Introduction in IICS
The Filter Transformation in Informatica IICS is used to filter records based on a specified condition.
Only records that satisfy the condition are passed to the next transformation or target.
How Filter Works
Source
↓
Filter
↓
Target
For example, if we want to process only customers from India:
Country = ‘India’
Only records satisfying this condition will continue through the pipeline.
Example
Source Data
| Customer_ID | Name | Country |
|---|---|---|
| 101 | Rahul | India |
| 102 | John | USA |
| 103 | Priya | India |
Country = ‘India’
Filter Condition
Country = ‘India’
Output
| Customer_ID | Name | Country |
|---|---|---|
| 101 | Rahul | India |
| 103 | Priya | India |
The USA record is filtered out.
Using Multiple Conditions
Multiple conditions can be combined using operators such as:
- AND
- OR
- =
- <
- =
- <=
Example:
Country = ‘India’ AND Sales > 5000
This passes only Indian customers whose sales are greater than 5000.
Real-World Use Case
A typical IICS pipeline can filter active customers before loading them into Snowflake:
Customer Data
↓
Filter
↓
Active Customers
↓
Snowflake
Filter condition:
Status = ‘Active’
This prevents unnecessary or unwanted records from being loaded into the target.
Best Practices
- Apply filters as early as possible when appropriate.
- Use clear and simple filter conditions.
- Avoid unnecessary complex expressions.
- Validate the number of records before and after filtering.
- Make sure business rules are correctly represented in the filter condition.
Conclusion
The Filter Transformation is a simple but important IICS transformation used to control which records continue through an ETL pipeline. It improves data quality and can reduce unnecessary processing by allowing only required records to reach downstream transformations and targets.