11/5/13

Incremental Aggregation in Informatica

Home

Data Warehouse

Informatica

Informatica Scenarios

Oracle

Unix

Hadoop

Forum

Subscribe

SoftLayer Official Site

SoftLayer.com/Asia-Pacific More Custom Hosting From A Trusted Source. 24x7 Support. Chat Now.

INCREMENTAL AGGREGATION IN INFORMATICA

Incremental Aggregation is the process of capturing the changes in the source and calculating the aggregations in a session. This process makes the integration service to update the target incrementally and avoids the process of calculating the aggregations on the entire source. Consider the below sales table as an example and see how the incremental aggregation works.

Search...

Search

S o u r c e : Y E A RP R I C E 2 0 1 01 0 0 2 0 1 02 0 0 2 0 1 03 0 0 2 0 1 15 0 0 2 0 1 16 0 0 2 0 1 27 0 0

UNIX RECENT POSTS Know the User Shell - Unix / Linux View Phpinfo from Command Line

For simplicity, I have used only the year and price columns of sales table. We need to do aggregation and find the total price in each year.
www.folkstalk.com/2012/02/incremental-aggregation-in-informatica.html

Unix Sed Command to Delete Lines in File - 15 Examples

1/6

11/5/13

Incremental Aggregation in Informatica

How to recover lost root user password in Unix /linux

When you run the session for the first time using the incremental aggregation, then integration service process the entire source and stores the data in two file, index and data file. The integration service creates the files in the cache directory specified in the aggregator transformation properties. After the aggregation, the target table will have the below data.

Sort Files By Size In Unix / Linux ls Command More Unix Tutorials

T a r g e t : Y E A RP R I C E 2 0 1 06 0 0 2 0 1 11 1 0 0 2 0 1 27 0 0

Now assume that the next day few more rows are added into the source table.
INFORMATICA RECENT POSTS

S o u r c e : Y E A RP R I C E 2 0 1 01 0 0 2 0 1 02 0 0 2 0 1 03 0 0 2 0 1 15 0 0 2 0 1 16 0 0 2 0 1 27 0 0 2 0 1 04 0 0 2 0 1 11 0 0 2 0 1 22 0 0 2 0 1 38 0 0

Session Logs Information - Informatica Load all records except last N - Informatica Cumulative Sum Calculation in Informatica Informatica PowerCenter Repository - Global and Local Different Types of Tracing Levels In Informatica More Informatica Articles ORACLE RECENT POSTS Cursor For Loop Example in Oracle PlSql Max of Product N consecutive Digits - Oracle Sql Query SQL Query to Group / Aggregate N Consecutive Rows Oracle Trigger After Insert Example & Create Oracle Trigger Before Insert Create & Example

www.folkstalk.com/2012/02/incremental-aggregation-in-informatica.html

2/6

11/5/13

Incremental Aggregation in Informatica

POPULAR POSTS

Now for the second run, you have to pass only the new data changes to the incremental aggregation. So, the source will contain the last four records. The incremental aggregation uses the data stored in the cache and calculates the aggregation. Once the aggregation is done, the integration service writes the changes to the target and the cache. The target table will contains the below data.

Top Examples of Awk Command in Unix Sed Command in Unix and Linux Examples Cut Command in Unix ( Linux) Examples Informatica Scenario Based Interview Questions with Answers - Part 1

T a r g e t :
Informatica Scenario Based Questions - Part 2

Y E A RP R I C E 2 0 1 01 0 0 0 2 0 1 11 2 0 0 2 0 1 29 0 0 2 0 1 38 0 0

Download Informatica PowerCenter Version 9.1 Tutorials (PDF Documents) String Functions in Hive Date Functions in Hive Unconnected Stored Procedure Transformation Example - Informatica

Points to remember
SQL Transformation in Informatica with examples

1. When you use incremental aggregation, first time you have to run the session with complete source data and in the subsequent runs you have to pass only the changes in the source data. 2. Use incremental aggregation only if the target is not going to change significantly. If the incremental aggregation process changes more than hhalf of the data in target, then the session perfromance many not benfit. In this case go for normal aggregation.

Follow me on Google+

Note: The integration service creates a new aggregate cache when A new version of mapping is saved Configure the session to reinitialize the aggregate cache Moving or deleting the aggregate files Decreasing the number of partitions Configuring the mapping for incremental aggregation

www.folkstalk.com/2012/02/incremental-aggregation-in-informatica.html

3/6

11/5/13

Incremental Aggregation in Informatica

Before enabling the incremental aggregation option, make sure that you capture the changes in the source data. You can use lookup transformation or stored procedure transformation to remove the data which is already processed. You can also create a trigger on the source database and can read only the source changes in the mapping.

Labels: Informatica

7 comments:
Neel 08 February, 2012 23:01 Hi, Is incremental aggregation so simple? If we implement d idea of incremental load or CDC, and by default aggregator has caching property...why do i need to excercise incremental aggregation as separate option. What is the advantage of using this over normal map. (using cdc and not using incremental aggregation property). Please explain. Reply Replies Anonymous 10 February, 2012 07:24 Normal aggregator also caches the data. However, this cache will be cleared when the session run completes. In case of incremental aggregation the cache will not be cleared and it is reused in the next session run. If you want to use normal aggregation, every time you run the session you have to pass the complete source data to calculate the aggregation. In case of incremental aggregation, as the processed data is stored in the cache, you just need to pass only the changes in the source. This way the data in cache and the changes form the complete source. Reply

Anderson Schmitt 14 February, 2012 09:27

www.folkstalk.com/2012/02/incremental-aggregation-in-informatica.html 4/6

11/5/13

Incremental Aggregation in Informatica

Thanks, this solved me a big problem! Reply

Rahul Srivastava 22 April, 2013 22:34 how to enable incremental aggregation in mapping is incremental aggregation only work with aggregator transformation if yes then how to enable it? Reply Replies vijay bhaskar 22 April, 2013 22:38

You have to use aggregator transformation. You can find the incremental aggregation in the session properties tab under the performance section. Reply

kalpesh 03 July, 2013 01:49 It is said that , use incremental aggregation only if the target is not going to change significantly OR If the incremental aggregation process changes more than hhalf of the data in target, then the session perfromance many not benfit. But in the given example all the target records are changed hence the increamental aggregation may not benefit. Could anyone explain why increammental aggregation may not benefit if target records change significantly... Reply Replies vijay bhaskar 03 July, 2013 02:00

Updates are lot costlier when compared to inserts. This is why if the target records changes significantly, then truncate the target and do aggregation on the complete data and then load into target. Reply

www.folkstalk.com/2012/02/incremental-aggregation-in-informatica.html

5/6

11/5/13

Incremental Aggregation in Informatica

Enter your comment...

Comment as:

Google Account

Publish

Preview

Newer Post
Subscribe to: Post Comments (Atom) FOLLOW US ON FACEBOOK

Home

Older Post

privacy policy. Powered by Blogger.

www.folkstalk.com/2012/02/incremental-aggregation-in-informatica.html

6/6