Incremental Loading with Informatica Cloud

VERSION 1 Created on: Dec 16, 2010 10:35 AM by Katie Herstein - Last Modified: Dec 16, 2010 10:35 AM by Katie Herstein You can easily implement incremental processing using the variable $LastRunTime in a filter condition as shown in the screenshot below. Its very simple to add and Informatica Cloud will remember the last run time for you!

The $LastRunTime variable is based on the last time that the integration task ran. If you want to base this on your source system, use the following:

Note that the filter is based on the source data type. For databases, look at the particular database SQL. For SFDC to a file or database, use SOQL syntax.

One thing to note here is the use of timezones. All communication with SFDC is in GMT. The $LastRunTime from the Secure Agent on your computer is in GMT. Your database or file is probably in your local timezone (CST, PST, etc). If you are integrating from SFDC --> DB and using $LastRunTime, you don't need to worry about making time zone adjustments. If you are integrating from DB --> SFDC, you will need to adjust the time in your filter by that much (minus daylight savings).

For example, if you are synching data from a database running on Central Standard (CST) time to Salesforce.com, you will add 5 hours to the filter using the database terminology. For MSSQL, use in your Advanced Filter

DATEADD(hour,5 , myLastModifiedField) >= $LastRunTime

An easier way is to continually load a windows of data consisting of the last day of data. That is accomplished in Data Filters as well.

For example, over the last Day

where mylastdatechangedfield >= MySQL - DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) Microsoft SQL Server - DATEADD(day, -1, getdate()) Oracle - SYSDATE - 1 Salesforce.com - LAST_N_DAYS:1

---------------------------------------------------------------------------------------------------

Files are a little finickier - for example, to get 5 days before the last date that the task ran. (lastAct is the date changed) to get same date as last run, change the -5 to 0

(IIF(INSTR(lastAct, ' ' ) > 0,to_date(SUBSTR(lastAct,0,INSTR(lastAct, ' ' )),'MM/DD/YYYY'),to_date(lastAct,'MM/DD/YYYY')))>=(add_to_date(to_date( substr($lastrundate,6,2) || '/' || substr($lastrundate,9,2) || '/' || '20' || substr($lastrundate,3,2),'MM/DD/YYYY'),'DD',-5)) Can I do incremental loading how to do it? You can easily implementing incremental processing using $LastRunTime in a filter condition as shown in the screenshot below. Its very simple to add and Informatica On Demand will remember the last run time for you!

The $LastRunTime variable is based on the last time that the integration task ran. If you want to base this on your source system, use the following:

Note that the filter is based on the source data type. For database, look at the particular database SQL. For SFDC to a file or Database, use SOQL syntax

One thing to note here is the use of Timezones. All Communication with SFDC is in GMT. The $LastRunTime from the Secure Agent on your computer is in GMT. Your Database or file is probably in your timezone (ie CST, PST, etc)

So if you are integrating from SFDC --> DB and using $LastRunTime you are good. If you are integrating from DB --> SFDC you will need to adjust the time in your filter by that much (minus daylight savings . . )

So from a Central Standard (CST) database time to Salesforce.com, you will add 5 hours to the Filter using the database terminology For MSSQL, use in your Advanced Filter

DATEADD(hour,5 , myLastModifiedField) >= $LastRunTime

An easier way is to continually load a windows of data consisting of the last day of data. That is accomplished in Data Filters as well.

For example, over the last Day

where mylastdatechangedfield >= MySQL - DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) Microsoft SQL Server - DATEADD(day, -1, getdate()) Oracle - SYSDATE - 1 Salesforce.com - LAST_N_DAYS:1

---------------------------------------------------------------------------------------------------

Files are a little finickier - for example, to get 5 days before the last date that the task ran. (lastAct is the date changed) to get same date as last run, change the -5 to 0

(IIF(INSTR(lastAct, ' ' ) > 0,to_date(SUBSTR(lastAct,0,INSTR(lastAct, ' ' )),'MM/DD/YYYY'),to_date(lastAct,'MM/DD/YYYY')))>=(add_to_date(to_date( substr($lastrundate,6,2) || '/' || substr($lastrundate,9,2) || '/' || '20' || substr($lastrundate,3,2),'MM/DD/YYYY'),'DD',-5))

This document was generated from the following discussion: The specified discussion was not found.