Solution Details

The rationale behind this approach was to divide the high volume data into smaller chunks (partitions) so that the select query would fetch data only from a single partition rather than scanning the full table. This section provides a complete example of using Informatica partitioning and EIM table cloning which can be implemented on any of the Siebel entity. The example demonstrates how one would load the Siebel entity Activity from Siebel 6.0 version to Siebel 8.0, as there is huge data volume required for migrating Activity related data. For this option indexes were created on S_EVT_ACT table and 8 Informatica partition points were used to load data into Clone EIM tables. Following are the steps to improve the performance of high volume data load. Step 1 Create Index on S_EVT_ACT On the basis of the column which can be used for Informatica partitioning we would require to create index on that specific column. As the source in the example is Siebel, the index would be created on S_EVT_ACT table and partitioning would be done on the basis of CREATED column. Using the below script, an index was created on the S_EVT_ACT table. By using Parallel and Nologging option, the index creation time was reduced drastically Database: Source CREATE INDEX TEMP_S_EVT_ACT ON S_EVT_ACT (CREATED) NOLOGGING TABLESPACE INDEX_LRG1 PCTFREE 10 INITRANS 2 MAXTRANS 255 STORAGE ( INITIAL 100M NEXT 100M MINEXTENTS 1 MAXEXTENTS 505 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT ) PARALLEL (DEGREE 6 INSTANCES 1);

Execution Time: 30-45 mins.

Step 2 Analyze Index Once the index is created Analyze script is executed as given below to build stats on the index. Building statshelped with the optimization to use the most efficient path Database Source ANALYZE INDEX TEMP_S_EVT_ACT ESTIMATE STATISTICS SAMPLE 5000 ROWS Execution Time: 5-10 mins Step 3 Clone EIM tables The next step in this process is to Clone the EIM tables. This creates multiple copies of the same EIM table with different names. The cloning of the EIM tables is required as each workflow will load data into a separate set of Clone tables. The script for creating the Clone of EIM tables on the Target database is attached below.

CLONE_EIM_TBL_SC RIPT.SQL

Step 4 Get S_EVT_ACT counts The group by SQL statement given below was executed to get the volume for each month. This information is to limit the records for each workflow and partition points will use. Database Source SELECT TO_CHAR (CREATED, 'YYYY/MM'), COUNT (8) FROM SIEBEL.S_EVT_ACT WHERE created > TO_DATE ('31-DEC-2007', 'DD-MON-YYYY') GROUP BY TO_CHAR (CREATED, 'YYYY/MM') Execution Time: 10-15 mins Step 5 Import Target definitions The Cloned EIM tables created in the above steps were imported to create the Informatica mappings using the Import Target definitions option in Informatica Step 6 Create Informatica Mapping Mapping 1 m_ACTIVITY_EIM_1was created using the below target tables. EIM_FN_ACT EIM_FN_ACT1 EIM_ACT_DTL

Figure 1 Workflow wf_INT_ACTIVITY_EIM_1 was created for the above mapping.

Figure 2

8 Partition Points were created as shown in the figure below.

Figure 3 The data set was limited in each Partition Point by using created column as shown below. The Workflow can be referred for more details. WHEREA.row_id=b.par_row_id(+) ANDA.created>TO_DATE('30-Jun-2011','DD-MON-YYYY') Step 7 Additional Mappings for Parallel execution The above approach was replicated for all the below mappings and workflows. Mappings m_ACTIVITY_EIM_CLONE_2 m_ACTIVITY_EIM_CLONE_3 m_ACTIVITY_EIM_CLONE_4 m_ACTIVITY_EIM_CLONE_5 m_ACTIVITY_EIM_CLONE_6 Workflows wf_INT_ACTIVITY_EIM_CLONE_2 wf_INT_ACTIVITY_EIM_CLONE_3 wf_INT_ACTIVITY_EIM_CLONE_4 wf_INT_ACTIVITY_EIM_CLONE_5

wf_INT_ACTIVITY_EIM_CLONE_6 Step 8 Drop EIM table indexes on the Target database The EIM table indexes were dropped on the target database. The Scripts are attached for reference.

DROP IDX_EI _FN_ ACT1.txt

DROP_IDX_EI _FN_ ACT.txt

Step 9 Execution of workflows

The workflows mentioned below were executed to run in parallel. wf_INT_ACTIVITY_EIM_1 wf_INT_ACTIVITY_EIM_CLONE_2 wf_INT_ACTIVITY_EIM_CLONE_3 wf_INT_ACTIVITY_EIM_CLONE_4 Execution Time: It took 4 hrs. to load 50 million records in each of the EIM tables

Step 10 Execution of workflows

Upon successful completion of the above workflows, the below scripts were executed in Parallel to load all the data in the 1st set of EIM tables. INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_FN_ACTSELECT * FROM SIEBEL.EIM_FN_ACT_C2 INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_FN_ACT1 SELECT * FROM SIEBEL.EIM_FN_ACT1_C2 INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_ACT_DTLSELECT * FROM SIEBEL.EIM_ACT_DTL_C2 INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_FN_ACTSELECT * FROM SIEBEL.EIM_FN_ACT_C3 INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_FN_ACT1 SELECT * FROM SIEBEL.EIM_FN_ACT1_C3 INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_ACT_DTLSELECT * FROM SIEBEL.EIM_ACT_DTL_C3 INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_FN_ACTSELECT * FROM SIEBEL.EIM_FN_ACT_C4

INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_FN_ACT1 SELECT * FROM SIEBEL.EIM_FN_ACT1_C4 INSERT/*+ APPEND NOLOGGING */INTO SIEBEL.EIM_ACT_DTLSELECT * FROM SIEBEL.EIM_ACT_DTL_C4 Please Note: This process can be done one by one too by moving data from first clone to vanilla EIM table and once EIM job is executed then moving the data from EIM table back to the clone table.

Step 11 Re-Create EIM table indexes

The indexes were recreated on EIM_FN_ACT using the attached scripts provided blow.The scripts were executed in parallel with multiple threads using Toads QSR

IDX_EIM_FN_AC

Similarly, the attached script provided below was used to create the indexes on EIM_FN_ACT1. . The script was executed in parallel with multiple threads using Toads QSR

IDX_EIM_FN_AC

Step 12 Build EIM table stats

The stats were run on the EIM table and their indexes using the scripts provided below.

ANALYZ IM BL & ID .t t