Académique Documents
Professionnel Documents
Culture Documents
What is a Transformation?...........................................................................................................................4
Types of Transformations............................................................................................................................4
Active Transformation.............................................................................................................................4
Passive Transformation............................................................................................................................4
What are the different ways to use a Transformation?.................................................................................5
Connected Transformation.......................................................................................................................5
Unconnected Transformation...................................................................................................................5
Commonly used Transformations................................................................................................................6
Source Qualifier..........................................................................................................................................7
Interview Questions related to Source Qualifier....................................................................................15
Filter..........................................................................................................................................................17
Very Important Interview Question........................................................................................................17
Steps to use Filter Transformation.........................................................................................................18
SORTER TRANSFORMATION...............................................................................................................33
TYPES OF LOADS (INTERVIEW QUESTION).....................................................................................47
Full Load...............................................................................................................................................47
Incremental Load...................................................................................................................................47
Refresh Load.........................................................................................................................................47
RANK TRANSFORMATION...................................................................................................................48
How to Define the Target Table?...........................................................................................................49
Interview Question................................................................................................................................58
Important Scenario................................................................................................................................60
EXPRESSION TRANSFORMATION......................................................................................................62
Interview Question................................................................................................................................62
Using Character Functions in an Expression.........................................................................................70
AGGREGATOR TRANSFORMATION...................................................................................................73
ROUTER TRANSFORMATION..............................................................................................................84
Interview Question for ROUTER..........................................................................................................96
UNION TRANSFORMATION.................................................................................................................97
JOINER TRANSFORMATION..............................................................................................................113
Join Types............................................................................................................................................113
Equi-Join..........................................................................................................................................113
MASTER OUTER...........................................................................................................................115
Detail Outer.....................................................................................................................................115
Full Outer.........................................................................................................................................115
Interview Question for JOINER..........................................................................................................115
SEQUENCE GENERATOR....................................................................................................................135
DEBUGGING.........................................................................................................................................138
CONNECTED LOOK-UP TRANSFORMATION..................................................................................148
TYPES OF CACHE............................................................................................................................150
Interview Questions.............................................................................................................................162
Question: What is the difference between Static and Dynamic cache? (V.V.Imp)............................162
Question: Which is the default cache of lookup?.............................................................................162
Question: How many output ports are allowed by Connected Lookup?..........................................162
Question: What are the different types of CACHE?........................................................................162
Question: Can you look up in a flat file?.........................................................................................163
Question: What types of Joins are allowed in Lookup Transformation?..........................................163
Question: Can a lookup allow Outer Joins?.....................................................................................163
UNCONNECTED LOOKUP TRANSFORMATION..............................................................................164
Advantages of Unconnected Lookup Transformation..........................................................................171
Differences Between Connected and Unconnected Lookups...............................................................172
UPDATE STRATEGY............................................................................................................................173
DYNAMIC LOOKUP WITH UPDATE STRATEGY.........................................................................175
STORED PROCEDURE.........................................................................................................................195
Operations accomplished through Stored Procedures..........................................................................195
Types of Stored Procedure Transformations........................................................................................195
Normal.............................................................................................................................................196
Source Pre-Load or Pre-Source Load..............................................................................................196
Source Post Load or Post Source Load............................................................................................196
Pre-target Load or Target Pre-Load..................................................................................................196
Post-Target Load or Target Post-Load.............................................................................................196
UNCONNECTED STORED PROCEDURE...........................................................................................208
Source Pre-load and Source Post-Load....................................................................................................215
Transaction Control Transformation........................................................................................................222
XML SOURCE QUALIFIER..................................................................................................................234
XML Source Qualifier.........................................................................................................................236
How to Create an XML file from a relational file (table).....................................................................238
NORMALIZER TRANSFORMATION..................................................................................................251
What is a Transformation?
Types of Transformations
Transformations are basically of two types:
Active
Passive
Active Transformation
A transformation is called as an active transformation if it allows change in the record count.
e.g. If there are 20 records coming from the source but only 5 are sent to the target after transformation.
Source
(20)
Transformati
on
Target (5)
Passive Transformation
A transformation is called as a passive transformation if it does not allow any change in the record count.
e.g. If there are 20 records coming from the source and all 20 are sent to the target after transformation.
Source
(20)
Transformati
on
Target (20)
I/p
O/p
->
->
->
->
->
->
->
->
Connected Transformation
In a pipeline .The transformation used in a mapping are said to be in a pipeline.
Unconnected Transformation
Outside the pipeline.
Transformation Logic
Source Qualifier
Filter
Sorter
Expression
Aggregator
RANK
ROUTER
Sequence Generator
Joiner
Union
Look-up(connected/unconnected)
Stored Procedure (connected/unconnected)
Update Strategy
Transaction Control
XML Source Qualifier
Normalizer
Application Source Qualifier (SAP Application Sources)
Application Multigroup Source Qualifier (SAP Application Sources)
MQ-Series (Message Queuing System)
Midstream XML Generator (TIBCO)
Midstream XML Parser (TIBCO)
Type
Active
Active
Active
Passive
Active
Active
Active
Passive
Active
Active
Passive
Passive
Active
Active
Active
Active
Active
Active
Active
Active
Active
Source Qualifier
It is an active transformation which acts like a source for Informatica.
Purpose:
Note: The Input and Output ports for the Source Qualifier are disabled (this means the data coming to the
input port and going from the output port cant be modified). Data will just pass through without any
transformation.
Go to the properties tab of the Edit Transformations Window and click on the drop-down button.
SQL Editor window opens up. Give ODBC data source, username and password and click the Generate
SQL button.
Other Questions:
Filter
Active Transformation
Used to filter the data or reduce the amount of data.
Single condition can be applied at a time.
Filter transformation accepts only two types of data - Boolean (True or False)
Filter accepts condition satisfied data.
The false data is rejected or dropped to the bad files. (Drawback)
Source
Qualifier
Filter1 (sal
> 2000)
Target1
Filter2
(deptno =
10)
Target2
Source
Qualifie
r
Target
Give a name to the transformation (T_FIL) and click on Create and then Done.
Select the required ports from the Source Qualifier (by using Ctrl key).Drag and drop on the filter
transformation.
Right click the Transformation object and click on edit or select Edit option from the Transformations
menu.
Click on the Properties tab and click the Filter condition down arrow button.
Click on Validate.
Project the required ports from the filter to the target.
Repository -> Save.
Select the dimension table from the Targets folder and then in the Properties window check the Truncate
Target Table option. Apply and then OK.
Now go to Informatica Designer. Right click on the target table DIM_EMP and select the option Preview
Data.
Give the database credentials and click on Connect, the result dataset is shown up.
SORTER TRANSFORMATION
Properties:
STEPS:
Define Source (EMP)
Drag the Source (EMP) and Target to the Mapping Designer Workspace.
From the Transformation Menu select Create Sorter
Edit the Sorter and in the Ports tab do the following settings:
Check the Sal and Deptno as Key and choose the order (direction asc or desc).
Our objective is to display the department wise salaries in descending order.So change the order of the
columns (make dept appear first then salary)
Suppose we just want to know the salaries of deptno 10 then we need to filter out the records.
Note: To enhance the performance, first apply sorter and then filter transformation .
Output:
Incremental Load
Next time onwards incremental load is done. The data is appended to the historical data.
We generally do this type of load.
Refresh Load
Suppose the data loaded is full of errors, entire data is reloaded. In this case only we need the Truncate
Target Table Option. Rarely used.
RANK TRANSFORMATION
Active Transformation.
Used to seek the position (at which position the data is).
We can see top or bottom positions.
By default, rank index port name is created along with the transformation object (To carry the
rankings rank index is prepared).
Ports:
Input
Output
Expression
Variable
Rank
Group-By
Steps:
1. Define the Source Definition (EMP).
2. Define the Target Definition.
Now go to the Columns tab and add a new column RANKINDEX and click Apply, then OK.
Till now the new table Dim_RANK is logically present in the WareHouse.
SQL> select * from tab;
TNAME
TABTYPE CLUSTERID
------------------------------ ------- ---------DIM_EMP
TABLE
DIM_EMP1
TABLE
To make it available physically go to the Targets Menu and click on Generate/Execute SQL.
Click the radio button Generate from Selected Tables , check the box Create Table.
Click Generate SQL file and the Close.
Step 3. Define the mapping by dragging the source and target. Create a RANK Transformation
T_RANK.
Project the required ports from the Source Qualifier to the transformation object.
Edit Rank Transformation.
Go to the Ports Tab and apply RANK on Salary Check the R box. Apply Group By on the
department no.
Now we have selected department wise top two salaries, but our objective is to fetch the second
highest salary so we will apply a fliter with the condition RANKINDEX = 2.
Create a Filter Transformation. Project the required ports from RANK to FILTER.
Edit Filter In the Properties Tab apply the condition RANKINDEX = 2.
Apply -> OK
Project the required ports from the Filter to the Target.
SQ -> T_RANK -> T_FIL -> DIM_RANK.
(To increase the performance, use SORTER Transformation before T_RANK)
Create a Session
Create a Workflow
Repository _> Save
Start Workflow
We encounter an error.
Interview Question
Note:
If there is an index on the target table (in this example DIM_RANK is indexed on EMPNO)
(Index or Primary Key), BULK LOAD cant happen.
Solution:
Option 1: Make it a NORMAL LOAD (By default the Target load type is BULK).
Option2: Data Warehouses require bulk load, drop the index, project the data and recreate the
index.
Generally done by the production support people.
Important Scenario
Theres a problem here instead of showing 3 records (one for each dept 10,20,20), it is showing
just two records.
Reason:
1 select empno,deptno,sal from emp
2* order by deptno,sal desc
L> /
EMPNO DEPTNO
-------- ---------- ---------7839
10
5000
7782
10
2450
7934
10
1300
7788
20
3000
7902
20
3000
7566
20
2975
7876
20
1100
SAL
7369
7698
7499
7844
20
30
30
30
800
2850
1600
1500
EMPNO DEPTNO
-------- ---------- ---------7521
30
1250
7654
30
1250
7900
30
950
SAL
The result is not showing the second highest salary for the dept 20 as it assigns rank 1 to the
records with empno 7788 and 7902. When it comes to empno 7566 it assigns rank as 3. So the
filter condition rank = 2 is not satisfied here.
Solution (later ?)
EXPRESSION TRANSFORMATION
Passive Transformation
It is used to build row-by-row calculations.
We can derive new information (new data) in expressions.
Ports:
Input
Output
Variable
Expression
(All the ports are enabled except expression)
Interview Question
When the expression port will be enabled?
When the input port is unchecked
Or
The variable port is checked
Note: Expression has no properties.
TNAME
TABTYPE CLUSTERID
TABLE
DIM_EMP1
TABLE
DIM_RANK
TABLE
DIM_EXP
TABLE
These ports now appear under the Expression list (highlighted). Click on Apply and then OK.
AGGREGATOR TRANSFORMATION
Aggregator is an active transformation used to calculate group values.
Ports:
Input
Output
Expression
Variable
Group By
Properties:
Data Cache
Index cache
Sorted Input
Steps:
1. Define Source
2. Define Target
Write the Target requirements and Generate/Execute SQL.
AGG_FACT
DEPTNO
TOT_SAL
MAX_SAL
MIN_SAL
AVG_SAL
COUNT_EMP
5. Go to the Transformation Menu and click on Create. Select the Transformation type as
AGGREGATOR T_AGG.
In the Functions tab expand Aggregate folder. Select the function. From the Ports tab select the port.
Validate -> OK.
Apply the functions to respective ports.
Check the Group By check box for the department.
In the Properties tab check the box Sorted Input if the data is already sorted (suppose sorter tranfn is
applied before the aggregator, otherwise the agg will sort the input)
ROUTER TRANSFORMATION
Active Transformation.
It is used to filter the data like the filter transformation.
Router allows to filter with multiple conditions and project data to multiple targets.
Unlike the Filter, Router allows rejected data also. It captures the rejected data in a default
group.
Router allows one input and sends it to many outputs.
Steps:
Define Source
Define Targets (multiple targets).
Create the mapping .Drag Source (EMP) and Targets (Dim_EMP,DIM_EXP,DIM_RANK for practice
only) to the Mapping Designer.
In the Transformation Menu select ROUTER and click on Create. Give the name as T_ROUT.
Click on Create and then Done.
Project the required ports from the Source Qualifier to Router. An INPUT GROUP is prepared.
Note: When we add a new group for the first time, by default it prepares a default group. Default group is
created only once.
Give Group names and then specify the group filter condition.
Apply and then OK.
All the Groups (INPUT, DEPTNO10, SAL_GT_2000, DEFAULT) can now be seen in Router.
EMPNAME
X
Y
Z
A
SAL
2000
3000
4000
3000
DEPTNO
10
10
20
10
Condition1: Deptno = 10 -> 3 records satisfy the 4th record goes to the Default Group.
Condition2: Sal > 2000 the record in the Default Group satisfies this condition and so it is now removed.
Rejected record is now 1 X 2000 10
Note: For a record in the default Group it should satisfy neither condition.
Now Project the required ports from ROUTER OUTPUT to Multiple TARGETS.
Note: In Router Transformation we may get more no. of records in the Target than the source.
Example: Source Rows 14, Target Rows 16
(See the screen shot)
UNION TRANSFORMATION
Active Transformation
Merges similar data coming from heterogeneous (Oracle,SQl Server) or homogenous sources.
These data can be maintained in the form of flat files.
Union uses UNION ALL function. It does not remove duplicates.
Multiple input groups but only one output group (opposite to Router).
Input and Output groups must have matching ports.
It does not generate transactions (just combining).
Steps:
1. Define Source Definition EMP(Oracle) and EMP_F (Flat file e.g. .txt file)
2. Create a .txt file EMP.txt
Go to Source Analyzer workspace, select Import from File option from the Sources menu.
Define Target
Define Mapping
Drag Sources,Target and create a Transfn. T_UNION
By default, NEWGROUP (Blue many) and OUTPUT (grey only one) are created. There are no ports
inside this blank object.
Create two Input Groups and give the INPUT group name as EMP_O and EMP_F.
In the mapping designer workspace we can now see the newly created groups.
Drag the inputs from SQ to UNION EMP_O. Likewise, drag the inputs from SQ to UNION EMP_F.
Define Session,Workflow.
Edit the Session and in the Mappings tab mention the Source File Directory C:\ and Source Filename
EMP.txt.
Output (Target table is having primary key here so the 3 duplicate records have been rejected)
If the Target Table has no Primary Key (All duplicate records will also get populated)
JOINER TRANSFORMATION
Active Transformation
Joiner relates two or more tables.
It allows homogeneous or heterogeneous sources.
Join Types
NORMAL (Equijoin default)
OUTER
Master Outer
Detail Outer
Full Outer
Does not allow non-equi joins.
Join Types
Equi-Join
Example
EMP (DETAIL )
Empno ename sal deptno
1 x 2000 10
2 y 3000 20
3 z 1000 10
4 A 3000
DEPT (MASTER)
DEPTNO DEPTNAME
10
HR
20
FI
30
SALES
Equi-Join (3 Records)
1 x 2000 10 HR
2 y 3000 20 FI
3 z 1000 10 HR
MASTER OUTER
Equi-Join + Remaining Records of Detail (4 records)
1 x 2000 10 HR
2 y 3000 20 FI
3 z 1000 10 HR
4 A 3000
Detail Outer
Equi-Join + Remaining Records of MASTER (4 records)
1 x 2000 10 HR
2 y 3000 20 FI
3 z 1000 10 HR
30 SALES
Full Outer
Equi-Join + Remaining Records of MASTER + Remaining Records of Detail (5 records)
1 x 2000 10 HR
2 y 3000 20 FI
3 z 1000 10 HR
30 SALES
4 A 3000
EMPNO ENAME
JOB
MGR HIREDATE
SAL
COMM
DEPTNO
SALESMAN
7698 20-FEB-81
1600
300
30
7521 WARD
SALESMAN
7698 22-FEB-81
1250
500
30
7566 JONES
MANAGER
7839 02-APR-81
2975
7654 MARTIN
SALESMAN
7698 BLAKE
20
7698 28-SEP-81
1250
MANAGER
7839 01-MAY-81
2850
30
7782 CLARK
MANAGER
7839 09-JUN-81
2450
10
7788 SCOTT
ANALYST
7566 19-APR-87
PRESIDENT
17-NOV-81
7839 KING
7844 TURNER
SALESMAN
7876 ADAMS
CLERK
7900 JAMES
CLERK
EMPNO ENAME
JOB
3000
7698 03-DEC-81
MGR HIREDATE
30
20
5000
7698 08-SEP-81
7788 23-MAY-87
1400
10
1500
1100
950
30
20
30
SAL
COMM
ANALYST
7566 03-DEC-81
7934 MILLER
CLERK
7782 23-JAN-82
9999 MONA
ANALYST
14 rows selected.
7566 06-AUG-09
3000
1300
2000
20
10
DEPTNO
DEPTNO DNAME
LOC
NEW YORK
DALLAS
CHICAGO
40 OPERATIONS
50 SUPPORT
BOSTON
BANGALORE
60 DISTRIBUTION DELHI
70 EDUCATION
80 XYZ
8 rows selected.
ABC
CHENNAI
STEPS:
1. Define Sources (EMP,DEPT)
2. Define Target (DIM_JOIN)
3. Define Mapping (M_JOIN) Drag two sources and one target in Mapping Designer
Workspace.
4. From the Transformation Menu select JOINER and create. T_JOIN
Drag the required ports from two source qualifiers to the JOINER.
Note: The first input to JOINER becomes Master and next Detail. (??? I got the detail as detail and master
as master only )
Go to JOINER,edit go to PORTS tab and select master port for deptno,dname and loc.
Ports: Input,Output,MAster
Properties:
Data Cache
Index Cache is going to use JOIN condition.
Sorted Input
Join Type
Join Condition
Cache Directory
Case sensitive string comparison
Join Type : NORMAL
NULL ordering in MASTER/DETAIL (By default it is high NULL values will come
at the bottom)
NOTE: For JOINER and some other transformations, we can edit the properties at the Session
level also.
Edit the Session and go to Mappings tab.
Select Transformations.
Master outer
(Equijoin + Remaining records of Detail Table)
Select the Join Type as Master Outer.
Output (13 + 1)
Output:
SEQUENCE GENERATOR
Passive Transformation
It is used to generate sequence numbers (to create Primary Key values)
By default, it creates NEXTVAL and CURRVAL port names which are in disabled state.
We cannot add new ports.
Sequence generator does not allow any inputs, allows output only.
STEPS:
Define Source
Define Target
Define Mapping Drag the source (EMP), target (should have an extra field for sequence number).
Rep -> save
Ports:
Properties:
Start Value: Start value is 0 by default. Used when the Cycle option is checked. It is used for starting the
next cycle.
Increment By
End Value
Cycle
Current value: The beginning value of the sequence. By default it is 1.
DEBUGGING
Mappings Menu -> Debugger -> Start Debugging or Press F9.
Click on NEXT.
Output Window:
Server initialized
Server Ready
Server initialization completed.
DSQ Execution Order:
1: SQ_EMP
Server is Ready.
No breakpoints to send to the server.
Target Instance
Instance
Mapping Menu -> Debugger -> Next Instance or Press F10
Passive Transformation
Used to search
To get the related values
To compare the values
Used in slowly changing dimensions (SCDs)
Source Connection
SOURCE
DATABA
SE
SR
C
Target Connection
SQ
DEPTN
ONO
LOOKUP
Transformati
on
Input Port =
deptno
TGT
DNAM
E
TARGET
DATABASE
Lookup connection
LOOKUP
DATABASE
(Source/Target/Oth
ers)
The source EMP does not have deptname. Target DB required deptname.
Lookup DB in this case is our Source DB which has DEPT table (SCOTT schema)
Note: To increase the performance all other ports are projected directly from SQ to the TGT.
Example:
Source
Target
1 X 3000 10
1 X 2000 10
TYPES OF CACHE
1.
Data Cache
2.
Index Cache
3.
Static cache
4.
Dynamic cache
5.
Persistent Cache
6.
7.
Shared Cache
Steps:
Define Source (EMP).
Define Target (DIM_JOIN).
Define Mapping M_LKUP_C_S (Lookup-connected-static)
Drag Source,Target to the Mapping Designer workspace.Create a Lookup Transformation T_LKP_C_S.
Click on Create.
Lookup table is DEPT which in this case in the Source Database itself. Select the Source tab and select
DEPT and then click on OK.
Note: From version 7.0 onwards, Lookup flat file concept has been introduced.
Edit the lookup transformation and in the ports tab see that the input port for deptno1 is checked.
The deptno and deptno1 ports should have same data type and size.
Select the condition tab.
Add a condition.
Lookup table column deptno = transformation port deptno1.
Click on Apply and then OK>
Properties Tab
Tracing level.
Lookup Cache Directory name
Re- cache from lookup source.
Flat file Properties:
Thousand Separator.
Decimal Separator.
Project the required ports from Lookup to the target and SQ to TGT.
Repository-> Save
Create a Session.
At the session level specify three connections STC,TGT and LKP.
TARGET CONNECTION
Interview Questions
Question: What is the difference between Static and Dynamic cache? (V.V.Imp)
STATIC CACHE
DYNAMIC CACHE
Data Cache
2.
Index Cache
3.
Static cache
4.
Dynamic cache
5.
Persistent Cache
6.
11 August, 2009
SOURCE
DATABA
SE
SR
C
TGT
DEPTNO (actual
parameter)
DNAM
E
DNAME
:LKP.U_LKP(DEPTNO)
EXPRESSION
Transformation
Unconnected
LOOKUP
Transformati
on
Input Port =
deptno
LOOKUP
DATABASE
(Source/Target/Oth
ers)
TARGET
DATABASE
Project the required ports from SQ to T_EXP and add a new port DNAME in T_EXP. Uncheck the Input
port.
Edit the Lookup Transformation and add a new port IN_DEPTNO (Formal parameter). Set the
datatype,size and precision same as that of DEPTNO.
Check the input port for IN_DEPTNO and return port for DNAME.
Go to Functions Tab, expand Lookup Folder. Double Click on Lookup Function :LKP.T_LKP_UNC().
Connected Lookup
Unconnected Lookup
UPDATE STRATEGY
It is an active transformation.
It is used to decide what type of data has to be stored in the target whether it is Insert, Update or
Delete.
Within a session. When we configure a session, we can instruct the PowerCenter Server to
either treat all rows in the same way (for example, treat all rows as inserts), or use
instructions coded into the session mapping to flag rows for different database operations.
Within a mapping. Within a mapping, we use the Update Strategy transformation to flag
rows for insert, delete, update, or reject.
Till now we have been unknowingly using the update strategy at the Session level.
Go to the Properties tab of the Session.
The Property Treat Source rows as is INSERT by default. Other options are Delete,Update,Data Driven.
Within a Mapping, we can set flag rows for INSERT, DELETE, UPDATE or REJECT.
DD_INSERT
DD_UPDATE 1
DD_DELETE 2
DD_REJECT 3
Note: Source ports (EMPNO1,ENAME1, etc. ) are created in the Lookup transformation. Edit the Lookup
transformation, Go to Properties Tab.
Check the property Dynamic Lookup Cache
Note that after this property is checked following options get highlighted:
An automatic projection happens from lookup transformation to the update strategy port
NewLookupRow.
At runtime it decides whether to INSERT or UPDATE.
By default, Session Property is INSERT.
Assign the connection for the Lookup Connection (MEEN_TGT because lookup table is DIM_JOIN
which is in the target database).
Select the Dynamic Lookup Cache option and Insert Else Update option.
In the Session properties set the attribute Treat source rows as to Data Driven.
Save the repository. Validate and start the workflow .
Normal output
Now let us observe the newlookuprow attribute.
INSERT a new record in the Source table EMP.
1 row created.
SQL> COMMIT;
Preview the output again
STORED PROCEDURE
It is a Passive Transformation
It is used to manage (USED BY DBA) and populate (USED BY DEVELOPER) the target
database.
Normal
By default, SP type is normal. It can be used for calculations. It can be used in two different types
of transformations connected SP or unconnected SP.
Source Pre-Load or Pre-Source Load
By using Source Pre-load, we can execute a procedure against Source Database before Source
(SQ) executes.
Source Post Load or Post Source Load
By using Source Post load, we can execute a procedure against Source database after Source
Query (SQ) executes. Example Indexes
Pre-target Load or Target Pre-Load
Before executing the target process or query on the target, we can use SP executing against the
target database.
Post-Target Load or Target Post-Load
After executing the target process or query on the target, we can use SP executing against the
target database. Example, Recreating the Index.
Ports:
Input,Output,Return
Define a Mapping M_SP. Drag Source (EMP),Target (EMP_SP) to the mapping designer.
Create a Stored Procedure transformation T_SP.
Connect to the database which has the required SP. In our case we have the SP to calculate annual
salary on the Source Database (SCOTT).
Form the Procedures tab select calc_an_sal.
In the Transformation T_SP we have P_SAL (input parameter) and P_AN_SAL (output
parameter).
Connect Source port Sal to Transformation port P_SAL, transformation port P_AN_SAL to the
target port AN_SAL. Project the remaining ports directly from source to the target.
Define a workflow.
Assign the source,target and Transfromation (SP) connections.
Start the workflow and preview the output.
DEFINE A MAPPING M_SP_UNC. Drag Source,Target EMP_SP_UNC and create two transformations:
Expression T_EXP_SP_UNC and Stored procedure T_SP_UNC.
Project the required ports from Source Qualifier to the Expression Transformation.
Edit the Expression Transformation and add two new port AN_SAL (change datatype, precision and
scale), HLF_YR_SAL .
Also add one port hlf_sal which is a variable port.
Go to Function tab of the Expression Editor, go to Stored procedures folder and Dbl click on T_SP_UNC
Assign the parameters SAL (IN), PROC_RESULT (Informatica Global Variable) and hlf_sal
(local variable).
:SP.SP_UNC(SAL,PROC_RESULT,hlf_sal)
Connections:
Source
Target
Stored Procedure
Define the workflow.
Assign source,target and transformation (sp) connections.
Start the workflow.
Preview
Steps:
Define a mapping M_SRC_PRE_POST_LOAD.
Drag Source (EMP),Target DIM_EMP to the mapping designer.
Create a SP Transformation T_SP_PRESOURCE_LOAD.
Import the stored procedure P_IN_EMP_NAME
Note: Give the execution order if there are more than one SPs in the Pre Source load.
MAPPING LEVEL
Transaction Control Transformation is used in a mapping to control transactions according to the userdefined conditions.
Use expression transformation to form the condition.
Built-in variables in the Expression Editor when we create a transaction control expression:
TC_COMMIT_AFTER. The PowerCenter Server writes the current row to the target,
commits the transaction, and begins a new transaction. The current row is in the
committed transaction.
If the transaction control expression evaluates to a value other than commit, rollback, or
continue, the PowerCenter Server fails the session.
Steps:
Define a mapping M_TC.
Drag Source EMP,Target DIM_EMP to the mapping designer.
Create a Transaction Control Transformation T_TC.
Project EMPNO,ENAME,JOB,SAL,DEPTNO from SQ to T_TC.
Project required ports from T_TC to the TGT.
Edit the Transformation T_TC and go to Properties Tab.
Go to Properties Tab and select the Variables tab. Select the built-in folder
Rejected: 0
Affected: 6
WRITER_1_*_1> WRT_8317
USER-DEFINED COMMIT POINT Fri Aug 14 14:01:12 2009
This means the records are inserted into a buffer till the time the record satisfying the condition deptno=10 is
feteched into the buffer.
Once that record is fetched, it satisfies our condition.This is also the commit point for our transaction. So the
transaction is committed after this record is fetched. All the 6 records go to target table.
The next record satisfying the condition is at position 8.
WRT_8036 Target: DIM_EMP (Instance Name: [DIM_EMP])
WRT_8038 Inserted rows - Requested: 8
Applied: 8
Rejected: 0
Affected: 8
WRITER_1_*_1> WRT_8317
USER-DEFINED COMMIT POINT Fri Aug 14 14:01:12 2009
===================================================
WRT_8036 Target: DIM_EMP (Instance Name: [DIM_EMP])
WRT_8038 Inserted rows - Requested: 13
Applied: 13
Rejected: 0
Affected: 13
WRITER_1_*_1> WRT_8317
USER-DEFINED COMMIT POINT Fri Aug 14 14:01:12 2009
===================================================
WRT_8036 Target: DIM_EMP (Instance Name: [DIM_EMP])
WRT_8038 Inserted rows - Requested: 14
Applied: 14
Rejected: 0
Affected: 14
WRITER_1_*_1> WRT_8168 End loading table [DIM_EMP] at: Fri Aug 14 14:01:12 2009
WRITER_1_*_1> WRT_8143
Commit at end of Load Order Group Fri Aug 14 14:01:12 2009
===================================================
WRT_8036 Target: DIM_EMP (Instance Name: [DIM_EMP])
WRT_8038 Inserted rows - Requested: 15
Applied: 15
Rejected: 0
Affected: 15
In the Session Properties the option Commit On End Of File is checked. So the last record which is mot
satisfying the condition also gets inserted and committed.
Uncheck the option Commit On End Of File and see the output
Affected: 6
WRITER_1_*_1> WRT_8317
USER-DEFINED COMMIT POINT Fri Aug 14 14:28:25 2009
===================================================
WRT_8036 Target: DIM_EMP (Instance Name: [DIM_EMP])
WRT_8038 Inserted rows - Requested: 8
Applied: 8
Rejected: 0
Affected: 8
WRITER_1_*_1> WRT_8317
USER-DEFINED COMMIT POINT Fri Aug 14 14:28:25 2009
===================================================
WRT_8036 Target: DIM_EMP (Instance Name: [DIM_EMP])
WRT_8038 Inserted rows - Requested: 13
Applied: 13
Rejected: 0
WRITER_1_*_1> WRT_8317
USER-DEFINED COMMIT POINT Fri Aug 14 14:28:25 2009
===================================================
Affected: 13
Rejected: 0
Affected: 14
Define Target.
Select the target(s) to create from XML source.
Relational -> OK.
XML requires a root.
Delete the root table from the WH designer.
Select all the tables and Generate/Execute SQL.
Connect these tables to the target.
Define the mapping drag xml source and target in the mapping designer.
Any transformation can be used in between.
Project the required ports from XML SQ to the TGT.
Save the repository.
Define the workflow.
Assign the connections.
Source File: C:\ (Directory) and assign the xml filename employees.xml
Multiple targets (Email,employees,Phone nos.,Salary)
When you add an XML source definition to a mapping, you need to connect it to an XML Source
Qualifier transformation. The XML Source Qualifier transformation defines the data elements
that the PowerCenter Server reads when it executes a session. It determines how the
PowerCenter reads the source data.
An XML Source Qualifier transformation always has one input or output port for every column
in the XML source. When you create an XML Source Qualifier transformation for a source
definition, the Designer links each port in the XML source definition to a port in the XML
Source Qualifier transformation. You cannot remove or edit any of the links. If you remove an
XML source definition from a mapping, the Designer also removes the corresponding XML
Source Qualifier transformation. You can link one XML source definition to one XML Source
Qualifier transformation
You can link ports of one XML Source Qualifier group to ports of different transformations to
form separate data flows. However, you cannot link ports from more than one group in an XML
Source Qualifier transformation to ports in the same target transformation.
You cannot use an XML Source Qualifier transformation in a mapplet.
You can add an XML Source Qualifier transformation to a mapping by dragging an XML source
definition to the Mapping Designer workspace or by manually creating one.
You can edit some of the properties and add metadata extensions to an XML Source Qualifier
transformation.
You can use an XML Parser transformation to extract XML inside a pipeline. The XML Parser
transformation enables you to extract XML data from messaging systems, such as TIBCO or MQ
Series, and from other sources, such as files or databases.
The XML Parser transformation functionality is similar to the XML source functionality, except
it parses the XML in the pipeline. For example, you might want to extract XML data from a
TIBCO source and pass the data to relational targets.
The XML Parser transformation reads XML data from a single input port and writes data to one
or more output ports.
You can use an XML Generator transformation to create XML inside a pipeline. The XML
Generator transformation enables you to read data from messaging systems, such as TIBCO and
MQ Series, or from other sources, such as files or databases. The XML Generator transformation
functionality is similar to the XML target functionality, except it generates the XML in the
pipeline. For example, you might want to extract data from relational sources and pass XML data
to targets.
The XML Generator transformation accepts data from multiple ports and writes XML through a
single output port.
Defining Targets
(Problem here)
NORMALIZER TRANSFORMATION
It is an active Transformation.
It is used to act like a SOURCE QUALIFIER when the sources are COBOL files.
Question: Apart from COBOL files can we use a NORMALIZER in between mappings?
Answer: Yes. It can also be used to convert de-normalized data to normalized.
Steps:
Define Source (COBOL file .cbl)
Structure and level of COBOL comes to the SQ.
Example:
NAME
LEVEL
SCHOOL_ID
CLASS
STUDENT
PARENT
TEACHER
SPORT
(6 target tables will be created here)
FILE_ONE
SCHOOL_ID
SCHOOL_NM
Occurrence
Data Type
CLASS_REC
GK_CLASS
GK_FILE_ONE
CLASS_ID
CLASS_NM
PARENT_NM
GK_STUDENT
PARENT_ID
PARENT_NM
STUDENT_REC
GK_STUDENT
GK_CLASS
STUDENT_ID
STUDENT_NM
TEACHER_REC
GK_TEACHER
TEACHER_ID
Create a mapping. Drag Source and Taget.
Autolink.
GK stand for Generated Key (it is prepared automatically)
Normalization is the process of organizing data. In database terms, this includes creating
normalized tables and establishing relationships between those tables according to rules designed
to both protect the data and make the database more flexible by eliminating redundancy and
inconsistent dependencies.
The Normalizer transformation normalizes records from COBOL and relational sources,
allowing you to organize the data according to your own needs. A Normalizer transformation can
appear anywhere in a pipeline when you normalize a relational source. Use a Normalizer
transformation instead of the Source Qualifier transformation when you normalize a COBOL
source. When you drag a COBOL source into the Mapping Designer workspace, the Mapping
Designer creates a Normalizer transformation with input and output ports for every column in the
source.
You primarily use the Normalizer transformation with COBOL sources, which are often stored in
a denormalized format. The OCCURS statement in a COBOL file nests multiple records of
information in a single record. Using the Normalizer transformation, you break out repeated data
within a record into separate records. For each new record it creates, the Normalizer
transformation generates a unique identifier. You can use this key value to join the normalized
records.
You can also use the Normalizer transformation with relational sources to create multiple rows
from a single row of data.
Rather than using two Normalizer transformations to handle the two different detail record sets,
you handle both normalizations in the same transformation.
Normalizer Ports
When you create a Normalizer for a COBOL source, or in the mapping pipeline, the Designer
identifies the OCCURS and REDEFINES statements and generates the following columns:
Generated key. One port for each REDEFINES clause. For more information, see Generated
Key.
Generated Column ID. One port for each OCCURS clause. For more information, see
Generated Column ID.
You can use these ports for primary and foreign key columns. The Normalizer key and column
ID columns are also useful when you want to pivot input columns into rows. You cannot delete
these ports.
Generated Key
The Designer generates a port for each REDEFINES clause to specify the generated key. You
can use the generated key as a primary key column in the target table and to create a primaryforeign key relationship. The naming convention for the Normalizer generated key is:
GK_<redefined_field_name>
As shown in Figure 10-1, the Designer adds one column (GK_FILE_ONE and GK_HST_AMT)
for each REDEFINES in the COBOL source. The Normalizer GK columns tell you the order of
records in a REDEFINES clause. For example, if a COBOL file has 10 records, when you run
the workflow, the PowerCenter Server numbers the first record 1, the second record 2, and so on.
You can create approximately two billion primary or foreign key values with the Normalizer by
connecting the GK port to the desired transformation or target and using the values ranging from
1 to 2147483647. At the end of each session, the PowerCenter Server updates the GK value to
the last value generated for the session plus one.
If you have multiple versions of the Normalizer transformation, the PowerCenter Server updates
the GK value across all versions when it runs a session.
If you open the mapping after you run the session, the current value displays the last value
generated for the session plus one. Since the PowerCenter Server uses the GK value to determine
the first value for each session, you should only edit the GK value if you want to reset the
sequence.
If you have multiple versions of the Normalizer, and you want to reset the sequence, you must
check in the mapping after you modify the GK value.
Generated Column ID
The Designer generates a port for each OCCURS clause to specify the positional index within an
OCCURS clause. You can use the generated column ID to create a primary-foreign key
relationship. The naming convention for the Normalizer generated column ID is:
GCID_<occuring_field_name>
As shown in Figure 10-1, the Designer adds one column (GCID_HST_MTH and
GCID_HST_AMT) for each OCCURS in the COBOL source. The Normalizer GCID columns
tell you the order of records in an OCCURS clause. For example, if a record occurs two times,
when you run the workflow, the PowerCenter Server numbers the first record 1 and the second
record 2.
Adding a COBOL Source to a Mapping
When you add a COBOL source to a mapping, the Mapping Designer inserts and configures a
Normalizer transformation. The Normalizer transformation identifies the nested records within
the COBOL source and displays them accordingly.
To add a COBOL source to a mapping:
1. In the Designer, create a new mapping or open an existing one.
2. Click and drag an imported COBOL source definition into the mapping.
If the Designer does not create a Normalizer transformation by default, manually create
the Normalizer transformation.
For example, when you add the COBOL source to a mapping, the Designer adds a
Normalizer transformation and connects it to the COBOL source definition.
Figure 10-1 illustrates that the ports representing HST_MTH appear separately within the
Normalizer transformation:
Figure 10-1. COBOL Source Definition and a Normalizer Transformation
If you connect the ports directly from the Normalizer transformation to targets, you
connect the records from HST_MTH, represented in the Normalizer transformation, to
their own target definition, distinct from any other target that may appear in the mapping.
3. Open the new Normalizer transformation.
4. Select the Ports tab and review the ports in the Normalizer transformation.
5. Click the Normalizer tab to review the original organization of the COBOL source.
This tab contains the same information as in the Columns tab of the source definition for
this COBOL source. However, you cannot modify the field definitions in the Normalizer
transformation. If you need to make modifications, open the source definition in the
Source Analyzer.
6. Select the Properties tab and enter the following settings:
Setting
Description
Reset
Resets the generated key value after the session finishes to its original value.
Restart
Restarts the generated key values from 1 every time you run a session.
Tracing
level
Determines the amount of information about this transformation that the PowerCenter
Server writes to the session log. You can override this tracing level when you
configure a session.
7. Click OK.
8. Connect the Normalizer transformation to the rest of the mapping.
If you have denormalized data for which the Normalizer transformation has created key values,
connect the ports representing the repeated data and the output port for the generated keys to a
different portion of the data flow in the mapping. Ultimately, you may want to write these values
to different targets.
To add a Normalizer transformation to a mapping:
1. In the Mapping Designer, choose Transformation-Create. Select Normalizer transformation. Enter
a name for the Normalizer transformation. Click Create.
Add a port corresponding to each column in the source record that contains denormalized
data. The new ports only allow the number or string datatypes. You can create only new
ports in the Normalizer tab, not the Ports tab.
Using the level controls in the Normalizer transformation, identify which ports belong to
the master and detail records. Adjust these ports so that the level setting for detail ports is
higher than the level setting for the master record. For example, if ports from the master
record are at level 1, the detail ports are at level 2. When you adjust the level setting for
the first detail port, the Normalizer transformation creates a heading for the detail record.
Enter the number of times detail records repeat within each master record.
6. After configuring the output ports, click Apply.
The Normalizer transformation creates all the input and output ports needed to connect
master and detail records to the rest of the mapping. In addition, the Normalizer
transformation creates a generated key column for joining master and detail records.
When you run a session, the PowerCenter Server generates unique IDs for these columns.
7. Select the Properties tab and enter the following settings:
Setting
Description
Reset
Restart
Tracing
level
Determines the amount of information PowerCenter Server writes to the session log.
You can override this tracing level when you configure a session.
8. Click OK.
9. Connect the Normalizer transformation to the rest of the mapping.
10. Choose Repository-Save.
You can only concatenate ports from level one. Remove the concatenation.
Pipeline Normalizer
Transformation
Connection
COBOL source
Any transformation
Port creation
Created manually
Ni-or-1 rule
Yes
Yes
No
Yes
Yes
Yes
Reusable
No
Yes
Ports
Input/Output
Input/Output
Note: Concatenation from the Normalizer transformation occurs only when the row sets being
concatenated are of the order one. You cannot concatenate row sets in which the order is greater
than one.
Scheduling Workflow
When we create a workflow using Wizard (i.e. non-reusable tasks)
By default, the run option is Run On Demand.
NUMBER := 0;
v_month
NUMBER := 0;
v_quarter
NUMBER := 0;
v_year
NUMBER := 0;
BEGIN
WHILE v_start_date <= v_end_date
LOOP
v_week := TO_CHAR(V_start_date,'ww');
v_month := TO_CHAR(V_start_date,'mm');
v_quarter := TO_CHAR(V_start_date,'q');
v_year:= TO_CHAR(V_start_date,'yyyy');
T_LKP_TIME
Condition
LKP_LOC
Condition
LKP_LOC_ID
Condition
Define parameter (constant value which is defined before the mapping is run).
Now Edit the Filter Transformation and in the Properties Tab click on Filter Condition.
In the Variables tab, select Mapping Parameters $$RANKSAL.
To change the parameter value modify .prm file and right click on the session and restart the task.
Mapping Variable
Consider the example of filter transformation M_FILTER_PARA in which we want to project the details
of employees belonging to detpno 10.
Define a variable $$DEPTNO.
Variable represents a value that can be changed during mapping run.
Define a Workflow.
Question: After running for the first time, where does the value get stored?
Answer: In the repository,
e.g. $$DNO = 0 (Initial Value)
$$DNO = 0 + 10 (Saved in the repository)
Next run $DNO is fetched from the repository.
$$DNO = $$DNO + 10
SQL OVERRIDE
Default Query
For relational sources, the PowerCenter Server generates a query for each Source Qualifier
transformation when it runs a session. The default query is a SELECT statement for each source
column used in the mapping. In other words, the PowerCenter Server reads only the columns that
are conneted to another transformation.
Although there are many columns in the source definition, only three columns are connected to
another transformation. In this case, the PowerCenter Server generates a default query that
selects only those three columns:
SELECT CUSTOMERS.CUSTOMER_ID, CUSTOMERS.COMPANY,
CUSTOMERS.FIRST_NAME
FROM CUSTOMERS
When generating the default query, the Designer delimits table and field names containing the
slash character (/) with double quotes.
Viewing the Default Query
You can view the default query in the Source Qualifier transformation.
To view the default query:
1. From the Properties tab, select SQL Query.
The SQL Editor displays the default query the PowerCenter Server uses to select source
data.
3. Click Cancel to exit.
Note: If you do not cancel the SQL query, the PowerCenter Server overrides the default query
with the custom SQL query.
Do not connect to the source database. You only connect to the source database when you enter
an SQL query that overrides the default query.
Tip: You must connect the columns in the Source Qualifier transformation to another
transformation or target before you can generate the default query.
Overriding the Default Query
You can alter or override the default query in the Source Qualifier transformation by changing
the default settings of the transformation properties. Do not change the list of selected ports or
the order in which they appear in the query. This list must match the connected transformation
output ports.
When you edit transformation properties, the Source Qualifier transformation includes these
settings in the default query. However, if you enter an SQL query, the PowerCenter Server uses
only the defined SQL statement. The SQL Query overrides the User-Defined Join, Source Filter,
Number of Sorted Ports, and Select Distinct settings in the Source Qualifier transformation.