Transform SQL queries and scripts in Informatica

6/4/2015
behappyandmakeotherstobehappy:TRANSFORMATIONSPART2ININFORMATICA
MONDAY,16JULY2012
TRANSFORMATIONSPART2ININFORMATICA
SQLTRANSFORMATION:
YoucanpassthedatabaseconnectioninformationtotheSQLtransformationasinputdataatruntime.Thetransformationprocessesexternal
SQLscriptsorSQLqueriesthatyoucreateinanSQLeditor.TheSQLtransformationprocessesthequeryandreturnsrowsanddatabase
errors.
WhenyoucreateanSQLtransformation,youconfigurethefollowingoptions:
Mode:TheSQLtransformationrunsinoneofthefollowingmodes:
Scriptmode.TheSQLtransformationrunsANSISQLscriptsthatareexternallylocated.Youpassascriptnametothe
transformationwitheachinputrow.TheSQLtransformationoutputsonerowforeachinputrow.
Querymode.TheSQLtransformationexecutesaquerythatyoudefineinaqueryeditor.Youcanpassstringsorparametersto
thequerytodefinedynamicqueriesorchangetheselectionparameters.Youcanoutputmultiplerowswhenthequeryhasa
SELECTstatement.
Passiveoractivetransformation.TheSQLtransformationisanactivetransformationbydefault.Youcanconfigureitasa
passivetransformationwhenyoucreatethetransformation.
Databasetype.ThetypeofdatabasetheSQLtransformationconnectsto.
Connectiontype.PassdatabaseconnectioninformationtotheSQLtransformationoruseaconnectionobject.
ScriptMode
AnSQLtransformationrunninginscriptmoderunsSQLscriptsfromtextfiles.YoupasseachscriptfilenamefromthesourcetotheSQL
transformationScriptNameport.Thescriptfilenamecontainsthecompletepathtothescriptfile.
Whenyouconfigurethetransformationtoruninscriptmode,youcreateapassivetransformation.Thetransformationreturnsonerowforeach
inputrow.Theoutputrowcontainsresultsofthequeryandanydatabaseerror.
RulesandGuidelinesforScriptMode
UsethefollowingrulesandguidelinesforanSQLtransformationthatrunsinscriptmode:
Youcanuseastaticordynamicdatabaseconnectionwithscriptmode.
http://krishnareddyoracleapps.blogspot.in/2012/07/transformationspart2ininformatica.html
1/51
6/4/2015
Toincludemultiplequerystatementsinascript,youcanseparatethemwithasemicolon.
Youcanusemappingvariablesorparametersinthescriptfilename.
Thescriptcodepagedefaultstothelocaleoftheoperatingsystem.Youcanchangethelocaleofthescript.
ThescriptfilemustbeaccessiblebytheIntegrationService.TheIntegrationServicemusthavereadpermissionsonthedirectory
thatcontainsthescript.
TheIntegrationServiceignorestheoutputofanySELECTstatementyouincludeintheSQLscript.TheSQLtransformationin
scriptmodedoesnotoutputmorethanonerowofdataforeachinputrow.
YoucannotusescriptinglanguagessuchasOraclePL/SQLorMicrosoft/SybaseTSQLinthescript.
YoucannotusenestedscriptswheretheSQLscriptcallsanotherSQLscript.
Ascriptcannotacceptruntimearguments.
QueryMode
WhenyouconfiguretheSQLtransformationtoruninquerymode,youcreateanactivetransformation.
WhenanSQLtransformationrunsinquerymode,itexecutesanSQLquerythatyoudefineinthetransformation.
Youpassstringsorparameterstothequeryfromthetransformationinputportstochangethequerystatementorthequerydata.
YoucancreatethefollowingtypesofSQLqueriesintheSQLtransformation:
StaticSQLquery.Thequerystatementdoesnotchange,butyoucanusequeryparameterstochangethedata.TheIntegration
Servicepreparesthequeryonceandrunsthequeryforallinputrows.
DynamicSQLquery.Youcanchangethequerystatementsandthedata.TheIntegrationServicepreparesaqueryforeach
inputrow.
RulesandGuidelinesforQueryMode
UsethefollowingrulesandguidelineswhenyouconfiguretheSQLtransformationtoruninquerymode:
ThenumberandtheorderoftheoutputportsmustmatchthenumberandorderofthefieldsinthequerySELECTclause.
Thenativedatatypeofanoutputportinthetransformationmustmatchthedatatypeofthecorrespondingcolumninthedatabase.
TheIntegrationServicegeneratesarowerrorwhenthedatatypesdonotmatch.
WhentheSQLquerycontainsanINSERT,UPDATE,orDELETEclause,thetransformationreturnsdatatotheSQLErrorport,
thepassthroughports,andtheNumRowsAffectedportwhenitisenabled.IfyouaddoutputportstheportsreceiveNULLdata
values.
2/51
6/4/2015
WhentheSQLquerycontainsaSELECTstatementandthetransformationhasapassthroughport,thetransformationreturns
datatothepassthroughportwhetherornotthequeryreturnsdatabasedata.TheSQLtransformationreturnsarowwithNULL
dataintheoutputports.
Youcannotaddthe"_output"suffixtooutputportnamesthatyoucreate.
YoucannotusethepassthroughporttoreturndatafromaSELECTquery.
WhenthenumberofoutputportsismorethanthenumberofcolumnsintheSELECTclause,theextraportsreceiveaNULL
value.
WhenthenumberofoutputportsislessthanthenumberofcolumnsintheSELECTclause,theIntegrationServicegeneratesa
rowerror.
Youcanusestringsubstitutioninsteadofparameterbindinginaquery.However,theinputportsmustbestringdatatypes.
SQLTransformationProperties
AfteryoucreatetheSQLtransformation,youcandefineportsandsetattributesinthefollowingtransformationtabs:
Ports.DisplaysthetransformationportsandattributesthatyoucreateontheSQLPortstab.
Properties.SQLtransformationgeneralproperties.
SQLSettings.AttributesuniquetotheSQLtransformation.
SQLPorts.SQLtransformationportsandattributes.
Note:YoucannotupdatethecolumnsonthePortstab.WhenyoudefineportsontheSQLPortstab,theydisplayonthePortstab.
PropertiesTab
ConfiguretheSQLtransformationgeneralpropertiesonthePropertiestab.SometransformationpropertiesdonotapplytotheSQL
transformationorarenotconfigurable.
ThefollowingtabledescribestheSQLtransformationproperties:
Property
Description
RunTimeLocation
EnterapathrelativetotheIntegrationServicenodethatrunsthe
SQLtransformationsession.
Ifthispropertyisblank,theIntegrationServiceusesthe
environmentvariabledefinedontheIntegrationServicenodeto
locatetheDLLorsharedlibrary.
YoumustcopyallDLLsorsharedlibrariestotheruntimelocation
3/51
6/4/2015
ortotheenvironmentvariabledefinedontheIntegrationService
node.TheIntegrationServicefailstoloadtheprocedurewhenit
cannotlocatetheDLL,sharedlibrary,orareferencedfile.
TracingLevel
Setstheamountofdetailincludedinthesessionlogwhenyou
runasessioncontainingthistransformation.Whenyouconfigure
theSQLtransformationtracingleveltoVerboseData,the
IntegrationServicewriteseachSQLqueryitpreparestothe
sessionlog.
IsPartitionable
Multiplepartitionsinapipelinecanusethistransformation.Use
thefollowingoptions:
No.Thetransformationcannotbepartitioned.The
transformationandothertransformationsinthesamepipelineare
limitedtoonepartition.YoumightchooseNoifthetransformation
processesalltheinputdatatogether,suchasdatacleansing.
Locally.Thetransformationcanbepartitioned,butthe
IntegrationServicemustrunallpartitionsinthepipelineonthe
samenode.ChooseLocallywhendifferentpartitionsofthe
transformationmustshareobjectsinmemory.
AcrossGrid.Thetransformationcanbepartitioned,andthe
IntegrationServicecandistributeeachpartitiontodifferentnodes.
DefaultisNo.
UpdateStrategy
Thetransformationdefinestheupdatestrategyforoutputrows.
YoucanenablethispropertyforquerymodeSQL
transformations.
Defaultisdisabled.
TransformationScope ThemethodinwhichtheIntegrationServiceappliesthe
transformationlogictoincomingdata.Usethefollowingoptions:
Row
Transaction
AllInput
4/51
6/4/2015
Settransactionscopetotransactionwhenyouusetransaction
controlinstaticquerymode.
DefaultisRowforscriptmodetransformations.DefaultisAllInput
forquerymodetransformations.
OutputisRepeatable
Indicatesiftheorderoftheoutputdataisconsistentbetween
sessionruns.
Never.Theorderoftheoutputdataisinconsistentbetween
sessionruns.
BasedOnInputOrder.Theoutputorderisconsistentbetween
sessionrunswhentheinputdataorderisconsistentbetween
sessionruns.
Always.Theorderoftheoutputdataisconsistentbetween
sessionrunseveniftheorderoftheinputdataisinconsistent
betweensessionruns.
DefaultisNever.
GenerateTransaction
Thetransformationgeneratestransactionrows.Enablethis
propertyforquerymodeSQLtransformationsthatcommitdatain
anSQLquery.
Defaultisdisabled.
RequiresSingle
ThreadPerPartition
IndicatesiftheIntegrationServiceprocesseseachpartitionofa
procedurewithonethread.
OutputisDeterministic Thetransformationgenerateconsistentoutputdatabetween
sessionruns.Enablethispropertytoperformrecoveryon
sessionsthatusethistransformation.
Defaultisenabled.
CreateMapping:
Step1:Creatingaflatfileandimportingthesourcefromtheflatfile.
CreateaNotepadandinitcreateatablebynamebikeswiththreecolumnsandthreerecordsinit.
5/51
6/4/2015
Createonemorenotepadandnameitaspathforthebikes.InsidetheNotepadjusttypein(C:\bikes.txt)andsaveit.
Importthesource(secondnotepad)usingthesource>importfromthefile.Afterwhichwearegoannagetawizardwiththree
subsequentwindowsandfollowtheonscreeninstructionstocompletetheprocessofimportingthesource.
Step2:Importingthetargetandapplyingthetransformation.
Inthesamewayasspecifiedabovegotothetargets>importfromfileandselectanemptynotepadunderthenametargetforbikes(thisisone
moreblanknotepadwhichweshouldcreateandsaveundertheabovespecifiednameintheC:\).
Createtwocolumnsinthetargettableunderthenamereportanderror.
Weareallsethere.NowapplytheSQLtransformation.
InthefirstwindowwhenyouapplytheSQLtransformationweshouldselectthescriptmode.
ConnecttheSQtotheScriptNameunderinputsandconnecttheothertwofieldstotheoutputcorrespondingly.
Snapshotfortheabovediscussedthingsisgivenbelow.
6/51
6/4/2015
Step3:Designtheworkflowandrunit.
Createthetaskandtheworkflowusingthenamingconventions.
GotothemappingstabandclickontheSourceonthelefthandpanetospecifythepathfortheoutputfile.
Step4:Previewtheoutputdataonthetargettable.
================================================================
NORMALIZERTRANSFORMATION:
ActiveandConnectedTransformation.
TheNormalizertransformationnormalizesrecordsfromCOBOLandrelationalsources,allowingustoorganizethedata.
7/51
6/4/2015
UseaNormalizertransformationinsteadoftheSourceQualifiertransformationwhenwenormalizeaCOBOLsource.
WecanalsousetheNormalizertransformationwithrelationalsourcestocreatemultiplerowsfromasinglerowofdata.
Example1:Tocreate4recordsofeveryemployeeinEMPtable.
EMPwillbesourcetable.
CreatetargettableNormalizer_Multiple_Records.StructuresameasEMPanddatatypeofHIREDATEasVARCHAR2.
Createshortcutsasnecessary.
CreatingMapping:
1. Openfolderwherewewanttocreatethemapping.
2. ClickTools>MappingDesigner.
3. ClickMapping>Create>Givename.Ex:m_Normalizer_Multiple_Records
4. DragEMPandTargettable.
5. Transformation>Create>SelectExpression>Givename,Clickcreate,done.
6. PassallportsfromSQ_EMPtoExpressiontransformation.
7. Transformation>Create>SelectNormalizer>Givename,create&done.
8. TrydraggingportsfromExpressiontoNormalizer.NotPossible.
9. EditNormalizerandNormalizerTab.Addcolumns.ColumnsequaltocolumnsinEMPtableanddatatypealsosame.
10. NormalizerdoesnthaveDATETIMEdatatype.SoconvertHIREDATEtocharinexpressiont/f.Createoutputportout_hdateanddothe
conversion.
11. ConnectportsfromExpressiontoNormalizer.
12. EditNormalizerandNormalizerTab.AsEMPNOidentifiessourcerecordsandwewant4recordsofeveryemployee,giveOCCURfor
EMPNOas4.
13.
14. ClickApplyandthenOK.
8/51
6/4/2015
15. Addlinkasshowninmappingbelow:
16. Mapping>Validate
17. Repository>Save
Makesessionandworkflow.
Giveconnectioninformationforsourceandtargettable.
Runworkflowandseeresult.
Example2:Tobreakrowsintocolumns
Source:
Roll_Number Name
ENG
HINDI
MATHS
100
Amit
78
76
90
101
Rahul
76
78
87
102
Jessie
65
98
79
Target:
Roll_Number
Name
Marks
100
Amit
78
100
Amit
76
100
Amit
90
101
Rahul
76
101
Rahul
78
101
Rahul
87
102
Jessie
65
102
Jessie
98
102
Jessie
79
9/51
6/4/2015
Makesourceasaflatfile.Importitandcreatetargettable.
CreateMappingasbefore.InNormalizertab,createonly3portsRoll_Number,NameandMarksasthereare3columnsintarget
table.
Alsoaswehave3marksinsource,giveOccursas3forMarksinNormalizertab.
Connectaccordinglyandconnecttotarget.
ValidateandSave
MakeSessionandworkflowandRunit.GiveSourceFileDirectoryandSourceFilenameforsourceflatfileinsourcepropertiesin
mappingtabofsession.
Seetheresult.
====================================================================
SEQUENCEGENERATORTRANSFORMATION:
PassiveandConnectedTransformation.
TheSequenceGeneratortransformationgeneratesnumericvalues.
UsetheSequenceGeneratortocreateuniqueprimarykeyvalues,replacemissingprimarykeys,orcyclethroughasequential
rangeofnumbers.
WeuseittogenerateSurrogateKeyinDWHenvironmentmostly.WhenwewanttoMaintainhistory,thenweneedakeyotherthanPrimary
Keytouniquelyidentifytherecord.SowecreateaSequence1,2,3,4andsoon.Weusethissequenceasthekey.Example:IfEMPNOisthe
key,wecankeeponlyonerecordintargetandcantmaintainhistory.SoweuseSurrogatekeyasPrimarykeyandnotEMPNO.
SequenceGeneratorPorts:
TheSequenceGeneratortransformationprovidestwooutputports:NEXTVALandCURRVAL.
Wecannoteditordeletetheseports.
Likewise,wecannotaddportstothetransformation.
NEXTVAL:
UsetheNEXTVALporttogeneratesequencenumbersbyconnectingittoaTransformationortarget.
Forexample,wemightconnectNEXTVALtotwotargettablesinamappingtogenerateuniqueprimarykeyvalues.
10/51
6/4/2015
SequenceinTable1willbegeneratedfirst.Whentable1hasbeenloaded,onlythenSequencefortable2willbegenerated.
CURRVAL:
CURRVALisNEXTVALplustheIncrementByvalue.
WetypicallyonlyconnecttheCURRVALportwhentheNEXTVALportisAlreadyconnectedtoadownstreamtransformation.
IfweconnecttheCURRVALportwithoutconnectingtheNEXTVALport,theIntegrationServicepassesaconstantvalueforeach
row.
whenweconnecttheCURRVALportinaSequenceGeneratorTransformation,theIntegrationServiceprocessesonerowin
eachblock.
WecanoptimizeperformancebyconnectingonlytheNEXTVALportinaMapping.
11/51
6/4/2015
Example:TouseSequenceGeneratortransformation
EMPwillbesource.
CreateatargetEMP_SEQ_GEN_EXAMPLEinsharedfolder.StructuresameasEMP.AddtwomoreportsNEXT_VALUEand
CURR_VALUEtothetargettable.
Createshortcutsasneeded.
CreatingMapping:
1.Openfolderwherewewanttocreatethemapping.
2.ClickTools>MappingDesigner.
3.ClickMapping>Create>Givename.Ex:m_seq_gen_example
4.DragEMPandTargettable.
5.ConnectallportsfromSQ_EMPtotargettable.
6.Transformation>Create>SelectSequenceGeneratorforlist>Create>Done
7.ConnectNEXT_VALandCURR_VALfromSequenceGeneratortotarget.
8.ValidateMapping
9.Repository>Save
CreateSessionandthenworkflow.
12/51
6/4/2015
Giveconnectioninformationforalltables.
Runworkflowandseetheresultintable.
SequenceGeneratorProperties:
Setting
Required/Optional Description
StartValue
Required
StartvalueofthegeneratedsequencethatwewantIS
touseifweuseCycleoption.Defaultis0.
IncrementBy Required
Differencebetweentwoconsecutivevaluesfromthe
NEXTVALport.
EndValue
MaximumvaluetheIntegrationServicegenerates.
Optional
CurrentValue Optional
Firstvalueinthesequence.Ifcycleoptionused,the
valuemustbegreaterthanorequaltothestartvalue
andlesstheendvalue.
Cycle
Optional
Ifselected,theIntegrationServicecyclesthroughthe
sequencerange.Ex:StartValue:1EndValue10
Sequencewillbefrom110andagainstartfrom1.
Reset
Optional
Bydefault,lastvalueofsequenceduringsessionis
savedtorepository.Nexttimethesequenceisstarted
fromthevaluedsaved.
Ifselected,theIntegrationServicegeneratesvalues
basedontheoriginalcurrentvalueforeachsession.
PointstoPonder:
IfCurrentvalueis1andendvalue10,nocycleoption.Thereare17recordsinsource.Inthiscasesessionwillfail.
IfweconnectjustCURR_VALonly,thevaluewillbesameforallrecords.
IfCurrentvalueis1andendvalue10,cycleoptionthere.Startvalueis0.
13/51
6/4/2015
Thereare17recordsinsource.Sequence:1210.0123
Tomakeabovesequenceas110120,giveStartValueas1.StartvalueisusedalongwithCycleoptiononly.
IfCurrentvalueis1andendvalue10,cycleoptionthere.Startvalueis1.
Thereare17recordsinsource.Sessionruns.11017.7willbesavedinrepository.Ifwerunsessionagain,sequencewillstart
from8.
UseresetoptionifyouwanttostartsequencefromCURR_VALeverytime.
=====================================================================
AGGREGATORTRANSFORMATION:
ConnectedandActiveTransformation
TheAggregatortransformationallowsustoperformaggregatecalculations,suchasaveragesandsums.
Aggregatortransformationallowsustoperformcalculationsongroups.
ComponentsoftheAggregatorTransformation
1. Aggregateexpression
2. Groupbyport
3. SortedInput
4. Aggregatecache
1)AggregateExpressions
Enteredinanoutputport.
Canincludenonaggregateexpressionsandconditionalclauses.
Thetransformationlanguageincludesthefollowingaggregatefunctions:
AVG,COUNT,MAX,MIN,SUM
FIRST,LAST
MEDIAN,PERCENTILE,STDDEV,VARIANCE
SingleLevelAggregateFunction:MAX(SAL)
NestedAggregateFunction:MAX(COUNT(ITEM))
NestedAggregateFunctions
14/51
6/4/2015
InAggregatortransformation,therecanbemultiplesinglelevelfunctionsormultiplenestedfunctions.
AnAggregatortransformationcannothavebothtypesoffunctionstogether.
MAX(COUNT(ITEM))iscorrect.
MIN(MAX(COUNT(ITEM)))isnotcorrect.Itcanalsoincludeoneaggregatefunctionnestedwithinanotheraggregatefunction
ConditionalClauses
Wecanuseconditionalclausesintheaggregateexpressiontoreducethenumberofrowsusedintheaggregation.Theconditionalclausecan
beanyclausethatevaluatestoTRUEorFALSE.
SUM(COMMISSION,COMMISSION>QUOTA)
NonAggregateFunctions
Wecanalsousenonaggregatefunctionsintheaggregateexpression.
IIF(MAX(QUANTITY)>0,MAX(QUANTITY),0))
2)GroupByPorts
Indicateshowtocreategroups.
Whengroupingdata,theAggregatortransformationoutputsthelastrowofeachgroupunlessotherwisespecified.
TheAggregatortransformationallowsustodefinegroupsforaggregations,ratherthanperformingtheaggregationacrossallinputdata.
Forexample,wecanfindMaximumSalaryforeveryDepartment.
InAggregatorTransformation,OpenPortstabandselectGroupByasneeded.
3)UsingSortedInput
Usetoimprovesessionperformance.
Tousesortedinput,wemustpassdatatotheAggregatortransformationsortedbygroupbyport,inascendingordescending
order.
Whenweusethisoption,wetellAggregatorthatdatacomingtoitisalreadysorted.
WechecktheSortedInputOptioninPropertiesTabofthetransformation.
Iftheoptionischeckedbutwearenotpassingsorteddatatothetransformation,thenthesessionfails.
4)AggregatorCaches
ThePowerCenterServerstoresdataintheaggregatecacheuntilitcompletesAggregatecalculations.
15/51
6/4/2015
Itstoresgroupvaluesinanindexcacheandrowdatainthedatacache.IfthePowerCenterServerrequiresmorespace,itstores
overflowvaluesincachefiles.
Note:ThePowerCenterServerusesmemorytoprocessanAggregatortransformationwithsortedports.Itdoesnotusecachememory.We
donotneedtoconfigurecachememoryforAggregatortransformationsthatusesortedports.
1)AggregatorIndexCache:
Theindexcacheholdsgroupinformationfromthegroupbyports.IfweareusingGroupByonDEPTNO,thenthiscachestoresvalues10,20,
30etc.
AllGroupByColumnsareinAGGREGATORINDEXCACHE.Ex.DEPTNO
2)AggregatorDataCache:
DATACACHEisgenerallylargerthantheAGGREGATORINDEXCACHE.
ColumnsinDataCache:
Variableportsifany
Nongroupbyinput/outputports.
Nongroupbyinputportsusedinnonaggregateoutputexpression.
Portcontainingaggregatefunction
16/51
6/4/2015
1)Example:TocalculateMAX,MIN,AVGandSUMofsalaryofEMPtable.
EMPwillbesourcetable.
CreateatargettableEMP_AGG_EXAMPLEintargetdesigner.TableshouldcontainDEPTNO,MAX_SAL,MIN_SAL,AVG_SAL
andSUM_SAL
Createtheshortcutsinyourfolder.
CreatingMapping:
3.ClickMapping>Create>Givemappingname.Ex:m_agg_example
4.DragEMPfromsourceinmapping.
5.ClickTransformation>Create>SelectAGGREGATORfromlist.GivenameandclickCreate.Nowclickdone.
6.PassSALandDEPTNOonlyfromSQ_EMPtoAGGREGATORTransformation.
7.EditAGGREGATORTransformation.GotoPortsTab
8.Create4outputports:OUT_MAX_SAL,OUT_MIN_SAL,OUT_AVG_SAL,
17/51
6/4/2015
OUT_SUM_SAL
9.OpenExpressionEditoronebyoneforalloutputportsandgivethe
calculations.Ex:MAX(SAL),MIN(SAL),AVG(SAL),SUM(SAL)
10.ClickApply>Ok.
11.Dragtargettablenow.
12.ConnecttheoutputportsfromRanktotargettable.
13.ClickMapping>Validate
14.Repository>Save
CreateSessionandWorkflowasdescribedearlier.RuntheWorkflowandseethedataintargettable.
Makesuretogiveconnectioninformationforalltables.
==================================================================
UNIONTRANSFORMATION:
ActiveandConnectedtransformation.
Uniontransformationisamultipleinputgrouptransformationthatyoucanusetomergedatafrommultiplepipelinesorpipelinebranchesinto
onepipelinebranch.ItmergesdatafrommultiplesourcessimilartotheUNIONALLSQLstatementtoCombinetheresultsfromtwoormore
SQLstatements.
UnionTransformationRulesandGuidelines
wecancreatemultipleinputgroups,butonlyoneoutputgroup.
wecanconnectheterogeneoussourcestoaUniontransformation.
allinputgroupsandtheoutputgroupmusthavematchingports.ThePrecision,datatype,andscalemustbeidenticalacrossall
groups.
TheUniontransformationdoesnotremoveduplicaterows.ToremoveDuplicaterows,wemustaddanothertransformationsuch
asaRouterorFilterTransformation.
wecannotuseaSequenceGeneratororUpdateStrategytransformationupstreamfromaUniontransformation.
UnionTransformationComponents
WhenweconfigureaUniontransformation,definethefollowingcomponents:
Transformationtab:Wecanrenamethetransformationandaddadescription.
Propertiestab:Wecanspecifythetracinglevel.
Groupstab:Wecancreateanddeleteinputgroups.TheDesignerdisplaysgroupswecreateonthePortstab.
18/51
6/4/2015
GroupPortstab:Wecancreateanddeleteportsfortheinputgroups.TheDesignerdisplaysportswecreateonthePortstab.
WecannotmodifythePorts,InitializationProperties,MetadataExtensions,orPortAttributeDefinitionstabsinaUniontransformation.
CreateinputgroupsontheGroupstab,andcreateportsontheGroupPortstab.WecancreateoneormoreinputgroupsontheGroupstab.
TheDesignercreatesoneoutputgroupbydefault.Wecannoteditordeletethedefaultoutputgroup.
Example:tocombinedataoftablesEMP_10,EMP_20andEMP_REST
ImporttablesEMP_10,EMP_20andEMP_RESTinsharedfolderinSources.
CreateatargettableEMP_UNION_EXAMPLEintargetdesigner.StructureshouldbesameEMPtable.
CreatingMapping:
19/51
6/4/2015
3. ClickMapping>Create>Givemappingname.Ex:m_union_example
4. DragEMP_10,EMP_20andEMP_RESTfromsourceinmapping.
5. ClickTransformation>Create>SelectUnionfromlist.GivenameandclickCreate.Nowclickdone.
6. PassportsfromSQ_EMP_10toUnionTransformation.
7. EditUnionTransformation.GotoGroupsTab
8. OnegroupwillbealreadythereaswedraggedportsfromSQ_DEPT_10toUnionTransformation.
9. Aswehave3sourcetables,we3need3inputgroups.Clickaddbuttontoadd2moregroups.SeeSampleMapping
10. Wecanalsomodifyportsinportstab.
11. ClickApply>Ok.
12. Dragtargettablenow.
13. ConnecttheoutputportsfromUniontotargettable.
14. ClickMapping>Validate
15. Repository>Save
Makesuretogiveconnectioninformationforall3sourceTables.
20/51
6/4/2015
Samplemappingpicture
=======================================================================
JOINERTRANSFORMATION:
ConnectedandActiveTransformation
UsedtojoinsourcedatafromtworelatedheterogeneoussourcesresidinginDifferentlocationsorfilesystems.Or,wecanjoin
datafromthesamesource.
Ifweneedtojoin3tables,thenweneed2JoinerTransformations.
TheJoinertransformationjoinstwosourceswithatleastonematchingport.TheJoinertransformationusesaconditionthat
matchesoneormorepairsofPortsbetweenthetwosources.
Example:TojoinEMPandDEPTtables.
21/51
6/4/2015
EMPandDEPTwillbesourcetable.
CreateatargettableJOINER_EXAMPLEintargetdesigner.TableshouldContainallportsofEMPtableplusDNAMEandLOC
asshownbelow.
CreatingMapping:
3. ClickMapping>Create>Givemappingname.Ex:m_joiner_example
4. DragEMP,DEPT,andTarget.CreateJoinerTransformation.Linkasshownbelow.
5.SpecifythejoinconditioninConditiontab.Seestepsonnextpage.
6.SetMasterinPortstab.Seestepsonnextpage.
7.Mapping>Validate
8.Repository>Save.
22/51
6/4/2015
JOINCONDITION:
ThejoinconditioncontainsportsfrombothinputsourcesthatmustmatchforthePowerCenterServertojointworows.
Example:DEPTNO=DEPTNO1inabove.
1. EditJoinerTransformation>ConditionTab
2. Addcondition
Wecanaddasmanyconditionsasneeded.
Only=operatorisallowed.
IfwejoinCharandVarchardatatypes,thePowerCenterServercountsanyspacesthatpadCharvaluesaspartofthestring.Soifyoutryto
jointhefollowing:
Char(40)=abcdandVarchar(40)=abcd
ThentheCharvalueisabcdpaddedwith36blankspaces,andthePowerCenterServerdoesnotjointhetwofieldsbecausetheCharfield
containstrailingspaces.
Note:TheJoinertransformationdoesnotmatchnullvalues.
MASTERandDETAILTABLES
InJoiner,onetableiscalledasMASTERandotherasDETAIL.
MASTERtableisalwayscached.WecanmakeanytableasMASTER.
EditJoinerTransformation>PortsTab>SelectMforMastertable.
TablewithlessnumberofrowsshouldbemadeMASTERtoimprovePerformance.
Reason:
WhenthePowerCenterServerprocessesaJoinertransformation,itreadsrowsfrombothsourcesconcurrentlyandbuildsthe
indexanddatacachebasedonthemasterrows.Sotablewithfewerrowswillbereadfastandcachecanbemadeastablewith
morerowsisstillbeingread.
Thefeweruniquerowsinthemaster,thefeweriterationsofthejoincomparisonoccur,whichspeedsthejoinprocess.
JOINERTRANSFORMATIONPROPERTIESTAB
CaseSensitiveStringComparison:Ifselected,thePowerCenterServerusescasesensitivestringcomparisonswhen
23/51
6/4/2015
performingjoinsonstringcolumns.
CacheDirectory:Specifiesthedirectoryusedtocachemasterordetailrowsandtheindextotheserows.
JoinType:Specifiesthetypeofjoin:Normal,MasterOuter,DetailOuter,orFullOuter.
TracingLevel
JoinerDataCacheSize
JoinerIndexCacheSize
SortedInput
JOINTYPES
InSQL,ajoinisarelationaloperatorthatcombinesdatafrommultipletablesintoasingleresultset.TheJoinertransformationactsinmuchthe
samemanner,exceptthattablescanoriginatefromdifferentdatabasesorflatfiles.
TypesofJoins:
Normal
MasterOuter
DetailOuter
FullOuter
Note:Anormalormasterouterjoinperformsfasterthanafullouterordetailouterjoin.
Example:InEMP,wehaveemployeeswithDEPTNO10,20,30and50.InDEPT,wehaveDEPTNO10,20,30and40.DEPTwillbe
MASTERtableasithaslessrows.
NormalJoin:
Withanormaljoin,thePowerCenterServerdiscardsallrowsofdatafromthemasteranddetailsourcethatdonotmatch,basedonthe
condition.
Allemployeesof10,20and30willbethereasonlytheyarematching.
MasterOuterJoin:
Thisjoinkeepsallrowsofdatafromthedetailsourceandthematchingrowsfromthemastersource.Itdiscardstheunmatchedrowsfromthe
mastersource.
Alldataofemployeesof10,20and30willbethere.
TherewillbeemployeesofDEPTNO50andcorrespondingDNAMEandLOCColumnswillbeNULL.
DetailOuterJoin:
24/51
6/4/2015
Thisjoinkeepsallrowsofdatafromthemastersourceandthematchingrowsfromthedetailsource.Itdiscardstheunmatchedrowsfromthe
detailsource.
Allemployeesof10,20and30willbethere.
TherewillbeonerecordforDEPTNO40andcorrespondingdataofEMPcolumnswillbeNULL.
FullOuterJoin:
Afullouterjoinkeepsallrowsofdatafromboththemasteranddetailsources.
Alldataofemployeesof10,20and30willbethere.
TherewillbeemployeesofDEPTNO50andcorrespondingDNAMEandLOCColumnswillbeNULL.
TherewillbeonerecordforDEPTNO40andcorrespondingdataofEMPColumnswillbeNULL.
USINGSORTEDINPUT
Usetoimprovesessionperformance.
tousesortedinput,wemustpassdatatotheJoinertransformationsortedbytheportsthatareusedinJoinCondition.
WechecktheSortedInputOptioninPropertiesTabofthetransformation.
IftheoptionischeckedbutwearenotpassingsorteddatatotheTransformation,thenthesessionfails.
WecanuseSORTERtosortdataorSourceQualifierincaseofRelationaltables.
JOINERCACHES
JoineralwayscachestheMASTERtable.Wecannotdisablecaching.ItbuildsIndexcacheandDataCachebasedonMASTERtable.
1)JoinerIndexCache:
AllColumnsofMASTERtableusedinJoinconditionareinJOINERINDEXCACHE.
Example:DEPTNOinourmapping.
2)JoinerDataCache:
MastercolumnnotinjoinconditionandusedforoutputtoothertransformationortargettableareinDataCache.
Example:DNAMEandLOCinourmappingexample.
PerformanceTuning:
Performjoinsinadatabasewhenpossible.
Joinsorteddatawhenpossible.
25/51
6/4/2015
ForasortedJoinertransformation,designateasthemastersourcethesourcewithfewerduplicatekeyvalues.
Joinercan'tbeusedinfollowingconditions:
1. EitherinputpipelinecontainsanUpdateStrategytransformation.
2. WeconnectaSequenceGeneratortransformationdirectlybeforetheJoinertransformation.
=====================================================================
UpdateStraegyTRANSFORMATION:
ActiveandConnectedTransformation
Tillnow,wehaveonlyinsertedrowsinourtargettables.Whatifwewanttoupdate,deleteorrejectrowscomingfromsourcebasedonsome
condition?
Example:IfAddressofaCUSTOMERchanges,wecanupdatetheoldaddressorkeepbotholdandnewaddress.Onerowisforoldand
onefornew.Thiswaywemaintainthehistoricaldata.
UpdateStrategyisusedwithLookupTransformation.InDWH,wecreateaLookupontargettabletodeterminewhetherarowalreadyexists
ornot.Thenweinsert,update,deleteorrejectthesourcerecordasperbusinessneed.
InPowerCenter,wesettheupdatestrategyattwodifferentlevels:
1. Withinasession
2. WithinaMapping
1.UpdateStrategywithinasession:
Whenweconfigureasession,wecaninstructtheIStoeithertreatallrowsinthesamewayoruseinstructionscodedintothesessionmapping
toflagrowsfordifferentdatabaseoperations.
SessionConfiguration:
EditSession>Properties>TreatSourceRowsas:(Insert,Update,Delete,andDataDriven).Insertisdefault.SpecifyingOperationsfor
IndividualTargetTables:
26/51
6/4/2015
Youcansetthefollowingupdatestrategyoptions:
Insert:Selectthisoptiontoinsertarowintoatargettable.
Delete:Selectthisoptiontodeletearowfromatable.
Update:Wehavethefollowingoptionsinthissituation:
UpdateasUpdate.Updateeachrowflaggedforupdateifitexistsinthetargettable.
UpdateasInsert.Inseteachrowflaggedforupdate.
UpdateelseInsert.Updatetherowifitexists.Otherwise,insertit.
Truncatetable:Selectthisoptiontotruncatethetargettablebeforeloadingdata.
2.FlaggingRowswithinaMapping
Withinamapping,weusetheUpdateStrategytransformationtoflagrowsforinsert,delete,update,orreject.
Operation
Constant
Numeric
Value
27/51
6/4/2015
INSERT
DD_INSERT
UPDATE
DD_UPDATE
DELETE
DD_DELETE
REJECT
DD_REJECT
UpdateStrategyExpressions:
Frequently,theupdatestrategyexpressionusestheIIForDECODEfunctionfromthetransformationlanguagetotesteachrowtoseeifit
meetsaparticularcondition.
IIF((ENTRY_DATE>APPLY_DATE),DD_REJECT,DD_UPDATE)
Or
IIF((ENTRY_DATE>APPLY_DATE),3,2)
TheaboveexpressioniswritteninPropertiesTabofUpdateStrategyT/f.
DDmeansDATADRIVEN
ForwardingRejectedRows:
WecanconfiguretheUpdateStrategytransformationtoeitherpassrejectedrowstothenexttransformationordropthem.
Steps:
1. CreateUpdateStrategyTransformation
2. Passallportsneededtoit.
3. SettheExpressioninPropertiesTab.
4. Connecttoothertransformationsortarget.
Performancetuning:
1. UseUpdateStrategytransformationaslessaspossibleinthemapping.
2. Donotuseupdatestrategytransformationifwejustwanttoinsertintotargettable,insteadusedirectmapping,directfilteringetc.
3. ForupdatingordeletingrowsfromthetargettablewecanuseUpdateStrategytransformationitself.
==================================================================
LookupTRANSFORMATION:
28/51
6/4/2015
PassiveTransformation
CanbeConnectedorUnconnected.Dynamiclookupisconnected.
UseaLookuptransformationinamappingtolookupdatainaflatfileorarelationaltable,view,orsynonym.
WecanimportalookupdefinitionfromanyflatfileorrelationaldatabasetowhichboththePowerCenterClientandServercan
connect.
WecanusemultipleLookuptransformationsinamapping.
ThePowerCenterServerqueriesthelookupsourcebasedonthelookupportsinthetransformation.ItcomparesLookuptransformationport
valuestolookupsourcecolumnvaluesbasedonthelookupcondition.Passtheresultofthelookuptoothertransformationsandatarget.
WecanusetheLookuptransformationtoperformfollowing:
Getarelatedvalue:EMPhasDEPTNObutDNAMEisnotthere.WeuseLookuptogetDNAMEfromDEPTtablebasedon
LookupCondition.
Performacalculation:WewantonlythoseEmployeeswhosSAL>Average(SAL).WewillwriteLookupOverridequery.
Updateslowlychangingdimensiontables:Mostimportantuse.WecanuseaLookuptransformationtodeterminewhether
rowsalreadyexistinthetarget.
1.LOOKUPTYPES
WecanconfiguretheLookuptransformationtoperformthefollowingtypesoflookups:
ConnectedorUnconnected
RelationalorFlatFile
CachedorUncached
RelationalLookup:
WhenwecreateaLookuptransformationusingarelationaltableasalookupsource,wecanconnecttothelookupsourceusingODBCand
importthetabledefinitionasthestructurefortheLookuptransformation.
WecanoverridethedefaultSQLstatementifwewanttoaddaWHEREclauseorquerymultipletables.
Wecanuseadynamiclookupcachewithrelationallookups.
FlatFileLookup:
Whenweuseaflatfileforalookupsource,wecanuseanyflatfiledefinitionintherepository,orwecanimportit.Whenweimportaflatfile
lookupsource,theDesignerinvokestheFlatFileWizard.
29/51
6/4/2015
CachedorUncachedLookup:
WecanchecktheoptioninPropertiesTabtoCachetolookupornot.Bydefault,lookupiscached.
ConnectedandUnconnectedLookup
ConnectedLookup
UnconnectedLookup
Receivesinputvaluesdirectlyfromthe
pipeline.
Receivesinputvaluesfromtheresultofa
:LKPexpressioninanothertransformation.
Wecanuseadynamicorstaticcache.
Wecanuseastaticcache.
Cacheincludesalllookupcolumnsusedin
themapping.
Cacheincludesalllookup/outputportsinthe
lookupconditionandthelookup/returnport.
Ifthereisnomatchforthelookupcondition,
thePowerCenterServerreturnsthedefault
valueforalloutputports.
Ifthereisnomatchforthelookupcondition,
thePowerCenterServerreturnsNULL.
Ifthereisamatchforthelookupcondition,
Ifthereisamatchforthelookupcondition,the
thePowerCenterServerreturnstheresultof PowerCenterServerreturnstheresultofthe
thelookupconditionforalllookup/output
lookupconditionintothereturnport.
ports.
Passmultipleoutputvaluestoanother
transformation.
Passoneoutputvaluetoanother
transformation.
Supportsuserdefineddefaultvalues
Doesnotsupportuserdefineddefaultvalues.
30/51
6/4/2015
2.LOOKUPT/FCOMPONENTS
DefinethefollowingcomponentswhenweconfigureaLookuptransformationinamapping:
Lookupsource
Ports
Properties
Condition
1.LookupSource:
Wecanuseaflatfileorarelationaltableforalookupsource.WhenwecreateaLookupt/f,wecanimportthelookupsourcefromthefollowing
locations:
Anyrelationalsourceortargetdefinitionintherepository
Anyflatfilesourceortargetdefinitionintherepository
AnytableorfilethatboththePowerCenterServerandClientmachinecanconnecttoThelookuptablecanbeasingletable,or
wecanjoinmultipletablesinthesamedatabaseusingalookupSQLoverrideinPropertiesTab.
2.Ports:
31/51
6/4/2015
Ports
Lookup
Type
Number
Needed
Description
Connected
Unconnected
Minimum1
InputporttoLookup.Usuallyportsusedfor
JoinconditionareInputports.
Connected
Unconnected
Minimum1
Portsgoingtoanothertransformationfrom
Lookup.
Connected
Unconnected
Minimum1
Lookupport.TheDesignerautomatically
Designateseachcolumninthelookup
sourceasalookup(L)andoutputport(O).
Unconnected
1Only
Returnport.Useonlyinunconnected
Lookupt/fonly.
3.PropertiesTab
Options
LookupType
Description
LookupSQL
Override
Relational
OverridesthedefaultSQLstatementtoquerythe
lookuptable.
LookupTable
Name
Relational
Specifiesthenameofthetablefromwhichthe
transformationlooksupandcachesvalues.
LookupCaching
Enabled
FlatFile,Relational IndicateswhetherthePowerCenterServer
cacheslookupvaluesduringthesession.
LookupPolicyon
MultipleMatch
FlatFile,Relational DetermineswhathappenswhentheLookup
transformationfindsmultiplerowsthatmatchthe
lookupcondition.Options:UseFirstValueorUse
LastValueorUseAnyValueorReportError
LookupCondition
FlatFile,Relational Displaysthelookupconditionyousetinthe
Conditiontab.
Connection
Information
Relational
Specifiesthedatabasecontainingthelookuptable.
32/51
6/4/2015
SourceType
FlatFile,Relational Lookupisfromadatabaseorflatfile.
LookupCache
DirectoryName
FlatFile,Relational Locationwherecacheisbuild.
LookupCache
Persistent
FlatFile,Relational WhethertousePersistentCacheornot.
DynamicLookup
Cache
FlatFile,Relational WhethertouseDynamicCacheornot.
RecacheFrom
LookupSource
FlatFile,Relational Torebuildcacheifcachesourcechangesandwe
areusingPersistentCache.
InsertElseUpdate Relational
Useonlywithdynamiccachingenabled.Appliesto
rowsenteringtheLookuptransformationwiththe
rowtypeofinsert.
LookupData
CacheSize
FlatFile,Relational DataCacheSize
LookupIndex
CacheSize
FlatFile,Relational IndexCacheSize
CacheFileName
Prefix
FlatFile,Relational Useonlywithpersistentlookupcache.Specifies
thefilenameprefixtousewithpersistentlookup
cachefiles.
SomeotherpropertiesforFlatFilesare:
DatetimeFormat
ThousandSeparator
DecimalSeparator
CaseSensitiveStringComparison
NullOrdering
SortedInput
33/51
6/4/2015
4:ConditionTab
WeentertheLookupCondition.ThePowerCenterServerusesthelookupconditiontotestincomingvalues.Wecomparetransformationinput
valueswithvaluesinthelookupsourceorcache,representedbylookupports.
Thedatatypesinaconditionmustmatch.
Whenweentermultipleconditions,thePowerCenterServerevaluateseachconditionasanAND,notanOR.
ThePowerCenterServermatchesnullvalues.
Theinputvaluemustmeetallconditionsforthelookuptoreturnavalue.
=,>,<,>=,<=,!=Operatorscanbeused.
Example:IN_DEPTNO=DEPTNO
In_DNAME='DELHI'
Tip:Ifweincludemorethanonelookupcondition,placetheconditionswithanequalsignfirsttooptimizelookupperformance.
Note:
1.Wecanuse=operatorincaseofDynamicCache.
2.ThePowerCenterServerfailsthesessionwhenitencountersmultiplekeysforaLookuptransformationconfiguredtouseadynamiccache.
3.ConnectedLookupTransformation
Example:TocreateaconnectedLookupTransformation
EMPwillbesourcetable.DEPTwillbeLOOKUPtable.
CreateatargettableCONN_Lookup_EXAMPLEintargetdesigner.TableshouldcontainallportsofEMPtableplusDNAMEand
LOCasshownbelow.
CreatingMapping:
34/51
6/4/2015
3.ClickMapping>Create>Givename.Ex:m_CONN_LOOKUP_EXAMPLE
5.ConnectallfieldsfromSQ_EMPtotargetexceptDNAMEandLOC.
6.Transformation>Create>SelectLOOKUPfromlist.Givenameandclick
Create.
7.TheFollowingscreenisdisplayed.
8.AsDEPTistheSourcedefinition,clickSourceandthenSelectDEPT.
9.ClickOk.
10.NowPassDEPTNOfromSQ_EMPtothisLookup.DEPTNOfromSQ_EMPwillbenamedasDEPTNO1.EditLookupandrenameitto
IN_DEPTNOinportstab.
11.NowgotoCONDITIONtabandaddCONDITION.
DEPTNO=IN_DEPTNOandClickApplyandthenOK.
Linkthemappingasshownbelow:
12.WearenotpassingIN_DEPTNOandDEPTNOtoanyothertransformationfromLOOKUPwecaneditthelookuptransformationand
removetheOUTPUTcheckfromthem.
35/51
6/4/2015
13.Mapping>Validate
14.Repository>Save
CreateSessionandWorkflowasdescribedearlier.Runtheworkflowandseethedataintargettable.
MakesuretogiveconnectionforLOOKUPTablealso.
WeuseConnectedLookupwhenweneedtoreturnmorethanonecolumnfromLookuptable.ThereisnouseofReturnPortinConnected
Lookup.
SEEPROPERTYTABFORADVANCEDSETTINGS
4.UnconnectedLookupTransformation
AnunconnectedLookuptransformationisseparatefromthepipelineinthemapping.Wewriteanexpressionusingthe:LKPreferencequalifier
tocallthelookupwithinanothertransformation.
StepstoconfigureUnconnectedLookup:
1. Addinputports.
2. Addthelookupcondition.
3. Designateareturnvalue.
4. Callthelookupfromanothertransformation.
36/51
6/4/2015
Example:TocreateaunconnectedLookupTransformation
EMPwillbesourcetable.DEPTwillbeLOOKUPtable.
CreateatargettableUNCONN_Lookup_EXAMPLEintargetdesigner.TableshouldcontainallportsofEMPtableplusDNAME
asshownbelow.
CreatingMapping:
3.ClickMapping>Create>Givename.Ex:m_UNCONN_LOOKUP_EXAMPLE
5.NowTransformation>Create>SelectEXPRESSIONfromlist.Givename
andclickCreate.ThenClickDone.
6.PassallportsfromSQ_EMPtoEXPRESSIONtransformation.
7.ConnectallfieldsfromEXPRESSIONtotargetexceptDNAME.
8.Transformation>Create>SelectLOOKUPfromlist.Givenameandclick
Create.
9.FollowthestepsasinConnectedabovetocreateLookuponDEPTtable.
10.ClickOk.
11.NowEdittheLookupTransformation.GotoPortstab.
12.AsDEPTNOiscommoninsourceandLookup,createaportIN_DEPTNO
portstab.MakeitInputportonlyandGiveDatatypesameasDEPTNO.
13.DesignateDNAMEasReturnPort.CheckonRtomakeit.
37/51
6/4/2015
14.NowaddaconditioninConditionTab.
DEPTNO=IN_DEPTNOandClickApplyandthenOK.
15.NowweneedtocallthisLookupfromExpressionTransformation.
16.EditExpressiont/fandcreateanewoutputportout_DNAMEofdatatypeasDNAME.OpentheExpressioneditorandcallLookupas
38/51
6/4/2015
givenbelow:
WedoubleclickUnconninbottomofFunctionstabandasweneedonly
DEPTNO,wepassonlyDEPTNOasinput.
17.ValidatethecallinExpressioneditorandClickOK.
18.Mapping>Validate
19.RepositorySave.
MakesuretogiveconnectionforLOOKUPTablealso.
5.LookupCaches
WecanconfigureaLookuptransformationtocachethelookuptable.TheIntegrationService(IS)buildsacacheinmemorywhenitprocesses
thefirstrowofdatainacachedLookuptransformation.
TheIntegrationServicealsocreatescachefilesbydefaultinthe$PMCacheDir.Ifthedatadoesnotfitinthememorycache,theISstoresthe
overflowvaluesinthecachefiles.Whensessioncompletes,ISreleasescachememoryanddeletesthecachefiles.
Ifweuseaflatfilelookup,theISalwayscachesthelookupsource.
WesettheCachetypeinLookupProperties.
LookupCacheFiles
1.LookupIndexCache:
Storesdataforthecolumnsusedinthelookupcondition.
2.LookupDataCache:
ForaconnectedLookuptransformation,storesdatafortheconnectedoutputports,notincludingportsusedinthelookupcondition.
ForanunconnectedLookuptransformation,storesdatafromthereturnport.
TypesofLookupCaches:
1.StaticCache
Bydefault,theIScreatesastaticcache.ItcachesthelookupfileortableandLooksupvaluesinthecacheforeachrowthatcomesintothe
transformation.TheISdoesnotupdatethecachewhileitprocessestheLookuptransformation.
2.DynamicCache
Tocacheatargettableorflatfilesourceandinsertnewrowsorupdateexistingrowsinthecache,useaLookuptransformationwitha
39/51
6/4/2015
dynamiccache.
TheISdynamicallyinsertsorupdatesdatainthelookupcacheandpassesdatatothetarget.Targettableisalsoourlookuptable.Nogoodfor
performanceiftableishuge.
3.PersistentCache
Ifthelookuptabledoesnotchangebetweensessions,wecanconfiguretheLookuptransformationtouseapersistentlookupcache.
TheISsavesandreusescachefilesfromsessiontosession,eliminatingthetimeRequiredtoreadthelookuptable.
4.RecachefromSource
Ifthepersistentcacheisnotsynchronizedwiththelookuptable,wecanConfiguretheLookuptransformationtorebuildthelookupcache.If
Lookuptablehaschanged,wecanusethistorebuildthelookupcache.
5.SharedCache
Unnamedcache:WhenLookuptransformationsinamappinghavecompatiblecachingstructures,theISsharesthecacheby
default.Youcanonlysharestaticunnamedcaches.
Namedcache:Useapersistentnamedcachewhenwewanttoshareacachefileacrossmappingsorshareadynamicanda
staticcache.Thecachingstructuresmustmatchorbecompatiblewithanamedcache.Youcansharestaticanddynamicnamed
caches.
BuildingConnectedLookupCaches
Wecanconfigurethesessiontobuildcachessequentiallyorconcurrently.
Whenwebuildsequentialcaches,theIScreatescachesasthesourcerowsentertheLookuptransformation.
Whenweconfigurethesessiontobuildconcurrentcaches,theISdoesnotwaitforthefirstrowtoentertheLookuptransformation
beforeitcreatescaches.Instead,itbuildsmultiplecachesconcurrently.
1.BuildingLookupCachesSequentially:
40/51
6/4/2015
2.BuildingLookupCachesConcurrently:
Toconfigurethesessiontocreateconcurrentcaches
EditSession>InConfigObjectTab>AdditionalConcurrentPipelinesfor
LookupCacheCreation>Giveavaluehere(AutoByDefault)
Note:TheISbuildscachesforunconnectedLookupssequentiallyonly
====================================================================
ExpressionTRANSFORMATION:
Passiveandconnectedtransformation.
UsetheExpressiontransformationtocalculatevaluesinasinglerowbeforewewritetothetarget.Forexample,wemightneedtoadjust
employeesalaries,concatenatefirstandlastnames,orconvertstringstonumbers.
UsetheExpressiontransformationtoperformanynonaggregatecalculations.
41/51
6/4/2015
Example:Addition,Subtraction,Multiplication,Division,Concat,Uppercaseconversion,lowercaseconversionetc.
WecanalsousetheExpressiontransformationtotestconditionalstatementsbeforeweoutputtheresultstotargettablesorother
transformations.Example:IF,Then,Decode
Thereare3typesofportsinExpressionTransformation:
Input
Output
Variable:Usedtostoreanytemporarycalculation.
CalculatingValues:
TousetheExpressiontransformationtocalculatevaluesforasinglerow,wemustincludethefollowingports:
Inputorinput/outputportsforeachvalueusedinthecalculation:Forexample:TocalculateTotalSalary,weneedsalaryand
commission.
Outputportfortheexpression:Weenteroneexpressionforeachoutputport.Thereturnvaluefortheoutputportneedsto
matchthereturnvalueoftheexpression.
WecanentermultipleexpressionsinasingleExpressiontransformation.Wecancreateanynumberofoutputportsinthetransformation.
Example:CalculatingTotalSalaryofanEmployee
ImportthesourcetableEMPinSharedfolder.Ifitisalreadythere,thendontimport.
Insharedfolder,createthetargettableEmp_Total_SAL.KeepallportsasinEMPtableexceptSalandCommintargettable.Add
Total_SALporttostorethecalculation.
Createthenecessaryshortcutsinthefolder.
42/51
6/4/2015
CreatingMapping:
3. ClickMapping>Create>Givemappingname.Ex:m_totalsal
4. DragEMPfromsourceinmapping.
5. ClickTransformation>Create>SelectExpressionfromlist.GivenameandclickCreate.Nowclickdone.
6. LinkportsfromSQ_EMPtoExpressionTransformation.
43/51
6/4/2015
7. EditExpressionTransformation.AswedonotwantSalandCommintarget,removecheckfromoutputportforbothcolumns.
8. Nowcreateanewportout_Total_SAL.Makeitasoutputportonly.
9. ClickthesmallbuttonthatappearsintheExpressionsectionofthedialogboxandentertheexpressionintheExpressionEditor.
10. EnterexpressionSAL+COMM.YoucanselectSALandCOMMfromPortstabinexpressioneditor.
11. ChecktheexpressionsyntaxbyclickingValidate.
12. ClickOK>ClickApply>ClickOk.
13. NowconnecttheportsfromExpressiontotargettable.
14. ClickMapping>Validate
15. Repository>Save
44/51
6/4/2015
AsCOMMisnull,Total_SALwillbenullinmostcases.Nowopenyourmappingandexpressiontransformation.SelectCOMMport,InDefault
Valuegive0.Nowapplychanges.ValidateMappingandSave.
Refreshthesessionandvalidateworkflowagain.Runtheworkflowandseetheresultagain.
NowuseERRORinDefaultvalueofCOMMtoskiprowswhereCOMMisnull.
Syntax:ERROR(Anymessagehere)
Similarly,wecanuseABORTfunctiontoabortthesessionifCOMMisnull.
Syntax:ABORT(Anymessagehere)
Makesuretodoubleclickthesessionafterdoinganychangesinmapping.Itwillpromptthatmappinghaschanged.ClickOKtorefreshthe
mapping.Runworkflowaftervalidatingandsavingtheworkflow.
Performancetuning:
45/51
6/4/2015
ExpressiontransformationisusedtoperformsimplecalculationsandalsotodoSourcelookups.
1. Useoperatorsinsteadoffunctions.
2. Minimizetheusageofstringfunctions.
3. Ifweuseacomplexexpressionmultipletimesintheexpressiontransformer,thenMakethatexpressionasavariable.Thenweneedto
useonlythisvariableforallcomputations.
===================================================================
RouterTransformation:
Activeandconnectedtransformation.
ARoutertransformationissimilartoaFiltertransformationbecausebothtransformationsallowyoutouseaconditiontotestdata.AFilter
transformationtestsdataforoneconditionanddropstherowsofdatathatdonotmeettheCondition.However,aRoutertransformationtests
dataforoneormoreconditionsAndgivesyoutheoptiontorouterowsofdatathatdonotmeetanyoftheconditionstoadefaultoutputgroup.
Example:IfwewanttokeepemployeesofFrance,India,USin3differenttables,thenwecanuse3Filtertransformationsor1
Routertransformation.
MappingAusesthreeFiltertransformationswhileMappingBproducesthesameresultwithoneRoutertransformation.
ARoutertransformationconsistsofinputandoutputgroups,inputandoutputports,groupfilterconditions,andpropertiesthatweconfigurein
theDesigner.
46/51
6/4/2015
47/51
6/4/2015
WorkingwithGroups
ARoutertransformationhasthefollowingtypesofgroups:
Input:TheGroupthatgetstheinputports.
Output:UserDefinedGroupsandDefaultGroup.WecannotmodifyordeleteOutputportsortheirproperties.
UserDefinedGroups:Wecreateauserdefinedgrouptotestaconditionbasedonincomingdata.Auserdefinedgroupconsistsofoutput
portsandagroupfilterCondition.WecancreateandedituserdefinedgroupsontheGroupstabwiththeDesigner.Createoneuserdefined
groupforeachconditionthatwewanttospecify.
TheDefaultGroup:TheDesignercreatesthedefaultgroupafterwecreateonenewuserdefinedgroup.TheDesignerdoesnotallowusto
editordeletethedefaultgroup.Thisgroupdoesnothaveagroupfilterconditionassociatedwithit.IfalloftheconditionsevaluatetoFALSE,
theISpassestherowtothedefaultgroup.
Example:FilteringemployeesofDepartment10toEMP_10,Department20toEMP_20andresttoEMP_REST
SourceisEMPTable.
Create3targettablesEMP_10,EMP_20andEMP_RESTinsharedfolder.StructureshouldbesameasEMPtable.
CreatingMapping:
3.ClickMapping>Create>Givemappingname.Ex:m_router_example
4.DragEMPfromsourceinmapping.
5.ClickTransformation>Create>SelectRouterfromlist.Givenameand
ClickCreate.Nowclickdone.
6.PassportsfromSQ_EMPtoRouterTransformation.
7.EditRouterTransformation.GotoGroupsTab
8.ClicktheGroupstab,andthenclicktheAddbuttontocreateauserdefinedGroup.Thedefaultgroupiscreatedautomatically..
9.ClicktheGroupFilterConditionfieldtoopentheExpressionEditor.
10.Enteragroupfiltercondition.Ex:DEPTNO=10
11.ClickValidatetocheckthesyntaxoftheconditionsyouentered.
48/51
6/4/2015
12.CreateanothergroupforEMP_20.Condition:DEPTNO=20
13.TherestoftherecordsnotmatchingtheabovetwoconditionswillbepassedtoDEFAULTgroup.Seesamplemapping
14.ClickOK>ClickApply>ClickOk.
15.Nowconnecttheportsfromroutertotargettables.
16.ClickMapping>Validate
17.Repository>Save
Makesuretogiveconnectioninformationforall3targettables.
49/51
6/4/2015
SampleMapping:
DifferencebetweenRouterandFilter:
Wecannotpassrejecteddataforwardinfilterbutwecanpassitinrouter.RejecteddataisinDefaultGroupofrouter.
========================================================
PostedbyKrishnareddyat08:48
Recommend this on Google
Labels:INFORMATICA
50/51
6/4/2015
51/51

Transform SQL queries and scripts in Informatica

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Transform SQL queries and scripts in Informatica

Transféré par

Droits d'auteur :

Formats disponibles

6/4/2015

Recommend this on Google

Vous aimerez peut-être aussi