Using dynamic lookup cache

August 21, 2009 Second Edition

Using dynamic lookup cache
by Matthias Urech

About
interface - development.com is the source for data integration expertise that provides customers valuable, convenient
and relevant experiences through a diversity of products and services. The main orientation of the project is to
respond to the increased demand for applications and services to help successfully building data integration solutions
and deliver the results developer and organization expects.

Matthias Urech, founder of interface - development.com, has a proven track record of applying data integration
solutions for several companies across the industry. He played a key role to implement projects in the area of data
integration, data migration, data consolidation and data warehousing. Matthias contributed several articles about data
integration best practices for Informatica Technology Network, resulting in nomination as member of Developer Wall of
Fame and receiving an award as Contributor of the Year in 2008.

Copyright
Copyright © 2005 - 2009 interface - development.com. All Rights Reserved.

No parts of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any
means – electronic, mechanical, photocopying, or otherwise – without the prior written permission of the author.

Limit of Liability / Disclaimer of Warranty

While every precaution has been taken in the preparation of this article, the publisher and author assume no
responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

Trademarks
Designations used by companies to distinguish their products are often claimed as trademarks or registered
trademarks. In all instances in which interface-development.com, the author or publisher is aware of claim, the
product names appear in initial capital or all capitall letters. Readers, however, should contact the appropriate
companies for more complete information regarding trademarks and registration.

Article History
April 8, 2005 First Edition.
July 20, 2010 Second Edition.

Table of Contents
Introduction ................................................................................................................................................ 1
Analyzing the problem................................................................................................................................. 1
Walkthrough: Using dynamic lookup cache ................................................................................................ 2
Step 1: Creating Source and Source Qualifier 2
Step 2: Creating Target 3
Step 3: Filter inactive records 3
Step 4: Sorting records for latest change 4
Step 5: Lookup target data 5
Step 6: Flagging data for insert or update 6
Step 7: Creating Workflow and Session 6

Copyright © 2005 - 2009 interface - development.com. All Rights Reserved. ii

Using dynamic lookup cache

Introduction
This article will dive into the deep blue of using a dynamic lookup cache that inserts and updates rows in the cache as
it passes rows to the target.

The example used in this article is dealing with the issue that a status is stored multiple times in the source system
without flagging the actual record. We will analyze the problem and go step by step in building a solution by using
dynamic lookup cache and storing only the actual status in the target.

Analyzing the problem

Figure 1 shows example table OBJECT_STATUS. We want to get the actual STATUS_ID for each OBJECT_ID. By taking
a closer look at the table we already recognize a pattern:
• Each status has a date stamp
• Equal date stamps contain a sequential number (attribute CHANGE_NO)
• An inactive status is marked

Figure 1: source table OBJECT_STATUS

Given the fact that we know the pattern, we are able to roughly draw the solution:
• Sort data by attribute OBJECT_ID, CHANGE_DATE, CHANGE_NO
• Use dynamic lookup cache to determine if current records exists in target
• Flag record to force insert or update in target
• Once the solution has been built and ran, the yellow marked records in Figure 1 are reflecting the result (=
unique value for each OBJECT_ID)

Copyright © 2005 - 2009 interface - development.com. All Rights Reserved. 1

Using dynamic lookup cache

Walkthrough: Using dynamic lookup cache

In this section you will accomplish the following activities:
1. Creating Source and Source Qualifier
2. Creating Target
3. Filter inactive records
4. Sorting records for latest change
5. Lookup target data
6. Flagging data for insert or update
7. Creating Workflow and Session

Figure 2 illustrates the result of our activities.

Figure 2: mapping m_USING_DYNAMIC_LOOKUP_CACHE

Step 1: Creating Source and Source Qualifier

To create Source and Source Qualifier:
1. Create a new mapping m_USING_DYNAMIC_LOOKUP_CACHE
2. Create a Source by defining the table definition. The source table has the following data definitions:

Figure 3: data definition of Source

3. Add Source and Source Qualifier to your mapping and connect the ports

Copyright © 2005 - 2009 interface - development.com. All Rights Reserved. 2

Using dynamic lookup cache

Step 2: Creating Target

To create a target:
1. Create a relational Target to stage the output. The Target has the following data definitions:

Figure 4: data definition of Target

2. Create physical target in your preferred database by running the create table statement
3. Add the Target to your mapping

Step 3: Filter inactive records

To filter inactive records:
1. Add a Filter transformation to your mapping
2. Select all ports of the Source Qualifier and drop them into the Filter transformation
3. Edit the transformation and specify the values in tab Properties as follows:

Figure 5: settings of Filter transformation

Copyright © 2005 - 2009 interface - development.com. All Rights Reserved. 3

Using dynamic lookup cache

Step 4: Sorting records for latest change

To sort records for latest change:
1. Add as Sorter transformation to your mapping
2. Select all ports of the Filter and drop them into the Sorter transformation
3. Edit the transformation, select checkbox Key and specify Ascending as sort order for the following ports:
OBJECT_ID, CHANGE_DATE, CHANGE_NO
4. Specify other settings according to figure 6:

Figure 6: settings of Sorter transformation

Copyright © 2005 - 2009 interface - development.com. All Rights Reserved. 4

Using dynamic lookup cache

Step 5: Lookup target data

To lookup target data:
1. Add a Lookup Transformation to your mapping. Select target OBJECT_STATUS in the popup window and click
[OK] button
2. Select all ports from the Sorter and drop it to the Lookup transformation
3. Edit the transformation and select the Condition tab. Add a new condition and specify OBJECT_ID as port to be
matched
4. Select the Properties tab and select checkbox Dynamic Lookup Cache and Insert Else Update
5. Associate lookup ports with input ports
6. Enter other settings according to figure 7:

Figure 7: settings of Lookup transformation

Copyright © 2005 - 2009 interface - development.com. All Rights Reserved. 5

Using dynamic lookup cache

Step 6: Flagging data for insert or update

To flag data for insert or update:
1. Add an Update Strategy transformation to your mapping
2. Select port NewLookupRow of the Lookup and drop it to the Update Strategy transformation
3. Select all ports of the Sorter and drop them to the Update Strategy transformation
4. Edit the Update Strategy transformation and select tab Properties
5. Enter the following value in the Update Strategy Expression:

IIF(NewLookupRow = 1, DD_INSERT, DD_UPDATE)

6. Note: The Informatica Server assigns a value to the port, depending on the action it performs to the lookup
cache:
NewLookupRow Value Description
0 The Informatica Server inserts the row into the cache.
1 The Informatica Server inserts the row into the cache.
2 The Informatica Server updates the row in the cache.
Table 1: NewLookupRow values
7. Select all ports of the Update Strategy transformation and connect them to the Target
8. Save your mapping m_USING_DYNAMIC_LOOKUP_CACHE

Step 7: Creating Workflow and Session

To create Workflow and Session:
1. Create a workflow wf_USING_DYNAMIC_LOOKUP_CACHE
2. Create a session for mapping m_USING_DYNAMIC_LOOKUP_CACHE
3. Select tab Sources and specify the source directory and file name (object_status.dat)
4. Select tab Targets, specify the database to connect and select Insert, Update Else Insert and Truncate
action in the Property section
5. Perform all other required settings
6. Save the workflow wf_USING_DYNAMIC_LOOKUP_CACHE

Figure 8: workflow wf_USING_DYNAMIC_LOOKUP_CACHE

7. Run the workflow and see the output:

Figure 9: output of session workflow wf_USING_DYNAMIC_LOOKUP_CACHE

Copyright © 2005 - 2009 interface - development.com. All Rights Reserved. 6