Vous êtes sur la page 1sur 143

IBM DB2 Information Integrator 

Operations Guide for Classic Event


Publishing
Version 8.2

SC18-9157-02
IBM DB2 Information Integrator 

Operations Guide for Classic Event


Publishing
Version 8.2

SC18-9157-02
Before using this information and the product it supports, be sure to read the general information under “Notices” on page 127.

This document contains proprietary information of IBM. It is provided under a license agreement and copyright law
protects it. The information contained in this publication does not include any product warranties, and any
statements provided in this manual should not be interpreted as such.
You can order IBM publications online or through your local IBM representative:
v To order publications online, go to the IBM Publications Center at www.ibm.com/shop/publications/order
v To find your local IBM representative, go to the IBM Directory of Worldwide Contacts at
www.ibm.com/planetwide
When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any
way it believes appropriate without incurring any obligation to you.
© Copyright International Business Machines Corporation 2003, 2004. All rights reserved.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
© CrossAccess Corporation 1993, 2003
Contents
Chapter 1. Overview of mapping data . . 1 | Recovery procedures . . . . . . . . . . . 28
Record selection exit . . . . . . . . . . . . 1 | Retaining changes . . . . . . . . . . . 28
Running the metadata utility . . . . . . . . . 3 | Starting a recovery change-capture agent . . . 29
Encrypting passwords for CICS VSAM . . . . . 4 | Reverting to active mode during development . . 32
| Local mode versus Central Version mode . . . . 33
Chapter 2. Operating correlation
services and publication services . . . 5 Chapter 4. IMS operations . . . . . . 35
IMS-specific configuration . . . . . . . . . . 5 Overview of DB2 Information Integrator Classic
Configuring the correlation service and publication Event Publisher for IMS . . . . . . . . . . 35
service . . . . . . . . . . . . . . . . 5 DB2 Information Integrator Classic Event
Configuring the maximum size of messages . . . . 8 Publisher implementation plan . . . . . . . 36
Configuring Cross Memory services . . . . . . 9 Supported environments and program types . . 37
Configuring the stores for messages and pending Change-capture agents . . . . . . . . . . 37
commits . . . . . . . . . . . . . . . 10 Active change-capture agents . . . . . . . 37
Creating publications . . . . . . . . . . . 10 Recovery change-capture agents . . . . . . 43
Creating the Classic Event Publisher recovery data Given a restart point, how can I identify the IMS log
sets . . . . . . . . . . . . . . . . . 12 files required? . . . . . . . . . . . . . 47
Starting the process of publishing . . . . . . . 12 What is log file tracking? . . . . . . . . . 47
Activating change capture for an IMS database or What is stored in an IMS log file tracking data
segment . . . . . . . . . . . . . . . 13 set? . . . . . . . . . . . . . . . . 48
Activating change capture for VSAM . . . . . . 13 How do I implement IMS log file tracking? . . . 49
Monitoring correlation services and publication How do I identify agents that are in recovery? . 50
services . . . . . . . . . . . . . . . . 13 How do I tell the correlation service about
Using the CSA reporting and maintenance utility . . 14 unknown agents that are in recovery mode? . . 51
Shutting down a correlation service . . . . . . 15 How to determine whether an IMS log file is
Recycling a correlation service . . . . . . . . 15 required in the recovery process when not using
Specifying monitored tables . . . . . . . . . 16 IMS log file tracking . . . . . . . . . . 51
What is the IMS log file recovery process? . . . 53
How do I get an agent out of recovery mode? . . . 61
| Chapter 3. CA-IDMS operations . . . . 17 When is it safe to place an agent back in active
| Change capture . . . . . . . . . . . . . 17 mode? . . . . . . . . . . . . . . . 61
| Recovery mode . . . . . . . . . . . . . 18 How do I manually create a recovery data set? 62
| The CA-IDMS recovery agent . . . . . . . 19 When is recovery not possible? . . . . . . . 65
| Determining prerequisites . . . . . . . . . 19 When is recovery not required? . . . . . . . 66
| Mapping CA-IDMS data for change capture . . . 20 What are IMS log records of interest? . . . . . . 66
| Mapping CA-IDMS paths for change capture . . 21 What are cascade deletes? . . . . . . . . . 67
| Running the metadata utility with CA-IDMS What is an XM queue overrun? . . . . . . . . 69
| metadata grammar . . . . . . . . . . . 22 How does the interrupt value work? . . . . . 70
| Filtering change capture by database when How does throttling work? . . . . . . . . 71
| accessing multiple databases in a CA-IDMS How do I install the IMS active change-capture
| Central Version . . . . . . . . . . . . 23 agent? . . . . . . . . . . . . . . . . 72
| Accessing multiple CA-IDMS Central Versions What if I already have an IMS logger exit
| from a single data server . . . . . . . . . 23 implemented? . . . . . . . . . . . . 72
| Configuring and running a correlation service for How do I activate change capture for an IMS
| CA-IDMS . . . . . . . . . . . . . . . 24 database or segment? . . . . . . . . . . 73
| Directly accessing CA-IDMS data . . . . . . 24 How do I augment a DBD for change capture? 75
| APF authorization of CA-IDMS.LOADLIB . . . 25 What are the recommended nonrelational-to-
| Setting up a data server to access a CA-IDMS relational mappings to use? . . . . . . . . 77
| Central Version . . . . . . . . . . . . 25 Mapping validation rules . . . . . . . . . . 78
| Running a CA-IDMS active change-capture agent . 26 Mapping considerations and information
| Configuring and handling error messages for contained within IMS data capture log records . 79
| missing correlation services . . . . . . . . 26 How do I deal with IMS test and production
| Setting up the IDMSJNL2 exit . . . . . . . 27 systems on the same MVS image? . . . . . . . 79
| Before starting a change-capture agent . . . . 28 I changed some IMS data and nothing happened . . 80
| Starting an active change-capture agent . . . . 28
| Stopping a change-capture agent . . . . . . 28

© Copyright IBM Corp. 2003, 2004 iii


What is the effect of DB2 Information Integrator Configuration parameter descriptions . . . . . 110
Classic Event Publisher on my IMS system? . . . 80 LD TEMP SPACE . . . . . . . . . . . 110
Are my IMS operating procedures affected by MESSAGE POOL SIZE . . . . . . . . . 111
DB2 Information Integrator Classic Event NL . . . . . . . . . . . . . . . . 111
Publisher? . . . . . . . . . . . . . . 81 NL CAT . . . . . . . . . . . . . . 111
Do I need to create DB2 Information Integrator INTERLEAVE INTERVAL . . . . . . . . 112
Classic Event Publisher operating procedures? . . 81 PUB . . . . . . . . . . . . . . . 112
What DB2 Information Integrator Classic Event SAF EXIT . . . . . . . . . . . . . . 113
Publisher monitoring capabilities are available for SERVICE INFO ENTRY . . . . . . . . . 113
IMS? . . . . . . . . . . . . . . . . 83 SMF EXIT . . . . . . . . . . . . . 114
STATIC CATALOGS . . . . . . . . . . 114
Chapter 5. VSAM operations . . . . . 85 TASK PARAMETERS . . . . . . . . . . 114
Introduction . . . . . . . . . . . . . . 85 VSAM AMPARMS . . . . . . . . . . . 115
Determining prerequisites . . . . . . . . . 85
Mapping application data . . . . . . . . . 85 DB2 Information Integrator
Change capture . . . . . . . . . . . . . 85 documentation . . . . . . . . . . . 117
Recovery with VSAM . . . . . . . . . . 86 Accessing DB2 Information Integrator
Moving from recovery to active mode with VSAM 86 documentation . . . . . . . . . . . . . 117
Starting a CICS VSAM change-capture agent . . . 86 Documentation about replication function on z/OS 119
Configuring CICS file definitions . . . . . . 86 Documentation about event publishing function for
Before starting a change-capture agent . . . . 86 DB2 Universal Database on z/OS . . . . . . 120
Starting a change-capture agent. . . . . . . 87 Documentation about event publishing function for
Starting a recovery agent . . . . . . . . . 87 IMS and VSAM on z/OS . . . . . . . . . 120
Stopping a change-capture agent . . . . . . 87 Documentation about event publishing and
Configuring and starting a CICS VSAM replication function on Linux, UNIX, and Windows 121
change-capture agent . . . . . . . . . . . 87 Documentation about federated function on z/OS 122
Service info entry . . . . . . . . . . . 87 Documentation about federated function on Linux,
Mapping CICS VSAM data . . . . . . . . . 89 UNIX, and Windows . . . . . . . . . . . 122
Running the metadata utility . . . . . . . 90 Documentation about enterprise search function on
Linux, UNIX, and Windows . . . . . . . . 124
Appendix A. The IMS recovery process 93 Release notes and installation requirements . . . 124
Identifying agents in recovery mode . . . . . . 97
Putting unknown agents in recovery mode . . . 101 Notices . . . . . . . . . . . . . . 127
Using DBRC to identify log files needed for Trademarks . . . . . . . . . . . . . . 129
recovery . . . . . . . . . . . . . . . 103
Recovering log files . . . . . . . . . . . 104 Index . . . . . . . . . . . . . . . 131
Returning agents back to active mode . . . . . 107

Contacting IBM . . . . . . . . . . 133


Appendix B. Configuration parameters 109
Product information . . . . . . . . . . . 133
Configuration parameter format . . . . . . . 109
Comments on the documentation. . . . . . . 133
Configuration parameters . . . . . . . . . 110

iv DB2 II Operations Guide for Classic Event Publishing


Chapter 1. Overview of mapping data
Part of the process of preparing for change capture is mapping data so that DB2®
Information Integrator Classic Event Publisher knows how to handle the data.

After you understand the issues involved in mapping database data structures,
you can map your databases using the Data Mapper.

The following sections cover information about mapping data for specific
databases. For more general information about using the Data Mapper, see the DB2
Information Integrator Data Mapper Guide for Classic Federation and Classic Event
Publisher.

For instructions on how to map data for a specific database, see the section that
corresponds to the type of database tables you want to map:
v “What are the recommended nonrelational-to-relational mappings to use?” on
page 77
v “Mapping CICS VSAM data” on page 89

Record selection exit


The correlation service supports an optional record selection exit that can be used
to match record change data to specific table mappings in the metadata catalog.
The purpose of this exit is to accept or reject change records as belonging to
specific table mappings in the catalog. This exit is specifically designed to handle
records that have multiple record layouts, and that therefore require a separate
table mapping for each record layout. Multiple layouts are generally defined in
COBOL with a REDEFINES clause.

This exit is implemented by assembling and linking the module CACRCSEL into
the DB2 Information Integrator Classic Event Publisher load library. Sample source
for the exit is in the CACRCSEL member in the DB2 Information Integrator Classic
Event Publisher SCACSAMP library. This assembler module contains a description
of the parameters that are passed and logic to accept or reject record data that is
passed as matching a mapped table name.

The parameters that are passed to the module CACRCSEL are:


v An uninitialized 256-byte work area. This can be used as a save area so the
module can call subroutines. Do not use this area for persistent storage across
invocations of the module.
v The address of the record data that is to be matched to a table name. If the data
was compressed in the database, it is decompressed prior to calling CACRCSEL.
Do not change the record data with CACRCSEL because it contains only the
before image for record updates and therefore any necessary changes will not be
applied to the after image.
v A fullword value that contains the binary length of the record data.
v An 8-byte, space-padded database type name of the mapped table. This
parameter contains one of the following values:
– ’$IMS ’--IMS database mapping.
– ’$VSAM ’--VSAM file system mapping.

© Copyright IBM Corp. 2003, 2004 1


v The 8-byte, space-padded table owner ID as specified in the metadata grammar
that mapped the table.
v The 18-byte, space-padded table name as specified in the metadata grammar
that mapped the table.
v A fullword binary field that contains the change type associated with the passed
data. The values in this field are:
– 1--The change is an update and the passed data is the before image of the
update.
– 2--The change is an insert and the passed data is the data inserted.
– 3--The change is a delete and the passed data is the record deleted.
v An optional fullword DBMS-specific parameter.
– VSAM--Not used.

The following IMS™ information is available:


v IMSSSID DS CL8 IMS SUBSYSTEM ID
v IMSJOB DS CL8 CHANGE-CAPTURE AGENT JOB NAME
v IMSPSB DS CL8 PSB NAME
v IMSTRAN DS CL8 TRANSACTION NAME
v IMSUSER DS CL8 USER ID
v IMSDBD DS CL8 DBD NAME
v IMSSEGM DS CL8 LEAF SEGMENT NAME
v IMSLOGN DS CL16 LOG RECORD SEQUENCE NUMBER (in hexadecimal
format)
v IMSSTCK DS CL16 SYSTEM CLOCK VALUE (in hexadecimal format)

If the record selection exit module exists in the DB2 Information Integrator Classic
Event Publisher loadlib, it is called each time the correlation service receives a
change from a change-capture agent. This call is made once for each table name
that matches the database or file record changed. The record selection exit must
determine whether the change passed is valid for the table name from the
metadata catalog. Accepting or rejecting a table name is indicated by the return
code issued by the selection exit. A return code of 0 indicates acceptance of the
record data for the table name passed and results in the conversion of the record
data to an SQLDA to forward to the publication service.

In cases such as IMS, where the table mapping can include parent segments, the
record data parameter contains information for all records in the defined path.
When determining offsets to specific fields in the data record, you must account
for the concatenated length of all parent segments in the beginning of the data
area. To determine offsets to specific data items, you can add debugging logic in
the sample record exit to dump passed records to the operator console.

In some instances, you might want to filter certain table names based on the action
value passed. For example, an application might be interested in capturing changes
to a database only when certain records are inserted. In this case, the selection exit
can be used to reject all UPDATE and DELETE messages for a particular table
name.

You can modify the following sample JCL and use it to assemble and link the
record selection exit:

2 DB2 II Operations Guide for Classic Event Publishing


//jobname JOB (ACCTINFO),’CACRCSEL’,CLASS=A,
// MSGCLASS=X,NOTIFY=&SYSUID
//* ASSEMBLE CACRCSEL
//ASSEMBLE EXEC PGM=ASMA90,
// PARM=’LIST,NODECK,RENT’
//SYSLIB DD DSN=SYS1.MACLIB,DISP=SHR
//SYSLIN DD DISP=(NEW,PASS),DSN=&&OBJMOD,
// UNIT=SYSDA,SPACE=(TRK,(10,1)),
// DCB=(LRECL=80,BLKSIZE=3200,RECFM=FB)
//SYSUT1 DD DSN=&&SYSUT1,UNIT=VIO,SPACE=(1700,(2000),,,ROUND)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DISP=SHR,DSN=CAC.SCACSAMP(CACRCSEL)
//* LINK CACRCSEL
//LINK EXEC PGM=IEWL,PARM=’LIST,RENT,REUS’,COND=(4,LT)
//SYSLIN DD DISP=(OLD,DELETE),DSN=&&OBJMOD
// DD *
ENTRY CACRCSEL
NAME CACRCSEL (R)
//SYSLMOD DD DISP=SHR,DSN=CAC.SCACLOAD(CACRCSEL)
//SYSUT1 DD UNIT=SYSDA,SPACE=(1024,(120,120),,,ROUND),
// DCB=BUFNO=1
//SYSPRINT DD SYSOUT=*

For more information about modifying the assembler source member CACRCSEL,
see the sample exit CACRCSEL, in the DB2 Information Integrator Classic Event
Publisher SCACSAMP library.

Running the metadata utility


The metadata utility takes metadata grammar as input, and updates the metadata
catalog based on the information provided in the metadata grammar. It is assumed
that you already generated metadata grammar from the Data Mapper, and that
you performed any post-processing of the metadata grammar (added ALTER
statements) prior to running the metadata utility.
To run the metadata utility to complete the mapping process:
1. Edit the metadata utility JCL.
Sample metadata utility JCL is found in SCACSAMP member CACMETAU,
which is shipped with your DB2 Information Integrator Classic Event Publisher
software.
2. Complete basic customization.
Supply a valid job card and modify the CAC high-level qualifier to reference
the high-level qualifier for the DB2 Information Integrator Classic Event
Publisher data sets. At this point, you might want to save the member because
the DB2 Information Integrator Classic Event Publisher high-level qualifier is
not likely to change after each execution of the metadata utility.
3. Complete database-specific customization.
You might need to make database-specific metadata utility JCL modifications.
Database system JCL modifications might be required for other database
systems. Specific changes are outlined below by database system.
a. For CICS® VSAM, add a DD statement named CICSUID to the metadata
utility JCL if the ATTACHSEC setting for the connection is set to anything
other than LOCAL. The data set specifies an 80-character sequential file or
member of a PDS that contains two parameters: USER and PASSWORD.
USER is an 8-character string that provides a user name that is used to
connect to the data source. PASSWORD is an encrypted or unencrypted
password that corresponds with the user name provided.

Chapter 1. Overview of mapping data 3


Note: If ATTACHSEC is set to anything other than local and CICSUID is
not defined, or if the file pointed to by CICSUID is inaccessible, the
system will generate an error when attempting to establish a
connection.
b. For IMS, uncomment the IMS substitutable variable and specify the
high-level qualifier of the IMS library that contains the DBD that has been
augmented for change-capture. Also, uncomment the DBDLIB DD
statement. Ensure this DD statement references the DBDLIB where the
augmented DBD is located.
4. Specify the name of the USE grammar input member.
5. Submit the metadata utility job.
6. Inspect the output. A return code of 4 is acceptable and expected the first time
that you add a table.

Encrypting passwords for CICS VSAM


You can encrypt the password used to connect to the metadata utility. The utility
used to generate the encrypted password runs on a Microsoft® Windows-based
machine.
To encrypt a password:
1. From an MS-DOS window, type the following command and press Enter:
cacencr /x
The /x command-line parameter indicates that you are generating an encrypted
password for CICS VSAM, which is encrypted using hexadecimal values.
2. At the Enter password ==> prompt, type in the password (up to 8 characters)
and press Enter.

Note: The password is not shown on the screen when you type, so enter your
password carefully.
The utility creates a file, password.txt, in the working directory, and then exits.
3. Open password.txt.
4. Cut and paste the password into the file pointed to by the CICSUID DD
statement.

4 DB2 II Operations Guide for Classic Event Publishing


Chapter 2. Operating correlation services and publication
services
This chapter covers:
v “IMS-specific configuration” on page 5
v “Configuring the correlation service and publication service” on page 5
v “Configuring the maximum size of messages” on page 8
v “Configuring Cross Memory services” on page 9
v “Configuring the stores for messages and pending commits” on page 10
v “Creating publications” on page 10
v “Creating the Classic Event Publisher recovery data sets” on page 12
v “Starting the process of publishing” on page 12
v “Activating change capture for an IMS database or segment” on page 13
v “Activating change capture for VSAM” on page 13
v “Monitoring correlation services and publication services” on page 13
v “Using the CSA reporting and maintenance utility” on page 14
v “Shutting down a correlation service” on page 15
v “Recycling a correlation service” on page 15
v “Specifying monitored tables” on page 16

IMS-specific configuration
The JCL for the correlation service that is correlating IMS changes must be
modified to include a DBDLIB DD statement. This DBD load library must
reference DBD definitions for each table that has been augmented for data capture.
The correlation service compares the DBD version information contained in the
IMS Data Capture log records against the DBD load module. If discrepancies are
found, an error is logged and correlation service processing is terminated. This
puts you into a recovery situation.

Configuring the correlation service and publication service


The configuration member CACCSCF is stored in the SCACCONF data set and
contains a Service Info Entry (SIE) that defines the various services. Within the SIE
are various fields that define the service, the number of tasks started at correlation
service start up, the minimum and maximum number of tasks allowed, timeout
values, and trace options. Service Info Entries are used in configuration files to
inform the Region Controller task that a service is to be activated and how that
service is to be controlled.

Multiple SIE parameters are required to activate multiple instances of a given


service if different sub-parameter values are needed. A single SIE parameter is
used if only a single instance is needed (or multiple instances using the same
sub-parameter values). A given service’s restrictions and allowable combinations of
multiple instances are discussed in that service’s specific descriptions.
Mutually-exclusive services are also noted in these descriptions.

© Copyright IBM Corp. 2003, 2004 5


The SIE parameter consists of ten sub-parameters, each delimited by at least one
space. The format of the first nine of these subfields is consistent across all
services. The format for the tenth subfield is service-dependent.

The following table shows sample SIEs for the correlation service and the
publication service.
Table 1. Sample service info entries for the correlation service and the publication service
Type of Sample SIEs
service
Correlation SERVICE INFO ENTRY = CACECA2 XM1/XQM/CSQ1 2 1 1 16 4 10MS 30S \
service TCP/111.111.111.111/SOCKET#,CSA=1K,CSARLSE=3,INT=1,WARMSTART
Publication SERVICE INFO ENTRY = CACPUB PUB1 2 1 1 1 4 5M 5M MQI/QM_P39D/QUEUE1
service

The order of SIEs for the correlation service and the publication service matters.
The entry for the correlation service must come before the entry for the publication
service. This order is particularly important on shutdown, when services are
stopped in LIFO (reverse) order. The publication service must stop first so that it
can send the proper quiesce command to the correlation service. If the publication
service does not stop first, the correlation service might go into recovery mode on
an otherwise normal shutdown. For this reason, if the publication service is
configured to start before its corresponding correlation service, the publication
service will fail on startup when it fails to detect that the correlation service exists.

The parameters for the SIEs are explained below:


Parameter One: Task Name
For correlation services, the token CACECA2 is the task name and the
name of the correlation service load module. Leave this value as is.
For publication services, the token CACPUB is the task name and the
name of the publication service load module. Leave this value as is.
Parameter Two: Service Name
For correlation services, the service name token defines the protocol and
queue name for receiving raw data changes from active and recovery
agents. In most cases, the protocol name should be XM1 for Cross Memory
services. The subtoken XQM/CSQ1 identifies the Cross Memory data space
and queue name and can be modified to suit any particular site standards
if applicable. The final subtoken 16 identifies the size (in megabytes) of the
change capture data space queue. This value can range from 1 to 2048 and
defaults to 8 if not specified. The size of the queue depends on the number
of expected active agents and the burst volume of changes from each
agent. A value of 16 is recommended to help ensure queue space during
peak periods.

Important: You must define a unique data space or queue name for each
correlation service that will be running at any one time.

For publication services, the service name can be a string 16 characters


long.
Parameter Three: Service Start Class

6 DB2 II Operations Guide for Classic Event Publishing


For both correlation services and publication services, leave this value set
to 2.
Parameter Four: Minimum Tasks
For both correlation services and publication services, leave this value set
to 1. A value of 0 is acceptable if you want to manually start the service
with an operator command, but changing the value is not recommended.
Parameter Five: Maximum Tasks
For both correlation services and publication services, leave this value set
to 1.
Parameter Six: Maximum Connections per Task
For correlation services, this value should be the maximum number of
active and recovery agents that the correlation service will service, plus
four. The additional connections are used by the publication service and
reporting utility.
For publication services, set this value to 200.
Parameter Seven: Trace Output Level
Leave this value set to 4 unless you are asked to change it by IBM®
technical support for problem diagnostics.
Parameter Eight: Response Timeout
This value determines the length of time that the service will listen for a
response to requests before timing out. The default value is 10
milliseconds. Keep this value low so that the transport queue is checked
frequently for messages arriving from change-capture agents.
Parameter Nine: Idle Timeout
This value sets a polling frequency for recovery restart and rules
confirmation messages. A value of 30 seconds is recommended.
Parameter Ten: Service Specific Information
For correlation services, the first token in the service-specific information
defines the queue for communication with recovery agents and the
publication service. Generally, this token defines a TCP/IP connection
string to which the publication service connects for receiving change
messages. The format of a TCP/IP connection string is:
TCP/ip address or hostname/port number

Examples:
TCP/192.123.456.11/5555
TCP/OS390/5555

For publication services, the tenth parameter specifies the WebSphere® MQ


message queue to use as the restart queue. The format of the parameter is
as follows:

mqi/queue manager/queue name

queue manager is the name of the local queue manager that manages the
message queue. queue name is the name of the local message queue to use
as the restart queue. If your correlation service is running remotely from
your publication service, you can follow the name with a comma-delimited

Chapter 2. Operating correlation services and publication services 7


communication string to describe how the publication service is to
communicate with the correlation service.

Additional service information for correlation services that you can specify:
CSA=nK
The number of kilobytes that each server is to allocate for CSA space. In
most cases, 1K should be enough to manage change capture on at least 50
tables.
CSARLSE=n
The number of seconds to wait before attempting to release CSA storage at
shutdown. A value of 0 leaves CSA allocated for reuse when the
correlation service is restarted. The default value of CSARLSE is 0, which
prevents CSA from being released. If CSARLSE is not 0, the server will
release CSA only if no other correlation services are still active in the
system.
INT=n The number of changes to process for a committed unit of recovery before
checking for incoming change data from active or recovery agents. This
parameter prevents large transactions from blocking the incoming raw data
queue while sending change messages to the publication service. A value
of 0 will not interrupt processing of committed messages to look for
incoming raw data messages. 1 is the default and recommended value.
NOSTAE
Disables abend detection in the correlation service. Do not specify this
value unless requested by IBM technical support for diagnostics.
NAME
Names the correlation service. If you leave this option out, the correlation
service will be started unnamed. See the DB2 Information Integrator Planning
Guide for Classic Event Publishing for more information about named
servers.
COLDSTART/WARMSTART
Specifies whether to cold start or warm start the server. A cold start
discards all recovery information and places all known agents in active
mode. A warm start retains the state of all known change-capture agents at
the time the correlation service was last shutdown.
WARMSTART is the default action.
If you set the SIE to perform a cold start, make sure that you reset the SIE
after you cold start the server so that you do not inadvertently cold start
the server in the future.

Configuring the maximum size of messages


The publication service draws memory buffers from the message pool (the size of
which is determined by the MESSAGE POOL SIZE parameter in the configuration
file) and constructs messages within these buffers. When the publication service is
sending messages of type TRANS, a large transaction can exceed the size of the
buffer in which it is being converted to messages. In such cases, it is useful to
allow the publication service to segment the transaction. The publication service
constructs two or more messages for the transaction and puts messages on the
queue in succession when each becomes a certain size.

8 DB2 II Operations Guide for Classic Event Publishing


Use the MAX TRANSPORT MESSAGE SIZE parameter in your configuration file
to specify in bytes the largest size that a message can be before the publication
service writes it to a message queue. For example, consider the following entry in
a configuration file:

MAX TRANSPORT MESSAGE SIZE = 262144

When the publication service constructs a message for a large transaction in the
message pool, whenever the publication service finds that the size of the message
reaches 256 KB, the publication service writes the current message to the
appropriate message queue and starts building another message to contain the
subsequent DML of the transaction. If this next message becomes 256 KB in size
before the end of the transaction is reached, the publication service writes this
message to the message queue and begins constructing another message. This
process continues until the publication service reaches the end of the transaction.

Segments are numbered sequentially by the publication service so that the


application that receives them can be sure that they are in sequence. The message
attribute isLast is set to 1 in the final message so that receiving applications can
tell when the transaction is finished.

The maximum value of MAX TRANSPORT MESSAGE SIZE is 10 percent of the


parameter MESSAGE POOL SIZE.

The minimum value is 64 KB, expressed as 65536.

The default value is 128 KB, expressed as 131072.

Configuring Cross Memory services


When configuring Cross Memory services, you must configure each correlation
service with a unique data space or queue name. You define Cross Memory
services with four tokens: protocol, data space, queue name, and data space queue
size.

If you use the same combination of data space and queue name for more than one
correlation service definition, then change-capture agents will send captured
changes to the least-busy server, which might be the correct server. Names are
intentionally shared in a DB2 Information Integrator Classic Federation enterprise
server environment for load balancing and because serialization is not an issue.
But, in an Classic Event Publisher environment, serialization is essential.

Unless you have a specific reason for sharing data spaces between multiple
correlation services, use a unique data space name for each server. If you choose to
share data spaces between servers by using a common data space name, make sure
that the queue name is unique for each server.

If you configure more than one correlation service in a single data space, then the
first correlation service that is started will set the size of the data space.

Chapter 2. Operating correlation services and publication services 9


Configuring the stores for messages and pending commits
The correlation service stores uncommitted raw changes in the message store while
the pending commit store contains information about the committed UORs that
have been received from the change-capture agents and that need to be sent to the
publication service for processing. These stores are implemented as B-tree data sets
and are configured using the LD TEMP SPACE parameter.

Both the message store and the pending commit store are temporary data sets. Any
information contained in these stores is automatically deleted during initialization
of the correlation service. For performance reasons, IBM recommends that
hiperspace is used for these stores as opposed to temporary DASD. Note, however,
that the supplied LD TEMP SPACE definition uses DASD.

If you use hiperspace, begin with a fairly small initial allocation (perhaps 16
megabytes) and allow the hiperspace to grow slowly up to the maximum
configured size. The maximum size is 2 gigabytes. Use a small initial allocation
because the LD TEMP SPACE parameter is used for both the message store and
the pending commit store. Ideally, there will be a single, or small number of, UORs
that are waiting to be sent to the publication service for processing in a normal
operational environment.

Determining the maximum size of the message store is more difficult and depends
on the number and size of the UORs that are being generated by your client
applications and captured by the change-capture agents. While a UOR is in-flight
(e.g., has not been committed or rolled back) the changes generated are stored in
the message store. Therefore, you must allocate enough space to hold all of these
changes. The message store must be configured to be large enough to hold all of
the changes for any committed UORs that are being sent, or a waiting to be sent,
to the publication service for committed UORs while new changes are arriving
from the change-capture agents.

Setting a moderate initial allocation size and moderate secondary allocation size
and specifying a large maximum size is recommended since it allows the system to
dynamically grow based on load. This approach is viable when your site has
sufficient auxiliary storage available to handle the peak load.

Creating publications
After you configure your correlation service and your publication service, you
must configure publications to indicate where changes to mapped tables will be
published and how. You do so in the same configuration file in which you
configured your correlation service and your publication service. (If your
correlation service and publication service are remote from each other and
therefore use two different configuration files, configure your publications in the
publication service’s configuration file.)

IMS example

The following example publication uses an IMS source:


PUB ALIAS=ims1,
MSGTYPE=TRANS,
TABLE=CAC.STOKSTAT,
TOPIC=Schema1/IMS_update,
QUEUE=MQI/CSQ1/one,
BEFORE_VALUES = YES

10 DB2 II Operations Guide for Classic Event Publishing


VSAM example

The following example publication uses a VSAM source:


PUB ALIAS=vsam1,
MSGTYPE=TRANS,
TABLE=CAC.EMPCICS,
TOPIC=Schema1/VSAM_update,
QUEUE=MQI/CSQ1/one,
BEFORE_VALUES = YES

Publications are composed of the following parts:


Alias parameter
An alias defines the unique name for a publication within a Data Server.
Topic parameter (optional)
Include a topic in your publication if you want to publish it to WebSphere
Business Integrator Event Broker. Topics tell the WBI Event Broker how to
route the messages for the publication.
Queue parameter
Queues are the WebSphere MQ queues where messages are put. The
format of this parameter is MQI/queue_manager/queue_name, where MQI is
the designator for WebSphere MQ, queue_manager is the name of the queue
manager that the publication service is working with, and queue_name is
the name of the queue on which messages for the publication are put.
Message output parameters
Message output parameters define how the messages are constructed. The
table below lists these parameters and describes them.
Table 2. Message output parameters
Message output parameter Default Possible values
value
MSGTYPE TRANS
TRANS
A message is published for each
committed transaction that affects the
source table. The message contains all
of the changes made to the source table
by the transaction.
ROWOP
A message is published for each
committed row operation on the source
table.
TABLE None The Table string is used to identify the mapped
table name from which changes will be
published. There can be only one table per
publication. This table is specified in the format :
ownerName.tableName (for example,
QAVSAM.EMPLOYEES). The example above
refers to a table, QAVSAM.EMPLOYEES, that
was mapped into the Classic Event Publisher
catalog and was altered for data capture
changes.

Chapter 2. Operating correlation services and publication services 11


Table 2. Message output parameters (continued)
Message output parameter Default Possible values
value
BEFORE_VALUES NO
NO When a row is updated, only the
current values for all columns are
included in the message.
YES When a row is updated, the previous
values and the current values for all
columns are included in the message.
This parameter is effective for UPDATE
operations only.
CHANGED_COLS_ONLY YES This parameter is not currently supported. Do
not change its value.
ALL_CHANGED_ROWS NO This parameter is not currently supported. Do
not change its value.

Creating the Classic Event Publisher recovery data sets


To create the Classic Event Publisher recovery data sets referenced in the
correlation service:
1. Run the SCACSAMP member, CACGDGA, to create GDG files and allocate the
first generation recovery data sets.
2. The CACGDGA member contains JCL to allocate the Classic Event Publisher
recovery data sets used by the correlation service.
a. Customize the JCL to run in your environment and submit it.
b. After this job completes, ensure that the correlation service procedure in the
PROCLIB points to the newly created data sets using the CACRCVD and
CACRCVX DD statements.
c. Ensure that the correlation service has the proper authority to allocate the
next generation of the recovery files.

Starting the process of publishing


Before starting to publish changes that are made to your sources, ensure that the
WebSphere MQ queue manager is running and then start the correlation service
and the publication service.

If the correlation service and publication service are configured in the same file,
you either issue a console command to start the correlation service JCL procedure,
or submit a batch job. The console command to start the correlation service is:

S procname

where procname is the 1-8 character proclib member name to be started. When you
issue commands from the SDSF product, prefix all operator commands with the
forward slash ( / ) character.

If the correlation service and publication service are configured in separate files,
you can issue a console command to start the correlation service JCL procedure
and another console command to start the publication service JCL procedure. The
console command is described above. You can also choose to submit a batch job for
each.

12 DB2 II Operations Guide for Classic Event Publishing


Activating change capture for an IMS database or segment
Change capture is implemented using two IMS features. The IMS active
change-capture agent is implemented as an IMS Logger Exit. This allows the IMS
system to be monitored for changes in near real-time.

Although IMS performs its recovery processing based on the normal contents of
the IMS log files, Classic Event Publisher does not use the “raw” log records that
IMS uses to capture changes. Classic Event Publisher does use the same log
records, in addition to some additional IMS synchpoint log records, to track the
state of an in-flight Unit of Recovery (UOR), but does not use the type 50 (undo or
redo) and other low-level change notification records that IMS uses for recovery
purposes.

Instead, Classic Event Publisher uses type 99 Data Capture log records to identify
changes to a monitored IMS database because these records contain more
information and are easier to deal with than the “raw” recovery records used by
IMS.

Data Capture log records are generated at the database or segment level and
require augmentation of your DBD definitions. This augmentation does not affect
the physical database definition; it adds additional information to the DBD and
ACB load module control blocks.

Augmentation consists of adding the EXIT= keyword parameter on the DBD or


individual SEGM statements in your DBD definition. You can supply default
capture values at the DBD level and override or even suppress data capture
altogether at the SEGM level.

After you augment your DBD, perform these steps:


v Run a DBDGEN for the updated DBD.
v Run the ACBGEN utility to update all PSBs that reference the DBD.

You can then put the updated DBD and PSB members into your production ACB
libraries. If you make these changes with the IMS Online Change facility, the
correlation service goes into recovery. You must recycle the correlation service and
put into active mode the IMS active change-capture agents that were monitoring
the online system where the Online Change was performed.

Activating change capture for VSAM


If you have a change-capture agent for VSAM defined in the same configuration
file as your correlation service, that agents starts when you start the correlation
service. After the server is initialized, you will see the following message on the
system log:
CACH105I CICS VSAM CAPTURE: Vv.r.m mmddyyyy READY

This is followed by a message that indicates the time that processing began:
CACH106I START PROCESSING AT mm/dd/yyyy hh:mm:ss

Monitoring correlation services and publication services


After you start your correlation service and publication service, you can use MTO
commands to display reports on their activities.

Chapter 2. Operating correlation services and publication services 13


Command for monitoring correlation services
cmd,name of correlation service,report
This command displays a report on the activity of the correlation service
and all change-capture agents that send it change-capture data.
The following shows an example of the results of this command.
CAC00200I CMD,XM1/IMSX/IMSX/200,REPORT
CACG150I CORRELATION SERVICE ACTIVITY REPORT
*************** Transactions ***************
Agent Processed Sent to Rules Confirmed Pending State
----------- ---------- ------------- --------- ------- -----
VSAVSAMECA 0000002 0000002 0000002 0000000 Active

CACG151I END OF REPORT, AGENT TOTAL=1


CACG152I PENDINGQ(0) MSGQ(0) UNCONFIRMED(0)
Command for monitoring publication services
Command to report activity: cmd,name of publication service,report
This command publishes a report of the number of change messages
received, the number of commit messages received, the number of commits
that are confirmed received from the correlation service, the number of
commit messages rejected by the publication service. Here is a sample
report:
CACJ001I DISTRIBUTION SERVER ACTIVITY REPORT
--------------------------------------------
Change Message(s) Received = 13
Commit Message(s) Received = 2
Commit Message(s) Confirmed = 2
Commit Message(s) Rejected = 0

Using the CSA reporting and maintenance utility


SCACSAMP member CACE2RPT is sample JCL that you can use to invoke the
CSA reporting and maintenance utility. This utility allows you to view and release
the contents of the CSA storage that is used for communications between
change-capture agents and the correlation service. The CSA reporting and
maintenance utility is controlled with command-line parameters. The following
parameters are supported:
REPORT | NOREPORT
Identifies whether you want the CSA reporting and maintenance utility to
generate an Event Publisher Correlation Service Report describing the CSA
usage and state for either an unnamed server, or the correlation service
identified by the SERVER command line parameter.
The following example output was produced for an active unnamed
server:
+---------- Event Publisher Correlation Service Report --------------
|
| Correlation Service CSA E2CSGBLA, Lth=2588, Lock=x0000
| Service Maximum Count=1
| Allocated Service Blocks=1
| Agents in Recovery Mode=0
|
| Server AREA XADECA2_WCA047CS, Lth=1024, Free=536 (Active)
| Catalog Name ’WCA047.V8R2M00.CATALOG,TCP/9.30.136.99/7056’
| Connector Filters( VSAM ), ECB(x80000000)
|
|
+--------------------------------------------------------------------

14 DB2 II Operations Guide for Classic Event Publishing


RELEASE | NORELASE
Identifies whether the CSA being used by a named or unnamed server
should be released (freed) or retained. Under normal circumstances, you
do not want to release the CSA storage. If you encounter error conditions
where a correlation service may not start and reports problems related to
CSA, you can run the CSA reporting and maintenance utility and specify
the RELEASE option to free the currently allocated CSA storage. When you
release CSA storage, the correlation service that owns the storage cannot be
executing.
SERVER=Name
Identifies the name of the correlation service to be reported on, or which
correlation service owns CSA storage that is being released. If a server is
not specified, the CSA reporting and maintenance utility uses the CSA
storage associated with the unnamed server.

Shutting down a correlation service


A correlation service is designed to run continuously, but when there is a system
failure, or when you add new metadata catalog information, and in certain
recovery situations, you might need to stop a correlation service.

The databases being monitored must be shut down before shutting down the
correlation service. If you do not shut down or halt the databases before shutting
down the correlation service, it will put the change-capture agents into recovery
mode. The shut-down process of the databases must notify the correlation service
that it is being shut down for this to work correctly.

Note: The steps outlined in this section represent the ideal method of shutting
down the Classic Event Publisher operational environment. When capturing
IMS changes, generally you might find the requirement to terminate IMS
before terminating the Event Publisher environment unacceptable.
Terminating Classic Event Publisher server while an IMS active
change-capture agent is connected to the server always causes a recovery
situation, even there is no activity going on in the IMS regions being
monitored. In these situations, you can use the appropriate methods
discussed in the section ″How do I get and agent out of recovery mode″
after you restart the correlation service.

For other databases, such as VSAM, the change-capture agent can tell when the
monitored database is being shut down, and it notifies the correlation service
automatically.
To shut down a correlation service:
1. Stop all monitored databases that have their changes sent to the correlation
service that you are shutting down.
2. Stop the correlation service by issuing the following command:
P servername

Recycling a correlation service


Simply stopping and restarting a correlation service does not guarantee that the
change-capture agents that are linked to the correlation service will be returned to
active mode, even if you set the Service Info Entry parameter to COLDSTART.

Chapter 2. Operating correlation services and publication services 15


For example, the operator message CACH002A is issued when the change-capture
agent needs to go into recovery mode. If CACH002A is issued and receives no
response (‘R’ to go into recovery, ‘A’ to remain in active mode), and you stop the
correlation service and restart it with the SIE parameter set to COLDSTART, the
correlation service removes the recovery record held for the agent, but immediately
re-initiates recovery mode using the recovery information from the change-capture
agent, itself.

If you are not going to bring down all change-capture agents that are linked to the
correlation service before recycling the correlation service, go through the following
checklist:
v IMS: Check the database server system log. Verify that the CACH001 message
has been issued to indicate that the change-capture agent has been successfully
installed into the database system. In most cases, message CACH001 will be
issued almost immediately after the IMS control region opens an IMS log file.
v IMS: Check the database server for either the CACH002A or CACH003A
operator reply messages. If either of these messages are outstanding, then all
change-capture agent processing is suspended until a reply is received.
Recycling the correlation service without issuing a response to the CACH002A
message will cause the change-capture agents to immediately re-enter recovery
mode.
v Check the correlation service system log for any messages related to the agent in
question. All messages particular to an agent status are prefixed with CACG and
include the agent name. The first message for an agent will usually be either
CACG109I (indicating that the agent is sending messages to the correlation
service) or CACG111W (indicating that the agent is in recovery mode). If the
correlation service is cold started, the message CACG113I is issued.

Specifying monitored tables


Change-capture agents perform simple filtering, and forward candidate inserts,
updates, and deletes that affect monitored tables to the correlation service. The
correlation service stores the candidate changes until a commit is sent, at which
point it forwards the change data to the publication service.

16 DB2 II Operations Guide for Classic Event Publishing


|

| Chapter 3. CA-IDMS operations


| This chapter covers:
| v “Change capture”
| v “Recovery mode” on page 18
| v “Determining prerequisites” on page 19
| v “Mapping CA-IDMS data for change capture” on page 20
| v “Configuring and running a correlation service for CA-IDMS” on page 24
| v “Running a CA-IDMS active change-capture agent” on page 26
| v “Recovery procedures” on page 28
| v “Local mode versus Central Version mode” on page 33
|
| Change capture
| The CA-IDMS active change-capture agent is implemented as a CA-IDMS database
| exit. The specific exit implemented is IDMSJNL2, which is described in the User
| Exits chapter of the CA-IDMS System Operations manual.

| The IDMSJNL2 exit runs under the CA-IDMS Central Version and receives control
| before and after journal records are written to the CA-IDMS journal files. These
| records include transaction start and end information, and before and after images
| of database record and set updates.

| The IDMSJNL2 exit extracts transaction and record image information from journal
| records after the records are successfully written to the database journal file. The
| extracted information is then sent to the correlation service using Cross Memory
| services.

| Implementing the change-capture agent in CA-IDMS requires relinking the


| CA-IDMS database module IDMSDBIO and including the DB2 Information
| Integrator Classic Event Publisher version of the IDMSJNL2 exit, CACECAID.
| After relinking, you must restart the CA-IDMS Central Versions to activate the exit.
| For the steps to relink the module, see “Relinking the CA-IDMS Database I/O
| Module IDMSDBIO,” in IBM DB2 Information Integrator Getting Started with Classic
| Event Publishing.

| If you implement your own version of the IDMSJNL2 exit, you can stack the exits.
| For more information, see “Setting up the IDMSJNL2 exit” on page 27.

| The exit does minimal processing when the correlation service is not active on the
| LPAR running the CA-IDMS Central Version. Upon setup and activation of a
| correlation service, the change-capture agent filters record IDs that were changed
| in the CA-IDMS Central Version, and forwards only those records that have
| change capture active in the DB2 II Classic Event Publisher metadata catalog.

| If a correlation service is activated without any change capture defined for


| CA-IDMS-mapped tables, the change-capture agent behaves as though no
| correlation service is running.

| When a CA-IDMS Central Version is started without a correlation service running,


| the CA-IDMS change-capture agent stores the CA-IDMS online journal starting

© Copyright IBM Corp. 2003, 2004 17


| point. The agent retains this initial starting point until a correlation service is
| started that can receive the starting location as a recovery point. This starting
| location is kept in a persistent data set allocated in the CA-IDMS Central Version
| JCL. After it is saved, the initial starting point is retained until a correlation service
| is started even if the CA-IDMS Central Version is started multiple times without a
| correlation service running. This retention ensures recovery will start from the
| journal location of the first start of CA-IDMS without a correlation service.

| To maintain the journal starting point, you add a CACRCV DD to the CA-IDMS
| Central Version JCL. The change-capture agent for the Central Version writes out
| the initial CA-IDMS starting point to the file.

| Instructions for preallocating a data set on z/OS® are provided in IBM DB2
| Information Integrator Getting Started with Classic Event Publishing. Also see
| “Configuring Error Messages and Handling for Missing correlation service,” for
| instructions on configuring the ddname used for the recovery data set.
|
| Recovery mode
| If, for any reason, the change-capture agent or the correlation service has an error
| in processing, the change-capture agent goes into recovery mode. A starting point
| is written in the correlation service’s restart data store so that the recovery
| change-capture agent can reposition in the CA-IDMS journal when restarting.

| The recovery agent is a program that you can start manually or initiate by
| integrating into an automatic procedure, such as journal archiving. For more
| information, see “Starting a Recovery change-capture agent.”

| After you start the agent, it will search for the starting point in the journal. When
| the agent finds the starting point, it sends raw change data for all the database
| updates to the correlation service from the starting point until it reaches where
| CA-IDMS is in real-time (assuming you are using CA-IDMS Central Version disk
| journal files).

| The source of update information can come from:


| v CA-IDMS Central Version disk journal files
| v CA-IDMS local client journal files
| v CA-IDMS Archive journal files on tape or disk

| The most effective means of recovery is to run the CA-IDMS recovery agent using
| the CA-IDMS Central Version disk journal files. In this mode, the recovery agent
| can run continuously while the Central Version is active and can forward changes
| made to the CA-IDMS database with minimal latency.

| After you start it, the CA-IDMS recovery agent receives an initial restart position in
| the journal from the correlation service. The agent then searches the allocated
| journal files for the restart sequence number and sends information from that point
| forward. If the agent is running with the Central Version disk journal files, the
| agent attempts to catch up with the active Central Version and then continues to
| monitor the active journal for new database changes.

| Recovery from Central Version disk journals is possible even if one or more of the
| disk journals have been archived to tape. The key to successful recovery from these
| files is starting the recovery agent before the CA-IDMS Central Version has
| reprocessed all the journal files and written over the disk journal containing the

18 DB2 II Operations Guide for Classic Event Publishing


| restart point. The amount of time available between detection of a recovery
| situation and starting the recovery agent depends on the number and size of
| journal files as well as the volume of transactions written to the journal files.

| The CA-IDMS recovery agent


| The CA-IDMS recovery change-capture agents process online journal files to send
| data changes to the correlation service. DB2 Information Integrator Classic Event
| Publisher can only process uncondensed journal files. If a recovery agent attempts
| to process a condensed journal file, an error message is displayed, and processing
| terminates with an error message. The CA-IDMS recovery change capture agent is
| a multi-purpose program that, in addition to recovering lost changes, can be used
| for the following tasks:
| v Maintaining an operational environment that is conducive to fast recovery
| operations for an IDMS Central Version. This is done by maintaining a fixed
| number of unarchived Central Version journals that can be used in the recovery
| processing.
| v Monitoring an IDMS Central Version to determine if recovery operations are
| necessary.
| v Performing continuous recovery operations for a Central Version until the
| Central Version is shutdown. In this mode of operation, you can start the IDMS
| recovery change-capture agent and the agent will automatically pick up changes
| that are made by the Central Version. The agent can be configured to
| automatically terminate when an IDMS shutdown is detected and optionally,
| after all changes are processed, return the IDMS change-capture agent for that
| Central Version back into active mode in preparation of the restart of IDMS.
| v Forcing an IDMS change-capture agent back into active mode.

| The load module CACEC1DR is the CA-IDMS recovery change-capture agent. This
| module is used to perform recovery operations when the active agent goes into
| recovery mode due to an error in processing. DB2 Information Integrator Classic
| Event Publisher provides recovery from the start of a CA-IDMS Central Version
| when no correlation services are running. To ensure that application changes are
| not lost, all changes are forwarded to the publication service and then confirmed
| as successfully processed before they are considered complete. Otherwise, you risk
| losing changes in-transit even though the changes were forwarded to the
| publication service.

| The recovery change-capture agent monitors the online Central Version journals
| (J1JRNL - J4JRNL). When you start the agent, it looks in the active journal to find
| the starting point. It then starts reading and forwarding messages from the
| CA-IDMS journal to the correlation service.

| The JCL for this agent is in the SCACSAMP library and named CACIDRC1. This
| JCL must be modified and journal files added to match the journal files used by
| the CA-IDMS Central Version. The DD names for the journal files must be
| J1JRNL-JxJRNL for the number of journal files used by the CA-IDMS Central
| Version.
|
| Determining prerequisites
| Depending on what you plan to use DB2 Information Integrator Classic Event
| Publisher for, there might be steps that you need to complete before going into
| production. The following list specifies some steps you might need to perform.

Chapter 3. CA-IDMS operations 19


| Initial synchronization. If you plan to use DB2 Information Integrator Classic
| Event Publisher to maintain synchronization between two or more databases, the
| databases must start out already synchronized.

| Increase the number of journal files maintained. The recovery change-capture


| agent uses unarchived journal files to process recovery data. The standard
| CA-IDMS process of archiving journal files immediately upon switching from one
| journal file to another does not provide enough data for the DB2 Information
| Integrator Classic Event Publisher agent to effectively run in recovery mode.
| Therefore, you need to add at least one, and preferably more, extra journal files to
| your CA-IDMS configuration. The more unarchived journal files that are available,
| the more likely it is that recovery will succeed. The best case scenario, if possible,
| is to add enough journal files to process a whole day’s worth of information, and
| stop automatic archiving altogether, unless you have an excessively high-volume
| day.

| Modify your automatic archiving. Add a step to your automatic archiving JCL to
| check the number of unarchived journal files, and skip the archive if the minimum
| number of unarchived journal files are not available. For more information, see
| “Modifying Automatic Journaling,” in DB2 Information Integrator Getting Started
| with Classic Event Publishing.

| Update CA-IDMS Central Version JCL. The CA-IDMS Central Version needs to
| inform the correlation service when CA-IDMS is shut down. You need to modify
| the Central Version JCL to add this notification.

| The following is a sample of the JCL:


| //CACIDTRM EXEC PGM=CACE1TRM,
| 000023 // PARM=’AGENT=IDMS_001’
| 000024 //STEPLIB DD DISP=SHR,DSN=ep_hlw.V8R2M00.SCACLOAD
| 000025 //CTRANS DD DISP=SHR,DSN=ep_hlw.V8R2M00.SCACSASC
| 000026 //SYSPRINT DD SYSOUT=*
| 000027 //SYSTERM DD SYSOUT=*

| A sample of this JCL is supplied in SCACSAMP member name CACIDTRM.


|
| Mapping CA-IDMS data for change capture
| Mapping CA-IDMS data includes defining logical tables which access single
| records or a specific path through a CA-IDMS database. To define a mapping, the
| Data Mapper loads a CA-IDMS schema and subschema report and converts record
| layouts into SQL column definitions. When mapping a path of records, mapping
| starts with a single record and optionally traverses sets to additional records in the
| schema.

| CA-IDMS schema and subschema reports are produced by running the CA-IDMS
| schema and subschema compilers and capturing the punched output into a z/OS
| data set. You can find JCL to punch these reports in the SCACSAMP library with
| the member name CAC: IDPCH.

| Note: Before running the supplied JCL, you must know which databases, schemas,
| and subschemas you want to map. DB2 II Classic Event Publisher for
| CA-IDMS can capture changes only for CA-IDMS databases that use 24-bit
| database keys. For change capture to be effective, the CA-IDMS SEGMENT
| definition that a CA-IDMS record is stored in must specify “MAXIMUM
| RECORDS PER PAGE 255”. If you activate change capture for a record that
| is stored in a segment that uses a different value for the maximum records
20 DB2 II Operations Guide for Classic Event Publishing
| per page, the change-capture agent does not capture any changes when
| updates occur to the record and does not issue any errors.

| The basic steps to map CA-IDMS data are as follow:

| Note: TCP/IP users can transfer mainframe files between host systems and the
| workstation from within the Data Mapper using an FTP facility included in
| the Data Mapper.
| 1. Punch CA-IDMS schema and subschema reports on the mainframe.
| 2. Transfer schema and subschema reports to the workstation on which the Data
| Mapper is installed.
| 3. Start the Data Mapper.
| 4. Load the schema and subschema report for which you want to create logical
| mappings.
| 5. Create a metadata catalog of type CA-IDMS.
| 6. Create logical tables for any desired records or paths through the schema.
| 7. Import the columns from the schema report.
| 8. Generate the metadata grammar.
| 9. Transfer the generated metadata grammar back to the mainframe.
| 10. Run the DB2 Information Integrator Classic Event Publisher metadata utility
| to populate the metadata catalog used by the server.

| Mapping CA-IDMS paths for change capture


| DB2 Information Integrator Classic Event Publisher supports mapping columns
| from multiple records by defining a path through a CA-IDMS schema. Using the
| column mapping feature is necessary when fields in related records are necessary
| to provide context to changed records in the database schema. For example, if a
| CA-IDMS schema contains salary records related to employee records by a set,
| inserting an instance of the salary record would be impossible to relate to a specific
| employee unless the employee id in the employee record is included in the salary
| change message sent to the publication service.

| You map paths by selecting a starting record in the schema and navigating set
| relationships to the final target record for change capture. While data mapped in
| the path is supplied with the information in the final change record, there are the
| following points to consider:
| v Only changes to the final record in the path result in change messages to the
| correlation service and publication service. Changes to any other record in the
| path are ignored unless you map a separate table to capture changes to other
| records.
| v Change data in records other than the final record is gathered outside the
| updating transaction. The latency of collecting this information is based on the
| amount of time it takes the change-capture agent to send the change to the
| correlation service plus the time required to package up the change and forward
| the change to the publication service.

| These points means that related change data itself might have been updated or
| deleted between the transaction and the forwarding of the information to the
| publication service. For this reason, map only identifying fields (i.e., primary or
| foreign keys) in related records that are not subject to change.

Chapter 3. CA-IDMS operations 21


| If a related record is deleted between the time that an update transaction takes
| place and the time that the change is forwarded to the publication service, related
| fields are passed to the publication service as SQL NULL to indicate that the
| information is not available.

| The set navigation path in the mapping must be from owner to member records
| only. The correlation service is limited to navigating up owner chains when
| gathering related information.

| Modifications to the application schema might be necessary to ensure related data


| information can be gathered in the case of record deletes. In some cases, CA-IDMS
| does not keep an owner pointer in the record prefix, and therefore owner
| information is not available if the record is deleted. DB2 Information Integrator
| Classic Event Publisher supports change capture in these cases but will always
| send SQL NULL for related record information when the record is deleted.

| If the owner information is not available for delete purposes, the DB2 Information
| Integrator Classic Event Publisher metadata utility issues a return code of 4 and
| the message:
| WARNING: NO PARENT DBKEY EXISTS IN MEMBER RECORD PREFIX EVENT PUBLISHER
| DELETE INFORMATION FOR PARENT NOT AVAILABLE

| If you see this message and require owner information for delete purposes, then
| you must change one or more set definitions in the path to include the ‘LINKED
| TO OWNER’ statement so that owner pointers are maintained in the record prefix
| information. For more information on updating the schema, refer to your
| CA/CA-IDMS reference manuals.

| In some cases, you might need to gather information from multiple record owners
| using different paths in the schema. Because DB2 Information Integrator Classic
| Event Publisher supports only a single path in the schema for change capture
| purposes, gathering identifying fields from multiple paths is not possible in a
| single table mapping. However, because change messages sent to the publication
| service are based on a single unit-of-work in the database, you can define multiple
| tables for change capture.

| Using the CA-IDMS employee demo schema as an example, assume that you are
| interested in capturing changes to the employee record but you require both the
| office code and department id associated with an employee when the employee is
| changed. You can map the path DEPARTMENT->EMPLOYEE as one table and
| OFFICE->EMPLOYEE as a second table can be used to gather both the department
| ID and office ID when an employee is changed. The application triggered by the
| publication service can combine the information from both mapped tables as a
| change to employee, and automatically trigger a change message for both table
| mappings.

| Running the metadata utility with CA-IDMS metadata grammar


| To run, the metadata utility needs the following items:
| v The metadata grammar created by the Data Mapper
| v Access to the CA-IDMS schema and subschema reports
| v Access to the CA-IDMS Central Version

| CA-IDMS Central Version access is required to map schema records that have
| SYNONYMS for a SHARED STRUCTURE. These SYNONYMS require Central
| Version access to determine whether any prefixes or suffixes are defined for the

22 DB2 II Operations Guide for Classic Event Publishing


| elements in a record. For the prefix/suffix look-up to work correctly, the SYSIDMS
| DD statement in the metadata utility JCL must contain a ’DBNAME=xxxxxx’
| specification for the dictionary database name that contains the mapped schemas.
| Central Version access is also required so that the compression control length and
| the owner DBKEY offset can be retrieved (when path mapping is used).

| To set up the metadata utility for CA-IDMS mapping, make the following changes
| to the standard metadata utility JCL:
| 1. Add a DDLPUNCH DD statement and allocate all schema and subschema
| report files referenced in the USE statements. You can concatenate multiple
| schemas and subschemas to this DD name if the USE statements apply to more
| than one schema and subschema combination.
| 2. Add a SYSCTL DD statement that references the Central Version that the
| schema and subschema reports were produced from.
| 3. Add a SYSIDMS DD statement with a DBNAME=dictionary-name specification
| where dictionary-name is the name of the dictionary containing the schema
| reports included in the DDLPUNCH DD statement.

| While the data server itself can access multiple CA-IDMS Central Versions, the
| metadata utility is restricted to a single dictionary in a single Central Version for
| SYNONYM lookup purposes. Therefore, you must restrict the schemas and table
| definitions for each use of the metadata utility to a single dictionary in a single
| CA-IDMS Central Version.

| Filtering change capture by database when accessing multiple


| databases in a CA-IDMS Central Version
| When you map a CA-IDMS table in the Data Mapper, you can specify a database
| name in the DBNAME field. When you specify a value for this field, the
| change-capture agent will only capture data changes made on the table within the
| specified database.

| Accessing multiple CA-IDMS Central Versions from a single


| data server
| The SYSCTL data set allocated to the data server identifies the default CA-IDMS
| Central Version to communicate with. You can allocate multiple SYSCTL data sets
| with unique DD names to a single server, and can select on a table-by-table basis
| using custom-built CA-IDMS ACCESS LOADMODs that reference the appropriate
| SYSCTL DD name.

| To build and reference a custom access load module whose SYSCTL DD name is
| SYSCTL1 instead of the default value of SYSCTL:
| 1. Code an assembler IDMSOPTI module containing the following assembler
| macro statement.
| IDMSOPTI CENTRAL=YES,SYSCTL=SYSCTL1
| END
| 2. Assemble the IDMSOPTI module supplied in the SCACSAMP library member
| CACIDACM.
| 3. Relink the supplied batch access module named IDMS, and include the
| IDMSOPTI module assembled in step 2. Sample link-edit control statements to
| build the new access module are:
| v INCLUDE IDMSLOAD (IDMS)
| v ENTRY IDMS

Chapter 3. CA-IDMS operations 23


| v NAME IDMSCTL1
| Be sure to create a new name for the CA-IDMS module, as the default module
| should be left as-is for other CA-IDMS batch applications.
| 4. Use the Data Mapper to specify the new batch access module name by
| specifying an access module name of IDMSCTL1.
| 5. Generate the new metadata grammar and run the metadata utility again to
| update the metadata catalog with the new access module.

| You can use IDMSOPTI modules to manage default databases and other
| CA-IDMS-specific parameters. For more information, see the CA/CA-IDMS System
| Operations manual.

| Issues with Multiple Central Versions


| For DB2 Information Integrator Classic Event Publisher to correctly filter records in
| a multiple Central Version environment, the metadata utility must connect to the
| correct Central Version to gather internal database information about the record
| mapped for change capture. To ensure that the metadata utility connects to the
| correct Central Version, create a CA-IDMS SYSCTL DD that points to the Central
| Version to communicate with to gather the information.

| Because the metadata utility cannot communicate with more than one Central
| Version per run, mapping grammars for different Central Versions must be kept in
| separate data set members, and must be mapped using different metadata utility
| JCL that point to the appropriate Central Version.

| Note: If you have difficulties getting the metadata utility to find the internal
| database information that is necessary to map DB2 Information Integrator
| Classic Event Publisher tables, change the SYSIDMS DD statement to
| include DICTNAME=SYSDIRL and DBNAME=application-directory-name.
|
| Configuring and running a correlation service for CA-IDMS
| The correlation service is fed data from a change-capture agent, so DB2
| Information Integrator Classic Event Publisher usually does not access CA-IDMS
| data directly. The main exception to this is when you map CA-IDMS paths for DB2
| Information Integrator Classic Event Publisher. Then, the correlation service must
| access CA-IDMS data to retrieve path (record owner) records associated with the
| record that was changed.

| For a correlation service to capture changes in a CA-IDMS environment, the


| correlation service must have access to these items:
| v CA-IDMS decompression software and Presspack DCTABLE modules so that
| compressed change capture journal records can be decompressed
| v CA-IDMS Central Versions for path data retrieval

| Directly accessing CA-IDMS data


| DB2 Information Integrator Classic Event Publisher accesses CA-IDMS data by
| using the supplied batch access module, IDMS, if no access module is defined in
| the DB2 Information Integrator Classic Event Publisher table mapping. If you need
| to get to more than one Central Version from a single server, you must define an
| access module.

| The supplied module can access CA-IDMS data in either Central Version or local
| mode. By default, all of the JCL and examples that are provided are configured to

24 DB2 II Operations Guide for Classic Event Publishing


| access CA-IDMS in Central Version mode. If you need local mode access, see the
| CA-IDMS manuals for the JCL changes that are necessary to allocate and access
| CA-IDMS databases in local mode.

| The underlying access to CA-IDMS is through native DML calls. Because the
| underlying access is non-SQL, you must run the metadata utility to map data from
| the CA-IDMS database and add the mappings to the metadata catalog.

| In Central Version mode, the CA-IDMS data access component in the server
| connects to the CA-IDMS DC/UCF system as external run-units.

| The number of run-units available in a single CA-IDMS DC/UCF region is a


| CA-IDMS-configurable value. Each active DC/UCF system has a maximum
| external run-units (MAXERUS ) value that limits the number of concurrent external
| connections. The MAXERUS value governs all external connection sources,
| including batch jobs and CICS. DB2 Information Integrator Classic Event Publisher
| uses a maximum of one external run-unit.

| To set up and configure the server, you must analyze the MAXERUS value for the
| CA-IDMS Central Versions that the server is accessing. The DB2 Information
| Integrator Classic Event Publisher correlation service will never use more than one
| external run-unit.

| APF authorization of CA-IDMS.LOADLIB


| Certain DB2 Information Integrator Classic Event Publisher functions, such as
| Cross Memory services, require the server’s STEPLIB to be APF-authorized. The
| CA-IDMS.LOADLIB is not usually APF-authorized, and some utility programs in
| that library will fail if they are run from an APF-authorized STEPLIB
| concatenation.

| To run DB2 Information Integrator Classic Event Publisher, you must create a
| separate authorized copy of the CA-IDMS.LOADLIB. If you are doing change
| capture on CA-IDMS records using Presspack compression, you must authorize the
| library containing DCTABLE modules and include the library in the server
| STEPLIB concatenation.

| Setting up a data server to access a CA-IDMS Central Version


| The following JCL changes are required to access a CA-IDMS Central Version:
| 1. If you reference any IDMS records in tables that you marked for change
| capture, add the CA-IDMS.DBA.LOADLIB and CA-IDMS.LOADLIB to the
| STEPLIB concatenation if the records contain decompression or DCTABLE
| modules. You must also APF-authorize the records to add them to the STEPLIB.
| 2. Add a SYSCTL DD statement and allocate the SYSCTL file used by the Central
| Version that you need access to.
| 3. Add a SYSIDMS DD statement with a ’DBNAME=default database name’ card so
| that tables that are mapped without an explicit database name have a default
| name.

| Note: APF-authorization is required in DB2 Information Integrator Classic Event


| Publisher.

Chapter 3. CA-IDMS operations 25


|
| Running a CA-IDMS active change-capture agent
| This section discusses how to start a CA-IDMS change-capture agent. For the DB2
| Information Integrator Classic Event Publisher system to be fully-functional, you
| must start the correlation service and publication service.

| Configuring and handling error messages for missing


| correlation services
| When a change-capture agent captures changes and no correlation service is
| running that can accept the changes, the change-capture agent issues an operator
| message. This is the default message:
| CACH002A XSYNC SERVER ’xxxxxxxx’ NOT FOUND BY AGENT ’agentname’,
| REPLY ’R’ OR ’A’ RECOVERY/ACTIVE

| DB2 Information Integrator Classic Event Publisher provides a source module


| (CACE1OPT located in the SCACSAMP library) with the installation that you can
| modify to do the following tasks:
| v Change the operator message displayed
| v Disable the response A

| After starting a correlation service, the operator can select R to put the
| change-capture agent into recovery mode, which will capture all of the data that
| has changed, or the operator can select A to put the change-capture agent into
| active mode, which will discard any data captured between the time that the
| change-capture agent was started and the time that the correlation service was
| started.

| You can remove the option to select active mode, if the operator might not
| understand the ramifications of selecting active mode.

| Also stored in the CACE1OPT module is the ddname for the recovery data set that
| stores the initial restart point across multiple CA-IDMS Central Version restarts.
| For more information, see “Considerations about correlation services.”

| Creating a recovery point file


| The CA-IDMS active change-capture agent that runs in the Central Version stores
| an initial recovery point for the correlation service in an 80-byte file when the
| CA-IDMS Central Version is run without an active correlation service. This
| information is retained until a correlation service is started and receives the initial
| recovery information from the active change-capture agent. Saving an initial
| recovery point ensures that change data will not be lost even if the CA-IDMS
| Central Version is run multiple times without starting an DB2 Information
| Integrator Classic Event Publisher correlation service.

| To create a file to store recovery point data:


| v Update the CA-IDMS Central Version JCL by including this JCL statement:
| //CACRCV DD DISP=SHR,DSN=filename
| Where filename is the name of the file that you allocated to store recovery point
| data.

| The attributes of the file are:


| LRECL=80,DSORG=PS,RECFM=FB

26 DB2 II Operations Guide for Classic Event Publishing


| The blocksize of the file does not matter, and you can allocate the file to use one
| track, as the file will never contain more than a single record.

| If you stop and restart CA-IDMS after the CACH002A message , you will see the
| following message:
| CACH003A RECOVERY RECORD EXISTS FOR AGENT ’................’, REPLY
| ’R’ OR ’A’ RECOVERY/ACTIVE

| This message indicates that a recovery point from a prior CA-IDMS start exists and
| that recovery should take place. This message will appear each time you start of
| CA-IDMS until a correlation service is available to receive the recovery point from
| CA-IDMS.

| Note: This message will be displayed even if the operator starts the correlation
| service prior to restarting CA-IDMS, unless the operator had responded to
| the original CACH002A message by selecting A.

| Customizing error messages


| DB2 II Classic Event Publisher comes with an options module that allows you to
| customize the CA-IDMS change-capture agent. With this customization module,
| you can do the following tasks:
| v Change the text of operator messages issued by the change-capture agent.
| v Remove the ACTIVE option from the operator replies to the CACH002A and
| CACH003A messages. In production environments (where it is critical to not
| lose changes), the operator probably should not have the option of going active
| when changes have been lost.
| v Change the recovery data set DD name ’CACRCV’ or disable the recovery data
| set altogether. Disabling the recovery data set is not recommended.

| The source for the options module CACE1OPT is in the SCACSAMP data set. The
| source documents how to make changes to the messages and options. The source
| also includes JCL to relink the change-capture agent software that is incorporated
| into each change-capture agent database system.

| Setting up the IDMSJNL2 exit


| The CA-IDMS change-capture agent is implemented as a database exit named
| IDMSJNL2. Because this is a general purpose exit, it is possible that you are using
| your own version of the exit, and will want to incorporate your exit along with the
| DB2 Information Integrator Classic Event Publisher version of the exit. DB2
| Information Integrator Classic Event Publisher supports stacking the exits.

| To incorporate your version of the exit:


| 1. Change the internal CSECT name of your exit to IDM2JNL2 instead of
| IDMSJNL2. You can change the name either by changing your exit source and
| re-assembling the exit, or by using the linkage-editor to rename the CSECT.
| Renaming using the linkage-editor is not recommended unless you do not have
| the program source, as it can be an error-prone process.
| 2. Link your renamed exit into the IDMSDBIO module along with the DB2
| Information Integrator Classic Event Publisher exit.

| The DB2 Information Integrator Classic Event Publisher exit contains a


| weak-external for the name IDM2JNL2. If this was resolved by link-editing your
| renamed exit, all calls to the DB2 Information Integrator Classic Event Publisher

Chapter 3. CA-IDMS operations 27


| version of the exit are passed on to your exit as though DB2 Information Integrator
| Classic Event Publisher were not part of the IDMSDBIO module at all.

| Before starting a change-capture agent


| Before you start a change-capture agent, you have start a correlation service. If you
| do not start a correlation service before starting a change-capture agent, then the
| change-capture agent is put into recovery mode immediately after the correlation
| service is started.

| Starting an active change-capture agent


| The change-capture agent is a CA-IDMS database exit. The agent is automatically
| installed when you relink the CA-IDMS database I/O module IDMSDBIO as
| described in the IBM DB2 Information Integrator Installation Guide for Classic
| Federation and Classic Event Publishing. Once you restart the Central Version, the exit
| is activated. To verify that the exit was installed successfully, you can look in the
| CA-IDMS JES log for the following message:
| CACH001I EVENT PUBLISHER AGENT ’IDMS_nnn ’ INSTALLED FOR SERVER ’xxxxxxxx’

| The message appears after the first journaled record is written in the log.

| Stopping a change-capture agent


| The active change-capture agent is started and stopped automatically when the
| CA-IDMS Central Version or local-mode client starts and stops.

| The recovery change-capture agent stops automatically when it determines


| processing is complete or a processing error occurs. With the exception of
| continuous Central Version mode, the recovery agent stops when it reaches the end
| of the journal file or files allocated for processing. In continuous Central Version
| mode, the recovery agent stops automatically when it detects that the Central
| Version has been stopped and that it has caught up with all records journaled by
| the Central Version.
|
| Recovery procedures
| Recovery is performed differently depending on the database. You need to
| determine the procedures that you will use to move from recovery mode back into
| active mode.

| Retaining changes
| The recovery change-capture agent runs continuously until the correlation service
| is stopped, the CA-IDMS Central Version is stopped, or an error occurs in
| processing that prevents the recovery agent from continuing. See the following
| sections for more information on these events.

| Correlation service is stopped


| In rare circumstances, you might need to intentionally stop a correlation service.
| See “Shutting down a correlation service” for information about how to stop a
| correlation service. For more information about what happens when you stop a
| correlation service, see “Recovery Mode”.

| The CA-IDMS Central Version is stopped


| The recovery process checks the state of the Central Version before and after
| processing a set of changes from the journal files. If CA-IDMS is inactive at both
| the start and finish, the recovery process resets the state of the change-capture

28 DB2 II Operations Guide for Classic Event Publishing


| agent to active and terminates with a 0 return code. After both the correlation
| service and CA-IDMS Central Version are running, the change-capture agent
| returns to active mode processing without any operator intervention.

| An error occurs, preventing the change-capture agent from


| continuing
| If an error occurs in recovery processing, the recovery agent must discontinue
| forwarding data changes, as changes are now in question or lost. You must restart
| your recovery change-capture agent to ensure that no changes are lost.

| Manual process: You can define a manual process for starting a recovery agent. If
| you select the manual method, the amount of time between the active mode
| change-capture agent halting and the recovery mode change-capture agent starting
| needs to be small enough for recovery to be practical.

| Automatic process for CA-IDMS users: You can eliminate the need for manually
| starting a procedure by adding JCL to automated journal archiving procedures (if
| you have any) to check to see if the change-capture agent is in recovery mode. If
| the agent is in recovery mode, you can set the archiving procedure to start a
| recovery change-capture agent.

| After an automated procedure starts the recovery change-capture agent, the agent
| reads the online journals to look for its starting point.

| The recovery agent starts just before the failed transaction, as determined by the
| correlation service. The correlation service uses the last message confirmed by the
| publication service to determine what the last failed transaction is.

| In cases where the error is related to the actual journal data, the recovery agent is
| likely to fail at the same point if restarted. One example of this type of error is an
| incorrect mapping in the DB2 II Classic Event Publisher system catalogs. If
| mapping error exists, it may be possible to correct the error in the DB2 II Classic
| Event Publisher metadata catalog and then restart the recovery agent and the
| correlation service

| Starting a recovery change-capture agent


| The recovery change-capture agent is a program that reads CA-IDMS journal files
| and forwards previously journaled changes to the DB2 Information Integrator
| Classic Event Publisher correlation service. You can start this agent as a batch job
| or through automated installation procedures when the active agent enters
| recovery mode.

| Unlike the active agent, which captures changes as they are written to the journal,
| the recovery agent can process historical changes that were lost by the active agent
| due to the lack of an active correlation service or some other system failure. The
| recovery agent can also use throttling to control the rate at which changes are sent
| to the correlation service so that other active and recovery agents can continue to
| operate normally without the risk of overrunning the message queue.

| The z/OS parameter controls the processing of the recovery agent. The format of
| the parameter is shown in the following example:
| PARM=’CV,Optkeyword=vvvv,...’
| ’LOCAL,Optkeyword=vvvv,...’
| ’ARCHIVE,Optkeyword=vvvv,...’

Chapter 3. CA-IDMS operations 29


|
| ’REPORT,Optkeyword=vvvv,...’
| ’MONITOR,Optkeyword=vvvv,...’
| ’COUNT’

| Where:
| v CV defines recovery from one or more CA-IDMS disk journal files written by a
| CA-IDMS Central Version. In CV mode, the Central Version can be either
| running or stopped.
| v LOCAL defines recovery from a single tape or disk journal file written by a local
| mode CA-IDMS application.
| v ARCHIVE defines recovery from an archived Central Version journal file. Do
| not use the AGENT optional keyword with PARM=’ARCHIVE’.
| v REPORT requests a report of the current recovery sequence and timestamp for
| the requested agent. The AGENT keyword is required with the REPORT option.
| v MONITOR indicates whether the recovery agent will do a single check for
| recovery state or will run as a continuous monitor for automatic recovery.
| v COUNT indicates to count the number of full recovery logs and skip automatic
| archiving if the minimum number of full journals is not available.
| v Optkeyword=vvvv is one or more optional keyword parameters that can be
| included in the parameter string to control recovery processing.

| The following list shows the optional keywords for the CV, LOCAL, and REPORT
| parameters:
| v ACTIVATE={Y | N}. Specifies whether or not to enable the active agent after the
| recovery agent successfully completes. Specify Y to inform the correlation service
| that all recovery messages were sent and that the active change-capture agent
| can begin forwarding messages again after the final recovery message is
| processed.
| Using ACTIVATE with the CV parameter is not recommended unless you need
| to disable automatic activation when continuous-mode processing ends because
| a CA-IDMS Central Version was shut down.
| If the ACTIVATE parameter is not specified, the agent remains in recovery mode
| at termination unless continuous mode CV processing detects that a CA-IDMS
| Central Version was shut down.
| v AGENT=agent name. Identifies the active change-capture agent name when you
| set PARM=LOCAL or the Central Version number is 0. For local agents, this
| name must be ’IDMS_jjjjjjjj’ where jjjjjjjj is the original z/OS job or started task
| name that performed the database update. For Central Version 0 systems, the
| name is ’IDMS_ssssssss’ where ssssssss is the Central Version started task name.
| For Central Version disk and tape archive files, the Central Version number
| carried in journal ‘TIME’ records is used to automatically determine the agent
| name. The name is always ‘IDMS_nnn’ where nnn is the Central Version
| number.
| PARM=’LOCAL,AGENT=IDMS_IDMJUPDT’
| You must specify this value to use the REPORT option.
| v RESTARTWAIT=nn{M|S}. Defines the wait interval in minutes or seconds for a
| continuous run operation (PARM=CV). Each time the recovery agent catches up
| to the last record written by an active CA-IDMS Central Version, the recovery
| agent suspends processing for the specified interval before querying the active
| journal for new change records.

30 DB2 II Operations Guide for Classic Event Publishing


| If M or S is not specified, the value supplied is assumed to be in seconds. If the
| parameter is not specified or is specified as 0, the agent runs in non-continuous
| mode and terminates when it catches up with the current position in the active
| journal file.
| PARM=’CV,RESTARTWAIT=5S’
| v THROTTLE=nnnn. Defines a throttle value to prevent the recovery agent from
| overrunning the correlation service with change messages. The throttle value
| defines the maximum number of messages to be queued to the correlation
| service at any point in time. This value ensures available space in the message
| queue for any active change-capture agents communicating with the correlation
| service.
| If the THROTTLE value is not specified, the throttle value defaults to 512. A
| value of 0 disables throttling of messages.
| PARM=’CV,THROTTLE=1024’
| v TIMEOUT=nn{M|S}. Defines a timeout value when throttling is used.
| Throttling relies on the correlation service responding to requests that messages
| be received. In the event that the correlation service is unable to respond within
| the specified TIMEOUT value, the recovery server stops with an error.
| If the TIMEOUT value is not specified, the value defaults to 5 minutes.
| PARM=’CV,THROTTLE=1024,TIMEOUT=1M’
| v SERVER=name. Specifies the name of the correlation service that you want this
| change-capture agent to communicate with.

| Periodic recovery-checking compared to monitoring


| You can periodically check to see if an agent has entered recovery mode by
| integrating the following jobstep into an existing JCL:
| PARM=’MONITOR,RESTARTWAIT=0S’

| When you run the recovery agent in this mode, the agent checks the state of
| recovery and whether an active correlation service exists. If the agent is in recovery
| mode and a correlation service is active and ready to receive messages, the
| program ends with a return code of 0; otherwise, the program ends with a return
| code of 4.

| You can use the return code in the JCL to start or skip a recovery agent. The JCL
| might look like the following example:
| //CHECK EXEC PGM=CACEC1DR,PARM=’MONITOR,RESTARTWAIT=0S’
| //...
| //RECOVER EXEC PGM=CACEC1DR,PARM=’CV,...’,COND=(0,NE,CHECK)
| //...

| You can also run a continuous monitor to check for recovery status. By specifying
| a RESTARTWAIT value in the parm (such as PARM=’MONITOR,RESTARTWAIT=10S), the
| recovery program will continuously monitor the state of recovery and the
| correlation service until either of the following conditions is true:
| v The agent goes into recovery mode and a correlation service is running and
| ready to receive messages
| v CA-IDMS is shut down

| When the first condition is met, the monitor ends with a 0 return code.

| To run a continuous monitor, run the following JCL each time the CA-IDMS
| Central Version is started:

Chapter 3. CA-IDMS operations 31


| //MONITOR EXEC PGM=CACEC1DR,PARM=’MONITOR,RESTARTWAIT=10S,TIMEOUT=5M’
| //...
| //RECOVER EXEC PGM=CACEC1DR,PARM=’CV,...’,COND=(0,NE,MONITOR)
| //...

| The RESTARTWAIT parameter determines how often the monitor checks the state
| of recovery. Use the TIMEOUT value to specify how long CA-IDMS can be down
| before the monitor terminates.

| By default, the monitor stops when CA-IDMS is stopped. If the monitor is stopped
| because CA-IDMS is brought down, the monitor ends with a return code of 4. If
| you want the monitor continue when CA-IDMS is shut down, set the TIMEOUT
| value to 24H.

| Parameter example
| The following example shows a parameter string to recover data from Central
| Version disk files in continuous mode:
| EXEC PGM=CACEC1DR,PARM=’CV,RESTARTWAIT=2S’

| Execution JCL
| //JOBNAME JOB (ACCT,INFO),’CA-IDMS RECOVERY’,CLASS=A,MSGCLASS=X,
| // NOTIFY=&SYSUID
| //CACEC1DR EXEC PGM=CACEC1DR,
| // PARM=’CV,RESTARTWAIT=5S,TIMEOUT=5M’
| //STEPLIB DD DISP=SHR,DSN=CAC.LOADLIB
| //CTRANSDD DISP=SHR,DSN=SYS2.SASC.C650.LINKLIB
| //** CV JOURNAL DATASETS AS DEFINED IN THE DMCL SOURCE
| //** FOR CV 120.
| //J1JRNL DD DISP=SHR,DSN=CAI.CA-IDMS.J1JRNL
| //J2JRNL DD DISP=SHR,DSN=CAI.CA-IDMS.J2JRNL
| //J3JRNL DD DISP=SHR,DSN=CAI.CA-IDMS.J3JRNL
| //J4JRNL DD DISP=SHR,DSN=CAI.CA-IDMS.J4JRNL
| //SYSPRINT DD SYSOUT=*
| //SYSTERMDD SYSOUT=*
| //

| Journal files in execution JCL


| CA-IDMS journals are read from the JnJRNL DD statements allocated in the
| recovery execution JCL. When you use Central Version mode (PARM=’CV,...’), you
| can include multiple journal files so that all journal files allocated in the Central
| Version startup JCL are processed by the recovery change-capture agent.

| When you allocate multiple files for Central Version mode, make sure that the
| order of data set allocations to JnJRNL matches the processing order as defined in
| the CREATE DISK JOURNAL statements in the DMCL.

| Reverting to active mode during development


| When you implement DB2 Information Integrator Classic Event Publisher on test
| servers to determine how best to deploy the application, the system will probably
| go into recovery mode regularly. To make development easier, DB2 Information
| Integrator Classic Event Publisher provides a way for you to put the system back
| into active mode without requiring you to complete the standard recovery process.
| To bring the system back to active mode:
| 1. If possible, stop CA-IDMS.
| Because it is frequently not possible to stop CA-IDMS, these steps will still
| work if CA-IDMS is running. If you are unable to stop CA-IDMS, run these
| steps when CA-IDMS is least busy.

32 DB2 II Operations Guide for Classic Event Publishing


| 2. If there is a CACH002A message on the operator screen, respond with A.
| 3. Configure and run the recovery change-capture agent using the following
| parameters:
| REPORT,AGENT=IDMS_nnn,ACTIVATE=Y
| If the correlation service is a named server, use the following parameters:
| REPORT,SERVER=servername,AGENT=IDMS_nnn,ACTIVATE=Y
| These parameters allow you to skip recovery data and switch to active mode
| with the correlation service and change-capture agent running.
| If the correlation service goes into active mode and then immediately reverts to
| recovery mode, the recovery agent was probably started when there were
| in-flight CA-IDMS transactions. If the correlation service determines that it has
| received an incomplete transaction, it will immediately enter recovery mode.
| 4. If the correlation service enters recovery mode, repeat step 3 until the
| correlation service remains in active mode.
|
| Local mode versus Central Version mode
| It is not possible to merge journal files from two different sources. For example, if
| you run a batch program in local mode with journaling, and you run another
| program in the CA-IDMS Central Version, you cannot merge the journal files.

| In a case like this, if the local mode files are for databases other than those
| accessed from the Central Version, you can consider the local mode journals to be
| a separate and unrelated change-capture agent from the Central Version
| change-capture agent.

| You can run both change-capture agents, and have both agents update separate
| databases. To do this, run the local mode clients with CA-IDMS journaling active.
| Journaling in local mode requires that you run local mode clients using a
| CA-IDMS DMCL that defines disk or tape journals. For more information on local
| mode journaling, see the CA/CA-IDMS Manuals.

| Recovery mode is an issue when local mode clients are updating databases
| normally managed by a Central Version. The problem is that you have multiple
| sets of journals for the same database. You cannot merge these sets of journals.

| To compensate for this problem, swap the Central Version journal files before
| running the local client. You can recover the local journal files between the
| swapped Central Version journal files, if necessary.

Chapter 3. CA-IDMS operations 33


34 DB2 II Operations Guide for Classic Event Publishing
Chapter 4. IMS operations
This chapter includes the following topics:
v “Overview of DB2 Information Integrator Classic Event Publisher for IMS”
v “Active change-capture agents” on page 37
v “Given a restart point, how can I identify the IMS log files required?” on page
47
v “How do I get an agent out of recovery mode?” on page 61
v “What are IMS log records of interest?” on page 66
v “What are cascade deletes?” on page 67
v “What is an XM queue overrun?” on page 69
v “How do I install the IMS active change-capture agent?” on page 72
v “Mapping validation rules” on page 78
v “How do I deal with IMS test and production systems on the same MVS
image?” on page 79
v “I changed some IMS data and nothing happened” on page 80
v “What is the effect of DB2 Information Integrator Classic Event Publisher on my
IMS system?” on page 80
v “What DB2 Information Integrator Classic Event Publisher monitoring
capabilities are available for IMS?” on page 83

Overview of DB2 Information Integrator Classic Event Publisher for


IMS
This chapter provides a conceptual and technical overview of how change capture
works for IMS data sources. It is assumed that you have a working knowledge of
the DB2 Information Integrator Classic Event Publisher correlation service and
active and recovery change-capture agents.

This information is presented with a quick overview of what the IMS active
change-capture agent is and does. The majority of the information, like large parts
of the IMS documentation, discusses how to recover a failed DB2 Information
Integrator Classic Event Publisher IMS system so that no changes are lost and that
prior changes are propagated as quickly as possible and in the correct
chronological order. This document assumes that you have a working knowledge
of IMS operational environments, recovery concepts, recovery techniques, and the
reliance on IMS log files in all recovery situations.

You will usually discover the need to deal with and understand recovery during a
proof-of-concept trial. In a test environment, the most likely response to a DB2
Information Integrator Classic Event Publisher recovery situation is to cold start
the system and not deal with recovery. This response is not acceptable in a
production environment!

Everything must be running perfectly for DB2 Information Integrator Classic Event
Publisher to operate in its optimum mode, referred to as active mode.
Unfortunately, because this is a multi-tiered application (essentially a Web Service),
failures will occur. You must be able to deal with these failures when they occur.

© Copyright IBM Corp. 2003, 2004 35


The more mundane topics of how to install and configure a DB2 Information
Integrator Classic Event Publisher IMS change capture system are covered in the
DB2 Information Integrator Getting Started with Classic Event Publishing.

Additional topics cover how to:


v Deal with situations like monitoring test and production IMS subsystems
running on the same LPAR (image)
v Monitor IMS DB2 Information Integrator Classic Event Publisher operations in
low-volume test environments

DB2 Information Integrator Classic Event Publisher


implementation plan
You should develop an implementation plan for your DB2 Information Integrator
Classic Event Publisher system. This plan would identify:
v Whether you plan to do a large-scale or small-scale IMS DB2 Information
Integrator Classic Event Publisher deployment
v The names of IMS active change-capture agents that will be executing
v The names of the change-capture agents’ recovery data sets
v If you are planning to implement IMS log file tracking

Also as part of this implementation plan, you need to identify:


v The names of the DBDs and segments to be monitored.
v The information that will be collected in the Data Capture log records.
v A checklist of steps required to augment each of your DBDs. You make these
DBD load modules accessible to the correlation service and coordinate the
changes to the DBDs. You install these augmented DBDs into your IMS
environment, restart your correlation service, and then restart IMS.

Note: The sequence of starting the correlation service and then restarting IMS is
the optimum method of initiating the change-capture process. However, if
you plan to use IMS online change to augment your DBDs for
change-capture and do not plan to bring IMS down, there are other
techniques that you can use. These changes will force you into recovery
mode. See the sections “How do I get an agent out of recovery mode?” on
page 61 and “Returning agents back to active mode” on page 107for more
details on how to get out of recovery mode.

Start by identifying the DBDs that you want to monitor. You can then identify the
DB/DC or DBCTL applications and batch applications that update these databases
based on the PSBs that contain update PROCOPTs for these databases. When you
know how many, if any, batch jobs are affected, you can then make an informed
decision on whether to perform a large- or small-scale deployment. You can then
determine how you will set up your DB2 Information Integrator Classic Event
Publisher system.

If your site consists only of a single DB/DC or DBCTL subsystem, and all of your
batch applications are converted to access or update your IMS data as BMPs, you
will find that setting up your DB2 Information Integrator Classic Event Publisher
system is easy and straightforward. If you have multiple DB/DC or DBCTL
subsystems in a data-sharing environment, or many IMS batch update applications,
then things become more complicated.

36 DB2 II Operations Guide for Classic Event Publishing


If you have a complex deployment that affects numerous databases, IMS batch
jobs, and started tasks, test your implementation plan against your IMS test system
before implementing DB2 Information Integrator Classic Event Publisher in your
IMS production environment.

Supported environments and program types


The following table identifies the IMS environments, and database types that are
currently supported for change capture.
Table 3. Supported IMS environments and database types
IMS environment Databases supported
DB Batch Full-function
TM Batch None
DB/DC Full-function and DEDB
Full-function and DEDB
Full-function and DEDB
DBCTL Full-function DEBD
DCCTL None
None
None

You can capture changes for the following types of full-function databases:
v HISAM
v HDAM
v HIDAM
v SHISAM
v DEDB

Data capture is supported only when the batch job allocates a non-dummy
IEFRDER DD statement.

This environment also supports capturing updates made from CICS applications,
DB2 Information Integrator Classic Federation, or ODBA clients using DRA.

Change-capture agents
The topics in this section cover the IMS change-capture agents, both active and
recovery.

Active change-capture agents


An IMS active change-capture agent is any IMS control region that has installed
the IMS Logger Exit (DFSFLGX0) supplied by DB2 Information Integrator Classic
Event Publisher. The types of IMS control regions that are of interest are:
v DB/DC subsystems
v DBCTL subsystems
v Batch jobs

The IMS Logger Exit can also be activated in DC subsystems and the IMS Batch
Backout Utility, but because IMS changes are not possible in these environments,

Chapter 4. IMS operations 37


the IMS Logger Exit will not attempt to communicate with a correlation service
when active in these types of IMS control regions.

The name of the IMS active change-capture agent is the name of the batch job or
started task. If batch jobs have multiple job steps that update IMS data, each job
step is considered a separate instance of the IMS active change-capture agent’s
execution.

Functionality
When an IMS active change-capture agent is invoked by IMS in a DB/DC, DBCTL,
or batch IMS control region, IMS passes the IMS Logger Exit a pointer to a buffer
containing one or more IMS log records that have been successfully written to an
IMS log file. The IMS Logger Exit is also called at these times:
v During control region initialization processing, after the IMS log files are opened
v During control region normal termination processing, after the IMS log files are
closed
v When an IMS control region abnormally terminates

The IBM description of the IMS Logger Exit indicates that it can be used for
recovery purposes.

When the IMS active change-capture agent is called by IMS after IMS log records
are written to the IMS log and when IMS is terminating (normally or abnormally),
the IMS active change-capture agent sends a single XM data gram to the
correlation service. No communication is attempted when the IMS change-capture
agent is called during IMS initialization processing, because at this time, a restart
point cannot be established. See ″Restart Points″ for more information about how
Classic Event Publisher recovers from failures. The data gram contains the
following information:
v IMS active change-capture agent identity information
v IMS log record suffix information for the first log record and last log record in
the buffer
v All of the IMS log records that are identified in the topic “What are IMS log
records of interest?” on page 66.

XM data grams represent a unidirectional asynchronous method of communication


with the correlation service. The IMS active change-capture agent puts the XM data
gram on a memory queue, and the correlation service gets the message when it is
not busy doing other work.

There is a bi-directional method of communications between the IMS active


change-capture agent and the correlation service using common memory (CSA).
The IMS active change-capture agent accesses the common memory each time the
agent is called to identify:
v Whether a correlation service is active
v The name of the XM queue to use
v The current status of the IMS active change-capture agent

The IMS active change-capture agent can also update common storage to report
that the agent is in recovery mode. The agent might go into recovery mode
because of an XM data gram failure or some other processing error. Some error
situations that are detected by the IMS active change-capture agent are so severe
that the execution environment is considered compromised. In these situations, no
attempt is made to notify the correlation service of the error condition. In these

38 DB2 II Operations Guide for Classic Event Publishing


situations, the IMS active change-capture agent issues WTO error messages so that
the operations personnel are aware that something is wrong.

From an IMS perspective, an IMS active change-capture agent is in one or two


modes of operation:
Active mode
A correlation service is active, not reporting any errors, and the IMS active
change-capture agent is successfully communicating with the correlation
service via XM and common memory.
Recovery mode
Communications were never successfully established with a correlation
service, the correlation service reported some form of error, or the IMS
active change-capture agent detected an error condition.

After an IMS active change-capture agent is in recovery mode, you must run the
IMS recovery change-capture agent to find any changes that were made between
the time the agent went into recovery mode until one of three points in time:
v The time that the IMS control region is terminated
v You quiesce the databases being monitored and recover all changes up to the
quiesce point
v You have a window of time where the monitored databases are not actively
being updated and you recover all changes up to that point
After all changes are recovered, you can use one of the techniques discussed in
“How do I get an agent out of recovery mode?” on page 61 to return the agent
and the Classic Event Publisher system back to active mode.

Maintaining state
Figure 1 shows the IMS environment that DB2 Information Integrator Classic Event
Publisher is designed to operate in to capture changes to IMS data. The DB2
Information Integrator Classic Event Publisher correlation service is designed to
operate continuously. For IMS batch jobs, the correlation service must be active
before the batch jobs are started and must remain active until the IMS batch jobs
complete. Ideally, the correlation service should be active before an IMS DB/DC or
DBCTL subsystem is started and remain operational until the DB/DC or DBCTL
subsystem is terminated. Due to the length of time that DB/DC and DBCTL
subsystems are typically operational, Classic Event Publisher does support starting
the correlation service while one of these subsystems is already operational and
provides mechanisms to recover from Classic Event Publisher failures while these
kinds of IMS subsystems are active.

Chapter 4. IMS operations 39


Time

Change-Capture Agent

IMS Control Region Application


Log File(s)

Correlation
Change-Capture Agent
Service
IMS Control Region
IMS Databases

Application
Change-Capture Agent

IMS Control Region


Log File(s)

Application

Figure 1. IMS Operational Environment

The preceding diagram shows three different IMS control regions that were started
and terminated while the correlation service was operational. In the diagram, the
first and third IMS control regions could represent a DB/DC or DBCTL subsystem
that has been recycled as a part of normal operations. Alternately, the first IMS
control region could have terminated abnormally, and the third IMS control region
would represent an emergency restart.

The second IMS control region could represent a batch job that was started while
the first subsystem was running and continued running until after the subsystem
was restarted, after which it subsequently terminated. Alternately, the second IMS
control region could be another DB/DC or DBCTL subsystem.

If the second control region was a batch job, then the diagram would show only
one client application accessing the IMS control region and the client’s run-time
duration would closely match that of the IMS control region. If you run a lot of
batch jobs, then you would see a series of control region instances, some of them
executing serially and potentially concurrently depending on the database being
accessed by the batch jobs and your data sharing environment.

The diagram shows that in a DB/DC or DBCTL environment, multiple concurrent


applications can be accessing or updating IMS data simultaneously. These
applications can be executing on the same LPAR or might be remote applications

40 DB2 II Operations Guide for Classic Event Publishing


that are gaining access to IMS using a program such as DB2 Information Integrator
Classic Federation and the DRA interface to get to the data.

The diagram also shows that as the applications issue access and update requests
to the IMS control region, the control region accesses the IMS databases on behalf
of the client application. When an update request is received, the IMS control
region updates the database and then generates undo or redo log records to be
able to back out the changes if the client application requests it or if the application
abnormally terminates.

The IMS control region also generates log records to track the state of the current
unit-of-work for each application as well as the type 99 data capture records that
DB2 Information Integrator Classic Event Publisher uses.

The log records written by the control region are buffered. When a buffer is filled
and successfully written to the log file, the IMS active change-capture agent is
called. When called, the IMS active change-capture agent sends a single Cross
Memory data gram to the correlation service. In the data gram are any log records
that DB2 Information Integrator Classic Event Publisher uses to track the state of
the current units-of-work that are in-flight and any type 99 data capture log
records that were in the buffer passed by IMS.

Log record sequence-checking


Even if no data capture or synchpoint log records are in the buffer passed by IMS,
a data gram is still sent to the correlation service. In these cases, the data gram just
contains information about the starting and ending log record sequence numbers
of the log records contained in the buffer. The correlation service tracks each active
change-capture agent’s log record sequence numbers, because IBM documents that
it is possible for DFSFLGX0 to be called with duplicated, lost, or out-of-sequence
buffers. The correlation service automatically discards duplicated buffers, though
when a lost or out-of-sequence condition is encountered it forces a recovery
situation.

Note: In situations where IMS has made a mistake because the log buffer has been
successfully written to the IMS log files, recovery is still possible because the
IMS log records are guaranteed to be in the correct sequence in the IMS log
files.

During development of the IMS DB2 Information Integrator Classic Event


Publisher product, a duplicated, lost, or out-of-sequence log buffer condition has
never been encountered. The correlation service is designed to detect and deal with
these situations. Likewise, the correlation service is also designed to track the state
and health of each IMS control region. This is done by sending additional log
records to the correlation service that are not related to data capture or tracking the
progress of the currently in-flight units-of-work. The main log record of interest for
tracking control region status is the type 06 IMS/VS Accounting Record.

Using the information provided in the IMS/VS Accounting Record, as well as


information provided by IMS to DFSFGLX0, the correlation service detects and
handles the following situations:
v Start of an IMS control region.
v Normal termination of a batch job.
v Abnormal termination of a batch job. If a unit-of-work is in-flight, it is
automatically rolled back. In this situation, you might have to run the IMS Batch

Chapter 4. IMS operations 41


Backout Utility, but from an DB2 Information Integrator Classic Event Publisher
perspective this is not an error condition.
v Normal termination of a DB/DC or DBCTL subsystem.
v Abnormal termination of a DB/DC or DBCTL subsystem. Any in-flight
units-of-work that have performed updates are retained, awaiting the control
regions restart.
v Emergency restart of a DB/DC or DBCTL subsystem.

For example, in the case of an emergency restart of a DB/DC or DBCTL


subsystem, the correlation service expects to see a series of synchpoint control
records that are reporting roll backs for the units-of-work that had updated a
database when the system failed. If the correlation service remained operational
between the termination and restart of the DB/DC or DBCTL subsystem, the
correlation service has retained tracking information about the subsystem that
terminated and its in-flight units-of-work. In this situation, these units-of-work are
rolled back just as if the application issued a roll back request.

Running IMS without a correlation service


When you run IMS without running a correlation service, recovery is required, but
the correlation service does not know this when it is started up.

To handle these situations you can update your IMS control region JCL and
include a reference to a recovery data set. By default, the recovery data set DD
name is CACRCV. The data set referenced by CACRCV needs to be allocated as an
80-byte fixed record length file. Only one record is ever written to this file, so it
does not have to be blocked and you can allocate the minimum of one track.

When a recovery data set exists, and the IMS active change-capture agent detects
an error, such as the correlation service not running, it records a restart point in the
recovery data set. When the IMS active change-capture agent is again able to
communicate with a correlation service, the agent forwards the restart information
to the correlation service, letting the correlation service know that a recovery
situation exists. For more information about the recovery data set, see “Recovery
data sets” on page 44.

If you are monitoring IMS for DB2 Information Integrator Classic Event Publisher
change capture in a single DB/DC or DBCTL subsystem environment, then this is
all the information you need to get DB2 Information Integrator Classic Event
Publisher in-sync with IMS. Given that you have a known active change-capture
agent in recovery mode, you can begin the recovery process. See the topic “What is
the IMS log file recovery process?” on page 53, for details on getting DB2
Information Integrator Classic Event Publisher synchronized with IMS.

If the other IMS control regions are DB/DC or DBCTL subsystems, then after the
first IMS subsystem is restarted and reports that it is in recovery mode, you can
start the other DB/DC or DBCTL subsystems, which should also report that they
are in recovery mode. If they do, then you are in a normal recovery situation.

Starting a DB/DC or DBCTL subsystem and having it report that it is in recovery


mode is counter-productive; it just lengthens the recovery process. Additionally, if
an IMS Batch job was run without a correlation service being active, and not in
recovery mode, it might be impractical to re-run the batch job and have it report
that it is in recovery mode.

42 DB2 II Operations Guide for Classic Event Publishing


Assuming that you know that a correlation service has failed, then by running the
IMS recovery change-capture agent it can tell you which IMS control regions need
to be recovered. If you do not know that the correlation service is not active or is
in recovery mode, then running additional IMS control regions and applications
lengthens the time it takes to get IMS DB2 Information Integrator Classic Event
Publisher out of recovery and in sync with IMS. As long as you have the IMS log
files for failed IMS active change-capture agents, recovery is always possible.

Recovery change-capture agents


The IMS recovery change-capture agent is a batch job that is used to get one or
more IMS active change-capture agents that are in recovery mode back in
synchronization with the IMS control region where the IMS active change-capture
agent is, or was, running. Synchronization is performed by reading the IMS log
files (created by the IMS control region that was being monitored) and forwarding
the appropriate IMS log records to the correlation service for processing. The JCL
to run the IMS recovery change-capture agent can be found in SCACSAMP
member name CACIMSRA.

Each IMS active change-capture agent that is in recovery mode has a restart point
that identifies a specific IMS log record where the recovery process needs to begin.

The IMS recovery change-capture agent is designed to execute in one of four


modes:
v Identify:
– Agent statuses,
– The restart point, and
– (Optionally) the IMS log files required for recovery.
v Place an unknown agent into recovery mode.
v IMS log file recovery.
v Moving agents that are in recovery mode back into active mode.

You identify the mode, agent names, IMS control region types, and file names
using a 80-byte fixed format SYSIN data set. This file is referred to as the IMS
recovery change-capture agent control file.

When running in IMS log file recovery mode, the data set name specified in the
control file refers to the name of an IMS log file to be recovered. Multiple IMS log
files can be input for a single agent, and multiple agents can be recovered
simultaneously. When running in the other modes, the data set name refers to the
name of a recovery data set.

Important: If you are in a data sharing environment and multiple IMS active
change-capture agents that updated shared data are in recovery, then
you must identify these agents and supply their IMS log files in the
IMS log file recovery process or you can propagate changes
out-of-sequence.

One difference between IMS recovery and IMS DB2 Information Integrator Classic
Event Publisher recovery is that IMS recovery is oriented towards recovering the
individual IMS databases that are out of sync. When all of the databases that are
out of sync are recovered, the databases can be restarted and normal operations
can resume. IMS DB2 Information Integrator Classic Event Publisher recovery has
a different perspective because, when DB2 Information Integrator Classic Event
Publisher fails, the IMS operational environment is not compromised and it might

Chapter 4. IMS operations 43


be that you continue running IMS applications, while performing DB2 Information
Integrator Classic Event Publisher recovery operations (all the while checking up
with IMS), until it is convenient to stop IMS, or you are at a point where no
changes are occurring to the monitored databases and you have recovered all
changes, at which point you can complete DB2 Information Integrator Classic
Event Publisher recovery and get the system back into active mode.

Another difference with DB2 Information Integrator Classic Event Publisher


recovery is that it has been designed to be application-oriented, and the IMS
recovery change-capture agent is designed to propagate the changes that were
made by your IMS applications in the time sequence in which the updates
occurred. The assumption is that these applications are inter-related and it is
important the actions resulting from these changes occur in the sequence that they
physically occurred. The assumption is that you are in a data sharing environment
where multiple IMS active change-capture agents that were monitoring changes to
shared data were executing concurrently. Hence the above warning. However, if
you have multiple IMS active change-capture agents that are in recovery mode, but
that are not in a data sharing environment then you can recover these agents
independently. It all depends on your environment.

Recovery data sets


A recovery data set is a single-record, 80-byte fixed-length sequential data set that
an IMS active change-capture agent updates when the agent is running and the
correlation service is not. The IMS active change-capture agent records in the
recovery data set information about the first log record that was passed in the IMS
log buffer on the first IMS Logger Exit write call. See the next topic ″Restart
Points″ for an exact description of the information recorded in the recovery data
set.

Use of recovery data sets is required when running the IMS recovery
change-capture agent. Update your IMS JCL to include a reference to a recovery
data set for each IMS active change-capture agent running at your site.

Use the following naming standard for recovery data sets:


&HLVLQUAL.DB2IIEP.&JOBNAME

where &HLVLQUAL is a high-level data set name qualifier that the IMS control region
job or started task name can access and update. The second-level qualifier is a
constant (in the example above, DB2IIEP). This can be any unique identifier up to
eight characters. Its sole purpose is to group the DB2 Information Integrator
Classic Event Publisher recovery data sets together so you can focus on them. The
file name suffix should be the job name or started task name of the IMS control
region being monitored.

If you do not add a recovery data set to one of your IMS jobs and run the job
without a correlation service being active, you can manually create a recovery data
set for that agent. To do so, you must know when the job was executed and the
file name of the first (or only) IMS log file created by IMS. See “How do I
manually create a recovery data set?” on page 62, for more information.

If an IMS active change-capture agent is in recovery mode and the correlation


service knows that the agent is in recovery, you do not have to have a recovery
data set for that agent. In this scenario, specify a non-existent recovery data set in
the IMS recovery change-capture agent control file.

44 DB2 II Operations Guide for Classic Event Publishing


Restart points
Every IMS active change-capture agent that is in recovery mode has a restart point.
A restart point is a location within an IMS log file where the recovery process
needs to begin. The IMS log file that contains the restart point and all log files
created thereafter that contain changes to the monitored databases, up until the
point that IMS is shutdown, or a quiesce point is reached, must be input into the
recovery process. If the agent associated with an IMS control region is in recovery
mode and that IMS job or subsystem is run again before the recovery process is
completed, then any IMS log files created in subsequent executions must also be
input into the recovery process.

An agent that is in recovery mode has a specific record within an IMS log file
where recovery needs to start. Initially, this is the first IMS log record when the
active change-capture agent was called for the first IMS Logger Exit write
operation. For an IMS batch application, the restart point identifies the first IMS
log record in the log file. For a DC/DC or DBCTL subsystem this is generally the
first IMS log record in the currently active online log data set. When a UOR (unit
of recovery) is committed by the correlation service, the agent’s restart point is the
IMS log record associated with the first change received from the oldest living
UOR when the committed UOR started. As UORs are committed, the restart point
keeps getting pushed farther into the future. See the discussion below that more
fully explains the concept of the oldest living UOR.

The individual IMS log records represent events (not DB2 Information Integrator
Classic Event Publisher events, but rather IMS events) that were created because an
IMS application did something. When the IMS active change-capture agent is
called with a log buffer, the individual IMS log records in the buffer were all
created in the past. In a high-volume system the time between when the IMS log
record was created and when the IMS active change-capture agent is called can be
only milliseconds off, however, IMS DB2 Information Integrator Classic Event
Publisher is always dealing with things that happened in the past.

When discussing restart points and IMS log files, you need to think of the
individual IMS log records as representing a stream of events in time. The way
IMS operates the IMS log records written to an IMS log file are always written in
time sequence. When discussing pushing restart points into the future, the future
in this context is forward in the IMS log files stream of time.

A restart point identifies a unique IMS log record in an IMS log file where the
recovery process needs to begin. The following information is associated with a
restart point:
v DB2 timestamp when the agent went into recovery
v Internal timestamp when the agent went into recovery
v The IMS log record suffix of the first IMS log record recorded by the
change-capture agent or the first type 99 log record received for the UOR that is
in recovery
v The IMS log record suffix of the first log record recorded by the change-capture
agent or the first type 99 log record for the oldest UOR in existence when the
UOR that is in recovery was created

If the restart point is associated with a UOR, you need the IMS UOR identifier.

Although multiple UORs might be in-flight when an agent goes into recovery, the
restart point is recorded for the UOR that is the oldest. To illustrate how restart

Chapter 4. IMS operations 45


points progress forward in time, the following figure shows a DB/DC or DBCTL
subsystem that has four UORs that are being tracked.

Time

UOR 1

UOR 2

UOR 3

UOR 4

Restart Point
UOR Time Line

Figure 2. Restart Point Tracking

UOR 1 has a restart point: when the UOR was started. Because UOR 1 and 2
overlap in time, UOR 2’s restart point is the time UOR 1 started. Likewise, UOR 2
and 3 overlap in time and therefore UOR 3’s restart point is when UOR 2 started.
UOR 4 does not overlap so its restart point is when UOR 4 started.

The DB2 timestamp for a restart point is based on the system clock value for the
oldest living UOR. This is either the IMS log record of the first type 99 log record,
if a UOR was in-flight when the system went into recovery, or the system clock
value of the first log record recorded by the change-capture agent. The time has
millisecond resolution. The internal timestamp consists of a DB2 timestamp with a
resolution of a tenth of a second, based on the oldest living UOR system clock
value.

The IMS log suffix information is the real key to a restart point. The IMS log record
suffix consists of an 8-byte system clock value that identifies when the IMS log
record was created. Also included in the suffix is an 8-byte double word IMS log
record sequence number generated by IMS. The combination of the system clock
value and the log sequence number identify a unique IMS log record that is used
to establish the restart point.

46 DB2 II Operations Guide for Classic Event Publishing


Given a restart point, how can I identify the IMS log files required?
You can run the IMS recovery change-capture agent to request agent status
information. If an agent is reported as being in recovery, either by the content of
the restart data set or by the correlation service, the IMS recovery change-capture
agent identifies a restart point for the agent that is in recovery mode.

After you run the IMS recovery change-capture agent to get restart information,
there are three ways of determining what log files you must supply to the IMS
recovery change-capture agent to bring DB2 Information Integrator Classic Event
Publisher back into sync:
v Manually track the IMS jobs and IMS log files that were created, and supply this
information to the IMS recovery change-capture agent.
v Run a DBRC LIST.LOG ALL (or variant) report if the IMS jobs are registered with
DBRC and review the DBRC output to identify the IMS log files associated with
the IMS control region for the agents in recovery.
v Have DB2 Information Integrator Classic Event Publisher track the IMS log files
associated with an agent. When this is done the IMS recovery change-capture
agent automatically identifies the IMS log files you need when you check an
agents status.

Using either of the first two methods to identify IMS log files that are required for
recovery are very error prone (especially if one or more agents are in recovery
mode for extended periods of time) and the likelihood of missing a log file is
greater than if you let DB2 Information Integrator Classic Event Publisher track the
log files. Even using DB2 Information Integrator Classic Event Publisher log file
tracking, it is possible to miss a log file if the control file is not updated properly.

When the IMS recovery change-capture agent is recovering IMS log files, the
recovery process is fairly sophisticated. The IMS recovery change-capture agent
accepts IMS log files that were created before the restart point and automatically
discards log records created before the restart point. If multiple IMS log files are
supplied for an agent, these files are automatically sorted in system clock time
sequence so that they are processed in the correct sequence. During the sorting
process, the IMS recovery change-capture agent automatically checks for
specification of an IMS log file more than once in the control file and for IMS log
files that have overlapping system times. (The latter condition indicates that an
IMS log file for a different IMS active change-capture agent was supplied.)
Additional checks are performed for IMS batch applications to insure that IMS log
files are not supplied from a different job.

However, the IMS recovery change-capture agent cannot detect when an IMS log
file that needs to be supplied was not. If you start an IMS recovery change-capture
agent without all of the necessary log files, the correlation service is likely to detect
this condition, but by that time, the restart point might be pushed into the future
and the content of the missing log file will no longer be recoverable.

What is log file tracking?


Log file tracking allows DB2 Information Integrator Classic Event Publisher to
track the IMS log files associated with an IMS active change-capture agent. When
log file tracking is implemented, all IMS log files associated with an agent are
tracked regardless of the state of the agent.

Log file tracking is accomplished by creating a log file tracking data set. Like the
recovery data set, this is an 80-byte sequential data set. However, unlike the

Chapter 4. IMS operations 47


recovery data set, the log file tracking data set can contain multiple records. You
can control how many log entries are maintained for each agent being tracked. You
need to create this file manually.

The section “Recovery data sets” on page 44, specified a naming standard for
identifying recovery data sets. When accessing IMS log file tracking information,
you do not explicitly tell the IMS recovery change-capture agent the name of the
IMS log file tracking data set that is associated with the agent. The name of the file
that the IMS recovery change-capture agent attempts to open is:
Recovery-File-Name.LOGS

Therefore, when you define an IMS log file tracking data set, you must follow the
above naming convention.

What is stored in an IMS log file tracking data set?


The IMS log tracking utility manages the content of an IMS log file tracking data
set. The IMS log file tracking data set contains a record for each primary and
secondary IMS log file created by IMS for a given agent. The content of each
record contains the information in the following table:
Table 4. IMS log file tracking data set content
Starting
Name Offset Length Description
Log File Type 1 1 Type of IMS log file. Values are:
v Primary Log File (1)
v Secondary Log File (2)
First Log 2 16 The IMS log record suffix from the first IMS log
Record Suffix record in the log file. The IMS log record suffix
consists of an 8-byte system clock value and a
8-byte double word IMS log sequence number.
Log File Name 18 44 Fully-qualified data set name of the IMS log file.

Given the information contained in an IMS log file tracking data set, the IMS
recovery change-capture agent can identify the IMS log files that need to be
recovered for an IMS active change-capture agent that is in recovery mode.

Based on a restart point, the IMS recovery change-capture agent can determine
whether the IMS log file was created before or after the agent went into recovery
mode. Any file created after the restart point must be recovered. Also, the last log
file with a system clock value that starts before the restart point is necessary. It is
assumed that this IMS log file contains the IMS log record that the restart point is
referring to.

When reporting about IMS log files required in the recovery process, the IMS
recovery change-capture agent identifies primary IMS system log files, or archived
IMS system log files in preference to secondary IMS system or archived log files.
This behavior is modified when there is no primary IMS system or archived log
file information and only secondary IMS log file information is available.

Note: In extreme recovery situations you will need to update the content of an
IMS log file tracking data set either by running the IMS log tracking utility
manually or by physically modifying the data set’s content. For best results,
make sure IMS log files are listed in the sequence in which they were

48 DB2 II Operations Guide for Classic Event Publishing


created. Also make sure that secondary IMS log files are listed immediately
after the corresponding primary log file entry. If you do not follow these
guidelines, the IMS recovery change-capture agent will terminate and report
errors when duplicate IMS log files are supplied, based on the output
generated by the IMS recovery change-capture agent when requesting agent
status information.

How do I implement IMS log file tracking?


Implementing IMS log file tracking differs slightly between DB/DC, DBCTL, and
IMS batch environments, but in all of these environments IMS log files are defined
to the tracking system using the IMS log tracking utility.

The goal of tracking IMS log files for a particular IMS active change-capture agent
is to retain information about all IMS log files that physically exist.

The IMS log tracking utility is a job-step that you must add to an IMS DB/DC or
DBCTL subsystem’s log archive JCL to register the archived IMS log file in the IMS
log file tracking data set. For IMS batch jobs, additional job-steps must be added
after each job-step that updates IMS data that is monitored by an IMS active
change-capture agent. Member name CACIMSLT in SCACSAMP contains sample
JCL used to execute the IMS log tracking utility.

The IMS log tracking utility is command-line driven, and uses fixed DD names to
identify the primary (referenced by the CACLOG1 DD statement) and secondary
(referenced by the CACLOG2 DD statement) log files to be registered and the
name of the IMS log file tracking data set (referenced by the CACTRACK DD
statement) to be updated.

Use the command-line parameters in the following table to control the actions of
the IMS log tracking utility:
Table 5. IMS log tracking utility command-line parameters
Keyword Value Description
DIALOGS Y|N Identifies whether the IMS Log File Tracking Utility should
collect information about a secondary IMS log file.
Y The IMS control region is using dual logging.
N The IMS control region is using single logging.

The default value is ‘N.’


PARM=’DUALLOGS=N’
ECHO Y|N Identifies whether the IMS Log File Tracking Utility should
issue informational WTO messages.
Y Issue informational WTO messages.
N Do not issue informational WTO messages.

The default value is ‘Y.’


PARM=’DUALLOGS=N,ECHO=N’

Chapter 4. IMS operations 49


Table 5. IMS log tracking utility command-line parameters (continued)
Keyword Value Description
MAXLOGS Number Identifies the maximum number of IMS log file entries that are
to be maintained in the IMS log file tracking data set.
Specifying a value of 0, or not supplying a MAXLOGS value
causes the IMS Log File Tracking Utility to maintain an
unlimited number of IMS log file entries.

Normally, IMS log files have a certain lifetime associated with


them. Generally, the IMS log files are defined as generation
data sets that have a fixed number of generations that are
retained. In these situations, you should supply a MAXLOGS
value that matches the number of generations being retained.
There is no sense in the IMS Log File Tracking Utility tracking
IMS log files that no longer exist.

If you have specified that dual logging is being used for this
IMS active change-capture agent, the IMS Log File Tracking
Utility automatically doubles the MAXLOGS values supplied
by you, under the assumption that the same number of
secondary IMS log files are retained.
PARM=’DUALLOGS=N,ECHO=N,MAXFILES=5’

How do I identify agents that are in recovery?


One way to identify that an IMS active change-capture agent is in recovery mode
is to monitor the IMS control region where the agent is active and the correlation
service. The IMS active change-capture agent issues WTO messages informing you
when the agent is in recovery mode or has experienced some other error that has
caused it to go into recovery mode.

Likewise, the correlation service issues normal processing and error WTO
messages. These messages inform you of any of the following states:
v Changes are being received from an IMS active change-capture agent.
v The agent has shutdown (that is, the IMS control region where the IMS active
change-capture agent was running, has terminated).
v The IMS active change-capture agent is in recovery mode.

If you have an automated operator facility, you might be able to identify the
various WTO messages that can be issued, monitor for these, and allow your
facility to notify you when one or more IMS active change-capture agents go into
recovery mode. Alternately, your operations personnel can just be on the lookout
for these messages.

A better approach is to create a special version of the IMS recovery change-capture


agent that is used to track the status of all of the IMS active agents at your site.
This is referred to as the IMS Active Agent Status Job, and the control file is
referred to as the IMS Status Control File. You run the IMS Active Agent Status Job
periodically, and it identifies:
v Whether any of the IMS active change-capture agents are in recovery
v The restart point for any agents in recovery
v The IMS log files that are presently available for recovery, for agents that are in
recovery, using IMS log file tracking.

50 DB2 II Operations Guide for Classic Event Publishing


If there are any IMS active change-capture agents that are in recovery, a WTO
message is issued identifying how many agents are in recovery mode. The return
code from the IMS Active Agent Status Job also identifies whether you have agents
in recovery mode. The following return codes are possible:
0 No agents are in recovery mode.
1 One or more IMS active change-capture agents are in recovery mode.
8 An error condition was reported while the IMS recovery change-capture
agent was running.

See “Identifying agents in recovery mode” on page 97 for more information about
how to use the IMS recovery change-capture agent to identify agents that are in
recovery mode.

How do I tell the correlation service about unknown agents


that are in recovery mode?
An unknown agent is an IMS active change-capture agent that executed when the
correlation service was not active. Assuming that you are using recovery data sets,
the output from the IMS Active Agent Status Job allows you to identify these
agents. They will have recovery data set information and a restart point, but the
correlation service reports the agent is not in recovery mode.

For these agents, you need to create a custom input control file and run the IMS
recovery change-capture agent in “set” mode. When run in set mode, the IMS
recovery change-capture agent communicates with the correlation service to notify
the correlation service that an IMS active change-capture agent is in recovery mode
and supplies the correlation service with the restart point for the agent.

See “Putting unknown agents in recovery mode” on page 101 for more information
about how to use the IMS recovery change-capture agent to identify agents that are
in recovery mode.

How to determine whether an IMS log file is required in the


recovery process when not using IMS log file tracking
When you supply the names of multiple IMS log files during IMS log file recovery,
the IMS recovery change-capture agent potentially reads each IMS log file twice.
Before actually processing the IMS log files, the IMS recovery change-capture agent
scans each input log file to determine whether the file is actually required, to
ensure that the IMS log files are processed in the correct chronological order and
that an invalid or duplicated log file has not been specified. Assuming that no
errors are encountered during this pre-scanning phase, the IMS recovery
change-capture agent discards any IMS log files that are not required and then
processes only the required log files. This process is referred to as log file sequence
checking. The IMS recovery change-capture agent issues WTO messages indicating
when sequence checking begins, provides status messages and indicates when log
file sequence checking is completed.

Even if a single IMS log file is supplied, WTO messages are issued indicating the
IMS log file sequence checking is occurring. However, because there is only a
single log file, in this situation the IMS log file is not actually scanned, because no
effective sequence checking can be performed and the IMS log file recovery process
being almost immediately.

Chapter 4. IMS operations 51


Regardless of whether you specify a single or multiple IMS log files, when
sequence checking is completed and IMS log file recovery begins, the IMS recovery
change-capture agents attempt to locate the restart point in the log files. If the
restart point cannot be found, WTO message CACH036E is issued, indicating that
the IMS log files specified do not contain the restart point.

If DBRC is being used to track the IMS log files for the IMS active change-capture
agent that is in recovery mode, you can use the output from the DBRC LIST.LOG
ALL report to determine whether the IMS log file needs to be input into the IMS
log file recovery process. When the DBRC report has been created, determine
whether the IMS log file is required or not by browsing the DBRC LIST.LOG report
output. Then, perform one of the following actions:
v For a batch application, find the section of the report that lists data set tracking
information for all PRILOG files being tracked for the batch job. The SSID
recorded by DBRC is the batch job name.
v For a DB/DC or DBCTL subsystem (if you are planning on using archived logs
as input), find the section of the report that lists PRILOG data set tracking
information for the DB/DC or DBCTL subsystem ID (SSID) associated with the
IMS active change-capture agent that is in recovery mode.
v For a DB/DC or DBCTL subsystem (if you are planning on using online logs as
input), find the PRIOLD data set tracking information section of the report for
the DB/DC or DBCTL subsystem ID (SSID) associated with the IMS active
change-capture agent that is in recovery mode.

For example, when you are using log file tracking, if the IMS recovery
change-capture agent reports that an IMS agent named IMSD is in recovery mode
and that IMS log file CXMAINT.IMSD.SLDSP.IMS3.LOGBKUP.G1161V00 is
required for recovery, then you would use the procedure that follows to verify that
the log file is actually required or not. You can use a similar procedure to identify
IMS log files that are required in the recovery process when you choose not to use
IMS log file tracking.

Note: In this example, the name of the IMS active change-capture agent is the
same as the IMS subsystem ID. Often this is not the case. However,
generally, the IMS active change-capture agent name is prefixed with the
IMS subsystem ID. For example, it would not be uncommon for the IMS
active change-capture agent name to be IMSDCR (or IMSDCR1) when the
IMS subsystem ID is IMSD. In this case, the IMS active change-capture agent
is the name of the IMS control region address space started task name.

In addition to reporting that IMS log file


CXMAINT.IMSD.SLDSP.IMS3.LOGBKUP.G1161V00 as a required IMS log file, the
IMS recovery change-capture agent will also report restart point information for
agent IMSD. The restart point for agent IMSD is:
DB2 RESTART TIME 20020501 16341629
IMS RESTART TIME 02.121 16:34:16.2
RESTART STORE CLOCK B79054D35F0F7640
RESTART LOG SEQ. # 00000000-00140149

After you run the DBRC LIST.LOG ALL report and find the PRILOG section of the
report for SSID=IMSD, you would see that
CXMAINT.IMSD.SLDSP.IMS3.LOGBKUP.G1161V00 is listed as one of the files
being tracked by DBRC and that the following information is in the report for the
log file:

52 DB2 II Operations Guide for Classic Event Publishing


DSN=CXMAINT.IMSD.SLDSP.IMS3.LOGBKUP.G1161V00 UNIT=3390
START = 02.121 16:33:40.7 FIRST DS LSN= 0000000000140121
STOP = 02.121 16:38:30.5 LAST DS LSN= 000000000014BF10
FILE SEQ=0001 #VOLUMES=0001

VOLSER=CXA008 STOPTIME = 02.121 16:38:30.5


CKPTCT=2 CHKPT ID = 02.121 16:38:19.7

The DBRC report identifies that the IMS log file was created on 02.121 at
16:33:40.7, which is why the IMS recovery change-capture agent reported it as a
required IMS log file. The report also identifies the file was closed at 02.121 at
16:38:30.5 and that the log file contains IMS log record sequence numbers
140121-14BF10 (in hexadecimal). Based on the restart point identified by the IMS
recovery change-capture agent, this IMS log file is definitely required in the IMS
log file recovery process because the restart time falls within the log file’s start or
stop times, and the restart log record sequence number falls within the range of the
IMS log records contained in the file.

If the IMS log file start or stop time is before the restart time reported by the IMS
recovery change-capture agent, then the IMS log file is not required in the IMS log
file recovery process. All IMS log files created after the restart point must also be
input into the IMS log file recovery process.

What is the IMS log file recovery process?


You use the IMS log file recovery process to push the restart point farther into the
future for one or more IMS active change-capture agents that are in recovery
mode. How complicated the IMS log file recovery process is, and how long it takes
to complete depends on three factors:
v The complexity of the recovery situation
v The length of time that the agents have been in recovery mode
v Whether the IMS control region (for an agent that is in recovery mode) is still
active or whether a quiesce point has been reached

The IMS recovery change-capture agent supports running in an incremental


recovery mode, which is required when multiple IMS control regions are in
recovery mode and the IMS control regions are still active. In this mode of
operation, IMS log file recovery can be performed multiple times as online log files
are archived. In each execution of the IMS recovery change-capture agent, one of
the IMS control regions is identified as the controlling IMS subsystem, and
processing is halted when all of the supplied IMS log files have been processed for
the controlling IMS subsystem. In this mode of operation, the controlling region is
flip-flopped based on the availability of IMS archived log files from the IMS
control regions that are in recovery mode.

Note: IMS log file recovery does not require the use of archived online log files if
the original online log (primary or secondary) has not been over written
(due to log roll-over). If IMS log file recovery is performed quickly enough
and the online logs are still available the recovery process is generally more
efficient because typically the archived logs are written to tape, which is
slower to process than the DASD online logs. Additionally, the IMS recovery
change-capture agent is capable of processing an active online log during
IMS log file recovery. The IMS recovery change-capture agent detects when
it has reached the current ″logical″ end-of-file and stops the IMS log file
recovery process when this condition is detected. Because it depends upon
how long an active change-capture agent has been in recovery mode and the

Chapter 4. IMS operations 53


frequency at which IMS online log file switches occur at your site, the
following examples describe the use of archived log file in the IMS log file
recovery process.

See the topic ″Recovering log files″ in Appendix A for more information about how
to use the IMS recovery change-capture agent to recover lost changes.

The diagrams in the following sections show three recovery situations that you
might encounter.
v “Single DB/DC or DBCTL subsystem in recovery mode”
v “Single DB/DC or DBCTL subsystem with IMS batch jobs In recovery mode” on
page 56
v “An incremental IMS log file recovery situation” on page 58

Although there are many other more-complex recovery situations, these three
examples represent the most common recovery scenarios with all others just being
a variation on one of these three themes.

Single DB/DC or DBCTL subsystem in recovery mode


The following diagram shows a simple IMS DB2 Information Integrator Classic
Event Publisher implementation. In this example, a single IMS DB/DC or DBCTL
subsystem is being monitored. The diagram shows that while the subsystem was
executing, six online log files were used. The diagram also shows that while using
the third online log file, the IMS active change-capture agent went into recovery
mode. Assume that the agent went into recovery mode due to an XM queue
overrun.

Note: XM queue overrun indicates that the amount of storage allocated for the XM
queue is too small and needs to be increased. In this situation, the
correlation service needs to be stopped and the correlation service SIE
definition needs to be changed to increase the size of the XM queue. In
particular, if an XM queue overrun occurs from the activity of an active
change-capture agent, the size of the XM queue needs to be increased
because the IMS recovery change-capture agent sends larger XM datagrams
than the active change-capture agent does and at a faster rate, when the
restart point has been reached in the IMS log files. Experience has shown
that you can use the THROTTLE parameter to slow down the IMS recovery
change-capture agent somewhat. However, in this type of recovery situation,
a larger XM queue is required. When any form of XM failure occurs, the
correlation service must be shutdown and restarted to clear up the XM
problem. When the correlation service is restarted ensure that it is warm
started so that the recovery process can be completed.

The diagram also shows that the IMS log archive utility has run for each of the
online IMS log files, and therefore six archived log files are available to be used in
the IMS log file recovery process. Although not shown, assume that the Log
Archive Utility job has been modified to include the use of the IMS Log File
Tracking Utility. Therefore, the IMS recovery change-capture agent knows about all
six archived log files.

54 DB2 II Operations Guide for Classic Event Publishing


Figure 3. IMS DB2 Information Integrator Classic Event Publisher Implementation

In this situation, when the IMS recovery change-capture agent is asked for
recovery status information about this agent, the IMS agent is going to report that
archived IMS log files A-3, A-4, A-5, and A-6 are required. Also assume that it was
not known that the agent was in recovery mode until after the IMS subsystem had
been shutdown.

In this situation, the IMS log file recovery process can be completed in one
operation by supplying the names of archived IMS log files A-3 through A-6.
When these IMS log files have been successfully processed by the IMS recovery
change-capture agent, the recovery process is complete and the agent can be placed
back into active mode, and then the IMS DB/DC or DBCTL subsystem can be
restarted.

Another variation on this theme is when you know that the agent went into
recovery mode and you want to start the IMS log file recovery process
immediately. In this situation, the IMS recovery change-capture agent would be
executed, requesting recovery status information for the agent that is in recovery.
The IMS recovery change-capture agent is going to report a restart point that is
located somewhere in the third IMS online log file.

If the Log Archive Utility has not been run or has not been completed when the
IMS recovery change-capture agent is executed requesting recovery status
information, then the second archived log file is identified as an IMS log file that is
needed for recovery. Running a DBRC LIST.LOG ALL report and locating the entry
for the second archived IMS log file will identify that the log file was closed before
the restart point reported by the IMS recovery change-capture agent. At this point
in time, IMS log file recovery needs to use the online version of the log O-3 as
input.

Regardless of whether you decide to perform IMS log file recovery with the active
online log or to wait a while until after the IMS Log Archive job for online log file
3 has completed, when the IMS recovery change-capture agent is re-executed and
requests recovery status information for the agent that is in recovery, the IMS

Chapter 4. IMS operations 55


recovery change-capture agent is going to report that archived IMS log 3 is
required. You could re-run the DBRC LIST.LOG ALL report and verify that archived
IMS log 3 is in fact the first file that needs to be input into the IMS log file
recovery process, or you can skip this step because (from knowing that archived
IMS log 2 does not contain the restart point) you know that this is the IMS log file
you need to start with.

When the third online log file has been archived, you can run the IMS recovery
change-capture agent in IMS log file recovery mode, supplying the name of the
IMS archived log 3. You then wait until the fourth online log file is archived. When
the fourth log is archived, you can run the IMS recovery change-capture agent and
request recovery status for the agent that you know is in recovery. Alternately, you
can perform IMS log file recovery using the active logs to keep pushing the restart
point further into the future as changes are accumulating in the active log file.

At this point in time, the IMS recovery change-capture agent is going to report that
archived log file 3 is a required IMS log file and that archived IMS log 4 is
required. When performing IMS log file recovery as archived IMS logs become
available from a DB/DC or DBCTL subsystem, you will almost always need to
input the IMS log file before the first IMS log file has been recovered because
usually, when a log file switch occurs, one or more UOR’s will be in-flight. As
shown in Figure 2 on page 46, an agent’s restart point is the first type 99 data
capture IMS log record for the oldest living UOR when tracking begins for a new
UOR. This almost guarantees that the restart point will exist somewhere in the
previous log file when a switch occurs.

Note: If you have IMS applications that run for extended periods of time that do
not issue frequent checkpoints, you might find that the restart point is
further in the past than you think. When you are feeding IMS logs into the
IMS log file recovery process as archived logs become available, run the IMS
Active Agent Status Job each time after the archive job completes to
determine the new restart point, if any.

After the sixth log file is archived (in this example, when the DB/DC or DBCTL
subsystem is shut down), then after archived log files 5 and 6 have been recovered,
the agent can be taken out of recovery mode and it is then safe to restart the
DB/DC or DBCTL subsystem.

Single DB/DC or DBCTL subsystem with IMS batch jobs In


recovery mode
In this example, there was a DB/DC or DBCTL subsystem that had been active
while two batch jobs were executed without a correlation service running. All three
of these agents are now in recovery mode, although the correlation service does
not know about this.

56 DB2 II Operations Guide for Classic Event Publishing


Figure 4. DB2 Information Integrator Classic Event Publisher Implementation with DB/DC or
DBCTL

After you start the correlation service, you can begin the recovery process for the
three IMS active change-capture agents that are in recovery. In this example,
assume that each of these agents has a recovery data set and has implemented IMS
log file tracking.

You begin the process by running the IMS Active Agent Status Job (see page 50).
The output will identify restart points for all three agents and will also indicate
that the correlation service does not know about these three IMS active
change-capture agents. The IMS recovery change-capture agent will also report on
the required IMS log files to be recovered.

The second step in the recovery process is to create a custom input control file that
is used to “set” the three agents into recovery mode. When you have run the IMS
recovery change-capture agent supplying the “set” input control file as input, the
correlation service knows that these agents are in recovery mode. Part of the
output from the “set” operation is the list of IMS log files that must be recovered
for each agent.

For the DB/DC or DBCTL subsystem, archive IMS log files A-2 through A-5 are
identified as required IMS log files. Likewise, for the two batch jobs, IMS log file
BL-1 and BL-2 are identified as potentially-necessary IMS log files.

If you run a DBRC LIST.LOG ALL report and find an entry for archive IMS log file
A-1, you will see that its creation date is right around the restart point for the
DB/DC or DBCTL subsystem. Likewise, if the two batch jobs are under DBRC
control, the creation dates for IMS log files BL-1 and BL-2 are around the restart
points for these two IMS active change-capture agents. Input all of the IMS log
files identified by the IMS recovery change-capture agent into the IMS log file
recovery process.

The easiest way to recovery these three agents is to create a custom input control
file, in which you specify that you are performing IMS log file recovery and
identify the names of all IMS log files reported by the IMS recovery change-capture

Chapter 4. IMS operations 57


agent. In this example, you would supply seven control cards, five for the DB/DC
or DBCTL region identifying the names of archive log files A-1 through A-6 and a
control card for each batch agent and their associated system log files (BL-1 and
BL-2).

After these IMS log files are successfully processed, DB2 Information Integrator
Classic Event Publisher will be synchronized with IMS, and the agents can be
moved from recovery mode into active mode. Assuming the correlation service
remains running, then the next time the DB/DC or DBCTL subsystem is started,
the IMS batch jobs are run again, or both, changes will be captured.

A second approach to this recovery situation is to recovery the three agents


separately. Although the IMS recovery change-capture agent allows you to do this,
depending on your data-sharing environment this might not be a wise move.
Regardless of the data-sharing environment, the five archived IMS log files
associated with the DB/DC or DBCTL subsystem can be recovered independently,
because no other IMS applications were running while the subsystem was
operational.

If the two IMS batch jobs updated different IMS databases, then each of these
agents’ IMS log files can be recovered independently. However, if these batch jobs
updated a shared database then these agents should be recovered together.

If the two IMS batch jobs updated the same IMS database record and these agents
are recovered independently, you run the risk of propagating updates
out-of-sequence. If the IMS batch jobs updated different IMS database records in
the shared database, then the agents can be recovered independently, but it is
difficult and time-consuming to determine whether an IMS application that
updated a shared database updated the same database record that another IMS
application updated.

IMS active change-capture agents and the correlation service do not know if you
are in a data-sharing environment. If you are in a data-sharing environment and an
IMS active change-capture agent goes into recovery mode, immediately stop the
correlation service, which will force any other IMS active change-capture agents to
go into recovery mode and will automatically place any new IMS applications into
recovery mode if they are run while the correlation service is down.

An incremental IMS log file recovery situation


The following example shows two DB/DC or DBCTL subsystems that are being
monitored for change capture. To simplify the diagram, the online log files are not
shown, nor are the execution of the IMS Log Archive Utility. The different sizes of
the archive log files indicates they are closed at different times based on system
activity. This example also assumes that these control regions are using recovery
data sets and have implemented IMS log file tracking.

58 DB2 II Operations Guide for Classic Event Publishing


Incremental Recovery Points

1 2 3 4 5 6 7 8 9 10

DB/DC or DBCTL Subsystem 1

A1-1 A1-2 A1-3 A1-4 A1-5 A1-6

DB/DC or DBCTL Subsystem 2

A2-1 A2-2 A2-3 A2-4 A2-5 A2-6

Time
Figure 5. Incremental Recovery Points

To further simplify this example, assume that the correlation service was not
running when these two IMS subsystems were started. The diagram shows that
there are ten different points where incremental IMS log file recovery can be
performed. In this situation, incremental recovery can be performed only after an
IMS online log file has been archived and only when there are IMS log files from
both subsystems that have overlapping creation dates or times.

You begin the recovery process by starting the correlation service and then running
the IMS Active Agent Status Job (see page 50). In this example, almost immediately
after starting these two subsystems, it was noticed that the correlation service was
not running, and it was started before the A2-1 archive log was created.

If this is the first time these IMS subsystems have been started after the IMS active
change-capture agent has been installed and IMS log file tracking has been
implemented, the IMS recovery change-capture agent will not identify any IMS log
files as candidates for the IMS log file recovery process. If this is not the first time
these subsystems have been executed after installing DB2 Information Integrator
Classic Event Publisher, then the IMS recovery change-capture agent will report the
last IMS archive log file created from the previous execution as being the required
IMS log file.

In the latter case, if you run a DBRC LIST.LOG ALL report, you will find that these
log files were closed before the restart point reported by the IMS recovery
change-capture agent. Until files A1-1 and A2-1 have been archived, incremental
recovery is not possible.

After these log files are created and you run the IMS Active Agent Status Job
again, then log files A1-1 and A2-1 are reported as being required IMS log files.

To perform incremental recovery for archived log files A1-1 and A2-1, create a
custom control input file and identify that you want to perform IMS log file
Chapter 4. IMS operations 59
recovery for file A1-1 and incremental recovery for file A2-1. In this case the
second subsystem is the controlling subsystem because its log file was closed
before the first subsystem’s log file.

The second incremental recovery point is reached when archive file A1-2 is created.
After file A1-2 is created and you run the IMS Active Agent Status job again, the
first subsystem files A1-1 and A1-2 are identified as required log files. Likewise, for
the second subsystem, file A2-1 is reported as a required IMS log file.

Running the DBRC LIST.LOG ALL report and checking out the creation and close
dates or times for the required IMS log files will verify that the IMS logs that you
can use for incremental recovery are A1-1, A1-2 and A2-1. There is a slight
problem, because archive file A2-2 has not been created yet, archive log A2-1 is still
the controlling subsystem and running incremental recovery at this point in time
does not push either restart points farther into the future.

Incremental recovery point three is the next point where the restart points can be
pushed into the future. When archive log A2-2 has been created then you can
perform incremental recovery. The input log files for the first subsystem are A1-1
and A1-2 and for the second subsystem A2-1 and A2-2. Because A1-2 was closed
before A2-2 the first subsystem is the controlling subsystem.

Incremental recovery point four is also another point in time where incremental
recovery does not buy you anything. Like incremental recovery point two, the first
subsystem is still the controlling subsystem and the restart points for neither
subsystem can be pushed any further into the future until archive log A1-3 has
been created.

The next incremental recovery point is number five where archive log A2-3 is the
controlling subsystems ending log file. Likewise incremental recovery point six is
another “null” point because A2-3 is still the controlling subsystems ending log file
and the recovery points cannot be pushed any further into the future.

Incremental recovery points seven, eight, nine, and ten are points where effective
incremental recovery can be performed to push the restart points forward in time.
Incremental recovery point ten is interesting because both archive files were closed
at the same time. You can either identify both subsystems as being the controlling
subsystem or just perform non-incremental IMS log file recovery. However, after
IMS log file recovery has been performed at incremental recovery point ten, these
two subsystems are still in recovery mode.

In reality, it is highly unlikely that two different archived log files were closed at
the exact same time. Use the following procedures each time you want to perform
incremental recovery to ensure that the correct IMS log files are supplied and the
correct controlling subsystem is identified:
1. Run the IMS Active Agent Status Job to identify the current restart point and
the list of IMS log files that are required.
2. Run a DBRC LIST.LOG ALL report and look up the create and close dates for
each IMS log file identified by the IMS recovery change-capture agent.
3. Eliminate IMS log files that were closed before the current restart point.
4. Identify the controlling subsystem. This is the subsystem whose last log file has
the earliest close time.

60 DB2 II Operations Guide for Classic Event Publishing


How do I get an agent out of recovery mode?
After you have one or more agents that are ready to be moved from recovery
mode, use the IMS recovery change-capture agent to perform this operation. Create
a custom input control file and supply “activate” control cards for each agent that
you want to place back in active mode. For each agent, identify the agent name
and the name of its associated recovery data set.

When the IMS recovery change-capture agent is run in this mode, the agent
informs the correlation service that it is no longer in recovery mode and has been
shutdown, thus putting the agent back into active mode. The IMS recovery
change-capture agent also deletes the contents of the recovery data set. If you
perform this operation and the change-capture agent is for a DB/DC or DBCTL
subsystem and the IMS control region is running, then the IMS recovery
change-capture agent receives an error when it attempts to delete the contents of
the recovery data set. In this instance, the IMS recovery change-capture agent will
end with a return code of 4 and in the job output you will see message LSCX872
(″File in use by another job″). The agent is nevertheless in active mode.

Note: If there is data in the recovery data set, the data set has not been deleted.
Additionally, the existence of a record in the recovery data set indicates that
IMS was started without a correlation service being active. After you have
activate the change-capture agent, it is no longer in recovery mode. If you
get into a situation where the correlation service is not active and you
shutdown and restart IMS, the information in the recovery data set is not
updated and the ″old″ restart point is still recorded. If subsequently you
cold start the correlation service, you are in a recovery situation and you
will need to manually edit the contents of the recovery data set to establish
the correct restart point. See “How do I manually create a recovery data
set?” on page 62 for details on the format of the information that is stored in
the recovery data set and for guidelines for identifying restart point
information.

See “Returning agents back to active mode” on page 107 for more information
about how to use the IMS recovery change-capture agent for instructions about
how to get an agent out of recovery mode.

When is it safe to place an agent back in active mode?


Use the following rules to determine when an IMS active change-capture agent can
be removed from recovery mode and placed back into active mode:
v The IMS control region for the failed agent is not active, or the databases being
monitored have been stopped so that changes cannot occur, or you have reached
a ″logical″ quiesce point and you know that changes to the monitored databases
will not occur.
v The IMS log file recovery process must be completed for each IMS log file (or
archived log files, for DB/DC or DBCTL subsystems) created by the IMS control
region from the failure point up to and including the last IMS log file created by
the IMS control region where the agent failed.

If an IMS active change-capture agent that is in recovery mode meets the above
requirements and you have a situation where multiple IMS active change-capture
agents are in recovery mode, leave all agents in recovery mode until all agents
meet the above requirements. Doing so lessens the likelihood of propagating
changes in the wrong chronological order.

Chapter 4. IMS operations 61


How do I manually create a recovery data set?
If you forget to update one of your IMS jobs or started task procedures to supply
the name of a recovery data set, and the IMS job or started task was started
without a correlation service being active, you can still recovery this agent if you
know the date and time that the IMS job or started task was started. Typically, the
IMS active change-capture agent will have issued a WTO message when it detected
that no correlation service was running. So, one way to figure out when the IMS
job or started task was started is to search the system log for DB2 Information
Integrator Classic Event Publisher WTO messages that begin with a prefix of CAC.

Another way to identify when the IMS job or started task was executed is to
search the system log for the IMS job or started task name to determine when the
IMS job or started task was started. Yet a third approach is to identify the creation
date or time of the IMS log files associated with the IMS job or started task.

You need to know the names of these IMS log files anyway, for recovery to be
possible. Presumably, because you forgot to update the JCL to specify the name of
the recovery data set, it is generally a safe assumption that the JCL also was not
updated to implement IMS log file tracking.

How you determine when the IMS job or started task was started and identify the
IMS log files that are required is up to you. Assuming you have this information,
the first step in recovering this agent is to create a recovery data set for the agent
and then identify the restart point.

Allocate an 80-byte fixed length physical sequential recovery data set. The block
size can also be defined as 80-bytes and you can take the minimum allocation size
of 1 block. No secondary extents are needed because this file contains only a single
record.

After you create the file, you need to manually edit the file’s contents using ISPF
to define the restart point. Insert one line into the recovery data set. The content
and format of the recovery record is identified in the table below.

62 DB2 II Operations Guide for Classic Event Publishing


Table 6. Recovery data set metadata
Starting
Name Offset Length Description
Agent Type 1 4 Identifies the type of agent that is being
recovered. Set this to
IMS_

.
Agent Name 5 12 Name of the IMS job or started task that you
want to recovery. The name must be padded with
trailing blanks.
DB2 Time Stamp 17 10 Approximate restart time in DB2 timestamp
format.
IMS UOR 27 16 The IMS unit-of-recovery identifier for a
Identifier committed UOR that was being processed when
the agent went into recovery. If the UOR was not
committed at the time the agent went into
recovery, this field contains null values (binary
zeros).
Current UOR 43 8 Binary, double-word IMS log sequence number
Starting Log for the first type 99 data capture log record
Record Sequence associated with the UOR identified by the IMS
Number UOR Identifier field, or, if the IMS UOR Identifier
is null, the IMS log sequence number of the first
IMS log record in the file when the IMS
subsystem was started.
Current® UOR 51 8 Binary, system clock value from the IMS log
Starting System record suffix for the first type 99 data capture log
Clock record associated with the UOR identified by the
IMS UOR Identifier field, or, if the IMS UOR
Identifier is null, the system clock value from the
log record suffix of the first IMS log record in the
file when the IMS subsystem was started.
Oldest UOR Log 59 8 Binary, double-word IMS log sequence number
Record Sequence for the first type 99 data capture log record for
Number the UOR that is the ″oldest″ UOR when the UOR
identified by the IMS UOR Identifier field started.
If the UOR Identifier is null, then the value that
must be supplied is the same value that is
contained in the Current UOR Starting Log
Record Sequence Number field.
Oldest UOR 67 8 Binary, system clock value from the IMS log
System Clock record suffix for the first type 99 data capture log
record for the ″oldest″ UOR when the UOR
identified by the IMS UOR Identifier field started.
If the UOR Identifier is null, then the value that
must be supplied is the same value that is
contained in the Current UOR Starting System
Clock field.

Following is the content of a recovery data set for agent DBCD. The information is
displayed in hexadecimal format with column numbers turned on.

Chapter 4. IMS operations 63


****** ***************************** Top of Data ******************************

------------------------------------------------------------------------------
=COLS> ----+----1----+----2----+----3----+----4----+----5----+----6----+----7--
----+----F----+----F----+----F----+----F----+----F----+----F----+----F--
+----1----+----2----+----3----+----4----+----5----+----6----+----7--
------------------------------------------------------------------------------
000001 IMS_DBCD ?h? ?](???? ?](????
CDE6CCCC4444444420000336840000000000000000000000CFB4E9D414000000CFB4E9D414000000
942D42340000000004618915820000000000000000000000EFBDAFF810000000EFBDAFF810000000
------------------------------------------------------------------------------
****** **************************** Bottom of Data ****************************

Specifying the agent type and agent name is pretty straightforward. However,
specifying the correct approximate restart point in a DB2 local timestamp format is
more difficult. The DB2 timestamp in the above example specifies the following
date or time:
2004/06/01 08:39:31.6.5.8.8.4.2

Which is June 1, 2004 at 08:39:31 AM local time, the trailing time resolution is in:
v Tenths of a second--6
v Hundredths of a second--5
v Thousandths of a second--8
v Ten thousandths of a second--8
v Hundred thousandths of a second--4
v Microseconds--2

In this kind of a recovery situation, you do not need to be as precise as the IMS
active change-capture agent is in identifying the restart point. Somehow you have
identified the IMS log files that the IMS job or start task created, which you will
feed into the IMS log file recovery process, after you get this agent into recovery
mode.

Regardless of how you determined the IMS log files, you must identify an
approximate restart point that is before the creation date or time of the first IMS
log file created by the IMS job or started task. You need to supply only the
following DB2 timestamp components:
v Century and year within the century
v Month of the year
v Day of the month
v Hour of the day, based on a 24-hour clock
v Minute of the hour
v Seconds of the hour

All other DB2 timestamp components should be zeros. Using the earlier example,
you would identify the approximate restart point in hexadecimal DB2 timestamp
format as:
20040601083931000000

The absolutely critical information that you must supply when manually creating a
recovery data set is contained in the Oldest UOR Log Record Sequence Number
and Oldest UOR Log Record Sequence Number fields. You need to identify the

64 DB2 II Operations Guide for Classic Event Publishing


first IMS log file created by the IMS region that you are specifying information for
and view or edit that IMS log files contents. You are interested only in the first
record in the file.

Place your editor into hexadecimal mode and scroll to the right until you reach the
end of the first log record. The IMS log record suffix consists of the last 16-bytes of
the log record. In the IMS log record, the suffix consists of the System Clock value
followed by the IMS Log Record Sequence Number. Below is the hexadecimal
representation of the IMS Log Record Suffix that corresponds to the information
stored in the recovery data set for this example:
B4E9D414000000CF
BDAFF810000000EF

The easiest way to transfer this information to the recovery data set is to copy and
paste the information from the IMS log file into the recovery data set.

After you have identified the required information, save the recovery data set and
start the correlation service if it is not running. When the correlation service is
ready, then to perform recovery for this agent:
1. Run the IMS recovery change-capture agent to “set” the agent in recovery
mode.
2. Run the IMS recovery change-capture agent in IMS log file recovery mode,
supplying all of the IMS log files associated with this agent.
3. Run the IMS recovery change-capture agent and “activate” the agent to take it
out of recovery mode

Note: If you have multiple agents that are in recovery mode and they you are in a
data sharing environment, ensure that you perform IMS log file recovery
concurrently for all agents that were active at the time of failure.

When is recovery not possible?


There are four conditions in which you have an IMS active change-capture agent
that is in recovery mode but it is impossible to complete the recovery process.
These include when:
v There is a media failure for an IMS log file that are needed in the IMS log file
recovery process and dual logging was not in effect, or both log files have media
problems.
v The IMS log or archived log files required for IMS log file recovery processing
no longer exist. This condition should occur only when you defer the recovery
process for an extended period of time.
v Using archived log files in the recovery process and the IMS Log Archive Utility
has supplied an SLDS control statement that removed log records of interest to
DB2 Information Integrator Classic Event Publisher from the archived log file.
v An IMS log file contains a type 99 Data Capture Log record that is marked “in
error.” According to the IMS documentation, this situation should occur only
when cascade delete information is being recorded.

Practically speaking, the first condition should never occur, nor the second
condition, assuming that operations personnel are monitoring DB2 Information
Integrator Classic Event Publisher’s operations. One way or another, WTO
messages will be issued by the IMS active change-capture agents, the correlation
service, or both when a problem is encountered. Assuming a recovery situation has

Chapter 4. IMS operations 65


been identified, begin the recovery process as soon as possible. This ensures that
the IMS log files required still exist and helps reduce the time agent remains in
recovery mode.

Recommendations on how to eliminate the other two failure reasons are discussed
in the sections that follow.

When is recovery not required?


If you have a known or unknown agent that is in recovery mode, then recovery is
not required if the IMS applications for the agents that failed did not update any
monitored databases. There are several methods that you can use to determine
whether the agents updated a monitored database. The recommended approach is
to use member name CACIMSLA in SCACSAMP to determine whether the IMS
log files associated with the IMS agent that is in recovery updated any monitored
databases.

CACIMSLA executes the IMS File Select and Formatting Print Utility (DFSERA10).
The input control cards search the IMS log files for type 99 data capture log
records. If a data capture log record exists in the IMS log files its content is printed
and execution of the utility is terminated.

If you update CACIMSLA to reference the IMS log files for the IMS active
change-capture agent that is in recovery mode and a data capture log record is
found, then recovery is required for this agent. If no data capture log records
exists, then recovery is not required.

In the latter case, assuming the correlation service is active (if not start the
correlation service), if the correlation service knows the IMS active change-capture
agent is in recovery mode, then all you need to do is “activate” the IMS agent that
is in recovery mode using the IMS recovery change-capture agent.

If the correlation service does not report the IMS agent in recovery mode, then run
the IMS recovery change-capture agent to “set” the agent in recovery mode. Then
run the IMS recovery change-capture agent again to “activate” the IMS agent. By
performing these operations the IMS agent will not go into recovery mode
(because the contents of the restart data set have been cleared) the next time that
IMS control region is executed.

What are IMS log records of interest?


Like IMS, DB2 Information Integrator Classic Event Publisher uses a variety of
types and sub-types of IMS log records to track the state of the IMS control region
being monitored, the synchpoint log records used to track UORs that are in-flight
and whether they are committed or rolled back, and the actual type 99 Data
Capture Log records that are used to capture changes. Many of the IMS log records
used by DB2 Information Integrator Classic Event Publisher are the same ones
used by IMS for its own recovery processes.

The IMS Log Archive Utility allows you to identify records that are not to be
recorded in the archive versions of the IMS logs. The IBM documentation states
that if you attempt to suppress an IMS record type used in IMS recovery that the
IMS Log Archive Utility will not operate.

66 DB2 II Operations Guide for Classic Event Publishing


Note: If you are running the IMS Log Archive Utility using the standard defaults,
all of the required types of IMS log records are retained, so no action is
required on your part.

To ensure that IMS DB2 Information Integrator Classic Event Publisher recovery
processing is possible using archived IMS log files, do not eliminate the IMS log
records identified in the following table from the archived log files.
Table 7. IMS log records
Record Sub-
Type types Description and Purpose
06 All IMS/VS Accounting Record--used to track the state of IMS batch
applications and to detect and handle IMS emergency restart
situations for a DB/DC or DBCTL subsystem.
37 All QBLK synchpoint Log Record--used to identify the start of a UOR
for a BMP application using extended checkpoints, or as the start
of Phase II synchpoint processing for a DRA application.
38 All SMB Application Abnormal Termination Log Record--used to
identify that a UOR has been rolled back.
41 All Batch Checkpoint Log Record--used to identify the end of a UOR
for a batch application that issues checkpoints.
59 37, 38 IMS/VS/FP Sync Point Log Record--for Fast Path applications
used to determine whether a UOR is being committed or rolled
back.
99 All Data Capture Log Record--used to capture the changes that
occurred to a monitored database. Also used to determine whether
an IMS batch application terminated abnormally.

What are cascade deletes?


Data Capture log records can contain information about deleted child segments.
IBM refers to this information as a cascade delete. In these situations a single
logical Data Capture log record is created that contains elements about the segment
that was deleted and all of its children. The Data Capture log record can span
multiple physical IMS log records.

The IMS documentation states that in cascade delete situations, the Data Capture
log record is constructed in virtual storage. If enough virtual storage cannot be
allocated to build the log record, it is marked as being “in error” and information
is lost.

The correlation service refuses to attempt to process Data Capture log records
marked “in error,” and if one is encountered, the correlation service reports that it
has encountered one of these records and immediately terminates processing. This
forces all IMS active change-capture agents to go into recovery mode. This is a
non-recoverable error condition and you cannot perform IMS log file recovery to
correct the situation. Your only option is to cold start the correlation service and
resume processing changes after the IMS type 99 log records. The changes made by
the application that issued the cascade delete and generated the ″in error″ data
capture log record are lost.

One way to eliminate this situation is to not capture cascade delete information. In
this scenario, the application that is processing the XML messages off of the queue
contains logic to take into account and deal with any child segments that might

Chapter 4. IMS operations 67


have been deleted when a parent segment delete is read off the queue. This
approach works well in situations like replication where the information captured
for the deleted segment contains enough information, in the form of keys, to allow
the application to delete rows in the associated child segment tables. This can be
done if the IMS database is in third-normal form.

A second approach that reduces the risk of creating a Data Capture log record
marked “in error” is to capture only concatenated key information for the deleted
child segments. This reduces the amount of information captured for each deleted
child segment and overall reduces the size of the Data Capture log record, making
it more likely that sufficient virtual storage can be obtained to construct the Data
Capture log record. This approach is viable if the IMS database is in third-normal
form.

Note: Only specify data capture and in particular cascade delete options on
segments whose contents you are interested in capturing changes for. For
example, if you have a three level database where the hierarchy is:

Root

Child1

Child2

If you do not care about changes to the contents of the Child2 segment, including
cascade delete notifications, then you should supply the following EXIT parameter
for Child2:
EXIT=(*,NOKEY,NODATA,NOPATH,(NOCASCADE),NOLOG)

A middle-of-the-road approach is to capture concatenated key information and


image data for the deleted child segments. This approach can be taken when the
IMS database is not in third-normal form and a column in the deleted child
segment contains some kind of a key that, in conjunction with the concatenated
key information, allows the application to identify the child segments that have
been deleted. This generates a larger Data Capture log record and you run the risk
of IMS marking the Data Capture log record “in error” when a large number of
child segments have been deleted and the contents of the deleted segments cannot
be compressed.

The capture options that generate the largest Data Capture log records is when you
capture segment data for the child segments that are deleted, as well as segment
data for the child’s parent segments. This is required only when you have a very
poorly-designed IMS database.

You have to deal with this situation only if you are monitoring an IMS database
that contains large database records and deletes of a database record are allowed at
your site. If these kinds of database records exist at your site, you must decide
how you want to handle these cascade delete situations so that DB2 Information

68 DB2 II Operations Guide for Classic Event Publishing


Integrator Classic Event Publisher IMS change capture does not fail. Remember,
Data Capture log records marked “in error” represent a non-recoverable situation.

What is an XM queue overrun?


The section “Functionality” on page 38, explained that the IMS active
change-capture agent uses XM data grams to send uncommitted Data Capture and
synchpoint log records to the correlation service for processing. The IMS recovery
change-capture agent also uses XM data grams to communicate with the
correlation service.

XM data grams are stored in an asynchronous queue. Each time the IMS active
change-capture agent is called with an IMS log buffer, it posts a single XM data
gram on the queue and exits. The correlation service picks messages up off of the
XM queue when it is not busy doing other work.

The IMS recovery change-capture agent operates in a similar fashion when


performing IMS log file recovery. However, in this mode, if a single agent is being
recovered, an XM data gram is posted when a 32-kilobyte buffer is filled up with
IMS log records of interest. If IMS log files for multiple agents are being recovered,
this behavior is modified and an XM data gram is posted when:
v A 32-kilobyte buffer is filled up with IMS log records of interest.
v A buffer is truncated due to system clock overlaps with another agent.

By default, the size of the XM queue allocated is 8-megabytes. On the correlation


service’s SIE entry, you can change the size of the queue to be any value between 1
and 256-megabytes. See DB2 Information Integrator Getting Started with Classic Event
Publishing for more information about modifying this setting.

For a DB/DC subsystem that is primarily processing transactions, the default 8


megabytes of queue space should be sufficient to handle the traffic between the
IMS active change-capture agent and the correlation service. Likewise, for a
DBCTL subsystem that is primarily processing DRA “transactions” (such as CICS
or DB2 Information Integrator Classic Federation), the default setting should be
adequate for normal operations.

There are some conditions under which the default XM queue size might not be
adequate, and might need to be increased. If you have a BMP or batch job that
performs a large number of updates to databases being monitored, and does not
issue frequent checkpoints, then large amounts of data are written to the message
store, which slows down the correlation service’s overall responsiveness. When a
checkpoint is issued, the checkpoint causes the UOR to be committed and the
correlation service will begin sending the data changes made by the UOR to the
publication service for processing.

By default, the correlation service will send all changes associated with a
committed UOR to the publication service, and wait for confirmation, before
checking the XM queue for more work. If you have applications that fit this
profile, increasing the size of the XM queue gives the system more breathing room
and helps reduce the possibility that the XM queue will fill up, at which point in
time the system must go into recovery mode. This situation would usually affect
all IMS active change-capture agents, as well as any IMS recovery change-capture
agents that are active when this situation occurs.

Chapter 4. IMS operations 69


To help alleviate problems with UOR’s that contain a large number of changes, the
correlation service allows you to specify an interrupt value that allows the
correlation service to interleave work between sending changes to the publication
service and checking for more work from active and recovery agents. See the
following section, “How does the interrupt value work?” for more information.

Another situation that would require that you increase the size of the XM queue is
one in which you have multiple IMS active change-capture agents monitoring
different IMS control regions in a high-volume environment communicating with
the same correlation service. All of the agents are sharing the same XM queue, and
it is possible that a queue overrun can occur if the correlation service is busy
processing committed UOR’s or is in the process of analyzing another agent’s XM
data gram, and cannot retrieve the data grams fast enough to empty the queue.

The IMS recovery change-capture agent is much more likely to encounter this
situation due to the fact that the IMS recovery change-capture agent is directly
reading the IMS log files and does not have to wait to be invoked by IMS. In
addition to increasing the size of the XM queue, setting an appropriate interrupt
value, or both, you can also specify a throttle value that causes the IMS recovery
change-capture agent to slow down and reduces the risk of an XM queue overrun.
See “How does throttling work?” on page 71, for more information.

There is no rule of thumb for determining the optimum size of the XM queue for
your site. It depends on how your applications are written and how active your
IMS control regions are. You need knowledge of the applications that are being
monitored to determine whether they generate large UOR’s containing many
changes, or whether the majority of the work being monitored is transaction based.
Likewise, you should have a good idea of how active your system is.

The best advice is to thoroughly test DB2 Information Integrator Classic Event
Publisher in your IMS test environment and attempt to simulate what will be
experienced in the production environment so that you can determine an optimum
XM queue size.

How does the interrupt value work?


The default behavior of the correlation service when it receives a committed UOR
is to send all changes to the publication service for processing. Each segment that
has been deleted, inserted, or updated by the IMS application causes one message
to be sent to the publication service for processing. If you have many changes in a
committed UOR, it could take some time before the correlation service is done
processing the UOR’s content and retrieves any more work from the IMS Active or
recovery change-capture agents in the XM queue.

The correlation service INT SIE parameter () can be used to change the default
commit processing behavior. Specifying a non-zero INT value identifies how many
changes in a committed UOR are sent to the publication service before the
correlation service interrupts commit processing and checks the XM queue to see if
there are outstanding XM data grams to be processed. See DB2 Information
Integrator Getting Started with Classic Event Publishing for more information about
the INT parameter.

Increasing the size of the XM queue and supplying an appropriate interrupt value
are two methods that you use to eliminate agents’ going into recovery due to an
XM queue overrun. There is no rule on what an appropriate interrupt value is. If

70 DB2 II Operations Guide for Classic Event Publishing


you have applications that generate a large number of updates, experiment with
this parameter setting to determine an optimum value.

How does throttling work?


In an IMS active change-capture agent, XM data grams are posted immediately on
the XM queue when the IMS active change-capture agent is called with an IMS log
buffer. The IMS active change-capture agent is called synchronously by IMS and
must return control as soon as possible to reduce the impact the agent has on your
IMS operations.

This is not the case with an IMS recovery change-capture agent, because it is
processing batch system log files or archived online log files. Naturally, you want
the IMS recovery process to be completed as quickly as possible, however, it is
permissible for the IMS recovery change-capture agent to “ping” the correlation
service at intervals to determine whether it is responding, or busy doing other
work. Periodically checking to see how the correlation service is doing is called
throttling. Throttling reduces the risk of an IMS recovery change-capture agent
getting into an XM queue overrun situation.

By default, the IMS recovery change-capture agent automatically sends a throttle


message to the correlation service after 128 buffers have been sent to the
correlation service for an IMS active change-capture agent that is in recovery. This
count is maintained on a per-agent basis.

If you are recovering a single agent, the buffers sent to the correlation service are
probably very large. The IMS recovery change-capture agent uses 32-kilobyte
buffers, and after no more IMS log records of interest can fit in the buffer, the XM
data gram is sent to the correlation service. This means that around 4 megabytes of
data grams will be posted in the XM queue before the IMS recovery
change-capture agent sends a throttle message to the correlation service to see if it
is still responding. If the correlation service does not respond within the TIMEOUT
value that you specify as an IMS recovery change-capture agent command line
parameter, the IMS recovery change-capture agent issues an error message and
terminates processing. The default time out value is 5 minutes.

In situations in which you are performing incremental recovery or recovering


multiple agents that were executing concurrently, you might want to increase the
throttling interval. In these kinds of recovery situations, the IMS recovery
change-capture agent is performing IMS system clock time stamp merging for all
of the agents that have been identified in the IMS log file recovery process. This
process begins by identifying the oldest agent and that agent’s XM buffer is filled
up until either:
v No more IMS log records can fit into the buffer.
v A IMS log record for a different agent is encountered with a younger system
clock value.

In the worst-case scenario, the XM data gram sent to the correlation service
contains only a single IMS log record. More typical situations will send some XM
data grams containing a large number of IMS log records, with occasional smaller
XM data grams containing only a few IMS log records. In these situations, you
might want to increase the throttle count so that more XM data grams are sent
before the IMS recovery change-capture agent checks to see if the correlation
service is still responding.

Chapter 4. IMS operations 71


How do I install the IMS active change-capture agent?
The IMS active change-capture agent is implemented as the IMS Logger Exit. The
IMS Logger Exit has the hard-coded name DFSFLGX0. IMS automatically invokes
the IMS Logger Exit during IMS log file initialization processing, if a module
named DFSFLGX0 is found in the STEPLIB concatenation, or the link-pack area.

One of the easiest ways to install the IMS active change-capture agent is to copy
module DFSFLGX0 from the Classic Event Publisher distribution libraries into the
IMS RESLIB. Another method is to concatenate the Classic Event Publisher load
library into your IMS batch jobs and started task procedures for the online DB/DC
or DBCTL regions.

Each approach has its advantages and disadvantages. You must decide which
approach you take based upon how pervasive the use of IMS DB2 Information
Integrator Classic Event Publisher will be at your site.

If you are implementing a large-scale deployment of IMS DB2 Information


Integrator Classic Event Publisher, then placing the IMS Logger Exit in the IMS
RESLIB is the easiest installation method. You have a large-scale deployment when
either:
v You are planning to augment the majority of your IMS databases for change
capture.
v You are augmenting an IMS database for change capture that is updated by the
majority of your IMS applications.

If you are planning a smaller-scale IMS DB2 Information Integrator Classic Event
Publisher implementation that monitors only a small number of IMS databases
updated by a small number of IMS applications, then you might want to modify
these IMS batch jobs and the DB/DC or DBCTL subsystems started task
procedures to reference the Classic Event Publisher-authorized load library.

In a large-scale deployment, you need to update each IMS batch job and DB/DC
or DBCTL subsystem’s started task JCL to include a recovery data set, and
optionally install IMS Log File Tracking. See “What is log file tracking?” on page
47, for more information about Log File Tracking.

In a small-scale implementation, there are fewer IMS batch jobs and started task
procedures that need to be updated, but if you forget to update one of your IMS
applications that updates a monitored database, these changes are lost and the
correlation service will have no knowledge that this has occurred.

If you install the IMS active change-capture agent in the IMS RESLIB and are
performing only a small-scale implementation of IMS DB2 Information Integrator
Classic Event Publisher, the correlation service still tracks all IMS control regions
that are referencing the IMS RESLIB where the IMS active change-capture agent is
installed, even though many of these IMS applications do not update databases
that are being monitored by DB2 Information Integrator Classic Event Publisher.
Likewise, if these IMS active change-capture agents go into recovery mode, you
have to recovery these failed agents using the techniques discussed earlier, even
though no IMS changes are being captured, which makes more work for you.

What if I already have an IMS logger exit implemented?


The IMS Logger Exit is somewhat of an esoteric IMS system exit. IBM does not
supply a sample for this exit, so normally this exit will not exist at your IMS site.

72 DB2 II Operations Guide for Classic Event Publishing


In case you have implemented your own IMS Logger Exit, or are using a
third-party exit, the version supplied with DB2 Information Integrator Classic
Event Publisher does contain support for invoking an existing IMS Logger Exit.

You can use the SCACSAMP member CACIMSLX to relink the IMS Logger Exit.
Modify the JCL to include a reference to the load library where the existing
DFSFLGX0 exit resides on the SYSLIB DD statement. If you chain an existing
DFSFLGX0 exit with the Classic Event Publisher version, the existing exit is
invoked after the IMS active change-capture agent has completed its processing.

How do I activate change capture for an IMS database or


segment?
IMS DB2 Information Integrator Classic Event Publisher change capture is
implemented using two IMS features. The IMS active change-capture agent is
implemented as an IMS Logger Exit. This allows the IMS system to be monitored
for changes in near real-time.

Although IMS performs its recovery processing based on the normal content of the
IMS log files, IMS DB2 Information Integrator Classic Event Publisher does not use
the “raw” log records that IMS uses to capture changes. IMS DB2 Information
Integrator Classic Event Publisher uses IMS synchpoint log records to track the
state of an in-flight UOR.

Instead of using the type 50 (undo or redo) and other low-level change notification
records that IMS uses for recovery purposes, DB2 Information Integrator Classic
Event Publisher uses type 99 Data Capture log records to identify changes to a
monitored IMS database. These records contain more information and are easier to
deal with than the “raw” recovery records used by IMS. In this respect, DB2
Information Integrator Classic Event Publisher is like the IBM DPropNR product
that is used to replicate IMS changes into DB2 databases. DPropNR had IBM
modify IMS to generate the Data Capture log records for asynchronous
propagation because the IMS log records that were required for IMS recovery did
not contain enough information for DPropNR to successfully replicate IMS changes
into DB2. DB2 Information Integrator Classic Event Publisher is no different in this
respect.

Generation of Data Capture log records is done at a database or segment level, and
requires augmentation of your DBD definitions. This augmentation does not affect
the physical database definition. It adds additional information to the DBD and
ACB load module control blocks.

Augmentation consists of adding the EXIT= keyword parameter on the DBD or


individual SEMG statements in you DBD definition. You can supply default
capture values at the DBD level and override or suppress generation of change
capture information altogether at the SEGM level.

After you have augmented your DBD, you run a DBDGEN for the updated DBD
and then run the ACBGEN utility and update all PSBs that reference the DBD. You
can then put the updated DBD and PSB members into your production ACB
libraries. If you perform this augmentation using the IMS Online Change facility,
the correlation service will eventually go into recovery, because the correlation
service also needs access to the augmented DBD load module. Recovery will occur
when the correlation service detects changes from the newer version of the DBD
that the applications are updating when the online change is completed. As part of

Chapter 4. IMS operations 73


Classic Event Publisher customization, you update the correlation service’s JCL and
add a DBDLIB statement that references your DBD load modules that are being
monitored for changes.

Presently, the correlation service gathers information about the DBDs being
monitored for changes only during initialization processing. You cannot add a new
DBD to be monitored while the correlation service is running, or modify the
capture options for a database that is being monitored for changes.

There is a second and potentially more serious problem with attempting to


augment a DBD while the correlation service is running. As mentioned above,
information about the DBDs being monitored by the correlation service is obtained
during correlation service initialization processing. For this discussion, the
information of interest is the DBD VERSION information associated with the DBD
and the EXIT data capture options for each of the segments in the DBD.

When the correlation service processes the contents of a UOR, for each Data
Capture log record encountered, it compares the DBD VERSION information
contained in the Data Capture log record with the VERSION information obtained
from the DBD load module. If the VERSION information does not match, the
correlation service goes into recovery mode. The correlation service will also go
into recovery mode if the data capture options contained in the Data Capture log
record do not match the data capture options defined in the DBD.

Note: If you have DPropNR installed at your site, any DBDs that have been
augmented for DPropNR’s use must also be defined in the DB2 Information
Integrator Classic Event Publisher metadata catalog.

If you augment a DBD for data capture using Online Change and then an
application updates it without recycling the correlation service, then the correlation
service does not know about the DBD. Likewise, if you change the data capture
options for an existing DBD and then an application updates the DBD, the
correlation service will go into recovery due to either a VERSION mismatch or
because the data capture options do not match. These are recoverable situations
after you synchronize the correlation service’s DBD information with IMS.

During initialization processing, the correlation service needs access to the


augmented DBD load module. The correlation service JCL must be modified to
include a DBDLIB DD statement that references the DBD library where the
augmented DBD resides.

The need to keep the load library that the correlation service references in sync
with the IMS DBD library introduces an additional point of failure. One common
failure is that the correlation service goes into recovery when processing a
committed UOR due to a discrepancy in the DBD VERSION contained in the Data
Capture log records and the VERSION information of the DBD referenced by the
correlation service.

By default, the VERSION identifier is the date and time that the DBD was
assembled. If you use default VERSION identifiers and some other group
re-assembles the augmented DBD and does not re-cycle the correlation service,
then when a Data Capture log record containing the new VERSION identifier is
received, the correlation service will go into recovery mode and terminate IMS
processing.

74 DB2 II Operations Guide for Classic Event Publishing


One way to reduce this possibility is to specify an explicit VERSION identifier in
the form of a string, such as the name of the DBD or application. Using explicit
VERSION identifiers reduces the possibility of the correlation service going into
recovery mode due to DBD VERSION mismatches.

How do I augment a DBD for change capture?


IMS was enhanced to generate IMS Data Capture log records for use by IBM’s
DPropNR for asynchronous replication purposes. Internally, IMS uses type 50 undo
or redo log records and others for its own recovery purposes. Unfortunately, these
log records do not contain enough information to be effectively used for change
capture purposes. IBM realized this and enhanced IMS to generate IMS Data
Capture log records that do contain all of the information necessary.

To have IMS generate IMS Data Capture records, the DBD for which information
needs to be captured must be augmented to specify what information you want
captured. These DBD modification affect only the actual DBD definition (stored in
the DBD/ACB library) and do not affect the physical database.

You use the EXIT= keyword to specify IMS Data Capture information. The EXIT
keyword is supported for the DBD control statement and the SEGM control
statement. Supplying an EXIT keyword on the DBD statement defines default
values for all segments in the DBD. Specifying an EXIT keyword on the SEGM
statement allows you to override the default values. This gives you great flexibility
about the types and amounts of information that is captured.

The format of the EXIT keyword is:


EXIT=(Exit-Name,KEY|NOKEY,DATA|NODATA,PATH|NOPATH,
(CASCADE, KEY|NOKEY,DATA|NODATA,PATH|NOPATH),
LOG|NOLOG)

The following table identifies the use of each of these parameters, keywords, or
keyword pairs.
Table 8. Parameter, keyword, keyword pair formats
Keyword Purpose
Exit-Name This parameter is used to specify the name of a
DPropNR synchronous data capture exit, or the fact that
there is no exit (by specifying *), or NONE to deactivate
an exit routine on a SEGM statement.

IBM does not use data capture exits, but co-exists if


your site is using DPropNR or you have implemented
your own exits at your site. If you do not have any data
capture exits, you should specify * for the Exit-Name
parameter.
KEY|NOKEY Identifies whether you want the IMS Data Capture log
records to contain physical path concatenated key
information for a segment that has been deleted,
inserted or updated. The default is KEY.
DATA|NODATA Identifies whether you want physical segment data
included in the IMS Data Capture log records for a
segment that has been deleted, inserted or updated. The
default is DATA.

Chapter 4. IMS operations 75


Table 8. Parameter, keyword, keyword pair formats (continued)
Keyword Purpose
PATH|NOPATH Identifies whether you want physical segment data
included in the IMS Data Capture log records for the
parents of a segment that has been deleted, inserted or
updated. The default is NOPATH.
CASCADE|NOCASCADE Identifies whether you want IMS Data Capture log
records to contain cascade delete information for a
segment that has been deleted that has child segments.
When CASCADE is specified, the next three sets of
keywords can be specified to identify what kind of
information you want included in the IMS Data Capture
log record for the deleted child segments.
KEY|NOKEY Identifies whether you want the child segment’s
concatenated key information included in the IMS Data
Capture log record. The default is KEY.
DATA|NODATA Identifies whether you want the physical segment data
included in the IMS Data Capture log record for the
deleted child segment. The default is DATA.
PATH|NOPATH Identifies whether you want physical segment data
included in the IMS Data Capture log records for the
parents of the child segment that has been deleted. The
default is NOPATH.
LOG|NOLOG Identifies whether you want IMS Data Capture log
records to be generated to the IMS log files. If an * is
specified for Exit-Name the default is LOG. You can
specify NOLOG on individual SEGM statements, if
these segments do not contain data that is to be
captured. Alternately, you can specify NOKEY,
NODATA for segments that you do not want data
captured for.

In addition to specifying EXIT information in the DBD, you can also supply
VERSION information about the DBD control statement. IBM recommends that
you allow IMS to generate the default DBD version identifier, which is the date or
time the DBD was assembled.

When the correlation service, during commit processing, processes a IMS Data
Capture record it compares the version information contained in the record against
the version information in the DBD load module. If the version information does
not match, the correlation service logs an error message and terminates. By using
the date or time stamp of DBD assembly you are pretty much guaranteed that the
DBDs that IMS is accessing are the same ones that the correlation service is using
for reference purposes.

As you can see, you have a great deal of flexibility about how much and what
kinds of data you want IMS to place in a IMS Data Capture log record. Also as
you might already suspect, the more information you request the larger
performance impact it will have on your IMS system. Some of these impact are
discussed in the next topic.

76 DB2 II Operations Guide for Classic Event Publishing


What are the recommended nonrelational-to-relational
mappings to use?
This section presents general guidelines for efficient use of IMS DB2 Information
Integrator Classic Event Publisher monitoring of your IMS environment. The
easiest way is to discuss this in relational terms. From an IMS perspective, this is a
root-only database. Things get more complicated when an IMS database record has
children and when you factor in whether the IMS database is designed using third
normal form (TNF) and the foreign key information is available from concatenated
key information, or whether foreign key information is buried in a parent
segment’s data.

These topics are discussed in the following sections.

Create columns only for the information that you really need
Typically, when you have DB2 Information Integrator Classic Event Publisher
installed you will also have Classic Federation installed and most likely will
already have one or more tables defined for the IMS DBD that has been
augmented for change capture. Past experience shows that typical mappings of an
IMS database for retrieval purposes contains column definitions for most, if not all,
of the data in the segments that are referenced by the table.

For retrieval purposes this is not a problem. But for update purposes, you should
define “custom” update versions of your tables that contain column definitions
only for the columns to be updated. In update situations, there are restrictions on
what can be inserted or updated that generally require you to create custom
“update” versions of your tables.

Likewise for DB2 Information Integrator Classic Event Publisher change capture,
create custom “data capture” tables that are mapped over the IMS database being
monitored for change capture. In these data capture tables you should only define
columns for the data that changes in your segments that you are actually interested
in capturing. If you only create the columns that you are actually interested in,
then when data changes you reduce the amount of information that is sent to the
publication service when a change does occur to the monitored database.

If you have redefinitions in your segments, then you should create multiple data
capture table definitions: one for each re-definition that exists. Ensure that each
mapping contains the “control column” that allows you to identify record type. In
these situations, when a change occurs to the database the record selection exit
(CACRCSEL) is called multiple times, once for each mapping associated with the
segment that changed. In these cases, the record selection exit should only accept
the change to the proper table mapping based on the ″control information″ that is
contained in the captured data. In the case where you have multiple redefinitions
and these redefinitions contain column mappings with different numeric data
types, you must use the record selection exit to ″filter″ the proper changes.
Otherwise, the correlation service goes into recovery mode when it detects invalid
data in the numeric columns when the data is converted from its native form into
its SQL representation.

Root-only database recommendations


For a root-only database, specify the following EXIT parameter on the DBD
statement:
EXIT=(*,NOKEY,DATA,NOPATH,(NOCASCADE),LOG)

Chapter 4. IMS operations 77


This will capture all changes to the root segment and informs IMS that path data
and cascade delete information is not required. It also does not request concatenated
key information because that will be available in the root segment’s before and
after images. Not requesting concatenated key information also reduces the size of
the IMS data capture log records that are created.

Dealing with child segments


When you are augmenting DBD’s with child segments, you must consider whether
your database is design in Third Normal Form, and how many child segments
physically exist.

Only augment and map child segments whose content you are actually interested
in. For segments that you are not interested in when they are created, updated, or
deleted specify the following EXIT parameter on that SEGM’s statement:
EXIT=(*,NOKEY,NODATA,NOPATH,(NOCASCADE),NOLOG)

When your IMS database is designed in Third Normal Form, then presumably the
foreign key information required is contained in the IMS concatenated key. In these
instances you will want to minimally code the following EXIT statement for the
child segment being monitored:
EXIT=(*,KEY,DATA,NOPATH,(?),LOG)

The question mark represent the cascade delete parameter and options that are in
effect. Recommendations on how to deal with cascade deletes is discussed in
“What are cascade deletes?” on page 67.

Ironically, your IMS database might be in Third Normal Form, but the application
to read the MQ message queues does not recognize the segment data and
concatenated key data as sufficient to create a foreign key for its own use. In these
situations you must capture path data. Minimally you must code the following
EXIT=(*,KEY,DATA,PATH,(?),LOG)

Mapping validation rules


When mapping tables against a segment that has data capture activated for a
tables leaf segment, the correlation service enforces the rules identified in the table
below:
Table 9. Mapping verification rules
Data Capture Options Leaf Segment Mapping Rules Non-leaf Segment Mapping
Rules
KEY Sequence Fields Only Sequence Fields Only
KEY,DATA Maximum Segment Length Sequence Fields Only
DATA,PATH Maximum Segment Length Maximum Segment Length
KEY,DATA,PATH* Maximum Segment Length Maximum Segment Length
DATA Maximum Segment Length No Mappings Allowed
PATH No Mappings Allowed Maximum Segment Length

Using this combination of mapping options is not recommended. You can obtain
the same information by using the DATA,PATH option. Using this option increases
the size of the Data Capture log records generated by IMS.

78 DB2 II Operations Guide for Classic Event Publishing


These rules are applied to both normal Data Capture options (for records
generated when a DLET, ISRT or REPL call is issued) and for Data Capture log
records generated for cascade deletes.

Mapping considerations and information contained within IMS


data capture log records
If you have segments that contain variable length segments and the physical
segment length is less than the maximum segment length, the trailing data
contains binary zeros.

How do I deal with IMS test and production systems on the same MVS
image?
It is not uncommon to have multiple IMS subsystems defined on the same MVS™
image, such as having test and production IMS systems on the same image. By
default, if you install IMS DB2 Information Integrator Classic Event Publisher in
both of these IMS systems, any changes in either system are forwarded to the
correlation service for processing. When IMS data in either system that has been
augmented for monitoring changes, these changes are forwarded to the publication
service where the changes are inserted into WebSphere MQ message queues. In
this situation, WebSphere MQ will contain the “real” changes from the production
IMS system as well as the changes made in the test system. The application
reading the WebSphere MQ message queues cannot tell the difference between real
and test changes.

To get around this problem, the correlation service supports the concept of named
correlation services. Using named correlation services; multiple correlation service
instances can run on the same MVS image. Each correlation service has its own
XM queue and its own copy of common memory.

To implement named correlation services, you specify the name of the correlation
service on the correlation service SIE entry, and create a customized version of the
IMS Logger Exit (DFSFLGX0) that identifies the name of the correlation service
that the IMS active change-capture agent communicates with. SCACSAMP member
name CACIMSLX is sample JCL to re-link the IMS Logger Exit after you have
customized and assembled the CACE1OPTS module to identify the correlation
service that the IMS active change-capture agent is to communicate with.

For example, if you have two different IMS systems, IMST for test and IMSP for
production, you would create two instances of the correlation service and name
them IMST and IMSP, respectively. You would them create two customized
versions of the IMS Logger Exit, DFSFLGX0. One version would use the name
IMST and the exit would be placed into the IMST RESLIB load library. The second
version, which would communicate with the IMSP correlation service, would be
placed in the IMSP RESLIB load library.

Remember that the name of the IMS Logger Exit load module, which is the IMS
active change-capture agent, is determined by IMS and is hard coded to be
DFSFLGX0. This requires that you have differently-named IMS RESLIBs, or that
you place each version of DFSFLGX0 in a different APF-authorized load library
(for DB/DC or DBCTL subsystems) and modify your IMS batch and started task
JCL to reference the correct version of DFSFLGX0.

Chapter 4. IMS operations 79


I changed some IMS data and nothing happened
When DB2 Information Integrator Classic Event Publisher for IMS is installed and
operating in an IMS test system and there is not a lot of activity in the DB/DC or
DBCTL subsystem, you can execute a transaction that updates IMS data that has
been augmented for data capture without seeing any activity in the correlation
service (if you are monitoring pending message or committed UOR counts) or in
WebSphere MQ (if you are monitoring queue statistics). There are two possible
reasons for this:
v The transaction did not change any data.
v IMS has not called the IMS Logger Exit because the IMS log buffer containing
the changes has not been written to disk.

For IMS full-function databases, when a REPL call is issued, IMS checks to see if
the contents of the segments being updated have actually changed. If not, IMS
does not generate any low-level undo or redo IMS log records nor any Data
Capture log records.

A second cause for changes not being propagated to the correlation service is that
there is not enough activity in the DB/DC or DBCTL subsystem for the IMS
Logger Exit to be called by IMS. This kind of behavior is not expected to occur in a
production IMS subsystem, but can occur on a test system, especially in a DBCTL
environment.

A way to force IMS to drive the IMS Logger Exit is to issued a simple /CHECKPOINT
command. When a simple checkpoint command is issued, if there has been any
system activity since the last time the IMS Logger Exit was called, IMS will force
the IMS log buffer to be written to disk thus causing the IMS Logger Exit to be
called and the changes to be forwarded to the correlation service for processing.

What is the effect of DB2 Information Integrator Classic Event


Publisher on my IMS system?
IMS DB2 Information Integrator Classic Event Publisher has been designed to have
a minimal impact on your IMS system. The IMS active change-capture agent is
designed to require minimal resources and, when called, to send the XM data gram
to the correlation service and return control to IMS as quickly as possible.

However, few things in life are free. As such, you will see the following changes in
your IMS operations:
v The more information you ask for, the more CPU time it takes IMS to synthesize
the IMS Data Capture records for each delete, insert, and update.
v For each insert, update, and delete call that you issue for a segment (or path of
segments) for which you have IMS Data Capture activated, an additional log
record is generated and written to the IMS log file.
The size of the segments, and in particular whether or not you have requested
path data, determines how large these log records are. For each DL/I call issued,
a single logical log record is generated. Based on the size of the log file buffer
size, IMS might have to write multiple physical records to the log file.
IBM has attempted to reduce the size of these log records by using ESA
compression services if they are available. The actual content of your data
determines how effective compression is.
v The size of the IMS log files will increase because more log records are being
written to the log files. You might have to change the allocation sizes for the log

80 DB2 II Operations Guide for Classic Event Publishing


files. For DB/DC, DBCTL, or DCCTL environments, the online log will be
archived more frequently. Because more log records are being written, system
level checkpoints occur more frequently, depending on the number of DBDs
augmented and how frequently they are updated.

Note: If you have IMS DB2 Information Integrator Classic Event Publisher
installed in a batch database load job, then you need to drastically
increase the size of the IMS log file used by the load job. Typically, the
IMS log file for a batch load job is quite small and does not contain any
IMS log records for the individual segments that were added to the
database. However, with DB2 Information Integrator Classic Event
Publisher installed and when the database being loaded has data capture
active, the IMS log file will now contain a type 99 data capture record for
each segment (or path of segments) inserted into the database.
v The IMS Logger Exit is invoked more often. The IBM IMS Logger Exit is
designed to be as “thin” as possible. Obviously, the more physical IMS Data
Capture log records are generated, the more times the IMS Logger Exit is called
and the more XM data grams will be sent to the correlation service. Also note
that even when you have no DBDs augmented for data capture, the IMS Logger
Exit will be sending non-type 99 log records to the correlation service for
correlation purposes for all in-flight UORs. Because the IMS Logger Exit does
not know whether a particular UOR contains any IMS Data Capture records
within it, the exit still sends these other types of log records to the correlation
service in case the UOR does contain updates to a database that has been
augmented for change capture.

Are my IMS operating procedures affected by DB2 Information


Integrator Classic Event Publisher?
Yes. You need to modify your IMS operating procedures to:
v Make sure that the correlation service is started before a DB/DC or DBCTL
subsystem is started and before any IMS batch job, with IMS DB2 Information
Integrator Classic Event Publisher installed, is executed.
v Make sure that a correlation service is not terminated while a monitored IMS
DB/DC or DBCTL subsystem is shut down, or while an IMS batch job, with DB2
Information Integrator Classic Event Publisher installed, is still executing.

Although the second rule is not technically part of the IMS operating procedures,
you might want to update those procedures to include this rule in addition to the
IMS DB2 Information Integrator Classic Event Publisher operating procedures that
you need to develop.

Do I need to create DB2 Information Integrator Classic Event


Publisher operating procedures?
You should develop a set of IMS DB2 Information Integrator Classic Event
Publisher operating procedures to cover normal operating procedures: starting the
correlation service before any IMS jobs or started tasks that have IMS active
change-capture agents installed are executed, and stopping the correlation service
after all IMS control regions have terminated.

You also need to develop procedures to monitor DB2 Information Integrator


Classic Event Publisher’s health and develop problem-determination procedures
and recovery procedures for when something fails. These problem-determination
and recovery procedures need to deal with the failure of DB2 Information
Integrator Classic Event Publisher components and for the applications that are
Chapter 4. IMS operations 81
processing the events that have been staged to the WebSphere MQ queues that are
being populated by the publication service.

Develop an implementation plan for your IMS DB2 Information Integrator Classic
Event Publisher system. This plan would identify whether you want a large-scale
or small-scale IMS DB2 Information Integrator Classic Event Publisher deployment,
how you want to name the IMS active change-capture agents, how many IMS
active change-capture agents you want to run, how you want to name the recovery
data sets, and whether you want to implement IMS Log File Tracking. See “What
is log file tracking?” on page 47 for more information about Log File Tracking.

As part of this implementation plan:


v Identify the names of the DBDs and segments to be monitored.
v Specify the information that will be collected in the Data Capture log records.
v Create a checklist of steps for:
– Augmenting each of your DBDs
– Making these DBD load modules accessible to the correlation service
– Coordinating the changes to the DBDs
– Installing these augmented DBDs into your IMS environment
– Restarting your correlation service

Start by identifying the DBDs that you want to monitor. You can then identify the
DB/DC or DBCTL applications and batch applications that update these databases,
based on the PSBs that contain update PROCOPTs for these databases. When you
know how many, if any, batch jobs are affected, you can make an informed
decision on whether to perform a large or small scale deployment, and then
further develop your plan for setting up your DB2 Information Integrator Classic
Event Publisher system.

If your site consists of a single DB/DC or DBCTL subsystem and all of your batch
applications have been converted to access or update your IMS data as BMP’s, you
will find that setting up your DB2 Information Integrator Classic Event Publisher
system is very easy and straightforward. If you have multiple DB/DC or DBCL
subsystems in a data-sharing environment or many IMS batch update applications,
then things become more difficult.

Ideally, you would develop an implementation plan prior to installing and


implementing IMS DB2 Information Integrator Classic Event Publisher in your IMS
test environment. However, system availability for an IMS test system is not as
critical as it is for your production environments, and it might be acceptable to
perform DB2 Information Integrator Classic Event Publisher installation and
implementation in multiple steps.

The same is not true for when you plan to implement DB2 Information Integrator
Classic Event Publisher in your IMS production environment. You want your IMS
subsystems to be unavailable for the shortest periods of time, so it is critical that
you develop an implementation plan so that DB2 Information Integrator Classic
Event Publisher can be installed and activated in the shortest possible time.

If you have a fairly complex deployment affecting numerous databases, IMS batch
jobs, started tasks, or any combination of the three, test out your implementation
plan against your IMS test system before implementing DB2 Information Integrator
Classic Event Publisher in your IMS production environment.

82 DB2 II Operations Guide for Classic Event Publishing


What DB2 Information Integrator Classic Event Publisher monitoring
capabilities are available for IMS?
During normal operations, the correlation service issues WTO messages when an
IMS active or recovery change-capture agent initially connects to the server.
Likewise, the correlation service issues a WTO message when an agent ends, and
IMS active change-capture agents issue them when they go into recovery mode.

IMS-specific WTO messages are also issued when an IMS active or recovery
change-capture agent terminates. These messages identify the agent that ended,
providing high-level summary information about the agent’s interaction with the
correlation service. They report any in-flight UORs that were being tracked by the
correlation service when it terminated. If the an IMS active change-capture agent
terminated abnormally, then additional messages are issued notifying you that an
abnormal termination was detected and the actions that the correlation service
took. These actions can include automatic roll back of an in-flight UOR for an IMS
batch application that abended.

When the correlation service is shutdown, IMS-specific summary WTO messages


are issued that identify the number of IMS active and recovery change-capture
agents that were serviced by the correlation service, and provides high-level
summary information about these agents’ interaction with the correlation service.
WTO messages are issued for any IMS active or recovery change-capture agents
that were still connected to the correlation service at termination, and for in-flight
UORs that were being tracked for these agents at correlation service termination.

If IMS change-capture agents were active when the correlation service was
shutdown or abnormally terminated, this indicates that you were either already in
a recovery situation, or shortly will enter a recovery situation when the IMS active
or recovery change-capture agents attempt to communicate with the correlation
service.

When an IMS agent terminates, or the correlation service is shutdown and UORs
are in-flight or have been discarded, information about these UORs is also written
to the system log. The correlation service automatically generates IMS event
metrics for the DBD and individual segments that have been augmented for IMS
change capture and are defined in the metadata catalog that the correlation service
references. These metrics include the number and types of events for committed
UORs that have been sent to the publication service for processing. The metric
information is always maintained, and is written to the system log when the
correlation service is shutdown and the trace level is set to four or less.

The correlation service also supports two levels of tracing that you might need to
activate in a test environment if you are experiencing problems with IMS DB2
Information Integrator Classic Event Publisher change capture. Setting the trace
level at two generates messages to the system log that:
v Identify the tables being monitored for change capture.
v Identify the start and stop of an IMS active or recovery change-capture agent.
v Provide details about the interaction of the correlation service with the message
store and the publication service.
v Detail tracking of UORs, which includes:
– The start of a UOR
– The end of a UOR and its outcome (committed or rolled back)

Chapter 4. IMS operations 83


– For committed UORs that contained changes, their forwarding to the
publication service and subsequent confirmation by the publication service

Setting the trace level to one generates all of these tracing messages, as well as
dumping the content of the headers in the XM data grams and the full content of
the individual IMS log records contained in the data grams.

The primary problem with using tracing is that to get to the information, you must
stop the correlation service. If there are IMS active change-capture agents
communicating with the correlation service, at shutdown, these agents will almost
immediately go into recovery mode.

84 DB2 II Operations Guide for Classic Event Publishing


Chapter 5. VSAM operations
This chapter covers:
v “Introduction” on page 85
v “Determining prerequisites” on page 85
v “Mapping application data” on page 85
v “Change capture” on page 85
v “Moving from recovery to active mode with VSAM” on page 86
v “Starting a CICS VSAM change-capture agent” on page 86
v “Configuring and starting a CICS VSAM change-capture agent” on page 87
v “Mapping CICS VSAM data” on page 89

Introduction
This chapter describes the tasks that you must perform before going live with your
DB2 Information Integrator Classic Event Publisher implementation. The
information is presented in the order in which you should perform it. The actual
steps that you perform are covered in the DB2 Information Integrator Operations
Guide for Classic Event Publishing.

Determining prerequisites
Depending on what you plan to use DB2 Information Integrator Classic Event
Publisher for, there might be steps that you will need to complete before going
live. The following is one of the steps you might need to perform.

Initial synchronization. If you are going to use DB2 Information Integrator Classic
Event Publisher to maintain synchronization between two or more databases, the
databases must start out already synchronized.

Mapping application data


The data mapping process involves gathering information about your existing
databases and assigning SQL names to the data. The information is stored in the
DB2 Information Integrator Classic Event Publisher metadata catalog.

For more information about using the Data Mapper, see the Data Mapper Guide for
Classic Federation and Classic Event Publishing.

Change capture
The VSAM change-capture agent runs in the same address space as the correlation
service. It is activated by uncommenting the SERVICE INFO ENTRY for
CACECA1V in the correlation service’s configuration file and specifying options on
field 10. These options include which CICS to monitor, the log streams to read, and
the starting point.

This agent reads system and user log streams where CICS writes file recovery
information and sends it to the correlation service. RECOVERY ALL must be

© Copyright IBM Corp. 2003, 2004 85


specified on the CICS file definition of the files and the retention period of the log
streams should be considered to allow for recovery operations.

Recovery with VSAM


If, for any reason, the change-capture agent or the correlation service has an error
in processing, the change-capture agent goes into recovery mode. A starting point
is written in the correlation service’s restart data store, so the recovery
change-capture agent can locate a starting position in the log stream files.

For VSAM, the same task serves as both active and recovery change-capture agent.
When the change-capture agent starts, it will determine if it is in recovery mode or
active mode. If it is in recovery mode, it will query the correlation service for the
restart point and start reading the log streams at that point. When the end of the
log streams are reached, it will set itself to active mode.

The retention period and AUTODELETE specifications of the system, user, and log
of log streams should be set so that the data will remain in the log stream for the
longest period of recovery desired. If the retention period and AUTODELETE
specifications are met, CICS purges completed units of work on the system log file
when CICS terminates.

Moving from recovery to active mode with VSAM


The VSAM change-capture agent automatically moves to active mode when it
reaches the end of file on the log streams. This could occur when CICS is shut
down or quiesced, or when activity decreases enough to allow the VSAM agent to
catch up.

Starting a CICS VSAM change-capture agent


For CICS VSAM, the change-capture agent and the correlation service reside in the
same address space. The change-capture agent reads the system and user
logstreams to gather the changed data.

The retention period and AUTODELETE specifications of the system, user, and log
of log streams should be set so that the data will remain in the log stream for the
longest period of recovery desired. If the retention period and AUTODELETE
specifications are met, CICS purges completed units of work on the system log file
when CICS terminates or does an activity keypoint (AKPFREQ SIT parameter).

Configuring CICS file definitions


You custom-configure CICS files to enable DB2 Information Integrator Classic
Event Publisher to capture the before and after images of a file. You configure
differently for SMS and non-SMS volumes.

On SMS volumes, you alter the clustername, setting it to log ALL, and specifying a
log stream name. The following is the syntax you would use:
ALTER clustername LOG(ALL) LOGSTREAMID(logstreamname)

Before starting a change-capture agent


For the DB2 Information Integrator Classic Event Publisher system to be
fully-functional, you must also start up the correlation service and publication
service. See DB2 Information Integrator Getting Started with Classic Event Publishing
for instructions on starting these servers.

86 DB2 II Operations Guide for Classic Event Publishing


Starting a change-capture agent
The CICS VSAM change-capture agent runs as part of the correlation service and is
defined by a SERVICE INFO ENTRY in the configuration file. When it starts, you
will see the following message on the system log:
CACH105I CICS VSAM CAPTURE: Vv.r.m mmddyyyy READY

This is followed by the following message, indicating the time processing began:
CACH106I START PROCESSING AT mm/dd/yyyy hh:mm:ss

Starting a recovery agent


If the change-capture agent is put into recovery mode, when it restarts, it will
automatically be put in recovery mode. Processing will begin at the time needed to
gather all the information needed when the active agent was terminated. When the
agent reads to the end of all the log stream files, it will automatically go into active
mode.

Stopping a change-capture agent


The change-capture agent will stop when the correlation service is terminated. It
can be manually started and stopped by the operator START and STOP commands
as shown below:
START,SERVICE=VSAMECA
STOP,SERVICE=VSAMECA

Configuring and starting a CICS VSAM change-capture agent


The correlation service configuration file contains SERVICE INFO ENTRY
statements to define the correlation service and the change-capture agent. The
following section describes the SERVICE INFO ENTRY for a change-capture agent
and the options associated with it.

Service info entry


Service Info Entries are used in configuration files to inform the region controller
task that a service should be activated and how that service should be controlled.

The Service Info Entry parameter consists of ten subparameters, each delimited by
at least one space. The format of the first nine of these subfields is consistent across
all services.

The following is a sample SERVICE INFO ENTRY for the change-capture agent:
SERVICE INFO ENTRY = CACECA1V VSAMECA 2 1 1 1 45M 5S \
APPLID=CICSAPPL STARTUP CICSUID.CICSAPPL.DFHLOG \
CICSUID.CICSAPPL.DFHJ01 CICSUID.CICSAPPL.DFHJ02 \
CICSD.CICSVR.DFHLGLOG
CACECA1V
Field 1 is the name of the change-capture agent load module.
VSAMECA
Field 2 is the task name of the change-capture agent. This can be any
unique value.
Service Start Class
Set the value of Field 3 to 2.

Chapter 5. VSAM operations 87


Minimum Tasks
Set the value of Field 4 to 1 unless you want to manually start the service
with an operator command, in which case you would set it to 0.
Maximum Tasks
Set the value of Field 5 to 1.
Maximum Connections
Set the value of Field 6 to 1.
Tracing Output Level
Leave the value of Field 7 set to 4 unless you are asked to change it by
IBM Technical Support for problem diagnostics.
Response Time Out
Field 8 is used when requesting information from the correlation service
for either a restart Log Sequence Number (LSN) or a throttling
confirmation. This value should be set large enough to allow the server
time to receive throttled messages. A value of 5 minutes (5M) should be
adequate in most cases.
Idle Time Out
Field 9 sets a polling frequency for recovery restart and rules confirmation
messages. A value of 5 seconds (5S) is recommended. For the
change-capture agent, this is the amount of time to wait when all of the
log streams have reached end of file. Decreasing this value will reduce the
latency in which changes are reflected to the correlation service at the
expense of increased overhead.
Service-specific information
Fields after Field 9 contain information specific to the CICS VSAM
change-capture agent.
APPLID=id
This is an optional parameter to allow selection of data from a specific
CICS. If APPLID is not specified, changes from all CICSs in the log streams
are processed. If you need to select multiple CICSs, create another
SERVICE INFO ENTRY.
START
This specifies the time to start processing for the first time the
change-capture agent is activated or when COLDSTART is specified for the
change-capture agent. Valid options are:
v STARTUP--Start processing at the time CICS was last started.
v OLDEST--Start processing at the beginning of the log stream file.
v YOUNGEST--Start processing with the next record written.
v TIME mmddyyyy hhmmss--Start processing at the indicated time
mmddyyyy is the month, day and year to start and hhmmss is the hour,
minute and second to start.

Note: The time to start processing should be set when CICS is inactive. If
a time is set to an active period, the start position could be in the
middle of a unit of work. If this happens, an error will occur or
updates will be missed.
Log streams
You list the log streams to read, separated by spaces. Specify the system

88 DB2 II Operations Guide for Classic Event Publishing


log stream and one user log stream. If you specify more than one user log
stream, you must also specify the log of logs log stream. The log streams
can be listed in any order.
COLDSTART
This keyword indicates to ignore the recovery position and start processing
at the position specified on the SERVICE INFO ENTRY for the
change-capture agent. In normal operations, this keyword should not be
required. If the change-capture agent is stopped and restarted, it will
resume processing at a position to gather all the information from when it
was last stopped.
THROTTLE=n
Specifies a throttling level for change messages when the agent is in
recovery mode and must catch up with the current activity point in the
logstream. Throttling prevents over-running the raw change data queue for
the correlation service when the recovery point in the logstream is
significantly behind the current activity point in the log. The value
specified for this parameter indicates the maximum number of messages
that will be in the raw data queue at any point in time. The default value
is THROTTLE=1024.
SERVER=name
Specifies the name of the named correlation service that you want this
change-capture agent to communicate with. For more information about
named servers, see DB2 Information Integrator Planning Guide for Classic
Event Publishing.

Mapping CICS VSAM data


Mapping CICS VSAM data includes defining logical tables which access single
records of a CICS VSAM file. To define a mapping, the Data Mapper loads a
COBOL copybook and converts record layouts into SQL column definitions.
The basic steps required to map CICS VSAM data are:
1. Transfer the COBOL copybook to the PC on which the Data Mapper is
installed.
2. Start the Data Mapper.
3. Open an existing, or create a new, Repository.
4. Select or create a Data Catalog of type CICS VSAM.
5. From the Window menu, choose List Tables.
6. From the Edit menu, choose Create a new Table.
7. Enter the table information using:
v The DD/CICS option
v CACCICS2 for the local Applid
v The desired CICS for the CICS Applid
v MTLU62 for the Logmode
v EXV2 for the CICS Transaction ID
8. From the File menu, choose Import External File to import the COBOL
copybook.
9. Delete any columns you are not interested in.
10. Close the Columns and Tables windows.
11. From the File menu, choose Generate USE Statement to generate the metadata
grammar

Chapter 5. VSAM operations 89


12. Click Yes to view the generated script.
13. Transfer the generated metadata grammar back to the mainframe.
14. Run the metadata utility to populate the metadata catalog used by the server.
TCP/IP users can transfer mainframe files between host systems and the PC
from within the Data Mapper using an FTP facility included in the Data
Mapper.

Running the metadata utility


The metadata utility requires a connection to CICS to validate the table and collect
information about it. This is accomplished using a VTAM® LU62 connection from
the metadata utility to CICS. To set up this connection, perform the steps
documented in the following sections.

VTAM resource definitions


A VTAM APPL definition is required to communicate with CICS and to create a
VTAM mode table.

Sample member CACCAPPL in the SCACSAMP data set contains sample VTAM
APPL definitions. It contains two APPL definitions. CACCICS1 is not required for
DB2 Information Integrator Classic Event Publisher, and can be removed.
CACCICS2 is used by the Metadata Utilities. The following is the sample member:
*
* SAMPLE APPL ID DEFINITIONS FOR CICS INTERFACE
*
CACCAPPL VBUILD TYPE=APPL
CACCICS1 APPLACBNAME=CACCICS1,
APPC=YES,
AUTOSES=1,
MODETAB=CACCMODE,
DLOGMOD=MTLU62,
AUTH=(ACQ),
EAS=100,PARSESS=YES,
SONSCIP=YES,
DMINWNL=0,
DMINWNR=1,
DSESLIM=100
CACCICS2 APPLACBNAME=CACCICS2,
APPC=YES,
AUTOSES=1,
MODETAB=CACCMODE,
DLOGMOD=MTLU62,
AUTH=(ACQ),
EAS=1,PARSESS=YES,
SONSCIP=YES,
DMINWNL=0,
DMINWNR=1,
DSESLIM=1

You need to create a Logon Mode Table entry. Member CACCMODE in


SCACSAMP data set contains the macro definitions to define it. Assemble and
catalog this member in VTAM’s VTAMLIB. The following is the member’s content:
CACCMODE MODETAB
MTLU62 MODEENT LOGMODE=MTLU62,
TYPE=0,
FMPROF=X’13’,
TSPROF=X’07’,
PRIPROT=X’B0’,
SECPROT=X’B0’,
COMPROT=X’D0B1’,

90 DB2 II Operations Guide for Classic Event Publishing


RUSIZES=X’8989’,
PSERVIC=X’060200000000000000000300’
MODEEND
END

CICS resource definitions


CICS SIT, transaction, program, connection, and session entries need to be added
to allow the metadata utility to communicate with CICS.

The CICS system initialization table (DFHSIT) definition or initialization overrides


must include ISC=YES to enable intercommunication programs. If this does not
already exist, add it and cycle CICS.

Copy the load modules CACCICAT from the load library to the CICS user load
library.

Install the IBM Language Environment (LE) product in CICS.

The file CACCDEF in the SCACSAMP data set contains a sample job. Add it to the
CICS transaction, program, connection, session, and file definitions required for
DB2 Information Integrator Classic Event Publisher.
To run the job:
1. Update the job card for your site specifications.
2. Update the STEPLIB for the correct CICS library.
3. Update the DFHCSD DD for the correct CSD file.
4. Remove the program definition for CACCIVS, the EXV1 transaction, the EXC1
connection, the EXS1 session and the CACEMP file definition.
5. After successful completion of the job, install the new definitions with the
following CICS transaction:
CEDA INSTALL GROUP(CACVSAM)
6. Next, add the CACVSAM group to your start-up group with the following
CICS transaction:
CEDA ADD GR(CACVSAM) LIST(xxxxxxxx)
where xxxxxxxx is the name of the start-up group from your SIT table.

Chapter 5. VSAM operations 91


92 DB2 II Operations Guide for Classic Event Publishing
Appendix A. The IMS recovery process
To successfully recover changes to IMS data that DB2 Information Integrator
Classic Event Publisher has lost, the following infrastructure needs to be in place:
1. The names of the IMS active change-capture agents that were active at the time
of failure.
2. An approximate restart date and time when the IMS active change-capture
agent failed.
3. The names of the IMS log files that are associated with the IMS Active Change
Capture Change Agents address space at, and after the point of failure up to
the present time, when no IMS Control Regions are operational.

The identification of the names of the IMS Change Capture Agents that are, or
potentially are, in recovery mode is, of course, the most important part of getting
IMS back in synchronization with DB2 Information Integrator Classic Event
Publisher. Also note that it is not possible to totally get in sync when an IMS
control region is active and DB2 Information Integrator Classic Event Publisher is
in a recovery situation.

However, for a single DB/DC or DBCTL subsystem that is in recovery mode, you
can use the IMS recovery change-capture agent to catch up with the IMS control
region, so that when you do shutdown the control region, the number of log files
that need to be recovered is kept to a minimum. For an IMS batch job that requires
DB2 Information Integrator Classic Event Publisher recovery (not necessarily IMS
recovery), DB2 Information Integrator Classic Event Publisher recovery can only be
performed after the batch job has completed executing.

The IMS recovery change-capture agent allows you to perform the following
operations:
1. Given a recovery data set name and IMS active change-capture agent name,
identify whether that IMS change-capture agent is in recovery mode due to no
active correlation service or other failure. If so, identify the approximate or
exact starting point in the IMS log files where the recovery process needs to be
initiated.
2. Place an IMS active change-capture agent in recovery mode.
3. Input one or more log files into the recovery process for an IMS active
change-capture agent that is in recovery mode.
4. Place an IMS active change-capture agent (that is in recovery mode) back into
active mode.

Generally, in a recovery situation, you will perform the 3rd step one or more times
and then, once the IMS control region is not executing and you have recovered all
of the regions log files, you will perform the last step and place the change-capture
agent back into active mode. In an DB2 Information Integrator Classic Event
Publisher recovery situation, you should always perform the first operation, which
will identify the IMS control regions that need to be recovered and an approximate
or exact starting position within the IMS log files where recovery needs to be
initiated. You only need to perform the 2nd operation when multiple IMS control
regions were active when the failure occurred – it allows you to place an IMS
Active Recovery Agent that the correlation service is not aware of into recovery
mode.

© Copyright IBM Corp. 2003, 2004 93


The DB2 Information Integrator Classic Event Publisher IMS recovery
change-capture agent runs as a batch job. First, there must be a correlation service
executing on the same MVS image (LPAR) where the IMS active change-capture
agent failed. The IMS recovery change-capture agent accepts an 80-character text
input file that identifies the operations that the agent is to perform. Recovery
requires executing the IMS recovery change-capture agent multiple times until all
IMS control regions are inactive and all IMS logs have been processed.

In the control file input you identify either the name of a recovery data set to be
checked, placed in recovery mode or placed in active mode, or the data set name
of an IMS log file to be accessed during the recovery process. These files are
dynamically allocated therefore the IMS recovery change-capture agent address
space requires read access to the IMS log files and read-write access to the recovery
data sets.

The contents of the control file identify the operations that the IMS recovery
change-capture agent is to perform. The format of the control file is identified in
the following table.
Table 10. Format of control files for IMS recovery change-capture agents
Parameter Name Starting Offset Length Description
Mode 1 1 Identifies the function
that the IMS recovery
change-capture agent
is to perform.
IMS change-capture 3 8 The name of an
agent name active change-capture
agent that you want
to:
v Check to see if it is
in recovery mode.
v Place the agent in
recovery mode.
v Perform recovery
on a log file.
v Place the agent in
active mode.
Region type 12 1 Identifies the IMS
control region type
being recovered when
an IMS log file is
supplied.
Data set name 14 44 Name of a recovery
data set to be
checked, placed in
recovery or placed
into active mode, or
the name of an IMS
log file to use in the
recovery process.

The IMS recovery change-capture agent is designed to run multiple times in order
to get synchronized with IMS. As mentioned before, ultimately, to get
synchronized you have to shut down your IMS online regions. To get you to the
synchronization goal the IMS recovery change-capture agent runs in four modes.

94 DB2 II Operations Guide for Classic Event Publishing


The Mode field in the IMS recovery change-capture agent control card file
determines the execution mode. These modes are mutually exclusive and are
identified in the following table.
Table 11. Execution modes
Mode Start description Action
C Check agent Check the recovery data set
to see if the agent is in
recovery mode. If so, identify
an initial recovery point.

The correlation service is also


accessed to see if the agent
name is currently in recovery
mode. If so a more exact
initial recovery starting point
is also identified.
S Set agent in recovery mode Given the name of a recovery
data set and a IMS
change-capture agent name,
place that agent in recovery.
The IMS change-capture
agent name must match the
name of the agent recorded
in the recovery data set.
L Log file recovery Given the name of an IMS
log file, an IMS
change-capture agent name
and the region type attempt
to move the recovery point
forward in time.
I Incremental log file recovery Perform Log File Recovery
processing and halt the Log
File Recovery process when
the last IMS log file has been
processed for this
change-capture agent.
A Place agent in active mode Remove the IMS
change-capture agent name
from recovery mode and
place it back into active
mode.
* Comment Contents are ignored.

When you are recovering from log files, then you must also identify the type if
IMS Control Region that is being recovered. The Region Type parameter is
therefore required and must be one of the values identified in the following table.

Appendix A. The IMS recovery process 95


Table 12. Region types
Keyword Value Description
REGION nnnnnnnn Defines the amount of MESSAGE POOL
storage to allocate for communications with the
correlation service and for IMS recovery
change-capture agent processing.

If this parameter is not specified the default


value is 4MB. PARM=’REGION=16000000’
THROTTLE nnnn Defines a throttling value to prevent the IMS
recovery change-capture agent from
overrunning the correlation service with change
and synchpoint log records. The throttle value
defines the maximum number of messages to
be queued to the correlation service at any
point in time. This helps ensure available space
in the message queue for communications with
the correlation service during the recovery
process.

If this value is not specified, the throttle value


defaults to 128. A value of 0 disables throttling
of messages.

The throttling only applies during IMS log file


recovery processing. The throttle value is
applied to each IMS active change-capture
agent in recovery mode for which IMS log file
have been supplied. For example, using the
default value and having two agents
performing IMS log file recovery, a throttle
message is issued for the first agent after 128
buffers have been sent to the correlation
service. Likewise, a throttle message is sent
after 128 buffers have been sent for the second
agent in recovery. PARM=’THROTTLE=512’
TIMEOUT nnM|S Defines a timeout value (in seconds or
milliseconds) when throttling is used.
Throttling relies on the correlation service
responding to requests that messages are
received. In the event that the correlation
service is unable to respond within the
specified TIMEOUT value, the IMS recovery
change-capture agent terminates with an error.

If this parameter is not supplied the default


TIMEOUT value is 5 minutes.
PARM=’THROTTLE=512,TIMEOUT=5M’
SERVER xxxxxxxx Defines the name of the correlation service to
communicate with when using Named
correlation services.

If this parameter is not supplied the default is


to communicate with an un-named correlation
service. PARM=’SERVER=IMSTEST’

96 DB2 II Operations Guide for Classic Event Publishing


When the IMS recovery change-capture agent is executed it issues a series of WTO
messages that identify the processing performed and any errors that were
encountered.

Identifying agents in recovery mode


IBM recommends that you create a special IMS recovery change-capture agent job
that is used to identify the status of each of the IMS active change-capture agents
that exist at your site. For purposes of clarification this job will be referred to as
the IMS active agent status job.

The control file for the IMS active agent status job is a more-or-less a static file that
you create once and only need to update when you add a new IMS active
change-capture agent at your site or remove an existing IMS active change-capture
agent. This file will be referred to as the IMS Status control file.

The IMS Status control file contains a control card for each of your IMS active
change-capture agents. The order of these control cards does not matter. For each
control card supply the following parameter values:
v Mode – Check Agent
v IMS Change-Capture Agent Name – The 8-character job name associated with
this agent.
v Region Type – not used.
v Data Set Name – The name of the recovery data set associated with this agent.

If you have followed our recovery data set naming conventions, then you can
easily identify the information that needs to be supplied in the IMS Status control
file.

When you execute the IMS active agent status job a series of WTO messages are
generated for each agent identified in the IMS Status control file. These WTO
messages identify whether the agent is in recovery mode and if so an initial
recovery restart point. The restart point identifies the approximate time when the
agent failed, and if the agent is known by the correlation service, possibly, the
exact position in the IMS log files where the restart operation needs to be initiated.

Identification of the agents that are in recovery mode and identification of recovery
restart points is crucial to successful recovery of DB2 Information Integrator Classic
Event Publisher without loss of changes. To illustrate how to identify agents in
recovery and identification of recovery restart points the following two examples
are provided.

In these examples, two IMS active change-capture agents exist. These happen to be
batch jobs, however, the same procedure is used for DB/DC or DBCTL regions that
are in recovery. The following table shows the IMS Status control file contents.
Table 13. IMS status control file contents
Mode Agent Data set name
C WCA008LA WCA008.XSYNC.WCA008LA
C WCA008IA WCA008.XSYNC.WCA008IA

Appendix A. The IMS recovery process 97


In the first example, job WCA008LA was run without a correlation service being
active and job WCA008LA has never been executed with change capture active.
Listed below is the output WTO messages that are issued:
(1) CACH061I RECOVERY MODE: CHECK AGENT STATUS
(2) CACH048I RECOVERY DATASET WCA008.XSYNC.WCA008LA OPENED
(3) CACH049I AGENT NAME IS ’WCA008LA’
(4) CACH039I DB2 RESTART TIME 20020409 14051629
(5) CACH040I IMS RESTART TIME 02.099 14:05:16.2
(6) CACH050I RECOVERY DATASET WCA008.XSYNC.WCA008IA REPORTS AGENT IS NOT IN RECOVERY
MODE
(7) CACH030I AGENT ’WCA008IA’ IS NOT IN RECOVERY MODE
(8) CACH062I RECOVERY PROCESSING COMPLETED SUCCESSFULLY

For purposes of discussion each of the WTO messages have been numbered.
Messages 2 through 5 are associated with job or agent WCA008LA while messages
6 and 7 are for job or agent WCA008IA. The meaning of these messages is
identified in the table below.
Table 14. Meanings of the WTO messages in the preceding sample output
Number WTO ID Meaning
1 CACH061I Identifies the mode the IMS
recovery change-capture
agent is processing. In this
example, checking agents
recovery status.
2 CACH048I Indicates that recovery data
set was opened successfully
and contains a record that
contains an approximate
recovery restart point.
3 CACH049I Identifies the name of the
active change-capture agent
that is in recovery mode.
4 CACH039I Identifies the approximate
recovery restart point date
and time in DB2 format. The
date or time that is identified
is April 8th, 2002 at 2:05:19
pm local time.
5 CACH040I Identifies the approximate
recovery restart point in the
date or time format used by
IMS in DBRC reports. The
date or time identified is the
99th day of 2002 at 2:05:09
pm and 200 th seconds local
time.
6 CACH050I Indicates that the recovery
data set is empty.
7 CACH030I Indicates that the correlation
service does not know about
this agent.

98 DB2 II Operations Guide for Classic Event Publishing


Table 14. Meanings of the WTO messages in the preceding sample output (continued)
Number WTO ID Meaning
8 CACH062I Final message issued that
identifies the overall success
or failure of the IMS recovery
change-capture agents
execution. There are three
forms of this message:
v CACH062I – Processing
completed successfully, no
errors were encountered.
v CACH062W – Processing
completed successfully,
however, one or more
agents were already in
recovery mode or already
in active mode.
v CACH062E – Fatal error
due to memory allocation
failure, failures while
communicating with the
correlation service, invalid
control card input or
invalid IMS log file input.
Preceding this message,
one or more messages will
have been issued that
identify the error
condition.

In this second example, job WCA008LA has either never been executed, or when it
was executed, the correlation service was active and all changes were processed
successfully. However, when job WCA008IA was executed, the correlation service
was active, however, IMS active change-capture agent WCA008LA went into
recovery from (a simulated) XM queue overflow situations. Listed below is the
output from IMS active agent status job.
(1) CACH061I RECOVERY MODE: CHECK AGENT STATUS
(2) CACH050I RECOVERY DATASET WCA008.XSYNC.WCA008LA REPORTS AGENT IS NOT IN
RECOVERY MODE
(3) CACH030I AGENT ’WCA008LA’ IS NOT IN RECOVERY MODE
(4) CACH050I RECOVERY DATASET WCA008.XSYNC.WCA008IA REPORTS AGENT IS NOT IN
RECOVERY MODE
(5) CACH051I correlation service REPORTS AGENT IN RECOVERY MODE
(6) CACH049I AGENT NAME IS ’WCA008IA’
(7) CACH039I DB2 RESTART TIME 20020411 07304733
(8) CACH040I IMS RESRART TIME 02.101 07:30:47.3
(9) CACH037I RESTART STORE CLOCK B776B62ED935A240
(10) CACH038I RESTART LOG SEQ. # 00000000-00000073
(11) CACH062I RECOVERY PROCESSING COMPLETED SUCCESSFULLY

Once again the WTO messages have been numbered and the meaning of each
message is described in the table below.

Appendix A. The IMS recovery process 99


Table 15. Meaning of the WTO messages in the preceding sample output
Number WTO ID Meaning
1 CACH061I Indicates that the IMS
recovery change-capture
agent is checking the
recovery status of one or
more agents.
2 CACH050I Indicates that recovery data
set is empty.
3 CACH030I Indicates that the correlation
service does not know about
this agent.
4 CACH050I Indicates that the recovery
data set is empty.
5 CACH051I Indicates that the correlation
service knows the agent is in
recovery mode.
6 CACH049I Identifies the name of the
agent that is in recovery.
7 CACH039I Identifies the recovery restart
point date and time in DB2
format. The date or time that
is identified is April 11th,
2002 at 7:30:47 am local time.
8 CACH040I Identifies the recovery restart
point in the date or time
format used by IMS in DBRC
reports. The date or time
identified is the 101st day of
2002 at 7:30:47 am and 300 th
seconds local time.
9 CACH037I Identifies the 8-byte STCK
(system clock) value
contained in the log record
suffix identifying the log
record where the recovery
process needs to begin.
10 CACH038I Identifies the double word
log record sequence number
(contained in the log record
suffix), identifying the log
record where the recovery
process needs to begin.
11 CACH062I Identifies that no errors were
encountered while checking
agents recovery status.

In this example, job WCA008IA has issued checkpoints that caused changes to be
forwarded to the Rule Server, which has confirmed processing of these changes.
Because changes have been committed and confirmed the correlation service has
an exact starting point in the IMS logs, for this IMS control region, where the
recovery process needs to begin.

100 DB2 II Operations Guide for Classic Event Publishing


Note: If an IMS active change-capture agent goes into recovery mode, before any
changes have been committed or confirmed by the publication service, then
the STCK time and log record sequence numbers are zeros. This indicates
that you only have an approximate recovery point from which to start the
recovery process. From a log file identification perspective, you treat this
situation in the same manner that you would when IMS is run without a
correlation service being active.

Putting unknown agents in recovery mode


In the first example, the correlation service does not know that job WCA008LA is
in recovery mode. One way to make the correlation service aware of this is to start
the correlation service and run job WCA008LA.

The IMS active change-capture agent will identify the correlation service that it is
in recovery mode because there is a record in the recovery data set. In the case of a
batch job it might be impractical to re-run the job. In any event starting an IMS
control region that is in recovery mode, for any reason, is un-desirable. Doing so
pushes the point where recovery can be completed farther into the future and
increases the time it takes to complete the recovery process due to need to feed
additional IMS log files into the recovery process. Also remember, DB2 Information
Integrator Classic Event Publisher recovery cannot be completed until all agents in
recovery mode are not running.

The IMS recovery change-capture agent can be used to set unknown IMS active
change-capture agents in recovery mode without having to push the completion
point farther into the future. To do this, run the IMS active agent status job, and
for each agent that is identified as being in recovery, but unknown (i.e., WTO
message CACH048I was issued) then you want to set that agent in recovery mode.

This requires creating a customized control file that only contains control cards for
the agents that need to be placed into recovery mode. For each agent or control
card supply the following parameter values:
v Mode – Set Agent In Recovery
v IMS Change-Capture Agent Name – The 8-character job name associated with
this agent.
v Region Type – not used.
v Data Set Name – The name of the recovery data set associated with this agent.

Going back to our first example, job WCA008LA needs to be recovered, however,
the correlation service does not know this. Supplying the following control file
input and running the IMS recovery change-capture agent will notify the
correlation service that agent WCA008LA is in recovery mode:
Table 16. Control file input
Mode Agent Data set name
S WCA008LA WCA008.XSYNC.WCA008LA

Listed below are the WTO messages returned from the IMS recovery
change-capture agent when this is done.
CACH061I RECOVERY MODE: SET AGENT IN RECOVERY
CACH048I RECOVERY DATASET WCA008.XSYNC.WCA008LA OPENED
CACH052I AGENT ’WCA008LA’ IS NOW IN RECOVERY MODE

Appendix A. The IMS recovery process 101


CACH039I DB2 RESTART TIME 20020411 10491967
CACH040I IMS RESTART TIME 02.101 10:49:19.6
CACH062I RECOVERY PROCESSING COMPLETED SUCCESSFULLY

If you mistakenly re-run this job, or supply a control card for an agent that has
recovery data set information and has already been placed in recovery, you will
receive the following WTO messages:
CACH061I RECOVERY MODE: SET AGENT IN RECOVERY
CACH048I RECOVERY DATASET WCA008.XSYNC.WCA008LA OPENED
CACH053I AGENT ’WCA008LA’ ALREADY IN RECOVERY MOD
CACH062W RECOVERY PROCESSING ENDED WITH WARNINGS

Also, note that if you run the IMS active agent status job again, you will receive
slightly different output. Here is the output from the IMS active agent status job
after job WCA008LA has been manually placed into recovery:
CACH061I RECOVERY MODE: CHECK AGENT STATUS
CACH048I RECOVERY DATASET WCA008.XSYNC.WCA008LA OPENED
CACH049I AGENT NAME IS ’WCA008LA’
CACH039I DB2 RESTART TIME 20020411 10491967
CACH040I IMS RESTART TIME 02.101 10:49:19.6
CACH051I correlation service REPORTS AGENT IN RECOVERY MODE
CACH049I AGENT NAME IS ’WCA008LA’
CACH039I DB2 RESTART TIME 20020411 10491967
CACH040I IMS RESTART TIME 02.101 10:49:19.6
CACH037I RESTART STORE CLOCK 0000000000000000
CACH038I RESTART LOG SEQ. # 00000000-00000000
CACH050I RECOVERY DATASET WCA008.XSYNC.WCA008IA
CACH030I AGENT ’WCA008IA’ IS NOT IN RECOVERY MODE
CACH062I RECOVERY PROCESSING COMPLETED SUCCESSFULLY

Notice that in addition to reporting the restart information contained in the


recovery data set there is also restart information being obtained from the
correlation service. The restart date or time will be the same, however, the
correlation service does not have exact IMS log file restart information.

After you identify the IMS Active Agents that need to be recovered, and for
unknown agents, placed them in recovery mode, you need to identify the IMS log
files that need to be feed into the recovery process. Identification of the IMS log
files that are needed in the recovery process is purely a manual operation. IBM
recommends that you register all IMS control regions being monitored with DBRC
as well as creating recovery data sets for each IMS control region.

For any IMS active change-capture agents that are reported in recovery mode, you
will be supplied with a DB2 and IMS DBRC date and time stamp of when the
agent failure occurred. In the case of a failure while the correlation service was
operational and the IMS active change-capture agent was communicating with the
correlation service and has committed and one, or more, UOR s then additional
restart information is supplied.

If you know the names of the IMS log files that need to be recovered, then proceed
to “Recovering log files” on page 104. If you do not know the names of the IMS
log files that need to be input into the recovery process, use DBRC to provide you
with a list of IMS log files that are available for input into the recovery process.

102 DB2 II Operations Guide for Classic Event Publishing


Using DBRC to identify log files needed for recovery
Assuming that your are using recovery data sets and have input the names of
these data set to the IMS recovery change-capture agent, then you have the names
of IMS control regions that need to be recovered. Given a recovery starting point
you can specify the name of an IMS log file that contains IMS log records that
where created before or on the recovery starting point recorded by DB2
Information Integrator Classic Event Publisher. The correlation service
automatically discards log records for units-of-work that were created before the
recovery point.

Based on the state of the IMS active change-capture agent when failure with the
correlation service occurred, then different restart information is available.
v If the IMS Control Region was activated without a correlation service, then log
recovery starts at the beginning of the first log file created by that IMS control
region when it was activated.
v For agents that failed while connected and active with a correlation service the
starting log sequence number and store clock time of the oldest existing in-flight
unit-of-work.

In either case you can use DBRC to determine the names of the IMS log files
associated with the IMS control region that needs to be recovered. You can obtain
information about the IMS log files associated with an IMS subsystem using the
DBRC LIST.LOG ALL command.

If you have an IMS batch job in recovery that is being tracked by DBRC, you can
issue the following command to obtain a list of IMS log files associated with the
batch job:

LIST.LOG ALL SSID(Batch-Job-Name)

The DBRC output lists all of the IMS log files for the IMS batch job that DBRC
knows about. If the IMS recovery change-capture agent reports an exact restart
point for the agent in recovery, that is a non-zero system store clock and non-zero
log sequence numbers are displayed, then the IMS log files that need to be input
into the recovery process are selected by identifying the IMS log file that was
active at time of failure. This file has a creation date that is on or before the IMS
restart time displayed and has a starting log record sequence number that is less
than or equal to the restart log record sequence number. All log files created after
the first log file was created.

If you only have an approximate recovery starting point (no system clock or IMS
log record sequence number), then you want to select as the initial starting IMS log
file an IMS log file that was created around the time of the approximate restart
date or time displayed by the IMS recovery change-capture agent. Generally, for
batch jobs the IMS log file is created first, the IMS active change-capture agent is
then called, detects that no correlation service is active and records the
approximate as the time the failure is detected. Therefore, the initial IMS log file
that needs to be input into the recovery process is created before or on the
approximate initial restart recovery time displayed by the IMS recovery
change-capture agent. After you identify the initial IMS log file you want to input
all other IMS log files created after the initial IMS log file into the recovery process.

Identifying IMS log files for a DB/DC or DBCTL subsystem in recovery is more
difficult. Generally, you need to supply one or more of the archived online log data

Appendix A. The IMS recovery process 103


sets into the recovery process. In DBRC terms these are identified as either PRISLD
(primary system log) data set s or SECSLD (secondary system log) data sets. It is
also possible to use the non-archived online log data sets as input under the
following conditions:
1. The IMS DB/DC or DBCTL subsystem is shut down.
2. The online logs contain the initial restart recovery point reported by the IMS
recovery change-capture agent.

Assuming that you are going to use archived logs, then the following DBRC
command can be issued:

LIST.LOG ALL

If you have multiple DB/DC or DBCTL subsystems you can further restrict the
DBRC output by specifying the SSID parameter identifying the 4-character
subsystem ID to be selected. In these cases the IMS active change-capture agent
name that is in recovery will not match the subsystem ID that needs to be
supplied, if the DB/DC or DBCTL started procedure name is not the same as the
IMS subsystem ID.

After the DBRC report is generated, you want to edit the report output and
perform a find for LOGALL you will find several of these entries in the report
output. The one of interest does not have any database names associated it with it
in the report header and has a sub-heading of PRISLD or SECSLD depending upon
whether you have dual logging activated. When dual logging is in effect either the
primary or secondary system logs can be used as input.

After you find the correct section of the report you will see a list of log files
created in ascending creation date or time sequence. Depending on whether you
have approximate or more exact restart information you use the same rules,
discussed above, for batch jobs in determining which IMS log files need to be
input into the IMS recovery change-capture agent IMS log file recovery process.

Recovering log files


After you have identify the IMS log files that need to be input into the recovery
process you can input these files into the recovery process and move the recovery
restart point forward in time for one or more IMS active change-capture agents
that are in recovery mode. If you have multiple IMS agents in recovery mode it is
important that you supply IMS log files for all of these agents each time log
recovery is performed. Failure to do so can cause changes to be propagated to the
Rules Service in the wrong time sequence.

Additionally, if you are supplying multiple IMS log files for an agent that is in
recovery, you need to supply all IMS log files for the time period being recovered.
The IMS recovery change-capture agent automatically checks the store clock
(system clock) time sequences of the IMS log files supplied for an agent in
recovery and automatically detects duplicated log files and log files that have
overlapping store clock timestamps.

However, it is not possible for the IMS recovery change-capture agent to detect
when you forgot to supply one or more of the IMS log files that need to be input
into the recovery process. If this condition occurs, the recovery process is likely to
fail, due to invalid IMS UOR synchpoint sequences when the next log file (after the

104 DB2 II Operations Guide for Classic Event Publishing


missing IMS log files) is processed. If this situation occurs, recovery might not be
possible depending upon when the failure is reported by the correlation service.

Use the following guidelines for identifying IMS log files to be recovered:
v IMS log files can be supplied that were created before the current restart point.
These files are read and log records that were created before the current restart
point are discarded.
v If you are supplying more than one IMS log file for an agent in recovery, you
can identify the input IMS log files in any sequence. The IMS recovery
change-capture agent automatically re-sequences these files into the correct
processing sequence based on the system clock values contained in each IMS log
record.
v If you are recovering multiple agents, agents can be identified in any sequence
as well as the names of the input IMS log files. The IMS recovery
change-capture agent automatically match merges multiple change-capture agent
IMS log files in store clock sequence and presents these log records to the
correlation service in the time sequence that the log records were created in.

Additionally, you can identify whether you want to perform a full log file recovery
or an incremental recovery. Incremental recovery is required when you are
recovering multiple IMS DB/DC or DBCTL subsystems or when one or more of
the subsystems are still running.

In this situation you have IMS log files that contain log records whose system time
stamps overlap and you have different end points in these logs. For this situation
you can only recover the log files contents up to the end of the IMS log files for
the older of the two subsystems. Incremental recovery is activated by specifying a
Mode of I (Incremental Log File Recovery) for the controlling subsystem.
Incremental Log File Recovery can be specified for any IMS log file for the
subsystem that is controlling the recovery process. You can also identify that
incremental recovery is to be performed on multiple active change-capture agents
that are in recovery.

When incremental recovery is specified for any agent in recovery, once all IMS log
files for that agent have been processed the IMS recovery change-capture agent
stops processing IMS log files for the other agents and (normally) terminates
processing. This allows you to move the recovery restart points for the agents in
recovery forward in time. When more log files are available, you can input these
files into the recovery process, possibly identifying a different agent as being the
controlling agent and continue to push the restart recovery point further into the
future. You continue performing incremental recoveries until all IMS control
regions can be shut down, it is then possible to complete the recovery process by
inputting all of the remaining IMS log files up to the last log file created by each
IMS control region that is in recovery mode.

Log file recovery requires creating a customized control file that contains control
cards for the agents in recovery. A control card must be supplied for each agent
and IMS log file that is to be recovered. For each agent or control card supply the
following parameter values:
v Mode – Log File Recovery or Incremental Log File Recovery
v IMS Change-Capture Agent Name – The 8-character job name associated with
this agent.
v Region Type – The type of IMS control region that is being recovered.

Appendix A. The IMS recovery process 105


v Data Set Name – The fully qualified name of an IMS log file that is to be
recovered.

Going back to our examples, WCA008IA went into recovery due to an XM queue
overrun. Let s also assume that there were three steps in job WCA008IA that
updated IMS. Also assume that the XM queue failure occurred while executing the
first job step that updated IMS data.

In this scenario, there are three log files that need to be input into the recovery
process. For this example, the IMS log files are being written to a generation data
set (GDG). When using GDG s you want to specify the fully qualified data set
name and not use relative generation numbers. This ensures that log files will not
be feed in out of sequence due to the creation of a new generation. Also, in
general, when using GDG s you need to perform the recovery process up to the
latest (current) generation. The table below identifies the input log file recovery
information for agent WCA008IA.
Table 17. Input log file recovery information
Mode Agent Region Data set name
L WCA008IA 2 WCA008.IMS.WCA008IA.LOG..G0002V00
L WCA008IA 2 WCA008.IMS.WCA008IA.LOG..G0001V00
L WCA008LA 2 WCA008.IMS.WCA008IA.LOG..G0003V00

In this example we are requesting a full log recovery and have identified agent
WCA008LA as being an IMS batch region. Also, note that the log files have not
been identified in the physical order that they need to be recovered in. The listing
below shows the WTO messages that are issued by the IMS recovery
change-capture agent when it processed the above input.
CACH061I RECOVERY MODE: IMS LOG FILE RECOVERY
CACH055I STARTING LOG FILE SEQUENCE CHECKING
CACH058I LOG FILE SEQUENCE CHECKING COMPLETED
CACH044I AGENT ’WCA008IA’ LOG OPENED: WCA008.IMS.WCA008IA.LOG.G0001V00
CACH045I AGENT ’WCA008IA’ LOG CLOSED: WCA008.IMS.WCA008IA.LOG.G0001V00
CACH044I AGENT ’WCA008IA’ LOG OPENED: WCA008.IMS.WCA008IA.LOG.G0002V00
CACH045I AGENT ’WCA008IA’ LOG CLOSED: WCA008.IMS.WCA008IA.LOG.G0002V00
CACH044I AGENT ’WCA008IA’ LOG OPENED: WCA008.IMS.WCA008IA.LOG.G0003V00
CACH045I AGENT ’WCA008IA’ LOG CLOSED: WCA008.IMS.WCA008IA.LOG.G0003V00
CACH062I RECOVERY PROCESSING COMPLETED SUCCESSFULLY

The first WTO message is CACH061I that identifies that the IMS recovery
change-capture agent is performing IMS log file recovery. Next, message
CACH055I is issued after the input file has been read, the IMS recovery
change-capture agent has verified that the agent is in recovery mode and
communications with the correlation service have been established. It is informing
you that the IMS recovery change-capture agent is in the process of verifying, to
the best of its ability, that you have supplied the correct set of IMS log files. During
this process, the IMS log files are opened and their contents read when either 1)
multiple IMS log files have been identified for an agent in recovery, or 2) when
you have requested incremental log file recovery.

After log file sequence checking is completed, then message CACH058I is issued
identifying that sequence checking has been completed successfully and that IMS
log file recovery is ready to commence. If an invalid IMS log file is supplied, one
of the following WTO messages is issued and the IMS recovery change-capture
agent immediately terminates with a non-zero return code:

106 DB2 II Operations Guide for Classic Event Publishing


v CACH056E AGENT Agent-Name DUPLICATE LOG: IMS-Log-File-Name
v CACH057E AGENT Agent-Name INVALID LOG: IMS-Log-File-Name

Message CACH056E is self explanatory. The IMS log file identified in the message
has the same starting or ending system time stamp values as another IMS log file
associated with the agent. Message CACH057E is more ambiguous. This message
is issued when an overlap in system time stamps has been discovered with the
identified IMS log file and another IMS log file associated with the agent. The
general implication of this message is that you specified the name of IMS log file
that is associated with some other IMS control region.

Message CACH044I is issued each time an IMS log file is opened for recovery
processing. After the entire contents of an IMS log file are read and processed,
message CACH045I is issued.

If you supply an IMS log file that contains log records that were created before the
recovery restart point, the number of IMS log records sent to the correlation service
is zero. The IMS recovery change-capture agent automatically filters these types of
record outs.

The last message issued is CACH062I or CACH062E that identify whether the IMS
log files were recovered successfully or whether errors were encountered. In this
example, all IMS log files were processed successfully and no errors were reported.

Note: When you are performing incremental recovery, after the IMS recovery
change-capture agent completes processing, always re-run the IMS active
agent status job to see how much farther into the future the recovery restart
points have been pushed. In most cases you will have to supply the last
processed IMS log file for each agent in recovery back into the IMS log file
recovery process when you perform the next incremental recovery run, or
the final recovery run.

The correlation service records, as the recovery restart point, the log record
of the oldest living unit-of-work when a unit-of-work is started. Generally,
for a DB/DC or DBCTL subsystem there are always active units-of-work so
that the restart recovery point almost never matches the physical end of file
for an IMS log file. An exception to this rule is when you supply an IMS log
file that was the last log file produced for a DB/DC or DBCTL subsystem
before the subsystem was shut down.

Returning agents back to active mode


After you complete the recovery process for all of your IMS active change-capture
agents that are in recovery mode, you use the IMS recovery change-capture agent
to place those agents in recovery mode back into active mode. Agents can be
successfully be placed back into active mode once the following conditions are
met:
v The IMS control region when the active agent was running is shut down.
v All log files created by the IMS control region in recovery have successfully gone
through the IMS log file recovery process.

Setting agents back into active mode requires creating a customized control file
that only contains control cards for the agents that need to be activated. For each
agent or control card supply the following parameter values:
v Mode – Place Agent In Active Mode

Appendix A. The IMS recovery process 107


v IMS Change-Capture Agent Name – The 8-character job name associated with
this agent.
v Region Type – not used.
v Data Set Name – The name of the recovery data set associated with this agent.

For example, assume that agent WCA008IA is in recovery mode and you have
recovered the three log files discussed in the previous topic – Recovering Log Files.
After IMS log file recovery is performed, and assuming that job WCA008IA has not
been re-executed during the recovery process, agent WCA008IA meets the
requirements for re-activating it back into active mode. Supply the following
control file input to re-active agent WCA008IA.
Table 18. Control file input
Mode Agent Data set name
A WCA008IA WCA008.XSYNC.WCA008IA

Listed below are the output WTO messages from the IMS recovery change-capture
agent when this is done.
CACH061I RECOVERY MODE: ACTIVATE AGENT
CACH054I PREPARING TO ACTIVATE AGENT WCA008IA
CACH031I AGENT WCA008IA SWITCHING TO ACTIVE MODE
CACH062I RECOVERY PROCESSING COMPLETED SUCCESSFULLY

The first message is CACH061I that identifies that the IMS recovery
change-capture agent is preparing to place one or more agents back into active
mode. For each agent being activated, message CACH054I is issued informing you
that the IMS recovery change-capture agent is about to attempt to re-activate the
agent. If the agent is successfully returned to active mode, then message
CACH031I is issued. If the agent is reported as not being in recovery or there are
problems communicating with the correlation service, message CACH031I is not
issued and you will see other error messages that identify what the problem is.

As part of activating an agent, the contents of the recovery data set are deleted.

After all IMS agents in recovery are placed into active mode, DB2 Information
Integrator Classic Event Publisher recovery is complete and you can resume
normal IMS and DB2 Information Integrator Classic Event Publisher operations.

108 DB2 II Operations Guide for Classic Event Publishing


Appendix B. Configuration parameters
This appendix contains the format and descriptions of the DB2 Information
Integrator Classic Event Publisher configuration parameters.

The following sections are discussed:


v “Configuration parameter format” on page 109
v “Configuration parameters” on page 110
v “Configuration parameter descriptions” on page 110

Configuration parameter format


Configuration parameters consist of fixed-length 80-byte records containing either a
parameter starting in column 1 or a comment, represented as an asterisk (*), in
column 1. The parameter syntax follows.

Example:
parameter name = value

In this example,
v Parameter name is one or more keywords beginning in the first column of the
record.
v A blank must exist on both sides of the equal sign.
v Value is any number of characters up to the end of the record.
v String values are not surrounded by delimiters.
v Comments after the value are not allowed.

The maximum parameter length is 255 characters, but parameters can be continued
across 80-byte records by using the backslash (\) as a continuation character. The
continuation character cannot be used until after the equal sign, and must be the
last non-blank character of the record. The backslash character will be discarded, as
will leading blanks on the continued record. Comment lines can be inserted
between the continued records.

When editing configuration data sets, do not insert sequence numbers at the end of
the records because they become part of the value assigned to the corresponding
keyword.

Note: At any level, if a configuration file contains invalid parameters, then none of
that configuration image is loaded into memory. If the Master Configuration
File cannot be loaded, DB2 Information Integrator Classic Event Publisher
will terminate. If ISPF is used, make sure that NUM OFF is set in the edit
profile when editing configuration members.

© Copyright IBM Corp. 2003, 2004 109


Configuration parameters
The following table lists the configuration parameters and whether the parameter
is required (R), optional (O), or not applicable (left blank).
Table 19. Configuration parameter classifications
Parameter Required
LD TEMP SPACE n N
MESSAGE POOL SIZE N
NL Y
NL CAT Y
INTERLEAVE INTERVAL N
PUB Y
SAF EXIT N
SERVER CODEPAGE Y
SERVICE INFO ENTRY Y
STATIC N
CATALOGS
TASK PARAMETERS N

Configuration parameter descriptions


The following is an alphabetical listing of configuration parameters and their
descriptions.

Parameter defaults are used when the parameter is not specified in the relevant
configuration file.

Invalid parameters will cause the file to be ignored. DB2 Information Integrator
Classic Event Publisher does not issue a message indicating that the default value
was applied or that a value in the configuration file is invalid.

LD TEMP SPACE
Description: Optional parameter that defines a temporary data set dynamically
allocated by a data server to store the intermediate result set. Temporary data set
information is a set of parameters separated by commas. Parameters not specified
are set to the defaults.

Set this parameter so the resulting file is large enough to hold any intermediate
result sets that are generated from a typical query running under a particular data
server. If your site has a storage unit name set up for VIO storage, specify VIO.

Hiperspace is a feature that allows the placement of temporary data files, such as
temporary files, spill files, and so on, in expanded storage. This results in
improved performance. This performance improvement mostly affects complex
queries, for example, ORDER BY. To specify Hiperspace™, set the LD TEMP SPACE
as follows:
LD TEMP SPACE = HIPERSPACE,INIT=16M,MAX=2G,EXTEND=8M

Where:

110 DB2 II Operations Guide for Classic Event Publishing


v INIT= initial region size for the hiperspace.
v MAX= maximum region size for the hiperspace.
v EXTEND= unit of growth when INIT is exceeded.

The estimate for determining these values is related to system installation limits
and expected query types. Roughly, the maximum size should be equivalent to that
of the regular temp space file as described for the non-hiperspace LD TEMP
SPACE setting.

Note: Using hiperspace requires APF authorization.

MESSAGE POOL SIZE


Description: Required parameter that specifies the size of the memory region used
for all memory allocation. The number is specified in bytes. The actual workable
maximum value should be set to 2MB less than the region size. If the value
specified is less than 1MB, 1MB is used. If the amount of storage that can be
obtained is less than the value specified, the maximum amount available is
obtained.

Example:
MESSAGE POOL SIZE = 16777216

Allowable value type: numeric

Representation: decimal

Maximum permitted value: 2147483648 (2GB)

Minimum permitted value: 1048576 (1MB)

Default: 1048575 (1MB)

NL
Description: Required parameter that specifies the language used for text messages
produced by DB2 Information Integrator Classic Event Publisher. US English
corresponds to standard English as used in the United States.

Example:
NL = US ENGLISH

Allowable value type: string

Default: US ENGLISH

NL CAT
Description: Required parameter that points to the language catalog that contains
DB2 Information Integrator Classic Event Publisher messages in a specified
language. It is defined by a DD statement in the start-up procedure.

Example:
NL CAT = DD:ENGCAT

Appendix B. Configuration parameters 111


Note: Different catalog file names can be specified. The one shown above is for the
US English language catalog (DD:ENGCAT).

Allowable value type: string DD:, followed by a character string or string DSN,
followed by a data set name.

Representation: string

Default value: DD:ENGCAT

INTERLEAVE INTERVAL
Description: Optional parameter. This value sets the interleaving interval from the
query processor. The unit of measurement for this interval is a result set row.
When multiple results sets are being processed on the same query processor
instance, the interleaving interval controls context switching between users and
result sets. For example, if INTERLEAVE INTERVAL is set to 100 then the query
processor will context switch between active users on that instance for every 100
rows produced.

Default Value: 100

Minimum Value: 100

Maximum Value: 4294967295

Note: The value of 0 (zero) is permitted. This value turns off context switching.

PUB
Description: Required parameter used to specify the parts of a publication.
Publications are composed of the following parts:
Alias parameter
An alias defines the unique name for a publication within a Data Server.
Topic parameter (optional)
Include a topic in your publication if you want to publish it to WebSphere
Business Integrator Event Broker. Topics tell the WBI Event Broker how to
route the messages for the publication.
Queue parameter
Queues are the WebSphere MQ queues where messages are put. The
format of this parameter is MQI/QUEUE_MANAGER/QUEUE_NAME, where MQI is
the designator for WebSphere MQ, QUEUE_MANAGER is the name of the queue
manager that the publication service is working with, and QUEUE_NAME is
the name of the queue on which messages for the publication are put.
Message output parameters
Message output parameters define how the messages are constructed. The
table below lists these parameters and describes them.

112 DB2 II Operations Guide for Classic Event Publishing


Table 20. Message output parameters
Message output parameter Default Possible values
value
MSGTYPE TRANS
TRANS
A message is published for each
committed transaction that affects the
source table. The message contains all
of the changes made to the source table
by the transaction.
ROWOP
A message is published for each
committed row operation on the source
table.
TABLE None The Table string is used to identify the mapped
table name from which changes will be
published. There can be only one table per
publication. This table is specified in the format :
ownerName.tableName (for example,
QAVSAM.EMPLOYEES). The example above
refers to a table, QAVSAM.EMPLOYEES, that
was mapped into the Event Publisher catalog
and was altered for data capture changes.
BEFORE_VALUES NO
NO When a row is updated, only the
current values for all columns are
included in the message.
YES When a row is updated, the previous
values and the current values for all
columns are included in the message.
This parameter is effective for UPDATE
operations only.
CHANGED_COLS_ONLY YES This parameter is not currently supported. Do
not change its value.
ALL_CHANGED_ROWS NO This parameter is not currently supported. Do
not change its value.

SAF EXIT
Description: Optional parameter used to specify the SAF system exit that will
perform authorization checks for the connector associated with a data server or
execute a stored procedure application program. The default is NONE.

Example:
SAF EXIT = CACSX04 IMS CLASS=IMSP,PSB PREFIX=IMSP

Default: none

SERVICE INFO ENTRY


Description: Required parameter used in server configuration files to inform the
region controller task that a service should be activated and how that service
should be controlled. It is valid only in server and enterprise server configuration
files.

Appendix B. Configuration parameters 113


For information about this parameter, see ″Configuring the correlation service and
publication service″ in Chapter two.

SMF EXIT
Description: Optional parameter used to report wall clock time and the CPU time
for an individual user session with a query processor Task.

Example:
SMF EXIT = CACSX02 RECTYPE=255,SYSID=JES2

Default: none

STATIC CATALOGS
Description: Optional parameter used to activate static catalog processing for the
metadata catalog data sets referenced by the data server. Static catalog close should
be used when the metadata catalogs referenced by the connector will not be
updated while the connector is executing (essentially, when the data server is
operating in production mode and the metadata catalogs are static).

When static catalog processing is activated, the metadata catalog files are opened
once for a query processor task. The metadata catalog files remain open until that
server is shut down. In normal operating mode, the metadata catalogs are closed
after the required table and column information has been retrieved in order to
process a query, for each query processed by the query processor.

Activation of static catalog processing can substantially improve query


performance in outer or inner cursor situations when a large number of queries are
being issued serially.

Example:
STATIC CATALOGS = 1

Allowable value type: numeric

Representation: decimal

Maximum permitted value: 1

Minimum permitted value: 0

Allowed value and results:


v 0 (close metadata catalog files and establish read locks for each query)
v 1 (close metadata catalog files when the server is shut down)

Default: 0

TASK PARAMETERS
Description: Optional parameter that specifies SAS/C runtime options passed to
system daughter tasks through the z/OS ATTACH macro.

One common use of this parameter is to pass TCP/IP information to the


Communications Interface task.

114 DB2 II Operations Guide for Classic Event Publishing


The TCPIP_PREFIX variable sets the high-level qualifier (hlq) for finding the
TCP/IP system data sets. It can be set to use the installation defined data sets, or a
user-defined data set.

The TCPIP_MACH variable sets the address space name or subsystem name of the
TCP/IP stack for Interlink. For IBM’s TCP/IP system utilizing the Berkeley Socket
interface, this parameter can also be specified in the hlq.TCPIP.DATA file under the
parameter TCPIPUSERID.

The default for both variables is TCPIP.


TASK PARAMETERS= =TCPIP_PREFIX=TCPIP =TCPIP_MACH=TCPIP

The Time Zone environment variable (TZ) must be set for each job on z/OS. The
variable sets the time zone in which the task will start, for example Pacific
Standard Time (PST).

Example:
TASK PARAMETERS = =MI =TZ=PST8PDT

This sets the time zone to PST plus 8 hours from Greenwich mean time (8) and
Pacific Daylight Time (PDT).

Using the same example for Eastern Standard Time (EST), enter the following
information:
TASK PARAMETERS = =MI =TZ=EST5EDT

For additional information about other valid TZ settings see www.sas.com.

Representation: string

Maximum permitted value: any valid parameter preceded by the equal (=) sign, as
documented at www.sas.com.

Default: none

VSAM AMPARMS
Description: Optional parameter string used to supply CICS VSAM buffer and
memory-tuning parameters when a CICS VSAM file is opened. The CICS VSAM
AMPARMS parameter specifies tuning parameters that are applied to all CICS
VSAM files opened when a single cursor is open.

The CICS VSAM AMPARMS parameter takes the form of a string of comma
delimited parameters that are passed to the SAS/C afopen call that is used to
access CICS VSAM files. The following parameters can be supplied:
v BUFND=n: Specifies the number of data I/O buffers CICS VSAM is to use.
Specification of this parameter is equivalent to coding the BUFND value on a
DD statement. A data buffer is the size of a control interval in the data
component of a CICS VSAM cluster. The default number of data buffers is the
number of strings plus one. If you are using the CICS VSAM service, the default
number of buffers would be 11. If you are not using the CICS VSAM service, the
default number of buffers is two.
Generally, with sequential access the optimum value for the data buffers is six
buffers or the size of the control area, whichever is less. When skip-sequential
processing (random keyed read access) is being performed, specifying two

Appendix B. Configuration parameters 115


buffers is optimum. Specifying a larger BUFND value when the CICS VSAM file
is being scanned during query processing generally yields performance
improvements. In keyed access situations specifying a larger BUFND might
show no performance improvements, or might actually degrade query
performance by tying up large amounts of virtual storage and causing excessive
paging.
v BUFNI=n: Specifies the number of index I/O buffers CICS VSAM is to use.
Specification of this parameter is equivalent to coding the BUFNI value on a DD
statement. An index buffer is the size of a control interval in the index
component of a keyed CICS VSAM cluster. If you are using the CICS VSAM
service, the default number of index buffers is 10. If you are not using the CICS
VSAM service, the default number of index buffers is 1.
For keyed access, the optimum BUFNI specification is the number of high-level
(non-sequence set) index buffers + 1. This number can be determined by
subtracting the number of data control areas from the total number of index
control intervals within the data set. An upper bound BUFNI specification of 32
can be used, which accommodates most CICS VSAM files with reasonable index
control interval and data control area sizes (cylinder allocated data component)
up to the 4GB maximum allowed data component size. Specification of a large
BUFNI value incurs little or no performance penalty, unless they are excessive.
v BUFSP=n: Specifies the maximum number of bytes of storage to be used by
CICS VSAM for file data and index I/O buffers. Specification of this parameter
is equivalent to coding the BUFSP value on a DD statement. A data or index
buffer is the size of a control interval in the data or index component.
A valid BUFSP specification generally overrides any BUFND or BUFNI
specification. However, the CICS VSAM rules for specifying an optimum BUFSP
value are fairly complex. The appropriate IBM-supplied information about the
ACB macro should be consulted to determine the rules for specifying a BUFSP
value.

Example:
VSAM AMPARMS = BUFND=20,BUFNI=15

Allowable value type: string

Representation: string

Maximum permitted value: 255 characters

Minimum permitted value: 7 characters

Default: None

116 DB2 II Operations Guide for Classic Event Publishing


DB2 Information Integrator documentation
This topic provides information about the documentation that is available for DB2
Information Integrator. The tables in this topic provide the official document title,
form number, and location of each PDF book. To order a printed book, you must
know either the official book title or the document form number. Titles, file names,
and the locations of the DB2 Information Integrator release notes and installation
requirements are also provided in this topic.

This topic contains the following sections:


v Accessing DB2 Information Integrator documentation
v Documentation for replication function on z/OS
v Documentation for event publishing function for DB2 Universal Database on
z/OS
v Documentation for event publishing function for IMS and VSAM on z/OS
v Documentation for event publishing and replication function on Linux, UNIX,
and Windows
v Documentation for federated function on z/OS
v Documentation for federated function on Linux, UNIX, and Windows
v Documentation for enterprise search on Linux, UNIX, and Windows
v Release notes and installation requirements

Accessing DB2 Information Integrator documentation


All DB2 Information Integrator books and release notes are available in PDF files
from the DB2 Information Integrator Support Web site at
www.ibm.com/software/data/integration/db2ii/support.html.

To access the latest DB2 Information Integrator product documentation, from the
DB2 Information Integrator Support Web site, click on the Product Information
link, as shown in Figure 6 on page 118.

© Copyright IBM Corp. 2003, 2004 117


Figure 6. Accessing the Product Information link from DB2 Information Integrator Support Web site

You can access the latest DB2 Information Integrator documentation, in all
supported languages, from the Product Information link:
v DB2 Information Integrator product documentation in PDF files
v Fix pack product documentation, including release notes
v Instructions for downloading and installing the DB2 Information Center for
Linux, UNIX, and Windows
v Links to the DB2 Information Center online

Scroll though the list to find the product documentation for the version of DB2
Information Integrator that you are using.

118 DB2 II Operations Guide for Classic Event Publishing


The DB2 Information Integrator Support Web site also provides support
documentation, IBM Redbooks, white papers, product downloads, links to user
groups, and news about DB2 Information Integrator.

You can also view and print the DB2 Information Integrator PDF books from the
DB2 PDF Documentation CD.

To view or print the PDF documentation:


1. From the root directory of the DB2 PDF Documentation CD, open the index.htm
file.
2. Click the language that you want to use.
3. Click the link for the document that you want to view.

Documentation about replication function on z/OS


Table 21. DB2 Information Integrator documentation about replication function on z/OS
Form
Name number Location
ASNCLP Program Reference for Replication N/A DB2 Information Integrator
and Event Publishing Support Web site
Introduction to Replication and Event GC18-7567 DB2 Information Integrator
Publishing Support Web site
Migrating to SQL Replication N/A DB2 Information Integrator
Support Web site
Replication and Event Publishing Guide and SC18-7568 v DB2 PDF Documentation CD
Reference
v DB2 Information Integrator
Support Web site
Replication Installation and Customization SC18-9127 DB2 Information Integrator
Guide for z/OS Support Web site
SQL Replication Guide and Reference SC27-1121 v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site
Tuning for Replication and Event Publishing N/A DB2 Information Integrator
Performance Support Web site
Tuning for SQL Replication Performance N/A DB2 Information Integrator
Support Web site
Release Notes for IBM DB2 Information N/A v In the DB2 Information
Integrator Standard Edition, Advanced Edition, Center, Product Overviews >
and Replication for z/OS Information Integration >
DB2 Information Integrator
overview > Problems,
workarounds, and
documentation updates
v DB2 Information Integrator
Installation launchpad
v DB2 Information Integrator
Support Web site
v The DB2 Information Integrator
product CD

DB2 Information Integrator documentation 119


Documentation about event publishing function for DB2 Universal
Database on z/OS
Table 22. DB2 Information Integrator documentation about event publishing function for DB2
Universal Database on z/OS
Form
Name number Location
ASNCLP Program Reference for Replication N/A DB2 Information Integrator
and Event Publishing Support Web site
Introduction to Replication and Event GC18-7567 v DB2 PDF Documentation CD
Publishing
v DB2 Information Integrator
Support Web site
Replication and Event Publishing Guide and SC18-7568 v DB2 PDF Documentation CD
Reference
v DB2 Information Integrator
Support Web site
Tuning for Replication and Event Publishing N/A DB2 Information Integrator
Performance Support Web site
Release Notes for IBM DB2 Information N/A v In the DB2 Information
Integrator Standard Edition, Advanced Edition, Center, Product Overviews >
and Replication for z/OS Information Integration >
DB2 Information Integrator
overview > Problems,
workarounds, and
documentation updates
v DB2 Information Integrator
Installation launchpad
v DB2 Information Integrator
Support Web site
v The DB2 Information Integrator
product CD

Documentation about event publishing function for IMS and VSAM on


z/OS
Table 23. DB2 Information Integrator documentation about event publishing function for IMS
and VSAM on z/OS
Form
Name number Location
Client Guide for Classic Federation and Event SC18-9160 DB2 Information Integrator
Publisher for z/OS Support Web site
Data Mapper Guide for Classic Federation and SC18-9163 DB2 Information Integrator
Event Publisher for z/OS Support Web site
Getting Started with Event Publisher for z/OS GC18-9186 DB2 Information Integrator
Support Web site
Installation Guide for Classic Federation and GC18-9301 DB2 Information Integrator
Event Publisher for z/OS Support Web site
Operations Guide for Event Publisher for z/OS SC18-9157 DB2 Information Integrator
Support Web site

120 DB2 II Operations Guide for Classic Event Publishing


Table 23. DB2 Information Integrator documentation about event publishing function for IMS
and VSAM on z/OS (continued)
Form
Name number Location
Planning Guide for Event Publisher for z/OS SC18-9158 DB2 Information Integrator
Support Web site
Reference for Classic Federation and Event SC18-9156 DB2 Information Integrator
Publisher for z/OS Support Web site
System Messages for Classic Federation and SC18-9162 DB2 Information Integrator
Event Publisher for z/OS Support Web site
Release Notes for IBM DB2 Information N/A DB2 Information Integrator
Integrator Event Publisher for IMS for z/OS Support Web site
Release Notes for IBM DB2 Information N/A DB2 Information Integrator
Integrator Event Publisher for VSAM for z/OS Support Web site

Documentation about event publishing and replication function on


Linux, UNIX, and Windows
Table 24. DB2 Information Integrator documentation about event publishing and replication
function on Linux, UNIX, and Windows
Name Form number Location
ASNCLP Program Reference for Replication and N/A DB2 Information Integrator
Event Publishing Support Web site
Installation Guide for Linux, UNIX, and GC18-7036 v DB2 PDF Documentation CD
Windows
v DB2 Information Integrator
Support Web site
Introduction to Replication and Event GC18-7567 v DB2 PDF Documentation CD
Publishing
v DB2 Information Integrator
Support Web site
Migrating to SQL Replication N/A DB2 Information Integrator
Support Web site
Replication and Event Publishing Guide and SC18-7568 v DB2 PDF Documentation CD
Reference
v DB2 Information Integrator
Support Web site
SQL Replication Guide and Reference SC27-1121 DB2 Information Integrator
Support Web site
Tuning for Replication and Event Publishing N/A DB2 Information Integrator
Performance Support Web site
Tuning for SQL Replication Performance N/A DB2 Information Integrator
Support Web site

DB2 Information Integrator documentation 121


Table 24. DB2 Information Integrator documentation about event publishing and replication
function on Linux, UNIX, and Windows (continued)
Name Form number Location
Release Notes for IBM DB2 Information N/A v In the DB2 Information
Integrator Standard Edition, Advanced Edition, Center, Product Overviews
and Replication for z/OS > Information Integration >
DB2 Information Integrator
overview > Problems,
workarounds, and
documentation updates
v DB2 Information Integrator
Installation launchpad
v DB2 Information Integrator
Support Web site
v The DB2 Information
Integrator product CD

Documentation about federated function on z/OS


Table 25. DB2 Information Integrator documentation about federated function on z/OS
Name Form number Location
Client Guide for Classic Federation and Event SC18-9160 DB2 Information Integrator
Publisher for z/OS Support Web site
Data Mapper Guide for Classic Federation and SC18-9163 DB2 Information Integrator
Event Publisher for z/OS Support Web site
Getting Started with Classic Federation for z/OS GC18-9155 DB2 Information Integrator
Support Web site
Installation Guide for Classic Federation and GC18-9301 DB2 Information Integrator
Event Publisher for z/OS Support Web site
Reference for Classic Federation and Event SC18-9156 DB2 Information Integrator
Publisher for z/OS Support Web site
System Messages for Classic Federation and SC18-9162 DB2 Information Integrator
Event Publisher for z/OS Support Web site
Transaction Services Guide for Classic SC18-9161 DB2 Information Integrator
Federation for z/OS Support Web site
Release Notes for IBM DB2 Information N/A DB2 Information Integrator
Integrator Classic Federation for z/OS Support Web site

Documentation about federated function on Linux, UNIX, and Windows


Table 26. DB2 Information Integrator documentation about federated function on Linux, UNIX,
and Windows
Form
Name number Location
Application Developer’s Guide SC18-7359 v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site

122 DB2 II Operations Guide for Classic Event Publishing


Table 26. DB2 Information Integrator documentation about federated function on Linux, UNIX,
and Windows (continued)
Form
Name number Location
C++ API Reference for Developing Wrappers SC18-9172 v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site
Data Source Configuration Guide N/A v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site
Federated Systems Guide SC18-7364 v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site
Guide to Configuring the Content Connector for N/A DB2 Information Integrator
VeniceBridge Support Web site
Installation Guide for Linux, UNIX, and GC18-7036 v DB2 PDF Documentation CD
Windows
v DB2 Information Integrator
Support Web site
Java API Reference for Developing Wrappers SC18-9173 v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site
Migration Guide SC18-7360 v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site
Wrapper Developer’s Guide SC18-9174 v DB2 PDF Documentation CD
v DB2 Information Integrator
Support Web site
Release Notes for IBM DB2 Information N/A v In the DB2 Information
Integrator Standard Edition, Advanced Edition, Center, Product Overviews
and Replication for z/OS > Information Integration >
DB2 Information Integrator
overview > Problems,
workarounds, and
documentation updates
v DB2 Information Integrator
Installation launchpad
v DB2 Information Integrator
Support Web site
v The DB2 Information
Integrator product CD

DB2 Information Integrator documentation 123


Documentation about enterprise search function on Linux, UNIX, and
Windows
Table 27. DB2 Information Integrator documentation about enterprise search function on
Linux, UNIX, and Windows
Name Form number Location
Administering Enterprise Search SC18-9283 DB2 Information
Integrator Support Web
site
Installation Guide for Enterprise Search GC18-9282 DB2 Information
Integrator Support Web
site
Programming Guide and API Reference for SC18-9284 DB2 Information
Enterprise Search Integrator Support Web
site
Release Notes for Enterprise Search N/A DB2 Information
Integrator Support Web
site

Release notes and installation requirements


Release notes provide information that is specific to the release and fix pack level
for your product and include the latest corrections to the documentation for each
release.

Installation requirements provide information that is specific to the release of your


product.
Table 28. DB2 Information Integrator Release Notes and Installation Requirements
Name File name Location
Installation Requirements for IBM Prereqs v The DB2 Information Integrator
DB2 Information Integrator Event product CD
Publishing Edition, Replication
v DB2 Information Integrator
Edition, Standard Edition, Advanced
Installation Launchpad
Edition, Advanced Edition Unlimited,
Developer Edition, and Replication for
z/OS
Release Notes for IBM DB2 ReleaseNotes v In the DB2 Information Center,
Information Integrator Standard Product Overviews > Information
Edition, Advanced Edition, and Integration > DB2 Information
Replication for z/OS Integrator overview > Problems,
workarounds, and documentation
updates
v DB2 Information Integrator
Installation launchpad
v DB2 Information Integrator Support
Web site
v The DB2 Information Integrator
product CD
Release Notes for IBM DB2 N/A DB2 Information Integrator Support
Information Integrator Event Web site
Publisher for IMS for z/OS

124 DB2 II Operations Guide for Classic Event Publishing


Table 28. DB2 Information Integrator Release Notes and Installation
Requirements (continued)
Name File name Location
Release Notes for IBM DB2 N/A DB2 Information Integrator Support
Information Integrator Event Web site
Publisher for VSAM for z/OS
Release Notes for IBM DB2 N/A DB2 Information Integrator Support
Information Integrator Classic Web site
Federation for z/OS
Release Notes for Enterprise Search N/A DB2 Information Integrator Support
Web site

To view the installation requirements and release notes that are on the product CD:
v On Windows operating systems, enter:
x:\doc\%L
x is the Windows CD drive letter and %L is the locale of the documentation that
you want to use, for example, en_US.
v On UNIX operating systems, enter:
/cdrom/doc/%L/
cdrom refers to the UNIX mount point of the CD and %L is the locale of the
documentation that you want to use, for example, en_US.

DB2 Information Integrator documentation 125


126 DB2 II Operations Guide for Classic Event Publishing
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in
all countries. Consult your local IBM representative for information on the
products and services currently available in your area. Any reference to an IBM
product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product,
program, or service that does not infringe any IBM intellectual property right may
be used instead. However, it is the user’s responsibility to evaluate and verify the
operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter
described in this document. The furnishing of this document does not give you
any license to these patents. You can send license inquiries, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.

For license inquiries regarding double-byte (DBCS) information, contact the IBM
Intellectual Property Department in your country/region or send inquiries, in
writing, to:
IBM World Trade Asia Corporation
Licensing
2-31 Roppongi 3-chome, Minato-ku
Tokyo 106-0032, Japan

The following paragraph does not apply to the United Kingdom or any other
country/region where such provisions are inconsistent with local law:
INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS
PUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS
FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or
implied warranties in certain transactions; therefore, this statement may not apply
to you.

This information could include technical inaccuracies or typographical errors.


Changes are periodically made to the information herein; these changes will be
incorporated in new editions of the publication. IBM may make improvements
and/or changes in the products and/or the programs described in this publication
at any time without notice.

Any references in this information to non-IBM Web sites are provided for
convenience only and do not in any manner serve as an endorsement of those Web
sites. The materials at those Web sites are not part of the materials for this IBM
product, and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it
believes appropriate without incurring any obligation to you.

© Copyright IBM Corp. 2003, 2004 127


Licensees of this program who wish to have information about it for the purpose
of enabling: (i) the exchange of information between independently created
programs and other programs (including this one) and (ii) the mutual use of the
information that has been exchanged, should contact:
IBM Corporation
J46A/G4
555 Bailey Avenue
San Jose, CA 95141-1003
U.S.A.

Such information may be available, subject to appropriate terms and conditions,


including in some cases payment of a fee.

The licensed program described in this document and all licensed material
available for it are provided by IBM under terms of the IBM Customer Agreement,
IBM International Program License Agreement, or any equivalent agreement
between us.

Any performance data contained herein was determined in a controlled


environment. Therefore, the results obtained in other operating environments may
vary significantly. Some measurements may have been made on development-level
systems, and there is no guarantee that these measurements will be the same on
generally available systems. Furthermore, some measurements may have been
estimated through extrapolation. Actual results may vary. Users of this document
should verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of


those products, their published announcements, or other publicly available sources.
IBM has not tested those products and cannot confirm the accuracy of
performance, compatibility, or any other claims related to non-IBM products.
Questions on the capabilities of non-IBM products should be addressed to the
suppliers of those products.

All statements regarding IBM’s future direction or intent are subject to change or
withdrawal without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily business
operations. To illustrate them as completely as possible, the examples include the
names of individuals, companies, brands, and products. All of these names are
fictitious, and any similarity to the names and addresses used by an actual
business enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs, in source language, which


illustrate programming techniques on various operating platforms. You may copy,
modify, and distribute these sample programs in any form without payment to
IBM for the purposes of developing, using, marketing, or distributing application
programs conforming to the application programming interface for the operating
platform for which the sample programs are written. These examples have not
been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or
imply reliability, serviceability, or function of these programs. You may copy,
modify, and distribute these sample programs in any form without payment to
IBM for the purposes of developing, using, marketing, or distributing application
programs conforming to IBM’s application programming interfaces.

128 DB2 II Operations Guide for Classic Event Publishing


Each copy or any portion of these sample programs or any derivative work must
include a copyright notice as follows:

© (your company name) (year). Portions of this code are derived from IBM Corp.
Sample Programs. © Copyright IBM Corp. _enter the year or years_. All rights
reserved.

Trademarks
The following terms are trademarks of International Business Machines
Corporation in the United States, other countries, or both:

IBM
CICS
DB2
IMS
MVS
VTAM
WebSphere
z/OS

The following terms are trademarks or registered trademarks of other companies:

Java and all Java-based trademarks and logos are trademarks or registered
trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of
Microsoft Corporation in the United States, other countries, or both.

Intel, Intel Inside (logos), MMX and Pentium are trademarks of Intel Corporation
in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other
countries.

Other company, product or service names may be trademarks or service marks of


others.

Notices 129
130 DB2 II Operations Guide for Classic Event Publishing
Index
A CACRCV data set 42
Central Version.
correlation service (continued)
restarting 15
APF authorization See CA-IDMS Central Version Service Info Entry 5
of CA-IDMS.LOADLIB 25 Change capture starting
enabling CA-IDMS 24
for an IMS database/segment 13 stopping 15
C filtering by database 23 tracing 7
CA-IDMS type of IMS log records used 13
accessing multiple Central change-capture agent
Versions 23 CA-IDMS 17
configuring
D
APF authorization 25 Database synchronization
automatic archiving 20 CICS VSAM 87
CA-IDMS 20
automatically checking for recovery determining status 47
VSAM 85
mode 29 moving from recovery to active
Databases
Central Version vs. local mode 33 mode 61
mapping data structures 1
change-capture agent 17 recovery mode
shutting down or halting 15
configuring error messages for restoring active status 30, 31, 32
DB2 Information Integrator Classic Event
missing correlation service 26 starting 29, 87
Publisher
creating a recovery point file 26 setting into recovery mode 57
configuration parameters
customizing error messages 27 starting
format 109
direct data access 24, 25 CA-IDMS 26, 32
Data Mapper 1
filtering change capture by CICS VSAM 86
DFSFLGX0 37, 41
database 23 stopping
JCL CA-IDMS 28
modifying 19 CICS VSAM 87
journal files 20 VSAM 85 E
maintaining a journal starting CICS Environments
point 18 configuring file definitions 86 supported by IMS for change
maintaining state 18 CICS VSAM capture 37
mapping data 20, 24 See also VSAM Error messages
mapping paths 21, 22 configuring a change-capture configuring for missing correlation
modifying Central Version JCL 20 agent 87 service
recovery encrypting metadata utility CA-IDMS 26
starting 18 password 4 customizing 27
writing starting point to log 18 mapping data 89
recovery agent 19 running the metadata utility 90
recovery mode 32 starting a change-capture agent 86
starting a recovery change-capture
F
recovery procedures 28 Filtering captured records
running the metadata utility 22, 23 agent 87
with record selection exit 2
server setup 25 stopping a change-capture agent 87
starting a correlation service 24 CICSUID DD 4
starting a recovery Change Capture Connections, maximum for a correlation
Agent 29 service 7 I
starting an active change-capture correlation service Idle time out, correlation service 7
agent 26, 32 calling record selection exit 2 IDMSDBIO, relinking 17
stopping a change-capture agent 28 communicating with IMS 38 IDMSJNL2 17
switching to active mode 32 configuration 5 IDMSJNL2 exit
synchronization 20 configuring setting up 27
CA-IDMS Central Version TCP/IP 7 IMS
accessing multiple from a single creating recovery data sets 12 abnormal termination 42
server 23 defining protocol 6 activating change capture for a
issues with multiple 24 defining queue name 6 database/segment 13
modifying JCL 20 duplicated buffers 41 Active Agent Status Job 50, 59
setting up a server to access 25 error messages for missing active change-capture agent 37
stopping 28 CA-IDMS 26 CACRCV data set 42
vs. local mode 33 idle time out 7 cascade delete 67, 68
CA/CA-IDMS. maximum connections 7 change-capture agent
See CA-IDMS minimum or maximum tasks 7 recovery mode 38
CACE2TRM 15 receiving XM data grams from checkpoints 56
CACEC1DR load module 19 IMS 38 common memory 38
CACRCSEL module 1 response time out 7 CSA 38

© Copyright IBM Corp. 2003, 2004 131


IMS (continued)
determining change-capture agent
L Service Info Entry
correlation service 5
status 47 LD TEMP SPACE parameter 110 SERVICE INFO ENTRY parameter 113
exact restart point 45 LIST.LOG ALL 47, 52 SIE
log archive JCL 49 Log file tracking 47, 49 See Service Info Entry
log file tracking 47, 49 data stored 48 SMF exit
data stored 48 Implementing 49 parameter 114
Log file tracking Log tracking utility, command-line STATIC CATALOGS parameter 114
Implementing 49 parameters 49 Synchronization
log tracking utility, command-line CA-IDMS 20
parameters 49 VSAM 85
operational environment 39 M
recovery Mapping data
creating data set manually 62
determining available log files 50
CA-IDMS 20, 24 T
CICS VSAM 89 TASK PARAMETERS 114
determining log files to use 47 VSAM 85 TCP/IP
going to active mode 61 Maximum connections, correlation configuring
incremental 58 service 7 for correlation service 7
incremental mode 53 Maximum tasks, correlation service 7 Tracing
log file recovery process 60 MESSAGE POOL SIZE parameter 111 correlation service 7
log records of interest 66 metadata catalog 17 Type 99 Data Capture Log records 66
setting agents into 57 metadata utility 3
single DB/DC or DBCTL error code 4 22
subsystem 54 running
unrecoverable situations 65 CA-IDMS 22, 23 V
with multiple agents 58 CICS VSAM 90 VSAM
Recovery Minimum tasks, correlation service 7 See also CICS VSAM
Log file recovery process 53 Multiple record layouts maintaining state 85
recovery change-capture agent control handling 1 VSAM AMPARMS parameter 115
file 43
recovery data sets 44
naming standard 44
recovery mode
N
processing 43, 46 NL CAT parameter 111
restart point 43, 45 NL parameter 111
running without correlation
service 42
sending XM data grams 38 P
Status Control File 50 Program types
supported environments and program supported by IMS for change
types 37 capture 37
synchronization 43 Protocol
type 06 records 41 defining for correlation service 6
Type 99 Data Capture Log records 66
type of log records used for change
capture 13
unknown agents 51
Q
XM data grams 41 Queue name, defining for correlation
IMS Log Archive Utility service 6
suppressing IMS record types 66
IMS Logger Exit 37
IMS/VS Accounting Record 41 R
Initial synchronization Record layouts
CA-IDMS 20 handling multiple 1
VSAM 85 Record Selection Exit 1, 3
INTERLEAVE INTERVAL parameter 112 linking 2
Recovery data sets
creating 12
J Recovery point file
JCL creating 26
customizing Response time out, correlation service 7
accessing CA-IDMS databases in
local mode 24
S
SAF exit
parameter description 113

132 DB2 II Operations Guide for Classic Event Publishing


Contacting IBM
To contact IBM customer service in the United States or Canada, call
1-800-IBM-SERV (1-800-426-7378).

To learn about available service options, call one of the following numbers:
v In the United States: 1-888-426-4343
v In Canada: 1-800-465-9600

To locate an IBM office in your country or region, see the IBM Directory of
Worldwide Contacts on the Web at www.ibm.com/planetwide.

Product information
Information about DB2 Information Integrator is available by telephone or on the
Web.

If you live in the United States, you can call one of the following numbers:
v To order products or to obtain general information: 1-800-IBM-CALL
(1-800-426-2255)
v To order publications: 1-800-879-2755

On the Web, go to www.ibm.com/software/data/integration/db2ii/support.html.


This site contains the latest information about:
v The technical library
v Ordering books
v Client downloads
v Newsgroups
v Fix packs
v News
v Links to Web resources

Comments on the documentation


Your feedback helps IBM to provide quality information. Please send any
comments that you have about this book or other DB2 Information Integrator
documentation. You can use any of the following methods to provide comments:
v Send your comments using the online readers’ comment form at
www.ibm.com/software/data/rcf.
v Send your comments by e-mail to comments@us.ibm.com. Include the name of
the product, the version number of the product, and the name and part number
of the book (if applicable). If you are commenting on specific text, please include
the location of the text (for example, a title, a table number, or a page number).

© Copyright IBM Corp. 2003, 2004 133


134 DB2 II Operations Guide for Classic Event Publishing


Printed in USA

SC18-9157-02
Spine information:

 IBM DB2 Information Integrator DB2 II Operations Guide for Classic Event Publishing Version 8.2