Log Apply for CICS/VSAM Concepts and Facilities

2007 E-Net Corporation 1

Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 1
Version 3 Version 3 Version 3 Version 3
Concepts and Facilities Concepts and Facilities Concepts and Facilities Concepts and Facilities
January 2007 January 2007 January 2007 January 2007
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 2
Log Apply for CICS/VSAM
RRDF feature module
RRDF provides log capture and transport services
Totally real-time apply process
Working with new RRDF RealTime Reformat (RTR)
architecture
Objective is to achieve sub-second latency during normal
operations
Synchronization with batch (offline) activities via VSAM
Backup Synchronization (VBS) feature
Uses CICS auto-journals or forward recovery logs
Log streams captured using Resource Manager interface or
Browse Interface
Supports Record Level Sharing via Browse Interface
Requires RRDF Version 3
In production at major account
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 2
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 3
VSAM Replication
using Log Apply for CICS/VSAM Feature Module
CICS/TS
VSAM
Files
RRDF Send Region
Remote
VSAM
Files
RRDF Receive Region
CICS/TS region
Log Apply
for
CICS/VSAM
Local Server Remote Server
Logger
RRDF Archive Job
RRDF RealTime
Reformat Region
Disk Journals AJNLs Logstreams
RRDF Reformat and Receive enhanced to run continuously
Log data is fed immediately from receive region to RealTime Reformat region
If sequence continuity is OK, then change data is passed directly to CICS apply transaction
AJNL files are still maintained and used for recovery and gap situations
Network
Connection
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 4
Logstream Capture using System Logger
Resource Manager Interface
MVS Logger MVS Logger MVS Logger MVS Logger
Cross Cross Cross Cross- -- -memory memory memory memory
services services services services
Log Log Log Log
Streams Streams Streams Streams
CICS / TS CICS / TS CICS / TS CICS / TS
CF
RRDF Send RRDF Send RRDF Send RRDF Send
Region Region Region Region
Resource Resource Resource Resource
Manager Manager Manager Manager
Interface Interface Interface Interface
Write
Log
Block
From
Buffer
Handle
Log
Writes
Queue Queue Queue Queue
DBMS exit DBMS exit DBMS exit DBMS exit
Copies log Copies log Copies log Copies log
block into block into block into block into
RRDF send RRDF send RRDF send RRDF send
region region region region
queue queue queue queue
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 3
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 5
Logstream Capture using System Logger
Browse Interface
MVS Logger
Cross-
memory
services
Log
Streams
CICS TS
CF
RRDF Send
Region
Write
Log
Block
From
Buffer
Handle
Log
Writes
DBMS exit
Browses log
blocks (using
IXGBRWSE) and
inserts them
into RRDF send
region queue
Queue
Browse Interface is
appropriate for any
logstream
Permits RRDF Send
Region to run on different
z/OS image
Required for logging
associated with RLS files
Browse Browse Browse Browse
Interface Interface Interface Interface
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 6
RRDF RealTime Reformat
Log Apply for
CICS/VSAM
Control Data
address space
Receive
Region
Receive Gap
Job
Archive Job
RealTime
Reformat
Region
Disk
Journal
T-record
dataspace
High Global
timestamp information
AJNL
File
Shadow DB2 databases
Shadow VSAM files
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 4
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 7
Log Apply for CICS/VSAM
Apply Transactions
Dataspace
. . . . . . . .
RTR
Reformat
Started
Task
PLTPI Task
(master task)
Daughter
Transaction
File A
Daughter
Transaction
File B
Daughter
Transaction
File C
CICS Region(s)
KSDS
File A
ESDS
File B
KSDS
File C
Apply tasks are
automatically initiated
during CICS system
startup
Target files may or may
not be journaled
Target files may be
inquired against at any
time
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 8
Special Features
Customizable and tunable log capture
Number of browse tasks and polling (no-data wait) intervals are under
user control
Partitioning capability
Active files can be logically partitioned by key range or other user-
defined algorithm via user exit (partitioning exit)
Apply processes work in parallel with as many partitions as desired
Fuzzy apply logic
VSAM logical errors need not be fatal!
Permits user to resolve logical conflicts gracefully
Can be tailored to exploit application-specific consistency tokens
User number (update sequence token) capability
Truncation groups
Enable logical consistency among groups of VSAM files and their
associated logstreams
Automated recovery
Checkpoint/restart for apply tasks
Fully integrated with RRDF spilling and gap recovery processes
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 5
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 9
ENT3 Statistics Latency by File/Partition
ENT3 STAT ENT01K
FILE NAME: ENT01K TARGET FILE NAME: ENT06K DATE: 2004033
INTERVAL COUNT : 0001000 TARGET TIME : 0000100 TIME: 08432163
INTERVAL QDEPTH: 0000041 HISTORICAL QDEPTH: 0000041 BLKID: 00000000162A61
PART INTERVAL INTERVAL INTERVAL HISTORICAL HISTORICAL HISTORICAL LAST
NO BEST LONGEST OVER BEST LONGEST OVER RECORD
01 00000043 00001120 0000208 00000043 00001120 0000208 00000094
02 00000072 00001153 0000189 00000072 00001153 0000189 00000088
03 00000023 00001152 0000140 00000023 00001152 0000140 00000141
04 00000021 00001154 0000201 00000021 00001154 0000201 00000090
05 00000094 00001150 0000136 00000094 00001150 0000136 00000094
06 00000098 00001143 0000123 00000098 00001143 0000123 00000104
07 00000047 00001122 0000132 00000047 00001122 0000132 00000161
08 00000063 00001142 0000175 00000063 00001142 0000175 00000252
09 00000090 00001144 0000138 00000090 00001144 0000138 00000102
10 00000072 00001169 0000141 00000072 00001169 0000141 00000129
11 00000066 00001140 0000137 00000066 00001140 0000137 00000110
12 00000070 00001134 0000111 00000070 00001134 0000111 00000121
13 00000052 00001132 0000144 00000052 00001132 0000144 00000164
14 00000106 00001112 0000116 00000106 00001112 0000116 00000207
15 00000076 00001129 0000151 00000076 00001129 0000151 00000076
16 00000084 00001131 0000125 00000084 00001131 0000125 00000114
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 10
ENT3 Statistics I/O Activity by File/Partition
ENT3 DISP ENT01K
FILE NAME: ENT01K TARGET FILE NAME: ENT06K
PART WRITES DELETES UPDATES DUPLICATES SKIPPED ERRORS TOTAL
01 0000044 0000000 0000246 0000019 0000319 0000000 0000628
02 0000023 0000000 0000218 0000038 0000336 0000000 0000615
03 0000028 0000000 0000245 0000027 0000310 0000000 0000610
04 0000034 0000000 0000272 0000022 0000304 0000000 0000632
05 0000023 0000000 0000241 0000023 0000293 0000000 0000580
06 0000019 0000000 0000167 0000028 0000322 0000000 0000536
07 0000023 0000000 0000207 0000021 0000294 0000000 0000545
08 0000040 0000000 0000204 0000030 0000285 0000000 0000559
09 0000030 0000000 0000198 0000027 0000319 0000000 0000574
10 0000036 0000000 0000185 0000024 0000297 0000000 0000542
11 0000025 0000000 0000187 0000018 0000296 0000000 0000526
12 0000030 0000000 0000128 0000035 0000329 0000000 0000522
13 0000045 0000000 0000156 0000019 0000310 0000000 0000530
14 0000037 0000000 0000127 0000029 0000333 0000000 0000526
15 0000034 0000000 0000222 0000017 0000278 0000000 0000551
16 0000029 0000000 0000185 0000023 0000283 0000000 0000520
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 6
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 11
ENT3 Statistics RRDF Stats by File/Partition
ENT3 STAT ENT01K
FILE NAME: ENT01K TARGET FILE NAME: ENT06K DATE: 2004033
INTERVAL COUNT : 0001000 TARGET TIME : 0000100 TIME: 08440035
INTERVAL QDEPTH: 0000041 HISTORICAL QDEPTH: 0000041 BLKID: 0000000021E7DE
PART CURRENT CURRENT CURRENT CURRENT CURRENT LAST
NO DATE TIME DATAGROUP SEQUENCE QDEPTH RECORD
01 2004033 08440030 0000000002 0000008982 0000000 00001001
02 2004033 08440026 0000000002 0000008983 0000000 00001198
03 2004033 08440010 0000000002 0000008984 0000000 00001053
04 2004033 08435993 0000000002 0000008985 0000000 00000990
05 2004033 08435991 0000000002 0000008986 0000000 00000958
06 2004033 08440020 0000000002 0000008987 0000000 00001001
07 2004033 08440015 0000000002 0000008988 0000000 00000982
08 2004033 08440033 0000000002 0000008989 0000000 00001025
09 2004033 08440035 0000000002 0000008990 0000000 00001072
10 2004033 08440034 0000000002 0000008991 0000000 00001025
11 2004033 08440023 0000000002 0000008992 0000000 00000999
12 2004033 08440009 0000000002 0000008993 0000000 00001149
13 2004033 08440005 0000000002 0000008994 0000000 00000966
14 2004033 08440006 0000000002 0000008995 0000000 00001007
15 2004033 08435998 0000000002 0000008996 0000000 00000958
16 2004033 08440002 0000000002 0000008981 0000000 00000974
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 12
VSAM Batch Considerations
VSAM Backup Synchronization (VBS) feature
Permits groups of VSAM files to be restored from full
copies (backups) and be synchronized with shadowing
processes
Can be used with BMC RUV, IBM CICS/VR, or new E-Net
Log Apply for CICS/VSAM feature
Batch journaling support
Already supported in recovery on demand scenarios
Using either BMC Recovery Utility for VSAM (RUV) or IBMs
CICS/VR
Planned support for replication solution using RRDF
together with Log Apply for CICS/VSAM feature
DFSMStvs
Planning to support Transactional VSAM
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 7
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 13
Using Shadow Databases for Continuity & Availability
Normal Operations
Site 1 Site 2
Logs/Journals from Site 1
applied to shadow database at
Site 2
Shadow database available for
read-only queries/reports
Switch-over
Fail-over
Database at Site 1 unavailable -
shadow database becomes the
production database
Logs/Journals accumulated at
site 2 but cannot be transmitted
and/or applied
PROD
PROD
Catch-up Phase
Logs/Journals from Site 2
applied to (shadow) database at
Site 1
Gap recovery used to acquire
accumulated logs/journals
PROD
Switch-back
Fail-back
Logs/Journals from Site 1
applied to shadow database at
Site 2
Return to normal operations
complete
PROD
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 14
Case Study: Bi-Directional VSAM Replication
using RRDF and Log Apply for CICS-VSAM
CICS/TS
Production Site Server Backup Site Server
CICS/TS
Group A
Primary
RRDF
Send
RRDF
Recv/RTR/
Apply
RRDF
Send
RRDF
Recv/RTR/
Apply
Group A
Shadow
Group B
Shadow
Group B
Primary
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 8
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 15
Peer-to-Peer Replication
Multi-Master or Master-Master Configuration
Each site has its own set of critical VSAM files
Each site is in update mode at all times
Permits aggressive use of load balancing and multi-site transaction
routing
Exploits redundant network topologies
Recursiveness Avoidance
Log Apply for CICS/VSAM filters out any log data associated with its
own update activity
Avoids possibility of recursiveness in peer-to-peer configurations
Conflict Detection and Resolution
Apply conflicts may occur using peer-to-peer replication
No serialization or lock manager functional across multiple sites
Record Level Sharing serialization helps only within a single parallel sysplex
(i.e. within one site)
User exits in Apply component permit application-specific
customization
Permits user to detect and resolve logical conflicts gracefully
Can be tailored to exploit application-specific consistency tokens
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 16
Case Study: Peer-to-Peer VSAM Replication
using RRDF and Log Apply for CICS-VSAM
CICS/TS
Site A Production Server
CICS/TS
File 1
Site A Copy
RRDF
Send
RRDF
Recv/RTR/
Apply
RRDF
Send
RRDF
Recv/RTR/
Apply
Site B Production Server
File 2
Site B Copy
File 1
Site B Copy
File 2
Site A Copy
Log Apply for CICS/VSAM Concepts and Facilities
2007 E-Net Corporation 9
Log Apply for CICS/VSAM Concepts and Facilities
E-Net 2007 Slide 17
Concepts and Facilities Concepts and Facilities Concepts and Facilities Concepts and Facilities