Oracle Data Integration Solutions

GoldenGate New Features Summary

Valarie Bedard
Principle Sales Consultant

Tony Velardo
DIS Account Manager
Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Oracle Data Integration

Moving Data
to Transform Business
1

Note to Reader - March 2014

This deck represents highlights of features in OGG 11.2 and

OGG 12.x
It does not represent ALL features in either of these releases
It is recommended that the reader review the Release Notes for
either version of OGG to get the complete picture of new features
per release
It is recommended that the reader review the OGG Install Guide,
Admin Guide, and Reference Guides for either version to
understand dependencies, usage and syntax for any functionality
or feature mentioned in this deck.

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Quick List of OGG Database Integration points

Ability to ignore triggers when applying data (OGG V 12 default)

Ability to ignore constraints (like cascade delete) when applying data
Optimized ASM capture - DBLOGREADER
Access to Oracles Encryption Methods
Transparent Data Encryption
Tablespace Encryption
RMAN Integration log retention
Automatic Schema level supplemental logging
Integrated Capture - Capture compressed data including EHCC
Database Vault access
Oracles AES encryption algorithms for trail files
OEM Grid Control Plug-in for Monitoring, control and metrics
Exadata DBFS optimizations for GoldenGate

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Oracle GoldenGate 11g Release2

Integrated Extract
Conflict Detection and Resolution
Manageability and Monitoring
Security and Performance
Globalization
Event Marker Enhancement
Coherence Integration

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Integrated Extract
Leverages Proprietary Oracle Database LCR API
Source database and Integrated Extract process are in the same machine
User/Application changes

Record
changes

Capture
changes

Redo Logs

Oracle Database

LCR1
LCR2
LCR3
.
.
.

Logical Change
Records

Capture

Trail
File

..

Integrated
Extract

Source Machine

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Integrated Extract Benefits - Summary

Category
Exadata
Compressions
Distributed Transactions
RAC
New Datatypes
LOB
REDO Processing

New Features
Capture support for EHCC compression

Comments
Enable capture from Exadata sources

Capture support for OLTP, Segment compression

XA-RAC, PDML

Support capture from XA transactions

originated from multiple nodes of RAC

Simplified RAC management

No need to modify parameter files, if an

instance is not available

XML OR, XML Binary

Full and Partial read from REDO log
Multithreaded support

Performance gain from parallel processing

Deployment

Source and Downstream capture support

Helps to offload some of the extract work

from the source database;

DDL

Captures tables created with column level

password specification

Others

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Support IOT with MAPPING table option

Integrated Extract Expanded Deployment Options

Source database and Integrated Extract process are in different machine
User/Application changes

Record
changes
Continuous REDO
Shipping
Redo Logs

Oracle Database
Source Machine

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Capture
changes

LCR1
LCR2
LCR3
.
.
.

Capture

Standby
Redo Logs

Logical Change
Records

Capture

Trail
File

Integrated
Extract

Oracle Database
Downstream Machine

..

Conflict Detection and Resolution

Conflicts occur when the timing of simultaneous changes results in one of these
out-of-sync conditions:
a replicated insert attempts to add a row that already exists in the target.
the before image of a replicated update does not match the current row in the target.
a replicated delete attempts to remove a row that does not exist in the target.

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Conflict Detection and Resolution

Overwrite

Insert

INSERTROWEXISTS

Ignore
Discard
USEMIN,USEMAX

Overwrite

UPDATEROWMISSING

Ignore
Discard

Update

Overwrite
Ignore

UPDATEROWEXISTS

Discard
USEMIN,USEMAX

USEDELTA

DELETEROWMISSING

Ignore
Discard

Delete
Overwrite

DELETEROWEXISTS

Ignore
Discard

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Performance - Streaming
Prior to Oracle GoldenGate 11gR2
Synchronous data communication between data pump
extract and remote collector process
Data sent by data pump extract to remote collector
process must be followed with ACK from the remote
collector. Extract can only send out new data after ACK.

Oracle GoldenGate 11gR2

Acknowledgement message can be skipped for every
data sent
Remote collector does not send an acknowledgement
to the data pump extract unless a packet contains a
flag requesting a response, typically when the data
pump extract must checkpoint or determine a write
position.

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

10

Security Enhancements: Encryption

GGSCI

GGSCI
Manager

Interactive
Interface

Manager

Interactive
Interface

LAN / WAN / INTERNET

Over TCP/IP

Source
Database

IPv6 support

Target
Database
PASSWORDS

PASSWORDS

Parameter Files

TCP/IP
Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Parameter Files

BLOWFISH, AES128, AES192,AES256

11

Globalization
Database Object Name Enhancement
Non US-ASCII Character Support (both DDL/DML)
Support database object name with any characters including European accent, multi-byte,
white space and symbols as long as database supports

Database Like Object Level Case Sensitivity Support

Oracle, DB2 and SQL/MX case sensitive object name
Case sensitive from/to case insensitive mapping

Partial Character Set Conversion

Implicit Character Set Conversion Support by replicat
Between two different character set CHAR/VARCHAR/CLOB columns
CHAR/VARCHAR/CLOB from/to NCHAR/NVARCHAR/NCLOB columns

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

12

Event Marker System 11gR2

ACTIONS
TRACE
LOG
CHECKPOINTBEFORE
IGNORE
DISCARD
ROLLOVER
REPORT

User/Application changes

ABORT
CHECKPOINTAFTER
FORCESTOP/STOP

Event
record

Event
record

SHELL with parameters

SUSPEND/RESUME

Event
record

DDL

LAN / WAN / INTERNET

Over TCP/IP

Source
Database

Transaction Logs

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Target
Database

13

Coherence HotCache
Coherence/GoldenGate Integration
Capture changes from DB
Pump the data to coherence node,
using Java
Toplink maps relational changes to
the objects
Updates the cache
User will see the latest data in their
cache in Real-Time after the
database is updated

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

14

Oracle GoldenGate 12c Highlights

Optimized for Oracle Database 12c

Non-consolidated, Multitenant and Cloud-based Real-Time Replication

Three part naming convention, New data types (Large VARCHAR)
Integrated Delivery for the Oracle Database

Leveraging high-speed, lightweight Streaming API built Exclusively for OGG to take
advantage of database parallel apply server for auto dependency aware parallel apply
Coordinated Apply for Non-Oracle Databases

Orchestrates the High-Speed Apply Processes & Simplifies Setup and Management
Improved Ease of Use

Automatic Discard File, Enhanced Debugging, Schema Wildcarding, and Improved Security
Expanded Heterogeneity

12c Brings Support for New Databases and Enhancements to Existing Supported Platforms
Enhanced High Availability for Zero Data Loss

Integration with Data Guard and FSFO for Automated & Transparent failover of Components
Expanded Oracle Application and Technology Support

Active/Active ATG, Low Downtime E-Business Suite Migrations and Coherence Integration

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

15

Oracle GoldenGate 12.1.2 New Functionality

New with OGG 12.x

Oracle Installer
Oracle database Version 12 CDB support
3 part object name support
Automatic Discard
Improved initial load with added data type support for remote task
Improved SHOWSYNTAX
Schema Wildcard Support
Credential store for userid/(encrypted)password >> Alias
Source time zone in trail, replicat sets its session to this
Extract start at CSN ( Oracle SCN)
Oracle dbms_goldengate_auth.grant-admin_priviledge Package

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

16

16

Integrated Replicat
Overview
Integrated Replicat for Oracle target databases only
12.1.0.1 and 11.2.0.4
Leverages database parallel apply servers for automatic dependency
aware parallel apply
Minimal changes to replicat configuration
Single replicat parameter file for all tables
Single Replicat, no need to use @RANGE or THREAD or other
splitting

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

17

17

Integrated Replicat
Architecture Diagram
Oracle Database
Inbound Server
Lightweight
Streaming API

Receiver

Applier 1
Preparer

Coordinator
Applier N

Replicat

Replicat

Inbound Server (Database Apply Process)

Reads the trail file

Receiver: Reads LCRs

Constructs logical change

records (LCRs)

Preparer: Computes the dependencies between the transactions (primary key, unique
indexes, foreign key) , grouping transactions and sorting in dependency order.

Transmits LCRs to Oracle

Database via the Lightweight
Streaming API

Coordinator: Coordinates transactions, maintains the order between applier processes.

Applier: Performs changes for assigned transactions, including conflict detection and
error handling.

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

18

Integrated Replicat - Dependency Aware Apply

Source Database Supplemental Logging

Supplemental logging of scheduling columns (PK, UI, FK) needed for dependency aware
apply processing at target
New default of ADD TRANDATA is to log all unique indexes and foreign key columns .
If Primary Key, Unique Index, Foreign Key columns are same at both source and
target, ADD TRANDATA provides requisite information. If target has different
scheduling columns, then ADD TRANDATA [table] COLS format should be
used to identify any additional columns
For old behavior specify NOSCHEDULINGCOLS option to only log primary key

ADD TRANDATA [table], NOSCHEDULINGCOLS

ADD SCHEMATRANDATA [schema], NOSCHEDULINGCOLS

Scheduling columns must be present in trail file

Extract parameter LOGALLSUPCOLS
Includes any supplementally logged column in trail file
Automatically enables GETUPDATEBEFORES and NOCOMPRESSDELETES

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

19

19

Converting to Integrated Replicat

Convert to integrated mode from existing nonintegrated replicat
GGSCI> STOP REPLICAT repdb
GGSCI> ALTER REPLICAT repdb, INTEGRATED
Updates checkpoint file to indicate INTEGRATED
DB checkpoint table is not maintained when using integrated replicat

Consolidate the parameter files into a single file

Start replicat with proper parameter file

GGSCI> START REPLICAT repdb
Registers replicat repdb1 database
Starts and attaches to inbound server
INFO REPLICAT repdb will show INTEGRATED in output

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

20

20

Coordinated Replicat
Applicable to all databases
Full barrier coordination across events requiring ordering

DDL (Oracle only)

Primary Key Update
EMI
SQLEXEC

Ease of Use
Simplify parameter file manageability (fewer param files)
Simplify error recovery

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

21

Unified Parameter File (THREAD)

MAP sales.acct1, TARGET sales.acct1;

MAP sales.acct2, TARGET sales.acct2;

MAP sales.acct1, TARGET sales.acct1,

THREAD(1);
MAP sales.acct2, TARGET sales.acct2,
THREAD(2);
MAP sales.acct,3 TARGET sales.acct3,
THREAD(3);

MAP sales.acct3, TARGET sales.acct3;

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

22

Unified Parameter File (THREADRANGE)

MAP sales.acct, TARGET sales.acct, FILTER
(@RANGE (1, 3, ID));

MAP sales.acct, TARGET sales.acct, FILTER

(@RANGE (2, 3, ID));

MAP sales.acct, TARGET sales.acct,

THREADRANGE(1-3, ID));

MAP sales.acct, TARGET sales.acct, FILTER

(@RANGE (3, 3, ID));

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

23

Key Concepts
Coordinated vs. Integrated Replicat
Coordinated Replicat

Integrated Replicat

User Specified Partitioning

Automatic Scheduling

Split transaction Semantics

Does not split transactions

All databases

Oracle specific with version requirements

Continues to employ SQL generation and

execution outside the database.

Apply is in the database server. Tight

integration with Oracle database.

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

24

24

Integrated Extract
Overview
Integrated Extract was introduced in Oracle GoldenGate 11.2

Oracle GoldenGate 12.1.2 enhances the feature.

With Oracle GoldenGate 12.1.2
By default, all newly created extracts will be integrated extract
Upgrading from old IE you can use TRANLOGOPTION _LCRCAPTUREPROTOCOL
V2
_LCRCAPTUREPROTOCOL V2 provides better performance by
Streamlined for improved performance (V2 as fast or faster than Classic Extract)
Drastic improvement in LOB performance
Reduced overhead on database
Available when the mining server is 11.2.0.3 BP 14 and higher or Oracle 12c

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

25

25

Integrated Extract
DDL Trigger Removal
Requirements:
Database must be running Oracle Database 12c (12.1.0.1), 11.2.0.4 and later versions.
DDL Trigger Removal is required for CDB.

Behavior
Integrated Extract running against Oracle Database 12c or 11.2.0.4 will automatically run in this
mode
No need to run DDL setup scripts
DDL trigger can remain enabled
Integrated Extract running against Oracle Database 11g Release 11.2.0.3 will get metadata
gathered from DDL trigger
Classic Extract will continue to require existence of DDL trigger

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

26

Integrated Extract and Replicat Review

Oracle Database Only Support
Integrated Extract
Oracle GoldenGate 12.1.2: the source database can be 11.1 or 11.2 or above. ... the MINING
database has to be version newer than the source database, but must be 11.2.0.3 BP 6 or higher.
Same as OGG 11.2.1.
If the source database is Oracle DB 12.1, then OGG 12.1.2 must be used.
For Oracle Database 12c with CDB setup, integrated extract is required.
Integrated Replicat
Target database has to be Oracle Database 12.1.0.1 and 11.2.0.4.
The source database (Extract) can be other databases but have to include required supplemental
logging information for dependency calculation.
Oracle GoldenGate 11.1.1 and 11.2.1 versions or non Oracle source databases, use 2
parameters: GETUPDATEBEFORES and NOCOMPRESSDELETES, these are required.
For Oracle GoldenGate 12.1.2 use LOGALLSUPCOLS and UPDATERECORDFORMAT COMPACT.
This works for classic and integrated extract only.

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

27

Oracle GoldenGate 12c Database Support

Oracle Database
Oracle Database 12c
Multitenant Container Database (CDB)
More Datatypes: Large VARCHAR2

Other Databases
MySQL 5.6 & NDB 7.x Clusters
Sybase 15.7
DB2/LUW 10.1

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

28

28

Join the Data Integration Community

LinkedIn
Oracle Data Integration

Twitter or Facebook
twitter.com/ORCLGoldenGate
facebook.com/OracleDataIntegration

Oracles Data Integration blog

blogs.oracle.com/dataintegration

Oracle GoldenGate YouTube channel

youtube.com/oraclegoldengate
MOS Knowledgebase
http://support.oracle.com/
Oracle Learning Library
http://apex.oracle.com/pls/apex/f?p
=44785:1:0::NO

Oracle
Technology
Network

Oracle
Technology
Network

OTN: GoldenGate
http://www.oracle.com/technetwork/middle
ware/goldengate/overview/index.html
Oracles Public Help Forum
http://forums.oracle.com/forums/forum.j
spa?forumID=860&start=0

Oracle.com/goto/dataintegration

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

29

Copyright 2013, Oracle and/or its affiliates. All rights reserved.

30