Académique Documents
Professionnel Documents
Culture Documents
Agenda
Introduction Use Cases Break / VirtualBox Installation & Import Architecture Configuration Overview ands!on "a#s
'
<*his is a critical re+uirement to enable hi$h volume "ata e,traction to support real&time "ata warehousin$ an" -., as well as master "ata mana$ement an" other operational use cases() Rob /arel, Forrester0
#.t1s +uite a catch 2or Oracle in that it1s a si$ni2icant a""ition to its "ata warehousin$ an" -. port2olio() 3ames /obielus, Forrester
Oracle $olden$ate
The Solution for *nterprise4"ide !eal Time 9ata =eeds
5atabase an" applications, 6i,e" sources, "istribute" s%stems, le$ac%, O7*P, O78P
!eal4time information
6ission Critical 8pplications 5ata, -usiness .ntelli$ence, Reportin$ 2or Customers, Partners 9mplo%ees
!eal4time 1ccess Oracle 'ol"en'ate "elivers real&time access o2 real&time in2ormation, enablin$ companies to "ramaticall% improve the availabilit%, reliabilit%, an" per2ormance o2 critical "ata across enterprise s%stems(
H*T*!OG*=*OAS
GoldenGate
?ast S@-
S@- @uery
9ata Integrator
>
Oracle 9. Stand+y
*5ample of ho" a glo+al hu+ ; spoke manufacturing operation may +e deployed using +oth 1cti e 9ata Guard and GoldenGate
'ol"en'ate *ar$et
/$
Oracle GoldenGate
*4-T Transformation
Heterogeneous Targets
!eal4time 9ata
Heterogeneous Sources
Oracle GoldenGate
//
transactional s(stems
' .on!invasive on sources
On49isk -ogs
Oracle GoldenGate
9I6 ?1CT
9I6
Oracle GoldenGate
On49isk -ogs
*6P
9*PT
9I6
9I6
O9S Schema
9W Schema
/3
Bank of America
VocaLINK
/$$E paperless en ironment dri es efficiency and impro es patient care across hospital
Montefiore
Swedbank
/$$E a aila+ility during phased upgrade for 3'5: critical C!6 application
HSN
Speed H Su+second -atency Iolume H Thousands of TPS -og4+ased Capture =ati e, -ocal 1pply *fficient IO and .and"idth Asage .idirectional Group Transactions .ulk Operations Compression One4to46any, 6any4to4One Cascade
Transaction Integrity Transparent Capture Guaranteed 9eli ery Conflict 9etection, !esolution 9ynamic !oll+ack Incremental T96 Initial 9ata -oad GAI4+ased 6onitoring and Configuration Proacti e 1lerts *ncryption !eal4Time 9eferred or .atch * ent 6arkers
Heterogeneous 9ata Sources 6apping Transformation *nrichment 9ecoupled 1rchitecture Ta+le, !o", Column ?iltering G6-, 1SCII, S@?ormats @ueue Interface Stored Procedures Aser *5its *T- Integration Ja a8J6S Integration
Capture9 Oracle IB/ *B: /icrosoft &2" &erver &(#ase A&% Ingres ,eradata %nscri#e &2"//) &2"//; *eliver(9 All listed a#ove /(&2"< ) .eoview< .ete00a An( O*BC compati#le data#ases %," products =/& message >ueues or topics
?indows :@@@< :@@A< ;) "inux &un &olaris ) .on&top )!U; ) ,+UBC ) OpenV/& IB/ AI; IB/ 0/O& IB/ O&/AD@ IB/ O&/C@@
*5tensi+le ; ?le5i+le
!elia+le
16
/F
Agenda
Introduction Use Cases Break / VirtualBox Installation & Import Architecture Configuration Overview ands!on "a#s
Oracle $olden$ate Use Cases %nterprise!wide &olution for +eal ,ime *ata .eeds
' &tandardi0e on &ingle ,echnolog( for /ultiple .eeds ' *eplo( for Continuous Availa#ilit( and +eal!time *ata Access for +eporting / BI
MOpen ; 1cti eN
Operational !eporting
OGG
!eporting 9ata+ase
*TO9S *T*9W
!eal4time .I
*9W
@uery Offloading
' ighl( 1lexi#le ' 1ast *eplo(ments ' "ower ,CO & Improved +OI
9ata 9istri+ution
/0
Oracle 9ata+ases
Fare
Rule !p"ates
<-ookupsO
GoldenGateprovided the glue to move the data across systems, evolving our overall approach in parallel with the future of online travel commerce. - James arding, !" of #ystems "lanning $ "erformance, #abre
3$
CH1--*=G*S 8 OPPO!TA=ITI*S -arge, Comple5 ; Heterogeneous en ironment for /$$ +ack office applications =umerous installations that re2uire a fle5i+le solution& MOracle, Sy+ase ; 6S S@- Ser erN for +oth production and de elopment&
3/
Oracle
Oracle
Oracle
S@Ser er
%ith GoldenGate it was boom, boom, boom & full speed ahead. - 'mit (u)her*ee, #enior +irector of ,eporting #ervices, '+"
33
enterprise data warehouse for #usiness users 5customer claims processing for maMor incidents8 Integrate data from multiple operational s(stems 5*B: on 0/O& and "U?8 "everage existing %," investment to achieve near real!time data feedsL
GoldenGate SolutionL
million transactions per da( from heterogeneous operational sourcesL +eal!,ime C*C co!exists with Informatica %," for data re>uiring heav( transformationsL
5ual 8ctive
!esultsL
Teradata Warehouse
ena#led #( faster and more responsive emplo(ee intelligence on claims dataL Improved resource utili0ation efficienc( #( 7I /anagement team can now d(namicall( monitor reserve threshold to increase profita#ilit(L
*ata is not a #usiness pro#lemK It is the foundation of our a#ilit( to #e profita#le in a rapidl( changing marketplaceL - -ric "ost, +irector of +ata %arehousing, .iberty (utual
3)
Agenda
Introduction Use Cases Break / VirtualBox Installation & Import Architecture Configuration Overview ands!on "a#s
Cop( #oth ?orkshop *V*s to a single director( on (our local harddrive *ownload and install lateset VirtualBox version from http9//virtual#oxLorg
.ote9 Virtual#ox vAL:LH for ?indows ;) is included on *V*
After Installation9
Start Iirtual.+o5 ?ile 4> Import 1ppliance Choose &o f file
Agenda
Introduction Use Cases Break / VirtualBox Installation & Import Architecture Configuration Overview ands!on "a#s
Oracle $olden$ate,ransactional *ata /anagement9 )rimaril( used for change data capture and deliver( from data#ase transaction logs Can optionall( #e used for initial load directl( from data#ase ta#les %speciall( useful for s(nchroni0ing heterogeneous data#ases *ata#ase!specific methods ma( #e prefera#le for homogeneous configurations
Capture
Tar" et Trai&
De&i)er
D e&i)er
Tar" et Trai&
Capture
Bi di r ect i onal
'ource Data%ase
-ana"er
-ana"er
'ource Data%ase
Net*or. (TCP/IP) +,tract Transactio n Lo" Loca& Trai& Data Pump 'er)er Co&&ector Remote Trai& Rep&icat
Tar"et Data%ase
-ana"er
-ana"er
-ana"er
)arameter 1ile
mgrLprm file in $$& L/dirprm director(
+,tract
Capture changed data from data#ase logs *istri#ute data from local trails to remote s(stems 5data pump8 Capture data directl( from source ta#les for initial data load
Data Pump
*ata is stored in a local trail file on the source s(stem A second %xtract process< the *ata )ump9
+eads this trail and sends to one or more targets /anipulates the data or passes it through without change
Trai&
Primar +,tract
Trai&
Data Pump
Trai&
Trai&
Data Pump 1
Trai&
Primar +,tract
Trai&
Data Pump /
Trai&
Data Pump 0
Trai&
'er)er Co&&ector
A &erver Collector process reassem#les and writes the data to a $olden$ate remote trail A +eplicat process reads this trail and applies it to the target data#ase
Rep&icat
)0
%xtract trails and files are unstructured< with varia#le length records
I/O performed using large #lock writes
'$
,rails and files can #e transported online using ,C)/I) or sent in #atch using an( file transfer method
'/
'3
''
&tart "ogdump from $olden$ate installation director( using logdump command "ogdump is documented in the Oracle $olden$ate ,rou#leshooting and ,uning $uide %xample9
Logdump> open dirdat/rt00000
Initial "oad
'ource Data%ase Ta%&es +,tract Net*or. (TCP/IP) Rep&icat 2r D3 3u&. Load 4ti&it Tar"et Data%ase
'er)er Co&&ector
1i&es
-ana"er
-ana"er
199 *GT!1CT <name>, SOA!C*IST1.-* *5tract parametersL !6TT1SK !*P-IC1T, G!OAP <name>
199 *GT!1CT <name>, SOA!C*IST1.-* *5tract parametersL !6TT1SK !*P-IC1T, G!OAP <name>
?ile
?ile
?ile SSIS
199 *GT!1CT <name> *5tract parametersL SOA!C*IST1.-* !6T?I-* <name> ?O!61T1SCII .CP or S@--O19*!
Checkpointing
Capture< )ump< and *eliver( save positions to a checkpoint file so the( can recover in case of failure
.egin, TG / Insert, TG / .egin, TG 3 Apdate, TG / Insert, TG 3 Commit, TG 3 .egin, TG ) Insert, TG ) .egin, TG ' Commit, TG ) 9elete, TG ' .egin, TG 3 Insert, TG 3 Commit, TG 3 .egin, TG ) Insert, TG ) Commit, TG )
Capture Checkpoint
Source 9ata+ase
Capture
Checkpointing
Capture< )ump< and *eliver( save positions to a checkpoint file so the( can recover in case of failure
.egin, TG / Insert, TG / .egin, TG 3 Apdate, TG / Insert, TG 3 Commit, TG 3 .egin, TG ) Insert, TG ) .egin, TG ' Commit, TG ) 9elete, TG ' .egin, TG 3 Insert, TG 3 Commit, TG 3 .egin, TG ) Insert, TG ) Commit, TG )
Capture Checkpoint
Source 9ata+ase
Capture
Pump
Checkpointing
Capture< )ump< and *eliver( save positions to a checkpoint file so the( can recover in case of failure
.egin, TG / Insert, TG / .egin, TG 3 Apdate, TG / Insert, TG 3 Commit, TG 3 .egin, TG ) Insert, TG ) .egin, TG ' Commit, TG ) 9elete, TG ' .egin, TG 3 Insert, TG 3 Commit, TG 3 .egin, TG ) Insert, TG ) Commit, TG )
Start of Oldest Open MAncommittedN Transaction Pump Checkpoint 9eli ery Checkpoint
Capture Checkpoint
Source 9ata+ase
Capture
Pump
9eli ery
Target 9ata+ase
Agenda
Introduction Use Cases Break / VirtualBox Installation & Import Architecture Configuration Overview ands!on "a#s
'ource Data%ase
Tar"et Data%ase
*5tract
$olden$ate Installation
Access the /edia )ack
Access the product media pack 5software and documentation8 at http9//edeliver(LoracleLcom &elect )roduct )ack9 Oracle 1usion /iddleware Identif( the proper release of $olden$ate for (our source and target environments9
Operating s(stem and version *ata#ase and version
$olden$ate Installation
$olden$ate *irectories
*irector( dirchk dirdat dirdef dirpcs dirprm dirrpt dirs>l dirtmp Contents $olden$ate checkpoint files $olden$ate trail and extract files *ata definitions produced #( *%1$%. and used to translate heterogeneous data )rocess status files )arameter files )rocess report files &2" scripts ,emporar( storage for transactions that exceed allocated memor(
$olden$ate Configuration
)arameters< )rocess $roups and Commands
$olden$ate processes are configured #( A&CII parameter filesL A process group consists of9 An %xtract or +eplicat process Associated parameter file Associated checkpoint file An( other files associated with that process %ach process group on a s(stem must have a uni>ue group nameL )rocesses are added and started using the $olden$ate &oftware Command Interface 5$$&CI8 with the group nameL $$&CI commands also add trails< check process status< etcL
$olden$ate Configuration
$$&CI %xamples
&tart a /anager process
GGSCI> START MGR
$olden$ate Configuration
$"OBA"& versus )rocess )arameters $"OBA"& parameters appl( to all processes
&et when /anager starts +eside in Q$olden$ate install director(R/$"OBA"&
)rocess parameters appl( to a specific process 5/anager< %xtract< &erver Collector< +eplicat< Utilities8
&et when the process starts Override $"OBA"& settings +eside #( default in the dirprm director( in files named QprocessnameRLprm /ost appl( to all ta#les processed #ut some can #e specified at the ta#le level
$olden$ate Configuration
%diting )arameter 1iles
%dit parameter files to configure $olden$ate processes ,he $"OBA"& parameter file is identified #( its file path
GGSCI> EDIT (ARAMS &/GL!"ALS
%xtract and +eplicat parameter files are identified #( the process group name
GGSCI> EDIT (ARAMS *group name>
$olden$ate Configuration
1unctions ! *ata ,ransformation $olden$ate provides the capa#ilit( to transform columns #( using a set of #uilt!in functions ,ransformation functions can #e applied for either %xtract or +eplicat If (ou re>uire more complex functions< (ou also have the a#ilit( to call (our own logic through user exits
$olden$ate Configuration
1unctions ! *ata ,ransformation Using column conversion functions (ou can9
)erform string and num#er conversions %xtract portions of strings or concatenate columns Compare strings or num#ers )erform a variet( of date mappings Use single or nested I1 statements to evaluate num#ers< strings< and other column values to determine the appropriate value and format of target columns
$olden$ate Configuration
&ource *efinition Configuration
Problem J Understanding source and target la(outs across disparate s(stems and data#ases Solution J the *%1$%. utilit( program
*%1$%. produces a file containing la(out definitions of the source files and ta#les ,his source definition file is used to interpret la(outs for data stored in $olden$ate trails At start up +eplicat reads the definition file specified with the &OU+C%*%1& parameter &erver Collector uses the Jd argument to specif( which definition file to read at startup
Can also capture target definitions on target s(stem and cop( to source s(stem for %xtract to use
Agenda
Introduction Use Cases Break / VirtualBox Installation & Import Architecture Configuration Overview ands!on "a#s
Oracle Oracle OS -oginL oracle 8 oracle GG OS -oginL orgg 8 oracle GG HomeL 8u$/8oraclegg Oracle Target SchemaL orgg
Ta+le 1CCOA=T 1CCOA=TUT!1=S .!1=CH .!1=CHU1T6 T*--*! T*--*!UT!1=S !o"s $ $ $ $ $ $
Transaction Lo"
T*--*! T*--*!UT!1=S
Redo Lo"
Oracle Oracle OS -oginL oracle 8 oracle GG OS -oginL orgg 8 oracle GG HomeL 8u$/8oraclegg GG 9. -oginL orgg8orgg 8u$/8oraclegg
8dirchk 8dirdat 8dirdef 8dirpcs 8dirprm 8dirrpt 8dirs2l 8dirtmp SchemaL orgg
Ta+le 1CCOA=T 1CCOA=TUT!1=S !o"s $ $
Oracle Target
9ata+aseL 6P.1=K
Ta+le 1CCOA=T 1CCOA=TUT!1=S
Transaction Lo"
T*--*!UT!1=S
Redo Lo"
+,tract +L2AD
Rep&icat RL2AD
2rac&e Tar"et
+,tract +-A'TR
Rep&icat R-A'TR
Transaction Lo"
Loca& Trai&
!edo -og
Oracle Oracle OS -oginL oracle 8 oracle GG OS -oginL orgg 8 oracle GG HomeL 8u$/8oraclegg GG 9. -oginL orgg8orgg
+,tract +-A'TR
Transaction Lo"
Loca& Trai&
Remote Trai&
!edo -og
-ana"er -5R
-ana"er -5R
6yS@9ata+ase
!eplicat M6P!TN
!emote Trail
-ocal Trail
Transaction -og
*5tract MO!GTN
Transaction -og
-ocal Trail 9ata Pump M6P9PN !emote Trail
Oracle 9ata+ase
!eplicat MO!!TN
*5tract M6PGTN
"a# B9 ,ransformations
6yS@6yS@- OS -oginL mys2l 8 oracle GG OS -oginL mygg 8 oracle GG HomeL 8u$/8mys2lgg GG 9. -oginL mygg8mygg
WG*T*=I M<GGH*19*!O, <CO66ITTI6*ST16PON
Oracle Oracle OS -oginL oracle 8 oracle GG OS -oginL orgg 8 oracle GG HomeL 8u$/8oraclegg GG 9. -oginL orgg8orgg O!GG&1CCOA=TU.1-UHISTO!P ' 1CCOA=TU=A6.*!
' T!1=SUTS ' 1CCOA=TU.1-1=C*
"a# T9 1iltering
6yS@6yS@- OS -oginL mys2l 8 oracle GG OS -oginL mygg 8 oracle GG HomeL 8u$/8mys2lgg GG 9. -oginL mygg8mygg Oracle Oracle OS -oginL oracle 8 oracle GG OS -oginL orgg 8 oracle GG HomeL 8u$/8oraclegg GG 9. -oginL orgg8orgg
<$
' 1CCOA=TU.1-1=C*