Applying PSU Patch in a dataguard (Physical Standby) environment

Posted on September 20, 2012by Shivananda Rao P

4 Votes

Here is a brief explanation on how to apply PSU (Patch Set Update) in a dataguard environment In this demo, I am applying PSU 11.2.0.2.4 on the Primary and standby databases. Primary database Server : dev Standby database Server : uat Primary database : srprim Standby database : srps Primary Server: 1 2 3 4 5 6 7 8 9 10
Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production Copyright (c) 1982, 2010, Oracle. All rights reserved. SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 10:43:50 2012 [oracle@dev ~]$ sqlplus sys/oracle@srprim as sysdba

With the Partitioning, Automatic Storage Management, OLAP, DataMining and Real A

11 12 13 14 15 16 17 18

SQL> select status,instance_name,database_role from v$database,v$instance;

STATUS INSTANCE_NAME DATABASE_ROLE ------ ------------- --------------OPEN srprim PRIMARY

SQL> select max(sequence#) from v$archived_log;

MAX(SEQUENCE#)

19
--------------

20
10

21 Standby Server:
[oracle@uat ~]$ sqlplus sys/oracle@srps as sysdba

1 2
SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 10:46:35 2012

3 4
Copyright (c) 1982, 2010, Oracle. All rights reserved.

5 6
Connected to:

7
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

8 9

With the Partitioning, Automatic Storage Management, OLAP, DataMining and Real A

10 11 12 13 14 15

SQL> select status,instance_name,database_role from v$database,v$instance;

STATUS

INSTANCE_NAME

DATABASE_ROLE

-------- -------------- ---------------------MOUNTED srps PHYSICAL STANDBY

SQL> select max(sequence#) from v$archived_log where applied='YES';

16 17
MAX(SEQUENCE#)

18
--------------

19
10

20 21 Step 1: Disable the log shipping from primary database to the standby database by setting the log_archive_dest_state_2 to defer on the primary database. Here log_archive_dest_state_2 is deferred because parameter log_archive_dest_2 is set on my primary database to point to the Standby Database. 1 2 3
System altered. SQL> alter system set log_archive_dest_state_2=defer;

Step 2: On the standby database cancel the Managed Recovery Process.

SQL> alter database recover managed standby database cancel;

2 3

Database altered.

Step 3: PSU (Patch Set Update)/CPU (Critical Patch Update)/ Patch Set patches always needs to be applied first on the standby database and then on the primary database. In order to apply it on the standby database, shutdown the standby database and also the listener running on the standby server. 1
SQL> shutdown immediate

2 3 4 5 6 7 8 9 10 11 12 13

ORA-01109: database not open

Database dismounted. ORACLE instance shut down.

[oracle@uat ~]$ lsnrctl stop

[oracle@uat ~]$ ps -ef | grep tns oracle 6958 5107 0 10:52 pts/1 00:00:00 grep tns [oracle@uat ~]$ [oracle@uat ~]$ ps -ef | grep pmon oracle 4788 1 0 09:56 ? 00:00:00 asm_pmon_+ASM oracle 6960 5107 0 10:52 pts/1 00:00:00 grep pmon

14 Step 4: Now apply the PSU on the standby database.

1 2 3 4 5 6 7 8

<pre>[oracle@uat ~]$ export PATH=$PATH:/u01/app/oracle/product/11.2.0.2/db_1/OPa [oracle@uat ~]$ opatch version OPatch Version: 11.2.0.3.0

OPatch succeeded.

[oracle@uat ~]$ <span style="color: #ff6600;">opatch prereq CheckConflictAgainst Oracle Interim Patch Installer version 11.2.0.3.0 Copyright (c) 2012, Oracle Corporation. All rights reserved.

9 10
PREREQ session

11 12
Oracle Home : /u01/app/oracle/product/11.2.0.2/db_1

13
Central Inventory : /u01/home/oraInventory

14 15 16 17 18 19 20 21

from : /u01/app/oracle/product/11.2.0.2/db_1/oraInst.loc OPatch version : 11.2.0.3.0 OUI version : 11.2.0.2.0

Log file location : /u01/app/oracle/product/11.2.0.2/db_1/cfgtoollogs/opatch/opa

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

22 23 24 25 26 27 28
Oracle Home : /u01/app/oracle/product/11.2.0.2/db_1 [oracle@uat ~]$ opatch apply /opt/12827726/ Oracle Interim Patch Installer version 11.2.0.3.0 Copyright (c) 2012, Oracle Corporation. All rights reserved. OPatch succeeded.

29
Central Inventory : /u01/home/oraInventory

30
from : /u01/app/oracle/product/11.2.0.2/db_1/oraInst.loc

31
OPatch version : 11.2.0.3.0

32 33 34 35 36 37 38 39 40 41 42

OUI version : 11.2.0.2.0

Log file location : /u01/app/oracle/product/11.2.0.2/db_1/cfgtoollogs/opatch/128

Applying interim patch '12827726' to OH '/u01/app/oracle/product/11.2.0.2/db_1' Verifying environment and performing prerequisite checks... All checks passed.

Provide your email address to be informed of security issues, install and initia address/User Name. Visit http://www.oracle.com/support/policies.html for details. Email address/User Name:

You have not provided an email address for notification of security issues.

43 44 45 46 47 48 49 50
User Responded with: Y Is the local system ready for patching? [y|n] y Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y

Please shutdown Oracle instances running out of this ORACLE_HOME on the local sy (Oracle Home = '/u01/app/oracle/product/11.2.0.2/db_1')

51
Backing up files...

52 53
Patching component oracle.rdbms.rsf, 11.2.0.2.0...

54 55 56 57 58 59 60 61 62 63
Patching component oracle.rdbms.dv, 11.2.0.2.0... Patching component oracle.ldap.rsf, 11.2.0.2.0... Patching component oracle.sysman.oms.core, 10.2.0.4.3... Patching component oracle.sysman.console.db, 11.2.0.2.0...

Patching component oracle.rdbms, 11.2.0.2.0...

64 65 66 67 68 69 70 71 72
Verifying the update... Patching component oracle.sdo.locator, 11.2.0.2.0... Patching component oracle.rdbms.rman, 11.2.0.2.0... Patching component oracle.sysman.plugin.db.main.repository, 11.2.0.2.0... Patching component oracle.rdbms.dbscripts, 11.2.0.2.0...

73
Patch 12827726 successfully applied

74 75 76
OPatch succeeded.

Log file location: /u01/app/oracle/product/11.2.0.2/db_1/cfgtoollogs/opatch/1282

77 78 79 80 Step 5: Once the patch has been applied on the standby database, start the listener and the standby database. 1 2
[oracle@uat ~]$ lsnrctl start

3 4 5 6 7 8 9 10

[oracle@uat ~]$ sqlplus sys/oracle@srps as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 11:40:02 2012

Copyright (c) 1982, 2010, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup mount

11
ORACLE instance started.

12 13
Total System Global Area 684785664 bytes

14
Fixed Size 2229640 bytes

15 16 17 18 19

Variable Size 197134968 bytes Database Buffers 482344960 bytes Redo Buffers 3076096 bytes Database mounted.

Note: Do not run any patching scripts on the standby database (Example: catbundle.sql). We are done with the patching on the standby database. Now lets move to the primary database. Step 6: Shutdown the Primary database and stop the listener running on the primary database server.

1 2 3 4 5 6 7 8 9 10 11

<pre> [oracle@dev ~]$ sqlplus sys/oracle@srprim as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 11:48:26 2012

Copyright (c) 1982, 2010, Oracle. All rights reserved.

Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

With the Partitioning, Automatic Storage Management, OLAP, DataMining and Real A

SQL> shutdown immediate

12
Database closed.

13
Database dismounted.

14 15 16 17 18 19 20 21

ORACLE instance shut down. SQL> exit

Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bi

With the Partitioning, Automatic Storage Management, OLAP, DataMining and Real A

[oracle@dev ~]$ lsnrctl stop

[oracle@dev ~]$ ps -ef | grep pmon

22 23 24 25 26 27

oracle 4618 1 0 09:53 ? 00:00:00 asm_pmon_+ASM oracle 10233 4998 0 11:50 pts/1 00:00:00 grep pmon [oracle@dev ~]$ [oracle@dev ~]$ ps -ef | grep tns oracle 10237 4998 0 11:50 pts/1 00:00:00 grep tns

Step 7: Now apply the PSU patch on the Primary database. 1 2 3 4

OPatch succeeded. [oracle@dev ~]$ export PATH=$PATH:/u01/app/oracle/product/11.2.0.2/db1/OPatch [oracle@dev ~]$ opatch version OPatch Version: 11.2.0.3.0

5 6 7
Oracle Interim Patch Installer version 11.2.0.3.0

[oracle@dev ~]$ opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /opt/1

8
Copyright (c) 2012, Oracle Corporation. All rights reserved.

9 10 11 12 13 14
Oracle Home : /u01/app/oracle/product/11.2.0.2/db1 Central Inventory : /u01/home/oraInventory from : /u01/app/oracle/product/11.2.0.2/db1/oraInst.loc

PREREQ session

15 16 17 18 19 20

OPatch version : 11.2.0.3.0 OUI version : 11.2.0.2.0

Log file location : /u01/app/oracle/product/11.2.0.2/db1/cfgtoollogs/opatch/opat

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

21 22
OPatch succeeded.

23 24
[oracle@dev ~]$ export PATH=$PATH:/u01/app/oracle/product/11.2.0.2/db1/OPatch

25
[oracle@dev ~]$ opatch apply /opt/12827726/

26 27 28 29 30 31 32 33 34 35

Oracle Interim Patch Installer version 11.2.0.3.0 Copyright (c) 2012, Oracle Corporation. All rights reserved.

Oracle Home : /u01/app/oracle/product/11.2.0.2/db1 Central Inventory : /u01/home/oraInventory from : /u01/app/oracle/product/11.2.0.2/db1/oraInst.loc OPatch version : 11.2.0.3.0 OUI version : 11.2.0.2.0

Log file location : /u01/app/oracle/product/11.2.0.2/db1/cfgtoollogs/opatch/1282

36 37 38 39 40 41 42 43 44 45 46

Applying interim patch '12827726' to OH '/u01/app/oracle/product/11.2.0.2/db1' Verifying environment and performing prerequisite checks... All checks passed.

Provide your email address to be informed of security issues, install and initia address/User Name. Visit http://www.oracle.com/support/policies.html for details. Email address/User Name:

You have not provided an email address for notification of security issues. Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y

Please shutdown Oracle instances running out of this ORACLE_HOME on the local sy (Oracle Home = '/u01/app/oracle/product/11.2.0.2/db1')

47 48
Is the local system ready for patching? [y|n]

49
y

50
User Responded with: Y

51 52 53 54 55 56

Backing up files...

Patching component oracle.rdbms.rsf, 11.2.0.2.0...

Patching component oracle.rdbms, 11.2.0.2.0...

57 58 59 60 61 62 63 64

Patching component oracle.sysman.console.db, 11.2.0.2.0...

Patching component oracle.sysman.oms.core, 10.2.0.4.3...

Patching component oracle.ldap.rsf, 11.2.0.2.0...

Patching component oracle.rdbms.dv, 11.2.0.2.0...

Patching component oracle.rdbms.dbscripts, 11.2.0.2.0...

65 66
Patching component oracle.sysman.plugin.db.main.repository, 11.2.0.2.0...

67 68
Patching component oracle.rdbms.rman, 11.2.0.2.0...

69 70
Patching component oracle.sdo.locator, 11.2.0.2.0...

71 72 73 74 75 76 77
OPatch succeeded.

Verifying the update... Patch 12827726 successfully applied

Log file location: /u01/app/oracle/product/11.2.0.2/db1/cfgtoollogs/opatch/12827

78 79 80 Step 8: Start the listener on the primary database server and also start the Primary database. 1 2 3 4 5 6
Copyright (c) 1982, 2010, Oracle. All rights reserved. SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 12:28:35 2012 [oracle@dev ~]$ sqlplus sys/oracle@srprim as sysdba [oracle@dev ~]$ lsnrctl start

7 8
Connected to an idle instance.

9 10
SQL> startup

11
ORACLE instance started.

12 13
Total System Global Area 684785664 bytes

14 15 16 17

Fixed Size 2229640 bytes Variable Size 222300792 bytes Database Buffers 457179136 bytes Redo Buffers 3076096 bytes

18 19 20 21

Database mounted. Database opened. SQL>

Step 9: Now enable log shipping on the primary database by setting the log_archive_dest_state_2 to enable. As I said earlier, parameter log_archive_dest_2 on my primary database is set to point to the standby database. 1 2 3
System altered. SQL> alter system set log_archive_dest_state_2=enable;

Step 10: Start the Managed Recovery Process (MRP) on the standby database. 1 2
SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 12:33:03 2012 [oracle@uat ~]$ sqlplus sys/oracle@srps as sysdba

3 4
Copyright (c) 1982, 2010, Oracle. All rights reserved.

5 6
Connected to:

7
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

8 9 10

With the Partitioning, Automatic Storage Management, OLAP, DataMining and Real A

SQL> alter database recover managed standby database disconnect;

11 12 13 14 15 16 17 18
ARCH CONNECTED CONNECTED CONNECTED IDLE IDLE IDLE IDLE WAIT_FOR_LOG 0 0 0 0 13 0 0 13 PROCESS STATUS SEQUENCE# SQL> select process,status,sequence# from v$managed_standby; Database altered.

------- ------------- ----------ARCH CONNECTED 0

19
ARCH

20
ARCH

21 22 23 24 25 26 27 28 29

RFS RFS RFS RFS MRP0

9 rows selected.

Step 11: On the primary database, run the patching scripts like catbundle.sql in this case. The script run generates archives and these archives would be shipped and applied to

the standby database. So, there is no requriement to run the patching scripts on the standby database. 1
[oracle@dev ~]$ sqlplus sys/oracle@srprim as sysdba

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
14 MAX(SEQUENCE#) -------------SQL> select max(sequence#) from v$archived_log; SQL> @?/rdbms/admin/catbundle.sql psu apply Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production Copyright (c) 1982, 2010, Oracle. All rights reserved. SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 12:31:52 2012

With the Partitioning, Automatic Storage Management, OLAP, DataMining and Real A

17 Step 12: Check if the PSU applied shows up in the primary database by querying the registry$history or dba_registry_history view.

1 2 3 4 5

SQL> select * from registry$history order by action_time desc;

ACTION_TIME ACTION NAMESPACE VERSION ID COMMENTS BUNDLE_SER

------------------------------ ---------- ---------- --------------- ---------- --18-SEP-12 12.32.44.728562 PM APPLY SERVER 11.2.0.2 4 PSU 11.2.0.2.4 PSU

Step 13: Make sure that the latest archive applied on the standby database is the latest archive generated on the primary database. You can see below that the latest archive sequence applied on the standby database is sequence 14 and the latest sequence generated on the primary database too is 14. Now, check if the PSU applied shows up in the standby database by querying the registry$history or dba_registry_history view.
[oracle@uat ~]$ sqlplus sys/oracle@srps as sysdba

1 2

SQL*Plus: Release 11.2.0.2.0 Production on Tue Sep 18 12:44:25 2012

3 4
Copyright (c) 1982, 2010, Oracle. All rights reserved.

5 6
Connected to:

7
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production

8 9 10 11

With the Partitioning, Automatic Storage Management, OLAP, DataMining and Real A

SQL> select max(sequence#) from v$archived_log where applied='YES';

12 13 14 15 16 17

MAX(SEQUENCE#) -------------14

SQL> select * from registry$history order by action_time desc;

ACTION_TIME ACTION NAMESPACE VERSION ID COMMENTS BUNDLE_SER

18 19
18-SEP-12 12.32.44.728562 PM APPLY SERVER 11.2.0.2 4 PSU 11.2.0.2.4 PSU

----------------------------------- ---------- ---------- ---------- ---------- --

20 21 We can see that the PSU is applied successfully on both Primary and standby databases.

Applying CPU patch on DataGuard Physical Standby Configuration.

May 26th, 2012 | Posted by Nassyam Basha in Data Guard | Patching, Installations & Upgrade

Applying CPU Patch in a DataGuard Physical Standby Database Configuration

I seen lot of questions in OTN on patching of DataGuard, Recently I have applied CPU Patch on production database, I like to give step-by-step procedure how to apply CPUJAN2012(13343244). Standby is in MAXIMUM PERFORMANCE Mode with Data Broker Enabled.When managing with broker, there are some steps which we need to manage with Broker instead of SQL. Which is highly recommended, If not there may be issues in Broker configuration. 1. In Primary disable log shipping to the standby & Stop MRP on standby 2. Shutdown Standby Database & Listener 3. Install CPU patch on Standby 4. Shutdown Primary database & Listener. 5. Apply Patch on Primary & Run Post installation Scripts(catbundle.sql) 6. Re-enable Log shipping on Primary 7. Monitor the redo apply from Primary to Standby Environment:-

SQL> select * from v$version; BANNER ------------------------------------------------------------------------------Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production PL/SQL Release 11.2.0.2.0 - Production CORE 11.2.0.2.0 Production

TNS for Linux: Version 11.2.0.2.0 - Production NLSRTL Version 11.2.0.2.0 - Production SQL>
oracle@primary> file /bin/ls

/bin/ls: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), for GNU/Linux 2.6.9, stripped oracle@primary>
Opatch Version To apply CPUJan2012, OPatch utility version 11.2.0.1.0 or later to apply this patch. Oracle recommends that you use the latest released OPatch 11.2, which is available for download from My Oracle Support patch 6880880 by selecting the 11.2.0.0.0 release

oracle-ckpt.com> export PATH=/u00/app/oracle/product/11.2.0/OPatch:$PATH oracle-ckpt.com> opatch lsinventory Invoking OPatch 11.2.0.1.1 Oracle Interim Patch Installer version 11.2.0.1.1 Copyright (c) 2009, Oracle Corporation. Oracle Home All rights reserved.

: /u00/app/oracle/product/11.2.0

Central Inventory : /u00/app/oraInventory from : /etc/oraInst.loc

OPatch version : 11.2.0.1.1 OUI version : 11.2.0.2.0 OUI location : /u00/app/oracle/product/11.2.0/oui Log file location : /u00/app/oracle/product/11.2.0/cfgtoollogs/opatch/opatch2012-03-03_06-3239AM.log Patch history file: /u00/app/oracle/product/11.2.0/cfgtoollogs/opatch/opatch_history.txt Lsinventory Output file location : /u00/app/oracle/product/11.2.0/cfgtoollogs/opatch/lsinv/lsinventory2012-0303_06-32-39AM.txt ------------------------------------------------------------------------------1) In Primary disable log shipping to the standby & Stop MRP on standby

oracle@primary> dgmgrl / DGMGRL for Linux: Version 11.2.0.2.0 - 64bit Production Copyright (c) 2000, 2009, Oracle. All rights reserved. Welcome to DGMGRL, type "help" for information. Connected. DGMGRL> edit database PSPRIM set state='LOG-TRANSPORT-OFF'; Succeeded. DGMGRL>
Check the remote destination Status:-

oracle@primary> sqlplus / as sysdba SQL*Plus: Release 11.2.0.2.0 Production on Sat Apr 28 22:43:55 2012 Copyright (c) 1982, 2010, Oracle. Connected to: All rights reserved.

Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL> show parameter dest_state_2 NAME TYPE VALUE

------------------------------------ ----------- ----------------------------log_archive_dest_state_2

Check the Archive Log status:-

string

RESET

ID STATUS DB_MODE TYPE RECOVERY_MODE PROTECTION_MODE ACTIVE ARCHIVED_SEQ#

SRLs

--- --------- --------------- ---- ----------------------- ------------------- ----------------------1 VALID PERFORMANCE 2 DEFERRED PERFORMANCE OPEN 0 ARCH IDLE 0 MAXIMUM

MOUNTED-STANDBY LGWR MANAGED REAL TIME APPLY MAXIMUM 10 1 106896

2) Shutdown Standby Database & Listener

oracle@standby> ps -ef|grep pmon oracle 8016 30235 0 02:17 pts/0 00:00:00 grep pmon

oracle@standby>> ps -ef|grep tns oracle 8019 30235 0 02:17 pts/0 00:00:00 grep tns

oracle@standby>
Take Backup of ORACLE_HOME & Inventory on Standby

oracle@standby> tar -zcpvf +%Y%m%d).tar.gz /u00

11.2.0_Home_Inventory_Backup_$(date

/app/oracle/product/11.2.0 /u00/app/oraInventory /u00/app/oracle/product/11.2.0/ /u00/app/oracle/product/11.2.0/database/ /u00/app/oracle/product/11.2.0/database/cv/ /u00/app/oracle/product/11.2.0/database/cv/cvutl/ /u00/app/oracle/product/11.2.0/database/cv/cvutl/check_nodeadd.pl /u00/app/oracle/product/11.2.0/database/cv/admin/

- - - - - /u00/app/oraInventory/ /u00/app/oraInventory/oraInstaller.properties /u00/app/oraInventory/install.platform /u00/app/oraInventory/orainstRoot.sh /u00/app/oraInventory/oui/ /u00/app/oraInventory/oui/srcs.lst /u00/app/oraInventory/logs/ /u00/app/oraInventory/logs/OPatch2011-06-22_09-50-15-PM.log /u00/app/oraInventory/logs/oraInstall2011-06-21_12-35-26AM.out /u00/app/oraInventory/logs/installActions2011-06-21_12-35-26AM.log /u00/app/oraInventory/logs/oraInstall2011-06-21_12-35-26AM.err /u00/app/oraInventory/oraInst.loc /u00/app/oraInventory/ContentsXML/ /u00/app/oraInventory/ContentsXML/comps.xml /u00/app/oraInventory/ContentsXML/libs.xml /u00/app/oraInventory/ContentsXML/inventory.xml oracle@standby>
3. Install CPU patch on Standby

oracle@standby> opatch napply -skip_subset -skip_duplicate Invoking OPatch 11.2.0.1.1 Oracle Interim Patch Installer version 11.2.0.1.1 Copyright (c) 2009, Oracle Corporation. UTIL session Oracle Home : /u00/app/oracle/product/11.2.0/database All rights reserved.

Central Inventory : /u00/app/oraInventory from OPatch version OUI version : /etc/oraInst.loc : 11.2.0.1.1 : 11.2.0.2.0

OUI location

: /u00/app/oracle/product/11.2.0/database/oui

Log file location : /u00/app/oracle/product/11.2.0/database/cfgtoollogs/opatch/opatch2012-0428_22-57-58PM.log Patch history file: /u00/app/oracle/product/11.2.0/database/cfgtoollogs/opatch/opatch_history.t xt Invoking utility "napply" Checking conflict among patches... Checking if Oracle Home has components required by patches... Checking skip_duplicate Checking skip_subset Checking conflicts against Oracle Home... OPatch continues with these patches: 11830776 11830777 12586486 12586487 12586488 12586489 1258 6491 12586492 12586493 12586494 12586495 12586496 12846268 12846269 13343244 13386082 13468884 Do you want to proceed? [y|n] y User Responded with: Y Running prerequisite checks... OPatch detected non-cluster Oracle Home from the inventory and will patch the local system only. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u00/app/oracle/product/11.2.0/database') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files affected by the patch 'NApply' for restore. This might take a while... Applying patch 11830776...

ApplySession applying interim patch '11830776' to OH '/u00/app/oracle/product/11.2.0/database' Backing up files affected by the patch '11830776' for rollback. This might take a while... Verifying the update... Inventory check OK: Patch ID 13343244 is registered in Oracle Home inventory with proper meta-data. Files check OK: Files from Patch ID 13343244 are present in Oracle Home. Applying patch 13386082... ApplySession applying interim patch '13386082' to OH '/u00/app/oracle/product/11.2.0/database' ApplySession: Optional component(s) [ oracle.network.cman, 11.2.0.2.0 ] not present in the Oracle Home or a higher version is found. Backing up files affected by the patch '13386082' for rollback. This might take a while... Patching component oracle.network.rsf, 11.2.0.2.0... Updating archive file "/u00/app/oracle/product/11.2.0/database/lib/libnro11.a" "lib/libnro11.a/ncrfidr.o" Patching component oracle.rdbms, 11.2.0.2.0... ApplySession adding interim patch '13386082' to inventory Verifying the update... Inventory check OK: Patch ID 13386082 is registered in Oracle Home inventory with proper meta-data. Files check OK: Files from Patch ID 13386082 are present in Oracle Home. Applying patch 13468884... ApplySession applying interim patch '13468884' to OH '/u00/app/oracle/product/11.2.0/database' Backing up files affected by the patch '13468884' for rollback. This might take a while... Verifying the update... Inventory check OK: Patch ID 13468884 is registered in Oracle Home inventory with proper meta-data. Files check OK: Files from Patch ID 13468884 are present in Oracle Home.

with

Running make for target client_sharedlib Running make for target client_sharedlib Running make for target ioracle The local system has been patched and can be restarted. UtilSession: N-Apply done. OPatch succeeded. oracle@standby>
Start Standby database in mount status

SQL> startup mount; Total System Global Area 3.2068E+10 bytes Fixed Size Variable Size Database Buffers Redo Buffers Database mounted. SQL>
DataGuard Broker enabled, So MRP will be started automatically, No need to start MRP manually 4. Shutdown Primary database & Listener.

2242320 bytes 2.2616E+10 bytes 9395240960 bytes 55267328 bytes

oracle@primary> ps -ef|grep pmon oracle 8016 30235 0 02:17 pts/0 00:00:00 grep pmon

oracle@standby>> ps -ef|grep tns oracle 8019 30235 0 02:17 pts/0 00:00:00 grep tns

oracle@primary>
Take Backup of ORACLE_HOME & Inventory on Primary

oracle@Primary> tar -zcpvf +%Y%m%d).tar.gz /u00

11.2.0_Home_Inventory_Backup_$(date

/app/oracle/product/11.2.0 /u00/app/oraInventory /u00/app/oracle/product/11.2.0/ /u00/app/oracle/product/11.2.0/database/ /u00/app/oracle/product/11.2.0/database/cv/ /u00/app/oracle/product/11.2.0/database/cv/cvutl/

/u00/app/oracle/product/11.2.0/database/cv/cvutl/check_nodeadd.pl /u00/app/oracle/product/11.2.0/database/cv/admin/ - - - - - /u00/app/oraInventory/ /u00/app/oraInventory/oraInstaller.properties /u00/app/oraInventory/install.platform /u00/app/oraInventory/orainstRoot.sh /u00/app/oraInventory/oui/ /u00/app/oraInventory/oui/srcs.lst /u00/app/oraInventory/logs/ /u00/app/oraInventory/logs/OPatch2011-06-22_09-50-15-PM.log /u00/app/oraInventory/logs/oraInstall2011-06-21_12-35-26AM.out /u00/app/oraInventory/logs/installActions2011-06-21_12-35-26AM.log /u00/app/oraInventory/logs/oraInstall2011-06-21_12-35-26AM.err /u00/app/oraInventory/oraInst.loc /u00/app/oraInventory/ContentsXML/ /u00/app/oraInventory/ContentsXML/comps.xml /u00/app/oraInventory/ContentsXML/libs.xml /u00/app/oraInventory/ContentsXML/inventory.xml oracle@primary>
5. Apply Patch on Primary & Run Post installation Scripts(catbundle.sql)

oracle@primary> export PATH=$ORACLE_HOME/OPatch:$PATH: oracle@primary> cd $ORACLE_HOME/../Patches/ 13343244 oracle@primary> ls -ltr total 4772 drwxrwxr-x 19 oracle oinstall -rw-r--r-x86-64.zip 1 oracle dba 4096 Jan 15 12:07 13343244 4823614 Jan 30 10:36 p13343244_112020_Linux-

oracle@primary> pwd

/u00/app/oracle/product/11.2.0/Patches oracle@primary> cd 13343244 oracle@primary> opatch napply -skip_subset -skip_duplicate

Invoking OPatch 11.2.0.1.1 Oracle Interim Patch Installer version 11.2.0.1.1 Copyright (c) 2009, Oracle Corporation. UTIL session Oracle Home : /u00/app/oracle/product/11.2.0/database All rights reserved.

Central Inventory : /u00/app/oracle/oraInventory from OPatch version OUI version OUI location : /etc/oraInst.loc : 11.2.0.1.1 : 11.2.0.2.0 : /u00/app/oracle/product/11.2.0/database/oui

Log file location : /u00/app/oracle/product/11.2.0/database/cfgtoollogs/opatch/opatch2012-0428_23-46-42PM.log Patch history file: /u00/app/oracle/product/11.2.0/database/cfgtoollogs/opatch/opatch_history.t xt Invoking utility "napply" Checking conflict among patches... Checking if Oracle Home has components required by patches... Checking skip_duplicate Checking skip_subset Checking conflicts against Oracle Home... OPatch continues with these patches: 11830776 11830777 12586486 12586487 12586488 12586489 1258 6491 12586492 12586493 12586494 12586495 12586496 12846268 12846269 13343244 13386082 13468884 Do you want to proceed? [y|n] y

User Responded with: Y Running prerequisite checks... OPatch detected non-cluster Oracle Home from the inventory and will patch the local system only. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u00/app/oracle/product/11.2.0/database') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files affected by the patch 'NApply' for restore. This might take a while... Applying patch 11830776... ApplySession applying interim patch '11830776' to OH '/u00/app/oracle/product/11.2.0/database' Backing up files affected by the patch '11830776' for rollback. This might take a while... Patching component oracle.sysman.console.db, 11.2.0.2.0... Updating jar file "/u00/app/oracle/product/11.2.0/database/sysman/jlib/emCORE.jar" with "/sysman/jlib/emCORE.jar/oracle/sysman/eml/admin/rep/AdminResourceBundle.cl ass" Updating jar file "/u00/app/oracle/product/11.2.0/database/sysman/jlib/emCORE.jar" with "/sysman/jlib/emCORE.jar/oracle/sysman/eml/admin/rep/AdminResourceBundleID. class" Updating jar file "/u00/app/oracle/product/11.2.0/database/sysman/jlib/emCORE.jar" with "/sysman/jlib/emCORE.jar/oracle/sysman/eml/admin/rep/UserData.class" Copying file to "/u00/app/oracle/product/11.2.0/database/oc4j/j2ee/oc4j_applications/applic ations/em/em/admin/rep/editUserSummary.uix" Verifying the update... Inventory check OK: Patch ID 13468884 is registered in Oracle Home inventory with proper meta-data.

Files check OK: Files from Patch ID 13468884 are present in Oracle Home. Running make for target client_sharedlib Running make for target client_sharedlib Running make for target ioracle The local system has been patched and can be restarted. UtilSession: N-Apply done. OPatch succeeded. oracle@primary>
Start Database in OPEN mode

oracle@primary> SQL> startup

sqlplus "/ as sysdba"

ORACLE instance started. Total System Global Area 3.2068E+10 bytes Fixed Size Variable Size Database Buffers Redo Buffers Database mounted. Database opened. SQL>
Run catbundle.sql

2242320 bytes 2.2616E+10 bytes 9395240960 bytes 55267328 bytes

SQL> @?/rdbms/admin/catbunldle.sql cpu apply PL/SQL procedure successfully completed. PL/SQL procedure successfully completed. Generating apply and rollback scripts... Check the following file for errors: /u00/app/oracle/product/cfgtoollogs/catbundle/catbundle_CPU_psprim_GENERATE _2012Apr28_23_53_16.log Apply script: /u00/app/oracle/product/11.2.0/database/rdbms/admin/catbundle_CPU_psprim_AP PLY.sql

Rollback script: /u00/app/oracle/product/11.2.0/database/rdbms/admin/catbundle_CPU_psprim_RO LLBACK.sql PL/SQL procedure successfully completed. Executing script file... SQL> COLUMN spool_file NEW_VALUE spool_file NOPRINT SQL> SELECT '/u00/app/oracle/product/cfgtoollogs/catbundle/' || 'catbundle_CPU_' || name || '_APPLY_' || TO_CHAR(SYSDATE, 'YYYYMonDD_hh24_mi_ss', 'NLS_DATE_LANGUAGE=''AMERICAN''') || '.log' AS spool_file FROM v$database; SQL> SPOOL &spool_file SQL> exec dbms_registry.set_session_namespace('SERVER') PL/SQL procedure successfully completed. SQL> PROMPT Processing EM Repository... Processing EM Repository... SQL> ALTER SESSION SET current_schema = SYSMAN; Session altered. SQL> @?/sysman/admin/emdrep/sql/db/latest/policy/config_util_pkgdef.sql SQL> Rem drv: <create type="pkgdefs"/> SQL> PROMPT Updating registry... Updating registry... SQL> INSERT INTO registry$history 2 3 4 5 6 7 8 9 (action_time, action, namespace, version, id, bundle_series, comments) VALUES (SYSTIMESTAMP, 'APPLY', SYS_CONTEXT('REGISTRY$CTX','NAMESPACE'), '11.2.0.2', 4,

10 11

'CPU', 'CPUJan2012');

1 row created. SQL> COMMIT; Commit complete. SQL> SPOOL off SQL> SET echo off Check the following log file for errors: /u00/app/oracle/product/cfgtoollogs/catbundle/catbundle_CPU_psprim_APPLY_20 12Apr28_23_53_21.log SQL>
Compile Invalid objects by executing utlrp.sql.

SQL> @?/rdbms/admin/utlrp.sql TIMESTAMP -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------COMP_TIMESTAMP UTLRP_BGN 2012-04-28 23:54:20

DOC> The following query reports the number of errors caught during DOC> recompilation. If this number is non-zero, please query the error DOC> messages in the table UTL_RECOMP_ERRORS to see if any of these errors DOC> are due to misconfiguration or resource constraints that must be DOC> fixed before objects can compile successfully. DOC># ERRORS DURING RECOMPILATION --------------------------0 Function created. PL/SQL procedure successfully completed. Function dropped.

PL/SQL procedure successfully completed. SQL>

6. Check Patch Registry & Re-enable Log shipping on Primary

SQL> select ACTION_TIME,VERSION,ID,COMMENTS from registry$history; ACTION_TIME VERSION ID COMMENTS

------------------------------ ---------- ---------- -------------------05-SEP-10 06.22.14.370943 AM 20-MAY-11 06.18.50.526828 AM 28-APR-12 11.53.31.208266 PM

Re-Enable Log Shipping on Primary:-

11.2.0.2 11.2.0.2 11.2.0.2

0 Patchset 11.2.0.2.0 0 Patchset 11.2.0.2.0 4 CPUJan2012

DGMGRL> edit database psprim set state='ONLINE'; Succeeded. DGMGRL> exit

7. Monitor the redo apply from Primary to Standby

oracle@standby> tail -f /u00/app/oracle/diag/rdbms/psprimsby1/psprim/trace/alert_psprim.log Archived Log entry 48459 added for thread 1 sequence 106898 ID 0x6bf95f8a dest 1: Sun Apr 29 00:07:58 2012 Media Recovery Log /u04/arch/psprim/1_106899_751616314.dbf Sun Apr 29 00:07:58 2012 Archived Log entry 48460 added for thread 1 sequence 106901 ID 0x6bf95f8a dest 1: Media Recovery Log /u04/arch/psprim/1_106900_751616314.dbf Media Recovery Log /u04/arch/psprim/1_106901_751616314.dbf Media Recovery Waiting for thread 1 sequence 106902 (in transit)
Primary:-

SQL> select max(sequence#) from v$archived_log; MAX(SEQUENCE#) -------------106916 SQL>

Standby:-

SQL>

select max(sequence#) from v$archived_log where applied='YES';

MAX(SEQUENCE#) -------------106915 SQL>

Check Broker Status:-

DGMGRL> show database verbose PSPRIM Database - PSPRIM Role: Intended State: Instance(s): PSPRIM Properties: DGConnectIdentifier ObserverConnectIdentifier LogXptMode DelayMins Binding MaxFailure MaxConnections ReopenSecs NetTimeout RedoCompression LogShipping PreferredApplyInstance ApplyInstanceTimeout ApplyParallel StandbyFileManagement ArchiveLagTarget LogArchiveMaxProcesses = 'PSPRIM' = '' = 'ASYNC' = '0' = 'optional' = '0' = '1' = '300' = '30' = 'DISABLE' = 'ON' = '' = '0' = 'AUTO' = 'AUTO' = '0' = '30' PRIMARY TRANSPORT-ON

LogArchiveMinSucceedDest DbFileNameConvert LogFileNameConvert /u01/oradata/PSPRIM' FastStartFailoverTarget InconsistentProperties InconsistentLogXptProps SendQEntries LogXptStatus RecvQEntries SidName

= '1' = '' = '/u01/oradata/PSPRIM,

= '' = '(monitor)' = '(monitor)' = '(monitor)' = '(monitor)' = '(monitor)' = 'PSPRIM'

StaticConnectIdentifier = '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oracleckpt.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=PSPRIM_DGMGRL)(INSTANCE_NA ME=PSPRIM)(SERVER=DEDICATED)))' StandbyArchiveLocation AlternateLocation LogArchiveTrace LogArchiveFormat TopWaitEvents Database Status: SUCCESS DGMGRL>
Make sure Broker configuration is everything fine. Hope this helps.

= '/u04/arch/PSPRIM' = '' = '0' = '%t_%s_%r.dbf' = '(monitor)'