Oracle 10g 2 RAC Installation/Configuration Troubleshoots on

RHEL5(For server USTSMCDCVTC007/ USTSMCDCVTC008)

1.RPM Packages issues:

Few of the RPM Packages(as per Oracle recommends OS package list) were
not installed before.

Asked SA to check/install all below RPM packages.

binutils-2.17.50 (x86_64)
compat-db-4.2.52 (x86_64)
compat-libstdc++-296(i386)
compat-libstdc++-33-3.2.3 (x86_64)
compat-libstdc++-33-3.2.3(i386)
control-center-2.16.0 (x86_64)
gcc-4.1.1 (x86_64)
gcc-c++-4.1.1 (x86_64)
glibc-2.5-12 (x86_64)
glibc-2.5-12 (i386)
glibc-common-2.5-12 (x86_64)
glibc-devel-2.5-12 (x86_64)
glibc-devel-2.5-12(i386)
glibc-headers-2.5-12 (x86_64)
ksh-20060214-1.4 (x86_64)
libaio-0.3.96 (x86_64)
libgcc-4.1.1(i386)
libgcc-4.1.1(x86_64)
libgnome-2.16.0 (x86_64)
libgnomeui-2.16.0 (x86_64)
libgomp-4.1.1 (x86_64)
libstdc++-4.1.1 (x86_64)
libstdc++-devel-4.1.1 (x86_64)
libXp-1.0.0-8 (i386)
libXtst-1.0.1-3.1(i386)
libXtst-1.0.1-3.1 (x86_64)
make-3.81 (x86_64)
sysstat-7.0.0 (x86_64)
xorg-x11 –deprecated-libs-6.8.2-l.EL.13.20.i386.rpm

2.IP Address setup issues:

a. Virtual ipaddress, private ipaddress were not added on /etc/hosts

file.

Asked SA to add private ipaddress and VIP address entries on /etc/hosts

file.
b.VIP address was up.

Should not be configured/up(test by pinging).Asked SA to keep VIP

address down as CRS will manage the VIP during installation.

c. Interface related Error Message during ASM installation:

Interface type 1 eth7 10.10.1.0 configured from OCR for use as a

cluster interconnect
WARNING 10.10.1.0 could not be translated to a network address
error 1 Interface type 1 bond0 10.206.4.0 configured from OCR
for use as a public interface
WARNING 10.206.4.0 could not be translated to a network address
WARNING: No cluster interconnect has been specified. Depending
on the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.

PRKS-1009 : Failed to start ASM instance "+ASM2" on node

"ustsmcdcvtc008", [PRKS-1009 : Failed to start ASM instance
"+ASM2" on node "ustsmcdcvtc008", [ustsmcdcvtc008
ra.ustsmcdcvtc008.vip:bond0: error fetching interface
information: Device not found ustsmcdcvtc008
ra.ustsmcdcvtc008.vip:checkIf: interface bond0 is down
ustsmcdcvtc008 ra.ustsmcdcvtc008.vip:Invalid parameters, or
failed to bring up VIP (host=ustsmcdcvtc008.convatec.com)
ustsmcdcvtc008 ra.ustsmcdcvtc008.vip:bond0: error fetching
interface information: Device not foundustsmcdcvtc008
ra.ustsmcdcvtc008.vip:checkIf: interface b

It is expecting the interface names to be the same on both boxes

and they are not.Run ifconfig –a on server to confirm you have
the proper interface names with the IPs that have been assigned
the public hostname and private interconnect.

3.KERNEL parameter setting issues for RAC:

Please make sure these parameters are set on /etc/sysctl.conf file:

Modify these two parameters:

kernel.shmall = 2097152
kernel.shmmax = 23072146 ( Should be 60%-75% of Physical RAM)

Add these below parameters as they need to support RAC DB:

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 32768 65000
net.core.rmem_default = 1048576
net.core.rmem_max = 1048576
net.core.wmem_default = 262144
net.core.wmem_max = 262144
4.Raw device issues:

"proprio-gid:1 Invalid format" error on ocrconfig_xxxxx.log(Issues faced

during crs installation).

OCR Raw device was not accessible. Ensure the raw devices defined for OCR and
Voting disk have the following permissions:
the crsocr raw device needs to be owned by root:oinstall with
permissions of 640 and crsvote should be owned by oracle:oinstall and
have permissions of 644.

5. Swap Memory size setup issues:

Swap Memory was before setup 10% of RAM .

Changed the swap memory size to 75 % of RAM as per Oracle recommends.

$ cat /proc/meminfo|grep MemTotal

MemTotal: 32960208 kB
$ cat /proc/meminfo|grep SwapTotal
SwapTotal: 25165808 Kb (75 % of MemTotal)

6.Root access for DBA( Needed during installation)

DBAs have access to run all below scripts as ROOT during install and
CRS management.

Please get the root access for below scripts from security team.

(root) /usr/bin/su - oracle

(root) /etc/init.d/init.crs
(root) /etc/init.d/init.crsd
(root) /etc/init.d/init.cssd
(root) /usr/bin/ls -l /var/core
(root) /usr/bin/ls -ltr /var/core
(root) /usr/bin/view /var/cron/syslog.log
(root) /usr/bin/view /var/cron/log
(root) /u001/app/oracle/product/oraInventory/orainstRoot.sh
(root) /u001/app/oracle/product/rdbms/10.2.0/root.sh
(root) /u001/app/oracle/product/crs/10.2.0/root.sh
(root) /u001/app/oracle/product/asm/10.2.0/root.sh
(root) /u001/app/oracle/product/crs/10.2.0/bin/crsctl
(root) /u001/app/oracle/product/crs/10.2.0/bin/vipca
(root) /u001/app/oracle/product/crs/10.2.0/install/root102.sh
 7. Set up oracle .profile:

ORACLE_OWNER=oracle
export ORACLE_OWNER
ORACLE_SID=xxxx
export ORACLE_SID
export ORACLE_BASE=/u001/app/oracle/product
export ORACLE_BASE
export ORACLE_HOME=$ORACLE_BASE/rdbms/10.2.0
export ORACLE_HOME
export ORA_RDBMS_HOME=$ORACLE_BASE/rdbms/10.2.0
export ORA_RDBMS_HOME
export ORA_CRS_HOME=$ORACLE_BASE/crs/10.2.0
export ORA_CRS_HOME
export ORA_ASM_HOME=$ORACLE_BASE/asm/10.2.0
export ORA_ASM_HOME
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH
PATH=.:/usr/bin:/usr/sbin:/etc:/usr/ccs/bin:/usr/openwin/bin:/usr
/local/bin:$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$ORA_ASM_HOME/bin:
$PATH
export PATH
if [ -t 0 ]; then
stty intr ^C
fi

8. CLUVFY checkup issues:

a. Prior to running cluvfy,

Ensure that ssh and scp reside in /usr/local/bin, if not, get SAs to create
links in /usr/local/bin to their actual location.

b. runcluvfy.sh comp sys -n node_list -p crs -osdba osdba_group –orainv

orainv_group –verbose

“User equivalence check failed for user Oracle”

Checked ssh configuration setting on both the nodes and resolve the issue.

NOTE: Ensure no banners are displayed when doing ssh using oracle ID or
install will fail. Banner can be put back after install is done.

c.If receive error related to not finding suitable VIPS

refer to Note: 338924.1

d.Private Interconnectivity check up failed.

Asked SA to provide and activate the private ipaddress.

9. OUI Failure Issues:

Oracle Universal Installer (OUI) initially fails with the following

error:

Checking installer requirements...

Checking operating system version: must be redhat-3, SuSE-9, redhat-4,
UnitedLinux-1.0, asianux-1 or asianux-2 Failed <<

To resolve the initial problem (starting OUI), use one of the following
workaround:

Ignore all the prerequisite checking by using

% ./runInstaller –ignoreSysPrereqs

10. CRS installation issues:

a. Prior to running root.sh during CRS installation,

put new version of clsfmt.bin into $ORA_CRS_HOME/bin via Bug 4679769.

b. During CRS Installation, make the below changes (Note 414163.1),

then run vipca manually

edit vipca (in the CRS bin directory on all nodes) to undo
the setting of LD_ASSUME_KERNEL. After the IF statement
around line 120 add an unset command to ensure
LD_ASSUME_KERNEL is not set as follows:

if [ "$arch" = "i686" -o "$arch" = "ia64" -o "$arch" = "x86

_64" ]
then
LD_ASSUME_KERNEL=2.4.19
export LD_ASSUME_KERNEL
fi

unset LD_ASSUME_KERNEL <<== Line to be added

Similarly for srvctl (in both the CRS and, when installed,
RDBMS and ASM bin directories on all nodes), unset
LD_ASSUME_KERNEL by adding one line, around line 168 should
look like this:

LD_ASSUME_KERNEL=2.4.19
export LD_ASSUME_KERNEL

unset LD_ASSUME_KERNEL <<== Line to be added

11. CRS bundle for 10.2.0.4 #3 – 7715304 to CRS, ASM and RDBMS oracle
Homes
Reference Metalink Doc ID 405820.1 for known issues.