A nR S G C ar Hd D ,B D RelcO a E, , A LB B zi la t r iVelcO OA u ar GD

GnJ uei ev N a s ld s 'i k OlLG aO r B e c

t ub oA

e m o H

t ub oA

1.3 reM elcO ga ar aM nV lcO )3.0.2.1 ( ar 1 taru i fnc ts t g o e

Oracle Virtualization Installing Oracle VM Server 3.1.1, Oracle VM Manager 3.1.1 and Deploying Oracle RAC 11gR2 (11.2.0.3) Oracle VM templates Linux x86 64 bit for test configuration
In the second post, first is here, for Oracle VM 3.1.1 technology you will look at how to upgrade the existing Oracle VM server and Oracle VM manager to version 3.1.1 and deploy Oracle RAC templates to build two node Oracle 11gR2 (11.2.0.3) cluster. All software can be downloaded from edelivery.oracle.com (Oracle Software Delivery Cloud) and the templates (version 11.2.0.3) can be downloaded as a patch from MOS (patch 12694014. The approach described in the article is applicable on any supported platform although I am using Oracle VirtualBox for testing. Software versions used are: Oracle VirtualBox 4.1.16 Oracle VM Server 3.1.1 Oracle VM Manager 3.1.1 OEL 6.2 Oracle RAC templates (patch 12694014) The article will cover the following: VirtualBox configuration for VM Server machine and VMManager machine Installation of Oracle VM Server

Guenadi Jilevski is a Senio r Oracle Pro fessio nal (OCP 7, 8 , 8 i, 9 i, 10 g, 11g, OCE 10 gR2 RAC, OCE Oracle RAC 11gR2 and Grid Infrastructure Administrato r, Oracle Exadata 11g Certified Implementatio n Specialist,Oracle Go ldenGate 10 Certified Implementatio n Specialist and OCM 9 i) with mo re than 20 years o f industrial experience in high end Oracle techno lo gy starting with Oracle RDBMS versio n 6 up to 11g. He has been invo lved with multi platfo rm High Availability So lutio ns starting with OPS and Veritas and Sun cluster as well as co mplex multiple no de RAC 9 i, 10 g and 11g implementatio ns o n AIX, Linux, So laris and HP. He has experience in deplo ying disaster reco very so lutio ns with standby databases co mpliant to Oracle MAA and third party so lutio ns such as Shareplex and Go ldengate. As a fo rmer emplo yee o f Oracle Co rpo ratio n Advanced Techno lo gy So lutio ns Co nsulting Practice in Americas, Guenadi has been invo lved in multiple visible high end pro jects fo r RAC enabling EBS, Peo pleso ft, Siebel ERP applicatio ns, end to end perfo rmance tuning, architectured High Availability and Disaster Reco very so lutio ns to name a few. Presently he is wo rking as A Senio r Oracle Co nsultant in EMEA. Guenadi Jilevski is Oracle OCP 7, 8 , 8 i, 9 i, 10 g, 11g, OCE 10 gR2 RAC, OCE Oracle RAC 11gR2 and Grid Infrastructure Administrato r, Oracle Exadata 11g Certified Implementatio n Specialist,Oracle Go ldenGate 10 Certified Implementatio n Specialist and Oracle OCM, Teradata Certified DBA, Info rmix Certified DBA.
PDFmyURL.com

Installation of Oracle VM Manager Deployment of Oracle RAC 11gR2 Oracle VM templates Linux x86 64bit for test configuration In real life Oracle VM server runs on a dedicated set of servers and is installed as a bare metal product without the need of an Operating system. Oracle VM Manager is installed on a Linux box. Oracle VirtualBox can be successfully used for testing and selfstudy of the new features. The main components in the architecture are two virtual machines.

The views expressed o n this blo g are my o wn and do no t necessarily reflect the views o f Current/Past/Future Emplo yers. The views and o pinio ns expressed by visito rs o n this blo g are theirs so lely and may no t reflect mine.

s v i h rA e c
No ve m be r 20 12 (8 ) Oct o be r 20 12 (6 ) J uly 20 12 (8 ) J une 20 12 (4) Fe bruary 20 12 (2) J anuary 20 12 (2) No ve m be r 20 11 (6 ) Oct o be r 20 11 (3) Se pt e m be r 20 11 (1) August 20 11 (2) J uly 20 11 (9 ) May 20 11 (6 )

Virtual Machine
OracleVMServer303 OracleVMManager303

IP Address
192.168.2.21 192.168.2.31

OS
OEL XEN with Oracle VM Server 3.0.3 OEL 6.2

Purpose
Oracle VM Server 3.0.3

s v i h rA e c
Select Mo nth

statS g lB o Oracle VM Manager 3.0.3 418 ,139 hits d ta pT eRo

Po st s | Page s | Co m m e nt s

s i roeta e g C
o racle

Product documentation for Oracle VM server 3.1.1 and Oracle VM Manager 3.1.1 is available here.
1.

Upgrade Oracle VM server 3.0.3 to Oracle VM server 3.1.1

Make the VMserver311 ISO available to the virtual machine OracleVMserver303 and start the VM. Once the boot screen appears press Enter.

S S R
Ent rie s RSS Co m m e nt s RSS

All | Today | T his We e k | T his Mont h

There are no rated items fo r this perio d.

l l org lB o
Wo rdPre ss.co m 0 Wo rdPre ss.o rg 0

ran l a de C
June 20 12 M T W T F S S 4 5 6 7 1 8 2 9 3 10

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
PDFmyURL.com

 Fe b

J ul

h raS c e
Search

n i tp i rcs u l i m o bS a E
Enter yo ur email address to subscribe to this blo g and receive no tificatio ns o f new po sts by email. Jo in 358 o ther fo llo wers

Press Skip.
Sign me up!

Ar NO a

PDFmyURL.com

Select keyboard.

Accept the license agreement.

PDFmyURL.com

Select an UPGRADE option and the Oracle VM server 3.0.3 to upgrade.

PDFmyURL.com

Upgrade the boot loader.

Take a note of the log file location and press OK.

PDFmyURL.com

Wait for the upgrade to continue.

Reboot.

PDFmyURL.com

Press the highlighted selection.

PDFmyURL.com

Wait for the boot to complete.

Press Alt-F2 to login

PDFmyURL.com

Login

PDFmyURL.com

2.

Upgrade Oracle VM Manager 3.0.3 to Oracle VM Manager 3.1.1

Transfer to OracleVMManager303 virtual machine OracleVMManager311.iso image a loop mount it.
[root@vmmgr u01]# ls app OracleVMManager311.iso r.txt stage t.txt VMManager303.iso [root@vmmgr u01]# mount -o loop OracleVMManager311.iso /mnt/stage [root@vmmgr u01]#

From the stage directory invoke the upgrade script.

[root@vmmgr stage]# ./runUpgrader.sh Stating OVM Manager upgrade on Sun May 20 16:28:43 EEST 2012 Oracle VM Manager 3.1.1.305 upgrade utility Upgrade logfile : /tmp/upgrade-2012-05-20-28.log It is highly recommended to do a full database repository backup prior to upgrading Oracle VM Manager Press any key to continue Oracle VM Manager is running Verifying installation status Read Oracle VM Manager config file Found Oracle VM Manager install files Found Oracle VM Manager upgrader Found Oracle WebLogic Server Found Java Using the following information : Database Host : localhost

PDFmyURL.com

Database SID : XE Database LSNR : 1521 Oracle VM Schema : ovs Oracle VM Manager UUID : 0004fb0000010000f8a01f5a1718ffd9 Current Build ID : 3.0.3.126 Upgrade from version : 3.0.3 Upgrade to version : 3.1.1 Using /tmp/workdir.0zgO6hQ9um for backup and export location. Using /tmp/patchdir.Glh0EQQcD for patching. Enter password for user ovs : Undeploying previous version of Oracle VM Manager application Undeploying Oracle VM Manager help Undeploying Oracle VM Manager console Undeploying Oracle VM Manager core Waiting for Oracle VM Manager core to fully undeploy Waiting Finished undeploying previous version Exporting Oracle VM Manager repository Please wait as this can take a long time Oracle VM Manager repository export completed Creating backup file Oracle VM Manager repository backup in /tmp/ovm-manager-3-backup2012-05-20.zip Upgrading Oracle VM Manager Backing up old files to /tmp/ovm-manager-3-backup-2012-05-20163052

PDFmyURL.com

Removing old files Unpacking Oracle VM Manager 3.1.1.305 `transform_003001001000_010.xsl -> `/tmp/patchdir.Glh0EQQcD/transform_003001001000_010.xsl `transform_003001001000_020.xsl -> `/tmp/patchdir.Glh0EQQcD/transform_003001001000_020.xsl `deletedClasses.xml -> `/tmp/patchdir.Glh0EQQcD/deletedClasses.xml Filtering full repository export to the selective export subset at /tmp/workdir_sel.XRjFLORxBo cp: omitting directory `/tmp/workdir.0zgO6hQ9um/jrnl cp: omitting directory `/tmp/workdir.0zgO6hQ9um/objs adding: objs/63/639.cl.xml (deflated 81%) adding: objs/9.cl.xml (deflated 93%) adding: objs/16/161.cl.xml (deflated 67%) adding: objs/19/193.cl.xml (deflated 70%) Selective export is at /tmp/workdir_sel.XRjFLORxBo 4 objects selected (out of 508) to be upgraded Transform XSL files used: -rw-rr. 1 root root 56678 May 20 16:30 /tmp/patchdir.Glh0EQQcD/transform_003001001000_010.xsl -rw-rr. 1 root root 10079 May 20 16:30 /tmp/patchdir.Glh0EQQcD/transform_003001001000_020.xsl Changed classes encountered in selective export set: com.oracle.ovm.mgr.api.manager.BusinessManagerDbImpl com.oracle.ovm.mgr.api.manager.ModelManagerDbImpl com.oracle.ovm.mgr.api.manager.RasManagerDbImpl

PDFmyURL.com

com.oracle.ovm.mgr.api.physical.ServerDbImpl Upgrading Oracle VM Manager repository Please wait as this can take a long time Oracle VM Manager repository upgrade completed Validating Oracle VM Manager repository Oracle VM Manager repository validation completed Refresh system-jazn-data.xml file Redeploying Oracle VM Manager core container Redeploying Oracle VM Manager console Redeploying Oracle VM Manager help Install ADF Patch Completed upgrade to 3.1.1.305 Writing updated config in /u01/app/oracle/ovm-manager-3/.config Restart WebLogic Stopping Oracle VM Manager [ OK ] Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout [ OK ] OVM Manager upgrade finished on Sun May 20 16:35:47 EEST 2012 [root@vmmgr stage]#

Lets test Oracle VM Manager 3.1.1

PDFmyURL.com

After login you have.

PDFmyURL.com

Starting and Stopping the OVM from OEL.

[root@vmmgr ~]# service ovmm stop Stopping Oracle VM Manager [ OK ] [root@vmmgr ~]# [root@vmmgr ~]# service ovmm start Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout [ OK ] [root@vmmgr ~]#

3.

Deploy two node Oracle 11gR2 (11.2.0.3) RAC

1. Add a file system /nfs that is to NFS mounted.

Add 50GB disk to the OracleVMManager303 virtual machine. This disk is visible as /dev/sdb. Create a volume, file system and mount the file system on this disk.
[root@vmmgr ~]# fdisk -l /dev/sdb Disk /dev/sdb: 53.7 GB, 53687091200 bytes 255 heads, 63 sectors/track, 6527 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 000000000 Disk /dev/sdb doesnt contain a valid partition table

PDFmyURL.com

[root@vmmgr ~]# [root@vmmgr ~]# pvcreate /dev/sdb Writing physical volume data to disk /dev/sdb Physical volume /dev/sdb successfully created [root@vmmgr ~]# [root@vmmgr ~]# fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xb618dbd7. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content wont be recoverable. Warning: invalid flag 00000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. Its strongly recommended to switch off the mode (command c) and change display units to sectors (command u). Command (m for help): c DOS Compatibility flag is not set Command (m for help): u Changing display/entry units to sectors Command (m for help): n Command action e extended p primary partition (1-4)

PDFmyURL.com

p Partition number (1-4): 1 First sector (2048-104857599, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): +50G Value out of range. Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): +49G Command (m for help): t Selected partition 1 Hex code (type L to list codes): 8e Changed system type of partition 1 to 8e (Linux LVM) Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@vmmgr ~]# [root@vmmgr ~]# fdisk -l /dev/sdb Disk /dev/sdb: 53.7 GB, 53687091200 bytes 171 heads, 46 sectors/track, 13330 cylinders Units = cylinders of 7866 * 512 = 4027392 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xb618dbd7 Device Boot Start End Blocks Id System

PDFmyURL.com

/dev/sdb1 1 13065 51380224 8e Linux LVM [root@vmmgr ~]# [root@vmmgr ~]# pvcreate /dev/sdb1 Writing physical volume data to disk /dev/sdb1 Physical volume /dev/sdb1 successfully created [root@vmmgr ~]# vgcreate NFSVolGroup /dev/sdb1 [root@vmmgr ~]# vgcreate NFSVolGroup /dev/sdb1 Volume group NFSVolGroup successfully created [root@vmmgr ~]# [root@vmmgr ~]# vgdisplay NFSVolGroup Volume group VG Name NFSVolGroup System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 49.00 GiB

PDFmyURL.com

PE Size 4.00 MiB Total PE 12543 Alloc PE / Size 0 / 0 Free PE / Size 12543 / 49.00 GiB VG UUID RnSh8A-fvtJ-97hA-KyX3-Lizm-Dpat-sDcWM9 [root@vmmgr ~]# lvcreate size 48G NFSVolGroup [root@vmmgr ~]# lvcreate size 48G NFSVolGroup Logical volume lvol0 created [root@vmmgr ~]# [root@vmmgr ~]# lvdisplay Logical volume LV Name /dev/NFSVolGroup/lvol0 VG Name NFSVolGroup LV UUID 0M6eKi-2uiG-Chjl-rk6E-8BUc-WngI-SvvZpl LV Write Access read/write LV Status available # open 0 LV Size 48.00 GiB Current LE 12288 Segments 1 Allocation inherit Read ahead sectors auto currently set to 256 Block device 253:3

PDFmyURL.com

 Logical volume LV Name /dev/vg_vmmgr/lv_root VG Name vg_vmmgr LV UUID kpXFYY-eAPW-zCXW-wHnN-QuKw-kFxh-vjeaRX LV Write Access read/write LV Status available # open 1 LV Size 28.37 GiB Current LE 7263 Segments 1 Allocation inherit Read ahead sectors auto currently set to 256 Block device 253:0 Logical volume LV Name /dev/vg_vmmgr/lv_home VG Name vg_vmmgr LV UUID gaY4wY-aN48-SemS-iUps-L8CX-w27t-OQqPdF LV Write Access read/write LV Status available # open 1 LV Size 25.26 GiB Current LE 6467 Segments 1 Allocation inherit Read ahead sectors auto

PDFmyURL.com

 currently set to 256 Block device 253:2 Logical volume LV Name /dev/vg_vmmgr/lv_swap VG Name vg_vmmgr LV UUID f3L33X-DIly-6iO3-hGgu-DrWM-0g1O-mACi0e LV Write Access read/write LV Status available # open 1 LV Size 5.88 GiB Current LE 1504 Segments 1 Allocation inherit Read ahead sectors auto currently set to 256 Block device 253:1 [root@vmmgr ~]# /dev/NFSVolGroup/lvol0 [root@vmmgr ~]# mkfs.ext3 /dev/NFSVolGroup/lvol0 mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks

PDFmyURL.com

3145728 inodes, 12582912 blocks 629145 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=4294967296 384 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424 Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 21 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. [root@vmmgr ~]# [root@vmmgr ~]# # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/vg_vmmgr-lv_root / ext4 defaults 1 1 UUID=19fcf4b8-c245-4f71-bd73-5bf56c6dada5 /boot ext4 defaults 1 2 /dev/mapper/vg_vmmgr-lv_home /home ext4 defaults 1 2

PDFmyURL.com

/dev/mapper/NFSVolGroup-lvol0 /nfs ext4 defaults 1 2 /dev/mapper/vg_vmmgr-lv_swap swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 [root@vmmgr mapper]#

2. Create NFS

Modify /etc/exports as follows.

[root@vmmgr ~]# cat /etc/export cat: /etc/export: No such file or directory [root@vmmgr ~]# cat /etc/exports /nfs/stage *(rw,sync,no_wdelay,insecure_locks,no_root_squash) /nfs/stage1 *(rw,sync,no_wdelay,insecure_locks,no_root_squash) [root@vmmgr ~]#

Create /nfs/stage and /nfs/stage1 Enable and start nfs service

[root@vmmgr stage]# chkconfig nfs on [root@vmmgr stage]# service nfs restart Shutting down NFS mountd: [FAILED] Shutting down NFS daemon: [FAILED] Shutting down NFS quotas: [FAILED] Starting NFS services: [ OK ] Starting NFS quotas: [ OK ] Starting NFS daemon: [ OK ] Starting NFS mountd: [ OK ]

PDFmyURL.com

Starting and Stopping the OVM

[root@vmmgr ~]# service ovmm stop Stopping Oracle VM Manager [ OK ] [root@vmmgr ~]# [root@vmmgr ~]# service ovmm start Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout [ OK ] [root@vmmgr ~]#

3. Discover a VM server from the Oracle VM Manager 3.1.1

Click discover servers in the tool bar.

PDFmyURL.com

Wait until the server get discovered.

4. Create a network

Click the Networking tab, then the Networks subtab. Click Create New Network + to start.

PDFmyURL.com

Select Create a network with bonds/ports only. Press Next.

PDFmyURL.com

Specify Name.

Select servers.

PDFmyURL.com

Select ports. Make sure ONLY eth0 is selected. Picture is for illustration and should have only eth0.

Configure IP addresses. Select None and press Finish.

PDFmyURL.com

After the network is created it will be displayed.

5. Create a file server

On the storage tab select discover file servers and

PDFmyURL.com

press Next once filled in the data.

Select server.

PDFmyURL.com

Select the nfs from /etc/exports. Here should show one of it and press Finish. (Instead of nfs:/u01/VM/nfs should be nfs:/nfs/stage or nfs:/nfs/stage1) /nfs/stage is for the file system and /nfs/stage1 is for the repository.

You will get something like

PDFmyURL.com

1. Create a Server pool

From Servers and VMs click Create Server Pool. Press Next.

Select the server press Finish.

PDFmyURL.com

You will see the nfs created earlier. Press OK.

PDFmyURL.com

Examine the Pool successfully created.

2. Create Storage Repository.

Make sure that storage is refreshed. If not refresh from file servers.
PDFmyURL.com

Click Create Repository.

PDFmyURL.com

Enter Repository name. Press the magnifier to select a nfs location.

Press OK and press Next.

PDFmyURL.com

Select server.

PDFmyURL.com

Verify the result.

3. Import the templates.

Templates are not anymore stored directly onto the Oracle VM server 3.1.1. Instead Oracle has provided a way to access them using either ftp or http. I will use ftp. So I will configure the ftp service.
[root@vmmgr stage]# service vsftpd start

PDFmyURL.com

Starting vsftpd for vsftpd: [ OK ] [root@vmmgr stage]# chkconfig vsftpd on [root@vmmgr stage]#

I am using OS user gjilevski on the OEL6.2 linux VM running the Oracle VM Manager 3.1.1 with home directory /home/gjilevski. Copy the templates into /home/gjilevski. When specifying the templates make sure that you list second before the first template.
ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM2of2.tgz ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM1of2.tgz

In the URL gjilevski is the OS user identified by the password. File names follows without a path as they are in the default user directory. From VM templates import the templates.

PDFmyURL.com

In the screen that appear enter

ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM2of2.tgz ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM1of2.tgz

instead of what is displayed.

Wait for the import to completes.

It will looks like this.

PDFmyURL.com

If you edit it properties will be as follows.

PDFmyURL.com

PDFmyURL.com

4. Create shared virtual disk

Installing RAC requires 5 shared disks each 2GB. From the screen click create virtual disk.

PDFmyURL.com

Create ASM1

Create ASM2

PDFmyURL.com

Create ASM3

Repeat same steps for ASM4 and ASM5. Verify that all are created.
PDFmyURL.com

5. Create VNIC

From Networking tab click virtual NICs.

Click Auto Fill and Create.

PDFmyURL.com

Initially I did not created VNIC and got the error.

begin() Appended operation Adding a queued job operation to object 0004fb00001900008ea92f8eb1b6abc2 (Create Statistics Log). com.oracle.odof.exception.ObjectException: Caught during invoke method: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_8004E There are no more VNICs available. Create More MAC Addresses. Mon May 21 17:33:13 EEST 2012 at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:151) at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100) at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81) at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773) at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401) at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459) at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42) at java.lang.Thread.run(Thread.java:662) Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_8004E There are no more VNICs available. Create

PDFmyURL.com

More MAC Addresses. Mon May 21 17:33:13 EEST 2012 at com.oracle.ovm.mgr.api.manager.VnicManagerDbImpl.getAvailableVnic(VnicManagerDbImpl.java:353) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822) at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245) at com.oracle.ovm.mgr.api.manager.VnicManagerProxy.getAvailableVnic(Unknown Source) at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.cloneNetworks(VirtualMachineConfigDbImpl.java:1065) at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:916) at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:860) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136) 7 more rollback()

6 . Create Virtual Machines

PDFmyURL.com

Once both are done you will have.

PDFmyURL.com

7. Assign NICs to Virtual Machines and make sure that

xenbr0 and xenbr1 networks exists. In the network you need at the end something like this.

If not create/modify xenbr0 and xenbr1. For xenbr0 press Next.

PDFmyURL.com

Select server.

Select the ports.

PDFmyURL.com

Press Next.

Press Next.

PDFmyURL.com

For xenbr1 Press Next.

PDFmyURL.com

Select server name and press Next.

Select ports and press Next.

PDFmyURL.com

Press Finish.

If you do not do so will get similar to

Job Construction Phase begin() Appended operation Adding a queued job operation to object 0004fb000019000038ea52da9f8f21ee (Create Statistics Log). Appended operation Virtual Disk/Virtual Cdrom Clone to object 0004fb00001200001de8348bd5ec83ab.img (System.img (2)). Appended operation Repository Element files refresh to

PDFmyURL.com

object 0004fb00001200001de8348bd5ec83ab.img (System.img (2)). Appended operation Server Copy File to object 0004fb00001200001de8348bd5ec83ab.img (System.img (2)). Appended operation Virtual Disk/Virtual Cdrom Clone to object 0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2)). Appended operation Repository Element files refresh to object 0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2)). Appended operation Server Copy File to object 0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2)). Appended operation Virtual Machine Cfg File Construct to object 0004fb0000220000aa9f213a1f521585. com.oracle.odof.exception.ObjectException: Caught during invoke method: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_9039E Cannot place clone VM: OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz, in Server Pool: MyRACPool. That server pool has no servers that can run the VM Mon May 21 18:42:36 EEST 2012 at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:151) at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100) at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81) at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773) at

PDFmyURL.com

com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401) at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459) at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42) at java.lang.Thread.run(Thread.java:662) Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_9039E Cannot place clone VM: OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz, in Server Pool: MyRACPool. That server pool has no servers that can run the VM Mon May 21 18:42:36 EEST 2012 at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:939) at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:860) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136) 7 more

8 . Assign ASM shared disks to VM

Make sure that you have the following assignment for the VMs. For racnode1

PDFmyURL.com

PDFmyURL.com

PDFmyURL.com

For racnode2

PDFmyURL.com

PDFmyURL.com

The file are as follows.

[root@vmmgr u01]# cat vm.cfg vif = ['mac=00:21:f6:00:00:2f,bridge=xenbr0', 'mac=00:21:f6:00:00:30,bridge=xenbr1'] OVM_simple_name = racnode2

PDFmyURL.com

disk = ['file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008699ec66aee7821a.img,xvda,w', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200006e90573639dac725.img,xvdb,w', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000cdf8b3cd91c68ebd.img,xvdc,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000fb986a9247406641.img,xvdd,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008d3f9019fe32a304.img,xvde,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000046a344c5efa19a02.img,xvdf,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000c65d6d0a62ec75a0.img,xvdg,w!'] uuid = 0004fb00-0006-0000-a2a3-03e4d1281181 on_reboot = restart cpu_weight = 27500 memory = 2048 cpu_cap = 0 maxvcpus = 1 OVM_high_availability = False maxmem = 2048 OVM_description = racnode2 on_poweroff = destroy on_crash = restart bootloader = /usr/bin/pygrub name = 0004fb0000060000a2a303e4d1281181 guest_os_type = default vif_other_config = [] vfb = ['type=vnc,vncunused=1,vnclisten=127.0.0.1,keymap=en-us'] vcpus = 1 OVM_os_type = None OVM_cpu_compat_group =

PDFmyURL.com

OVM_domain_type = xen_pvm [root@vmmgr u01]# [root@vmmgr u01]# cat vm.cfg vif = ['mac=00:21:f6:00:00:1a,bridge=xenbr0', 'mac=00:21:f6:00:00:05,bridge=xenbr1'] OVM_simple_name = racnode1 disk = ['file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000a3949c3f3d394b21.img,xvda,w', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000065511085c4ad6864.img,xvdb,w', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000cdf8b3cd91c68ebd.img,xvdc,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000fb986a9247406641.img,xvdd,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008d3f9019fe32a304.img,xvde,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000046a344c5efa19a02.img,xvdf,w!', 'file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000c65d6d0a62ec75a0.img,xvdg,w!'] uuid = 0004fb00-0006-0000-cfab-2af11ac1f125 on_reboot = restart cpu_weight = 27500 memory = 2048 cpu_cap = 0 maxvcpus = 1 OVM_high_availability = False maxmem = 2048 OVM_description = racnode1 on_poweroff = destroy on_crash = restart bootloader = /usr/bin/pygrub name = 0004fb0000060000cfab2af11ac1f125 guest_os_type = default

PDFmyURL.com

vif_other_config = [] vfb = ['type=vnc,vncunused=1,vnclisten=127.0.0.1,keymap=en-us'] vcpus = 1 OVM_os_type = None OVM_cpu_compat_group = OVM_domain_type = xen_pvm [root@vmmgr u01]#

9 . Start the VMS

10 . Establish VNC to both nodes

After both VM are started establish a VNC sessions to both nodes for configuration of GI and RAC. Select and dedicate which node will be the first and second node.

PDFmyURL.com

Enter for each node hostname, public IP and private IP address, VIP, VIP name, domain, DNS, public and private interface etc as prompted.

Review and acknowledge by pressing YES.

PDFmyURL.com

Press YES

PDFmyURL.com

Wait for the installation to complete. Progress is

PDFmyURL.com

At the end

11. Verify Oracle 11gR2 11.2.03 GI and RAC installation.

You can verify the status using sqlplus.

PDFmyURL.com

You can verify the status using srvctl.

PDFmyURL.com

Some notes. Acknowledging events may clear some errors after a crash of the Oracle VM Server 3.1.1 VirtualBox VM.

Verifying the VM properties resolves the error

Unexpected error during login (com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_6000E Internal Error: Connection refused Connection refused Mon May 21 23:18:19 EEST 2012), please consult logs for details. Job Construction Phase begin() Appended operation Virtual Machine Start to object 0004fb0000060000cfab2af11ac1f125 (racnode1). Appended operation Bridge Configure Operation to object 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com). Appended operation Bridge Configure Operation to object 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com).

PDFmyURL.com

commit() Completed Step: COMMIT Objects and Operations Object (IN_USE): [EthernetPort] 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com) Operation: Bridge Configure Operation Object (IN_USE): [Vnic] 0004fb000007000047de4cdfa34b0a41 (00:21:f6:00:00:1a) Object (IN_USE): [Vnic] 0004fb000007000020f4233448d8d770 (00:21:f6:00:00:05) Object (IN_USE): [VmDiskMapping] 0004fb00001300006e3c1bc79509f857 Object (IN_USE): [VmDiskMapping] 0004fb0000130000ce67b1e5241157d7 Object (IN_USE): [VirtualDisk] 0004fb00001200008d3f9019fe32a304.img (ASM3) Object (IN_USE): [VmDiskMapping] 0004fb0000130000494d049341598540 Object (IN_USE): [VirtualDisk] 0004fb0000120000c65d6d0a62ec75a0.img (ASM5) Object (IN_USE): [VmDiskMapping] 0004fb0000130000267c771177271b00 Object (IN_USE): [VirtualMachine] 0004fb0000060000cfab2af11ac1f125 (racnode1) Operation: Virtual Machine Start Object (IN_USE): [VirtualDisk] 0004fb0000120000cdf8b3cd91c68ebd.img (ASM1)

PDFmyURL.com

Object (IN_USE): [VmDiskMapping] 0004fb00001300002036b8246f86e2c4 Object (IN_USE): [VirtualDisk] 0004fb000012000046a344c5efa19a02.img (ASM4) Object (IN_USE): [VirtualDisk] 0004fb000012000065511085c4ad6864.img (Oracle11203RAC_x86_64-xvdb.img (2)) Object (IN_USE): [EthernetPort] 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com) Operation: Bridge Configure Operation Object (IN_USE): [VirtualDisk] 0004fb0000120000fb986a9247406641.img (ASM2) Object (IN_USE): [VmDiskMapping] 0004fb00001300007b4b45d8dab8f783 Object (IN_USE): [VirtualDisk] 0004fb0000120000a3949c3f3d394b21.img (System.img (2)) Object (IN_USE): [VmDiskMapping] 0004fb000013000040f7df759023d559 Job Running Phase at 23:27 on Mon, May 21, 2012 Job Participants: [b6:10:7c:ba:39:89:4e:e8:95:b7:ca:1a:61:e9:81:69 (vmsrv33.gj.com)] Actioner Starting operation Bridge Configure Operation on object 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com) Bridge [xenbr1] already exists (and should exist) on

PDFmyURL.com

interface [eth2] on server [vmsrv33.gj.com]; skipping bridge creation Completed operation Bridge Configure Operation completed with direction ==> DONE Starting operation Virtual Machine Start on object 0004fb0000060000cfab2af11ac1f125 (racnode1) Completed operation Virtual Machine Start completed with direction ==> LATER Starting operation Bridge Configure Operation on object 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com) Bridge [xenbr0] already exists (and should exist) on interface [eth3] on server [vmsrv33.gj.com]; skipping bridge creation Completed operation Bridge Configure Operation completed with direction ==> DONE Starting operation Virtual Machine Start on object 0004fb0000060000cfab2af11ac1f125 (racnode1) Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg'] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected. stdout: Using config file /OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg .

PDFmyURL.com

Mon May 21 23:28:12 EEST 2012 Mon May 21 23:28:12 EEST 2012 at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507) at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444) at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378) at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121) at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65) at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:999) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822) at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245) at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source) at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596) at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52) at

PDFmyURL.com

com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822) at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245) at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source) at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218) at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309) at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136) at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100) at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81) at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)

PDFmyURL.com

at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401) at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459) at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42) at java.lang.Thread.run(Thread.java:662) Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://? uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg'] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected. stdout: Using config file /OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg . Mon May 21 23:28:12 EEST 2012 at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798) at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503) 42 more FailedOperationCleanup Starting failed operation Virtual Machine Start cleanup on object racnode1

PDFmyURL.com

Complete rollback operation Virtual Machine Start completed with direction=racnode1 Rollbacker Executing rollback operation Bridge Configure Operation on object 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com) Complete rollback operation Bridge Configure Operation completed with direction=DONE Executing rollback operation Virtual Machine Start on object 0004fb0000060000cfab2af11ac1f125 (racnode1) Complete rollback operation Virtual Machine Start completed with direction=DONE Executing rollback operation Bridge Configure Operation on object 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com) Complete rollback operation Bridge Configure Operation completed with direction=DONE Objects To Be Rolled Back Object (IN_USE): [EthernetPort] 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com) Object (IN_USE): [Vnic] 0004fb000007000047de4cdfa34b0a41 (00:21:f6:00:00:1a) Object (IN_USE): [Vnic] 0004fb000007000020f4233448d8d770 (00:21:f6:00:00:05) Object (IN_USE): [VmDiskMapping] 0004fb00001300006e3c1bc79509f857 Object (IN_USE): [VmDiskMapping]

PDFmyURL.com

0004fb0000130000ce67b1e5241157d7 Object (IN_USE): [VirtualDisk] 0004fb00001200008d3f9019fe32a304.img (ASM3) Object (IN_USE): [VmDiskMapping] 0004fb0000130000494d049341598540 Object (IN_USE): [VirtualDisk] 0004fb0000120000c65d6d0a62ec75a0.img (ASM5) Object (IN_USE): [VmDiskMapping] 0004fb0000130000267c771177271b00 Object (IN_USE): [VirtualMachine] 0004fb0000060000cfab2af11ac1f125 (racnode1) Object (IN_USE): [VirtualDisk] 0004fb0000120000cdf8b3cd91c68ebd.img (ASM1) Object (IN_USE): [VmDiskMapping] 0004fb00001300002036b8246f86e2c4 Object (IN_USE): [VirtualDisk] 0004fb000012000046a344c5efa19a02.img (ASM4) Object (IN_USE): [VirtualDisk] 0004fb000012000065511085c4ad6864.img (Oracle11203RAC_x86_64-xvdb.img (2)) Object (IN_USE): [EthernetPort] 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com) Object (IN_USE): [VirtualDisk] 0004fb0000120000fb986a9247406641.img (ASM2) Object (IN_USE): [VmDiskMapping] 0004fb00001300007b4b45d8dab8f783 Object (IN_USE): [VirtualDisk] 0004fb0000120000a3949c3f3d394b21.img (System.img (2)) Object (IN_USE): [VmDiskMapping] 0004fb000013000040f7df759023d559

PDFmyURL.com

Write Methods Invoked Class=InternalJobDbImpl vessel_id=10367 method=addTransactionIdentifier accessLevel=6 Class=VirtualMachineDbImpl vessel_id=9317 method=start accessLevel=6 Class=VnicDbImpl vessel_id=8010 method=lock accessLevel=6 Class=VnicDbImpl vessel_id=7888 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=9329 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=9339 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7354 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7404 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7529 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7571 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7590 method=lock accessLevel=6 Class=VnicDbImpl vessel_id=8010 method=lock accessLevel=6 Class=VnicDbImpl vessel_id=7888 method=lock accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9334 method=lock accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9344 method=lock accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9726 method=lock

PDFmyURL.com

accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9731 method=lock accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9736 method=lock accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9741 method=lock accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9746 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=9329 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=9339 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7354 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7404 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7529 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7571 method=lock accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7590 method=lock accessLevel=6 Class=VirtualMachineStartingEventDbImpl vessel_id=10376 method=notify accessLevel=6 Class=VirtualMachineStartingEventDbImpl vessel_id=10376 method=setDescription accessLevel=6 Class=EthernetPortDbImpl vessel_id=1888 method=addJobOperation accessLevel=6 Class=EthernetPortDbImpl vessel_id=1882

PDFmyURL.com

method=addJobOperation accessLevel=6 Class=InternalJobDbImpl vessel_id=10367 method=setCompletedStep accessLevel=6 Class=InternalJobDbImpl vessel_id=10367 method=setAssociatedHandles accessLevel=6 Class=EthernetPortDbImpl vessel_id=1882 method=setCurrentJobOperationComplete accessLevel=6 Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6 Class=InternalJobDbImpl vessel_id=10367 method=setTuringMachineFlag accessLevel=6 Class=VirtualMachineDbImpl vessel_id=9317 method=setCurrentOperationToLater accessLevel=6 Class=InternalJobDbImpl vessel_id=10367 method=setTuringMachineFlag accessLevel=6 Class=EthernetPortDbImpl vessel_id=1888 method=setCurrentJobOperationComplete accessLevel=6 Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6 Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6 Class=InternalJobDbImpl vessel_id=10367 method=setFailedOperation accessLevel=6 Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6 Class=VnicDbImpl vessel_id=8010 method=nextJobOperation accessLevel=6 Class=VnicDbImpl vessel_id=7888 method=nextJobOperation accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9741

PDFmyURL.com

method=nextJobOperation accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9344 method=nextJobOperation accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7529 method=nextJobOperation accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9736 method=nextJobOperation accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7590 method=nextJobOperation accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9726 method=nextJobOperation accessLevel=6 Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7354 method=nextJobOperation accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9731 method=nextJobOperation accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7571 method=nextJobOperation accessLevel=6 Class=VirtualDiskDbImpl vessel_id=9339 method=nextJobOperation accessLevel=6 Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6 Class=VirtualDiskDbImpl vessel_id=7404 method=nextJobOperation accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9334 method=nextJobOperation accessLevel=6 Class=VirtualDiskDbImpl vessel_id=9329 method=nextJobOperation accessLevel=6 Class=VmDiskMappingDbImpl vessel_id=9746

PDFmyURL.com

method=nextJobOperation accessLevel=6 Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6 Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6 Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6 Completed Step: ROLLBACK Job failed commit (internal) due to OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg'] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected. stdout: Using config file /OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg . Mon May 21 23:28:12 EEST 2012 Mon May 21 23:28:12 EEST 2012 com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg']

PDFmyURL.com

failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected. stdout: Using config file /OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg . Mon May 21 23:28:12 EEST 2012 Mon May 21 23:28:12 EEST 2012 at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507) at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444) at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378) at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121) at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65) at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:999) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

PDFmyURL.com

at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822) at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245) at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source) at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596) at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52) at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009) at sun.reflect.GeneratedMethodAccessor886.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822) at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245) at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source) at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)

PDFmyURL.com

at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309) at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330) at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290) at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136) at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100) at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81) at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773) at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401) at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459) at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42) at java.lang.Thread.run(Thread.java:662)

PDFmyURL.com

Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://? uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg'] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected. stdout: Using config file /OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg . Mon May 21 23:28:12 EEST 2012 at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798) at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503) 42 more End of Job -

1.

Conclusion

You had a look at the steps to upgrade Oracle VM server and Oracle VM Manager to version 3.1.1. You looked at the step by step guide to install two node Oracle 11gR2 11.2.0.3 RAC using templates. The errors that I encountered were also included as well as A way to work it around.

Do wnlo ad Oracle Virt ualizat io n Inst alling Oracle VM Se rve r 3.1.1, Oracle VM Manage r
PDFmyURL.com

3.1.2 and De plo ying Oracle RAC 11gR2 (11.2.0 .3) Oracle VM t e m plat e s Linux x86 6 4 bit f o r t e st co nf igurat io n Share t his:

Like t his:

Be the first to like this.

June 20, 2012 - Posted by g jile vski | o racle

stm7 n 1 o C e
1. 0 0

Rate This Thanks fo r yo ur effo rts. If, I have a hard disk which is directly attached to the OVM servers OS, ho w to deal with that? The OVM Manager is no t able to pick up the hard disk partitio n Eg. /dev/sda4 Comment by Praveen | August 24, 2012 | Reply 2. 0 0 Rate This

Thanks fo r the article. But I have a FC SAN as the sto rage. I fo und it impo ssible to create the server po o l in VM Manager since it always says that jo b is time-o ut. I searched all the do cs o n the net but fo und no thing helpful. Any suggestio ns? Thanks. Comment by Michael | September 4, 2012 | Reply 0 0 Rate This

So mebo dy suggests me to wipe o ut all the data o n the sto rage. Ive used dd co mmand to write the first 10 0 M o f the SAN to zero . But its in vain. The same erro r sho wed up again. To my surprise, Oracle o fficial do cumentatio n o n this is unclear and vague. I cant believe that Oracle VM, as Oracles the main virtualizatio n pro duct fo r x8 6 , is so fragile. Comment by Michael | September 4, 2012 | Reply 0 Hi, Lo o k at the erro r and wo rk it o ut. I canno t understand what is the pro blem that yo u
PDFmyURL.com

Rate This

are facing o ther than that the Server po o l creatio n times o ut. Co uld yo u bo unce the VM manager and give it a try? Make sure that yo u fo llo w the o rdered steps, do no t miss a step, and each step co mpletes succesfully befo re mo ving to the nest o ne. Did yo u synchro nise the NFS and repo sito ry co nfiguratio n? Try to get so mething that pro vides an indicatio n o f a pro blem Regards, Comment by gjilevski | September 4, 2012 | Reply 3. 0 0 Rate This

@Praveen, What yo u sho uld do is: lo gin to yo ur OVM server as ro o t and use parted co mmand to lo o k at the unused disk. So , fo r example, to carve o ut ~50 GB space and make it available to the VM Manager: [ro o t@o vm1 dev]# parted GNU Parted 1.8 .1 Using /dev/sda Welco me to GNU Parted! Type help to view a list o f co mmands. (parted) print free Mo del: HITACHI H10 1414SCSUN146 G (scsi) Disk /dev/sda: 147GB Secto r size (lo gical/physical): 512B/512B Partitio n Table: msdo s Number Start End Size Type File system Flags 1 32.3kB 10 7MB 10 7MB primary ext3 bo o t 2 10 7MB 118 4MB 10 78 MB primary linux-swap 118 4MB 147GB 146 GB Free Space (parted) mkpart primary 118 4MB 50 GB (parted) print free Mo del: HITACHI H10 1414SCSUN146 G (scsi) Disk /dev/sda: 147GB Secto r size (lo gical/physical): 512B/512B Partitio n Table: msdo s Number Start End Size Type File system Flags 1 32.3kB 10 7MB 10 7MB primary ext3 bo o t 2 10 7MB 118 4MB 10 78 MB primary linux-swap
PDFmyURL.com

3 118 4MB 50 .0 GB 48 .8 GB primary 50 .0 GB 147GB 9 6 .8 GB Free Space No w, yo u will see a /dev/sda3 device. Just create a fs o n it: [ro o t@o vm1 ~]# ll /dev/sd* brw-r 1 ro o t disk 8 , 0 Sep 6 16 :41 /dev/sda brw-r 1 ro o t disk 8 , 1 Sep 6 16 :27 /dev/sda1 brw-r 1 ro o t disk 8 , 2 Sep 6 16 :27 /dev/sda2 brw-r 1 ro o t disk 8 , 3 Sep 6 16 :41 /dev/sda3 brw-r 1 ro o t disk 8 , 16 Sep 6 16 :27 /dev/sdb brw-r 1 ro o t disk 8 , 17 Sep 6 16 :27 /dev/sdb1 brw-r 1 ro o t disk 8 , 18 Sep 6 16 :27 /dev/sdb2 [ro o t@o vm1 ~]# [ro o t@o vm1 ~]# [ro o t@o vm1 ~]# mkfs -t ext3 /dev/sda3 mke2fs 1.39 (29 -May-20 0 6 ) Filesystem label= OS type: Linux Blo ck size=40 9 6 (lo g=2) Fragment size=40 9 6 (lo g=2) 59 6 3776 ino des, 119 18 221 blo cks 59 59 11 blo cks (5.0 0 %) reserved fo r the super user First data blo ck=0 Maximum filesystem blo cks=429 49 6 729 6 36 4 blo ck gro ups 3276 8 blo cks per gro up, 3276 8 fragments per gro up 16 38 4 ino des per gro up Superblo ck backups sto red o n blo cks: 3276 8 , 9 8 30 4, 16 38 40 , 229 376 , 29 49 12, 8 19 20 0 , 8 8 4736 , 16 0 56 32, 26 5420 8 , 40 9 6 0 0 0 , 79 6 26 24, 11239 424 Writing ino de tables: do ne Creating jo urnal (3276 8 blo cks): do ne Writing superblo cks and filesystem acco unting info rmatio n: do ne This filesystem will be auto matically checked every 39 mo unts o r 18 0 days, whichever co mes first. Use tune2fs -c o r -i to o verride. [ro o t@o vm1 ~]# mkdir /1st_50 G [ro o t@o vm1 ~]# mo unt /dev/sda3 /1st_50 G [ro o t@o vm1 ~]# df -h
PDFmyURL.com

[ro o t@o vm1 ~]# df -h Filesystem Size Used Avail Use% Mo unted o n /dev/sdb1 3.0 G 6 35M 2.2G 23% / /dev/sda1 9 9 M 28 M 6 7M 30 % /bo o t tmpfs 36 0 M 0 36 0 M 0 % /dev/shm no ne 36 0 M 40 K 36 0 M 1% /var/lib/xensto red /dev/sda3 45G 18 1M 43G 1% /1st_50 G [ro o t@o vm1 ~]# Add to /etc/fstab: /dev/sda3 /1st_50 G ext3 defaults 0 0 No w, yo u can expo rt it as NFS share as Guenadi has sho wn abo ve: [ro o t@o vm1 ~]# cat /etc/expo rts /ist_50 G *(rw,sync,no _wdelay,insecure_lo cks,no _ro o t_squash) [ro o t@o vm1 ~]# service nfs status rpc.mo untd (pid 36 9 9 ) is running nfsd (pid 36 9 6 36 9 5 36 9 4 36 9 3 36 9 2 36 9 1 36 9 0 36 8 9 ) is running [ro o t@o vm1 ~]# service nfs restart Shutting do wn NFS mo untd: [ OK ] Shutting do wn NFS daemo n: [ OK ] Shutting do wn NFS services: [ OK ] Starting NFS services: [ OK ] Starting NFS daemo n: [ OK ] Starting NFS mo untd: [ OK ] [ro o t@o vm1 ~]# No w, yo u can use this 50 G space fro m yo ur OVM Manager co nso le. Ho pe that helps. Comment by jcnars | September 6, 2012 | Reply 0 0 Rate This

Thanks Guenadi fo r taking the time to list the steps. I go t it wo rking(finally) with the lo cal hard disk by keeping it as raw i.e. initializatio n fro m BIOS but NO initializatio n fro m the OS and file system o f these hard disks. This wo rks no t THE so lutio n to be used in pro ductio n env. tho ugh. I will keep yo ur NFS suggestio n if need be fo r future additio nal disks. Thanks again. Regards, Praveen
PDFmyURL.com

Comment by Praveen | September 7, 2012 | Reply 0 0 Rate This

@Praveen, Thanks. Co uld yo u be mo re specific as to the exact details? I didnt quite fo llo w ho w yo u made it to wo rk. Maybe a blo g entry? I am sure theres a lo t o f interest in the so lutio n o f using up the lo cal disks wo rk (fo r TEST/DEV OVM setup) and yo urs co uld be a useful o ne. Regards jcnars Comment by jcnars | September 7, 2012 0 Hi, Hi, Co uld yo u please elabo rate further o n the steps that yo u did as I believe readers can be interested in Regards, Comment by gjilevski | September 7, 2012 0 0 Rate This 0 Rate This

Happy to do this, when I can do so mething fro m scratch so that I co uld get the right screen-sho ts. No t sure ho w so o n but. Thanks fo r yo ur interest. -Praveen On Sat, Sep 8 , 20 12 at 5:38 AM, Guenadi N Jilevskis Oracle BLOG wro te: > ** > gjilevski co mmented: Hi, Co uld yo u please elabo rate further o n he > steps that yo u did as I belive reades can be interested in. Regards, > Comment by Praveen Rajendra Babu | September 13, 2012 0 0 Rate This

Happy to do this, when I can do so mething fro m scratch so that I co uld get the right screen-sho ts. No t sure ho w so o n but. Thanks fo r yo ur interest. If yo u have clean slate hard-disks, I co uld spare an ho ur fo r a remo te sessio n
PDFmyURL.com

fo r Aus.(Melbo urne) time. My co nfig. is 4 separate hard disks. Kept 1 disk separate and merged the o ther 3 as RAID. The 1 separate hard disk fo r the OVM server installatio n and the o ther fo r the OVM repo sito ry. -Praveen Comment by Praveen | September 13, 2012 0 Hi, Do no t wo rry. If yo u can po int it o ut is fine else no biggie. Regards, Comment by gjilevski | September 13, 2012 4. 0 0 Rate This 0 Rate This

co uld yo u please po int o ut the Oracle VM Manager Installatio n Instructio ns and filesystems layo ut thanks in advance Comment by samjaz | September 16, 2012 | Reply 0 Hi, Lo o k at Upgrade Oracle VM Manager 3.0 .3 to Oracle VM Manager 3.1.1 ht t p://gjile vski.wo rdpre ss.co m /20 12/0 6 /20 /o racle -virt ualizat io n-inst allingo racle -vm -se rve r-3-1-1-o racle -vm -m anage r-3-1-1-and-de plo ying-o racle rac-11gr2-11-2-0 -3-o racle -vm -t e m plat e s-linux-x86 -6 4 -bit -f o r-t e st co nf igurat io n/? pre vie w=t rue &pre vie w_id=4 6 4 4 &pre vie w_no nce =6 0 5 f 7 b5 7 dd Install and co nfigure Oracle VM Manager 3.0 .3 ht t p://gjile vski.wo rdpre ss.co m /20 12/0 6 /19 /o racle -virt ualizat io n-inst allingo racle -vm -se rve r-3-0 -3-o racle -vm -m anage r-3-0 -3-and-de plo ying-o racle rac-11gr2-11-2-0 -3-o racle -vm -t e m plat e s-linux-x86 -6 4 -bit -f o r-t e st co nf igurat io n/ Regards, 0 Rate This

PDFmyURL.com

Comment by gjilevski | September 20, 2012 | Reply 5. 0 0 Rate This

Reblo gged this o n Apps DBA no t e s. Comment by Mohammed | September 26, 2012 | Reply 6. 0 0 Rate This

Hi Guenadi thanks fo r the info o n OVM3.1 quite different to OVM 2 nice no te. Regards, Gavin Comment by Gavin | October 17, 2012 | Reply 7. 0 0 Rate This

Hi gjilevski, Great po st! Very useful info rmatio n. Few questio n to s ask: 1. do yo u need to build the Virtual Machine racno de1 and racno de2 o n VirtualBo x like yo u did fo r svr33 & vmmgr? 2. ho w do yo u lo gged into racno de1 and racno de2? 3. Have yo u tried the newest deplo ycluster to o l with OVM 3.1.1? (ht t p://www.o racle .co m /t e chne t wo rk/pro duct s/clust e rware /o ve rvie w/raco vm de plo yclust e r-t o o l-16 35 5 19 .pdf ) Acco rding to the Do c, I can skip yo ur step 9 Start the VMS. But no t sure! Po ssible to update this po st with the newest deplo ycluster to o l?. Best Regards, Comment by jlong | November 8, 2012 | Reply 0 Hi, 1. No . I build using VirtualBo x o nly vmmgr and vmsrv33. The virtual servers raclinux1 and raclinux2 are built o n vmsrv33 as a result o f creating virtual machines using the RAC templates fro m VM Manager. 2. Lo o k in the po st fo r establishing a vnc co nnectio n. I did it fro m the VM Manager.
PDFmyURL.com

Rate This

Yo u co uld try using vncviewer as well. It is the same but invo ke it o n the co mmand line instead o f fro m the VMManager. 3. The to o l was no t available when I wro te the article. I can give it a try. It sho uld simplify the setup. I did no t use it. I can update the article when get a chance to play with the to o l. Regards, Comment by gjilevski | November 8, 2012 | Reply

Leave a Reply
Enter your comment here...

Pre vio us | Ne xt

i d a n e uG
Theme: Andreas0 4 by Andreas Viklund. Blo g at Wo rdPress.co m .

PDFmyURL.com