Académique Documents
Professionnel Documents
Culture Documents
Kai Yu is a Senior System Engineer and works within the Dell Oracle Solutions Lab. He has over 14 years Oracle DBA and Solutions Engineering experience. His specialties include Oracle RAC, Oracle EBS and OVM. He is an avid author of various Oracle technology articles and frequent presenter at Oracle Open World and Collaborate. He is also the Oracle RAC SIG US Event Chair. David Mar is an Engineering Manager for the Dell | Oracle Solutions group. He directs a team across the globe to create Oracle RAC reference configurations and build best practices based upon tested and validated configurations. He has worked for Dell since 2000 and has a MS in Computer Science and BS in Computer Engineering from Texas A&M University.
Physical View
Logical View
Physical View
Scalability
Database 3 Database 4
Physical Grid
DB 1 OS DB 2
OS
OS
VM Server Pool
DB 3 OS OS OS OS OS
OVERVIEW OF EQUALLOGIC
Storage Pool
4 Gigabit network ports per controller 2 GB cache per controller Intelligent automated management tools Self Managing Array Linear Scalability Data Protection Software
Storage Pool
Front View
Back View
Energy efficient PowerEge M100e enclosure 11th generation blade servers available Intel Nehalem processors Intel QuickPath memory technology Three redundant fabrics 16 half height blade servers
11
1 PHYSICAL GRID
3 MANAGEMENT
Oracle Grid Control to manage both physical Grid and virtual Grid Consider HA for production management Manage externally out of the Blade Cluster, to avoid single point of failure Dell Servers enable health monitoring from EM
4 - STORAGE
GRID IMPLEMENTATION
16
GRID IMPLEMENTATION
Implementation Tasks
Configure Grid Control Management Infrastructure Configure EqualLogic Shared storage Configure Physical Grid Configure Virtual Grid
19
iSCSI storage connections Open-iSCSI administration utility to configure host access to storage volumes: eth2-iface on eth2, eth3-iface on eth3 Linux Device Mapper to establish mutipath devices to storage alias:
/etc/multipath:
multipaths { multipath { } wwid 36090a028e093dc7c6099140639aae1c7 alias ocr-crs
22
23
25
eth0 : public in dom0, presented to domU through xenbr0 eth2, eth3: iSCSI connections eh4 and eth4 bonded to bond0 for the private interconnect between VMs for 11g RAC configuration through xenbr1 Secure OVM agent is running: service ovs-agent status
26
IO modules
A1 B1, B2 C1,C2
connection
Public Network iSCSI connections Bonded to bond0
IP
155.16.9.71-78 10.16.7.238-235 192.168.9.82-85
domU
eth0
eth1
Customize the default Xen bridge configuration a. Stop the default bridges/etc/xen/scripts/network-bridges stop b. Edit /etc/xen/xend-config.sxp: replace line: (network-script network-bridges) with (network-script network-bridges-dummy) a. Edit /etc/xen/scripts/network-bridges-dummy : /bin/true b. Create infcg-bond0, ifcfg-xenbr0, ifcfg-xenbr1 scripts in /etc/sysconfig/network-scripts
27
ifcfg-eth4
ifcfg-bond0
ifcfg-xenbr0
ifcfg-xenbr1
DEVICE=xenbr1
ONBOOT=yes
DEVICE=eth4 DEVICE=bond0 DEVICE=xenbr0 ONBOOT=yes ONBOOT=yes TYPE=Ethernet MASTER=bond0 BOOTPROTO=none TYPE=Bridge USERCTL=no BRIDGE=xenbr1 IPADDR=155.16.9.82
NETMASK=255.255.0.0
TYPE=Bridge
TYPE=Bridge NETMASK=255.255.255.0
Check the multipath devices : ls /dev/mapper/* /dev/mapper/blade-data6 /dev/mapper/mpath5 /dev/mapper/ovs_data4p1 /dev/mapper/blade-data6p1 /dev/mapper/vmocr-css1 /dev/mapper/vmocr-css1p1 /dev/mapper/vmracdb1 /dev/mapper/vmracdb1p1 /dev/mapper/ovs_data4
28
Fcb
/dev/mapper/ovs_data4p1
f) edit /etc/fstab:
#/dev/sda3 /dev/mapper/ovs_data4p1 /OVS /OVS ocfs2 defaults ocfs2 _netdev,datavolume,nointr 1,0 00
29
Connect the VM servers to Enterprise Manager Grid Control Meet pre-requisites: a. Oracle user with oinstall group b. Oracle users ssh user equivalence between dom0 & OMS
30
31
32
33
34
35
Shared disk
36
Attached the storage to to the guest VM a) vm.cfg: disk = [''file:/OVS/sharedDisk/racdb.img,xvdc,w!', b) vm.cfg: disk = ['phy: /dev/mapper/vmracdbp1, xvdc,w!', Exposed Xen bridge for virtual network interface vm.cfg: vif = ['bridge=xenbr0,mac=00:16:3E:11:8E:CE,type=netfront', 'bridge=xenbr1,mac=00:16:3E:50:63:25,type=netfront',] xenbr0 - eth0 in the guest VM xenbr1 eth1 in the guest VM
37
38
41
42
SUMMARY
Dell Grid POC Project: Pre-built Grid with physical and virtual grids Grid control as the unified management solution for the Grid Dell blades as the platform of the Grid: RAC node and VM servers EqualLogic provides the shared storage for the physical and virtual Grid The Grid provides the infrastructure to consolidate the enterprise applications as well as the RAC databases Acknowledge the support of Oracle engineers: Akanksha Sheoran, Rajat Nigam, Daniel Dibbets, Kurt Hackel, Channabasappa Nekar,
Premjith Rayaroth, and Dell Engineer: Roger Lopez
References:
Practices of Deployment of Oracle ASM with Dell EqualLogic PS iSCSI Storage System, Dell White Paper 2 . Oracle Press Release: Oracle Enterprise Manager Extends Management to Oracle VM Server Virtualization 3. Oracle Enterprise Manager Concepts 10g Release 5 (10.2.0.5) Part Number B31949-10 4. Oracle Enterprise Manager Grid Control ReadMe for Linux x86-64 10g Release 5 (10.2.0.5) April 2009 5. How to Enable Oracle VM Management Pack in EM Grid Control 10.2.0.5, Metalink Note #: 781879.1 6. Oracle VM: Converting from a Local to Shared OVS Repository, Metalink note # 756968.1 7. How to Add Shared Repositories to Oracle VM Pools with Multiple Servers, Metalink Note # 869430.1 8. Implementing Oracle Grid: A successful Customer Case Study, Kai Yu, Dan Brint, IOUG Select Journal Volume 16, Number 2, Second Quarter 2009 9. Deploying Oracle VM Release 2.1 on Dell PowewrEdge servers and Dell/EMC storage Dell white paper 10. Dell Reference Configuration Deploying Oracle Database on Dell EqualLogic PS5000XV iSCSI Storage A Dell Technical White Paper 11. Technical Best Practices for Virtualization & RAC Oracle RAC SIG webseminar slides, Michael Timpanaro-Perota & Daniel Dibbets
1. Best
44