An Oracle White Paper

August 2014

Oracle VM 3:
Change the Server Pool File System &
Repositories (SAN)

White Paper Change the Server Pool File System & Repositories (SAN)

Contents
Introduction ...................................................................................... 1
Solution Overview............................................................................ 2
Only supported with Oracle VM 3.2.8+ .................................................2
Important terms used throughout the document ...................................2
Goal of the solution...............................................................................3
A note about destroying the server pool ...............................................3
The overall solution at a glance ............................................................3
Key solution highlights explained ..........................................................4
Maintain continuous replication of storage repositories .....................4
Use replicated disks to create a new pool .........................................5
The device special file name will change ..........................................5
The storage repository names remain the same ...............................5
The server pool name remains the same ..........................................6
Process flow .........................................................................................6

Step 1: Preparation Prior to Implementation .................................... 7

Step 1.1: Write down the server pool VIP .............................................7
Step 1.2: Catalog current storage .........................................................7
Step 1.3: Establish continuous replication of storage ............................7
Step 1.4: Perform ad-hoc backup of Oracle VM servers .......................8

Step 2: Release Ownership of Repositories .................................... 9

Step 2.1: Perform ad-hoc backup of Oracle VM database ....................9
Step 2.2: Stop Oracle VM guests running in the pool ..........................11
Step 2.3: Release ownership of current repositories ...........................11

Step 3: Destroy Current Server Pool ............................................. 13

Step 3.1: Remove Oracle VM servers from pool .................................14
Step 3.2: Delete the empty server pool ...............................................15
Step 3.3: Perform final synchronization of storage ..............................16

Step 4: Delete Source Disks .......................................................... 16

Step 4.1: Un-map source LUNs ..........................................................17

White Paper Change the Server Pool File System & Repositories (SAN)

Step 4.2: Delete the physical disks from DB .......................................17

Step 5: Discover Target Disks ....................................................... 18

Step 5.1: Map the target LUNs ...........................................................19
Step 5.2: Discover the new storage platform ......................................19
Step 5.3: Refresh the Oracle VM database.........................................19
Step 5.3.1: If you are using Fibre Channel ......................................19
Step 5.3.2: If you are using iSCSI ...................................................20

Step 6: Rebuild the Server Pool .................................................... 21

Step 7: Take Ownership of Target Repositories ............................ 24
Step 7.1: Refresh the target disks .......................................................24
Step 7.2: Refresh the target repositories ............................................25

Step 8: Start Oracle VM Guests .................................................... 27

White Paper Change the Server Pool File System & Repositories (SAN)

Introduction
This white paper explains how to change the device special file (DSF) being used as a server
pool file system. This solution can be applied to a variety of situations, but the most common
need for this solution is migrating the pool file system and storage repositories used by a
server pool from a legacy storage array to a new or different storage array. In this case,
system administrators are charged with migrating storage repositories as well as pool file
systems. It is the pool file system that poses the challenge.
Oracle VM 3 does not currently have a built-in tool or way to change the pool file system
without incurring downtime of all running Oracle VM guests. This is because the server pool
must be temporarily destroyed and rebuilt using the new pool file system. There are two major
ways to approach the migration from one pool file system to another.
The first approach presented in this white paper is straightforward with well defined stages that
make it easier for people to follow. This approach involves stopping all Oracle VM guests,
temporarily destroying the server pool and rebuilding the server pool again using a new pool
file system.
Other than this brief explanation, we will not be explaining how to accomplish the second
approach at all since it is a little more complex with slightly blurred stages and doesnt really
provide any advantage over the first approach. This approach involves using one Oracle VM
server from the legacy server pool to build a new server pool using the new pool file system,
then migrating the remaining Oracle VM servers one-by-one until everything is up and running
in the new pool. You still incur an outage of all Oracle VM guests at various stages using this
method and you will need another server pool VIP, so there is really no advantage to this
process except in cases where you cannot tolerate an outage of all virtual machines during the
same maintenance window.

White Paper Change the Server Pool File System & Repositories (SAN)

Solution Overview
This solution assumes all SAN storage related to a single server pool is being migrated to a new storage
platform including the server pool file system and all storage repositories. In addition, the process of
destroying and then creating a new server pool also changes the OCFS2 cluster ID. This means the
storage repositories will not be available to the new server pool unless these they are first released from
ownership by the Oracle VM Manager.
This document is divided into several parts representing each step in the entire process. The first part
of the document explains the process, process flow, concepts and terms used throughout the
document. The remaining parts of the document are dedicated to accomplishing the steps one-by-one.

Important!

Please take the time to read and understand the solution before moving directly to the step-by-step process
articulated in this document.

Only supported with Oracle VM 3.2.8+

Your Oracle VM platform must be at release 3.2.8 or higher; you must upgrade your Oracle VM
platform to a minimum release level of Oracle VM 3.2.8 before you can successfully use this solution.
The following table shows supported and unsupported versions of Oracle VM:
ORACLE VM RELEASE

SUPPORT STATUS

REMOTE REPLICATION

Oracle VM 3.2.1

Not supported

Oracle VM 3.2.2

Not supported

This release of Oracle VM does not allow Oracle VM guests to be migrated from
SAN repositories to Unassigned Virtual Machines.
Therefore ownership of
repositories cannot be released which is a critical step in the process.

Oracle VM 3.2.3

Not supported

Oracle VM 3.2.4

Not supported

Oracle VM 3.2.6

Not supported

Oracle VM 3.2.7

Not supported

Oracle VM 3.2.8

Supported

Solution is fully tested and supported

Oracle VM 3.3.1

Supported

Solution is supported

Table 1: Supported releases of Oracle VM 3

Important terms used throughout the document

For the purpose of this paper, we will use the following terms throughout the document:
Model: A common term used to everything contained in the entire Oracle VM management
database; everything you can see and access in the Oracle VM Manager user interface.
PoolFS: Server pool file system
SAN: Any storage presented using block protocols such as Fibre Channel (FCP) or iSCSI
Source storage: This refers to the legacy, old or current PoolFS and storage repositories associated
with a single server pool; the source contains the original data that is being migrated from the legacy
storage platform to the new storage platform.
Storage platform: This refers to either a physical storage array or logical volumes on a physical
storage array. Perhaps you are migrating existing storage from one physical array to another, or
perhaps you are instead migrating storage from one logical volume to another logical volume that
both reside on the same physical storage array.

White Paper Change the Server Pool File System & Repositories (SAN)

Target storage: This refers to the new, cloned or replicated PoolFS and storage repositories that will
be used to create the new server pool.

Goal of the solution

The goal of this solution is to migrate all of the storage being used for a server pool from a legacy
storage array or volume to a new storage array or volume. This particular solution assumes you are
migrating the storage repositories and your PoolFS from your current storage platform to a new
storage platform. The process will require downtime during the final storage replication and migration
stages.
As stated previously, this process will require you to rebuild the server pool using the new PoolFS.
Despite the ominous sounding step of destroying the server pool, it does not impact the Oracle VM
Manager, Oracle VM servers, Oracle VM guests, networking or anything else related to your existing
Oracle VM platform. The actual rebuilding of the server pool should take a minimal about of time.
Basically, you remove the servers from the source pool, destroy the source pool, refresh the storage
and then build a new pool with the same servers. There is no need to reinstall or reconfigure anything
other than creating the target server pool.

A note about destroying the server pool

The process described in this document requires you to destroy the current server pool and then
rebuild the same server pool using the replicated disks from the new storage platform. This is a
requirement and there is no way around it.
However, destroying the server pool is not as dramatic as it sounds. Nothing about Oracle VM is really
impacted by this step other than the server pool wont exist for a few minutes while we build the new
server pool. You will not need to reinstall or reconfigure anything on the Oracle VM servers; all of the
bonding, networking and anything else about the Oracle VM servers remain unchanged.
In fact, there is nothing destructive about any of the steps related to your Oracle VM guests; you still
have all the parts and pieces to rebuild the same server pool and your virtual machines are quite safe.
We explain how to back up your Oracle VM management database and the Oracle VM servers in steps
1 & 2 that make it very easy to recover in the unlikely case you need to back out of the solution.

Important!

Destroying the server pool is not as dramatic as it sounds. Everything related to your Oracle VM guests is
still there for you to use and it is very easy to recover if you follow the instructions in steps 1 and 3.

The overall solution at a glance

Figure 1 below shows the state of the storage for a fictional server pool prior to beginning the
migration process while Figure 2 the state of the storage for the new server pool after completing the
migration process.

White Paper Change the Server Pool File System & Repositories (SAN)

This is shown only to explain the differences between the beginning and end of the solution. The
numbered items in each of the illustrations highlight key points that are accompanied by detailed
explanations below.

Oracle VM Servers
Legacy Storage
/poolfsmnt/0004fbxxxx6eb0 [Mypool2]

/dev/mapper/360a98xxxx3056

PoolFS LUN

/OVS/Repositories/0004fbxxxx20f8 [Mypool2 Repo1]

/dev/mapper/360a98xxxx302d

RepoFS LUN

/OVS/Repositories/0004fbxxxx01fa [Mypool2 Repo2]

/dev/mapper/360a98xxxx3845

RepoFS LUN

New Storage
PoolFS Clone

RepoFS Clone
RepoFS Clone

Figure 1: Showing the overall state of the storage before beginning the process

Oracle VM Servers
Legacy Storage
/poolfsmnt/0004fb00xxxx0ad6 [Mypool2]

/dev/mapper/360a98xxxx82a0

PoolFS LUN

/OVS/Repositories/0004fbxxxx6cc2 [Mypool2 Repo1]

/dev/mapper/360a98xxxx4453

RepoFS LUN

/OVS/Repositories/0004fbxxxx1b3c [Mypool2 Repo2]

/dev/mapper/360a98xxxx4456

RepoFS LUN

New Storage
PoolFS Clone

RepoFS Clone
RepoFS Clone

Figure 2: Showing the final overall state of the storage after completing the process

Key solution highlights explained

So lets break down some of the more important concepts shown in Figure 1 & Figure 2 above needed
to successfully complete the migration process.
Maintain continuous replication of storage repositories

To decrease the downtime associated with the migration process, ensure that you maintain a
continuous replication relationship between the source and target storage platforms until you are ready
to actually stop your Oracle VM guests and release ownership of the storage repositories (see Figure 1,
item 1). You should start the replication process well before you begin the actual migration
documented in this white paper.

White Paper Change the Server Pool File System & Repositories (SAN)

The replication relationship should be maintained and periodically refreshed minimize the delta
between the source and replica storage; this just means periodic replication, not synchronous
replication. For example, you would set up continuous replication between the source and target
storage platforms with scheduled refreshes until the server pool is destroyed in Step 3 of this
document. Notice in Figure 2, item 1 that the replication relationship no longer exists between the
source and target storage repositories after the final refresh has been completed and the old server pool
is destroyed.
Use replicated disks to create a new pool

The process is completely predicated on replicating/cloning the current LUNs being used for the
current server pool and then completely rebuilding the server pool using the replicated disks. You will
use the current LUNs as shown in Figure 1, item 2 until Step 6 where you build an entirely new server
pool using the replicated LUNs from the new storage platform as shown in Figure 2, item 2.
The device special file name will change

Compare item 3 in Figure 1 & Figure 2; notice that the WWID (page83 ID) of the device special files
(DSF) for the repository LUNs are different. Our illustration shows a truncated device name showing
only the last four significant digits of the page83 ID this is simply to help us fit the illustrations on
the page.
During the process you will stop all the Oracle VM guests and destroy the server pool. Destroying the
server pool is a requirement because the DSF name for the PoolFS will be different when you present
the new PoolFS from the new storage platform. Since Oracle VM has no built-in features to simply
change the device special file, we need to destroy the current server pool and create a new one using
the same servers but a different PoolFS. The new PoolFS DSF will have a different WWID.
The storage repository names remain the same

Notice in both Figure 1 & Figure 2, item 4 that the storage repository names will remain the same
throughout the entire process. The simple name of the repository is contained in a hidden file within
the top level directory named .ovsrepo which will be seen in the Repositories tab of Oracle VM
Manager. Each newly discovered repository will be assigned a random ID by the Oracle VM Manager
as it is added to the management database and will be mounted to /OVS/repositories on the Oracle
VM servers using the new repository ID.
Note

Keep in mind that the Oracle VM guests will not be seen in the new server pool until the target repositories
have been refreshed in Step 9 of this document.

Your Oracle VM guests will remain safe and entirely intact during the entire process. It is normal for
the Oracle VM guests to remain undiscovered in the new server pool until the repositories have been
discovered, presented to Oracle VM servers in the new pool and the repositories refreshed in Step 9 of
this document.

White Paper Change the Server Pool File System & Repositories (SAN)

The server pool name remains the same

Figure 2, item 5 shows that you will use the same server pool simple name when you create the new
pool. Item 5 also shows that the newly discovered PoolFS will be assigned a random ID by the Oracle
VM Manager as it is added to the management database and will be mounted to /poolfsmnt on the
Oracle VM servers using the new PoolFS ID.

Process flow
The table of contents shows all the steps for the entire process. However, we will show the following
process flow diagram at the beginning of each step to help you keep track of where you are in the
overall process.
Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

Figure 3: Process flow

White Paper Change the Server Pool File System & Repositories (SAN)

Step 1: Preparation Prior to Implementation

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

There will be an outage associated with the migration project since you will be destroying and
rebuilding your target server pool You should plan ahead to ensure the outage window is as small as
possible.
The following steps should be performed weeks or days before the planned outage. The choice of
timing to begin the following steps is completely at your discretion since you know how much storage
you will be migrating and how long the preparatory steps might take in your data center given your
operational governance and standard operating procedures.

Step 1.1: Write down the server pool VIP

You will need to remember the server pool VIP since you will use the same one to create the new
server pool after deleting the current server pool.

Step 1.2: Catalog current storage

You will need to get a list of current OCFS2 formatted devices from one of your servers before you
start the process. This will provide a list of the PoolFS and storage repositories that are using OCFS2.

Step 1.3: Establish continuous replication of storage

Begin the storage replication of PoolFS and repositories well ahead of your scheduled outage. You will
perform a final refresh/synchronization of the PoolFS and repositories as part of Step 3 when we
destroy the server pool. Perform periodic refreshes of the replicated storage in between the time you
first create the replication relationship and the final refresh. Performing periodic refreshes of the
replica storage will reduce the time needed for the outage window.

Important!

Start your storage replication a day or two before your scheduled outage

White Paper Change the Server Pool File System & Repositories (SAN)

The replication relationship between the source and target storage platform should remain in place
until you destroy the server pool. You will use software on your storage array to accomplish the
storage replication. The following table shows some examples of storage replication software from
various storage vendors.
STORAGE VENDOR

LOCAL REPLICATION

REMOTE REPLICATION

EMC

TimeFinder

SRDF

Hitachi Data Systems

ShadowImage

TrueCopy

Network Appliance

Snapshot cloning

SnapMirror

Oracle ZFS storage appliance

Snapshot cloning

Remote replication

Table 2: Examples of replication software

If you do not have enterprise class storage with robust replication software, then you can use whatever
standard Linux tools you are comfortable using. The following table shows a few Linux commands
that can be used if you dont have enterprise class replication tools.
LINUX COMMAND

COMMENTS

rsync

Recommended: Faster, displays progress of copies and only copies changed blocks on
subsequent writes

cpio

Not recommended: Slow, complex to use for remote copies and copies both changed
blocks and unchanged blocks on subsequent writes

dd

Not recommended: Slow, complex to use for remote copies and copies both changed
blocks and unchanged blocks on subsequent writes

scp

Not recommended: Slow and copies both changed blocks and unchanged blocks on
subsequent writes

Table 3: Examples of common Linux tools

The rsync command is probably the best Linux tool to use since it will keep track of changed blocks
between the two copies. The rsync command will only copy changed blocks of data on subsequent
copies of the same repository, so rsync will be much faster for the final synchronization of all storage
repositories during Step 3.

Step 1.4: Perform ad-hoc backup of Oracle VM servers

This step combined with the ad-hoc Oracle VM management database backup in step 2 will ensure an
easy way to back out of the destroying the server pool performed in Step 3. It is highly unlikely that
you will need to back out of the process after destroying the server pool, but it is safer to error on the
side of caution.
The Oracle VM related configuration information on the Oracle VM servers is almost static with very
few changes ever being written to the servers once they have been put into production. It is assumed
you will not be making any major changes to the servers prior to the scheduled outage. So, it is okay to
back the servers up weeks or days in advance of the planned outage.
We will simply make copies of a few key directories and files on the Oracle VM servers. But the copies
we make on the Oracle VM servers combined with the Oracle VM management database backup we
perform in step 2 should allow you to completely recover from the destroyed server pool in less than
twenty or thirty minutes.

White Paper Change the Server Pool File System & Repositories (SAN)

It should be fine to perform the Oracle VM server backup a day or two prior to your outage, but the
backup of the Oracle VM management database should be performed in Step 2 just before you begin
the outage.
Log onto each Oracle VM server in the target server pool and perform the three tasks show in the
following two screen shots. The first task is to save a copy of the local Berkeley DB on the server as
shown in

[root@myserver1 ~]# cd /etc/ovs-agent

[root@myserver1 ovs-agent]# cp -rp db db.save
[root@myserver1 ovs-agent]#

Figure 4: Save a copy of the local Berkeley DB on each server

Save the OCFS2 configuration as shown in Figure 5 below.

[root@myserver1
[root@myserver1
[root@myserver1
[root@myserver1

~]# cd /etc
etc]# cp -rp ocfs2 ocfs2.save
etc]# cd /etc/sysconfig
sysconfig]# cp -rp o2cb o2cb.save

Figure 5: Save a copy of the current OCFS2 configuration

The local server side database and OCFS configuration files are the only things that are changed when
you delete or remove an Oracle VM server from a server pool. If you need to back out of the process,
the above files and directories can simply be restored along with the Oracle VM management database
you will back up in the next step.

Step 2: Release Ownership of Repositories

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

You must release ownership of the storage repositories before you perform the final refresh or
synchronization of the replicated storage.

Step 2.1: Perform ad-hoc backup of Oracle VM database

You need to take an ad-hoc backup of your Oracle VM management database as a first step. This is
documented in the Oracle VM administration guide, but the following screen shot shows an example
of creating a backup of the database. This should only take a few minutes and will be worth your time.

White Paper Change the Server Pool File System & Repositories (SAN)

Oracle VM 3.2 and 3.3 have slight variations in the name and location of the backup script. Use the
backup process show in Figure 6 for Oracle VM 3.2:

[root@mymanager ~]# cd /u01/app/oracle/ovm-manager-3/bin/

[root@mymanager bin]# ./createBackup.sh -u Admin n before-deleting-mypool2
Backing up the Oracle VM Manager MySQL Database...
Please enter the Oracle VM manager user password:
INFO: Succesfully backed up database as before-deleteing-mypool2-20140821_073506
[root@mymanager bin]#

Figure 6: Oracle VM 3.2 - creating a very quick ad-hoc backup of the Oracle VM management database (MySQL)

Use the backup process show in Figure 7 for Oracle VM 3.3:

[root@mymanager ~]# cd /u01/app/oracle/ovm-manager-3/ovm_tools/bin

[root@mymanager bin]# ./BackupDatabase -u admin -n before-deleting-mypool2
Enter your OVM Manager password:
INFO:

Backup job starting with destination:

/ovmm-backups/before-deleting-mypool2-20140821_074327

Job Id
= 'Start Backup to: before-deleting-mypool2(1408974206937) Uri: https://
localhost:7002/ovm/core/wsapi/rest/Job/1408974206937'
Job Name = 'Start Backup to: before-deleting-mypool2'
INFO: Backup job finished
[root@mymanager bin]#

Figure 7: Oracle VM 3.3 - creating a very quick ad-hoc backup of the Oracle VM management database (MySQL)

You can use this backup along with the saved files and directories from the Oracle VM servers taken in
the previous step to back out of this entire process.
Note

If you did not choose to use MySQL as the database engine during the install of Oracle VM Manager, then
you will need to contact your Oracle DBA and ask them to back up the database for you.

10

White Paper Change the Server Pool File System & Repositories (SAN)

Step 2.2: Stop Oracle VM guests running in the pool

The screen shot in Figure 8, item 2 below shows that all Oracle VM guests running on the target server
pool must be stopped and migrated to the Unassigned Virtual Machines folder in the navigation pane.

Figure 8: Stop all Oracle VM guests running on the target server pool

Stopping the Oracle VM guests is accomplished in the Servers and VMs tab (see Figure 8, item 1).
Notice that Figure 8, items 3 & 4 show our fictional virtual machines stopped and are no longer
assigned to any servers (our example uses Oracle VM guests named myguest51 through myguest56).
Note

You do not need to stop any Oracle VM guests belonging to other server pools being managed by the same
Oracle VM Manager

Step 2.3: Release ownership of current repositories

You must now release the ownership of the storage repositories once all of the Oracle VM guests have
been stopped. It is very important that you release ownership of the storage repositories before you
complete the final refresh of data between the source and target storage platforms. The new server
pool will not be able to use the storage repositories if you do not release ownership before the final
refresh of data between the two storage platforms.
Stop!

Do not skip this step if SAN storage repositories are part of your migration project. Ownership must be
released BEFORE you begin the final refresh of repositories for later steps to succeed. You may not be able
to complete the migration to new storage if you skip this step.

11

White Paper Change the Server Pool File System & Repositories (SAN)

The following three screen shots explain how to release ownership of the storage repositories. This
must be completed for all storage repositories that are part of the target server pool. Change to the
Repositories tab as shown in Figure 9 item 1 below, select the first storage repository as shown in
item2 and then choose the Edit Repository icon shown in item 3.

3
2

Figure 9: Showing ownership of storage repositories

Make sure you check the Release Ownership checkbox as shown in Figure 10 below and then choose
OK to complete.

Figure 10: Releasing ownership of storage repositories

Ensure the ownership has been released for all storage repositories before moving on to the next step.
If the operation was successful, then you will need to select the Show All Repositories radio button to

12

White Paper Change the Server Pool File System & Repositories (SAN)

reveal the un-owned repositories (see Figure 11, item 1). Select each repository (Figure 11, item 2) and
ensure that Ownership: indicates Unowned (Figure 11, item 3).

1
3

Figure 11: Showing storage repository is no longer owned by the Oracle VM Manager

Step 3: Destroy Current Server Pool

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

You will delete the server pool after releasing ownership of the storage repositories. This step is
tedious, but not as dramatic as it sounds. Nothing about Oracle VM is really impacted by this step
other than the server pool wont exist for a few minutes while we build the new server pool. You will
not need to reinstall or reconfigure anything on the Oracle VM servers; all of the bonding, networking
and anything else about the Oracle VM servers remain unchanged.
In fact, once the server pool is destroyed you can still remount the old storage repositories and start
your VMs by hand without the Oracle VM servers even being part of a server pool if you ever needed
to recover. It is highly unlikely you would need to do such a thing and the only reason we mention this
is to let you know nothing is irreversible once the pool is gone you still have all the parts and pieces
to rebuild the same server pool and your virtual machines are quite safe.
Note

Destroying the server pool is not as dramatic as it sounds. Everything is still there for you to use and it is
very easy to recover

13

White Paper Change the Server Pool File System & Repositories (SAN)

Step 3.1: Remove Oracle VM servers from pool

Remove the servers rather than delete the servers. Deleting the servers does not cause a problem it just
adds the unnecessary step having to discover them again. Change to the Servers and VMs tab and
select the target server pool to remove the servers from the pool as shown in Figure 12 below.

Figure 12: Select the target server pool

Move all Oracle VM servers from the Selected Server(s) box on the right to the Available Server(s)
box on the left as shown in Figure 13 below.

Figure 13: Move all servers to the Available Server(s) box

14

White Paper Change the Server Pool File System & Repositories (SAN)

All Oracle VM servers from the target server pool should now reside in the Unassigned Servers
folder in the navigation pane as shown in Figure 14 below.

Figure 14: All Oracle VM servers from target pool should be in Unassigned Servers folder

Step 3.2: Delete the empty server pool

Now delete the target server pool as shown in Figure 15 below. From the Servers and VMs tab,
select the target server pool (item 1) and then select the Delete Server Pool icon from the tool bar in
the navigation pane (item 2).

Figure 15: Delete the target server pool

15

White Paper Change the Server Pool File System & Repositories (SAN)

As shown in Figure 16 below, all of your Oracle VM servers from the target pool should be contained
in the Unassigned Servers folder (item 1) and all remaining unaffected server pools should be the
only pools displayed in either the management pane (item 2) or navigation pane.

Figure 16: Only unaffected server pools should remain

Step 3.3: Perform final synchronization of storage

You will need to perform a final refresh or synchronization of data between the source and target
storage platforms now that the Oracle VM guests have been stopped. All I/O activity should be fully
quiesced at this point so you should have complete data integrity when you perform the final refresh.
You should have already established a replication relationship between the source storage platform and
the target platform as part of step 1. The data you are replicating between the source and target storage
platforms should include the storage repositories and any other LUNs being presented as physical disks
to the Oracle VM guests.
Note

Oracle VM is not used in any way to accomplish cloning or replication of storage repositories. Therefore
storage replication is beyond the scope or purpose of this document.

We assume you know how to

accomplish the cloning or replication of the storage repositories and physical disks that are part of your
migration project.

Step 4: Delete Source Disks

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

Now we will delete the source disks (LUNs) from the Oracle VM management database (model).
Deleting physical disks in this case means deleting the database record for each storage object, not
actually deleting the LUNs that reside on the underlying storage platform. This is a non-destructive
process which means all the data on the LUNs will remain in place and unaffected.

16

White Paper Change the Server Pool File System & Repositories (SAN)

Deleting the physical disks is an important step since it will help reduce confusion between which disks
are the old source disks and which are the new target disks. You can always rediscover the deleted
disks if you need them again to back out of this solution.

Step 4.1: Un-map source LUNs

You will have your storage admin un-map or un-present the old source LUNs from the Oracle VM
servers so they can no longer access or see them from the source storage platform/array. Oracle VM
is not used or involved at all for this step; it is all accomplished on the storage array.
You will not be able to properly delete the source physical disks from the model until the Oracle VM
servers can no longer access the old source LUNs.

Step 4.2: Delete the physical disks from DB

As shown in Figure 17 below, change to the Storage tab (item 1) and then select whatever SAN server
you are using for your source SAN LUNs (item 2). The screen shot shows an iSCSI SAN server
selected; you should select Unmanaged FibreChannel Storage Array if you are using Fibre Channel
instead.
Ensure you are looking at the physical disks by selecting that from the Perspective (item 3). Notice
that the old source disks show an Event Severity of Warning (item 4). This means the old source
disks are no longer accessible by the Oracle VM servers this is a desired condition since it means we
are able to delete the physical disk records from the database.
The reason that Oracle VM does not automatically remove the disks from the database is simply
because it has no idea if this is a temporary condition due to the temporary loss of access, a human
error or if the disks are no longer being used. So, the record of each disk will remain in the database
until a human confirms that these particular physical disks are no longer being used and it is okay to
remove the disks.

17

White Paper Change the Server Pool File System & Repositories (SAN)

To delete the disk, simply select all of the disks (item 4) and then choose the Delete Disks icon from
the tool bar above the management pane (item 5).

Figure 17: Cleaning up the Oracle VM database to reduce confusion

You should no longer see any of the source physical disks once you have deleted them from the model
as seen in Figure 18 below.
Our screen shots assume these are the only physical disks being served from the legacy storage
platform, so you may still see other disks on the legacy storage platform if you are using them for other
server pools that are not part of your migration project.

Figure 18: All of the source physical disks have been deleted

Step 5: Discover Target Disks

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

18

White Paper Change the Server Pool File System & Repositories (SAN)

We deleted the source physical disks from the model simply to reduce confusion over which disks are
which in later steps. We can now discover replicated physical disks from the new target storage
platform.

Step 5.1: Map the target LUNs

You will have your storage admin map or present the replicated target LUNs to the Oracle VM servers
so they can access or see them from the new target storage platform/array. Oracle VM is not used or
involved at all in this step; it is all accomplished on the storage array.
You will not be able to discover the physical until the Oracle VM servers can access the new target
LUNs.

Step 5.2: Discover the new storage platform

Skip this step if:
You are using Fibre Channel
You are using iSCSI but the new SAN server already exists in your model
This paper assumes target SAN server containing the replicated LUNs has already been discovered
prior to this project. If you have not discovered the new iSCSI SAN server yet, then please run
through the SAN server discovery process. This process is documented in the latest Oracle VM user
guide for your particular version of Oracle VM.
Oracle VM is not really discovering the storage array, it is simply probing the SCSI bus on the Oracle
VM servers designated as admin servers for the SAN server and then cataloguing the LUNs it finds
into the Oracle VM management database. The discovery process really just adds LUNs to the Oracle
VM management database for the first time.

Step 5.3: Refresh the Oracle VM database

Skip this step if you performed the previous step of discovering a new iSCSI SAN server.
You will need to refresh the target SAN server containing the replicated LUNs before you attempt to
use them. You should perform this process even if you can see the new replicated LUNs in the Oracle
VM Manager. This will ensure that no new LUNs are overlooked and that the Oracle VM
management database contains all of the latest replicated LUNs.
Step 5.3.1: If you are using Fibre Channel

If you are using Fibre Channel to access your LUNs, then you should simply refresh the Unmanaged
FibreChannel Storage Array to have the Oracle VM servers reprobe the SCSI bus for new LUNs.
This process is documented in the latest Oracle VM user guide for your particular version of Oracle
VM.

19

White Paper Change the Server Pool File System & Repositories (SAN)

Ensure you are viewing the Storage tab as shown in Figure 19 below (item 1). Select the Unmanaged
FibreChannel Storage Array in the navigation pane (item 2), then choose the Refresh icon from the
tool bar above the navigation pane (item 3).

2
Figure 19: Refreshing Fibre Channel

You should now see all the replicated physical disks being served from the new storage platform as
shown in Figure 20 below. The LUNs have been catalogued as physical disks in the Oracle VM
management database at this point.

Figure 20: New target disks have now been added to the Oracle VM management database

Note

All Oracle VM servers accessing Fibre Channel disks must be designated as admin servers for the
Unmanaged FibreChannel Storage Array in order to reprobe the SCSI bus on the Oracle VM servers without
a reboot.

Step 5.3.2: If you are using iSCSI

Ensure you are viewing the Storage tab as shown in Figure 21 below (item 1). Select the iSCSI SAN
server in the navigation pane (item 2), then choose the Refresh icon from the tool bar above the
navigation pane (item 3).

20

White Paper Change the Server Pool File System & Repositories (SAN)

Figure 21: Refreshing iSCSI

You should now see all the replicated physical disks being served from the new storage platform as
shown in Figure 22 below. The LUNs have been catalogued as physical disks in the Oracle VM
management database at this point.

Figure 22: New target disks have now been added to the Oracle VM management database

Step 6: Rebuild the Server Pool

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

We can now rebuild the same server pool using the new PoolFS being presented from the target
storage platform.

21

White Paper Change the Server Pool File System & Repositories (SAN)

Go to the Servers and VMs tab as shown in Figure 23 below (item 1) and then choose the Create
Server Pool icon from the tool bar above the navigation pane (item 2).

Figure 23: Create the same server pool

Looking at Figure 24 below, give the new server pool the same name as the old server pool you
destroyed earlier (item 1), enter the same VIP from the destroyed server pool (item 2). Assign a
PoolFS by selecting the Physical Disk radio button (item 3) for Storage Location and then choose
the Search icon to select the new PoolFS (item 4).

Figure 24: Enter the server pool details

22

White Paper Change the Server Pool File System & Repositories (SAN)

Make sure you chose the correct SAN server from the Select Physical Disk dialog shown in Figure
25 below. Here we show our fictional iSCSI server; you should choose whatever is appropriate for
your environment.

Figure 25: Select the correct SAN server

You should now see the Storage Location field populated with your choice for PoolFS as shown in
Figure 26 below.

Figure 26: The new pool file system is selected

23

White Paper Change the Server Pool File System & Repositories (SAN)

Finally, add all the Oracle VM servers you removed from the old server pool as shown in Figure 27
below.

Figure 27: Add all of Oracle VM servers

Step 7: Take Ownership of Target Repositories

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

You have recreated the same server pool at this point using the PoolFS residing on the new storage
platform. There are no additional steps needed at this point other than incorporating the new
replicated storage repositories into the server pool and starting your Oracle VM guests again.
This should be a very easy step as long as you followed the instructions for releasing ownership of the
source repositories before the final replication refresh/synchronization.

Step 7.1: Refresh the target disks

We need to refresh the individual disks representing the replicated storage repositories before we can
use them as repositories. This step is slightly different from when we refreshed the SAN server; the
SAN server refresh only updates the Oracle VM management database with physical disks that are new
or missing. Refreshing the individual disks forces the Oracle VM Manager to examine the physical
disks for file systems. If the Oracle VM Manager discovers a know file system type on the physical
disk, it adds additional attributes to the database record for the physical disk indicating if the file system
contains a PoolFS or storage repository.

24

White Paper Change the Server Pool File System & Repositories (SAN)

We only need to select the physical disk we know to be storage repositories. Ensure you change to the
Storage tab (item 1) as shown in Figure 28 below. Select the appropriate iSCSI or Fibre SAN server
for your environment (item 2), select all the disks you know to contain replicated repositories (item 3)
and then choose the Refresh Disks icon from the management pane tool bar (item 4).

4
2
3
Figure 28: Refresh the replicated physical disks to update the Oracle VM management database

Step 7.2: Refresh the target repositories

The Oracle VM Manager will now recognize the physical disks containing storage repositories. Go to
the Repositories tab (item 1) as shown in Figure 29 below. The repositories are Unowned at this
point so you wont be able see them until you change the context of view. This is expected since you
released the ownership of the old source repositories before the final replication
refresh/synchronization before destroying the old server pool which leaves them un-owned at this
point.
Using Figure 29 below as a guide you will need to change the context by selecting the Show All
Repositories radio button (item 2). Select the top level folder for Repositories in the navigation pane
(item 3) and notice that the repositories show up with their old names and that they are Unowned
(item 4).

2
3
4
Figure 29: View the newly discovered storage repositories

25

White Paper Change the Server Pool File System & Repositories (SAN)

Select each replicated repository in the navigation pane and then choose the Edit Repository icon
(item 1) as shown in Figure 30 below. Use the first sub tab of the wizard to assign each repository to
the correct Server Pool (item 2) and then Take Ownership (item 3).

Figure 30: Take ownership of the repositories

Use the Present Repositories sub tab in the Edit Repository wizard as shown in Figure 31 below to
present each repository to the newly reconstituted server pool.

Figure 31: Present each repository to the newly rebuilt server pool

Notice that the replicated repositories are now Owned by You (item 1) and presented to all Oracle
VM servers in Figure 32 below. We can now rediscover the Oracle VM guests residing in the

26

White Paper Change the Server Pool File System & Repositories (SAN)

replicated repositories by selecting the two repositories in the management pane (item 1) and then
choosing Refresh Repositories icon from the tool bar above the management pane (item 2).

1
Figure 32: All repositories are now owned by the Oracle VM Manager again

Step 8: Start Oracle VM Guests

Migrating a Server Pool to a New Storage Platform
Step 1:
Preparation

Step 2:
Release
ownership of
repos

Step 3:
Destroy current
pool

Step 4:
Delete source
disks

Step 5:
Discover target
disks

Step 6:
Rebuild server
pool

Step 7:
Take ownership
of repos

Step 8:
Start Oracle VM
guests

The final step is simply to assign the Oracle VM guests to servers and start them.
To start the Oracle VM guests, you will need to select the Unassigned Virtual Machines folder in
the navigation pane as shown in Figure 33 below (item 1). Select one or more Oracle VM guests in the
management pane (item 2) and then choose the Migrate VMs icon from the tool bar above the
management pane (item 3). After that you can start

Figure 33: Migrate Oracle VM guests to Oracle VM servers

27

White Paper Change the Server Pool File System & Repositories (SAN)

You can drag and drop multiple Oracle VM guests to Oracle VM servers and start them in mass, or
you can migrate and start them one at a time. The point is that you should now be able to run the
same Oracle VM guests from the new replicated repositories.

Figure 34: The final step is to start the Oracle VM guests

28

Oracle VM 3:

Copyright 2014, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the

Change the Server Pool File System &

contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other

Repositories (SAN)

warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or

August 2014

fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are

Document: SN05220 rev. 5

formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any

Author: Gregory King

means, electronic or mechanical, for any purpose, without our prior written permission.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Oracle Corporation

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and

World Headquarters

are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are

500 Oracle Parkway

trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0612

Redwood Shores, CA 94065

U.S.A.
Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200
oracle.com