Xyratex StorView Path Manager for Linux User Guide

Part No. 0084354-06A Issue 1.5 November 7, 2008

Xyratex StorView Path Manager for Linux User Guide

Notices
The information in this document is subject to change without notice. While every effort has been made to ensure that all information in this document is accurate, the Authors accept no liability for any errors that may arise. No part of this document may be transmitted or copied in any form, or by any means, for any purpose, without the written permission of the Authors. Issue 1.5 November 7, 2008 Part No. 0084354-06A

Acknowledgments Linux is a trademark of Linus Torvalds. Red Hat is a trademark of Red Hat, Inc. SUSE is a trademark of Novell, Inc. All names, brands, products or services are trademarks or registered trademarks of their respective companies.

ii

Contents

Contents
Preface ................................................................................................................................................... v Revision History ...................................................................................................................................... v 1 Introduction ..................................................................................................................................... 1.1 Introduction .................................................................................................................................. 1.2 System Requirements ................................................................................................................. 1.2.1 Hardware Requirements .................................................................................................... 1.2.2 Supported Operating Systems ........................................................................................... 1.2.3 Supported Architectures ..................................................................................................... 1.2.4 Software ............................................................................................................................. 1 1 1 1 2 2 2

Installation ....................................................................................................................................... 3 2.1 Introduction .................................................................................................................................. 3 2.2 Red Hat Enterprise Linux 4 ......................................................................................................... 3 2.2.1 Check the operating system level ...................................................................................... 3 2.2.2 Install device mapper and device-mapper-multipath .......................................................... 3 2.2.3 Install StorView Path Manager for Linux ............................................................................ 4 2.3 Red Hat Enterprise Linux 5 ......................................................................................................... 5 2.3.1 Check the operating system level ...................................................................................... 5 2.3.2 Install device mapper and device-mapper-multipath .......................................................... 5 2.3.3 Install StorView Path Manager for Linux ............................................................................ 6 2.4 SUSE Linux Enterprise Server 9 ................................................................................................. 7 2.4.1 Checking the operating system level .................................................................................. 7 2.4.2 Install device-mapper and multipath-tools .......................................................................... 7 2.4.3 Install StorView Path Manager for Linux ............................................................................ 7 2.5 SUSE Linux Enterprise Server 10 ............................................................................................... 9 2.5.1 Checking the operating system level .................................................................................. 9 2.5.2 Install device-mapper and multipath-tools .......................................................................... 9 2.5.3 Install StorView Path Manager for Linux ............................................................................ 9 2.6 Multipathing on the Boot Device ................................................................................................ 10 2.6.1 How to Install RHEL 5.1 with Boot Disk Multipathing ....................................................... 11 2.7 Configuring HBAs for multipath use .......................................................................................... 13 2.8 Verifying the Installation ............................................................................................................ 13 2.9 Uninstallation ............................................................................................................................. 13 Overview ........................................................................................................................................ 3.1 Paths ......................................................................................................................................... 3.2 Software Capabilities ................................................................................................................. 3.2.1 svpm Man Page ............................................................................................................... 3.2.2 svpm-dm Man Page ......................................................................................................... 3.2.3 mpath_prio-svpm Man Page ............................................................................................ How To ....................................................................................................................................... 4.1 How to Get Information on Multipath Devices ........................................................................... 4.2 How to Determine Which Path is in Use ................................................................................... 4.3 How to Interpret SVPM Path Priorities ...................................................................................... 15 15 15 16 16 16 17 17 18 19

iii

Xyratex StorView Path Manager for Linux

4.4 How to Set Path Priorities ......................................................................................................... 19 4.5 How to Blacklist local-attached SCSI devices ........................................................................... 20 4.6 How to Automount a Multipath Device ...................................................................................... 20 5 6 Known Issues / Limitations ......................................................................................................... 23 Troubleshooting ........................................................................................................................... 6.1 Installation fails .......................................................................................................................... 6.2 Failback doesn't occur .............................................................................................................. 6.2.1 The multipathd daemon has failed ................................................................................... 6.2.2 The device is not mounted properly ................................................................................. 6.2.3 All required services have not been started ..................................................................... 6.2.4 Multipath.conf entries are not valid .................................................................................. 6.2.5 No LUNs are available ..................................................................................................... 6.2.6 The system needs to be rebooted ................................................................................... 6.3 Other problems ......................................................................................................................... 25 25 25 25 25 26 26 26 26 26

Glossary ............................................................................................................................................... 27

iv

Preface

Preface
What is in this guide This user guide gives you an overview of how to use the Xyratex StorView Path Manager for Linux to configure multipathing in a RAID system. Who should use this guide This user guide is for those needing to set up a multipath solution in the Linux environment. About this guide This user guide provides the following information: Chapter 1, Introduction, on page 1 provides an introduction and system requirements. Chapter 2, Installation, on page 3 gives details on how to set up the software, how configure the system for multipath use, and how to uninstall. Chapter 3, Overview, on page 15 describes multipathing and outlines the softwares capabilities. Chapter 4, How To , on page 17 provides step-by-step instructions for a number of common tasks. Chapter 5, Known Issues / Limitations, on page 23 gives warning of any limitations in the software. Chapter 6, Troubleshooting, on page 25 provides resolutions for some common problems. The Glossary, on page 27, defines some of the terms used throughout this user guide.

Revision History
Version 1.0 1.1 1.2 1.3 1.4 1.5 Date June 14, 2007 Description of Change Initial Release.

September 19, 2007 Support for EM64T architetures. March 12, 2008 July 9, 2008 August 22, 2008 November 7, 2008 Support for the RS-1220-F4-6412E storage system (F6412E). Support for RHEL5 and IA64; multipath booting on RHEL5.1; update to output from svpm -l command. Minor updates. Support for F6500E controllers.

Xyratex StorView Path Manager for Linux

vi

Introduction

Chapter 1

Introduction
1.1 Introduction
StorView Path Manager for Linux (svpm) works in conjunction with the Linux device-mapper and multipath tools to provide multipath capability for the F5402E, E5402E, F5404E, F5412E, E5412E, F6412E and F6500E controllers. Multipathing is the use of multiple physical components (such as host bus adapters, cables and switches) to provide two or more data paths to a storage device and hence avoid a single point of failure. When a component in one path fails, a different path to the device will still be available for data transfer. Multiple paths can be used simultaneously if the storage device supports it. StorView Path Manager for Linux provides failover/failback, path priorities and load balancing features.

1.2 System Requirements

Use of StorView Path Manager for Linux requires the following:

1.2.1 Hardware Requirements

The following enclosures and RAID Controllers are supported: RS-1220-F4-5402E (F5402E) RS-1220-E3-5402E (E5402E) RS-4835-F4-5404E (F5404E) RS-1220-F4-5412E (F5412E) RS-1220-E3-5412E (E5412E)

Xyratex StorView Path Manager for Linux

RS-1220-F4-6412E (F6412E) HB-2435-F8-6500E (F6500E) HB-2435-E311-XPN (F6500E) A storage configuration where there are at least two paths from the host to the storage.

1.2.2 Supported Operating Systems

Red Hat Enterprise Linux (RHEL) ES/AS 4 Update 4. Red Hat Enterprise Linux (RHEL) ES/AS 5.1. SUSE Linux Enterprise Server (SLES) 9 Service Pack 4. SUSE Linux Enterprise Server (SLES) 10 Service Pack 1.

1.2.3 Supported Architectures

32-bit x86. 64-bit EM64T (not on SLES 9). 64-bit Itanium IA64 (not on SLES 9).

1.2.4 Software
Device Mapper (included with RHEL 4, RHEL 5, SLES 9, SLES 10). Multipath tools (included with RHEL 4, RHEL 5, SLES 9, SLES 10). For more information, see Installation beginning on page 3.

Installation

Chapter 2

Installation
2.1 Introduction
The pre-requisites for installation are included in the installation instructions that follow.

2.2 Red Hat Enterprise Linux 4

2.2.1 Check the operating system level
1 Type the following:
cat /etc/redhat-release

The output should look similar to this:

Red Hat Enterprise Linux ES release 4 (Nahant Update 4)

You must be running the Enterprise version of Red Hat Linux, version ES or AS, release 4, with update 4 or greater.

2.2.2 Install device mapper and device-mapper-multipath

1 Type the following commands to determine whether the device mapper and multipath tools are already present on the system:
rpm -q device-mapper rpm -q device-mapper-multipath

The output should be as follows:

device-mapper-1.02.07-4.0.RHEL4 device-mapper-multipath-0.4.5-16.1.RHEL4

Note

The only supported versions of device-mapper and device-mapper-multipath are those that shipped with the operating system distribution.

Xyratex StorView Path Manager for Linux

If the commands return nothing, the components are not installed. The RPMs can be found on the installation media for your operating system: [CD2]/RedHat/RPMS/device-mapper-1.02.07-4.0.RHEL4.i386.rpm [CD4]/RedHat/RPMS/device-mapper-multipath-0.4.5-16.1.RHEL4.i386.rpm where [CD2] and [CD4] are the mount directories of the second and fourth installation CDs.

Note 4

The location of the RPMs are current at the time of the printing of this document. Change to the directory containing the RPMs and install them:
cd [CD2]/RedHat/RPMS rpm -ivh device-mapper-1.02.07-4.0.RHEL4.i386.rpm cd [CD4]/RedHat/RPMS rpm -ivh device-mapper-multipath-0.4.5-16.1.RHEL4.i386.rpm

2.2.3 Install StorView Path Manager for Linux

1 Type the following command to determine whether svpm is already installed on the system:
rpm -q svpm-dm

If a version of svpm not is installed on the system you should get a message: package svpm-dm is not installed. If a version of svpm is installed on the system you should get a message: svpm-dm-<version>.rhel4.xyr and you must uninstall the svpm rpm before continuing (see 2.9, Uninstallation, on page 13). 2 Type the following commands:
cd [CD path]/drivers/svpm/linux rpm -ihv svpm-dm-<version>.rhel4.xyr.i386.rpm

Determine if the rpm installation was successful by typing the following command:
rpm -q svpm-dm

If the installation was successful the command should return svpm-dm-<version>.rhel4.xyr. If the command returns nothing, the component was not installed. 4 Important Using a text editor, merge the contents from /etc/multipath.conf.svpm (the section devices {}) into /etc/multipath.conf. If /etc/multipath.conf is empty, also merge the section defaults {}. If the step above is not performed, svpm-dm will not operate correctly and results will be unpredictable. You will need to make sure the multipath and svpm daemons are started when the system boots:
/sbin/chkconfig multipathd on

It is now necessary to restart the multipath daemon:

/etc/init.d/multipathd restart

If nothing is returned, the daemons have started successfully. 7 Follow the instructions in 2.7, Configuring HBAs for multipath use, on page 13.

Installation

2.3 Red Hat Enterprise Linux 5

Note If you intend to use multipathing on the boot device, read section 2.6, Multipathing on the Boot Device, on page 10.

2.3.1 Check the operating system level

1 Type the following:
cat /etc/redhat-release

The output should look similar to this:

Red Hat Enterprise Linux Server release 5.1 (Tikanga)

You must be running the Enterprise version of Red Hat Linux, version ES or AS, release 5 with update 1.

2.3.2 Install device mapper and device-mapper-multipath

1 Type the following commands to determine whether the device mapper and multipath tools are already present on the system:
rpm -q device-mapper rpm -q device-mapper-multipath

The output should be as follows:

device-mapper-1.02.20-1.el5 device-mapper-multipath-0.4.7-12.el5

Note

The only supported versions of device-mapper and device-mapper-multipath are those that shipped with the operating system distribution. If the commands return nothing, the components are not installed. The RPMs can be found on the installation media for your operating system: [CD1]/Server/device-mapper-1.02.20-1.el5.i386.rpm [CD3]/Server/device-mapper-multipath-0.4.7-12.el5.i386.rpm where [CD1] and [CD3] are the mount directories of the second and fourth installation CDs. Or [DVD]/Server/device-mapper-1.02.20-1.el5.i386.rpm [DVD]/Server/device-mapper-multipath-0.4.7-12.el5.i386.rpm where [DVD] is the mount directory of the installation DVD.

Note

The location of the RPMs are current at the time of the printing of this document.

Xyratex StorView Path Manager for Linux

Change to the directory containing the RPMs and install them:

cd [CD1]/Server rpm -ivh device-mapper-1.02.20-1.el5.i386.rpm cd [CD3]/Server rpm -ivh device-mapper-multipath-0.4.7-12.el5.i386.rpm

Or
cd [DVD]/Server rpm -ivh device-mapper-1.02.20-1.el5.i386.rpm rpm -ivh device-mapper-multipath-0.4.7-12.el5.i386.rpm

2.3.3 Install StorView Path Manager for Linux

1 Type the following command to determine whether svpm is already installed on the system:
rpm -q svpm-dm

If a version of svpm not is installed on the system you should get a message: package svpm-dm is not installed. If a version of svpm is installed on the system you should get a message: svpm-dm-<version>.rhel5.xyr and you must uninstall the svpm rpm before continuing (see 2.9, Uninstallation, on page 13). 2 Type the following commands:
cd [CD path]/drivers/svpm/linux rpm -ihv svpm-dm-<version>.rhel5.xyr.i386.rpm

Determine if the rpm installation was successful by typing the following command:
rpm -q svpm-dm

If the installation was successful the command should return svpm-dm-<version>.rhel5.xyr. If the command returns nothing, the component was not installed. 4 Important Using a text editor, merge the contents from /etc/multipath.conf.svpm (the section devices {}) into /etc/multipath.conf. If /etc/multipath.conf is empty, also merge the section defaults {}. If the step above is not performed, svpm-dm will not operate correctly and results will be unpredictable. You will need to make sure the multipath and svpm daemons are started when the system boots:
/sbin/chkconfig multipathd on

It is now necessary to restart the multipath daemon:

/etc/init.d/multipathd restart

If nothing is returned, the daemons have started successfully. 7 Follow the instructions in 2.7, Configuring HBAs for multipath use, on page 13.

Installation

2.4 SUSE Linux Enterprise Server 9

2.4.1 Checking the operating system level
1 Type the following:
cat /etc/SuSE-release

The output should look similar to this:

SUSE LINUX Enterprise Server 9 (i586) VERSION = 9 PATCHLEVEL = 4

You must be running the Enterprise version of SUSE Linux, version 9, with patch level 4.

2.4.2 Install device-mapper and multipath-tools

1 Type the following commands to determine whether the device mapper and multipath tools are already present on the system:
rpm -q device-mapper rpm -q multipath-tools

The output should be as follows:

device-mapper-1.01.01-1.6 multipath-tools-0.4.7-75.7

Note

The only supported versions of device-mapper and device-mapper-multipath are those that shipped with the operating system distribution. If the commands return nothing, the components are not installed. The RPMs can be found on the installation media for your operating system. [CD1]/suse/i586/device-mapper-1.01.01-1.6.i586.rpm [CD1]/suse/i586/device-mapper-multipath-0.4.7-75.7.i586.rpm where [CD1] is the mount directories of the first installation CDs.

Note 4

The location of the RPMs are current at the time of the printing of this document. Change to the directory containing the RPMs and install them:
cd [CD1]/suse/i586 rpm -ivh device-mapper-1.01.01-1.6.i586.rpm cd [CD1]/suse/i586 rpm -ivh multipath-tools-0.4.7-75.7.i586.rpm

2.4.3 Install StorView Path Manager for Linux

1 Type the following command to determine whether svpm is already installed on the system:
rpm -q svpm-dm

Xyratex StorView Path Manager for Linux

If a version of svpm not is installed on the system you should get a message: package svpm-dm is not installed. If a version of svpm is installed on the system you should get a message: svpm-dm-<version>.sles9.xyr and you must uninstall the svpm rpm before continuing (see 2.9, Uninstallation, on page 13). 2 Type the following commands:
cd [CD path]/drivers/svpm/linux rpm -ihv svpm-dm-<version>.sles9.xyr.i586.rpm

Determine if the rpm installation was successful by typing the following command:
rpm -q svpm-dm

If the installation was successful the command should return svpm-dm-<version>.sles9.xyr. If the command returns nothing, the component was not installed. 4 Important Using a text editor, merge the contents from /etc/multipath.conf.svpm (the section devices {}) into /etc/multipath.conf. If /etc/multipath.conf is empty, also merge the section defaults {}. If the step above is not performed, svpm-dm will not operate correctly and results will be unpredictable. You will need to make sure the multipath and svpm daemons are started when the system boots:
/sbin/chkconfig /sbin/multipathd on

It is now necessary to restart the multipath daemon:

/etc/init.d/multipathd restart

If nothing is returned, the daemons have started successfully. 7 Use YaST - SYSTEMS / Runlevel Editor / expert mode to enable the following modules: boot.multipath level B boot.device-mapper level B boot.udev level B multipathd level 3 and 5 8 9 Follow the instructions in 2.7, Configuring HBAs for multipath use, on page 13. The system must be rebooted before newly installed devices and LVM entries will be recognized.

Installation

2.5 SUSE Linux Enterprise Server 10

2.5.1 Checking the operating system level
1 Type the following:
cat /etc/SuSE-release

The output should look similar to this:

SUSE LINUX Enterprise Server 10 (i586) VERSION = 10 PATCHLEVEL = 1

You must be running the Enterprise version of SUSE Linux, version 10, with patch level 1.

2.5.2 Install device-mapper and multipath-tools

1 Type the following commands to determine whether the device mapper and multipath tools are already present on the system:
rpm -q device-mapper rpm -q multipath-tools

The output should be as follows:

device-mapper-1.02.13-6.9 multipath-tools-0.4.7-34.18

Note

The only supported versions of device-mapper and device-mapper-multipath are those that shipped with the operating system distribution. If the commands return nothing, the components are not installed. The RPMs can be found on the installation media for your operating system. [CD1]/suse/i586/device-mapper-1.02.13-6.9.i586.rpm [CD1]/suse/i586/multipath-tools-0.4.7-34.18.i586.rpm where [CD1] is the mount directories of the first installation CDs.

Note 4

The location of the RPMs are current at the time of the printing of this document. Change to the directory containing the RPMs and install them:
cd [CD1]/suse/i586 rpm -ivh device-mapper-1.02.13-6.9.i586.rpm cd [CD1]/suse/i586 rpm -ivh multipath-tools-0.4.7-34.18.i586.rpm

2.5.3 Install StorView Path Manager for Linux

1 Type the following command to determine whether svpm is already installed on the system:
rpm -q svpm-dm

Xyratex StorView Path Manager for Linux

If a version of svpm not is installed on the system you should get a message: package svpm-dm is not installed. If a version of svpm is installed on the system you should get a message: svpm-dm<version>.sles10.xyr and you must uninstall the svpm rpm before continuing (see 2.9, Uninstallation, on page 13). 2 Type the following commands:
cd [CD path]/drivers/svpm/linux rpm -ihv svpm-dm-<version>.sles10.xyr.i586.rpm

Determine if the rpm installation was successful by typing the following command:
rpm -q svpm-dm

If the installation was successful the command should return svpm-dm-<version>.sles10.xyr. If the command returns nothing, the component was not installed. 4 Important Using a text editor, merge the contents from /etc/multipath.conf.svpm (the section devices {}) into / etc/multipath.conf. If /etc/multipath.conf is empty, also merge the section defaults {}. If the step above is not performed, svpm-dm will not operate correctly and results will be unpredictable. You will need to make sure the multipath and svpm daemons are started when the system boots:
/sbin/chkconfig /sbin/multipathd on

It is now necessary to restart the multipath daemon:

/etc/init.d/multipathd restart

If nothing is returned, the daemons have started successfully. 7 Use YaST - SYSTEMS / system services / expert mode to enable the following modules: boot.multipath level B boot.device-mapper level B boot.udev level B multipathd level 3 and 5 8 9 Follow the instructions in 2.7, Configuring HBAs for multipath use, on page 13. The system must be rebooted before newly installed devices and LVM entries will be recognized.

2.6 Multipathing on the Boot Device

Red Hat has added support for multipathing on the boot device to Red Hat Enterprise Linux 5.1. This allows you the protection of multipathing on your operating system partition. Note This is only supported with Red Hat Enterprise Linux 5.1 and later, not on SuSE Linux, nor on any version of RHEL prior to 5.1.

10

Installation

2.6.1 How to Install RHEL 5.1 with Boot Disk Multipathing

The following steps explain how to install RHEL 5.1 onto your Xyratex storage, such that multipathing is used on the boot device. Note In order to have multipathing on the boot device, the system must be configured at the time the operating system is installed.

2.6.1.1
1 2 3

Pre-Installation Setup
Perform the following steps before installing RHEL 5.1: Using StorView, configure the LUN you wish to use for your operating system partition. Using StorView SAN Mapping or Port Mapping, ensure that the LUN has 2 or more paths from the host. It is recommended that only one LUN is visible to the host before performing the install. This will simplify your install process. Additionally, any LUNs that can be seen during install will not have their aliases reflected by svpm-dm. To make the non-OS LUNs invisible to the host, use StorView SAN Mapping or Port Mapping. Configure your HBA to allow booting from the SAN.

2.6.1.2
1 2

Configuring your HBA to allow booting from the SAN

For QLogic HBAs perform the following steps: Press CTRL+Q to access the BIOS setup utility (Fast!UTIL) during system boot-up. From the Fast!UTIL Options menu, enable the host adapter BIOS by selecting Configuration Settings > Adapter Settings -> Host Adapter BIOS. From the Configuration Settings menu, select Selectable Boot Settings -> Selectable Boot. From the (Primary) Boot Port Name, LUN field, press <Enter> to select the primary boot device. Press <Esc> to exit Fast!UTIL. Select Save changes. Reboot the system. For all other HBAs, consult your HBA documentation.

3 4 5 6 7

2.6.1.3
1 2 3 4

Linux Installation
Insert your operating system CD or DVD and boot from the disc. From the boot: prompt, enter linux mpath. Follow the prompts given by the Red Hat installer to choose your language and keyboard setup. Choose the option to Install Red Hat Enterprise Linux Server.

11

Xyratex StorView Path Manager for Linux

You will then see a list of available partitions. Make sure your install partition is listed as mapper/mpath#. If the partition does not have a mapper/mpath listing, make sure the LUN is visible by more than one path, and restart the process. If you are installing to mapper/mpath0, choose the correct partition and click Next. If more than one LUN is visible to the host and/or you are not installing to mapper/mpath0, you must perform the following additional steps or the system will not be bootable: 1 Note which path is being used for the installation. For example: mapper/mpath1. 2 Check Review and modify partitioning layout and click Next. 3 On the screen that shows the drive layout, click Next. 4 The top line on the screen will indicate where the GRUB boot loader will be installed. By default, this will be /dev/mapper/mpath0. You will need to change this to install the boot loader onto the drive being used for the installation, as follows: 5 Check Configure advanced boot loader options and click Next. 6 Click Change Drive Order. 7 Arrange the drives so that the one being installed is at the top of the list. For example, if you are installing to mapper/mpath1 move /dev/mapper/mpath1 to the top of the list. Click OK. 8 Confirm that the next screen indicates that the boot loader will be installed to the correct device.

6 Note

7 8 9 10 11

Choose your network settings, timezone, and root password as normal. When prompted to customize the software selection, choose Customize now. Select Base System, then highlight Base and click Optional packages. Make sure that device-mapper-multipath is selected. Continue with the install process and reboot when prompted.

2.6.1.4
1 2

svpm-dm Installation and Setup

Install svpm-dm as described in this user manual. In order to make sure any new LUNs are not blacklisted, edit the /etc/multipath.conf file in one of the following ways: Comment out the wwid * line in the blacklist section. Add the wwid of any LUNs you create to the blacklist_exceptions section.

12

Installation

2.7 Configuring HBAs for multipath use

If you are using a Qlogic or Emulex HBA, you must edit the following files: Redhat Enterprise Linux 4 or 5: SuSE Linux Enterprise Sever 9 or 10: Add the line below for Qlogic HBA:
options qla2xxx ql2xfailover=0 qlport_down_retry=1 ql2xretrycount=5

/etc/modprobe.conf /etc/modprobe.conf.local

Add the line below for Emulex HBA:

options lpfc lpfc_nodev_tmo=14 lpfc_lun_queue_depth=30 lpfc_discovery_threads=1

2.8 Verifying the Installation

If the installation has been successful, you should be able to see each of the RAID LUNs using the following commands:
multipath ll svpm -l

See sections 4.2, How to Determine Which Path is in Use and 4.3, How to Interpret SVPM Path Priorities for the correct output.

2.9 Uninstallation
To uninstall StorView Path Manager for Linux: 1 Using a text editor, the /etc/multipath.conf file should be edited to remove the lines added during installation from /etc/multipath.conf.svpm Type the following:
rpm -e svpm-dm

The /etc/lvm/cache/svpm.conf file may need to be removed:

rm /etc/lvm/cache/svpm.conf

13

Xyratex StorView Path Manager for Linux

14

Overview

Chapter 3

Overview
3.1 Paths
A path is defined as the complete route from host to controller, including the HBA, cabling and possibly a switch. A multipath solution requires that there is more than one of these paths. The number of paths is easy to see if direct connections are made (it is simply the number of physical cables), but less obvious if a switch is used. If, for example, there are 3 HBAs connected to a switch, and the switch is connected to both ports of a controller, this results in a total of 6 paths. When using Device Mapper each path is given a standard linux block device name (/dev/sda, /dev/sdb, and so on), but these are also merged into a single multipath block device in the /dev/mapper directory. The name of the device is the SCSI ID which will be in one of the following forms: /dev/mapper/36000bb500020c5f20000000000000009 /dev/mapper/mpath1 This device behaves much like a standard Linux block device. It can be mounted, or checked using the fsck command. In order to use multipath correctly, you must mount the mapper device (/dev/mapper/ 36000bb500020c5f20000000000000009 or /dev/mapper/mpath1) and not one of the path block devices (/dev/sdb). Note The selection of SCSI ID or mpath device name is controlled by the setting of user-friendly-name in the defaults section of the multipath.conf file.

3.2 Software Capabilities

StorView Path Manager for Linux functions in a load balancing capacity and uses Device Mapper/ multipathd to provide failover/failback support. The svpm command can be used to do the following: Change the priority of the paths in a multipath system (svpm --prio).

15

Xyratex StorView Path Manager for Linux

Allow the system to automatically select the path priorities (svpm -F). Determine the alias of a multipath device (svpm --alias --get). Additional information is available in the man pages once the RPM is installed. Below is a list of svpm man pages. See the man pages and help for additional command explanation and examples.

3.2.1 svpm Man Page

svpm - StorView Path Manager Change to output verbosity level. Flush all current path preference settings and allow the svpm to determine the optimum pathing preference. This will erase any previously defined user pathing preferences. svpm l Display the current svpm configuration. svpm V Display the current svpm version svpm --alias --get path Get the logical drive alias name and world wide name svpm --prio --get path Get the path preference svpm --prio --set path preference Set the path preference svpm help Display svpm help information and command examples svpm [v level] [-FlV] svpm F

3.2.2 svpm-dm Man Page

svpm-dm Provides a brief overview of StorView Path Manager and Device Mapper operations. Also provides required packages, installation instructions and command examples.

3.2.3 mpath_prio-svpm Man Page

mpath_prio_svpm mpath_prio_svpm path Display the multipathd priority callout Display the priority callout used in the configuration file for multipathd. Gets the priority for the selected path.

16

How To ...

Chapter 4

How To
4.1 How to Get Information on Multipath Devices
Type the following command:
multipath -ll

The output will be something like the following for each multipath device:
mpath1 (36000bb500020c5f20000000000000009) dm-5 XYRATEX,F5412E [size=23G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=3][active] \_ 2:0:1:1 sdc 8:32 [active][ready] \_ round-robin 0 [prio=2][enabled] \_ 2:0:0:1 sda 8:0 [active][ready]

In this example, the multipath device with SCSI ID 36000bb500020c5f20000000000000009 and alias mpath1 consists of two path groups containing the Linux devices /dev/sdc and /dev/sda. Linux device / dev/sdc has the higher prio value and therefore has priority. Broken down into parts, the most useful information is as labeled:
Alias (friendly name) SCSI ID mpath1 (36000bb500020c5f20000000000000009) [size=23 G][features=1 queue_if_no_path][hwhandler=0]

Device capacity in GigaBytes

17

Xyratex StorView Path Manager for Linux

Device priority, set using svpm --prio Path Group State \_ round-robin 0 [prio=3][active] \_ 2:0:0:0 sdb 8:32 [active][ready]

Path Condition Path State Linux block device major:minor number Linux device (for example, /dev/sdb) SCSI Port:Bus:Target:LUN

Path Group States: [active] - The path group is receiving I/O requests. [enabled] - The path group will be used if the [active] path group has no paths in the [ready] state. [disabled] - The path group will be used if the [active] path group and all [enabled] path groups have no paths in the [ready] state. Path States: [ready] - The path is ready to deal with I/O requests. [active] - The path is up and is the currently active path. [faulty] - The path is not ready to deal with I/O requests. [failed] - The path is not ready to deal with I/O requests.

4.2 How to Determine Which Path is in Use

Type the following command:
multipath -ll

The output will be something like the following for each multipath device:
mpath1 (36000bb500020c5f20000000000000009) dm-5 XYRATEX,F5412E [size=23G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=3][active] \_ 2:0:1:1 sdc 8:32 [active][ready] \_ round-robin 0 [prio=2][enabled] \_ 2:0:0:1 sda 8:0 [active][ready]

The path that is currently in use is the one marked [active] at every stage of the tree view. In this case, /dev/sdc is being used to route I/O.

18

How To ...

4.3 How to Interpret SVPM Path Priorities

Type the following command:
svpm l

The output will be something like the following for each multipath device:
Alias set when logical drive was created SCSI ID LD_1 (36000bb500020c5f20000000000000009) \_ C0P1 [2:0:0:1] [priority=2] [/dev/sda] \_ C1P0 [2:0:1:1] [priority=3] [/dev/sdc]

Linux block device Current priority setting SCSI Port:Bus:Target:LUN Controller port number Controller number

4.4 How to Set Path Priorities

A path group with a high priority will be used in preference to a path group with a lower priority. The simplest method of selecting path priorities is to allow StorView Path Manager for Linux to choose them automatically using the following command:
svpm -F

Alternatively, you can set a path priority yourself with the following command:
svpm --prio --set <path> <preference>

where: <path> is the linux block device path (for example, /dev/sdb). <preference> is the priority number (5-8) where 8 is the highest priority. StorView Path Manager for Linux automatically assigns the values 1-4, so any value set using this method will override the system's automatic path choice. If you are using dissimilar hardware for different paths, you may want to assign a high priority to: The path with the highest quality components, which may be more reliable. The fastest path. A high performance LUN so that it has exclusive use of a path.

19

Xyratex StorView Path Manager for Linux

4.5 How to Blacklist local-attached SCSI devices

Blacklisting a local device causes multipath-tools to ignore it. This is done by adding come configuration lines to the /etc/multipath.conf file. Below is as example of how to blacklist a device from multipath-tools. Note It is advised to blacklist using the WWID method.
devnode_blacklist { wwid 3CD1011Q00007120M9YT devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|sdb)[0-9]*" devnode "^hd[a-z][0-9]*" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]" }

#Example #Example #Example #Example

wwid defaults defaults defaults

Warning

If you do not verify that you have your host disk blacklisted you are taking a risk in accidentally formatting your host drive containing your operating system.

4.6 How to Automount a Multipath Device

Note This procedure requires knowledge of the Linux boot process and the format of the /etc/fstab file. By adding the mounts to /etc/fstab, filesystem checks and mounting can be automated for the multipath device. Make sure that the multipath daemon (multipathd) starts early in the boot process. Some distributions automate the steps when multipath is enabled: Red Hat 4 will move symbolic links from /etc/init.d/multipathd into the /etc/rc*.d/S87multipathd files for runlevels S, 2, 3 and 5, removing the K87multipathd files. Red Hat 5 has early startup of the multipath, on by default within rc.sysinit when multipath is installed. SLES10 will require the addition of the B (boot), 2, 3 and 5 runlevel in /etc/init.d/boot.multipathd and /etc/init.d/multipathd scripts. If the operating system offers support for mount by UUID or WWN, then it is recommended to utilize those instead of mpath numerical device references. Research the possible mount device references in /dev/ mapper/ (in other words, WWN, UUID or mpath#) and make the appropriate arrangements for the selected filesystem, labels, udev configuration and devices. Once you have selected the device path you will use, check man fstab for details on the correct format and options, which will involve adding a line to /etc/fstab of the following format:
/dev/mapper/mpath206 /mnt/mnt1 ext3 defaults 0 3

It is recommended that the final value (filesystem checking) is set to 2 or higher to ensure the multipath device is checked after the root filesystem(s).

20

How To ...

Check the mounted mpath after reboot. If you have difficulties booting with the specified mount, either use a rescue disk or reboot into single user mode and comment out the selection with a # to permit debugging.

21

Xyratex StorView Path Manager for Linux

22

Known Issues / Limitations

Chapter 5

Known Issues / Limitations

The following issues and limitations are present in the current release of StorView Path Manager for Linux: 1 2 A maximum of 8 paths per device is supported (a limitation of DM). A maximum of 4 paths per controller port is supported. That is, only 4 host ports can be connected to a single controller port. For RHEL 4 or SUSE 9, multipathing cannot be used with the root (/) or boot (/boot) filesystems because the bootloader cannot handle multipath I/O. Linux SCSI Disk Timeout Value It is suggested that the Linux SCSI Disk Timeout value be set to no less than 60 seconds to allow the controller ample time to complete the failover/failback process. In the /sys/class/scsi_devices/ directory is a list of SCSI devices. For each of these, there is a subdirectory called device containing a file called timeout. Change the contents of each timeout file to the value 60. 5 Timeouts when using SATA drives If you experience a significant number of drive timeouts, it is recommended you change the /etc/multipath.conf pathchecker line from readsector0 to tur. Path priorities are stored based on controller port and host SCSI Port. If the SCSI Port number changes for an HBA, you will have to reset your path priorities by issuing svpm -F. For this reason, it is advised not to restart the HBA driver in order to rescan for new paths.

23

Xyratex StorView Path Manager for Linux

24

Troubleshooting

Chapter 6

Troubleshooting
6.1 Installation fails
If a message stating Error: failed dependencies is given, make sure the stated software has been installed and meets the minimum version requirements. For more details, see Installation beginning on page 3.

6.2 Failback doesn't occur

This can happen for a number of reasons:

6.2.1 The multipathd daemon has failed

Type the following:
ps -A | grep multipathd

If multipathd not listed in the output, the daemon has failed. To restart it, type: On Red Hat Enterprise Linux:
/sbin/service multipathd restart

On SUSE Linux Enterprise Server:

/etc/init.d/multipathd restart

6.2.2 The device is not mounted properly

In order for multipath to work correctly, the mapper device must be mounted (for example: /dev/mapper/ 36000bb500020c5f20000000000000009) and not the block device (for example: /dev/sdb). The mapper device (/dev/mapper/36000bb500020c5f20000000000000009) is a virtual device labeled to represent multiple LUN mappings to the same logical drive.

25

Xyratex StorView Path Manager for Linux

6.2.3 All required services have not been started

If /dev/mapper entries are not present, check that all required services have been started. See Chapter 2, Installation, on page 3 for details of which services are required on which Linux distributions.

6.2.4 Multipath.conf entries are not valid

The entries in the /etc/multipath.conf file may not be valid for the detected LUNs. Ensure that the available LUNs are represented in the /etc/multipath.conf file.

6.2.5 No LUNs are available

Determine whether the host system has detected the configured LUNs by typing the following command:
cat /proc/scsi/scsi

The output should look something like this:

Host: scsi2 Channel: 00 Id: 00 Lun:00 Vendor: XYRATEX Model F5402E Rev: Type: Direct-Access ANSI SCSI revision: 04

6.2.6 The system needs to be rebooted

Occasionally the system must be rebooted to recognize the available LUNs.

6.3 Other problems

It may be possible to diagnose other problems by looking for looking for multipath related entries in the Linux error log:
/var/log/messages

26

Glossary

Glossary
DM DM (Device-Mapper) is a block system device introduced in the 2.6 version of the Linux kernel. It handles logical volume management. References to DM in this document concern dm-multipath - the DM subsystem that handles multipath I/O. Failover/Failback Failover occurs when a path becomes unavailable and another path is used to route I/O. Failback occurs when a path with a higher priority than the currently active path is restored. In this case I/O will fail back to the higher priority path once it is available again. This type of multipath behavior can be set using the multipath -p failover command. I/O must be running to allow the Failover/Failback paths to reflect the active state. Load Balancing The process where I/O to a device uses all available paths to that device. If a path is made unavailable (by component failure, for example), I/O is re-directed down the remaining paths. If paths are later added, they will be automatically detected and used. This type of multipath behavior can be set using the multipath -p multibus command. Multipath Tools A user-space application for creating, configuring, manipulating and maintaining DM multipath devices. It consists of the multipath command and the multipathd daemon. Documentation can be found at the following web page: http://christophe.varoqui.free.fr/ Path Group A path group consists of one or more paths and has a single device entry in the /dev/ mapper directory. By default, the name of the path group is the SCSI ID of the device in hex, but this can be changed to something more meaningful by assigning the path group an alias. Path Priority The path with the highest priority value (as reported by the multipath -ll command) will be used in preference to other paths. Upon failure, the path with the next highest priority will be used. If a higher priority path is later restored, I/O will switch back to it. The operating system uses priorities 1-4 automatically, leaving priorities 5-8 for the user to specify.

27

Xyratex StorView Path Manager for Linux

28