86A293ER11

StoreWay Multipath for Linux
User's Guide
REFERENCE 86 A2 93ER 11
STOREWAY
STOREWAY StoreWay Multipath for Linux

User's Guide
Software
June 2011
BULL CEDOC 357 AVENUE PATTON B.P.20845 49008 ANGERS CEDEX 01 FRANCE REFERENCE 86 A2 93ER 11
The following copyright notice protects this book under Copyright laws which prohibit such actions as, but not limited to, copying, distributing, modifying, and making derivative works.
Copyright Bull SAS 2011 Printed in France
Trademarks and Acknowledgements

We acknowledge the rights of the proprietors of the trademarks mentioned in this manual. All brand names and software and hardware product names are subject to trademark and/or patent protection. Quoting of brand and product names is for information purposes only and does not represent trademark misuse.
The information in this document is subject to change without notice. Bull will not be liable for errors contained herein, or for incidental or consequential damages in connection with the use of this material.
Table of Contents
Chapter 1.About this Guide............................................................................................................. 1 Chapter 2.Overview........................................................................................................................ 3 2.1Purpose................................................................................................................................... 3 2.2Supported Disk Subsystems........................................................................................................ 3 2.3Supported Architectures............................................................................................................ 3 2.4Supported HBA........................................................................................................................ 4 2.5Packages................................................................................................................................ 4 2.6CD-ROM Contents.................................................................................................................... 5 Chapter 3.Multipathing Feature........................................................................................................ 7 3.1Installation and Configuration.................................................................................................... 7 3.1.1Automatic Installation..................................................................................................... 7 3.1.2Manual Installation......................................................................................................... 9 3.2Advanced Multipathing Configuration.......................................................................................11 3.2.1Multipathing over ISCSI configuration.............................................................................11 3.3Multipathing Un-installation...................................................................................................... 12 3.3.1Automatic Un-installation............................................................................................... 12 3.3.2Manual Un-installation..................................................................................................12 3.4Multipathing Status Checking: multipath Command..................................................................14 3.5Using Multipath Devices.......................................................................................................... 15 3.6Using LVM2 on top of the Multipath devices...............................................................................17 3.7Partitions on top of Multipath Devices.......................................................................................20 3.8Boot Support......................................................................................................................... 21 3.9Taking into Account Newly Created Devices..............................................................................23 3.10Checking Path at Boot Time................................................................................................... 24 3.10.1Automatic Checks...................................................................................................... 24 3.10.2How to Reinitialize Manually the File Paths....................................................................24 3.10.3How to change multipath policy...................................................................................24 3.11Wiring Subsystems............................................................................................................... 25 Chapter 4.Messages..................................................................................................................... 29 4.1FC Multipathing errlog............................................................................................................ 29 4.2Trace example after the first reboot with only a path available.....................................................29 4.3Trace example after a reboot with only a path available.............................................................29 4.4iSCSI Multipath errlog............................................................................................................. 30 Chapter 5.The bsan Tool................................................................................................................ 31 StoreWay Multipath for Linux - User's Guide i
Chapter 6.References.................................................................................................................... 37 Chapter 7.Terminology and Concepts.............................................................................................. 39
ii
StoreWay Multipath for Linux - User's Guide
Chapter 1. About this Guide

This document describes how to install and use the StoreWay Multipath product on Linux Operating System Environment. Current supported Linux distributions are: SUSE SLES9 SP3 Red Hat RHEL 4 up2/up4/up5 Red Hat RH 5 up1/up2/up3/up4/up5. Red Hat RH 6 up1
It is assumed that the reader has a good knowledge of the Linux native I/O concepts and features of Linux kernel 2.6.
Chapter 2. Overview
2.1 Purpose
The Bull StoreWay Multipath for Linux product provides two complementary features: Scripts to automatically configure and setup the native multipathing failover / load balancing features of the Device-Mapper module of Linux v2.6 kernel. An additional Nec prio feature to select the bester path.
This document describes the installation and use of these Linux packages.
2.2
Supported Disk Subsystems

The following disk subsystems have been tested and are supported by the Multipathing: StoreWay FDA subsystems family: 1000 and 2000 series with failover and loadbalancing mode. NetApp subsystem (only available on RHEL4 up4) with failover mode. StoreWay Optima1200, Optima 1250 subsystem with failover mode. StoreWay Optima1500/2000/3000 and Optima5000 with failover and loadbalancing mode.
The following disk subsystems have been tested and are supported by the Multipathing feature: FRA1622 subsystem with failover mode.
2.3
Supported Architectures
The following architectures, platforms and distributions are supported: NovaScale SLES9 sp3 ia64, RHEL4 up2/up4/up5 ia64, RHEL5 up1/up2/up3/up4/up5.
NovaScale Universal / NovaScale Blade SLES9 sp3 ia32, RHEL4 up2/up4/up5 ia32, RHEL5 up1/up2/up3/up4/up5. SLES9 sp3 EM-64T, RHEL4 up2/up4/up5 EM-64T, RHEL5 up1/up2/up3/up4/up5 EM-64T.
Novascale F2 RHEL5 up4/up5 ia32, EM-64T. RHEL6 up1 ia32, EM-64T.
2.4
Supported HBA
Emulex HBA / CNA Qlogic HBA Brocade HBA / CNA
mportant All HBA Emulex, Brocade and QLogic models supported by the drivers of the distributions can be used with the multipathing.
2.5
Packages
The multipathing function basic code and modules are directly available on SuSe SLES9SP, RedHat EL 4 up2 /up4/up5 and RedHat EL 5 up1 and upper Linux distributions. The multipath-tools-x.x.x.rpm on SLES9 or device-mapper-multipath-x.x.x.rpm on RHEL4/5 package is also included in these distributions and must be installed on your system. Additional specific packages are needed for SCSI-reservation and will be installed by the install.sh installation script provided by this StoreWay Multipath for Linux product (see the relevant chapter below).
2.6
CD-ROM Contents
The StoreWay Multipath for Linux CD-ROM contains the following items. Packages are delivering the Optima prio feature and the bsan tool that could gather all the disk-related information for a better understanding of the SAN configuration. These features are described in the next chapters. Packages: sles9 i386 bsan_lib-X.X.X-Y.i586.rpm scsi_reserve_gui-X.X.X-Y.i586.rpm bsan_monitor-X.X.X-Y.i586.rpm ia64 bsan_lib-X.X.X-Y.ia64.rpm scsi_reserve_gui-X.X.X-Y.ia64.rpm bsan_monitor-X.X.X-Y.ia64.rpm x86_64 bsan_lib-X.X.X-Y.x86_64.rpm scsi_reserve_gui-X.X.X-Y.x86_64.rpm bsan_monitor-X.X.X-Y.x86_64.rpm rh4/rh5 i386 bsan_lib-X.X.X-Y.i386.rpm bsan-X.X.X-Y.i386.rpm multipath-tools-nec-X.X.X-Y.i386.rpm ia64 bsan_lib-X.X.X-Y.ia64.rpm bsan-X.X.X-Y.ia64.rpm multipath-tools-nec-0.4.7-1.ia64.rpm x86_64 bsan_lib-X.X.X-Y.x86_64.rpm bsan-X.X.X-Y.x86_64.rpm multipath-tools-nec-X.X.X-Y.x86_64.rpm device-mapper-multipath-0.x.x-x.0.BullRH4.x86_64.rpm rh6 x86_64 multipath-tools-nec-X.X.X-Y.x86_64.rpm bsan_lib-X.X.X-Y.x86_64.rpm bsan-X.X.X-Y.x86_64.rpm i386 multipath-tools-nec-X.X.X-Y.i386.rpm bsan_lib-X.X.X-Y.i386.rpm bsan-X.X.X-Y.i386.rpm setup.sh readme 86A293ER11.pdf (this manual)
Chapter 3. Multipathing Feature

3.1 Installation and Configuration
The package multipath-tools-x.x.x.rpm (for SLES9) or device-mapper-multipath-x.x.x.rpm (for RHEL4/5) must be installed before any installation and configuration. Note If you envisage to boot on SAN, consult section 3.8 Boot Support before starting the distribution installation.
3.1.1
Automatic Installation
An installation script is located on the CD-ROM media, which performs all the installation and configuration actions described below in section 3.1.2 Manual Installation. Note On RHEL 6, check the presence of the following package: libsysfs To perform the automatic installation, mount the CD-ROM, go to the mounted cdrom directory and launch the setup.sh script. Example:
The title and CD-ROM version are displayed on the first line. The first choice (Check Supported Distributions) checks and displays the supported distributions, as illustrated below:
The second choice (Activate multipath) installs the bsan packages and configures the multipath. The install is interactive, you must reply to some questions.
Then re-launch the script to continue.
After the Multipath installation, you can either uninstall Multipath or quit. Note The displayed choices are context-dependent. Uninstall packages choice uninstalls all packages. Upgrade packages choice updates the packages (this choice is not displayed in the above screen).
3.1.2
Manual Installation
1. Add the following line in the /etc/modprobe.conf.local file on SLES9 or in the /etc/modprobe.conf file on RHEL4
options lpfc lpfc_nodev_tmo=5
2. 3.
Reload the lpfc driver. Enter the following commands:

# modprobe r lpfc # modprobe lpfc
4.
Rebuild the initrd file.
5.
Activate Multipathing feature. According to the distribution: On SLES9: The Multipathing feature is activated by the following commands:
# /etc/init.d/boot.multipath start # /etc/init.d/multipathd start
To be directly taken into account at the next reboot, the corresponding service can be activated by the command:
# insserv boot.multipath multipathd
Also check that the /etc/sysconfig/kernel file correctly references the lpfc driver in the INITRD_MODULES variable before issuing the mkinitrd command. On RHEL4/5/6: The Multipathing feature is activated by the following commands:
# /etc/rc.d/init.d/multipathd start
To be directly taken into account at the next reboot, the corresponding service can be activated by the command:
# chkconfig --level 235 multipathd on # mkinitrd f /boot//initrd kernel
Comment the three lines on the top of the /etc/multipath.conf file (as stated in the comments):
# Blacklist all devices by default. Remove this to enable multipathing # on the default devices. # devnode_blacklist { # #} ## devnode "*"
6.
Reboot the system.
No further configuration should be required for the default options.
10
3.2
Advanced Multipathing Configuration

Advanced configuration features are available using the /etc/multipath.conf configuration file, but should be changed only with a good knowledge of their meaning. Default values are most often correct, but the following features can be useful: the polling_interval, which defines the frequency of path-health checking the capability to blacklist some devices for which you dont want the Multipathing feature to apply.
3.2.1
Multipathing over ISCSI configuration

To support the multipathing over iSCSI you must modify the /etc/iscsi/iscsid.conf file and then reboot or restart the iSCSI service. Following are the recommended new settings:
node.session.timeo.replacement_timeout = 15 node.conn[0].timeo.noop_out_interval = 5 node.conn[0].timeo.noop_out_timeout = 30
11
3.3
Multipathing Un-installation
You can un-install Multipathing either automatically or manually.
3.3.1
Automatic Un-installation
mportant The un-installation must be done with the SAME version of the CD-ROM.
To perform the automatic un-installation, do as follows: 1. 2. 3. Mount the CD-ROM, Go to the mounted cdrom directory Launch the setup.sh script. The following screen appears:
4.
Select choice 3.
3.3.2
Manual Un-installation
To un-install manually the Multipathing feature (unsuitable), do as follows: 1. Remove the bsan tool:
rpm e bsan_lib-w.x.y-z rpm e bsan[_monitor]-w.x.y-z
12
2.
According to the distribution: On SLES9 Remove the lpfc option (options lpfc lpfc_nodev_tmo=5) in the /etc/modprobe.conf.local file. Run:
mkinitrd
Remove the Multipathing service:

insserv r boot.multipath multipathd
On RHEL4/5/6 Remove the lpfc option (options lpfc lpfc_nodev_tmo=5) in /etc/modprobe.conf file Run:
mkinitrd f /boot//<initrd-file> <kernel>
Remove the Multipathing service :

chkconfig --level 235 multipathd off
13
3.4
Multipathing Status Checking: multipath Command

The multipath command must be used to manage and display the multipathing feature. Example:
# multipath -l 1NEC_iStorage_2000_00000009279003580010A [size=8 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active][best] \_ 2:0:1:1 sde 8:64 [active] \_ round-robin 0 [enabled] \_ 3:0:1:1 sdi 8:128 [active]
In this example: The hexadecimal string (1NEC_iStorage_2000_00000009279003580010A) is the unique identification of the device given back by the device-mapper layer. This identification can be used as unique name of devices on the SAN, bringing the capability of persistent naming over reconfiguration events. It also shows the size, the feature and hardware handlers involved, as well as the priority groups: path are grouped into priority groups where only one priority group is active for I/O usage. To model an active/active configuration, all paths are included in the same group. To model an active/passive configuration, the paths which should not be active in parallel are placed in several distinct priority groups. This setting is completely automatic at device discovery time. Both paths on the same priority group are used with round-robin/load balancing algorithm. For each path, its physical address (host:bus:target:lun), device node name in /dev and major:minor are also displayed. The current path status (active or not) is also shown.
Notes: a) You may change the active priority group with the switchgroup command. Usage: switchgroup mapname index Example:
switchgroup1NEC_iStorage_2000_00000009279003580010A 1
b)
With a NetApp subsystem one of the paths is the main one (i.e. the more efficient one for IOs), the second path is a backup one (see bsan command to display such information). On a failure, the backup path will be activated. After the repair of the failed path: with RHEL4 up2, you can use the perf_netapp command to restore the IOs on the main path. with RHEL4 up4 and upper, it is automatic.
14
3.5
Using Multipath Devices

The corresponding link with the device-mapper object can be found in the /dev/disk/byname directory (on SLES9) or in the /dev/mapper directory (on RHEL4). Example with SLES9:
# cd /dev/disk/by-name # ll total 0 drwxr-xr-x 2 root root 320 Jan 5 09:02 . drwxr-xr-x 5 root root 120 Jan 5 09:02 .. lrwxrwxrwx 1 root root 10 Jan 5 09:02 1NEC_iStorage_2000_00000009279003580010A -> ../../dm-2
The device mapper identification can be directly used for file-system or physical volume creation. Filesystem creation:
# mke2fs /dev/disk/by-name/1NEC_iStorage_2000_0000009279003580010A mke2fs 1.36 (05-Feb-2005) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 1093440 inodes, 2184448 blocks 109222 blocks (5.00%) reserved for the super user First data block=0 67 block groups 32768 blocks per group, 32768 fragments per group 16320 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 Writing inode tables: done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. # mount /dev/disk/byname/1NEC_iStorage_2000_00000009279003580010A /mnt/workdir # mount /dev/sda3 on / type reiserfs (rw,acl,user_xattr) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) /dev/dm-3 on /mnt/workdir type ext2 (rw)
15
Example with RHEL4/5/6:

# ll /dev/mapper
brw-rw---- 1 root disk 253, 3 Aug 1 11:46 1NEC_iStorage_2000_0000009279003580010A crw------1 root root 10, 63 Aug 1 18:22 control
# mke2fs /dev/mapper/1NEC_iStorage_2000_0000009279003580010A mke2fs 1.35 (28-Feb-2004) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 2621440 inodes, 5242880 blocks 262144 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=4294967296 160 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000 Writing inode tables: done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 39 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. # mount /dev/mapper/1NEC_iStorage_2000_0000009279003580010A /mnt/workdir # mount | grep workdir
/dev/mapper/1NEC_iStorage_2000_0000009279003580010A on /mnt/workdir type ext3 (rw)
16
3.6
Using LVM2 on top of the Multipath devices

To make LVM2 recognize the Multipath devices as possible Physical Volumes (PVs), you must modify the /etc/lvm/lvm.conf configuration file, such that LVM2 does not scan and use the physical paths, but only accesses your storage via the Multipathing layer. To perform that, change the filter entry in the /etc/lvm/lvm.conf file as following, and add the extension types to make LVM2 recognize them: For SLES9:
# types = [ "fd", 16 ] types = [ "device-mapper", 16 ]
For RHEL4/5:
# types = [ "fd", 16 ] types = [ "device-mapper", 16 ]
This will allow LVM2 to scan only the /dev/disk/by-name paths with SLES9 or /dev/mapper with RHEL4 and reject everything else. (If you are using LVM2 on non-MPIO devices too, of course you will need to make the necessary adjustments to suit your setup). Then, you can use pvcreate and the other LVM2 commands as usual on the /dev/disk/by-name/ path with SLES9 or /dev/mapper/ path with RHEL4. Warning: Using LVM2, our recommendation is to create Physical Volumes (PVs) on a full disk (and not on a disk partition). If the lun has been previously used with parted (ex filesystem), it must be fully erased before issuing pvcreate, with this command: For SLES9:
clean_parts /dev/disk/by-name/lun-name
or, if this command does not exit, with:

dd >/dev/disk/by-name/lun-name </dev/zero bs=1024k
For RHEL4/5:
clean_parts /dev/mapper/lun-name
or, if this command does not exit, with:

dd >/dev/mapper/lun-name </dev/zero bs=1024k
17
Example with SLES9:

# pvcreate /dev/disk/byname/1NEC_iStorage_2000_00000009279003580010A
Physical volume "/dev/disk/byname/1NEC_iStorage_2000_00000009279003580010A" successfully created # vgcreate vg001 /dev/disk/byname/1NEC_iStorage_2000_00000009279003580010A
Volume group "vg001" successfully created # pvdisplay --- Physical volume --PV Name /dev/dm-2 VG Name vg001 PV Size 8.33 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 2133 Free PE 2133 Allocated PE 0 PV UUID 7m0tyb-k27W-ppxA-upjw-EAUn-PULX-lyiMi0
# lvcreate -L 2G vg001
Logical volume "lvol0" created # mke2fs /dev/vg001/lvol0 mke2fs 1.36 (05-Feb-2005) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 262144 inodes, 524288 blocks 26214 blocks (5.00%) reserved for the super user First data block=0 16 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Writing inode tables: done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 39 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
18
# mount /dev/vg001/lvol0 /mnt/workdir # mount /dev/sda3 on / type reiserfs (rw,acl,user_xattr) /dev/mapper/vg001-lvol0 on /mnt/workdir type ext2 (rw)
Example with RHEL4/5:

# pvcreate /dev/mapper/1NEC_iStorage_2000_00000009279003580010A
Physical volume "/dev/mapper/1NEC_iStorage_2000_00000009279003580010A " successfully created # vgcreate vg3 /dev/mapper/1NEC_iStorage_2000_00000009279003580010A
Volume group "vg3" successfully created # pvdisplay /dev/mapper/1NEC_iStorage_2000_00000009279003580010A --- Physical volume --PV Name /dev/dm-2 VG Name vg3 PV Size 20.00 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 5119 Free PE 5119 Allocated PE 0 PV UUID WuXHVw-Yufj-O1UY-kvVx-tAM5-xkoU-nKdi4N # lvcreate -L 2G vg3
Logical volume "lvol0" created # mkfs /dev/vg3/lvol0 mke2f1.35 (28-Feb-2004) Fesystem label= OSype: Linux Bck size=4096 (log=2) Fagmet size=4096 (log=2) 262144nodes, 524288 blocks 26214bocks (5.00%) reserved for the super user Frt data block=0 Mamum filesystem blocks=536870912 16bock groups 32768bocks per group, 32768 fragments per group 16384nodes per group Sperbock backups stored on blocks:
19
32768, 98304, 163840, 229376, 294912 Wting inode tables: done Wting superblocks and filesystem accounting information: done Th filesystem will be automatically checked every 38 mounts or 180day, whichever comes first. Use tune2fs -c or -i to override. # mount /dev/vg3/lvol0 /mnt/workdir # mount | grep workdir
/dev/mapper/vg3-vol0 on /mnt/workdir type ext3 (rw)
3.7
Partitions on top of Multipath Devices

It is not currently possible to partition the Multipath devices themselves. However, if the underlaying physical device is partitioned, the Multipath device will reflect those partitions and the device-mapper layer will provide /dev/disk/by-name/<name>p1 ... pN devices with SLES9 or /dev/mapper/<name>p1pN with RHEL4 so you can access the partitions. So you will have to partition the devices prior to enabling Multipathing; if you change the partitioning in the running system, the device-mapper will not automatically detect this change and reflect the changes. You'll need to reinit it, which in a running system with active access to the devices will likely imply a reboot. Thus, it is recommended to size the LUNs on your disk subsystem to the needed size of the partition and use them directly, or use LVM2 if you want to aggregate LUNs to make a larger container.
20
3.8
Boot Support
Currently, the Multipathing feature is not available neither for the root nor for the boot partition with SLES9. How to add boot support with RHEL4/RHEL5 The boot support is available only with RHEL4 up4/up5, RHEL5 up1 and with /root filesystem and swap on LVs and /boot on a partition (fat with IA64 or ext3 with IA32 and EM64T). The installation phase must be done with only one path connected. The second path will be connected when the installation is complete. Then: Mount the StoreWay Multipath for Linux CD-ROM Run the script setup.sh and reply to the questions. Shutdown. Connect the second path. Reboot. At this step, the boot partition is not mounted. The access path to this boot partition needs to be determined and updated in the /etc/fstab file. Example with Storeway FDA2300 on IA64:
[root@tigre3 ~]# bsan
#Copyright (C) by BULL 2003,2006 #cmd: 0.1.2 #lib: 2.1.3 all rights reserved. # ============== ADP info ====================== eth0 == ??? == FW:? V:? == IP@:172.16.101.57 = MAC:00:0e:0c:08:c1:22 eth1 == ??? == FW:? V:? == MAC:00:0e:0c:08:c1:23 eth2 == ??? == FW:? V:? == MAC:00:0e:0c:72:53:08 lo == XER == FW:? V:? == MAC:00:00:00:00:00:00 sit0 == XER == FW:? V:? == MAC:00:00:00:00 host0 == scsi0 FW:? V:? == host1 == scsi1 FW:? V:? == ses0 OK == sg0 , raw? == ESG-SHV SCA HSBP M24 6,0 host2 == scsi2 EMU == FW:1.80A2 (T2D1.80A2) V:8.0.16.27 == DID:030000 = 20000000c938c7e1:10000000c938c7e1 tgt2.0 == NEC CORPORATION == 2300 == WWN:200000004c7f0445:210000004c7f0445 DID:0119ef sda OK == sg1 , raw? == NEC iStorage 2000 19900 8,0 0,0 30 R5 sdb OK == sg2 , raw? == NEC iStorage 2000 19900 8,16 0,1 30 R5 host3 == scsi3 EMU == FW:3.93A0 (C2D3.93A0) V:8.0.16.27 == DID:030100 = 20000000c9272023:10000000c9272023 tgt3.0 == XYRATEX == WWN:20000050cc2000b8:24000050cc2000b8 = DID:030d00 ses1 OK == sg3 , raw? == XYRATEX F5402E 0,0 ses2 OK == sg4 , raw? == XYRATEX F5402E 0,2 tgt3.1 == NEC CORPORATION == 2300 == WWN:200000004c7f0445:230000004c7f0445 DID:011ce8 sdc OK == sg5 , raw? == NEC iStorage 2000 19900 8,32 1,0 30 R5 sdd OK == sg6 , raw? == NEC iStorage 2000 19900 8,48 1,1 30 R5 ============== MPIO info ====================== dm-1 == 1NEC_____iStorage_2000___00000009279003580008F pgrp 0 == 2 paths == active /dev/sdb == active Ready == failcount 0 /dev/sdd == active Ready == failcount 0 dm-0 == 1NEC_____iStorage_2000___00000009279003580008E pgrp 0 == 2 paths == active
= 008E 008F
= 008E 008F
= 1 pg/2 paths lb
= 1 pg/2 paths lb
21
/dev/sda /dev/sdc ============== VolGroup00 LogVol01 LogVol02 LogVol00
== active Ready == active Ready
== failcount 0 == failcount 0
LVM info ======================= == 19.31 GB = resizable == sdc sda (dm-6) == 5.88 GB = linear = available (dm-7) == 3.91 GB = linear = available (dm-8) == 1.97 GB = linear = available
= dm-5 = dm-5 = dm-5
= / ext3 (rw) = /home ext3 (rw) = swap prio=-1 used=n
To determine the boot partition path enter:

[root@tigre3 hg]# ll /dev/mapper/
total 0 brw-rw---brw-rw---brw-rw---brw-rw---brw-rw---brw-rw---crw------brw-rw---brw-rw---brw-rw---1 root disk 253, 1 root disk 253, 1 root disk 253, 1 root disk 253, 1 root disk 253, 1 root disk 253, 1 root root 1 root disk 253, 1 root disk 253, 1 root disk 253, 0 Oct 10 4 Oct 10 5 Oct 10 1 Oct 10 2 Oct 10 3 Oct 10 8 Oct 10 6 Oct 10 7 Oct 10 2006 1NEC_____iStorage_2000___00000009279003580008E 2006 1NEC_____iStorage_2000___00000009279003580008E1 2006 1NEC_____iStorage_2000___00000009279003580008E2 2006 1NEC_____iStorage_2000___00000009279003580008F 2006 1NEC_____iStorage_2000___00000009279003580008F1 2006 1NEC_____iStorage_2000___00000009279003580008F2 2006 control 2006 VolGroup00-LogVol00 2006 VolGroup00-LogVol01 2006 VolGroup00-LogVol02
10, 63 Oct 10
And you can see that the first sda partition is:
1NEC_____iStorage_2000___00000009279003580008E1,
Now modify the /etc/fstab file like this:

/dev/mapper/1NEC_____iStorage_2000___00000009279003580008E1 defaults 0 0 /boot/efi vfat
Then issue the mount /boot/efi command (/boot on IA32 or EM64T machine). To verify, enter:
[root@tigre3 hg]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol01 6063688 1917012 3838660 34% / none 2034912 0 2034912 0% /dev/shm /dev/mapper/VolGroup00-LogVol02 4031680 78712 3748168 3% /home /dev/mapper/1NEC_____iStorage_2000___00000009279003580008E1 104170 11984 92186 12% /boot/efi [root@tigre3 hg]# swapon -as
Filename /dev/mapper/VolGroup00-LogVol00
Type partition
Size
Used
Priority -1
2064352 0
22
3.9
Taking into Account Newly Created Devices

With the SLES9 distribution, before issuing the multipath command, the newly created LUNs must be created in the low-level Linux tables using the /bin/rescan-scsi-bus.sh script command. This command must be carefully used to avoid problems. If you don't know exactly the parameter values to enter, a system reboot will perform the new configuration for you without risks. To use the rescan-scsi-bus.sh script: Determine the fibre-channel host adapters to rescan, by looking into the /proc/scsi directory Use the "-w" ("wide") parameter for high-numbered ports scanning Limit the LUN range to the needed one, Run the command with the following syntax (here host4 and host5 are the chosen fibre-channel adapters):
/bin/rescan-scsi-bus.sh -w --hosts=4-5 --luns=0-31
The same command may be used with "-r" option to remove LUNs in the configuration. This command must also be used very carefully, as there is a risk to corrupt the system disk ("/" directory) when it is used with inadequate parameters. So, it is advised to reboot the system to remove LUNs from your configuration. With the RedHat distribution, issue the following command on the adapter X:
echo - - - > /sys/class/scsi_host/hostX/scan
23
3.10
Checking Path at Boot Time

The Multipath system manages only the paths discovered during the boot phase. To compensate for the problem we propose a safety mechanism.
3.10.1
Automatic Checks
After the first boot (after the Multipath installation) StoreWay Multipath checks automatically the paths and if the path number on a device is equal to one, a message is logged as in the example in section 4.2 . After the next reboots StoreWay Multipath checks automatically the paths and if the path number on a device is lesser than the first boot, a message is logged as in the example in section 4.3. The storeway_mpio_check repair command brings back the paths of the multipathing system after the physical path(s) reparation.
3.10.2
How to Reinitialize Manually the File Paths

After a configuration change for example, you can reinitialize the file paths used by the storeway_mpio_check command with the following commands sequence:
root# storeway_mpio_check clear root# storeway_mpio_check init
3.10.3
How to change multipath policy

The Red Hat Enterprise Linux 6 release provides two new path selector algorithms which determine which path to use for the next I/O operation: queue-length and service-time. The queue-length algorithm looks at the amount of outstanding I/O to the paths to determine which path to use next. The service-time algorithm looks at the amount of outstanding I/O and the relative throughput of the paths to determine which path to use next. To change the algorithm, open the /etc/multipath.conf file and coment/uncoment the corresponding lines and execute the following commandes
# multipath -F # multipath
Or reboot the system.
24
3.11
Wiring Subsystems
FRA1622 subsystem direct attach:
Note
The hub mode must be set on the subsystem. FRA1622 subsystem SAN attach:
Note
The hub mode must be reset on the subsystem.
25
StoreWay FDA Optima1500 Optima3000 or Optima5000 FC subsystem direct attach:
StoreWay FDA Optima1500/2000/3000 or Optima5000 FC subsystem SAN attach:
26
StoreWay Optima1200 Optima1250 FC subsystem direct attach:
StoreWay Optima1200 Optima1250 FC subsystem SAN attach:
27
NetApp FC subsystem direct attach:
NetApp FC subsystem SAN attach:
28
Chapter 4. Messages
4.1 FC Multipathing errlog
Here is an example of failover/failback messages in /var/log/messages file: Dec 5 11:10:59 tigre3 kernel: lpfc 0000:06:01.0: 0:0203 Nodev timeout on NPort x119ef Data: x2010808 x7 x6 Dec 5 11:10:59 tigre3 multipathd: devmap event (2) on 1NEC_iStorage_2000_000000092790035800109 Dec 5 11:10:59 tigre3 multipathd: 8:48: mark as failed Dec 5 11:10:59 tigre3 kernel: SCSI error : <2 0 1 0> return code = 0x10000 Dec 5 11:10:59 tigre3 kernel: end_request: I/O error, dev sdd, sector 2922240 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: 8:48 (#2922272): IO error - error: -5 - bi_rw: 8 - bi_flags: 18 - bi_error: 01000005 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: Failing path 8:48 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: 8:48 (#2922240): Requeued sector as #1 Dec 5 11:10:59 tigre3 kernel: end_request: I/O error, dev sdd, sector 2922272 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: 8:48 (#2922304): IO error - error: -5 - bi_rw: 8 - bi_flags: 18 - bi_error: 01000005 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: 8:48 (#2922272): Requeued sector as #2 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: 8:48 (#2922336): IO error - error: -5 - bi_rw: 8 - bi_flags: 18 - bi_error: 01000005 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: 8:48 (#2922304): Requeued sector as #3 Dec 5 11:10:59 tigre3 kernel: device-mapper: dm-multipath: 8:48 (#2922368): IO error - error: -5 - bi_rw: 8 - bi_flags: 18 - bi_error: 01000005 Dec 5 11:11:05 tigre3 multipathd: 8:48: readsector0 checker reports path is up Dec 5 11:11:05 tigre3 kernel: SCSI error : <2 0 1 0> return code = 0x10000 Dec 5 11:11:05 tigre3 multipathd: 8:48: reinstated Dec 5 11:11:05 tigre3 multipathd: devmap event (3) on1NEC_iStorage_2000_000000092790035800109
4.2
Trace example after the first reboot with only a path available
Jun 1 10:15:54 yack23 StoreWay: Init paths file ... Jun 1 10:16:09 yack23 StoreWay: Init multipath with not enough path with 360060160192712008f5027c1f8cbdb11 Jun 1 10:16:09 yack23 StoreWay: Init multipath with not enough path with 360060160192712008e5027c1f8cbdb11 Jun 1 10:16:09 yack23 rc: Starting storeway_mpio: succeeded
4.3
Trace example after a reboot with only a path available

Jun 1 13:03:54 yack23 StoreWay: Check paths ... Jun 1 13:04:08 yack23 StoreWay: Degraded multipath with 360060160192712008f5027c1f8cbdb11 (dm-13 ) Jun 1 13:04:08 yack23 StoreWay: Degraded multipath with 360060160192712008e5027c1f8cbdb11 (dm-12 )
29
4.4
iSCSI Multipath errlog

Aug 6 08:37:12 zeta14 kernel: connection1:0: ping timeout of 5 secs expired, last rx 4791894917, last ping 4791899917, now 4791904917 Aug 6 08:37:12 zeta14 kernel: connection1:0: detected conn error (1011) Aug 6 08:37:12 zeta14 multipathd: sddm: tur checker reports path is down Aug 6 08:37:12 zeta14 kernel: device-mapper: multipath: Failing path 71:64. Aug 6 08:37:12 zeta14 iscsid: Kernel reported iSCSI connection 1:0 error (1011) state (3) Aug 6 08:37:51 zeta14 iscsid: connect failed (113) Aug 6 08:37:57 zeta14 iscsid: connect failed (113) Aug 6 08:38:00 zeta14 multipathd: dm-92: add map (uevent) Aug 6 08:38:00 zeta14 iscsid: connection1:0 is operational after recovery (5 attempts) Aug 6 08:38:05 zeta14 multipathd: 71:64: reinstated Aug 6 08:38:05 zeta14 multipathd: dm-92: add map (uevent)
30
Chapter 5. The bsan Tool

The bsan command displays information on the SCSI hosts and devices. Options : -h -o -O displays the present Help Object Device machine list Object Device machine list verbose
The adapter/Interface lines display: Host Ven FW Ver DID host name Vendor (3 ASCII characters) Firmware version Driver version switchDomain, portId, loopId (with brocade device)
WWN host WorldWideName The target lines display:

Tgt Ven Mod DID Target identificator Vendor Name Target model (if available) switchDomain, portId, loopId (with brocade device)
WWN Target WorldWideName
The devices columns display:

Device State Identify the Linux device name (sd/raw/sg) Device state (test unit ready / reservation).
OK => device ready NOK => device not ready *OK => device ready and registered
!OK => device ready and reserved Parent Ven Type MB dev sid lun Qd Ty Port DEV scsi name of the device Vendor (3 ASCII characters) Device type returned by Inquiry Capacity in M bytes major, minor device Target Id (Decimal) Lun (from 0 to 31) retrieve the actual Queue Depth (HDS, SYMM, DGC, STK) Device Type (R5,Rs,R1,R0,R6+, i, B for BCV) Subsystem attachment Port name Device number inside the subsystem (EMC) Symmetrix DEV Id
Comment : (DGC) Assigned or NotAssigned

Supported subsystems: EMC: NEC: Symmetrix up to Symm7, DAS CX/AX Series 1000 & 2000, Optima5000
31
NetApp: 270, 30xx
The partition lines display:

Device State
Ven MB dev
Identify the device partition name (ex: sda1) Partition state Ex: BSY => busy Ex: IDL => idle Partition type (primary, extended, logical) Capacity in M bytes major, minor device
The filesystem lines display: Fs Label Type BSF ISF Filesystem identiticator (ex: fs0) Filesystem label Filesystem type (ext3, xfs etc...) Block space free (%) Inode space free (%)
The DM part displays device-mapper mirror and device-mapper multipath information: The device-mapper mirror lines display: Device alias log sync Identify the Linux device-mapper name (dm-#) Identify the persistante name (link) to the dm-# Log type {core or disk} Sync type {sync or nosync}
mirrors Mirrors number
The next lines describe the mirror components The device-mapper multipath lines display: Device Alias Pg Path Type Identify the Linux device-mapper name (dm-#) Identify the persistant name (link) to the dm-# Priority group number Path number multipath type (fo=failover, lb=loadbalancing)
The priority group lines display: pgrp# path stat Identify Priority group number Path number per Priority group Priority group status
The Path description lines display: Device Status tur fail Identify the path name (tgt:/dev/sdx) path status test unit ready result fail counter
32
The LVM part displays Logical Volume Group information: The volume group lines display:
VG Size Status Deps Identify the volume group name volume group size volume group status volume group dependence
The logical volume lines display:

LV dm size type status deps Identify the logical volume name associated device-mapper name logical volume size logical volume type logical volume status logical volume dependence
The filesystem lines display: Fs Label Type BSF ISF Filesystem identiticator (ex: fs0) Filesystem label Filesystem type (ext3, xfs etc...) Block space free (%) Inode space free (%)
Example1: FC SAN
#Copyright (C) by BULL 2003,2006 #cmd: 0.1.2 #lib: 2.1.3 all rights reserved. # ============== ADP info ====================== eth0 == NEC == FW:? V:? == IP@:172.16.101.56 = MAC:00:00:4c:71:3f:10 eth1 == Int == FW:? V:? == IP@:192.168.1.56 = MAC:00:03:47:e0:79:b8 eth2 == ZNY == FW:? V:? == IP@:192.168.2.56 = MAC:00:c0:95:f8:8c:14 lo == XER == FW:? V:? == MAC:00:00:00:00:00:00 sit0 == XER == FW:? V:? == MAC:00:00:00:00 host0 == scsi0 ADP == FW:? V:6.2.36 == host1 == scsi1 ADP == FW:? V:6.2.36 == st0 OK == sg0 , raw? == TANDBERG TDC 4222 Bull 5,0 host2 == scsi2 EMU == FW:1.90A3 (E2M1.90A3) V:8.0.16.17 == DID:30600 = 20000000c93fcdef:0x10000000c93fcdef tgt2.0 == NETWORK APPLIANCE CORP. == FAS3020 == WWN:500a09808617b917:500a09818617b917 = DID:030400 sda OK == sg1 , raw? == NETAPP LUN 10240 8,0 0,0 30 sdb OK == sg2 , raw? == NETAPP LUN 10240 8,16 0,1 30 tgt2.1 == XYRATEX == Optima1200 == WWN:20000050cc2000b8:23000050cc2000b8 = DID:030c00 sdc OK == sg3 , raw? == XYRATEX F5402E 9540 8,32 1,0 30 R? 2 sdd OK == sg4 , raw? == XYRATEX F5402E 9540 8,48 1,1 30 R? 3 ses0 OK == sg5 , raw? == XYRATEX F5402E 1,2 tgt2.2 == EMC CORPORATION == Symm7 == WWN:50060482d52cda47:50060482d52cda47 = DID:000000 sde OK == sg6 , raw? == EMC SYMMETRIX 8631 8,64 2,0 30 QT 0080 sdf OK == sg7 , raw? == EMC SYMMETRIX 8631 8,80 2,1 30 QT 0081 sdg OK == sg8 , raw? == EMC SYMMETRIX 8631 8,96 2,2 30 QT 0082 sdh OK == sg9 , raw? == EMC SYMMETRIX 8631 8,112 2,3 30 QT 0083 sdi OK == sg10 , raw? == EMC SYMMETRIX 8631 8,128 2,4 30 QT 0084 sdj OK == sg11 , raw? == EMC SYMMETRIX 8631 8,144 2,5 30 QT 0085 sdk OK == sg12 , raw? == EMC SYMMETRIX 8631 8,160 2,6 30 QT 0086 sdl OK == sg13 , raw? == EMC SYMMETRIX 8631 8,176 2,7 30 QT 0087 sdm OK == sg14 , raw? == EMC SYMMETRIX 8631 8,192 2,8 30 QT 0088 sdn OK == sg15 , raw? == EMC SYMMETRIX 8631 8,208 2,9 30 QT 0089 tgt2.3 == NEC CORPORATION == 2300 == WWN:200000004c7f0445:210000004c7f0445 = DID:0019ef sdo OK == sg16 , raw? == NEC iStorage 2000 8533 8,224 3,0 30 R1 0109 sdp OK == sg17 , raw? == NEC iStorage 2000 8533 8,240 3,1 30 R1 010A
33
host3 == scsi3 EMU == FW:1.00A4 (H2D1.00A4) V:8.0.16.17 == DID:30700 = 20000000c93393dc:0x10000000c93393dc tgt3.0 == NETWORK APPLIANCE CORP. == FAS3020 == WWN:500a09808617b917:500a098a8617b917 = DID:030900 sdq OK == sg18 , raw? == NETAPP LUN 10240 65,0 0,0 30 sdr OK == sg19 , raw? == NETAPP LUN 10240 65,16 0,1 30 tgt3.1 == XYRATEX == Optima1200 == WWN:20000050cc2000b8:22000050cc2000b8 = DID:030d00 sds OK == sg20 , raw? == XYRATEX F5402E 9540 65,32 1,0 30 R? 2 sdt OK == sg21 , raw? == XYRATEX F5402E 9540 65,48 1,1 30 R? 3 ses1 OK == sg22 , raw? == XYRATEX F5402E 1,2 tgt3.2 == EMC CORPORATION == Symm7 == WWN:50060482d52cda49:50060482d52cda49 = DID:000400 sdu OK == sg23 , raw? == EMC SYMMETRIX 8631 65,64 2,0 30 QT 0080 sdv OK == sg24 , raw? == EMC SYMMETRIX 8631 65,80 2,1 30 QT 0081 sdw OK == sg25 , raw? == EMC SYMMETRIX 8631 65,96 2,2 30 QT 0082 sdx OK == sg26 , raw? == EMC SYMMETRIX 8631 65,112 2,3 30 QT 0083 sdy OK == sg27 , raw? == EMC SYMMETRIX 8631 65,128 2,4 30 QT 0084 sdz OK == sg28 , raw? == EMC SYMMETRIX 8631 65,144 2,5 30 QT 0085 sdaa OK == sg29 , raw? == EMC SYMMETRIX 8631 65,160 2,6 30 QT 0086 sdab OK == sg30 , raw? == EMC SYMMETRIX 8631 65,176 2,7 30 QT 0087 sdac OK == sg31 , raw? == EMC SYMMETRIX 8631 65,192 2,8 30 QT 0088 sdad OK == sg32 , raw? == EMC SYMMETRIX 8631 65,208 2,9 30 QT 0089 tgt3.3 == NEC CORPORATION == 2300 == WWN:200000004c7f0445:230000004c7f0445 = DID:001ce8 sdae OK == sg33 , raw? == NEC iStorage 2000 8533 65,224 3,0 30 R1 0109 sdaf OK == sg34 , raw? == NEC iStorage 2000 8533 65,240 3,1 30 R1 010A host4 == scsi4 FW:3.00.00.04 V:5.00.04 == iqn.2000-04.com.qlogic:qla4010.fs20517b00891 rdc0 == DAC == FW:4.07-0-29 V:2.5.47 == c0d0 OK == raw? == Disk RAID-5 48,0 c0d1 OK == raw? == Disk RAID-7 48,8 ============== MPIO info ====================== dm-3 == SEMC_____SYMMETRIX______100329089000 = 1 pg/2 pgrp 0 == 2 paths == active /dev/sdad == active Ready == failcount 2 /dev/sdn == active Ready == failcount 3 dm-9 == 320030050cc2000b8 = 2 pg/2 pgrp 0 == 1 paths == enabled /dev/sdd == active Ready == failcount 2 pgrp 1 == 1 paths == active /dev/sdt == active Ready == failcount 0 dm-10 == SEMC_____SYMMETRIX______100329080000 = 1 pg/2 pgrp 0 == 2 paths == active /dev/sde == active Ready == failcount 3 /dev/sdu == active Ready == failcount 2 dm-14 == SEMC_____SYMMETRIX______100329084000 = 1 pg/2 pgrp 0 == 2 paths == active /dev/sdi == active Ready == failcount 3 /dev/sdy == active Ready == failcount 2 dm-7 == 360a9800043346234525a356f61395478 = 2 pg/2 pgrp 0 == 1 paths == active /dev/sdb == active Ready == failcount 5 == main ok pgrp 1 == 1 paths == enabled /dev/sdr == active Ready == failcount 4 == backup ok dm-2 == SEMC_____SYMMETRIX______100329088000 = 1 pg/2 pgrp 0 == 2 paths == active /dev/sdac == active Ready == failcount 2 /dev/sdm == active Ready == failcount 3 dm-4 == 1NEC_____iStorage_2000___000000092790035800109 = 1 pg/2 pgrp 0 == 2 paths == active /dev/sdae == active Ready == failcount 2 /dev/sdo == active Ready == failcount 2 dm-13 == SEMC_____SYMMETRIX______100329083000 = 1 pg/2 pgrp 0 == 2 paths == active /dev/sdh == active Ready == failcount 4 /dev/sdx == active Ready == failcount 3 dm-8 == 320020050cc2000b8 = 2 pg/2 pgrp 0 == 1 paths == enabled /dev/sdc == active Ready == failcount 2 pgrp 1 == 1 paths == active /dev/sds == active Ready == failcount 0 dm-1 == SEMC_____SYMMETRIX______100329087000 = 1 pg/2 pgrp 0 == 2 paths == active /dev/sdab == active Ready == failcount 2 /dev/sdl == active Ready == failcount 3 dm-6 == 360a9800043346234525a356f47566d33 = 2 pg/2 pgrp 0 == 1 paths == active /dev/sda == active Ready == failcount 5 == main ok pgrp 1 == 1 paths == enabled /dev/sdq == active Ready == failcount 4 == backup ok paths lb
paths fo
paths lb
paths lb
paths fo
paths lb
paths lb
paths lb
paths fo
paths lb
paths fo
34
dm-12 == SEMC_____SYMMETRIX______100329082000 pgrp 0 == 2 paths == active /dev/sdg == active Ready == failcount 3 /dev/sdw == active Ready == failcount 2 dm-5 == 1NEC_____iStorage_2000___00000009279003580010A pgrp 0 == 2 paths == active /dev/sdaf == active Ready == failcount 2 /dev/sdp == active Ready == failcount 2 dm-0 == SEMC_____SYMMETRIX______100329086000 pgrp 0 == 2 paths == active /dev/sdaa == active Ready == failcount 2 /dev/sdk == active Ready == failcount 3 dm-11 == SEMC_____SYMMETRIX______100329081000 pgrp 0 == 2 paths == active /dev/sdf == active Ready == failcount 3 /dev/sdv == active Ready == failcount 2 dm-15 == SEMC_____SYMMETRIX______100329085000 pgrp 0 == 2 paths == active /dev/sdj == active Ready == failcount 3 /dev/sdz == active Ready == failcount 2
= 1 pg/2 paths lb
= 1 pg/2 paths lb
= 1 pg/2 paths lb
= 1 pg/2 paths lb
= 1 pg/2 paths lb
============== LVM info ====================== VG1 == 16.66 GB = resizable == sdo sdae sdp sdaf lvol0 (dm-16) == 3.00 GB = linear = available = dm-4
Example2: iSCSI SAN

#Copyright (C) by BULL 2003-2009 #cmd: 0.2.4 #lib: 2.2.8 all rights reserved. # ============== ADP info ====================== eth0 == ??? = up = FW:? V:? == IP@:192.168.95.8 = MAC:00:15:17:6c:47:e4 eth1 == ??? = up = FW:? V:? == IP@:192.168.1.102 = MAC:00:16:97:c4:a9:b4 eth2 == ??? = up = FW:? V:? == IP@:192.168.0.103 = MAC:00:15:17:6c:47:e5 eth3 == ??? = unknown = FW:? V:? == MAC:00:16:97:c4:a9:b5 lo == XER = unknown = FW:? V:? == MAC:00:00:00:00:00:00 ... host5 == scsi5 sys0 == FW:? V:? == ssn1 == ISCSI Session == Optima1500 == IQN:iqn.2001-03.jp.nec:storage01:ist-3-10-sn-0000000935082027.lxrhel00.target0000 cnx1-0 == ISCSI Connection == 192.168.0.35:3260 [eth2] sddl OK == sg118, raw? == NEC iStorage 1000 1.6GB 71,48 0,0 32 R0 0001 sddm OK == sg119, raw? == NEC iStorage 1000 1.6GB 71,64 0,1 32 R0 0002 host6 == scsi6 sys0 == FW:? V:? == ssn2 == ISCSI Session == Optima1500 == IQN:iqn.2001-03.jp.nec:storage01:ist-3-10-sn-0000000935082027.lxrhel00.target0000 cnx2-0 == ISCSI Connection == 192.168.1.35:3260 [eth1] sddn OK == sg120, raw? == NEC iStorage 1000 1.6GB 71,80 0,0 32 R0 0001 sddo OK == sg121, raw? == NEC iStorage 1000 1.6GB 71,96 0,1 32 R0 0002 ... ===========-== DM info ======================= ... dm-92 == 20030138419850002 = 2 pg/2 paths fo pgrp 0 == 1 paths == active ssn1:/dev/sddm == active Ready == failcount 4 pgrp 1 == 1 paths == enabled ssn2:/dev/sddo == active Ready == failcount 0 dm-93 == 20030138419850001 = 2 pg/2 paths fo pgrp 0 == 1 paths == active ssn1:/dev/sddl == active Ready == failcount 4 pgrp 1 == 1 paths == enabled ssn2:/dev/sddn == active Ready == failcount 0 ... ============== LVM info ======================= ...
35
36
Chapter 6. References
Multipathing online documentation: http://christophe.varoqui.free.fr/multipath.html http://sources.redhat.com/dm For RHEL6: http://docs.redhat.com/docs/enUS/Red_Hat_Enterprise_Linux/6/html/DM_Multipath/index.html
LVM: LVM-HOWTO, CD 4: /usr/share/doc/howto/en/txt/LVM-HOWTO.gz /usr/share/doc/howto/en/html/LVM-HOWTO/index.html http://sourceware.org/lvm For RHEL6: http://docs.redhat.com/docs/enUS/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/index.htm l
37
38
Chapter 7. Terminology and Concepts

alias: The alias is the name of a Multipath device. By default, the alias is set to the WWID. However, by setting the "user_friendly_names" configuration option, the alias is set to a unique name of the form mpath<n>. The alias name can also be explicitly set for each Multipath device in the configuration file. NOTE: While the alias in guaranteed to be unique on a node, it is not guaranteed to be the same on all nodes using the Multipath device. Also, it may change. Failback: At regular intervals, multipathd checks the current priority of all path groups. If the current path group is not the highest priority path group, multipathd reacts according to the failback mode. By default, multipathd immediately switches to the highest priority path group. Other options for multipathd are to (a) wait for a user-defined length of time (for the path groups to stabilize) and then switch or (b) for multipathd to do nothing and wait for manual intervention. Failback can be forced at any time by running the multipath command. Failover: When I/O to a path fails, the dm-multipath module tries to switch to an enabled path group. If there are no enabled path groups with any paths in the ready state, dm-multipath tries to switch to a disabled path group. If necessary, dm-multipath runs the hardware handler for the Multipath device. Hardware Handler: A kernel module that performs hardware-specific actions when switching path groups and dealing with I/O errors. Multipath device: The Multipath device is the device mapper device created by dm-multipath. A Multipath device can be identified by either its WWID or its alias. A Multipath device has one or more path groups. It also has numerous attributes defined in the following file: /usr/share/doc/device-mapper-multipathd-0.4.5/multipath.conf.annotated. Path: The connection from an HBA port to a storage controller port for a LUN. Each path appears as a separate device. Paths can be in various states (refer to "Path States"). Path Group: A grouping of paths. With DM-MP, only one path group--the active path group--receives I/O at any time. Within a path group, DM-MP selects which ready path should receive I/O in a round robin fashion. Path groups can be in various states (refer to "Path Group States"). Path Group Priority: Each path group has a priority that is equal to the sum of the priorities of all the non-faulty paths in the group. By default, the multipathd daemon tries to ensure that the path group with the highest priority is always in the active state.
39
Path Group States:
active - Path group currently receiving I/O requests. enabled - Path groups to try if the active path group has no paths in the ready state. disabled - Path groups to try if the active path group and all enabled path groups have no paths in the active state. NOTE: The disabled state only exists for certain storage arrays.
Path Priority: Each path can have a priority assigned to it by a callout program. Path priorities can be used to group paths by priority and change their relative weights for the round robin path selector. Path States: ready - Path is able to handle I/O requests. shaky - Path is up, but temporarily not available for normal operations. faulty - Path is unable to handle I/O requests. ghost - Path is a passive path, on an active/passive controller. NOTE: The shaky and ghost states only exist for certain storage arrays. WWID: The WWID (World Wide Identifier) is an identifier for the Multipath device that is guaranteed to be globally unique and unchanging. It is determined by the getuid callout program.
40
BULL CEDOC 357 AVENUE PATTON B.P.20845 49008 ANGERS CEDEX 01 FRANCE
REFERENCE
86 A2 93ER 11

86A293ER11

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

86A293ER11

Transféré par

Droits d'auteur :

Formats disponibles

StoreWay Multipath for Linux

STOREWAY StoreWay Multipath for Linux

Copyright Bull SAS 2011 Printed in France

Trademarks and Acknowledgements

Chapter 6.References.................................................................................................................... 37 Chapter 7.Terminology and Concepts.............................................................................................. 39

StoreWay Multipath for Linux - User's Guide

Chapter 1. About this Guide

StoreWay Multipath for Linux - User's Guide

StoreWay Multipath for Linux - User's Guide

Supported Disk Subsystems

Novascale F2 RHEL5 up4/up5 ia32, EM-64T. RHEL6 up1 ia32, EM-64T.

StoreWay Multipath for Linux - User's Guide

StoreWay Multipath for Linux - User's Guide

StoreWay Multipath for Linux - User's Guide

StoreWay Multipath for Linux - User's Guide

Chapter 3. Multipathing Feature

StoreWay Multipath for Linux - User's Guide

Then re-launch the script to continue.

StoreWay Multipath for Linux - User's Guide

Reload the lpfc driver. Enter the following commands:

Rebuild the initrd file.

StoreWay Multipath for Linux - User's Guide

Reboot the system.

No further configuration should be required for the default options.

StoreWay Multipath for Linux - User's Guide

Advanced Multipathing Configuration

Multipathing over ISCSI configuration

StoreWay Multipath for Linux - User's Guide

StoreWay Multipath for Linux - User's Guide

Remove the Multipathing service:

Remove the Multipathing service :

StoreWay Multipath for Linux - User's Guide

Multipathing Status Checking: multipath Command

StoreWay Multipath for Linux - User's Guide

Using Multipath Devices

StoreWay Multipath for Linux - User's Guide

Example with RHEL4/5/6:

/dev/mapper/1NEC_iStorage_2000_0000009279003580010A on /mnt/workdir type ext3 (rw)

StoreWay Multipath for Linux - User's Guide

Using LVM2 on top of the Multipath devices

or, if this command does not exit, with:

or, if this command does not exit, with:

StoreWay Multipath for Linux - User's Guide

Example with SLES9:

Physical volume "/dev/disk/byname/1NEC_iStorage_2000_00000009279003580010A" successfully created # vgcreate vg001 /dev/disk/byname/1NEC_iStorage_2000_00000009279003580010A

StoreWay Multipath for Linux - User's Guide

Example with RHEL4/5:

Physical volume "/dev/mapper/1NEC_iStorage_2000_00000009279003580010A " successfully created # vgcreate vg3 /dev/mapper/1NEC_iStorage_2000_00000009279003580010A

StoreWay Multipath for Linux - User's Guide

/dev/mapper/vg3-vol0 on /mnt/workdir type ext3 (rw)

Partitions on top of Multipath Devices

StoreWay Multipath for Linux - User's Guide

StoreWay Multipath for Linux - User's Guide

/dev/sda /dev/sdc ============== VolGroup00 LogVol01 LogVol02 LogVol00

== active Ready == active Ready

= dm-5 = dm-5 = dm-5

= / ext3 (rw) = /home ext3 (rw) = swap prio=-1 used=n

To determine the boot partition path enter:

Now modify the /etc/fstab file like this:

StoreWay Multipath for Linux - User's Guide

Taking into Account Newly Created Devices

StoreWay Multipath for Linux - User's Guide

Checking Path at Boot Time