Recommended Best Practices Considerations for High

Availability on IBM® System Storage™ SAN Volume

Controller Solution

Prepared by: Cam-Thuy Do and John Sing

Reviewed by: Ian Wright and Stephen Edel
IBM High Availability Center of Competency
September 2008

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

Trademarks
IBM, IBM eServer, IBM logo, e-business logo, CICS, DB2, MQ, ESCON, Enterprise Storage Server, GDPS, IMS, MVS,
OS/390, Parallel Sysplex, Redbook, Resource Link, S/390, System z9.iSeries, pSeries, xSeries, OS/400, i15OS, System
Storage, TotalStorage, VM/ESA, VSE/ESA, WebSphere, z/OS, z/VM, z/VSE, and zSeries are trademarks or registered
trademarks of International Business Machines Corp. in the United States, other countries, or both.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

Microsoft is a registered trademark of Microsoft Corporation in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States, other countries, or both.

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

IBM SAN Volume Controller

SVC two node cluster

SVC four node cluster

ƒ This document provides a summary of High Availability best practices for the IBM
SAN Volume Controller storage virtualization

ƒ The reader is assumed to have a baseline understanding of the concepts and

facilities of these products

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

System Storage SAN Volume Controller (SVC) Practices

ƒ Configuration
• Deploy SAN Volume Controller (SVC) in node pair (2, 4, 6, 8) to meet performance
requirements
• Connect each SVC node of a pair to a different uninterruptible power supply (UPS)
• Configure MDisk group comprising disks from one disk subsystem
• Plan carefully when configured multiple SVC clusters sharing the same disk subsystem
• Configure quorum disks for each SVC cluster on separate storage controllers
• Configure storage subsystems in same manner as without SVC
• Test error alerting and Call-Home fully before put SVC into production operation

ƒ Enable Alert Notification

• Setting up Simple Network Management Protocol (SNMP) alerts for errors
• Having all devices capable of sending SNMP alerts to the appropriate SNMP manager

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

System Storage SAN Volume Controller (SVC) Practices

ƒ Monitor
• Error logs and SNMP traps for all devices on the fabric regularly
• All I/O paths between all hosts and SANs are working and online
• SDD/SDDPCM Server daemon is running and configured to start at system initialization
on all servers connected to SVC

ƒ SVC Master Console Software Installation – ensure your system satisfies both
hardware and software installation prerequisites before installing
• IBM TotalStorage Master Console: Installation and User’s Guide, GC30-4090

ƒ Exploit available options

• Adding a new storage controller dynamically
• Replacing nodes non-disruptively
• Migrate data from one backend controller to another non-disruptively
• Concurrent Maintenance for all field replaceable units (FRUs)
• Concurrent Code Update SAN Volume Controller software

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

System Storage SAN Volume Controller (SVC) Practices

ƒ Maintain Currency
• Create a regular maintenance window for SAN and SVC
™ Order of priority check
9 SVC
9 Storage
9 Switches
9 Host support levels
™ Quarterly update to the latest supported levels for SAN code and bi-annually for major
code
• Subscribe to MySupport/SupportNotification for Flashes and Alert
• Follow the maintenance instructions that are provided for your storage subsystem
• Monitor “Latest Updates” page on the SVC Support website for new releases, hints & tips
• Understand what fixes/upgrades are required
• Test prior to production
• Back up the SVC configuration data after each major change in the environment
• Check for support when adding new servers, switches, storage etc to the SVC

ƒ Perform the following steps before conducting maintenance on the SVC:

• Verify all I/O paths between all hosts and SANs are working
• Verify no unfixed errors in the SVC error log
• Upgrade SVC code
™ Back up the configuration data and save the configuration file in a safe place
™ Save the data collection for support diagnosis just in case of problem

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

System Storage SAN Volume Controller (SVC) Practices

ƒ Storage/Server
• Configure SVC nodes in redundant SAN fabrics
• Distribute FC ports of each HBA card across the dual fabric
• Configure paths to hosts
™ Zoning to limit from minimum of two paths to the maximum of eight paths
(typically four paths)
™ Zoning each port to a different set of SVC control ports to maximize performance and
redundancy

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

VDisk Mirroring (VDM)

ƒ VDisk Mirroring allows creation of VDisk with more than one copy
• Low level RAID1 mirroring to protect against controller / MDisk group failure
• The copies can be in different MDisk groups
• The copies can be of complete different structure (image, striped, sequential, SE)

ƒ VDisk Mirroring sits below the cache and replication services

• Cache, Metro Mirror, Global Mirror have no awareness that a VDisk is mirrored
• Anything that can be done today with a VDisk can be done with a mirrored VDisk
Š FlashCopy, data migration, expand size, etc.
• Mirrored VDisks have the same restrictions that a VDisk has today
Š e.g. A mirrored VDisk is owned by an IO group; if an IO group goes down, access is lost

ƒ Must allocate bitmap space prior to using VDisk Mirroring

• Use “chiogrp” command to allocate space
• V4.2.1 introduced configurable bitmap space that can be used by MM/GM/FC/VDM
Š 512MB per I/O group which equates to 1PB of VDisk space for copy services, source and
target VDisks, and for VDM copies

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

Optional Advanced Copy Functions Overview for Availability

ƒ Point in Time Copy (FlashCopy)

• Minimize application / database downtime required to make local point in time copies for:
™ Production backup, data cloning, data warehouse, test and development
• Disk subsystem microcode creates internal copy of data (FlashCopy)
™ Copy of large terabytes of data can be accomplished in seconds

ƒ Remote Mirroring (Metro Mirror, Global Mirror)

• Create real-time, continuously updated remote copies of disk subsystem data
™ Campus, metropolitan, or geographically distant site
• Data suitable for High Availability fast failover and failback
• Supports large amounts of data, at the terabyte level
• Disk subsystem microcode mirrors volumes/LUNs to remote disk subsystem
™ Synchronous capability (Metro Mirror)
™ Asynchronous capability (Global Mirror)

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

FlashCopy - Point in Time internal Data Replication

ƒ Internal data replication capability (name: FlashCopy)
• Used to create internal copies of data for backup, cloning, data mining, etc.
ƒ Physical configuration
• Assure sufficient target disk space allocated
ƒ Usage practices:
• Assure data integrity is maintained in the Point in Time Copy
™ Plan databases/applications to be in hot backup mode or quiesce
• Implement operational procedures to update internal volume/LUN metadata as
required for:
™ Operating System catalogs, etc.

™ Database/application metadata

ƒ Capacity Planning
• Plan and configure source and target physical volumes / subsystems to be of similar
speed and throughput
ƒ Minimize or avoid FlashCopy interplay with volumes/LUNs in remote mirroring
relationships
• Typically should avoid FlashCopy onto in-session remote mirroring volumes

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

Metro Mirror - Synchronous Data Replication

ƒ Applicability:
• General: provide synchronous data replication of data under control of SVC, at
volume / LUN level
ƒ Physical configuration, link and infrastructure planning
• Must perform initial and ongoing analysis of write workload – To identify peaks and
assure ongoing sufficient SAN/WAN/telecom infrastructure bandwidth
• Should have similar speed and throughput characteristics on source and target
volumes
ƒ Automation
• Plan for highly automated operational control of mirroring (TPC-R is recommended)
™To mask complexity, and assure reliability, repeatability, testability
ƒ Testing and testing resource expectations
• Plan to provide Tertiary Copy storage at remote site
™ For every production TB to be mirrored, ideally 2x that TB at remote site
™ To provide ongoing testing environment for resync protection, testing, setup,
validation, problem determination

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

Global Mirror - Asynchronous Data Replication

ƒ Applicability of IBM SVC Global Mirror :
• General: provide asynchronous data replication at volume/LUN level for data
under control of SAN Volume Controller
ƒ Physical configuration, link and infrastructure planning
• Essential to perform initial and ongoing analysis of write workload
To identify peaks and assure ongoing sufficient SAN/WAN/telecom
™
infrastructure bandwidth - SVC Global Mirror must have sufficient bandwidth
to handle peaks
ƒ Automation
• Plan for highly automated operational control of mirroring
™ To mask complexity and assure reliability, repeatability, testability
ƒ Testing and testing resource expectations
• Plan to provide sufficient Tertiary Copy storage at remote site
™ For every production TB to be mirrored, ideally 2x that TB at remote site
™ To provide resync protection, ongoing testing environment for setup,
validation, problem determination

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

System Storage SAN Volume Controller (SVC) Practices -

Management of Replication

ƒ Plan for highly automated disk mirroring environment

ƒ Provides foundation for Reliability, Repeatability, Scalability, Testability

ƒ Recommendations for automation software:

ƒ Recommendation: TPC for Replication

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

System Storage SAN Volume Controller Practices Resources

References
ƒ IBM System Storage SAN Volume Controller Planning Guide (GA32-0551)
ƒ IBM System Storage SAN Volume Controller Installation Guide (GC26-7900)
ƒ IBM System Storage SAN Volume Controller Configuration Guide (SC26-7902)
ƒ IBM System Storage SAN Volume Controller Host Attachment User’s Guide (SC26-7905)
ƒ IBM System Storage Master Console for SAN Volume Controller: Installation & User’s Guide
ƒ IBM System Storage SAN Volume Controller (SG24-6423)

Class
ƒ SAN Volume Controller Planning and Implementation Workshop (Course code SN83)

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

System Storage SAN Volume Controller Practices Resources

System Storage SAN Volume Controller Technology Center

ƒ http://www-03.ibm.com/servers/storage/software/virtualization/svc
ƒ SVC Education http://www-03.ibm.com/systems/education/cust/software/custsoft.html

System Storage Services

ƒ http://www-03.ibm.com/systems/storage/services/index.html

Redbooks/Redpapers
ƒ http://www.redbooks.ibm.com/redbooks.nsf/portals/Storage
ƒ IBM System Storage SAN Volume Controller (SG24-6423-04)
ƒ Understanding the IBM TotalStorage Software Family (SG24-7098-00)
ƒ IBM System Storage Solutions Handbook (SG24-5250-06)
ƒ SAN Volume Controller: Best Practices and Performance Guidelines (SG24-7521-01)

White papers
ƒ SAN Volume Controller Global Mirror statistics using TPC for Disk v3.1.3
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101033

ƒ IBM System Storage SAN Volume Controller 4.3: Proven, Enterprise-Ready and Scalable
Storage Virtualization Solution (Paper by Taneja Group)
http://www-03.ibm.com/systems/storage/software/virtualization/saying.html

© Copyright 2007 IBM Corporation. All rights reserved.

 IBM High Availability Center of Competency

Disclaimers

Copyright © 2007 by International Business Machines Corporation.

No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation.

Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This
information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the
product(s) and/or programs(s) at any time without notice.

Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and
objectives only.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such such products,
programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in
this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that
does not infringe IBM’s intellectually property rights, may be used instead. It is the user’s responsibility to evaluate and verify the
operation of any on-IBM product, program or service.

THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR
IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR
NONINFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted according to the terms
and conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License
Agreement, etc.) under which they are provided. IBM is not responsible for the performance or interoperability of any non-IBM products
discussed herein.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or
copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:

IBM Director of Licensing

IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.

© Copyright 2007 IBM Corporation. All rights reserved.