DDM v3 0 53-0000366-03 C

Brocade SilkWorm
Design, Deployment, and

Management Guide
SAN DDM
Version 3.0
Publication Number: 53-0000366-03

Publication Date: April 8, 2004
Copyright © 2004, Brocade Communications Systems, Incorporated.
ALL RIGHTS RESERVED. Publication Number: 53-0000366-03
Brocade, the Brocade B weave logo, Secure Fabric OS, and SilkWorm are registered trademarks of Brocade
Communications Systems, Inc., in the United States and/or in other countries. FICON is a registered trademark of IBM
Corporation in the U.S. and other countries. All other brands, products, or service names are or may be trademarks or
service marks of, and are used to identify, products or services of their respective owners. Notice: This document is for
informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment,
equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this
document at any time, without notice, and assumes no responsibility for its use. This informational document describes
features that may not be currently available. Contact a Brocade sales office for information on feature and product
availability.
The authors and Brocade Communications Systems, Inc. shall have no liability or responsibility to any person or entity
with respect to any loss, cost, liability, or damages arising from the information contained in this book or the computer
programs that accompany it.
Notice: The product described by this document may contain “open source” software covered by the GNU General
Public License or other open source license agreements. To find-out which open source software is included in Brocade
products, view the licensing terms applicable to the open source software, and obtain a copy of the programming source
code, please visit http://www.brocade.com/support/oscd.
Export of technical data contained in this document may require an export license from the United States Government.
Brocade Communications Systems, Incorporated
Corporate Headquarters
1745 Technology Drive
San Jose, CA 95110
T: (408) 487-8000
F: (408) 487-8101
Email: info@brocade.com
Asia-Pacific Headquarters
9/F, One International Finance
Center, 1 Harbour View Street,
Central, Hong Kong
Tel: +852 2539 0600
Fax: +852 2539 0613
Email: apac-info@brocade.com
European Headquarters
29, route de l’Aeroport
Case Postale 105
CH-1211 Geneva 15,
Switzerland
T: +41 22 799 56 40
F: +41 22 799 56 41
Email: europe-info@brocade.com
Latin America Headquarters

5201 Blue Lagoon Drive
Miami, FL 33126
T: (305) 716-4165
Email: latinam-sales@brocade.com
Document History
The table below lists all versions of the Brocade SilkWorm Design, Deployment, and Management Guide.
Document version Publication Number Publication Date
First Publication 53-0000366-01 4/30/2003

Version 2.1: 53-0000366-02 08/15/03
Updates to the Design chapter include additional SAN
Design concepts and background information, Secure Fabric
OS implementation. The Design chapter has also been
expanded into two chapters, with additional information in
Chapter 3, Architecting SANs With SilkWorm Switches.
Updates to the Deployment chapter include the addition of
Section 4.2.6 Staging SAN Fabrics with Secure Fabric
OS (SFOS),
Updates to the Management chapter includes the update of
graphics.
Version 3.0: 53-0000366-03 4/8/2004
The document has been updated to include information about
Fabric OS 4.2 and related features in all sections. The book is
now three sections with multiple new chapters and
appendices added.
Table of Contents
Preface xvii
Chapter 1 Product Introduction
Section I SAN Design

Chapter 2 SAN Design & Architecture Concepts
2.1 SAN Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
2.1.1 Storage Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
2.1.2 Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
2.1.3 High Availability / Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3
2.1.4 Extended Distance Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
2.2 SAN Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
2.2.1 Fabric Resiliency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
2.2.2 SAN Availability Classifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6
2.2.3 Redundant Fabric SANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7
2.3 Trunking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9
2.4 ISL Oversubscription Ratios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-13
2.4.1 Recommended ISL Oversubscription Ratios. . . . . . . . . . . . . . . . . . . . . . . . . . . 2-14
2.5 Locality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-17
2.6 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19
2.7 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-21
2.7.1 I/O Profiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-22
2.7.2 Fabric Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-23
Brocade SAN Design, Deployment, and Management Guide i

2.8 SAN Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-24
2.8.1 Simple Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-24
2.8.2 Hybrid Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-32
Chapter 3 Architecting SANs With SilkWorm Switches

3.1 Device Attachment Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1
3.1.1 Trunk and ISL Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1
3.1.2 Edge Switch ISL/Trunk Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
3.1.3 Core Switch and Standalone ISL/Trunk and Device Connections . . . . . . . . . . 3-10
3.1.4 Attaching SAN Devices For Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12
3.1.5 Connecting Devices To The Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13
3.1.6 Tiering (Low Locality Device Attachment) . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14
3.2 Platform Specific Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17
3.2.1 SilkWorm 2000 Series, 32x0, and 38x0 Switches. . . . . . . . . . . . . . . . . . . . . . . 3-17
3.2.2 SilkWorm 24000, 12000 and 3900 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18
3.3 Zoning Design Considerations & Guidelines . . . . . . . . . . . . . . . . . . . . . . . . 3-20
3.3.1 Zoning and Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20
3.3.2 Zoning Database Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-21
3.4 Designing SANs With Secure Fabric OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22
3.5 Switch Location In The Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24
3.5.1 Management and Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24
3.5.2 2 Gbit/sec Switch Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-26
3.5.3 Locating A Switch For Fabric and I/O Availability . . . . . . . . . . . . . . . . . . . . . 3-27
3.6 Scalability Support and Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29
3.7 Recommended Fabric Topologies and SAN Designs . . . . . . . . . . . . . . . . . 3-30
ii Brocade SAN Design, Deployment, and Management Guide

Section II SAN Deployment
Chapter 4 SAN Deployment Overview
4.1 Introduction to Fabric OS 2.6.2, 3.1.2, 4.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
4.1.1 Fabric OS 2.6.2/3.2/4.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
4.2 Fabric OS 4.2 Platform Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3
4.2.1 SilkWorm 24000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3
4.2.2 SilkWorm 3850 and SilkWorm 3250 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3
4.3 Fabric OS 2.6.2, 3.1.2, 4.2 Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4
Chapter 5 Deployment - Planning

5.1 Site Environment Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1
5.2 SAN Project Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
5.3 Planning for Power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4
5.3.1 SilkWorm 2000 and 3000 Power Requirements . . . . . . . . . . . . . . . . . . . . . . . . 5-4
5.3.2 SilkWorm 12000 and 24000 Director Power Requirements . . . . . . . . . . . . . . . 5-4
5.4 Cable Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
5.5 Platform Specific Cable Management Guidelines . . . . . . . . . . . . . . . . . . . . 5-5
5.5.1 Cable Management for the SilkWorm 2000 and 3000 . . . . . . . . . . . . . . . . . . . 5-5
5.5.2 SilkWorm 12000/24000 Cable Management Planning . . . . . . . . . . . . . . . . . . . 5-8
5.5.3 SilkWorm 12000/24000 Cabling Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9
5.6 The Rack Layout Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10
5.6.1 Racking Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-11
5.7 Documentation Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-12
5.8 Zoning Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-15
5.8.1 Zoning Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16
5.8.2 Zoning Enforcement Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-17
5.8.3 Zone Port Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-17
Brocade SAN Design, Deployment, and Management Guide iii

5.9 Planning the Upgrade of Fabric OS 2.x/3.x/4.x
to Fabric OS 2.6.1/3.1/4.1 or Later Versions . . . . . . . . . . . . . . . . . . . . 5-18
5.9.1 Planning Principal Switch Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-19
5.9.2 Planning for Secure Fabric OS Security Measures . . . . . . . . . . . . . . . . . . . . . . 5-19
5.10 Secure Fabric OS Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-21
5.10.1 Secure Fabric OS Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-21
5.10.2 Secure Fabric OS Switch Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-22
5.10.3 Secure Fabric OS Default Policy Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-23
5.11 Secure Fabric OS Pre-Installation Planning . . . . . . . . . . . . . . . . . . . . . . . . 5-23
5.12 Secure Fabric OS Pre-Installation Checklist . . . . . . . . . . . . . . . . . . . . . . . . 5-24
5.13 Planning the Secure Fabric OS Implementation . . . . . . . . . . . . . . . . . . . . 5-26
5.14 SAN Secure Fabric OS Software Utility Considerations . . . . . . . . . . . . . 5-28
5.14.1 PKIcert Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
5.14.2 Brocade Secure Telnet (SecTelnet) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
5.14.3 Using Secure Shell (SSH) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
5.15 LAN Planning Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
5.16 Extended Fabrics Planning Essentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
5.16.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
5.16.2 Compatibility and Interoperability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
5.16.3 Extended Fabrics Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-30
5.17 SilkWorm 12000 Upgrade Planning Guidelines . . . . . . . . . . . . . . . . . . . . . 5-32
Chapter 6 Deployment - Staging and Validation

6.1 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2
6.1.1 Case Study SAN Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2
6.2 Powering the SAN Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-3
iv Brocade SAN Design, Deployment, and Management Guide

6.3 Preparing the Switches for the SAN Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . 6-3
6.3.1 Gather Planning Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4
6.3.2 Check Fabric OS Version and Environmental Status . . . . . . . . . . . . . . . . . . . . 6-4
6.3.3 Set IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-6
6.3.4 Set Date and Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
6.3.5 Set the Switch Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
6.3.6 Set Domain ID and Core PID Format in the Same Session (On Non-Fabric OS 4.x
Switches). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
6.3.7 (Optional) Extended Edge PID Format Setting . . . . . . . . . . . . . . . . . . . . . . . . . 6-9
6.3.8 Add Fabric OS Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-14
6.3.9 (Optional) Name Devices with PortName . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-15
6.3.10 (Optional) Setting up a Preferred Principal Switch in the Fabric (Fabric OS 4.1 or
Higher) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-16
6.3.11 Set Telnet Session Timeout Value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17
6.3.12 (Optional) Disabling the Telnet Daemon When Secure Mode is Enabled (Fabric OS
4.1 or greater only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-18
6.3.13 (Optional) Setup Ports for Extended Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . 6-19
6.3.14 (Optional) Setup Fabric Watch, SNMP Traps, switchstatuspolicyset . . . . . . . 6-20
6.3.15 Baseline and Backup the Switch With Configupload . . . . . . . . . . . . . . . . . . . 6-20
6.4 SAN Fabric Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-21
6.4.1 Gather Planning Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-21
6.4.2 Cable ISLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-21
6.4.3 Cable Hosts and Storage Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-21
6.4.4 Use Documentation to Label All Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-23
6.4.5 (Optional) Dummy Zone Configuration to Prevent Device Access . . . . . . . . . 6-23
6.4.6 Setup NTP Time Server on the Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-23
6.4.7 Run Fabric Diagnostics with PortTest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-24
6.4.8 Pre-Zoning Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-25
6.4.9 Implement Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-26
6.4.10 Profile The SAN Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-28
Brocade SAN Design, Deployment, and Management Guide v

6.5 Staging SAN Fabrics with Secure Fabric OS (SFOS) . . . . . . . . . . . . . . . . . 6-29
6.5.1 Staging Secure Fabric OS with Fabric Manager . . . . . . . . . . . . . . . . . . . . . . . . 6-29
6.5.2 Secure Fabric OS and Extended Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-29
6.5.3 Staging Secure Fabric OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-30
6.5.4 Example of Managing Secure Fabric OS with Fabric Manager . . . . . . . . . . . . 6-48
6.5.5 Managing Fabric Changes: Secure Fabric OS for Change Management Control 6-51
6.5.6 Adding a Switch to a Secure Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-55
6.5.7 Removing a Switch from a Secure Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-57
6.6 Validating the SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-58
6.6.1 Sample Script to Generate I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-58
6.6.2 Sample Validation Recommendations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-58
Chapter 7 Maintenance and Operations

7.1 Upgrading from Fabric OS 2.x/3.x/4.x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1
7.1.1 Fabric OS Upgrade Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1
7.1.2 Upgrade Guidelines from Fabric OS 4.0.x on a SilkWorm 12000 . . . . . . . . . . 7-2
7.2 Fabric OS Version 4.1 (or higher) Ultra High Availability (HA) . . . . . . . . . 7-3
7.2.1 Ultra HA Functionality Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3
7.2.2 HA Caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3
7.2.3 HA Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4
7.3 Support for Boot Over SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-5
7.4 Compact Flash Space Monitoring and Management. . . . . . . . . . . . . . . . . . 7-5
vi Brocade SAN Design, Deployment, and Management Guide

7.5 Guidelines for Command Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-6
7.5.1 Pathinfo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-6
7.5.2 Wwncardshow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-8
7.5.3 Cfgtransabort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9
7.5.4 Sfpshow with -all Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9
7.5.5 Quietmode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-10
7.5.6 Nscamshow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-10
7.5.7 Portloginshow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-11
7.5.8 PortCamShow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-12
7.5.9 Portzoneshow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-13
7.5.10 Nodefind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14
7.5.11 Nsaliasshow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14
7.5.12 Cfgactvshow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-15
7.5.13 NsZoneMember . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-15
7.5.14 Port Swapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-16
7.5.15 Switch Rebooting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-18
7.6 Using Fabric OS Troubleshooting Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-20
7.6.1 Supportshow Command Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-20
7.6.2 Track Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-22
7.6.3 Persistently Disabling a Switch or Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-23
7.6.4 FDMIshow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-26
Brocade SAN Design, Deployment, and Management Guide vii

Section III SAN Management
Chapter 8 Introduction to SAN Management
8.1 SAN Management Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1
8.2 SAN Management Scope. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2
8.3 Brocade SAN Management Tools & Interfaces. . . . . . . . . . . . . . . . . . . . . . . 8-3
8.3.1 Brocade Fabric Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
8.3.2 Brocade Advanced Web Tools (HTTP). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
8.3.3 Command Line (Telnet). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
8.3.4 SNMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
8.3.5 Brocade API Scripting Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
8.4 Tool Selection Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5
Chapter 9 Brocade Fabric Manager 4.1.x

9.1 Key Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-1
9.2 Fabric Manager 4.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-3
9.3 Installation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-4
9.4 Switch Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-6
9.4.1 Configuration Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-6
9.5 Fabric Manager Navigation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-7
9.5.1 SAN Element Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-8
9.5.2 Information View Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-8
9.5.3 Customizing View Window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-9
viii Brocade SAN Design, Deployment, and Management Guide

9.6 Fabric Manager 4.1.x Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-9
9.6.1 SAN Discovery (Intelligent Baseline Profiling) . . . . . . . . . . . . . . . . . . . . . . . . 9-9
9.6.2 SAN Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-10
9.6.3 SAN Element Logical Grouping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-12
9.6.4 Fabric Login (Global Access Setup) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-16
9.6.5 Firmware Download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-18
9.6.6 Sequence Switch Reboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-21
9.6.7 Firmware Download to FDMI Capable Host Bus Adapter . . . . . . . . . . . . . . . . 9-24
9.6.8 Fabric Merge Check. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-24
9.6.9 Complete Fabric Backup and Compare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-26
9.6.10 License Key Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-31
9.6.11 Synchronize Fabric-Wide Time and Date . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-35
9.6.12 Fabric-Wide Event Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-35
9.6.13 Inter-Switch Link (ISL) Checking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-36
9.6.14 Manually Refreshing the Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-36
9.6.15 Call Home Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-36
9.6.16 Secure Fabric OS Policy Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-38
9.7 Switch and Port Level Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-46
Chapter 10 Brocade Advanced Web Tools Overview

10.1 Launching Web Tools from a Workstation . . . . . . . . . . . . . . . . . . . . . . . . . 10-2
10.2 Web Tools Access via Fabric Manager Client. . . . . . . . . . . . . . . . . . . . . . . 10-3
10.3 Web Tools in Secure Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-5
10.4 Brocade Advanced Web Tools Features . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-6
10.4.1 Fabric-Wide Zone Administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-6
10.4.2 Accessing Zone Administration Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-8
10.4.3 Zoning Configuration Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-9
10.5 Guidelines For Using Web Tools with Versions of Web Tools Prior to Fabric
OS 2.6.2/3.1.2/4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-10
Brocade SAN Design, Deployment, and Management Guide ix

10.6 Switch/Port Level Management Functions . . . . . . . . . . . . . . . . . . . . . . . . . 10-11
10.6.1 Switch Information Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-11
10.6.2 Network Config Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-13
10.6.3 Upload/Download Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-13
10.6.4 SNMP Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-14
10.6.5 License Admin Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-14
10.6.6 Configure Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-14
10.6.7 Routing Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-14
10.7 Port Management Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-15
10.7.1 Port Setting Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-15
10.7.2 Extended Fabrics Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-17
10.7.3 Trunk Information Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-17
Chapter 11 Command Line Interface (Telnet) Overview

11.1 Ethernet IP Address Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-1
11.2 SwitchStatusPolicySet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-2
11.3 Configuring Trunking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-3
11.4 SNMP Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-4
11.5 Fabric Watch Initialization (fwhelp) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-4
11.6 Port Swapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-4
11.7 Security Enable/Disable (sechelp) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-5
11.8 Command Sets (Reference Use) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-7
11.8.1 Security Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-7
11.8.2 Diagnostic (diaghelp). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-8
Chapter 12 SNMP Overview

12.1 Configuring and Reviewing SNMP Settings . . . . . . . . . . . . . . . . . . . . . . . . 12-3
12.1.1 agtcfgset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-3
12.1.2 Specifying Trap Level (swEventTrapLevel) * . . . . . . . . . . . . . . . . . . . . . . . . 12-4
12.2 Enabling Authentication on Community Names Verification (authTraps) 12-
5
12.3 SNMP Access Control List (ACL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-5
x Brocade SAN Design, Deployment, and Management Guide

12.4 SNMP MIB Support Set - Executing snmpMibCapSet Command . . . . . 12-6
12.5 Track Changes Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-7
12.5.1 Viewing Default Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-7
12.5.2 Resetting Agent Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-7
12.5.3 SNMP Setup in Web Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-7
12.5.4 SNMP MIBs/Trap Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-8
12.5.5 Trap Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-10
12.6 SNMP in Secure Fabric Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-11
12.6.1 Community Strings Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-11
12.6.2 SNMP MAC Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-11
Chapter 13 Brocade Fabric Watch Overview

13.1 Fabric Watch Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-2
13.2 Fabric Watch Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-3
13.3 Fabric Watch Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-4
13.3.1 Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-4
13.3.2 Traits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-5
13.3.3 Behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-5
13.3.4 Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-5
13.3.5 Alarm Notification Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-6
13.3.6 Alarm Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-7
13.4 Theory of Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-8
13.4.1 Counter Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-8
13.4.2 Time Base Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-8
13.5 Defining Threshold Boundary Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-11
13.5.1 Defining the Threshold for the Above Trigger . . . . . . . . . . . . . . . . . . . . . . . . 13-11
13.5.2 Defining Threshold for Below Trigger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-11
13.5.3 Defining Threshold for Changed Trigger . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-12
13.5.4 Defining Threshold for Exceeded Trigger. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-13
13.5.5 Defining Threshold for In-between Trigger . . . . . . . . . . . . . . . . . . . . . . . . . . 13-13
Brocade SAN Design, Deployment, and Management Guide xi

13.6 Advanced Thresholds Configuration (CLI) . . . . . . . . . . . . . . . . . . . . . . . . . 13-14
13.6.1 Behavior Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-15
13.6.2 Threshold Boundary Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-15
13.6.3 Threshold Alarm Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-15
13.6.4 Advanced Configuration Command Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-16
13.7 Advanced Threshold Configuration Using Brocade Web Tools . . . . . . 13-17
13.8 Fabric Watch Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-18
13.8.1 Class Based Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-18
13.9 Default and Custom Configuration Tables. . . . . . . . . . . . . . . . . . . . . . . . . . 13-23
Chapter 14 Advanced Performance Monitor (APM) Overview

14.1 Performance Monitoring via Web Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-2
14.1.1 Performance Graph Canvas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-3
14.1.2 Advanced Performance Monitor Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-3
14.1.3 End-to End Monitor Setup Using Web Tools . . . . . . . . . . . . . . . . . . . . . . . . . 14-5
14.1.4 Filter-based Performance Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-6
14.1.5 Masking Setup for End-to-End Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-6
14.2 Fabric Watch Supervised Advanced Performance Monitoring Features 14-7
14.2.1 Performance Class Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-7
14.2.2 AL_PA Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-7
14.2.3 Filter-Based Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-7
Chapter 15 Brocade API Scripting Toolkit Overview

15.1 API Scripting Interface Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-2
15.2 Basic Communication Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-3
15.3 Basic Communication Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-4
15.3.1 Session Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-4
15.3.2 Platform Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-4
15.4 Fabric Access Supported Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-5
xii Brocade SAN Design, Deployment, and Management Guide

Section IV Appendices
Appendix A Cabling Design and Management
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1
A.2 Needs Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1
A.2.1 Existing Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2
A.2.2 Site Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3
A.3 Design Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-5
A.3.1 Site Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-5
A.3.2 Inter-Site Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-6
A.3.3 Patch Panel Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-6
A.3.4 Patch Panel Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9
A.3.5 Cable Management Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-10
A.4 Cable Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-12
A.4.1 Typical Cable Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-12
A.4.2 Planning Cable Layouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-16
A.4.3 Cabling Standards within a single Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-17
A.4.4 Cabling Standards Between Racks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-18
A.4.5 Cabling Standards to Patch Panels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-18
A.5 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19
A.6 Cable Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19
A.6.1 Cable Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19
A.6.2 Cabling Best practices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-23
A.6.3 Cable Hazards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-23
A.7 Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-24
Appendix B Long-distance Technologies for

Storage Area Networks
B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1
Brocade SAN Design, Deployment, and Management Guide xiii

B.2 SilkWorm Product Family Configuration and Functionality . . . . . . . . . . . B-1
B.2.1 Brocade Extended Fabrics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1
B.2.2 Performance Over Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-3
B.2.3 ISL R_Rdy Mode and Remote Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-4
B.3 Fiber Optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-5
B.3.1 Optical Fiber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-5
B.3.2 Optical Transceivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-6
B.3.3 Fiber Loss and Link Budgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-6
B.3.4 Diagnostics and Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-7
B.4 Wave Division Multiplexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-8
B.4.1 Optical Components of WDM Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-8
B.4.2 Topologies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-9
B.4.3 Switch Configuration and Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-9
B.5 Multiprotocol Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-10
B.5.1 Switch Configuration and Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-10
B.6 SAN Distance Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-11
B.6.1 Remote Data Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-11
B.6.2 Centralized Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-12
B.6.3 Storage Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-13
Appendix C Glossary
C.1 Terms and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1
Appendix D Site Survey Templates
Index
xiv Brocade SAN Design, Deployment, and Management Guide

Preface
The SilkWorm Design, Deployment, and Management (DDM) Guide is focused on covering detailed “how to” information for
Brocade products from a design, deployment, and management perspective. The DDM is intended to be used in conjunction
with existing Brocade manuals, release notes, and related Brocade publications. The DDM is effective at focusing experienced
Brocade SAN professionals on a specific area or subject of the SAN life cycle. The emphasis of the DDM is on breadth, with
depth where new features or complexity dictate. When multiple choices are available for adopting a particular approach or
strategy and there are clear benefits to implementing a particular approach, guidelines are provided.
The SilkWorm 2000 series, 3200, 3250, 3800, 3850, 3900, 12000, and 24000 platforms and features up to Fabric OS 4.2, 3.1.2,
and 2.6.2 are covered in this document. The following Brocade software features are addressed:
• Advanced Web Tools
• Advanced Zoning
• Fabric Watch
• Trunking
• Advanced Performance Monitoring
• Extended Fabrics
• Secure Fabric OS
• Fabric Manager
• API
The flow and organization of the document follows the process of first designing a Brocade SAN, followed by the deployment
and operation of that SAN. It is important to understand how new features such as Secure Fabric OS, non-disruptive code
activation, and scalability impact a SAN and how these topics relate across the SilkWorm and Fabric OS family. Many
Brocade features span the disciplines of SAN design, deployment, and management. For example, ISL Trunking influences a
SAN design, has specific deployment tips, and can be managed via various interfaces such as the CLI, Web Tools, and Fabric
Manager.
Discussed in Section I SAN Design, are topics such as device attachment strategies, switch placement in a fabric, design
related Security topics, and zoning guidelines. Once the SAN design and other planning has taken place, the switches and
devices require deployment. Section II SAN Deployment, covers subjects such as installation preparation and planning, usage
of new features, Security planning, migrating to a secure fabric, staging equipment, validating a fabric, and fabric
troubleshooting. A rich set of management interfaces exists for the SilkWorm family of switches. Effectively integrating a
particular management interface, such as Fabric Manager or Fabric Watch, into the enterprise management system, capacity
planning, and SAN management with SNMP are just a few examples of topics addressed in Section III SAN Management.
Note: FICON design requirements and practices differ significantly from those used for open systems Fibre Channel SANs.
FICON considerations are addressed in the IBM Redbook titled “Getting Started with the IBM 2109 M12 FICON
Director” (ISBN 0738499390).
Brocade SAN Design, Deployment, and Management Guide xvii

Audience
The DDM is targeted for use by storage administrators, SAN administrators, system administrators, SAN architects, systems
engineers, and SAN operators that are involved with the design, deployment, and management of SANs. The DDM is an
advanced document and is very concise. Background information and supporting information for a particular topic are kept to
a minimum and as appropriate, the reader is referred to supporting documentation. The reader is expected to have working
experience with Brocade products. General computer system level troubleshooting skills are always important when
configuring sophisticated enterprise solutions. System administration or storage administration experience is also helpful in
comprehending this document.
Guidelines are provided throughout the document. Guidelines are recommendations for consideration. The adoption of these
guidelines is a function of the user’s ability to interpret and correlate relevant SAN information and make decisions based
upon their operational policies and specific SAN requirements.
Guideline Conventions
The formatting and conventions used in this document are designed to help the reader locate and comprehend information
quickly. In addition to the information provided in standard text, there are Guidelines, Notes, and Cautions to help focus the
reader on important information.
Formatting
The following table describes the formatting conventions that are used in this book:
Convention Purpose
bold text • identifies GUI elements

• identifies keywords/operands
• identifies menu selections at the GUI or CLI
italic text • provides emphasis
• identifies variables
• identifies paths and internet addresses
• identifies book titles and cross references
code text • identifies commands in line with text
• identifies CLI output
• identifies syntax examples
xviii Brocade SAN Design, Deployment, and Management Guide

Notes and Guidelines
Note: Notes emphasize important information.
Guideline: Guidelines are recommendations for consideration. The adoption of these guidelines is a function of the
user’s ability to interpret and correlate relevant SAN information and make decisions based upon their
organization and SAN requirements.
Warning: Warnings alert you to potential damage to hardware, firmware, software, or data.
Caution: Cautions indicate that a particular action or type of connection is not recommended as it may cause failure of
the switch or fabric.
Brocade SAN Design, Deployment, and Management Guide xix

Reference Documentation
Brocade Documentation
The following related publications are provided on the Brocade Documentation CD-ROM and on the BrocadeConnect web
site. To access the BrocadeConnect web site go to www.brocade.com and select the appropriate link.
• Brocade Fabric OS Documentation
- Brocade Fabric OS Procedures Guide
- Brocade Fabric OS Reference
- Brocade Diagnostic and System Error Message Reference Manual
- Brocade Fabric Manager User’s Guide
- Brocade MIB Reference Manual
• Brocade Fabric OS Optional Features Documentation
- Brocade Advanced Web Tools Administrator's Guide
- Brocade Fabric OS Features Guide
- Brocade Fabric Watch User’s Guide
- Brocade Secure Fabric OS® User's Guide
- Secure Fabric OS Quickstart Guide
• Brocade Hardware Documentation
- Brocade SilkWorm 24000 Hardware Reference
- Brocade SilkWorm 3250/3850 Hardware Reference
Additional Resource Information

The following related publications are provided on the Brocade Partner web site and are an excellent resource for additional
information.
• Brocade Fabric Watch Guidelines (part number: 53-0000434-01)
• Brocade Guide to Understanding Zoning (53-0000213-01)
• Cable Management Guidelines (53-0000824-01)
• LAN Guidelines For Brocade SilkWorm Switches (publication number: 53-0000350-01)
• SAN Migration Guide (publication number: 53-0000360-02)
• SAN Security: A Best Practices Guide (publication number: GA-RG-250-00)
• Designing Next-Generation Fabrics With Brocade Switches (whitepaper http://www.brocade.com)
• Exploring Brocade ISL Trunking (publication number: 53-0000263-01)
• Zoning Implementation Strategies For Brocade San Fabrics (whitepaper http://www.brocade.com)
xx Brocade SAN Design, Deployment, and Management Guide

Chapter
Product Introduction
1
The SilkWorm Design, Deployment and Management (DDM) Guide provides detailed information to help better utilize
Brocade SilkWorm switches, Brocade Fabric OS features and Brocade applications. Brocade SilkWorm switches are available
in models ranging from 8-port switches to high port count Directors, all of which provide many licensable features to help
better utilize your Brocade SAN fabric investment.
1.1. Brocade Fibre Channel Switches

Brocade offers Fibre Channel switches ranging from eight ports to as many as 128-ports. These switches can be interconnected
to form a fabric able to support your port count and availability requirements. For platform specific information about
incorporating any Brocade products into your SAN design, refer to Section I SAN Design.
Table 1-1 Switch Product Summary

SilkWorm Switch Fabric OS Port Count Number of Power 1G/2G
Model Supplies Capable
(Fixed/Removable)
SilkWorm 24000 Fabric OS 4.2 up to 128 ports 2 (Removable) 2 Gbit/sec
SilkWorm 12000 Fabric OS 4.2 up to 64 ports per logical 4 (Removable) 2 Gbit/sec
switch (128 per chassis)
SilkWorm 3850 Fabric OS 4.2 16 ports 2 (Fixed) 2 Gbit/sec
SilkWorm 3250 Fabric OS 4.2 8 ports 1 (Fixed) 2 Gbit/sec
SilkWorm 3900 Fabric OS 4.2 32 ports 2 (Removable) 2 Gbit/sec
SilkWorm 3800 Fabric OS 3.1.2 16 ports 2 (Removable) 2 Gbit/sec
SilkWorm 3200 Fabric OS 3.1.2 8 ports 2 (Removable) 2 Gbit/sec
SilkWorm 2000 Series (End of Life)
SilkWorm 2x00 Family Fabric OS 2.6.2 8 ports (SilkWorm 2400 2 (Removable) 1 Gbit/sec
and 2100)
16 ports (SilkWorm 2800)
SilkWorm 2xx0 Family Fabric OS 2.6.2 8-16 ports 1 (Fixed) 1 Gbit/sec
Brocade SAN Design, Deployment, and Management Guide 1-1

1 Product Introduction
1.1.1. Brocade SilkWorm 24000

The SilkWorm 24000 is an enterprise-level Fibre Channel gigabit director which operates on Brocade Fabric OS 4.2 and later.
It provides director-class availability and link speeds up to 2 Gbit/sec. Possible single domain configurations range from a
minimum of 32-ports to a maximum of 128-ports in a single 14U enclosure. The SilkWorm 24000 employs Brocade
Inter-Switch Link (ISL) Trunking to provide a high-speed data path between switches. The SilkWorm 24000 provides a
complete solution for a departmental SAN fabric or as a director of an enterprise core-to-edge SAN.
The SilkWorm 24000 meets mission critical availability requirements with network resiliency, automatic data path routing,
seamless loading of switch software, and redundant hot Pluggable components. With Inter-Switch Link (ISL) Trunking, a
high-speed data path is provided between switches within a fabric that can scale up to 8 Gbit/sec per trunk. This is done
non-disruptively by adding connections to an existing trunk group.

The SilkWorm 12000 has two possible two-domain configurations ranging from minimum of 32-ports to a maximum of
64-ports per domain in a single 14U enclosure. The SilkWorm 12000 supports up to 128-ports per chassis. The SilkWorm
12000 includes the Brocade Fabric Operating System™ version 4.x, and is compatible with the entire Brocade SilkWorm
product family. The SilkWorm 12000 provides a complete solution for a departmental SAN fabric or as a core switch of an
enterprise core-to-edge SAN.

With 32-ports in a 1.5 U chassis, the SilkWorm 3900 supports link speeds up to 2 Gbit/sec. The SilkWorm 3900 includes the
Brocade Fabric Operating System™ version 4.x, and is compatible with the entire Brocade SilkWorm product family. The
SilkWorm 3900 provides a complete solution for a departmental SAN fabric as a a small core, or an edge switch extending the
reach of an enterprise core-to-edge SAN built around Brocade SilkWorm 24000 core Fabric switches.
The SilkWorm 3900 meets mission critical availability requirements with network resiliency, automatic data path routing
seamless loading of switch software, and redundant hot Pluggable components. With Inter-Switch Link (ISL) Trunking to
provide a high-speed data path between switches, performance can be increased up to 8 Gbit/sec per trunk.
1.1.4. Brocade SilkWorm 3250/3850

Both the SilkWorm 3250 (an 8-port switch) and SilkWorm 3850 (a 16-port switch) are new to the line of Brocade SilkWorm
switches. Both Fibre Channel switches support link speeds up to 2 Gbit/sec. The SilkWorm 3250 and SilkWorm 3850 utilize
Brocade Fabric Operating System™ version 4.2, and are compatible with the entire Brocade SilkWorm product family. The
SilkWorm 3850 features dual power supplies and no Field Replaceable Units (FRUs). The SilkWorm 3250 features a single
power supply and no Field Replaceable Units (FRUs).
1.1.5. Brocade SilkWorm 3200/3800

The SilkWorm 3200 is an 8-port switch; the SilkWorm 3800 is a 16-port switch. Both Fibre Channel switches support link
speeds up to 2 Gbit/sec. The SilkWorm 3200 and SilkWorm 3800 include the Brocade Fabric Operating System™ version 3.x,
and are compatible with the entire Brocade SilkWorm product family.
1-2 Brocade SAN Design, Deployment, and Management Guide

Product Introduction 1
1.2. Fabric Operating System (FOS)

For information and considerations about incorporating Brocade Fabric OS into your SAN, refer to Section II SAN
Deployment and Section III SAN Management.
1.2.1. Brocade Fabric OS 4.2

Brocade Fabric OS is an operating system that provides the core infrastructure growing businesses need to deploy scalable and
robust Storage Area Networks (SANs). Fabric OS 4.2 runs on the following SilkWorm switches.
• SilkWorm 24000
• SilkWorm 12000
• SilkWorm 3900
• SilkWorm 3850
• SilkWorm 3250
Fabric OS 4.2 is the common base OS for the switches listed above and for future products. It supports scalable SAN fabrics
that interconnect hundreds of devices while ensuring high-performance data transfer among connected resources and servers.
Fabric OS easily manages both large switch fabrics and Fibre Channel Arbitrated Loop (FC-AL) configurations.
1.2.2. Brocade Fabric OS 3.1.2

Brocade Fabric OS 3.1.+2 provides the core functionality for the SilkWorm 3800. It supports connectivity to all Brocade
switches, including switches running Brocade Fabric OS 2.6.2 and 4.2.
1.2.3. Brocade Fabric OS 2.6.2

Brocade Fabric OS version 2.6.2 provides the core functionality for the SilkWorm 2000 family of Fibre Channel switches.
Fabric OS 2.6.2 runs only the SilkWorm 2000 family of Fibre Channel switches. It supports connectivity to all Brocade
switches, including switches running Brocade Fabric OS 3.1.2 and 4.2.
1.3. Fabric OS Licensed Features

For information and considerations about incorporating Fabric OS licensed features into your SAN, refer to Section II SAN
Deployment and Section III SAN Management.
Note: The following product discussion does not include bundled options. Licensed features may be bundled, however when
the command licenseshow is issued each license will be listed individually.
1.3.1. Brocade Advanced Web Tools

Brocade Advanced Web Tools provides a graphical interface (GUI) that allows the SAN administrator to monitor and manage
entire fabrics and individual switches and ports from a standard workstation. Web Tools runs on Fabric OS. All switches in the
fabric are displayed in the main Web Tools window.
Guidelines on management activities utilizing Web Tools can be found in Chapter 10, Brocade Advanced Web Tools
Overview. For more information about Brocade Web Tools, please refer to the Brocade Advanced Web Tools User's Guide.

1.3.2. Brocade Advanced ZONING

Brocade Advanced ZONING is a feature allowing an administrator to create segmentation or zones within a Brocade Fabric,
comprised of selected storage, servers, or even workstations. It also enforces access of information to only the devices in the
defined zone. Zoning ensures security and enables optimization of IT resources in response to user demand:
• Customize Environments. You can use zones to create logical subsets of the fabric to accommodate environments such
as closed user groups or functional areas within the fabric. For example, you can identify selected devices within a zone
for the exclusive use of zone members, or you can define a zone to create separate test or maintenance areas within the
fabric.
• Optimize Resources. You can use zones to logically consolidate equipment for efficiency or to facilitate time-sensitive
functions: for example, to create a temporary zone to back up nonmember devices.
A zone is a specified group of fabric-connected devices, also called zone objects, that have access to one another. Zone objects
are grouped into zones, and zones are grouped into a zone configuration. Zones can overlap; that is, a zone object can belong
to more than one zone and a fabric can have multiple zones. A switch can have any number of defined resident zone
configurations; however, only one configuration can be enabled at a time. Figure 1-1 illustrates a fabric with three zones.
Any zone object connected to the fabric can be included in one or more zones. For example, in Figure 1-1, the RAID device is
included in the Blue zone and the Green zone.
JBOD
Loop 2
Server2
Blue zone
Fib
bre Channe
el Fabric
RAID
Hub
Server1 Loop 1 Server3
Red zone Green zone
Figure 1-1 Fabric with Three Zones
For more information about Brocade Advanced Zoning refer to the Brocade Fabric OS Features Guide. Guidelines on
management activities can be found in Section III SAN Management.
1.3.3. Brocade ISL Trunking

Brocade Inter-switch Link (ISL) Trunking is an optionally licensed product available on all SilkWorm switches running Fabric
OS 3.1/4.1 or greater, on a per-switch basis. It optimizes network performance by forming trunking groups that can distribute
traffic across the shared bandwidth of all the ISLs (interswitch links). A trunk group is a group of up to four ISLs logically
merged together into a single link, delivering ISL throughput of up to 8 Gbit/sec. This enables traffic to be routed through any
available ISL in the group rather than being restricted to a specific, potentially congested ISL. ISL Trunking distributes traffic
dynamically across the trunk at the Fibre Channel frame level while preserving in-order delivery.

Brocade ISL Trunking simplifies network design and reduces the cost of storage management by optimizing bandwidth
utilization and enabling load balancing of traffic at the frame-level. ISL Trunking is compatible with both short wavelength
(SWL) and long wavelength (LWL) fiber optic cables and transceivers.
The ISL Trunking software identifies and constructs trunking groups as soon as the ISL Trunking license is activated. The
ISLs and ports that participate in trunking groups are referred to as trunks and trunking ports
Figure 1-2 illustrates how trunking can result in more throughput by avoiding congestion. In this example, the data available
for transmission is distributed over the four ISLs with no congestion, since it is below the total 8 Gbit/sec capacity of the
combined ISLs. In a fabric that does not have trunking capability, some paths could be congested and other paths could be
under utilized.
Figure 1-2 Distribution of Traffic Over an ISL Trunking Group
Guidelines on management activities can be found in Section III SAN Management. For more information about Brocade
Trunking, refer to the Brocade Fabric OS Features Guide.
1.3.4. Brocade Secure Fabric OS

As organizations grow their Storage Area Networks (SANs) and interconnect them over longer distances through existing
networks, they have a greater need to effectively manage SAN security and policy requirements. Secure Fabric OS is a
comprehensive security solution that is flexible and configurable to meet the specific policy requirements of your
organization. Secure Fabric OS works with Brocade Advanced Zoning to provide a secure environment utilizing your current
zoning policies.
The Deployment section of the DDM has an in-depth discussion of these features and best practices and guidelines as to how
to utilize them for the maximum benefit in your fabric.
Secure Fabric OS includes the following features:
• Fabric Configuration Servers
• Management Access Control
• Device Connect Controls
• Switch Connect Controls
• Secure Management Communications
Guidelines on management activities can be found in Section III SAN Management.
For more information about Brocade Secure Fabric OS, please refer to the Brocade Secure Fabric OS User's Guide.

1.3.5. Brocade Fabric Watch

Brocade Fabric Watch, an optionally licensed product, monitors the health status of fabric elements by continuously ensuring
that they are operating within specified thresholds. When a configured alarm condition is detected then an appropriate message
is forwarded to user by one or more pre-selected method. The severity state appears in the message to indicate the urgency of
the event to assist the administrator to take appropriate action. The Fabric Watch accomplishes this in three steps:
1. Measuring values
2. Comparing against threshold boundary limits
3. Event generation and notification
Fabric Watch, please refer to the Brocade Fabric Watch User's Guide.
1.3.6. Brocade Advanced Performance Monitoring

Brocade Advanced Performance Monitoring is an optionally licensed product used for monitoring the performance of
networked storage resources. It helps reduce over-provisioning while enabling SAN performance tuning and increasing
administrator productivity. This feature provides SAN performance monitoring through an end-to-end monitoring system that
provides
• Increased end-to-end visibility into the fabric.
• Accurate reporting for service-level agreements and charged access applications.
• Shortened troubleshooting time.
• Better capacity planning.
• Increased productivity via pre-formatted and customized screens and reports.
Advanced Performance Monitoring is administered through either command line interface (CLI) or Brocade Advanced Web
Tools. To use Advanced Web Tools, an Advanced Web Tools license must be activated on the switch.
Advanced Performance Monitoring, refer to the Brocade Fabric OS Features Guide.
1.3.6.1. Types of Performance Monitoring

Using Advanced Performance Monitoring, you can track the following:
• Number of CRC errors for AL_PA devices
• Number of words received and transmitted in Fibre Channel frames with a defined SID/DID pair
• Number of frames with CRC errors received at the port with a defined SID/DID pair
• Number of times a particular filter pattern in a frame is transmitted by a port
Through the use of Telnet commands and Web Tools, the following types of performance monitoring are available:
• AL_PA monitoring
• End-to-end monitoring
• Filter-based monitoring
1.3.7. Extended Fabrics

Brocade Extended Fabrics is an optionally licensed product that enables interswitch links (ISLs) to operate at full bandwidth
up to 100 Km. This is achieved by optimizing the internal buffering algorithm used by SilkWorm switches. It provides

maximum buffering between E_Ports connected over an extended distance. The buffer reconfiguration results in line speed
performance of 2 Gbit/sec for switches interconnected at up to 60 Km, and of 1 Gbit/sec for switches interconnected at up to
100km.
Brocade Extended Fabrics achieves long-distance connections by allocating more frame buffers for Fibre Channel traffic.
Long-distance connections require more frame buffers than regular ISL connections. The greater the distance, the more frame
buffers required. If the long-distance port is part of a quad, this limits the buffer space left over for the remaining ports in the
quad, which must, therefore, be configured appropriately.
When configuring long distance ISLs, make sure to balance the need between long distance ISL connections and Core-to-Edge
ISL connections within a switch. Configuring long distance ISLs between Core and Edge switches is possible, but is not a
recommended practice.
The long-distance Extended Fabrics configuration can be established among SilkWorm 3200, 3250, 3800, 3850, 3900, 12000,
and 24000 switches. Long-distance ports consume more buffers than regular ISL ports, which means that configuration of a
long-distance port could disable other ports in the same quad due to lack of buffers.
Valid Long-Distance
ISL Connections
SW3850 SW3250
SW2800 SW3900 SW24000 SW2800
SW3800 SW12000
Invalid Long-Distance
ISL Connection
SW2800 SW3000-series
ISL Connection
SW2800 SW12000
ISL Connection
SW2800 SW24000
Figure 1-3 Extended Fabric Configuration

Extended Fabrics, please refer to the Brocade Fabric OS Features Guide.

1.3.8. Remote Switch

Remote Switch is an optionally licensed product that enables you to connect two remote Brocade fabrics over an IP network,
enabling communication of IP or ATM protocols as well as Fibre Channel traffic.
The Brocade Remote Switch feature functions with the aid of a “bridging device” or Fibre Channel gateway. The gateway
supports both a Fibre Channel physical interface and a secondary, non-Fibre Channel physical interface, such as IP, SONET, or
ATM. Remote Switch functions over E_Port connections. With Remote Switch on both fabrics, the gateway accepts Fibre
Channel frames from one fabric, tunnels them across the network, and passes them to the other fabric. From the viewpoint of
the connected hosts and storage devices, fabrics using Remote Switch interact the same as locally connected switches.
Remote Switch provides many of the same capabilities of normal ISL links including:
• Coordinated fabric services
The Remote Switch fabric configuration fully supports all fabric services, including Distributed Name Service, Registered
State Change Notification, and Alias Service.
• Distributed management
Management tools such as Advanced Web Tools, Fabric OS, and SNMP are available from both the local switch and the
remote switch. Switch management is routed through the Fibre Channel connection; thus, no additional network
connection is required between sites.
• Support for interswitch links (ISLs)
Sites requiring redundant configurations can connect multiple E_Ports to remote sites by using multiple gateways.
Standard Fabric OS routing facilities automatically maximize throughput and provide automatic failover during
interruption on the WAN connection.
The Remote Switch feature operates in conjunction with a gateway. The gateway provides an E_Port interface that links to the
SilkWorm E_Port. After the link between the two E_Ports has been negotiated, the gateway E_Port moves to “pass through”
mode and passes Fibre Channel traffic from the SilkWorm E_Port to the WAN. The gateway accepts Fibre Channel frames
from one side of a Remote Switch fabric, transfers them across a WAN, and passes them to the other side of the Remote
Switch fabric.
Guidelines on management activities can be found in Appendix B, Long-distance Technologies for Storage Area Networks.
For more information about Brocade Extended Fabrics, please refer to the Brocade Fabric OS Features Guide.
1.3.9. Brocade Support for FICON®

Brocade Support for FICON® available in the Brocade Fabric OS. IBM Fibre Connections (FICON®) is an industry-standard,
high-speed input/output (I/O) interface for mainframe connections to storage devices.
FICON® configurations are evolving so that a single switch can run FICON® as well as Fibre Channel technology. This type
of configuration is called intermix mode. Support for the operation and management of switches running in intermix mode is
available in Fabric OS v4.1.2 and later.
For further information about specific FICON® implementation, refer to the IBM Redbook, FICON® Native Implementation
and Reference Guide. Refer to the Brocade Fabric OS Features Guide for more information about Brocade support of
FICON®.

1.4. Brocade Fabric Manager

Brocade Fabric Manager is a java-based management application that provides a central point of control. In addition, Fabric
Manager is tightly coupled with Brocade Web Tools, Fabric Watch, and Advanced Performance Monitor. The GUI simplifies
task administration at the fabric, switch, and port levels in a medium-to-large size Brocade SAN environment.
Brocade Fabric Manager 4.1.x key features are briefly described below.
• SAN Discovery
• SAN Element Logical Grouping
• Firmware/Configuration Download
• Firmware Download to FDMI capable Host Bus Adapter
• Sequence Switch Reboot
• Fabric Merge Check
• Fabric Checking
• Complete Fabric Backup and Compare
• Event Monitoring
• Call Home support
• Secure Fabric OS Policy Management
• Switch/Port Administration
Fabric Manager, please refer to the Brocade Fabric Manager User’s Guide.


Section
SAN Design I
Brocade SAN Design, Deployment, and Management Guide

Chapter
SAN Design & Architecture Concepts
2
This chapter discusses fundamental SAN Design & Architecture concepts and associated guidelines for developing an
effective SAN design. A background on SAN design solutions, Trunking, locality, availability, performance, and topologies is
provided as a foundation for the more advanced topics discussed in Chapter 3, Architecting SANs With SilkWorm Switches.
2.1. SAN Solutions

While many SAN users begin their SAN experience with one particular SAN solution, the SAN quickly becomes the basis for
many other applications. For example, a company may start out with SAN-based backup and quickly integrate storage
consolidation and clustering into the existing SAN foundation. In that respect, a SAN decision is a strategic one, and should
receive an appropriate level of attention. The adoption of SANs is being driven by a variety of objectives. Some examples are:
• The need for more efficient usage of enterprise storage arrays
• Decreasing size of backup/restore windows
• Increasing size of data set to be backed up
• The need for improved high availability and disaster tolerance
• The need to enhance storage resource management
• Decreased total cost of ownership for storage
Four popular SAN solution categories are Storage Consolidation, LAN-Free Backup, High Availability, and Business
Continuance. Each of these SAN solutions are generically described in the following sections and key attributes of each are
discussed in terms of their affect on SAN design.
2.1.1. Storage Consolidation

Storage consolidation is a way of optimizing storage resource utilization. It is often the result of migrating Directly Attached
Storage (DAS) and hosts to a SAN environment. In a SAN, it is no longer necessary to have a one-to-one correspondence
between a host port and a storage port. Instead, many hosts can share a single storage port, and a single host port can access
many storage devices. This immediately reduces cost on hosts because fewer HBAs are needed, and on storage because fewer
controllers are needed. In addition, savings can be accrued by reducing storage management, power, cooling, and, floor space
costs. Savings also come from improved utilization of free space on enterprise storage subsystems. With the lowering cost of
Fibre Channel HBAs and switch infrastructure, the storage consolidation value proposition has never been better.
For example, assume that 20 hosts each connect to 100 GB of storage in a direct attach environment, requiring a total 2000 GB
of storage. Some space on each system is free. This is known as white space, or headroom. The average utilization of this
directly attached storage (DAS) is 50%, leaving 50% white space. The total storage utilized is 1000 GB, which leaves 1000
GB of white space.

2 SAN Design & Architecture Concepts
With the use of a SAN, it is possible to achieve much higher utilization since every host can access to all storage in the SAN.
In this example, a modest 10-20% improvement in storage utilization could result in a savings of several hundred GB of
storage. In addition, a reduction in associated ownership costs of that surplus storage would occur. In the storage consolidation
model, if a host is not using all of its storage, it is possible to rapidly reallocate this extra storage to a different host. It is also
possible to add additional storage for all servers to access, rather than having to purchase storage for specific hosts. In a direct
attach environment, it is more difficult to do so, forcing the need to have very high white space overhead to allow growth. A
conservative 60% utilization scenario with DAS and storage consolidation environments are compared in Figure 2-1.
Storage Host
80% utilization
60% utilization
SAN
Figure 2-1 Direct Attach / Storage Consolidation Comparison
Since many hosts depend upon continuous access to their storage in a storage consolidation solution, designing a highly
available SAN to ensure this continuous access is critical. Resilient and redundant fabric designs are highly recommended,
especially in large storage consolidation solutions. In a storage consolidation solution, many devices contend for a shared
storage port. The performance-limiting factor is often the over-subscription or fan-out ratio of that port, and not the network.
Because of this, it is possible to design SANs with a certain amount of over-subscription without adversely affecting
application performance. Because the benefits of storage consolidation grow proportionally with the number of hosts and
storage, the capability for a SAN to scale is important. It is possible to choose a SAN architecture that can grow from tens of
ports to hundreds, and in some cases, thousands of ports, while minimizing or eliminating downtime. Topologies such as the
Core/Edge are optimal for enabling this type of scaling and availability.
2.1.2. Backup
A SAN-based backup is, in some respects, a form of storage consolidation in that an I/O device (the tape drive) is available to
be shared by many hosts. The difference is that the shared device is tape, rather than disk. This distinction can affect SAN
design in several ways:
• Currently, tape libraries tend to be single-attach, so the multi-pathing approaches used in storage consolidation will
usually not work.
• Backup devices tend to be more sensitive to I/O disruption than disk arrays. Arrays can recover from small glitches; tape
solutions sometimes do not recover as easily. This is a known issue in the industry and something being addressed with
the emergence and adoption of the FC-TAPE standard.
• The availability of tape drives is usually not as critical as that of disk arrays.

SAN Design & Architecture Concepts 2
• The performance requirements of tape drives typically involves the streaming of large blocks of I/O with bandwidth
requirements of 30-60 Mbyte/sec.
Non-SAN based backups take the form of direct attach tape drives, or backup over IP networks. IP backups contend with the
normal day-to-day traffic already on the Local Area Network (LAN). Using direct attach tape on each host is costly because of
the number of tape devices, tape management, and increased infrastructure cost for floor space, power, cooling, etc.
High-speed SAN-enabled backups reduce backup and restore windows and can enable disaster tolerance by locating libraries
at remote sites. SAN based backup improves on traditional backup by enabling the sharing of fewer, larger tape libraries and
by minimizing or eliminating the performance issues associated with traditional backup architectures, as depicted in
Figure 2-2.
Tape Host
Ethernet
or SAN
Figure 2-2 Direct Attach and LAN Based Backup Compared to a SAN Based Backup
A disruption in a backup SAN is usually not as critical as a disruption in a storage SAN. Mission critical applications require
continuous access to storage, while a tape backup normally can be restarted without end users seeing the effect. If a SAN is
designed for tape device attachment only, a single resilient fabric may provide sufficient availability.
2.1.3. High Availability / Clustering

High Availability (HA) clusters are used to support critical business applications. They provide a redundant, fail-safe
installation that can tolerate equipment, software, and/or network failures, and continue running with as little impact upon
business as possible. HA clusters have been in use for some time now. However, until the advent of Fibre Channel, they were
very limited in size and reliability. This is because clusters require shared storage, and sharing SCSI storage subsystems is
difficult and unreliable. In fact, sharing a SCSI device between more than two initiators can be difficult due to SCSI cabling
limitations, and the SCSI support-level for multiple initiators.
Clustering technology has therefore been greatly enhanced by the network architecture of SANs. SANs provide ease of
connectivity, and the ability to interconnect an arbitrarily large number of devices. SANs can support as few as two hosts in a
failover configuration, and can be expanded to support “many-to-one” configurations. The primary advantages that a SAN
affords a cluster are connectivity, scalability, and reliability.

2.1.4. Extended Distance Solutions

SANs enable the ability to replicate data over distances of hundreds of meters to thousands of kilometers. The data is
replicated via mirroring at the host level or at the storage level. Data can also be backed up to the remote site. Solutions can
involve disk mirroring by day and tape vaulting, over IP, by night. Should the primary site become disabled, the data located at
the remote site is accessible immediately so that critical applications can continue operation. Some business continuance
implementations deploy standby hosts that remain idle until needed. Other sites utilize the idle equipment for testing and
development purposes until that equipment is needed for production purposes. A key element to this strategy is having the
hosts boot from the SAN and the existence of a boot image of the primary site hosts located at the remote site.
Underlying connections to a remote site of up to 100 KM can be enabled by use of Fibre Channel technology such as long
wave length laser, extended long length lasers, or Wave Division Multiplexing (WDM) as shown in Figure 2-3. Use of Wide
Area Network technology such as IP (internet protocol) / Fibre Channel bridges or SONET / Fibre Channel bridges enables the
remote site to be separated from the primary site by thousands of kilometers.
Performance over long distance links can vary for multiple reasons. The number of buffer-to-buffer credits determines the
number of Fibre Channel frames that a switch can transmit on a link at one time before requiring an acknowledgement back
from the receiver. If there are not enough frames to fill the pipe, then performance degradation may result. Another
performance factor on long distance links is the response time for SCSI transactions. For more information about long distance
connectivity and solutions, refer to Appendix B, Long-distance Technologies for Storage Area Networks.
ELWL = Extended FC over SFPs (ELWL), 0 – 80 Km (1)

Long Wave Length Synchronous
FC over WDM
Applications
0 - 300 km (1)
Asynchronous
Translated FC over SONET, ATM or IP
Applications
0 – 20,000km
Figure 2-3 Business Continuance Distances and Technology

2.2. SAN Availability

A computer system or application is only as available as the weakest link. To build a highly available computer system it is not
sufficient to only have a highly available SAN. It is necessary to account for availability throughout the entire computer
system: dual HBAs, multi-pathing software, highly available and multi-ported storage subsystems, and clustering software are
some of the components that may make up such a system.
2.2.1. Fabric Resiliency

Many fabric topologies provide at least two internal fabric routes between all switches that comprise that fabric. These
topologies are considered resilient because each topology can withstand a switch or ISL failure while the remaining switches
and overall fabric remain operational. This self-healing capability is enabled by the Brocade-authored Fabric Shortest Path
First (FSPF) protocol. While originally a Brocade-only protocol, FSPF has been accepted by the standards bodies as the
standard protocol for Fibre Channel fabric routing.
Figure 2-4 depicts the failure of a switch in a Cascade topology, as an example of lack of resilience. Switches A and B are
unable to communicate with the remaining switches when the switch marked with the “X” fails, resulting in the fabric
segmenting into three separate fabrics. Figure 2-5 shows the resilience present in the resilient Ring and Core/Edge topologies.
If switch B fails, switch A can still communicate with switch C through the alternate path indicated by the arrows. The fail
over to alternate paths is effectively transparent to the attached devices. This fail over is performed by FSPF, which
automatically reroutes the data around the failure.
Figure 2-4 Lack of Resilience in a Cascade Topology
A
A
B X B X
C C
Figure 2-5 Resilience in Core/Edge and Ring Topologies

2.2.2. SAN Availability Classifications

Devices attached to a fabric may require highly reliable access to support applications such as storage consolidation, server
clustering, high availability, or business continuance operations. Both resilient and non-resilient dual fabrics can be referred to
as redundant fabric SANs. Redundant designs are always recommended for high availability systems, and any large SAN
deployment where downtime for the entire SAN could affect hundreds of servers. There are four primary categories of
availability in SAN architecture. In order of increasing availability, they are:
Single fabric, non-resilient: All switches are connected to form a single fabric, which contains at least one single point of
failure. The Cascade topology is an example of this category of SAN (see Figure 2-4).
Single fabric, resilient: All switches are connected to form a single fabric, but there is no single point of failure that could
cause the fabric to segment. Topologies such as Ring, Full Mesh, and Core/Edge topologies are examples of single, resilient
fabrics (see Figure 2-5).
Multi-fabric, non-resilient: In a dual fabric non-resilient SAN, half of the switches are connected to form one fabric, and the
other half form a separate fabric. This model can be extended to more than two fabrics if desired. Within each fabric, at least
one single point of failure exists. This design can be used in combination with dual-attached hosts and storage devices to keep
a solution running even if one fabric fails, or if a rolling upgrade is needed. An example of this type of SAN is a dual-fabric
SAN built with Core/Edge fabrics — each fabric only containing a single Core.
Multi-fabric, resilient: The most common multi-fabric SAN is the dual-fabric SAN. In a dual-fabric resilient SAN, half of the
switches are connected to form one fabric, and the other half form a separate fabric. This model can be extended to more than
two fabrics if desired. No fabric has a single point of failure that could cause the fabric to segment. This design can be used in
combination with dual-attached hosts and storage devices to keep an application running even if one entire fabric fails due to
operator error, catastrophe, or quality issues. This is the best design approach for high-availability environments. Another key
benefit of this design is the ability to take part of the SAN offline for online upgrades or maintenance without affecting
production operations on the remaining fabric(s). An example of this type of SAN is a dual-fabric SAN built with Core/Edge
fabrics — each fabric containing two or more Cores.

2.2.3. Redundant Fabric SANs

Resilient fabrics and the fault tolerant components that comprise them are very reliable. However, no single fabric can ever
truly be a High Availability (HA) solution. The fabric itself is still potentially subject to failures caused by things like disaster,
operator error, or software malfunctions. To account for those categories of error, another level of availability must be used:
the redundant fabric SAN, also known as a multi-fabric or dual-fabric SAN.
Redundancy in SAN design is the duplication of components up to and including the entire fabric to prevent the failure of the
SAN solution. Using a fully redundant fabric makes it possible to have an entire fabric fail as a unit or be taken offline for
maintenance without causing downtime for the attached nodes. When describing availability characteristics, what we are
concerned with is path availability. If a particular link fails, but the path to the data is still there, no downtime is experienced
by the users of the SAN. It is possible that a performance impact may occur, but this is a very small event compared to one or
many crashed servers.
In a redundant SAN architecture, there must be at least two completely separate fabrics – just as a high-availability server
solution requires at least two completely separate servers. Duplicating components and providing switch-over software is well
established as the most effective way to build high availability systems. Similarly, multi-fabric SAN architectures are the best
way to achieve high availability in a SAN.
Two or more fabrics must be used in conjunction with multiple HBAs, multiple RAID controllers, and path switch-over
software to be effective for those SAN devices that require the highest availability. Figure 2-6 illustrates the ability of
redundant fabrics to withstand large-scale failures. Note that tape drives can be effectively utilized in a redundant fabric
environment as well – even if they are single attached. Please reference the SOLUTIONware Enterprise LAN-Free Backup of
Consolidated Storage on a SilkWorm 3800 Based Redundant 96-Port Integrated Fabric (publication number 53-0000229-01)
for a discussion regarding the effective use of tapes in a dual-fabric environment.
Fabric A Fabric B
Redundant Fabric SAN
Figure 2-6 Failure of an Entire Fabric
In addition to enhancing availability, using redundant fabrics also enhances scalability. Using dual-fabrics essentially doubles
the maximum size of a SAN. If a fabric is limited by vendor support levels to 34 switches/1200 user ports and a single fabric
solution with dual-attached devices is utilized, then the SAN is limited to 600 devices. However, if a dual-fabric with
dual-attach device solution is utilized, the SAN is capable of supporting 2400 user ports or 1200 devices.

Any devices that are dual-attached and are capable of supporting an active-active or active-passive dual-path essentially
double the potential bandwidth. An active-active dual-path means that I/O is capable of using both paths in normal operation.
Some SAN devices only support active-passive dual-pathing. With active-passive dual-pathing, the passive path is utilized
only when the primary path fails.
Some devices, such as tape drives, are typically not capable of supporting multiple paths. It is possible to address this issue by
equally distributing tape devices between the redundant fabrics and configuring the backup applications to use an alternate
tape drive should an outage on one of the fabrics occur. However, some elements of a tape backup solution, such as robot
control within a single library enclosure, do not currently map well into a redundant fabric environment.
Single attached devices, including tape drives, non-critical storage, or hosts can be included in a redundant SAN, by
alternately assigning them between the fabrics. When implementing a logical group of single-attached devices, such as a tape
library with multiple tape drives and a robot, ensure that these devices reside on the same fabric, and if possible, on the same
switch.
When deploying redundant fabrics, it is not always necessary to deploy symmetrical fabrics. For example, when using a
dual-fabric, the first fabric could consist of several interconnected SAN islands, while the second fabric consists of isolated
islands, as shown in Figure 2-7. Redundancy is still maintained for dual-attach devices.
Single Single Dual

Attach Attach Attach
Single
Host Tape Host
Attach
Host
Single
Attach
Storage
Fabric A
Multiple Independent fabrics
Figure 2-7 Asymmetric Redundant SAN

2.3. Trunking
Brocade ISL Trunking is a licensed feature that enables traffic to be optimally shared across available inter-switch links (ISLs)
while preserving in-order delivery. A trunk group logically joins two, three, or four ISLs into one logical ISL. Trunking
optimizes ISL utilization, minimizing or eliminating congestion in the SAN. Trunking manages ISLs as a group instead of
individually; therefore, the effort of managing a SAN is reduced.
Trunking can also increase availability. As long as at least one ISL link remains, I/O continues if an ISL failure occurs – albeit
at a lower bandwidth. It is also possible to dynamically increase bandwidth by adding ISLs to a trunk – without impacting I/O
-- to enable up to 8 Gbit/sec of bandwidth over a single logical link. Trunking is available on the SilkWorm 3200, 3250, 3800,
3850, 3900, 12000, and 24000 platforms. The ports that form a trunk must reside in the same contiguous four-port groups,
which are known as quads, and are as shown in Figure 2-8, Figure 2-9, and Figure 2-10. For additional discussion about
Trunking, reference Exploring Brocade ISL Trunking (publication number: 53-0000263-01). An octet is a group of two
adjacent quads. The SilkWorm 3900 and 24000 switches implement octets. Note that the SilkWorm 3900 and 24000 octets are
highlighted in red dotted boxes in Figure 2-8 and Figure 2-10. The way that devices are connected to quads and octets can
impact the performance of switches for extreme I/O use cases where the majority of ports are utilized at or near full bandwidth.
Octets consist of two quads, each quad capable of supporting up to an 8 Gbit/sec trunk, ISLs, or SAN devices. An octet has no
bearing on trunking.
Note: Fabric OS versions 3.1 and 4.1 do not support Trunking with LE, L1, and L2 portcfglongdistance modes.
Please refer to Appendix B, Long-distance Technologies for Storage Area Networks for further discussion on this
topic.

0 1 2 3 4 5 6 7
3200 Q uads
3250 Q uads
0 1 2 3
4 5 6 7
3850 Q uads
0 1 2 3 8 9 10 11
4 5 6 7 12 13 14 15
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 3800 Q uads
3900 Q uads O ctet
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Number above and below each switch indicate port numbers
Figure 2-8 SilkWorm 3200, 3250, 3800, 3850, and 3900 Quads; SilkWorm 3900 Octets

P
O
R
T
S
15
14
13
SilkWorm
12 12000 Quads
11
10
9
8
7
6
5
4
3
2
1
0
Figure 2-9 SilkWorm 12000 Quads Used for Trunking

Switch Port Card
15
12 13 14 15
14
Quad
13
12 15 15 15 15 15 15 15 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
14 14 14 14 14 14 14 14
13 13 13 13
11 13 13 13 13
12 12 12 12 12 12 12 12
11 11 11
10 11 11 11 11 11
10 10 10 10 10 10 10 10
9 9 9
9 9 9 9 9 9
8 8 8 8 8 8 8 8
8 7 7 7 7 7 7 7 7
Octet 6 6 6 6 6 6 6
7
6
7
7
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
5 5 5 5 5 5 5 5
7 4 4 4 4 4 4 4 4
3 3 3 3 3 3 3 3
6
7
2 2 2 2 2 2 2 2
1 1 1 1 1 1 1 1
5
4 5 6
0 0 0 0 0 0 0 0
2 Red Boxes = Octets

Black and White Boxes = Quads
1
Figure 2-10 SilkWorm 24000 Quads and Octets

2.4. ISL Oversubscription Ratios

When designing a SAN, it is important to understand the performance boundaries such as storage fan-out ratios and storage
performance. While any SAN device that connects to a SAN at 2 Gbit/sec is theoretically capable of 2 Gbit/sec, in reality, that
device is most likely capable of a much lower performance. If a device truly is capable of generating 2 Gbit/sec of I/O, then the
principles of locality should be applied or sufficient bandwidth should be provisioned for the ISLs. A very popular SAN
application is storage consolidation, where many hosts share a storage device or port. Several popular storage vendors target
an average of a 6:1 fan-out. This means that on average six hosts are sharing a single storage port. If there were 32 storage
ports in a fabric, then one would expect to find an average of 192 hosts. Even if every host requires 1 Gbit/sec or 2 Gbit/sec of
bandwidth, the storage devices in the fabric are only capable of delivering 32 Gbit/sec (1Gbit/sec ports) or 64 Gbit/sec
(2 Gbit/sec ports). This equates to 3-6 MB/sec per host. While some ports in the fabric may require maximal bandwidth, not all
ports require sustained maximal bandwidth and rarely, if ever, do these ports require maximal bandwidth simultaneously.
When all ports operate at the same speed, ISL over-subscription is the ratio of device, or data input ports that might drive I/O
between switches to the number of ISLs over which the traffic could cross. In Figure 2-11, the over-subscription ratio on the
switch to the far left is three device ports to one ISL. This is usually abbreviated as 3:1. There are twelve hosts connected to the
upper left edge switch and only four ISLs to the Core. Thus, there are three hosts for each ISL. If all of these hosts tried to
simultaneously use the ISLs at full speed in a sustained manner — even if the hosts were accessing different storage devices —
each would receive only about one-third of the potential bandwidth available.
The basic over-subscription formula is “ISL Over-Subscription = Number of Nodes: Number of ISLs”, or Io=Nn:Ni. This is
reduced as a fraction so that Ni=1 (for example: 3:1).
3 to 1 ISL Over-subscription
12
x 1G
b/s
P o rt
s
4 x 1 Gb/s ISLs
Figure 2-11 ISL Over-Subscription with 1 Gbit/sec Devices

Since SAN-attached devices can operate at various speeds, it is necessary to put some additional thought into calculating ISL
over-subscription with variable speed hosts, storage, and ISLs. In Figure 2-12 on page 2-14, six 1 Gbit/sec hosts and six 2
Gbit/sec hosts are depicted. These share access to four 2 Gbit/sec ISLs. To calculate the ISL over-subscription ratio, average
the speed of the input ports and divide this result by the speed of the output ports. Multiply the node portion of the ratio by that
number, as shown in Figure 2-12.
2.25 to 1 ISL Over-subscription
12
mi
xe
d sp
6 x 2 Gb/s 6 x 1 Gb/s ee
d po
Hosts Hosts rts
4 x 2 Gb/s ISLs
((6*1) + (6*2)) / (4*2) = 2.25 to 1
Figure 2-12 ISL Over-Subscription with Mixed-Speed Devices
2.4.1. Recommended ISL Oversubscription Ratios

ISL oversubscription ratios apply in practice to Core/Edge fabrics, as the role of a Core switch is to connect other switches.
The calculations for ISL oversubscription ratios for a Core/Edge fabric are simple and straightforward, while these same
calculations become more complex and less pertinent for other topologies, such as a Ring topology. Connecting devices to a
Core switch is supported and makes sense for particular scenarios, such as when there are excess ports available on the Core
switch or for performance purposes. When devices are connected to the Core switch, the number of ISLs/trunks is usually
equal to or greater than the number of devices and the devices actually are undersubscribed or at a 1:1 ISL oversubscription
ratio – minimizing the value of this metric for Core switches. The ISL oversubscription ratio does become more meaningful
for a Core switch when devices are connected to the Core and there are more devices than ISLs/trunks.
A 7:1 ISL oversubscription ratio is aligned with an industry average of 6:1 fan-out. The trend in the storage industry is that the
hosts to storage ratios are increasing, as is the performance of storage devices. A 7:1 ISL oversubscription ratio should be
targeted in SAN designs, with the ISL oversubscription ratio being adjusted higher or lower to meet particular performance
requirements. While this ISL oversubscription ratio is conservative, it is felt that the cost of not having enough performance
and having to reshuffle devices and ISLs is much greater than the cost of having a few extra spare ports that can be used to
connect SAN devices at a later point in time. Note that if the SAN devices connected are 1 Gbit/sec devices, the ISL over
subscription ratio decreases since the lower bandwidth 1 Gbit/sec SAN devices are now aggregated across 2 Gbit/sec ISLs and
4-8 Gbit/sec trunks. This means that a 7:1 ISL over subscription ratio drops to 3.5:1 and a 3:1 ISL over subscription ratio drops
to 1.5:1. The higher the ISL oversubscription ratio, the lower the performance and conversely, the lower the ISL
oversubscription ratio, the higher the performance. An ISL oversubscription ratio of 3:1 results in high performance and fewer
available ports while an ISL oversubscription ratio of 15:1 results in lower potential performance and more available user
ports. The practical boundaries for ISL oversubscription ratios is 3:1 for high performance SANs and 15:1 where lower
performance is sufficient. The factors that influence this position include resiliency, number of ports available on a SilkWorm
platform, and industry host to storage fan-out ratio. Other ISL oversubscription ratios outside the practical boundaries, such as

1:1 or 31:1 are supported and make sense for specific requirements such a any-to-any high performance connectivity (1:1) or
high locality SANs (30:1). Intermediate ISL oversubscription ratios that fall within the practical boundaries are also valid,
such as 4.3:1, which can be realized on a 16-port switch with three ISLs, a 32-port switch with six ISLs, or 64-port switch with
twelve ISLs.
Guideline: As a starting point or if specific performance requirements are not available, it is suggested to connect
at a 7:1 ISL oversubscription ratio.
Table 2-1 identifies the number of ports required per platform to achieve a targeted ISL oversubscription ratio and the
recommended number of ports to reserve to scale performance. Note that for 8-port switches, the lowest practical ISL
oversubscription that maintains resiliency (i.e. more than one connection into the fabric) is 3:1 and two ports are necessary. For
the larger switches like the SilkWorm 3900 or 24000, higher ISL oversubscription ratios are possible while maintaining
resiliency. With higher ISL oversubscription ratios, it is important to utilize low bandwidth devices or to employ locality.
When a particular ISL oversubscription ratio is not recommended for resiliency purposes, the table cell is labeled with N/A
(not applicable).
Guideline: When designing a Core/Edge fabric, attach edge switch ISLs for the target ISL oversubscription ratio,
but reserve ports to enable the scaling of performance. For example, connect ISLs at a 7:1 ISL
oversubscription ratio, but reserve ports in the same quad (see section 3.1.1. Trunk and ISL
Connections on page 3-1) for further detail regarding the definition of quad) for a 3:1 ISL
oversubscription ratio.
Table 2-1 Recommended Trunk Configurations to Attain Target ISL Oversubscription Ratios for Switches Used
on the Edge in a Core/Edge Fabric
SilkWorm Number of 3:1 7:1 15:1

Switch Ports Per #trunks x # ISLs in #trunks x # ISLs in #trunks x # ISLs in
Switch the trunks the trunks the trunks
Required Reserved Required Reserved Required Reserved
SilkWorm 16 2x2 0 N/A N/A N/A N/A

3800/3850
SilkWorm 3900 32 2x4 0 2x2 2x2 N/A N/A
SilkWorm 12000 64 4x4 0 4x2 4x2 2x2 2x2
SilkWorm 24000 128 8x4 0 8x2 8x2 4x2 4x2
The information in Table 2-1 and Table 2-2 presents a detailed view of the exact number of ports and trunks to configure and
reserve to attain a particular ISL oversubscription ratio for switches used on the edge in a Core/Edge topology. In Table 2-1,
using the SilkWorm 12000 and a 7:1 ISL target connection rate as an example, it is necessary to connect two 2-ISL trunks and
reserve four ports (two 2-ISL trunk’s worth) of ports to enable the scaling of performance.
Table 2-2 identifies recommended number of ports to reserve for ISLs/trunks on an edge switch when designing a Core/Edge
fabric. The resulting configuration yields a 7:1 ISL oversubscription ratio that provisions for scaling to a 3:1 ISL
oversubscription ratio should performance requirements dictate.

Table 2-2 Recommended ISL/Trunk Port Allocations for SilkWorm Switches Used as Edge Switches in a
Core/Edge Fabric
SilkWorm Number Ports Per Edge Number Of Ports Recommended For

Platform Switch Allocation Per Edge Switch For ISLs/trunks
8-port 2000 Series 8 2
3200/3250 8 2
16-port 2000 Series 16 4
3800/3850 16 4
3900 32 8
12000 64 16
24000 128 32

2.5. Locality
If devices that communicate with each other are connected to the same switch or groups of switches then these devices have
high locality. If two devices must cross an ISL/trunk to communicate, then these devices have low locality. The higher the
locality, the less traffic crosses ISLs/trunks and therefore, fewer ISLs/trunks are needed. The lower the locality, the more traffic
crosses ISLs/trunks and therefore, more ISLs/trunks may be needed.
Figure 2-13 depicts the scenario of low locality. When host and storage devices need to communicate in low locality scenarios,
all traffic must traverse through ISLs/trunks. If four 2 Gbit/sec hosts in Figure 2-13 need to concurrently communicate with
four 2 Gbit/sec storage devices/connection at full bandwidth, congestion occurs in the ISLs. This is because eight devices
(four hosts, four storage devices) that could potentially generate 1600 MB/sec of I/O, must share only 800 MB/sec of
bandwidth. Of course, in reality, most devices cannot sustain full throughput and they would not all peak at the same time.
Note that this example involves simplex I/O, meaning only one path is utilized for I/O. One example of a simplex operation is
a scenario where all hosts are reading from the storage 100%. Under full duplex operations, the maximum potential bandwidth
is 3200 MB/sec. This is why many hosts can share a single storage port, and why many devices can share a single ISL. If all
eight devices were connected to the same switch, they could communicate with each other at a potential aggregate bandwidth
of 1600 MB/sec without congestion. When a single switch is not large enough to support the number of required devices, a
network of switches is needed.
Possible Congestion
Data Flow
Data Flow
Data Flow
4 Storage Devices 4 Hosts
Figure 2-13 Low Locality SAN

With a little planning, it is usually possible to design a SAN with a significant degree of locality, as shown in Figure 2-14.
While higher levels of locality are desirable, it is still possible to build very effective SANs with minimal to no locality. In fact,
some SANs are deliberately designed with low locality to maximize the administrative simplicity that a low locality design
provides. It is a straightforward process to design a tiered SAN that delivers sufficient bandwidth in a low locality
environment. The value in doing so is that tiered SANs require minimal planning and management to add hosts or storage -
just attach hosts to host-designated switches and storage to storage-designated switches. See section 3.1.6. Tiering (Low
Locality Device Attachment) on page 3-14 for further discussion on this topic.
Minimal Data Flow

Data Flow
Data Flow
Figure 2-14 High Locality SAN

2.6. Scalability
The scalability of a SAN is the size to which that SAN could be expanded without fundamental restructuring. Scalability is so
important to SAN design that it is frequently the first criteria used in deciding how to approach the SAN architecture. The
designer starts by asking two questions: 1) “how many ports does the SAN need now?” and 2) “how many ports will the SAN
need in the near future?” The solution is then designed to meet the port count requirements.
SANs should be designed to scale to the largest size that is expected in a reasonable time frame, rather than merely using the
requirements at the time of implementation as a target. This prevents the SAN from being “painted into a corner” and needing
to be fundamentally restructured after entering production.
Investment protection is another area that relates to scalability. If an existing switch is replaced with a newer or higher port
count switch to increase scalability, it is valuable to reuse the existing switch elsewhere in the fabric. Proper initial planning
facilitates this as well.
The Core/Edge fabric topology is the most frequently deployed topology in cases where scalability needs are great. It is
derived from the star topology, which is common in traditional data networks. The Core/Edge fabric topology (see
Figure 2-15) is a similar design, except that the Core is normally redundant, and there is typically only one level of edge
switches (few hops). Some Core/Edge implementations opt for a single Core switch when deploying the fabric in a dual-fabric
SAN architecture. The logic behind this approach is that should a single Core fail, the remaining and redundant fabric can
maintain SAN operations.
Edge
Core
Figure 2-15 Core/Edge Fabric Topology
Core/Edge topology is scalable from many perspectives. It is possible to use variable size switches in the Cores and the Edges.
The larger the Core switch, the larger the fabric can grow. If large Cores and Edges are utilized, it is possible to build very
large fabrics. The concept of scaling a fabric by using variable port count switches is shown in Figure 2-16.
A reasonable network progression might start with 16-port or 32-port Core switches and migrate to 64-port or 128-port Cores
when the scalability limits of the smaller Cores are reached. See the SAN Migration Guide (part number 53-0000360) for
detailed information on how such a migration would be performed. When additional ports are required, the lower port count
switches in the Core can be replaced with the higher density 64-port or 128-port switches. The lower port count switches can
then be redeployed at the edge.

16-port Core
64-port Core
Figure 2-16 Core/Edge Topology with Superior Scalability
If greater bandwidth is required between the edge switches, it is possible to scale the number of ISLs from the edge switches to
the Core, as shown in Figure 2-17. It is also possible to scale the bandwidth between any two-edge switches by increasing the
number of Core switches and the associated ISL/trunk connections.
Scale # ISLs
Scale # Core Switches
Figure 2-17 Increased bandwidth between Edge switches through the addition of ISLs or the addition of Core
switches

2.7. Performance
An over-subscribed link is one on which multiple devices might contend for bandwidth. Traditional data networks have been
built with very high levels of over-subscription on links for years. The Internet is probably the best-known example of this. A
congested link is one on which multiple devices actually are contending for bandwidth.
While not capable of supporting Internet-like over-subscription ratios, real-world SANs can be expected to have several
characteristics that enable them to function well, even with over-subscribed links. These characteristics include bursty traffic,
shared resources, low peak usage by devices, good locality, and devices that can generate only a small fraction of the I/O as
compared to the available bandwidth. Most networks have all of these characteristics to some degree. Moreover, organizations
can often realize substantial cost savings by deliberately designing a SAN with a certain amount of over-subscription.
When performance service levels are critical and the bandwidth requirements are high, lower over-subscription levels or traffic
localization should be targeted.
Today, many devices attached to a SAN are not capable of generating traffic at the full Fibre Channel bandwidth of 100
MB/sec or 200 MB/sec. Figure 2-18 and Figure 2-19 detail a simplified scenario using a handful of devices to explain SAN
bandwidth consumption.
Note that in the ISL in Figure 2-18, the total amount of traffic that is intended to cross between switches never exceeds the 200
MByte/sec capacity of the link (assuming a 2 Gbit/sec ISL). Even if the servers in Figure 2-18 are running at their theoretical
maximum performance, there still might be performance bottlenecks with the storage devices. In Figure 2-19, the two servers
are accessing a single storage port, so the 2:1 fan-out of the storage port becomes the limiting factor.
Figure 2-18 Low Server I/O Utilization
The key to managing bandwidth is capturing or estimating performance requirements and matching these requirements to an
appropriately designed SAN. If the servers are capable of generating 400 MB/sec of traffic and there are two 2 Gbit/sec ISLs
(see Figure 2-19), the network routes the traffic over both of them, and congestion does not occur. The storage port can operate
at only 50 MB/sec; therefore, each server can average only 25 MB/sec. This scenario is common in storage consolidation
environments where many servers need to share a single storage port. However, the I/O requirements for most servers can be
surprisingly low (1 or 2 MB/sec) and a single storage port can sustain many hosts without overwhelming its I/O capability.

M ax 2
00 M B
Hosts /s ec
e c
0 M B /s
M ax 20 SW12000 SW12000
Storage
Figure 2-19 High-Bandwidth Consumption with Two ISLs
2.7.1. I/O Profiles

Understanding an application’s I/O requirements is essential to the SAN design process. Classifying I/O as described below
will help you create an I/O profile which can be compared to Table 2-3.
• An individual I/O can be classified as either a read or a write operation. Although I/O is usually a mixture of reads
and writes, some applications are strongly biased. For example, video server I/O activity is normally almost 100
percent reads, while video editing cluster I/O may be mostly writes. Tape I/O is unique in that it typically consists of
large block transfers with reads and writes. When reading from disk, the I/O access may be random. When writing,
blocks are sent to streaming tape in a sequential fashion. Restores have similar I/O characteristics, except of course
that blocks are read from tape and written to disk.
• I/O can further be classified as random or sequential. Examples of random I/O include an e-mail server or an OLTP
server. Sequential I/O is characteristic of decision support (such as data warehousing), scientific modeling
applications, or backup applications.
• The third characteristic of I/O is size, which typically ranges from 2 KB to over 1 MB. Typically, user file systems
have smaller I/O sizes, whereas video servers or backups may have very large sizes.
• For SAN design performance purposes, I/O is classified by bandwidth utilization: light, medium, and heavy. It is very
important to support test assumptions by gathering actual data when possible. You can gauge the type of I/O activity
in your existing environment by using I/O measurement tools such as iostat and sar (UNIX) or diskperf (Microsoft).
Table 2-3 illustrates the application I/O profiles that establish the typical magnitude of application bandwidth consumption.
Table 2-3 Application I/O Profiles
Application Bandwidth Read/Write Max Typical Access Typical Block

Utilization Size
OLTP, e-mail, UFS e-commerce, Light 80% read Random 8 KB

CIFS 20% write
OLTP (raw) Light 80% read Random 2 KB to 8 KB

20% write
Decision support, HPC, seismic, Medium to 90% read Sequential 16 KB to 128 K

imaging Heavy 10% write (except
during “builds”)
Video Server Heavy 98% read Sequential > 64 KB

2% write
SAN applications: LAN-Free Medium to Variable Sequential >= 64 KB

backup, snapshots, third-party copy Heavy

2.7.2. Fabric Latency

The time it takes a frame to traverse from its source to its destination is referred to as the latency of the link. Sometimes a
frame is switched from source to destination on a single switch and other times a frame must traverse one or more hops
between switches before it reaches its destination.
A common misconception is that hop counts introduce unacceptable latency. For the vast majority of Fibre Channel devices,
the latency associated with traversing one or more ISLs is small relative to other data path delays. I/O for disk devices is
measured in milliseconds. Every hop in the Brocade SAN fabric adds no more than two microseconds of latency. This means
that in a large fabric designed with seven hops between two devices (the Brocade-supported maximum), the latency could be
up to fourteen microseconds. The distance between switches also introduces latency, especially for long-distance solutions
spread over larger metropolitan areas. The speed of light in optics is approximately five microseconds per kilometer. Brocade
addresses the need for long distance performance with Brocade Extended Fabrics™. This product enables full-bandwidth
performance across distances spanning up to 100 km, with greater distances possible at lower speeds. Please refer to Appendix
B, Long-distance Technologies for Storage Area Networks for more information.
For most I/O profiles, hop-count latency is inconsequential, from both a switch latency and optical latency standpoint. This is
because the millisecond disk I/O latency is several orders of magnitude greater than the microsecond latency of a Fibre
Channel fabric. Because the latency is so small, virtually no applications will be affected by the added latency.
As a result, hop latency is not a reason to keep hop counts low in a SAN design. A more pertinent reason to reduce hop count
is over-subscription: the more ISLs a frame has to traverse, the more likely it is to cross a congested ISL. The best hop count
for reducing over-subscription is, of course, zero hops (localized traffic) if performance is a driving requirement.
Table 2-4 shows collected I/O performance data representing random read/write operations (typical of E-commerce or OLTP
applications) and large, sequential reads (typical of decision-support, backup, or video applications). Tests were run on a 1
Gbit/sec host and storage connected to the same switch and across a SAN where the host and storage were separated by four 2
Gbit/sec hops. There was no significant difference in performance between I/Os run locally or across the SAN, so I/O latency
was not an issue in either configuration.
Table 2-4 I/O Latency Test Results
HOPS I/O Size Read % Write % Access Response Time Throughput

(milliseconds) (KB/sec)
0 2 KB 80% 20% Random 8 262
4 2 KB 80% 20% Random 8 261
0 8 KB 60% 40% Random 9 843
4 8 KB 60% 40% Random 10 833
0 64 KB 100% 0% Sequential 3 20,741
4 64 KB 100% 0% Sequential 3 20,652
0 128 KB 0% 100% Sequential 22 5890
4 128 KB 0% 100% Sequential 22 5899

2.8. SAN Topologies

There are multitudes of options for architecting a fabric infrastructure. A number of factors influence a choice of fabric
topology, such as scalability, performance, and availability. This section discusses some of these factors, and how they can
affect a topology choice. It also describes some of the most effective fabric topologies, and gives guidance on when to use
each. To simplify the SAN design process, consider the use of any of the reference topologies that are detailed in section 2.8.1.
Simple Topologies on page 2-24 or section 2.8.2. Hybrid Topologies on page 2-32. These topologies are effective for a wide
variety of applications, are tested extensively within Brocade, are in wide deployment in customer environments, are high
performance, and scalable.
The flexible fabric architecture of Brocade switches allows arbitrarily complex fabrics to be built when it is necessary to solve
complex problems, but also allows simple solutions to be built to solve simple problems.
2.8.1. Simple Topologies

A simple topology is one that is geometrically basic. A Ring of six switches is easy to recognize as a Ring. A Ring of six
switches, each of which has some variable number of switches attached to it is not as easy to define, and would be considered
a hybrid of a Ring and some number of other topologies. Hybrid topologies are discussed later.
The simple topologies discussed in this section include:
• section 2.8.1.1. Cascade on page 2-25
• section 2.8.1.2. Ring on page 2-26
• section 2.8.1.3. Full Mesh on page 2-27
• section 2.8.1.4. Partial Mesh on page 2-28
• section 2.8.1.5. Core/Edge on page 2-29
• section 2.8.2.2. Composite Core/Edge on page 2-33

2.8.1.1. Cascade
A Cascaded topology, which is shown in Figure 2-20, is similar to a bus or daisy chain topology: it is a line of switches with
one connection between each switch and the switch next to it. The switches on the ends have only one connection.
Cascaded fabrics are very inexpensive, easy to deploy, and easy to expand. However, they have the lowest reliability, are
subject to congestion, and have limited scalability. They are most appropriate in situations where most if not all traffic can be
localized onto individual switches, and the ISLs are used primarily for management traffic or low bandwidth SAN
applications. See section 2.5. Locality on page 2-17. for more information on the principal of locality.
There are variations of a Cascade topology that use more than one ISL between switches. This will eliminate ISLs as a single
point of failure, and increase the reliability of the solution. However, this also increases the cost of the solution, and each
switch can still be a single point of failure. Table 2-5 charts the properties of a Cascade topology.
Figure 2-20 A Cascaded Fabric
Table 2-5 Properties of the Cascade Topology
Ratings indicate how well the topology meets the ideal requirements
of that property (1 = Not well, 2 = Moderately well, 3 = Very well).
Properties Ratings
Ease of scalability 3
Performance 1
Ease of deployment 3
Reliability 1
Cost 3

2.8.1.2. Ring
A Ring (see Figure 2-21) is like a Cascaded fabric, but with the ends connected. The Ring has superior reliability to the
Cascade because traffic can route around an ISL failure or a switch failure. However, it does cost slightly more than a Cascade.
The Ring is usually preferable to the Cascade for that reason. Like the Cascade, the Ring is most suitable when locality is used
to optimize traffic patterns in the fabric. This design is effective for configurations that start small and stay small. It can also be
used when implementing SAN over MAN or WAN, where the topology of the MAN/WAN might dictate the topology of the
Fibre Channel network -- Rings are common MAN/WAN topologies. Finally, a Ring topology is a good choice when the ISLs
are mostly used for management or low bandwidth SAN applications. Table 2-6 charts the properties of a Ring topology.
Figure 2-21 A Ring Topology
Table 2-6 Properties of a Ring Topology
Properties Ratings
Performance 1
Reliability 2
Cost 3

2.8.1.3. Full Mesh
In a Full Mesh topology (see Figure 2-22 and Figure 2-23), every switch is connected directly to every other switch. Using
16-port switches, the largest useful Full Mesh consists of eight switches, each of which has nine available ports. This provides
72 available ports. Adding more than eight switches will actually reduce the number of available ports. A Full Mesh topology
is best used when the fabric is not expected to grow beyond four or five switches, since the cost of the ISLs becomes
prohibitive after that. Full Mesh fabrics are best used when any-to-any connectivity is needed. In addition, traffic patterns
should be evenly distributed, but overall bandwidth consumption low. Otherwise, a Core/Edge SAN is a better fit. Technically,
almost any topology could be described as some sort of mesh. Since this is not a very useful definition, working definitions for
two meshes are provided: the Full Mesh and the Partial Mesh. There are two special cases for a Full Mesh:
• A 2-switch Full Mesh is identical to a 2-switch Cascade.
• A 3-switch Full Mesh is identical to a 3-switch Ring.
Scaling a Full Mesh may require unplugging edge devices. If using a 4-switch full mesh (52 edge ports) and all the ports with
edge devices are in use, it will be necessary to unplug one device from each switch in the mesh in order to add another switch.
Because of this, Full Mesh topologies do not have a high rating for ease of scalability. Table 2-7 charts the properties of a Full
Mesh topology.
Figure 2-22 A Four-Switch Full-Mesh Topology
Figure 2-23 Maximum Size of a Full-Mesh Topology with 16-port Switches
Table 2-7 Properties of a Full-Mesh Topology
Properties Ratings
Performance 2
Reliability 3
Cost 1

2.8.1.4. Partial Mesh

A partial mesh is similar to a Full Mesh, but with some of the ISLs removed. In most cases, this is done in a structured pattern.
The common definition for a Partial Mesh (see Figure 2-24) is broad enough to encompass almost all fabrics that are not Full
Meshes. A Core/Edge topology is considered a partial mesh topology.
The topology in Figure 2-24 might be useful if minimal traffic is expected flow horizontally (i.e. from left to right) and that the
majority of traffic will flow vertically (i.e. top to bottom). For example, hosts would be connected to the top switches and
storage connected to the bottom switches. The network still is fully resilient to failure, and there is no price premium for an
ISL that will not be used (i.e. a horizontal ISL). Partial meshes also scale further than Full Meshes.
Figure 2-24 A Partial-Mesh Topology
While partial mesh networks can be scaled to produce a large number of ports, they still have less than ideal performance
characteristics. In addition, partial meshes can be difficult to scale without downtime. As a result, meshes – either full or
partial – are recommended only for networks that will change infrequently, have limited scaling requirements, and where the
traffic patterns of the connected devices are known. Note that the Core/Edge topology is considered a partial mesh and does
not suffer from the previously mentioned performance and scaling issues. With the advent of high port count switches, it is
possible to build reasonably large fabrics, that are considered partial meshes, that only consist of four switches. For example,
using four 64-port switches, it is possible to build a fabric that consists of 256 total ports. Table 2-8 charts the properties of a
partial-mesh topology.
Table 2-8 Properties of a Partial-Mesh Topology
Properties Ratings
Performance 1
Reliability 3
Cost 2

2.8.1.5. Core/Edge
With a Core/Edge topology, it is easy to satisfy SAN functional requirements. Given a diverse set of requirements:
performance, locality, connectivity, and scalability, the Core/Edge topology provides the most flexible architecture to address
these overall requirements.
The Core/Edge fabric is a variation on the well-established “star” topology popular in Ethernet LANs. There are a few
differences. Because Fibre Channel uses a routing protocol (i.e. FSPF) with load sharing, Fibre Channel fabrics can take full
advantage of multiple Core switches. In an Ethernet network, multiple switches at the center of a star would usually act in an
active/passive backup relationship, using a Spanning Tree Protocol or some variation.
These differences make multi-core fabrics very popular, since it is possible to easily scale the fabric’s bandwidth by adding
Core elements. In addition, the requirements of a Core fabric switch are more stringent than those of the center switch in an
Ethernet star. Due to the properties of Fibre Channel, the acceptable performance and reliability characteristics are very high.
The introduction of trunking (see section 2.3. Trunking on page 2-9) further increases the effectiveness of a Core/Edge fabric
due to more efficient utilization of the ISLs and lessened management requirements. Some Core/Edge implementations opt for
a single Core switch when deploying the fabric in a dual-fabric SAN architecture (see Figure 2-25). The logic behind this
approach is that should a single Core fail, the second fabric in the SAN can maintain operations. In a resilient Core/Edge fabric
(shown in Figure 2-26) two or more switches reside in the center of the fabric (the Core) and interconnect a number of other
switches (the edge).
SW2800 SW12000 SW3800 SW3900
SW24000
SW2800 SW12000 SW3800 SW3900
Highest
Availability
SW24000 SW24000
Figure 2-25 Single Core/Edge Fabric

Switches that reside in the middle of the fabric are referred to as Core switches. The switches that are interconnected by the
Core switches are referred to as edge switches. The simple form of the Core/Edge fabric has two or more Core elements, each
of which consists of a single switch. In a simple Core, the Core switches do not connect with each other. Edge switches in a
Core/Edge fabric also do not connect to each other. They only connect to the Core switch. Several variants of the Core/Edge
network do not meet this definition. These are discussed later in this section (see section 2.8.2. Hybrid Topologies on page
2-32). Devices such as hosts and storage are attached to free ports on the edge switches. These ports are referred to as edge
ports or user ports. Free ports on the Core switches should usually be reserved for additional edge switches when using
16-port switches and can connect SAN devices for higher port count switches. The scalability of a Core/Edge fabric is reduced
when a device is attached to a Core switch (see section 3.1.5. Connecting Devices To The Core on page 3-13).
Note: A Core/Edge fabric is typically built with two or more Core switches, but can be built with a single Core switch if that
fabric is used in a dual-fabric SAN - the redundant fabric maintains the SAN availability should the single Core switch
become unavailable.
A Core/Edge topology (shown in Figure 2-26) can be built with a variety of switch platforms, such as the SilkWorm 2000
series, 3200, 3250, 3800, 3850, 3900, 12000, and 24000. Selecting the right switch for a Core or Edge location can affect the
maximum size of the fabric. A SilkWorm 24000 is used as the Core in Figure 2-26, as the 128-ports per domain support
enables the size of the fabric to grow to hundreds of ports by connecting edge switches. The edge can be built with a variety of
switch platforms. Note that the SilkWorm 12000 can contain up to 128 ports in a 14U chassis, configured as two 64-port
switches (each switch is known as a logical switch and may also be referred to as a domain). The SilkWorm 24000 can contain
up to 128 ports in a 14U chassis, configured as one 128-port domain.
0 or more
0 or more 0 or more 0 or more
Core (2 or more) Edge
2000 Series
38x0 or 32x0
3900
12000
24000
Figure 2-26 The Core/Edge Topology
A key benefit of the Core/Edge topology is the use of FSPF, which automatically distributes the load across all paths equally.
There are two or more paths between any two edge switches in a resilient Core/Edge topology. Because of this, Core/Edge
fabrics have very good performance under varying to zero locality conditions. This concept is depicted in Figure 2-27.

Figure 2-27 Equal Load Distribution and Access in a Core/Edge Topology
Additional benefits of the Core/Edge topology:

• Well-tested and reliable
• Widely deployed in production environments
• Simple and easy to understand
• Able to solve most design problems, fits well with many SAN solutions, and is an effective choice when design
requirements are not well known
• Easy to grow without downtime or disconnection of links and devices
• Pay as you grow
• Flexible
• Capable of exhibiting stellar performance, with full utilization of FSPF load sharing and resiliency features
• Conducive to performance analysis. Because the Core/Edge topology is symmetrical, it is a straightforward process to
identify performance issues. Every device has an equivalent path to any other device and the same available bandwidth
between any two devices. To identify a SAN performance issue it is only necessary to monitor the Core switches. With
other topologies, this is not the case.
• The potential to scale to hundreds of ports (using high port count switches)

Table 2-9 Properties of a Simple Core/Edge Topology When Using
Ratings indicate how well the topology meets the ideal requirements of that property
(1 = Not well, 2 = Moderately well, 3 = Very well).
Properties Ratings
Performance 3
Reliability 3
Cost 2
2.8.2. Hybrid Topologies

In some cases, it may be desirable to use a topology that is a composite of simple topologies, which is known as a hybrid
topology. For example, if a SAN is spread across a MAN or WAN link, it can be an effective strategy to “glue” two Core/Edge
SANs together using a complex Core approach. There are several reasons to build a hybrid SAN:
• Geographically distributed nodes
• Cable plant restrictions
• Scalability requirements
It is unrealistic to expect all SAN users to deploy a Core/Edge topology or for that matter even a simple topology. Hybrid
topologies are more complex and can be more expensive to build and maintain – especially for high port count fabrics.
However, in certain cases it is necessary to expand beyond several hundred ports using 16-port switches or (theoretically)
several thousand ports using 128-port switches. It is also possible that geographic/cable restrictions need to be designed into
the solution. In these cases, hybrid topologies are an option. The elegance of Brocade SilkWorm switches and Fabric OS is the
freedom to design and implement any number of topologies to suit the specified requirements. The key to designing hybrid
topologies is not to exceed the recommended number of switches and hop count per fabric.
It is possible to build a fabric consisting of hundreds of ports using 128-port switches in a simple Core/Edge topology. Even
with higher port count switches, the need to design fabrics that take into account massive fabric sizes, cable plant restrictions
or geographically distributed fabrics will not go away. The same hybrid topologies that work for 16-port switches can also be
deployed using higher port count switches to create mega-SANs. What seems large today will appear small tomorrow.
2.8.2.1. Complex Core

A resilient complex Core/Edge fabric has two or more Core elements, each of which consists of multiple interconnected
switches. In contrast, a simple Core consists of two or more Core elements, each of which consists of a single switch. The Core
elements can be constructed out of any of the simple topologies previously discussed. However, it is generally best to construct
them out of high performance topologies, like a Full Mesh or simple Core/Edge if there is to be significant edge-to-edge
traffic. Keep the hop count within each Core as low as possible so that the fabric is within the Brocade maximum of seven
hops. Figure 2-28 is an example of a resilient complex Core/Edge fabric and is an effective design for spanning a fabric
between two separate sites. The complex Core is actually a partial mesh.

Site A
0-100 KM
Site B
Figure 2-28 Resilient Complex Core/Edge Fabric
2.8.2.2. Composite Core/Edge

A composite resilient Core/Edge fabric has two or more Cores, as shown in Figure 2-29. It is useful to achieve a higher port
count than available from a simple topology and is a topology that works well with a tiered approach to performance
optimization. Each Core consists of two or more single-switch elements. A composite resilient Core/Edge fabric could be
viewed as two Core/Edge fabrics “glued together at the edge.” Generally, any host needs access to any storage and there are
usually more hosts than storage devices in a fabric. In the configuration shown in Figure 2-29, all hosts have equal access
(2-hops) to any storage device; however it takes 4-hops should it be necessary for a host on the left tier to communicate with a
host on the right tier, which is acceptable since hosts normally do not communicate with each other.
HostTier
Blue Red
Core Core
StorageTier
y Tw o cores (Blue, Red)

y Each core composed of tw o sw itches
y Edge sw itches asymetrically connected
Figure 2-29 A Composite Resilient Core/Edge Topology


Chapter
Architecting SANs With SilkWorm Switches
3
Many SAN related elements such as device attachment strategies, platform specific topics, zoning, Secure Fabric OS, switch
location in the fabric, scalability, extended fabrics, and supportability have some bearing on the design of a SAN. It is these
more in depth topics and the usage of SilkWorm switches in a SAN architecture that are discussed in this section. A set of
recommended fabric topologies is also provided.
Note: The Core/Edge topology is identified as a reference topology for the guidelines presented in this chapter.
The Core/Edge topology is preferred for scalable, available, and high performance fabrics for a number of reasons (see section
2.8.1.5. Core/Edge on page 2-29). The guidelines established in this chapter can also apply to other topologies, such as a full
mesh or ring. It is up to the reader to interpret the guidelines in this section for topologies other than Core/Edge. Regardless of
topology chosen, a redundant fabric (i.e. dual fabric) SAN is recommended.
3.1. Device Attachment Strategies

How switches connect to other switches and how devices connect to those switches significantly influences the performance
and availability of a SAN. Easy to understand and consistent device attachment strategies also simplifies the operation and
maintenance of a SAN. This section details effective techniques for connecting devices, and ISLs/trunks to the SilkWorm 2000
series, 3200, 3250, 3800, 3850, 3900, 12000, and 24000 for availability, scalability, performance, and operational efficiency.
Note: The switch designator 38x0 indicates either a SilkWorm 3800 or 3850, and the switch designator 32x0 indicates either
a SilkWorm 3200 or 3250.
3.1.1. Trunk and ISL Connections

Guidelines for connecting ISLs/trunks, which take into consideration a the architecture of the switch, availability,
performance, and operational efficiency, are presented in this section.
Connecting ISLs/trunks across switch port cards horizontally on the SilkWorm 12000 or 24000 prevents a switch port card
failure from segmenting that switch from the fabric when there is more than one ISL or trunk connecting that switch to the
fabric. Connecting ISLs/trunks diagonally to the SilkWorm 3900 results in a configuration optimized for performance.
Note: The diagonal recommendations for the SilkWorm 3900 starts in the lower left hand corner of the switch (ports 0-4) and
progress to the upper right hand corner (ports 28-31). Connecting in the opposite order and starting in the upper left
hand corner of the switch (ports 16-19) and progressing to the lower right hand corner (ports 2-15) is also an acceptable
connection strategy.

3 Architecting SANs With SilkWorm Switches
Connecting ISLs/trunks to the left or right hand side of a SilkWorm series 2000, 32x0, or 38x0 switches does not yield any
availability or performance benefits; however, doing so does result in a switch that is easier to operate since the cable locations
are standardized. For all switches, the adoption of standardized ISL/trunk connections makes it easier to operate and maintain
those switches. If it is possible to connect more than one ISL from an edge switch to a core switch and both switches are trunk
capable, a choice exists to connect the ISLs to different quads for availability purposes or to use trunking. Under these
circumstances, the performance benefits of trunking combined with the resiliency of the Core/Edge topology are felt to result
in a simpler and superior solution. For example when connecting a SilkWorm 38x0 edge switch to a SilkWorm 12000 core
switch with a 3:1 ISL oversubscription ratio, connect using a 2-ISL trunk from the edge to each core instead of connecting two
ISLs spread across separate blades on each core.
SAN SAN
Devices Devices
ISLs/trunks ISLs/trunks
Figure 3-1 SilkWorm 12000 Horizontal ISL and Device Attachment
Guideline: Place trunking-capable switches adjacent to each other. This maximizes the number of trunking groups
that can be created. If using a Core/Edge topology, place trunking-capable switches at the core of the
fabric and switches that are not capable of trunking at the edge of the fabric. This allows for the
maximum amount of trunking between core switches and edge switches that are capable of trunking.
Guideline: When more than one ISL connects two trunk capable switches there are two option:
- spread the connections across different quads for availability purposes and not use trunking
- locate the ISLs on the same quad and use trunking (recommended)
Guideline: If trunking is not possible, or more than one trunk/ISL is required between switches, connect trunks and
ISLs from the same switch as follows:
- On the SilkWorm 12000 and 24000, connect multiple ISLs/trunks in redundant groups and connect
these ISLs/trunks horizontally (see Figure 3-3 and Figure 3-4).
- On the SilkWorm 3900, connect the ISLs/trunks on corners of the switch that are diagonally opposed
(see Figure 3-6).
- On 8-port and 16-port SilkWorm switches (SilkWorm 2000 series, 32x0, 38z0), connect the trunks/ISLs
to the either the left or right hand side of the switch (for consistency) see Figure 3-7 and Figure 3-8.

Architecting SANs With SilkWorm Switches 3
Guideline: For the SilkWorm 24000, 12000, and 3900, create redundant trunking groups when possible. This
protects against multiple ISL failures, optimizes performance, and for the SilkWorm directors it protects
against the rare occurrence of a blade failure. This means that instead of using a single 4-ISL trunk to
connect two SilkWorm 24000s, 12000s or 3900s, utilize two 2-ISL trunks, as shown in Figure 3-2.
1
1 1 ,
nk
rt u
L s
IS nk
4- tru
e L
on IS
of o 2-
ad w
ste e t
In us
1 2 1 2
1 2
Figure 3-2 Use redundant trunk groups when possible for the SilkWorm 3900, 12000, and 24000.

3.1.2. Edge Switch ISL/Trunk Connections

The recommended ISL/trunk connection guidelines for edge switches that are part of a Core/Edge topology are shown in
Figure 3-4, Figure 3-5, Figure 3-6, Figure 3-7, and Figure 3-8. Keeping in line with the recommended ISL oversubscription
ratios discussed in section 2.4. ISL Oversubscription Ratios on page 2-13, the connection areas for ISLs/trunks are grouped
into blocks of thirty-two ports for the SilkWorm 24000, sixteen ports for the SilkWorm 12000, eight ports for the SilkWorm
3900, four ports for 16-port switches, and two ports for 8-port switches. The connection schemes for the SilkWorm 3900,
16-port, and 8-port switches locates the trunks on the left and right hand sides of each switch. While more ISLs/trunks may be
connected for certain configurations, the connection schemes recommended account for SilkWorm switches used as edge
switches and provisioning for a 3:1 ISL oversubscription ratio.
SilkWorm 24000
Slot 1 Slot 2 Slot 3 Slot 4 Slot 7 Slot 8 Slot 9 Slot 10
15 15 15 15 15 15 15 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
14 14 14 14 14 14 14 14
13 13 13 13 13 13 13 13
12 12 12 12 12 12 12 12
11 11 11 11 11 11 11 11
10 10 10 10 10 10 10 10
9 9 9 9 9 9 9 9
8 8 8 8 8 8 8 8
7 7 7 7 7 7 7 7
6 6 6 6 6 6 6 6
7
7
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
4 5 6
5 5 5 5 5 5 5 5
4 4 4 4 4 4 4 4
3 3 3 3 3 3 3 3
4-ports
2 2 2 2 2 2 2 2
1 1 1 1 1 1 1 1
0 0 0 0 0 0 0 0
1 1
A llocate 32-ports fo r
Instead of one 4-ISL trunk, use two 2-
IS Ls/tru nks in e igh t
ISL trunks g ro up s for E d g e
sw itch
1 2 1 2
Figure 3-3 Recommended ISL/Trunk Connection Scheme for SilkWorm 24000 Switches Used as Edge Switches
in a Core/Edge Fabric

15 15 15 15
12 13 14 15
12 13 14 15
12 13 14 15
12 13 14 15
14 14 14 14
SilkWorm 12000 13 13 13 13
12 12 12 12
11 11 11 11
10 10 10 10
Allocate 16-ports for 9 9 9 9
ISLs/trunks in four 8 8 8 8
groups for Edge

7 7
switch 7 7
6 6 6 6
7
7
4 5 6
4 5 6
4 5 6
4 5 6
5 5 5 5
4 4 4 4
4 ports 3 3 3 3
2 2 2 2
1 1 1 1
0 0 0 0
2 2
1 Yes
1 2
2
1
Figure 3-4 Recommended ISL/Trunk Connection Scheme for SilkWorm 12000 Switches Used as Edge Switches
in a Core/Edge Fabric

Figure 3-5 Do not connect more than one ISL to the same switch port card unless it is part of a trunk.

SilkWorm 3900
Allocate 8-ports for ISLs/trunks in two groups for Edge switch
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
0 1 2 3 4 5 6 7 8 9 10
19 11 12 13 14 15
4-ports
2
1 Yes 2
1
2
1 1 2
If trunking is not possible, or more than one trunk/ISL is

required between switches connect the ISLs/trunks to
diagonally opposed corners of the switch.
Figure 3-6 Recommended ISL/trunk connection scheme for SilkWorm 3900 switches used as Edge Switches in
a Core/Edge fabric

Allocate 4-ports for ISLs/trunks for Edge switch
4-ports
0 1 2 3 8 9 10 11
4 45 56 67 7 12 13 14 15
SilkWorm 3850
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
SilkWorm 3800
4-ports
Note: ISL/trunk location can be either

the left or right hand side of the switch
Figure 3-7 Recommended ISL/trunk connection scheme for SilkWorm 16-port switches used as Edge switches
in a Core/Edge fabric

Allocate 2-ports for ISLs/trunks for Edge switch
2-ports
0 1 2 3
SilkWorm 3250
4 45 56 67 7
0 1 2 3 4 5 6 7 SilkWorm 3200
2-ports
Note: ISL/trunk location can be either

the left or right hand side of the switch
Figure 3-8 Recommended ISL/trunk connection scheme for SilkWorm 8-port switches used as Edge switches in
a Core/Edge fabric

3.1.3. Core Switch and Standalone ISL/Trunk and Device

Connections
The purpose of a core switch is to connect other switches and SAN devices. When a SilkWorm switch is utilized as a core or
standalone switch, similar guidelines for ISL/trunk connections as outlined in the previous sections apply. When connecting
edge switches or storage to a core or standalone switch, there are some attachment strategies for the SilkWorm 3900, 12000,
and 24000 switches that enable better performance under some circumstances. These connection strategies are similar to the
edge switch connection strategies and provision for up to a 3:1 host to storage ratio and assume a tiering/low-locality
implementation. The premise that hosts do not communicate with hosts and storage does not communicate with storage is the
basis for these recommendations. For edge switches that mix host and storage on the same switch, the edge to core connections
are not as important; however, if that storage is shared with devices located on other edge switches, it is suggested to connect
these edge switches as if they were storage edge switches. In the majority of SAN deployments, there are significantly more
hosts devices than storage devices and the trend is to aggregate many low bandwidth demand hosts to a few high bandwidth
demand storage ports. It is these factors that contribute to the recommendation to connect the storage edge switches and
storage diagonally opposed corners for the SilkWorm 3900, as shown in Figure 3-10. There are no performance or availability
benefits for adopting a connection plan for 8 and 16-port switches; however, standardization on a connection plan simplifies
operation and management of these switches (see Figure 3-11).
Guideline: Connect storage edge switches and SAN devices horizontally to the SilkWorm 12000 and 24000.
Guideline: Connect storage edge switches and storage to diagonally opposed corners for the SilkWorm 3900. Fill
the remaining quads with host edge switches or host devices.
Guideline: Connect 8 and 16-port storage switches, host edge switches, and SAN devices in groups to realize
operation and maintenance simplicity.
Core
SAN Devices
ISLs/
Or ISLs/
Trunks
Trunks
Figure 3-9 ISL/trunk connection method for SilkWorm 12000 and 24000 switch used as core switch in a
Core/Edge Fabric

S ilk W o r m 3900
S to ra g e E d g e
H o s t E d g e S w itc h C o n n e c tio n s S w itc h e s o r
S to ra g e
S to ra g e E d g e
S w itc h e s o r H o s t E d g e S w itc h C o n n e c tio n s
S to ra g e
S to ra g e E d g e
S w it c h e s o r
S to ra g e
H o st E dge
S w itc h e s
Figure 3-10 ISL/trunk connection method for SilkWorm 3900 switches used as core or standalone switches in a
Core/Edge fabric
Storage Edge 16-port switch

Switches or Host Edge Switches
Storage
Storage Edge
Switches or
Storage
Host Edge
Storage Switches or
Edge Hosts
Host Edge
Switches
Switches 8-port switch
or
Storage
Figure 3-11 ISL/trunk connection method for SilkWorm 8 and 16-port switches used as core or standalone
switches in a Core/Edge fabric

3.1.4. Attaching SAN Devices For Availability

When attaching devices to a SilkWorm 24000 or 12000, it is recommended to distribute the connections across blades to
minimize the impact of a switch port card failure. To effectively distribute the connections, it is important to understand the
connection types and relationships. For example a large storage array may support sixteen or more connections. If all sixteen
connections were made to a single switch port card, then the failure of that switch port card would sever all sixteen
connections to that array. However, if these sixteen connections were evenly distributed across a SilkWorm 24000 or 12000
switch, the failure of a switch port card would only affect four of the sixteen array ports. Of course, if a redundant (i.e. dual)
fabric SAN architecture is implemented, a switch port card failure would have minimal or no impact on operations.
Figure 3-12 depicts attaching devices across switch port cards for availability. Note the high port count devices are distributed
across all switch port cards and that devices are partitioned and distributed by type across switch port cards. While it is not
necessary to attach devices in groups, as shown in Figure 3-12 it does make it easier to manage the device connections. This
same guidance holds true for hosts that connect into the same fabric with multiple connections. For SilkWorm 8-port, 16-port,
and 3900 switches, connect devices as shown in Figure 3-13.
Guideline: Distribute devices on the SilkWorm 24000 and 12000 across switch port cards from left to right for
optimal availability.
D istribute H igh P ort C ount D evices,

Such as A rrays or T ape L ibraries
A cross M ultiple Sw itch P ort C ards
Ta p e L ib ra ry D isk S to ra g e H o st
SAN
SAN SAN D e v ic e s
Devices Devices
IS Ls/T runks
ISLs/trunks ISLs/trunks
D istrib u te D evices A cro ss IS L s/T ru n ks

S w itch P o rt C ard s
S A N D evices
Figure 3-12 Attaching devices for availability on a SilkWorm 24000/12000 used as an edge switch

SilkWorm 3900
4 ports ISLs/
12 ports SAN Devices
trunks
4 ports ISLs/
trunks
4 ports 16-port switch

trunks/ISLs
2 ports 6 ports
ISLs/ SAN 8-port switch
trunks Devices
Figure 3-13 SilkWorm 8-port, 16-port, and 3900 device connection plans for edge switches
3.1.5. Connecting Devices To The Core

When designing SilkWorm based Core/Edge SANs it is key to identify the scaling requirements for the SAN. Once these
scaling requirements are known, it is possible to allocate switch ports for ISLs needed for current switch connections as well as
future switch connections. If the scaling requirements are not too large and there are unallocated ports, then the remaining
ports can be utilized for attaching SAN devices to the core. In many situations, the core switch may be the most highly
available switch and it is desirable to attach mission critical devices to the most available switches. Connecting a device that is
shared by many other devices (i.e. many to one) may result in improved performance. Scaling for performance is also a
consideration. Following the guidelines in Figure 3-12 and Figure 3-13 results in a SAN with enough ports provisioned to
enable the scaling of performance.
It is important to understand the implications of attaching SAN devices to the core switch. While device placement does not
constitute fabric topology, it may very well effect and be affected by topology. The example in Figure 3-14 illustrates how the
placement of a device in a fabric can impact scalability. Scenario “A” (Local Attach) in Figure 3-14 depicts a disk system
attached to the same switch as the host that needs to access it. This is an effective configuration, because it eliminates the need
to manage ISL over-subscription. This configuration is useful when most traffic can be localized and congestion is a greater
concern.
Scenario “B” (Core Attach) depicts the case where not all ports on the core are being used by ISLs, and the storage device is
directly attached to the core. This configuration has two impacts. First, the number of available ports in the SAN is reduced
because core ports are no longer available for connecting additional switches. This means that the connection of a single
device to the core could reduce the potential size of the SAN by 128 ports or more. A high performance device that is accessed
by many other devices (i.e. a storage device), may realize some performance benefit in connecting to the core. This is because
the many devices can spread their load across many ISLs/trunks as opposed to only a few ISLs/trunks if that device were
placed on the edge. In Figure 3-14, scenario “B”, if there were many hosts connected across the edge switches, these hosts
would spread their load across eight ISLs, instead of two ISL as shown in scenario “C”.

Scenario “C” (Edge Attach) is the typical case. The number of available paths between the host and storage is two. In addition,
the core switch ports are available for increasing the size of the SAN by adding new edge switches. The hop count from the
server to the storage is 2-hops. Note that hop latency is two microseconds per hop – an inconsequential latency when
compared to disk I/O, which is measured in milliseconds.
Guideline: Connect devices to a core switch after validating your scaling requirements. Attaching devices to a core
switch limits the size of your SAN, but can result in higher performance.
Figure 3-14 How Device Placement Can Impact Performance and Scalability in a Core/Edge Fabric
A Core/Edge fabric built with two 16-port core switches, while maintaining a 7:1 ISL oversubscription ratio, can scale to 224
user ports. Using two 64-port core switches and maintaining a 7:1 ISL oversubscription ratio, a Core/Edge fabric can scale to
896 user ports and if four 64-port switches are used in the core, a fabric can grow to 1792 user ports. If a higher ISL
oversubscription ratio of 15:1 is used, a fabric built with two 64-port core switches can grow to 1920 user ports and 3840 ports
using four 64-port core switches. The connection of devices to the core lowers the potential size of these SANs. If the fabric
size is expected to exceed 896 user ports, to maintain the flexibility to scale performance and the size of a fabric or if it is
expected to connect devices into the core, it is recommended that four SilkWorm 12000 or 24000 switches be utilized in the
core.
3.1.6. Tiering (Low Locality Device Attachment)

Tiering is the process of grouping particular devices by function and then attaching these devices to particular switches or
groups of switches based on that function. Tiering is the opposite of high locality: in a highly localized SAN, hosts are attached
to the same switches as their storage devices; in a tiered SAN, hosts are rarely attached to the same switches as storage arrays.
It requires some level of effort to plan and manage the layout of a fabric for optimal locality. Sometimes this effort is not
necessary if there is a sufficient level of available ISL/trunk bandwidth. For example, if it is known that the peak bandwidth
that a host generates is 10 MB/sec and there are fourteen hosts on a switch, it is sufficient to only have one ISL (2 Gbit/sec)
connecting that switch to the remainder of the fabric and tiering is a viable design option. However, if those hosts generate 50
MB/sec concurrently, it is probably more appropriate to adopt a device attachment strategy that involves a high degree of
locality, or to use more ISLs.
From a cabling and maintenance perspective, tiering is quite effective. In Figure 3-15, a group of switches is designated as the
storage switch group, another group designated as the tape group, and a final group is designated as the host group. When it
becomes necessary to expand backup, storage, or hosts, it becomes a straightforward effort to attach the new devices to an
open port on the appropriate tier and to then enable access (i.e. zoning, configure hosts). If a particular tier requires expansion,
add a new switch to that group.

Host Tier
Core Core
Storage Tier Tape Tier
Figure 3-15 A Tiered SAN
The performance characteristics of a Core/Edge fabric make this topology an excellent candidate for tiering (see the discussion
in section 2.8.1.5. Core/Edge on page 2-29 for the reasons why). Also, note the flexibility to increase bandwidth between
devices by adding ISLs to account for varying performance requirements. It is not required to deploy an entirely tiered
architecture. For performance reasons, it may be desirable to establish a hybrid of tiered switches and some switches that are
not tiered. For example, it may be appropriate to connect a high performance host and storage device on the same switch while
maintaining a tiered approach for the other devices in the SAN.
An interesting aspect of a tiered SAN is the visual layout of the switches in the SAN architecture. Note that the two SANs
depicted in Figure 3-16 are identical: each SAN is built with the same number of switches, number of ISLs, ISL connection
points, and device attachment points. The only difference is how the switches are laid out in the figure.
Storage Tier Host Tier Tape Tier
Host Tier
Figure 3-16 Two different graphical representations of the same SAN

In Figure 3-17, the SANs have the same number of switches, number of ISLs, and ISL connection points; however, the device
connection points are different, as the core switches are utilized for device attachment. These two SANs are topologically
identical, but functionally different. Scalability applies as discussed earlier in section 3.1.5. Connecting Devices To The Core
on page 3-13, when attaching devices to the core scalability is diminished. However, for many to one SAN devices — a single
SAN device that is accessed by many other SAN devices, such as a storage device — connecting devices to the core offers
some performance advantages. The top SAN in Figure 3-17 is sometimes called a two-tier SAN and the bottom SAN is
sometimes called a three-tier SAN. The device attachment points, not the layout of the switches, differentiate a two-tier SAN
from a three-tier SAN.
Host Tier
Host Tier
Figure 3-17 Same Topology, but Different Device Attachment Strategies

3
3.2. Platform Specific Design Considerations

Several design considerations such as switch role, usage, availability, and scalability capabilities are covered in this section.
All SilkWorm switches can function as standalone switches or as part of a fabric. When participating in a fabric, some
switches are better suited for the core than others – depending upon factors such as scalability and availability. All SilkWorm
switches can perform in the role of an edge switch; however, depending upon requirements, some SilkWorm switches make
better edge switches than others. When building big fabrics, use high port count switches.
Guideline: Significant scalability enhancements, such as improved performance of zone checking, RSCN
distribution, and name server, along with improved quality and new functionality are incorporated into
Fabric OS versions 2.6.1, 3.1, and 4.1 and later versions. It is recommended to utilize these versions of
Fabric OS at a minimum and it is preferred to run versions 2.6.2, 3.1.2, and 4.2 versions of Fabric OS.
Guideline: For large SANs that exceed several hundred devices, it is necessary to run at a minimum Fabric OS
versions 2.6.1, 3.1, and 4.1 or later versions.
Note: The term Control Processor is associated with a SilkWorm 24000 and 12000 component/FRU (field replacable unit).
The SilkWorm 2000 series, 32x0, 38x0, and 3900 series switches do not have a FRU specifically associated with it and
when CP is used in the context of other SilkWorm switches, the reference is to the switch CPU and not a FRU.
3.2.1. SilkWorm 2000 Series, 32x0, and 38x0 Switches

Since its inception, Brocade has been the leader in developing technology and testing practices to expand the limits of Fibre
Channel fabrics. As fabrics increase in size, the numbers of switches, inter-switch links (ISLs), and edge devices increase
rapidly. This in turn increases the demand on the fundamental computing resources of the Control Processor in each switch, as
it must rapidly complete tasks such as processing Zoning configuration updates distributed by other switches, analyzing and
distributing RSCNs, responding to Name Server queries from hosts logging into the switch, etc. Brocade testing of fabrics
upwards of 1,300 total ports with Fabric OS v3.1.0 and v4.1.0, indicates the SilkWorm 2000 family of switches has
approached the limits of its supported capacity. For this reason, the maximum supported size for fabrics containing SilkWorm
2000 family switches remains 728 user ports (see Appendix C, Glossary for the definitions of user ports and total ports). This
limitation is not expected to change in future releases. Customers planning to build larger fabrics should plan on implementing
them solely with the Brocade 2 Gbit/sec switch family. Brocade Fabric OS v3.1.0 and v4.1.0 releases allow the SilkWorm
2 Gbit/sec switches to scale well past this 728 port limit. For fabrics needing to scale beyond 728 user ports, it is recommended
to build those fabrics with SilkWorm 24000, 12000 and 3900 switches, as the total cost of ownership and efficiencies of
building large fabrics with large switches are significant. Additionally, the processing power and memory of the SilkWorm
24000, 12000 and 3900 are significantly better than the SilkWorm 3800 or 3200 switches.
Guideline: Do not exceed 728 user ports in a fabric that uses SilkWorm 2000 series switches.
The SilkWorm 3250 and SilkWorm 3850 utilize a different and more powerful control processor than the legacy SilkWorm
3200 and 3800 switches. Additionally, the SilkWorm 3250 and 3850 are configured with more memory than the SilkWorm
2000 and 3800 switches. These additional resource position the SilkWorm 3250 and 3850 to function in larger fabrics;
however, the trade-offs of managing numerous lower port count switches versus fewer high port count switches need to be
considered as part of the SAN design and deployment planning process.

3
3.2.2. SilkWorm 24000, 12000 and 3900

The SilkWorm 12000 is currently capable of supporting up to two domains per chassis, while the SilkWorm 24000 currently
supports only one domain per chassis. Using one fabric per SilkWorm 12000 chassis prevents the same Fabric OS from
populating two fabrics (see Figure 3-18). It is not desirable to populate both fabrics of a redundant fabric SAN with the same
Fabric OS at the same time during an upgrade, since a human error or software error could cause both fabrics to be impacted.
Once a new version of Fabric OS is loaded and validated, then the new version of Fabric OS can be loaded on the other fabric.
If there are two fabrics on the same chassis, it is not possible to sequentially load a new version of Fabric OS. One fabric per
SilkWorm 12000 chassis limits operator error to a single fabric. One fabric per SilkWorm 12000 chassis is simpler to manage,
as it is not necessary to account for both fabrics when doing administration. Separation and compartmentalization are
fundamental to mitigating single points of failure. A single chassis effectively joins the two fabrics from a risk perspective.
When a chassis participates in more than one fabric, there exists a potential to introduce faults into both fabrics. With one
fabric per chassis, this risk does not exist.
All SilkWorm switches, except the SilkWorm 24000 and 12000, implement a fixed chassis design. The SilkWorm 24000 and
12000 offers a bladed design, which enables the non-disruptive expansion of ports on the chassis as well as hot swap of the CP
and switch port cards.
Guideline: Use only one fabric per SilkWorm 12000 chassis.
Note: The SilkWorm 12000 currently supports up to two domains per chassis, while the SilkWorm 24000 currently supports
one domain per chassis.
Note: With Secure Fabric OS, only one fabric per SilkWorm chassis is supported.
Guideline: For maximum fabric scalability and availability, it is highly recommended to use FOS 2.6.1, 3.1 or 4.1
or later on all Brocade SilkWorm switches.

3
A A B B
A B A B Two Fabrics Per Chassis
A A B B
One Fabric Per Chassis A A B B
Figure 3-18 One Fabric per SilkWorm 12000 Chassis is Recommended
Hot code activation (HCA) is one of the features in Fabric OS 4.1 and later versions. The defining characteristic of hot code
activation is that while a new firmware image is being activated on a switch there is no disruption of end-to-end data flow
between the hosts and storage devices. No disruption means no dropped frames, no retries, and no time-outs. The ASICs on
the switch continue to process frames while the new firmware is being activated. Hosts that are logged into their targets will
never be aware that anything has happened. This lack of end-to-end data disruption can be achieved with Fabric OS 4.1 on the
SilkWorm 24000, 12000, 3900, 3850, and 3250.
A redundantly configured SilkWorm 24000 and 12000 can pass control to its standby Control Processor as part of the
firmware activation. Critical fabric services are available in about 2 seconds, with all fabric and management services fully
available in less than ten seconds. The SilkWorm 3900, 3850, and 3250, with only one Control Processor, must shutdown and
reboot Fabric OS as part of the HCA process. This process completes in less than 60 seconds, with the critical fabric services
available in about 50 seconds. Again, although there is a window when fabric services are unavailable, there is no disruption to
end-to-end data flow between SAN devices.
Because it takes longer to do hot code activation on the SilkWorm 3900, 3850, and 3250, the switches directly linked to the
SilkWorm 3900, 3850, or 3250 need to be tolerant of the 60 seconds when fabric services from the SilkWorm 3900, 3850, or
3250 are not available. Brocade has made the necessary modifications to Fabric OS v3.1.0 and v2.6.1 and later versions to
extend the time-out values so that link and fabric re-configuration is avoided.
Guideline: It is strongly recommended that customers deploy neighboring switches (i.e. immediately connected via
an E-port) to the SilkWorm 3900, 3850, and 3250 with Fabric OS v2.6.1 (or later) on the SilkWorm 2000s
series, Fabric v3.1.0 (or later) on the SilkWorm 3200/3800, or Fabric v4.1.0 on the SilkWorm
3250/3850/3900/12000/24000.

3
During the SilkWorm 3900, 3850, or 3250 hot code activation, if earlier releases (Fabric OS v2.6.0x, Fabric OS v3.0.2x, or
v4.0.x) are deployed on the neighboring switches to a SilkWorm 3900, 3850, or 3250, a time-out will occur on these
neighboring switches resulting in a fabric re-configuration.
The SilkWorm 3900, 3850, and 3250 HCA procedure should always be performed when the fabric is stable. However
recovery actions will be taken upon completion of the HCA to ensure that no RSCNs, or fabric configuration changes are
missed. If any new hosts or targets were added to the SilkWorm 3900 switch during the HCA reboot time, then the initial
FLOGI will time out. After the reboot, the switch will reset that port to cause the FLOGI to happen again. If any device is
added elsewhere in the fabric, the RSCN will be delivered after the reboot completes. Finally, if any E-port cables are pulled,
or the fabric rebuilds for any reason, then after the HCA reboot completes, the SilkWorm 3900, 3850, or 3250 will cause the
fabric to rebuild again so that it may participate in the fabric.
Guideline: To prevent unnecessary disruptions in the fabric when firmware is activated on a SilkWorm 3900, 3850,
or 3250, it is recommended that any switches directly connected to the SilkWorm 3900, 3850, or 3250
use Fabric OS v2.6.1, v3.1.0, or v4.1.0 (or subsequent versions).
3.3. Zoning Design Considerations & Guidelines

Zoning is an important element of a secure and healthy SAN. Zoning does have an impact on SAN designs. For an overview of
how zoning works refer to the Brocade Zoning User’s Guide Version 3.1/4.1 (publication number: 53-0000523) and the white
paper titled Brocade Guide to Understanding Zoning (part number: 53-0000213), these documents also provide guidelines for
implementing zoning. This section highlights key elements of zoning that relate to a SAN design.
3.3.1. Zoning and Scalability

Zoning optimizes fabric services, such as RSCN distribution and name server response, and limits unnecessary device
discovery. With the new zoning and related name server changes in Fabric OS 3.1 and 4.1 and subsequent releases, zoning
becomes necessary for the proper functioning of large fabrics. For instance, the distribution of RSCNs (registered state change
notifications) is reduced to only devices affected by a zone change. In prior releases of Fabric OS 3.x, 4.x, and all versions of
Fabric OS 2.x, a zone activation (for example, executing the command cfgEnable) resulted in an RSCN being distributed to
all devices – regardless of whether these devices were affected by a zone change. Additionally, not using zoning results in
unnecessary delays during device discovery for some hosts, especially when a host pointlessly authenticates with hundreds of
devices. These delays can last minutes, pause ongoing I/O, and cause unpredictable behavior on a host. Use of zoning on the
switches limits the number of devices visible to a host and eliminates this host-based scalability problem.
Guideline: The implementation of zoning is recommended for any SAN and especially critical for any large fabric.
Zoning is fundamental to the functioning of multi-hundred port fabrics.
With Fabric OS v3.1/v4. 1 and later, zoning changes cause different RSCN (Registered State Change Notification) behavior. In
Fabric OS v3.1/v4. 1 and later, when zone changes are enabled or disabled, fabric RSCNs are only sent to devices that
completed an SCR (State Change Registration) and that are in the affected zones. In all Fabric OS v2.x releases, the locally
connected devices that completed an SCR will receive these RSCNs, regardless if the device is affected by a zone change.
With a mixed fabric, the devices in the zones that are affected, as well as all devices local to the Fabric OS v2.x switches,
receive an RSCN. The RSCN filtering of a device is handled by the Name Server of the switch to which it is attached. The
Fabric OS of the switch that originates a zoning change is irrelevant.
Guideline: Make certain devices which require RCSN suppression are directly attached to a switch running Fabric
OS version 3.1 or 4.1 or higher.

3
3.3.2. Zoning Database Size

Zoning consumes a finite amount of processing and memory resources. As the number of devices in a SAN grows, so do the
demands on these same resources. The zoning implementation is optimized to minimize processing resources and leverage
ASIC capabilities as much as possible. The zoning database size for SilkWorm 2000 series, 3200, and 3800 switches is 96 KB
and 128 KB for SilkWorm 3250, 3850, 3900, 12000, and 24000 switches. To check the size of a zone database, use the
command cfgSize. A switch with a zoning database size limit of 96 KB limits the size of the zoning database for the
whole fabric – even if a SilkWorm 3250, 3850, 3900, 12000, or 24000 switch is present in the fabric. As the size of a SAN
grows, it is important to monitor the zoning database. Typically, the zone database size needs to be of concern as the size of a
SAN exceeds several hundred ports. The size of an alias name, zone name, or configuration name is limited to 64 characters
for Fabric OS versions 2.6.1, 3.1, and Fabric OS 4.1 or later releases. While it is possible to create 64-character zone, alias, or
configuration names, doing so consumes more memory than a shorter name. Additionally, shorter names are easier to
remember and less prone to typing errors. Be wary of sacrificing meaning for shortness. See the whitepaper Zoning
Implementation Strategies For Brocade SAN Fabrics for effective guidance for naming aliases, zones, and configurations. The
variable size of zone objects makes it very difficult to state guidelines as a number of zone entries or alias. A zone database
size is similar to disk storage. The usage is not measured so much by how many files are located on the storage, but by the
amount of space taken up by the files. It is recommended to review the zoning configuration of a fabric periodically for unused
alias, zoning, and configuration entries and to then delete these unnecessary entries. Unnecessary alias, zone, and
configuration entries frequently result from the merging of fabrics or the addition of a switch with predefined zones into an
existing fabric.
Note: The maximum zoning database size for SilkWorm 2000 series, 3200, and 3800 switches is 96 KB and 128 KB for
SilkWorm 3250, 3850, 3900, 12000, and 24000 switches. A switch with a zoning database size limit of 96 KB limits
the size of the zoning database for the whole fabric to 96 KB.
Guideline: Limit the name of an alias, zone or configuration to as few characters as possible while maintaining
meaning of that name. Target 16-characters or less for an alias, zone, or configuration name.
Guideline: For SANs that exceed several hundred ports, monitor the size of the fabric-zoning database with the
command cfgSize.
Guideline: Routinely review a zoning configuration to identify unused aliases, zones, and configurations and then
remove these unused entries.

3
3.4. Designing SANs With Secure Fabric OS

A secured fabric must be entirely secured and all switches in a secured fabric must run a version of Fabric OS that supports
security and these switches must be licensed to run security. Scalability testing, which is the testing performed by Brocade of
fabrics consisting of hundred and thousands of device ports, with fabrics that have secure mode enabled and with fabrics that
have secure mode disabled.
Fabrics consisting solely of SilkWorm 2000 series switches need to run a minimum version of Fabric OS v2.6.0. If it is desired
to run with secure mode enabled in fabrics containing SilkWorm 2000 series switches and any of the following SilkWorm
switches: 3200, 3800, 3900, 12000, the minimum version of Fabric OS is V2.6.1. Secure Fabric OS was introduced for the
SilkWorm 3200 and 3800 switches in Fabric OS v3.1 and v4.1 for SilkWorm 3900 and 12000 switches. SilkWorm 3250, 3850,
and 24000 switches need to run Fabric OS version 4.2 or higher to utilize secure mode.
Note: The following minimum version requirements apply for fabrics that need secure mode enabled:
SilkWorm 2000 Series
- Fabric OS version 2.6.0 (or higher) with the Security license - for fabrics containing only SilkWorm 2000 series
switches
- Fabric OS version 2.6.2 (or higher) with the Security license - for secure fabrics containing a SilkWorm 2000
series switch and any of the following SilkWorm switches: 3200, 3250, 3800, 3850, 3900, 12000, 24000.
-SilkWorm 3200 and 3800 switches
- Secure Fabric OS version 3.1 (or higher) with the Security license
-SilkWorm 3900 and 12000 switches
SilkWorm 3250, 3850, and 24000 switches
Note: In a fabric that contains SilkWorm 2000 series switches, the maximum security DB size is limited to 32 KB, with
16 KB active. In a fabric containing SilkWorm 32x0, 38x0, 3900, or 12000 switches, the security DB size maximum
is 128 KB, with 64 KB active. For all fabrics, the maximum number of DCC policies is limited to 620. Use the
command secactivesize to monitor database usage.
Guideline: It is highly recommended to upgrade to the latest version of Fabric OS compatible with your switch, prior
to introducing a SilkWorm 24000 to your SAN. Contact your switch vendor for details.
To run with secure mode enabled, designate one or more switches as the Fabric Configuration Server (FCS). FCS switches are
“trusted” switches and are used for managing fabrics where secure mode is enabled. These switches should be both
electronically and physically secure. You can specify a Primary FCS switch and one or more Backup FCS switches, to provide
failover ability should the Primary FCS switch fails. All management access to the fabric must flow through the Primary FCS
switch. Should the Primary FCS switch be unavailable, it then becomes necessary to use the first available Backup FCS switch
for managing the fabric. Please reference Brocade Secure Fabric OS® User’s Guide Version 3.1 / 4.1 (part number:
53-0000526-02) and the Brocade Secure Fabric OS® Quickstart Guide Version 2.6.1/3.1 / 4. (part number: 53-0000352-02)
for further detail about Secure Fabric OS. The ability exists to use the Backup FCS via automatic failover or manually. In case
where a Backup FCS is utilized manually, the user can pick any switch designated as a Backup FCS. In cases of automatic
failover, the first Backup FCS switch designated will be used.
The Primary FCS switch is a central point for distributing fabric configuration information and management changes.
Establishing the core switch in a Core/Edge fabric as the Primary FCS is recommended since the core switch is optimally
located to communicate with all other switches in the fabric. There are several other reasons, which are discussed in section
3.5. Switch Location In The Fabric on page 3-24, for configuring one of the core switches as the Primary FCS. For the same

3
reasons it is also recommended to establish the other core as the Backup FCS. Note that some Core/Edge topologies utilize
three and even four cores. A second Backup FCS switch is also recommended and this switch should be an edge switch. One
Backup FCS is switch is necessary in a Core/Edge topology since a single switch failure does not cause total failure of the
fabric, so means to manage the fabric should a single switch fail should exist. Conceivably all the remaining switches could
also be established as Backup FCS switches. There is a fine balance between being overly cautious and creating unnecessary
complexity. For this reason a total of two Backup FCS switches are recommended. Figure 3-19 is an example implementation
of the recommendations for quantity, type, and location of FCS switches in a secure fabric. If a fabric spans multiple sites,
locate at least one Backup FCS at each site.
Guideline: Use one of the core switches in a Core/Edge topology as the Primary FCS switch for secure fabrics.
Guideline: In a Core/Edge topology, designate at least two additional switches as Backup FCS switches. It is
recommended to use the other core as the first Backup FCS and an edge switch as the second Backup
FCS.
Guideline: When deploying a secure fabric that spans multiple sites, ensure that there is at least one Backup FCS
at each site.
Primary FCS Backup (1) FCS
Backup (2) FCS
Figure 3-19 Recommended Location of Primary and Backup FCS Switches in a Core/Edge Topology

3
3.5. Switch Location In The Fabric

The SilkWorm family of Brocade switches offers a variety of port counts, availability levels, and performance. The SAN
designer has multiple options for placing a new switch into an existing fabric or creating fabric with new switches. Several
variables influence switch location in the fabric, including management, control, switch speed, availability, interoperability,
and availability. These variables are discussed in this section with recommendation and guidelines relating to:
• Which switch to use for fabric management
• Where to locate 2 Gbit/sec switches as they are integrated into an existing fabric of 1 Gbit/sec switches
• Where and what switches to place to match fabric availability with overall availability requirements
3.5.1. Management and Control

It is important that the SAN administration team select one switch in the fabric as the administration switch. Using one switch
for access lessens the possibility of multiple administrators making changes to different switches in the fabric at the same time.
Keeping in line with the terminology used by Secure Fabric OS, this switch is termed the primary management switch. There
are several reasons to select the core switch as the primary management switch for zoning, time services, Fabric Manager, Web
Tools, and general administrative access.
The core switch is optimally located to communicate with all other switches in the fabric. The core switch is commonly the
larger switch with a more powerful control processor. When zoning information is propagated, the switch where the zone
information changed is responsible for distributing the zone information to all other switches in the fabric. The core switch is
directly connected to all other switches in the fabric. Typically, the core switch is a later model switch providing the
administrator access via the latest Web Tools interface. The primary management switch can also be used by Fabric manager
as the main access point for that fabric. Once selected, all the other portions of Fabric manager will primarily access this
switch for fabric information. The ability to synchronize time within a fabric is available as of Fabric OS versions 2.6.1, 3.1,
and 4.1. All switches, in a fabric where secure mode is not enabled, synchronize their time with the principal switch in the
fabric.The principal switch in the fabric can synchronize its clock with an NTP timeserver, by identifying the timeserver to the
principal switch with the tsclockserver command. In a fabric where secure mode is enabled, switches synchronize time
with the Primary FCS, which may or may not be the principal switch.
Guideline: Select a core switch in a Core/Edge topology as the primary management switch for zoning, time
services, Fabric Manager, Web Tools, and general administrative access.
The establishment of a principal switch in a fabric can vary based on upon the state of the fabric, switch WWN, and whether
other switches/fabrics are merging into that fabric. A switch that is the principal switch in a fabric today may not be a principal
switch after a new switch or switches are added to that fabric or if that fabric reconfigures. The implementation of the
fabricprincipal command is based solely on mechanisms specified in the Fibre Channel standards. These mechanisms
provide a preference for a switch requesting to be the principal switch in a fabric, but they do not provide an absolute
guarantee that a switch requesting to be the principal switch will actually achieve this status.
Note: Note that the fabricprincipal is only available in Fabric OS version 4.1.0 or later.
Guideline: Identify the primary management switch in the fabric as the preferred principal switch by using the
command fabricprincipal.

3
With the use of secure Fabric OS, the management of a fabric must be originate from the Primary FCS switch and should the
Primary FCS switch be unavailable, a designated Backup FCS switch can be used. The use of a primary switch as a focal point
for SAN management is a guideline, and any switch can be used for these purposes. For the reasons stated in this section, it
makes sense for the core switch to be the primary switch for management purposes. Should the primary switch fail in a
non-secured fabric (i.e. a fabric not running Secure Fabric OS), then it is suggested to utilize the remaining core as primary
backup.
The primary management switch can also be used as an access point for management server access, access by SNMP software
that polls for fabric status, a focal point for fabric related SNMP traps, and as an access point for SAN management software.
Figure 3-20 summarizes the recommended location of a primary switch in a non-secured fabric, the recommended functions
that the primary should perform, and the location of a backup management switch.
Primary Backup
Management Management
Switch Switch
Management Switch Functions:
9 Zoning 9 Time Server for the fabric

9 Webtools 9 Fabric Manager
9 Principal switch 9 General administration
Figure 3-20 Primary and Backup Management Switch Location and Functions in a Core/Edge Topology

3
3.5.2. 2 Gbit/sec Switch Placement

When designing a SAN with 2 Gbit/sec switches, the similar guidelines that apply to trunking also apply to 2 Gbit/sec
capabilities. Place these switches adjacent to each other to take advantage of 2 Gbit/sec ISLs. Of course, it is also possible to
connect a SilkWorm 2000 series switch to a trunking capable switch, as Brocade Trunking capable switches are backwards
compatible and will negotiate a 1 Gbit/sec ISL.
For Core/Edge topologies, place 2 Gbit/sec switches in the core. If 2 Gbit/sec connectivity is required, it is acceptable to attach
these devices to the 2 Gbit/sec cores if 2 Gbit/sec edge switches are not yet implemented. By placing 2 Gbit/sec switches in the
core, it ensures that a 2 Gbit/sec path exists end to end. If a significant number of 2 Gbit/sec devices are required and the
performance requirements are high, an effective strategy is to localize the 2 Gbit/sec devices on the same switch or group of
switches.
Figure 3-21 depicts 2 Gbit/sec devices with and without 2 Gbit/sec end-to-end paths as well as the localization of some
2 Gbit/sec devices.
Guideline: Place 2 Gbit/sec switches adjacent to each other and for Core/Edge topologies, place the first 2 Gbit/sec
switches in the core and subsequent 2 Gbit/sec switches at the edge.
3800 2800 3800
Not
2Gbit/sec
2 Gbit/sec
end to end
2800 3800 end to end
3800 3900 3900
Localize 2 Gbit/sec dev ices

1 Gbit/sec ISL
2 Gbit/sec connect
4 Gbit/sec Trunk
Warning: This example is presented to establish the benefits of placing a 2 Gbit/sec

switch in the core. This example is a poor design by choice and is not something that a
designer should implement unless there are compelling reasons to do so.
Figure 3-21 End to End 2 Gbit/sec Paths and 2 Gbit/sec Device Locality

3
3.5.3. Locating A Switch For Fabric and I/O Availability

The type of switch and placement of that switch in a fabric impacts availability of the fabric and I/O operations. While
redundant fabrics enable redundancy, failing a path or fabric is the last thing many SilkWorm users want to do, as maintaining
continuous I/O is paramount. An effectively designed SAN can match variable availability requirements for storage and hosts.
Enabling the SAN administrator to offer different levels of availability provides significant flexibility.
The SilkWorm 3250, 3850, 3900, 12000, and the SilkWorm 24000 are capable of non-disruptive operation during hot code
activation when running Fabric OS 4.1 or subsequent versions. The SilkWorm 12000 and 24000 are capable of non-disruptive
operation during a failover. A failover can be initiated manually or happen automatically due to the detection of a fault, such as
a control processor failure. While a new firmware image is being activated on a switch or during a failover there is no
disruption of end-to-end data flow between hosts and storage devices. No disruption means just that – no dropped frames, no
retries, no time-outs. The SilkWorm 2400, 2800, 3800, 3900, 12000 and 24000 all implement hot swap and redundant power
and cooling. The SilkWorm 12000 and 24000 are the only switch that implement redundant control processors. This means
that the SilkWorm 12000 and 24000 are the only SilkWorm switches capable of operating in the event of a control processor
failure. The SilkWorm 12000 and 24000 provide the highest availability in the SilkWorm family of switches. The next highest
level of availability offered by a SilkWorm switch is provided by the SilkWorm 3900. Table 3-1 ranks the SilkWorm family of
switches by availability levels.
Table 3-1 SilkWorm Switch Availability Levels
SilkWorm Level Of Availability Features

Switch Availability
SilkWorm 12000 Ultra High Redundant control processors, Non-disruptive failover, Non-dis-
and 24000 ruptive port expansion, hot swap WWN card, hot code activation,
redundant and hot swappable power and cooling
SilkWorm 3900 Very High Hot code activation, redundant and hot swappable power and cooling
SilkWorm 3850 Very High Hot code activation, redundant power and cooling
SilkWorm 3800, High Redundant and hot swappable power and cooling
2800, 2400
SilkWorm 3200, Medium No redundant or hot swappable power and cooling. SilkWorm 3250
3250, 2x00, 20x0 implements hot code activation
A dual Core/Edge fabric SAN is redundant and resilient. A Core/Edge SAN is resilient and can sustain operations in the event
that a single core fails. I/O operation can continue when an edge switch fails if a redundant fabric with multipathing software
is implemented. From an availability perspective, any SilkWorm can perform as a core switch in a Core/Edge fabric. To
eliminate disruptions in the fabric and to I/O due to firmware activation or during a failure, a SilkWorm 12000 or 24000 is
recommended. The SilkWorm 12000 and 24000 offer the highest availability and up to 128-ports per chassis. If scaling
requirements do not dictate a several hundred port SAN, consider still using the SilkWorm 12000 or 24000 in the core,
populating the core remaining ports with devices that require the highest availability of I/O or the performance of a core
connection, and connecting devices that require lower availability to lower availability edge switches. Select and connect edge
switches based upon the SAN device availability requirements, as shown in Figure 3-22.
Guideline: For the highest fabric and I/O availability, deploy SilkWorm 12000 or 24000 switches in the core and
either SilkWorm 3250, 3850, 3900, 12000 or 24000 switches on the edge of a Core/Edge fabric. SAN
devices requiring lower availability can connect to SilkWorm 2000 series, 3200, and 3800 edge
switches.

3
Guideline: Using two separate SilkWorm 12000 or 24000 chassis in the core increases availability even further.
High Availability Very High Ultra High

Availability Availability
12000 or 12000 or
24000 24000
3800 3800 2800 2800 3850 3900
Note: For Ultra High 12000 or 12000 or

availability, utilize two 24000 24000
SilkWorm 12000/24000
chassis in the core.
Ultra High Availability
Figure 3-22 Connect Devices According to Availability Requirements

3
3.6. Scalability Support and Testing

Brocade is currently testing and validating fabrics consisting of thousands of ports. The ultimate limitation in a fabric design
today, as defined in the Fibre Channel standards, is a maximum of 239 physical switches, whether they are 8, 16, 64 or
128-port versions. As a practical matter, no vendor has yet tested networks of this size due to the expense and complexity of
implementing such a network. The current practical switch-count limit is lower than 239 switches, based upon empirical
testing. Other limits on a SAN design are the count and number of ISLs/trunks and the size of the zoning database.
Brocade partners with many OEMs and resellers who supply switches to end-users. Many of these partners provide direct
support for the switches they sell. These partners extensively test and support specific configurations, including switches,
storage, HBAs, and other SAN components. In some cases, the large fabric configurations supported by Brocade partners will
differ from the guidelines presented in this document. In fact, several Brocade switch partners have developed their own SAN
design guidelines, including in-depth support matrixes that specify support configurations and firmware versions.
To determine whether or not a SAN is supported, it is necessary to work with your support provider to determine if your SAN
design is valid. Important variables that determine the supportability of a particular SAN are the number of switches, version
of Fabric OS, number of ISLs/trunks, number of connected devices, and the hop count.
The Brocade approach to testing fabrics consisting of hundreds and even thousands of ports establishes an effective balance
between quality and building a solid foundation from which a wide range of fabric configurations can be derived. Brocade
scalability testing validates a range of ISL oversubscription ratios (from 3:1 to 15:1 or higher), mixture of switches, hop counts
(up to 7), and ISL densities. Many customers implement varying degrees of ISL oversubscription in the same fabric based
upon their performance needs. Some customers out of necessity deploy SANs that are composed of loosely connected
Core/Edge islands. Some customers require lower performance and low cost per port, while other customers need high
performance and are willing to pay a premium. The value in the current testing is the ability to offer SilkWorm users very
simple support determination metrics that enable the deployment of a wide-ranging, robust, and flexible set of topologies. In
Figure 3-23 is a sample of the type of configuration Brocade tests. Note that a range of ISL oversubscription ratios, trunk sizes,
number of ISLs/trunks, switches, switch type, and hop count are part of the configuration. Testing such a configuration
validates a wide variety of fabrics and derivatives and validates the backwards compatibility of Brocade products.
7:1 3:1 7:1 3:1 7:1 3:1 7:1 3:1
7:1 7:1 7:1
y 2 x 17 + 2 x 14 — 35 switches total
y 760 usable ports
1 ISL
38x0
2 ISLs
3900
4 ISLs
12000 or 24000
8 ISLs
Figure 3-23 Sample Brocade Scalability Test Configuration

3
3.7. Recommended Fabric Topologies and SAN

Designs
When building big fabrics use big switches. A list of guidelines and recommended topologies follows. Port counts in this
section are specified as user ports, meaning that ISLs ports are subtracted from the total number of ports in the fabric.
Guideline: When designing a fabric that exceeds 728 user ports or is expected to grow past 728 user ports, it is
suggested to exclusively utilize SilkWorm 3900, 12000 and 24000 switches in the core and the edge
since it is felt to be easier to manage fewer large switches versus many smaller switches.
Guideline: Use four SilkWorm 12000 domains or two SilkWorm 24000 domains in the core of a Core/Edge topology
if the size of the fabric is expected to exceed 896 user ports or a large number of devices are going to
be connected to the core.
Guideline: The practical limit for a simple Core/Edge fabric using 16-port core switches is 224 user ports. Use a
SilkWorm 24000, 12000 or 3900 in the core if the Core/Edge fabric is expected to exceed 224 user
ports.
The fabric topologies recommended are all Core/Edge topologies. The formats for these recommendations are templates. The
SAN designer can use these templates to adapt to meet their requirements. The key is to map these requirements to a supported
SAN design. The definition of whether or not a SAN design is supportable is up to the support partner and there are several
variables involved with determining this support (see section 3.6. Scalability Support and Testing on page 3-29). Six
recommended fabric topologies are provided:
Highest Availability Fabric Topology: A fabric built with the most highly available switches, implementing a 7:1 ISL
oversubscription ratio, and offering the highest fabric and I/O availability (see Figure 3-25).
Low Cost Per Port Fabric Topology: A fabric topology built with high ISL oversubscription ratios (low performance,
fewer ports used for ISLs), resulting in a higher number of user ports (see Figure 3-26).
High Performance Fabric Topology: A fabric built with low ISL oversubscription ratios (high performance, more ports
used for ISLs), resulting in a lower number of user ports (see Figure 3-27).
Very Large Fabric Topology: A fabric topology using two SilkWorm 24000 cores that is capable of scaling to 4352/3840
total/user ports (see Figure 3-28).
Small Fabric Topology: A fabric that uses smaller switches and is capable of scaling to 288/224 total/user ports (see
Figure 3-29).
Extended Distance Fabric Topology: A fabric topology recommended for connecting SANs that span multiple sites and
where the availability of connections between sites is limited (see Figure 3-30).
A topology can be selected based on cost, performance requirements, availability requirements, or scaling requirements. Once
a topology is selected, it is up to the SAN professional to determine the quantity and type of edge switches, locality model, and
to customize the ISL oversubscription ratios.

3
Note: All Core/Edge topologies can start small (see Figure 3-24) and grow as large as the maximum port counts specified.
In some topologies, the maximum port count is specified as a range, since this count is a function of the ISL
oversubscription ratio implemented and the type of switch utilized.
0 or more 0 or more 3800 0 or more
2000 Series
38x0 or 32x0
1 ISL
3900
2 ISLs
Figure 3-24 A Four Switch Partial Mesh is the Start of Every Core/Edge Fabric Topology
Guideline: Regardless of selected Core/Edge fabric topology, it is recommended to implement the selected
topology as a dual fabric SAN.
Note: Use these topology templates in conjunction with your support provider to develop a supported SAN design. The port
counts indicate what the topology is capable of scaling to, not what a support provider will support.
Note: The SilkWorm 12000 switches represented in Figure 3-25, Figure 3-26, Figure 3-27, and Figure 3-28 represent logical
switches. Each switch could reside in a chassis by itself or with another SilkWorm 12000 logical switch in the same
chassis.

3
0 or more 12000 / 24000

0 or more 3900
y Up to 2304 total ports

3900 2 ISL y Up to 1792 user ports
12000 4 ISLs y 7:1 ISL oversubscription ratio
24000
Figure 3-25 Highest Availability Fabric Topology
0 or more 0 or more
0 or more 3900 0 or more 12000/24000
2000 series 38x0/32x0
2000 Series y Up to 1152-4352 total ports*

38x0 or 32x0 1 ISL y Up to 896-3840 user ports*
3900 2 ISLs y Up to 7:1-15:1 ISL oversubscription ratio*
12000/24000
* depends on type of switch used
Figure 3-26 Low Cost Per Port Fabric Topology

3
0 or more 0 or more 0 or more 12000/24000

0 or more 3900
2000 series 32x0/38x0
2000 Series
38x0 or 32x0
3900 2 ISLs y Up to 1024 total ports
12000/24000 4 ISLs y Up to 768 user ports
8 ISLs y 3:1 ISL oversubscription ratio
Figure 3-27 High Performance Fabric Topology
0 or more 12000/24000
0 or more 3900
3900 y 2304-8704 total ports

2-4 ISL
12000 y 1792-7680 user ports
4-8 ISLs
24000 y 7:1 - 15:1 ISL oversubscription ratio
Figure 3-28 Very Large Fabric Topology

3
0 or more 0 or more
0 or more 3900
2000 series 32x0/38x0
2000 Series
y Up to 288 total ports
38x0 or 32x0
1 ISL y Up to 224 user ports
3900
2 ISLs y 7:1 ISL oversubscription ratio
Figure 3-29 Small Fabric Topology
Site A
0-100 KM
Site B
Figure 3-30 Extended Distance Fabric Topology

Section
SAN Deployment II

Chapter
SAN Deployment Overview
4
Once the SAN is designed using sound principles as detailed in Section I SAN Design, it needs to be deployed. The process of
SAN deployment is more than plugging in cables, turning on the power and setting IP addresses. In fact, there are four distinct
phases all SAN deployments go through. Briefly, these four phases are planning, staging, validation and maintenance. The
definition and benefits of doing each of them properly are summarized below:
1. Planning - Preparing for the staging of the SAN and related equipment is important; therefore, a chapter is dedicated
to this subject. Proper planning allows for estimation of time and effort and provides justification for resources. A
good plan provides a means of measuring progress and greatly assists in avoiding potential pitfalls that prevent timely
project completion. Refer to Chapter 5, Deployment - Planning.
2. Staging - After the planning phase the SAN needs to be put together. Staging covers everything from uncrating and
racking the switches to configuring Brocade Fabric OS and the applications that will run on the hosts, storage and
other devices that are attached to the SAN. Refer to Chapter 6, Deployment - Staging and Validation.
3. SAN Validation - Once staged, the entire SAN configuration needs to be tested and validated to confirm it is ready
for production. The tests should verify device connections, check for the SAN robustness, and most importantly, test
the application availability under varying failure sceneries. Refer to section 6.6. Validating the SAN on page 6-58.
4. Maintenance and Operations - Once the SAN transitions to an operational state, changes are likely to occur, such as
the addition of hosts or storage. This may require more switches if all user ports are allocated. The Fabric OS or other
firmware and software may need to be upgraded. Maintaining the SAN is all about the day-to-day activities that keep
the SAN running smoothly and efficiently. Refer to Chapter 7, Maintenance and Operations.
Section II SAN Deployment takes a hands on approach to illustrate the four deployment phases, using a case study. The case
study is an actual SAN designed, deployed and managed at the Brocade Technical Solutions Lab. In addition to being used for
illustration purposes, the case study SAN provides a validation point for the concepts and procedures discussed in this Section.
The case study SAN will also be used to demonstrate key operational tips of Fabric OS and used to point out important
subtleties and guidelines.
Like any complex project, there are many different ways of deploying a SAN. Even though there is a level of complexity, there
are still general guidelines and practices that should be followed. Many guidelines in this Section are in the form of checklists.
Checklists provide the essential SAN deployment information to ease the transition from the planning stage to production. All
checklists can be used as is, or modified to fit specific SAN environments. Taken as a whole, the guidelines and practices
discussed within each checklist will allow for sound decisions; optimizing the IT investment once in production. The
checklists are not meant to be all inclusive. Other methods of performing the activities discussed are possible and encouraged.
It is crucial that the production SAN be supportable, manageable, simple to maintain, and easily scaled. Proper planning and
documentation is critical to make these goals attainable. Effective up front planning simplifies the actual staging phase. Node
location will be known, storage requirements met and cabling simplified. Proper documentation functions much like a map,
allowing the quick identification of devices and configuration settings. This reduces the potential downtime, whether it be
scheduled or not. Since these benefits, and others, are so important, there is emphasis on putting together an effective planning
strategy.
Besides planning and documentation, this section focuses on features and functions of new releases of the Brocade products.
This is to assist the user who is familiar with Brocade technology to quickly make use of them. New features and
enhancements covered in Fabric OS 2.6.x, 3.1.x and 4.2 include Advanced Zoning, Trunking, Secure Fabric OS, and Hot Code
Activation (HCA). Other functions, such as firmware updates, are more user friendly. There are also new commands that allow
for better online switch documentation and for gathering diagnostic information. Fabric Manager is greatly improved and is
used to illustrate how to simplify many tasks, especially Secure Fabric OS policy management activities.

4 SAN Deployment Overview
4.1. Introduction to Fabric OS 2.6.2, 3.1.2, 4.2

This section introduces new features and functions introduced in Fabric OS 4.2, 3.1.2 and 2.6.2. All versions of Fabric OS
(FOS) are being updated to allow for overall greater consistency. Some new functions are added as needed, for example so that
Web Tools can discover new platforms.
The most recent Fabric OS release versions are OS 2.6.2 (SilkWorm 2000), 3.1.2 (SilkWorm 3800, 3200), 4.2 (SilkWorm
3250, 3850, 3900, 12000 and 24000).
With the introduction of Fabric OS 4.2, Fabric OS 4.x firmware now runs on five switch platforms. Non-disruptive code load,
Secure Fabric OS (SFOS) and other new features in Fabric OS 4.2 can now be utilized on Brocade switches ranging from 8 to
128-ports. Fabric OS 3.1.2 and 2.6.2 were enhanced to provide full GUI discovery and support of the SilkWorm 24000
Director as well as the 16-port SilkWorm 3850 and 8-port SilkWorm 3250.
Note: Fabric Manager 4.1.1 is required for management of the SilkWorm 24000, 3850 and 3250 (switches running Fabric
OS 4.2). Older versions of Fabric Manager will not support these new platforms.
Guideline: For full management support of the SilkWorm 24000, 3850 and 3250 from switches that support Fabric
OS 2.x or 3.x only, use Fabric OS versions 2.6.2 and 3.1.2 or greater.
Older versions of firmware have been tested in various SAN fabric configurations. For supported backwards compatibility
information, please refer to the switch provider documentation and official support statements.
4.1.1. Fabric OS 2.6.2/3.2/4.2 Overview

Each release of Brocade Fabric OS runs on a specific set of switches. Table 4-1 explains which release of Fabric OS is required
for each model of SilkWorm switch.
Table 4-1 Fabric OS Requirements
Switch Model Minimum Fabric OS Recommended Kernel

Fabric OS
SilkWorm 24000 Fabric OS 4.2 or later Fabric OS 4.2 or later Linux kernel
SilkWorm 3850 Fabric OS 4.2 or later Fabric OS 4.2 or later
SilkWorm 12000 Fabric OS 4.1.x or later Fabric OS 4.2 or later
SilkWorm 3900 Fabric OS 4.1.x or later Fabric OS 4.2 or later
SilkWorm 3800 Fabric OS 3.1 or later Fabric OS 3.2 or later VXWORKS
UNIX
SilkWorm 2000 Family Fabric OS 2.6.1 or later Fabric OS 2.6.2 or later
For complete list of all commands, refer to the appropriate Brocade Fabric OS Reference Guide for each version of Fabric OS.
For typical Fabric OS configuration tasks, refer to the appropriate Brocade Fabric OS Procedures Guide for each version of
Fabric OS. The Brocade Fabric OS Procedures Guide provides instruction on how to do most of the configuration activities
discussed in this section. These manuals are available on the Documentation CD shipped with Brocade SilkWorm switches.

SAN Deployment Overview 4
4.2. Fabric OS 4.2 Platform Support

The following provides highlights of the new platform characteristics and an overview of the platform support provided by
Fabric OS 4.2.
4.2.1. SilkWorm 24000

The SilkWorm 24000 is a director class switch that provides up to 128 ports with five nines of availability (99.999%) in a
single chassis. Substantial updates have been made to the processing power. With greater memory and a faster processor, the
SilkWorm 24000 CP is at least 3-4 times faster than its predecessor. A true Core/Edge design now exists within the chassis,
which provides improved deterministic I/O patterns.
The SilkWorm 24000 chassis is identical to the one used by the SilkWorm 12000, however the CP and Port blades are
different. As in the SilkWorm 12000, there are two Control Processors (CP) but now only a single domain. Maximum
supported distances have been increased to 100 Km. The ports are physically grouped as quads, which provide easier
identification of adjacent ports. This is useful when cabling the switch to utilize features such as Brocade Extended Fabrics
and or Brocade Trunking. Power consumption is much reduced, thus two power supplies are required to run the chassis.
Important Fabric OS 4.2 changes that allow SilkWorm 24000 support are provided below.
• New Active LED (Blue) to indicate the active CP blade
• Only one domain per chassis is supported and all slots are part of that single switch
• Web Tools, SNMP and API changes to a support 128-port domain
• Commands, such as switchshow adjusted to handle port count greater than 64-ports
• Commands such as bladeDisable adjusted to accept physical slots 7-10 as part of a single domain
• Commands such as chassisshow and slotshow can now display different blade ID
Note: Fabric OS 4.2 only supports a single domain within a SilkWorm 24000 chassis.
When the SilkWorm 24000 is fully populated, POST takes up to 15 minutes to complete from a cold start. POST occurs in two
phases. Each phase is a series of pre-defined diagnostic tests that checks everything from flash memory to internal link
integrity. Once the SilkWorm 24000 is installed, configured, and the diagnostic tests have completed successfully, there is no
requirement to have them turned on. Use diagdisablepost to prevent the switch diagnostic tests from running after a reboot or
cold start. The default setting has diagnostics turned on.
4.2.2. SilkWorm 3850 and SilkWorm 3250

The SilkWorm 3850 and 3250 run Fabric OS 4.2 or later. The SilkWorm 3850 is 16-ports and is configured with two fixed
power supplies. The SilkWorm 3250 has 8-ports and is configured with one fixed power supply. Like the SilkWorm 24000,
maximum supported distances have been increased to 100 Km.
A number of Brocade switch vendors provide a licensed option that allow for a maximum of two or four domain fabrics. For
fabrics with maximum port counts of 64 or less, this provides an excellent cost effective choice while still deriving benefit
from non-disruptive code load and other new Fabric OS 4.x features.
Guideline: Upgrade to the Full Fabric licensed option if using the SilkWorm 3850 or 3250 as part of a fabric of four
or more switches.

4 SAN Deployment Overview
4.3. Fabric OS 2.6.2, 3.1.2, 4.2 Enhancements

The following enhancements were made to improve the usability of Fabric OS. The following sections will provide detailed
examples and further explanations of the commands as appropriate.
Table 4-2 Enhancements to Fabric OS
Fabric OS Version Enhancement

2.6.2 3.1.2 4.2
X X X New Extended Edge PID Format (2).
X Compact Flash Free Space Monitoring and Management
X X X Online help text for zoning and other commands more consistent across FABOS versions.
X X X Boot Over SAN enhancements to allow greater HBA and switch interoperability
X New wwncardshow command and -all option to sfpshow command
X X X New quietmode command to v4.2. Improvements to existing quietmode behavior in

v2.6.2, v3.1.2.
X X X New pathinfo command which better displays FSPF selected routes.
X X New -portcount option to switchshow command. This allows easier viewing of

switchshow output in large port count switches.
X Increased maximum command line length to 256 characters to match 4.x.
X Flash the SilkWorm 12000 and 24000 blower LED when a blower has been disabled.

Chapter
Deployment - Planning
5
Having an effective plan prior to the staging of the equipment is critical for overall SAN deployment success. This success is
measured in many forms. The greatest benefit of a good plan is that it gets the SAN deployed on time and within budget.
Doing this ensures the ROI to be realized in the shortest time possible.
Planning is all about understanding the requirements and allocating necessary resources. Proper planning does take extra
up-front effort and cost. For higher port-count SANs, it is critical that there be at least one person who is accountable for
developing and driving it. With a good plan, progress towards completion can be measured, the right persons identified, the
roles and responsibilities defined, the site and SAN documented, and SAN resources can be efficiently utilized. With proper
planning, less effort is needed to maintain the infrastructure. When the staging requirements are known, obstacles that may
impede progress can be avoided.
This chapter provides some guidelines on the what essential information is needed for putting an effective plan together.
Checklists are used throughout this chapter to provide a framework for gathering requirements to meet specific site needs. To
clarify key points, a case study SAN will be referenced.
5.1. Site Environment Assessment

The first step is to prepare the site for the SAN. One effective way of doing this is to survey the site. The survey information
provides a list of items that require completion before the hardware arrives. As a side benefit, once filled out, the site will then
be documented. This is highly desirable in that if any issues arise, the pertinent information can be located quickly and the
issue can be proactively solved. For smaller port-count fabrics, the effort may be minimal. Even for smaller SANs, adherence
to this process still provides benefits, especially through the creation of the documentation. Site Survey Templates are
available in Appendix D, Site Survey Templates.
Here are some questions to consider when defining a survey:
• Is sufficient power available?
• Are the appropriate cables and receptacles in place?
• Are patch panels required?
• Effective and standardized cable labeling methodology in place? (Refer to Appendix A, Cabling Design and
Management for more information.)
• Is there sufficient cooling?
• Is there sufficient rack space and footprint available?
• How should the SAN be attached to the LAN infrastructure?
• What management station will be used to administer the fabric? Is there a serial port available?
• What are the special site-specific policies?
It is critical to weigh the constraints the to answer these and other questions, so that when the Brocade fabric switches arrive,
everything that is needed will be in place and the installation process can begin immediately and proceed smoothly. Knowing
the requirements ahead of time really helps in planning the activities. As an example, storage switches may require extra
security. This will prevent the highly sensitive data from being compromised. The switches, hosts and storage need to be
locked down, to prevent unauthorized physical access. Now that this is known, the SAN Project Manager can plan for the
space and budget.

5 Deployment - Planning
5.2. SAN Project Checklist

Table 5-1 is an itemized SAN project checklist that provides guidelines on gathering the essential information for driving the
project to completion. Following the checklist, the items that require further explanation are expanded upon. For an example
of a complete survey form, refer to Appendix D, Site Survey Templates.
Table 5-1 SAN Project Checklist
SAN Project Checklist

1. Identify SAN Project Manager
2. Site Contacts, Name, Location
3. Site Preparation/Space Planning
4. Project Plan
• Site Specific Policies
• Required Product Training
5. SAN Equipment Assessment. Consider choosing a
management switch for the SAN fabric.
6. Appropriate licenses ordered
7. Application Software Considerations
8. Approval Sheet
1. Identify the SAN Project Manager

This is the most important item on the checklist. The SAN Project Manager co-ordinates the entire effort. The paper
deliverable is a project plan. The co-ordination effort includes holding regular meetings, creating action items and driving
the decision making process to resolve them. Many times, it also involves getting the right folks to talk with each other,
and informing everyone of the plan and progress. Without this role clearly defined, the SAN project will quickly turn into
chaos.
2. Site Contacts, Name, Location
Identify each person on the deployment team, their roles and responsibilities, and the site where the equipment will be
staged. The SAN Project Manager should put together this list.
3. Site Preparation / Space Planning
The list of items for site preparation should include (but is not limited to): air-conditioning, power, fiber optic cable
requirements (including location, lengths, labels, etc.), any fiber patch panels, proposed rack layout, etc. As an example,
the project manager may have to work with the site facility staff to make sure that the proper cooling vents and cut tiles
are available for those locations that have cables run under raised floor tiles.
Once the equipment list is captured, having adequate space planned out is extremely helpful. By knowing where things go
ahead of time, it becomes easier to discuss cabling, patch panels, DWDM attachments, etc. At a minimum, a high-level
site schematic should illustrate the location of the Brocade SilkWorm fabric switches and other SAN-related equipment.
Give consideration to including host, storage devices and other auxiliary equipment such as the power infrastructure and
air conditioning as well.
Label each piece of equipment for quick and easy identification. A scale drawing is really helpful if a large number of
racks and other equipment are required. Another guideline is to create a rack layout diagram. This will illustrate all the
equipment locations within the 42 U of space. For large deployments where the same equipment will be replicated at
multiple locations, this becomes invaluable. The detailed information, like the port to which each device is attached, is not
important at this time. This will be covered in a later section. It is important that all of the equipment is accounted for so
that the other preparation steps can be completed.

Deployment - Planning 5
4. Project Plan
A project plan is the tool used by the SAN project manager to get the SAN in place on time and within budget. The survey
is used as a starting point to determine what is needed and when. Keep in mind any site-specific policies that may affect
the installation. Include diagrams of proposed rack layouts and an installation schedule, which shows all of the
dependencies. Training on the Brocade product family may be required for the staff who will own and operate the SAN.
Put this in the plan as well. For information regarding training on Brocade products please visit:
http://www.brocade.com/education_services/index.jsp
When the project plan is complete, the SAN project manager will understand the staging and other requirements. Armed
with this information, assigning and scheduling resources to meet the project milestones should be easily justified.
5. SAN Equipment Assessment
Find out whether there is an existing SAN in place or if this is a new installation. In the case of an existing SAN, a
migration plan may be required, especially if hosts are using the 24-bit PID address for persistent binding. For details
about planning and implementing a migration, reference the SAN Migration Guide (publication number: 53-0000360-02).
As part of the assessment, generate a high level list of all fabric switches, hubs, hosts and storage. For the Brocade SAN
Fabric switches, define the roles. As an example, consider choosing a management switch. This switch will be the
management point into the fabric. The ideal choice is a switch that is robust and central to the fabric, such as a core
switch. As for the other equipment, make sure the SFPs, GBICs, fiber optic cables, and any media converters are
available. Include any DWDMs, Gateways, and other devices for connecting SANs over distance. Include any LAN
Ethernet hubs/switches that require Brocade switch attachment. Refer to LAN Guidelines For Brocade SilkWorm Switches
(publication number: 53-0000350-01) for effective LAN Integration guidance.
6. Appropriate licenses ordered
Make sure that the appropriate licenses are ordered given the customer requirements.
7. Application Software
Put together a high level list of the application software to be used such as: Database, CRM, E-mail, Web Services, and
the associated host. This may drive switch and device placement within a data center. The objective is to make sure that
the application storage requirements are taken into consideration, so that the storage placement is optimal. For more
information about switch placement refer to section 3.5. Switch Location In The Fabric on page 3-24.
8. Approval sheet
Once the site survey and SAN Project plan are complete, a substantial amount of the pre-work will be complete. For larger
port-count SANs used in the enterprise, a survey and project plan becomes critical for successful deployment. SANs with
smaller port-counts may not require an extensive survey, or even a robust project plan. Perhaps all that is required is to
plan power and rack space. In all cases, it is important to focus on getting everything documented so that information is
readily available.
Table 5-2 Site Prep Tips
Site Prep Tips
• Some equipment may have long order lead times, so it is important to identify
what is needed as soon as possible.
• For cables that will be run underneath a raised floor, not having enough cut tiles
is a potential stumbling block.
• Heavy equipment, such as the SilkWorm 12000/24000, should be near the
bottom of the rack for maximum stability.

5.3. Planning for Power

This section provides some power guidelines to assist with the deployment planning of Brocade switch platforms. Please refer
to the appropriate Brocade SilkWorm hardware reference manual for all the geo-regional power requirements. A list of all
manuals is provided in the Preface. Cable types also differ by region and are required to be a separate line item in the Brocade
product order.
5.3.1. SilkWorm 2000 and 3000 Power Requirements

The hardware reference manuals provide information about amperage level of each switch. Use information when selecting
the appropriate cord for your switch. When ordering Brocade products, work with the switch vendor to get the appropriate
power cord set.
The SilkWorm 3800, 3850 and 3900 each have two power supplies. For maximum availability be sure both supplies are used.
Each power supply is a Field Replaceable Unit (FRU) and can be hot swapped. The SilkWorm 3250 and 3200, being low cost
products, each have a single non-replaceable power supply.
Note: The power supplies are not interchangeable between the SilkWorm 2000 family of switches and the SilkWorm
3200/3800. Power supplies cannot be interchanged between the SilkWorm 3200/3800 and SilkWorm 3900.
Guideline: If only one power supply is used, the default settings of switch environmental policies will cause the
switch to be in a marginal status. Use switchstatuspolicyshow at the command line to see the
current environmental settings and switchstatuspolicyset to change them. Web Tools will show
the switch image highlighted in an amber color as a warning. This setting can be configured
administratively as discussed in Section III SAN Management.
5.3.2. SilkWorm 12000 and 24000 Director Power

Requirements
SilkWorm Directors have voltage requirements of 200 to 240 VAC, 50 Hz or 60 Hz. Two dedicated branch circuits are
required for redundancy. Confirm the power cords ordered with the system to ensure the correct power receptacles are
installed.
• In the US and most of North America, the available voltage is usually 208 or 240 VAC, the receptacles are NEMA
L6-20R, on individual branch circuits, (each rated 20 amps).
• In UK, Ireland, Hong Kong, two UK-standard 13 amp receptacles, on individual branch circuits, are required.
• In most of continental Europe, two CEE7/7 “Schuko”-compatible receptacles are required, each rated 16 amps, OR
two IEC60309, 230V~16A-6h receptacles. Verify with the system order.
• In Australia and New Zealand, 2 Australian-standard, 15 amp receptacles, on individual branch circuits rated 15 amps
each, are required.
• For any location in which the US, UK, “Schuko” or AUS/NZ receptacles types are not accepted, the International
standard, IEC-60309, 230V~, 16A-6h, receptacles should be planned, and the system order confirmed.
For proper power on the SilkWorm 12000 and 24000, check for a steady green LED on all power supplies on the cable side of
the unit. For all products other than the SilkWorm 12000 and 24000, the single power LED should be a steady green. If it
blinks or is a different color, this indicates there is problem. The WWN card on the non-cable side of the chassis also displays
power supply status as “good” with four steady green LEDs.

5.4. Cable Planning

Cabling is one of the most important, yet somewhat overlooked, aspects of deployment. Poor cable management commonly
causes issues within IT departments. While there is no fail-safe way to prevent problems, this section contains some tips and
guidelines to help reduce the risk. For additional cable management guidelines please refer to Appendix A, Cabling Design
and Management.
All Brocade products use LC connectors for 2 Gbit/sec and SC connectors for 1 Gbit/sec type connections. For older hosts and
storage that support only 1 Gbit/sec, an SC to LC cable is required when mixing speeds. The switch will automatically detect
the speed and provide the appropriate connection speed during link initialization.
5.5. Platform Specific Cable Management Guidelines

Proper cable management reduces the cost of SAN maintenance and helps connectors last longer, as there is weight relief at
the point where the cable sheath joins at the connector (see Figure 5-1). For fiber optic cables, which are made of high quality
glass, proper management reduces the risk of link failure by preventing the cable from exceeding the maximum bend radius.
Figure 5-1 Cable management provides weight relief for the cable sheath at the connector.
Proper cable management facilitates easy cable identification, which is important for accommodating the growth of the SAN.
Good cable management is an art. Everyone has a different style that can be applied to the unique structural characteristics of
the site. Thus, there is no single solution for proper cable management. There are endless variations on doing it right. The next
few sections will highlight some general guidelines on managing cable connections that can be applied to almost every
situation. For additional cable management guidelines please refer to Appendix A, Cabling Design and Management.
5.5.1. Cable Management for the SilkWorm 2000 and 3000

When racking SilkWorm 2000 and 3000 series switches, it is highly recommended to have space to alleviate the bending of the
fiber optic cable connector sheaths while they are plugged into the switch ports. Effective cable management prevents this
problem, which occurs over time, as the weight of the cable not only bends the connector but can loosen the connector-cable
attachment. If this is not managed, eventually there will be signal loss that could cause loss of frames and, potentially, data
corruption. Cables that obstruct the face of the switch make it more difficult and time consuming to replace them. Good
management prevents this. As a side benefit, cable management also makes it easier to identify the cables and to read each
label. Figure 5-2 illustrates poor cable management and shows the difficulty of replacing the SilkWorm 3900.

Figure 5-2 Improper Cable Management
Use cable guides to solve these problems. There are horizontal and vertical cable guides for standard EIA racks. Horizontal
cable guides come in a variety of shapes and generally take up at least one rack unit. Vertical cable guides are generally much
wider and up to 42 U in height, which is the standard size of an EIA rack. These generally are mounted along the side, so extra
floor space is required for them. Telco racks have their own methods of management. Be sure to ask the Telco rack vendor for
cable management options. Both types have “fingers” that allow the cables to be held while being run across the rack or
plugged into a switch port. This may be expensive when considering the value of rack space so the customer may choose not
to do it. However, the organizational benefits are enough to justify the expense alone. Figure 5-3 shows how effective cable
guides are at achieving proper management.
Guideline: Use horizontal and vertical cable guides for cable management. Include them in your rack planning.
Guideline: Plan out the floor space and install the guides before staging the racks with the equipment. Often times,
the cable guides are secured to the racks themselves. Once the equipment is plugged in and the cables
have been run, installing cable guides in the racks become much more costly as power cables and racks
must be moved or removed.
Guideline: If using seismic platforms, allow enough cable slack for effective movement.

Note that the cables in Figure 5-3 are lined up nicely and the strain on the connectors is minimized. The cables are not
blocking the switch face and ports. If a switch needs to be replaced, there is no need to unplug unnecessary cables from
surrounding switches or the switches themselves. Downtime is reduced, and money is saved.
Figure 5-3 Proper Cable Management
Guideline: For Trunk groups, use Velcro tape and spacers (sometimes called pillars) to bundle the cables into
groups. Not only does this alleviate the weight, it also allows for effective cable organization. As an
added plus, trunk group identification is easily attained when this is accomplished.
Guideline: Rack the cable side of the switches on the same side where the HBAs and storage ports are located.
Typically these ports are located on the “back” of those devices. This will prevent cables having to be
run along the inside and side of an EIA rack. Be aware of potential cooling issues when doing this.
Guideline: Label the fiber optic cables or use fiber optic cables that are already numbered, such as with serial
numbers. This information can be used to build a spreadsheet of devices, switch ports and the cables
that connect them together.
Guideline: Allow manageable cable slack for sliders on rack mount kits.
Guideline: Cable guides should always be used in patch panels. This is because patch panels are a natural cable
collection point and the scene of many potential problems.
Figure 5-4 illustrates an effective use of cable trays.

Figure 5-4 Patch Panel With Cable Guides
Note: Some of the cable trays are open to show how the cables are run.
5.5.2. SilkWorm 12000/24000 Cable Management Planning

A prime consideration of cable management for the SilkWorm 12000/24000 is the ability to remove and replace the Field
Replaceable Unit (FRU) components without having to unplug cables. Since the port cards, CP cards, power supplies, and
blower assemblies are all hot-swappable, ensuring easy access to each component is vital toward contributing to the uptime of
the SAN. In addition, it is important that the LEDs on the individual components and the WWN card remain visible.
If Trunking is being used, the ports and cables used in trunking groups must meet specific requirements. For a list of these
requirements, refer to the Brocade Fabric OS Features Guide (part number: 53-0000395-01). Figure 5-5 shows an example of
effective cable management, with no cables crossing in front of the FRU and an example of how not to cable a SilkWorm
12000/24000. Note that any single FRU is impossible to replace without having to unplug devices and/or other switches.

Proper Cable Management Improper Cable Management
Figure 5-5 Proper and Improper SilkWorm 12000 Cable Management
5.5.3. SilkWorm 12000/24000 Cabling Guidelines

The following list provides cabling guidelines for the SilkWorm 12000 and 24000 directors. Use this information as part of
your cabling plan.
• Leave at least one meter of slack for each fiber optic cable. This provides room to remove and replace the port card,
allows for inadvertent movement of the rack, and helps prevent the cables from being bent to less than the minimum
recommended bend radius.
• Route fiber optic and other cables down along the front of the card to which they are connected, to prevent having to
disconnect them when neighboring cards are replaced. Do not route across adjacent cards or in front of the power
supplies.
• Use the Cable Pillars provided with the rack kits to bundle the fiber optic cables from each quad of ports. These
guides help to keep individual ports accessible by keeping the cables evenly spaced, and also help to provide
clearance for the replacement of a port card or CP card.
• Use Velcro wraps (not included with the switch) to further bundle the cables on a per-card basis. Tie wraps are not
recommended for optical cables because they can easily be overtightened, damaging the optical fibers.
• Use the cable management tray to route the bundled fiber optic cables, Ethernet cables, and any serial cables down
the front of the chassis.
• The power cord requires a minimum service loop of six inches at the switch to ensure enough freedom of movement
to plug and unplug it.
• Route the power cables to each side of the switch instead of through the cable management tray. The power cable
connectors are designed with right and left bends to facilitate cable management.
• Use a spreadsheet to track which devices are connected to which switch, port card, and port in the SilkWorm
12000/24000. The serial numbers or other identifiers of the fiber optic cables can also be tracked.
• Keep the chassis door on the SilkWorm 12000/24000 closed to protect cables from inadvertent movement.

5.6. The Rack Layout Plan

When it comes to racking, usually some creativity is required. There is no single correct technique. The following section
presents some guidelines to keep in mind when planning and racking the switches and other equipment.
From the site survey, get the equipment list and put together a spreadsheet showing the number of required rack units (U) that
are required for each device. Include switches as well as any hosts and storage. Once again, there is no one right way of doing
this. Here is one template that has proven effective with the case study equipment shown (for more information refer to section
6.1. Case Study on page 6-2). Table 5-3 shows the essential information for a standard 42 U rack. If desired, a scale diagram
can be drawn to show the actual rack space constraints. This template is also available in Appendix D, Site Survey Templates.
Table 5-3 Sample Rack Layout Spread Sheet Template.
D e v ice N a m e N u m b e r o f D e v ice s R a ck U n it p e r D e v ice T o ta l U se d R a c k U n its
F ib er P atc h P an el 1 3 3
K V M U nit 1 3 3
Hos t 1 1 1 1
Hos t 2 1 3 3
C a ble G u id e 1 1 1
S ilk W o rm 3 25 0 1 1 1
S ilk W o rm 3 85 0 1 1 1
S ilk W o rm 2 80 0 2 2 4
S ilk W o rm 3 80 0 2 1 2
S ilk W o rm 3 90 0 1 1.5 1 .5
S ilk W o rm 1 20 00 1 14 14
S W 12 00 0 C ab le G uide 1 2 2
T o ta l U n its A va ila b le 42
T o ta l U n its U se d 3 9.5
T o ta l U n its R e m a in in g 2 .5
Note: In this example the extra space for airflow is made available through the use of cable guides.

5.6.1. Racking Guidelines

There are two styles of racks, EIA and Telco. EIA racks typically have four posts, while Telco racks have only two. All
Brocade switches are available with EIA rack kits as a separate option. Telco racks typically provide better cooling potential as
there is typically more open space. Notify the switch vendor if a Telco rack is to be used. Assuming sufficient cooling and
airflow exists, which should have been determined by the site survey, then there is no physical need for having extra space
between each rack mounted switch.
Brocade SilkWorm switch airflow is from the non-cable side to cable side. When racking switches, totaling ten or more rack
units, make sure sufficient air flow is available to cool the switches. The inlet ambient air temperature must be below 40
degrees Celsius, refer to the appropriate Hardware Reference Guide for your switch model. This can be done by spacing the
switches appropriately or using a rack fan. Extra rack space is always desirable from a cooling standpoint, as more surface area
will be exposed. Another reason for having extra space between the switches is to allow for flexible cable management.
Guideline: Avoid crossing over rack unit boundaries. Count them before installing the equipment. Some EIA racks
have rack units pre-numbered.
Guideline: For larger chassis, such as the SilkWorm 12000/24000, create a hole schedule template to assist with
planning the rail locations. This is useful when a variety of different sized equipment is installed.
Guideline: When racking, use cage nuts with built in threads for the screws. To facilitate installation, mark the unit
boundaries on the rack before installing the nuts.
Guideline: To simplify cable management, rack the cable side of the switches on the same side where the HBAs
and storage ports are located.
Guideline: Consider ordering pre-numbered fiber optic cables.
Guideline: Cable guides should always be used in patch panels.
Guideline: When racking switches, totaling ten or more rack units, make sure sufficient air flow is available to cool
the switches. This can be done by spacing the switches appropriately or using a rack fan.
Guideline: Forced airflow is required when using solid doors in the rack, such as those made of Plexiglas.

5.7. Documentation Guidelines

This section will provide some guidelines regarding documenting the SAN. Knowing what documentation is needed allows
for planning IP addresses, switch domain numbers, what ports should be used for ISLs, etc. Once created, having the
documentation readily available allows change management of all components. In addition, being organized with updated
documentation saves time and effort when referencing equipment for service calls. Even while not being serviced, when
information is needed, it can be referenced quickly and easily. Here is a checklist that provides a recommendation as to what
documentation should be created before and during the staging of the equipment. Refer to Appendix D, Site Survey Templates
for templates that can be used to assist in documentation.
Table 5-4 Documentation Checklist
Documentation Checklist
1. Get an Equipment Binder for each rack
2. Logical Design Diagram
3. Switch Spreadsheet
4. ISL Port Map
5. Device Spreadsheet
6. Label All Cables
7. SAN Verification Test Plan Requirements
8. SAN Verification Test Plan
1. Equipment Binder
It is good practice to a put together a binder for each rack. Keep all of the configuration information in the binder,
including the rack layout, make and model of each piece of racked equipment, serial numbers, physical cable connections,
the names on the labels, etc. As part of the binder, keep all of the service logs to track that history for each piece of
equipment. If at all possible, keep this attached to the rack physically.
2. Switch Spreadsheet
Keep track of all Brocade switches. Include switch name, IP address and domain name as well as the role. An example of
a spreadsheet is shown in Table 5-5. This may be reflected in the switch name, for example A-C1, designates this switch
as the first core switch in Fabric A.
Guideline: Set unique domain numbers for each switch in the SAN. This allows for simpler merging of fabrics.
Guideline: As a convention, consider setting the domain ID of each switch to the last octet of its IP address. Be
aware that the highest allowed domain number is 239.

Table 5-5 Switch Device Spreadsheet
Fabric A Fabric B
Switch Name Domain IP Address Switch Name Domain IP Address
A-C1 100 192.168.122.100 B-C1 110 192.168.122.110
A-C2 101 192.168.122.101 B-C2 111 192.168.122.111
A-E1 102 192.168.122.102 B-E1 112 192.168.122.112
A-E2 103 192.168.122.103 B-E2 113 192.168.122.113
A-E3 104 192.168.122.104 B-E3 114 192.168.122.114
A-E4 105 192.168.122.105 B-E4 115 192.168.122.115
A-E5 106 192.168.122.106 B-E5 116 192.168.122.116
A-E6 107 192.168.122.107 B-E6 117 192.168.122.117
3. ISL Port Map

Create a port map that shows how each switch is interconnected. This is handy for tracing cables and can be used to
reproduce the configuration easily. Table 5-6 provides an example.
Table 5-6 ISL Port Map
Fabric A Fabric B
From Cable To From Cable To
FabA-E1-15 I1 FabA-C1-15 FabB-E1-15 I25 FabB-C1-15
4. Device Spreadsheet
The other piece of information that should be included is a device spreadsheet from the host perspective. Include
hostname, IP address, HBA make model and WWN, host and storage physical connections, storage LUN assignment map
etc. For an example of a device spreadsheet, refer to Appendix D, Site Survey Templates.
5. Logical Design Diagram
Create a picture of the logical design. Keep it simple and be careful of adding too much information. The idea is to
represent the SAN topology as well as host and storage connections. As an example, see Figure 6-1 on page 6-2, which
depicts the case study.

6. Label All Cables

Label all of the switches as well with IP addresses and/or switch host names. The SilkWorm 12000 has four IP addresses
and requires up four hostnames, one per logical switch and one per CP. Place a label on the side of the chassis for quick
and easy access. Choose a naming convention, keep it simple and be consistent. As for information, ISLs should have the
destination switch name and port number, for example for a SilkWorm 3800 connected to a SilkWorm 12000 on sw1
physical slot 7 and port 15, use “sw12000 7/15”. Devices should have its name (IP hostname works well), the fabric
plugged into and port number, for example “int124 A14”. For the SilkWorm 12000/24000 use the slot/port designation.
These can be handwritten on stickers or generated by a labeling machine. Use the same techniques for other types of
cables. Once complete, create a spreadsheet with all of this info and put into a binder with all of the other documentation.
7. SAN Verification Test Plan Requirements
At this stage it is important to plan for validating the SAN. The requirements for this depend upon the specific SAN
application and its purpose.
8. SAN Verification Test Plan
Once the requirements are understood, put together a verification test plan. If possible, (especially for enterprise class
SANs) it is highly recommended to do this with a proof of concept test SAN that is not in production.

5.8. Zoning Plan

Zoning allows the hosts to access specific storage devices on the SAN. For those SANs with multiple OS platforms, zoning
allows for OS separation and co-existence. With no zoning defined on the SAN, any device can see any other device. This is
the default setting. Once zoning is in place, all devices must be members of a defined zone. Those devices that are not will be
blind to all others. This section will provide some guidelines as to zoning plan definition. For additional information reference
the Brocade Zoning User’s Guide (part number: 53-0000523-02) and the whitepaper titled Brocade Guide to Understanding
Zoning (part number: 53-0000213-01).
Zoning requires careful thought and planning. Armed with the documentation created in the previous section, and
understanding the requirements, allows the creation of a zoning plan. Creativity is important here as there is no one “correct”
zoning configuration for a given SAN fabric configuration. In general, follow any specific zoning recommendations provided
by the switch vendor. Refer to section 3.3.1. Zoning and Scalability on page 3-20 for more information about implementing
Zoning into your SAN architecture.
Table 5-7 Zoning Plan Checklist
Zoning Plan Checklist

1. Gather the list of host and storage devices to be zoned from the device
spreadsheet.
2. Define the storage requirements for each host based upon software
application requirements.
3. Adhere to recommended storage device configurations such as LUN
masking, LUN security, and other specific features supported by the vendor.
4. Consider specific host requirements for storage value-added feature sets such
as Server Free backup, LUN snapshots, or LUN mirroring over distance.
1. List host and storage devices to be zoned

Gather the list of host and storage devices to be zoned from the device spreadsheet.
2. Consider specific host requirements
Consider specific host requirements for storage value-added feature sets such as Server Free backup, LUN snapshots, or
LUN mirroring over distance.
3. List the storage requirements for each host
Define the storage requirements for each host based upon software application requirements.
4. Define the recommended storage device configurations
Adhere to recommended storage device configurations such as LUN masking, LUN security, and other specific features
supported by the vendor.

Some key points when planning for zoning:

• Clearly understand the storage requirements for each host. This means understanding specifically what storage is
needed for the software application that will be in production. To understand this, the number and size of LUN
presentations on each storage array Fibre Channel port must be clearly defined.
• Be sure to adhere to recommended storage configurations by the switch vendor for LUN masking and other storage
specific features. Some vendors may recommend using a separate HBA for tape devices. For this case, define the
zoning configuration to fence off tape devices from any other HBAs, which see the disk storage, in the same host.
• Keep in mind the different OS platforms, backup application requirements and the number of paths to each LUN
which may drive the zoning plan. There may be specific host requirements for storage value-added feature sets such
as Server Free backup, LUN snapshots, or LUN mirroring over distance.
• As a general rule, have overlapping zones in all cases. An overlapping zone has the HBAs share one or more storage
ports, but with the HBAs separate from each other. This is sometimes referred to as a single initiator zone. For
specific guidelines please see Table 5-8 on page 5-17. Also, be aware that device placement is critical for WWN
based hard zoning. Refer to section 5.8.2. Zoning Enforcement Notes on page 5-17 for details.
5.8.1. Zoning Guidelines
Guideline: If possible, use persistent binding on the host. This will provide consistent controller, target and LUN
numbers for each storage LUN. Backup applications are especially sensitive, as these numbers map
directly to the backup application device identities.
Guideline: If using WWNs, zone by World Wide Port Name (WWPN) rather than World Wide Node Name (WWNN).
This is because a WWPN uniquely identifies a port to which a target is attached. Some Multipathing
software may get confused and not be able to discover targets properly. This is especially true when
using multi-port HBAs.
Guideline: Be aware of mixing different HBA vendors in a single zone. Each vendor HBA responds differently to
RSCNs, a method to notify an HBA for device discovery, and may cause one of the HBAs to lose the
zoned device.
Guideline: In addition it is recommended to have single initiator zones, that is one HBA per zone.
Guideline: Separate HBAs from each other for clustered hosts. Allow each HBA to see the same storage but not
each other. Once again, RSCNs, may cause the clustered host HBA to lose the storage array.

5.8.2. Zoning Enforcement Notes

All zoning employs the Name Server to limit the information returned to an initiator in response to a Name Server query. This
is referred to as soft zoning. When hardware enforced or hard zoning is active, the Brocade switch will monitor Fibre Channel
traffic and block any frames that do not comply with the effective zone configuration on the port to which targets are attached.
Since all hard zoning is enforced at this location, only targets attached to switches running Fabric OS 3.x and 4.x. support
WWN based hard zoning. On switches running Fabric OS 2.x, hard zoning can only be accomplished with Domain/Port
numbers. Thus to do hardware zoning in a mixed fabric, use WWN for switches running Fabric OS 3.x and 4.x and
Domain/Port for switches running Fabric OS 2.x. For more detail, refer to the whitepaper titled: Zoning Implementation
Strategies for Brocade SAN Fabrics.
5.8.3. Zone Port Map

It is good practice to create a zoning port map. This shows which hosts and storage belong in a particular zone. Using color is
really effective in displaying zone membership. For example, Table 5-8 shows the HPUX zone in blue, the Solaris zone in
yellow and un-zoned storage in white. The device aliases are labeled as shown. For larger fabrics this approach has its
limitations due to the color spectrum limits. For this case, build out a table that checks off each device in a zone. This may or
may not be part of the device spreadsheet. Another suggestion is to use a database that houses each device and its zone
membership. With this method, any zone can be selected out of the table space. If possible, include the last 4 or 6 digits of the
WWPN, this will help in identifying unique devices that will require zoning.
Table 5-8 Partial Port Map and Zoning Table for a SilkWorm 12000 Switch

5.9. Planning the Upgrade of Fabric OS 2.x/3.x/4.x

to Fabric OS 2.6.1/3.1/4.1 or Later Versions
This section will provide some high level guidelines when defining a strategy for doing a firmware upgrade of Fabric OS on an
existing Brocade fabric. This activity typically happens in the maintenance phase of a SAN deployment. For the purposes of
this discussion, it is assumed that no new switches will be added or removed from the existing SAN infrastructure. For that
case, and more details, please see the SAN Migration Guide (publication number: 53-0000360-02). For detailed instructions on
all upgrades, refer to the Brocade Fabric OS Procedures Guide for the specific version of Fabric OS used on the switch. Be
sure to upgrade to the firmware qualified or recommended by the switch provider. Also refer to the Brocade SAN Migration
Guide v 1.1 (publication number: 53-0000360-02).
Each Brocade-based SAN is unique. This is due to the wide variety of OS platforms, HBAs and storage arrays that may be
attached. This uniqueness means that each upgrade needs to be carefully planned to minimize the risk of unscheduled
downtime. All updates from a Fabric OS prior to 2.6.1, 3.1 and 4.1 will be disruptive. This includes updates to Fabric OS 2.6.1
and 3.1 and later versions. Scheduled downtime is normally required for single fabric SANs containing switches that run those
firmware versions. For dual fabric SANs, upgrade one fabric at a time. For Fabric OS 2.x or 3.x, either RSH or the FTP
protocol can be used to execute the upgrade. Since Fabric OS 2.x and 3.x allow only one telnet session, the
firmwaredownloadstatus command cannot be used.
Caution: All updates from a Fabric OS prior to 2.6.1, 3.1 and 4.1 will be disruptive.
For Fabric OS 4.x, only the FTP protocol is supported for firmware upgrades. Upgrades from Fabric OS 4.0.x to 4.1 or 4.2 —
depending on SAN architecture — are disruptive, so be sure to schedule downtime. Upgrades from Fabric OS 4.1 are
non-disruptive.
The SilkWorm 3900 and 24000 allow two admin telnet sessions. The SilkWorm 12000 allows two telnet sessions per logical
switch. Use one for firmwaredownload and the other for firmwaredownloadstatus.
Firmwaredownloadstatus is a handy command that shows a log of each upgrade phase. When complete, use
firmwareshow to display the firmware version on each compact flash partition.
Table 5-9 Upgrade Planning Checklist
Upgrade Planning Checklist

1. Analyze the potential risks and impact to each device on the SAN.
2. In order to maximize fall back capability, preserve each fabric switch
configuration with configupload.
3. Use Fabric Manager for larger multi-fabric SANs.
4. Verify the upgrade version is supportable.
5. Gather the documentation and readme notes for the firmware release.
6. Schedule downtime for single fabric updates of 2.x and 3.x and 4.0.x to 4.1.
7. For dual fabrics, update one fabric at a time.

5.9.1. Planning Principal Switch Placement

One of the features introduced in Fabric OS 4.1 is the ability to hard set a preferred principal switch in the fabric. For
Core/Edge topologies, the principal switch should be a core switch for optimal fabric operation. If using a SilkWorm 12000 or
24000 in the core, utilize it as the principal, as it has the greatest availability. Refer to section 3.5. Switch Location In The
Fabric on page 3-24.
Warning: Hard setting a preferred principal switch is not completely deterministic, especially in large fabrics. Secure Fabric
OS also affects its reliability. Use of sequenced reboot feature of Fabric Manager mitigates these two limitations.
After an un-managed reboot (such as a power failure recovery) do a sequenced reboot under Fabric Manager
control to ensure an orderly switch-by-switch reboot.
5.9.2. Planning for Secure Fabric OS Security Measures

There are some SAN Security measures that should be in place before implementing Secure Fabric OS (SFOS). Here are some
guidelines in the form of a checklist to assist with the planning process. These steps can be taken to provide some initial
restrictions on accessing the SAN and to provide some control over change management. For maximum SAN Security, these
measures should be used in conjunction with Secure Fabric OS. Secure Fabric OS provides a single point of management and
policies which allow complete control over what switches, devices and management stations are allowed to access the SAN.
Table 5-10 Planning for Secure Fabric OS Security Measures Checklist
Planning for Secure Fabric OS Security Measures Checklist
1. Prevent Physical Access

2. Prevent Remote Access through IP security measures
3. Hard Zone the devices
4. Lock Down E_port creation with portCfgEport
1. Prevent Physical Access

Use a cage, or some other method, to only allow authorized personnel to access the switches physically. This is important
in that serial port access is a real security risk.
2. Prevent Remote Access through IP security measures
Follow the IP-based security policies for the Brocade Ethernet port attachments and the IP Subnet they are located on. For
example, lock down IP access to the switches by putting them on a separate (Virtual LAN) VLAN segment with separate
IP router ACLs. This prevents illicit access of switch Ethernet connections.
3. Hard Zoning
Hard zoning works in much the same way as a limited Access Control List (ACL). It works by restricting the hosts and
storage access. For a mixed Operating System environment, this is generally required anyway.

4. Lock Down E_ports

This limits the creation of switch-to-switch ports. Use portCfgEport to lock down all ports except those that are to be
ISLs. Note that if Trunking is used, and additional bandwidth is required in the trunk group, the port must be re-enabled to
allow E-Port configuration. An alternate method is to use the command portCfgPersistantDisable to
persistently disable a port. Refer to section 7.6.3. Persistently Disabling a Switch or Port on page 7-23.
Figure 5-6 Using PortCfgEport to disable E_port creation

5.10. Secure Fabric OS Planning

Planning for SAN security and change management is important. Organizations understand that the data managed in their
SAN environment is often highly sensitive and must have controlled access properly to ensure confidentiality, integrity, and
availability. A compromise in any of these areas could have unintended consequences, resulting in the loss of proprietary
information, capital, or other core business resources. Proper SAN Security planning with Secure Fabric OS (SFOS) mitigates
these risks by ensuring proper SAN security access controls are in place and enforced. Because SAN security is, by its own
right a separate subject, a comprehensive treatment will not be discussed in this document. In order to be effective in the
implementation of Secure Fabric OS, there are two assumptions that are made for the duration of the discussion: 1) Significant
non-SFOS security measures are already in place and 2) proper security practices exist within the IT infrastructure.
Rather than talk about broad security objectives, the goal of this section will be to provide an overview of SFOS and provide
essential planning checklists. These planning checklists and other guidelines can be applied to any organization wishing to
implement SFOS and should be used as a foundation for tailored SFOS planning. To clarify important steps, the case study
SAN will be referenced as needed.
For all the details on Brocade SAN Security background information, theory, and SFOS implementation details please refer to
the SAN Security: A Best Practices Guide (publication number GA-RG-250-00). This is an excellent document that covers
current industry Security practices as well as illustrates practical examples on SFOS features usage.
Brocade Secure Fabric OS (SFOS) is a separately licensed Brocade product that provides a comprehensive SAN security
solution for Brocade fabric member switches and the devices that are attached to them. All SilkWorm switches are supported,
except for the SilkWorm 1000 series of products. To use SFOS in a mixed environment, the minimum firmware version must
be Fabric OS versions 2.6.1, 3.1, 4.1 or later. Once the SAN is properly prepared, it is just a matter of enabling security mode
and implementing the previously planned switch roles and policies that have been planned out in advance.
5.10.1. Secure Fabric OS Concepts

Secure Fabric OS functionality falls into five basic areas:
• Fabric Configuration Server (FCS) provides a centralized way to manage fabric-wide configurations and policies.
• Management Access Control (MAC) adds additional layers of granularity when enforcing which devices can access
SAN switches by way of which applications.
• Secure Management Channel provides a more secure method for running management applications that use
encrypted passwords and certificates for authentication.
• Switch Connection Control (SCC) improves switch-to-switch authentication by allowing the use of digital
certificates as well as locking down which ports can become E_ports.
• Device Connection Control (DCC) allows only specific devices into the fabric (per their WWNs) from a specific
port or group of ports.
Each of these areas are enforced by using policies. The concept is similar to security related administration tasks in Windows
or UNIX. When complete, the security information is stored in several databases distributed and enforced by the Primary FCS
switch.

5.10.2. Secure Fabric OS Switch Roles

The implementation of secure mode on a fabric requires grouping the switches logically into three areas:
• Primary FCS Switch: This label applies to a single, uniquely powerful switch that is the sole owner of read/write
privilege for fabric-wide operations. Design criteria for selecting this switch include:
• The switch that is in the most secure, best controlled physical location (typically not at a remote office)
• The most robust switch in the fabric (fastest CPU and highest availability)
• A core switch that is physically near the largest number of switches in the fabric
• Backup FCS Switch: One or more switches that can become the Primary FCS Switch if it becomes unavailable. All FCS
switches conform to a conventional order, in which the first switch is the Primary FCS Switch, the second switch is the
first Backup FCS switch to take over in the event of a failure, and so on. Automatic failover occurs to the first Backup
FCS switch. Manual failover can be initiated by the SAN administrator to any Backup FCS switch defined in the FCS list.
These switches do not have the ability to make changes to fabric-wide configurations unless they become the Primary
FCS Switch.
• Non-FCS Switch: This third class of switch encompasses all the remaining switches in the fabric. Any device not
designated as an FCS switch type simply functions as a member switch that will never have the ability to modify
fabric-wide configuration parameters.
Think of the Primary and Backup FCS switches as members of a trusted switch group. The Primary FCS (Trusted) switch
becomes the management point for fabric-wide configuration changes. The Backup FCS switches are trusted members of the
fabric and are there to provide redundancy in the event the Primary FCS switch goes down. To maintain Secure Fabric OS
policy settings, the Primary FCS switch is solely responsible for applying configurations, which are contained in four separate
databases, centrally managed on it. One of these contain fabric-wide passwords for each user account. A temporary password
can be assigned to a single switch for maintenance purposes without having to give away the FCS password. There is a
separate password list for FCS and Non-FCS switches. Non-FCS switches have only admin and user level account access. The
Non-FCS switches are just members of the fabric with policies enforced by the Primary FCS switch.
In order to join a SFOS fabric, new switches must have their digital certificates authenticated. Brocade has developed an
additional protocol, called Fibre Channel Authentication Protocol (FCAP) that is used as part of E_port link initialization
when Secure Fabric OS is turned on.
When enabling secure mode for the first time on the Primary FCS switch, be aware that each switch in the fabric must be
rebooted for the Primary FCS switch to propagate all of the information under its control to the Backup FCS switches. The
Primary FCS switch propagates the following fabric-wide configuration databases as shown in Table 5-11.
Table 5-11 Propagated Primary FCS Configuration Databases
Propagated Primary FCS Switch Configurations

1. Security Polices
2. Zoning Configuration Database
3. Fabric Password Database
4. SNMP Community Strings
5. Date and Time using the Primary FCS Time Server
Guideline: For Core/Edge topologies, such as the case study, it is recommended to select a core switch as the
Primary FCS. Use a SilkWorm 12000 or 24000 core fabric switch as the Primary FCS or trusted switch
if available.

Warning: All the items in Table 5-11 will be wiped out on switches that are allowed to be added to a Secure Fabric OS
enabled fabric, and replaced by the Primary FCS switch. It is highly recommended to NOT add switches that
contain zoning information, before making a backup with configupload, since all zoning configurations will
be written over.
Guideline: During the configuration of the SNMP community strings, change the default passwords of “public” and
“private”.
5.10.3. Secure Fabric OS Default Policy Settings

With Secure Fabric OS (SFOS) enabled, the default policy settings allow for any switch or device attached to fabric access to
any resource. For reference Table 5-12 describes each of the SFOS features and the applied default settings once SFOS is
turned on.
Table 5-12 Default Secure Fabric OS Policy settings
Secure Fabric OS Security Setting

Feature
FCS_POLICY This policy must exist and cannot be empty. It is a list of all FCS switches. Any
switch that is not in the FCS_POLICY list is a Non-FCS switch member.
The FCS_POLICY lists members by switch name and WWN. This information
is contained on the Primary FCS switch and it is the first switch in the list dis-
played by secmodeshow.
MAC Policies: SNMP, Telnet, These policies are in the “No Policy” state by default; as such, they do not limit
HTTP, SES, Management Server or block access.
(MS) Serial,
Front Panel (SilkWorm 2800 only)
Switch Connection Control (SCC) The SCC is set to “No Policy” by default: a switch with any WWN can connect
to the fabric.
Device Connection Control (DCC) The DCC is set to “No Policy” by default: any device can connect to any port.
Options By default, options are not enabled and therefore allow WWW zoning, for exam-
ple.
5.11. Secure Fabric OS Pre-Installation Planning

As with any new security measures, there needs to be a plan in place before staging Secure Fabric OS (SFOS). This section
will provide a SFOS Pre-installation checklist that outlines at a high level what needs to be accomplished before the SAN is
ready for SFOS. Once these steps are complete, the SAN is prepared.
An SFOS Implementation Plan checklist follows. This checklist provides high-level guidelines as to what is needed for a
successful staging of SFOS. The details of each checklist item follow the high level recommendations and is meant to help
with putting together a plan that is tailored for a specific implementation.

5.12. Secure Fabric OS Pre-Installation Checklist

Here are some pre–installation steps that require completion prior to enabling Secure Fabric OS. It is important to do this
ahead of time so that the SAN is prepared for SFOS security to be enabled. Table 5-13 contains a checklist that will help in
defining a tailored plan.
Note: The SilkWorm 1000 series of switches do not support SFOS.
Note: If more information is desired, each checklist item is expanded upon in the subsequent sections.
Table 5-13 Secure Fabric OS Preparation Checklist
Secure Fabric OS Preparation Checklist

1. Obtain and read Secure Fabric OS documentation. See the list of recommended documents below.
2. Be safe. Backup switch configurations with configupload. Prior information such as zoning will be
wiped out when a switch or fabric is allowed to join a Secure Fabric OS enabled fabric.
3. Verify PKI Objects Exist. This is required for Secure Fabric OS implementation.
• Pkishow (Fabric OS 4.1 and later)
• Configshow “pki” (Fabric OS 3.1/2.6.1 and later)
4. If the PKI objects do not exist, obtain the PKICert tool to setup the fabric for secure mode. This
is required for older switches. This utility runs on Windows and Solaris only.
5. Download and Install Brocade SecTelnet and Secure Shell client (SSH) Security Software
Utilities
6. Verify Fabric OS Version. Update as required.
7. Install Security and Zoning Licenses on all switches in the SAN. This is required for Secure Fabric
OS.
8. Schedule downtime when enabling secure mode. A reboot of each switch in the fabric SAN is
required.
9. Highly Recommended: Set the Core PID on all switches running Fabric OS 2.x and 3.x.
1. Obtain and Read Secure Fabric OS Documentation

For comprehensive information on Brocade Secure Fabric OS implementation, it is highly recommended to obtain and
read the following documentation. Refer to the Preface for additional references.
• Secure Fabric OS Quickstart Guide (publication number: 53-0000352-03)
• Secure Fabric OS Users Guide (publication number: 53-0000526-03)
• SAN Security: A Best Practices Guide (publication number: GA-RG-250-00)
2. Backup the Brocade Switch Configurations
When any big change is going to take place within the SAN, it is a good idea to be prepared with a backup of each switch
configuration. Implementing Secure Fabric OS (SFOS) definitely fits into that category. The fact of the matter is that even
with planning, something may go astray. Use configupload to preserve a backup of the switch before enabling SFOS.
For larger fabrics of more than four switches, use Brocade Fabric Manager, as it can backup all switches at once. Give
strong consideration to making this a baseline configuration.

3. Verify PKI Objects Exist
For new and existing switches, verify PKI objects exist by issuing pkishow at the command line for Fabric OS version 4.1.
Use configshow “pki” for all other versions of Fabric OS. Use an ordinary telnet client since Secure Fabric OS is
not enabled yet. Before Secure Fabric OS can be enabled and SecTelnet used, these objects must exist.
4. Obtain the PKICert tool to setup the fabric for Secure Mode
If the PKI objects do not exist, install PKICert to obtain the Certificate Signing Request (CSR) and install the certificates.
Usually these tasks only need to be performed on older switches that were factory installed prior to January 2002. There
are three overall tasks to perform with PKIcert, as listed below. PKICert can be used to obtain the objects on an entire
fabric. For multiple fabrics, it is recommended to do one fabric at a time. Scheduled downtime in single fabrics is not
required, but is recommended as part of Secure FOS implementation preparation.
• Request certificates with the Certificate Signing Request (CSR).
• Obtain the Certificate Signing Request from the fabric. This is an XML file and is required to get the digital
certificates. PKIcert will prompt the administrator for a name.
• Submit the CSR, obtain the digital certificates. These are contained in a second XML file that will be E-mailed after
submission to a secure site. The file will have a unique number in the filename.
• Load Digital Certificates onto the fabric. New switches from the factory with Fabric OS 2.6.1, 3.1 and 4.1 have them
pre-installed.
5. Download and Install Brocade Security Software Tools
Install the following tools on the hosts that will be used for Secure Fabric OS administration purposes. Refer to the
procedure offered by the switch supplier for the location and download instructions.
• Brocade SecTelnet 1.0 or higher
• SSH Client – any client that supports version 2 of the protocol
6. Verify Fabric OS Version
To verify the firmware versions, use the version command on all switches except those running Fabric OS 4.x. For any
switch running Fabric OS 4.x, use the firmwareshow command. Before upgrading, be sure to audit the current
end-to-end configuration to understand potential impacts during the upgrade.
7. Verify All Switches Have Zoning and Security License Keys
Both the Zoning and Security licenses are required for enabling Secure Fabric OS. This is because the Primary FCS
(Trusted) switch is responsible for doing all zoning management and updates to other switches in the fabric. These other
switches must be able to accept these changes. If not installed, follow the switch provider procedure to get them installed.
8. Schedule Downtime When Secure Mode is Enabled
Select the Primary FCS switch in the fabric. After enabling secure mode, all the switches will be rebooted in the fabric in
order for the Primary FCS switch to propagate and enforce the Secure Fabric OS SAN security features.
9. Set the Core PID Format
While not required for implementing Secure Fabric OS, now is the time to plan for it. Setting the Core PID format on
switches running Fabric OS 2.x and 3.x is required when attaching to switches running Fabric OS 4.x. It is strongly
recommended that all initial configurations of Fabric OS 2.x or 3.x based switches have the Core PID set.
Guideline: Setting the Core PID at the initial staging phase, on Fabric OS 2.x or 3.x based switches, will allow for
a seamless introduction of a switch running Fabric OS 4.x into the SAN fabric.
Now that these steps are complete, the SAN is ready for Secure Fabric OS deployment. In order for deployment to be
effective, there needs to be an understanding of what is going to be done. 5.13. Planning the Secure Fabric OS Implementation
on page 4-26 provides some guidelines to consider when putting together this plan.

5.13. Planning the Secure Fabric OS Implementation

Proper planning is essential to create the most secure environment possible. It is highly recommended that the checklist of
activities in Table 5-14 be used as a basis for any specific plan, which should be followed rigorously. The case study SAN will
be used as an example to provide details for each checklist item.
Table 5-14 Secure Fabric OS Implementation Plan Checklist
Secure Fabric OS Implementation Plan Checklist

1. Create a SFOS switch and device list for SFOS policies. This list should contain hostnames, switch
names, IP addresses, and WWNs
2. Plan the FCS placements for each switch. Select a Primary and Backup FCS switch.
3. Covertly mark FCS Switches. Use a small physical mark so that FCS switches are easily located.
4. Determine policy requirements for each device and host.
5. Select the SFOS management hosts. Brocade SecTelnet and a Secure Shell (SSH) client may be required.
6. Perform a final review of all configuration selections. Verify all changes have been included.
7. (Optional) Disable Telnet Daemon on switches running Fabric OS 4.1or later.
8. Set the switch Recovery Password and Boot Password (Fabric OS 4.1or later).
9. Enable SFOS and verify its operation.
10. Backup Primary FCS switch configurations with configupload.
Warning: Each switch in the fabric needs to be rebooted to activate Secure Fabric OS. Be aware that enabling Secure Fabric
OS will reboot BOTH CPs simultaneously on the SilkWorm 12000 and 24000. It is recommended to schedule
downtime on single fabric SANs before enabling Secure Fabric OS.
1. Create a Secure Fabric OS Switch and Device List

The list should contain switch and host Names, WWNs and IP addresses that will have polices applied to them. This is
where having a complete record of SAN components is helpful. If this information is readily available, it is easy to
consolidate that information from previously created spreadsheets.
2. Plan FCS Switch Placements
Select the Primary FCS and Backup FCS switch locations, based upon the location within the fabric topology. All
remaining switches will be Non-FCS switches. The Primary FCS will become the administrative point in the fabric;
therefore select a core switch as the Primary FCS. If there is a second Core switch, utilize it as a Backup FCS switch.
Should the Primary FCS switch become unavailable, the first available Backup FCS switch will be used to manage the
fabric. The SilkWorm 24000 makes the best choice as the Primary FCS switch given it is the highest available switch
platform in the Brocade SilkWorm product family.
Guideline: It is highly recommended to select a Core switch (preferably a SilkWorm 24000 or 12000) as the
Primary FCS switch. If the fabric has contains more than four switches, select a minimum of three
Backup FCS switches.
Guideline: Consider a locked closet to physically secure the Primary and Backup FCS switches and/or the
management station.

3. Covertly mark FCS Switches
Use a small physical mark so that FCS switches are easily located.
4. Determine the Policy Requirements
Consider which devices should be under access control before creating policies. Record each device and host, define the
Secure Fabric OS policy to be applied to each. When making these choices, be sure to work within general corporate
security guidelines being applied to other IT equipment.
5. Choose the Secure Fabric OS Management Hosts
The Secure Fabric OS management hosts will have SSH and SecTelnet installed for administrative purposes. Select at
least two, but not more than five hosts. To maximize overall security effectiveness, these hosts should be locked down. At
a minimum, they should have enterprise LAN based security and be barred from physical access. Two are required for
redundancy purposes. More than five yields too many access points into the fabric.
6. Perform a Final Review
To make sure all the policies are in place, review the policies with the cross-functional team responsible for making the
decisions and implementing the configuration. This will prevent last minute changes from being over-looked.
7. (Optional) Disable Telnet on Version 4.1 Only
For the highest level of security, it is recommended to disable the telnet daemon. As of Fabric OS v4.1, the telnet daemon
can be disabled on a per switch basis. Disabling telnet is done with the configure command. The switch does not need
to be brought offline for this task. Use an console port connection when enabling/disabling the telnet daemon. Refer to
section 6.3.12. (Optional) Disabling the Telnet Daemon When Secure Mode is Enabled (Fabric OS 4.1 or greater only) on
page 6-18.
8. Set Recovery Password and Boot Password
For maximum security change the passwords from the defaults. Write down and secure the new passwords. To change the
passwords, follow the procedures in the Secure Fabric OS Users Guide (part number: 53-0000526-02).
9. Backup the Brocade Switch Configurations
It is highly recommended to backup the Primary FCS using the command configupload, after enabling secure mode. This
will preserve the FCS_LIST, passwords and other policy configurations that are set on the Primary FCS. If required, use
configdownload to restore the data to the Primary FCS.
Warning: The Primary FCS databases are not automatically backed up when secure mode is enabled. Data will be lost if the
FCS switch is not backed up before the command secmodedisable is done on the Primary FCS switch. To
backup the Secure Fabric OS data, use the command configupload on the Primary FCS.
10. Verification Plan

After the SAN security measures and secure mode is enabled with policies set, they need to be tested using a verification
plan. Put together a plan that makes sense for the specific SFOS environment that is going to be implemented. Here are
some suggestions for the verification plan:
• Add some switches and devices that do not have access rights.
• Try adding a switch to the fabric, which is locked out by an SCC policy.
• Add an HBA that is not in the DCC list.
• Try to telnet to the Primary FCS with a host that should not have access.

5.14. SAN Secure Fabric OS Software Utility

Considerations
This section will provide an overview of SecTelnet and SSH. SSH can be used as a standalone utility for secure management
(with or without Secure Fabric OS). SecTelnet requires Secure Fabric OS (SFOS) for secure management of the SAN. Both
are referred to as “out of band” management. To get these applications, contact the switch provider. Brocade does not supply
the SSH client software.
5.14.1. PKIcert Utility

Only binaries for Windows and Solaris have been developed. Use Version 1.0.5 or higher to do the certificate installation
procedure. Generally older fabrics with switches that pre-date Brocade Secure Fabric OS are suspect. If required, this
procedure only has to be done once per fabric. To find out if the required digital certificates and other PKI infrastructure items
exist, such as public and private keys on Fabric OS 2.6.1 or 3.1, use configshow “pki”. For switches running Fabric OS
4.x, use pkishow.
5.14.2. Brocade Secure Telnet (SecTelnet)

The Brocade SecTelnet client allows for password encryption upon logging in. The SecTelnet binaries are available for
Windows and Solaris only. In addition, use of SecTelnet requires that digital certificates be installed on each switch that is
accessed. If installed, use the Brocade PKIcert application to obtain the Digital Certificates. Use of SecTelnet outside of
Secure Fabric OS does work but is not supported by Brocade. SecTelnet works for all versions of Fabric OS, however Secure
Fabric OS must be enabled for support. Secure Fabric OS is available on Fabric OS versions 2.6.x, 3.1 and 4.1 or higher.
Note: Sectelnet or SSH must be used to administer the Primary FCS switch when running Secure Fabric OS. Brocade
Sectelnet requires digital certificates are installed on each switch to be administered. This utility only encrypts the
passwords sent over the LAN, all other commands are sent as clear text. Refer to section 6.5.3.2. Verify Digital
Certificates Exist on page 6-31 for more the commands used to verify digital certificates.
Note: In a fabric that contains switches running Fabric OS 2.x, the maximum security DB size is limited to 32 KB, 16 KB of
which can be active. In a fabric containing switches running Fabric OS 3.x or 4.x, the security DB size maximum is
128 KB, 64 KB of which can be active. For all fabrics, the maximum number of DCC policies is limited to 620.
Caution: When using a switch with Fabric OS 2.6.1, and Secure mode enabled as an FCS switch, the maximum
database size is 16 KB.
5.14.3. Using Secure Shell (SSH)

Secure Shell (SSH) is a standards based secure method for accessing SilkWorm switches running Fabric OS 4.1 or later. Any
SSH client that supports version 2 of the protocol can be used. There are literally hundreds of freeware SSH clients available
that have this capability. On the switch side, SSH is only supported on Fabric OS 4.1 or higher. Two popular clients have been
tested, Putty and Fsecure.

5.15. LAN Planning Considerations

There are a few guidelines to consider when attaching the fabric to the corporate LAN infrastructure. In general, it is highly
recommended to configure a separate VLAN/subnet for each fabric. If at all possible, avoid the use of proxy servers from the
SAN management stations outside the local subnet. In fact, it is recommended to use a management station on the same
VLAN/subnet. For detailed guidelines on how to connect Brocade switches to the corporate network, please refer to Appendix
B, Long-distance Technologies for Storage Area Networks and the technical white paper titled: LAN Guidelines for Brocade
SilkWorm Switches (publication number: 53-0000350-01).
5.16. Extended Fabrics Planning Essentials

This section will provide the essential information for the required for planning connections of Brocade SilkWorm fabrics over
longer distances. One common reason is for data replication which gives site redundancy. In this way, if one site goes down
due to a disaster, the data can be recovered and brought online in minutes rather than days. Another typical use is consolidated
remote data archival to tape.
SAN long distance connectivity may involve single mode fiber or through more sophisticated network equipment that allows
for greater line availability. For Metro Area Networks (MAN), those distances up to 100 Km, DWDM equipment is generally
used. Longer distances that go up to thousands of kilometers generally require FC protocol conversion. This means a Wide
Area Network (WAN) transport method is required. Different equipment, such as an ATM long haul switch, maybe required.
These types of devices are out of scope for this document. The focus will be on the Brocade SilkWorm long distance
connectivity and the essential information for utilizing Brocade Extended Fabrics. For additional information refer to
Appendix B, Long-distance Technologies for Storage Area Networks.
5.16.1. Overview
All Brocade SilkWorm switches, except for the SilkWorm 1000 Series, support a separate buffer-to-buffer flow control circuit
for each of the eight virtual channels (VCs) used on ISLs. Of the eight VCs used, four are used for management of Fibre
Channel unicast frames and each are loaded with five credits on a Normal (L0) ISL. This allows switches to be interconnected
at distances of up to 5km for 2 Gbit/sec links and 10km for 1 Gbit/sec links.
The Brocade Extended Fabrics license allows ISLs to be connected at up to 60km for 2 Gbit/sec links and up to 100km for
1 Gbit/sec links, while maintaining maximum bandwidth. This is accomplished by compacting credits on all four virtual
channels normally used for unicast frames onto a single virtual channel (VC 2). Fabric OS v2.6 has three EF modes that can be
used to configure the amount of credits available to ISLs on long distance links. An additional LE mode that supports
2 Gbit/sec Fibre Channel (FC) performance at 10 km without requiring an EF license was introduced in Fabric OS v3.0/v4.0.
Two new modes were introduced in Fabric OS v3.1 and v4.1 that allow for further enhancements to extended distance links.
5.16.2. Compatibility and Interoperability

If an Extended Fabrics port is to be installed on a SilkWorm 2000 Series switch, the fabric wide configuration parameter
fabric.ops.mode.longDistance must be set to 1 on all switches operating within the fabric. Additionally, each
long distance port must be set using the portCfgLongDistance command. Each of the two ports within a long
distance ISL must be configured identically, otherwise fabric segmentation will occur.
Switches running Fabric OS 3.x or 4.x have new features that do not require long distance ports to be configured at the fabric
level. Only individual port configuration is necessary.

In a mixed fabric configuration, where long distance ports are installed on switches running Fabric OS 3.x and/or 4.x, only
port level configuration is required. If a long distance ISL is created between two SilkWorm 2000 Series switches in a mixed
fabric, then the fabric wide long distance parameter must be set on all switches within the fabric, as well as port level
configuration on the long distance ports.
Brocade Security features are supported on Extended Fabric links. This includes connections over dark fiber and DWDM
networks.
Note: When upgrading from Fabric OS v4.0.x to Fabric OS v4.1.x (or later) all Extended Fabric ports will be set to L0 mode.
Be sure to reconfigure the Extended Fabric ports appropriately if using Extended Fabrics.
Note: Long distance links are not supported from a SilkWorm 2000 series switch to a SilkWorm 3200/3800 or 12000. The
links following links are not supported: SilkWorm 2x00-to-3x00 or 2x00-to-12000/24000.
Note: The Brocade Trunking feature, which allows multiple ISLs within a quad to load balance traffic, is not currently
supported on ports configured for Extended Fabrics.
5.16.3. Extended Fabrics Modes

Table 5-15 lists the various Extended Fabrics modes and requirements. Two new Extended Fabrics modes introduced in Fabric
OS v3.1 and Fabric OS v4.1 are L0.5 and LD. L0.5 can support distances at up to 25 km. LD, or Dynamic long distance mode,
can automatically configure the required amount of credits based on the actual link distance. This is determined by calculating
latency on the link using the link round trip timer in the ASIC.
Table 5-15 Extended Fabrics Modes
EF Buffer Allocation Distance Distance Fabric EF License

Mode @ @ OS Required
1 Gbit/sec 2 Gbit/sec Release
1 Gbit/sec 2 Gbit/sec
L0 5 (26) 5 (26) 10 km 5 km All NO
LE 13 19 ~ 10 km 3.x, 4.x NO
L0.5 19 34 25 km 25 km 3.1, 4.1 YES
L1 27 54 50 km 50 km All YES
L2 60 64 100 km 60 km All YES
LD* Auto Auto Auto Auto 3.1, 4.1 YES
*The maximum distance for LD is the same as for L2.

For dynamic long distance links, the number of credits can be approximated using the following formula. The data rate is
1.0625 for 1 Gbit/sec and 2.125 for 2 Gbit/sec Fibre Channel. This equation can only be used as an estimation for the number
of credits that will be allocated to a given port. The actual amount will most likely be slightly higher.
Buffer Credits = ((Distance in km) * (Data Rate) * 1000) / 2112

Since only 108 credits are available for use on ports within each quad, configuring long distance ports may cause other ports to
become disabled if there are not enough credits available. For example, if two 2 Gbit/sec ports in a quad are configured for L1
mode, each will be allocated 54 buffer-to-buffer credits and cause the other two ports within the quad to become disabled.
Table 3-2 lists some possible port configurations that can be utilized with static long distance modes.
Table 5-16 Port Configurations
Speed Port a Port b Port c Port d Total Credits

1 Gbit/sec L2(60) L1(27)/E(27) Fx(16) X 103
1 Gbit/sec L2(60) Fx(16) Fx(16) Fx(16) 108
1 Gbit/sec L1(27)/E(27)/ L1(27)/E(27)/ L1(27)/E(27)/ L1(27)/E(27)/ 64-108

Fx(16) Fx(16) Fx(16) Fx(16)
2 Gbit/sec L2(108) X X X 108
2 Gbit/sec L1(54) L1(54)/E(27) X X 108(81)
2 Gbit/sec L1(54) E(27) E(27)/Fx(16) X 108 (97)
2 Gbit/sec L1(54) Fx(16) Fx(16) Fx(16) 102
2 Gbit/sec E(17)Fx(16) E(17)/Fx(16) E(27)/Fx(16) E(27)/Fx(16) 108 (64)
Guideline: For switches running Fabric OS 3.1/4.1 or greater, use LD mode for long distance connectivity.

5.17. SilkWorm 12000 Upgrade Planning Guidelines

SilkWorm 24000 blades will seamlessly seat into SilkWorm 12000 chassis. This makes an upgrade of a SilkWorm 12000
possible, though it is not recommended in certain cases. For the detailed upgrade plan and procedure contact your switch
provider.
The upgrade process will require scheduled downtime and extensive planning as well as a number of manual configuration
steps. Be aware that Fabric OS 4.2 does not allow a SilkWorm 12000 chassis to have both 12000 and 24000 blades. To
perform an upgrade on a chassis running Fabric OS 4.2, all blades, including the CP cards, must be swapped out of the
SilkWorm 12000 chassis.
In terms of SilkWorm 12000 chassis configurations, single domains are the easiest to upgrade. A dual domain SilkWorm
12000 is significantly more complex to upgrade. The chassis domain count will be reduced from 2 to 1, among other
complications. Split chassis upgrades, where each logical switch is used in a separate fabrics are extremely complex. An
upgrade in this case is highly discouraged. For details on the planning, implementation and testing of a SilkWorm 12000 to
24000 upgrade please refer to the SilkWorm 12000 to 24000 Upgrade Guide.
Note: Upgrades are straight forward on a single domain SilkWorm 12000 and can easily be done without down time
assuming either a dual fabric or that the SilkWorm 12000 is part of a dual core in a Core/Edge fabric. If the SilkWorm
12000 is part of a dual domain or both domains are utilized, please refer to your switch vendor for detailed guidelines.
Guideline: For minimal disruption when scaling a SAN Fabric, it is strongly recommended to build out the SilkWorm
12000 chassis with SilkWorm 12000 port blades. Non-disruptively add a SilkWorm 24000 chassis as
required to increase the desired port count.

Chapter
Deployment - Staging and Validation
6
Once the plan is complete and the resources readied, the new SAN can be built and prepared for production. Staging the SAN
is more than uncrating, racking and installing the Brocade switches and other components. Staging also includes configuring
the firmware and software of each component. Guidelines on how to accomplish these tasks are presented in this chapter in the
form of checklists. The case study will be used extensively in this chapter to provide examples of commands used in the
staging phase of SAN deployment.
This chapter assumes that a new Core/Edge fabric with “clean” Brocade switches will be staged. A “clean” switch has no
defined or active zoning configuration and has all default settings. It is assumed throughout this chapter that a Core/Edge
topology has been chosen as part of the design criteria. The guidelines in this chapter may need to be modified for other
topologies. This chapter will not explicitly cover an existing SAN fabric migration to other Brocade switch platforms. Please
refer to the SAN Migration Guide (publication number 53-0000360-02) for recommended guidelines and procedures.
Staging a new SAN Fabric requires essentially two tasks. The first task is uncrating, racking, cabling and providing power to
the Brocade switches. The second task is configuring the Brocade Fabric Operating System firmware. The focus of this chapter
will be on the commands used in staging Brocade SAN fabrics with Fabric OS 2.6.2, 3.2 and 4.2. New high-level
troubleshooting functions are also introduced within these Fabric OS releases and will be discussed at a high level in this
chapter. Since most of the new functionality is in Fabric OS 3.2 and 4.2, Fabric OS 2.6.2 will get limited coverage.
This chapter will not provide recommendations on racking, cabling or power installation for the Brocade SilkWorm Fabric
Switch Family. Guidelines for these tasks are provided in section 5.6. The Rack Layout Plan on page 5-10. Nor will this
chapter focus on the devices, such as HBAs and storage targets, attached to the Brocade SAN Fabric.
All of these functions will be shown as a case study example using Telnet based commands. Differences among commands in
each of the Fabric OS releases will be pointed out as each task is discussed. Many of these same functions can be accomplished
with the Brocade Web Tools and Fabric Manager, refer to Section III SAN Management for more information. Brocade Fabric
Manager and Web Tools will be referenced at a high level in certain situations, as some tasks are easier with these software
tools, especially with larger port count SANs. Please see the Brocade Fabric Manager User’s Guide (part number:
53-0000823-06) and the Brocade Advanced Web Tools Administrator’s Guide (part number: 53-0000522-05) for detailed
instructions on usage.
Guideline: For an online list of commands, use the help command. To view the online command reference, use the
command help <command>. For example, help tsclockserver will provide the online reference
for the tsclockserver command. For more information please refer to the Brocade Fabric OS
Reference Guide.

6 Deployment - Staging and Validation
6.1. Case Study

A case study is used as an example to provide context as concepts are introduced and new features explained.
Please note that the case study was created using a SilkWorm 12000 as the core switch. The new SAN configuration can use
either a SilkWorm 12000 or the high port count single domain SilkWorm 24000 as the core switch. A fabric consisting of a
combination of Brocade SilkWorm 12000, 24000, 3900, 3850 and 3250 models has a unique advantage of a common code set
across these models.
6.1.1. Case Study SAN Description

The case study SAN contains two fabrics that are redundant. Each fabric contains six switches in resilient Core/Edge topology.
Each fabric has two SilkWorm 2800 fabric switches, two SilkWorm 3800 fabric switches, a SilkWorm 3900 switch, and a
SilkWorm 12000 core fabric switch. The SilkWorm 2800 switches are connected with two 1 Gbit/sec ISLs. The SilkWorm
12000 switches are deployed at the core to provide the maximum scalability and availability. The SilkWorm 3800 and
SilkWorm 3900 switches are connected at the edge using two ISLs per trunk group. This yields 4 Gbit/sec bandwidth for each
logical connection. The design provides 86 device ports per fabric, with a total of 172 ports available in the SAN. Brocade
Trunking technology provides up to four ISLs that form a single 8 Gbit/sec logical ISL. If the design recommendations are
followed, bandwidth can be added on the fly, just by adding a cable to any trunk group. With the SilkWorm 12000 director at
the core, the SAN used in this solution can scale quickly and easily to hundreds of ports using SilkWorm 16, 32, 64, and 128
port switches on the edge.
The attached hosts include Solaris, Windows, and HPUX systems. These hosts share the RAID array connections that are
connected on the SAN. To maximize the SAN availability, all hosts are running multi-pathing software. Figure 6-1 shows the
SAN design.
Sun W2000
2800 2800 3800 3800 3900 2800 2800 3800 3800 3900
2ISLtrunk
1ISL
Figure 6-1 Case Study SAN Design

Deployment - Staging and Validation 6
The assumptions used in the case study are:
• The installation and configuration of the LAN infrastructure, HBA drivers, multi-pathing software, and host
applications in the case study SAN is beyond the document scope. These areas are covered only as it pertains to this
chapter.
• All host bus adaptors (HBA) are installed, the appropriate drivers loaded and configured for F_port (point-to-point)
attachment.
• The host and storage devices are powered on, initialized and are ready to be attached to each fabric.
The SAN has been designed following Brocade recommendations in Chapter 2, SAN Design & Architecture Concepts.
6.2. Powering the SAN Equipment

In the event of that the SAN needs to be brought online, it is recommended to follow the power on sequence in Table 6-1. This
will ensure a clean bring up of all devices.
Caution: If the sequence is not followed, some hosts may not be able to access the storage devices.
Table 6-1 Recommended Power On Sequence
Recommended Power On Sequence
1. IP Infrastructure
2. Brocade Switches
3. Storage Devices
4. All Hosts
6.3. Preparing the Switches for the SAN Fabric

There are many possible ways to configure the switches that make up a fabric. Such as the configuration of a switch is
dependant upon its role. During the planning phase of deployment, the following questions should have been addressed:
• Is it a principal switch?
• Is it a core or edge switch?
• Is long distance required?
• Are the correct license keys installed?
Once the answers are known, the goal of this section is to provide guidance on preparing Brocade Fabric OS for production in
a multi-switch SAN fabric. There are two major steps. First, prepare each switch for attaching to the corporate LAN
infrastructure and joining a fabric. The second step is to do the fabric wide configuration, this being primarily zoning. Each
step will have a separate checklist. As with the rest of the document, this section first provides “bare bones” guidelines. Each
checklist item is then expanded upon to provide detail into how each task is completed. Examples will be used throughout.
Most of these steps apply to all versions of Fabric OS. Those steps that are unique to a particular version will be called out
separately. Some guidelines will be considered optional and will be noted as such in Table 6-2. All commands discussed in this
section are available to the admin user.

Table 6-2 Switch Preparation Checklist
Brocade Switch Preparation Checklist

1. Gather Planning Information (Switch Spreadsheet)
2. Check Fabric OS version and status
3. Set IP Address(es)
4. Set Date and Time
5. Set the Switch Name
6. Set the Domain ID Number and Set Core PID on non-Fabric OS 4.x switches
7. (Optional) Extended Edge PID Format Setting
8. Add Fabric OS licenses
9. Optional: Name devices with PortName
10. Optional: Setting up a preferred principal switch in the fabric (Fabric OS 4.1 or later)
11. Set Telnet Session Timeout Value with the timeout command
12. Optional: Disabling the telnet daemon when secure mode is enabled (Fabric OS 4.1 or later)
13. Optional: Setup ports for Extended Fabrics (see section 5.16. Extended Fabrics Planning Essentials on page 5-29)
14. Optional: Set up Fabric Watch, SNMP Traps, switchstatuspolicyset (see Section III SAN Management)
15. Baseline and backup the switch with configupload
6.3.1. Gather Planning Documentation

Before getting started, gather the switch spreadsheet compiled during the planning phase, as discussed in section 5.7.
Documentation Guidelines on page 5-12. This shows the planned IP addresses and domain names as well as the switch roles
(Core Switch, Edge switch, Management Switch, etc.) Once the IP addresses and domain numbers are known, its just a matter
of executing the appropriate commands to set these values on the associated switch.
6.3.2. Check Fabric OS Version and Environmental Status

There are several commands that are recommended to be run to check the Fabric OS (FOS) version and the overall switch
environmental status after accessing the switch through a serial cable (see section 6.6. Validating the SAN on page 6-58). The
recommended commands to execute on Fabric OS 2.6.x and 3.1.x are: version (firmwareshow for Fabric OS 4.x),
uptime, switchstatushow, sensorshow and portcfgshow. Outputs from the case study are shown below.
Switchstatusshow displays any triggered messages set by switchstatuspolicyset.
Switchstatuspolicyset, discussed in the management section, allows the environmental settings that trigger warning
messages to be customized.
sialab89:admin> version
Kernel: 5.4
Fabric OS: v3.1.0
Made on: Thu Feb 20 15:21:32 PST 2003
Flash: Thu Feb 20 15:22:24 PST 2003
BootProm: Tue Oct 30 10:24:38 PST 2001
Figure 6-2 Version Command
sialab89:admin> uptime
8:01am up 1 day, 12:32, 1 user, load average: 1.29, 1.45, 1.27
int219:admin> switchstatusshow

The overall switch status is HEALTHY/OK
poc166:admin> firmwareshow
Local CP (Slot 5, CP0): Active
Primary partition: v4.1.0
Secondary Partition: v4.1.0
Remote CP (Slot 6, CP1): Standby
Note: If Local CP and Remote CP have different versions of
firmware, please retry firmwaredownload command.
Figure 6-3 Uptime Command
int195:admin> switchstatusshow
The overall switch status is Marginal/Warning
Contributing factors:
* 1 bad power supply and 0 missing power supply triggered the Marginal/Warning status
Figure 6-4 Switchstatusshow Command
As of Fabric OS 2.6.1, 3.1 and 4.1, Sensorshow can be used to display the status and values of each FRU.
int219:admin> sensorshow
sensor 1: (Temperature) is Ok, value is 46 C
sensor 6: (Fan ) is Ok, speed is 3308 RPM
sensor 12: (Power Supply ) is Ok
sensor 13: (Power Supply ) is Ok
Figure 6-5 Sensorshow Command Output
int195:admin> portcfgshow
Ports 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Persistent Disable .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
where ..:OFF, ??:INVALID.
Portcfgshow Command Output for Fabric OS 2.6.1
sialab89:admin> portcfgshow
Ports 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN
Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON
Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC link init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
where AN:AutoNegotiate, ..:OFF, ??:INVALID. LM:L0.5
Figure 6-6 Portcfgshow Command Output for Fabric OS 3.1

6.3.3. Set IP Address

To set IP Addresses on all of Brocade SilkWorm 2 Gbit/sec family of switches, console access to the serial port is required.
To do this, attach a straight through serial cable to the console port on a host and the switch serial port. Configure the terminal
program with the proper serial port settings. These are shown in Table 6-3.
Table 6-3 Brocade Switch Serial Port Settings
Serial Port Settings

• 8-bit
• No parity
• No Flow Control
• One stop bit
• 9600 baud
Once attached, and with the terminal program running, hit Enter a few times to initiate the connection. The switch will
respond with a login prompt.
1. At the prompt, log on as admin and type the command ipAddrSet to set the IP address.
sw0:admin> ipAddrSet
2. Enter the IP Address at the prompt. The example below uses 10.1.23.92.
Ethernet IP Address [192.168.155.89]: 10.1.23.92
3. At the Ethernet subnet mask prompt enter the subnet mask. The example uses 255.255.255.0.
Ethernet Subnet Mask [255.255.255.0]: 255.255.255.0
4. For the Fibre Channel IP address and subnet mask, make sure these are at the default value which is [none]. All factory
installed units are pre-set with this. To reset to the default values enter 0.0.0.0 as shown.
Fibre Channel IP Address [none]: 0.0.0.0
Fibre Channel Subnet Mask [none]: 0.0.0.0
5. Enter the Gateway Address. Since this is on a private VLAN with the Management Station, keep the default of [none] by
hitting Enter. If not at the default value, use 0.0.0.0 to reset it to [none], this is shown below. Enter Y to the configure the
IP addresses immediately.
Gateway Address [none]: 10.1.23.1
Set IP addresses now?
[y = set now, n = next reboot]: y
Committing configuration...done.
The IP configuration is complete. Repeat 1-5 for the remaining switches in each fabric.
When the switch has been configured with an IP address, a telnet, Web Tools or Fabric Manager session can be initiated from
a management host for administrative purposes.
Guideline: Like Fabric OS 2.x and 3.x, Fabric OS 4.x contains ifmodeset which allows the Ethernet port speed
to be set at 10 or 100 Mb/sec. This may be required when attaching to some brands of Ethernet
switches.
Guideline: If using a terminal server to manage serial port connections to all Brocade switches in the SAN be sure
to DISABLE flow control on every virtual serial port on the terminal server side.

6.3.4. Set Date and Time

To set the date and time use the date command. Here is an example that sets the date and time to 4:02pm March 22, 2003.
Using date with no arguments provides the current day and time.
sialab89:admin> date “0322160203”
Sunday March 22 16:02:00 UTC 2003
Date Command
6.3.5. Set the Switch Name

Once the IP address is set, use switchname to set the name of the switch. Setting a name helps to quickly identify a switch
and its role within the fabric. The example below sets the name of the switch to be FabA-C1, since it has been designated to be
the first core switch in the fabric.
sw77:admin> switchname "FabA-C1"
FabA-C1:admin>
Note the prompt changes to reflect the new switch name which is now FabA-C1.
6.3.6. Set Domain ID and Core PID Format in the Same

Session (On Non-Fabric OS 4.x Switches)
As part of the introduction of Fabric OS 4.0, for greater scalability and compatibility, Brocade has provided an option to
change the 24-bit PID format in Fabric OS versions 2.x and 3.x. This is required to support attachment to the higher port count
switches. As an example, this means that an old PID format of 0 would have a 24-bit address of 091500 now when set to 1
becomes 090500. Changing the Core PID only needs to be done once. Because of this, it is highly recommended to change it
on each switch that requires it during the initial staging process.
Guideline: Set the Core PID format to 1 on all 2.x and 3.x based switches during the initial configuration procedure.
This will allow seamless introduction of higher port count switches that run Fabric OS 4.x.
This section will just provide a high level recommended procedure done on the case study SAN. Changing the Core PID is
disruptive, so SANs with single fabrics will require scheduled downtime. For all the recommendations and details for setting
the Core PID format, reference the Brocade Fabric OS Procedures Guide (part number: 53-0000518-03).
Note: The procedure to set the PID format to Extended Edge PID (PID format 2) is identical to the procedure described
below. Extended Edge PID (PID format 2) is available on Fabric OS versions 2.6.2, 3.1.2, 4.2 and higher, as discussed
in section 6.3.7.1. Understanding the Extended Edge PID Format on page 6-12. Contact your switch provider for
details.
To set the switch Domain ID numbers, follow the procedure below. Since the Core PID format can be set in the same session
as the Domain ID, it is highly recommended to take advantage of the initial setup and set it to a value of 1. For SANs with
more than one fabric, do one fabric at a time. While the Domain ID needs to be set on all switches, the Core PID needs only to
be applied to each switch that is running Fabric OS 2.x and 3.x.
1. Login as the admin user. Open a telnet session and issue the switchdisable command. This will cause the switch to
go offline.
2. Now that the switch is offline, type configure and press Enter. The configure menu appears.
int170:admin> configure
Configure...

3. Select the Fabric Parameters submenu by typing Y and pressing Enter. The Domain ID setting appears. Set the domain ID
by entering a number between 1 and 239. The example uses 170 and the line is bold for emphasis.
Fabric parameters (yes, y, no, n): [no] y
Domain: (1..239) [1] 170
4. Press Enter until the Core PID prompt appears and set the Core PID to 1. This will change the default value which is 0.
The change is shown in bold text below.
____ BB credit: (1..27) [16]
R_A_TOV: (4000..120000) [10000]
E_D_TOV: (1000..5000) [2000]
WAN_TOV: (1000..120000) [0]
Data field size: (256..2112) [2112]
Sequence Level Switching: (0..1) [0]
Disable Device Probing: (0..1) [0]
Suppress Class F Traffic: (0..1) [0]
SYNC IO mode: (0..1) [0]
VC Encoded Address Mode: (0..1) [0]
Core Switch PID Format: (0..2) [0] 1
Per-frame Route Priority: (0..1) [0]
Long Distance Fabric: (0..1) [0]
5. Both the Domain ID and Core PID settings have been configured. Press Enter to the rest of the configure sub-menu
choices to skip through the settings as shown.
Virtual Channel parameters (yes, y, no, n): [no]
Zoning Operation parameters (yes, y, no, n): [no]
RSCN Transmission Mode (yes, y, no, n): [no]
Arbitrated Loop parameters (yes, y, no, n): [no]
System services (yes, y, no, n): [no]
_Portlog events enable (yes, y, no, n): [no]
6. The Domain ID and Core PID format have now been set. DO NOT bring the switch online. This will be done in an orderly
fashion in step 8.
7. Repeat steps 1-5 of the procedure for each switch in Fabric A only.
8. Once all of the switches have been set, enable each core switch using switchenable. Ignore any error messages that may
appear.
9. Now enable each edge switch with switchenable.
10. Verify the trunk groups form by issuing trunkshow on each core switch.
11. Verify the fabric is stable using fabricshow from any switch. Check the output and make sure all the switches are
present in Fabric A.
12. Repeat steps 1-11 for Fabric B.
The Switch Domain number and Core PID format is now set for each switch in the SAN fabrics.
6.3.6.1. Set Domain ID Number Only

While it is not necessary, it is highly recommended to set a domain. If no domain is set, the switch will automatically derive a
domain as part of a fabric rebuild. To automatically obtain a domain from the fabric, it is necessary that the switch connect to
the fabric in a disabled state and then be enabled once the connection is complete.
Setting the domain ID number of the switch applies to all versions of Fabric OS. The domain ID number is the first 8-bits of
the 24-bit port ID (PID). The default domain number is 1 for all switches. The following steps provide a high level outline of
the procedure, for more in depth information please refer to the Fabric OS Procedures Guide.
1. Disable the switch with switchdisable. This must be done for all versions of Fabric OS.
2. Use the configure command to set the Domain ID. Running configure will invoke several prompts.

3. Enter Y at the Fabric parameters prompt. The first entry will be the Domain ID. Type in the number. The valid range is
1-239.
Guideline: Use the last octet of the switch IP address for the number.
4. Press Enter to take the other default values.

5. Once the configuration is committed, re-enable the switch with switchenable.
6.3.6.2. Core PID On Switches with Fabric OS 4.2 (or Higher)

The SilkWorm 3850 and 3250 have the Core PID format ON by default. Fabric OS 4.2 or greater only supports PID Formats 1
and 2. If these switches are introduced to a fabric that has switches running Fabric OS 2.x or 3.x with the default Core PID
format setting of 0, the SilkWorm 3850 and 3250 will segment. When introducing these switches to the fabric, the Core PID
formats on the existing switches must be set to 1, as this setting is a fabric wide parameter.
Note: If the PID Formats on the existing switches must use the Extended Edge PID format, the existing SilkWorm switches
must be updated to 2.6.2, or 3.1.2 or greater to use this new setting. However, it is still highly recommended to set all
switches to PID Format 1.
6.3.7. (Optional) Extended Edge PID Format Setting
Note: Displaying and Configuring Extended Edge PID format is available in Fabric OS 2.6.2, 3.1.2 and 4.2 and higher.
This section provides some detail on how to validating the PID Format setting. The best way to display the configuration of the
PID format is to use configshow. The current setting is shown by the fabric.ops.mode.pidFormat variable. Depending on
the switch type, several screens of output may need to be scrolled through. Partial output, showing the setting is shown below.
fabric.ops.mode.pidFormat:1
In this case the PID Format is set to 1. This means the switch is set to the Core PID setting. Other commands to display the PID
setting are portshow and portswapshow.
Warning: Configure may be tempting to use for displaying the PID format. However, in order to see the PID Format setting,
the switch must be disabled.
Please note that the switch port number and area ID are not the same. The switch port number is the physical port on the
switch. The area number is a logical number that represents the logical port and is the second byte of the PID. The switch port
number may consist of physical slot and port numbers or just the switch port number on non-bladed Brocade switch platforms.
To display the PID, use switchshow. It will display both the Area ID and the Port Number on bladed switches. Portshow
displays the PID address for individual ports on all switch types. Other commands that display PID address include
nsallshow, nsshow, portswapshow

To set the PID format, use the configure command. The method of doing this is equivalent to what was shown in section
6.3.6. Set Domain ID and Core PID Format in the Same Session (On Non-Fabric OS 4.x Switches) on page 6-7. Please refer to
this section for details.
Note: As of Fabric OS 2.6.2, 3.1.2 and 4.2, support for a PID format 2 available. This setting is an alternative to the Core
PID format (PID format 1).
PID format 2 was created for hosts which use persistent binding from having to reboot or involve the execution of commands
on the host to recognize devices utilizing new PIDs. This new PID format setting is referred to as Extended Edge PID.
Note: No format setting is required when introducing a SilkWorm 24000, 3250, and/or 3850 to a fabric with a SilkWorm
12000 and/or 3900.
Warning: Be aware that a PID format setting is a fabric wide parameter. All switches in a fabric MUST have the same value
or the fabric will segment.
The PID is the 24-bit address assigned to fabric attached hosts and storage. It contains three bytes: the first byte is the Domain
Number; the second byte is the Area and the third byte the ALPA. Different PID Format values toggle Brocades addressing
scheme in the second byte. Table 6-4 provides a summary of the current available PID formats.
Table 6-4 Available PID Formats
PID Format PID Area ID Affected What it is / What Example/Notes

Name Format Calculated Value Switch Model it does
Value and Range
Native PID 0 0x10 + port number SilkWorm 2000 Sets the first nibble db1a00 (SW 2250)
Format Range: 10-1f and 3000 series of PID Area byte to Device on port 10 of a 16
be 1. Applies to port switch.
switches of up to 16
Only applies on Fabric OS
ports
2.x and 3.x.
Core Switch 1 port number All Sets the first nibble 141a00 (SW 3900)
PID Format Range: 00-ff of PID to start at 0. Device on port 26 of a 32
Applies to switches port switch
of up to 256 ports
Extended 2 0x10 + port number All Sets the first nibble 3f5a00 (SW 24000)
Edge PID Range: 10-00 (with of PID Area byte to Device on port 10 on slot 6.
Format wrap) start at 1 and wraps
On SilkWorm 24000,
(Newly added to 0, with 7 as the
area_ID runs from 16 - 127,
to Fabric OS next to last number.
0 - 15. For port counts of
2.6.2, 3.1.2 Applies to switches
16 or less, Native PID For-
and 4.2) of up to 256 ports
mat is equivalent to
Extended Edge

The SilkWorm 2000 series and 3200/3800 use Core PID format 0 as default. Core PID Format 1 was introduced in the
SilkWorm 3900 and the 12000 director. Core PID Format 1 has been extended to the SilkWorm 24000, 3250 and 3850. Your
existing fabric may have one of the following three configurations.
1. The fabric consists of SilkWorm 2000 series (Fabric OS 2.x) and/or SilkWorm 3200/3800 (Fabric OS 3.x).
2. The fabric consists of SilkWorm 2000 series (Fabric OS 2.x) and/or 3800 (Fabric OS 3.x) and at least one SilkWorm
3900 or SilkWorm 12000 (Fabric OS 4.x) is present.
3. The fabric consists of a SilkWorm 12000 and/or 3900 (Fabric OS 4.x) only.
The Core PID format of fabric configurations 2 and 3 are already set to format 1, therefore no action is required when adding a
SilkWorm 24000, 3250 and/or 3850. However, fabric configuration 1 requires the Core PID format be set to 1 or 2 when
adding a switch running Fabric OS 4.2 or later. It is highly recommended to use Core PID format 1 to be compatible with
newer switches. The following example illustrates a rare special case when you should consider upgrading switches in fabric
configuration 1 to Core PID format 2.
Condition 1: Host binds storage by Port ID (PID) (solaris, AIX, HPUX hosts)
Condition 2: Host config.file update and System reboot is not permissible.
Condition 3: You are not planning to change the Domain ID of the existing switch.
Condition 4: You are not planning to move the physical storage port from its current location.
Note: If you must upgrade to Core PID format 2 on all switches in fabric configuration 1 to avoid host reboot, they must first
be upgraded to the latest available Fabric OS (Fabric OS 2.6.2/3.1.2). Prior versions of Fabric OS do not offer setting
of Core PID Format 2. The switch that you would like to add to this existing fabric must also be set to Core PID format
2. All switches in the fabric must have a uniform format type.
Example: Core PID Setting

To illustrate the use of the Extended Edge PID setting, consider the following scenario:
A SilkWorm switch running Fabric OS 4.1 or greater needs to be integrated into an existing fabric that contains SilkWorm
switches running Fabric OS 2.x or 3.x, all of which have the default Native PID format (PID 0) configured. Hosts on these
fabrics use the PID for persistent binding. Because of the difference in Core PID format, the Brocade switches running Fabric
OS 4.1 or greater will not be able to join the existing fabric unless the Core PID format on the exiting fabric switches is
upgraded to a compatible PID format of the newer switch (default Core PID Format is 1). Setting the Core PID on existing
switches is a disruptive event as it requires the switch to be disabled and rebooted. In this case if the Core PID format is set to
1, as is required, the area field of 24-bit Port ID of all pre-configured devices is changed. The hosts that are binding the storage
port by Port ID require the Config.file to be updated to reflect the new storage port ID, and the system must be rebooted. All
hosts must be rebooted also, and perhaps certain OS settings may need to be changed as well, in order to rediscover the SAN
attached storage.
With the Extended Edge PID Format (PID format 2) set on all switches (old and new), the PID address of the pre-configured
devices will be the same. Thus, there is no change to the config.file and the host does need to be rebooted.
While not discussed in the example, the SilkWorm 3850/3250 (Fabric OS 4.2 or greater) is factory installed with Core PID on
(PID 1). These switches also support the Extended Edge PID Format.
Warning: In all cases where the PID format is changed, all switches in the fabric must be rebooted or disabled and then
re-enabled.

6.3.7.1. Understanding the Extended Edge PID Format

The Extended Edge PID format supports up to 256 ports, as does the Core PID format. The difference lies within how the first
nibble of the second byte is handled. For example a disk device on a SilkWorm 3800 in Native PID mode (PID format 0) has
24-bit address of db1a00. Converting to decimal values, this corresponds to a fabric device physically attached to port 10 of
switch domain 219. When the Core PID value is set, the PID format parameter becomes 1. The address will now be changed to
db0a00. If a host device was using the PID db1a00 for binding a target/LUN number, it will no longer be discovered properly.
With the Extended Edge PID (PID format 2) set, the PID will remain unchanged as db1a00, preserving the target/LUN
binding, as shown in Figure 6-7.
On switches with greater than 16-ports, the area value will be shifted. For example a disk device on a SilkWorm 24000 that is
set to the default Core PID mode (PID format 1) has an address of 143b00. Converted to decimal value, this corresponds to a
Domain of 20 and an area of 59. Note that this means that the device is physically located on slot 3, port 11. When the
Extended Edge PID format is set (PID format 2), the address will become 144b00. This allows the SilkWorm 24000 now to
have PIDs on the first slot consistent with the Native PID Format used on older 16 and 8-port switches. This is because devices
attached on any of the first 16-ports, will have the first nibble of the area set to 1, rather than 0. Thus with the Extended Edge
PID format (PID format 2) set, devices on physical slot 0 will have address of 141x00, where x corresponds to the physical
port number in hex rather than 140x00 with the default Core PID format setting of 1. As stated earlier, this effectively means
the area mappings have been shifted.

Sw itch Ports 0-15
Port ID (10- 1F ) PID Format 0 (Native - Fabric OS2.x and 3.x Only)
16-portswitch
Port ID (00- 0F ) PID Format 1 (Core)
Area Byte Range
Port ID (10- 1F ) PID Format 2 (Displaced)
Sw itch Ports 0-15 Sw itch Ports 16-31

PID Format 1
Port ID ( 00- 0F ) Port ID ( 10- 1F )
32-portswitch (Core)
Area Byte Range PID Format 2
Port ID ( 10- 1F ) Port ID ( 00- 0F ) (Displaced)
Port Card Ports 0-15 Port Card Ports 0-15

Core PID Format 1 Displaced PID Format 2
Slot1 Port ID ( 00- 0F ) Port ID ( 10- 1F )
64-portdirector Slot2 Port ID ( 10- 1F ) Port ID ( 20- 2F )

Area Byte Range Port ID ( 30- 3F )
Slot3 Port ID ( 20- 2F )
Slot4 Port ID ( 30- 3F ) Port ID ( 00- 0F )
Key:
Older PIDFormat Mapping
Displaced PID Format Mapping
Figure 6-7 PID Format Area Byte Mappings by Port Count
When the Extended Edge PID is set (PID format 2), and the switches are rebooted these settings are automatically updated. Be
aware that this new setting will impact Zoning, Secure Fabric OS DCC policy and other switch or fabric configurations. To
find out all the details of the Extended Edge PID format, and impact to Brocade Advanced Fabric Services and other switch
configuration parameters, please refer to the Brocade Fabric OS 4.2 Procedures Guide (part number: 53-0000518-03).
Guideline: Since the Extended Edge PID is a fabric wide setting, take great care to understand the impact to a
production fabric. As with any fabric wide change, before the Extended Edge PID format is set be sure
to go through the proper risk assessment and planning procedure to understand and minimize any the
negative impact to the fabric operation.

Guideline: Always backup the switch configurations before making the change. For dual fabrics, configure the
Extended Edge PID one fabric at a time and allow the fabric to reform and become stable before
continuing operations.
Table 6-5 shows the Default and supported PID settings for each version of Fabric OS.
Table 6-5
Fabric OS Version Default PID Setting Supported PID Values

2.6.2 0 (Native PID) 0, 1, 2
3.1.2 0 (Native PID) 0, 1, 2
4.2 1 (Core PID) 1, 2
6.3.8. Add Fabric OS Licenses

Follow the instructions given by the switch provider. This normally entails using the paper instructions and going to the
Brocade web site (brocade.com) and entering a registration number. A high level procedure for adding a trunking license is
illustrated in the following example.
1. Get the license ID of the switch. All licenses are based on the WWN. To find out the switch license ID for Fabric OS
versions 2.x and 3.x, use wwn. For Fabric OS 4.x use licenseidshow.
Example for Fabric OS 2.x and 3.x:
sialab89:admin> wwn
10:00:00:60:69:51:10:42
Example for Fabric OS 4.x:
poc166:admin> licenseidshow
10:00:00:60:69:80:0f:ac
2. Add a license with licenseadd. Use the key as generated by the instructions in the paper pack.
sialab89:admin> licenseadd "SeQedReQRSbfRfeB"
3. Check the licenses with licenseshow. Note that the Trunking License now exists.
sialab89:admin> licenseshow
SeQedReQRSbfRfeB:
Trunking license
SebSeyQy9cafcTft:
Web license
Zoning license
QuickLoop license
Fabric license
Remote Switch license
Remote Fabric license
Extended Fabric license
Fabric Watch license
Performance Monitor license
Note: After adding a Trunking License, use switchcfgtrunk to enable all ports for trunking. The switch does not need
to be disabled with switchdisable to do this. Most other licensed features are not activated automatically and
require an extra step or two. Please refer to the Brocade Fabric OS Procedures Guide for specific information about
licensed options.

6.3.9. (Optional) Name Devices with PortName

In Fabric OS 3.1 and 4.1 the command, portname, can be used to label a port. When defining the label use the port number
(or area number) as the argument, for all switches except the SilkWorm 12000 or 24000. When defining the label for a port on
a SilkWorm 12000 or 24000 the port number argument is entered in the physical port/slot convention.
Use portname to label port 26 on a SilkWorm 3900 as shown.
int219:admin> portname 26, "int124_HBA_B"
View the port name labels by using portname with no arguments.

int219:admin> portname
port 26: int124_HBA_B
To remove the label, use “ “ as shown.

sialab90:admin> portname 8, ""
Use portname again to display the port name. Note that the name no longer exists.
sialab90:admin> portname
sialab90:admin>
Port name can also be viewed using from the Name Server window within Web Tools, as shown in Figure 6-8.
Figure 6-8 Names Server Window Showing Port Name Values
Note: Port names do not change with the configdefault command. However, they can be cleared on port by port basis
with portcfgdefault. The portshow command displays port name in the first line of the output. Switchshow
does NOT display the portName. The port name label is persistent across switch reboots and power cycles.
Nsaliasshow displays the zoning alias, not the portname value.

6.3.10. (Optional) Setting up a Preferred Principal Switch in

the Fabric (Fabric OS 4.1 or Higher)
The principal switch is responsible for handing out domain IDs to the rest of the fabric upon a fabric build. In some cases it
may be desirable to hard set a switch to always be the principal switch. In Fabric OS 4.1 or greater only, a preferred principal
switch can be selected. Brocade has done extensive testing with this feature. The results have shown that with large fabrics
and/or Secure Fabric OS enabled the selection is not completely deterministic. For the vast majority of fabrics, there is no
issue.
The recommended way to set the principal switch is shown in the next example. This sets the current switch as the preferred
principal switch and forces a fabric build to enable it. Fabric builds are not disruptive.
poc165:admin> fabricprincipal -f 1
fabric: Reconfiguration due to Principal Selection Mode
fabric: Reconfiguring at Mon Mar 17 15:54:59 2003
Principal Selection Mode enabled (Forcing fabric rebuild)
poc165:admin>
5 4 3 2 1
10 9 8 7 6 5 4 3 2 1
fabric: Principal switch

fabric: Domain 165
To enable preferred principal switch selection without doing a rebuild do the following:
poc165:admin> fabricprincipal 1
Principal Selection Mode enabled (Activate in next fabric rebuild)
To turn off the persistent selection mode on the switch, use 0 as the argument as shown:
poc165:admin> fabricprincipal 0
Principal Selection Mode disabled
Use fabric principal with no argument to check the current mode. This example shows the current state as enabled.
poc165:admin> fabricprincipal
Principal Selection Mode: Enable
poc165:admin>
Note: When another switch that is configured to be the preferred principal switch the lower WWN switch will win and the
Higher WWN switch will log an error saying “PSS principal failed (Lost to WWN: <wwn>)”. The preferred principal
switch remains selected across reboots, power cycles, and upon a fabric rebuilds.

6.3.11. Set Telnet Session Timeout Value

It is highly recommended to set the admin telnet session timeout values to 10 minutes. This prevents a telnet session from
locking up access to a switch. Table 6-6 shows the number of allowed telnet sessions per switch model for the admin user. For
the SilkWorm 24000, 3850 and 3250 Fabric OS 4.2 or greater is required.
Table 6-6 Number of Telnet Sessions for Admin User
SilkWorm Switch Model Number of Telnet Sessions

SilkWorm 2xxx series 1
SilkWorm 3200 (Fabric OS 3.x) 1

SilkWorm 3800 (Fabric OS 3.x)
SilkWorm 3900 (Fabric OS 4.x) 2

SilkWorm 3850 (Fabric OS 4.2)
SilkWorm 3250 (Fabric OS 4.2
SilkWorm 12000 (Fabric OS 4.x) 4 (2 per logical switch)
SilkWorm 24000 (Fabric OS 4.2) 2
This example shows how to set the timeout value on Fabric OS 2.6.1 and 3.1.
sialab89:admin> timeout
TimeOut is Disabled
sialab89:admin> timeout 10
TimeOut is now 10 minutes
sialab89:admin> timeout
TimeOut is 10 minutes
sialab89:admin> timeout 0
TimeOut is now Disabled
Figure 6-9 Timeout Command Output in Fabric OS 2.6.1 and 3.1
Note, when issuing the command in Fabric OS 4.x you will get the following messages:
IDLE Timeout Changed to 10 minutes
The modified IDLE Timeout will be in effect after NEXT login

6.3.12. (Optional) Disabling the Telnet Daemon When

Secure Mode is Enabled (Fabric OS 4.1 or greater only)
To maintain a high level of security, Brocade switches running Fabric OS 4.1 or greater have the option to disable the telnet
daemon using the configure command. Unlike most configuration changes with configure, this can be done while the
switch is still online. It is highly recommended to login via a console port connection when performing the configuration. The
example below illustrates the procedure.
Note: Switches running Fabric OS 2.x or 3.x do not have the capability to disable the telnet daemon. However, policies can
be configured with SFOS to block telnet.
Note: In most cases the switch must be disabled with switchdisable prior to using the configure command, however
when disabling the telnet daemon the switch remains online while the configure command is invoked.
int219 login: admin

Password:
Please change your passwords now.
Use Control-C to exit or press 'Enter' key to proceed.
Password was not changed. Will prompt again at next login

until password is changed.
Not all options will be available on an enabled switch.

To disable the switch, use the "switchDisable" command.
Configure...
System services (yes, y, no, n): [no] y

rstatd (on, off): [off]
rusersd (on, off): [off]
telnetd (on, off): [on] off
Broadcast message from root (pts/0) Thu Apr 17 17:39:11 2003...
Security policy change: TTY pts on switch instance 0 will be logged out.
Connection closed...
6.3.12.1. Enabling a Disabled Telnet Daemon

To enable a disabled telnet daemon connect to the switch serial port and follow the procedure below.
Not all options will be available on an enabled switch.

To disable the switch, use the "switchDisable" command.
Configure...
System services (yes, y, no, n): [no] y

rstatd (on, off): [off]
rusersd (on, off): [off]
telnetd (on, off): [off] on
int219:admin> 0x2b2 (fabos): Switch: 0, Warning FW-STATUS_SWITCH, 3,
Switch status changed from Marginal/Warning to HEALTHY/OK
int219:admin>

6.3.13. (Optional) Setup Ports for Extended Fabrics

Configuring a port for Extended Fabrics can be performed using the portCfgLongDistance CLI (Command Line Interface)
utility. Specify the port and Extended Fabrics level as arguments to the portCfgLongDistance command. If an Extended
Fabrics port is to be configured on a SilkWorm 2000 series switch, the fabric-wide long distance parameter
fabric.ops.mode.longDistance must be set in the configure CLI menu. Configuration of this parameter requires the switch to
be disabled.
Guideline: Extended Fabrics can also be configured via Brocade Fabric Manager and Web Tools, however the
procedures are not given in this document. Please refer to the appropriate User's Guide.
Revisions of Fabric OS v3.0.2 and greater contain an additional optional parameter, VC Translation Link Initialization, to the
portCfgLongDistance CLI command. When set to 1, this parameter indicates that enhanced link reset protocol should be used
on the port. The default value for this parameter is 0 and is compatible with earlier Fabric OS v3.x implementations. For
optimal performance, specify 1 when E_Port links are between switches with Fabric OS v3.0.2 and greater, or Fabric OS
v4.0.2 and greater. Specify 0, or nothing, when connecting a switch with Fabric OS v3.0.2 or above switch to previous releases
of Fabric OS.
To configure Extended Fabrics on SilkWorm 2000 series switches, the following procedures must be performed.
1. Set the fabric-wide configuration parameter fabric.ops.mode.longDistance to 1 using the configure command. This
parameter must be set on all switches within the fabric.
2. Set the appropriate Extended Fabrics mode for each long-distance port using the portCfgLongDistance command.
3. When configuring Extended Fabrics on SilkWorm 3000 series switches and above, only port level configuration is
necessary.
4. Set the appropriate Extended Fabrics mode for each long-distance port using the CLI command portCfgLongDistance. At
the same time, set the VC translation link initialization bit to 1 on long-distance switches running Fabric OS v3.0.2 or
above.
Brocade supports Extended Fabrics ports between same series switches (i.e. SilkWorm 2000 series to SilkWorm 2000 series
switches) as well as SilkWorm 3000 series switches to SilkWorm 12000 switches. Directly connecting a SilkWorm 2000 series
switch to a SilkWorm 3000/12000 series switch is unsupported when using Extended Fabrics. For a mixed SilkWorm 2000 and
SilkWorm 3000/12000 fabric, where the long-distance ports are between SilkWorm 2000 series switches, the fabric-wide
parameter fabric.ops.mode.longDistance must be set to a value of 1 on all switches within the fabric. For mixed fabric
configurations where long-distance ports are located between SilkWorm 3000 and/or SilkWorm 12000 series switches, the
fabric-wide long distance parameter is not required.
The output in Figure 6-10 shows portCfgLongDistance usage and an example of how to configure port 0 on a SilkWorm 3800
switch running Fabric OS v3.1 for L2 Extended Fabrics mode with VC translation link initialization set to 1.
mw123:root> portcfglongdistanceUsage: portCfgLongDistance port ,"distance_level",<vc translation link
init>distance_level: L0 - normal LE <= 10km L0.5
<= 25km L1 <= 50km L2 <= 100km LD
- autovc trans link init: 0 normal1 vc translation
mw123:root> portcfglongdistance 0,"L2",1 Committing configuration...done.
Figure 6-10

The portCfgShow command can be used to determine which ports are configured as long-distance ports. The switch output in
Figure 6-11 shows that port 0 has been configured for L2 Extended Fabrics and VC Translation Link Initialization has been
turned on.
mw123:root> portcfgshowPorts
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
15-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Long Distance L2 .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC link init ON .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Figure 6-11
For additional information refer to Appendix B, Long-distance Technologies for Storage Area Networks.
6.3.14. (Optional) Setup Fabric Watch, SNMP Traps,

switchstatuspolicyset
Although optional, it is highly recommended to define the environmental and SNMP management items at this point in time.
For guidelines on setting these refer to Section III SAN Management.
6.3.15. Baseline and Backup the Switch With Configupload

Once all the settings are complete, use configupload to backup the switch configuration. This uploads all of the
configshow parameters that define the switch configuration. This is highly recommended in the event the original settings
need to be restored. If minor changes to the configuration is required, just edit the configuration file and re-download the
configuration to the switch. One further benefit is that a “standard” switch configuration can be defined and uploaded to all of
the remaining switches in the fabric. This is known as baselining. Refer to section 6.5.3.10. Backup the Configuration with
configupload on page 6-47.
Guideline: Define a “golden” switch. For larger SAN fabrics, or for the staging of many smaller fabrics, use
configupload to baseline the “golden” switch. The “golden” switch configuration can be downloaded
with Fabric Manager to all other switches in the fabric. Do baselines by Fabric OS version. In other
words, if the fabric contains switches running Fabric OS 3.1.2 and 4.2, create two “golden” switch
configurations. This will help with change management.

6.4. SAN Fabric Configuration

Once each switch has been prepped, the SAN fabric can be built and configured. A checklist provides some essential
high-level guidelines and task order. Most of what needs to be done is the zoning of devices. Once these steps are complete,
the staging is essentially complete.
Table 6-7 SAN Fabric Configuration Checklist
SAN Fabric Configuration Checklist

1. Gather Planning Documentation (ISL Map, Device Spreadsheet, Logical Design Diagram)
2. Cable ISLs. Check the fabric using the command: islshow, trunkshow
3. Cable Host and Storage Devices. Check using the commands: switchshow and nsshow
4. Use documentation to label all cables.
5. Optional: Create a Dummy Zone to prevent device access.
6. Setup NTP Time Server on the Fabric. Use the Principal or Primary FCS Switch only.
7. Validate the fabric using the command: PortTest
8. Pre-Zoning Check. Verify hosts and storage are in the fabric that are to be zoned. On a core
switch use the command: nscamshow
9. Implement Zoning. It is good practice to backup the zoning configuration once it is enabled.
10. Profile each SAN fabric.
6.4.1. Gather Planning Documentation

Gather the ISL Port Map, Device Spreadsheet and Logical Design Diagram that was put together during the planning phase.
These items will be used as the SAN map that shows how to build the fabric and where to attach all the devices. Once the
required information is readily available cabling will be simplified.
6.4.2. Cable ISLs

Cable the ISLs as documented by the ISL port map. section 5.4. Cable Planning on page 5-5 provides guidelines on cabling.
The commands islshow and trunkshow (output not shown) can be used to verify the E_port and trunking link states.
6.4.3. Cable Hosts and Storage Devices

Cable the Hosts and Storage devices as documented by the Device Spreadsheet. section 5.4. Cable Planning on page 5-5
provides guidelines on cabling. Once cabling is complete use fabricshow to verify the fabric is built. Use switchshow
and nsshow to verify devices are online and registered with the Name Server. Some example outputs from the case study
SAN are shown in the following illustrations. Note that fabricshow shows the six switches as expected.
sialab89:admin> fabricshow
Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
-------------------------------------------------------------------------
89: fffc59 10:00:00:60:69:51:10:42 192.168.155.89 0.0.0.0 "sialab89"
90: fffc5a 10:00:00:60:69:51:0f:2b 192.168.155.90 0.0.0.0 >"sialab90"
166: fffca6 10:00:00:60:69:80:0f:ad 192.168.173.166 0.0.0.0 "poc166"
195: fffcc3 10:00:00:60:69:10:93:14 192.168.162.195 0.0.0.0 "int195"
196: fffcc4 10:00:00:60:69:10:62:2c 192.168.162.196 0.0.0.0 "int196"
219: fffcdb 10:00:00:60:69:90:04:1a 192.168.162.219 0.0.0.0 "int219"
Note: > indicates Principal switch.
Figure 6-12 Fabricshow Output for Fabric OS 3.1

sialab89:admin> switchshow
switchName: sialab89
switchType: 9.2
switchState: Online
switchMode: Native
switchRole: Subordinate
switchDomain: 89
switchId: fffc59
switchWwn: 10:00:00:60:69:51:10:42
switchBeacon: OFF
Zoning: ON (bkup_cfg_B)
port 0: id N2 Online E-Port 10:00:00:60:69:80:0f:ad "poc166" (upstream)
(Trunk master)
port 1: id N2 Online E-Port (Trunk port, master is port #0)
port 2: -- N2 No_Module
port 4: id N2 No_Light
port 15: id N2 Online L-Port 15 public
sialab89:admin>
Figure 6-13 Switchshow Output for Fabric OS 3.1

The nsshow –r option shows only the devices that have done a State Change Registration (SCR). These devices accept
Registered State Change Notifications (RSCNs) which allow for device discovery when changes occur in the fabric.
int219:admin> nsshow -r
{
Type Pid COS PortName NodeName TTL(sec)
N db0e00; 3;50:00:60:e8:02:ee:78:18;50:00:60:e8:02:ee:78:18; na
FC4s: FCP [HITACHI OPEN-E -SUN0118]
Fabric Port Name: 20:0e:00:60:69:90:04:1a
N db0f00; 3;50:00:60:e8:02:ee:78:19;50:00:60:e8:02:ee:78:19; na
FC4s: FCP [HITACHI OPEN-E 0118]
Fabric Port Name: 20:0f:00:60:69:90:04:1a
N db1a00; 2,3;10:00:00:00:c9:30:d0:62;20:00:00:00:c9:30:d0:62; na
FC4s: FCP
Fabric Port Name: 20:1a:00:60:69:90:04:1a
N db1b00; 2,3;10:00:00:00:c9:30:d0:9d;20:00:00:00:c9:30:d0:9d; na
FC4s: FCP
NodeSymb: [35] "Emulex LP9002 FV3.90A7 DV5-4.82A4 "
Fabric Port Name: 20:1b:00:60:69:90:04:1a
NsShow Output with -r Option Displaying Devices Registered for State Change Notification
To show all the devices in the fabric, use nsallshow. An example output is shown below.
sialab89:admin> nsallshow
{
590fd1 590fd2 590fd3 590fd4 590fd5 590fd6 590fd9 590fda
590fdc 590fe0 590fe1 590fe2 590fe4 590fe8 590fef 5a0800
c30600 c30800 c40d00 db0e00 db0f00 db1a00 db1b00
23 Nx_Ports in the Fabric }
Figure 6-14 NsallShow Output Showing All Devices Participating in the Fabric

6.4.4. Use Documentation to Label All Cables

Having labels on every ISL and device connection is one of the most important tasks. When troubleshooting or servicing a
device, this allows for greatly reduced downtime. Make sure all cables are labeled and that each label is easily read.
6.4.5. (Optional) Dummy Zone Configuration to Prevent

Device Access
The default zoning configuration allows any device to see any other device. If the devices are plugged in and the desire is to
have them locked out, define a dummy-zoning configuration. To do this, create a zone configuration with an unused port. The
example in Figure 6-15 shows the four steps that creates a dummy zone for switch domain 101 and port 5. With this complete,
no devices are allowed to access any other device.
int219:admin> zonecreate "dummyzone","101,5"
int219:admin> cfgcreate "dummycfg","dummyzone"
int219:admin> cfgenable "dummycfg"
int219:admin> cfgsave
Figure 6-15 Creating a Dummy Zone
6.4.6. Setup NTP Time Server on the Fabric

An extension of the Time Service capability is available in Fabric OS 2.6.1, 3.1 and 4.1 and later with tsclockserver. The
purpose of this command is to provide synchronization of the fabric time with an external NTP server. The time server only
needs to be configured on one switch in the fabric. Configure the time server on either the Principal switch or while in Secure
Fabric OS mode, the Primary FCS (trusted) switch. Once setup, the time server configuration will be propagated in band to
each other switch in the fabric.
The default value for tsclockserver is LOCL, as shown in Figure 6-16. This means no time server has been setup for the
fabric and each switch updates its own local time value. If no argument is used, the current configuration is shown. The CLI
command has the following use: tsClockServer [<NTP Server address>]. Any switch in the fabric can be used
to set the parameter, as it is a fabric wide setting. An example of how to use tsclockserver is shown in Figure 6-17. Refer
to section 3.5.1. Management and Control on page 3-24 for additional information about synchronizing time within a fabric.
Guideline: If desired, use the primary management switch in the fabric, generally this would be the principal or
Primary FCS Switch for NTP service setup.
int219:admin> tsclockserver
LOCL
Figure 6-16 Default Setting
sialab89:admin> tsclockserver "192.168.126.60"
Updating Clock Server configuration...
Figure 6-17 Example of Setting Tsclockserver

6.4.7. Run Fabric Diagnostics with PortTest

It is good practice to run diagnostics after building the fabric. The command porttest as of, Fabric OS 3.1 and 4.1, allows
diagnostics to be run on a local switch that has both online devices and E_ports. The test only runs on connected ports, whether
it is a device or ISL. This test will determine if a faulty SFP is on an online port. There are many options to this command, all
of which are described in the online help page.
Note: The parameters of the command porttest are slightly different in Fabric OS 3.x and 4.x.
Note: Some HBAs require a specific payload size since the Echo response is limited is payload size. Please refer to the HBA
manufacturer’s documentation for details.
For Fabric OS 3.1 the following porttest runs for 1000 iterations on all ports with -1 option. The test lasts about seven
minutes. Porttestshow displays the current status. Check for PASS on the first line after the port number. This indicates a
successful test run. To avoid unnecessary output, run the test only on ports with connected devices.
The example in Figure 6-18 illustrates the use of this command in Fabric OS 3.1. The example in Figure 6-19 illustrates the
use of this command in Fabric OS 4.1.
sialab90:admin> portTest -1,1000
sialab90:admin> porttestshow
Port 0 : PASS
PortType: E PORT(SLAVE) PortState: TESTING
PortInternalState: TX PortTypeToTest: ALL_PORTS
Pattern: 0xb Seed: 0xaa UserDelay: 10
TotalIteration: 1000 CurrentIteration: 575
TotalFail: 0 ConsecutiveFail: 0
StartTime: Mar 22 03:58:06
StopTime: NONE
Timeout: 0 ErrorCode: 0
Port 8 : PASS
PortType: F PORT PortState: TEST DONE
PortInternalState: INIT PortTypeToTest: NO_TEST
Pattern: 0xb Seed: 0xaa UserDelay: 10
TotalIteration: 1000 CurrentIteration: 1000
TotalFail: 0 ConsecutiveFail: 0
StartTime: Mar 22 03:58:06
StopTime: Mar 22 04:03:58
Timeout: 0 ErrorCode: 0
Figure 6-18 PortTestShow on Fabric OS 3.1
When running the command porttest in Fabric OS 4.1, the arguments are more intuitive. The example of porttest
below runs for 1000 iterations on all ports with -1 option. As in the previous example, porttest lasts about 7 minutes with
these arguments. To avoid unnecessary output run the test only on ports with the connected devices. If there are a large number
of devices, use the terminal emulator software to save the output to a log file.
int218:admin> portTest -iteration 1000
Figure 6-19 PortTest on Fabric OS 4.1
Note: If porttest is configured to run indefinitely, use the stopporttest command to stop it.

6.4.8. Pre-Zoning Check

Before configuring Zoning run the command nscamshow to perform a sanity check on device connectivity. Capture the
output to a file and compare to the Device Spreadsheet. All devices should be present.
Nscamshow displays all the devices for each non-local switch domain that have been registered with the name server on the
other switches in the fabric. For a Core/Edge topology, such as the case study, execute this command from a Core switch as
Edge devices are not registered with the local name servers. If devices are attached locally, use nsshow in addition, to capture
devices registered in the local name server database. The partial output from a core switch in the case study is shown. Note that
private loop HBAs will not be captured.
Guideline: For a Core/Edge topology, such as the case study, use a core switch to execute nscamshow, as there
are no edge devices registered with the local name servers. If devices are attached locally, use nsshow
in addition, to capture devices registered in the local name server database.
int63:admin> nscamshow
nscam show for remote switches:
Switch entry for 75
state rev owner
known v260 0xfffc3f
Device list: count 1
Type Pid COS PortName NodeName
N 4b0f00; 3;20:05:00:a0:b8:07:5d:c7;20:04:00:a0:b8:07:5d:c6;
FC4s: FCP
Fabric Port Name: 20:0f:00:60:69:10:10:9d
Switch entry for 88

state rev owner
known v310 0xfffc3f
N 580800; 2,3;10:00:00:00:c9:29:04:8f;20:00:00:00:c9:29:04:8f;
FC4s: FCP
Fabric Port Name: 20:08:00:60:69:51:0e:0a
Switch entry for 217

state rev owner
known v410 0xfffc3f
N d90e00; 3;50:00:60:e8:02:ee:78:08;50:00:60:e8:02:ee:78:08;
FC4s: FCP
Fabric Port Name: 20:0e:00:60:69:90:03:fa
N d90f00; 3;50:00:60:e8:02:ee:78:09;50:00:60:e8:02:ee:78:09;
FC4s: FCP
Fabric Port Name: 20:0f:00:60:69:90:03:fa
N d91a00; 2,3;10:00:00:00:c9:30:d0:66;20:00:00:00:c9:30:d0:66;
FC4s: FCP
Fabric Port Name: 20:1a:00:60:69:90:03:fa
N d91b00; 2,3;10:00:00:00:c9:30:d0:c2;20:00:00:00:c9:30:d0:c2;
FC4s: FCP
Fabric Port Name: 20:1b:00:60:69:90:03:fa
Figure 6-20 Nscamshow Output

6.4.9. Implement Zoning

The SAN fabric case study will be used to illustrate the command line. Zoning can be configured using the command line
interface (CLI), Brocade Fabric Manager, or Web Tools. Only the CLI method will be discussed in this section. For more
information about how to use Brocade Fabric Manager or Web Tools to configure Zoning, refer to the Brocade Fabric OS
Features Guide (part number: 53-0000395-01). Please note that although it is possible to have many zoning configurations
defined on a fabric, only one configuration can be active.
Guideline: It is recommended to backup the zoning configuration once Zoning has been configured and enabled.
Note: The steps shown in the procedure below will not re-create the configuration.
1. To configure a hardware zone on a SilkWorm 2000 series switch, first use alicreate with domain ID, port ID as the
second argument. In the example below it is “218,8”. As discussed in the Zoning section earlier, the WWN can be used for
hardware zoning on all switches that run Fabric OS 3.x or 4.x. The example will use the WWN for the following steps.
int195:admin> alicreate "E250_OrA","218,8"
int219:admin> alicreate "int122_HBA_B","10:00:00:00:c9:24:f5:f9"
int219:admin> alicreate "stor_port_B","20:04:00:a0:b8:07:5d:c7"
2. Use the command zonecreate to define a zone using the aliases. The zoneshow command is used to check the newly
created aliases are correct.
int219:admin> zonecreate "int122_B_zone","int122_HBA_B; stor_port_B"
int219:admin> zoneshow
Defined configuration:
alias: int122_HBA_B 10:00:00:00:c9:24:f5:f9
alias: stor_port_B 20:04:00:a0:b8:07:5d:c7
3. Use the command cfgcreate to define a configuration with one or more zones. This example has only one, defined as
“E250_A_ZONE”. Enable the new zone configuration with the command cfgenable.
int219:admin> cfgcreate "bkup_cfg_B","int122_B_zone"
int219:admin> cfgenable "bkup_cfg_B"
zone config "bkup_cfg_B" is in effect
4. Use the command cfgsave to write the configuration to flash memory.

int219:admin> cfgsave
Updating flash ...
5. Verify the new configuration with the command zoneshow.

int219:admin> zoneshow
Defined configuration:
cfg: bkup_cfg_B int122_B_zone
zone: int122_B_zone int122_B_zone; stor_port_B
alias: int122_HBA_B 10:00:00:00:c9:24:f5:f9
alias: stor_port_B 20:04:00:a0:b8:07:5d:c7
Effective configuration:
cfg: OracleBackup_cfg
zone: int122_B_zone
10:00:00:00:c9:24:f5:f9
20:04:00:a0:b8:07:5d:c7
6. Use the command cfgActvshow to show the current active configuration. Note: This command is not available in
Fabric OS 2.6.1.
sialab89:admin> cfgActvshow
cfg: bkup_cfg_B
zone: int121_B
10:00:00:00:c9:24:f5:f9
20:04:00:a0:b8:07:5d:c7

7. To display the amount of memory the zone database has and the amount used, use the command cfgsize. In Fabric OS
2.6.1 and 3.x the maximum allowed size is 96 KB. For Fabric OS 4.x the maximum size is 128 KB. The output from each
is shown below.
Example: Fabric OS 2.6.x/3.x
sialab89:admin> cfgsize
Zone DB max size - 98232 bytes
committed - 274
transaction – 0
Example: Fabric OS 4.x

int219:admin> cfgsize
Zone DB max size - 130956 bytes
committed - 274
transaction - 0
value = 0
8. Backup the zoning configuration using the command configupload.

For more details on the recommended zoning practices please reference the whitepaper titled Zoning Implementation
Strategies for Brocade SAN Fabrics (part number: GA-WP-522-00).
Warning: For mixed fabrics that contain switches with Fabric OS 2.x and 3.x, the maximum zoning database size will be
about 96 KB. Be aware that, the vast majority of fabrics will not even be close to this limit. The largest ones
deployed today on the order of a thousand ports, use about 40 KB for the zoning database.
Below are some additional guidelines when implementing zoning for larger sized fabrics and dual fabric SANs.
Guideline: To simplify the zoning configuration alias creation on dual fabric SANs, do the following on a
non-production fabric. It’s best to use clean switches, with no zones enabled or defined.
1. Configupload to a host.
2. Edit the zoning configuration appropriately.
3. ConfigDownload the zoning configuration from the host to a target switch in each fabric.
Guideline: For a large number of zone configurations, generally over 15, use Fabric Manager or Web Tools. These
tools vastly simplify the zoning implementation. WWNs of devices and the ports on the switch they are
attached to are also seen automatically in the GUI, so validation of connectivity can be done
simultaneously.

6.4.10. Profile The SAN Fabrics

Once the SAN fabrics are built and zoning has been configured, it is good practice to capture a profile of the fabric. The
“show” commands are ideal for this. The commands in Table 6-8 were used to capture the profile of the SilkWorm
12000/24000 used in the case study SAN. The list in Table 6-9 can be used for SilkWorm 3200/3800 switches. The list in
Table 6-10 can be used for SilkWorm 3250/3850 switches which run Fabric OS 4.2 or greater. It also applies to the 3900.
These commands can be easily scripted using the Brocade API Scripting Toolkit. As changes take place in the fabric, consider
periodically updating the profile information. This will proactively simplify change management and provide a living
documentation set that can be referenced when technical support is required.
Table 6-8
SilkWorm 12000/24000 Profiling Commands

firmwareshow sensorshow
hashow switchshow
licensidshow nsshow
licenseshow nsallshow
ipaddrshow 4 islshow
portcfgshow trunkshow
slotshow fabricshow
chassisshow zoneshow
Table 6-9
SilkWorm 3200/3800 Profiling Commands

version configshow
licensidshow switchshow
licenseshow nsshow
ipaddrshow nsallshow
portcfgshow islshow
sensorshow trunkshow
fabricshow
zoneshow
Table 6-10
SilkWorm 3250/3850/3900 Profiling Commands

version configshow
firmwareshow switchshow
licensidshow nsshow
licenseshow nsallshow
ipaddrshow islshow
portcfgshow trunkshow
sensorshow fabricshow
zoneshow

6.5. Staging SAN Fabrics with Secure Fabric OS

(SFOS)
Prior to staging Secure Fabric OS, it is highly recommended to follow the planning guidelines in section 5.9.2. Planning for
Secure Fabric OS Security Measures on page 5-19. It is recommended to use the checklists provided in each section as a
framework for developing a customized Secure Fabric OS solution. The dual fabric SAN case study will be referenced
throughout the discussion in order to clearly illustrate each checklist item.
6.5.1. Staging Secure Fabric OS with Fabric Manager

Fabric Manager provides a simplified means to accomplish important Secure Fabric OS configuration tasks on high port count
SAN fabrics. In order to use Fabric Manager, some preparation must take place. This section provides the required information
to enable the use of Fabric Manager.
Before Fabric Manager can be used to manage Secure Fabric OS, the command secmodeenable must be executed at a
sectelnet/SSH session. After Secure Fabric OS has been enabled, the following security policies must be defined before Fabric
Manager can be used to access secure switches:
• The IP address of any host must be added that runs the Fabric Manager server to the IP policy of your fabric. The
server cannot communicate with the fabric if you do not include this IP address. This is true regardless of whether or
not the machine also runs the Fabric Manager 4.1 client.
• Every client that is desired to run the API or the API policy of your fabric MUST BE added using the command line.
Once Secure Fabric OS is enabled and the passwords are reset, an existing Fabric Manager installation must have the password
authentication database updated. From the main Fabric Manager administration window click on Actions -> Security.
Refer the online help or the Brocade Fabric Manager 4.1 Users Guide (part number: 53-0000823-03) for details on Fabric
Manager.
Guideline: Update the Fabric Manager password authentication after Secure Fabric OS is enabled.
Warning: A telnet session cannot be used simultaneously with Fabric Manager when managing Secure Fabric OS. This is
indicated by a “not transaction owner” message when attempting to make changes at the command line. The
Fabric Manager Security Admin window must be closed to manage Secure Fabric OS at the command line.
6.5.2. Secure Fabric OS and Extended Fabrics

Connectivity between switches over an extended distance does not affect Secure Fabric OS (SFOS) policy configuration tasks.
Settings may be required on the MAN/WAN equipment ports that provide the Fibre Channel link connections for FCAP
protocol authentication. This is especially true for equipment that encapsulates Fibre Channel into ATM, IP or another
transport over long distance. Consult the vendor's documentation or professional services organizations for recommendations
and guidelines on setting up the hardware for this purpose.
Guideline: When a fabric spans multiple sites be sure to assign at least one FCS switch per site.

6.5.3. Staging Secure Fabric OS

The checklist in Table 6-11 is a suggested process which use the following four SAN security design choices based on a
defined Secure FOS plan:
• Define three FCS switches with the highest available switch (preferably the SilkWorm 24000) as the Primary FCS.
• Configure the access for a single Management Station.
• Lock down the fabric to prevent any access by any unauthorized switch.
• Lock down all host and storage devices to prevent any access by any unauthorized device.
These design choices also provide the initial requirements for the initial Secure FOS policy definition. Additional policies may
be created using the procedure in section 6.5.3.9. Creating Policies and Validating Functionality on page 6-40.
Note: Please work with the IT staff responsible for corporate wide IT security policies definition and enforcement for specific
requirements as part of the planning effort before staging Secure Fabric OS.
Warning: Every switch in the entire fabric needs to be rebooted in order to activate Secure Fabric OS. Be aware that enabling
Secure Fabric OS on a SilkWorm 12000 will reboot BOTH CPs simultaneously. It is recommended to schedule
downtime on single fabric SANs before enabling Secure Fabric OS. Downtime to the SAN can be avoided if using
a redundant fabric SAN.
Guideline: Any time a change is required, the Secure Fabric OS policy set MUST be updated BEFORE the new
device is allowed to inhabit the fabric. This prevents accidental or malicious attempts at accessing fabric
wide information and more importantly, provides complete control over the SAN environment. It is for
this reason that defining and implementing policies for each attached switch or node is highly
recommended.
Upon the Secure Fabric OS being enabled, there are benefits available without any policies in place. These benefits are:
• With Secure Fabric OS enabled in the SAN, each fabric can only be administered through a secure session with a
single management point; the Primary FCS switch. A management control mechanism is now in place.
• New passwords will be defined for the user, root and admin administrative accounts. These will be centrally managed
by the Primary FCS switch.
• A secure management station with Brocade SecTelnet software utility or an SSH client must be defined to access the
Primary FCS switch. This means that passwords will no longer be sent out-of-band in clear text.
Secure FOS policy definition is both flexible and comprehensive. Adding switches, devices and/or out of band management
stations to the existing SAN architecture is no longer an ad-hoc task. With Secure FOS policies in place, the following benefits
are realized:
• Extreme control over access to SAN components and a framework for change management enforcement. With
Secure Fabric OS policies in place, a SAN fabric becomes a fully controlled environment.
• Any physical or logical changes that occur within a fabric must be first authorized before the actual implementation
takes place. Secure Fabric OS thus provides complete control over a continually changing fabric environment.
• Integration into the existing IT security framework. Already in place on the IP network infrastructure and hosts,
policy based security is now available in the SAN.
The following page provides a checklist of items to install, configure and add policies using Secure FOS so that the above
benefits are put into practice. The items in the checklist are then expounded upon to show detailed examples and provide
further guidelines on usage during the configuration phase. This list should be used as a starting point for a custom installation.

Table 6-11 Secure Fabric OS Staging Checklist
Secure Fabric OS Staging Checklist

1. Follow the Planning guidelines in sections section 5.10. Secure Fabric OS Planning on page 5-21, section 5.11.
Secure Fabric OS Pre-Installation Planning on page 5-23, and section 5.13. Planning the Secure Fabric OS
Implementation on page 5-26.
2. From the SAN Management Station, verify Digital Certificates exist on the switches. To simplify the process
Fabric Manager can be used. Be sure to follow the preparation steps in section 6.5.3.2. Verify Digital Certificates
Exist on page 6-31.
3. Gather SAN Configuration documentation as illustrated in section 5.7. Documentation Guidelines on page 5-12.
4. Develop a policy plan:.
• Select an external NTP Time Server
• Select and document switch Roles.
• Select and document fabric devices for policy enforcement.
• Select a Secure Fabric OS management station.
5. Install Brocade Secure Telnet and other utilities on Management Station.
6. Configure the time server for the fabric with the command: tsclockserver
7. Enable Secure Fabric OS with secmodeenable from a SecTelnet session.
8. If Fabric Manager is being used, update the fabric password authentication.
9. Create the policies and validate functionality.
10. Backup the Configuration with configupload.
11. Optional: Document the Secure Fabric OS Configuration.
Warning: Be aware that enabling Secure Fabric OS on a SilkWorm 12000 will reboot BOTH CPs simultaneously. Although
not recommended, some fabrics may use each logical switch in a single chassis in separate fabrics in a dual SAN
fabric configuration. In this case, BOTH fabrics will be OFFLINE while the fabric reboots and the Primary FCS
switch establishes itself. For SANs that will have Secure Fabric OS enabled, DO NOT use both logical switches
in separate fabrics as this is an unsupported configuration.
6.5.3.1. Follow the Planning Guidelines

Be sure the fabric is ready for enabling Secure Fabric OS. Proper planning is critical to a successful deployment. It is
highly recommended to follow the Secure Fabric OS planning guidelines in sections section 5.10. Secure Fabric OS
Planning on page 5-21 and section 5.13. Planning the Secure Fabric OS Implementation on page 5-26.
6.5.3.2. Verify Digital Certificates Exist

Verify that digital certificates exist before enabling Secure Fabric OS. When doing this from the command line, you will
be required to log into each switch from the fabric. For Fabric OS 4.1 and later, based switches, use the command
pkishow. On switches with Fabric OS 2.6.x or 3.1 or later, use the command configshow “pki”. Note that when
the digital certificate is properly installed, the Certificate object is flagged with exist.
int217:admin> pkishow
Passphrase : Exist
Private Key : Exist
CSR : Exist
Certificate : Exist
Root Certificate: Exist

Brocade Fabric Manager can simplify the verification of digital certificates as you will not be required to log into each
switch. To verify the installation of the digital certificates, launch the Fabric Manager client and use the following
procedure:
1. Discover the SAN fabrics by entering the IP address of one switch in each fabric in the Address Bar, one at a time.
2. Once each fabric is discovered in the SAN, click on the SAN Elements tab. To select all fabrics in the SAN, click on
Fabrics under the My SAN tree location. To select a single fabric, as shown in the example, click on the Fabric
Name, in this case it is DDM-A. Once selected, DDM-A is highlighted.
3. To check if the digital certificates exist click on the Switches button on the detail pane as shown. Use the scroll bar at
the bottom to locate the Has Certificate column. Note that in Figure 6-21 all switches in DDM-A have this value set
to true. If digital certificates did not exist, the value would be set to false.
As the output illustrates, all switches have the required digital certificates. Thus the fabric is ready for Secure Fabric OS to
be enabled. If digital certificates or other PKI objects do not exist, the Brocade PKIcert utility is required to prepare them.
This procedure is generally required for legacy Brocade SilkWorm switches. All new switches have certificates installed
at the factory. For details on installing and using PKIcert, please contact the switch provider or visit the Brocade web site
and select Partner Network.
Warning: Digital Certificates must exist on all switches in the fabric before enabling Secure Fabric OS.
Figure 6-21 Fabric Manager
6.5.3.3. Gather SAN Configuration Documentation

Before defining the policies, gather the SAN Configuration Documentation created as part of the planning process. This
will provide a starting point when thinking about what policies should be created and the devices and switches that need to
be added to them.

6.5.3.4. Develop a Policy Plan
Before Brocade Secure Fabric OS (SFOS) can be initialized, an external fabric wide time server must be selected as part
of the plan. Record the IP address. Once SFOS is setup, the Primary FCS switch will be responsible to update the time on
all other member switches.
Use the guidelines provided in Chapter 2, SAN Design & Architecture Concepts to determine what the switch roles will
be. Using the output of the command fabricshow put together a table of switches and each SFOS role as shown in
Table 6-12. From the table, select and document fabric devices for policy enforcement and select a Secure Fabric OS
management station.
Plan to use robust, secure passwords as these are set during the initial secmodeenable. Be aware that the new
passwords must be different than the default setting. Depending on the site policies, the password may or may not be
recorded in Table 6-12 as shown. If passwords are recorded they must be protected, as they are a security risk. The
switchname, domain number or switch WWN is used to define and activate the initial FCS_POLICY configuration.
int217:admin> fabricshow
-------------------------------------------------------------------------
63: fffc3f 10:00:00:60:69:80:4d:fc 192.168.162.63 0.0.0.0 >"int63"
75: fffc4b 10:00:00:60:69:10:10:9d 192.168.162.75 0.0.0.0 "int75"
88: fffc58 10:00:00:60:69:51:0e:0a 192.168.155.88 0.0.0.0 "sialab88"
154: fffc9a 10:00:00:60:69:10:68:fa 192.168.162.154 0.0.0.0 "int154"
170: fffcaa 10:00:00:60:69:50:10:90 192.168.162.170 0.0.0.0 "int170"
217: fffcd9 10:00:00:60:69:90:03:fa 192.168.162.217 0.0.0.0 "int217"
The Fabric has 6 switches
Table 6-12 Fabric A Switch Table
Switch Domain Secure Name WWN Root Admin

FOS Role Password Password
SilkWorm 63 Primary FCS int63 10:00:00:60:69:80:4d:fc ddmfabra ddmfabra

12000
SilkWorm 217 Backup FCS int217 10:00:00:60:69:90:03:fa ddmfabra ddmfabra

3900
SilkWorm 170 Backup FCS int170 10:00:00:60:69:50:10:90 ddmfabra ddmfabra

3800
All Other 75, 88, Non-FCS int 75 10:00:00:60:69:10:10:9d password1 password1

Switches 154 Switches int 88 10:00:00:60:69:51:0e:0a
int154
10:00:00:60:69:10:68:fa
Table 6-13 Fabric A Device Table (all devices not shown)
Device Name Role Switch and Port Location Device Port WWN
poc50 Management Station N/A N/A

(ethernet connection)
int121 - HBA_A File Server Domain 154, Port 8 10:00:00:00:c9:24:f6:5d
RAID - Port 1 LUNs for int121 Domain 217, Port 13 10:00:00:00:ae:37:b6:70

6.5.3.5. Install Brocade SecTelnet and other utilities on Management Station

Install the Brocade Secure FOS software utilities such as SecTelnet. Note that the secmodenable command must be
executed from a SecTelent client. The SecTelnet utility is a minimum requirement to enable Secure Fabric OS. As part of
the preparation, it is recommended to install it on the SAN Management station where Fabric Manager exits. Also install
the utilities on the backup management station, if one is to be used.
6.5.3.6. Configure the time server

Enable the time server selected in section 6.5.3.4. Develop a Policy Plan on page 6-33. To enable the time server on a core
switch, use the command tsclockserver, as shown below.
int63:admin> tsclockserver "192.168.126.60"
Updating Clock Server configuration...done.
Please note Switch 0 configuration takes priority over Switch 1.
To verify the status of the time server, use the command tsclockserver with no operand. If the out put is LOCL, then
the external time server is not set.
int63:admin> tsclockserver
192.168.126.60
Please note Switch 0 configuration takes priority over Switch 1.
6.5.3.7. Enable Secure Fabric OS

Secure FOS requires several steps to be completed. This section will walk through an example used on the Case Study
SAN. Scheduled downtime is required for this activity on SANs that contain single fabrics. To enable Secure Fabric OS
use the following procedure:
1. As the admin user, open a secure telnet session with the Brocade SecTelnet client.
2. Use the command secmodeenable to enable secure mode in the fabric. Although any switch within the fabric will
suffice, it is good practice to use a core switch. This will minimize the hop count to the other switches. In the case
study, the switch int63, a SilkWorm 12000 was used as a core. For dual fabric SANs, first do this on Fabric A only.
3. Verify the devices are present with nscamshow or Brocade Fabric Manager before continuing. Since a core switch
with no locally attached devices was used in the example, nscamshow is sufficient. If using a switch with local
devices, use nsshow and nscamshow.
Guideline: For dual fabric SANs, enable security on Fabric A and verify basic device connectivity with nscamshow
before enabling Secure Fabric OS on Fabric B. It is recommended to schedule downtime on single
fabric SANs before enabling Secure Fabric OS. Once online, verify the all devices are logged in with
nscamshow. Downtime to the SAN can be avoided if using a redundant fabric SAN.
4. During the initial configuration with secmodeenable, the FCS and non-FCS switch roles need to be defined and
new passwords set. Gather this information from a list of switch roles created as part of the second checklist item.
Guideline: While Secure Fabric OS is being configured with secmodeenable, it is recommended to save all
events to a log file. To create a log file with SecTelnet, from the SecTelnet client window, click on the
Terminal -> Session Log -> Start. A dialog box appears. Select a location and enter a filename. Click
on Open to begin recording the telnet session. If logging is stopped and restarted using the same
filename, SecTelnet does not append the captured output. The file will be over-written and any previous
logging information will be lost. Be sure to save the captured session or use a different filename.

5. The output of secmodeenable (below) shows the case study session. The first entry in the FCS_POLICY list will be
the Primary FCS. The inputs are highlighted in bold text.
int63:admin> secmodeenable
Your use of the certificate-based security features of the software installed on this equipment is
subject to the End User License Agreement provided with the equipment and the Certification Practices
Statement, which you may review at http://www.switchkeyactivation.com/cps. By using these security
features, you are consenting to be bound by the terms of these documents. If you do not agree to the
terms of these documents, promptly contact the entity from which you obtained this software and do not
use these security features.
Do you agree to these terms? (yes, y, no, n): [no] yes
This command requires Switch Certificate, Security license and Zoning license to be installed on every
switch in the fabric.
PLEASE NOTE: On successful completion of this command, all login sessions will be closed and all
switches will go through a reboot to form a secure fabric.
This is an interactive session to create a FCS list.
The new FCS list is empty.
Enter WWN, Domain, or switch name (Leave blank when done): int63
Switch WWN is 10:00:00:60:69:80:4d:fc.
The new FCS list:
10:00:00:60:69:80:4d:fc
Switch WWN is 10:00:00:60:69:90:03:fa.
The new FCS list:
10:00:00:60:69:80:4d:fc
10:00:00:60:69:90:03:fa
Switch WWN is 10:00:00:60:69:50:10:90.
The new FCS list:
10:00:00:60:69:80:4d:fc
10:00:00:60:69:90:03:fa
10:00:00:60:69:50:10:90
Enter WWN, Domain, or switch name (Leave blank when done):

Are you done? (yes, y, no, n): [no] yes
Is the new FCS list correct? (yes, y, no, n): [no] yes
One or more Ulysses boxes are in the dual fabric. The switches might not work properly. Please, make
sure external NTP server is configured on all switches in the fabric before setting secure mode.
About to enable security.
ARE YOU SURE (yes, y, no, n): [no] yes
Please enter current admin account password: xxxxxxxx
Changing password for root
New FCS switch root password: xxxxxxx
Password must be between 8 and 40 characters long.
New FCS switch root password: xxxxxxxx
Re-type new password: xxxxxxxx
Changing password for factory
New FCS switch factory password: xxxxxxxx
Changing password for admin
New FCS switch admin password: xxxxxxxx
Changing password for user
New fabric wide user password: xxxxxxxx
Changing password for admin
New Non FCS switch admin password: xxxxxxxx
You cannot reuse the old password.
New Non FCS switch admin password: xxxxxxxx
_
Broadcast message from root Sat Jul 19 00:39:39 2003...
Security Policy or Password Change: root factory admin user will be logged out on switch 0

6. All switches will be rebooted automatically and the SecTelnet session will end. When the switches come online, the fabric
will re-build. Fabric Manager can be used to check the switches as they come online. Once the fabric is online, a secure
telnet session must be opened using the new password settings to the Primary FCS. All switches in the fabric are now
managed by it. To view the defined and active FCS_POLICY list use the command secpolicyshow.
Warning: All switches in the fabric will require a reboot for Secure Fabric OS to be brought online.
int63:admin> secpolicyshow
____________________________________________________
DEFINED POLICY SET
FCS_POLICY
Pos Primary WWN DId swName
__________________________________________________
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
____________________________________________________
____________________________________________________
ACTIVE POLICY SET
FCS_POLICY
__________________________________________________
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
____________________________________________________
Note: Be aware of the switch order as defined by the Pos (Position) column. In this configuration there are three FCS
switches. The first one listed is the Primary FCS and thus is in Position 1. If it were to fail, the Position 2 switch would
take over. If 1 and 2 were to fail than the Pos 3 switch would take over. If desired, the Primary FCS switch can be
changed with the secPolicyFCSMove command.
7. For a summary of the Secure Fabric OS configuration use the command secfabricshow.
int63:admin> secfabricshow
Role WWN DId Status Enet IP Addr Name
================================================================
Primary 10:00:00:60:69:80:4d:fc 63 Ready 192.168.162.63 "int63"
non-FCS 10:00:00:60:69:10:10:9d 75 Ready 192.168.162.75 "int75"
non-FCS 10:00:00:60:69:51:0e:0a 88 Ready 192.168.155.88 "sialab88"
non-FCS 10:00:00:60:69:10:68:fa 154 Ready 192.168.162.154 "int154"
Backup 10:00:00:60:69:50:10:90 170 Ready 192.168.162.170 "int170"
Backup 10:00:00:60:69:90:03:fa 217 Ready 192.168.162.217 "int217"
___________________________________________
Secured switches in the fabric: 6
Once Secure FOS is enabled administrative tasks can be accomplished at the command line. If using Fabric Manager for
Secure FOS administrative tasks (and it is highly recommended to do so), follow the steps in the next subsection.

6.5.3.8. Update the Fabric Manager Password Authentication
Before defining the policies, it is prudent to prepare Fabric Manager with the new switch passwords so that it does proper
authentication. Use the following procedure to update Fabric Manager for Secure FOS administrative tasks.
1. Click on Actions -> Security, a warning menu will appear as shown in Figure 6-22.
Figure 6-22 Login Information Warning Dialog Box

2. Click Yes.
3. Select the switches using the blue arrow button and enter the new password for the FCS switch admin user, as shown
in Figure 6-23. Note that the non-FCS switches have different set passwords and thus the authorization fails.
Figure 6-23 Fabric Manager - Fabric Login - Authorization Failed

4. Select the non-FCS switches by removing the FCS switches (not shown) from the Selected Switches pane on the right
and repeat the process. Click Apply to set and validate the new password settings. The Fabric Login appears as
shown in Figure 6-24.
Figure 6-24 Fabric Manager - Fabric Login - Successful Login

The Fabric is ready for Secure Fabric OS management. After a few moments, the Security Admin window appears in
Figure 6-25. This window provides a summary of the current policy configuration. Fabric Manager is now ready for
Secure Fabric OS policy management.
Figure 6-25 Security Admin Window

6.5.3.9. Creating Policies and Validating Functionality

Creating and adding members to policies at the command line is done much in the same way as zoning. Once a policy is
defined, only the members of the policy are granted access to a management area.
command secactivesize to monitor database usage.
The following overview describes the steps used to configure policies. For purposes of illustration, the following four SAN
Secure FOS design choices will be used as examples:
A. Define three FCS switches using the most highly available switch (preferably the SilkWorm 24000) as the Primary
FCS.
B. Configure the Management Station Access with SecTelnet and Fabric Manager installed.
C. Lock down the fabric to prevent any access by any unauthorized switch.
D. Lock down all host and storage devices to prevent any access by any unauthorized device.
6.5.3.9.1 Policy Creation Overview

This section will provide an overview of the required procedure to define all Secure FOS policies. Detailed examples follow
this explanation.
Note: The entire set of created policies will not be shown in the following examples, however, the examples provide the
essential steps to configure additional policies.
To make policy configuration changes, log onto the Primary FCS as the admin user with the Brocade SecTelnet utility. The
following five high level steps are used to configure Secure Fabric OS Policies. The steps are essentially the same for each
design choice.
1. Create the policy and populate it with the required device information.
Note: The policy must be selected from a pre-defined list. This list can be seen on the help pages at the command line or with
the Fabric Manager online help. If a policy is attempted to be defined that is not on this list, an error message will be
generated. Policy creation may require IP address, WWN, Domain ID or Port Number depending on the device to be
added. Multiple devices can be added at once.
2. Validate the Secure Fabric OS polices have the appropriate information.

3. Save the policy to NVRAM on the Primary FCS switch to have it persistent across reboots.
4. Activate the policies. This will cause the Primary FCS to copy the policy database to the Backup FCS switches and
begin enforcement.
5. Validate the changes are in place and test the access of the devices. Make sure that devices that should have access do
and those that should not have access are locked out.

Guideline: All Secure FOS policy changes can be made real time without rebooting the Primary FCS switch. During
the use of the case study SAN, changes were made with I/O activity and there was no negative affect.
Secure FOS Design choices A through D will be walked through step-by-step using the five steps above.
A. Define three FCS switches with the most highly available switch (preferably the SilkWorm 24000) as the Primary
FCS.
The three FCS switches were defined during the initialization of Secure Fabric OS with secmodeenable. The
FCS_POLICY list is created and populated by default and must exist at all times. FCS switches can be added/removed
from the FCS_POLICY with the commands secmodeadd/secmoderemove as needed.
Guideline: It is highly recommended to have three FCS switches in the list. For fabrics with three or fewer switches,
it is recommended to add all switches to the FCS_POLICY list.
B. Configure the Management Station Access.

To setup the access rights to a single management station that has Fabric Manager and SecTelnet installed, as in the case
study, three policies must be defined. They are the TELNET_POLICY, API_POLICY and HTTP_POLICY. Only the IP
Addresses in each policy list will be allowed access each of these management access points. The API_POLICY is
required to lock down access by a single Management Station through Fabric Manager. The HTTP_POLICY will prevent
another host from accessing the switches through a web client with Brocade Web Tools.
1. Create the following policies with secpolicycreate as shown. For each of these policies the IP Address of the
Management station is required.
int63:admin> secpolicycreate "TELNET_POLICY","192.168.173.50"
TELNET_POLICY has been created.
int63:admin> secpolicycreate "HTTP_POLICY","192.168.173.50"
HTTP_POLICY has been created.
int63:admin> secpolicycreate "API_POLICY","192.168.173.50"
API_POLICY has been created.
Guideline: To ensure that the secure SAN Fabrics can be managed, define a second management station as a
backup. Using secpolicyadd and adding the secondary management host’s IP address to the
TELNET_POLICY, API_POLICY and HTTP_POLICY is all that is required. Should the primary station
become unavailable, the SAN administrator will not be locked out of the fabrics. Refer to section 6.5.5.
Managing Fabric Changes: Secure Fabric OS for Change Management Control on page 6-51 for
information about adding a management station.
2. Validate the policies are correct with secpolicyshow. Note that the policies are defined but are not yet activated.
This allows a degree of flexibility by the SAN Administrator. Multiple changes can be made at once before having
the Primary FCS enforce them. Also, in the event of an entry error, changes can be made without affecting the current
active configuration.
____________________________________________________
DEFINED POLICY SET
FCS_POLICY
__________________________________________________
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
TELNET_POLICY

IpAddr
__________________________________________________
192.168.173.50
HTTP_POLICY
IpAddr
__________________________________________________
192.168.173.50
API_POLICY
IpAddr
__________________________________________________
192.168.173.50
___________________________________________________
ACTIVE POLICY SET
FCS_POLICY
__________________________________________________
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
____________________________________________________
Note: If desired, the policy change can be nullified using secpolicyabort. This command will write the current saved
policy set in NVRAM to the defined policy set. An example is shown below.
int63:admin> secpolicyabort
Unsaved data has been aborted.
Note: Save the new policy definition with secpolicysave. This will preserve the policies as they will be copied to
NVRAM flash on the Primary FCS switch. Once saved, the changes cannot be backed out with secpolicyabort.
Each change must be removed or deleted either at the command line or preferably, Fabric Manager.
int63:admin> secpolicysave
secpolicysave command was completed successfully.
3. Activate the enforcement of the policies with secpolicyactivate. Once executed, the Primary FCS switch will
write over all active policies on all other switches in the fabric.
int63:admin> secpolicyactivate
About to overwrite the current Active Policy Set.
secpolicyactivate command was completed successfully..
4. Once activated, validate the policies exist once again with secpolicyshow. Note that the new policies are now
part of the ACTIVE POLICY SET.
____________________________________________________
DEFINED POLICY SET
FCS_POLICY
__________________________________________________
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
TELNET_POLICY
IpAddr
__________________________________________________
192.168.173.50
HTTP_POLICY
IpAddr
__________________________________________________

192.168.173.50
API_POLICY
IpAddr
__________________________________________________
192.168.173.50
____________________________________________________
ACTIVE POLICY SET
FCS_POLICY
__________________________________________________
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
TELNET_POLICY
IpAddr
__________________________________________________
192.168.173.50
HTTP_POLICY
IpAddr
__________________________________________________
192.168.173.50
API_POLICY
IpAddr
__________________________________________________
192.168.173.50
5. Test the policy enforcement.

To make sure that only the defined management station has access, try to access the fabric with another host with a telnet
session, web client and Fabric Manager. Those without access should be prevented and the user notified that access is not
granted.
C. Lock down the fabric to prevent any access by any unauthorized switch.
Use the SCC_POLICY list, which is not created by default, to provide greater granularity for change management and to
prevent access by rogue switches. With no policy in place, all switches are allowed to join the fabric if they have valid
digital certificates. The SCC_POLICY list cannot be empty. This policy must contain the FCS switches at a minimum. Be
prepared with documentation of switch WWNs and roles before implementing the SCC_POLICY.
Note: This procedure can also be accomplished using Fabric Manager 4.0 as shown in section 6.5.4.1. Add All Switches to
The SCC_POLICY on page 6-48.
Warning: Once the SCC_POLICY is implemented, any switch listed in SCC_POLICY that is not in the FCS_POLICY list
will be forced to be a non-FCS switch. Non-FCS switches cannot become a Primary FCS Switch.
1. Create the SCC_POLICY. All switches can be added at once using the switch WWNs as shown in the following
example:
int63:admin> secpolicycreate
"SCC_POLICY","10:00:00:60:69:51:0e:0a;10:00:00:60:69:10:10:9d;10:00:00:60:69:10:68:fa;10:00:00:60
:69:50:10:90; 10:00:00:60:69:90:03:fa;10:00:00:60:69:80:4d:fc"
SCC_POLICY has been created.
If the policy is created without adding the non-FCS switches the following warning appears as shown:
int63:admin> secpolicycreate "SCC_POLICY"
One or more non-FCS switches are not in the SCC_POLICY. They will be excluded from the fabric when
activating the policy, unless you add them by using secPolicyAdd command later.
ARE YOU SURE (yes, y, no, n): [no] no

2. Once complete the SCC_POLICY will be created with only the FCS Members and stored in the defined policy set.
Do not activate the policy until all current non-FCS switches are added as members to the SCC_POLICY. Using the
case study SAN as an example, all non-FCS switches are must be added with the following command:
int63:admin> secpolicycadd
"SCC_POLICY","10:00:00:60:69:51:0e:0a;10:00:00:60:69:10:10:9d;10:00:00:60:69:10:68:fa"
Member(s) have been added to SCC_POLICY.
3. Use secpolicyshow to confirm the SCC_POLICY configuration. The case study SAN output is shown as
truncated output below:
____________________________________________________
DEFINED POLICY SET
FCS_POLICY
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
SCC_POLICY
WWN DId swName
10:00:00:60:69:51:0e:0a 88 sialab88
10:00:00:60:69:10:10:9d 75 int75
10:00:00:60:69:10:68:fa 154 int154
10:00:00:60:69:50:10:90 170 int170
10:00:00:60:69:90:03:fa 217 int217
10:00:00:60:69:80:4d:fc 63 int63
4. Use secpolicysave and secpolicyactivate to save and activate the defined configuration.
secpolicyactivate command was completed successfully.
5. Validate the policy. If another clean switch is available, attempt to add it to the fabric. When a switch is added that is
not authorized, the secure switch will the following message sent to the screen:
0x1030cd50 (tTransmit): Jul 20 03:13:03
INFO SEC-SECVIOL_SCC, 4, Security violation: Unauthorized switch 10:00:00:60:69:90:04:3d tries to
join secure fabric.
0x1030cd50 (tTransmit): Jul 20 03:13:03
WARNING FABRIC-SEGMENTED, 3, port 5, SCC violation
Furthermore, the port that had the violation is disabled, as shown by this partial output of switchshow:
int170:admin> switchshow
switchName: int170
switchType: 9.1
switchState: Online
switchMode: Native
switchRole: Subordinate
switchDomain: 170
switchId: fffcaa
switchWwn: 10:00:00:60:69:50:10:90
switchBeacon: OFF
Zoning: ON (ddm_cfg_a)
port 0: id N2 Online E-Port (Trunk port, master is port #1)
port 1: id N2 Online E-Port 10:00:00:60:69:80:4d:fc "int63" (upstream) (Trunk master)
port 5: id N2 In_Sync Disabled (Security violation)
To undo the violation, the port must be configured online with portenable. Once it is verified, the fabric is locked down by
the SCC_POLICY, the task is complete. The next step is to control device access to the SAN.

D. Lock down all host and storage devices to prevent access by any unauthorized device.
To lock down all host and storage devices connected to all fabric edge switches, the DCC_POLICY_ must be used. This
policy is unique in that many instances of it can exist. The example which uses the case study SAN Fabric, will
demonstrate how to create a policies for hosts connected as an F_Port and a JBOD which is connected as an FL_Port.
These two cases differ in that while the host will have a single node attached to the port, the JBOD will have multiple
nodes attached, since it is a loop device. The DCC_POLICY_ must contain Port World Wide Names (PWWN). This may
seem complicated, however as illustrated in the example that follows, there are multiple command line shortcuts. Fabric
Manager can be used for this as well. These steps are not provided in this section. Please refer to section 6.5.4.2. Create a
DCC_POLICY_ALL on page 6-50.
A DCC_POLICY_ for a node can be thought of as a set of rules that allow binding device nodes to specific switch ports.
With no policy (the default), any device can connect to any switch port on the fabric. An empty created DCC_POLICY_
is the same as having no policy. It is possible to have more than one policy for a single port. Since this can be somewhat
confusing, it is good practice to avoid doing this. There are subtleties and nuances that are out of the scope of this section.
Please refer to the Brocade Secure Fabric OS User’s Guide (part number: 53-0000526-02) or SAN Security: A Best
Practices Guide (part number: GA-RG-250-00) for details.
1. Create a switch wide DCC_POLICY_ for each edge switch in the fabric that includes all locally attached devices and
all switch ports using secpolicycreate. This is shown in the example below. The first operand is the policy name; in
this case it is DCC_POLICY_170. The second operand contains the domain number (170) and a list of ports. The *
refers to all ports on the local switch. The [] refers to an operation that automatically populates the scanned device
Port WWNs into the switch wide policy DCC_POLICY_170. Repeat secpolicycreate with similar arguments
for each edge switch in Fabric A.
int63:admin> secpolicycreate "DCC_POLICY_170","170[*]"
DCC_POLICY_170 has been created.
int63:admin> secpolicycreate "DCC_POLICY_88","88[*]"
Any number of ports can be specified using [] or (). If () were used instead, the device Port WWN would have to be
specified as part of the second operand. For example: secpolicycreate
“DCC_POLICY_tst”,”10:00:00:00:c9:29:04:8f;170(1,5-8,14,15)” will bind the host HBA Port
WWN F-Port 10:00:00:00:c9:29:04:8f to ports 1,5,6,7,8,14,15 on domain 170. All other device Port WWNs will be
denied access to those specified ports. Access will be granted to any device for all other ports, which are in this case are
0,2,3,4,9,10,11,12,13.
Guideline: Avoid nested DCC_POLICY_ definitions. Not only are they hard to document, they are difficult to
understand. There are rules that determine how the nested policies are enforced but applying the rules
and determining the outcome can be a complex task.
Guideline: For easy policy identification, append the domain number or switch name to the DCC_POLICY_ name.
For example for a switch with domain 170, create a policy called DCC_POLICY_170.
Guideline: The command line for this definition becomes complex, but it is simplified using Fabric Manager 4.0, as
shown in section 6.5.4.2. Create a DCC_POLICY_ALL on page 6-50.
2. Prevent nodes from connecting to the core switch(es). Since no device was specified as part of the second operand, all
nodes will be denied access to every port on the switch. The [] could be used as well, since no devices are attached.
int63:admin> secpolicycreate "DCC_POLICY_63","63(*)"

Guideline: For core switches, it is a good idea to lock down all unused ports to disallow any device from logging
into the fabric. To do this, use (*) or [*] as the port designation in the second operand with out any
devices attached. With [*], all switch ports to be scanned and no device will be added to the
DCC_POLICY_ list. With (*), and no device listed, the policy will be created with ports and no device
definitions. This too will prevent any device accessing all the ports on the core switches.
Guideline: Use secpolicyshow to verify the configuration. If correct it will only show the switch ports and domain
number. If only a DCC_POLICY_ exists, new switches can still be fabric members. Use the
SCC_POLICY to control fabric access by new switches.
Guideline: It is good practice to lock down all devices that connect to single switch, by creating a DCC_POLICY_
that binds all device Port WWN to all ports on a particular switch. Use [ ] as a shortcut to automatically
scan and populate the Port WWNs of the attached devices to the DCC_POLICY_ list. Once configured,
only those Port WWNs included in the policy list will be granted access to all local switch ports.
3. Validate the configuration with secpolicyshow before activating the configuration. Note that the
DCC_POLICY_63 have no devices listed. This means all devices are denied access to the switch. Unlike the example
which has the Area field truncated, the real output will show all ports.
DCC_POLICY_63
Type WWN DId swName
__________________________________________________
Switch 10:00:00:60:69:80:4d:fc 63 int63.
=Area=> 0,1,2,3,4,5,6,7,8,9,10,...,62,63.
DCC_POLICY_170
Type WWN DId swName
__________________________________________________
Switch 10:00:00:60:69:50:10:90 170 int170.
=Area=> 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15.
Device 22:00:00:20:37:15:1f:d0
Device 22:00:00:20:37:15:1a:f9
Device 22:00:00:20:37:15:0a:a5
Device 22:00:00:20:37:15:0c:0b
Device 22:00:00:20:37:e6:9a:8c
Device 22:00:00:20:37:e6:9a:6c
Device 22:00:00:20:37:15:0b:a5
4. Once the configuration has been verified, save and activate the configuration as before. List the new policy
configuration with secpolicyshow and verify the active policy has been updated to reflect the changes. This
output is not shown.
5. Validate the configuration. Attempt to add a device that is not authorized.

As before, a security violation message will be sent to the screen and the port disabled. Once again, the port will have
to be manually put online with portenable. Be aware that E_Ports are allowed if an ISL is attached from a switch that
does not violate the SCC_POLICY. If the SCC_POLICY did not exist, any switch could be added.

6.5.3.10. Backup the Configuration with configupload
Use configupload at the command line from the Management Station to save the configuration from the Primary FCS
switch. As noted earlier, this is very important the entire Secure Fabric OS configuration will be wiped clean if secure mode is
turned off.
int63:admin> configupload
Server Name or IP Address [host]: 192.168.173.50
User Name [user]: root
File Name [config.txt]: int63_primaryfcs_config.txt
Password:XXXX
Upload complete
Although not shown, Fabric Manager can be used to make a backup copy of the configuration.
Warning: If Secure Fabric OS is disabled, all policy changes will be lost. This includes the FCS_POLICY list that defines
the FCS switch members. All configuration steps will have to be redone once the Secure Fabric OS is re-enabled.
Guideline: Fabric Manager can be used to make backups as well. Once a policy is defined it is highly
recommended to save a baseline of the Primary FCS switch and store in a secure location. Be sure to
update the baseline on a regular basis as policy changes are implemented.
6.5.3.11. Document the Secure Fabric OS Configuration

Once the initial policies have been configured, capture the output of secpolicyshow. This command will output a
comprehensive list of all defined and active policies. Unfortunately, a number will refer to devices and switches in each
defined and active policy; whether it be IP address, Domain Number or Port WWN. To clarify the documentation, add host
names, switch names and device names to make the policy documentation comprehensive and easy to read. As changes occur,
keep the policy documentation current. This is critical for site preparedness in the event of a disaster, quick support turnaround
and as part of doing proper change management practices.
Guideline: Add meaningful names to the policy documentation. When changes occur, keep the information
updated. This is critical for quick support turnaround and site preparedness in the even of a disaster.
command secactivesize to monitor database useage.

6.5.4. Example of Managing Secure Fabric OS with Fabric

Manager
Fabric Manager can be used to manage Secure Fabric OS, simplifying the command line process. This section provides some
examples of how to best use Fabric Manager when initially staging Secure FOS and for administrative upkeep. Policy
management is greatly simplified with the use of this tool and therefore it is highly recommended for all aspects of day-to-day
operations.
6.5.4.1. Add All Switches to The SCC_POLICY

The following procedure is used to add all switches to the SCC_POLICY.
1. Click on the SCC tab of the Security Admin tab after selecting the fabric to be managed from in the SAN Elements Pane.
Enter * to select all switches in the current fabric and click on Create Policy as shown in Figure 6-26.
Figure 6-26 SCC Tab

2. Once the Create Policy button has been selected, the Security Admin window opens as shown in Figure 6-27. To
complete the process click Save, and then click the Activate button. Some informational and confirmation windows will
open (not shown). Once completed, the Defined and Active policy sets will be updated and the SCC_POLICY will be
enabled.
Figure 6-27 Adding a switch to a policy

3. To view the changes, click the Summary tab. Note the SCC_POLICY is now in the defined and active policy columns, as
shown in Figure 6-28.
Figure 6-28 Summary Tab

6.5.4.2. Create a DCC_POLICY_ALL

The following procedure is used to create a DCC_POLICY_ALL that includes all current switches and devices using Fabric
Manager.
1. Select the fabric and open the Security Admin window.
2. Click on the DCC tab and create a policy called ALL.
3. To add all current switches and attached devices, shift-select all of the switches and click on Add Member. This will
create the required DCC_POLICY_ALL in one step.
4. To complete the process click Save, and then click the Activate button. Some informational and confirmation windows
will open (not shown). Be sure to spot check the proposed changes. Once completed, the Defined and Active policy sets
will be updated.
Figure 6-29 Security Admin Window

6.5.5. Managing Fabric Changes: Secure Fabric OS for

Change Management Control
This section will focus on guidelines and practices associated with the use of Secure Fabric OS management tools to properly
track and manage the addition/removal of different SAN components. For large changes, exhaustive planning should take
place to ensure that the right changes are documented, authorized and validated.
6.5.5.1. Specific Case Study Goals/Requirements

The following scenario is used as a basis for the case study policy changes. These tasks were chosen to emulate typical
changes that occur within a Secure FOS environment.
• Adding a new host, with hostname int212, as a backup management station.
• Removing a Fibre Channel HBA from the configuration.
• Addition of a new SilkWorm 3900 switch called int218 to Fabric A (and int220 to Fabric B) to meet future increased
port count requirements.
• Changing passwords on the non-FCS and FCS switches.
All changes for the first two tasks are made using secpolicyadd and secpolicyremove. Adding a new switch requires
the same level of preparation as the current members of the Secure Fabric OS fabric plus a few additional steps. Because of
this, a separate checklist for adding a new switch will be provided. To view the details of this, refer to the section 6.5.6. Adding
a Switch to a Secure Fabric on page 6-55. Changing passwords for the non-FCS switches is accomplished with
secNonFCSpasswd. The commands will be shown as needed for Fabric A. The steps for Fabric B will not be shown, as
they are the same.
Warning: Do not use secpolicydelete or secpolicycreate when making the changes. secpolicydelete
will permanently delete a policy and all members. secpolicycreate defines and creates a brand new policy
type. Use secpolicyremove to remove a device from a current defined policy and secpolicyadd to add
a device.
Guideline: Although the command line is shown for change management activities, Fabric Manager can be used
as an alternative. The GUI interface allows for complex changes to be made in an intuitive manner.
Please refer to section 6.5.5. Managing Fabric Changes: Secure Fabric OS for Change Management
Control on page 6-51.

Table 6-14 Change Management Checklist
Change Management Checklist

1. Gather the stakeholders. Use the current Secure Fabric OS policy documentation as a basis for the
update.
2. Create a Policy Action Table using a spreadsheet or other means.
3. Authorize the policy changes. If adding a switch, refer to item 4.
4. Optional: Prepare and add a new switch. (Refer to section 6.5.6. Adding a Switch to a Secure
Fabric on page 6-55).
5. Update the defined Secure Fabric OS Policy Configuration with secpolicyadd or
secpolicyremove.
6. Save and activate the new policy configuration after verifying the policy changes with
secPolicyadd.
7. Validate the new policy is being enforced by the Primary FCS.
8. Optional: Change the passwords.
9. Backup the Secure Fabric OS configuration with configupload.
1. Gather the Stakeholders

Gather the stakeholders and documentation that was created when the configuration was baselined. For larger
configurations, this documentation may be online in a database. It is important to have the baseline configuration
documentation available as part of the preparation process so that the stakeholders can make the right decisions. This
becomes critical for larger configurations and changes. For smaller configurations and/or changes to the Secure Fabric OS
policies, the overall documentation effort may be minimal. All changes should be double checked to prevent initiators
from losing their targets when the configuration is put back into production.
Guideline: Be thorough in the planning process to prevent mistakes such as preventing initiators from seeing their
targets.
2. Create a Policy Action Table

Record Port WWNs, IP address, MAC addresses of all SAN Components being acted on by Secure Fabric OS policy
changes before actually implementing them. An effective technique is to create a table that contains these components as
well as the desired actions to take place, as shown in Table 6-15. Use the table as a checklist while entering in the
commands that update the defined policy set.

Table 6-15 Secure Fabric OS Policy Action Table
SAN Component Role Action Required Policy

Information
Host int212 Management Station Add to TELNET_POLICY, IP Address
API_POLICY and
HTTP_POLICY
Host int121-HBA_A File Server Remove from Port WWN of HBA

DCC_POLICY_75
Switch int218 Non-FCS Edge Switch Prepare using the checklist and Switch WWN
(used SilkWorm 3900) add to SCC_POLICY
3. Authorize the policy changes

It is important to get the policy changes authorized before the implementation. For larger and/or more critical changes,
have a final meeting to spot check all of the changes in the documentation and make sure the requirements are met and to
catch mistakes. For smaller configuration changes, having a single individual responsible for authorizing Secure Fabric
OS policy changes will be likely sufficient.
4. Optional: Prepare and add a new switch
In this case, a new switch is desired to increase the port count on each fabric. Refer to section 6.5.6. Adding a Switch to a
Secure Fabric on page 6-55 for a checklist of the required steps to add the SilkWorm 3900 as a non-FCS member.
5. Update the defined Secure Fabric OS Policy Configuration
The following commands at the SecTelnet prompt will make the changes using the information in Table 6-14. Although
the command line is shown, Fabric Manager can be used to simplify the process.
Guideline: If Fabric Manager is installed, using it to make changes is the preferred method. The GUI interface is
intuitive. Before changes are made, a validation window appears that allows the change to be verified
before being saved to the defined policy.
a. Adding the int212 management station:

int63:admin> secpolicyadd "TELNET_POLICY","192.168.162.212"
Member(s) have been added to TELNET_POLICY
int63:admin> secpolicyadd "HTTP_POLICY","192.168.162.212"
Member(s) have been added to HTTP_POLICY
int63:admin> secpolicyadd "API_POLICY","192.168.162.212"
Member(s) have been added to API_POLICY
Guideline: If access to a serial connection is to be required, use the SERIAL_POLICY list to define the hosts to be
added. As before, use secpolicycreate to create a new policy list and secpolicyadd to add a
management host to the existing SERIAL_POLICY.
b. Removing an HBA from the DCC_POLICY_154:

int63:admin> secpolicyremove "DCC_POLICY_154","10:00:00:00:c9:24:f6:5d"
Member(s) have been removed from DCC_POLICY_154
c. Validate the changes have been made to the defined policy set with secpolicyshow (output not shown).
The host int212 should be added to the TELNET_POLICY, HTTP_POLICY and API_POLICY and the HBA removed
from the DCC_POLICY_154.

6. Save and activate the new policy configuration.

Once the changes have been made, save and activate the configuration as shown below.
7. Validate the new policy is being enforced.

Once the new policy has been activated, verify the new Secure Fabric OS configuration. As an example, the new
management station int212 was tested by installing Brocade SecTelnet and attempting to log on to the Primary FCS. If the
policy is successfully implemented, int212 will successfully be able to log on. If not, check the policy an make sure the IP
address, etc. is correct. To test the new DCC_POLICY_154, the old HBA in int121 was re-attached. A security violation
was reported on the switch and the port disabled as expected. These are just a few of the validation steps that can be
performed.
Guideline: It is good practice to validate all the changes that occur in the Secure Fabric OS policy configuration
before going into production. Test each change; make sure that all have been activated.
8. Optional: Change the passwords on the non-FCS switches.

If desired the passwords can be set on the non-FCS switches in the fabric with secnonfcspasswd. The change is
stored in non-volatile flash memory and is persistent across reboots.
int63:admin> secnonfcspasswd 88,”admin”
New Non FCS switch admin password:xxxxxxxx
Re-type new password:xxxxxxxx
The password has been successfully changed.
Note: There is no Secure Fabric OS command that allows the permanent password change to all FCS switch members. Use
sectemppasswdset to set a temporary password for the FCS switches. To remove the temporary password, use
the command sectemppasswdreset. To permanently change the FCS passwords for each user account, root
access is required. Use the passwd command as is normally done on UNIX hosts. If the password has been lost, please
contact the switch vendor for the recovery process.
9. Backup the Secure Fabric OS Configuration.

Use configupload to save the Primary FCS configuration as shown in the example below.
int63:admin> configupload
Server Name or IP Address [host]: 192.168.173.50
User Name [user]: root
File Name [config.txt]: int63_primaryfcs_config_07272003.txt
Password:XXXX
Upload complete
It is recommended to save the prior two changes so that if a roll-back to earlier configuration versions are desired, the
option will be available. This is critical for troubleshooting or if a new Primary FCS is to be brought online.
Guideline: Save the prior two changes to the Secure FOS configuration. This allows for two roll back choices in the
event problems occur.

6.5.6. Adding a Switch to a Secure Fabric
Note: Although other switches may be added to a secure fabric, a SilkWorm 3900 is used in this example.
Table 6-16 Checklist for adding a SilkWorm 3900 as a non-FCS member
Checklist for Adding a SilkWorm 3900 Non-FCS member

1. Prepare the switch for secure mode.
2. Validate the Secure Fabric OS requirements on the new switch, int 218.
3. Obtain the WWNs of the current FCS members of the Secure Fabric OS.
4. Enable Secure Fabric OS on the new switch. The WWN of each FCS switch in the secure Fabric A
must be used in the SAME ORDER on the new switch.
5. Prepare the new switch to accept the current Primary FCS databases by issuing the command
secversionreset.
6. Add the new SilkWorm 3900 to the SCC_POLICY on the Primary FCS of the existing Secure
Fabric OS fabric with secpolicyadd. Save and activate the configuration with
secpolicysave and secpolicyactivate.
7. Physically attach the new switch to Fabric A. Establish the new switch as an edge switch. Follow
the cabling guidelines in Chapter 2, SAN Design & Architecture Concepts.
8. Validate the configuration with secfabricshow. Verify the new switch is now a new non-FCS
member of Fabric A.
Warning: DO NOT use secversionreset on the existing fabric. This will wipe out the Primary FCS databases and
update each switch in the fabric accordingly.
Warning: If the procedure in Table 6-16 is not followed, the new switch will have a security violation on its E_Port
connection as shown by switchshow below:
19 id N2 Online E-Port segmented,(Security Violation)(Trunk master)
1. Prepare the switch for secure mode.

The SilkWorm 3900 was a used switch. If not fresh out of the box with the factory install, as in this case, the following
steps were followed.
a. Backup the switch configuration if desired with configupload.
b. Clean up the zoning of the switch with cfgdisable, cfgclear and cfgsave.
c. Use configdefault to wipe out other parameters. Note the IP address, domain number and switch name will not
be set to factory defaults. Make sure the domain number is set to a unique value.
d. If the switch does not have FOS Version 4.1, and is being added to Fabric OS 4.x based fabric, be sure to set the Core
PID format to 1 using the configure command. Note that the switch must first be disabled with
switchdisable.
The switch is now prepared for Secure Fabric OS installation tasks.

2. Validate the Secure Fabric OS requirements.

The new switch must have Fabric OS version 4.1 or later (or 3.1 or 2.6.1, or later, for other switch models), licenses for
Secure Fabric OS and Zoning, and has the PKI Objects installed. This can be verified with configshow “pki” or
pkishow (4.x only). Refer to Table 5-13 on page 5-24.
3. Obtain the WWNs of the current FCS members
This is best done with secmodeshow on the existing secure fabric. The order of the WWNs is important.
int63:admin> secmodeshow
Secure Mode: ENABLED.
Version Stamp: 419825780, Sat Jul 26 19:32:56 2003.
Pos Primary WWN DId swName.
=================================================
1 Yes 10:00:00:60:69:80:4d:fc 63 int63.
2 No 10:00:00:60:69:90:03:fa 217 int217.
3 No 10:00:00:60:69:50:10:90 170 int170.
4. Enable Secure Fabric OS on the new switch

Use the WWN of the current FCS switch members, in the SAME ORDER as in the FCS List when going through the
process of enabling Secure FOS. Note that the domain names or switch names will NOT work in this case since there is no
connection to the existing secure Fabric A. Assign new passwords. The passwords do not have to be identical to the other
fabric FCS and non-FCS members. Once complete the switch will reboot into Secure Mode. The command is shown
below with no output.
int218:admin> secmodeenable
Caution: The WWNs of the current FCS members must be entered in the same order as the secmodeshow displays
when enabling Secure FOS with secmodeenable.
5. Prepare the new switch to accept the current Primary FCS databases
Log in as admin with the Brocade SecTelnet utility and issue the command secversionreset. In the example, the
same management station that manages the Secure Mode on Fabric A was used.
int218:admin> secversionreset
About to reset version stamp to 0.
Security Policy Version Stamp has been set to zero.
If the version stamp is set to zero, as shown by secmodeshow, the output will be:
int218:admin> secversionreset
Stamp is 0.
Warning: Be sure that the switch to be added is what is logged on to before executing this command. If not, the version
information will be wiped away and the Primary FCS database cleaned out.
6. Add and Save the new SilkWorm 3900 entry to the existing SCC_POLICY
Log into the Primary FCS of the existing Secure Fabric OS fabric and use secpolicyadd to add the new switch to the
existing SCC_POLICY. Validate the configuration with secpolicyshow. Note that the new switch is registered in the
SCC_POLICY with an unknown status. This is expected at this stage.
int63:admin> secpolicyadd “SCC_POLICY”,”10:00:00:60:69:90:04:3d”
Member(s) have been added to SCC_POLICY

____________________________________________________
DEFINED POLICY SET
FCS_POLICY
__________________________________________________
1 Yes 10:00:00:60:69:80:4d:fc 63 int63
2 No 10:00:00:60:69:90:03:fa 217 int217
3 No 10:00:00:60:69:50:10:90 170 int170
SCC_POLICY
WWN DId swName
__________________________________________________
10:00:00:60:69:51:0e:0a 88 sialab88
10:00:00:60:69:50:10:90 170 int170
10:00:00:60:69:90:03:fa 217 int217
10:00:00:60:69:80:4d:fc 63 int63
10:00:00:60:69:10:10:9d 75 int75
10:00:00:60:69:10:68:fa 154 int154
10:00:00:60:69:90:04:3d - Unknown
7. Physically Attach the new switch to Fabric A

Establish the new switch as an edge switch by cabling it to the SilkWorm 12000 core switch acting as the Primary FCS.
When cabling, follow the cable layout guidelines in Chapter 2, SAN Design & Architecture Concepts. For other cabling
practices, please see section 5.4. Cable Planning on page 5-5. As soon as the new switch is attached, a build fabric will
occur. If this is being done for the first time, the new switch will reboot to acquire the new password database from the
Primary FCS. The other switches in the fabric will not reboot. The following output will be observed on the Primary FCS:
int63:admin> fabric: Reconfiguration due to Rcv BF(port 46)
fabric: Reconfiguring at Mon Jul 28 21:03:02 2003
8 7 6 5 4 3 2 1
8. Validate the configuration

Verify the new switch is a new non-FCS member of Fabric using secfabricshow as shown in the bold output below.
int63:admin> secfabricshow
Role WWN DId Status Enet IP Addr Name
================================================================
Primary 10:00:00:60:69:80:4d:fc 63 Ready 192.168.162.63 "int63"
non-FCS 10:00:00:60:69:51:0e:0a 88 Ready 192.168.155.88 "sialab88"
non-FCS 10:00:00:60:69:10:68:fa 154 Ready 192.168.162.154 "int154"
Backup 10:00:00:60:69:50:10:90 170 Ready 192.168.162.170 "int170"
Backup 10:00:00:60:69:90:03:fa 217 Ready 192.168.162.217 "int217"
___________________________________________
Secured switches in the fabric: 7
The new switch has been added successfully. To promote the switch to an FCS backup role, add it to the FCC_POLICY
list using secpolicyadd.
6.5.7. Removing a Switch from a Secure Fabric

Removing a switch is very straightforward. Use secpolicyremove on the SCC_POLICY and FCC_POLICY lists.
Also remove any DCC_POLICY_ that has been defined. This can be a bit tricky if there are nested DCC_POLICY_s
defined and activated. Fabric Manager can be used to simplify this process.

6.6. Validating the SAN

Once the SAN is staged, it is highly recommended to verify its functionality and robustness before going into production.
While less important for the entry-level environment, validation becomes critical for SANs with higher port counts. While all
pertinent tests for a particular implementation will not be discussed, this section should provide a framework for a customized
validation plan. Sample procedures in this section will demonstrate how to check the SAN stability and High Availability
(HA). While not covered, it is important to validate the Secure FOS environment as well. If at all possible, it is a good idea to
do these tests with generated I/O, preferably with the application up and running.
These tests are meant to be used as guidelines and a proof point that the SAN is operating properly before it is put into
production. This section will focus on validating a Core/Edge SAN. If Core/Edge is not used, all tests in this section can be
tailored for other fabric topologies. Separate tests will be required for the particular application in use in the SAN.
6.6.1. Sample Script to Generate I/O

With no application, it is possible to generate I/O. If using UNIX hosts the following sample script can be used. For Windows
hosts, use an I/O tool such as Iometer. This script creates and writes to a file and then does continuous reads of it. The path and
size in blocks need to be specified. As an example, sbtest /hds01 1000 will create a file of size 1000 blocks in /hds01
and once created, it will do successive reads until terminated. More I/O can be generated on a single host using multiple
instances of this script running in the background. Portperfshow is a helpful command line tool that can be used to quickly
check expected I/O performance and functionality. The sample script is shown below.
#!/bin/sh
PATH=$1
SIZE=$2
COUNT=`/usr/bin/expr $SIZE \* 2`
TMPFILE="$PATH/sbtest.$$"
RUN=0
echo "Building test file ($TMPFILE)..."
/usr/bin/dd if=/dev/zero of=$TMPFILE bs=512k count=$COUNT > /dev/null 2>&1
echo "Done."
while [ 0 -eq 0 ];
do
DATE=`/usr/bin/date`
echo "Run #: $RUN Timestamp: $DATE"
/usr/bin/dd if=$TMPFILE of=/dev/null bs=512k count=$COUNT >/dev/null 2>&1
RUN=`/usr/bin/expr $RUN + 1`
done
Figure 6-30 Sample Script
6.6.2. Sample Validation Recommendations

Table 6-17 SAN Fabric Validation Checklist
Validation Checklist
1. Fabric Stability Validation - Refer to section 6.6.2.1. Fabric Stability Validation

on page 6-59.
2. High Availability ISL Failure Simulation - Refer to section 6.6.2.2. High
Availability ISL Failure Simulation on page 6-59.
3. High Availability Switch Failure Simulation - Refer to section 6.6.2.3. High
Availability Switch Failure Simulation on page 6-59.

6.6.2.1. Fabric Stability Validation
For validating the stability of the fabric, run the host application software for a period of time. If available, use simulation tools
provided by the software application vendor. In the case study, the sample script was run for 72 hours to check for I/O stability
and no problems were observed. Switch event logs should be setup to capture any problems in the fabric. In addition, a host
syslogd can be setup to capture event data. No problems should be observed. If there are, troubleshoot them at this time.
Guideline: Check the switch event logs for any issues with the fabric while validating it’s stability. Setup the syslogd
system log on a host and configure to capture fabric event messages.
6.6.2.2. High Availability ISL Failure Simulation

All hosts are attached to one edge switch as shown in Figure 6-1. From this switch, one trunk with two ISLs are attached to
each core switch to form a trunk group with 4 Gbit/sec of available bandwidth. These trunk groups are contain an ISL trunk
master and a second ISL as a trunk member. A maximum of 400 MB/sec should be generated across them to provide
maximum ISL stressing. The switch tests simulated failures on an edge and core switch. Hot code load should also be tested
before going into production.
Note: When attaching ISLs to the SilkWorm 2800 there will be no trunking as it is not supported in Fabric OS 2.x. The
connections will be auto-negotiated at 1 Gbit/sec as non-trunked E_Ports.
Case 1 Member ISL

With I/O, the non-trunk master ISL was removed for ten seconds and then replaced. Repeat this test three times. I/O should
continue without any effect. Some frames may be “lost” when the cable is pulled. If this is the case, the host will log messages
indicating that the I/O is being retried.
Case 2 Trunk Master ISL
The Trunk master ISL was removed for ten seconds and then replaced. Repeat this test three times. I/O traffic may be paused
briefly on the second fabric, however the I/O should not timeout on the host.
6.6.2.3. High Availability Switch Failure Simulation

Case 1 Edge Switch Failure
A fastboot was initiated on a switch with no I/O to simulate edge switch failure. Repeat this test three times in succession.
The switch should fully recover after the fastboot and the fabric should rebuild. Other devices not associated with the rebooted
switch should continue to perform I/O. Verify this with the fabricshow command.
Case 2 Core Switch Failure
A fastboot was initiated on a core switch. Repeat this test three times in succession. The switch should full recovery after
the fastboot and the fabric should rebuild. Verify this with the fabricshow command.
Case 3 CP Failover (SilkWorm 12000/24000)
Initiate an hafailover on the active CP with I/O running in the fabric. Full recovery without an I/O pause should occur in
all cases. Refer to section 7.2. Fabric OS Version 4.1 (or higher) Ultra High Availability (HA) on page 7-3 for information
about the expected behavior.
Case 4 Non-Disruptive Code Load (Fabric OS 4.1 or higher only)
Initiate a firmwaredownload on a switch with Fabric OS 4.1 or higher. Full recovery without an I/O pause should occur in
all cases. Refer to section 7.2. Fabric OS Version 4.1 (or higher) Ultra High Availability (HA) on page 7-3 for information
about the expected behavior.


Chapter
Maintenance and Operations
7
After the Brocade SAN fabrics have been staged it is important to maintain them for proper and continued operation. This
chapter will provide guidelines on firmware upgrades, Ultra High Availability and as well as other useful commands for
gathering important information about the SAN.
7.1. Upgrading from Fabric OS 2.x/3.x/4.x

This section will provide some high level guidelines when upgrading firmware on Brocade SilkWorm switches from Fabric
OS 2.x/3.x/4.x to Fabric OS 2.6.1/3.1/4.1 or later. The context of this section will be using the command line to upgrade a
single switch. For upgrading multiple switches, Brocade Fabric Manager is suggested, though it will not be discussed here.
Please see Section III SAN Management for details on using Fabric Manager. There are some significant differences between
upgrading Fabric OS 2.x/3.x and 4.x which will be noted. For detailed instructions on upgrading Fabric OS 2.x/3.x and 4.x
reference the SAN Migration Guide v1.1 (part number: 53-0000360-02). For detailed instructions for upgrading from 4.x
please refer to the tech note titled: Upgrading to Fabric OS v4.1.x (publication number: 53-0000380-01).
7.1.1. Fabric OS Upgrade Overview

All upgrades from Fabric OS versions 2.x and 3.x will be disruptive. When upgrading to Fabric OS 2.6.1 and 3.1 or later, either
RSH or FTP can be used to execute the upgrade. Since only one telnet session is allowed for Fabric OS versions 2.6.1 and 3.1,
there is no firmwaredownloadstatus command.
In Fabric OS 4.x, firmwaredownload only allows the FTP protocol to be used for upgrades. Be aware that upgrading from
Fabric OS 4.0.x is disruptive. A scheduled downtime is required for these upgrades. All further upgrades from Fabric OS 4.1
will be non-disruptive. For Fabric OS 4.0.0d or higher, firmwaredownload performs a check on the High Availability
status. If this check fails, then the firmware cannot be upgraded automatically. Instead, firmwaredownload must be
executed with the -s option.
When upgrading from Fabric OS 4.1/4.2, the disruption will last longer on the non-bladed platforms as there is only one
processor and the firmware must failover to itself. Essentially, this is because the Fabric OS Linux kernel and other processes
that run in user space must be stopped and restarted through a fastboot. By default, a firmwarecommit is launched after
the switch is rebooted. This process runs in the background and copies the new firmware from the flash memory primary
partition (just upgraded) to the backup partition.
Firmwaredownloadstatus can be used on another telnet session to check the upgrade progress. This essentially plays
back the firmwaredownload log. The only message that will be received during the upgrade is “Firmwaredownload
has started.” After the fastboot, which happens automatically, run firmwaredownloadstatus. The total upgrade
time is about 13 minutes, including the required firmwarecommit. This process backs up the primary flash partition to a
secondary one. Although it takes about 13 minutes to complete the upgrade on a SilkWorm 3900, it will be non-disruptive. The
fabric state will be saved in non-volatile memory, and after the reboot, all of the fabric services start up first.
int219:admin> firmwaredownloadstatus
[0]: Sun Mar 16 20:41:07 2003
Firmwaredownload has started.
[1]: Sun Mar 16 20:47:08 2003
Firmwaredownload has completed successfully.
[2]: Sun Mar 16 20:48:52 2003
Firmwarecommit has started.
[3]: Sun Mar 16 20:53:50 2003
Firmwarecommit has completed successfully.
[4]: Sun Mar 16 20:53:50 2003
Firmwaredownload command has completed successfully.

7 Maintenance and Operations
7.1.2. Upgrade Guidelines from Fabric OS 4.0.x on a

SilkWorm 12000
On a SilkWorm 12000 running Fabric OS versions 4.0.0 to 4.0.0c the firmwaredownload command must be used
individually on each CP. To minimize downtime, start with the Standby CP and then do a failover with hafailover. Find
out which is active by executing hashow. These procedures are outlined in the Brocade Fabric OS Procedures Guide.
For Fabric OS versions 4.0.0d or higher, firmwaredownload, which is invoked from a logical switch, will automatically
upgrade both logical switches in a SilkWorm 12000. This also includes all upgrades for Fabric OS 4.1 and higher. When
running it, the output of firmwaredownload does not show each package being downloaded. This makes monitoring the
status difficult. Before performing the upgrade, open a second telnet session on the logical switch and run
firmwaredownloadstatus. This will show a log of each step in the process.
Guideline: The -s option provides additional control over the download process.
In Fabric OS 4.0.0d or higher, the -s option can be used on a SilkWorm 12000. This allows a single
CP to be upgraded at a time. Use this command first on the Standby CP and then the Active CP.
As of Fabric OS 4.1.2, the -s option is supported on the SilkWorm 3900.
It is recommended to use the command line when using the -s option.
When complete, about 20 minutes, use firmwareshow to display current firmware versions.
poc165:admin>
Remote CP (Slot 6, CP1): Standby
When upgrading from Fabric OS 4.1/4.2 and later, the process is non-disruptive. This is much faster on the SilkWorm 12000
as there are two CPUs and two fabric state images, one per CP. The standby simply becomes the active during the failover.
Thus, the impact on a production fabric is much less, when using these bladed platforms. When upgrading from Fabric OS 4.1
and beyond, use hashow first before launching firmwaredownload. The hashow output should display as follows on
Fabric OS 4.1.
poc165:admin> hashow
Local CP (Slot 6, CP1): Standby
Remote CP (Slot 5, CP0): Active
HA enabled, Heartbeat Up, HA State synchronize
This means that a firmwaredownload will be allowed to start non-disruptively. If hashow displays anything else, the
firmwaredownload will not start.
Caution: Do not attempt to load Fabric OS 4.x on a switch currently running Fabric OS 2.x or 3.x. Reference the Brocade
Fabric OS Procedure Guide for Fabric OS upgrade process for your switch type. Fabric OS versions 3.1/4.1 and
greater have a feature called firmware watermarking. This prevents other Fabric OS firmware versions
from writing over it.

Maintenance and Operations 7
7.2. Fabric OS Version 4.1 (or higher) Ultra High

Availability (HA)
It is recommended to utilize high availability on switches running Fabric OS version 4.1 and greater. This section will provide
information and guidelines on how to verify that HA is functional.
Non-disruptive means that data traffic continues to flow during a failover or firmware upgrade. The non-disruptive highlights
that Fabric OS version 4.1 and later provide are listed below.
• Non-disruptive fail-over for the SilkWorm 12000 and 24000 Active and Standby CP Cards.
• Non-disruptive firmware activation on the SilkWorm 3250, 3850, 3900, 12000 and 24000 platforms.
• Introduction of a standby CP health monitor for the SilkWorm 12000 and SilkWorm 24000. This function monitors
conditions that might prevent the fail-over mechanism from working.
Note: High availability is not available on Fabric OS 2.x and 3.x.
Table 7-1 Expected HA Functionality
Ultra HA Feature Data Flow Delay Fabric Services Pause

(Reads, Writes) (Name Server, FLOGI, etc.)
CP Failover (SilkWorm 12000, 24000) 0 (seconds) Between 5-10 seconds
Firmware Activation (SilkWorm 12000, 24000) 0 (seconds) About 2 seconds
Firmware Activation (SilkWorm 3900, 3250, 3850) 0 (seconds) Between 30-50 seconds
7.2.1. Ultra HA Functionality Overview

The the two CPs on the SilkWorm 12000 and 24000 maintain a synchronized state through Ethernet data packets transmitted
across the backplane. This means that there is essentially a mirror image of the fabric configuration that is stored on each CP.
Note: The SilkWorm 3900, 3850 and 3250 each have one processor, therefore failover does not apply.
In order to facilitate this, the environment must be stable for synchronization to occur. After initial power up, the SilkWorm
12000/24000 waits for a stable environment before attempting to synchronize the state between the two CPs. If the switch is
standalone, then the system requires 3 X FS_TOV or 15 seconds. After this time, whenever a change is detected the Active CP
updates the standby with the required information.
7.2.2. HA Caveats
HA is available only in Fabric OS 4.1 or later. Telnet commands run during non-disruptive failover should be reissued, as the
sessions will be dropped. Within a maximum 30 seconds, IP will restart. I/O continues through the fabric while this process
takes place. Port LED Lights may stop blinking but the I/O will still continues. Also note that the firmwaredownload
process will still take the same amount of time as before. On the SilkWorm 3900/3850/3250 Non-Disruptive Code load, takes
longer since there is only one processor.

7.2.3. HA Commands
As a result of HA capability several commands have been developed. The most important command is hashow. This displays
the state of the HA present on the local switch. The older HA mode is still supported, but not recommended. The four
commands are, haShow, haSyncStart, haSyncStop, and hadump. The rest of this section will discuss these
commands and provide examples of usage.
7.2.3.1. Using haShow

This is haShow when both of the CPs on the SilkWorm 12000/24000 are properly synchronized.
Remote CP (Slot 6, CP1): Standby, Healthy
HA Enabled, Heartbeat Up, State Synchronized
When the SilkWorm 12000/24000 does not have synchronized CPs, haShow will display the following output:
Remote CP (Slot 6, CP1): Standby, Healthy
HA Enabled, Heartbeat Up, HA Not Synchronized
7.2.3.2. Guidelines For Using haSyncStop, haSyncStart, and hadump

The command hasyncstop should be used only for troubleshooting purposes, as it will temporarily turn off non-disruptive
failover or non-disruptive code activation. This means that the next fail-over or firmware download and activation will be
disruptive. When haSyncStop is run, all ports WILL be reset. The command will only be effective until next fail-over or if
the command haSyncStart is issued. Note that the two CPs will re-synchronize as normal only after the next failover.
Therefore this command should not be executed command during normal operations. Here is an example of using the
commend haSyncStop.
poc165:admin> hasyncstop
Stop synchronize 0x228 (fabos): Switch: 0, Info FSS_ME-FORCELOG, 4, Software out of sync!
HaSyncStart is used to re-activate synchronous HA between the CPs. It only needs to be executed after an haSyncStop.
Here is an example of usage and output.
poc165:admin> hasyncstart
Start synchronize ...
done
poc165:admin> 0x223 (fabos): Switch: 0, Info FSS_ME-FORCELOG, 4, Software is in sync!
Hadump is a supportability command that displays HA related logging information. Hadump should not have to be used by
the general SAN administrator. It is really meant to be a troubleshooting tool for technical support.

7.3. Support for Boot Over SAN

Note: Boot Over SAN is officially supported in Fabric OS 2.6.2, 3.1.2 and 4.2.
The following HBA Vendors and Operating Systems were tested. For the latest support, please consult your switch vendor.
Table 7-2 Tested Boot Over SAN Configurations.
Operating Systems (OS) Host Bus Adapters (HBAs) Test Areas

Windows NT, 2000 Qlogic, Emulex, JNI Boot up verification, ISLs, FSPF Routing, Zoning,
HPUX, Solaris, Linux, AIX (2 Gb/sec models) Secure Fabric OS, General Fabric Changes
7.4. Compact Flash Space Monitoring and

Management
Note: Compact flash space monitoring is supported in Fabric OS 4.2 or greater.
CF usage on the primary partition is now monitored on a regular basis by a cron job that is run at consistent time intervals. If
the CF utilization is >= 80%, then a warning message is entered into the persistent error log, and the offending files are
truncated to 0 bytes. If the CF utilization is still >= 80% after the truncation, a warning message is sent to all active Telnet
sessions, the console, and the persistent error log. The persistent error log will contain the CF utilization percentage as well as
a warning message. Other potential offending files are checked and truncated as needed. In addition to these steps, a new file
system will be used on each Fabric OS 4.2 based switch. This will allow for improved space management and file handling.

7.5. Guidelines for Command Usage

This section discusses features and functions within the Fabric OS family. Many commands and the usage will be discussed.
Most features and functions apply to Fabric OS 3.1/4.1 or greater only, as noted.
7.5.1. Pathinfo
Note: Pathinfo is available in Fabric OS 2.6.2, 3.1.2, 4.1.1 and higher.
The pathinfo command provides traceroute functionality for the SAN. It allows the paths to be discovered and viewed in a
usable manner. The previous method of tracing paths used urouteshow.
The fabric topology information is actually known at every switch, however FSPF routing is known only locally; there is no
global, end-to-end routing table. In this context, end-to-end refers to a source switch with connected E_Ports and a destination
switch also with connected E_Ports and connected devices. In general, each switch in the fabric contains a table of routes that
enable frames to be forwarded to adjoining switches. If desired, an end-to-end routing table can be constructed by logging into
every switch and reading the local routing tables. Pathinfo simplifies this task by providing a flexible command line and an
interactive menu. This interactive menu is displayed by default if no arguments are supplied.
PathInfo provides additional routing information including.
• Destination port state
• Link statistics for every hop from source to destination
• Link utilization for each hop from source to destination
7.5.1.1. PathInfo Examples

As designed, pathinfo is intended to gather information on a specific data stream, not the entire fabric. The simplest
example uses a destination domain as an argument. When used, this will provide routing information from the embedded port
on the local switch to the embedded port on the remote switch domain. The example below shows the command and expected
output for the destination domain of 32. Each hop is shown along the way to the destination.
ess031:admin> pathinfo 32
Target port is Embedded
Hop In Port Domain ID (Name) Out Port BW Cost

---------------------------------------------------------
0 E 31 (ess031) 9 1G 1000
1 23 93 (ess093) 5 2G 500
2 6 32 (ESS032) E - -
value = 3 = 0x3

Another common usage is to gather the routing information from a source port on a local switch to a destination port on a
remote switch. This requires three arguments; the source domain, the source port and the destination port. To use the
embedded port, use -1. The example below uses a source domain of 32, it’s embedded port and a destination port of 13. To
determine more complex pathing, it is recommended to use a diagram of the fabric with labeled ISL connections and device
attachments. This allows arguments to be quickly determined. If the appropriate information is not entered properly, pathinfo
may provide error messages.
ess031:admin> pathinfo 32, -1, 13
Target port is F_Port

---------------------------------------------------------
0 E 31 (ess031) 9 1G 1000
1 23 93 (ess093) 5 2G 500
2 6 32 (ESS032) 13 1G -
value = 3 = 0x3
An example of this is shown next. Note that if no devices are attached to the destination port being used as the argument, the
error message displayed is target port not active. Normally this does not mean that no frames are being passed to the device or
that it is not online, it simply means no device is attached. If a device is attached, this may mean the device is not online.
ess031:admin> pathinfo 32, -1, 14
Target port not active

---------------------------------------------------------
0 E 31 (ess031) 9 1G 1000
1 23 93 (ess093) 5 2G 500
2 6 32 (ESS032) 14 1G -
Guideline: For determining complex path information quickly and easily, create or use a diagram of the SAN fabric
with labeled ISL connections and device locations. This simplifies the process of gathering the
appropriate pathinfo arguments. Use -1 as the argument for an embedded switch port.
This next example shows how to display the return path. Use the –r option as shown. Depending on the fabric topology, it may
not be the same as the outbound path. The following example shows that the reverse path in this case is equivalent.
ess031:admin> pathinfo "-r", 32, -1, 13
Target port is F_Port

---------------------------------------------------------
0 E 31 (ess031) 9 1G 1000
1 23 93 (ess093) 5 2G 500
2 6 32 (ESS032) 13 1G -
Reverse path
3 13 32 (ESS032) 6 2G 500
4 5 93 (ess093) 23 1G 1000
5 9 31 (ess031) E - -
value = 5 = 0x5

If no parameters are given, the PathInfo command works in interactive mode. The menu is displayed much like the configure
command. Interactive mode allows more parameters to be specified. The choices are shown below with default values.
• max hops (default = 25)
• domain (required)
• source port (default = embedded)
• destination port (default = embedded)
• enable basic statistics (default = no)
• enable extended statistics (default = no)
• trace reverse path (default = no)
• source route (default = no)
• strict source routing (may be specified if source route enabled)
• timeout (default = 5 seconds)
The interactive mode menu is shown with various choices being made are shown in the following output:
ess031:admin> pathinfo
Max hops: (1..127) [25]

Domain: (1..239) [-1] 32
Source port: (0..15) [-1]
Destination port: (0..255) [-1] 13
Basic stats (yes, y, no, n): [no] y
Extended stats (yes, y, no, n): [no] y
Trace reverse path (yes, y, no, n): [no] y
Source route (yes, y, no, n): [no] n
Timeout: (1..30) [5]
Pathinfo can also be used in advanced mode to display basic statistics. The following example shows partial output with
extended statistics for one hop.
---------------------------------------------------------
1 23 93 (ess093) 5 2G 500
Port 23 5
Tx Rx Tx Rx
-----------------------------------------------
F/s (1s) 0 0 0 0
F/s (64s) 0 0 0 0
Words 9270 8003 9296 9741
Frames 435 450 544 535
Errors - 0 - 0
7.5.2. Wwncardshow
This root level command provides information about information stored in the World Wide Name Card found on the
SilkWorm 12000 and 24000, thus it only works on Fabric OS 4.2 or greater. wwncardshow only accepts IPdata as a parameter.
When executed with this parameter it will only show the IP configuration stored in the WWN Card.
Note: wwncardshow is available only in Fabric OS 4.2 and higher.
Note: Currently this command can be executed only when logged on as root.

As shown below, the command shows IP parameters stored in the WWN card.
sw12000_1:root> wwnCardShow ipdata
++ Wwn Card IP Data ++

Type Num Field Address Mask Cfg/Zone
---------------------------------------------------------
CP 0 Eth IP: 192.168.166.172 255.255.255.0
CP 1 Eth IP: 192.168.166.91 255.255.255.0
Chassis GW IP: 192.168.166.1
LicID: 10:00:00:60:69:80:02:3c
Name: Silkworm12000 Gen#: 0/0
Sw 0 Eth IP: 192.168.166.178 255.255.255.0

FC IP: 0.0.0.0 0.0.0.0
GW IP: 0.0.0.0
WWN: 10:00:00:60:69:80:02:3c
Name: sw12000_1 Gen#: 183/0
Sw 1 Eth IP: 192.168.166.179 255.255.255.0

FC IP: 0.0.0.0 0.0.0.0
GW IP: 0.0.0.0
WWN: 10:00:00:60:69:80:02:3d
Name: sw12000_1 Gen#: 0/0
7.5.3. Cfgtransabort
Note: cfgtransabort is available in Fabric OS 2.6.2, 3.1.2 and 4.2 and higher.
With larger fabrics, zoning configuration changes may take a period of time to complete. At certain times, it may be desired to
abort the changes from occurring. This is achieved with cfgtransabort. This is new to Fabric OS 4.2. To use this
command, the transaction id must be known. This is gathered from the cfgtransshow command. A typical sequence is
shown below:
switch:admin> cfgtranshow
++Outstanding Transactions++
23426
switch:admin> cfgtransabort 23426
7.5.4. Sfpshow with -all Option

Note: sfpshow -all is only available in Fabric OS 4.2 and higher.
Fabric OS 4.2 have included enhancements to the sfpshow command. Fabric OS 2.x executes on switches use the older GBICs
and not SFPs, so this command does not apply. Fabric OS 3.1.2 does not have this option. sfpshow -all displays
extensive information including connector type, supported baud rate, encoding scheme, transceiver information, and vendor
information. The output from this command provides helpful online documentation.
Guideline: It is not recommended to issue this command for troubleshooting. It does however provide great online
documentation for support purposes. Because the output is so voluminous, it is helpful to capture
sfpshow -all to a text file and perform searches on the appropriate information.

7.5.5. Quietmode
Note: Quietmode is available in Fabric OS 2.6.2, 3.1.2 and 4.2 and higher.
Quietmode has been used on Fabric OS 2.x and 3.x for quite some time. This command suppresses output from various
Fabric OS processes and daemons. To turn off quietmode, use an argument of 0. To turn it on, use an argument of 1. No
argument displays the current setting. This command is useful if large amounts of output is being displayed to the screen while
commands are being issued. The Fabric OS 4.2 implementation is similar to, but not identical to, the Fabric OS 2.x and 3.x
quietmode commands. Unlike Fabric OS 2.x or 3.x, some error and debug messages may not be suppressed. The console log
output will not be suppressed. One minor change has occurred in v2.6.2 and v3.1.2. For these Fabric OS versions, syslog is no
longer suppressed in quietmode.
Guideline: Use quietmode when output of Fabric OS processes are needed. If being used to suppress Fabric
Watch messages, be aware that Fabric Watch can be tuned to provide the right level of output. This is
discussed in Chapter 13, Brocade Fabric Watch Overview. Also, refer to the Fabric Watch Guidelines
technical guide for details.
Sample output of quietmode is shown below.

switch:admin> quietmode
quietMode: OFF
Usage: quietMode 0|1

0: to turn it off
1: to turn it on
7.5.6. Nscamshow
Nscamshow shows the non-local device name server registration on all of the other switches in the fabric. This is useful to
identify other devices on other switches. The count variable shows the number of devices that are registered. Note that the core
switch, int63 in the example below, does not have any name server entries, as it only has other switches attached to it.
Guideline: To view all devices registered with each local name server use nscamshow on a core switch (with no
attached devices) in a Core/Edge fabric. For other topologies, use nsshow to view all device
registrations with the local name server in addition to nscamshow. Normally all attached hosts, storage
and other devices such as in-band storage management stations, SCSI/FC bridges, tape drives will be
shown.
int170:admin> nscamshow
Switch entry for 63
state rev owner
known v410 0xfffcaa
No entry is found!
Switch entry for 75

state rev owner
known v260 0xfffcaa
N 4b0f00; 3;20:05:00:a0:b8:07:5d:c7;20:04:00:a0:b8:07:5d:c6;
FC4s: FCP
Fabric Port Name: 20:0f:00:60:69:10:10:9d

Switch entry for 88

state rev owner
known v310 0xfffcaa
N 580800; 2,3;10:00:00:00:c9:29:04:8f;20:00:00:00:c9:29:04:8f;
FC4s: FCP
Switch entry for 154

state rev owner
known v260 0xfffcaa
N 9a0800; 2,3;10:00:00:00:c9:24:f6:5d;20:00:00:00:c9:24:f6:5d;
FC4s: FCP
Fabric Port Name: 20:08:00:60:69:10:68:fa
N 9a0a00; 3;50:06:0b:00:00:0a:48:4a;50:06:0b:00:00:0a:48:4b;
FC4s: FCP
Fabric Port Name: 20:0a:00:60:69:10:68:fa
----output truncated---
7.5.7. Portloginshow
The command portloginshow displays the Fibre Channel services for which a device has registered. In the command
output, there will be one login record per service. As shown in the example, which in this case is an HBA, there are two
entries. One is for the FLOGI to the well-known address FFFFFE, which is the Fabric Port Login service for fabric
registration, (that provides the 24-bit PID address). The other is for the PLOGI to the well-known address FFFFFC, which is
the name server.
int195:admin> portloginshow 6
Type PID World Wide Name credit df_sz cos
-----------------------------------------------------
fe c30600 10:00:00:00:c9:24:f5:f9 64 2048 c scr=3
fc c30600 10:00:00:00:c9:24:f5:f9 12 2048 c c2sema=0x100ba770
The credit on the first entry is in reference to BB_Credit (buffer-to-buffer credit), on the second it references the EE_Credit
(end-to-end credit). COS represents the class of service a bit map column that is displayed in hex. The SCR is in reference to a
state change registration with a class of 3.
This next example shows a different HBA. The df_size refers to the FC maximum payload size, for this HBA it is 2048 bytes.
For all HBAs, this is a typical value.
sialab90:admin> portloginshow 8
-----------------------------------------------------
fe 5a0800 10:00:00:00:c9:2b:4f:75 16 2048 c scr=3
This command is also useful to display all of the devices that are logged into a port. For JBODs, which are Fibre Channel loop
devices, this is invaluable as each PID (with the AL_PA) is shown. In the example, a JBOD is attached to port 15 on a
SilkWorm 3800 with Fabric OS 3.1. Note that the AL_PA and port WWNs are in a tabular list and thus are easily identified.
Df_size refers to the maximum Fibre Channel payload size, for the JBOD it is 2112 bytes, the largest possible.
int170:admin> portloginshow 15
-----------------------------------------------------
ff aa0fef 22:00:00:20:37:15:1f:f7 0 2112 8
ff aa0fe8 22:00:00:20:37:e6:9a:a3 0 2112 8
ff aa0fe4 22:00:00:20:37:e6:9a:84 0 2112 8
ff aa0fe2 22:00:00:20:37:e6:9a:7d 0 2112 8
ff aa0fe1 22:00:00:20:37:e6:99:be 0 2112 8

ff aa0fe0 22:00:00:20:37:15:1f:d6 0 2112 8

ff aa0fdc 22:00:00:20:37:15:1d:51 0 2112 8
ff aa0fda 22:00:00:20:37:15:21:21 0 2112 8
ff aa0fd9 22:00:00:20:37:15:0b:a5 0 2112 8
ff aa0fd6 22:00:00:20:37:e6:9a:6c 0 2112 8
ff aa0fd5 22:00:00:20:37:e6:9a:8c 0 2112 8
ff aa0fd4 22:00:00:20:37:15:0c:0b 0 2112 8
ff aa0fd3 22:00:00:20:37:15:0a:a5 0 2112 8
ff aa0fd2 22:00:00:20:37:15:1a:f9 0 2112 8
ff aa0fd1 22:00:00:20:37:15:1f:d0 0 2112 8
int170:admin>
7.5.8. PortCamShow
Note: PortCamShow is only available in Fabric OS 3.1/4.1 and later.
In some environments, exceeding the total cached limit of 64 SIDs and 512 DIDs per quad may be an issue. If the limit is
exceeded, there is no impact to I/O and the zoning enforcement reverts to session level enforcement. In order to address this
need, Brocade created a command to monitor these values. This command is called PortCamShow. It displays the number of
Fibre Channel SIDs and DIDs used out of the maximum possible. One useful tip is to use nsaliasshow along with
PortCamShow to identify what devices are allowed to see each other. In this example, there is an HBA zoned to a single
storage port by port WWN.
First use nsshow to find out the PID of the HBA and in which quad it is located. For this case, it is 5a0800.
sialab90:admin> nsaliasshow
{
N 5a0800; 2,3;10:00:00:00:c9:2b:4f:75;20:00:00:00:c9:2b:4f:75; na
FC4s: FCP
Fabric Port Name: 20:08:00:60:69:51:0f:2b
Aliases: int124_w2k_B
The Local Name Server has 1 entry }
PortCamShow with no argument displays the used SID/DID on every port as shown.
sialab90:admin> portcamshow
Port 0 to Port 15:
-----------------------------
Port SID used DID used
0 0 0
1 0 0
2 0 0
3 0 0
4 0 0
5 0 0
6 0 0
7 0 0
8 2 2
9 0 0
10 0 0
11 0 0
12 0 0
13 0 0
14 0 0
15 0 0
-----------------------------
Quad ports (SID Free, DID Free)
0-3 (64, 512) 4-7 (64, 512) 8-11 (64, 512) 12-15 (64, 512)
Using the port number as the argument for PortCamShow (in this case 8) shows the number of SID/DID used, 24-bit address
of each device that is in the SID and DID tables and the total SID/DID entries that are free.

sialab90:admin> portcamshow 8
-------------------------------------------------------
Port SID used DID used SID entries DID entries
8 2 2 5a0800 5a0800
c40d00 5a0800
Quad ports (SID Free, DID Free)
8-11 (62, 510)
From the PortCamShow table, note the C40d00 device. To find out what it is, telnet to the switch with domain ID C4 hex
(196). From the output of nsaliasshow, the device is identified.
int196:admin> nsaliasshow
{
N c40d00; 3;20:04:00:a0:b8:07:5d:c7;20:04:00:a0:b8:07:5d:c6; na
FC4s: FCP [LSI INF-01-00 0401]
Fabric Port Name: 20:0d:00:60:69:10:62:2c
Aliases: LSI_B
7.5.9. Portzoneshow
Note: Portzoneshow is only available in Fabric OS 3.1/4.1 and later.
The command Portzoneshow shows how the zoning on the ports is enforced. An example is shown below. From the output
shown in the example, note that port 8 is HARD enforced by WWN. This is shown in bold. Also note that although not able to
be zoned E_Ports are displayed as well.
sialab88:admin> portzoneshow
Local Zoning Port-level information:
PORT: 0 enforcement: E-Port defaultSoftZone: 0 defaultHardZone:0
PORT: 1 enforcement: E-Port defaultSoftZone: 0 defaultHardZone:0
PORT: 2 enforcement: Not Zoned defaultSoftZone: 0 defaultHardZone: 0
PORT: 8 enforcement: HARD WWN defaultSoftZone: 0 defaultHardZone:0
This is the output on a SilkWorm 3900. Note there is some additional information, as the port type is shown. Only partial
output is shown.
int219:root> portzoneshow
PORT: 26 Enforcement: HARD PORT defaultHard: 1 F-port: 1
PORT: 27 Enforcement: HARD PORT defaultHard: 1 F-port: 1
PORT: 28 Not Zoned
PORT: 29 Not Zoned
PORT: 30 Not Zoned
PORT: 31 Not Zoned

7.5.10. Nodefind
Note: Nodefind is only available in Fabric OS 3.1/4.1 and later.
The command Nodefind provides device searching capability within a single fabric. An example is shown below. In order to
find the location of a node, the WWN of interest must be known. The command returns the 24-bit port ID of the device. The
first 8-bits is the switch domain. The second 8-bits is the port number.
sialab89:admin> nodefind "10:00:00:00:c9:30:d0:62"
db1a00
1 device with wwn 10:00:00:00:c9:30:d0:62 }
sialab89:admin>
In this example the Domain ID is db and the port number is 1a. To get effective use of the command, issue fabricshow.
Fabricshow assists with finding the switch the device is on. The switch in the ID column with db as the last byte identifies
the switch. Looking at the Name column (bold) reveals that the switch name where the device is located is int219. The port
number is already known as 1a in hex which is 26 in decimal. Thus the device is on switch int219, port 26.
sialab89:admin> fabricshow
-------------------------------------------------------------------------
64: fffc40 10:00:00:60:69:80:4d:fd 192.168.162.64 0.0.0.0 "int64"
89: fffc59 10:00:00:60:69:51:10:42 192.168.155.89 0.0.0.0 "sialab89"
90: fffc5a 10:00:00:60:69:51:0f:2b 192.168.155.90 0.0.0.0 "sialab90"
195: fffcc3 10:00:00:60:69:10:93:14 192.168.162.195 0.0.0.0 "int195"
196: fffcc4 10:00:00:60:69:10:62:2c 192.168.162.196 0.0.0.0 "int196"
219: fffcdb 10:00:00:60:69:90:04:1a 192.168.162.219 0.0.0.0 >"int219"
The Fabric has 6 switches
7.5.11. Nsaliasshow
Note: Nsaliasshow is only available in Fabric OS 3.1/4.1 and later.
Nsaliasshow is very much like nsshow. The only difference is that the zone alias is now displayed in the output. Be aware
that this is not the port name assigned by portname. The example shows the Alias name assigned through zoning in bold.
{
N 5a0800; 2,3;10:00:00:00:c9:2b:4f:75;20:00:00:00:c9:2b:4f:75; na
FC4s: FCP
Fabric Port Name: 20:08:00:60:69:51:0f:2b
Aliases: int124_w2k_B

7.5.12. Cfgactvshow
Note: Cfgshow (or zoneshow) is only available in Fabric OS 3.1/4.1 and later.
Cfgshow (or zoneshow) can be lengthy. It shows all the alias names and member, zone names and members and every
zoning configuration defined. To enhance the usability of viewing zoning information at the command line, cfgactvshow
was created. Cfgactvshow displays only the active configuration and the zones that are defined that are part of it. As shown
in the example, this provides a summary of which devices are defined in the active zoning configuration.
sialab90:admin> cfgactvshow
cfg: bkup_cfg_B
zone: int121_B
10:00:00:00:c9:24:f5:f9
20:04:00:a0:b8:07:5d:c7
zone: int124_B
10:00:00:00:c9:2b:4f:75
20:04:00:a0:b8:07:5d:c7
zone: int202_B
50:06:0b:00:00:0a:48:84
20:04:00:a0:b8:07:5d:c7
7.5.13. NsZoneMember
Note: Nszonemember is only available in Fabric OS 3.1/4.1 and later.
Nszonemember displays the zoned members, both local and remote, for a particular device. For the purposes of this
command, the port ID identifies the device for which the zoned members are to be identified.
First determine the port ID (PID). To find this information for a particular device on the fabric that is of interest use
nsaliasshow. This will not only show the PID but the device zoning alias name as well. In the example, an HBA was
picked on Fabric A that is in the host int124. The desire was to find out what storage was supposed to be seen.
{
N 580800; 2,3;10:00:00:00:c9:29:04:8f;20:00:00:00:c9:29:04:8f; na
FC4s: FCP
Aliases: int124_w2k_A
Once the PID of the device is known, use it as the argument of Nszonemember. Note that in this case there are no local or
remote storage devices zoned with the HBA.
sialab88:admin> nszonemember "580800"
No local zoned members
No remote zoned members

7.5.14. Port Swapping

Note: Portswapping is only available in Fabric OS 4.1 and later.
A function of Fabric OS 4.1 (and later) is the ability to do port swapping on a local switch in a fabric. For users of Operating
Systems that use the PID for persistent binding, this function adds a lot of value. Now, if for troubleshooting purposes, the port
of the host HBA needs to be moved on the same switch, the cable can be moved without the PID changing. Some hosts require
a reboot for certain operating system commands to be executed if the PID changes. An example will be shown to illustrate the
steps required to perform a swap. This section will provide and overview of port swapping and provide guidelines on its usage.
7.5.14.1. Port Swap Functionality Overview

Swapping ports re-assigns the area numbers for the pair ports being swapped. That is, after the swap, port A will get the area
number of port B. In order to accomplish this, both switch ports must be disabled first. When complete, the result of the
Portswap operation is persistent. This means that the swapped area numbers stay swapped across reboots and power cycles.
Portswap information is kept in separate data base and can not be manipulated by editing the switch configuration database.
By default, port swapping is turned off. To turn on port swapping, use portswapenable. This allows port swapping to be
configured. Disabling the ability to swap ports is done with portswapdisable. This does not allow further ports to be
swapped BUT does keep the current swapped area numbers in place. To reset the ports with the original area numbers, port
swap must be done again to undo the configuration. To view the current ports being swapped, use portswapshow. The next
two examples illustrate these points and show how to use the command.
7.5.14.2. Port Swap Examples

1. Enable port swapping with the command portswapenable.
int219:admin> portswapenable
2. Disable the ports that are to be swapped. In this case, the ports are 14 and 15. For the SilkWorm 12000 use physical slot
number/port number. As an example, portdisable 1/14 will disable port 14 on physical slot 1.
int219:admin> portdisable 14
3. Swap the ports with portswap. A few moments pass before the configuration completes.
int219:admin> portswap 14 15
portswap done
4. Check to see if the port area numbers are swapped with portswapshow. Note that the swapping is persistent across
power cycles and switch reboots.
int219:admin> portswapshow
PortSwap is enabled
Port Area
====================
14 15
15 14
This completes the swapping of the ports. The cables can now be moved.
5. The final step is to enable both of the ports. Note that the devices will redo the FLOGI and PLOGI as well as go through
the normal registration process.
int219:admin> portenable 14

To disable the creation of port swapping configurations, use portswapdisable. It is important to note that the original
swap still is in effect as shown with portswapshow. Once again, this configuration is persistent across power cycles and
reboots.
int219:admin> portswapdisable
PortSwap is disabled.
Existing Portswap condition is still effective.
Only future Portswap operations are not allowed.
Port Area
====================
14 15
15 14
7.5.14.3. Undoing the Port Swap

This next procedure shows how to undo the port swap done earlier.
1. Make sure that port swapping is enabled.
int219:admin> portswapenable
2. Disable both ports that are to be re-configured to their original Domain IDs.
3. Do the swap to return to the original area numbers.

int219:admin> portswap 14 15
portswap done
4. Check the swap configuration to verify that the ports now have their original area numbers.
PortSwap is enabled
Port Area
====================
No ports have been swapped
5. Optional, disable port swapping using portswapdisable. This is the default setting for Fabric OS 4.1.
int219:admin> portswapdisable
PortSwap is disabled.
Existing Portswap condition is still effective.
Only future Portswap operations are not allowed.
Port Area
====================
No ports have been swapped
6. Move the cables back to the original locations.

7. Enable each of the affected ports. Once again, the devices will do the appropriate FLOGI and PLOGI to re-establish
themselves with the fabric.
Note: The portswap information is kept in separate database and thus cannot be manipulated by editing the regular switch
configuration database gathered with configupload.

7.5.15. Switch Rebooting Methods

This section will provide a high level overview of methods of rebooting a switch. Table 7-3 summarizes each reboot method
by switch platform. All methods of rebooting are meant to be disruptive.
The command reboot works on all switch platforms. When it is issued, the switch shuts down gracefully and restarts. As it
comes online, multiple POST and self-diagnostic tests are run.
int219:admin> reboot
The command fastboot works on all switch platforms. This is the same as doing a reboot with diagnostics turned off
with diagdisablepost. Diagnostics can be re-enabled with diagenablepost. This reduces the time for it to come
back online.
int219:admin> fastboot
Guideline: Issue the command diagdisablepost if frequent reboots are expected and the switch is not participating
in a production environment. Once the switch is placed in a production environment, it is recommended
to enable POST (Power On Self Test) by issuing the command diagpostenable.
7.5.15.1. Switch Reboot for the SilkWorm 12000 and 24000

The command, switchreboot is used to reboot a single logical switch image and thus will always be disruptive. The
command switchreboot was designed to clean up and restart all Fabric OS applications. When running this command, the
disruption is assumed to have been planned. An example of a switchreboot is shown below.
Note: Only non-bladed switches that run Fabric OS 4.x support the reboot and fastboot commands.
int64:admin> switchreboot
int64:admin> Selecting i2c bus...Done.
Stopping all switch daemons...Done.
Releasing i2c bus...Done.
Powering off slot 7...Done.
Checking all slots are powered off....Done.
Cleaning up kernel modules...Done.
Done.
Powering on slot 7...Done.
Checking diagnostics..............................
.......Done.
10 9 8
fabric: Subordinate switch
fabric: Reconfiguration due to Fabric Merge(port 47)
fabric: Reconfiguring at Tue Mar 25 06:48:15 2003
5 4 3 2 1
fabric: Domain 64

Table 7-3 Reboot Methods
Switch Model Fabric OS Version Reboot Command

SilkWorm 2xxx Fabric OS 2.6.1 (and later) reboot
fastboot
SilkWorm 3200 Fabric OS 3.1 (and later) reboot

fastboot

fastboot

fastboot

fastboot

fastboot

fastboot
switchreboot

fastboot
switchreboot
NOTE: The commands reboot and fastboot are chassis wide, meaning
both switches get rebooted, for the SilkWorm 12000 and 24000.
Guideline: On the SilkWorm 12000 and SilkWorm 24000, it is recommended that if an Active CP requires a reboot,
that the command hafailover is executed. This makes the process non-disruptive. That way, the
other CP becomes the Active and the new standby can be rebooted non-disruptively.

7.6. Using Fabric OS Troubleshooting Tools

This section will just provide an overview of Brocade Fabric OS troubleshooting tools. Some examples will be given to
demonstrate the practical use. There have been big changes with error logging and the use of portlogdump. For details and
examples for setting up persistent error logs and the uses of portlogdump, please see the Brocade Fabric OS Procedures
Guide (part number: 53-0000518-02).
7.6.1. Supportshow Command Groups
Note: Supportshow command groups are only available in Fabric OS 3.1/4.1 and later.
Supportshow can be run with any combination of 11 command groups. This makes Supportshow more flexible and
easier to capture the desired information.
If required for support reasons, supportshow can always be reconfigured to display additional information. Commands for
the FC Fabric command group 4 are listed in Table 7-4.
Table 7-4 Command Group 4
Command Group 4: FC Fabric
fabricShow qlShow
islShow cfgShow
trunkShow fabStatsShow
topologyShow fabLogDump
faShow
This section will provide some guidelines on setting up supportshow command groups and make a recommendation on
which ones to use. By default, eight command groups are enabled for supportshow. These are shown by
supportshowcfgshow output below. This command can be run as admin or root. Changes to the supportshow
configuration can only be made as the root user.
int219:root> supportshowcfgshow
os enabled
exception enabled
port enabled
fabric enabled
services enabled
security enabled
network enabled
portlog enabled
system enabled
extend disabled
filter disabled
perfmon disabled
Figure 7-1 supportshowcfgshow output

If you have the root password consider configuring supportshow to disable the following groups. For general information
about the fabric and the switch supportshow is running on, these groups are really all that is needed. For most SAN
administrative troubleshooting cases, the data provided by the remaining groups will do.
int219:root> supportshowcfgdisable "os"
Config update Succeeded
int219:root> supportshowcfgdisable "port"
int219:root> supportshowcfgdisable "security"
int219:root> supportshowcfgdisable "portlog"
int219:root> supportshowcfgdisable "network"
Note: If using Secure Fabric OS, do not disable the security command group as shown. Even the remaining groups will
generate a lot of output.
Finally, verify the setting with supportshowcfgshow. The output that should be seen is shown in the following example.
int219:root> supportshowcfgshow
os disabled
exception enabled
port disabled
fabric enabled
services enabled
security disabled
network disabled
portlog disabled
system enabled
extend disabled
filter disabled
perfmon disabled
int219:root>
Figure 7-2 Supportshowcfgshow Command Output
To clarify the groups being used, please refer to Table 7-5. The security group is for Secure Fabric OS support information and
it is group 6.
Table 7-5 Group Name and Number
Group Name Group Number

exception 2
fabric 4
services 5
security 6
system 10

Table 7-6 is a summary of the supportshow configuration commands and the function they perform. Once again, only the
root user is allowed to make changes.
Table 7-6 Supportshow Configuration Commands
Supportshow Configuration Function

Command
supportShowCfgShow Displays list of command groups and whether they are enabled.
supportShowCfgEnable Allows root user to enable a single command group
supportShowCfgDisable Allows root user to disable a single command group
For all supportShow output, no matter how its configured, there are certain commands that will always be executed. These
are, in the order of execution date, version and supportshowcfgshow.
Enabling and disabling is persistent except for filter and extended groups. This is not a concern since those two command
groups are rarely used. Two command groups have only one member. The exception group has errdump. The portlog group
only has portlogdump.
Note: Supportshow for Fabric OS 4.2 and later, includes output for FICON enabled switches, enhanced portlogdump
output, and portswapshow and dbgshow have been added to the System Group. For usage guidelines and a
comprehensive discussion of supportshow for recent versions of Fabric OS, refer to the Supportshow Reference
Guide. No substantial changes have taken place for Fabric OS 2.6.2 or 3.1.2.
7.6.2. Track Changes

Note: Track changes groups are only available in Fabric OS 3.1/4.1 and later.
The change management tool trackChangesSet allows SAN administrator to track successful and unsuccessful logins,
logouts and configfile changes via a log file or SNMP on a local switch. It is disabled by default and must be admistratively
turned on by trackChangesSet. Refer to 7.6.2. Track Changes on page 7-22.
This example shows trackchangesshow. It displays the default setting.
int170:admin> trackChangesShow
Track changes status: OFF
Track changes generate SNMP-TRAP: NO
The example shows how to turn on track changes but not use SNMP to send traps.
int170:admin> trackChangesSet 1, 0
0x102d7450 (tShell): Mar 28 04:12:08
INFO TRACK-TRACK_ON, 4, Track-changes on
0x102d7450 (tShell): Mar 28 04:12:11
INFO TRACK-CONFIG_CHANGE, 4, Config file change from task:tShell
Now trackchangesshow now shows the logging capability enabled. Note that SNMP is still turned off.
int170:admin> trackchangesshow
Track changes status: ON
Track changes generate SNMP-TRAP: NO
With failed logins, the information is logged to the screen after the unsuccessful login as shown.
Login incorrect

0x102d7450 (tShell): Mar 28 04:23:50
INFO TRACK-FAILED_LOGIN, 4, Unsuccessful login
Use errshow to display the changes. As stated in the Fabric OS Procedures Guide (part number: 53-0000518-02), by default
there are only 1024 entries. By default, the errlog is cleared after a reboot. With Fabric OS 3.1 and 4.1, this log can be
expand to 2048 entries maximum and set to be persistent.
int170:admin> errshow
Error 27
--------
0x102d7450 (tShell): Mar 28 04:25:31
INFO TRACK-LOGIN, 4, Successful login
Type <CR> to continue, Q<CR> to stop:
Error 26
--------
0x102d7450 (tShell): Mar 28 04:23:50
INFO TRACK-FAILED_LOGIN, 4, Unsuccessful login
Error 25
--------
0x102d7450 (tShell): Mar 28 04:23:50
INFO SEC-SECVIOL_LOGIN, 4, Security violation: Login failure attempt via TEL
NET. Peer IP: 192.168.162.211
Error 15
--------
0x102d7450 (tShell): Mar 28 04:20:02
INFO TRACK-LOGIN, 4, Successful login
Error 14
--------
0x102d7450 (tShell): Mar 28 04:19:53
INFO TRACK-LOGOUT, 4, Logout
Note: Fabric OS 3.1/4.1 and later allow three unsuccessful logins before disconnecting the current telnet session.
7.6.3. Persistently Disabling a Switch or Port

There are four commands to allow for persistently disabling ports or switches. When configured, the state of the switch or port
will remain disabled through power cycles or reboots. Reasons why this may be done include:
• There may be a bad SFP or switch which causes fabric instability. These may need to be brought down temporarily
until a replacement is found.
• Unused ports maybe persistently disabled for security concerns. When in this state, no device or switch will be
allowed to join the fabric on that port.
• To prevent operator error when devices or switches are connected to the wrong port.
To disable a switch persistently, use switchCfgPersistentDisable. After a few moments, the switch is disabled. To
verify, use switchshow to display the current state. Note that SwitchRole is now Disabled (Persistent). This indicates that
the command has taken effect.
int170:admin> switchCfgPersistentDisable
int170:admin> switchshow
switchName: int170
switchType: 9.1
switchState: Offline
switchMode: Native
switchRole: Disabled (Persistent)
switchDomain: 170 (unconfirmed)
switchId: fffcaa
switchWwn: 10:00:00:60:69:50:10:90

switchBeacon: OFF
Zoning: ON (bkup_cfg_A)
port 0: id N2 In_Sync Disabled
port 2: -- N2 No_Module Disabled
port 4: id N2 No_Light Disabled
Figure 7-3 Disabling a switch
Note that portcfgshow still shows all ports as not disabled. This is fine, since the switch as a whole is disabled.
nt170:admin> portcfgshow
Ports 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC link init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
where AN:AutoNegotiate, ..:OFF, ??:INVALID.
LM:L0.5
Figure 7-4 Portcfgshow Output
To re-enable the switch persistently, use switchCfgPersistentEnable. Note that the fabric must re-configure after it is
brought back online.
int170:admin> switchCfgPersistentEnable
Command in progress
fabric: Domain 170
. . . . . . . . . . . . . . done
Figure 7-5 switchCfgPersistentEnable Output
Note: If the switch is re-enabled with switchenable it will be enabled temporarily. The next power cycle, reboot or
fastboot will cause the switch to be disabled, persistently. This is because the state of the switch is now stored in
the flash non-volatile memory.
Use portCfgPersistentDisable to persistently disable a port. Use portcfgshow to check the status. An example of
this that shows port 7 being disabled persistently is shown next.
int170:admin> portCfgPersistentDisable 7

Ports 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC link init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Persistent Disable .. .. .. .. .. .. .. ON .. .. .. .. .. .. .. ..
ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
LM:L0.5
Figure 7-6 portCfgPersistentDisable Output
To re-enable a port persistently, use portcfgpersistentenable. To temporarily enabled a port, use portenable.
int170:admin> portcfgpersistentenable 7
Ports 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC link init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
LM:L0.5
Note that portcfgdefault will turn off persistent disabling of a port. The command portenable is required to enable
the port. Portcfgdefault will set all other port settings to default values. This sequence is shown in Figure 7-7.
int170:admin> portCfgPersistentDisable 7
int170:admin>
int170:admin> portcfgdefault 1
Ports 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC link init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Figure 7-7 portCfgPersistentDisable Output
Warning: When a portcfgpersistentdisable is done on an enabled E_port, a fabric reconfiguration may occur.
This is the same behavior as the portcfgdisable command. Persistently disabling groups of ports is not
supported. Each port must be persistently disabled individually.

7.6.4. FDMIshow
Note: FDMIshow is only available in Fabric OS 3.1/4.1 and later.
FDMI stands for Fabric Device Management Interface. This function provides the ability to manage the HBA’s in-band using
Fibre Channel. This capability is a result of a Brocade partnership with Emulex. FDMI is supported on all versions of Fabric
OS and is fully backward compatible with all other versions of Fabric OS. There are two commands (fdmishow and
fdmicacheshow) that support this function. FdmiShow displays local FDMI information that include, local HBAs, HBA
port list, HBA attributes and Port attributes. FdmiCacheShow displays remote FDMI information such as Remote HBAs and
Remote HBA port lists.
To use the FDMI feature both the firmware on the HBA and switch must support it. For the Brocade SilkWorm Switches, the
Fabric OS version must be at 3.1 or higher or 4.1 or higher. If the Emulex HBA does not support FDMI, the following entries
will exist from fdmishow and fdmicacheshow.
int170:admin> fdmishow
Local HBA database contains no entry.
Local Port database contains no entry.
Remote HBA database contains no entry.
Remote Port database contains no entry.
int170:admin> fdmicacheshow
Switch entry for domain 63
state: known
version: v410
wwn: 10:00:00:60:69:80:4d:fc
No devices.
Total count of devices on the switch is 0
Switch entry for domain 75
state: unsupported
version: v260
wwn: 10:00:00:60:69:10:10:9d
No devices.

Section
SAN Management III

Chapter
Introduction to SAN Management
8
8.1. SAN Management Philosophy
The trend of fabrics increasing in switch count, and geographically separate locations of fabrics and switches, has created the
need for centralized, secure, and cost effective SAN management. A SAN consists of switches, host systems and storage
devices. Switch vendors have provided several command and web based tools to manage switch configurations. A few major
storage providers have integrated a basic level of switch management utilities into their existing proprietary management
software packages. However, their primary focus remains on managing storage, not the fabric. As SAN fabrics become larger,
managing a fabric by accessing each individual switch from a command line interface can be an inefficient and time
consuming process. SAN management software must be capable of performing complex tasks including configuring,
maintaining, monitoring and troubleshooting a SAN in a simple and effective manner. Although, there are many SAN
management software packages in the market place, they are somewhat limited in providing SAN level management
functionality. Today’s SAN Management software applications must include:
• SAN Security
• Graphical display of SAN devices
• Easy access to all switches to maintain consistent configurations
• Real time self-monitoring status and advance warning
• Real time critical fabric event detection and notification
• Fault isolation and corrective action
• Analyze long-term behavioral trends
SAN security is a crucial issue that must be addressed by ensuring that only a select group of authorized users are given access
to make SAN configuration changes in a controlled manner. SAN access can be secured by implementing secure policies at the
system as well as the Fabric OS level.
Graphical display of a SAN provides a quick view of its components and their relationship to one another. The device type and
in some cases the status of a device should be determined by the device icon coloring scheme. It also simplifies the SAN
device access process utilizing the point and click method.
Maintaining a consistent configuration within the fabric, and across fabrics, requires performing a variety of tasks on more
than one switch in a fabric. For example, upgrading firmware can be a time consuming task if a large number of switches are
involved. An efficient method of upgrading firmware is to discover the switches by logging into a single switch and then
initiating the firmware download procedure on a selected group of switches.
A real time self-monitoring status and advanced warning can be very helpful to maintain the health of the SAN by avoiding
costly down time in an enterprise environment. In a redundant system, the failure of a FRU may not be necessarily a disastrous
event, however the failure does increase the vulnerability of the overall system. An advanced warning helps the administrator
plan corrective action in time to restore redundancy.
A fabric event critical in nature can disrupt the fabric operation. For example, a failure of an Inter Switch Link (ISL) between
two switches may result in fabric segmentation. When a faulty condition is self-detected and immediate notification is sent by
an appropriate method(s), corrective action may be initiated before the event escalates. Thus a self-monitoring system can
assist a SAN administrator minimize disruption time.
Diagnostic utilities are desirable to identify a failing component and possible root cause of the failure. A diagnostic utility
should be able to run in the background without disturbing the fabric operation. When implemented, it can minimize the
maintenance time significantly.
Analyzing behavior patterns on a switch port for an extended period can expose a deficiency and/or anomaly affecting the
overall throughput performance of the SAN. Detailed information must be gathered over a period of time, using an application
which can probe the switch on a periodic basis.

8 Introduction to SAN Management
8.2. SAN Management Scope

The SAN Management administrative scope can be broken down into three distinct areas; Fabric, Switch and Port. Fabric
administration tasks typically involve parameters that are applied to many switches at once. Switch tasks generally involve
environmental monitoring. Port level tasks generally involve performance monitoring, setting up port level definitions etc. Of
course, there is overlap between these areas.
Figure 8-1 summarizes the SAN Management at the Fabric, Switch and Port level and the various administrative tasks
associated with them. In each area, a recommended Brocade management tool for these tasks is shown at the top of each box.
Fabric Adm inistration (Fabric Manager 4.1.x)
Fabric discovery (Subnet scan)

SAN Element Grouping
Fabric Login (Global access)
Firmw are /configuration dow n load
Firmw are dow nload to FDMI capable Host Bus Adapter
Sequence sw itch reboot
Fabric Merge check
Fabric Checking
Complete Fabric backup and compare
License key management
Synchronize Time and Date
Fabric Event monitoring
ISL Checking
Fabric refreshing
Call home support
Secure Fabric OS policy management
Sw itch Adm inistration (Web Tools or CLI)
Zone Administration
Sw itch information
Netw orkconfiguration
Firmw are/Configuration upload/ dow nload
SNMP setup
License Administration
Fabric parameter configuration
Routing, DLS and IOD setup
Port Adm inistration
Port Configuration setup

Extended Fabric setup
Trunk information
TelnetCom m and Line Access
Brocade Licensed products

Fabric Watch
Advance Perform ance Monitoring
Figure 8-1 Brocade SAN administration

Introduction to SAN Management 8
8.3. Brocade SAN Management Tools & Interfaces

While other storage management software packages offer basic features of switch and port level management, Brocade's
approach is SAN management centric. The main emphasis is to simplify the entire SAN management process by simplifying
the three levels of management for Brocade switches from a centralized management server. Brocade has developed and
refined a rich command set for Web Tools. Licensed fabric health monitoring tools, including Brocade Fabric Watch and
Advanced Performance Monitoring, were introduced to SAN administrators to assist with SAN monitoring and improving the
operating environment. As SAN configurations grew larger and more complex, managing them using multiple management
tools became very challenging. Brocade Fabric Manager was introduced to assist users, by simplifying the entire management
process by providing centralized management. Brocade Fabric Manger is capable of managing complex tasks at the fabric,
switch, and port levels. In addition, access to Brocade Web Tools, Fabric Watch, and Advanced Performance Monitoring is
made available via the Fabric Manager interface. This allows for complete SAN management from a single user interface.
Brocade SAN management products include:
• Brocade Fabric Manager 4.1.x
• Brocade Advanced Web Tools (HTTP)
• Command Line (Telnet)
• Event management via SNMP
• Brocade Fabric access API
• Brocade Advanced Performance Monitoring
• Brocade Fabric Watch
API T ools
Fabric Manager
Fabric-A Fabric-B Fabric-C
Web T ools
Fabric Watch SNMP
Advance performance monitoring
Figure 8-2 Brocade SAN Management Tools
Brocade SilkWorm series switches offer industry standard interfaces. Management tools have been developed using these
interfaces. These interfaces are briefly discussed here.

8.3.1. Brocade Fabric Manager

Fabric Manager provides a graphical interface that enables the administrator to monitor and manage an entire fabric from a
standard workstation. It provides a highly scalable Java-based application capable of managing multiple switches and multiple
fabrics in real -time. Fabric Manager provides high-level information about all switches in the fabric, additional switch
information can be obtained by launching Brocade Web Tools. Fabric Manager models the fabric based on an interface called
Model.
8.3.2. Brocade Advanced Web Tools (HTTP)

Brocade Web Tools provides a java based graphical interface that enables an administrator to monitor and manage the entire
fabric from a standard workstation by establishing a Hypertext Transport Protocol (HTTP) server connection on one of the
switches in a fabric. HTTP is based on request and response messages. After securing a connection, the client sends a request
for a particular resource and the HTTP sever responds by returning the requested resource or a reasoning message. Web Tools
is an optionally licensed product that runs on the Fabric OS. Brocade switches without the Web Tools license must be managed
through CLI or other management applications.
8.3.3. Command Line (Telnet)

The very first and basic form of switch management is command line interface (CLI) that can be established via a Telnet
session. It is an efficient and cost effective method of managing an individual switch or a small fabric of four switches or less.
Each switch in the fabric must be accessed and managed individually. The tasks can be overwhelming and time consuming
when managing a large number of switches in the fabric. CLI does not offer management beyond a single switch level,
therefore it is not an ideal solution to manage medium to large fabrics.
8.3.4. SNMP
SNMP is a standard protocol that SNMP manager uses to monitor their Fibre Channel SAN. SNMP manager is responsible for
managing SNMP agents remotely. A manager is capable of querying agents, setting variables, getting responses and
acknowledging asynchronous events (traps) from agents in the network. All brocade fabric switches support SNMPv1C and
capable of responding to SNMPv2C commands using SNMPv1 community names.
8.3.5. Brocade API Scripting Toolkit

The Brocade API Scripting Toolkit consists of a host-based library that uses the Brocade API and interfaces the application to
one or more SAN fabrics. The application connects to each fabric either out-of-band over a TCP/IP connection, or in-band
using an IP capable Fibre Channel host bus adapter. The application function calls are interpreted by the library, which in turn
issues commands to the fabric via the gateway switch. The information returned is then received by the library, formatted and
presented to the application. With the Brocade API Scripting Toolkit, the Perl based application scripts can be used for
discovering switches, managing zones, and retrieving switch configuration information.

Introduction to SAN Management 8
8.4. Tool Selection Guidelines

As mentioned previously, there are several management tools available to end-users. Although Brocade Fabric Manager
provides a complete set of management functionality, consider selecting an appropriate tool to suit your fabric environment
and budget. Table 8-1 provides rough guidelines based on the number of switches and the Brocade licensed products that need
to be monitored.
Table 8-1 Brocade Management Product Usage Guidelines

Brocade Licensed Management Usage Guideline
Product
Brocade Fabric Manager Single or multiple fabrics with a large number of switches in an enterprise envi-
ronment. Ideal for managing Fabric level management functions on a group of
switches.
Brocade Advanced Web Tools A user friendly interface for managing an individual switch which has a Web
Tools license installed. Ideal for managing small fabrics.
Telnet (CLI) Ideal for managing basic and advanced features on an individual fabric or
switch.
Brocade Fabric Access API Application based
SNMP Third party management software packages


Chapter
Brocade Fabric Manager 4.1.x
9
Brocade Fabric Manager is a java-based fabric-centric monitor and management application that provides a central point of
control. In addition, Fabric Manager is tightly coupled with Brocade Advanced Web Tools, Fabric Watch and Advanced
Performance Monitoring applications. The graphical interface simplifies administering tasks at the fabric, switch and port
levels in a medium to large size Brocade SAN environment.
The overall goal is to deliver an intelligent scalable management of storage fabric resources that enables customers to reduce
infrastructure cost and improve efficiency of Brocade SAN management function.
9.1. Key Features

Brocade Fabric Manager 4.1.x key features are briefly described in this section.
SAN Discovery
• Discover all fabric switches.
• Discovery of all switches and fabric via subnet scanning.
• Device information includes FDMI data if available.
• SAN Element display by WWN, Name, Domain or IP address.
• Tree view of SAN elements organized by fabrics, switch/port groups.
• Drill-down ability allows detail information on selected element.
SAN Element Logical Grouping
• SAN Elements can be placed in one or more groups to manage them as a unit.
• Simplify time consuming repetitive tasks by executing on a predefined group of elements.
• Groups can be exported and imported for the purpose of sharing across multiple Fabric Manager client instances.
Firmware/Configuration Download
• Firmware download on one or more switches of the fabric simultaneously.
• Allows configuration upload and download across fabrics to all Brocade switches.
Firmware Download to FDMI capable Host Bus Adapter
• In-band firmware download to FDMI capable HBA.
Sequence Switch Reboot
• Define a group of switches that need to be rebooted.
• Define a reboot sequence order.
• Allows programmable delay between switch reboot to enable a pre-defined reboot sequence in the fabric.
Fabric Merge Check
• Pre-qualify two independent fabrics for merging.
• Checks current configurations of both fabrics for compatibility.
• Presents conflicting information across zoning, security and other areas that can prevent fabrics from merging.

9 Brocade Fabric Manager 4.1.x
Fabric Checking
• Retrieve and save the current state of the fabric.
• Retrieve and save the ISL information including trunking.
• Detect and display the differences between the current and the saved states.
• The Fabric Manager database preserves application specific data such as fabric/switch/port/group names, group
memberships, re-boot sequences and existing license keys across sessions. The data is made available for subsequent
Fabric Manager sessions.
Complete Fabric Backup and Compare
• The Fabric Manager backup process creates a complete backup file of the existing fabric configuration that includes
switch configuration, license keys, fabric configuration etc.
• Compare with a baseline saved file to identify any change.
Event Monitoring
• Monitor ISL connectivity, Fabric and Switch health status.
• Displays Error log events and propagates status upwards within user-defined groups.
• Displays Fabric Watch events.
Call Home support
• Client side-GUI allows user to configure the Call Home trigger conditions.
• Server monitors a user-configurable set of switches for changes/events in order to send a request for action based on
configured parameters.
Secure Fabric OS Policy Management
• This feature is enabled only if Fabric is operating in Secure Mode using Brocade Secure Fabric OS 4.1.0 or higher.
• The fabric security is managed by defining Secure policies. Policies can be created or modified when operating in
secure mode.
Switch/Port Administration
• Invokes Advanced Web Tools from a specific switch to perform switch/port level management.
• Telnet window provides a command line interface.
• Fabric switches and ports can be renamed.
• Fabric switches and ports can be enabled/disabled.
• Switch login credentials are saved for a specific session. Maintain session once authentication with a switch has
succeeded. The same credentials can be used across multiple switches.
• Time synchronization across all fabric switches.

Brocade Fabric Manager 4.1.x 9
9.2. Fabric Manager 4.1 Installation

Please refer to Brocade Fabric Manager User's Guide v4.1 for step-by-step installation, configuration and instructions.
Brocade Fabric Manager v4.1.x Release Notes also include valuable information about features and important notices.
A Client/Server architecture based Fabric Manager 4.1 offers the following installation options.
• Fabric Manager Server only
• Fabric Manager Client only
• Fabric Manager Server and Client (Same system)
Client workstations
Fabric ManagerServer
Fabric-A Fabric-B RemoteFabric-C
Figure 9-1 Client/Server Configuration

Figure 9-1 illustrates a Client/Server configuration where each component is installed separately. The server is accessible via
one or more local or remote client stations as long as a valid IP connection remains in place. A Fabric Manager Server can
support up to five Fabric Manager Clients. This Client/Server approach is very desirable as clients can be located remotely.
Installing the Server and Client components of Fabric Manager on the same host is a cost effective approach suitable for
managing small to medium size fabric(s). However in an enterprise environment managing multiple fabrics consisting of a
large number of switches, a Client/Server configuration is recommended.
Note: Each Fabric Manager Server can support up to five Fabric Manager Clients.

9.3. Installation Requirements

The Fabric Manager 4.1 has the following listed hardware and software requirements.
Table 9-1 Host System Configuration

Operating System Windows 2000, SP4 (Client and Server)
Windows XP (Client and Server)
Windows 2003 (Client and Server)
Windows NT (Client only)
Solaris 2.8, 2.9 (Client and Server)
Solaris 2.7 (Client only)
CPU and Memory 256 -512 MB RAM (Recommended 512 MB, CPU 1.5 GHz)
Disk Space 70 MB
Browser Internet Explorer 5.5 or higher
(Internet Explorer 6.0 Recommended)
Plug-ins Java runtime Environment plug-in V 1.4.2_03 (Recommended)
RAM Minimum 8 MB of video RAM is recommended
Note: When monitoring more than 30 switches while using the Call Home feature it is recommended to install Fabric
Manager Server on a dedicated machine.
Table 9-2 Product S/N and License Key

Procedure Serial Number and License Key Requirement
Installing Fabric Manager Use an existing valid Product Serial number and License key. If none exists,
over an existing 4.1.x version you are requested to enter the S/N and License key
Upgrading Fabric Manager Installer requests Product Serial number and license key re-entry. (Retrieve
from version 4.0.x or earlier S/N and license key prior to upgrade)
New Installation full version Product Serial number and License key is required
Evaluation version Not required for initial evaluation period
Table 9-3 Fabric Manager configuration parameters
Client /Server Setup On Same Host Server Only Client Only
Windows Domain Name (Note 1) Windows Domain Name -

Starting port number (Note 2) Starting port number Starting port number
(default=24600)
Mail SMTP Host Mail SMTP Host -
Email address Email address -
Server IP address (default = - Server IP address
local host)

Note: 1 Fabric Manager server becomes a member of the domain specified during the Fabric Manager server installation. A
DNS server IP address can also be entered here. To find the Domain name to use as the windows authentication domain
that must be specified during installation, open DOS window and enter “set”. The alias USERDOMAIN indicates the
active domain.
Note: 2 The Default starting port number is 24600. It requires six consecutive free ports to work correctly. In this case the
default free port address range is 24600-24606. To change port assignment, make sure six consecutive ports are free.
Document the new starting port number, as it is required during the client installation.
Warnings for Windows Installations
Warning: If the client and server are residing in different domains, both domains must have trust established between them,
or Fabric Manager will not be able to authenticate the client. Please take a precautionary note that the domain is
referenced to the domain name Microsoft uses for authentication, it is not an internet domain such as
corp.mycompany.com.
Warning: In a Windows environment, if the server is not a member of the specified domain, Fabric Manager authentication
will succeed for any user credentials, compromising the security of the Fabric Manager Server. Windows guest
user permissions must be disabled and a domain must be specified. For instructions on disabling guest user
permissions, refer to Microsoft Windows documentation.
Warnings for Solaris Installations
Warning: Solaris: Enter your NIS hostname or IP address and domain name for the Fabric Manager server user
authentication. If the NIS server host name or IP address is not assigned, and no NIS server exists on the same
subnet as the Fabric Manager server, all authentication requests will fail. Valid domain names can include no more
than 67 characters (including: .com, .net, .org at the end). Only alphanumeric values and dashes (-) are allowed.
Spaces and other characters are not permitted.
Other Installation Warnings
Warning: The client polls the fabric information directly, therefore the client must be able to access each switch via Ethernet
IP connection. Make sure the network environment does not have a proxy server or firewall between the client
and server and the switches. If one exists, ensure that the access is properly setup.
Warning: In order to monitor switches for Call Home events, only the server needs IP connectivity to the switches.
Warning: Failure to enter a starting port-number and/or finding the consecutive six ports free will cause server not to start
up correctly.

9.4. Switch Requirements

All switches in the fabric are represented in the main window of Fabric Manager, but only those with an Advanced Web Tools
license can be managed through Fabric Manager.
Table 9-4
Brocade Switches FOS version Brocade Switch licenses
Advanced Fabric Watch Advanced Performance Secure
Web Tools Monitoring Fabric OS
SilkWorm 2000 Series 2.6.0c- 2.6.1 Enable Recommended Optional Optional
SilkWorm 3000 Series 3.0, 3.1.0c-3.1.2 Enable Recommended Optional Optional
SilkWorm 3900, 12000 4.1.0- 4.2 Enable Recommended Optional Optional
SilkWorm 3250, 3850, 24000 4.2.0 Enable Recommended Optional Optional
9.4.1. Configuration Guidelines

Guideline: All switches must have a Brocade Advanced Web Tools license enabled.
Guideline: HTTP (tcp port 80) protocol must be enabled on every switch that needs to be discovered, monitored,
and configured via Fabric Manager. Fabric OS supports a maximum of five simultaneous HTTP
sessions on a switch. The HTTP sessions are leveraged by each copy of Fabric Manager and Web
Tools on a switch.
Guideline: For firmware download specific features to function, TCP/UDP ports 20 and 21 must be free.
Guideline: For Fabric Manager features set time, Security and FDMI to function, TCP/UDP port number 111 (rpc
mapping) and ports 600-1023 must not be blocked by a network firewall or proxy server.
Guideline: Fabric Manager features available on Fabric OS versions 4.2 /3.1.2 may not be available on previous
versions. It is always a good practice to upgrade Brocade SilkWorm switches to the latest Fabric OS
release.
Guideline: To monitor fabric health, a Fabric Watch license must be installed and Fabric Watch parameters must
be setup and enabled correctly on all fabric switches. Please refer to the Brocade Fabric Watch
Guidelines for threshold setup planning.
Guideline: Basic Performance Monitoring does not require any license, however Advanced Performance
Monitoring requires the activation of the Advanced Performance Monitoring license.
Guideline: Secure Fabric Policy management features are only applicable to fabrics configured to operate in
Brocade Secure Fabric mode.

9.5. Fabric Manager Navigation

The Fabric Manager navigation tools are designed to provide an easy to use interface. The view menu lists the various Fabric
Manager views. Use the view menus to navigate Fabric Manager.
FabricElement Discovery
Connectivity display selection
Fabric Manager Function management menu
Information ViewWindow View tabs
SAN Element Window Icon color represents current status T opology view
Figure 9-2 Fabric Manager Window

9.5.1. SAN Element Window

After an initial discovery is performed, discovered SAN elements are displayed in a tree structure hierarchy. For example, a
single SAN fabric consisting of five Brocade SilkWorm switches is discovered when a discovery is performed given the IP
address of 10.64.147.48. The element list option is determine by the ID field. The elements can be listed either by IP, Name,
Domain ID or WWN. The element icon color represents its current status. In a large and complex environment the switch and
port group can be created to enhance the visibility and manage devices as a single entity.
The maintenance tasks that can be performed in SAN Element Window
• Discover a fabric
• Fabric login
• Delete a fabric
• Rename a fabric
• Manually Refresh a Fabric
• Rename a switch
• Rename a port
• Create, Edit, Import or Export switch or port group
9.5.2. Information View Window

Fabric Manager probes the attached devices on discovery and retrieves the relative information. First select a device (fabric,
switch or port) in the SAN Element View window and then select one of the following listed tabs from the Information View
Window. The requested details are then displayed in the Information View window. For example, Figure 9-2 illustrates the
Fabric (IP=10.64.147.48) topology details. The following views in Fabric Manager can be accessed either from the View
menu or from the Information View window.
Table 9-5 Summary of Views
Tabs Information View Widow

Detail Detail information of selected Elements (SAN, Fabric, Switch ports)
Device Port Lists the device ports attached to selected SAN element
Devices Lists the information of all devices, ex; device node name, SCSI inquiry string
Event Display Current status reason and event log of the selected Element
Portgrid Provides the host and storage devices connectivity map
Ports Display detail information of all ports example, name, port type, current status
Summary Provides Summary view of fabric switches or switch ports
Switches Provides the configuration detail on all of the switches
Topology Provides a graphical representation of the fabric elements Fabric Manager monitors.

9.5.3. Customizing View Window

The information in the View window is presented in a table format organized by rows and columns. Each column header
identifies the type of information. An information column can be added, removed or re-arranged to create a meaningful
customize view so that each time a view tab is selected, the relative information is displayed in prescribed format for clarity.
Customizing can be accomplished for a SAN element by selecting the element and the appropriate view tab. Access the view
option table from Edit menu and selecting “display items” from the list. Figure 9-3 is an example of editing the Devices table
for a fabric.
Figure 9-3 Fabric Manager - View Window
9.6. Fabric Manager 4.1.x Features

Apart from assisting SAN administrators performing routine maintenance tasks, Fabric Manager addresses and simplifies
some of the SAN upgrade and topology change related tasks that can be time consuming and overwhelming depending on the
complexity of the existing SAN configuration and size. Fabric Manager features are designed to simplify the entire process of
configuring, maintaining and upgrading a SAN are discussed in this section.
Note: For a complete set of Fabric Manager features please refer to Brocade Fabric Manager User's Guide v4.1.
9.6.1. SAN Discovery (Intelligent Baseline Profiling)

One of the primary tasks of Fabric Manager is to discover physical elements of the SAN and draw an accurate Topology. Once
a SAN topology becomes known, the detail information is obtained by probing each individual element. A SAN administrator
can then easily navigate through this information with the help of Fabric Manager tools. The requested information is logically
displayed in a simple and easy to understand format.
A baseline profiling is consists of:
• Discovering fabric switches
• Discovering of all switches and fabrics via subnet scanning
• SAN Element display by WWN, Name, Domain or IP address
• Tree view of SAN elements organized by fabrics, switch/port groups
• Displaying current health status of SAN elements by icon color.
(Green - healthy, Orange - marginal, Red - down, Gray- un-monitored, Blue - unknown)

9.6.2. SAN Discovery

After launching Fabric Manager, SAN discovery can be initiated from Fabric Manager starting screen either by providing an
IP address of a switch that is part of the SAN in the Address field of the Fabric Manager screen or scanning entire subnet in
case the fabric switch IP address is not available.
To initiate a SAN Discovery:
Set the Address field to a switch IP address (for example: 192.168.11.20) and press Enter.
The discovered fabric is displayed in the SAN Element panel tree. The fabric assumes the name of the switch for which
the IP address is used in the address field.
Figure 9-4 shows the discovered fabric which has assumed the switch name “emc13900A00” (the switch name of IP
address 192.168.11.20). The Fabric Elements can be sorted by name, IP address, or WWN by selecting the appropriate
tab. The fabric can be renamed by the user for clarity.
The Switches tab, located on the Information View window, displays all switches and related information for the selected
fabric.
Figure 9-4 Fabric Discovery by IP Address of a Fabric Switch

9.6.2.1. Subnet Scan
The subnet scan feature is useful when the exact IP address of any switch in the fabric is not available. To initiate a subnet scan
follow the steps below:
1. From the Tools menu, select Subnet scan.
2. Set the IP Address Range field to the first three known bytes (for example: 192.168.11.*) and click the Scan button. The
discovered switches are displayed in the Scan Result field, as shown in Figure 9-5.
Subnet scan search can also be limited to a narrow range of devices. for example,
• 192.168.11. * Device search range 1-254
• 192.168.11.1* Device search range 100-199
• 192.168.11.11* Device search range 110-119
3. Select the fabric or switches by checking the appropriate box then click the Add button.
4. Double click the fabric name to view all switches in the selected fabric.
5. Click the Add button after checking the box to add the selected fabric to the SAN Element display window of Fabric
Manager.
Figure 9-5 Subnet Scan Results

9.6.3. SAN Element Logical Grouping

Logical groups consist of switches or switch ports can be formed for the purpose of managing and monitoring them as a single
entity. Grouping simplifies the change management process by initiating the process uniformly on a defined switch or port
group. One of the primary requirements is that all the switches of a group must have user access with a common user ID and
password. A group can have switches from one or more fabrics. When creating a switch group, first evaluate the functional
requirements of the individual group. For example, if a switch group is created for the purpose of downloading firmware make
sure all switches of the group are accessible (user ID and password) and use the same firmware file.
Consider logical grouping for simplifying the following tasks:
• Monitoring a small group of ports on a switch that has large number of ports
• Firmware downloads on multiple switches that use the same Fabric OS file.
• Fabric-wide license key retrieving from/to a file.
• Fabric Login to a group of switches.
The current status of the group can be viewed anytime by selecting the group name in the SAN element display window and
then selecting Summary view from the Information view window.
The following is a list of suggested groups that can be created for the purpose of performing a specific task that applies to all
switch members of the group.
Switch group name = Project-x Group
A group name “Project-x “can be created to monitor and manage the fabric switches that belongs to this specific project.
Project-x utilizes switches from multiple fabrics (Fabric-A, Fabric-B and Fabric-C). Thus limiting the scope to a small number
of switches that belong to this project only provide visibility and increases efficiency.
Switch group name = Tape-backup Group
This group can be created to monitor the tape backup path status during backup operation by including the switches that
provide connectivity to backup host, storage and tape library. This group provides a specific task oriented monitoring during
the critical backup operation.
Switch group name = Firmware Download group
Firmware update can be a time consuming task in a large fabric configuration when all switches of the fabric require a
firmware upgrade. This process can be simplified by automating a download process on a group of Brocade SilkWorm
switches. The grouping not only saves the time by automating the download process, it also ensure that the entire group is
updated to the same level of OS version. This switch group must meet the following criteria:
• The group member switches must have a common login - User Id and password.
• The group member switches must be using the same Firmware File.
For example:
• SilkWorm 3250, 3850, 3900, 12000 and 24000 use Brocade Fabric OS 4.2.x.
• SilkWorm 3800, 3200 use Brocade Fabric OS 3.1.x
• SilkWorm 2000 Series use Brocade Fabric OS 2.6.x.
Note: Make sure the current firmware version on all switches is qualified for the firmware upgrade. Contact your switch
provider for details.
Guideline: A firmware upgrade from Fabric OS 2.6.x and 3.1.x require the switch to reboot. This will result in a
fabric re-configuration. To prevent more than one fabric from rebooting simultaneously, limit the group
membership to a single fabric.

Guideline: When upgrading a SilkWorm 12000 which is utilizing both domains, only include one domain in a group.
Both CP0 and CP1 gets updated automatically.
Guideline: Fabric OS version 4.1, 4.2 and later support hot code load. Brocade SilkWorm switches using these
Fabric OS versions do not require the switch to reboot after a firmware upgrade.
Switch group name = Core switch Group, Edge switch group

In a large SAN environment utilizing a Core/Edge topology, creating the following switch groups can provide high visibility
focusing on a specific segment of the SAN. For example, creating a Core switch group will limit fabric monitoring to the Core
switch segment of the fabric. The Edge switch group can limit the monitoring status to the host or storage segment. Both of
these groups can also be used in defining a reboot sequence. Reboot a Core switch group before groups containing Edge
switches to minimize the number of fabric disruptions.
switch group name = Long Distance Switch Group Monitor status of switches providing Long Distance connectivity.
Switch group name = MPR Group Monitor status of Brocade SilkWorm Multi-Protocol Routers.
9.6.3.1. Creating a Firmware Download Group for Fabric-A

1. From the Fabric Manager File menu select Groups> Edit Switch Groups.
2. From the Edit Switch Groups window, create a group name (for example: “Firmware 4.x group of fabric-A”).
3. Select the Firmware Download group and move switches that use OS version 4.x to the right pane of the window from
Fabric-A. Click OK when done.
I
Figure 9-6 Creating a switch group for 4.x firmware download and adding members

Fabric-A switches that use Brocade Os version 4.x

Notice from this screen that there are only three switches of Fabric-A that use Firmware file 4.x.
switch name = emc13900A00 SilkWorm 3900 firmware Version = v4.2.0
switch name = hds112kA00sw0 SilkWorm 12000 Logical switch 0 OS version = 4.2.0
switch name = hds112kA00sw1 SilkWorm 12000 Logical switch 1 OS version = 4.2.0*
Note: *On a SilkWorm 12000 with two logical switches only one logical switch needs to be included in the group.
The newly created group name “firmware 4.x group of fabric-A” shows up in the SAN element window under the “switch
group” element. When this group is selected, the menu tabs in the Information View window limit the switch monitoring
display to the group members only, as shown in Figure 9-7. This group includes switches that use the same firmware file, and
therefore can be used to download Fabric OS 4.x as described in section 9.6.5. Firmware Download on page 9-18.
Figure 9-7 Firmware 4.x Group of fabric-A member switches only display

9.6.3.2. Create Port Groups
Monitoring a large number of ports across multiple SilkWorm platforms in a fabric can be a very challenging task. The
grouping of ports within the switch or across fabric switches can provide a more focus monitoring on selected ports. For
example, consider creating the following port groups.
· Storage port Group Monitor Storage device availability status
· E_port Group Monitor Fabric wide E_Port status
· Trunk port GroupMonitor the Trunk port status
· Long Distance port groupMonitor Long Distance port status
· Host Port GroupMonitor Host availability status.
· Project-x port GroupMonitor ports associate only with project-x [host port(s), device port(s), Link ports]
Creating a Trunk port Group
1. Select Edit Port Group from File > Group menu of the Fabric Manager menu bar.
2. From the Edit port Groups window, Create a group name (for example: Trunk port Group Fabric-A).
3. Select the “Trunk port Group Fabric-A” and move member ports from one or more switches to the right pane of the
window.
4. Click OK after all the required ports have been included in the group.
Figure 9-8 Creating a port group name “Trunk port Group Fabric-A”

The newly created port group shows up in the SAN element tree under the Port Group element. The Information view window
ports tab shows the current status of the trunk ports of Fabric-A.
Figure 9-9 “Trunk port Group Fabric-A” port status monitoring
Guideline: Organize switches by functions, switch type, firmware (Fabric OS) version or any selected criteria.
Create groups of similar switches and ports so that they can be can be monitored and configured as a
unit. For example, creating a group of same model type fabric switches running the same Fabric OS,
the updating process can be initiated uniformly on a group basis, thus eliminating the need for accessing
each switch individually.
Note: A switch can appear in multiple groups at the same time. Switches remain in a group even when removed from their
source fabrics from Fabric Manager.
9.6.4. Fabric Login (Global Access Setup)

A global password will allow logging into multiple switches simultaneously eliminating the need for logging into each switch
of the fabric individually. After logging into a switch, Fabric Manager stores your login information and automatically logs
into the switches in the fabric.

Guideline: This time saving and convenient feature must be considered when performing the following Fabric
Manager tasks that impact multiple switches. All Fabric switches that need to be accessed, as a group
must have same user ID and password pre-assigned.
List of group maintenance tasks:

• License key management
• Date/time synchronization
• Baseline configuration upload/download
• Firmware download to switches
• Sequenced reboot
• Firmware download to HBAs
• Fabric compare and merge
Note: In-order to perform any of the tasks listed above on a switch group, the login must be performed first.
Select Fabric Login from the File menu and select the entire fabric or switch group.Supply user ID and password and depress
Apply button. A successful login status means the switches are reachable by the supplied user ID and password parameters.
Figure 9-10 Fabric login on Fabric-A switches

9.6.5. Firmware Download

Perform a Firmware Download with Fabric Manager to concurrently download firmware to a group of switches that use the
same firmware (Fabric OS) file and download procedure. Since a firmware upgrade on switches using Fabric OS 2.6.x and 3.1
require a switch reboot, the switch reboot sequence can also be planned best suited for the current fabric environment.
SilkWorm switches using Fabric OS 4.x support hot code load and do not require reboot.
1. From Tools menu select Firmware download to switches.
2. Select a switch group whose members use the same Firmware file (Fabric OS).
Figure 9-11 shows firmware download on switch group name “Firmware 4.x group of fabric-A” that was created in section
9.6.3. SAN Element Logical Grouping on page 9-12. The group has two switches with the current version of Fabric OS 4.2.0.
The firmware will be upgraded to revision level 4.2.0a.
Figure 9-11 Firmware download on a switch group name “Firmware 4.x group of fabric-A”
3. Provide the Host IP address, user ID and password.
Host IP address of the server to download firmware file from (ex: 192.168.162.107)
Remote User ID = (ex: root)
Password required for FTP: *******
Firmware file = Provide the complete directory path of the firmware file (ex: \import\fw\v4.x)
Select protocol FTP or rhsd

Figure 9-12 Server information to access the firmware v4.2.0a file.

4. Initiate a fabric login from the Fabric Login tab. Supply Fabric Login - user Id and password of the switches and click
Apply. After success fabric login close the window.
Figure 9-13 Fabric login successful completion

5. Start firmware download by clicking Download

The firmware downloading process begins if there is no server file access issue. The intermediate messages are displayed in
the status column showing the progress. A success status is posted upon completion and the color of status column the changes
from blue to green.
Figure 9-14 Firmware Download status blue= in progress, Green=done
Guideline: All switches selected for firmware download must be upgraded to the same version of Fabric OS.
Warning: Verify the current OS version(s) of all switches in the group for backward OS compatibility. There may be a switch
in the group whose current code may not be directly upgradable to the latest version. Contact your switch vendor
for details.
Guideline: All Brocade SilkWorm switches selected to simultaneously reboot must reside on the same fabric.
Guideline: Only a single logical switch (either logical switch 0 or logical switch 1) is permitted when upgrading
firmware on a SilkWorm 12000 chassis with two logical switches. Including one logical switch will
upgrade firmware on both CP0 and CP1.

Guideline: Customize the Switch Reboot Sequence to make the new firmware effective with less interruption. For
example, if you have a Core/Edge fabric configuration, you must reboot the Core switch before the Edge
switches. Thus, when the devices connected to Edge switch start the log in process, the fabric is already
stable.
Guideline: TCP/UDP ports 20 and 21 must be available for the firmware download to function correctly.
Warning: When upgrading from Fabric OS v3.0.0/4.0.0 to Fabric OS v3.1.0/4.1.0, any port name changes that have been
made in Fabric Manager are lost. Also, make sure during the firmware upgrade if multiple Fabric Manager clients
are simultaneously active, they do not overwrite each other's port names.
Note: The firmware download process will time out in the event network connectivity to the switch is lost. Switches running
Fabric OS v2.x/3.x have a 25 minute time out period and switches running Fabric OS v4.x have a 30 minute time out
period. No error message is returned when the firmware download process gets interrupted.
9.6.6. Sequence Switch Reboot

Sequence switch reboot provides the functionality to define a reboot sequence for an orderly bringup of the fabric from a down
state. Fabric Manager lets you plan in advance to control the reboot sequence with in a switch group to minimize the number
of interruptions caused by a fabric re-configuration. When creating a new switch reboot group, the switches can be selected
from the same group and re-ordered. A Fabric wide reboot sequence including all fabric switches can be defined for an orderly
bringup of the fabric. For example, in a Core -Edge Topology SAN, a boot sequence can be defined in the following order
1. Boot Core switches
2. Boot Storage device edge switches
3. Boot Host edge switches
Guideline: Reboot the Core (backbone) of a large SAN first, and then reboot Edge switches (remaining sections).
Reboot distant physical locations sequentially.
Guideline: Define a sequence to re-boot the Core switches before the Edge switches, to provide a stable fabric
before connected devices perform fabric login.
Guideline: The fabric consists of SilkWorm 2000 series, 3800 and 3200 switches also require switch reboot to
activate new firmware after firmware download process completion. Rebooting of a switch cause fabric
reconfiguration which is somewhat disruptive in a non_redundant fabric environment. The core switches
of the fabric must be online first for a successful fabric re-configuration. The edge switches that provides
device connection can be re-booted after the core switches are up. The SilkWorm switches using 4.x
code supports hot code load do not require rebooting after firmware download. Therefore exclude these
switches from the reboot sequence.

Creating Core/Edge Fabric -A reboot sequence group:

1. From Tools > Reboot > Create Reboot Sequence. Click Create and provide sequence group name along with requested
parameters.
Figure 9-15 Specifying reboot sequence parameters

In case a reboot sequence process times out, an appropriate action can be defined to Prompt, Continue or Abort. The Fabric
stabilization time-out parameter defines the time limit for a successful completion of fabric stabilization after re-configuration.
This time-out period can vary depending upon the size of fabric and type of SilkWorm switches involved. The delay after
Fabric stabilization parameter determines the elapsed time before the next reboot in the sequence begins.
Note: Fabric Manager considers a fabric stable when all WWNs appear in the fabricshow command output.
2. The sequence is defined to bringup the Fabric-A fabric from a down state. Select all unassigned switches (cntrl+left
mouse button) and move to the group using left arrow button.
Figure 9-16 Assigning Fabric-A switches to a Reboot Group

3. Select only one logical switch of a SilkWorm 12000 chassis that houses two logical switches.
Figure 9-17 Fabric-A cold boot sequence group and member switches
4. The switches can be re-ordered using up and down arrow buttons. When done click Apply.
5. The Reboot button will start the reboot sequence in the order defined after a successful login to the switches. The
sequence can also be started from the Tools menu. The following screen shows the reboot process started from the Tools
menu. The switch status is monitored during the reboot process. A successful completion of the reboot process is
represented by a green status. The switches with red status are still in the process of rebooting.

Figure 9-18 Rebooting Fabric-A cold boot sequence Group switch status
9.6.7. Firmware Download to FDMI Capable Host Bus

Adapter
The firmware download process can be initiated only on FDMI capable HBAs that are connected to a Brocade SilkWorm
switch running Fabric OS 3.1.0 or 4.1.0 (or greater). They must also be logged into the Name Server. The Non-FDMI capable,
and FDMI capable HBAs currently logged out of the name server are grayed out in the “Firmware Download to HBA” menu
indicating their unavailability.
Note: Fabric Manager supports up to a maximum of 50 firmware downloads to multiple HBAs simultaneously.
9.6.8. Fabric Merge Check

The Fabric Merge feature of Fabric Manager simplifies the process of merging two separate fabrics by performing a complete
configuration compatibility check and validates both fabrics beforehand. Any configuration inconsistency between both
fabrics can prematurely abort the merge process leaving the fabric segmented. Fabric Manager extracts copies of configuration
elements from each fabric and compares them in memory. The corrective actions must be taken to remove the inconsistencies
that are found and displayed in the merge-check results window.
To Perform a Fabric Check:
From the Tools menu, select Fabric Merge. Then from the menu, select one of the two fabrics to merge and click Check.
A Merge Check Results list appears identifying the inconsistencies between the fabrics. Merge check process will first
ensure that the fabrics are accessible.
Checking a Fabric Merge on fabric Name fabric-hds112kA00sw and fabric-SW24000_32

Figure 9-19
Fabric Merge test results are displayed upon completion in status window. It is safe to merge when no inconsistency exists
between fabrics. The following list shows that switch domain-4 exists on both fabrics that must be resolved prior to
actually merging fabrics.
Merge Check Results ---
hds112kA00sw0 merge checked against SW24000_32
DomainIDTest
both fabrics contain domain: 4
Failed Merge Check
TOVTest
Merge Check Successful
BufferBufferCreditTest
DisableDeviceProbe Test
RoutePriorityPerFrame Test
SeqLevelSwitching Test
SuppressClassF Test
LongDistanceTest
InteropMode Test
MgmtServerPlatformTest
Test not applicable.
SecurityTest
Neither fabric has security enabled
FCSPoliciesTest
Not Applicable to subject fabrics.

SCCPolicyTest
Not Applicable to subject fabrics.
DataFieldSizeTest
VCEncodingTest
VC Priority Test
PIDTest
ZoningTest
Zoning Configuration from SW24000_32 will be applied to hds112kA00sw0
Guideline: It is important to establish the current operating mode of both fabrics. If both fabrics are operating in
non-secure mode then perform merge check and resolve any conflict that is reported in results display
window.
Guideline: Merging a Secure fabric with a non-secure Fabric or Merging Secure fabrics requires additional steps
of verifying current security policies. The security policies are fabric wide, therefore all the conflicts that
may arise for the resulting fabric after merging must be resolved.
Note: When a Merge Check is performed between a secure fabric and a non-secure fabric, the message “Not applicable to
subject fabrics” will given as a result for the following tests: Security, FCS Policies, Version Stamp, and Management
Server Platform. Merging a secure and non-secure fabric requires a careful planning based on the current fabric
configurations. These steps are provided in detail in the Brocade Secure Fabric OS Users Guide 4.1.0.
Another obstacle which can prevent fabrics from merging is the zoning configuration. When merging fabrics that use zoning,
the check will identify any zoning mismatches. Launching the Zone Merge Manager tool will highlight all zoning conflicts
between the two fabrics giving the administrator an opportunity to resolve the existing zoning conflicts. The Merge Check
message confirms the resolution by displaying the “Merge check process successful” status.
Guideline: All zoning related conflicts must be resolved between fabrics before extending an Inter Switch Link
(ISL). After Fabric Merge is successfully accomplished verify that each host can access its respective
storage device.
9.6.9. Complete Fabric Backup and Compare

The Fabric Manager backup process creates a complete backup file of the existing fabric configuration that includes the
following listed files for each fabric switch.
• Switch Configuration file
• License keys for every switch in the fabric
• A list of switches that belong to the fabric
• An ISL stamp (Inter Switch Link mapping at the time of backup)
• All zone definitions (and notes the active zone)
• Name server information
Once a fabric is backed-up, the current fabric settings can be compared to a saved baseline file. Fabric Manager displays the
discrepancies between the two in HTTP file format.

Note: This feature is available at the fabric level only. If a backup for each switch is desired, all switches in the fabric must
be configured individually to transfer files using the upload/download function in Web Tools.
Note: Verify that Fabric Manager has discovered all fabric information before performing a backup. Wait at least 60 seconds
after discovering a fabric before running a backup. This will allow Fabric Manager to discover all ports, devices, and
ISL information. Not waiting the 60 seconds after discovering a fabric before running a backup may result in
incomplete results when running a backup compare.
Example: Backing up Fabric-A configuration.
Figure 9-20 Fabric-A configuration

1. From Fabric Manager Action menu select backup. Specify the backup file path.
2. Backup now button initiates the backup process.
Figure 9-21 Figure - Specifying file path and starting backup process on selected fabric
Upon completion the back up process summary is displayed. The following screen shows that all required files for Fabric- A
switches are successfully backed up.

Figure 9-22 Fabric-A switch backup results

The backup files are saved in the specified folder for the Fabric-A switches.
Figure 9-23 Fabric-A backup files

9.6.9.1. DIFF with Backup

Compare the current Fabric-A configuration with the saved baseline configuration files to determine the change, if any.
For Example:
From the Fabric Manager Action menu select Diff with Backup. From the subsequent screen menu click Diff to start the
compare process. Upon completion a summary of the difference is displayed. As shown in the following screen, the current
fabric configuration differs from baseline in ISL configuration.
Figure 9-24 Difference details between current and baseline configuration

9.6.10. License Key Management

Fabric Manager can display, store, load, and reload license keys so they will not be lost in the unlikely event of switch failure.
With E-Licensing, license keys can be requested online.
Import and Export License Keys
License keys can be exported from healthy switch to a file so can they can be restored in the unlikely event of switch failure.
The following management functions are available.
• load fabric switch license from a switch into a.XML file
• Import licenses from a file
• Load license back to switches
• Remove license from a switch
• Print license hard copy for reference
9.6.10.1. Perform E-Licensing

E-Licensing provides users with the ability to acquire Licenses online for switch-based software features. In order to use this
feature, users need to have already purchased the licenses and obtained a Transaction Key in electronic form. Electronic
Transaction Keys are provided as a file, typically delivered as an attachment to an email.
Note: Not all switch vendors support the delivery of Electronic Transaction Keys, and therefore, this feature may not be
available to some users.
Example: Loading switch license from Fabric-A switches into a.XML file
1. From Fabric Manager tool bar select Tools > Licensing> select the option Load from switch.
Figure 9-25 Figure - Fabric-A switches

2. Select fabric-A switches by moving it to the right pane of the window using right arrow button and click OK when done.
Figure 9-26 Fabric-A switch selection for license management
3. License information loading from the selected switches is completed and displayed as shown in the following screen.
Figure 9-27 License information retrieval from Fabric-A switches.

4. The license data can be exported to a file - by selecting the Export to File button. Provide the complete path and file name
and then click Export.
Figure 9-28 Exporting license information to a file name Fabric-A switch License.
Note: If you are exporting to an external server make sure the server configuration is setup. From the File menu select
Options and then File Transfer. Configure the server for file transfer by entering the correct parameters and verify
accessibility by clicking Test.
Figure 9-29 File Transfer setup

5. License information can also be imported from a saved file. From the Fabric Manager tool bar select Tools > Licensing>
and select import from file. Provide the complete path of file (Fabric-A switch License.XML) and click Open.
Figure 9-30 Importing license information from a saved XML file

6. The following screen shows the License information of the imported file. The License information can be restored on one
or more switches of the Fabric-A by selecting switches and then depressing Download to Switch button.
Figure 9-31 Importing a file name “Fabric-A switch License”

After the downloading is complete, select the switch tab in the Fabric Manager information window to view the updated
license information on the switches. A copy of the license information can be printed by clicking Print.

9.6.11. Synchronize Fabric-Wide Time and Date

The switch date and time is set on each switch that may slightly differ from each other. The fabric-wide error log reporting also
includes the timestamp to indicate the exact time of the event. Unless the time and date is synchronized in the fabric, especially
if they are in different time zones, the error log event timestamp do not represent the accurate sequence of the events in the port
log. Therefore it is necessary to synchronize time and date across the entire fabric.
Note: To synchronize time and date, select fabrics, not switch groups.
9.6.12. Fabric-Wide Event Monitoring

The primary purpose of event monitoring is to ensure that the SAN elements are continuously operating with in the specified
limits of their safe operational range. In the event a safe operating limit is breached, an appropriate message is forwarded to the
error log. The message can be displayed for an entire fabric or an individual SAN element by selecting the SAN Element
Window and selecting Event tab. The messages are logged in chronological order. They can also be re-ordered by switch,
number, time and level fields of simply clicking the appropriate column tab in the Event log field.
Figure 9-32 shows Fabric-wide events on all switches of Fabric-A. The Current status reason window describes the reason
for a state change. Two switches of Fabric-A indicate marginal (orange color) operating status due to a loss of a redundant
power supply. The message received from the other switches are displayed with a time stamp that determine the order of
received events. This is an example of why it is important to synchronize the time in the fabric as described in section 9.6.11.
Synchronize Fabric-Wide Time and Date on page 9-35. The messages can be viewed by sorting them either by switch, Time
stamp or level of severity columns for clarity.
Figure 9-32 Event Monitoring for Switches in Fabric-A
Note: Event monitoring used in conjunction with Brocade Fabric Watch is an efficient way to monitor the state of the SAN
(refer to Chapter 13, Brocade Fabric Watch Overview). The Error Log message and status reason direct the
administrator to the source of the fault, saving the time and cost associated with debugging.

9.6.13. Inter-Switch Link (ISL) Checking

Enabling ISL Checking reports any status change in the event log and topology display. Fabric Manager takes a stamp of
current topology so it can compare changes and register events. Taking a re-stamp will accept the change permanently and use
updated topology for future comparisons.
Note: ISL Checking is supported on SilkWorm 2000 series switches running Fabric OS 2.6.0 or later.
9.6.14. Manually Refreshing the Fabric

Fabric Manager Views are updated at 20 second intervals; everything else in the GUI is updated whenever there is a change in
current data. In case of discrepancy it is always a good idea to perform a manual refresh that will re-discover the fabric and
refresh the display to its current status.
Note: Manually refreshing the fabric disables ISL Checking and Fabric Checking. The features must be re-enabled after the
completion of fabric refreshing.
9.6.15. Call Home Support

The Call Home feature of Fabric Manager continuously monitors the health status of the fabric switches and is capable of
delivering a notification via a user-defined email address when a critical event takes place. The forwarded message includes a
brief description of the failure, switch identification data, current status, and Fabric OS version etc. The Fabric Manager server
monitors the switches that have been discovered. The Fabric Manager client can be used to configure which switches to
monitor for Global Call Home functionality.
Note: Fabric Manager enables Call Home by default on the Fabric Manager server. However, you must configure the client
to select fabrics to monitor before the Call Home server can monitor switches.
Configuring Call Home

1. External executable on server Optional (see the note below).
Note: The Fabric Manager Call Home feature can accept an external executable that runs when a Call Home event occurs.
For example, entering C:\executable.exe in the External Executable on the Server field launches C:\executable.exe
filename.xml. Fabric Manager passes an XML file to the executable. The executable must be capable of being executed
by the OS on which the Fabric Manager server is installed and it must be a valid binary for that OS (Windows or
Solaris). It must be able to handle Fabric Manager passing it a command-line argument. The argument is the name of
an XML file that Call Home generates when an event occurs.
2. Click Commit button to commit a configuration. A configuration is committed only if at least one switch and one email
address or the path of an external executable is specified.

For Example: From the following Call Home window, a configuration name Call Home is created using Add button. The
monitoring is limited to only two switches of Fabric-A, named emc13800A00 and emc13900A00. Finally, a valid email
address is entered in the Email address pane. A switch downtime is specified for 50 seconds indicating that the event will
trigger an email message only if a switch downtime exceeds the 50 seconds threshold. No executable script or path is defined
(see note below). Click Commit to accept the parameters of the configuration name Call Home. A configuration is committed
only if at least one switch and email address or path of an external executable is specified.
Figure 9-33 A configuration name Call Home created for two switches of Fabric-A
Finally, the configuration can be Enabled/Disabled from the Control tab. When enabled, a notification message is sent out to
the specified email address.
Guideline: The Call Home feature must be used with some discretion. Consider limiting the email setup for critical
events that require your immediate attention. For example a failure of an ISL may result in fabric
segmentation thus disrupting the normal fabric operations. A switch FRU failure may cause the
additional damage if timely corrective actions are not taken.
Guideline: The minimum value accepted for Switch Unreachable Duration is 40 seconds. Consider setting this
period based on your fabric size, switch type and fabric environment.
Guideline: Checking the box labeled “Send server is running message every”, sends out a message at the
specified period. To avoid frequent messages set the interval period to a greater time interval.
Guideline: When setting an “External Executable on Server” feature consider small scripts. The external
executable runs as a background process, and the Task Manager monitors the process. A large script
may impair the performance of the server.

9.6.16. Secure Fabric OS Policy Management

Secure Fabric features introduced in Brocade Secure Fabric OS 4.1 and later can be efficiently managed via Fabric Manager.
Once the security feature is enabled on all switches from the command line interface, the Fabric Manager access to the fabric
is enabled on the primary FCS (Fabric Configuration Server) “trusted” switch by defining the following security policies using
the command line interface.
Fabric Manager manages the remaining Secure Fabric Policies via accessing the Primary FCS switch. A policy based security
management provides the flexibility of implementing SAN security as per unique SAN environment.
Note: For detailed information on Security, please refer to the Secure Fabric OS User's Guide.
Fabric Manager administers security policies via the Security Admin window selected from the Action menu of Fabric
Manager. The following examples shows how Security policies are created, updated and deleted from Fabric Manager.
Security on Fabric-D consists of four fabric switches is enabled from CLI.
Figure 9-34 A Secure Fabric-D and member switches
Note: When making changes in the Security Admin window, changes can be activated immediately by clicking Activate, or
changes can be saved without activation from Save button. When creating /updating multiple policies first save each
individual policy update and when done Activate. For information about activation of security policies refer to section
9.6.16. Secure Fabric OS Policy Management on page 9-38.

9.6.16.1. Switch Connection Control (SCC) Policy
The SCC policy defines all switches in the secure fabric (FCS and non-FCS). To add a switch to a secure fabric the switch
must be defined in the SCC policy. The SCC policy does not exist when security is enabled for the first time. Using the Fabric
Manager Security Action menu, first create the SCC policy and then add the switch members. The SCC policy can be created;
members can be added or removed easily from the Switch Connection Control tab of Fabric Manager.
Note: New fabric switches will not be able to join the secure fabric until the switch is added to the SCC policy.
For Example: This example shows the SCC policy that includes all Fabric-D switches. Since the SCC policy is created, no
new switch will be able to join the fabric until the WWN of the new switch is added to the SCC policy list. Save changes if
another policy needs to be created or updated. When done making changes to one or more policies, the new policies can be
activated by clicking Activate.
Figure 9-35 Defining SCC policy of the Secure Fabric

9.6.16.2. Fabric Configuration Server (FCS) Policy

Switches included in the FCS policy server lists are “trusted switches”. The first switch in the policy serves as the Primary
FCS from which the entire fabric is managed. Each subsequent switch serves as a Backup FCS. The order of the entry list
determines which Backup FCS switch becomes eligible for promotion in case of a Primary FCS switch failure. Switches can
be added, removed or re-ordered by accessing the FCS policy from the Security Admin window of Fabric Manager.
Guideline: The FCS policy must have at least one back up switch configured. If there is no back up switch
configured in FCS policy you will lose management access to the fabric in case the primary FCS fails.
The management interface access to a secure fabric is extended only via the primary FCS switch,
therefore it is recommended to specify as many backup switches as possible considering the order in
which they are promoted.
Example: From the Actions menu, select Security. The Security Admin window appears. Select the appropriate policy tab.
This example shows how to add a switch to a FCS policy already in place. A new switch name het94 is added to the current
FCS list. The list can also be re-ordered by using the up and down arrow keys. Save or Activate changes when complete.
Figure 9-36 Adding a new switch to FCS policy list

9.6.16.3. Device Connection Control (DCC) Policy
The DCC policy is defined to bind device ports to a specific switch ports. If the device is a fabric switch the binding is defined
by the SCC policy. If the device is a host or storage, the DCC binds those devices to a particular switch ports, thus restricting
the device connectivity. Fabric Manager lets you create and configure multiple DCC policies with unique names.
Example: Creating a DCC Policy named “Project-x” to for the purpose of controlling device connection. A device WWN
10:00:00:00:00:79:86:10 is binding to port 5 of switch luf128. Additional devices can be added by entering a device WWN
and corresponding switch port to the policy name Project-x. Select Save or Activate when done.
Figure 9-37 Creating a DCC policy and adding devices

9.6.16.4. Telnet Policy

A telnet session provides a command line interface to fabric switches. In a Secure fabric environment the fabric access
restriction can be defined by a telnet policy. The telnet policy contains IP address of trusted hosts or subnet that can establish
telnet connections to fabric switches. When a subnet is defined in the policy list, all hosts on that subnet can telnet to the fabric.
Creating an empty Telnet policy will prevent all telnet access to your fabric.
Example: Configuring Telnet policy of a Secure Fabric by providing IP address of the trusted Fabric manager clients/server
stations. In this case, the fabric access is limited to only these two trusted hosts (Fabric Manager Server= 192.168.162.78
Fabric Manager Client=192.168.173.213). Select Save or Activate when done (see note above)
Figure 9-38 Configuring Telnet access
Note: Telnet, RSNMP, WSNMP, HTTP and API access policies can be defined same way as shown by entering the IP address
of the trusted hosts to the Permitted Access Point list. Please refer to section 9.6.16. Secure Fabric OS Policy
Management on page 9-38 for activating updated Security Policies.
9.6.16.5. WSNMP Policy

Configure the WSNMP policy to limit SNMP access to specific, “trusted” management stations.
Note: Members of the WSNMP policy automatically gain RSNMP access.

9.6.16.6. RSNMP Policy
Configure the RSNMP policy to limit SNMP access to specific, trusted management stations.
Note: Creating a RSNMP policy requires the presence of a WSNMP policy. If not already present, create a WSNMP policy
first.
9.6.16.7. Hyper Text Terminal Protocol (HTTP) Server Policy

Configure the HTTP policy to grant access to IP addresses and/or subnets so they can establish HTTP connections to the
switches in the fabric.
Note: The policy must have the IP address of Fabric Manager client included to access the fabric via Fabric Manager.
9.6.16.8. Application Programming Interface (API) Policy

In a Secure fabric environment, create an API policy to control the workstations that can use the API to write to the fabric.
Warning: If Fabric Manager is used to update the API policy to disable API access from the current host either by creating
an empty policy, or by specifically excluding this host from the API policy list, the security transaction will be
locked and can take up to two hours before Fabric OS releases the security transaction. The policies cannot be
modified until the security transaction is released.
9.6.16.9. SCSI Enclosure Services (SES) Policy

The SES policy can be used to restrict which devices can be managed by SES commands. The policy is named SES_POLICY
and contains a list of device port WWNs that are allowed to access SES and from which SES commands are accepted and
acted upon.
Note: SES client must be directly attached to the primary FCS switch. Then the SES client can be used to manage all the
switches in the fabric through the SES product for SilkWorm. The current SES implementation does not support the
SES commands Read Buffer or Write Buffer for remote switches.
9.6.16.10. Management Server (MS) Policy

The Management Server policy can be used to restrict which devices can be accessed by the management server. The policy
contains a list of device port WWNs for which the management server implementation in Fabric OS (designed according to
FC-GS-3 standard) accepts and acts on requests.
Note: Fabric configuration and control functions can be performed only by the requesters that are directly connected to the
primary FCS switch.
9.6.16.11. Serial Port Policy

The Serial Port policy can be used to restrict which switches can be accessed by the serial port. The policy contains a list of
switch WWNs, domain IDs, or switch names for which serial port access is enabled. The serial policy is checked before the
account login is accepted. If the Serial Port Policy exists and the switch is not included in the policy, the session is terminated.

9.6.16.12. Front Panel Policy

The Front Panel policy can be used to restrict which switches can be accessed through the front panel. The policy contains a
list of switch WWNs, domain IDs, or switch names for which front panel access is enabled.
Note: This policy only applies to SilkWorm 2800 switches, as no other switches contain front panels.
9.6.16.13. Creating Options Policy

The Options policy can be used to prevent the use of Node WWNs to add members to zones. This policy is named
OPTIONS_POLICY and has only one valid value, “NoNodeWWNZoning”. Adding this value to the policy prevents use of
Node WWNs for WWN-based zoning.
Note: The use of node WWNs can introduce ambiguity because the node WWN may also be used for one of the device ports,
as may be true with a host bus adapter (HBA). If the policy does not exist or is empty, node WWNs may be used for
WWN-based zoning. Only one Option policy can be created. This policy cannot be used to control use of port WWNs
for Zoning. By default, use of Node WWNs is allowed; the Options policy does not exist until it is created by the
administrator.
9.6.16.14. Activating Security Policies

Once all the needed policies are created or updated they need to be activated. Depressing Activate button will open the
Security Policy Review window for the purpose of verifying changes before committing. This information can also be copied
to a file for future reference. Selecting continue will complete the activation process. Upon successful completion the updated
policies become active.
Figure 9-39 Security policy review screen before committing changes

Note: The updated policies can be saved without activating using save button. The defined and active policies can be viewed
from Summary button for the purpose of comparison.
Figure 9-40 Define and Active policies Summary view

9.7. Switch and Port Level Management

Brocade Fabric Manager simplifies the entire management process by providing a centralized management. Brocade Fabric
Manager is capable of managing complex tasks at fabric, switch and port level by integrating Brocade Web Tools, Telnet
(CLI), SNMP, Fabric Watch and Advance performance monitoring access, thus simplifying the over all three levels of
management from a single users interface. The following listed sections discuss these topics in detail. Please refer to the
appropriate Chapter.
• Switch and port level management
• Chapter 10, Brocade Advanced Web Tools Overview
• Chapter 11, Command Line Interface (Telnet) Overview
• Chapter 12, SNMP Overview
• Monitoring Fabric via integrated software packages
• Chapter 13, Brocade Fabric Watch Overview
• Chapter 14, Advanced Performance Monitor (APM) Overview
• Chapter 15, Brocade API Scripting Toolkit Overview

Chapter
Brocade Advanced Web Tools Overview
10
Brocade Advanced Web Tools is a graphical interface that enables an administrator to monitor and manage fabrics, switches,
and ports from a standard workstation or Fabric Manager 4.1 client. It is an optionally licensed product that runs on all versions
of Fabric OS. All switches in the fabric are displayed in the Fabric Tree window of Web Tools, including switches that do not
have a Web Tools license. However, only switches that have a Web Tools license installed can be managed through Web Tools.
Non-Licensed switches can be managed via command line interface (telnet, sectelnet, or ssh) session. Table 10-1 and
Table 10-2 detail the Web Tools browser requirements for switches running Fabric OS 2.6.1/3.1/4.1 and Fabric OS
2.6.2/3.1.2/4.2.
Table 10-1 Fabric OS 2.6.1/3.1/4.1 Web Tools Browser Requirements

Operating System Browser Java Plug-in
Microsoft Windows 2000,XP, NT 4.0 Microsoft Internet Explorer 5.5 Java Plug-in 1.3.1_04 or later
Solaris 2.7, 2.8 Netscape 4.77 Java plug-in 1.3.1_04 or later
Table 10-2 Fabric OS 2.6.2/3.1.2/4.2 Web Tools Browser Requirements

Operating System Browser Java Plug-in
Microsoft Windows 2000, 2003, XP Microsoft Internet Explorer 6.0 Java Plug-in 1.4.1_02 (preferred)
Solaris 2.8, 2.9 Mozilla 1.4 Java plug-in 1.4.2
Red Hat Linux 9.0 Mozilla 1.4 Java plug-in 1.4.2
Note: For all switches in a fabric to be managed with Web Tools, all switches must have a Web Tools license installed and
enabled.
Guideline: When using Web Tools in a fabric utilizing multiple versions of Fabric OS, it is recommended to launch
Web Tools from a switch running the latest version of Fabric OS.
Note: Although Web Tools is backwards compatible management function availability may vary. When using Web Tools in
a fabric utilizing multiple versions of Fabric OS, some features and enhancements available in later versions of Web
Tools may not be available of older versions.
Guideline: Launch Web Tools by using an IP address of a switch that has the latest Fabric OS.

10 Brocade Advanced Web Tools Overview
10.1. Launching Web Tools from a Workstation

1. From an from an independent workstation, launch the web browser and enter the IP address of the licensed switch in the
Location/Address field. For example: http://10.64.148.32 (SilkWorm 24000 Fabric OS 4.2) discovers the fabric switches
as shown in the following Web Tools screen. The functions in the left pane provide fabric level access and display window
controls. Clicking on a switch icon provides a detail switch view and related management tabs. Frequently accessed
switch management icons Switch event, Admin View and telnet are provided next to switch icon.
Fabric Tree Switch Information Window
Fabric Tool Bar Switch View Menu Tabs
Figure 10-1 Web Tools - Fabric View
Web Tools Main Views

• Fabric Tree - Displays a list of all the switches in the fabric.
• Fabric Tool bar - Provides access to fabric wide management interfaces such as Name Server, Fabric events and Zoning
Administration.
• Switch Information - Displays information about the switch such as: Switch Name, Window Switch Status, Fabric OS
Version, Domain ID, IP address, WWN, and current zone configuration.
• Switch View - Displays an interactive graphical representation of the switch.
• Frequently access menu icon - Switch Events, Admin View and Telnet

Brocade Advanced Web Tools Overview 10
10.2. Web Tools Access via Fabric Manager Client

Web Tools access to a fabric switch is available by selecting a fabric in the Element window and selecting an icon or the
Summary view tab. The switch management feature is available by clicking on an appropriate function icon. The Actions
menu provides a detail graphical view of a fabric switch along with the management tabs. The same functions available from
the Actions menu are available by right-clicking on a switch in the Element window.
Actions Menu
Switch View
Switch Events
Admin view
Fabric Watch
T elnet Summary View of Fabric switches
Fabric Element Display Window
Fabric Watch
Switch Events T elnet Update
Admin view Fabric Watch
Figure 10-2 SilkWorm 24000 - Fabric View

Slot Number
Slot Status LED
16-port Blade
CP Status LED
Port Status LED
Switch View
Menu Tabs
Figure 10-3 SilkWorm 24000 - Switch View
The Switch View displays a graphical representation for the selected switch, including a real-time view of switch and port
status. The Switch View also includes the following listed launch point and health monitoring buttons.
Table 10-3 Switch View Menu Items
Menu Item Function Menu Item Function
Status Displays the current operational status of the Info Opens a window that displays
switch. information on the switch.
Event Displays the switch events log. Watch Access to Brocade Fabric Watch
Admin Provides access to the switch management Fan Displays the switch fan status.
functions panel.
Telnet Opens a telnet session to the switch providing a Temp Displays the temperature readings
Command line (CLI) interface.
Perf Access to Performance Monitoring management Power Displays the power supply status.
Beacon Enables beaconing on the switch Hi Avail Displays the High Availability (HA)
Admin panels

10.3. Web Tools in Secure Mode

The Security feature may change your ability to access Web Tools functionality when Secure Mode is enabled. For more
information on the Security feature refer to the Secure Fabric OS User's Guide.
Note: A Secure mode Enable/Disable or any change to the Primary FCS requires the current Web Tools session be closed
and re-established after the change occurs.
Guideline: When Secure Mode is enabled, the Web Tools interface is controlled by the HTTP_POLICY. Unless the
workstation IP address is included in the policy, Web Tools access will be denied.
Guideline: The following functions are available only via Primary FCS switch of the Secure Fabric.
- Zoning Administration is available only on the Primary FCS switch. The Zoning button is disabled on
remaining secure fabric switches.
- SNMP Access Control Lists and the SNMP Community Strings can only be modified from the Primary
FCS switch. For Non-FCS switches, the SNMP Community Strings are configured read only, and the
SNMP Access Control Lists are not displayed.
Warning: Telnet access to a switch and the Telnet button in Web Tools are both disabled when secure mode is enabled for a
fabric. You must use sectelnet or SSH to access the Fabric OS CLI in a secure fabric. Web Tools does not provide
sectelnet or ssh capability.
Warning: When working with more than one Web Tools module in a secure fabric, login to one module at a time, and
complete the login process before proceeding to another task. For example, if you want to access “Zone
Administration” and “Switch Admin” modules, open one of the modules, login, and wait for it to fully load before
opening the second module. Abnormal behavior may be seen if attempting to open two modules simultaneously
in a secure fabric.

10.4. Brocade Advanced Web Tools Features

This section provides a brief overview of Web Tools features. For detail information please refer to the Brocade Advanced Web
Tools Users Guide.
10.4.1. Fabric-Wide Zone Administration

Brocade Advanced Zoning allows you to partition a SAN into logical groupings of devices that can access each other. Brocade
Advanced Zoning can be configured and administered via telnet or Brocade Advanced Web Tools. Using Zoning via Web
Tools, you can arrange fabric-connected devices into logical groups or zones for the purpose of providing a selective access.
For example, a host can gain access to a storage device when both are members of the same logical group or zone.
Non-member access is restricted to the member devices of the defined zone. Zones can be configured dynamically.
Configuring new zones do not interrupt traffic on unaffected ports, devices, or ISLs. They can vary in size depending on the
number of fabric-connected devices, and any device can belong to more than one zone.
Brocade Advanced Zoning can be implemented from any switch in the fabric. Changes configured to one switch automatically
replicate to all switches in the fabric. If a new switch is added to the existing fabric, all zone characteristics are automatically
applied to the new switch. Because each switch in the fabric stores zoning information, Brocade Advanced Zoning ensures a
high level of reliability and redundancy.
Brocade Advanced Zoning uses policy based administration, separating zone specification from zone enforcement. Multiple
zone configurations can be managed and enabled easily. A fabric can store any number of configurations; however only one
configuration can be active at a time. Thus configurations can be built in advance and saved and easily activated on demand.
10.4.1.1. Zones for Controlled Access

Use zones to provide controlled access to fabric devices and to establish barriers between operating environments. For
example, isolate Widows and UNIX hosts by creating separate zones.
10.4.1.2. Customize Environment

Use zones to create logical subsets of the fabric to accommodate closed user groups or to create functional areas within the
fabric. For example, include selected devices within a zone for the exclusive use of zone members or create separate test or
maintenance areas in the fabric.
10.4.1.3. Optimize IT Resources

Use zones to consolidate equipment, logically, for IT efficiency, or to facilitate time-sensitive functions. For example, create a
temporary zone to back up non-member devices
Note: A Zoning license and admin privileges are required to access this module. In Secure Fabric Mode the Zoning access
is available only via the Primary FCS switch.

10.4.1.4. Zone Configurations
A zone configuration often called a “config” consists of one or more zones. The config file is created and saved under a unique
name, thus there can be multiple config files but only one config can be active at a given time. A Configuration file may have
one or more of the following group types.
• Zone names
• QuickLoop names
• FA (Fabric Assist) zone names
The member of each group can be identified either via Alias name, WWNs, domain and port area number or Quickloop
AL_PAs. There are four methods of defining members for zoning:
• Mixed mode (Soft zoning) When zone members are identified using one or more naming convention. (Alias, Domain
& port number, WWNs and Quickloop AL_PAs)
• Port zoning (Hard zoning) Zone members are identified using domain & port naming convention only.
• WWN zoning (Hard zoning) Define zone members using device's WWN.
• AL_PA zoning (Hard zoning) Define zone members using QuickLoop AL_PAs only
Guideline: The Zoning configuration, when activated, is enforced either via SilkWorm hardware or software.
Whether you are using soft zoning or hard zoning is determined by the way the zone members are
defined. To define hard zoning, all members of the zone must be defined either by WWN or domain &
port number convention.
Guideline: A device Alias or Zone name can be defined with a maximum of 64 characters. To make an efficient use
of allocated zoning database memory, define meaningful alias and zone names by limiting the number
of characters to minimum. Also, routinely review a zoning configuration and remove alias and zone
names that are no longer in use. The size of the zone database can be monitored by running cfgsize
command.
Note: The maximum zoning database size for a switch running Fabric OS 2.x or 3.x is 96 KB. A switch running Fabric OS
4.x has a maximum zoning database size of 128 KB. If a switch running Fabric OS 2.x/3.x is part of a fabric utilizing
Fabric OS 4.x, the fabric wide zoning database size is limited to 96 KB.
Note: QuickLoop (QL) and QuickLoop Fabric Access (QLFA) are supported only on SilkWorm 2000, 3200 and 3800
switches.

10.4.2. Accessing Zone Administration Window

1. Launch Web Tools from a work station. The fabric will be discovered automatically.
2. Click the Zone Administration icon.
3. Enter the admin user name and password.
The Zone Administration window appears, shown in Figure 10-4. Sample Configuration name vendor consists of a single zone
name system. The three members of the zone are defined by using domain and port number naming convention.
Configuration Name Function Tabs Web Tools Refresh Zone Configuration

Window (Config Tab View) Members
Zones Members Status Window
Figure 10-4 Zone Administration Window

10.4.3. Zoning Configuration Tasks

Zoning configuration related tasks can be performed from Web Tools. The current configuration can be viewed by selecting
the appropriate function tab. The existing configurations can be modified or a new one can be created. The Edit, View and
Actions menus provide the following options. For detailed Zone management operations please refer to the Brocade Web Tools
User’s Guide.
Edit Menu options
• Adding a WWN in the Zoning Database
• Deleting a WWN in the Zoning Database
• Replacing a WWN in the Zoning Database
• Searching for a Zone Member
View Menu options
• Selecting a Zoning Method
• Refresh the Zone Database.
• Refresh the Fabric
Actions Menu options
• Enabling a Configuration
• Disable Zoning
• Saving Changes to Configuration
• Clearing the Zoning Database
• Alias Tab - Create, modify, rename, or delete aliases in the zoning database.
• Zone Tab - Create, delete, rename or modified members of an existing zone.
• Config Tab - Create, delete, rename or modified an existing Zone configuration.
• QuickLoop Tab - QuickLoop tab to manages loops in the zoning database. For more information regarding
QuickLoop, refer to the QuickLoop User's Guide.

10.5. Guidelines For Using Web Tools with Versions

of Web Tools Prior to Fabric OS 2.6.2/3.1.2/4.2
This section provides notes and guidelines for using older versions (pre-Fabric OS 2.6.2, 3.1.2 and 4.2) of Web Tools in SAN
fabrics which have a SilkWorm 24000, 3850 and/or 3250 as a member.
To verify the JRE Version on the supported OS platforms do the following:
• On Windows, check the JRE version in the Java Console or looking for the register entry:
“HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment\CurrentVersion”.
• On Solaris/Linux: check the JRE version by typing “java -version”.
When launching Web Tools on a switch running a Fabric OS version prior to 2.6.2, 3.1.2 or 4.2, Web Tools may generate
errors while attempting to discover switches running Fabric OS 4.2.
Guideline: Always launch Web Tools from the switch running the latest version of Fabric OS.
Guideline: On Solaris and Linux use JRE 1.4.2 or greater. If using Solaris or Linux, Mozilla 1.4 or higher in general
is OK. Avoid Netscape unless using Web Tools from Fabric OS 2.6.2, 3.1.2 and 4.2. Be aware that Linux
requires the use of the CLI to bypass the Switch Admin window.
Guideline: Avoid popup blockers. They disrupt and prevent JRE windows from appearing. It is highly
recommended to uninstall them from the host accessing Web Tools.

10.6. Switch/Port Level Management Functions

Switch Management tasks are performed from the Switch Admin Window. Selecting the appropriate tab opens up a new
window displaying the current setup. The user can then either view or edit the related parameters.
Accessing the Switch Admin Window
1. Launch Web Tools by entering the desired IP address in a browser window.
2. Select a switch from the Fabric Tree. The selected switch appears in the Switch View.
3. Select the Admin icon from the switch View. The login dialog box appears.
(From Fabric Manager select appropriate fabric switch from SAN element window and then from Actions menu select Admin
or click the Admin icon on the appropriate switch.)
4. Enter the admin user name and password.
5. Switch information window opens up by default. Select appropriate tab to manage the required function.
Note: The Switch Admin Window can be entered with User level access, but certain areas require Admin level access.
10.6.1. Switch Information Tab

Use the Switch Information tab to manage basic switch setup for items such as switch name, switch domain ID, and
enable/disable the switch. For additional information on switch parameter settings, refer to the Configure command in the
Brocade Fabric OS Reference.
Note: After modifying parameters click Apply and refresh the view.

Selected Switch Switch Administration Window Tabs Switch Information Tab
Fabric Information Status Window Control Buttons
Figure 10-5 Switch Admin View
Table 10-4 summarizes the switch and port level management tasks that can be performed via the management window tabs.
Table 10-4
Tabs Switch Administration Tasks
Switch information Set basic Switch configuration variable (switch name, domain ID etc.)
Network Config Set basic network address variables such as FC Net IP, Ethernet IP, Syslog IP
Upload/Download Updates firmware, backup/download a new switch parameter configuration
SNMP Configure SNMP parameters such as community string, access control, Traplevel
License Admin Install license keys for advance features
Configure Set switch parameters such as BB credit, TOV, virtual channels, AL etc.

Table 10-4
Tabs Switch Administration Tasks

Routing Set frame routing features such as link cost, static routing, IOD or DLS
Port Admin Tabs Port Administrative tasks
Port setting Enable/disable port, configure speed, enable trunking
Extended Fabric Setup long distance licensed features
Trunk information Configure Trunking licensed feature
Note: For more information about tab views, configuring, updating and applying parameters, please refer to Brocade
Advanced Web Tools User’s Guide v4.2.
10.6.2. Network Config Tab

Administrative interface to manage the IP networking functionality of a fabric switch.
• Configuring an Ethernet IP
• Configure FC IP Address
• Configure Syslog IP
Note: When both FC IP and Ethernet IP addresses are configured for a switch, the switch is identified by the FC IP address.
Warning: When modifying either the Ethernet IP or FC IP configuration from the Network Config tab, you will loose the
network connection to the switch while these parameters are updated. If the IP address of the switch is changed,
you must close all windows and restart Web Tools using the updated IP address.
Note: The Syslog IP represents the IP address of the server that is running the syslog process. The Syslog daemon reads and
forwards system messages to the appropriate log files and/or users, depending on the system configuration. When one
or more IP addresses are configured, the switch forwards all error log entries to the syslog on the specified server(s).
Up to six servers are supported. Refer to Fabric OS Procedures Guide for more information on configuring the Syslog
daemon.
10.6.3. Upload/Download Tab

Use the Upload/Download tab of the Switch Admin window to complete the following tasks
• Download firmware to a switch.
• Upload a switch configuration file to the host.
• Download a saved switch configuration from host to switch.
Note: The host information must be provided by the user for all upload /download tasks.

10.6.4. SNMP Tab

To perform administration of the SNMP subsystem, use the SNMP tab to specify the switch community string, location, trap
level and trap recipients. For more detailed information regarding SNMP, refer to the agtcfgset command in the Brocade
Fabric OS Command Reference.
Note: The SNMP tab is affected by the use of Secure Fabric OS; the ACL list will not be visible if security is enabled. For
specific information regarding security, refer to the Brocade Secure Fabric OS User's Guide.
Note: In order for the switches to send SNMP traps, first enable the MIBs on all fabric switches from the command line
interface executing snmpmibcapset command.
10.6.5. License Admin Tab

View, install, or remove license keys to enable or disable Brocade licensed features. The license key is case sensitive. Enter the
license key exactly as listed, using upper and lower case characters.
Note: To enable a new licensed feature, first obtain a license key from Brocade. Removal of a license key disables the feature
permanently. Save your license key for future use before removing it.
10.6.6. Configure Tab

View or update the current Fabric Configuration
• Fabric Parameters
• Virtual Channel parameters
• Arbitrated Loop parameters
• System Services parameters.
For more detailed information regarding the fields available in this tab, refer to the configure command in the Brocade Fabric
OS Command Reference.
Note: Most of the updates to the switch parameters require the switch to be disabled first. A grayed out parameter field is
read only and cannot be modified.
10.6.7. Routing Tab

Use the Routing Tab of the Administrative Interface to perform following tasks
• View the Fabric Shortest Path First (FSPF) routing information.
• Add or delete a static route.
• Set the link cost for the selected ports.
• Dynamic Load Sharing (DLS)
• In-Order Delivery (IOD)
Note: If a switch has one or more ISLs and no attached devices, the Routing tab will not display any information.

10.7. Port Management Functions

Switch Port management functions can be performed using the following tabs.
• 10.7.1. Port Setting Tab on page 10-15
• 10.7.2. Extended Fabrics Tab on page 10-17
• 10.7.3. Trunk Information Tab on page 10-17
10.7.1. Port Setting Tab

The Port Setting tab of the Administrative interface allows the setup of a switch port configuration.
• Disable/Enable a port
• Configure port speed 1GB/2GB/Auto Negotiate on 2 Gbit/sec capable switches
• Naming a port
• Enable/Disable Trunking
• Set persistent Domain ID
• View current online status and negotiated speed
• Viewing Swapped Port Area IDs
Note: The additional features of port management are available in Command line interface. Please refer to Chapter 11,
Command Line Interface (Telnet) Overview for a complete set of port configuration commands.
Guideline: Brocade 2 Gbit /sec switches support both 1 Gbit/sec and 2 Gbit/sec transmission. Negotiation mode
negotiates port speed. The port can also be locked in to supporting a single speed. You should consider
locking a port speed when connected to a legacy device not capable of negotiating speed.
Guideline: Port swapping feature available in Fabric OS version 4.1.0 and greater can be set only from the
command line interface. Web Tools only allows viewing of the current port swap status that has been
changed from the CLI. The Port number (Area ID) column shows the swapped port number followed by
the Area ID in parenthesis.

Port Setting Tab View Enabled when checked Port Speed Port Name
Port Blade Slot Status Window Port Status Control Buttons
Figure 10-6 Port Setting Tab View

10.7.2. Extended Fabrics Tab

The Extended Fabrics tab is useful when setting up port configuration parameters to support long distance connectivity. From
the Extended Fabrics tab you can specify which ports to be configured for distance and at what level. All switches come with
L0 and LE (extended normal) settings. An Extended Fabrics license enables additional settings. The support for an Extended
Fabric varies depending on port speed and distance. Please refer to Appendix B, Long-distance Technologies for Storage Area
Networks for more information. For more detailed information regarding the Extended Fabrics feature, refer to the Brocade
Fabric OS Features Guide.
10.7.3. Trunk Information Tab

The Trunk Information tab is a read-only tab. This window displays the trunking groups, master port and member port
information. The trunk management is done from the command line interface. Please refer to the portCfgTrunkPort command
to configure trunking on a port (Chapter 11, Command Line Interface (Telnet) Overview).


Chapter
Command Line Interface (Telnet) Overview
11
The Switch and Port level management feature offered in Brocade Web Tools focuses on common and important features. The
Brocade command line interface (CLI) offers a complete superset of management features. A complete list of commands can
be obtained either from the Brocade Fabric OS Reference Guide or by using the help command in a telnet window after
logging into a switch. The help pages on each command can be listed by simply typing help followed by a specific command.
The command line interface access is very important to setup new switch configuration parameters. Also, some specific
features such as enabling Secure Fabric OS mode and Brocade Fabric Watch require command line interface, although they
can be managed from Brocade Web Tools. Key CLI use cases are defined in this section.
Note: Telnet access is required to perform the following tasks:

- IP address configuration
- SwitchStatusPolicySet
- Trunking configuration
- SNMP
- Fabric Watch initialization
- Port swapping
- Secure Mode enable/disable
11.1. Ethernet IP Address Setup

A new switch must be configured for Ethernet IP address via serial port connected to a hyper terminal. Once network access
configuration is complete, a telnet session can be opened.
Example: SilkWorm 3250 (non-bladed switch)
Dazzler8_48:admin> ipaddrset
Ethernet IP Address [0.0.0.0]: 10.64.147.48
Ethernet Subnetmask [255.255.255.0]: 255.255.240.0
Fibre Channel IP Address [0.0.0.0]:
Fibre Channel Subnetmask [0.0.0.0]:
Gateway IP Address [0.0.0.0]: 10.64.144.1
Issuing gratituous arp...Done.
IP address is being changed...Done.
Committing configuration...Done.
Example: SilkWorm 24000 (bladed switch) which requires IP address configuration for CP0, CP1 and logical switch 0
SW24000_32:admin> ipAddrSet -cp 0
Host Name [cp0]:
Ethernet IP Address [10.64.148.34]:
Ethernet Subnetmask [255.255.240.0]:
Gateway IP Address [10.64.144.1]:
IP address of remote CP is being changed...Done.
SW24000_32:admin> ipAddrSet -cp 1

Host Name [cp1]:

11 Command Line Interface (Telnet) Overview

Gateway IP Address [10.64.144.1]:
IP address is being changed...Done.
SW24000_32:admin> ipAddrSet -sw 0

Fibre Channel IP Address [0.0.0.0]:
Fibre Channel Subnetmask [0.0.0.0]:
Issuing gratituous arp...Done.
SW24000_32:admin> ipAddrShow
SWITCH
Ethernet IP Address: 10.64.148.32
Ethernet Subnetmask: 255.255.240.0
Fibre Channel IP Address: 0.0.0.0
Fibre Channel Subnetmask: 0.0.0.0
CP0
HostName : cp0
Gateway Address: 10.64.144.1
CP1
HostName : cp1
Gateway Address: 10.64.144.1
Backplane IP address of CP0 : 10.0.0.5

Backplane IP address of CP1 : 10.0.0.6
11.2. SwitchStatusPolicySet
The switchstatuspolicyset command is used to change the overall switch status policy parameters. The current policy setup can
be viewed by executing SwitchStatusPolicyShow. These thresholds are used to determine the health status of the switch and
can be viewed in the Event window or by selecting the Status tab from the Action menu. The switch icon will also change
color from green (healthy) to orange (marginal) or red (down).
The default switch status policy is based on the switch model and configuration. For example, the SilkWorm 12000 and 24000
use the same chassis but the power supply requirements vary. A SilkWorm 12000 chassis may have 2 to 4 power supplies
depending on the switch configuration. This command allows the thresholds to be adjusted accordingly.
Example:
poc165:admin> switchStatusPolicySet
The current overall switch status policy parameters:

Down Marginal
--------------------------------------------------------------
FaultyPorts 2 1
MissingSFPs 0 0
PowerSupplies 3 1
Temperatures 2 1
Fans 2 1
PortStatus 0 0

Command Line Interface (Telnet) Overview 11
ISLStatus 0 0
CP 0 1
WWN 0 1
Blade 0 1
Note that the value, 0, for a parameter, means that it is NOT used in the calculation.
** In addition, if the range of settable values in the prompt is (0..0),
** the policy parameter is NOT applicable to the switch.
** Simply hit the Return key.
The minimum number of

FaultyPorts contributing toDOWN status: (0..64) [2]
FaultyPorts contributing toMARGINAL status: (0..64) [1]
MissingSFPs contributing toDOWN status: (0..64) [0]
MissingSFPs contributing toMARGINAL status: (0..64) [0]
Bad PowerSupplies contributing toDOWN status: (0..4) [3]
Bad PowerSupplies contributing toMARGINAL status: (0..4) [1]
Bad Temperatures contributing toDOWN status: (0..6) [2]
Bad Temperatures contributing toMARGINAL status: (0..6) [1]
Bad Fans contributing toDOWN status: (0..3) [2]
Bad Fans contributing toMARGINAL status: (0..3) [1]
Down PortStatus contributing toDOWN status: (0..64) [0]
Down PortStatus contributing toMARGINAL status: (0..64) [0]
Down ISLStatus contributing toDOWN status: (0..64) [0]
Down ISLStatus contributing toMARGINAL status: (0..64) [0]
Down CP contributing toDOWN status: (0..2) [0]
Down CP contributing toMARGINAL status: (0..2) [1]
Down WWN contributing toDOWN status: (0..2) [0]
Down WWN contributing toMARGINAL status: (0..2) [1]
Down Blade contributing toDOWN status: (0..4) [0]
Down Blade contributing toMARGINAL status: (0..4) [1]
No change
poc165:admin>
11.3. Configuring Trunking

Trunking can be enabled either on a port by port basis or for an entire switch, after installing and activating the Trunking
license. There are other limitations and specific requirements you should consider before enabling this feature.
switchCfgTrunk mode parameter Configure all ports on the switch for trunking
(Mode parameter 1=enable 0=disable)
Example: switchCfgTrunk - Enables trunking on all E-ports
portCfgTrunkPort port number mode Configure a specific port for trunking
(Mode parameter 1=enable 0=disable)
Example: portCfgTrunk 2 - Enables trunking on switch port 2 only
Note: Trunking is supported up to 10 Km, provided the deskew is tightly controlled by keeping the cable lengths equal with
in the trunk group. Trunking is not supported in Extended Fabric mode at this time.

11.4. SNMP Commands

• snmpMibCapSet - Set options for configuring SNMP MIBS/Trap capability
• trackChangesSet - Enables track-changes feature
These commands and their corresponding parameters must be issued and configured from the command line interface.
11.5. Fabric Watch Initialization (fwhelp)

fwClassInit - This command initializes all classes under Fabric Watch.
poc165:admin> fwclassinit
fwClassInit: Fabric Watch is updating...
fwClassInit: Fabric Watch has been updated.
Note: This command must be executed immediately after installing the Fabric Watch license to start Fabric Watch classes.
fwAlarmsFilterShow Verify current on /off status of alarms filtering and use the fwAlarmfilterset to setup the alarm
reporting.
fwAlarmsFilterSet [ mode] 1= alarm on ; 0=off
poc165:admin> fwAlarmsFilterShow
FW: Alarms are disabled
poc165:admin> fwAlarmsFilterSet 1
FW: Alarms are enabled
Note: The Alarms are suppressed for all non-environment Fabric Watch classes when turned off.
11.6. Port Swapping

portSwap [slot/]port1 [slot/]port2
The port swapping feature is a useful option that was introduced in Fabric OS 4.1 and later versions. This option allows a new
healthy port of the switch to assume the PID of the failed port. Port swapping is the easiest way to keep the pre-defined storage
link consistency. After a port swap, the new port FLOGI (the alternate port) should be the same as the original (failed) port.
The Port Swapping feature is a temporary solution with no impact on other switch functionality. It does not require switch
disable or reboot. The traffic on the remaining switch ports, other than ports being swapped, remain undisturbed. The result of
the portswap operation is persistent across reboots and power cycles, which mean ports remain swapped after reboots or power
cycles.
1. Log in to the switch as admin then first enable the feature by executing portswapenable.
2. Disable both ports using the portdisable command.
3. Enable the port swap by executing portSwap command.
4. Enable the ports by executing portEnable command
Example: Failed Port slot1/port0, alternate healthy port Slot10/15 on A SilkWorm 24000 switch,
SW24000_32:admin> portswapenable
SW24000_32:admin> portdisable 1/0
SW24000_32:admin> portdisable 10/15

SW24000_32:admin> portswap 1/0 10/15
portswap done
SW24000_32:admin> portenable 1/0
SW24000_32:admin> portenable 10/15
Guideline: This option is useful when the failing port is a storage port and the hosts accessing the devices on the
port are using the Port ID (PID) bindings method. In this case, moving storage to a different healthy
switch port will result in miss-matching the actual storage link path with the host configuration file
bindings entry. One alternate to this is to update the configuration file storage device binding entries to
reflect the new Port ID, which is not always an easy task and some OS also requires System reboot to
make the configuration file changes effective.
11.7. Security Enable/Disable (sechelp)

• PkiShow - Show existing pki objects
• pkiCreate - Create new pki objects
• secModeEnable - Enable security mode
• secModeDisable - Disable security mode
• secModeShow - Show current mode of security
• secFabricShow-Show security related information
switch:admin> pkiCreate
Installing Private Key and Csr...
Use the pkiCreate command in non-Secure mode to create pki objects (i.e. switch private key, private key pass phrase, CSR an
install root certificate). This command does not install switch certificates. Switch certificates should be obtained offline from
Certificate Authority. In Secure mode this command will exit with a warning and will not create pki objects. Pkishow
command verifies the desistance of pki objects.
switch:admin> pkishow
Passphrase : Exist
Private Key : Exist
CSR : Exist
Certificate : Exist /Note: If Certificate is empty - obtain certificate first/
Root Certificate: Exist
Note: The switches must have all the required PKI objects correctly installed.
secModeEnable Commands
• secModeEnable Activates security mode on all switches in the fabric.
• Creates the security database populated with a list of FCS switches in the FCS_POLICY.
• Distributes the security database to all switches in the fabric.
• Resets the root, factory, admin, and user account passwords on all FCS switches.
• Resets the admin account password on all non-FCS switches.
Note: This command fails if any switch in the fabric is not capable of enforcing the security policies defined in the security
database. The fcslist option let you specify one or more FCS switches by either WWN, switch name or domain ID. If
no operand is specified the command becomes interactive.

switch:admin> secModeEnable ( interactive session to create fcs list)

Your use of the certificate-based security features of the software installed on this equipment is
subject to the End User License Agreement provided with the equipment and the Certification Practices
Statement, which you may review at http://www.switchkeyactivation.com/cps. By using these security
features, you are consenting to be bound by the terms of these documents. If you do not agree to the
terms of these documents, promptly contact the entity from which you obtained this software and do not
use these security features.
Do you agree to these terms? (yes, y, no, n): [no] y
This command requires Switch Certificate, Security license and Zoning license to be installed on every
switch in the fabric.
PLEASE NOTE: On successful completion of this command, all login sessions will be closed and all
switches will go through a reboot to form a secure fabric.
This is an interactive session to create a FCS list.
The new FCS list is empty.

Enter WWN, Domain, or switch name(Leave blank when done): 10:00:00:60:69:80:0f:8a
Switch WWN is 10:00:00:60:69:80:0f:8a.
The new FCS list:
10:00:00:60:69:80:0f:8a
Enter WWN, Domain, or switch name(Leave blank when done): 10:00:00:60:69:51:0e:8b
Switch WWN is 10:00:00:60:69:51:0e:8b.
The new FCS list:( The following two switches are entered to the fcs list)
10:00:00:60:69:80:0f:8a
10:00:00:60:69:51:0e:8b
Enter WWN, Domain, or switch name(Leave blank when done):
Are you done? (yes, y, no, n): [no] y
:
User will be prompted to enter the new passwords for all accounts. Enter new passwords:
When complete the switches will be rebooted and the current telnet session is closed. After the reboot, fabric wide security is
enabled. Open a sectelnet session and login to fabric switches using the new password. Once Security mode is enabled, the
Security icon from the Action menu of Fabric Manager becomes active. The remaining policies can be created and managed
using Fabric Manager.
Warning: If the fabric is not in Secure mode and this command is issued, all switches in the fabric will be rebooted
automatically after the command is successfully executed.
Warning: Once security is enabled, a secure telnet session (sectelnet or SSH) must be used for issuing the command.
Warning: If security is enabled first time on the fabric and no FCS switches are present in the fabric, the command can be
issued on any switch.
Warning: If the fabric is in secure mode and no FCS switches are present in the fabric, the command can be issued on any
switch. This is used to recover a secure fabric that has no FCS switch.
switch:admin> secModeShow
Secure Mode: ENABLED.
Version Stamp: 1522496366, Wed Mar 31 20:53:14 2004.
Pos Primary WWN DId swName.
=================================================
1 Yes 10:00:00:60:69:80:0f:8a 5 luf128.
2No 10:00:00:60:69:51:0e:8b 2 fabb3800.
Additional Secure Fabric commands are listed in section 11.8.1. Security Commands on page 11-7.

11.8. Command Sets (Reference Use)
11.8.1. Security Commands

Table 11-1 Security Commands
Command Function
pkiRemove Remove pki objects
pkiShow Display existence of pki objects
secActiveSize Display the size of the active database
secDefineSize Display the size of the defined database
secFabricShow Display security related fabric information
secFCSFailover Force primary role to this FCS switch
secGlobalShow Display the current internal state information
secModeEnable Enable security mode
secModeDisable Disable security mode
secModeShow Show current mode of security
secNonFCSPasswd Set non-FCS password
secPolicyAbort Abort changes to defined policy
secPolicyActivate Activate all policy sets
secPolicyAdd Add members to a policy
secPolicyCreate Create a policy
secPolicyDelete Delete a policy
secPolicyFCSMove Move a FCS member in the FCS list
secPolicyRemove Remove members from a policy
secPolicySave Save all policy sets and send to switches
secPolicyShow Show members of one or more policies
secPolicyDump Dump all policies
secStatsReset Reset security statistics
secStatsShow Display security statistics
secTempPasswdSet Set temporary password
secTempPasswdReset Reset temporary password
secTransAbort Abort current transaction
secVersionReset Reset version stamp

11.8.2. Diagnostic (diaghelp)

A complete list of SilkWorm Diagnostic command is available by typing diaghelp from command line interface. Some of
these tests can be run in the background without impacting switch operations and frame traffic. However, others can be
disruptive. Please use the help command (example: help backplanetest) to obtain man pages on a specific command before
executing it.
Table 11-2 Diagnostic Commands
Command Function
backplanetest Backplane connection test for multi-blade systems

backport Test for back-end ASIC pair to ASIC pair links.
bladediag Run a suit of diagnostic tests on a switch blade
bladediagshort Run a suit of diagnostic tests on a switch blade
centralmemorytest Test ASIC central memory operation
cftest Run compact flash memory sanity check
cmemretentiontest Data retention test of the central memory SRAMs
cmitest Verify CMI bus between ASICs
cptest CP test
crossporttest Functional test of port external transmit and receive path
diagclearerror Clears diagnostics failure status
diagdisablepost Disable diagnostic POST
diagenablepost Enable diagnostic POST
diagoktorun Check to see if it is OK to run a diagnostic test
diagpost Set or display diagnostic POST configuration
diagshow Display diagnostics status
diagstatus Display currently running diagnostic tests
kerneltest Boot PROM test
kerneltest Kernel flash test
ledtest Exercise the port, and status LEDs in a system
minicycle Functional test of internal and external transmit and receive paths at full speed
portledtest Cycle user port LEDs
portregtest Write/read test of the ASIC SRAMs & registers
porttest Functional test on a live fabric. Starts porttest.
porttestshow Retrieve information from porttest.
ptbufshow Dump port buffer contents
ptcreditshow Display port credits
ptregshow Display contents of port registers

Table 11-2 Diagnostic Commands
Command Function
ptrouteshow Display port routing tables
ptstatsshow Display port statistics
ramdump Display the contents of port internal registers
spinfab Functional test of switch to switch ISL cabling and trunk group operation
spinjitter Line-speed jitter measurement
spinsilk Functional test of internal and external transmit and receive paths at full speed
supportshow Prints switch information for debugging purposes.
switchdiag Run a suit of diagnostic tests on a switch blade
systemtest Run a series of diagnostic tests on a switch blade without removing cables
systemverification Run a suit of diagnostic tests on all switches in a system
turboramtest Turbo SRAM test for bloom ASICs
txdpath Functional test of ASIC pair TXA, TXD connections
voltagemargin Set the slot voltage margin
Table 11-3 Switch Level Commands
Command Function
switchBeacon Set switch beacon on or off

switchCfgPersistentDisable Persistently disable a switch
switchCfgPersistentEnable Enable a persistently disabled switch
switchCfgSpeed Configures all ports of the switch to a particular speed level
switchCfgTrunk Configure all ports on the switch for trunking
switchDisable Disable this switch
switchEnable Enable this switch
switchStatusPolicySet Set policy parameters for overall switch status
switchStatusPolicyShow Print policy parameters for overall switch status
switchStatusShow Print overall switch status
trunkDebug Debug a trunk link failure
trunkShow Display trunking information
tsClockServer Display or set the NTP server address
tsTimeZone Display or set Time Zone
wwn Display the world wide name
errClear Clear error log
errDump Print error log (no page breaks)

Table 11-3 Switch Level Commands
Command Function
errNvLogSizeSet Resize non-volatile (persistent) error log
errNvLogSizeShow Show persistent error log configuration
errSaveLvlSet Set error save level
errShow Print error log
i Display process summary
ifModeSet Set the link operating mode for a network interface
interopMode Displays Brocade switch interoperability mode with switches
Table 11-4 Port Level Commands
Port Commands Function
portCfgDefault Restore the port configuration to defaults

portCfgEport Enable/Disable a port from becoming E_Port
portCfgGport Lock a port as a G_Port
portCfgLport Lock a port as a L_Port
portCfgPersistentDisable Persistently disable a port
portCfgPersistentEnable Enable a persistently disabled port
portCfgShow Displays port configuration settings.
portCfgTrunkPort Configure a port for trunking
portErrShow Print port error summary
portFlagsShow Display the port status bitmaps
portLogClear Clear port activity log
portLogDump Print port log (no page breaks)
portLogDumpPort Print port log (no page breaks)
portLogShow Print port activity log
portLogShowPort Print port activity log on a specified port
portPerfShow Print port throughput numbers
portRouteShow Display various routing tables for a port
portShow Print state of specified port

Chapter
SNMP Overview
12
One of the standard methods for monitoring and managing a network device is through Simple Network Management Protocol
(SNMP). It is a universally accepted protocol that is portable, lightweight, and is widely deployed. SNMP allows an
administrator to monitor the health and performance of countless devices locally or remotely via an Ethernet port. Enterprise
management software, like HP OpenView, Tivoli, CA Unicenter, and EMC Open edition that monitor thousands of devices in
an enterprise have extended their support to manage Brocade SANs. These are commercially available packages that can be
run separately by the SAN administrator so that they can get alerts, trend performance and capture details of error status on
switches separately. Within the SNMP model, a manageable network consists of one or more management stations and a
collection of agent systems that are also known as network elements. The manager communicates with the agent using SNMP
Protocol. Brocade currently supports SNMP version 1 (SNMPv1) and community based SNMP version 2 (SNMPv2C) which
means that the switches will respond to SNMP v2 commands but use SNMP v1 community names.
The data can be retrieved both actively via SNMP queries or can be received passively by SNMP traps. The switch event
messages are generally sent in the form of SNMP traps.
Figure 12-1 illustrates the Basics of SNMP structure. The SNMP agent and Management Information Base (MIB) reside on
every Brocade SilkWorm switch. SNMP resembles a client server model with the management station being the client and the
agent being the server. A management station communicates with an agent in order to alter or inspect variables. The station can
monitor an agent in active or passive mode.
In active mode a station queries the agent, actively inspecting (get) or altering (set) variables. GET, GETNEXT, and SET are
the active mode commands. Once the value has been set or obtained by the agent a reply is sent back to the station.
In passive mode an event on the agent element (switch) triggers an unsolicited message (Trap) to be sent to the station. An
SNMP agent can receive queries from one or more stations and can send traps up to six stations.
SNMP Agent - MIBS

SNMP
SNMP Server SilkWorm
Get,Get Next,Set
Reply
SNMP Query
T rap
SNMP T rap
Figure 12-1 SNMP Structure
One other SNMP feature to understand is the concept of a community. A community is a pairing of an agent and a
management station, as shown in Figure 12-2. Each community has a name and access rights. The default READ-ONLY
community is PUBLIC and the default READ-WRITE community is PRIVATE. For a management station to query a switch,
the query command must contain the name of an existing community on the switch. The most common community is public.

12 SNMP Overview
SNMP Server
SNMP Server
Agent
Agent
SilkWorm Agent
Figure 12-2 Community String Concept
The fabric switches must be properly configured either from the command line interface or using Brocade Web Tools. Web
Tools can also be accessed from the Fabric Manager Admin menu. A switch configuration setup is consists of:
• Configuring and reviewing Brocade SNMP settings
• SAN monitoring with SNMP (Please refer to Chapter 13, Brocade Fabric Watch Overview)
• Performance Monitoring with SNMP (Please refer to Chapter 14, Advanced Performance Monitor (APM) Overview)
• SNMP in Secure Fabric Mode

SNMP Overview 12
12.1. Configuring and Reviewing SNMP Settings

There are four commands that are helpful in configuring SNMP on a Brocade switch as listed:
Table 12-1 SNMP Configuration Commands

SNMP Command Function
agtcfgset Set SNMP agent configuration

agtcfgDefault Resets the SNMP agent configuration to factory defaults
agtcfgShow Lists SNMP agent configuration
snmpMibCapSet Set options for configuring SNMP MIBS/Trap capability
trackChangesSet Enables track-changes feature
12.1.1. agtcfgset
This command allows the administrator to change the configuration of the SNMP agent in the switch. The major settings in
this command are the trap destinations, Access Control List (ACL), and the value of swEventTrapLevel. Traps can be sent to
six different destinations or IP addresses. The community string should be between 2 and 16 characters. Make sure the trap
receiver has the same community string.
The ACL has six IP entries, if all entries are empty any IP address may access the switch via SNMP. Access to a specific IP
addresses can be given, as well as access to an entire subnet, by using a zero as a wildcard in the appropriate IP address octet.
Note: You must conform to the standard IP subnetting rules for this wildcard functionality to work. For example, the entire
subnet 192.168.162.0 has access to the SNMP agent on this switch, as shown in the following example.
The agtcfgset command configures the following MIB-II values on a Brocade switch.
• SysDescr - The system description (in MIB-II definition). The default value is set as “Fibre Channel Switch”.
• SysLocation - The location of the system (switch) (in MIB-II). The default value is set as “End User Premise”.
• Syscontact - The contact information for this system (switch). the default value is set as “Field Support”.
• swEventTrapLevel * - This value is set at 0 by default, implying that no swEventTrap is sent. Possible values are 0-5 (see
below)
• authTraps* - The default value for this parameter is 0 (disabled).
• SNMP Access Control (ACL) - There are six ACL (Access Control List) to restrict SNMP get/set/trap operations to hosts
under a host-subnet-area.
Example: Using the agtcfgset command to configure an ACL to limit access to one subnet.
switch:admin> agtcfgset
sysDescr = Fibre Channel Switch.
sysLocation = End User Premise
Current SNMP Agent Configuration
Customizable MIB-II system variables:
sysDescr = Fibre Channel Switch.
sysLocation = End User Premise
sysContact = Field Support.
swEventTrapLevel = 4 ( *see details below)
authTraps = 1 (On) ( *see details below)
SNMPv1 community and trap recipient configuration:

12 SNMP Overview
Community 1: Secret Code (rw)

No trap recipient configured yet
Community 2: OrigEquipMfr (rw)
Community 3: private (rw)
Community 4: public (ro)
Community 5: common (ro)
Community 6: FibreChannel (ro)
SNMP access list configuration:
Entry 0: No access host configured yet
Entry 1: 192.168.162.0
switch:admin>
12.1.2. Specifying Trap Level (swEventTrapLevel) *

The trap is generated when an event occurs whose level is at or below specified “swEventTrapLevel”. This acts as a filter to
control the traps that may be sent by the agent whenever an event is written to the error log. For example, swEventTraplevel 0
will prevent the agent from sending out any event trap. A setting of level 5 will enable all traps to be sent out. The level is set
via agtcfgset command. A level 4 setting is appropriate to receive all traps other than “debug” from agent.
All Brocade fabric switch errors, regardless of their message severity level, are stored in a persistent storage. The error severity
levels are defined as:
Error Level
0 - panic (the switch reboots)
1 - critical
2 - error
3 - warning
4 - information
5 - debug
It is important to know that all Fabric Watch traps including FRU failure are sent out as level 3 and 4 traps. The error level that
defines the error severity in switch error log must not be confused with the “swEventTrap level” setting in Table 12-2.
Table 12-2 Trap Categories/Levels
swEventTrapLevel Setting Message Severity

0 no trap sent
1 critical
2 error
3 warning
4 information
5 debug

SNMP Overview 12
12.2. Enabling Authentication on Community Names

Verification (authTraps)
The default value for the authentication trap is 0 (off). When enabled the authentication trap, authentication failure is
transmitted to a configured trap recipient in the event. The agent receives a protocol message that is not properly authenticated.
In the context of SNMPv1 and SNMPv2, this means that a request contains a community string that is not known to the agent.
Community names are used as a sort of Password in the SNMP world.
Note: The community name must be the same between the manager and the Brocade switch agent.
12.3. SNMP Access Control List (ACL)

There are six ACLs to restrict SNMP get/set operations to a host or a host subnet -area. Host subnet -area is defined by
comparing non-zero IP octets. For example an ACL of 192.168.162.0 enables access for any host that start with
192.168.162.xx.
Note: An ACL Check is disabled when all six entries contain 0.0.0.0. As described in section 12.1.1. agtcfgset on page 12-3,
the individual ACL host IP address can be specified instead specifying an ACL-subnet.

12 SNMP Overview
12.4. SNMP MIB Support Set - Executing

snmpMibCapSet Command
This command allows users to turn ON/OFF support for certain MIBS and traps. It displays current settings and then prompts
the user to change the values for each parameter. By default, support for SW-MIB, SW-TRAP and FE-MIB are all enabled in
Fabric OS 3.1/4.1 and later. During a firmware upgrade all settings are persistent and will be saved through the Fabric OS
upgrade. All MIB support can be turned off except for SW-MIB and FE-MIB. Ensure the SW-TRAP is enabled, as this is the
support required to send Brocade traps. The following example shows the default MIB setup of Brocade Fabric OS 4.2.
Example: snmpMibCapSet
SW24000_32:admin> snmpMibCapSet
The SNMP Mib/Trap Capability has been set to support
FE-MIB SW-MIB FA-MIB FICON-MIB HA-MIB SW-TRAP FA-TRAP FICON-TRAP HA-TRAP
FA-MIB (yes, y, no, n): [yes]
FICON-MIB (yes, y, no, n): [yes]
HA-MIB (yes, y, no, n): [yes]
SW-TRAP (yes, y, no, n): [yes]
FA-TRAP (yes, y, no, n): [yes]
SW-EXTTRAP (yes, y, no, n): [no]
FICON-TRAP (yes, y, no, n): [yes]
HA-TRAP (yes, y, no, n): [yes]
Notice several new MIBs in the list and their corresponding traps. MIB and trap support for specific features such as High
Availability (HA) can easily be turned on or off depending upon the switch.
SW-EXTTRAP allows Brocade SilkWorm 6400 group as well as soft serial number (SSN) information to be sent in traps.
Guideline: If using third-party SAN management software, it is advisable to turn on support for the FA-MIB as
several companies use values within that MIB to discover the presence of switches. Some later versions
of Fabric OS have the FA-MIB selected by default.

SNMP Overview 12
12.5. Track Changes Set

trackChangesSet [mode], [snmp-trap mode ]
This command enables the track changes feature that keeps track of successful and unsuccessful logins, as well as any changes
to the configuration file. There are two parameters for this feature:
• Mode - toggles track-changes on=1/off =0
• SNMP-trap - toggles the trap on=1/off=0
Example: The following example would turn on the feature and place the messages in the errlog but not send out a trap:
SW24000_32:admin> trackChangesSet 1,0
Committing configuration...done
Note: The trackchanges message placed in the errlog will be sent as specific trap 4 if swEventTrapLevel is set to severity
level 4. The trap which is specific trap 6 will be sent regardless of the swEventTrapLevel setting.
12.5.1. Viewing Default Settings

agtcfgShow - This command shows the current MIB-II variable settings and the trap level settings.
Note: There is one agent per logical switch in the SilkWorm 12000. This command is specific to the logical switch to which
you are logged in.
12.5.2. Resetting Agent Configuration

agtcfgDefault - This command allows the administrator to reset the configuration of the SNMP agent to factory defaults. The
command will prompt the user for confirmation of the action and will only proceed to reset if accepted.
12.5.3. SNMP Setup in Web Tools

The ability to change SNMP within Brocade Web Tools is functionally the same except for these two commands and their
corresponding parameters must be configured from command line interface.
• snmpMibCapSet
• trackChangesSet

12 SNMP Overview
Figure 12-3 SNMP Tab of the Switch Admin View
12.5.4. SNMP MIBs/Trap Support

MIB-II module which is typically supported by all SNMP agents on TCP/IP enabled devices or systems contains a description
of the objects hierarchy on the managed device, as well as the name (object ID), syntax and access privileges for each variable
in the MIB.
Note: For detailed information about specific MIBs, please refer to the Brocade MIB Reference Manual.
Brocade supports the following MIBs/Traps:

• SW-MIB
• FE-MIB
• FA-MIB
• FICON-MIB Brocade platform specific
• HA-MIB Brocade platform specific
• SW-TRAP
• SW-EXTTRAP
• FA-TRAP
• FICON-TRAP

SNMP Overview 12
12.5.4.1. SW-MIB
The SW-MIB is Brocade's enterprise -specific MIB. It provides the following definitions for gathering and setting specific
information in the Brocade switch that is not found on any other MIBs.
• Switch Configuration
• Switch Environmental values
• Fabric Topology
• Port status change
• Errors
• Port Performance
• Advance performance end-to-end monitoring
• Fabric Watch Traps
Note: For Fabric OS v2.6.2/3.1.2/4.2, only one SW-MIB is required and it is backward compatible with prior Fabric OS
versions.
12.5.4.2. FE-MIB
Brocade defines and supports FE MIB (RFC2837). It is open to any switch vendor to use. All the FE MIB features listed below
are also incorporated in SW MIB.
• Fibre Channel Frame values
• Port Operational Status
• Port Link failures
• Port loss of signal
• Invalid Transmitted words
Note: FIBRE-CHANNEL-FE-MIB (RFC2837) in the MIB-2 branch is supported in Fabric OS 4.0.x (and higher) and 3.0.x
(and higher). FCFABRIC-ELEMENT-MIB in the experimental branch is supported in Fabric FOS 3.0.x (and higher)
and 2.6.x (and higher).
12.5.4.3. FA-MIB
Fibre Alliance MIB was developed to implement a common method for managing heterogeneous Fibre Channel based Storage
Area Network (SAN) devices. This MIB provides many of the capabilities of the SW MIB.
• Environmental (Switch Temperature, Fans, Power supplies)
• Port values (Speed, Performance, Error)
Note: The latest FA-MIB from the Alliance is v4.0. Brocade supports version 3.0.

12 SNMP Overview
12.5.4.4. FICON-MIB
The FICON MIB module (LINK-INCIDENT-MIB) defines support for FICON in Fabric OS. This MIB addresses link
incident and link failure data for FICON hosts and devices connected to Brocade switch. This MIB is supported only in Fabric
OS v4.1.2 and later.
The descriptions of each of the MIB variables in this chapter come directly from the FICON MIB itself. The object types in the
FICON MIB are organized into the following groupings:
• Request Node Identification Data (RNID)
• Link Incident Record Registration (LIRR)
• Registered Link Incident Report (RLIR)
• Traps
Note: SNMP traps for FICON are generated when, a FICON device is added to the switch, a FICON device is removed from
the switch, a new “listener” is added (once the LIRR handshake is complete), a “listener” entry is deleted or a link
incident occurs.
12.5.4.5. HA-MIB
The HA-MIB provides information about the High Availability features of Fabric OS v4.x. This MIB is supported only in
Fabric OS v4.1 and later. The HA-MIB has a dependency on the SW-MIB. This dependency requires a management
application to load the SNMP-FRAMEWORK MIB, then the SW-MIB, followed by the Entity MIB before it can load the
HA-MIB. The object types in HA-MIB are organized into the following groupings:
• High Availability Group (CP card status, HA status)
• HA MIB Traps (CP card failover HA events)
Note: Brocade SilkWorm 12000 and 24000 switches offer high availability.
12.5.4.6. TRAP-MIB
The Trap-MIB file provides additional trap definitions that the SNMP manager station would use to decode Brocade SNMP
traps it receives. The Trap file should be compiled after the SW-MIB file is compiled in to monitoring station.
This file Provides trap decoding definitions for:
• Track changes
• Fabric Watch Threshold Traps
• Sensor events
• Switch Faults
• State changes
12.5.5. Trap Support

The trap information is received by the management application from the switch when SW, FA or FICON traps are enabled.
• SW-TRAP
• FA-TRAP
• FICON-TRAP
Note: SW-EXTTRAP - This trap is disabled by default. The Switch Serial Number (SSN) is sent in the trap when enabled.

SNMP Overview 12
12.6. SNMP in Secure Fabric Mode

When running Secure Fabric OS, there are several subtle changes to SNMP that need to be considered. The concept of Fabric
Configuration Servers (FCS) switch is introduced and discussed in more detail in the Secure Fabric OS User's Guide (part
number: 53-0000526-03). The FCS_Policy maintains the lists of Primary and Backup FCS switches. The first switch on the
top of the list is considered a Primary FCS switch that provides access to the rest of the fabric switches. It is the responsibility
of Primary FCS to propagate changes fabric-wide.
12.6.1. Community Strings Setup

The community strings are one of the management functions that are controlled by the Primary FCS (PFCS). The
configuration of these strings is performed only on the PFCS. The PFCS then propagates the community strings to all other
participating Secure Fabric OS switches.
12.6.2. SNMP MAC Policy

In a non-secure fabric mode, The Access Controls (ACLs) are configured using agtcfgset command as discussed earlier. It is
replaced with the following listed SNMP Management Access Control (MAC) Policies in Secure Fabric Mode.
• RSNMP_POLICY (read access)
• WSNMP_POLICY (write access)
These policies contain a list of host IP addresses from which connections or messages are accepted by any switch in the secure
fabric. The SNMP MAC Policies can be used to limit switch access to specific, trusted workstations in the customer's
environment, and is done so with differing read/write access levels. The SNMP host must send its request to the Primary FCS
switch to perform write operations.
Each policy has three different settings, Non-existent, Empty and some defined parameter listing of any number of IP
addresses. There are nine different combinations listed in Table 12-3. Take note of case number 4 and 7 that cause ambiguity
and are therefore illegal.

12 SNMP Overview
Table 12-3 SNMP Rules
RSNMP WSNMP [RO] Result [RW] Result
1. Non-existent Non-existent All hosts allowed to All hosts allowed to read-write [RW]
read-only [RO]
2. Non-existent Empty All hosts allowed to No host allowed to read-write [RW]

read-only [RO]
3. Non-existent Having host B All hosts allowed to Only host B allowed to read-write
in policy read-only [RO] [RW]
4. Empty Non-existent This is ambiguous, therefore it is illegal. Warning is displayed and

cannot be saved with secPolicysave.
5. Empty Empty No host allowed to No host allowed to read-write [RW]

read-only [RO]
6. Empty Having host B No host allowed to Host B allowed to read-write [RW]

in policy read-only [RO]
7. Having host A Non-existent This is ambiguous, therefore it is illegal. Warning is displayed and
in policy cannot be saved with secPolicysave.
8. Have host A in Empty Host A allowed to No host allowed to read-write [RW]

policy read-only [RO]
9. Having host A Having host B Host A allowed to Host B allowed to read-write [RW]
in policy in policy read-only [RO]
Note: If a switch running Fabric OS 2.6.0 with illegal policy combinations and tries to join into a fabric consisting of switches
running Fabric OS 2.6.1/ 3.1/4.1, the Fabric OS 2.6.0 switch will not be allowed to join the fabric. The reason for this
is the security behavior will not be consistent across the fabric. For consistent behavior, it is recommended to upgrade
to Fabric OS v2.6.1 or turn off security on that particular switch. When a switch goes from secure mode to non-secure
mode, the community names stay the same, but can be configured on a per switch basis. The ACLs that had
configuration parameters are now reinstated and the security policies RSNMP and WSNMP no longer apply.

Chapter
Brocade Fabric Watch Overview
13
This chapter provides comprehensive information to guide advance users to customize and administer Fabric Watch on
SilkWorm switches. The primary purpose of implementing Fabric Watch is to monitor the health status of the fabric elements
by continuously ensuring that they are operating within the specified threshold boundaries. In the event a safe operating limit
of an element is breached, an appropriate message is forwarded to the user by one or more pre-selected method. The severity
state appears in the message to indicate the urgency of the event to assist the administrator to take appropriate action. The
Fabric Watch accomplishes this in three steps:
1. Measuring values
2. Comparing against threshold boundary limits
3. Event generation and notification
The Fabric Watch classes of elements are predefined for Brocade SilkWorm switches. The threshold levels for these element
classes are provided in a default configuration file. The default configuration saves valuable configuration setup time for a new
user. A customize configuration setup provides a much-needed flexibility to advance users to fine-tune these thresholds to their
unique fabric environment.

13 Brocade Fabric Watch Overview
13.1. Fabric Watch Installation

Brocade Fabric Watch is a licensed product which requires a license key to enable functionality. A licenseshow command lists
the currently installed licenses. If not already enabled, obtain a license key for a switch and enable by using licenseAdd
command.
Example: Licenseshow
domain_01:admin> licenseshow
bdQQQRQez9peRhTB:
Release v4.0 license
bcybzdd9Qyedzc0Q:
Fabric Watch license
dybczSdd9uzcd0m:
Web license
Zoning license
Security license
Note: After the license key is entered, it must be initialized by executing the command Fwclassinit. This step can be skipped
if the license key was installed at the factory. After initialization, Fabric Watch will start monitoring the class elements
using the default configuration.
Example: Fwclassinit
domain_01:admin> fwclassinit
fwClassInit: Fabric Watch is updating...
fwClassInit: Fabric Watch has been updated.
Fabric Watch alarms occur only after they are activated
Example: Verifying Fabric Watch Alarms active status.

domain_01 : admin> fwalarmsfiltershow
FW: Alarms are disabled
Example: Enabling Fabric Watch Alarms

domain_01 : admin> fwalarmsfilterset 1 ( 1=Enable , 0=disable)
Commiting configuration ........... done
domain_01 : admin> fwalarmsfiltershow

Brocade Fabric Watch Overview 13
13.2. Fabric Watch Commands

Table 13-1 is a complete list of Fabric Watch management commands for the command line interface (CLI).
Table 13-1 Fabric Watch Commands
Command Function
fwAlarmsFilterSet Configure alarms filtering for Fabric Watch
fwAlarmsFilterShow Show alarms filtering for Fabric Watch
fwClassInit Initialize all Fabric Watch classes
fwConfigure Configure Fabric Watch
fwConfigReload Reload Fabric Watch configuration
fwSetToCustom Set boundary & alarm level to custom
fwSetToDefault Set boundary & alarm level to default
fwShow Show thresholds monitored by Fabric Watch
fwMailCfg Configure Fabric Watch Email Alert
fwFruCfg Configure FRU state and notification
fwSamShow Show availability monitor information
switchStatusPolicyShow Show switch status policy parameters
switchStatusPolicySet Set switch status policy parameters
switchStatusShow Show overall switch status
tempShow Show switch temp readings
sensorShow Show sensor readings
Note: Fabric Watch can be administered either from CLI or Brocade Web Tools. Brocade Fabric Manager is also capable of
accessing Fabric Watch. For detailed information about Brocade Web Tools and Fabric Manager, please refer to the
Brocade Fabric Manager User's Guide and the Brocade Web Tools User's Guide. For Fabric Watch detailed setup
information refer to the Brocade Fabric Watch User's Guide.

13.3. Fabric Watch Components

The Fabric Watch components are organized in the following order:
• Class: A Class is defined as a high level category of switch or group of fabric elements sharing the same traits that
Fabric Watch monitors. For Example, an Environment class monitors Fans, Power supplies and chassis temperature
• Area: Area represents behavior that Fabric Watch monitors. For example, environment class is consists of three
areas named Fan, Power Supply and Temperature. All fans of the system are monitored for speed; power supplies for
functional status and chassis for operating temperature limits. A behavior template is defined per area basis.
• Element: Element is an individual component of an area or switch condition that Fabric Watch monitors. For
example, an area defined by fan of environment class may have multiple fans being monitored.
The following is a complete list of class, area and elements defined by Fabric Watch.
Table 13-2 Fabric Watch Classes, Areas, and Traits
Classes Areas Elements
Environment Fan, Power Supply, Temperature Fans, chassis, Power supply
SFP Temperature, Rx/Tx performance, current, voltage All SFPs
Port Linkloss, Syncloss, Signal loss, Protocol error, Invalid words, All ports
Invalid CRCs, Rx/TX performance, State changes
E-Port *Same as port class E_ports
F/FL Port (Optical) * Same as port class Active F-ports
Fabric class E_port down, Fabric reconfiguration, Domain changes Fabric Fabric wide
Segmentation, Zone change, Fabric logins, SFP state change
Alpa Performance Invalid CRC Loop port
Monitor
EE Performance Invalid CRC, Rx/Tx performance Fabric port
Monitor
Filter Performance Custom defined Fabric port
Monitor
Security Violations: Telnet, HTTP, API, RSNMP, SES, M, Serial port, Security base policies
Front panel, SCC, DCC, Login, Invalid time stamp, Invalid
certificate, Invalid signature, SLAP failure, SLAP Bad packet, TS
out of sync, No FCS, Inc. security database, Illegal command
SAMSwitch Port downtime, Average duration, Frequency, Total up time Exports, F-Ports
Availability Monitor percentage
13.3.1. Thresholds
Thresholds identify a value or range of values to which Fabric Watch compares counter values to determine if a given element
warrants an alarm notification. Fabric Watch uses components (including traits, behavior, and alarms) to determine how and
when to check the status of a variable. Fabric Watch groups these components, identifies them as threshold, and stores them in
non-volatile memory.

13.3.2. Traits
Traits are characteristics that define a threshold. They are defined individually for each area and uniformly applied to all
elements of the area.
Table 13-3 Threshold Traits
Traits Description
Units The counter value units of measurement

Time Base A time interval (sec, minute, hour or day) to confirm an event
Low boundary Low boundary is the lowest acceptable counter values without triggering an event
High boundary High boundary is the highest acceptable counter value without triggering an event
Buffer size The buffer size is a value assigned below the threshold boundary to prevent multiple
triggering in case counter value fluctuates.
13.3.3. Behaviors
Threshold behavior defines the monitoring status (enable/disable) and event triggering behavior.
Table 13-4 Triggering Behavior s
Behavior Description
Status Enable or Disable status monitoring

Behavior type Determine Event triggering type - single or continuous mode
Behavior interval If continuous mode, determine the elapse time between continuous alarms
13.3.4. Alarms
An alarm is a response to a triggering event. The captured counter value of a fabric element, absolute or cumulative, when
compared against pre-defined threshold limits can trigger one of the listed event alarms.
Table 13-5
Alarm Description
Above A counter value rises above a high boundary
Below A counter value falls below a low boundary
Changed A counter value is changed
Exceed A counter value falls outside the acceptable range.
In -between The counter value returns to acceptable level after exceeding the defined range

13.3.5. Alarm Notification Types

Specify one or more preferred methods of Alarm notification based on the convenience and urgency. A brief discussion of
Alarm notification follows.
Table 13-6 Alarm Notification Types
Notification Description
Types
Error log A message is sent to the system Error Log that can be displayed on a telnet window using the Errshow
command. The message also carries a timestamp. It can also be monitored via Brocade Web Tools or
Fabric manager interface, as these tools derive their messages from the error log. The message text
provides information related to the type of event.
Port log Lock The port log locks to retain the detail data related to an event so it does not scroll out of the log. This
action assists administrator to examine data at later time, it does not actively alert administrator.
Note: The Error log alarm notification option for a fabric element must be enabled
Note: Typically, locking the port log is used for debugging purposes and under the guidance of your
switch support provider.
SNMPTrap Message can be configured to Send SNMP trap to user defined destination
RapiTrap Once enabled, all event information is forwarded to a designated proxy switch. The host API
automatically configures the proxy switch. The switch then forwards the alert message to a server where it
can be viewed. Third party applications using Brocade API can determine how RapiTrap alarm message is
presented to the user.
Email Alert Email alarm sends information about a switch event to a specified user email address. The email alert
notification can be set for any element event. The email also specifies the threshold and describes the
event much like an error message. You must configure Email using the fwmailCfg command (CLI) or
graphical user interface.
Guideline: Even though Email notification can be specified for any class element, it is recommended that it should
be set to receive notification only for critical alerts that require immediate attention. For example, a SAN
administrator would like to informed instances like fabric re-configuration, fabric segmentation, switch
failure or Security violation. The remaining informational and warning messages can be left directed to
system Error log.

13.3.6. Alarm Messages

The Fabric Watch alarm message provides useful information concerning the current status of the fabric element. Based on the
threshold boundary configuration, a message could fall into one of the three categories.
• Normal - The state of the element has return to Normal operating mode
• Faulty - The preset boundary threshold limit is exceeded
• Informative - The counter value has changed
In addition to Fabric Watch messages, Switch Status Policy based messages are also displayed in Brocade Web Tools and
Fabric Manager. The switch policy is configured to alert the user of any switch environmental degradation. The
switchstatuspolicyset command sets the current policy parameters for calculating the overall status of the chassis and its
components. The change in status is color-coded and can be easily spotted by the color of the switch or related component
icon.
• Down/Failed (Red) - The Faulty/Down condition set by switchpolicyset command is met. The individual component
or switch operation is critical or ceased; an immediate intervention is required to restore the service.
• Marginal/Warning (Yellow) - The switch policy marginal limit is met. The component or switch is operational in
non-redundant mode. This is a warning message to user to take corrective action to prevent future failure/down time.
• Healthy/OK (Green) - Normal Switch operating status with in the pre-defined switch policy limits.
For detail Fabric Watch configuration setup please review Brocade Fabric Watch User's Guide v 4.1.0 (Publication number:
53-0000524-03).

13.4. Theory of Operation

As described in the previous section, thresholds identify a value or range of values to which Fabric Watch compares measured
counter value to determine if a given element warrants an alarm notification. They consist of traits, behavior and alarms. These
component settings significantly influence the triggering behavior of Fabric Watch; therefore it is extremely important to have
a good understanding of their working relationship in order to achieve consistently predictable results. Please remember that
each element of the fabric has unique measurement requirements that are set based on the following listed criteria.
• Counter values
• Time Base Event
• Threshold boundary limits
13.4.1. Counter Values

A Counter represents the value of a behavior variable. Counter value can be cumulative or absolute. A counter may represent
the total number of times that a given error occurred in a defined Time Base (cumulative) or it may represent the current value
of a particular behavior such as temperature or fan speed (absolute). Fabric Watch compares a counter value to a preset
threshold boundary limit to determine an event; therefore it is important to understand whether the counter value is cumulative
or absolute so that the appropriate threshold boundaries and Time Base can be defined for triggering an event.
13.4.2. Time Base Event

1. A Time Base set to none means that an absolute value of the monitoring counter shall be compared against a threshold
boundary level. The event is triggered as soon as the absolute value of the measuring counter exceeds the threshold
boundary.
2. A Time Base set to other than none (seconds, minute, hour or day) do not use the absolute value of the counter, instead it
calculates the rate of change for the specified Time Base interval and compare this value to threshold boundary limit. For
example, a Time Base of minute calculates the counter value difference between two samples a minute apart. Then the
difference (Current counter value - Last value) over one minute period is compared against the preset thresholds boundary
limit. When an event trigger is Time Base defined, there are two possible outcomes.
• The event is triggered only if the difference in the counter value exceeds the preset threshold boundary limit.
• The current Counter value may be exceeding the threshold but after subtracting last value if the rate of change is
lower than the threshold limit, event will not trigger.
The following examples illustrate both Time Base events.
Example: 1
A sample is obtained and placed in a circular buffer. A high limit threshold value of 2 is specified to trigger an event to detect
a security violation. A Time Base of minute is defined by default. An event will trigger only if the calculated rate of change
within the specified interval period (minute) is greater than the threshold high limit. As illustrated on 10th sample, the counter
value changes form 0 to 1; hence calculated rate of change is 1 per minute. After the 13th sample the rate of change is 2 per
minute. The rate of change must be at least 3 per minute in order to meet the event-triggering requirement of high limit of 2,
which is met on 19th sample.

Absolute Counter value
Event trigger
3
2
Measurement units
1 High lim it=2

0
Normal Operating range
Measurement
sample
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Rate of change/minute =(1-0)

T ime base =1 minute
Rate ofchange/minute =(2-0)
Rateof change/minute=(3-0)
Figure 13-1 Triggering Event

Example: 2
Consider the following example that illustrates a calculated rate of change in counter value always less than or equal to a high
threshold limit of 2. At 10th sample the rate of change is one per minute, at 14th, 21st and 26th sample the rate of change
remains equal to high limit of 2. The event will never trigger in this case even though the absolute value of the counter reaches
4 that is well above the triggering limit.
Absolute counter value
4
3
2
Measurement units
1 High limit=2
0
Measurement
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 sample

T ime base 1 minute
Rate of change/minute =(2-0)
Figure 13-2 Non-Triggering Event
The examples above show a Time Base event triggering must be applied selectively as each class element has unique
requirements.
Guideline: Do not use Time Base where absolute Counter value measurement is required such as monitoring
environmental class elements.
Guideline: When an absolute counter value is monitored, an event triggering can be defined either by setting up
high or low thresholds or any change in the counter value. The latter method must be used when the
element traits are such that a change in counter value is least expected and not very frequent. For
example, a power supply failure can be triggered on Changed value
Guideline: Consider using appropriate Time Base where frequency of an event occurrence is high with in the
defined Time Base interval. For example, it is inappropriate configuring Fabric Watch Security log in
violation on each occurrence. It is quite possible an authorized user may have entered password or user
id incorrectly one or more time while trying to login. To avoid unnecessary messages, a Time Base
trigger can be defined after a number of unsuccessful retries.

13.5. Defining Threshold Boundary Limits

The captured counter value of a fabric element absolute or cumulative must be compared against threshold limits in order to
trigger an event. This section describes the threshold boundary limits and how to define a buffer zone can impact the triggering
of an event. The Fabric Watch event types are listed below:
• Above: A counter value rises above a high boundary
• Below: A counter value falls below a low boundary
• Changed: The value of a counter is changed
• Exceed: A counter value rise or falls below a range of acceptable values
• In -between: The counter value returns from a value outside of an acceptable range to a value within the acceptable
range
The following figures illustrate the number of ways an event trigger threshold can be setup based on the traits of a fabric
element.
13.5.1. Defining the Threshold for the Above Trigger

This trigger threshold is set for an element that requires only the high limit monitoring. An event is triggered immediately after
the counter value reaches above the high threshold limit (event 1). A buffer zone can be defined with in the operational limit of
an element to suppress multiple triggering incase the counter value fluctuates above high limit and buffer zone. In the event
counter value returns to normal operating range after falling below the buffer zone, event 2 is triggered. This event can be
setup to notify user of returning to normal operating status.
1 Above T rigger Buffer

Measurement units
High limit
Operating upper limit 2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Sample period (~6 sec)
Measurement Sample
Figure 13-3 Above Trigger Event with Buffer zone
13.5.2. Defining Threshold for Below Trigger

The below threshold trigger threshold works exactly the same way as described above except the trigger is generated crossing
the low boundary. If a class element must operate with in a defined range then both limits can be set.
Normally, the low boundary is specified in conjunction with upper limit to reset a counter to re-arm it for the next event. For
example, an event trigger will occur when high limit is exceeded. Unless this counter is reset by crossing the below limit, the
event will not trigger again.

As Figure 13-4 illustrates, a high threshold limit is set to define an above event trigger. A measured counter value exceeding
this limit will generate an event notification. However if you are interested in finding out how often the event takes place, the
counter value must be reset to re-arm it for next event detection. The low limit event (2) will reset the counter value for the
next event.
Above T rigger
Measurement units
High limit
Threshold boundary setup

2
Lowlimit
Below T rigger
Sample period (~6 sec)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Measurment Sample
Figure 13-4 Below Event Trigger Configured to reset counter value
13.5.3. Defining Threshold for Changed Trigger

The element counter is updated on each sample period. An event can be set upon detecting a change in the counter value
between two-sample periods regardless of high or low threshold settings. The comparison elapse time can be defined by Time
Base. Figure 13-5 illustrates generating events each time counter value is found changed within the normal operating range.
A change in counter value does not necessarily constitute a failure. Event triggering on a change value must be used with some
discretion where a change in value is least expected. A Fabric element whose value is changing frequently will generate
repeated messages of little or no value to a user. For example, this trigger type is appropriate for FRU failure. It is not
appropriate for temperature monitoring as counter value changes more often with in valid temperature range.
3 Changed trigger
Counter=2
Counter=1
Counter=0
Measurement units
No limits defined
Normal Operatingrange

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Measurement Sample
Figure 13-5 Changed Event Trigger

13.5.4. Defining Threshold for Exceeded Trigger

This Alarm status is applicable to entire operating range defined by high and low limit of the element. When set, the event
trigger is generated in both cases either crossing the high or low boundary. It indicates that the counter value has exceeded one
of the threshold limits. This setup is useful for monitoring an element that operates with in a specified range such as Fan. An
operating range of a fan is defined by high and low limit including buffer zone. Operating outside these limit is indication of
faulty fan and exceeding either high or low limit will generate an exceed trigger (4). The same results can be obtained by
setting above and below event triggering independently.
4 Above T rigger
Measurement units
High limit

T hreshold boundary
Lowlimit
Below T rigger 4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Measurment Sample
Figure 13-6 Exceeded above or below Threshold Boundary
13.5.5. Defining Threshold for In-between Trigger

Fabric Watch event trigger is usually set to notify user of a warning or failure condition. The in-between trigger can be defined
to receive a notification of fault recovery. A buffer zone must be setup in order to generate an in-between event. For example,
when measuring a port performance, crossing high threshold limit triggers above event sending a warning message to a user. It
is quite possible that limit is crossed for a brief period that is not a cause for an alarm. An in-between Trigger (5) indicates that
the port performance is within acceptable range. The length of the time period can be obtained from the timestamp of both
triggering events to evaluate the port performance over the length of period. The in-between event triggering is appropriate for
the following cases.
• Verification of successful recovery from a faulty condition is needed.
• The counter value must be reset for the next event.
• Identifying an element that is consistently operating under marginal condition.
5 In-BetweenT rigger
Buffer
Measurement units
High limit
Normal Operating range T hreshold Boundary
Lowlimit
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Sample period (~6 sec)
Measurment Sample
Figure 13-7 In-between Trigger Event

13.6. Advanced Thresholds Configuration (CLI)

Fabric Watch provides a default configuration for each element. The default configuration is always preserved to provide user
an option to revert to original settings any time; therefore users are not allowed to modify default settings. However a user can
always modify custom settings from the Advanced Configuration menu. To customize Fabric Watch thresholds and alarm
notification, first update the thresholds of the column label custom and then configure Fabric Watch to use the custom
thresholds instead of default
The “Advanced Configuration setup” example discusses the user selectable parameters.
Example: Advance Configuration setup
1. A Fabric Watch fwconfigure command lists the element classes. You must select a class from the list.
2. Select an AREA (AREA represents the behavior that Fabric Watch monitors).
3. Selecting an area lists all the classes being monitored. For example, there may be one or more E_ports configured on this
switch, when selecting an E-port class they are represented via index number.
4. To customize Fabric Watch configuration for thresholds boundary and Alarm level select advance configuration option.
Figure 13-8 describes the Advanced Configuration parameters that can be updated by user.
Index Threshold Name BehaviorType BehaviorInt
Element 0 eportRXPerf000 Triggered 1

Trigger or continuous
Measurement unit Threshold boundarylevelis set at : Default T hreshold level

Default or Custom
Default Custom
Counter value check interval Unit KB/s KB/s
(second, minute, hour, day) Timebase
T hresholdlimits
Low 60000 0
High 80000 0
BufSize 0 0
Alarm level
Threshold alarmlevelis setat : Custom default or Custom
Errlog-1, SnmpTrap-2, PortLogLock-4

RapiTrap-8, EmailAlert-16
Valid alarmmatrix is 31 (1+2+4+8+16)
Any one or combination
Default Custom of Alarm setting is valid
Changed 0 0
Exceeded 0 0
Send Message to Error log
Below 1 0 for above and in-between
Above 1 1 trigger events
InBetween 0 1
Figure 13-8 Advanced Configuration Menu

13.6.1. Behavior Type

• Trigger mode - Triggering event occurs only once after trigger condition is met. Select to avoid multiple triggering of
the same event.
• Continuous - Triggering event occurs on Time Base interval as long as trigger condition persists. Configure only if
continuous monitoring is required. (For example, if a Time Base is minute, the event will trigger every minute
providing repetitive messages)
13.6.2. Threshold Boundary Level

Determine applicable threshold type Custom or Default. You must define Custom values.
• Measurement unit - Define appropriate measurement unit applied to measured value
• Time Base - Defines a period specifying how often the change in counter value will be examined. (second, minute,
hour, day) if applicable to the trigger threshold type otherwise leave it alone.
• High /Low limit - Specify applicable High and /or Low limit that an element must not exceed.
• Buffer - Specify a buffer zone if applicable to suppress multiple triggering.
13.6.3. Threshold Alarm Level

A valid Alarm Matrix of 31 (1+2+4+8+16) indicates that all alarms are available for this element. Table 13-7 lists a few
popular Alarm level combinations enabling one or more alarm notification.
Table 13-7 Alarm Matrix
Typical Email Alert RapiTrap PortlogLock SNMP Trap Error Log

Alarm level (16) (8) (4) (2) (1)
1 1
2 2
3 2 1
5 4 1
7 4 2 1
17 16 1
19 16 2 1
Example: Alarm level= 19 (16+2+1) Enables Email + SNMP Trap + Error log

13.6.4. Advanced Configuration Command Set

The following is a complete set of commands to customize Fabric Watch configuration.
Table 13-8 Fabric Watch Commands
1: change behavior type 11: change threshold alarm level
2: change behavior interval 12: change changed alarm
3: change threshold boundary level 13: change exceeded alarm
4: change custom unit 14: change below alarm
5: change custom time base 15: change above alarm
6: change custom low 16: change in-Between alarm
7: change custom high 17: apply threshold alarm changes
8: change custom buffer 18: cancel threshold alarm changes
9: apply threshold boundary changes 19: return to previous page
10: cancel threshold boundary changes
Commands 1 through 10 are used to modify the behavior and Threshold boundary level
(1) The user must change threshold boundary level to custom (command 3).
(2) After modifying the Custom Threshold parameters (command 1,2 and 4-7), the user must execute “apply threshold
boundary changes” to make the changes effective (command 9).
Commands 11 through 19 are used to modify the Thresholds Alarm level
(1) When customizing the Alarm levels, the user must select custom from Change threshold alarm level (command 11).
(2) After modifying the alarm thresholds (commands 12-16), a user must execute “apply threshold alarm changes” to
make changes effective (command 18).
Guideline: Set custom values for a class element only if they differ from the default. This will preserve system
memory. For example, you can customize threshold boundary limits of an element but if you are
satisfied with the alarm threshold settings leave them as default.
Guideline: While making changes disable the threshold of the element from the Advance Configuration menu to
prevent unnecessary false reporting.
Guideline: It is recommended that the threshold parameters for any element class be defined uniformly for all fabric
switches when customizing it. This can be accomplished by creating a template of the customized
Fabric Watch settings from the file created with the configupload command. This template can then be
loaded, via configdownload to other switches in the fabric. Fabric Manager simplifies this process and
has tools to perform these activities. Alternatively the command line interface can also be used.
Note: In a mixed Brocade fabric (i.e., Brocade SilkWorm 12000, 24000, 3800, 3900) a separate Fabric Watch configuration
must be defined for each switch environment class. This is because the operating temperature range, fan speed and
number of required power supplies for each product is slightly different.

13.7. Advanced Threshold Configuration Using

Brocade Web Tools
Brocade Web Tools simplifies the Fabric Watch Advance configuration setup process as described below.
1. Using a web browser, access the fabric switch by entering the switch IP address. Example: http: //192.162.173.xxx
2. After a switch discovery is completed, select the Watch button located at the bottom row of the switch screen. Provide the
necessary user ID and password to obtain switch access.
3. The following Fabric Watch window for the selected switch domain appears with Alarm Notification, Thresholds
Configuration, and Email Configuration menu tabs.
4. Select a class of element from the left pane. (Example: E-port class)
5. Select Threshold Configuration tab.
6. Select appropriate area from the menu of the select area window. (Example: Rx performance)
7. Select boundary level to Custom.
Figure 13-9 Advance configuration Web Tools setup screen
7. Set or modify existing threshold boundary parameters.

(Example: Custom Performance monitoring high limit is set 190 MB/sec, Buffer=10 MB)
8. Specify Alarm threshold type from the first column of Alarm setting section of the screen.
(Example: above and in-between boxes are checked)
9. Select the Notification method for each Alarm type by checking the appropriate box.
(Example: Both above and in-between alarm notification is forwarded to Error log and Email)
10. After setting all required parameters, check Save configuration to switch box and depress Apply button to save changes in
a non-volatile memory.
11. The above steps can be repeated to define all areas of E-port class or other Fabric Watch classes.

13.8. Fabric Watch Best Practices

A default Fabric Watch configuration is available for the purpose of saving setup time. As a user gains familiarity with Fabric
Watch features, Fabric Watch can be tailored to suite the fabric environment. The Fabric Watch custom settings provide an
advanced user the flexibility of re-defining boundary threshold limits and alarm notification methods. The basic concept of
Fabric Watch is to monitor the health of an element by sampling status, comparing the sample data and, if found outside the
threshold limits, notify user of the event by one or more selected methods. Since Fabric Watch monitors a variety of classes
and class elements, each element with a unique trait must be evaluated prior to defining custom thresholds to meet a specific
objective. This section discusses some of the changes that one should consider implementing to improve the overall efficiency
of Fabric Watch. A summary of the custom configuration for a Brocade SilkWorm 12000/24000 is provided in section 13.9.
Default and Custom Configuration Tables on page 13-23.
The customization is recommended to achieve the following objectives.
• Selecting appropriate message delivery method for critical and non-critical events.
• Selecting appropriate thresholds and alarm levels relevant to each class element.
• Define appropriate Time Base event triggering based on the class element traits.
• Eliminating message delivery that has a very little or no practical value to administrator.
• Consolidating multiple messages, generated from a single event.
13.8.1. Class Based Recommendations

The default setting for each class element is evaluated for its overall effectiveness in the user's fabric environment. The
customization is discussed below for each class and recommendations are made where applicable to improve the overall
efficiency of early warning and messaging system.
Guideline: An administrator must consider including Email notification on selected events that are critical in nature
and require immediate attention. As described in section 13.8.1.4. Fabric Class on page 13-19, email
notification is recommended only for fabric reconfiguration and/or segmentation. The remaining fabric
class elements should be configured to only forward messages to the error log.
13.8.1.1. Environment Class

Class Element: Temperature
A rise in temperature above the safe operating limit is possible either from a fan failure, blocked airflow path or improper
room temperature. Exceeding the operating limit can completely shut down the blade circuitry, thus hindering the fabric
operation. The default high threshold limit for SilkWorm 12000 is configured at 75o Celsius. An event is triggered upon
exceeding this limit and a blade circuitry is shutdown simultaneously giving administrator no early warning to take corrective
action.
Objective-1: The administrator must be notified prior to temperature reaching 75o Celsius so that a corrective action can be
initiated to avoid complete shutdown.
Solution: Consider setting the upper threshold limit of 65o Celsius with a buffer-zone defined as 10o Celsius. This
configuration provides a wide operating temperature range 0-65o Celsius. Triggering an above event exceeding 65o Celsius
serves as an advanced warning indicating that a switch is operating at the high end of the temperature range. The advanced
warning allows for time to investigate and initiate corrective action while switch is operational.
Objective-2: To obtain immediate response in case of FRU failure (Port blade, fan, power supply)
Solution: Email notification must be enabled for a timely response.

13.8.1.2. SFP Class (Accept Default)
The default SFP Class element thresholds are set based on the manufacturer specification. The informative message is sent to
Error log by default. This category is useful to isolate marginal or sensitive SFP.
Note: SFP class monitors smart SFP(s) only.
13.8.1.3. Port Class

The port class monitors all switch ports regardless of their configuration status. A configured port is further monitored by
either E-Port Class or Fabric port class applying the stringent thresholds. The default thresholds for Port class element are
extremely lenient to the extent that the probability of triggering an event is very low to none. The E_port or Fabric port Class,
monitoring the same elements with a very low threshold level will trigger the event before port class.
Objective-1: To setup Port class element thresholds for the purpose of monitoring port health on long term basis.
Solution: For example, monitoring a port experiencing link loss or sync loss type errors over a longer period can determine the
severity of the problem. The accumulative counter values obtained over a longer period (day) can assist the administrator to
make a correct decision by determining the frequency of the error.
Objective-2: Minimizing the number of messages
Solution: The performance monitoring is available on active ports via E-Port and Fabric Port class. It does not serve any
useful purpose therefore can be disabled to avoid repetition.
13.8.1.4. Fabric Class

Class element: Fabric re-configuration, Segmentation
A Fabric wide event that impacts an ongoing operation requires immediate attention. A common denominator in all fabric
class elements is fabric re-configuration. A fabric reconfiguration has two possible out comes. After a brief fabric disruption
either the fabric recovers or segments. Given the urgency of the response time an Email alarm notification must be received by
an administrator. At the same time receiving an Email notification for each element of the fabric class can be overwhelming.
Objective-1: Email Notification setup on Fabric reconfiguration.
Objective-2: Minimize the number of Email messages.
Solution: The fabric segmentation is the worst-case outcome of a fabric reconfiguration. However a frequent fabric
reconfiguration also requires a prompt response therefore, an Email notification is recommended for either Fabric
reconfiguration and/or Segmentation. Please refer to section 13.8.1.9. Email Notification Setup on page 13-21.
13.8.1.5. E-port Class

Class element: Rx and Tx performance
Objective-1: To monitor throughput of an E-Port, capable of delivering a maximum of 200 MB/sec for the purpose of
identifying congestion point and fine-tuning the fabric.
Solution: Receive an early alarm notification from Fabric Watch in the event an E-Port is operating consistently in the upper
range of the bandwidth. Once an alarm is received the port or switch aggregate bandwidth can be monitored from Brocade
Basic Performance monitoring tool over a longer period. If an E-Port is found to be consistently operating in the upper range,
the administrator can initiate further actions to avoid the bandwidth saturation.
Class element: Sync loss, protocol err, invalid words, invalid CRC
Objective-2: Reducing the number of Error Log messages.

The current notification thresholds are set to forward message crossing both above and lower threshold limits. An above
threshold crossing indicates an error status and dropping below limit indicates it has returned to its normal state. The current
configuration is setup such that a recovery message is followed by a failure message. When a port is disabled it generates a
stream of messages, sync loss, link loss etc. In most cases the message are overwhelming. The same way recovery produces
the same number of informative messages. The recovery messages that provide very little or no useful information to
administrator can be selectively disabled, thus reducing the overall number of messages.
Solution: For example, except link loss and signal loss elements, the recovery message notification can be disabled for
remaining E-port elements. Please refer to Table 13-9 and Table 13-10.
13.8.1.6. Fabric F/FL (Optics) Class

Fabric Port set up consideration same as E_port class. Please refer to Table 13-9 and Table 13-10.
13.8.1.7. Security Class

There are a total of 21 Security elements that can be monitored for security breaches. Again, forwarding an Email on each
security element can be overwhelming and serve no useful purpose; therefore, it must be limited to address a specific concern
in your fabric environment. For example, a security conscious administrator may want to receive immediate notification in
case an unauthorized login is attempted.
Objective-1: A system administrator may select to setup Email notification to identify un-authorized logins
Class element: login violation, serial violation
Solution: The default setup or all security elements trigger an event on 3rd violation attempt exceeding the high limit of two
per minute. All three violations must occur with in one-minute period (Time Base = minute). The event will not trigger for the
violation rate less than three per minute. Currently, the security violation message by default is forward to Error log and SNMP
trap. Unless the error log is being monitored on real time basis, the notification is irrelevant. A security breach warrants a real
time notification to respond in time. This can be accomplished by setting up Email notification on areas of concern and the
policies implemented. The security elements that do not warrant immediate response can be left to default.
Note: Response to a security class message depends on user policy. The user policies must be defined and activated for the
security elements enabled on Fabric Watch.
13.8.1.8. Advanced Performance Class (Accept Default)

Brocade Performance Monitor offers Basic and Advanced performance monitoring. There is no licensing requirement for
using the basic functionality of Performance Monitor. Brocade Advanced Performance Monitor is a licensed product. Please
refer to Chapter 14, Advanced Performance Monitor (APM) Overview for details.

13.8.1.9. Email Notification Setup
The following example shows an E-Port class trigger event forwarding an email message to specified user's address when
email notification option is enabled.
1. When the email box is checked for alarm notification for a class, a valid forwarding email address must be provided. The
mailing status must be enabled from the Email Configuration tab. The Apply button updates the changes
Figure 13-10 Email Configuration Tab
Note: An Email address is applicable to all areas with in a class. A different forwarding email address can be specified for
each class basis. For example, the email address is applied to all E-port class areas. However, the email notification is
enabled on area basis. Figure 13-9 shows Email Notification is only turned on area name Rx performance of the E-Port
class.
2. Configure the Email Domain Server and Domain name on the switch (Figure 13-11).
a. Using Web Tools access the fabric switch by entering IP address of the switch. (for example: http: //192.162.173.xx)
b. After a switch discovery is completed, select the Admin button. Provide the necessary user ID and password to
obtain switch access.
c. Setup the Email address, Domain name server (s), Email forwarding domain name etc. as shown in the following
screen. Click Apply to enable the changes.

Figure 13-11 Email Configuration in Web Tools

13.9. Default and Custom Configuration Tables

Table 13-9 is a summary of the recommended Custom Fabric Watch settings for a SilkWorm 12000/24000. Table 13-10 is a
summary of the default configuration for a SilkWorm 12000/24000. By default, the Error Log alarm notification is always
enabled. The SNMP and Email notification methods are optional. The use of optional methods must be considered for
effective management based on their availability in your fabric environment. If using third party fabric management software
packages such as Open View, Tivoli, Open Edition or CA Unicenter you must enable SNMP notification.
Table 13-9 Custom Configuration for SilkWorm 12000/24000

Class Elements
Changed (Inform.)
In-betw.(Marginal)
Exceeded (Faulty)
(Faulty)
(Faulty)
High Threshold
Trigger Enable
Low Threshold
Measure Units
Email Alert
Time Base
SnmpTrap
Error Log
RapiTrap
Above
Buffer
Below
Threshold Settings Event Triggers Alarm Notification
1. Environmental
1. Temperature * C 0 65 10 * * * * opt *
2. Fan * rpm 2k 3.4k 3 * * * * opt *
3. Power Supply * - 1 0 * * * * opt *
2. SFP
1. Temperature * C -10 45 3 * * opt opt opt
2. RX Power * uW 0 5k 25 * * opt opt opt
3. TX Power * uW 0 5k 25 * * opt opt opt
4. Current * uI 0 50 1 * * opt opt opt
5. Voltage * uV 3.15k 3.6k 10 * * * opt opt opt
3. Port class
1. Link loss * err day 1 60 * * opt opt opt
2. Sync loss * err day 1 120 * * opt opt opt
3. Signal loss * err day 1 120 * * opt opt opt
4. Protocol Error * err day 1 60 * * opt opt opt
5. Invalid words * err day 1 60 * * opt opt opt
6. Invalid CRCS * err day 1 60 * * opt opt opt
7. RX Perform. di
8. TX Perform. di
9. State changes * err day 1 120 * * opt opt opt
4. Fabric
1. E_Port down * * * * opt opt
2. Fabric re-config. * * * * opt *
3. Domain change * * * * opt opt


Class Elements
Changed (Inform.)
Exceeded (Faulty)
In-betw.(Marginal)
(Faulty)
(Faulty)
High Threshold
Trigger Enable
Low Threshold
Measure Units
Email Alert
Time Base
SnmpTrap
Error Log
RapiTrap
Above
Buffer
Below
4. Segmentation * * * * opt *
5. Zone changes * * * * opt opt
6. Fabric QL di
7. Fabric Logins di
8. SFPstate change di
5. E-port
1. Link loss * err min 1 5 * * * * opt opt
2. Sync loss * err min 1 5 * * opt opt opt
3. Signal loss * err min 1 5 * * * * opt opt
4. Protocol Error * err min 1 5 * * opt opt opt
5. Invalid words * err min 1 5 * * opt opt opt
6. Invalid CRCS * err min 1 5 * * opt opt opt
7. RX Perform. * kbs sec 0 190k 10k * * * opt opt opt
8. TX Perform. * kbs sec 0 190k 10k * * * opt opt opt
9. State changes * err min 1 15 * * * * opt opt
6. Fabric Ports
1. Link loss * err min 1 5 * * * * opt opt
2. Sync loss * err min 1 5 * * opt opt opt
3. Signal loss * err min 1 5 * * * * opt opt
4. Protocol Error * err min 1 0 * * opt opt opt
5. Invalid words * err min 1 0 * * opt opt opt
6. Invalid CRCS * err min 0 5 * * opt opt opt
7. Rx Performance * kb sec 0 190k 10k * * opt opt opt
8. Tx Performance * kb sec 0 190k 10k * * opt opt opt
9. State changes * err min 2 15 * * * *
Advance Performance monitoring license is required

7 Alpa Perform.
1. Invalid CRCs
8. EE Perform
1. Invalid CRCs
2. Rx Performance
3. Tx Performance

Class Elements
Changed (Inform.)
Exceeded (Faulty)
In-betw.(Marginal)
(Faulty)
(Faulty)
High Threshold
Trigger Enable
Low Threshold
Measure Units
Email Alert
Time Base
SnmpTrap
Error Log
RapiTrap
Above
Buffer
Below
9 Filter Perform
1. Custom define
10. Security
1. Telnet * err min 1 2 * * * opt opt
2. HTTP violation * err min 1 2 * * * opt opt
3. API violation * err min 1 2 * * * opt opt
4. RSNMP violation * err min 1 2 * * opt opt opt
5. WSNMP violation * err min 1 2 * * opt opt opt
6. SES violation * err min 1 2 * * opt opt opt
7. MS violation * err min 1 2 * * opt opt opt
8. Serial violation * err Hr 1 2 * * * opt *
9. Front Panel violation * err min 1 2 * * * opt *
10. SCC violation * err min 1 2 * * * opt opt
11. DCC violation * err min 1 4 * * * opt opt
12. Login violation * err Hr 1 7 * * * opt *
13 Inv Time stamp * err min 1 2 * * opt opt opt
14. Inv. signatures * err min 1 2 * * opt opt opt
15.Inv. certificate * err min 1 2 * * opt opt opt
16. FCAP Failure * err min 1 2 * * opt opt opt
17. FCAP bad packet * err min 1 2 * * opt opt opt
18. TS out of sync * err min 1 2 * * opt opt opt
19. No-FCS * err mon 1 2 * * opt opt opt
20. Inc. data base * err min 1 2 * * opt opt opt
21. Illegal Command * err min 1 2 * * opt opt opt
11 SAM
1. Down Time * % 0 5 * * opt opt opt
2. Total Uptime * % 95 100 * * opt opt opt
3 Avg. Duration * 0 5 * * opt opt opt
4. Frequency * 0 1 * * opt opt opt
* = Enable; di = Disable; Opt = Optional

Note: SNMP Trap, Rapi Trap, Email alert notifications settings are user specified. The SNMP Trap notification when enabled
must be limited to a number of selected messages as recommended.
Note: For information concerning the environmental class specification, please refer to the appropriate Brocade SilkWorm
Hardware Reference Guide.
Note: Finisar GBIC/SFP specifications are available on the Finisar web site: www.finisar.com

Table 13-10 Default Configuration for SilkWorm 12000/24000
Class Elements
Changed (Inform.)
Exceeded (Faulty)
In-betw.(Marginal)
(Faulty)
(Faulty)
High Threshold
Trigger Enable
Low Threshold
Measure Units
Email Alert
Time Base
SnmpTrap
Error Log
RapiTrap
Above
Below
Buffer
Threshold Settings Event Triggers Alarm Notification
1. Environmental
1. Temperature * C 0 75 10 * * * *
2. Fan * rpm 2k 3.4k 3 * * *
3. Power Supply * - 1 0 * * *
2. SFP
1. Temperature * C -10 45 3 * * *
2. RX Power * uW 0 5k 25 *
3. TX Power * uW 0 5k 25 * * *
4. Current * uI 0 50 1 * * *
5. Voltage * uV 3.15k 3.6k 10 * * *
3. Port class
1. Link loss * err min 1 60 * *
2. Sync loss * err min 1 120 * *
3. Signal loss * err min 1 120 * *
4. Protocol Error * err min 1 60 * *
5. Invalid words * err min 1 60 * *
6. Invalid CRCS * err min 1 60 * *
7. RX Perform. * kbs sec
8. TX Perform. * kbs sec
9. State changes * err min 1 120 * *
4. Fabric
1. E_Port down *
2. Fabric re-config. *
3. Domain change *
4. Segmentation *
5. Zone changes *
6. Fabric QL *
7. Fabric Logins *
8. SFPstate change *
5. E-port
1. Link loss * err min 1 5 * *

Class Elements
Changed (Inform.)
Exceeded (Faulty)
In-betw.(Marginal)
(Faulty)
(Faulty)
High Threshold
Trigger Enable
Low Threshold
Measure Units
Email Alert
Time Base
SnmpTrap
Error Log
RapiTrap
Above
Buffer
Below
7. RX Perform. * kbs sec 120k 209k * *
8. TX Perform. * kbs sec 120k 209k * *
6. Fabric F/FL Ports

1. Link loss * err min 1 5 * * *
7. Rx Performance * kb sec
8. Tx Performance * kb sec
Advance Performance monitoring license is required

7 Alpa Perform.
1. Invalid CRCs
8. EE Perform
1. Invalid CRCs 1 10
2. Rx Performance
3. Tx Performance
9 Filter Perform
1. Custom define
10. Security
1. Telnet * err min 1 2 * *
2. HTTP violation * err min 1 2 * *
3. API violation * err min 1 2 * *
4. RSNMP violation * err min 1 2 * *
5. WSNMP violation * err min 1 2 * *

Class Elements
Changed (Inform.)
Exceeded (Faulty)
In-betw.(Marginal)
(Faulty)
(Faulty)
High Threshold
Trigger Enable
Low Threshold
Measure Units
Email Alert
Time Base
SnmpTrap
Error Log
RapiTrap
Above
Buffer
Below
6. SES violation * err min 1 2 * *
7. MS violation * err min 1 2 * *
8. Serial violation * err min 1 2 * *
9. Front Panel violation * err min 1 2 * *
10. SCC violation * err min 1 2 * *
11. DCC violation * err min 1 4 * *
12. Login violation * err min 1 2 * *
13 Inv Time stamp * err min 1 2 * *
14. Inv. signatures * err min 1 2 * *
15.Inv. certificate * err min 1 2 * *
16. FCAP Failure * err min 1 2 * *
17 FCAP bad packet * err min 1 2 * *
18. TS out of sync * err min 1 2 * *
19. No-FCS * err mon 1 2 * *
20. Inc. data base * err min 1 2 * *
21. Illegal Command * err min 1 2 * *
11. SAM
1. Down Time * % 0 5 * *
2. Total Uptime * % 95 100 1 * *
3 Avg. Duration * 0 5 * * *
4. Frequency * 0 1 * * *


Chapter
Advanced Performance Monitor (APM) Overview
14
Brocade Advanced Performance Monitoring is a tool which helps SAN managers measure the efficiency of their SAN
resources. Trends and patterns can be read using charts and graphs.
The overall throughput performance of an application is entirely dependent on the configuration of the host and storage
hardware being utilized. A SAN configuration providing connectivity to host and storage devices must not be a performance
factor. Ideally, a storage port providing connectivity to one or more hosts must be able to satisfy the aggregated bandwidth
requirement of all the hosts. However if not carefully planned, a congestion condition may exist when a port is
over-subscribed. Advanced Performance Monitoring helps identify and correct this over-provisioning condition. Other SAN
management areas which can benefit from the monitoring system include:
• Capacity Planning
• SAN Performance tuning
• End-to-end visibility into fabric
• Increasing productivity via Pre-formatted Reports
Fabric T uning
Input =Output
Hosts
End-to-End Monitoring
Host SID Storage DID
Figure 14-1 End-to-End Monitoring
Performance Monitoring is administered through either telnet commands or Web Tools. A Web Tools license must be activated
on the switch in order to use Web Tools.
Note: The Basic Performance monitoring is enabled by default, however to activate advance features an Advanced
Performance Monitor license key is required.

14 Advanced Performance Monitor (APM) Overview
14.1. Performance Monitoring via Web Tools

Web Tools is the most convenient way to view, customize, or monitor Brocade APM. Launch Web Tools either directly or
access it via Brocade Fabric Manager client workstation.
1. Select an appropriate switch from the SAN Element window and then select a switch view from the Action menu.
2. From the switch view select Perf tab. (The performance window opens.)
3. The Actions or Performance Graphs tabs let you perform the tasks listed Table 14-1.
Port PerformanceMB/sec
PerformanceMonitorwindow
Switch Ports (Slot/port) T hroughput scale MB/Sec
Figure 14-2 Performance Monitor
Table 14-1 Performance Monitor Menus and Functions
Menu Functions
Actions Menu Save Current Canvas Configuration
Display canvas configurations
Display resource Usage
Print all graphs
Basic Monitoring menu Graphs
Port throughput Display port performance MB/sec
Switch Aggregate Throughput Display aggregate throughput for all switch ports
Switch Throughput Utilization Displays the throughput at the time sample is taken
Port Error Displays a CRC error for a given port

Advanced Performance Monitor (APM) Overview 14
Table 14-1 Performance Monitor Menus and Functions
Menu Functions
Switch Percent Utilization Displays Percentage uses of a switch at the sample time
Port Snapshot Error Displays CRC error for all switch port between sampling period
Advance Monitoring menu Graphs
SID/DID Performance Displays the performance between specified SID/DID pair
SCSI vs. IP traffic Displays traffic in percentage on each port
AL_PA Error Displays a CRC error on a given port AL_PA
SCSI Commands Graphs
SCSI Read/Write on a LUN/ port Displays SCSI read/write on a given port and specific LUN
SCSI read on a LUN per port Displays SCSI read on a given port and specific LUN
SCSI Write on a LUN per port Displays SCSI write on a given port and specific LUN
SCSI Read/Write per port Displays SCSI read/write on a given port
SCSI Read per port Displays SCSI read on a given port
SCSI Write per Port Displays SCSI write on a given port
14.1.1. Performance Graph Canvas

Performance Monitoring allows you to set up a canvas of performance graphs from the Action menu.
• An existing report can be selected from a list of reports that are predefined or in some cases can be customized to
monitor the specific objects of the fabric.
• The canvas window displaying the graph can hold up to a maximum of eight graphs simultaneously. Any graph can
be magnified, added or removed from the main canvas.
• Up to 20 individual canvases can be saved for later retrieval.
• Graphs can be printed.
14.1.2. Advanced Performance Monitor Features

Advanced Performance Monitor for Fabric OS v3.1/4.1 and higher features enhance the integration support of the Fabric
Access API. The distributed framework of Fabric OS 3.x/4.x allows Advanced Performance Monitor (APM) to communicate
with the APM running on other switches of the fabric, thus allowing the administrator to create or delete monitors on a remote
switch in the fabric. It is capable of setting up end-to-end monitor to measure throughputs between source (SID) and
destination (DID) on a port. SID or DID mask can be setup for partial matches of the frame ID. The frame filtering parameters
can also be applied to detect specific frame patterns. For example, comparing IP versus SCSI commands on a port, measuring
SCSI read or write type commands. It is also capable of reporting CRC error statistics on a specified AL-PA. The example
below illustrates end-to-end monitor setup and operation.
Note: The distributed framework for Advanced Performance Monitoring is not supported in Fabric OS v3.0.x/4.0.x. The
Fabric Access API will also not be able to setup any performance monitors on a switch running Fabric OS
v3.0.x/v4.0.x.

Example: End-to-End Monitor

End-to-End monitoring provides information regarding performance between the source (SID) and destination (DID) on a
fabric or a loop. Up to eight SID-DID pairs can be specified per port. For each of the source ID and destination (SID-DID)
pairs, the monitor counts the number of words received, number of words transmitted, and number of CRC errors detected in
frames qualified using either of following two conditions
1. For frames Received (Rx) at the port with End-to-End monitor
Frame SID = Source ID
Frame DID = Destination ID.
RX_COUNT and CRC_COUNT will be updated accordingly.
2. For frames Transmitted (Tx) from the port with End-to-End monitor
Frame DID = Source ID
Frame SID= Destination ID
TX_COUNT, and CRC_COUNT will be updated accordingly.
Switch x Switch y
Tx Rx Rx Rx
Rx Rx Rx Tx
Host A SID =x020b00 StorageBDID=x0108ef
Monitor0 Monitor1
Switch domain=02 Switch domain=01
Port=12 Port=08
ALPA=00 ALPA=ef
Settingend-to-endmonitor
Figure 14-3 Monitor 0 counts the frames that have an SID of 0x020b00 and a DID of 0x0108ef.
For monitor 0,
RX_COUNT is the number of words from Host A to Dev B
TX_COUNT is the number of words from Dev B to Host A
CRC_COUNT is the number of frames in both directions with CRC errors.
Monitor 1 counts the frames that have an SID of 0x0108ef and a DID of 0x020b00.
For monitor 1,
RX_COUNT is the number of words from Dev B to Host A
TX_COUNT is the number of words from Host A to Dev B
CRC_COUNT is the number of frames in both directions with CRC errors.
Note: End-to-end performance monitoring monitors traffic on the receiving port respective to the SID only. As illustrated in
Figure 14-3, if you add Monitor-1 specifying Dev B as the SID and Host A as the DID, no counters, except CRC, will
be incremented on Monitor-1.

Depending on the application, any port along the routing path can be selected for such monitoring.
Switch x Switch y Switch z

Tx Rx Rx Rx
Host A-SID Storage B DID
Monitor setting points with respect to SID

Multi-switch configuration
Figure 14-4 Multi-switch Configuration of Performance Monitor
14.1.3. End-to End Monitor Setup Using Web Tools

From the Performance Graphs menu, select Advanced Monitoring and then SID/DID Performance. Enter the monitoring
port number, SID and DID of the pair as shown in Figure 14-5.
Figure 14-5 SID/DID Performance Setup
Note: In Fabric OS 4.x the implementation of End-to-End (or Filter-based) monitors dictates that hardware counters to be
probed at an interval that is a multiple of 5 seconds. The recommended the probing interval is 10 seconds. This
restriction does not exist in Fabric OS 3.x.
Note: The Command Line Interface also provides access to the Advanced Performance Monitoring. Please refer to the
Brocade Fabric OS Features Guide for details.

14.1.4. Filter-based Performance Monitoring

Filter-based monitoring counts the number of times a frame with a particular pattern is received by a port. Filter-based
monitoring is achieved by configuring a filter for a particular purpose. The filter can be a standard filter (for example, a read
command filter that counts the number of read commands that have been received by the port) or a user-defined custom filter.
The maximum number of filters is eight per port, in any combination of standard filters and user-defined filters. For example,
Filter can be added to count the SCSI traffic frames or IP vs. SCSI traffic frames. The following example shows setting SCSI
Read/Write on switch Domain 2, port 2 on LUN 00.
From the Performance Graphs menu, select Advanced Monitoring and then the SCSI command menu. Select SCSI
Read/Write on a LUN per port. Enter the monitoring port number and LUN number as shown.
Figure 14-6 SCSI Read/Write on a LUN per port Setup
14.1.5. Masking Setup for End-to-End Monitor

End-to-End monitors count the number of words in Fibre Channel frames that match a specific SID/DID pair. The mask can be
set on a port to compare only certain parts of the SID or DID. With no mask set, the frame must match the entire SID and DID
to trigger the monitor. For example, by setting a mask, you can choose to have the frame match only one or two bytes of the
three byte field (Domain ID, Area ID, AL_PA) to trigger the monitor.
For example, the default end-to-end mask for SID and DID is FF:FF:FF that means the entire SID and DID must match. A
value of 00 means that field must be ignored. When mask is set to FF:FF:00, the first two bytes are looked at and third byte
AL_PA filed is completely ignored.
Note: Only one mask per port can be set. When setting a mask, all existing end-to-end monitors will be deleted.

14.2. Fabric Watch Supervised Advanced

Performance Monitoring Features
Advanced Performance Monitoring can be an effective tool when used in conjunction with Brocade Fabric Watch.
14.2.1. Performance Class Monitoring

The Fabric Watch performance class lets you specify thresholds and notification methods on performance areas (AL_PA,
End-to End, Filter based). A notification is sent out on triggering an event by a threshold breach. The area in question then can
be monitored graphically. For example, in case a Tx/Rx traffic breaches pre-defined threshold limits of Fabric Watch, an event
is triggered notifying the user. Advanced Performance Monitor can be set to monitor traffic patterns and behavior over a
reasonable period of time. Having a graphical representation of that particular port provides much needed visibility to SAN
administrator for taking corrective actions. The threshold base event triggering eliminates the need for constant monitoring of
the port activity and the visual representation of throughput data helps minimizing the troubleshooting time. Thus improving
the overall SAN management and productivity.
The process of setting and monitoring Rx performance on a port is detailed in section 14.2.3.1. Fabric Watch Threshold Setup
for Rx performance on page 14-7.
14.2.2. AL_PA Monitoring

AL_PA monitoring provides information regarding the number of CRC errors in a loop configuration. AL_PA monitoring
collects CRC error counts for each AL_PA attached to a specific port. Upon exceeding CRC threshold a message is forwarded
as per alarm settings. Upon receiving an alarm a user can choose to monitor this port from the Advanced Performance menu.
14.2.3. Filter-Based Monitoring

Filter-based option will let you configure thresholds and alarm notification on custom filter
Example: Rx performance monitoring
Figure 14-7 and Figure 14-8 show threshold configuration setup measuring Rx performance of a port supporting 2 Gbit/sec
transfer rate. The high (above) threshold is setup at 190 MB/sec. The Alarm triggers only when above threshold is exceeded.
The user notification is set for Error log and Email. A buffer zone of 10 MB/sec is specified to re-arm the trigger when
throughput falls below 180 MB/sec. The buffer also suppresses multiple triggering if the port is consistently operating in the
range of 180-190 MB/sec. The second trigger will occur only after the throughput falls below 180 MB/sec. Thus multiple
triggering of the event indicates that the port is continuously operating in the upper range and must be examined over a period
to verify that it is not reaching its saturation point.
14.2.3.1. Fabric Watch Threshold Setup for Rx performance

1. From the Fabric Manager Actions menu select Switch view.
2. Select Watch from Switch view.
3. Select End-to-End area of the Performance class in element window.
4. Select the Threshold Configuration tab and setup parameters for Rx performance.

Figure 14-7
Note: The Email notification can be set from the Email Configuration tab which is explained in section 13.8.1.9. Email
Notification Setup on page 13-21.
Example 2: The performance monitoring after the Rx performance threshold trigger event notification is received.
1. From the Fabric Manager Actions menu select Switch view.
2. Select Perf from Switch view.
3. Select Basic monitoring from Performance Graph menu.
4. Select Port throughput and specify the port number.
Figure 14-8
Note: The port performance over one hour period showing the Rx/Tx throughput trend.

Chapter
Brocade API Scripting Toolkit Overview
15
Brocade was the first to introduce a management API for SAN fabrics, the Fabric Access API. The Fabric Access API is made
available to application development partners, and customers through the API Scripting Toolkit. Please go to
www.brocadeconnect.com to obtain the API Scripting Toolkit.
The Brocade API Scripting Toolkit facilitates customization of SAN management environment by providing highly scalable
and robust interfaces for integrating SAN management functions with mission-critical management applications. Over a dozen
companies have delivered products to market that take advantage of the Fabric Access API. Scripting can easily integrate the
intelligence of the Brocade SAN fabrics into existing management applications. The Fabric Access API Scripting Toolkit
provides the following key features:
• Fabric-wide and multi fabric support - Scripting simplifies multi-switch management task by coordinating access to
multiple switches in the SAN fabric through a single session as opposed to telnet session that requires establishing a telnet
session on each switch of the fabric.
• Multi-user access - Scripting supports up to 10 simultaneous sessions without interrupting simultaneous real-time access
to the telnet interface.
• Embedded functions - The scripting Toolkit contains most of the commonly performed functions that can be integrated
rapidly. SAN discovery, switch and port control type embedded functions can be accessed via Perl code or a direct
interface to the Fabric Access API.
• Direct Function Access - Offers Perl interface to advance users to directly access Fabric Access API functions
The Brocade API Scripting Toolkit includes the following key components:
• Fabric Access Library
• Fabric Access Perl Adaptation Interface
• Basic scripting functions
• Advance scripting functions
• Direct interface to the Fabric Access API library
• Error handling capabilities
• Perl code examples

15 Brocade API Scripting Toolkit Overview
15.1. API Scripting Interface Levels

The Brocade API Scripting Toolkit provides the end-user with three interface levels.
User Written Perl Programs
BasicInterface Pre-developed telnet

like functions
Advance Interface
.XS(PerlAdapter) Direct manipulation

of desired objects
Scripting Tool Kit
XML
Passes XML strings
Fabric OS API to C library
HostLibrary
Figure 15-1
• Basic Interface
Closely simulates the key telnet functions. Several basic interface functions allow you to perform discovery on a fabric,
switch, port, device, zone, and license.
• Advanced Interface
Provides a Perl-native equivalent of the API Scripting Toolkit object model. The advanced interface is a set of Brocade API
functions that have been mapped into the Perl interface. This is a strict one-to-one mapping with no external logic provided.
This interface is for more advanced users who are familiar with the Brocade API and the concepts it uses. The advanced
interface is suited for applications because objects and their attributes are returned as Perl variables instead of text strings.
• Direct Interface
Provides a direct interface to the API Scripting Toolkit XML interface. The direct interface is used to send and receive XML
through the Brocade API library. No error checking is performed on the validity of the XML functions that is passed in. This
level of interface is for the most advanced users who use their own error checking routines and are trying to integrate with an
XML/Perl interface.

Brocade API Scripting Toolkit Overview 15
15.2. Basic Communication Process

Brocade API Scripting Toolkit consists of a host-based library that interfaces the application to one or more fabrics. The
application connects to each fabric either out-of-band TCP/IP connection or in-band using an IP-capable Fibre Channel Host
Adapter. The Perl application links the API library to its environment. The access to a SAN begins through a set of routines to
initialize access points to the SAN and create authenticated sessions to manage the switch in the fabric. For each fabric, the
library establishes an IP session with a switch in the fabric. The Application function calls are interpreted by the library and
commands are issue to the fabric via the gateway switch. The resulting output is received by the library, formatted and returned
to the application.
Host system
PerlInterpreter/Compiler
T CP/IP
API Functionsplatform Libraries Fabric Access Layer
Fabric OS
Figure 15-2

15.3. Basic Communication Model
15.3.1. Session Management

The user must provide a login and password to a single switch in the fabric. Information for all switches in the fabric can be
obtained without additional logins to the other switches of the fabric as oppose to a telnet session that requires logins to each
individual switch and limit session to one per switch. The API allows the user to have multiple sessions open at a given time.
Each session may be with the same fabric or different fabric. The gateway switch limits the number of session available per
fabric. Each switch can maintain up to 10 active sessions.
Note: If more than 10 sessions are required on a fabric, try using a different proxy switch of the fabric.
Note: The API Scripting toolkit uses a different TCP/IP interface that does not conflict with the telnet interface. Therefore
an API session and the telnet session can be used at the same time.
15.3.2. Platform Support

Brocade API Scripting Toolkit v3.x is supported on the following platforms. Familiarity with Perl development and
understanding of telnet interface implementation on Brocade product are the primary requirements.
Table 15-1
Operating system Brocade Fabric OS Perl
AIX 5.1 /5.2 v2.6.0j, v3.0.2q, v4.0.2c, v5.6.1, v5.8

HP-UX 11.0/11i or later
Red Hat Linux 8
Sun Solaris 2.7/2.8/2.9
Windows NT 4.0
Windows XP-2000/2003

Brocade API Scripting Toolkit Overview 15
15.4. Fabric Access Supported Features

• XML/Perl interface
• Session management
• Discovery
• fabric discovery
• switch discovery
• device discovery
• zoning discovery
• license discovery
• Control
• switch control
• port control
• zoning control
• license control
• Firmware download
• Configuration upload and download
• Advance Performance Monitoring (APM)
• FDMI support
• Error handling
Note: For step-by-step installation, configuration setup and usage instructions please refer to Brocade API Toolkit User's
Guide v 3.x. The release notes also include valuable information about features and important notices.
Note: The latest version of Brocade API Toolkit can be downloaded from the Brocade Connect web site
(www.brocadeconnect.com). Brocade also make API shared scripts and utilities available along with complete
documentation. Please visit Brocade Connect for more information and API updates.


Section
Appendices IV

Appendix
Cabling Design and Management
A
A.1. Introduction
In a SAN environment, the strict uptime requirements do not allow sufficient time for the daunting task of redeploying a cable
strategy. To prevent additional cost and downtime, a reliable cable management strategy needs to be implemented when a SAN
is first deployed. A complete layout and documentation plan needs to be created during the planning phase. Every SAN has
constraints and factors that dominate design choices, such that no single cabling design can meet all requirements. While a
single design plan will not solve all problems, it is possible to follow a reliable process to make sure that the majority of design
concerns are addressed. By implementing a reliable cable management strategy, a SAN environment can gain the following
benefits:
• Improved supportability and fault identification
• Improved ability to isolate a fault - i.e. prevent a single cable problem from affecting multiple components
• Improved scalability - scale a design and prevent the management decay
• Reduced fault count
• Improved time to resolution
This is intended to provide an overview of the process used to develop a cable management strategy. Best practices and
guidelines are presented throughout the document. The process and guidelines included in this document have been developed
in the Test labs at Brocade. While this process has been developed to help account for the dynamic needs of interoperability,
these are generic principles that apply to any SAN in a corporate environment.
The cable processes discussed in this document will focus on the following areas:
• Data collection -
• Analyzing the needs of the site and of the SAN -
• Design criteria -
• Design criteria -
• Implementation best practices -
• Ongoing management and maintenance -
A.2. Needs Analysis

Before designing a plan, it is critical to complete a “needs” analysis, including a list of all assumptions and dependencies. No
one cable management plan will be perfect. Each site will have to weigh factors such as: cost, management overhead,
reliability, scalability, and flexibility. Before creating a cable plan, the requirements for each site needs to be clearly
understood. This section provides some details into the type of data that should be collected during this phase. Fundamentally
two areas must be analyzed: 1) the current infrastructure, and 2) the requirements and assumptions that will guide the
implementation.
Brocade SAN Design, Deployment, and Management Guide A-1

A Cabling Design and Management
A.2.1. Existing Infrastructure

The existing infrastructure, which will contain the SAN environment, must be analyzed as it can place a significant number of
constraints on a cable management strategy. Here are some of the basic questions that need to be asked:
Site Connectivity: The site may have a patching infrastructure in place. Before any plans are made, this should be
documented and the capacity clearly understood. If the site is connected by a patching infrastructure, the following questions
should be asked.
• Are sites connected together by a core distribution panel or are they linked in a series?
• What is the maximum number of patch panel hops between two labs?
• How many ports are available for in each lab?
• What type of connectors are used (LC, SC, MTP)?
• What types of fiber is the installation based on (50 micron or 62.5 micron, Single mode or multi-mode)?
• Is the infrastructure reliable (has this been well managed, clear of dirt contamination, etc.)?
Existing Site Layout: Most of the design restrictions are based on the current layout. The following questions should be
answered:
• How many racks are in each site?
• What is the total number of switch ports in each site?
• What is the total number of nodes in each site?
• What percent of the racks have cables routed internally?
• What percent of the devices are cabled to a nearby rack?
• What percent of the devices are cabled to a distant rack (across the site or to another site)?
Existing Cables: Make sure that you understand what current standards have been put in place for cables. When a design is
complete do not accidentally mix incorrect cable types or us unreliable cables.
• What types of cables are used?
• For Multi-mode cables, what percent are 62.5 micron and what percent is 50 micron?
• Have the cables been kept free from dirt and other contaminates?
• Develop a breakdown of all cables categorized by length (1m, 5m, 10m, etc.) and type (50 micron, 62.5 micron, etc.).
Site Infrastructure: Each site will probably have some level of cabling infrastructure already in place. The following
questions should be answered:
• Where are the cabinets and other storage facilities located?
• Are raised floors or ceiling cable guides used?
• Where are the patch panels located?
• Where is the Ethernet connectivity located?
• What is the power distribution plan for each site?
• Where are the windows (sun light) located?
• Where are the sprinklers, vents, fans, and cooling located?
• Where are the ceiling and floor panels located which maintenance will use for access to (such as ceiling access for the
cooling system)?
• Where are the fire lanes and other zones that must be clear of cables?
• Where are the doors and emergency exits located?
• What is the emergency fire and disaster plan?
• Where is the current site documentation?
These points should be covered in the documentation for each site. If it is not documented, it is critical that this data is
collected before designing the site layout. Once this data is gathered, the next step is to document the assumptions for each
site.
A-2 Brocade SAN Design, Deployment, and Management Guide

Cabling Design and Management A
A.2.2. Site Requirements

It is important to document all requirements and assumptions. Undocumented requirements are a major source of confusion in
SAN implementation strategies. Each requirement should be rated by priority. The rating scheme selected should be able to
differentiate absolute requirements from others that are “nice to have”. A recommended ranking system is listed below:
• R2: Required Level 2. This feature is mandatory. The need for this requirement out ways any reasonable cost
considerations. If this feature cannot be met then the entire requirements list and implementation strategy must be
reanalyzed.
• R1: Required Level 1. The featured is required. There is a strong need for this requirement. If the addition of any
single R1 requirement resulted in a significant scope change (2x increase in scope) then this could be dropped. If a
group of R1 priorities had to be dropped then the requirements should need to be re-analyzed.
• D2: Desired Level 2. This feature is desired if there is no significant addition to the scope. If a small addition in
scope can cover this requirements or a group of D2 requirements then this is beneficial.
• D1: Desired Level 1. This feature is desired if there is no addition to the scope.
This is one possible classification list, but many others will serve as well. When documenting requirements, several categories
need to be analyzed. Table A-1 and Table A-2 provide an outline of the types of requirements that should be collected and
documented. The key objective is to document all assumptions and user requirements that could affect the design choices.
Table A-1 Growth and Density Requirements
Ref Description Required Nice To Notes

Have
1.1 Define the capacity each site is expected to scale
to in the next 12 months.
1.2 Define the minimum capacity each site is
expected to scale in the next 12-24 months.
1.3 Define the maximum capacity each site is
expected to scale to over the next 12-24 months.
1.4 Define the frequency that you expect to be able to
make additions to the fabric.
1.5 Define if the site infrastructure is intended to stay
static or if it needs to be a dynamic area with
many changes. (This is a critical requirement to
analyze as it can affect your design choices.)

Table A-1 Growth and Density Requirements

Have
1.6 Define the maximum capacity that each SAN is
planned to support.
1.7 Define the maximum number of connections
between each site (this does not have to be lim-
ited to the current infrastructure).
1.8 Define the maximum number of ports expected
per rack.
1.9 Define the percent of rack space allocated for
cable guides within each lab. Define the maxi-
mum, minimum, and average if possible.
Table A-2 Usability Requirements

Have
2.1 Define fire lanes and other areas that must be
clear of all cables.
2.2 Define the space behind or between racks and the
expected level of access.
2.3 Define which type of user will have access to
which labs.
2.4 Define all areas where frequent access is required
and a large snag potential exists. These areas will
require more strict cable management.
2.5 Define any appearance requirements and the
audience for the site. (A visually appealing site
may not be optimized for support or fault
isolation. The value of appearance over ease of
use must be weighed).
2.6 Define the importance of being able to isolate
faulty cables and replace them. Define the
acceptable level of impact that removing one
cable can have on other cables.
2.7 Define the need for any racks to roll in and out of
the site. Include the required clearance height.
Define the path that all equipment entering the
site will follow and define that this area must be
free from any located snag.

A.3. Design Process

During the design process, document the full layout including all required equipment. Continuing through the design process,
follow the steps below. This is a proposed process only. Any site design process will generally be sufficient. The following
sections will focus in more depth in each of these areas:
• SAN Design
• Inter-site connection design
• Site patch panel design
• Cable Management Requirements
• Cable Requirements (patch cords, bundle cable etc.)
• Site Documentation Standards
A.3.1. Site Design

Define the layout for each site before implementing a routing plan. The layout out plan should provide a logical naming
convention for each site and each rack location within the site. Link the naming convention to the Ethernet port and patch
panel port naming convention. Standardizing on a common naming convention will eliminate confusion when discussing the
location of a device, port, or rack. Many site problems are introduced because of bad of improper documentation.
Standardizing the naming convention can greatly reduce this risk. A suggested naming convention is below:
SJ1-5W2-B3
SJ1 - Building Identifier, San Jose 1
5th Floor
W2 - West Side, lab 2
B3 - Row B and Rack 3
This provides a name convention for the rack. An Ethernet port or patch panel port can be indicated using an additional
identifier. In this way when a port is referenced, the exact location of a problem can be located. Typically, the building
identifier is left of any ports that are local to the lab. Any ports that connect the labs (or buildings) through a central
distribution panel should include the building identifier.
Static vs. Dynamic: This is a critical design constraint on the site layout. Each rack in each site needs to be rated by how often
changes are expected to be made. Select cables based on the frequency and magnitude of the changes.
• Dynamic: Significant modifications and changes are made to the rack cable layout about every six months.
• Static: Minor changes are made every year or two.
Note: All the cases between these two extremes are a judgment call.
Once the current layout is complete, evaluate the impact of both the minimum and maximum growth projections. Make sure
that the process to scale is clearly planned. Even the best site design plan will decay into chaos over time. This is so predictable
that you can frequently define a half-life (like a radio active half-life) that is based on frequency of changes and the number of
additions to the site. Another major factor is the growth plan for the lab. With a solid growth plan the decay problem can be
significantly mitigated or controlled. Without a proactive growth plan, no amount of reactive support can save a site
infrastructure, and it will revert to a chaotic system. If a site is planned to be live for more than three years, a growth plan is
mandatory.

A.3.2. Inter-Site Connectivity

Once the site design is completed and the growth plan documented the inter-site connections should be clear. Make sure that
you have sufficient capacity to connect you labs together and that there is 10-20% extra capacity beyond your expected needs.
The primary cost for a patching infrastructure is installation, not the cost of materials. Adding two to four extra ports is usually
a minimal addition when connecting labs within the same building. Several deigns can be used for connecting labs together.
Typical layouts include: A ring configuration where each site is connected in a series. Take the case where five labs exist A, B,
C, D, and E. If these are connected in a series then each site can only connect to each of the neighbors. This can work well with
three labs but beyond this number, management becomes increasingly complex and there is a larger risk that there will be a
conflict. An alternate connection method is to have the labs connected to a distribution panel. Frequently you will find all the
labs on a floor attached to a common distribution panels and then the entire then floor through a second Panel. This can be an
expensive solution and the capacity requirements for connection to labs on the same floor and other floors will need to be
clearly understood.
A.3.3. Patch Panel Design

Each site will need to have a complete patch panel design completed. Consider the following when designing the patch panel
infrastructure:
• Accessibility: How frequently will the panel need to be accessed and how often will changes be made?
• Capacity: How many ports are required in each location?
• Flexibility: Will a set of nodes always require patching to a specific location or do they need any-to-any access.
Any-to-Any access requirements should be limited if possible. This increased flexibility comes at and added
management burden.
• Location: Each location may require different patch panels (roof, rack, or wall mounted, etc.) Consider proximity to
hazards (heat, light, water, dirt, grease, humans, etc.).
The following figures illustrate two different types of patch panels. Figure A-1 shows a patch panel designed for a static
installation. This provides for high capacity in a compact space and provides a cover for protection. On the downside, this
panel is hard to fit fingers into and removing just one cable can be challenging. When fully loaded, this patch panel can hold
72 ports. All 72 cables are routed in from the sides making well-planned management mandatory. This is great for a static
configuration, but it can be a burden in a dynamic environment.

Figure A-1 Patch Panel - Static Installation
Figure A-2 illustrates another patch panel design. This provides a 1U patch panel with no enclosure. Notice how each of the
ports is spaced apart to allow for easy access.
Figure A-2 Patch Panel - No Enclosure

The type patch panel in Figure A-2 can be used as a distribution panel or rack mounted, as shown in Figure A-3.
Figure A-3 Patch Panel - Rack Mounted
One of the disadvantages to this type of panel is that there is no dust cover. In Figure A-2 several ports without fillers are
shown. These ports should be considered contaminated and will need to be cleaned.
Another point is that both allow for easy maintenance. In Figure A-1 the patch panel has inserts, which can move, allowing
easy access to the cable. The panel in Figure A-2 and Figure A-3 has a tray that pulls out to allow access to the ports. Although
these access methods are convenient, it is critical that the cables are properly manages with sufficient slack to allow for the
maintenance. If one port requires work, a poorly designed cables management system might require that all the ports are
unplugged before removing the panel.
After selecting the patch panels, complete the final layout. In the final layout, consider the following design requirements:
• Will each patch panel be connected to a central distribution area or to a fixed location only?
Scenario 1: A set of storage ports on the west end of the lab need to be connected to the fabric. This is the only location to
which these storage ports need to be connected.
Scenario 2: A set of Unix hosts in a lab need to be able to get connectivity to any rack in the lab through the patch panel
system. This will require a distribution panel, since direct connectivity is not possible. Try to design around this requirement,
since the additional flexibility requires increased cost and a larger support burden.
• Will the patch panels be mounted in the ceiling, floor, or racks?
For the edge devices mounting the patch panel in the ceiling (see Figure A-2) can allow for added flexibility. This allows for
the replacement of the rack allocated to that panel without disrupting the infrastructure. Mounting the panel in a rack with
other devices will require disconnecting the entire panel, not just the patch cords, when moving the rack.
One option is to have a special rack dedicated to patch panels. By alternating patch panel racks and switch racks, it is possible
to bring a large number of edge devices to a central patching infrastructure.
Note: Floor mounted patch panels are not recommended, as they can be hard to access and easily contaminated by dirt.
• What are the environmental hazards?

Make sure that the patch panel infrastructure is free of hazards. Design it to be static and low maintenance. The following
typical hazards should be considered:
Human: Be aware of technicians moving a rack or cable: The patch panel should be located so that no expected activity
or use can result in a failure.
Light: Do not expose the cables to direct sunlight.
Water: Do not expose the patch panels to condensation or water.
Heat and exhaust: Do not mount patch panels near equipment exhaust. This can heat the patch panel and cause dirt to
contaminate the panel.
Accessibility: Do not mount patch panels in locations that block access to lights, sprinklers, A/C access ways. Do not
block a zone that needs to be free of cables.

A.3.4. Patch Panel Guidelines

Patch panels are great for providing a centralized method for managing host and storage connections to the SAN and other IT
equipment. Here are a few guidelines for making effective use of them. Also pointed out are some common problems and
solutions.
Guideline: Every fiber optic connection point generates a few dBs of signal loss. Keep the number of connections
to a minimum, generally eight or less. If a connection seems intermittent, too many patched connections
may be the cause.
Guideline: Be aware of distance limitations. The total distance can add up quickly as cables are run through
conduits and patched. For 2 Gbit/sec connections, using shortwave SFPs and 50 micron cables, the
maximum distance is 300 meters. Like too many patch panel connections, exceeding this distance limit
may also cause an intermittent loss of signal.
Guideline: Avoid using older fiber optic cables with a 62.5 micron core. For 2 Gbit/sec connections, a 62.5 micron
core only yields 90 meters as the maximum total distance. The newer standard fiber optic cable, 50
micron core, yields a maximum of 300 meters.
Guideline: At times, when plugging in a fiber optic cable from a patch panel, there may be a no light indication on
the switch. This may indicate that a Tx and Rx swap has occurred. For Tx and Rx connections to be
lined up properly between the switch and devices, a net ½ twist is required along the connection path.
Often times, the fiber optic cable connecting between patch panels adds an extra whole number plus ½
number of twists, yielding a net of 0. One way to fix this is to swap the Tx and Rx on the LC or SC
connector at the switch port. If using patch panels, be aware of this and use straight through fiber optic
cables if at all possible.
Warning: Do not mix single (long-wave) and multi-mode (short-wave) cables in Patch Panels. The light wavelengths are
different in each type. This make the fiber cables incompatible for connecting together.
Warning: Do not patch together fiber optic cables with a 62.5 micron core to fiber cables with a 50 micron core.
Keeping these tips in mind should allow for more effective and reliable use of a patch panel infrastructure for connecting the
SAN components.

A.3.5. Cable Management Requirements

The only way to safely route a large number of cables in a rack, or across a site, is to correctly use the appropriate cable
management product. There is a variety of cable management guides available. Figure A-4 shows three common cable guides.
Figure A-4 Three Types of Cable Guides
The guide in the front (A) is designed as a 0U cable management guide. This can be mounted in front of a device and will
allow not take up any addition space. An illustration of how this can be racked in shown in Figure A-2. This single cable guide
is sufficient for a pair of patch panels. If a 0U patch panel is selected make sure that the main support bar is removable. Access
to the device behind the cable guide will be required at some point (pull out a drawer for maintenance, etc.), and if the cable
guide does not have a removable bar then the entire guide will have to be removed. In Figure A-5 notice the pin in the bottom
right corner.
Main Bar Release

Pin
Figure A-5 Cable Guide with Pin Release

The second guide (B) in Figure A-4 is a 1U guide designed for fiber cables. The curves help prevent bend radius violations.
The last guide shown (C) is a 2U cable guide. This type of guide is more common in Ethernet cable management but can also
work well for fiber. Extra care is required so that a bend radius violation does not occur.
In addition to horizontal cable guides, it is critical to select an appropriate vertical cable guide, as shown in Figure A-6.
Vertical guides provide capacity for a large quantity of cables. Curved fingers to help prevent bend radius violations. The oval
support in the center is useful when managing slack and controlling the cables.
Figure A-6 Vertical Cable Guide
When selecting a cable guide, consider the following:

1. How much space is the available? Can 20-50% of your rack space be spared for cable management?
2. How many cables do you need to manage?
3. Will the guide block access to any equipment?
4. Will the guide and your cable design support your bend radius requirements?
5. Will it be possible to isolate bad cables?
6. Does the guide allow back access? (Notice the guides in Figure A-4 have holes to support rear access.)

A.4. Cable Requirements
One of the most critical components of a cable design is proper cable selection. There are many types of cable available, each
offering a variety of benefits. When investigating cables, consider the following:
• Core diameter (5 or 62.5 micron)
• Quality of the glass used in the cable
• Thickness of the cable
• Protecting sheath strength
• Flexibility
• Cable memory
• Connector quality
• Cable labeling
A.4.1. Typical Cable Types

The following figures illustrate some of the typical cable choices available.
Figure A-7 illustrates a typical multi-mode fiber cable. This cable has sufficient flexibility for nearly any cable guide. Also,
note that there are no kinks in the cable. This type of cable does not exhibit memory - the tendency to permanently form kinks
due to previous cable management. Another feature to note is that each end of the cable has shrink-wrap with a serial number
that identifies that cable. This is extremely helpful in documentation and tracing cables.
Figure A-7 typical Multi-Mode Fiber Cable

A
Figure A-8 illustrates a cable form a different manufacturer. This cable is serialized and is more rigid, but easily routed.
Evaluate each cable for the intended environment.
Figure A-8 Rigid Serialized Cable
Figure A-9 illustrates a thinner cable. A large quantity will take up less space. This particular cable also exhibits memory.
Notice the severe kinks. This cable could become a problem if routed in this condition. It also makes compact storage difficult.
Figure A-9 Thin Cable Exhibiting Memory

Figure A-10 illustrates a breakout cable this provides a single bundle that separates into individually protected cables. This is
cable has the highest level of protection but is also thicker than alternates. This cable is good for dynamic areas, as the
individual ports are exposed.
Figure A-10 Breakout Cable
Figure A-11 illustrates a bundled cable. Like Figure A-10, this type of cable bundles multiple fibers. The key difference is that
each fiber has thin protection. This is ideal for a static configuration, and in a rack with high-density constraints.
Figure A-11 Bundled Cable

A
Figure A-12 illustrates a ribbon cable with MTP connectors. The MTP connector can be seen on the right. In this cable, all the
fibers run as a group and are not individually protected. The light dots in the center of the adaptor are the actual fiber strands.
An adaptor can be used to connect this to a patch panel or allow it to fan out.
Figure A-12 Ribbon Cable with MTP Connectors
An MTP patch panel is shown in Figure A-13. The MTP plugs in to the rear and the panel fans out to the ports. MTP cables are
not standard at this time, but will be more common upon the standardization of 10Gb. Integration of this with an existing
infrastructure can be difficult and costly.
Figure A-13 MTP Patch Panel
When selecting the appropriate cable, consider the following:

• Do not mix 50-micron and 62.5-micron cables.
• Select components maintained easily.
• Understand your density requirements before cables are selected.
• Try to order the correct cable length. Do not use a 5-meter cable if only a 3-meter cable is required.
• Try to avoid cable with bad connectors or a cable that maintains kinks and loops.

A.4.2. Planning Cable Layouts
When planning the cable layout in a corporate environment, the following factors should be considered and evaluated:
• Number of sites
• Equipment layout for each site
• Percentage of localization for each site
• Connectivity between sites
• Environmental threats
Before a cable management plan can be completed, each of the factors must be analyzed. The most critical factor when
determining the layout is the percentage of localization for each site. This single factor will dictate the number of patch panel
ports and remote cabling. The 80/20 rule applies well to cable design. This rule is completely dependant on the site
architecture and design constraints. If a design exceeds 30-40% remote cabling, an alternative SAN design should be
investigated. Deploying a SAN with a Core/Edge architecture, and cabling only the ISLs remotely, will reduce excess cabling.
Once the level of localized cabling in each fabric has been determined, the connectivity between fabrics can be evaluated.
When designing fabric-to-fabric connectivity be aware of all environmental hazards. Environmental hazards such as water,
equipment exhaust, sunlight, or any other environmental hazards, should be considered during cable selection.
Use the following the table as a guide when designing the cabling infrastructure, keeping in mind the appropriate distance
limitations.
Table A-3 Cable Usage by Distance
Bandwidth Cable Type Maximum Notes

Supported Distance
1 Gbit/sec Single Mode (9 um) cable 10 KM With an Extended Fabrics license, distances
(Long-wave) can be increased.
Multi Mode (62.5 or 50 um) cable (50um) 500 meters The distance will be less when using older
(Shortwave) style 62.5 um fiber cables.
2 Gbit/sec Single Mode (9 um) cable 10 KM With an Extended Fabrics license, distances
(Long-wave) can be increased.
Multi Mode (62.5 or 50 um) cable (50um) 300 meters The distance will be less when using older
(Shortwave) style 62.5 um fiber cables.
Note: Use the appropriate SFPs, depending upon the connector type.

A
A.4.3. Cabling Standards within a single Rack

When designing a cable layout within a single rack the first goal is to make sure that all equipment can be easily removed and
that no devices are blocked. In general, two forms of cable guides can be used.
• If space utilization is not a concern 1 or 2U cable guides can be used for cable routing across the rack. These guides
should be placed between every other device at a minimum. When using cable guides, make sure to properly feed the
cable into the guide. A common issue is having the device too close to the guide, forcing a tight bend radius.
Preventive measures include recessing the device or using a wider cable guide.
• If space utilization is a concern 0U cable guides can be used. These guides mount in front of devices instead of above
or below. The cable guide should not hinder the removal of a device, should a device need to be replaced. Typically,
one 0U guide placed between two devices is sufficient.
When cabling a rack, it is recommended that the ISL cables and the device cables be separated. By selecting a consistent
standard (for example: all ISLs get routed to the right and devices to the left) fault isolation can be simplified. Cables should be
routed from devices to the outer edge of the rack using the correct cable guide. Never allocate more than two devices per cable
guide, which can result in an overlap. Cable guides should be mounted at the edge of the rack and used to route cables to the
correct level. Cables should be routed back toward the center of the rack using the appropriate guide. Excess cable should be
wound on a cylinder that is the correct bend radius.
High-density cabling is recommended in some conditions, such as when routing 6 to 12 ports from one location to another.
There are three key options:
• Breakout cable: Each strand of the breakout is individually protected for maximum reliability. The down side is the
large diameter of the bundle.
• Bundled cables: A compact bundle with very light protection on each strand. This is reliable in a static environment,
but not recommended for patching devices in a dynamic environment. The advantage to this cable is the compact
form factor.
• Ribbon cable: Specialized cable with 8 to 24 strands running in a single cable. Fan out cables or patch panels can be
plugged into the ends. These are very compact and stable, however the breakout connectors are expensive and
difficult to make compatible with an LC interface. This cable is not recommended, unless the SAN is planned to
standardize on this interface. Off the shelf, components can be limiting and most solutions require custom
(expensive) parts. This solution may be more viable in the future.

A.4.4. Cabling Standards Between Racks
When cabling between racks, the same guidelines as same rack cable management should be followed. Cable guides must be
used and cables should be routed to the rack edge and then to the correct height. Try to group cables into common types
(storage ports, host ports, ISLs, etc.). It is generally easier to trace an individual cable by grouping common types. It is also
recommended to standardize the side of the rack which is used for devices and which side is used for ISLs. This will further
help isolate the different the cables. Another recommendation is to make sure that any cables that have to cross a rack are not
routed within the rack. It may be necessary to remove a single rack and routing a cable through an adjacent rack will require
that cables to be disconnected. Make sure that the cable design minimizes the impact of any device or rack removal.
Figure A-14 Cabling Between Racks
A.4.5. Cabling Standards to Patch Panels

When using patch panels a few unique constraints must be considered.
Cable type: In a patching infrastructure it is key that all of the cables are the same diameter. 50 micron cables should be used
in a patching infrastructure if possible. Do not mix 62.5 and 50 micron cables in the same patching structure. Doing so may
lead to intermittent link loss issues than can be difficult to troubleshoot.
Bundled or breakout cables: Since patch panel cables are static, it is recommended to use bundled cables. Breakout cables
are sufficient, but the added protection results in a wider diameter that is more difficult to route. Position patch panels close to
the ceiling to prevent contamination from dirt. It is critical that any unused port remain covered at all times.

A
A.5. Documentation
Documenting the cable plan is critical for future management. A reliable standard for labeling all cables and should be clearly
established, including:
• All site-to-site patch panels
• All building-building patch panels
• All site patch panels
• The source, destination, and serial number of all cables at the site
• Cable Properties
• Cable manufacturer
• Cable length
• Cable type
• Other Cable properties
This is a significant documentation burden, however skipping this step will result in more work in the future. Documentation
is must be well maintained in a static environment. In a dynamic environment, balance the level of documentation with the
frequency of changes. Use the level of documentation that works for your site, but make sure it is consistent across the site.
A.6. Cable Implementation

Good cable management standards are a requirement in any corporate environment. It is clear that a poor cable management
plan will result in chaotic infrastructure and an increased management burden to trace cables. In addition, a poor cable
management plan can put a highly available SAN at risk. It is important to follow good cabling standards to minimize this
potential threat.
A.6.1. Cable Standards

A common issue with cable management is cables overlapping multiple pieces of equipment. This simple design flaw has two
major affects on infrastructure.
A single component may be hindered or prevented from being replaced without added interruption. A typical corporate
environment implements a SAN design that has both resilient and redundant components. In order to benefit from this
architecture, it is critical that a component can be replaced without interruption to the SAN. Due to overlapping cables,
adjacent devices may need to be disconnected to remove or replace a component. This can turn a simple procedure that would
only slight reduce the integrity into a massive disruption. Figure A-15 shows a well cabled environment that is open to this
flaw. If the center switch requires replacement it may be necessary to unplug cables on the adjacent switch. This flaw can be
avoided easily and should not be tolerated. Figure A-16 shows a configuration with no overlap.

Figure A-15 Well Cabled Environment Exhibiting Overlap
Figure A-16 Well Cabled Environment Without Overlap
Overlapping cables can cross the exhaust of a device. This needs to be carefully monitored so that exhaust heat does not reduce
cable integrity.
When designing a cable management strategy the following cable limitations must be considered:
Bend radius: Do not violate the minimum bend radius anywhere in the entire path of the cable.
Shear Force: Any object that has the potential of applying a contact shear force to a cable must be carefully analyzed. Doors
and other moving components should be carefully analyzed to make sure that in all positions they do not apply a shear force.
Cable Strain: A good cable design will provide sufficient slack to prevent a significant strain. While this is often sufficient,
the weight of a group of cables hanging without support must be considered. This is especially a concern if any device exhaust
heats the cable. Another case frequently missed is the effect of doors, sliding patch panels, and moving components.
Violating these rules can result in internal faults in the cable. In some cases this can cause a complete failure of a cable. Often,
faults result in intermittent problems that require a specific cable orientation. This type of fault can be difficult to isolate and
the best resolution for this is preventive maintenance. Use the following guidelines when planning a cable layout.

A
Adhere to manufacture recommended bend radius limitation. As a general rule a bend should not have a radius of less that one
inch but each manufacture can provide more precise guidelines for their cable. A common mistake is to route cables over a 90
degree angle. When loose this does not cause a problem, but if the cables are pulled taught then a 90 degree bend can occur.
Over time this can destroy a cable even if there is only limited strain.
Verify that there is no shear force applied to the cable. Any potential must be carefully analyzed and eliminated. Look for
doors, weights, and tight corners or narrow ridges. Some cable management guides designed for CAT5 have thin plastic
fingers. If the cable is pulled in the wrong direction these can result in a significant shear force.
Verify that no strain is applied to the cables. Analyze the full path of the cables and make sure that there is sufficient slack for
patch panels and devices to move. Moving a patch panel may be required for maintenance and testing purposes.
The following figures illustrate some common cable management mistakes:
Figure A-17 Cables Under Tension - Tight Bend
Through bad cable management, it is possible to force a bend radius violation. In Figure A-17, notice the cable on the left. If
left slack, these cables would be fine. Under tension the cable guide forces a bend radius violation and magnifies the strain on
the cable. This is a good example of how tension on the cable can result in a shear force. This also illustrates why care must be
taken with cable guides.

The cable guide in Figure A-18 is designed to prevent a bend radius violation. The bend on the left has been deliberately
exaggerated to show that even with a custom-made cable guide, it is possible to cause a significant bend. However, this is an
extreme case and it is much more difficult with this type of guide.
Figure A-18 Preventing a Bend Radius Violation
Illustrated on the left side of Figure A-19 is a well-managed environment. A closer investigation reveals that the number of
cables routed to each guide exceeds capacity. On the right, the cables are visible after removing the faceplate on the horizontal
cable guide. Notice the crease down the center of the cables caused by the faceplate. Do not underestimate the weight or
pressure that a mass of cables creates. A mass of cables in a small area is likely to create an extreme stress on a few cables.
Forcing too many cables into a small space is a common cause for cable failure.
Figure A-19 Exceeding Cable Guide Capacity
Contamination from dirt and debris is another concern that must be accounted for in a SAN environment. Dirt and debris are
the potential cause of intermittent failures. All cables that are not connected should be properly capped and protected.
Carefully monitor patch panel ports that are near a system exhaust, and if possible try to move the patch panel away from any
exhaust.
Most cables can be cleaned on site. Basic kits are available that will allow engineers to test their infrastructure. These tools are
a requirement. Without these tools fault isolation is completely impossible.

A
A.6.2. Cabling Best practices

• Label each end of the cable with a removable label that identifies the source and destination. Be very careful if zip
ties are used. If pulled too tight, sever pressure may be applied to the cable.
• Bundle cables together in groups of four whenever possible. A bundle is easier to manage. Bundle the cables with
wraps, wire ties, or Velcro every 18 to 24 inches. This can vastly improve the ability to isolate a bad cable and remove
it with minimal impact.
• In a high-density environment, try using a combination of thinner patch cables, patch panels, and bundled cables.
This can be effective in a static environment. Do not design a dynamic high-density system.
• Use the correct length cable
• Try to separate ISL cables and device cables.
• Use horizontal guides to route cables to the edge of a rack. Use vertical guides to route cables to the correct height.
• Do not mix 50 micron and 62.5-micron cables.
• Keep patch panels clean and all ports capped.
• Keep all cables bagged and capped when not in use.
A.6.3. Cable Hazards

One of the major problems in a site is unplanned disconnects. The following is a list of general guidelines to minimize this
risk:
• Incorrect cable removal: Reliable documentation is required to prevent an unintentional discontent. Documentation
needs to be standard across all sites. All patch panels, nodes, switch ports, and patch cables should be documented.
• Snag Threats: Human intervention is the most common fault in a cable environment. Keep cable plans neat and
compact. Do not leave cables free to allow a snag or to trip an engineer. Be aware of the path when moving racks, and
keep cables clear. Keep cables clear of all door ways.
• Maintenance: Frequently, correcting one problem causes another. Make sure that all fiber cables are isolated from
power cords and Ethernet cords. Be sure to route cables around racks, not through racks and to route cables around
devices, not through across devices. Leave sufficient slack for access to required areas.
• Environmental: Keep sunlight, moisture, chemicals, and chemical fumes away from cables. Many cables have a
semi permeable sheath and direct contact with water or chemicals can cause a failure.
• Other cables: One cable by itself is not an issue. However, one cable with hundreds of cables pressing on it could be
a significant problem. The weight of the cables can cause a fault over time, as shown in Figure A-20.
Figure A-20 Cable Weight Causing Failure

A.7. Maintenance
Without regular maintenance the quality of a site infrastructure will start to decay. Two key factors can help prevent this decay:
Comprehensive documentation and a clear growth plan. Once these are established, two primary activities must be covered.
• Be proactive about scheduled maintenance or site expansion
• Be reactive about any crisis affecting the SAN. Clean up around cables and document all changes.

Appendix
Long-distance Technologies for
Storage Area Networks B
B.1. Introduction
As Storage Area Network infrastructures continue to expand, so does the need to connect storage devices over longer distances
in heterogeneous environments. In fact, many organizations are beginning to connect local SAN islands over existing
high-speed public and private networks-an approach that enables new types of applications that leverage a geographically
dispersed, yet interconnected SAN infrastructure. Typical applications include wide area data replication, high-speed remote
centralized backup, cost-effective remote storage centralization, business continuity, and storage outsourcing. This document
describes some technologies available for connecting Fibre Channel SANs over metropolitan and wide area networks as well
as implementation details and features unique to Brocade SilkWorm switches that enable enterprise SAN solutions.
B.2. SilkWorm Product Family Configuration and

Functionality
Brocade SilkWorm switches support two methods of flow control over ISLs: Virtual Channel (VC) flow control and R_Rdy
mode flow control. VC flow control is unique to Brocade switches and provides a more efficient mechanism for transporting
Fibre Channel frames across ISLs. SilkWorm switches provide eight virtual channels for prioritizing frame traffic and
congestion management. Eight separate buffer-to-buffer flow control circuits are used to support independent credit for each
VC. Each VC is assigned a specific priority; for example, the VC that carries class F traffic (switch management) is given
highest priority to provide greater stability and ensure the fabric remains synchronized. R_Rdy flow control is defined in Fibre
Channel standards specifications and is essentially VC flow control with a single virtual channel for all Fibre Channel traffic.
Any E_Port on a SilkWorm switch can be configured to use either VC or R_Rdy mode flow control.
B.2.1. Brocade Extended Fabrics

When a switch port is configured for Extended Fabrics, additional credit is given to virtual channels that carry class 2 or 3
traffic. This allows distances between switches to be extended over greater distances while maintaining maximum
performance over ISLs. The Brocade Extended Fabrics license allows ISLs to be connected at up to 60 KM for 2 Gbit/sec links
and up to 100 KM for 1 Gbit/sec links without degradation of performance.
Brocade SAN Design, Deployment, and Management Guide B-1

B Long-distance Technologies for Storage Area Networks
B.2.1.1. Extended Fabrics Modes

As the distance between switches increases, additional buffer-to-buffer credits are required to maintain maximum
performance. Credits are allocated from a common pool of memory on the switch ASIC. From an external point of view, every
four ports, or quad, on a SilkWorm switch share a common pool of credits. Since the total number of frame buffers is limited,
consideration should be given when deciding for which Extended Fabrics level a port should be configured. Table B-1 lists the
various Extended Fabrics modes and Fabric OS requirements.
Table B-1 Extended Fabrics Modes
Extended Distance @ Distance @ Fabric OS Extended Fabrics

Fabrics Mode 1 Gbit/sec 2 Gbit/sec Release License Required
L0 10 KM 5 KM All NO
LE N/A 10 KM 3.x, 4.x NO
L0.5 25 KM 25 KM 3.1, 4.1 YES
L1 50 KM 50 KM All YES
L2 100 KM 60 KM All YES
LD Auto Auto 3.1, 4.1 YES
A new feature in Fabric OS v3.1 and v4.1 allows the number of credits assigned to a port to be automatically determined and
configured based on link latency. Setting a port to LD mode will enable the use of the link round trip timer on the switch to
calculate latency and allocate the required and optimal number of credits a port needs to maintain maximum performance.
B.2.1.2. Configuring Extended Fabrics

Configuring a port for Extended Fabrics can be performed using the portCfgLongDistance CLI (Command Line Interface)
utility. Specify the port and Extended Fabrics level as arguments to the portCfgLongDistance command. If an Extended
Fabrics port is to be configured on a SilkWorm 2000 series switch, the fabric-wide long distance parameter
fabric.ops.mode.longDistance must be set in the configure CLI menu. Configuration of this parameter requires the switch to
be disabled.
Guideline: Extended Fabrics can also be configured via Brocade Fabric Manager and Web Tools, however the
procedures are not given in this document. Please refer to the appropriate User's Guide.
Revisions of Fabric OS v3.0.2 and greater contain an additional optional parameter, VC Translation Link Initialization, to the
portCfgLongDistance CLI command. When set to 1, this parameter indicates that enhanced link reset protocol should be used
on the port. The default value for this parameter is 0 and is compatible with earlier Fabric OS v3.x implementations. For
optimal performance, specify 1 when E_Port links are between switches with Fabric OS v3.0.2 and greater, or Fabric OS
v4.0.2 and greater. Specify 0, or nothing, when connecting a switch with Fabric OS v3.0.2 or above switch to previous releases
of Fabric OS.
To configure Extended Fabrics on SilkWorm 2000 series switches, the following procedures must be performed.
1. Set the fabric-wide configuration parameter fabric.ops.mode.longDistance to 1 using the configure command. This
parameter must be set on all switches within the fabric.
2. Set the appropriate Extended Fabrics mode for each long-distance port using the portCfgLongDistance command.
B-2 Brocade SAN Design, Deployment, and Management Guide

Long-distance Technologies for Storage Area Networks B
3. When configuring Extended Fabrics on SilkWorm 3000 series switches and above, only port level configuration is
necessary.
4. Set the appropriate Extended Fabrics mode for each long-distance port using the CLI command portCfgLongDistance. At
the same time, set the VC translation link initialization bit to 1 on long-distance switches running Fabric OS v3.0.2 or
above.
Brocade supports Extended Fabrics ports between same series switches (i.e. SilkWorm 2000 series to SilkWorm 2000 series
switches) as well as SilkWorm 3000 series switches to SilkWorm 12000 switches. Directly connecting a SilkWorm 2000 series
switch to a SilkWorm 3000/12000 series switch is unsupported when using Extended Fabrics. For a mixed SilkWorm 2000 and
SilkWorm 3000/12000 fabric, where the long-distance ports are between SilkWorm 2000 series switches, the fabric-wide
parameter fabric.ops.mode.longDistance must be set to a value of 1 on all switches within the fabric. For mixed fabric
configurations where long-distance ports are located between SilkWorm 3000 and/or SilkWorm 12000 series switches, the
fabric-wide long distance parameter is not required.
The output in Figure B-1 shows portCfgLongDistance usage and an example of how to configure port 0 on a SilkWorm 3800
switch running Fabric OS v3.1 for L2 Extended Fabrics mode with VC translation link initialization set to 1.
mw123:root> portcfglongdistanceUsage: portCfgLongDistance port ,"distance_level",<vc translation link
init>distance_level: L0 - normal LE <= 10km L0.5
<= 25km L1 <= 50km L2 <= 100km LD
- autovc trans link init: 0 normal1 vc translation
mw123:root> portcfglongdistance 0,"L2",1 Committing configuration...done.
Figure B-1
The portCfgShow command can be used to determine which ports are configured as long-distance ports. The switch output in
Figure B-1 shows that port 0 has been configured for L2 Extended Fabrics and VC Translation Link Initialization has been
turned on.
mw123:root> portcfgshowPorts
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
15-------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+--
Long Distance L2 .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
VC link init ON .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Locked G_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Disabled E_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
Figure B-2
B.2.2. Performance Over Distance

Performance over long-distance links can vary for multiple reasons. The number of buffer-to-buffer credits determines the
number of Fibre Channel frames that a switch can transmit on a link at one time before requiring an acknowledgement back
from the receiver, thus performance degradation may occur if there are not enough credits available. If a SAN application does
not require or utilize the full bandwidth on the link then maximizing credits may not be of high concern. For example, a tape
backup application that generates a 10 MB/sec stream over a 100 KM ISL using full size Fibre Channel frames would require
approximately five credits to maintain performance. If the backup were to generate 100 MB/sec, then approximately 50 credits
would be required to maintain performance.

The number of credits required to maintain full performance on an ISL can be approximated using the following formula. This
equation can only be used as estimation for the number of credits required at a particular distance. The actual amount allocated
by a switch port will most likely be slightly higher and will also include extra credits for class F traffic.
Buffer Credits = ((Distance in KM) * (Data Rate) * 1000) / 2112
Data Rate = 1.0625 Mbaud for 1 Gbit/sec Fibre Channel
Data Rate = 2.1250 Mbaud for 2 Gbit/sec Fibre Channel
Another performance factor of long-distance links is response times for SCSI transactions. For example, in class 3 traffic,
every write transaction requires two round trips; one for the SCSI write request command and another for the transmission of
data. Transmission of data cannot occur until the initiator receives a transfer ready response. Response time can be calculated
by multiplying the link propagation time by 4. The effect of response times can be minimized if an application allows the
number of outstanding I/Os to be increased. Application tolerance to latency must be examined when extending storage
networks over long distances. For example, remote replication solutions may either be synchronous or asynchronous.
Synchronous replication requires disk arrays to maintain consistency at all times; if latency over a link is too large
performance degradation may result.
B.2.3. ISL R_Rdy Mode and Remote Switch

Brocade supports two methods in which to enable R_Rdy mode flow control across ISLs. ISL R_Rdy mode, available on
Fabric OS v3.1 and v4.1 and above, allows per-port configuration of E_Port flow control via command line interface. The
default operation for all ISLs is to use VC mode flow control, however some gateway devices require configuring the port
R_Rdy mode. In order to enable R_Rdy mode flow control, use the portCfgISLMode CLI utility. Figure B-3 shows command
line usage and then enables R_Rdy mode on port 2 of a SilkWorm switch.
mw75:admin> portcfgislmodeUsage: portCfgISLMode port_number, [1 | 0]mw75:admin> portcfgislmode
2,1Committing configuration...done.ISL R_RDY Mode is enabled for port 2. Please make sure the
PIDformats are consistent across the entire fabric.
Figure B-3
Remote Switch is an optionally licensed software feature that can be used to enable R_Rdy mode flow control automatically
when connecting to supported Fibre Channel gateway devices. Fibre Channel gateways generally provide a method of
extending SAN fabrics over metropolitan or wide area networks by encapsulating frames into another protocol. Refer to
section B.5. Multiprotocol Devices on page B-10 for more information.
Note: Core PID mode and fabric long distance settings are not verified for consistency when interconnecting switches using
R_Rdy mode or Remote Switch; the administrator should verify that these fabric-wide settings are consistent.

B.3. Fiber Optics
B.3.1. Optical Fiber

There are two basic types of optical fiber: multi-mode and single-mode. Multi-mode fiber is generally used for short distance
applications and is more prevalent in Fibre Channel SANs within the datacenter. Multi-mode fiber has a large enough core
diameter that numerous modes of light are carried simultaneously through the waveguide. As distance increases this
characteristic can result in what is called modal dispersion. The refraction differences of some modes of light may cause them
to propagate through a fiber at different speeds causing them to arrive at different times. This effect can cause the receiver to
be unable to decipher the incoming data stream. Single-mode fiber, which has a smaller core diameter that allows only one
mode of light to propagate, does not exhibit this effect. Single-mode fiber is also better at retaining the fidelity of each light
pulse over long distances and results in lower attenuation.
Table B-2 FC-PI Multi-mode Cable Specifications
Parameter Unit 50 mm MMF 62.5 mm MMF

Modal MHz*KM 500 200
Bandwidth
Data Rate Gbit/sec 100 200 100 200
Operating Meters 2-500 2-300 2-300 2-150
Distance
Attenuation is caused by absorption (light losing energy to atoms in the fiber) and scattering (because the core is not always a
perfect cylinder). The three main transmission windows where loss is minimal are in the 850, 1310, and 1550 nanometer
range. Table B-3 lists various types of fiber and optical loss incurred by distance.
Fibre Channel links longer than 500 meters require the use of single-mode fiber for transmission. Single-mode cabling has an
approximate core/cladding diameter of 9 micron and 125 micron respectively. The latency of light through fiber is about 100
microseconds round trip for every 10 KM.
Table B-3 Attenuation Caused by Distance
Fiber Size Type Optical Loss (dB/KM)

850 nm 1310 nm 1550 nm
9/125 mm SM ~ 0.35 - 0.8 0.2 - 0.3
50/125 mm MM 3.0 - 7.0 ~ ~
62.5/125 mm MM 3.0 - 7.0 ~ ~

B.3.2. Optical Transceivers

Optical GBIC and SFP transceivers are available in short and long wavelength versions. Short wavelength transceivers
transmit at 850 nm and are used with 50 or 62.5 mm multi-mode fiber cabling. For long-distance applications, long
wavelength transceivers should be used with 9 mm single-mode fiber. Long wavelength GBIC and SFP transceivers operate in
the 1310 or 1550 nm range. Table B-4 lists a subset of GBIC and SFPs that have been qualified with Brocade switches.
Table B-4 Various GBIC and SFP Transceivers
Vendor/Model Type Wavelength Typical Distance

Agilent HFBR-5720L SFP 850 nm 150m - 300m
IBM 42P21SNYAA10 SFP 850 nm 150m - 300m
IBM SOC-1063N-LW GBIC 1310 nm 10 KM
Finisar FTR-8519 GBIC 850 nm 300m - 500m
Finisar FTRJ-8519 SFP 850 nm 150m - 300m
Finisar FTR-1319 GBIC 1310 nm 10 KM
Finisar FTR-1519 SFP 1310 nm 40 KM
Finisar FTRJ-1319 SFP 1310 nm 10 KM
B.3.3. Fiber Loss and Link Budgets

A key part of designing SANs over long distance optical networks involves analyzing fiber loss and power budgets. The
decibel (dB) unit of measurement can be used to describe loss mechanisms in the in the optical path of a fiber link. dB loss is
usually determined by comparing the launch power of a device to the receive power. Launch and receive power are expressed
as decibel milliwatt (dBm) units, which is the measure of signal power in relation to 1 mW.
The link power budget can be determined by finding the difference between worst-case launch power and receiver sensitivity.
Optical specifications for GBIC and SFP transceivers vary and can be obtained from the manufacturer. DWDM and other
optical equipment vendors will provide specifications for their equipment. A value of 1.0 dB can be used to approximate
attenuation caused by a connector/patch panel. Optionally, an optical path penalty of 2.7 dB can also be added to the equation
to account for miscellaneous signal degradation other than attenuation (safety margin).
Power Budget = (Worst Case Launch Power) - (Worst Case Receiver Sensitivity) + (Connector Attenuation)
To calculate the maximum signal loss across an existing fiber segment, use the following equation.
Signal Loss = (Fiber Attenuation * Distance) + (Connector Attenuation) + (Safety Margin)
Table B-3 describes standard optical loss characteristics in various fibers, although loss may vary depending on type and
quality. A better solution is to measure actual optical loss of fiber with an optical power meter.
These equations can be used to determine the validity of fiber links. If an optical signal is greater than the maximum receiver
sensitivity, data loss or receiver damage may occur. Fiber attenuators can be used to resolve the problem.

B.3.4. Diagnostics and Troubleshooting

Most problems that arise in long distance optical applications are caused by attenuation problems. An optical signal may be
too powerful and overdrive a receiver, or may be too weak to maintain synchronization. The resulting symptoms include ports
going offline at unexpected times or not coming up at all. Receive power can be verified with an Optical Power Meter (OPM).
If the output is too high, attenuators can be used to dampen the signal, if output is too low, verify that proper cabling and
transceivers are being used and fiber distance is supported. Refer to section B.3.3. Fiber Loss and Link Budgets on page B-6
for additional information about link power budgets.
Defective or dirty cabling and transceivers can also cause marginal link problems. This is especially true when using
single-mode fiber. Clean the ferrule, or fiber tip, with a cleaning kit and always replace caps on unused cabling to protect
against dust and scratches. Brocade switches have various utilities to troubleshoot marginal links. The portStatsShow CLI
command can be used to monitor port statistics related to bad links such as encoding errors, CRC errors, and truncated frames.
Bonnie:root> portstatsshow 1/15
stat_wtx 36527704
4-byte words transmittedstat_wrx 2403345873
4-byte words receivedstat_ftx 45870232
Frames transmittedstat_frx 4015901700
Frames receivedstat_c2_frx 0
Class 2 frames receivedstat_c3_frx 4015800596
Class 3 frames receivedstat_lc_rx 48640
Link control frames receivedstat_mc_rx 0
Multicast frames receivedstat_mc_to 0
Multicast timeoutsstat_mc_tx 0
Multicast frames transmittedtim_rdy_pri 348431876
Time R_RDY high prioritytim_txcrd_z 476885060
Time BB_credit zeroer_enc_in 0
Encoding errors inside of frameser_crc 0
Frames with CRC errorser_trunc 0
Frames shorter than minimumer_toolong 0
Frames longer than maximumer_bad_eof 0
Frames with bad end-of-frameer_enc_out 3089371
Encoding error outside of frameser_disc_c3 0
Class 3 frames discardedopen 0
loop_opentransfer 0
loop_transferopened 0
FL_Port openedstarve_stop 0
tenancies stopped due to starvationfl_tenancy 0
number of times FL has the tenancynl_tenancy 0
number of times NL has the tenancy
Figure B-4
The portErrShow command displays error statistics for all ports within a switch. It is often useful to run these commands
multiple times in a row, or over various time periods, to see if error counters are incrementing at unexpected rates.
mw75:root> porterrshow frames enc crc too too bad enc disc link loss loss frjt fbsy
tx rx in err shrt long eof out c3 fail sync sig
---------------------------------------------------------------------
0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1: 0 0 0 0 0 0 0 0 0 0 0 0 0 0
2: 0 0 0 0 0 0 0 0 0 0 3 0 0 0
3: 1.7m 78k 0 0 0 0 0 19 0 1 12 14 0 0
4: 0 0 0 0 0 0 0 0 0 0 0 2 0 0
5: 0 0 0 0 0 0 0 0 0 0 0 0 0 0
6: 2.1m 214k 2 2 1 0 2 21 0 10 15 7 0 0
Figure B-5

B.4. Wave Division Multiplexing

Wave Division Multiplexing (WDM) is a technology that can be used to increase bandwidth over existing fiber optic
communications networks by combining multiple optical signals onto a single fiber using different wavelengths. Different
networking protocols, such as Fibre Channel, gigabit Ethernet, and SONET can be transmitted concurrently over the same
fiber. WDM can be used in cooperation with Brocade SilkWorm switches and Extended Fabrics to extend Fibre Channel over
metropolitan networks and enable enterprise SAN solutions such as remote replication, backup, and storage consolidation.
There are two general WDM technologies available in today's networks: DWDM and CWDM.
Note: Brocade has performed extensive interoperability and SAN solution testing with many optical transport vendors. To
find out which CWDM and DWDM devices are Brocade Fabric Aware certified, refer to the Brocade Interoperability
Matrix, available at www.Brocadeconnect.com.
DWDM: Dense Wavelength Division Multiplexing (DWDM) is optimized for high speed, high capacity networks and long
distances. DWDM is suitable for large enterprise companies and service providers who lease wavelengths to customers. Most
equipment vendors can support 32, 64, or more channels over a fiber pair at speeds up to 10 Gbit/sec. Fiber distances between
nodes can generally extend up to 100 KM or farther. DWDM equipment usually provides a path protection scheme in case of
link failure and allows for switching in less than 50 ms. In short, DWDM offers the following advantages for SAN
connectivity:
Extended Distance - Enables SAN connectivity over metropolitan area networks
Scalability - Increases capacity by multiplexing channels over a single fiber
Self healing - Fast protection switching for high availability
CWDM: Course Wavelength Division Multiplexing (CWDM) provides the same optical transport and features of DWDM,
but at a lower capacity, which allows for lower cost. CWDM is generally designed for shorter distances (typically 50 to 80
KM) and thus does not require specialized amplifiers and high-precision lasers (lower cost). Most CWDM devices will
support up to 8 or 16 channels. CWDM generally operates at a lower bit rate than higher end DWDM systems, typically up to
2.5 Gbit/sec.
Two types of CWDM Solutions:
Transponder Based Solutions - Allows connectivity to switches with standard 850 or 1310 nm optical SFP or GBIC
transceivers. A transponder is used to convert these signals using O-E-O conversion to CWDM frequencies for transport
across a single fiber.
GBIC Based Solutions - These eliminate the need for transponders by requiring switch equipment to utilize special
CWDM transceivers reducing overall cost. CWDM GBICs and SFPs are just like any standard transceiver used in Fibre
Channel switches, except they transmit on a particular CWDM frequency. GBIC based CWDM are generally passive
devices.
B.4.1. Optical Components of WDM Systems

• Optical transmitters - Optical transceivers that may be pluggable (GBIC or SFP) or fixed.
• Transponders - converts optical signals to a specific wavelength using O-E-O conversion
• Optical amplifiers - used to boost signal strength allowing light to travel greater distances
• Multiplexers and demultiplexers - combine and separate optical wavelengths for transmission and reception
• OADMs - Optical Add Drop Multiplexers (OADMs) allow the addition or removal of wavelengths at designated points in
the network.

B.4.2. Topologies
DWDM providers generally offer several types of infrastructure topologies including point-to-point, hub-and-spoke, and ring.
Each Fibre Channel switch port connected to the ingress port of a DWDM creates a single point-to-point ISL with another
Fibre Channel switch at a remote site. Figure B-6 describes a point-to-point topology where two ISLs from separate fabrics
share a common fiber across the DWDM network. In the event of a connectivity failure between DWDM nodes, a protection
switch will occur to re-establish connection to the remote switches.
λ1 λ1
FC Switch 100 Km FC Switch
DWDM DWDM
λ2 λ2
FC Switch λ1 FC Switch
λ2
Figure B-6 Point-to-Point Topology With DWDM Path Protection
In the Ring topology shown in Figure B-7, SANs from three separate sites are interconnected. Site A has two ISLs to site C
connected over 45 KM. Site B has two ISLs to site C connected over 50 KM. In the event of a fiber failure between DWDM
nodes, all sites will maintain connectivity through the redundant 10 KM path between sites A and B.
FC Fabric 10km FC Fabric

A B
DWDM
DWDM
DWDM
Ring
45km 50km
DWDM
FC Fabric
C
Figure B-7 Ring Topology with Protection
B.4.3. Switch Configuration and Notes

Although not a requirement, it is recommended to configure Brocade Extended Fabrics on E_Ports connected over long
distance DWDM links to enable greater performance. Use the portCfgLongDistance CLI utility as described in the Extended
Fabrics chapter.
When utilizing the path protection capabilities of DWDM equipment, the Fibre Channel signal received at the ingress port of a
DWDM line card will be split and transmitted concurrently over both the active and protected links. The remote equipment
will then choose one of these signals to transmit to the end device. The signal chosen may be programmed by the administrator
of the DWDM or determined by the equipment itself. When a protection switch occurs due to a failure, services are generally
restored within 50 ms. Although this seems like an extraordinarily short time, there is still a high probability that a Fibre
Channel switch will detect a loss of sync condition and bring the link down (and then back up once restoration is complete). If
this occurs on a principal ISL, the fabric will perform a non-disruptive reconfiguration.

B.5. Multiprotocol Devices

Enterprise business requirements may necessitate SAN connectivity over greater distances than what native metropolitan
DWDM networks are able to provide. Fibre Channel gateways can be used to extend SANs over wide area networks by
encapsulating Fibre Channel frames into the payload of another protocol for transport over the WAN. A gateway will provide
point-to-point E_Port connectivity between Fibre Channel switches, separated by a protocol independent metro or wide area
network such as ATM, IP, or SONET.
E_Port E_Port
FC Switch Gateway Gateway FC Switch
MAN/WAN
Network
ISL
Figure B-8 SAN Gateway
Normal flow control based on available buffer credits on a switch port can limit overall performance when interconnecting
SANs over higher latency, long-distance WAN links. A gateway can compensate for delays by providing additional memory
for buffering and intelligent flow control that enables Fibre Channel transport at full speed over distances up to thousands of
kilometers.
A gateway device can maintain buffer-to-buffer flow control between its Fibre Channel port and the locally connected switch
port. This means that R_RDY primitives can be generated locally upon receiving a frame from the switch, keeping the pipe
full and reducing latency costs incurred over the long haul link to improve performance.
Note: To find out which Fibre Channel Gateway devices are Brocade Fabric Aware certified, see the Brocade Interoperability
Matrix, available on www.Brocadeconnect.com.
B.5.1. Switch Configuration and Usage

SilkWorm switch configuration for interoperability with gateway devices varies from vendor to vendor. Most will require the
use of Brocade Remote Switch license or enabling of ISL R_Rdy mode on the switch port connected to the gateway device.
See section B.2. SilkWorm Product Family Configuration and Functionality on page B-1 for information on enabling these
features.

B.6. SAN Distance Solutions

Companies maintaining critical data and applications requiring minimal or no downtime must be prepared in the event of
disaster. Careful planning of a disaster recovery solution is imperative to make the impact of disaster as transparent and
minimal as possible. Depending on costs and business requirements, this may mean restoring services within hours or
immediately after a failure. Since there is no single disaster recovery solution for every business, companies must assess all
requirements and define a solution that works best.
B.6.1. Remote Data Replication

Remote data replication or mirroring allows data to be sent off-site where it can be protected from hardware failures and other
disasters. Replication solutions may transfer data synchronously, asynchronously, or by point-in-time copies. The speed or
latency of the remote connection, how current remote data is required to be, and replication performance impacts must be
considered when making off-site copies of data. Figure B-9 displays an SAN topology for remote data replication over IP
networks using a Fibre Channel gateway.
FC Switch FC Switch
FC-IP IP FC-IP
Gateway Network Gateway
FC Switch FC Switch
Replication
Figure B-9 Data Replication Over FC-IP
Using a split mirror or snapshot copy of a volume for tape backup allows applications to remain available at all times and
eliminates backup windows. The SAN described in Figure B-10 mirrors volumes locally at the production site and creates a
point in time copy of data at a backup site 100 KM away. The volume copy can then be used to create tape backups whenever
needed.
FC Switch
100 Km
FC Switch
DWDM DWDM
FC Switch FC Switch
Snapshot Copy
Locally mirrored Snapshot for

volumes tape backup
Figure B-10 Remote Snapshot for Tape Backup

B.6.2. Centralized Backup

Companies in a campus or multi-site environment may wish to optimize tape library utilization and reduce costs by
implementing a centralized remote tape backup strategy. Centralized remote tape backup can enable a disaster protection
solution allowing multiple sites to share a common tape library. Centralized backup can also reduce costs associated with
maintaining individual distributed backups and eliminates manual transport of tapes to off-site locations.
The scenario in Figure B-11 describes a centralized backup solution for Sites A and B. Both sites share the tape library
provided by site C through a redundant SAN fabric interconnected by DWDM. The backup management server also located at
site C oversees backup windows for clients at each location.
Site A: Backup Site B: Backup

Client and local Client and local
disk Storage disk Storage
FC Switch FC Switch
10km
FC Switch FC Switch
DWDM
DWDM
DWDM
Ring
45km 50km
DWDM
FC Switch FC Switch
Site C: Remote
Backup Site
Backup Tape
Management Library
Server
Figure B-11 Tape Backup Consolidation Over DWDM

B.6.3. Storage Consolidation

Organizations can centralize storage across a campus or geographically dispersed environment, or even remotely out-source
the work to a Storage Service Provider (SSP). Storage consolidation can streamline administration tasks by managing storage
resources at a single site and maximize utilization. Brocade Zoning and Secure Fabric OS can be used to control access to
storage devices over the network.
Site A Site B
Datacenter Datacenter
FC Switch FC Switch
10km
FC Switch FC Switch
DWDM
DWDM
DWDM
Ring
45km 50km
DWDM
FC Switch FC Switch
Site C
Data center with
shared storage
Figure B-12 Remote Storage Consolidation


Appendix
Glossary
C
C.1. Terms and Definitions

These terms and definitions are provided to ensure that a consistent language for describing SANs is used throughout the
document and so that the reader understands what these terms mean. This section is not intended to be all-inclusive.
Table C-1 Key Terms and Definitions
Terms Definitions and Impact

“golden” switch For larger SAN fabrics, or for the staging of many smaller fabrics, use
configurations configupload to baseline the “golden” switch. The “golden” switch configuration
can be downloaded with Fabric Manager to all others in the fabric. Do baselines by
Fabric OS version. In other words, if the fabric contains Fabric OS 3.1 and 4.1 switches
create two “golden” switch configurations.
Blocking The inability of one device to connect to another device. The Brocade Virtual Channel
implementation of Fibre Channel does not block. The term blocking is often confused
with the term congestion.
Congestion If two or more sources contend for the same destination, performance for each source
may decrease; however, available bandwidth is shared fairly by all sources contending
for the same destination. Congestion is the realization of the potential of
over-subscription. Congestion may be due to contention for a shared storage port or host
port, or an ISL.
Control Processor The term control processor is associated with a SilkWorm 12000 component/FRU (field
replacable unit). The SilkWorm 2000, 3200, 3800, and 3900 switches do not have a
FRU specifically associated with it and when CP is used in the context of other
SilkWorm switches, the reference is to the switch CPU and not a FRU.
Core PID Format The 24-bit Switch Fabric Port Identification (PID) also known as SID consists of
Domain_ID, Area and AL_PA fields.
Core Switch Also known as a “core fabric switch.” This is one of the switches at the logical center of
a Core/Edge fabric. There are generally at least two core switches per Core/Edge fabric
to enable resiliency within the fabric. Ports on a core switch are normally used for ISLs.
Hosts and storage also connect to the core switch.
DASD Direct Attach Storage Device
Edge Switch This is one of the switches on the logical outside edge of a Core/Edge fabric. There are
generally many more edge switches than core switches. Ports on edge switches are used
for SAN device connections.
Brocade SAN Design, Deployment, and Management Guide C-1

C Glossary
Fabric One or more interconnected Fibre Channel switches. The term “Fabric” usually refers to
the interconnected switche(s), but can also account for the interconnected switches and
the devices connected to the fabric.
Fabric build The build fabric Switch Fabric Internal Link Service requests a non-disruptive
(BF) configuration to the entire fabric. A BF process shall not cause the Domain_ID list to be
cleared. This preserves existing node port addresses and allows open exchanges to be
completed.
Impact: Fabric build is a non-disruptive process to I/O.
Fabric Port Count The number of ports available to connect SAN devices in a fabric. ISLs ports (E-ports)
are not included in this count. (Also known as user port count.)
Fabric Re-Configura- Fabric reconfiguration is a disruptive fabric operation during which domain IDs may
tion (RCF) change. If the Domain_ID changes, all attached node ports must re-login with the fabric
and be assigned new N-Ports identifiers reflecting the change in Domain-IDs.
Impact: Reconfigure causes Class-n frames (1,2,3,4 or 6) to be discarded and class 1
connection to be abnormally removed.
Fabric Segmentation A fabric is unable to resolve the switch configuration parameters during the rebuild
process with one or more switches, and may isolate them from the fabric, causing fabric
segmentation.
Impact: I/O operations are ceased only on those devices losing their access due to
segmentation.
Fabric Topology A topology is “the logical layout of the components of a computer system or network
and their interconnections.” A fabric topology is the layout of the switches that form a
fabric.
Fan-in The ratio of storage ports to a single host port.
Fan-out The ratio of host ports to a single storage port.
FRU Field Replaceable Unit
FSPF Fabric Shortest Path First protocol. The FSPF protocol was developed by Brocade and
subsequently adopted by the Fibre Channel standards community for allowing switches
to discover the fabric topology and route frames correctly. It is now the industry
standard routing protocol for Fibre Channel networks.
HA High Availability
High Locality If devices that communicate with each other are connected to the same switch or groups
of switches then these devices have high locality. The higher the locality, the less traffic
crosses ISLs/trunks and therefore, fewer ISLs/trunks are needed.
Hop Count For evaluating SAN designs, the hop count is identical to the number of ISLs that a
frame must traverse to reach its destination.
Host Edge Switch Edge switch with host device connections only.
Incremental Upgrade Replacing one switch at a time in an online fabric.
ISL Inter-Switch Link. ISLs connect two switches via E-ports.
C-2 Brocade SAN Design, Deployment, and Management Guide

Glossary C
ISL Over-Subscription In networks where all ports operate at the same speed, the over-subscription ratio for an
Ratio ISL is the number of different ports that could contend for the use of its bandwidth. If
there are 14 node ports on a switch and two ISLs, the ratio is 14:2, or 7:1. When there is
a mixture of port speeds, the exact calculation is not as simple. The rule of thumb is that
the lower the ratio is, the better performance is likely to be.
Latency The time it takes for a frame to traverse from its source to its destination is referred to as
the latency of the link. Sometimes a frame is switched from source to destination on a
single switch and other times a frames must traverse several hops between switches
before it reaches its destination.
Locality The degree that I/O is confined to a particular switch or segment of a fabric. If two
devices that need to communicate with each other are located on the same switch or
fabric segment, then these two devices are said to have high locality. If these same
devices are located on different switches or segments of a fabric and these two devices
need to communicate with each other, then these devices are said to have low locality.
Logical Switch The SilkWorm 12000 can contain up to 128 ports in a 14U chassis, configured as two
64-port switches. Each switch is known as a logical switch and may also be referred to
as a domain.
Low Locality If two devices must cross an ISL/Trunk to communicate, then these devices have low
locality. The lower the locality, the more traffic crosses ISLs/trunks and therefore, more
ISLs/trunks are needed.
NMS Network Management Software
Node Any SAN device – usually either a host or storage device – that attaches to a fabric.
Node Count The number of nodes attached to a fabric.
Octet An octet is a group of two horizontally adjacent quads. The SilkWorm 3900 is the only
SilkWorm switch that implements octets. Octets are used primarily to define boundaries
for performance tuning purposes.
Offline Fabric A non-functional state of a fabric unsuitable for I/O operation.
Online Fabric A functional stable state of a fabric performing reliable I/O fabric operations.
Over-Subscription A condition where more nodes could potentially contend for the use of a resource – such
as an ISL – than that resource could simultaneously support, that resource is said to be
over-subscribed.
PID bindings Static mapping between physical and logical devices on a host accomplished via
Port_ID (PID).
Redundant Fabric A SAN composed of two or more independent fabrics The multiple fabric architecture
makes dual fabric SANs redundant.
Resilience The ability of a fabric to adapt to or tolerate a failure of a component.
SAN A Storage Area Network (SAN) can consist of one or more related fabrics and the
connected SAN devices.
Brocade SAN Design, Deployment, and Management Guide C-3

C Glossary
SAN Architecture The overall design or structure of a storage area network solution. This includes one or
more related fabrics, each of which has a topology. Other components may also be
included, such as host, storage, and other SAN devices.
SAN Port Count The number of ports available for connection by nodes in the entire SAN. The SAN Port
Count equals the fabric port count in a single fabric SAN and is equal to the sum of each
fabric’s port count in a multi-fabric SAN.
Scalability The ease with which a particular design can grow and adapt without requiring a
significant change in SAN architecture or requiring a substantial re-layout of existing
SAN devices.
Secure Mode Disabled An operating mode where all switches that participate in the fabric are unable to
successfully execute the command secModeEnable or if the command
secModeDisable is successfully executed in the fabric.
Secure Mode Enabled An operating mode where all switches that participate in the fabric are running a version
of Fabric OS that supports the security feature, have licenses to run security, and the
command secModeEnable has been successfully executed.
Single Fabric A SAN composed of a single fabric may be configured to provide one or more paths via
different switches of the fabric.
Impact: Offers no Protection at fabric level. All paths are closed when fabric is offline,
completely stopping I/Os.
SPOF A single point of failure. A SPOF in a SAN is any component – either hardware or
software – that could cause a fabric or a SAN to fail.
Storage Edge Switch Edge switch with storage device connections only.
Tiering The process of grouping particular SAN devices by function and then attaching these
devices to particular switches or groups of switches based on that function
Total Ports The total number of ports of all the switches
User Ports Total number of switch ports less ports used for ISLs/trunks
C-4 Brocade SAN Design, Deployment, and Management Guide

Appendix
Site Survey Templates
D
Brocade SAN Design, Deployment, and Management Guide D-1

1 of 3
Brocade SAN Site Survey

Completed By:
First Name: Last Name: Phone: Mobile
Email: Pager: Fax
Address: Country/City/State/Zip
Site Name and Location

Name: Phone: Fax
Site Contact #1
First Name: Last Nam e: Phone: Mobile
Email: Pager: Fax
Site Contact #2
Email: Pager: Fax
Site Contact #3
Email: Pager: Fax
Site Profile Summary & Implementation Schedule:

Include the purpose or purposes of the SAN. For example, LAN-Free Backup, Storage Consolidation, Remote Replication, etc. In
addition, please add any other critical information regarding this site. The list of items should include but is not limited to site power, fiber
cable requirements (including location, lengths, etc), any fiber patch panels, proposed rack layout, etc.
Attach a project plan with a Visio diagram, proposed rack layouts and an installation schedule. When complete, the SAN project
manager/SAN Architect will be able to assess the staging requirements and be able to assign/schedule resources as needed.
Survey Authorization
Site Representative: Date:
2 of 3

SAN Topology Assessment
Item 1 Please attach a high level list of all hardware devices to be attached. Completed
Notes: Each device should have the make/model identified. If 2 topologies are identical in every way then a single list will be sufficient. This list must
include all switches, HUBS, DWDM, Gateways, and other devices for connecting SANs over distance.
Please attach a high level topology diagram that illustrates the location of each existing
Item 2 or new fabric member switches in the SAN at this site. If possible, include the edge Completed
devices as well.
Notes: Each SAN should have every device represented by an ICON. Each fabric should be represented as a cloud that can be referenced to the correct
topology. A single SAN frequently incorporates more than one fabric (redundant fabric architectures for example). When illustrating such a SAN it is
important to show how all devices connect to each fabric. Since clouds will be used it is not necessary to identify which switch/port each device has been
connected to.
Switch Output
If migrating into an existing SAN please attach the supportShow output from each of
Item 3
the existing switches
Notes: The output of supportShow should be saved to a file and then attached to this document. If there are any non-brocade switches in the fabric then
similar information should be captured.
Device Configuration
For each device in the SAN please provide a detailed profile. See below for a list of
Item 4 Completed
information that should be included
Complete hardware profile
Server Storage Miscellaneous Device

OS Patch level Disk type Disk Capacity Make Model
Bus Speed Bus Type Disk Firmware # of Disks Firmware Driver
CPU Other Data RAID Controller Firmware Other Information
Port Identfication/Information (For Affected Existing SANs)
Identification of each Fibre Channel card in that device. Include the Firmware and Driver
For each card, identify each port and the switch/port it is connected to.
For each port, identify what device(s) it has access to across the SAN.
For each port, identify what fabric zones it is a member of
Notes: Each device should be reference as follows xxxxxxxxxx-yy-zz , where the first segment is a 10 character ID for a device, the second
segment is a 2 digit number identifying the slot, and the last segment is a 2 digit number identifying the port.
Software Application Information
Identify each software application required for that device.
Backup Applications
Multi-path Applications
Management Application
Other applications
LUN Security Configuration
3 of 3

SAN Requirements Checklist
Target number of user device ports in SAN environment (with any
phasing of implementation)
Number of Fabrics in SAN
Number of sites in environment
Distance requirements
Number /types of hosts
Number /types of storage devices
Number /types of tapes
Other devices
Customer requirement for fail over/redundancy, reliability of SAN
Customer scaling plans
SilkWorm 12000 Pre -Install Checklist
Task Complete
The voltage required is 200 to 240 VAC, 50 Hz or 60 Hz. Two dedicated branch circuits required for redundancy. RECEPTACLE
Confirm the power cords ordered with the system to ensure the correct power receptacles are installed.
NEMA
o In the US and most of North America, the available voltage is usually 208 or 240 VAC, the receptacles are
NEMA L6-20R, on individual branch circuits, each rated 20 amps.) UK
o In UK, Ireland, Hong Kong, two UK-standard 13 amp receptacles, on individual branch circuits, are required.
o In most of continental Europe, two CEE7/7 “Schuko”-co mpatible receptacles are required, each rated 16 amps, CEE7/7
OR two IEC60309, 230V~16A-6h receptacles. Verify with the system order.
o In Australia and New Zealand, 2 Australian-standard, 15 amp receptacles, on individual branch circuits rated 15 AUS/NZ
amps each, are required.
o For any location in which the US, UK, “Schuko” or AUS/NZ receptacles types are not accepted, the International IEC-60309
standard, IEC-60309, 230V~, 16A-6h, receptacles should be planned, and the system order confirmed.
Appropriate licenses ordered
Soft copy of manuals provided to customer
Customer training scheduled
Brocade personnel have access to customer site
Verify that customer loading docks are adequate for delivery
Verify appropriate shelf kits are available: standard (4 posts) or telco (2 posts)
Assure adequate aisle space from staging to install area for 12000(s)
Verify cooling vents and cut tiles available
Cables on site (LC-LC, SC-LC)
If existing switches are going to be connected to the 12k, are these switches running at least
v2.6.0c (2000 series) or v3.0.2c (3000 series) ?
Is core pid format enabled on existing SilkWorm 2000 & 3000 series switches?
SAN Components Inventory
The worksheets in this Excel workbook are intended to be used as a template and starting point for you to
record:
§ an inventory of your current storage infrastructure and analysis of whether those existing components
should be used in your Storage Area Network (SAN)
§ new components needed for your SAN
§ component compatibility
§ needed port count for your SAN
Enhance the worksheets as you see fit to meet your needs. The completed spreadsheet
can be used to help you purchase your SAN components.
For more detailed information on how to use these worksheets, please visit the following steps in the Planning
and Design section of the Brocade SAN Info Center (www.brocade.com/san):
Step 2: Inventory and Analyze Your Environment
Step 3: Determine Your SAN Components
We appreciate your feedback regarding the usefulness of these worksheets, and how we might improve them
for use by others. Please send your comments to:
saninfocenter@brocade.com
All contents herein and in the attached worksheets are Copyright 2002 Brocade Communications Systems, Inc. All rights reserved. ALL INFORMATION AND
OTHER MATERIALS AND IN THE ATTACHED WORKSHEETS ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. Version 2.1, June 2002
SAN Components Inventory: HOSTS
Host Identifier
New (N) or existing (E)
Manufacturer
Make
Model
Operating System
Total Slots
Type A
Type (PCI, SBUS, etc)
Number
Size
Speed (Mhz)
Type B
Type (PCI, SBUS, etc)
Number
Size
Speed (Mhz)
Single (S) or Dual (D) Attached
Ports
# Ports required for single attached
# Ports required for dual attached
HBA
General information
- make
- model
- version
- number of ports
- number of HBAs
- slot numbers
- DMP/Failover Application
Driver type: (check applicable type)
-fabric
-PTP
-private loop
-public loop
Connections supported per HBA
Fibre Channel Ready? (Y or N)
Server
- Dimensions
- Power Requirements
- Console location
- Ethernet interface list
- Phone line required for mgmt? (Y/N)
- Physical location
Application list (with version)
Application Name 1
Application Name 2
Application Name 3
Application Name 4
SAN Components Inventory: HOSTS
Host Identifier
Storage requirements
Requirement 1
Application Name
Initial Requirements
Projected Requirements
Requirement 2
Application Name
Requirement 3
Application Name
Requirement 4
Application Name
SAN Components Inventory: DISK STORAGE DEVICES
Storage Identifier
New (N) or existing (E)
Manufacturer
Make/model
Firmware version
Ports
Max # servable hosts per frame
Max # hosts per port
Interfaces
SCSI (SC) or Fibre Channel (FC)
Type of SCSI
- wide/narrow
- differential/single ended
Fibre Channel Ready? (Y/N)
# Fibre Channel interfaces needed
# Existing Fibre Channel interfaces
- make/model
# Fibre Channel interfaces to purchase
- make/model
Ethernet interface list
Connection supported
(check applicable connection type)
-fabric
-PTP
-private loop
-public loop
- SCSI
- SSA
- ESCON/FICON
SAN Components Inventory: TAPE LIBRARIES
Tape Identifier
New or existing (N/E)
Manufacturer
Make/model
Firmware version
Ports
Interfaces
SCSI (SC) or Fibre Channel (FC)
Type of SCSI
-- wide/narrow
-- differential/single ended
Fibre Channel Ready? (Y/N)
# Fibre Channel interfaces needed
# Existing Fibre Channel interfaces
-- make/model
# Fibre Channel interfaces to purchase
-- make/model
Ethernet interface list
10BaseT, 100BaseT, etc.
Connection supported
(check applicable connection type)
-fabric
-PTP
-private loop
-public loop
-private loop
-public loop
- SCSI
- SSA
- ESCON/FICON
SAN Components Inventory: SWITCHES
Switch Identifier
Zoning info
Firmware Version
IP Address(es)
Gateway
Port configurations
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
Switch Identifier
55
56
57
58
59
60
61
62
63
Devices attached to each port
(type, WWN, etc.)
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Switch Identifier
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
Licensing
Security information
Passwords
Control Port info
Ethernet, telnet
Special settings
Operating temperature
Power requirements
Phone line required for mgmt?
Yes or No (Y/N)
Physical location
Firmware level
Device Name Number of Devices Rack Unit per Device Total Used Rack Units
Fiber Patch Panel 0
KVM Unit 0
Host 1 0
Host 2 0
Cable Guide 0
Silkworm 2800 2 0
Cable Guide 0
Silkworm 3800 1 0
Cable Guide 0
Silkworm 3900 1.5 0
Cable Guide 0
Silkworm 12000 14 0
SW 12000 Cable Guide 0
Total Units Available 42

Total Units Used 0
Total Units Remaining 42
Hosts IP Address HBA KVM HBA Model HBA Host Fab A/B Storage 1 Fab A/B Storage 1 Storage 1 Storage 1 Storage 1
Windows Port # PWWN Switch Ports Switch Ports Controller Ports PWWN Vol Assigned Vol Size
3. w2k CPQ123 Hba0 16G LP9002L 7482 100/s3/p0 100/s2/p4 3bA-3052 71,72
Hba1 LP9002L 4eeb 150/s3/p0 151/s9/p4 4b-3053 71,72
Chassis Name: <name> (Brocade SilkWorm 12000)
Area for Logical Switch 0 CPs Area for Logical Switch 1
Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7 Slot 8 Slot 9 Slot 10
I 15 15 15 15 15 15 15 15
S
14 14 14 14 14 14 14 14
L
' 13 13 13 13 13 13 13 13
s
12 12 12 12 12 12 12 12
11 11 11 11 11 11 11 11
10 10 10 10 10 10 10 10
9 9 9 9 9 9 9 9
H
O 8 8 8 8 8 8 8 8
S
T
CP1 CP2
7 7 7 7 7 7 7 7
S
6 6 6 6 6 6 6 6
5 5 5 5 5 5 5 5
4 4 4 4 4 4 4 4
S
T 3 3 3 3 3 3 3 3
O
2 2 2 2 2 2 2 2
R
A 4 1 1 1 1 1 1 1
G
E
0 0 0 0 0 0 0 0
Grey - mark as OPEN PORT

Index
A breakout cable 14
ACL 5 Brocade Extended Fabrics 1
Advanced Configuration Extended Fabrics Modes 2
Command Set 16 buffer-to-buffer credits 2
Advanced Performance Monitoring bundled cable 14
AL_PA Monitoring 7 C
End-to End Monitor 5 cable hazards 23
Fabric Watch 7 Cable Layouts
Features 3 planning 16
Filter-Based Monitoring 7 Cable Management
Filter-based Monitoring 6 bend radius 21
Performance Class 7 best practices 23
Performance Graph Canvas 3 cable guides 11
Performance Monitor Menus 2 cable types 12
Threshold Setup 7 Design Process 5
Web Tools 2 documentation 19
Advanced Thresholds Configuration implementation 19
Behavior Type 15 Infrastructure 2
Threshold Alarm Level 15 Inter-Site Connectivity 6
Threshold Boundary Level 15 maintenance 24
AL_PA Monitoring 7 needs analysis 1
Alarm Notication Patch Panel Design 6
Types 6 requirements 10
API Scripting Site Design 5
Advanced Interface 2 Site Requirements 3
Basic interface 2 Cable Types 12
Communication Process 3 cables
Direct Interface 2 breakout 14
interface levels 2 bundled 14
Platform Support 4 multi-mode 12
Session Management 4 ribbon 15
Application Programming Interface (API) Policy Cables Management
43 hazards 23
ATM gateway 8 Cabling Standards
Availability 12 between racks 18
B patch panels 18
Backup FCS Switch 22 single rack 17
bend radius limitation 21 Call Home Support 36
Brocade SAN Design, Deployment, and Management Guide Index-1

Centralized Backup 12 Web Tools 3
CLI Fabric Manager
diagnostic commands 8 Call Home 36
Community Names Customizing View Window 9
Verification 5 Event Monitoring 35
Community Strings Fabric Backup and Compare 26
Setup 11 Fabric Login 16
Connecting Devices Fabric Merge Check 24
Core 13 Fabric Refresh 36
Creating Options Policy 44 Firmware Download 18
customizing zoning environments 4 Information View Window 8
D Installation 3
DCC 41 Installation Requirements 4
Device Connection Control 21 ISL checking 36
Device Attachment 1 Key Features 1
Switch ISL/Trunk Connections 4 License Key Management 31
Trunk and ISL Connections 1 Navigation 7
Device Connection Control 41 SAN Element Window 8
Device Connection Control (DCC) 21 Secure Fabric OS
Device Connections API Policy 43
Tiering 14 Creating Options Policy 44
diaghelp 8 DCC Policy 41
Diagnostic FCS Policy 40
commands 8 Front Panel Policy 44
Diagnostics HTTP Policy 43
distance 7 MS Policy 43
Distance Policy Management 38
performance 3 RSNMP Policy 43
DWDM 10 SCC Policy 39
E Serial Port Policy 43
email notification setup 21 SES Policy 43
End-to-End Monitor Telnet Policy 42
masking setup 6 WSNMP Policy 42
Event Monitoring 35 Sequence Switch Reboot 21
Extended Fabrics 6 switch configuration guidelines 6
Configuring 2 switch requirements 6
configuring 2 Fabric Manger
F SAN Discovery 9
Fabric Access Fabric Merge Check 24
supported features 5 Fabric Resiliency 5
Fabric Backup and Compare 26 Fabric Watch
Fabric Configuration Server 40 Advanced Configuration 16
Fabric Configuration Server (FCS) 21 Advanced Configuration Menu 14
Fabric Login 16 advanced threshold configuration 17
FAbric Manager Advanced Thresholds Configuration 14
Index-2 Brocade SAN Design, Deployment, and Management Guide

Alarm Messages 7 Hyper Text Terminal Protocol (HTTP) Server Pol-
Alarm Notification 6 icy 43
Alarms 5 I
Behaviors 5 I/O Profiles 22
Best Practice 18 IBM Fibre Connections (FICON®) 8
commands 3, 16 intermix mode 8
components 4 Inter-Switch Link
Configuration Tables 23 checking 36
configuration with Web Tools 17 ISL checking 36
Counter Values 8 ISL Oversubscription Ratios 13
Email Notification Setup 21 ISL R_Rdy Mode 4
initialization 2 L
installation 2 License Key Management 31
Recommendations 18 Link Budgets 6
Advanced Performance Class 20 Locality 17
Environment Class 18 long distance ISLs 7
E-port Class 19 LWL
Fabric Class 19 ISL Trunking support for 5
Fabric F/FL Class 20 M
Port Class 19 MAC
Security Class 20 Management Access Control 21
SFP Class 19 Management Access Control (MAC) 21
Theory of operation 8 Management Functions
Threshold Boundary Limits Switch and Port level 11
Above Trigger 11 Switch Information 11
Below Trigger 11 Management Server (MS) Policy 43
Changed Trigger 12 Manual Fabric Refresh 36
defining 11 MTP patch panel 15
Exceeded Trigger 13 Multi-fabric, non-resilient 6
In-between Trigger 13 Multi-fabric, resilient
Thresholds 4 6
Time Base Event 8 multi-mode fiber cable 12
Traits 5 multiple zones 4
FCS 40 Multiprotocol Devices 10
Fabric Configuration Server 21 N
Fiber Loss and Link Budgets 6 Non-FCS Switch 22
Firmware Download 18 O
frame buffers 2 Optical Fiber
Front Panel Policy 44 consideration for distance 5
G Optical Transceivers 6
gateway, Remote Switch 8 optimizing resources through zoning 4
H overlapping zones 4
hop counts 23 P
horizontal cable guides 11 Patch Panel Design 6
Hot code activation 19 patch panels

MTP 15 SES Policy 43
Performance 21 Switch Connection Control Policy 39
performance Telnet Policy 42
distance 3 WSNMP Policy 42
Port Management Secure Management Channel 21
Extended Fabrics 17 Secure Mode
Functions 15 Web Tools 5
Port Setting 15 Serial Port Policy 43
Trunk Information 17 Single fabric, non-resilient 6
Primary FCS Switch 22 Single fabric, resilient 6
R SNMP
Remote Data Replication 11 Access Control 5
Remote Switch 8, 4 authTraps 5
ribbon cable 15 MAC Policy 11
RSNMP Policy 43 MIB Support Set 6
S MIBs/Trap Support 8
SAN Availability Classifications 6 Secure Fabric Mode 11
SAN Discovery 9 Track Changes Set 7
SAN Distance Solutions 11 Web Tools 7
SAN Element Logical Grouping 12 SNMP settings
SAN Management agtcfgset 3
Philosophy 1 configuring and reviewing 3
Scope 2 Specifying Trap Level 4
Tool Selection Guidelines 5 swEventTrapLevel 4
Tools & Interfaces 3 Storage Consolidation 1, 13
SAN Solutions 1 Switch Connection Control 39
Backup 2 Switch Connection Control (SCC) 21
Business Continuance 4 Switch ISL/Trunk Connections 4
High Availability 3 Availability 12
Storage Consolidation 1 Core Switch 10
Scalability 19 Switch Management Functions
SCC 39 Configure 14
Switch Connection Control 21 License Admin 14
SCSI Enclosure Services (SES) Policy 43 Network Config 13
Secure Fabric OS Routing 14
API Policy 43 SNMP 14
Creating Options Policy 44 Upload/Download 13
Device Connection Control Policy 41 SWL
Fabric Configuration Server Policy 40 ISL Trunking support for 5
Front Panel Policy 44 T
HTTP Policy 43 Telnet Policy 42
MS Policy 43 Tiering 14
Policy Management 38 Time and Date Synchronize
RSNMP Policy 43 Fabric Manager
Serial Port Policy 43 Time and Date 35
Index-4 Brocade SAN Design, Deployment, and Management Guide

Topologies
Cascade 25
Complex Core 32
Core/Edge 29
Full Mesh 27
Hybrid 32
Partial Mesh 28
Ring 26
Track Changes
Agent Configuration 7
Default Settings 7
Troubleshooting
distance 7
Trunk and ISL Connections 1
Trunking 9
W
Wave Division Multiplexing
about 8
components 8
Switch Configuration 9
topologies 9
WDM
see Wave Division Multiplexing 8
Web Tools
Fabric Manager Client 3
Features 6
launching 2
Main views 2
requirements
switch 1
workstation 1
Secure Mode 5
Switch view 4
Zone Administration 6, 8
WSNMP Policy 42
Z
Zone 6
zone configurations 4
zone objects 4
zones
multiple 4
overlapping 4
zoning
customization 4

DDM v3 0 53-0000366-03 C

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

DDM v3 0 53-0000366-03 C

Transféré par

Droits d'auteur :

Formats disponibles

Brocade SilkWorm

Design, Deployment, and

Publication Number: 53-0000366-03

Latin America Headquarters

Document version Publication Number Publication Date

First Publication 53-0000366-01 4/30/2003

Chapter 1 Product Introduction

Section I SAN Design

Brocade SAN Design, Deployment, and Management Guide i

Chapter 3 Architecting SANs With SilkWorm Switches

ii Brocade SAN Design, Deployment, and Management Guide

Chapter 5 Deployment - Planning

Brocade SAN Design, Deployment, and Management Guide iii

Chapter 6 Deployment - Staging and Validation

iv Brocade SAN Design, Deployment, and Management Guide

Brocade SAN Design, Deployment, and Management Guide v

Chapter 7 Maintenance and Operations

vi Brocade SAN Design, Deployment, and Management Guide

Brocade SAN Design, Deployment, and Management Guide vii

Chapter 9 Brocade Fabric Manager 4.1.x

viii Brocade SAN Design, Deployment, and Management Guide

Chapter 10 Brocade Advanced Web Tools Overview

Brocade SAN Design, Deployment, and Management Guide ix

Chapter 11 Command Line Interface (Telnet) Overview

Chapter 12 SNMP Overview

x Brocade SAN Design, Deployment, and Management Guide

Chapter 13 Brocade Fabric Watch Overview

Brocade SAN Design, Deployment, and Management Guide xi

Chapter 14 Advanced Performance Monitor (APM) Overview

Chapter 15 Brocade API Scripting Toolkit Overview

xii Brocade SAN Design, Deployment, and Management Guide

Appendix B Long-distance Technologies for

Brocade SAN Design, Deployment, and Management Guide xiii

Appendix D Site Survey Templates

xiv Brocade SAN Design, Deployment, and Management Guide

Brocade SAN Design, Deployment, and Management Guide xvii

bold text • identifies GUI elements

xviii Brocade SAN Design, Deployment, and Management Guide

Note: Notes emphasize important information.

Brocade SAN Design, Deployment, and Management Guide xix

Additional Resource Information

xx Brocade SAN Design, Deployment, and Management Guide

1.1. Brocade Fibre Channel Switches

Table 1-1 Switch Product Summary

Brocade SAN Design, Deployment, and Management Guide 1-1

1.1.1. Brocade SilkWorm 24000

1.1.2. Brocade SilkWorm 12000

1.1.3. Brocade SilkWorm 3900

1.1.4. Brocade SilkWorm 3250/3850

1.1.5. Brocade SilkWorm 3200/3800

1-2 Brocade SAN Design, Deployment, and Management Guide

1.2. Fabric Operating System (FOS)

1.2.1. Brocade Fabric OS 4.2

1.2.2. Brocade Fabric OS 3.1.2

1.2.3. Brocade Fabric OS 2.6.2

1.3. Fabric OS Licensed Features

1.3.1. Brocade Advanced Web Tools

Brocade SAN Design, Deployment, and Management Guide 1-3

1.3.2. Brocade Advanced ZONING

Server1 Loop 1 Server3

Red zone Green zone

Figure 1-1 Fabric with Three Zones

((61) + (62)) / (4*2) = 2.25 to 1