Configuring I/O modules

CX4 Family

Dan Nowlan
Mike Hamel

10/26/2010 Rev 4

EMC CONFIDENTIALINTERNAL USE ONLY 1

 I/O modules

Supported I/O modules on CX4 Arrays

SLIC Code Name SLIC Label Ports BE/FE? Minimum Flare Rev

Tomahawk 4Gb Fibre 4 BE/FE 04.28.000.5.003

Harpoon 1Gb GbE/TOE 2 FE only 04.28.000.5.003
Glacier 8Gb Fibre 4 BE/FE 04.28.000.5.704
Poseidon (optical) 10Gb GbE 2 FE only 04.29.000.5.001
Heatwave 10Gb FCoE 2 FE only 04.30.000.5.507

EMC CONFIDENTIALINTERNAL USE ONLY 2

 Port Limits enforced by Flare (per SP)

CX4-120 CX4-240 CX4-480 CX4-960 CX4-960

w/960 enabler

Max FC BE 1 2 4 4 8

Max FC (FE + BE) 8 8 12 16 20

Max iSCSI (1G + 10G) 4 6 6 8 8

Max 10G iSCSI 2 2 4 4 4

Max 10Gb FCoE 2 2 4 4 4

y FLARE maximum port limitations/allowances

y Purple indicates New for Taurus (R29)
y Blue indicates New for Jupiter (R30+)
EMC CONFIDENTIALINTERNAL USE ONLY 3
 SFP Rules

y Removing an SFP will NOT cause the SP to reboot

y To replace an SFP, simply pull out the bad SFP and insert a good one.
No need to reboot the SP.
y SFPs can be inserted into a port at any time
y To persist ports on a 4Gb or 8Gb Fibre SLICs, you must have SFPs
inserted.
y Both ports will be persisted on a 10GbE SLIC even if there are no SFPs
inserted.

EMC CONFIDENTIALINTERNAL USE ONLY 4

 SFP rules for Fibre Front-End ports

y CX4 arrays support two different classes of Fibre Channel SFPs for front-
end ports
One type supports 1Gb, 2Gb and 4Gb speeds
The other type supports 2Gb, 4Gb and 8Gb speeds

y A port can only run at speed X if both the SFP and the SLIC support that
speed. Mixing-and-matching qualified SFPs and SLICs is allowed, but
restricts the speed that a port can run at.
y Navisphere will list the available speeds for a particular port see
values below in blue.

If SFP supports If SFP supports

1Gb/2Gb/4Gb 2Gb/4Gb/8Gb
4Gb Fibre SLIC 1Gb, 2Gb, 4Gb 2Gb, 4Gb
Tomahawk
8Gb Fibre SLIC 2Gb, 4Gb 2Gb, 4Gb, 8Gb
Glacier
EMC CONFIDENTIALINTERNAL USE ONLY 5
 Handling I/O modules

y New I/O modules can be inserted into an open slot in the array at any
time
y CAUTION: Removing a powered up I/O module will cause the SP to
immediately reboot
This reboot is not graceful the reboot occurs immediately, regardless of what the
software on the SP is doing
Exception: The SP will NOT reboot if the I/O module is not powered up*

y *When an I/O module is first inserted, it will not power up until the SP is
rebooted

EMC CONFIDENTIALINTERNAL USE ONLY 6

 Adding Ports

y New backend or frontend ports can be persisted at any time, but

requires rebooting both SPs one at a time (coordinated reboots)
Must adhere to the arrays port limits (See Slide 3 - these limits are platform-specific)
Both SPs must have the same port configuration

y NST is the preferred method

Runs HA validation checks before rebooting SPs to avoid DU/DL
Checks for symmetry between SPs

y NaviSecCLI procedure is also supported

NaviSecCli ioportconfig persist will reboot both SPs, one at a time in a coordinated
fashion. Allow the command to perform both reboots.
IMPORTANT: The use of ioportconfig persist does NOT perform any HA validation,
symmetry checks or if you have exceeded your FE/BE port counts. This can be a
dangerous command so KNOW YOUR CONFIGURATION before using it.
Must do these checks manually
Symmetry is important: If the physical port configuration on the two SPs does not match, the
first SPs configuration is persisted, and the second SP will have missing and/or faulted ports
until you correct the problem on the second SP

Caution: You cannot change a port after it is persisted!

EMC CONFIDENTIALINTERNAL USE ONLY 7
 Adding Ports (continued)

y If you are adding a new BE port and DAE, there is a two step process:
(1) Use the NST or USM to persist the new BE port on the I/O module
The NST/USM will ask you to confirm you have inserted the SFP. For a BE this is the SFP
end of the SFP to HSSDC cable.
It does not matter if the other end of the cable is connected or not connected to a DAE. The
cable just needs to be plugged into the port on the SLIC.
(2) Use the NST or USM to add DAE(s) to the newly persisted BE port

y IMPORTANT INFO REGARDING WHERE AN I/O MODULE CAN BE

INSTALLED: There is much confusion regarding slot usage and where certain I/O
modules can be placed in an array. Documentation has been confusing but its actually
not that complicated.
Other than slot 0, any I/O module can go in any open SLIC slot as long as you do not go over the
BE or FE port limits set by FLARE. (See slide 3 for these limits) This is true for all arrays
regardless of what the docs say.
Example: When adding an I/O module for usage with the Backend Enabler on a CX4-960 it is
perfectly allowable to put a 4 port FC module in any slot, including the annex and configure them
all as BE ports. Annex location installation is completely allowable, it has been done before and
there are no restrictions from engineering.
RULE: As long as you have not gone over the maximum FE or BE port allowances set by FLARE
you can add I/O modules into any available slot.

EMC CONFIDENTIALINTERNAL USE ONLY 8

 Replacing a failed I/O module

1. Shut down power to the SP that includes the failed I/O module.
(i) if the SP with the bad I/O Module is accessible using Navisphere, use the
following command:
naviseccli h <IP address> -user <rname> -password <password> -scope 0
shutdownSP

(ii) if the SP with the bad I/O Module is not accessible using Navisphere, use
the following naviseccli command:
naviseccli h <IP address> -user <rname> -password <password> -scope 0
shutdownpeerSP
NOTE: If you use shutdownpeerSP, make sure to use the IP address of the
storage processor that does not include the failed I/O module (SP B, if the
failed module is in SP A, SP A if the failed module is in SP B).
2. Replace the failed module and reboot the storage processor from its peer SP as
follows. Issue the following CLI command to the SP that is still running:
naviseccli -h <IP address> -user <rname> -password <password> -scope 0
rebootpeersp
EMC CONFIDENTIALINTERNAL USE ONLY 9
 I/O module Conversions

Requires Minimum Taurus (04.29.000.5.001)

y Replace a pair of SLICs, one on each SP, with faster performing SLICs
SLICs must still be symmetrical between SPs when the SLIC Conversion is complete
Port numbers and port locations do not change
Must reboot both SPs, one at a time
Can replace multiple pairs of SLICs during a conversion

y Two Supported Upgrade Paths:

Tomahawk to Glacier (4Gb FC to 8Gb FC)
Harpoon to Poseidon (1Gb iSCSI to 10Gb iSCSI)
If any other upgrade is attempted, Flare will fault the replacement SLICs

y No Navisphere Wizard support must use NaviSecCLI

Always commit FLARE-Operating-Environment bundle before starting
Use naviseccli ioportconfig upgrade
Sets a registry flag and then powers off the targeted SP
Command must be issued to both SPs, one at a time

EMC CONFIDENTIALINTERNAL USE ONLY 10

 I/O module Conversions - Example

Customer wants to upgrade one pair of 4Gb Fibre SLICs to 8Gb Fibre SLICs
1. Starting State (view from the back of a CX4-480 array)
y Two 4Gb Fibre SLICs and One (1Gb GbE/TOE) SLIC per SP

SPB 4Gb FC 4Gb FC 1GbE SPA 4Gb FC 4Gb FC 1GbE

iSCSI iSCSI

FE/BE FE/BE FE/BE FE/BE

FE FE
FE/BE FE/BE FE/BE FE/BE

FE/BE FE/BE FE/BE FE/BE

FE FE
FE/BE FE/BE FE/BE FE/BE

Slot 0 Slot 1 Slot 2 Slot 3 Slot 4 Slot 0 Slot 1 Slot 2 Slot 3 Slot 4

2. Upgrade SPA first: navisecclih <IP address SPA> -user <rname> -

password <password> -scope 0 ioportconfig upgrade
3. Wait for SPA to power off

EMC CONFIDENTIALINTERNAL USE ONLY 11

 I/O module Conversions - Continued

4. Replace one 4Gb Fibre SLIC with an 8Gb Fibre SLIC on SPA side
SPB 4Gb FC 4Gb FC 1GbE SPA 4Gb FC 8Gb FC 1GbE
iSCSI iSCSI

FE/BE FE/BE FE/BE FE/BE

FE FE
FE/BE FE/BE FE/BE FE/BE

FE/BE FE/BE FE/BE FE/BE

FE FE
FE/BE FE/BE FE/BE FE/BE

Slot 0 Slot 1 Slot 2 Slot 3 Slot 4 Slot 0 Slot 1 Slot 2 Slot 3 Slot 4

5. Reboot SPA: naviseccli h <ip address SPB> -user <rname> -

password <password> -scope 0 rebootpeersp
6. Wait for SPA to boot - its SP Fault LED will turn off

EMC CONFIDENTIALINTERNAL USE ONLY 12

 I/O module Conversions - Continued

7. Now upgrade the SPB side: navisecclih <IP address SPA> -user <rname> -
password <password> -scope 0 ioportconfig upgrade
8. Wait for SPB to power off
9. Replace SLIC on SPB side

SPB 4Gb FC 8Gb FC 1GbE SPA 4Gb FC 8Gb FC 1GbE

iSCSI iSCSI

FE/BE FE/BE FE/BE FE/BE

FE FE
FE/BE FE/BE FE/BE FE/BE

FE/BE FE/BE FE/BE FE/BE

FE FE
FE/BE FE/BE FE/BE FE/BE

Slot 0 Slot 1 Slot 2 Slot 3 Slot 4 Slot 0 Slot 1 Slot 2 Slot 3 Slot 4

10. Reboot SPB: naviseccli h <ip address SPA> -user <rname> -password
<password> -scope 0 rebootpeersp
11. Wait for SPB to boot to complete conversion

EMC CONFIDENTIALINTERNAL USE ONLY 13

 I/O module Conversions
Special Cases

y Always commit FLARE-Operating-Environment bundle before starting a SLIC

Conversion, or some SLICs may not be recognized
y If you are going to replace a 4Gb Fibre SLIC in Slot 0 with an 8Gb Fibre SLIC,
make sure the array has a Libra (04.28.000.5.7xx) or newer Utility Partition
installed older Utility Partitions cannot boot with an 8Gb SLIC in Slot 0.
y If you upgrade a 1Gb iSCSI SLIC to a 10Gb iSCSI SLIC, the port settings for the
two virtual ports associated with each physical port on the 1Gb SLIC will be
applied to the first two virtual ports for each physical port on the 10Gb SLIC
10Gb SLICs support additional virtual ports for each physical port
y If any of the front-end ports on a 4Gb Fibre SLIC are set to run at 1Gb, and you
upgrade to an 8Gb Fibre SLIC (Glacier), the 1Gb port(s) will not be usable.
Glacier does not support 1Gb; You must use Navisphere to select a valid port speed
Note: Due to DIMS 233237 (deferred to R30) the port will be displayed as Missing instead of Faulted in
Navi Mgr and CLI. However, the naviseccli port command will show the port as DISABLED, with an invalid
Requested Speed of 1Gb while the Available Speeds are listed as 2Gb, 4Gb, 8Gb, Auto
Assuming that the customers infrastructure does not require the ports to run at 1Gb, the correct
procedure is to change these ports to Auto-negotiate before starting the SLIC Conversion. The
ports will correctly auto-negotiate after the conversion.
y On all CX4 arrays, 10Gb iSCSI port limits are smaller than 1Gb iSCSI port limits,
so you cannot upgrade all of your 1Gb SLICs to 10Gb.

EMC CONFIDENTIALINTERNAL USE ONLY 14

 MirrorView & I/O module ports

y MirrorView port(s) are no longer consistent within a platform

Physical location and Logical port numbers will vary
y MirrorView port(s) are automatically assigned by Flare. It will be:
The highest-numbered FC front-end port when Flare first persists FC front-end port(s)
The highest-numbered iSCSI port when Flare first persists iSCSI port(s)
Assignment is done even if no MirrorView enablers are installed
y If you add more front-end ports later, the MirrorView ports do not move
So you cannot assume that the highest-numbered ports are always the MirrorView ports
y You can find the MirrorView port(s) using Navisphere
Navi UI: Special icons are used
Icon present even if no MirrorView enabler is installed, but is grayed out
NaviSecCLI: port -list -sp all
Will display usage as MirrorView only if a MirrorView enabler is installed
NaviSecCLI: ioportconfig list
Will display usage as Special even if a MirrorView enabler is NOT installed

MirrorView port in use MirrorView port not in use

EMC CONFIDENTIALINTERNAL USE ONLY 15

 Deleting or Changing Ports

y Not Supported
y The only supported method for changing the port configuration is
destructive
y You must wipe out the entire array configuration and start over
Lose all LUNs, RAID Groups, port settings, etc.
Requires re-imaging or a special NaviSecCLI command

EMC CONFIDENTIALINTERNAL USE ONLY 16