Configuring I/O modules

CX4 Family

Dan Nowlan
Mike Hamel
10/26/2010

EMC CONFIDENTIALINTERNAL USE ONLY

Rev 4

I/O modules

Supported I/O modules on CX4 Arrays

SLIC Code Name

SLIC Label

Ports

BE/FE?

Minimum Flare Rev

Tomahawk

4Gb Fibre

BE/FE

04.28.000.5.003

Harpoon

1Gb GbE/TOE

FE only

04.28.000.5.003

Glacier

8Gb Fibre

BE/FE

04.28.000.5.704

Poseidon (optical)

10Gb GbE

FE only

04.29.000.5.001

Heatwave

10Gb FCoE

FE only

04.30.000.5.507

EMC CONFIDENTIALINTERNAL USE ONLY

Port Limits enforced by Flare (per SP)

CX4-480 CX4-960

CX4-960

CX4-120

CX4-240

Max FC BE

Max FC (FE + BE)

12

16

20

Max iSCSI (1G + 10G)

Max 10G iSCSI

Max 10Gb FCoE

w/960 enabler

y FLARE maximum port limitations/allowances

y Purple indicates New for Taurus (R29)
y Blue indicates New for Jupiter (R30+)
EMC CONFIDENTIALINTERNAL USE ONLY

SFP Rules
y Removing an SFP will NOT cause the SP to reboot
y To replace an SFP, simply pull out the bad SFP and insert a good one.
No need to reboot the SP.
y SFPs can be inserted into a port at any time
y To persist ports on a 4Gb or 8Gb Fibre SLICs, you must have SFPs
inserted.
y Both ports will be persisted on a 10GbE SLIC even if there are no SFPs
inserted.

EMC CONFIDENTIALINTERNAL USE ONLY

SFP rules for Fibre Front-End ports

y CX4 arrays support two different classes of Fibre Channel SFPs for frontend ports
One type supports 1Gb, 2Gb and 4Gb speeds
The other type supports 2Gb, 4Gb and 8Gb speeds

y A port can only run at speed X if both the SFP and the SLIC support that
speed. Mixing-and-matching qualified SFPs and SLICs is allowed, but
restricts the speed that a port can run at.
y Navisphere will list the available speeds for a particular port see
values below in blue.

4Gb Fibre SLIC

Tomahawk
8Gb Fibre SLIC
Glacier
EMC CONFIDENTIALINTERNAL USE ONLY

If SFP supports
1Gb/2Gb/4Gb

If SFP supports
2Gb/4Gb/8Gb

2Gb, 4Gb

2Gb, 4Gb, 8Gb

1Gb, 2Gb, 4Gb

2Gb, 4Gb

Handling I/O modules

y New I/O modules can be inserted into an open slot in the array at any
time
y CAUTION: Removing a powered up I/O module will cause the SP to
immediately reboot
This reboot is not graceful the reboot occurs immediately, regardless of what the
software on the SP is doing
Exception: The SP will NOT reboot if the I/O module is not powered up*

y *When an I/O module is first inserted, it will not power up until the SP is
rebooted

EMC CONFIDENTIALINTERNAL USE ONLY

Adding Ports
y New backend or frontend ports can be persisted at any time, but
requires rebooting both SPs one at a time (coordinated reboots)
Must adhere to the arrays port limits (See Slide 3 - these limits are platform-specific)
Both SPs must have the same port configuration

y NST is the preferred method

Runs HA validation checks before rebooting SPs to avoid DU/DL
Checks for symmetry between SPs

y NaviSecCLI procedure is also supported

NaviSecCli ioportconfig persist will reboot both SPs, one at a time in a coordinated
fashion. Allow the command to perform both reboots.
IMPORTANT: The use of ioportconfig persist does NOT perform any HA validation,
symmetry checks or if you have exceeded your FE/BE port counts. This can be a
dangerous command so KNOW YOUR CONFIGURATION before using it.
Must do these checks manually
Symmetry is important: If the physical port configuration on the two SPs does not match, the
first SPs configuration is persisted, and the second SP will have missing and/or faulted ports
until you correct the problem on the second SP

Caution: You cannot change a port after it is persisted!

EMC CONFIDENTIALINTERNAL USE ONLY

Adding Ports (continued)

y If you are adding a new BE port and DAE, there is a two step process:
(1) Use the NST or USM to persist the new BE port on the I/O module
The NST/USM will ask you to confirm you have inserted the SFP. For a BE this is the SFP
end of the SFP to HSSDC cable.
It does not matter if the other end of the cable is connected or not connected to a DAE. The
cable just needs to be plugged into the port on the SLIC.

(2) Use the NST or USM to add DAE(s) to the newly persisted BE port

y IMPORTANT INFO REGARDING WHERE AN I/O MODULE CAN BE

INSTALLED: There is much confusion regarding slot usage and where certain I/O
modules can be placed in an array. Documentation has been confusing but its actually
not that complicated.
Other than slot 0, any I/O module can go in any open SLIC slot as long as you do not go over the
BE or FE port limits set by FLARE. (See slide 3 for these limits) This is true for all arrays
regardless of what the docs say.
Example: When adding an I/O module for usage with the Backend Enabler on a CX4-960 it is
perfectly allowable to put a 4 port FC module in any slot, including the annex and configure them
all as BE ports. Annex location installation is completely allowable, it has been done before and
there are no restrictions from engineering.
RULE: As long as you have not gone over the maximum FE or BE port allowances set by FLARE
you can add I/O modules into any available slot.
EMC CONFIDENTIALINTERNAL USE ONLY

Replacing a failed I/O module

1. Shut down power to the SP that includes the failed I/O module.
(i) if the SP with the bad I/O Module is accessible using Navisphere, use the
following command:
naviseccli h <IP address> -user <rname> -password <password> -scope 0
shutdownSP
(ii) if the SP with the bad I/O Module is not accessible using Navisphere, use
the following naviseccli command:
naviseccli h <IP address> -user <rname> -password <password> -scope 0
shutdownpeerSP
NOTE: If you use shutdownpeerSP, make sure to use the IP address of the
storage processor that does not include the failed I/O module (SP B, if the
failed module is in SP A, SP A if the failed module is in SP B).
2. Replace the failed module and reboot the storage processor from its peer SP as
follows. Issue the following CLI command to the SP that is still running:
naviseccli -h <IP address> -user <rname> -password <password> -scope 0
rebootpeersp
EMC CONFIDENTIALINTERNAL USE ONLY

I/O module Conversions

Requires Minimum Taurus (04.29.000.5.001)
y Replace a pair of SLICs, one on each SP, with faster performing SLICs

SLICs must still be symmetrical between SPs when the SLIC Conversion is complete
Port numbers and port locations do not change
Must reboot both SPs, one at a time
Can replace multiple pairs of SLICs during a conversion

y Two Supported Upgrade Paths:

Tomahawk to Glacier (4Gb FC to 8Gb FC)
Harpoon to Poseidon (1Gb iSCSI to 10Gb iSCSI)
If any other upgrade is attempted, Flare will fault the replacement SLICs

y No Navisphere Wizard support must use NaviSecCLI

Always commit FLARE-Operating-Environment bundle before starting
Use naviseccli ioportconfig upgrade
Sets a registry flag and then powers off the targeted SP

Command must be issued to both SPs, one at a time

EMC CONFIDENTIALINTERNAL USE ONLY

10

I/O module Conversions - Example

Customer wants to upgrade one pair of 4Gb Fibre SLICs to 8Gb Fibre SLICs
1. Starting State (view from the back of a CX4-480 array)
y

Two 4Gb Fibre SLICs and One (1Gb GbE/TOE) SLIC per SP
SPB

4Gb FC

4Gb FC

FE/BE

FE/BE

1GbE
iSCSI

SPA

4Gb FC

4Gb FC

FE/BE

FE/BE

FE

FE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE
FE/BE

FE/BE

Slot 0

Slot 1

Slot 2

1GbE
iSCSI

FE

Slot 3

Slot 4

FE/BE

FE/BE

Slot 0

Slot 1

Slot 2

Slot 3

Slot 4

2. Upgrade SPA first: navisecclih <IP address SPA> -user <rname> password <password> -scope 0 ioportconfig upgrade
3. Wait for SPA to power off

EMC CONFIDENTIALINTERNAL USE ONLY

11

I/O module Conversions - Continued

4. Replace one 4Gb Fibre SLIC with an 8Gb Fibre SLIC on SPA side
SPB

4Gb FC

4Gb FC

FE/BE

FE/BE

1GbE
iSCSI

SPA

4Gb FC

8Gb FC

FE/BE

FE/BE

FE

FE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE
FE/BE

FE/BE

Slot 0

Slot 1

Slot 2

1GbE
iSCSI

FE

Slot 3

Slot 4

FE/BE

FE/BE

Slot 0

Slot 1

Slot 2

Slot 3

Slot 4

5. Reboot SPA: naviseccli h <ip address SPB> -user <rname> password <password> -scope 0 rebootpeersp
6. Wait for SPA to boot - its SP Fault LED will turn off

EMC CONFIDENTIALINTERNAL USE ONLY

12

I/O module Conversions - Continued

7. Now upgrade the SPB side: navisecclih <IP address SPA> -user <rname> password <password> -scope 0 ioportconfig upgrade
8. Wait for SPB to power off
9. Replace SLIC on SPB side
SPB

4Gb FC

8Gb FC

FE/BE

FE/BE

1GbE
iSCSI

SPA

4Gb FC

8Gb FC

FE/BE

FE/BE

FE

FE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE/BE

FE
FE/BE

FE/BE

Slot 0

Slot 1

Slot 2

1GbE
iSCSI

FE

Slot 3

Slot 4

FE/BE

FE/BE

Slot 0

Slot 1

Slot 2

Slot 3

Slot 4

10. Reboot SPB: naviseccli h <ip address SPA> -user <rname> -password
<password> -scope 0 rebootpeersp
11. Wait for SPB to boot to complete conversion

EMC CONFIDENTIALINTERNAL USE ONLY

13

I/O module Conversions

Special Cases
y Always commit FLARE-Operating-Environment bundle before starting a SLIC
Conversion, or some SLICs may not be recognized
y If you are going to replace a 4Gb Fibre SLIC in Slot 0 with an 8Gb Fibre SLIC,
make sure the array has a Libra (04.28.000.5.7xx) or newer Utility Partition
installed older Utility Partitions cannot boot with an 8Gb SLIC in Slot 0.
y If you upgrade a 1Gb iSCSI SLIC to a 10Gb iSCSI SLIC, the port settings for the
two virtual ports associated with each physical port on the 1Gb SLIC will be
applied to the first two virtual ports for each physical port on the 10Gb SLIC
10Gb SLICs support additional virtual ports for each physical port

y If any of the front-end ports on a 4Gb Fibre SLIC are set to run at 1Gb, and you
upgrade to an 8Gb Fibre SLIC (Glacier), the 1Gb port(s) will not be usable.
Glacier does not support 1Gb; You must use Navisphere to select a valid port speed
Note: Due to DIMS 233237 (deferred to R30) the port will be displayed as Missing instead of Faulted in
Navi Mgr and CLI. However, the naviseccli port command will show the port as DISABLED, with an invalid
Requested Speed of 1Gb while the Available Speeds are listed as 2Gb, 4Gb, 8Gb, Auto

Assuming that the customers infrastructure does not require the ports to run at 1Gb, the correct
procedure is to change these ports to Auto-negotiate before starting the SLIC Conversion. The
ports will correctly auto-negotiate after the conversion.

y On all CX4 arrays, 10Gb iSCSI port limits are smaller than 1Gb iSCSI port limits,
so you cannot upgrade all of your 1Gb SLICs to 10Gb.
EMC CONFIDENTIALINTERNAL USE ONLY

14

MirrorView & I/O module ports

y MirrorView port(s) are no longer consistent within a platform
Physical location and Logical port numbers will vary

y MirrorView port(s) are automatically assigned by Flare. It will be:

The highest-numbered FC front-end port when Flare first persists FC front-end port(s)
The highest-numbered iSCSI port when Flare first persists iSCSI port(s)
Assignment is done even if no MirrorView enablers are installed

y If you add more front-end ports later, the MirrorView ports do not move
So you cannot assume that the highest-numbered ports are always the MirrorView ports

y You can find the MirrorView port(s) using Navisphere

Navi UI: Special icons are used
Icon present even if no MirrorView enabler is installed, but is grayed out

NaviSecCLI: port -list -sp all

Will display usage as MirrorView only if a MirrorView enabler is installed

NaviSecCLI: ioportconfig list

Will display usage as Special even if a MirrorView enabler is NOT installed
MirrorView port in use

EMC CONFIDENTIALINTERNAL USE ONLY

MirrorView port not in use

15

Deleting or Changing Ports

y Not Supported
y The only supported method for changing the port configuration is
destructive
y You must wipe out the entire array configuration and start over
Lose all LUNs, RAID Groups, port settings, etc.
Requires re-imaging or a special NaviSecCLI command

EMC CONFIDENTIALINTERNAL USE ONLY

16