IBM Storage Networking

Leading the Evolution of Data Center Networks

SSAP/IBM SAN Sales Specialist
Luo Huazhong
13701102824

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

IBM System Storage SAN Product Family

Building Blocks for the DYNAMIC INFRASTRUCTURE

IBM SAN384B
(2499-192)

IBM SAN04B-R
(2005-R04)

IBM Converged Switch B32

(3758-B32)

Brocade DCX-4S

Brocade 7500E

Brocade 8000

Consolidation and Performance

24-ports 10Gb FCoCEE

8-ports 1, 2, 4, 8Gb FC

SystemX Brocade Converged

Network Adapters (CNAs)
10Gb CEE Dual Port

SystemX Brocade FC HBAs

8G FC Single and Dual Port

IBM SAN06B-R
(2498-R06)
Brocade 7800

IBM SAN80B-4
(2498-B80)

IBM SAN256B
(2109-M48)

16 to 192-ports
1, 2, 4, 8, 10Gbps
FC, FICON

Brocade 48000

Brocade 5300

IBM SAN40B-4
(2498-B40)

for IBM BladeCenter

1, 2, 4 Gbps FC

Brocade DCX

16 to 384-ports
1, 2, 4, 8, 10Gbps
FC, FICON

16 to 384-ports
1, 2, 4, 8, 10 Gbps
FC, FICON

48, 64, 80-ports

4 Gbps SAN
Switch Modules

IBM SAN768B
(2499-384)

Enterprise

1, 2, 4, 8 Gbps
FC, FICON

Brocade 5100

24, 32, 40-ports

IBM SAN24B-4
(2498-B24)

1, 2, 4, 8 Gbps
FC, FICON

Brocade 300

Midrange

8, 16, 24-ports
1, 2, 4, 8 Gbps FC

XIV

N7000

Entry Level

DS4000
DS5000

DS6000
Data Center
Fabric Manager

DS8000

TotalStorage
Productivity Center

N6000
N3000

DS3000

SVC

Scalability, Availability, Business Continuity

2009 IBM Corporation

FCoCEE PROTOCOL,
COMPONENTS,
MANAGEMENT

2009 IBM Corporation

Converged Enhanced Ethernet (CEE)

Protocol Standard Initiatives

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

FCoCEE and CEE Standards Activities

IEEE

T11 ANSI

802.1Qbb Priority Flow

Control (PFC)

FCoE
SPMA FPMA address
discovery

802.1Qaz Enhanced
Transmission Selection (ETS)
DCBX Data Center Bridging
Exchange protocol proposal for
802.1Qaz

IETF

802.1Qau congestion notification

Implementation Agreement

Approved

TRILL - Transparent
Interconnection of Lots of
Links

In Development

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

CEE: Evolving Ethernet with Fibre Channel Characteristics

Transport for FCoE storage traffic
Lossless Ethernet that creates the potential for I/O consolidation at the server edge by transporting
storage and networking traffic over CEE

Feature/Standard

Benefit

Priority Flow Control (PFC)

IEEE 802.1Qbb

Enable storage and networking traffic types to share a common CEE link
without interfering with each other

Enhanced Transmission Selection

(Bandwidth Management)
IEEE 802.1Qaz

Enable bandwidth management by assigning bandwidth segments to

different traffic flows

Congestion Management
IEEE 802.1Qau

End-to-end congestion management for L2 network

Data Center Bridging Exchange Protocol

(DCBX)

Management protocol for CEE

L2 Multipathing: TRILL in IETF

Recover bandwidth, multiple active paths; no spanning tree

FCoE/FC Awareness

Preserves SAN Management Practices

2009 IBM Corporation

Fibre Channel over Ethernet (FCoE)

Protocol and Terminology

2009 IBM Corporation

FCoE and Other Protocols: OSI Layers

FCP

FC

Operating System / Applications

SCSI Layer
FCP
iSCSI
FCP
FCP
SRP/iSER
FCoE
FCIP iFCP
TCP
TCP
TCP
IP
IP
IP
CEE
Ethernet
IB

FCoE is an encapsulation protocol with less overhead than iSCSI but more
overhead than FC
Encapsulation protocols do not replace storage device interfaces such as
FC or SCSI
Brocade is a leader in developing and supporting encapsulation protocols

June 2009

Brocade Data Center Convergence Overview

Dynamic Infrastructure Data Center Networking

Fibre Channel Gets a New Transport Layer

Encapsulation

FC-4
FC-3

Fibre Channel
Services

FC-2
IP
Legacy
Ethernet

Encapsulation
Layer 3
Layer 2
CEE
Layer
1
Transport

Legacy
Ethernet
Today

FCoE

FC-1
FC-0

Fibre Channel
Transport

Fibre Channel
Today

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

Fibre Channel over Converged Enhanced Ethernet

Encapsulation protocol for transporting FC over CEE
(Lossless Ethernet: CEE)

Traditional
Ethernet
Enhanced
Ethernet
Transport

IP Traffic
FC Traffic
IP Traffic

FCS

EOF

FC Payload

CRC

FC
Header

FCoE
Header

Ethernet
Header

Ethernet Frame, Ethertype = FCoE=8906h

Same as a physical FC frame

Control information: version, ordered sets (SOF, EOF)

10

FC frame remains intact: FC does not change

Ethernet needs a larger frame: Larger than 1.5 KB
Ethernet must become lossless to carry storage data with integrity
Brocade is leading the development of FCoE and will drive adoption

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

IBM Converged Switch B32 (3758-B32) Top of Rack

Multi-protocol, multi-layer DCF switch

FCoCEE switching for server-tostorage connect

CEE and L2 switching for server-server
& server-LAN connections
Cut Through, Non-Blocking Switch
Architecture
Compatible with Brocade FC installed
base & works with FCoE CNAs
Built on Brocade Fabric OSTM
v6.1.2_cee
11

1 U form factor, 350W redundant

power supply, standard rack width,
hot swappable Fan FRUs
Back to Front Airflow (port side front)
24 ports 10 GbE FCoE + 8 ports
1/2/4/8G FC
Support for SR, LR SFP+ Optics
Twinax (Copper - 10GigE only)
2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

IBM Converged Switch B32 (3758-B32) FC & FCoE Features

8 FC ports with 1/2/4/8G FC

Universal (E, F, M, Ex and FL)
Fibre Channel Media type
Hot-pluggable, industry-standard Small Form Factor Pluggable (SFP+),
LC connector , SWL, LWL and ELWL

Supports all features of Brocade FC Switch

12

SFP+ to SFP+

Native FCoE Forwarding

FCoE Hosts to FCoE Target
Native FCoE VE Port Support
{ VF, FC-ID } based Zoning (ACLs)
FCoE to FC Translation

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

IBM Converged Switch B32 (3758-B32) CEE Features

24 ports 10 GbE CEE

CEE Media Type: Hot pluggable,10GbE SFP+ Short Reach (SR) and Long
Reach (LR) optical transceivers or SFP+ Cu Twinax active and passive cables
Converged Enhanced Ethernet (CEE) Switching

13

Enables connectivity to servers and corporate network

802.1Qbb Priority-based flow control
802.1Qaz Enhanced transmission selection
DCBX Capability exchange protocol
IETF: TRILL (Transparent Interconnection of Lots of Links) (future)

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

FCoCEE Switch Model

FC
Port

FC
Port

FC
Port

FC
Port

FC Switch
FCoCEE
Enhanced Ethernet Switch
Enet
Port

Enet
Port

Enet
Port

Enet
Port

FC switch is the FCoCEE/FC

forwarding entity
Has multiple FC ports that connect to
the SAN
Has multiple 10 GbE ports that
connect to the messaging network
FCoE entity that handles FCoCEE/FC
encap/decap function
Enhanced Ethernet switch handles
LAN and SAN traffic:
Forwards FCoE traffic to FCoE entity
Forwards Enet traffic to Enet ports

IBM Converged Switch B32

14

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

CEE and FCoE

IBM B32 Switch

24 x 10GbE CEE ports

8 x 8Gbps FC ports
SFP+ and Twinax

Product Highlights

Feature

15

Benefit

Highest FC bandwidth, 2X closest

competing product

Ready to handle applications with

demanding I/O needs

True FC switch connects directly to FC

storage

Integrates into existing FC fabrics

Eliminates need for multiple switches

Lowest power consumption (350 W) less

than half of the leading competitor

Contributes to greener data centers

Lower operating costs

Hardware-based link aggregation: 40Gbps

frame-based trunking

Future ready with 40Gbps

Port mirroring and flow mirroring

Robust high availability

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

IBM System X Brocade Converged Network Adapter (42C1820)

Brocade 1020 CNA
Dual-port 10Gb CEE
PCIe 2.0 Gen 2 x8 Host Bus Interface
Four independent PCI functions
Wire speed
SFP+ Copper and Optical Models
Multifunction Capability
NIC: Advanced Link Layer Offload: TCP/IP
acceleration, RSS, jumbo frames, VLANs, Priority,
PAUSE, link aggregation
Storage: Hardware FCoE Offload

10GbE
10GbE

NIC

FCoE
FC

PCI
Ethernet
Drivers

FC Drivers

Operating System

16

2009 IBM Corporation

FCoCEE and CEE Usage Models &

Server I/O Consolidation

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

FCoCEE/CEE solutions Today

Brocade 10Gb Converged Network
Adapter for IBM System x

CEE IBM Converged Switch B32

Top-of-Rack switch

10GbE Ports

24

FC ports

8 @ 8Gbps

Total 10GbE
Bandwidth

240 Gbps

Total FC
Bandwidth

64 Gbps

IP Traffic

CEE
ports

FC
ports

Brocade
DCFM

FC Traffic
Fibre Channel
SAN

LAN
FC Traffic

18

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

Benefits of FCoCEE/CEE
Reduces number of physical network interfaces in the server
Maintains the same management paradigm and tools for storage
Lower CapEx: Fewer CNAs, cables, and switch ports
Lower OpEx: Reduced power usage and cooling costs

19

FC HBA
FC HBA
NIC

FC Traffic
FC Traffic
Enet Traffic

NIC

Enet Traffic

CNA
CNA

Consolidated
I/O Ports

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

Benefits of FCoCEE/CEE
Top-of-Rack
Configuration

SAN A
SAN B

Reduced
number of server ports
Fewer cables
Reduce number of switch ports

LAN

Reduced cabling

Fewer
adapters

Reduced
power consumption
Increased speed of links
Increased utilization of links

Fewer switches

Consolidating I/O Interfaces Lowers CapEx and OpEx

20

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

Top-of-Rack Switch Deployment Example

2

LAN

LAN
2

2
2

SAN A

SAN A

SAN B

SAN B

50% fewer
adapters,
switches,
and cables
18 Servers

21

Ethernet

FC

Total

Adapters*

18

18

36

Switches

Cables

36

Uplink Ports

18 Servers

CEE

Ethernet

FC

Total

Adapters*

18

18

Switches

36

72

Cables

36

36

Uplink Ports

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

Inhibitors and Accelerators to FCoCEE

INHIBITORS
10GE Cost
Curve

Large FC Security, SLA,Maturity

Politics /
FCoE, CEE FC Market
Investment
QoS
Of Team
Standards Momentum
s
concerns
Technology
Issues

FCoCEE

Server
Virtualization

SRM/SAN
Mgmt S/W
Power
Efficiencies

ACCELERATORS
22

Desire for
Convergence
Simplified,
Unified
Mgmt.

10GE
Performance

Blade Server
Convergence

Better Asset
Utilization

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

Summary
Industrys first end-to-end FCoCEE & CEE solution comprising:
IBM Converged Switch B32 FCoE switch
Brocade 10Gb Converged Network Adapters for IBM System x
IBM DCFM management framework
IBM FCoE/CEE leadership
First 10Gbpse CEE, 8Gbps FC, FCoCEE switch
First single ASIC 10Gbps CNAs
Highest FC bandwidth in an FCoCEE switch (2x competition)
Truly integrated management application covering FC, FCoE and CEE
resources
Best server I/O consolidation end-to-end solution
Collaborating with industry partners to enable pragmatic evolution of data
centers using FCoE & CEE

April 2009

23

IBM B32 Switch:

A Top-of-Rack FCoE Switch

2009 IBM Corporation

Dynamic Infrastructure Data Center Networking

When to Deploy FCoCEE?

In 2009, test and try out FCoCEE:
Get comfortable with FCoCEE and see what it can do
In 2010, start deploying more stable FCoCEE:
We anticipate better stability and maturity to start taking hold sometime in 2010 or later
For price/performance value, customers should:
Consider deploying FCoCEE in greenfield environments
The new server network should consist of open systems
Depending on various components and trends, we expects this to come through in 2010
to 2011

24

2009 IBM Corporation

Thank you!

2009 IBM Corporation