HP Blade System Virtual Connect

Technology Workshop
Glyn Pollington UK Technical Services

2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

Virtual Connect environment 3 Key Components to a VC Infrastructure

Virtual Connect Ethernet Modules

side by side for redundancy

Connect selected server Ethernet ports to specific data center networks. Supports aggregation/tagging of uplinks to data center. LAN-safe for connection to any data center switch environment (Cisco, Nortel, HP ProCurve, etc.)

+ +

Virtual Connect Fibre Channel Modules

Selectively aggregate multiple server FC HBA ports (Qlogic/Emulex) on a FC uplink using NPIV. Connect enclosure to Brocade, Cisco, McData, or Qlogic data center FC switches. Doesnt appear as a switch to the FC fabric.

Virtual Connect Manager (embedded!)

Manage server connections to the data center without impacting the LAN or SAN Move/upgrade/change servers without impacting the LAN or SAN

6 January, 2010

HP 1/10Gb VC-Enet Module

MidPlane

HP Virtual Connect Ethernet Module

16x 1Gb EthernetConnects to one NIC in each HH blade server bay 1x 10GbCross-link between adjacent VC-Enet modules Mgmt Interfaces to Onboard Administrator (Enet & I2C)

2x 10Gb Ethernet (CX4)

Usable as data center links or as stacking links.

Recessed module reset button

8x 1Gb Ethernet (RJ45)

Usable as data center links or as stacking links. Next button (unused in initial release firmware)

Port Number & Status Indicators

Indicates whether a data center link (green), stacking link (amber), or highlighted port (blue).
3 6 January, 2010

HP 1/10Gb-F VC-Enet Module

MidPlane

HP Virtual Connect Ethernet Module

16x 1Gb EthernetConnects to one NIC in each HH blade server bay 1x 10GbCross-link between adjacent VC-Enet modules Mgmt Interfaces to Onboard Administrator (Enet & I2C)

1x 10Gb Ethernet (CX4)* 2x 10Gb Ethernet (XFP module)* Port Number & Status Indicators

2x 1Gb Ethernet (SFP modules)*

Next button (unused in initial release firmware)

4x 1Gb Ethernet (RJ45)

*Usable as data center links or as stacking links. Recessed module reset button

Indicates whether a data center link (green), stacking link (amber), or highlighted port (blue).
4 6 January, 2010

HP Virtual Connect FC Module

MidPlane 16x 4Gb FC
Connects to one HBA port in each HH blade server bay

Mgmt Interfaces to Onboard Administrator

Module LEDs
Health and UID

Port LEDs
Link and Activity

4x 1/2/4 Gb FC (SFP)
Connects to data center Fibre Channel switches.

Port Number & Status Indicators Faceplate

5 6 January, 2010

Indicates whether a port is configured (green) or highlighted (blue). Note: VC-FC requires VC-Ethernet also installed in same VC domain

Virtual Connect Manager

6 January, 2010

Why Virtual Connect?

Overcome

Interconnectivity Issues Simplify Management Separate Network and Server Management Manage Change

6 January, 2010

Too many cables

Problem: Error prone & costly

32 cables 64 cables 128 cables

Datacenter

6 January, 2010

Too many cables

Problem: Error prone & costly

128 cables 512 cables!!!!

Datacenter

6 January, 2010

Too many switches

Cisco director-class -- 425 downlinks

Problem: 26X the amount of work

Cisco 3020 -- 16 downlinks

425 servers 1 switch to manage 425/16 > 26 26 switches to manage for the same number of downlinks!

10

6 January, 2010

Too many switches

Problem: SAN fabrics are limited

Brocade 4Gb SAN Switch -- 16 downlinks Each vendor has upper limit to number of switch domains: McData Cisco FC Brocade 24 40 56

At only 16 downlinks per switch, that makes for a small fabric!

11

6 January, 2010

Traditional server-network connectivity

Each server needs unique MAC and WWN assignments
Blade Servers HBA NIC

1 2 3 4 5
12 6 January, 2010

MAC address World Wide Name

MAC WWN
Ethernet Switch

HBA NIC

HBA NIC

LAN

MAC WWN

HBA NIC

MAC WWN
Fibre Channel Switch

SAN

HBA NIC

MAC WWN

HP BladeSystem and Virtual Connect

Simplifies blade server connectivity
HP BladeSystem Enclosure HBA NIC

1 2 3 4 5
13 6 January, 2010

MAC address pools

LAN Uplinks
Ethernet Switch

HBA NIC

LAN

HBA NIC

Virtual Connect

HBA NIC

SAN
SAN Uplinks
Fibre Channel Switch

HBA NIC

WWN address pools

Reduces cabling Provides constant connections to LANs/SANs Server changes have no impact on networks

Keep the fabric outside the enclosure

Enclosure

Standard Blade LAN or SAN Configuration Today

Server 1 Server 2 Server 3 SWITCH SWITCH

Brocade, Cisco, or McData Switch Brocade, Cisco, or McData Switch

Fabric

Server 16

Enclosure

Blade LAN or SAN Configuration with Virtual Connect

Server 1 Server 2 Server 3

HP Virtual Connect Module HP Virtual Connect Module

Brocade, Cisco, or McData Switch

Fabric

Server 16

Brocade, Cisco, or McData Switch

14

6 January, 2010

traditional servers

everyone must follow the moving server!

Server Administrator

A B C

HBA NIC

MAC A WWN A

HBA NIC

Network Administrator

LAN
Find the new port

Server Administrator

ENET Switch

Configuration of the new port: VLAN, Security, STP, etc

HBA NIC

Storage Administrator

LAN Administrator

Spare

MAC B WWN B

HBA NIC

SAN
LUN investigation
FC Switch

Zoning modification based on new WWN LUN masking modification

15

6 January, 2010

**ANIMATED SLIDE** VIEW IN SLIDE SHOW MODE

SAN Administrator

Virtual Connect solves these

Server Administrator

& makes IT change-ready

Network Administrator

WWN A

HP Virtual Connect Modules

A B C

HBA NIC

MAC A

HBA NIC

LAN
ENET Switch

Server Administrator

Find the new port

Configuration of the new port: VLAN, Security, STP, etc

HBA NIC

Storage Administrator

LAN Administrator

Spare

HBA NIC

SAN
LUN investigation
FC Switch

Zoning modification based on new WWN LUN masking modification

16

6 January, 2010

**ANIMATED SLIDE** VIEW IN SLIDE SHOW MODE

SAN Administrator

Logical Server Mobility

Physical Server A Physical Server B

Virtual and Physical Server Comparison

Physical Server B Physical Server A

OS Image
Guest Server 1
VLAN 1 Fabric A

OS Image
Guest Server 2
VLAN 3 Fabric A

OS Image
Guest Server 3
VLAN 4 Fabric B

OS Image
(Boot from SAN)

Hypervisor

Hypervisor

VMotion OS Image
Host Server A (VMware ESX)
SSN: AABB UUID: 1122AB MACs: A1 & B1 WWNs: C1 & D1

OS Image
Host Server B (VMware ESX)
SSN: CCDD UUID: 3344CD MACs: A2 & B2 WWNs: C2 & D2

Host Server 1
(Windows, ESX, Linux, etc)

VC SSN: SSN: AABB VC1DD VC UUID: UUID: 1122AB 5566EF VC MACs: MACs: A1 A3 && B1 B3 VC WWNs: WWNs: C1 C3 && D1 D3
FC Fabric A VLANs 1 & 2

SSN: CCDD UUID: 3344CD MACs: A2 & B2 WWNs: C2 & D2

FC Fabrics A&B VLANs 1-5

FC Fabrics A&B VLANs 1-5

17

6 January, 2010

**ANIMATED SLIDE** FC Fabrics A& B; VLANs 1- 5 External Network Fabric VIEW IN SLIDE SHOW MODE External Network Fabric

Virtual Connect Interconnect Fabric

Scalable for small and large infrastructures

Virtual Connect Manager Virtual Connect Enterprise Manager

LAN Addresses (MAC) SAN Addresses (WWN) Server Connection Profiles

For single rack datacenters Each domain configured and managed independently Multiple consoles and address pools
18 6 January, 2010

For multiple racks of enclosures /domains Configure and manage groups of enclosures & domains centrally Single console and address pool Import existing Virtual Connect domains

Multi-enclosure VC Domains
Enclosure Stacking (Ethernet)
Any server NIC can access any data center uplink
If 4 enclosures, 64 servers in Virtual Connect Domain Every server in domain has access to any data center LAN connection Every server has fault-tolerant access to every uplink port With this stacking method, every Virtual Connect module still has 8 -1GbE and 1 -10GbE uplink ports available
10 GbE Internal Cross-Connect c-Class Enclosure
58 5/1/2007 HP Con fide ntial -- INTERNA L USE ONLY

Up

to 4 c-Class enclosures Up to16 VC-Enet Modules Up to16 VC-FC Modules

Requires

Virtual Connect Ethernet Modules

10GE ports still available as additional stacking links or data center upli nks

One base enclosure OAs & VC modules must be on same management network (IP subnet) All Ethernet modules interconnected All enclosures must have the same VC-FC configuration (no stacking)
No
19

merge-domain feature (yet)

6 January, 2010

Stacking up to four VC Enclosures

Definition Interconnecting up to 4 VC enclosures to be managed as a single VC domain
Server Blade 1
pNIC pNIC

Server Blade 2
pNIC pNIC

Server Blade 3
pNIC pNIC

Server Blade 1
pNIC pNIC

Server Blade 2
pNIC pNIC

Server Blade 3
pNIC pNIC

Server Blade 1
pNIC pNIC

Server Blade 2
pNIC pNIC

Server Blade 3
pNIC pNIC

Enclosure 1

Enclosure 2

Enclosure 3

vNet_2

vNet_1

Spanning Stacking links links

LAN 1 (Production)

LAN 2 (Backup)
20 6 January, 2010

Traditional Switch and Virtual Connect?

Whats the difference?

Switch Part of LAN or SAN network Managed from network Server directly connected, so any server change impacts network

LAN\SAN Server
Server Server Server Server 21 6 January, 2010

Virtual Connect Part of server system Managed with servers Layer between servers and network, so external network isnt negatively impacted server changes Allows flexibility and control of system resources Ideal for virtualization environments
Server
Virtual Connect Server Server Server

LAN\SAN

LAN

LAN

SAN

SAN

Server

VMware Switch
assigned MACs

Is Virtual Connect different?

NIC VM VM VM VM NIC

NIC NIC

VMWARE VSWITCH

assigned MACs

VIRTUAL CONNECT

Datacenter LAN
22 6 January, 2010

Virtual Connect Fibre Channel

Fibre Channel VC
Methodology Management

23

6 January, 2010

HP Virtual Connect FC Module uses N_Port_ID Virtualization

Defined by the T11 FC standards

FC Device Attach (FC-DA) Specification, Section 4.13 FC Link Services (FC-LS) Specification

NPIV provides a Fibre Channel facility for assigning multiple N_Port_IDs to a single N_Port, thereby allowing multiple distinguishable entities on the same physical port. Where is it used?
When implemented within a FC HBA, allows unique WWNs & IDs for each virtual machine within a server Can be used by a transparent HBA aggregator device (like HP VCFC module) to reduce cables in a vendor-neutral fashion.

24

6 January, 2010

N_Port_ID Virtualization
with an HBA aggregator
Server #1
(WWN A) 2a
FLOGI A ACC A 2b FDISC A 1 FLOGI X

As used by HP VC-FC module

ACC X

Server #2
(WWN B)
3a FLOGI B ACC B

HBA Aggr.
(WWN X)

ACC A 3b

FDISC B ACC B

Full FC Switch with NPIV Support

Server #3
(WWN C)
4a FLOGI C ACC C

4b

FDISC C ACC C 5

1.

Fabric Login using HBA aggregators WWN (WWN X). Sets up the buffer credits for the overall link. Receives overall Port ID. Server HBA does normal fabric login with their WWNs. Server HBA fabric logins translated to FDISC. Traffic for all 4 N-port IDs carried on the same link.

2-4a 2-4b 5.
25

6 January, 2010

N_Port_ID Virtualization
with an HBA aggregator
Server #1
(WWN A) 2a
FLOGI A ACC A 2b FDISC A 1 FLOGI X

As used by HP VC-FC module

ACC X

Server #2
(WWN B)
3a FLOGI B ACC B

HBA Aggr.
(WWN X)

ACC A 3b

FDISC B ACC B

Full FC Switch with NPIV Support

Server #3
(WWN C)
4a FLOGI C ACC C

4b

FDISC C ACC C 5

Uses standard FC HBAs (Qlogic, Emulex) No changes to server OS or device drivers

Once server is logged in, the FC frames pass through unchanged! Doesnt interfere with server/SAN compatibility.

Works with a wide range of Brocade, McData, or Cisco FC Switches (with recent firmware)

26

6 January, 2010

Using NPIV with a Brocade Switch

How does it look from the switch management?

HBA #1
10:00:00:00:c9:38:96:3f

N-port HBA Aggr. N-port (with NPIV) Brocade SilkWorm 200E

HBA #2
10:00:00:00:c9:39:5f:64

N-port

Example Configuration

27

6 January, 2010

Using NPIV with a Brocade Switch

How does it look from the switch management?

Uses same fabric port

Identifies port as NPIV Switch tools show the port with the same WWN that the server tools see.

28

6 January, 2010

Why did we do it that way?

VC-FC Uplink Mapping

29

6 January, 2010

VC-FC uplink usage 1 active uplink

Uplink 1 Uplink 2 Uplink 3 Uplink 4

30

6 January, 2010