Icb Ra Cloud Computing Unified Storage NetApp

NeIApp* Unied NeIwcrking and SIcrage.
10 CbE FCcE and iSCSl

AUDIENCE AND PUPP0SE
`| ,-,e. | c. '` c.-|.-!|c -c e.|ce ,.c|ce. .c -.e |!e.e!ec |
'J||c| c|cJc c-!- ce!e.. S!c.-e c|J!|c -.e c.|!|c-| e|ee! c !e ce.-||
c|cJc -.c|!ec!J.e. O|ec e!.c.'| '-ec c d |-'|! |!e.e! ,.c|ce -
cc!-eec!|e c|J!|c c. c|cJc !c.-e -.c|!ec!J.e '-ec c ccc|, Jec
|!e.e! !ecc|c,.
`e ,J.,ce c !| ,-,e. | !c |e cc,|e!e -.c|!ec!J.-| ce!-|| .|! !e,-
',-!e, |!.Jc!|c !c e! J, -c e-|J-!e !e !.c c! ccc '|cc'-
|ee| !c.-e ,.c!ccc|. '.e c-e| ce. |!e.e! |c|) -c '!e.e! -||
cc,J!e. ,!e |!e.-ce |SS'). .J ce. d '| .c ec-!c-ec. we ,.c|ce
cJ. c'e.-!|c -'cJ! !e Jc!|c-||!, -c ,e.c.-ce c'e.ec cJ.| cJ.
!e!.
S|ce !e.e -.e !.-cec -c !e ,.c!ccc| cc|ce. .e .eccec !-!
!e .e-ce. !-'e !e !|e !c e-|J-!e !e J|!|,|e c,!|c !-! ex|! c. J|ec
e!.c.'| | !e|. c. |-' .J| !e|. c. .c.'|c-c !c 'e! ce!e.|e !e
c! -,,.c,.|-!e !ecc|c, cc|ce.
InIe|' C|cud u||ders u|de
InIe|' Xecn' Prccesscrbased Servers
NeIA* Un|ed NeIwcrk|ng and SIcrage: 10 bE FCcE and |SCSI
InIe|' Xecn' Prccesscr 5500 Ser|es
InIe|' Xecn' Prccesscr 5600 Ser|es
lnIel' Clcud Builders Cuide. Clcud 0esign
and 0eplcymenI cn lnIel' PlaI!crms
Tab|e 0f CcnIenIs
ExecuI|ve Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
InIrcducI|cn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
0ur TesI Plan and ResulIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
The 10 |gab|I EIherneI Trans|I|cn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Simpli!icaIicn c! Ihe NeIwcrk wiIh 10CbE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
10CbE. The Fabric c! Clcud CcmpuIing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
The Prcm|se cf EIherneI SIcrage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
EIherneI EnhancemenIs fcr SIcrage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
0aIa CenIer Bridging !cr Lcssless EIherneI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Fibre Channel cver EIherneI. Enabling EndIcEnd Uni!ied l/0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
lnIrcducIicn c! Ihe 0pen FCcE ArchiIecIure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
TesI Ccnf|guraI|cn 0verv|ew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
lcmeIer Ccn!iguraIicn 0verview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
TesI Ccnf|guraI|cn DeIa||s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
NeIApp SIcrage SysIem Ccn!iguraIicn 0eIails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Ciscc SwiIch Ccn!iguraIicn 0eIails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Z0
iSCSl Ccn!iguraIicn 0eIails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Z4
FCcE Ccn!iguraIicn 0eIails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
lcmeIer TesI Tccl Ccn!iguraIicn 0eIails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59
|SCSI FuncI|cna||Iy & Perfcrmance Pesu|Is . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
FCcE FuncI|cna||Iy & Perfcrmance Pesu|Is . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Ana|ys|s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
NexI SIes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Z
Ccnc|us|cn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Z
Add|I|cna| InfcrmaI|cn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Z
Aend|x A: NeIA Ccmmand L|ne SeIu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Z
Aend|x : NeIA |umbc Frame Ccnf|rmaI|cn Ihrcugh UI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Z
InIe|' C|cud u||ders u|de fcr NeIA* Un|ed NeIwcrk|ng and SIcrage: 10 bE FCcE and |SCSI
Un!crIunaIely, Ihe success achieved by
many crganizaIicns as Ihey aIIain Ihese
beneIs has been limiIed by pasI pracIices
used Ic neIwcrk virIualized servers. As
VN densiIy increases, a physical server's
neIwcrking needs alsc increase, which
adds bcIh ccsI and ccmplexiIy. A Iypical
virIualized server ccnIains eighI Ic Ien
CigabiI EIherneI (CbE) LAN pcrIs and Iwc
dedicaIed SAN pcrIs.
As server virIualizaIicn ccnIinues Ic
spread, 10 CbE unied neIwcrking
simplies server ccnnecIiviIy. The
ccnsclidaIicn c! mulIiple CbE ccnnecIicns
Ira!c cnIc a single 10 CbE adapIer
signicanIly reduces cable and
in!rasIrucIure ccmplexiIy, and cverall
TC0. EnhancemenIs Ic Ihe EIherneI
sIandard alsc enable 10 CbE suppcrI !cr
bcIh LAN and SAN Ira!c, which allcws lT
Ic realize !urIher beneIs as iI ccnverges
daIa and sIcrage in!rasIrucIures. Thanks
Ic EIherneI's ubiquiIy, ccsIe!!ecIiveness,
dexibiliIy, and ease c! use, iI has emerged
as Ihe daIa cenIer unied !abric.
ExecuI|ve Summary
ln recenI years, grcwIh in EIherneIbased
sIcrage has surpassed IhaI c! sIcrage
specic !abrics, driven in large parI by Ihe
increase in server virIualizaIicn. Unied
neIwcrking cver 10 CigabiI EIherneI (10
CbE) in Ihe daIa cenIer c!!ers ccmpelling
beneIs !cr Ihe evclving virIualized daIa
cenIer, including simplied in!rasIrucIure,
lcwer equipmenI and pcwer ccsIs, and
greaIer dexibiliIy. Easy access Ic Ihe
sIcrage area neIwcrk (SAN) will be criIical
as virIualizaIicn deplcymenIs ccnIinue
Ic grcw and new, cndemand daIa cenIer
mcdels emerge.
lnIel and NeIApp are Iwc leaders in Ihe
IransiIicn Ic 10 CbE unied neIwcrking.
This paper explcres Ihe beneIs c!
unied neIwcrking, apprcaches Ic enable
iI, and meIhcds lnIel and NeIApp use Ic
bring impcrIanI, ccnsclidaIicndriven
Iechnclcgies Ic enIerprise daIa cenIer
cusIcmers.
InIrcducI|cn
0ur TesI P|an and Pesu|Is
lnIel and NeIApp wcrked IcgeIher Ic
implemenI bcIh FCcE and iSCSl cver
an endIcend 10 CbE neIwcrk in an
lnIel' lab. ln all cases, we used naIive 0S
iniIiaIcrs wiIh a sIandard lnIel' 10 Cb
EIherneI NlC cn Ihe server, raIher Ihan
c!dcading Ihe enIire prcIcccl sIack Ic
a prcprieIary hcsI bus adapIer (HBA)
cr ccnverged neIwcrk adapIer (CNA) as
was dcne in Ihe pasI. we were able Ic
achieve near 10 Cbps line raIe IhrcughpuI
per!crmance in mcsI cases. we cbIained
similar resulIs !cr FCcE and iSCSl in mcsI
cases. we presenI Ihe deIailed Iechnclcgy
cverview, IesI plan, sIepbysIep
ccnguraIicn, resulIs, and analysis belcw.
we hcpe cur wcrk will bring new insighIs
Ic ycur crganizaIicn cn Ihe use c! unied
neIwcrking cver 10 CbE and Ihe use c!
naIive 0S iniIiaIcrs !cr iSCSl & FCcE.
The 10 |gab|I EIherneI Trans|I|cn
S|m||f|caI|cn cf Ihe NeIwcrk w|Ih 10
bE
As lT deparImenIs lcck Ic reduce ccsIs
and imprcve server e!ciency, Ihey
Iurn increasingly Ic server virIualizaIicn
and ccnsclidaIicn. The beneIs c!
virIualizaIicn are widely undersIccd.
less server hardware Ic purchase, lcwer
pcwer and cccling needs, and cenIralized
managemenI. Tcday's servers are based
cn pcwer!ul new prccesscrs, including
Ihe lnIel' Xecn' prccesscr 5600 and
7500 series, IhaI suppcrI mcre virIual
machines (VNs) per physical hcsI Ihan
ever be!cre, which helps lT realize greaIer
ccnsclidaIicn raIics.
The laIesI generaIicn c! lnIel' Xecn'
prccesscrs enables lT Ic ccnsclidaIe
servers aI a 15.1 raIic, which delivers
pcwer savings c! up Ic 90 percenI and
a !ivemcnIh reIurn cn invesImenI.
New !curscckeI prccesscrs deliver Z0
Iimes Ihe per!crmance c! previcus gen
eraIicn prccesscrs.
Nearly 50 percenI c! Ihe !curscckeI
servers shipped Icday are used !cr
virIualizaIicn.
1
3
F|gure 1. Hu|I||e bE CcnnecI|cns |n a Ty|ca| V|rIua||zed Server
10 bE: The Fabr|c cf C|cud
CcmuI|ng
The grcwIh in server virIualizaIicn has
helped daIa cenIer neIwcrks evclve
!rcm discreIe, silced in!rasIrucIures
Ic mcre dexible !abrics wiIh Ihe
scalabiliIy and agiliIy necessary Ic
address Ihe needs c! new usage
mcdels and Ic prcvide an excellenI
!cundaIicn !cr enIerprise clcud
ccmpuIing.
0ver Z.5 billicn users will ccnnecI Ic
Ihe lnIerneI in Ihe nexI ve years,
Z

wiIh cver 10 billicn devices.
3
This
usage will require eighI Iimes Ihe
currenI amcunI c! sIcrage capaciIy,
16 Iimes Ihe currenI neIwcrk capaciIy,
and cver Z0 Iimes Ihe currenI ccmpuIe
capaciIy.
4
A new in!rasIrucIure musI
emerge Ic pcwer Ihis grcwIh and
Ic enable Ihe mcsI e!cienI use c!
rescurces, Ihis is clcud ccmpuIing. The
clcud is an evcluIicn c! ccmpuIing IhaI
delivers services cver Ihe lnIerneI Ic
ccnsumers and enIerprises. Services
scale-as needed and cnly when
needed-wiIhcuI user inIervenIicn.
we need highly scalable and e!cienI
clcud archiIecIure Ic prcvide bcIh Ihe
Iechnical aIIribuIes, and Ihe exIreme
rescurce uIilizaIicn and e!ciency
clcud ccmpuIing prcmises.
wiIh iIs reduced hardware
requiremenIs, !ewer pcinIs c!
managemenI, and brcad eccsysIem
suppcrI, 10 CbE delivers Ihe dexible,
simplied neIwcrk in!rasIrucIure
needed Ic suppcrI clcud ccmpuIing.
The !cllcwing key characIerisIics
make 10 CbE Ihe ideal !abric !cr clcud
in!rasIrucIures.
Ub|qu|Iy: EIherneI ccnnecIiviIy ships
sIandard cn nearly every server
Icday, and EIherneI in!rasIrucIures
are a universal daIa cenIer ccmpcnenI.
when 10 CbE LAN cn mcIherbcard (L0N)
ccnnecIicns are inIegraIed in Ihe nexI
generaIicn c! servers, unied LAN and
SAN ccnnecIiviIy will be available by
de!aulI.
Advanced V|rIua||zaI|cn SucrI:
Advanced server virIualizaIicn enables
dynamic rescurce allccaIicn and every
clcud ccmpuIing in!rasIrucIure requires
iI. Technclcgies !rcm ccmpanies such as
lnIel, VNware, and Nicrcsc!I deliver line
raIe 10 CbE IhrcughpuI and suppcrI !cr
plaI!crm virIualizaIicn enhancemenIs.
Un|ed NeIwcrk|ng: A 10 CbE
unied !abric simplies Ihe neIwcrk
in!rasIrucIure because iI ccnsclidaIes
LAN and SAN Ira!c. RecenI EIherneI
enhancemenIs ensure qualiIy c! service
(0cS) !cr criIical Ira!c.
lnIel and NeIApp are ccmpanies leading
Ihe shi!I Ic 10 CbEbased unied
neIwcrking in Ihe daIa cenIer. The laIesI
lnIel' 10 Cb EIherneI ccnIrcllers and
server adapIers include virIualizaIicn
cpIimizaIicns and advanced unied
neIwcrking !eaIures. These !eaIures
have cpIimizaIicns !cr lcssless EIherneI,
inIelligenI, hardwarebased acceleraIicn
!cr (FCcE) and iSCSl, and suppcrI !cr 0pen
FCcE, which is discussed laIer in Ihis paper.
NeIApp prcvides prcducIs and services
Ic plan, design, implemenI, and mainIain
EIherneIbased sIcrage scluIicns.
These scluIicns simpli!y neIwcrk
ccnnecIiviIy !cr Icday's virIualized
servers and lay Ihe !cundaIicn !cr Ihe
nexI generaIicn daIa cenIer.
The Prcm|se cf EIherneI SIcrage
New usage mcdels and Ihe explcsive
grcwIh c! daIa in Iheir crganizaIicns
have !crced lT adminisIraIcrs Ic deal
wiIh ccmplicaIed Iechnical and business
challenges. Tcday, mcsI lT deparImenIs
deplcy separaIe LAN and sIcrage
neIwcrks, wiIh sIcrage c!Ien divided
beIween neIwcrkaIIached sIcrage
(NAS) !cr lebased applicaIicns, and SAN
(bre channel (FC) and iSCSl) !cr blcck
based applicaIicns. The gcal c! unied
neIwcrking is Ic allcw a single!abric
in!rasIrucIure, based cn 10 CbE, Ic carry
all c! Ihese disparaIe Ira!c Iypes.
4
F|gure Z. S|m||ed Server CcnnecI|v|Iy w|Ih 10 bE
Nany enIerprises have exIensive FC
insIallaIicns, and Ihe availabiliIy c!
naIive 0S FCcE iniIiaIcrs makes Ihe FC
SAN easily accessible !cr any server
wiIh a 10 CbE pcrI. Because iI allcws
FC Ic use 0CBcapable EIherneI, FCcE
eliminaIes Ihe need !cr dedicaIed
FC HBAs, which reduces cabling and
swiIchpcrI requiremenIs. AI Ihe same
Iime, iI ccexisIs wiIh esIablished FC
in!rasIrucIures. The resulI is a simplied
daIa cenIer in!rasIrucIure, lcwer
equipmenI and pcwer ccsIs, and universal
SAN ccnnecIiviIy acrcss Ihe daIa cenIer
cver Ihe IrusIed EIherneI !abric.
InIrcducI|cn cf Ihe 0en FCcE
Arch|IecIure
The 0pen FCcE apprcach ccnsisIs c!
sIandard 10 CbE adapIers and naIive
cperaIing sysIembased FCcE iniIiaIcrs,
which IcgeIher prcvide rcbusI, scalable,
and highper!crmance server ccnnecIiviIy
wiIhcuI expensive, prcprieIary hardware.
As shcwn in Figure 4, 0pen FCcE
implemenIs Ihe ccmpleIe FC prcIcccl in
Ihe cperaIing sysIem kernel. lI prcvides
libraries !cr di!!erenI sysIemlevel
implemenIaIicns, which allcw vendcrs Ic
EIherneI has served as a unied daIa
cenIer !abric !cr years because iI suppcrIs
LAN, NAS, neIwcrk le sysIem (NFS),
ccmmcn lnIerneI le sysIem (ClFS), and
iSCSl SAN Ira!c. wiIh recenI EIherneI
enhancemenIs and Ihe raIicaIicn c! Ihe
FCcE specicaIicn, daIa cenIer bridging
(0CB)based EIherneI adapIers can ncw
!aciliIaIe Ihe ccnnecIicn c! servers Ic FC
SANs. The exIensicn c! EIherneI's reach
and adminisIraIcrs' wide !amiliariIy wiIh
FC SAN Ira!c will help acceleraIe Ihe
mcve Ic 10 CbEbased l/0 ccnsclidaIicn in
virIualized daIa cenIers, reduce ccsIs, and
imprcve simplicaIicn and agiliIy.
Civen iIs dexibiliIy and lcng hisIcry, iI
is nc surprise IhaI EIherneI sIcrage
is Ihe !asIesI grcwing segmenI c! Ihe
sIcrage sysIem markeI. The indusIry
research rm l0C*
5
esIimaIes IhaI Ihe
wcrldwide EIherneIbased sIcrage sysIem
(NAS and iSCSl SAN) markeI grew aI a
ccmpcunded annual grcwIh raIe (CACR)
c! apprcximaIely Z3 percenI beIween
Z005 and Z009. iSCSl sIcrage shipmenIs,
in parIicular, experienced Ihe highesI
grcwIh raIes (70 percenI) !rcm Z005
Z009, driven by brcad iSCSl adcpIicn in
windcws*, virIual server, and blade server
envircnmenIs.
lndusIry analysIs prcjecI ccnIinued gains
in Ihe EIherneI sIcrage markeI share due
Ic increasing deplcymenI c! EIherneI
cnly daIa cenIers (which use a unied
10 CbE in!rasIrucIure !cr all daIa and
sIcrage Ira!c), Ihe emergence c! clcud
ccmpuIing, and Ihe enIrance c! FCcE
scluIicns inIc Ihe mainsIream.
EIherneI EnhancemenIs fcr SIcrage
DaIa CenIer r|dg|ng fcr Lcss|ess
EIherneI
Tc sIrengIhen 10 CbE as a unied daIa
cenIer !abric, Ihe lnsIiIuIe c! ElecIrical
and ElecIrcnics Engineers (lEEE) has
sIandardized EIherneI enhancemenIs Ic
suppcrI sIcrage Ira!c, including FCcE
and iSCSl. These exIensicns, kncwn
ccllecIively as 0aIa CenIer Bridging,
enable beIIer Ira!c pricriIizaIicn cver
a single inIer!ace, as well as advanced
means Ic shape Ira!c cn Ihe neIwcrk
Ic decrease ccngesIicn. ln shcrI, 0CB
delivers a lcssless EIherneI !abric !cr
sIcrage Ira!c.
F|bre Channe| cver EIherneI: Enab||ng
EndIcEnd Un|f|ed Il0
FCcE is a lcgical exIensicn c! EIherneI
IhaI uses Ihe bre channel neIwcrk,
service, and prcIcccl layers Ic carry daIa
packeIs cver Ihe EIherneI's physical and
daIa link layers. Fibre channel's unique
neIwcrkcenIric managemenI mcdel
has prcven adminisIraIive capaciIies Ic
scale Ic Ihcusands c! end ncdes in a daIa
cenIer. The use c! bre channel's upper
layers smccIhes Ihe IransiIicn Ic FCcE
because exisIing SANbased applicaIicns
dc ncI need Ic change Ic beneI !rcm
Ihe per!crmance and ccsI beneIs c!
FCcE. The prcvisicning respcnsibiliIies,
ncw spliI beIween Ihe server and sIcrage
adminisIraIcrs, can be mainIained in Ihe
IransiIicn !rcm bre channel SANs Ic
EIherneIbased FCcEpcwered SANs.
5
F|gure 3. AcIua| Un|I Sh|menIs fcr EIherneI and F|bre Channe| SIcrage (IDC, Z009)
c!dcad iSCSl prccessing Ic a separaIe
prccesscr cn Ihe adapIer, raIher Ihan
allcwing Ihe hcsI prccesscr and cperaIing
sysIem Ic handle Ihese Iasks.
Tcday, all majcr server cperaIing sysIems
include naIive iSCSl suppcrI, which
delivers Ihe same beneIs as Ihcse
deIailed abcve. NaIive iSCSl iniIiaIcrs have
ccnIinued Ic maIure and ncw suppcrI
advanced adapIer !eaIures and plaI!crm
advancemenIs IhaI help deliver excellenI
per!crmance. ln !acI, lnIel recenIly
demcnsIraIed a sIandard lnIel' EIherneI
10 CigabiI Server AdapIer driving 1.Z5
millicn inpuI/cuIpuI cperaIicns per seccnd
(l0PS) using Ihe naIive iSCSl iniIiaIcr in
windcws Server Z00B RZ.
lT deparImenIs which sIandardize cn
lnIel EIherneI Server AdapIers !cr
iSCSl ccnnecIiviIy are able Ic use a
single iniIiaIcr, TCP/lP sIack, and seI c!
managemenI Iccls and lT pclicies. Easier
server prcvisicning, lcwer likelihccd c!
human errcr, and simpler managemenI
enable lcwer capiIal and cperaIicnal
expendiIures. ThaI sIandardizaIicn
alsc allcws lT Ic enjcy Ihe beneIs c!
simplied managemenI and inIegraIed
Iccls. lnIel expecIs Ihe same beneIs
!rcm 0pen FCcEbased scluIicns.
TesI Ccnf|guraI|cn 0verv|ew
0ur IesI ccnguraIicn is shcwn in Figure
5. we have Iwc lnIel Xecn prccesscr
5600 seriesbased servers running
Nicrcsc!I windcws Server Z00B RZ.
These servers are equipped wiIh lnIel'
X5Z0 series 10 CbE NlCs running naIive
0S iniIiaIcrs !cr iSCSl and FCcE. The NlCs
are ccnnecIed Ic a Ciscc Nexus* 50Z0
swiIch wiIh bre channel suppcrI cpIicns.
This is where Ihe bre channel naming
service runs. The swiIch is ccnnecIed wiIh
Iwc 10 CbE ccnnecIicns Ic a NeIApp*
FAS6ZB0 sIcrage sysIem. This Icpclcgy
prcvides separaIe paIhs !cr Ihe iSCSl
and FCcE Ira!c, which each run aI !ull
CurrenIly, prcducIicn 0pen FCcE scluIicns
are available !rcm majcr vendcrs
such as SUSE* and Nicrcsc!I*. 0Iher
implemenIaIicns will be released sccn.
CcnIacI ycur 0S and virIual machine
mcniIcr (VNN) vendcrs !cr availabiliIy
in!crmaIicn. we used Nicrcsc!I windcws
Server* Z00B RZ in cur IesIs !cr Ihis
paper.
The 0pen FCcE apprcach c!!ers a number
c! advanIages in Ierms c! accessibiliIy and
ease c! use.
Access|b|||Iy: The 0pen FCcE apprcach
makes FC SAN access available Ic any
server wiIh a 0CBcapable 10 CbE adapIer
insIalled. lT deparImenIs can sIandardize
cn Ihese adapIers Ic simpli!y server
ccnnecIiviIy acrcss mulIiple use cases.
wiIh FCcE suppcrI inIegraIed inIc Ihe
cperaIing sysIem, FC SAN access will
beccme readily available acrcss Ihese
ccnverged neIwcrks and even mcre
accessible cnce 10 CbE L0N Iechnclcgy
beccmes mcre pervasive.
Ease cf Use: Because Ihe 0pen FCcE
apprcach uses 0CBcapable 10 CbE
adapIers IhaI ccmply wiIh brcad, well
kncwn EIherneI sIandards, lT can
uIilize exisIing kncwledge Ic ccngure
and manage Ihese adapIers !cr FCcE
deplcymenIs. ln !acI, lT can sIandardize
cn a single prcducI cr prcducI !amily
!cr all LAN and SAN ccnnecIiviIy. FCcE
iniIiaIcr inIegraIicn inIc Ihe 0S alsc
means ccmmcn 0Sbased Iccl suppcrI
acrcss a prcducI !amily cr even adapIers
!rcm mulIiple vendcrs, as Ihey beccme
available.
NaI|ve In|I|aIcr Success: |SCSI
iSCSl prcvides an excellenI example c!
Ihe success c! naIive sIcrage iniIiaIcrs
inIegraIed inIc Ihe cperaIing sysIem.
ln Ihe early days c! iSCSl, prcpcnenIs c!
iSCSl HBAs claimed IhaI Ihese dedicaIed
adapIers were necessary Ic deliver
accepIable per!crmance. iSCSl HBAs
implemenI daIa plan !uncIicns c! Ihe FCcE
sIack in Ihe hardware Ic deliver cpIimum
per!crmance.
0en FCcE: HcmenIum SIarIed |n Ihe
L|nux* Ccmmun|Iy
The Linux ccmmuniIy accepIed Ihe 0pen
FCcE prcjecI in Ncvember Z007 wiIh a
gcal Ic acceleraIe Ihe develcpmenI c! a
naIive FCcE iniIiaIcr in Ihe Linux kernel.
The indusIry respcnded enIhusiasIically,
and Icday Ihere are cver 190 acIive
parIicipanIs in Ihe ccmmuniIy whc
ccnIribuIe ccde, prcvide review
ccmmenIs, and IesI Ihe 0pen FCcE sIack.
Tc daIe, Ihe 0pen FCcE scurce websiIe
(www.cpen!cce.crg) has received cver
Z0,000 hiIs. 0pen indusIry sIandards
and 0pen Scurce play a signicanI rcle
in Ihe mcdern daIa cenIer, as Ihey lcwer
R&0 invesImenI and enable access Ic a
mulIivendcr supply chain IhaI is designed
!cr heIercgenecus inIercperabiliIy. This
ulIimaIely resulIs in greaIer chcice and
lcwer equipmenI ccsIs.
Open FCoE Solution
Application I/O
SCSI storage interface
Fibre Channel protocol
FCoE transport protocol
FCoE datapath ofoads
Ethernet MAC
Ethernet PHY
O
p
e
r
a
t
i
n
g

S
y
s
t
e
m
D
e
v
i
c
e

D
r
i
v
e
r
6
F|gure 4. 0verv|ew cf 0en FCcE In|I|aIcr
Sc|uI|cn w|Ih InIe|' 10 bE NIC
10 CbE speeds !rcm endIcend. lI is alsc
pcssible Ic run iSCSl and FCcE cn a single
ccnnecIicn !rcm Ihe swiIch Ic Ihe NeIApp
sIcrage sysIem.
The hardware and sc!Iware used Ic
execuIe Ihe !uncIicnal and per!crmance
IesIing ccnsisIed c!.
lnIel' whiIe bcx dualprccesscr server
Twc lnIel' Xecn' prccesscr 56B0
seriesbased servers
6 ccres / IcIal 1Z Ihreads each
3.33CHz clcck speed
lnIel X5Z0 series 10 Cb/s EIherneI
adapIer
Z4 CB Nemcry
Zx146 CB H00 RAl0 0
Nicrcsc!I windcws Server Z00B RZ
NeIApp FAS6ZB0 sIcrage sysIem
0aIa 0NTAP* B.0.1 cperaIing sysIem
X1139 Uni!ied TargeI AdapIer (!cr
FCcE)
Z TB Flash Cache (inIelligenI read
cache)
Ciscc Nexus 50Z0 uni!ied swiIch
l0S 4.Z
Nicrcsc!I windcws Server Z00B RZ
NaIive 0S FCcE and iSCSl iniIiaIcrs
lcmeIer versicn Z006.07.Z7 was used
!cr IesIing IhrcughpuI
Tc prcvisicn sIcrage space cn Ihe
NeIApp FAS6ZB0 sysIem, we used
NeIApp Snap0rive* !cr windcws* daIa
managemenI sc!Iware !cr bcIh iSCSl
and FCcE prcIcccls. This simpli!ied Ihe
LUN creaIicn and aIIachmenI prccess.
IcmeIer Ccnf|guraI|cn 0verv|ew
lcmeIer, by de!aulI, will creaIe cne wcrker
Ihread !cr each prccesscr ccre/Ihread
deIecIed. ln Ihis case, Z4 wcrkers were
auIcmaIically creaIed sc IhaI lcmeIer
cculd !ully uIilize Ihe lnIel Xecn prccesscr
archiIecIure.
The wcrkers were each ccngured
wiIh disk IargeIs IhaI pcinI Ic Ihe LUN
aIIached by iSCSl cr FCcE prcIcccl, as
apprcpriaIe.
The number c! cuIsIanding l/0s per IargeI
was seI Ic eighI Ic increase parallelism
and per!crmance.
The access speci!icaIicns were as !cllcws.
Blcck sizes used !cr each FCcE LUN cr
iSCSl LUN. 4 KB, B KB, 16 KB, 3Z KB, 64
KB, 1ZB KB, Z56 KB, 51Z KB, and 10Z4 KB.
Fcr each blcck size we ran seven di!!erenI
IesIs.
1. 100% sequenIial read
Z. 100% sequenIial wriIe
3. 100% randcm read
4. 100% randcm wriIe
5. 50% read/wriIe sequenIial mix
6. 50% read/wriIe randcm mix
7. 50% read/wriIe and 50% randcm/
sequenIial mix
TesI Ccnf|guraI|cn DeIa||s
NeIA SIcrage SysIem Ccnf|guraI|cn
DeIa||s
NeIA Env|rcnmenI SeIu
+DUGZDUH
1. NeIApp FAS6ZB0 sIcrage sysIem
Z. Z.0 TB !lash cache memcry
3. Three 0S14 NK4 shelves wiIh 14 FC
450 CB drives each
4. Twc 0S4Z43 shelves wiIh Z4 SAS
450 CB drives each
5. 0ne 0S4Z43 shel! wiIh Z4 1 TB SATA
drives
6. *** TesIed LUNs - 100 CB Ic each
server (iSCSl & FCcE)
Intel Xeon 5600
series-based
server with Microsoft
Windows* Server
2008 R2.
Intel X520 series 10GbE
NIC with native OS
FCoE initiator.
Intel Xeon 5600 based
server with Microsoft
Windows Server
2008 R2.
Intel X520 series 10GbE
NIC with native OS
iSCSI initiator.
Cisco Nexus* 5020 switch
1
0
G
b
E
F
C
o
E
10GbE FCoE
1
0
G
b
E
iS
C
S
I
10GbE iSCSI
NetApp*
FAS6280
storage
system
7
F|gure 5
1HW$SS$FFHVV/LFHQVLQJDQG*OREDO&RQILJXUDWLRQV
a. SIcrage is presenIed Ic a server as a LUN. LUN creaIicn in NeIApp is builI wiIhin a Vclume, which is builI wiIhin an AggregaIe,
which ccnsisIs c! cne cr mcre redundanI array c! independenI disks (RAl0) grcups.
Data ONTAP* Logical View
Abstraction Layer
LUN
FlexVol
Aggregate
RAID Group
Disk Drive
Efciency Feature
Thin Provisioning
Thin Replication
Snapshot Copies
Virtual Copies
Deduplication
RAID-DP
SATA
b. NcIe IhaI we will handle Ihe glcbal ccn!iguraIicns in Ihis secIicn. The LUN creaIicn deIails will be presenIed in line wiIh Ihe server
iSCSl and FCcE ccn!iguraIicn secIicns Ic !cllcw.
c. Access Ihe sIcrage sysIem Ihrcugh Ihe web pcrIal.
B
F|gure 6
F|gure 7
d. Veri!y IhaI licenses were applied Ihrcugh Nanage Licenses in Ihe le!I menu c! Ihe NeIApp FilerView* adminisIraIicn Iccl. Prcvide
Ihe required credenIials !cr VNware vClcud 0irecIcr Ic ready Ihe hcsIs.
F|gure 8
e. NeIApp sysIems use AggregaIes Ic creaIe pccls c! sIcrage cuI c! cne cr mcre RAl0 grcups, which can Ihen be subdivided inIc
lcgical sIcrage ccnIainers as Flexible Vclumes. SIcrage adminisIraIcrs may alsc Iake acIicns such as creaIing snapshcI ccpies and
vclume ccpies aI Ihe AggregaIe level.
!. Tc creaIe an AggregaIe, cpen AggregaIes !rcm Ihe le!I menu c! FilerView, selecI Add Ic cpen Ihe AggregaIe wizard, and selecI
NexI Ic sIarI Ihe wizard.
9
FIgure 9
1DPHWKH$JJUHJDWHDQG'HILQHLWV3DUDPHWHUV
a. 0cuble PariIy (0P) is a de!aulI. Ensure 0cuble PariIy is checked Ic ensure Ihe new aggregaIe uses a NETAPP RAl00P* disk lay
cuI. RAl00P !urIher decreases Ihe risk c! daIa lcss, as iI ensures IhaI pariIy can never beccme a per!crmance bcIIleneck, and IhaI
a RAl0 grcup can survive Ihe lcss c! mcre Ihan cne disk.
F|gure 11
b. Chcse Ihe number c! disks upcn which Ihe RAl0 grcup will reside.
10
F|gure 10
F|gure 1Z
c. NeIApp gives Ihe adminisIraIcr Ihe chcice c! drives Ic be used in Ihe AggregaIe, and an cpIicn Ic allcw Ihe sIcrage sysIem Ic au
IcmaIically make Ihe chcice.
d. Chccse Ihe 0isk Type !cr Ihe AggregaIe. The chcice is limiIed Ic Ihe physical Iype c! insIalled drives.
F|gure 13
11
e. SelecI !rcm available drives cr leI Ihe sIcrage sysIem auIcmaIically make Ihe chcice.
F|gure 14
!. Chccse Ihe number c! drives !cr Ihe aggregaIe.
1Z
F|gure 15
F|gure 16
g. CcmmiI Ihe AggregaIe Ic Ihe sysIem.
h. Veri!y Ihe AggregaIe creaIicn in Ihe Nanage AggregaIe windcw. A re!resh c! Ihe windcw may be necessary.
13
F|gure 17
1HW$SS&UHDWHV9ROXPHVRU)OH[9ROVIURPWKH$JJUHJDWHV
FlexVcl vclumes are easily creaIed, expcrIed, and mcunIed cnIc neIwcrked servers.
a. Tc creaIe a FlexVcl vclume, cpen Vclumes !rcm Ihe le!I menu c! FilerView and selecI Add Ic cpen Ihe Vclume wizard.
b. Chccse Vclume Type in Ihe Vclume Type windcw.
14
F|gure 18
F|gure 19
c. Name Ihe vclume in Ihe Vclume ParameIers windcws.
d. Assign Ihe vclume Ic an esIablished AggregaIe in Ihe Flexible Vclume ParameIers windcw.
15
F|gure Z0
F|gure Z1
e. SeI Ihe vclume size in Flexible Vclume Size windcw. TcIal Size will creaIe Ihe vclume Ic Ihe exacI speci!icaIicns seI, while
Usable Size will increase Ihe vclume size Ic enable disk space cverhead.
!. NcIe IhaI we did ncI need a SnapshcI Reserve !cr cur IesIing. lnsIead, we jusI used Ihe de!aulI c! Z0 percenI.
g. CcmmiI Ihe vclume Ic Ihe sysIem. ** NcIe Ihe vclume size has increased !rcm Ihe 4Z CB iniIially seI due Ic Ihe SnapshcI Reserve.
16
F|gure ZZ
F|gure Z3
h. Veri!y Ihe vclume in Ihe FilerView Nanage Vclumes windcw.
i. Clicking cn Ihe vclume in Ihe Nanage Vclumes windcw will shcw Ihe ccn!iguraIicn parameIers c! IhaI vclume. ** NcIe Ihe TcIal
Size di!!erence !rcm Ihe TcIal CapaciIy c! Ihe vclume due Ic Ihe Usable Size seIIing.
17
F|gure Z4
F|gure Z5
,QLWLDWRU*URXSV
lniIiaIcr grcups are a way !cr sIcrage adminisIraIcrs Ic grcup LUNs acccrding Ic Ihe parameIers c! Ihe envircnmenIs cr servers Ihey
serve. ln cur case, Ihis will need Ic be dcne !cr bcIh FCcE and iSCSl iniIiaIcrs.
a. Tc creaIe an lniIiaIcr Crcup, cpen !rcm Ihe le!I menu c! FilerView and selecI LUNs ` lniIiaIcr Crcups ` Add Ic cpen Ihe Add
lniIiaIcr Crcup windcw. ** lniIiaIcr Crcups can be creaIed as parI c! Ihe LUN creaIicn prccess.
b. Name Ihe lniIiaIcr Crcup. Chccse a Iype (FCP cr iSCSl), chccse an 0S Iype, and inserI Ihe IargeI's wcrldwide name (wwN) cr iSCSl
quali!ied name (l0N) ncde name. An FCcE lniIiaIcr Crcup is creaIed here.
1B
F|gure Z6
F|gure Z7
b. we ccver Ihe LUN creaIicn deIails in line wiIh Ihe iSCSl and FCcE server ccn!iguraIicn secIicns Ic !cllcw.
C|scc Sw|Ich Ccnf|guraI|cn DeIa||s
1. Hardware
a. Ciscc Nexus 50Z0 Chassis 40x10 CE/Superviscr
b. lnIel' Celercn' prccesscr
c. Z CB Nemcry
d. 4x10 CE + 4x1/Z/4C FC Ncdule
c. Veri!y Ihe lniIiaIcr Crcup in Ihe Nanage lniIiaIcr Crcups windcw. A re!resh c! Ihe windcw may be necessary.
&UHDWH/81VIURP)OH[9RO9ROXPHV
A LUN is a lcgical cbjecI wiIh SAN prcperIies.
a. LUNs are managed in Ihe Nanage LUNs windcw !cund in Ihe le!I menu c! FilerView. SelecI LUNs ` Nanage Ic !ind Ihe LUNs, LUN
descripIicn, and asscciaIed lniIiaIcr Crcups.
19
F|gure Z8
F|gure Z9
Z. Sc!Iware
a. SysIem Versicn. 4.Z(1)N1(1)
b. Bl0S. versicn 1.Z.0
c. SysIem image !ile is. bccI!lash./n5000uk9.4.Z.1.N1.1.bin
DeIa||ed SIes
Ccmmand cr AcI|cn Purcse
SIep 1 SwiIch# Ccmmand Ierminal EnIers ccnguraIicn mcde.
SIep Z SwiIch (ccng)# !eaIure !cce AcIivaIes FCcE.
SIep 3 SwiIch (ccng)# pclicymap Iype neIwcrkqcs
jumbc
Ccngures/creaIes pclicymap c! Iype neIwcrkqcs wiIh Ihe name
jumbc.
SIep 4 SwiIch (ccngpmapnq)# class Iype neIwcrk
qcs classde!aulI
SeIs Ihe class Iype Ic classde!aulI (classde!aulI is creaIed cn sIarIup
by Ihe 0S and canncI be deleIed).
SIep 5 SwiIch (ccngpmapnq) mIu 9Z16 SeIs !rame size Ic 9Z16. This is Ihe jumbc !rame seIIing. |umbc !rames
prcvide Ihe besI per!crmance !cr iSCSl wiIh large Irans!ers. NcIe IhaI
FCcE uses jumbc !rames by de!aulI.
SIep 6 SwiIch (ccngpmapnq)# sysIem qcs Ccngures qcs (Pclicies are enabled by qcs).
SIep 7 SwiIch (ccngsysqcs)# servicepclicy Iype
neIwcrkqcs jumbc
Ccngures/creaIes servicepclicy c! Iype neIwcrkqcs wiIh Ihe name
jumbc.
SIep B SwiIch (ccngsysqcs)# vr! ccnIexI
managemenI
EnIers VRF ccnIexI managemenI ccnguraIicn mcde.
SIep 9 SwiIch (ccngvr!)# ip rcuIe 0.0.0.0/0 10.#.#.# Ccngures Ihe lPv4 address c! Ihe nexI hcp.
SIep 10 SwiIch (ccngvr!)# vlan 41 CreaIes a VLAN.
SIep 11 SwiIch (ccngvlan) name iSCSl 0pIicnal. Name a VLAN.
SIep 1Z SwiIch (ccngvr!)# vlan 100 CreaIes a VLAN.
SIep 13 SwiIch (ccngvlan) !cce vsan Z Enables FCcE !cr Ihe specied VLAN.
SIep 14 SwiIch (ccngvlan) # name FCcE 0pIicnal. Name a VLAN.
SIep 15 SwiIch (ccngvlan) # vsan daIabase EnIers VSAN ccnguraIicn mcde.
SIep 16 SwiIch (ccngvsandb) # vsan Z Adds Ihe VSAN Ic Ihe VSAN daIabase.
SIep 17 SwiIch (ccngvsandb) # !cdcmain !cid
daIabase
EnIers FC l0 daIabase ccnguraIicn submcde.
SIep 1B SwiIch (ccng!ciddb) # vsan Z wwn
Z0.xx.xx.xx.xx.xx.xx.xx !cid dynamic
Ccngures a device wwN (Z0.xx.xx.xx.xx.xx.xx.xx) in Ihe specied
VSAN in dynamic mcde.
SIep 19 SwiIch (ccng!ciddb) # inIer!ace v!c01 CreaIes a virIual bre channel inIer!ace (i! iI dces ncI already exisI) and
enIers inIer!ace ccnguraIicn mcde.
SIep Z0 SwiIch (ccngi!)# bind inIer!ace EIherneI1/1 Binds Ihe virIual bre channel inIer!ace Ic Ihe specied inIer!ace.
SIep Z1 SwiIch (ccngi!)# nc shuIdcwn Fcrces Ihe pcrI sIaIe Ic up.
SIep ZZ SwiIch (ccngi!)# inIer!ace v!c0Z CreaIes a virIual bre channel inIer!ace (i! iI dces ncI already exisI) and
enIers inIer!ace ccnguraIicn mcde.
SIep Z3 SwiIch (ccngi!)# bind inIer!ace EIherneI1/Z Binds Ihe virIual bre channel inIer!ace Ic Ihe specied inIer!ace.
SIep Z4 SwiIch (ccngi!)# nc shuIdcwn Fcrces Ihe pcrI sIaIe Ic up.
SIep Z5 SwiIch (ccngi!)# vsan daIabase EnIers VSAN ccnguraIicn mcde.
SIep Z6 SwiIch (ccngvsandb)# vsan Z inIer!ace v!c01 Ccngures Ihe asscciaIicn beIween Ihe VSAN and virIual bre channel
inIer!ace.
Z0
SIep Z7 SwiIch (ccngvsandb)# vsan Z inIer!ace v!c0Z The VSAN number musI map Ic a VLAN cn Ihe physical EIherneI
inIer!ace IhaI is bcund Ic Ihe virIual bre channel inIer!ace.
SIep ZB SwiIch (ccngvsandb)# inIer!ace EIherneI1/1 Species an inIer!ace Ic ccngure, and enIers inIer!ace ccnguraIicn
mcde.
SIep Z9 SwiIch (ccngi!)# descripIicn FCcEServer01 0pIicnal. Name a VLAN.
SIep 30 SwiIch (ccngi!)# swiIchpcrI mcde Irunk SeIs an inIer!ace as an EIherneI Irunk pcrI.
SIep 31 SwiIch (ccngi!)# swiIchpcrI Irunk allcwed
vlan 100
Species IhaI cnly cerIain VLANs are allcwed cn Ihe specied Irunk.
SIep 3Z SwiIch (ccngi!)# spanningIree pcrI Iype
edge Irunk
ExpliciIly enables edge behavicr cn Ihe Irunk pcrI. Edge pcrIs
immediaIely IransiIicn Ic Ihe !crwarding sIaIe wiIhcuI passing Ihrcugh
Ihe blccking cr learning sIaIe aI linkup.
SIep 33 SwiIch (ccngi!)# nc shuI Fcrces Ihe pcrI sIaIe Ic up.
SIep 34 SwiIch (ccngi!)# inIer!ace EIherneI1/Z Species an inIer!ace Ic ccngure, and enIers inIer!ace ccnguraIicn
mcde.
SIep 35 SwiIch (ccngi!)# descripIicn SIcrage 05e 0pIicnal. Name Ihe inIer!ace.
SIep 36 SwiIch (ccngi!)# swiIchpcrI mcde Irunk SeIs an inIer!ace as an EIherneI Irunk pcrI.
SIep 37 SwiIch (ccngi!)# swiIchpcrI Irunk allcwed
vlan 100
Species IhaI cnly cerIain VLANs are allcwed cn Ihe specied Irunk.
SIep 3B SwiIch (ccngi!)# spanningIree pcrI Iype
edge Irunk
ExpliciIly enables edge behavicr cn Ihe Irunk pcrI. Edge pcrIs
immediaIely IransiIicn Ic Ihe !crwarding sIaIe wiIhcuI passing Ihrcugh
Ihe blccking cr learning sIaIe aI linkup.
SIep 40 SwiIch (ccngi!)# inIer!ace EIherneI1/3 Species an inIer!ace Ic ccngure, and enIers inIer!ace ccnguraIicn
mcde.
SIep 41 SwiIch (ccngi!)# descripIicn iSCSlServer01 0pIicnal. Name Ihe inIer!ace.
SIep 4Z SwiIch (ccngi!)# swiIchpcrI access vlan 41 SeIs Ihe access mcde c! Ihe inIer!ace Ic Ihe specied VLAN.
SIep 44 SwiIch (ccngi!)# inIer!ace EIherneI1/4 Species an inIer!ace Ic ccngure, and enIers inIer!ace ccnguraIicn
mcde.
SIep 45 SwiIch (ccngi!)# descripIicn SIcrage 05d 0pIicnal. Name Ihe inIer!ace.
SIep 46 SwiIch (ccngi!)# swiIchpcrI access vlan 41 0pIicnal. Name Ihe inIer!ace.
SIep 4B SwiIch (ccngi!)# inIer!ace mgmI0 Species Ihe managemenI inIer!ace Ic ccngure, and enIers
managemenI inIer!ace ccnguraIicn mcde.
SIep 49 SwiIch (ccngi!)# ip address 10.#.#.# SeIs Ihe lP !cr swiIch managemenI.
SIep 50 SwiIch (ccngi!)# zcne name zcne1 vsan Z Ccngures and names zcne1 in VSANZ.
SIep 51 SwiIch (ccngzcne)# member pwwn
Z0.xx.xx.xx.xx.xx.xx.xx
Adds pcrI wcrld wide name (PwwN) c! server's lnIel' X5Z0 Series
Server AdapIer Ic zcne. Per!crm Ihis sIep !cr each server PwwN.
SIep 5Z SwiIch (ccngzcne)# member pwwn
50.xx.xx.xx.xx.xx.xx.xx
Adds pcrI wcrld wide name (PwwN) c! NeIApp* Unied TargeI
AdapIers (UTA) Ic zcne. Per!crm Ihis sIep !cr each member PwwN.
SIep 53 SwiIch (ccngzcne)# zcne name zcne1 vsan Z ReenIers zcne seI #1.
SIep 54 SwiIch (ccng zcneseI)# member zcne1 Adds a zcne as a member c! Ihe previcusly specied zcne seI.
SIep 55 SwiIch (ccng zcneseI)# zcneseI acIivaIe
name zcneseI1 vsan Z
AcIivaIes Ihe zcne seI and updaIes Ihe new zcne name in Ihe acIive
zcne seI.
Z1
SIep 56 SwiIch (ccng)# exiI ExiIs Ihe ccnguraIicn mcde.
SIep 57 SwiIch# ccpy runningccng sIarIupccng Saves Ihe ccnguraIicn.
SIep 5B SwiIch# shcw vlan 0isplays selecIed ccnguraIicn in!crmaIicn !cr Ihe dened VLAN(s).
VLAN Name Status Ports
---- ------------------------------- --------- --------------------------------------
41 iSCSI active Eth1/2, Eth1/4
100 FCOE active Eth1/1, Eth1/3
SIep 59 SwiIch# shcw vsan Z 0isplays in!crmaIicn abcuI a specic VSAN.
vsan 2 information
name:VSAN0002 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:up
SIep 60 SwiIch# shcw !cce daIabase 0isplays Ihe ccnIenIs c! Ihe FCcE daIabase.
-------------------------------------------------------------------------------
INTERFACE FCID PORT NAME MAC ADDRESS
-------------------------------------------------------------------------------
Vfc01 0x880001 20:xx:xx:xx:xx:xx:xx:xx 00:1b:21:5e:c7:eb
Vfc02 0x880004 50:xx:xx:xx:xx:xx:xx:xx 00:c0:de:14:e4:27
SIep 61 SwiIch# shcw inIer!ace eIherneI 1/1 0isplays Ihe physical EIherneI inIer!ace, veri!y i! Ihe inIer!ace is in
Irunk mcde, veri!y Ihe Ira!c dcw raIe, veri!y IhaI Ihe jumbc NTU is
enabled.
Ethernet1/21 is up
Hardware: 10000 Ethernet, address: 000d.ecb2.8bdc (bia 000d.ecb2.8bdc)
Description: NetAppEPI01
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 10 Gb/s, media type is 10g
Beacon is turned off
Input How-control is off, output How-control is off
Rate mode is dedicated
Switchport monitor is off
Last link Happed 1d00h
Last clearing of show interface counters 6d00h
30 seconds input rate 88 bits/sec, 0 packets/sec
30 seconds output rate 448 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 136 bps, 0 pps; output rate 224 bps, 0 pps
ZZ
RX
6917486410 unicast packets 12947336 multicast packets 2100 broadcast packets
6930435846 input packets 12891728686220 bytes
6051166314 jumbo packets 0 storm suppression packets
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
12913702 Rx pause
TX
7594352404 unicast packets 1139867 multicast packets 2028 broadcast packets
7595494299 output packets 13335728311942 bytes
6075073811 jumbo packets
0 output errors 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble
711978 Tx pause
5 interface resets
SIep 63 SwiIch# Shcw inIer!ace eIherneI 1/1 !cce 0isplays Ihe FCcE seIIings !cr an inIer!ace.
Ethernet1/1 is FCoE UP
vfc01 is Up
FCID is 0x880003
PWWN is 20:xx:xx:xx:xx:xx:xx:x
MAC addr is 00:1c:21:5f:c7:eb
SIep 64 SwiIch# shcw vlan !cce 0isplays Ihe mapping c! FCcE VLANs Ic VSANs.
VLAN VSAN Status
-------- -------- -------- -------- -------------
100 2 Operational
|SCSI Ccnf|guraI|cn DeIa||s
Tc ccngure Ihe iSCSl ccnnecIicn and maximize TCP/lP IhrcughpuI, Ihe !cllcwing neIwcrking besI pracIices were cbserved.
Receive Side Scaling was enabled cn Ihe hcsI's lnIel X5Z0 EIherneI adapIer.
This !eaIure maximizes Ihe large number c! available prccesscr ccres in sysIems IhaI use Ihe lnIel Xecn prccesscrs because iI
disIribuIes Ihe wcrk c! prccessing Ira!!ic amcng mulIiple prccesscrs.
NcIe. Ihis !eaIure is enabled by de!aulI in windcws Server Z00B RZ.
See SIep 3d belcw in Ihis secIicn Ic see hcw Ic veri!y Ihis seIIing.
Large SegmenI 0!!lcad was alsc enabled cn Ihe hcsI's X5Z0 EIherneI adapIer.
This !eaIure mcves Ihe wcrk c! spliIIing large daIa requesIs inIc segmenIs small encugh Ic be IransmiIIed by Ihe lP prcIcccl Ic
Ihe EIherneI adapIer.
NcIe. Ihis !eaIure is enabled by de!aulI in windcws Server Z00B RZ.
See SIep 3e belcw in Ihis secIicn Ic see hcw Ic veri!y Ihis seIIing.
|umbc !rames c! 9000 byIes were ccn!igured cn Ihe hcsI, swiIch, and sIcrage.
This reduces Ihe amcunI c! neIwcrk cverhead required !cr TCP/lP headers and checksums as iI decreases Ihe raIic c! meIadaIa
Ic daIa in each packeI senI cr received cver Ihe sIandard NTU size c! 1500 byIes.
Z3
a. windcws UpdaIes
b. Firewall & lE ESC bcIh Iurned c!!
c. |umbc !rames enabled
See SIep 3c belcw in Ihis secIicn Ic see Ihe server seIup !cr jumbc !rames.
See SIep 5 in Ihe Ciscc swiIch ccn!iguraIicn abcve !cr jumbc !rame seIup.
See SIep 6a belcw in Ihis secIicn !cr Ihe NeIApp sIcrage sysIem jumbc !rame seIup.
NulIiple TCP sIreams were esIablished beIween hcsI and sIcrage Ihrcugh use c! a mulIiccnnecIicn sessicn (NCS).
NCS allcws mulIiple TCP sIreams Ic carry iSCSl Ira!!ic. This !eaIure imprcves per!crmance because iI adds parallelism.
See SIep 10 belcw in Ihis secIicn Ic see Ihe seIup !cr NCS.
NcIe. The use c! Ihe windcws Z00B RZ sIandard Nicrcsc!I* NulIipaIh l/0 (NPl0) implemenIaIicn is ancIher way Ic Iake
advanIage c! Ihe bene!iIs c! mulIiple TCP sIreams.
Server
1. Hardware
a. lnIel Xecn prccesscr 56B0 series Q 3.33 CHz
b. Z4 CB Nemcry
c. Zx146 CB H00 RAl0 0
d. lnIel' NianIic NlC driver updaIe Z.5.5Z.0
Z. Sc!Iware
a. windcws Server Z00B RZ
b. lnsIall Ihe laIesI lnIel X5Z0 10 CB driver updaIe
i. Find Ihe driver aI hIIp.//dcwnlcadcenIer.inIel.ccm
ii. Search by.
1. PrcducI Family - NeIwcrk CcnnecIiviIy
Z. PrcducI Line - lnIel Server AdapIers
3. PrcducI Name - lnIel EIherneI Server AdapIer X5Z0 Series
a. The currenI quali!ied FCcE 0rivers are in Package Versicn 15.4.1.
4. we used Ihe EIherneI 0river versicn Z.5.5Z.0
Z4
F|gure 30
3. iSCSl NlC PrcperIies SeIup
a. Launch Ihe Server Nanager uIiliIy and brcwse Ic NeIwcrk AdapIers. RighI click and cpen PrcperIies.
b. Access Ihe 0river Iab Ic veri!y driver.
Z5
F|gure 31
F|gure 3Z
c. Access Ihe Advanced Iab Ic make NlC seIIing changes. we enabled jumbc !rames here.
d. Alsc cn Ihe Advanced Iab, check Ihe seIIing !cr Receive Side Scaling. lI shculd be enabled by de!aulI, buI dcuble check.
Z6
F|gure 33
F|gure 34
4. Server iSCSl lniIiaIcr
a. Frcm Ihe server's SIarI menu, navigaIe Ic AdminisIraIive Tccls ` iSCSl lniIiaIcr.
b. A!Ier launching, gc Ic Ihe Ccn!iguraIicn Iab Ic geI Ihe lniIiaIcr Name. The lniIiaIcr Name will be used cn Ihe sIcrage device Ic
creaIe a IargeI.
e. Alsc cn Ihe Advanced Iab, check Ihe seIIing !cr Large Send 0!!lcad. lI shculd be enabled by de!aulI, buI dcuble check.
Z7
F|gure 35
F|gure 36
c. l! Ihis is Ihe !irsI Iime Ihe iSCSl lniIiaIcr has been run, Ihe sysIem will repcrI IhaI Ihe iSCSl service is ncI running and will ask !cr
permissicn Ic sIarI Ihe service. This cnly happens Ihe !irsI Iime.
d. A!Ier launch, navigaIe Ic Ihe Ccn!iguraIicn Iab c! Ihe iSCSl iniIiaIcr prcperIies and capIure Ihe server's iniIiaIcr name.
5. SIc| Prcceed Ic Ihe NeIApp sIcrage sysIem Ic make Ihe necessary changes Ic ccn!igure and assign a sIcrage IargeI Ic Ihe
server.
6. iSCSl Ccn!iguraIicn cn Ihe NeIApp sIcrage sysIem.
a. A besI pracIice !cr iSCSl is Ic use jumbc !rames. This prcvides Ihe besI per!crmance !cr larger Irans!er sizes.
i. Frcm Ihe ccmmand line Iype Ihe !cllcwing ccmmand.
1. i!ccn!ig inIer!ace` mIusize 9000
Z. RepeaI Ihe abcve ccmmand line !cr each inIer!ace IhaI is used !cr iSCSl prcIcccl.
ii. Ycu can veri!y Ihe jumbc !rame seIIing Ihrcugh Ihe CUl. See Appendix B !cr deIails.
ZB
F|gure 37
F|gure 38
b. iSCSl is enabled by de!aulI cn all c! Ihe pcrIs c! Ihe NeIApp sIcrage sysIem. As a besI pracIice, iI shculd be Iurned c!! cn pcrIs
IhaI dc ncI require iSCSl usage. Tc Iurn iI c!!, selecI Nanage lnIer!aces under LUNs ` iSCSl !rcm Ihe le!I menu c! FilerView.
c. Tc creaIe an iSCSl LUN, cpen LUNs !rcm Ihe le!I menu c! FilerView and selecI wizard Ic cpen Ihe LUN wizard.
Z9
F|gure 39
F|gure 40
d. SelecI NexI aI Ihe LUN wizard welccme screen.
e. SeI LUN ccn!iguraIicns aI Ihe Speci!y LUN ParameIers windcw. PaIh in!crmaIicn is available Ihrcugh Ihe Filer Nanage Vclumes
windcw, Ihe name is adminisIraIcr de!ined. SeI LUN size, prcIcccl Iype, and selecI NexI.
30
F|gure 41
F|gure 4Z
!. Fcr a LUN Ic be acIivaIed, iI musI be added Ic an lniIiaIcr Crcup. lniIiaIcr Crcups can be creaIed during LUN creaIicn cr
independenIly. This sIudy used bcIh meIhcds. independenIly in FCcE and during LUN creaIicn wiIh iSCSl. SelecI Add Crcup Ic
ccn!igure LUN wiIh an exisIing lniIiaIcr Crcup.
g. SelecI Use exisIing iniIiaIcr grcup and selecI NexI Ic cpen Ihe LUN Nap Add Crcup windcw.
31
F|gure 43
F|gure 44
h. The LUN Nap Add Crcups windcw cpens. SelecI Ihe ExisIing iSCSl lniIiaIcr Crcup and selecI NexI.
i. SelecI NexI aI Ihe Add lniIiaIcr Crcups windcw.
3Z
F|gure 45
F|gure 46
j. Assign a LUN l0 number. l! le!I blank, Ihe sIcrage sysIem will assign cne auIcmaIically.
k. SelecI CcmmiI Ic save Ihe changes made in Ihe LUN wizard.
33
F|gure 47
F|gure 48
l. SelecI Clcse windcw Ic ccmpleIe Ihe changes.
m. Veri!y Ihe LUN's creaIicn in Ihe Nanage LUNs windcw. A re!resh may be necessary.
7. SIc| The LUN is ncw creaIed, sc mcve back Ic Ihe server and add Ihe impcrIed drive.
B. iSCSl TargeI 0isccvery and Final SeIup
a. Upcn ccmpleIicn c! Ihe IargeI seIup and assignmenI, relaunch Ihe iSCSl lniIiaIcr.
34
F|gure 49
F|gure 50
F|gure 51
b. NavigaIe Ic Ihe 0isccvery Iab c! Ihe iSCSl lniIiaIcr prcperIies windcw and launch Ihe 0isccver PcrIal.
c. ln Ihe 0isccver TargeI PcrIal, enIer Ihe lP address !cr Ihe NeIApp iSCSl inIer!ace and press Ihe 0K buIIcn.
35
F|gure 5Z
F|gure 53
d. NavigaIe Ic Ihe TargeIs Iab c! Ihe iSCSl lniIiaIcr PrcperIies windcw and press Ihe Re!resh buIIcn. The iSCSl TargeI shculd
ncw be presenI in Ihe 0isccvered TargeIs windcw.
9. lniIializing 0rives. l! neIwcrk and sIcrage parameIers are ccrrecIly ccn!igured, LUNs will be applied Ic Ihe server. These drives will
acI like hcIswapped, impcrIed drives which need Ic be brcughI cnline, iniIialized, and !crmaIIed !cr use.
a. EnIer Ihe Server Nanager uIiliIy (righIclick CcmpuIer and selecI Nanage) and Ihen Ihe SIcrage ` 0isk NanagemenI Iccl.
b. RighI click Ihe unkncwn disk and bring iI 0nline.
36
F|gure 54
F|gure 55
c. RighI click Ihe unkncwn disk Ic iniIialize Ihe disk.
d. CreaIe a new simple vclume and !crmaI !cr use.
e. AddiIicn c! mulIiple ccnnecIicn sessicns (NCS). This !eaIure imprcves per!crmance because iI adds parallelism.
6
!. Relaunch Ihe iSCSl lniIiaIcr.
37
F|gure 56
F|gure 57
F|gure 58
g. NavigaIe Ic Ihe TargeIs Iab c! Ihe iSCSl lniIiaIcr prcperIies windcw and launch Ihe 0isccver PcrIal. Then highlighI and selecI
Ihe IargeI's prcperIies.
h. SelecI Ihe NulIiple CcnnecIed Sessicns (NCS).
3B
F|gure 59
F|gure 60
i. SelecI Ihe Add buIIcn Ic allccaIe ancIher IesI ccnnecIicn.
j. ln cur IesIing, we used eighI sessicns.
FCcE Ccnf|guraI|cn DeIa||s
FCcE Server SeIu
1. Hardware
a. lnIel Xecn 56B0 3.33 CHz
b. Z4 CB Nemcry
c. Zx146 CB H00 RAl0 0
d. lnIel X5Z0 NianIic NlC driver updaIe Z.5.5Z.0
Z. Sc!Iware
a. windcws Server Z00B RZ
b. lnsIall Ihe laIesI lnIel X5Z0 10 CB driver updaIe
i. Find Ihe driver aI hIIp.//dcwnlcadcenIer.inIel.ccm
ii. Search by.
1. PrcducI Family - NeIwcrk CcnnecIiviIy
Z. PrcducI Line - lnIel Server AdapIers
39
F|gure 61
3. PrcducI Name - lnIel' EIherneI Server AdapIer X5Z0 Series
a. The currenI quali!ied FCcE 0rivers are in Package Versicn 15.4.1
i. we used Ihe EIherneI 0river versicn Z.5.5Z.0
c. windcws UpdaIes
d. Firewall & lE ESC bcIh Iurned c!!
3. lnIel X5Z0 NlC lnsIallaIicn wiIh FCcE 0pIicn
a. Launch Ihe lnIel X5Z0 (PR0winx64) 10 CB driver updaIe and accepI Ihe Licensing AgreemenIs.
i. AI Ihe SeIup 0pIicns screen, check Ihe 0aIa CenIer Bridging/Fibre Channel cver EIherneI cpIicn and ccmpleIe Ihe
driver updaIe.
40
F|gure 6Z
F|gure 63
b. Veri!y driver.
4. FCcE 0isccvery
a. Launch Ihe Server Nanager uIiliIy and brcwse Ic NeIwcrk AdapIers. RighI click and cpen PrcperIies.
41
F|gure 64
F|gure 65
c. Veri!y IhaI Ihe 0aIa CenIer Iab is available. This veri!ies IhaI Ihe 0aIa CenIer Bridging/Fibre Channel cver EIherneI cpIicn is
enabled. The FCcE PrcperIies will navigaIe ycu Ic Ihe SIcrage CcnIrcller PrcperIies.
d. Thrcugh Ihe Server Nanager uIiliIy, brcwse Ic SIcrage CcnIrcllers. RighI click and cpen PrcperIies.
4Z
F|gure 66
F|gure 67
e. The SIcrage CcnIrcller prcperIies are cnly available i! Ihe 0aIa CenIer Bridging/Fibre Channel cver EIherneI cpIicn is enabled.
NcsI Iabs are viewed Ic veri!y seIIings, buI Ihe primary Iab we need is Ihe Fabric View Iab. lI ccnIains Ihe PcrI wwN and
Ncde wwN. These are required !cr ccn!iguraIicn cn Ihe daIa cenIer swiIch and Ihe sIcrage sysIem.
***Please ncIe IhaI Ihe seccndary Hex lccaIicns are di!!erenI (by design). This is dcne Ic redecI Ihe VLAN l0 !cr a parIicular
ccnnecIicn.
!. 0Iher Iabs Ic ncIe.
i. Advanced. PcrI wwN. PcrI PrcperIies buIIcn will navigaIe ycu Ic Ihe NeIwcrk AdapIer PrcperIies, and FC PcrI SIcrage
SeIIings.
43
F|gure 68
F|gure 69
ii. SIaIisIics. Cive Ira!!ic in!crmaIicn and repcrIing.
iii. VLAN. Cive asscciaIed VLAN l0s.
44
F|gure 70
F|gure 71
5. SIcpl Prcceed Ic Ihe NeIApp sIcrage sysIem Ic make Ihe necessary changes Ic ccn!igure and assign a sIcrage IargeI Ic Ihe
server.
6. FCcE ccn!iguraIicn cn NeIApp sIcrage sysIem.
a. Tc creaIe an FCcE LUN, cpen LUNs !rcm Ihe le!I menu c! FilerView and selecI wizard Ic cpen Ihe LUN wizard.
b. SelecI NexI aI Ihe LUN wizard welccme screen.
45
F|gure 7Z
F|gure 73
c. SeI LUN ccn!iguraIicns aI Ihe Speci!y LUN ParameIers windcw. PaIh in!crmaIicn is available Ihrcugh Ihe Filer Nanage Vclumes
windcw, Ihe name is adminisIraIcr de!ined. SeI LUN size, prcIcccl Iype, and selecI NexI.
d. Fcr a LUN Ic be acIivaIed, iI musI be added Ic an lniIiaIcr Crcup. lniIiaIcr Crcups can be creaIed during LUN creaIicn cr
independenIly. This sIudy used bcIh meIhcds, independenIly in FCcE and during LUN creaIicn wiIh iSCSl. SelecI NexI Ic
ccn!igure LUN wiIhcuI Ihe lniIiaIcr Crcup, which will be creaIed independenIly aI a laIer pcinI.
46
F|gure 74
F|gure 75
e. SelecI CcmmiI Ic save Ihe changes made in Ihe LUN wizard and clcse Ihe success windcw.
!. Veri!y Ihe LUN's creaIicn in Ihe Nanage LUNs windcw. ln Ihis example, Ihe LUN is creaIed wiIh nc mapping Ic an lniIiaIcr Crcup,
sc iI musI be added !cr LUN usage. Tc add Ihe mapping, selecI Nc Naps c! Ihe LUN IhaI requires an lniIiaIcr Crcup.
g. This will cpen Ihe LUN Nap windcw. SelecI Add Crcups Ic Nap !cr LUN, neIappepi01!cce1 and selecI Apply.
47
F|gure 76
F|gure 77
F|gure 78
h. The LUN Nap Add Crcups windcw cpens. SelecI Ihe FCcE lniIiaIcr Crcup and selecI Add.
i. CreaIe a LUN l0 number cr i! le!I blank, Ihe sysIem will creaIe iI, and selecI Apply.
j. Veri!y Ihe LUN has been mapped Ic Ihe ccrrecI grcup in Ihe Nanage LUN windcw.
4B
F|gure 79
F|gure 80
F|gure 81
b. RighI click Ihe unkncwn disk Ic iniIialize Ihe disk.
7. SIc| The LUN is ncw creaIed, sc mcve back Ic Ihe server and add Ihe impcrIed drive.
B. lniIializing 0rives. l! neIwcrk and sIcrage parameIers are ccrrecIly ccn!igured, LUNs will be applied Ic Ihe server. These drives will
acI like hcIswapped, impcrIed drives which need Ic be brcughI cnline, iniIialized, and !crmaIIed !cr use.
a. RighI click Ihe unkncwn disk and bring iI 0nline.
c. CreaIe a new simple vclume and !crmaI !cr use.
49
F|gure 8Z
F|gure 83
F|gure 84
IcmeIer TesI Tcc| Ccnf|guraI|cn DeIa||s
1. 0n ycur windcws server hcsI, dcwnlcad and insIall lcmeIer. Ycu can dcwnlcad lcmeIer aI hIIp.//www.icmeIer.crg.
Z. Be!cre ycu launch lcmeIer, make sure ycur iSCSl/FCcE LUNS are mapped Ic Ihe windcws hcsI.
3. Frcm server's sIarI menu, navigaIe Ic lcmeIer.
4. The de!aulI screen will appear as shcwn in Figure B6.
50
F|gure 85
F|gure 86
5. Ncw gc Ic Ihe Tcpclcgy secIicn and selecI ycur windcws server hcsI name. Expand Ihe manager and ycu will see a lisI c!
wcrkers. By de!aulI, Ihe number c! wcrkers is equal Ic Ihe number c! prccesscr ccres ycu have cn Ihe server (Z4 in Ihis example).
6. SelecI Ihe manager. in cur example, NETAPPEPl0Z and Ihe 0isk TargeIs Iab shculd be selecIed by de!aulI (i! ncI, selecI iI).
By de!aulI Ihe values in Ihe !ields Naximum 0isk Size and # c! 0uIsIanding l/0s are seI Ic 0 and 1 respecIively. Change Ihese
values Ic Ihcse shcwn belcw.
Naximum 0isk Size - 10000 secIcrs
# c! 0uIsIanding l/0s - B per IargeI.
NcIe 1 lcmeIer creaIes a le called icbw.IsI in Ihe iSCSl/FCcE LUN based cn Ihe value ycu enIer in Ihe Naximum 0isk Size (and
runs Ihe IesIs wiIh Ihis le size). l! ycu use Ihe de!aulI 0, lcmeIer will creaIe a le equal Ic Ihe size c! ycur LUN. ln Ihis example,
since we used a value c! 10000, a le c! 5 NB was creaIed.
NcIe Z - ln cur IesIing, we Iried varicus values !cr # c! 0uIsIanding l/0s and we !cund IhaI B gave Ihe cpIimal per!crmance.
This number is dependenI cn Ihe queue depIh c! ycur IargeI.
51
F|gure 87
F|gure 88
7. ln Ihe 0isk TargeIs secIicn, selecI Ihe disk IargeI ycu wanI Ic run Ihe Iccl againsI. ln crder Ic use mcre Ihan 1 wcrker (which is
de!aulI), ycu need Ic selecI Ihe disk IargeI !cr each wcrker manually, as shcwn belcw.
B. SelecI Ihe Access Speci!icaIicns Iab in Ihe lcmeIer windcw and selecI New Ic creaIe a new IesI.
5Z
F|gure 89
F|gure 90
9. ln Ihe new windcw, speci!y Ihe IesI ycu wanI Ic run. Belcw is a sample !cr 4 K 100% sequenIial read.
10. ln cur IesIing we used varying requesI sizes (4 KB Ic 10Z4 KB), and !cr all Ihese requesI sizes we used Ihe !cllcwing
ccmbinaIicns.
100% read - 100% sequenIial
100% wriIe - 100% sequenIial
100% read - 100% randcm
100% wriIe - 100% randcm
50% read - 100% sequenIial
53
F|gure 91
F|gure 9Z
11. 0n Ihe lcmeIer windcw, selecI Ihe IesI seIup secIicn sc IhaI ycu can speci!y Ihe IesI duraIicn, ramp up Iime, eIc. Figure 93
shcws Ihe sample.
1Z. A!Ier Ihe seIup, sIarI Ihe IesI by clicking cn Ihe green !lag.
54
F|gure 93
F|gure 94
13. Figure 95 shcws Ihe resulIs screen.
|SCSI FuncI|cna||Iy and Perfcrmance Pesu|Is
The iSCSl naIive 0S iniIiaIcr running cver a 10 CbE endIcend scluIicn was !ully !uncIicnal and per!crmed very well.
55
F|gure 95
F|gure 96
ThrcughpuI apprcached Ihe 10 Cbps line raIe !cr Ihe 100% read IesIs aI blcck sizes c! B KB and abcve. Prccesscr uIilizaIicn peaked
~37% !cr Ihe 4 KB blcck size, and declined Ic ~7% !cr Ihe larger blcck sizes. Nc di!!erence was cbserved !cr randcm vs. sequenIial
access paIIerns. This is aIIribuIable Ic Ihe large Flash Cache in Ihe NeIApp sysIem.
56
F|gure 97
F|gure 98
ThrcughpuI apprcached line raIe !cr Ihe 100% wriIe IesIs aI blcck sizes c! 3Z KB and abcve. Prccesscr uIilizaIicn peaked ~33%
!cr Ihe 4 KB blcck size, and declined Ic ~5% !cr Ihe larger blcck sizes. Nc di!!erence was cbserved !cr randcm vs. sequenIial access
paIIerns. This is aIIribuIable Ic Ihe large Flash Cache in Ihe NeIApp sysIem.
57
F|gure 99
F|gure 100
5B
F|gure 101
F|gure 10Z
ThrcughpuI peaked aI ~1900 NB/sec in Ihe read/wriIe 50% IesIs even Ihcugh read and wriIe individually bcIh peaked aI cver 1100
NB/sec. we are ncI sure why read/wriIe is belcw Ihe sum c! read and wriIe individually. we will have Ic dc scme mcre per!crmance
Iuning Ic cpIimize Ihis. Prccesscr uIilizaIicn peaked ~39% !cr Ihe 4 KB blcck size and declined Ic ~9% !cr Ihe larger blcck sizes. Nc
di!!erence was cbserved !cr randcm vs. sequenIial access paIIerns. This is aIIribuIable Ic Ihe large Flash Cache in Ihe NeIApp sysIem.
FCcE FuncI|cna||Iy & Perfcrmance Pesu|Is
The FCcE naIive 0S iniIiaIcr running cver a 10 CbE endIcend scluIicn was !ully !uncIicnal and per!crmed very well.
59
F|gure 103
F|gure 104
ThrcughpuI apprcached Ihe 10 Cbps line raIe !cr Ihe 100% read IesIs aI blcck sizes c! B KB and abcve. Prccesscr uIilizaIicn peaked
~37% !cr Ihe 4 KB blcck size, and declined Ic ~Z% !cr Ihe larger blcck sizes. Nc di!!erence was cbserved !cr randcm vs. sequenIial
access paIIerns. This is aIIribuIable Ic Ihe large Flash Cache in Ihe NeIApp sysIem.
60
F|gure 105
F|gure 106
ThrcughpuI apprcached line raIe !cr Ihe 100% wriIe IesIs aI blcck sizes c! 3Z KB and abcve. Prccesscr uIilizaIicn peaked ~ZZ%
!cr Ihe 4 KB blcck size, and declined Ic ~Z% !cr Ihe larger blcck sizes. Nc di!!erence was cbserved !cr randcm vs. sequenIial access
paIIerns. This is aIIribuIable Ic Ihe large Flash Cache in Ihe NeIApp sysIem.
61
F|gure 107
F|gure 108
ThrcughpuI peaked aI ~ZZ00 NB/sec in Ihe read/wriIe 50% IesIs. This resulI is nearly Ihe sum c! cur individual read and wriIe IesI
resulIs, which is beIIer Ihan Ihe iSCSl IesI. Prccesscr uIilizaIicn peaked ~30% !cr Ihe 4 KB blcck size, and declined Ic ~4% !cr Ihe
larger blcck sizes. Nc di!!erence was cbserved !cr randcm vs. sequenIial access paIIerns. This is aIIribuIable Ic Ihe large Flash Cache
in Ihe NeIApp sysIem.
Ana|ys|s
BcIh Ihe iSCSl and FCcE naIive 0S iniIiaIcrs running cver a 10 CbE endIcend scluIicn were !ully !uncIicnal and per!crmed very well.
ThrcughpuI apprcached Ihe 10 Cbps line raIe !cr Ihe 100% read IesIs aI blcck sizes c! B KB and abcve.
ThrcughpuI apprcached line raIe !cr Ihe 100% wriIe IesIs aI blcck sizes c! 3Z KB and abcve.
ThrcughpuI peaked aI ~ZZ00 NB/sec in Ihe read/wriIe 50% IesIs !cr FCcE, buI cnly ~1900 NB/sec !cr iSCSl. This may be aIIribuIable
Ic Ihe TCP/lP sIack cverhead and pcssibly redecIs an imper!ecIly Iuned sysIem. we ccnIinue Ic wcrk cn imprcvemenI c! Ihis resulI
ncw.
Prccesscr uIilizaIicn cn Ihe server was scmewhaI higher !cr Ihe iSCSl cases. This may be aIIribuIable Ic Ihe TCP/lP sIack cverhead
and pcssibly redecIs an imper!ecIly Iuned sysIem.
Nc di!!erence was cbserved !cr randcm vs. sequenIial access paIIerns. This is aIIribuIable Ic Ihe large Flash Cache in Ihe NeIApp
sysIem.
0ne key !acIcr in Ihe success c! naIive 0S iniIiaIcr scluIicns !cr unied neIwcrking is Ihe subsIanIially increased prccessing
per!crmance c! Ihe lnIel' Xecn' prccesscr 5500 & 5600 series prcducI lines available !cr Ihe lasI !ew years. AncIher key !acIcr is
Ihe per!crmance cpIimizaIicn in Ihe laIesI lnIel X5Z0 series c! NlCs. These devices prcvide hardware suppcrI Ic spread Ihe neIwcrk
prccessing lcad acrcss Ihe many ccres c! mcdern server plaI!crms. They alsc have prcIcccl specic hardware cpIimizaIicns !cr bcIh
iSCSl and FCcE.
Because Ihe IesI bed was available Ic us !cr cnly a !ew weeks, we cculd ncI dc all c! Ihe IesIs and per!crmance Iuning IhaI we wculd
have liked Ic dc. Hcwever, we have demcnsIraIed near 10 Cbps line speed per!crmance !cr bcIh iSCSl and FCcE.
6Z
F|gure 109
seIup dialcgues !cund in Ihe NeIApp
EnvircnmenI SeIup secIicn c! Ihis
dccumenI.
iSCSl inIer!ace managemenI.
> iscsi interface disable
e2b
> iscsi interface enable e0c
AggregaIe creaIicn.
> aggr create -t EPIaggr2
raid_dp 3
Vclume creaIicn.
> vol create EPIVol01 EPI-
Aggr2 42M
lgrcup creaIicn and pcpulaIicn (FCcE)
> igroup create -f -t win-
dows EPI-FCoE-Initiator-
Group
> igroup add EPI-
FCoE-Initiator-Group
20:64:00:1b:21:56f:c7:db
lgrcup creaIicn and pcpulaIicn (iSCSl)
> igroup create -I -t win-
dows EPI-iSCSI-Initiator-
Group
> igroup add EPI-iSCSI-Ini-
tiator-Group iqn.1991-05.
com.microsoft:netappepi01
LUN creaIicn and mapping (FCcE)
> lun create -s 20G -t win-
dows-2008 -o noreserve /vol/
vol7/netappepi01_fcoe
> lun map /vol/vol7/netap-
pepi01_fcoe EPI-FCoE-Initia-
tor-Group 0
LUN creaIicn and mapping (iSCSl)
> lun create -s 20G -t win-
dows-2008 -o noreserve /vol/
vol7/netappepi02-iSCSI
> lun map /vol/vol7/netappe-
pi01-iSCSI EPI-iSCSI-Initia-
tor-Group 0
NexI SIes
Fcllcwing Ihis research, nexI sIeps
include.
0pIimizaIicn c! Iuning in iSCSl cases Ic
maximize per!crmance
Ccn!iguraIicn c! NFS & ClFS prcIcccls.
IesIing & ccmpariscn wiIh iSCSl & FCcE
resulIs
ExIensicn c! IesIing Ic cIher 0S & VNN
envircnmenIs beycnd windcws Z00B
ExIensicn c! IesIing Ic mulIipaIh Icpcl
cgies Ic !ind Ihe maximum per!crmance
pcssible
Ccnc|us|cn
The simpliciIy, dexibiliIy, and !amiliariIy
c! Ihe 10 CigabiI EIherneI make iI Ihe
ideal !abric !cr a unied daIa cenIer
neIwcrk !abric. lI enables new ccmpuIe
deplcymenI mcdels, including IhaI c!
clcud ccmpuIing, which will deliver mcre
inIelligenI, respcnsive daIa cenIers, and
greaIer business agiliIy. Thrcugh Iheir
prcducI and indusIry leadership, lnIel and
NeIApp help lT crganizaIicns IransiIicn
Ic 10 CbE and build nexIgeneraIicn daIa
cenIer in!rasIrucIures.
Users have a chcice c! several viable
cpIicns !cr unied neIwcrking prcIcccls
running cn 10 CbE. All c! Ihese chcices run
very well wiIh naIive 0S iniIiaIcrs running
cn sIandard 10 CbE NlCs. The use c!
naIive 0S iniIiaIcr scluIicns and sIandard
NlCs !cr unied neIwcrking delivers
ccsIe!!ecIive scluIicns, ubiquiIcus
deplcymenI, and dexible chcices c!
prcIcccls.
we enccurage Ihe reader Ic explcre 10
CbE unied neIwcrking in Iheir cwn labs
Ic deIermine Ihe besI chcices !cr Iheir
cwn applicaIicns and envircnmenIs.
Add|I|cna| InfcrmaI|cn
1. lnIel' EIherneI X5Z0 Server
AdapIers. hIIp.//www.inIel.ccm/
PrcducIs/Server/AdapIers/X5Z0/
eIherneIX5Z0cverview.hIm
Z. lnIel' Xecn' Prccesscr Family. hIIp.//
www.inIel.ccm/iIcenIer/prcducIs/
xecn/index.hIm
3. lnIel' Server SysIem SRZ600UR.
hIIp.//www.inIel.ccm/prcducIs/
server/sysIems/srZ600ur/srZ600ur
cverview.hIm
4. NeIApp Snap0rive* admin guide.
hIIp.//ncw.neIapp.ccm/kncwledge/
dccs/snapdrive/relsnap63/pd!s/
admin.pd!
5. NeIApp RAl00P*. hIIp.//www.
neIapp.ccm/us/prcducIs/plaI!crmcs/
raiddp.hIml , hIIp.//www.neIapp.
ccm/us/library/IechnicalrepcrIs/Ir
3Z9B.hIml
6. NeIApp Flash Cache*. hIIp.//
www.neIapp.ccm/us/prcducIs/
sIcragesysIems/!lashcache/
7. 0aIa CenIer Bridging. hIIp.//www.
ieeeB0Z.crg/1/pages/dcbridges.hIml
B. Fibre Channel cver EIherneI. hIIp.//
en.wikipedia.crg/wiki/FCcE
9. 0pen FCcE. hIIp.//cpen!cce.crg/
10. lnIerneI Small CcmpuIer SysIem
lnIer!ace. hIIp.//en.wikipedia.crg/
wiki/lscsi
Aend|x A: NeIA Ccmmand L|ne
SeIu
Fcr CLl ccmmand re!erence see hIIp.//
ncw.neIapp.ccm/N0w/kncwledge/dccs/
cnIap/relB0/pd!s/cnIap/cmdre!1.pd! and
hIIp.//ncw.neIapp.ccm/N0w/kncwledge/
dccs/cnIap/relB0/pd!s/cnIap/cmdre!Z.
pd!. Belcw are Ihe 0aIa 0NTAP ccmmand
line equivalenIs Ic Ihe FilerViewbased
63
Aend|x : NeIA |umbc Frame Ccnf|rmaI|cn Ihrcugh UI
1. SelecI Nanage lnIer!aces under NeIwcrk !rcm Ihe le!I menu c! FilerView ` SelecI Ncdi!y Ic view cr change parameIers c! Ihe
inIer!ace.
Z. lnIer!ace e3a shcws IhaI jumbc !rames are cn.
64
F|gure 110
F|gure 111
3. Tc view all neIwcrk inIer!ace parameIer seIIings, selecI Shcw All lnIer!ace 0eIails !rcm Ihe Nanage NeIwcrk lnIer!ace windcw.
65
F|gure 11Z
F|gure 113
5. l0C ww SIcrage SysIems Tracker,
0ecember Z009
6. NulIiple CcnnecIicns per Sessicn
(NCS) suppcrI is dened in Ihe iSCSl RFC
Ic allcw mulIiple TCP/lP ccnnecIicns
!rcm Ihe iniIiaIcr Ic Ihe IargeI !cr Ihe
same iSCSl sessicn. This is iSCSl prcIcccl
specic. This allcws l/0 Ic be senI cver
eiIher TCP/lP ccnnecIicn Ic Ihe IargeI. l!
cne ccnnecIicn !ails, ancIher ccnnecIicn
can ccnIinue prccessing l/0 wiIhcuI
inIerrupIing Ihe applicaIicn. NcIe IhaI ncI
all iSCSl IargeIs suppcrI NCS.
EndncIes
1. Scurce. lnIel esIimaIes as c! |anuary
Z010. Per!crmance ccmpariscn using
SPECjbb*Z005 bcps (business cperaIicns
per seccnd). ResulIs have been esIimaIed
based cn inIernal lnIel analysis and are
prcvided !cr in!crmaIicnal purpcses cnly.
Any di!!erence in sysIem hardware cr
sc!Iware design cr ccnguraIicn may
a!!ecI acIual per!crmance.
Z. l0C, Server wcrklcads FcrecasI, Z009.
3. l0C, The lnIerneI Reaches LaIe
Adclescence, 0ecember Z009.
4. Bx NeIwcrk. B00 IerabyIes per seccnd
c! lP Ira!c esIimaIed cn inIernal lnIel
analysis NeIwcrk Supply/0emand
Z010Z0Z0 !crecasI, 16x SIcrage. 60
exabyIes c! daIa sIcred !rcm Barclays
CapiIal SIcrage BiIs SepIember Z009,
exIrapclaIicn by lnIel !cr Z015, Z0x
CcmpuIe. lnIel inIernal lcngrange
planning !crecasI.
66
Tc learn mcre abcuI deplcymenI c! clcud scluIicns,
visiI www.inIel.ccm/clcudbuilders
D|sc|a|mers
A lrle| processor ruroers are rol a reasure ol perlorrarce. Processor ruroers d|llererl|ale lealures W|lr|r eacr processor lar||y, rol across d|llererl processor lar|||es. 3ee WWW.|rle|.cor/producls/processor_ruroer lor dela||s.
lNF0RVATl0N lN Tll3 00CuVENT l3 PR0vl0E0 lN C0NNECTl0N wlTl lNTEL PR00uCT3. N0 LlCEN3E, EXPRE33 0R lVPLlE0, 8Y E3T0PPEL 0R 0TlERwl3E, T0 ANY lNTELLECTuAL PR0PERTY Rl0lT3 l3 0RANTE0
8Y Tll3 00CuVENT. EXCEPT A3 PR0vl0E0 lN lNTEL'3 TERV3 AN0 C0N0lTl0N3 0F 3ALE F0R 3uCl PR00uCT3, lNTEL A33uVE3 N0 LlA8lLlTY wlAT30EvER, AN0 lNTEL 0l3CLAlV3 ANY EXPRE33 0R lVPLlE0
wARRANTY, RELATlN0 T0 3ALE AN0/0R u3E 0F lNTEL PR00uCT3 lNCLu0lN0 LlA8lLlTY 0R wARRANTlE3 RELATlN0 T0 FlTNE33 F0R A PARTlCuLAR PuRP03E, VERClANTA8lLlTY, 0R lNFRlN0EVENT 0F ANY PAT-
ENT, C0PYRl0lT 0R 0TlER lNTELLECTuAL PR0PERTY Rl0lT. uNLE33 0TlERwl3E A0REE0 lN wRlTlN0 8Y lNTEL, TlE lNTEL PR00uCT3 ARE N0T 0E3l0NE0 N0R lNTEN0E0 F0R ANY APPLlCATl0N lN wllCl TlE
FAlLuRE 0F TlE lNTEL PR00uCT C0uL0 CREATE A 3lTuATl0N wlERE PER30NAL lNJuRY 0R 0EATl VAY 0CCuR.
lrle| ray ra|e crarges lo spec|lcal|ors ard producl descr|pl|ors al ary l|re, W|lroul rol|ce. 0es|grers rusl rol re|y or lre aoserce or craracler|sl|cs ol ary lealures or |rslrucl|ors rar|ed 'reserved or 'urdelred. lrle| reserves lrese lor
lulure delr|l|or ard sra|| rave ro respors|o|||ly Wralsoever lor corl|cls or |rcorpal|o|||l|es ar|s|rg lror lulure crarges lo lrer. Tre |rlorral|or rere |s suojecl lo crarge W|lroul rol|ce. 0o rol lra||ze a des|gr W|lr lr|s |rlorral|or.
Tre producls descr|oed |r lr|s docurerl ray corla|r des|gr delecls or errors |roWr as errala Wr|cr ray cause lre producl lo dev|ale lror puo||sred spec|lcal|ors. Currerl craracler|zed errala are ava||ao|e or requesl. Corlacl your |oca|
lrle| sa|es ollce or your d|slr|oulor lo oola|r lre |alesl spec|lcal|ors ard oelore p|ac|rg your producl order. Cop|es ol docurerls Wr|cr rave ar order ruroer ard are relererced |r lr|s docurerl, or olrer lrle| ||leralure, ray oe oola|red oy
ca|||rg 1-800-518-1Z25, or oy v|s|l|rg lrle|'s weo s|le al WWW.|rle|.cor.
Copyr|grl @ 2010 lrle| Corporal|or. A|| r|grls reserved. lrle|, lre lrle| |ogo, lrle| Xeor, lrle| Xeor |rs|de, lrle| Ce|eror ard lrle| Ce|eror |rs|de are lraderar|s ol lrle| Corporal|or |r
lre u.3. ard olrer courlr|es.
Copyr|grl @ 2010 NelApp, lrc. NelApp, lre NelApp |ogo, 0o lurlrer, lasler, 0ala 0NTAP, F||erv|eW, F|exvo|, RAl0-0P, 3rap0r|ve, ard 3rapsrol are lraderar|s or reg|slered
lraderar|s ol NelApp, lrc. |r lre ur|led 3lales ard/or olrer courlr|es.
0lrer rares ard orards ray oe c|a|red as lre properly ol olrers.

Pr|rled |r u3A 1010/8A/PRw/P0F P|ease Recyc|e 321131-002 u3

Icb Ra Cloud Computing Unified Storage NetApp

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Icb Ra Cloud Computing Unified Storage NetApp

Transféré par

Droits d'auteur :

Formats disponibles

NeIApp* Unied NeIwcrking and SIcrage.

10 CbE FCcE and iSCSl

Vous aimerez peut-être aussi