Vous êtes sur la page 1sur 316

CCIE Data Center Techtorial

Mubasher Nawaz Exam Program Manager


Robert Burns Technical Leader Services
Mike Brown Technical Leader Services
Peppe Monterosso Technical Leader Services
TECCCIE-3644
Session Abstract
If you're starting the journey to gain your CCIE Data Center number, this session
is for you!
It introduces the new curriculum of the CCIE Data Center Program and
highlights the changes the exams (written and lab).
The main objective of this session is to provide candidates with clear
expectations of what to expect with the new exams.
There is no prerequisite to this session.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Disclaimer
Not all topics discussed today
appear on every exam
Due to time restraints, we are
unable to discuss every feature and
topic described in the exam
blueprint

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Timing for the Day

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Agenda

Introduction
L2/L3 Technologies
DC Storage and Compute
CCIE Next Gen
Automation and Orchestration
Fabric Infrastructure
Study, Preparation and Closing
Cisco Certified Internetwork Expert
(CCIE)
Program Overview
Cisco Certifications

https://learningnetwork.cisco.com/community/certifications

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
CCIEs Worldwide
Most highly respected IT certification
for more than 20+ years!
Industry standard
validating expert skills and experience
Demonstrate strong commitment and investment
to networking career, life-long learning, and
dedication to remaining an active CCIE

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Expert Level Tracks

Emphasizes network Collaboration, Unified Datacenters Networking across LAN


design principles and Communications, or infrastructure, storage, and WAN interfaces and
theory at the Voice and Video compute and variety or routers and
infrastructure level Network for design, virtualization switches
implementation and
troubleshooting

VPN solutions and IP fundamentals and Wireless networking


security for Layer 2 and technologies in building with solid understanding
Layer 3 network an extensible service of WLAN technologies
infrastructure, provider network from Cisco
application protocols
and OS

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
CCIE / CCDE Certification Process

CCIE /
Written Exam Practical Exam CCDE

First attempt
within 18
Pearson Location months Select Cisco Locations
2 hours 8 hours lab
Configurations
90-110 questions
Troubleshooting
No documentation
Cisco Documentation
Immediately scored
Score within 48 hours

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Proactive and Holistic Candidate Feedback
Input Feedback
Candidate Exam and Item
Cisco Business Units Comments
Cisco Technology groups Candidate Satisfaction Surveys
Cisco Technical Support teams Create or Refresh
Customer Service Cases
(TAC, AS, ..) Exam Content
EAG (Exam Advisory Groups)
Cisco-Internal and Cisco-External
Subject Matter Experts Cisco Learning Network
Customer Advisory Boards Blogs
Customer Focus Groups
Customer and Cisco field surveys
(Marketing) Launch
Cisco Product Manager, Marketing Exam
Manager, Program Manager

Exams
Exam Live

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Performance Assessment
Validity
Reliability
Fairness
Congruency
Relevancy

Intended use of the test scores


Definition of Minimally Qualified Candidate

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
CCIE Data Center
Written Exam Overview v1.0
Available till 22nd July 2016
CCIE Data Center Written Exam (350-080) version
1.0 Curriculum Overview
# Topic % in exam

1.0 Cisco Data Center Architecture 10%

2.0 Cisco Data Center Infrastructure-Cisco NX-OS 20%

3.0 Cisco Storage Networking 15%

4.0 Cisco Data Center Virtualization 20%

5.0 Cisco Unified Computing System 30%

6.0 Cisco Application Networking Services 5%

Full blueprint available on the Cisco Learning Network:


https://learningnetwork.cisco.com/docs/DOC-13984

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Step 1: CCIE DC Written Exam:
Available worldwide at any Pearson VUE testing facility. Costs may vary due to
exchange rates and local taxes (VAT, GST)
Two-hour exam with 90-110 multiple-choice questions
Closed book; no outside reference materials allowed
Pass/Fail results are available immediately following the exam;
the passing score is set by statistical analysis and is subject to periodic change
Candidates who pass a CCIE written exam must wait a minimum of 180 days
before taking the same number exam
From passing written, candidate must take first lab exam attempt within 18
months
No skip-question functionality
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Written Exam Objective
Candidates who fail any CCIE or CCDE written exam must wait for a period of
15 calendar days, beginning the day after the failed attempt, before retaking the
same exam.(Effective August 2nd,2014)
http://www.cisco.com/web/learning/exams/policies.html#~Written,
The goal of the DC written exam is to test concepts and theoretical knowledge of
Cisco Data Center Technologies in the blue print
Awareness of industry standard best practices, standard bodies, policy
frameworks, and common RFC/BCPs
Lays foundation for Data Center lab exam
http://www.cisco.com/c/en/us/training-events/training-
certifications/exams/policies.html#~written

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Written Exam: Sample Question 1 MC-SA
Q. What it is the best description of the FCoE Initiation Protocol
FIP function?

A. It is required to establish the point-to-point FCoE links with the first


switch in the path
B. It is required to establish the point-to-point FCoE links with any
switch across multiple Ethernet segment
C. It is not used to build the FCoE links
D. It is used to ensure lossless transport

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Written Exam: Sample Question 2 MC-MA
Q: Which of the following two server characteristics cannot be
configured via the UCS service profile? (Choose 2)

A. The number of vNICs and vHBAs to present to the OS


B. The server boot order
C. The amount of CPU and memory to present to the OS.
D. The server BIOS settings
E. The operating system to install

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Written Exam: Sample Question 3 Exhibit
Nexus5k-B# show run Nexus5k-A Nexus5k-B
interface Ethernet1/17
switchport mode trunk
channel-group 17 mode active
Eth 1/17 Eth 1/17
interface port-channel17
switchport mode trunk Port-Channel 1
vpc 17

Nexus5k-B# show vpc 17


vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
17 Po17 up success 100-200
Nexus 1000v
VEM
Q: Which of the following port-channel modes is appropriate for this
topology?

A. lacp port-channel
B. vPC-HM with manual subgroups
C. static port-channel
D. vPC-HM mac-pinning

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Drag and Drop
Drag and drop from the column on the left to the column on the right the correct
color scheme in right order for the UCS DIMM bank color.

Black DIMM Bank COLOR

White

Blue

Yellow

Red

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Drag and Drop
Drag and drop from the column on the left to the column on the right the correct
color scheme in right order for the UCS DIMM bank color.

Black DIMM Bank COLOR

White

Blue

Yellow

Red

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
CCIE Data Center
Lab Exam Overview v1.0
CCIE Data Center Lab Exam version 1.0 Curriculum
Overview
# Topic % in exam

1.0 Cisco Data Center Infrastructure-Cisco NX-OS 30%

2.0 Cisco Storage Networking 20%

3.0 Cisco Data Center Virtualization 10%

4.0 Cisco Unified Computing System 30%

5.0 Cisco Application Networking Services 10%

Full blueprint available on the Cisco Learning Network:


https://learningnetwork.cisco.com/docs/DOC-13992

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
CCIE Data Center Lab Exam
Candidates build a data center configuration based on supplied specifications
Eight-hour exam requires working configurations and troubleshooting to
demonstrate expertise
Must achieve a pass mark scored from several sections that cover configuration
and troubleshooting as per lab exam blueprint
The point values for each question are shown on the exam
Some questions depend upon completion of previous parts of the network
https://learningnetwork.cisco.com/community/certifications/ccie_data_center
http://www.cisco.com/c/en/us/training-events/training-
certifications/exams/policies.html#~lab

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
CCIE DC Mobile Labs and Locations

Brussels
Dubai
Bangalore
Hong Kong
Beijing
Tokyo
Sydney
San Jose
RTP

https://learningnetwork.cisco.com/docs/DOC-3224
https://learningnetwork.cisco.com/community/certifications/ccie_data_center/lab_exam?tab=take-your-lab-exam

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Data Center Lab Exam:
Equipment and Software Versions
The lab exam tests any feature that can be configured on the equipment and the NXOS versions
indicated below. Occasionally, you may see more recent NXOS versions installed in the lab, but you
will not be tested on the new features of a release unless indicated below.
MDS 9222i NXOS v6.x on Nexus 7000 Switches
Nexus 7009 NXOS v5.x on Nexus 5000 Switches
Nexus 5548 NXOS v4.2.x on Nexus 1000v
Nexus 2224 / 2232 NXOS v5.x on MDS 9222i Switches
Nexus 1000v UCS Software release 2.x for UCS-6248 Fabric
UCS C200 Series Server Interconnect
UCS-6248 Fabric Interconnects Software Release A5(1.x) for ACE 4710
UCS-5108 Blade Chassis (B200) Cisco Data Center Manager software v5.x
Cisco Application Control Engine Appliance -
ACE4710
Dual attached JBODs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
CCIE DC Lab Exam:
Pre-Configuration

The Routers and Switches in Your Topology Are Preconfigured With:

Basic IP addressing, hostname, passwords


Please read all instructions carefully

Do NOT change any pre-configuration on any devices unless


explicitly stated in a question

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
CCIE Lab Exam:
Grading
Proctors grade all lab exams
Automatic tools aid proctors with simple grading tasks
Automatic tools are never solely responsible for lab exam gradingproctors are
Proctors complete grading of the exam and submits the final score within 48
hours
No partial credit awarded on questions
Points are awarded for working solutions only
Some questions have multiple solutions

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
CCIE Data Center
V2.0 Update
CCIE Data Center v2.0 Curriculum Overview
Certification process unchanged
Exam curriculum and format changed (July 2016)
Designed and validated with industry experts
(Cisco internals and externals)
Aligned with evolution of job role and relevant technologies

Check the official information on CLN


https://learningnetwork.cisco.com/community/certifications/ccie_data_center

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
CCIE Data Center v2.0 (Unified Blueprint)
% in Written exam
# Topic % in Lab Exam
(400-151)

1.0 Cisco Data Center L2/L3 Technologies 24% 27%

2.0 Cisco Data Center Network Services 12% 13%

3.0 Data Center Storage Networking and Compute 23% 26%

4.0 Data Center Automation and Orchestration 13% 14%

5.0 Data Center Fabric Infrastructure 18% 20%

6.0 Evolving Technologies 10% N/A

Full blueprint available on the Cisco Learning Network

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
CCIE DCv2 Certification Process

Written Exam Practical Exam


pass pass CCIE
400-151
DIAG CFG/TS
120min 60min 420 min

1. Diagnostic
2.Configuration/Troubleshoot

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
CCIE Data Center v2.0 Written Exam
New Number: 400-151
120 minutes, 90 110 independent items
MC-SA/MA; DnD; Point & Click
English only
Pearson VUE
Closed-book
Score directly available

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
CCIE Data Center v2.0 Lab exam
480 minutes, multiple exam modules
Configure, Troubleshoot scenarios to given specifications
English only
Cisco Lab locations including mobile labs.
Cisco Documentation
Score available usually within 48h

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
CCIE Data Center v2.0 vs v1.0 Equipment List
CCIE Data Center v1 .0 CCIE Data Center v2 .0

MDS 9222i APIC Cluster


Nexus7009 Nexus 9336 ACI Spine
o (1) Sup Nexus 9372
o (1) 32 Port 10Gb (F1 Module) Nexus 7004
o (1) 32 Port 10Gb (M1 Module) o (1) Sup2E
Nexus5548 o (2) 48 Port 10Gb (F3 Module)
Nexus2232 Nexus 5672
Nexus 1000v Nexus 2348
UCS C200 Series Server Nexus 1000v
o vic card for c-series UCS C220 M4 Series Rack Server
UCS-6248 Fabric Interconnects o VIC card for C-Series
UCS-5108 Blade Chassis UCS-6248 Fabric Interconnects
o B-200 Series Blades UCS-5108 B-Series Chassis
o Palo mezzanine card o B-200 M4 Series Blades
o Emulex mezzanine card o Palo mezzanine card
Cisco Application Control Engine Appliance o VIC 1340 Card for B-Series
- ACE4710 Dual attached JBODs
Dual attached JBODs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
CCIE Data Center v2.0 Lab Exam Format
New DIAG module
Existing Configuration and Troubleshooting Module
Overall cut-score AND per-module minimum score

Web-based delivery

DIAG Configuration and Troubleshooting


(1 h) (7 h)

No Device Devices

minScore minScore

Cut Score

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
CCIE Data Center v2.0 New Diagnostic Module
Web-based delivery

DIAG Configuration and Troubleshooting


(1 h) (7 h)

No Device Devices

minScore minScore

Cut Score

Assessing new skills


Analyzing, correlating and discerning multiple sources of documentation

Support ticket scenario


Fixed 60 minutes, 100% Web-based, no device needed
Deterministic grading
Item format similar to multiple choices item

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
CCIE Data Center v2.0 Scoring Logic
Web-based delivery

DIAG Configuration and Troubleshooting


(1 h) (7 h)

No Device Devices

minScore minScore

Cut Score

2 required conditions to PASS:


#1: MUST meet or exceed each modules minScore
#2: MUST meet or exceed the Labs TOTAL cutScore

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Example#1: FAIL+ PASS = FAIL!
Module Total score Min score Cut score Candidate 1 Score PASS/FAIL

DIAG 10 4 7 DIAG 2 FAIL

CFG/TS 90 40 68 CFG/TS 78 PASS

100 75 LAB 80 FAIL

#1: DIDNt meet or exceed each modules minScore


#2: met or exceeded the Labs TOTAL cutScore

Strong in both CFG/TS but very weak in DIAG.


!This is just an illustration! Actual values vary per exam questionnaire!

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Example#1: FAIL+ PASS = FAIL!
Module Total score Min score Cut score Candidate 1 Score PASS/FAIL

DIAG 10 4 7 DIAG 5 PASS

CFG/TS 90 40 68 CFG/TS 45 PASS

100 75 LAB 50 FAIL

#1: met or exceeded each modules minScore


#2: DIDNT meet or exceed the Labs TOTAL cutScore

Passed all modules minScore, but total < cutScore!

!This is just an illustration! Actual values vary per exam questionnaire!

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Example#1: PASS + PASS = PASS!
Module Total score Min score Cut score Candidate 1 Score PASS/FAIL

DIAG 10 4 7 DIAG 6 PASS

CFG/TS 90 40 68 CFG/TS 72 PASS

100 75 LAB 78 PASS

#1: met or exceeded each modules minScore


#2: met or exceeded the Labs TOTAL cutScore

Compensated a weakness in Diag with strength in CFG!

!This is just an illustration! Actual values vary per exam questionnaire!

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
CCIE Data Center v2.0 Lab Summary

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Demo Diagnostic Section
Agenda

Introduction
L2/L3 Technologies
DC Storage and Compute
CCIE Next Gen
Automation and Orchestration
Fabric Infrastructure
Study, Preparation and Closing
L2/L3 Technologies Agenda

Virtual Port Channels (vPC)


FabricPath
Overlay Transport Virtualization (OTV)
Virtual Extensible LAN (VXLAN)
DEMO
CCIE Data Center
L2/L3 Technologies NXOS
Virtual Port Channels (vPC)
Virtual Port Channel (vPC)
Multi-Chassis EtherChannel (MCEC)
vPC allows a single device to use a port channel across two neighbor switches
(vPC peers)
vPC Peers
Eliminate STP blocked ports & Provide fast convergence upon link/device failure

Supports back-to-back connection of different vPC domains

vPC is a layer 2 only port channel


MCEC
Available on Nexus 3000, 5000/5500 and 7000

Available as of 5.0(3)U2(1) on Nexus 3000, NX-OS 4.1(3)N1 on the Nexus 5000 vPC Peers
& NX-OS 4.1(3) on the Nexus 7000
MCEC

! Enable vpc on the switch


vPC
dc11-5020-1(config)# feature vpc

! Check the feature status


dc11-5020-1(config)# show feature | include vpc
vpc 1 enabled

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Feature Overview
How does vPC help with STP?
Primary Secondary
Root Root
Before vPC
STP blocks redundant uplinks
VLAN based load balancing
Loop Resolution relies on STP
Protocol Failure

With vPC
No blocked uplinks
Lower oversubscription
EtherChannel load balancing (hash)
Loop Free Topology

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
vPC Terminology (1 of 2)
vPC Peer-
keepalive
link
vPC Domain - A pair of vPC switches in a vpc system vPC Domain
vPC peer-link

vPC Peer - A vPC switch, one of a pair

vPC member port - one of a set of ports (port channels) that form a vPC

vPC - the port channel between the vPC peer and the downstream
vPC peer
device
vPC
vPC peer-link - Link used to synchronize state between vPC peer
vPC
vPC
devices, must be 10GE member
member
port
port
vPC peer-keepalive link - The keep-alive link between vPC peer devices
vPC
vPC
member
port

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
vPC Terminology (2 of 2)

vPC VLAN - Any of the VLANs carried over the peer-link and used to
communicate via vPC with a peer device CFS protocol

non-vPC VLAN - Any of the STP VLANs not carried over the peer-link

CFS - Cisco Fabric Services protocol, used for state synchronization and
configuration validation between vPC peer devices

Orphan Port An orphan port is a interface which connects to an orphan Orphan Port
device
Orphan
Orphan Device An orphan device is a device which is on a VPC vlan but Device
only connected to one VPC peer and not to both

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Building a vPC Domain
Configuration Steps
Following steps are needed to build a vPC (Order does Matter!)
Create vPC domain
Establish Peer Keepalive connectivity
Create a Peer link vPC

1 2 3 4
Create vPCs

* Make sure configurations are consistent * 5 6 7 8


vPC

vPC member
vPC
Routed Interface

Host Port

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Building a vPC Domain
Step 1: Create vPC Domain vPC Domain 10
vPC Domain defines the grouping of switches participating in the vPC

Provides for definition of global vPC system parameters

The vPC peer devices use the vPC domain ID to automatically assign a unique
vPC system MAC address

You MUST utilize unique Domain ids for all vPC pairs defined in a contiguous vPC Domain 20
layer 2 domain

! Configure the vPC Domain ID It should be unique within the layer 2 domain
NX-1(config)# feature vpc
NX-1(config)# vpc domain ? vPC System MAC identifies the
<1-1000> Domain id
Logical Switch in the network
NX-1(config)# vpc domain 20 topology
! Check the vPC system MAC address
NX-1# show vpc role
<snip>
vPC system-mac : 00:23:04:ee:be:14

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Virtual Port Channel (vPC) System Mac
LACP neighbour needs to see the same System ID from both vPC peers
The vPC system-mac is used by both vPC peers
NX-1# sh vpc role NX-2# sh vpc role
<snip> <snip>
vPC system-mac : 00:23:04:ee:be:14 vPC system-mac : 00:23:04:ee:be:14
vPC system-priority : 1024 vPC system-priority : 1024
vPC local system-mac : 00:0d:ec:a4:53:3c vPC local system-mac : 00:0d:ec:a4:5f:7c
vPC local role-priority : 1024 vPC local role-priority : 32667

NX-1 NX-2

Remember: local system-mac is


used for regular portchannels

1/33 1/34

dc11-4948-1
dc11-4948-1#sh lacp neighbor
<snip>
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi1/33 SA 32768 0023.04ee.be14 9s 0x0 0x801E 0x4104 0x3D
Gi1/34 SA 32768 0023.04ee.be14 21s 0x0 0x801E 0x104 0x3D

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
vPC peer-keepalive link
Building a vPC Domain vPC PKL
messages
Step 2: Establish vPC Peer-Keepalive (VPC PKL) should NOT be
routed over the
vPC PL !
Definition:
Heartbeat between vPC peers thru L3 connection
Active/Active detection (in case vPC Peer-Link is down)
Non-fatal to the operation of vPC

Packet Structure: NEXUS 7X00 / NEXUS 6000 /


9X000 5X00 / 3X00
UDP message on port 3200, 96 bytes long (32 byte payload),
1- Dedicated link(s) 1- mgmt0 interface
includes: version, time stamp, local and remote IPs, and domain ID
(1GE LC) (along with
Default timers : interval 1 sec / timeout 5sec management traffic)
Recommendations (in order of preference): 2- mgmt0 interface 2- Dedicated link(s)
(along with (1/10GE front panel
management traffic) ports)
NX-1(config-vpc-domain)# peer-keepalive destination 10.4.1.5 source 10.4.1.1 vrf default 3- As last resort, 3 - As last resort,
can be routed over can be routed over
L3 infrastructure L3 infrastructure

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Building a vPC Domain
Step 3: Create vPC Peer-Link (VPC PL) vPC imposes the
rule that the peer link vPC peer-link
Definition: should never be
Standard 802.1Q Trunk which carries CFS (Cisco Fabric Services) messages blocking !
Carries flooded traffic from the vPC peer , STP BPDUs, HSRP Hellos, IGMP
updates, etc.
Always use identical
Requirements: modules on either
Peer-Link member ports must be 10/40/100GE interfaces :
sides of the peer-link PO100

32 port 10GE M1 or 8 port 10GE-X2 M1 modules


24 Port 10GE M2, 6 Port 40GE M2 or 2 Port 100GE Modules
32 port 10GE F1 or 48 port 10GE F2 /F2e fiber or 48 port F2e copper Modules
Any 10GE port on NEXUS 5000/5500 series
The peer link is always forwarding
vPC Peer-link should be a point-to-point connection for any VLAN that is a member !
(No other device between the vPC peers)

Recommendations (strong ones!)


Minimum 2x 10GE ports NX-1(config-vpc-domain)# interface port-channel 100
(on NEXUS 7000 : use 2 separate cards for best resiliency) NX-1(config-if)# vpc peer-link
10GE ports in dedicated mode (for oversubscribed modules)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Building a vPC Domain
Step 4: Create vPC
Definition:
NX-1 NX-2
Port-channel member of a vPC

Requirements
Configuration needs to match other vPC peer member
In case of inconsistency a VLAN or the entire port-channel may be suspended (e.g.
MTU mismatch) vPC 201
Up to 16 active ports between both vPC peers with M series LC.
Up to 32 active ports between both vPC peers with F series LC

vPC
member
NX-1 : NX-2 : port
interface port-channel201 interface port-channel201
switchport mode trunk switchport mode trunk
switchport trunk native vlan 100 switchport trunk native vlan 100
switchport trunk allowed vlan 100-105 switchport trunk allowed vlan 100-105
vpc 201 vpc 201

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
CCIE Data Center
L2/L3 Technologies NXOS
FabricPath
Introduction to FabricPath
Intelligent L2 Domains Evolution
STP -> vPC -> FabricPath Inter-POD Connectivity across L3
LISP VM mobility
Shipping OTV Failure Boundary Preservation
Nexus 7k
IP Cloud

Core
L3

L3 vPC Aggregation

L2
FabricPath
vPC vPC vPC+ Access
L2

Virtual Access

vPC FabricPath
STP+
NIC Teaming
STP Enhancements 16x ECMP
Simplified loop-free trees
Bridge Assurance Low Latency / Lossless
2x Multi-pathing
MAC Scaling

Shipping Nexus Shipping Nexus Shipping


7k/5k 7k/5k Nexus 7k/5k

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
FabricPath: Goal

Switching Routing
Easy Configuration Multi-pathing (ECMP)
Plug & Play Fast Convergence
Provisioning Flexibility Highly Scalable

FabricPath

FabricPath brings Layer 3 routing benefits to flexible Layer


2 bridged Ethernet networks

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
FabricPath: An Ethernet Fabric
Turn the Network into a Fabric

FabricPath

Connect a group of switches using an arbitrary topology


With a simple CLI, aggregate them into a Fabric:

N7K(config)# interface ethernet 1/1


N7K(config-if)# switchport mode fabricpath

No STP inside. An open protocol based on L3 technology provides


Fabric-wide intelligence and ties the elements together.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
FabricPath IS-IS
Replaces STP as control-plane Improves failure detection,
protocol reconvergence, and high availability
Link-state protocol with support for Minimal IS-IS knowledge required
ECMP at Layer 2 no user configuration by default
Exchanges reachability of Switch
IDs and builds forwarding trees
STP STP FabricPath IS-IS
BPDU BPDU

STP
FabricPath

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Why IS-IS?
No IP dependency no need for IP reachability in order to form adjacency
between devices
Easily extensible Using custom TLVs, IS-IS devices can exchange information
about virtually anything
Provides SPF routing Excellent topology building and re-convergence
characteristics

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
FabricPath Encapsulation
16-Byte MAC-in-MAC Header

Classical Ethernet Frame DMAC SMAC 802.1Q Etype Payload CRC

Original CE Frame

Outer Outer FP
Cisco FabricPath DA SA Tag DMAC SMAC 802.1Q Etype Payload
CRC
(new)
Frame (48) (48) (32)

6 bits 1 1 2 bits 1 1 12 bits 8 bits 16 bits 16 bits 10 bits 6 bits

OOO/DL
RSVD
Endnode ID Endnode ID Sub Etype
U/L
I/G

Switch ID Port ID Ftag TTL


(5:0) (7:6) Switch ID 0x8903

Switch ID Unique number identifying each FabricPath switch


Sub-Switch ID Identifies devices/hosts connected via VPC+
Port ID Identifies the destination or source interface
Ftag (Forwarding tag) Unique number identifying topology and/or multidestination
distribution tree
TTL Decremented at each switch hop to prevent frames looping infinitely

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
FabricPath Technical Overview
Plug-n-Play L2 IS-IS Manages Forwarding Topology
New Control Plane
IS-IS assigns addresses to all FabricPath switches automatically
Compute shortest, pair-wise paths
Support equal-cost paths between any FabricPath switch pairs

S10 S20 S30 S40

FabricPath
Routing Table
Switch IF
S10 L1
S20 L2
FabricPath
S30 L3
S40 L4
L1 L2 L3
L4
S200 L1, L2, L3, L4

S400 L1, L2, L3, L4
S100 S200 S300 S400

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
FabricPath Technical Overview
New Data Plane
The association MAC address/Switch ID is maintained at the edge
Traffic is encapsulated across the Fabric
S10 S20 S30 S40
Switch ID space:
Routing decisions
are made based on S300: FabricPath
the FabricPath Routing Table
routing table AB S100 S300
Switch IF
FabricPath (FP)
S100 S200 S300
S100 L1, L2, L3, L4
MAC address space:
Switching based on
MAC address tables
1/1 1/2 S300: CE MAC
Address Table
Classical Ethernet (CE) MAC IF
A B B 1/2

A
S100

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
FabricPath Technical Overview
Terminology

Interface connected to another FabricPath device
Sends/receives traffic with FabricPath header
Does not run spanning tree
Does not perform MAC learning!
Exchanges topology info through L2 ISIS adjacency
FP Core Ports Forwarding based on Switch ID Table
S10 S20 S30 S40

Spine Switch

FabricPath (FP)
S100 S200 S300

Leaf Switch

1/1 1/2

Classical Ethernet (CE)


A B

CE Edge Ports Interface connected to traditional network device


Sends/receives traffic in standard 802.3 Ethernet frame format
Participates in STP domain
Forwarding based on MAC table

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
FabricPath MAC Learning
Unknown Unicast
S10 S20 S30 S40

A B S100 M

FabricPath Lookup B: Hit


S100 S200 S300
Learn source A
Lookup B: Miss
Flood Lookup B: Miss
Dont learn
S100: CE MAC 1/1 S200: CE MAC 1/2 S300: CE MAC
Address Table Address Table Address Table
MAC IF MAC IF MAC IF
A B
A 1/1 B 1/2

A
S100

Classical Ethernet

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
FabricPath MAC Learning
Known Unicast, Conversational Learning
S10 S20 S30 S40

S300: FabricPath
Routing Table
B A S300 S100
Lookup A: Hit Lookup A: Hit Switch IF
Learn source B FabricPath Send to S100

S100 S200 S300
S100 L1, L2, L3, L4

S100: CE MAC 1/1 S200: CE MAC 1/2 S300: CE MAC


Address Table Address Table Address Table
MAC IF MAC IF MAC IF
A B
A 1/1 B 1/2

B
S300
A
S100

Classical Ethernet

Conversational Learning

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
FabricPath and vPC+
MAC flap issue, if FabricPath edge switches are vPC peers
Emulated switch is used to present vPC peers as single switch to FabricPath network
FabricPath network see emulated switch reachable via S200 and S300.
S10 S20 S30 S40

vpc domain 200


fabricpath switch-id 400

FabricPath (FP)
Interface port-channel 1
S100 S200 S300 switchport mode fabricpath

S100: CE MAC 1/1


Address Table S400 Emulated
MAC IF
Classical Ethernet (CE) vPC+ Switch
A 1/1 A
B S400
MAC flap
B

S200 S300

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
FabricPath Configuration
Checklist
Ensure you have Nexus devices that supports FabricPath.
System is running minimum NX-OS 5.1.1 (Nexus 7000) / NX-OS 5.1.3 (Nexus
5500) software release
Obtain and install Enhanced Layer 2 license. You will need to obtain the host
id of the switch show license host-id
Install the license install license <file>
Install FabricPath feature set FabricPath depends on several discrete
processes and functions; ensures all required system plugins loaded into
memory by issuing install feature-set fabricpath

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
FabricPath Configuration
Plug-and-Play
Once FabricPath feature-set installed:
Enable FabricPath feature set
feature-set fabricpath
Define FabricPath VLANs
vlan <range>
mode fabricpath
Identify FabricPath interfaces
interface <name>
switchport mode fabricpath
FabricPath devices will form adjacencies, exchange unicast and multicast
routing information, and begin forwarding traffic
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
CCIE Data Center
L2/L3 Technologies NXOS
Overlay Transport Virtualization
(OTV)
OTV Feature Overview
Data Center Interconnect
Challenges in Traditional Layer 2 VPNs (EoMPLS, VPLS, Dark Fiber)
Flooding Behavior Pseudo-wire Maintenance Multi-Homing

- Unknown Unicast - Full mesh of Pseudo-wire - Requires additional


for MAC propagation is complex Protocols & extends STP
- Unicast Flooding reaches - Head-End replication is - Malfunctions impacts
all sites a common problem multiple sites

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
OTV changes the game
Flooding Based Learning Control-Plane Based Learning
Move to a Control Plane protocol that proactively advertises MAC addresses and their
reachability instead of the current flooding mechanism
Pseudo-wires and Tunnels Dynamic Encapsulation
Not require static tunnel or pseudo-wire configuration
Offer optimal replication of traffic done closer to the destination, which translates into
much more efficient bandwidth utilization in the core
Complex Dual-homing Native Automated Multi-homing
Allow load balancing of flows within a single VLAN across the active devices in the
same site, while preserving the independence of the sites. STP confined within the site
(each site with its own STP Root bridge)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Overlay Transport Virtualization
OTV in a Nutshell
OTV is a MAC-in-IP method that extends Layer 2 connectivity across a transport
network infrastructure
OTV supports both multicast and unicast-only transport networks
OTV uses ISIS as the control protocol
OTV on Nexus7000 does not encrypt encapsulated payload

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Terminology
OTV Edge Device
Performs all OTV functionality Transport Infrastructure*

Usually located at the Aggregation Layer


or at the Core Layer
OTV Edge OTV Edge
Device Device
Support for multiple OTV Edge Devices
(multi-homing) in the same site OTV OTV

L
3
L
2

* It can be owned by the Enterprise


or by the Service Provider

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Terminology
Internal Interfaces
Site facing Interfaces of the Edge Devices Transport Infrastructure

Carry VLANs extended through OTV


Regular Layer 2 interfaces OTV Internal
Interfaces
OTV Internal
Interfaces

No OTV configuration required OTV OTV

L
3
L
2

OTV Internal
= Interface

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Terminology
Join Interface
One of the uplink of the Edge Device Transport Infrastructure

Point-to-point routed interface (physical


interface, sub-interface or port-channel OTV Join OTV Join
supported) Interface Interface

Used to physically join the Overlay OTV OTV

network L
3
L
2
No OTV specific configuration required

= OTV Join Interface

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
Terminology
Overlay Interface
Virtual interface with most of the OTV Transport Infrastructure
configuration
Logical multi-access multicast-capable
Overlay
interface Interface
Overlay
Interface

Encapsulates Layer 2 frames in IP OTV OTV

unicast or multicast L
3
L
2

= Overlay Interface

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
OTV Control Plane
Neighbor Discovery and Adjacency Formation
Before any MAC address can be advertised the OTV Edge Devices must:
Discover each other
Build a neighbor relationship with each other

Neighbor Relationship built over a transport infrastructure:


Multicast-enabled (all shipping releases)
Unicast-only (from NX-OS release 5.2)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
OTV Control Plane
Neighbor Discovery (over Multicast Transport)
Multicast-enable
Transport
OTV OTV
OTV Control Plane
OTV Control Plane

IP A IP B
West East

Mechanism End Result


Edge Devices (EDs) join an Adjacencies are maintained
multicast group in the transport, as over the multicast group
they were hosts (no PIM on EDs) A single update reaches all
OTV hellos and updates are neighbors
encapsulated in the multicast group

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
OTV Control Plane
Building the MAC Tables
No unknown unicast flooding (selective unicast flooding in 6.2)
Control Plane Learning with proactive MAC advertisement
Background process with no specific configuration
IS-IS used between OTV Edge Devices
MAC Addresses
OTV Advertisements OTV

IP A IP B
West East

IP C
OTV

South

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
OTV Control Plane
MAC Advertisements (over Multicast Transport)
Craft OTV
2 update with
new MACs
VLAN MAC IF
100 MAC A IP A
Update A
OTV
Update
100 AMAC B
100 MAC C
IP A
IP A
6
OTV

Update A IP A G Multicast-enabled Update


UpdateAA IP A G

Transport East
West
3 MAC Table
5
MAC Table Encap Decap VLAN MAC IF
100 MAC A IP A
VLAN MAC IF
100 MAC A e1/1
4 101 MAC B IP A
102 MAC C IP A
100
101 MAC B e1/1 Update A IP A G
100
102 MAC C e1/1 Update A IP A G

Add MACs
learned
through OTV
1 Decap
New MACs learned 7
in VLANs that are 5
OTV 7
OTV extended MAC Table
Update
UpdateAA IP A G VLAN MAC IF Add MACs
100 MAC A IP A learned
100
101 MAC B IP A
VLAN
100
MAC IF
MAC A IP A 100
102 MAC C IP A through OTV
6 Update
100
100
MACAB
MAC C
IP A
IP A
South

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
OTV Configuration Multicast Transport
Building a OTV Infrastructure: OTV Join Interface
Valid Join Interface: L3 Port Channel / Physical Routed Interface / Routed Sub-Int
Join Interface and neighboring Core Interface need a little bit of common
configuration
MTU of 1542 is a hard recommendation OTV Edge Device
interface port-channel1
description Join Interface
OTV OTV
ip address 10.10.10.n1-4/30
ip igmp version 3
mtu 1542
!

Core Router
DC
interface port-channel1 DC
West description Core Interface East
ip address 10.10.10.r1-4/30
ip router ospf TAG area 0.0.0.0
Join Interface ip ospf passive-interface
ip pim sparse-mode
Core Interface OTV ip igmp version 3
OTV
mtu 1542
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
OTV Configuration Multicast Transport
Building a OTV Infrastructure: OTV Overlay Interface
Overlay Interface is the heart of OTV
Prepare the different Parameters you require in Advance
OTV Edge Device (VDC)
feature otv
!
otv site-vlan 99
otv site-identifier 1111.1111.1111
OTV ! OTV
interface Overlay100
otv join-interface port-channel1
otv control-group 239.1.1.1
otv data-group 232.192.1.0/24
otv extend-vlan 100-150

DC
DC
West
East

Overlay Interface OTV OTV

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 86
OTV Configuration Unicast Transport
Unicast Transport: Overlay
OTV can run across a unicast only transport (Ideal for a small number of sites)
Unicast Transport requires the configuration of one or more adjacency servers. OTV
devices register with the adjacency server which in turn provides each with an OTV
Neighbor List (oNL).
Think of the adjacency server as a special process running on a generic OTV edge device
A primary and secondary adjacency server can be configured for redundancy
Primary and Secondary Adjacency servers are stateless; every OTV client must register
with both servers
OTV uses graceful exit of Adjacency Servers. If the primary server is rebooted or
reconfigured, it can notify the OTV clients allowing them to immediately use the
secondary.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
OTV Configuration Unicast Transport
Unicast Transport: Primary Adjacency Server Overlay

interface Overlay1
otv join-interface port-channel100
otv extend-vlan 200-209
West
otv adjacency-server unicast-only East
otv otv

otv
Core
otv

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
OTV Configuration Unicast Transport
Unicast Transport: Secondary Adjacency Server Overlay

interface Overlay1 Primary Server


otv join-interface port-channel100
otv extend-vlan 200-209
otv use-adjacency-server 172.16.1.34 unicast-only
West otv adjacency-server unicast-only East
otv otv

otv
Core
otv

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 89
Unicast Transport: Configuration
Unicast Transport: Client Overlay
Primary Server Secondary Server
interface Overlay1
otv join-interface port-channel100
otv extend-vlan 200-209
otv use-adjacency-server 172.16.1.34 172.16.1.26 unicast-only
West East
otv otv

otv
Core
otv

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
OTV Data Plane
Inter-Site Packet Flow

4
MAC TABLE Transport MAC TABLE
VLAN MAC IF
Infrastructure VLAN MAC IF
Decap
100 MAC 1 Eth 2 IP A IP B 100 MAC 1 IP A
3 5
2 100 OTV MAC 2 Eth 1 OTV OTV OTV 6
Encap 100 MAC 2 IP A
Layer MAC 1 MAC 3 IP A IP B Layer 2
100 MAC 3 IP B MAC 1 MAC 3 IP A IP B 100 MAC 3 Eth 3
2 Lookup
100 MAC 4 IP B 100 MAC 4 Eth 4
Looku
p

MAC 1 MAC 3
MAC 1 MAC 3 West East
Server 1 Site Site Server 3
1 7

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 91
For Your
OTV Control Plane Reference
CLI Verification
Establishment of control plane adjacencies between
OTV Edge Devices (multicast or unicast transport):
dc1-agg-7k1# show otv adjacency

Overlay Adjacency database


Overlay-Interface Overlay100 :
Hostname System-ID Dest Addr Up Time Adj-State
dc2-agg-7k1 001b.54c2.efc2 20.11.23.2 15:08:53 UP
dc1-agg-7k2 001b.54c2.e1c3 20.12.23.2 15:43:27 UP
dc2-agg-7k2 001b.54c2.e142 20.22.23.2 14:49:11 UP

Unicast MAC reachability information:


dc1-agg-7k1# show otv route
OTV Unicast MAC Routing Table For Overlay100
VLAN MAC-Address Metric Uptime Owner Next-hop(s)
---- -------------- ------ -------- --------- -----------
2001 0000.0c07.ac01 1 3d15h site Ethernet1/1 Local Site
2001 0000.1641.d70e 1 3d15h site Ethernet1/2 MAC
2001 0000.49f3.88ff 42 2d22h overlay dc2-agg-7k1
2001 0000.49f3.8900 42 2d22h overlay dc2-agg-7k2 Remote Site
MAC

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
OTV Multi-homing
Fully Automated Multi-homing
No additional protocols required (i.e. BGP)
OTV site-vlan used to discover OTV neighbor in the same site
Authoritative Edge Device (AED) Election takes place
Extended VLANs are split across the AEDs
The AED is responsible for: AED OTV OTV AED

MAC address advertisement for its VLANs Site Adjacency


L3
L2
Forwarding its VLANs traffic inside and outside the site

Site Adjacency used for AED election

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
OTV Multi-homing
Terminology: Authoritative Edge Device AED for odd
OTV supports multiple edge devices VLANs
per site
A single OTV device is elected as otv otv
AED on a per-vlan basis
The AED is responsible for
advertising MAC reachability and
forwarding traffic into and out of the
site for its VLANs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 94
OTV Multi-homing
Terminology: Authoritative Edge Device AED for even
OTV supports multiple edge devices VLANs
per site
A single OTV device is elected as otv otv
AED on a per-vlan basis
The AED is responsible for
advertising MAC reachability and
forwarding traffic into and out of the
site for its VLANs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
OTV Multi-homing Core
Terminology: Site VLAN and Site Identifier
Dual Site Adjacency, 5.2(1) and Im AED for Im AED for
Even VLANs Odd VLANs
above
Site Adjacency established across OTV Hello OTV Hello
otv
Site-ID 1.1.1 otv
Site-ID 1.1.1
the site vlan
Overlay Adjacency established via
the Join interface across Layer 3 Full
network Adjacency

OTV Hello OTV Hello


Site-ID 1.1.1 Site-ID 1.1.1

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
OTV Multi-homing Core
Terminology: Site VLAN and Site Identifier
Im now AED
Dual Site Adjacency also has Im not AED Im AED for Im AED for
ALL VLANs
capable Even VLANs Odd VLANs
mechanism for advertising AED
capabilities on local failure to
otv otv
improve convergence
Join interface down
Partial
Adjacency

Im not AED
capable

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
OTV Multi-homing Core
Terminology: Site VLAN and Site Identifier
Im not AED Im now AED
Dual Site Adjacency also has capable
Im AED for Im AED for
ALL VLANs
Even VLANs Odd VLANs
mechanism for advertising AED
capabilities on local failure to Im not AED
otv
capable otv
improve convergence
Join interface down
Partial
Internal Vlans down
Adjacency

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 98
Release 5.2
and above
Hardened Multi-homing
Introducing OTV Site-identifier
Same site devices must use common site-identifier
Site-id information is included in the control plane
Makes OTV multi-homing more robust and resilient
Site Adjacency and Overlay Adjacency are now both leveraged for AED election
An overlay will not come up until a site-id is configured
Site and Overlay Adjacency are both leveraged for AED election

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 99
Configuration
Site VLAN and Site Identifier

otv site-vlan 210 otv site-vlan 210


otv site-identifier 0001.0001.0001 otv site-identifier 0002.0002.0002

West East
otv otv

otv
Core
otv

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
OTV Multi-homing
VLANs Split across AEDs Remote OTV Device
MAC Table
VLAN MAC IF
Automated and deterministic algorithm 100 MAC 1 IP A

101 MAC 2 IP B
In a dual-homed site:
Lower IS-IS System-ID (Ordinal 0) = EVEN VLANs
Higher IS-IS System-ID (Ordinal 1) = ODD VLANs
OTV-a# show otv vlan

OTV Extended VLANs and Edge Device State Information (* - AED)


AED IP A Overlay Adjacency IP B AED
VLAN Auth. Edge Device Vlan State Overlay ODD VLANs OTV OTV EVEN VLANs
---- ------------------ ---------- -------
100 East-b inactive(Non AED) Overlay100
101* East-a active Overlay100
102 East-b inactive(Non AED) Overlay100

Site Adjacency
OTV-b# show otv vlan OTV-a OTV-b

OTV Extended VLANs and Edge Device State Information (* - AED)

VLAN Auth. Edge Device Vlan State Overlay


---- ------------------ ---------- -------
100* East-b active Overlay100
101 East-a inactive(Non AED) Overlay100
102* East-b active Overlay100

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 101
OTV Multi-homing
AED and Broadcast Handling
Broadcast reaches all the Edge Devices within the site
Only the AED forwards the traffic to the Overlay
All the Edge Devices at the other sites receive the broadcast
At the remote sites only the AEDs forward it into the site
OTV

Broadcast OTV Broadcast


stops here stops here

OTV

Bcast
pkt
OTV
Core

AED
AED

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 102
Path Optimization
Egress Routing with LAN Extension
Extended VLANs typically have associated HSRP groups

By default, only one HSRP router elected active, with all servers pointing to HSRP VIP as default gateway
Packet from
Result: sub-optimal routing HSRP Hellos Vlan for
ARP 10 to Vlan 20
DMAC
HSRP VIP = DGW
ARP reply
Routing

HSRP HSRP HSRP HSRP


Active Standby Listen Listen

Packet from
Vlan 10 to Vlan 20
DMAC = Host Vlan 20

VLAN VLAN
20 10

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 103
Egress Routing Localization
FHRP Filtering Solution
Filter FHRP with combination of VACL and MAC route filter
Result: Still have one HSRP group with one VIP, but now have active router at
each site for optimal first-hop routing


HSRP Hellos HSRP Hellos

HSRP Filter
HSRP HSRP HSRP HSRP
Active Standby Active
Listen Listen
Standby

ARP for
HSRP VIP

ARP reply

VLAN VLAN
20 10

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
For Your
Egress Routing Localization Reference
First Hop Redundancy Protocols (FHRP) Isolation
VLAN Access List (VACL) to drop Hellos

! IP ACL's to drop HSRP Hellos,


! Create the VACL
! forward other traffic
vlan access-map HSRP_Localization 10
ip access-list HSRP_IP
match mac address HSRP_VMAC
10 permit udp any 224.0.0.2/32 eq 1985
match ip address HSRP_IP
20 permit udp any 224.0.0.102/32 eq 1985
action drop
ip access-list ALL_IPs
vlan access-map HSRP_Localization 20
10 permit ip any any
match mac address ALL_MACs
! MAC ACL's to drop non-IP HSRP traffic,
match ip address ALL_IPs
! forward other traffic
action forward
mac access-list HSRP_VMAC
10 permit 0000.0c07.ac00 0000.0000.00ff any
! Apply the VACL to each extended vlan
20 permit 0000.0c9f.f000 0000.0000.0fff any
vlan filter HSRP_Localization vlan-list
mac access-list ALL_MACs
<OTV_Extended_VLANs>
10 permit any any

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 105
For Your
Egress Routing Localization Reference
First Hop Redundancy Protocols (FHRP) Isolation
ARP Inspection Filter to drop ARP sourced from the Virtual MAC
(preventing duplicate IP messages between Active Devices at each site)

! Feature dhcp required for ARP inspection


feature dhcp
! Create the ARP access-list to deny traffic from Virtual MAC
arp access-list HSRP_VMAC_ARP
10 deny ip any mac 0000.0c07.ac00 ffff.ffff.ff00
20 deny ip any mac 0000.0c9f.f000 ffff.ffff.f000
30 permit ip any mac any
! Apply ARP ACL to each extended VLAN
ip arp inspection filter HSRP_VMAC_ARP vlan <OTV_Extended_VLANs>

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
For Your
Egress Routing Localization Reference
First Hop Redundancy Protocols (FHRP) Isolation
Apply Route-Map to each Overlay to filter Virtual MAC
(prevents virtual MAC from flapping between sites)

! mac-list to deny advertising virtual MAC


mac-list OTV_HSRP_VMAC_deny seq 10 deny 0000.0c07.ac00 ffff.ffff.ff00
mac-list OTV_HSRP_VMAC_deny seq 11 deny 0000.0c9f.f000 ffff.ffff.f000
mac-list OTV_HSRP_VMAC_deny seq 20 permit 0000.0000.0000 0000.0000.0000
! create the route-map
route-map OTV_HSRP_filter permit 10
match mac-list OTV_HSRP_VMAC_deny
! apply route-map to each overlay
otv-isis default
vpn Overlay1
redistribute filter route-map OTV_HSRP_filter

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 107
CCIE Data Center
L2/L3 Technologies NXOS
Virtual Extensible LAN (VXLAN)
Why Overlays?
Seek well integrated best in class Overlays and Underlays

Robust Underlay/Fabric Flexible Overlay Virtual Network


High Capacity Resilient Fabric Mobility Track end-point attach at edges
Intelligent Packet Handling Scale Reduce core state
Distribute and partition state to network edge
Programmable & Manageable
Flexibility/Programmability
Reduced number of touch points

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 109
Overlay Taxonomy

Overlay Control Plane


Service = Virtual Network Instance (VNI) VTEPs
Identifier = VN Identifier (VNID)
NVE = Network Virtualization Edge Encapsulation
VTEP = VXLAN Tunnel End-Point
Edge Device (NVE) Edge Devices (NVE)

Hosts
Underlay Network (end-points)

Underlay Control Plane

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 110
VXLAN is an Overlay Encapsulation
Data Plane Learning Protocol Learning
Flood and Learn over a multidestination Advertise hosts in a protocol
distribution tree joined by all edge devices amongst edge devices

Overlay Control Plane

Encapsulation

VXLAN
t
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 111
VXLAN Packet Structure
Ethernet in IP with a shim for scalable segmentation

FCS
VXLAN Header Original L2 Frame

Ethernet Payload

Large scale
Src VTEP MAC Address Src and Dst addresses
Allows for 16M segmentation
of the VTEPs UDP 4789 possible segments
Next-Hop MAC Address
Hash of the inner L2/L3/L4
headers of the original frame.
50 Bytes of overhead Enables entropy for ECMP Load Tunnel Entropy
balancing in the Network.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 112
Data Plane Learning
Dedicated Multicast Distribution Tree per VNI
Web DB DB Web
VM VM VM VM
VTEP VTEP VTEP

PIM Join for Multicast PIM Join for Multicast Group


Group 239.1.1.1 239.2.2.2

PIM Join for Multicast PIM Join for Multicast Group


Group 239.1.1.1 239.2.2.2

Multicast-enabled
Transport
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 113
Data Plane Learning
Learning on Broadcast Source - ARP Request Example

ARP Req VM 1 VM 2 VM 3
MAC IP Addr MAC IP Addr
VM 1 VTEP 1 VM 1 VTEP 1

VTEP 1 ARP Req


VTEP 2 ARP Req
VTEP 3
1.1.1.1 2.2.2.2 3.3.3.3

ARP Req IP A G

ARP Req IP A G

Multicast-enabled
Transport

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 114
Data Plane Learning
Learning on Unicast Source - ARP Response Example

VM 1 MAC IP Addr VM 2 MAC IP Addr VM 3


VM 2 VTEP 2 VM 1 VTEP 1

VTEP 1 VTEP 2 ARP Resp VTEP 3


ARP Resp
1.1.1.1 2.2.2.2 3.3.3.3
ARP Resp VTEP 2 VTEP 1

Multicast-enabled
Transport

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 115
Data Plane Learning
Sharing Multicast Groups across VNIs

Web Blue VNI on DB DB Purple VNI on Web


VM Group G VM VM Group G VM

VTEP 1 VTEP 2 VTEP 3


1.1.1.1 2.2.2.2 3.3.3.3

Org Frame IP A G

Org Frame IP A G

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 116
VXLAN Evolution
Head-end replication enables unicast-only mode
Multicast Independent Control Plane provides dynamic VTEP discovery

Workload MAC addresses learnt by VXLAN NVEs


Protocol Learning
Advertise L2/L3 address-to-VTEP association
prevents floods information in a protocol

VXLAN HW Gateways to other encaps/networks


External Connectivity VXLAN HW Gateway redundancy
Enable hybrid overlays

VXLAN Routing
IP Services Distributed IP Gateways

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 117
VXLAN Evolution: Using a Control Protocol
VTEP Discovery
2 BGP consolidates and
propagates VTEP list for VNI
BGP Route
VTEPs advertise their VNI membership in BGP
Reflector
1
1

VTEP VTEP
IP A
IP B
POD1 1 POD2
Overlay Neighbors
POD3 , IP C
POD2 , IP B
3
VTEP obtains list of
VTEP neighbors for
VTEP
each VNI
IP C 4 VTEP can perform
POD3 Head-End Replication

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 118
VXLAN Unicast Mode
Head-end replication
A host sends a L2 BUM* frame *Broadcast, Unknown Unicast or Multicast
1
BUM Frame
5 Frames are unicast to
3 VTEP performs Head- the neighbors
End Replication
IP A IP B
Unicast-Only
BUM Frame
VTEP Transport VTEP
BUM Frame IP A IP C
IP A
IP B
4 VXLAN Encap
POD1 POD2
Overlay Neighbors
POD3 , IP C
POD2 , IP B
2
VTEP retrieves the list
of Overlay Neighbors**
VTEP
**Information statically configured or dynamically retrieved via control plane (VTEP discovery)
IP C
POD3
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 119
VXLAN Evolution: Using a Control Protocol
Protocol Learning: MP-BGP EVPN Control Plane
2 BGP propagates routes for
the host to all other VTEPs
BGP Route
VTEPs advertise host routes (IP+MAC) to
Reflector
local hosts
1

VTEP VTEP
IP A
IP B
POD1 Overlay Forwarding Table POD2
Host1 <MAC,IP> , VTEP IP A

3
VTEPs obtain host
Overlay Forwarding Table routes for remote hosts
Host1 <MAC,IP> , VTEP IP A
and install in RIB/FIB
Host2 <MAC,IP> , VTEP IP B VTEP
3 IP C
POD3

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 120
Evolution of the VXLAN Data Plane
Beyond Ethernet in IP GPE: Generic Protocol Encapsulation

FCS
VXLAN Header Original L2 Frame

Payload:
IP
Ethernet
other

Src VTEP MAC Address Src and Dst addresses


of the VTEPs UDP 4789 Allows for 16M
Next-Hop MAC Address possible segments

Hash of the inner L2/L3/L4


headers of the original frame. 24 bit Protocol
50 Bytes of overhead Enables entropy for ECMP Load Type field
balancing in the Network. (previously reserved)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 121
For Your
MP-BGP EVPN VXLAN Configuration Reference
Building your VTEP (VXLAN Tunnel End-Point)

# Features & Globals Enables VTEP (only required on Leaf or Border)


feature bgp
feature nv overlay
nv overlay evpn
Enables EVPN Control-Plane in BGP
# Spine (S1)

# Leaf (V1) Configure the VTEP interface


interface nve1
RR RR RR RR
source-interface loopback0
host-reachability protocol bgp
Use a Loopback for Source Interface

iBGP
Enable BGP for Host reachabilityV1 V2

*Simplified BGP configuration; would have 4 BGP peers (RR)


IGP not shown
V3

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 122
For Your
MP-BGP EVPN VXLAN Configuration Reference
Building your Overlay Control-Plane
# Features & Globals
feature bgp
feature nv overlay
nv overlay evpn
Enables EVPN Control-Plane in BGP
# Spine (S1)
router bgp 65500
router-id 10.10.10.S1
address-family ipv4 unicast
address-family l2vpn evpn Activate L2VPN EVPN under each BGP neighbor
RR RR RR RR

neighbor 10.10.10.0/24 remote-as 65500


update-source loopback0
address-family l2vpn evpn iBGP
send-community both V2
V1
route-reflector-client

# Leaf (V1)
router bgp 65500 Send Extended BGP Community
router-id 10.10.10.V1 to distribute EVPN route attributes
address-family ipv4 unicast V3
neighbor 10.10.10.S1 remote-as 65500
update-source loopback0 *Simplified BGP configuration; would have 4 BGP peers (RR)
address-family l2vpn evpn IGP not shown
send-community both
*

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 123
Demo
vPC peer-link Goal:
CE link Extend VLAN 700-701 to allow communication between
FP link Host 1 and Host 2
Fabricpath
70.70.70.2 70.70.70.50
Nexus7010 Nexus7010-FP
OTV-West-1 Nexus5k1
vPC
vPC OTV vPC+

Host 1 Host 2
VLAN 700 OTV2
VLAN 700
70.70.70.100 70.70.70.200
vPC OTV-West-2 Nexus5k2
Nexus7018 Nexus7018-FP
70.70.70.2 70.70.70.51

VLAN 700-701

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 125
DEMO
Steps:
Configure vPC
Verify vPC and make sure ping works
Configure OTV
Verify OTV and make sure ping works
Configure Fabricpath and vPC+
Verify Fabricpath and vPC+
Test end to end connectivity

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 126
Agenda

Introduction
L2/L3 Technologies
DC Storage and Compute
CCIE Next Gen
Automation and Orchestration
Fabric Infrastructure
Study, Preparation and Closing
Storage Networking
Agenda

What is a SAN?
Fibre Channel
Fibre Channel over Ethernet
Fibre Channel over IP
Conclusion
Sample Storage Area Networking in the
CCIE Sample Topology

MDS Switches w/
attached Storage

Fibre Channel configuration &


Troubleshooting within the
Nexus N5K & N7K

Fibre Channel
Technologies
SAN Port-Channel &
F-Port Trunking

Host to Storage Zoning


UCS Fabric Interconnect SAN
Interface, Switch Mode or NPV
Mode

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 130
Glossary of Terms
SAN Storage Area Network. A network of switches, typically fibre channel used for carrying SCSI or FICON traffic

FC Fibre Channel. A protocol used to carry SCSI or FICON packets containing IO commands from a server to a storage array

SCSI Small Computer System Interface. A bus based system or protocol used to carry block based storage commands

iSCSI An IP based protocol capable of carrying SCSI commands to and from storage devices

FICON The protocol used to carry mainframe based IO

MDS The Cisco family of datacenter switches capable of carrying fiber channel traffic

VSAN Virtual SANs. A feature capable of creating logical SANs on a physical SAN infrastructure

FCIP Fibre Channel over IP. The protocol used to tunnel fiber channel packets over an IP infrastructure. Used for extending a
Fibre Channel SAN over long distances

ISL Inter Switch Link.

FCoE Fibre Channel over Ethernet. An encapsulation of FC traffic over an enhanced Ethernet topology.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 131
What Is a SAN?
The SCSI I/O Channel - Starting Point
SCSI is a standard that defines an interface
between an initiator (usually a computer) and a Applications
Half-Duplex
target (usually a storage device such as a hard File System I/O Channel
disk)
Block Device
Logical Unit Number (LUN): A 64-bit field within
SCSI Generic
SCSI that identifies the logically addressable unit
within a target SCSI device
TCP/IP Stack
SCSI I/O channel provides half-duplex pipe for

SCSI
NIC Driver Adapter Driver
SCSI command, data, and status
NIC Adapter SCSI Adapter
SCSI I/O channel can be internal or external to
host
Multiple SCSI I/O channels can exist within host SCSI SCSI
Initiator Target

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 132
Storage Area Network (SAN)
Clients
Same SCSI protocol carried over a network transport via
serial implementation
Transport must not jeopardize SCSI payload (security,
integrity, latency) LAN
Two primary transports to choose from today, namely IP and
Servers
Fibre Channel
Fibre Channel provides high-speed transport for SCSI Fibre Channel
payload via Host SAN
Bus Adapter (HBA)
Fibre Channel overcomes many shortcomings of parallel I/O Block
and combines best attributes of a channel and a network Storage
together Devices

Characteristics and requirements of the SCSI protocol and emulating raw block disk to
the OS define the necessary fabric capabilities and design

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 133
Storage Area Networking
Fibre Channel
Fibre Channel Communications Model
Fibre Channel Has Many Similarities to IP (TCP)

Point to point oriented Facilitated through device login


Similar to TCP session establishment

N_Port to N_Port connection Logical node connection point


Similar to TCP/UDP sockets

Flow Controlled Hop-by-hop and End-to-End basis


Similar to TCP flow control Different mechanism (no drops)

Acknowledged For certain classes of traffic, none for others


Similar to TCP / UDP acknowledgement models

Multiple connections allowed per device


Similar to multiple TCP / UDP sockets

Node Node
Transmitter Receiver
N_Port N_Port
Receiver Transmitter
Link
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 135
Fibre Channel Port Types
Fibre Channel Switch
Input Fabric Output
Port X Port

Fabric E_Port
E_Port NPV
Switch F_Port NP_Port
Switch

End
Fabric TE_Port TE_Port F_Port N_Port Node
Switch
F_Port N_Port End
Node

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 136
Fibre Channel Addressing
World Wide Names (WWN)
WWNs are used as burnt-in unique addresses assigned to fabric switches, ports, and nodes by the
manufacturer
Each switch is assigned a WWN at time of manufacture
Each switch port is assigned a WWN at the time of manufacture
Each HBA port is assigned a WWN at the time of manufacture

WWNs are created using a MAC address and a prefix to ensure a globally unique address
These addresses are registered in the fabric and mapped to an FC_ID

Eg. IEEE Extended Name Format


4 Bits 12 Bits 24 Bits 24 Bits

N_Port or IEEE Organizational Unique ID


0010 Locally Assigned Identifier
F_Port Identifier (OUI)
Format
Port Identifier Assigned to Each Vendor Vendor-Unique Assignment
Identifier

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 137
Fibre Channel Addressing
FC_ID Address Model
FC_ID address models help speed up routing

Switches assign FC_ID addresses to N_Ports

Some addresses are reserved for fabric services

Private loop devices only understand 8-bit address (0x0000xx)

TL_Port can provide proxy service for private-to-public address translation

Maximum switch domains = 239 (based on standard)


8 Bits 8 Bits 8 Bits

Switch Topology Model Switch


Area Device
Domain

Arbitrated Loop
Private Loop Device 00 00 Physical Address
Address Model (AL_PA)

Arbitrated Loop
Public Loop Device Switch
Area Physical Address
Address Model Domain (AL_PA)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 138
Fibre Channel Communications
FC-2 Hierarchy
Multiple exchanges are initiated between initiators (hosts) and targets (disks)
Each exchange consists of one or more bi-directional sequences
Each sequence consists of one or more frames
For the SCSI3 ULP, each exchange maps to a SCSI command

OX_ID &
Exchange
RX_ID

SEQ_ID Sequence Sequence Sequence

SEQ_CNT Frame Frame Frame

Frame
Fields ULP Information Unit

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 139
Fibre Channel Fabric Topology
Trunking and Channeling
Port Channels
Higher aggregate bandwidth
Hardware-based load balancing
Only supported on switch to switch connections
(E_Port to E_Port and NP_Port to F_Port)

Trunking
Trunking E_Port (TE_Port)
Carries tagged frames from multiple VSANs
Trunking
Enhanced ISL (EISL) link E_Port
(TE_Port)

Standardization of Enhanced Capabilities Is Less Mature in the Fibre Channel


Fabric than You May Be Used to in the Ethernet and IP World

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 140
Virtual SANs (VSANs)
VLAN or 802.1q for FC VSANs Supported on MDS, Nexus 5000/7000
and UCS Product Lines
A Virtual SAN (VSAN) Provides a Method to
Allocate Ports within a Physical Fabric and
Create Virtual Fabrics

Analogous to VLANs in Ethernet Physical SAN Islands


Are Virtualized onto
Virtual fabrics created from larger cost-effective Common SAN
Infrastructure
redundant physical fabric
Reduces wasted ports of a SAN island approach
Fabric events are isolated per VSAN which gives
further isolation for High Availability
FC Features can be configured on a per VSAN
basis.
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 141
Start from the Beginning Storage
Target
Start with a host and a target that need to FABRIC A FABRIC B
communicate
FC Fabric FC Fabric
Typical Host has 2 HBAs (one per fabric)
each with a unique WWN
Target has multiple ports to connect to
multiple fabrics
Connect them to a FC Switch
Port Type Negotiation
Speed Negotiation
FC Switch is part of the SAN fabric
HBA
Most commonly, dual fabrics are deployed for
redundancy
Initiator

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 142
Buffer to Buffer Credits Fibre Channel Switch
Fibre Channel Flow Control
B2B Credits used to ensure that FC transport is lossless
Number of credits negotiated mandated between ports when
link is brought up RX 16

R_RDY
Transmit Credits are decremented with each packet placed

Packet
on the wire
Independent of packet size
If # TX credits == 0, no more packet transmission

Transmit Credits are incremented with each transfer ready


received. TX 16
TX 15
B2B Credits need to be taken into consideration as distance
and/or bandwidth increases, along with average packet size. TX 16
Host

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 143
Fabric Shortest Path First
Fibre Channel Forwarding Just Like OSPF
FSPF routes traffic based on destination Domain ID
Each node calculates fabric topology and computes preferred routes.
For FSPF a Domain ID identifies a single switch
This limits the max number of switches that can support
in the Fabric to 239 when FSPF is supported
FSPF performs hop-by-hop routing
FSPF uses total cost as the metric to determine most efficient path
FSPF supports equal cost load balancing across links

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 144
Storage Security
Fabric Zoning Target
FC Fabric
Zones are the basic form of data path security access
to the physical storage array not the actual LUN
A bidirectional ACL
Zone members can only see and talk to other
members of the zone
Devices can be members of more than one zone
By default, devices not in a zone are isolated from
other devices
Zones belong to a zoneset
Zoneset must be active to enforce zoning
Only one active zoneset per fabric or per VSAN
Not the only security required Storage admins must
still expose / export LUNs to hosts Initiator

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 145
LUN masking and mapping
Logical Unit Number (formated to the specific operating system using it)
Typically seen by operating system as a disk drive Array
Internal
Server
PWWN
Export as
LUN#
LUN#
Windows = e:\ or g:\ 100 11:22:33 1

Linux = /mnt/volumeE or /mnt/volumeG 101 55:66:77 1


102 55:66:77 2
Disk LUNs are normally dedicated 1 specific server
Mapped by storage array to specific server by PWWN
What are my LUNs?
Your LUN is 1
11:22:33
What are my LUNs?
Your LUNs are 1 and 2
55:66:77

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 146
Target
My Port Is Up Can I Talk Now?
FLOGIs / PLOGIs
Fabric Login (FLOGI)
FC Fabric
Determines the presence or absence of a Fabric
Exchanges Service Parameters with the Fabric E_Port
Switch identifies the WWN
in the service parameters of the accept frame and
assigns a Fibre Channel ID (FCID)
Initializes the buffer-to-buffer credits
Port Login (PLOGI) F_Port
Required between nodes that want to
communicate N_Port
Similar to FLOGI transports a PLOGI frame to
the designation node port HBA
In p2p topology (no fabric present), initializes
buffer-to-buffer credits
Initiator

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 147
What is NPIV?
N-Port ID Virtualization (NPIV) provides a means to assign multiple FCIDs to a single N_Port

Allows multiple applications to share the same Fiber Channel adapter port

Different pWWN allows access control, zoning, and port security to be implemented at the application level

Usage intended for platforms such as Storage Arrays

Storage Array FC NPIV Core Switch

Email LUNs Email I/O F_Port


N_Port_ID 1
Web LUNs Web I/O F_Port
N_Port_ID 2
File Services LUNs File Services I/O
N_Port_ID 3
N_Port

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 148
What is NPV
N-Port Virtualizer (NPV) utilizes NPIV functionality to allow a switch to act like a server performing multiple logins through a single physical link

Physical servers connected to the NPV switch login to the upstream NPIV core switch
Physical uplink from NPV switch to FC NPIV core switch does actual FLOGI
Subsequent logins are converted (proxy) to FDISC to login to upstream FC switch

No local switching is done on an FC switch in NPV mode

FC edge switch in NPV mode Does not take up a Domain ID, it is an extension of the Core Domain.

Scalability will be dependent on FC login limitation


Nexus 5000, MDS 91xx, MDS Blade Switches, UCS Fabric Interconnect FC NPIV Core Switch
F-Port

Eth1/1 Server1 TNP-Port TF-Port


N_Port_ID 1
Eth1/2 Server2 F_Port
N_Port_ID 2
Eth1/3 Server3
N_Port_ID 3
N-Port
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 149
Storage Area Networking
FCoE
Sample Storage Area
Networking in the CCIE
Topology

FCoE configuration and


troubleshooting within N5K &
N7K Storage vDC

Fibre Channel over Ethernet


Technologies

FCoE within the Fabric


Interconnect

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 151
FC over Ethernet (FCoE)
FCoE Benefits
Mapping of FC Frames over Ethernet Fewer Cables
Both block I/O & Ethernet traffic co-
Enables FC to Run
exist on same cable
on a Lossless
Ethernet Network Fewer adapters needed
Overall less power

Ethernet Interoperates with existing SANs


Management SANs remains constant
Fibre No Gateway
Channel
Traffic

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 152
FCoE Connectivity Extends FC SANs

FC FC
SAN Extension
FICON VSAN
SAN Security FC
Zoning
iSCSI QoS FICON

FCoE FCIP
SAN Fabric

Preserves FC investments
Simplifies SAN-attach of servers

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 153
PFC: Priority Flow Control
IEEE 802.1Qbb
VLAN Tag enables 8 priorities for
Ethernet traffic
PFC enables Flow Control on a
Per-Priority basis using PAUSE
frames (802.1p)
Therefore, we have the ability to
have lossless and lossy priorities at
the same time on the same wire
Allows FCoE to operate over a
lossless priority independent of other
priorities FCoE
Ethernet Wire

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 154
ETS: Enhanced Transmission Selection
IEEE 802.1Qaz
Allows you to create priority groups
Can guarantee bandwidth
Can assign bandwidth percentages to groups
Not all priorities need to be used or in groups

80% 20% 80% 20%


FCoE

Ethernet Wire
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 155
DCBX: Data Center Bridging eXchange
IEEE 802.1Qaz
Allows network devices to Hello?

advertise their identities and Hello?


Looks Like We Hello.
capabilities over the network All Speak the
Enables hosts to pick up proper Hello. Same Language.
Hello?
configuration from the network
Enables switches to verify proper
configuration Hello.

Provides support for:


PFC
ETS
Applications (e.g., FCoE)

Ethernet Wire

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 156
Protocol Organization

FCoE Itself FIP (FCoE Initialization Protocol)

Is the data plane protocol It is the control plane protocol

It is used to carry most of the It is used to discover the FC entities connected


FC frames and all the to an Ethernet cloud
SCSI traffic
It is also used to login to and logout from the FC
fabric

Uses unique BIA on CNA for MAC


The Two Protocols Have:
Two different Ethertypes
Two different frame formats
Both are defined in FC-BB-5

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/white_paper_c11-560403.html

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 157
FCoE Frame
Ethernet 12 Bytes (MAC Addresses) +
Header 4 Bytes (802.1Q Tag)
FCoE
Header
16 Bytes
FC
Header
Total: 2180 Bytes

24 Bytes
FC Payload

Up to 2112 Bytes
FCoE Standard (FC-BB-5) Requires
Jumbo Support;
4 Bytes 2.5KB = Baby Jumbo

1 Byte (EOF) + 3 Bytes (Padding)


CRC
EOF 4 Bytes
FCS

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 158
FCoE Port Types
Fibre Channel or Ethernet Switch

FCF VE_Port VNP_Port E_NPV


VE_Port VF_Port
Switch Switch

VF_Port VN_Port End


Unchanged from previous FC standard Node
VN_Port: Virtual N_Port End
VF_Port: Virtual F_Port VF_Port VN_Port
Node
VE_Port: Virtual E_Port
FCoE Switch : FCF
Added to support FCoE
FCoE_LEP (FCoE link endpoint): The data forwarding component that handles
FC frame encapsulation/decapsulation, and transmission/reception of FCoE
frames
FCoE Controller: the entity that implements the FIP protocol

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 159
FCoE Forwarding
FCF (Fibre Channel Forwarder) is a logical FC switch inside an FCoE switch
- Fibre Channel login happens at the FCF
- Contains an FCF-MAC address
- Consumes a Domain ID

FCoE encapsulation/decapsulation happens within the FCF


NPV devices are not FCFs and do not have domains
FC
Port
FCoE Switch
FC
FCF Port

FC
Ethernet Bridge Port

FC
Port
Eth Eth Eth Eth Eth Eth Eth Eth
Port Port Port Port Port Port Port Port

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 160
Unified Port Overview
Ports on Nexus 5548UP & Nexus 5596UP and Unified port GEM can be configured to be in
Ethernet or FC mode.

Ethernet or Fibre Channel

In each module, continuous set of ports can be in


Slot 2 GEM Slot 3 GEM Slot 4 GEM
configured as Ethernet or FC
Eth Ports Eth FC Eth FC
Eth ports have to be the first set and they have to be one
contiguous range. FC ports have to be second set and FC Ports 41 -
they have to be contiguous as well Eth Ports 1 - 40 48

Number of Ethernet and FC ports on a module can be


changed
Configuration example:
GEM power off/on or switch reload required to complete n5k(config)# slot 1
change in port types. n5k(config-slot)# port 41-48 type fc
n5k(config-slot)# port 1-40 type ethernet

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 161
What is FCoE-NPV FC
FABRIC A Target
FCoE Pass through device
All FCoE Switching is performed at the upstream FCF FCF
Addressing is pass out by the upstream FCF
Domain ID and N7K, MDS or N5K
More FCoE connectivity to hosts without: FC-MAP come
from the FCF
Running into the domain ID issue VF
Less-expensive
Consistent management
VNP
Proxys FIP functions between a CNA and an FCF N5K in
FCoE VLAN configuration and assignment
FCoE_NPV Mode
FCF Assignment
VF
FCoE_NPV does not

FLOG
FCoE-NPV load balance logins from the CNAs evenly across consume a domain ID

I
the available FCF uplink ports VN
FCoE-NPV will take VSAN into account when mapping or
pinning logins from a CNA to an FCF uplink
Operations and management process are in line with todays E_Node
SAN-Admin practices MAC Address

Similar to NPV in a native Fibre Channel network Dedicated FCoE Link


Converged Link

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 162
FCoE - NPV configuration Details
MDS w/
N7K w/
N7K Storage VDC release
release 5.2.x
n7k-fcoe(config)# feature npiv 5.2.x

MDS Global command


MDS9513-71# feature npiv
Becomes VNP to VF

N5Ks with N5Ks with


release 5.0.3 or n5k(config)# feature fcoe-npv release 5.0.3
later or later
LACP Port-channels can be configured between switches for high availability

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 163
FCoE Port Configurations
feature fcoe
vlan 100 LAN Fabric
fcoe vsan 100
Fabric A Fabric B

interface vfc20
bind interface Ethernet1/20
no shutdown
VLAN VSAN
1 100
vsan database
vsan 100 interface vfc20
vfc20
interface Ethernet1/20
switchport mode trunk
switchport trunk allowed vlan 1,100
Ethernet 1/20
spanning-tree port type edge trunk
Can also be configured with DCNM Device Manager

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 164
FCoE

FCoE Multihop Configuration example


N7K-50-fcoe(config)# vsan database
N7K-50-fcoe(config-vsan-db)# vsan 50
N7K-50-fcoe-1(config-vlan)# interface ethernet 4/11-12 N7K-50-fcoe(config-vsan-db)# vlan 50
N7K-50-fcoe-1(config-if-range)# switchport mode trunk N7K-50-fcoe(config-vlan)# fcoe vsan 50
N7K-50-fcoe-1(config-if-range)# switchport trunk allowed vlan 50
N7K-50-fcoe-1(config-if-range)# channel-group 50 force mode active
N7k-50-fcoe-1(config-if-range)# no shut
N7K-50-fcoe-1(config)# interface vfc-port-channel 50
N7K-50-fcoe-1(config-if)# switchport mode f
N7K-50-fcoe-1(config-if)# switchport trunk allowed vsan 50
N7K-50-fcoe-1(config-if)# no shut

SAN A SAN B

n5k-2(config-vlan)# interface ethernet 1/1-2


n5k-2(config-if-range)# switchport mode trunk
n5k-2(config-if-range)# switchport trunk allowed vlan 50
n5k-2(config-if-range)# channel-group 350 mode active

n5k-2-104(config)# interface vfc350


n5k-2-104(config-if)# switchport mode np
n5k-2-104(config-if)# bind interface port-channel 350
n5k-2-104(config-if)# switchport trun allowed vsan 50
n5k-2-104(config-if)# no shut

n5k-2-104(config)# vsan database


n5k-2-104(config-vsan-db)# vsan 50
n5k-2-104(config-vsan-db)# vlan 50
n5k-2-104(config-vlan)# fcoe vsan 50

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 165
Storage Area Networking
FCIP (Fibre Channel over IP)
Sample Storage Area Networking in the
CCIE Topology

Fibre Channel Over IP


MDS configuration and
deployment

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 167
FCIP - Fibre Channel Over IP
FCIP provides FC fabric connectivity over long distance
FCIP provides a standard way of encapsulating FC frames within TCP/IP,
allowing islands of FC SANs to be interconnected over an IP-based network
TCP/IP is used as the underlying transport to provide congestion control and
in-order delivery of error-free data
One or two TCP sessions can be used
FC frames are treated the same as datagrams
It is not IPFC, iSCSI Transports or extended FC Fabric
Can be routed in the IP network, unlike FCoE

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 168
Cisco IPS Module FCIP Basic Configuration Steps
Perform these basic configuration steps on both MDS 9000 switches to
configure IPS modules and FCIP links

Step 1 - Perform FCIP pre-configuration planning


Step 2 - Configure the Gigabit Ethernet interface(s) remember MTU setting.
Step 3 - Measure RTT
Step 4 - Create a FCIP profile and assign a Gigabit Ethernet interface IP address to
that profile
Step 5 - Create a FCIP interface and assign a profile to that interface
Step 6 - Configure peer information for the FCIP interface(s)
Step 7 - Enable the interface(s)
TSI-9222I-B-134# ips measure-rtt 1.1.1.125 interface gigabitethernet 1/1
Round trip time is 68 micro seconds (0.07 milli seconds)
TSI-9222I-B-134#

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 169
FCIP Configuration Example: MDS9000
fcip profile 10
ip address 10.1.4.2
tcp max-bandwidth-mbps 155 min-available-bandwidth-mbps 20 round-
Site A trip-time-ms 1
VSAN 1 VSAN 20
interface fcip50
switchport mode E
Switch-A no shutdown RTT will autconfigure
switchport trunk allowed vsan 1 and adapt to network
10.1.4.2 switchport trunk allowed vsan add 20 changes during idle
use-profile 10 periods
peer-info ipaddr 10.4.8.2
Shared Jumbo Frame MTU -
155Mbps interface GigabitEthernet2/5 2300 Bytes will handle
WAN link ip address 10.1.4.2 255.255.255.0 largest FC frame
switchport mtu 2300
no shutdown
10.4.8.2

Switch-B Three steps for FCIP config Profile, GigE i/f and FCIP i/f

VSAN 1 VSAN 20 Min-bandwidth set to minimum bandwidth available (through


QoS or other means). Sender will start at this rate
Site B
Peer FCIP interface configured similarly

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 170
Example: FCIP Interface Show Command
SI-9222I-B-134# show interface fcip 1
fcip1 is trunking
Hardware is GigabitEthernet
Port WWN is 20:14:00:0d:ec:4a:cb:40
Peer port WWN is 20:14:00:0d:ec:39:07:00
Admin port mode is auto, trunk mode is on
snmp link state traps are enabled
Port mode is TE Local Interface verification
Port vsan is 1
Speed is 1 Gbps
E_Port Operation Trunk vsans (admin allowed and active) (13)
Enabled Trunk vsans (up) (13)
Trunk vsans (isolated) ()
Trunk vsans (initializing) ()
Interface last changed at Wed Mar 25 02:08:27 2015

Using Profile id 1 (interface GigabitEthernet1/1)


Peer Information Peer Information
IP Address and Peer Internet address is 1.1.1.125 and port is 3225
TCP port Write acceleration mode is configured off
Tape acceleration mode is configured off
Tape Accelerator flow control buffer size is automatic
FICON XRC Accelerator is configured off
Ficon Tape acceleration configured off for all vsans
IP Compression is disabled
Maximum number of TCP connections is 2
QOS control code point is 0
QOS data code point is 0
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 171
Example: FCIP Interface Show Command..cont
TCP Connection Information
MDS to MDS Two 2 Active TCP connections
TCP Connections Control connection: Local 1.1.1.134:3225, Remote 1.1.1.125:61635
Data connection: Local 1.1.1.134:3225, Remote 1.1.1.125:61637
One Connection Is 54 Attempts for active connections, 5 close of connections
F-Class (Control) TCP Parameters TCP MTU
Second Path MTU 2300 bytes We can transmit a full
Connection Is the Current retransmission timeout is 200 ms size FC frame
Data Round trip time: Smoothed 8 ms, Variance: 4 Jitter: 150 us
Advertized window: Current: 24580 KB, Maximum: 24580 KB, Scale: 5
Peer receive window: Current: 33 KB, Maximum: 33 KB, Scale: 5
Congestion window: Current: 30 KB, Slow start threshold: 112 KB
Current Send Buffer Size: 24580 KB, Requested Send Buffer Size: 0 KB
CWM Burst Size: 50 KB
Measured RTT : 500000 us Min RTT: 500000 us Max RTT: 0 us
TCP Parameters 5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec Fragments
9947 frames input, 955600 bytes Are we re-assembling
9928 Class F frames input, 953212 bytes frames due to MTU
19 Class 2/3 frames input, 2388 bytes issues?
0 Reass frames
0 Error frames timestamp error 0
10117 frames output, 961460 bytes
10117 Class F frames output, 961460 bytes
0 Class 2/3 frames output, 0 bytes
0 Error frames

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 172
Computing with UCS
UCS in the DC CCIE
High Level Overview
Features you should know for the exam
Possibly Topology Scenarios

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 174
UCS Physical Building Blocks
UCS Manager
Embedded manages entire system

UCS Fabric Interconnect


48 Port 10Gb FCoE with Unified Ports

UCS Fabric Extender


Remote line card

UCS Blade Server Chassis


Flexible bay configurations

UCS Server
Industry-standard architecture
Blade and rack-mount, 2 and 4 socket

UCS Virtual Adapters


Choice of multiple adapters

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 175
UCS in the CCIE Sample
Topology
SAN & LAN connectivity to
Northbound switches

Multihop FCoE

UCS NPV and FC switching


modes on Upstream Nexus

Blade and component Discovery and


base configuration

Address Pools and Profile, Configuration UCSM and CLI


& Deployment Proficiency

SAN boot, FC, PXE, iSCSI

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 176
UCS Manager
Complete management and configuration is driven by the GUI interface of the UCSM.
FI setup wizard is used to do initial install of IP addresses and start to Cluster.

Browser pointed at Cluster IP


Java Client

HTML client
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 177
End Host Mode - Fabric Failover for Ethernet
Fabric provides NIC failover LAN SAN A SAN B
capabilities chosen when defining
a service profile
UCS Fabric
Interconnects
Traditionally done using NIC
bonding driver in the OS
Chassis
Provides failover for both unicast
and multicast traffic Fabric Extender Fabric Extender

Works for any OS on


bare metal Adapter Adapter

vNIC
vNIC

vNIC
vNIC
Recommended in case
of OS on bare metal for BMC BMC

non hypervisor-based servers Half Width Blade Half Width Blade

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 178
Configuring Unified Ports

Fibre
Channel

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 179
Disjointed L2 Feature and Configuration
Prior to UCS 2.x code, network facing interfaces are
Dev2 Backup
Network 3 configured to support all VLANs in the system
Prod
Network 1 Network
VLAN 177 VLAN 178
VLANs 21- VLAN
VLAN 18331
VLANs 1-20 There is no way to configure a subset of VLANs on a
30
network facing interface
A single network-facing interface is selected to receive
multicast/broadcast traffic from the upstream network
BIF1
BIF2 BIF3 BIF1 BIF2 BIF3 This limits UCS to deployments where the upstream
UCS A UCS B networks are symmetrical (all LAN segments are reachable
by each border interface)
Assumptions
There is no overlap in VLAN IDs between the disparate
networks
Both FIs have access to the same set of VLANs. This
ensures the function of fabric failover

Using LAN Uplink Manager to configure

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 180
Traffic Behavior Summary
RPF Check
Border Links Deja-Vu Check

FI

Server Links
Traffic from Server Traffic from Network
Unknown Unicast Forwarded to its pinned border interface Unknown Unicast Dropped

Known Unicast Switched to server-facing interface based on DMAC Known Unicast Accepted only on the pinned border and
lookup forwarded to the server port based on DMAC-
Broadcast/L-2 multicast Forwarded to pinned border-interface and other server lookup.
ports Broadcast/L-2 Accepted only on the flood-pinned border and
multicast forwarded to all server ports
IP-Multicast (Un- Forwarded to pinned border-interface
registered)
IP-Multicast (Un- Not forwarded to any interface.
registered)
IP-Multicast Forwarded to pinned border-interface and to all server
(Registered) ports that registered for the group. IP-Multicast Accepted only on the g-pinned border and
(Registered) forwarded to all server ports that registered
for the group.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 181
CCIE SAN Configuration Possibilities

Several Types of SAN


connections
UCS in NPV mode
UCS in FC Switch Mode Fibre
Channel
Direct Attach Storage Ports Direct Attach

SAN Port Channel


F-Port Trunking
Watch the model of MDS
some do not support FC Port
Channel + Trunking! (During
Self Study)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 182
N-Port Virtualization (NPV) mode
UCS FI work in NPV mode by default
Server-facing ports are regular F ports
Uplinks towards SAN core fabric are NP ports

UCS distributes (relays) FCIDs to attached devices


No domain ID to maintain locally
Zoning, FSPF, DPVM, etc are not configured on the UCS Fabrics
Domain mgr, FSPF, zone server, fabric login server, name server
They do not run on UCS Fabrics
No local switching
All FC traffic routed via the core SAN switches

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 183
UCS operating in FC Switching Mode
Global setting: FC Switching Mode (requires a reboot)
Why? Direct connectivity of FC and FCoE Storage Arrays
Connecting a NAS is totally independent of the FC mode of operation
Be aware of:
UCS provides limited FC switching features
No interop mode per VSAN (keep that in mind!)
Direct connect from Fabric Interconnect to Storage Array FC targets
Designed for small scale
Limited interoperability with storage ecosystem

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 184
UCS Service Profiles Entities

Server Storage Network


Identity (UUID) Optional Disk usage Uplinks
Adapters SAN settings LAN settings
Number VLAN
LUNs
Type: FC, Ethernet QoS
Persistent Binding
Identity etc
Characteristics SAN settings Firmware
Firmware vSAN Revisions
Revisions Firmware LAN Connectivity
Configuration Revisions Policies
settings
SAN Connectivity Polices
Boot Policies

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 185
Pools, Policies, Templates Oh my!
Before we start creating Service Profiles we can make use of a few building
blocks
Pools Predefined Resources
Policies Rules to be followed
Templates Common configuration built using pools and policies that can be
applied for a specific Host types

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 186
UCS Templates & Connectivity Policies
vNIC Template
vHBA Template
Service Profile Template (Initial vs. Updating)

LAN Connectivity Policy


SAN Connectivity Policy

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 187
SAN Boot On UCS
Required for true stateless computing
Failed Local Disk doesnt render your host useless
Simple re-association to mitigate HW failures or upgrades
Servers identity follows with its service profile 1:1

Must be block level storage (FC or iSCSI)


iSCSI boot supported on all VIC adapters, and Broadcom
Watch jumbo frames if using iSCSI!
Array and SAN Switch configuration out of UCS scope

Storage networks (aka Fabrics) are typically segregated


Best Practice

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 188
Sample FC SAN configuration
Vsan 10 Vsan 20

Tgt A- 50:0A:09:81:78:3B:98 Tgt B - 50:0A:09:86:78:3B:98

fc0 20:00:00:25:B5:00:10:0E fc1 20:00:00:25:B5:00:10:0F

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 189
SAN configuration
6x00 FLOGI into MDS/N5K

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 190
Cisco VIC Fibre Channel Option ROM

The VIC does not have an Option ROM to break into during POST
You can connect to the adapter and check configurations
It will show you if there is connectivity, but only at the moment the VIC is trying
to initialize.
You must run the commands while the VIC is initializing

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 191
Cisco VIC Fibre Channel Option ROM
The VIC does not have an Fibre Channel Option ROM you can query during
Boot up

You can make use of the attach-fls commands to view the configured
settings

cae-sj-ca3-A# connect adapter 1/1/1 Chassis/Slot/ID


adapter 1/1/1 # connect
adapter 1/1/1 (top):1# attach-fls Fabric Login Services
adapter 1/1/1 (fls):1# vnic
---- ---- ---- ------- -------
vnic ecpu type state lif
---- ---- ---- ------- -------
5 1 fc active 3
6 2 fc active 4

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 192
Cisco VIC Fibre Channel Option ROM
VIC Attempting to Initialize

adapter 1/1/1 (fls):1# vnic


---- ---- ---- ------- -------
vnic ecpu type state lif
---- ---- ---- ------- -------
5 1 fc active 3
6 2 fc active 4

adapter 1/1/1 (fls):2# lunmap 5


lunmapid: 0 port_cnt: 1
lif_id: 3
PORTNAME NODENAME LUN PLOGI
50:0a:09:81:86:78:3b:98 00:00:00:00:00:00:00:00 0000000000000000 N
adapter 1/1/1 (fls):12# login 5
lifid: 3
ID PORTNAME NODENAME FID
0: 50:0a:09:81:86:78:3b:98 00:00:00:00:00:00:00:00 0x000000

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 193
Cisco VIC Fibre Channel Option ROM
If executed when you see the Palo Screen. The VIC has initialized
PLOGI = Y
FID = Configured Storage tgt FCID
You cannot scan for valid LunIDs

adapter 1/1/1 (fls):2# lunmap 5


lunmapid: 0 port_cnt: 1
lif_id: 3
PORTNAME NODENAME LUN PLOGI
50:0a:09:81:86:78:3b:98 00:00:00:00:00:00:00:00 0000000000000000 Y
adapter 1/1/1 (fls):12# login 5
lifid: 3
ID PORTNAME NODENAME FID
0: 50:0a:09:81:86:78:3b:98 00:00:00:00:00:00:00:00 0x530001

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 194
SAN Failures
Most SAN Failures are caused by one of the following
Incorrect Zoning
Incorrect Masking
Incorrect LUN ID
VSAN Misconfig

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 195
Possible Topologies
Native FC and Ethernet Uplinks
Storage Array

Ethernet
Fibre Channel
FCoE

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 197
Separate FCoE and Ethernet VPC Uplinks
FC Storage Array

Ethernet
Fibre Channel
FCoE

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 198
Converged FCoE and Ethernet Uplinks
FCoE Storage Array

Ethernet
Fibre Channel
FCoE

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 199
Possible Exam Scenarios/Tasks
Configure UCSM Policies (Power, Discovery, Uplink, Firmware)
Create/Modify USC Service Profiles
Configure RBAC Authentication
Create LAN/SAN Policies (FC Modes, Disjoint L2, VLAN, SAN)
Configure Remote Boot (iSCSI, FC SAN, FC Direct, FCoE mhop)
Configure/Modify Templates (SP, vNIC, vHBA, LAN/SAN Connectivity)
Configure/Apply Server Pools & Compute Autodiscovery

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 200
UCSM Training Resources
UCSM External Web Page
http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-manager/index.html

UCSM Community Page


https://communities.cisco.com/community/technology/datacenter/compute-and-
storage/ucs_management
Live & previously recorded UCSM Tech Talks

UCSM Getting Started & Common Practice Guides


http://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-
manager/whitepaper_c11-697337.html
Learning Network DC CCIE Technical VoDs
https://learningnetwork.cisco.com/community/learning_center/ccie_datacenter_tech_sem
inars

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 201
UCS Study & Practice Resources
Self Guided Labs available on dCloud
https://dclould.cisco.com

UCSPE (Platform Emulator)


https://communities.cisco.com/docs/DOC-57526
Virtual Machine which simulates UCS Domain (blade & racks)
Allows you to import config from a physical UCS domain

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 202
Demo
UCSM Sample Question 1
The company you work for, Win Tire S. Coming, is separated into 3 houses; Lannister, Targaryen and
Stark. Each house is managed by a junior squire. Each junior squire has access only to their houses
respective house (Org)

As the senior Maester for your kingdom, youve been challenged with assigning 100 new compute
blades to various houses as quickly as possible. Two blades have already been installed, with the
remainder being installed within the next hour. You need to make each houses respective servers are
made available to the junior squires as soon as they are installed/connected.
Additional Info
You have no idea which of the 10 Chassis the blades will be inserted into upon arrival
The first two compute nodes have already arrived and should belong to the Starks.
The details of what each department purchased has been provided.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 204
Win Tire S. Coming - Compute Purchase Details
Department Model CPU/Core Memory Adapter Qty

Lannister B200-M4 Dual/12 128GB mLOM 15

Lannister B420-M4 Dual/8 16GB mLOM, VIC-M82-8P 15

Targaryen C240-M4L Dual/4 128GB VIC 1225 10

Stark B200-M4 Dual/2 64GB mLOM 30

Stark C220-M4L Dual/4 64GB VIC 1225 30

How the are we going to do this?


Server Pools
Qualification Policies

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 205
UCSM Sample Question 2
A junior compute engineer was
configuring UCSM and reports
the host his service profile is
associated to blade 1/1 is
unable to reach any of the
required network resources in
either the Data or the IP
Storage networks. 10.1.1.61 10.1.1.62
Your task is to identify &
resolve the issue while
maintaining access to all
existing network resources.
Host credentials are: VIP:10.1.1.40
root/Cisco!123
UCS/N5K Credentials are:
admin/Cisco!123

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 206
UCSM Sample Question 3
Boot from SAN Troubleshooting
A study partner of yours has configured a service profile called CCIE-Demo-BFS and installed
vSphere on the remote LUN. For practice, hes gone and broken the profiles ability to boot to the
remote LUN ID 1. Your mission, should you choose to accept it is to find the mistake, fix it and allow
the profile to successful boot the OS. The following diagram is the only other information your friend
has given you.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 207
Agenda

Introduction
L2/L3 Technologies
DC Storage and Compute
CCIE Next Gen
Automation and Orchestration
Fabric Infrastructure
Study, Preparation and Closing
The Future of a Digital Workforce
In the Internet of Everything Age
Global social-economic problems to solve

Digital technology

Worlds converge
Future of work

Market and consumer driven demand


210
Digital Experience Architect Robotics Specialist
Platform Developer

Cyber
Data Scientist
Security
Analyst
Cloud Broker Business Transformation Practitioner

Customer Success
Cognitive Engineer
Network Programmer (SDN)
Industrial Network Engineer
Customer
Enterprise Architect Makers

Machine Learning Scientist Social Intelligence Manager

Job roles of the future


TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 211
Job-Role Evolution
Technology Evolution

Training and Certifications


The workforce of the future
Will require
continual learning and upskilling
CCIE Update
Aligning to Evolving Industry Job Roles and Technology Evolution
CCIE NextGen Program Update
CCIE Data Center v2.0

TECCCIE-3644
TECCCIE-3000 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 213
Having practical IT expertise is
no longer a differentiator

IT professionals need a clear


A new, understanding of evolving and disruptive
technologies that fuel innovation
enhanced Ciscos enhanced expert-level
CCIE certifications providing IT professionals
with the most advanced program

Expands career paths to address


shifts in critical industry roles

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 214
CCIE NextGen Framework
Blueprint Weights

WRT% LAB%
NEW EXAM TOPICS Evolving Technologies (Common across all Tracks)
REST Automation and 10% N/A
SDN IoT DevOps XaaS OpenStack Cloud NFV/AFV
API Orchestration

CCIE Data CCIE CCIE CCIE CCIE CCIE CCDE


Center v2.0 SP Security v5.0 Wireless Collaboration R&S Current BP
Consolidated/ Current Consolidated/ Current BP Current BP Current per track 90% 100%
Unified BP Unified per track per track BP
Blueprint per track Blueprint per track

100% 100%
New Evolving Technologies section across all CCIE/CCDE tracks
Future proofing IT professional skills
Holistic assessment of each learning domain
New Written Exams go-live on July 25

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 215
CCIE Community Events (CiscoTV Broadcast)
CCIE Community Events are interactive online events where leaders of
Learning@Cisco discuss the state of the industry, updates to the CCIE program
and items that are top of mind to the community.
The events are held twice a year and are invite-only for the active community.

https://learningnetwork.cisco.com/community/archived_events/ccie-community-events

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 216
CCIE Webinar Series
OpenStack
Technical Sessions focused on new technologies
4 6 Sessions per year
OpenStack May 2015 Fog
Architecture
Fog Architecture August 2015
Cisco NetFlow and Big Data Analytics for Cybersecurity October 2015
Neutron Deep Dive March 2016 Cisco
NetFlow &
DNA Deep Dive June 2016 Big Data
Analytics

https://learningnetwork.cisco.com/community/archived_events/ccie-community-events

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 217
Never Received an Invitation? Opt-In

http://mkto.cisco.com/CCIE-Opt-In.html
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 218
CCIE Rollout Plan
CCIE Data Center v2.0 evolving
Announcement technologies and revised exam
on 11/19 topics July 2016

All written blueprints Future revisions of


incorporate new technology Expert-level tracks will
domain July 2016 incorporate new CCIE
framework

2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Benchmark Technology Topics Incorporated
Across the Learning@Cisco Portfolio

Offers a new skills


The most advanced Bridges technology
framework for a
program for IT expertise and
disruptive and
professionals business impact
evolving environment

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 220
Agenda

Introduction
L2/L3 Technologies
DC Storage and Compute
CCIE Next Gen
Automation and Orchestration
Fabric Infrastructure
Study, Preparation and Closing
Automation and Orchestration
Orchestration & Automation
There are a number of solutions that fall under this category the DC CCIE
candidate should be familiar with:
UCS Central
Today well take a look at these two
UCS Director
IMC Supervisor
Cisco Process Orchestrator
Open Network Environment Suite

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 223
UCS Management Portfolio
UCS Director API
UCS Director
Non-Cisco
Policy Driven, Application Centric Infrastructure Management and
Infrastructure Orchestration

API API API


UCS Central UCS Performance
Policy Driven Multi DC,
IMC Supervisor Manager
Multi-Domain
Virtual Machines Management Performance Monitoring

Network Devices API API


UCS Manager UCS Manager
Domain 1 Domain x
API

CIMC
Storage
FlexPod vBlock
Stand-Alone UCS
C-Series Unified Computing Integrated & Converged
System Infrastructure
Servers
Basic Management Functionality Advanced Infrastructure Abstraction & Automation

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 224
UCS Central Overview
UCS Central Introduction
Many Domains

One Console

UCS Central: Centralized management for multiple UCS


Domains
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 226
UCS Central: Is/Is Not

UCS Central is scalable compute UCS Central is not:


infrastructure management Supporting capabilities beyond UCS
Automated, policy-based hardware Manager
configuration across domains Operating System or Hypervisor
Centralized ID Management deployment or management
Centralized Hardware Inventory Software patch management
Centralized Hardware Fault Storage Management
Management Network Management outside of the
Simplified cross-domain firmware UCS domains
management Cloud/IaaS controller
Detailed bandwidth, power, and
thermal statistics collection

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 227
UCS Manager Registration

UCS Central UCSM


UCSM
UCSM
UCSM

At Registration:
1. Registration initiated by admin on UCS Manager
2. Requires UCS Central IP or DNS name and (optionally) domain group
3. Secure process through the use of Shared Secret

After Registration:
1. All policies for the domain group take effect at registration
2. All resources from the local pools become available in the Global Pools

Bulk Registration:
1. Registrations can be done through the XML API
2. Scripts can be written with lists of UCS Domain IP addresses to bulk register

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 228
Why Should Customers Use UCS Central?
Feature Functions Benefit
Information Dashboard Centralized hardware inventory, centralized Improved visibility across local and remote domains
faults/logs, and up to one year of statistics reduces the administrative time to monitor,
across UCS Mini and Classic UCS domains troubleshoot, inventory, and do capacity planning

Centralized KVM Access KVM sessions on all UCS managed Users no longer need to know which domain to access
servers from a single location to set up a KVM session.

Centralized Backup Scheduled backup of UCS Manager and UCS Automated backups to a central location improves
Central instances. resiliency with minimal administrative impact

Administrative Configuration Cross-domain administrative settings and Set up new domains in minutes with limited
cross-domain ID pools that new UCS Manager administrative effort while maintaining cross-domain
instances have access to upon registration consistency saving hours of set up time per domain.

Operational Control Global policies and settings that can deployed Policy and settings standardization and enforcement
and enforced across domains across domains helps ensure compliance, reduces
configuration issues, and reduces troubleshooting time.

Workload Mobility Global Service Profiles with optional site Consistent deployments across UCS domains with the
specific settings for localization flexibility to quickly provision, de-provision, or move
workloads between servers or domains.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 229
UCS Central Domain Groups
UCS Central

UCSM 1
Domain
Group 1
Domain UCSM 2
Group 2
Domain
Group 3
UCSM 3

Domain Group (DG) is arbitrary grouping of UCS domains UCSM 5


Domains can be a part of only one DG at a time
Policies defined in the DG are in effect for all domains in the DG UCSM 6
Domains can move between DGs
DG to DG move for domain can be disruptive depending on new policies
Domain can auto-join DG based on qualification policies at registration

UCSM 7

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 230
UCS Central Domain Groups
UCS Central
e.g.

PRODUCTION
PRODUCTION IT
Geographic UCSM 1

LONDON
Domain Domain
Group 1 Groups
Domain UCSM 2

IT
Group 2
e.g.
Domain Organization
Group 3 within Domain
Groups
UCSM 3

ENGINEERING
ENGINEERING IT
NEW YORK
UCSM 5

IT
Domain Groups can be created based on operational needs UCSM 6

BANGALORE

LAB IT
UCSM 7

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 231
Global Admin Policies UCS Domain 1

Global Admin Policies in a Domain Group

Date & Time: NTP, Timezone


DNS
Remote Access
SNMP UCS Domain 2
Debug Settings
Call Home
Authentication (LDAP, Radius, TACACS)
Equipment Power and SEL policies

UCS Domain 3

Admin Policies are defined at the domain group


Any domain that is a member of the DG inherits policies

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 232
Cross Launch of UCS Manager and KVM
UCS Central

UCS
Manager

UCS Central

Server
KVM
Console

Access to all registered UCS Managers and server consoles from one location

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 233
Global Service Profiles & Templates UCS Domain 1

Global Templates defined in Global Service Profile Template


HR-Apps
UCS Central Network: HR-VLAN

Network QoS: High


BIOS: Version 1.03

Global templates use global Boot Order: SAN, LAN

policies

Global Service Profiles Global Service Profile


HR-App1
derived from Global SP Network: HR-VLAN
Network QoS: High
UCS Domain 2
MAC: 67:6f:74:75:63:73:21:20

templates WWN: 00:05:9b:67:6f:75:63:70


BIOS: Version 1.03
Boot Order: SAN, LAN

Global Service Profiles can


be attached to Global
Server Pools and Identifier Global Service Profile
HR-App2
Pools Network: HR-VLAN
Network QoS: High
MAC: 67:6f:74:75:63:73:21:21
WWN: 00:05:9b:67:6f:75:63:72
BIOS: Version 1.03
Boot Order: SAN, LAN

Global Server Pools can UCS Domain 3

have members from multiple


domains
Global Service Profile
HR-App3
Global SPs can be deployed Network: HR-VLAN
Network QoS: High
MAC: 67:6f:74:75:63:73:21:23

to domain of choice WWN: 00:05:9b:67:6f:75:63:73


BIOS: Version 1.03
Boot Order: SAN, LAN

manually or through
automatic association to a
server in a pool
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 234
Global ID Pooling
UCS1
Global Pool
67:6c:6f:62:61:6c:69:64
68:70:73:75:63:6b:73:21
Pool1
75:75:69:64:72:6f:63:6b
67:72:6f:77:75:63:73:21
22:6d:61:63:69:64:21:22
62:75:79:75:63:73:21:21
27:77:77:6e:66:75:6e:27

UCS Central
UCS2

Pool2
66:63:6f:65:62:61:62:79
75:63:73:72:6f:63:6b:73
ID usage from
Both local and global pools
67:6c:6f:62:61:6c:69:64
68:70:73:75:63:6b:73:21 UCS3
75:75:69:64:72:6f:63:6b

Pool3
Centralized sourcing of IDs from global pools 76:69:63:70:6f:77:65:72
Real-time ID usage summaries 73:76:63:70:72:6f:66:6c
Avoidance of ID conflicts among UCS domains 75:63:73:6d:63:6f:6f:6c
Reporting on ID usage

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 235
VLAN Aliasing Global Service Profile
Allows a single VLAN alias to be
used that can reference different VLAN Alias Prod
VLAN IDs in different domain groups
Ex. Prod VLAN can resolve VLAN ID
65 in a domain group in one data
center and resolve to VLAN ID 66 in Domain
another domain group in a different Group A
data center.
VLAN Prod Domain
The single VLAN alias can be used in ID 65
a global service profile that is Group B Domain
deployed across multiple domain VLAN Prod Group C
groups. ID 66
VLAN Prod
ID 67

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 236
HTML 5 User Interface Enhancements
HTML 5 UI is the default UCS Central UI
Old Flash-base UI is still available but deprecated and hasnt received any
enhancements
User Experience Enhancements
Unified KVM browser plus KVM user role
Additional and Improved Widgets
Table Export for Reports
Managing multiple vLAN permissions
vNIC and vHBA in a Global Service Profile in addition to LAN and SAN connectivity
policies

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 237
Overview Browsing Resources
Each item opens up a table of
all resources of that type within
the system

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 238
Overview Searching

Search for any policy and optionally provide a name to


filter your results

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 239
Overview Tasks
Perform actions such
as Create a Policy,
Schedule a Backup
and Install a FW
Bundle
You can also perform
operational tasks such
as creating Local
Users and setting up
Smart Call Home

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 240
dCloud Labs for UCS Central
Currently based on UCS
Central 1.3
9 labs that guide users
through domain registration,
pool setup, policy setup, etc.
One of the best lab guides
most reviewers have seen
A great way for Cisco,
partners, and even customers
to learn about UCS Central

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 241
Possible Exam Scenarios/Tasks
Register UCS Domain(s)
Assign Domains to appropriate Domain Groups
Create/Assign Global Resources
Create/Clone/Assign Global Service Profiles
Troubleshoot UCS Central Configuration Issues
Troubleshoot Global Service Profile Deployment

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 242
UCS Central Resources
UCS Central External Web Page
http://www.cisco.com/en/US/products/ps12502/index.html
UCS Community Page
https://communities.cisco.com/ucs
Live & previously recorded UCS Management Tech Talks

UCS Central Best Practice Guide


https://supportforums.cisco.com/docs/DOC-32946
Learning about UCS Central through dCloud Labs
https://dclould.cisco.com

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 243
UCS Director
Cisco UCS Director
A multi-vendor, multi-tenant, multi-hypervisor provisioning and management
solution that provides comprehensive infrastructure control, management and
monitoring via a single pane of glass

Cisco UCSD automates the provisioning of resource pools across physical and
virtual from a unified centralized management console, reducing time-to-value
for both applications and end users.

Cisco UCSD delivers unified management for the industrys leading converged
infrastructure solutions, which are based on the Cisco Unified Computing
System (UCS) and Nexus platforms.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 245
Cisco UCSD Turn-Key Solution Overview
Mobile Devices
LDAP, Single Sign On
RBAC
IT Ticketing Systems
End CMDB,
Admins Operations
Users Metering/Chargeback

Self Service Admin System


Dashboard REST API
Portal Console Integration

UCSD Unified Infrastructure Controller Amazon,


Multi-tenant & integrated cloud platform Rackspace,

Provider API
UCS Director
Integrated Multi-tenant Cloud Platform Public Clouds

Server Storage Network RHE-Virtualization


Managers APIs Manager vCenter System Center
Manager
Custom
UCS Nexus
Connector

VMware HyperV
Open KVM
Automation Infrastructure Cloud Infrastructure

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 246
Discovering Infrastructure
Discover UCS Compute Domain Discover Network

Discover Virtual Infrastructure

Discover Storage

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 247
Task Library 1000+ Tasks
Plus create Custom Tasks in minutes

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 248
Introducing UCS Director Orchestration and
Workflows
my-workflow
Start

Storage Resume Worfklow


Task-1

Network Rollback
Task-2
Unprovision
Compute
Task-3
Approvals
Virtualization Task-4
Publish to Catalog

End

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 249
Drag n Drop Workflow Creation
Orchestration, Workflow and Tasks Defined

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 250
Key Use Cases
Infrastructure
Application Provisioning
Infra. for with
provisioning Applications : 100%
UCS-Director VirtualVMs
: 100% & Mixed

Secure Tenant Application Profile Self-Service


on-boarding Definition Portal

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 251
Bare Metal Server Provisioning Overview
Mgmt
VLAN PXE Record
MAC Address
Mgmt VLAN
BMA PXE Record
IP Address
1 Network mask
HTTP
UCS Director Hostname
TFTP Gateway
DNS
DHCP Root password
PXE DHCP request 2 Time zone
OS Type
3 DHCP response
Bare Metal Server
PXE boot 4
OS Installer/Image 5

PXE VLAN

Image Repo

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 252
Deployment Automation Scenario
Automating Server Deployment within a Single
Work Flow Legend
Virutalization
Tasks
Network Tasks
Storage Tasks
Compute Tasks

Create Update Create Network Deploy SP from


Create LUN
VLANs Trunks Policies SP Template
Workflow

Create and Create Zones Create


Activate Zoneset PXE Boot
Configure IG and Zonesets PXE Record Blade Power ON

Change Boot Register with Send Complete


OS Installation Reboot PXE Verification
Order vCenter Notifications

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 253
Possible Exam Scenarios/Tasks
Device Discovery
Create Virtual Datacenter (VDC)
Configure RBAC Authentication
Create/Modify Workflows
Create/Modify Orchestration Tasks
Manage Task Inputs/Outputs
Publish Tasks to Service Catalogue

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 254
UCS Director Resources
UCS Director External Web Page
http://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-director/index.html
UCS Director Community Page
https://communities.cisco.com/community/partner/datacenter/unifiedmanagement/ucs_di
rector
Live & previously recorded UCS Director Tech Talks

UCS Director Getting Started Whitepaper


http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-
director/le-41601-ucsd-gsd.pdf
Learning about UCS Director through dCloud Labs
https://dclould.cisco.com

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 255
UCS Central Sample Question 1
To make your job of managing multiple UCS domains easier, youve decided to deploy
UCS Central. Youve been asked by your manager to register the first UCS domain with
Central, and ensure that only the following UCS policies will be managed by Central. All
policies/features not listed below should remain within the control of the UCS domain.
UCS Central Shared Secret: C1scoucs
Policies Managed by Central:
Call Home
Power Redundancy
Date & Time
Firmware

Lastly, if the UCS domain is ever de-registered from Central, all global polices should be removed.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 256
UCS Central Sample Question 2
Now that youve registered your first UCS domain, you need to configure a few policies.
Create the following UCS Central resources:
Global VLANs 21-25
Global User-Ack Maintenance Policy
Global RAID-1 Disk Policy
Global MAC Pool (size 10)
For any resources created, you may use your own naming convention.

Create an updating Global Service Profile Template leveraging the global policies above called central-sp-templ with
the following attributes:
Dual redundant vNICs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 257
Agenda

Introduction
L2/L3 Technologies
DC Storage and Compute
CCIE Next Gen
Automation and Orchestration
Fabric Infrastructure
Study, Preparation and Closing
Application Centric Infrastructure

Its time to ad just everything you thought


you knew about networking.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 259
ACI primer and demo exam items.
Agenda
ACI terminology recap
ACI fabric hierarchy
ACI fabric constructs review
ACI fabric constructs
FEX and VPC in ACI
L3 out
Contracts refresher
Demos:
FEX and VPC
L3 out route leaking
Contracts
Verifications
Applications What are we talking about here?
Consider the Interaction between the endpoints
Web App DB
External QoS QoS QoS
Network Filter Service Filter

ACI Fabric
Non-Blocking Penalty Free Overlay

APIC
APIC
APIC

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 262
Remember UCS & Stateless Computing?
Service Profile

Storage Server Network


Identity (UUID) Uplinks
Optional Disk usage
Adapters
SAN settings LAN settings
Number
LUNs VLAN
Type: FC, Ethernet
QoS
Persistent Binding Identity
Characteristics etc
SAN settings
Firmware Firmware
vSAN
Revisions Revisions
Firmware
Configuration settings
Revisions

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 263
Enter Stateless Application Policies
Application
Profile
QoS QoS QoS

Service
Service

Filter
EPG Web Service

Filter
EPG App Filter
EPG DB

End Points Network & Security L4 L7 Services


Single or Device Groups Quality of Service (QoS) Firewalls
Virtual / Physical Contracts & Filters (TCP/UDP) Load Balancers
Single/Multiple Subnets Redirection Orchestration & Management
Health Monitoring SPAN & Monitoring Network Analysis

There is stateless filtering between End Point Groups (EPGs) that may be
able to eliminate the need for some firewalls within the datacenter. Contracts
define what an EPG exposes to other app tiers and how. In other words,
any communication not explicitly allowed, is denied.
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 264
ACI terminology recap
EPG (end point group) A group of devices that we want to apply like policy to.
IE: web servers, application servers, database servers.
L2out and L3out is how we extend layer 2 and layer 3 into and out of an ACI
fabric.
Contract, how we define what traffic is permitted between EPGs.
Fabric Access Policies, how we define the physical connectivity for devices to
connect to the fabric.
Tenant, is a logical container for 1 or more Application Profiles.
Application Profile a logical container where we define EPGs and the contracts
they consume and provide.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 265
ACI terminology recap (cont.)
BD (bridge domain) a forwarding domain used by ACI when deciding how to
forward a packet.
VRF (virtual routing facility) a layer 3 routing domain.
Domains how ACI locally sees things like physical servers, virtual machines,
external L2 switches, and external L3 routers.
Route Leaking term used when we inject routes from one VRF into another
VRF.
Provider/Consumer this is how the directionality of a contract is referenced. IE:
EPG-A can provide contract X and EPG-B can consume contract-X.
vzAny special contract that applies to all EPGs that reside under the same VRF.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 266
ACI fabric policy hierarchy

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 267
ACI fabric constructs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 268
Sample FEX and vPC topology
9508 SPINE 201 9508 SPINE 202

FAB1 Leaf 101 FAB1 Leaf 103


VPC peers

1/36 1/36
BD = 192.168.1.254/24 PC
PC
FEX 2232PP FEX 2232PP
FEX 111 FEX 122
111/1/15 111/1/10 122/1/10
PC
PC
Bare Metal
Server 1 2
vPC
Orphan device Bare Metal Server

2 port 10G NIC


IP = 192.168.1.100/24

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 269
FEX and vPC highlights
Fabric EXtender and Virtual Port-Channels can be use together or individually
within ACI.
The FEX attachment is via straight through port-channel even if single link is
used.
vPC from leaf port to the FEX is not supported.
FEX ports are supported for endpoint only, not for L2 or L3 out attachments.
vPC is supported for L2 and L3 attachment (leaf ports only, not FEX ports)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 270
An L3 out is a construct that represents external IP connectivity.

Inside

Border
leaves

Routed
Outside
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 271
L3 out high level points
OSPF, eBGP, iBGP, EIGRP, and static routing are supported for L3 out.
SVI, routed, and routed sub-interfaces are supported on the border leaf interface
type. No support for L3 port-channel. Use SVI over L2 port channel.
An internal VRF is extended out of the fabric using L3 out. BD subnets can be
advertised out.
An L3 out is an EPG (end point group), and must consume/provide a contract
before it can be used by internal endpoints.
No internal endpoints can be in the L3 out EPG.
We can leak routes learned in the extended VRF into non-extended VRFs.
It is possible to transit route (learn routes on VRF-A via L3 and leak them into
VRF-B and advertise them out via L3)
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 272
Contracts refresher
Used between EPGs to permit or deny traffic. (Think access-list)
Comprised of subjects and filters, and are applied with varying levels of usability.
IE: within a tenant, within a VRF, within an Application Profile, or global.
Subjects are used to provide directionality of filter between consumer and
provider. Contracts are only used if VRF(s) are in enforced mode.
Filters are use to determine specifically what is permitted.
Example: contract
Filter = from TCP port any to TCP port 22 (ssh)
Subject = apply both directions and reverse filter ports = true
EPG-A provides contract, EPG-B consumes contract
EP in EPG-B can open SSH to EP in EPG-A however EP in EPG-A can not open SSH to EP in EPG-B.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 273
Contracts Enable Inter-EPG Communication

Tenant
Application Profile
C EPG Web C EPG App C EPG DB

Contracts Group of Subjects. Define Scope (Global, Tenant, AP)

Subjects Group of Filters. Unidirectional / Bi-direction,


QoS & Service Graph Insertion Point
Filters Lowest Level ACL Entries

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 274
Contracts GUI View

Contracts

EPGs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 275
VLAN Normalization
Understanding FD and Encap VLANs
Once a packet enters the Fabric and it gets classified into an EPG, the Encap (wire) VLAN is no longer relevant. (Encap
VLAN only important when enter/exiting fabric)

Encap VLANs get mapped to a System/FD VLAN which are switch specific!

Therefore you need to know how to identify each to understand & verify if an EP has been learned within the correct EPG

Lets take a look at this endpoint

Leaf1 Leaf3
System VLAN = 17 System VLAN = 9
Encap VLAN = 100 Encap VLAN = 100
Eth1/15

EP-A EP-B CCIE


EPG

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 277
Verifying FD and Encap VLANs
leaf1# show vlan extended

VLAN Name Status Ports

---- -------------------------------- --------- -------------------------------

13 infra:default active Eth1/1, Eth1/15, Eth1/16,

Eth1/47, Eth1/48, Po2, Po3, Po4

14 tenant1:bd1 active Eth1/31, Po1

16 ccie:bd1 active Eth1/15

17 ccie:App1:ccie active Eth1/15

<snip>

System
Tenant App Profile EPG Programmed
VLAN Interfaces
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 278
Verifying FD and Encap VLANs
<contd>

VLAN Type Vlan-mode Encap

---- ----- ---------- -------------------------------

13 enet CE vxlan-16777209, vlan-4093

14 enet CE vxlan-14811120, vlan-200

16 enet CE vxlan-15105997

17 enet CE vlan-100

leaf1#

System Enap/Wire
VLAN VLAN

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 279
Verifying FD and Encap VLANs
I know the System & Encap VLAN for my Endpoint, now what?
Check the MAC table for the EP
leaf1# show mac address-table vlan 17

Legend:

* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC

age - seconds since last seen,+ - primary entry using vPC Peer-Link,

(T) - True, (F) - False

VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID

---------+-----------------+--------+---------+------+----+------------------

* 17 0050.5680.fe11 dynamic - F F eth1/15

leaf1#

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 280
VMM Integration
Topology for exam item demo
ACI ACI ACI ACI
LEAF-1 LEAF-2 LEAF-3 LEAF-1
1/45-48 1/45-48 1/17 1/14

1/17 3/15
FEX FEX
112 113 N5K N7K

HOST

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 282
Hypervisor Integration with ACI
Control Channel - VMM Domains
Relationship is formed between
APIC and Virtual Machine Manager
(VMM)
Multiple VMMs likely on a single
ACI Fabric
Each VMM and associated Virtual
hosts are grouped within APIC
vCenter DVS vCenter AVS SCVMM
Called VMM Domain
There is 1:1 relationship between a
VMM Domain 1 VMM Domain 2 VMM Domain 3 Virtual Switch and VMM Domain

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 283
VMware Integration
Three Different Options
Distributed Virtual Switch Application Virtual Switch
vCenter + vShield
(DVS) (AVS)

Encapsulations: VLAN Encapsulations: VLAN, Encapsulations: VLAN,


Installation: Native VXLAN VXLAN
Installation: Native Installation: VIB through
VM discovery:
VUM or Console
LLDP/CDP VM discovery:
LLDP/CDP VM discovery: OpFlex
Software/Licenses:
vCenter with Software/Licenses: Software/Licenses:
Enterprise+ License vCenter with vCenter with
Enterprise+ License, Enterprise+ License
vShield Manager with
vShield License

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 284
ACI Policies Used in VMM Integration
Which Switches

Interface
Configuration

Which Interfaces
Which EPG

Logical &
Physical
Connector
Which VMM
Which VLANs

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 285
Application Virtual Switch (AVS)
Integration Overview

OpFlex Control protocol


- Control channel Hypervisor
- VM attach/detach, link state notifications Manager

VEM extension to the fabric Southbound vSphere

vSphere 5.1 and above OpFlex API

BPDU Filter/BPDU Guard


SPAN/ERSPAN VM VM VM VM

Microsegmentation (uSeg)
Port level stats collection N1KV VEM

Remote Virtual Leaf Support


(future)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 286
Demos
FEX sample exam item demo
You have been asked to add FEX 113 into the fabric, and make the fabric
ready for a new host being added. The host will be attached to FEX 112
and 113 and will the host NIC will be configured for active/active teaming.
The Leaf/host port configuration will be performed at a later time.

We need to add FEX 113. The diagram shows FEX 113 is connected to leaf 3, ports 1/45-48 and
they are in a port-channel. This is MUST for a FEX attachment. Even if the FEX is attached with 1
link to the leaf, it must be a 1 link port-channel.
SecondThe NIC will be configured for active/active teaming, but connected to 2 different devices
on the other end. Those being FEX 112 and FEX 113. We must configure the leafs to be in a virtual
port-channel pair. There is no mention of vPC domain ID, so we are free to choose the value.
There is no need to configure the host ports as a vPC as the exam item only mentions adding the
FEX and making the fabric ready for vPC.
The item does not mention any names for access policies, only that the FEX id is 113.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 288
OSPF sample exam item demo
Your customer is reporting issues with an OSPF connection between leaf
4 and an N5K. You must get the OSPF neighbor up. The L3 out is in the
common tenant. The 192.168.101.0/24 subnet on the bd-101 Bridge
Domain in CCIE-demo tenant should be visible in the N5K vrf ccie
routing table. You may not make any changes to the N5K, create any new
contracts in ACI, or use contract vzAny. All VRFs used here must remain
in enforced mode.

We can look at the N5K, but are not allowed to alter its configuration. Make note of int E1/17 and
OSPF configuration for VRF ccie.
We need to advertise the noted BD route, this may indicate we need to route leak as they also
mention multiple VRFs.
We can not create new contracts or use vzAny, so we need to use an existing contract.
We are looking at 2 tenants per the item, common and CCIE-demo, this further adds that route
leaking might be needed, and a unique type of contract might be needed.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 289
Contract exam item demo
Endpoints in tenant CCIE-demos, epg2 need to open only HTTP sessions
to endpoints in epg3. Endpoints in epg3 must not be able to open HTTP
sessions to endpoints in epg2. Use contract name web and filter name
http for the contract. Limit the contract to use within the Tenant only.

We have been told what names to use for the contract and filter, but not the subject name. This
indicates we can use any name for the subject we desire.
Directionality has been specified, so we need to ensure our filter and subject are correctly
configured for only one specific TCP port to be opened in a specific direction.
HTTP sessions only means we are limited to what we are allowed to permit in the contract.
We were told to limit the contract use to the Tenant only.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 290
Verifications
show fex
show port-channel database
show port-channel summary
show vpc
show ip route vrf common:default
show ip route vrf neighbors vrf all
Faults, faults, faultscheck for faults!

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 291
Agenda

Introduction
L2/L3 Technologies
DC Storage and Compute
CCIE Next Gen
Automation and Orchestration
Fabric Infrastructure
Study, Preparation and Closing
Preparation and Study
Keeping your Eye on the Prize
Be prepared to commit to at least 4-8 months

Studying becomes a work/life commitment

Home Lab where possible (N1K, UCSPE, VIRL)

Hands on Experience is a MUST (Remote labs included)

Plan your success!


Set milestones/goals and do whats needs to be done to achieve them.
Pop Quiz Next

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 294
Pop Quiz
Count the # of Fs on this page

FINISHED FILES ARE OFTEN THE


RESULT OF YEARS OF SCIENTIFIC
STUDY COMBINED WITH THE
EXPERIENCE OF YEARS
How many did you count?
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 295
Pop Quiz
Count the # of Fs on this page

FINISHED FILES ARE OFTEN THE


RESULT OF YEARS OF SCIENTIFIC
STUDY COMBINED WITH THE
EXPERIENCE OF YEARS
How many did you count?
TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 296
What does it take to pass a CCIE Lab Exam?
Skills

Technical Compentancy

Time Management

Knowing Where to find


information
Attention to Detail

Toubleshooting Skills

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 297
Lab Exam: Tips & Tricks
Before the exam
Prepare for the exam!
Plan your study
Do self assessment, know what I dont know
Dedicate time per day
Always ask What if
Practice, practice and practice

Learn how to browse on Cisco Documentation (sort, dont search)


Choose materials from trustworthy source
Practice for speed and troubleshooting

Build a study plan that works for you

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 298
Dont do it alone
There are many groups, forums and study groups available.
95% of successful CCIEs participate in a study group of some form.
On going groups available including Learning @ Cisco etc

If you cant team up locally, do it virtually

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 299
Lab Exam: Tips & Tricks
Day Before the Lab Exam
Arrive the day prior, if you have to travel
Check Visa requirement in advanced
Survey the lab location
Know exactly how to get to the office
Plan the trip to the lab location
train timetable
book a taxi
etc

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 300
Lab Exam: Tips & Tricks
Night Before the Big Day
Have a good dinner
Have a good sleep
Do whatever you enjoy
Ensure readiness in both mind
and body
Mental Readiness for the BIG DAY

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 301
Lab Exam: Tips & Tricks THE BIG DAY

Take time for Breakfast. Most important Meal of the Day.


Reduce stress, arrive early and prepare IDs!
Listen to the proctors guidelines
Re-draw the topology: physical and logical(if needed)
Manage your time! Stick to your strategy!
Read the whole module, dont forget the guidelines!
Read, read and read the questions before asking for clarification from the proctor
Save the configuration often!
Avoid last minute change!
Plan for regression tests and overall validations at the end of each module!

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 302
What Happens if I Get Stuck???
If you get into a question and hit a wall (not sure what to do), make a note, move
on and come back to it.
Lab Exams are composed of multiple questions and multiple tasks. Weigh the
score value against the time invested. Sometimes its better to skip a question
and focus on the rest.
Some questions may affect others. Many lab scenarios are treated as a
datacenter solution questions may have an impact on other outcome of
another.

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 303
A Note on Lab Proctors
Proctors are there to run the exam
They are not there to help you on any technically related questions

A Proctor will:
Clarify a Question
Deal with Hardware Issues if encountered

A Proctor will not:


Solve or Troubleshoot Configuration Issues
Answer questions on how to configure devices (Confirming good/bad configuration)
Answer Questions regarding a choice of how to configure something

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 304
Lab Exam: Tips & Tricks
Aftermath
If you pass
CONGRATULATION!

If you fail
Release the anger! Do whatever you have to do
Try to switch from Denial to Curious quickly
Start looking for your mistakes
Repeat the scenarios in your own lab
Back to lab practice focusing on the failed scenarios
Book the next lab exam in 4 weeks time.
Even some of the best TAC engineers require multiple attempts!
If you are 100% sure the CCIE Program team is wrong ask for review*
*Additional Costs involved TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 305
Available Resources
Learning @ Cisco Forum for asking questions, support and free online
resources such as webinars and other virtual events
https://learningnetwork.cisco.com/community/certifications/ccie_data_center
Recommended Reading List
https://learningnetwork.cisco.com/docs/DOC-13986
Recommended Training
https://learningnetwork.cisco.com/docs/DOC-13985
Online Resources
https://learningnetwork.cisco.com/docs/DOC-13987
Other Courses
http://www.cisco.com/web/learning/le31/ase/offerings/datacenter/index.html

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 306
Got a question after the session?
Join the CCIE Data Center Study Group on CLN
https://learningnetwork.cisco.com/groups/ccie-data-center-study-group
Ask technical questions
Find study partner(s)

Open a CertSupport case at http://www.cisco.com/go/certsupport

Send me an email at munawaz@cisco.com

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 307
Cisco Certifications SME* Recruitment Program
http://www.cisco.com/go/certsme
Directly influence Cisco Career Certifications (Design, Author, Review)
Give back to community
Experience with assessment techniques Apply
Now!
Join creativity with experience, knowledge and skills
Use and sharpen technical expertise
Collaborate and network with other engineers

SME= Subject Matter Expert

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 308
Complete Your Online Session Evaluation
Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 Amazon gift card.
Complete your session surveys
through the Cisco Live mobile
app or from the Session Catalog
on CiscoLive.com/us.

Dont forget: Cisco Live sessions will be available


for viewing on-demand after the event at
CiscoLive.com/Online

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 309
Continue Your Education
Demos in the Cisco campus
Walk-in Self-Paced Labs(Add relevant Labs here)
Lunch & Learn
Meet the Engineer 1:1 meetings
Related sessions(Add Related Session Here)

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 310
Please join us for the Service Provider Innovation Talk featuring:
Yvette Kanouff | Senior Vice President and General Manager, SP Business
Joe Cozzolino | Senior Vice President, Cisco Services

Thursday, July 14th, 2016


11:30 am - 12:30pm, In the Oceanside A room

What to expect from this innovation talk


Insights on market trends and forecasts
Preview of key technologies and capabilities
Innovative demonstrations of the latest and greatest products
Better understanding of how Cisco can help you succeed

Register to attend the session live now or


watch the broadcast on cisco.com
Q&A
Thank you
Data Center / Virtualization Cisco Education
Offerings
Course Description Cisco Certification
Introducing Cisco Data Center Networking (DCICN); Learn basic data center technologies and skills to build a CCNA Data Center
Introducing Cisco Data Center Technologies (DCICT) data center infrastructure.

Implementing Cisco Data Center Unified Fabric (DCUFI); Obtain professional level skills to design, configure, CCNP Data Center
Implementing Cisco Data Center Unified Computing (DCUCI) implement, troubleshoot data center network infrastructure.
Designing Cisco Data Center Unified Computing (DCUDC)
Designing Cisco Data Center Unified Fabric (DCUFD)
Troubleshooting Cisco Data Center Unified Computing
(DCUCT)
Troubleshooting Cisco Data Center Unified Fabric (DCUFT)

Product Training Portfolio: DCNMM, DCAC9K, DCINX9K, Gain hands-on skills using Cisco solutions to configure,
DCMDS, DCUCS, DCNX1K, DCNX5K, DCNX7K deploy, manage and troubleshoot unified computing, policy-
driven and virtualized data center network infrastructure.

Designing the FlexPod Solution (FPDESIGN); Learn how to design, implement and administer FlexPod Cisco and NetApp Certified
Implementing and Administering the FlexPod Solution solutions FlexPod Specialist
(FPIMPADM)

For more details, please visit: http://learningnetwork.cisco.com


Questions? Visit the Learning@Cisco Booth or contact ask-edu-pm-dcv@cisco.com

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 315
Cloud Cisco Education Offerings
Course Description Cisco Certification
Understanding Cloud Fundamentals Learn how to perform foundational tasks related to Cloud computing, and the essentials
(CLDFND) of Cloud infrastructure
CCNA Cloud
Introducing Cloud Administration Learn the essentials of Cloud administration and operations, including how to provision,
(CLDADM) manage, monitor, report and remediate.

Implementing and Troubleshooting the Learn how to implement and troubleshoot Cisco Cloud infrastructure: compute,
Cisco Cloud Infrastructure (CLDINF) network, storage.

Learn how to design private and hybrid Clouds including infrastructure, automation,
Designing the Cisco Cloud (CLDDES)*
security and virtual network services
CCNP Cloud
Automating the Cisco Enterprise Cloud Learn how to automate Cloud deployments provisioning IaaS (private, private with
(CLDAUT)* network automation and hybrid) and applications, life cycle management
Building the Cisco Cloud with Application Learn how to build Cloud infrastructures based on Cisco Application Centric
Centric Infrastructure (CLDACI)* Infrastructure, including design, implementation and automation

Learn how to manage physical and virtual infrastructure using orchestration and
UCS Director Foundation (UCSDF)
automation functions of UCS Director.

* Available Q2CY2016

For more details, please visit: http://learningnetwork.cisco.com


Questions? Visit the Learning@Cisco Booth or contact ask-edu-pm-dcv@cisco.com

TECCCIE-3644 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 316