Académique Documents
Professionnel Documents
Culture Documents
Architecture
RST-3465
RST-3465
12523_04_2006_c2 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 1
Session Goal
To provide you with a thorough
understanding of the Catalyst® 6500
switching architecture, packet flow, and
key forwarding engine functions
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 2 2
Agenda
• Chassis Architecture
• Supervisor Engine and Switch Fabric Architecture
• Switching Module Architecture
• Layer 2 Forwarding
• IPv4 Forwarding
• IPv4 Multicast Forwarding
• Security and Feature ACLs
• QoS
• NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 3
Chassis Architecture
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 4
Catalyst 6500 Chassis Architecture
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 5
Catalyst 6503/6503E and 6504E
• Slots 1 and 2—Supervisor
engine, or switching module
Power Power
• Other slots—Any switching module Supply Supply
• 2 fabric channels per slot
VTT/Clock Modules EEPROMs
• Power supplies in rear
6503/6503E—Power entry modules (PEMs) in Dual Channels Slot 1
Fan Tray
front of chassis provides power connection
Dual Channels Slot 2
• 950W AC/DC and 1400W AC power Dual Channels Slot 3
supplies for 6503/6503E
Dual Channels Slot 4
• 2700W AC/DC power supplies for
6504E Crossbar Shared Bus
5 RU
4 RU
Note: CEF720 modules
not supported in
Catalyst 6503 (non-E) chassis
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 6
Catalyst 6506/6509 and 6506E/6509E
• Slots 1 and 2—Supervisor Engine 2, or VTT/Clock Modules EEPROMs
switching module
• Slots 5 and 6—Supervisor Engine 32/720, or Dual Channels Slot 1
switching module Dual Channels Slot 2
• Other slots—Any switching module
Dual Channels Slot 3
• 2 fabric channels per slot
Fan Tray
Dual Channels Slot 4
• Wide variety of power supplies, from legacy
1000W to new 6000W—E chassis requires at Dual Channels Slot 5
least 2500W PS Dual Channels Slot 6
• NEB-A chassis has vertical slot
alignment, dual fan trays, front-to-back air flow, Dual Channels Slot 7
air filtration system Dual Channels Slot 8
Dual Channels Slot 9
21 RU Crossbar Shared Bus
Power Power
Supply Supply
12 RU
15 RU
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 7
Catalyst 6513
VTT/Clock Modules EEPROMs
• Slots 1 and 2—Supervisor Single Channel Slot 1
Engine 2, or switching module
Single Channel Slot 2
• Slots 7 and 8—Supervisor Single Channel Slot 3
Engine 32/720, or switching
module Single Channel Slot 4
Fan Tray
Single Channel Slot 6
from 2500W to
Single Channel Slot 7
new 6000W
Single Channel Slot 8
• 1 fabric channel
slots 1–8 Dual Channels Slot 9
Dual-fabric modules Dual Channels Slot 10
not supported in
slots 1–8! Dual Channels Slot 11
• 2 fabric 19 RU Dual Channels Slot 12
channels Dual Channels Slot 13
slots 9–13 Crossbar Shared Bus
Any switching module
Power Power
RST-3465 Supply Supply
Cisco Public
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. 8
Agenda
• Chassis Architecture
• Supervisor Engine and Switch Fabric Architecture
• Switching Module Architecture
• Layer 2 Forwarding
• IPv4 Forwarding
• IPv4 Multicast Forwarding
• Security and Feature ACLs
• QoS
• NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 9
Supervisor Engine and Switch Fabric
Architecture
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 10
Supervisor 2
• PFC2 forwarding engine • Internal RP and SP bootflash
daughter card (32MB each)
• Switch Processor CPU (300MHz • External PCMCIA flash slot
R7000)
• Supports optional Switch
• Optional MSFC2 daughter card Fabric Module (SFM)/SFM2
with Route Processor CPU
• 2 x 1GE GBIC uplink ports
(300MHz R7000)
• 256MB/256MB (Sup2) or
256MB/512MB (Sup2U) DRAM
8 Gbps
MET
multicast/
SPAN
L2 CAM DBUS
contains 16 Gbps
RBUS To SFM/SFM2 Interface to
Bus fabric and bus
MAC entries
RST-3465 EOBC
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 12
Supervisor 720
• 720Gbps crossbar fabric • 512/512MB (3A/B) or 1/1GB
(3BXL) DRAM
• PFC3 forwarding engine
daughter card • Internal RP and SP bootflash
(64MB each)
• Integrated RP/SP CPUs on
MSFC3 daughter card (600MHz • Optional 512MB CF bootflash
MIPS) upgrade for SP (WS-CF-UPG=)
• Dual external compact flash
slots
• 2 x GbE uplink ports—
2 x SFP <or>
1 x SFP and 1 x 10/100/1000
GbE Uplinks
Supervisor 720 Baseboard CPU Daughter Card
Counter QoS FIB 1 Gbps RP (MSFC3) DRAM
FPGA ADJ
TCAM TCAM Port ASIC CPU
(B/BXL Only)
1 Gbps SP (NMP) DRAM
ACL L3/4 NetFlow CPU
TCAM Engine MET
Fabric Integrated
L2 Engine PFC3
20 Gbps 720 Gbps
L2 Daughter Interface/
Replication Switch Fabric
CAM Card
Engine
17 x 20 Gbps
Fabric
…
Channels
L2 CAM moved Crossbar switch
DBUS
on-chip for RBUS fabric integrated
higher 16 Gbps EOBC on supervisor
Bus baseboard
RST-3465
performance
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 14
Supervisor 32
• Classic supervisor—no fabric, uses
16Gig bus only
• PFC3B forwarding engine daughter
card
• SP CPU (400MHz Sibyte) 2 10GE Xenpak +
1 10/100/1000 RJ-45 uplink ports
• MSFC2a routing engine
• 256MB/256MB DRAM (512MB/512MB
with non-$0 feature set)
• Internal CF bootdisk (256MB) and
MSFC2A bootflash (64MB)
• External CF slot
• Uplink options:
8 1GE SFP +
8 SFP + 1 10/100/1000 1 10/100/1000 RJ-45
uplink ports
2 10GE + 1 10/100/1000
L2 Engine PFC3
L2 Daughter Replication
CAM Card Engine
MET WS-SUP32-GE-3B
DBUS
16 Gbps
RBUS Bus attached only;
EOBC
Bus no fabric support
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 16
Supervisor 32-10GE / PFC3 Architecture
Dual port ASICs to
support two 10GE
interfaces
MET WS-SUP32-10GE-3B
DBUS
RBUS
16 Gbps EOBC
Bus
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 17
Supervisor Chassis Requirements
Supervisor 720 and Supervisor 32 require:
• Catalyst 6500 or 6500-E chassis
• High speed fan tray (FAN2/E-FAN)
• 2500W power supply (AC or DC) or greater
3000W supply recommended for new deployments
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 18
Crossbar Switch Fabric
• Provides multiple conflict-free paths between switching
modules
Dedicated bandwidth per slot
Compare to system bus which is shared by all bus-attached modules
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 19
Switch Fabric Module and SFM2
• 256 Gbps crossbar switch fabric
• Works with Supervisor 2 and CEF256/dCEF256 modules
• Fabric channels run at 8 Gbps full duplex
8 Gbps in/8 Gbps out per channel
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 20
Supervisor 720 Switch Fabric
• 720 Gbps crossbar switch fabric
• Integrated on Supervisor 720 baseboard
• Fabric channels run at 20 Gbps
full duplex
20 Gbps in/20 Gbps out per channel
• Works with all fabric-capable modules
Fabric channels auto-sync speed on
per-slot basis (8 Gbps or 20Gbps)
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 21
Monitoring Fabric Status and Utilization
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 23
Policy Feature Cards
• Mandatory daughter card for supervisor engine
• Provides the key components enabling high-performance
hardware packet processing
• Supervisor 2 supports PFC2
• Supervisor 32 supports PFC3B
• Supervisor 720 supports:
PFC3A
PFC3B
PFC3BXL
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 24
Policy Feature Cards (Cont.)
Key hardware-enabled features:
• Layer 2 switching
• IPv4 unicast forwarding
• IPv4 multicast forwarding
• Security ACLs
• QoS/policing
• NetFlow accounting
PFC3 also supports:
• IPv6, MPLS*/VRF-lite, Bidir PIM, NAT/PAT, GRE/v6
tunnels, CoPP
RST-3465
* MPLS on 3B/3BXL only
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 25
High-Level Forwarding Engine Logic
Frame Input Layer 2 Ingress
received lookup FIB TCAM Forwarding
Engine
Layer 2 Table FIB lookup
ACL TCAM
Router Yes Input QoS Input ACL
MAC? lookup lookup
NetFlow lookup
Input QoS
lookup
QoS TCAM
RST-3465
NetFlow Table QoS TCAM ACL TCAM *PFC3 only
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 26
PFC TCAM Technology
• TCAM—Ternary Content Value 1
Addressable Memory Value 2
Mask 1 Value 3
• Leveraged heavily in Catalyst
Value 4
6500
Value 5
FIB, ACL, QoS, NetFlow all utilize Value 6
TCAM memory
Value 7
• All entries accessed in Value 8
parallel—fixed performance
Value 1
independent of number of
Value 2
entries
Mask 2 Value 3
• Memory consists of groups of Value 4
values and associated masks
Value 5
8:1 ratio of values to masks
Value 6
• Masks are used to “wildcard” Value 7
some portion of values
Value 8
Masks Values
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 27
Generic TCAM Lookup Logic
1 3 Compare
Generate
Packet Fields Lookup
Key
01101010
011010xx
0110xxxx 110110xx 1
Lookup Key 000111xx 2
2
11111100 101101xx 3
100111xx 4
000000xx 5
1. Relevant fields read from 010010xx 6
contents of packet 1=“Compare”
111111xx 7
0=“Mask”
2. Lookup key created 001100xx 8
0011xxxx 6
0000xxxx 7
1000xxxx 8
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved.
Masks Values Cisco Public 28
Agenda
• Chassis Architecture
• Supervisor Engine and Switch Fabric Architecture
• Switching Module Architecture
• Layer 2 Forwarding
• IPv4 Forwarding
• IPv4 Multicast Forwarding
• Security and Feature ACLs
• QoS
• NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 29
Switching Module Architecture
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 30
Classic Module
DBUS
Example: WS-X6416-GBIC
RBUS
Classic Module
DBUS
Port ASICs for physical RBUS
connectivity, buffering,
and queueing Classic Module
Port
ASIC
48x10/100
Example: WS-X6148A-RJ-45
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 31
CEF256 Module
Example: WS-X6516-GBIC
CEF256
Module
Fabric
Interface
Replication
MET
Engine Port Port Port Port
ASIC ASIC ASIC ASIC
4xGE 4xGE 4xGE 4xGE
LCDBUS
LCRBUS
Replication
MET
Engine Port Port Port Port
ASIC ASIC ASIC ASIC
4xGE 4xGE 4xGE 4xGE
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 33
CEF720 Module
Example: WS-X6748-SFP
DBUS
20Gbps Fabric 20Gbps Fabric
RBUS
Channel Channel
Combined fabric
interface and
replication engine
Transparent bus
interface
Layer3/4
Engine for
FIB/Adj, ACL, Layer 2
QoS and Engine
NetFlow for L2
lookups lookups
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 35
Distributed Forwarding
• One or more modules have local forwarding engine
(DFC—Distributed Forwarding Card)
• Central engine and distributed engines perform different lookups
independently and simultaneously
• Implementation is fully distributed
All hardware from PFC is present on the DFC
Full Layer 2, Layer 3, ACL/QoS information downloaded from Supervisor
Ingress DFC performs all lookups locally
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 36
Distributed Forwarding Cards
• DFCs work in conjunction with specific
supervisor
DFC works with PFC2 on Supervisor 2
DFC3A/3B/3BXL works with PFC3 on Supervisor 720
• PFC/DFC “major” module version must be identical
PFC/DFC “minor” module version mismatch supported in lowest common denominator mode
Example: System with PFC3B and DFC3As runs in PFC3A mode
• DFC is optional daughter card for CEF256 modules
• DFC3 is optional daughter card for CEF256/CEF720 modules
Several flavors and form factors available
• WS-X6816-GBIC module REQUIRES either DFC or DFC3
• Local CPU for managing hardware tables
• Use remote login module command to access DFC console
Commands available on DFC console for troubleshooting use, under direction from Cisco
TAC/escalation
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 37
Centralized Forwarding
Port Port4
Classic
L2 Engine ASIC ASIC
Module B
2
3 PFC3
DBUS
RBUS
Source S
Classic Port Port Destination D
Module A ASIC ASIC Blue VLAN
1
Red VLAN
Blue S
Entire Packet
Packet Header
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 38
Centralized Forwarding with Fabric
Red
D
L2 Engine 720Gbps
Switch 6
Fabric
3 8Gbps CEF256
PFC3 Fabric Interface
Module B
DBUS
RBUS
2
Source S
Fabric 5 8Gbps CEF256 Destination D
Interface
4 Module A
Blue VLAN
LCDBUS Red VLAN
LCRBUS
Entire Packet
Port Port
ASIC ASIC Packet Header
1
Blue S
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 39
Distributed Forwarding
Red
D
Port Port
CEF720
ASIC ASIC DFC3
Supervisor Engine 720 L3/4 Module B
Engine w/DFC3
5
720Gbps Fabric Interface/
PFC3 Switch 20Gbps Replication Layer 2
Fabric Engine Engine
20Gbps
CEF720 Source S
4 Module A
Fabric Interface/ 2 Layer 2 Destination D
w/DFC3
Replication 3Engine Blue VLAN
Engine Red VLAN
L3/4
Port Port Engine
DFC3 Entire Packet
ASIC ASIC
Packet Header
1
Blue S
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 40
Agenda
• Chassis Architecture
• Supervisor Engine and Switch Fabric Architecture
• Switching Module Architecture
• Layer 2 Forwarding
• IPv4 Forwarding
• IPv4 Multicast Forwarding
• Security and Feature ACLs
• QoS
• NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 41
Layer 2 Forwarding
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 42
Layer 2 Lookups
Frame Input Layer 2 Ingress
received lookup FIB TCAM Forwarding
Engine
Layer 2 Table FIB lookup
ACL TCAM
Router Yes Input QoS Input ACL
MAC? lookup lookup
NetFlow lookup
Input QoS
lookup
QoS TCAM
RST-3465
NetFlow Table QoS TCAM ACL TCAM
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 43
Layer 2 Forwarding
• Layer 2 forwarding based on {VLAN, MAC} pairs
Same MAC can be learned in multiple VLANs
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 44
Layer 2 Forwarding Logic
Frame received
L2 flooding
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 45
Layer 2 Forwarding Table Design
PFC2
PFC3
16384 rows
MAC Table
8 pages
RST-3465
MAC Table 16K*8=128K entries 4K*16=64K entries
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 46
PFC2 Layer 2 Lookup
Frame
1
16384 rows
Lookup Key
2
VLAN MAC Address
10 | 0000.aaaa.aaaa
Destination
5 4000 interface(s)
40| 20
3233.1111.3333
| 0000.1111.2222
| 0000.cccc.cccc
Compare 111 |
3999 9000.8000.7000
| 9090.9090.9090
100 DMAC lookup
3 10| 0000.1111.1111
| 0000.bbbb.bbbb
2101 |
44444 4334.5445.6556
| 6666.6666.6666
| 0100.5e01.0101
10 | 0000.aaaa.aaaa 6
Hash Function HIT!
44 | 2468.ace0.2468
30 | 0000.dddd.dddd SMAC lookup
Update
Starting Page Entry
and Row 4
Lookup Key
2
VLAN MAC Address
10 | 0000.aaaa.aaaa
DMAC lookup
Destination
5 interface(s)
3 Compare
Hash Function
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 49
Agenda
• Chassis Architecture
• Supervisor Engine and Switch Fabric Architecture
• Switching Module Architecture
• Layer 2 Forwarding
• IPv4 Forwarding
• IPv4 Multicast Forwarding
• Security and Feature ACLs
• QoS
• NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 50
IPv4 Forwarding
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 51
IPv4 Lookups
Frame Input Layer 2 Ingress
received lookup FIB TCAM Forwarding
Engine
Layer 2 Table FIB lookup
ACL TCAM
Router Yes Input QoS Input ACL
MAC? lookup lookup
NetFlow lookup
Input QoS
lookup
QoS TCAM
RST-3465
NetFlow Table QoS TCAM ACL TCAM
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 52
Hardware-Based CEF
• Catalyst 6500 leverages existing software Cisco Express
Forwarding (CEF) model
• Supervisor 2, Supervisor 32, Supervisor 720 extend CEF to
hardware
• What is CEF, in a nutshell?
Boil down the routing table = FIB table
Boil down the ARP table = adjacency table
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 53
Hardware-Based CEF (Cont.)
• Decouples control plane and data plane
Forwarding tables built on control plane
Tables downloaded to hardware for data plane forwarding
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 54
FIB TCAM and Adjacency Entries
FIB:
• IPv4 entries logically arranged from 172.20.45.1
most to least specific
10.1.1.100
• 0/0 default entry terminates unicast MASK (/32)
FIB entries … IF, MACs, MTU
• Overall FIB hardware shared by 10.1.3.0
IF, MACs, MTU
IPv4 unicast 10.1.2.0
IPv4 multicast MASK (/24) IF, MACs, MTU
IPv6 unicast …
IF, MACs, MTU
IPv6 multicast 10.1.0.0
MPLS 172.16.0.0
Adjacency table: MASK (/16)
…
• Hardware adjacency table also Adjacency Table
shared among protocols 0.0.0.0
MASK (/0)
• Actual adjacency table entries are
NOT shared FIB TCAM
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 55
IPv4 FIB TCAM Lookup
Compare 3
Generate
Lookup
Key
10.1.1.10
10.1.1.xx
2 10.1.1.2 1
Lookup Key
DIP 10.1.1.3 2
1 10.1.1.10 FFFFFFFF 10.1.1.4 3 Flow Data
Packet 10.10.0.10 4
10.10.0.100 5
10.10.0.33 6 IF, MACs, MTU
/32 entries 10.100.1.1 Load-Sharing
7
(compare all Hash IF, MACs, MTU
bits) 10.100.1.2 8 Offset
5
6 IF, MACs, MTU
10.1.2.xx 1
10.1.3.xx 2 IF, MACs, MTU
FFFFFF00 10.10.100.xx 3
HIT! 10.1.1.xx 4 Adj Index
10.100.1.xx 45
/24 entries
(mask last Result
10.10.0.xx 6 Adjacency Table
octet)
10.100.1.xx 7
8
RST-3465
12523_04_2006_c1 Masks FIB TCAM
© 2006 Cisco Systems, Inc. All rights reserved. Values Cisco Public 56
Displaying IPv4 Forwarding Summary
Information
6509-neb#
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 57
Displaying Hardware IPv4 Prefix Entries
6509-neb#show mls cef
Codes: decap - Decapsulation, + - Push Label
Index Prefix Adjacency
64 127.0.0.51/32 receive
• Cisco IOS: show mls
65 127.0.0.0/32 receive
66 127.255.255.255/32 receive
cef
67 0.0.0.0/32 receive • Catalyst OS: show mls
68 255.255.255.255/32 receive entry cef ip
75 10.10.1.1/32 receive
76 10.10.1.0/32 receive
77 10.10.1.255/32 receive
78 10.10.1.2/32 Gi1/1, 0030.f272.31fe
3200 224.0.0.0/24 receive
3201 10.10.1.0/24 glean
3202 10.100.0.0/24 Gi1/1, 0030.f272.31fe
3203 10.100.1.0/24 Gi1/1, 0030.f272.31fe
3204 10.100.2.0/24 Gi1/1, 0030.f272.31fe
3205 10.100.3.0/24 Gi1/1, 0030.f272.31fe
<…>
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 58
Displaying Detailed Hardware Entries
• Cisco IOS:
show mls cef <prefix> [detail]
show mls cef adjacency [entry <entry> [detail]]
• Catalyst OS:
show mls entry cef ip <prefix/mask> [adjacency]
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 59
Finding the Longest-Match Prefix Entry
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 60
IPv4 CEF Load Sharing
• Up to 8 hardware load-sharing paths per
prefix
• Use maximum-paths command in routing
protocols to control number of load-sharing 10.10.0.0/16
via Rtr-A
paths via Rtr-B
• IPv4 CEF load-sharing is per-IP flow
• Per-packet load-balancing NOT supported
• Load-sharing based on Source and
Destination IP addresses by default A B
“Unique ID” in PFC3 prevents polarization
• Configuration option supports inclusion of L4
ports in the hash 10.10.0.0/16
mls ip cef load-sharing full
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 61
Load-Sharing Prefix Entry Example
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 62
Identifying the Load-Sharing Path
Interface: Gi1/1, Next Hop: 10.10.1.2, Vlan: 1019, Destination Mac: 0030.f272.31fe
Interface: Gi2/2, Next Hop: 10.40.1.2, Vlan: 1018, Destination Mac: 000d.6550.a8ea
6509-neb#
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 63
IPv4 Unicast RPF Check
6500 Routing Table
Prefix Next Hop Interface
10.255.0.0/16 10.10.1.1 gig 1/1
g1/1 10.20.1.1 gig 1/2
10.30.1.1 gig 2/1
10.40.1.1 gig 2/2
10.20.0.0/16 10.20.1.1 gig 6/3
g1/2
Gotcha: System supports only a global uRPF mode—strict or loose—last configured mode overrides
Gotcha: uRPF with exception ACL not recommended due to software processing
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 64
Verifying uRPF Check Configuration
Index Interfaces
Global uRPF
-------+---------------------------------------- multipath mode
0
1
2 uRPF interface groups
3 (not configured)
6509#show mls cef ip rpf 192.168.1.0
RPF information for prefix 192.168.1.0
uRPF check performed in the hardware for interfaces:
Vlan776
Vlan777 uRPF details for
uRPF check punted to software for interfaces: specific IP prefix
uRPF check disabled for interfaces:
6509#
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 65
Agenda
• Chassis Architecture
• Supervisor Engine and Switch Fabric Architecture
• Switching Module Architecture
• Layer 2 Forwarding
• IPv4 Forwarding
• IPv4 Multicast Forwarding
• Security and Feature ACLs
• QoS
• NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 66
IPv4 Multicast Forwarding
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 67
IPv4 Multicast Lookups
Frame Input Layer 2 Ingress
received lookup FIB TCAM Forwarding
Engine
Layer 2 Table FIB lookup
ACL TCAM
Router Yes Input QoS Input ACL
MAC? lookup lookup
NetFlow lookup
Input QoS
lookup
QoS TCAM
RST-3465
NetFlow Table QoS TCAM ACL TCAM
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 68
IPv4 Multicast Forwarding
• Central and distributed IPv4 multicast
hardware forwarding
• Distributed multicast replication with
appropriate switching modules†
• PIM-SSM and PIM-SM forwarding in
hardware
• BiDir-PIM forwarding in hardware‡
• Off-loads majority of forwarding tasks from
RP CPU
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 70
Multicast Hardware Entries
• FIB
MAC, MET Index
IPv4 multicast entries arranged 172.21.4.19, 225.3.3.3
logically from most to least specific 10.1.44.199, 240.9.8.1 MAC, MET Index
10.1.1.1, 239.1.1.1
• Adjacency table MAC, MET Index
…
Different format than unicast MAC, MET Index
MASK (S,G) /32
Key piece of data is MET index …
*, 234.0.1.1
• MET
MASK BiDir Entries
Contains OILs for multicast routes
… Adjacency Table
Memory resident on replication 10.1.1.0, 224.0.0.0
engines (not PFC/DFC) OIL #1
MASK IF 224/4 Entries
… OIL #2
OIL #3
*, 229.0.1.1
MASK PIM-SM (*,G) /32 OIL #4
FIB TCAM
RST-3465
MET
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 71
Multicast FIB TCAM Lookup
Compare 3
Generate
Lookup
Key
10.1.1.10, 239.1.1.1
10.1.1.10, 239.1.1.1 1
Lookup Key 2
10.1.1.10, 239.1.1.1 2
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 73
Displaying Hardware Multicast
Forwarding Entries
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 75
Security and Feature ACLs
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 76
ACL Lookups
Frame Input Layer 2 Ingress
received lookup FIB TCAM Forwarding
Engine
Layer 2 Table FIB lookup
ACL TCAM
Router Yes Input QoS Input ACL
MAC? lookup lookup
NetFlow lookup
Input QoS
lookup
QoS TCAM
RST-3465
NetFlow Table QoS TCAM ACL TCAM
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 77
Security ACLs
• Enforce security policies based on Layer 2,
Layer 3, and Layer 4 information
• Dedicated ACL TCAM ensures security ACLs
do not affect system performance
• Router ACL (RACL)—Enforced for all traffic
crossing a Layer 3 interface in a specified
direction
IPv4, IPX†, IPv6‡ RACLs supported
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 79
ACL Merge
• Sophisticated feature merge algorithm allows multiple security and
feature ACLs to be applied to a single interface/VLAN
• What is merging?
PFC/DFC hardware supports limited number of ACL lookups on a single packet
May need two or more ACL features on a single interface (e.g., RACL and PBR)
Merge produces ACEs that return correct result in a single lookup
QOS_TCAM
--------
Masks: 7 4089 0 18
Entries: 32 32736 0 144
LOU: 47 81 36
ANDOR: 1 15 6
ORAND: 0 16 0
ADJ: 0 2048 0
6509
- neb#
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 83
Verifying Hardware ACL Enforcement
• show fm summary
6509-neb#show fm summary
Interface: Vlan199 is up
TCAM screening for features: ACTIVE inbound
Interface: Vlan400 is up
TCAM screening for features: ACTIVE inbound
TCAM screening for features: ACTIVE outbound
Interface: Vlan402 is up
TCAM screening for features: ACTIVE inbound
TCAM screening for features: ACTIVE outbound
Interface: Vlan404 is up
TCAM screening for features: INACTIVE inbound
Interface: Vlan405 is up
TCAM screening for features: ACTIVE inbound
6509-neb#
fm = “Feature Manager”
ACTIVE = ACL policy is installed in hardware
INACTIVE = ACL policy is NOT installed in hardware
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 84
Displaying Hardware ACL “Hit Counters”
Cisco IOS: show tcam interface <interface> acl {in | out} ip
6509-neb#show tcam interface vlan199 acl in ip
<…>
permit udp any 10.89.210.0 0.0.0.255 (234265 matches)
permit udp any 10.90.143.0 0.0.0.255 (6860 matches)
permit udp any 10.91.25.0 0.0.0.255 (23 matches)
permit udp any 10.92.82.0 0.0.0.255 (23662 matches)
permit udp any 10.93.154.0 0.0.0.255 (3232 matches)
permit udp any 10.94.1.0 0.0.0.255 (12113 matches)
permit udp any 10.95.109.0 0.0.0.255 (247878 matches)
permit udp any 10.96.201.0 0.0.0.255 (33234 matches)
permit udp any 10.97.16.0 0.0.0.255 (6855 matches)
permit udp any 10.98.43.0 0.0.0.255 (89745 matches)
permit udp any 10.1.1.0 0.0.0.255 (7893485 matches)
deny ip any any (448691555 matches)
6509-neb#
Global or per-ACL entry
ACL Hit Counters Supported on PFC3B/BXL Only! (use [no] mls acl tcam
share-global to toggle)
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 85
Agenda
• Chassis Architecture
• Supervisor Engine and Switch Fabric Architecture
• Switching Module Architecture
• Layer 2 Forwarding
• IPv4 Forwarding
• IPv4 Multicast Forwarding
• Security and Feature ACLs
• QoS
• NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 86
QoS
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 87
Catalyst 6500 QoS Model
Receive
Interface
Input Output
Ingress Egress Congestion
Queue Classify Mark Queue
Police Police Avoidance
Schedule Schedule
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 88
QoS Lookups
Frame Input Layer 2 Ingress
received lookup FIB TCAM Forwarding
Engine
Layer 2 Table FIB lookup
ACL TCAM
Router Yes Input QoS Input ACL
MAC? lookup lookup
NetFlow lookup
Input QoS
lookup
QoS TCAM
RST-3465
NetFlow Table QoS TCAM ACL TCAM *PFC3 only
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 89
Classification
• Based on—
Port trust
QoS ACLs
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 90
QoS ACLs
• Support standard and extended IPv4,
IPv6,† and MAC ACLs for classification
• Use QoS TCAM to classify traffic for
marking and policing
• Leverage dedicated QoS TCAM
32K entries/4K masks
† PFC3 only
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 91
QoS ACL Lookup Results
• QoS TCAM lookups behave exactly the same as ACL TCAM
lookups
• But, returned result differs:
Index into Aggregate table (identifies aggregate policer to use)
Index into Microflow table (identifies microflow policer to use)
Remarked DSCP/IP precedence value
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 92
Marking
• Untrusted port—Set a default QoS
value
• Trusted port—Use the marking
(COS, precedence, DSCP)
provided by upstream device
• QoS ACLs / service-policies—Set
QoS values based on standard or
extended ACL match
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 93
Policing
• Enforces a policy on a port or VLAN for traffic
matching classification policy
Markdown
Police (drop)
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 95
Microflow Policing
• Bandwidth limit applied separately to each individual flow that
matches the associated class
Every individual FTP flow limited to configured rate
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 96
Remarking Traffic with Policers
• Policing action may remark certain traffic
For example, transmit with marked-down DSCP
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 97
Monitoring Service Policies
(Marking and Policing)
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 99
NetFlow
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 100
NetFlow Lookups
Frame Input Layer 2 Ingress
received lookup FIB TCAM Forwarding
Engine
Layer 2 Table FIB lookup
ACL TCAM
Router Yes Input QoS Input ACL
MAC? lookup lookup
NetFlow lookup
Input QoS
lookup
QoS TCAM
RST-3465
NetFlow Table QoS TCAM ACL TCAM
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 101
IPv4 NetFlow
• Tracks statistics for traffic flows through the system
• Entries created in NetFlow table when new flows
start
Flow mask determines format of entries
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 102
Displaying NetFlow Statistics Entries
• Cisco IOS: show mls netflow ip Which fields are
populated depends on
• Catalyst OS: show mls statistics entry the configured flow mask
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 103
NetFlow Table Utilization
• PFC2
NetFlow table contains 128K entries
Hash ~25% efficient (32K entries)
Probability of collision increases after 32K
entries
• PFC3
NetFlow table size varies
• PFC3A/B—128K entries
• PFC3BXL—256K entries
Hash ~50–90% efficient (64/96/230K entries for
PFC3A/B/BXL)
Probability of collision increases after
64K/96K/230K entries
Alias CAM handles hash collision cases
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 104
PFC2 NetFlow Table Architecture
Packet
Flow Key
2 16K rows
SIP 10.10.20.1
DIP | 10.20.2.2
Proto | 6SPort
| 1044 | 80
DPort
172.16.8.2 | 192.168.1.2 | 6 | 1025 | 80
5 172.16.1.1
10.1.1.1 | 10.1.1.2 | 172.16.2.2
| 6 | 1030 || 80 17 | 2334 | 23
10.10.10.1
10.1.1.1 | 10.1.1.2| |10.20.1.1
6 | 1030 || 80 6 | 2334 | 80
Compare 10.1.1.1 | 10.1.1.2 | 6 | 1030 | 80
10.1.1.2 |192.168.1.1
10.1.1.1 | 10.1.1.2
10.1.1.1
10.4.4.4 | 6 || 10.1.1.2
6 | 1030| 1
|80
| 172.16.8.8
|0|0
| 80
10.1.1.1
10.1.1.2 | 10.1.1.2
| 10.1.1.1 | 6 | |806 | |1030
17 | 1025
| 80 | 514
10.1.1.1 | 10.1.1.2
10.1.1.2 | 10.1.1.1 | 6 | 80 | 6 | 1030 | 80
3 10.1.1.1
10.1.1.2 | 239.1.1.1
| 10.1.1.1 | 6 | 17
80 | 5000
1030 | 5000 Update
10.10.20.1
10.1.1.2 | 10.20.2.2
| 10.1.1.1 | 6 | 80| 6 | 1044 | 80
| 1030
Hash Function
10.1.1.2 | 10.1.1.1 | 6 | 80 | 1030
HIT! 10.1.1.2 | 10.1.1.1 | 6 | 22 | 3245 6 Statistics
10.99.1.1 | 10.99.100.1 | 6 | 4444 | 25
10.99.100.1 | 10.4.5.6 | 6 | 25 | 1080
Starting Page 4
and Row
8 pages
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. NetFlow Table Cisco Public 105
PFC3 NetFlow Lookups
Key Flow Data
Packet Key Flow Data
1 Key Flow Data
Key 6 Flow Data HIT!
2 Mask Compare
Key Flow Data Statistics
7
Flow
Flow Key
Key Key Flow Data
Key Flow Data
Key Flow Data
NetFlow
HIT! Key Flow Data
Table Index
3 Key 5 Flow Data
Result
Hash Function Key Flow Data
Flow Data
Mask
4
Hash Key
Hash Key 128K/256K 128K/256K
Compare
entries rows
Key
128 entries
RST-3465
12523_04_2006_c1 Alias CAM
© 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 106
Monitoring NetFlow Table Usage
• Cisco IOS: show mls netflow table-contention
• Catalyst OS: show mls debug
6506#show mls netflow table
- contention detailed
Earl in Module 6
Detailed Netflow CAM (TCAM and ICAM) Utilization
================================================
TCAM Utilization : 100%
ICAM Utilization : 82%
Current utilization
Netflow TCAM count : 131072
Netflow ICAM count : 105
Netflow Creation Failures : 3432605
Clear on read
Netflow CAM aliases : 8
6506#show mls netflow table
- contention aggregate
Earl in Module 6
Aggregate Netflow CAM Contention Information
=============================================
Netflow Creation Failures : 222917949
Cumulative
Netflow Hash Aliases : 834
RST-3465 6506# Cisco Public 107
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved.
NetFlow Aging
• Process of removing stale NetFlow
entries
• Types of aging
Normal—Fixed idle time for flows
Fast—Threshold-based aging of flows
Long—Maximum lifetime for flows
Session-based—Based on TCP FIN/RST flags
• Cisco IOS:
mls aging {normal | fast | long}
show mls netflow aging
• Catalyst OS:
set mls agingtime [fast | long-duration]
show mls
6506#
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 109
Conclusion
• You should now have a thorough
understanding of the Catalyst 6500
switching architecture, packet flow, and
key forwarding engine functions…
ANY QUESTIONS?
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 110
Related Networkers Sessions
• RST-3262: IP Multicast Architecture and Troubleshooting for
the Cisco Catalyst 6500 Series
• RST-3143: Troubleshooting Catalyst 6500 Series Switches
• RST-2031: Multilayer Campus Architectures and
Design Principles
• RST-3466: Cisco IOS Software Modularity—Architecture and
Deployment
• TECRST-3101: Troubleshooting Cisco Catalyst Switches
• TECRST-2001: Enterprise High Availability
• BoF-06: Enterprise Switching
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 111
Q and A
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 112
Recommended Reading
• Continue your Cisco Networkers
learning experience with further
reading from Cisco Press
• Check the Recommended Reading
flyer for suggested books
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 113
Complete Your Online Session Evaluation
• Win fabulous prizes; Give us your feedback
• Receive ten Passport Points for each
session evaluation you complete
• Go to the Internet stations located
throughout the Convention Center to complete
your session evaluation
• Drawings will be held in the
World of Solutions
Tuesday, June 20 at 12:15 p.m.
Wednesday, June 21 at 12:15 p.m.
Thursday, June 22 at 12:15 p.m. and 2:00 p.m.
RST-3465
12523_04_2006_c1 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Public 114
RST-4501
11366_06_2005_x © 2005 Cisco Systems, Inc. All rights reserved. 115