Académique Documents
Professionnel Documents
Culture Documents
Troubleshooting Lab
LTRSEC-2740
Agenda
Overview (30min)
LT RSEC-2740
Ether-Channel (ECLB)
4.
5.
Cisco Public
LT RSEC-2740
Cisco Public
Simple Mgmt
State Sharing
CCL
One Master
One Config
LT RSEC-2740
Cisco Public
High Availability
Deployment Options
Overview of ASA cluster types, firewall and context modes
Must configure L2 spanned mode cluster to use Transparent firewall
L3 Individual mode requires Routed firewall
Multiple context mode works in both types of clustering
Load Balancing
Transparent
Routed
Multiple Contexts
Individual Interface
L3 Method ECMP/ PBR
N/A*
Spanned Interface
L2 Method Ether-Channel LB
LT RSEC-2740
Cisco Public
Prep
Lab Portal
https://labops-out.cisco.com/labops/ilt
LT RSEC-2740
Cisco Public
Prep
Pick a Pod
LT RSEC-2740
Cisco Public
Prep
LT RSEC-2740
Cisco Public
Prep
10
LT RSEC-2740
Click to RDP
login:
Administrator
password:
stgscvt
Cisco Public
10
Prep
ASAs, CSRs, and test hosts are open via SuperPutty shortcut, using credentials:
ASA console: enable password is cisco
CSR SSH: auto-login: admin/cisco
Linux host SSH: auto-login: user/cisco
LT RSEC-2740
Cisco Public
11
ASA2
Enable Passwd: cisco
CSR1
Login: admin/cisco
CSR2
Login: admin/cisco
Inside-host
Login: user/cisco
Outside-host
Login: user/cisco
LT RSEC-2740
Cisco Public
12
Prep
Prep
Reconnect via
Layouts
Double-click on
ASA-CSR-ENDHOSTS
Inside-host
(IP 10.10.140.30)
./client.iperf
CSR2
show ip route
terminal monitor
(to view log msgs)
CSR1
show ip route
terminal monitor
(to view log msgs)
Outside-host
(IP 172.16.2.44)
./server.iperf
ASA1
show route
show conn
Outside-host
(IP 172.16.2.44)
ping 10.10.140.30
ssh user@10.10.140.30
Inside-host
(IP 10.10.140.30)
ping 172.16.2.44
ssh user@172.16.2.44
ASA2
show route
show conn
LT RSEC-2740
Cisco Public
13
Prep
Open IE or Firefox
Home Page inside RDP
LT RSEC-2740
Cisco Public
14
Prep
Tasks 1-5
Two IP Paths
1. Stand-alone ASAs as two equal OSPF paths for CSRs
2. Move to L3 cluster with CSR OSPF ECMP
3. Switch to IP SLA, by removing OSPF on ASA L3 cluster
LT RSEC-2740
Cisco Public
15
Prep
Outside
ASA1
CSR1
CSR2
ASA2
Cisco Public
16
Success
UDP
PASS
ping
FAIL
ssh
FAIL
Inspected or Stateful
Connections traversing
ASAs
IP 1.1.1.2
Details
Down ASA2
Open Conns
Up ASA2
IP 1.1.2.2
ASA1
Inside
host
Outside
host
CSR1
CSR2
IP 1.1.2.3
IP 1.1.1.3
ASA2
Steps
Task 1
Cisco Public
17
Etherchannel
Etherchannel
Layer 3 Adjacent
Task 2 and 3
LT RSEC-2740
Task 4 and 5
Cisco Public
18
CSR1#sh ip route
Task 2 & 3
(snip)
O 172.16.2.0/24 [110/12] via 1.1.1.3, 00:07:41, Gig1
[110/12] via 1.1.1.2, 00:18:25, Gig1
CSR2#sh ip route
(snip)
O
10.10.140.0 [110/12] via 1.1.2.3, 00:10:58, Gig1
[110/12] via 1.1.2.2, 00:11:08, Gig1
Inside
IP-A1
Tw o IP paths
IP-B1
ASA1
Outside
Tw o paths
CSR1
CSR2
IP-A2
IP-B2
ASA2
LT RSEC-2740
Protocol
Success
UDP
PASS
ping
PASS
ssh
PASS
Slave
Cisco Public
19
Task 2 & 3
Po1.7
.1 (.2)
Master
ASA1
Outside
VLAN 8
Po1.8
.1 (.2)
CCL
10.10.140.0/24
172.16.2.0/24
Po1
1.1.1.0/24
Inside
Host
CSR1
1.1.2.0/24
Outside
Host
CSR2
Po2
CCL
Po1.7
(.3)
ASA2
Slave
LT RSEC-2740
Cisco Public
20
Po1.8
(.3)
Workf low:
(1) Open test connections
(2) Determine the connection owner
(3) Proceed to f ail the owner ASA
(4) Measure conv ergence
(5) Recov er down ASA
UP
G0/2
Down
or
ASA2
UP
G0/2
ASA1
G0/3
ASA1
Down
UP
Down
or
Down
Po1
Inside
Host
Outside
Host
CSR1
CSR2
Po2
Test 2: Simulate
ASA crash w ith
crashinfo force page-fault
CCL
ASA2
UP
G0/3
CCL
LT RSEC-2740
ASA2
Cisco Public
21
UDP outside
172.16.2.44:5001 inside
TCP outside
172.16.2.44:55505 inside
10.10.140.30:38842, idle 0:00:00, bytes 883470, flags 10.10.140.30:22, idle 0:01:01, bytes 0, flags
asa2:*****************************************************************
7 in use, 17 most used
Cluster stub connections: 1 in use, 212 most used
TCP outside
172.16.2.44:55505 inside
UDP outside
172.16.2.44:5001 inside
master/a/admin(config)#
LT RSEC-2740
Cisco Public
Active UDP
connection
22
Active TCP
connection
Measuring Convergence
Count (nan%)
UDP packets
that were lost,
and record in your
convergence table
Lost
Pkts/Secs
ping
9 (322-330)
UDP iPerf
9 (326-334)
ssh
N/A
ASA detects
that owner unit
went down
Count the
missed
PINGs
LT RSEC-2740
Protocol
Cisco Public
23
Task 4 & 5
CSR2# sh ip route
(snip)
Gateway of last resort is 172.16.2.1 to network 0.0.0.0
O
10.10.140.0 [110/12] via 1.1.2.1, 00:21:20, Gig1
CCL
Switch
Inside
IP-A1
ASA1
Outside
IP-B1
CSR1
CSR2
ASA2
LT RSEC-2740
Cisco Public
24
Task 4
Po4.7
.1
ASA1
1.1.1.0/24
Outside
VLAN 8
Po4.8
.1
CCL
Po4
1.1.2.0/24
Inside
Host
Outside
Host
CSR1
.200
.200
Po4
CCL
ASA2
Slave
LT RSEC-2740
Cisco Public
25
CSR2
Task 5
Po4.7
BVI1
ASA1
Po4.8
BVI1
Outside
VLAN 8
CCL
10.10.140.0/24
172.16.2.0/24
Po4
Inside
Host
Outside
Host
CSR1
1.1.1.200/16
1.1.2.200/16
Po4
CCL
ASA2
Slave
LT RSEC-2740
Cisco Public
26
CSR2
Workf low:
(1) Open test connections
(2) Determine the connection owner
(3) Proceed to f ail the owner ASA
(4) Measure conv ergence
(5) Recov er down ASA
UP
G0/0
Down
or
ASA2
UP
G0/0
ASA1
ASA1
Down
UP
G0/1
or
ASA2
UP
Down
G0/1
Down
CCL
Po4
Inside
Host
Outside
Host
CSR1
CSR2
Po4
G0/3
ASA1
UP
Test 2: Simulate
ASA crash w ith
crashinfo force page-fault
CCL
ASA2
LT RSEC-2740
Cisco Public
27
Down
or
ASA2
UP
G0/3
Down
Task 1
Preview
IP 1.1.1.2
Tw o paths
IP 1.1.2.2
ASA1
External
Tw o paths
CSR1
CSR2
IP 1.1.1.3
IP 1.1.2.3
ASA2
Tests
Down ASA2
Two paths provided by ASA1 and ASA2,
stand-alone firewalls NOT in failover or cluster Attempt connections between hosts
Verify OSPF routes on CSR1 to outside
LT RSEC-2740
Cisco Public
28
Task 1
1.1.1.0/24
Inside
VLAN 7
Po1.7
(.2)
Outside
VLAN 8
Po1.8
(.2)
10.10.140.0/24
172.16.2.0/24
VLAN 15
VLAN 4
gig1
gig2
.30
Inside
host
1.1.2.0/24
Master
.200
.200
.44
Outside
host
Internal
CSR1
CSR2
Po2.7
(.3)
Po2.8
(.3)
ASA2
LT RSEC-2740
gig2
gig1
.200
.1
Cisco Public
29
External
Verify
CSR2
CSR1
!CSR1 OSPF routes
!CSR1#
!CSR2#
sh ip route ospf
sh ip route ospf
(snip)
(snip)
O*E2
O*E2
CSR2#
CSR1#
LT RSEC-2740
Cisco Public
Task 1
30
Verify
Task 1
ASA2
ASA1
!changeto context admin to show OSPF routes
!asa2#
!asa1#
!asa2/admin#
!asa1/admin#
sh route
sh route
Gateway of last resort is 1.1.2.200 to network 0.0.0.0
O
172.16.3.1 255.255.255.255 [110/12] via 1.1.2.200, 1:35:58,
outside
asa2/admin#
asa1/admin#
LT RSEC-2740
Cisco Public
31
To shutdown ASA2 ports on the switch, use browser home page on jumpbox PC,
pointing to link: http://172.16.2.40/
Disable ASA2
G0/3 port
Disable ASA2
G0/2 port
LT RSEC-2740
Task 1
Cisco Public
32
Verify
CSR1
!CSR1 OSPF routes
!CSR1#
!CSR2#
sh ip route ospf
sh ip route ospf
(snip)
(snip)
O*E2
O*E2
Task 1
Cisco Public
CSR1#
LT RSEC-2740
33
Task 1
InsideHost
#user@lubuntu:~$
cat client.iperf
iperf -u t 260 -i 1 -c 172.16.2.44 -b 0.0941m
user@lubuntu:~$
#Change t flag to 20000, to allow iPerf to send for 4 hours
#You can use your favorite UNIX editor installed, vi or pico
#This will allow you to run UDP traffic throughout duration of the lab
pico client.iperf
#Change to: -t 20000
#user@lubuntu:~$
cat client.iperf
iperf -u t 20000 -i 1 -c 172.16.2.44 -b 0.0941m
user@lubuntu:~$
LT RSEC-2740
Cisco Public
34
Task 1
Outside-host
(IP 172.16.2.44)
./server.iperf
Inside-host
(IP 10.10.140.30)
ping 172.16.2.44
Outside-host
(IP 172.16.2.44)
ssh user@10.10.140.30
(passwd: cisco)
Cisco Public
35
Task 1
InsideHost
#On top left terminal, ping to outside -lnx
#user@inside-lnx:~$
ping 172.16.2.44
PING 172.16.2.44 (172.16.2.44) 56(84) bytes of data.
#On top right terminal, Server listens and receives client UDP traffic
#user@outside-lnx:~$
Verify if you
can ping.
./server.iperf
------------------------------------------------------------
#on bottom left terminal, start a 4min iperf UDP connection to outside -lnx
------------------------------------------------------------
#user@inside-lnx:~$
./client.iperf
Transfer
Bandwidth
Jitter
------------------------------------------------------------
[ ID] Interval
Datagrams
11.5 KBytes
94.1 Kbits/sec
0.075 ms
3]
11.5 KBytes
94.1 Kbits/sec
0.087 ms
0/
8 (0%)
3]
28.7 KBytes
94.1 Kbits/sec
0.083 ms
0/
20 (0% )
3]
Lost/Total
0/
8 (0%)
-----------------------------------------------------------[
### When server is not receiving packets, output will show (-nan%)
[ ID] Interval
Transfer
3]
12.9 KBytes
106 Kbits/sec
3]
11.5 KBytes
LT RSEC-2740
### You can count the number of seconds server could not receive packets
[ 3] 21.0-22.0 sec 0.00 Bytes 0.00 bits/sec
0.067 ms
0/
0 (-nan%)
[ 3] 22.0-23.0 sec 0.00 Bytes 0.00 bits/sec
0.067 ms
0/
0 (-nan%)
[ 3] 23.0-24.0 sec 0.00 Bytes 0.00 bits/sec
0.067 ms
0/
0 (-nan%)
Bandwidth
Cisco Public
36
Task 1
OutsideHost
OutsideHost
#On bottom right terminal, open ssh connection outside to inside
#If this session locks up, it should drop out within 5min w/ error
user@outside-lnx:~$
user@lubuntu:~$
#You can kill it by typing ~. w/ no single quotes
(snip)
#Then re-open it
user@outside-lnx:~$
user@inside-lnx:~$
user@inside-lnx:~$
LT RSEC-2740
Cisco Public
37
Re-enable ASA2
Open IE link
inside RDP
To shutdown ASA1 or ASA2 ports on the switch, use browser on jumpbox PC and
open link: http://172.16.2.40/
Enable ASA2
G0/2
Task 1
Enable ASA2
G0/3
Tw o paths
ASA1
CSR1
Tw o paths
CSR2
ASA2
Cisco Public
38
Verify
CSR2
CSR1
!CSR1 OSPF routes
!CSR1#
CSR2#
sh ip route ospf
sh ip route ospf
(snip)
(snip)
O*E2
O*E2
CSR2#
CSR1#
LT RSEC-2740
Cisco Public
Task 1
39
Task 1
Protocol
Outside-host
(IP 172.16.2.44)
UDP traffic still being
received?
Task 1
Pass / Fail
ping
UDP iPerf
ssh
Outside-host
(IP 172.16.2.44)
ssh session still working?
Inside-host
(IP 10.10.140.30)
ping still working?
Cisco Public
40
Task 2
CCL
IP 1.1.1.2
Interna
l
IP 1.1.2.2
External
ASA1
IP 1.1.1.3
IP 1.1.2.3
CSR2
Tests
Enter configuration on ASA2 slave via CLI and Open connections through cluster
watch it detect and sync config from master
Down ASA that owns the connection using
one of four failure scenarios
CSR1/CSR2 are still load-balancing via OSPF
Two paths provided by ASA1 and ASA2, now
maintain state as L3/Individual cluster
LT RSEC-2740
Cisco Public
41
Task 2
1.1.1.0/24
1.1.2.0/24
Master
Inside
VLAN 7
Po1.7
.1 (.2)
Po1.8
.1 (.2)
Master
10.10.140.0/24
Outside
VLAN 8
G0/3
.1
172.16.2.0/24
VLAN 15
VLAN 4
gig1
gig2
.30
Inside
host
2.2.2.0/24
gig2
gig1
.200
.1
.200
.200
CCL VLAN 25
Outside
host
Internal
CSR1
Po2.7
(.3)
CSR2
G0/3
.2
Slave
Po2.8
(.3)
mgmt_pool
Inside_pool
Outside_pool
172.16.1.2-172.16.1.10
1.1.1.2-1.1.1.10
1.1.2.2-1.1.2.10
ASA2
LT RSEC-2740
.44
Cisco Public
42
External
Task 2
ASA2
!In system
!Feedback
context
from
clear
ASA2
cfg,
after
enable
enabling
cluster mode, and apply ASA2 cfg
changeto
system
asa2/a#(config)#ena
changeto system
config terminal
config terminal
ClusterDisabled/a(cfg-cluster)#
clear config
allCluster Master.
Detected
no cluster
INFO:interface-mode
UC-IME is enabled, issuing 0 free TLS licenses for UC-IME
clusterBeginning
interface-mode
individual
force
configuration
replication
from Master .
!Bring up
interface
for CCL
WARNING:
Removing
all contexts in the system
interface
GigabitEthernet0/3
Removing context 'admin' (7)... Done
no shut
INFO: Admin context is required to get the interfaces
!Define***
cluster
group
Output
from config line 64, "arp timeout 14400"
clusterINFO:
groupAdmin
fw
context is required to get the interfaces
local-unit
asa2 from config line 65, "no arp permit-nonconnect..."
*** Output
cluster-interface
GigabitEthernet0/3
ip 2.2.2.2
255.255.255.0
Creating context
'admin'... Done.
(8)
priority
20
*** Output from config line 68, "admin -context admin"
!If prompted, you MUST confirm Y for YES, remove these commands
console-replicate
health-check
3
WARNING:holdtime
Skip fetching
the URL disk0:/a.cfg
clacp system-mac auto system-priority 1
ClusterDisabled/a/asa1(config)#
!Now wait 1 min for ASA1 to become Master through election process
!Cluster unit asa1 transitioned from DISABLED to MASTER
!Cluster
unit asa2
from DISABLED
to SLAVE
Cluster
unit transitioned
asa2 transitioned
from DISABLED
to SLAVE
LT RSEC-2740
Cisco Public
43
ASA1
Task 2
ASA1
!master/a/admin(config)#
sh run router
router ospf 1
ID
: 0
Ver sion
: 9 .3(2)
CCL IP
: 2 .2.2. 1
CCL MAC
log-adj-changes
(snip)
!master/a/admin(config)#
sh route ospf
ID
: 1
(snip)
Ver sion
: 9 .3(2)
O*E2
CCL IP
: 2 .2.2. 2
CCL MAC
(snip)
!master /a/a sa1(c onfig )#
LT RSEC-2740
Cisco Public
44
Verify
CSR2
CSR1
!CSR1 OSPF routes
!CSR1#
!CSR2#
sh ip route ospf
sh ip route ospf
(snip)
(snip)
O*E2
O*E2
CSR2#
CSR1#
LT RSEC-2740
Cisco Public
Task 2
45
Task 2
Outside-host
(IP 172.16.2.44)
./server.iperf
Inside-host
(IP 10.10.140.30)
ping 172.16.2.44
Outside-host
(IP 172.16.2.44)
ssh user@10.10.140.30
(passwd: cisco)
Cisco Public
46
Task 2
OutsideHost
#user@outside-lnx:~$
#user@inside-lnx:~$
./server.iperf
ping 172.16.2.44
------------------------------------------------------------
user@inside-lnx:~$
Transfer
Bandwidth
Jitter
./client.iperf
[ ID] Interval
Datagrams
------------------------------------------------------------
3]
11.5 KBytes
94.1 Kbits/sec
0.075 ms
0/
3]
11.5 KBytes
94.1 Kbits/sec
0.087 ms
0/
8 (0%)
3]
28.7 KBytes
94.1 Kbits/sec
0.083 ms
0/
20 (0% )
8 (0%)
### Again, when server is not receiving packets, output will show (-nan%)
### You can count the number of seconds server could not receive packets
[ 3] 21.0-22.0 sec 0.00 Bytes 0.00 bits/sec
0.067 ms
0/
0 (-nan%)
[ 3] 22.0-23.0 sec 0.00 Bytes 0.00 bits/sec
0.067 ms
0/
0 (-nan%)
[ 3] 23.0-24.0 sec 0.00 Bytes 0.00 bits/sec
0.067 ms
0/
0 (-nan%)
[ ID] Interval
Transfer
3]
12.9 KBytes
106 Kbits/sec
3]
11.5 KBytes
LT RSEC-2740
Lost/Total
-----------------------------------------------------------[
Bandwidth
Cisco Public
47
LT RSEC-2740
Cisco Public
48
Task 2
Task 2
ASA1
!master/a/admin(config)#
172.16.2.44:5001 inside
TCP outside
172.16.2.44:55505 inside
10.10.140.30:38842, idle 0:00:00, bytes 883470, flags 10.10.140.30:22, idle 0:01:01, bytes 0, flags y
asa2:*****************************************************************
7 in use, 17 most used
172.16.2.44:55505 inside
UDP outside
172.16.2.44:5001 inside
master/a/admin(config)#
LT RSEC-2740
Cisco Public
Active UDP
connection
49
Active TCP
connection
Task 2
UP
G0/2
Down
or
ASA2
UP
G0/2
ASA1
G0/3
ASA1
Down
UP
Down
or
Down
Po1
Inside
Host
Outside
Host
CSR1
CSR2
Po2
Test 2: Simulate
ASA crash w ith
crashinfo force page-fault
CCL
ASA2
UP
G0/3
CCL
LT RSEC-2740
ASA2
Cisco Public
50
Task 2
Outside-host
(IP 172.16.2.44)
UDP packets arriving?
./server.iperf
Outside-host
(IP 172.16.2.44)
ssh session still working?
ssh -l user 10.10.140.30
Inside-host
(IP 10.10.140.30)
ping still working?
Ping 172.16.2.44
Cisco Public
51
Test 1
Protocol
Task 1
Lost
Pkts/Secs
To shutdown ASA2 ports on the switch, use browser home page on jumpbox PC,
pointing to link: http://172.16.2.40/
ping
UDP iPerf
ssh
Disable ASA
G0/2 port
LT RSEC-2740
Task 2
Cisco Public
52
Measure
Task 2
Count (nan%)
UDP packets
that were lost,
and record in your
convergence table
Compare PING
req counts
to find lost
pkt count
LT RSEC-2740
Cisco Public
53
Task 2
Enable ASA
G0/2 port
enable
!Wait for ASA to detect master, finish sync, and become a Slave unit
!Cluster unit asa2 transitioned from DISABLED to SLAVE
Cisco Public
54
Task 2
Outside-host
(IP 172.16.2.44)
UDP packets arriving?
./server.iperf
Outside-host
(IP 172.16.2.44)
ssh session still working?
ssh -l user 10.10.140.30
Inside-host
(IP 10.10.140.30)
ping still working?
Ping 172.16.2.44
Cisco Public
55
Task 2
ASA1
!master/a/admin(config)#
172.16.2.44:5001 inside
TCP outside
172.16.2.44:55505 inside
10.10.140.30:38842, idle 0:00:00, bytes 883470, flags 10.10.140.30:22, idle 0:01:01, bytes 0, flags y
asa2:*****************************************************************
7 in use, 17 most used
172.16.2.44:55505 inside
UDP outside
172.16.2.44:5001 inside
master/a/admin(config)#
LT RSEC-2740
Cisco Public
Active UDP
connection
56
Active TCP
connection
Test 2
Task 2
Simulate crash
on owner ASA
Protocol
Crash owner
ASA w/ CLI
ping
UDP iPerf
ssh
LT RSEC-2740
Cisco Public
57
Task 2
Lost
Pkts/Secs
Measure
Task 2
Count (nan%)
UDP packets
that were lost,
and record in your
convergence table
ASA detects
that owner unit
went down
LT RSEC-2740
Cisco Public
58
LT RSEC-2740
Cisco Public
Task 2
59
Task 2
Outside-host
(IP 172.16.2.44)
UDP packets arriving?
./server.iperf
Outside-host
(IP 172.16.2.44)
ssh session still working?
ssh -l user 10.10.140.30
Inside-host
(IP 10.10.140.30)
ping still working?
Ping 172.16.2.44
Cisco Public
60
Task 2
ASA1
!master/a/admin(config)#
172.16.2.44:5001 inside
TCP outside
172.16.2.44:55505 inside
10.10.140.30:38842, idle 0:00:00, bytes 883470, flags 10.10.140.30:22, idle 0:01:01, bytes 0, flags y
asa2:*****************************************************************
7 in use, 17 most used
172.16.2.44:55505 inside
UDP outside
172.16.2.44:5001 inside
master/a/admin(config)#
LT RSEC-2740
Cisco Public
Active UDP
connection
61
Active TCP
connection
CSR2
CSR1
!change spf dead-interval from 30sec to 3sec
!CSR1#
!CSR1#
interface GigabitEthernet1
interface GigabitEthernet1
ip ospf dead-interval 3
ip ospf dead-interval 3
ASA
Master
ASA
Master
!change spf dead-interval from 30sec to 3sec
!master/a/asa1/admin(config)#
!master/a/asa1/admin(config)#
sh route ospf
interface inside
(snip)
ospf dead-interval 3
O*E2
interface outside
ospf dead-interval 3
LT RSEC-2740
Cisco Public
62
Task 2
Test 3
Protocol
Task 3
Lost
Pkts/Secs
ping
UDP iPerf
ssh
Disable ASA
CCL port
LT RSEC-2740
Task 2
Cisco Public
63
Measure
Task 2
Count (nan%)
UDP packets
that were lost,
and record in your
convergence table
Count the
missed
PINGs
LT RSEC-2740
ASA detects
that owner unit
went down
Cisco Public
64
Task 2
config terminal
!Define cluster group
cluster group fw
enable
!Wait for ASA to detect master, finish sync, and become a Slave unit
!Cluster unit asa2 transitioned from DISABLED to SLAVE
Enable ASA
CCL port
LT RSEC-2740
Cisco Public
65
Task 3
CCL
IP 1.1.1.2
Interna
l
IP 1.1.2.2
IP 1.1.1.3
Preview
External
ASA1
IP 1.1.2.3
CSR1
CSR2
ASA2
Tests
Cisco Public
66
Task 3
1.1.1.0/24
1.1.2.0/24
Master
Inside
VLAN 7
Po1.7
.1 (.2)
Po1.8
.1 (.2)
Master
10.10.140.0/24
Outside
VLAN 8
G0/3
.1
172.16.2.0/24
VLAN 15
VLAN 4
gig1
gig2
.30
Inside
host
2.2.2.0/24
gig2
gig1
.200
.1
.200
.200
CCL VLAN 25
Outside
host
Internal
CSR1
Po2.7
(.3)
CSR2
G0/3
.2
Slave
Po2.8
(.3)
mgmt_pool
Inside_pool
Outside_pool
172.16.1.2-172.16.1.10
1.1.1.2-1.1.1.10
1.1.2.2-1.1.2.10
ASA2
LT RSEC-2740
.44
Cisco Public
67
External
Verify
CSR2
CSR1
!CSR1 OSPF routes
!CSR1#
!CSR2#
sh ip route ospf
sh ip route ospf
(snip)
(snip)
O*E2
O*E2
CSR2#
CSR1#
LT RSEC-2740
Cisco Public
Task 3
68
CLI
ASA
Master
ASA
Master
!ASA1 i n th is ca se is Mast er
!master/a/asa1/admin(config)#
!
show route
Gateway of last resort is 1.1.2.200 to network 0.0.0.0
S*
sh run rout er
master/a/asa1/admin(config)#
!ASA2 Slave
router ospf 1
!slave/a/asa2/admin(config)#
sh route
timers spf 1 1
S*
slave/a/asa2/admin(config)#
no rout er o spf 1
LT RSEC-2740
Cisco Public
69
Task 3
Verify
CSR2
CSR1
!CSR1 IP SLA routes
!CSR1#
!CSR2#
sh ip route
sh ip route
(snip)
(snip)
S*
O*E2
Task 3
CSR1#
CSR2#
LT RSEC-2740
Cisco Public
70
Task 3
Outside-host
(IP 172.16.2.44)
UDP packets arriving?
./server.iperf
Outside-host
(IP 172.16.2.44)
ssh session still working?
ssh -l user 10.10.140.30
Inside-host
(IP 10.10.140.30)
ping still working?
Ping 172.16.2.44
Cisco Public
71
ASA1
!master/a/admin(config)#
cluster exec sh conn
asa1(LOCAL):**********************************************************
7 in use, 18 most used
Cluster stub connections: 1 in use, 50 most used
UDP outside
172.16.2.44:5001 inside
TCP outside
172.16.2.44:58952 inside
10.10.140.30:60810, idle 0:00:00, bytes 170520, flags 10.10.140.30:22, idle 0:02:39, bytes 0, flags
asa2:*****************************************************************
172.16.2.44:58952 inside
UDP outside
172.16.2.44:5001 inside
master/a/admin(config)#
LT RSEC-2740
Task 3
Cisco Public
72
Task 3
UP
G0/2
Down
or
ASA2
UP
G0/2
ASA1
G0/3
ASA1
Down
UP
Down
or
Down
Po1
Inside
Host
Outside
Host
CSR1
CSR2
Po2
Test 2: Simulate
ASA crash w ith
crashinfo force page-fault
CCL
ASA2
UP
G0/3
CCL
LT RSEC-2740
ASA2
Cisco Public
73
Test 1
Protocol
Task 1
Lost
Pkts/Secs
To shutdown ASA2 ports on the switch, use browser home page on jumpbox PC,
pointing to link: http://172.16.2.40/
ping
UDP iPerf
ssh
Disable ASA
G0/2 port
LT RSEC-2740
Task 3
Cisco Public
74
Measure
Task 3
Count (nan%)
UDP packets
that were lost,
and record in your
convergence table
Count the
missed
PINGs
LT RSEC-2740
ASA detects
that owner unit
went down
Cisco Public
75
Verify
Task 3
CSR2
CSR1
!CSR1 IP SLA routes
!CSR1#
!CSR2#
sh ip route
sh ip route
(snip)
(snip)
S*
O*E2
CSR1#
CSR2#
LT RSEC-2740
Cisco Public
76
Task 3
Enable ASA
G0/2 port
enable
!Wait for ASA to detect master, finish sync, and become a Slave unit
!Cluster unit asa2 transitioned from DISABLED to SLAVE
LT RSEC-2740
Cisco Public
77
ASA1
!master/a/admin(config)#
cluster exec sh conn
asa1(LOCAL):**********************************************************
7 in use, 18 most used
Cluster stub connections: 1 in use, 50 most used
UDP outside
172.16.2.44:5001 inside
TCP outside
172.16.2.44:58952 inside
10.10.140.30:60810, idle 0:00:00, bytes 170520, flags 10.10.140.30:22, idle 0:02:39, bytes 0, flags
asa2:*****************************************************************
172.16.2.44:58952 inside
UDP outside
172.16.2.44:5001 inside
master/a/admin(config)#
LT RSEC-2740
Task 3
Cisco Public
78
Test 2
Task 3
Simulate crash
on owner ASA
Protocol
Crash owner
ASA w/ CLI
ping
UDP iPerf
ssh
LT RSEC-2740
Cisco Public
79
Task 2
Lost
Pkts/Secs
Measure
Task 3
Count (nan%)
UDP packets
that were lost,
and record in your
convergence table
ASA crashes
Count the
missed
PINGs
LT RSEC-2740
Cisco Public
80
LT RSEC-2740
Cisco Public
Task 3
81
Task 3
Outside-host
(IP 172.16.2.44)
UDP packets arriving?
./server.iperf
Outside-host
(IP 172.16.2.44)
ssh session still working?
ssh -l user 10.10.140.30
Inside-host
(IP 10.10.140.30)
ping still working?
Ping 172.16.2.44
Cisco Public
82
ASA1
!master/a/admin(config)#
cluster exec sh conn
asa1(LOCAL):**********************************************************
7 in use, 18 most used
Cluster stub connections: 1 in use, 50 most used
UDP outside
172.16.2.44:5001 inside
TCP outside
172.16.2.44:58952 inside
10.10.140.30:60810, idle 0:00:00, bytes 170520, flags 10.10.140.30:22, idle 0:02:39, bytes 0, flags
asa2:*****************************************************************
172.16.2.44:58952 inside
UDP outside
172.16.2.44:5001 inside
master/a/admin(config)#
LT RSEC-2740
Task 3
Cisco Public
83
Test 3
Protocol
Task 3
Lost
Pkts/Secs
ping
UDP iPerf
ssh
Disable ASA
CCL port
LT RSEC-2740
Task 3
Cisco Public
84
Measure
Task 3
Count (nan%)
UDP packets
that were lost,
and record in your
convergence table
ASA switches
to Master role
Count the
missed
PINGs
LT RSEC-2740
Cisco Public
85
Task 3
Enable ASA
CCL port
enable
!Wait for ASA to detect master, finish sync, and become a Slave unit
!Cluster unit asa2 transitioned from DISABLED to SLAVE
LT RSEC-2740
Cisco Public
86
Task 3*
CCL
IP 1.1.1.2
IP 1.1.2.2
(optional)
Interna
l
IP 1.1.1.3
Preview
External
ASA1
IP 1.1.2.3
CSR1
CSR2
ASA2
Tests
LT RSEC-2740
Cisco Public
87
CLI
CSR2
!CSR2
config terminal
ASA
Master
!ASA master
master/a/asa1(config)#
(snip)
S
config terminal
object network pat-ips
OutsideHost
LT RSEC-2740
user@lubuntu:~$
Cisco Public
88
Task 3*
Task 3*
Inside-host
(IP 10.10.140.30)
./client.iperf
Outside-host
(IP 172.16.2.44)
Can not go to inside now
without a static NAT
Inside-host
(IP 10.10.140.30)
ping 172.16.2.44 or
Ssh user@172.16.2.44
Cisco Public
89
Verify translations
Task 3*
ASA
Master
ASA
Master
! You can also try show conn detail to decode the flags
master/a/asa1/admin(config)#
asa2(LOCAL):**********************************************************
asa2(LOCAL):**********************************************************
TCP outside 172.16.2.44:22 inside
bytes 0, flags Y
UDP outside 172.16.2.44:5001 inside
bytes 2072700, flags -
asa1:*****************************************************************
asa1:*****************************************************************
LT RSEC-2740
master/a/asa1/admin(config)#
Cisco Public
90
Remove PAT
ASA
Master
CSR2
config terminal
config terminal
exit
write memory
LT RSEC-2740
Task 3*
Cisco Public
91
Task 4
CCL
IP 1.1.1.1
Interna
l
Preview
CSR1
One path
CSR2
ASA2
Tests
Open test connections through cluster
One hop
External
ASA1
LT RSEC-2740
IP 1.1.2.1
92
Task 4
1.1.1.0/24
Master
Inside
VLAN 7
G0/0
VLAN 15
gig1
gig2
Inside
host
G0/1
ASA1
10.10.140.0/24
.30
Outside
VLAN 8
.1
.200
172.16.2.0/24
VLAN 4
G0/3
.1
Po4.7
.1
gig1
gig2
.200
.200
Po4.8
.1
Outside
host
Internal
G0/3
.2
CSR1
CSR2
mgmt_pool
172.16.1.2-172.16.1.10
G0/0
G0/1
ASA2
Slave
CCL
VLAN 25
2.2.2.0/24
Cisco Public
93
.44
External
CLI
Task 4
ASA1
! Disable clustring on ASA1 unit
changeto system
changeto system
config terminal
config terminal
cluster group fw
cluster group fw
no enable
no enable
!All data interfaces have been shutdown due to clustering being disabled.
To recover either enable clustering or remove cluster group configuration.
ClusterDisabled/a/asa1(cfg-cluster)#
ClusterDisabled/a/asa2(cfg -cluster)#
LT RSEC-2740
Cisco Public
94
CLI
Task 4
ASA1
! Execute CLI to convert to L2 or Spanned interface mode
changeto system
changeto system
config term
config terminal
no shut
cluster group fw
local-unit asa2
cluster-interface GigabitEthernet0/3 ip 2.2.2.2 255.255.255.0
priority 20
console-replicate
health-check holdtime 3
clacp system-mac auto system-priority 1
enable
!Wait for ASA2 to detect master, finish sync, and become a Slave unit
LT RSEC-2740
Cisco Public
95
Verify
ASA1
Master
ASA1
Master
!master/a/asa1#
!master/a/asa1#
changeto system
show cluster info
Cluster fw: On
Interface mode: spanned
This is "asa1" in state MASTER
ID
: 0
Version
: 9.3(2)
Serial No.: FCH16097J8X
CCL IP
: 2.2.2.1
CCL MAC
: c464.1339.1841
Last join : 18:43:37 UTC Jan 14 2015
Last leave: N/A
Other members in the cluster:
Unit "asa2" in state SLAVE
ID
: 1
Version
: 9.3(2)
Serial No.: FCH16097J78
CCL IP
: 2.2.2.2
CCL MAC
: c464.1339.1481
Last join : 19:17:36 UTC Jan 14 2015
Last leave: N/A
master/a/asa1(config)#
LT RSEC-2740
Task 4
asa1(LOCAL):**********************************************************
Group
Port-channel
Protocol
Span-cluster
Ports
------+-------------+---------+------------+--------------2
Po2(U)
LACP
Yes
Gi0/0(P)
Gi0/1(P)
asa2:*****************************************************************
Group
Port-channel
Protocol
Span-cluster
Ports
------+-------------+---------+------------+--------------2
Po2(U)
LACP
Yes
Gi0/0(P)
!master/a/asa1#
!Notice that Non-Stop Forwarding is enabled for ASA now
changeto context admin
show run router
Cisco Public
96
Gi0/1(P)
CSR1
CSR2
CSR1#
Task 4
CSR2#
sh ip route
sh ip route
O*E2
O*E2
O
C
L
O
C
L
O
CSR2#
CSR1#
LT RSEC-2740
Cisco Public
97
Task 4
For each Test, observe and record packets lost for UDP and PING, and manually
Inside-host
(IP 10.10.140.30)
Still sending packets
./client.iperf
Outside-host
(IP 172.16.2.44)
UDP packets arriving?
./server.iperf
Outside-host
(IP 172.16.2.44)
ssh session still working?
Type one char and wait
ssh -l user 10.10.140.30
Inside-host
(IP 10.10.140.30)
ping still working?
Ping 172.16.2.44
Cisco Public
98
Task 4
UP
G0/0
Down
or
ASA2
UP
G0/0
ASA1
ASA1
Down
UP
G0/1
or
ASA2
UP
Down
G0/1
Down
CCL
Po4
Inside
Host
Outside
Host
CSR1
CSR2
Po4
G0/3
ASA1
UP
Test 2: Simulate
ASA crash w ith
crashinfo force page-fault
CCL
ASA2
Cisco Public
99
Down
or
ASA2
UP
G0/3
Down
Test 1A
Protocol
Task 1A
Lost
Pkts/Secs
To shutdown ASA2 ports on the switch, use browser home page on jumpbox PC,
pointing to link: http://172.16.2.40/
ping
UDP iPerf
ssh
Disable ASA
G0/0 port
LT RSEC-2740
Task 4
Cisco Public
100
Test 1B
Protocol
Task 1B
Lost
Pkts/Secs
Task 4
To shutdown ASA2 ports on the switch, use browser home page on jumpbox PC,
pointing to link: http://172.16.2.40/
ping
UDP iPerf
ssh
Disable ASA
G0/1 port
LT RSEC-2740
Cisco Public
101
Task 4
To shutdown ASA2 ports on the switch, use browser home page on jumpbox PC,
pointing to link: http://172.16.2.40/
Up the ASA
G0/1 port
Up the ASA
G0/0 port
Down ASA
! Re-join approriate ASA unit
changeto system
config terminal
LT RSEC-2740
Cisco Public
102
Test 2
Protocol
Task 4
Task 2
Lost
Pkts/Secs
Owner ASA
ping
UDP iPerf
ssh
cluster group fw
enable
!Wait for ASA2 to detect master, finish sync, and become
a Slave unit
!Cluster unit asa2 transitioned from DISABLED to SLAVE
Crash owner
ASA w/ CLI
LT RSEC-2740
Cisco Public
103
Test 3
Protocol
Task 4
Task 3
Lost
Pkts/Secs
Owner ASA
ping
UDP iPerf
cluster group fw
ssh
no enable
!Or you can down the CCL for owner ASA via web page
!As shown below in the home web page
Down CCL on
owner ASA
LT RSEC-2740
Cisco Public
104
Task 4
Down ASA
Bring UP CCL on
owner ASA
!ClusterDisabled/a/asa1/admin(config)#
changeto context sys
!ClusterDisabled/a/asa1(config)#
cluster group fw
Enable
!Detected Cluster Master.
(snip)
Cisco Public
105
Task 4*
CCL
IP 1.1.1.1
IP 1.1.2.1
(optional)
Interna
l
Preview
External
ASA1
CSR1
One path
One Hop
Aw ay
CSR2
ASA2
Tests
LT RSEC-2740
Cisco Public
106
CLI
Task 4*
ASA1
CSR2
! If you skipped Task 3*, you will need pat -ips object
changeto context admin
sh ip route
(snip)
InsideHost
user@172.16.2.44's password:
user@inside-lnx:~$
Cisco Public
107
Task 4*
Inside-host
(IP 10.10.140.30)
./client.iperf
Outside-host
(IP 172.16.2.44)
Can not go to inside now
without a static NAT
Inside-host
(IP 10.10.140.30)
ping 172.16.2.44
Ssh user@172.16.2.44
Cisco Public
108
Verify
ASA1
CSR1
!master/a/asa1/admin(config)#
!CSR1#
sh ip route
O*E2
asa2:*****************************************************************
C
L
O
O E2
C
L
O
O
CSR1#
asa2:*****************************************************************
1 in use, 9 most used
Cluster stub connections: 1 in use, 0 most used
TCP outside 172.16.2.44:22 inside
bytes 4102, flags UxIO
master/a/asa1/admin(config)#
LT RSEC-2740
Task 4*
Cisco Public
109
Remove PAT
ASA
Master
CSR2
config terminal
config terminal
LT RSEC-2740
Task 4*
Cisco Public
110
Task 5
CCL
IP 1.1.2.200/16
Interna
l
Preview
External
ASA1
CSR1
One Subnet
Directly
Connected
CSR2
ASA2
Tests
Measure convergence
LT RSEC-2740
Cisco Public
111
Master
1.1.0.0/16
1.1.0.0/16
Master
Inside
VLAN 7
G0/0
VLAN 15
Inside
host
VLAN 4
gig1
.1.200
.1
172.16.2.0/24
0/3
.1
gig1
gig2
.30
Outside
VLAN 8
G0/1
ASA1
10.10.140.0/24
gig2
.2.200
.200
Po4.8
BVI1
Po4.7
BVI1
CSR1
0/3
.2
mgmt_pool
G0/0
G0/1
ASA2
Slave
2.2.2.0/24
CCL
VLAN 25
LT RSEC-2740
Cisco Public
112
.44
Outside
host
Internal
172.16.1.2-172.16.1.10
Task 5
CSR2
External
CLI
Task 5
ASA1
!Install a transparent firewall context config for current admin context
!master/a/asa1/admin(config -if)#
config terminal
sh mac-address-table
changeto system
copy /noconfirm milan/task5 -admin.cfg task5-admin.cfg
interface
bridge-group
context admin
.
Cryptochecksum (unchanged): dcf70f21 bc4b86f6 c570e03f 2093dcd6
INFO: Context admin was created with URL disk0:/task5-admin.cfg
INFO: Admin context will take some time to come up .... please wait.
type
Age(min)
Cisco Public
inside
0050.56bf.34b8
dynamic
inside
0016.9cd3.b780
dynamic
outside
0050.56bf.dbc2
dynamic
master/a/asa1/admin(config -if)#
master/a/asa1(config -ctx)#
address
----------------------------------------------------------------------------------
config-url disk0:/task5-admin.cfg
LT RSEC-2740
mac
113
CLI
CSR1
CSR2
!Change CSR subnet to /16 so they can peer through ASA cluster
!Change CSR subnet to /16 so they can peer through ASA cluster
config terminal
interface GigabitEthernet1
ip address 1.1.2.200 255.255.0.0
config terminal
interface GigabitEthernet1
ip address 1.1.1.200 255.255.0.0
router ospf 1
no network 1.1.2.0 0.0.0.255 area 0
network 1.1.0.0 0.0.255.255 area 0
router ospf 1
no network 1.1.1.0 0.0.0.255 area 0
network 1.1.0.0 0.0.255.255 area 0
! Verify routes on CSRs, once they can ping each other and peer directly
! Verify routes on CSRs, once they can ping each other and peer directly
O*E2
O
O
CSR1#
O*E2
LT RSEC-2740
Task 5
Cisco Public
O
O
CSR2#
114
Verify
ASA1
Master
!master/a/asa1/admin(config)#
cluster exec show conn
asa1(LOCAL):**********************************************************
0 in use, 19 most used
Cluster stub connections: 0 in use, 6 most used
asa2:*****************************************************************
2 in use, 8 most used
Cluster stub connections: 0 in use, 117 most used
OSPF outside 224.0.0.5 inside 1.1.1.200, idle 0:00:00, bytes 181176, flags
OSPF outside 1.1.2.200 inside 224.0.0.5, idle 0:00:00, bytes 179984, flags
master/a/asa1/admin(config)#
LT RSEC-2740
Cisco Public
115
Task 5
Task 5
Outside-host
(IP 172.16.2.44)
UDP packets arriving?
./server.iperf
Outside-host
(IP 172.16.2.44)
Restart ssh session
ssh -l user 10.10.140.30
Inside-host
(IP 10.10.140.30)
ping still working?
Ping 172.16.2.44
Cisco Public
116
ASA1
!master/a/asa1/admin(config)#
cluster exec show conn
asa1(LOCAL):**********************************************************
4 in use, 10 most used
Cluster stub connections: 2 in use, 0 most used
OSPF outside 224.0.0.5 inside
172.16.2.44:55501 inside
172.16.2.44:5001 inside
asa2:*****************************************************************
3 in use, 3 most used
Cluster stub connections: 4 in use, 96 most used
OSPF outside 1.1.2.200 inside
UDP outside
172.16.2.44:5001 inside
172.16.2.44:55501 inside
master/a/asa1/admin(config)#
LT RSEC-2740
Task 5
Cisco Public
117
Task 5
UP
G0/0
Down
or
ASA2
UP
G0/0
ASA1
ASA1
Down
UP
G0/1
or
ASA2
UP
Down
G0/1
Down
CCL
Po4
Inside
Host
Outside
Host
CSR1
CSR2
Po4
G0/3
ASA1
UP
Test 2: Simulate
ASA crash w ith
crashinfo force page-fault
CCL
ASA2
Cisco Public
118
Down
or
ASA2
UP
G0/3
Down
Test 1B
Protocol
Task 1B
Lost
Pkts/Secs
Task 5
To shutdown ASA2 ports on the switch, use browser home page on jumpbox PC,
pointing to link: http://172.16.2.40/
ping
UDP iPerf
ssh
Disable ASA
G0/0 port
LT RSEC-2740
Disable ASA
G0/1 port
Cisco Public
119
Task 5
Up the ASA
G0/0 port
Down ASA
! Re-join approriate ASA unit
changeto system
config terminal
LT RSEC-2740
Cisco Public
120
Test 2
Protocol
Task 5
Task 2
Lost
Pkts/Secs
Owner ASA
ping
UDP iPerf
ssh
Crash owner
ASA w/ CLI
LT RSEC-2740
Cisco Public
121
Test 3
Protocol
Task 5
Task 3
Lost
Pkts/Secs
Owner ASA
ping
UDP iPerf
changeto system
ssh
cluster group fw
no enable
!Or you can down the CCL for owner ASA via web page
!As shown below in the home web page
Down CCL on
owner ASA
LT RSEC-2740
Cisco Public
122
Task 5
Bring UP CCL on
owner ASA
!ClusterDisabled/a/asa1(config)#
cluster group fw
enable
!Detected Cluster Master.
(snip)
End configuration replication from Master.
Cluster unit asa1 transitioned from DISABLED to SLAVE
LT RSEC-2740
Cisco Public
123
Task 5*
CCL
IP 1.1.1.200/16
IP 1.1.2.200/16
Interna
l
Preview
External
ASA1
CSR1
One Subnet
Directly
Connected
CSR2
ASA2
Tests
LT RSEC-2740
Cisco Public
124
Introduce PAT
CLI
Task 5*
ASA1
! If you skipped Task 3*, you will need pat -ips and inside-network objects
!CSR2#
show ip route
O*E2
clear local
CSR2#
LT RSEC-2740
Cisco Public
125
Task 5*
Inside-host
(IP 10.10.140.30)
./client.iperf
Outside-host
(IP 172.16.2.44)
Can not go to inside now
without a static NAT, so
SSH from inside
to outside
Inside-host
(IP 10.10.140.30)
ping 172.16.2.44
ssh user@172.16.2.44
Cisco Public
126
Verify
Task 5*
ASA1
cluster exec show conn
asa1(LOCAL):**********************************************************
asa1(LOCAL):**********************************************************
asa2:*****************************************************************
asa2:*****************************************************************
master/a/asa1/admin(config)#
10.10.140.30:50519, idle 0:00:06,
master/a/asa1/admin(config)#
LT RSEC-2740
Cisco Public
127
CONGRATULATIONS.
on completing the LTRSEC-2740 lab
128
Call to Action
Visit the World of Solutions for
Cisco Campus Visit Network and Content Security Booths
Technical Solution Clinics
Meet the Engineer ASA experts from our team will be available to meet you
Lunch time Table Topics
DevNet zone related labs and sessions
Recommended Reading: for reading material and further resources for this
session, please visit www.pearson-books.com/CLMilan2015
LT RSEC-2740
Cisco Public
129
LT RSEC-2740
Cisco Public
130
Additional Slides
IP 1.1.2.1
IP 1.1.1.1
vPC
IP 1.1.2.2
IP 1.1.2.1
Po 200
IP 1.1.2.2
IP 1.1.1.1
IP 1.1.1.2
IP 1.1.1.2
Po 201
IP 1.1.1.3
Po 202
Po 203
IP 1.1.2.4
Po 101
IP 1.1.2.3
IP 1.1.1.3
IP 1.1.2.3
Po 102
IP 1.1.2.4
IP 1.1.1.4
Po 103
IP 1.1.1.4
Outside
Outside
Inside
CCL
LT RSEC-2740
Po 100
Inside
CCL
Cisco Public
132
vPC
cLACP
ASA Po 10
LACP
vPC 100
Po 100
Classic Switch
N7K/vPC
Cat/VSS
CCL
LT RSEC-2740
CCL
Cisco Public
133
134