Brkapp 3006

Troubleshooting Cisco Wide Area Application Services (WAAS)
BRKAPP-3006
Zach Seils
2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
Session Abstract
This session provides troubleshooting techniques for analyzing the behavior and performance of Cisco Wide Area Application Services (WAAS). The session begins with an overview of the various WAE hardware platforms, followed by key platform health and liveliness indicators. Next, an analysis of common network interception issues and their resolution is covered followed by the WAE auto-discovery process, including tips and techniques for analyzing optimized traffic using a packet capture. Once the basics have been established, the session provides a detailed analysis of optimized connection counters and performance. Particular focus is given to understanding system performance. The session finishes by looking at common issues with application acceleration and CIFS auto-discovery. This session is intended for network engineering and operations support personnel responsible for deploying and supporting Cisco WAAS.
Cisco Public
Related Sessions
BRKAPP-1004: Introduction to Cisco Wide Area Application Services (WAAS) BRKAPP-2005: Deploying Cisco Wide Area Application Services (WAAS)
Cisco Public
Skill Level Assumptions

Understanding of TCP/IP Understanding of WAAS Components and their functions, including:
ATP, TFO, DRE, WAFS, Central Manager, etc.
Familiarity with WAAS configuration and management interfaces, including:

CM GUI, WAE GUI and CLI
Cisco Public
Session Topics
WAE Platform Overview Platform Health and Liveliness Network Interception TFO Auto-Discovery WAN Optimization Application Acceleration
Cisco Public
WAE Platform Overview
Cisco Public
WAE Hardware Options

WAE-512 Appliance
Single processor, 1 or 2GB of memory Supports up to 20Mbps WAN and up to 1,500 optimized TCP connections Software RAID-1, No Hot Swap Support
WAE-512 Remote Office Appliance
WAE-612 Appliance
Dual-core processor, 2 or 4GB of memory Supports up to 90Mbps WAN and up to 6,000 optimized TCP connections Software RAID-1, Hot Swap Support*
WAE-612 Regional Hub and Data Center Appliance
* Requires WAAS 4.0.13 or later

2008 Cisco Systems, Inc. All rights reserved. Cisco Public

WAE-7326 Appliance
Dual processor, 4GB of memory Supports up to 155Mbps WAN and up to 7,500 optimized TCP connections Software RAID-1, Hot Swap Support*
WAE-7326 Enterprise Data Center Appliance
WAE-7341 Appliance
Quad-core processor, 12GB of memory Supports up to 310Mbps WAN and up to 12,000 optimized TCP connections Hardware RAID-5, Hot Swap Support*
WAE-7371 Appliance
Dual Quad-core processor, 24GB of memory Supports up to 1000Mbps WAN and up to 50,000 optimized TCP connections Hardware RAID-5, Hot Swap Support*

8

NME-WAE-302
Single processor, 512MB of memory Supports up to 4Mbps WAN and up to 250 optimized TCP connections Single drive, no RAID support Transport license only
NME-WAE-502
Single processor, 1GB of memory Supports up to 4Mbps WAN and up to 500 optimized TCP connections Single drive, no RAID support Enterprise license supported (WAFS Edge) NME-WAE Router-Integrated Network Module for the Cisco Integrated Services Router
NME-WAE-522
Single processor, 2GB of memory Supports up to 8Mbps WAN and up to 800 optimized TCP connections Single drive, no RAID support Enterprise license supported

Inline Module
Supported on all WAE appliance models 2 pairs of inline ports Ports fail-to-wire Configurable interception based on Vlan
Cisco Public
10
Cisco WAE Family Performance and Scalability

CURRENT HARDWARE PLATFORMS Drive Unit Capacity (GB) / Maximum Usable Capacity (GB) CM Scalability (Devices Managed) Core Fan-out (Number of Peers)
Hardware Configuration
Max Optimized TCP Connections
Max CIFS Sessions
Max Drives
Memory (GB)
WAN Link Capacity (Mbps)
NME-WAE-302 NME-WAE-502 NME-WAE-522 WAE-512-1GB WAE-512-2GB WAE-612-2GB WAE-612-4GB WAE-7326 WAE-7341 WAE-7371
250 500 800 750 1500 2000 6000 7500 12000 50000
N/A 500 800 750 1500 2000 2500 2500 12000 32000
80/80 120/120 160/160 250/250 250/250 300/300 300/300 300/900 300/840 300/1400
1 1 1 2 2 2 2 6 4 6
.5 1 2 1 2 2 4 4 12 24
4 4 8 8 20 45 90 155 310 1000
N/A N/A N/A 500 1000 2000 2500 N/A N/A N/A
1 1 1 5 10 30 50 96 200 400
Note: These are guidelines for sizing based on certain assumptions. Enabling multiple features will have an impact on scalability.
11
Software Options Image Naming Convention WAAS-w.w.x.y-K9.z

Major Version Level Minor Version Level Build Number Image Type
WAAS-4.0.9.10-K9.binRelease Image WAAS-4.0.9.10-K9.sysimgSystem Image WAAS-4.0.9.10-rescue-cdrom-K9.isoRescue CD-ROM
Cisco Public
12
Software Options Upgrade and Recovery Images

Release Image (*.bin)
Regular software file used for copy ftp install type of installation or upgrade from GUI. Contains the System Image and other disk-based software
System Image (*.sysimg)

Used when the system image on flash is corrupt and cannot boot. Can be loaded from the bootloader/rescue image by typing *** during WAE startup
Rescue CD (*.iso)
Used to create bootable CD-ROM for manufacturing flash, installing bootloader/rescue image, system image, and/or release image
Cisco Public
13
Checking WAAS Software Version

What is the current running version?
WAE-612# show version Cisco Wide Area Application Services Software (WAAS) Cisco Wide Area Application Services Software Release 4.0.9 (build b10
What was the previous version?

WAE-612# show version last Saved version is WAAS 4.0.7-b46, built on 10:22:10 Feb 25 2007 ... It can be restored by running restore rollback command
What is the pending version?

WAE-612# show version pending Pending version is WAAS 4.0.9-b10, built on 17:07:16 Apr 19 2007 by ... It will take effect after reload
Cisco Public
14
Verify Flash and Disk-based Code

WAAS software is made up of both flash memory and disk-based software components The software version on flash and disk must match for proper system operation
WAE-612# show flash WAAS software version (disk-based code): WAAS-4.0.9-b10 System image on flash: Version: 4.0.9.b10 System flash directory: System image: 123 sectors Bootloader, rescue image, and other reserved areas: 24 sectors 256 sectors total, 109 sectors free. WAE-612#
Cisco Public
15
Platform Health and Liveliness
Cisco Public
16
A good place to start

4.0.15 release added support for self-diagnostic tool Executes series of tests to check correctness of configuration and device operation Test output saved in local diagnostic_report.txt file, which is included as part of sysreport
WAE7326-CCO# test self-diagnostic ? all Run all self-diagnostic tests basic Basic device configuration connectivity Basic device connectivity inline Inline groups and ports interfaces Physical interfaces system Device operation tfo TFO/DRE configuration wafs WAFS connectivity wccp WCCP configuration and operation WAE7326-CCO#
Cisco Public
17
Sample Diagnostic Report

WAE7326-CCO# running test running test running test running test running test running test running test running test test self-diagnostic all `basic' `connectivity' ìnterfaces' `tfo' `wccp' ìnline' `system' `wafs'
-----------------------------------------------------------Diagnostic Report performed on 12/21/2007 11:22:59 Test NOTE [basic] NOTE NO_NTP_CFG Device ntp time synchronization is not configured Action: Use 'ntp server' cli command to configure NTP server. Test PASS Test WARN WARN [connectivity]
[interfaces] IFACE_HD Interface GigabitEthernet 1/0 is in half-duplex mode. Please check the Switch/Router configuration. Action: Half-Duplex is configured in the interface , Use 'interface type slot/group full-duplex' cli command to configure Full-Duplex for better performance. If autosense is configured then change the duplex settings configured in the Switch/Router. NOTE IFACE_DOWN Interface GigabitEthernet 2/0 is not up Action: Use 'interface type slot/group' cli command followed by 'no shutdown' to enable the interface.
18
Sample Diagnostic Report (cont.)

Test WARN WARN [tfo] PE_OTHER Un-classified traffic is configured for by-pass and is not optimized Action: Use policy engine 'policy-engine application map other' cli command to enable optimization of un-classified traffic. [wccp] [inline] Skipping test because inline card(s) is not present
Test PASS Test PASS NONE Test WARN WARN 1.
[system] HAS_ALARM 1 Major Alarms are raised in the device. Please find the list below. The interface GigabitEthernet 1/0 is not in full-duplex mode.
Action: Critical/major alarms are raised. Check device alarms using 'show alarms detail' cli command for details. Test PASS [wafs]
WAE7326-CCO#
Cisco Public
19
Interface Duplex Issues

WAE7326-CCO# show int gi 1/0 Type:Ethernet Ethernet address:00:14:5E:AC:2D:79 Internet address:10.88.80.135 Broadcast address:10.88.80.255 Netmask:255.255.255.128 Verify interface Maximum Transfer Unit Size:1500 transitions and Metric:1 Packets Received: 14298 Input Errors: 0 Input Packets Dropped: 0 Input Packets Overruns: 0 Input Packets Frames: 0 Packet Sent: 12946 Output Errors: 0 Output Packets Dropped: 0 Output Packets Overruns: 0 Output Packets Carrier: 0 Output Queue Length:1000 Collisions: 64 Interrupts:16 Flags:UP BROADCAST RUNNING MULTICAST Mode: autoselect, half-duplex, 100baseTX WAE7326-CCO#
configuration, errors
Cisco Public
20
Sample Diagnostic Report

Targeted diagnostic report
WAE7326-CCO# test self-diagnostic interfaces running test ìnterfaces' -----------------------------------------------------------Diagnostic Report performed on 12/21/2007 11:39:20 Test NOTE [interfaces] NOTE IFACE_DOWN Interface GigabitEthernet 2/0 is not up Action: Test for packet loss usingby ping Use 'interface type slot/group' cli command followed 'no flood shutdown to enable the interface. WAE7326-CCO# ping -c 1000 -f 10.88.80.129 PING 10.88.80.129 (10.88.80.129) from 10.88.80.135 : 56(84) bytes of data. . --- 10.88.80.129 ping statistics --1000 packets transmitted, 1000 packets received, 0% packet loss round-trip min/avg/max/mdev = 0.465/0.535/4.202/0.337 ms WAE7326-CCO#
Cisco Public
21
Verify CPU Utilization

WAE CPU utilization is available in three (3) places:
Central Manager GUI, WAE Manager GUI, CLI
Use CLI for real-time CPU utilization monitoring:

WAE-612# show processes system count 10 delay 5 < .. > procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 0 772548 151944 161944 0 0 2 7 227 185 1 1 98 0 1 0 0 772628 151948 161940 0 0 0 15 1010 1518 1 1 98 0 0 0 0 772436 151956 161932 0 0 0 18 1006 1401 0 0 99 0 0 0 0 772140 151956 161932 0 0 0 8 1010 1400 1 0 98 0 Subtract id column from 1 0 0 772204 151956 161932 0 0 0 6 1009 1421 1 0 98 0 100 to get current 0 0 0 772092 151956 161932 CPU 0 0 0 6 1005 1422 0 0 99 0 1 0 0 772300 151956 161932 0 0 0 6 1005 1413 0 0 99 0 utilization 0 0 0 772380 151956 161932 0 0 0 8 1003 1578 1 1 98 0 0 0 0 772652 151960 161996 0 0 0 6 1005 1349 0 0 99 0 1 0 0 772492 151964 161992 0 0 0 8 1007 1508 0 0 99 0 WAE-612#
22
Verify Disk Health and Status

Check disk drive status through CLI for failed drives
WAE-612# show disks details Physical disk information: disk00: Present (h00 c00 i128 l00 - Int DAS-SAS) disk01: Present (h00 c01 i128 l00 - Int DAS-SAS) < .. > Software RAID devices: DEVICE NAME TYPE /dev/md0 RAID-1 /dev/md1 RAID-1 /dev/md2 RAID-1 /dev/md3 RAID-1 /dev/md4 RAID-1 /dev/md5 RAID-1 /dev/md6 RAID-1
286095MB(279.4GB) 286095MB(279.4GB)
STATUS NORMAL NORMAL NORMAL NORMAL NORMAL NORMAL NORMAL
OPERATION OPERATION OPERATION OPERATION OPERATION OPERATION OPERATION
PHYSICAL DEVICES disk00/00[GOOD] disk00/01[GOOD] disk00/02[GOOD] disk00/03[GOOD] disk00/04[GOOD] disk00/05[GOOD] disk00/06[GOOD]
AND STATUS disk01/00[GOOD] disk01/01[GOOD] disk01/02[GOOD] disk01/03[GOOD] disk01/04[GOOD] disk01/05[GOOD] disk01/06[GOOD]
Disk encryption feature is disabled. WAE-612#

23
Verify Disk Health and Status

Check for early indications of disk drive failure and proactively replace them
WAE-612# show disks failed-sectors disk00 ========= (none) WAE-612# WAE-612# show disks tech-support === disk00 === Device: IBM-ESXS GNA300C3ESTT0Z N Version: BH0G Serial number: J80LP6XK Device type: disk Transport protocol: SAS Local Time is: Wed Apr 25 19:32:30 2007 UTC Device supports SMART and is Enabled Temperature Warning Enabled SMART Health Status: OK WAE-612#
24
Troubleshooting Management Services
WAE is offline or is unable to reach the CM
Cisco Public
25
Verify WAE CMS Service Application Accelerator

Verify service status
WAE-612# show cms info Device registration information : Device Id Device registered as Current WAAS Central Manager Registered with WAAS Central Manager Status Time of last config-sync CMS services information : Service cms_ce is not running WAE-612#
= = = = = =
191 WAAS Application Engine 10.88.80.133 10.88.80.133 Offline Wed Apr 25 22:17:57 2007
Service status on CM
Service is not running

26
Verify WAE CMS Service Central Manager

Verify service status
WAE-612# show cms info Device registration information : Device Id Device registered as Current WAAS Central Manager role CMS services information : Service cms_httpd is running Service cms_cdm is running WAE-612#
= 144 = WAAS Central Manager = Primary
Current CM role
Service status
27
Restoring CMS Operation

Registration information is kept on both the WAE and Central Manager (CM) If the information is out of sync, the WAE will not be able to communicate with the CM Possible scenarios:
CM database has been lost. WAE still has information and tries to contact the CM. WAE is not known to the CM
WAE-612# cms deregister force
WAE lost database, but it still registered to CM. Delete WAE from CM database and re-enable CMS
Cisco Public
28
Verify Clock Synchronization

Each WAE system clock must be synchronized with the CM clock for proper reporting and ease of troubleshooting
Cisco Public
29
Verify Clock Synchronization

Verify clock details and NTP status from the CLI
WAE-612# show ntp status Synchronization with CM ntp enabled is based on UTC time server list: 10.88.121.253 remote refid st t when poll reach delay offset jitter ============================================================================== 10.88.121.253 10.81.254.202 2 11 64 1 1.199 0.499 0.000 WAE-612# WAE-612# show clock detail Local time: Sun Apr 29 01:28:25 CST 2007 UTC time: Sun Apr 29 07:28:25 UTC 2007 Epoch: 1177831705 seconds UTC offset: -21600 seconds (-6 hours 0 minutes) WAE-612#
Cisco Public
30
Central Manager Alarms

The Cisco WAAS Central Manager provides an alarm reporting facility Device alarms can be triggered, which cause the system or device status to change, notifying the administrator of a condition that requires attention
Cisco Public
31
Central Manager Alarms (Cont.)

When you click the severity column in the device list, or click the system status severity indicator, a popup window appears This window contains a list of all alarms that require attention. Moving the mouse over the alarm provides a menu of options for troubleshooting the alarm A list of alarms and a download location for the alarm book are contained in the notes
Cisco Public
32
WAAS Log Files

The following directories are used by Cisco WAAS for log files:
/local1 - Root directory for all log files /local1/logs - Service log files (aka admin logs) /local1/errorlog - Service log files (aka debug logs) /local1/core_dir - Process core cump files
File system navigation commands;

cd pwd dir type-tail <filename> <lines> [| | follow] find-pattern
33
WAAS System Report

Help Us Help You The WAAS system report (sysreport) contains a compressed archive containing all relevant support and system health information The sysreport includes the following:
CLI command output, platform configuration and logs, platform state information, print services configuration and logs, authentication configuration and logs, logs for internal services and acceleration, CMS configuration and logs, system logs, etc.
The sysreport can be generated from the WAE Manager GUI or CLI:
WAE-612# copy sysreport <disk | ftp | tftp> ...
Cisco Public
34
Network Interception
Cisco Public
35
Network Interception Methods

Web Cache Communication Protocol (WCCP) Policy-Based Routing (PBR) L4 Content Switch
Content Switching Module (CSM) Application Control Engine (ACE)
Physical Inline
Cisco Public
36
WCCP
Platform Recommendations
Function Support / Recommend
Assignment
Software ISR & 7200

Hash Only
Cat 6500 Sup720 Sup32

Mask or Hash / Mask
Cat 6500 Sup2

Mask or Hash / Mask L2 or GRE / L2 Full extended ACL In or Out / In IP Forward or L2 / IP Forward 12.1(27)E; 12.2(18)SXF12 IP Forward or L2 / IP Forward 12.2(31)SG IP Forward or L2 / IP Forward 12.2(37)SE No Redirect List Support (planned) In only Extended ACL no deny support In only L2 only L2 only
Cat 4500
Mask only
Cat 3560/3750
Mask only
Forwarding
GRE Only
L2 or GRE / L2 or GRE
Forwarding Redirect List Direction
Full extended ACL In or Out / In
Full extended ACL In or Out / In IP Forward only
Return
GRE only
IOS
12.1(14); 12.2(26); 12.3(13); 12.4(10); 12.1(3)T; 12.2(14)T; 12.3(14)T5; 12.4(9)T1

12.2(18)SXF12
Cisco Public
37
WCCPv2 Interception Verification (Router)

Router# show ip wccp Global WCCP information: Router information: Router Identifier: Protocol Version: Service Identifier: 61 Number of Service Group Clients: Number of Service Group Routers: Total Packets s/w Redirected: Process: Fast: CEF: Service mode: Service access-list: Total Packets Dropped Closed: Redirect access-list: Total Packets Denied Redirect: Total Packets Unassigned: Group access-list: Total Messages Denied to Group: Total Authentication failures: Total Bypassed Packets Received: --More- 2008 Cisco Systems, Inc. All rights reserved. Cisco Public
Client = WAE
10.88.81.242 2.0
1 1 68755 2 0 68753 Open -none0 -none0 0 -none0 0 0
38

Router# show ip wccp Global WCCP information: Router information: Router Identifier: Protocol Version: 10.88.81.242 2.0
Service Identifier: 61 Number of Service Group Clients: 1 Number of Service Group Routers: 1 Total Packets s/w Redirected: 68755 Process: 2 Fast: 0 CEF: 68753 Service mode: Open Service access-list: -noneTotal Packets Dropped Closed: 0 Redirect access-list: -noneVerify that counters are Total Packets Denied Redirect: 0 incrementing on software-based Total Packets Unassigned: 0 Group access-list: -noneplatforms Total Messages Denied to Group: 0 Total Authentication failures: 0 Total Bypassed Packets Received: 0 --More- 2008 Cisco Systems, Inc. All rights reserved. Cisco Public
39

Router# show ip wccp Global WCCP information: Router information: Router Identifier: Protocol Version: Service Identifier: 61 Number of Service Group Clients: Number of Service Group Routers: Total Packets s/w Redirected: Process: Fast: CEF: Service mode: Service access-list: Total Packets Dropped Closed: Redirect access-list: Total Packets Denied Redirect: Total Packets Unassigned: Group access-list: Total Messages Denied to Group: Total Authentication failures: Total Bypassed Packets Received: --More- 2008 Cisco Systems, Inc. All rights reserved. Cisco Public
10.88.81.242 2.0
Counter will increment for 1 68755 that match service packets 2 group but do not match 0 redirect-list 68753
Open -none0 -none0 0 -none0 0 0
40

Router# show ip wccp Global WCCP information: Router information: Router Identifier: Protocol Version: 10.88.81.242 2.0
Service Identifier: 61 Number of Service Group Clients: 1 Number of Service Group Routers: 1 Total Packets s/w Redirected: 68755 Process: 2 Fast: 0 for every packet Increments CEF: 68753 received with Service mode: Openincorrect service Service access-list: -nonegroup password Total Packets Dropped Closed: 0 Redirect access-list: -noneTotal Packets Denied Redirect: 0 Total Packets Unassigned: 0 Group access-list: -noneTotal Messages Denied to Group: 0 Total Authentication failures: 0 Total Bypassed Packets Received: 0 --More- 2008 Cisco Systems, Inc. All rights reserved. Cisco Public
41
Interception Verification (Router) (Cont.)

Verify WAE state in service group
Router# show ip wccp 61 detail WCCP Client information: WCCP Client ID: Protocol Version: State: Initial Hash Info: Assigned Hash Info: Hash Allotment: Packets s/w Redirected: Connect Time: Bypassed Packets Process: Fast: CEF:
10.88.81.4 2.0 Usable 00000000000000000000000000000000 00000000000000000000000000000000 FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF 256 (100.00%) 2452 01:19:46 0 0 0
% of hash buckets assigned
Current time in the service group

42
WCCPv2 Interception Verification (WAE)

WAE-612# show wccp services Services configured on this File Engine TCP Promiscuous 61 Verify TCP Promiscuous 62
WAE-612# show wccp status WCCP version 2 is enabled and currently active WAE-612# show wccp routers
WCCP is configured and enabled
Router Information for Service: TCP Promiscuous 61 Routers Configured and Seeing this File Engine(1) Router Id Sent To Recv ID 10.88.81.242 10.88.81.1 0000A631 Routers not Seeing this File Engine -NONERouters Notified of but not Configured -NONEMulticast Addresses Configured Verify bi-directional -NONE-
communication with WCCPenabled routers

43
Cisco Public
show wccp gre

WAE-612# show wccp gre Transparent GRE packets received: 55311 Transparent non-GRE packets received: 0 Transparent non-GRE non-WCCP packets received: 0 Total packets accepted: 5051 Invalid packets received: 0 Packets received with invalid service: 0 Packets received on a disabled service: 0 Packets received too small: 0 Either of these counters Packets dropped due to zero TTL: 0 should be incrementing if Packets dropped due to bad buckets: 0 Packets dropped due to no redirect address: WCCP 0 redirection is Packets dropped due to loopback redirect: 0 working Pass-through pkts dropped on assignment update:0 Connections bypassed due to load: 0 Packets sent back to router: 0 Packets sent to another WAE: 0 GRE fragments redirected: 0 GRE encapsulated fragments received: 0 Packets failed encapsulated reassembly: 0 Packets failed GRE encapsulation: 0 --More--
Cisco Public
44
show wccp gre

WAE-612# show wccp gre Transparent GRE packets received: 55311 Transparent non-GRE packets received: 0 Transparent non-GRE non-WCCP packets received: 0 Total packets accepted: 5051 Invalid packets received: 0 Packets received with invalid service: 0 Packets received on a disabled service: 0 Packets received too small: 0 packets redirected For Packets dropped due to zero TTL: 0 using WCCP l2-redirect Packets dropped due to bad buckets: 0 Packets dropped due to no redirect address: forwarding 0 method Packets dropped due to loopback redirect: 0 Pass-through pkts dropped on assignment update:0 Connections bypassed due to load: 0 Packets sent back to router: 0 Packets sent to another WAE: 0 GRE fragments redirected: 0 GRE encapsulated fragments received: 0 Packets failed encapsulated reassembly: 0 Packets failed GRE encapsulation: 0 --More--
Cisco Public
45
show wccp gre

WAE-612# show wccp gre Transparent GRE packets received: 55311 Transparent non-GRE packets received: 0 Transparent non-GRE non-WCCP packets received: 0 Total packets accepted: 5051 Invalid packets received: 0 Packets received with invalid service: 0 Packets received on a disabled service: 0 Packets received too small: 0 packets L2 redirected For Packets dropped due to zero TTL: 0 using non-WCCP (L4, PBR, Packets dropped due to bad buckets: 0 Packets dropped due to no redirect address: etc.) 0 interception method Packets dropped due to loopback redirect: 0 Pass-through pkts dropped on assignment update:0 Connections bypassed due to load: 0 Packets sent back to router: 0 Packets sent to another WAE: 0 GRE fragments redirected: 0 GRE encapsulated fragments received: 0 Packets failed encapsulated reassembly: 0 Packets failed GRE encapsulation: 0 --More--
Cisco Public
46
show wccp gre

WAE-612# show wccp gre Transparent GRE packets received: 55311 Transparent non-GRE packets received: 0 Transparent non-GRE non-WCCP packets received: 0 Total packets accepted: 5051 Invalid packets received: 0 Packets received with invalid service: 0 Packets received on a disabled service: 0 Packets received too small: 0 Packets accepted for Packets dropped due to zero TTL: 0 optimization (i.e. autoPackets dropped due to bad buckets: 0 Packets dropped due to no redirect address: discovery 0 found peer WAE) Packets dropped due to loopback redirect: 0 Pass-through pkts dropped on assignment update:0 Connections bypassed due to load: 0 Packets sent back to router: 0 Packets sent to another WAE: 0 GRE fragments redirected: 0 GRE encapsulated fragments received: 0 Packets failed encapsulated reassembly: 0 Packets failed GRE encapsulation: 0 --More--
Cisco Public
47
show wccp gre

WAE-612# show wccp gre Transparent GRE packets received: 55311 Transparent non-GRE packets received: 0 Transparent non-GRE non-WCCP packets received: 0 Only includes packets Total packets accepted: 5051 Invalid packets received: 0 handled using WCCP Packets received with invalid service: 0 return method Packets received on a disabled service: 0 Packets received too small: 0 Packets dropped due to zero TTL: 0 Packets dropped due to bad buckets: 0 Packets dropped due to no redirect address: 0 Packets dropped due to loopback redirect: 0 Pass-through pkts dropped on assignment update:0 Connections bypassed due to load: 0 Packets sent back to router: 0 Packets sent to another WAE: 0 GRE fragments redirected: 0 GRE encapsulated fragments received: 0 Packets failed encapsulated reassembly: 0 Packets failed GRE encapsulation: 0 --More--
Cisco Public
48
show wccp gre

WAE-612# show wccp gre Transparent GRE packets received: 55311 Transparent non-GRE packets received: 0 Transparent non-GRE non-WCCP packets received: 0 Packets forwarded directly Total packets accepted: 5051 Invalid packets received: 0 between WAEs due to Packets received with invalid service: 0 WCCP flow protection Packets received on a disabled service: 0 Packets received too small: 0 Packets dropped due to zero TTL: 0 Packets dropped due to bad buckets: 0 Packets dropped due to no redirect address: 0 Packets dropped due to loopback redirect: 0 Pass-through pkts dropped on assignment update:0 Connections bypassed due to load: 0 Packets sent back to router: 0 Packets sent to another WAE: 0 GRE fragments redirected: 0 GRE encapsulated fragments received: 0 Packets failed encapsulated reassembly: 0 Packets failed GRE encapsulation: 0 --More--
Cisco Public
49
Packet Capture Debugs

Packets can be captured on all WAAS interfaces using one of the following CLI tools:
tethereal tcpdump
The Problem?
A packet capture taken on the WAE will contain packets of all TCP segments
How can you differentiate between original and optimized connections?
Cisco Public
50
Multiple TCP Segments

header IP eth src Client Client dst Server Router header IP eth src Client Router dst Server
61 in Client
62 in
header IP eth
src Server Router
dst Client Client
exclude in
header IP eth
src Server
dst Client Router
header IP eth
src Client Router
dst Server WAE
header IP eth
src Server Router
dst Client WAE
WAE
header IP eth src Server WAE dst Client Router header IP eth src Client WAE dst Server Router
Cisco Public
51
Displaying Optimized Segments

61 in Client
62 in
exclude in
header IP eth
src Server
dst Client Router
header IP eth
src Server Router
dst Client WAE
WAE
Wireshark Display Filter: tcp && ip.src == <ServerIP> && eth.dst == <WAE MAC addr>
52
Displaying Original Segments

61 in Client
62 in
header IP eth
src Server Router
dst Client Client
exclude in
header IP eth
src Client Router
dst Server WAE
WAE
header IP eth
src Server WAE
dst Client Router
Wireshark Display Filter: tcp && ip.src == <ServerIP> && eth.src == <WAE MAC addr>
Cisco Public
53
Inline Interception
WAE installed in-path between switch and router or firewall 2 pairs of inline ports Ports fail-to-wire Configurable interception based on VLANs Supported on all WAE appliances On a WAE, either Inline or WCCP may be used, but not both
WAN1 LAN1 WAN0 LAN0
Cisco WAE 4-port inline card
WAN WAE1
54
Inline Module: show hardware

WAE-511# show hardware Cisco Wide Area Application Services Software (WAAS) Copyright (c) 1999-2007 by Cisco Systems, Inc. Cisco Wide Area Application Services Software Release 4.0.7 ... Compiled 10:22:10 Feb 25 2007 by cnbuild System was restarted on Tue Apr 24 20:45:39 2007. The system has been up for 5 days, 9 hours, 4 minutes, 54 seconds. CPU 0 is GenuineIntel Intel(R) Celeron(R) CPU 2.80GHz (rev 3) ... Total 1 CPU. 512 Mbytes of Physical memory. 1 CD ROM drive (CD-224E) 2 GigabitEthernet interfaces 2 InlineGroup interfaces. 1 Console interface
Cisco Public
55
show interface inlineGroup

WAE-612# show interface inlineGroup 1/0 Interface is in intercept operating mode. Standard NIC mode is off. Disable bypass mode is off. VLAN IDs configured for inline interception: All Watchdog timer is enabled. Timer frequency: 1600 ms. Autoreset frequency 500 ms. The watchdog timer will expire in 1452 ms. Intercept WAE-612#
operating mode or bypass operating mode
Intercept operating modePacket are passed to WAAS for (potential) optimization Bypass operating modeMechanical bypass between ports in InlineGroup
56
show interface inlinePort

WAE-612# show interface inlinePort 1/0/wan Device name : eth4. Bypass master interface. Packets Received : 54231 Packets Intercepted: 0 Packets Bridged : 54231 Packets Forwarded : 0 Packets Dropped : 0 Packets Received on native :0 Active flows for this interface : 0 ... WAE-612# show interface inlinePort 1/0/lan Device name : eth5. Bypass slave interface. Packets Received : 334602 Packets Intercepted: 0 Packets Bridged : 334599 Packets Forwarded : 0 Packets Dropped : 3 Packets Received on native :0 Active flows for this interface : 0 ... WAE-612#
Use sh int inlinep to determine device name for any inlinePort The device name is needed for packet captures
Cisco Public
57
show interface inlinePort

WAE-612# show interface inlinePort 1/0/wan Device name : eth4. Bypass master interface. Packets Received : 54231 Packets Intercepted: 0 Packets Bridged : 54231 Packets Forwarded : 0 Packets Dropped : 0 Packets Received on native :0 Active flows for this interface : 0 ... WAE-612# show interface inlinePort 1/0/lan Device name : eth5. Bypass slave interface. Packets Received : 334602 Packets Intercepted: 0 Packets Bridged : 334599 Packets Forwarded : 0 Packets Dropped : 3 Packets Received on native :0 Active flows for this interface : 0 ... WAE-612#
Packets that the bridging layer decided to pass-through
Packets from L3 and above layers that were sent out from an inline port
Cisco Public
58
Auto-Discovery
Cisco Public
59
Auto Discovery - Refresher

Clients SYN forwarded by WAE1 with Auto Discovery Options SYN forwarded again by WAE2 with Auto Discovery Options SYN/ACK processed by WAE2, Auto Discovery Options added SYN/ACK processed by WAE1, to determine Auto Discovery success, SYN/ACK sent to Client and ACK with options sent to WAE2 ACK with options processed by WAE2 to determine Auto Discovery success, ACK sent to server.
WCCPv2 WCCPv2 or or PBR PBR WCCPv2 WCCPv2 or or PBR PBR
WAN
WAE1
WAE2
Cisco Public
60
Automatic Discovery Verification (Cont.)

Client WAE WAE Server
Client:Server TCP SYN
Cisco Public
61

Client:Server TCP SYN
Client:Server TCP SYN+OPT
Cisco Public
62

Cisco Public
63

Server:Client TCP SYN-ACK OPT
Server:Client TCP SYN-ACK
Cisco Public
64
Auto-Discovery Connection List

WAE511-CCO# sh tfo conn summ Optimized Connection List Policy summary order: Our's, Peer's, Negotiated, Applied F: Full optimization, D: DRE only, L: LZ Compression, T: TCP Optimization Local-IP:Port 10.88.81.2:25967 10.88.81.2:25968 10.88.81.2:25969 10.88.81.25:15560 10.88.81.25:15810 10.88.81.25:15812 10.88.81.25:15828 Remote-IP:Port 10.88.80.135:4050 10.88.80.135:4050 10.88.80.135:4050 10.88.80.53:389 10.88.80.53:389 10.88.80.53:389 10.88.80.53:389 ConId 5 6 7 17 252 253 263 PeerId 00:14:5e:ac:2d:79 00:14:5e:ac:2d:79 00:14:5e:ac:2d:79 00:14:5e:ac:2d:79 00:14:5e:ac:2d:79 00:14:5e:ac:2d:79 00:14:5e:ac:2d:79 Policy F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F
Auto-Discovery Connection List E: Established, S: Syn, A: Ack, F: Fin, R: Reset s: sent, r: received, O: Options, P: Passthrough Local-IP:Port Remote-IP:Port Conn-Type 10.88.80.52:80 10.88.81.25:15900 Ext. Client 10.88.80.52:80 10.88.81.25:15868 Ext. Client 10.88.80.52:80 10.88.81.25:15836 Ext. Client 10.88.80.52:80 10.88.81.25:15901 Ext. Client 10.88.80.52:80 10.88.81.25:15869 Ext. Client 10.88.80.52:80 10.88.81.25:15837 Ext. Client
Orig-St Sr Sr Sr Sr Sr Sr
Term-St SsO SsO SsO SsO SsO SsO

65
Auto-Discovery States
Orig-St: Sr
Term-St: SsO
Orig-St: SrO
Term-St: SsO
Orig-St: SAs
Term-St: SArO
Orig-St: SAsO
Term-St: SAr
X
F,F,F,F Orig-St: EO Term-St: As F,F,F,F
66
Auto-Discovery Statistics
WAE-612# show tfo auto-discovery ... Auto discovery failure: No peer or asymmetric route: 0 Insufficient option space: 0 Invalid connection state: 0 Missing Ack conf: 0 Auto discovery success TO: Internal server: 40 External server: 0 Auto discovery success FOR: Internal client: 2902 External client: 0 Auto discovery success SYN retransmission: Zero retransmit: 2902 Not enough space in TCP One retransmit: 0 Two+ retransmit: 0 AD Options header for Auto discovery Miscellaneous: option Intermediate device: 0 RST received: 0 SYNs found with our device id: 0 SYN retransmit count resets: 0
67
WAE-612# show tfo auto-discovery ... Auto discovery failure: No peer or asymmetric route: Insufficient option space: Invalid connection state: Missing Ack conf: Auto discovery success TO: Internal server: External server: Auto discovery success FOR: Internal client: External client: Auto discovery success SYN retransmission: Zero retransmit: One retransmit: Two+ retransmit: Auto discovery Miscellaneous: Intermediate device: RST received: SYNs found with our device id: SYN retransmit count resets:
0 0 0 0 40 0 2902 0 2902 0 0 0 0 0 0
68
WAE-612# show tfo auto-discovery ... Auto discovery failure: No peer or asymmetric route: 0 Insufficient option space: 0 Invalid connection state: 0 Missing Ack conf: 0 Auto discovery success TO: Internal server: 40 Packet received with our External server: 0 Auto discovery success FOR: TCP option - indicates a Internal client: 2902 redirection loop External client: 0 Auto discovery success SYN retransmission: Zero retransmit: 2902 One retransmit: 0 Two+ retransmit: 0 Auto discovery Miscellaneous: Intermediate device: 0 RST received: 0 SYNs found with our device id: 0 SYN retransmit count resets: 0
own
69
Debug Auto-Discovery Using ACL

Create the ACL
WAE-612# conf t WAE-612(config)# ip access-list extended 199 WAE-612(config-ext-nacl)# permit ip host 10.88.81.25 any WAE-612(config-ext-nacl)# permit ip any host 10.88.81.25
Enable Logging
WAE-612(config)# logging console priority debug WAE-612(config)# logging console enable WAE-612(config)# end WAE-612# term mon
Enable Debugging
WAE-612# debug tfo connection auto-discovery 199
Cisco Public
70
Sample Debug Output

10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 drop table! Process it! 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 Ret=1, Send OPT! 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 10.88.81.25:3389 - 10.89.13.149:49152 10.89.13.149:49152 - 10.88.81.25:3389 10.88.81.25:3389 - 10.89.13.149:49152 10.88.81.25:3389 - 10.89.13.149:49152 found in drop table! Process it! 10.89.13.149:49152 - 10.88.81.25:3389 from table. ret=-2 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 10.88.81.25:3389 - 10.89.13.149:49152 10.88.81.25:3389 - 10.89.13.149:49152 10.89.13.149:49152 - 10.88.81.25:3389 10.88.81.25:3389 - 10.89.13.149:49152 10.88.81.25:3389 - 10.89.13.149:49152 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 10.89.13.149:49152 - 10.88.81.25:3389 peer_id=00:11:25:ab:43:32
- Entering syncache_rcv(). - Entering nostate_sm(). - nostate_sm: Address 1951580a NOT found in - Entering ad_parse_options(). - nostate_sm: Server 1951580a Not in table. Entering syncache_insert(). Entering ad_add_update_optimize_options(). Entering ad_add_opt_post(). Entering syncache_rcv(). syncache_lookup() - match found Entering syns_with_adopt_sm(). syns_with_adopt_sm: Address 950d590a NOT
- syns_with_adopt_sm: Remove addr 1951580a Entering ad_term_tcpip_params_save(). Entering ad_add_update_optimize_ok_options() Entering ad_prepare_skb(). ad_prepare_skb(). Alloc. new skb. Entering ad_parse_options(). Entering ad_add_opt_post(). Entering ad_transmit_skb(). Entering syncache_rcv(). syncache_lookup() - match found Entering synacks_with_adopt_sm(). Entering ad_parse_options(). ad_split_conn_common: AD Successful;
Cisco Public
71
Sample Debug Output

10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_term_split_conn(). 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_sock_init(). 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_sock_init(). 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_sock_nonopt_term_init(). 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_route_req(). dir: orig -> term 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_sock_opt_init(). 10.89.13.149:49152 - 10.88.81.25:3389 - seq nos: rcv_nxt=-749206040, snd_nxt=-334819512 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_route_req(). dir: term -> orig 10.89.13.149:49152 - 10.88.81.25:3389 - init policy: apply_dre=0, apply_lz=0, peer policy: apply_dre=0, apply_lz=0, final policy:apply_dre=0, apply_lz=0 10.89.13.149:49152 - 10.88.81.25:3389 - ad_wccp_add_flow adding flow to svc 62, bucket 26 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_gen_term_ack(). 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_alloc_prepare_skb(). 10.89.13.149:49152 - 10.88.81.25:3389 - opt_sk = 0xf1898980 10.89.13.149:49152 - 10.88.81.25:3389 - tp = 0xf125a28c, tp->write_seq = 1398277608 10.89.13.149:49152 - 10.88.81.25:3389 - ad_wccp_add_flow adding flow to svc 61, bucket 26 10.89.13.149:49152 - 10.88.81.25:3389 - Entering ad_transmit_skb(). 10.88.81.25:3389 - 10.89.13.149:49152 - Entering syncache_rcv(). 10.89.13.149:49152 - 10.88.81.25:3389 - syncache_lookup() - match found 10.88.81.25:3389 - 10.89.13.149:49152 - Entering acks_sm(). 10.89.13.149:49152 - 10.88.81.25:3389 - Entering syncache_drop().
Cisco Public
72
Redirection Problems
Proper network interception configuration is critical to the success of a WAAS deployment Failure to insure that egress packets from the WAE are not re-intercepted can lead to a redirection loop:
%WAAS-SYS-3-900000: 137.34.79.11:1192 - 137.34.77.196:139 - opt_syn_rcv: Routing Loop detected - Packet has our own devid. Packet dropped.
You can search syslog.txt for instances of the above message using the following command:
WAE-612# find match Routing Loop syslog.txt
As well as in the TFO filtering statistics:

WAE-612# show tfo filtering .. Syn packets dropped with our own id in the options: ..
0
73
Redirection Problems
During normal operation, there are 3 different connection segments for an optimized flow
WAN
In the event of an interception failure, packets from an original connection could be received on the optimized segment by a peer WAE
X
WAN
%WAAS-SYS-3-900000:src=10.56.46.183:4386 dst=10.56.46.164:445 Unoptimised packet received on optimized socket: seq=2796284443 end_seq=279628444

74
Firewall Integration
Stateful inspection of WAAS optimized traffic requires that the inspecting device understand the sequence number shift on optimized TCP connections The following software versions provide 100% interoperability with WAAS optimized connections:
IOSFW (Zone-based): 12.4(11)T2 or later
ASA/PIX: 7.2.3 or later
FWSM: 3.2.1 or later
Cisco Public
75
WAN Optimization
Cisco Public
76
Policy Engine Refresher

Application Definitionlogical grouping of traffic to help report statistics on the type of traffic Traffic ClassifierAccess Control List which helps choose connections based on IP addresses and ports Policy MapBinds the Application and Classifier with an action, which specifies the type of optimization, if any, to be applied
Static Policy MapConfigured on the device via CLI/GUI, is persistent until unconfigured Dynamic Policy MapAuto-configured and has a short life time, just enough to accept a new connection
Cisco Public
77
Policy Engine Example

policy-engine application name Streaming classifier Liquid-Audio match dst port eq 18888 exit classifier MS-NetShow match dst port eq 1755 exit classifier RTSP match dst port eq 554 match dst port eq 8554 exit classifier VDOLive match dst port eq 7000 exit map basic name Streaming classifier name Streaming classifier name Streaming classifier name Streaming classifier exit
Application Name
Well known application ports
Liquid-Audio action optimize full MS-NetShow action optimize full RTSP action optimize full VDOLive action optimize full
Maps binding the protocol to the application, and specifying an action

78
Verify Global TFO Status

WAE-612# show statistics tfo Total number of optimized connections : 4121 No. of active connections : 7 No. of pending (to be accepted) connections : 0 No. of connections closed normally : 3377 No. of connections closed with error : 737 Total number of peers : 1 No. of entries into overload mode : 0 No. of connections reset due to Socket write failure : 0 Socket read failure : 0 Number Opt socket close while waiting to write : of 14 Unopt socket close while waiting to write : 0 currently connections Opt socket error close while waiting to read : 429 being optimized Unopt socket error close while waiting to read : 5 DRE decode failure : 0 DRE encode failure : 0 Connection init failure : 0 Opt socket unexpected close while waiting to read : 0 Exceeded maximum number of supported connections : 0 Buffer allocation or manipulation failed : 0 Peer received reset from end host : 289 DRE connection state out of sync : 0 Memory allocation failed for buffer heads : 0
79

WAE-612# show statistics tfo Total number of optimized connections : 4121 No. of active connections : 7 No. of pending (to be accepted) connections : 0 No. of connections closed normally : 3377 No. of connections closed with error : 737 Total number of peers : 1 No. of entries into overload mode : 0 No. of connections reset due to Socket write failure : 0 Socket read failure : 0 Opt socket close while waiting to write : 14 Unopt socket close while waiting to write : 0 Opt socket error close while waiting to read : 429 Unopt socket error close while waiting to read : 5 DRE decode failure : 0 Current number DRE encode failure : 0 peer WAEs Connection init failure : 0 Opt socket unexpected close while waiting to read : 0 Exceeded maximum number of supported connections : 0 Buffer allocation or manipulation failed : 0 Peer received reset from end host : 289 DRE connection state out of sync : 0 Memory allocation failed for buffer heads : 0
of
80

WAE-612# show statistics tfo Total number of optimized connections : 4121 No. of active connections : 7 No. of pending (to be accepted) connections : 0 No. of connections closed normally : 3377 No. of connections closed with error : 737 Total number of peers : 1 No. of entries into overload mode : 0 No. of connections reset due to Socket write failure : 0 Socket read failure : 0 Opt socket close while waiting to write : 14 Unopt socket close while waiting to write : 0 Opt socket error close while waiting to read : 429 Unopt socket error close while waiting to read : 5 DRE decode failure : the 0 maximum Number of times DRE encode failure : 0 number of supported (optimized) Connection init failure : 0 connections has Opt socket unexpected close while waiting to read : been 0 exceeded Exceeded maximum number of supported connections : 0 Buffer allocation or manipulation failed : 0 Peer received reset from end host : 289 DRE connection state out of sync : 0 Memory allocation failed for buffer heads : 0
81
Inspect TFO Connections

show tfo connection summary
WAE-612# show tfo connection summary Optimized Connection List Policy summary order: Our's, Peer's, Negotiated, Applied F: Full optimization, D: DRE only, L: LZ Compression, T: TCP Optimization Local-IP:Port 10.88.81.18:18238 10.88.81.25:1471 10.88.81.25:1663 10.88.81.25:1665 10.88.81.25:1666 10.88.81.18:48864 10.88.81.25:1882 Remote-IP:Port 10.88.80.53:445 10.88.80.53:3268 10.88.80.53:389 10.88.80.53:389 10.88.80.53:389 10.88.80.53:389 10.88.80.53:445 ConId 14 930 1885 1886 1887 2270 2315 PeerId 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 Policy F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F
Pass-Through Connections Local-IP:Port Remote-IP:Port 10.88.81.2:43520 10.88.80.130:4050 10.88.80.130:4050 10.88.81.2:43520 WAE-612#
Conn Type PT In Progress PT In Progress
Includes passthrough connections

82
Cisco Public

show tfo connection
WAE-612# show tfo connection Connection Id: 930 Start time: Peer Id: Connection type: Source IP Address: Source Port Number: Destination IP Address: Destination Port Number: Our policy: Peer policy: Negotiated policy: Applied policy: Mon Apr 30 11:10:38 2007 00:11:25:ab:43:32 Ext. Client 10.88.81.25 1471 10.88.80.53 3268 TCP_OPTIMIZE + DRE + LZ TCP_OPTIMIZE + DRE + LZ TCP_OPTIMIZE + DRE + LZ TCP_OPTIMIZE + DRE + LZ Source <-> WAAS (Original) 0x52ffce40 N. Read Wait P. Read Wait Write Shutdown Writing N/A 0 0 0 9477 6528 35 35 68 0 0.067(34) msec: 0.063(68) 0.104(35)
Cisco Public
Peer WAE for optimized connection
Current Read State: Previous Read State: Current Write State: Previous Write State: TCP opt. only xfer mode: Read Buffer Size: Write Buffer Size: Work Buffer Size: Bytes Read: Bytes Written: Number of Reads: Number of Writes: Number of Encodes: Number of Decodes: Read Q. latency in msec: Encode/decode latency in Write Q latency in msec: WAE-612#
WAAS <-> Dest (Optimized) 0x52ffccd8 N. Read Wait P. Read Wait D. Write Wait Writing N/A 0 0 0 2837 4011 71 72 0 69 0.059(69) 0.045(35) 0.096(35) 83

show tfo connection
Cisco Public
WAAS <-> Dest (Optimized) 0x52ffccd8 N. Read Wait P. Read Wait D. Write Wait Writing N/A 0 0 0 2837 4011 71 72 0 69 0.059(69) 0.045(35) 0.096(35)
Connection was initiated from outside the WAE
84

show tfo connection
Cisco Public
Basic connection details
85

show tfo connection
Cisco Public
Optimization levels negotiated during auto-discovery
86

show tfo connection
Cisco Public
This is an Edge WAE, so the original connection is the nonoptimized connection between the client and the WAE On the Core WAE, the original connection is the one between the WAE and the origin server

show tfo connection
Cisco Public
Statistics pertaining to each connection

show tfo connection
Cisco Public
The amount of data buffered in various queues at the instant the command was executed

show tfo connection
Cisco Public
The average amount of time after data is read from the network WAAS <-> Dest before processing begins (Optimized) 0x52ffccd8 (the # is brackets in the # of samples) N. Read Wait
P. Read Wait D. Write Wait Writing N/A 0 0 0 2837 4011 71 72 0 69 0.059(69) 0.045(35) 0.096(35) 90

show tfo connection
Cisco Public
The average amount of time taken by DRE to encode/decode the data
91

show tfo connection
Cisco Public
The average amount of time after data is returned from DRE before WAAS <-> Dest being written to the network (the # (Optimized) 0x52ffccd8 is brackets in the # of samples) N. Read Wait
P. Read Wait D. Write Wait Writing N/A 0 0 0 2837 4011 71 72 0 69 0.059(69) 0.045(35) 0.096(35) 92
TFO Error Logs

Extensive connection logging information is written in error logs under:
/local1/errorlog/errlog-tcpproxy-<timestamp>
WAE-612# lls errlog-tcpproxy-0000-20070424* size time of last change -------------- ------------------------6382170 Mon Apr 30 17:52:30 2007 errlog-tcpproxy-0000-20070424-204731 WAE-612# name ----------/local1/errorlog/
WAE0612# type-tail errlog-tcpproxy-0000-20070424-204731 100 Mon Apr 30 17:48:30 2007: monitor_worker_thread: polling ok. Mon Apr 30 17:48:31 2007: 10.88.81.25:1964 - 10.88.80.51:80 - net_reset:1230: Entering (reset code=13, Peer received reset from end host) Mon Apr 30 17:48:31 2007: 10.88.81.25:1963 - 10.88.80.51:80 - net_reset:1230: Entering (reset code=13, Peer received reset from end host) Mon Apr 30 17:48:35 2007: monitor_worker_thread: polling ok. ...
Connection was reset by end-host at peer location

93
Inspect DRE Connections

show statistics dre connection
WAE-612# show statistics dre connection Conn Id Peer No Client-ip:port Server-ip:port Encode-in/ Decode-in 1KB/ 1KB/ 1KB/ 1KB/ 1KB/ 1KB/ 1KB/ 0B/ 1KB/ 1KB/ 1KB/ 1KB/ Status (A-Active) (C-Closed) 1KB C(3m45s) 1KB C(3m45s) 1KB C(3m45s) 1KB C(3m45s) 1KB C(3m45s) 1KB C(3m45s) 1KB C(3m46s) 0B C(7m15s) 3KB C(7m4s) 1KB A 1KB C(8m45s) 1KB C(8m45s)
13726 3 13725 3 13724 3 13723 3 13722 3 13721 3 13720 3 13714 3 13713 3 13712 3 13710 3 13709 3 WAE-612#
10.88.81.18:10887 10.88.81.18:10886 10.88.81.18:10885 10.88.81.18:10884 10.88.81.18:10883 10.88.81.18:10882 10.88.81.18:10881 10.88.81.25:2119 10.88.81.25:2118 10.88.81.18:35078 10.88.81.18:52316 10.88.81.18:52315
10.88.80.53:445 10.88.80.53:445 10.88.80.53:445 10.88.80.53:445 10.88.80.53:445 10.88.80.53:445 10.88.80.53:445 10.88.80.53:139 10.88.80.53:445 10.88.80.53:389 10.88.80.53:445 10.88.80.53:445
Connections optimized using DRE are listed here (some historic connections included)
Cisco Public
94

show statistics dre connection id <id>
WAE-612# show statistics dre connection id 13712 Conn-ID: 13712 10.88.81.18:35078 -- 10.88.80.53:389 Peer No: 3 Status: Closed -----------------------------------------------------------------------------Open at 04/30/2007 20:00:21, Close at 04/30/2007 20:15:41, Duration: 920 secs Encode: Overall: msg: 9, in: 1903 B, out: 478 B, ratio: 74.88% DRE: msg: 9, in: 1903 B, out: 527 B, ratio: 72.31% DRE Bypass: msg: 0, in: 0 B LZ: msg: 4, in: 400 B, out: 351 B, ratio: 12.25% LZ Bypass: msg: 5, in: 127 B Avg latency: 0.199 ms Message size distribution: Connected related information: 0-1K=100% 1K-5K=0% 5K-15K=0% 15K-25K=0% 25K-40K=0% >40K=0% - ID, Client IP/port, etc. Decode: - Status, Active or Closed Overall: msg: 9, in: 1666 B, out: 2513 B, ratio: 33.70% DRE: msg: 9, in: 1740 B, out: 2513 B, ratio: 30.76% - Time and Duration DRE Bypass: msg: 0, in: 0 B LZ: msg: 5, in: 1566 B, out: 1640 B, ratio: 4.51% LZ Bypass: msg: 4, in: 100 B Avg latency: 0.121 ms Message size distribution: 0-1K=88% 1K-5K=11% 5K-15K=0% 15K-25K=0% 25K-40K=0% >40K=0% WAE-612#
95

show statistics dre connection id <id>
WAE-612# show statistics dre connection id 13712 Conn-ID: 13712 10.88.81.18:35078 -- 10.88.80.53:389 Peer No: 3 Status: Closed -----------------------------------------------------------------------------Open at 04/30/2007 20:00:21, Close at 04/30/2007 20:15:41, Duration: 920 secs Encode: Overall: msg: 9, in: 1903 B, out: 478 B, ratio: 74.88% DRE: msg: 9, in: 1903 B, out: 527 B, ratio: 72.31% DRE Bypass: msg: 0, in: 0 B LZ: msg: 4, in: 400 B, out: 351 B, ratio: 12.25% LZ Bypass: msg: 5, in: 127 B Avg latency: 0.199 ms Message size distribution: 0-1K=100% 1K-5K=0% 5K-15K=0% 15K-25K=0% 25K-40K=0% >40K=0% Decode: Overall: msg: 9, in: 1666 B, out: 2513 B, ratio: 33.70% DRE: msg: 9, in: 1740 B, out: 2513 B, ratio: 30.76% DRE Bypass: msg: 0, in: 0 B Encode/Decode related statistics: LZ: msg: 5, in: 1566 B, out: 1640 B, ratio: 4.51% Number of messages LZ Bypass: msg: 4, in: 100 B - How many DRE applied Avg latency: 0.121 ms Message size distribution: - How many LZ applied 0-1K=88% 1K-5K=11% 5K-15K=0% 15K-25K=0% 25K-40K=0% >40K=0% - How many bypassed WAE-612#
96
DRE Debugging: Low Compression

Messages Are Too Small
WAE-612# show statistics dre connection id 13712 Conn-ID: 13712 10.88.81.18:35078 -- 10.88.80.53:389 Peer No: 3 Status: Closed -----------------------------------------------------------------------------are the message Open at 04/30/2007 20:00:21, Close at 04/30/2007 What 20:15:41, Duration: 920sizes? secs Encode: Overall: msg: 9, in: 1903 B, out: 478 B, ratio: 74.88% DRE: msg: 9, in: 1903 B, out: 527 B, ratio: 72.31% DRE Bypass: msg: 0, in: 0 B LZ: msg: 4, in: 400 B, out: 351 B, ratio: 12.25% LZ Bypass: msg: 5, in: 127 B Avg latency: 0.199 ms Message size distribution: 0-1K=100% 1K-5K=0% 5K-15K=0% 15K-25K=0% 25K-40K=0% >40K=0% Decode: Overall: msg: 9, in: 1666 B, out: 2513 B, ratio: 33.70% DRE: msg: 9, in: 1740 B, out: 2513 B, ratio: 30.76% DRE Bypass: msg: 0, in: 0 B LZ: msg: 5, in: 1566 B, out: 1640 B, ratio: 4.51% LZ Bypass: msg: 4, in: 100 B Avg latency: 0.121 ms Message size distribution: 0-1K=88% 1K-5K=11% 5K-15K=0% 15K-25K=0% 25K-40K=0% >40K=0% WAE-612#
97

DRE Cache Is Too Small
AST6-CCO-01#show statistics dre Cache: Status: Usable, Oldest Data (age): 24d22h Total usable disk size: 30719 MB, Used: 13.50% Hash table RAM size: 66 MB, Used: 20.00% Connections: ... AST6-CCO-01# Total (cumulative): 3241 Active: 12
Check DRE cache age

Should be several weeks old Minimum of 2 weeks
98
Cisco Public

DRE Bypassed
WAE-612# show statistics dre ... Encode: Overall: msg: 89617, in: DRE: msg: 89617, in: DRE Bypass: msg: 6, in: LZ: msg: 24068, in: LZ Bypass: msg: 65549, in: Avg latency: 0.187 ms Message size distribution: 0-1K=99% 1K-5K=0% 5K-15K=0% Decode: Overall: msg: 88088, in: DRE: msg: 88084, in: DRE Bypass: msg: 4, in: LZ: msg: 31372, in: LZ Bypass: msg: 56716, in: Avg latency: 0.154 ms Message size distribution: 0-1K=95% 1K-5K=4% 5K-15K=0% WAE-612#
4911 4910 1428 3553 1413
KB, out: KB, out: B KB, out: KB
3247 KB, ratio: 4965 KB, ratio: 1834 KB, ratio:
33.88% 0.00% 48.37%
15K-25K=0% 5439 8541 4258 4093 1346
How much data bypassed DRE?

11139 KB, ratio: 11135 KB, ratio: 7200 KB, ratio: 51.17% 23.29% 43.15%
25K-40K=0%
>40K=0%
KB, out: KB, out: B KB, out: KB
15K-25K=0%
25K-40K=0%
>40K=0%
99

DRE Bypassed
WAE-612# show statistics dre detail Cache: Status: Usable, Oldest Data (age): 24d22h Total usable disk size: 30719 MB, Used: 13.50% Hash table RAM size: 66 MB, Used: 20.00% ............................................................................. Connection details: Chunks: encoded 193379, decoded 126026, anchor(forced) 795(6955) Ack: msg 88202, size 250 KB Encode bypass due to: remote wait-Q due to nack: messages: 6, size: 1428 B last partial chunk: chunks: 70091, size: 1165 KB Nacks: total 0 R-tx: total 3 chunk-miss 3 Aggregation encode: Retrasmissions: 3 level 0: chunks: 103855 hits: 6005 miss: 27231 level 1: chunks: 144 hits: 119 miss: 25 level 2: chunks: 2 hits: 0 miss: 2 level 3: chunks: 0 hits: 0 miss: 0 Aggregation decode: Collisions: 0 level 0: chunks: 125704 hits: 18121 miss: 0 level 1: chunks: 993 hits: 76 miss: 0 And why ... level 2: chunks: 2231 hits: 2064 miss: 0 level 3: chunks: 52 hits: 0 miss: 0 WAE-612#
100
Cisco Public
DRE Debugging: Fan Out

WAE-7326# show statistics dre config DRE configuration: Mac-id: 00:14:5e:67:13:2a Comp-Type: DRE + LZ Max concurrent connections: 7500, max fan-out: 100 TFO working thread number: 20 DRE cache data version: 7 Total DRE cache diskspace: 17835 MB Total DRE cache memory: 1888 MB, hash table memory: 70 MB PLZ memory: 300 MB, PLZ buffer queue size: 997 PLZ high-watermark=80, low-watermark=50 Total TFO buffer size: 942 MB ACK TFO buffer queue cap: 235 MB, ACK queue diskspace cap: 1024 MB Wait queue cap: 1024 MB Max data-unit number: 24814 Sign-block read pool size: 265, data-segment read pool size: 66 Default anchor distance: 8 Default average chunk size: 256, default min chunk size: 32 LZ algorithm: ZLIB, adaptive LZ: on Adaptive threshold: 95, adaptive LZ use entropy: on Big message checksum: off WAE-7326#
101
Debugging DRE: Fan Out

System Operation When Exceeding Fan Out
%WAAS-RE-3-690433: (678578) RE Cache error: too many used data-unit 47 in the write-pool, current fan-out 137, cap=100
Active peers are counted towards the fan out threshold When the threshold is exceeded, traffic through that peer will continue to benefit from TFO+DRE+LZ, with the exception that new patterns from that peer will not be added to the DRE cache Peer inactivity timers expire after 10 minutes, after which the peer will not be counted towards the fan out threshold
102
Application Acceleration
Cisco Public
103
Verify CIFS Accelerator Service Status

Represents the WAFS Edge service only Registered, Handling Level: 100%
WAE-612# show tfo accelerators Name: TFO State: Keepalive timeout: 3.0 seconds, Session timeouts: 0, Total timeouts: 0 Last keepalive received 00.0 Secs ago Last registration occurred 6:08:19:28.5 Days:Hours:Mins:Secs ago Name: EPM State: Not Registered, Handling Level: 100% Keepalive timeout: 5.0 seconds, Session timeouts: 0, Total timeouts: 0 Last keepalive received 6:07:15:21.5 Days:Hours:Mins:Secs ago Last registration occurred 6:08:19:25.7 Days:Hours:Mins:Secs ago Name: CIFS State: Registered, Handling Level: 100% Keepalive timeout: 4.0 seconds, Session timeouts: 0, Total timeouts: 0 Last keepalive received 00.8 Secs ago Last registration occurred 4:22:25:52.9 Days:Hours:Mins:Secs ago WAE-612#
Cisco Public
104
Verify CIFS Accelerator Connections

WAE-612# show tfo connection summary
Multiple connections are established between the WAFS Edge and WAFS Core
Optimized Connection List Policy summary order: Our's, Peer's, Negotiated, Applied F: Full optimization, D: DRE only, L: LZ Compression, T: TCP Optimization Local-IP:Port 10.88.81.2:51655 10.88.81.2:51656 10.88.81.2:51657 10.88.81.2:51658 10.88.81.2:51659 10.88.81.2:51660 Remote-IP:Port 10.88.80.130:4050 10.88.80.130:4050 10.88.80.130:4050 10.88.80.130:4050 10.88.80.130:4050 10.88.80.130:4050 ConId 1265 1266 1267 1268 1269 1270 PeerId 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 00:11:25:ab:43:32 Policy F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F F,F,F,F
Pass-Through Connections Local-IP:Port Remote-IP:Port 10.88.81.2:43520 10.88.80.130:4050 WAE-612#
Conn Type App Dyn Mtch Optimized
Cisco Public
105
Where Are the CIFS Connections?

If you see CIFS connections between the client and server, they are not being accelerated by WAFS
WAE-612# show tfo connection summary
Optimized Connection List Policy summary order: Our's, Peer's, Negotiated, Applied F: Full optimization, D: DRE only, L: LZ Compression, T: TCP Optimization Local-IP:Port 10.88.81.25:1372 10.88.81.25:5156 Remote-IP:Port 10.88.80.130:445 10.18.20.32:3389 ConId 1265 1266 PeerId Policy 00:11:25:ab:43:32 F,F,F,F 00:11:25:ab:43:32 T,T,T,T
Pass-Through Connections Local-IP:Port Remote-IP:Port 10.88.81.25:1520 10.88.80.53:139 WAE-612#
Conn Type PT In Progress
Cisco Public
106
Troubleshooting File Server AD

If CIFS AD is working, you should see newly discovered file servers being added to the PE dynamic map:
WAE-612# show policy-engine application dynamic Dynamic Match Freelist Information: Allocated: 32768 In Use: 1 Max In Use: 67 Allocations: 1786 Individual Dynamic Match Information: Number: 1 Type: Any->Host (5) User Id: WAFS Edge (1) Src: ANY:ANY Dst: 10.88.80.53:445 Map Name: basic Server Flags: ACNT_NONOPT | ACCEPT Seconds: 0 Remaining: - NA - Hits: 8 WAE-612#
IP address
ACCEPT = Accelerated by WAFS AO DENY = Passed back to TFO for normal AD

107

Viewed cached discovery results and latest log entries
Cached WAE-612# show cifs auto-discovery host-db [3502] (alias=, host=10.88.80.53:139, type=8) discovery results [3502] (alias=, host=10.88.80.53:445, type=8) [3502] (alias=, host=10.88.80.51:445, type=8) WAE-612# WAE-612# show cifs auto-discovery last [Mon May 21 18:26:14 UTC 2007] 10.88.80.51:445 discovered=true, clusterID=3502, freshness=180000 [Mon May 21 18:25:48 UTC 2007] 10.88.80.53:139 discovered=true, clusterID=3502, freshness=180000 [Mon May 21 18:25:48 UTC 2007] 10.88.80.53:445 discovered=true, clusterID=3502, freshness=180000 [Mon May 21 18:22:37 UTC 2007] 10.88.80.53:445 discovered=false, reason="local latency is lower than Core<=>Filer latency, possibly local server", freshness=60 000 [Mon May 21 18:20:47 UTC 2007] 10.88.80.53:445 discovered=false, reason="local latency is lower than Core<=>Filer latency, possibly local server", freshness=60 000 WAE-612# Most recent CIFS AD log message entries
108

CIFS AD reason codes
[Mon May 21 18:22:37 UTC 2007] 10.88.80.53:445 discovered=false, reason="local latency is lower than Core<=>Filer latency, possibly local server", freshness=60 000
server not found = Unable to connect to server or reverse DNS fails high/unknown Core<=>Filer latency = RTT latency between WAFS Core and file server is > 25 msec digital signatures required by server = SMB Signing required by server (ex: Win2k3 DC) no connected clusters = No transport peers connected timed out = No response from WAFS Core to AD query within 4 seconds max session limit is reached = Concurrent number of CIFS sessions has been reached port closed = File server not listening on TCP port 139 or 445 host is not exported = Server is statically defined in CM, but not exported or accessible
Cisco Public
109
Which CIFS Clients Are Accelerated?

WAE-612# show cifs sessions count 473 Current # of CIFS sessions WAE-612# show cifs sessions list SMBSession:[client=10.88.81.25:8392,server=ast6-fs03-10.88.80.53, port=445,closed=false] SMBSession:[client=10.88.81.26:7666,server=ast6-fs02-10.88.80.51, port=445,closed=false] SMBSession:[client=10.88.81.27:13482,server=ast6-fs01-10.88.80.50, port=445,closed=false] SMBSession:[client=10.88.81.28:22387,server=ast6-fs02-10.88.80.51, port=445,closed=false] SMBSession:[client=10.88.81.29:1554,server=ast6-fs03-10.88.80.53, port=445,closed=false] SMBSession:[client=10.88.81.30:10213,server=ast6-fs02-10.88.80.51, port=139,closed=false] < .. > WAE-612#
Lists all CIFS sessions accelerated by WAFS

110
Which files are opened?
WAE511-CCO# show stat wafs expert "-server Rx -mbean CifsFileSystemDB -oper query" -------------------------------------------------------------------------------| Operation | query | | Parameters | None | | Returns | java.lang.String[] | | Value | ast6-fs03.asdcnp-waas.cisco.com\ipc$\lsarpc Optimizations leve | | | l:0, Open count: 1, Ref count: 1, Type: 2, delete on close: | | | NONE, exclusive? false, writers count: -1, readers count: - | | | 1, deleters count: 0, deny-write count: 0, deny-read count: | | | 0, deny-delete count: -1, must update attributes: false | -------------------------------------------------------------------------------WAE511-CCO#
Indicates presence/type of oplock: 0 = None 1 = Exclusive 2 = Batch 3 = Level 2

111
CIFS Request Statistics

Local versus remote requests WAe-612# show statistics cifs requests Statistics gathering period: minutes: 33 seconds: 9 ms: 3 Total: 453 Response time for all cmds Remote: 214
ALL_COMMANDS total:453 remote:214 async:21 avg local:2.164ms avg remote:123.877ms CLOSE_FILE total:31 remote:3 async:14 avg local:1.443ms avg remote:90.772ms CONNECT total:15 remote:3 async:0 avg local:11.055ms avg remote:209.193ms Cancel total:3 remote:3 async:0 avg local:0.0ms avg remote:95.094ms DCERPC total:93 remote:93 async:0 avg local:0.0ms avg remote:95.671ms DCERPC_SRVSVC total:25 remote:20 async:0 avg local:0.743ms avg remote:89.509ms DCERPC_WKSSRV total:15 remote:11 async:0 avg local:1.134ms avg remote:90.786ms ECHO total:2 remote:0 async:0 avg local:1.448ms avg remote:0.0ms FIND_CLOSE2 total:1 remote:0 async:0 avg local:0.595ms avg remote:0.0ms IOCTL total:3 remote:3 async:0 avg local:0.0ms avg remote:94.818ms LOGOFF_ANDX total:3 remote:0 async:3 avg local:1.396ms avg remote:0.0ms NB_SESSION_REQ total:6 remote:0 async:0 avg local:1.455ms avg remote:0.0ms NEGOTIATE total:3 remote:3 async:0 avg local:0.0ms avg remote:99.003ms NT_CREATE_ANDX total:137 remote:29 async:0 avg local:0.549ms avg remote:130.642ms < .. > WAE-612#
Breakdown per CIFS cmd

112
Q and A
Cisco Public
113
Recommended Reading
Continue your Networkers at Cisco Live learning experience with further reading from Cisco Press Check the Recommended Reading flyer for suggested books
Available Onsite at the Cisco Company Store

114
Complete Your Online Session Evaluation

Win fabulous prizes; Give us your feedback Receive ten Passport Points for each session evaluation you complete Go to the Internet stations located throughout the Convention Center to complete your session evaluation Drawings will be held in the World of Solutions
Tuesday, June 20 at 12:15 p.m. Wednesday, June 21 at 12:15 p.m. Thursday, June 22 at 12:15 p.m. and 2:00 p.m.
Cisco Public
115
Cisco Public
116

Brkapp 3006

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Brkapp 3006

Transféré par

Droits d'auteur :

Formats disponibles

Troubleshooting Cisco Wide Area Application Services (WAAS)

2008 Cisco Systems, Inc. All rights reserved.

2008 Cisco Systems, Inc. All rights reserved.

2008 Cisco Systems, Inc. All rights reserved.

Skill Level Assumptions

Familiarity with WAAS configuration and management interfaces, including:

2008 Cisco Systems, Inc. All rights reserved.

2008 Cisco Systems, Inc. All rights reserved.

WAE Platform Overview

2008 Cisco Systems, Inc. All rights reserved.

WAE Hardware Options

WAE-512 Remote Office Appliance

WAE-612 Regional Hub and Data Center Appliance

* Requires WAAS 4.0.13 or later

WAE Hardware Options

WAE-7326 Enterprise Data Center Appliance

WAE-7341 Enterprise Data Center Appliance

WAE-7371 Enterprise Data Center Appliance

WAE Hardware Options

WAE Hardware Options

2008 Cisco Systems, Inc. All rights reserved.

Cisco WAE Family Performance and Scalability

Max Optimized TCP Connections

Max CIFS Sessions

WAN Link Capacity (Mbps)

4 4 8 8 20 45 90 155 310 1000

Software Options Image Naming Convention WAAS-w.w.x.y-K9.z

WAAS-4.0.9.10-K9.binRelease Image WAAS-4.0.9.10-K9.sysimgSystem Image WAAS-4.0.9.10-rescue-cdrom-K9.isoRescue CD-ROM

2008 Cisco Systems, Inc. All rights reserved.

Software Options Upgrade and Recovery Images

System Image (*.sysimg)

2008 Cisco Systems, Inc. All rights reserved.

Checking WAAS Software Version

What was the previous version?

What is the pending version?

2008 Cisco Systems, Inc. All rights reserved.

Verify Flash and Disk-based Code

2008 Cisco Systems, Inc. All rights reserved.

Platform Health and Liveliness

2008 Cisco Systems, Inc. All rights reserved.

A good place to start

2008 Cisco Systems, Inc. All rights reserved.

Sample Diagnostic Report

Sample Diagnostic Report (cont.)

Test PASS Test PASS NONE Test WARN WARN 1.

2008 Cisco Systems, Inc. All rights reserved.

Interface Duplex Issues

2008 Cisco Systems, Inc. All rights reserved.

Sample Diagnostic Report

2008 Cisco Systems, Inc. All rights reserved.

Verify CPU Utilization

Use CLI for real-time CPU utilization monitoring:

Verify Disk Health and Status

STATUS NORMAL NORMAL NORMAL NORMAL NORMAL NORMAL NORMAL

OPERATION OPERATION OPERATION OPERATION OPERATION OPERATION OPERATION

PHYSICAL DEVICES disk00/00[GOOD] disk00/01[GOOD] disk00/02[GOOD] disk00/03[GOOD] disk00/04[GOOD] disk00/05[GOOD] disk00/06[GOOD]

AND STATUS disk01/00[GOOD] disk01/01[GOOD] disk01/02[GOOD] disk01/03[GOOD] disk01/04[GOOD] disk01/05[GOOD] disk01/06[GOOD]

Disk encryption feature is disabled. WAE-612#

Verify Disk Health and Status

Troubleshooting Management Services

WAE is offline or is unable to reach the CM

2008 Cisco Systems, Inc. All rights reserved.