VoLTE E2E Analysis and Troubleshooting

Tools

July 2018

The information contained in this document is proprietary and confidential to Sprint and is
intended solely for internal use by Sprint employees and approved vendors. Any unauthorized
distribution of this information without the written consent of Sprint is strictly prohibited.

SPRINT PROPRITERAY & CONFIDENTIAL 1

 VOLTE E2E Analysis & Troubleshooting Tools

TABLE OF CONTENTS
1 GENERAL 3
1.1 PURPOSE 3
1.2 SCOPE 3
1.3 RESPONSIBILITY 3
1.4 REVISION HISTORY 3

2 NG1 4

3 TRUECALL 7

4 ISA 9

5 NBA 11

6 PRTS 14

7 WINDCATCHER 18

8 LIST OF VOLTE KPIS 20

9 GLOSSARY 22

10 APPENDIX 22

2
VoLTE E2E Analysis & Troubleshooting Tools

1 GENERAL

1.1 PURPOSE
This document contain details on the various tools and reporting that are available
to support VoLTE optimization and performance activities.

1.2 SCOPE
Scope of this document is related to Netscout suite of tools and PRTS. This document
is not intended for training on the tools but rather as a means to educate the regions
on the tool and reporting enhancements that are available to support VoLTE.

Recordings of the NETSCOUT Knowledge Transfer (KT) sessions are available at the
link below (ref. “NetScout VoLTE KT Sessions June 2018” folder): https://doc-
share.corp.sprint.com/livelink/llisapi.dll?func=ll&objId=89648775&objAction=browse
&viewType=1

1.3 RESPONSIBILITY
The TP&SD Performance Standard team is responsible for updating, and monitoring
the application of the guidelines, specifications, practices, procedures, and plans
described in this document. Please contact the following, if there are questions
regarding these planning guidelines.

1.4 REVISION HISTORY

Date Version Author Contributors Comments

07/27/2018 1.0 Amit Roya Johnson, Hugh Park, Scott First Release
Mahajan, Rinderle, Rahul
Krishna Meenakshisundaram, Temo
Sitaram, Reyes, Michael Silvey, John
Nee-Nee Davidian, Sasan Saifnoorian, Chris
Yang Shrader, Geoff Lubeskie, Kathy
Rubeo, Ravindra Wasalwar

SPRINT PROPRITERAY & CONFIDENTIAL 3

VoLTE E2E Analysis & Troubleshooting Tools

2 NG1
NetScout nG1 tool provides information from probes on different interfaces as well as
signaling and media KPIs. It also provides end to end view of calls and analytics that
can be used for further troubleshooting. nG1 is a DPI-based platform that supports
for both top-down (network level) and bottom-up (individual call level) analysis of
the VoLTE network. Using the top-down approach, engineers can proactively look at
the trending on the performance of the network and where issues are seen, drill down
into those issues to the individual call level. There are a number of different
applications in nG1 that allow for rapid issue identification.

Service Dashboard

The Service Dashboard can be used to gain visibility into critical service issues. It
can be viewed on a vEPC/vIMS basis and can be monitored on different regional
and local domains. Each local domain (e.g., AKRNOHIJ vEPC) consists of two
categories, Call Model and Device View. Call Model and Device View has
multiple dashboards corresponding to a service and an interface.

For example:
AKRNOHIJ vEPC (Domain)Call Model View  Advanced Voice Statistics
(Service) S1-U Interface

SPRINT PROPRITERAY & CONFIDENTIAL 4

VoLTE E2E Analysis & Troubleshooting Tools

Service Dashboard

Clicking on the drilldown to Service Monitor icon in the service dashboard can lead
you to different service monitor panels that are used for analysis and
troubleshooting of different call characteristics.

Advanced Voice Statistics (Signaling- Call Setups and Failures)

The Advanced Voice Statistics Monitor is primarily used to look at accessibility and
retainability metrics. KPIs are rolled up for the server or community (user) in question
and each KPI can be looked at via their individual error codes.

Advanced Voice Statistics Monitor Looking at Call Failures

SPRINT PROPRITERAY & CONFIDENTIAL 5

VoLTE E2E Analysis & Troubleshooting Tools

Universal Monitor (Registrations and Mobility)

The Universal Monitor is primarily used to troubleshoot registration issues, mobility
issues (S1-U and S11), and also can be used to look at accessibility and retainability
metrics.

Universal Monitor View Looking at SIP Registrations

Media Service Monitor (Media- MOS, Jitter, Gaps, One-Way Audio)

The Media Service Monitor is primarily used to look at the quality of the call, from a
media perspective. Typical KPIs are MOS, Packer Loss Rate, Jitter, Gaps, QoS
Mismatch, and others.

Media Service Monitor Looking at Media Quality

SPRINT PROPRITERAY & CONFIDENTIAL 6

VoLTE E2E Analysis & Troubleshooting Tools

Single Call View

Packet Analysis View

Using the monitors gives the engineering team insight into the scale and severity of
the issue via KPIs, and also allows the user to drill down on an hourly basis into
individual call flows. Identifying the calls that contributed to the KPI degradations
can be leveraged for use in another NetScout platform: Iris Session Analyzer (ISA).

3 TRUECALL
NetScout also provides another platform for service monitoring, but focuses on the RAN
RRC and S1-AP, and NAS messaging (MME, eNB). TrueCall shows the geolocation of
events (ie, call failures) and helps in identifying and substantiating the findings from the

SPRINT PROPRITERAY & CONFIDENTIAL 7

VoLTE E2E Analysis & Troubleshooting Tools

drive test data. TrueCall can also provide a number of KPIs to assess service quality for
VoLTE.

Call Failure Analysis Using TrueCall and Geolocation

Heat Maps

SPRINT PROPRITERAY & CONFIDENTIAL 8

VoLTE E2E Analysis & Troubleshooting Tools

VoLTE Analysis

4 ISA
NetScout Iris Session Analyzer (ISA) is a platform that pulls data from the same
probes as those used by NetScout nG1, but is used for detailed call analysis to
locate where the origination of failures or issues is seen. It provides an end to end
view of the failures and events. The tool provides fully correlated end to end ladder
diagrams and color coded failure analysis, which can assist in easy inspection of
complex session flows to quickly identify trouble spots.

SPRINT PROPRITERAY & CONFIDENTIAL 9

VoLTE E2E Analysis & Troubleshooting Tools

Using ISA a user can filter on specific MDNs or IMSIs to look at ladder diagrams and
their associated messaging to find root cause of issues down to the network
element level.

Selecting an MDN for Session Analysis

To analyze a particular call, the user selects the call from the upper window,
whereby the ladder diagram and flow of the media and signaling for the call will
be displayed in the window immediately below. A user can see the summary status

SPRINT PROPRITERAY & CONFIDENTIAL 10

VoLTE E2E Analysis & Troubleshooting Tools

for each message in the bottom window, and also export PCAP captures to look at
the call in other tools, such as Wireshark.

Call Failure Analysis for a Particular MDN

5 NBA
nBA is an analytics platform that delivers subscriber analytics and allows for executive
reporting and dashboards. Through nBA Explorer, users can also build custom KPIs that
can be visualized on custom dashboards.

The VoLTE KPIs in nBA are currently under validation by Netscout, so there could be some
data inaccuracies. The link to access nBA is https://10.24.202.253:8443/explorer/ .

Three main views are available for monitoring and analyzing VoLTE performance.
• VoLTE Executive Dashboard provides an overview of the major VoLTE KPIs

• VoLTE Market Reports contains a more comprehensive list of VoLTE KPIs

• VoLTE Market Trends provide graphical views of the VoLTE KPIs

VoLTE Executive Dashboard – National and 99 market level

SPRINT PROPRITERAY & CONFIDENTIAL 11

VoLTE E2E Analysis & Troubleshooting Tools

Some of the major KPIs that are reported include the following:
• Registration Success Rate

• Session Establishment Ratio (SER) - Percentage of successfully established sessions

 SER = # of INVITE Requests w/associated 200 OK

(Total # of INVITE Requests) - (# of INVITE Requests w/ 3XX Response)

• Session Establishment Effectiveness Ratio (SEER) - Percentage of successfully

established sessions less User-related failures e.g. busy, no answer

 SEER = # of INVITE Requests w/associated 200 OK, 480, 486, 487, 600, or 603

(Total # of INVITE Requests) - (# of INVITE Requests w/ 3XX Response)

• Drop Rate

• MOS

• Packet Loss %

• Registration Latency

• Post Dial Delay

• Call Setup Time

VoLTE Executive Dashboard View

SPRINT PROPRITERAY & CONFIDENTIAL 12

VoLTE E2E Analysis & Troubleshooting Tools

VoLTE Market Reports – Market, Cascade, ECI level

KPIs specific to Accessibility, Sustainability, Quality, Latency KPIs are provided in the VoLTE
Market Reports.

VoLTE Market Report View

VoLTE Market Trends – Market, Cascade, ECI level

Charts for Accessibility, Sustainability, Quality, Latency KPIs are provided in the VoLTE
Market Trends.

VoLTE Market Trend View

SPRINT PROPRITERAY & CONFIDENTIAL 13

VoLTE E2E Analysis & Troubleshooting Tools

6 PRTS
Below are some VoLTE KPI trends that are available in PRTS. The availability of the VoLTE KPIs
are dependent on each RAN OEM. To view the VoLTE KPIs, select “VoLTE” under the KPI
Group pull down menu or select “Others” under the KPI Group pull down menu to manually
select specific VoLTE KPIs.

SPRINT PROPRITERAY & CONFIDENTIAL 14

VoLTE E2E Analysis & Troubleshooting Tools

SPRINT PROPRITERAY & CONFIDENTIAL 15

VoLTE E2E Analysis & Troubleshooting Tools

SPRINT PROPRITERAY & CONFIDENTIAL 16

VoLTE E2E Analysis & Troubleshooting Tools

SPRINT PROPRITERAY & CONFIDENTIAL 17

VoLTE E2E Analysis & Troubleshooting Tools

7 WINDCATCHER
Post processing tool can be used to process the volte drive logs and look at the RF
conditions, layer 3 as well as SIP & RTP messaging between the device and the rest of the
network. While processing VoLTE log files, the engineer has to make sure of the
appropriate VoLTE selections.

Once the setting selections have been done, the user can process the data and look at
views from the device side including SIP, RTP, RRC messaging along with the RF conditions.

SPRINT PROPRITERAY & CONFIDENTIAL 18

VoLTE E2E Analysis & Troubleshooting Tools

SPRINT PROPRITERAY & CONFIDENTIAL 19

VoLTE E2E Analysis & Troubleshooting Tools

8 VOLTE KPIS & TOOLS MAPPING

Sprint VoLTE KPI List Category Availability Tool
Data Connection Failure Rate Accessibility PRTS
VoLTE Connection Failure Rate Accessibility PRTS
Initial ERAB QCI[i] Setup Failure Rate Accessibility PRTS
Initial ERAB QCI[i] Setup Request Accessibility PRTS
Initial ERAB QCI[i] Setup Success Accessibility PRTS
Additional E-RAB QCi Failure rate Accessibility PRTS
Session Establishment Ratio (SER) Accessibility Netscout nG1/nBA
Session Establishment Effectiveness Ratio (SEER) Accessibility Netscout nG1/nBA
Setup Failure Accessibility Netscout nG1/nBA
LTE RRC Connection Setup Success Rate Accessibility PRTS/Netscout TrueCall
QCI[i] ERAB Setup Failure Rate Accessibility PRTS/Netscout TrueCall
VoLTE Call Setup Success Rate (SIP Invite to 200 OK) Accessibility Netscout nG1/nBA
VoLTE Registration Failure Rate Accessibility Netscout nG1/nBA
VoLTE (SIP) Re-registration Success Rate (RRSR) Accessibility Netscout nG1/nBA
UE In Session Activity Time(MOUs) Capacity PRTS
QCI[i] In Session Activity Time MOUs Capacity PRTS
DL Average QCI[i]Active User Number Capacity PRTS
DL Average RF Utilization Capacity PRTS
DL Average QCI[i] Payload Utilization Capacity PRTS
UL Average RF Utilization Capacity PRTS
UL Average QCI[i] Payload Utilization Capacity PRTS
DL Cell PDCP SDU QCI[i] Average Bitrate Capacity PRTS
UL Cell PDCP SDU QCI[i] Average Bitrate Capacity PRTS
DL Cell PDCP SDU Max Bitrate Capacity PRTS
UL Cell PDCP SDU Max Bitrate Capacity PRTS
DL Cell Control Plane PDCP SDU Average Bitrate Capacity PRTS
UL Cell Control Plane PDCP SDU Average Bitrate Capacity PRTS
ERAB QCI[i] Average Connections Capacity PRTS
ERAB QCI[i] Max Connections Capacity PRTS
ERAB QCI[i] GBR Satisfied Rate Capacity PRTS
UL Average QCI1 Active TTI Bundling User Number Capacity PRTS
VoLTE Media UL Average TTI Bundling PRB Ratio Capacity PRTS
VoLTE Media UL Average TTI Bundling PRBs Capacity PRTS
VoLTE Media UL TTI Bundling MAC Packet Loss Rate Capacity PRTS
VoLTE Media UL Initial TTI Bundling HARQ Success Rate Capacity PRTS
DL IP QCI[i] Latency Latency/Delay PRTS
DL PDCP SDU QCI[i] Latency Latency/Delay PRTS
ERAB QCI[i] Average Setup Latency Latency/Delay Netscout TrueCall
ERAB QCI[i] Max Setup Latency Latency/Delay Netscout TrueCall
Average PDD (Post Dial Delay) Latency/Delay Netscout nG1/nBA
Average Disconnect Time Latency/Delay Netscout nG1/nBA
SIP Setup Time 200 OK [SIP Invite to 200 OK] Latency/Delay Netscout nG1/nBA
SIP Registration Latency Latency/Delay Netscout nG1/nBA
SIP Handshake Latency (PRACK to PRACK 200 OK) Latency/Delay Netscout nG1/nBA
Round Trip Delay Latency/Delay Netscout nG1/nBA
ERAB QCI[i] Average Setup Latency Latency/Delay Netscout TrueCall
ERAB QCI[i] Max Setup Latency Latency/Delay Netscout TrueCall

SPRINT PROPRITERAY & CONFIDENTIAL 20

VoLTE E2E Analysis & Troubleshooting Tools

Sprint VoLTE KPI List Category Availability Tool

Intra ENB HO QCI[i] Success Rate (QCI1 is also known as VoLTE Media Mobility PRTS
Bearer Intra-ENB HO Success Rate in PRTS)
Intra ENB HO QCI[i] Preparation Success Rate (QCI1 is also known as Mobility PRTS/Netscout TrueCall
VoLTE Media Bearer Intra-ENB HO Preparation Success Rate in PRTS)
Intra ENB HO QCI[i] Execution Success Rate (QCI1 is also known as VoLTE Mobility PRTS/Netscout TrueCall
Media Bearer Intra-ENB HO Execution Success Rate in PRTS)
Inter ENB HO QCI[i] Success Rate (QCI1 is also known as VoLTE Media Mobility PRTS
Bearer Inter-ENB HO Success Rate in PRTS)
Inter ENB HO QCI[i] Preparation Success Rate (QCI1 is also known as Mobility PRTS/Netscout TrueCall
VoLTE Media Bearer Inter-ENB HO Preparation Success Rate in PRTS)
Inter ENB HO QCI[i] Execution Success Rate (QCI1 is also known as VoLTE Mobility PRTS/Netscout TrueCall
Media Bearer Inter-ENB HO Execution Success Rate in PRTS)
VoLTE Media Bearer MRO HO Failure Rate Mobility PRTS
VoLTE Media Bearer MRO Too Early HO Failure Rate Mobility PRTS
VoLTE Media Bearer MRO Too Late HO Failure Rate Mobility PRTS
VoLTE Media Bearer MRO To Wrong Cell HO Failure Rate Mobility PRTS
Intra−ENB QCI[i] HO Latency(QCI1 is also known as Mobility PRTS
VoLTE_Media_Bearer_Intra_eNB_Average_HO_Latency
in PRTS)
Inter−ENB QCI[i] HO Latency(QCI1 is also known as Mobility PRTS
VoLTE_Media_Bearer_Inter_eNB_Average_HO_Latency
in PRTS)
UL Average QCI[i]Active User Number Quality PRTS
DL QCI[i] Packet Discard Rate Quality PRTS
DL QCI[i] Packet Loss Rate Quality PRTS
UL QCI[i] Packet Loss Rate Quality PRTS
VoLTE Media Average CQI Quality PRTS
VoLTE Media CQI[i]Reported Count Quality PRTS
DL MOS (Mean Opinion Score) Quality Netscout nG1/nBA
UL MOS (Mean Opinion Score) Quality Netscout nG1/nBA
DL Jitter Quality Netscout nBA
UL Jitter Quality Netscout nBA
Packets Out Of Sequence (OOS) Quality Netscout nG1
Audio Gaps Rate Quality Netscout nG1/nBA
% Single Stream (i.e.One Way Audio) Quality Netscout nG1/nBA
RTP Packet Loss Rate Quality Netscout nG1/nBA
QoS Mismatch Rate Quality Netscout nG1/nBA
Voice Compression MOS degradation Rate [%] Quality Netscout nG1
UL MOS Degradation % Quality Netscout nG1/nBA
DL MOS Degradtion % Quality Netscout nG1/nBA
Jitter Degradation % Quality Netscout nG1/nBA
Compression Degradation Quality Netscout nG1/nBA
Connection QCI[i] Drop Rate Sustainability PRTS
Network Connection QCI[i] Drop Rate Sustainability PRTS
ERAB QCI[i] RRC Connection Reestablishment Failure Rate Sustainability Netscout TrueCall
VoLTE Call Drop Rate Sustainability Netscout nG1/nBA
LTE RRC Connection Drop Rate Sustainability Netscout TrueCall
QCI[i] ERAB Drop Rate Sustainability PRTS/Netscout TrueCall
VoLTE Call completion Success Rate (SIP Invite 200 OK to Bye 200 OK) Sustainability Netscout nG1/nBA

SPRINT PROPRITERAY & CONFIDENTIAL 21

VoLTE E2E Analysis & Troubleshooting Tools

9 GLOSSARY
AMR- WB – Adaptive Multi Rate- Wide Band Codecs

BLER – Block Error Rate

EVS – Enhanced voice services codec

MOS – Mean Opinion Score

QCI - Quality of Service Indicator

QoS – Quality of Service

TM – Traffic Management

VoLTE - Voice over LTE

TBS=Transport Block size

TTI=Transmit time interval

10 APPENDIX
 None

SPRINT PROPRITERAY & CONFIDENTIAL 22