White Paper

SIP Peering KPIs - How to Measure Answer Seize Ratio [ASR] and other Key Performance Indicators in a VoIP World

Teraquant Corporation 2400 Central Ave, Suite P-2 Boulder CO 80301 Phone: +1 719 488 1003 Fax: +1 303 547 3248 www.teraquant.com

SIP Peering KPIs

Page 2/12

IXCs always measure KPIs such as ASR [Answer Seize Ratio] or Post Dial Delay at SS7 Interconnects with other operators. This allows them to optimize Billable MOUs versus cost of termination.
Service providers have for many decades measured key performance indicators for their SS7 interconnects with long-distance or international operators or peering partners. Such measurements are defined in ITU-T Recommendation E.411 "International Network Management Operational Guidance" and E.422 "Quality of Service for Outgoing International Calls" and include Answer Seize Ratio [ASR], Post Dial Delay [PDD] and Network Efficiency Ratio [NER]. Name Counted ASR SSB CGC SEC CFL RSC UNN ADI CLF LOS rt wt wna ct ht Minutes Description Number of calls Answered calls (percent) Subscriber busy (percent) Circuit Group Congestion (percent) Switching Equipment Congestion (percent) Call Failure (percent) Reset Circuit Signal (percent) Unallocated Number (percent) Address Incomplete (percent) Clear Forward (percent) Line Out of Service (percent) Response time (average) Wait time with answer (average) Wait time with no answer (average) Call time (average) Hold time (average) Total Call time in minutes Table 1

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 3/12

Answer Seize Ratio [ASR] is the common measure for call completion but it discounts calls that are not answered by the Callee. This is not necessarily a network fault but it does depress the success rate.
Answer Seize Ratio [ASR] is used as a measure of network quality although the measurement also includes user behavior. In other words, if the call was not answered, the network could not be faulted, although the ASR measurement would be reduced by the uncompleted call, indicating lower quality. However, because for a given hour within a day, unanswered calls would always represent the same percentage, from day-to-day, this offset would be normalized out and carriers are able to monitor the trend of ASR and treat it as a relative measurement

Network Efficiency Ratio is a measurement that more accurately shows calls succeeding and calls failing due to network errors.
Network Efficiency Ratio was designed to eliminate the impact of user behavior on the measurement and better represent pure network performance.
Network Efficiency Ratio [NER] is defined as: User Answers or Normal call clearing + User Busy + Ring No Answer + Terminal Rejects) Cause code: 16 Cause code: 17 Cause code: 18 & 19 Cause code: 21

NER = ----------------------------------------------------------------------------------(Total # of Call Attempts i.e. IAMs)

x 100

SIP is a more flexible protocol and has wider uses than simple call control. Therefore, use cases differ widely and systems such as voicemail and call forwarding skew expected behavior for answered calls Furthermore, SIP is a more flexible protocol and has wider range of response messages which can be used to specifically indicate certain types of failures from either servers, network devices or the movement or absence or other behavior of users. Accordingly, IETF has defined KPI's equivalent to Answer Seize Ratio and Network Efficiency Ratio. These are described under SIP End-to-End Performance Metrics draft ietf-pmol-sip-perf-metrics-04. For example, the equivalent of ASR is Session Establishment Ratio (SER)
Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 4/12

Session Establishment Ratio (SER) is defined as follows, to quote the IETF

The SER is calculated using the following formula: # of INVITE Requests w/ associated 200 OK

SER = -------------------------------------------- x 100

(Total # of INVITE Requests)

(# of INVITE Requests w/ 3XX Response)

Here is the message flow which defines the SER.

UA1 UA2 | | |INVITE | +-------------------->|------------------>| | | 180| | |<------------------| Session Established | | | | | | | 200| +-------------------->|<------------------| | |

A more realistic representation of a SIP call setup is shown below using IPTEGO's Palladion message flow presentation

IPTEGO Palladions Message Flow

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 5/12

In SS7, the fate of the call is determined by the RELEASE message which ends the call. The fate of a SIP call is determined by the Response to each Request for each transaction within the SIP call. For example as above, session establishment is determined by a successful outcome from the call session establishment phase

Similar to NER in SS7 ISUP, Session Establishment Effectiveness Ratio (SEER) is SIP INVITES which successfully setup a voice call plus busy scenarios or VoIP Phones has been moved i.e. the network has done its job
Similar to NER in SS7 ISUP, SIP End-to-End Performance Metrics also defines Session Establishment Effectiveness Ratio (SEER) This metric is complimentary to SER, but is intended to exclude the potential effects of the terminating UAS from the metric and therefore more closely reflect the performance of the network. SEER is defined as the number of INVITE requests resulting in a 200 OK response and INVITE requests resulting in a 480, 486, or 600; to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX, 401, 402, and 407 response. In order to simplify the formula, the following variable a is used to summarize multiple SIP responses a = 3XX, 401, 402, and 407 The SEER is calculated using the following formula: # of INVITE Requests w/ associated 200 OK, 480, 486, or 600

SEER = ------------------------------------------------------------------- x 100

(Total # of INVITE Requests) (# of INVITE Requests w/ 'a' Response) All these KPIs for both SS7 ISUP and for SIP are easily selected for graphing with in IPTEGO's advanced service assurance monitoring platform, Palladion. As shown below, such KPI's can be selected from a GUI, graphed and alerts can be raised [SNMP, PCAP/Wireshark Trace, or e-mail sent to or communication to

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 6/12

other network devices over API] in response to KPI thresholds being exceeded. In addition, these Palladion KPI metrics can be selected for network elements, realms or trunks specified by you in your network. So this makes it very easy for you to monitor the performance provided to you SIP Response Codes by each peering partner, SIP trunk or route/LCR or the o 2xxSuccessful Responses quality you are delivering to o 200 OK each of your realms or o 4xxClient Failure Responses enterprise customers. o 401 Unauthorized (Used only by registrars or user agents. Proxies should use proxy authorization 407) o 402 Payment Required (Reserved for future use) o 480 Temporarily Unavailable o 486 Busy Here o 6xxGlobal Failure Responses o 600 Busy Everywhere

IPTEGO Palladion supports hundreds of KPIs measured for each network device or Trunk

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 7/12

Another very useful measurement defined by the IETF here is Session Defects Ratio (SDR), also graphed by Palladion by checking a box. 503 SIP Response messages commonly indicate a route to one of your peering partners is failing due to congestion of the Gateway or SoftSwitch. Session Defects Ratio (SDR) is the percentage of call attempts receiving the following responses, in relation to total call attempts or INVITES:
o o o

500 Server Internal Error 503 Service Unavailable 504 Server Timeout

The SDR is calculated using the following formula: # of INVITE Requests w/ associated 500, 503, or 504

SDR =-------------------------------------------------------------- x 100

Total # of INVITE Requests

Palladion can be set up to monitor, measure, graph and alert on any specific SIP response code. The diagram below shows 404s and 503's

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 8/12

IPTEGO Palladion set to measure 503s for each network device or Trunk In addition to this, some carriers like to plot Post Dial Delay, [PDD] and to alert when it exceeds a certain time threshold. PDD is defined as the time interval between transmission of the INVITE and reception of the 180 Ringing response message. This is an earlier indication of congestion soon to occur on a given route and at this time, Palladion can be configured to provide an SNMP trap all other notification to the Network Operations Center [NOC].

A trend of increasing Post Dial Delay, [PDD] is used to indicate the on-site or imminence of congestion at a peering point. Early alerts can allow rerouting of LCRs to avoid dropped calls

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 9/12

The IETF defines many more KPIs. the table given below maps these to the equivalent SS7 or circuit switched performance measurement, wherever a comparison makes sense or is valid.
SIP End-to-End Performance Metrics draft-ietf-pmolsip-perf-metrics04 Successful Request Attempt [SRD] Session Request Delay [SRD] Failed Session Setup [SRD] and SIP Definition ISUP Equivalent ISUP Definition

INVITE INVITE Trying eg 180

to 100 to non-100

Response Time Post-Dial Delay (PDD) N/A

IAM to ACM IAM to ANM IAM to REL

Successful session completion [SDD] Failed session completion SDD Session Duration Time [SDT] Successful Session Duration Time [SSDT] Failed session duration SDT Session Establishment [SER ] Ratio Session Establishment Effectiveness Ratio (SEER) Session Defects Ratio (SDR) Ineffective Session

INVITE to 4XX (excluding 401, 402, and 407 nonfailure challenge response codes), 5XX, or possible 6XX message. BYE, to 2XX Ack BYE, to Timer F Expires

Time to Clear Call or CIC Release with no release complete

REL to RLC Missing RLC

Average Call Hold Time (ACHT) 200 OK response to an INVITE to BYE

200 OK response to an INVITE, and the resulting Timer F expiration.

x 100/----------------------(Total # of INVITE Requests)

(# of INVITE Requests w/ associated 200 OK)

Average Call Hold Time (ACHT) ASR

(# of INVITE Requests w/ 3XX Response)

x 100------------------------------

(# of INVITE Requests w/ associated 200 OK, 480, 486, or 600) (Total # of INVITE Requests)

N/A

(# of INVITE Requests w/ 'a' Response) (# of INVITE Requests w/ associated 500, 503, or 504) Total # of INVITE Requests

N/A
Ineffective

x100---------------------------

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 10/12 Machine Attempts (IMA) in telephony applications of SIP, and was adopted from Telcordia GR512-CORE [GR-512].

Attempts (ISA) Invites resulting in:o 408 Request Timeout o 500 Server Internal Error o 503 Service Unavailable o 504 Server Timeout Session Disconnect Failures [SDF]
Session disconnect failures occur when an active session is terminated due to a failure condition that can be identified by a REASON header [RFC3326] in a BYE or CANCEL request.

# of ISA x 100 ----------------------------Total # of Session Requests

Cutoff Calls # of SDF's x 100 ----------------------------Total # of Session Requests

Telcordia GR512-CORE [GR-512].

Session Completion Ratio (SCR)

# of Successfully Completed Sessions x 100 -----------------------------------------Total # of Session Requests A session completes successfully when it begins with a setup request and ends with a session completion message ie SIP 200
Session completion fails when an INVITE is sent from a UAC, but there is no indication the INVITE reached the intended UAS. This can also occur if the intended UAS does not respond to the UAC or the response never reaches the UAC associated with the session. Eg only responses are from proxies eg 407 or 408

Call Completion Ratio (CCR)

Successful Session Completion

Failed Session Completion Session Success Ratio [SSR]

SSR = 100% - (ISA% + SDF%)

Call Success Ratio (CSR)

References

SIP End-to-End Performance Metrics metrics-04

draft-ietf-pmol-sip-perf-

ITU-T Recommendation E.411 : International network management Operational guidance ITU-T Recommendation E.422 : Observations on international outgoing telephone calls for quality of service Telcordia GR-512-CORE - LSSGR: Reliability, Section 12

Teraquant Corporation All rights reserved www.teraquant.com

SIP Peering KPIs

Page 11/12

For more information on SIP KPI's, visit our website at www.teraquant.com For more information on Palladion, please click the "mail to" or email admin@teraquant.com Or call +1 719 488 1003 www.teraquant.com

Teraquant Corporation All rights reserved www.teraquant.com