Performance optimization, troubleshooting & dimensioning of a radio access network through the "raw counters KPI

2nd TMA PhD School Napoli June 6th 2011

Roberto Carretto Vodafone Omnitel N.V. Technology Network Engineering RAN Optimization

 a challenging world for a RF engineer

Competition and cost reduction require very high level of effectiveness and efficiency The progressive refinement of the tools for network management is a way to achieve both targets

Crisis

Customer very sensitive

Mature Market

WEB2.0

Digital divide

Total convergence

RAN raw counters: a deposit of information

NodeB
21-mar 22-mar 23-mar 24-mar 25-mar 28-mar 29-mar 30-mar 31-mar 01-apr 04-apr 05-apr 06-apr 07-apr 08-apr 11-apr 12-apr 13-apr 14-apr 15-apr 18-apr 19-apr 20-apr 21-apr 22-apr 26-apr 27-apr 28-apr 29-apr

NodeB

Vodka Data OS S
Dat aba se

RNC

DrawSiti
NodeB

mRAB calls F1 mRAB DCR F1 mRAB calls F2 mRAB DCR F2 174,101 0.54 51,170 0.92 180,301 0.56 54,605 0.89 183,745 0.55 55,218 0.87 188,331 0.51 56,323 0.9 192,585 0.58 58,650 0.89 179,874 0.62 54,629 0.95 188,439 0.74 58,747 1.06 192,917 0.58 61,549 0.97 194,007 0.59 59,995 0.93 197,977 0.58 59,439 0.8 186,883 0.62 56,184 0.91 210,744 0.73 69,523 1.25 189,757 0.67 57,869 1.07 193,859 0.68 61,765 1.04 204,914 0.58 64,804 0.95 197,798 0.59 67,311 0.99 217,676 0.63 83,416 1.58 227,818 0.6 87,703 2.1 225,366 0.65 87,875 2.06 225,895 0.56 86,817 1.7 187,266 0.48 59,052 0.91 199,003 0.47 61,991 0.9 201,159 0.49 63,340 0.9 185,576 0.48 57,720 0.81 138,839 0.43 42,599 0.73 147,160 0.44 45,423 0.83 177,756 0.43 55,720 0.76 183,240 0.49 56,605 0.85 185,505 0.49 57,408 0.93

The RAN raw counters are the perfect feeder for most of the management tools and process in place in Vodafone Italy All the radio procedures can be deeply monitored The raw counters based KPIs allows to summarize the carried traffic and its achieved performance, as well as to highlight any occurred issues The steering of their evolution and their deep validation has become one of the most important task for the Operators, now having the same priority put on the development of the radio features

Radio Network Optimization

The configuration of the deployed radio features can be tuned and progressively optimized The performance of the new services can be monitored and improved The configuration of each cell can be tailored to the actual needs of the offered traffic, potentially leading to the Self-OrganizedNetwork (SON) approach

H P abnorm release rate SD A al 9.00 8.00 7.00 6.00 Abn rel rate% 5.00 4.00 3.00 2.00 1.00 0.00 m ar-23 m ar-26 m ar-31 apr-03 apr-08 apr-13 apr-16 apr-21 apr-24 apr-29 m ag-04 m ag-08 m ag-13 m ag-18 m ag-21 m ag-26 giu-01 giu-04 giu-09 giu-12 giu-17 giu-23 giu-26 lug-01 lug-06 lug-10 lug-15 lug-20 lug-23 lug-28 ago-03 ago-06 ago-11 ago-14 ago-19 ago-25 ago-28 set-02 set-07 set-10 set-16 set-21 set-24 set-30 ott-05 1,800,000 1,600,000 1,400,000 1,200,000 1,000,000 800,000 600,000 400,000 200,000 0 H P SD A_M _D AC _all H P SD A_norm al_rel Abn_R el_HSD A P

N abs

D ata

Radio Network Dimensioning

380,000 Meas urement Eo_Month Max t raff ic @ Target Ef fic ienc y = 65.8% Max t raff ic @ Target Ef fic ienc y = 63.3% Outline Revis ione outlin e + You & Me (21 Apr 08) Baseline 2+10 ( 01- Ago-08) 340,000

300,000 Erlang BH

260,000

Both the radio base stations and the related controllers can be properly dimensioned

220,000

180,000 ott-07 giu-07 lug-07 dic-07 giu-08 lug-08 ott-08 set-07 mar-08 set-08 dic-08 mag-07 mag-08 mar-09 apr-07 ago-07 nov-07 gen-08 apr-08 ago-08 nov-08 gen-09 feb-08 feb-09

4000 3500 3000 2500 2000 1500 1000 500 0

Distribuzione siti fra classi e criticit (week-end febbraio)

The real-time monitoring of the radio KPIs can allow to immediately cope with lack of capacity issues or to unexpected peak of traffic

3111 1177

3065

214 692 78 81 6 5

High Loaded

Mid Loaded

Low Loaded

Radio Network Troubleshooting

The real-time monitoring of the radio KPis and its correlation with the alarms reported by the RAN equipments speed up the maintenance of the radio network, thus limiting the impact of the occurred issues

However, they are not perfect .

User experience
Customer Smith
#Calls = 3 #Customer Dropped = 2 #Customer successful call = 1 Customer Drop Call rate = 66%

Cell KPIs
Cell NA0101
#Calls = 692 #Dropped = 2 #Successful call = 690 Drop Call rate = 0.29%

Very effective from a pure network perspective, the radio rawcounters provide a partial view only of the actual user experience They doesnt allow to have real customer based KPIs or to monitor the actual achieved end-to-end performance Different approach are required to close such a gap (probes, collection of the network-elements internal log, etc.), all of them requiring huge additional investments

RAN raw counters next steps: Automated Network Troubleshooting

Network Monitoring Fault Detection

Among all the potential usage of Radio raw counters, the one which may enable the automated troubleshooting processes is certainly the most innovative, as well as the most interesting for an Operator In the usual troubleshooting process, most of the effort is spent in the Cause Diagnosis phase an automated correlation of raw counters, alarms, skills and experience could lead to a dramatic reduction of both effort and time spent

Solution Deployment

Cause Diagnosis

Fault Detection (based on symptom, determine Which cells have a problem)

10% Effort

Cause Diagnosis (determine What causes the problem)

70% Effort

Solution Deployment (implement solution How to fix the problem)

20% Effort

Automated Network Troubleshooting: the Vodafone-Italy case 1. Detect a wrong behavior, e.g. cells w/high DCR
2. What could have caused the fault?

3. How do we detect the presence of Interference in TRX? 4. KPI, Alarms ... which of them?

Interference in TRX Bad Coverage at cell Border HW Fault Outage of Neighbor Cell Other

Increased TCH Fail Rate IOI Levels increase Excessive TCH alarms seen Increased number of intra-cell HOs Increased number of UL Quality HOs Increased number of UL Interference Hos Others ...

Bayesian Networks? Bayesian Networks are based on Bayes formula for reversing the causal direction of conditional probabilities, allowing one to reason about causes based on information about the effects or symptoms

The Bayesian Networks approach

High DCR
Causes: logical or physical problem to be diagnosed Macro problem which defines the model to be used

Prob(Ck) determined by the planner expertise (faults probability), it represents the a priori knowledge Prob(Sj | Ck) calculated from network data and alarms according to following process:

Ck, Prob(Ck) Interference in TRX

Ck+1, Prob(Ck+1 ) Bad Coverage

Ck+2, Prob(Ck+2 ) HW Fault

1. Identify faulty cells (i.e. with DCR > x%) 2. Discard those cells not relevant for analysis (e.g. due low traffic or know issues) 3. Analyze cells from step 1 and perform fault diagnosis (e.g. cell LA0151 high DCR caused by fault in MHA) 4. Collect symptoms statistics for the cause identified at step 3

Prob(Sj+2 /Ck) Prob(Sj/Ck) Prob(Sj+1 /Ck) Prob(Sj+2 /Ck+2 ) Prob(S /C ) j+ 3 k1 + Prob(Sj+4 /Ck+1 )

Prob(Sj/Ck+2 )

Sj
Incr. TCH fail rate

Sj+1
IOI Levels increase

Sj+2
Excessive TCH Alarms

Sj+3 Incr. Qual. HOs

Sj+4 Avg. DL signal lev < -95dbm

Symptoms (KPI, alarms) observed during daily network monitoring

Automated Network Troubleshooting: many things still to do .

Engineer
Faulty Data Verification & Model Tuning

Knowledge

(Sj, Ck) Knowledge Builder

Faulty Cells & Prob(Ck, Sj))

The attempt has failed so far, due to the issues experienced in exhaustively collecting the actual fault statistics For the same reason, an exhaustive collection of the symptoms statistics has been impossible so far

Model

Decision Engine

Fault Probability

Daily KPIs

. but the principle is certainly good and promising . and, furthermore, absolutely required by a mature operator !

Conclusions
A mature Cellular Operator playing in a mature market needs to automate more and more its operational processes Most of the needed tools are already available, inherently implemented into the network elements currently in place The Operators need to learn how to use them in more innovative, as well as more effective ways The support of all the competence centres, like e.g. Universities, is obviously mandatory

Thank you for your attention ! Any questions?