Académique Documents
Professionnel Documents
Culture Documents
LTE and EPC Test An Overview of Test Concepts and Tools for Trials
Table of Contents
Specific LTE Test Areas 3 Overview3 Gainingdeeperinsights4 Performancetesting5 Single-userthroughputtesting5 Cellperformanceverificationwithmulti-userthroughputtesting 6 Realisticmulti-userthroughputtesting8 Idle-to-activetransitiontimes9 Latencytesting10 KPIverificationandcalculation10 ValidatingLTEvoice12 Signalingvalidation 13 VoiceQoSandQoE(MOS) 13 TestingQoSandQoEofLTEstreamingvideo14 EvaluatingLTEMIMOandfrequency-selectivescheduling 15 Testingnetworkcoverage16 TestingLTEhandover16 ValidatingLTEbackhaul18 VerifyingLTEhandsetIOT18 Validatingdeviceconfiguration18
WEBSITE: www.jdsu.com/test
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
Appendix: Outline for a Basic Phase 1 LTE Test Plan 20 Overview20 Achievabledataratesandlatency:single-userthroughputforUL/DLandTCP/UDP20 Achievabledataratesandlatency:cellthroughputandMUthroughputforUL/DLandTCP/UDP21 Achievabledataratesandlatency:latency21 Intra-LTEmobility:mobilityandhandoverperformance 21 Achievabledataratesandlatency:applicationperformance 22 Coverageandcapacityradiofeaturesefficiencyandgainassessment:linkbudget 22 Coverageandcapacityradiofeaturesefficiencyandgainassessment:scheduler22 Evaluationofantennaconfigurationoptions23 Self-configurationandself-organizingnetworkfeatures 23 Evaluationoffrequencyreuse:onedeploymentscenario23 BasicQoS:userdifferentiationbetweennon-GBRuserswithdifferentQCI 24 BasicQoS:user-differentiationbetweenGBRandnon-GBRusers24 Basicapplicationperformance:webbrowsing,streaming,voicecalls,e-mail,VPN,on-linegaming25 References 26 EPSspecificationreferences26 3GPPreferences26 NGMNreference28 ETSIreference28 Glossary 29
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
Lab trials
Commercial launch
TheexampleslistedbelowaddressdifferentaspectsoftestingatvariousstagesintherolloutofLTE/ SAETheseareofferedasanoverviewofthemajorelementsoftheLTEdeploymentlifecycle Technologyfieldtrials EvaluatethetechnologyagainstNGMNorotherindustryrequirements Testandvalidatetechnologyimplementations Adoptaprocessthatismoreopenthanfullyclosedbilateralfieldtrials Labtesting EvaluateeNodeBschedulingperformance Evaluatesecurityandbillingpolicies Evaluateserviceperformanceincontrolledenvironments EvaluateUEperformance EvaluateMIMOperformancegains Fieldtrialsandvendorevaluation Evaluateend-to-end(E2E)performance Evaluatenetworkcoverage Evaluatecellandnodeperformanceunderloadedconditions Evaluateself-optimizingnetworkcapabilities EvaluateIRATcapabilitiesandperformance PerformKPImonitoringandbenchmarking
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
Friendlycustomertrials Understandend-userQoE/QoS Confirmhistorictroubleshootingcapabilities TesthandsetIOT,conformanceandpre-conformance Monitorhandsetperformance Includetrending,statisticsandVIPreporting Commerciallaunch Performservice-andaccesstechnology-awaremonitoring VerifyE2Evisibilityandtroubleshootingcapabilities Validateandunderstandsubscriberbehavior UnderstandoverallnetworkandIRAThandover(HO)performance Includeintegrationwithnode-loggingcapabilities Optimizationandwiderdeploymentphase ValidateandensureSONcapabilitiesareworking Locateareasforexpansion Monitorbackhaulperformanceimpactonend-userQoE Benchmarkserviceperformancebetweenmacroandfemtoroll-out TheremainderofthisdocumentfocusesonthefieldtrialsportionoftheprocessEachsection describesthetestcasesmostoperatorswillwanttoundertakeandalsooutlinesthetoolsthatwillensure effectiveperformanceofsuchtestsItisassumedthatnotalloperatorswillperformallstepssowillneed tomatchthetestcases(andpossiblesolutions)touniqueneeds Gaining deeper insights Foryears,personnelfromacrossAgilentTechnologieshavebeen involvedwithtechnologystandardizationandtestdevelopmentfor LTEandSAESince2008,Agilenthasactivelyprovidedtoolstohelp developerstakeLTE/SAEforwardtothemarketplaceInApril2009,the companypublishedacomprehensivebookcalledLTEandtheEvolution to4GWireless:DesignandMeasurementChallenges(ISBN:978-0-47068261-6)Thisresourcehasreceivedsignificantandwidespreadpositive feedbackfromthewirelessindustryInMay2010,JDSUacquiredthe NetworkSolutionsDivisionfromAgilentWiththisacquisition,several contributingauthorstothisLTEbookincludingtheauthorofthis applicationnotetransitionedfromAgilenttoJDSUThebookis justoneexampleofhowAgilentandnowJDSUcontributetothe overalllandscapeoftheLTEandSAEindustryAsacompaniontothebook,thisapplicationnote containsrelevantreferencestotheLTEbook,whichcanbestudiedseparatelyinpursuitofadeeper understandingofthevarioustopicsandconceptsinvolvedinLTEtesting
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
Performance testing WithLTE,muchofthefocushasbeenonincreasingsystemperformanceinareassuchasend-user throughput,latency,andidle-to-activetransitionsAlthoughtheseimprovementsalongwith significantlyimpressivesingle-userdataratesareimportantformarketingreasons,inthemselves theyplayalimitedroleinhowrealuserswillutilizeadeployedLTEnetworkThus,measurementsare neededforbothsingle-userpeakratesandmulti-userthroughputInanalyzingmulti-userthroughput andoverallcellcapacity,itisimportanttounderstandtheimpactofacellsrealdistributionofusersas wellasthemobilityandusagepatternsofthoseusers Single-user throughput testing Intheory,single-userthroughputtestingisquitesimpleHowever,inrealityitcanprovetobequitetricky Asanexample,thefirstaspectstounderstandarewhat to measure and how to benchmark the result LookingatthepublicresultsoftheLTESAETrialInitiative(LSTI)ProofofConcept(PoC)group,it canbeseenthatpeakrateswillvaryfromafewhundredkilobitspersecondatthecelledgetoover 150Mbpsinverygoodradioconditions(forexample,ina20MHz2x2MIMOsystem)Inpractice, measuringthiswiderangeofratescouldbeperformedwithadedicatedhardwaretrafficgenerator thatwillhaveguaranteedperformanceandwillproducetrafficpreciselyaccordingtoananticipated trafficprofileSuchaprofilecouldincludevaryingdatarates,differenttypesoftraffic,anddelay characteristicssuchasjitterOtheralternativesincludesoftware-basedtrafficgenerators(for example,iperf)orsimplygeneratingthetrafficfromexistingstandardapplicationserversusedfor FTPorvideostreaming Ifasoftware-basedtrafficgeneratorisused,itisimportanttounderstandtheconditionsunderwhich itwilldeliverreliableresultsIngeneral,currentlyavailablesoftware-basedtrafficgeneratorsproduce acorrectaveragethroughput;however,theinstantaneousvariationscouldbesignificantTogenerate highrates,dedicatedCPUresourcesmustbeconsistentlyavailabletothegeneratorShouldother processesstartonthetrafficPC,andthisisoftenthecasewithcertainoperatingsystems,theremay becomplicatedsideeffectsintheEPCortheeUTRANbecausebufferingmightoccurunexpectedly Consequently,theremaybeagapinthegeneratedtrafficandthusnothingtotransferovertheair interfaceforagivenTTI Ifnotmanagedproperly,theuseofanexistingapplicationserver(forexample,anFTPserver)to generatethetrafficcouldalsoproduceunexpectedsideeffectsForexample,anFTPservernormally accessesafilefromatraditionalharddiskIfmultipleuserstrytodownloadthesamefileatthesame time,therecouldbeabottleneckcausedbytheFTPserverratherthantheLTEairinterface(ingood radioconditions)ItispossibletosetupanFTPservertomanagethisifpropercareistakenandthe appropriatedisksystemisapplied;however,oversightsassimpleasanimproperlyconfigureddiskcan skewtheoverallresults Whenitcomestomeasuringtheperformance,oneessentialruleofthumbistounderstandthebasic aspectsthatwillcontroltheresultsForexample,anunderstandingoftheunderlyingradiolayer transportsettingsiscrucialHARQprovidesanexample:ifHARQisnotenabled,thelinkwillactually deliverahigherthroughputingoodradioconditionscomparedtotimeswhenHARQisnotswitched onOntheotherhand,reliabletransportdependsonHARQbeingswitchedonIfnot,problemssuch asasignificantamountofTCPretransmissionwillcropup,leadingtoaverypooreffectivethroughput EvenifHARQisswitchedon,theamountofHARQretransmissioncanbeconfiguredSettingthisto averylowvaluewillincreasetheinstantaneousthroughputbutwillleadtoapooreffectivethroughput undernon-idealradioconditions
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
Foradetailedunderstandingofbearerthroughput,measurementsshouldbemadeatdifferentlayers intheprotocolstackMAClayer,IPlayerandUDPorTCPlayersafterapplicableretransmissions Duringthesemeasurements,itisimportanttounderstandandrecordtheactualsettingsthatwere configuredfortheMAClayerandtheTCPlayer,andifanyspecificserviceorapplicationlayersettings havebeenapplied(forexample,iftheapplicationusingTCPhadoneormultipleTCPflows) InsomeveryspecificcasesitmaybeusefultomeasurethehighestpossiblebitratethattheLTEair interfacecandeliverForthismeasurement,therecommendedapproachistomeasurethelossless (forexample,zeropacket-lossthroughput)ofasustainedUDPstreamInsuchcases,thespecific MAClayersettingsshouldberecordedaswellastheUDPpacketsizeItcouldaddsubstantialvalueto performanRFC2544testovertheLTEconnection;thiswillstepthroughseveraldifferentpacketsizes andthroughputratestolocatethelosslessthroughputforeachrelevantpacketsize Tounderstandwhythroughputischangingindifferentenvironmentsatdifferenttimes,thebest approachistorecordasetofLTE-relevantparameterswhileperformingthethroughputtestThese parametersshouldincludetheinputstotheeNodeBschedulingdecisions(forexample,CQIforall oftheranks(widebandandsub-band),thePMI,andtherankindicator)andinformationaboutthe resultingeNodeBschedulingdecisions(forexample,selectedmodulationandcoding,MIMOmode, etc)Whentestinguplinkperformance,therelevantpower-controlinformationshouldalsobelogged JDSUprovidesallofthetoolsandprocessesneededtoplan,perform,evaluateresults,andprovide reportsforsingle-andmulti-userthroughputtestingThisincludesbothtoolstogenerateE2Etraffic aswellastoolstologdatafromhandsetsandrelevantnetworkinterfacesFortestinginthefield,signal sourcesavailablefromothervendorsemulateuplinkinterferenceandloadingcausedbyusersinother cellsBecauseloadinghasasignificantimpactonthroughputratesitshouldbeincludedaspartofany realisticevaluationThegeneratedinterferenceshouldideallyberepresentativeofanLTEairinterface inbothULandDLAlthoughwhitenoisewasacceptableforUMTS,whichusesanoise-likeCDMAair interface,itdoesnotrepresentthepowervariationsacrossfrequencyandtimethatcharacterizeLTEs OFDMandSC-FDMAmodulationschemesRapidvariationsmaytrickschedulersintomakingeither optimisticorpessimisticpredictionsofthechannelconditionsandsubsequentmodulationandcoding schemethatcanbereliablytransmittedJDSUtoolsprovidethemeanstoevaluatethisimpactbefore networksbecomeheavilyloaded Cell performance verification with multi-user throughput testing MeasuringtheperformanceofanLTEsysteminasingle-usercaseprovidesabasicunderstanding uponwhichamorerealisticanalysisofthemulti-usercasecanbeperformedItshouldbe understoodthat,inmanycases,single-userpeakthroughputperformancewillbemuchhigherthan eventheaggregatecellthroughputwithseveralusersactivelydownloadingFromindustrystudies performedbyLSTIandothers,DLcellthroughputisexpectedtobearound35-40Mbps(assuming 20MHzwith2x2MIMO)with10usersspreadoverthecell,allwithfullbufferdownloadsanda proportionalfairscheduler1,2
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
Whenbasicmulti-usertestingisperformed,theresultsmayvarygreatly,spanningfromvalues belowtheanticipatednumberstoratesnearthesingle-userpeakperformanceThereasonforthis widevariationisduetotwokeyfactors:theoverallbehavioroftheeNodeBsschedulingandthe environmentforeachindividualuserItisthereforeimportanttosetupthetestinarepeatablewayto ensurethatresultsareusefulontheirownandcanbecomparedagainstsimilartestsineitheradjacent cellsinthesamenetworkorwithcellsthatusenetworkequipmentfromanothervendor Theprocessofsettingupatesttoverifycellperformancewithmultipleusershastoomanystepsto presenthereindetailInoutlineform,herearethekeysteps: 1 Evaluatethedistributionofsignalqualityinthenetworkthroughexhaustivedriveorwalktesting tobuildaCDF a IftheCDFsfordifferentcellsaresignificantlydifferent,theendresultwillbesignificantlydifferent b Thedistributionwilldependonthetypeofterraincoveredandwherethewalkordrivetestis performedForexample,includingindoorlocationswillproducelowersignalqualitiesthan outdoor-onlydrivetesting c Itisexpectedthatroughly80percentoftheLTEdatatrafficwillbegeneratedindoors 2DistributetheUEsaccordingtothesignalqualityCDFtoobtainapropertrafficprofile a ThechoiceofdistributionshouldmatchnotonlythemeasuredCDFbutalsotheanticipated distributionofrealusersintherealnetworkForexample,ifitisanticipatedthatseveralusersare locatedinagroupinapubliclocation(forexample,anairportorcaf)thenitisadvisabletoplace someoftheusersinsimilarpatterns b DifferentMIMOconditionsmightbestressedduetothistypeofdistribution(forexample,multiusergainsfromMIMO) 3Intherealnetwork,locatetheusersaccordingtotheplanneddistribution a Notethatitwillbenearlyimpossibletogetanexactmatchsoitisimportanttoinsteadlocatethe usersinanareasimilartotheoneidentifiedMakesurethattheactualradioconditionsofthe chosenlocationsareloggedandstoredforthecompletedurationofthetest b Forcasesinwhichsomeoftheuserswillbeinamovingenvironment,makesurethatthiscanbe managedinacontrolledway 4 EnsureOCNGisenabledintheDLfortheadjacentcellstocreateDLloadinginthecell 5EnsurerealisticULloadingisgeneratedfromUEsinadjacentcells 6GeneratetraffictothetestUEsandensurethateachUEissetuptoreceiveortransmittrafficwith fullbuffers 7Logalloftherelevantparametersasidentifiedearlierincludingthelocation(latitudeand longitude)ofeachUE a Logallrelevanttrafficfromthenetworkinterfacestoensurethatfullbufferswerepresentforall UEsinallconditions b VerifythatallUEswereactiveandthattheindividualUEthroughputratesarerealisticcompared toexpectations 8CorrelatetheUEsentireindividualthroughputforMAC,IP,andUDPorTCPlayers a IfoneorseveraloftheUEsweremoving,theaggregatedthroughputmaychangeovertime dependingontheschedulingandspecificradioperformanceofeachdevicesenvironment
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
Itshouldbenotedthatthisisnotarealisticwaytotestacellsactualperformanceinarealnetwork scenariowhenbuffersarenotalwaysfullInstead,itisatestofhowtheeNodeBwouldbehavein averyspecificscenarioThiscouldnormallybeconsideredaworst-casescenarioforaspecific distributionofusers Realistic multi-user throughput testing Despitethecomplexity,therearestilllimitationsinmulti-usertestingthatusesfullbufferdownloads anduploadsInarealistictrafficscenario,usersareexpectedtohaveburstytrafficprofiles,possibly comprisingHTTP,voice,FTP,IM,e-mail,etcInthefull-buffercase,userswithgoodconnections downloadmoredatathanuserswithlowdatarates,whichbiasesresultsAlthoughthisispartlytrue, userstendtodomorewithafasterconnectionthefull-buffercaseisperhapstooextremeThe question,then,iswhatwouldbeamorerealistictrafficmodel?Woulditbevaluabletopursuethis testingand,ifso,whatistheaddedvalue?Theanswerdependsonwhichphaseofthetechnologythe specificoperatorisinatthetimeIftheoperatorisabouttoperformvendorselection,thiscouldprove tobeanessentialtesttoensurethattheequipmentbehavesappropriatelyintermsofscheduling,ability todeliverthedesiredQoS,andoverallfairnesstothedifferentusers ItisalsoimportanttounderstandthatdeliveringtheexpectedQoSorbeingabletoprovidefairness inthesystemisnotnecessarilyadifficulttaskforaneNodeBInpractice,thedifficultyistheabilityto delivertheexpectedQoSwithaminimumofoverheadfactorsthatimpacttheoverallcellcapacity Thefollowingisaproposedtesttohighlightcertainpossibledeficienciesinasystem 1 QoSoverheadprovisioninganalysis a Setupastaticmulti-usercelldownloadwith6to10usersperformingfull-bufferdownloads b Measuretheoverallcellcapacity c Reassignoneuserasamobileuserwithafixed-rateUDPstreamwiththesamemegabit-persecondratethatwasachievedinthefull-bufferdownload i SettheQoSparametersforthisusertomatchthefixedrateoftheUDPstream ii Verifythatthethroughputrateisintherangeof2to5Mbps d Measuretheoverallcellcapacity e Movetheuserfrommediumtopoorradioconditioninfivestepsandrepeatthecell-capacity measurementsforeachstep f Movetheusertothecelledgeforthespecificdataratethatwasprovisioned g Measuretheoverallcellcapacity ComparethecellcapacityfirstwithandthenwithouttheQoSprovisionedforthebandwidththat canbedeliveredwithoutanyactualimpactontheoverallcellcapacityThencompare,stepbystep, theoverallimpactofusermobilityoncellcapacityandguaranteedQoS 2Testtheimpactofrealistictrafficbehaviorontheoverallscheduling a Setupagroupofusers(6to10)witharealistictrafficprofileforthetypeofusersenvisionedfor theLTEnetworkThefollowingareexamplesofusescenarios i Downloadfivewebpageswitha30-seconddelaybetweeneachdownloadEachpage contains20objectsforatotalof14MBperpage ii Receiveane-mailmessagewithalarge(forexample,10MB)attachment
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
iii Send10e-mailmessages,fourwith2-MBattachmentsandsixsmalloneswithonly50KBof dataineach iv ListentoInternetradio(256Kbps) v PerformabackgroundFTPdownloadofsevenlargefiles(totalof100MB,similartoa monthlyoperatingsystemupdate) vi MakeoneVoIPcallusing122KbpsonaguaranteedQoSbearer Notethatthetestconditionsmustbesetupsuchthattheradiointerfacewillbecongestedin certainconditionsduringtestexecutionUsethepreviouslymeasuredanticipatedcellcapacity numberstoselectthecorrecttest-caseparameters
b Thesequenceofeventsoccursinparallelforeachuser;however,theusersareseparatedintime by10secondseach c EachuserwouldbelocatedinaspecificradioenvironmentdefinedbytheCDFforthespecificcell d LetthetestruntocompletionwhileloggingalldatafromboththetraceUEsandthenetwork basedprobesIfpossible,alsologthedatafromtheLTEUuinterface ThedatashouldthenbeanalyzedtorevealactualbehaviorduringcongestedconditionsOne shouldspecificallystudytheimpactofthescheduleronboththeguaranteedQoStraffic(VoIP) andthestaticRTPstreamfortheInternetradiobecausethesetwoservicesarethemostlikely todegrade Asimpleandbasicmeasureistobenchmarkthetotaltimeofcompletionforthetestsequence (excludingtheInternetradiostreamandtheVoIPsession,whichcouldrunindefinitely)The minimumcompletiontimeshouldberelativelyeasytocalculatebasedonthepreviously measuredperformanceforthemulti-userandfull-bufferdownloadscenariosThedifference fromthisminimumtimeshouldbeanalyzedtounderstandtheoverallefficiencyofthesystem AddingmobilitytothistestcasewouldallowforfurtheranalysisHowever,itwouldprobablyadd somuchcomplexitythatitcouldnotbejustifiedasabasiccase Allaspectsoftheschedulingandperformanceoftheradioenvironmentshouldbeanalyzedto understandtheefficiencyineachlayerandduringeachprocessThiswillalsohelprevealwhich conditionsleadtosuccessfulschedulingresultsandwhichcauseschedulingissues
JDSUcanprovidethetoolsandtheprocessestoexecuteandanalyzetheresultsfromthesetypesof testsJDSUtoolscanhelpcharacterizethebehaviorandallowforbothcompetitivebenchmarkingand regressiontestingwhenalaundrylistmustbemaintainedforaspecificsupplier Idle-to-active transition times OneofthemainexpectationsofLTEistoprovideanalwaysconnectedexperienceforendusersThis isachievedinpartbyensuringaswifttransitionbetweentheidleandactivemodesTheoverallRRC statemachinehasbeenoptimizedandthenumberofpossiblestateshasbeenminimizedtoensure reducedcomplexity,lowerpowerconsumption,andfastertransitiontimes Tomeasuretheidle-to-activetransitiontimesonemustbeabletoeitherfullycontrolUEbehavior orbeabletologalloftheassociatedsignalingtoensurethedataisavailabletobemeasuredfromthe overalltraffic JDSUtoolscanmeasureidle-to-activetimeaswellasotherrelevanttransactionandproceduraltimes Thesemeasurementscanbeperformedusingeitherdatafromatracemobilealoneordatacombined fromtheLTEandEPCnetworklinkstoenabletruecorrelatedE2Emeasurements
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
10
Latency testing Theoverallexperienceismorethanthespecificbandwidththatanendusercanreceiveandhow quicklythenetworkwillenableswitchingbetweentheidleandactivemodesTheexperiencealso dependsontheE2ElatencyofapackettransitioningthroughthecompleteEPCandeUTRAN KPI verification and calculation TheJDSUbookLTE and the Evolution to 4G Wireless: Design and Measurement Challengesincludes anextensivesectionaboutKPIs,thecalculationmethods,andtheoverallmethodology(pleasereferto Chapter6)Asaresult,theKPImaterialcoveredhereiskepttoaminimum ItisunfortunatethatthephraseskeyperformanceindicatorandKPIhavebecomecommonly misunderstoodandmisusedAtthemostbasiclevel,aKPIisnothingmorethanastatisticora measurementHowever,itisthetestobjectiveormarketrequirementforagivenservicethatallowsa particularstatisticormeasurementtobeconsideredasakeyindicatorofperformance EventhetermperformancecanmeanverydifferentthingsdependingonthetestingcontextFor example,serviceperformanceforVoIPmaybemeasuredintermsofjitter,latency,anddropped packetsNetworkperformancemaybemeasuredbythenumberconcurrentVoIPusersthatcanbe servedwithanacceptablelevelofjitter,latency,andpacketlossThus,whenmeasuringqualityor performance,oneofthekeychallengesisagreeingondefinitionsthatenableconsistentinterpretations ofresults Accordingto3GPP,KPIsgenerallyfitintooneoffivecategories:accessibility,retainability, integrity,availability,andmobilityThelistissometimesexpandedtoincludethecategoriesof utilizationandusability 3GPPKPIstandardizationeffortsarelargelyfocusedonmeasurementsrelatedtotheendusers perceivedQoSThesemetricstendtobemoreoperator-centricastheyrelatespecificallytomeasuring theabilityofcustomerstoobtainandmaintainaconnectiontothenetworkandtherebymakeuseof oneormoreavailableservices KPIsarebestunderstoodinthecontextoftheactualobjectiveofthemeasurementEachpartofthe networkhasdifferentresponsibilitiesassociatedwithdeliveringasingleserviceTherefore,LTEspecificKPIsfocusontheeUTRANitselfand,inmanycases,relyontheeNodeBtoactuallymeasure itsownperformanceOnechallengeforanNEMisdefiningawaytoverifythatKPIscalculatedbythe eNodeBarecorrect,especiallywhenrunningathighloadorfullcapacityAnotherchallengeforboth NEMsandWSPsismakingtheshiftfromsimplylookingatKPIstotroubleshootingandidentifying therootcausesofproblems ForeachKPIcategory,eachservicemayhaveadifferentQoSprofileorQCIlabelToidentifythe performanceofeachtypeofservicebeingaccessed,themeasurementsshouldbemadeonaper-QCI basisAdditionalKPIsineachofthesecategoriesshouldbeconsideredinordertoevaluatetheend-toendusabilityandmanageabilityofaservice Keyaspectsofawell-designedend-to-endtestsystemarethedatasourcesandthepossiblemonitoring pointsthatexistinanLTEandSAEnetworkSomeofthetopicsdiscussedherearealreadypartofthe industrystandardsOtherareasmayormaynotbepartoffuturestandardizationeffortsby3GPP,ETSI, orotherindustrybodies
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
11
Fundamentaltothetopicofdatasourcesandmonitoringpointsisanunderstandingofmeasurement theory,basicphysicallaws,andhowtheLTE/SAEtechnologyworksandisdeployedConsiderthe followingexamplecommontobothLTE/SAEandUMTSinwhichtwoengineersneedtomeasurethe RRCconnectionsuccessratiointhenetwork efirstengineer,whoisexperiencedindrivetesting,commissionsatargeteddrive-testcampaign, Th measuringtheRRCconnectionsetupsuccessratioforawideareaClearly,thenumberof measurementpointsisdirectlyrelatedtothedurationofthetestingandthenumberofactual attemptsperunitoftimeForthisexample,theengineerconcludedthattheRRCconnection successratiowas985percent esecondengineerisaccustomedtonetworkcountersandlink-monitoringtoolsAsaresult, Th heextractslogsfromthesystemThisprovideshimwithareportofalloftheRRCconnection attemptsforthefullnetwork;hismeasuredRRCconnectionsuccessratiois995percent Whyisthereadifferenceofnearlyonefullpercentagepoint?Theanswerisfundamentaltotherestof thisdiscussion Thedifferenceintheresultsisnotduetoflawsinthedatasource,beitthedrivetest,thenetwork counters,orthelink-monitoringtoolsThedisparityiscausedbythetwoengineersmeasuringdifferent networkproceduresfromdifferentangles Thedrive-testmethodanalyzesnetworkperformanceasseenfromasinglehandsetatspecific physicalpointsinthenetworkatdistinctpointsintime Thenetworkcountersandlink-monitoringtoolsrecordallofthetrafficandalloftheoccurrences ofsignalingandusertraffictheyaredesignedtomonitorThisisadifferentframeofreference:it analyzesnetworkperformanceasseenonthenetworkandatthenetworkmonitoringpoints Differencesbegintoaccumulateifanydrive-testlocationsarewelloutsidetheactual,andpotentially, intendednetworkcoverageareaAsaresult,RRCconnectionrequestsmadeoutsideofthenetwork coverageareawillberecordedbythedrive-testsystembutnotbythenetworkcountersorlinkmonitoringsolutionsThishighlightsakeypoint:anextensivedrive-testcampaignprovidesadditional informationbeyondwhatnetworkcountersorlink-monitoringtoolscanprovide Intheoptimizationcommunity,itisgenerallyagreedthatKPIsshouldbecomparedtoeachotheronly whentheyarederivedfromthesamedatasourceorwhentheyarenormalizedtoremoveanybiasdue tomethodorsourceThisisespeciallytrueifcomparisonsshowunexpectedresultsToday,thelackof propercomparisonsisoneofthelargestcontributorstounsoundoptimizationdecisionsinthemobile industryValidcomparisonsandmeaningfuloptimizationcanbeensuredonlyifastringentand coherentagreementondatasourcesandmonitoringpointshasbeensettledinadvance3 WhetherengagedintheR&Dprocessortheoptimizationphase,onemustoftenchoosebetween severaldifferentstrategieswhendevelopinganLTEtestplanSelectingthemostcost-effectiveand results-effectivestrategyisoneofthemostimportantdecisionstobemadeearlyineachphaseofthe workOnceastrategyisselected,theboundaryconditionsofitsapplicabilitymustbeestablishedItis importanttonotethatastrategythatisappropriateforonephaseoftheworkprobablyhassignificant shortcomingsinanyotherphaseInotherwords,itisseldomagoodideatousethesamefundamental KPIsbecausethedatasourceswillprovidedifferentresultsindifferentphasesofanetworks deploymentandmaintenance
3PleasenotethatthisissimplyamatteroffundamentalmeasurementtheoryItisnotintendedtobeadiscussionaboutthepotentialriskof resultsbeingincorrectduetoacertainmeasurementtoolnotworkingasdefined
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
12
Toillustratethisidea,consideracaseinwhichQoSiscontrastedwithQoEMonitoringtheend-user IPtrafficonamobilenetworkprovidesafullanddetailedunderstandingofthetrafficflows(TCPor UDP),theapplications(voice,video,HTTP,e-mail,etc),andpotentiallyperformanceSomebelieve thatthemonitoringofonlytheUDPorTCPflowswillprovideenoughinformationabouttheend-user QoStobeabletodeduceagoodapproximationoftheend-userQoEAsthefollowingscenarioshows, thisisnotthecaseforapplicationssuchasstreamingvideo useriswatchingstreamingvideoonhishandset,buttheradioqualityisnotsufficienttodeliver A thefullbandwidthovertheairinterfaceWhentheUDPstreamismeasuredinthecorenetwork, nodegradationoftheRTP/UDPstreamisobservedThreeotherlocationsprovideabetterplaceto observethedegradation:ontheairinterface,ontheusershandset,orbetweenthetwoendpoints oftheRLCentity(intheUTRANbetweentheUEandtheRNCorintheLTEeUTRANbetween theUEandtheeNodeB)Thenetworkmonitoringtoolinthecorewouldreportahighandstable bitrate(goodQoS);however,theenduserwouldreportapoorQoE hiftingthescenarios,assumethattheendusersapplicationisquality-awareand,duetotheradio S conditions,signalsthatthevideostreamingservershouldchangethebitrateofitscodecWhen thisnewRTP/UDPbitstreamappearsonthenetwork,thenetworkmonitoringtoolwillassociate thechangewithalowerQoSstreambecauseithasalowerbitrateOntheotherhand,theradio conditionsaregoodenoughtodeliverthisadaptedbitstream,andtheenduserQoEhasnowincreased ThisscenarioshowsthatthecrucialelementintheQoEisnotthebitratemeasuredinthecorenetwork, butrathertheend-to-endabilitytodeliveraspecificservicetotheenduserTheapplicationdomain will,inthiscase,ensurethatthebestpossibleQoEisachievedTherefore,themonitoringtoolmustbe application-awaretodelivercorrectQoSmeasurementsthatleadtothecorrectestimationofend-userQoE Validating LTE voice Oneoftheto-be-defineditemsfortheEPSishowcircuit-switchedservicessuchasvoice,CSUDIvideo, SMS,LCS,andUSSDwillbemanagedFouralternativesarecommonlyconsidered Circuit-switchedfallback(CSFB) VoiceoverLTEgenericaccess(VoLGA) VoiceoverIMS(VoIMS) Proprietaryoptions CSFBandVoLGAarebothstandardizedandcouldbereadilyimplementedVoIMSislikelytofollow andisanticipatedtobeawidespread,long-termsolutionAmongthese,CSfallbackinEPSisdescribed indetailbelow;theotheroptionsarecoveredinbrief Anotheraspectthatisnotyetspecifiedin3GPPR8istheuseofavoicecodecSeveraldifferentoptions areathand;however,duetoalackofagreement,thispartofthestandardizationmightbedelayeduntil R10Onekeyreasonforthedelayisthelackofclarityontheobjective:shouldthequalitybeimproved orshouldthecapacitybeimprovedbythechoiceofcodec?Itislikelythatachoiceofcodecforinitial EPSdeploymentswillbebasedonamutualbilateralagreementbetweentheUEvendors,theoperators, andtheEPSproviderItwouldnotbeasurpriseifAMRandAMR-WBwereusedinitially ItiscrucialtounderstandthattheITUhasalreadymovedaheadwiththedefinitionoftheG718codec G718isbuiltontheAMR-WBfoundationand,forthemostpart,providesthequalityofAMR-WB at1265KbpsonthesamecapacityasAMR795KbpsOverall,thismeansthatG718providesa57 percentincreaseincapacitywithaverylimitedimpactonspeechdelayAdetaileddescriptionis outsidethescopeofthispapersothereaderisencouragedtostudyITU-TRecG718infulltofurther understandthistopic
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
13
Signaling validation Figure2isatypicalexampleofanIMScallflowforLTEinteractingwiththeSS7networkAswillbe shown,thisisfarfromonlySIPandsomeofthecriticalaspectscannoteasilybeseeninthissimplified viewOneexampleisthelackofvisibilityintothechoiceofthebearersthathavebeensetupandhow theymaptoacertainQCI,etc Anyshort-termanalysisofavoice-over-LTEimplementationwillmostprobablybeimpactedbyone orseveralshortcutseveniftheintentionistobestandardscompliantThisshouldbeconsideredwhen performinganalysisanddrawingconclusionsKeyaspectstoconsideraretheusageofROHCforthe airinterface;theusageofacodecforE2EspeechandtheintegrationofapagingprocedurewhentheUE isinidlemode;and,howthemobilitybetweenaccesstechnologieswouldbemanaged
eNB
MME
Core IMS
PSTN
Terminating UE
IDLE TO ACTIVE
RRC Connection Setup RRC Connection Setup complete + NAS service request
Security Mode Command + RRC Connection Recon guration RRC Connection Recon guration complete
IAM
CALL SETUP
IAM
COT ACM/CPG
COT ACM/CPG
Figure 2. Typical IMS call flow for LTE interacting with SS7
Voice QoS and QoE (MOS) VoiceQoSandtheresultingQoEisatopicofmanygoodpublicationsandthusthissectionwillnotgo intotoomanydetails Overall,onemustbecarefulaboutwhatistestedunderwhichconditions,andonemustbeawareofthe governingfactorsthatcontroltheoutcomeIfacertainvoiceserviceistestedwithabearerdeliveringa specificQoS,thentheresultingvoicequalitywillbelimitedbythequalitydeliveredbythebearer
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
14
Whilethissoundssimpleintheory,itcausessignificantpracticalconcernsinalmostallnew technologiesbeforetheE2EstructureiswellunderstoodInUMTS,voicewasnormallydelivered usingaradiolinkwithaone-percentBLERtargetCertainimplementationsusedonepercentasa minimumqualitytargetbutover-deliveredonqualityifandwhenresourceswereavailableThis meantthatcomparingasituationinwhichtheBLERwasalwaysheldattheconfiguredBLERtarget (regardlessofavailableresources)andBLERwasadjustedtodeliverthebestpossibleservicewithout causingdegradationforothers,theresultsinanunloadednetworkwouldalwaysfavortheflexible implementationThisisnotalwaystheintendedtestobjectandthusthevalueofvoice-qualitytesting canbedegradedorworse,beconsidereduselessTherefore,itiscriticaltounderstandtheunderlying conditionsthatwillimpactvoiceserviceandtoeithercontrolthesefullyorrecordonlythose parametersusedThiswillensurefaircomparisonswhenbenchmarkingaresulteitherovertimeor betweenimplementations JDSUsuppliestoolsthatcanperformE2Evoice-qualitytestingandalsobenchmarkvoicequality passivelyinsidethenetwork(Figure3)
JDSU J7830A Signaling Analyzer and J6900A Triple Play Analyzer JDSU VoIP Office End
EPC
Serving/PDN GW
Testing QoS and QoE of LTE streaming video QoSandQoEmonitoringofavideoserviceisdirectlyanalogoustotheearlierdiscussionofKPIsIn short,allofthesamemethodsandissuesidentifiedintheKPIsectionapplytothesemeasurements Ratherthanrepeatingthatmaterial,pleasereferbacktotheearlierdiscussion Thereisonekeypointtoadd:understandingvideoQoSandtheresultingQoEshouldbedoneacrossall relevantaspectsoftheLTEandEPCThismeansthatonecannotonlyidentifytheactualQoSatapoint butcanalsoensuretheserviceupstreamfromthispointifthetrafficflowisgoodatthemonitored locationOntheotherhand,ifthetrafficflowispooratthemonitoringlocation,thenafurther investigationupstreamshouldbeperformedNotethatinmanycasestheclientcanusejitterbuffersto compensateforacertainamountandtypeofQoSimpairmentsThus,adegradedQoSdoesnotalways leadtothedegradationofQoE
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
15
JDSUprovidesalloftherelevanttoolstomonitorvideoQoSandQoEfromeitherapassiveperspective orfromanactiveperspective(Figure4)SupportisprovidedforbothstandardIPTVandforMSTV
Internet
Uu
UEs
Evolved Node B
MME
HSS
CSCF
MRF
PCRF
Figure 4. Tools for monitoring end-to-end LTE QoS and QoE (VoIP, IPTV, data)
Evaluating LTE MIMO and frequency-selective scheduling ComparedtoHSPAsystems,muchofthevalueinLTEcomesfromtheeffectiveusageofbothMIMO andfrequency-selectivescheduling(FSS)Inmanyaspectsthesearetechnologiesthathaveyettobe provedintermsofaddingrealtangiblevaluetothecustomerinafieldenvironmentTheirrealvalue dependsonthreethings:theactualdeploymentscenario;theoveralltrafficmodeling;and,thespecific implementationandconfigurationofthesystem AfeaturesuchasFSSmakesittheoreticallypossibletoaddsignificantperformancetothesystemIn practice,however,performanceislimitedbythealgorithmusedtocontrolschedulingThisisdue tooperationaltradeoffssuchasreducingsignificantuplinktrafficforsub-bandCQIreporting,or technicalfactorssuchasthepracticalavailablecomputationalperformanceintheeNodeB AsfarasMIMOisconcerned,itshouldbeunderstoodthatMIMOusageorthemeasurements leadingtoaspecificMIMOconfigurationforspecifictransmissionstoauserduringaspecificTTIis typicallynotastaticbehaviorWhatsmore,anyexpectationthatMIMOcanbecontrolled,modeled, orunderstoodfromoneorafewmeasurementsistypicallyunrealisticAsaresult,itisimportantto identifytwokeyelements:thoseaspectsofthetechnologythatshouldbeevaluatedfromastatistical behaviorpointofview;and,thoseparametersthatcanbeanalyzedfromasinglediscretemeasurement withoutlookingatalargersample ItisvitaltounderstandandcharacterizetheactualbehaviorofbothMIMOandFSStoensurethe properdimensioningofthecells,thebackhaul,andtheQoSprovisioningThestatisticalnatureof MIMOandFSScanbewellunderstoodonadetailedTTIlevelonlyiftherighttoolsandprocessare appliedThesecapabilitiesareallavailableusingJDSUtoolsandmethodsThetoolsallowadetailed analysisandcharacterization,enablingaclearunderstandingofactualbehavior
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
16
Testing network coverage Duringnetworkplanning,modelingisperformedandassumptionsaremadeIftheseareatall inaccurate,theundesirableconsequencesincludeeitherunplannedcoverageholesorunwantedsignal leakageintoadjacentcellsEarlyintheplanningandtuningprocess,aclearunderstandingofactual networkcoverageversustheplannednetworklayoutenablestheplanningandengineeringteamto developastrongmethodologythatallowsforverycost-effectivenetworkdeploymentandtuningEarly verificationofthecoveragewillallowcreationofaCDFofthenetworkperformanceandallowfor propertestingofcellperformance JDSUprovidesdetailedRFcoveragemeasurementsincludingRSSI,RSRP,RSRQ,andRS_CINR Thesecapabilitiescaneffectivelycoveralargenumberofbandsinverycomplexnetworktopologies Theresultsfromthemeasurementscanbepost-processedwitheithercustomsoftwareoranyofthe industry-standardpost-processingapplications Testing LTE handover CoveragetestingiscloselylinkedtothecharacterizationofHOvalidationandperformanceTofully graspHOperformance,onehastofirststudyandunderstandtheoverallproceduresandprocessesthat underliethehandoverinLTEAsiswidelyunderstood,nosofthandoverispresentinLTEAsaresult,the handoverisalwaysahardbreakbeforemaketransitionTheresultingimpactonend-userQoEmust beunderstoodandoptimizedItisalsoimportanttounderstandthehandoversoverheadimpacton theeNodeBbecauseLTEhandoverscanbeusedtoproactivelymanageserviceloadinginaspecificarea Severaldifferenttypesofhandoversarepresent:thosebetweencellsinthesameeNodeB;thosebetween cellsindifferenteNodeBswithoutanyX2dataforwarding;and,thoseinwhichX2dataforwarding isensuringaminimuminterruptionofdatatrafficItiscommonlyunderstoodthattheX2data forwardingfeaturemightbeimplementedinlaterreleasesoftheeUTRANsoftware;however,some vendorsmightmakethiscapabilityavailableearlyinthelifecycle AUEcannormallydiscriminatebetweenthetypesofhandoversbymonitoringtheSFNaroundtheHO timeandcheckiftheSFNisjumpingoriscontinuous Figure5isagoodexampleofahandoffbetweentwocellsinasingleeNodeBTheplotshowsthatthe BLERisincreasingthemomentsbeforetheHOandthatthisresultsindegradedthroughputonthe MAClayerAftertheHOisperformed(forexample,anRRCconnectionreconfiguration),MAClayer throughputisincreasedandnoBLERisreported
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
17
Figure 5. MAC throughput (green line) sags before an RRC event but recovers immediately after.
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
18
Asinallmobiletechnologies,itisrelativelystraightforwardtoachievesuccessfulhandoversHowever, achievinganoptimizednetworkwithoptimizedhandoversrequiresgreatinsightandtremendousskill intheartofoptimization Validating LTE backhaul LTEoffersgreatlyincreasedbandwidthtotheenduserInturn,itisveryimportantthatthisbandwidth capabilityisavailablethrougheverypartofthenetworkOneofthebigareasoffocusforthisisthe mobilebackhaulnetwork,typicallyEthernet,betweenthecellsite(eNodeB)andthecorenetwork InLTEfieldtrialswehavewitnessedtodate,thewirelessbackhaulnetworkisbroughton-lineprior toimplementationoftheLTE-specifictestcasesCommissioningtheEthernetbackhaulquicklyand easilyisvitaltokeepinganLTEfieldtrialonscheduleTheJDSUNetCompleteServiceAssurance SolutionforWirelessBackhaulverifiesEthernetserviceoperationtotrialcellsitesinadvanceofthe LTEtrialexecutionAdherencetoRFC-2544standardsismaintainedwithcost-savingderivedthrough automationandefficientdeploymentoftechniciansevenacrossmultiplecellsitesHowever,since suchtestingisnotuniquetoanLTEfieldtrial,itisbeyondthescopeofthisapplicationnoteGeneral informationisavailablesuchastheIETFsRFC2544(1999),theIEEE,andtheITUwhichdefinetest andperformancemonitoringmethodologiesforEthernetnetworkinterfacedevices Verifying LTE handset IOT ForeachoperatorintheearlyphasesofbringingLTEtomarket,itisessentialtoensureproper interoperabilityofLTEhandsetsordatacardsOfcourse,theseareearlydevicesthatwillcontinue toevolverapidlyovertimeConsequently,itisimportanttounderstandthecapabilitiesandensure interoperabilitybetweenthehandsetandtherelevantnetworkelementsItwillnotbepossibleto simplyrelyonpre-conformanceorconformancetestresultsbecausethesewillseldomreflectactual userbehaviororissuesfoundintheearlyphasesofnetworkdeployment IndustryforumssuchasLSTIhavedefinedaminimumfeaturesetandacorrespondingIoDTand IOTtestplanTheseareavailableonlytoLSTImembersandmaybeusedonlyforLSTIpurposesAsa result,non-membersmustrelyonothermeanstosecurehandsetinteroperabilityAsanLSTImember, AgilenthasmadesignificantcontributionstotheIoDTandIOTphasesThroughthisexperience personnelinbothAgilentandJDSUarebetterabletocontributeeffectivelytothewiderindustry JDSUprovidesmonitoringtoolssuchastheSignalingAnalyzerthatprovidethecapabilitytoanalyze andcorrelateinformationfromeachinterfaceandhighlightanydiscrepanciesversustheanticipated behavioronasignalinglevelTheJDSUNiXTapplicationcanactivelystimulateahandsettoexecute differentapplicationsandtasksThiscorrelatestocontroloftherelevantRFparameters(forexample, fading,MIMOprofiles,etc)andaneffectiveLTEhandsetinteroperabilityenvironmentcanbe providedIfappropriatelyandeffectivelyautomated,thiswillensurethatcost-effectiveandreliable testingisperformed,therebymaximizingassetutilization Validating device configuration OnepartofLTEisyettobefullystandardizedandwidelyagreedupon:theeffectiveperformanceof deviceconfigurationEvenifsignificantinteroperabilitytestinghasbeenperformed,anyoperator launchinganLTEserviceearlywillberequiredtoremotelyupdatehandsetsandparameterssuchas networksettings,software,orotherrelevantparametersduringoperationofthenetwork4
4ManyofthetraditionalOTAsystemsarebasedonshortmessageservices(SMS)thatarenoteasilysuppliedinanLTEenvironmentwithout theuseofeitherCSFBIMSorVoLGA,bothofwhicharerelativelycomplexanderror-pronetechnologies
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
19
Itisessentialtoprovidefullvisibilityintodevicebehaviorduringpre-configuration,configuration, andpost-configurationphasesandtocomprehendthefullE2Esignalingneededtodeliverthe configurationenvironmentWithJDSUtools,thepre-configurationstatecanbeanalyzedboth qualitatively(forexample,theperformanceofalloperationalservices)andinsimplego/no-gotesting (whichservicesworkordontwork)Specificnetworkbehaviorsthatidentifyincorrectlyconfigured devicescanbeanalyzedanddocumentedtobeusedinsubsequentnetwork-widemonitoringsystems Duringtheconfigurationprocedure,differenttypesofpositiveandnegativetestingcanbeperformed tostimulateerrorconditionsandtoprovethattheconfigurationissuccessfulifcertainconditionsare metThenegativeconditionsareespeciallycriticalbecauseitisoftendifficulttoevaluatethestateofan end-userdeviceinarealnetwork
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
20
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
21
Achievable data rates and latency: cell throughput and MU throughput for UL/DL and TCP/UDP Basic test overview GenerateaCDFofthecellaccordingtothecommontestdescriptionPlace10UEsaccordingto theCDFdistribution Verifyplacementofthe10UEsinappropriateandrepresentativelocations GenerateULandDLloadingof70percentfromadjacentcells PerformseparateULandDLthroughputtestsindividuallywithTCPandUDPMeasureaverage throughputfor30secondsduringstationaryconditionsTheTCPandUDPtestsshouldmeasure losslessthroughputratherthanrawthroughput Logallappropriateparametersandconditionsincludingfulltracesofthecontrolanduserplanes fromthenetworkandUEsides Providerawresultsandnormalizedresultsaccordingtothecommontestdescription Achievable data rates and latency: latency Basic test overview Evaluatetheend-to-endlatencyinarangeofradioconditionsandloadingscenarios No-loadscenario Performend-to-pinginarangeofradioconditionsforaminimumof100samplesperlocation andpacketsize(32,1000,1500bytes) With-loadscenario GenerateULandDLloadingof70percentfromadjacentcells Generatesame-cellDLloadfromoneUEingoodradioconditionswithfullUDPdownload Generatesame-cellULloadfrom1UEinpoorradioconditionswithfullUDPupload Performend-to-endping(32,1000,1500bytes)inarangeofradioconditionsforaminimumof 100samplesperlocationandpacketsize Logallappropriateparametersandconditionsincludingfulltracesofcontrolanduserplanefrom thenetworkandUEsides ProvideRANandEPClatencyand,ifrelevant,thebackhaultransmissionlatency Providerawresultsplusminimum,maximumandaveragevaluesforeachradioconditionand packetsize Intra-LTE mobility: mobility and handover performance Basic test overview TestandcompareresultsforhandoverbasedonbothS1aloneandonX2forwarding;testsamecell loadedandunloaded Samemethodologyforboth(S1andX2)testcases Repeattestforatleast20HOofeachtype PerformthetestwithDLandULTCPandUDPtrafficsubsequently MeasureandreportHOsuccessrate MeasureandreportcontrolplaneHOtime MeasureandreportuserplaneHOinterruptiontimeandpacketloss
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
22
Logallappropriateparametersandconditionsincludingfulltracesofcontrolanduserplanefrom thenetworkandUEsides ForloadedHOperformance,testaddtwoUEsingoodradioconditionsinbothcellsperforming full-bufferDLandUL Othercellloadshouldbe70percentinDLInthemobilitycase,notethatULloadingisnot requiredduetopracticalissues Achievable data rates and latency: application performance Basic test overview Fromidlemode,connectanddownloadtheCopernicuswebsite(seecommontestdescription)and measurethedownloadtimeinarangeofradioconditionsAminimumoffiverepetitionsateach locationisrequiredAminimumoffive(10isrecommended)differentradioconditionsshouldbe coveredfromgoodradioconditionstocell-edgeconditions Theothercellloadshouldbe70percentinULandDL Logallappropriateparametersandconditionsincludingfulltracesofthecontrolanduserplanes fromthenetworkandUEsides Coverage and capacity radio features efficiency and gain assessment: link budget Basic test overview Testfirstinstaticlocations: TheUEislocatedclosetothecellcenterAUDPDLtransferwithfullbufferisinitiated Theusermovesawayfromthebasestation,sopathlossincreasesRepeatthemeasurementsat thenewlocation Thisisrepeatedforatleast10differentlocationsacrossthecellAmajorityofthelocations shouldbeinpoorradioconditions;continueuntilcoverageislost Repeatthesametestwhileinmobileconditions:drivefromcellcentertocelledgeandrepeat severaltimes Logallappropriateparametersandconditionsincludingfulltracesofcontrolanduserplanesfrom thenetworkandUEsides Coverage and capacity radio features efficiency and gain assessment: scheduler Basic test overview LocatefourUEsingoodradioconditionsascloseaspossibletoequalradioconditionsEnsurethat allUEshavethesamepriorityandQoSsettings SetupDLloadingofUDPwiththefollowingtrafficcharacteristics:UE1=X;UE2=2*X;UE3=3*X; UE4=4*XEnsureXischosensothattheaggregatedthroughputoftheusersexceedsthecellcapacity Measuretheresultingbehavior LocatetheUEsatfourdifferentlocationsandgeneratethesameUDPstreamthroughputtoeach UEEnsureaggregatedthroughputoftheusersexceedsthecellcapacity Measuretheresultingbehavior Logallappropriateparametersandconditionsincludingfulltracesofthecontrolanduser planesfromthenetworkandUEsides
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
23
Evaluation of antenna configuration options Basic test overview Performthesingle-userthroughputtestcaseandthemultiusercellcapacitytestcasewiththe followingantennaconfigurations: SIMO 2x2MIMO 4x4MIMO(Optional:Performifpossibleforthevendor) Self-configuration and self-organizing network features Basic test overview EnsureatleasttwoeNodeBsareoperationalandnoneighborsexistinthetargetcell VerifyintheO&MsystemthatnoneighborsorX2interfacesareconfigured ActivateANRineNodeB TurnontheUEinacellwherenoneighborsaredefinedAfterUEreportingofsignalstrengthsof surroundingsectors,theANRfunctionshalladdneighborstotheneighborlist Performahandover VerifyintheO&MsystemthatallcorrectneighborsandX2interfaceshavebeenadded PerformanHOtoandfromthenewneighbor Logallappropriateparametersandconditionsincludingfulltracesofthecontrolanduserplanes fromthenetworkandUEsides Evaluation of frequency reuse: one deployment scenario Basic test overview LocateaUEatthecelledge(in100percentothercellload)andpreparefull-bufferdownloads Withoutothercellload,measureaveragethroughputfor30secondsforbothTCPandUDP With50percentothercellload,measuretheaveragethroughputfor30secondsforbothTCP andUDP With70percentothercellload,measuretheaveragethroughputfor30secondsforbothTCP andUDP With100percentothercellload,measuretheaveragethroughputfor30secondsforbothTCP andUDP Logallappropriateparametersandconditionsincludingfulltracesofthecontrolanduserplanes fromthenetworkandUEsides Comparethecell-edgeperformanceunderthevariousloadingconditionsandcomparethe measuredSINRforeachloadingconditionatthesamephysicallocation
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
24
Basic QoS: user differentiation between non-GBR users with different QCI Basic test overview OneUE(A)shouldhaveafull-bufferdownloadofUDPtrafficingoodradioconditionsandwith theQCIoflowestpriority OneUE(B)shouldhaveadownloadofaUDPstreamof10Mbpsandbelocatedinmediumradio conditionswithaQCIofsecond-lowestpriority OneUE(C)shouldhaveadownloadofaUDPstreamof10Mbpsandbelocatedinmediumradio conditionswithaQCIofhighestpriority Measuretheresultingthroughputandpacketloss(onUEBandC)andreporttheactualreceived throughputandtheS1andSGithroughput Othercellloadshouldbe70percentinULandDL Logallappropriateparametersandconditionsincludingfulltracesofcontrolanduserplanefrom thenetworkandUEsides Note:Toensurethatcongestionoccurs,itmaybenecessarytonormalizethethroughputofUEBandC Basic QoS: user-differentiation between GBR and non-GBR users Basic test overview
User GBR DL (Mbps) GBR UL (Mbps) MBR DL (Mbps) MBR UL (Mbps) ARP (Allocation and Retention Priority is NOT equal for all flows) UE1 X1 Y1 A1 B1 N1 UE2 X2 Y2 A2 B2 N2 UE3 X3 Y3 A3 B3 N3 UE4 X4 Y4 A4 B4 N4
Assume:N1>N2>N3>N4(User1hasthehighestpriorityandUser4hastheleastpriority)
LocatefourUEingoodradioconditionsConfigureDLbearerswithGBRvaluesforUE1,UE2, UE3,andUE4EnsurethatX1,X2,X3,andX4DLthroughputsareaggregatedabovethecell capacityatallUElocationsbutthatX1,X2,andX3aggregatedarebelowthecellcapacity GenerateUDPDLtraffictoclientsUE1,UE2,UE3,andUE4 StopDLtransferafteratleast30secondsofdatatransfertotheUEclients MeasureDLthroughput VerifythattheDLratesforUE1,UE2,andUE3arenolessthantheirpredefinedGBRvalues(X1, X2,andX3,respectively) VerifythatUE4(lowest-prioritytraffic)wasthemostnegativelyaffected(theDLbitratefailedto achieveX4,whichisthedesiredDL-GBRforUE4) ChangetheARPsettingofUE4tobethehighestpriority(N1) RepeatwithsimultaneousDLtransfertoallfourUEclients VerifythattheDLdataratetoUE3(lowestpriorityamongallUEs)ismostnegativelyaffected(its dataratewillbelowerthanitsdesiredDL-GBR)
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
25
PerformthesametestbutlimittheS1downlinkperformancetocausecongestionatalevelbelow theRFcellcapacitybutabovetheaggregatedX1,X2,andX3VerifythroughputisbelowtheS1 peakthroughputlimitation ThelimitationcanbeachievedviaVLANtuning Basic application performance: web browsing, streaming, voice calls, e-mail, VPN, on-line gaming Basic test overview Identifytherelevantservicesforeachlocalenvironment Selectaheavilytraffickedlocalwebsite: 1-5MBforthemainpage(includingimages) SelectaVoIPapplicationsupportedbytheoperatorandthelocalenvironment(forexample, Skype,GoogleChat,MSN,etc) SelectalocalcorporatecustomerusingaVPNinamobileenvironment;theVPNconnection shouldhaveadatarateontheorderof2-10Mbpsend-to-end Identifythreelocal,well-recognizedservices: Ahigh-requirement(low-latency)onlinegame Ahighperformancefixed-lineInternetservice(forexample,xDSL,packetcable,etc) Astreamingvideo(RTP/UDP)service Executeandcomparetheapplicationperformanceonthefollowingaccesstechnologies: HSPA(bestlocalcommercialavailableservice) Fixed-lineInternetservice LTEingood,medium,andpoorradioconditions Logallappropriateparametersandconditionsincludingfulltracesofthecontrolanduserplanes fromthenetworkandUEsidesforLTEand,ifpossible,forHSPAandthefixed-lineInternet service Forthevoiceservice,measureandcompareMOS Forthestreamingvideoservice,measureandcomparevideoMOSorasubjectivequality comparisonwithfriendlyusers Fortheonlinegame,getasubjectivestatementfromanexperiencedgamerandanE2Elatency measurement Reporttheabsoluteandrelativeperformancebetweenthedifferentaccesstechnologies
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
26
References
EPS specification references
3GPP TS 24.301 36.413 36.423 29.118
29.168 29.272 29.274 29.275 29.276 29.277 29.280 29.281
Title Non-Access-Stratum (NAS) Protocol for Evolved Packet System (EPS); Stage 3 E-UTRAN: S1 Application Protocol (S1AP) E-UTRAN: X2 Application Protocol (X2AP) Mobility Management Entity (MME) -Visitor Location Register (VLR) SGs Interface Specs. Cell Broadcast Center Interfaces with the Evolved Packet Core; Stage 3 MME Related Interfaces Based on Diameter Protocol Evolved General Packet Radio Service (GPRS) Tunneling Protocol for Control plane (GTPv2-C); Proxy Mobile IPv6 (PMIPv6) based Mobility and Tunneling Protocols; Stage 3 Optimized Handover Procedures and Protocols between EUTRAN Access and cdma2000 HRPD Optimized Handover Procedures and Protocols between EUTRAN Access and 1xRTT Access 3GPP EPS Sv Interface (MME to MSC) for SRVCC GPRS Tunneling Protocol User Plane (GTPv1-U)
3GPP references 3GPP TS 23272:CircuitSwitchedFallbackinEvolvedPacketSystem;Stage2 3GPP TS 24301:Non-Access-Stratum(NAS)protocolforEvolvedPacketSystem(EPS);Stage3 3GPP TS 29118:MobilityManagementEntity(MME)VisitorLocationRegister(VLR)SGs interfacespecification TS 23401:GeneralPacketRadioService(GPRS)enhancementsforEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN)access 3GPP TS 21905:Vocabularyfor3GPPspecifications 3GPP TS 22278:ServicerequirementsfortheEvolvedPacketSystem(EPS) 3GPP TS 43318:GenericAccessNetwork(GAN);Stage2 TS 23401:GeneralPacketRadioService(GPRS)enhancementsforEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN)access TS 24301:Non-Access-Stratum(NAS)protocolforEvolvedPacketSystem(EPS);Stage3 TS 29272:EvolvedPacketSystem;MMEandSGSNrelatedinterfacesbasedonDiameterprotocol TS 33102:3GSecurity;Securityarchitecture TS 33203:AccesssecurityforIP-basedservices TS 33210:3GSecurity;NetworkDomainSecurity;IPnetworklayersecurity
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
27
TS 33401:3GPPSystemArchitectureEvolution(SAE):SecurityArchitecture; TS 36300:EvolvedUniversalTerrestrialRadioAccess(E-UTRA)andEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN);Overalldescription;Stage2 TS 23401:GeneralPacketRadioService(GPRS)enhancementsforEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN)access TS 24301:Non-Access-Stratum(NAS)protocolforEvolvedPacketSystem(EPS);Stage3 TS 29272:EvolvedPacketSystem;MMEandSGSNrelatedinterfacesbasedonDiameterprotocol TS 33102:3GSecurity;Securityarchitecture TS 33203:AccesssecurityforIP-basedservices TS 33210:3GSecurity;NetworkDomainSecurity;IPnetworklayersecurity TS 33401:3GPPSystemArchitectureEvolution(SAE):SecurityArchitecture; TS 36300:EvolvedUniversalTerrestrialRadioAccess(E-UTRA)andEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN);Overalldescription;Stage2 TS 36420:X2layer1generalaspectsandprinciples TS 36421:X2layer1 TS 36422:X2signallingtransport TS 36423:X2ApplicationProtocol(X2AP) TS 36424:S2datatransport TS 29281:GPRSTunnelingprotocolforuserplane(GTPv1-U) TS 23401:GeneralPacketRadioService(GPRS)enhancementsforEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN)access TS 36300:EvolvedUniversalTerrestrialRadioAccess(E-UTRA)andEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN);Overalldescription;Stage2 TS 29305:InterworkingFunction(IWF)betweenMAP-basedandDiameter-basedinterfaces TS 29274:TunnelingprotocolforControlplane(GTPv2-C);Stage3 TS 23060:GeneralPacketRadioService(GPRS);servicedescription;Stage2 TS 36410:S1layer1generalaspectsandprinciples TS 36412:S1signallingtransport TS 36413:S1Applicationprotocol(S1AP) TS 36414:S1datatransport TS 29281:GPRSTunnelingprotocolforuserplane(GTPv1-U)
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
28
TS 24301:Non-Access-Stratum(NAS)protocolforEvolvedPacketSystem(EPS);Stage3 3GPP TR 22968:StudyforrequirementsforaPublicWarningSystem(PWS) 3GPP TS 22168:EarthquakeandTsunamiWarningSystem(ETWS)requirements;Stage1 3GPP TR 23828:EarthquakeandTsunamiWarningSystem(ETWS),requirementsandsolutions TS 23060:GeneralPacketRadioService(GPRS);servicedescription;Stage2 TS 23107:QualityofService(QoS)conceptandarchitecture TS 23203:PolicyandChargingControlArchitecture TS 23401:GeneralPacketRadioService(GPRS)enhancementsforEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN)access TS 36300:EvolvedUniversalTerrestrialRadioAccess(E-UTRA)andEvolvedUniversalTerrestrial RadioAccessNetwork(E-UTRAN);Overalldescription;Stage2 TS 23002:Networkarchitecture TS 23003:Numbering,addressingandidentification NGMN reference wwwngmnorg/uploads/media/White_Paper_NGMN_Beyond_HSPA_and_EVDOpdf ETSI reference ETSI TS 102 250-1:Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks: Part 1: Identification of Quality of Service aspects
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
29
Glossary
AAA AMR AMR-WB APN ASP BICC BLER BSC BTS CAPEX CDF CDMA CLID CLIP CSUDI CSFB DL DMT E2E eNodeB EPC EPS ESN ETSI eUTRAN FSS GGSN GPRS GUI HA HARQ HLR HO HTTP IM IMS IMSI authentication,authorization,andaccounting adaptivemulti-rate adaptivemulti-ratewideband accesspointname applicationserviceprovider bearer-independentcallcontrol blockerrorrate basestationcontroller basetransceiverstation capitalexpenditures cumulativedistributionfunction codedivisionmultipleaccess callinglineidentification callinglineidentificationpresentation circuit-switchedunrestricteddigitalinformation circuit-switchedfallback downlink dataminingtoolkit end-to-end evolvedNodeB;NodeBisaUMTSbasetransceiverstation(BTS) evolvedpacketcore evolvedpacketsystem electronicserialnumber EuropeanTelecommunicationsStandardsInstitute evolvedUMTSterrestrialradioaccessnetwork;alsoabbreviatedasE-UTRANorEUTRAN frequency-selectivescheduling gatewayGPRSsupportnode generalpacketradioservice graphicaluserinterface homeagent hybridautomaticrepeatrequest homelocationregister handover hypertexttransferprotocol instantmessaging instantmessagingservice internationalmobilesubscriberidentity
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
30
IP-TV IOT IP IRAT KPI LCS LSTI LTE MGW MIMO MSC MSC-S MSS MSTV NAI NE NEM NGMN OCNG OFDM OPEX OLAP PCF PDSN-FA PDSN-HA PoC PSTN QCI QoE QoS QoSM RNC ROHC RRC SAE SC-FDMA SCTP SFN SGSN
Internetprotocoltelevision inter-officetrunk Internetprotocol inter-radioaccesstechnology keyperformanceindicator locationservices LTESAEtrialinitiative Longtermevolution mediagateway multiple-input/multiple-output mobileswitchingcenter mobileswitchingcenterserver mobilesoftswitch MaximumServiceTelevision networkaddressindicator networkelement networkequipmentmanufacturer next-generationmobilenetworks OFDMAchannel-noisegeneration orthogonalfrequency-divisionmultiplexing operatingexpenses onlineanalyticalprocessing packetcontrolfunction packetdataservingnode,foreignagent packetdataservingnode,homeagent proofofconcept publicswitchedtelephonenetwork QoSclassidentifier qualityofexperience qualityofservice QualityofServiceManager radionetworkcontroller robustheadercompression radioresourcecontrol SystemArchitectureEvolution single-carrierfrequency-divisionmultipleaccess StandardControlTransmissionProtocol single-frequencynetwork servingGPRSsupportnode
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
31
SIP SMS SON STP TDR TTI UE UMTS UL URI USSD VIP VoIMS VoLGA WAP WSP
sessioninitiationprotocol shortmessagingservice self-optimizingnetwork signalingtransferpoint transactiondetailrecord transmissiontimeinterval userequipment UniversalMobileTelecommunicationsSystem uplink uniformresourceindicator unstructuredsupplementaryservicedata veryimportantperson voiceoverinstantmessagingservice voiceoverLTEgenericaccess WirelessApplicationProtocol wirelessserviceprovider
Application Note: LTE and EPC TestAn Overview of Test Concepts and Tools for Trials
32
Productspecificationsanddescriptionsinthisdocumentsubjecttochangewithoutnotice2010JDSUniphaseCorporation301682005000810LTEEPCTESTANNSDTMAEAugust2010