UMTS terminal testing: A practical perspective

Olaf Bergengruen May, 2003

Agere Systems - Optimay GmbH

May, 2003

-1-

We will not survive without a proper regression test system,

an Optimay engineer

Agere Systems - Optimay GmbH

May, 2003

-2-

Overview

Virtual testing

Logical view (what we want to test) Why virtual testing Requirements to the test system

3GPP approach Test model Test cases

Optimays implementation
GSM/GPRS virtual test system GSM/GPRS/UMTS virtual test system
Agere Systems - Optimay GmbH May, 2003
-3-

Test environment / logical view (conformance testing, FTA)

GSM/GPRS tests GSM 11.10 SAT tests GSM 11.10-4 GERAN simulation

Um Cu MS
SIM simulation

UMTS tests 3GPP 34.123

UTRAN simulation

Uu

Agere Systems - Optimay GmbH

May, 2003

-4-

Test environment / logical view (2)

WAP TCP/IP test cases

USB 802.11 Bluteooth simulation

GERAN simulation

Um
WAP tcp/ip

Cu
SIM simulation

MS

UTRAN simulation

Uu

MMI simulation

Agere Systems - Optimay GmbH

May, 2003

-5-

Why virtual test environment

We need a SW only environment (no specific HW)

HW is expensive or does not exist

HW debugging is difficult and time consuming SW developers dont want to be bothered with HW issues or dont have the specific skills

We need to be much faster than real time Developers want to use standard tools (compilers, editors, debuggers)

Agere Systems - Optimay GmbH

May, 2003

-6-

Requirements to the virtual test system

Test system shall be much faster than real time Support the design of test scenarios at a high level of abstraction All test suites need to run over night: GSM/GPRS tests (GSM 11.10), UMTS test (3GPP 34.123), STK tests (GSM 11.10-4), MMI test cases and possibly other suites (TCP/IP, WAP, ...) Test system shall be available to anybody, at anytime, on any PC

Agere Systems - Optimay GmbH

May, 2003

-7-

GSM/GPRS test architecture

Agere Systems - Optimay GmbH

May, 2003

-8-

GSM/GPRS test architecture

Optimay test scripts (GSM/GPRS test cases)

MS SS
MMI control Phone Apps GMM, MM, CC, etc GRR MAC/RLC RLC/MAC Logical L1 L2 Logical L1 DSP emulation RR L2

L1 simulation (Radio Block transfer)

Agere Systems - Optimay GmbH

May, 2003

-9-

Great approach: L1 simulated interface

Basic communication with the Tester via three functions: RxRadioBlock, TxRadioBlock, TimerTick These functions can be extended as needed to simulate DSP code or other HW related functionality MS is the master of system ticks, about 100 times faster than real time Tester and MS are tightly coupled which allows debugging using standard tools
Agere Systems - Optimay GmbH May, 2003
- 10 -

Debugging GMM code ...

static void { MMRAUpdPending = FALSE; GMRauType = RauType; GMStopT3302( ); GMStopT3311( ); if ( ! FirstAttempt ) { GMT3330Expiries = 0; if ( GMActState == GMActDetachInit ) { GMStopT3321( );
GMDetachPending = TRUE;

GMMDoRoutingAreaUpdate

( GM_IE_UPDATE_TYPE RauType, BOOLEAN FirstAttempt )

} } if ( GMTriggerAction == GM_GS_ATTACH_COMP ) { /* Reinitialise the trigger action variable. */ GMTriggerAction = GM_NO_TRIGGER_ACTION;

/* Confirm the end of the ATTACH procedure to GS */ SEND( GSGMEstCnf ) ( );

Agere Systems - Optimay GmbH

May, 2003

- 11 -

Highlights of our GSM/GPRS virtual test system

Extremely efficient performance (around 1300 tests in 20 minutes) Includes testing of DSP code About 30 man-years development effort (including test case development)

Agere Systems - Optimay GmbH

May, 2003

- 12 -

Drawbacks of our GSM/GPRS simulation

Tester uses proprietary (in house) test script languages and encoders which are difficult to maintain It is difficult to write test cases

Tester is not nicely layered as shown in the previous slide, so it is difficult to extend and maintain Summarizing: the system can not be extended to implement an UMTS System Simulator
Agere Systems - Optimay GmbH May, 2003
- 13 -

3GPP test model (TS 34.123-3)

TTCN test scenarios

UE SS
RRCadapt. PDCP RLC MAC PHY (UTRAN) MMI / E-MMI control Phone Apps NAS (GMM, MM, CC, etc) RRC SNDCP LLC RLC/MAC L2 L1 (GERAN) RLC MAC PHY MAC/RLC L2 L1 RR / GRR

RF cable

Agere Systems - Optimay GmbH

May, 2003

- 14 -

Why we need a Test Model

Main reason for us: If we comply with the 3GPP Test Model we can use the TTCN test cases developped here at ETSI / STF 160 In general:
Clear and stable interfaces to the SS enables manufacturers to develop the test equipment It reduces ambiguities within test case scenarios It enables outsourcing of testing activities and SW reuse It provides a common consens or understanding of the complete system to manufacturers and operators

Agere Systems - Optimay GmbH

May, 2003

- 15 -

A test case (1/2) ...

Configure SS
Configure PHY / L1
Configure MAC Configure RLC

Schedule and send System Information Blocks Test case preamble

Bring UE into initial state (e.g. CS and PS registration) Perform Location Update procedure

Perform GPRS Attach procedure

Agere Systems - Optimay GmbH

May, 2003

- 16 -

A test case (2/2) ...

Test case body Stimulate the UE, e.g.

Send X message to the UE, or Change cell power levels, or Trigger the UE via AT command to initiate a call

Verify responses
Match messages from the UE to expected values Assign a verdict (PASS, FAIL, INCONC)

Test case postamble Complete signalling to bring UE into stable state

Agere Systems - Optimay GmbH

May, 2003

- 17 -

So, how to design a Virtual Tester which matches the 3GPP model and our requirements ?

Agere Systems - Optimay GmbH

May, 2003

- 18 -

Problems designing an overall GSM/GPRS/UMTS Test Env.

The clean 3GPP model is not enough for a complete MS test environment We need to re-use tools, tracers, and most of all test cases developed for GSM in the new environment We can not afford to develop the UMTS test suite in house, we need to re-use the 3GPP tests The L1 and DSP simulation strategies for GSM do not match those of UMTS
Agere Systems - Optimay GmbH May, 2003
- 19 -

Current approach (a compromise)

Multi-threaded system: The proper MS, the GSM SS, the UMTS SS and the SIM simulator All communicating via shared buffers The tick-master is the MS, when all jobs for current frame are completed, it issues a tick and all System Simulators prepare data (radio blocks) for next frame. Each SS will send a tick-ack to the MS when it is ready for next frame When all SSs are done: the MS L1 engine will fetch or store a radio block
Agere Systems - Optimay GmbH May, 2003
- 20 -

GSM/GPRS/UMTS Test Environment

.bst, .asc files (Optimay tests) TTCN suite (3GPP tests)

Thread 1 Thread 2

GERAN simulation
SNDCP/ LLC RLC/ MAC L2 Thread 0

UTRAN simulation
PDCP RLC MAC PHY emulation

L1 emulation

Um

MS
Cu

Uu

Thread 4

Tracer
.sat files (Optimay) SIM simulation
Thread 3

Agere Systems - Optimay GmbH

May, 2003

- 21 -

Summary

A Virtual Test Environment is an essential tool for development for quality assurance

3GPP test model

A rough overview of our implementation

Agere Systems - Optimay GmbH

May, 2003

- 22 -

Thank you.

Agere Systems - Optimay GmbH

May, 2003

- 23 -