Test Driven Development For Embedded C: The Physics of Debug Later Programming (DLP)

The Physics of Debug Later Programming (DLP)
Test Driven Development for Embedded C

Presented by James Grenning at Agile China 2010 twitter: jwgrenning
Copyright 2008-2010 James W. Grenning All Rights Reserved. For use by training attendees.
Td
Tnd
T x Time
Mistake made (bug injection)
Bug discovery Bug found Bug xed
As Td increases, Tfind increases dramatically Tfix is usually short, but can increase with Td
http://pragprog.com/titles/jgade/
www.renaissancesoftware.net james@renaissancesoftware.net
Presented at Agile China October 14, 2010 Test Driven Development
1
1
Copyright 2008-2010 James W. Grenning All Rights Reserved.
2
2
Monday, October 18, 2010
The Physics of Test Driven Development

T d Tnd T x Time Mistake Mistake xed made Mistake Root cause found discovery
TDD Micro Cycle

Write a test Watch it fail Make it pass Refactor (clean up any mess) Repeat until done
When Td approaches zero, Tfind approaches zero In many cases, bugs are not around long enough to be considered bugs. See: http://www.renaissancesoftware.net/blog/archives/16
Copyright 2008-2010 James W. Grenning All Rights Reserved. www.renaissancesoftware.net james@renaissancesoftware.net
3
3
4
4
TDD State Machine in C/C++

Start
Test Driving a Circular Buffer (FIFO)
Choose a test No more tests All tests pass
Write the test
Compilation error
23
66
12
99
16
90
99
Compiles Clean
Refactor (Make it right) DONE! All tests pass Link error Make the test compile Compilation error
Out-Index
In-Index
Programming error
Make the test pass
New test fails
Make the test link
Link error
5
5
6
6
Create a Test-List (avoid analysis paralysis)

Circular Buffer Tests Initially Empty Transition to empty Transition to Full Transition from Full Put-Get FIFO Put to full Get from empty Filled but not wrapped around Wrap around
Set up a Test Fixture

TEST_GROUP(CircularBuffer) { CircularBuffer* buffer; void setup() { buffer = CircularBuffer_Create(); } void teardown() { CircularBuffer_Destroy(buffer); } }; TEST(CircularBuffer, TestName) { }
7
7
8
8
Make a bare Skeleton of the Production Code

#ifndef D_CircularBuffer_H #define D_CircularBuffer_H typedef struct CircularBuffer CircularBuffer; CircularBuffer * CircularBuffer_Create(int capacity); void CircularBuffer_Destroy(CircularBuffer *); #endif // D_CircularBuffer_H
Make a bare Skeleton of the Production Code

#include "CircularBuffer.h" #include <stdlib.h> struct CircularBuffer { int dummy; } ; CircularBuffer* CircularBuffer_Create() { CircularBuffer* self = calloc(capacity, sizeof(CircularBuffer)); return self; } void CircularBuffer_Destroy(CircularBuffer* self) { free(self); }
9
9
10
10
Choose a Test
Start
Which Test?
Write the test
Compilation error
Compiles Clean
Out-Index
In-Index
41
New test fails Make the test link Link error
59
14
33
31
Programming error
Make the test pass
In-Index
Out-Index
11
11
12
12
Production Code is Grown One Test at a Time
Write a Test
Start
Choose a test No more tests
Write the test
Code without new feature
Code with new tested feature

DONE!
All tests pass
Compilation error
Compiles Clean
Refactor (Make it right) Make the test compile Link error Compilation error
All tests pass
Programming error
Make the test pass
New test fails
Make the test link
Link error
13
13
14
14
Start with a Test That is Easy to Get to Pass -- Designing the Interface
TEST_GROUP(CircularBuffer) { CircularBuffer* buffer; void setup() { buffer = CircularBuffer_Create(); } void teardown() { CircularBuffer_Destroy(buffer); } }; TEST(CircularBuffer, ShouldBeEmptyAfterCreate) {
CHECK(CircularBuffer_IsEmpty(buffer));
Programming error
Make the Test Compile

Start
Write the test
Compilation error
Compiles Clean
Make the test pass
New test fails
Make the test link
Link error
}
15
15
16
16
Make the Test Compile

#ifndef D_CircularBuffer_H #define D_CircularBuffer_H typedef struct CircularBuffer CircularBuffer;
Make the Test Link

Start
Choose a test
Write the test
CircularBuffer * CircularBuffer_Create(int capacity); void CircularBuffer_Destroy(CircularBuffer *); int CircularBuffer_IsEmpty(CircularBuffer *); #endif // D_CircularBuffer_H
No more tests All tests pass Compilation error
Compiles Clean
Programming error
Make the test pass
New test fails
Make the test link
Link error
17
17
18
18
Make the Test Link and Intentionally Fail

#include "CircularBuffer.h" #include <stdlib.h> struct CircularBuffer { int dummy; } ; ... not all code shown ... int CircularBuffer_IsEmpty(CircularBuffer* self) { return 0; }
Watch it Fail
CircularBufferTest.cpp:48: error: Failure in TEST(CircularBuffer, EmptyAfterCreation) CHECK(CircularBuffer_IsEmpty(buffer)) failed
19
19
20
20
Make the Test Pass

Start
Write no More Code than is Necessary to Pass the Current Test

Hard code behavior or partial implementations are OK and often preferred.
Write the test
Compilation error
Compiles Clean
Programming error
Make the test pass
New test fails
Make the test link
Link error
21
21
22
22
Make the Test Pass

#include "CircularBuffer.h" #include <stdlib.h> struct CircularBuffer { int dummy; } ; ... not all code shown ... int CircularBuffer_IsEmpty(CircularBuffer* self) { return 1; }
Repeat until You Run Out of Tests

The first tests drive the interface definition Later tests complete the behavior You will think of more tests as you go
Write the new test or Add it to the test list
23
23
24
24
Tests are FIRST
Thank you: Tim Ottinger

Fast Independent Repeatable Self-validating Timely

Adaptation for Embedded Software

What is scarce during embedded development?
25
25
26
26
Hardware is Scarce!
It does not exist. It is being used by someone else.
Hardware is Scarce!
It does not exist. It is being used by someone else.
27
27
28
28
Hardware Has Bugs of its Own

bool BlindsScheduler::DoesLightRespondToday (TimeService::Day reactionDay) { int today = timeService->GetDay(); if (reactionDay == TimeService::EVERYDAY) return true; if (reactionDay == today) return true; if (reactionDay == TimeService::WEEKEND && (TimeService::SATURDAY == today || TimeService::SUNDAY == today)) return true; if (reactionDay == TimeService::WEEKDAY && today >= TimeService::MONDAY && today <= TimeService::FRIDAY) return true; return false; } void BlindsScheduler::CheckEvent(ScheduledBlindsEvent* blindsEvent) { if (blindsEvent->extraRandomEventPreventer > 0) { blindsEvent->extraRandomEventPreventer--; return; } if (!DoesLightRespondToday(blindsEvent->day)) return; if (timeService->GetMinute() != blindsEvent->minute + blindsEvent->randomMinutes) return; switch (blindsEvent->blindsType) { HorizontalBlinds* theBlinds; case Horizontal: switch (blindsEvent->operation) { case Raise:
Tests Must be Fast and We Can Run Them NOW
29
29
30
30
Use Your Development System for a Test Bed

Multi-targeted code. Must beware of hardware and OS dependencies. Object Oriented approach to Dependency Management. Avoid inefficiencies
Waiting for hardware. Waiting for restart. Waiting for downloads. Waiting for long compiles. Debugging on the target.
But There are Risks with Development System Tests

Architecture differences
Word size Big-endian Little-endian Alignment
Compiler differences Library differences Execution differences
31
31
32
32
TDD Adaptation for Embedded Development

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
TDD Adaptation for Embedded Development

Stage 1 Stage 2 Stage 3 Run Tests in the Eval Hardware or Simulator Stage 4 Stage 5
Write a Test Make it Pass Refactor
Compile for Target Processor
Run Tests in the Eval Hardware
Run Tests in Target Hardware
Acceptance Tests
Write a Test Make it Pass Refactor
Compile for Target Processor
Run Tests in Target Hardware
Run Manual Tests in Target
More Frequent
More Frequent
Less Frequent
Less Frequent
See : http://renaissancesoftware.net/files/articles/ProgressBeforeHardware.pdf
33
33
34
34
Automate Stages 2-5
Source Code Repository
Continuous Integration Server
Test Management System
TDD and the Code in the Middle

(most of the code!)
Eval/Reference System Developer Workstation Target System Simulator
35
35
36
36
What about Code in the Middle
Testing a Module in the Middle

The test case takes the role of the client
The Module under test is tested independent of other modules

The servers might need to be stubbed out

37
37
38
38
A Module with Collaborators

Every minute, the RTOS wakes up the Light Scheduler. If it is time for one of the lights to be controlled, the LightController is told to turn on/off the light.
Admin Console Light Scheduler +addSchedule() +removeSchedule() +wakeUp()
<<anonymous callback>>
Program to Interfaces
Separate interface and implementation as separate entities. This design has good separation of responsibilities
Admin Console Light Scheduler +addSchedule() +removeSchedule() +wakeUp()
<<anonymous callback>>
Light Controller + on(id) + off(id)
Time Service + getDay() + getTimeOfDay() + setPeriodicAlarm() RTOS
<<interface>>
Time Service + getDay() + getTimeOfDay() + setPeriodicAlarm()

<<implements>>
<<interface>>
Hardware
<<implements>>
Real Light Controller
Real Time Service
Hardware
RTOS
39
39
40
40
Testing a Module with Collaborators

Use the real collaborators if you can. Use fakes when you must.
Light Scheduler Test Light Scheduler +addSchedule() +removeSchedule() +wakeUp()
Testing the Scheduler

TEST(LightScheduler, ScheduleOnTodayNotTimeYet) { LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(999); LightScheduler_Wakeup(); LONGS_EQUAL(LIGHT_NA, FakeLightController_getState(3));
<<interface>>
Time Service + getDay() + getTimeOfDay() + setPeriodicAlarm()

<<implements>>
<<interface>>
} TEST(LightScheduler, ScheduleOnTodayItsTime) { LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1000); FakeTimeSource_SetMinute(1000); LightScheduler_Wakeup(); LONGS_EQUAL(LIGHT_ON, FakeLightController_getState(3)); }
<<implements>>
Light Controller Spy
Fake Time Service
41
41
42
42
Light Controller Interface

#ifndef D_LightController_H #define D_LightController_H enum { MAX_LIGHTS = 32 }; void void void void LightController_Create(void); LightController_Destroy(); LightController_On(int id); LightController_Off(int id); // D_LightController_H
Fake Light Controller Interface

#ifndef D_FakeLightController_H #define D_FakeLightController_H #include "LightController.h" typedef enum { NO_ID = -1, UNKNOWN_STATE = -1, LIGHT_OFF = 0, LIGHT_ON = 1} LightState; int FakeLightController_getLastId(void); int FakeLightController_getLastState(void); int FakeLightController_getState(int id); #endif // D_FakeLightController_H
#endif
43
43
44
44
Fake Light Controller Implementation

#include "FakeLightController.h" #include "memory.h" static int lastId; static int lastLevel; void LightController_Create(void) { lastId = NO_ID; lastLevel = UNKNOWN_STATE; } void LightController_Destroy() { }
Fake Light Controller Implementation

int FakeLightController_getLastId(void) { return lastId; } int FakeLightController_getLastState(void) { return lastLevel; } void LightController_On(int id) { lastId = id; lastLevel = LIGHT_ON; } void LightController_Off(int id) { lastId = id; lastLevel = LIGHT_OFF; }
45
45
46
46

TEST(LightScheduler, NoScheduleNothingHappens) { LightScheduler_Wakeup(); LONGS_EQUAL(NO_ID, FakeLightController_getLastId()); LONGS_EQUAL(UNKNOWN_STATE, FakeLightController_getLastState()); } TEST(LightScheduler, ScheduleOnTodayNotTimeYet) { LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1200); FakeTimeService_SetMinute(1199); LightScheduler_Wakeup(); LONGS_EQUAL(NO_ID, FakeLightController_getLastId()); LONGS_EQUAL(UNKNOWN_STATE, FakeLightController_getLastState()); }

TEST(LightScheduler, ScheduleWeekdayItsSunday) { LightScheduler_ScheduleTurnOn(3, EVERYDAY, 1200); FakeTimeService_SetDay(SUNDAY); FakeTimeService_SetMinute(1200); LightScheduler_Wakeup(); LONGS_EQUAL(3, FakeLightController_getLastId()); LONGS_EQUAL(LIGHT_ON, FakeLightController_getLastState()); }
47
47
48
48
When must you use fakes?

When the Code under test cannot be fully tested with the real collaborators. Examples:
When manual verification is needed (Printed output, LEDs) When the results change (Time, random events) When failures need to be simulated (Network down) When hardware is involved (LEDs, USB, Sensors, Motors, IO pins, Flash...) Operating system calls (RTOS)
Breaking Dependencies with Function Pointers
49
49
50
50
Loosening Compile-Time and Run-Time Dependencies

Problem: an existing C function needs to be stubbed out for some tests...
//From the .h file ... int RandomMinuteGenerator_Get(); ...
Convert Direct Dependency to Function Pointer

Modify the function declaration in the header file
Add extern Add (*)
//From the .c file ... int RandomMinuteGenerator_Get() { ...the implementation... } ...
//From the .h file ... extern void (*RandomMinuteGenerator_Get)(); ...
51
51
52
52
Convert Direct Dependency to Function Pointer

Modify the function definition in the .c file
Change function name by appending _impl Hide the _impl by making it static
No Changes are Needed to Callers of the Function Pointer

Calling code does not change. No pointer dereferencing is needed
//from some production code... ... RandomMinuteGenerator_Get(); ...
Add defining instance of the function pointer and initialize the pointer with the the _impl
//From the .c file ... static int RandomMinuteGenerator_Get_impl() { ...the implementation... } int (*RandomMinuteGenerator_Get)() = RandomMinuteGenerator_Get_impl;
53
53
54
54
Runtime Substitution Test setup and teardown

Override the function by assigning the test version of the function during setup Dont forget to restore the original
void setup() { RandomMinuteGenerator_Get = RandomMinuteGenerator_Get_fake; } void teardown() { RandomMinuteGenerator_Get = RandomMinuteGenerator_Get_impl; }
Use CppUTests UT_PTR_SET
void setup() { UT_PTR_SET(RandomMinuteGenerator_Get, RandomMinuteGenerator_Get_fake); } void teardown() { }
55
55
56
56
Message Flow for Flash Memory Block Erase with Error

FlashDriver FlashDevice
Self Verifying Mock Objects
Flash_Program(offset, data)
IOWrite(CommandRegister, 0x40)
IOWrite(offset, data)
IORead(StatusRegister) b7 == 0 IORead(StatusRegister) b7 == 1, other bits == 0 IORead(offset) FlashSuccess data
Repeats while b7 == 0
57
57
58
58
Flash Program Flow Chart How Many Tests are Needed?

Start b3 == 0 Program Command Write 0x40 to 0x0
Mock Object
Problem - Complex collaborator interactions cannot be captured with a simple spy. Solution - Mock Object
A Mock Object is a Test Double that verifies that the code being tested interacts with its collaborator properly. The test tells the mock
The expected calls In the expected order What to return.
NO
Vpp Error
YES
Write data to address
b4 == 0
NO
Program Error
How do you test these errors in the target?
YES
Read status register Protected Block Error
b1 == 0
NO
NO YES
b7 == 1 Start/Stop Clear status Write 0xFF to 0x0
YES
59
59
60
60
Flash Driver Test

TEST(Flash, ProgramSucceedsReadyImmediately1) { int result = 0; MockIO_Expect_IOWrite(0, 0x40); MockIO_Expect_IOWrite(0x1000, 0xBEEF); MockIO_Expect_IORead(0, 1<<7); MockIO_Expect_IORead(0x1000, 0xBEEF); result = Flash_Program(0x1000, 0xBEEF); LONGS_EQUAL(0, result); MockIO_Assert_AllExpectationsMet(); }
void void void void void
Mock Flash Read/Write

//MockIO.h #include "IOReadWrite.h" MockIO_Create(int maxExpectations); MockIO_Destroy(); MockIO_Expect_IOWrite(ioAddress_t offset, uint16_t data); MockIO_Expect_IORead(ioAddress_t offset, uint16_t data); MockIO_Assert_AllExpectationsMet();
//IOReadWrite.h #include <stdint.h> typedef uint32_t ioAddress_t; typedef uint16_t ioData_t; ioData_t IORead(ioAddress_t offset ); void IOWrite(ioAddress_t offset, ioData_t data);
61
61
62
62
Mock Flash Read/Write

//MockIO.c typedef struct Expectation { int kind; ioAddress_t addr; ioData_t value; } Expectation; static static static static static Expectation* expectations = 0; int setExpectationCount; int getExpectationCount; int maxExpectationCount; int failureAlreadyReported = 0;
Capturing Flash Read/Write Expectations

void MockIO_Expect_IOWrite(ioAddress_t addr, ioAddress_t value) { failWhenNoMoreRoomForExpectations(report_too_many_write_expectations); recordExpectation(FLASH_WRITE, addr, value); } void MockIO_Expect_IORead(ioAddress_t addr, ioAddress_t value) { failWhenNoMoreRoomForExpectations(report_too_many_read_expectations); recordExpectation(FLASH_READ, addr, value); }
//Snip void MockIO_Create(int maxExpectations) { expectations = malloc(sizeof(Expectation)*maxExpectations); memset(expectations, 0, sizeof(expectations)); setExpectationCount = 0; getExpectationCount = 0; maxExpectationCount = maxExpectations; failureAlreadyReported = 0; }
63
63
64
64
Mock Flash Read

ioData_t IORead(ioAddress_t addr) { setExpectedAndActual(addr, -1); failWhenNotInitialized(); failWhenNoUnusedExpectations(report_read_but_out_of_expectations); failWhen(expectationIsNot(FLASH_READ), report_expect_write_was_read); failWhen(expectedAddressIsNot(addr), report_read_wrong_address); return expectations[getExpectationCount++].value; }
Mock Flash Write

void IOWrite(ioAddress_t addr, ioData_t value) { setExpectedAndActual(addr, value); failWhenNotInitialized(); failWhenNoUnusedExpectations(report_write_but_out_of_expectations); failWhen(expectationIsNot(FLASH_WRITE), report_expect_read_was_write); failWhen(expectedAddressIsNot(addr), report_write_does_not_match); failWhen(expectedDataIsNot(value), report_write_does_not_match); getExpectationCount++; }
65
65
66
66
All Expectations Must Be Met

void MockIO_Assert_AllExpectationsMet() { if (failureAlreadyReported) return; failWhenNotAllExpectationsUsed(); }
Minimize DOH!
Debug On Hardware
67
67
68
68
On-line
Test harnesses
[CPPTEST] www.sourceforge.org, project CppUTest [FITNESSE] www.fitnesse.org
The End
Groups
http://groups.yahoo.com/group/testdrivendevelopment http://groups.yahoo.com/group/AgileEmbedded
69
69
70
70
See Embedded TDD and Related Blogs and Papers

http://www.renaissanceSoftware.net http://www.renaissancesoftware.net/blog/
Embedded TDD Zune Bug: Test Driven Bug Fix Learning Tests are Free! TDD as a Design Rot Prevention System Crashing Your Way to Great Legacy C Tests TDD and the Big Framework Part Bug Fixes and TDD Physics of Test Driven Development Tests vs. Short Term Cache Between Your Ears Embedded Systems Conference FAQ I miss constructors Who says you cant test drive a device driver? Why are You Still Using C? Planing Poker Agile Embedded Software Development (ESC) Launching Extreme Programming at a Process Intensive Company (IEEE) Test Driven Development for Embedded Software Progress Before Hardware Agile Times - Containing Progress Before Hardware Test-Driven Development for Embedded C++ Programmers
References and Sources

[RCM] Robert C. Martin, Clean Code, 2008 [SLAD] Craig Larman and Bas Voode, Scaling Lean & Agile Development [WELC] Michael Feathers, Working Effectively with Legacy Code [REF] Martin Fowler, Refactoring [TDD] Kent Beck, Test-Driven Development, 2003 [xUNIT] Gerard Meszaros - xUnit Testing Patterns, 2008 [MOCK] Tim Mackinnon and Steve Freeman and Philip Craig, EndoTesting: Unit Testing with Mock Objects, 2001 [TD] Lasse Koskela, Test Driven, 2007 [LESSONS] Bret Pettichord Cem Kaner, James Bach. Lessons Learned in Software Testing: A Context-Driven Approach. John Wiley & Sons, New York, NY, 2002.
71
71
72
72
Available now in Beta

http://pragprog.com/titles/jgade/
In print early 2011
73
73

Test Driven Development For Embedded C: The Physics of Debug Later Programming (DLP)

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Test Driven Development For Embedded C: The Physics of Debug Later Programming (DLP)

Transféré par

Droits d'auteur :

Formats disponibles

The Physics of Debug Later Programming (DLP)

Test Driven Development for Embedded C

Mistake made (bug injection)

Bug discovery Bug found Bug xed

Presented at Agile China October 14, 2010 Test Driven Development

Copyright 2008-2010 James W. Grenning All Rights Reserved.

Monday, October 18, 2010

Monday, October 18, 2010

The Physics of Test Driven Development

TDD Micro Cycle

Presented at Agile China October 14, 2010 Test Driven Development

Monday, October 18, 2010

Monday, October 18, 2010

TDD State Machine in C/C++

Test Driving a Circular Buffer (FIFO)

Choose a test No more tests All tests pass

Write the test

Make the test pass

New test fails

Make the test link

Presented at Agile China October 14, 2010 Test Driven Development

Presented at Agile China October 14, 2010 Test Driven Development

Monday, October 18, 2010

Monday, October 18, 2010

Create a Test-List (avoid analysis paralysis)

Set up a Test Fixture

Presented at Agile China October 14, 2010 Test Driven Development

Presented at Agile China October 14, 2010 Test Driven Development

Monday, October 18, 2010

Monday, October 18, 2010

Make a bare Skeleton of the Production Code

Make a bare Skeleton of the Production Code

Presented at Agile China October 14, 2010 Test Driven Development

Presented at Agile China October 14, 2010 Test Driven Development

Monday, October 18, 2010

Monday, October 18, 2010

Choose a test No more tests All tests pass

Write the test

Make the test pass

Presented at Agile China October 14, 2010 Test Driven Development

Presented at Agile China October 14, 2010 Test Driven Development

Monday, October 18, 2010

Monday, October 18, 2010

Production Code is Grown One Test at a Time

Choose a test No more tests

Write the test

Code without new feature

Code with new tested feature

All tests pass

All tests pass

Make the test pass

New test fails

Make the test link

Presented at Agile China October 14, 2010 Test Driven Development

Presented at Agile China October 14, 2010 Test Driven Development

Monday, October 18, 2010

Monday, October 18, 2010

Make the Test Compile

Choose a test No more tests All tests pass

Write the test

Make the test pass

New test fails

Make the test link