Vous êtes sur la page 1sur 39

ICONIX PROCESS ROADMAPS

DOUG ROSENBERG

FINGERPRESSLTD
LONDON

Findmoregreatbooksat:
www.fingerpress.co.uk

This PDF contains Appendix A of ICONIX Process Roadmaps by Doug Rosenberg.



Find out more at:
http://software.fingerpress.co.uk/iconixprocessroadmaps.html

Or buy the book direct:


www.iconix.org/product.sc?productId=92&categoryId=18

CopyrightDougRosenberg2011.WeencourageyoutoforwardthisPDFaroundandredistributeitfor
noncommercialuse.WejustaskthatyoudontmodifythePDF;andifyoulikeit,tobuyacopyofthe
book!

APPENDIX A
Bonus Roadmap: Design Driven Testing for Systems
ByDougRosenbergmanythankstoMichaelSieversattheJetPropulsionLaboratory
(Pasadena,CA)forhisvaluableinputs.

DesignDrivenTesting(DDT)wasfirstoutlinedinthebookUseCaseDrivenObjectModelingwith
UML:TheoryandPractice(byDougRosenbergandMattStephens),andthendescribedindetail
inDesignDrivenTesting:TestSmarter,NotHarderbythesameauthors.AlthoughDDTcanbe
successfullyusedinconjunctionwithTestDrivenDevelopment(TDD),itreallyfollowsthe
oppositedevelopmentphilosophy:startingwithadesign(andrequirementsandusecases)and
drivingtestsfromthem.
DDTisahighlymethodicalapproachtotesting,allowingyoutoknowwhenyouvefinished
i.e.whenyouvewrittenenoughteststocoverthedesignandtherequirements.Ithelpsyouto
zoominandwritealgorithmicteststocoverintensiveormissioncriticalsectionsofcode,and
toknowwhenitssafetozoomoutandwritefewertestsforboilerplateorlesscriticalcode.
ConsequentlyyoutendtoendupwithfewertestsoverallthanwithTDDbutcomparatively
moretestscenariospertestcase.
InthisAppendix,DougextendstheconceptofDDTforhardware/softwaresystems,allowing
SysMLbaseddesignstobetestedinahighlyrigorous,systematicway.ItsstillDesignDriven
Testing,butnowthedesignelementsthatneedtobetestedincludeallofthefourpillarsof
SysML,whereasDDTforsoftwarefocusesontestingbehavior.
Doug(notbeinganactualrocketscientist)wouldliketoacknowledgesomeinformalbutvery
valuableandentertainingdiscussionswithDr.MichaelSieversofNASAsJetPropulsion
Laboratory(JPL)regardinghowrealrocketscientistswouldapproachtheproblemofdetecting
potholesfromouterspaceandhowtheywouldtesttheirdesigns.

APPENDIXA

Introduction
ThisAppendixdiscussesanintegrated,modelbaseddesignapproachbasedonanextensionof
DesignDrivenTesting(DDT)54thatiscompatiblewithValidation&Verification(V&V)activitiesat
alldesignlevels.(WedefineV&VinthesidebaronPage216).Modelsarevalidatedateachstep
andthevalidatedmodelsarethenusedforverificationtests.Essentiallythedesignisthemodel
andthemodelisthedesign.Atsomelevelofdetailthemodelisphysicalhardwareand
executablesoftwareandtheV&Vprocessusesfamiliarunit,bench,andintegrationtests.
WhileDDTisrelatedtosoftwaresimulationscurrentlyusedintheindustry,itdiffersinat
leastonekeyaspect:typicallysoftwaresimulationsarebuiltbyindependentteamsfrom
independentrequirements.Thereisalwaysaquestionofhowclosethesoftwaresimulationis
totherealsystem.Inourapproach,thereisonesystemmodelwhichmaybeviewedindifferent
waysandatdifferentlevelsofdetailassuitsausersneeds.Moreover,aswillbeseeninthe
DesignDrivenTestingsection,ourapproachguidessmartselectionoftestcaseswhichsaves
costandschedule.Thisisparticularlyimportantbecausewhilesystemshavebecomemore
complex,thetimeallocatedfortestinghasnotchanged.Wemustnowtestmore,butinthe
sameamountoftimeassimplersystems.Thiscanonlybecomepracticalbyimprovingtest
efficiency.
DDTisintimatelylinkedtomodern,modelbasedsystemsengineeringusingtheSysML
paradigm.LikeSysMLwhichestablishesasingle,commonsourcefordesigntruth,DDTprovides
asingle,consistentsourceofV&Vtruthandestablishesclearlinesofresponsibility.Equally
importantisthatDDTisintegratedwiththeiterativedesignprocessassuringthateachlevelof
designisconsistentwithitsparentandchildren.

54

DesignDrivenTesting:TestSmarter,NotHarderbyMattStephensandDougRosenberg(Apress,2010).

210

ROADMAP:DDT/SYSTEMS

ROCKET SCIENCE:
Why Use a Space Mission to Illustrate DDT/Systems?
A space mission is a system of interacting systems that implement a set of behaviors defined by
operational concepts. Operations concepts describe all phases of the mission from groundbased testing through pre-launch, launch, separation, commissioning, nominal operations and
eventually disposal or decommissioning. Each of these use cases comprises complex actions,
alternate paths and exceptional conditions. When a typical mission costs taxpayers hundreds of
millions to billions of dollars, it is crucial that these systems are validated and verified from the
highest conceptual levels down to the detailed code that defines the functions in an FPGA. The
cost of finding and fixing problems escalates by orders of magnitude as the mission progresses
through its design and deployment stages hence, systematic and early Validation and Verification
(V&V) are central aspects of all space programs.

A Model System Design


UsingSysML,wedescribethedesignofafanciful,yetsomewhatrealisticspacecraftmissionfor
arunningexampleinthispaper.ThenextsectiondiscussesthesystemV&Vprocessasan
extensiontoDDT.WethenillustratethesystemV&Vmethodappliedtoourspacecraftdesign.
Ourmissiongoalisasystemthatimagesthecitystreetsitpassesoverwithasufficiently
largetelescopethatitcanresolvelargestreetimperfections.Wecallourspacecraft:Persistent
OpenTrafficHazardObstacles,LargeandElongated(POTHOLE)becauseallspacecraftmust
haveaclevername.POTHOLEsendsimagestothegroundwheretheyareprocessedandnewor
growingpotholescanbereportedtolocalmaintenancebureaus.
FigureA1showsaconceptualdiagramofPOTHOLEspacecraft.Majorvisiblecomponents
arethesolararrayingreen,telescopeindarkblue,sunshadeinlightblue,downlinkantennain
silverandtheBusinyellow.TheBusconsistsofthefunctionsneededtomaintainthevehicle
andcommunicatewiththeground.Thetelescopeandrelatedimagingfunctionsarecollectively
calledtheInstrument.ThemissionblockdiagraminFigure3showsthatthePOTHOLE
spacecraftisacompositionoftheBusandInstrumentsystems.Themissionalsorequiresa
GroundStationforcontrollingthespacecraftandacceptingandprocessingimages.

211

APPENDIXA

FigureA1.POTHOLESpacecraft
bdd [SysML Block Definition] Block Diagrams [POTHOLE Context]
Context
POTHOLE Mission

System of Systems
POTHOLE Spacecraft

System
Bus

System of Systems
Ground Station

System
Instrument

FigureA2.POTHOLEMissionBlockDiagram

Wenextdefineafewkeybusinessrequirementsconstitutingthecontractbetweenthe
customerandtheimplementer.Businessrulesdefinetheconstraintsandfunctionsthatmustbe
deliveredbytheimplementerandarethebasisforthecustomersacceptancecriteria.

212

ROADMAP:DDT/SYSTEMS

ROCKET SCIENCE: V&V Challenges


Several challenges are inherent in current V&V methodologies.

There are multiple layers of requirements (refer to Figure A-3) mission, systems,
subsystems, assemblies, units and components.

Mission requirements are often driven by capabilities. Instead of asking, What do you
want us to build? we are often asked, What do you have and how can we use it?

A design is V&V'd through tests, analysis, demonstration and inspection.

Keeping track of who does what and how lower level V&V rolls up into higher levels is not
managed very well.

Tests at higher levels wait until tests at lower levels have completed. So problems related
to validating higher level functions arent recognized until later in the development cycle
when it is more expensive and time consuming to fix them.

Models (e.g. aerothermal and reliability models) are often used for various aspects of
V&V. These models are not integrated and do not necessarily provide an easy path for
incorporating effects in one model into another.

213

APPENDIXA

Context
Mission

1..*
Level 2
Proj ect

1..
Level 3
1..*
Payload System

Level 3
Spacecraft Bus

1..*

Level 3
Ground Systems

1..*
Level 4
Instrument
Subsystems

.
.
.

Level 4
Spacecraft
Subsystems

1..*
Level 5
Assemblies

1..*
Level 6
Boards

1..*
Level 4
Ground
Subsystems

.
.
.

Level 7
Components

FigureA3.TypicalspacesystemdesignhierarchyincludesuptosevenlevelsofspecificationandV&V

214

ROADMAP:DDT/SYSTEMS

POTHOLE Business Requirements


FigureA4showsthetenrequirements,asmodeledinEnterpriseArchitect.Mission
requirementsderivefromtheopportunitiesandlimitationsinsystemuse,environmental
conditions,functionalcriticality,andtheothersystemsthecustomeremploysinsupportofthe
system.

req Functional Requirements


The spacecraft shall store images for
at least one hour

The telescope shall have an


adjustable aim point

The spacecraft shall transmit images


to the ground in real-time when a
ground station is available

Each image pixel shall have ground


resolution of no more than 0.1 meters

The spacecraft shall retransmit stored


images when requested by ground
command

Images shall be transmitted to the


ground at 150 Mbps

Retransmitted images shall be


interleaved with real-time images
according to a ground configurable
downlink schedule

Images shall be downlinked in groups


of 16 using CCSDS format

The spacecraft shall store images in a


circular buffer over-writing older
images with newer images

The field of view shall be at least 1.0


Km x 1.0 Km

FigureA4.POTHOLEMissionRequirements

215

APPENDIXA

ROCKET SCIENCE: Validation and Verification (V&V)


We define validation as the processes performed to assure that we are building the right thing
for specified needs. If a user wants to measure magnetic fields around a planet then our design
must include a magnetometer. We should not build our spacecraft with something else unless we
have discussed it with our customers and they agree that our alternative is acceptable. In a space
flight mission, validation focuses on certifying that the system meets a set of mission objectives
documented in operations concepts and mission plans while operating in flight-like conditions.
Validation is often subjective because it asks whether the system is able to accomplish its
mission goals.
Verification is defined as the processes performed that assure we have built the system right.
Verification determines that the system has the right components to do its job and that at each
level of design those components satisfy a set of requirements. Collectively, requirements are a
formal contract between the customers and implementers of a system. Unlike validation,
verification is associated with formal, objective tests that have either a pass or fail result.
Defining comprehensive yet efficient V&V for complex systems is a difficult problem and is often
made more difficult by fuzzy requirements, incomplete definition of desired behaviors, multiple
teams, and unclear responsibilities. Inefficient V&V resulting in unnecessary overlaps or
repeated tests adds costs time and money but is far less dangerous than incomplete or missing
tests. Yet testing all combinations and permutations of a complex system is impossible both for
the size of the problem and for the lack of visibility into internal operations as systems are
integrated.
Typically, a space system is designed top-down and bottom-up. Mission planners think through
the type of data they want to collect which drives lower-level allocations and requirements. At the
same time, designers are often looking at what hardware and software they can reuse even
though they may not yet have a complete understanding of what functions they must provide.
Often bottom-up selection and top-down mission analyses and decompositions meet in a
integration and test facility where disconnects are frequently found.

TheBusandInstrumentstructuresarefurtherdetailedintheblockdefinitiondiagrams
(BDDs)showninFiguresA5andA6respectively.
TherearesixmajorsubsystemsthatformtheBus:Telecomwhichconsistsofradiosfor
groundcommunication.GuidanceandNavigationisresponsibleforknowwherethevehicleis
relativetothegroundandformaintainingpointingsothatthetelescopepointstotheEarths
surfacewhilethesolararraypointstothesun.Avionicshousestheflightcomputersonwhich
flightsoftwareexecutes.ThePropulsionsubsystemincludesthethrusters,valves,and
propulsiontanks.TemperaturesaremaintainedbytheThermalsubsystemandthePower
subsystemmanageselectricalpower.

216

ROADMAP:DDT/SYSTEMS

Lunchtime Conversation with a Rocket Scientist


We discussed our fictitious mission requirements with a friendly rocket scientist over lunch. The
conversation went something like this:
Well, the first requirement tells us how long we have to keep images onboard. Once we know the
image capture rate and image size, we can place a lower bound on how much storage we need.
The seventh requirement tells us that our field of view (FOV) is 1.0 Km x 1.0 Km. Assuming we
want to take an image in every 1.0 Km x 1.0 Km patch, then we have to take an image every time
the ground track of the spacecraft moves 1.0 Km which can be computed from the spacecraft
orbit (inclination and altitude).
Pizza arrives
For example, at an altitude of 20,000 Km (GPS altitude), we need to image roughly every 100
milliseconds at the equator. Requirement 9 tells us that our image sensor is 10K pixels * 10K
pixels = 100M pixels. At 12 bits per pixel, each image is 1.2 Gb. Storing images for an hour
translates into around 4.3Tb of onboard storage a large number, but still within the realm of
current solid state recorder technology.
Time for a second slice
In a real design, we would be quite concerned about the practicalities of building a telescope and
optics with the required precision. However, for our purposes, we are only interested in data
collection, transport, and processing. Our back-of-the-envelope analysis gives us interface rates,
storage requirements, image sensor size, and hints of needed processing throughput. Were now
ready to define our domain model and use cases.

TheInstrumentcomprisesDataStorageforholdingimageswhengroundstationsarenotin
viewandforretransmission,aCamerathattakesimages,anInstrumentComputerthat
managestheinstrumentandimagecollectionfunctions,andtheTelescopewhichconsistsof
mirrorsandoptics.
FigureA7isanIBDthatshowsinterfacesbetweentheCDHprocessor,LocalEngineeringUnit
(LEU)usedforanalogdatasampling,NonVolatileMemory(NVM)usedtostorecriticalvariables
andsoftwareimages,andtheCommunicationsInterface(CommInterface)thatsendsand
receivesmessagesfromthetelecomradios.WeusethehighlightedNVMtodefineinterface
testsinthevalidationsectionbelow.

217

APPENDIXA
System
Bus

Subsystem
Guidance and Nav

Subsystem
Telecom

Subsystem
Av ionics

Subsystem
Pow er

block
FSW

block
CDH

block
Processor

Subsystem
Thermal

Subsystem
Prop

block
EngineeringDataCollection

block
TelecomInterface

block
Non-VolatileMemory

FigureA5.POTHOLESpacecraftBusBDD
System
Instrument

System
Instrument
Computer

Subsystem
DataStore

Subsystem
Camera

Subsystem
Telescope

FigureA6.POTHOLEInstrumentBDD

218

ROADMAP:DDT/SYSTEMS
: CDH
Board
Processor

Board
flowPort
EngrCtl

flowPort EngrIn

LEU
flowPort CtlIn

flowPort EngrOut

AnalogIn

DigitalIn

DigitalIn

Board

flowPort
FIFOCtlOut

NVM
flowPort
FIFOCtlIn

flowPort
FIFOStatusIn

AnalogIn

flowPort
InstrDataIn

flowPort
FIFOStatusOut

flowPort EngrOut

flowPort
XbandDL

flowPort
InstrDataIn

flowPort
XbandDL

Board
flowPort EngrDL

CommInterface

flowPort CmdIn

flowPort CmdUL

flowPort
SbandDL

flowPort
SbandUL

flowPort
SbandTlm

flowPort
SbandCmd

FigureA7.ExamplesofCDHinterfaces

EarlierwemadeacasefortopdownandintegratedV&V,yethereweareshowing
significantdetailforourhypotheticalspacecraft.Naturalquestionsmightbe:howdidwecome
tothispartitioning,howdoweknowitisright,andcouldtherebeabetterdecompositionthat
doeswhatweneed?Thesimpleansweristhatthesearebasicfunctionsofa3axisstabilized
spacecraftthatcancaptureimagesandsendthemtotheground.Withinthisstructure
designersarefreetoallocatespecificbehaviorsandrequirements.Moreover,wearefreeto
modifythestructureifourV&Vprocessdeterminesthatitisnotadequate.
WhydistinguishbetweenBus,andInstrument,couldntthesebejustpartofaFlight
Vehicleandavoidtheintermediatepartition?TheBusandInstrumentpartitionsareneither
requirednornecessarybutreflecttherealitythatdifferentdevelopmentgroupsareusually
responsibleforprovidingBusandInstrumentfunctions.Thesegroupsindependentlydesignand
testtheirportionofthespacecraftandtheneventuallythosesystemsarebroughttogetherand
tested.ThishighlightsthepointmadeearlierthattraditionalV&Vwaitsforverificationoflower
levelfunctionsbeforetestinghigherlevelfunctions.Electricalandfunctionalinterface
documentsdefinehowthisshouldhappenandwheneverythingisproperlyspecifiedand
implementedthenallworksout.But,thatsnotalwaysthecase.

219

APPENDIXA

POTHOLE Domain and Use Case Models


WehaveagoodideaofwhatbehaviorsareneededinthePOTHOLEspacecraftforcollectingand
transmittingimagestothegroundforpotholeanalysis.Thosebehaviorsalsohelpdefineour
domainmodel.Domainmodelsandusecasesaretightlycoupledbecauseusecasesreferto
domainelementsanddomainelementsmustbepresentinatleastoneusecase.Consequently,
domainandusecasemodelinteractionsarethebasisofinitialmodelvalidationbecauseeach
modelcheckstheother.Neithermodelcanbeconsideredcompleteorcorrectunlesseachis
consistentwithandsupportiveoftheother.
Fromtherequirementsandmissiongoal,weknowthatourdomainmodelmusthavean
imageelement.Wealsoknowthatindividualimagesarepackagedonthespacecraftpriorto
downlink,soourdomainneedsaconceptofimagecollectionwhichiscomposedofimages.
Similarly,weknowthatwewillneedsomesortofimagequeueinourdomainmodelthat
feedsimagestoanalysissoftwarebecauseimagescannotbeprocessedconcurrently.
ThedomainmodelinFigureA8andthecorrespondingusecasemodelinFigureA9result
fromiteratingdomainelementsandusecasedescriptions.FigureA8populatesmodelelements
withconceptualfunctiondefinitions.Thesemayneedrevisionasmoredetailisknown,butat
thispoint,theelementsandfunctionsareconsistentwiththeusecasesinFigureA9andthe
missionrequirements.
ThedomainmodelincludestheInstrumentthatproducesanImageCollectionwhichis
createdasanaggregationofindividualImages.TheImageCollectionhasanassociationwiththe
GroundStationthroughtheTelecomsubsystem.TheGroundStationmaintainsanImageQueue
foranalyzingandannotatingtheImages.ImagesannotatedwithPotholesaresenttothe
Subscriberbythegroundstation.

220

ROADMAP:DDT/SYSTEMS
class Domain Model

Image Collection

Telecom

produces
Instrument

sends images to

Ground Station

Image Queue

sends annotated image to

Image

analyzes

produces
Subscriber

Annotated Image

Image Analyzer

detects

Pothole

FigureA8.POTHOLEImagingDomainDiagram

ThemissionlevelusecasesforourreferencefunctionshowninFigureA9complementthe
domainmodelandreflectourunderstandingoftheimagingcollectionandanalysisprocess.The
spacecraftmustachieveandmaintainthecorrectorbit(MaintainOrbit).Theinstrumentmust
beaimedatapointonthegroundpointpriortolookingforpotholes(AimInstrument).
Spacecrafthealthmustbemaintainedthroughoutthemission(MaintainSpacecraftHealth)
includingdealingwithspacecraftemergencies(enteringsafemodewhichkeepsthespacecraft
powerpositiveandincommunicationwiththeground).Whencommanded,wecaptureand
processimages(LookforPotholes)andifimagesarecorruptedintransmission,thegroundmay
requestthattheyberetransmitted(RetransmitImages).

221

APPENDIXA

Aim Instrument

precedes

Maintain Orbit

Retransmit Images

precedes
precedes

invokes

Ground Station

Look for Potholes

notify

Subscriber

invokes

Ongoing

Maintain Spacecraft
Health

FigureA9.POTHOLEMissionLevelUseCases

Becauseourprimarymissionispotholedetection,wefocusontheLookforPotholes
scenariothatcollectsimageswithsufficientresolutiontodetectlargepotholesandsendsthe
imagestothegroundforprocessing.Moreprecisely:
Basic Course:

Ground Station sends a command to Spacecraft to collect Images.


Instrument collects Images, resulting in an Image Collection
Telecom sends the Image Collection to the Ground Station
Ground Station adds Images to Image Queue
Image Analyzer gets Image from Image Queue
Image Analyzer detects Potholes and produces Annotated Image showing Pothole
locations
Image Analyzer sends Annotated Image to Subscriber

Alternate Courses:

Ground Station sends a command to Spacecraft to retransmit Image Collection:


Invoke Retransmit Images
Spacecraft enters safemode: invoke Maintain Spacecraft Health
Ground Station sends a command to Spacecraft to stop collecting images: invoke
Maintain Spacecraft Health

222

ROADMAP:DDT/SYSTEMS
AfewthingsareworthpointingoutinthisICONIXprocessstyleusecase55.Thescenariois
writtenpreciselyusingnouns(showninbold)thatrepresentobjectsinthedomainorcontext
models.Verbsapplyactionsthattransformonenounintoanothernounortransportanounto
anotheraction.Bolditalicizedtextreferstousecasesintheusecasediagram.
Comparingthenounsandverbsinourusecasewiththedomainmodelelementsgivesus
confidencethatourmodelsareconsistent.Butwedontyetknowthatwedefinedtheusecase
scenariocorrectlyandsowecantyetclaimthatwehavevalidatedourinitialconcepts.Ournext
stepcreatesaconceptualdesignintheformofarobustnessdiagram56thatfacilitatesadesign
walkthroughforvalidatingtheusecases.Wewillseelaterthattherobustnessdiagramisalso
anessentialtoolfordefiningtestcases.

POTHOLE Conceptual Design


FigureA10showsarobustnessdiagramfortheLookforPotholesusecase.Inarobustness
diagram,thelogicalandconceptualstructuresareanalyzedensuringthattheusecasesit
modelsaresufficientlyrobustfortheintendedusagebehaviorsandrequirements.Robustness
diagramsareannotatedwithrequirementsestablishingcompletenessandconsistencydesign.
Missing,incorrect,andincompleterequirementsbecomereadilyapparent.
Arobustnessdiagramcomprises:actors(peopleorexternalsystemsthatinteractwiththe
systemunderconsideration,boundaries(interfacesthatactorsinteractwith),controllers(the
verbsintheusecasestatements),andentities(elementsfromthedomainorcontextmodels).
Afewsimpleinteractionrulesdefinetheconstructionofrobustnessdiagrams:

Entitiesandcontrollersmayinteract.

Actorsandboundariesmayinteract.

Boundariesandcontrollersmayinteract.

Entitiescannotdirectlyinteractwithotherentities.

InFigureA10,circleswitharrowsrepresentcontrollers,underlinedcirclesareentitiesand
circlesattachedtoverticallinesareboundaries.Requirementsareassociatedwithcontrollers.
EverycontrollermusthaveatleastonerequirementsoFigureA10isnotcomplete.Moreover,

55

UseCaseDrivenObjectModelingwithUML:TheoryandPracticebyDougRosenbergandMattStephens.

56

IvarJacobson,et.al.(1992),ObjectOrientedSoftwareEngineeringAUseCaseDrivenApproach,AddisonWesley.

223

APPENDIXA
everyrequirementmustbeassociatedwithacontrollerandmissingassociationsmeaneither
omissionsintherobustnessdiagramorunnecessaryrequirements.
WecanwalkthroughtheLookforPotholesstepsandfollowtheflowintherobustness
diagram.Inconsistenciesarereconciledbyalteringrobustnesselementstoreflecttheusecases
itrepresentsand/orchangingtheusecasestomirrortherobustnessdiagram.Sinceupdatesto
usecasesmayalsocausedomainmodelrevisions,wheniterationsarecomplete,the
robustness,usecase,anddomainmodelsareselfconsistent.

Basic Course:
Ground Station sends a command to Spaceraft to
collect images.
Instrument collects Images, resulting in an Image
Collection
Telecom sends the Image Collection to the Ground
Station
Ground Station adds images to Image Queue
Image Analyzer gets Image from Image Queue
Image Analyzer detects Potholes and produces
Annotated Image showing pothole locations
Image Analyzer sends Annotated Image to
Subscriber
Alternate Courses:
1.
2.
3.

The telescope shall have an


adjustable aim point

Each image pixel shall have ground


resolution of no more than 0.1 meters
(from Functional Requirements)

(from Functional Requirements)

The spacecraft shall store images in a


circular buffer over-writing older
images with newer images
(from Functional Requirements)

Instrument
The spacecraft shall retransmit stored
images when requested by ground
command

capture images

(from Functional Requirements)

Ground Station sends a command to Spacecraft


to retransmit images: Invoke Retransmit Images
Spacecraft enters safemode: invoke Maintain
Spacecraft Health
Ground Station sends a command to Spacecraft
to stop collecting images: invoke Maintain
Spacecraft Health

Image Collection
Retransmit Images

Pothole
send images
Telecom

detect pothole

Ground Station

Subscriber

send annotated image


Image Analyzer

annotate image
Annotated Image

add image

get image

Image Queue
Image

FigureA10.POTHOLERobustnessDiagramwithafewexamplerequirements

FigureA11showsaportionofthesequencediagramassociatedwiththeLookforPotholes
usecase.Behaviorsareallocatedtothesystemelementsdefinedinthedomainmodeland
blockdiagrams.Eachbehaviormustbeunittested,asshowninFigureA25.
TheExtendedDomainModelinFigureA12showsanobjectorientedallocationofbehavior
tosystemelements.Allocationssatisfyallbehaviorsspecifiedintheusecaseasshowninthe
sequencediagram.

224

ROADMAP:DDT/SYSTEMS

Ground Station

Basic Course:
Ground Station sends a
command to Spaceraft to
collect images.
Instrument collects Images,
resulting in an Image
Collection
Telecom sends the Image
Collection to the Ground
Station
Ground Station adds images
to Image Queue
Image Analyzer gets Image
from Image Queue
Image Analyzer detects
Potholes and produces
Annotated Image showing
pothole locations
Image Analyzer sends
Annotated Image to
Subscriber
Alternate Courses:
1.

2.

3.

Ground Station sends a


command to Spacecraft
to retransmit images:
Invoke Retransmit
Images
Spacecraft enters
safemode: invoke
Maintain Spacecraft
Health
Ground Station sends a
command to Spacecraft
to stop collecting images:
invoke Maintain
Spacecraft Health

Telecom

Instrument

Image Queue

Image Analyzer

Subscriber

CAPTURE_IMAGES_CMD()
CaptureImages()
loop
[16 images per collection]

Image Collection

SaveImageInOnboardStorage()

GetImages()
TRANSMIT_IMAGES()

State Machines :
Capture Images

RECEIVE_IMAGES()
AddImage()

loop all images in queue


GetImage()

Create()

Image
DetectPothole()

Annotated Image

Pothole
CreateBoundaryOutline()

AnnotateImageWithPothole()

SendAnnotatedImage()
ReceiveAnnotatedImage()

FigureA11.Sequencediagramshowswhatthesystemmustdoandwhichelementsareresponsiblefor
eachbehavior

225

APPENDIXA

Instrument
+

CaptureImages() : void
produces

Telecom
+
+
+
+
+

CAPTURE_IMAGES_CMD() : void
ENTER_SAFEMODE_CMD() : void
RETRANSMIT_IMAGES_CMD() : void
STOP_IMAGE_CAPTURE_CMD() : void
TRANSMIT_IMAGES() : void

Image Collection
+
+

GetImages() : void
SaveImageInOnboardStorage() : void

sends images to

Image Queue

Ground Station
+

RECEIVE_IMAGES() : void

+
+

Image

AddImage() : void
GetImage() : void

sends annotated image to

analyzes

Image Analyzer
Subscriber
+

ReceiveAnnotatedImage() : void

+
+

Annotated Image

produces
+
+

DetectPothole() : void
SendAnnotatedImage() : void

AnnotatePothole() : void
Create() : void

detects
Pothole
+

CreateBoundaryOutline() : void

FigureA12.ExpandedDomainModelshowingbehaviorsallocatedtosystemelements

FigureA13showsthestatediagramforinitializingtheinstrumentandcapturingimages.
Afterinitializingbuffersandthecamera,theinstrumententerstheIdlestatewhereitwaitsfora
groundcommand.ReceivingaCAPTURE_IMG_CMDcausesatransitiontotheCapturingImages
Stateinwhichthecameraismadeready.
Imagesarecapturedandsavedbythedobehaviors.Imagesarecapturedinthecircular
bufferuntiltheSTOP_IMAGE_CAPTURE_CMDisreceived.Thiscommanddisablesthecamera
afterthecurrentimagecaptureiscompleteandtheinstrumentreturnstotheIdlestate.
FigureA14showsapowerparametricmodelcomprisingcamera,datastore,telescope,and
instrumentcomputerpowermodels.Theparametricmodelsumsthesecomponentstoforma
totalpowerusage.

226

ROADMAP:DDT/SYSTEMS
Ground Sends Stop Command

Initializing
Instrument
Initial

+ do / initialize

[STOP_IMAGE_CAPTURE_CMD]
/CompleteImageCapture

Idle
+ do / waitForCommand

Capturing Images

[CAPTURE_IMG_CMD]

[STOP_IMAGE_CAPTURE_CMD]

+
+
+
+

do / captureImage
exit / disableCamera
entry / initializeCamera
do / saveImage

FigureA13.Allstatesandtransitionsmustbetested

Instrument Power = Telescope Power + Camera Power +


Data Store Power + Instrument Computer Power

Subsystem
Camera

Subsystem
DataStore

Camera Power

Data Store Power

Instrument Power
Instrument Power
Subsystem
Telescope

System
Instrument
Computer

Telescope Power

Instrument Computer
Power

FigureA14.InstrumentPowerParametricModel

ThiscompletesourdescriptionofbasicSysMLartifactsandsetsupthetestingdiscussionthat
follows.WhileourPOTHOLEexampleissimplisticandfanciful,themodelswedescribedarethe
basisformanysystems.
ThenextsectiondescribesthebaselineDDTmethodologyandrationale.Wefollowwitha
descriptionofanextensionforsystems.

227

APPENDIXA

Design Driven Testing


StephensandRosenberg57defineDesignDrivenTesting(DDT)asalogicalextensiontoUML
basedsoftwaredesign.ThepremiseofDDTisthatthebottomupagilepracticeofdriving
designsfromunittests(testdrivendesignTDD)isinherentlybackwards.InTDD,testcasesare
developedbeforecodedesign.Codeisthenwrittentomakethetestcasespassandrecodedas
necessaryiftestcasesfail.TDDproducesalotoftestcasesbutmoreseriouslybecauseofitsad
hocnature,doesntfullyaccountforacustomersneeds.Infact,theuserneedsaredetermined
bythetestresults!
Arguably,ignoringthecostofunnecessarytestingandrecoding,TDDworksinanagile
developmentcycle.However,thetypeofcodeproducedisnotsuitableforcriticalapplications
andmostuserscantolerateoddbehavioruntilthenextupdateispushedout.Noonewould
everdevelopspacecraftoranyothercriticalapplicationsusingTDD.
TheDDTapproachfollowsatopdownmethodologyinwhichthetestingisautomatically
generatedfromsoftwaremodels.Automationassuresthattestcasesarentmissedandthelink
tosoftwaremodelsassuresthattestcasesaredrivenfromthedesign.
DDTisanappropriatestartingplacefordevelopingSystemsV&Vtestmethodsbecausemost
systembehaviorisaccomplishedinsoftware.WedefineSysMLbasedextensionstoDDT
(DDT/Systems)laterinthepaper.SincewearestartingfromDDT,wepresentahighlevel
summaryoftheapproachhere,whichwewillillustratebyexampleshortly.
DDTautomatestestcasegenerationforbothdevelopersusingunittestsandforQAtesters
usingacceptancetests.FigureA15showsacceptancetestrelationships.Missionrequirements
aredrivenbycustomerneedsandusecasescenariosdefinewhatauserdoeswiththesystem.
DDTgeneratesteststhatevaluaterequirementcomplianceandcorrectimplementationof
desiredbehavior.AsshowninFigureA16,DDTalsoappliesduringthedevelopmentcycleby
automatingcreationofcontrollertestsfromrobustnessmodels(seenextsection)andunittests.

57

DesignDrivenTesting:TestSmarter,NotHarderbyMattStephensandDougRosenberg(Apress,2010).Alsosee:

www.designdriventesting.com

228

ROADMAP:DDT/SYSTEMS

FigureA15.AcceptanceTestingFlow

FigureA16.DeveloperTestingFlow

229

APPENDIXA
TheDDTmethodologycanbesummarizedas:

Generatetestcasesfromrequirements.
Generatethreadtestsfromscenarios.
Generateunittestsfromconceptualanddetaileddesigns.

WewillillustrateRequirement,Scenario,andUnittestgenerationbyexampleinafewpages.
However,thesetestsarenotsufficienttotestallaspectsofahardware/softwaresystem.Our
nextsectiondescribesanextensiontoDDTsuitableforembeddedsystems.
DDT/SystemsExtensions
ExtendingDDTforsystemsnecessitatescoveringadditionalelementsasshowninFigureA
17.

System Tests

DDT Software
Tests

FigureA17:DDT/SystemsincludesDDTSoftwareandadditionaltests

Theseelementsinclude:

StateMachines
ActivityDiagrams
BlockDefinitionDiagrams
InternalBlockDiagrams
ConstraintBlockDiagrams
FigureA18showsthesystemextensionstoDDT.

230

ROADMAP:DDT/SYSTEMS

DDT/Systems
DDT

- State Machine Tests


- Activity Tests
- Interface Tests (BDD/IBD)
- Parametric Tests

FigureA18:DDTSystemsextendsDDTforsoftwarewithadditionalsystemtests

WenextapplyDDT/SystemstothePOTHOLEspacecraft.

231

APPENDIXA

Validating POTHOLE with DDT/Systems


WefocusourPOTHOLEdesignandV&Vexamplesonasinglereferencefunction:takingand
processingimagesforfindingpotholes.WewalkthroughasimplifiedV&Vprocessbyexamplein
thefiguresbelow.ThenextsectionsummarizestherealworldV&Vprocess.

Nowwecometothenextvalidationstep:checkingthecompleteness,correctness,and
consistencyofourusecasedescription.Wewanttoassureourselvesthatwehaventforgotten
somethingorspecifiedsomethingtoovaguelyorincorrectly.Anyerrorsinourusecasemustbe
fixedandasnecessaryandthedomainmodeliterated.
Wearenowreadytodefinetestcases.Wecanautomaticallygenerateatestcaseelement
foreveryrequirementonarequirementdiagram,asshowninFigureA19.Requirementtests
areanintegralpartofthetestplan.Generatingtestcasesautomaticallyfromallrequirements
ensuresthatnoneareforgotten.

232

ROADMAP:DDT/SYSTEMS
Images shall be transmitted to the ground at
150 Mbps

(from Functional Requirements)

The spacecraft shall transmit images to the


ground in real-time when a ground station is
available

TestCase
Images shall be transmitted to the ground at 150 Mbps

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
The spacecraft shall transmit images to the ground in
real-time w hen a ground station is av ailable

(from Functional Requirements)


test scripts
Unit: : (Not Run) Default Run Scenario
The spacecraft shall store images for at least
one hour

(from Functional Requirements)

The spacecraft shall retransmit stored images


when requested by ground command

TestCase
The spacecraft shall store images for at least one hour

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
The spacecraft shall retransmit stored images w hen
requested by ground command

(from Functional Requirements)


test scripts
Unit: : (Not Run) Default Run Scenario
Retransmitted images shall be interleaved with
real-time images according to a ground
configurable downlink schedule

TestCase
Retransmitted images shall be interleav ed w ith
real-time images according to a ground configurable
dow nlink schedule

(from Functional Requirements)

The spacecraft shall store images in a circular


buffer over-writing older images with newer
images

test scripts
Unit: : (Not Run) Default Run Scenario
TestCase
The spacecraft shall store images in a circular buffer
ov er-w riting older images w ith new er images

(from Functional Requirements)


test scripts
Unit: : (Not Run) Default Run Scenario
Images shall be downlinked in groups of 16
using CCSDS format

TestCase
Images shall be dow nlinked in groups of 16 using
CCSDS format

(from Functional Requirements)


test scripts
Unit: : (Not Run) Default Run Scenario
The field of view shall be at least 1.0 Km x 1.0
Km

(from Functional Requirements)

The telescope shall have an adjustable aim


point

(from Functional Requirements)

Each image pixel shall have ground resolution


of no more than 0.25 meters

TestCase
The field of v iew shall be at least 1.0 Km x 1.0 Km

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
The telescope shall hav e an adj ustable aim point

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Each image pixel shall hav e ground resolution of no
more than 0.25 meters

(from Functional Requirements)

test scripts
Unit: : (Not Run) Default Run Scenario

FigureA19.POTHOLEMissionRequirementTestCases

Weuseausecasethreadexpandertoautomaticallygeneratetestscriptsforallsunny
day/rainydaypermutationswithinausecase.Thepremiseofthisapproachisthatausecase
withonesunnydayscenarioandthreerainydayscenariosrequiresatleastfourtestscriptsin

233

APPENDIXA
ordertoexerciseallusagepaths.ThisprocessisillustratedinFiguresA19toA22.FigureA20
showsastructuredscenariothatspecifiesstepnumbersandjoinlogicforourusecase.

FigureA20.Expandedsunny/rainydaythreads:createastructuredscenario

FigureA21showsanautomaticallygeneratedactivitydiagramthatisusedtoverifythelogic
ofthescenariobeforetestsaregenerated.

234

ROADMAP:DDT/SYSTEMS

FigureA21.Generateanactivitydiagramandcheckflowagainstusecases

235

APPENDIXA
FigureA22showsatestcasediagram,showingthecompleteusecasetextinanote,anda
testcaseelementwithatestscenario(threadtest)foreachCourseofActionwithintheuse
case.

FigureA22.ScenarioTesting

Usagepathsareautomaticallygeneratedforeachthread,asshowninFigureA23.These
threadsshowportionsoftheBasicCourseandportionsoftheAlternateCourses,asrelevantfor
eachthread.

236

ROADMAP:DDT/SYSTEMS

FigureA23.Generatetestscriptsfromusecasetext

DDTfollowsausecasedrivenapproachtodesigninwhicheachusecaseperformedbythe
systemisfirstelaboratedataconceptuallevel(refinestheusecaseandvalidatesthesystem)
andthenatadetaileddesignlevel(forsystemverification).DDTgeneratestestcaseelements
fromcontrollersonconceptualdesign(robustness)diagrams,asshowninFigureA24.
Unittestsarealsogeneratedfrommessagesondesignlevelsequencediagramsasshownin
FigureA25.AftertestscenarioshavebeendetailedwithInputs,Outputs,andSuccessCriteria,
thetestcasesareautomaticallytransformedintounittestcodesuitableforregressiontesting
frameworkssuchasJUnit,NUnitandFlexUnit.58

58

See:www.junit.org,www.nunit.organdwww.flexunit.orgrespectively.

237

APPENDIXA
TestCase
Retransmit Images Test
Retransmit Images
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Analyzer Test
Image Analyzer
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
capture images Test
capture images
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
send images Test
send images
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
add image Test
add image
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
get image Test
get image
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
detect pothole Test
detect pothole
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
annotate image Test
annotate image
test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
send annotated image Test
end annotated image
test scripts
Unit: : (Not Run) Default Run Scenario

FigureA24.POTHOLEControllerTestCases

238

ROADMAP:DDT/SYSTEMS
TestCase
Ground Station RECEIVE_IMAGES Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Queue AddImage Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Queue GetImage Test

test scripts
Unit: : (Not Run) Default Run Scenario

ref
Look for Potholes Sequence

TestCase
Image Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Annotated Image Create Test

test scripts
Unit: : (Not Run) Default Run Scenario

TestCase
Image Analyzer DetectPothole Test

test scripts
Unit: : (Not Run) Default Run Scenario

FigureA25.SubsetofAutogeneratedLookforPotholesSequenceMessageTests

FigureA26showsstate,statetransitions,andbehaviortestcasesforthecaptureimage
statemachinedescribedearlier.Allbehaviors,transitionsandtriggersmustbetestedincluding
entry,do,andexitbehaviorsonstates.Notethatsimilartestsmustalsobedefinedfor
activitydiagrambehavioraldescriptions.
InterfaceControlDefinitionsmustalsobetested.Thisincludeselectricalandlogical
interfacedefinitionsasshowninFigureA27.Thisincludesport,required/providedinterfaces,
electricalandtiming,andfaultbehaviortests.

239

APPENDIXA
TestCase
Test Capturing Images
Capturing Images
+
+
+
+

do / captureImage
exit / disableCamera
entry / initializeCamera
do / saveImage

Unit:
Unit:
Unit:
Unit:
Unit:
Unit:
Unit:
Unit:

:
:
:
:
:
:
:
:

(Not
(Not
(Not
(Not
(Not
(Not
(Not
(Not

Run) Test
Run) Test
Run) Test
Run) Test
Run) Test
Run) Test
Run) Test
Run) Test

Behaviors,
transitions and
triggers must all
be tested.

TestCase
Test Idle

Idle
+

test scripts
captureImage
completeImageCapture
disableCamera
initializeCamera
Recieve Capture Images Command
Recieve Stop Image Capture Command
saveImage
Stop Image Capture

do / waitForCommand
test scripts
Unit: : (Not Run) Test waitForCommand

TestCase
Test Initializing Instrument
Initializing Instrument
+

do / initialize

test scripts
Unit: : (Not Run) Test initialize

FigureA26:Examplesofstatetests

Board
NVM
TestCase
Test Buffer Control
Interface

flowPort
InstrDataIn

TestCase
Test Instrument Data Input

flowPort
FIFOCtlIn

flowPort
FIFOStatusOut

flowPort
XbandDL

(from Block Diagrams)

TestCase
Test XBand Output

FigureA27.NeedtotestallbehaviorsandinterfacesdefinedinanInterfaceControlDocument(ICD)

Inmanycases,analyticalmodelsareneededforevaluatingnonbehavioralrequirements.
Thesemodelscouldevaluatemass,power,radiationeffects,andsoforth.FigureA28showsa
parametricdiagramthatevaluatesinstrumentpower.Thecomputedtotalpoweriscomparedto
amaximuminstrumentpowerrequirement.

240

ROADMAP:DDT/SYSTEMS
par [SysML Parametric] Block Diagrams [Instrument Parametric Diagram]

Instrument Power = Telescope Power + Camera Power +


Data Store Power + Instrument Computer Power

Subsystem
Camera

Subsystem
DataStore

Camera Power

Data Store Power

Max Instrument Power

Instrument Power

notes
Not to exceed 500W

Instrument Power
Subsystem
Telescope

System
Instrument
Computer

Telescope Power

Instrument Computer
Power

TestCase
Test Instrument Pow er

test scripts
Unit: : (Not Run) Test Instrument Power does not exceed Max Instrument Power

FigureA28.Parametricevaluationofinstrumentpower

ThisconcludesourillustrationofDDT/SystemsusingthePOTHOLEexample.
Asillustratedinthissection,DDT/Systemscomprisesthefollowingsetoftests:
Requirementtests
Scenariotests
Controllertests
Sequencetests
Statemachine&Activitytests
Interfacetests
Parametrictests
ThisisthemostcomprehensiveandefficientsystemV&Vapproachweareawareof.

Conclusions

DDTautomatesgenerationoftestcasesfromUMLmodelsforsoftware
SystemsV&VrequiresadditionalmodelingandtestcasegenerationfromSysML
OurfictitiousPOTHOLEspacecraftillustratessomeoftheseextensions
WehavedescribedproposedadditiontoDDTthatencompassessystems(DDT/Systems)

241

COMINGSOONFROM

fingerpress

HAMSTER
DRIVEN
DEVELOPMENT

(itsnotforthesqueamish)
SatirefromthecodingroomfloorfromtheauthorsofDesignDrivenTesting:TestSmarter,
NotHarderandExtremeProgrammingRefactored:TheCaseAgainstXP(featuringSongsof
theExtremos)

www.fingerpress.co.uk

DESIGN DRIVEN TESTING

ThegroundbreakingbookDesignDrivenTesting
bringssanitybacktothesoftwaredevelopment
processbyflippingaroundtheconceptofTest
DrivenDevelopment(TDD)restoringthe
conceptofusingtestingtoverifyadesigninstead
ofpretendingthatunittestsareareplacement
fordesign.
AnyonewhofeelsthatTDDisTooDamn
Difficultwillappreciatethisbook.

Visitthewebsiteformoreinfo:

www.designdriventesting.com

DesignDrivenTestingshowsthat,by
combiningaforwardthinking
developmentprocesswithcuttingedge
automation,testingcanbeafinely
targeted,businessdriven,rewarding
effort.Inotherwords,youlllearnhow
totestsmarter,notharder.

OPEN
ENROLLMENT
TRAINING

Youvereadthebook,nowgetthetraining!

Learn how to apply the process roadmaps youve just read


about in this book from Doug at an ICONIX Open Enrollment
workshop

Developers:Learnarigorous,systematicpathfromrequirementstosourcecode
BusinessAnalysts:Accomplishbusinessprocessmodelingforrequirementselicitationand
processreengineering
SystemsEngineers:ApplyarigorousdesignandtestingmethodologyusingSysML

TheDesignDrivenTestingcourseusesasitsexampleaninteractivehotelmappingsystemwhich
youcantryoutforrealat:www.vresorts.com

ForthelatestOpenEnrollmentschedule,pleasevisit:
www.iconixsw.com/EA/PublicClasses.html