Testing in Agile

1 hLLp//qLpblogspoLcom
Table of contents
1. Traditional style QA
2. Agile Approaches
3. An Agile Tester
4. Traditional vs. Agile
5. Role Testers Play
6. Testing & Testers on Agile Projects
7. Question: While a developer is coding a task, it is impossible for a tester to
test it (it doesn't exist yet). What then is the role of a tester at this point.
8. Question: s the tester now involved in unit testing? s this done parallel to
black box testing?
9. Question: What does the tester do during a sprint where primarily
infrastructural changes have been made, that may only be testable in unit
testing?
10. Specific Technical Skills For Agile Tester's Toolkit
hLLp//qLpblogspoLcom
TraditionaI StyIe QuaIity Assurance
Process and tools are a key part of QA and Testing.
QA people seem to love documentation.
QA people want to see the written specification.
And where is testing without a plan?
So, is there a role for QA in Agile projects?
Maybe, but the role is different, the tasks are different.
hLLp//qLpblogspoLcom
AgiIe approaches are changing the
conversation about software deveIopment
Agilc sIificd our aiicniion io small icams.
.
ncrcmcnially dclivcring qualiiy sofiwarc.
TIc old idcas aloui icsiing ai iIc cnd of iIc coding Iasc no longcr alicallc.
Wc nccd io iIinl aloui iIc rolc of Qualiiy.
Assurancc in Agilc Projccis.
Dcfiniicly NOT lusincss-as-usual.
hLLp//qLpblogspoLcom
An AgiIe Tester
A professional tester who embraces change,
collaborates well with both technical and business
people, and understands the concept of using
tests to document requirements and drive
development.
Agile testers tend to have good technical skills,
know how to collaborate with others to automate
tests, and are also experienced exploratory
testers.
They're willing to learn what customers do so that
they can better understand the customers'
software requirements.
hLLp//qLpblogspoLcom
TraditionaI vs. AgiIe Testing
kequ|rements Spec|f|cat|ons
Code 1est|ng ke|ease
@me
A 8
C
A 8
C
A
8
u
C
A
8
u
L
l
Agle
lLeraLe lncremenLal
Lach sLory s expanded coded LesLed
ossble release afLer each LeraLon
hased Model eg WaLerfall
6 hLLp//qLpblogspoLcom
TraditionaI vs. AgiIe Testing
1rad|t|ona| Ag||e
n the phased approach diagram (see previous
slide), it is clear that testing happens at the end,
right before release. The diagram is idealistic,
because it gives the impression there is as much
time for testing as there is for coding. n many
projects, this is not the case. The testing gets
"squished because coding takes longer than
expected, and because teams get into a code-
and-fix cycle at the end.
Agile is iterative and incremental (see previous
slide). This means that the testers test each
increment of coding as soon as it is finished. An
iteration might be as short as one week, or as
long as a month. The team builds and tests a little
bit of code, making sure it works correctly, and
then moves on to next piece that needs to be
built. Programmers never get ahead of the
testers, because a story is not "done until it has
been tested.
Tests are usually created from a requirements
document.
Rather than creating tests from a requirements
document that was created by business analysts
before anyone ever thought of writing a line of
code, someone will need to write tests that
illustrate the requirements for each story days or
hours before coding begins.
hLLp//qLpblogspoLcom
#oIe Testers PIay
DnL @esLng
AccepLance
@esLng

SysLem @esLers are no longer
requred?
The role of the tester with agile methods is an area that has received increasing attention.
nitially with a focus on unit testing and 'acceptance' testing it appeared the system tester
did not have a role in agile.
1
-laclLaLe communcaLon beLween Lhe Lechncal busness sLakeholders

-SupporL early aldaLon of requremenLs

-Pelp Lhe busness sLakeholders defne accepLance crLera

-CreaLe auLomaLed accepLance LesLs

-erform manual/exploraLory LesLs on earlysLage code

As Cem Kaner put it:
'The nature of the tester's role
changes in iterative projects.
We are no longer the high-
profile victims, we are no
longer the lonely advocates of
quality, we are merely (!)
competent service providers,
collaborating with a group that
wants to achieve high quality.'
Typical responsibilities for testers in agile can include (not limited to):
hLLp//qLpblogspoLcom
Testing and Testers on AgiIe Projects
hLLp//qLpblogspoLcom
t comes as no surprise to testers that
working software is not the same as code
the tester clearly needs to be involved in not
only assessing the product, but in deciding
how the product is to be assessed. However,
with automated unit tests in the hands of the
coders, and confirmation-focused acceptance
testing driven by the customer, testers should
be aware that they will not be the sole or
even the primary owner of deciding what
works, and what doesn't.
10 hLLp//qLpblogspoLcom
Testers need to be able to interact directly
with designers and coders to understand the
technological imperatives and restrictions that
affect the software and its unit tests.
||| take the
p|ace of
some documentation
Conversations and shared
understandings
erall less
documenLaLon
requred
11 hLLp//qLpblogspoLcom
Testing will be driven by what is important to a user, rather than to
fulfill a procedural requirement. t is better to have
communication between tester, customer and designer than to
maintain independence of the test team.
n practice, it is common to find large-scale automated unit
testing on agile projects, to confirm that code works as expected.
%e product will be judged by te customer typically by manual,
confirmatory tests, wit close observation for undesirable
beaviors. Testing by testers is often driven by the need to
measure the system's performance and to find surprises tools
are very much in evidence, but rigid test scripts and procedures
do not give the requisite opportunity for discovery, diagnosis and
exploitation.
1 hLLp//qLpblogspoLcom
Testers are key collaborators with the customer, and on
some agile projects will take on much of the role of the
customer in designing and executing confirmation-driven
acceptance tests. However, although testers traditionally
make good customer advocates, working closely with a
customer is preferable to becoming a proxy.
Test strategies which lean heavily on an unchanging set
of requirements (for example: designing and coding tests to be
bougt togeter wit code late in te project; prioritizing tests based on
a fixed risk assessment; testing only wat as been agreed in te
contract; reporting bugs only against fixed requirements) may be
considered to be fatally flawed in the light of this value.
terative collaboration is favored over a negotiated bill of
work
1 hLLp//qLpblogspoLcom
hiIe a deveIoper is coding a task, it is
impossibIe for a tester to test it (it
doesn't exist yet). hat then is the roIe
of a tester at this point.
1 hLLp//qLpblogspoLcom
Testers can prepare their test plans, test
cases, and automated tests for the user
stories before (or while) they are
implemented. This helps the team
discover any inconsistency or ambiguity
in the user stories even before the
developers write any code.
1 hLLp//qLpblogspoLcom
The tester could be working with the
customer to fine tune the stories in the
sprint.
16 hLLp//qLpblogspoLcom
They can often be involved in designing
the tests that the coder will write to
perform TDD.
1 hLLp//qLpblogspoLcom
f the agile team is fairly advanced then
the tester would normally be writing the
ATDD (Acceptance Test Driven
Development) tests. These could be in a
tool such as Fitnesse or Robot Framework
or they could be more advanced ruby tests
or even some other programming
language. Or in some cases, simple
record and playback can often be
beneficial for a small number of tests.
1 hLLp//qLpblogspoLcom
They would obviously be writing planning
some exploratory testing scenarios or
ideas.
1 hLLp//qLpblogspoLcom
The tricky thing to comprehend sometimes
for the team is that the story does not have
to be complete in order to drop it to the
test stack for testing. For example the
coders could drop a screen with half of the
fields planned on it. The tester could test
this half whilst the other half is being
coded and hence feedback in with early
test results. Testing doesn't have to take
place on "finished" stories.
0 hLLp//qLpblogspoLcom
s the tester now invoIved in unit
testing? s this done paraIIeI to bIack
box testing?
1 hLLp//qLpblogspoLcom
1esters
DnL @esLs
uon'L do
Testers only test code that
passes all of the automated
unit, integration and acceptance
tests, which are all written by
the developers. This split may
be different elsewhere, though;
for example your testers could
be writing automated
acceptance tests.
@he whole ponL of unL LesLs s Lo LesL
Lhe sofLware s correcL Lo aod wasLng
Lme laLer down Lhe lne lLs all abouL
nsLanL feedback
hLLp//qLpblogspoLcom
hat does the tester do during a sprint
where primariIy infrastructuraI changes
have been made, that may onIy be
testabIe in unit testing?
hLLp//qLpblogspoLcom
Testers workload will vary between sprints,
but regression tests still need to be run on
these changes...
hLLp//qLpblogspoLcom
ou may also find that having the testers
spend the first couple of days of each sprint
testing the tasks from the previous sprint may
help, however it's better to get them to nail
down the things that the developers are going
to be working on by writing their test plans.
hLLp//qLpblogspoLcom
nsure that project or sprint requirements are
clear, measurable and testable. n an ideal
world each requirement will have a fit criterion
written down at this stage. Determine what
information needs to be automatically logged
to troubleshoot any defects.
6 hLLp//qLpblogspoLcom
Prepare a project specific test strategy and
determine which QA steps are going to be
required and at which project stages:
integration, stress, compatibility, usability,
performance, beta testing etc. Determine
acceptable defect thresholds and work out
classification system for defect severity,
specify guidelines for defect reporting.
hLLp//qLpblogspoLcom
Specify, arrange and prepare test environment:
test infrastructure and mock services as
necessary; prepare test data; write scripts to
quickly refresh test environment when necessary;
establish processes for defect tracking,
communication and resolution; prepare for
recruitment or recruit users for beta, usability or
acceptance testing. Write test scripts.
hLLp//qLpblogspoLcom
deally the tester would be working with the team
and the customer (who by the way, is part of the
team!) to define the planned stories and build in
some good, detailed acceptance criteria. This is
invaluable and can save loads of time later down
the line. The tester could also be learning new
automation techniques, planning test
environments, helping to document the outcome
of the planning.
hLLp//qLpblogspoLcom
Specific Technical Skills For Agile
Tester's Toolkit
0 hLLp//qLpblogspoLcom
lor auLomaLon Lo
succeed
we need Lo apply
good desgn pracLces
We strive to keep each automated test to
a single &
clear purpose
earn how to evaluate and choose the right tools, so you
can help your team create maintainabIe automated
regression tests. ou can free up time for essential testing
activities such as exploratory testing.
AuLomaLon Sklls
1 hLLp//qLpblogspoLcom
Communication skills and good domain
understanding enable testers to help business
experts give good examples of both desired and
undesired tyttem behuuior.
We can turn these examples into tests that
help the programmers understand what code to
write. This is called acceptance test-driven
development, and it is a major step toward
building quality into the code and preventing
defects.
AccepLance @esLdren ueelopmenL
hLLp//qLpblogspoLcom
earnng
SLyles
audLory
learners
learn by
lsLenng
sual
learners
earn by
see pcLures
We all have blind spots that may prevent us from learning or triggers
where we shut down and don't hear the message anymore. Keep
your emotional "hot buttons in mind and focus on what you can learn
from instructors, material, or teammates to enhance your abilities.
Mentors with different backgrounds or from other industries besides
testing and software development might work best with your learning
style. Don't limit yourself to coaches, mentors, and instructors who
work specifically in software testing.
earnng SLyles
hLLp//qLpblogspoLcom
Many good sofLware
@esLng books
lenLy of free maLeral on Lhe lnLerneL
8esources your peers can prode
CommunLes of pracLce are anoLher good place Lo
fnd menLors and learn LogeLher
Conferences are an obous way Lo geL a loL of new
deas n a ery shorL perod of Lme
Malng lsLs and socal neLworks
@esLng communLes such as Weekend @esLers
earnng 8esources Lxamples
hLLp//qLpblogspoLcom
8eferences
W AClL @LS@lnC A 8AC@lCA CDluL l8 @LS@L8S Anu AClL
@LAMS by sa Crspn !aneL Cregory
W hLLp//sqasLackexchangecom/quesLons/1/Lheroleofa
sofLwareLesLernanagleenronmenL
W hLLp//searchsofLwarequalLyLechLargeLcom/news/10/8ole
ofLesLngnaglepro[ecLs
W hLLp//wwwkohlca/blog/arches/000116hLml
W hLLp//wwwmcbreenabca/Lalks/CAMDCpdf
W hLLp//newsweaere/qualLech/e_arLcle0001cfm?xb110w
W hLLp//sLackoerflowcom/quesLons/16011/roleofLesLersn
agle
W LA8nlnC l8 AClL @LS@L8S arL by sa Crspn and !aneL
Cregor 8eLLer SofLware Magazne SepL/cL 011
hLLp//qLpblogspoLcom