Académique Documents
Professionnel Documents
Culture Documents
SACHIN 1
Today’s Topics
Chapters 17 & 18 in SEPA 5/e
Testing Principles & Testability
Test Characteristics
Black-Box vs. White-Box Testing
Flow Graphs & Basis Path Testing
Testing & Integration Strategies
SACHIN 2
Software Testing
Opportunities for human error
Specifications, design, coding
Communication
“Testing is the ultimate review”
Can take 30-40% of total effort
For critical apps, can be 3 to 5 times all
other efforts combined!
SACHIN 3
Testing Objectives
Execute a program with the intent of
finding errors
Good tests have a high probability of
discovering errors
Successful tests uncover errors
‘No errors found’: not a good test!
Verifying functionality is a secondary goal
SACHIN 4
Testing Principles
Tests traceable to requirements
Tests planned before testing
Pareto principle: majority of errors traced
to minority of components
Component testing first, then
integrated testing
Exhaustive testing is not possible
Independent tests: more effective
SACHIN 5
Software Testability
Characteristics that lead to
testable software:
Operability • Simplicity
Observability
• Stability
Controllability
Decomposability • Understandability
SACHIN 6
Operability
The better it works, the more efficiently
it can be tested
SACHIN 7
Observability
What you see is what you test
Distinct output for each input
States & variables may be queried
Past states are logged
Factors affecting output are visible
Incorrect output easily identified
Internal errors reported
Source code accessible
SACHIN 8
Controllability
The better we can control the software,
the more the testing can be automated
All possible outputs can be generated by
some input
All code executable by some input
States, variables directly controlled
Input/output consistent, structured
Tests are specified, automated, and
reproduced
SACHIN 9
Decomposability
By controlling the scope of testing, we
can more quickly isolate problems and
perform smarter retesting
Independent modules
Modules can be tested separately
SACHIN 10
Simplicity
The less there is to test, the more
quickly we can test it
SACHIN 11
Stability
The fewer the changes, the fewer the
disruptions to testing
SACHIN 12
Understandability
The fewer the changes, the fewer the
disruptions to testing
Design is well-understood
Dependencies are well understood
Design changes are communicated
Documentation is:
accessible
well-organized
specific, detailed and accurate
SACHIN 13
Test Characteristics
Good test has a high probability of finding
an error
Good test is not redundant
A good test should be “best of breed”
A good test is neither too simple nor too
complex
SACHIN 14
Test Case Design
‘Black Box’ Testing
Consider only inputs and outputs
‘White Box’ or ‘Glass Box’ Testing
Also consider internal logic paths, program
states, intermediate data structures, etc.
SACHIN 15
White-Box Testing
Guarantee that all independent paths have
been tested
Exercise all conditions for ‘true’ and ‘false’
Execute all loops for boundary conditions
Exercise internal data structures
SACHIN 16
Why White-Box Testing?
More errors in ‘special case’ code which is
infrequently executed
Control flow can’t be predicted accurately
in black-box testing
Typo errors can happen anywhere!
SACHIN 17
Basis Path Testing
White-box method [McCabe ‘76]
Analyze procedural design
Define basis set of execution paths
Test cases for basis set execute every
program statement at least once
SACHIN 18
Basis Path Testing [2]
Flow Graph: Representation of
Structured Programming Constructs
SACHIN 19
Cyclomatic Complexity
V(G)=E-N+2 = 4
Independent Paths
1: 1,11
2: 1,2,3,4,5,10,1,11
3: 1,2,3,6,8,9,10,1,11
4: 1,2,3,6,7,9,10,1,11
SACHIN 20
Black Box Testing
Focus on functional requirements
Incorrect / missing functions
Interface errors
Errors in external data access
Performance errors
Initialization and termination errors
SACHIN 21
Black Box Testing [2]
How is functional validity tested?
What classes of input will make good test
cases?
Is the system sensitive to certain inputs?
How are data boundaries isolated?
SACHIN 22
Black Box Testing [3]
What data rates and volume can the system
tolerate?
What effect will specific combinations of data
have on system operation?
SACHIN 23
Comparison Testing
Compare software versions
“Regression testing”: finding the outputs
that changed
Improvements vs. degradations
Net effect depends on frequency and
impact of degradations
When error rate is low, a large corpus can
be used
SACHIN 24
Generic Testing
Strategies
Testing starts at module level and moves
“outward”
Different testing techniques used at
different times
Testing by developer(s) and independent
testers
Testing and debugging are separate
activities
SACHIN 25
Verification and
Validation
Verification
“Are we building the product right?”
Validation
“Are we building the right product?”
Achieved by life-cycle SQA activities,
assessed by testing
“You can’t create quality by testing”
SACHIN 26
Organization of Testing
[From SEPA 5/e]
SACHIN 27
Logarithmic Poisson
execution-time model
SACHIN 28
[From SEPA 5/e]
SACHIN 29
[From SEPA 5/e]
SACHIN 30
PRO: Lower-level (reusable) modules tested early
CON: Higher-level (logic) modules tested late
SACHIN 31
Hybrid Approaches
Sandwich Integration: combination of top-
down and bottom-up
Critical Modules
address several requirements
high level of control
complex or error prone
definite performance requirements
Test Critical Modules ASAP!
SACHIN 32
SACHIN 33