Chapter 17

Software
Testing
Strategies
Unit III

Software Testing
Testing is the process of exercising
a program with the specific intent of
finding errors prior to delivery to the
end user.

Unit III

Easy Language : :

Software testing is a process used to identify the

correctness, completeness, and quality of developed
computer software. It includes a set of activities
conducted with the intent of finding errors in software so
that it could be corrected before the product is released
to the end users.
In simple words, software testing is an activity to check
whether the actual results match the expected results
and to ensure that the software system is defect free.

Unit III

Why is testing is important?

Unit III

This isChinaAirlines Airbus A300 crashing due to a software bug on April 26,
1994 killing 264 innocent lives.
Software bugs can potentially cause monetary and human loss, history is full
of such examples.
In 1985,Canada's Therac-25 radiation therapy machine malfunctioned due to
software bug and delivered lethal radiation doses to patients ,leaving 3 people
dead and critically injuring 3 others.
In April of 1999 ,a software bug caused the failure of a $1.2 billion military
satellite launch, the costliest accident in history.
In may of 1996, a software bug caused the bank accounts of 823 customers of
a major U.S. bank to be credited with 920 million US dollars.
As you see, testing is important because software bugs could be expensive or
even dangerous.
As Paul Elrich puts it - "To err is human, but to really foul things up you need a
computer.

What Testing Shows

errors
requirements conformance
performance

an indication
of quality

Unit III

Example:

Consider a scenario where you are moving a file from folder A to

Folder B. Think of all the possible ways you can test this.
Apart from the usual scenarios, you can also test the following
conditions.
Trying to move the file when it is Open.
You do not have the security rights to paste the file in Folder B.
Folder B is on a shared drive and storage capacity is full.
Folder B already has a file with the same name, infact the list is
endless.
Or suppose you have 15 input fields to test ,each having 5 possible
values , the number of combinations to be tested would be 5^15.

Unit III

Strategic Approach

Unit III

To perform effective testing, you should conduct effective

technical reviews. By doing this, many errors will be
eliminated before testing commences.
Testing begins at the component level and works
"outward" toward the integration of the entire computerbased system.
Different testing techniques are appropriate for different
software engineering approaches and at different points
in time.
Testing is conducted by the developer of the software
and (for large projects) an independent test group.
Testing and debugging are different activities, but
debugging must be accommodated in any testing
strategy.
7

V&V

Verification refers to the set of tasks that ensure

that software correctly implements a specific
function.
Validation refers to a different set of tasks that
ensure that the software that has been built is
traceable to customer requirements. Boehm
[Boe81] states this another way:

Unit III

Verification: "Are we building the product right?"

Validation: "Are we building the right product?"

Who Tests the Software?

developer
Understands the system
but, will test "gently"
and, is driven by "delivery"
Unit III

independent tester
Must learn about the system,
but, will attempt to break it
and, is driven by quality
9

Software Testing Life Cycle STLC

Unit III

10

A Strategy for Testing

Conventional Software

Unit III

11

Testing Strategy

We begin by testing-in-the-small and move

toward testing-in-the-large
For conventional software

For OO software

Unit III

The module (component) is our initial focus

Integration of modules follows
our focus when testing in the small changes from
an individual module (the conventional view) to an
OO class that encompasses attributes and
operations and implies communication and
collaboration
12

Strategic Issues

Unit III

Specify product requirements in a quantifiable manner

long before testing commences.
State testing objectives explicitly.
Understand the users of the software and develop a
profile for each user category.
Develop a testing plan that emphasizes rapid cycle
testing.
Build robust software that is designed to test itself
Use effective technical reviews as a filter prior to testing
Conduct technical reviews to assess the test strategy
and test cases themselves.
Develop a continuous improvement approach for the
testing process.
13

Manual Testing types:

Unit III

14

The goal of unit testing is to isolate each part of the program and show that the
individual parts are correct. Unit testing is usually performed by the developer.

Unit Testing
module
to be
tested
results
software
engineer

Unit III

test cases

15

Unit testing increase the speed of

development.

Unit III

16

Unit Testing
module
to be
tested

interface
local data structures
boundary conditions
independent paths
error handling paths

test cases
Unit III

17

Unit Test Environment

driver
interface
local data structures

Module

boundary conditions
independent paths
error handling paths

stub

stub

test cases
RESULTS
Unit III

18

Benefits and Limitations

Unit Testing benefits
Developers

looking to learn what functionality is provided by a unit and how to use it can look
at the unit tests to gain a basic understanding of the unit API.
Unit testing allows the programmer to refactor code at a later date, and make sure the module
still works correctly (i.e. Regression testing). The procedure is to write test cases for all
functions and methods so that whenever a change causes a fault, it can be quickly identified
and fixed.
Due to the modular nature of the unit testing, we can tests parts of project without waiting for
others to be completed.

Unit Testing Limitations

Unit

testing can't be expected to catch every error in a program. It is not possible to evaluate
all execution paths even in the most trivial programs
Unit testing by its very nature focuses on a unit of code. Hence it can't catch integration errors
or broad system level errors.
Unit III

19

Integration Testing Strategies

Options:
the big bang approach
an incremental construction strategy

Unit III

20

Need of Integration Testing

A Module in general is designed by an individual software developer

whose understanding and programming logic may differ from other
programmers. Integration testing becomes necessary to verify the
software modules work in unity
At the time of module development, there are wide chances of change
in requirements by the clients. These new requirements may not be
unit tested and hence integration testing becomes necessary.
Interfaces of the software modules with the database could be
erroneous
External Hardware interfaces, if any, could be erroneous
Inadequate exception handling could cause issues.

Unit III

21

Example
Integration Test case differs from other test cases in the sense it focuses mainly on the
interfaces & flow of data/information between the modules. Here priority is to be given for
the integrating links rather than the unit functions which are already tested.
Sample Integration Test Cases for the following scenario: Application has 3 modules say
'Login Page', 'Mail box' and 'Delete mails' and each of them are integrated logically.
Here do not concentrate much on the Login Page testing as it's already been done in Unit
Testing. But check how it's linked to the Mail Box Page.
Similarly Mail Box: Check its integration to the Delete Mails Module.

Unit III

22

Big Bang Approach:

Here all component are integrated together at once, and then tested.
Advantages:
Convenient

for small systems.

Disadvantages:
Fault

Localization is difficult.
Given the sheer number of interfaces that need to be tested in this approach, some
interfaces links to be tested could be missed easily.
Since the integration testing can commence only after "all" the modules are designed, testing
team will have less time for execution in the testing phase.
Since all modules are tested at once, high risk critical modules are not isolated and tested on
priority. Peripheral modules which deal with user interfaces are also not isolated and tested on
priority.

Unit III

23

Bottom Up Integration
In the bottom up strategy, each module at lower levels is
tested with higher modules until all modules are tested. It
takes help of Drivers for testing
Diagrammatic Representation:

Unit III

24

In

Incremental Approach:

this approach, testing is done by joining two or more modules that are logically
related. Then the other related modules are added and tested for the proper functioning.
Process continues until all of the modules are joined and tested successfully.
This

process is carried out by using dummy programs called Stubs and Drivers. Stubs
and Drivers do not implement the entire programming logic of the software module but
just simulate data communication with the calling module.
Stub:

Is called by the Module under Test.

Driver:

Calls the Module to be tested.

Incremental

Approach in turn is carried out by two different Methods:

Bottom Up
Top Down
Unit III

25

Bottom Up Integration
Advantages:
Fault

localization is easier.
No time is wasted waiting for all modules to be developed unlike Bigbang approach
Disadvantages:
Critical

modules (at the top level of software architecture) which control

the flow of application are tested last and may be prone to defects.
Early prototype is not possible

Unit III

26

Top down Integration:

In Top to down approach, testing takes place from top to down following the
control flow of the software system.
Takes help of stubs for testing.

Unit III

27

Top Down Integration

Advantages:
Fault Localization is easier.
Possibility to obtain an early prototype.
Critical Modules are tested on priority; major design flaws could be found and
fixed first.

Disadvantages:

Needs many Stubs.

Modules at lower level are tested inadequately.

Unit III

28

Regression Testing

Unit III

Regression testing is the re-execution of some subset of

tests that have already been conducted to ensure that
changes have not propagated unintended side effects
Whenever software is corrected, some aspect of the
software configuration (the program, its documentation,
or the data that support it) is changed.
Regression testing helps to ensure that changes (due to
testing or for other reasons) do not introduce
unintended behavior or additional errors.
Regression testing may be conducted manually, by reexecuting a subset of all test cases or using automated
capture/playback tools.

29

Smoke Testing

A common approach for creating daily builds for product

software
Smoke testing steps:

Software components that have been translated into code are

integrated into a build.

A build includes all data files, libraries, reusable modules, and engineered
components that are required to implement one or more product
functions.

A series of tests is designed to expose errors that will keep the build
from properly performing its function.
The intent should be to uncover show stopper errors that have the
highest likelihood of throwing the software project behind schedule.

The build is integrated with other builds and the entire product (in its
current form) is smoke tested daily.
The integration approach may be top down or bottom up.

Unit III

30

Object-Oriented Testing

begins by evaluating the correctness and

consistency of the analysis and design models
testing strategy changes

Unit III

the concept of the unit broadens due to

encapsulation
integration focuses on classes and their execution
across a thread or in the context of a usage
scenario
validation uses conventional black box methods

test case design draws on conventional

methods, but also encompasses special
features

31

Broadening the View of Testing

ItcanbearguedthatthereviewofOOanalysisand
designmodelsisespeciallyusefulbecausethe
samesemanticconstructs(e.g.,classes,attributes,
operations,messages)appearattheanalysis,
design,andcodelevel.Therefore,aprobleminthe
definitionofclassattributesthatisuncovered
duringanalysiswillcircumventsideeffectsthat
mightoccuriftheproblemwerenotdiscovered
untildesignorcode(oreventhenextiterationof
analysis).
Unit III

32

Testing the CRC Model

1. Revisit the CRC model and the object-relationship model.
2. Inspect the description of each CRC index card to determine if a
delegated responsibility is part of the collaborators definition.
3. Invert the connection to ensure that each collaborator that is
asked for service is receiving requests from a reasonable source.
4. Using the inverted connections examined in step 3, determine
whether other classes might be required or whether responsibilities
are properly grouped among the classes.
5. Determine whether widely requested responsibilities might be
combined into a single responsibility.
6. Steps 1 to 5 are applied iteratively to each class and through
each evolution of the analysis model.

Unit III

33

OO Testing Strategy

class testing is the equivalent of unit testing

integration applied three different strategies

Unit III

operations within the class are tested

the state behavior of the class is examined
thread-based testingintegrates the set of
classes required to respond to one input or event
use-based testingintegrates the set of classes
required to respond to one use case
cluster testingintegrates the set of classes
required to demonstrate one collaboration

34

WebApp Testing - I

Unit III

The content model for the WebApp is reviewed

to uncover errors.
The interface model is reviewed to ensure that
all use cases can be accommodated.
The design model for the WebApp is reviewed
to uncover navigation errors.
The user interface is tested to uncover errors in
presentation and/or navigation mechanics.
Each functional component is unit tested.

35

WebApp Testing - II

Unit III

Navigation throughout the architecture is tested.

The WebApp is implemented in a variety of different
environmental configurations and is tested for
compatibility with each configuration.
Security tests are conducted in an attempt to exploit
vulnerabilities in the WebApp or within its environment.
Performance tests are conducted.
The WebApp is tested by a controlled and monitored
population of end-users. The results of their interaction
with the system are evaluated for content and navigation
errors, usability concerns, compatibility concerns, and
WebApp reliability and performance.

36

High Order Testing

Validation testing

System testing

executes a system in a manner that demands resources in abnormal quantity,

frequency, or volume

Performance Testing

Unit III

verifies that protection mechanisms built into a system will, in fact, protect it
from improper penetration

Stress testing

forces the software to fail in a variety of ways and verifies that recovery is
properly performed

Security testing

Focus is on customer usage

Recovery testing

Focus is on system integration

Alpha/Beta testing

Focus is on software requirements

test the run-time performance of software within the context of an integrated

system

37

Debugging: A Diagnostic Process

Unit III

38

The Debugging Process

Unit III

39

Debugging Effort
time required
to correct the error
and conduct
regression tests

Unit III

time required
to diagnose the
symptom and
determine the
cause

40

Symptoms & Causes

symptom and cause may be
geographically separated
symptom may disappear when
another problem is fixed
cause may be due to a
combination of non-errors
cause may be due to a system
or compiler error

symptom

cause

cause may be due to

assumptions that everyone
believes
symptom may be intermittent

Unit III

41

Consequences of Bugs
infectious

damage

catastrophic
extreme
serious
disturbing
mild

annoying
Bug Type

Bug Categories: function-related bugs,

system-related bugs, data bugs, coding bugs,
design bugs, documentation bugs, standards
violations, etc.
Unit III

42

Debugging Techniques
brute force / testing
backtracking
induction
deduction

Unit III

43

Correcting the Error

Unit III

Is the cause of the bug reproduced in another part of the program? In

many situations, a program defect is caused by an erroneous
pattern of logic that may be reproduced elsewhere.
What "next bug" might be introduced by the fix I'm about to make?
Before the correction is made, the source code (or, better, the
design) should be evaluated to assess coupling of logic and
data structures.
What could we have done to prevent this bug in the first place? This
question is the first step toward establishing a statistical
software quality assurance approach. If you correct the process
as well as the product, the bug will be removed from the
current program and may be eliminated from all future
programs.

44

Final Thoughts

Unit III

Think -- before you act to correct

Use tools to gain additional insight
If youre at an impasse, get help from someone
else
Once you correct the bug, use regression
testing to uncover any side effects

45