Software Testing

SEN-269 : Software Engineering

Tazeen Muzammil
 Testing
A Process of evaluating a particular
product to determine whether the
product contain any defects

“Testing proves that the system works correctly

as it was intended to and that it demonstrates
that the developers have understood and met
customer requirements” (Tayntor, 1998). 
 Software Testing
Software Testing is a process of evaluating a system by
manual or automatic means and verify that it satisfies
specified requirements or identify differences between
expected and actual results.
• “Testing is the process of exercising or evaluating a system or system
component by manual or automated means to verify that it satisfies
specified requirements, or to identify differences between expected and
actual results." (IEEE)

• Testing is not just another phase in the system

development life cycle (SDLC); it is part of all phases.

• Software testing is the most visible activity in the

Software Quality Assurance, which is, the most critical
activity in a software project.

• Software Testing is the process of executing a program or

system with the intent of finding errors.
 Software Testing
• Testing is an integral part in software
development.
• It is broadly deployed in every phase in the
software development cycle.
• Typically, more than 50% percent of the
development time is spent in testing.
• Testing is usually performed for the following
purposes:
– To improve quality.
– For Verification & Validation (V&V)
– For reliability estimation
– Defect Detection
 Testing Principles
All tests should be traceable to customer
requirements.
Tests should be planned long before testing
begins.
Exhaustive testing is no possible.
Testing should begin “in the small” and progress
towards “ in the large”
Should be conducted by independent third
party.

5
 The V-Shaped Model
• V-Model evolved from waterfall Model.
• Each phase must be completed before the next phase
begins.
• Instead of moving down in a linear way, the process
steps are bent upwards after the coding phase, to form
the typical V shape.
• Testing is emphasized in this model more than in the
waterfall model.
• It is a structured approach to testing.
• Brings high quality into the development of our
products.
 Steps in the V-Shaped Model

Quality is guaranteed at each project stage.

 When to start Testing
• Testing early in the life cycle reduces the errors.
• The number one cause of Software bugs is the
Specification/Requirements.
• As soon as we have a single requirement in our net
we can start testing. The aim is to trap requirements-
related defects as early as they can be identified.
• We prevent incorrect requirements from being
incorporated in the design and implementation
where they will be more difficult and expensive to
find and correct.
• The next largest source of bugs is the Design.
• The other category is the catch-all for what is left.
 When to Stop Testing
• Many modern software applications are so complex,
and run in such as interdependent environment, that
complete testing can never be done.
• Common factors in deciding when to stop are:

– Deadlines (release deadlines, testing deadlines)

– Test cases completed with certain percentages passed
– Test budget depleted
– Coverage of code/functionality/requirements reaches a
specified point
– The rate at which Bugs can be found is too small
– Beta or Alpha Testing period ends
– The risk in the project is under acceptable limit.
Software Testing Types

STATIC DYNAMIC

Logical Errors

Structural Testing Functional

Testing
(White Box
testing) (Black Box
testing)
Nothing else but
functioning
 Static Testing
Static testing is also called static analysis.
The Verification activities fall into the category of
Static Testing.
During static testing, you have a checklist to check
whether the work you are doing is going as per the
set standards of the organization. These standards
can be for Coding, Integrating and Deployment.
Static testing methodologies:
Reviews
Inspection's and
Walkthrough's
 Dynamic Testing
Dynamic testing is sometime called software
testing or simply dynamic analysis.
Dynamic Testing involves working with the
software, giving input values and checking if the
output is as expected.
These are the Validation activities.
Dynamic Testing methodologies:
Unit Tests
Integration Tests
System Tests and
Acceptance Tests
 Static v/s Dynamic
Static
Static testing is about prevention
Static testing is a form of software
testing where the software isn’t
actually used.
It is generally not detailed testing,
but checks mainly for the sanity of
the code, algorithm, or document. It
is primarily syntax checking of the
code or and manually reading of the
code or document to find errors
Dynamic
Dynamic testing is about
cure.
In dynamic testing the
software must actually be
compiled and run.
Dynamic analysis refers to the
examination of the physical
response from the system to
variables that are not
constant and change with
time
Functional Testing (Black Box
testing)
Black Box Testing is a testing strategy, which
does not need any knowledge of internal design
or code etc.
The types of testing under this strategy are
totally based/focused on the testing for
requirements and functionality of the work
product/software application.
Black box testing is sometimes also called as
"Opaque Testing", "Functional/Behavioral Testing"
and "Closed Box Testing”.
In order to implement Black Box Testing Strategy,
the tester is needed to be thorough with the
 Black Box testing
Advantages
Black box tests are reproducible.
The environment the program is running is also tested.
The invested effort can be used multiple times.
Disadvantages
The results are often overestimated.
Not all properties of a software product can be tested.
The reason for a failure is not found.
Structural Testing (White Box
testing)
White box testing strategy deals with the internal logic and
structure of the code.
White box testing is also called as glass, structural, open box
or a clear box testing.
The tests written based on the white box testing strategy
incorporate coverage of the code written, branches, paths,
statements and internal logic of the code etc.
In order to implement white box testing, the tester has to
deal with the code and hence is needed to possess
knowledge of coding and logic i.e. internal working of the
code.
White box test also needs the tester to look into the code and
find out which unit/statement/chunk of the code is
malfunctioning.
 White Box testing
Advantages
As the knowledge of internal coding structure is prerequisite, it becomes very
easy to find out which type of input/data can help in testing the application
effectively.
The other advantage of white box testing is that it helps in optimizing the code
It helps in removing the extra lines of code, which can bring in hidden defects.

Disadvantages
As knowledge of code and internal structure is a prerequisite, a skilled tester is
needed to carry out this type of testing, which increases the cost.
And it is nearly impossible to look into every bit of code to find out hidden
errors, which may create problems, resulting in failure of the application.
 Gray Box Testing
Gray Box Testing is a software testing method which is a combination of Black Box
Testing method and White Box Testing method.

In Gray Box Testing, the internal structure is partially known. This involves having
access to internal data structures and algorithms for purposes of designing the test
cases, but testing at the user, or black-box level.

Gray Box Testing is named so because the software program, in the eyes of the
tester is like a gray/semi-transparent box; inside which one can partially see.

EXAMPLE

An example of Gray Box Testing would be when the codes for two units/modules are
studied (White Box Testing method) for designing test cases and actual tests are
conducted using the exposed interfaces (Black Box Testing method).
 Test Case Design

Rule: Cover all factors but do not make

too many test cases.
White Box Testing: Test cases are derived to
ensure that all statements in the program have
been executed at least once.
Black Box Testing: Test designed to validate
functional requirements without regard to
internal working of the program

19
Levels of Software Testing
 Unit Testing

Exercising the smallest individually

executable code units.
Objectives:
Find faults in the units.
Assure correct functional behavior of units.
Usually performed by programmers.
 Integration Testing
Exercising two or more units or components.
Objectives:
Detect interface errors.
Assure the functionality of combined units.
Performed by programmers or testing group.
Issues:
Strategy for combining units?
Compatibility with third-party components?
Correctness of third-party components?
Types of Integration Testing:
Top Down Integration
Bottom Up Integration
 System Testing
Exercising the functionality, performance,
reliability, and security of the entire system.
Objectives:
Find errors in the overall system behavior.
Establish confidence in system functionality.
Validate non-functional system requirements.
Usually performed by a separate test group.
Validation Testing/Acceptance
Testing
Operating the system in the user
environment with standard user input
scenarios.
Objectives:
Evaluate whether the system meets the
customer criteria.
Determine whether the customer will accept the
system.
Usually performed by the end user.
 Regression Testing
Testing modified versions of a previously
validated system.
Objective: Assuring that changes to the
system have not introduced new errors.
Performed by the system itself or by a
regression test group.
Capture / Replay (CR) tools
 Testing categorization

Fault-directed testing:
Unit testing
Integration testing

Conformance-directed testing:
System testing
Acceptance testing
Verification and Validation

Assuring that a software system

meets a user's needs

27
Verification vs validation

Verification:
"Are we building the product right"
The software should conform to its
specification
Validation:
"Are we building the right product"
The software should do what the user really
requires

28
 The V & V process

Is a whole life-cycle process - V & V

must be applied at each stage in the
software process.
Has two principal objectives
The discovery of defects in a system
The assessment of whether or not the
system is usable in an operational situation.

29
 Static and dynamic
verification
Software inspections Concerned with
analysis of the static system representation
to discover problems (static verification)
May be supplement by tool-based document and
code analysis
Software testing Concerned with exercising
and observing product behavior (dynamic
verification)
The system is executed with test data and its
operational behavior is observed

30
Static and dynamic V&V

31
 V& V goals
Verification and validation should
establish confidence that the software is
fit for purpose
This does NOT mean completely free of
defects
Rather, it must be good enough for its
intended use and the type of use will
determine the degree of confidence that
is needed

32
 Testing and Debugging
Defect testing and debugging are distinct processes
Verification and validation is concerned with
establishing the existence of defects in a program
Debugging is concerned with locating and repairing
these errors
Debugging involves formulating a hypothesis about
program behavior then testing these hypotheses to find
the system error
The Debugging Process

34
 Reading Assignment

Equivalence Partitioning
Boundary Value Analysis
Comparison Testing
Orthogonal Array Testing