Software Verification and

Validation
Testing Fundamentals
1
Testing Fundamentals
Lecture Overview
Basic concepts and goals of software testing
Testing process
Defects and software quality
Creating effective test cases
2
Automated and manual testing
General testing strategies
Testers tasks and roles
Debugging process and techniques
Fault model and root cause analysis for defect prevention
Cost of software quality
Errors, Defects, Failures, and Reliability
Errors
Mistakes, misconceptions, or misunderstandings on the part of a
software developer.
Different types of errors can be made in each phase of the software development
process, e.g., requirements errors, design errors, and code errors.
Defects (or faults)
3
Introduced into the software as the result of an error that may cause it
to behave incorrectly, not according to its specification, and not as
intended.
Failures
The inability of a software or component to perform its required
functions within specified performance requirements
Reliability
The ability of a system or component to perform its required functions
under stated conditions for a specified period of time
Software Testing
Software testing
The process of analyzing and evaluating the features of the software item to
detect possible defects and to establish that the software has attained a
specified degree of quality with respect to selected attributes.
Verification
The process of evaluating a system or component to determine whether the
products of a given development phase satisfy the conditions imposed at the
start of that phase [IEEE-STD-610].
The process of checking that a software system meets specifications.
4
The process of checking that a software system meets specifications.
The human based activities for verifying design, code related artifacts such as design
documents and files through inspections and reviews
Are we building the product right [Boehm89]?
Validation
The process of evaluating a system or component during or at the end of the
development process to determine whether it satisfies specified requirements
[IEEE-STD-610].
The process of checking that a software fulfills its intended purpose (meeting users
needs).
The execution based checking and testing of the real product
Are we building the right product [Boehm89]?
Primary Goals of Testing
Primary goals of testing
To check if the software meets specified requirements and customer/
users expectations by:
detecting defects related to functions and quality
determining the software quality checking quality attributes
Some examples of quality attributes:
5
Some examples of quality attributes:
Correctness
Reliability
Usability
Maintainability
Portability
Interoperability
Integrity
Importance of Software Quality
Software quality affects:
Money
Risk
Security
Liability
People bet their jobs, their comforts, their safety, their
entertainment, their decisions, and their very lives on
computer software. It better be right.
Sources of Defects
Sources of defects
Lack of education, poor
communication, oversight,
mistakes while performing tasks,
especially in an immature process
Defects can be injected in every Defects can be injected in every
phase of the development.
Defects injected in early phases
can persist and be very difficult
and costly to remove in later
phases.
7
Major Defect Classes
Requirements defects
Functional description, feature, feature interaction, interface description
Ambiguous, contradictory, imprecise, incomplete, missing, redundant, incorrect
Design defects
Low level design such as algorithmic design, logical flow design, etc.
High level design such as architecture design for module interface or
external interface, data design, etc.
The design may be incorrect, poor, or not detailed enough for programmer. The design may be incorrect, poor, or not detailed enough for programmer.
Coding defects
Algorithmic and processing, control, logic, and sequence, data, code
documentation, etc.
Testing defects
Test harness, test design, test procedure
Other defects from the development process
Process defects due to immature process
Some defects (found in previous phases) may be related to all
development phases.
8
Relative Cost of Correcting Defects
The relative costs to find and repair defects increase dramatically as
we go from prevention to detection and to internal failure to external
failure costs.
Prevention costs include
quality planning
technical reviews
test equipment test equipment
training
Internal failure costs
include
rework
repair
failure mode analysis
External failure costs are
complaint resolution
product return and
replacement
help line support
warranty work
Testability of Software
Testability is how easily a program can be tested.
Developers should design and implement software with testability in
mind.
A well-defined process should be established.
Testability characteristics
Understandabilityare requirements and design easy to understand?
Observabilityit operates cleanly, so the results of each test case are
easily observable
Controllability/automatabilitytesting can be automated
Isolateability/decomposabilityis isolated testing possible?
Stabilityfew changes are requested