Lecture1 - Extensive Prologue For Software Quality

An Extensive Prologue for
Software Quality
Introduction to Software Testing

Lecture Outline
Software testing motivation.

What is a software testing?
Verification and Validation?
Error , Defect and Bug
Quality Control and Quality Assurance
Why software fail?
What is cost of software failures?
What makes a good software tester?
22
© Syed Raheel Abbas- Delivered at Mohammad Ali Jinnah University
Software Testing Motivation
The Lion King Animated Storybook

Disney’s first multimedia CD-ROM game for
kids. Released at Christmas season.
26th December….. Customer Support’s
Nightmare.
The CD was testing only for specific PC
platform.
It failed on many popular PC operating system.
33
Intel Pentium Floating-Point Bug

(4195835/3145727)* 3145727 - 4195835
If answer is ZERO, your computer is fine.
Anything else, you have old Intel Pentium CPU
with floating-point division bug.
A software bug burned into a computer chip and
reproduced over and over in the manufacturing
process.
44
NASA Mars Polar Lander, 1999

3rd December 1999, Mars Polar Lander
disappeared during its landing attempt.
Failure Review Board concludes the likely failure
reason was the unexpected setting of a single
data bit.
Most alarming was why the problem was not
caught by internal tests.
55
Malaysia Airlines Jetliner, August 2005

Flight between Perth, Australia and Kuala
Lampur, Malaysia zoomed 3,000 feet upwards.
A defective software program had provided
incorrect data about the aircraft’s speed and
acceleration, confusing flight computers.
66
What is Software Testing?
The process of finding evidence of defects in

software systems.
Establishing confidence that a program does
what it is supposed to do.
1. Software testing is not debugging.

2. Software testing is not quality assurance
77
What is Software Testing?
Correctness of software with respect to

requirements or intent;
Performance of software under various
conditions;
Robustness of software, its ability to handle
erroneous input and unanticipated
conditions;
Installation and other facets of a software
release.
88
Software Testing
 To find and prevent defect
To have confidence that the system
performs adequately
To provide information about quality
To provide the insight of product risk
and quality
To ensure a program corresponds to its
specification
Defect Detection Mechanism
Software Test Process
 Test Strategy
Test Plan
Test Design
Setup Test Environment
Test Execution
Defect Tracking
Testing Tools
Evaluate Test Exit Criteria
Reporting
Improve Test Process
Some terminologies
Related to
human=Mistake
Related to product
Bug= slang / dialect
Error
Related to event
Defect, Fault, Bug Accident
Incident, Issue
Risk = likelihood of failure
x impact
Failure
Risk
Simply any thing find by tester its defect,
when the developer accepts the defect its
bug. Any thing found faulty i.e. fault is an
Summary
error.
Fault--> Error--> Defect--> Bug
12
© Syed Raheel Abbas- Delivered at Mohammad Ali Jinnah University 12
Trivial Example
Failure
For any integer n, square (n) = n*n.
int square (int x)

{ Square (3) = 6
return x*2;
} Fault
13
Trivial Example
Correct Result
For any integer n, square (n) = n*n.
int square (int x)

{ Square (2) = 4
return x*2;
}
14
Testing...Definition Revisited
Myers
Testing is the process of executing a program with
intention of finding errors
Paul Jorgensen
Testing is obviously concerned with errors, faults,
failures and incidents. A test is the act of exercising
software with test cases with an objective of Finding
failure and Demonstrate correct execution
ISO
Technical operation that consist of the determination of
one or more characteristics of a given product, process
or service according to a specified procedure
15
Verification and Validation
• Validation
– does the software do what was wanted?
– “Are we building the right system?”
– Validation is the process of evaluating software artifacts during the software
development process in an attempt to determine if the system works as
required by the customers.
– Verification
– does the software meet its specification?
– “Are we building the System right??”
– This can only be objective if the specifications are sufficiently precise
• Everything must be verified

– …including the verification process itself
Verification
Verification – Are we building the product
right?
Verification is any checking process
conducted on software artifacts in an
attempt to determine if they work as
specified by the designers of the system.
Includes reviews, inspections, walkthroughs,
unit testing and integration testing.

Validation
Validation – Are we building the right product?
Validation is the process of evaluating software
artifacts during the software development
process in an attempt to determine if the
system works as required by the customers.
Any evaluation activity that involves the
customer can be used for validation purposes.
Includes program reviews, system testing,
customer acceptance testing.

Verification vs. Validation
 Verification
 Main purpose is to detect defects
in the artifacts of the system under

development.
 Validation
 Main purpose is to show that the
system under development meets

user needs, requirements, and
expectations.
Validation and Verification
process continue in a cycle till
What is the objectives of
Software Testing?
20
20
Quality Assurance vs.
Quality Control
Quality Assurance Quality Control
A series of
An overall
analytical
management plan to
measurements used
guarantee the
to assess the
integrity of data
quality of the
(The “system”)
analytical data
(The “tools”)
Direct Objectives
To identify and reveal as many errors as
possible in the tested software
To bring the tested software, after correction
of the identified errors and retesting to an
acceptable level of quality
To perform the required tests efficiency and
effectively, within budgetary and scheduling
limitation
22
Indirect Objective
To compile a record of software errors for use
in error prevention (by corrective and
preventive actions)
23
Goals of Testing
• Goal: show a program meets its
specification
– But: testing can never be complete for non-
trivial programs
• What is a successful test?
– One in which no errors were found?
– One in which one or more errors were
found?
Goals of Testing - 2
• Testing should be:
– repeatable
– if you find an error, you want to repeat the test to
show others
– if you correct an error, you want to repeat the test to
check that you fixed it
– systematic
– random testing is not enough
– select test sets that are representative of real uses
– select test sets that cover the range of behaviors of
the program
– documented
– keep track of what tests were performed, and what
the results were
Goals of Testing - 3.
• Therefore you need a way to document
test cases showing:
– the input
– the expected output
– the actual result
• These test plans/scripts are critical to
project success!
Why do Failure Occur?
27
Root Causes of Failures
Inaccurate understanding of end user

requirements.
Inability to deal with changing requirements.
Late discovery of serious project flaws.
For example, modules that do not fit
together.
Untrustworthy build & release process.
Implementation team’s chaos.
28
Failure Costs
Disney’s Lion King CD
29
Software Testers
Good understanding of the development

process and its products.
Ability to anticipate likely faults and errors.
30
31
Software Quality Factors
● Correctness
– accuracy, completeness of required
output
– uptodateness, availability of the
information
● Reliability
– maximum failure rate
● Efficiency
– resources needed to perform software function
● Integrity
– software system security, access rights
● Usability
– ability to learn, perform required task
32
Software Quality Factors…….
● Maintainability
– effort to identify and fix software failures (modularity,
documentation, etc)
● Flexibility
– degree of adaptability (to new customers, tasks, etc)
● Testability
– support for testing (e.g. log files, automatic diagnostics,
etc)
33
Software Quality Factors…….
● Portability
– adaptation to other environments (hardware,
software)
● Reusability
– use of software components for other projects
● Interoperability
– ability to interface with other
components/systems
34
Software Testing proverbs
1. It is impossible to test a program completely.

2. Software testing is a risk-based exercise.
3. Testing cannot show the absence of bugs.
4. The more bugs you find, the more bugs there are.
5. Not all bugs found will be fixed.
6. It is difficult to say when a bug is indeed a bug.
7. Specifications are never final.
8. Software testers are not the most popular members of a
project.
9. Software testing is a disciplined and technical profession.
35
proverb 1
It is impossible to test a program
completely
How many test cases do you need to exhaustively test:
Powerpoint
A calculator
MS Word
Any interesting software!
The only way to be absolutely sure software works is to
run it against all possible inputs and observe all of its
outputs …
Oh, and the specification must be correct and complete.
36
proverb 1 (cont’d)
It is impossible to test a program
completely
The number of possible inputs is very large.
The number of possible outputs is very
large.
The number of paths through the software is
very large.
The software specification open to
interpretation.
37
proverb 2
Software testing is a risk-based exercise
If you do not test the software for all inputs (a wise
choice) you take a risk.
Hopefully you will skip a lot of inputs that work
correctly.
What if you skip inputs that cause a fault?
Risk: financial loss, security, loss of money, loss of life!
That is a lot of pressure for a tester!
This course is all about techniques and practices to
help reduce the risk without breaking the bank.
38
proverb 2 (cont’d)
Software testing is a risk-based exercise
Number of Cost of
If you try to test too Missed Bugs Testing
much, the
development cost Q
becomes prohibitive. u
Testing
a Under Equilibrium
If you test too little, n
t
Testing Over
the probability of i
Testing
software failure t
y
increases and as we
discussed … software
failures can cost us
big time! Amount of Testing
39
proverb 3
Testing cannot show the absence of bugs
“Program testing can be used to

show the presence of bugs, but never
to show their absence!”
- Edsger Dijkstra
40
proverb 4
The more bugs you find, the more bugs there
are
Bugs appear in groups, where you see one you will likely find more
… Why?
Programmers can have bad days
Programmers tend to make the same mistakes
Some bugs are just the tip of the iceberg.
Boris Beizer coined the term pesticide paradox to describe the

phenomenon that the more you test software the more immune it
becomes to your test cases.
Remedy: continually write new and different tests to exercise different parts of
the software.
41
proverb 5
Not all bugs found will be fixed
Why wouldn’t you fix a bug you knew about?
There’s not enough time
● Some deadlines cannot be extended (e.g., Y2K)
It’s not really a bug
● Specifications can be wrong
It’s too risky to fix
● “I’m not touching Shahryarzzzz code!”
It’s just not worth it
● Bugs in fringe features may have to wait
● Why not charge the customer for bug fixes in the next release
(sound familiar?) :-)
42
proverb 6
It is difficult to say when a bug is indeed a bug
If there is a problem in the software but no one

ever discovers it … is it a bug?
Parody of “if a tree falls in the forest … does it
really make a noise?”
What is your opinion? Does a bug have to be
observable in order for it to me a bug?
Bugs that are undiscovered are called latent bugs.
43
proverb 7
Specifications are never final
Building a product based on a “moving target” specification

is fairly unique to software development.
Competition is fierce
Very rapid release cycles
Software is “easy” to change
Not true in other engineering domains

E.g., the Brooklyn Bridge could not be adjusted to allow train traffic to
cross it once its construction started.
44
proverb 8
Software testers are not the most popular
members of a project
Goal of a software tester:
Find bugs
Find bugs early
Make sure bugs get fixed
Tips to avoid becoming unpopular:

Find bugs early
Temper your enthusiasm … act in a professional manner
Don’t report just the bad news
45
proverb 9
Software testing is a disciplined and technical
profession
When software was simpler and more manageable software
testers were often untrained and testing was not done
methodically.
Contrary to the urban legend, if you hire a million monkeys and have them
test for a million years … you will not find all of the bugs in your software.
It is too costly to build buggy software. As a result testing has

matured as a discipline.
Sophisticated techniques
Tool support
Rewarding careers
46
Key Points
Software testing is the process of discovering

evidence of defects and failures in software
systems.
Test early, test often, test enough.
Testers should have good understanding of
the development process, product.
47
You now know …
… the 9 proverbs of software testing
… what is software verification
… what is software validation
… the relationship between software quality and
reliability
… the relationship between Quality Control and quality
assurance or Software Testing and Software Quality
Priority and Severity of Defect.
48

Lecture1 - Extensive Prologue For Software Quality

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Lecture1 - Extensive Prologue For Software Quality

Transféré par

Droits d'auteur :

Formats disponibles

An Extensive Prologue for

Introduction to Software Testing

Software testing motivation.

The Lion King Animated Storybook

Intel Pentium Floating-Point Bug

NASA Mars Polar Lander, 1999

Malaysia Airlines Jetliner, August 2005

The process of finding evidence of defects in

1. Software testing is not debugging.

Correctness of software with respect to

Fault--> Error--> Defect--> Bug

For any integer n, square (n) = n*n.

int square (int x)

For any integer n, square (n) = n*n.

int square (int x)

• Everything must be verified

© Syed Raheel Abbas- Delivered at Mohammad Ali Jinnah University

© Syed Raheel Abbas- Delivered at Mohammad Ali Jinnah University

in the artifacts of the system under

system under development meets

Inaccurate understanding of end user

Disney’s Lion King CD

Good understanding of the development

1. It is impossible to test a program completely.

“Program testing can be used to

Boris Beizer coined the term pesticide paradox to describe the

If there is a problem in the software but no one

Building a product based on a “moving target” specification

Not true in other engineering domains

Tips to avoid becoming unpopular:

It is too costly to build buggy software. As a result testing has

Software testing is the process of discovering

Vous aimerez peut-être aussi