Software Testing

Software Testing Foundations

 Agenda
1. ISTQB and IEEE

2. Quality, Testing, QA, QC

3. Testing Principles

4. Who is who in the team?

5. Software Development Life Cycle

6. Specification

7. Why do we test?

8. Common misconceptions

9. The role of Software QA

 ISTQB, IEEE
• ISTQB - The International Software Testing Qualifications Board (ISTQB) is a software testing
qualification certification organization that operates internationally. Founded in Edinburgh in
November 2002, ISTQB is a non-profit association legally registered in Belgium.

• IEEE - The Institute of Electrical and Electronics Engineers (IEEE) is a professional association with
its corporate office in New York City and its operations center in Piscataway, New Jersey.
 Quality

What is quality?
 Quality
Definitions of Quality:

By ISTQB:
The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.

By IEEE:
• The degree to which a system, component, or process meets specified requirements.
• The degree to which a system, component, or process meets customer or user needs or expectations.
 QA, QC, Testing
Quality Assurance:
• A set of activities designed to ensure that the development and/or maintenance process is adequate to ensure a
system will meet its objectives.

Quality Control:
• A set of activities designed to evaluate a developed work product.

Testing:
• The process of executing a system with the intent of finding defects.
 QA, QC, Testing
Quality
Assurance
Definition Activities which ensure the
implementation of
processes, procedures and
standards in context to
verification of developed
software and specified
requirements.
Focuses on Focuses on processes and
procedures rather then
conducting actual testing
on the system.
Relate to Process oriented
activities.
Goal Prevention
Quality Control
Activities which ensure the
verification of developed
software with respect to
documented (or not in
some cases) requirements
Focuses on actual testing
by executing Software with
intend to identify
bug/defect through
implementation of
procedures and process.
Product oriented
activities.
Detection
Testing
Activities which ensure the
identification of
bugs/error/defects in the
Software.
Focuses on actual testing
Product oriented
activities.
Detection
 Audit

Audit
Standardization organizations

- ISO (International Organization for Standardization)

- IEC (International Engineering Consortium)
- IEEE (Institute of Electrical and Electronics Engineers)
 Testing vs Debugging
Testing Debugging
Purpose Finding and locating of a Identifying, isolating and
defects in a Testing phase fixing the defects/bug
Executed by Done by Testing Team Done by Development
Team
Goal To find as many defects To remove the defects
as possible
 Testing Principles
7 principles of testing

 Testing shows presence of defects

 Exhaustive testing is impossible
 Early testing
 Defect clustering
 Pesticide paradox
 Testing is context depending
 Absence of errors fallacy
 Who is who in the team
1. QAs
2. Developers
3. Business Analysts
4. Project Manager
5. Solution Designed/Solution Architect/Solution Strategist
6. Technical Writers
SDLC
Example
 Cost of errors
• The later the a fault is found,
the higher the cost

• Small error in specification can result

in huge error in production

• Error in production can

ruin the company
 Examples
1. In October of 1999 the $125 million NASA Mars Climate Orbiter spacecraft was believed to be lost in
space due to a simple data conversion error. It was determined that spacecraft software used certain
data in English units that should have been in metric units.

2. A small town in Illinois received an unusually large monthly electric bill of $7 million in March of 1999.
This was about 700 times larger than its normal bill. It turned out to be due to faults in new software
that had been purchased by the local power company to deal with Y2K software issues.

3. In early 1999 a major computer game company recalled all copies of a popular new product due to
software problems. The company made a public apology for releasing a product before it was ready
 Mistake and Error
• Humans make mistakes: No one is perfect. We all make mistakes or omissions

• Aggressive schedules and pressure: The more pressure we are under the more likely we are
to make mistakes. In IT development we have time and budgetary deadlines to meet

• Poor Communication: Poor Training. Poor Communication

• Poor Documentation: Requirements not clearly defined. Requirements change & requirements
not properly documented. Data specifications not complete

• Assumptions
 Specification
• Description of how software should work/look

- A document that specifies, ideally in a complete, precise and

verifiable manner, the requirements, design, behavior, or other

characteristics of a component or system, and, often, the procedures

for determining whether these provisions have been satisfied. [IEEE 610]

• Different types of documents

- Use cases, User Stories, Technical Design, Mock-ups

• Bug in specification

• Lack of specification
 TROUBLESHOOT!

• Try to find where is the problem

• What caused the problem

• How to prevent its reappearance

 Why do we test?

• Find defects before the customer

• Measure quality

• Gain confidence in the level of quality

• Increase reliability

• Increase profitability
 Misconceptions
• MYTH: Software quality depends on the testing team
FACT: Software quality depends on the development team
• MYTH: The objective of Testing is to ensure a 100% defect- free product.
FACT: The objective of testing is to uncover as many defects as possible while
ensuring that the software meets the requirements. Identifying and getting rid of all
defects is impossible.
• MYTH: Testing is easy.
FACT: Testing can be difficult and challenging (sometimes, even more so than
coding).
 Misconceptions
• MYTH: Anyone can test.
FACT: Testing is a rigorous discipline and requires many kinds of skills.
• MYTH: There is no creativity in testing.
FACT: Creativity can be applied when formulating test approaches, when designing tests,
and even when executing tests.

• MYTH: Automated testing eliminates the need for manual testing.

FACT: 100% test automation cannot be achieved. Manual Testing, to some level, is always
necessary.
 Misconceptions

MYTH: Quality Assurance = Testing

 Tester vs QA
Tester QA Engineer
• Takes part in all stages of SDLC and TLC
• Participates in the testing phase of the project • Determines testing strategy and approach
• Writes and verifies defects • System technical knowledge
• Test case execution • Gives suggestions and ideas how to improve the
• Reports results product
• Facilitates implementation of processes and procedures
 The role of QA
What is special about QA?
• Broad competence
• Special attitude

Levels of testing team independence

• No testing team – developers do the testing
• Test specialists as part of development team
• Separate testing team inside the organization (internal)
• Separate testing team outside the organization (external/outsource)
QUESTIONS?