Black Box Test Techniques - Equivalence

Partitioning and Boundary Value Analysis

Author: Ian R McDonald

2010, 2013 Ian McDonald

Overview
In this presentation you will learn about Dynamic Testing Techniques.
This is where data is injected and the outputs monitored for the expected output.
The method is dynamic, since we have to supply data to the system under test.
We can design our test data with no understanding of the internal working of the
system called a Black Box Technique.
Alternatively we may choose to structure our data based on a full knowledge of the
internal workings called a White Box Technique. Sometimes also known as Clear
Box, Glass Box, Structural or Crystal techniques.
Another approach is to use our experience an judgement to guess what is the best
values to use for data called Error Guessing.
In this unit we will learn about 2 specific Black Box techniques to design tests. These
are:
1.

Equivalence Partitioning

2.

Boundary Value Analysis

Objectives
By the end of this presentation you should be able to:
1. Understand the terms:
1. Dynamic Testing
2. Black Box
3. Equivalence Partitioning
4. Boundary Value Analysis
2. Understand when it is appropriate to use Black Box techniques
3. Be able to identify equivalence partitions within data.
4. Be able to identify boundary values for a system under test.
5. Use material from this unit and reference it to BS7925-1 and BS7925-2

Dynamic Analysis
Dynamic Analysis is the process of evaluating a system or component
based upon its behaviour during execution BS7925-1 adopted from the
IEEE (Institute of Electrical and Electronics Engineers)
We use Dynamic Analysis techniques to define test inputs and matching
expected outputs to exercise a software component or a full system.
There are several Black Box techniques in designing data for testing:

Equivalence Partitioning
Boundary Value Analysis
Cause-effect Graphing (not covered here)
Other techniques e.g. Classification Tree Analysis (not covered here)

Note: The opposite of a Dynamic technique is a Static Technique. Static

Testing is the term applied in reviewing code and documents, where the
programme is not executed (i.e. the programme is in a static state).

Test Techniques
We apply test techniques to ensure:
1. To ensure a systematic approach to testing so that we use the
minimum number of tests to achieve as much testing as possible in a
cost affective way. In life critical systems we want to ensure that we have
not missed out any specific values.
2. We want to ensure that tests are repeatable. We may have to justify to
our managers, customers and even a court that we tested a specific set
of values.
3. We want to provide confidence that we are applying a well structured
testing approach.
NOTE: We can use tools to help us in defining tests. E.g. the
Classification Tree Approach used the Classification Tree Editor (CTE),
which is covered in a separate presentation.
Black box approaches to testing require no knowledge of the internal
working of an application. In contrast White box approaches do require
knowledge. White box approaches can sometimes be called clear box,
transparent or crystal approaches.

Functional Test Case Design

A Test Case is a set of inputs, execution preconditions, and

expected outcomes developed for a particular objective, such as
to exercise a particular programme path or to verify compliance
with a specific requirement. BS7925-1
Functional Test Case Design, selects the test cases that is based
on an analysis of the specification of the component without
reference to the internal workings. BS7925-1
Black Box testing is the use of functional test case design, using
the requirements, component and system specifications and
other documentation describing what is required of the product
under development.

Black Box Testing Exercise

For this exercise, you are a
salesman in a television
shop. A customer comes
into your shop and asks you
to demonstrate that the TV
they intend to purchase
works.
To see if the TV works, do
you need to know what
goes on inside the set?
What tests do you think you
will want to include?
How could we be
systematic in our test
approach?

Black Box Testing Exercise Suggested Answers

There are a range of possible answers including:

Check standby light operates from off to standby, standby to on
and on to standby.
Check the control panels to verify, volume, brightness, contrast,
colour.
Check main terrestrial TV stations.
Check Digital TV channels.
Check headphones socket functions.
Check other sockets for format eg can you plug a video recorder
into TV?

Notes:
If checking Volume level, you may want a decibel meter and
specify the signal strength at the aerial input.
Would you want to check any ranges for each identified input?
8

Smoke Test

You may come across the term smoke test.

This was originally applied to civil engineering when smoke was put down
a pipe and checks could be quickly made for more obvious leeks.
In hardware engineering, a quick way to check if all the power lines were
correctly set up, electrolytic capacitors set up the right way, would be to
switch on the circuit and see if any smoke appears.
Smoke tests provide initial confidence, they are however no substitute for
detailed testing. Sometimes a test manager will do a smoke test before
accepting a build into detailed testing. This provides confidence that the
build is at an appropriate state to enter system test.
The TV could be turned on and we can check for a picture of some sort
and for sound. This does not mean that the sound matches the picture,
or you can change channel.
Would you be happy with a medical system or an aircraft that had only
been smoke tested? Probably and most likely no. In the case of the TV,
they will have been tested using automation before leaving the factory.
9

Black Box Testing

Black

Box testing

Focuses

on WHAT a system does not HOW it does it.

Focuses on the functional capabilities of the system
Is also known as functional testing.

Data input

Black Box

Actual Results

If the actual results are as expected then the test has passed
10

Where do we use Black Box Testing

Black Box testing is used in the following stages of the software

development life cycle.

Component Testing
Integration of Components (called integration in the small)
Systems Testing
Systems Integration (called integration in the large)
User Acceptance Testing (UAT)
Assisting in the business verification and validation of the application
Operational Acceptance Testing (OAT)
In short you can use Black Box analysis at every phase. White box
testing however helps supplement the testing analysis for code
development and integration (in the small).

11

Equivalence Partitioning #1
It

is very difficult, expensive and time

consuming if not at times impossible to test
every single input value combination for a
system.
We can break our set of test cases into subsets. We then choose representative values
for each subset and ensure that we test these.
Each subset of tests is thought of as a partition
between neighbouring subsets or domains.
12

Equivalence Partitioning #2
Equivalence

Partitioning:

Makes use of the principle that software acts in a

general way (generalises) in the way it deals with
subsets of data,
Selects Groups of inputs that will be expected to be
handled in the same way.
Within a group of data, a representative input can be
selected for testing.
For many professional testers this is fairly intuitive.
The approach formalises the technique allowing an
intuitive approach to become repeatable.

13

Equivalence Partitioning #3

EP Example:
Consider a requirement for a software system:

The

customer is eligible for a life assurance

discount if they are at least 18 and no older
than 56 years of age.

For the exercise only consider integer years.

14

Equivalence Partitioning #4
The

customer is eligible for a life assurance discount if they are at

least 18 and no older than 56 years of age.

18

56

Invalid Partition

Valid Partition

Invalid Partition

Less than or
equal to 18

Range 19 to 56

Greater than 56

15

Equivalence Partitioning #5

What if our developer incorrectly interpreted the requirement as:

The customer is eligible for a life assurance discount if they are over 18 and less than 56 years of age.
People aged exactly 18 or exactly 56 would now not get a discount.

18

56

Invalid Partition

Valid Partition

Invalid Partition

=< 18

18< Range <56

<56

Errors are more common at boundary values, either just

below, just above or specifically on the boundary value.
16

Boundary Analysis #1

The customer is eligible for a life assurance discount if they are at

least 18 and no older than 56 years of age.

17, 18, 19

55, 56, 57
Boundaries

Invalid Partition

Valid Partition

Invalid Partition

Less than 18

Range 17 to 56

Greater than 56

Test values would be: 17, 18, 19, 55, 56 and 57.
This assumes that we are dealing with integers and so least significant digit
is 1 either side of boundary.
17

Boundary Analysis #2
For

each boundary we test +/- 1 in the

least significant digit of either side of the
boundary.
Boundary Limit

Boundary Limit -1

Boundary Limit + 1

If significant digit was second decimal place, then the

limits above would be +/- 0.01
18

Boundary Analysis #3

While the textbooks may limit testing to the

boundaries, we are interested in how software
normally behaves and how it reacts to handling error
conditions. Therefore it is normal to treat NOT ONLY
the boundaries but also:
A typical mid range value e.g. 37
Zero (since divide by 0 errors can occur).
Negative values
Numbers out of range by a long way e.g. +/-1000
Illegal data entries like nineteen as letters, Fred,
banana.
Illegal characters such as # $ & @ : ;
19

Taking EP and BVA Further

Consider the following requirement:

The customers must be at least 18. Customers are eligible for a
life assurance discount of 40% if they are at least 18 and no older
than 25 years of age. Customers are entitled to a 30% discount if
they are older than 25 years of age, but under 40. Customers are
entitled to a 10% discount if they are 40 or over, but no older than
56. Over 56 customers are not entitled to a discount.

What are the equivalence partitions?

What are the boundary values to be tested?
What other values might you test?

20

Taking EP and BVA Further Answer

The customers must be at least 18. Customers are eligible for a life
assurance discount of 40% if they are at least 18 and no older than 25
years of age. Age is only recorded in integer years. Customers are
entitled to a 30% discount if they are older than 25 years of age, but under
40. Customers are entitled to a 10% discount if they are 40 or over, but no
older than 56. Over 56 customers are not entitled to a discount.

17, 18, 19
invalid

24, 25, 26

40%
discount

38, 39, 40

10%
discount

55, 56, 57

10%
discount

0%
discount

Might also test: 0, -5, 200, Fred, 0.00000001, some typical mid
range values: 21, 32, 47. Note boundary values tested +/- least
significant recorded digit.
21

Invalid Partitions and ISEB / ISTQB Examination

Note

when taking the ISEB / ISTQB

examinations, they may specifically
require that you identify the valid
partitions and not the invalid partitions.
Be aware of this when studying for the
examination and consult the syllabus for
specific instructions.

22

Question 1
One

of the fields on a form contains a

text box which accepts numeric values in
the range of 18 to 26. Identify the invalid
Equivalence class.
a) 17
b) 19
c) 25
d) 21
23

Solution 1

The text box accepts numeric values in the range 18

to 25 (18 and 25 are also part of the class). So this
class becomes our valid class. But the question is to
identify invalid equivalence class. The classes will be
as follows:
Class I: values < 18 => invalid class
Class II: 18 to 25
=> valid class
Class III: values > 25 => invalid class
17 fall under invalid class. 19, 25 and 21 fall under
valid class.
So answer is a (17)
24

Question 2
In

an Examination a candidate has to score

minimum of 25 marks in order to pass the
exam. The maximum that he can score is 50
marks. Identify the Valid Equivalence values if
the student passes the exam.
a) 22,24,27
b) 21,39,40
c) 29,30,31
d) 0,15,22
25

Solution 2
The

classes will be as follows:

Class I: values < 25 => invalid class
Class II: 25 to 50
=> valid class
Class III: values > 50 => invalid class
We have to indentify Valid Equivalence
values. Valid Equivalence values will be
there in Valid Equivalence class.
All the values should be in Class II.
So answer is c ( 29,30,31)
26

Ariane 5 Lessons Learned

Take a look on YouTube the Ariane 5 Takeoff e.g:
http://www.youtube.com/watch?v=c9Hf4qTxdxs
The video shows the takeoff of Arianne 5. The rocket used the
navigation code of Ariane 4. However as the code was from a
working rocket system it was not fully tested in the context of the new
Arianne 5 it was assumed there would be no surprises.
Normally we would want to test the interface between new and
existing code and we would want to use boundary value analysis as
part of this.
As the rocket took off one partition set of values was specifically
reached.

27

Testing is about reducing risk

Remember testing is about reducing risk to an acceptable level.

Not removing all risk.
Targeting testing using systematic design techniques, such as
equivalence partitions and boundaries significantly, traps more
error conditions for fewer tests.
Boundaries between software interfaces presents a significant
risk. Do not assume that new code will always work with well
established older interfaces.
Typically within projects test analyst specialists are brought onto
projects too late. They need to be present from day one, when
producing requirements.
Testing can also be underestimated by project managers. In the
case of Arianne 5, this was a project that was delivered to time at
the launch site and was successful. However was it really
successful? A wise project manager mitigates risk early and
invests in testing.
28