Data flow testing

SV thực hiện:
1.Nguyễn Thị Trâm Anh
2.Lê Nguyễn Thùy Dương
3.Ông Mỹ Linh
4.Võ Thị Minh Hòa
Ch 13, slide 1
 Learning objectives
• Understand why data flow criteria have been
designed and used
• Recognize and distinguish basic DF criteria
– All DU pairs, all DU paths, all definitions
• Understand how the infeasibility problem
impacts data flow testing
• Appreciate limits and potential practical uses of
data flow testing

Ch 13, slide 2
 Motivation
• Middle ground in structural testing
– Node and edge coverage don’t test interactions
– Path-based criteria require impractical number of
test cases
• And only a few paths uncover additional faults, anyway
– Need to distinguish “important” paths
• Intuition: Statements interact through data
flow
– Value computed in one statement, used in another
– Bad value computation revealed only when it is used

Ch 13, slide 3
 Outline
• What’s data flow testing?
• Definition-Use Associations
• Data Flow Testing Criteria
• Data Flow Coverage with Complex Structures
• The Infeasibility Problem
• Data Flow Guidelines

Ch 13, slide 4
 Outline
• What’s data flow testing?
• Definition-use associations
• Data flow testing criteria
• Data flow coverage with complex structures
• The infeasibility problem
• Data Flow Guidelines

Ch 13, slide 5
 What’s Data Flow Testing?
• Data flow testing: Testing in which test cases
are designed based on variable usage within the
code
BS 7925-1.British Computer Society Specialist
Interest Group in Software Testing (BCS SIGIST)

• Data Flow Testing bases on white-box

technique

Ch 13, slide 6
 What’s white-box technique?
• Tạo kiểm thử chủ yếu nhờ vào phân tích cấu trúc
của các thành phần hay hệ thống
• Tìm kiếm lỗi theo cách mà hệ thống được xây
dựng
• Bao gồm:
– Cấu trúc hệ thống (mã, thiết kế…) thường dẫn tới kịch
bản kiểm thử
– Quy mô bao phủ kiến trúc
– Kiểm thử xa hơn có thể được dẫn ra có hệ thống hóa
để gia tăng khả năng bao phủ

Ch 13, slide 7
 Outline
• What’s data flow testing?
• Definition-use associations
• Data flow testing criteria
• Data flow coverage with complex structures
• The infeasibility problem
• Data Flow Guidelines

Ch 13, slide 8
 Terms
• DU pair: a pair of definition and use for some
variable, such that at least one DU path exists
from the definition to the use
x = ... is a definition of x
= ... x ... is a use of x
• DU path: a definition-clear path on the CFG
starting from a definition to a use of a same
variable
– Definition clear: Value is not replaced on path
– Note – loops could create infinite DU paths between
a def and a use
Ch 13, slide 9
 Defs, uses examples
• 1 public int gcd(int x, int y) { /* A: def x,y
*/
• 2 int tmp; /* def tmp */
• 3 while (y != 0) { /* B: use y */
• 4 tmp=x%y; /* C: use x,y, def tmp */
• 5 x=y; /* D: use y, def x */
• 6 y=tmp; /* E: use tmp, def y */
• 7}
• 8 return x; /* F: use x */
• 9}
Ch 13, slide 10
Defs, uses examples

Ch 13, slide 11
 Defs, uses examples
• Những dòng nào xuất hiện định nghĩa của biến i?
Dòng 1, 4 và 9
• Những dòng nào xảy ra việc sử dụng biến i?
Dòng 15 và 18

Ch 13, slide 12
 DU pairs examples
• Có bao nhiêu cặp định nghĩa – sử dụng đối với
biến i?
Có 6 cặp, nhưng chỉ có 5 cặp có thể thực hiện
được (feasible)
• Gồm:
– 1- 18 (a=35, b=32)
– 1- 15 (a=35, b=29)
– 4- 15 (a=25, b=24)
– 4- 18 (not feasible)
– 9- 15 (a=29, b=25)
– 9- 18 (a=35, b=34)

Ch 13, slide 13
 Data flow concept
• Value of x at 6 could
1
x = .... be computed at 1 or at
2
4
if ....
• Bad computation at 1
3
.... 4 x = ....
or 4 could be revealed
only if they are used at
5
... 6
6 • (1,6) and (4,6) are
y = x + ...
def-use (DU) pairs
– defs at 1,4
– use at 6
Ch 13, slide 14
DU path example

Ch 13, slide 15
DU path example

Ch 13, slide 16
DU path example

Ch 13, slide 17
DU path example

Ch 13, slide 18
DU path example

Ch 13, slide 19
 Definition-clear path
• 1,2,3,5,6 is a definition-
1
x = ....
clear path from 1 to 6
– x is not re-assigned
2 between 1 and 6
if ....
• 1,2,4,5,6 is not a
3 4
.... x = .... definition-clear path
5
from 1 to 6
... – the value of x is “killed”
(reassigned) at node 4
6
y = x + ... • (1,6) is a DU pair because
1,2,3,5,6 is a definition-
clear path

Ch 13, slide 20
 Outline
• What’s data flow testing?
• Definition-use associations
• Data flow testing criteria
• Data flow coverage with complex structures
• The infeasibility problem
• Data Flow Guidelines

Ch 13, slide 21
 Data Flow Testing Criteria
• Goal: Try to ensure that values are computed
and used correctly

Ch 13, slide 22
 Data Flow Test Criteria
• First, we make sure every def reaches a use
• Then we make sure that every def reaches all
possible uses
• Finally, we cover all the paths between defs
and uses

Ch 13, slide 23
 Data Flow Testing Criteria
• A test suite T for a program P satisfies the all
DU pairs adequacy criterion iff, for each DU
pair du of P, at least one test case T exercises
du

Ch 13, slide 24
 The all DU pairs adequacy criterion
• All DU pairs: Each DU pair exercised by at least
one test case
• The all DU pairs coverage CDU pairs of T for P is
the fraction of DU pairs of program P exercised
by at least one test case in T

Ch 13, slide 25
 The all DU paths adequacy criterion
• All DU paths: Each simple (non looping) DU path
is exercised by at least one test case
• The all DU paths coverage CDU paths of T for P is
the fraction of DU paths of program P exercised
by at least one test case in T

Ch 13, slide 26
 The all definitions adequacy criterion
• All definitions: For each definition, there is at
least one test case which exercises a DU pair
containing it
• The all definitions Cdefs of T for P is the fraction
of definitions of program P covered by at least
one test case in T

Ch 13, slide 27
 Outline
• What’s data flow testing?
• Definition-use associations
• Data flow testing criteria
• Data flow coverage with complex structures
• The infeasibility problem
• Data Flow Guidelines

Ch 13, slide 28
Data flow coverage with complex structures
• Data flow analysis approximates the effects of
program executions.
• It suffers imprecision in modeling dynamic
constructs, in particular dynamic access to
storage, such as indexed access to array
elements or pointer access to dynamically
allocated storage.

Ch 13, slide 29
Data flow coverage with complex structures
• The precision of data flow analysis depends on
the precision of alias information used in the
analysis.
• Alias analysis requires a trade-off between
precision and computational expense, with
significant overestimation of alias sets for
approaches that can be practically applied to
real programs.

Ch 13, slide 30
Data flow coverage with complex structures
• From a global analysis, that the parameters
encoded and decoded never refer to the same
or overlapping memory regions.

Ch 13, slide 31
Data flow coverage with complex structures
• A good alias analysis can greatly improve the
applicability of data flow testing but cannot
eliminate all problems.
• A good alias analysis can greatly improve the
applicability of data flow testing but cannot
eliminate all problems.

Ch 13, slide 32
Data flow coverage with complex structures
• Arrays and pointers are critical for data flow analysis
– Under-estimation of aliases may fail to include some DU pairs
– Over-estimation, on the other hand, may introduce unfeasible
test obligations
• For testing, it may be preferrable to accept under-
estimation of alias set rather than over-estimation or
expensive analysis
– Controversial: In other applications (e.g., compilers), a
conservative over-estimation of aliases is usually required
– Alias analysis may rely on external guidance or other global
analysis to calculate good estimates
– Undisciplined use of dynamic storage, pointer arithmetic, etc.
may make the whole analysis infeasible

Ch 13, slide 33
Data flow coverage with complex structures
Example
1 void pointer abuse() {
2 int i=5, j=10, k=20;
3 int *p, *q;
4 p=&j+1;
5 q=&k;
6 *p = 30;
7 *q=*q+55;
8 printf("p=%d, q=%d\n", *p, *q);
9 }

Ch 13, slide 34
Data flow coverage with complex structures
Example
• Ví dụ cho thấy các con trỏ trỏ tới các đối tượng
trong chương trình ngăn xếp có thể tạo ra định
nghĩa cơ bản sử dụng tùy ý, đặc biệt là khi kết
hợp với số học con trỏ như trong ví dụ này.

Ch 13, slide 35
 Outline
• What’s data flow testing?
• Definition-use associations
• Data flow testing criteria
• Data flow coverage with complex structures
• The infeasibility problem
• Data Flow Guidelines

Ch 13, slide 36
 Infeasible Test Requirements
• An infeasible test requirement cannot be
satisfied
– Unreachable statement (dead code)
– A subpath that can only be executed if a contradiction occurs (X
> 0 and X < 0)
• Most test criteria have some infeasible test
requirements
• It is usually undecidable whether all test
requirements are feasible
• When sidetrips are not allowed, many structural
criteria have more infeasible test requirements
• However, always allowing sidetrips weakens the
test criteria
Ch 13, slide 37
 The infeasible path

Định nghĩa :
- Cho b là 1 nhánh điều kiện với:

        +p là 1 biểu thức

        +c là 1 đường dẫn từ điểm bắt đầu của chương

trình tới true (false) out-edge của b.
 
- Path c là infeasible path nếu như biểu thức p luôn
mang giá trị true(false)

Ch 13, slide 38
The infeasible path example

Ch 13, slide 39
The infeasible path example

Ch 13, slide 40
The infeasible path example

Ch 13, slide 41
 Marking the shortest infeasible path

 
 
- Định nghĩa : 1 infeasible path e1,e2,...,e(n) là 1
shortest infeasible path nếu e2,..,e(n) không là
infeasible path

Ch 13, slide 42
 Marking the shortest infeasible path
 
[p]: có nghĩa là 1 subpath p tùy ý bao gồm trong path
 
   vd:    [3,4]: là subpath p tùy ý tồn tại trong 1 path
 
(p)*: có nghĩa là subpath p có thể không được lặp lại hoặc là
được lặp lại nhiều lần
 
  vd:    ( 14     10    11    13    6) * : là subpath p có thể được lặp
lại nhiều lần

Ch 13, slide 43
Marking the shortest infeasible path

Ch 13, slide 44
 Infeasibility
• Suppose cond has not
1
if (cond) changed between 1 and 5
• Or the conditions could be
2 3 different, but the first
.... x = ....
implies the second
4
...
• Then (3,5) is not a
(feasible) DU pair
5 if (cond) • But it is difficult or
impossible to determine
6 7 which pairs are infeasible
y = x + ... ....
• Infeasible test
obligations are a problem
• No test case can cover
them

Ch 13, slide 45
 Infeasibility
• The path-oriented nature of data flow analysis
makes the infeasibility problem especially
relevant
– Combinations of elements matter!
– Impossible to (infallibly) distinguish feasible from
infeasible paths. More paths = more work to check
manually.
• In practice, reasonable coverage is (often, not
always) achievable
– Number of paths is exponential in worst case, but
often linear
– All DU paths is more often impractical

Ch 13, slide 46
 Data Flow Guidelines
• Data Flow Testing is good for computationally
intensive program
• Define / use testing provides a rigorous,
systematic way to examine points at which
faults may occur
• Aliasing of variables causes serious problems!
• Working things out by hand for anything but
small methods is hopeless
• Compiler-based tools help in determining
coverage values
Ch 13, slide 47
 Summary
• Data flow testing attempts to distinguish
“important” paths: Interactions between
statements
• Intermediate between simple statement and branch
coverage and more expensive path-based structural testing
• Cover Def-Use (DU) pairs: From computation of
value to its use
• Intuition: Bad computed value is revealed only when it is
used
• Levels: All DU pairs, all DU paths, all defs (some use)
• Limits: Aliases, infeasible paths
• Worst case is bad (undecidable properties, exponential
blowup of paths), so pragmatic compromises are required

Ch 13, slide 48
 Tài liệu tham khảo
• Mauro PezzandMichal Young. Software Testing and
Analysis: Process, Principles and Techniques. John Wiley
& Sons 2008
• Chang Liu, Teaching Data Flow Testing in an Software
Engineering Course
• Thầy Nguyễn Ngọc Tú, Slide bài giảng môn Kiểm tra phần
mềm
• Paul Ammann,Jeff Offutt. Introduction to Software
Testing. Cambridge University Press, 2008
• http://www.cse.yorku.ca/course_archive/2009-10/W/4313
/slides/09-DataFlowTesting.pdf
• Data Flow Testing CS-399: Avanced topics in Computer
Science Mark new (321917)

Ch 13, slide 49
Thank you for your listening

Ch 13, slide 50