IEEE CONSEG 2009
A New Fuzzing Technique for
Software Vulnerability Testing
Zhiyong Wu1 J. William Atwood2
1,3
Network Information Center
University of Science and
Technology of China
Hefei, Anhui, China
Xueyong Zhu3
2
Department of Computer Science
and Software Engineering
Concordia University
Montreal, Quebec, Canada
 Contents

1. Introduction and Motivation

2. FTSG Model
3. Related Techniques
• Static analysis
• Dynamic binary instrument and dynamic trace
• I/O analysis
4. GAMutator
5. Prototype System: DXFuzzing
6. Validation
7. Experiments
8. Conclusion

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 2

 1 Introduction and Motivation
C code of a vulnerable procedure
int process_chunck(char* head_str, char* data_str, char* program checksum){
char buf[60];
char buf1[32]; knowledge-based fuzzing
char buf2[32]; could pass it easily
memset(buf, 0, 60);
if ( true == strong_check(head_str,data_str,program checksum)){
if （ strlen(head_str) > 32 || strlen(data_str) >32)
return -1;
strcpy(buf1, head_str);
one-dimension m&g strategy can’t
strcpy(buf2, data_str);
overflow if length(head_str) = 16
strcat(buf, head_str);
and length(data_str) = 20
strcat(buf, data_str);//error
return 1;
}
else
return -1;
}
2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 3
 2 FTSG Model

FTSG: Fuzzing Test Suites Generation

FTSG ＝ (s,L,N,C,F,OP,Result) ,

OP ＝ {M, Slv},

Result ＝ {sampletree, mediumtree, newtree, testcase, testsuite}.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 4

 2 FTSG: Procedure for generating test
cases by Mutation Operators and Slv
M = {m1, …, mi, …, mk, GAMutator}
F = {f1,f2, …, fe, …,fv}
for (each mi in M except GAMutator)
{
while (!(mediumtree = mi (sampletree)) )
{
newtree=Slv(mediumtree, C)
}
}
for (each fe in F)
{
while (!(mediumtree = GAMutator (sampletree, fe))
)
{
newtree=Slv(mediumtree, C)
}
}
2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 5
 2 FTSG: Total number of test
cases

k
T  testsuite   mi ( sampletree )
i 1

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 6

 3 Related Techniques:
Static analysis ， dynamic binary
instrument and dynamic trace

Technique Usage Tool

Static identify insecure IDA PRO
analysis functions
Dynamic get insecure Pin
binary functions’ dynamic
instrument input arguments
values to calculate
fitness value
Dynamic monitor buffer Pydbg
trace coverage

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 7

 3 Related Techniques:
I/O analysis

Method Instrument Characteristic

Target
static analysis source code false alarm
execution- binary code simple and
oriented precise
analysis

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 8

 3 Related Techniques:
I/O analysis: execution-oriented analysis

INPUT OUTPUT VALUE of Ok

t1 = (a1,a2,…,as,…,an) O = {o1,o2, …, ok, … on} V1

t2 = (a1,a2,…,as,…,an) O = {o1,o2, …, ok, … on} V2

t3 = (a1,a2,…,as’,…,an) O = {o1,o2, …, ok, … on} V3

xs influences output ok if and only if

V1 =V2 ≠V3
where ai∈D(xi), as’∈D(xi), as≠as’

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 9

 GAMutator
 GAMutator mutates relative l or n in sampletree
to trigger suspend vulnerability in fe.
 l or n are the inputs that influence some
arguments of fe.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 10

 Cont.
 Special Characteristics of GAMutator:
 A multi-dimension mutation operator.
 A demand-oriented operator.
 The number of test cases that GAMutator generates is
not fixed.
 Communicates with outside system.
 The genetic algorithm here is used to generate test
cases to trigger vulnerability in unsafe functions
 The number of test cases generated by GAMutator is
O(h).

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 11

 4 GAMutator:
Heuristics and fitness function

Heuristics are used to generate test cases more likely to trigger vulnerability in fe
in F.

TWO EXAMPLES:
1 strcpy( dst, src)

 size(d )
 , if len( s )  0,
f (X )   len( s)

 MAX _ DEFAULT _ FITNESS , if len( s ) ＝ 0.

2 malloc(a)
A  a , when a  A,

f ( X )  0, when a  A and (a% A)  B,
a % A  B , when a  A and ( a% A)  B.


2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 12

 5 Prototype System: DXFuzzing
1) Locate insecure functions positions in target binary code
by Program Analyzer. Record their information into
database;
2) Analyze corresponding network protocols or file format
in target application according to related knowledge,
choose a sample file s and write a primitive xml test
script manually which contains a sampletree;
3) Scheduling Engine calls XFuzzing to fuzz target
application with mi and records runtime information with
Program Analyzer when it is necessary.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 13

 Cont.

4) Data Mapper constructs relationships between

X and F based on collected runtime information.
5) Scheduling Engine calls XFuzzing to fuzz target
application with GAMutator.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 14

 6 Validation

1) Based on application-specific knowledge,

DXFuzzing could generate test cases which
easily pass strong program checks and
validations in the program.
2) The problem of finding new combinations to
trigger possible vulnerability in fe in F is
especially suitable for genetic algorithm to solve
.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 15

 Cont.

3) GAMutator does not only care about the

relationships between li and fe , but also cares
about nj and fe. Because some fe in F is
influenced by the nj, however, the nj is neglected
in general.
4) Different from combinatorial test in black-box
testing, the combination of li or nj in DXFuzzing
is decided by the I/O analysis; the values of li or
nj in some combination are refined by every
generation.
2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 16
 Cont.
 Execution-oriented I/O analysis in DXFuzzing is
preferred here.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 17

 7 Experiments
LibPng library as the target application
Some data are as follows:
Table I insecure functions in target application
Function name usePng.exe LibPng.dll v1.0.6
strcpy 1 6
memcpy 0 77
sprintf 0 16
malloc 18 113

Table II Input nodes

ID INPUT ELEMENTS
101 PngFile..IHDA_CHUNK_DATA.BitDepth
102 PngFile..IHDA_CHUNK_DATA.ColorType
109 PngFile..IHDA_CHUNK_DATA.Height
111 PngFile..IHDA_CHUNK_DATA.Width

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 18

 Cont.

Table III Insecure functions influenced by input nodes

ID INSECURE FUNCTIONS
72 pngrutil.c(2939):png_ptr-
>row_buf=(png_bytep)png_malloc(png_ptr,row_bytes)
73 pngrutil.c(2945):png_ptr-
>prev_row=(png_bytep)png_malloc(png_ptr, png_uint_32)
( png_ptr->rowbytes + 1))
89 pngread.c(1301):info_ptr-
>row_pointers=(png_bytepp)png_malloc(png_ptr,info_ptr-
>height * sizeof(png_bytep))

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 19

 Cont.

111 102 101 109

72 73 89
Figure 4. Relationships between inputs and insecure functions by static analysis
111 102 101 109

simple and precise

72 73 89

Figure 5. Relationships between inputs and outputs by dynamic execution

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 20

 Cont.

w width 111
d BitDepth 101
z Argument value of png_malloc 73

Initial Values: w = 0x20, d = 0x01 w∈[0,0xfffffff] ， d∈[0,0xff].

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 21

 Cont.
 Further analyzing, we got d∈{1,2,4}.
 w and d will generate 3×0x100000000 ＝
12884901888 combination test cases.
 However, there are only 262148 of them that
could trigger this vulnerability if we set B=100000
 For this case png_malloc could successfully
allocate memory.
 So the possibility is 262148/12884901888 =
0.00002.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 22

 Cont.

Width, BitDepth distribution when they trigger this vulnerability

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 23

 Cont.

Table IV Vulnerabilities Found by Different Fuzzing Tools

Tools Number of vulnerability checked Number of test cases
Smart Fuzzer 0 1000000
GAFuzzing 0 1000000
Peach 2.3 4 31026
DXFuzzing 7 34222

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 24

 Conclusion

 Whitebox fuzzing is complex, time costly and

there are still some problems such as path
explosion, and is hard to pass strong program
checks fully automatically.
 Peach is an outstanding knowledge-based
fuzzing tool.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 25

 Conclusion
 DXFuzzing enriches current mutation
methodology with multi-dimension input nodes
mutation strategy without combinatorial
explosion. So DXFuzzing could find some
vulnerabilities that never will been found by one-
dimension mutation fuzzing.

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 26

 9 For More Information

For More Questions and Comments:

wuzhiyong0127@gmail.com

bill@cse.concordia.ca

zhuxy@ustc.edu.cn

2009/12/19 Conseg 09 Fuzzing for Software Vulnerability 27