Académique Documents
Professionnel Documents
Culture Documents
to create and creating fault tolerant software is an even Throughout the literature certain questions reoccur,
greater challenge. Fault tolerant software must successfully which one would anticipate when a new field emerges in
recover from a multitude of error conditions to prevent commercial software fault tolerance. People are interested,
harmful system failures. and want to understand and define commercial software
Software testing cannot demonstrate that a software reliability and fault tolerance, so the following questions
system is completely correct. An enormous number of which are recurrent throughout the literature are not
possible executions that must be examined in any real-world surprising:
sized software system. Fault tolerance expands the number Why study Fault Injection Modelling?
of states (and thus execution histories) that must be tested, Why study business software fault tolerance requirements?
because inconsistent or erroneous states must also be tested. Why are they called ‘Fault Injection & Error Seeding’?
Mailing lists, websites, research and forums have been Why review Software Implemented Fault Injection (SWIFI)?
created in which all aspects of this fresh new niche software What work was performed, current status and work
engineering area are discussed. People are interested, partly proposed?
because it is a new area but also because the whole field of These questions will be expanded upon throughout the
commercial software reliability is in itself so interesting; as research, and seek to bring clarity to those who want to find
it holds so many wide ranging disciplines, perspectives and the answers to the above, or to see if there truly are any
logic at its core. Software reliability engineering is uniting answers!
professionals in disciplines that previously had little to do
with one another, it is creating more opportunities for
4.2 Background Concepts
employment in the online environment, and it is changing
the face and structure of all information that we seek out on A fault is a hardware or software defect, inconsistency,
the web. In the era of economic recession, customer transient electrical field, or other abnormal circumstance.
demands reliable, certified and fault tolerant commercial An error is an invalid internal state, which may or may not
and business software applications. be detected by the system.
In this research, the focus is on software testing A failure is an invalid output. Thus a fault or error
techniques that use fault injection. Several potentially becomes a failure when it propagates to the output. There is
powerful existing systems have drawbacks for practical a natural progression from fault to error to failure. Recovery
application. We first examine existing fault injection code is the part of a program that is designed to respond to
techniques and evaluate their potential for practical error states. Recovery code executes after the program
application in commercial and business software recognizes that some erroneous or abnormal state has been
applications. Available and accessible literature entered. This code should gracefully restore the system to a
infrastructure including premium subscribed IEEE and valid state before a failure occurs.
ACM resources were studied and summarized for literature Figure 1 shows the progression from faults to errors and
review from 1986 (20 years). finally to failures. The recovery code should serve as a safety
net to prevent the progression from error to failure. A fault
4. Fault Injection Modeling tolerant system should never fail, even if it has faults.
Fault Injection Modelling (FIM) involves the deliberate
insertion of faults or errors into a computer system in order
to determine its response. It has proven to be an effective
method for measuring and studying response of defects,
validating fault-tolerant systems, and observing how systems
behave in the presence of faults. In this study, faults are
injected in all phases of Software Development Life Cycle
viz., Requirements, Design and Source Code.
4.1 Objectives
The key objective of this research is to understand and Figure 1. Fault Tolerance Terms
statistically analyze the behaviour of faults and defects
pattern by injecting known defects in business software Testing recovery code requires the modeling of bad states
applications artifacts (Software Requirements Specification, that accurately simulate exceptional situations. As much as
Design, Source Code) to study the “Amplification Index”, 50% of a fault tolerant program can consist of recovery
Domino Effect and defect leakage in key Software code. Although testing might include invalid data that
Development Life Cycle (SDLC) phases (Requirements, executes some of the recovery code, often much of this code
Design, Coding, Testing) of business application is never executed during normal testing.
development, with domain variants (education, e- Any recovery code testing technique must be based upon
governance, retail, systems) build on same technology(C an assumed fault model [7]. We assume that all faults will
#.Net). behave according to some specific rules. Any fault model
The goal of this research is to understand the behaviour of can only consider a subset of all possible faults.
faults and defects pattern in commercial and business
software application and defect leakage in each phase of
application development.
(IJCNS) International Journal of Computer and Network Security, 9
Vol. 1, No. 3, December 2009
For example, a common debugging practice is to insert a system behaviour in presence of errors. All of the injection
series of \print" statements in key positions. This debugging failures methods are based on concrete hardware or software
practice assumes a particular fault model. characteristics associated to systems which are applied,
Faults will cause the program to execute in the incorrect then, to realize generalizations is a very complicated task.
order and will be demonstrated
Figure 2: Taxonomy of Fault Injection Techniques in the
4.3 Prior Work on Fault Injection
printed output. Clearly, not all faults will adhere to this
model. Fault injection can be used to modify either a program's
source code text or the machine state of an executing
program. Figure 2 shows taxonomy of the key methods of
fault injection. Fault injection techniques based on static
analysis -program source modification - are modelled by the
left sub tree.
The most common static fault injection is mutation
testing. The right sub tree in Figure 2 models dynamic fault
injection techniques where changes are made to an actively
running program's state. Much of the recent fault injection
research is concerned with dynamic injection.
4. 4 Domino’s effect
Domino’s effect is the cascading effect of defects from the
initial stages of the project to all the subsequent stages of the
software life cycle. Errors undetected in one work product
are ‘leaked’ to the child work product and amplifies defects
Figure 2. Taxonomy Of Fault Injection Techniques
in the child work product. This chain reaction causes an
No one fault model will fit all faults. However, a fault exponential defect leakage. E.g.: undetected errors in
model can be very effective in detecting faults that fit the requirements leak and cause a significant number of defects
model. in design which, in turn, causes more defects in the source
Fault Injection technique of fault injection dates back to code. The result of this study is to arrive at an
the 1970s when it was first used to induce faults at a “Amplification Index” which will characterize the extent of
hardware level. This type of fault injection is called impact or damage of phase-wise defects in subsequent
Hardware Implemented Fault Injection (HWIFI) and Software Development Life Cycle (SDLC) phases.
attempts to simulate hardware failures within a system. The The defect components in a work product and leakage
first experiments in hardware fault injection involved into subsequent phases is illustrated below:
nothing more than shorting connections on circuit boards
and observing the effect on the system (bridging faults). It
was used primarily as a test of the dependability of the
hardware system. Later specialised hardware was developed
to extend this technique, such as devices to bombard specific
areas of a circuit board with heavy radiation. It was soon
found that faults could be induced by software techniques
and that aspects of this technique could be useful for
assessing software systems. Collectively these techniques are
known as Software Implemented Fault Injection (SWIFI) [8].
Martin defines software fault injections as faults which
are injected at the software level by corrupting code or data.
So faults are applicable at the implementation phase when
the code of the system is available, and it can be applied on Figure 3. Fault Injection Patten
an application to simulate either internal or external faults.
Internal faults represent design and implementation
5. Trimentus Approach For Fault Injection
faults, such as variables/parameters that are wrong or not
initialized, incorrect assignments or condition checks. Experiments
External faults represent all external factors that are not Defects were deliberately injected into each phase (work
related to faults in the code itself but that alter the system's product) in the software development life cycle of a typical
state. application development project and the effect of the defects
The injection of failures can discover errors that normal injected was studied subsequently. The injected defects are
procedures cannot. First, it tests the mechanisms of typical defects that are characteristic of the software systems
exception and treatment of failures that in normal of a commercial application developed in the following
circumstances are not sufficiently proven and, helps to domains;
evaluate the risk, verifying how much defective can be the
10 (IJCNS) International Journal of Computer and Network Security,
Vol. 1, No. 3, December 2009
A Point Of Sales (POS) is retail systems for recording 8.2.1 Design Review
sales and billing at a cash counter of typical establishment.
POS is developed to automate the operations and billing Table 6: SRS And Design Of The Four Applications
applications of super markets. The information system
application was used to conduct the fault injection
experiments. The same application was developed in the
following technologies in 3G languages;
Table 4: POS
An Audit Tracking System (ATS) is a systems application 8.2.2 Design Defect Amplification: Domain Variant
used to conduct the fault injection experiments. The The following chart represents the comparison of
application chosen was the Audit Tracking System (ATS) amplification index between the applications developed on
that is used to automate the process for recording, same technology. The amplification of design defects caused
communicating and monitoring all the audit findings that due to the injected requirement defects in the application is
take place within the organization. The same application evidenced in all domains and least prominent in retail
was developed in the following technologies in 3G application.
languages;
AI Trend - Application Wise
0.3
0.2
0.2
1.0 0.9
0.9
0.8
0.7 0.6
0.6 0.5 0.5
0.5
AI
0.4
0.3
0.2
0.1
0.0
LMS POMS POS ATS
Coding Phase - Applications
9. CONCLUSIONS
9.1 AI Trend Analysis
The Amplification Index indicates the extent of damage
caused by a defect in various phases of the project. The
index increases with every step in the life cycle of the
project in all domains. Initial analysis of these defects in the
education domain application shows substantial increase in
the amplification index across phases compared other
application domains.
Within the experimentation limits, it was concluded and
8.3.4 Statistical Analysis and Validation validated statistically that there is no statistically significant
Similarly, based on the AI derived from the above design relationship, and correlation between defects amplification
data analysis, a statistical study was carried out to index on requirements vs design and design vs code on
understand and analyse the statistical significance and applications developed under different domains under a
relationship of AI across design phases. common technology
A regression analysis using was carried out to under the
relationship between a dependent variable, i.e., Code AI to AI Trend - Application Wise
independent variable i.e., Design AI.
Minitab tool was used to analyse the data set of 1.9
2.0
Amplification Indexes.
Analysis Results: 1.8
1.6
Table 11: AI Derived From Design Data Analysis 1.4 LMS
1.2 1.1
0.9 PMOS
1.0 0.8
AI
POMS
8 87
Tolerant Digital Systems. Addison-Wesley,
7 POS
6 6 Massachusetts, 1989.
5 ATS
4 4 45 [8] Daniel Dreilinger, Lijun Lin. Using Fault Injection to
3
2 Test Software Recovery Code, November 1995
0 [9] Leme, Nelson G. M.; Martins, Eliane; Rubira, Cecilia
Req Design Coding Testing
M. F. “A Software Fault Injection Pattern System”.
Phases Proceedings of the IX Brazilian Symposium on Fault-
Tolerant Computing. Florianópolis, SC, Brazil, March
Figure 8. Defect Leakage – Application Wise
5th-7th, 2001, pages 99-113.