Vous êtes sur la page 1sur 6

Software reliability testing - Wikipedia, the free encyclopedia

1 of 6


Software reliability testing

From Wikipedia, the free encyclopedia

Software reliability testing is a field of software testing that relates to testing a software's ability to function,
given environmental conditions, for a particular amount of time. Software reliability testing helps discover many
problems in the software design and functionality.

1 Overview
2 Measurement
3 Objectives of reliability testing
3.1 Secondary objectives
3.2 Points for defining objectives
4 Importance of reliability testing
5 Types of reliability testing
5.1 Feature test
5.2 Load test
5.3 Regression test
6 Test planning
6.1 Problems in designing test cases
7 Reliability enhancement through testing
7.1 Reliability growth testing
7.2 Designing test cases for current release
8 Reliability evaluation based on operational testing
8.1 Reliability growth assessment and prediction
8.2 Reliability estimation based on failure-free working
9 See also
10 References
11 External links

Software reliability is the probability that software will work properly in a specified environment and for a given
amount of time. Using the following formula, the probability of failure is calculated by testing a sample of all
available input states. Mean Time Between Failure(MTBF)=Mean Time To Failure(MTTF)+ Mean Time To
Probability = Number of failing cases / Total number of cases under consideration
The set of all possible input states is called the input space. To find reliability of software, we need to find output
space from given input space and software.[1]
For reliability testing, data is gathered from various stages of development, such as the design and operating
stages. The tests are limited due to restrictions such as cost and time restrictions. Statistical samples are obtained
from the software products to test for the reliability of the software. Once sufficient data or information is

10/5/2016 2:30 AM

Software reliability testing - Wikipedia, the free encyclopedia

2 of 6


gathered, statistical studies are done. Time constraints are handled by applying fixed dates or deadlines for the
tests to be performed. After this phase, design of the software is stopped and the actual implementation phase
starts. As there are restrictions on costs and time, the data is gathered carefully so that each data has some
purpose and gets its expected precision.[2] To achieve the satisfactory results from reliability testing one must
take care of some reliability characteristics. For example, Mean Time to Failure (MTTF) [3] is measured in terms
of three factors:
1. operating time,
2. number of on off cycles,
3. and calendar time.
If the restrictions are on operation time or if the focus is on first point for improvement, then one can apply
compressed time accelerations to reduce the testing time. If the focus is on calendar time (i.e. if there are
predefined deadlines), then intensified stress testing is used.[2]

Software reliability is measured in terms of mean time between failures(MTBF).[4]
MTBF consists of mean time to failure (MTTF) and mean time to repair(MTTR). MTTF is the difference of time
between two consecutive failures and MTTR is the time required to fix the failure.[5] Reliability for good
software is a number between 0 and 1. Reliability increases when errors or bugs from the program are
For example, if MTBF = 1000 hours for average software, then the software should work for 1000 hours for
continuous operations.

Objectives of reliability testing

The main objective of the reliability testing is to test software performance under given conditions without any
type of corrective measure using known fixed procedures considering its specifications.

Secondary objectives
The secondary objectives of reliability testing is:
1. To find perceptual structure of repeating failures.
2. To find the number of failures occurring in a specified amount of time.
3. To find the mean life of the software.
4. To discover the main cause of failure.
5. Checking the performance of different units of software after taking preventive actions.

Points for defining objectives

Some restrictions on creating objectives include:
1. Behaviour of the software should be defined in given conditions.
2. The objective should be feasible.
3. Time constraints should be provided.[7]

10/5/2016 2:30 AM

Software reliability testing - Wikipedia, the free encyclopedia

3 of 6


Importance of reliability testing

The application of computer software has crossed into many different fields, with software being an essential
part of industrial, commercial and military systems. Because of its many applications in safety critical systems,
software reliability is now an important research area. Although software engineering is becoming the fastest
developing technology of the last century, there is no complete, scientific, quantitative measure to assess them.
Software reliability testing is being used as a tool to help assess these software engineering technologies. [8]
To improve the performance of software product and software development process, a thorough assessment of
reliability is required. Testing software reliability is important because it is of great use for software managers
and practitioners.[9]
To verify the reliability of the software via testing:
1. A sufficient number of test cases should be executed for a sufficient amount of time to get a reasonable
estimate of how long the software will execute without failure. Long duration tests are needed to identify
defects (such as memory leakage and buffer overflows) that take time to cause a fault or failure to occur.
2. The distribution of test cases should match the actual or planned operational profile of the software. The
more often a function or subset of the software is executed, the greater the percentage of test cases that
should be allocated to that function or subset.

Types of reliability testing

Software reliability testing includes feature testing, load testing, and regression testing.[10]

Feature test
Feature testing checks the features provided by the software and is conducted in the following steps:
Each operation in the software is executed once.
Interaction between the two operations is reduced and
Each operation is checked for its proper execution.
The feature test is followed by the load test.[10]

Load test
This test is conducted to check the performance of the software under maximum work load. Any software
performs better up to some amount of workload, after which the response time of the software starts degrading.
For example, a web site can be tested to see how many simultaneous users it can support without performance
degradation. This testing mainly helps for Databases and Application servers. Load testing also requires software
performance testing, which checks how well some software performs under workload.[10]

Regression test
Regression testing is used to check if any new bugs have been introduced through previous bug fixes.
Regression testing is conducted after every change or update in the software features. This testing is periodic,
depending on the length and features of the software.[10]

10/5/2016 2:30 AM

Software reliability testing - Wikipedia, the free encyclopedia

4 of 6


Test planning
Reliability testing is more costly compared to other types of testing. Thus while doing reliability testing, proper
management and planning is required. This plan includes testing process to be implemented, data about its
environment, test schedule, test points etc.

Problems in designing test cases

Some common problems that occur when designing test cases include:
Test cases can be designed simply by selecting only valid input values for each field in the software.
When changes are made in a particular module, the previous values may not actually test the new features
introduced after the older version of software.
There may be some critical runs in the software which are not handled by any existing test case.
Therefore, it is necessary to ensure that all possible types of test cases are considered through careful test
case selection.[10]

Reliability enhancement through testing

Studies during development and design of software help for improving the reliability of a product. Reliability
testing is essentially performed to eliminate the failure mode of the software. Life testing of the product should
always be done after the design part is finished or at least the complete design is finalized.[11] Failure analysis
and design improvement is achieved through testings.

Reliability growth testing

[11] This

testing is used to check new prototypes of the software which are initially supposed to fail frequently.
The causes of failure are detected and actions are taken to reduce defects. Suppose T is total accumulated time
for prototype. n(T) is number of failure from start to time T. The graph drawn for n(T)/T is a straight line. This
graph is called Duane Plot. One can get how much reliability can be gained after all other cycles of test and fix

solving eq.1 for n(T),

where K is e^b. If the value of alpha in the equation is zero the reliability can not be improved as expected for
given number of failure. For alpha greater than zero, cumulative time T increases. This explains that number of
the failures doesn't depends on test lengths.

Designing test cases for current release

If we are adding new features to the current version of software,then writing a test case for that operation is
done differently.

10/5/2016 2:30 AM

Software reliability testing - Wikipedia, the free encyclopedia

5 of 6


First plan how many new test cases are to be written for current version.
If the new feature is part of any existing feature, then share the test cases of new and existing features
among them.
Finally combine all test cases from current version and previous one and record all the results.[10]
There is a predefined rule to calculate count of new test cases for the software. if N is the probability of
occurrence of new operations for new release of the software, R is the probability of occurrence of used
operations in the current release and T is the number of all previously used test cases then

Reliability evaluation based on operational testing

The method of operational testing is used to test the reliability of software. Here one checks how the software
works in its relevant operational environment. The main problem with this type of evaluation is constructing
such an operational environment. Such type of simulation is observed in some industries like nuclear industries,
in aircraft etc. Predicting future reliability is a part of reliability evaluation.
There are two techniques used for this:
Steady state reliability estimation
In this case, we use feedback from delivered software products. Depending on those results, we can
predict the future reliability for the next version of product. This is similar to sample testing for physical
Reliability growth based prediction
This method uses documentation of the testing procedure. For example, consider a developed software
and that we are creating different new versions of that software. We consider data on the testing of each
version and based on the observed trend, we predict the reliability of the new version of software.[12]

Reliability growth assessment and prediction

In the assessment and prediction of software reliability, we use the reliability growth model. During operation of
the software, any data about its failure is stored in statistical form and is given as input to the reliability growth
model. Using this data, the reliability growth model can evaluate the reliability of software. Lots of data about
reliability growth model is available with probability models claiming to represent failure process. But there is no
model which is best suited for all conditions. Therefore, we must choose a model based on the appropriate

Reliability estimation based on failure-free working

In this case, the reliability of the software is estimated with assumptions like the following:
If a bug is found, then it is sure that it is going to be fixed by someone.
Fixing the bug will not have any effect on the reliability of the software.
Each fix in the software is accurate.[12]

See also

10/5/2016 2:30 AM

Software reliability testing - Wikipedia, the free encyclopedia

6 of 6


Software testing
Load testing
Regression testing
Reliability engineering

1. Software Reliability. Hoang Pham.
2. E.E.Lewis. Introduction to Reliability Engineering.
3. "MTTF".
4. Roger Pressman. Software Engineering A Practitioner's Approach. McGrawHill.
5. "Approaches to Reliability Testing & Setting of Reliability Test Objectives".
6. Aditya P. Mathur. Foundations of Software Testing. Pearson publications.
7. Reliability and life testing handbook. Dimitri kececioglu.
8. A Statistical Basis for Software Reliability Assessment. M. xie.
9. Software Reliability modelling. M. Xie.
10. John D. Musa. Software reliability engineering: more reliable software, faster and cheaper. McGraw-Hill.
ISBN 0-07-060319-7.
11. E.E.Liwis. Introduction to Reliability Engineering. ISBN 0-471-01833-3.
12. "Problem of Assessing reliability". CiteSeerX:

External links
Mean Time Between Failure (http://www.weibull.com/hotwire/issue94/relbasics94.htm/)
Software Life Testing (http://www.weibull.com/basics/accelerated.htm/)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Software_reliability_testing&oldid=730477331"
Categories: Software testing
This page was last modified on 19 July 2016, at 07:22.
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may
apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia is a registered
trademark of the Wikimedia Foundation, Inc., a non-profit organization.

10/5/2016 2:30 AM