Software Quality

Overview
Everything we have been talking about all semester has had something to do with software quality.
This is perhaps the most central and important topic of the course.

We will discuss the meaning of quality, and how to achieve and assure it. We will then turn our attention to defects: where they come from, what to do with them, and how to prevent them. We then discuss the Quality Triangle: People, Processes, and Tools. We conclude with a discussion of software metrics.
2

Objectives
Here is what you should be able to do upon completion of this module:
Define Software Quality. Achieve and assure software quality. Discuss the Return on Investment (ROI) of quality processes. Explain how to prevent defects.

Describe the interrelationship of People, Process and Tools. Describe the Quality organization: CM, Testing, and SQA. Discuss the use of metrics in a software engineering process.

Outline
Software Quality.
Definition. Assuring and Achieving quality. ROI (return on investment).

Defects:
Definition. Preventing defects. Finding and fixing defects.

The Quality Triangle: People, Processes, and Tools. Software Metrics.

4

Software Quality
Definition of quality: The degree to which a system, component, or process meets (1) specified requirements, and (2) customer or user needs or expectations. 1

Software Quality Criteria

Correctness Efficiency Flexibility Robustness Interoperability Maintainability Performance Portability Reliability Reusability Testability Usability Availability Understandability

Achieving Quality
How do we build quality into a product?

Assuring Quality
How do we know we have quality? How do we demonstrate this to others?

Return On Investment
What is the cost of quality? What is the cost of lack of quality?

Defects
Definitions. Preventing defects. Finding and fixing defects.

10

Some Definitions
Error: A conceptual, syntactic or clerical discrepancy which results in one or more faults in the software. 1 Fault: A specific manifestation of an error. A discrepancy in the software which can impair its ability to function as intended. An error may be the cause for several faults. 1

11

More Definitions
Failure: A software failure occurs when a fault in the computer program is evoked by some input data, resulting in the computer program not correctly computing the required function in an exact manner. 2 Defect: Either a fault or discrepancy between code and documentation that results in mischief in the testing, installation, maintenance, or use of software. 3
12

References
of Software Engineering Terminology, IEEE Std 610.12-1990 2 Lloyd, D.K., and M. Lipow, Reliability, Management, Methods and Mathematics, 2 nd Edition, published by the authors, 1977. 3 Dunn, Robert, and Ullman, Richard, Quality Assurance for Computer Software, McGraw-Hill, New York, 1982.
1 Glossary

13

Categorization of Defects
Classified by phase in which defect is introduced:
Inadequate or incorrect requirements definition. Inadequate or noncompliant top-level design. Errors in detailed design. Coding errors.
14

Generic Faults:
Computational. Logic. I/O. Data Handling. Interface. Database & Data Definition. Other.

Preventing Defects
Standards. Software Engineering Methods. Languages (let the compiler do some of the checking). Formal proofs of correctness. Prototyping. Continuous Process Improvement.

15

Finding Defects
Reviews. Walkthroughs. Formal Inspections. Testing. Defect Reporting.

16

Removing Defects
Configuration Management.
Baselines. Change Control Board. Software Configuration.

Regression testing. Ripple effect.

17

The Quality Triangle

Processes
18

Quality Assurance
Organization.
Configuration Management. Testing (IV&V). SQA.

Standards. Tools / Support Software.

19

Configuration Management
Bill of Materials hierarchy. Software Library.
Each version of each component.

Configuration Matrix.
Version control.

Change Control Board (CCB).

20

10

Testing
Purpose of testing:
Defect detection. Verification. Validation. Acceptance.

21

Phases of Testing
System Specification Software Requirements Software Design Code Qualification Test Integration Test String Test Unit Test

22

11

Software Quality Assurance

Independent organization. Concerned with the software process. Guided by Standards.

23

Standards
Software Quality Assurance Program. Tools, Techniques and Methodologies. Computer Program Design. Work Certification. Documentation. Library Control. Reviews and Audits. Configuration Management. Testing. Corrective Action. Subcontractor Control.

Source: Mil-S-52779A, Software Quality Assurance Program Requirements

24

12

Tools
Requirements. Analysis and Design. Coding. Configuration Management. Testing. Software Quality Assurance. Program Management.

25

Metrics
Definitions.
Product Metrics. Process Metrics.

Use of Metrics. Establishing a metrics program.

26

13

Metrics
Process Metrics measure aspects of the process. Product Metrics measure aspects of the product.

27

Software Product Metrics

Examples of software metrics:
Complexity. Reliability. Usability.

Types of metrics:
Static can be counted without running the software. Dynamic must run the software with instrumentation.

28

14

Reliability -- MTBF
MTBF: Mean time between failures. As testing proceeds, the rate of defects found (hopefully) decreases. As the defect rate decreases, the MTBF increases.
Defect rate Goal MTBF

Test time
29

Software Process Metrics

Examples:
Productivity. Manageability. Scrap & Rework. Turnover in Personnel.

Types of metrics:
Static counting things. Dynamic change over time.

30

15

Use of Metrics
Proper uses of metrics:
Statistical Process Control. Continuous Process Improvement.

Improper uses of metrics:

Using product or process metrics to measure people. Setting arbitrary measurement targets.

31

Establishing a Metrics Program

How do you decide what to measure?
Measure everything you can think of, and then see what is useful. or Determine what is useful (i.e., what your goals are) in advance, then see what needs to be measured to support those goals.
GQM Goal Question Metric

32

16