Vous êtes sur la page 1sur 33

Unit 5

September 19, 2011

Defect

For the development phases before testing, the development activities themselves are subject to defect injection, and the reviews or inspections at end-of-phase activities are the key vehicles for defect removal. For the testing phases, the testing itself is for defect removal. When the problems found by testing are xed incorrectly, there is another chance to inject defects. In fact,even for the inspection steps, there are chances for bad xes. The Figure below describes the detailed mechanics of defect injection and removal at each step of the development process. From the gure, defect removal eectiveness for each development step, therefore, can be dened as:

P.Selvapriyavadhana,Asst.Prof,ACT

Figure 1: Defect Injection and Removal During One Process Step

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

Defect Removal Eectiveness and Quality Planning

Phase defect removal eectiveness and related metrics associated with eectiveness analyzes (such as defect removal and defect injection rates) are useful for quality planning and quality management. These measurements clearly indicate which phase of the development process we should focus on for improvement. Eectiveness analyzes can be done for the entire project as well as for local areas, such as at the component level and specic departments in an organization, and the control chart technique can be used to enforce consistent improvement across the board. Longitudinal release-to-release monitoring of these metrics can give a good feel for the process capability of the development organization. In addition, experiences from previous releases provide the basis for phase-specic target setting and for quality planning.
Phase-Based Defect Removal Model

The phase-based defect removal model (DRM) summarizes the relationships among three metricsdefect injection, defect removal, and eectiveness. The DRM takes a set of error-injection rates and a set of phase-eectiveness rates as input, then models the defect removal pattern step by step. It takes a simplied view of Figure 6.3 and works like this:

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

Figure 2: Dimensions of the Productivity Concept

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

Productivity Metrics

As stated in the preface, productivity metrics are outside the scope of this book. Software productivity is a complex subject that deserves a much more complete treatment than a brief discussion in a book that focuses on quality and quality metrics. For non-OO projects, much research has been done in assessing and measuring productivity and there are a number of well-known books in the literature. For example, see Joness work (1986, 1991, 1993, 2000). Metrics like lines of code per hour, function points per person-month (PM), number of classes per person-year (PY) or person-month, number of methods per PM, average person-days per class, or even hours per class and average number of classes per developer have been proposed or reported in the literature for OO productivity (Card and Scalzo, 1999; Chidamer et al., 1997; IBM OOTC, 1993; Lorenz and Kidd, 1994). Despite the differences in units of measurement, these metrics all measure the same concept of productivity, which is the number of units of output per unit of eort. In OO development, the unit of output is class or method and the common units of eort are PY and PM. The productivity concept in software, especially at the project level, however, is three-dimensional: output (size or function of deliverables), eort, and time.This is because the tradeo between time and eort is not linear, and therefore the dimension of time must be addressed.If quality is included as yet another variable, the productivity concept would be four-dimensional.Assuming quality is held constant or quality criteria can be established as part of the requirements for the deliverable, we can avoid the confusion of mixing productivity and quality, and productivity remains a 3D concept.

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

Figure 3: Defect Removal

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

As shown in Figure, if one holds any of the two dimensions constant, a change in the third dimension is a statement of productivity. For example, if eort (resources) and development time are xed, then the more output (function) a project produces, the more productive is the project team. Likewise, if resources and output (required functions) are xed, then the faster the team delivers, the more productive it is. It appears then that the two-dimensional metrics are really not adequate to measure software productivity.

Ishikawa-Diagram

Basic concept

The Idea: Think about possible causes and reasons leading to an effect or a problem nd solution for preventing those problems. 7 causes lead to the problem/eect. The causes are divided into main-and side causes. The7 causes are: 1. Methods 2. Machinery 3. Management 4. Materials 5. Manpower 6. Environment 7. Measurement

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

Figure 4: Ishikawa-Diagram

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

Aim

Find the causes, main and side causes. Clarity Interdependence of the causes. Improve them for having the wanted eect or eliminate them for solving the problem.
theoretical conversion

1. sketch the diagram and in script the needed causes. 2. work the main and side causes out 3. check the completeness 4. weight the main & side causes in terms of meaning & inuence 5. check the selected causes for rightness 6. The team discusses about the solution. causes that can be improved or eliminated easily will be nished rst of all (no need to be weighted) The weighted causes are in a list of priority and will be nished in turn.
Example

Rise in productivity 1.sketch the diagram and in script the needed causes.

SQM

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

10

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

11

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

12

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

13

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

14

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

15

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

16

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

17

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

18

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

19

P.Selvapriyavadhana,Asst.Prof,ACT

SQM

20

P.Selvapriyavadhana,Asst.Prof,ACT

4. weight the main & side causes in terms of meaning & inuence. Lean Management Standardization Motivation Education 5. check the selected causes for rightness. 6. The team discusses about the solution causes that can be improved or eliminated easily: Hardware Software Temperature Noise weighted causes Lean Management Standardization Motivation Education

SQM

21

P.Selvapriyavadhana,Asst.Prof,ACT

Applying the Seven Basic Quality Tools in Software Development

The basic statistical tools for quality control promoted by Ishikawa (1989) are widely used in manufacturing productions. They have indeed become an integral part of the quality control literature, and have been known as Ishikawas seven basic tools. This chapter describes the application of these tools for process and quality control in software development. There are many ways to analyze software metrics; the applications of Ishikawas seven tools represent a set of basic operations. Keep in mind that these statistical tools are for process and quality control at the project and organization level and, hence, are useful for project leaders and process experts. In contrast, they do not provide specic information to software developers on how to improve the quality of their designs or implementation. Also, because not all these tools are equally useful for small projects where statistical patterns of parameters of the development process are less obvious, the benets of statistics may not be realized. The box at the end of the chapter oers specic recommendations for small teams. In addition, although the benets of these tools have long been proved in manufacturing operations, their use and roles in software development has not been widely recognized. For instance, the use of control charts in manufacturing production can ensure a certain endproduct quality once the process is dened and the control limits are set. In software development, however, the process is complex and involves a high degree of creativity and mental activity. It is extremely dicult, if not impossible, to dene the process capability of software development in statistical terms. Therefore, achieving statistical process control in software development may mean a lot more than control charting. It may require,
SQM 22

P.Selvapriyavadhana,Asst.Prof,ACT

for example, new development technology, CASE tools, and the use of defect models and reliability estimating techniques. However, good use of the seven basic tools can lead to positive long-term results for process improvement and quality management in software development. The following sections begin with a brief description of the tools, followed by a discussion of each tool with examples of its applications. Where appropriate, the inuences of these tools on process improvement and on decision making are also described. The examples are either from software engineering literature or from software projects developed at IBM in Rochester, Minnesota. In addition to the seven basic tools, we discuss the relations diagram, which is eective for small team brainstorming and particularly useful in displaying cause-and-eect relationships.

SQM

23

P.Selvapriyavadhana,Asst.Prof,ACT

Figure 5: Ishikawas Seven Basic Tools for Quality Control

SQM

24

P.Selvapriyavadhana,Asst.Prof,ACT

A check sheet is a paper form with printed items to be checked. Its main purposes are to facilitate gathering data and to arrange data while collecting it so the data can be easily used later. Another type of check sheet is the check-up conrmation sheet. It is concerned mainly with the quality characteristics of a process or a product. To distinguish this conrmation check sheet from the ordinary data-gathering check sheet, we use the term checklist. In most software development environments, the data-gathering aspect is automated electronically and goes far beyond the data-gathering check sheet approach, which has been used in manufacturing production. Our discussion on this tool, therefore, is conned to checklists. A Pareto diagram is a frequency chart of bars in descending order; the frequency bars are usually associated with types of problems. It is named after a nineteenthcentury Italian economist named Vilfredo Pareto (1848 1923), who expounded his principle in terms of the distribution of wealththat a large share of the wealth is owned by a small percentage of the population. In 1950 Juran applied the principle to the identication of quality problemsthat most of the quality problems are due to a small percentage of the possible causes. In software development, the X -axis for a Pareto diagram is usually the defect cause and the Y -axis the defect count. By arranging the causes based on defect frequency, a Pareto diagram can identify the few causes that account for the majority of defects. It indicates which problems should be solved rst in eliminating defects and improving the operation. Pareto analysis is commonly referred to as the 8020 principle (20% of the causes account for 80% of the defects), although the cause-defect relationship is not always in an 8020 distribution.

SQM

25

P.Selvapriyavadhana,Asst.Prof,ACT

The histogram is a graphic representation of frequency counts of a sample or a population. The X-axis lists the unit intervals of a parameter (e.g., severity level of software defects) ranked in ascending order from left to right, and the Y-axis contains the frequency counts. In a histogram, the frequency bars are shown by the order of thXe variable, whereas in a Pareto diagram the frequency bars are shown by order of the frequency counts. The purpose of the histogram is to show the distribution characteristics of a parameter such as overall shape, central tendency, dispersion, and skewness. It enhances understanding of the parameter of interest. A scatter diagram vividly portrays the relationship of two interval variables. In a cause-eect relationship, the X -axis is for the independent variable and the Y -axis for the dependent variable. Each point in a scatter diagram represents an observation of both the dependent and independent variables. Scatter diagrams aid databased decision making (e.g., if action is planned on the X variable and some eect is expected on the Y variable). One should always look for a scatter diagram when the correlation coecient of two variables is presented. A run chart tracks the performance of the parameter of interest over time. The X-axis is time and the Y -axis is the value of the parameter. A run chart is best used for trend analysis, especially if historical data are available for comparisons with the current trend. Ishikawa (1989) includes various graphs such as the pie chart, bar graph, compound bar graph, and circle graph under the section that discusses run charts. An example of a run chart in software is the weekly number of open problems in the backlog; it shows the development teams workload of software xes. A control chart can be regarded as an advanced form of a run chart for situations where the process capabilSQM 26

P.Selvapriyavadhana,Asst.Prof,ACT

ity can be dened. It consists of a central line, a pair of control limits (and sometimes a pair of warning limits within the control limits), and values of the parameter of interest plotted on the chart, which represent the state of a process. The X -axis is real time. If all values of the parameter are within the control limits and show no particular tendency, the process is regarded as being in a controlled state. If they fall outside the control limits or indicate a trend, the process is considered out of control. Such cases call for causal analysis and corrective actions are to be taken. The cause-and-eect diagram, also known as the shbone diagram, was developed by Ishikawa and associates in the early 1950s in Japan. It was rst used to explain factors that aect the production of steel. It is included in the Japanese Industrial Standards terminology for quality control (Kume, 1989). It shows the relationship between a quality characteristic and factors that aect that characteristic. Its layout resembles a shbone, with the quality characteristic of interest labeled at the sh head, and factors aecting the characteristics placed where the bones are located. While the scatter diagram describes a specic bivariate relationship in detail, the cause-and-eect diagram identies all causal factors of a quality characteristic in one chart. 4 Measuring And Improving The Development Process

Why should we improve?

1. Lower development costs 2. Lower maintenance costs 3. Less bugs


SQM 27

P.Selvapriyavadhana,Asst.Prof,ACT

4. Higher code quality 5. Be exible - adapt to change more easily 6. Happier more productive users
How can we improve our development process?

1. Standardized build practices 2. Better testing practices 3. Better visibility 4. Faster feedback 5. Quality metrics 6. Automate
Standardized build practices

1. Development process - It is important to choose the appropriate development lifecycle process to the project at hand because all other activities are derived from the process. For most modern software development projects, some kind of spiral-based methodology is used over a waterfall process. There are several choices, including the Rational Unied Process (RUP), IBM Global Services Method, and eXtreme Programming (XP). Having a process is better than not having one at all, and in many cases it is less important on what process is used than how well it is executed. The commonly used methodologies listed above all contain guidance about how to execute the process and templates for artifacts. In addition, the RUP has a series of books that describe the best practices for using RUP although if you do not choose to use RUP, these books still provide an excellent source of best practices. It is also possible to add plugins to the RUP. For a list of available plug-ins, see Plug-in
SQM 28

P.Selvapriyavadhana,Asst.Prof,ACT

Central. 2. Requirements - Gathering and agreeing on requirements is fundamental to a successful project. This does not necessarily imply that all requirements need to be xed before any architecture, design, and coding are done, but it is important for the development team to understand what needs to be built. Quality requirements are broken up into two kinds: functional and nonfunctional. A good way to document functional requirements is using Use Cases. Note that Use Cases are used for non-OO projects. A denitive book on the subject of use cases is by Armour and Non-functional requirements describe the performance and system characteristics of the application. It is important to gather them because they have a major impact on the application architecture, design, and performance. See the non-functional requirements checklist on the Construx Web site. 3. Architecture - Choosing the appropriate architecture for your application is key. Many times IBM is asked to review a project in trouble and we have found that the development team did not apply well-known industry architecture best practices. A good way to avoid this type of problem is to contact IBM. Our consultants can work side by side with your team and ensure that the projects get started on the right track. Tried and true practices are called patterns and they range from the classic Gang of Four patterns, Java patterns , to EJB design patterns . Suns equivalent is the Core J2EE Patterns catalog . Many projects fail as discussed in the introduction. The study of these failures has given rise to the concept of antipatterns. They are valuable because they provide useful knowledge of what does not work, and why. 4. Design - Even with a good architecture it is still possible to have a bad design. Many applications are either over-designed or under-designed. The two basic principles here are Keep it Simple and information hiding. For many projects, it is important to perform
SQM 29

P.Selvapriyavadhana,Asst.Prof,ACT

Object-Oriented Analysis and Design using UML. There are many books on UML, but we recommend UML User Guide and Applying UML and Patterns . Reuse is one of the great promises of OO, but it is often unrealized because of the additional eort required to create reusable assets. Code reuse is but one form of reuse and there are other kinds of reuse that can provide better productivity gains. 5. WebSphere application design - IBM has extensive knowledge of the best practices and design patterns for the WebSphere product family. Each project is dierent and our consultants have the experience to help you. There is still a tremendous return on investment (ROI) even if you only use the consultants for a short time because you save the costs later in the project. Our experts have also published a great deal of this wisdom, including considerations for high-performance Web sites and guidelines for autonomic computing. 6. Construction of the code - Construction of the code is a fraction of the total project eort, but it is often the most visible. Other work equally important includes requirements, architecture, analysis, design, and test. In projects with no development process (so-called code and x), these tasks are also happening, but under the guise of programming. A best practice for constructing code includes the daily build and smoke test. Martin Fowler goes one step further and suggests continuous integration that also integrates the concept of unit tests and self-testing code. Note that even though continuous integration and unit tests have gained popularity through XP, you can use these best practices on all types of projects. I recommend using standard frameworks to automate builds and testing, such as Ant and JUnit. 7. Peer reviews - It is important to review other peoples work. Experience has shown that problems are eliminated earlier this way and reviews are as eective or even more eective than testing. Any artifact from
SQM 30

P.Selvapriyavadhana,Asst.Prof,ACT

the development process is reviewed, including plans, requirements, architecture, design, code, and test cases. Karl Wiegers paper on the Seven Deadly Sins of Software Reviews explains the correct ways to perform peer reviews. Peer reviews are helpful in trying to produce software quality at top speed. 8. Testing - Testing is not an afterthought or cutback when the schedule gets tight. It is an integral part of software development that needs to be planned. It is also important that testing is done proactively; meaning that test cases are planned before coding starts, and test cases are developed while the application is being designed and coded. There are also a number of testing patterns that have been developed. 9. Performance testing - Testing is usually the last resort to catch application defects. It is labor intensive and usually only catches coding defects. Architecture and design defects may be missed. One method to catch some architectural defects is to simulate load testing on the application before it is deployed and to deal with performance issues before they become problems. 10. Conguration management - Conguration management involves knowing the state of all artifacts that make up your system or project, managing the state of those artifacts, and releasing distinct versions of a system. There is more to conguration management than just source control systems, such as Rational Clearcase. There are also best practices and patterns for conguration management. 11. Quality and defects management - It is important to establish quality priorities and release criteria for the project so that a plan is constructed to help the team achieve quality software. As the project is coded and tested, the defect arrival and x rate can help measure the maturity of the code. It is important that a defect tracking system is used that is linked to the source control management system. For example, projects using
SQM 31

P.Selvapriyavadhana,Asst.Prof,ACT

Rational ClearCase may also use Rational ClearQuest. By using defect tracking, it is possible to gauge when a project is ready to release. 12. Deployment - Deployment is the nal stage of releasing an application for users. If you get this far in your project - congratulations! However, there are still things that can go wrong. You need to plan for deployment and you can use a deployment checklist on the Construx Web site. 13. System operations and support - Without the operations department, you cannot deploy and support a new application. The support area is a vital factor to respond and resolve user problems. To ease the ow of problems, the support problem database is hooked into the application defect tracking system. 14. Data migration - Most applications are not brand new, but are enhancements or rewrites of existing applications. Data migration from the existing data sources is usually a major project by itself. This is not a project for your junior programmers. It is as important as the new application. Usually the new application has better business rules and expects higher quality data. Improving the quality of data is a complex subject outside the scope of this article. 15. Project management - Project management is key to a successful project. Many of the other best practice areas described in this article are related to project management and a good project manager is already aware of the existence of these best practices. Our recommended bible for project management is Rapid Development by Steve McConnell . Given the number of other checklists and tip sheets for project management, it is surprising how many project managers are not aware of them and do not apply lessons learned from previous projects, such as: if you fail to plan, you plan to fail. One way to manage a dicult project is through timeboxing. 16. Measuring success - You can measure your develSQM 32

P.Selvapriyavadhana,Asst.Prof,ACT

opment process against an industry standard known as the Capability Maturity Model (CMM) from the Software Engineering Institute at Carnegie Mellon University. Most projects are at level 1 (initial). If you implement the best practices described above and the guidelines in the companion article, Guide to Running Software Development Projects, then you could be well on the way to achieving a higher maturity level and a successful project.
Tools

1. Build scripting 2. Automated code quality 3. Automated testing

SQM

33