UNIVERSITY OF LJUBLJANA FACULTY OF ECONOMICS

Requirements development Risks of error and how to prevent them

Ljubljana, May 2012

- name removed -

TABLE OF CONTENTS
INTRODUCTION .................................................................................................................... 1 1 2 3 REQUIREMENTS DEVELOPMENT ........................................................................... 1 COST OF ERRORS ......................................................................................................... 3 ISSUES ............................................................................................................................... 4 3.1 3.2 3.3 3.4 3.5 3.6 3.7 4 Skills ............................................................................................................................ 4 Tools ............................................................................................................................ 5 Control ........................................................................................................................ 5 Involvement / commitment ....................................................................................... 6 Detail ........................................................................................................................... 6 Priority ........................................................................................................................ 7 Differences among parties involved ......................................................................... 8

FRAMEWORK ................................................................................................................. 8

CONCLUSION ....................................................................................................................... 10 REFERENCES ....................................................................................................................... 11

TABLE OF FIGURES
Figure 1: The requirements development part of the engineering process ........................ 2 Figure 2: Relative cost of making errors ................................................................................ 3 Figure 3: Key reasons for project failure ............................................................................... 4 Figure 4: Types of requirements ............................................................................................. 7

TABLE OF TABLES
Table 1: Activities, risks and tips per phase of the requirements development process ... 9

TABLE OF APPENDICES
Appendix I ................................................................................................................................. 1

INTRODUCTION
Grady (1999) shows it costs far more in order to correct a mistake that is found late in the lifecycle of a project, than fixing the error shortly after creating it. In many cases, customers do not see and understand how essential it is to assure high quality requirements from the beginning (Wiegers, 2003). Because these customers are the same stakeholders that eventually have to work with the software, it is essential the Business Analyst (BA) ensures the quality of the requirement development process. In this paper aimed at (future) Business Analysts the author explains what requirements development is, what the costs of bad requirements development are, and where the risk of error exists most. Where elaboration is deemed necessary, a concise solution is provided with each pitfall or risk. After that, a more detailed overview for the benefit of the BA is provided that shows several issues and (other) risks divided per phase of the development process.

1 REQUIREMENTS DEVELOPMENT
We can say that a software project failed when the stakeholders are unhappy with the end result, and the software is not doing what the stakeholders wanted it to do. Only two possible reasons for this failure are possible: wrong expectations or wrong software (or a combination of the two). Making the software is a straightforward process in the sense that when requirements are clear, there is not much room for accidental deviation. Wrong expectation however is not, and is linked with the complex process of the development of the requirements. As authors use different definitions of terms used in business analysis, a short definition of the term requirements development as chosen by the author of this paper is provided. A requirement is a condition or capability that is needed by a stakeholder to either solve a problem, achieve an objective or that must be met to satisfy a formally imposed document (e.g. a contract, specification, etc.) (IIBA, 2009). There are 3 main types of requirements; business requirements, which are organizational goals and needs, user requirements, concerning the necessary tasks to perform with the software solution, and solution requirements. Solution requirements are characteristics of the solution itself and can be functional or non-functional. Besides the main criteria, some authors use the term transition requirements to define temporary needed capabilities during the development process (Modernanalyst.com, 2012). A graphical representation is found in Figure 4 later in this paper. Requirements form the very basis of a project and define what is needed by the stakeholders in a potential new system. Requirements development is therefore the set of activities that produces these conditions or capabilities and is part of the requirements engineering process, which is how most authors identify the complete process of both requirements development and requirements management (Jackson, Dick, & Hull, 2010; Leffingwell & Widrig, 2003; Sommerville & Kotonya, 1998). A widely accepted definition of the different steps that are part of the development process does not exist, although the differences are minor. Figure 1 shows how Wiegers (2003) and Moore, Bourque, Dupuis, 1

Abran, and Trip (2001) describe requirements development. They define the steps as elicitation, analysis, specification and validation.1 This is never a straightforward, linear process but needs an iterative approach because you are dealing with different people that may add to or contradict one another (Westfall). Figure 1: The requirements development part of the engineering process

(Wiegers, Software Requirements, 2nd Edition, 2003)

In order to avoid confusion, a short explanation of each basic step is provided, based on the descriptions used in the Business Analysis Body of Knowledge (BABOK) as well as other literature. Because, as Figure 1 shows, the development process is not a linear one, it is often not helpful to relate the development issues discussed later in this paper to one specific step, but the steps might help the reader to visualize them better (Hickey & Davis, 2003; IIBA, 2009; Bahill & Henderson, 2004). Elicitation describes our work with stakeholders to find out their needs. Ensuring we correctly understand those needs is vital here, as well as closely documenting it for feedback and for the next steps in the process. Several elicitation techniques can be used at the same time. Analysis is the elaboration of the elicitated requirements to a level of detail that is sufficient for designing and building a solution. This step includes further assessment of the stated requirements and the current state of the business to know what level of detail is needed, and to be able to verify the results. Specification or modeling comprises the documentation of all the input and derived requirements into usable models. Validation means (quoted) ensuring that (1) the set of requirements is correct, complete, and consistent, (2) a model that satisfies the requirements can be created, and (3) a realworld solution can be built and tested to prove that it satisfies the requirements.. Customer validation is an integral part of this step.

Other authors use less or more steps to describe the process, including enterprise analysis, solution assessment or triage, modeling, verification, negotiation and requirements documentation, which may or may not overlap with the Requirement Management process (Hickey & Davis, 2003; Pohl, 1996; University of Ljubljana, Faculty of Economics, 2012; IIBA, 2009; Wahono, 2003)

2 COST OF ERRORS
Making mistakes is expensive. The size of the cost however depends on how far you are in a project and how much it is completed. Study shows it costs much more to correct mistakes later in a project then those fixed in an early stage. Grady (1999) found that correcting defects during testing is already 10 times more expensive than correcting them during the requirement phase, shown in Figure 2, and attributes this to errors in the (development of) requirements. Waters (1999) shows that a six-month delay can cost firms 33 percent of return on investment in a five year business case. Figure 2: Relative cost of making errors
Relative cost to correct a defect
120 100 80 60 40 20 0 Requirements Design Code Test Operation Development phase (Serena, 2005)

The place of requirement development in this failure is remarkable. Figure 3 shows that poor requirements definition account for half of all project failures. Other statistics go as high as 70 percent (ESI International, 2008). Clearly the process of requirements development is one where much can be lost (and won) and asks for a more detailed look at what the underlying reasons are. A case study by Snyder and Shumate shows that 4 to 7 percent of requirement defects leak into the design of the project, and even 4 percent stays unnoticed until maintenance (Snyder & Shumate, 1992). Studying and improving Hughes Aircrafts requirement detection process, their changes led to a requirements review efficiency of 43 percent before improvement to 84 percent after (Willis, Ron R.; Rova, Robert M.; Scott, Mike D.; Johnson, Martha J.; Ryskowski, John F.; Moon, Jane A.; Winfield, Thomas O.; Shumate, Ken C., 1998).

Figure 3: Key reasons for project failure

Lack of qualified resources 3% Communication problems 14% Inadequate risk management 17% Poor scope control 15%

Other 1%

Poor requirements defining 50%

(ESI International, 2005)

3 ISSUES
Two famous examples of requirement development failures outside software engineering are the RMS Titanic, and the World Trade Towers in New York. The HMS Titanic satisfied the needs (requirements) of the ship owners and passengers before it sank, meaning the validation of the requirements was done well. But the requirements were wrong, because the makers did not build the ship in the right way. This shows very well that though a requirement may satisfy the need of stakeholders, it may not be suitable for the business need (which may or may not be known at the time of creation). The other example, the World Trade Center in New York, shows that it is debatable which real needs or threats should be taken into account when developing the requirements. There are people saying the towers should have possessed the strength to withstand a crash of an airplane. Again the structure conformed to the original design, so it was validated perfectly. The requirements however, may (or may not) have been wrongly defined (Bahill & Henderson, 2004). Construction is similar to software engineering; if the right requirements are not set, disasters can happen. In order to give a complete overview of the most important risks, errors and pitfalls that surround the requirement development process in software engineering, a list of themes is presented. Using this approach many themes will show to be interrelated and will apply to one or more of the defined steps; elicitation, analysis, specification and / or validation. Also, where possible the issues are repeated per phase in a table (along with other risks and tips) for the benefit of the BA. 3.1 Skills Some well known issues with any profession concern the lack of skill. Outside of the obviously needed procedural and communication skills needed by the BA, there is experience, 4

technical expertise and also language skills that may lack. This is true also for the stakeholders. In case they are technically not sophisticated enough, the process can slow down (Naz & Khokhar, 2009). Also stakeholders may not understand the development process at all, although this might be directly linked with the communication skills of the BA (McConnell, Rapid Development: Taming Wild Software Schedules, 1996). The BA can also have little knowledge of the problem domain (Wahono, 2003). 3.2 Tools Using the wrong software tools like common office tools as a BA is labor intensive, also for the stakeholders. With unintegrated tools it relies on the BA to relate data across them. Also review of requirement analysis by the stakeholders is harder, making it harder to gain consensus. 3.3 Control Planning to catch up later: Just as any part of the process, analysis can take too long. This can be a result of the wrong idea that coding cannot begin until the requirement specification is perfect and a hundred percent complete. McConnell (1996) states Piling on new features without adjusting the schedule guarantees that you will miss your deadline.. If you are working on a four-month project, but it takes two months to do what should have taken you one, what do you do? You can plan to catch up later, but generally you never do. Just as you learn during the project about how much time something actually takes, your schedule needs to be flexible and adjusted. Catching up later generally is a recipe for failure. Requirement changes: On average 25 percent of requirements change during a project lifecycle. This in turn means often 25 percent more time needed to produce the software. This is not just dangerous, but can be fatal for the whole project (McConnell, 1996). The same goes for new requirements. There will be always new, better ways of changing your model, but unless you set hard formal limits on what can be changed until which phase, the system will never be completed (Dorsey, 2000). Also, proposed changes may not be feasible (money, time, complexity). Saying sure, no problem is a major pitfall in this process. Version control: in case there are documents that are only distinguishable by the BA but not by the other parties involved, the process is broken and will probably fail. And finally, coding: when the requirements process is not taken seriously as a preparation or when the requirements themselves are not taken seriously, chances are that coding begins (a) too soon and (b) with only interest in the broad requirements. In projects this is called cutting out non-essentials and are easy targets (Dorsey, 2000). Solution: Risk assessment is the key in deciding how far to go and how many changes are allowed (Westfall). The first question to ask if a new requirement comes, or when a change happens is Is this in scope?. The goal of requirements development is to get requirements that are good enough, with an acceptable level of risk (Wiegers, 2005). After realizing perfect 5

requirements dont exist, risk and scope can be set. An example is not allowing major changes when a module reaches 75 percent completion (TechRepublic, 2007). If possible, set up a change control board governing this in combination with a proper issue-tracking tool to ensure changes are processed in the right way (Wiegers, 2000). It is very important that this includes testing change proposals on feasibility (money, time, complexity). Also, proper automated version control tools are recommended. 3.4 Involvement / commitment Users not participating (enough) in the reviewing of the requirements. The result of this is a growing gap between the original idea of the stakeholders and the final product (Naz & Khokhar, 2009). This is not a responsibility that is (just) on the user however, but of the BA. If the BA did not gather input from some of the potential user groups, someone will likely be unhappy with the final product (Wiegers, 2000). Specifically be cautious when user surrogates (such as marketing staff, or managers) are the ones supplying the requirements. Solution: Identify the different user groups before starting the elicitation. These are groups of users that differ from other groups by the frequency of using the software, which features they use, their security access level or other ways (Wiegers, 2000; Gause & Lawrence, 1999). Also use participation and communication techniques to involve everybody. 3.5 Detail Ambiguity is probably the number one mistake one can make. When a single requirement can be read in different ways, its there. Worse is when the BA believes it is formal and unambiguous, but all parties interpret the requirements in their own way. This means the mistake does not become clear until after the software has been created, causing severe amounts of rework (Wiegers, Software Requirements, 2nd Edition, 2003). Another cause connected with detail is that, because of lack of skill of the user or the BA, obvious information is easily omitted. This will not be visible until the testing phase (Wahono, 2003). Solution: It is important to realize that there are several levels of requirements. Wiegers (2000) thinks in three levels or requirements; business requirements, representing high-level objectives; user requirements, describing tasks that users should be able to carry out using the potential product; and solution requirements, specific behavioral (functional and nonfunctional) needs that the software must have (IIBA, 2009). Figure 4 shows this graphically. Against ambiguity, be sure to begin writing test cases early in the development phase and not to use subjective word in the requirements. See Appendix I for a list of terms to avoid. It is also important that not only the analyst should formally inspect the requirement documents, but also the customer, developer and tester.

Figure 4: Types of requirements

(Wiegers, Software Requirements, 2nd Edition, 2003)

3.6 Priority This is best shown by a symptom Wiegers (2000) gives in his excellent work 10 requirement traps to avoid: We dont need to prioritize requirements, said the user representative. Theyre all important, or I wouldnt have given them to you.. Although seemingly true, in practice it works very different. If all requirements are said to be equally critical, the project manager will have no adequate way to respond to new requirements, or for example changed quality goals or amount of staff. Second, because they are all important, everybody involved in the process can make their own trade-off decisions. Given the very different personalities and roles people have in the requirement development process, this is not a good idea. Another issue connected with this is requiring functionality that no one uses. Users may tend to express their need for functionalities they will never use. A sharp eye of the BA is needed in this area. This also goes for the developers side that can add unnecessary functionality that the users are going to love, so called gold-plating (McConnell, 1996). Solution: Make priorities based on whether they support key business objectives and processes, but also potential usage frequency, stakeholder importance, formal functionality demand, level of complexity and influence on other requirements. One way of doing this is by checking all use cases and prioritize them based on those criteria. Being strict and making all functional requirements traceable to their origin (with the help of your use cases and priorities) is vital in order to avoid unnecessary requirements (Wiegers, 2000). Also to avoid

gold-plating with the users, as what they need to be able to do with the system rather than what they want (Westfall).2 3.7 Differences among parties involved Unrealistic expectations: Original ideas that sprung up in stakeholders minds may not be complete and too optimistic but still very strong and therefore hard to deal with. Worse is when the BA actually promises those high expectations. Unknown expectations: Users may not know what it is they exactly want or need. Too late: Users may come with new requirements once costs and plan are already set. (McConnell, Rapid Development: Taming Wild Software Schedules, 1996) Vocabularies: The word requirement already means different things to different people (Wiegers, 2000). This is connected to skill, and the BA may not be suitable to bridge the gap between technicians and users. More dangerous even is a belief by the BA that agreement has been reached while in fact it has not (Naz & Khokhar, 2009). This will not show up until the final product is there.3 Friction: as a predecessor to poor communication is friction, caused by the differences between customers and developers. Developers can perceive the customers schedules or (timing of coming with) requirements unreasonable, and the customers may perceive this as not cooperative or wrong supply of the software. If communication is poor, all other steps will not be complete and a poor or unaccepted design may be the effect (Naz & Khokhar, 2009). Solution: Be sure to take notes at every meeting and share these notes with all parties involved. Also make sure to be consistent in use of words. It is most useful to make a glossary of terms, give stakeholders a copy, and always stick to those terms (TechRepublic, 2007). Also bear in mind that even the best requirements cannot replace human dialog (Wiegers, 2005).

4 FRAMEWORK
Although each error could occur in several phases of the development process, it might prove useful to be able to look, aggregated per phase and sub-phase, what else to watch out for. For that end the author includes a table with, per specific activity, aforementioned and other risks and tips that the BA can use as a quick reference4. These are listed in Table 1 (next page).

See http://eprints.ucl.ac.uk/749/1/2.2_rtprob.pdf for an extensive discussion on requirement traceability (Gotel & Finkelstein) 3 See http://tinyurl.com/7ssfu6k for an excellent resource on the characteristics of good software requirements: IEEE (Institute of Electrical and Electronics Engineers) 830: Standard for Software Requirements Specification [IEEE-830] 4 In the case the solution for a risk is obvious (e.g. the risk Lack of an elicitation plan), understanding the solution is assumed (in this case, ensuring to have an elicitation plan before elicitation starts).

Table 1: Activities, risks and tips per phase of the requirements development process
Elicitation phase Acitivity Stakeholder analysis - Identify stakeholders - Define influence, authority, attitude Preperation on elicitation Organize and schedule all needed resources Have a defined business problem / opportunity / case / solution scope Risks Not including all stakeholders Wrongly defining influence Lack of an elicitation plan Inadequate version control No knowledge of domain No well defined problem / case / client Ignoring influence (potential champions) Creating wrong assumptions by focusing too much or too little Implied consent Reading wishes as needs Accepting without impact analysis Lack of commitment of stakeholders Friction Tips Make sure all stakeholders and client(s) are defined

Elicitation Meet with stakeholders to elicit needs Record information provided by stakeholders Validate stakeholder intentions as recorded

Documentation and confirmation

Activity Organize requirements Structure requirements Prioritize requirements

Inadequate document version control Lack of documentation Analysis phase Risks Tips Attempting to satisfy all stakeholders (equal or no prioritizing) Gold-plating: adding unnecessary features Unrealistic or unknown expectations

Use best practice documents available at Business Analysis institutions or in literature. Use a document versioning software tool Set acceptable level of risk and scope Be sure to derive the needs of the stakeholder from the wishes Ask the right question at the right time (open closed) Ensure consent on how non-anonymous information can be used. Use participation and conversation techniques Document everything and disperse copies to stakeholders Use automated version control tools

Prioritize all use cases based on standard criteria Make requirements traceable Test proposals on feasibility Acceptable risk? In scope?

Verify & validate requirements Make sure requirements are correctly defined Assess requirements on

satisfying business needs Specification phase Activity Risks Specify requirements Cutting out nonessentials due to time Specify requirements into one understood by constraints all stakeholders Omitting obvious Model requirements information Translate requirements Ambiguity into models understood by specific stakeholders Verify requirements and models Make sure models are correctly defined Validation phase Activity Risks Validate requirements (final) Requirement changes Validate that model or additions while in satisfies the validation phase (or requirement earlier) Ensure a real-world solution can be build with the model

Tips

Let the predefined scope determine essentials Use all levels of requirement to describe the situation Write tests cases from early on Use a glossary and divide this among stakeholders

Tips

Set a phase or time limit of major requirement change Ask is this in scope? Set up a change control board

(IIBA, 2009; Varvasovszky & Brugha, 2000; Naz & Khokhar, 2009; Wiegers, Karl Wiegers Describes 10 Requirements Traps to Avoid, 2000; Wiegers, Cosmic Truths About Software Requirements, 2005; McConnell, Classic Mistakes Enumerated, 1996; Dorsey, 2000; Westfall; Wahono, 2003; University of Ljubljana, Faculty of Economics, 2012)

CONCLUSION
Requirements are the roots of the final product. The issues presented in this paper show requirement development as a vulnerable process. The author describes the steps in requirements development, the cost of error and therefore the need to press these costs. Issues judged by the author as main issues based on qualitative research followed, including solutions. A summary of these and other risks and tips is added in a table for the benefit of the BA. Results of studies show different cost of errors, but all show these costs are substantial. The particular issues that form or precede these errors are connected with skills, tools, control, involvement and commitment, detail, priority and differences among the parties involved. The solutions are practical and show that good requirements do not come for free but need ever-high focus.

10

REFERENCES
1. Bahill, A. T., & Henderson, S. J. (2004). Requirements Development, Verification and Validation Exhibited in Famous Failures. Systems Engineering, Vol. 8, No. 1. 2. Dorsey, D. P. (2000). Top 10 Reasons Why Systems Projects Fail. Dulcian, Inc. 3. ESI International. (2005). ESI International survey of 2,000 industry professionals. 4. ESI International. (2008). Managing Business Requirements: Implementing Best Practices to Reduce Risk, Rework and Recriminations. ESI International. 5. Gause, D. C., & Lawrence, B. (1999, Jan./Feb.). User driven design. Software Testing & Quality Engineering, 23-27. 6. Gotel, O. C., & Finkelstein, A. C. (n.d.). An Analysis of the Requirements Traceability Problem. London: Imperial College of Science, Technology & Medicine: Department of Computing. 7. Grady, R. B. (1999). An Economic Release Decision Model: Insights into Software Project Management. In Proceedings of the Applications of Software Measurement Conference (p. 227-239). Orange Park, FL: Software Quality Engineering. 8. Grady, R. B. (1999). An Economic Release Decision Model: Insight into Software Project Management. In proceedings of the Applications of Software Measurement Conference, (p. 227-239). Orange Park, USA. 9. Hickey, A. M., & Davis, A. M. (2003). Requirements Elicitation and Elicitation Technique Selection: A Model for Two Knowledge-Intensive Software Development Processes. Proceedings of the 36th Hawaii International Conference on System Sciences. Hawii: Computer Society. 10. IIBA. (2009). The Guide to the Business Analysis Body of Knowledge. Toronto, Canada: International Institute of Business Analysis (IIBA). 11. Jackson, K., Dick, J., & Hull, E. (2010). Requirements Engineering (3rd ed. ed.). London: Springer. 12. Leffingwell, D., & Widrig, D. (2003). Managing software requirements: a use case approach. Boston, U.S.A.: Pearson Education, Inc. 13. McConnell, S. (1996). Classic Mistakes Enumerated. Retrieved May 11, 2012, from Website of Steve McConnell: http://www.stevemcconnell.com/rdenum.htm 14. McConnell, S. (1996). Rapid Development: Taming Wild Software Schedules. Redmond, WA: Microsoft Press. 11

15. Modernanalyst.com. (2012). Interview questions for Business Analysts and System Analysts. Retrieved May 23, 2012, from ModernAnalyst.com`: http://www.modernanalyst.com/Careers/InterviewQuestions/tabid/128/Default.aspx?A rticleType=ArticleView&ArticleID=2033 16. Moore, J. W., Bourque, P., Dupuis, R., Abran, A., & Trip, L. L. (2001). Software Engineering Body Of Knowlegde Project. In J. Marciniak, Encyclopedia of Sotware Engineering (2nd edition ed., p. 1405). New York: John Wiley & Sons Inc. 17. Naz, H., & Khokhar, M. N. (2009). Critical Requirements Engineering Issues & their Solution . International Conference on Computer Modeling and Simulation (p. 218222). IEEE Computer Society. 18. Pohl, K. (1996). Process-Centered Requirements Engineering. Wiley. 19. Serena. (2005). Requirements Management: The Proven Way to Accelerate Development. Serena Software, Inc. 20. Snyder, T., & Shumate, K. (1992). Kaizen Project Management. American Programmer. 21. Sommerville, I., & Kotonya, G. (1998). Requirements engineering. London: John Wiley & sons. 22. TechRepublic. (2007, January 2). Five common errors in requirement analysis (and how to avoid them). Retrieved May 10, 2012, from TechRepublic: http://www.techrepublic.com/article/five-common-errors-in-requirements-analysisand-how-to-avoid-them/6146544 23. University of Ljubljana, Faculty of Economics. (2012). Business Analysis (presentation). Ljubljana. 24. Varvasovszky, Z., & Brugha, R. (2000). A stakeholder analysis. Health Policy and Planning, p. 338 - 345. 25. Wahono, R. S. (2003). ANALYZING REQUIREMENTS ENGINEERING PROBLEMS. Proceedings of the IECI Japan Workshop 2003 (pp. 55-58). Chofu Bunka Kaikan Tazukuri, Japan: Department of Information and Computer Sciences, Saitana University. 26. Waters, D. (1999). Operations Management. Kogan Page Ltd. 27. Westfall, L. (n.d.). The What, Why, Who, When and How of Software Requirements. Plano, United States of America. 28. Wiegers, K. E. (2000, Jan./Feb.). Karl Wiegers Describes 10 Requirements Traps to Avoid. Software Testing and Quality Engineering(Vol. 2, No.1), p. 34-40. 12

29. Wiegers, K. E. (2003). Software Requirements, 2nd Edition. Microsoft Press. 30. Wiegers, K. E. (2005). Cosmic Truths About Software Requirements. Sponsored by Brewer Consulting Services. 31. Willis, R. R., Rova, R. M., Scott, M. D., Johnson, M. J., Ryskowski, J. F., Moon, J. A., . . . Shumate, K. C. (1998). Hughes Aircraft's Widespread Deployment of a Contiuously Improving Software Process. Software Engineering Instiute.

13

Appendix I Ambigous words to avoid when writing requirements (Wiegers, 2000) Minimize Maximize Optimize Rapid User-friendly Easy Simple Often Normal Usual Large Intuitive Robust State-of-the-art Improved Efficient Flexible and/or etc.