Académique Documents
Professionnel Documents
Culture Documents
Agile Methods
Using Return on Investment
Dr. David F. Rico, PMP, CSM
Agenda
) Introduction
Sources of Business Value
Surveys of Business Value
Measures of Business Value
Models of Business Value
Estimation of Business Value
Comparison of Business Value
Summary of Business Value
2
Author
DoD contractor with 25+ years of IT experience
B.S. Comp. Sci., M.S. Soft. Eng., D.M. Info. Tech.
Large NASA & DoD programs (U.S., Japan, Europe)
* Published five textbooks and over 15 articles on various topics in return on investment, information technology, agile methods, etc.
3
Purpose
Provide an overview of the business value of
Agile Methods using return on investment:
Business value is an approach for estimating the
tangible and intangible worth of organizational assets
Business value is an appraisal of intellectual
assets such as knowledge, experience, and skills
Business value is a technique for determining the
complete worth of an investment to an enterprise
Business value is a method of determining the
health and well-being of a firm in the long-run
Business value includes employee, customer,
supplier, alliance, management, and societal value
4
What is Agility?
A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness,
and ease of movement; nimbleness
Agility is the ability to create and respond to change
in order to profit in a turbulent business environment
Agility is reprioritizing for maneuverability because
of shifting requirements, technology, and knowledge
Agility is a very fast response to changes in customer
requirements through intensive customer interaction
Agility is the use of adaptability and evolutionary
delivery to promote rapid customer responsiveness
Agility is a better way of developing products using
collaboration, teamwork, iterations, and flexibility
5
What are Agile Methods?
‘Adaptable’ software development methodologies
‘Human-centric’ method for creating business value
‘Alternative’ to large document-based methodologies
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
6
Essence of Agile Methods
High degree of customer & developer interaction
Highly-skilled teams producing frequent iterations
Right-sized, just-enough, and just-in-time process
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
8
Antecedents of Agile Methods
Rooted in management evolution from early 1900s
Evolved from software methods from 1950s/1960s
Spinoffs of NPD/RAD approaches from the 1980s
Rico, D. F., Sayani, H. H., & Field, R. F. (2008). History of computers, electronic commerce, and agile methods. In M. V. Zelkowitz (Ed.), Advances in
computers: Emerging technologies, Vol. 73. San Diego, CA: Elsevier. 9
Agenda
Introduction
) Sources of Business Value
Surveys of Business Value
Measures of Business Value
Models of Business Value
Estimation of Business Value
Comparison of Business Value
Summary of Business Value
10
Types of Agile Methods
Crystal Methods and Scrum 1st Agile Methods
Extreme Programming swept the globe by 2002
Scrum/Extreme Programming hybrids are popular
Year Method Author Firm Major Features
Crystal Use Cases, Domain Models, Frequent Delivery,
1991 Cockburn IBM
Methods Reflection Workshops, Risk Management
Dynamic
User Involvement, Time Boxes and Prototypes
1993 Systems Millington DSDM
Development
(Iterations), Testing and Quality Assurance
Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.
13
Dynamic Systems Develop.
Created by group of British firms in 1993
15 practices, 12 roles, and 23 work products
Non-proprietary RAD approach from early 1990s
Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.
14
Feature Driven Development
Created by Jeff De Luca at Nebulon in 1997
Has 8 practices, 14 roles, and 16 work products
Uses object-oriented design and code inspections
Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.
15
Extreme Programming
Created by Kent Beck at Chrysler in 1998
Has 28 practices, 7 roles, and 7 work products
Popularized pair programming and test-driven dev.
Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.
16
Extreme Programming (cont’d)
RELEASE PLANNING — Best Practice
Created by Kent Beck at Chrysler in 1998
Used for managing both XP and Scrum projects
Light, flexible, and adaptable project mgt. framework
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
17
Extreme Programming (cont’d)
PAIR PROGRAMMING — Best Practice
Term coined by Jim Coplien in 1995
Consists of two side-by-side programmers
Highly-effective group problem-solving technique
Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.
18
Extreme Programming (cont’d)
TEST-DRIVEN DEV. — Best Practice
Term coined by Kent Beck in 2003
Consists of writing unit tests before coding
Subject to automated testing/continuous integration
Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.
20
Agenda
Introduction
Sources of Business Value
) Surveys of Business Value
Measures of Business Value
Models of Business Value
Estimation of Business Value
Comparison of Business Value
Summary of Business Value
21
Surveys of Agile Methods
Numerous surveys of Agile Methods since 2003
AmbySoft and Version One collect annual data
Generally include both hard and soft benefits
Rico, D. F. (2008). What is the return-on-investment of agile methods? Retrieved February 3, 2009, from http://davidfrico.com/rico08a.pdf
22
Shine Technologies
Survey of 131 international respondents
Extreme Programming (58%) and Scrum (8%)
85% of respondents were experts in Agile Methods
Cost
Satisfaction
Quality
Productivity
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Improvement
Johnson, M. (2003). Agile methodologies: Survey results. Victoria, Australia: Shine Technologies.
23
Agile Journal
Survey of 400 international respondents
Extreme programming (28%) and Scrum (20%)
80% using Agile Methods to deliver maximum value
Cost
Alignment
Quality
Time to Market
Improvement
Barnett, L. (2006). And the agile survey says. Agile Journal, 1(1).
24
Microsoft
Survey of 492 Microsoft respondents
Scrum (65%) and Extreme Programming (5%)
65% using Agile Methods in virtual distributed teams
Productivity
Satisfaction
Quality
Flexibility
Time to Market
Communication
Improvement
Begel, A., & Nagappan, N. (2007). Usage and perceptions of agile software development in an industrial context: An exploratory study.
Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, Madrid, Spain, 255-264. 25
UMUC
Survey of 250 international respondents
70% of respondents using Agile Methods
83% of were from small-to-medium sized firms
Cost
Quality
Cycle Time
Productivity
Satisfaction
Improvement
Rico, D. F., Sayani, H. H., Stewart, J. J., & Field, R. F. (2007). A model for measuring agile methods and website quality. TickIT International, 9(3), 3-15.
26
AmbySoft
Survey of 642 international respondents
69% of firms had adopted an Agile Method
62% were from firms with less than 1,000 people
Virtual Success
Cost
Quality
Project Success
Satisfaction
Productivity
Improvement
Ambler, S. W. (2008). Agile adoption survey. Retrieved October 17, 2008, from http://www.ambysoft.com/downloads/surveys/AgileAdoption2008.ppt
27
IT Agile
Survey of 207 respondents in Germany
Scrum (21%) and Extreme Programming (14%)
97% of respondents are satisfied with Agile Methods
Quality
Project Status
Productivity
Job Satisfaction
Flexibility
Improvement
Wolf, H., & Roock, A. (2008). Agile becomes mainstream: Results of an Online Survey. Object Spektrum, 15(3), 10-13.
28
Version One
Survey of 3,061 respondents from 80 countries
Scrum (49%), Scrum/XP (22%), and XP (8%)
68% from small firms and 57% distributed
Cost
Maintainability
Quality
Productivity
Project Visibility
Cycle Time
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Improvement
Version One. (2008). The state of agile development: Third Annual Survey. Alpharetta, GA: Author.
29
Agenda
Introduction
Sources of Business Value
Surveys of Business Value
) Measures of Business Value
Models of Business Value
Estimation of Business Value
Comparison of Business Value
Summary of Business Value
30
Measures of Business Value
A major principle of Agile Methods is creating value
ROI is the measure of value within Agile Methods
There are seven closely related ROI measures
Metric Definition Formula
Costs
n
Sum of Costs
Total amount of money spent ∑ Cost
i =1
i
Benefits
n
Sum of Benefits
Total amount of money gained ∑ Benefit
i =1
i
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
31
Costs
Total amount of money spent on Agile Methods
Includes training, coaching, automated tools, etc.
Minimally, includes the dev. effort of Agile Methods
∑ Cost
i =1
i
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
32
Benefits
Total amount of money gained from Agile Methods
Includes economic benefit from using new system
Minimally, includes maintenance rework savings
∑ Benefit
i =1
i
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
33
Benefit to Cost Ratio
Ratio of total benefits to total costs of Agile Methods
Includes development, maintenance, and business
Minimally, benefits should be larger than all costs
Benefits
Costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
34
Return on Investment
Ratio of adjusted benefits to costs of Agile Methods
Benefits are adjusted downward using total costs
Minimally, benefits should be larger than costs
Benefits − Costs
× 100%
Costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
35
Net Present Value
Discounted benefits of using Agile Methods
Future benefits are discounted due to inflation
Minimally, future benefits should exceed all costs
Years
Benefitsi
∑
i = 1 (1 + Discount Rate)
Years
− Costs0
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
36
Breakeven Point
Point when benefits exceed costs of Agile Methods
Point where slope of benefits and costs intersect
Minimally, old costs should exceed new costs
New Costs
Old Costs New Costs − 1
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
37
Real Options Analysis
Iterative benefits gained from using Agile Methods
Future benefits are increased because of risks
Minimally, benefits should exceed costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
38
Agenda
Introduction
Sources of Business Value
Surveys of Business Value
) Models of Business Value
Measures of Business Value
Estimation of Business Value
Comparison of Business Value
Summary of Business Value
39
Software Lifecycle Costs
1:10:100 is a classical ratio of dev. to maint. hours
Defects have negative multiplicative effect on cost
A conservative and contemporary ratio is 1:6:30
Relative Cost to Fix Error
Benediktsson, O., & Dalcher, D. (2005). Estimating size in incremental software development projects. Journal of Engineering Manufacture, 152(6), 253-259.
41
Total Lifecycle Costs
0.51 hours/line of code for Traditional Methods
10% defect inject rate (1,000 defects/10 KLOC)
67% of defects in test (33% in maintenance)
Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.
In, H. P., et al. (2006). A quality-based cost estimation model for the product line life cycle. Communications of the ACM, 49(12), 85-88.
McCann, B. (2007). The relative cost of interchanging, adding, or dropping quality practices. Crosstalk, 20(6), 25-28. 42
Agile Productivity Studies
Productivity data found in 26 Agile Methods studies
Studies conducted from 2002 to the present time
Average productivity 21.2374 LOC per hour
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
43
Agile Productivity Models
Based on 13 studies of Extreme Programming (XP)
Also based on 7 studies of pair programming (PP)
“Pair programming” had highest productivity
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
44
Agile Quality Studies
Defect density found in 21 studies of Agile Methods
Studies conducted from 2002 to the present time
Average quality 1.7972 defects per KLOC
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
45
Quality Models
Based on 10 studies of Extreme Programming (XP)
Also based on 6 studies of pair programming (PP)
“Extreme Programming” had the highest quality
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
46
Agenda
Introduction
Sources of Business Value
Surveys of Business Value
Measures of Business Value
Models of Business Value
) Estimation of Business Value
Comparison of Business Value
Summary of Business Value
47
Agile Lifecycle Costs
Costs based on productivity and quality models
Development costs based on LOC ÷ productivity rate
Maintenance costs based on defects × KLOC × MH
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
48
Agile Lifecycle Benefits
Benefits based on total traditional less agile costs
Traditional costs based LOC × dev. + maint. effort
Traditional costs credited testing effort applied
No. Method Agile Lifecycle Benefit Models Benefits
1. XP (10,000 3.51 – 3,651.48 4.47) 100 – $208,069 $1,667,079
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
49
Extreme Programming
Costs based on avg. productivity and quality
Productivity moderated from 16.1575 to 5.3858
Costs were $208,069, benefits were $1,667,079
∑
5
i =1
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
50
Test Driven Development
Costs based on avg. productivity and quality
Productivity moderated from 29.2800 to 9.7600
Costs were $167,109, benefits were $1,708,039
∑
5
i =1
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
51
Pair Programming
Costs based on avg. productivity and quality
Productivity moderated from 33.4044 to 11.135
Costs were $160,459, benefits were $1,714,690
∑
5
i =1
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
52
Scrum
Costs based on avg. productivity and quality
Productivity data remained the same at 5.4436
Costs were $302,052, benefits were $1,573,096
∑
5
i =1
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
53
Agile Methods
Costs based on avg. productivity and quality
Productivity data resulted in average of 7.9311
Costs were $180,002, benefits were $1,695,146
∑
5
i =1
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
54
Agenda
Introduction
Sources of Business Value
Surveys of Business Value
Measures of Business Value
Models of Business Value
Estimation of Business Value
) Comparison of Business Value
Summary of Business Value
55
Data for Agile Methods
Agile Methods were ranked based on ROI
Agile Methods with high quality had lower ROI
Agile Methods with high productivity had high ROI
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
56
ROI of Agile Methods
Agile Methods were ordered based on ROI
Agile Methods had a high ROI value of 969%
Agile Methods yielded an average ROI of 842%
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
57
Data for Traditional Methods
Traditional Methods were ranked based on ROI
Methods with good cost and quality had higher ROI
Agile Methods had better ROI than heaviest methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
58
ROI of Traditional Methods
Traditional Methods were ordered using ROI
Traditional Methods had high ROI value of 1,562%
Agile Methods had better ROI than heaviest methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
59
Data for All Methods
Software methods were ranked based on ROI
Methods with good cost and quality had best ROI
Best Agile and Traditional Methods had similar ROI
Type Method Costs Benefits B/CR ROI NPV BEP ROA
Traditional PSPsm $105,600 $1,755,148 17:1 1,562% $1,414,174 $945 $1,672,907
Traditional TSPsm $148,400 $1,706,648 12:1 1,050% $1,329,379 $5,760 $1,591,127
Traditional Inspections $82,073 $897,499 11:1 994% $695,067 $51,677 $833,681
Agile PP $160,459 $1,714,690 11:1 969% $1,324,283 $5,919 $1,590,034
Agile TDD $167,109 $1,708,039 10:1 922% $1,311,874 $6,430 $1,578,575
Agile Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,556,997
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
60
ROI of All Methods
Software methods were ordered by ROI
Agile Methods had a high ROI value of 969%
Traditional Methods had high ROI value of 1,562%
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
61
Unadjusted ROI of All Methods
Are data based on unrealistic laboratory conditions?
Are productivity data from lab studies optimistic?
Are total lifecycle costs closer to 1:10:100?
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
62
Agenda
Introduction
Sources of Business Value
Surveys of Business Value
Measures of Business Value
Models of Business Value
Estimation of Business Value
Comparison of Business Value
) Summary of Business Value
63
Benefit Summary
Agile (138 pt.) and Traditional Methods (99 pt.)
Agile Methods fare better in all benefits categories
Agile Methods 459% better than Traditional Methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
64
Cost of Quality
Apply traditional reliability and quality theory
Defects are inexpensive to remove early in cycle
Late bug removal has negative, multiplicative effect
Inspection Cost (57X PSP) Ad Hoc (326X)
PSP Cost (326X lower than Ad Hoc) Test Cost (138X PSP)
Software Defects
PSP
Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI) strategies. Rome, NY: DACS.
65
Real Options
NPV models losses of Traditional Methods
Real options model profits from Agile Methods
Agile Methods incur less initial risk and higher ROI
Probability
Probability
Fichman, R. G., Keil, M., & Tiwana, A. (2005). Beyond valuation: Options thinking in IT project management. California Management Review, 47(2), 74-96.
66
Agile vs. Traditional Metrics
Agile Methods are a fundamentally new paradigm
Agile Methods are “not” lighter Traditional Methods
They should not be viewed through a Traditional lens
Rico, D. F. (2009). Metrics for agile methods. Retrieved February 7, 2009, from http://davidfrico.com/agile-metrics.pdf
67
New Book
Guide to Agile Methods for business leaders
Communicates business value of Agile Methods
Rosetta stone to Agile Methods for Traditional folks
THE BUSINESS VALUE Table of Contents
OF AGILE METHODS 1. Introduction
2. Values of Agile Methods
Maximizing ROI with Right-Sized, Just-Enough, 3. History of Agile Methods
and Just-in-Time Processes and Documentation 4. Antecedents of Agile Methods
5. Types of Agile Methods
6. Practices of Agile Methods
7. Agile Project Management
8. Agile Software Engineering
9. Agile Support Processes
10. Agile Tools and Technologies
11. Comparison of Agile Methods
12. Agile Metrics and Models
13. Costs of Agile Methods
14. Benefits of Agile Methods
15. ROI of Agile Methods
DR. DAVID F. RICO, DR. HASAN H. SAYANI 16. NPV of Agile Methods
AND DR. SAYA SONE 17. Real Options of Agile Methods
Forward by Dr. Jeffrey V. Sutherland 18. Conclusion
* Rosetta stone to the business value and culture of Agile Methods for executives, managers, and thought leaders in the field of software methods.
68
References
Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI)
strategies. Rome, NY: DACS.
Rico, D. F. (2002). How to Estimate ROI for Inspections, PSP, TSP, SW-CMM, ISO 9001, and
CMMI. Software Tech News, 5(4), 23-31.
Rico, D. F. (2002). The Return on investment in quality. TickIT International, 4(4), 13-18.
Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and
software engineers. Boca Raton, FL: J. Ross Publishing.
Rico, D. F. (2005). Practical metrics and models for return on investment. TickIT International, 7(2),
10-16.
Rico, D. F. (2006). A framework for measuring the ROI of enterprise architecture. International
Journal of End User Computing, 18(2), 1-12.
Rico, D. F. (2007). Optimizing the ROI of enterprise architecture using real options. In S. Clarke
(Ed.), End user computing challenges and technologies: Emerging tools and applications.
Hershey, PA: Information Science Reference.
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? An analysis of extreme
programming, test-driven development, pair programming, and scrum (using real options). TickIT
International, 10(4), 9-18.
Solingen, R. A., & Rico, D. F. (2006). Calculating software process improvement’s return on
investment. In M. V. Zelkowitz (Ed.), Advances in computers: Quality software development, Vol.
66 (pp. 1-41). San Diego, CA: Elsevier.
69
Contact Information
Website: http://davidfrico.com
Biography: http://www.linkedin.com/in/davidfrico
Capabilities: http://davidfrico.com/rico-capability.pdf
70