Académique Documents
Professionnel Documents
Culture Documents
Rob Morris
CDL Systems
Outline
1. My Background
2. Why do we estimate?
3. How to estimate
a. Traditionally
b. Agilely
4. Firm Fixed Price Estimation
5. The need for Actuals
CDL Systems
October 12, 2006 www.cdlsystems.com
Background
Pre 1995 - How do we manage software development projects?
1995 - CDL – small contractor, control station for UAVs
1997 - GD Canada (CDC) – fairly large Canadian defence contractor
1998 - Alliant Techsystems – larger American defence contractor
1999 - AAI – Airplane defence contractor – CMM level 5
2002 - Northrop Grumman – Huge defence contractor, F14, F18,
Stealth Bomber CMMI level 5
CDL Systems
October 12, 2006 www.cdlsystems.com
Why do we estimate?
To Plan – When will something be done?
To Schedule – What order should we do things in?
To Hire – Do we need more people to do the work?
To Price – How much will it cost?
To Guide Investment – Is doing something worth it?
CDL Systems
October 12, 2006 www.cdlsystems.com
Definitions
• Size – something that can be counted/measured.
Hopefully it is representative of effort
• Effort – the actual hours required to write the software.
• Duration – the time on the calendar to get something
done
• Cost – strongly correlated with effort, but duration also
plays a role
CDL Systems
October 12, 2006 www.cdlsystems.com
Estimation is a Probability
0.25
0.2
0.15
0.1
0.05
0
0 5 10 15 20 25
CDL Systems
October 12, 2006 www.cdlsystems.com
Cumulative Distribution
1.2
0.8
0.6
0.4
0.2
0
0 5 10 15 20 25
CDL Systems
October 12, 2006 www.cdlsystems.com
Reality of Estimates
Todd Little at Landmark Graphics did a
study of 100+ projects.
Data coincided with results found by Tom
Demarco
CDL Systems
October 12, 2006 www.cdlsystems.com
Results
CDL Systems
October 12, 2006 www.cdlsystems.com
Results con’t
CDL Systems
October 12, 2006 www.cdlsystems.com
Parkinson’s Law
Effort expands to fill the time
Time
0.25
0.2
0.15
0.1
0.05
0
0 5 10 15 20 25
-0.05
CDL Systems
October 12, 2006 www.cdlsystems.com
Traditional Estimation
Pick a size metric – SLOC, function points
Select the value for a variety of cost
drivers
Experience of requirements analyst
Type of project
CDL Systems
October 12, 2006 www.cdlsystems.com
COCOMO II
16 cost drivers
Product of the influence of all 16 cost drivers is
1454.
Two different projects of the same size (in
SLOC)
$10,000 and $14.54 million
Changes in the cost drivers from one project to
the next can have a large impact on projects of
the same ‘size’.
CDL Systems
October 12, 2006 www.cdlsystems.com
Actual Data at CDL Systems
Post mortem analysis within CDL Systems
saw variations from 6 LOC/hr to 18 LOC/hr
on very similar projects.
Factor of 3 with perfect knowledge of
SLOC (not a very good estimate)
CDL Systems
October 12, 2006 www.cdlsystems.com
Lesson Learned
The client (or even one person) can have a huge
impact on cost and it can be completely out of
your control.
Classify clients like hurricanes, Category 1 to 5.
Tardy clients can be costly if they can’t make a
decision
Agile – Don’t start developments without
requirements for the iteration.
CDL Systems
October 12, 2006 www.cdlsystems.com
Agile Estimation
Estimate at 3 scales
Iteration Plan Estimation
Release Plan Estimation
Project Estimation
CDL Systems
October 12, 2006 www.cdlsystems.com
Estimation Units
Ideal Time – how long a task takes if there were no
interruptions
Story Point – relative measurement among User Stories
User Requirements
Feature Lists
Use Case Scenario
CDL Systems
October 12, 2006 www.cdlsystems.com
Techniques
Analogous Estimation
Planning Poker
CDL Systems
October 12, 2006 www.cdlsystems.com
Analogous Estimation
This Story is like another Story (maybe a little
more difficult, maybe a little less)
Give this Story a comparable estimated value.
Estimate against multiple Stories of the same
effort.
Analogous estimation is successful due to our
inherent ability to better measure relative size
than absolute size.
CDL Systems
October 12, 2006 www.cdlsystems.com
Planning Poker
1. Each estimator is given a deck of cards, each card contains a valid
estimate.
Power of 2 – 1,2,4,8,16,32
Fib – 1,2,3,5,8,13,20,30
2. Story is read and discussed briefly
3. Each estimator selects a card that reflects their estimate.
4. Cards are turned over for all to see.
5. Discussion takes place
6. Re-estimate to try to get convergence.
CDL Systems
October 12, 2006 www.cdlsystems.com
Iteration Planning
Entire team gets together at the beginning
of an iteration.
Can include outside stakeholders
Estimate each of the items
Prioritize the items
Use the team velocity to stack the tasks
into the iteration.
CDL Systems
October 12, 2006 www.cdlsystems.com
Estimating Velocity
Use historical data
Run an iteration
Make a forecast
CDL Systems
October 12, 2006 www.cdlsystems.com
Historical Data
Previous iteration
Running average of past iterations
Valid, only when little has changed from one
iteration to the next
Technology
Domain
Product Owner
Tools
Working Environment
Team Size and Makeup
Estimators
CDL Systems
October 12, 2006 www.cdlsystems.com
Run an Iteration
Essentially performs a trial
Calibrates the historical data for the
current environment.
CDL Systems
October 12, 2006 www.cdlsystems.com
Forecast
Estimate ideal hours (per developer, per iteration)
Allowances for staff meetings, status meeting, planning session
Allowance for new staff, interns, etc. (60% for 6 months)
Allowances for training
Allowances for stat holidays, vacation, flex time
Be aware of summer projects
Allowances for doctor, dentist, cable guy, Friday lunch
25% overhead to the team lead per developer
Eg – 5 person team, 1 intern, 75 hrs/iteration/person
On paper 375 hrs
Actually with allowances – 217 (~40%)
Determine ideal hours per iteration
Expand user stories into tasks, estimate the tasks, fill the
iteration.
Tasks are like Lego blocks and must completely fit within an iteration
CDL Systems
October 12, 2006 www.cdlsystems.com
Release Planning
Similar to iteration planning but extend the technique to multiple
iterations
Tasks are generally coarser.
Two basic techniques
Given a release date, continue filling content until the date is reached.
Given content, continue adding iterations until content is completed.
Schedule can be shrunk somewhat with additional resources but the
best that can be hoped for is about 25% improvement in schedule
with a 50% increase in staff.
Maximum team size 5 (including lead)
Use forecasting and history if available.
Allow time for bug fixing – perhaps allocate an entire iteration
CDL Systems
October 12, 2006 www.cdlsystems.com
Firm Fixed Price Estimate
Similar to Rough order of magnitude
(ROM), or Not to exceed (NTE)
Inherently risky if we use 50% probability
estimates
Many of us have to live in this world
Sole Source and Competitive Bid
CDL Systems
October 12, 2006 www.cdlsystems.com
Firm Fixed Price Technique
CDL Systems
October 12, 2006 www.cdlsystems.com
Example
Story 50% (days) 90% (days) Δ2
Plan Flight 15 25 100
Control Weapons 5 10 25
Launch Vehicle 5 7 4
Land Vehicle 20 40 400
Total 45 529
Total = 45 + √529 = 68
CDL Systems
October 12, 2006 www.cdlsystems.com
Advantages
The greater the difference between the 90%
estimate and the 50% estimate is an indicator
of risk
Provide developers with the 50% numbers to
complete the tasks.
50% and 90% confidence can be tested for
accuracy by looking at actuals for all tasks.
CDL Systems
October 12, 2006 www.cdlsystems.com
Scope Creep and Rework
Scope Creep – new requirements that
were never allocated for in the estimate
Rework – Doing work more than once
because client couldn’t make up their
mind
CDL Systems
October 12, 2006 www.cdlsystems.com
Scope Creep and Rework
Keep track of out of scope and rework items
Every item that is added to your backlog qualify it as in
scope, rework or out of scope
Treat them like any other backlog item
Even if you can’t recapture out of scope and rework in
this contract, beware of this contractor and adjust
estimates in subsequent contracts with this contractor
accordingly
Trying to tighten loopholes in contracts doesn’t work –
leads to red tape and animosity
Important to keep track of to determine the accuracy of
the original estimate
CDL Systems
October 12, 2006 www.cdlsystems.com
Actuals
Pure Agilist – Actuals aren’t necessary
Scrum – more concerned with time remaining to create a burn
down
XP – weaker concept of ‘Yesterday’s weather’ which is just
number of story points completed by a team in an iteration.
Use of velocity somewhat circumvents need for actuals,
but it is course grained.
Velocity is a team actual
CDL Systems
October 12, 2006 www.cdlsystems.com
Need for Actuals
Firm Fixed Price Bidding needs Actuals
Need the historical data as a frame of reference for
forecasting
When teams are shuffled, individual velocity can aid in
estimation
Team concept is great, however we don’t pay teams, we
pay individuals
Individual accomplishments should not be overlooked.
Actuals provide feedback to staff to improve their
estimates on subsequent iterations
CDL Systems
October 12, 2006 www.cdlsystems.com
Final Words
Being agile does not circumvent need for
estimates
Traditional ‘size to effort’ techniques have their
problems (cost driver sensitivity, size estimation)
Can still use agile techniques in Firm Fixed Price
environments
Clients are a huge cost driver – make allowances
for the client
Still need actuals
CDL Systems
October 12, 2006 www.cdlsystems.com
References
CDL Systems
October 12, 2006 www.cdlsystems.com