AACE International

Risk Management SK.02

Skills & Knowledge of Cost Engineering
2003 Annual Meeting Orlando, FL
Copyright 2003 Decision Sciences Corp.

Michael W. Curran
u u

Education:
Mathematics and Physics, with honors Washington University

Employment:
1968-Present: Founder, President and CEO Decision Sciences Corporation Before 1968: Research Monsanto; Physics Instructor Washington University

Professional Organizations:
AACE: Chair Decision and Risk Management Committee; Editorial Advisor Cost Engineering magazine infORMS, PMI, SCEA, Sigma Xi

Developer of:
Range Estimating, Bracket Budgeting, codeveloper of Risk Established Value (REVTM)

Publications:
Handbook of Budgeting, Effective Project Management through Applied Cost and Schedule Control, Professional Practice Guide to Risk, et alia

u u

Affiliations:
Former Advisor Entrepreneurial Business Center, University of Missouri

Recognition:
AACE Technical Excellence Award Whos Who in Science and Engineering, Whos Who in Finance and Industry Whos Who in America, Whos Who in the World

Copyright 2003 Decision Sciences Corp.

What is risk?
Risk An ambiguous term that can mean any of the following:
All uncertainty (threats and opportunities); Downside uncertainty (a.k.a. threats); or The net impact or effect of uncertainty (threats minus opportunities)

The convention used in any work should be clearly stated to avoid misunderstandings.
AACE Internationals Risk Management Dictionary
Copyright 2003 Decision Sciences Corp.

What is Risk Management?

The number of definitions of the term Risk Management is perhaps equal to the number of its practitioners. Simply put however, Risk Management (RM) is the application of scientific management principles to the identification, analysis, mitigation, and control of risk.

Copyright 2003 Decision Sciences Corp.

Overview
u

A peek into the real world of RM

the Good, the Bad, the Truly Ugly

The steps in RM
Identify Analyze Mitigate Control the Risk

On becoming a risk analyst

5

Copyright 2003 Decision Sciences Corp.

A peek into the real world of RM

Founded in 1968, Decision Sciences Corporation (DSC) is a consulting firm whose practice is self-limited to decision sciences, with particular interest in risk-based decisions faced by top executives. For over three decades, DSC has counseled senior managers in thousands of high risk decisions ranging in size from the very small to the extremely large, many in the multi-billion dollar range. The applications have been equally diverse: capital projects, operational budgeting and control, competitive bidding, R&D, new products and services, mergers and acquisitions, enterprise resource planning (ERP), joint ventures, negotiating, and production enhancements. The spectrum of clients includes varied industries, education, and many governmental units (municipal to federal, including the U.S. Central Intelligence Agency). What has been learned from all of this? How do risk analysts measure up? How much of a role does technology play? How does institutional culture help or hinder the process? How can decision makers be best prepared for making risky decisions? What are the common (and uncommon) pitfalls? How can the potholes be avoided? What path will this form of managerial assistance likely take over this decade? What will be the proper role of risk? The following evidence provides some important clues!
Copyright 2003 Decision Sciences Corp.

Growth of interest in RM
AACE Articles on RM
180 160 140 120 100 80 60 40 20 0 1960s 1970s 1980s 1990s
Copyright 2003 Decision Sciences Corp.

A Google Web search yielded 3,660,000 hits on Risk Analysis

7

Flawed RM
u u u u

Frequently looks valid Introduces more risk Is a juicy litigation target Becoming ever more common

Copyright 2003 Decision Sciences Corp.

Flawed RM: Examples

u

Major petrochemical company

More than 20% of projects finish with actual costs above their risk analyzed ranges

Engineering firm
Going from 60% to 90% confidence required but $31,000 in $26 million early stage project (Firm teaches risk analysis to its clients!)

u u

Manufacturing company
$4 million risk error in $89 million project

U.S. research facility

Overruns risk budgets 94% of the time
9

Copyright 2003 Decision Sciences Corp.

Flawed RM: Origins

u u

Lack of commitment and understanding Belief that Risk Management is primarily a technical consideration rather than a psychological and political one Belief that risk cannot be adequately controlled

Copyright 2003 Decision Sciences Corp.

10

Flawed RM: Symptoms

u

Lack of commitment and understanding

Used to sell the plan to management No action plans developed from risk analysis Little or no use of RM after appropriation Few releases of contingency to management Contingency regularly exhausted (Parkinson) Actual bottom line often outside of its range

Copyright 2003 Decision Sciences Corp.

11

Flawed RM: Symptoms

u

Belief that Risk Management is primarily a technical consideration rather than a psychological and political one
Inordinate focus on
software statistics industry expertise

Undue attention paid to decisions technical characteristics Results vary with decision size or complexity Risk surprises occur regularly
Copyright 2003 Decision Sciences Corp.

12

Flawed RM: Symptoms

u

Belief that risk cannot be adequately controlled

Applying RM only when mandated Performing cookie cutter risk analysis Discarding input data extremes that werent Explaining errors by hiding behind the original assumptions Blaming poor results on scope creep Ignoring de-scoping in budgetary successes Allowing success to mask lack of expertise
13

Copyright 2003 Decision Sciences Corp.

Flawed RM: Symptoms

u

Belief that the decisions downstream characteristics cannot be analyzed

Belief that most business risks are not within the grasp of modern management techniques Heavy reliance on exceptional execution to bring about business or operational success Fear of finding killer downstream risks

Copyright 2003 Decision Sciences Corp.

14

Good RM: Bedrock components

u

Managers exposed to decision analysis u Ardent management commitment u True risk analysis expertise (per person)
25 to 50 risk analyses performed per year Solid cross-industry experience Multiple business function experience

Risk Management culture in place u Business risk also analyzed u Benchmark and grade no excuses!
u
Copyright 2003 Decision Sciences Corp.

15

Good RM: Characteristics

u u

Predictable results Monitoring to determine mid-course corrections Risk-based action plans developed, analyzed, and implemented No hiding behind assumptions and scope creep Periodic releases of contingency to improve overall capital effectiveness
16

Copyright 2003 Decision Sciences Corp.

Good RM: Example 1

u

Chemical Company
Circumstances
Goal was to achieve cost underruns in 95% of projects $1.00 over budget is classified as an overrun

Results of RM
More than 500 projects completed and analyzed 96% of those projects underran

Copyright 2003 Decision Sciences Corp.

17

Good RM: Example 2

u

Heavy Industrial Company

Circumstances
Goal was to achieve cost underruns in 90% of projects $1.00 over budget is classified as an overrun

Results of RM
Over $600 million of projects finished and analyzed 92% of those projects underran Total of $600 million was underrun by about 1% All projects that overran did so by less than 10% No actual bottom line materialized outside its range Contingency was released whenever Risk < Budget

Copyright 2003 Decision Sciences Corp.

18

Good RM: Example 3

u

Research & Development

Circumstances
New weapons $42 million budget was risk analyzed A risk analysis audit was mandated by U.S. Army

Results of RM
Audit found 3X more risk than original risk analysis Contingency underfunded by $12 million -- $11 million of which was expended Settled dispute between contractor and U.S. General Accounting Office (GAO) Increased confidence of U.S. Congress in its weapons appropriation process Process was mandated for joint U.S. and European weapons development
Copyright 2003 Decision Sciences Corp.

19

Good RM: Example 4

u

Information Technology
Circumstances
Enterprise Resource Planning (ERP) project Original budget: $23 million, 22 months, 12 modules

Results of RM
Exposed integration firms lack of understanding of the unique nature of risk in the project Integration firm fired due to its inability to look beyond the sales bias in its original estimate Owners staff managed project to completion by adhering to recommendations resulting from RM Actual (Predicted): $54 million, 27 months, 8 modules ERP vendor refers to this as best in that industry Lead executive promoted to CFO of holding company
Copyright 2003 Decision Sciences Corp.

20

The future of RM
u u u u u u

Benchmarking and Grading will become de rigueur Faster, better decisions with limited and faulty data Expanded use will heighten profits Forensic use in the legal area will bring on better planning and control systems Risk Management will become holistic RM expertise will be ever more scarce
21

Copyright 2003 Decision Sciences Corp.

The future of Risk Management

More Golf
Copyright 2003 Decision Sciences Corp.

22

The steps in RM
u u u u

Identify ... Analyze ... Mitigate ... Control the Risk

Copyright 2003 Decision Sciences Corp.

23

Steps in RM: Identify the risks

u u u u

Search through historical information? Interview key people, one-on-one? Hold a brainstorming session? Hold a risk oriented brainstorming session facilitated by a risk expert?

Copyright 2003 Decision Sciences Corp.

24

Steps in RM: Identify the risks

u

Search through historical information?

Advantages
Such information is often highly useful

Disadvantages
Historical information tends to be voluminous It is often corrupted in unknown ways It is rarely accessible in a risk friendly format

Copyright 2003 Decision Sciences Corp.

25

Steps in RM: Identify the risks

u

Interview key people, one-on-one?

Advantages
Key people have key information lots of it

Disadvantages
One-on-one interviews take too much time Conflicting opinions can be difficult to resolve quickly Lack of interaction among the key people leads to a lower quality of data

Copyright 2003 Decision Sciences Corp.

26

Steps in RM: Identify the risks

u

Hold a brainstorming session?

Advantages
The group is an excellent filter of historical information Brainstorming is faster than one-on-one interviewing It is a great forum for resolving conflicting opinions The group interaction produces a higher quality of data

Disadvantages
It may be difficult to schedule at a suitable time for all It will not surface many important facets of risk if it is not conducted by a risk expert

Copyright 2003 Decision Sciences Corp.

27

Steps in RM: Identify the risks

u

Hold a risk oriented brainstorming session facilitated by a risk expert?

Advantages versus traditional brainstorming
Risk oriented brainstorming facilitated by a risk expert bestows all of the benefits of traditional brainstorming, plus It maximizes the amount and quality of vital risk data that will be uncovered

Disadvantages versus traditional brainstorming

None

Copyright 2003 Decision Sciences Corp.

28

Steps in RM: Analyze the risk

u

Many tools exist for analyzing risk in PM. However, there is but one tool deserving of first place on that list, simply because it provides for a comprehensive analysis of the fourth variable in management decision making: probability.

Copyright 2003 Decision Sciences Corp.

29

Steps in RM: Analyze the risk

Monte Carlo Simulation (MCS)
Stanislaw M. Ulam was a brilliant mathematician who made contributions in such varied fields as physics, biology, and astronomy. Many scholars claim that Monte Carlo Simulation (MCS), a way to solve vexing problems by electronic random sampling, was his greatest achievement. Yet he is mostly remembered for his key role in developing the hydrogen bomb. The idea for MCS came to him in 1945 when playing solitaire while recuperating from a nearly fatal case of viral encephalitis. He recognized that a computer could deal numerous hands and thus calculate the probability of any given hand. He also realized this method could answer a difficult question in neutron diffusion. He related this to John von Neumann, the codeveloper of game theory, and the rest is history. The term Monte Carlo was coined by an associate of Ulams, Nicholas Metropolis.
Photo from Stanislaw Ulam Papers American Philosophical Society

Adventures of a Mathematician is Ulams highly readable autobiography. He died in 1984 at age 75.
30

Copyright 2003 Decision Sciences Corp.

Steps in RM: Mitigate the risk

u

Analyzing risk is useless if it does not lead to an action plan that enhances the chance of success in decision making. Development of a good action plan is frequently characterized by creativity and synergy (group think) that results in a list of specific actions which will:
Protect opportunities Remove or reduce threats

Copyright 2003 Decision Sciences Corp.

31

Steps in RM: Control the risk

u

The final step, controlling risk, is simply a dedication to do the obvious: repeat Steps 1-3 throughout the life cycle of the given decision process. Two tools can play a vital role in that regard:
Benchmarking the RM process itself Grading the risk analysis data streams

Copyright 2003 Decision Sciences Corp.

32

On becoming a risk analyst

u

Explosive growth in the application of Risk Management has led to a shortage of qualified risk analysts To become even a minimally qualified risk analyst requires taking the following steps or assuring yourself you have already done so

Copyright 2003 Decision Sciences Corp.

33

On becoming a risk analyst

u

Sharpen your people skills as much as possible u Acquire an understanding of modern management science methods and tools
Decision Analysis Influence Diagrams Decision Trees Monte Carlo Simulation Stochastic Scheduling Discrete Event Simulation
34

Copyright 2003 Decision Sciences Corp.

On becoming a risk analyst

u

Seek out a mentor to work with you, if only occasionally or for short periods of time For the first ten or so analyses, choose fairly simple problems, not those having high importance and visibility like an M&A in which the CEO has keen interest Conduct post mortems on your mistakes Perform as many analyses as possible
35

u u

Copyright 2003 Decision Sciences Corp.

What we have covered

u

A peek into the real world of RM

the Good, the Bad, the Truly Ugly

The steps in RM
Identify Analyze Mitigate Control the Risk

On becoming a risk analyst

36

Copyright 2003 Decision Sciences Corp.

Da, da, da Thats all folks!

Copyright 2003 Decision Sciences Corp.

37