Quantifying, Not Assuming: Making Sense of Intangibles, Uncertainties, and Risks

Quan%fying
-‐ Not Assuming

Making Sense of Intangibles, Uncertain%es and Risks
Crea%ng Value in IT Design, User Adop%on, Risk Management

Business Performance & Return on Investment
Gerald A. Bush, Ph.D.

Robert D. Brown III
What OHen Happens in Business?
  The largest risks and most uncertain issues get the least quan%ta%ve analysis
  “It can’t be measured, anything could happen.”
  “The %me and cost of the analysis would not pay for itself.”
  Time and effort spent on improving the precision of available data, while risks
are treated using broad, ambiguous assump%ons
  Risk scores can be ambiguous and misleading, but using them creates a
placebo effect for decision makers
The result is that:

  None of the real issues get quan%fied (leading to real risk)
  Analysis paralysis occurs when issues remain unresolved
Robert D. Brown III and Gerald A. Bush Copyright © 2010 All Rights Reserved 2
An Epiphany for Decision Makers
  Intangibles can be measured – despite the objec%ons
  “How much is it worth to assure that SLAs are met?”
  “What is the risk vs. savings of moving into the cloud?”
  A first pass analysis can tell you the value of informa%on, i.e. the value of
learning more about an uncertainty or risk.
  Over 90% of the variables have $0 value of informa%on, you are
was%ng your %me over-‐analyzing them.
  But, a few can make a difference with huge value, and focus the
analysis where the %me and energy are worth it.
  The goal is to make the best choice among your op%ons, not to
aaempt to prove the future.
An Epiphany is Worth It
  By quan%fying the Value of Informa%on:
  An airline saved over $360 MM/year by understanding fuel use and local
pricing, instead of doing just back office automa%on.
  A soHware startup created $ Billions in value by understanding the whole
market instead of selling a “work for hire” to one client.
  An oil company purchased expensive seismic studies and shiHed their

drilling site to a huge producer, avoiding many dry holes.
  A pharmaceu%cal company did the cri%cal studies first, doubling its R&D
efficiency and reducing new drug development %me.
  A chemicals company spent 3 months studying their compe%%on,
selec%ng the more difficult strategy that made them into a leader.
How Can Intangibles be Quan%fied?
Intangibles can be measured – despite objec%ons
  Airline problem
  Tangible benefit of fuel system ini%ally focused on back office automa%on and
reduc%on in payables errors.
  Intangible value of new informa%on use for decisions was not allowed, since it
could not be proven with exis%ng data.
  Airline epiphany
  Total cost of back office opera%on -‐ $720k per year did not pay for the $4.8 MM
development cost of new system
  Cost of the pricing and route planning tools plus fuel weight carrying costs was
$9.2 MM, nearly doubling the basic project.
  Value of “tankering” to selec%vely purchase fuel in lower cost airports suggested
a range of $84MM to $420MM per year.
  High uncertainty, but with more than enough value of informa%on to pay for a
study of the market and route impact.
  Result – an en%rely different design and $360 MM/yr savings
Case Study – ERP System
Major oil company approved a capital spending
project of $100 million for a new ERP system.
The business case value of the world-‐wide project

was assessed to be a whopping…
What went wrong?
  The team solved the wrong problem precisely, framing the problem
primarily in terms of cost reduc%on, not system performance and the
company’s strategic objec%ves
  The team performed mul%ple sensi%vity analyses on inconsequen%al

variables, effec%vely hiding the real opportunity for value improvement
and system success
  The team did not consider the underlying connec%ons between key
implementa%on decisions and the overall system performance and user
adop%on
Much hand wringing ensued…
How can we jus%fy the spend?

What should we do?
Maybe we should cut

capital costs!
Would this push the project

NPV back into the black?
Quan%fying a few ranges of uncertainty
showed that this was going the wrong way
simple business
case value = $0 $80 = informed business case value
NPV $million
Reducing the capital investment User adop>on
would nega%vely impact user System reliability
adop%on, system reliability and data
integrity – killing the value. Data integrity
The range of CapEx spending was

not a major driver of value by itself, CapEx
since it would be a one-‐%me cost.
Analysis of alterna%ve project strategies revealed a hybrid
strategy that mi%gated the cri%cal risks and found new value
1
0.9
Cumula%ve Probability
0.8
0.7
Original strategy Hybrid strategy
0.6
0.5
0.4
0.3
0.2
0.1
0
$0 $80 $380
NPV, $million
Some underlying behaviors and tradi%ons can obstruct
the crea%on of value in technical organiza%ons
  Rather than quan%fying the high value (oHen intangible) business factors,
risk assump%ons, technical arguments or cost subs%tu%on become the
basis for investment decisions
  Nego%a%ons or power-‐plays are used to select ventures rather than
focusing on expected value or fit with the business strategy
  Bad ventures frequently persist for too long, was%ng %me and resources
that could assure the value is achieved in good projects
The key to understanding and ac%ng on these issues is…
quan&fying the value of the intangibles
Management of complex technical ini%a%ves requires a
collabora%ve method for making high quality decisions
  Frame: Focused on the right goals and
issues for stakeholders
  Informa>on: Understanding the range of

possible outcomes (quan%fying the few
issues that impact the value of the
decision)
  Alterna>ves: Crea%ng and structuring a

wide array of contras%ng alterna%ves
  Values: Evalua%ng within the context of

stakeholder interests, using appropriate
metrics
Increasing complexity due to ambiguity and uncertainty
increases the need for a structured methodology
Unclear
Future
  Mul%ple stakeholders with
Increasing
complexity increasingly ambiguous goals
Uncertainty
  Mul%ple moving parts (external

& internal) with increasingly
uncertain outcomes
Clear
Future
Clear Goals Ambiguity Conflic%ng
Goals
Essen%al warran%es that decision-‐makers need:
Warranty of Clarity
• The terms are well defined
• The right problem has been iden%fied Avoids garbage in
Warranty of Coherence

• Reasoning is logical
• Preferences do not violate transi%vity
Avoids garbage in
• Evalua%ons cohere with the best knowledge of the day
between
Warranty of Thoroughly Considered

Consequences
• A crea%ve set of alterna%ves are considered Avoids garbage out
• Implica%ons of choosing are clear
• The best alterna%ve is revealed
Adapted from Howard, Ronald A. “Heathens, Here%cs, and Cults”. Interfaces 22:6. November-‐December 1992. pp. 15-‐27.
Each stage of the process resolves key ques%ons for
managers prior to making a financial commitment
Discovery Framing Evalua>on Agreement

 What is the real  What are the decision  What are the effects of  Which path best meets
situa%on? boundaries and open uncertain%es on our objec%ves?
decisions? solu%on goals?
 What is the  How much should we
opportunity?  What are the sources  What is the difference spend to control
of uncertainty? in value among the uncertain%es?
 What are our goals alterna%ves?
and objec%ves?  Can we develop an  What are the keys to
80% confidence range  How much risk do we implementa%on?
for the uncertain%es? face with each
alterna%ve?  What resources will
 Is value of informa%on we need for
worth the %me and  What insights can we implementa%on?
cost of analysis? create for con%ngency
plans or op%ons?  What con%ngent
ac%ons should we
take?
Focusing on the desired objec%ves is more produc%ve
than arguing the pros/cons of advocated ac%ons
Maximize
Primary stakeholder
Value
Capital Opera%onal Service

Fundamentals Costs Costs Execu%on
Excellence
System System System Data Service Appropriate

Means Arch.
Hardware SoHware
Overhead Complexity Integrity Reliability Tools
  Reveal and priori%ze ambiguous or conflic%ng goals and sources of poten%al trade-‐offs
  Agree on the clearly defined scope of decision
  Iden%fy the objec%ve basis for making a decision and the metric by which compe%ng
alterna%ves are compared
Establishing boundaries clarifies the important choices
to be made now as the focus for the analysis work
  Generate a wide array of crea%ve,

Policy (givens, boundaries) doable alterna%ves that are
Decisions already made, foregone consistent with objec%ves
  Create alterna%ves as a coherent
set of ac%ons, usually one op%on
Strategic (focus, open op&ons) from each decision category
Team Focus
for Analysis Decisions categories with sub-‐op%ons
  Alterna%ves should represent
to be considered now
strategically different themes
versus the full combina%on of all
Tac&cal (execu&on dependent) op%ons
Decisions to be made later, deferred   Sets the stage to consider
opportunity costs thoroughly
Clarifying sources of uncertainty and their interac%ons
helps us explore quan%ta%ve effects of decisions
  Captures the essence of the problem

and facilitates the dialog between
the team members
  Becomes a well defined model of the

situa%on, containing all the
necessary and relevant informa%on
needed to assess the situa%on
  Used as a means to communicate

the shared knowledge of the team to
the organiza%on
Quan%fying uncertain%es with an 80% confidence
interval helps us avoid the blind spots of assump%ons
We recognize assump%ons We observe results

are really uncertain%es as distribu%ons
Uncertainty
Uncertainty
Probability
Simulation Objective
Uncertainty
Engine!
Uncertainty
$0 $ NPV
Uncertainty
The right type of expert can explore the causes of
uncertainty and avoid unpleasant surprises
•  Tradi%onal Analysis (Forecaster)

–  Understand the assump%ons around a
forecast value
–  Focus on what has to go right (wrong) for
success (failure)
–  Assess a risk factor for the forecast
•  Uncertainty Analysis (Desired Expert)

–  Explore the factors that create extreme
outcomes or contribute to uncertainty
–  Understand what could go wrong and what
the upside and downside may be
–  Develop a range of possible outcomes and the
shape of the probability distribu%on
Monte Carlo simula%on reveals the full range of
varia%on of value given the ranges of uncertain%es
1
A B C   Demonstrates the aggregate
0.9
effect of all the uncertain%es
0.8
0.7 by both their impact &

0.6 prevalence
0.5
0.4
  Shows the probability of
0.3 important intervals and the
0.2 opportunity for op%miza%on
0.1 & tradeoffs
0
  Shows if one decision
Value Measure of Primary Objec>ve dominates another
Sensi%vity analysis of uncertain%es enables quan%fying
the value of informa%on for risk management
Value Measure of Primary Objec>ve
0
Effect of Uncertaintyi on Value Measure
Cri>cal
Value
Drivers
Minor
Value
Drivers
Two tools to deal with uncertain%es and risks can
create tremendous value in a decision
Value of Informa&on (the clairvoyant) quan%fies

the trade-‐offs of gathering more informa%on to
insure that you will not regret commitng to a
given decision; i.e., the price of choosing clearly.
Value of Control (the wizard) quan%fies the max

value for taking certain ac%ons to control your
outcome; i.e., the price of conver%ng an
uncertainty into a decision.
Value of Control leads to the formula%on of hybrid strategies
that capture the best elements of compe%ng alterna%ves
Hybrid
1
0.9 B C   Captures risk limita%ons
0.8 of Alterna%ve B
0.7
0.6   Captures value poten%al
0.5 of Alterna%ve C
0.4
0.3
0.2
0.1
0
Value Measure of Primary Objec>ve
Using an objec%ve decision process provides your
organiza%on with numerous qualita%ve benefits
Decisions will be…
•  Tenable
–  Resolves ambiguous and/or conflic%ng values
–  Clearly communicates the purpose and aims of the
decision
–  Aligns with fiduciary requirements including duty of care
•  Transparent
–  Based on collabora%ve exper%se and ra%onale
–  Aligns personal & organiza%onal agendas
–  Clearly iden%fies the implica%ons of uncertainty,
tradeoffs, complexity, and risk
•  Transferable
–  Does not rely on one individual's special insight
–  Promotes systemic thinking and comprehension
–  Builds an environment for effec%ve and ethical
collabora%on
A London Business School study found objec%ve
evidence of the value of this approach
Set Direc>on Iden>fy Op>ons Do Evalua>on Implement Ac>on
100%
% Successful A_er 2 Years
Key findings:
Improvement
Dialog (internal experts) or Pilot

Crea>ve Synthesis of Op>ons
Collabora>ve Par>cipa>on
in understanding,
par%cipant buy-‐in,
50%
use of crea%ve ideas
Framing
and achievement of

Data and business results.
Idea Simula>on Edict or
Jus>fica>on Single
Persuasion
Op>on
0%
Frequency of Use (bar width) of each Method

Out of the 127 cases studied in North America
(by Paul Nu\, Ph.D., Ohio State University)
Robert D. Brown III and Gerald A. Bush Copyright © 2010 All Rights Reserved
Page 26
Ques%ons?
Robert D. Brown III
Tel: (678) 947-‐5997
Email: rdbrown@incitedecisiontech.com
Gerald A. Bush, Ph.D.

Tel: (678) 641-‐6254
Email: gabush@mac.com
Web: www. incitedecisiontech.com
Robert D. Brown III and Gerald A. Bush Copyright © 2010 All Rights Reserved
Page 27

Quantifying, Not Assuming: Making Sense of Intangibles, Uncertainties, and Risks

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Quantifying, Not Assuming: Making Sense of Intangibles, Uncertainties, and Risks

Transféré par

Droits d'auteur :

Formats disponibles

Quan%fying

-­‐ Not Assuming

Crea%ng Value in IT Design, User Adop%on, Risk Management

Gerald A. Bush, Ph.D.

The result is that:

 An oil company purchased expensive seismic studies and shiHed their

The business case value of the world-­‐wide project

 The team performed mul%ple sensi%vity analyses on inconsequen%al

How can we jus%fy the spend?

Maybe we should cut

Would this push the project

The range of CapEx spending was

 Informa>on: Understanding the range of

 Alterna>ves: Crea%ng and structuring a

 Values: Evalua%ng within the context of

 Mul%ple moving parts (external

Warranty of Coherence

Warranty of Thoroughly Considered

Discovery Framing Evalua>on Agreement

Capital Opera%onal Service

System System System Data Service Appropriate

 Generate a wide array of crea%ve,

 Captures the essence of the problem

 Becomes a well deﬁned model of the

 Used as a means to communicate

We recognize assump%ons We observe results

• Tradi%onal Analysis (Forecaster)

• Uncertainty Analysis (Desired Expert)

0.7 by both their impact &

Eﬀect of Uncertaintyi on Value Measure

Value of Informa&on (the clairvoyant) quan%ﬁes

Value of Control (the wizard) quan%ﬁes the max

Set Direc>on Iden>fy Op>ons Do Evalua>on Implement Ac>on

Dialog (internal experts) or Pilot

and achievement of

Frequency of Use (bar width) of each Method

Gerald A. Bush, Ph.D.

Web: www. incitedecisiontech.com

Vous aimerez peut-être aussi

-‐ Not Assuming

  An oil company purchased expensive seismic studies and shiHed their

The business case value of the world-‐wide project

  The team performed mul%ple sensi%vity analyses on inconsequen%al

  Informa>on: Understanding the range of

  Alterna>ves: Crea%ng and structuring a

  Values: Evalua%ng within the context of

  Mul%ple moving parts (external

  Generate a wide array of crea%ve,

  Captures the essence of the problem

  Becomes a well deﬁned model of the

  Used as a means to communicate

•  Tradi%onal Analysis (Forecaster)

•  Uncertainty Analysis (Desired Expert)