Vous êtes sur la page 1sur 42

Monte Carlo Analysis

David M. Hassenzahl

Copyright 2004 David M. Hassenzahl

Purpose of lecture
Introduce Monte Carlo Analysis as a tool for managing uncertainty Demonstrate how it can be used in the policy setting Discuss its uses and shortcomings, and how they are relevant to policy making processes
Copyright 2004 David M. Hassenzahl

What is Monte Carlo Analysis?


It is a tool for combining distributions, and thereby propagating more than just summary statistics It uses random number generation, rather than analytic calculations It is increasingly popular due to high speed personal computers
Copyright 2004 David M. Hassenzahl

Background/History
Monte Carlo from the gambling town of the same name (no surprise) First applied in 1947 to model diffusion of neutrons through fissile materials Limited use because time consuming Much more common since late 80s Too easy now? Nameis EPA gambling with peoples lives (anecdotal, but reasonable).
Copyright 2004 David M. Hassenzahl

Why Perform Monte Carlo Analysis?


Combining distributions With more than two distributions, solving analytically is very difficult Simple calculations lose information
Mean mean = mean 95% %ile 95%ile 95%ile! Gets worse with 3 or more distributions
Copyright 2004 David M. Hassenzahl

Monte Carlo Analysis


Takes an equation
example: Risk = probability consequence

Instead of simple numbers, draws randomly from defined distributions Multiplies the two, stores the answer Repeats this over and over and over Then the set of results is displayed as a new, combined distribution
Copyright 2004 David M. Hassenzahl

Simple (hypothetical) example


Skin cream additive is an irritant Many samples of cream provide information on concentration:
mean 0.02 mg chemical standard dev. 0.005 mg chemical

Two tests show probability of irritation given application


low freq of effect per mg exposure = 5/100/mg high freq of effect per mg exposure = 10/100/mg

Copyright 2004 David M. Hassenzahl

Analytical results
Risk = exposure potency
Mean risk = 0.02 mg 0.075 / mg = 0.0015
or 15 out of 10,000 applications will result in irritation

Copyright 2004 David M. Hassenzahl

Analytical results
Conservative estimate
Use upper 95th %ile Risk = 0.03 mg 0.0975 / mg = 0.0029

Copyright 2004 David M. Hassenzahl

Monte Carlo: Visual example

0.01

0.02

0.03

0.05

0.10

Exposure (mg chemical)

Potency (probability of irritation per mg chemical)

Exposure = normal(mean 0.02 mg, s.d. = 0.005 mg) potency = uniform (range 0.05 / mg to 0.10 / mg)
Copyright 2004 David M. Hassenzahl

Random draw one


0.0165 0.063

0.01

0.02

0.03

0.05

0.10

Exposure (mg chemical)

Potency (probability of irritation per mg chemical)

p(irritate) = 0.0165 mg 0.063/mg = 0.0010

Copyright 2004 David M. Hassenzahl

Random draw two


0.0175 0.089

0.01

0.02

0.03

0.05

0.10

Exposure (mg chemical)

Potency (probability of irritation per mg chemical)

p(irritate) = 0.0175 mg 0.089 /mg = 0.0016

Summary: {0.0010, 0.0016}


Copyright 2004 David M. Hassenzahl

Random draw three


0.0152 0.057

0.01

0.02

0.03

0.05

0.10

Exposure (mg chemical)

Potency (probability of irritation per mg chemical)

p(irritate) = 0.152 mg 0.057 /mg = 0.0087

Summary: {0.0010, 0.0016, 0.00087}


Copyright 2004 David M. Hassenzahl

Random draw four


0.0238 0.085

0.01

0.02

0.03

0.05

0.10

Exposure (mg chemical)

Potency (probability of irritation per mg chemical)

p(irritate) = 0.0238 mg 0.085 /mg = 0.0020

Summary: {0.0010, 0.0016, 0.00087, 0.0020}


Copyright 2004 David M. Hassenzahl

After ten random draws


Summary {0.0010, 0.0016, 0.00087, 0.0020, 0.0011, 0.0018, 0.0024, 0.0016, 0.0015, 0.00062}
mean 0.0014 standard deviation (0.00055)
Copyright 2004 David M. Hassenzahl

Using software
Could write this program using a random number generator But, several software packages out there. I use Crystal Ball
user friendly customizable r.n.g. good up to about 10,000 iterations
Copyright 2004 David M. Hassenzahl

100 iterations (about two seconds)


Monte Carlo results
Mean Standard Deviation Conservative estimate Mean standard deviation Conservative estimate 0.0016 0.00048 0.0026 0.0015 n/a 0.0029

Compare to analytical results

Copyright 2004 David M. Hassenzahl

Summary chart - 100 trials


Forecast: P(Irritation) 100 Trials
.050

FrequencyChart
5

1 Outlier

.038

3.75

.025

2.5

.013

1.25

.000 0.00 0.00103 0.00 0.00 0.00161 0.00 0.00 0.00311

Copyright 2004 David M. Hassenzahl

Summary - 10,000 trials


Monte Carlo results
Mean Standard Deviation Conservative estimate Mean standard deviation Conservative estimate 0.0015 0.000472 0.0024 0.0015 n/a 0.0029

Compare to analytical results

Copyright 2004 David M. Hassenzahl

Summary chart - 10,000 trials


Forecast: P(Irritation) 10,000 Trials
.023

FrequencyChart

88 Outliers
226

.017

169.5

.011

113

.006

56.5

.000 0.00 0.00069 0.00 0.00 0.00150 0.00 0.00 0.00331

About 1.5 minutes run time


Copyright 2004 David M. Hassenzahl

Policy applications
When there are many distributional inputs Concern about excessive conservatism
multiplying 95th percentiles multiple exposures

Because we can Bayesian calculations


Copyright 2004 David M. Hassenzahl

Issues: Sensitivity Analysis


Sensitivity analysis looks at which input distributions have the greatest effect on the eventual distribution Helps to understand which parameters can both be influenced by policy and reduce risks Helps understand when better data can be most valuable (information isnt freenor even cheap)
Copyright 2004 David M. Hassenzahl

Issues: Correlation
Two distributions are correlated when a change in one causes a change in another Example: People who eat lots of peas may eat less broccoli (or may eat more) Usually doesnt have much effect unless significant correlation (||>0.75)
Copyright 2004 David M. Hassenzahl

Generating Distributions
Invalid distributions create invalid results, which leads to inappropriate policies Two options
empirical theoretical

Copyright 2004 David M. Hassenzahl

Empirical Distributions
Most appropriate when developed for the issue at hand. Example: local fish consumption
survey individuals or otherwise estimate data from individuals elsewhere may be very misleading

A number of very large data sets have been developed and published
Copyright 2004 David M. Hassenzahl

Empirical Distributions
Challenge: when theres very little data Example of two data points
uniform distribution? triangular distribution? not a hypothetical issueis an ongoing debate in the literature

Key is to state clearly your assumptions Better yetdo it both ways!


Copyright 2004 David M. Hassenzahl

Which Distribution?

0.05

0.10

0.05

0.10

Potency (probability of irritation per mg chemical)

Potency (probability of irritation per mg chemical)

0.05

0.10

0.05

0.10

Potency (probability of irritation per mg chemical)

Potency (probability of irritation per mg chemical)

Copyright 2004 David M. Hassenzahl

Random number generation


Shouldnt be an issue@Risk and Crystal Ball are both good to at least 10,000 iterations 10,000 iterations is typically enough, even with many input distributions

Copyright 2004 David M. Hassenzahl

Theoretical Distributions
Appropriate when theres some mechanistic or probabilistic basis Example: small sample (say 50 test animals) establishes a binomial distribution Lognormal distributions show up often in nature
Copyright 2004 David M. Hassenzahl

Some Caveats
Beware believing that youve really understood uncertainty Beware: misapplication
ignorance at best fraudulent at worstporcine hoof blister

Copyright 2004 David M. Hassenzahl

Example (after Finkel)


Alar versus aflatoxin
Exposure has two elements
Peanut butter consumption aflatoxin residue Juice consumption Alar/UDMH residue

Potency has one element


aflatoxin potency UDMH potency

Risk = (consumption residue potency)/body weight


Copyright 2004 David M. Hassenzahl

Inputs for Alar & aflatoxin


Variable Units Mean 5th %ile 95th %ile Percentile location of the mean. Peanut butter consumption Apple juice consumption aflatoxin residue UDMH residue aflatoxin potency UDMH potency g/g g/g kgday/mg kgday/mg
Copyright 2004 David M. Hassenzahl

g/day

11.38

2.00

31.86

66

g/day

136.84

16.02

430.02

69

2.82 13.75 17.5

1.00 0.5 4.02

6.50 42.00 28.23

61 67 61

0.49

0.00

0.85

43

Alar and aflatoxin point estimates


aflatoxin estimates:
Mean
11 .38 g 2.82 g 17 .5kg day mg 20 kg 1000 g day g mg

= 0.028 Conservative = 0.29

Alar (UDMH) estimates:


Mean = 0.046 Conservative = 0.77
Copyright 2004 David M. Hassenzahl

Alar and aflatoxin Monte Carlo


10,000 runs Generate distributions
(dont allow 0)

Dont expect correlation

Copyright 2004 David M. Hassenzahl

Aflatoxin and Alar Monte Carlo results (point values)


Aflatoxin
Mean Conservative Analytical 0.028 0.29 Monte Carlo 0.028 0.095

Alar
Mean Conservative Analytical 0.046 0.77 Monte Carlo 0.046 0.18

Copyright 2004 David M. Hassenzahl

Aflatoxin and Alar Monte Carlo results (distributions)


Forecast: peanut butter risk 10,000 Trials
.016

FrequencyChart

192 Outliers
163

.012

122.2

.008

81.5

.004

40.75

.000 0 0.0375 0.075 Certainty is 98.05% from -Infinity to 0.1495 0.1125 0.15

Copyright 2004 David M. Hassenzahl

Aflatoxin and Alar Monte Carlo results (distributions)


Forecast: apple juice risk 10,000 Trials
.102

Frequency Chart

125 Outliers
1020

.077

765

.051

510

.026

255

.000 0 0.1125 0.225 0.3375 0.45

Certainty is 93.93% from -Infinity to 0.15

Copyright 2004 David M. Hassenzahl

Aflatoxin and Alar Monte Carlo results (distributions)


Forecast: peanut butter risk 10,000 Trials
1.000

Cumulative Chart

192 Outliers
10000

.750

.500

.250

.000 0 0.0375 0.075 0.1125 0.15

Certainty is 98.04% from -Infinity to 0.1495

Copyright 2004 David M. Hassenzahl

Aflatoxin and Alar Monte Carlo results (distributions)


Forecast: apple juice risk 10,000 Trials
1.000

Cumulative Chart

125 Outliers
10000

.750

.500

.250

.000 0 0.1125 0.225 0.3375 0.45

Certainty is 93.93% from -Infinity to 0.15

Copyright 2004 David M. Hassenzahl

Aflatoxin and Alar Monte Carlo results (distributions)


OverlayChart Frequencydistribution--comparison
.102

.077

peanut butter risk

.051

.026 apple juice risk .000 0 0.1125 0.225 0.3375 0.45

Copyright 2004 David M. Hassenzahl

Aflatoxin and Alar Monte Carlo results (distributions)


OverlayChart Cumulative distribution--comparison
1.000

.750

peanut butter risk

.500

.250 apple juice risk .000 0 0.1125 0.225 0.3375 0.45

Copyright 2004 David M. Hassenzahl

References and Further Reading


Burmaster, D.E and Anderson, P.D. (1994). Principles of good practice for the use of Monte Carlo techniques in human health and ecological risk assessments. Risk Analysis 14(4):447-81 Finkel, A (1995). Towards less misleading comparisons of uncertain risks: the example of aflatoxin and Alar. Environmental Health Perspectives 103(4):376-85. Kammen, D.M and Hassenzahl D.M. (1999). Should We Risk It? Exploring Environmental, Health and Technological Problem Solving. Princeton University Press, Princeton, NJ. Thompson, K. M., D. E. Burmaster, et al. (1992). "Monte Carlo techniques for uncertainty analysis in public health risk assessments." Risk Analysis 12(1): 53-63. Vose, David (1997) Monte Carlo Risk Analysis Modeling in Molak, Ed., Fundamentals of Risk Analysis and Risk Management.
Copyright 2004 David M. Hassenzahl