Académique Documents
Professionnel Documents
Culture Documents
Day 1
Module 5
How Many Samples do I Need?
Part 2
Presenter: Sebastian Tindall
60 minutes
(15 minute 2nd Afternoon Break) 1 of 40
Summary
Use Classical Statistical sampling approach:
Very likely to fail to get representative data in most cases
MASSIVE
Bayesian
Geo-statistics
DATA Required
Kriging
[ AL] 3
then almost never fail
(Run Simulations)
3 of 40
Uncertainty is Additive!
Remember the uncertainty is additive for
all steps in sampling and analysis
Analytical
+
Sampling & Sub-sampling
+
Natural heterogeneity of the site
=
Total Uncertainty
4 of 40
What is the one phenomenon that
causes
ALL sampling error?
HETEROGENEITY
5 of 40
The SYSTEM functions as if it
believes that
Prescriptive
Decision
Analytical
Methods = Data
Quality = Quality
{
{
{Analytical
Uncertainty
Automatically
Managed
Data
Uncertainty
Automatically
Managed
Decision
Uncertainty
Automatically
Managed
6 of 40
Take-Home Message
Perfect Non-
Analytical + Representative
Chemistry Sample
BAD DATA
7 of 40
Representativeness
8 of 40
Representativeness
Soil Soil
Sample Sample
Representative Non-Representative
Soil Sample Soil Sample
9 of 40
Sample vs. Analytical Certainty
Analytical = 5%
5 4
24,000 Onsite 27,800 Onsite
27,700 Lab 42,800 Lab
11 of 40
Typical Sampling Design
EPA Methods for
Evaluating the
Attainment of Soil
Cleanup Standards -
n
2
Z 1 Z1 2
0.5 Z1 2
AL 1 2
Vol 1, 1989
Equation 6.6
Estimate of
usually way off
or
unknown
Wrong
Assumed Normal Distribution Often
12 of 40
Typical Sampling Design
(cont.)
n5 1. Will usually fail to truly
capture heterogeneity. of
population(s)
13 of 40
Uncertainty
Mo = Md = Mn Mo M d Mn
Normal Lognormal
Answer: GIGO; your 5 data points did not include data for heavy
rainfall years.
(ProUCL uses bootstrap techniques on small data sets. But remember,
Statistics cannot create information where there is none.)
17 of 40
Why Decisions are suspect
Failure to define population accurately
Failure to collect representative samples from the population
of interest
Failure to obtain representative data from the population of
interest
Failure to accurately determine the frequency distribution of
the COPCs
Failure to accurately determine the standard deviation of the
COPCs
Failure to select the appropriate statistical method for
generating adequate samples
Failure to use the appropriate UCL in making the decision
18 of 40
Definitions of Representativeness
A sample collected in such a manner that
the sampling error is less than a specified
amount.
A sample of a universe or whole that can be
expected to exhibit the average properties of
the universe or whole (40 CFR 260.10).
A sample that answers a question about a
population with a specified confidence
Sampling for Environmental Activities, Envirostat, 2003
19 of 40
Definitions of Representativeness
Representativeness expresses the degree to which sample
data accurately and precisely represents a characteristic of
a population, parameter variations at a sampling point, or
an environmental condition. Representativeness is a
qualitative parameter which is most concerned with the
proper design of the sampling program. The
representativeness criterion is best satisfied by making
certain that sampling locations are selected properly and a
sufficient number of samples are collected.
Representativeness is addressed by describing sampling
techniques and the rational used to select sampling
locations.
20 of 40
Definitions of Representativeness
A sample is representative when it is taken by
a selection method that is both accurate and
reproducible. Thus representativeness is
characterized by the absence of bias and an
acceptable variance. As far as the author is
aware, this is the only possible objective and
scientific definition of representativeness.
Sampling for Analytical Purpose, Pierre Gy, J. Wiley & Sons, 1998; pg 30
21 of 40
Definitions of Representativeness
A correct sampling method is always structurally accurate. In
addition, its variance is minimal so that its representativeness
is maximal.
Non-correct sampling is always structurally biased. It may be
accurate over short periods, but these cannot be forecast and
so are unusable. This makes the tests of accuracy
recommended by certain standards (the so-called bias tests)
not only useless but also dangerous as they offer a false sense
of security.
As well as having a negligible bias, representativeness
requires reproducibility, i.e. a minimum variance, which itself
depends on the quantitative properties of the sample (e.g. the
mass and the number of increments).
Sampling for Analytical Purpose, Pierre Gy, J. Wiley & Sons, 1998; pg 31
22 of 40
Definitions of Representativeness
A sample is representative when the mean square,
r2 SE , of Sampling Error (SE) is not larger than a
certain standard of representativeness regarded as
acceptable.
Representativeness is the sum of the square of the
mean of SE (mSE), and the variance of the SE
(s2SE).
Preparation of Soil Sampling Protocols: Sampling Techniques and Strategies, EPA/600/R-92/128, July 1992
23 of 40
Typical Values of Bias
Primary sample (non-probabilistic): up to
1000%
Secondary sample (probabilistic but
incorrect): up 50% (and probably much more)
Analysis: 0.1-1.0%
24 of 40
Concepts
Homogeneous: when all its units are strictly identical to each
other.
Homogeneity is an abstract mathematical concept that does not
exist in the real, material world.
Heterogeneous: when all the units are not identical to each other.
Heterogeneity is the only state in which a set of material units or
groups of units can be observed in practice.
Heterogeneity is seen as the sole source of all sampling errors
Homogeneity is the inaccessible condition of zero Heterogeneity
Sampling for Analytical Purpose, Pierre Gy, J. Wiley & Sons, 1998; pg 24-25
25 of 40
Quantity of Data Matters. Why?
26 of 40
Sample Size Rules of Thumb
27 of 40
Sample Size Rules of Thumb
In order to choose a specific classical statistical method (equation)
information regarding the distribution of the contaminant within the
decision unit is usually required.
If the tests fail, then the sampling results are in jeopardy and the
data maybe invalidated, which could lead to another round of
sampling. 28 of 40
Sample Size Rules of Thumb
(continued)
The sampling data is presented graphically (usually in the form of
a histogram) in order to assess the distribution of the contaminant.
29 of 40
Sample Size Rules of Thumb
30 of 40
Sample Size Rules of Thumb
31 of 40
Sample Size Rules of Thumb
32 of 40
Quantity of Data Matters. Why?
33 of 40
Lacking distribution information, it is
impossible to devise an optimal sampling
strategy.
-
Jenkins, et. al. 1996. Assessment of
Sampling Error Associated with Collection and Analysis of
Soil Samples at Explosives-Contaminated Sites U.S.
Army Corps of Engineers, Cold Regions Research &
Engineering Laboratory, p. 1.
http://www.crrel.usace.army.mil/techpub/CRREL_Reports/reports/SR96_15.pdf
34 of 40
Begin with the End in Mind
DATA Contaminant Concentrations in the Spatial
Distribution of the Population
The end 35 of 40
Q: Where do you obtain the contaminant distribution
information in order to select the correct sampling
design to ensure representativeness, etc?
36 of 40
Sample Representativeness
Are we honestly addressing Heterogeneity
(sampling uncertainty)?
Now we are finally able to address this issue,
defensibly and affordably!
Use cheaper analytical technologies that allow you to
increase sample density
Use real-time measurements at the site of the
sample to support real-time decision-making
IF we are willing to honestly balance analytical
uncertainty against overall data uncertainty
37 of 40
Managing Uncertainty
Systematic Dynamic
Planning Work Plan
Real-Time Measurement
Technologies
38 of 40
Managing Uncertainty
Systematic planning
Identify decision goals w/ tolerable overall
uncertainty
Identify major uncertainties (cause decision error)
Identify strategy to manage each major uncertainty
Use the Field Analytical Method (FAM) and a
Dynamic Work Plan (DWP) to effectively manage
sampling uncertainty (ensure sample
representativeness)
39 of 40
End of Module 5
Thank you
Questions?
40 of 40