Accred Qual Assur (2009) 14:95–106
DOI 10.1007/s00769-008-0475-6
PRACTITIONER’S REPORT
Implementation in MATLAB of the adaptive Monte Carlo method
for the evaluation of measurement uncertainties
M. Solaguren-Beascoa Fernández Æ
J. M. Alegre Calderón Æ P. M. Bravo Dı́ez
Received: 5 May 2008 / Accepted: 21 November 2008 / Published online: 19 December 2008

Springer-Verlag 2008
Abstract The ISO 98:1995 Guide to the expression of
uncertainty in measurement (GUM) presents important
application limitations. For its improvement, different
supplements are being developed that will progressively
enter into effect. The first of these supplements describes
an alternative method for calculating uncertainties, the
Monte Carlo method (MCM), which is not restricted to the
conditions of the method described in the GUM: the line-
arity of the model and the application of the central limit
theorem. MCM requires computer calculation systems for
generating pseudo-random numbers and for evaluating the
model a large number of times. There are software appli-
cations that have been specifically developed for
calculating uncertainties, some of which include MCM; but
they do not allow the user to control all factors in the
process, particularly the result stabilization criteria. On the
contrary, its implementation in a mathematical program for
general purposes such as MATLAB, enables total control
over the process, is simple and benefits from its calculation
speed. This article details programming in MATLAB for
the implementation of the adaptive MCM method.
Keywords Uncertainty
Measurement
Monte Carlo

MATLAB
Metrology
M. Solaguren-Beascoa Fernández (&)
J. M. Alegre Calderón

P. M. Bravo Dı́ez
Group of Structural Integrity, Superior Polytechnic School,
University of Burgos, Avda. de Cantabria s/n,
09005 Burgos, Spain
e-mail: msolaguren@ubu.es
Introduction
Metrological activities are fundamental for guaranteeing
the quality of scientific and industrial activities. The results
of such measurements must be valid, comparable, and
reproducible; and their uncertainty is the quantitative
measurement that expresses the quality of such results. In
accordance with the ISO/IEC 17025:1999 [1] standard, all
calibration or testing laboratories must have and apply
procedures to evaluate uncertainty in measurements as a
guarantee of their technical competence. In order to per-
form this evaluation, the ISO 98:1995 guide commonly
known as the ‘Guide to the expression of uncertainty in
measurement’ (GUM) [2] has been widely used and
accepted by the metrological accreditation organizations.
With its publication as an ISO standard, the method for
evaluating measurement uncertainties described in the
GUM structure—referred to as the GUM Uncertainty
Framework (GUF)—signified an important benchmark that
unified methods and was accepted with enthusiasm by the
international metrological community.
In 1997, the International Office of Weights and Mea-
sures (BIPM, Bureau International des Poids et Mesures)
created the Joint Committee for Guides in Metrology
(JCGM) with the mission of revising GUM in order to
improve its ease of use and expand its range of application,
since it displayed certain limitations. A variety of supple-
ments to GUM are being developed, which will
progressively enter into effect. In the first of these sup-
plements [3], an alternative procedure is described for the
calculation of uncertainties: the Monte Carlo Method
(MCM). This method is shown as a practical alternative to
GUF in cases in which the latter is not applicable or its
validity is not clear. The fundamental difference between
both methods lies in that GUF is a deterministic method
123
96
based on the mathematical combination of probability
distribution (by means of the law of propagation of
uncertainty and the central limit theorem), while the MCM
is a probabilistic method that performs the combination of
probability distributions by numerical simulation (through
calculation systems for generating random numbers).
GUF is based on the law of propagation of uncertainty
and on the characterization of the quantities measured by
means of either a normal (Gaussian) distribution or a
t-distribution, which allows measurement uncertainty to
be delimited by means of a confidence interval. Unfor-
tunately, to determine this confidence interval, GUF may
only be applied if two conditions are met: firstly, lineal
dependence must exist between the measurand and the
arguments of the measurement process; and secondly, in
most cases it must be justifiable the application of the
central limit theorem. In order to be able to apply the
method, if the mathematical model is not linear, it is
linearized based on the first order Taylor series approxi-
mation; however, not complying with either of the two
mentioned conditions may lead to an inadequate repre-
sentation and, therefore, to confidence intervals for
uncertainty that are hardly reliable [4].
MCM is an alternative to GUF. MCM performs a
characterization of the quantities measured based on the
random sampling of the probability distribution functions.
This numerical simulation tends to require around 106 trials
(model evaluations), which require a computer. Since the
domain of validity for MCM is broader than that for the
GUF, MCM can be applied in all those situations in which
the conditions for applying GUF are not met or are not
clear. An important aspect to bear in mind is that the MCM
can be used to determine whether all the conditions for
applying GUF hold.
There are software applications that are specifically
developed for calculating measurement uncertainties [5–7],
some of which use the MCM, such as WINCERT [8],
Evaluator [9], MUSE [10] or UncertaintyManager [11].
Other software applications meant for statistical systems
and risk analysis can also be used for uncertainty calcula-
tions, such as Minitab [12], LabVIEW [13], Analytica [14]
or Microsoft Excel @Risk [15] and Crystal Ball [16] add-
ins. In addition, it is easy to implement the MCM in general
purpose mathematical programs such as MATLAB [17],
MathCad [18], Mathematica [19], Maple [20], etc.
The examples developed with both the specific pro-
grams and those used for risk and system analysis [21–23]
have not been able to apply the MCM in an adaptive way;
simply choosing the number of iterations to perform
beforehand or, at the most, performing a preliminary study
to determine the adequate number of them [24]. If the
MCM is not adaptive, there is no direct control over the
quality of the results obtained. The implementation of the
123
Accred Qual Assur (2009) 14:95–106
MCM in a general purpose mathematical program is not
complex and allows control over all of its possibilities.
Similarly, some specific programs are beginning to
implement the adaptive MCM [25].
From among the commercial mathematical analysis
software applications, MATLAB is widely distributed
worldwide and is widely used in universities as well as by
all kinds of professionals. As of version 7.4, it uses the
Mersenne Twister algorithm [26] for generating pseudo-
random numbers, which has passed numerous randomness
tests, among them the strict Diehard test [27]. MATLAB
was the software used in developing the examples in sup-
plement 1 to the GUM. This article explains the basic
implementation of the adaptive Monte Carlo method in
MATLAB, which may be used in other similar programs
with each program’s variations.
GUF and MCM
A measurand, denoted by Y, is a quantity subject to mea-
surement. An estimate for Y, denoted by y, is a central
value of the distribution of values that could be reasonably
be attributed to Y. The uncertainty is a parameter associated
with the estimate y which characterizes the dispersion of
the values that could be attributed to Y based upon all
available information. The standard uncertainty is uncer-
tainty expressed as a standard deviation, denoted by u(y).
Methods used for evaluation of measurement uncertainties
are based on the concept of a measurement equation:
Y ¼ f ðX1 ; X2 ; . . .; XN Þ ð1Þ
The input quantities Xi are all those that have an
influence on the result of the measurement, including
correction factors as well as quantities that take into account
other variability sources. The output quantity is the
measurand Y. Given that its values are not known with
absolute certainty, all the quantities involved are considered
random variables. Each one of them is characterized by its
probability density function, the mathematical expectation
of which is the estimate of the quantity value and the
standard deviation of which is the standard uncertainty
related to the estimate.
The fundamental difference between GUF and MCM
lies in the way they characterize the output quantity. GUF
determines, from the standard uncertainties of the input
quantities, what will be the standard uncertainty for the
output quantity; and, to delimit measurement uncertainty
by means of a confidence interval, characterizes the output
quantity by means of either a normal (Gaussian) distribu-
tion or a t-distribution. MCM obtains, from the probability
distributions of the input quantities, an approximate
numerical representation of the probability distribution for
Accred Qual Assur (2009) 14:95–106
the output quantity. GUF propagates the standard uncer-
tainties and MCM propagates the probability distributions.
GUF performs the propagation of standard uncertainties
using the so called ‘‘law of propagation of uncertainty’’
which is, simply, the expression of the standard deviation
of a linear combination of random variables. The func-
tional relation between the output quantity Y and the input
quantities Xi must be linearized by approximating it
through the first order terms of the Taylor series in a
neighborhood of its estimates xi, denoted by a lower case
letter:
X
N
Y ¼ f ðX1 ; X2 ; . . .; XN Þ ’ yðx1 ; x2 ; . . .; xN Þ þ ci X i ð2Þ
i¼1
where the coefficients ci are known as sensitivity
coefficients:

output quantity is close to normal, but not completely since
of ðX1 ; X2 ; . . .; XN Þ


ci ¼
ð3Þ
oX i x1 ;x2 ;...;xN
Once linearized, it is possible to use the law of
propagation of uncertainty:
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u N
uX N 1 X
X N  
uð y Þ ¼ t c2i u2 ðxi Þ þ 2 ci cj u xi ; xj ð4Þ
i¼1 i¼1 j¼iþ1
In this expression, u(xi) are the standard uncertainties
related to the input estimates, and u(xi, xj) are the possible
covariances between them.
The input variables may be evaluated from statistical
methods or determined by other means. An input variable
is referred to as Type A or Type B depending on whether
its probability distribution or its parameters are specified by
statistical methods or by other means:
• In a Type A evaluation of an input quantity Xi, the
estimate xi and the uncertainty uA(xi) are determined
from sampling theory (the subscript A indicates that it is a
Type A standard uncertainty). Depending on the number
mi of independent measurements, the Type A standard
uncertainty is uncertain because of the statistical reason
of limited sampling. Following the ISO-GUM, the
statistical uncertainty concerning uA(xi) is quantified by
degrees of freedom. The degrees of freedom associated
with uA(xi) are mi = mi - 1.
• A Type B evaluation of an input quantity Xi is
commonly obtained by assigning a state-of-knowledge
probability distribution for Xi. Then, the estimate xi is
the expected value E(Xi) and the standard uncertainty
uB(xi) is the standard deviation S(Xi) of the assigned
distribution (the subscript B indicates that it is a Type B
standard uncertainty). It is common practice to carry
out a Type B evaluation in a manner that ensures that
any underestimation is avoided. Under the assumption
97
that this practice is followed, the degrees of freedom of
the standard uncertainty uB(xi) obtained from a Type B
evaluation may be taken to be mi ? ?.
If the probability distribution of the output quantity were
normal, the interval defined by its standard uncertainty u(y)
would have a level of confidence or coverage probability of
approximately 68%. Since, in general, this is required to be
95 or 99%, the uncertainty interval is usually expressed
through the expanded uncertainty, which is simply the
standard uncertainty multiplied by a coverage factor ky:
U ð y Þ ¼ k y uð y Þ ð5Þ
For normal distribution, a value ky = 2 corresponds to
an approximate coverage probability of 95%, and ky = 3 of
99%. In many practical cases, the conditions of the central
limit theorem are met and the probability distribution of the
its standard deviation has been estimated. Its probability
distribution is then a t-distribution, the effective degrees of
freedom meff of which may be calculated from the degrees
of freedom considered for the input estimates mi by means
of the Welch-Satterthwaite formula:
u4 ð y Þ
meff ¼ P ð6Þ
N c4i
u4 ðxi Þ
i¼1 mi
Knowing that the probability distribution is a t-
distribution with meff degrees of freedom, it is possible to
determine the coverage factor ky for the desired level of
confidence.
An alternative to this approximate t-distribution based
on the Welch-Satterthwaite formula is the use of the
Bayesian extension [27]. The assumptions that underlie
both methods are essentially identical. Nevertheless, the
use of the Bayesian extension greatly simplifies the
expression of uncertainty by eliminating altogether
the need for calculating effective degrees of freedom from
the Welch-Satterthwaite formula.
MCM carries out the propagation of probability distri-
butions sampling the probability distribution of the output
quantity Y. In this method, probability density functions
(PDF) are explicitly assigned to all input quantities Xi
based on information concerning these quantities. GUM
Supplement 1 [3] gives guidance on the assignment, in
some common circumstances, of PDFs to the input quan-
tities Xi. In order to construct a PDF for a quantity, on the
basis of a series of indications, Bayes’ theorem can be
applied. When appropriate information is available con-
cerning systematic effects, the principle of maximum
entropy can be used to assign a suitable PDF. The GUM
Supplement 1 does not distinguish between the Types A
and B state-of-knowledge probability distributions for the
input variables. Nevertheless, this classification is still
123
98
important [28] since the information the metrologist have
for quantities of Type A or Type B is very different (sta-
tistical analysis or other means, respectively). Each value yr
(r = 1,…, M) is obtained by performing a random sam-
pling of each of the probability density functions of the
input quantities Xi and evaluating the model with the values
found. The M values of Y thus obtained must be arranged in
a non-decreasing order, thus providing an approximate
numerical representation G of its distribution function
GY(g). This vector G can be used to obtain an estimate y of
Y, the standard uncertainty u(y) related to y, as well as an
appropriate coverage interval for Y, for a stipulated cov-
erage probability p. The effectiveness of MCM in obtaining
y, u(y) and a coverage interval for Y depend on the
appropriate choice of a sufficiently large M value.
The conditions for a valid application of MCM are less
restrictive than those of GUF. However, MCM is very
difficult to use in some cases, for example when the input
quantities are correlated. The use of both methods leads to
a better understanding of the measurement process.
The adaptive Monte Carlo method
A fundamental parameter in order to obtain reliable results
through MCM is the number M of trials or evaluations to
be performed by the model. If M is chosen beforehand,
there will not be direct control over the results. A value
M = 106 is often considered appropriate in order to pro-
vide a coverage interval of 95%; however, the random
nature of the process and the nature of the probability
distribution of the output quantity Y have an influence on
the value needed for M, which will vary in each case. For
this reason, the performance of the MCM must be carried
out in an adaptive way, progressively increasing the
number of iterations until the stabilization of the results.
The stabilization criterion for the results is established
in accordance with the aim of arriving at a given
numerical tolerance. The numerical tolerance d is defined
as the semi-width of the shortest interval containing all
numbers that can be expressed to a specified number of
significant decimal digits. For example, the numerical
tolerance associated with the numerical value 2.3 is
d = 0.05, as that number can express any value included
between 2.25 and 2.35. In supplement 1 to the GUM the
results stabilization criterion is the following: a numerical
result is deemed to have stabilized in a statistical sense
when twice the standard deviation associated with it is
less than the numerical tolerance associated with the
standard uncertainty u(y).
The calculation algorithm of the flexible MCM com-
prises the following steps:
123
Accred Qual Assur (2009) 14:95–106
(a) Choose the desired coverage probability p for the
interval to be obtained.
(b) Choose the number of significant decimal digits ndig for
the uncertainty u(y). It is normal to take a value of 1 or 2.
(c) Choose the number M of trials to perform in each
sequence of the application of the process. Standard
practice is to make:
  
M ¼ max 100 ð1  pÞ  104 : ð7Þ
These two values have been intentionally chosen to be
much smaller than the foreseeable total number of
iterations (of the order of 106) in order to be able to
analyze, after every sequence, the variability of the
statistics of the parameters to be determined. In addi-
tion, this value M is taken as a multiple of 1/1(1 - p)
in order to help in the determination of the shortest
coverage interval (104 is a multiple of 1/1(1 - p) for
the usual cases in which p = 0.95 and 0.99).
(d) The variable h counts the number of MC-simulations.
In order to carry out the first one, h = 1 is
established.
(e) For each h sequence, M trials or evaluations of the
model are carried out, which gives the values yr
(r = 1,…, M) and the following estimated parameters:
• Average is taken as an estimate y of Y:
1X M
yðhÞ ¼ yr ð8Þ
M r¼1
• Standard deviation is taken as the standard
uncertainty u(y) associated with y:
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
  u u 1 X M
u yðhÞ ¼ t ðy r  y Þ2 ð9Þ
M  1 r¼1
• Let q the integer part of pM ? 1/2. Sorting the
values yr (r = 1,…, M) in a non-decreasing order
y(r) (r = 1,…, M), the probabilistically symmetric
ðhÞ ðhÞ
coverage interval for Y will be [ylow ; yhigh ]. The
ðhÞ ðhÞ
interval extremes are ylow ¼ yðrÞ and yhigh ¼ yðrþqÞ ;
where r is the integer part of (M - q)/2 ? 1/2. If
the desired result is the shortest coverage interval,
the value r* should be determined such that
yðr þqÞ  yðr Þ  yðrþqÞ  yðrÞ for each and everyone
of the values r = 1,…, (M - q).
(f) In order to analyze the variability of the parameters,
more than one sequence is required, thus if h = 1, it
should be increased by one unit and return to step (e).
(g) After every sequence, the average and standard
deviation of these latter parameters must be
calculated:
Accred Qual Assur (2009) 14:95–106 99

• For the estimate: u(y) have already been calculated in steps (g) and (h),
respectively.
1X h
y^ ¼ y ¼ yðiÞ ð10Þ
h i¼1
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u Programming in MATLAB
u 1 X h
sy^ ¼ t
2
ðyðiÞ  y^Þ ð11Þ
hðh  1Þ i¼1 MATLAB is software developed for performing numerical
calculations with vectors and matrices. The programming
• For the standard uncertainty: language used constitutes a high level tool that is very fast
1X h   against other application development environments. A
u^ð yÞ ¼ u yðiÞ ð12Þ program in MATLAB, as in any programming language,
h i¼1
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi consists of a series of instructions that run sequentially,
u with the possibility of containing certain instructions that
u 1 X h
su^ðyÞ ¼ t
2
ðuðyðiÞ Þ  u^ð yÞÞ ð13Þ control those which will run in each circumstance. Creating
hðh  1Þ i¼1 a program in MATLAB is simple: just open a text docu-
ment (it can be done in any editor, if there is one built in),
• For the lowest extreme of the coverage interval:
write the instructions as they would be entered into the
1X h
ð iÞ command window and save the document with the exten-
y^low ¼ y ð14Þ sion.m. MATLAB automatically recognizes files with this
h i¼1 low
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi extension. In addition, it contains a set of tools called
u h  2 ‘‘user interface’’ that allow giving the programs made
u 1 X
¼t
ðiÞ
sy^low ylow  y^low ð15Þ with it a user-friendly appearance, similar to a Windows
hðh  1Þ i¼1
application.
Implementation of the Monte Carlo method in
• For the highest extreme of the coverage interval:
MATLAB, according to the algorithm described in the
1X h
ðiÞ previous section, requires the user to enter a series of
y^high ¼ y ð16Þ
h i¼1 high information: the functional relation of the measurement
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi process, the probability distributions of the input quantities,
u h  2
u 1 X their parameters, the required coverage probability, whe-
¼t
ð iÞ
sy^high yhigh  y^high ð17Þ ther the coverage interval being sought is probabilistically
hðh  1Þ i¼1
symmetrical or the shortest, and the number of significant
(h) In order to apply the stabilization criterion to the decimal digits for the standard uncertainty. User interaction
results, the numerical tolerance d related to the with the developed program can take place through ‘‘user
standard uncertainty u(y) must be calculated. The interface’’ instructions, which allows buttons, menus,
uncertainty u(y) is calculated as in step (e), but using all windows, to be created, etc.; however, in order not to
the values h 9 M of the model. The tolerance is simply excessively complicate the program and thus make it more
half of the last significant digit of the uncertainty. For understandable, they will not be used and information will
its computer calculation, the uncertainty must be be directly entered in the first lines of the program.
expressed in the form u(y) = c 9 10d, where c is an Table 1 details the program developed for the calcula-
ndig decimal digit integer and d an integer; then, the tion of uncertainty in the calibration of a mass. For any
related numerical tolerance is: other case, it is enough to modify the first lines, since they
are the ones that establish the variables that are charac-
1 teristic of each case:
d ¼ 10d ð18Þ
2 • In line 2, the coverage probability p is specified, the
(i) The stabilization criterion for the results establishes number of significant decimal digits for the standard
that if any of the values of 2sy^; 2su^ðyÞ ; 2sy^low or 2sy^high uncertainty n_digit, (normally 1 or 2) and a value called
are greater than d, h should be increased by one unit tol_divisor that divides the numerical tolerance. This
and step (e) should be repeated. value is entered in order to take into account recom-
(j) Once the stabilization criterion has been verified, all mendation [3], which states that when MCM is used to
the values h 9 M of the model should be used to validate GUF, a numerical tolerance five times less than
calculate ylow and yhigh in the same way that it was that related to the standard uncertainty is taken in the
done in step (e) for each sequence. The values of y and application of the stabilization criterion.

123
100 Accred Qual Assur (2009) 14:95–106

Table 1 Implementation program of the Monte Carlo method in MATLAB

Line Programme Flow Diagram

1 clear;
2 p=0.95; n_digit=1; tol_divisor=5; Establishment of the process parameters and the
3 interval=1; % (1 'shortest', other value 'symmetric') default values of the control variables
4 M=max((100/1-p),10000); comp=0; h=0;
5 while comp==0
6 h=h+1;
7 mass_rc=random('norm',100000,0.05,1,M);
8 dmass_rc=random('norm',1.234,0.02,1,M);
9 air=random('unif',1.1,1.3,1,M); Creation of M size row vectors for each of the input
10 air0=1.2; variables
11 dens_w=random('unif',7000,9000,1,M);
12 dens_r=random('unif',7950,8050,1,M);
13 nom_mass=100000;
14 y=(mass_rc+dmass_rc).*[1+(air-air0).*(1./dens_w-1./dens_r)]-nom_mass;
15 y=sort(y); Evaluation of the model. M size file vector
16 y_mean=mean(y);
17 y_std=std(y); Calculation of the average, standard deviation and
18 q=round(p*M); r=round((M-q)/2); symmetrical interval of the M value sequence
19 y_low=y(r); y_high=y(r+q);
20 if interval==1 NO
21 r=1; Shortest interval?
22 while r<=(M-q)
YES: interval=1
23 if y(r+q)-y(r)<=y_high-y_low;
24 y_low=y(r); y_high=y(r+q); Calculation of the shortest coverage interval of the
25 end M value sequence
26 r=r+1;
27 end
28 end NO
29 if h==1 First sequence?
30 Y=y; YES: h=1
31 Y_mean=y_mean; Y_std=y_std; Y_low=y_low; Y_high=y_high; Value matrices and parameter vectors are formed
32 else
33 Y=[Y;y];
34 Y_mean=[Y_mean;y_mean]; Y_std=[Y_std;y_std]; Add row to value matrices and parameter vectors
35 Y_low=[Y_low;y_low]; Y_high=[Y_high;y_high];
36 y_mean_std=std(Y_mean)/(h^0.5);
37 y_std_std=std(Y_std)/(h^0.5);
38 y_low_std=std(Y_low)/(h^0.5); Calculation of standard deviation of the parameters
39 y_high_std=std(Y_high)/(h^0.5);
40 y_values=reshape(Y,1,h*M);
41 y_standard=std(y_values), h*M Calculation of the total standard deviation
42 a=y_standard; b=0;
43 if a>=10^n_digit
44 while a>=10^n_digit
45 a=a/10; b=b+1;
46 end
47 tol=(0.5/tol_divisor)*10^b;
Calculation of the numerical tolerance related to the
48 elseif a<10^(n_digit-1)
standard deviation
49 while a<10^(n_digit-1)
50 a=a*10; b=b+1;
51 end
52 tol=(0.5/tol_divisor)*10^-b;
53 else tol=(0.5/tol_divisor);
54 end
55 if (2*y_mean_std<tol & 2*y_std_std<tol & 2*y_low_std<tol & 2*y_high_std<tol)
56 comp=1;
57 end NO
Stabilization?
58 end
59 end YES: comp=1
60 y_values=sort(y_values);
61 y_mean=mean(y_values); Calculation of the average and the symmetrical
62 q=round(p*h*M); r=round((h*M-q)/2); coverage interval of all the values
63 y_limit_low=y_values(r); y_limit_high=y_values(r+q);
64 if interval==1 NO
Shortest interval?
65 r=1;
66 while r<=(h*M-q)
YES: interval=1
67 if y_values(r+q)-y_values(r)<=y_limit_high-y_limit_low;
68 y_limit_low=y_values(r); y_limit_high=y_values(r+q); Calculation of the shortest coverage interval of all
69 end the values
70 r=r+1;
71 end
72 end
73 y_mean, y_standard, y_limit_low, y_limit_high
74 lower=min(y_values); upper=max(y_values); Shows results on screen and draws the histogram
75 xc=lower:(upper-lower)/99:upper; with 100 bars
76 n=hist(y_values,xc);
77 bar(xc,n./(((upper-lower)/99)*h*M),1);

123
Accred Qual Assur (2009) 14:95–106 101

• In line 3, the user specifies whether the coverage dm ¼ mW;c  mnom

interval to be calculated is probabilistically symmetri-  

    1 1
cal or the shortest. The interval parameter value ¼ mR;c þ dmR;c 1 þ qa  qa0 
qW qR
determines one case or another: 1 in case it is the  mnom ð23Þ
shortest and any other value in a probabilistically
symmetrical case. The information on each of the input quantities must be
• In lines 7–13, the probability distributions of the input used to assign a probability distribution to them:
quantities are specified, and in line 14, the functional • The available information on mR,c and dmR,c are usually
relation of the measurement process. the values of their estimates and their standard uncer-
tainty, obtained from their calibration certificates. A
Gaussian distribution can be assigned to these quanti-
Application example 1: mass calibration ties, whose average and standard deviation are the latter
values of the estimate and the standard uncertainty.
To demonstrate how the program works, its application is • If the only available information on qa, qW and qR are
illustrated with afore-mentioned example of the calculation their lowest and highest value limits, a rectangular
of uncertainty in the calibration of a mass. The results will distribution can be assigned to them with said limit
be compared with those given by GUF. This first example values as extremes of the distribution.
has been taken from supplement 1 to the GUM [3].
Consider the calibration of a weight W against another of Table 2 provides the probability distribution parameters
reference R, both with the same nominal mass mnom = assigned to the different input quantities. None of the input
100 g, using a balance. Since this operates in the air (with a quantities are considered to be correlated to any significant
mass density qa) and the mass densities of both weights, qW extent.
and qR, will be generally different, it is necessary to consider From this data, it is possible to use GUF as well as
the buoyancy effects. Applying the Archimedes’ principle, MCM to obtain an estimate dm ^ of dm, its related standard
the functional relation of the model is: uncertainty uðdm^ Þ, and the shortest 95% coverage interval.
  The GUF and the adaptive MCM were each used (a high
qa qa
mW 1  ¼ ðmR þ dmR Þ 1  ð19Þ number of significant figures have been taken for com-
qW qR
parison purposes only):
where dmR is the mass of the small weight that must be
• In order to carry out the GUF method is necessary to
added to R in order to balance W. Given that the weights calculate the partial derivatives for the model with
are generally used in the air, it is usual to work in terms of
respect to the input parameters. These derivatives,
conventional values. Conventional mass mW,c is defined as
evaluated at the best estimates of the input quantities,
the mass of a hypothetical weight of density are the sensitivity coefficients. The law of propagation
q0 = 8,000 kg/m3 that balances W in air with a density ^¼
of uncertainty (Table 3) gives the estimate dm
of qa0 ¼ 1:2 kg/m3 : Such that:
  1:2340 mg and associated standard uncertainty
qa0 qa0 uð dm
^ Þ ¼ 0:0539 mg. A probabilistically symmetric
mW 1  ¼ mW;c 1  ð20Þ
qW q0 (and shortest) 95% coverage interval for dm, based on
a coverage factor of 1.96 for the Gaussian probability
Thus, in terms of conventional masses, the model (21)
density function, is [1.1284, 1.3396]mg.
comes to:
  • Carrying out the adaptive MCM by running the
qa qa program in MATLAB, for n_digit = 1 it performed
1 1
qW   qR 0.72 9 106 evaluations of the model until the stabil-
mW;c  ¼ mR;c þ dmR;c  ð21Þ
qa0 qa0 ization of the results, in barely 6 s (on a Windows XP
1 1
qW qR Pro with Intel Pentium processor @ 1.86 GHz, 512 MB
of RAM). It gives the estimate dm ^ ¼ 1:2341 mg with
The previous expression can be approximated in most
associated standard uncertainty uðdm ^ Þ ¼ 0:0755 mg
practical cases by the following, more simple:

  and a shortest 95% coverage interval [1.0849,
    1 1 1.3839]mg.
mW;c ¼ mR;c þ dmR;c 1 þ qa  qa0 
qW qR
The results show that, although GUF and MCM give
ð22Þ
estimates of dm in good agreement, the numerical values
Finally, the deviation of mW,c regarding the nominal for the associated standard uncertainty are noticeably dif-
mass may be expressed as: ferent. The MATLAB program also generates a graphical

123
102 Accred Qual Assur (2009) 14:95–106

Table 2 Assigned input

Xi Distribution Parameters
quantities and probability
distributions for the mass Expectation l Standard Expectation Semi-width
calibration model (mg) deviation r (mg) x = (a ? b)/2 (kg/m3) (a - b)/2 (kg/m3)

mR,c N(l, r2) 100000.000 0.050

dmR,c N(l, r2) 1.234 0.020
qa R(a, b) 1.20 0.10
qW R(a, b) 8 9 103 1 9 103
3
qR R(a, b) 8.00 9 10 0.05 9 103

Table 3 Uncertainty budget in

Quantity Estimate Probability Standard Sensitivity Uncertainty
the calibration of a mass using
distribution uncertainty coefficient contribution
GUF method
Xi xi u(xi) ci ui(y) (mg)

mR,c (mg) 100000.000 Normal 0.050 1 0.050

dmR,c (mg) 1.234 Normal 0.020 1 0.020
3
pffiffiffi
qa (kg/m ) 1.20 Rectangular 1:2 3 0 0
pffiffiffi
qW (kg/m3) 8 9 103 Rectangular 1  103 3 0 0
pffiffiffi
qR (kg/m3) 8.00 9 103 Rectangular 0:05  103 3 0 0
mnom (mg) 100000 – – – –
dm (mg) 1.2340 0.0539

approximation in the form of a histogram of the probability

density function of the output quantity, represented in
Fig. 1, onto which the curve of a Gaussian distribution of
parameters obtained from GUF has been superimposed
(continuous curve named GUF1). It can clearly be seen
how the probability density functions obtained by both
methods differ considerably.
The bell-shaped (Gaussian like) probability density
function obtained by MCM and the good agreement
between the best estimates from GUF and MCM are indi-
cations pointing in direction of the underlying problem. The
reason for the difference in the results from GUF and MCM
is immediately visible analyzing equation (25): there is a
product of two quantities (qa  qa0 and 1/1qW - 1/1qR)
with zero expectation in the model function. In this well
known problem [30, p. 37] the usual GUF method of Fig. 1 Approximation of the probability density function for the
evaluation of uncertainty, based on the linearization of the output quantity using GUF, with and without linearization, and MCM
model function, has to be modified to avoid the missing of
an uncertainty component. For the product of two quantities application of GUF. For a numerical tolerance of one
with zero expectation, the standard uncertainty is not zero significant decimal digit in u(y) (d = 0.005), the validation
(as it is considered by applying the linearization of the of GUF is positive only if all uncertainty components are
model function in the GUF), but the product of the indi- accounted for. Thus, the 95% coverage interval is [1.08,
vidual standard uncertainties. If this is taken into account in 1.38]mg.
the GUF evaluation of the uncertainty (broken curve named
GUF2 in Fig. 1) it gives the estimate dm ^ ¼ 1:2340 mg with
associated standard uncertainty uðdm ^ Þ ¼ 0:0750 mg and a Application example 2: calibration of a Vernier calliper
shortest 95% coverage interval [1.0870, 1.3810]mg.
Table 4 expresses the results obtained with each of the This other case study was adapted from supplement 2 of
methods, as well as its comparison in order to validate the [30] and deals with the calibration of a Vernier calliper

123
Accred Qual Assur (2009) 14:95–106 103

Table 4 Results of the

Method dm
^ uðdm
^Þ Shortest 95% dlow (mg) dhigh (mg) GUF validated
calculation stage in the
(mg) (mg) coverage (d = 0.005)?
calibration of a mass using GUF
interval (mg)
method, with and without
linearization, and MCM MCM 1.2341 0.0755 [1.0849, 1.3839]
GUF1 1.2340 0.0539 [1.1284, 1.3396] 0.0435 0.0443 No
GUF2 1.2340 0.0750 [1.0870, 1.3810] 0.0021 0.0029 Yes

against grade I gauge block of steel. The example concerns
the 150 mm calibration point for measurement of external
dimensions. The error of the indication Ex of the calliper at
the reference temperature t0 = 20 C is obtained from the
relation:
Ex ¼ lix  ls þ Ls


a
Dt þ dlix þ dlM ð24Þ
where lix is the indication of the calliper, ls the length of the
gauge block, Ls the nominal length of the gauge block,
a
the average thermal expansion coefficient of the calliper
and the gauge block, Dt the difference in temperature
between the calliper and the gauge block, dlix the correc-
tion due to the finite resolution of the calliper and dlM the
correction due to mechanical effects.
The information on each of the input quantities is used
to assign a probability distribution to them:
• The result of measurement for the 150 mm gauge block
is lix = 150.10 mm. The measurement is repeated
several times without detecting any scatter in the
observations. Thus, uncertainty due to limited repeat-
ability does not give a contribution.
• The length of the reference gauge block used as working
standard, together with its associated expanded uncer-
tainty of measurement, is given in the calibration
certificate. This certificate confirms that the gauge block
comply with the requirements for grade I gauge blocks
according to ISO 3650, i.e. that the central length ls of the
gauge block coincides within ± 0.8 lm with the nom-
inal length Ls = 150 mm. A rectangular distribution can
be assigned to the quantity ls, taking the tolerance limits
as the upper and lower limits of the interval of variability.
• The average thermal expansion coefficient is

a ¼ 11:5  106  C1 . Its uncertainty is considered
negligible for the present case.
• After an adequate stabilisation time, the temperatures
of the calliper and the gauge block are equal within
±2 C. A rectangular distribution can be assigned to
this quantity Dt, with said limit values as extremes of
the distribution.
• The scale interval of the Vernier scale is 0.05 mm.
Thus, variations dlix due to the finite resolution are
estimated to have rectangular limits of ±25 lm.
• Mechanical effects dlM include the applied measure-
ment force, the Abbe error and the play between the
beam and the sliding jaw. Additional effects may be
caused by the fact that the measuring faces of the jaws
are not exactly flat, not parallel to each other and not
perpendicular to the beam. To minimize effort, only the
range of the total variation, equal to ±25 lm is
considered as interval of variability of a rectangular
distribution.
From this data, it is possible to use GUF as well as
MCM to obtain ^
 an estimate Ex of Ex, its related standard
uncertainty u E^x , and the shortest 95% coverage interval.
None of the input quantities are considered to be correlated
to any significant extent. The GUF and the adaptive MCM
were each used (a high number of significant figures have
been taken for comparison purposes only):
• GUF method (Table 5) gives the estimate E^x ¼
0:1000
  mm and associated standard uncertainty
u E^x ¼ 0:0323 mm. A probabilistically symmetric

Table 5 Uncertainty budget in

Quantity Estimate Probability Standard Sensitivity Uncertainty
the calibration of a Vernier
distribution uncertainty coefficient contribution
calliper using GUF method
Xi xi u(xi) ci ui(y) (mm)

lix (mm) 150.10 – – – –

pffiffiffi
ls (mm) 150.0000 Rectangular 0:0008 3 -1 -0.0005
Ls (mm) 150 – – – –

(C-1)
a 11.5 9 10 -6
– – – –
pffiffiffi
Dt (C) 0 Rectangular 2 3 0.001725 0.0020
pffiffiffi
dlix (mm) 0.000 Rectangular 0:025 3 1 0.0144
pffiffiffi
dlM (mm) 0.000 Rectangular 0:050 3 1 0.0289
Ex (mm) 0.1000 0.0323

123
104 Accred Qual Assur (2009) 14:95–106

Table 6 First lines of the MATLAB program for the calibration of a Vernier calliper

Line Programme Flow Diagram

1 clear;
2 p=0.95; n_digit=2; tol_divisor=5; Establishment of the process parameters and the
3 interval=1; % (1 'shortest', other value 'symmetric') default values of the control variables
4 M=max((100/1-p),10000); comp=0; h=0;
5 while comp==0
6 h=h+1;
7 ind=150.10;
8 block_length=random('unif',149.9992,150.0008,1,M);
9 nom_length=150; Creation of M size row vectors for each of the input
10 thermal_coef=11.5e-6; variables
11 temp_diff=random('unif',-2,2,1,M);
12 resol=random('unif',-0.025,0.025,1,M);
13 mech_effects=random('unif',-0.05,0.05,1,M);
14 y=ind-block_length+nom_length*thermal_coef.*temp_diff+resol+mech_effects;
15 ... Evaluation of the model. M size file vector
16 ...
17 ...

(and shortest) 95% coverage interval for Ex, based on a
coverage factor of 1.96 for the Gaussian probability
density function, is [0.0366, 0.1634]mm.
• To carry out the adaptive MCM, the first lines of the
program must be modified (Table 6). Running the
program, this time for n_digit = 2, it performed
6.74 9 106 evaluations of the model until the stabil-
ization of the results in 5 min. It gives the estimate
E^x ¼0:1000 mm with associated standard uncertainty
u E^x ¼ 0:0323 mm and a shortest 95% coverage
interval [0.0409, 0.1595]mm.
The results show that GUF and MCM give estimates and
associated standard uncertainties in good agreement, but
there are differences in the coverage intervals. Figure 2
represents the histogram generates by the MATLAB pro-
gram, onto which the curve of a Gaussian distribution of
Fig. 2 Approximation of the probability density function for the
output quantity using GUF, with Gaussian or trapezoidal distribution,
and MCM
parameters obtained from GUF has been superimposed
(continuous curve named GUFG). It can be seen how the
probability density functions obtained by both methods
differ significantly.
The trapezoidal-shaped probability density function
obtained by MCM and the uncertainty budget show that
the uncertainty of measurement associated with the result
is clearly dominated by two influences: the mechanical
effects and the finite resolution of the Vernier scale.
Thus, the assumption of a normal distribution for the
output quantity is not justified. If the two dominant
contributions arise from rectangular distributions, the final
distribution would be essentially trapezoidal. The distri-
bution resulting of convolving the two rectangular
distributions associated with the dominant contributions is
a symmetrical trapezoidal distribution (broken curve
named GUFT in Fig. 2). The half-widths of the base and
the top of the resulting symmetrical trapezoid are 75 and
25 lm, respectively. Ninety-five percentage of the area of
the trapezoid is encompassed by an interval ±60 lm
around its symmetric axes, corresponding to a coverage
factor of 1.83 [30, p. 62]. If this trapezoidal distribution
is taken into account in the GUF evaluation of the
uncertainty, it gives a shortest 95% coverage interval
[0.0408, 0.1592]mm.
Table 7 expresses the results obtained with each of the
methods, as well as its comparison in order to validate the
application of GUF. For a numerical tolerance of two
significant decimal digits in u(y) (d = 0.0005), the vali-
dation of GUF is positive only if a trapezoidal probability
distribution is assumed. In this case, the 95% coverage
interval is [0.041, 0.159]mm. For a numerical tolerance of
one significant decimal digit in u(y) (d = 0.005), the val-
idation status would be positive in both cases, Gaussian or
trapezoidal probability distribution, and the 95% coverage
interval would be [0.04, 0.16]mm.

123
Accred Qual Assur (2009) 14:95–106 105

Table 7 Results of the calculation stage in the calibration of a Vernier calliper using GUF method, with Gaussian or trapezoidal distribution, and
MCM
 
Method dE^x u dE^x (mm) Shortest 95% cove-rage dlow (mm) dhigh (mm) GUF validated GUF validated
(mm) interval (mm) (d = 0.0005)? (d = 0.005)?

MCM 1.0000 0.0323 [0.0409, 0.1595]

GUFG 1.0000 0.0323 [0.0366, 0.1634] 0.0043 0.0039 No Yes
GUFT 1.0000 0.0323 [0.0408, 0.1592] 0.0001 0.0003 Yes Yes

Conclusions Propagation of distributions using a Monte Carlo method (2008)

The ISO GUM guide is been adapted to new and improved
methods for the evaluation of measurement uncertainties.
The application of the Monte Carlo method is described in
the first of the supplements that are being written to com-
plete it. It gets over the limitations that apply in certain
cases to the method described in GUM, using the com-
puterized computation to perform the propagation of
probability distributions. Although many specific software
applications for evaluating uncertainties include the
application of the Monte Carlo method, it is not difficult to
implement in a general purpose mathematical program
such as MATLAB. In this way, there is total control over
the parameters of the process and over the stabilization
criteria of the results, benefiting as well from the calcula-
tion speed afforded by its programming language.
The program has been developed in MATLAB in order
to apply the adaptive Monte Carlo method. The instructions
list can be downloaded from the web page of the Group of
Structural Integrity [31]. It is intended to be simple to
facilitate comprehension, yet a more advanced user of
MATLAB can complete it using the ‘‘user interface’’
instructions. A friendlier environment could thus be created
similar to those in the specific programs for uncertainty
calculation, improving the presentation of the results and
entering the required information in each case using the
keyboard or mouse instead of in the program list itself.
The implementation in MATLAB of the adaptive MCM
has been illustrated by two application examples. The cases
selected show that combination of GUF and MCM leads to
a better understanding of the measurement process.
References
1. ISO IEC 17025:1999 General Requirements for the Competence
of Calibration and Testing Laboratories (1999) International
Organization for Standardization, Geneva
2. ISO 98:1995 Guide to the expression of Uncertainty in Mea-
surement 2nd edn (1995) International Organization for
Standardization, Geneva
3. JCGM 101:2008 Evaluation of measurement data – Supplement 1
to the ‘‘Guide to the expression of uncertainty in measurement’’ –
Sèvres, France: BIPM Joint Committee for Guides in Metrology
4. Bich W, Cox MG, Harris PM (2006) Evolution of the guide to the
expression of uncertainty in measurement. Metrologia 43:161–
166. doi:10.1088/0026-1394/43/4/S01
5. Rasmussen SN (2003) Software tools for the expression of
uncertainty in measurement MetroTrade workshop on traceability
and measurement uncertainty in testing (Berlin). Danish Tech-
nological Institute. http://www.teknologisk.dk/:
6. Castrup S (2004) A comprehensive comparison of uncertainty
analysis tools, measurement science conference (California).
http://www.isgmax.com/articles_papers.asp. Accessed 14 Nov
2008
7. Jurado JM, Alcázar A (2005) A software package comparison for
uncertainty measurement estimation according to GUM. Accredit
Qual Assur 10:373–381. doi:10.1007/s00769-005-0005-8
8. WINCERT Implex. http://www.implex.fr/. Accessed 14 Nov
2008
9. Evaluator, Newton Metrology Inc. http://metrologyforum.tm.
agilent.com/download.shtml. Accessed 14 Nov 2008
10. MUSE Measurement Uncertainty Research Group. Swiss Federal
Institute of technology, Zurich. http://www.mu.ethz.ch/. Acces-
sed 14 Nov 2008
11. UncertaintyManager The ChromSword Group. http://www.
iristech.net/. Accessed 14 Nov 2008
12. Minitab Statistical Software Minitab Inc. http://www.minitab.
com/. Accessed 14 Nov 2008
13. LabVIEW National Instruments. http://www.ni.com/. Accessed
14 Nov 2008
14. Analytica, Lumina Decision Systems Inc. http://www.lumina.
com/. Accessed 14 Nov 2008
15. @Risk, Palisade Corporation. http://www.palisade-europe.com/.
Accessed 14 Nov 2008
16. Ball C Oracle. http://www.crystalball.com/. Accessed 14 Nov 2008
17. MATLAB The MathWorks. http://www.mathworks.com/. Acces-
sed 14 Nov 2008
18. MathCad Parametric Technology Corporation. http://www.ptc.
com/. Accessed 14 Nov 2008
19. Mathematica, Wolfram Research. http://www.wolfram.com/.
Accessed 14 Nov 2008
20. Maple M. http://www.maplesoft.com/. Accessed 14 Nov 2008
21. Gonzalez AG, Herrador MA, Asuero AG (2004) Uncertainty
evaluation from Monte Carlo simulations by using Crystal Ball
software. Accredit Qual Assur 10:149–154. doi:10.1007/s00769-
004-0896-9
22. Damasceno JC, Borges RMH, Ordine AP, Getrouw MA, Fraga
ICS, Borges PP, Souza V (2005) Evaluation of Ag/AgCl-elec-
trode standard potential uncertainty used in primary pH
measurements by Monte Carlo simulation. Meas Sci Rev 5:49–52
23. Damasceno JC, Borges RMH, Couto PRG, Ordine AP, Getrouw
MA, Borges PP, Fraga ICS (2006) Estimation of primary pH
measurement uncertainty using Monte Carlo simulation. Metro-
logia 43:306–310. doi:10.1088/0026-1394/43/3/014

123
106
24. Canaves M Jr, Pompèia PJ (2006) Uncertainty of the density of
moist air: Gum 9 Monte Carlo. Braz Arch Biol Technol 49:87–95
25. Müller M, Wolf M, Rösslein M (2008) MUSE: computational
aspects of a GUM supplement 1 implementation. Metrologia
45:586–594. doi:10.1088/0026-1394/45/5/013
26. Matsumoto M, Nishimura T (1998) Mersenne Twister: a 623-
dimensionally equidistributed uniform pseudo-random number
generator. ACM Trans Model Comput Simul 8:3–30. doi:
10.1145/272991.272995
27. Marsaglia G (1995) The Diehard battery of tests of randomness,
CDROM (Supercomputer Computations Research Institute and
Department of Statistics, Florida State University, Florida)
123
Accred Qual Assur (2009) 14:95–106
28. Kacker RN (2006) Bayesian alternative to the ISO-GUM’s use of
the Welch-Satterthwaite formula. Metrologia 43:1–11. doi:
10.1088/0026-1394/43/1/001
29. Kacker RN, Toman B, Huang D (2006) Comparison of ISO-
GUM, draft GUM Supplement 1 and Bayesian statistics using
simple linear calibration. Metrologia 43:167–177. doi:10.1088/
0026-1394/43/4/S02
30. EA4 02 Expression of the Uncertainty of Measurement in Cali-
bration (1999) European Cooperation for Accreditation
31. Group of Structural Integrity (GIE Grupo de Integridad Estruc-
tural), Universidad de Burgos. http://web.ubu.es/investig/grupos/
ing_const/mmc-2/enlaces.html. Accessed 14 Nov 2008