Analytica Chimica Acta 480 (2003) 3952

Comparison of different approaches to estimate the

uncertainty of a liquid chromatographic assay
Edelgard Hund, D. Luc Massart, Johanna Smeyers-Verbeke
Chemo AC, Pharmaceutical Institute, Vrije Universiteit Brussel, Laarbeeklaan 103, B-1090 Brussels, Belgium
Received 23 July 2002; received in revised form 14 November 2002; accepted 10 December 2002

Abstract
A measurement result cannot be properly interpreted without knowledge about its uncertainty. Several concepts to estimate
the uncertainty of a measurement result have been developed. Here, four different approaches for uncertainty estimation are
compared on the example of the RP-high-performance liquid chromatography (HPLC) assay for tylosin for veterinary use:
the guide to the expression of uncertainty in measurement (GUM) approach, which derives the uncertainty of a measurement
result by combining the uncertainties related to the uncertainty sources of the measurement process; the top-down approach,
which uses the reproducibility estimate from an inter-laboratory study as uncertainty estimate; an approach recently presented
by Barwick and Ellison, which combines precision, trueness and robustness data to obtain an uncertainty estimate of the
measurement result and finally a further approach, which directly estimates the measurement uncertainty from a robustness
test. The comparison shows that the different approaches lead to comparable uncertainty estimates.
2002 Elsevier Science B.V. All rights reserved.
Keywords: Measurement uncertainty; GUM approach; Error-propagation; Error-budget; Top-down approach

1. Introduction
Uncertainty of a measurement is defined as a
parameter, which is associated with the result of
a measurement and characterizes the dispersion of
the values that could reasonably be attributed to the
measurand [1]. The result of a measurement is considered the best estimate of the value of the measurand, and all sources of uncertainty contribute to the
spread of the result [2]. As a consequence, a measurement result cannot be properly interpreted without
knowledge about the uncertainty of this result [2].

Corresponding author. Tel.: +32-2-477-4737;

fax: +32-2-477-4735.
E-mail address: asmeyers@vub.vub.ac.be (J. Smeyers-Verbeke).

Uncertainty can be expressed in two different forms,

the so-called standard and expanded uncertainty. The
uncertainty evaluation for the results of a measurement
leads to the standard uncertainty, u(x), which is expressed as a standard deviation. If the standard uncertainty is derived from different sources of uncertainty,
it is referred to as combined standard uncertainty, the
individual error components are expressed as standard
uncertainties u(yi ). The expanded uncertainty U(x)
defines an interval around the measurement result,
which contains the unknown true value with a defined
probability: X = x U (x). U(x) is obtained by multiplying u(x) by a coverage factor k, U (x) = ku(x) [3].
For k = 2, the expanded uncertainty roughly corresponds to half the length of a 95% confidence interval.
Several concepts were developed for the estimation
of the uncertainty related to a measurement result. In

0003-2670/03/$ see front matter 2002 Elsevier Science B.V. All rights reserved.
doi:10.1016/S0003-2670(02)01591-X

40

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

fact, one of the first approaches to uncertainty estimation in analytical chemistry was published in the
1980s. Wernimont used the precision estimates from
inter-laboratory method-performance studies for uncertainty estimations [4]. A completely different approach, referred to as the bottom-up, error-budget or
error-propagation approach, is proposed in the guide
to the expression of uncertainty in measurement
(GUM) [3]. This guideline was developed by several international organizations, mainly from the
physicalmetrological field. It derives the uncertainty
of a measurement result by combining the contributions of all uncertainty sources. EURACHEM [5]
adopted the GUM in the early 1990s for analytical
chemistry. However, since it is usually not evident to
quantify all sources of uncertainty, analytical chemists
were reluctant to adopt this approach. Instead, the
Analytical Methods Committee (AMC) of the Royal
Society of Chemistry resumed Wernimonts proposal
by introducing the top-down approach based on an
inter-laboratory study [2]. Shortly after, the Nordic
Committee for Food Analysis (NMKL) suggested a
related approach, based only on intra-laboratory data
[6]. The data required for the top-down approach, or
at least a part of them, are usually already available
from the method validation.
The discussion on the estimation of uncertainty of
a measurement has also influenced the ISO guideline
17025 [7]. In the section about uncertainty, it states
that reasonable estimation shall be based on knowledge of the performance of the method and on the
measurement scope and shall make use of, for example, previous experience and validation data. The
guideline explicitly refers to the GUM [3], but also
to the ISO standard on the accuracy of measurement
results [8]. The second edition of the EURACHEM
guideline [9] as well as IUPAC [10] suggest to use
validation data in the uncertainty estimation, too. The
latter does not only refer to inter-laboratory studies,
but also mentions the possibility to include information from robustness tests. Robustness tests check in
an intra-laboratory approach whether the method is
susceptible to small but deliberately introduced modifications of the method parameters, which might occur
when the method is transferred between laboratories
(or analysts, instruments). If the modifications introduced are adequately chosen, a robustness test can be
considered as a simulation of an inter-laboratory study.

The intra-laboratory estimate of the reproducibility

standard deviation obtained in this way could be used
as uncertainty estimate. Barwick and Ellison [11] propose a different approach to integrate robustness data
into uncertainty estimation. Their error-budget uses
robustness data as a supplement to precision and trueness data. Uncertainty sources which are not covered
by the latter are obtained from robustness tests.
For primary analytical chemical methods [12],
such as titrimetry [13] or isotope dilution with
mass spectroscopy [14,15], the application of the
bottom-up approach has already been demonstrated.
For more complex analytical chemical methods, e.g.
high-performance liquid chromatography, the quantification of all relevant uncertainty contributions is
very laborious, as follows from [16] and from a recently published case study [17]. Especially in the
pharmaceutical field, robustness tests are performed
during method validation. These validation data can
be reconsidered for the uncertainty evaluation. If data
from inter-laboratory studies are available, they can
serve as a base for uncertainty estimates, too.
For the HPLC-assay for tylosin in veterinary
use [18], an inter-laboratory precision evaluation
[19] as well as robustness testing [20] have been
performed earlier. Reconsidering these data, the
bottom-up approach is compared to uncertainty evaluation based on either an inter-laboratory approach,
robustness data or validation data in general. The
uncertainty evaluation only considers the analytical
process, uncertainty due to sampling is not taken into
account.
2. Experimental
2.1. Analyte
Tylosin for veterinary use consists of a mixture of
macrolide antibiotics. The main component is referred
to as tylosin A (TA). Moreover, tylosin B (TB), tylosin
C (TC) and tylosin D (TD) also contribute to the potency of the substance. Some impurities are described
as well [18]. Since there was a considerable lapse of
time between the inter-laboratory study and the robustness testthe latter being performed 2 years later
than the formersome degradation has to be taken
into account. This means that during the experiments

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

41

Table 1
Chromatographic conditions for the HPLC-assay for tylosin

2.3. Inter-laboratory study

Chromatographic column

Further information about the inter-laboratory

study, such as the chromatographic columns used by
the participating laboratories can be found in [19].
In [19], the method considered here is referred to as
method I. Laboratory 2 of the inter-laboratory study
corresponds to the one that performed the robustness
test. Twelve laboratories analyzed seven samples in
duplicate (n = 2) under repeatability conditions. Samples S3, S4 and S5 from the inter-laboratory study
are reconsidered and are referred to here as samples
A, B and C, respectively. The inter-laboratory study
additionally considered a normal-phase HPLC assay,
referred to as method II [19]. Here, this method is
used as a reference method in the trueness assessment
in the approach according to Barwick and Ellison.

Flow
Detection wavelength
Column temperature
Mobile phase A
Mobile phase B

Ratio A:B
Concentration (sample or substance)
Solvent for sample or substance
Injection volume

Stainless steel, packed

with octadecyl silica gel
R, 5 m diameter of the
particles, 250 mm
lengtha , 4.6 mm i.d.
1 ml/min
290 nm
35 C
Acetonitrile (ACN)
200 g/l NaClO4 in water,
pH set to 2.5 with 1 M
HCl
40:60 (v/v)
20 mg/100 ml solvent
Water:ACN, 50:50 (v/v)
20 l

a The EP-assay [18] prescribes 200 mm column length and

allows a deviation of up to 70%.

2.4. Robustness test

for the robustness test, the standard was not pure TA,
as in the inter-laboratory study. In fact, the other active
components can be detected in the standard at the time
of the robustness test as well. The fact that a different altering of samples and standards could introduce
a bias is not taken into account.
The European Pharmacopoeia prescribes a minimum content both for TA and for the total content,
which is the sum of the contents of TA, TB, TC and
TD [18]. Therefore, these two responses are mainly
considered in the uncertainty evaluation.
2.2. Chromatographic procedure
The conditions of the RP-HPLC assay with external
standard are shown in Table 1.

The experimental set-up of the robustness test is

given in [20]. It comprises eight experiments. Besides
six quantitative factors, it examines two chromatographic columns of the same type but from different
batches. Table 2 gives an overview of the factors and
levels examined. The modifications introduced were
chosen in a range that can usually be expected when
transferring the method between different laboratories. The robustness test considered those three samples from the inter-laboratory study that are already
mentioned above (A, B and C) [20]. Before and after
the design experiments, experiments at the nominal
conditions were performed during 4 days. From these
data, an estimate of the time-different intermediate
precision could also be obtained. All samples and
standards were prepared daily.

Table 2
Factors and levels examined in the robustness test
Physicochemical factor

Flow (ml/min)
(nm)
Temperature ( C)
A:B
NaClO4 (g/l)
pH (aqueous phase)

Level
1

Nominal

0.9
287
32
37:63
190
2.3

1.0
290
35
40:60
200
2.5

1.1
293
38
43:57
210
2.7

test

real

0.2
6
6
6%
20
0.4

0.01
4
2
2%
2
0.2

0.05
0.667
0.333
0.333
0.1
0.5

42

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

Fig. 1. Chromatogram for sample A (TA: tylosin A; TB: tylosin B; TC: tylosin C; TD: tylosin D).

A chromatogram of sample A obtained in the robustness test is given in Fig. 1. It is obvious that no
baseline separation was obtained, as was already observed earlier [19,20]. This was also the case for the
other samples.
The error-propagation approach makes use of some
of the results of the robustness test. For the approach proposed by Barwick and Ellison, data from
the inter-laboratory study and the robustness test are
combined.

3.1. The GUM approach [3]

where x is the measurement result which depends on

several parameters yi , each yi being a certain uncertainty source; u(yi ) is the standard uncertainty related
to this uncertainty source and x/yi the partial
derivative of x with respect to yi . A complete errorpropagation would also comprise the covariances.
Since independence of the effects is assumed, the covariances are generally neglected here. For measurement models that are characterized by a multiplicative
relationship, it can be shown that the error-propagation
algorithm of Eq. (1) corresponds to the summation of
the squares of the relative uncertainties [21]


 n 

 u(yi ) 2
u(x(yi , . . . , yn )) = x 
(2)
yi

The GUM approach consists of the identification

and quantification of the relevant sources of uncertainty followed by the combination of the individual
uncertainty estimates. The latter is done by means of
the error-propagation algorithm, which consists of a
first order Taylor series


 n 
2
 x

u(x(y1 , . . . , yn )) =
(1)
u(yi )
yi

The evaluation of the different uncertainty contributions can be classified into two groups. The type
A evaluation uses the statistical analysis of repeated
measurements to estimate the uncertainty u(yi ). It is
however not defined under which conditions the precision should be evaluated. In a type B evaluation,
the uncertainty contribution is assessed by a further
split-up of the uncertainty contribution or by considering information from different sources, including
manufacturers specifications, calibration certificates,

3. The different uncertainty approaches

i=1

i=1

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

experience or general knowledge of the behavior and

properties of relevant materials or instruments, etc.
The error-propagation approach requires the identification and quantification of all relevant uncertainty
contributions, which has to be performed separately
for each individual analytical method. Barwick [16]
provides an overview on sources of uncertainty in
HPLC and GC, but gives quantitative estimates only
for very few uncertainty sources. Recently, a case
study to apply the GUM to a HPLC method with
UV-Vis detection has been published [17]. However,
it only focuses on the uncertainty in the determination of peak areas and peak heights for samples with
known concentration and does not consider any content determination. For the determination of a peak
area, it identifies a list of parameters to be considered
in the uncertainty evaluation. If a content determination is considered, the uncertainty sources identified
have to be taken into account both in the calibration
and in the sample measurement. In HPLC, especially
if one examines pharmaceutical formulations within
a restricted concentration range, one often applies a
one-point calibration. For the assay considered here,
this approach is also used [18]. The response factor
R is then determined as
Ac
R=
(3)
mc
with Ac the peak area measured for the analyte in
the calibration and mc the amount of analyte in the
calibration solution. The uncertainty in the response
factor R, uR , follows from Eq. (1)
u2A
A2
(4)
u2R = 2c + c4 u2mc
mc
mc
where uAc is the uncertainty in the peak area measured in the calibration and umc the uncertainty in
the amount of analyte injected in the calibration. This
latter uncertainty (umc ) can be derived from the relative uncertainties in the purity of the solid standard
Pc , the weighing of the standard Wc , the volumetric
flask of the calibration solution Vfl and the injection
volume Vinj according to Eq. (2)






uPc 2
uWc 2
uVfl 2
2
2
+
+
umc = mc
Pc
Wc
Vfl

2 
uVinj
+
(5)
Vinj

43

In a type B approach, these uncertainties can be deduced from the uncertainty specifications of the manufacturer. For these specifications, one can assume
a rectangular distribution,which in practice means
that they are divided by 3 [5]. Theoretically, the
uncertainty in the molar mass M of the substance also
influences the uncertainty in umc , but this uncertainty
can generally be considered negligible. The uncertainty in the peak area (uAc ) is influenced by the uncertainty in the flow rate (uU ) and by the uncertainty
in absorbance, uabs . The latter uncertainty contribution is assessed in a type A approach from repeated
absorbance measurements, as a separate estimation of
the uncertainty in the detection wavelength and the integration is much more complicated. The uncertainty
in the flow rate, uU , can be estimated experimentally
(type A approach) or taken from the specification of
the instrument provider (type B approach); the latter
approach has been used here assuming a rectangular
distribution.
The content of the analyte in the sample, x, is determined as
x=

As
R

(6)

with As the peak area measured for the sample and R

the response factor. The corresponding uncertainty is
derived according to Eq. (1)
u2x =

u2As
R2

A2s 2
u
R4 R

(7)

with uAs the uncertainty in the peak area measured

for the sample and uR the uncertainty in the response
factor as obtained from Eq. (4). The uncertainty in the
peak area of the sample depends on the uncertainty
of the absorbance (uabs ), the uncertainty of the flow
rate (uU ) and the combined uncertainty of the weighing, dilution and injection of the sample, which here
is abbreviated as uDinj . The first two of these uncertainty sources, uabs and uU , are influenced by the same
factors as in the calibration measurement (see above).
The last mentioned source, uDinj , allows a type B approach using the uncertainties specified by the manufacturers of balances (uWs ), volumetric flasks (uVfl )
and the injection volume (uVinj ). The respective un
certainty specifications have to be divided by 3 if a
rectangular distribution is assumed [5]. Eq. (2) is used

44

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

to calculate the error-budget of uDinj






 
uVinj 2
uWs 2
uVfl 2
2
2
uDinj = Dinj
+
+
(8)
Ws
Vfl
Vinj
The uncertainty in the peak area of the sample of unknown concentration can then also be derived according to Eq. (2)


u
2  u D 2 u
2
U
abs
inj
2
2
uAs = As
(9)
+
+
abs
Dinj
U
where abs is the absorbance measured, Dinj the combined dilution and weighing factor for the injected
sample and U the flow rate of the mobile phase.
Thus, the error-budget for the uncertainty of the
content is not calculated in one step but split into different sub-budgets (type B approach). In the last step
of these calculations, the uncertainty of the content is
obtained from Eq. (7).
3.2. The top-down approach: uncertainty estimation
from reproducibility estimates
Most inter-laboratory precision evaluations are
based on the ISO standard 5725-2 [8]. Using an analysis of variance (ANOVA) approach, different variance
estimates are obtained from the inter-laboratory study.
In the top-down approach, the reproducibility standard
deviation obtained in this way, sR , is considered as the
standard uncertainty of a measurement result [2,4].
If several samples, e.g. at different concentrations
or with different matrices, are included in the uncertainty evaluation, a standard uncertainty of a measurement result can be obtained by pooling the relative
reproducibility standard deviations provided that the
relative standard deviations for the different samples
are comparable. If this is not the case, the uncertainty
is individually determined for different concentration
ranges or matrices.
3.3. The approach by Barwick and Ellison:
combination of validation data
Barwick and Ellison propose to combine three
different elements of method validation in order to
assess the uncertainty of a measurement result [11]:
precision, trueness and robustness. Time- and/or

operator-different (sI(T) , sI(O) or sI(OT) ) intermediate

precision estimates are considered.
In order to cover the uncertainties related to method
bias, trueness estimates should be considered as well.
Barwick and Ellison describe several possibilities to
estimate the uncertainty related to trueness, including
the analysis of certified reference materials (CRMs),
spiking studies and comparison with a reference
method. In the examples considered here, only the latter approach is applied. The mean method recovery is
given by R m = C method /C standard where C method is the
mean of the results obtained using the method under
consideration and C standard is the mean of the results
obtained using the standard method. The uncertainty
in the recovery, u(R m ), is given by


 2


 s
u(Cstandard ) 2
+
(10)
u(R m ) = R m  method
C standard
nC 2
method

where smethod is the standard deviation of the results obtained using the method, n is the number of
replicates and u(Cstandard ) is the standard uncertainty
associated with the standard method. The standard
method is preferably assessed according to ISO [22],
the reproducibility standard deviation of the standard
method is then used as the standard uncertainty of
the standard method. If the recovery is significantly
different from 1, this should be corrected for in the
measurement result.
As both precision and trueness are assessed within
a single laboratory, one should also consider uncertainties, which are due to different conditions, e.g. different laboratories. As a robustness test examines the
modifications of the method parameters (factors)
expected in a transfer between laboratories, robustness data should additionally be considered in the
uncertainty assessment. The significance of the effects
is determined by comparison with a critical effect,
which derives the standard error from a precision
estimate. Although Barwick and Ellison suggest to
use a precision estimate that is assessed over a short
period of time [11], here, the time-different intermediate precision standard deviation sI(T) is used, as it
has been shown that the repeatability generally leads
to an overestimation of the number of significant effects [23]. For those factors that yield no significant
effects, the uncertainty takes into consideration that
in real situations the variability of some factors might

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

be smaller than in the robustness test. Consequently,

the standard uncertainty corresponds to the standard
deviation that must be expected with the real variation

2tcrit sI(T) real

2tcrit sI(T)
u(x(yi )) =
=
i
n1.96 test
n1.96
= sI(T),corr i
(11)
where real is the usual variation of a factor i if the
method is working under control and test the variation introduced in the robustness test; i = real /test
is used to indicate their ratio, sI(T),corr is the corrected
standard deviation. For a two-level design, n = N/2
with N the number of design experiments and the
critical t-value tcrit is t(1/2,N1) with = 0.05. For
factors that lead to significant effects, the uncertainty
is estimated as
u(x(yi )) = u(yi )ci
with ci the sensitivity coefficient for factor i


 observed change in result 


ci = 

change in factor

(12)

(13)

45

is considered. Thus, the precision data from robustness tests are used in the same way as reproducibility
estimates (see Section 3.2).
Usually, in pharmaceutical analysis, a robustness
test that yields significant effects for the content determination leads to a further optimization of the method.
Consequently, uncertainty evaluation should only be
performed after the method has shown to be robust.
Notice that this deviates from the approach by Barwick and Ellison, which provide different algorithms
for significant and non-significant factors [11].

4. Results and discussion

Since the different samples have a different composition, the comparison of the uncertainty estimates
is based on the relative (standard) uncertainty, which
corresponds to a relative standard deviation. The values given in Tables 36 are standard uncertainties and
relative standard uncertainties.

4.1. The GUM approach

which can directly be deduced from the robustness
test. The uncertainty of the factor that leads to the
The GUM approach requires a quantification of
significant effect in the robustness test, u(yi ), is estiall relevant uncertainty contributions. First, the unmated from a rectangular probability distribution.
certainty in the response factor R was estimated. As
To obtain the combined uncertainty u(x), the uncerfollows from Eq. (4), this uncertainty depends on the
tainty estimates for the different factors p, q, r, . . . of
uncertainty of the amount of analyte injected and the
the robustness test are combined with the uncertainty
uncertainty of the peak area of the calibration standard.
estimates for precision and trueness in an error-budget
The former uncertainty, umc , is calculated according
comparable to Eq. (2)

 2






u(x)
u(Rm )
sI(T)
2
u(p) 2
u(q) 2
u(r) 2
+
+
+
+
+
(14)
=
x
x
p
q
r
R m
3.4. Uncertainty estimation from robustness
test data only
Robustness tests can be considered to be intralaboratory simulations of inter-laboratory studies,
if the modifications introduced are suitably chosen.
This allows a further approach to estimate the uncertainty of a measurement result. In contrast to the
approach by Barwick and Ellison (see Section 3.3),
one does not separately consider the different factors
examined in the robustness test, instead the variation
in the content observed over the design experiments

to Eq. (7). The uncertainty in the purity of the standard is extrapolated from the specification for another
in-house standard produced in the same way [24],
while the other uncertainty contributions are obtained
from manufacturers specifications. The corresponding uncertainty components are specified in Table 3.
As is also the case for the sample, the stock solution
was injected without further dilution. For the example considered, (2.175 0.015) 108 mol of TA
were injected for the calibration. The uncertainty of
the peak area of the TA peak is combined from contributions of the flow rate and of the absorbance. For the

46

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

Table 3
Uncertainty sources considered in the GUM approach; for demonstration purposes, determinations of sample B and C have been randomly
selected
Source

Value

Uncertainty (ui )

Percentageb

Calibration
Purity of standard, Pc
Weighing of standard, Wc
Volume of flask, Vfl a
Injection volume, Vinj a
Amount injected, mc
Flow rate, Ua
Absorbance (peak height), abs
Peak area standard, Ac
Response factor, R

100.0%
0.0249 g
25.00 ml
20.00 l
2.175 108 mol
1.00 ml/min
0.4199 V
8.394 V min
386.00 106 V min/mol

0.6%
0.00006 g
0.023 ml
0.058 l
1.546 1010 mol
0.003 ml/min
0.0042 V
0.088 V min
4.881 106 V min/mol

0.60
0.23
0.09
0.29
0.71
0.29
1.01
1.05
1.26

Sample B
Weighing of sample, Ws
Injection and dilution, Dinj
Peak area, As TC
Peak area, As TB
Peak area, As TD
Peak area, As TA
Content TC, x (mass%)
Content TB, x (mass%)
Content TD, x (mass%)
Content TA, x (mass%)

0.0250 g
1.996 105 g
1.033 V min
0.877 V min
0.552 V min
3.080 V min
12.89
10.94
6.89
38.41

0.00006 g
7.61 108 g
0.0108 V min
0.0092 V min
0.0058 V min
0.0322 V min
0.22
0.18
0.12
0.65

0.23
0.38
1.05
1.05
1.05
1.05
1.69
1.69
1.69
1.69

69.13

2.34

3.38

0.0251 g
2.008 105 g
0.343 V min
0.749 V min
0.562 V min
4.866 V min
4.26
9.29
6.97
60.34

0.00006 g
7.64 108 g
0.0036 V min
0.0078 V min
0.0059 V min
0.0509 V min
0.07
0.16
0.12
1.02

0.23
0.38
1.05
1.05
1.05
1.05
1.69
1.69
1.69
1.69

80.86

2.73

3.38

Total content, x (mass%)

Sample C
Weighing of sample, Ws
Injection and dilution, Dinj
Peak area, As TC
Peak area, As TB
Peak area, As TD
Peak area, As TA
Content TC, x (mass%)
Content TB, x (mass%)
Content TD, x (mass%)
Content TA, x (mass%)
Total content, x (mass%)
a
b

Values apply both for calibration and samples.

Relative uncertainty 100%.

estimation of the latter, 14 samples have been injected

in duplicate. The uncertainty was then estimated by
pooling the relative standard deviations of these 14
paired measurements for the peak heights of TA. One
might argue that these injections are also influenced
by the variability in the flow rate. However, it is assumed that this potential overestimation of the uncertainty should compensate the possible underestimation owing to the replication conditions chosen. The

relative uncertainty obtained in this way for the absorbance corresponds to about twice the values that
Hibbert et al. estimate or simulate [17]. Unfortunately,
there exist only few literature estimates for the uncertainty of the absorbance of UV detectors in HPLC.
The estimates obtained here correspond to the lowest ones within a larger range of values reported in
[25]. The relative uncertainty estimated in this way for
the absorbance of TA is also used as an estimate of

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

Table 4
Uncertainties estimated from the inter-laboratory study (top-down
approach)
Response
Sample A
Content
Content
Content
Content

TC
TB
TD
TA

Total content
Sample B
Content
Content
Content
Content

TC
TB
TD
TA

Total content
Sample C
Content
Content
Content
Content

TC
TB
TD
TA

Total content

Content
(mass%)

u (mass%)

sR2

%R.S.D.

1.49
4.43
5.24
77.44

0.22
0.49
0.87
3.22

0.05
0.24
0.76
10.36

14.73
10.95
16.61
4.16

88.61

3.48

12.12

3.93

11.79
8.67
5.61
38.04

1.29
0.84
0.46
1.11

1.66
0.70
0.21
1.24

10.91
9.67
8.11
2.92

64.11

2.65

7.03

4.13

4.11
7.94
5.53
60.68

0.51
0.92
0.75
1.65

0.26
0.85
0.57
2.71

12.49
11.58
13.64
2.71

78.26

3.52

12.40

4.50

the relative uncertainty of the absorbance of the other

active components. The peak area observed in this
calibration example and the response factor derived,
R, are given together with their uncertainties in the
upper part of Table 3.
For demonstration purposes, two injections of samples B and C have been randomly selected from a
larger number of determinations (nominal experiments
of the robustness test [20]). The uncertainty in the content determination is calculated according to Eq. (7).
The uncertainty budgets are outlined in Table 3. It is
obvious that, due to the approach used, the relative uncertainty in the content of the different components is
equal for all samples. Here, the relative uncertainty of
the total content is combined from the relative uncertainties of the contents of the individual active components according to Eq. (2). Therefore, the uncertainty
of the total content is larger than the uncertainty of the
individual components. It is obvious that the assumption of independence of the effects is certainly not true
for the uncertainties in the contents of the different
active components. Table 3 clearly shows that among
the parameters, the absorbance gives the largest uncertainty contribution.

47

4.2. The top-down approach: uncertainty

estimation from reproducibility estimates
The uncertainty estimates based on the interlaboratory study for tylosin [19] for the samples A,
B and C are summarized in Table 4. The uncertainty
of the total content is derived from the different results for the total content in the different laboratories.
Thus, it corresponds to the reproducibility standard
deviation.
The relative uncertainty estimates for the total
content in the different samples are comparable. In
contrast, the content of TA shows a somewhat larger
uncertainty for sample A than for samples B and C.
For control purposes, the estimated relative uncertainties are compared with the values expected from the
Horwitz function (%R.S.D.R ) [26]. With the exception of the uncertainty of the total content in sample
C (2.2 times the Horwitz value), the values lie within
the expected interval of 0.52.0 times the Horwitz
value [27]. Thus, the relative uncertainties show no
unexpected behavior.
4.3. The approach by Barwick and Ellison:
combination of validation data
While the approach described in Section 4.2 only
considers precision estimates from an inter-laboratory
study, here the time-different intermediate precision
estimated from the robustness data is considered, but
it is complemented by uncertainty estimates from the
trueness and robustness assessment. The mean content
for the nominal experiments of the robustness test is
given together with sI(T) and the corresponding relative
standard deviation in Table 5.
The trueness assessment (recovery study) is performed by a comparison of the method under study
with method II from the inter-laboratory study [19].
The mean values obtained with both methods are given
in Table 5. While for the method under study, only the
results obtained in laboratory 2 of the inter-laboratory
study are considered, the standard method always
considers the (mean) values of all laboratories. An estimate of the relative standard uncertainty of the standard method, (u(Cstandard )/C standard ) in Eq. (10) can
be derived from the inter-laboratory study by pooling
the reproducibility estimates for TA of all seven samples, which are rather comparable: 1.76%. Notice that

48

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

Table 5
Uncertainties components derived according to Barwick and Ellison
Content TA (mass%)

Total content (mass%)

68.09

34.82

54.04

82.00

62.47

74.26

1.92
2.81

1.38
3.95

0.63
1.17

2.41
2.94

1.55
2.48

1.18
1.58

Uncertainty components from recovery measurements

Mean content (laboratory 2, inter-laboratory study)
Mean content standard method
Recovery, R
u(R)
u(%)

76.05
77.20
0.99
0.02
1.94

37.29
37.40
1.00
0.02
2.10

60.17
61.30
0.98
0.02
1.79

86.21
87.39
0.99
0.02
1.99

61.26
62.75
0.98
0.02
1.95

76.39
77.46
0.99
0.02
1.83

Uncertainty components from robustness data

Ecrit
E(column)
E(flow)
E()
E(temperature)
E(A:B)
E()
E(pH)
sI(T),corr a
u(column)
%u(column)
u(flow)
%u(flow)
u()
%u()
u(temperature)
%u(temperature)
u(A:B)
%u(A:B)
u()
%u()
u(pH)
%u(pH)

0.0320
0.0319
0.0175
0.0130
0.0059
0.0013
0.0071
0.0047
0.0163
0.0082
1.200
0.0008
0.120
0.0109
1.600
0.0054
0.800
0.0054
0.800
0.0016
0.240
0.0082
1.200

0.0230
0.0028
0.0179
0.0145
0.0056
0.0011
0.0075
0.0178
0.0117
0.0059
1.686
0.0006
0.169
0.0078
2.249
0.0039
1.124
0.0039
1.124
0.0012
0.337
0.0059
1.686

0.0106
0.0018
0.0224
0.0047
0.0011
0.0126
0.0033
0.0135
0.0054
0.0027
0.499
0.0006
0.120
0.0036
0.665
0.0018
0.332
0.0024
0.448
0.0005
0.100
0.0039
0.721

0.0403
0.0295
0.0308
0.0123
0.0101
0.0025
0.0025
0.0009
0.0206
0.0103
1.255
0.0010
0.125
0.0137
1.673
0.0069
0.837
0.0069
0.837
0.0021
0.251
0.0103
1.255

0.0259
0.0086
0.0373
0.0124
0.0143
0.0147
0.0050
0.0148
0.0132
0.0066
1.060
0.0011
0.173
0.0088
1.413
0.0044
0.706
0.0044
0.706
0.0013
0.212
0.0066
1.060

0.0197
0.0076
0.0369
0.0043
0.0016
0.0071
0.0070
0.0148
0.0100
0.0050
0.676
0.0011
0.144
0.0067
0.901
0.0033
0.451
0.0033
0.451
0.0010
0.135
0.0050
0.676

Mean content
u(content)
%u(content)

68.09
2.93
4.30

34.82
2.01
5.78

54.04
1.34
2.47

82.00
3.67
4.48

62.47
2.44
3.91

74.26
2.10
2.83

Mean content (robustness test)

Uncertainty components from precision data
sI(T) (n = 4)
%R.S.D.

To be multiplied with = real /test according to Eq. (11).

for pooling, the number of degrees of freedom given

by Vander Heyden et al. were considered, which are
derived from a Satterthwaite approximation [19]. For
the method under study, the time-different intermediate precision estimates, determined as described
2
above, are used as smethod
in Eq. (10). The recoveries
and the related uncertainties calculated according to
Eq. (10) are given in Table 5. The recoveries are not

significantly different from 1. The uncertainty in the

recovery for the different samples is comparable.
The uncertainty estimation according to Barwick
and Ellison is completed by those uncertainty sources,
which must be expected in a method transfer between
laboratories. These uncertainty contributions are estimated from the robustness test [20]. The effects
E(factor) from the robustness test are given separately

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

Table 6
Uncertainties estimated from the robustness test
Compound

Content (mass%)

u (mass%)

%R.S.D.

Sample A
TA
Total

70.80
86.20

2.24
2.55

3.16
2.96

Sample B
TA
Total

36.60
66.50

1.71
2.66

4.69
4.00

Sample C
TA
Total

55.30
77.00

1.66
2.34

3.00
3.04

for the different samples in Table 5. The critical effects derived according to [11] are indicated, too.
Significant effects are underlined and additionally
marked with an asterisk. Table 5 gives the values
for sI(T),corr for the different samples, which must be
multiplied with = real /test according to Eq. (11).
For the factor column, the ratio between the modification introduced (test ) and the real modification
(real ) is estimated, as a quantitative assessment is not
possible: column = 0.5. This is rather conservative,
considering that a change between columns does not
occur between all analyses. Moreover, aging effects
occur only gradually. For the other factors, Table 2
gives an overview on test , real and = real /test .
The real -values are either based on the literature
information given by Barwick and Ellison [11] or
on specifications in the manuals of the equipment

49

used. The uncertainties derived in this way are given

in Table 5 together with the corresponding relative
uncertainties.
For those effects that lead to statistically significant effects, the uncertainty contribution is calculated
according to Eq. (12). The uncertainties in the factors, u(yi ), are derived from rectangular distributions
[9]: u(flow) = 0.0029 ml/min, u(A : B) = 0.577%
and u(pH) = 0.0577. The sensitivity coefficients of
Eq. (13) for the significant effects on the content
of TA for sample C are 0.224 min/ml, 0.0042 and
0.0675 for the flow rate, A:B and pH, respectively.
For the significant effects of the flow rate on the total
content, comparable sensitivity coefficients are calculated for samples B and C: 0.373 and 0.369 min/ml,
respectively. The uncertainties and relative uncertainties calculated for these significant factor-response
combinations are given in Table 5, too.
It is assumed that all relevant uncertainty contributions are covered by the precision, trueness and
robustness assessment. Finally, the relative uncertainties are combined according to Eq. (14). Fig. 2 gives
an overview of the contribution of the different uncertainty sources (in terms of relative uncertainties) on
the combined standard uncertainty for sample A. It
is obvious that precision and recovery (method bias)
give the largest contribution to the combined uncertainty. Among the uncertainty contributions assessed
in the robustness tests, the wavelength predominates,
which corresponds to the large uncertainty contribution of the absorbance in the GUM approach.

Fig. 2. Contribution of the different uncertainty sources in the uncertainty assessed according to Barwick and Ellison.

50

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

4.4. Uncertainty estimation from robustness

test data only
For this approach, the significance of the effects
in the robustness test was determined as described in
[20]. Notice that this deviates from the method applied
by Barwick and Ellison [11], and results in fewer significant effects. None of the samples shows significant
effects on the total content or on the content of TA.
The uncertainty estimates obtained from the variance of the design experiments of the robustness test
are shown in Table 6. Comparison with the expected
values for %R.S.D.R from the Horwitz function [26]
shows that the estimated relative uncertainties range
between 1.4 (TA, sample C) and 2.0 (TA, sample B)
times the Horwitz estimate, which means that they
still lie within the expected interval of 0.52.0 times
the Horwitz value [27] and therefore fulfil the expectation.
4.5. Comparison of the different approaches
An overview of the relative uncertainties for the different samples estimated with the different methods
is given in Table 7. Both the content of TA and the
total content are considered. The values for the relative uncertainty for the total content range between
2.83% (sample C, BarwickEllison) and 4.50% (samTable 7
Overview on the relative uncertainties for the different methods
and samples
Sample A

Sample B

Sample C

GUM approach
TA
1.69a
Total
3.38a

1.69
3.38

1.69
3.38

Top-down approach
TA
4.16
Total
3.93

2.92
4.13

2.71
4.50

Barwick and Ellison

TA
4.30
Total
4.48

5.78
3.91

2.47
2.83

Robustness data only

TA
3.16
Total
2.96

4.69
4.00

3.00
3.04

a Estimated values, the same uncertainty can be expected independent of the sample.

ple C, top-down approach) and are therefore quite

comparable.
It is obvious that the GUM approach as applied
here yields relative uncertainty estimates that are independent of the sample considered, while the other
methods lead to relative uncertainty estimates, which
can vary between the different samples. Especially for
the content of TA, the GUM estimates of the relative
uncertainty are clearly smaller than the estimates from
the other approaches. This might be due to the fact
that some uncertainty sources have been overlooked
and/or that the ones considered were underestimated.
The considerable difference between the uncertainty
of the total content and the uncertainty related to the
content of TA in the GUM approach follows from
the error-propagation. A smaller uncertainty of the
total content is to be expected if the covariances of
the contents of the different active components are
considered.
The top-down approach and the method by Barwick and Ellison usually result in slightly larger
uncertainty estimates than the two other methods considered. This is not illogic as these two approaches
explicitly consider the uncertainty due to a transfer
between laboratories, the influence of which might be
underestimated in the other two approaches.
The comparability of the results of the top-down
approach and the approach according to Barwick and
Ellison indicates that the smaller precision estimates
(intermediate precision instead of reproducibility)
compensate for the consideration of method bias
(recovery/trueness) in the latter approach.
At least for the total content, the uncertainties estimated from the robustness test are somewhat smaller
than the estimates from the inter-laboratory study. This
indicates that the intervals for the modification of the
parameters in the robustness test might be slightly too
small to simulate an inter-laboratory transfer. Moreover, in the robustness test, the interval for the modification in the detection wavelength was also chosen
symmetrically around the wavelength prescribed. This
could result in an underestimation of the variability,
as the absorbances can be expected to symmetrically
decrease in a small interval around the maximum
wavelength (nominal level). Thus, for this factor, an
asymmetrical interval, e.g. (290 nm; 293 nm) with the
maximum wavelength as one of the extreme levels in
the robustness test might have been more realistic.

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

Both methods that individually consider the uncertainty contribution of the different experimental
parameters, the GUM approach and the approach according to Barwick and Ellison, show that the wavelength gives the largest contribution in the uncertainty.

[2]
[3]

[4]

5. Conclusions
It was demonstrated that all four approaches lead
to comparable uncertainty estimates for the total
content. The results suggest that the GUM approach
can easily lead to an over- or underestimation of the
uncertainty as some uncertainty sources can be overlooked or wrongly quantified. Moreover, if the uncertainty of the total is combined from the uncertainty
in the content of the different active components, the
covariances should be considered to prevent from
an overestimation of the uncertainty. The approach
according to Barwick and Ellison and the approach
involving only robustness data can be advantageous
over the top-down approach, as they require less experimental expense. Moreover, the number of experiments to be performed in addition to the validation
experiments already performed, is rather low. As it
follows from theory, only the GUM approach and the
approach according to Barwick and Ellison provide
information about the contributions of the different
method parameters to the combined uncertainty. If
the uncertainty of a method is directly evaluated from
robustness data, one has to make sure that the robustness test indeed examines those modifications that are
likely to occur in a transfer between laboratories.

Acknowledgements

[5]
[6]

[7]

[8]

[9]
[10]

[11]

[12]

[13]
[14]
[15]
[16]
[17]
[18]

This work was supported by the Research contract no. NM/03/24 of the Belgian government (The
Prime Ministers ServiceFederal Office for Scientific, Technical and Cultural Affairs, Standardisation
Program).

[19]

[20]

References

[21]
[22]

[1] International Organization for Standardization, International

Vocabulary of Basic and General Terms in Metrology, second

51

ed., International Organization for Standardization, Geneva,

1993.
Analytical Methods Committee, Analyst 120 (1995)
23032308.
International Organization for Standardization, Guide for the
Expression of Uncertainty in Measurements, International
Organization for Standardization, Geneva, 1993.
G.T. Wernimont, Use of Statistics to Develop and Evaluate
Analytical Methods, AOAC, Arlington, VA, 1985.
EURACHEM, Quantifying Uncertainty in Analytical
Measurement, first ed., EURACHEM, 1995.
NMKL procedure no. 5, Estimation and Expression of
Measurement Uncertainty in Chemical Analysis, NMKL,
1997 (http://www.snt.no/nmkl).
ISO/IEC International Standard 17025, General Requirements
for the Competence of Testing and Calibration Laboratories,
first ed., International Organization for Standardization,
Geneva, 1999.
International Standard, Accuracy (Trueness and Precision) of
Measurement Methods and Results, ISO 5725-2, International
Organization for Standardization, Geneva, 1994.
EURACHEM, Quantifying Uncertainty in Analytical
Measurement, second ed., EURACHEM, 2000.
Report on the FAO, IAEA, AOAC INT/IUPAC International
Workshop on Principles and Practices of Method Validation,
Budapest, 46 November 1999 (IUPAC Projects 5/97/8 and
5/2/99).
V.J. Barwick, S.L.R. Ellison, VAM Project 3.2.1,
Development and Harmonisation of Measurement Uncertainty
Principles. Part d. Protocol for Uncertainty Evaluation from
Validation Data, Version 5.1, January 2000 (http://www.caeal.
ca/VAM%20uncertainty.pdf).
BCR Information, Metrology in Chemistry and Biology:
A Practical Approach, Report EUR 18405 EN, European
Commission DG 12, Brussels, 1998 (ISBN 1018-5593).
T. Anglov, I.M. Petersen, J. Kristiansen, Accred. Qual. Assur.
4 (1999) 504510.
I. Papadakis, P.D.P. Taylor, P. de Bivre, J. Anal. Atom.
Spectrom. 12 (1997) 791796.
U. rnemark, P.D.P. Taylor, P. de Bivre, J. Anal. Atom.
Spectrom. 12 (1997) 567572.
V.J. Barwick, J. Chromatogr. A 849 (1999) 1335.
D.B. Hibbert, J. Jiang, M.I. Mulholland, Anal. Chim. Acta
443 (2001) 205214.
European Pharmacopoeia, third ed. (Supplement), European
Pharmacopoeia, Strasbourg, 1998, pp. 504505.
Y. Vander Heyden, J. Saevels, E. Roets, J. Hoogmartens, D.
Decolin, M.G. Quaglia, W. Van den Bossche, R. Leemans,
O. Smeets, F. Van de Vaart, B. Mason, G.C. Taylor, W.
Underberg, A. Bult, P. Chiap, J. Crommen, J. De Beer, S.H.
Hansen, D.L. Massart, J. Chromatogr. A 830 (1999) 328.
E. Hund, Y. Vander Heyden, D.L. Massart, J. SmeyersVerbeke, J. Pharm. Biomed. Anal. 30 (2002) 11971206.
E. Hund, D.L. Massart, J. Smeyers-Verbeke, Trends Anal.
Chem. 20 (2001) 394406.
International Standard, Accuracy (Trueness and Precision) of
Measurement Methods and Results, ISO 5725-6-1994, ISO,
Geneva, 1994 (especially paragraph 8.1).

52

E. Hund et al. / Analytica Chimica Acta 480 (2003) 3952

[23] Y. Vander Heyden, K. Luypaert, C. Hartmann, D.L. Massart,

J. Hoogmartens, J. De Beer, Anal. Chim. Acta 312 (1995)
245262.
[24] E. Roets, P. Beirinckx, I. Quintens, J. Hoogmartens, J.
Chromatogr. 630 (1993) 159166.

[25] R.P.W. Scott, C.E. Reese, J. Chromatogr. 138 (1977) 283307.

[26] W. Horwitz, L.R. Kamps, K.W. Boyer, J. Assoc. Off. Anal.
Chem. 63 (1980) 13441354.
[27] M. Thompson, P.J. Lowthian, J. AOAC Int. 80 (1997) 676
679.