See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/315921140

Evaluation of the measurement uncertainty: Some common mistakes with a

focus on the uncertainty from linear calibration

Article  in  Journal of Chromatography A · March 2017

DOI: 10.1016/j.chroma.2017.03.078

CITATIONS READS

3 91

1 author:

Rouvim Kadis
D.I. Mendeleyev Institute for Metrology
22 PUBLICATIONS   80 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Rouvim Kadis on 21 February 2019.

The user has requested enhancement of the downloaded file.

 Journal of Chromatography A, 1499 (2017) 226–229

Contents lists available at ScienceDirect

Journal of Chromatography A
journal homepage: www.elsevier.com/locate/chroma

Discussion

Evaluation of the measurement uncertainty: Some common mistakes

with a focus on the uncertainty from linear calibration
Rouvim Kadis
Department of State Measurement Standards in Physico-Chemical Measurements, D. I. Mendeleyev Institute for Metrology, 19 Moskovsky pr.,190005 St.
Petersburg, Russian Federation

a r t i c l e i n f o a b s t r a c t

Article history: The rational strategy in the evaluation of analytical measurement uncertainty is to combine the “whole
Received 24 January 2017 method” performance data, such as precision and recovery, with the uncertainty contributions from
Received in revised form 27 March 2017 sources not adequately covered by those data. This paper highlights some common mistakes in evalu-
Accepted 28 March 2017
ating the uncertainty when pursuing that strategy, as revealed in current chromatographic literature.
Available online 29 March 2017
The list of the uncertainty components usually taken into account is discussed first and fallacies with
the LOD- and recovery uncertainties are noted. Close attention is paid to the uncertainty arising from a
Keywords:
linear calibration normally used. It is demonstrated that following a well-known formula for the standard
Chromatographic analysis
Measurement uncertainty evaluation
deviation of an analytical result obtained from a straight line calibration leads to double counting the
Combined bottom up/in-house validation precision contribution to the uncertainty budget. Furthermore, the precision component itself is often
approach estimated improperly, based on the number of replicates taken from the precision assessment experi-
Uncertainty from linear calibration ment. As a result, the relative uncertainty from linear calibration is overestimated in the budget and may
Uncertainty budget become the largest contribution to the combined uncertainty, which is clearly shown with an example
calculation based on the literature data.
© 2017 Elsevier B.V. All rights reserved.

1. Introduction the expression of uncertainty in measurement (GUM) [2] and the

There is a general trend in modern analytical science and service,
encouraged by laboratory accreditation requirements [1], towards
providing analytical measurement results with their uncertainty,
a key indicator of both reliability and fitness for purpose. In view
of lack of a standardized procedure of the uncertainty evaluation,
the task is far from being simple; it requires deep understanding of
the measurement process and mastery of statistical tools, in which
most analytical chemists do not take an active interest. Eventually
it leads to incorrect evaluation, with the resulting uncertainty and
the conclusions about the relative magnitudes of its constituents
being questionable. This is particularly the case with regard to pub-
lications on measurement uncertainty in chromatographic analysis
constituting perhaps the most part of the analytical uncertainty
evaluations published.
The problem of the measurement uncertainty evaluation is com-
plicated by the fact that there is no single way to do it, which
would be universally recommended in all practical situations. Two
principal approaches to measurement uncertainty are the mod-
elling (or “bottom-up”) approach set forth in the basic Guide to
E-mail address: rouvim.kadis@gmail.com
empirical (or “top-down”) approach that encompasses three “whole
method” performance methodologies: single-laboratory valida-
tion, interlaboratory studies, and proficiency testing, as outlined
in the EUROLAB report [3].
The modelling approach based on a thorough analysis of
the measurement system, modelling, and propagating individual
uncertainties through the measurement model relating the mea-
surand to other quantities upon which it depends is rather difficult
to implement in chemical analysis. Actually, an exhaustive model
of a chemical measurement process is not always possible to build
because of complexity of the process. Even if the model is con-
sidered known, the difficulty consists in evaluating independent
uncertainty components each corresponding to an input quantity
in the model.
On the other hand, the empirical approach that is based on the
estimated “whole method” performance makes allowance for the
all-inclusive variability at once, but immediately poses the question
of sufficiency, the extent of coverage, and “extrapolation” from the
specific conditions of the performance study to the actual condi-
tions.
The helpful adaptation of the basic approach [2] to analyti-
cal chemical problems, made in the EURACHEM/CITAC Guide [4],
allowed using the “whole method” performance parameters, such

http://dx.doi.org/10.1016/j.chroma.2017.03.078
0021-9673/© 2017 Elsevier B.V. All rights reserved.
 R. Kadis / J. Chromatogr. A 1499 (2017) 226–229
as precision and recovery, alongside with individual uncertainty
components emerging from the model. The rational strategy is
thus to combine the performance information, commonly obtained
during in-house validation studies, with uncertainty contributions
from sources not adequately covered in those studies. This is a
kind of combination of the bottom-up uncertainty approach and
in-house validation data.
With this strategy, however, the risk is run to take account of a
source of uncertainty twice: once as an individual contribution and
second one within the measure of the overall performance. Here
we face a problem of double counting, an issue often overlooked in
estimating the uncertainty of analytical measurement.
In the present paper, the problem of double counting is high-
lighted with regard to the uncertainty arising from a linear
calibration function normally used. Calibration of the measurement
system with reference materials is known to be an important point
in instrumental analysis. However, is the uncertainty from calibra-
tion really the largest contribution to the combined uncertainty as
suggested in many chromatography uncertainty evaluations pub-
lished? As is seen from the discussion that follows, this finding is a
consequence of inadequate evaluation rather than may reflect the
reality.
2. The relevant uncertainty components
As may be seen from current publications [5–14],1 the typical
list of the component uncertainties to be taken into account in
chromatographic analysis, following the combined bottom up/in-
house validation approach, includes (as relative contributions): (i)
the uncertainty associated with sample preparation, u(sample)rel ,
(ii) the uncertainty associated with calibration (including calibra-
tion standards uncertainty), u(calibr)rel , (iii) the uncertainty arising
from random effects (precision), u(prec)rel , and (iv) the uncertainty
associated with recovery, u(rec)rel , so that the expression for the
combined standard uncertainty will run as follows:
 studies. As is seen from Eq. (2), the uncertainty in the recovery esti-
(uc )rel = u(sample)rel 2 + u(calibr)rel 2 + u(prec)rel 2 + u(rec)rel 2 (1)
Some authors [15–19] take also account of the limit of detection
(LOD) as a separate source (an element) of uncertainty, quantified in
relative form as the reciprocal of the analyte level: u(LOD)rel = LOD
c .
The reasoning of the proponents of this view [15] such as “the
uncertainty is 100% when the concentration level is equal to LOD”
does not seem to be convincing enough to consider the LOD to be
a source of uncertainty in analytical measurement. The LOD is not
of direct relevance to the uncertainty evaluation. In fact, involv-
ing the term LOD c in Eq. (1) is an attempt to model the variation of
the combined uncertainty with the level. The general mathematical
expression proposed for this aim (termed uncertainty or character-
istic function [20]) is based on a two-component model combining
both constant (˛) and proportional (ˇc) effects,  which in case of
u ˛2
the relative uncertainty takes the form: c = + ˇ2 . It is evi-
c2
dent that the squared LOD-component suggested to be included
in Eq. (1) acts as the first term in this model, intended merely to
provide rapidly increasing values as concentrations fall towards
zero. To sum it up, the dependence on the level (where a wide
working range is covered) should be accounted for in the precision
component u(prec)rel , without invoking the LOD.
1
Reports on quantitative analysis using chromatographic techniques with
detailed uncertainty evaluations following the combined approach above, published
from 2010 to 2016, have been taken into consideration only.
227
Dictated by common sense, the constituents of the combined
uncertainty on Eq. (1) are not objectionable, but their quantification
in the evaluations cited rises questions.
For example, the recovery term u(rec)rel reckons with the situ-
ation where the recovery factor or, more precisely, the “apparent
recovery” [21], is (reciprocally) involved in the measurement equa-
tion so that the result of analysis is explicitly corrected for observed
bias. Then the uncertainty associated with (corrected) recovery is
given by [22]:

(srec )2rel
u(rec)rel = + (uref )2rel (2)
m
where (srec )rel is the relative standard deviation of the results during
estimation of the recovery, m is the number of replicates performed,
and (uref )rel is the relative standard uncertainty of the reference
value carried by a CRM or spiked material used for estimating the
recovery.
Very often, however, even bad recoveries, such as 75–125% [23],
are considered acceptable. (There must be technical and economic
reasons that would justify a correction [24]; otherwise the correc-
tion is not applied.) In this case many authors simply omit the term
u(rec)rel considering that is all there is to it. However, an addi-
tional uncertainty contribution accounting for that uncorrected
bias should be taken into consideration; a number of approaches
for doing this was discussed in the literature (see the review articles
[24,25]). As a matter of fact, the need for increasing the uncer-
tainty to account for an estimated but uncorrected bias is ignored
in papers on the uncertainty in chromatographic analysis.
It is generally assumed that the separate contributions in the
uncertainty assessment budget expressed by Eq. (1) are not “over-
lapped”. For instance, the uncertainty arising from random effects,
if assessed under repeatability or even day-to-day precision con-
ditions with calibration being unchanged, does not include the
calibration uncertainty that needs therefore to be estimated sepa-
rately. “Overlapping” may be difficult to avoid as is specifically the
case with the contributions derived from precision and recovery
mate inevitably includes the precision of the observed mean value
that is used in calculation of the recovery. Bearing this in mind,
some authors [6,18,26,27] do not include the term u(prec)rel into
the budget in addition to the u(rec)rel thus trying to avoid double
counting.
In the following, we will focus mainly on the two uncertainty
components in Eq. (1), u(calibr)rel and u(prec)rel . Usually, it does
not present any difficulty to quantify the component u(sample)rel
that incorporates contributions from the sample mass or vol-
ume measurement as well as from other operations during a
sample preparation procedure (which are not covered by the pre-
cision/recovery experiment).
3. Calibration and its associated uncertainty
In a typical case of multipoint calibration, either external or
internal, the linear calibration function y = a + bx is used relating
an analytical signal y to the analyte content x in the prepared
sample solution, with the peak area or peak height, or the respec-
tive signal-ratio as the analytical signal. Initially, the least-squares
regression is performed on a set of calibration data (xi , yi ) to derive
the parameters a and b of the best fit line. Thereafter the inverse of
the calibration function
y0 − b
x0 = (3)
a
can be utilized to infer (or as they often say “to predict”)
the unknown analyte content x0 corresponding to an observed
response y0 .
228 R. Kadis / J. Chromatogr. A 1499 (2017) 226–229
According to Eq. (3), the uncertainty in the estimated value x0 ,
which is often referred to as calibration uncertainty, arises from
the variability in y0 combined with the variability in location of
the regression line (due to the uncertainties in a and b). An addi-
tional uncertainty due to the fact that the reference values xi are
not known exactly, but with some uncertainty, can be accounted
separately in the budget; this uncertainty commonly contributed
by the degree of purity of the material, weighing and volumetric
operations, and so on will not be considered here.
The calibration uncertainty u(x0 ) is estimated using the well-
known formula for the standard deviation/confidence interval on
x0 , deduced in the linear regression analysis [28]:

sy/x  1
1 (x − x̄)2
u(x0 ) =  + + n 0
b m n 
 (xi − x̄)2
i=1
In this equation sy/x is the residual standard deviation (of the data
points from the regression line), m is the number of replicates on
the sample performed to give the value of x0 , n is the number of
calibration points (multiplied where appropriate by the number of
replicate measurements at each point), x̄ is the average value of x
in the calibration set, and the summation is over all data pairs.
The formula, Eq. (4), has long been involved in standard texts
on statistics in analytical chemistry, such as [29]; its derivation,
conditions for validity, and the use with regard to analytical cal-
ibration were explained in the literature [30,31]; the formula is
recommended in the EURACHEM uncertainty Guide [4 (E.4)]. And
yet, if calibration uncertainty is considered in the budget along with
the “whole method” performance data, calculation following Eq. (4)
will be wrong.
Indeed, the term sy/x /bm (or simply sy/x /b in case m = 1) in this
equation characterizes the estimated precision in the quantity x
from a calibration experiment. Meanwhile the observed precision
(estimated from replicate determinations during validation study
or quality control checks) is directly included in the budget, Eq.
(1), with the term u(prec)rel . The fact that the observed precision
incorporates all the individual sources of variability, including that
relating to calibration, renders the sy/x /bm contribution superflu-
ous within calibration uncertainty. It follows that the first term
under the radical in Eq. (4) needs to be omitted in the budget-based
uncertainty evaluation to avoid double counting. Unfortunately,
this fact has been overlooked in the uncertainty evaluations cited
[5–19], resulting in overestimation of the calibration uncertainty
in varying degrees.
There may be cases where calibration covers most sources of
s
variation inherent in the whole measurement, and so, the y/x b
statistic in Eq. (4) represents the analytical precision. Then, the
component u(prec)rel may alternatively be omitted in the budget.
Table 1
The relative calibration uncertainties and the relative precision uncertainty as derived from the data in Ref. [7].
u(x0 )
Concentration, mg/l u(cal)rel = x
,% a
Calculation using Eq. (4)
1 1
with m
0.8 125
4 23.9
8 11.6
12 7.6
20 4.6
40 3.2
a
Raw data from Table 2 in [7] (m = 2, n = 6).
b
Raw data from Table 3 in [7], available for concentration of 4.0 mg/l only.
It is this approach that was used in studies on the uncertainty in
chromatography water analysis [26,27].
4. How many replicate determinations are required to
obtain the measurement result?
Double counting above is not the only cause of overestimation
of the relative calibration uncertainty calculated from Eq. (4); the
other one is as follows.
Figure m in Eq. (4) should be read as signifying the number of
replicate determinations carried out on the analytical procedure
in its actual or intended use. It is this number of replicates that is
required to produce the result of analysis having a stated uncer-
tainty. Accordingly, the precision estimate in the budget should be
(4) adjusted for this m value by dividing the standard deviation s(single)
√
of a single determination by m. The surprising thing in nearly
all the evaluations cited is that the “uncertainty associated to the
method precision” calculated in relative form as
s(sin gle)
u(prec)rel = √ (5)
x̄0 m
(with x̄0 standing for the mean result) takes account of the number
of replicates m from the precision assessment experiment (where
usually m ≥ 6), not from the analytical procedure specification (that
is lacking); as if someone would perform six or more replicate
determinations in practical analysis. As a consequence, the preci-
sion of the result, that usually contributes significantly, turns out
to be underestimated in the uncertainty budget in favor of other
constituents, first of all that arising from calibration.
It is instructive to recall why clear specification is necessary:
only specified analytical procedure can produce the result with a
stated uncertainty, while “analytical method” since it is described
in general terms cannot.
Another reason why the relative calibration uncertainty is exag-
gerated in the budget, especially at low concentration levels, comes
from limitations of the least squares statistical model. One of the
assumptions of the simple least squares method is that the random
variation of the response (expressed as the sy/x statistic in Eq. (4)) is
independent of the variable x, so that the uncertainty u(x0 ) from Eq.
(4) does not convey the notable analytical feature that the uncer-
tainty tends to decrease as x decreases. Because of this, the relative
u(x )
quantity x0 that behaves as the reciprocal of x may show a large
(but conceptually unjustified) increase in its magnitude when the
analyte level is diminished.
5. Example calculation
The features highlighted above can be exemplified with the
component uncertainties estimated from published data from the
study of the quantification of pentachlorophenol (PCP) in water by
SPME-GC–MS [7]. The Table 1 gives the relative calibration uncer-
S(single)
u(prec)rel = √
x̄0 m
, %b
without m
m=6 m=2
78
13.8 9.6 16.6
6.3
3.9
2.4
2.6
 R. Kadis / J. Chromatogr. A 1499 (2017) 226–229
tainties at different analyte concentrations, calculated both with
1 uncertainty of polychlorinated biphenyls, polycyclic aromatic hydrocarbons
and without the term m in Eq. (4), and the relative precision uncer-
tainties (from a series of six repeats) calculated by Eq. (5) with m
equal to both 6 and 2. As is clear from the table, the calibration
uncertainty that showed up as dominating in the original budget [7]
is less than the precision contribution if both of the components are
properly estimated. The same holds true of other uncertainty eval-
uations cited where the calibration uncertainty was unjustifiably
overstated.
6. Conclusions
As revealed by analysis of the current literature, fallacies in the
evaluation of the component uncertainties and hence the com-
bined uncertainty of the result are common with chromatographic
analysis. In particular, the uncertainty from linear calibration is
overestimated because of double counting the precision contribu-
tion, let alone the confusion in estimating the precision component
itself.
Duplication resulting from the use of the well-known formula,
Eq. (4), for calibration uncertainty, when the observed precision is
included in the budget, should be resolved by simply omitting the
first component in that formula.
Clearing up these issues will hopefully improve quantification
of the uncertainty in chromatographic analysis, making the uncer-
tainty statement more realistic.
References
[1] ISO/IEC 17025:2005, General Requirements for the Competence of Testing
and Calibration Laboratories, International Organization for Standardization,
2005.
[2] JCGM 100:2008, Evaluation of Measurement Data—Guide to the Expression of
Uncertainty in Measurement, Joint Committee for Guides in Metrology, 2008.
[3] EUROLAB Technical Report 1/2007, Measurement uncertainty revisited:
Alternative approaches to uncertainty evaluation, Eurolab, 2007.
[4] EURACHEM/CITAC Guide CG 4, Quantifying Uncertainty in Analytical
Measurement, Third edition, 2012.
[5] B. Kanrar, S. Mandal, A. Bhattacharyya, Validation and uncertainty analysis of
a multiresidue method for 42 pesticides in made tea, tea infusion and spent
leaves using ethyl acetate extraction and liquid chromatography–tandem
mass spectrometry, J. Chromatogr. A 1217 (2010) 1926–1933.
[6] H. Tian, Uncertainty analysis of chlormequat residue in fruits by LC–MS–MS,
Chromatographia 73 (2011) 457–462.
[7] I. Brás, N. Ratola, A. Alves, Uncertainty in the quantification of
pentachlorophenol in wood processing wastewaters by SPME-GC–MS, J. Anal.
Chem. 66 (2011) 756–762.
[8] T. Tomić, N.U. Nasipak, S. Babić, Estimating measurement uncertainty in
high-performance liquid chromatography methods, Accred. Qual. Assur. 17
(2012) 291–300.
[9] F.O. Pelit, L. Pelit, H. Ertaş, F.N. Ertaş, Development of a gas chromatographic
method for the determination of Chlorpyrifos and its metabolite
Chlorpyrifos-oxon in wine matrix, J. Chromatogr. B 904 (2012) 35–41.
[10] J.J. Jimenez, Determination of aminopolycarboxylic acids in river water by
solid-phase extraction on activated charcoal cartridges and gas
chromatography with mass spectrometric detection. Method performance
characteristics and estimation of the uncertainty, Anal. Chim. Acta 770 (2013)
94–102.
View publication stats
229
[11] G. Aslan-Sungur, E.O. Gaga, S. Yenisoy-Karakas, Estimation of measurement
and organochlorine pesticides in the atmosphere using gas
chromatography-mass spectrometry and gas chromatography-electron
capture detector, J. Chromatogr. A 1325 (2014) 40–48.
[12] J.Y. Kim, W. Kwon, H.S. Kim, S.I. Suh, M.K. In, Estimation of measurement
uncertainty for the quantification of 11-nor-delta
9-tetrahydrocannabinol-9-carboxylic acid and its glucuronide in urine using
liquid chromatography–tandem mass spectrometry, J. Anal. Toxicol. 38
(2014) 164–170.
[13] C. Li, H. Jia, Y. Wang, M. Shen, S. Nie, M. Xie, Determination of
3-monochloropropane-1, 2-diol esters in edible oil—method validation and
estimation of measurement uncertainty, Food Anal. Methods 9 (2016)
845–855.
[14] S.C.W. de Souza Eller, F. de Oliveira, M. Yonamine, Measurement uncertainty
for the determination of amphetamines in urine by liquid-phase
microextraction and gas chromatography–mass spectrometry, Forensic Sci.
Int. 265 (2016) 81–88.
[15] P. Konieczka, J. Namieśnik, Estimating uncertainty in analytical procedures
based on chromatographic techniques, J. Chromatogr. A 1217 (2010) 882–891.
[16] M. Borecka, A. Białk-Bielińska, G. Siedlewicz, K. Kornowska, J. Kumirska, P.
Stepnowski, K. Pazdro, A new approach for the estimation of expanded
uncertainty of results of an analytical method developed for determining
antibiotics in seawater using solid-phase extraction disks and liquid
chromatography coupled with tandem mass spectrometry technique, J.
Chromatogr. A 1304 (2013) 138–146.
[17] D.G. Hela, V.D. Papadopoulos, Estimating uncertainty in matrix solid phase
extraction methodology for the determination of chlorinated compounds in
fish, Int. J. Environ. Anal. Chem. 93 (2013) 1676–1697.
[18] A. Pavlova, O. Stoyanova, P. Ivanova, T. Dimova, Uncertainty estimation
related to analysis of volatile aromatic hydrocarbons by static headspace gas
chromatography, J. Chromatogr. Sci. 52 (2014) 567–572.
[19] W. Wang, X. Ma, X. Guo, M. Zhao, P. Tu, Y. Jiang, A series of strategies for
solving the shortage of reference standardsfor multi-components
determination of traditional Chinese medicine, Mahoniae Caulis as a case, J.
Chromatogr. A 2015 (1412) 100–111.
[20] M. Thompson, Uncertainty functions, a compact way of summarising or
specifying the behaviour of analytical systems, Trends Anal. Chem. 30 (2011)
1168–1175.
[21] D.T. Burns, K. Danzer, A. Townshend, Use of the terms recovery and apparent
recovery in analytical procedures, Pure Appl. Chem. 74 (2002) 2220–2225.
[22] V.J. Barwick, S.L.R. Ellison, Protocol for uncertainty evaluation from validation
data, LGC/VAM/1998/088, LGC (Teddington), 2000.
[23] D.M. Bliesner, Validating Chromatographic Methods: A Practical Guide,
Wiley-Interscience, Hoboken, NJ, 2006, Attachment IV.
[24] B. Magnusson, S.L.R. Ellison, Treatment of uncorrected measurement bias in
uncertainty estimation for chemical measurements, Anal. Bioanal. Chem. 390
(2008) 201–213.
[25] T.P.J. Lisinger, Use of recovery and bias information in analytical chemistry
and estimation of its uncertainty contribution, Trends Anal. Chem. 27 (2008)
916–923.
[26] N. Gros, M.F. Camões, R.J.N. Bettencourt da Silva, Uncertainty budget for
simultaneous determination of minor and major ions in seawater with ion
chromatography confronted with uncertainties in concentrations of
calibration standards, Anal. Lett. 43 (2010) 1317–1329.
[27] I. Komorowicz, D. Barałkiewicz, Arsenic speciation in water by
high-performance liquid chromatography/inductively coupled plasma mass
spectrometry—method validation and uncertainty estimation, Rapid
Commun. Mass Spectrom. 28 (2014) 159–168.
[28] G.A.F. Seber, A.J. Lee, Linear Regression Analysis, Second edition,
Wiley-Interscience, Hoboken, NJ, 2003, Par. 6.1.5.
[29] J.N. Miller, J.C. Miller, Statistics and Chemometrics for Analytical Chemistry,
Sixth edition, Pearson Education Limited, Harlow, 2010, Par. 5.6.
[30] J.N. Miller, Basic statistical methods for analytical chemistry. Part 2.
Calibration and regression methods. A review, Analyst 116 (1991) 3–14.
[31] D.B. Hibbert, The uncertainty of a result from a linear calibration, Analyst 131
(2006) 1273–1278.