Vous êtes sur la page 1sur 10

DIABETES TECHNOLOGY & THERAPEUTICS

Volume 16, Number 2, 2014


Mary Ann Liebert, Inc.
DOI: 10.1089/dia.2013.0208

ORIGINAL ARTICLE

Evaluation of 12 Blood Glucose Monitoring


Systems for Self-Testing:
System Accuracy and Measurement Reproducibility
Guido Freckmann, MD, Annette Baumstark, PhD, Christina Schmid, PhD,
Stefan Pleus, MS, Manuela Link, ME, and Cornelia Haug, MD

Abstract
Background: Systems for self-monitoring of blood glucose (SMBG) have to provide accurate and reproducible blood glucose
(BG) values in order to ensure adequate therapeutic decisions by people with diabetes.
Materials and Methods: Twelve SMBG systems were compared in a standardized manner under controlled laboratory
conditions: nine systems were available on the German market and were purchased from a local pharmacy, and three
systems were obtained from the manufacturer (two systems were available on the U.S. market, and one system was not yet
introduced to the German market). System accuracy was evaluated following DIN EN ISO (International Organization for
Standardization) 15197:2003. In addition, measurement reproducibility was assessed following a modified TNO (Netherlands
Organization for Applied Scientific Research) procedure. Comparison measurements were performed with either the glucose
oxidase method (YSI 2300 STAT Plus glucose analyzer; YSI Life Sciences, Yellow Springs, OH) or the hexokinase method
(cobas c111; Roche Diagnostics GmbH, Mannheim, Germany) according to the manufacturers measurement procedure.
Results: The 12 evaluated systems showed between 71.5% and 100% of the measurement results within the required system
accuracy limits. Ten systems fulfilled with the evaluated test strip lot minimum accuracy requirements specified by DIN EN
ISO 15197:2003. In addition, accuracy limits of the recently published revision ISO 15197:2013 were applied and showed
between 54.5% and 100% of the systems measurement results within the required accuracy limits. Regarding measurement
reproducibility, each of the 12 tested systems met the applied performance criteria.
Conclusions: In summary, 83% of the systems fulfilled with the evaluated test strip lot minimum system accuracy requirements of DIN EN ISO 15197:2003. Each of the tested systems showed acceptable measurement reproducibility. In order
to ensure sufficient measurement quality of each distributed test strip lot, regular evaluations are required.
Background

elf-monitoring of blood glucose (SMBG) plays an


important role in the maintenance of glycemic control in
both type 1 and type 2 insulin-treated diabetes patients.13 In
order to ensure adequate therapeutic decisions, it is crucial
that glucose monitoring systems intended for self-testing by
people with diabetes provide accurate and reproducible
blood glucose (BG) values.
The international norm DIN EN ISO (International Organization for Standardization) 15197:20034 is one of the most
widely accepted standards to assess the accuracy of SMBG
systems. In DIN EN ISO 15197:2003, system accuracy is defined as closeness of agreement between a measurement result and the accepted reference value determined by the
manufacturers measurement procedure. According to this
norm, at least 95% of the system measurement results shall fall

within 15 mg/dL of the results of the manufacturers measurement procedure at BG concentrations < 75 mg/dL and
within 20% at BG concentrations 75 mg/dL. In the recently published revision ISO 15197:2013,5 criteria for system
accuracy are more stringent, with at least 95% of the system
measurement results within 15 mg/dL of the results of the
manufacturers measurement procedure at BG concentrations < 100 mg/dL and within 15% at BG concentrations
100 mg/dL. In this revision, mandatory compliance is recommended after a 36-month transition period.
In order to obtain the CE (Conformite Europeenne) mark
for SMBG meters, a minimum requirement for being marketed in Europe, manufacturers have to provide evidence of
conformity with DIN EN ISO 15197:2003. However, studies
have repeatedly shown that individual test strip lots of
available systems do not comply with accuracy requirements
stated in DIN EN ISO 15197:2003, notwithstanding their

Institut fur Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universitat Ulm, Ulm, Germany.

113

114
certification with the CE mark.611 Currently, application of
the CE mark on a BG meter is a one-time procedure; regular
and independent quality controls are not mandatory after the
market approval.
In this study, the measurement quality of 12 SMBG systems
was compared in a standardized manner under controlled
laboratory conditions. The main objective was the assessment
of system accuracy following the requirements stated by DIN
EN ISO 15197:2003. In addition, reproducibility of each systems measurement results was assessed following a modified
TNO (Netherlands Organization for Applied Scientific Research) procedure.12
Materials and Methods
The study was performed between July and October 2012
at the Institut fur Diabetes-Technologie Forschungs- und
Entwicklungsgesellschaft mbH an der Universitat Ulm, Ulm,
Germany in compliance with the German Medical Devices
Act. The study protocol was approved by the Ulm University
Ethics Committee, and the applicable authority was notified.
Informed consent forms were signed by all participants prior
to study procedures.
Study population
Male and female subjects ( 18 years old) with diabetes
type 1 or type 2 and subjects without diabetes were included.
Exclusion criteria were as follows: pregnancy or lactation
period, severe acute disease, and/or severe chronic disease.
The subjects medical history and medication was reviewed
by a physician to determine any usage of interfering substances (e.g., acetaminophen, salicylates, ascorbic acid, dopamine) given in the manufacturers labeling.
BG monitoring systems for self-testing
In this study, 12 systems were evaluated (Table 1): AccuChek Aviva by Roche Diagnostics GmbH, Mannheim,
Germany; BGStar by AgaMatrix Inc., Salem, NH; Contour XT by Bayer Consumer Care AG, Basel, Switzerland;
GE100, GE200, mylife Pura, and mylife Unio by Bionime Corp., Taichung City, Taiwan; GL40 from Beurer
GmbH, Ulm; Omnitest 3 by B. Braun Melsungen AG,
Melsungen, Germany; OneTouch Verio Pro by LifeScan
Europe, Division of Cilag GmbH, Zug, Switzerland; and
Systems x and y (owing to legal reasons, the authors/study
site decided not to show brand names of two systems that
missed accuracy requirements specified by DIN EN ISO
15197:2003 [see Results]). The systems displayed plasma BG
values in mg/dL. mylife Unio was not yet introduced to the
German market. GE100 and GE200 were available on the
U.S. market. GE100, GE200, and mylife Unio were obtained
from the manufacturer (Bionime Corp.); the other nine systems were available on the German market and were purchased from a local pharmacy. These nine systems were
selected in order to assess the performance of systems of
different manufacturers and price ranges. The systems were
stored, used, and maintained in compliance with the manufacturers labeling. In order to ensure the proper function of
each tested system, control measurements according to the
manufacturers labeling were performed daily prior to the
test procedure.

FRECKMANN ET AL.
Comparison measurement
Comparison measurements were performed with the following two different methods, according to the manufacturers labeling. For systems using a glucose oxidase method to
measure reference values (Table 1), the YSI 2300 STAT Plus
glucose analyzer (YSI Life Sciences, Yellow Springs, OH) was
used. The glucose oxidase method was also used for GE100
and mylife Pura according to the manufacturers information.
For systems using a hexokinase method to measure reference
values (Table 1), the cobas c111 (Roche Diagnostics GmbH)
was used. The hexokinase method was also used for GE200
according to the manufacturers information. Trueness of the
glucose oxidase method (YSI 2300 STAT Plus) was assessed
by assaying bioanalytical glucose standards (YSI). Precision
was assessed by assaying quality controls (liquid assayed
Multiqual; Bio-Rad Laboratories GmbH, Munich, Germany). Trueness and precision of the hexokinase method
(cobas c111) were verified by assaying control material provided by the manufacturer (Precipath U/Precinorm U;
Roche Diagnostics GmbH). Additionally, regular internal and
external quality control measurements were performed, as
required by the German national standard.13
Comparison measurements were performed from capillary
plasma. Both methods used for comparison measurements
provided BG values in mg/dL.
Evaluation procedures
Test procedures were performed by clinical personnel well
trained to the limitations of the tested systems, the manufacturers labeling, the safety practices, and the test protocol. Test
procedures were carried out in a laboratory setting with
controlled room temperature (23 5C) and humidity. Each
participant washed his or her hands with soap and water and
dried them before the finger puncture and the measurement
procedure were performed. The hematocrit value of each
blood sample was checked to be within 30% and 55% (based
on the smallest range indicated in the manufacturers labeling). BG results of comparison measurements had to be within
the range of 20600 mg/dL (based on the smallest range indicated in the manufacturers labeling).
System accuracy evaluation
In this study, system accuracy evaluation was performed
following the procedures prescribed in detail in DIN EN ISO
15197:2003.4 Deviations from this standard are described in
the following section.
Each system was tested on at least 100 capillary blood
samples from different subjects over at least 10 days. Samples
were measured with one test strip lot and two meters for each
system. Test strips were taken from at least 10 different vials,
using test strips of the same vial for the two measurements of
an individual sample. The vials were changed after approximately 10 subjects. In addition, a sample for determination of
hematocrit in duplicate was collected using heparinized
capillaries. Capillaries were centrifuged, and the hematocrit
was read on an alignment chart.
DIN EN ISO 15197:2003 prescribes the distribution of the
blood samples into different BG concentration categories.
Concentration categories with the following distribution were
used: 5% had to be 50 mg/dL; 15% between >50 and

115

GOD

GOD

GOD

OneTouch
Verio Pro

System x

System y

GOD

GDH

GDH

GOD

GDH

GOD

GOD

GDH

GOD

20600

20600

20600

10600

10600

10600

20630

20600

20600

10600

20600

10600

Measurement
range (mg/dL)

1040

1040

644

1040

644

1040

1030

644

1040

545

1040

844

Temperature
(C)

2060

2060
2060

< 85
85

2060

1070

3060

2060

3055 (1070)c

3060

1090

1090

1090

1090

1085

1090

1090

1070

2060

90

1093

1065

Hematocrit
(%)

1090

Humidity
(%)

Measurement conditions

xxx-1
(December 2013)
yyy-1
(November 2013)

112823A
( June 2014)
G5MA30
( January 2014)
3299831
(November 2013)

1122161
(February 2014)
1128238
( June 2014)
X06/3
(February 2014)
1121125
( January 2014)

DP2BFEC02F
(August 2013)

490873
(August 2013)
JJ16WQ63G
(November 2013)

Lot 1 (expiry date)

xxx-2
(November 2013)
yyy-2
(November 2013)

112823B
( June 2014)
G2LK02
(November 2013)
3309296
(October 2013)

1121183
( January 2014)
1128239
( June 2014)
X05/5
(February 2014)
1117084
( June 2013)

DP2AFEC04A
( July 2013)

490866
(August 2013)
JJ17WQ65E
(November 2013)

Lot 2 (expiry date)

Test strips

Beurer GmbH, Ulm,


Germany
Bionime Corp. (Ypsomed
GmbH, Liederbach,
Germany)
Bionime Corp. (Ypsomed
GmbH)
B. Braun Melsungen AG,
Melsungen, Germany
LifeScan Europe, Division
of Cilag GmbH, Zug,
Switzerland (LifeScan
Deutschland,
Neckargemund,
Germany)

Roche Diagnostics GmbH,


Mannheim, Germany
AgaMatrix Inc., Salem,
NH (Sanofi-Aventis
Deutschland GmbH,
Frankfurt, Germany)
Bayer Consumer Care AG,
Basel, Switzerland (Bayer
Vital GmbH, Leverkusen,
Germany)
Bionime Corp., Taichung
City, Taiwan
Bionime Corp.

Manufacturer (distributor)a

The systems are listed alphabetically. Brand names of systems that missed accuracy requirements specified by DIN EN ISO 15197:2003 are not shown (Systems x and y). Comparison measurement
methods (glucose oxidase [GOD] or hexokinase [HK]), test strip enzyme (glucose dehydrogenase [GDH] or GOD), calibration, measurement range, and measurement conditions according to the
manufacturers labeling. Test strip lot 1 was used for system accuracy evaluation; test strip lot 1 and test strip lot 2 were used for evaluation of measurement reproducibility.
a
Distributor is given if different from manufacturer (according to the manufacturers labeling).
b
Obtained from the manufacturer, not purchased commercially (mylife Unio is not yet introduced into the German market; GE100 and GE200 are available on the U.S. market).
c
After the performance of the study, a hematocrit value of 1070% was specified by the manufacturer.

GOD

Omnitest 3

GOD

GL40

HK

HK

GE200b

mylife Uniob

GOD

GE100b

GOD

GOD

Contour XT

mylife Pura

GOD

GOD

GDH

GDH

HK

Accu-Chek
Aviva
BGStar

System

Comparison
method

Test
strip
enzyme

Table 1. The 12 Systems Evaluated

116
FIG. 1.

Continued.

117

FIG. 1. Difference plots of the 12 systems evaluated. Solid lines indicate criteria for system accuracy following DIN EN ISO 15197:2003. Dashed lines indicate criteria for system
accuracy following the recently published revision ISO 15197:2013. Brand names of systems that missed accuracy requirements specified by DIN EN ISO 15197:2003 are not shown
(Systems x and y).

118
80 mg/dL; 20% between >80 and 120 mg/dL; 30% between
>120 and 200 mg/dL; 15% between >200 and 300 mg/dL;
10% between >300 and 400 mg/dL; and 5% had to be
>400 mg/dL. Samples were assigned to the respective category according to the mean BG result of the respective comparison
measurement
(manufacturers
measurement
procedure).
Samples were collected from fingertips by skin puncture.
Before measurement with each system and before each sample collection for comparison measurements, residual blood
was wiped off from the finger. For BG concentrations
>50 mg/dL and 400 mg/dL only native blood samples
were used. For native samples, BG measurements were performed with up to three systems (two meters per system,
respectively) directly from fingertips. Before and after the
measurements with the systems, samples (200 lL) for comparison measurements were collected in lithium heparin
tubes.
The performance of a controlled and safe human study that
ensures the availability of sufficient native blood samples in
the lowest and highest concentration categories is very difficult. In this study, samples could be adjusted to evaluate the
systems accuracy at BG concentrations of 50 mg/dL and
>400 mg/dL. Fresh capillary blood samples designated for
adjustment were collected in lithium heparin tubes. Adjustment was performed either by incubation to allow for glycolysis or by supplementation with glucose (stock solution;
40% glucose in 0.9% NaCl). Adjusted blood samples were
gently mixed, and BG measurements were performed with up
to six systems. Immediately after the sampling and measurement procedure, the partial pressure of oxygen (pO2) in
adjusted blood samples was checked (Opti Check; OPTI
Medical Systems Inc., Roswell, GA) in order to ensure a pO2
that is comparable to the pO2 in capillary blood.14 Before and
after the measurements with the systems, samples for comparison measurements were collected.
Comparison measurements of native and adjusted blood
samples were performed after the measurements with the
systems. Therefore, samples for comparison measurements
were centrifuged, plasma was separated, and measurements
were performed immediately after centrifugation.
In order to verify sample stability, the drift between the first
and the second comparison measurement had to be 4 mg/
dL at BG concentrations 100 mg/dL and 4% at BG concentrations >100 mg/dL.
Evaluation of reproducibility
The precision of measurement is defined as the closeness of
the agreement between independent measurement results
obtained under the stipulated conditions.15
In this study, evaluation of measurement reproducibility
was performed following the TNO guideline12 with slight
modifications as follows. Ten venous blood samples were
collected in lithium heparin tubes each from one of 10 different subjects. These samples were distributed to five glucose
concentration categories: 5080 mg/dL, 80120 mg/dL, 120
200 mg/dL, 200300 mg/dL, and 300400 mg/dL, with two
samples within each category. Hematocrit values were
checked to be within the ranges given by the manufacturers.
Following the TNO guideline, a minimum of three samples
with a BG value <6.5 mmol/L ( <117 mg/dL) was required.

FRECKMANN ET AL.
Measurements were performed with two lots of test strips and
two meters per system. For each concentration category
sample 1 was measured with meter 1 and test strip lot 1;
sample 2 was measured with meter 2 and test strip lot 2. For
each sample and each system, 10 measurements were performed within 30 min. In order to assign the samples to the
respective BG concentration category, BG comparison measurements were performed before and after the measurement
with the system (as described above for system accuracy
evaluation).
Data analysis
Data management and evaluation were performed at the
study site.
System accuracy. Data were excluded from analysis if
 the drift between the first and the second comparison

measurement exceeded the specified acceptance criteria


(as mentioned above).
 the required number of samples in a BG concentration
category was already reached.
 the pO2 of adjusted samples was >100 mm Hg.
In this study, 200 results of 100 subjects were included for
each system according to DIN EN ISO 15197:2003.
System accuracy was assessed by comparison of the systems measurement results with the respective mean result of
the comparison measurement (obtained immediately before
and after the measurements with the system). At BG concentrations <75 mg/dL, the relative number of system results within 15 mg/dL, 10 mg/dL, and 5 mg/dL of the
comparison measurement was calculated. At BG concentrations 75 mg/dL, the relative number of system results
within 20%, 15%, 10%, and 5% of the comparison
measurement was calculated. Acceptability of a system was
determined by adding the number of system results within
15 mg/dL at BG concentrations <75 mg/dL to the number
of system results within 20% at BG concentrations 75 mg/
dL. The agreement between each system and the mean comparison result was plotted in a difference-plot as recommended in DIN EN ISO 15197:2003. In the difference-plot,
the deviation of a single BG system measurement from the
respective mean comparison measurement result is shown
(Fig. 1).
In addition, accuracy limits of the recently published revision
ISO 15197:20135 were applied, and the relative numbers of
system results within 15 mg/dL, 10 mg/dL, and 5 mg/
dL of the comparison measurement at BG concentrations of
<100 mg/dL and within 15%, 10%, and 5% of the comparison measurement at BG concentrations of 100 mg/dL
were calculated.
In order to assess potential clinically significant deviations,
consensus error grid analysis was performed for each system.16 For consensus error grid analysis, the agreement between individual measurement results and the respective
mean comparison result was determined. Then, the number of
a systems results within five zones of different clinical significance was counted: Zone A, no effect on clinical action;
Zone B, altered clinical action but little or no effect on clinical
outcome; Zone C, altered clinical action likely to affect clinical
outcome; Zone D, altered clinical action that could have

119

490873

Accu-Chek
Aviva
BGStar
Contour XT
GE100
GE200
GL40
mylife Pura
mylife Unio
Omnitest 3
OneTouch
Verio Pro
System x
System y

93 186/200
71.5 143/200

198/200
200/200
200/200
199/200
197/200
199/200
200/200
193/200
197/200

75 mg/dL

100
61

100
100
100
100
100
100
100
100
97

100

97
42

100
100
100
100
95
98
100
100
74

97

68
16

88
88
95
71
53
53
82
71
42

47

91
74

99
100
100
99
98
99
100
96
99

99

80
51

96
99
98
99
91
99
100
90
97

99

62
33

88
93
88
97
80
84
96
70
82

90

38
13

61
48
60
81
43
47
65
43
57

62

15 mg/dL 10 mg/dL 5 mg/dL 20% 15% 10% 5%

< 75 mg/dL

% at BG concentration of

196/200
199/200
197/200
199/200
187/200
199/200
200/200
183/200
195/200

198/200

84.5 169/200
54.5 109/200

98
99.5
98.5
99.5
93.5
99.5
100
91.5
97.5

99

Overall result
within
accuracy limits
( 15 mg/dL
and 15%)

Brand names of systems that missed accuracy requirements specified by DIN EN ISO 15197:2003 are not shown (Systems x and y).
BG, blood glucose.

xxx-1
yyy-1

99.5 199/200

JJ16WQ63G
99
DP2BFEC02F 100
1122161
100
1128238
99.5
X06/3
98.5
1121125
99.5
112823A
100
G5MA30
96.5
3299831
98.5

Test strip lot

Meter

System

Overall result
within
accuracy limits
( 15 mg/dL
and 20%)

DIN EN ISO 15197:2003

Table 2. System Accuracy

100 mg/dL

100
53

95
100
100
98
97
100
100
95
97

97

94
31

87
95
95
95
87
97
98
87
73

87

66
13

67
73
74
73
47
60
73
63
43

44

78
55

99
99
98
100
92
99
100
90
98

100

58
36

92
94
90
99
80
83
97
71
84

95

33
14

66
50
63
82
45
43
67
42
59

67

15 mg/dL 10 mg/dL 5 mg/dL 15% 10% 5%

< 100 mg/dL

% at BG concentration of

ISO 15197:2013

120
significant medical risk; and Zone E, altered clinical action
that could have dangerous consequences.
Reproducibility according to TNO quality guideline. For
evaluation of reproducibility, results of 10 subjects (one
sample per subject, 10 measurements per sample) were included for each system. For each sample, the coefficient of
variation (CV) of 10 independent measurements and the SD
were calculated. Acceptance criteria were defined as follows: the permissible CV had to be 5% for BG values
100 mg/dL, and the SD had to be 10 mg/dL for BG
values < 100 mg/dL.
Results
In this study, 10 of the 12 evaluated systems fulfilled the
accuracy requirements specified by DIN EN ISO 15197:2003
with at least 95% of the system measurement results within 15 mg/dL of the results of the manufacturers measurement procedure at BG concentrations < 75 mg/dL and
within 20% at BG concentrations 75 mg/dL (Table 2 and
Fig. 1). The 12 evaluated systems showed between 71.5% and
100% of the measurement results within the required accuracy
limits.
According to ISO 15197:2013 with the BG concentration
threshold of 100 mg/dL (instead of 75 mg/dL as described in
the ISO 15197:2003 standard), accuracy limits for BG concentrations between 75 mg/dL and 100 mg/dL are more
stringent. In this study, eight of the 12 systems showed at least
95% of the system measurement results within 15 mg/dL of
the results of the manufacturers measurement procedure at
BG concentrations of < 100 mg/dL and within 15% at BG
concentrations of 100 mg/dL (Table 2 and Fig. 1). The systems showed between 54.5% and 100% of the measurement
results within the required accuracy limits.
Consensus error grid analysis showed that all tested systems had 100% of the results within Zone A (no effect on
clinical action) and Zone B (altered clinical action but little or
no effect on clinical outcome). For six systems (Contour XT,
BGStar, Omnitest 3, GE100, GE200, and mylife Unio) 100% of
the results fell within Zone A.
Each of the 12 tested systems met the modified TNO performance criteria for reproducibility with 5% CV at BG
values of 100 mg/dL or 10 mg/dL SD for BG values of
<100 mg/dL (Table 3) for both test strip lots. CV for BG concentrations of 100 mg/dL ranged from 0.77% to 4.84%; SD
for BG concentrations of <100 mg/dL ranged from 0.88 mg/
dL to 4.33 mg/dL. Half of the tested systems reproducibility
results showed a CV <3% and an SD <3 mg/dL. For two
systems, measurement reproducibility with a CV <2% and a
SD <2 mg/dL was found (Table 3).
Discussion
In this study, system accuracy and measurement reproducibility of 12 systems for SMBG were evaluated in a standardized manner over the clinically relevant BG
concentration range. The study was performed under controlled laboratory conditions in which influencing factors
with potential impact on BG measurements were reduced to a
minimum.
The main objective was the assessment of system accuracy
following DIN EN ISO 15197:2003.4 In this study, 10 of the 12

FRECKMANN ET AL.
systems fulfilled with the evaluated test strip lot the minimum
system accuracy requirements of this norm. The 12 evaluated
systems showed between 100% and 71.5% of the measurement results within the required accuracy limits; comparable
results for some of the investigated systems have been reported in other studies.6,8,17 Considering the recently published revision ISO 15197:2013,5 the systems measurement
results within the required limits varied between 54.5% and
100%. Because many currently available SMBG systems presumably were developed and marketed in compliance with
ISO 15197:2003, a 36-month transition period is recommended in ISO 15197:2013 until mandatory compliance to
this standard is required.
Assessment of system accuracy according to criteria stated
in DIN EN ISO 15197:2003 should ensure an evaluation
procedure that follows as closely as possible the recommendations of this norm. However, the evaluation of system accuracy is complex, and the designs of most of the published
studies show remarkable variations to the recommendations
of DIN EN ISO 15197:2003.18 Evaluation procedures in our
study were performed under standardized conditions; however, several factors may contribute to variations of the systems accuracy results. A possible bias of the comparison
method and reported differences between the widely used
reference methods (glucose oxidase, hexokinase) contribute to
inaccuracies of a systems measurement results.19,20
In this study, system accuracy of only one lot of test strips of
each of the 12 systems was evaluated. As system accuracy
results can remarkably vary between different test strip
lots,7,11,21 the results of the test strip lot presented here do not
allow for an overall conclusion for a given system. In order to
consider test strip variations more closely, the recently published revision ISO 15197:2013 prescribes the performance of
system accuracy evaluation studies with at least three different lots of test strips. Nevertheless, each lot of test strips that is
distributed and available to people with diabetes has to provide sufficient measurement quality.
Adequate storage conditions of test strips are also an important aspect that potentially affects measurement quality22
and should be ensured over the entire transport chain from
the manufacturer to the pharmacies and from the pharmacies
to the patient. In this context it should be mentioned that in
this study, nine systems were purchased commercially,
whereas three systems were obtained from the manufacturer.
When interpreting accuracy results of SMBG systems it
should be taken into account that several factors potentially
affect accuracy results, and it should also be recognized that
not only a bias that is as small as possible but also high precision is important. Both the precision and the trueness of the
measurement results contribute to a systems accuracy. Systems with a high trueness can be imprecise, and vice versa.
According to DIN EN ISO 15197, system accuracy (trueness
and precision) is sufficient if 95% of single measurement results are within the funnel of the difference plot (Fig. 1). Precision is shown as the closeness of agreement between
independent test results under stipulated conditions.15 In this
study, each of the tested systems showed acceptable measurement reproducibility (precision under reproducible conditions). Other studies in which measurement reproducibility
was investigated showed comparable results.10,23 Trueness of
a system is shown as the closeness of agreement between the
average value obtained from a large series of test results and

SYSTEM ACCURACY/MEASUREMENT REPRODUCIBILITY

121

Table 3. Measurement Reproducibility


Glucose concentration range with test strip lot
5080 mg/dL
System, parameter
Accu-Chek Aviva
Comparison mean
SD (mg/dL)
CV (%)
BGStar
Comparison mean
SD (mg/dL)
CV (%)
Contour XT
Comparison mean
SD (mg/dL)
CV (%)
GE100
Comparison mean
SD (mg/dL)
CV (%)
GE200
Comparison mean
SD (mg/dL)
CV (%)
GL40
Comparison mean
SD (mg/dL)
CV (%)
mylife Pura
Comparison mean
SD (mg/dL)
CV (%)
mylife Unio
Comparison mean
SD (mg/dL)
CV (%)
Omnitest 3
Comparison mean
SD (mg/dL)
CV (%)
OneTouch Verio Pro
Comparison mean
SD (mg/dL)
CV (%)
System x
Comparison mean
SD (mg/dL)
CV (%)
System y
Comparison mean
SD (mg/dL)
CV (%)

(mg/dL)

80120 mg/dL

(mg/dL)

(mg/dL)

(mg/dL)

(mg/dL)

63.6
1.48

63.6
1.89

101.3

99.8
2.39

185.6

167.0

237.9

245.1

373.0

337.3

1.30

3.08

0.77

1.59

1.94

2.05

182.0

165.3

232.8

236.5

370.3

332.8

1.99

4.08

3.31

1.67

1.37

3.13

182.5

165.3

234.3

239.0

373.5

330.3

1.68

1.19

1.59

1.55

1.49

1.72

181.5

161.5

231.3

239.0

373.5

329.0

1.40

1.22

1.64

1.83

1.19

2.18

104.6

174.2

156.6

257.7

234.6

341.7

374.3

1.37

1.43

1.43

1.31

1.61

0.80

1.37

94.1
3.79

181.5

161.5

231.3

239.0

373.5

329.0

2.51

2.77

3.15

2.90

2.87

1.70

182.5

165.3

234.3

239.0

373.5

330.3

1.17

1.20

0.99

0.90

1.79

2.39

104.6

174.2

156.6

257.7

234.6

341.7

374.3

1.75

1.93

1.28

1.20

1.59

1.10

1.56

96.5
2.84

181.5

163.8

233.0

237.5

370.0

331.5

2.07

4.04

2.71

2.95

2.23

4.49

182.0

165.3

232.8

236.5

370.3

332.8

1.79

1.54

1.31

2.08

1.65

2.12

181.5

163.8

233.0

237.5

370.0

331.5

1.59

2.47

1.90

1.07

1.76

2.81

183.5

164.8

236.8

241.3

372.3

327.8

1.55

4.84

2.10

1.96

1.96

2.57

61.4
2.07

63.5
1.03

61.5
2.18

62.2
0.99

98.5
1.29

100.3

97.7
1.87

98.2
1.03

54.8
0.88

50.3
1.05

54.8
1.70

63.5
1.14

59.2
1.32

59.6
1.51

59.2
1.32

62.2
1.34

94.8
1.65

92.9
2.74

94.8
4.22

100.3

94.1
4.42

98.2
2.00

1.43
(mg/dL)

(mg/dL)

(mg/dL)

(mg/dL)

(mg/dL)

300400 mg/dL

1.38
(mg/dL)

200300 mg/dL

1.05
(mg/dL)

120200 mg/dL

50.3
1.75

58.1
2.18

61.4
1.10

58.1
1.07

64.7
2.08

59.6
1.37

60.2
2.83

61.5
1.58

60.2
1.81

63.3
1.35

92.9
1.78

95.5
4.33

98.5
2.42

95.5
2.33

101.0
3.07

97.7
2.49

96.5
2.06

99.0
1.95

The SD is given for comparison of measurement results of < 100 mg/dL; the coefficient of variation (CV) is given for comparison of
measurement results of 100 mg/dL. Brand names of systems that missed accuracy requirements specified by DIN EN ISO 15197:2003 are
not shown.

122
an accepted reference value.15 In daily routine, both low imprecision and low trueness bear the risk of inadequate therapeutic decisions.

FRECKMANN ET AL.

9.

Conclusions
In this study, 83% of the systems evaluated fulfilled minimum system accuracy requirements of DIN EN ISO
15197:2003. Considering the more stringent criteria of the recently published revision ISO 15197:2013, 67% of the evaluated test strip lots showed at least 95% of the values within
accuracy limits. In order to ensure reliable SMBG results,
sufficient measurement quality should be provided for each
BG meter and each test strip lot distributed on the market. We
recommend the establishment of regular independent evaluations of SMBG systems in a standardized manner.
Acknowledgments

10.

11.

12.

13.

This study was funded by a grant from Ypsomed AG,


Burgdorf, Switzerland.
14.

Author Disclosure Statement


All authors are employees of the Institut fur DiabetesTechnologie Forschungs- und Entwicklungsgesellschaft mbH
an der Universitat Ulm (IDT), Ulm, Germany. G.F. is general
manager of the IDT, which carries out studies evaluating BG
meters and medical devices for diabetes therapy on behalf of
various companies. G.F./IDT have received speakers honoraria or consulting fees from Abbott, Bayer, Menarini Diagnostics, Roche Diagnostics, Sanofi, and Ypsomed.

15.

16.

17.

References
1. Bergenstal RM, Gavin JR III: The role of self-monitoring of
blood glucose in the care of people with diabetes: report of a
global consensus conference. Am J Med 2005;118(Suppl
9A):1S6S.
2. Karter AJ, Ackerson LM, Darbinian JA, DAgostino RB Jr,
Ferrara A, Liu J, Selby JV: Self-monitoring of blood glucose
levels and glycemic control: the Northern California Kaiser
Permanente Diabetes registry. Am J Med 2001;111:19.
3. Blonde L, Karter AJ: Current evidence regarding the value
of self-monitored blood glucose testing. Am J Med 2005;
118(Suppl 9A):20S26S.
4. In Vitro Diagnostic Test SystemsRequirements for BloodGlucose Monitoring Systems for Self-Testing in Managing
Diabetes Mellitus. DIN EN ISO 15197 2003. Geneva: International Organization for Standardization, 2003.
5. In Vitro Diagnostic Test SystemsRequirements for BloodGlucose Monitoring Systems for Self-Testing in Managing
Diabetes Mellitus. ISO 15197:2013 2013. Geneva: International Organization for Standardization, 2013.
6. Freckmann G, Baumstark A, Jendrike N, Zschornack E, Kocher S, Tshiananga J, Heister F, Haug C: System accuracy
evaluation of 27 blood glucose monitoring systems according
to DIN EN ISO 15197. Diabetes Technol Ther 2010;12:221231.
7. Baumstark A, Pleus S, Schmid C, Link M, Haug C, Freckmann
G: Lot-to-lot variability of test strips and accuracy assessment
of systems for self-monitoring of blood glucose according to
ISO 15197. J Diabetes Sci Technol 2012;6:10761086.
8. Brazg RL, Klaff LJ, Parkin CG: Performance variability of
seven commonly used self-monitoring of blood glucose

18.

19.

20.

21.

22.

23.

systems: clinical considerations for patients and providers.


J Diabetes Sci Technol 2013;7:144152.
Tack C, Pohlmeier H, Behnke T, Schmid V, Grenningloh M,
Forst T, Pfutzner A: Accuracy evaluation of five blood glucose monitoring systems obtained from the pharmacy: a
European multicenter study with 453 subjects. Diabetes
Technol Ther 2011;14:18.
Kuo CY, Hsu CT, Ho CS, Su TE, Wu MH, Wang CJ: Accuracy and precision evaluation of seven self-monitoring blood
glucose systems. Diabetes Technol Ther 2011;13:596600.
Kristensen GB, Monsen G, Skeie S, Sandberg S: Standardized evaluation of nine instruments for self-monitoring of
blood glucose. Diabetes Technol Ther 2008;10:467477.
Post H, van Keulen GJ, Muller WJ, Miedema K: Portable InVitro Blood Monitor Systems for (Self)-MonitoringBlood
Glucose MonitorsParticular Requirements and Test
Methods. TNO Quality Guideline PG/TG/2001 045 2001.
Delft: TNO, 2001.
Bundesarztekammer: Richtlinie der Bundesarztekammer zur
Qualitatssicherung laboratoriumsmedizinischer Untersu rzteblatt 2008;105:341355.
chungen. Deutsches A
Higgins C: Capillary blood gases: to arterialize or not. MLO
Med Lab Obs 2008;40:4247.
Accuracy (Trueness and Precision) of Measurement Methods and ResultsPart 1: General Principles and Definitions.
ISO 5725-1. Geneva: International Organization for Standardization, 1994.
Parkes JL, Slatin SL, Pardo S, Ginsberg BH: A new consensus
error grid to evaluate the clinical significance of inaccuracies
in the measurement of blood glucose. Diabetes Care
2000;23:11431148.
Pfuetzner A, Mitri M, Musholt PB, Sachsenheimer D,
Borchert M, Yap A, Forst T: Clinical assessment of the accuracy of blood glucose measurement devices. Curr Med
Res Opin 2012;28:525531.
Thorpe GH: Assessing the quality of publications evaluating
the accuracy of blood glucose monitoring systems. Diabetes
Technol Ther 2013;15:253259.
Twomey PJ: Plasma glucose measurement with the Yellow
Springs Glucose 2300 STAT and the Olympus AU640. J Clin
Pathol 2004;57:752754.
Genter PM, Ipp E: Accuracy of plasma glucose measurements in the hypoglycemic range. Diabetes Care 1994;17:
595598.
Kristensen GB, Christensen NG, Thue G, Sandberg S:
Between-lot variation in external quality assessment of
glucose: clinical importance and effect on participant performance evaluation. Clin Chem 2005;51:16321636.
Bamberg R, Schulman K, MacKenzie M, Moore J, Olchesky
S: Effect of adverse storage conditions on performance of
glucometer test strips. Clin Lab Sci 2005;18:203209.
Philis-Tsimikas A, Chang A, Miller L: Precision, accuracy,
and user acceptance of the OneTouch SelectSimple blood
glucose monitoring system. J Diabetes Sci Technol 2011;5:
16021609.

Address correspondence to:


Guido Freckmann, MD
Institut fur Diabetes-Technologie Forschungs- und
Entwicklungsgesellschaft mbH an der Universitat Ulm
Helmholtzstrasse 20
89081 Ulm, Germany
E-mail: guido.freckmann@uni-ulm.de