See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/12955957

Reliability of a New Caries Diagnostic System Differentiating between Active and

Inactive Caries Lesions

Article  in  Caries Research · July 1999

DOI: 10.1159/000016526 · Source: PubMed

CITATIONS READS

382 1,538

3 authors:

Bente Nyvad Vita Machiulskiene

Aarhus University Lithuanian University of Health Sciences
129 PUBLICATIONS   6,243 CITATIONS    48 PUBLICATIONS   1,829 CITATIONS   

SEE PROFILE SEE PROFILE

Vibeke Baelum
Aarhus University
196 PUBLICATIONS   7,126 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Non-bactericidal therapeutic approaches to caries control View project

CPITN for periodontal epidemiology View project

All content following this page was uploaded by Vita Machiulskiene on 04 May 2015.

The user has requested enhancement of the downloaded file.

 Original Paper
Caries Res 1999;33:252–260
Reliability of a New Caries Diagnostic
System Differentiating between
Active and Inactive Caries Lesions
B. Nyvad a V. Machiulskiene c V. Baelum b
a Department of Dental Pathology, Operative Dentistry and Endodontics, and b Department of Periodontology and
Oral Gerontology, Royal Dental College, Faculty of Health Sciences, Aarhus University, Aarhus, Denmark;
c Department of Therapeutic Stomatology, Faculty of Stomatology, Kaunas Medical Academy, Kaunas, Lithuania
Key Words
Caries activity · Diagnosis · Non-cavitated caries ·
Reliability
Abstract
Current scoring systems for dental caries do not consid-
er the dynamic nature of the disease. The aims of the
present study were to describe a new set of clinical
caries diagnostic criteria which differentiate between
active and inactive caries lesions at both the cavitated
and non-cavitated levels and to evaluate the reliability
of this criteria system in a population with high caries
experience. Ten diagnostic codes were defined: 0 =
sound; 1 = active (intact); 2 = active (surface discontinu-
ity); 3 = active (cavity); 4 = inactive (intact); 5 = inactive
(surface discontinuity); 6 = inactive (cavity); 7 = filling; 8
= filling with active caries; 9 = filling with inactive caries.
Distinction between active and inactive caries lesions
was made on the basis of a combination of visual and
tactile criteria. The inter- and intra-examiner reliability
was assessed through repeated examinations of 50 chil-
dren by 2 recorders over a period of 3 years. The per-
centage agreement of caries diagnoses varied between
94.2 and 96.2%. The kappa values ranged between 0.74
and 0.85 for intra-examiner examinations and between
0.78 and 0.80 for inter-examiner examinations; 81.6% of
all misclassifications involved non-cavitated caries le-
쑕 1999 S.Karger AG, Basel
0008–6568/99/0334–0252 $17.50/0
Fax +41 61 306 12 34
E-Mail karger@karger.ch Accessible online at:
www.karger.com http://BioMedNet.com/karger
Received: February 23, 1998
Accepted after revision: September 29, 1998
sions. Disagreement between sound surfaces and non-
cavitated active or non-cavitated inactive lesions (31.3
and 31.2%, respectively) was more common than dis-
agreement between non-cavitated active and non-cavi-
tated inactive lesions (10.6%). The probability of recon-
firming a sound, non-cavitated active or non-cavitated
inactive caries lesion – given that the surface was diag-
nosed as either sound, non-cavitated active or non-cav-
itated inactive at the first examination – was 98.0, 68.7
and 72.5%, respectively. The results show that the use of
a new set of clinical caries diagnostic criteria based on
activity assessment can be performed with a high relia-
bility, even when non-cavitated diagnoses are included
in the criteria system.
The decrease in the prevalence of dental caries in chil-
dren and adolescents, which has been reported worldwide
over the past 20 years [Glass, 1982; Brunelle and Carlos,
1990; Marthaler, 1990; Marthaler et al., 1996], may require
that in the future more sensitive diagnostic criteria are em-
ployed for the recording of caries. Thus, the traditional mea-
surement of caries at the stage of cavitation, excluding pre-
cavitation stages of caries [WHO, 1997], may no longer be
sufficient to reflect changes in the incidence of caries in pre-
sent-day populations exhibiting an overall slow rate of
193.219.162.172 - 11/13/2013 9:32:03 AM
caries progression [Glass et al., 1983]. Furthermore, it has
Eur.Org.for Caries Research
Bente Nyvad
Department of Dental Pathology, Operative Dentistry and Endodontics
Royal Dental College, Faculty of Health Sciences, Aarhus University, Vennelyst Boulevard
Downloaded by:
DK-8000 Århus C (Denmark)
Tel. +45-89424074, Fax +45-86202202, E-Mail nyvad@odont.aau.dk
been shown that diagnosis of caries at the cavitation level
results in a significant underestimation of the actual caries
experience in populations [Pitts and Fyffe, 1988; Manji et
al., 1989; Ismail et al., 1992; Bjarnason et al., 1993; Kuzmi-
na et al., 1995; Sköld et al., 1995; Machiulskiene et al.,
1998].
For many years, recording of non-cavitated caries le-
sions was deliberately avoided due to the belief that it is not
possible to achieve a reliable diagnosis of precavitation
stages of caries [WHO, 1997]. However, several studies
contradict this statement [Backer-Dirks, 1966; Pitts and
Fyffe, 1988; Manji et al., 1989; Neilson and Pitts, 1991;
Ismail et al., 1992], and it has been demonstrated that the in-
ter- and intra-examiner reliability is not necessarily reduced
when non-cavitated caries lesions are included in the
recording system, provided that the examiners are thor-
oughly trained and calibrated prior to the study [Pitts and
Fyffe, 1988].
Use of a caries diagnostic system which includes non-
cavitated caries has the distinct advantage that the classical
stages of lesion formation – development of cavitation
through non-cavitated stages of caries – may be reflected
in the recordings. Caries lesion progression is a highly dy-
namic process characterized by alternating periods of disso-
lution and redeposition of minerals in the dental hard tissue
[Larsen and Bruun, 1994; Fejerskov and Clarkson, 1996].
When the outcome of these processes over time is a net loss
of mineral, a caries lesion develops or progresses [Fejer-
skov and Manji, 1990; Fejerskov, 1997]. However, when re-
deposition of mineral predominates, the result may be arrest
of lesion progression or ‘remineralization’. Clinical obser-
vations suggest that caries lesions can be arrested at any
stage of lesion development – even at the cavitation level –
provided that clinically plaque-free conditions are main-
tained [for review, see Nyvad and Fejerskov, 1997]. The
transition of an active lesion into an arrested/inactive lesion
is accompanied by characteristic changes of the surface fea-
tures of the lesion. Thus, the typical initial active caries le-
sion in the enamel exhibits a whitish opaque appearance
with a rough surface [Holmen et al., 1987], whereas the ac-
tive lesion of root/dentin is soft or leathery and discoloured
[Nyvad and Fejerskov, 1986]. As lesions transform into an
inactive stage they acquire a smooth/hard surface. The clin-
ical distinction between active and arrested caries has been
supported by several histological and histochemical studies
[reviewed by Nyvad and Fejerskov, 1997].
In recent years there has been an increased interest
in studying the effect of various non-operative/preventive
interventions of caries. Recording of these phenomena
requires a diagnostic system which reflects the dynamic na-
Reproducibility of Caries Diagnoses
ture of caries at all stages of lesion progression. The aim of
the present study was therefore (1) to describe a set of clin-
ical caries diagnostic criteria which differentiate between
active and inactive caries lesions at both the cavitated and
non-cavitated levels, (2) to assess the inter- and intra-exam-
iner reliability of this criteria system over 3 years in a pop-
ulation of children with a high caries experience, and (3) to
compare the degree of agreement obtained with the new cri-
teria system with that of other commonly used diagnostic
systems such as the WHO criteria [WHO, 1997].
Materials and Methods
Caries Diagnostic Criteria
The caries diagnostic criteria were developed on the basis of in-
formation from the literature as well as on personal experience with
clinical caries diagnosis. A detailed description and visualization of
the criteria is presented in table 1 and figure 1, respectively.
Active and inactive caries lesions were distinguished on the basis
of a combination of visual and tactile criteria (table 1). The assess-
ment was carried out at three levels of increasing severity, depending
on the depth of penetration of the lesions (intact surface, surface dis-
continuity in enamel or manifest cavity in dentin). Explorers were
used to gently clean the tooth surface from bacterial deposits and to
check for loss of tooth structure (cavitation) and surface texture (hard
or rough/soft/leathery). Probing of lesions was deliberately avoided
unless plain visual criteria (e.g. opaque versus shiny) were not suffi-
cient to assign a lesion into the active or inactive category. As surface
texture is considered a more reliable indicator of activity than colour
[Beighton et al., 1993]; colour was never used as the sole diagnostic
criterium. ‘Mixed’ lesions containing elements of both active and in-
active caries were diagnosed as active.
Subjects and Examinations
The subjects were selected among a group of 889 9-14-year-old
children with a high caries prevalence living in the city of Kaunas,
Lithuania [Machiulskiene et al., 1998]. These children participated in
a clinical caries trial and were available for repeated caries examina-
tions for 3 consecutive years (1994–1996). Each year, 50 children
were selected for assessment of inter- and intra-examiner repro-
ducibility. A total of 7 sets of duplicate caries recordings were col-
lected during the study period.
The caries examinations were carried out independently by 2 of
the authors (B.N. and V.M.) using the criteria described in table 1. Pri-
or to the study the examiners had been extensively calibrated through
discussions and practical exercises for a period of 1 month. Immedi-
ately before the examinations, each child cleaned his or her teeth with
a toothbrush to remove gross masses of plaque. The caries examina-
tions were conducted under standardized conditions using the dental
chairs available in school dental offices, and a portable dental unit
equipped with fibre-optic operating light, compressed air, and a suc-
tion device. All permanent teeth were recorded at the surface level
using cotton rolls, plane mouth mirrors and standard explorers after
drying the teeth with a blast of compressed air for 3–5 s per surface.
In addition to the codes shown in table 1, specific codes were includ-
193.219.162.172 - 11/13/2013 9:32:03 AM
ed for extracted teeth and unerupted teeth/surfaces. Tooth surfaces
Eur.Org.for Caries Research
Caries Res 1999;33:252–260 253
Downloaded by:
 Table 1. Description of the new caries diagnostic criteria

Score Category Criteria

0 Sound
1 Active caries
(intact surface)
2 Active caries
(surface discontinuity)
3 Active caries
(cavity)
4 Inactive caries
(intact surface)
5 Inactive caries
(surface discontinuity)
6 Inactive caries
(cavity)
7 Filling (sound surface)
8 Filling + active caries
9 Filling + inactive caries
Normal enamel translucency and texture (slight staining allowed in otherwise
sound fissure).
Surface of enamel is whitish/yellowish opaque with loss of luster;
feels rough when the tip of the probe is moved gently across the surface;
generally covered with plaque. No clinically detectable loss of substance.
Smooth surface: Caries lesion typically located close to gingival margin.
Fissure/pit: Intact fissure morphology; lesion extending along the walls of the
fissure.
Same criteria as score 1. Localized surface defect (microcavity) in enamel only.
No undermined enamel or softened floor detectable with the explorer.
Enamel/dentin cavity easily visible with the naked eye; surface of cavity feels soft
or leathery on gentle probing. There may or may not be pulpal involvement.
Surface of enamel is whitish, brownish or black. Enamel may be shiny and feels
hard and smooth when the tip of the probe is moved gently across the surface.
No clinically detectable loss of substance.
Smooth surface: Caries lesion typically located at some distance from gingival
margin.
Fissure/pit: Intact fissure morphology; lesion extending along the walls of the
fissure.
Same criteria as score 4. Localized surface defect (microcavity) in enamel only.
No undermined enamel or softened floor detectable with the explorer.
Enamel/dentin cavity easily visible with the naked eye; surface of cavity may be
shiny and feels hard on probing with gentle pressure. No pulpal involvement.
Caries lesion may be cavitated or non-cavitated.
Caries lesion may be cavitated or non-cavitated.

with two or more lesions were classified on the basis of the most
severe of these lesions (active lesion c inactive lesion; manifest cav-
ity c surface discontinuity c non-cavitated lesion). Intra-examiner
examinations were repeated with an interval of 1–2 weeks. The time
spent per examination of each child was approximately 5–8 min.
Assessment of Reliability
The intra- and inter-examiner reliability of the caries diagnostic
criteria was assessed at the tooth surface level using five different di-
agnostic thresholds, as outlined in table 2. The cut-off points were the
following: sound versus diseased (diseased including all visible signs
of caries); active versus not active (cavitated and non-cavitated le-
sions pooled); cavity level (non-cavitated caries and caries with dis-
continuity recorded as sound); discontinuity level (non-cavitated
caries recorded as sound), and cavity level according to WHO (non-
cavitated caries and cavities with hard floors recorded as sound). The
results were expressed as percentage agreement and Cohen’s kappa.
The probability of confirming a sound or non-cavitated active or non-
cavitated inactive caries diagnosis at repeated examination was as-
sessed by examining the distribution of pooled diagnoses (1994–
1996) from the intra- and inter-examiner examinations. The group of
disagreements was analysed separately, and the percentage distribu-
tion of misclassified non-cavitated active and inactive caries lesions
was calculated.
Results
The percentage of agreement and the kappa values for
the different intra- and inter-examiner examinations are
shown in tables 3 and 4, respectively. When applying the
new diagnostic criteria (ten diagnostic categories) the per-
centage agreement of the caries diagnoses varied between
94.2 and 96.2%. The kappa values ranged between 0.74 and
0.85 for intra-examiner examinations and between 0.78 and
0.80 for inter-examiner examinations. The high level of
reliability was maintained over the 3-year period (kappa =
0.74–0.84, table 3; examiner V.M.).
When kappa values were calculated on the basis of the
same data sets using the alternative diagnostic thresholds
(two diagnostic categories; table 2), the kappa values varied
somewhat more than would be expected on the basis of the
percentage agreement, which remained high at all diagnos-
tic thresholds (tables 3, 4). The most dramatic change in
kappa was observed when the diagnostic threshold was
changed from ‘sound versus diseased’ to ‘active versus in-
193.219.162.172 - 11/13/2013 9:32:03 AM

active’ in the new criteria system.

Eur.Org.for Caries Research

254 Caries Res 1999;33:252–260 Nyvad/Machiulskiene/Baelum

Downloaded by:
 Fig. 1. Clinical characteristics of the caries diagnostic criteria. A detailed description of the individual diagnostic
codes is given in table 1.
193.219.162.172 - 11/13/2013 9:32:03 AM
Eur.Org.for Caries Research

Reproducibility of Caries Diagnoses Caries Res 1999;33:252–260 255

Downloaded by:
Table 2. Overview of the diagnostic thresholds used for the calculation of Cohen’s kappa

New diagnostic criteria WHO

sound vs. diseased active vs. not active discontinuity level cavity level cavity level

– Sound Sound Sound Sound Sound

Inactive, surface intact Active, surface intact Active, intact Active, intact
Inactive, surface Inactive, surface intact Active, surface Inactive, intact
discontinuity Filled discontinuity Inactive, surface
Inactive, cavity Filled + inactive lesion Inactive, intact discontinuity
Filled Inactive, surface Inactive, cavity
Filled + inactive lesion discontinuity Filled
Filled Filled + inactive lesion
Filled + inactive lesion
+ Active, surface intact Active, surface intact Active, surface Active, cavity Active, surface
Active, surface Active, surface discontinuity Inactive, cavity discontinuity
discontinuity discontinuity Inactive, surface Filled + active lesion Active, cavity
Active, cavity Active, cavity discontinuity Filled + active lesion
Inactive, surface intact Filled + active lesion Active, cavity
Inactive, surface Inactive, cavity
discontinuity Filled + active lesion
Inactive, cavity
Filled
Filled + active lesion
Filled + inactive lesion

– and + represent negative and positive diagnoses, respectively.

Table 5 shows the distribution of the intra-examiner data
for examiner V.M., pooled over the 3 years (1994–1996).
Given that a tooth surface was recorded as sound at one
examination, the probability of diagnosing the same surface
as sound at the other examination was 98.0%
[=100 E 2E14,486/(14,764+14,786)]. The corresponding
figures for non-cavitated active lesions and non-cavitated
inactive lesions were 68.7 and 72.5%, respectively. The
probability of reconfirming a non-cavitated caries diagnosis
was moderately higher for occlusal surfaces than for
smooth surfaces (data not shown). The results for the inter-
examiner data and for the intra-examiner data for examiner
B.N. were essentially similar (data not shown).
The total number of misclassifications in the pooled data
set (table 5) amounted to 821 (= 17,164–14,486–398–521–
232–9–556–73–68); 670 of the misclassifications (81.6%)
involved the two non-cavitated diagnoses (tables 5, 6). A
total of 257 misclassifications (31.3%) concerned disagree-
ment between sound surfaces and non-cavitated active le-
sions, 256 misclassifications (31.2%) concerned disagree-
ment between sound surfaces and non-cavitated inactive
ments between active and inactive non-cavitated lesions.
The remaining 221 misclassifications involved other com-
binations (tables 5, 6). Table 6 shows that the distribution
of the misclassifications was rather similar for the pooled
inter-examiner data and for the intra-examiner data for ex-
aminer B.N.
Discussion
It is a fundamental premise for both clinical and epide-
miological studies of dental conditions that diagnostic
methods exist which may provide consistent and standard-
ized expressions of the condition in question. This require-
ment places emphasis on the issue of reliability of the diag-
nostic methods available. Thus, the purpose of the present
study was to evaluate the reliability a new set of clinical
caries diagnostic criteria which provides information about
the activity of caries lesions in addition to distinguishing
between cavitated and non-cavitated stages of caries lesion
formation.
193.219.162.172 - 11/13/2013 9:32:03 AM

lesions, and 87 misclassifications (10.6%) were disagree-

Eur.Org.for Caries Research

256 Caries Res 1999;33:252–260 Nyvad/Machiulskiene/Baelum

Downloaded by:
Table 3. Percentage agreement and kappa values for the intra-examiner examinations

Intra-examiner V.M. 1994 Intra-examiner V.M. 1995 Intra-examiner V.M. 1996 Intra-examiner B.N. 1996
(ns = 5,510) (ns = 5,450) (ns = 6,204) (ns = 6,118)

New diagnostic criteria % agreement 95.5 % agreement 94.2 % agreement 95.6 % agreement 96.2
(ten categories) kappa 0.84 kappa 0.74 kappa 0.83 kappa 0.85
Diagnostic threshold – + – + – + – +
Sound vs. diseased – 4,594 74 4,685 107 5,207 119 5,190 73
(two categories) + , 86 756 ,116 542 , 76 802 , 77 778
% agreement 97.1 % agreement 95.9 % agreement 96.9 % agreement 97.5
kappa 0.89 kappa 0.81 kappa 0.87 kappa 0.90
Active vs. inactive – 5,065 45 5,109 75 5,812 92 5,714 58
(two categories) + , 95 305 , 86 180 , 61 239 , 71 275
% agreement 97.5 % agreement 97.0 % agreement 97.5 % agreement 97.9
kappa 0.80 kappa 0.68 kappa 0.74 kappa 0.80
Discontinuity level – 5,362 15 5,258 22 6,095 14 5,897 16
(two categories) + , 22 111 , 47 123 , 6 89 , 42 163
% agreement 99.3 % agreement 98.7 % agreement 99.7 % agreement 99.1
kappa 0.85 kappa 0.77 kappa 0.90 kappa 0.84
Cavity level – 5,406 14 5,341 10 6,154 6 6,006 11
(two categories) + , 17 73 , 25 74 , 3 41 , 16 85
% agreement 99.4 % agreement 99.4 % agreement 99.9 % agreement 99.6
kappa 0.82 kappa 0.81 kappa 0.90 kappa 0.86
WHO cavity level – 5,369 16 5,263 23 6,098 13 5,908 15
(two categories) + , 19 106 , 46 118 , 7 86 , 42 153
% agreement 99.4 % agreement 98.7 % agreement 99.7 % agreement 99.1
kappa 0.86 kappa 0.77 kappa 0.89 kappa 0.84
The data are given for the new diagnostic system (ten categories) as well as for different diagnostic thresholds collapsing the ten categories
into two. – and + represent negative and positive diagnoses, respectively.

The results of the present study demonstrate that when
the new criteria are applied to the site-specific diagnosis of
caries lesions under real-life conditions in a clinical trial
they can be reproduced by the same examiner or by another
examiner to the extent of almost perfect agreement. This
conclusion applies both when evaluated in terms of the per-
centage agreement of diagnoses, which always exceeded
94%, and in terms of the value of the kappa statistics, which
essentially always exceeded 0.70, and usually exceeded
0.80, indicating almost perfect agreement [Landis and
Koch, 1977]. Moreover, the kappa values reported here are
in the same order of magnitude as those reported by other
researchers who have included non-cavitated caries diag-
noses into the criteria system [Pitts and Fyffe, 1988; Manji
et al., 1989; Ismail et al., 1992].
Use of the kappa statistics to evaluate the reliability of a
given diagnostic method has the advantage over the value
agreement that is expected as a result of chance. This said,
it is also clear that the kappa statistics has several draw-
backs. One of these is that the kappa value depends on the
underlying true, but unknown, prevalence of the condition
being diagnosed [Altman, 1991]. This may be illustrated by
the results in tables 3 and 4 which show that with approxi-
mately the same proportional agreement (95.9–99.9%), the
kappa values varied considerably more (0.68–0.90). The
consequence of this prevalence dependence is that poten-
tially misleading conclusions may be drawn if comparisons
are made of the kappa values obtained in different studies
where the caries prevalence rates are likely to be different.
For the same reason it may be misleading to make use of a
comparison of the different kappa values obtained for dif-
ferent caries diagnostic thresholds within a given study to
determine which cut-point yields the highest reliability
[Pitts and Fyffe, 1988]. Therefore, the only tenable conclu-
193.219.162.172 - 11/13/2013 9:32:03 AM

for percentage agreement in that it takes into account the sion that may be drawn on the basis of the results presented
Eur.Org.for Caries Research

Reproducibility of Caries Diagnoses Caries Res 1999;33:252–260 257

Downloaded by:
 Table 4. Percentage agreement and kappa values for the inter-examiner examinations

Inter-examiner 1994 Inter-examiner 1995 Inter-examiner 1996

(ns = 4,884) (ns = 5,598) (ns = 5,790)

New diagnostic criteria % agreement 94.9 % agreement 95.7 % agreement 94.6

(ten categories) kappa 0.80 kappa 0.80 kappa 0.78
Diagnostic threshold – + – + – +
Sound vs. diseased – 4,144 92 4,880 126 4,890 99
(two categories) + , 63 585 , 46 546 ,112 689
% agreement 96.8 % agreement 96.9 % agreement 96.4
kappa 0.86 kappa 0.85 kappa 0.85
Active vs. inactive – 4,486 65 5,287 69 5,350 89
(two categories) + , 65 268 , 40 202 , 84 267
% agreement 97.3 % agreement 98.1 % agreement 97.0
kappa 0.79 kappa 0.78 kappa 0.74
Discontinuity level – 4,628 26 5,403 29 5,529 37
(two categories) + , 36 194 , 23 143 , 36 188
% agreement 98.7 % agreement 99.1 % agreement 98.7
kappa 0.86 kappa 0.84 kappa 0.83
Cavity level – 4,736 15 5,485 2 5,650 13
(two categories) + , 17 116 , 12 99 , 15 112
% agreement 99.3 % agreement 99.7 % agreement 99.5
kappa 0.88 kappa 0.93 kappa 0.89
WHO Cavity level – 4,630 26 5,407 29 5,547 35
(two categories) + , 34 194 , 25 137 , 31 177
% agreement 98.8 % agreement 99.0 % agreement 98.9
kappa 0.86 kappa 0.83 kappa 0.84
The data are given for the new diagnostic system (ten categories) as well as for different diagnostic thresholds col-
lapsing the ten categories into two. – and + represent negative and positive diagnoses, respectively.

in tables 3 and 4 is that the present study provides no evi-
dence that the reliability of a simple caries recording system
in which only cavitated caries lesions are included would be
substantially higher than the reliability of more elaborate
and sensitive caries diagnostic systems. This conclusion
corroborates the observations by Pitts and Fyffe [1988] and
Ismail et al. [1992].
A second drawback of the use of the kappa statistics to
evaluate diagnostic reliability is that the kappa statistic is
rather insensitive to the type of examiner disagreements, i.e.
whether they are systematic or not. However, such know-
ledge could be important, since systematic deviations may
indicate a diagnostic slide (in case of systematic intra-ex-
aminer disagreements) or the use of different diagnostic
thresholds (in case of systematic inter-examiner disagree-
ments). In the present study, there were no indications of
such systematic inter- or intra-examiner deviations.
The third drawback of the kappa statistics as a measure
of diagnostic reliability is that the kappa value depends on
the number of categories in the diagnostic system. Tables 3
and 4 thus illustrate that the kappa values for the original
diagnostic system with ten categories were generally lower
– although remaining within the range of good to excellent
agreement – than the kappa values for the same data sets
obtained when collapsing the ten categories into only two
(‘sound versus diseased’).
In spite of these shortcomings, we may, in concert with
Altman [1991], conclude that the kappa statistic remains the
best approach to assessing the reliability of caries diagnos-
tic criteria, when the scope is to evaluate the reliability of
the site-specific caries diagnoses. However, it should be
recognized that no single value of kappa exists which can be
regarded universally as indicating good agreement.
As previously noted the kappa value cannot be used to
193.219.162.172 - 11/13/2013 9:32:03 AM

disclose whether disagreements are concentrated around

Eur.Org.for Caries Research

258 Caries Res 1999;33:252–260 Nyvad/Machiulskiene/Baelum

Downloaded by:
 Table 5. Distribution of diagnoses at the intra-examiner examinations (V.M.) in 1994–1996 (pooled data)

Exam 1 Exam 2
Sound non-cavitated cavitated fillings fillings total
active inactive active inactive active inactive

Sound 14,486 138 133 7 0 22 0 0 14,786

Non-cavitated
Active , 119 398 56 5 0 2 4 0 , 584
Inactive , 123 31 521 16 2 5 0 2 700
Cavitated
Active , 21 6 16 232 1 8 3 0 , 287
Inactive , 0 0 4 3 9 0 0 0 , 16
Fillings , 14 0 4 3 0 556 5 9 , 591
Fillings
Active , 0 1 0 2 0 12 73 7 , 95
Inactive , 1 0 3 0 0 24 9 68 , 105
Total 14,764 574 737 268 12 629 94 86 17,164

Table 6. Total numbers and percentage distribution of misclassifications recorded at the seven intra- and inter-examiner examinations
1994–1996 (pooled data)

Duplicate Disagreements Disagreements Disagreements involving non-cavitated Other

assessments n involving lesions, n disagreements, n
n non-cavitated
sound/active sound/inactive active/inactive
lesions, n

Intra-examiner V.M. 17,164 821 670 (81.6%)a 257 (31.3%) 256 (31.2%) 87 (10.6%) 221 (26.9%)
Intra-examiner B.N. 6,118 235 175 (74.4%) 55 (23.4%) 69 (29.4%) 25 (10.6%) 86 (36.6%)
Inter-examiner
V.M./B.N. 16,272 783 653 (83.4%) 194 (24.8%) 289 (36.9%) 63 (8.0%) 237 (30.3%)

a Percentages are based on the total number of misclassifications.

specific categories of diagnoses. A closer look at the data
sets from the present study revealed that most of the mis-
classifications (approximately 80%) involved disagreement
between sound surfaces and non-cavitated caries lesions
(active or inactive). Therefore, a detailed analysis of the dis-
tribution of misclassifications within this particular catego-
ry was performed (table 5). Although most of these misclas-
sifications concerned the distinction between sound and
non-cavitated active lesions or sound and non-cavitated
inactive lesions, the probability of confirming the diagnosis
‘sound’, ‘non-cavitated active’, or ‘non-cavitated inactive’
was 98, 69 and 73%, respectively. These figures are in the
same order of magnitude as those reported by Ismail et al.
ly, these findings contradict the general opinion [WHO,
1997] that the reproducibility of diagnostic systems includ-
ing non-cavitated caries lesions is poor in clinical settings.
Disagreements between sound surfaces and non-cavitat-
ed active lesions and sound surfaces and non-cavitated
inactive lesions occurred with about the same frequency
(about 30%). Only about 10% involved disagreement
between non-cavitated active and non-cavitated inactive
lesions. This pattern of distribution shows that the new
diagnostic criteria are effective in separating active lesions
from inactive lesions. However, non-cavitated caries le-
sions, being active or inactive, are difficult to differentiate
from sound surfaces. Ismail et al. [1992] have previously
193.219.162.172 - 11/13/2013 9:32:03 AM

[1992] for incipient lesions in pits and fissures. Collective- drawn attention to this diagnostic problem after observing
Eur.Org.for Caries Research

Reproducibility of Caries Diagnoses Caries Res 1999;33:252–260 259

Downloaded by:
that trained examiners misclassified incipient carious le-
sions 17 times more likely as sound rather than cavitated
caries. While it could be argued that professional plaque re-
moval immediately prior to the clinical examination might
be helpful in reducing the number of misclassifications, it
should be borne in mind that such strategies may not neces-
sarily be feasible with all types of caries epidemiological
studies. Others have suggested that the use of magnification
lenses [Ismail et al., 1992] may lead to fewer misclassifica-
tions of non-cavitated caries lesions, but in any case the ef-
fectiveness of either of these approaches remains unclear.
This study has shown that caries diagnosis based on ac-
tivity assessment can be performed with a high reliability,
even when non-cavitated caries diagnoses are included in
the scoring system. However, diagnosis of non-cavitated
caries lesions, with or without activity assessment, in-
evitably results in misclassifications. It cannot be conclud-
ed from the present data whether the new diagnostic criteria
are suitable for monitoring changes in the activity of caries
lesions over time. Longitudinal clinical studies are present-
ly in progress in order to address this issue.

References

Altman DG: Practical Statistics for Medical Re-
search. London, Chapman & Hall, 1991, pp
396–409.
Backer-Dirks O: Posteruptive changes in dental
enamel. J Dent Res 1966;45:503–511.
Beighton D, Lynch E, Heath MR: A microbiologi-
cal study of primary root-caries lesions with
different treatment needs. J Dent Res 1993;72:
623–629.
Bjarnason S, Finnbogason SY, Holbrook P, Köhler
B: Caries experience in Icelandic 12-year-old
urban children between 1984 and 1991. Com-
munity Dent Oral Epidemiol 1993;21:194–
197.
Brunelle JA, Carlos JP: Recent trends in dental
caries in US children and the effect of water
fluoridation. J Dent Res 1990;69(special is-
sue):723–727.
Fejerskov O: Concepts of dental caries and their
consequences for understanding the disease.
Community Dent Oral Epidemiol 1997;25:
5–12.
Fejerskov O, Clarkson BH: Dynamics of caries le-
sion formation; in Fejerskov O, Ekstrand J,
Burt BA (eds): Fluoride in Dentistry, ed 2.
Copenhagen, Munksgaard, 1996, pp 187–213.
Fejerskov O, Manji F: Risk assessment in dental
caries; in Bader JD (ed): Risk Assessment in
Dentistry. Chapel Hill, University of North
Carolina Dental Ecology, 1990, pp 215–217.
Glass RL: The First International Conference on
the Declining Prevalence of Dental Caries. J
Dent Res 1982;61(special issue):1301–1388.
Glass RL, Peterson JK, Bixler D: The effects of
changing caries prevalence and diagnostic cri-
teria on clinical caries trials. Caries Res 1983;
17:145–151.
Holmen L, Thylstrup A, Årtun J: Surface changes
during the arrest of active enamel caries le-
sions in vivo. A scanning electron microscope
study. Acta Odontol Scand 1987;45:383–390.
Ismail AI, Brodeur J-M, Gagnon P, Payette M, Pi-
card D, Hamalian T, Oliver M, Eastwood BJ:
Prevalence of non-cavitated and cavitated car-
ious lesions in a random sample of 7–9-year-
old schoolchildren in Montreal, Quebec. Com-
munity Dent Oral Epidemiol 1992;20:250–
255.
Kuzmina IN, Kuzmina E, Ekstrand KR: Dental
caries among children from Solntsevsky – a
district in Moscow, 1993. Community Dent
Oral Epidemiol 1995;23:266–270.
Landis JR, Koch GG: The measurement of observ-
er agreement for categorical data. Biometrics
1977;33:159–174.
Larsen MJ, Bruun C: Caries chemistry and fluo-
ride – mechanisms of action; in Thylstrup A,
Fejerskov O (eds): Textbook of Clinical Cari-
ology, ed 2. Copenhagen, Munksgaard, 1994,
pp 231–257.
Machiulskiene V, Nyvad B, Baelum V: Prevalence
and severity of dental caries in 12-year-old
children in Kaunas, Lithuania 1995. Caries
Res 1998;32:175–180.
Manji F, Fejerskov O, Baelum V: Pattern of dental
caries in an adult rural population. Caries Res
1989;23:55–62.
Marthaler TM: Caries status in Europe and predic-
tions of future trends. Caries Res 1990;24:
381–396.
Marthaler TM, O’Mullane DM, Vrbic V: The
prevalence of dental caries in Europe 1990–
1995. Caries Res 1996;30:237–255.
Neilson A, Pitts NB: The clinical behavior of free
smooth surface carious lesions monitored over
2 years in a group of Scottish children. Br Dent
J 1991;171:313–318.
Nyvad B, Fejerskov O: Active root surface caries
converted into inactive caries as a response to
oral hygiene. Scand J Dent Res 1986;94:281–
284.
Nyvad B, Fejerskov O: Assessing the stage of
caries lesion activity on the basis of clinical
and microbiological examination. Community
Dent Oral Epidemiol 1997;25:69–75.
Pitts NB, Fyffe HE: The effect of varying diagnos-
tic thresholds upon clinical caries data for a
low prevalence group. J Dent Res 1988;67:
592–596.
Sköld UM, Klock B, Rasmusson C-G, Torstensson
T: Is caries prevalence underestimated in to-
day’s caries examination? Swed Dent J 1995;
19:213–217.
WHO: Oral Health Surveys: Basic Methods, ed 4.
Geneva, World Health Organization, 1997.
193.219.162.172 - 11/13/2013 9:32:03 AM
Eur.Org.for Caries Research

260 Caries Res 1999;33:252–260 Nyvad/Machiulskiene/Baelum

Downloaded by:

View publication stats