Académique Documents
Professionnel Documents
Culture Documents
F.M. Mendes b
a
Universidade Cruzeiro do Sul and b Department of Pediatric Dentistry, School of Dentistry, University of
São Paulo, São Paulo, Brazil; c Department of Dentistry, Schulich School of Medicine and Dentistry, University of
Mendes
tively. For the radiographic method, a score of 1 was the cutoff [Jablonski-Momeni et al., 2008; Rodrigues et al., 2008; Braga et
point for the NC threshold, and a score of 2 was the cutoff value al., 2009; Diniz et al., 2009], and the values of the radiographic
for the D3 threshold. For the fluorescence-based methods, the method were calculated based on a systematic review [Bader et al.,
cutoff points are described below. 2002]. Subsequently, with the estimation of the number of true-
The sensitivity, specificity, and accuracy values and the 95% positive, true-negative, false-positive, and false-negative results,
confidence intervals (95% CI) were calculated for each method at these values were considered in all teeth which were not subjected
the NC threshold. The examined teeth are clustered because we to operative intervention. In the teeth which opening with bur
opted to choose more than 1 tooth per subject. Therefore, we em- was performed, the actual results were used. As the reference
ployed an approach previously described [Rao and Scott, 1992; standard was related to visual inspection and radiographic meth-
Durkalski et al., 2003] to calculate sensitivity, specificity, and ac- ods, the correction at the D3 threshold was performed only for the
curacy values and 95% CI. Receiver operating characteristics fluorescence-based methods.
(ROC) analyses were also performed and the area under the ROC For the NC threshold, the estimated sensitivity and specificity
curve (Az) was calculated. The first assessments of the examiners values of the reference standard method were 0.796 and 0.799, re-
were used to calculate the parameters. Since visual inspection was spectively. For the D3 threshold, the sensitivity of the reference
employed as the reference standard method, the performance and standard method employed in the present study was 0.786, and
comparisons were carried out only for radiographic and fluores- the specificity was 0.995. Then, after estimation of the sensitivity
cence-based methods. The best cutoff points for the fluorescence- and specificity of the reference standard methods, the corrections
based methods were obtained through ROC analysis, considering were performed using the formulas described.
the sum of specificity and sensitivity. For the LFpen method values
higher than 4 were considered NC lesions, and for the FC the equiv-
alent cutoff point was 1.1. Az values were compared among the
methods using a nonparametric approach, and sensitivity, speci- Results
ficity and accuracy values were compared using the McNemar test.
At the D3 threshold, the same parameters and comparisons With regard to interexaminer reproducibility, all
were performed. The cutoff point for the LFpen method to detect methods presented similar ICC values when considering
D3 lesions, obtained by ROC analysis, was 34, and for the FC the the entire values of the scores or of the devices. All figures
best cutoff point was 1.4. All analyses were carried out using ap-
propriate software (MedCalc 9.3.0.0; Mariakerke, Belgium). For were around 0.9, showing good interexaminer agree-
all statistical analyses, the level of significance was p ! 0.05. ment. Intraexaminer ICC values were higher than inter-
Since imperfect reference standard methods were employed, examiner ones, except for the visual inspection method.
adjustment of sensitivity and specificity values was performed us- However, all values were higher than 0.9 (table 1).
ing an approach previously proposed [Brenner, 1996]. The correc- When the collapsed results were considered, both fluo-
tions were performed using the formula assuming independence
of classification by diagnostic and reference standard methods. rescence-based methods presented similar interexamin-
The formulas are: er kappa values which were higher than those for visual
P q Sers q Sedm 1 P
q 1 Sprs
q 1 Spdm
inspection in detecting NC caries lesions. Nevertheless,
Secor for intraexaminer agreement, FC presented a higher kap-
P q Sers 1 P
q 1 Sprs
Interexaminer (n = 407)
LFpen 0.918 (0.901–0.932) 0.759 (0.694–0.825) 0.810 (0.733–0.887)
FC 0.894 (0.872–0.912) 0.786 (0.724–0.847) 0.791 (0.706–0.876)
Visual inspection 0.895 (0.874–0.913) 0.695 (0.576–0.813) 1.000 (1.000–1.000)
Radiographic 0.883 (0.860–0.903) 0.868 (0.778–0.959) 0.892 (0.806–0.978)
Intraexaminer (n = 97)
LFpen 0.986 (0.979–0.990) 0.790 (0.667–0.913) 0.863 (0.709–1.000)
FC 0.948 (0.924–0.965) 0.856 (0.732–0.979) 0.942 (0.828–1.000)
Visual inspection 0.901 (0.856–0.933) 0.795 (0.395–1.000) 1.000 (1.000–1.000)
Radiographic 1.000 (1.000–1.000) 1.000 (1.000–1.000) 1.000 (1.000–1.000)
a
Calculated using the values of the scores or of the devices.
b
Calculated after division into sound vs. NC + D3 according to the described cutoff points (LFpen: sound,
0–4; NC lesions, >4; FC: sound, 0–1.1; NC lesions, >1.1).
c Calculated after division into sound + NC vs. D3 according to the described cutoff points (LFpen: sound,
0–34; dentine caries lesion, >34; FC: sound, 0–1.4; dentine lesions, >1.4).
With regard to the performance of the methods with cence methods were also similar, but visual inspection
no correction, at the NC threshold both fluorescence- and radiographic methods showed significantly higher
based methods presented higher sensitivities but lower specificity values. There were no significant differences
specificities than the radiographic method. Sensitivity between visual inspection and radiographic methods
values were statistically significantly higher for the LFpen concerning sensitivity and specificity. The accuracy val-
method than for the FC method. Accuracy and Az values ues of both fluorescence-based methods were around 0.9,
were also significantly higher for the LFpen than for other with no significant differences, but they were significant-
methods, and the FC method presented higher accuracy ly lower than the accuracies of the visual and radiograph-
and Az values than did the radiographic method (table 2). ic methods (10.95). Concerning the Az, there was no sig-
At the D3 threshold, 25 teeth were considered carious nificant difference among both fluorescence-based meth-
using visual inspection and/or the radiographic method. ods and visual inspection, but the radiograph presented
From these teeth, 14 were considered to have dentine car- significantly lower values of Az (table 3). The values ob-
ies lesions by both methods (after operative intervention, tained by 2 examiners followed the same trends with all
1 had only enamel caries lesions, 11 had initial dentine of the methods in both the NC and D3 thresholds.
caries lesions, and 2 had deep dentine caries lesions). Four When correction of the sensitivity and specificity val-
teeth were classified as carious only by the radiographic ues was performed considering the imperfect reference
method (1 had enamel caries lesions and 3 had initial den- standard method it was observed at the NC threshold that
tine caries lesions after operative intervention), and 7 there were little changes in the sensitivity values for all
teeth were classified as carious only by visual inspection methods, but there was a significant decrease in the spec-
(2 with enamel caries lesions and 5 with initial caries le- ificity values. The decrease in sensitivities for all methods
sions). Thus, of 407 teeth, 386 were considered sound was around 2%. On the other hand, the specificities of
(94.8%) and 21 (5.2%) had a dentine caries lesions as- fluorescence-based methods decreased from 27 to 48%
sessed by operative intervention. after correction (FC and LFpen methods, respectively).
The sensitivities of fluorescence-based methods at the Radiographic specificity values decrease by around 7%
D3 threshold were similar. Visual inspection and radio- (table 4).
graphic methods presented lower values of sensitivity but At the D3 threshold, on the other hand, the decrease
had no significant differences compared to the fluores- in the values was more significant in the sensitivities than
cence-based methods. The specificities of the fluores- in the specificities after the correction. While the sensi-
Mendes
Table 2. Performance of the different methods in detecting NC caries lesions (383 teeth in 68 subjects) on occlusal surfaces of prima-
ry molars
Method Sensitivity (95% CI) Specificity (95% CI) Accuracy (95% CI) Az (95% CI)
Examiner 1
LFpen 0.687a (0.607–0.766) 0.813a (0.640–0.985) 0.697a (0.623–0.771) 0.799a (0.756–0.838)
FC 0.444b (0.362–0.526) 0.906a (0.761–1.000) 0.483b (0.403–0.564) 0.674b (0.624–0.721)
Radiographic 0.088c (0.048–0.129) 1.000b (0.890–1.000) 0.164c (0.106–0.223) 0.544c (0.493–0.595)
Examiner 2
LFpen 0.675a (0.606–0.745) 0.813a (0.636–0.989) 0.687a (0.621–0.753) 0.764a (0.719–0.806)
FC 0.450b (0.369–0.532) 0.875a (0.740–1.000) 0.486b (0.407–0.564) 0.690a (0.641–0.736)
Radiographic 0.094c (0.052–0.136) 0.969a (0.911–1.000) 0.167c (0.106–0.228) 0.533b (0.481–0.584)
LFpen cutoff points: sound, 0–4; NC lesions, >4. FC cutoff points: sound, 0–1.1; NC lesions, >1.1.
Different letters indicate statistically significant differences among values within the same column in the examinations performed
by the same examiner (p < 0.05; McNemar test).
Table 3. Performance of fluorescence-based methods in detecting dentine caries lesions (407 sites in 68 subjects) at occlusal surfaces
of primary molars
Method Sensitivity (95% CI) Specificity (95% CI) Accuracy (95% CI) Az (95% CI)
Examiner 1
LFpen 0.952a (0.864–1.000) 0.883a (0.839–0.927) 0.887a (0.845–0.929) 0.973a (0.952–0.986)
FC 1.000a (0.837–1.000) 0.902a (0.863–0.940) 0.907a (0.871–0.943) 0.970a (0.948–0.984)
Radiographic 0.762a (0.579–0.944) 0.966b (0.944–0.989) 0.956b (0.933–0.978) 0.862b (0.825–0.894)
Visual inspection 0.857a (0.675–1.000) 0.984b (0.970–0.999) 0.978b (0.961–0.995) 0.965a, b (0.942–0.981)
Examiner 2
LFpen 1.000a (0.837–1.000) 0.860a (0.815–0.905) 0.867a (0.826–0.909) 0.958a (0.934–0.975)
FC 0.905a (0.780–1.000) 0.881a (0.836–0.925) 0.882a (0.840–0.924) 0.973a (0.952–0.986)
Radiographic 0.762a (0.579–0.944) 0.961b (0.936–0.986) 0.951b (0.923–0.978) 0.857b (0.819–0.890)
Visual inspection 0.857a (0.675–1.000) 0.984b (0.970–0.999) 0.978b (0.961–0.995) 0.966a, b (0.943–0.981)
LFpen cutoff points: sound, 0–34; dentine caries lesion, >34. FC cutoff points: sound, 0–1.4; dentine lesions, >1.4.
Different letters indicate statistically significant differences among values within the same column in the examinations performed
by the same examiner (p < 0.05; McNemar test).
tivities of both fluorescence methods decreased by around [Lussi and Hellwig, 2006; Rodrigues et al., 2008] and
9%, the specificities decreased only by around 1%. The they showed similar values of reproducibility compared
methods presented similar performance after the adjust- to the values obtained in the present study. Concerning
ments (table 4). validity, in vitro studies have shown lower specificity
and sensitivity values [Lussi and Hellwig, 2006; Ro-
drigues et al., 2008] compared to the values obtained in
Discussion the present study. In vivo studies have also demonstrat-
ed lower values of performance in detecting occlusal
This was the first in vivo study testing the new LFpen caries lesions in permanent teeth [Krause et al., 2007;
method for detecting occlusal caries in primary teeth, Huth et al., 2008].
and it was the first study to evaluate the clinical perfor- These differences could be explained by the different
mance of a new FC. With regard to the LFpen, some types of teeth. The enamel of primary teeth has about
in vitro studies were performed using permanent teeth half the thickness found in permanent teeth, and some
Mendes
tine caries lesions. However, the low prevalence of den- In conclusion, both fluorescence-based methods pre-
tine caries lesions raises some doubts concerning the util- sent similar performances in detecting dentine occlusal
ity of additional methods [Baelum, 2010], and this issue caries lesions in primary teeth but they usually yield more
should be addressed in further studies. false-positive results than visual and radiographic meth-
On the other hand, in detecting noncavitated caries ods. Therefore, they could be considered as adjuncts of
lesions, both radiographic and fluorescence methods did visual inspection in order to improve the sensitivity. Fur-
not show good performance. The radiographic method thermore, owing to the use of an imprecise reference
did not usually perform well in detecting early caries le- standard method, the correction of values related to the
sions [Braga et al., 2010]. Concerning the fluorescence- diagnostic performance shows that they could be overes-
based methods, as dentine caries lesions are more infect- timated. Therefore, adjustment of the values related to
ed than initial lesions [Kidd et al., 2003], a worse perfor- the performance of the methods could be an important
mance of fluorescence-based methods in detecting initial procedure to be performed in studies of caries detection
lesions would be expected since the methods detect the which use imperfect reference standard methods, but
presence of bacterial metabolites. Thus, for noncavitated further studies need to be conducted to validate this ap-
caries lesions, visual inspection performed with exten- proach.
sive drying of the site on cleaned teeth is the most accu-
rate method for use in clinical practice [Ekstrand, 2004;
Braga et al., 2010]. Acknowledgements
Additionally, since an imperfect reference method was
This study was supported by the Conselho Nacional de Des-
used, a suggestion to correct the values of performance
envolvimento Científico e Tecnológico (CNPq – process No.
was made. The intention was for these values to be more 476372/2006-2, 302368/2008-6, and 565061/2008-9), Pró-Reito-
realistic than values of tables 2 and 3 or than those values ria de Pesquisa e de Pós-Graduação da USP, and Fundação de
obtained in previous studies using the same reference Amparo à Pesquisa do Estado de São Paulo (FAPESP). This study
standard method [Krause et al., 2007; Huth et al., 2008]. was also supported by Canadian Institutes of Health Research
(CIHR grant # 106657 and CIHR grant # 97577). Dr. Siqueira is a
After the correction, we noticed a significant decrease in
recipient of the CIHR New Investigator Salary Award.
some values. The decrease was more pronounced in the
specificities at the NC threshold and in the sensitivities at
the D3 threshold. This pattern is because of the preva- Disclosure Statement
lence of the disease. In fact, when using an imperfect ref-
erence standard method in a low-prevalence population The authors certify that they have no affiliation with or finan-
sensitivities are usually overestimated, while in high- cial involvement in any organization or entity with a direct finan-
prevalence populations specificities are overestimated cial or personal interest in the subject matter or materials dis-
cussed in this paper.
[Irwig et al., 2002].
These results raise some limitations of caries detection
studies. Mainly in the in vivo setting, the reference stan-
dard methods for the presence of caries lesions are far References Abalos C, Herrera M, Jimenez-Planas A, Llamas
R: Performance of laser fluorescence for de-
from ideal. Therefore, the performance of the caries de- tection of occlusal dentinal caries lesions in
tection methods in studies is usually overestimated. The permanent molars: an in vivo study with to-
correction proposed in the present study corroborates tal validation of the sample. Caries Res 2009;
43:137–141.
this issue, and the results after the correction are proba- Bader JD, Shugars DA: A systematic review of the
bly closer to the actual performance of the methods. performance of a laser fluorescence device
Thus, further studies should take into account the limita- for detecting caries. J Am Dent Assoc 2004;
135:1413–1426.
tions of the reference standard method, and procedures Bader JD, Shugars DA, Bonito AJ: A systematic
to correct this performance should be carried out in an review of the performance of methods for
attempt to increase the external validity of the studies. To identifying carious lesions. J Public Health
Dent 2002;62:201–213.
the best of our knowledge, this is the first study of caries Baelum V: What is an appropriate caries diagno-
detection methods that performs a correction of obtained sis? Acta Odontol Scand 2010;68:65–79.
results considering the imperfect reference standard Braga MM, Mendes FM, Ekstrand KR: Detec-
tion activity assessment and diagnosis of
method. Additional studies, however, should be carried dental caries lesions. Dent Clin North Am
out in order to validate this statistical approach. 2010;54:479–493.
Mendes
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.