Académique Documents
Professionnel Documents
Culture Documents
DOI 10.1007/s00247-010-1812-6
ORIGINAL ARTICLE
Materials and methods Class IV: Disagreement in diagnosis that, if left un-corrected,
would have led to a major change in management.
Between December 30, 1998, and July 29, 2009, 131,728 (Example: Free intraperitoneal gas not detected on
cases were reviewed as part of our ongoing quality- abdominal radiograph).
assurance process. There are five general mechanisms by
which cases are identified and reported: the first is a Of the 131,728 cases subjected to additional review, a
hospital-wide sentinel event reporting system (SERS) with subset of 4,051 cases was identified in which a diagnostic
which clinicians report cases where there was a discrepancy error had been reported to our departmental Quality
or diagnostic error related to the Radiology Service. The Improvement Committee. Of the 4,051 discrepancies
second is the radiologist double-read process in which each reported, we limited our review to 265 cases that were
radiologist in the department reviews a series of randomly initially classified as Class III or IV disagreements.
selected cases interpreted by other staff radiologists. The Two senior pediatric radiologists reviewed each case
third method identifies discrepancies that arise during individually and categorized the etiology of the diagnostic
weekly multidisciplinary conferences (i.e. solid tumor, bone error according to the method developed by Graber [9] and
tumor, neuro-oncology, general surgical, urology and used widely in evaluating medical errors [10–14]. Disagree-
advanced fetal care conferences) in the radiology depart- ments were resolved by consensus.
ment. These represent a random selection of active cases in A diagnostic error was defined as a diagnosis that was
the hospital at any given time. Identified discrepancies are delayed, wrong or missed [12]. Errors were further classified
assessed with regard to clinical progression of disease, as perceptual, cognitive, system-related or unavoidable [9,
surgical or pathological findings. The fourth method is 12–14]. Perceptual errors were defined as non-recognition of
reporting by radiologists who either missed cases them-
selves or identified cases that were missed by other
radiologists. These are reported at the monthly morbidity Table 1 Reader status, imaging modality and initial severity rating in
and mortality conference of the department. Finally, there is 265 cases with diagnostic errors
the daily review of images interpreted initially by resident n=265 n %
and fellow trainees during after-hours shifts. Although there
are biases in each of the sampling methods, the combina- Reader status
tion of inputs gives us a broad look at the type and Staff 121 45.7
distribution of cases as well as their severity. These Fellow 57 21.5
discrepancies were initially classified by consensus by our Resident 85 32.1
departmental Quality Improvement Committee according to Technologist/clinician 2 0.8
a four-step ordinal scale: Image modality
X-ray 143 54.0
Class I: Minor disagreement in observation that would Fluoroscopy 21 7.9
have led to no change in management. (Example: US 20 7.5
Small post-operative pneumomediastinum not CT 65 24.5
detected in a child in the intensive care unit MRI 14 5.3
who already has a thoracostomy tube in place). Nuclear medicine 2 0.8
Class II: Disagreement in diagnosis that would have led to Initial severity
no change in management. (Example: Initial Class I (minor event) 6 2.3
interpretation of pre-operative chest radiograph Class II (moderate event) 36 13.5
failed to diagnose cardiomegaly and shunt vascu- Class III (major event) 138 52.1
larity in a child with known VSD being prepared Class IV (severe event) 85 32.1
for surgical correction. Decision to operate and
operative planning would not have been affected). Definitions
Class III: Disagreement in diagnosis that, if left un-corrected, • Class I: Disagreement in observation that would have led to no
would have led to a minor change in management. change in therapy
(Example: Initial impression on a skull radiograph • Class II: Disagreement in diagnosis that would have led to no change
in therapy
of Luchenshadel skull, with final diagnosis of
• Class III: Disagreement in diagnosis that would have led to a minor
normal convolutional markings—had this error
change in therapy
gone unrecognized, the child might have had an
• Class IV: Disagreement in diagnosis that, if left un-corrected, would
unnecessary MRI with sedation to assess for Chiari have led to a major change in therapy. (Example: free peritoneal gas
II malformation). not detected on abdominal radiograph)
Pediatr Radiol
an imaging abnormality. Errors were defined as cognitive in training of the radiologist involved (resident, pediatric
nature when due to faulty information processing (over- radiology fellow or staff radiologist).
interpretation of an imaging finding, misinterpretation of a This study was approved by our institution’s Internal Review
finding or failure to consider a different diagnosis for a given Board, and the requirement for informed consent was waived.
finding [premature closure]); faulty data gathering (poorly
performed imaging examination, inadequate review of
patient history or lack of consideration of a patient’s Results
underlying condition), or insufficient knowledge base.
System-related errors were defined as technical (due to Table 1 shows the distribution of discrepancies by reader
equipment failure) or organizational flaws such as ineffective status, imaging modality and initial severity score for the 265
policies, inadequate training or supervision, and defective cases reviewed in detail. Radiology trainees were responsible
communication. Repeated instances of the same error for the majority of discrepancies reviewed (53.6%), followed
(clustering) were also included as a system-based error. An by staff radiologists (45.7%). This is consistent with the
error was considered unavoidable when abnormal imaging workload distribution in our department in which 48% of the
findings were absent or masked or so atypical that arriving at studies are interpreted by staff radiologists alone. A technol-
a correct diagnosis would not be expected. An error was ogist or clinician was associated with an error in less than 1%
considered multi-factorial when more than one type of error of cases. The two modalities most frequently involved with
was identified in a given clinical case. discrepancies were radiography (54%) and CT (24.5%).
The individual associated with the error (technologist, Fluoroscopy (8%), US (7.5%) and nuclear medicine (0.8%)
radiologist or clinician) was tabulated along with level of were involved less often.
Initial severity classification category (29 errors), while errors were attributable to
insufficient skills at performing an imaging test in only eight
We classified the discrepancies as major events (if left instances. Cognitive errors were seen in association with
un-corrected, these would have led to a minor change in perceptual or system-related errors in 83/151 cases (55%).
management) in 138 of 265 cases reviewed (52%), and However, multiple cognitive errors were present in the
as severe events (if left un-corrected, these would have majority of cases with at least one cognitive error.
led to a major change in management) in 85 cases Perceptual errors were the next most common category,
(32.1%). During the study, 42 cases were re-classified as accounting for 165/484 (34.1% of errors, Table 2, Fig. 3).
class I (6) or class II (36) disagreements but were still This class of error occurred as an isolated cause of error in
included in the overall data and did not affect the overall 97 cases (59%), and was associated with cognitive or
distribution of discrepancies in any category. system-related errors or both in 68 cases (41%). Table 3
shows the type and frequency of the 54 system-related errors
Etiology of diagnostic error we encountered. Organizational errors were most frequent in
this category (48 errors), consisting primarily of repeating
We identified 484 errors in the 265 cases reviewed (mean instances of the same error type (clustering, 18 errors), faulty
1.8 errors/case, Tables 2 and 3). The most common type of or incomplete medical history (13 errors, Fig. 4), and
error was cognitive, accounting for 258 (53.3%) errors. difficulties in communication (6 errors). Inefficient workflow
Within this category, faulty information processing processes, inadequate policies and procedures, and failure to
accounted for the majority of errors, including misinterpre- supervise a system or trainee were less commonly attribut-
tation of a test (78 errors), premature closure (71 errors) and able causes of error. Technical errors consisting of a technical
over-interpretation of a finding (13 errors) (Figs. 1 and 2). or equipment failure could be identified in only six cases.
Failures in data gathering were the next most common Errors were considered unavoidable in only seven instances.
source of cognitive error (47 errors), consisting of situations
when an imaging test was performed either incorrectly or
incompletely (39 errors) or in which insufficient information Discussion
was collected before starting a diagnostic test (8 errors). In our
environment, faulty knowledge was a relatively uncommon The majority of the diagnostic errors documented in our
etiology, accounting for only 37 cognitive errors. An study were cognitive and multi-factorial. These errors were
inadequate knowledge base contributed to most errors in this not commonly caused by a lack of medical knowledge but
rather by a combination of lack of recognition of an (over- or faulty interpretation of a finding) were not as common
important imaging finding, faulty interpretation of a (151/265 cases). These findings are consistent with an earlier
finding, and premature closure. A number of cases were review of 182 radiology errors at a university hospital that
assigned to multiple categories of error. For example, the identified 69% of diagnostic errors as perceptual/cognitive in
cases illustrated in Figs. 1 and 2 and included in Table 2 nature, of which only two were related to lack of knowledge
were included as examples of a specific type of cognitive [8]. Under-interpretation of findings can be related to a
error, yet these two cases were categorized as both faulty number of perceptual and visual phenomena. The first is
interpretation and premature closure. visual isolation, where attention is selectively focused on a
Radiologists engage in two interrelated processes when main area of the image while less or no attention is given to
interpreting imaging studies: perception and analysis [15]. secondary areas. A second cause has been termed "satisfaction
Kundel [16] defines perception as “the unified awareness of of search," which occurs when additional lesions remain
the content of a displayed image” and analysis as undetected after detection of an initial lesion [8, 17].
“determining the meaning of the perception in the context Students of cognitive psychology suggest that cognitive
of the medical problem that initiated the acquisition.” errors are not the act of a few bad performers or the result
Failures in perception show up as failure to identify an of ignorance but rather a result of systematic factors that are
important finding on an imaging study, the most common predictable and occur routinely in clinical medicine [18].
source of diagnostic error identified in our study (165/265 Thus, understanding how these mistakes are made might be
cases), while failures in the analytic portion of the process helpful in correcting their underlying causes. Many diag-
Fig. 4 Organizational
error (faulty medical history).
A 14-year-old boy with history
of exposure to tuberculosis,
presenting with wheezing.
AP (a) and lateral (b) chest
radiographs show aspirated
tooth and pulmonary opacities
misinterpreted as a probable
Ghon complex. History of
recent fight and missing tooth
was not provided
study (69%) involved recurrent under-interpretation of effect of these variables on radiologist performance and to
buckle, Salter II, avulsion and transverse bony fractures. better define modality and radiologist-specific error rates,
This information was used to modify their training as well as effective strategies for long-term and sustainable
programs to emphasize these fractures early in the improvement.
residents’ training and thus reduce the risk of this recurrent
error.
There are limitations to our methodology. The optimal References
method of identification of diagnostic discrepancies would be
to perform a prospective study of a large random sample of 1. Berlin L (2007) Accuracy of diagnostic procedures: has it
imaging studies performed in our department, with represen- improved over the past five decades? AJR 188:1173–1178
tative distribution among imaging modalities and radiologists 2. Siegle RL, Baram EM, Reuter SR et al (1998) Rates of
disagreement in imaging interpretation in a group of community
in near-real time. This would allow for a detailed root-cause hospitals. Acad Radiol 5:148–154
analysis of factors associated with missed diagnoses [10, 11]. 3. Borgstede JP, Lewis RS, Bhargavan M et al (2004) RADPEER
Although we relied on a number of mechanisms to identify quality assurance program: a multifacility study of interpretive
errors, the data collected were non-random and had the disagreement rates. J Am Coll Radiol 1:59–65
4. Soffa DJ, Lewis RS, Sunshine JH et al (2004) Disagreement in
potential for a number of systematic biases, including the interpretation: a method for the development of benchmarks for
over- or under-reporting of errors in certain modalities or quality assurance in imaging. J Am Coll Radiol 1:212–217
categories of error and self-reporting biases. The classification 5. Miyakoshi A, Nguyen QT, Cohen WA et al (2009) Accuracy of
of potential patient harm was based on the theoretical risk to preliminary interpretation of neurologic CT examinations by on-
call radiology residents and assessment of patient outcomes at a
patient if a diagnostic error remained undetected. It was not level I trauma center. J Am Coll Radiol 6:864–870
based on documented, actual patient outcomes. In the majority 6. Berbaum KS, Caldwell RT, Schartz KM et al (2007) Does
of these cases, the discrepancy was identified before harm computer-aided diagnosis for lung tumors change satisfaction of
came to a patient or therapy was changed. search in chest radiography? Acad Radiol 14:1069–1076
7. Miglioretti DL, Haneuse SJ, Anderson ML (2009) Statistical
approaches for modeling radiologists' interpretive performance.
Acad Radiol 16:227–238
Conclusion 8. Renfrew DL, Franken EA, Berbaum KS et al (1992) Error in
radiology: classification and lessons in 182 cases presented at a
problem case conference. Radiology 183:145–150
Our study defines a taxonomy of errors encountered in a 9. Graber ML, Franklin N, Gordon R (2005) Diagnostic error in
large academic pediatric radiology practice and suggests internal medicine. Arch Intern Med 165:1493–1499
that most are multi-factorial in etiology. Yet, many 10. VA National Center for Patient Safety (2002) The basics of
radiologists believe that increasing workload, increasing healthcare failure mode and effect analysis. Available at www.
patientsafety.gov/SafetyTopics.html. Accessed 6 July 2006
numbers of images in CT and MRI studies, interruptions 11. Bagian JP, Gosbee J, Lee CZ et al (2002) The veterans affairs root
and multi-tasking during the workday also have an impact cause analysis system in action. Jt Comm J Qual Improv 28:531–
on error rates. Further studies are needed to evaluate the 545
Pediatr Radiol
12. Benavidez OJ, Gauvreau K, Jenkins KJ et al (2008) Diagnostic 19. Tversky A, Kahneman D (1986) Rational choice and the framing
errors in pediatric echocardiography: development of taxonomy of decisions. J Bus 59:S251–S278
and identification of risk factors. Circulation 117:2995–3001 20. Berner ES, Graber ML (2008) Overconfidence as a cause of
13. Kassirer JP, Kopelman RI (1989) Cognitive errors in diagnosis: diagnostic error in medicine. Am J Med 121:S2–S23
instantiation, classification, and consequences. Am J Med 86:433–441 21. Potchen EJ (2006) Measuring observer performance in chest
14. Bordage G (1999) Why did I miss the diagnosis? Some cognitive radiology: some experiences. J Am Coll Radiol 3:423–432
explanations and educational implications. Acad Med 74:S138–S143 22. Eva KW, Norman GR (2005) Heuristics and biases—a biased
15. Kundel HL (2006) History of research in medical image perspective on clinical reasoning. Med Educ 39:870–872
perception. J Am Coll Radiol 3:402–408 23. Eva KW, Norman GR (2010) Diagnostic error and clinical
16. Kundel HL (1979) Images, image quality and observer perfor- reasoning. Med Educ 44:94–100
mance. Radiology 132:265–271 24. Croskerry P (2003) The importance of cognitive errors in
17. Berbaum KS, Franken EA Jr, Dorfman DD et al (1992) Satisfaction of diagnosis and strategies to minimize them. Acad Med 78:775–780
search in diagnostic radiology. Invest Radiol 27:571–573 25. Halsted MJ, Kumar H, Paquin JJ et al (2004) Diagnostic errors by
18. Redelmeier DA (2005) The cognitive psychology of missed radiology residents in interpreting pediatric radiographs in an
diagnoses. Ann Intern Med 142:115–120 emergency setting. Pediatr Radiol 34:331–336