Académique Documents
Professionnel Documents
Culture Documents
com
PGMJ Online First, published on December 23, 2016 as 10.1136/postgradmedj-2016-134491
Original article
Original article
Analysis
We considered the quality of the evidence for each outcome
according to the GRADE criteria, which generates a rating of the
evidence by evaluating the study design and a number of modify-
ing factors that can upgrade or (more likely) downgrade the
quality of evidence.10 The concordance data from each study
were pooled using the DerSimonian-Laird random effects model
and the pooled effect size was reported as a proportion with 95%
CI. Heterogeneity was assessed using the I2 statistic. A subgroup
analysis was planned in order to identify the population of inter-
est, emergency medical staff. Statistical analysis was conducted
using Stata V.11.0 (StataCorp, College Station, Texas, USA).
RESULTS
There were 41 eligible studies identied for full-text review,
after which a further 20 were excluded. Figure 1 depicts the
process of study selection.
Figure 1 Description of study selection using the preferred reporting
items for systematic reviews and meta-analysis method. Among the 20 studies that were excluded on full-text review,
17 did not meet eligibility criteria and the remainder met exclu-
sion criteria. Of the 13 studies that did not report a relevant
Study selection primary outcome, 5 assessed diagnostic variance on CTB
The authors reviewed abstracts and full-text articles where rele- between radiology specialists, 5 compared different CT tech-
vant, with any inconsistency resolved by consensus. Eligible nologies and the nal 3 studies only reported the incidence of
studies reported a primary outcome that measured the concord- CTB use. All four studies that failed to report on the interpret-
ance of non-contrast CTB interpretation between non-radiology ation of CTB assessed cerebral angiography instead. There were
medical specialists and qualied radiologists. Exclusion criteria two further studies that were excluded due to poor reporting of
included the use of paediatric patient populations, a gold stand- the primary outcome. The rst did not clearly describe the rele-
ard for CTB interpretation non-equivalent to a radiology special- vant outcome and the second reported clinician performance on
ist and unclear or insufcient reporting of the primary outcome. a non-clinical test from which the authors were unable to
extract concordance data. The nal study excluded was an
Data extraction abstract only.
Data from each eligible study were collected using a standard Table 1 reports the characteristics of the 21 studies identied
template that focused on methodology, results and relevant as appropriate for quantitative synthesis. There was signicant
Original article
Al-Reesi11 (2010) 0.93 0.07 0.05 0.78 (0.670.86) 0.96 (0.930.98) 0.83
Alfaro12 (1994) 0.63 0.37 0.24
Ardic13 (2015) 0.95 0.05 0.03 0.93 0.93 0.82
Arendts14 (2003) 0.85 0.15 0.06 0.69
Boyle15 (2009) 0.92 (0.870.96) 0.75 (0.650.84)
Cheung16 (2015) 0.91 0.09 0.09 0.66 (0.600.71) 0.96 (0.950.97)
Dolatabadi17 (2013) 0.84 0.16
Harding18 (2010) 0.64 0.36 0.02
Khan19 (2013) 0.87 0.13 0.06 0.96 (0.930.99) 0.63 (0.460.77) 0.64
Khoo20 (2007) 0.67 0.33 0.11 0.57 (0.450.69) 0.63 (0.460.77) 0.24
Mucci21 (2005) 0.87 0.13 0.04
Talebian22 (2015) 0.86 0.14 0.03 0.62 0.86 0.68
Chun23 (2010) 0.92 0.08 0.79 0.88 0.57
Dourado24 (2015) 0.76
Mukerji25 (2009) 0.70
Vorhies26 (2002) 0.94 0.06 0.02 0.89 1.00
Erly27 (2002) 0.91 0.09 0.02
Miyakoshi28 (2009) 0.97 0.03 0.01
Roszler29 (1991) 0.96 0.04 0.02
Strub30 (2006) 0.96 0.04 0.001
Wysoki31 (1998) 0.96 0.04 0.02
CSM, clinically significant misinterpretation.
Original article
Figure 2 Forest plot describing the rates of concordance in the emergency department subgroup.
Figure 3 Forest plot describing the rates of clinically signicant misinterpretations in the emergency department subgroup.
for CTB is signicantly higher than that reported for other homogeneity of studies that assessed radiology trainees. As
imaging modalities interpreted by the ED staff. Nitowski32 expected, the concordance data showed a higher level of accur-
reports a CSM rate of 0.002 for emergency physicians interpret- acy in the radiology trainee subgroup compared with the ED
ing plain radiography of the limbs, while Safari et al33 found staff. Furthermore, the rate of CSMs for radiology trainees
a total discrepancy rate of 0.015 for interpretation of the (0.0010.025) is consistent with the approximate CT misinter-
chest X-ray. pretation rate of 0.02 that is reported in the radiology litera-
The primary comparison made in this analysis was between ture.28 Gallagher et al,34 who were excluded from quantitative
the ED and radiology trainee subgroups. First, the range of con- analysis, directly compared the performance of radiologists,
cordance was wider in the ED subgroup and reects the relative radiology trainees, neuroradiographers and emergency clinicians
4 Evans LR, et al. Postgrad Med J 2016;0:16. doi:10.1136/postgradmedj-2016-134491
Downloaded from http://pmj.bmj.com/ on June 14, 2017 - Published by group.bmj.com
Original article
and found that the latter had the highest error rate for CTB light of the frequency of CTBs being requested these ndings
interpretation. As noted previously, the ACEM concept of indicate the importance of educational and institution-based
expert prociency correlates practically with the level of skill interventions directed towards improving the accuracy of CTB
expected of a consultant in that eld. Therefore, comparison of interpretation by emergency clinicians.
ED performance to both radiology specialists and trainees indi-
cates a possible deciency in the skillset required of specialist
emergency physicians.
The studies that reported on neurosurgical interpretation of Main messages
CTB varied between prospective evaluation of images and retro-
spective, exam-based assessment. Chun23 reported a concord-
CT of the brain is a common and important emergency
ance rate of 0.92 (95% CI 0.90 to 0.95). However, this study
department investigation.
was limited by the inclusion of only ve participants and by
The capacity of emergency clinicians to accurately interpret
focusing on scoring systems that are not commonly used in clin-
this type of imaging is unclear.
ical practice. All three studies in the neurosurgery subgroup
We identied 21 studies describing the concordance of
reported a moderatehigh level of inter-rater agreement with
radiologists with non-radiologists; 12 reported on emergency
Cohens coefcients of 0.570.76. The nal study by Vorhies
clinicians, 5 on radiology trainees and 4 on surgeons.
et al26 reported the accuracy of senior general surgical trainees
The rate of concordance was lowest in the emergency
interpreting a variety of modalities and noted a CTB accuracy of
clinician subgroup with a range of 0.630.95 and a clinically
0.94 (95% CI 0.90 to 0.97). This study was restricted in regards
signicant error rate of 0.020.24.
to the small number of scans included (n=47) and the exclusion
Given the frequency with which CT of the brain is used in
of any scan interpreted in the presence of a senior surgeon.
the emergency department, interventions towards improving
The major limitation to this review is the presence of signi-
accuracy may be warranted.
cant heterogeneity. As described previously, the variability in
study design and the lack of a consistent method of assessing
accuracy have impeded meta-analysis. Substantial bias was
present in each individual study, most commonly in regards to
Current research questions
the baseline characteristics of selected participants and their
level of experience. The total number of scans interpreted in
each study varied considerably and the proportion of those 1. The use of CT of the brain in the emergency department has
scans that were abnormal had a similarly wide range (0.16 decreased with the development of MRI:
0.72). The majority of studies noted that there was a likely rela- a. True
tionship between the proportion of abnormal scans and the b. False
number of misinterpretations. There was further ambiguity in 2. Suspected traumatic brain injury is the most common
the way in which a participants interpretation was classied as indication for CT of the brain in the emergency department:
concordant or discordant. Several studies noted that incomplete a. True
or poor documentation frequently impaired the extraction of b. False
data, which was resolved by the investigators subjectively decid- 3. Emergency clinicians and radiology trainees have been
ing the result. The variety of study limitations and the signi- shown to interpret CT of the brain with similar rates of
cant presence of bias is reected in the formal classication of accuracy:
the quality of evidence on this topic as very low, as dened by a. True
the GRADE criteria.10 b. False
4. The error rate for emergency clinicians interpreting CT of the
brain is similar to plain radiography of the chest:
CONCLUSION
a. True
Statistical and clinical heterogeneity limit the strength of this
b. False
reviews ndings. However, the individual conclusions drawn
5. Ongoing educational interventions are indicated to reduce
from each study indicate variance between ED interpretation of
the error rate for emergency clinicians interpreting CT of the
CTB and the nal radiology report. Of note, the rate of clinic-
brain:
ally signicant errors ranged from 0.02 to 0.24. High-quality
a. True
prospective research is required to further elucidate the ability
b. False
of emergency clinicians in this important area. Nonetheless, in
Evans LR, et al. Postgrad Med J 2016;0:16. doi:10.1136/postgradmedj-2016-134491 5
Downloaded from http://pmj.bmj.com/ on June 14, 2017 - Published by group.bmj.com
Original article
Acknowledgements The authors acknowledge the authors of all original studies 15 Boyle A, Staniciu D, Lewis S, et al. Can middle grade and consultant emergency
included in this review. physicians accurately interpret computed tomography scans performed for head
trauma? Cross-sectional study. Emerg Med J 2009;26:5835.
Contributors LRE was responsible for the planning and execution of the systematic
16 Cheung LW, Shih YN, Leung CS, et al. How accurate are emergency department
review, drafting of the article and submission to the journal. BM reviewed the search
medical staff in the interpretation of head injury related computed tomography?
strategy and carried out the statistical analyses. BM, MCF and DV contributed to the
Hong Kong J Emerg Med 2015;22:2417.
editing and revision of the manuscript. As rst author and guarantor, LE is
17 Arhami Dolatabadi A, Baratloo A, Rouhipour A, et al. Interpretation of computed
ultimately responsible for the content and format of the nal manuscript.
tomography of the head: Emergency physicians versus radiologists. Trauma Mon
Competing interests None declared. 2013;18:869.
Provenance and peer review Not commissioned; externally peer reviewed. 18 Harding J, Craig M, Jakeman N, et al. Emergency physician interpretation of head
CT in trauma and suspected subarachnoid haemorrhageis it viable? An audit of
current practice. Emerg Med J 2010;27:11620.
19 Khan A, Qashqari S, Al-Ali AA. Accuracy of non-contrast CT brain interpretation by
REFERENCES emergency physicians: a cohort study. Pak J Med Sci 2013;29:54953.
1 Raja AS, Andruchow J, Zane R, et al. Use of neuroimaging in US emergency 20 Khoo NC, Duffy M. Out of hours non-contrast head CT scan interpretation by
departments. Arch Intern Med 2011;171:2602. senior emergency department medical staff. Emerg Med Australas 2007;19:1228.
2 Larson DB, Johnson LW, Schnell BM, et al. National trends in CT use in the 21 Mucci B, Brett C, Huntley LS, et al. Cranial computed tomography in trauma: The
emergency department: 1995-2007. Radiology 2011;258:16473. accuracy of interpretation by staff in the emergency department. Emerg Med J
3 Australian College for Emergency Medicine. ACEM Curriculum Framework 2015. 2005;22:53840.
p. 68. 22 Talebian MT, Kavandi E, Farahmand S, et al. Comparing the brain CT scan
4 Goergen SK, Grimm J, Paul E, et al. Audit of demand for after-hours CT scanning interpretation of emergency medicine team with radiologists report and its impact
services in RANZCR-accredited training departments. J Med Imaging Radiat Oncol on patients outcome. Emerg Radiol 2015;22:2618.
2016;60:3546. 23 Chun KA, Manley GT, Stiver SI, et al. Interobserver variability in the
5 Hrtl R, Gerber LM, Iacono L, et al. Direct transport within an organized state assessment of CT imaging features of traumatic brain injury. J Neurotrauma
trauma system reduces mortality in patients with severe traumatic brain injury. 2010;27:32530.
J Trauma 2006;60:12506; discussion 6. 24 Dourado JC, Pereira JL, Albuquerque LA, et al. Indices of agreement between
6 Seelig JM, Becker DP, Miller JD, et al. Traumatic acute subdural hematoma: major neurosurgeons and a radiologist in interpreting tomography scans in an emergency
mortality reduction in comatose patients treated within four hours. N Engl J Med department. Arq Neuropsiquiatr 2015;73:68891.
1981;304:151118. 25 Mukerji N, Cahill J, Paluzzi A, et al. Emergency head CT scans: can neurosurgical
7 Mellanby E, Clark R, McLellan S. Time from CT scan to surgery in traumatic brain registrars be relied upon to interpret them? Br J Neurosurg 2009;23:15861.
injury. Room for improvement? Anaesthesia. Conference:Scottish Society of 26 Vorhies RW, Harrison PB, Smith RS, et al. Senior surgical residents can accurately
Anaesthetists Annual Meeting 2011 Crieff, Perthshire United Kingdom. Conference interpret trauma radiographs. Am Surg 2002;68:2215; discussion 5-6.
Publication: 66. 27 Erly WK, Berger WG, Krupinski E, et al. Radiology resident evaluation of
8 Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews head CT scan orders in the emergency department. AJNR Am J Neuroradiol
and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009;62:100612. 2002;23:1037.
9 Evans L, Fitzgerald M, Mitra B. Assessing the accuracy of non-radiologists in 28 Miyakoshi A, Nguyen QT, Cohen WA, et al. Accuracy of preliminary interpretation of
interpreting computed tomography of the brain. PROSPERO 2016: neurologic CT examinations by on-call radiology residents and assessment of patient
CRD42016037210. http://www.crd.york.ac.uk/PROSPERO/display_record.asp? outcomes at a level I trauma center. J Am Coll Radiol 2009;6:86470.
ID=CRD42016037210 29 Roszler MH, McCarroll KA, Rashid T, et al. Resident interpretation of emergency
10 Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE computed tomographic scans. Invest Radiol 1991;26:3746.
evidence proles and summary of ndings tables. J Clin Epidemiol 30 Strub WM, Vagal AA, Tomsick T, et al. Overnight resident preliminary interpretations
2011;64:38394. on CT examinations: should the process continue? Emerg Radiol 2006;13:1923.
11 Al-Reesi A, Stiell IG, Al-Zadjali N, et al. Comparison of CT head interpretation 31 Wysoki MG, Nassar CJ, Koenigsberg RA, et al. Head trauma: CT scan interpretation
between emergency physicians and neuroradiologists. Euro J Emerg Med by radiology residents versus staff radiologists. Radiology 1998;208:1258.
2010;17:2802. 32 Nitowski LA, OConnor RE, Reese CL IV. The rate of clinically signicant plain
12 Alfaro D, Levitt MA, English DK, et al. Accuracy of interpretation of cranial radiograph misinterpretation by faculty in an emergency medicine residency
computed tomography scans in an emergency medicine residency program. Ann program. Acad Emerg Med 1996;3:7829.
Emerg Med 1994;25:16974. 33 Safari S, Baratloo A, Negida AS, et al. Comparing the interpretation of traumatic chest
13 Ardic S, Guneysel O. Emergency medicine residents can assess cranial computed x-ray by emergency medicine specialists and radiologists. Arch Trauma Res 2014;3.
tomography scans consistently with radiologists. J Acute Med 2015;5:925. 34 Gallagher FA, Tay KY, Vowler SL, et al. Comparing the accuracy of initial head CT
14 Arendts G, Manovel A, Chai A. Cranial CT interpretation by senior emergency reporting by radiologists, radiology trainees, neuroradiographers and emergency
department staff. J Med Imaging Radiat Oncol 2003;47:36874. doctors. Br J Radiol 2011;84:10405.
These include:
References This article cites 30 articles, 5 of which you can access for free at:
http://pmj.bmj.com/content/early/2016/12/23/postgradmedj-2016-134
491#BIBL
Email alerting Receive free email alerts when new articles cite this article. Sign up in the
service box at the top right corner of the online article.
Notes