Vous êtes sur la page 1sur 8

ORIGINAL RESEARCH

Small Pulmonary Artery Defects Are Not Reliable Indicators of


Pulmonary Embolism
Wallace T. Miller, Jr.1, Lawrence A. Marinari2, Eduardo Barbosa, Jr.1, Harold I. Litt1, James E. Schmitt1, Anton Mahne3,
Victor Lee4, and Scott R. Akers5
1
Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia Pennsylvania; 2Department of
Medicine, and 4Department of Radiology, The Bryn Mawr Hospital, Bryn Mawr, Pennsylvania; 3Department of Radiology and
Outpatient Diagnostic Testing, Cooper University Health Care, Camden, New Jersey; and 5Veterans Affairs Medical Center,
Philadelphia, Pennsylvania

Abstract (short-axis diameter) of the lesion, and (3) diminishing quality of


the CT examination. Forty-eight of 177 (27%) of subsegmental
Objectives: To evaluate the rate of agreement of pulmonary vascular defects identified by community radiologists were deemed
embolism diagnosis in computed tomography (CT) pulmonary indeterminate, and 27 of 177 (15%) of subsegmental vascular defects
angiogram studies and to evaluate the rate of inaccurate were judged to be false positive for pulmonary embolism by the
interpretations in the community hospital setting. consensus diagnosis. Fifty-four of 274 (20%) vascular defects with
short axis less than 6 mm were indeterminate for pulmonary
Methods: Using the keywords “pulmonary embolism/embolus/ embolism, and 37 of 274 (14%) of vascular defects with short axis
emboli,” the radiology information system was searched for CT less than 6 mm were false positive for pulmonary embolism. Eleven
pulmonary angiograms performed over a 3-year period at three of 13 (85%) of vascular lesions identified as pulmonary emboli
U.S. community hospitals. Studies containing probable or definite on the lowest-quality CT examinations were false positive or
pulmonary emboli were independently reviewed by four subspecialty indeterminate for pulmonary embolism. False-positive
thoracic radiologists. examinations were most often due to respiratory motion artifact
Measurements and Main Results: Agreement about the (19/38, 50%).
presence of pulmonary embolism progressively decreased with Conclusions: There is relatively poor interobserver agreement for
decreasing diameter of pulmonary vascular lesions (P , 0.0001). subsegmental and/or small pulmonary artery defects, especially in
There was a sharp fall in observer agreement for pulmonary CT pulmonary angiograms degraded by technical artifacts. These
embolism of subsegmental lesions (P , 0.0001). The frequency factors can lead to an increased frequency of inaccurate interpretation
of agreement decreased with decreasing quality of the imaging or indeterminate diagnosis of subsegmental and/or small defects.
examination (P , 0.0001). Community radiologists were prone to Caution is indicated in interpreting the significance of small vascular
false-positive pulmonary embolism diagnosis of subsegmental and/ defects in CT pulmonary angiograms.
or small pulmonary arterial defects. The probability of a false-positive
diagnosis and indeterminate examinations progressively increased Keywords: pulmonary embolism; X-ray computed tomography;
with: (1) more peripheral location of the lesion, (2) decreased size diagnostic errors; false-positive reactions; community hospitals

(Received in original form February 25, 2015; accepted in final form April 8, 2015 )
Supported by the Sharpe-Strumia Research Foundation.
Author Contributions: W.T.M. takes responsibility for the content of the study as a whole, including study design, data collection, data analysis and
interpretation, and the writing of the manuscript. L.A.M. contributed substantially to the study design, data analysis and interpretation, and the writing of the
manuscript. E.B., H.I.L., A.M., V.L., and S.R.A. contributed to data collection and interpretation and the writing of the manuscript. J.E.S. contributed to
statistical analysis, data interpretation, and the writing of the manuscript.
Correspondence and requests for reprints should be addressed to Wallace T. Miller, Jr., M.D., Department of Radiology, Perelman School of Medicine University
of Pennsylvania, Silverstein 1, 3400 Spruce Street, Philadelphia PA 19104. E-mail: millerw@uphs.upenn.edu
Ann Am Thorac Soc Vol 12, No 7, pp 1022–1029, Jul 2015
Copyright © 2015 by the American Thoracic Society
DOI: 10.1513/AnnalsATS.201502-105OC
Internet address: www.atsjournals.org

1022 AnnalsATS Volume 12 Number 7 | July 2015


ORIGINAL RESEARCH

Pulmonary embolism is the third most a 5-point scale: 1, definitely no trunk, right/left pulmonary artery, lobar
commonly diagnosed acute cardiovascular pulmonary embolism; 2, probably pulmonary artery, segmental pulmonary
disease after myocardial infarction and no pulmonary embolism; 3, artery, subsegmental pulmonary artery) in
cerebrovascular accident (1). The signs and indeterminate pulmonary embolism; 4, which the embolus was identified. The
symptoms of pulmonary embolism are not probable pulmonary embolism; 5, definite short-axis diameter of the largest pulmonary
specific, and clinicians rely on imaging pulmonary embolism. Reports with embolism was measured by electronic
examinations for the diagnosis. Computed descriptions like “multiple pulmonary calipers. Each study was scored for technical
tomography (CT) pulmonary angiography emboli present” or “small filling defect quality using a 5-point scale: 5, excellent
is fast, noninvasive, and widely available, consistent with a pulmonary embolism” quality (no technical limitation); 4,
and has emerged as the modality of choice were scored as a “5” (definite pulmonary good quality (minor technical limitations
for the diagnosis of pulmonary embolism embolism). Reports using qualifiers such as not likely to affect the detection of
(2, 3). “probable,” “suspicious for,” or “likely to pulmonary embolism); 3, moderate quality
Prior studies have suggested that the represent” when describing a defect were (technical defects that could affect the
interobserver agreement of the presence scored as a “4” (probable pulmonary detection of small pulmonary embolism);
of a pulmonary embolism is dependent embolism). Reports using the terms “could 2, poor quality (examination in which only
on the location of the embolus (central represent” or “possibly representing” large central pulmonary embolism can
vs. peripheral) and that there is a high pulmonary embolism were scored as “3” reliably be detected); and 1, nondiagnostic
false-positive rate in the diagnosis of (indeterminate pulmonary embolism). study. This initial review suggested a high
subsegmental pulmonary arteries (2, 3). The Reports with qualifiers such as “probably false-positive rate in the diagnosis of
purpose of this study was to investigate not” or “not likely to represent” pulmonary pulmonary embolism for peripheral and/or
interobserver agreement as a function of embolism were scored as “2” (probably no small pulmonary artery defects. Additional
lesion size and location and how it affects pulmonary embolism). If the report noted radiologist review was sought to buttress
the reliability of a diagnosis of pulmonary no pulmonary embolism but expressed the conclusions concerning false-positive
embolism as identified by CT pulmonary a limitation in interpretation based on diagnosis.
angiograms in routine clinical practice. suboptimal quality of the study, the report Three additional thoracic radiologists
was scored as a “2.” Reports with strong with 10 (Reader 2), 8 (Reader 3), and 2
descriptors of the absence of pulmonary (Reader 4) years of subspecialty experience
Methods embolism, such as “no evidence of reviewed a subset of the CT pulmonary
pulmonary embolism” or “no pulmonary angiogram examinations. We attempted to
Study Design embolism,” were scored as “1” (definite include all segmental and subsegmental
The study was approved by the center’s absence of pulmonary embolism). lesions in the second review both because
Institutional Review Board (Main Line Of the 7,900 examinations, a total our initial review indicated a problem with
Health IRB: F/N-R09-2783BLP) and of 728 examinations were interpreted as small, peripheral lesions and because prior
is Health Insurance Portability and a score of 4 to 5 for pulmonary embolism studies have indicated lower interobserver
Accountability Act compliant. Informed and represented our study group. agreement for more peripheral pulmonary
consent was waived due to the retrospective Examinations that were interpreted as emboli (2, 3). We were able to include
nature of the project. Using the keywords inadequate to evaluate for pulmonary 177 of 179 subsegmental and 192 of 195
“pulmonary embolism/embolus/emboli,” embolism (106), interpreted has having no segmental lesions. Five lesions could not be
the radiology information system was pulmonary embolism (score 1–2, 7,035 successfully transferred to an independent
searched for CT pulmonary angiograms examinations), or interpreted as being workstation where the second review was
performed from January 1, 2006 to indeterminate for pulmonary embolism performed and were excluded. We included
December 31, 2008 at three suburban (score 3, 31 examinations) were excluded. all examinations with main and lobar
community hospitals in the mid-Atlantic A thoracic radiologist (Reader 1) with pulmonary embolism that were scored as
region of the United States. We identified 19 years of subspecialty experience reviewed a 1 to 3 for likelihood of PE by Reader 1
8,414 reports that fulfilled criteria. We all 728 CT pulmonary angiograms for the and a random sampling of the remainder,
excluded 514 examinations from further presence of pulmonary embolism and including 19 of 126 (15%) lesions of the
review because they were: (1) non– evaluated the specific lesions that were main pulmonary arteries and 27 of 228
pulmonary embolism protocol studies identified by the primary reader. In all cases, (12%) lesions of the lobar pulmonary
(n = 369), (2) repeat examinations (n = 140), the location of the presumed embolus was arteries. Readers 2 to 4 randomly reviewed
or (3) thrombus within the stump of the identified in the report, by slice location the subset and scored both the likelihood of
pulmonary artery after lobectomy (n = 5). and lobe, by electronic arrows or circles, by pulmonary embolism and the quality of
One of the authors (A.M.) reviewed the pulmonary segment, or in some cases by the examination, using our 5-point scales.
the reports of the remaining 7,900 CT more general terms like “multiple small A composite study quality was derived by
pulmonary angiograms for the following: emboli in both lung bases.” averaging the study quality across the four
(1) presence or absence of pulmonary Reader 1 scored each study for the reviewers.
embolism, (2) location of pulmonary embolism, likelihood of pulmonary embolism using Each imaging examination was scored
and (3) confidence of pulmonary the same 5-point scale and identified the as positive, negative, or indeterminate for
embolism. The confidence of pulmonary largest pulmonary embolism based on the the presence of pulmonary embolism by the
embolism in the report was scored on generation of pulmonary artery (pulmonary five readers (the original report and the four

Miller, Marinari, Barbosa, et al.: Small Pulmonary Artery Defects and Predicting PE 1023
ORIGINAL RESEARCH

reviewers). A score of a “4” or “5” was Table 1. Locations of the largest pulmonary artery defect in the original and second
considered positive for pulmonary review computed tomography studies
embolism; a score of “1” or “2” was
considered negative for pulmonary Location Original Dataset Second Review
embolism, and a score of “3” was Subset
considered indeterminate for pulmonary
embolism. If four of the five readers agreed Main 127 (17) 19 (4.6)
on the presence of pulmonary embolism, Lobar 228 (30) 27 (6.5)
then the examination was considered Segmental 207 (27) 192 (46)
positive for pulmonary embolism. Subsegmental 197 (26) 177 (43)
Total 759 415
For purposes of evaluating the
community radiologist, if four of four Data are presented as n (%).
thoracic radiologists agreed on the absence
of pulmonary embolism, then the
examination was considered negative for Statistical Analysis exact test, with control for multiple
pulmonary embolism and a false-positive The data were subsequently imported testing using Bonferroni correction. All
interpretation by the community into the R computing environment for P values reported are two sided, with
radiologist. For purposes of evaluating statistical analysis (4). Several metrics a P value less than 0.05 considered
the thoracic radiologists, if three of four of interrater reliability were calculated significant.
thoracic radiologists agreed on the absence on the sample as a whole and after
of pulmonary embolism, the examination subsampling by pulmonary embolism
was considered negative for pulmonary size, location, and CT pulmonary Results
embolism and a false-positive angiogram scan quality. First, the simple
interpretation for the dissenting thoracic fraction of cases with complete or Community Radiologists
radiologist. near-complete agreement (e.g., five- or The radiology group that initially
four-rater agreement) was calculated. interpreted the studies was a practice
CT Examinations Interrater reliability was estimated using covering three suburban, northeastern
CT examinations were performed on four Randolph’s Free Marginal Multirater United States hospitals. Forty-two
multidetector spiral CT scanners (GE kappa, which accounts for differences participating community staff radiologists
Medical Systems, Milwaukee, WI): Light in group prevalence (5, 6). Ninety-five each interpreted between 3 and 35 CT
speed QXi (n = 22), Lightspeed Ultra percent confidence intervals were pulmonary angiogram examinations. Of the
(n = 190), Light speed 16 (n = 116), and calculated via bootstrap using 1,000 42 radiologists, all were board certified and
Lightspeed VCT (n = 87). Ninety-four replicates. Fisher exact test was used to 31 had subspecialty training (7 body
percent (n = 708) were performed with compare differences in agreement based imaging, 6 interventional radiology, 7
a slice thickness of 1.25 mm and 6% on pulmonary embolism size, location, neuroradiology, 5 breast imaging, 4
(n = 46) with a slice thickness of 2.5 mm. and scan quality. Post hoc pairwise pediatric radiology, 2 nuclear medicine). Six
Intravenous contrast used included: differences also were assessed with Fisher board-certified moonlighting radiologists
ioversol 320 mg I/mL (Optiray 320;
Covidien, Dublin, Ireland) (n = 27), ioversol
350 mg I/mL (Optiray 350; Covidien) Table 2. Agreement for pulmonary embolism as a function of pulmonary embolism
(n = 315), and iodixanol 320 mg I/mL location, pulmonary embolism diameter, and study quality
(Visipaque 320; GE Healthcare, Milwaukee,
WI) (n = 51). In 22 cases, the type of 5 of 5 4 of 5 Indeterminate* Kappa
contrast was not recorded. Readers Agree Readers Agree

Image Review Location


Evaluation by the community radiologists Main (n = 19) 17 (89) 2 (11) 0 0.94
Lobar (n = 27) 24 (89) 2 (7.4) 1 (3.7) 0.92
and Reader 1 was performed using the Segment (n = 192) 169 (88) 12 (6.3) 11 (5.7) 0.90
picture archiving system (McKesson Subsegment (n = 177) 70 (40) 59 (33) 48 (27) 0.53
Radiology, McKesson, San Francisco, CA). Diameter
Images were available in the axial, sagittal, .9.0 (N = 42) 40 (95) 0 2 (4.8) 0.95
and coronal planes. Readers 2 through 4 6.1–9.0 (N = 99) 87 (88) 8 (8.1) 4 (4.0) 0.91
3.1–6.0 (N = 207) 133 (64) 41 (20) 33 (16) 0.72
reviewed examinations on a thin-client 0–3.0 (N = 67) 20 (30) 26 (39) 21 (31) 0.46
viewing platform (AQNet; Terarecon, Study quality
Foster City, CA) where Digital Imaging and 4.0–5.0 (N = 251) 198 (79) 31 (12) 22 (8.7) 0.84
Communications in Medicine (DICOM)- 3.0–3.9 (N = 151) 81 (54) 38 (25) 32 (21) 0.63
2.0–2.9 (N = 13) 1 (7.7) 6 (46) 6 (46) 0.22
formatted images were loaded, patient-
specific information was removed, and the Numbers in parentheses are percentages.
patient order was randomized. *Fewer than four of five readers agreeing on the presence or absence of pulmonary emboli.

1024 AnnalsATS Volume 12 Number 7 | July 2015


ORIGINAL RESEARCH

interpreted 12 examinations (1.6%). The A


group practice had a diagnostic radiology 1.0

Free Marginal Multirater Kappa


residency. Of the 759 examinations, 261
studies (35%) were reviewed by 26 residents 0.8
in conjunction with staff radiologists.
0.6
Location of Pulmonary Artery Lesions
The locations of pulmonary artery lesions in 0.4
the original and second review subsets are
listed in Table 1. In the original dataset, 0.2
locations of pulmonary artery lesions were
nearly uniformly distributed between
0.0
central and peripheral locations. By study
design, the second review subset was highly Main Lobar Segmental Subsegment
skewed toward smaller lesions in the PE Location
segmental and subsegmental arteries.
B
1.0
Free Marginal Multirater Kappa
Agreement among Readers
Table 2 reports agreement among readers
0.8
based on pulmonary embolism location,
pulmonary embolism diameter, and study
0.6
quality. The frequency of agreement
decreased and the proportion of
indeterminate lesions increased with more 0.4
peripheral location of the pulmonary artery
defect. 0.2
Kappa scores typically are rated on the
following scale: poor (0.00), slight (0.01– 0.0
0.20), fair (0.21–0.40), moderate (0.41– >9 6–9 3–6 <3
0.60), substantial (0.61–0.80), or almost PE size (mm)
perfect (0.81–1.0) agreement. By this scale,
agreement among readers was almost C
perfect for main, lobar, and segmental 1.0
Free Marginal Multirater Kappa

emboli. However, there was notable fall


in agreement with subsegmental lesions, 0.8
which was only moderate, with a k = 0.53
(Figure 1A). Five-rater agreement was 0.6
significantly different based on lesion
location (P , 0.0001). Post hoc analysis 0.4
demonstrated significant differences
between subsegmental lesions and all other 0.2
locations (P , 00001), with other pairwise
group comparisons not reaching statistical
0.0
significance (P . 0.9999). There was lower
five-reader agreement for subsegmental High Intermediate Low
lesions than for larger branches (40% Scan Quality
[70/177] vs. 88% [210/238], P , 0.0001).
Similarly, subsegmental lesions were judged Figure 1. (A) Agreement among five readers as a function of lesion location. (B) Agreement as
as indeterminate for pulmonary embolism a function of pulmonary embolism size. (C) Agreement as a function of study quality. Bars are the
95% confidence intervals. PE = pulmonary embolism.
more commonly than for larger vessels
(27% [48/177] vs. 5% [12/238], P , 0.0001).
The frequency of agreement decreased differences in agreement by size for the difference of agreement for lesions 6 to
and the indeterminate fraction increased entire population (P , 0.0001). There was 9 mm and smaller size groups (P , 0.0001).
as the short-axis diameter of the defect a significant difference between agreement There was a significant difference of
decreased (Table 2). Agreement regarding for lesions less than 3 mm and all other agreement for lesions greater than 9 mm
lesions with a short axis greater than 6 mm size groups (P , 0.0001). There was and groups of less than 6 mm (P , 0.0001).
was nearly perfect, but fell for lesions with a significant difference of agreement for However, there was not a significant
short axis of 6 mm or smaller (Figure 1B). lesions 3 to 6 mm and all other size groups difference in agreement between emboli of
There were statistically significant group (P , 0.0001). There was a significant size 6 to 9 mm and emboli of size greater

Miller, Marinari, Barbosa, et al.: Small Pulmonary Artery Defects and Predicting PE 1025
ORIGINAL RESEARCH

than 9 mm. Five-reader agreement for Table 3. Frequency of false-positive interpretations by individual readers as a function
lesions with a short-axis diameter of less of location of pulmonary embolism, diameter of pulmonary embolism, and study quality
than or equal to 6 mm was significantly
lower than larger lesions (56% [153/274] vs. R1 R2 R3 R4 Community Radiologist (%)
90% [127/141], P , 0.0001), and frequency
of indeterminate lesions was increased Location
(20% [54/274] vs. 4% [6/141], P , 0.0001). Main (N = 19) 0 0 0 0 2
The frequency of agreement decreased Lobar (N = 27) 0 0 0 0 2
with decreasing quality of examination Segment (N = 192) 0 1 (0.5) 2 (1.0) 0 7 (3.6)
Subsegment (N = 177) 7 (4.0) 1 (0.6) 2 (1.1) 0 27 (15)
(P , 0.0001), with post hoc analysis Size
demonstrating significant differences in .9.0 (N = 42) 0 0 0 0 0
five-rater agreement between all pairwise 6.1–9.0 (N = 99) 0 0 1 (1.0) 0 1 (1.0)
levels of scan quality (P = 0.0014 to 3.1–6.0 (N = 207) 3 (1.4) 0 2 (1.0) 0 26 (13)
0–3.0 (N = 67) 4 (6.0) 2 (3.0) 1 (1.5) 0 11 (16)
,0.0001) (Table 2). Agreement regarding Study quality
lesions in studies with a quality of 4 4.0–5.0 (N = 251) 4 (0.4) 0 4 (0.4) 0 8 (0.3)
(“minor technical limitations not likely 3.0–3.9 (N = 151) 3 (2.0) 2 (1.3) 0 0 25 (17)
to affect the detection of pulmonary 2.0–2.9 (N = 13) 0 0 0 0 5 (38)
embolism”) or better was substantial
False positive for community radiologist is defined as four of four thoracic radiologists interpreted
(k = 0.84) but fell progressively with qualities the examination as negative for pulmonary embolism. False positive for the thoracic radiologists was
less than 4 (Figure 1C). Agreement of all defined as three of three other thoracic radiologists interpreted the examination as negative for
five readers for lesions on examinations pulmonary embolism. R1, R2, R3, R4 indicate thoracic radiology reviewers 1 through 4.
of quality greater than or equal to 4 was
significantly better than examinations of
lower quality (79% [198/251] vs. 50% [82/ 0.7% [1/141]). Furthermore, of the highest- lesions, and motion artifact and mucous
164], P , 0.0001), and the indeterminate quality examinations (quality score > 4), plug are typically confused with peripheral
fraction was less than those of lower quality 0.3% (8/251) were false positive compared lesions.
(9% [22/251] vs. 23% [38/164], P , 0.0001). with 18% (30/164; P , 0.0001) in those
with lower quality (quality score , 4). Accuracy of Thoracic Radiologists
Accuracy of Interpretations by Motion artifact (Figure 2) accounted There was a range of false-positive rates
Community Radiologists for 50% (19/38) of false-positive lesions among chest radiologists, with a low of 0%
Community radiologists were prone to (Table 4). Other less-common causes false-positive interpretations to a maximum
false-positive diagnosis (Table 3). The included streak artifact, pulsation artifact, of 1.7% (7/415) false-positive interpretations
consensus diagnosis identified 38 of 759 mucous plug in a bronchus adjacent to the of pulmonary embolism. The likelihood of
examinations (5.0%) as false positive for pulmonary artery, and hilar lymph nodes a false-positive interpretation increased with
pulmonary embolism. The probability of (Figure 3). The number of causes other more peripheral location and decreasing
a false-positive diagnosis increased with (1) than motion artifact are small but suggest short-axis diameter of the lesion. A
more peripheral location of the lesion, (2) that streak artifact and lymphadenopathy maximum of 4.0% (7/177) of subsegmental
decreased short-axis diameter of the lesion, are typically confused with centrally located lesions compared with 0.8% (2/238,
and (3) decreasing quality of the CT
examination (Table 3).
False-positive reports were more likely
to describe subsegmental lesions than more
central lesions (15% [27/177] vs. 4.6%
[11/238]; P = 0.0002). Because not all main
and lobar emboli examinations were
reviewed by the entire panel, we cannot
absolutely determine the positive predictive
values for all locations. However, assuming
that Reader 1 identified all false-positive
pulmonary embolisms in the main and
lobar arteries, the positive predictive values
for locations of emboli were 98.4% for
main pulmonary emboli, 99.1% for lobar
pulmonary emboli, 96.6% for segmental
pulmonary emboli, and only 86.3% for
subsegmental emboli. Figure 2. This 46-year-old woman complained of chest pain. (A) Computed tomography pulmonary
The false-positive rate of lesions less angiogram through the left lower lobe shows apparent filling defects (arrows) in several subsegmental
than or equal to 6 mm was greater than arteries. (B) Lung windows of the same slice show moderate blurring of the pulmonary vessels
lesions greater than 6 mm (14% [37/274] vs. indicating motion artifact as the most likely cause for the lesions seen in the pulmonary arteries.

1026 AnnalsATS Volume 12 Number 7 | July 2015


ORIGINAL RESEARCH

P , 0001) of central lesions were false Table 4. Causes of 38 false-positive examinations as judged by thoracic radiologists
positive for pulmonary embolism. Six
percent (4/67) of lesions with short axis less Location Motion Streak Pulsation Mucous Plug Lymph Node No Agreement
than or equal to 3 mm were false positive
compared with 0.9% (3/348, P = 0.015) of Main 0 1 0 0 1 0
lesions with short axis greater than 3 mm. Lobar 0 1 0 0 0 1
Scan quality differences were not associated Segment 3 1 0 0 0 3
with false-positive interpretations by Subsegment 16 0 1 2 0 8
Total 19 (50) 3 (7.8) 1 (2.6) 2 (5.2) 1 (2.6) 12 (32)
thoracic radiologists.
There was a range of false-negative Numbers in parenthesis are percentages. Cause established if more than three readers identified the
rates among chest radiologists, with a low same cause. Fewer than three readers classified as no agreement on cause of false positive.
of 0% false-negative interpretations to
a maximum of 4.8% (20/415) incidence
false-negative interpretations of pulmonary 26 proximal vessels, but is impossible to study quality, analogous to decreases in
embolism (Table 5). The likelihood of directly inspect each subsegmental artery. agreement among radiologists.
a false-negative diagnosis increased with Radiologists rely on “peripheral vision” to Of subsegmental pulmonary emboli
more peripheral location and decreasing draw their eye to subsegmental lesions, with identified by community radiologists, 27%
short-axis diameter of the pulmonary artery relatively low accuracy. (48/177) were judged to be indeterminate
lesion. As high as 11% (19/177) of Our thoracic radiologist reviewers and 15% (27/177) false positive by
subsegmental lesions compared with 0.8% rated the quality of the examination based consensus diagnosis. In most cases, false-
(2/238, P , 0.0001) of central lesions were on a subjective assessment of how the positive lesions represented artifacts,
false negative for a thoracic radiologist, and quality of the examination might influence primarily motion artifact, and streak artifact
up to 16% (11/67) of lesions with a short the detection of pulmonary embolism. We (Figure 2). Hilar lymph nodes and small
axis less than 3 mm compared with 2.6% found that they could detect falls in quality peripheral mucous plugs were also
(9/348, P , 0.0001) of larger lesions were that would result in errors of pulmonary confused with pulmonary embolism (7, 8)
missed by a thoracic radiologist. embolism detection. Agreement about the (Figure 3). Because the majority of false-
presence of pulmonary embolism fell as positive lesions represented artifacts, it is
radiologists identified decreasing quality of not surprising that decreasing quality of the
Discussion examinations. examination led to increasing incidence of
Compared with thoracic radiologists, false-positive interpretations.
Our study indicates that the interpretation community radiologists were more likely to Several other studies have described
of subsegmental and/or small pulmonary identify false-positive defects as pulmonary the inaccuracies associated with small CT
artery defects has a high rate of inaccuracy embolism. The probability of a false-positive pulmonary angiography lesions. Stein and
among both community radiologists examination was directly correlated with colleagues reported a false-positive rate of
and academic thoracic radiologists. subsegmental location and small size of the 32% (15/47) of segmental defects and 75%
Interobserver agreement fell relatively defect and was inversely correlated with (6/8) of subsegmental defects (3). Among
continuously as the lesion diameter
decreased (Figure 1B). Using post hoc
review by a panel of academic chest
radiologists to define a “gold standard,” Table 5. Frequency of false-negative interpretations by individual readers as a function
this result suggests that there is a direct of location of pulmonary embolism, diameter of pulmonary embolism, and study quality
correlation between the size of a pulmonary
embolism and the reliability of detection. R1 R2 R3 R4
By necessity, subsegmental pulmonary
emboli must be small in size because of Location
the small caliber of the vessel. However, Main (N = 19) 0 0 0 0
interobserver agreement dropped sharply Lobar (N = 27) 0 0 0 0
Segment (N = 192) 0 0 3 (1.5) 1 (0.5)
from segmental to subsegmental pulmonary Subsegment (N = 177) 0 8 (4.5) 3 (1.7) 19 (11)
artery lesions (Figure 1A), suggesting that Size
additional factors other than lesion size .9.0 (N = 42) 0 0 0 0
contribute to inaccuracy in detection of 6.1–9.0 (N = 99) 0 1 (1.0) 3 (3.0) 2 (2.0)
subsegmental pulmonary embolism. We 3.1–6.0 (N = 207) 0 4 (1.9) 2 (0.9) 7 (3.3)
0–3.0 (N = 67) 0 3 (4.5) 1 (1.5) 11 (16)
suspect that this factor is the sheer number Study quality
of subsegmental vessels. There is one 4.0–5.0 (N = 251) 0 4 (1.6) 4 (1.6) 14 (5.6)
pulmonary trunk, two main pulmonary 3.0–3.9 (N = 151) 0 4 (2.6) 2 (1.3) 6 (4.0)
arteries, five lobar arteries, and 18 2.0–2.9 (N = 13) 0 0 0 0
Total (N = 415) 0 8 (1.9) 6 (1.4) 20 (4.8)
segmental arteries, and hundreds of visible
subsegmental pulmonary arteries. It is Numbers in parenthesis are percentages. Note: by study design, Reader 1 could have no false
a reasonable task to look at each of the negatives.

Miller, Marinari, Barbosa, et al.: Small Pulmonary Artery Defects and Predicting PE 1027
ORIGINAL RESEARCH

“The imaging exam is degraded by


motion artifact, and emboli can only be
reliably interpreted to the level of the
segmental arteries.”
The primary limitation of this study is the
absence of definitive proof of pulmonary
embolism. We have relied on a five-radiologist
panel as the arbiter of “truth.” A second
limitation is the evaluation of a single
community radiology group. It is unknown if
these results apply to practicing radiologists in
general; however, there is no obvious reason,
other than random variation, why this group
would be different from other radiologists.
Our community radiologists were
evaluating clinical cases, whereas our
Figure 3. This 70-year-old woman with a history of breast cancer had progressive dyspnea,
thoracic radiologists were involved in
diaphoresis, and lightheadedness. (A) The small low attenuation lesion (arrow) in the right lower lobe
was interpreted initially as representing a small subsegmental pulmonary embolus. (B) Sagittal oblique
a research study. Thus, the consequences of
reconstructions of the lesion show it to represent a small mucus plug. a false-negative study were higher for the
community radiologists than the thoracic
radiologists, which may have led to a greater
patients with a low clinical probability of reviewers knew that another radiologist had probability of community radiologist calling
pulmonary embolism, CT pulmonary already diagnosed a pulmonary embolism a pulmonary artery defect a positive lesion.
angiography had a 42% false-positive rate and they were looking more diligently for Our study did not review the 6,095 CT
(3). Perrier and colleagues demonstrated small lesions. This study was not designed pulmonary angiography examinations
a false-positive rate of 0% in the main to measure the false-negative rate of interpreted as negative for pulmonary
pulmonary artery, 15% in the lobar identifying pulmonary emboli but indicates embolism; therefore, we have no measure of
pulmonary arteries, and 38% in the that even experienced readers can miss the incidence of false-negative studies. We
segmental pulmonary arteries (2). a significant percentage of small peripheral selected this design because it was our intent
Since the emergence of CT pulmonary emboli. Our study suggests that both to study “overdiagnosis” rather than
angiography as the primary diagnostic small short-axis diameter and peripheral “underdiagnosis” of pulmonary embolism.
study for pulmonary embolism, there has location play a role in false-negative Many recent studies have demonstrated
been a near doubling in the detection interpretations. that overdiagnosis is likely to be a greater
of pulmonary embolism without Given our results, we suggest three problem than underdiagnosis in the
a corresponding decrease in mortality (8). policies regarding subsegmental pulmonary interpretation of CT pulmonary
Some have suggested that this phenomenon artery defects and small lesions in any angiograms (2, 6, 7, 9–13).
is due to the detection of small clinically location. In conclusion, our study indicates that
insignificant pulmonary embolism (2, 9– 1. Subsegmental pulmonary artery defects there is often lack of agreement among
16). Our study suggests the increase in have a high probability of inaccurate radiologists concerning subsegmental and/
detection of pulmonary embolism may in interpretation. A diagnosis of pulmonary or small vascular defects in CT pulmonary
part be due to false-positive diagnosis. embolism should be made only when angiograms. Furthermore, a review by
Although our thoracic radiologists several readers agree that the lesion(s) a panel of thoracic radiologist specialists
performed better than the community represent pulmonary embolism. suggests that community radiologists have
radiologists at detecting pulmonary 2. Small defects (,6 mm short axis) a tendency to falsely interpret imaging
embolism, their performance was also identified on lower-quality examinations artifacts as subsegmental and/or small
adversely affected by subsegmental location have a high probability of inaccurate pulmonary emboli. A confident diagnosis of
and/or small size. Up to 4.0% (7/177) of interpretation. A diagnosis of pulmonary pulmonary embolism for subsegmental or
subsegmental lesions identified by one embolism should be made only when small pulmonary artery defects should only
thoracic radiologist were judged false several readers agree that the lesion(s) be made if multiple readers agree on the
positive by the remaining three thoracic represent pulmonary embolism. presence of an embolism. Diminished
radiologists, and up to 10% (19/195) of 3. Radiologists should report the quality of examination quality, especially as a result
subsegmental emboli determined by imaging examinations. We believe that of motion artifact or poor vascular
consensus as positive for pulmonary this is best reported by indicating to opacification, increases the likelihood of
embolism were read as negative what generation of pulmonary arteries inaccurate interpretations for peripheral
examinations by one thoracic radiologist. (main, lobar, segmental, subsegmental) and/or small pulmonary embolism. n
The false-negative rate is likely an the interpreting radiologist can reliably
underrepresentation of the true rate of confirm or exclude a diagnosis of Author disclosures are available with the text
false-negative examinations because our pulmonary embolism. For example: of this article at www.atsjournals.org.

1028 AnnalsATS Volume 12 Number 7 | July 2015


ORIGINAL RESEARCH

References 9 Wiener RS, Schwartz LM, Woloshin S. Time trends in pulmonary


embolism in the United States: evidence of overdiagnosis. Arch
1 British Thoracic Society Standards of Care Committee Pulmonary Intern Med 2011;171:831–837.
Embolism Guideline Development Group. British Thoracic Society 10 Ryu JH, Olson EJ, Pellikka PA. Clinical recognition of pulmonary
guidelines for the management of suspected acute pulmonary embolism: problem of unrecognized and asymptomatic cases. Mayo
embolism. Thorax 2003;58:470–483. Clin Proc 1998;73:873–879.
2 Perrier A, Roy PM, Sanchez O, Le Gal G, Meyer G, Gourdier AL, 11 Musset D, Parent F, Meyer G, Maı̂tre S, Girard P, Leroyer C, Revel MP,
Furber A, Revel MP, Howarth N, Davido A, et al. Multidetector-row Carette MF, Laurent M, Charbonnier B, et al.; Evaluation du Scanner
computed tomography in suspected pulmonary embolism. N Engl J Spiralé dans l’Embolie Pulmonaire study group. Diagnostic strategy
Med 2005;352:1760–1768. for patients with suspected pulmonary embolism: a prospective
3 Stein PD, Fowler SE, Goodman LR, Gottschalk A, Hales CA, Hull RD, multicentre outcome study. Lancet 2002;360:1914–1920.
Leeper KV Jr, Popovich J Jr, Quinn DA, Sos TA, et al.; PIOPED II 12 Ghaye B, Szapiro D, Mastora I, Delannoy V, Duhamel A, Remy J, Remy-
Investigators. Multidetector computed tomography for acute Jardin M. Peripheral pulmonary arteries: how far in the lung does
pulmonary embolism. N Engl J Med 2006;354:2317–2327. multi-detector row spiral CT allow analysis? Radiology 2001;219:
4 R Development Core Team. R: a language and environment for 629–636.
statistical computing. Vienna, Austria; 2014. 13 Schoepf UJ, Goldhaber SZ, Costello P. Spiral computed tomography
5 Randolph JJ. Free-marginal multirater kappa: an alternative to Fleiss’ for acute pulmonary embolism. Circulation 2004;109:2160–2167.
fixed marginal multirater kappa. Joensuu University Learning and 14 Carrier M, Righini M, Wells PS, Perrier A, Anderson DR, Rodger MA,
Instruction Symposium, 2005. Pleasance S, Le Gal G. Subsegmental pulmonary embolism diagnosed
6 Dahiya A, Bolen M, Grimm RA, Rodriguez LL, Thomas JD, Marwick TH; by computed tomography: incidence and clinical implications. A
AR Concordance Investigators. Development of a consensus systematic review and meta-analysis of the management outcome
document to improve multireader concordance and accuracy of studies. J Thromb Haemost 2010;8:1716–1722.
aortic regurgitation severity grading by echocardiography versus 15 Coche E, Verschuren F, Keyeux A, Goffette P, Goncette L, Hainaut P,
cardiac magnetic resonance imaging. Am J Cardiol 2012;110: Hammer F, Lavenne E, Zech F, Meert P, et al. Diagnosis of acute
709–714. pulmonary embolism in outpatients: comparison of thin-collimation
7 Beigelman C, Chartrand-Lefebvre C, Howarth N, Grenier P. Pitfalls in multi-detector row spiral CT and planar ventilation-perfusion
diagnosis of pulmonary embolism with helical CT angiography. AJR scintigraphy. Radiology 2003;229:757–765.
Am J Roentgenol 1998;171:579–585. 16 Eyer B, Goodman L, Washington L. Clinicians’ response to radiologists’
8 Remy-Jardin M, Remy J, Artaud D, Deschildre F, Fribourg M, Beregi JP. reports of isolated subsegmental pulmonary embolism or
Spiral CT of pulmonary embolism: technical considerations and inconclusive interpretation of pulmonary embolism using MDCT.
interpretive pitfalls. J Thorac Imaging 1997;12:103–117. AJR 2005;184:623–628.

Miller, Marinari, Barbosa, et al.: Small Pulmonary Artery Defects and Predicting PE 1029