Clinical Imaging 48 (2018) 12–16

Contents lists available at ScienceDirect

Clinical Imaging
journal homepage: www.elsevier.com/locate/clinimag

Case based simulation in MRI for suspected appendicitis in children☆ T

a,⁎ b c d
Hansel J. Otero , Anilawan Smitthimedhin , Crystal C. Wang , Mark D. Heitzmann
a
Department of Radiology, Children's Hospital of Philadelphia, 3401 Civic Center blvd, Philadelphia, PA 19104, United States
b
Department of Diagnostic Imaging and Radiology, Children's National Health System, 111 Michigan Avenue, NW, Washington, DC 20010, United States
c
Department of Economics, Princeton University, Princeton, NJ 08540, United States
d
Naval Hospital Camp Pendleton, Santa Margarita rd, bldg h100, Camp Pendleton, CA 92055, United States

A R T I C L E I N F O A B S T R A C T

Keywords: Purpose: To establish the effect on diagnostic confidence of a simulation setting, in which radiologists re-in-
MRI terpret anonymized pediatric MRI cases.
Appendicitis Materials: In this IRB-approved study, participants completed surveys rating confidence before and after inter-
Children preting 10 MRI cases for suspected appendicitis in children.
Simulation
Results: 18 radiologists (4 faculty, 5 fellows, and 9 residents) correctly identified an average of 7.44 cases (range
Diagnostic confidence
5–9). Self-described confidence regarding technique and interpretation increased from 2.0 (SD 0.77) and 2.33
(SD 0.69) to 2.83 (SD 0.71) and 2.94 (SD 0.73), respectively.
Conclusion: Simulated interpretation of pediatric MRI in suspected appendicitis results in increased self-describe
confidence without requiring additional capital/equipment expenses.

1. Introduction sought to establish the effect on perceived diagnostic confidence of a

Magnetic resonance imaging (MRI) is an established imaging mod-
ality for the detection and evaluation of diseases with an increasing
demand in the pediatric emergency setting [1,2].
MRI offers a safer alternative to CT, and is central to strategies that
try to limit the exposure of children to ionizing radiation. In acute
appendicitis, the most common condition for urgent abdominal surgery
in children, ultrasound is the preferred initial imaging modality fol-
lowed by CT for inconclusive cases [3]. In the last few years, alternative
imaging algorithms that incorporate magnetic resonance imaging (MRI)
have been proposed [4–6]. MRI has been reported to be accurate for
acute appendicitis in children and does not change time to antibiotic
administration, time to appendectomy, negative appendectomy rate,
perforation rate, or length of stay compared to CT [4,7–9]. However,
plans to adopt MRI for suspected appendicitis in clinical practice might
be hindered by unfamiliarity with the technology by the interpreting
radiologist or by the residents and fellows, who operate independently
after hours.
We developed a workstation simulation format for radiologists and
radiologists-in-training, in which they re-interpret anonymized MRI
studies in children with suspected appendicitis as if it were firsthand.
The interpretation was followed by a “read out” session for trainees and
key-image responses for faculty to replicate on-the-job learning. We
simulation-based curriculum
2. Method and materials
This HIPAA compliant study was approved by our institution review
board. We evaluated diagnostic performance and self-reported compe-
tence level on interpreting MRI for suspected appendicitis of radi-
ologists and radiologists-in-training at a single stand-alone pediatric
training hospital.
All participants were distributed a video lecture of MR for suspected
appendicitis and completed pre-test surveys rating competence on their
technical and interpretative skills regarding MRI for appendicitis using
the following five-point scale: 1. Novice, 2. Advanced Beginner, 3.
Competent, 4. Proficient, and 5. Expert. The categories were adapted
from Benner's stages of clinical competence [10], as follows: Novice: No
previous experience, lacks confidence and requires continual cues from
supervisor. Advanced Beginner: marginally acceptable performance
because of prior experience but requires occasional support. Compe-
tent: competence and efficiency is demonstrated. The reader is co-
ordinated and has confidence in his/her actions. Reports are completed
within a suitable time frame without supporting cues. Proficient: can
now recognize abnormalities, fit them into a category and develop a
plan without supervision. Expert: deep understanding of the situation.

☆
No conflict of interest to report.
⁎
Corresponding author.
E-mail address: Oteroh@email.chop.edu (H.J. Otero).

http://dx.doi.org/10.1016/j.clinimag.2017.09.012
Received 10 August 2017; Received in revised form 9 September 2017; Accepted 19 September 2017
0899-7071/ © 2017 Elsevier Inc. All rights reserved.
H.J. Otero et al. Clinical Imaging 48 (2018) 12–16

Fig. 1. Standard PACS viewer tool use during the simulation.

Fig. 2. Key images from anonymized study showing uncomplicated appendicitis (arrow) in a 12-year old boy, including axial T2-weighted images with and without fat saturation, coronal
T2-weighted image without fat-saturation, and axial Diffusion Weighted Imaging.

13
H.J. Otero et al.
Fig. 3. Key images from anonymized study showing right ovarian hemorrhagic cyst (arrow) in a 14-year old girl, including axial T2-weighted images with and without fat saturation,
coronal T2-weighted image with fat saturation, and axial Diffusion Weighted Imaging.
“Experts have automated most of the basic skills of observation, and
they fit observations into diagnostic schemata”. The expert radiologist
distinguishes between relevant and irrelevant information and novel
factors in the case. The expert will make more inferences from relevant
information and may be more reluctant than the novice to reject in-
formation that appears not to fit the puzzle.
The participants then completed the simulation test of 10 unknown
anonymized MRI cases following a suspected appendicitis protocol on
an image-sharing standard PACS viewer (LifeImage, Newton, Mass)
with a provided clinical history of “right lower quadrant pain”. The
cases were assigned to participants using the “share” feature, which
make the cases populate their personal “library” (Fig. 1). All partici-
pants were familiar with the software, which is used for uploading
outside studies needing a second interpretation or for later transfer to
the standard departmental PACS. Radiologists re-interpreted anon-
ymized MRI cases as if they were diagnosing firsthand rendering a final
impression of appendicitis or alternative diagnosis. Afterwards, there
was a one-on-one “read out” session with a faculty member for trainees
and key-image responses for faculty that replicates on-the-job learning.
After reviewing their reports, the participants were administered post-
test surveys re-rating their competence regarding MRI for appendicitis
using the same five-point scale as well as a feedback survey on the si-
mulation-based format. The feedback survey used a 5-point Likert scale
(i.e; from strongly disagree to strongly agree) for each of the following
statements: 1) This course helped to address an area needing im-
provement, 2) I found this activity easy to understand/navigate, 3) I
feel better prepare to interpret similar studies in the future, and 4)
Based upon what I have learned I plan to change how I report similar
studies.
All MRI studies were performed in a 1.5 Tesla magnet using a
standard body 12-channel phased array coil. The protocol included
14
Clinical Imaging 48 (2018) 12–16
axial and coronal ultrafast spin echo (Single Shot Fast Spin Echo-SSFSE)
with and without fat saturation, sagittal SSFSE with fat saturation as
well as axial diffusion weighted images. Patients, aged 5 to 17 years,
received no IV or oral contrast nor did they receive sedation or anxio-
lysis for the study. The MRI protocol takes approximately 8 to 11 min.
The final sample of diagnoses included: uncomplicated acute ap-
pendicitis (n = 3) (Fig. 2), perforated appendicitis with abscess
(n = 2), right ovarian hemorrhagic cyst (n = 1) (Fig. 3), acute pan-
creatitis (n = 1), right base pneumonia (n = 1) (Fig. 4), vaso-occlusive
disease in the setting of sickle cell anemia (n = 1), and acute pelvic
inflammatory disease (n = 1). Diagnoses were confirmed by pathology
(all cases of appendicitis), laboratory (pancreatitis and pelvic in-
flammatory disease), additional imaging (right lower lobe pneumonia)
or follow up (ovarian hemorrhagic cyst and sickle cell disease). The
number of correct diagnoses made was recorded for each participant.
Descriptive statistics (mean, standard deviation, and variance) were
calculated. Then, we compared differences between two groups using
two-tailed t-test. A p value of < 0.05 was considered statistically sig-
nificant.
3. Results
18 radiologists (4 pediatric radiology faculty, 5 pediatric radiology
fellows, and 9 rotating radiology residents) completed the exercise.
Overall, radiologists diagnosed correctly an average of 7.44 cases out of
10 (averages of 8.3, 8.0 and 6.8 for the attending, fellow and residents
groups, respectively; range from 5 to 9 correct diagnoses). Self-de-
scribed competence regarding technique and image interpretation in-
creased from an average of 2.0 (SD 0.77) and 2.33 (SD 0.69) (i.e.
Advanced Beginner) to an average of 2.83 (SD 0.71) and 2.94 (SD 0.73)
(i.e. competent), respectively (p = 0.001 and 0.01). See Table 1.
H.J. Otero et al. Clinical Imaging 48 (2018) 12–16

Fig. 4. Key images from anonymized study showing right lower lobe pneumonia (arrow) in a 7-year old boy, including axial T2-weighted images with and without fat saturation, coronal
and sagittal T2-weighted with fat saturation images.

Table 1
Pre-test and post-test scores for all participants.

Mean (SD) All Resident Fellow Faculty

What best describes your level of competence regarding “TECHNIQUE” in the presented pediatric Pre-test 2.0 (0.77) 1.67 (0.71) 1.8 (0.45) 3 (0)
topic? Post-test 2.83 2.56 (0.73) 3 (0.71) 3.25 (0.5)
(0.71)
Increased in confident 0.33 0.89 (0.6) 1.2 (0.45) 0.25 (0.5)
score
p 0.002 0.0185 0.0128 0.3559
What best describes your level of competence regarding “IMAGE INTERPRETATION (abnormality Pre-test 2.33 1.89 (0.6) 2.6 (0.58) 3 (0)
identification, description, reporting)” in the presented pediatric topic? (0.69)
Post-test 2.94 2.67 (0.71) 3 (0.71) 3.5 (0.58)
(0.73)
Increased in confident 0.61 0.78 (0.44) 0.4 (0.89) 0.5 (0.58)
score
p 0.01 0.0229 0.3578 0.1354

Statistical significance (P < 0.05) presented in bold.

Table 2 Self-described competence regarding technique and image inter-

Evaluation of the simulation exercise.
Do you need additional education on this topic?
This course helped to address an area needing improvement in my
training
I found this activity easy to understand/navigate
Do you feel better prepared to interpret this topic in your future
practice?
Based upon what I have learned I plan to change how I report
similar studies
pretation of the resident group increased from 1.67 (SD 0.71) and 1.89
(SD 0.6) to 2.56 (SD 0.72) and 2.67 (SD 0.71) (p = 0.01 and 0.02),
Mean (SD)
respectively. For fellows, self-described competence regarding tech-
3.89 (0.96) nique and image interpretation increased from 1.8 (SD 0.45) and 2.6
4.39 (0.50) (SD 0.58) to 3 (SD 0.71) and 3.0 (SD 0.71), (p = 0.01 and p = 0.36),
respectively. There was no significant change in self-described compe-
4.06 (0.64)
4.39 (0.50)
tence regarding technique or image interpretation in the faculty group
(p = 0.36 and p = 0.13, respectively).
3.94 (0.73) Participants assigned a score of 4.39 (SD 0.50) for the format's
capability to address areas needing improvement in their training, 4.06
(SD 0.64) for its ease of navigation, 4.39 (SD 0.50) for preparing them
to interpret similar studies in future practice, and 3.94 (SD 0.73) for
inducing a change in how they report these studies in the future. See

15
H.J. Otero et al.
Table 2.
4. Discussion
The aim of the study was to establish the effect on perceived com-
petence in interpreting MRI for suspected appendicitis using a simula-
tion approach. We found a significant increase in self-reported com-
petence in the group as a whole with competence level above the
threshold from advanced beginner to competent. The significance of the
increase holds true when further subdivided into residents and fellows.
However, the increase was not significant for the faculty group, which
is likely due to the low number of faculty participants and a higher
reported competence level to start with.
While our results only show increased self-perceived diagnostic
confidence, previous literature, previous reports demonstrate that
training with direct feedback resulted in improved diagnostic perfor-
mance of inexperienced readers in the interpretation of MRI for sus-
pected appendicitis [11]. Similar findings were reported for radiology
residents interpreting mammograms with known outcomes as part of
their training [12]. Improved diagnostic performance and increased
diagnostic confidence are ultimately related to experience, which leads
to expertise. We believe the presented format is an accelerated form of
experience by grouping and interpreting multiple similar studies and
guaranteeing feedback to close the learning cycle. Expert radiologists
are highly adept at the visual management and synthesis of disease
characteristics, which are achieved by translating radiologic and ana-
tomic patterns into diagnostic and management decisions [13]. Im-
proved reading skills were directly related to volume in another study
considering mammograms [14]. When adopting a new imaging appli-
cation or technology, the interpretation of several similar studies has
the potential to help developing radiologic patterns and expertise, as
described by Wood [13].
By creating a simulation setting from readily available existing cases
with known outcomes and existing software, radiologists would be able
to increase exposure to such cases and practice their interpretative skills
on a more realistic environment than traditional interesting cases and
“case of the day” conferences. This experience is also felt to be valuable
to a learner, because uncommon cases are available for sharing among
the group. Residents and fellows, which were provided with immediate
feedback from attending radiologists, generally agreed that the simu-
lation-based format is easy to navigate and prepares them for future
practice. This study opens the possibility of developing additional
dedicated modules for a full curriculum of pediatric MRI applications
that may improve diagnostic confidence in addition to their daily work,
didactic lectures and academic conferences. These cases also comple-
ment our workflow and help us overcome a sense of low real time ex-
posure to advance imaging cases among our trainees and faculty
members. An additional advantage to the presented approach is the
creation of a new educational tool with existing departmental re-
sources. At a time of limited financial support for educational projects,
this project had the advantage of using existing software and hardware,
hence avoiding capital cost expenses. Other than the time invested by
the faculty, the project did not interfere with other ongoing projects and
required no additional funding.
This study has several limitations; first, it was meant as a proof-of-
concept, measuring subjective improvement in diagnostic confidence
and not changes in actual accuracy. A larger set of cases that allows to
see improvement in accuracy can be done in the future. Second, the
number of participants was low, even more so for the faculty and fel-
lows subgroups, limiting the possibility of more robust statistical ana-
lysis and inter-group comparisons. However, as a stand-alone pediatric
16
Clinical Imaging 48 (2018) 12–16
hospital, we believe these numbers are somewhat representative of
larger groups in similar settings. Third, another potential limitation is
related to the selection of non-appendicitis cases, which do not include
pathologic confirmation. Moreover, we did not record the identification
of the normal appendix by each participant in those cases. The identi-
fication of the normal appendix, albeit not crucial in the presence of an
alternative diagnosis such as right lower lobe pneumonia or pancrea-
titis, is important to more confidently exclude appendicitis and even
more so if negative cases are added to the simulation to more closely
resemble daily practice. Last, the pre-simulation video lecture, which
was requested by trainees before attempting to interpret studies with an
unfamiliar technology, can act as a confounding factor on confidence as
participants might feel better prepared by the lecture itself. Future re-
search will look at expanding the cases and participants for this exercise
and at the implementation of a similar environment with other clinical
MR applications. This approach might be particularly useful as new
techniques are being introduced to a radiology department, especially
those requiring standardized interpretation by different readers.
5. Conclusion
Simulation-based training for interpretation of MRI in suspected
appendicitis results in increased self-describe confidence and does not
require additional capital/equipment expenses. Moreover, a similar
format can be used to facilitate adoption of new technologies and ap-
plications by increasing self-perceived competence of interpreting
radiologists. Reinterpreting cases might also allow departments with
lower volume of cases or uneven case load distribution to disseminate
knowledge and improve training within the department.
References
[1] Scheinfeld MH, Moon J-Y, Fagan MJ, Davoudzadeh R, Wang D, Taragin BH. MRI
usage in a pediatric emergency department: an analysis of usage and usage trends
over 5 years. Pediatr Radiol 2017;47(3):327–32.
[2] O'Connor M, Ryan J, Foley S. A review of cross-sectional imaging, ultrasound and
nuclear medicine utilization patterns in paediatric patients in Ireland, 2003-12. Br J
Radiol 2015;88(1048):20140767.
[3] Smith MP, Katz DS, Lalani T, Carucci LR, Cash BD, Kim DH, et al. ACR
Appropriateness Criteria® Right lower quadrant pain—suspected appendicitis.
Ultrasound Q 2015;31(2):85–91.
[4] Aspelund G, Fingeret A, Gross E, et al. Ultrasonography/MRI versus CT for diag-
nosing appendicitis. Pediatrics 2014;133(4):586–93.
[5] Moore MM, Kulaylat AN, Hollenbeak CS, Engbrecht BW, Dillman JR, Methratta ST.
Magnetic resonance imaging in pediatric appendicitis: a systematic review. Pediatr
Radiol 2016;46(6):928–39.
[6] Epifanio M, Antonio de Medeiros Lima M, Corrêa P, Baldisserotto M. An imaging
diagnostic protocol in children with clinically suspected acute appendicitis. Am
Surg 2016;82(5):390–6.
[7] Koning JL, Naheedy JH, Kruk PG. Does abdominal pain duration affect the accuracy
of first-line MRI for pediatric appendicitis? Abdom Imaging 2015;40(2):352–9.
[8] Orth RC, Guillerman RP, Zhang W, Masand P, Bisset GS. Prospective comparison of
MR imaging and US for the diagnosis of pediatric appendicitis. Radiology 2014
Jul;272(1):233–40.
[9] Thieme ME, Leeuwenburgh MMN, Valdehueza ZD, Bouman DE, de Bruin IGJM,
Schreurs WH, et al. Diagnostic accuracy and patient acceptance of MRI in children
with suspected appendicitis. Eur Radiol 2014;24(3):630–7.
[10] Gardner L. From novice to expert: Benner's legacy for nurse education. Nurse Educ
Today 2012;32(4):339–40.
[11] Leeuwenburgh MMN, Wiarda BM, Bipat S, Nio CY, Bollen TL, Kardux JJ, et al.
Acute appendicitis on abdominal MR images: training readers to improve diagnostic
accuracy. Radiology 2012;264(2):455–63.
[12] Poot JD, Chetlen AL. A simulation screening mammography module created for
instruction and assessment: radiology residents vs national benchmarks. Acad
Radiol 2016;23(11):1454–62.
[13] Wood BP. Visual expertise. Radiology 1999;211(1):1–3.
[14] Nodine CF, Kundel HL, Mello-Thoms C, Weinstein SP, Orel SG, Sullivan DC, et al.
How experience and training influence mammography expertise. Acad Radiol
1999;6(10):575–85.
Reproduced with permission of copyright owner. Further reproduction
prohibited without permission.