Vous êtes sur la page 1sur 11


AJMXXX10.1177/1062860615597058American Journal of Medical QualityGupte et al

American Journal of Medical Quality

Together We Learn: Analyzing the

© The Author(s) 2015
Reprints and permissions:
Interprofessional Internal Medicine sagepub.com/journalsPermissions.nav
DOI: 10.1177/1062860615597058

Residents’ and Master of Public Health ajmq.sagepub.com

Students’ Quality Improvement Education


Gouri Gupte, MHA, PhD1, Craig Noronha, MD1, Michal Horný, MSc1,
Karin Sloan, MD1, and Winnie Suen, MD, MSc2

Although the value of interprofessional collaborative education has been promoted, it is unclear how teams of clinical
and nonclinical learners perceive this experience. The authors studied an interprofessional quality improvement (QI)
curriculum implemented in 2013 integrating internal medicine residents (n = 90) and Master of Public Health (MPH)
students (n = 33) at an urban safety net academic medical center. Pre and post curriculum surveys assessed attitudes
toward QI and interprofessional education and team performance. Resident attitudes toward learning and engaging
in QI work improved at the end of the curriculum. Overall, MPH students demonstrated significantly more positive
attitudes about interprofessional learning and work than residents. They also agreed more strongly than residents that
patients would benefit if residents and public health students worked together. As health care organizations evolve to
become more integrated, it is crucial that interprofessional educational opportunities be developed and evaluated to
help encourage a culture of collaboration among health care providers.

quality improvement, interprofessional, residents, evaluation

Effective collaborations among clinicians, support staff, solve patient safety problems.3-5 Although the value of
administrators, and patients are integral to making interprofessional collaborative education has been pro-
health care patient-centered, safe, timely, equitable, moted, the approach has rarely been implemented in resi-
effective, and efficient.1 Creating opportunities for dency curricula, especially focusing on nonclinical health
trainees to learn and work together may foster an under- professionals as colearners.6-10 It is unclear how teams of
standing of the importance of an interdisciplinary clinical and nonclinical learners would perceive their
approach and produce better working relationships.2 In experiences with interprofessional education and collab-
2011, the Interprofessional Education Collaborative oration as each type of learner may have different
expert panel (IPEC) identified the following core com- approaches to and perspectives on problem solving and
petencies for interprofessional collaborative practice: learning.10 Existing limited research has shown that some
values/ethics for interprofessional practice, roles/ health professionals do have positive attitudes toward
responsibilities, interprofessional communication, and interprofessional learning.6
teams and teamwork.3 The report encouraged integrating An interprofessional quality improvement (QI) curric-
interprofessionalism practice into curricula and assessing ulum was implemented in the 2012-2013 academic year at
achievement of these competencies.
The IPEC report focuses on systems thinking and col- 1
Boston University, Boston, MA
laborative approaches that align with the Accreditation 2
Virginia Commonwealth University, Washington, DC
Council for Graduate Medical Education competencies
Corresponding Author:
of practice-based learning and improvement and systems- Gouri Gupte, MHA, PhD, Boston University School of Public Health,
based practice, and the new Next Accreditation System Room 264, 715 Albany Street, Boston, MA 02118.
requirement of working in multidisciplinary teams to Email: gourig@bu.edu

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

2 American Journal of Medical Quality

Table 1.  List of Final Quality Improvement (QI) Projects Completed by Internal Medicine Residents and Masters in Public
Health Students During the QI Curriculum.

Intensive care unit (ICU)

•• Improving ICU family meetings
•• Implementation of a spontaneous breathing trial protocol in the ICU
•• Improving time to antibiotic delivery in sepsis
•• Improving inpatient hypoglycemia prevention
•• Improving antiemetics for prevention of chemotherapy-induced nausea and vomiting in inpatients receiving chemotherapy
•• Improving cellulitis management
•• Improving blood transfusion management
•• Improving the early patient discharge process
•• Improving laboratory follow-up by providers
Primary care/outpatient
•• Improving the resident continuity clinic experience in the primary care clinic
•• Improving diabetes management point of care laboratory testing in a primary care clinic
•• Improving diabetes and cholesterol management in a primary care clinic
•• Improving primary care clinic postdischarge follow-up
•• Improving provider access to behavioral health services for patients
•• Improving the smoking cessation program
•• Increasing hepatitis C testing in the suboxone program

Boston Medical Center (BMC), an urban safety net aca- and 33 MPH students enrolled in the Lean Management
demic medical center in Boston, Massachusetts. The cur- and Operations Management in Healthcare courses at
riculum involved internal medicine residents working on BUSPH participated in the QI curriculum. Four faculty
projects with Master of Public Health (MPH) students, members—a BUSM clinician experienced with QI and
who functioned as QI consultants. The curriculum was medical education, a BUSPH faculty member with exper-
based on the pilot implemented during the 2011-2012 aca- tise in QI methodologies and an instructor for the Lean
demic year, which successfully improved resident QI and Operations Management Courses, the director of
knowledge and promoted positive attitudes toward QI.7 clinical quality at BUSM, and an associate program
Based on feedback received from students and residents, director in the internal medicine residency program—
major changes such as improving the time allocation for collaborated to direct the curriculum. In addition, 16
projects, providing sponsorship for the projects, and eval- BUSM faculty members and staff participated as spon-
uating the interprofessional experience were made to the sors for the chosen QI projects (Table 1).
2012-2013 curriculum. This article describes the updated
curriculum structure and results of the interprofessional Planning
experience of the 2012-2013 academic year curriculum.
Curriculum planning started 6 months prior to the first
session. The course directors applied the driver diagram
Methods tool, used primarily to guide change in QI implementa-
tion work, to map out curricular improvements (see
Setting online Appendix A, available at http://ajmq.sagepub.
Members of the Boston University School of Medicine’s com/supplemental).11 The pre-curriculum planning steps
(BUSM) Department of Medicine and Boston University involved identifying clinical sponsors to champion proj-
School of Public Health’s (BUSPH) Department of Health ects in clinical areas and to link QI teams with relevant
Policy and Management collaborated to conduct the sec- stakeholders, as well as soliciting interest and commit-
ond iteration of the interprofessional QI curriculum. ment from MPH students to participate in the residents’
QI curriculum. Students who elected to participate used
the QI project as their mandatory course assignment.
Participants They then completed BMC’s human resources require-
Ninety BMC second postgraduate year (PGY-2) and third ments. One month prior to the start of the curriculum,
postgraduate year (PGY-3) internal medicine residents course faculty provided the residents and MPH students

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

Gupte et al 3

Table 2.  Content Delivered During the Didactic Sessions Held for the Quality Improvement (QI) Curriculum Implemented at
Boston Medical Center for Residents.

Learning Goals Session Activities Suggested Homework

Session 1
  Introduction to QI •• Explain the hospital QI •• Introductions and survey •• Engage clinical staff on
curriculum process approach of plan- project
do-study-act (PDSA)
  Gap Analysis/ •• Develop a process map •• Brief overview of the QI •• Develop current process
Identifying Wastes in process approach map, identification of wastes,
the System obtain available data
  •• Develop a fishbone/5 whys •• Describe curriculum, project,  
diagram and deliverables
  •• Identify wastes in the system •• Process mapping, fishbone  
diagram, identification of
  •• Identify team roles •• Team communication/work  
plan discussion
Session 2
  Data Analysis •• Use data analysis tools in QI •• Paper puppet making activity •• Develop root cause analysis/
work (eg, Pareto, control to demonstrate use of data fishbone diagram
charts, histograms) analysis tools
  •• Team meeting •• Start PDSA
Session 3
  Challenges in QI work •• Identify reasons why QI •• Discussion of challenges that •• Complete PDSA
projects could fail arise in QI
  •• Successful project showcase •• Complete Presentation
  •• Team meeting  
Session 4
 Presentations •• Coherently and effectively •• 10-minute presentation,  
present project to audience 3-minute question session for
each group in front of peers,
staff, and judges
  •• Curriculum evaluation  

with a list of 28 projects to select from according to their invited to attend all sessions, although many were unable
personal or career interests. Sixteen teams, each consist- to attend because of scheduling conflicts.
ing of 1 to 2 MPH students and 4 to 6 residents, were then Concurrently, the MPH students received in-depth
formed based on individual preferences (Table 1). training on various tools that are important in operations
management, project planning, as well as QI tools and
methods during their week-long Lean Management
Curriculum Structure and Content course prior to the start of the resident-MPH QI curricu-
The 16-week QI curriculum (January-May 2013) was lum and their semester-long Operations Management
incorporated into the 3-week inpatient 1-week ambula- course. Similar to the pilot curriculum, MPH students
tory (3+1) residency schedule model, in which QI session then served as QI consultants and were required to attend
time was provided for a half day during the ambulatory all 4 QI sessions with the residents.7
week. The QI curriculum (Table 2) ran for 4 months with
each group meeting for 90 minutes once per month.
Data Collection and Analysis
Given this 3 + 1 residency schedule, residents were
divided equally into 4 groups and further subdivided into A 38-question pre-curriculum survey adapted from 3
project teams.7 Half of the session time provided just-in- instruments—Quality Improvement Knowledge
time learning on QI topics, while the remainder was used Application Tool (QIKAT), the Tess et al14 QI assessment,
for project work. The teams created their own meeting and the Readiness for Interprofessional Learning Scale
and project work schedules outside of these sessions. The (RIPLS)—was administered in Session 1 to evaluate QI
groups also were provided with another hour for project content knowledge and attitudes (see online Appendix B,
work during the ambulatory weeks. Sponsors were available at http://ajmq.sagepub.com/supplemental).12-15

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

4 American Journal of Medical Quality

The survey included 5 demographic questions and 33 questionnaire showed an excellent internal consistency
five-point Likert scale questions (1 = strongly disagree; 5 with a Cronbach α of .974.
= strongly agree). The post survey provided in Session 4
included 3 demographic questions, the 33 questions from
the QIKAT and RIPLS, and 18 six-point Likert scale ques-
tions (1 = none of the time; 6 = all of the time) from the A total of 123 individuals (90 residents and 33 MPH stu-
Team Performance Scale.16 The RIPLS and Team dents) participated in the 2012-2013 academic year cur-
Performance Scale were administered to both MPH stu- riculum. In all, 67.5% of the participants (34 PGY-2
dents and residents, while the condensed version of the residents, 28 PGY-3 residents, and 21 MPH students)
QIKAT survey was administered only to the residents. completed the pre-curriculum survey, and 66.7% of the
Demographic information was examined using descrip- participants (34 PGY-2 residents, 27 PGY-3 residents,
tive statistics and differences between the subpopulations and 21 MPH students) completed the post-curriculum
were assessed by a χ2 test. Prior to the response analysis, survey. There were approximately equal numbers of
the internal consistency and reliability of the 3 question- males and females among residents (30 females, 32
naires was verified using the Cronbach α analysis. The males), but substantially more females than males among
attitudes toward interprofessional education questionnaire MPH students (17 females, 2 males, 2 did not disclose
contained several questions with opposite scoring and their sex). More residents (61%, n = 38) than MPH stu-
meaning from the remaining items; therefore, the dents (29%, n = 6) had prior exposure to QI (eg, personal
responses to these questions were reverse coded for the improvement project, workplace initiative, other experi-
purposes of the Cronbach α analysis. Differences in ence that used formal improvement principles and meth-
responses to individual questions were assessed by the ods) (P = .04). However, a similar share of residents
Wilcoxon rank sum test. Because of a very low proportion (29%, n = 18) and MPH students (29%, n = 6) had prior
of males among MPH students, a robustness check of the formal training in QI techniques (P = .73).
results was performed by rerunning all analyses for
females only. All analyses were generated using SAS soft-
ware, version 9.3 (SAS Institute Inc., Cary, North Assessment of Attitudes Toward
Carolina). Figures were generated using the statistical Interprofessional Education
software R, version 3.2.0. Finally, free-text answers found The responses of MPH students and residents both before
in the comments section of the survey were analyzed and and after the curriculum are summarized in Figure 1.
common themes were derived. The BMC Institutional MPH students demonstrated significantly more positive
Review Board deemed this project exempt from review. attitudes toward interprofessional learning and work than
residents for all questions before the curriculum and in
Results almost all questions after the curriculum. MPH students
agreed more strongly than residents both before and after
Survey the curriculum that patients would benefit if medical resi-
In the pre-curriculum survey, the attitudes toward QI edu- dents and public health students worked together, that
cation and interprofessional education questionnaires shared learning would increase understanding of clinical
showed good internal consistency (Cronbach α of .825 problems, that communication skills should be learned
and .894, respectively). The Team Performance Survey with other medical residents and public health students,

Pre-Curriculum Post-Curriculum

EssPre How essential do you consider quality improvement in your future professional work EssPost

MPH Students 0% 0% 100% 5% 0% 95%

p < .0001 p = .36

Residents 3% 10% 87% 0% 98%

Response Very non-essential Non-essential Neither essential nor non-essential Essential Very essential

ConfiPre ConfiPost
How confident are you that you can make a change to improve health care in a local setting

MPH Students 0% 32% 68% 0% 40% 60%

p = .0002 p = .75
Residents 24% 48% 27% 17% 29% 54%

Response Not confident at all Non confident Reasonably confident Confident Very confident

Figure 1.  Survey results comparing Master of Public Health (MPH) students’ and residents’ attitudes toward interprofessional education.

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

Gupte et al 5

Pre-Curriculum Post-Curriculum
ProfPrewith other professionals will make me a more effective member of a medical and public health team

MPH Students 0% 0% 100% 0% 0% 100%

p < .0001 p < .0001

Residents 0% 11% 89% 0% 2% 98%

Patients would ultimately benefit if medical residents and public health students worked together
BenefPre BenefPost

MPH Students 0% 0% 100% 0% 0% 100%

p < .0001 p = .0001

Residents 2% 15% 84% 5% 9% 86%

Shared learning SharedPost
with other medical residents and public health students will increase my ability to understand clinical problems

MPH Students 0% 5% 95% 0% 0% 100%

p = .0001 p = .0001
Residents 5% 11% 84% 2% 20% 78%

CommPre CommPost
Communications skills should be learned with other medical residents and public health students

MPH Students 0% 0% 100% 0% 0% 100%

p < .0001 p < .0001

Residents 2% 21% 77% 7% 88%

100 50 0
TeamPre Team-working50
skills are vital for all100 100and public health students
medical residents 50 to learnTeamPost 50 100
Percentage Percentage
MPH Students 0% 0% 100% 0% 95%

p < .0001 p < .0001

Residents Response
0% Strongly disagree Disagree
6% Neutral Agree Strongly agree
94% Response
0% Strongly disagree Disagree Agree Strongly agree

LimitPre Shared learning will help me to understand my own professional limitations LimitPost

MPH Students 5% 0% 95% 0% 0% 100%

p < .0001 p < .0001

Residents 0% 16% 84% 0% 11% 89%

Learning between medical residents FinishPost
and public health students before finishing residency/school would improve working relationships after finishing residency/school

MPH Students 0% 10% 90% 0% 10% 90%

p < .0001 p < .0001

Residents 2% 31% 68% 0% 23% 77%

Shared learning will help me think positively about other medical residents and public health professionals
PositPre PositPost

MPH Students 0% 10% 90% 10% 0% 90%

p < .0001 p < .0001

Residents 3% 16% 80% 0% 20% 80%

SmallPre SmallPost
For small-group learning to work, students/medical residents need to respect and trust each other

MPH Students 0% 0% 100% 10% 90%

p < .0001 p = .01

Residents 2% 2% 97% 5% 95%

WastePre WastePost
I don't want to waste time learning with other medical residents and public health students / professionals

MPH Students 95% 0% 5% 100% 0% 0%

p < .0001 p < .0001

Residents 73% 13% 15% 71% 25% 4%

100 50 0
NecesPre 50
It is not necessary 100
for medical residents 100health students to50
and public 0
learn together NecesPost 50 100
Percentage Percentage
MPH Students 90% 10% 0% 95% 5% 0%

p < .0001 p < .0001

Residents Response
76% Strongly disagree Disagree
13% Neutral Agree Strongly agree
11% Response Strongly disagree Disagree
29% Neutral Agree Strongly agree

Clinical problemClinicPre ClinicPost

solving can only be learned effectively with medical residents/ students / professionals from my own school / organization

MPH Students 95% 5% 0% 14% 10%

p < .0001 p = .16

Residents 66% 21% 13% 11% 7%

Shared learning with Comm1Post
other medical residents and public health students will help me to communicate better with patients and other professionals

MPH Students 0% 0% 100% 5% 0% 95%

p < .0001 p < .0001

Residents 6% 13% 81% 5% 20% 75%

I would welcome the opportunity to work on small group projects with other medical residents and public health WelcoPost
students / professionals

MPH Students 0% 0% 100% 5% 0% 95%

p < .0001 p < .0001

Residents 3% 15% 82% 2% 41% 57%

I would welcome the opportunity GenerPost
to share some generic lectures, tutorials or workshops with other medical residents and public health students / professionals

MPH Students 0% 5% 95% 0% 5% 95%

p < .0001 p < .0001

Residents 3% 16% 81% 7% 38% 55%

ClariPreShared learning and practice will help me clarify the nature of patients' or clients' problemsClariPost

MPH Students 0% 5% 95% 0% 10% 90%

p < .0001 p < .0001

Residents 2% 23% 76% 7% 27% 66%

100 50 BetterPre
0 Shared learning
50 before and after100
residency will help
100me become a better
50 team worker BetterPost
0 50 100
Percentage Percentage
MPH Students 0% 0% 0% 0% 0% 0%

Residents Response
2% Strongly disagree Disagree
16% Neutral Agree Strongly agree
82% Response
0% Strongly disagree Disagree
18% Neutral Agree Strongly agree

UnsurePre I am not sure what my professional role will be / is UnsurePost

MPH Students 0% 0% 0% 0% 0% 0%

Residents 44% 34% 23% 77% 16% 7%

I have to acquire much more knowledge and skill than other students / professionals in my own faculty
AcquirePre / organization

MPH Students 0% 0% 0% 0% 0% 0%

Residents 21% 52% 26% 53% 35% 13%

100 50 0 50 100 100 50 0 50 100

Percentage Percentage

Response Strongly disagree Disagree Response

Neutral Strongly
Agree disagree Disagree Response
Strongly agree Neutral Agree disagree
Strongly Strongly agree
Disagree Neutral Agree Strongly agree

Figure 1. (continued)
Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
6 American Journal of Medical Quality

and that shared learning would help them better under- Reflections by Residents and MPH Students
stand their own professional limitations (P < .01 for all).
The results were robust to gender differences. The next An open-ended question in the post survey invited resi-
assessment focused on how attitudes toward interprofes- dents and MPH students to comment about the curriculum
sional education changed after the curriculum for the experience. Residents requested more time for the projects
samples of MPH students and residents separately. The (n = 20), to start the curriculum earlier in the semester (n =
findings were that attitudes improved significantly among 4), and reported individual team member issues (n = 3).
residents while there were essentially no significant Residents also commented on the difficulty of prioritizing
changes in the attitudes of the MPH students. and balancing clinical work while working as a team
because of busy schedules (n = 2). MPH students made
comments about difficulties communicating and contact-
Assessment of Residents’ Attitudes Toward QI ing team members, as residents were not responsive to
in General and QI in the Residency Program e-mails when clinically busy with challenging work sched-
The pre- and post-curriculum QIKAT questionnaires ules and clinical priorities (n = 11). One MPH student com-
showed that the QI curriculum and projects did signifi- mented that the experience was “worthwhile, with initial
cantly improve residents’ attitudes toward QI work and ‘growing pains’ while working with the residents that later
the residency program’s focus on QI (Figure 2)—although normalized.” MPH students reported concerns of not being
the results of improvement in attitudes assessment were respected during their teamwork experience (n = 13). One
more pronounced among PGY2 residents. After the cur- resident made a comment that was representative of others
riculum, residents felt more strongly that QI is an impor- regarding working with MPH students:
tant educational topic (P < .01) and that they play a role
QI is very important to me and I truly believe that it enhances
in designing QI changes in the hospital and/or clinic (P <
medical education/clinical training. . . . Additionally, while
.01). The results were robust to gender differences. gaining perspective from MPH students is valuable, I feel it
On further analyzing differences in responses by sex didn’t help much in this situation because both sides (MPH
prior to the curriculum, it was found that female residents & MD) were on two different wavelengths with two different
felt more strongly than male residents (P < .01) that their goals of implementation, which really didn’t come to a
ideas to improve patient care were sought and used con- common ground.
structively by hospital and/or clinic leaders, but this
difference was no longer observed after the curriculum
(P = .18). Understanding one’s own role within the mul- Discussion
tidisciplinary team generally improved throughout the QI Accrediting bodies have encouraged interprofessional
curriculum (P < .01) for both males and females. teamwork and collaboration in medicine to improve
However, male residents demonstrated stronger attitudes care delivery processes and outcomes; to reduce staff
than female residents consistently before (P = .029) and turnover, absenteeism, and burnout; to increase staff
after (P = .031) the QI curriculum. motivation; and to reduce staff conflict.5,17-20 A QI cur-
Overall, there were no significant differences in atti- riculum created at BMC with residents and MPH stu-
tudes toward QI education between PGY-2 and PGY-3 dents promoted an interprofessional learning and
residents before the curriculum; however, PGY-2 residents working environment while teaching QI tools and sys-
had more positive feelings about how the residency was tems thinking. Surprisingly, the findings show glaring
preparing them to deliver high-quality care (medianPGY-2 = differences between MPH students and residents with
strongly agree/agree; medianPGY-3 = agree; P = .046) and regard to interprofessional engagement and collabora-
to think analytically in clinical situations (medianPGY-2 = tion. MPH students had more favorable attitudes than
strongly agree; medianPGY-3 = agree; P = .001) than PGY-3 residents toward working in teams, understanding clini-
residents after the QI curriculum. cal problems, and improving patient care. Additionally,
MPH students were more interested in improving their
communication and understanding of their professional
Assessment of Team Performance limitations through these shared learning activities than
MPH students evaluated team performance post curricu- residents. Gender-based differences were observed, with
lum slightly better than residents (Figure 3); however, women participants placing significantly more impor-
these differences were no longer significant when the 2 tance on QI knowledge and interprofessional work than
data samples were restricted to females only. In general, male colleagues from the residency program.
both MPH students and residents rated team performance Many of the previously published interprofessional QI
relatively high, with median responses ≥5 on a 6-point curricula included only clinical professionals working in
scale. teams.14,21,22 A survey focusing on attitudes of faculty

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

Gupte et al 7

I feel that quality improvement is an important educational topic

Pre-Curriculum 2% 12% 87%

p = .001
Post-Curriculum 0% 0% 100%

I play a role in designing quality improvement changes in the hospital and/or clinic

Pre-Curriculum 30% 33% 37%

p = .0003
Post-Curriculum 8% 21% 71%

I play a role in implementing quality improvement changes in the hospital and/or clinic

Pre-Curriculum 20% 17% 63%

p = .13
Post-Curriculum 8% 13% 79%

I want to learn more about how to improve quality in a hospital/clinic

Pre-Curriculum 3% 15% 82%

p = .23
Post-Curriculum 0% 8% 92%

My ideas to improve patient care are sought and used constructively by hospital and/or clinic leaders

Pre-Curriculum 25% 42% 32%

p = .17
Post-Curriculum 17% 36% 47%

I feel the education I receive about patient safety and health care quality is relevant to the clinical care I deliver

Pre-Curriculum 2% 13% 85%

p = .029
Post-Curriculum 0% 2% 98%

100 50 0 to practice medicine safely

I feel residency is preparing me 50 100
Pre-Curriculum 2% 3% 95%

p = .004
Post-Curriculum 0%
Response Strongly disagree Disagree
Neutral Agree Strongly agree 100%

I feel residency encourages me to analyze my own practice and make changes based on what I learn

Pre-Curriculum 0% 18% 82%

p = .007
Post-Curriculum 0% 6% 94%

I feel I understand my role within the multidisciplinary team (physicians, nurses, social workers, case managers, PT, and OT) caring for patients on the medical floors

Pre-Curriculum 0% 12% 88%

p = .007
Post-Curriculum 0% 0% 100%

I feel residency is preparing me to deliver high-quality care to my patients

Pre-Curriculum 2% 7% 92%

p = .007
Post-Curriculum 0% 0% 100%

I feel residency is preparing me to think analytically in clinical situations

Pre-Curriculum 0% 2% 98%

p = .046
Post-Curriculum 0% 0% 100%

I feel our residency encourages me to learn with other professionals and my peers

Pre-Curriculum 2% 5% 93%

p = .35
Post-Curriculum 0% 2% 98%

100 50 0 errors and waste in healthcare

I feel comfortable understanding 50 100
Pre-Curriculum 8% 30% 62%

p < .0001
Post-Curriculum 2%
Response Strongly disagree Disagree
Neutral Agree Strongly agree 94%

I am satisfied overall with the quality of care I deliver

Pre-Curriculum 3% 13% 83%

p = .13
Post-Curriculum 2% 4% 94%

100 50 0 50 100

Response Strongly disagree Disagree Neutral Agree Strongly agree

Figure 2.  Survey comparing residents’ attitudes toward quality improvement (QI) in general and QI in the residency program
before and after the quality improvement curriculum.

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

8 American Journal of Medical Quality

All team members made an effort to participate in discussions

MPH Students 21% 79%

p = .46
Residents 17% 83%

When team members had different opinions, each member explained his or her point of view

MPH Students 11% 89%

p = .29
Residents 2% 98%

Team members encouraged one another to express their opinions and thoughts

MPH Students 11% 89%

p = .24
Residents 5% 95%

Team members shared and received criticism without making it person

MPH Students 0% 100%

p = .20
Residents 0% 100%

Different points of view were respected by team members

MPH Students 0% 100%

p = .053
Residents 0% 100%

Often members helped a fellow team member to be understood by paraphrasing what he or she was saying

MPH Students 11% 89%

p = .03
Residents 7% 93%

100 50 for problem solving (such as brainstorming)

My team used several techniques 0 with each team member50
presenting his or her best ideas 100
MPH Students 5% 95%
p = .009
Residents 2% Response None of the time Very rarely Rarely Occasionally Very frequently All of the time 98%

Team members worked to come up with solutions that satisfied all members

MPH Students 0% 100%

p = .01
Residents 0% 100%

All team members consistently paid attention during group discussions

MPH Students 11% 89%

p = .04
Residents 14% 86%

My team actively elicited multiple points of view before deciding on a final answer

MPH Students 5% 95%

p = .23
Residents 0% 100%

Team members listened to each other when someone expressed a concern about individual or team performance

MPH Students 0% 100%

p = .18
Residents 5% 95%

Team members willingly participated in all relevant aspects of the team

MPH Students 11% 89%

p = .05
Residents 14% 86%

100 50 members resolved differences of0 opinion by openly speaking their mind
Team 50 100
MPH Students 0% 100%
p = .10
Residents 10% Response None of the time Very rarely Rarely Occasionally Very frequently All of the time 90%

Team members used feedback about individual or team performance to help the team be more effective

MPH Students 16% 84%

p = .04
Residents 18% 82%

Team members seemed attentive to what other team members were saying when they spoke

MPH Students 5% 95%

p = .11
Residents 7% 93%

My team resolved many conflicts by compromising between team members, with each one giving a little

MPH Students 5% 95%

p = .16
Residents 8% 92%

Members who had different opinions explained their point of view to the team

MPH Students 11% 89%

p = .16
Residents 5% 95%

Team members were recognized when something they said helped the team reach a good decision

MPH Students 0% 100%

p = .12
Residents 2% 98%

100 50 0 50 100

Response None of the time Very rarely Rarely Occasionally Very frequently All of the time

Figure 3.  Survey results comparing Master of Public Health (MPH) students’ and residents’ perceptions of team performance
post curriculum. Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015
Gupte et al 9

members from various health care professions toward Strengths of the Program
interprofessional education had reported that, overall,
medical faculty had less interest in working with indi- One of the strengths of this curriculum is the unique col-
viduals from other health professions who are not usually laboration between medicine and public health providers,
in their immediate clinical team, such as pharmacists or both as learners and as instructors. The curriculum was
social workers.23 The present study at BMC showed simi- made possible because BUSPH was physically next to
lar results for collaborations between clinical and non- the hospital where the internal medicine residents were
clinical professionals. The authors partly subscribe to the training. In addition, the residency program director,
theory that professional cultures may explain the differ- chair of medicine, internal medicine faculty, School of
ences between attitudes toward interprofessional team- Public Health faculty, and members of the hospital qual-
work and education. Traditionally, different paradigms ity department were supportive of the curriculum. This
drive different disciplines. Physician training has focused allowed for collaborations across schools, departments,
more on teaching learners to be team leaders and to work and the hospital.
with other clinical professionals, to be able to think inde- Another strength is in the design of the program. The
pendently about the situation and derive treatment plans QI curriculum was structured in a way to incorporate
that may be applied in a more paternalistic and authoritar- into the 3 + 1 residency scheduling system for all resi-
ian manner.9,24,25 Traditionally, MPH students as well as dents and matched the semester schedule of the MPH
pharmacists and social workers have learned and worked students taking either Operations Management or Lean
on projects that involve more collaborative practices.26-28 Management courses. The program also was designed to
Although current training and competencies have been promote active and reflective learning styles based on
developed to encourage interprofessional teamwork and principles of renowned education frameworks such as
relationship building in medical programs, culture change Bloom’s taxonomy, Knowles adult learning theory, and
and historical hierarchies will take time to evolve and be the Kirkpatrick framework.36-38
ingrained into practice. The evaluation of interprofessional attitudes of the
Another potential explanation for the differences in clinical and nonclinical learners toward the curriculum is
attitudes toward interprofessional work between MPH a strength of the curriculum. There are tools to evaluate
students and residents may be attributable to how distinct QI knowledge and tools to assess interprofessionalism,
professions perceive and interpret issues based on their but it is unique to combine these 2 parts into an evalua-
cognitive map.9,29,30 These cognitive maps are likely cre- tion of a curriculum that encompasses both.
ated because instructors teaching different professions
may create curricula, objectives, and competencies in iso- Challenges for the Program
lation of each other. As a result “. . . quite literally, two
opposing ‘disciplinarians’ can look at the same thing and One of the major challenges for the curriculum was the
not see the same thing . . .”29(p. 35) Therefore, students not incorporation of more project and meeting time for resi-
only learn the skills and values of their profession but also dents who were trying to balance other clinical and
assume the occupational identity commonly referred to as research obligations. The lack of protected time outside
“professionalization.”29,31,32 Identifying the differences in the ambulatory week may have affected both residents’
cognitive maps and professional perceptions and raising and MPH students’ attitudes toward interprofessional
them for discussion during the interprofessional curricu- work. However, the results may also be a true reflection
lum introduction may help learners better understand their of real life where many clinicians have to participate in
team experience, roles, and potential challenges.29 QI work as part of their clinical expectations and mainte-
The observed gender differences wherein females nance of certification requirements. Many physicians
had a more positive attitude toward teamwork and col- refer to it as a “second unpaid job.”
laboration corresponded to already established find- Another challenge the participants faced was defining
ings.23,33 Various studies have suggested that there are clear roles for each other, as well as goals and objectives
important differences in the way the 2 genders interact for the project. Because residents were frequently clini-
and adopt new methodologies. Women are known to be cally busy for several weeks at a time during their inpa-
more sensitive to relationships and work harder to main- tient rotation, the MPH students often had to carry the
tain a team environment.34,35 In addition, women tend to team and hold the project together. Because of a lack of
show less skepticism toward new interventions or meth- effective communication at times, residents and MPH
odologies.34 Consciously assigning and evaluating a students ultimately focused on different parts of the proj-
certain gender mix on the teams in future iterations of ect, leading to disjointed end products that required addi-
the curriculum may be helpful in determining how this tional time to be made cohesive.
characteristic could influence overall attitudes toward The team structure may affect the effectiveness of the
interprofessionalism. teamwork and outcomes of the project. This curriculum

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

10 American Journal of Medical Quality

included both PGY-2 and PGY-3 residents. Based on the recommend that schools actively seek opportunities for
evaluation results, the authors believe that the PGY-3 collaboration in education to increase the number of grad-
residents may have been less invested in the curriculum uates who will collaborate across disciplines.26,39 The sus-
and could have negatively affected the attitudes of the tainability of these kinds of interprofessional curricula can
group. The relatively large size of each group (ie, 4-6 be made possible with institutional support and strategi-
residents and 1-2 MPH students) also may have reduced cally progressive medical education programs that under-
the sense of teamwork and individual responsibility of stand the upcoming needs of health care. QI is a “team
team members. sport”; therefore, these curricula can be used as a good
The perceived and actual role of the sponsors also starting point for current residents. However, the authors
affected the interprofessional team experience. In some encourage that these assessments and strategies for more
cases, the sponsor’s expectations differed from the resi- collaboration start very early in the training programs.
dent-MPH team’s expectations, which again led to a frag-
mented project product. However, the authors have Authors’ Note
observed that some projects have led to ongoing work Prior presentation: Gupte G, Suen W, Sloan K, Noronha C. An
and improved processes in patient care thanks to the addi- interprofessional quality improvement curriculum for internal
tion of faculty sponsors to the curriculum. medicine residents and masters of public health students: the
This study has some limitations. First, although there lean way; Association of University Programs in Health
was a large cohort of residents and MPH students, the gen- Administration Annual Meeting; San Antonio, Texas; 2014.
eralizability of the findings is limited because the partici-
pants represent only a single cohort, specialty, department, Declaration of Conflicting Interests
and institution. Second, despite the strong validity of the The authors declared no potential conflicts of interest with
constructs in the questionnaire, there is room for improve- respect to the research, authorship, and/or publication of this
ment in future versions (eg, controlling for the colearning article.
environment, considering the context of daily work).
The authors received the following financial support for the
Future of This Curriculum research, authorship, and/or publication of this article: Education
As part of future steps, the authors plan to develop a com- Pilot Grant, Department of Medicine, Boston University, July 1,
prehensive 3-year evaluative strategy using psychometric 2012 to June 30, 2013.
assessment tools and 360-degree feedback mechanisms
from main stakeholders of the curriculum inclusive of resi- References
dents, public health students, and participating faculty and
1. Institute of Medicine. Crossing the Quality Chasm: A New
sponsors of the QI projects; to build in more training for
Health System for the 21st Century. Washington, DC:
team formation and consideration of gender differences; to National Academies Press; 2001.
create more partnerships with other professions such as 2. Lerner S, Magrane D, Friedman E. Teaching teamwork in
nursing and engineering; and to involve medical students medical education. Mt Sinai J Med. 2009;76:318-329.
to provide residents with the opportunity to examine QI 3. Interprofessional Education Collaborative Expert Panel. Core
from multiple perspectives. Last, the authors plan to evolve Competencies for Interprofessional Collaborative Practice:
the content to the next level through introduction of the Report of an Expert Panel. Washington, DC: Interprofessional
concepts of human factors and system errors. Education Collaborative Expert Panel; 2011.
4. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME
accreditation system—rationale and benefits. N Engl J
Conclusion Med. 2012;366:1051-1056.
As health care organizations evolve to become more inte- 5. Accreditation Council of Graduate Medical Education.
Next accreditation system. http://www.acgme.org/acg-
grated, it is crucial that interprofessional educational
opportunities be developed and evaluated to help encour- NextAccreditationSystem.aspx. Accessed November 14,
age a culture of collaboration among health care provid- 2014.
ers. The approach needs to be more interdisciplinary, not 6. Giordano C, Umland E, Lyons KJ. Attitudes of faculty
just multidisciplinary. Future studies of interprofessional and students in medicine and the health professions toward
curricula also should investigate the effect of gender mix interprofessional education. J Allied Health. 2012;41(1):
on the teams and consider collaborative learning early in 21-25.
the medical education process to foster improved attitudes 7. Weigel C, Suen W, Gupte G. Using lean methodology to
toward interprofessionalism. National School of Public teach quality improvement to internal medicine residents at
Health and School of Medicine committees both a safety net hospital. Am J Med Qual. 2013;28:392-399.

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015

Gupte et al 11

8. Reid R, Bruce D, Allstaff K, McLernon D. Validating the 23. Curran VR, Sharpe D, Forristall J, Flynn K. Attitudes of
Readiness for Interprofessional Learning Scale (RIPLS) health sciences students towards interprofessional team-
in the postgraduate context: are health care professionals work and education. Learning in Health and Social Care.
ready for IPL? Med Educ. 2006;40:415-422. 2008;7:146-156.
9. Hall P. Interprofessional teamwork: professional cultures 24. Robinson M, Cottrell D. Health professionals in multi-dis-
as barriers. J Interprof Care. 2005;19(suppl 1):188-196. ciplinary and multi-agency teams: changing professional
10. Payler J, Meyer E, Humphris D. Pedagogy for interprofes- practice. J Interprof Care. 2005;19:547-560.
sional education—what do we know and how can we eval- 25. Reese DJ, Sontag MA. Successful interprofessional collabora-
uate it? Learning in Health and Social Care. 2008;7:64-78. tion on the hospice team. Health Soc Work. 2001;26:167-175.
11. Langley GJ, Moen R, Nolan KM, Nolan TW, Norman 26. Maeshiro R, Johnson I, Koo D, et al. Medical education
CL, Provost LP. The Improvement Guide: A Practical for a healthier population: reflections on the Flexner Report
Approach to Enhancing Organizational Performance. San from a public health perspective. Acad Med. 2010;85:
Francisco, CA: Jossey-Bass; 2009. 211-219.
12. Morrison L, Headrick L, Ogrinc G, Foster T. The quality 27. Varda D, Shoup JA, Miller S. A systematic review of col-
improvement knowledge application tool: an instrument laboration and network research in the public affairs litera-
to assess knowledge application in practice-based learning ture: implications for public health practice and research.
and improvement. Paper presented at: Society of General Am J Public Health. 2012;102:564-571.
Internal Medicine meeting; Vancouver, British Columbia, 28. Horsburgh M, Lamdin R, Williamson E. Multiprofessional
Canada; May 7, 2003. learning: the attitudes of medical, nursing and pharmacy
13. Ogrinc G, Headrick LA, Morrison LJ, Foster T. Teaching and students to shared learning. Med Educ. 2001;35:876-883.
assessing resident competence in practice-based learning and 29. Petrie HG. Do you see what I see? The epistemology of inter-
improvement. J Gen Intern Med. 2004;19(5 pt 2):496-500. disciplinary inquiry. J Aesthetic Educ. 1976;10(1):29-43.
14. Tess AV, Yang JJ, Smith CC, Fawcett CM, Bates CK, 30. Itano JK, Williams J, Deaton MD, Oishi N. Impact of a
Reynolds EE. Combining clinical microsystems and an student interdisciplinary oncology team project. J Cancer
experiential quality improvement curriculum to improve Educ. 1991;6:219-226.
residency education in internal medicine. Acad Med. 31. Hall P, Weaver L. Interdisciplinary education and teamwork:
2009;84:326-334. a long and winding road. Med Educ. 2001;35:867-875.
15. Parsell G, Bligh J. The development of a questionnaire to 32. Loseke DR, Cahill SE. Actors in search of a character:
assess the readiness of health care students for interprofes- student social workers’ quest for professional identity.
sional learning (RIPLS). Med Educ. 1999;33:95-100. Symbolic Interaction. 1986;9:245-258.
16. Thompson BM, Levine RE, Kennedy F, et al. Evaluating 33. Hertweck M, Hawkins S, Bednarek M, Goreczny A,

the quality of learning-team processes in medical educa- Schreiber J, Sterrett S. Attitudes toward interprofes-
tion: development and validation of a new measure. Acad sional education: comparing physician assistant and other
Med. 2009;84(10 suppl):S124-S127. health care professions students. J Physician Assist Educ.
17. Zwarenstein M, Goldman J, Reeves S. Interprofessional 2012;23(2):8-15.
collaboration: effects of practice-based interventions on 34. Wilhelmsson M, Ponzer S, Dahlgren LO, Timpka T,
professional practice and healthcare outcomes. Cochrane Faresjö T. Are female students in general and nurs-
Database Syst Rev. 2009;(3):CD000072. ing students more ready for teamwork and interprofes-
18. Coleman MT, Roberts K, Wulff D, van Zyl R, Newton K. sional collaboration in healthcare? BMC Med Educ.
Interprofessional ambulatory primary care practice-based 2011;11(1):15.
educational program. J Interprof Care. 2008;22(1):69-84. 35. Fox MF. Women, science, and academia: graduate educa-
19. McPherson K, Headrick L, Moss F. Working and learning tion and careers. Gender Society. 2001;15:654-666.
together: good quality care depends on it, but how can we 36. Bloom BS, Krathwohl DR. Taxonomy of Educational

achieve it? Qual Health Care. 2001;10(suppl 2):ii46-ii53. Objectives: The Classification of Educational Goals.
20. Varkey P, Reller MK, Smith A, Ponto J, Osborn M. An Handbook I: Cognitive Domain. New York, NY: David
experiential interdisciplinary quality improvement educa- McKay; 1956.
tion initiative. Am J Med Qual. 2006;21:317-322. 37. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training

21. Varkey P, Karlapudi SP. Lessons learned from a 5-year Programs: The Four Levels. San Francisco, CA: Berrett-
experience with a 4-week experiential quality improve- Koehler; 2006.
ment curriculum in a preventive medicine fellowship. J 38. Knowles MS. The Modern Practice of Adult Education. Vol
Gen Intern Med. 2009;1:93-99. 41. New York, NY: New York Association Press; 1970.
22. Wong BM, Etchells EE, Kuper A, Levinson W, Shojania 39. Gebbie K, Rosenstock L, Hernandez LM. Who Will Keep
KG. Teaching quality improvement and patient safety the Public Healthy? Educating Public Health Professionals
to trainees: a systematic review. Acad Med. 2010;85: for the 21st Century. Washington, DC: National Academies
1425-1439. Press; 2003.

Downloaded from ajm.sagepub.com at Universitas Gadjah Mada on November 13, 2015