Otolaryngol Clin N Am

40 (2007) 1215–1225

Competency-Based Resident Education

Bradley F. Marple, MD
Department of Otolaryngology/Head and Neck Surgery, University of Texas,
Southwestern Medical Center at Dallas, 5323 Harry Hines Boulevard, Dallas,
TX 75390-9035, USA

Medical education in the United States has evolved over the course of the
last century from that of an apprentice-based system to its current state of or-
ganization through a series of stepwise changes. Early organizational efforts
implemented by pioneers such as William Halstead [1] and Abraham Flexner
(Flexner report, 1910) around the turn of the century revealed the poor state
of medical education at the time, and set into motion a series of expectations
and resultant improvements that ultimately impacted both medical education
and public safety. The current energy surrounding the ACGME Outcomes
Project and its impact on transition to a competency-based resident training
paradigm are the latest steps in the evolution of medical education.

Brief history of the Accreditation Council for Graduate Medical

Education/role of the Residency Review Committees
The Accreditation Council for Graduate Medical Education (ACGME)
was established in 1981 to serve as a national governing body charged
with accreditation of post-MD residency training programs within the
United States. In effect, the fiduciary responsibility of the ACGME is to
the public as demonstrated in its mission statement: ‘‘We improve health
care by assessing and advancing the quality of resident physicians’ education
through accreditation.’’ In its role as a national governing body, the
ACGME sets general policy and the direction of resident education. How-
ever, recognizing that significant differences exist between specialties, over-
sight at the specialty level is relegated to 27 specialty-specific committees.
It is the responsibility of these Residency Review Committees (RRCs) to
periodically review every residency training program to ensure program-
matic compliance with the standards that are set forth by the ACGME.

E-mail address: bradley.marple@utsouthwestern.edu

0030-6665/07/$ - see front matter Ó 2007 Elsevier Inc. All rights reserved.
doi:10.1016/j.otc.2007.07.003 oto.theclinics.com
1216 MARPLE

ACGME Outcomes Project

In February 1999, the ACGME endorsed the Outcomes Project, which is
funded by the Robert Wood Johnson Foundation and designed to focus on
educational outcomes [1]. This project would eventually lead to fundamental
changes in the way residents are educated. Accreditation would no longer be
granted on the basis of a residency program’s potential to educate trainees
as demonstrated by process assessment (process-oriented education).
Instead, accreditation would be granted based on a series of measurable out-
comes intended to demonstrate an individual resident’s ability to provide
effective patient care, as well as the residency program’s ability to provide
such training (outcomes-oriented education).
With the Outcomes Project in mind, the ACGME introduced a new
model of accreditation in July 2002 that was based on two major require-
ments. First and foremost, it was assumed that the effectiveness of resident
education would be more efficient if alterations were made in resident
lifestyle that decreased fatigue. The first requirement, therefore, addressed
a reduction of the resident workweek to a maximum of 80 hours. In
many cases, this requirement caused a significant decrease in the number
of hours logged by residents during training. The resultant impact on work-
flow, in turn, resulted in major changes to the residency process. Education,
communication, and transfer of patient care between health care providers,
along with other traditional processes of training programs, were modified,
thus leading to the development of novel training strategies. Although it was
initially feared that this potential for reduction in resident exposure to
patient care would lead to physicians ill equipped to function effectively fol-
lowing graduation [2,3], it appears that more effective educational processes
and less fatigued residents have largely overcome these concerns [4]. On the
other hand, limiting the resident workweek has effectively resulted in
measurable decreases in clinical experience during residency training [5,6].
Resolving the mutual demands of a balancing of reduced patient care expo-
sure with an optimization of educational opportunities has given rise to
novel approaches to resident education [7].
In light of the concerns about the ACGME requirement limiting resident
duty hours, the second requirement was appropriately focused on improve-
ments in the quality of the resident educational experience [8]. Under the
model proposed by the ACGME, accreditation would be based on a series
of outcomes-based measures. This approach has created a need to refine rea-
sonable goals and objectives specific to each program, to develop ways of
effectively capturing and optimizing available clinical experiences, to
develop a series of reasonable and validated measures of outcome, and to
develop processes by which collected data would facilitate improvement in
resident and residency program performance. Further, development of the
tools necessary to teach trainees to implement a problem-based approach
to medical care should serve to develop a culture of lifelong learning.
 COMPETENCY-BASED RESIDENT EDUCATION 1217

To date, implementation of the Outcomes Project has been the responsi-

bility of the individual programs, with guidance by the ACGME and the
individual RRCs.

Timeline of the ACGME Outcomes Project

The Outcomes Project is a long-term initiative by which the ACGME is
increasing emphasis on educational outcomes as the basis for accreditation
of residency education programs [8]. In contrast to the current practice of
accreditation based on a potential to educate as measured by determining
compliance with a set of existing requirements, the goal of the Outcomes
Project is to emphasize the actual accomplishments of a training program
as assessed by measurable outcomes. Stated differently, it intends to mea-
sure how effectively education in each program impacts its trainees’ abilities.
The Outcomes Project has been implemented in phases that were pro-
posed on a strict timeline [9]. In each phase a set of programmatic focuses
and accreditation focuses were listed. Under the guidance of the RRC pro-
gram evaluation process, each program would receive feedback regarding
progress.
Phase 1 (July 2001 through June 2002) required that an initial response be
formed to the changes in ACGME requirements. Specifically, each program
was required to define specific objectives that would allow residents to dem-
onstrate learning in the six general competencies. Additionally, the general
competencies were to be integrated into resident didactic and clinical educa-
tional experiences.
In Phase 2 (July 2002 through June 2006), the focus and definition of the
competencies and assessment tools used to evaluate outcomes were to be
sharpened. Learning opportunities in all six of the competency domains
were to be demonstrated. The evaluation processes were to be improved
as needed by each program, under the guidance of the RRC. In addition,
aggregated resident performance data would be accumulated for each
program’s Graduate Medical Education (GME) Committee internal review.
In Phase 3 (July 2006 through June 2011), each program is to fully inte-
grate the general competencies and their assessment into both learning and
clinical care. Resident performance data are to be used as the basis for im-
provement and to provide evidence for the program’s accreditation review.
In essence, this enables Phase 3 of the implementation of the Outcomes
Project to be the point at which competency-based training is to become
operational.
Finally, in Phase 4 (July 2011), the general competencies and their assess-
ment are to be expanded to develop models of excellence. In this phase, as in
the preceding phases, site review by the RRCs will provide an opportunity
to identify and disseminate models of excellence developed within individual
programs. In theory, this process will lead to identification of benchmarks
that will serve to facilitate continual improvement.
1218 MARPLE

The six general competencies

The first major activity of the Outcomes Project was the development of the
six general competencies for residency training, which were endorsed by the
ACGME in 1999 [10]. This process of development was stimulated by an
introspective assessment of how adequately physicians in the current model
of residency training were prepared to practice medicine in the arena of the
changing health care delivery system. The general competencies were
derived through evaluation of existing research on qualities necessary for
physicians to function effectively. Further input was gathered from a variety
of GME stakeholders.
The ACGME approved specific language regarding the general compe-
tencies and evaluation processes in September 1999 [10]. For the pur-
poses of accuracy, the description of the ACGME general competencies
used is that approved by the ACGME [11]. The six general competencies
are
1. Patient care
2. Medical knowledge
3. Practice-based learning and improvement
4. Interpersonal and communication skills
5. Professionalism
6. Systems-based practice

Patient care
Residents must provide patient care that is compassionate, appropriate,
and effective for the treatment of health problems and the promotion of
health. Residents are expected to
 Communicate effectively and demonstrate caring and respectful behav-
iors when interacting with patients and their families
 Gather essential and accurate information about their patients
 Make informed decisions about diagnostic and therapeutic interventions
based on patient information and preferences, up-to-date scientific
evidence, and clinical judgment
 Develop and carry out patient management plans
 Counsel and educate patients and their families
 Use information technology to support patient care decisions and
patient education
 Perform competently all medical and invasive procedures considered
essential for the area of practice
 Provide health care services aimed at preventing health problems or
maintaining health
 Work with health care professionals, including those from other
disciplines, to provide patient-focused care
 COMPETENCY-BASED RESIDENT EDUCATION 1219

Medical knowledge
Residents must demonstrate knowledge about established and evolving
biomedical, clinical, and cognate (eg, epidemiological and social-behavioral)
sciences and the application of this knowledge to patient care. Residents are
expected to
 Demonstrate an investigatory and analytic thinking approach to clinical
situations
 Know and apply the basic and clinically supportive sciences that are
appropriate to their discipline

Practice-based learning and improvement

Residents must investigate and evaluate their patient care practices,
appraise and assimilate scientific evidence, and improve their patient care
practices. Residents are expected to
 Analyze practice experience and perform practice-based improvement
activities using a systematic methodology
 Locate, appraise, and assimilate evidence from scientific studies related
to their patients’ health problems
 Obtain and use information about their own population of patients and
the larger population from which their patients are drawn
 Apply knowledge of study designs and statistical methods to the
appraisal of clinical studies and other information on diagnostic and
therapeutic effectiveness
 Use information technology to manage information, access on-line
medical information, and support their own education
 Facilitate the learning of students and other health care professionals
Interpersonal and communication skills
Residents must demonstrate interpersonal and communication skills that
result in effective information exchange and teaming with patients, their
patients families, and professional associates. Residents are expected to
 Create and sustain a therapeutic and ethically sound relationship with
patients
 Use effective listening skills and elicit and provide information using
effective nonverbal, explanatory, questioning, and writing skills
 Work effectively with others as a member or leader of a health care team
or other professional group
Professionalism
Residents must demonstrate a commitment to performing professional
responsibilities, adherence to ethical principles, and sensitivity to a diverse
patient population. Residents are expected to demonstrate
 Respect, compassion, and integrity; a responsiveness to the needs of
patients and society that supercedes self-interest; accountability to
1220 MARPLE

patients, society, and the profession; and a commitment to excellence

and ongoing professional development
 Commitment to ethical principles pertaining to provision or withholding
of clinical care, confidentiality of patient information, informed consent,
and business practices
 Sensitivity and responsiveness to patients’ culture, age, gender, and
disabilities
Systems-based practice
Residents must demonstrate an awareness of and responsiveness to the
larger context and system of health care and the ability to effectively call
on system resources to provide care that is of optimal value. Residents
are expected to
 Understand how their patient care and other professional practices
affect other health care professionals, the health care organization,
and the larger society and how these elements of the system affect their
own practice
 Know how types of medical practice and delivery systems differ from
one another, including methods of controlling health care costs and
allocating resources
 Practice cost-effective health care and resource allocation that does not
compromise quality of care
 Advocate for quality patient care and assist patients in dealing with sys-
tem complexities
 Know how to partner with health care managers and health care pro-
viders to assess, coordinate, and improve health care and know how
these activities can affect system performance
Components of a competency-based education program
In essence, the current Outcomes Project is an example of a competency-
based educational model. This model is designed to focus on the perfor-
mance of the individual trainee, as measured in outcomes, as he or she
works to master educational goals and objectives. For this form of educa-
tion to be effective, a number of elements must be in place. Further, these
elements should be integrated in such a way that meaningful data are gen-
erated and continually integrated into the educational environment, thus
providing a progressive, graduated experience for the learner.
A clear set of educational goals and objectives serves to set the stage for the
educational environment and aligns the expectations of the learners with
those of the program. Goals and objectives should be specific to the educa-
tional site and experience level of the resident, in effect serving to continually
challenge and develop trainees’ skill sets. It is important that this set of goals
and objectives can be objectively measured to track progress and identify
areas that need remediation. Finally, the goals and objectives should attempt
to reflect the need to gain experience in all six of the general competencies.
 COMPETENCY-BASED RESIDENT EDUCATION 1221

At first glance, the efforts required to transform residency programs as

suggested by the ACGME Outcomes Project appeared monumental, but
most programs have been able to modify existing didactic and clinical expe-
riences to align with the general competencies required by the ACGME
[12,13]. A recommended approach to help assess programmatic alignment
with new ACGME regulations is to address three questions: Do the resi-
dents achieve the learning objectives set forth by the program? What evi-
dence can the program provide that it does so? How does the program
demonstrate continuous improvement in its educational process? [1].
Measurement, tracking, and documentation of the goals and objects set
forth by each program provide a mechanism by which to ensure effective
education. However, this step in the process of educational transition may
also present a challenge. Large amounts of data generated by this process
will require secure storage and organization if they are to be useful. Many
programs have used computer-based systems that can log resident duty
hours, track a variety of performance measures, and identify areas of non-
compliance/substandard performance. Further, these systems can prepare
reports (ie, performance ‘‘dashboards’’) and alert program directors early
in the course of deviations from expected performance [1]. Varying levels
of access can be provided to residents, faculty, program directors, and chair-
men ‘‘chairpersons’’ (or simply ‘‘others’’)? to allow for ongoing access to
pertinent information.
A variety of assessment tools has been developed and made available as out-
come metrics through the ACGME Outcomes Project Toolbox [8]. Table 1 is
a partial list of assessment tools that are currently in common use. Each of the
listed metrics has been made available for use by residency programs.
Ultimately, programs are given the flexibility to use the assessment tools
made available by the ACGME to facilitate measurement of the effective-
ness of a residency program in its preparation of residents. Although
some assessment tools may be tailored to assess most, if not all, of the gen-
eral competencies, some are less capable of this function. Thus, the use of
several tools is recommended to optimize the validity of the assessment port-
folio collected for each resident. Given that each program may approach
integration of common program requirements, goals and objectives, and
the way in which data are gathered in a different fashion, some general
guidelines are offered by the ACGME. Programs are expected to [8]
 Provide learning opportunities in each general competency domain
 Use multiple and overlapping assessment methods
 Aggregate data such that they improve the educational program

Expectations for the future

As noted previously, the ACGME Outcomes Project is intended to facil-
itate a change in the current system of medical education. Fundamental to

Table 1
Common methods of assessment
Assessment tool
360 evaluation instrument
Global rating of live or recorded performance
Procedure, operative, or case logs
1222
Description
360 evaluations are measurement tools designed to be
completed by a variety of individuals involved in the
examinee’s sphere of influence. Evaluations are completed in a ‘‘360 ’’
fashion, signifying participation by peers, superiors, and subordinates.
A standard survey or questionnaire is used to gather information
and can be tailored to easily align with the general competencies.
MARPLE
A variety of rating scales can be used.
A global rating is a retrospective assessment in which a rater
assesses general categories of an ability (eg, patient care skills, medical
knowledge, interpersonal and communication skills). The
rating is based on general information and impressions derived from
multiple sources of information (eg, direct observations or interactions;
input from other faculty, residents, or patients; review of work
products or written materials) and is collected over a period of time
(eg, end of a clinical rotation). Rating forms frequently allow
the evaluator to make scaled responses (eg, 5 ¼ outstanding,
4 ¼ good, 3 ¼ fair, 2 ¼ marginal, 1 ¼ unsatisfactory).
Written comments allow evaluators to expand on responses.
Procedure, operative, or case logs document patient encounters,
surgeries, or procedures performed. The logs include
counts of cases, operations, or procedures.
This type of information is intended to
document sufficient exposure to clinical experience.
Patient surveys Patient surveys assess satisfaction of the health care experience from the
perspective of the patient and frequently include comments about the
physician’s care. The questions often assess satisfaction with physician
communication skills, time spent with the patient, perceived competency,
and so forth. Patients are typically asked to rate satisfaction through
the use of rating categories ranging from excellent to poor.
Each rating is given a value that can then be averaged to generate
a comprehensive score overall or separated into
separate clinical domains.
Portfolios A portfolio is a collection of resident-collected products that provides

COMPETENCY-BASED RESIDENT EDUCATION

evidence of learning and achievement related to a learning plan. This
portfolio may contain written documents, photographs, case logs, and
other forms of information. Resident preparation of these materials
provides an important opportunity to reflect on what he or
she had learned. Additionally, a written synopsis of the
portfolio provided by the resident can be used to identify
areas of strength and weakness.
Simulations and models Simulations use models that duplicate actual clinical
scenarios and are used to assess clinical skills and
performance. Effective simulations allow examinees
to perform skills or act out potentially harmful situations
without placing a patient at risk. Additionally, simulations
can be used to provide a safe environment within which a skill can be
practiced and acquired before its introduction into the
patient care setting. Feedback can be immediate,
allowing the learner to modify mistakes and reinforce successes in
subsequent simulations.
(continued on next page)

1223

Table 1 (continued )
Assessment tool
Standardized ral examination
Written examination
Checklist evaluation
1224
Description
The standardized oral examination uses a realistic patient
case as the basis of a series of questions administered by
a trained physician examiner. Questions assess acquisition of clinical
information, rationale for requesting testing, interpretation
of findings, and treatment plans.
MARPLE
A written examination is composed of multiple-choice
questions created to sample medical knowledge. Each
question consists of an introductory statement
(or question), which is followed by a series
(typically four or five) options. The examinee
selects one of the options as the correct response.
Checklist evaluations use a series of ‘‘yes’’ or ‘‘no’’ questions addressing
specific behaviors, activities, or steps that make up a more complex
competency or competency component.
 COMPETENCY-BASED RESIDENT EDUCATION 1225

this change is a migration from the current process-based model to one of

a competency-based model in hopes of generating data on which continuous
refinements can be made. Effective use of these data, once processes are in
place, can enable individualized training that is tailored to reinforce
strengths and remediate problems early in the training process. This change
has the potential to result in modifications of the residency accreditation
process for residencies as well as the certification process for trainees. As
an example, case logs emphasizing absolute numbers of operative cases
may be replaced, or supplemented, by outcome metrics that demonstrate
an individual resident’s proficiency in the skills necessary to effectively
accomplish a range of related procedures. Simultaneously, a program is
provided with continuous information that enables improvements in the
methods by which training takes place. Making use of such an example of
competency-based training may provide for individual certification based
on demonstrated proficiency.

References
[1] Joyner BD. An historical review of graduate medical education and a protocol of accredita-
tion Council for Graduate Medical Education compliance. J Urol 2004;179:34–9.
[2] Winslow ER, Bowman MC, Klingensmith ME. Surgeon workhours in the era of limited
resident workhours. J Am Coll Surg 2004;198(1):111–7.
[3] Romano M. Hours of doctors-in-training: who’s counting? ACGME’s restrictions limiting
workhours denounced as inadequate. Mod Healthc 2002;32(33):18–9.
[4] deVirgillo C, Yaghoubian A, Lewis RJ, et al. The 80-hour workweek does not adversely
affect patient outcomes of resident education. Curr Surg 2006;63(6):435–9.
[5] Weatherby BA, Rudd JN, Ervin TB, et al. The effect of resident work hour regulations on
orthopaedic surgical education. J Surg Orthop Adv 2007;16(1):19–22.
[6] Short AC, Rogers SJ, Magann EF, et al. The 80-hour workweek restriction: How are
OB/GYN resident procedure numbers affected? J Matern Fetal Neonatal Med 2006;
19(12):801–6.
[7] Woodrow SI, Segouin C, Armbruster J, et al. Duty hours reforms in the United States,
France, and Canada: is it time to refocus our attention on education? Acad Med 2006;
81(12):1045–51.
[8] Accreditation Council on Graduate Medical Education. Available at: http://www.acgme.
org. Accessed May, 2007.
[9] Accreditation Council on Graduate Medical Education. Available at: www.acgme.org/
outcome/project/timeline/TIMELINE_index_frame.html. Accessed May, 2007.
[10] Accreditation Council on Graduate Medical Education. Available at: http://www.acgme.
org/outcome/project/OPintrorev1_7–05.ppt-12. Accessed May, 2007.
[11] Accreditation Council on Graduate Medical Education. Available at: www.acgme.org/
outcome/comp/compFull.asp. Accessed May, 2007.
[12] Dyre PL, Strauss RW, Rinnert S. Systems-based practice: the sixth core competency. Acad
Emerg Med 2002;9:1270–7.
[13] Hobgood CO, Riviello RJ, Jouriles N, et al. Assessment of communication and interpersonal
skills competencies. Acad Emerg Med 2002;9:1305–9.