American Journal of Pharmaceutical Education-7

Vol ume 78, Issue 7, 2014
I SSN: 0002-9459
About AJPE
The American Journal of Pharmaceutical Education (AJPE) is the official publication of the
American Association of Colleges of Pharmacy (AACP). Its purpose is to document and advance
pharmaceutical education in the United States and Internationally. The Journal considers material
in all areas related to pharmaceutical education. Through open-access Internet publication the
Journal intends to take full advantage of the electronic medium; this includes the publication of
articles with multimedia features, encompassing 3D graphics, video, interactive figures and
databases, and sound. The Journal Editor is Joseph T. DiPiro, Professor and Executive Dean at
the South Carolina Colege of Pharmacy.
The electronic Journal will be issued quarterly. In addition, a year-end print volume will be made
available to AACP member institutions and to others upon request for a nominal fee. Access to
the electronic Journal will not be restricted by password.
The Journal accepts unsolicited manuscripts that have not been published and are not under
consideration for publication elsewhere. All manuscripts are subject to peer review and approval
by the Editor before publication. Authors must prepare manuscripts to conform to the Journal
style. Authors are not assessed page charges for publication. All reports and papers presented at
the Annual Meeting of the AACP, as well as manuscripts accepted for publication, become the
property of AACP. All requests for reassignment of the above rights should be directed to the
Editor.
Referees are assigned by the Editor with the advice of the Editorial Board as needed.
Manuscripts are processed on line by the Editor, Editorial Assistant, and the American
Association of Pharmaceutical Scientists using a computer-based tracking system that relies on
keyword selection and automatic time lines for review. The electronic submission and review
process is intended to expeditiously complete the publication process using electronic
communication between the editorial office, editors, referees, and authors. Once an article is
accepted, a final version is sent to the author for approval by email. The article is published on
the AJPE web site shortly thereafter. All expressions of opinion and statements of supposed fact
appearing in the Journal are not to be
regarded as necessarily expressing the policy or views of the Editor or of AACP.
Articles appearing in the Journal are indexed in: PubMed/Medline; Current Contents, Education;
Current Contents, Life Sciences; International Pharmaceutical Abstracts; Current Index to
Journals in Education; and by PROQuest and EBSCO.
The Journal has been published continuously since 1937. Past Editors of the Journal were:
Rufus A. Lyman, University of Nebraska 1937-1955

Melvin R. Gibson, Washington State University 1956-1960
C. Boyd Granberg, Drake University 1961-1974
Marvin M. Malone, University of the Pacific 1975-1979
George H. Cocolas, University of North Carolina 1980-2002
Editorial Office
The Journal office is located on the University of South Carolina campus in Columbia. Address all
communications concerning manuscripts to the Editorial office. Mailing address and contact
information is as follows:
Address:
American Journal of Pharmaceutical Education

715 Sumter St.
Columbia, SC 29208
Editorial Office:
Fax:
803-777-3096
803-777-3097
Business Office
The business office is located at the AACP office. Mailing address and contact information is as
follows:
Address:
727 King Street, Alexandria, VA 22314
Phone:
703-739-2330
Fax:
703-836-8982
Copyright 2011, American Association of Colleges of Pharmacy, 1426 Prince Street,

Alexandria, VA 22314. All rights reserved.
Editorial Board-AJPE
Joseph T. DiPiro, PharmD
Editor, South Carolina College of Pharmacy
Gayle A. Brazeau, PhD
Associate Editor, University of New England
Jack E. Fincham, PhD
Associate Editor, University of Missouri - Kansas City
Claire Anderson, PhD, BPharm
International Associate Editor, University of Nottingham, United Kingdom
Naser Alsharif, PhD

Creighton University
Zubin Austin, PhD
University of Toronto
Alicia Bouldin, PhD
The University of Mississippi
Eric Boyce, PharmD
University of the Pacific
Susan Burton
South Africa
Shelley Chambers, PhD
Washington State University
Sudip K. Das, PhD
Butler University
Patrick J. Davis, PhD
University of Texas-Austin
Gary E. DeLander, PhD
Oregon State University
JoLaine R. Draugalis, PhD
University of Oklahoma
Catherine A. Elstad, PhD
Washington State University
Deborah Harper-Brown, PharmD

Chicago State University
Mohamed Azmi Ahmad Hassali
University Sains Malaysia
Monica Holiday-Goodman, PhD
University of Toledo
Peter Hurd, PhD
St. Louis College of Pharmacy
Harold Kirschenbaum, PharmD
Long Island University
Jennifer Marriott
Monash University
Susan Meyer, PhD
University of Pittsburgh
Adam Persky, PhD
University of North Carolina
Anna Ratka, PharmD, PhD
Texas A&M University
Frank Romanelli, PharmD, MPH
University of Kentucky
Amy Seybert, PharmD
University of Pittsburgh
Brent Fox, PhD
Auburn University
Stuart Haines, PharmD

University of Maryland
Cecilia Plaza, PharmD, PhD

AACP Liaison
Vol ume 78, Issue 7, 2014

I SSN: 0002-9459
Contents
VIEWPOINTS
1.
Turning the World of Pharmacy Education into a Global Community

Through Sharing
Claire AndersonClaire Anderson, Jennifer L. MarriottJennifer L. Marriott,
Joana CarrasqueiraJoana Carrasqueira, Tina P. BrockTina P. Brock,
Timothy RennieTimothy Rennie, Andreia F. BrunoAndreia F. Bruno,
Ian BatesIan Bates
1-2
SPECIAL ARTICLES
2.
Complex Issues Affecting Student Pharmacist Debt

3-6
Jeff CainJeff Cain, Tom CampbellTom Campbell, Heather Brennan CongdonHeather Brennan
Congdon, Kim HancockKim Hancock, Megan KaunMegan Kaun, Paul R. LockmanPaul R.
Lockman, R. Lee EvansR. Lee Evans
RESEARCH
3.
A Three-Year Study of the Impact of Instructor Attitude, Enthusiasm, and

Teaching Style on Student Learning in a Medicinal Chemistry Course
Naser Z. AlsharifNaser Z. Alsharif, Yongyue QiYongyue Qi
4.
Identifying Achievement Goals and Their Relationship to Academic

20-27
Achievement in Undergraduate Pharmacy Students
Saleh AlrakafSaleh Alrakaf, Erica SainsburyErica Sainsbury, Grenville RoseGrenville Rose,
Lorraine SmithLorraine Smith
5.
Self-Efficacy and Self-Esteem in Third-Year Pharmacy Students

Mark L. YorraMark L. Yorra
6.
Investigating the Relationship Between Pharmacy Students Achievement

33-38
Goal Orientations and Preferred Teacher Qualities
Saleh AlrakafSaleh Alrakaf, Erica SainsburyErica Sainsbury, Grenville RoseGrenville Rose,
Lorraine SmithLorraine Smith
7-19
28-32
INSTRUCTIONAL DESIGN AND ASSESSMENT

7.
Long-term Effectiveness of Online Anaphylaxis Education for Pharmacists

39-49
Sandra M SalterSandra M Salter, Sandra ValeSandra Vale, Frank M SanfilippoFrank M Sanfilippo,
Richard LohRichard Loh, Rhonda M CliffordRhonda M Clifford
8.
A Physical Assessment Skills Module on Vital Signs

50-56
Christine LeongChristine Leong, Christopher LouizosChristopher Louizos, Grace FrankelGrace
Frankel, Sheila NgSheila Ng, Harris IacovidesHarris Iacovides, Jamie FalkJamie Falk, Drena
DunfordDrena Dunford, Kelly BrinkKelly Brink, Nancy KleimanNancy Kleiman, Christine
DavisChristine Davis, Robert RenaudRobert Renaud
9.
Student Self-Screening for Methicillin-Resistant Staphylococcus Aureus

57-63
(MRSA) Nasal Colonization in Hand Hygiene Education
Tia LumTia Lum, Kristin PicardoKristin Picardo, Theresa WestbayTheresa Westbay, Amber
BarnelloAmber Barnello, Lynn FineLynn Fine, Jill LavigneJill Lavigne
10.
Using Curriculum Mapping to Engage Faculty Members in the Analysis

64-72
of a Pharmacy Program
Sheryl ZelenitskySheryl Zelenitsky, Lavern VercaigneLavern Vercaigne, Neal M. DaviesNeal M.
Davies, Christine DavisChristine Davis, Robert RenaudRobert Renaud, Cheryl KristjansonCheryl
Kristjanson
11.
Improved Knowledge Retention Among Clinical Pharmacy Students

Using an Anthropology Classroom Assessment Technique
Heather P. WhitleyHeather P. Whitley, Jason M. PartonJason M. Parton
73-82
12.
An Active Learning Complementary and Alternative Medicine Session

in a Self-Care Therapeutics Class
Melissa J. MattisonMelissa J. Mattison, Eric C. NemecEric C. Nemec
83-86
13.
A Multiyear Analysis of Team-Based Learning in a Pharmacotherapeutics Course 87-95

June Felice JohnsonJune Felice Johnson, Edward BellEdward Bell,
Michelle BottenbergMichelle Bottenberg, Darla EastmanDarla Eastman,
Sarah GradySarah Grady, Carrie KoenigsfeldCarrie Koenigsfeld, Erik MakiErik Maki, Kristin
MeyerKristin Meyer, Chuck PhillipsChuck Phillips, Lori SchirmerLori Schirmer
American Journal of Pharmaceutical Education 2014; 78 (7) Article 1.
VIEWPOINTS
Turning the World of Pharmacy Education into a Global Community
Through Sharing
Claire Anderson, PhD, BPharm,ab Jennifer L. Marriott, PhD, BPharm,b Joana Carrasqueira, MPharm,b
Tina P. Brock, EdD, MS,b Timothy Rennie, PhD, MPharm,b Andreia F. Bruno, PhD, MPharm,b Ian Bates,
PhD, BPharmb
a
b
Associate Editor, American Journal of Pharmaceutical Education

University of Nottingham, Nottingham, United Kingdom
Barriers to delivering quality pharmacy education in

developing countries have been identified as limited infrastructure, access to teaching resources, academic staff
development, and research capacity.1 In 2010, the International Pharmaceutical Federation (FIP) and the United
Nations Educational, Scientic and Cultural Organization (UNESCO) signed an agreement to develop a FIP
UNESCO-UNITWIN Global Pharmacy Education Development (GPhED) program. The GPhED network
strives to raise the standard of education globally and incorporate best practices through shared resources, expertise, and experience. UNESCO and FIP consider one of
the essential factors favoring development in the eld of
professional competence the exchange of experience and
knowledge between universities and other learning institutions. Additionally, collaborative practice by university
teachers, researchers, and administrators from different
regions across the world could benet the entire professional community.
The aims of the FIP UNESCO-UNITWIN program
are global higher education and its development, with
a special interest in African nations and low-income countries, empowerment of women as pharmaceutical scientists
and academics, sustainable health workforce development,
academic capacity building, quality assurance, and accreditation standards, which are all compatible with the overarching aims of UNESCO.2,3 Objectives of the program
include promotion of an integrated system of research,
training, information, and documentation in the eld of
pharmacy education through Centres of Excellence,
which will facilitate intra-regional cooperation.
Sharing in all areas is the key and to this end the
GPhED program is currently developing three areas of
activity that focus on sharing of teaching and learning
resources, pharmaceutical education expertise and experience, and research capacity.

Resource sharing within the program is being facilitated through SABER (Sharing and Building Educational
Resources), an online platform (www.saber.monash.edu)
from which resources can be downloaded, used, and shared
internationally between academic institutions to improve
teaching and learning. Enhanced pharmacy education creates better pharmacists, but creating high-quality student
learning resources takes time and money. Sharing educational resources makes sense so students can benet and
learn more, especially in a global profession such as pharmacy where teaching needs are often similar. Sharing and
collaborating creates and strengthens partnerships based
on goodwill and a shared commitment to educational excellence. The SABER platform is a place to share, discover, acquire, and re-purpose resources for pharmacy
education. It also encourages the collaborative creation
of new content. SABER is available to educators in pharmacy schools around the world, hosting quality-assured
educational resources and providing a trustworthy source
of relevant and current material. Being multi-tiered, it
allows various access levels for a broad range of users.
SABERs intuitive interface allows users to quickly
nd existing assets, and up-to-date social-networking
tools let them build peer networks to adapt resources
and create new material. The site is administered by
Monash University on behalf of the GPHeED network
and is accessible to all for free. Under a Creative Commons license, SABER enables academics anywhere in
the world to share something as simple as an image or a
PowerPoint presentation, to large purpose-designed teaching programs in areas relating to pharmacy and pharmaceutical science. The success of SABER depends on
academics contributing resources they have developed
to share via the database.
Another important component in the GPhED program is the formation of the African Centre of Excellence
in Pharmacy Education. Ideally, the Centre will facilitate
Corresponding Author: Claire Anderson, PhD, BPharm,

University of Nottingham, University Park, Nottingham
NG7 2RD, United Kingdom. E-mail: Claire.Anderson@
Nottingham.ac.uk
American Journal of Pharmaceutical Education 2014; 78 (7) Article 1.

the sharing of ideas, skills, resources, and good practice,
including staff exchange for skills and capacity building.
The Centre will provide a forum for discussion and debate
on trends and developments in pharmacy education, facilitated by network partners, including NGOs and professional agencies, and coordinated by the founding
partners. Africa was chosen to establish the first FIP
UNESCO-UNITWIN Centre of Excellence because the
region is in great need of a pharmacy workforce and lacks
educational resources for universities.4
A recent meeting of the 5 founding partner countries
Ghana, Namibia, Nigeria, Uganda and Zambia was held
in Lusaka, Zambia to determine the future activities of the
Centre. Five domains for the Centre of Excellence were
determined (communication, capability, quality, innovation, and clinical). Each founding partner agreed to take
the lead in one of these domains and coordinate projects or
activities within them. The founding partners developed
and agreed to a communications strategy and will contribute to advocacy, local network building, and communications to ensure the wider success of the Centre of
Excellence. Proposed projects of the Centre include a survey of African colleges and schools of pharmacy to establish a database of academic capacity and expertise to
facilitate intra-regional sharing of expertise through a visiting academic program. Another project is the development of a Lab-box of basic laboratory equipment to
improve students ability to undertake laboratory experiments to support their learning of basic science concepts.
The Centre of Excellence will invite additional countries
to become part of the centre to expand its activities and
provide a broader base of communication and support in
the region. Centre of Excellence activities will also seek
to promote gender equality and empowerment for women
academics and scientists in collaborative research and

policy development.
The GPhED hopes to foster international collaborative research by having prospective research students
complete a higher degree in their home university in
Africa under the guidance of a local supervisor and an
experienced international researcher with expertise in
the chosen research area. The hope is that the arrangement
of shared supervision will enable collaboration in policy
synthesis and strategic development, with a focus on international development issues and sustainable practitioner development policies. Public health or service
delivery projects are favored by the African universities
due to the limited laboratory facilities available, but such
projects will contribute relevant health data in the country
and increase the capacity for improved health programs
and delivery of pharmacy services.
The FIP UNESCO-UNITWIN Global Pharmacy Education Development network is in the nascent stages of
development, but through the goodwill, enthusiasm, and
support of academics across the globe, sharing educational resources, expertise, and experience through such
networks will facilitate the provision of quality pharmacy
education in developing countries.
REFERENCES
1. FIPEd Global Education Report 2013 http://www.p.org/
educationreports. Accessed June 30, 2014.
2. UNESCO 2008-2009 Biennium Priorities (34 C/5): http://www.
unesco.org/science/doc/SC_34_C5.pdf. Accessed June 30, 2014.
3. UNESCO 2008-2013 Mid-term Strategy (34 C/4): http://unesdoc.
unesco.org/images/0014/001499/149999e.pdf. Accessed June 30, 2014.
4. FIP Global Pharmacy Workforce Report https://www.p.org/
static/peducation/2012/FIP-Workforce-Report-2012/?
page5hr2012. Accessed June 30, 2014.
American Journal of Pharmaceutical Education 2014;78(7) Article 2.
SPECIAL ARTICLES
Complex Issues Affecting Student Pharmacist Debt
Jeff Cain, EdD, MS,a Tom Campbell, PharmD,b Heather Brennan Congdon, PharmD,c
Kim Hancock, PhD,d Megan Kaun, PharmD,e Paul R. Lockman, PhD,f and R. Lee Evans, PharmDg
a
College of Pharmacy, University of Kentucky, Lexington, Kentucky

College of Pharmacy, Lipscomb University, Nashville, Tennessee
c
School of Pharmacy, University of Maryland, Baltimore, Maryland
d
College of Pharmacy, Ferris State University, Big Rapids, Michigan
e
College of Pharmacy and Pharmaceutical Sciences, The University of Toledo, Toledo, Ohio
f
School of Pharmacy, West Virginia University, Morgantown, West Virginia
g
Harrison School of Pharmacy, Auburn University, Auburn, Alabama
b
Submitted November 25, 2013; accepted January 26, 2014; published September 15, 2014.
It is time for colleges and schools of pharmacy to examine and confront the rising costs of pharmacy
education and the increasing student loan debt borne by graduates. These phenomena likely result from
a variety of complex factors. The academy should begin addressing these issues before pharmacy education
becomes cost-prohibitive for future generations. This paper discusses some of the more salient drivers of cost
and student debt load and offers suggestions that may help alleviate some of the financial pressures.
Keywords: student loan, debt, tuition, higher education
higher education funding over the last several years. At first,

most public institutions absorbed state funding reductions
through spending cuts and efficiency measures. However,
after those means were exhausted, the only major recourse
to fund educational activity was through tuition increases,
effectively shifting more of the costs to students.2 In 2008,
31.6% of revenue used to cover public higher education
operating expenses came from tuition, compared to 42.4%
in 2012.3After ination adjustments, annual tuition at 4-year
public colleges has increased by $1,850, or 27%, since the
2007-08 school year.2 Compounded over several semesters,
the increased tuition adds substantially to the total cost of
higher education for an individual and is a primary factor for
increased student debt.4
INTRODUCTION
The rising cost of higher education in the United
States is a disturbing trend. Pharmacy school tuition and
resulting student indebtedness have risen significantly
over the last decade.1 Initial assumptions typically suggest that the responsibility for increased student loan debt
belongs to students. However, we assert that rising tuition
and student debt is a multifaceted, complex issue that has
origins within the academy, the accreditation process,
federal and state governments, universities, and nally
student and faculty culture. Reducing or even maintaining
current cost levels for pharmacy education and concomitant student debt will not be straightforward, easy, or
without controversy, but it is imperative that the academy
confront the issues before they worsen. In this paper, we
discuss some of the more notable inuences on cost and
student debt load and suggest potential actions that may
allay the nancial burdens.
GOVERNMENT ISSUES
Government regulation of loan eligibility, loan
amounts, interest rates, repayment structures, loan deferment eligibility, and government loan forgiveness programs all impact the total cost of student loan debt.
Student loan interest rates arguably have the single largest impact on student loan debt. On July 1, 2013, interest
rates on certain types of loans doubled because Congress
failed to prevent an expiration of subsidies.5 The federal
government, as opposed to the borrower, pays interest
accrued on subsidized loans during periods of eligible
deferment. This change to subsidized loans had the
potential to nearly double the total payment amount of
REDUCED STATE SUPPORT OF HIGHER

EDUCATION
Faced with financial pressures from the recent economic recession, state legislatures have significantly reduced
Corresponding Author: Jeff Cain, EdD, MS, Department of
Pharmacy Practice & Science, University of Kentucky
College of Pharmacy, 789 South Limestone, Lexington, KY
40536-0596. Tel: 859-257-4429. Fax: 859-257-7297.
E-mail: jeff.cain@uky.edu

a loan. However, in August 2013, Congress passed a bipartisan deal to lower interest rates and tie them to market rates. Although this legislation has led to immediate
relief for many borrowers, it will not protect future borrowers in a stronger economy when interest rates may
rise to as much as 9.5% for graduate and professional
students.6
Another factor affecting pharmacy graduates is recent
change to federal regulations that no longer require lenders
to place student loans into forbearance for a pharmacy residency or fellowship. The new regulation states that only
medical and dental residencies qualify for mandatory loan
forbearance.7 While this has a smaller effect on student
debt, it could cause some pharmacy students to opt out of
postgraduate training because of inability to afford or unwillingness to make loan payments.
part because of the extensive and often redundant governmental, funding agency, and accreditation requirements, institutions and schools now employ large
numbers of noninstructional staff members to develop
and/or maintain technical systems and to capture, track,
record, analyze, and report data (nancial, assessment,
accreditation, etc.) for accountability purposes. These
increases in staff members have been disproportionate
to increases in student enrollment.10 In 2007, after
adjusting for increased student enrollment, it took
13.1% more employees than it did in 1993 to educate
the same number of students.11 While accountability is
a necessary and desirable aspect of public institutions,
the nancial effects of maintaining it are potentially
borne, at least partially, by students through rising tuition and fees.
STUDENT PERSONAL FINANCE

AWARENESS AND KNOWLEDGE
CURRICULAR ISSUES
The time required to earn a degree and the students/
potential applicants ability to earn money throughout an
educational career are often important factors for selection of a major. Faculty members set the standards of
what, how much, and how in depth the educational process will be throughout the degree program. From a students perspective on nancing an education, the
educational and experiential schedules of pharmacy
school may limit their ability to earn income while in
school. Furthermore, some curricula seem to focus too
much on delivering an increasing amount of content instead of focusing on more selective content paired with
better instructional strategies. One of the more difcult
questions that pharmacy educators need to ask themselves
is: Can we design effective curricula that is delivered
more quickly and/or more efciently (without sacricing
quality), saving students either money or time? The higher
education landscape is shifting, with more attention to
widespread online delivery12 and reconsideration of the
credit hour as a metric for student abilities.13 Calls are
being made to reduce the cost of medical education14 and
pharmacy school administrators should be doing the same
for pharmacy education.
Preprofessional requirements have also grown by
525% from 2006 to 2011 in the number of pharmacy programs requiring 3 years of prerequisite courses or a bachelors degree, versus 2 years of courses for admittance.15
There is currently no consensus regarding the ideal length
or content for prepharmacy curricula.16 While theoretically
the more undergraduate work that entering students have
completed, the more mature and better equipped they will
be for pharmacy school, this potentially comes at the cost of
another year of educational expenses and the effective loss
of a years salary.
Students and their families must accept personal

responsibility for aspects of their financial future. While
factors pertaining to tuition, salaries, and job prospects
are out of their control, students do make choices that
affect their education-related debt. Some students engage in lifestyles that significantly exceed their income
while in college, purchasing, for example, automobiles,
clothing, and electronic devices, and even taking elaborate vacations.8As a result, student debt extends beyond
school-related nancial aid to include additional credit
card debt. In 2009, a study by Sallie Mae found that 84%
of undergraduates have at least one credit card and 50%
of undergrads have 4 or more credit cards and use these
because they have insufcient savings or nancial aid to
cover all of their expenditures. Furthermore, 68% of
college students have charged items to their credit cards
knowing they did not have sufcient funds to pay the bill.
The long-term effects of nancing these discretionary
purchases can result in student loan and credit card debt
considerably higher than what is necessary. Most students are aware that this is a problem, with 84% of undergraduate students stating they want more education
on nancial management topics.9
INCREASED PERSONNEL FOR

ADMINISTRATIVE AND TECHNICAL
TASKS
Higher education has traditionally been a laborintensive profession, primarily because of the specialized set of faculty skills and expertise. In recent years,
however, the bulk of labor costs have begun to shift away
from faculty toward managerial, technical, and support
staff.10 In addition to other types of support staff and in
4

ARMS RACE FOR FACILITIES AND
RESOURCES
principles to students prior to graduation, but some students

may already be deep in debt before those lessons are taught.
We propose 5 recommendations that the academy could
employ to minimize this problem.
(1) Integrate required nancial management coursework into the curriculum. This coursework needs
to be employed early (preferably within the rst
year of the doctor of pharmacy program) so that
the principles can be employed throughout the
students educational career. Curriculum should focus on designing and living within a budget, avoiding unnecessary debt and overuse of credit cards,
managing nancial aid in a responsible manner, and
obtaining nancial advice for postgraduate loan repayment, insurance coverage, and timing of large
purchases.
(2) Designate an appropriate faculty/staff member or
external expert to provide ongoing nancial counseling to students throughout their educational career. Access to nancial aid ofces varies among
college campuses, and even in the best of scenarios,
those services may lack the scope of counseling that
students need related to nancial management. A
designated counselor who can build a level of trust
with students might encourage a more open line of
communication with and a more receptive response
from students.
(3) Inform all students of the Federal Student Loan
Forgiveness Program, which forgives student debt
after a period of service. To qualify for the program, students must work for a public institution
for 10 years (nonconsecutive) and make 10 years
worth of qualifying payments. All remaining student debt will be forgiven after the 10 years.25
Pharmacists who take advantage of this program
can potentially reduce their student debt by tens of
thousands of dollars.
(4) Encourage national pharmacy organizations to
provide nancial management programming at
association meetings that student pharmacists attend. Programming should include student-tostudent teaching and best practices (ie, roundtable
discussions), allowing students to share what has
worked as well as mistakes to avoid.
(5) Encourage administrators at colleges and schools
of pharmacy to assess their institution to ensure
that students are not overburdened by the cost of
their education. This might necessitate scrutiny of
tuition models and internal operations to determine
if students are paying only their fair share. At the
university level, use of technology should be explored to reduce staff numbers and time necessary
Recent significant expansion of pharmacy education

has led to competitive student recruitment based more
and more on reputation. Factors that enhance reputation
(buildings, technology, student amenities) are all costly,
thereby incentivizing an academic arms race to see who
can spend the most money.17 Colleges and schools of pharmacy are not immune and are continually seeking every
possible edge in attracting the highest quality students. A
National Bureau of Economic Research working paper
reported that college students place a high value on consumption amenities such as student activities, dormitories,
and sports.18 The recent addition and enhancement of
student services to include social, emotional, and career
counseling has also accounted for spending growth.19,20
Institutional marketing and recruitment strategies now
highlight key amenities such as technology, cutting edge
pedagogical approaches, and state-of-the-art facilities in
order to attract students to their programs.21 However, even
if these amenities are deemed necessary, the price tag associated with new infrastructures, technology implementation, and student services is substantial, driving the cost of
an education even higher with little to no evidence that
learning is improved. Moreover, higher education might
benet if every institution ceased the expansion of facilities,
athletics, and student amenities. Unfortunately, market
forces perpetuate continued escalation, and no institution
can unilaterally withdraw from this arms race without putting itself at risk of falling behind.22
RECOMMENDATIONS
A Center for College Affordability policy paper has
been released that offers systemic methods universities can
use to make education more affordable (such as reforming
financial aid, digitizing academic libraries, and streamlining redundant programs) and reverse the trend of rising
costs.17 We offer some additional recommendations for
the academy to consider.
While public funding of higher education will not
likely revert completely to previous levels, the academy
must continue to lobby for federal and state support. Without pressure, state and federal legislatures may reduce financial allocations even further. Additionally, lobbying
efforts should be directed toward securing adequate financial aid funding for students, particularly grants and low
interest loans.
As mentioned previously, many students today
struggle with increased loan balances and significant
debt related to unwise personal financial management.
Many colleges and schools of pharmacy offer personal finance courses23,24 or provide brief nancial management
5

to comply with accreditation and other accountability demands. Extracurricular amenities should
be closely scrutinized and avoided if they result in
additional costs to students without providing
added value to their professional education. Additionally, the curriculum (including prepharmacy)
should be analyzed to determine if students are
loaded with coursework that adds little value to
their overall degree and their ability to practice
pharmacy. This type of scrutiny may be a painful
process for faculty members, but it could result in
a much more streamlined and effective curriculum.
While fraught with numerous issues, there are also
intriguing possibilities for collaboration within the
academy with regard to sharing educational content online. We encourage the American Association of Colleges of Pharmacy to explore potential
content sharing mechanisms that create pedagogical efciencies. Pharmacy educators have the responsibility as faculty members of not acting out of
self-interest, but in providing the best and most
economical education to students.
for College Affordability and Productivity. http://www.

centerforcollegeaffordability.org/uploads/Trends_LaborForce.pdf.
Accessed November 4, 2013.
11. Greene JP, Kisida B, Mills J. Administrative bloat at American
universities: the real reason for high hosts in higher education.
Goldwater Institute Policy Report No. 239. August 17, 2010. http://
goldwaterinstitute.org/sites/default/les/Administrative%20Bloat.
pdf. Accessed September 3, 2013.
12. Pirani J. A compendium of MOOC perspectives, research, and
resources. Educause Rev. http://www.educause.edu/ero/article/
compendium-mooc-perspectives-research-and-resources. Published
November 4, 2013. Accessed November 6, 2013.
13. Carnegie Foundation for the Advancement of Teaching. Carnegie
Foundation for the Advancement of Teaching receives funding to
rethink the Carnegie Unit. http://www.carnegiefoundation.org/
newsroom/press-releases/carnegie-foundation-receives-fundingrethink-the-carnegie-unit. Decmeber 2012. Accessed Nov 4, 2013.
14. Asch DA, Nicholson S, Vujici M. Are we in a medical education
bubble market? N Engl J Med. 2013;369:1973-1975. http://www.
nejm.org/doi/pdf/10.1056/NEJMp1310778. Accessed November 18,
2013.
15. Gleason B, Siracuse M, Moniri N, Birnie C, Okamoto C, Crouch
M. Evolution of preprofessional pharmacy curricula. Am J Pharm
Educ. 2013;77(5):Article 95.
16. Broedel-Zaugg K, Buring SM, Shanker N, et al. Academic
pharmacy administrators perceptions of core requirements for entry
into professional pharmacy programs. Am J Pharm Educ. 2008;72(3):
Article 52.
17. Vedder R, Gillen A, Bennett D, et al. 25 ways to reduce the cost
of college. Center for College Affordability and Productivity; http://
www.centerforcollegeaffordability.org/uploads/25Ways_to_
Reduce_the_Cost_of_College.pdf. Accessed November 18, 2013.
18. Jacob B, McCall B, Stange KM. College as country club: do
colleges cater to students preferences for consumption. The National
Bureau of Economic Research. http://www.nber.org/papers/w18745.
pdf. Accessed November 5, 2013.
19. Jaschik S. The spending side of the equation. Inside Higher Educ.
http://www.insidehighered.com/news/2008/05/01/spending.
20. Davis Educational Foundation. An inquiry into the rising cost of
higher education: Summary of responses from seventy college and
university presidents. http://www.davisfoundations.org/site/
documents/AnInquiryintotheRisingCostofHigherEducation_003.pdf.
21. National Public Radio. Whats driving college costs higher?
http://www.npr.org/2012/06/26/155766786/whats-driving-collegecosts-higher. Accessed November 4, 2013.
22. Winston G. The positional arms race in higher education.
Educause. http://net.educause.edu/ir/library/pdf/ffp0103s.pdf.
23. Fox J, Bartholomae S, Trombitas K. Evaluating nancial
education programs. In: Durband DB, Britt SL, eds. Student
Financial Literacy. Springer, US; 2012:141-166.
24. Chui MA. An elective course in personal nance for health care
professionals. Am J Pharm Educ. 2009;73(1):Article 6.
25. Federal Student Aid, The US Department of Education. Federal
student aid: public service loan forgiveness. http://studentaid.ed.gov/
repay-loans/forgiveness-cancellation/charts/public-service. Accessed
May 5, 2013.
REFERENCES
1. Cain J, Campbell T, Congdon HB, et al. Pharmacy student debt
and return on investment of a pharmacy education: issues for the
academy. Am J Pharm Educ. 2014;78(1):Article 5.
2. Oliff P, Palacios V, Johnson I, Leachman M. Recent deep state
higher education cuts may harm students and the economy for years
to come. http://www.cbpp.org/cms/?fa5view&id53927. Accessed
September 9, 2013.
3. State Higher Education Executive Ofcers. State Higher
Education Finance: FY 2012. http://www.sheeo.org/sites/default/
les/publications/SHEF%20FY%2012-20130322rev.pdf. Accessed
September 10, 2013.
4. Macy A, Terry N. The determinants of student college debt.
Southwest Econ Rev. 2007;34(1):15-25.
5. Nawaguna E. Congress nally votes to cut student loan interest
rates. Reuters. July 31, 2013http://www.reuters.com/article/2013/07/
31/us-usa-studentloans-rates-idUSBRE96U1G220130731. Accessed
November 1, 2013.
6. Johnson J. Obama signs student loan interest rate legislation into
law. The Washington Post. August 9, 2013. http://articles.
washingtonpost.com/2013-08-09/politics/41223761_1_interest-ratesplus-loans-graduate-students. Accessed November 1, 2013.
7. Federal Student Aid, The US Department of Education. Federal
student aid: deferment and forbearance.http://studentaid.ed.gov/
repay-loans/deferment-forbearance. Accessed May 5, 2013.
8. Roberts JA, Jones ELI. Money attitudes, credit card use, and
compulsive buying among American college students. J Consumer
Aff. 2001;35(2):213-240.
9. Sallie Mae. How undergraduate students use credit cards: Sallie
Maes national study of usage rates and trends. http://static.
mgnetwork.com/rtd/pdfs/20090830_iris.pdf. Accessed Oct 4, 2013.
10. Bennett DL. Trends in the higher education labor force:
identifying changes in worker composition and productivity. Center
RESEARCH
A Three-Year Study of the Impact of Instructor Attitude,
Enthusiasm, and Teaching Style on Student Learning in a
Medicinal Chemistry Course
Naser Z. Alsharif, PharmD, PhD, and Yongyue Qi, MS
School of Pharmacy and Health Professions, Creighton University Medical Center, Omaha, Nebraska
Submitted August 9, 2013; accepted January 20, 2014; published September 15, 2014.
Objective. To determine the effect of instructor attitude, enthusiasm, and teaching style on learning for
distance and campus pharmacy students.
Methods. Over a 3-year period, distance and campus students enrolled in the spring semester of a medicinal
chemistry course were asked to complete a survey instrument with questions related to instructor attitude,
enthusiasm, and teaching style, as well as items to measure student intrinsic motivation and vitality.
Results. More positive responses were observed among distance students and older students. Gender did
not impact student perspectives on 25 of the 26 survey questions. Student-related items were signicantly
correlated with instructor-related items. Also, student-related items and second-year cumulative grade point
average were predictive of students nal course grades. Instructor enthusiasm demonstrated the highest
correlation with student intrinsic motivation and vitality.
Conclusion. While this study addresses the importance of content mastery and instructional methodologies, it focuses on issues related to instructor attitude, instructor enthusiasm, and teaching style,
which all play a critical role in the learning process. Thus, instructors have a responsibility to
evaluate, reevaluate, and analyze the above factors to address any related issues that impact the
learning process, including their inuence on professional students intrinsic motivation and vitality,
and ability to meet educational outcomes.
Keywords: instructor enthusiasm, science courses, medicinal chemistry, student learning, intrinsic motivation,
vitality, teaching
Key to this relationship is having instructors who can

unlock the dormant energy inside their students and
instill vitality in how students pursue the learning process.9-13 The instructor has to use strategies in the classroom that are student-centered and challenging, and that
put the responsibility on the student as much as on the
instructor to enhance student engagement in learning.14-16
The Center for the Advancement of Pharmacy Education
(CAPE) Educational Outcomes 2013 called for the inclusion of an affective domain that would address personal
and professional skills, attitudes, and attributes required
for the delivery of patient-centered care.17
By assessing student intrinsic motivation and vitality, both of which can impact student affective domain,
this study conducted at the School of Pharmacy and
Health Professions, Creighton University, analyzed the
implications of instructor-related factors such as attitude, enthusiasm, and teaching style on student learning.
The study addresses this topic from the perspective of 2
student cohorts (campus and distance pathway students)
enrolled in a required Chemical Basis of Drug Action
INTRODUCTION
The art of teaching is a continuous journey to find the
right combination of pedagogy, instructional methodologies, and more recently, technology to help students
learn.1-6 Student evaluations have always been an important
tool to gauge how students perceive the classroom environment and the learning process.1-6 Over the years, one theme
that seems to be prevalent in student evaluations is how
instructors demeanor, behavior, and/or attitude in the classroom affect student learning.5,7 This focus is in contrast to
pharmacy educators sometimes elaborate emphasis on innovative learning theories, lecture notes, and classroom activities to bring clinical relevance to course content.
A relationship has been identified between instructor
enthusiasm and students intrinsic motivation to learn.8-12
Corresponding Author: Naser Z. Alsharif, PharmD, PhD,
School of Pharmacy and Health Professions, Department of
Pharmacy Sciences, Creighton University Medical Center,
2500 California Plaza, Omaha, NE 68178. Tel: 402-280-1857.
Fax: 402-280-1883. E-mail: nalshari@creighton.edu.

course in the second year of the doctor of pharmacy
(PharmD) curriculum.
less than 0.05 was considered signicant. An exempt status

for the study was obtained from the Creighton University
Institutional Review Board.
METHODS
RESULTS
A convenience sample that included distance

(n5187) and campus (n5285) pathway students at
Creighton University was used for the study.18 Both cohorts were registered for the required Chemical Basis of
Drug Action course in the spring semester of 2011, 2012, or
2013. This 2-credit hour course, taught in the second year
of the curriculum, is delivered concurrently with pharmacology and follows completion of course work in biochemistry, physiology, pathology, anatomy, pharmaceutics, and
communication skills. The campus students were required
to attend class. The distance students, who were located
throughout the country, followed the same course syllabus,
learning objectives, lesson outline, and course activities as
did the campus students.18 Distance students viewed
videos of classroom lectures that were made available to
them within 2 hours after each class. The course instructor
was a tenured professor who had taught the course for 20
years.
A department instructor evaluation tool that had been
used for more than 15 years was administered to the students at the end of the spring semester in each of the years
of the study. The survey instrument also included studyspecific instructor-related items that addressed the course
instructors attitude, enthusiasm, and teaching style (ie,
items that were not related to instructional techniques).
The survey items were adopted from several studies that
addressed instructor enthusiasm, student intrinsic motivation, and student vitality.10,13,19 Although most of the
items were rated using a 5-point Likert scale, students also
were asked to provide written responses. Both the quantitative and qualitative sets of data were analyzed.
Statistical analysis included descriptive statistics and
independent t test for comparing mean evaluation scores
between pathways, age groups, and genders. Mean evaluation scores also were compared between students whose
grade point average (GPA) ranked in the upper 40% of
their class (approximately 70% of the students, Group 1)
and those whose GPA ranked in the lower 60% (Group 2),
as well as by students rst-year cumulative GPA, secondyear cumulative GPA, and rst- and second-year cumulative GPA. Pearson correlation analysis was conducted to
address the association between instructor and studentrelated items. In the multiple regression model, the course
score was treated as a response variable while rst- and
second-year cumulative GPA, age, student-related items,
pathway, and gender were treated as predictor variables.
All statistical analyses were conducted using SAS, version
9.3 (SAS Institute Inc, Gary, North Carolina). A p value
The study involved 187 (39.6%) distance students

and 285 (60.4%) campus students who were admitted to
the PharmD program at Creighton University in 2009,
2010, and 2011. The average age for distance students
was 33.1 years, and for campus students, 24.8 years (Table 1). The response rate was approximately 100% because it was incorporated into the required end-of-class
instructor evaluation. There were 180 (38.1%) male students and 292 (61.9%) female students. Distance students average prerequisite GPA (Pre-GPA), rst-year
cumulative GPA, and second-year cumulative GPA were
3.5, 3.6, and 3.3, respectively, while these variables for
campus students were 3.4, 3.4, and 3.2, respectively.
Almost all campus (93%) and distance students
(84%) strongly agreed or agreed that the instructor related
to them and other students in a manner that promoted
mutual respect. In addition, 96% of the distance students
and 81% of the campus students indicated that they
strongly agreed or agreed that the instructor demonstrated
interest in their success. Further, 94% of the distance
students and 87% of the campus students indicated that
they strongly agreed or agreed that the instructor demonstrated professionalism in interaction with them or other
students (Table 2). An independent t test for comparing
mean evaluation scores showed a signicant difference
between the distance students and the campus students
(p,0.001) for all the items related to instructor attitude
(Table 2), with signicantly higher ratings given by distance students. The majority of the distance students were
older than 27 years (average age 33.1 years, Table 1), and
the majority of the campus students were 27 years of age
or younger (average age 24.8 years, Table 1). Overall, age
was a signicant factor (p,0.001) in all items related to
instructor attitude (Table 3).
There was a difference in the ratio of male to female
students between the distance and campus students in the
admitted classes of 2009, 2010, and 2011 (Table 1), with
a higher ratio of female to male students in the distance
classes. The t test analysis for gender did not demonstrate
any signicant difference in student responses for any of
the instructor-related items except for question 19 (provides different ways to learn the content, p,0.05), or for
the student-related items.
Instructor enthusiasm was measured based on responses to general questions and to questions related
to verbal and nonverbal behaviors (Tables 2 and 3). With
regard to the instructor enthusiasm general question
8

Table 1. Student Demographics
Demographics
Age
Campus
Distance
Gender
Campus
Male, %
Female, %
Distance
Male, %
Female, %
Pre-GPA
Campus
Distance
Pa
CGPA1
Campus
Distance
Pa
CGPA2
Campus
Distance
Pa
CGPA12
Campus
Distance
Pa
2009
Admitted Year
2011
2010
Three Year Average
25.6 (N5107)
32.8 (N566)
24.9 (N577)
33.1 (N555)
23.9 (N5101)
33.5 (N566)
24.8 (N5285)
33.1 (N5187)
44.9
55.1
41.6
58.4
47.5
55.5
44.9
55.1
19.7
80.3
36.4
63.6
28.8
71.2
27.8
72.2
3.4
3.5
0.004
3.4
3.5
0.006
3.4
3.5
0.13
3.4
3.5
0.001
3.5
3.7
,0.001
3.3
3.5
0.002
3.4
3.6
0.005
3.1
3.4
0.002
3.1
3.1
0.96
3.3
3.3
0.66
3.12
3.3
0.048
3.3
3.5
,0.001
3.2
3.3
0.16
3.4
3.5
0.06
3.3
3.4
,0.001
3.4
3.6
,0.001
Abbreviations: Pre-GPA5prerequisite grade point average; CGPA5rst-year cumulative grade point average; CGPA25second-year cumulative
grade point average; CGPA125rst- and second-year cumulative grade point average.
a
p value as determined by independent t test.
items, the distance students (96%) and campus students

(89%) strongly agreed or agreed with the statement that
the instructor demonstrated a passionate interest in the
topic. Also, 97% of the distance students and 87% of
the campus students strongly agreed or agreed with the
statement that the instructor demonstrated mastery of the
topic. When asked about their agreement with the statement that the instructor was full of energy when teaching, 99% of the distance students and 94% of the campus
students strongly agreed or agreed with it. Further, the
distance students (80% and 77%, respectively) and the
campus students (58% and 50%, respectively) strongly
agreed or agreed with the statements that the instructor
enhanced my motivation for the pharmacy program and
that the instructor enhanced my motivation for medicinal
chemistry (Table 2). Again, independent t test for comparing mean evaluation scores showed signicantly
higher scores among distance students than among campus students (p,0.001) for all items related to instructor
enthusiasm (Table 2).
Distance students responded more positively to

the instructor enthusiasm questions based on some of
the instructors verbal and nonverbal signs such as vocal
delivery, vocal volume, apparent emotion and level of
energy. These responses were signicantly different
from those of campus students (p,0.001) (Table 2).
Campus students perception about hand and body gesture inuence on learning was slightly higher than that of
distance students. However, these were not signicantly
different (Table 2).
Table 3 shows that age was a signicant factor (p,0.01)
in all items related to instructor enthusiasm (Table 3). However, age was not a factor in 4 of 7 verbal and nonverbal items
related to instructor enthusiasm (Table 3).
In general, distance student perceptions were significantly higher (p,0.001) compared to those of campus students in their agreement with statements related to the
instructor expecting student participation, having appropriate expectations, providing relevance for the information
presented, showing genuine concern for (their) learning,
9

Table 2. Percent Response Frequencies, Means, and p values of Independent t Test Based on Pathway
Survey Items
Instructor Related Items
Instructor Attitude
1. The instructor related to me and/or other students
in a manner that promoted mutual respect.
2. The instructor demonstrated interest in my success.
3. The instructor demonstrated professionalism
in interactions with me and/or other students.
Instructor Enthusiasm (General Items)
4. Demonstrates a passionate interest in his topic.
5. Demonstrates mastery of his topic.
6. Is full of energy when teaching.
7. Enhanced my motivation for the pharmacy program.
8. Enhanced my motivation for medicinal chemistry.
Instructor Enthusiasm (verbal and non-verbal signs)
9. Vocal delivery.
10. Vocal volume.
11. Facial expressions.
12. Apparent emotion.
13. Hands and other gestures.
14. Observed body gestures.
15. High level of energy.
Instructor Teaching Style
16. Expects student participation.
17. Has appropriate expectations.
18. Provides relevance for the information presented.
19. Shows genuine concern for my learning.
20. Provides different ways to learn the content.
Student Related Items
Student Intrinsic Motivation
21. I am intrinsically motivated to learn.
Pathway (N)
Response, %a
SA A N D SD
Score,
Mean
Campus
Distance
Campus
Distance
Campus
Distance
(N5283)
(N5187)
(N5282)
(N5187)
(N5282)
(N5187)
33
61
40
69
36
63
51
32
41
27
51
31
9
6
12
3
8
5
6
1
6
1
5
1
1
1
1
1
1
1
4.1
4.5
4.1
4.6
4.2
4.6
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
(N5280)
(N5185)
(N5279)
(N5186)
(N5278)
(N5183)
(N5279)
(N5185)
(N5278)
(N5186)
54
77
48
70
63
76
18
39
16
39
35
19
39
27
31
23
40
41
34
38
8
4
11
3
6
1
25
17
27
19
2
0
1
0
0
0
12
4
14
4
1
0
0
0
0
0
5
0
9
1
4.4
4.7
4.3
4.7
4.6
4.7
3.6
4.1
3.3
4.1
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
(N5278)
(N5186)
(N5279)
(N5187)
(N5281)
(N5187)
(N5279)
(N5185)
(N5277)
(N5186)
(N5274)
(N5185)
(N5278)
(N5187)
30
47
32
48
30
33
31
49
29
32
29
30
42
61
44
39
40
37
45
22
43
35
42
24
44
25
40
34
13
9
11
10
18
44
16
13
17
38
19
44
14
5
10
3
12
2
5
1
7
3
8
5
7
1
4
0
3
1
4
1
3
1
2
1
3
1
2
1
1
1
3.9
4.3
3.8
4.3
3.9
3.9
3.9
4.3
3.9
3.8
3.9
3.8
4.1
4.5
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
(N5277)
(N5187)
(N5278)
(N5186)
(N5279)
(N5185)
(N5279)
(N5185)
(N5279)
(N5185)
75
86
26
53
31
61
39
65
26
45
23
13
40
33
56
35
46
33
44
44
2
1
15
10
10
3
10
3
19
10
0
0
14
3
2
1
4
0
8
2
0
0
5
1
0
1
1
0
3
0
4.7
4.9
3.7
4.3
4.2
4.6
4.2
4.6
3.8
4.3
Campus (N5282)
Distance (N5184)
44
62
46
33
9
6
1
0
0
0
4.3
4.7
,0.001
,0.001
,0.001
,0.001
,0.001
,0.001
,0.001
,0.001
,0.001
,0.001
0.39
,0.001
0.60
0.52
,0.001
,0.001
,0.001
,0.001
,0.001
,0.001
,0.001
(Continued)
10

Table 2. (Continued )
Survey Items
22. I am highly motivated about the pharmacy program.
23. I am highly motivated about the Chem.
Basis of Drug Action course.
24. I have greater motivation for clinical courses
compared to science courses.
25. I would like to learn more about the
content from this course.
Student Vitality
26. When I am in this class, I feel alive and vital.
a
Pathway (N)
Campus
Distance
Campus
Distance
Campus
Distance
Campus
Distance
Response, %a
SA A N D SD
Score,
Mean
(N5280)
(N5187)
(N5279)
(N5187)
(N5280)
(N5185)
(N5280)
(N5186)
51
67
19
33
37
30
12
21
43
30
43
43
42
28
32
41
4
3
23
18
16
29
32
31
2
0
8
4
4
13
15
4
0
0
7
1
1
2
9
2
4.4
4.6
3.6
4.0
4.1
3.7
3.2
3.8
Campus (N5281)
Distance (N5186)
12
18
24
38
33
34
20
7
12
3
3.0
3.6
P
,0.001
,0.001
,0.001
,0.001
,0.001
Scale responses include: Strongly Disagree (SD)51, Disagree (D)52; Neutral (N)53; Agree (A)54; Strongly Agree (SA)55.
and providing different ways to learn content (Table 3).

Table 3 also shows that age was a signicant factor in all
items related to instructor teaching style.
Almost all distance (95%) and campus students
(90%) strongly agreed or agreed that they were intrinsically motivated to learn. When asked if they were highly
motivated about the pharmacy program, 97% of the distance students and 94% of the campus students strongly
agreed or agreed. However, the percentages who strongly
agreed or agreed were lower for distance and campus
students when asked if they were highly motivated about
the Chemical Basis of Drug Action course (76% and 58%,
respectively) and whether they had greater motivation for
clinical courses compared to science courses (62% and
79%, respectively). Another finding was that only 62% of
the distance students and 44% of the campus students
indicated that they strongly agreed or agreed that they
would like to learn more course content (Table 2). All
the comparisons between distance and campus students
responses for all of the above items (Table 2) were signicant (p,0.001). Table 3 shows that age was a signicant factor (p values ranged from 0.022 to less than 0.001)
in all items related to student intrinsic motivation.
When asked about their agreement with the statement,
when I am in this class, I feel alive and vital, 56% of the
distance students and 36% of the campus students strongly
agreed or agreed with the statement (p,0.001) (Table 2).
Age was a signicant factor (p,0.001) in the item related
to student vitality (Table 3).
Students whose first-year cumulative GPA ranked in
the upper 40% of their class GPA had significantly higher
evaluation scores for instructor enthusiasm, teaching style,
and student intrinsic motivation than those whose GPA
ranked in the lower 60% (p,0.001) (Table 4). A signicant
difference was also seen for the second-year cumulative
GPA with teaching style and student intrinsic motivation.

When looking at cumulative GPA for the rst 2 years of
pharmacy school, students who ranked in the upper 40%
had signicantly higher evaluation scores for instructor
attitude, enthusiasm, teaching style, and student intrinsic
motivation items than those whose GPAs ranked in the
lower 60% (p,0.001) (Table 4).
Multiple regression analysis between selected predictors such as first-year cumulative GPA, second-year
cumulative GPA, age, student related items, female gender, or campus pathway and student course scores
revealed the significant factors of second-year cumulative
GPA and student-related item scores in predicting student
final course scores. The 2 factors were positively associated with course scores. The model p value was ,0.001
and the R2 (coefcient of determination) was 0.32.
DISCUSSION
In this study, we attempted to identify the impact of
instructor-related factors, including attitude, enthusiasm,
and teaching style, on both campus and distance students
intrinsic motivation and vitality. Our objective was to improve on the learning experience for all students enrolled in
the Chemical Basis of Drug Action course. Prior to this
study, the percentage of students choosing strongly agree
or agree in response to the 3 statements addressing instructor attitude (Table 2) was in the mid to upper 70% range
(data not shown). These percentages have steadily increased, especially over the last 3 years during which time
the percentage of students who strongly agreed or agreed
improved by as much as 10%. Distance students have always scored these items higher than campus students have
and this may reect the differences in the dynamics of the
classroom that each group experiences. In the traditional
classroom, campus students are challenged to be interactive
11

Table 3. Percent Response Frequencies, Means and p values of Independent t Test Based on Age
Survey Items
Instructor-Related Items
Instructor Attitude
1. The speaker related to me and/or other students
in a manner that promoted mutual respect.
2. The speaker demonstrated interest in my success.
3. The speaker demonstrated professionalism
in interactions with me and/or other students.
Instructor Enthusiasm (General Items)
4. Demonstrates a passionate interest in his topic.
5. Demonstrates mastery of his topic.
6. Is full of energy when teaching.
7. Enhanced my motivation for the pharmacy program.
8. Enhanced my motivation for medicinal chemistry.
Instructor Enthusiasm (verbal and non-verbal Signs)
9. Vocal delivery.
10. Vocal volume.
11. Facial expressions.
12. Apparent emotion.
13. Hands and other gestures.
14. Observed body gestures.
15. High level of energy.
Instructor Teaching Style
16. Expects student participation.
17. Has appropriate expectations.
18. Provides relevance for the information presented.
19. Shows genuine concern for my learning.
20. Provides different ways to learn the content.
Student-Related Items
Student Intrinsic Motivation
21. I am intrinsically motivated to learn.
Age (N)
SAa
Percentage
A
N
D
SD
5-point
Scale
Mean
#27
.27
#27
.27
#27
.27
(N5259)
(N5211)
(N5258)
(N5211)
(N5259)
(N5210)
37
56
43
66
38
60
48
34
41
27
49
32
9
8
11
5
9
5
5
2
4
2
3
2
1
0
1
0
1
0
4.2
4.4
4.2
4.6
4.2
4.5
,0.001
#27
.27
#27
.27
#27
.27
#27
.27
#27
.27
(N5258)
(N5207)
(N5256)
(N5209)
(N5255)
(N5206)
(N5255)
(N5209)
(N5255)
(N5209)
57
73
52
65
64
75
23
32
22
29
32
23
35
30
31
22
40
44
34
41
8
4
11
5
4
2
24
18
25
21
3
0
1
0
0
0
10
6
13
7
1
0
0
0
0
0
3
1
6
2
4.4
4.7
4.4
4.6
4.6
4.7
3.7
4.0
3.5
3.9
,0.001
#27
.27
#27
.27
#27
.27
#27
.27
#27
.27
#27
.27
#27
.27
(N5256)
(N5208)
(N5256)
(N5210)
(N5258)
(N5210)
(N5255)
(N5209)
(N5256)
(N5207)
(N5252)
(N5207)
(N5257)
(N5208)
34
42
36
44
31
31
35
44
33
29
32
28
47
55
43
38
39
38
41
27
41
37
38
29
40
29
34
36
13
12
11
12
22
38
16
14
20
34
21
41
14
7
8
6
11
5
3
2
6
4
7
5
5
2
3
1
3
1
4
1
2
1
2
1
3
2
2
1
1
1
4.0
4.1
3.9
4.2
4.0
3.8
4.0
4.2
3.9
3.8
3.9
3.8
4.2
4.4
0.07
#27
.27
#27
.27
#27
.27
#27
.27
#27
.27
(N5254)
(N5210)
(N5256)
(N5208)
(N5258)
(N5206)
(N5256)
(N5208)
(N5257)
(N5207)
75
85
30
47
36
54
43
61
30
39
22
15
39
37
51
39
43
34
42
44
3
0
15
11
11
3
10
4
17
12
0
0
13
5
2
2
2
1
8
3
0
0
4
1
0
0
1
0
2
1
4.7
4.9
3.8
4.2
4.2
4.5
4.3
4.5
3.9
4.2
0.001
#27 (N5259)
.27 (N5207)
44
57
47
38
7
5
1
0
0
0
4.3
4.5
,0.001
,0.001
0.001
0.006
0.001
,0.001
0.005
0.20
0.03
0.16
0.14
0.01
,0.001
,0.001
,0.001
0.003
0.002
(Continued)
12

Table 3. (Continued )
Survey Items
Age (N)
#27
.27
#27
.27
#27
.27
#27
.27
22. I am highly motivated about the

pharmacy program.
23. I am highly motivated about the Chem.
Basis of Drug Action course.
24. I have greater motivation for clinical courses
compared to science courses.
25. I would like to learn more about the
content from this course.
Student Vitality
26. When I am in this class, I feel alive and vital.
a
SAa
Percentage
A
N
D
SD
5-point
Scale
Mean
(N5258)
(N5209)
(N5257)
(N5209)
(N5257)
(N5208)
(N5257)
(N5209)
52
61
21
27
33
30
13
16
41
36
42
43
46
28
37
38
5
3
22
22
18
29
31
36
2
0
8
5
3
12
11
8
0
0
7
3
1
1
8
2
4.4
4.6
3.6
3.8
4.1
3.7
3.4
3.6
#27 (N5258)
.27 (N5209)
12
14
27
34
32
39
19
10
9
3
3.1
3.5
P
0.02
0.02
,0.001
0.02
0.001
Scale responses include: Strongly Disagree (SD), Disagree (D); Neutral (N); Agree (A); Strongly Agree (SA)55.
and engaged in the class. However, it is difcult to get all of

the students to appreciate this, and some are instead intimidated and negatively interpret this continuous challenge for
them to be an active participant in the classroom. As one
student expressed, I have a difcult time in being put on the
spot when being asked questions, so that teaching style is
difcult to me.
Distance students on the other hand, do not have that
same pressure and in many cases they have shared on the
end of class evaluations how they are excited about answering questions posed in the classroom as they watch
the video. Also, distance students are older than campus
students by an average age of 8.3 years (Table 1). In
addition, our results may represent the difference between
generations where the majority of campus students are
mostly considered generation Y (1980-1994), who are
more comfortable with e-mail and text communication

than face-to-face communication20-21 while the distance
students are mostly generation X (1965-1979) who can be
ercely independent, self-directed learners who enjoy
question-and-answer sessions.20,21 Another factor that
could have impacted the results is student bias related to
age, with younger students preferring younger professors
and older students preferring older professors.22 Overall,
our data support this because age was clearly a factor that
positively impacted how students perceived instructor attitude (Table 3).
Over the years in which the Chemical Basis of Drug
Action course has been taught, the instructor identified
and adopted several key behaviors to help students recognize that he was a partner in their learning process and
interested in their success, including: (1) mastering the
Table 4. Response Comparisons Between Groups Who Ranked Upper 40% (Group 1) and Lower 60% of GPAs (Group 2)
Survey Questions
Faculty-related items
Faculty/instructor attitude
Faculty/instructor enthusiasm
Faculty/instructor enthusiasm
(verbal and non-verbal signs)
Faculty/instructor teaching
Student-related items
Student intrinsic motivation
Student vitality
Pre-GPA
(Group 1 mean, Group 2 mean, p value for t Test)

b
c
CGPA1
CGPA2
CGPA12
(13.0, 13.0, 0.82)

(21.2, 21.1, 0.85)
(28.2, 28.0, 0.64)
(12.8, 13.2, 0.09)

(20.9, 21.6, 0.03)
(28.1, 28.2, 0.80)
(12.9,13.1, 0.20)
(20.9, 21.4, 0.12)
(28.1, 28.1, 0.97)
(12.8, 13.3, 0.02)

(20.8, 21.7, 0.01)
(28.1, 28.2, 0.77)
(21.5, 21.4, 0.97)
(21.2, 21.9, 0.02)
(21.2, 21.8, 0.05)
(21.1, 21.9, 0.01)
(20.0, 19.9, 0.70)

(3.3, 3.2, 0.20)
(19.7, 20.4, 0.01)

(3.2, 3.3, 0.36)
(19.6, 20.5, ,0.001)

(3.2, 3.4, 0.36)
(19.6, 20.5, ,0.001)

(3.2, 3.4, 0.23)
Group 1: Students who ranked upper 40% of their GPAs; Group 2: Students who ranked lower 60% of their GPAs.
a
Pre-GPA: prerequisite grade point average.
b
CGPA1: rst-year cumulative grade point average.
c
CGPA2: Second-year cumulative grade point average.
d
CGPA12: rst and second year cumulative grade point average.
13

course content; (2) developing different strategies to deliver the content; (3) stating expectations clearly at the
beginning of the semester; (4) being a role model in interaction with students; (5) being present in the learning
environment whether it is the classroom, online, in the
office, or as it happens, in the hallway; (6) ensuring
prompt communications with students as logistical or academic issues arise; and (7) responding to concerns by
students. These behaviors matched well with some of the
principles for good teaching articulated by other authors.23,24 However, as shown by the study data, implementing all of the above does not ensure a successful
classroom environment and learning experience. Clearly
students perceptions of the overall process are important.
Thus, this 3-year study, as well as an ongoing evaluation
process, end-of-course evaluations, and input from class
ofcers during the semester are strategies that have been
used and will continue to be developed and enhanced to
address concerns by both student cohorts regarding these
issues outside of the traditional aspects of teaching. The
goal is to improve the relationship between the instructor
and the students and to enhance the interactive classroom
environment.
As a result of some of the concerns identified by students related to instructor attitude, several strategies have
been implemented to address this. The first strategy was
taking more time at the beginning of each class to review
key concepts. The second strategy was implementing an
instructional model5,25 to help transition the students to
a higher level of thinking and interactivity in the class
and to decrease the feeling of intimidation from the demands of the in-class time and the perceived energy of
the instructor. The third strategy was exhibiting more patience with students and accuracy in gauging when the instructor needs to answer his own questions if there is no
response by the students. In support of this, Monteiro and
associates demonstrated that the patience, availability, and
openness of the instructor had a positive correlation with
student academic engagement. 26 An instructor answering
his or her own questions also ensures that there will not be
too much idle time spent waiting for an answer. The idle
time resulting from waiting for an answer from the campus
students was actually a concern that was shared by some
students from both cohorts. Also, both student cohorts and
more especially students who were struggling were encouraged to feel comfortable to approach the instructor for help
and to take advantage of an open door policy or to make an
appointment for a phone call (distance students). The
fourth strategy was identifying any signs of frustration or
perceived negative language (eg, statements such as Are
you with me? Does this make sense? I hope you recognize this is not rocket science) for the students lack of
participation or perceived motivation. This is important so

that students are not deterred by such statements or perceive them as unprofessional behavior. This is critical as
instructors perceived misbehavior, including being offensive (eg, verbally abuse, humiliate, embarrass, or insult
students) is viewed negatively by students.27 The latter is
also important because low motivation among students has
been associated with teacher discouragement.28 A fth
strategy was recognizing the importance of not exhibiting
any reactions that may be perceived as disappointment or
ridicule when a student answers a question incorrectly, and
taking time to openly and enthusiastically recognize students when they answer questions correctly or when they
demonstrate the ability to connect information and concepts. Recognition is greatly valued by students, especially
high achievers. It is one of the criteria they look for in an
effective instructor and it serves as a positive reinforcement
and motivator for them.26
Gender was not a factor in how students responded
on instructor-related items, including instructor attitude,
or student-related items. While some studies22, 28,29 identied gender bias in how students evaluate instructors,
other studies did not.30-32 Also some studies found that
women are more self-determined in the learning process.33,34 Although our gender data did not show such
positive ndings, it is critical to continue to evaluate gender-related factors and incorporate teaching and learning
strategies that have been shown to be effective for both
genders, such as use of gender-inclusive language.
The majority of both student cohorts responded positively regarding the general question items related to instructor enthusiasm (Table 2). The written responses
showed similar agreement, with the theme of enthusiasm
prevalent in many of the students comments. Thus, the
data (Table 2) clearly demonstrate that for the preponderance of students, the apparent enthusiastic attitude of the
instructor was a positive factor in their learning. A major
aspect of that is the perception by students that the instructor had mastery of the course subject matter. Mastery of the
subject in combination with mastery of teaching methodologies are perceived very positively by students and considered as characteristic of the best teachers.23,35,36
In regard to the questions related to instructor enthusiasm based on the verbal and nonverbal behavior of the
instructor (Table 2), the difference in the signicantly
higher response by the distance students to how vocal
delivery, apparent emotion, and level of energy inuenced their learning, and how the campus students perceived the hand and body gestures slightly more
favorably, is likely related to the classroom environment.
Campus students may have been more distracted by the
instructors high volume than by his hand and body ges14

of deeper approaches to learning,23,24,41-43 the instructor
should challenge students to a higher level of thinking
but with efforts also to adapt the learning process to the
needs of all students including students who are performance oriented. This can be accomplished by providing
more structured presentations, taking more time to explain concepts, and offering tips for success. 23,44,45
Overall, the majority of both student cohorts and age
groups (Tables 2 and 3, respectively) recognized the efforts done to explain the relevance of the information
presented. In fact, Ismail and Hayes showed that course
topics that are difcult to make relevant or fun can negatively inuence motivation.46 Some of the students mentioned including test questions based on patient cases on
examinations and relating chemistry to clinical practice in
lectures as helpful ways in which the instructor showed
the clinical relevance of the course.
As stated above, the combination of mastery of subject and mastery of instructional methodologies are identified as key characteristics of the best instructors.36,41
Thus, over the years, several mechanisms were developed
to help students explore new and different ways to learn
the content and the effectiveness of these different
mechanisms continues to be evaluated.1-6 The importance of having different ways to learn course content
is supported by the literature36,41 and by students written comments, in which they mentioned the various
teaching tools used such as interactive PowerPoint
slides, Softchalk lessons, and short video reviews prepared by the instructor.
While over 90% of both the distance and campus
students responded that they were intrinsically motivated
to learn and were highly motivated about the pharmacy
program, there was much less motivation for taking the
Chemical Basis of Drug Action course and learning its
content, especially among campus students (Table 2),
75% of whom had greater motivation for clinical courses.
Similar data were observed based on age (Table 3). The
lack of student motivation is always an issue with science
courses taught in a health sciences professional program,
especially when instructors do not try to make their courses
relevant for their students.47,48 Instructor enthusiasm has
been identied as an external catalyst for the intrinsic motivational energy that may be lying dormant within students.9-12 It is even more critical when teaching college
students as many of them are used to a system of external
incentives (eg, grades). The combination of a positive student perception of instructor attitude, enthusiasm, and
teaching style appears to make a difference in students
overall perception of their learning experience in the
course, as is demonstrated by the high correlation of the
student-related items with the instructor-related items (Fig-
tures, which may have been less intimidating or more

subtle. A few distance students commented that the instructor clapping his hands to emphasize certain concepts
seemed disruptive to them, but the instructors vocal delivery did not appear to bother them. Thus, student input
related to the above issues was instrumental in recognizing that, for some students, verbal or nonverbal demonstrations of enthusiasm may be detrimental to the learning
process. As a result, it is important for instructors to monitor their volume and not to sound or appear as if screaming into the microphone or being aggressive and to control
any perceived negative emotions that may be distracting
or alienating to students.27 However, as suggested by student responses (Tables 2 and 3) and studies in the eld of
communication, immediate behavior identied by vocal
expressions, communicating at a close distance, smiling,
engaging in eye contact, and exhibiting body gestures is
associated with reducing physical and/or psychological
distance between instructor and student and have a positive effect on learning.37,38 On the other hand, as discussed above, instructor verbal aggressiveness has been
viewed very negatively by the students.27,39 Thus, gauging the students is important to ensure that balance exists
and that behavior and emotions are not perceived in a negative way. One aspect that has been identied in the literature to be helpful also as part of immediate behavior is
humor.37,38 Inserting more humor in the handouts (eg,
cartoons) and in vocal delivery may also contribute to
a more positive experience.37,38
The instructor setting high but appropriate educational expectations, setting them early, stressing such expectations continuously, reevaluating them based on
student input, and helping students transition to meet
those expectations is critical. Although the majority of
students felt the instructors expectations for students
were appropriate, a signicant percentage of campus students disagreed or were neutral. This nding may reect
the feelings of students who are not doing as well in the
course as they may be concentrating on their grade for the
course rather than on meeting the challenge of the instructor, which is to have a deeper understanding of the content. Senko and associates have shown that students who
pursue mastery goals favor instructors who stimulate and
challenge them intellectually, while those who pursue
performance goals favor instructors who present the material clearly and provide clear cues about how to succeed. 40
Also, challenging assessments are favored by students
who are high achievers26 which may explain why the top
40% of students in this study had more favorable responses
to all instructor-related items than did other students. Based
on the literature that emphasizes the role of the instructor to
promote student intrinsic motivation and the development
15

ure 1). However, work still needs to be done to improve
overall student interest in medicinal chemistry as a discipline and its importance in pharmacy students overall understanding of drug action.
Science instructors in professional health sciences
degree programs may need to address students lack of
interest in basic science courses more, not only in the
classroom but possibly during student orientation, by giving, for example, seminars on opportunities for graduate
education and by stressing the value of obtaining both
a PharmD and a PhD degree. It is also important for instructors to work and communicate with other science and
clinical instructors to explain the relevance of medicinal
chemistry by incorporating aspects of drug structure and
the science behind it into their teaching of drug action and
drug clinical use. However, while establishing the clinical
relevance for learning medicinal chemistry is important,
science instructors should not dilute their discipline.
Medicinal chemists must not take the chemistry out of
medicinal chemistry. Doing so to placate a minority of
students would be an injustice to the majority of students
who enjoy chemistry and appreciate its role in pharmacy
a view that has been expressed by many students. Also,
student comments related to the lack of relevance of the
course or that medicinal chemistry should not be in the
curriculum have decreased dramatically and more positive comments related to the importance of the knowledge
gained in this course have increased.5,13 This is more
evident in the last 2 years with a purposeful attempt by
both the medicinal chemistry course instructor and the
pharmacology instructors to synchronize the content of
the 2 courses. This is also supported by the association
between student intrinsic motivation and course score and
is reinforced by the multiple regression analysis, which as
discussed above, showed that the second-year cumulative
GPA and student-related item scores predicted student

nal course scores. This nding will certainly be shared
with students in future classes to encourage them to be
more open minded about the course, their experience in
the course, and their overall responsibility for their
learning.
Intrinsically motivated behavior is performed simply
for the pleasure inherent in the activity itself,49 occurs in
the absence of rewards or reinforcements,50 and is characterized by the experience of interest, enjoyment, and
curiosity.11 Intrinsic motivation is empirically linked to
achievement test scores and report card grades51 and positive emotions in school.10,52 The literature identies supporting self-determination/autonomy49,50 and promoting
perceived competence49,50 as 2 critical components in
fostering intrinsically motivated behavior in students. In
contrast, research describes individuals exhibiting amotivation as not being able to complete or value an activity, having no sense of purpose, or exhibiting feelings of
incompetence or learned helplessness.53 The literature
further identies providing autonomy to the students in
respect to choices and decisions about their study with
high levels of intrinsic motivation.49,50 A summary of
more specic strategies to support the above ndings
from the literature are also included in Appendix 1 and
will be continuously evaluated.
The low percentages of students who strongly agreed
or agreed with the student-related items (Tables 2 and 3,
question 22) clearly indicate that much effort is needed to
help both student cohorts become more energetic about
being in the course and studying medicinal chemistry.
However, although vitality is linked to intrinsic motivation, enthusiasm is identied as a behavioral manifestation of intrinsic motivation while vitality is a subjective
one,11 and many other factors may affect it; thus, it is
Figure 1. Schematic illustration of study ndings. Values represent Pearsons r correlation coefcients (*p,0.001) between
faculty and student related items. Pre-GPA (Pre-requisite Grade Point Average), CGPA1 and CGPA2 (Cumulative Grade Point
Average academic year 1 and academic year 2, respectively). Gender did not show correlation to any of the variables.
16

much more difcult to inuence. For example, some
students may have fear of chemistry, which they may
carry with them into the professional program. However, depending on the extent of such an attitude, it is
very difcult to rid students of it, and this attitude will
reect on their level of energy and enthusiasm in the
classroom. Other factors that could affect students vitality could be workload in the respective semester and
personal issues. While assessing congruence between
credit hours per semester and the amount of work required for each course based on its allocated credit hours
and addressing any personal issues that arise is important, these efforts may be delayed and may not completely address the impact of workload and personal
issues on students vitality. Efforts to coordinate a master
examination schedule, limit the number of examinations
within a week, establish policies that are sensitive to
legitimate personal student issues, and to evaluate
courses after they are offered and make recommendations to the curriculum committee to improve on for the
next offering can be helpful. Also, this emphasizes the
importance of the instructor-related items since all have
been shown to positively enhance student intrinsic motivation and student vitality (Figure 1).
While our study addressed many factors that affect the
learning process for students, many other factors may also
be in play that have not been directly addressed in this
study. These include the ethnic background of the students
and instructor, cultural values and beliefs of the students
and the instructor, learning style of the students, and the
difficulty of the course. Also, this study did not show
a causal relationship between some of the variables but
rather a correlational one (Figure 1). In addition, the study
depended on self-reporting by the students. Further, we did
not attempt to measure students motivation and vitality at
the beginning of the course or to control for any of the
instructor-related factors tested in this study. Finally, our
data were obtained from college students enrolled in a medicinal chemistry course in a private professional school in
the United States and are specic to one course instructor;
therefore, some ndings may not be transferable to other
instructors. Our results do not prove, for example, that an
instructor who is low key, soft-spoken, and demonstrates
a low level of enthusiasm is not an effective teacher. Nonetheless, our study is a 3-year study with a large sample size,
and it provides unique results related to distance and campus students in a professional pharmacy program. In addition, the ndings related to student factors are important in
light of the new CAPE Educational Outcomes 2013, which
emphasize the affective domain aspects of students personal and professional growth.17 Further, Figure 1 data
may lend support that faculty enthusiasm items (which
demonstrate the highest correlation with student intrinsic

motivation and vitality) may be a catalyst for all other
factors to t in place, establish a healthy faculty-student
relationship, and improve student learning.
CONCLUSION
While this study does address the importance of content mastery and instructional methodologies, it focuses
on issues related to instructor attitude, instructor enthusiasm, and teaching style, which all were shown to play
a critical role in the learning process. Thus, instructors
have a responsibility to evaluate, re-evaluate, and analyze
the above factors to address any related issues that impact
the learning process, including their influence on professional students intrinsic motivation, vitality, and ability
to meet educational outcomes.
REFERENCES
1. Alsharif NZ, Roche VF. Optimizing the four important
interactions in distance education. Curr Pharm Teach Learn. 2010;
2(2):114-125.
2. Alsharif NZ. Drug Structure and treatment algorithm: treatment of
hypertension. Curr Pharm Teach Learn. 2010;2(1):52-66.
3. Alsharif NZ, Henriksen B. Assessment of electronic integration of
prerequisite content into a medicinal Chemistry course on student
learning. Am J Pharm Educ. 2009;73(8):Article 150.
4. Alsharif NZ. Clinical toxicology: a practical and patient-oriented
approach. Am J Pharm Educ. 2008;72(5):Article 120.
5. Alsharif NZ, Galt KA. Evaluation of an instructional model to
teach clinically relevant medicinal chemistry in a campus and
distance pathway. Am J Pharm Educ. 2008;72(2):Article 31.
6. Alsharif NZ, Shara MA, Roche VF. The structurally-based
therapeutic evaluation concept: An opportunity for curriculum
integration and interdisciplinary teaching. Am J Pharm Educ.
2001;65:314-323.
7. Alsharif NZ. Faculty enthusiasm: a blessing or a curse. Am
J Pharm Educ. 2011;75(2):Article 23.
8. Dille AK, Placone D. Teacher characteristics and student learning.
J Econ Econ Educ Res. 2008; 9(3):15-28.
9. DeLong M, Winter D. Learning to Teaching and Teaching to
Learn Mathematics: Resources for Professional Development.
Mathematical Association of America; 2002: 163.
10. Patrick B, Hisley J, Kempler, T. Whats everybody so excited
about?: the effects of teacher enthusiasm on student intrinsic
motivation and vitality. J Exp Educ. 2000;68(3):217-236.
11. Deci EL. Interest and the intrinsic motivation of behavior. In:
Renninger KA, Hidi S, Krapp A, eds. The Role of Interest in Learning
and Development. Hillsdale, NJ: Erlbaum; 1992:43-70.
12. Deci EL, Vallerand RJ, Pelletier LG, Ryan, RM. Motivation and
education: the self-determination perspective. Educ Psychol. 1991;
26(3/4):325-346.
13. Ryan RM, Frederick C. On energy, personality and health:
subjective vitality as a dynamic reection of well-being. J Pers.
1997;65(3):529-565.
14. Yair G. Educational battleelds in America: the tug-of-war
over students engagement with instruction. Sociol Educ. 2000;
73(4):155-174.
17

35. Davies J, Arlett C, Carpenter S, Lamb F, Donaghy L. What
makes a good engineering lecturer? Students put their thoughts in
writing. Eur J Eng Educ. 2006;31(5):543-553.
36. Krauss S, Brunner M, Kunter M, et al. Pedagogical content
knowledge and content knowledge of secondary mathematics
teachers. J Educ Psychol. 2008;100(3):716-725.
37. Witt PL, Wheeless LR. Nonverbal communication expectancies
about teachers and enrollment behavior in distance learning. Commun
Educ. 1999;48:149-154.
38. Will PL, Wheeless LR. An experimental study of teachers
verbal and nonverbal immediacy and students affective and
cognitive learning. Commun Educ. 2001;50(4):327-342.
39. Myers SA, Rocca KA. Perceived instructor argumentativeness
and verbal aggressiveness in the college classroom: Effects on
student perceptions of climate, apprehension and state motivation.
West J Commun. 2001;65(2);113-137.
40. Senko C, Hulleman CS, Harackiewiez JM. Achievement goal
theory at the crossroads: old controversies, current challenges and
new direction. Educ Psychol. 2011;46(1):26-47.
41. Strahan D. Successful teachers develop academic momentum
with reluctant students. Middle Sch J. 2008;39(5):4-12.
42. Biggs J. Teaching for Quality at University: What the Student
Does. Buckingham: Society for Research into Higher Education; 2000.
43. Menges RJ, Austin AE. Teaching in higher education. In:
Richardson V, ed. Handbook of Research on Teaching. Washington,
DC: American Educational Research Association; 2001:1123-1156.
44. Heller KA. Identication of gifted and talented students. Psychol
Sci. 2004;46(3):302-323.
45. Tomlinson CA, Brighton C, Hertberg H, et al. Differentiating
instruction in response to student readiness, interest and learning
prole in academically diverse classrooms: a review of literature.
J Educ Gifted. 2003;27(2/3):119-145.
46. Ismail B, Hayes K. Factors that affect student motivation in
a dairy products elective course. J Food Science Educ. 2004;
4(1):15-17.
47. Harrold MW. Educational challenges facing basis science
faculties. Am J Pharm Educ. 2004;68(1):Article 15.
48. Woster PM. Maintaining basic science content throughout the
PharmD curriculum. Am J Pharm Educ. 2003;67(3):Article 99.
49. Deci EL, Ryan RM. A motivational approach to self: Integration
in personality. In: Dienstbier R, ed. The Role of Interest in Learning
and Development. Hillsdale, NJ: Erlbaum; 1991:43-70.
50. Deci EL. Effects of externally mediated rewards on intrinsic
motivation. J Pers Soc Psychol. 1971;18(1):105-115.
51. Gottfried AE. Academic intrinsic motivation in young
elementary school children. J Educ Psychol. 1990;82(3):525-538.
52. Patrick BC, Skinner EA, Connell JP. What motivates childrens
behavior and emotion? The joint effects of perceived control and
autonomy in the academic domain. J Pers Soc Psychol. 1993;
65(4):781-791.
53. Baker S. Intrinsic, extrinsic and amotivational orientation: their
role in university adjustment, stress, well-being and subsequent
academic performance. Curr Psychol. 2004;23(3):189-202.
15. Brown M. Dialogue for engagement. Educause Rev. 2010;

45(5):38-40.
16. Stoner SC, Fincham JE. Faculty role in classroom engagement
and attendance. Am J Pharm Educ. 2012;76(5):Article 75.
17. The Center for the Advancement of Pharmacy Education.
Educational Outcomes 2013. http://www.aacp.org/resources/
education/cape/Open%20Access%20Documents/
CAPEoutcomes2013.pdf. Accessed November 5, 2013.
18. Malone PM, Glynn GE, Stohs SJ. The development and structure
of a web-based entry-level doctor of pharmacy pathway at Creighton
University Medical Center. Am J Pharm Educ. 2004;68(2):Article 46.
19. Collins ML. Effects of enthusiasm training on pre-service
elementary teachers. Res Teach Educ. 1978;29(1):53-57.
20. Coates J. Generational Learning Styles. River Falls, WI: Lern
Books; 2007.
21. Fisher T, Crabtree J. Generational cohort theory: have we
overlooked an important aspect of the entry-level occupational
therapy doctoral debate. Am J Occup Ther. 2009;63(5):656-660.
22. Sprinkle JE. Student perceptions of effectiveness: an examination
of the inuence of student biases. Coll Stud J. 2008;42(2):276-293.
23. Chickering AW, Gamson ZF. Seven principles for good practice
in undergraduate education. AAHE Bulletin. 1987;39(7):3-7.
24. Kuh GD, Kinzie J, Buckley JA, Bridges BK, Hayek JC. (2006).
What matters to student success: a review of the literature.
Commissioned report for the National Symposium on Postsecondary
Student Success: Spearheading a Dialogue on Student Success. http://
nces.ed.gov/npec/pdf/Kuh_Team_Report.pdf. Accessed August 2, 2014.
25. Alsharif NZ, Galt K, Mehanna A, Chapman R, Ogunbadeniyi
AM. Instructional model to teach clinically relevant medicinal
chemistry. Am J Pharm Educ. 2006;70(4):Article 90.
26. Monteiro S, Almeida LS, Vasconcelos RM. The role of teachers
at university: what do high achiever students look for? J Scholarsh
Teach Learn. 2012;12(2):65-77.
27. Thweatt KS, McCroskey JC. Teacher non-immediacy and
misbehavior: unintentional negative communication. Commun Res
Rep. 1996;13(2):198-204.
28. Arbuckle J, Williams BD. Students perceptions of
expressiveness: age and gender effects on teacher evaluations. Sex
Roles. 2003;49(9/10):507-516.
29. Amin ME. Gender as a discriminating factor in the evaluation of
teaching. Asses Eval Higher Educ. 1994;19(2):134-143.
30. Feldman KA. College students views of male and female
college teachers: Part I. Evidence from the social laboratory and
experiments. Res Higher Educ. 1992;33(3):317-375.
31. Feldman KA. College students views of male and female
college teachers: Part II. Evidence from the students evaluations of
their classroom teachers. Res Higher Educ. 1993;34(2):151-191.
32. Fernandez J, Mateo MA. Student and faculty gender in ratings of
university teaching quality. Sex Roles. 1997;37(11/12):997-1003.
33. Karesenti T, Thivert G. The inuence of gender on within-term
changes in junior college student motivation. Paper presented at:
American Educational Research Association; New Orleans, LA.
34. Vallerand RJ, Fortier MS, Guay F. Self-determination and
persistence in a real-life setting: Toward motivational model of high
school dropout. J Pers Soc Psychol. 1997;72(5):1161-1176.
18

Appendix I. Course Specific Strategies to Support Components of Intrinsic Motivation for Students
Intrinsic Motivation Component
1. Supporting of
self-determination
Course Specific Strategies to Achieve

d
d
d
d
d
d
d
d
2. Promotion of perceived
competence
Conduct pre-assessment quiz to prepare for the classroom session.

Provide interactive classroom session.
Provide several active learning exercises on the course website and in the in-class
interactive PowerPoint presentations.
Provide interactive Softchalk lesson handout with learning activities.
Encourage use of analogies (e.g. food analogy exercise and
structure activity relationship).
Encourage students to write own innovative short story related to the content being
taught (eg, The magnicent penicillins, The Town of Neurotransmitoron,
The Attack on Muskulopolis, The Tale of Aminoglycosides).
Encourage students to write their own take home message.
Encourage students to nd the clinical relevance of what is being taught.
Help students characterize what they are learning by utilizing strategies based on
Krathwol Taxonomy.53
Encourage notion of faculty member as a facilitator/partner in the learning process.
Help the students transition to the higher level of thinking required in this course.
Provide a standardized lesson handout based on Blooms and
Krathwols Taxonomy.53
Challenge students to answer questions in classroom
Challenge students to answer their own questions in the classroom, online, and
in face-to-face and virtual review sessions.
Provide practice exams and case studies.
Challenge the students to write an original case study for the exam.
Provide student answers as key answers for questions on the exam.
Provide constructive comments on assignments and exams.
19
RESEARCH
Identifying Achievement Goals and Their Relationship to Academic
Achievement in Undergraduate Pharmacy Students
Saleh Alrakaf, MSc Pharm,a Erica Sainsbury, PhD,a Grenville Rose, PhD,b and Lorraine Smith, PhDa
a
b
Faculty of Pharmacy, The University of Sydney, Sydney, New South Wales, Australia
Aftercare, Sydney, New South Wales, Australia
Submitted July 1, 2013; accepted February 9, 2014; published September 15, 2014.
Objectives. To compare the achievement goal orientations of rst-year with those of third-year undergraduate Australian pharmacy students and to examine the relationship of goal orientations to
academic achievement.
Methods. The Achievement Goal Questionnaire was administered to rst-year and third-year students
during class time. Students grades were obtained from course coordinators.
Results. More rst-year students adopted performance-approach and mastery-approach goals than did
third-year students. Performance-approach goals were positively correlated with academic achievement in the rst year. Chinese Australian students scored the highest in adopting performance-approach
goals. Vietnamese Australian students adopted mastery-avoidance goals more than other ethnicities.
First-year students were more strongly performance approach goal-oriented than third-year students.
Conclusion. Adopting performance-approach goals was positively correlated with academic achievement, while adopting avoidance goals was not. Ethnicity has an effect on the adoption of achievement
goals and academic achievement.
Keywords: achievement goals, achievement goal questionnaire, academic achievement, ethnicity
goals and students who adopt performance goals view

ability and dene success vs failure differently.3
Students who adopt mastery goals tend to view their
abilities as a flexible trait that can be enhanced by hard
work, persistence, and continuous development of their
skills,10 while students who adopt performance goals
view ability as a xed trait that cannot be enhanced.11
Mastery students use self-referential criteria in differentiating between success and failure (ie, feeling they learn
what they need to learn or improve),10 whereas performance students dene success as outperforming their
peers.6 Thus, performance students who believe they have
high ability will enjoy outperforming their peers, while
performance students who believe they have low ability
will avoid such challenges.3
In recent years, achievement goal theorists have further divided mastery goals and performance goals into 4
types: mastery-approach (ie, aiming to learn and understand the task at hand thoroughly), mastery-avoidance (ie,
aiming to avoid losing previously acquired skills or to
avoid not understanding the task at hand thoroughly),
performance-approach (ie, aiming to outperform ones
peers or to demonstrate ones ability to others), and
performance-avoidance (ie, aiming to avoid performing
worse than ones peers).12-15 These distinctions are
INTRODUCTION
The role of goals in human motivation is critical.1

Goals can be dened as a cognitive representation of future aims that a person is committed either to approach or
avoid.2 A class of goals that has received considerable
attention in the educational eld for more than 2 decades
is achievement goals.3,4 According to achievement goal
theory, achievement goals are goals in which competence is the main aim for an individual.5 Thus, achievement goals are dened as a future-focused cognitive
representation that guides behavior to a competencerelated end state that the individual is committed to either
approach or avoid.6 Achievement goal theory tries to
describe and understand the goals students adopt when
dealing with academic activities and the reasons behind
such adoption.7,8 For example, when students face an
academic activity, they adopt either 1 of 2 major types
of achievement goals: mastery goals (ie, to try to learn and
understand the task on hand) or performance goals (ie, to
try to perform well compared to peers).1,9,10 Achievement
goal theorists believe that students who adopt mastery
Corresponding Author: Saleh Alrakaf, Faculty of Pharmacy,
Room S114, Pharmacy and Bank Building A15, The
University of Sydney, NSW, 2006. Tel: 161-2-9351-4501.
Fax: 161-2-9351-4451. E-mail: salr4982@uni.sydney.edu.au
20

supported by a large body of empirical research and are
robust in predicting and understanding student engagement and achievement.6,16,17
The mastery-approach goal has been linked to a number of positive effects such as deep learning,18 high interest,19 high persistence,20 and help seeking.21 Despite
these benecial effects, evidence suggests that students
who adopt this type of goal orientation rarely attain high
academic achievement (ie, grades).6,10 The performanceapproach goal, on the other hand, is associated with shallow learning strategies such as memorization22 but linked
to high academic achievement.8,10,23,24 Performanceavoidance and mastery-avoidance goals have been associated with negative effects, especially in Western
culture, such as stress and anxiety,25-27 low academic
achievement,28,29 and low intrinsic motivation.30,31
In general, cross-sectional studies designed to compare undergraduate students achievement goals are
scarce. However, there is some evidence that suggests
there are differences in the achievement goals adopted
by students in different academic years. Lieberman and
Remedios examined the achievement goals of 1857 undergraduate students from rst, second, third, and fourth
years who were studying in different disciplines such as
psychology, business, biology, art, English, history,
mathematics, and nursing at a Scottish university, and
found that students in the rst year were more masteryapproach oriented than students from any other year.32
The authors attribute their ndings to increased pressure
on students as they advance through their academic
life. This academic pressure undermines interest and
enjoyment, and thus, signicantly decreases pursuing
mastery-approach goals. Another study conducted by
Remedios et al to identify and compare the achievement
goals of Russian undergraduate students, who were taking
English studies course for business in different academic
years, yielded strikingly similar results.9 However, the
authors explained their results in the context of the cultural shift in Russia caused by globalization, which inuenced students to be more individualistic and pragmatic,
with more emphasis on performance than mastery goals.
Only a few studies have aimed to investigate the relationship between ethnicity and achievement goals.33
For example, Elliot and his colleagues found Asian
American undergraduate psychology students adopted
performance-avoidance goals more than their Anglo
American peers.34 The authors attributed these ndings
to subcultural differences between the 2 groups. In general, students from Asian backgrounds valued avoiding
negative outcomes, whereas approaching positive outcomes was valued in Anglo American culture.34 Similar
ndings have been found by Zusho et al; however, they
found that undergraduate Asian American students outscored their Anglo American peers in mathematics.35
Although more than 1000 publications and dissertations report the application of achievement goal theory,6
only one of these (to our knowledge) is in the pharmacy
education setting. Waskiewicz used a framework based
on achievement goal theory to determine student motivation to achieve in a low stakes examination, compared
to their motivation to pursue a doctor of pharmacy
program.36 The author found a signicant relationship
between situational motivation and performanceapproach goal. However, no signicant relationship was
found between the same motivation and masteryapproach goal.
Achievement goal theory provides academics with
invaluable understanding of how their students respond
when they encounter academic activity.29 By understanding students achievement goals, academics might try to
create an environment that can encourage those benecial
goals and limit the non-benecial ones.7 Yet the rst step
is to understand more about pharmacy students achievement goals.
Therefore, the aims of this study are to compare the
achievement goal orientations between first-year and
third-year undergraduate pharmacy students, investigate
Australian undergraduate pharmacy students achievement goals and their relationship to their academic
achievement, and examine the inuence of different ethnicities on achievement goals and academic achievement.
METHODS
This study received approval from the Human Research Ethics Committee, The University of Sydney,
NSW Australia.
The bachelor of pharmacy degree program at the
University of Sydney is a 4-year undergraduate course
of study that qualifies graduates to apply for registration
as a pharmacist in Australia.37 The participants for this
study were rst-year and third-year undergraduate students in this program. In total, 380 students agreed to
participate in the study.
We used the Achievement Goal Questionnaire
(AGQ), which contains 12 items intended to measure
the 4 types of achievement goals on a 7-point Likert scale
(15not at all true of me to 75very true of me). Sociodemographic indicators included in the survey were
gender, age, language spoken at home, and student identication (ID) number. Student ID numbers were used
only for matching students grades with the different
types of achievement goals. Individual students could
not be identied in the analysis. Completion of the survey instrument took approximately 10 minutes.
21

Data regarding participants ethnicities were gathered by asking students to indicate the language spoken
at home, which may be interpreted as more accurately
reecting the cultures of participants than asking for ethnicity in a general question.39 Culture is a variable of interest as it is the prism through which individuals view the
world and may specically affect their approach to education.40 Another benet of this question is that it enabled
us to identify participants ethnicities with greater precision. For example, instead of writing Asian in answering an ethnicity question, a participant would indicate the
precise ethnicity, such as Vietnamese or Korean, when
identifying the language spoken at home.
The study was initiated in the second semester of the
academic year in 2012. Students were invited to participate in the study during normal lectures or tutorials (ie,
small group discussions). They were advised that participation was voluntary and if they chose to participate they
could withdraw from the study at any time. In addition,
students were advised that their decision to participate
would not impact their academic performance results or
influence student-teacher relationships. Researchers
approached students as a group.
At the end of the semester, students raw grades in 2
courses, Foundations of Pharmacy and Endocrine, Diabetes and Reproductive [System], were obtained from
course coordinators. Foundations in Pharmacy is a rstyear course aiming to introduce students to the pharmacy
profession and the roles of pharmacists in the health care
system.41 Endocrine, Diabetes and Reproductive is
a course taken in the third year that covers the pharmacotherapeutics of endocrine, diabetes and reproductive
disorders.41
SPSS 20 (SPSS Inc, Chicago, Illinois) was used for
all statistical analyses. Descriptive statistics regarding
year group, gender, age, and language spoken at home
were reported. Correlation analysis was used to determine
the strength and direction of the relationships between
achievement goals and academic achievement. An independent sample t test was used to compare the achievement goal orientations between rst-year and third-year
students. One-way analysis of variance (ANOVA) was
used as preliminary analysis for multiple comparisons
of predominant languages spoken at home and each type
of achievement goal. A 2-way ANOVA was conducted to
explore the impact of academic year (rst or third) and
predominant ethnicities on each achievement goal. Similar analysis was used to explore the same impact on academic achievement. All mean difference analyses were
subjected to post hoc tests (Tukey test).
A direct logistic regression procedure was performed
to determine the extent to which achievement goals and
languages spoken at home contributed to academic

achievement. Academic achievement was transformed
into a binary variable using the grade 74 as a cut point.
Thus, students grades were regressed as pass and credit
vs distinction and high distinction. Languages spoken
at home also were transformed into a binary variable
(English/other languages). The Forced Entry Method was
used to examine the odds ratios of all variables, even if not
signicant. A p value of less than 0.05 was considered
signicant for all analyses.
RESULTS
Three hundred eighty (251 female, 128 male, and 1
gender unspecified) undergraduate pharmacy students
from years 1 and 3, with a mean age of 19.7 years, agreed
to participate in this study (76% response rate). Descriptive statistics for participants are reported in Table 1.
The predominant languages spoken at home (ethnicities) in approximately 90% of both classes of students
were English, Chinese, Vietnamese, Korean, and Arabic.
The number of different ethnicities reported by first-year
and third-year students was 22 and 13, respectively.
Independent t test results (Table 2) revealed differences between rst-year and third-year students in
performance-approach and mastery-approach goals, with
rst-year students scoring signicantly higher than thirdyear students in both. In contrast, no signicant differences
in the scores of rst- and third-year students were observed
for performance-avoidance and mastery-avoidance goals.
Correlations between achievement goals and grades
are reported in Table 3. Among rst-year students, higher
scores on performance-approach goals were associated with higher grades. In the same year, adoption of
performance-avoidance goals signicantly correlated
Table 1. Demographics of First- and Third-Year Pharmacy
Students
Academic
Year
First-year
260
67.7
Third-year 120
62.5
22
Gender, Age, Mean

% Female
(SD)
Language, %
18.8 (2.12) English, 28.4

Chinese,a 24.1
Vietnamese, 15.2
Arabic, 11.7
Korean, 8.2
Other, 12.4
21.5 (3.56) English, 39.0
Chinese,a 27.0
Korean, 13.6
Vietnamese, 8.5
Arabic, 5.1
Other, 6.8
Chinese 5 Cantonese, Mandarin, Chinese, and Teochew languages.

signicant. The impact of academic year and ethnicity on
mastery-avoidance goals was signicant. Post hoc comparisons using the Tukey test indicated that Vietnamese
Australian students reported higher adoption of masteryavoidance goals than their Anglo Australian and Arab
Australian peers. The interaction effect between predominant ethnicities and students academic year was not
signicant.
A two-way, between-groups ANOVA was also conducted to explore the impact of academic year and predominant ethnicity on students grades and are shown in
Table 5. All effects were signicant at the 0.05 level. The
two-way ANOVA of students grades based on year
group showed that mean scores were signicantly higher
for third-year (Mean6SD, 74.768.8) compared to rstyear (Mean6SD, 70.468.3. There was a signicant main
effect for ethnicity. Post hoc comparisons using the Tukey
test lacked the power to determine where that difference
was, beyond that it was between the ethnicities scoring
highest and lowest in this study, which was determined
from the main effect. The interaction effect between predominant ethnicities and academic year was not signicant. Direct logistic regression was performed to assess
the impact of a number of factors on the students grades.
The model contained 5 independent variables (the 4
achievement goals, and ethnicities). The full model
containing all predictors was signicant. The model
as a whole explained between 5.1% (Cox and Snell R
squared) and 6.9% (Nagelkerke R squared) of the variance, and correctly classied 63.9% of cases.
As shown in Table 6, only 2 of the independent variables made a unique signicant contribution to the model
(mastery-avoidance and other ethnicities). The strongest
predictor of grades was mastery avoidance, recording an
odds ratio of 0.83. This indicated that students who pursued the mastery-avoidance goal were less likely to
achieve high grades than those who did not pursue this
goal, controlling for all other factors in the model. Other
ethnicities were also signicantly predictive of higher
academic achievement with an odds ratio of 1.62.
Table 2. Achievement Goal Values in First- and Third-Year

Pharmacy Students
Performance-approach
First-year
Third-year
Performance-avoidance
First-year
Third-year
Mastery-approach
First-year
Third-year
Mastery-avoidance
First-year
Third-year
Mean (SD)
5.1 (1.3)
4.5 (1.4)
,0.001
5.6 (1.3)
5.5 (1.3)
0.25
5.9 (1.0)
5.6 (1.2)
0.01
4.8 (1.5)
4.6 (1.4)
0.15
with lower grades. Among third-year students, adoption

of mastery-avoidance goals signicantly correlated with
lower grades. Although collapsing different Asian ethnicities (ie, Chinese, Vietnamese, and Korean) into one
group has statistical appeal, yielding greater power,
a one-way, between group ANOVA revealed signicant
differences at the p,0.05 level in performance-approach
and mastery-avoidance goals and academic achievement
scores among the 3 ethnic groups. Thus, each predominant Asian ethnicity was analyzed separately.
A 2-way, ANOVA was conducted to explore the
impact of students academic year and predominant ethnicity on each achievement goal. Ethnic differences in
mean levels of goals and year are reported in Table 4.
Regarding performance-approach goals, both academic
years and ethnicity had signicant impact. Post hoc
comparisons of the main effect using the Tukey test indicated that Chinese Australian students reported higher
performance-approach goals than their Anglo Australian
and Korean Australian peers. No signicant interaction
was found between the predominant ethnicities and
students academic year. No signicant impact was
found from academic year (or predominant ethnicity) on
performance-avoidance goals. The interaction effect between predominant ethnicity and academic year was not
signicant. Only academic year had a signicant impact
on mastery-approach goals. The interaction effect between predominant ethnicity and academic year was not
DISCUSSION
For more than 2 decades, achievement goal theory
has captured a considerable amount of attention in
Table 3. Correlations between Grades and Achievement Goals

Year
Grades
R
P
R
P
Mastery-approach
Mastery-avoidance
0.14
0.04
0.05
0.61
-0.14
0.03
-0.18
0.06
0.06
0.32
-0.16
0.09
-0.07
0.27
-0.31
0.001
23

Table 4. Ethnic and Year Group Achievement Goals , Mean (SD)
Goals, n=1st yr/3rd yr
Year 1
5.0 (1.4)e
Year 3
4.6 (1.4)f
Year 1
5.6 (1.3)a
Year 3
5.5 (1.2)a
Mastery-approach
Year 1
5.6 (1.0)e
Year 3
5.5 (1.2)f
Mastery-avoidance
Year 1
4.9 (1.5)e
Year 3
4.5 (1.3)f
Anglo 73/46
Chinese 62/32
Vietnamese 39/10
Korean 21/16
Arabic 30/6
4.9 (1.5)a
5.3 (1.2)b
5.4 (1.1)a,b,c
4.6 (1.6)a,c
4.7 (1.4)a,b,c
4.4 (1.6)
5.0 (1.2)
4.5 (1.4)
4.3 (1.3)
4.4 (1.3)
5.6 (1.3)a
5.6 (1.2)a
5.9 (1.1)a
5.8 (1.4)a
5.3 (1.5)a
5.6 (1.3)
5.4 (1.2)
5.8 (1.3)
5.3 (1.1)
6.3 (0.8)
6.0 (1.0)a
5.9 (0.8)a
6.0 (1.2)a
5.4 (1.4)a
5.7 (1.1)a
5.5 (1.3)
5.3 (1.2)
5.8 (1.0)
5.7 (1.0)
4.4 (1.3)
4.6 (1.6)a,c
5.1 (1.1)a,b,c
5.5 (1.3)b
4.5 (1.5)a,b,c
4.3 (1.6)a,c
4.2 (1.5)
4.7 (1.3)
4.8 (1.3)
4.6 (1.2)
4.3 (0.9)
Means in the same row that do not share the same superscripts differ signicantly at p,0.05.
education, with more than 1000 articles and dissertations

being written using it as a framework.3,4,6 Four types of
achievement goals are acknowledged: mastery-approach,
master-avoidance, performance-approach, and performanceavoidance.12-15 The primary aims of this study were to
identify Australian undergraduate pharmacy students
achievement goals, determine the relationships between
goals and academic achievement, and compare the
achievement goals of 2 different cohorts of undergraduate
students. A secondary aim of this study was to investigate
any relationships between ethnicity, type of achievement
goals, and academic achievement.
Comparison of first-year and third-year students results showed that rst-year students were oriented more
strongly toward performance-approach and masteryapproach goals than third-year students. Our nding that
rst-year students adopted mastery-approach goals more
than third-year students is consistent with Lieberman and
Remedios32 and Remedios et al.9 However, our ndings
regarding performance-approach differed from both previous studies. Results from Remedios et al. found no signicant differences in the adoption of this goal among
rst-year, second-year, third-year, and fourth year students.9 Lieberman and Remedios found third-year students adopted performance-approach goals more than
rst-year students. In our study, the higher adoption of
performance-approach goals by rst-year students compared to third-year students might have been a result of
competitive high school environments from which rstyear students had just come.
Our results show that first-year students who adopted
performance-approach goals received higher grades in
their subject compared to their peers who adopted any
other type of achievement goals. These findings are consistent with several previous studies that indicate the positive association between performance-approach and
academic achievement.8,10,23,24,28,29,42 Perhaps students
Table 5. Academic Achievement Mean (SD) Scores for Each
Ethnicity in Both Year Groups
Ethnicity
Anglo
Year 1
Year 3
Chinese
Year 1
Year 3
Vietnamese
Year 1
Year 3
Korean
Year 1
Year 3
Arabic
Year 1
Year 3
24
Year (n)
Mean (SD)
67
45
71.5 (9.0)
75.0 (9.6)
58
29
70.7 (6.7)
76.0 (7.1)
38
8
68.7 (7.4)
70.6 (7.6)
17
15
65.0 (9.4)
72.7 (9.2)
28
6
72.5 (8.5)
75.8 (10.4)

Table 6. Logistic Regression Analysis Examining Different Factors That Might Predict Academic Achievement.
Age
Females
Mastery-approach
Mastery-avoidance
Other ethnicities
Constant
S.E.
Odds Ratio
-0.046
0.563
0.073
-0.157
-0.113
-0.191
-0.558
1.446
0.046
0.260
0.086
0.090
0.112
0.085
0.248
1.177
0.319
0.030
0.395
0.082
0.310
0.024
0.024
0.219
0.955
1.756
1.076
0.854
0.893
0.826
0.572
4.245
95% C.I. for EXP(B)

Lower
Upper
0.872
1.056
0.909
0.716
0.717
0.700
0.352
1.046
2.922
1.273
1.020
1.111
0.975
0.930
The overall model is signicant at P , 0.05.
who adopt a strong performance-approach goal orientation focus on topics that appear important and testable
for their teachers. In contrast, students who are strongly
mastery-oriented are more likely to follow their own interest and study subject material that is appealing to them
regardless of its testability.43 Almost all faculty members
want their students to be curious and interested, and to use
deep-learning strategies (ie, adopt a mastery-approach
goal) when they study and, at the same time, attain
higher grades (ie, adopt a performance-approach goal).
Although reaping the benets of both types of achievement goals is clearly benecial, the task for educators is to
develop ways to foster this combination. One way is by
helping students pursue mastery-approach goals throughout the semester and then encouraging them to pursue
performance-approach goals when preparing for examinations.19 This can be achieved through appropriate curriculum development and an understanding of teacher
qualities that enhance and support the delivery of course
curricula.19 These qualities, if adopted by academicians,
might help create a combined mastery-approach and
performance-approach environment.
Surprisingly, among the third year students, there was
no significant relationship between academic achievement
and performance-approach goals. This result was inconsistent with previous research findings.8,22,23,27-30 Although
our data did not allow us to elucidate why this was, we posit
that the nature of the examined course (ie, Endocrine, Diabetes and Reproductive) did not support shallow learning
strategies such as memorization. Thus, adoption of this
type of achievement goal had no signicant association
with academic achievement.
In contrast to much of the published literature,33,34,44,45
which has grouped different Asian ethnicities under one
umbrella and applied ndings to the whole group, our
study clearly revealed that individual Asian ethnicities
varied in their adoption of each type of achievement
goal. Vietnamese students, for example, had signicantly
higher scores on mastery-avoidance goal than their Korean peers, whereas Chinese students had signicantly
higher performance-approach goal scores than Korean
students. To the best of our knowledge, this is the rst
study to analyze each Asian ethnicity separately, and
doing so has yielded signicant conclusions. Zusho et al
did not nd any signicant difference between Asian
American and Anglo American students in pursuing
performance-approach goals,35 but our study showed that
Chinese Australian students were adopting performanceapproach goals signicantly more than their Anglo Australian peers, possibly because Chinese Australian parents
expect high academic performance from their children.45
However, there were no signicant differences between
Anglo Australian and Vietnamese Australian or Korean
Australian students. Our nding that more Vietnamese
Australian students adopted mastery-avoidance goals
than did their Anglo Australian peers was consistent, to
some degree, with literature that found more Asian
students adopted avoidance goals than did Caucasian
students.33,34,46
The contradictory findings of this study in comparison with previous research may be attributed to 3 factors.
First, this study made a clear distinction between Asian
ethnicities while most other studies have not, suggesting
that a one group ts all approach misses the opportunity
to more precisely understand different ethnic groups. Second, most published literature focuses on psychology
students.34,35 There might be a correlation among
discipline-specic subjects, achievement goals, and academic achievement. Third, this study was conducted in
Australia and given the multicultural nature of the country, particularly in Sydney, the study suggests that no
single strategy may suit all Australian students, and that
future work should address cultural differences more
directly.
Although there was a significant impact of predominant ethnicities upon academic achievement, post hoc
25

comparisons were unable to determine the exact locations
of the differences. Either a larger sample size of different
ethnicities or a replication study with planned comparisons based on the current study would allow us to explore
the fine-grain effect of each ethnicity.
This study was important in identifying undergraduate pharmacy students achievement goals and the relationship of those goals to academic achievement. In
addition, this study shed some light on the relationship
between different ethnicities and achievement goals. As
quantitative studies do not answer the question why such
phenomena occur, a qualitative investigation of this
phenomenon may yield useful additional results. In-depth
interviews with a purposive sample of students from this
study may yield more information regarding student adoption of one achievement goal over another, the relationship
between academic achievement and performance-approach
goals in third-year students, and ethnic differences. Further,
understanding the qualities that mastery-approach and
performance-approach students would like to see in their
instructors may help academics create environments that
foster the adoption of both goals.
Limitations of the study include the small sample
size of some of the ethnic groups. For example, despite
the significant impact of predominant ethnicities upon
academic achievement, the Tukey test failed to determine
where the significant differences were between each ethnicity. Another limitation was using cohorts from only
one university. A study that includes undergraduate pharmacy students from different Australian universities may
yield more generalizable results.
3. Senko C, Hulleman CS, Harackiewicz JM. Achievement goal

theory at the crossroads: old controversies, current challenges, and
new directions. Educ Psychol. 2011;46(1):26-47.
4. Meece JL, Anderman EM, Anderman LH. Classroom goal
structure, student motivation, and academic achievement. Annu Rev
Psychol. 2006;57:487-503.
5. Elliot AJ, Dweck CS. Competence and motivation: competence as
the core of achievement motivition. In: Elliot AJ, Dweck CS, eds.
Handbook of Competence and Motivation. New York, NY: The
Guiforsd Publications Inc.; 2005:3-12.
6. Hulleman CS, Schrager SM, Bodmann SM, Harackiewicz JM. A
meta-analytic review of achievement goal measures: different labels
for the same constructs or different constructs with similar labels?
Psychol Bull. 2010;136(3):422-449.
7. Kaplan A, Maehr M. The contributions and prospects of
orientation theory. Educ Psychol Rev. 2007;19(2):141-184.
8. Harackiewicz JM, Barron KE, Tauer JM, Elliot AJ. Predicting
success in college: a longitudinal study of achievement goals and ability
measures as predictors of interest and performance from freshman year
through graduation. J Educ Psychol. 2002;94(3):562-575.
9. Remedios R, Kiseleva Z, Elliott J. Goal orientations in Russian
university students: from mastery to performance? Educ Psychol.
2008;28(6):677-691.
10. Hulleman CS, Senko C. Up round the bend: forcasts for
achievement goal theory and research in 2020. In: Urdan TC,
Karabenick SA, eds. The Decade Ahead: Theoritical Perspectives on
Motivation and Achievement. Vol 16A. Bingley, UK: Emerald Group
Publishing Limited; 2010:71-104.
11. Dweck CS. Motivational processes affecting learning. Am
Psychol. 1986;41(10):1040-1048.
12. Elliot AJ. Approach and avoidance motivation and achievement
goals. Educ Psychol. 1999;34(3):149-169.
13. Elliott JG, McGregor HA. A 2x2 achievement goal framwork.
J Pers Soc Psychol. 2001;80(3):501-519.
14. Elliot AJ, Thrash TM. Achievement goals and the hierarchical
model of achievement motivation. Educ Psychol Rev. 2001;13(2):
139-156.
15. Isoard-Gautheur S, Guillet-Descas E, Duda JL. How to achieve
in elite training centers without burning out? An achievement goal
theory perspective. Psychol Sport Exerc. 2013;14(1):72-83.
16. Huang C. Discriminant and criterion-related validity of
achievement goals in predicting academic achievement: a metaanalysis. J Educ Psychol. 2012;104(1):48-73.
17. Shim SS, Cho Y, Wang C. Classroom goal structures, social
achievement goals, and adjustment in middle school. Learn Instr.
2013;23:69-77.
18. Diseth A. Self-efcacy, goal orientations and learning strategies
as mediators between preceding and subsequent academic
achievement. Learn Individ Differ. 2011;21(2):191-195.
19. Harackiewicz JM, Barron KE, Pintrich PR, Elliot AJ, Thrash
TM. Revision of achievement goal theory: necessary and
illuminating. J Educ Psychol. 2002;94(3):638-645.
20. Wolters CA. Advancing achievement goal theory: using goal
structures and goal orientations to predict students motivation,
cognition, and achievement. J Educ Psychol. 2004;96(2):236-250.
21. Roussel P, Elliot AJ, Feltman R. The inuence of achievement
goals and social goals on help-seeking from peers in an academic
context. Learn Instr. 2011;21(3):394-402.
22. Elliot AJ, McGregor HA, Gable S. Achievement goals, study
strategies, and exam performance: A mediational analysis. J Educ
Psychol. 1999;91(3):549-563.
CONCLUSION
Adopting performance-approach goals positively
correlated with academic achievement, while adopting
either performance-avoidance or mastery-avoidance
goals did not. First-year students were more performanceapproach and mastery-approach oriented than thirdyear students. Ethnicity affected achievement goals
and academic achievement. Chinese Australian students
indicated stronger preferences for adopting performanceapproach goals, whereas Vietnamese Australian students adopted mastery-avoidance goals more than any
other ethnicities.
REFERENCES
1. Murayama K, Elliot AJ, Friedman R. Achievement goals. In: Ryan
RM, ed. The Oxford Handbook of Human Motivation. New York,
NY: Oxford University Press; 2012:191-207.
2. Elliot AJ, Fryer JW. The goal concept in psychology. In: Shah
Gardner W, eds. Handbook of Motivational Science. New York, NY:
Guilford Press.; 2008:235-550.
26

34. Elliot AJ, Chirkov VI, Kim Y, Sheldon KM. A cross-cultural
analysis of avoidance (relative to approach) personal goals. Psychol
Sci. 2001;12(6):505-510.
35. Zusho A, Pintrich PR, Cortina KS. Motives, goals, and adaptive
patterns of performance in Asian American and Anglo American
students. Learn Individ Differ. 2005;15(2):141-158.
36. Waskiewicz RA. Achievement goal orientation and situational
motivation for a low-stakes test of content knowledge. Am J Pharm
Educ. 2012;76(4):Article 65.
37. Smith L, Saini B, Krass I, Chen T, Bosnic-Anticevich S,
Sainsbury E. Pharmacy students approaches to learning in an
Australian university. Am J Pharm Educ. 2007;71(6):Article 120.
38. Elliot AJ, Murayama K. On the measurement of achievement
goals: critique, illustration, and application. J Educ Psychol.
2008;100(3):613-628.
39. Marian V, Kaushanskaya M. Self-construal and emotion in
bicultural bilinguals. J MemLang. 2004;51(2):190-201.
40. Yamauchi LA, Tharp RG. Culturally compatible
conversations in Native American classrooms. Linguist Educ.
1995;7(4):349-367.
41. Sydney Uo. University of Sydney HandBooks. In: Sydney
TUo, ed. 00026A. Sydney, Australia: The University of Sydney
2012.
42. Harackiewicz JM, Barron KE, Tauer JM, Carter SM, Elliot AJ.
Short-term and long-term consequences of achievement goals:
predicting interest and performance over time. J Educ Psychol.
2000;92(2):316-330.
43. Senko C, Miles KM. Pursuing their own learning agenda: how
mastery-oriented students jeopardize their class performance.
Contemp Educ Psychol. 2008;33(4):561-583.
44. Kao G. Asian-Americans as model minorities? A look at their
academic performance. Am J Educ. 1995;103(2):121-159.
45. Li J. Expectations of Chinese immigrant parents for their
childrens education: the interplay of Chinese tradition and the
Canadian context. Can J Educ. 2001;26(4):477-494.
46. Lee AY, Aaker JL, Gardner WL. The pleasures and pains of
distinct self-construals: the role of interdependence in regulatory
focus. J Pers Soc Psychol. 2000;78(6):1122-1134.
23. Cury F, Elliot AJ, Da Fonseca D, Moller AC. The socialcognitive model of achievement motivation and the 2* 2 achievement
goal framework. J Pers Soc Psychol. 2006;90(4):666-679.
24. Barron KE, Harackiewicz JM. Revisiting the benets of
performance-approach goals in the college classroom: Exploring
the role of goals in advanced college courses. Int J Educ Res.
2003;39(4/5):357-374.
25. Putwain DW, Symes W. Achievement goals as mediators of the
relationship between competence beliefs and test anxiety. Br J Educ
Psychol. 2012;82(2):207-224.
26. Elliot AJ, Pekrun R. Emotion in the hierarchical model of
approach-avoidance achievement motivation. In: Schutz PA, Pekrun
R, eds. Emotion in Education. San Diego, CA: Elsevier Academic
Press; 2007:57-73.
27. Sideridis GD. The regulation of affect, anxiety, and stressful
arousal from adopting mastery-avoidance goal orientations. Stress
Health. 2008;24(1):55-69.
28. Murayama K, Elliot AJ. The competition-performance relation:
a meta-analytic review and test of the opposing processes model of
competition and performance. Psychol Bull. 2012;138(6):1035-1070.
29. Van Yperen NW, Elliot AJ, Anseel F. The inuence of masteryavoidance goals on performance improvement. Eur J Soc Psychol.
2009;39(6):932-943.
30. Linnenbrink-Garcia L, Middleton MJ, Ciani KD, Easter MA,
OKeefe PA, Zusho A. The strength of the relation between
performance-approach and performance-avoidance goal orientations:
theoretical, methodological, and instructional implications. Educ
Psychol. 2012;47(4):281-301.
31. Van Yperen NW. A novel approach to assessing achievement
goals in the context of the 2 x 2 framework: identifying distinct
proles of individuals with different dominant achievement goals.
Pers Soc Psychol Bull. 2006;32(11):1432-1445.
32. Lieberman DA, Remedios R. Do undergraduates motives for
studying change as they progress through their degrees? Br J Educ
Psychol. 2007;77(2):379-395.
33. Witkow MR, Fuligni AJ. Achievement goals and daily school
experiences among adolescents with Asian, Latino, and European
American backgrounds. J Educ Psychol. 2007;99(3):584-596.
27
RESEARCH
Self-Efficacy and Self-Esteem in Third-Year Pharmacy Students
Mark L. Yorra, EdD
School of Pharmacy, Northeastern University, Boston, Massachusetts
Submitted October 2, 2014; accepted February 8, 2014; published September 15, 2014.
Objective. To identify the experiential and demographic factors affecting the self-efcacy and selfesteem of third-year pharmacy (P3) students.
Methods. A 25-item survey that included the Rosenberg Self-Esteem Scale and the General SelfEfcacy Scale, as well as types and length of pharmacy practice experiences and demographic information was administered to doctor of pharmacy (PharmD) students from 5 schools of pharmacy in
New England at the completion of their P3 year.
Results. The survey response rate was approximately 50% of the total target population (399/820).
Students with a grade point average (GPA)$3.0 demonstrated a higher signicant effect from unpaid
introductory pharmacy practice experiences (IPPEs) on their self-efcacy scores ( p,0.05) compared
to students with lower GPAs. Students who had completed more than the required amount of pharmacy
experiences had higher levels of self-efcacy and self-esteem ( p,0.05). Ethnicity also was related to
students levels of self-efcacy and self-esteem.
Conclusion. Self-efcacy and self-esteem are two important factors in pharmacy practice. Colleges
and schools of pharmacy should ensure that students complete enough practice experiences, beyond the
minimum of 300 IPPE hours, as one way to improve their self-efcacy and self-esteem.
Keywords: self-efcacy, self-esteem, experiential learning, introductory pharmacy practice experience
of pharmacy occurs outside of the classroom.5 Experiential learning through IPPEs and APPEs, both unpaid and
paid, provide an opportunity for students to receive formal
appraisals as well as informal feedback from coworkers,
preceptors, and others they interact with during their experiences. The Accreditation Council for Pharmacy Education (ACPE) requires (1) 300 IPPE hours, of which 20%
or 60 hours can be earned through completing simulated
experiences; and (2) 36 weeks at 40 hours a week of
APPEs, as the minimum to be eligible to take the North
American Pharmacist Licensure Examination.6,7 Nevertheless, this amount may still not be sufcient and students without additional pharmacy work experience
outside of the required school-provided experiences
may be unprepared to work independently as pharmacists
at graduation. Such graduates may require several weeks
of additional training on the job in order to function in
their new role as a pharmacist (J. Gallagher, personal
communication, January 25, 2012; C. Perry, personal
communication, January 25, 2012)
A literature review did not yield any pharmacy specific articles on self-efficacy and self-esteem in relation to
students early professional experiences. Several articles
addressed the benet of a cooperative education model
where students received extended practical experiences
on their self-efcacy and self-esteem.8-11 A few other
INTRODUCTION
The personal transformation that pharmacy students
undergo as they complete their academic career and enter
their professional lives is arguably one of the most significant and difficult transitions they will experience. Ideally, upon entering practice, a graduates self-efcacy and
self-esteem should be high, allowing the new pharmacist
to perform with great condence at a high level of competence.1 Self-efcacy is an individuals belief in their
ability to perform well in a variety of situations.2 People
with a high level of self-efcacy approach difcult tasks
as a challenge to be mastered rather than a threat to be
avoided.3 Self-esteem is a certain attitude and a perception of ones self.4 Although self-esteem is an internal
perception of ones self, it can be affected by external
comparison to peers or role models.
One way for pharmacy students to improve their selfefficacy and self-esteem prior to graduation is to gain experience by working in pharmacy settings. Introductory
and advanced pharmacy practice experiences (IPPEs and
APPEs) are where important learning about the profession
Corresponding Author: Mark L. Yorra, EdD, School of
Pharmacy, Northeastern University, 360 Huntington Avenue,
140 The Fenway, Mailstop 218TF, Boston, MA 02115. Tel:
617-373-3433. Fax: 617-373-7504. E-mail: m.yorra@neu.edu
28

studies found a relationship between students ethnicity
and level of self-efcacy.12-14 This study was conducted
to determine whether students who had spent extended
time in paid and/or unpaid IPPEs, through summer jobs,
part-time jobs, or other work opportunities, had greater
self-efcacy and self-esteem than those who spent less
time in these work settings. The impact of several demographic factors also was examined.
Students from a 0-6 PharmD program may have worked in

a paid position from 400-500 hours (10-12 weeks at
40 hrs/wk) during 2 summer periods and another 200
hours (8 hrs/wk for 25 weeks) during the school year.
Students from the 2-4 PharmD programs had 2 summer
periods to obtain a full-time experience. Only a few could
have worked in a paid part-time position on an occasional
basis because of the rural location of the schools. Students
from the accelerated program had the lowest opportunity
to work part-time because they did not have any summer
breaks. The cooperative education experience was approximately 600 hours during one 4-month work period,
with each student required to complete 2 work periods
with a curricular requirement of 1 work period of 600
hours in a community setting and 1 work period of 600
hours in an institutional setting. These students would
have more than 1200 hours from the 2 work periods.
The majority of the positions were paid. Approval for this
project was granted by the Institutional Review Board at
Northeastern University, Boston, MA.
The author examined the student responses to the
self-efficacy and self-esteem questions, and tested for
significant effects from the independent variables, particularly paid and unpaid pharmacy experiences. For the
purposes of analyses, students were divided into 2 groups
by age, under 25 years and 25 years and over, and 2 groups
by GPA, ,3.0 and $ 3.0. Analysis of Variance was used
to test the hypothesis using SPSS v 21 (IBM). Ethnicity
was examined using chi-square testing to determine differences within the variable.
METHODS
The author developed the Pharmacy Self-efficacy
and Self-esteem Study Questionnaire using the General
Self-efficacy Scale and Rosenberg Self-esteem Scale.
The General Self-efficacy Scale was selected because it
measures the most desirable psychometric properties in
a wide variety of settings.2 The Rosenberg Self-esteem
Scale is a 10-item scale and has been used to measure
general self-esteem in various populations.15-17
In addition to the self-efficacy and self-esteem
questions, the survey collected the following demographic information: date of birth, ethnicity, school
attended, gender, grade point average (GPA), hours of
IPPEs completed, and years of paid pharmacy work experience. One of the research objectives was to explore
how selected demographic variables related to students
self-efcacy and self-esteem in order to conrm or disprove previously reported ndings associating students
gender, ethnicity, GPA, and/or age with their self-efcacy
and self-esteem.
The author selected 5 colleges and schools in the
New England area as a convenience sample, which provided a mix of educational models, including a 3-year
accelerated program, two 6-year private universities,
and two 4-year state universities. Students who had completed their third year of pharmacy school were selected to
participate in the study. Three institutions opted to use
a paper survey instrument, which was distributed by a faculty member during a class and then collected and
returned to the author for processing. Two institutions
requested the electronic version of the survey instrument,
a link to which was e-mailed to P3 students by their deans
ofce. The author sent a reminder e-mail to the students
14 days after the initial request and closed access to the
electronic survey instrument after 60 days.
Students IPPEs were categorized based on number
of hours completed. Paid experiences were categorized
based on time worked as none, ,1 year, 1-2 years, and .2
years of experience. Paid experience included both parttime and full-time experience. In order to keep the unit of
time consistent, the time in years was converted to approximate hours based upon the school a student attended
to determine their availability to work in a paid position.
RESULTS
Three hundred ninety-nine students completed the survey instrument, approximately a 50% response rate based
on the total estimated student population of 820 students at
the end of the P3 year. The reliability of the General Selfefficacy Scale was a50.907 and the Rosenberg Self-esteem
Scale showed a reliability factor a50.888. Students gender
mirrored that of the national pharmacy student population,18
ie, 35% male and 65% female. The ethnic distribution was
4% African American/black, 29.4% Asian/Pacic Islanders,
2.2% Spanish/Hispanic, and 59% white/Caucasian, and
4.7% indicated other or no response. Seventy-three percent
of students were 25 years of age or under and 27% were over
25 years of age. A correlation was performed on the demographic variables in Table 1. Students self-esteem scores
were signicantly correlated with number of IPPE hours
and number of paid practice experiences.
Self-Efficacy
Among students with a GPA,3.0, there were no factors associated with higher self-efcacy. Among students
29

Table 1. Correlation Between Demographic Variables and Experiences
Gender
Pearson Correlation
N
Ethnicity
Pearson Correlation
N
IPPE
Pearson Correlation
N
Paid Experiences
Pearson Correlation
N
YOB by 25
Pearson Correlation
N
Self Esteem Score
Pearson Correlation
N
a
b
Gender
Ethnicity
IPPE
Paid
Experiences
YOB ,25
Self-Esteem
Score
GSE Score
0.007
399
?0.014
362
0.147b
360
?0.037
387
?0.032
388
?0.025
391
0.055
360
0.164b
358
0.125a
385
0.017
386
0.043
389
0.089
345
0.074
352
?0.132a
353
?0.062
354
0.168b
350
0.135a
350
0.114a
352
?0.016
374
?0.032
378
0.656b
380
2 tailed Correlation signicant p,0.05 level

2 tailed Correlation signicant p,0.01 level.
with a GPA$3.0, number of hours of IPPEs was signicantly associated with higher self-efcacy (p,0.05).
There was a significant correlation between mean
scores on the General Self-efficacy Scale items and
a higher number of hours spent in paid or extended pharmacy practice experiences (r50.114, p,0.05). This correlation did not exist for hours spent in unpaid experiences
or IPPE experiences. Ethnicity was examined using a chisquare test with a signicant nding (p,0.001) for the
Asian/Pacic Islander group. Chi-square testing was also
performed with the variables of age, gender, and the
school a student attended, but none of the ndings were
signicant.
chi-square testing (Table 2), resulting in a nding of

higher self-esteem among African American/black students (p5.003) and lower self-esteem among Asian/Pacic Islander students (p,.001).
DISCUSSION
This research examined several variables to determine if any had an effect on the self-efficacy and selfesteem of pharmacy students. The finding that students
with a GPA.3.0 beneted more from extended work experiences than students with a GPA,3.0 can be explained
by the difference between academic self-efcacy and
work-based self-efcacy. Students who do well academically may not have as much work or practical experience
because they are focused on academics and they may
benet more in terms of self-efcacy from practical experience than the average student. Gender showed no
relationship to self-efcacy, although the literature did
Self-Esteem
There was a significant correlation between which
school a student attended and the students level of selfesteem, with students attending a 4-year state school in
Rhode Island having greater self-esteem (r5-0.101,
p,0.05). There was also a signicant correlation between
number of hours of IPPEs a student completed and level
of self-esteem (r5-0.132, p#0.05) with students completing 320 hours having the highest self-esteem. Finally,
there was a correlation between students hours of paid
experiences and self-esteem (r50.135, p#0.05) with students who worked for 1 to 2 years in paid positions having
the highest levels of self-esteem. No signicant correlations were found between gender or age and students
level of self-esteem. Ethnicity was examined using
Table 2. Self-Esteem Scores by Ethnicity
30
Ethnicity
No.
African-American/Black
Spanish, Hispanic, Latino
Asian/Pacic Islander
White/Caucasian
Other
Prefer not to state
Total
16
8
114
230
10
7
385
Mean (SD)
9.1
9.0
8.3
9.0
8.1
7.6
8.8
(1.3)
(1.4)
(1.6)
(1.1)
(2.1)
(1.7)
(1.3)

state that female students had lower self-efcacy than
male students, possibly as a result of their upbringing.
Because much of the literature was published prior to
2000, continued changes in gender roles in the last few
decades may have inuenced the ndings, resulting in
no differences in self-efcacy between genders.19 Age
also was not a factor in self-efcacy as students were
experiencing the profession at the same time regardless
of age.
Levels of self-efficacy and self-esteem were related
to ethnicity. Asian/Pacific Islander students reported having lower self-efficacy and self-esteem than the other
ethnic groups, while African-American students reported
having higher self-esteem than the other ethnic groups. In
a study of college students, African-American students
reported the highest levels of self-esteem compared with
their peer groups, which can be partially attributed to their
internalized self-images in the later stages of their educational careers.20 African-American students who achieved
a college education had higher levels of self-esteem compared to their peers who did not go to college. There is
ample evidence that as an ethnic group, Asians are modest
when responding to surveys assessing self-efcacy.21 This
stems from a cultural tendency to be more moderate in
their private beliefs, values, and preferences than typical
Western students.21 The question the author now faces is
whether the Asian students in the current study in fact had
low levels of self-efcacy compared to other ethnic
groups, or whether their responses understated their actual level of self-efcacy? Choi suggests that responses by
Asians on self-assessments should be taken as genuine
and not adjusted for any perceived modesty bias.21 Although the ndings may not indicate a need for Asian
students to improve their self-efcacy, they may indicate the need for Asian students in the health professions
to better project their self-efcacy, as high levels of selfefcacy and self-esteem are essential to earn the respect
and condence of future employers, colleagues, and
patients.1,22
The reason why students from some colleges and
schools had better self-esteem could be explained by the
attitude promoted by the college or schools faculty members and administration. Also, because the response rates
from 2 institutions were low, the results may not accurately represent the self-esteem of all students at the
school.
Self-esteem fluctuates over time and in various settings.23 Graduates may enter the work setting with a high
level of self-esteem based on their success as a student in
the academic setting, but this may decrease as they begin
to compare themselves to other practitioners.24 As students gain more experience, their self-esteem increases
based upon their knowledge and acquired skills. Students

may not have enough contact time during an IPPE to increase self-esteem substantially without additional contact hours through paid or unpaid experiences.
There were 2 main limitations in this study. The first
was the use of the General Self-efficacy Scale and Rosenbergs Self-esteem Scale. Both scales are valid and reliable, but they need to include more specic questions
pertaining to work-based self-efcacy. Joseph Raelin,
a researcher on experiential education, suggested the
use of academic, career, and work self-efcacy tools to
examine a student population (personal communication
April 20, 2012). The use of a more specic scale might
have yielded different ndings and probably would have
strengthened the results of the study.
The second limitation was the use of a regional sample of schools from the Northeast United States, which
may not have been representative of a national sample,
limiting the ability to generalize the findings. The individual response rates from 2 of the schools were less
than 25, which made any correlations from these schools
unreliable.
Other limitations were that the responses to the survey were self-reported by the students and not corroborated by external validation. Nonresponders, particularly
students attending the public universities, may have affected the results by the lack of full representation. There
was not a survey item about students socioeconomic status, which may have potentially affected the results based
on the type of students who attend a public university vs
a private institution. Students who had a greater nancial
need may have worked more to earn money for school
than did more afuent students. Finally, students who
were higher achievers may have been more likely to complete the survey instrument, thus introducing a selection
bias to the survey. The researcher did not test for response
bias.
CONCLUSION
The objective of this study was to determine if there
is a relationship between experiential education/work experience and self-efficacy and self-esteem, based upon
the desirability to have new graduates entering the workforce possess these characteristics. Though schools provide 300 hours of practical experience through IPPEs, the
author found that additional hours would enable students
to achieve higher levels of self-efficacy and self-esteem.
Whether IPPEs are expanded or a new requirement is
implemented for students to obtain experiences outside
of the IPPE program, the additional experience would
provide important benefits to a students development
of self-efcacy and self-esteem.
31

REFERENCES
13. Jaret C, Reitzes DC. Currents in a Stream: College Student

Identities and Ethnic Identities and Their Relationship with SelfEsteem, Efcacy, and Grade Point Average in an Urban University.
Social Science Quarterly 2009;90:345-67.
14. Vuong M, Brown-Welty S, Tracz S. The Effects of Self-Efcacy
on Academic Success of First-Generation College Sophomore
Students. Journal of College Student Development 2010;51:50-64.
15. Rosenberg M. Society and the adolescent self-image: Princeton,
N.J., Princeton University Press; 1965.
16. Classen S, Velozo CA, Mann WC. Rosenberg Self-Esteem Scale
as a measure of self-esteem for the noninstitutionalized elderly.
Clinical Gerontologist 2007;31:77-93.
17. Dahlbeck DT, Lightsey OR, Jr. Generalized self-efcacy,
coping, and self-esteem as predictors of psychological adjustment
among children with disabilities or chronic illnesses. Childrens
Health Care 2008;37:293-315.
18. American Association of Colleges of Pharmacy. Academic
Pharmacys Vital Statisitcs. Washington DC; 2011.
19. Bandura A, Barbaranelli C. Multifaceted impact of self-efcacy
beliefs on academic functioning. Child Development 1996;67:1206-22.
20. Elion AA, Wang KT, Slaney RB, French BH. Perfectionism in
African American students: Relationship to racial identity, GPA, selfesteem, and depression. Cultural Diversity and Ethnic Minority
Psychology 2012.
21. Choi I, Choi Y. Culture and Self-Concept Flexibility. Personality
and Social Psychology Bulletin 2002;28:1508-17.
22. Freudenberg B, Cameron C, Brimble M. The Importance of Self:
Developing Students Self Efcacy Through Work Integrated
Learning. International Journal of Learning 2010;17:479-96.
23. Ferris DL, Lian H, Brown DJ, Pang FXJ, Keeping LM. Selfesteem and job performance: the moderating role of self-esteem
contingencies. Personnel Psychology 2010;63:561-93.
24. Crocker J, Brook AT, Niiya Y, Villacorta M. The Pursuit of SelfEsteem: Contingencies of Self-Worth and Self-Regulation. Journal of
Personality 2006;74:1749-71.
1. Bandura A. Self-efcacy: the exercise of control. New York: W.H.

Freeman; 1997.
2. Scherbaum CA, Cohen-Charash Y, Kern MJ. Measuring General
Self-Efcacy. Educational and Psychological Measurement
2006;66:1047-63.
3. Bandura A. Regulation of cognitive processes through perceived
self-efcacy. Developmental Psychology 1989;25:729-35.
4. Mruk CJ. Self-esteem research, theory, and practice: toward
a positive psychology of self-esteem. New York: Springer; 2006.
5. DiPiro JT. Making the most of pharmacy school. Am J of Pharm
Educ 2008;72(1):Article15.
6. American Association of Colleges of Pharmacy. Accreditation
Standards and Guidelines for the Professional Program in Pharmacy
Leading to the Doctor of Pharmacy Degree. Washington D.C.;
2011.
7. American Association of Colleges of Pharmacy. Accreditation
Leading to the Doctor of Pharmacy Degree 2011:92.
8. Abrahamsson K. Co-operative education, experiential learning
and personal knowledge. In: International Conference on Higher
Education; 1981; Lancaster, England; 1981.
9. Billett S. Learning Through Practice: Models, Traditions,
Orientations and Approaches. Professional and Practice-Based
Learning 2010;1.
10. Blair BF, Millea M, Hammer J. The Impact of Cooperative
Education on Academic Performance and Compensation of
Engineering Majors. J of Engineering Educ 2004;93:333-8.
11. Yin A. Learning on the job: Cooperative education, internships
and engineering problem-solving skills. Philadelphia: The
Pennsylvania State University; 2009.
12. Concannon JP, Barrow LH. A Cross-Sectional Study of
Engineering Students Self-Efcacy by Gender, Ethnicity, Year, and
Transfer Status. Journal of Science Education & Technology
2009;18:163-72.
32
RESEARCH
Investigating the Relationship Between Pharmacy Students
Achievement Goal Orientations and Preferred Teacher Qualities
Saleh Alrakaf, MSc Pharm,a Erica Sainsbury, PhD,a Grenville Rose, PhD,b and
Lorraine Smith, PhDa
a
Faculty of Pharmacy, The University of Sydney, Sydney, New South Wales, Australia
Innovation and Evaluation, Aftercare, Sydney, New South Wales, Australia
Submitted December 15, 2013; accepted March 17, 2014; published September 15, 2014.
Objective. To investigate the relationships between pharmacy students preferred teacher qualities and
their academic achievement goal orientations.
Methods. Participants completed an achievement goal questionnaire and a build-a-teacher task. For the
latter, students were given a $20 hypothetical budget to purchase amounts of 9 widely valued teachers
qualities.
Results. Three hundred sixty-six students participated. Students spent most of their budget on the traits of
enthusiasm, expertise, and clear presentation style, and the least amount of money on interactive teaching, reasonable workload, warm personality, and intellectually challenging. In relation to achievement
goals, negative associations were found between avoidance goals and preferences for teachers who
encourage rigorous thinking and self-direction.
Conclusion. These novel ndings provide a richer prole of the ways students respond to their learning
environment. Understanding the relationships between teachers characteristics and pharmacy students
achievement goal orientations will contribute to improving the quality of pharmacy learning and teaching
environments.
Keywords: achievement goals, motivation, pharmacy education, teacher qualities, student preferences
as performing well compared to other students and they

dene success vs failure based on teacher-referential
standards.8,11
Elliot and McGregor have proposed that mastery and
performance goals can be further divided into approach and
avoidance components.12 Students who adopt a masteryapproach goal aim to learn and understand the course
materials as thoroughly as possible, whereas those who
are oriented towards the mastery-avoidance goal aim to
avoid not understanding the course materials thoroughly.
Students adopting the performance-approach goal are
motivated to outperform other students or to demonstrate
their ability to either teachers or peers, whereas students
adopting the performance-avoidance goal aim to avoid doing worse than other students or appearing less talented.
This distinction is supported by a large body of empirical
research and is robust in predicting and understanding students engagement and achievement.13-15
These goal orientations are differentially associated
with a range of motivation, academic, and psychological
correlates. Avoidance goals are associated with negative
outcomes, for example, performance-avoidance and masteryavoidance goals have been linked to depression16 and low
INTRODUCTION
Most faculties (schools of pharmacy) seek their students opinions regarding teaching and instructor qualities, and it is not uncommon to nd that students rate the
same instructor differently. However, it is unclear why
different students rate an instructor differently. The goals
that university students adopt in class may be the answer
to this question,1 specically achievement goals, which
theorists believe play a major role in education.2,3
According to achievement goal theorists, students
engage in educational activities with 1 of 2 broad goals
in mind: mastery goals or performance goals.4-6 For either
goal, gaining competence is the students primary aim.7
However, they perceive competence in different ways.
Mastery-oriented students view competence as learning and
understanding the task thoroughly and use self-referential
standards to dene success vs failure.8-10 On the other
hand, performance-oriented students view competence
Corresponding Author: Saleh Alrakaf, Faculty of Pharmacy,
Room S114, Pharmacy and Bank Building A15, The
University of Sydney, NSW, 2006. Tel: 161-2-9351-4501.
Fax: 161-2-9351-4451. E-mail: salr4982@uni.sydney.edu.au
33

grades on examinations.17,18 In contrast, the masteryapproach goal has been associated with deep learning,19
high individual interest,20 high self-regulation,21 and
willingness to cooperate.22 Yet, to the achievement theorists surprise, the mastery-approach goal can rarely predict high academic achievement (ie, grades).13 The
performance-approach goal, however, is associated with
high grades on examinations,24,25 but also with surface
learning approaches such as memorization.23
To find an explanation for the unexpected relationship
between the approach types of achievement goals and
academic achievement, Senko and his colleagues hypothesized that each type of goal affects students learning
strategies differently.11 According to the authors, students
who adopt the mastery-approach goal tend to study materials that are interesting to them regardless of the subject
matters importance or testability, while students who are
performance-approach oriented do not. The latter will
study what they think will appear on the examination and
try to gure out what is important to their teachers instead
of following their own interests. As a result, they gain
higher grades than their mastery-approach peers.26
The quality of higher education largely depends on
the qualities of teachers in this sector.27 Determining
which qualities are considered essential and effective
can be difcult to dene as stakeholders in higher education (eg, students, teachers, administrators, and scholars)
have individual views and opinions about what these qualities are.28 However, they all believe that teacher qualities
have a great impact not only on students education but
also on students futures as well.29 One area that teacher
qualities play a major role in is students achievement
goals.30,31 A recent study conducted by Shim and colleagues
found that teachers who strongly pursue mastery goals tend
to foster the adoption of mastery goals by their students,
while teachers who strongly pursue performance goals tend
to foster the adoption of those goals by their students.32
Although such impacts are well documented, little is known
about how students achievement goals might inuence their
preferences for teachers qualities.33
Senko and colleagues found that mastery-approach
and performance-approach goals adopted by students
did affect the qualities and traits that students wanted to
see in their teachers.33 Students who adopted a masteryapproach goal most valued teachers who challenged them
intellectually and had an extensive experience in their subject areas. In contrast, students who adopted a performanceapproach goal valued teachers who provided suggestions
about how to gain high marks on examinations and who
presented their material clearly. Valuing these qualities
did not suggest that these students did not like other qualities such as warmth and enthusiasm. It simply meant that
students considered the latter qualities less important and

not necessities. These so-called luxury qualities were
desirable only after obtaining the essential ones. While
Senko and colleagues tested the effect of both mastery and
performance approach types, we believe that investigating the impact of mastery and performance avoidance
types and their relationship to teacher qualities is also
important. Given that avoidance types are maladaptive
and unproductive, knowing the preferred teacher qualities
of students who strongly adopt them is benecial in order
to review teaching methods that might foster adoption of
these goals.
Thus, our study had 3 aims: to investigate which
qualities pharmacy students most preferred in their
teachers; to test assumptions about how masteryapproach and performance-approach goals affect students
preferences for various teacher qualities in a pharmacy education setting;33 and to investigate the effects of avoidance
types of achievement goals (mastery avoidance and performance avoidance) on teacher qualities. To our knowledge,
no study has assessed the effects of the 4 types of achievement goals on students preferences of teacher qualities.
METHODS
The participants for this study were second-year and
fourth-year undergraduate pharmacy students enrolled in
a bachelors degree program at The University of Sydneys
Faculty of Pharmacy. Completion of this 4-year program
enables graduates to register as a pharmacist in Australia.34
The survey comprised 2 measures: the Achievement
Goal Questionnaire12 (AGQ) and the build-a-teacher task.33
Both instruments are available from the corresponding author. In addition to these measures, gender and age were
included as socio-demographic indicators in the survey.
The AGQ is a validated and psychometrically robust
instrument35 intended to measure the 4 types of student
achievement goals and contained 12 items. Students rated
each item on a 1 to 7 scale (15not at all true of me, 75very
true of me).
The build-a-teacher task is a validated and commonly used instrument for measuring teacher qualities.33
It contains a list of 9 widely valued teacher qualities. The
task required students to design their ideal teacher by
buying qualities with a limited budget. The purchasing
scale ranged from $0 to $10. This method encouraged
students to carefully consider their choices as the more
they spent on one quality the less money was left to spend
on other qualities.33,36
The study was initiated in the first semester of the
academic year. Students were invited to participate in the
study during normal lectures or tutorials (small group
discussions). They were advised that participation was
34

voluntary, and if they chose to participate, they could
withdraw from the study at any time. In addition, students
were advised that their decision to participate would not
impact on their academic performance results or influence
student-teacher relationships. Researchers approached students as a group and not individually. The first author
administered the survey instrument.
Students completed the survey in paper form. For the
build-a-teacher task, students were given a hypothetical
$20 budget to purchase the 9 teachers qualities. The written instructions explained that the maximum amount of
money students could spend on any 1 quality was $10.
Students were asked to spend their full budget in a way
that reected their preferences. Completing the task took
approximately 20 minutes.
SPSS 21 (SPSS Inc, Chicago, Illinois) was used for
all statistical analyses. Descriptive statistics regarding
gender and age were reported. A split-plot ANOVA design (SPANOVA), with academic year as the betweensubjects factor and teacher qualities as the within-subjects
factor, was used to investigate the impact of academic
year upon student preferences for teacher qualities and
to compare student responses to the 9 different teacher
qualities.37 If the sphericity assumption was violated, the
Huynh-Feldt degrees of freedom were reported. Bonferroni
correction was performed as needed. A multiple regression analysis procedure was performed to assess the effect
of each achievement goal type on student spending on
teacher qualities.
Conduct of this study was approved by the Human
Research Ethics Committee, The University of Sydney.
the least essential (Table 1). The effect comparing the

2 academic years was not signicant (p50.23), suggesting no difference between the 2 academic years.
Bonferroni pairwise comparisons were performed and
the variables were placed in groups where there were no
signicant differences. No signicant differences were
found among the qualities of enthusiasm, topic expertise and clear presentation style. However, these qualities did signicantly differ in mean scores from other
teacher qualities such as good feedback, intellectually
challenging, warm/compassionate personality, reasonable workload, and interactive teaching style.
Multiple regression was performed to assess the impact
of the different types of achievement goals on the 9 teacher
qualities. The model contained 4 independent variables
(performance-approach, mastery-approach, performanceavoidance and mastery-avoidance goals). The relationships
between students achievement goals and the teacher
qualities they preferred were determined by any signicant relationship between a goal and the money spent on
a teacher quality.
As shown in Table 2, the more students pursued
mastery-avoidance goals, the less they spent on the
teacher quality of enthusiasm (p50.03). Furthermore,
the more students pursued performance-avoidance goals,
the less they wanted their teacher to challenge them intellectually (p50.01). In addition, the more students pursued performance-approach goals, the less they spent on
the quality of warm/compassionate personality (p50.01).
DISCUSSION
This study tried to answer 3 important questions:
What teacher qualities do students most prefer? To
what extent do mastery- and performance-approach
goals influence student preferences for teacher qualities? To what extent do mastery- and performance-
RESULTS
Three hundred sixty-six students (235 female, 128
male, and 3 who preferred not to reveal their gender)
participated in this study. The mean age of the students
was 21.3 years (standard deviation52.7 years). The survey yielded a response rate of 73.2%.
The Mauchly test indicated that the assumption of
sphericity had been violated (p,0.05), therefore, degrees of freedom were corrected using Huynh-Feldt estimates of sphericity. A SPANOVA test revealed no
signicant impact of academic year on student preferences
for teacher qualities (p50.66). However, there were signicant differences between teacher qualities that students
prefer in that the test showed students prioritized some
qualities over others (p,0.01). Students most preferred
quality was enthusiasm/entertaining (mean6SD,
3.162.2), followed closely by topic expertise, clear presentation style, and clarity about how to succeed. They
considered reasonable workload (mean6SD, 1.661.6)
and interactive teaching style (mean6SD, 1.661.5)
Table 1. Pharmacy Students Ratings of Desirable Teacher

Qualities
Teachers Qualities
Mean (SD)
Enthusiastic/entertaining
Topic expertise
Clear presentation style
Clear about how to succeed
Good feedback
Intellectually challenging
Warm/compassionate personality
Reasonable workload
Interactive teaching style
3.1a (2.2)
3.1a (2.0)
2.8a,b (2.2)
2.4b (2.0)
1.9c (1.7)
1.8c (1.7)
1.7c (1.7)
1.6c (1.6)
1.6c (1.5)
Note: Qualities that do not share the same superscript are signicantly
different using Bonferroni correction at the 0.05 level.
35

both cross-sectionally and longitudinally that pharmacy
students preferred to learn through dependence on
teacher-sourced strategies rather than self-sourced strategies, and that deep processing and critical thinking were not
routinely favored by students.34,42
The low ranking that the quality of having an interactive teaching style received may have resulted from the
introduction of the online-recorded lecture system, which
enables academics to record lectures and make them
available to students electronically. Although all other
pharmacy classes (workshops, tutorials, and laboratories)
are face to face, no attendance is required at recorded
lectures. Thus, students may have felt that having
a teacher with an interactive teaching style was not as
essential as in the past. The ability to use Internet sites
such as YouTube as a source for information may also
explain why students considered interactive teaching
style the least essential teacher quality. The use of Internet
technology is a defining feature of this generation of students, because they are the first generation to have had the
Internet as a part of their lives from birth.43,44
Our findings in relation to our first aim supported
those that Senko and colleagues found in their study,33
yet regarding the second aim, our results were quite different. In contrast to Senko and colleagues results, the
only signicant relationship we found was a negative one
between performance-approach goals and buying the
teacher quality of a warm personality. Students who more
strongly pursued performance-approach goals were less
likely to prefer a warm and compassionate teacher. This
result might be attributed to the competitive nature of
performance-approach-oriented students who tend to afrm their competence by outperforming their peers. Evidence suggests that warm and compassionate teachers
may be willing to take into account the circumstances
of struggling students and give preferential treatment with
respect to grades.45
Our study expanded upon previous research by examining the impact of mastery- and performance-avoidance
goals,33 showing that they had signicant negative relationships with the enthusiasm and intellectually challenging teacher qualities, respectively. This indicated that the
more strongly students adopted mastery- and performanceavoidance goals, the less necessary it was that their
teachers be enthusiastic or challenge them intellectually.
These ndings could be attributed to the specic motivational characteristics of students who adopted the avoidance types of goals. Fear of facing shame, being
embarrassed, and/or being criticized by teachers have
been highly linked to students who pursue these goals.46
The aim of students who adopt the mastery-avoidance
goal is to avoid not understanding the course materials
Table 2. Regression Analyses (p-value) of Goal Type with

Teacher Quality
Teachers Qualities
Enthusiastic/entertaining
Intellectually challenging
Topic expertise
Clear about how to succeed
Clear presentation style
Reasonable workload
Interactive teaching style
Warm/compassionate
personality
Good feedback
M-APa M-AVb P-APc P-AVd

0.16
0.46
0.06
0.76
0.07
0.19
0.37
0.73
0.03
0.28
0.96
0.76
0.39
0.05
0.18
0.36
0.14
0.76
0.27
0.09
0.96
0.76
0.83
0.01
0.06
0.01
0.53
0.06
0.63
0.08
0.20
0.98
0.21
0.71
0.18
0.53
The overall model is signicant at p,0.05 according to ANOVA

results.
a
M-AP: Mastery-approach goal
b
M-AV: Mastery-avoidance goal
c
P-AP: Performance-approach goal
d
P-AV: Performance-avoidance goal
avoidance goals influence student preferences for teacher

qualities? To answer all of these questions precisely,
we used a budget methodology specifically designed
to differentiate between essential and nonessential
teacher qualities33 and a validated measure of achievement motivation.35
That the enthusiasm quality emerged as one of the
most preferred teacher qualities was not a surprise to us. A
qualitative study conducted by Alrakaf and colleagues to
investigate undergraduate pharmacy students preferences for teaching indicated, without prompting, that students highly value this quality.38 Interestingly, the bottom
ranked quality was interactive teaching style, which is
viewed by many scholars as highly valued by students
and benecial in terms of academic achievements.39-41
A closer look at the teacher qualities students preferred revealed that on the whole, the highly valued qualities were those that reflected teacher engagement with the
learning process where the emphasis was on the level of
teacher commitment to the task of optimizing student
learning and achievement. The least-valued qualities, on
the other hand, were those that reflected student engagement with the learning process, where the emphasis was on
student commitment to optimizing their own learning and
achievement. Take for example, the low ranking for the
teacher quality intellectually challenging. This quality
requires student commitment to learning and an ability to
perform self-directed learning tasks. These results were
supported by the ndings of our previous work regarding
our pharmacy students approaches to learning, in which
students demonstrated being dependent upon and valuing
external sources of support and found self-directed learning approaches challenging. Our previous research showed
36

thoroughly, so a teacher who uses humor and anecdotes
might be seen as a distraction from this effort. Also, an
intellectually challenging teacher may inadvertently create an intimidating environment for students who pursue
a performance-avoidance goal as these students tend to be
afraid of being criticized and appearing untalented in
front of the teacher and their peers. Furthermore, students
who adopt either type of avoidance goals perceive challenging activities as a threat to their self-esteem.47
Using a pharmacy cohort from only one institution is
a limitation for this study. However, the faculty of pharmacy where the study was conducted is the only school in
Sydney that offers a bachelors degree in Pharmacy. In
order to generalize these results, a national study of Australian pharmacy students would be a good next step, as
well as a multinational study on pharmacy students. The
strengths of the study are that we used 2 validated measuring instruments and a unique and engaging method of determining student preferences for teacher qualities.
7. Elliot AJ, Dweck CS. Competence and motivation: competence as

the core of achievement motivition. In: Elliot AJ, Dweck CS, eds.
Handbook of Competence and Motivation. New York, NY: The
Guiforsd Publications Inc.; 2005:3-12.
8. Murayama K, Elliot AJ, Friedman R. Achievement goals. In: Ryan
RM, ed. The Oxford Handbook of Human Motivation. New York,
NY: Oxford University Press; US; 2012:191-207.
9. Remedios R, Kiseleva Z, Elliott J. Goal orientations in Russian
university students: from mastery to performance? Educ Psychol.
2008;28(6):677-691.
10. Senko C, Harackiewicz JM. Regulation of achievement goals:
the role of competence feedback. J Educ Psychol. 2005;97(3):320.
11. Senko C, Miles KM. Pursuing their own learning agenda: how
mastery-oriented students jeopardize their class performance.
Contemp Educ Psychol. 2008;33(4):561-583.
12. Elliot AJ, McGregor H. A 2x2 achievement goal framwork.
J Pers Soc Psychol. 2001;80(3):501-519.
13. Hulleman CS, Schrager SM, Bodmann SM, Harackiewicz JM. A
meta-analytic review of achievement goal measures: different labels
for the same constructs or different constructs with similar labels?
Psychol Bull. 2010;136(3):422-449.
14. Huang C. Discriminant and criterion-related validity of
achievement goals in predicting academic achievement: a metaanalysis. J Educ Psychol. 2012;104(1):48-73.
15. Shim SS, Cho Y, Wang C. Classroom goal structures, social
achievement goals, and adjustment in middle school. Learn Instr.
2013;23:69-77.
16. Akin A. Self-efcacy, achievement goals, and depression,
anxiety, and stress: a structural equation modelling. World Appl Sci
J. 2008;3(5):725-732.
17. Murayama K, Elliot AJ. The competition-performance relation:
a meta-analytic review and test of the opposing processes model of
competition and performance. Psychol Bull. 2012;138(6):1035-1070.
18. Van Yperen NW, Elliot AJ, Anseel F. The inuence of masteryavoidance goals on performance improvement. Eur J Soc Psychol.
2009;39(6):932-943.
19. Diseth A. Self-efcacy, goal orientations and learning strategies
as mediators between preceding and subsequent academic
achievement. Learn Individ Differ. 2011;21(2):191-195.
20. Harackiewicz JM, Barron KE, Pintrich PR, Elliot AJ, Thrash
TM. Revision of achievement goal theory: necessary and
illuminating. J Educ Psychol. 2002;94(3):638-645.
21. Wolters CA. Self-regulated learning and college students
regulation of motivation. J Educ Psychol. 1998;90(2):224.
22. Harris A, Yuill N, Luckin R. The inuence of context-specic
and dispositional achievement goals on childrens paired
collaborative interaction. Br J Educ Psychol. 2008;78(3):355374.
23. Elliot AJ, McGregor H, Gable S. Achievement goals, study
strategies, and exam performance: a mediational analysis. J Educ
Psychol. 1999;91(3):549-563.
24. Harackiewicz JM, Barron KE, Tauer JM, Elliot AJ. Predicting
success in college: a longitudinal study of achievement goals and
ability measures as predictors of interest and performance from
freshman year through graduation. J Educ Psychol. 2002;94
(3):562-575.
25. Barron KE, Harackiewicz JM. Revisiting the benets of
performance-approach goals in the college classroom: Exploring the
role of goals in advanced college courses. Int J Educ Res. 2003;39
(4-5):357-374.
CONCLUSION
Pharmacy students value a range of teacher qualities
that are stimulating and promote achievement rather than
deep thinking. Students engagement with learning is
characterized by a preference for teacher-focused strategies rather than self-focused strategies. In keeping with
this approach to learning, students who adopt avoidancetype achievement goals value least of all those teacher
qualities that promote self-directed learning. These ndings highlight the nexus between teaching and learning
and can be used in the development of learning, teaching,
and assessment strategies that optimize topic mastery,
critical thinking, and academic achievement.
REFERENCES
1. Murphy KR, Cleveland JN, Skattebo AL, Kinney TB. Raters who
pursue different goals give different ratings. J Appl Psychol. 2004;
89(1):158-164.
2. Hulleman CS, Senko C. Up round the bend: forcasts for
achievement goal theory and research in 2020. In: Urdan TC,
Karabenick SA, eds. The Decade Ahead: Theoritical Perspectives on
Motivation and Achievement. Vol 16A. Bingley, UK: Emerald Group
Publishing Limited; 2010:71-104.
3. Wolters CA. Advancing achievement goal theory: using goal
structures and goal orientations to predict students motivation,
cognition, and achievement. J Educ Psychol. 2004;96(2):236-250.
4. Ames C. Classrooms: goals, structures, and students motivation.
J Educ Psychol. 1992;84(3):261-271.
5. Pintrich PR. An achievement goal theory perspective on issues in
motivation terminology, theory, and research. Contemp Educ
Psychol. 2000;25(1):92-104.
6. Nicholls JG. Achievement motivation: Conceptions of ability,
subjective experience, task choice, and performance. Psychol Rev.
1984;91(3):328-346.
37

26. Senko C, Hulleman CS, Harackiewicz JM. Achievement goal
theory at the crossroads: old controversies, current challenges, and
new directions. Educ Psychol. 2011;46(1):26-47.
27. Hill Y, Lomas L, MacGregor J. Students perceptions of quality
in higher education. Qual Assur Educ. 2003;11(1):15-20.
28. Voss R, Gruber T. The desired teaching qualities of lecturers in
higher education: a means end analysis. Qual Assur Educ. 2006;
14(3):217-242.
29. Stronge JH. Qualities of Effective Teachers. 2nd edition.
Alexandria, VA: ASCD Publications; 2007.
30. Murayama K, Elliot AJ. The joint inuence of personal
achievement goals and classroom goal structures on achievementrelevant outcomes. J Educ Psychol. 2009;101(2):432-447.
31. Wolters CA, Daugherty SG. Goal structures and teachers sense
of efcacy: their relation and association to teaching experience and
academic level. J Educ Psychol. 2007;99(1):181-193.
32. Shim SS, Cho Y, Cassady J. Goal structures: the role of teachers
achievement goals and theories of intelligence. J Exp Educ. 2013;81
(1):84-104.
33. Senko C, Belmonte K, Yakhkind A. How students achievement
goals shape their beliefs about effective teaching: a build-aprofessor study. Br J Educ Psychol. 2012;82(3):420-435.
34. Smith L, Saini B, Krass I, Chen T, Bosnic-Anticevich S,
Sainsbury E. Pharmacy students approaches to learning in an
Australian university. Am J Pharm Educ. 2007;71(6):Article 120.
35. Alrakaf S, Abdelmageed A, Kiersma M, et al. An international
validation study of two student achievement goal questionnaires.
Australasian Pharmaceutical Science Association Conference.
Dunedin, New Zealand: APSA; 2013.
36. Li NP, Bailey JM, Kenrick DT, Linsenmeier JA. The necessities
and luxuries of mate preferences: testing the tradeoffs. J Pers Soc
Psychol. 2002;82(6):947.
37. Pallant J. SPSS SURVIVAL MANUAL: A Step by Step Guide to

Data Analysis Using SPSS. 4th edition. Crows Nest, Australia: Allen
& Unwin; 2011.
38. Alrakaf S, Sainsbury E, Smith L. First year undergraduate
pharmacy students and academics views of and preferences for
learning and teaching. A preliminary investigation. Res J Pharm
Technol. In press.
39. Reynolds D, Farrell S. Worlds Apart?: A Review of International
Surveys Of Educational Achievement Involving England. London,
England: Stationery Ofce Books; 1996.
40. Costa ML, van Rensburg L, Rushton N. Does teaching style
matter? A randomised trial of group discussion versus lectures
in orthopaedic undergraduate teaching. Med Educ. 2007;
41(2):214-217.
41. Knight JK, Wood WB. Teaching more by lecturing less. Cell
Biol Educ. 2005;4(4):298-310.
42. Smith L, Krass I, Sainsbury E, Rose G. Pharmacy Students
Approaches to Learning in Undergraduate and Graduate Entry
Programs. Am J Pharm Educ. 2010;74(6):Article 106.
43. Hughes A. Higher education in a web 2.0 world, Joint
Information Systems Committee Report. Jisc. http://www.jisc.ac.uk/
publications/generalpublications/2009/heweb2.aspx#downloads.
Accessed February 6, 2012.
44. Doiron R, Asselin M. Exploring a new learning landscape in
tertiary education. New Libr World. 2011;112(5/6):222-235.
45. Crocker J, Olivier M-A, Nuer N. Self-image goals
and compassionate goals: costs and benets. Self Identity. 2009;
8(2-3):251-269.
46. Conroy DE, Elliot AJ. Fear of failure and achievement goals in
sport: Addressing the issue of the chicken and the egg. Anxiety Stress
Coping. 2004;17(3):271-285.
47. Smith L. The Role of Affective Distress in Final Year High
School Student Achievement Motivation. [Doctorate thesis]. Sydney:
The University of Sydney; 2003.
38

Long-term Effectiveness of Online Anaphylaxis Education for Pharmacists
Sandra M Salter, BPharm,a Sandra Vale, BSc,b Frank M Sanlippo, PhD,c
Richard Loh, MBBS,*b,d and Rhonda M Clifford, PhDa
a
School of Medicine and Pharmacology, The University of Western Australia, Crawley, Western Australia
The Australasian Society of Clinical Allergy and Immunology, Sydney, New South Wales, Australia
c
School of Population Health, The University of Western Australia, Crawley, Western Australia
d
School of Paediatrics and Child Health, The University of Western Australia, Crawley, Western Australia
*
Bachelor of Medicine/Bachelor of Surgery
b
Submitted December 16, 2013; accepted February 26, 2014; published September 15, 2014.
Objective. To evaluate the long-term effectiveness of an Australasian Society of Clinical Immunology

and Allergy (ASCIA) anaphylaxis e-learning program compared to lectures or no training.
Design. A controlled interrupted-time-series study of Australian pharmacists and pharmacy students
who completed ASCIA anaphylaxis e-learning or lecture programs was conducted during 2011-2013.
Effectiveness was measured using a validated test administered pretraining, posttraining, and 3 and 7
months after training.
Assessment. All learning groups performed signicantly better on all posttests compared to the pretest,
and compared to a control group (p,0.001). The proportion of e-learners achieving the minimum
standard for anaphylaxis knowledge improved from 45% at pretest to 87% at 7 months.
Conclusion. The ASCIA e-learning program signicantly increased anaphylaxis knowledge. The high
proportion of participants achieving the minimum standard at 7 months indicates long-term knowledge
change.
Keywords: e-learning, knowledge, evaluation, Australasian Society of Clinical Immunology and Allergy,
adrenaline auto-injector.
or to those patients without a prescription when an individual therapeutic need is established by the pharmacist.
In addition, pharmacists sell these devices to Australian
schools and childcare services to facilitate emergency
treatment.20-23 With each distribution, pharmacists
should educate patients (or their agents) about anaphylaxis, conrm they have a device-specic ASCIA Action
Plan for Anaphylaxis, and advise them regarding the correct use and storage of the adrenaline auto-injector.14,20,24
Pharmacists also provide collaborative care (usually with
a family physician or specialist physician) to patients
with comorbid conditions including asthma, offer advice
about and sell medicines for the treatment of allergies, and
are sometimes called upon to provide rst aid for patients
with acute anaphylaxis. Changes to devices in Australia,
including the addition of Anapen in 2010 and the change
of EpiPen to a new-look device in 2011, highlighted the
potential for patient confusion and the importance of upto-date pharmacist advice. Therefore, pharmacists need to
have a thorough knowledge of anaphylaxis as well as
adrenaline auto-injectors.
In 2011, the Australasian Society of Clinical
Immunology and Allergy (ASCIA) launched ASCIA
INTRODUCTION
Anaphylaxis is a severe, progressive allergic reaction that is rapid in onset and may cause death.1 The incidence of anaphylaxis has dramatically increased over
the past decade,2-9 with more cases occurring in the community setting than in the hospital setting.10 Early diagnosis of anaphylaxis and treatment with adrenaline is
essential to prevent fatalities, and deaths are more common in patients with a history of asthma.10-14 Adrenaline
is internationally recognized as the rst-line treatment for
anaphylaxis, with auto-injector devices universally recommended as rst aid for anaphylaxis occurring in the
community setting. Prescriptions for adrenaline autoinjector devices should be accompanied by a devicespecic emergency action plan.11,14-19
In Australia, pharmacists supply adrenaline autoinjectors to patients who present a physicians prescription,
Corresponding author: Sandra Salter, BPharm, School of
Medicine and Pharmacology, The University of Western
Australia, M315, 35 Stirling Highway, Crawley WA 6009,
Australia. Tel: 161 416 003 808. Fax: 161 8 9389 7628.
Email: sandra.salter@uwa.edu.au
39

Anaphylaxis e-training for pharmacists to meet the need
for accurate, consistent anaphylaxis education. This
e-learning package complemented existing ASCIA anaphylaxis e-training programs for schools and childcare
services and other health professionals. The importance
of ensuring that this e-training is effective at increasing
anaphylaxis knowledge is paramount to reducing the risk
of fatal anaphylaxis in the community. Long-term effectiveness is of prime importance because the incidence of
anaphylaxis is increasing and errors in management because of waning knowledge may result in a poor outcome
for the patient.
Effectiveness studies of e-learning in health professionals education indicate e-learning is as effective as
traditional methods at increasing knowledge immediately
after training.25-29 However, there is little evidence to
support the long-term effectiveness of e-learning to enhance knowledge, or to meet a minimum knowledge requirement, such as a minimum pass score. In this study,
we sought to evaluate the immediate and long-term impact of ASCIA Anaphylaxis e-training for pharmacists on
anaphylaxis knowledge, compared to ASCIA anaphylaxis lecture training or no training. We hypothesized that
ASCIA Anaphylaxis e-training for pharmacists would be
as effective as ASCIA anaphylaxis lecture training at increasing short and long-term knowledge, meeting a minimum standard for anaphylaxis knowledge, and teaching
the steps required for adrenaline auto-injector device administration. We also hypothesized that both programs
would be superior to no training.
anaphylaxis e-learning between September 2011 and

May 2012. Lecture participants were recruited while
attending ASCIA anaphylaxis lectures in Perth, Western
Australia, between August and September 2011. As the
e-learning and lecture participants were separated by
both place and time, randomization to either intervention arm was not possible. Control participants were
recruited while attending regular university lectures
and tutorials in Perth, Western Australia, in September
2012. The aims, objectives, relevance of the study, and
option to participate were explained, and all participants
provided written, informed consent prior to enrollment in
the study (e-learning participants gave consent by selecting an I Agree checkbox online). Participants also
completed a short demographic survey, which included
the variables gender, age group, main job in pharmacy,
type of control student, postal code of main workplace,
and graduation year.
ASCIA Anaphylaxis Training for Pharmacists
The training program was developed by ASCIA in
consultation with the Pharmaceutical Society of Western
Australia, the Pharmaceutical Society of Australia, the
Pharmacy Guild of Australia, and the Society of Hospital
Pharmacists of Australia. The training was advertised as an
accredited continuing professional development (CPD)
activity with these organizations, as well as through professional newsletters, magazines, and websites. Table 1
provides an overview of the training. Briey, e-learning
and face-to-face lecture programs consisted of the same 4
modules, each designed to take 15 minutes to complete.
E-learning was presented as a series of slides using Metamorphosis software (Easy Authoring, Sydney, Australia).
Face-to-face lectures were delivered as Microsoft PowerPoint slides.
E-learning participants were allowed to complete
training at their own pace, although it was recommended
that all modules and tests be completed within a 2-week
period. Explanatory notes for slides accompanied the
e-learning program to ensure equivalence with spoken
material presented in face-to-face lectures. Participants
were encouraged to obtain their own trainer adrenaline
auto-injector devices and practice the steps required for
their administration while completing the program.
Lecture participants attended one of three 1-hour,
face-to-face lectures. To ensure consistency across lectures, a dedicated ASCIA-approved lecturer (a clinical
immunology/allergy medical specialist) delivered all lectures in the study. Participants were provided with trainer
adrenaline auto-injector devices for the duration of the
lecture only, and a hands-on activity was included to
demonstrate the steps required for administration.
DESIGN
This controlled, interrupted time-series study was
conducted in Australia between August 2011 and April
2013. The University of Western Australia Human Research Ethics Committee gave ethics approval for the
study in July 2011.
Intervention participants were eligible if they were
pharmacists or pharmacy students within Australia.
Pharmacists included professionals registered with the
Pharmacy Board of Australia (PBA) and pharmacy interns who held provisional registration as a pharmacist
with PBA and who were completing practice hours under the direct supervision of a registered pharmacist.
Pharmacy students were individuals enrolled in an approved course of study in the field of pharmacy at an
Australian university. Control participants were students of medicine or pharmacy at the University of
Western Australia.
All participants were recruited using a convenience
approach. E-learning participants were recruited from
across Australia while registering online for ASCIA
40

Table 1. An Overview of ASCIA Anaphylaxis Training for Pharmacists
Aim
To provide ready access to accurate and consistent anaphylaxis education to pharmacists throughout Australia and New Zealand.
Learning objectives
On completion of this program participants should be able to:
d Dene anaphylaxis.
d Identify common causes of anaphylaxis.
d Identify the signs and symptoms of a mild to moderate allergic reaction.
d Identify the signs and symptoms of anaphylaxis.
d Outline the acute management for anaphylaxis.
d Describe the effects of adrenaline on the body.
d List the side effects of adrenaline.
d Explain how to correctly store adrenaline auto-injector devices.
d Differentiate between the EpiPen and Anapen devices.
d Differentiate between junior and adult adrenaline auto-injector devices.
d Demonstrate how to use the EpiPen and Anapen auto-injectors using trainer devices.
d Outline management required after an adrenaline auto-injector has been administered.
d Explain the purpose of the ASCIA Action Plan.
d Identify the most appropriate Action Plan for the patient.
d Identify the roles of the pharmacist in anaphylaxis management.
Program Content
Module 1
What is allergy and anaphylaxis?
Module 2
Acute management of anaphylaxis
Module 3
Adrenaline auto-injectors
Module 4
ASCIA Action Plans and the role of pharmacists in anaphylaxis management
Module 5
Assessment
Program Delivery
E-learning or face-to-face lectures
15 minutes per module (total of 60 minutes of training, plus assessment)
Assessment
Twelve knowledge assessment questions multiple choice, yes/no, and order-the-steps questions.
Abbreviations: ASCIA5 Australasian Society of Clinical Allergy and Immunology.
Completion of a posttest was a requirement for CPD

credits in both programs. Understanding the correct answer is considered part of the learning experience, and elearning participants received brief and immediate online
feedback (as part of the learning program) on their test
results, including the correct answers to questions. Lecture participants were able to access the correct answers
from researchers in the lecture room after completing the
posttest. Neither group received a link to or copy of the
test answers, nor were answers provided at the 3-month or
7-month follow-up tests. For students, the training did not
form part of any university assessment.
Control participants attended a lecture on womens
health, participated in a discussion session on professional pharmacy practice, or completed a pharmacydispensing laboratory session. All control interventions
lasted 60 minutes.
Tool (AT-PAsT), which we developed and validated prior

to use in the study.30 We used a combination of multiplechoice, yes/no, and order-the-steps questions to measure
knowledge of the prevention, identication, and management of anaphylaxis in the community setting. An expert
group of 10 allergy and immunology physicians and 2
clinical pharmacists developed the test questions and
assessed content validity. Modications to wording and
content changes were made to 2 questions. Face validity
was evaluated in a group of 15 pharmacists and 5 pharmacy students, and all agreed they understood the questions and response options. This test was pilot tested on
a group of 67 pharmacists who attended an ASCIA anaphylaxis lecture in Adelaide, South Australia, in July
2011. Although the test demonstrated a signicant improvement in knowledge scores after the lecture
(8.2-11.2 points, paired t test; p,0.001), 4 questions did
not show response change and thus may have overstated
knowledge (McNemar test; p.0.5). These questions were
redeveloped, reviewed by the expert group and pharmacists
EVALUATION AND ASSESSMENT

Knowledge gain was assessed using a 12-question
test, the Anaphylaxis Training Pharmacist Assessment
41

for content and face validity, and incorporated into the nal
version of the AT-PAsT.
The test was administered immediately before
training and immediately after training, then 3 and 7
months after training. To reduce practice effect, the
questions and their response options were reordered on
each test. Participants in the e-learning group completed
the pretest and posttest online as part of the e-learning
program. Participants in the lecture and control groups
completed the pretest and posttest on paper in the lecture
or tutorial room. Pharmacy students completed the
3-month follow-up test on paper. All other tests were
completed through the online research suite Qualtrics
(Qualtrics, Utah). When follow-up tests were due, participants received an e-mail notification and up to 5
e-mail reminders. The follow-up tests remained accessible for 2 weeks. Three prizes (cinema tickets or retail
vouchers), with a maximum value of AU$100, were
provided as an incentive to complete each of the follow-up tests. Of the participants who completed the 3month and 7-month follow-up tests, 1 winner from each
group (e-learning, lecture or control), was drawn at random. There were no other incentives provided in the
study.
As there were no reliable estimates for expected
standard deviation in score, we did not conduct a priori
sample-size calculations. However, a post hoc power
calculation, using the 7-month posttest sample size of
30 in the e-learning group and 50 controls with an observed standard deviation of 1.4 points, showed that
the study had 86% power to detect a difference in score
of 1 point between groups at the 5% level of significance. Calculations for all other sample sizes in the
study groups yielded power estimates between
86% and 100% for between-group and within-group
comparisons.31
the postal code of main workplace to 2 geographic areas,

Western Australia or all other Australian states. Analyses
were restricted to participants who had valid, non-missing
data for all variables in the model.
We compared the proportion of participants within
and between learning groups who, at each test, achieved
the minimum standard for anaphylaxis knowledge
(score$9 out of 12) and correctly ordered the steps for
EpiPen and Anapen device administration. The Pearson
chi-square test was used for between-group comparisons
and the McNemar test was used for within-group comparisons. Data for individual answers to the device-ordering
questions for the e-learning group were not available for
the pretest and posttest (only the overall scores were available). Therefore, we could only make comparisons between the 3-month and 7-month tests in the e-learning
group.
Results
We recruited 383 participants (277 intervention and
106 controls) to the study (Table 2). There was signicant
diversity across all 4 groups based on demographic variables (p,0.001). E-learning and lecture pharmacists
groups were similar by age group and years since graduation, but differed by gender, main job in pharmacy, and
location of main job (Table 2). Completion rates across
the 4 tests ranged from 100% at posttest, to 47.2% at 7
months (Figure 1), and were similar between groups
(p50.91 at 7 months).
Mean knowledge scores were significantly different
by group and test (p,0.001, Table 3). With all demographic variables in the model, there were no signicant
differences in score by age group (p50.28), main job in
pharmacy (p50.06), type of control student (p50.082),
state of main workplace (p50.96), or years since graduation (p50.56). Score initially differed signicantly by
gender (p50.05); however, this effect was lost when
non-signicant variables were removed from the model
(p50.06).
Figure 2 and Table 3 show mean AT-PAsT scores
by group and test. There was a signicant and sustained
improvement in anaphylaxis knowledge after training
in all learning groups (paired t tests, p,0.001 for all
comparisons). Mean scores improved by 3.3, 2.8, and
4.6 points immediately after training in the e-learning,
lecture pharmacists, and lecture pharmacy students
groups, respectively, but decreased in the control group.
Mean scores decreased signicantly from posttest
scores in all learning groups at the 3-month test (a respective score decrease of 1.6, 1.4, 1.7 points). At 7
months, mean scores improved and were above the minimum standard in all learning groups. There were no
Analysis
All analyses were performed using SPSS version 21
(IBM, New York), and reported as 2-sided p-values with
a 5% level of signicance. A linear mixed-effects model
with post hoc pairwise analysis was used to evaluate
changes in short-term and long-term knowledge within
and between learning and control groups. We specied
score as the dependent variable, with group (e-learning,
lecture pharmacists, lecture pharmacy students, or control) and test (pretest, posttest, 3-month and 7-month
tests) as covariates. We compared models with and without demographic covariates (gender, age group, main job
in pharmacy, type of control student, postal code of main
workplace, and years since graduation). As the majority
of the sample was from Western Australia, we converted
42

Table 2. Participant Characteristics by Intervention and Control Group at Pretest (count and %)
E-learning
n=57
Lecture
Pharmacists
n=154
Lecture
Pharmacy
Students n=66
Control
n=106
Total
n=383
Gender
Male
43 (75.4)
41 (26.6)
22 (33.3)
36 (34.0)
142 (37.1)
Age group (years)

18-24
25-34
35-44
45-54
551
7
24
9
11
6
(12.3)
(42.1)
(15.8)
(19.3)
(10.5)
32
51
23
19
27
37
23
4
1
82 (77.4)
18 (17.0)
1 (0.9)
0
0
158
116
37
31
33
(41.2)
(30.3)
(9.7)
(8.1)
(8.6)
Main job in pharmacyc,d

Community pharmacist
Hospital pharmacist
Pharmacy intern
Pharmacy student
Pharmacy academic
27
16
4
3
3
(47.3)
(28.0)
(7.0)
(5.3)
(5.3)
99 (64.3)
15 (9.7)
22 (14.3)
0
9 (5.8)
NA
NA
NA
65 (100)
NA
NA
NA
NA
NA
NA
126
31
26
68
12
(32.9)
(8.1)
(6.8)
(17.8)
(3.1)
NA
NA
NA
NA
66 (62.3)
35 (33.0)
0
(29.5)
(22.1)
(8.7)
(39.6)
65 (98.5)
0
0
0
0
101 (95.2)
0
0
0
0
166
52
50
22
82
(43.3)
(13.6)
(13.0)
(5.7)
(21.4)
(85.9)
(3.6)
(3.4)
(3.1)
(1.0)
(,1)
(,1)
(,1)
Characteristica
Pb
,0.001
,0.001
(20.8)
(33.1)
(14.9)
(12.3)
(17.5)
(56.0)
(34.8)
(6.1)
(1.5)
0
,0.001
Type of control
Medical student
Pharmacy student
NA
NA
Years since graduatione

Not graduated
Less than 5
5-10
11-15
More than 15
8
17
9
23
0
(14.0)
(29.8)
(15.8)
(40.4)
Location of main job

Western Australia
New South Wales
Victoria
Queensland
South Australia
Tasmania
Australian Capital Territory
Northern Territory
9
14
13
12
4
2
2
1
154 (100)
0
0
0
0
0
0
0
65 (98.5)
0
0
0
0
0
0
0
101 (95.2)
0
0
0
0
0
0
0
329
14
13
12
4
2
2
1
Main job by region

Western Australia
Rest of Australia
9 (15.8)
48 (84.2)
154 (100)
0
65 (98.5)
0
101 (95.2)
0
329 (85.9)
48 (12.5)
66 (17.2)
35 (9.1)
,0.001
44
33
13
59
,0.001
Eight participants (2 lecture pharmacists, 1 lecture pharmacy student and 5 controls) did not provide any demographic data.
Pearson chi-square p value for comparison of demographic variables across all 4 groups. Location of main job was compared by region only.
P values for comparison of demographic variables between e-learning and lecture pharmacists groups were: gender: p,0.001; age group: p50.11;
main job: p,0.001; years since graduation: p50.08; main job by region: p,0.001.
c
Of those completing the demographic questionnaire, 7 lecture group pharmacists did not answer the main job question.
d
Of those completing the demographic questionnaire, 4 e-learning participants stated other main job including defence force, industrial, and
compounding pharmacist jobs.
e
3 lecture group pharmacists did not answer the years since graduation question.
NA: not applicable
b
signicant changes in mean score in the control group

after posttest.
Figure 3 shows the change in mean AT-PAsT scores
by group over time. All learning groups performed
signicantly better on all posttests compared to control

(p,0.001 for all comparisons). E-learning and lecture
pharmacist participants had similar scores across all tests
except posttest, where e-learning scores were slightly
43
Figure 1. Study groups, participation and completion rates by group and test.
DISCUSSION
higher (0.65 points, p50.04). Lecture pharmacy students

had the greatest gains in knowledge of all learning groups,
yet lower scores. It was not possible to compare e-learning
scores for pharmacy students with lecture pharmacy students scores, as only 3 pharmacy students completed the
e-learning program.
There were significant and sustained improvements
in the proportion of learners achieving the minimum standard for anaphylaxis knowledge after training (Table 4).
Less than 46% of e-learning and lecture pharmacists
achieved the minimum standard before training; however, 7 months after training, over 80% achieved this
standard. The improvement in the proportion of lecture
pharmacy students achieving the standard was almost tenfold: from 6.7% pretest to 61.8% at 7 months.
Although there were sizeable gains in the proportion of lecture participants who passed the deviceordering questions after training, these gains were not
sustained over time (Table 4). At 7 months, 63.3% of elearning participants and 61.5% of lecture pharmacists
correctly ordered the 4 steps for both EpiPen and
Anapen, an improvement of around 15% in each group
from pretest.
Pharmacists play a vital role in the management of

anaphylaxis patients. Easily accessible, effective anaphylaxis education is essential to fulfil this role.32-35 However, there is little evidence of the effectiveness of
anaphylaxis training for pharmacists. We evaluated the
e-learning program, ASCIA Anaphylaxis e-training for
pharmacists, and measured its effectiveness in terms of
knowledge change.
This education program was associated with significant and sustained improvements in anaphylaxis knowledge. Short-term knowledge gains (on average, a 39%
improvement in mean score) were similar to immediate
gains seen in other pharmacy e-learning effectiveness
studies.36-39 Persistence of knowledge 7 months after
training was high: almost 90% of e-learners achieved at
or above our minimum standard for anaphylaxis knowledge, compared to 45% of the same learners before training. Thus, the results add long-term effectiveness to the
existing body of e-learning pedagogical research,25-29 and
more importantly, demonstrate that this education program is effective long-term. ASCIA Anaphylaxis e-training
for pharmacists was as effective as lecture training,
44

Table 3. Mean Anaphylaxis Training Knowledge Assessment Score by Group and Testa
E-learning
Lecture Pharmacists
Pretest
8.27 (7.80-8.43) n551
8.11 (7.79-8.43) n5153
Posttest
11.53 (11.0-12.0) n552 10.88 (10.56-11.19) n5151
9.50 (9.11-9.89) n589
3-month testc 9.96 (9.0-10.18) n540
7-month testd 10.05 (9.40-10.71) n530 9.66 (9.25-10.06) n578
Lecture Pharmacy Students

5.40
9.96
8.25
9.05
(4.90-5.90) n560
(9.47-10.46) n562
(7.70-8.80) n545
(8.43-9.67) n534
Control
4.57
3.72
3.63
3.68
(4.18-4.99)
(3.34-4.01)
(3.15-4.12)
(3.17-4.18)
n5106
n5106
n555
n550
Reported as estimated marginal mean (95% CI) for each test; maximum test score512. Type III tests of xed effects with mean score as
dependent variable: p,0.001 for group and test.
b
Pairwise comparisons of pretest with posttest, 3-month test, and 7-month test, by group.
c
Pairwise comparison of 3-month test with posttest, p,0.001 for e-learning, lecture pharmacists and lecture pharmacy students; p50.74 for
control.
d
Pairwise comparison of 7-month test with 3-month test; p50.22, 0.51, 0.02, and 0.88 for e-learning, lecture pharmacists, lecture pharmacy
students, and control, respectively.
and signicantly more effective than no training, at improving short-term and long-term anaphylaxis knowledge
in pharmacists. We were unable to demonstrate effectiveness of this e-learning program in pharmacy students due
to low numbers of student participants. Even so, lecture
training was effective at improving short-term and longterm anaphylaxis knowledge in pharmacy students, and
other research has demonstrated short-term effectiveness
of e-learning in pharmacy students in different subject
areas.38,40-42 Therefore, it is likely that this e-learning
program would also be effective for pharmacy students.
There was no change in anaphylaxis knowledge in those
who did not receive training. This is consistent with the
broader literature for short-term e-learning effectiveness.26,29 However, as far as we know, this is the
rst study to demonstrate long-term differences in an
e-learning group compared to a group who did not receive
training.
An essential part of anaphylaxis education for patients is hands-on training in the use of adrenaline autoinjectors. Although pharmacists are ideally placed to
deliver this training, there is evidence that the majority
of anaphylaxis patients do not receive it.24,43,44 People
who do not know how or when to use their adrenaline
auto-injector may elect not to do so in an emergency, or
may incorrectly activate the device.45 Devices and procedures change over time, and there is a constant need to
improve pharmacists skills in this area, so they can better
train those at risk of anaphylaxis.13,35,43,44,46 Approximately two-thirds of e-learners in our study were able to
correctly order all of the steps required for both EpiPen
and Anapen administration 7 months after training. Lecture participants achieved results similar to those for elearners, even though they had hands-on practice with
devices during training. Although long-term device recall
was poorer compared to anaphylaxis knowledge, other
research has shown device recall may wane over
time.47,48 In a group of physician trainees, only one-third
accurately demonstrated devices 6 months after training.48 In our study, the complexities of the different devices, lack of regular experience with them, and the fact
they were new to many pharmacists at the time of training
may have impacted pharmacists long-term recall. As the
participants were geographically diverse, we did not evaluate device demonstration as a skill. Thus, while knowledge of device administration steps improved at 7 months,
application of this knowledge was not assessed.
Strengths and limitations
This study has a number of strengths. The training
program and assessment test were developed using a rigorous approach and validated prior to use. We included 2
comparator groups in our study: traditional lecture training and no training. Further, we conducted 3 posttraining
tests, with a follow-up period considerably longer than
that of other e-learning effectiveness studies. Retention
rates were high: almost all participants completed the
posttest, and around 50% completed all 4 tests. This compares favorably with response rates to e-mailed surveys
(where the average response rate is 33%).49 The study had
sufcient power to detect a mean score difference of at
least 1 point within and between groups. Finally, there
was no duplication in recruitment of pharmacists to intervention groups (pharmacists who participated in the elearning group could not participate in the lecture group
and vice versa).
However, we did not randomize participants to intervention or control groups, and as we adopted a convenience method of recruitment, the study may have been
affected by selection bias. The lack of randomization
would only affect between-group comparisons. Nevertheless, generalization of the e-learning results may be limited to people with a high comfort level with learning via
the Internet and/or who have experience using multimedia online. Given that the study sample represented
well-educated professionals who had daily exposure to
45

The control group did not include pharmacists and
began the study at a different time than the intervention
groups. We chose to use students as controls because we
could ensure that they did not receive inadvertent exposure to anaphylaxis training and thus contamination during follow up. Nonetheless, we acknowledge that control
scores were significantly lower than intervention scores at
pretest. This ultimately impacted pairwise comparisons
and may have distorted the magnitude of the difference
between training and no training. Moreover, the control
scores did not change over time, despite participants completing the same test questions on 4 occasions. This may
have been because of/the result of lack of interest in the
topic, lack of perceived relevance to practice, fatigue
from completing multiple tests, or a true effect.
We used the same 12 questions for each of the 4 tests.
There was the potential for a learning effect from the test
itself, although we did attempt to control for practice
effect, and it was unlikely given there was no change in
control scores. Although we did not adjust for multiple
comparisons in the analyses, we do not consider this to be
a limitation. The key effectiveness measurelong-term
knowledge changewas assessed in 3 post hoc tests
(e-learning, lecture training, or no training groups, comparing 7-month tests and pretests), and the magnitude of
the change in knowledge at all tests was large. Therefore,
with low numbers of multiple comparisons, an effect size
of practical relevance, and very low p-values (p,0.001),
there was no need for adjustment.54
Finally, we acknowledge that this training may not
have been wholly responsible for knowledge demonstrated at 7 months. There is the potential for academic
dishonesty with tests completed remotely. However, participants were de-identified and study incentives were not
dependent on scores, so we consider the motivation to
deceive was low. Although exposure to alternate anaphylaxis information over time (eg, through general media or
through self-study) may have confounded the results,
knowledge gain across learner groups was consistent
(with no gain in the control group) over 7 months.
Figure 2. Mean anaphylaxis knowledge assessment scores by

group and test.
Internet-related technologies, we expected knowledge

and use of the Internet to be high in this population. Lecture participants also were required to show a high level of
comfort with Internet use, as they were required to complete all follow-up tests online. Further, the vast literature
evaluating e-learning programs, the increasing delivery
of online education, a historical early acceptance of technology in the pharmacy profession (all suggesting pharmacists are confident Internet users), and the difficulties
achieving a true random sample in online research may
have combined to reduce the effect of selection bias in our
study.50-52 In addition, we evaluated ASCIA Anaphylaxis
e-training for pharmacists in a context where learners now
dene their education strategies (eg, choosing rather than
being recruited to undertake this program),53 which may
have provided real-world evidence for effectiveness.
Implications and recommendations

ASCIA Anaphylaxis e-training for pharmacists is
part of a group of e-learning packages available to pharmacists and other health professionals, school and childcare workers, and the general community throughout
Australia and New Zealand. Since 2011, more than 760
pharmacists, 4600 health professionals, 130 000 school
and childcare workers, and 1100 members of the general
public, have completed this training.55 The key messages
in each of these programs are equivalent, and the language
used in each program is appropriate for the intended
Figure 3. Change in mean anaphylaxis knowledge assessment

scores by group over time. E-learning scores were similar to lecture pharmacist scores at pre-test (p=0.62), 3-month test (p=0.79)
and 7-month test (p=0.31). Control scores were signicantly
lower than all intervention scores after training (p,0.001).
46

Table 4. Learners Achieving the Minimum Standard for Anaphylaxis Knowledge and the Correct Device Administration Steps by
Group and Test.
p all
groupsa
P e-Learning vs
Lecture
Pharmacistsb
6.7
85.5
53.3
61.8
,0.001
,0.001
0.002
0.004
0.03
0.94
0.66
0.17
0.47
28.3
77.4
40.0
50.0
0.096
0.43
0.07
0.73
0.45
0.82
0.86
E-learning
Lecture
Pharmacists
Lecture Pharmacy Students
Proportion achieving
minimum standard, %
Pretest
Posttest
3-month
7-month
pc
45.1
96.2
85.0
86.7
0.021
45.8
97.4
74.2
80.8
0.001
Proportion correctly
ordering device
administration steps, %
Pretest
Posttest
3-month
7-month
pc
45.0
63.3
-
34.0
87.4
47.2
61.5
0.002
Pearson chi-square test for difference in proportions across all learning groups at each test.
Pearson chi-square test for difference in proportions between e-learning and lecture pharmacists groups at each test.
c
McNemar test for difference in proportions between the 7-month test and pretest for each group.
Proportion achieving minimum standard - percentage of participants completing the test who achieved a score $9 out of 12.
Proportion correctly ordering device administration steps - percentage of participants completing the test who correctly ordered all 4 steps
required for both EpiPen and Anapen device administration.
NA - data were not available for this group and these tests.
b
learner. Despite the success in implementation, ASCIA

anaphylaxis e-training programs have not previously
been evaluated for effectiveness. The study demonstrates
that ASCIA Anaphylaxis e-training for pharmacists is
effective at increasing and maintaining long-term anaphylaxis knowledge across a demographically and geographically diverse population of pharmacists.
Because accurate and current anaphylaxis knowledge
is an essential part of anaphylaxis management, the question of when to retrain should be considered. As the majority of e-learners met the minimum standard for anaphylaxis
knowledge 7 months after training, it is difficult to define
a retraining interval based on declining knowledge. An
additional follow-up evaluation of the same participants
at 18-24 months may be a realistic timeframe. For pharmacy students in the era of the flipped classroom, the addition of this e-learning program would increase their
anaphylaxis knowledge while allowing them to actively
practice with adrenaline auto-injector devices. Investigating the effectiveness of the e-learning program in this context would be useful.
Pharmacists have been identified as an underutilized resource for providing anaphylaxis education and
device training at the time of adrenaline auto-injector
supply.43,44 One-third of e-learners in the study did
not correctly order the steps for EpiPen and Anapen

device administration, and this may impact the quality
of advice provided with these devices. Covert or overt
simulation-based research is required to determine what
happens at the time of adrenaline auto-injector distribution in pharmacies, as a measure of translation of anaphylaxis learning to practice. Research options include
simulated patient methodology to assess device demonstration and anaphylaxis knowledge, or the use of overt
simulation (for example, using mannequins) to investigate the pharmacists response to acute anaphylaxis.
SUMMARY
Regular education updates are required for pharmacists to maintain current knowledge about the prevention
and treatment of anaphylaxis and how to supply and use
adrenaline auto-injectors. ASCIA Anaphylaxis e-training
for pharmacists increased anaphylaxis knowledge longterm. Knowledge gains were similar to ASCIA lecture
training and superior to no training. This e-learning program offers a convenient, effective, no-cost option for
pharmacists to improve and maintain their anaphylaxis
knowledge. Future evaluations should seek to define an
interval for retraining and investigate translation of anaphylaxis knowledge to practice.
47

13. Simons FE. Anaphylaxis. J Allergy Clin Immunol. 2010;125
(2 Suppl 2):S161-S181.
14. Simons FE, Ardusso LR, Bilo MB, et al. World allergy
organization guidelines for the assessment and management of
anaphylaxis. World Allergy Organ J. 2011;4(2):13-37.
15. Kemp SF, Lockey RF, Simons FE. Epinephrine: the drug of
choice for anaphylaxis. A statement of the world allergy organization.
Allergy. 2008;63(8):1061-1070.
16. ASCIA Action Plans for Anaphylaxis, The Australasian Society
of Clinical Immunology and Allergy, AU. http://www.allergy.org.au/
health-professionals/ascia-plans-action-and-treatment. Accessed
November 26, 2013.
17. Emergency treatment of anaphylactic reactions. Guidelines for
healthcare providers, Working Group of the Resuscitation Council,
UK. http://www.resus.org.uk/pages/reaction.pdf. Accessed
November 12, 2013.
18. Muraro A, Roberts G, Clark A, et al. The management of
anaphylaxis in childhood: position paper of the european academy of
allergology and clinical immunology. Allergy. 2007;62(8):857-871.
19. Acute management of anaphylaxis guidelines, Australasian
Society of Clinical Immunology and Allergy, AU. http://www.
allergy.org.au/images/stories/pospapers/ASCIA_Acute_Management_
of_Anaphylaxis_Guidelines_September_2013.pdf. Accessed
November 26, 2013.
20. Pharmacists Information Bulletin: Supplying adrenaline
autoinjectors - schedule 3 medicines, Pharmaceutical Services
Branch, Health Department, Government of Western Australia, AU.
http://www.health.wa.gov.au/anaphylaxis/HP/. Accessed November
26, 2013.
21. Anaphylaxis guidelines for Queensland state schools,
Queensland Department of Education,Training and Employment,
AU. http://education.qld.gov.au/schools/healthy/docs/
anaphylaxis_guidelines_for_queensland_state_schools.pdf.
Accessed 30 October, 2013.
22. Anaphylaxis guidelines. A resource for managing severe
allergies in Victorian schools, Department of Education and Early
Childhood Development, Government of Victoria, AU. http://www.
education.vic.gov.au/school/principals/health/Pages/
anaphylaxisschools.aspx. Accessed August 20, 2014.
23. Anaphylaxis Procedures for Schools 2012, Education and
Communities, NSW Government, AU. http://www.schools.nsw.edu.
au/media/downloads/schoolsweb/studentsupport/studenthealth/
conditions/anaphylaxis/guidelines/anaphylaxis-procedures.pdf.
Accessed October 30, 2013.
24. Diamond S, Salter J, Hummel D. The role of pharmacists in
anaphylaxis education. J Allergy Clin Immunol. 2003;111(1):S102.
25. Chumley-Jones HS, Dobbie A, Alford CL. Web-based learning:
sound educational method or hype? A review of the evaluation
literature. Acad Med. 2002;77(10 Suppl):S86-S93.
26. Cook DA. Levinson AJ, Garside S, Dupras DM, Erwin PJ,
Montori VM. Internet-based learning in the health professions:
a meta-analysis. JAMA. 2008;300(10):1181-1196.
27. Curran VR, Fleet L. A review of evaluation outcomes of webbased continuing medical education. Med Educ. 2005;39(6):561-7.
28. Lahti M, Hatonen H, Valimaki M. Impact of e-learning on nurses
and student nurses knowledge, skills, and satisfaction: a systematic
review and meta-analysis. Int J Nurs Stud. 2014; 51(1):136-149.
29. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning in
medical education. Acad Med. 2006; 81(3):207-212.
30. Salter SM, Loh R, Vale S, Clifford RM, editors. Evaluation of the
anaphylaxis training pharmacist assessment tool (AT-PAsT): a pilot
ACKNOWLEDGMENTS
The authors acknowledge the Australasian Society of
Clinical Immunology and Allergy (ASCIA) for creating
and delivering anaphylaxis education to all members of
the community and for enabling us to evaluate the effectiveness of their anaphylaxis training for pharmacists.
The authors thank Ms. Suzanne Grainger, Impagination
(http://www.impagination.com.au), Victoria, Australia,
for her assistance with the development and implementation of the online data collection forms for pretests and
posttests for the e-learning participants, and Ms. Laura
Firth, Department of Mathematics and Statistics, The
University of Western Australia, for her assistance with
planning the statistical analyses.
The first author, Ms. Sandra Salter, was the recipient
of a University Postgraduate Award and UWA Top-Up
Scholarship, provided by The University of Western
Australia.
REFERENCES
1. Sampson HA, Munoz-Furlong A, Campbell RL, et al. Second
symposium on the denition and management of anaphylaxis:
summary report-Second National Institute of Allergy and Infectious
Disease/Food Allergy and Anaphylaxis Network symposium.
J Allergy Clin Immunol. 2006;117(2):391-7.
2. Lieberman P. Epidemiology of anaphylaxis. Curr Opin Allergy
Clin Immunol. 2008;8(4):316-20.
3. Lieberman P, Camargo CA, Bohlke K, et al. Epidemiology of
anaphylaxis: ndings of the american college of allergy, asthma and
immunology epidemiology of anaphylaxis working group. Ann
Allergy Asthma Immunol. 2006;97(5):596-602.
4. Liew WK, Williamson E, Tang ML. Anaphylaxis fatalities and
admissions in australia. J Allergy Clin Immunol. 2009;123(2):434442.
5. Lin RY, Anderson AS, Shah SN, Nurruzzaman F. Increasing
anaphylaxis hospitalizations in the rst 2 decades of life: New York
State, 19902006. Ann Allergy Asthma Immunol. 2008;101(4):387393.
6. Poulos LM, Waters AM, Correll PK, Loblay RH, Marks GB.
Trends in hospitalizations for anaphylaxis, angioedema, and urticaria
in Australia, 1993-1994 to 2004-2005. J Allergy Clin Immunol.
2007;120(4):878-884.
7. Prescott S. The Allergy Epidemic A Mystery of Modern Life.
Perth, WA: UWA Publishing; 2011.
8. Simons FER, Peterson S, Black CD. Epinephrine dispensing
patterns for an out-of-hospital population: a novel approach to
studying the epidemiology of anaphylaxis. J Allergy Clin Immunol.
2002;110(4):647-651.
9. Tang ML, Osborne N, Allen K. Epidemiology of anaphylaxis.
Curr Opin Allergy Clin Immunol. 2009; 9(4):351-6.
10. Simons FE. Anaphylaxis, killer allergy: long-term
management in the community. J Allergy Clin Immunol. 2006;117
(2):367-377.
11. Sicherer SH, Simons FE. Self-injectable epinephrine for rst-aid
management of anaphylaxis. Pediatrics. 2007;119(3):638-646.
12. Simons FE. Anaphylaxis: recent advances in assessment and
treatment. J Allergy Clin Immunol. 2009; 124(4):625-636.
48

43. Barnett CW. Need for community pharmacist-provided foodallergy education and auto-injectable epinephrine training. J Am
Pharm Assoc. 2005;45(4):479-485.
44. Simons KJ, Simons FE. Epinephrine and its use in anaphylaxis:
current issues. Curr Opin Allergy Clin Immunol. 2010;10(4):354-361.
45. Simons FE, Clark S, Camargo Jr CA. Anaphylaxis in the
community: learning from the survivors. J Allergy Clin Immunol.
2009;124(2):3016.
46. Simons FE, Lieberman PL, Read Jr EJ, Edwards, ES. Hazards
of unintentional injection of epinephrine from autoinjectors:
a systematic review. Ann Allergy Asthma Immunol. 2009;102
(4):2827.
47. Sicherer SH, Vargas PA, Groetch ME, et al. Development and
validation of educational materials for food allergy. J Pediatr.
2012;160(4):651-6.
48. Topal E, Bakirtas A, Yilmaz O, et al. When should we perform
a repeat training on adrenaline auto-injector use for physician
trainees? Allergol Immunopathol (Madr). 2013; In press.
49. Shih T-H, Fan X. Comparing response rates in e-mail and paper
surveys: a meta-analysis. Educ Res Rev. 2009;4(1):26-40.
50. Hesse-Biber S, Grifn AJ. Internet-mediated technologies and
mixed methods research: problems and prospects. J of Mixed
Methods Res. 2013;7(1):43-61.
51. Monaghan MS, Cain JJ, Malone PM, et al. Educational
technology use among US colleges and schools of pharmacy. Am J
Pharm Educ. 2011;75(5):Article 87.
52. Bartolini EZ, Hubbard T. Community pharmacies meet
mobile technologies: a whole new world of opportunity. US
Pharm. 2013;38(8):43-50. http://www.uspharmacist.com/
content/d/featured%20articles/c/42380/. Accessed August 20,
2014.
53. Mascolo MF. Beyond student-centered and teacher-centered
pedagogy: teaching and learning as guided participation. Pedagog
Human Sci. 2009;1(1):3-27.
54. Feise R. Do multiple outcome measures require p-value
adjustment? BMC Med Res Methodol. 2002;2(1):8.
55. ASCIA e-training courses, Australasian Soceity of Clinical
Allergy and Immunology, Sydney, NSW, AU. http://www.allergy.
org.au. Accessed 12 November, 2013.
study. Australasian Pharmaceutical Sciences Association

Conference; December 2011; Adelaide, Australia.
31. PS: Power and Sample Size Calculation, Vanderbilt University
School of Medicine, Department of Biostatistics, Nashville, TN, US.
http://biostat.mc.vanderbilt.edu/wiki/Main/PowerSampleSize.
Accessed September 20, 2013.
32. Tang MLK, Kang LW. Prevention and treatment of anaphylaxis.
Paediatr Child Health. 2008;18(7):309-316.
33. Loh RKS, Vale S. Adrenaline autoinjectors - what pharmacists
need to know. Aust Pharm. 2011; 30(8):692,46.
34. Vale S, Mullins R, Smith J, Loh R. Anaphylaxis training courses
for pharmacists in Australia and New Zealand. J Allergy Clin
Immunol. 2012;129(2):AB178.
35. Simons FE. Anaphylaxis: evidence-based long-term risk reduction
in the community. Immunol Allergy Clin North Am. 2007; 27(2):231-248.
36. Flowers SK, Vanderbush RE, Hastings JK. Web-based
multimedia vignettes in advanced community pharmacy practice
experiences. Am J Pharm Educ. 2010;74(3):Article 39.
37. Sweet BV, Welage LS, Johnston JP. Effect of a Web-based
continuing-education program on pharmacist learning. Am J HealthSyst Pharm. 2009;66(21):1902-3.
38. Hall DL, Corman SL, Drab SR, Smith RB, Meyer SM.
Application of a technology-based instructional resource in diabetes
education at multiple schools of pharmacy: evaluation of student
learning and satisfaction. Curr Pharm Teach Learn. 2010; 2(2):108113.
39. Legris ME, Seguin NC, Desforges K, et al. Pharmacist webbased training program on medication use in chronic kidney disease
patients: impact on knowledge, skills, and satisfaction. J Contin Educ
HealthProf. 2011;31(3):140-150.
40. Congdon HB, Nutter DA, Charneski L, Butko P et al. Impact of
hybrid delivery of education on student academic performance and
the student experience. Am J Pharm Educ. 2009;73(7):Article 121.
41. Erickson SR, Chang A, Johnson CE, Gruppen LD. Lecture
versus web tutorial for pharmacy students learning of MDI
technique. Ann Pharmacother. 2003;37(4):500-5.
42. Lancaster JW, McQueeney ML, Van Amburgh JA. Online
lecture delivery paired with in class problem-based learning. . .does it
enhance student learning? Curr Pharm Teach Learn. 2011;3(1):23-9.
49

A Physical Assessment Skills Module on Vital Signs
Christine Leong, PharmD,a Christopher Louizos, BSc(Pharm),a Grace Frankel, PharmD,a
Sheila Ng, BSc(Pharm),a Harris Iacovides, PharmD, MSc,a Jamie Falk, PharmD,a
Drena Dunford, BSc(Pharm),a Kelly Brink, BSc(Pharm),a Nancy Kleiman, MBA,a
Christine Davis, PharmD,a and Robert Renaud, PhDa,b
a
Faculty of Pharmacy, University of Manitoba, Winnipeg, Canada

Department of Educational Administration, Foundations & Psychology, Faculty of Education, University of Manitoba,
Winnipeg, Canada
b
Submitted November 12, 2013; accepted January 31, 2014; published September 15, 2014.
Objectives. To implement and evaluate a physical assessment module for pharmacy students.
Design. A physical assessment module focusing on vital signs was incorporated into the curriculum for
third-year pharmacy students. This module consisted of an online component, a practical skills workshop, and a clinical practice site.
Assessment. The mean score on the in-class quiz, which evaluated students knowledge of physical
assessment after completion of the online module, was 94%. During the practical skills laboratory, 48%
of student-measured systolic blood pressure (BP) readings and 60% of student-measured diastolic BP
readings were within 5 mmHg of the machine reading. In the assessment of blood pressure technique, areas
of difculty included detection of Korotkoff sounds; steady deation of cuff; and hand-eye coordination.
Conclusion. Students more frequently underestimated systolic BP than the diastolic BP when compared to the automated machine readings. Findings from this study will be used to improve existing
modules and evaluation methods on the physical assessment of vital signs.
Keywords: blood pressure, physical assessment, pharmacy education, pharmacy students, vital signs
examinations.7 The Association of Faculties of Pharmacy

of Canada and the American Association of Colleges of
Pharmacys Center for the Advancement of Pharmacy
Education have identied the performance and interpretation of physical assessment ndings as an educational
outcome for pharmacy graduates of entry-to-practice
pharmacy programs.8,9 However, few reports have described strategies for implementing a physical assessment
program in pharmacy education.10-15 More specically,
many of the previous studies used surveys to assess student satisfaction with the implementation of a new program rather than to assess the impact of the program on
student learning. Others have described the way in which
a physical assessment course is integrated within an existing course. Only one study compared student-measured
blood pressure readings to machine-measured readings,
and no studies have specically reported on common
areas of difculty in learning physical assessment skills
for new learners in a skills laboratory environment. As
a result, the purpose of this study was to implement a physical assessment skills module on vital signs for third-year
pharmacy students enrolled in a Bachelor of Science in
INTRODUCTION
The application of physical assessment skills is recognized as an important part of providing pharmaceutical
care.1-8 Advantages include being able to monitor and
optimize medications more effectively, screen patients
at risk for chronic disease states, promote better communication among health care practitioners, and improve our
overall understanding of patient care.6,7 As pharmacists
scope of practice continues to expand into more patientcentered roles, pharmacy education will require the
incorporation of courses into the curriculum that will develop skills to fulll such roles.
At present, most Canadian pharmacists have not received training in physical assessment skills. One survey
reported 82.4% of Canadian pharmacists never received any type of formal training in conducting physical
Corresponding Author: Christine Leong, PharmD, Faculty
of Pharmacy, Apotex Centre, 750 McDermot Avenue,
University of Manitoba, Winnipeg, MB, R3E 0T5.
Tel: 1-204-318-5276. Fax: 1-204-474-7617. E-mail: christine.
leong@umanitoba.ca
50

Pharmacy program at the University of Manitoba, and to
evaluate student learning of physical assessment skills.
Specific objectives of the study were: to evaluate
students objective knowledge of physical assessment
skills based on the information provided in the online
module; to compare the difference in blood pressure
values obtained between a manual aneroid sphygmomanometer and an automated blood pressure machine when
conducted by a third-year pharmacy student; and to identify common areas of difculty in developing skills in
obtaining a blood pressure reading with a manual aneroid
sphygmomanometer.
standing on the online module. A PASS standing is required of students in order to participate in the practical
skills laboratory workshop and clinical practice site. The
multiple-choice options were randomized where appropriate to prevent students from circulating the answers.
Students also were directed to useful videos demonstrating the measurement of blood pressure and an audio of
Korotkoff sounds.
After completion of the online module, pharmacy
students were required to take an in-class quiz based on
the content provided in the online module prior to participating in the practical skills laboratory workshop. The inclass quiz was administered to evaluate the students
baseline knowledge of physical assessment skills prior
to attending the practical skills workshop. The quiz consisted of 30 multiple-choice questions and 5 short-answer
questions. The multiple-choice section comprised questions focused on the technical performance of vital signs
(eg, steps on how to perform a blood pressure reading).
The short-answer component focused on the clinical application of vital signs measurement (eg, identication of
medical conditions that warrant immediate referral). Students had to receive a grade of 60% or greater to receive
a PASS standing, and 5% of the grade contributed to their
overall grade for the course.
DESIGN
A module on Skills in Physical Assessment focusing
on vital signs was integrated into the Pharmacy Skills
Laboratory 3 course for third-year pharmacy students
(n548) in the 2013-2014 academic year. The course
was designed to apply and develop skills related to pharmacy practice using a wide range of interactive and collaborative learning strategies.
The physical assessment module consisted of 3 components: an online module, a practical skills workshop,
and a clinical experiential practice site (at a periodontal
clinic). The learning objectives for the module were as
follows: (1) to recognize the importance of developing
skills in physical assessment; (2) to demonstrate how to
measure blood pressure, pulse rate, respiratory rate, and
body temperature; (3) to explain and interpret findings
obtained from a physical assessment of vitals; and (4) to
apply physical assessment skills on selected patients for
the purpose of evaluating and monitoring drug therapy
response in a clinical setting. All content was developed
by a registered clinical pharmacist with experience and
formal training in physical assessment, with input on
strategies for providing formal training and evaluation
of instructors and students provided by the director of
the Clinical Learning and Simulation Facility (CLSF) at
the University of Manitoba. This study was exempted for
full review by the Institutional Review Board.
Practical Skills Laboratory Workshop

Students had the opportunity to practice the measurement of vital signs (blood pressure, pulse rate, respiratory
rate, and tympanic temperature) on a classmate during
a 3-hour practical skills workshop. Students were divided into groups of 6, and further divided into pairs to
practice the measurement of vital signs on their classmate
(Appendix 1). The room was set up so that each pair had
their own blood pressure station. The room was also
equipped with a simulator arm station and a thermometer
station. Eight instructors from the faculty of pharmacy
were involved in the supervision and evaluation of pharmacy students during this workshop. Each instructor was
responsible for the supervision of 1 group of 6 pharmacy
students, and each group carried out the activities of the
workshop in a tutorial room. These instructors attended 2
training sessions led by the coordinator of the physical
assessment module to become familiar with the equipment and procient in the techniques for measuring vital
signs. The instructors primary role was to provide feedback as students practiced the measurement of vital
signs and to provide a nal evaluation on blood pressure
technique.
Students assignment to earn their class participation
grade for the course was to assess vital signs and document the readings obtained. Students performed 3 manual
Online Module
The online module included 3 voice-over PowerPoint lectures (approximately 20 minutes each). The
online lectures were hosted on a secured online portal
called Desire2Learn. The 3 lectures included: Introduction to Physical Assessment Techniques; Vital Signs:
Blood Pressure; and Vital Signs: Pulse Rate, Respiratory
Rate, Temperature. A 5-question multiple-choice quiz
followed each lecture. Students were required to receive
a grade of 60% or greater on each quiz to receive a PASS
51

blood pressure readings using a manual aneroid sphygmomanometer (Pocket Nurse Proshyg BP Cuff, Monaca,
PA) and stethoscope (Littmann Classic II S.E., St. Paul,
MN), followed by 1 blood pressure reading with an automated blood pressure machine (American Diagnostic
Corporation Semi-Automatic Adult Blood Pressure Monitor 6012, Hauppauge, NY). The average of the 3 manual
blood pressure readings was calculated. The ADC SemiAutomatic Blood Pressure Monitor has an accuracy of 63
mmHg for blood pressure.15 Students also obtained a heart
rate, respiratory rate, and tympanic temperature reading
on their classmate.
During the workshop, a simulator arm (Nasco Lifeform, Fort Atkinson, WI) was also present to allow students to practice obtaining accurate blood pressure and
pulse readings. The simulator arm allowed the instructor
or student to program a specific blood pressure and heart
rate reading for the learner to practice on using a manual
blood pressure cuff and stethoscope. Students who participated and completed the assignment received a PASS on
the assignment.
A final assessment of blood pressure on a simulated
patient (classmate) was performed during the workshop.
Students were given 15 minutes to demonstrate their ability to perform a blood pressure measurement based on
a checklist derived from Bickleys Bates Guide to Physical Examination and History Taking.17 Evaluation of
students technical skill rather than the accuracy of the
blood pressure values they obtained was the focus of this
assessment as additional practice would be necessary for
some students to become procient. Students who missed
critical steps in obtaining a blood pressure reading or who
demonstrated weaknesses in any of these areas (eg, steady
coordination of the control valve during cuff deation)
were advised to gain additional practice prior to their
Periodontal Clinic exposure. A date was set for students
to gain additional practice and to be reevaluated. The
second session involved practicing 3 to 4 times on different individuals (classmates and instructor).
Table 1. Distribution of Grades on the Short-Answer

Component of In-Class Quiz (n548)
Letter Grade
A1
A
B1
B
C1
C
D
F
Students, No. (%)

19
21
2
3
1
2
0
0
(39.6)
(43.8)
(4.2)
(6.3)
(2.1)
(4.2)
blood pressure cuff were 125.3613.7 mmHg (range 94.0

mmHg to 153.0 mmHg) and 77.769.3 mmHg (range 52.0
mmHg to 97.0 mmHg), respectively. Independent t tests
were conducted to compare the mean student and automated values for each of the systolic and diastolic readings. The mean student value did not differ signicantly
from the mean automated value in both the systolic (BP
t(94)51.55, p50.13) and diastolic (BP t(94)50.57,
p50.57) readings.
Although the differences between the mean student
and automated values for systolic and diastolic BPs were
not significant, there were other differences worth noting
related to the use of a manual vs an automated blood
pressure machine. The mean absolute difference between
the student-measured and automated blood pressure machine for the systolic and diastolic BPs was 6.666.1
mmHg (range 0.3 mmHg to 35.3 mmHg) and 5.063.8
mmHg (range 0 mmHg to 14.0 mmHg), respectively.
The manual aneroid blood pressure cuff operated by the
students appeared to underestimate the systolic BP more
frequently than the diastolic BP when compared with
readings obtained using the automated blood pressure
machine (Table 2). Conversely, students tended to overestimate diastolic BP more often than systolic BP. Figure 1
shows the distribution of absolute differences in systolic
and diastolic BPs in 5 mmHg increments starting from
0 to 5 mmHg to over 20 mmHg. Most of the students were
able to achieve a manual reading within 5 mmHg of the
automated reading. However, the majority of the students
who under- or overestimated the blood pressure readings
by greater than 5 mmHg, fell in the 5 mmHg to 10 mmHg
range. From a clinical standpoint, a blood pressure reading that is more than 5 mmHg either under or over a persons true systolic or diastolic BP can increase the
likelihood of a misdiagnosis or an inaccurate measurement of medication efcacy.

The mean score on the in-class quiz was 94%. The
mean score on the multiple-choice section of the test was
97%, and the mean score on the short-answer component
was 84.7%. The distribution of grades in the short-answer
component is shown in Table 1.
The mean student-measured (using a manual aneroid
sphygmomanometer) systolic blood pressure (SBP) was
121.2612.4 mmHg (range 92.0 mmHg to 150.7 mmHg)
and the mean student-measured diastolic BP was
78.869.0 mmHg (range 60.0 mmHg to 98.7 mmHg). The
mean systolic and diastolic BP measured by an automated
DISCUSSION
Designing a physical assessment course is a relatively
new and important area of interest to many educators in
52

Table 2. Proportion of Students Underestimating or Overestimating the SBP and DBP by Greater than 5 mmHg Using a Manual
Device in Comparison to the Automated Machine
SBP
Underestimating
Overestimating
DBP
No. (%)
Range (mmHg)
No. (%)
Range (mmHg)
20 (41.7)
5 (10.4)
-5.3 to -35.3
5.3 to 14.0
8 (16.7)
11 (29.9)
-6.7 to -13.0
5.3 to 12.7
pharmacy and other health care professions. In this study,

students demonstrated baseline knowledge of physical
assessment through an in-class test prior to attending the
practical skills laboratory workshop. However, a pretest on
content presented in an online module was not administered and therefore student learning as a result of the online
module could not be evaluated.
The practical skills workshop revealed that new
learners of blood pressure assessment technique using
a manual cuff more frequently underestimated systolic
BP. In contrast, student-measured diastolic BP more
closely reflected the readings obtained from an automated
blood pressure machine. These observations might be
expected of new learners given their initial unfamiliarity
with Korotkoff sounds and skill in achieving a steady deflation rate. However, it could be argued that the diastolic
BP should be more difficult to detect since it is potentially
more difficult to determine the disappearance or muffling
sounds of the diastolic BP. McCall and colleagues
reported that among BP assessments taken by 83 secondyear pharmacy students, 51% of systolic readings and
47% of diastolic readings were within 5 units of the machine reading,14 compared to 48% and 60%, respectively,
in our study. McCall and colleagues noted nal digit bias,
inappropriate cuff size, and variability in deation rate as
common errors leading to an inaccurate diastolic BP readings. In our study, the accuracy of the blood pressure
readings could also have been inuenced by inappropriate
cuff size (only standard and large cuff sizes were available), ambient room noise during the exercises, and improperly calibrated equipment (ie, new equipment not
tested for accuracy). In addition, students commented
on how the release valves on the new equipment were
initially difcult to adjust, which could have affected students ability to deate the cuff at a steady rate.
The final assessment of students on their blood pressure technique using a manual cuff identified common
major and minor issues (Table 3). Major areas of difculty
required considerably more practice to master blood pressure technique. Minor issues, on the other hand, were
easily addressed and corrected on the subsequent trial.
Identication of common major and minor areas of difculty during the study allowed instructors to develop future strategies for teaching and evaluating new learners of
manual blood pressure technique.
While the instructor could observe and evaluate technique in measuring blood pressure, it was difficult for the
instructor to decipher whether the student heard and correctly interpreted the Korotkoff sounds, which is necessary to accurately obtain the true blood pressure value of
an individual. Students who expressed the inability to detect the Korotkoff sounds were told to practice obtaining
a reading on a simulator arm and on different classmates.
The simulator arm can be programmed to assess the students ability to obtain an accurate blood pressure value.
However, the Korotkoff beats produced by the simulator
Figure 1. Absolute difference between student and machine measurements (N548).
53

common areas of difficulty for new learners of blood
pressure measurement technique using a manual aneroid
sphygmomanometer were identified. Identifying common
issues observed during the performance of blood pressure
assessment and recognizing how student-measured readings compared to machine-measured readings allowed the
instructors to tailor strategies for improving existing
models and evaluation methods for teaching physical
assessment skills. These findings have important clinical
implications as they relate to teaching pharmacy students
to identify accurately patients with high blood pressure and
to monitor effectively those on antihypertensive therapy.
Table 3. Major and Minor Areas of Difficulty Experienced by

the Student While Learning Blood Pressure Technique
Major Areas of Difculty
Korotkoff sounds could not be heard
Cuff could not be deated at a steady rate of 2 to
3 mmHg per second
Overall hand-eye coordination in operating the
manual device was poor
Minor Areas of Difculty
Patient was not properly positioned
Stethoscope was donned incorrectly
Stethoscope was not turned on
Stethoscope was not placed correctly over the
brachial artery
Cuff was not deated completely after estimating the SBP
ACKNOWLEDGMENTS
The authors acknowledge the Faculty of Medicine
Director of the CLSF at the University of Manitoba for
his consultation on strategies for providing formal training and evaluation of instructors and students. The
authors thank Cheryl Kristjanson for her consultation on
program evaluation and review of the manuscript.
arm were perceived by students and instructors as obvious (ie, easy to identify) compared to those of a real
patient. Additional practice on different individuals
would allow for more exposure and familiarity with
Korotkoff sounds. In addition, teaching stethoscopes
(with 2 sets of ear prongs) could provide a means for
the instructor to listen to the Korotkoff sounds with the
student. In our experience, these stethoscopes were unable to produce clearly audible sounds when compared
with those produced by the Littmann Classic II S.E.
stethoscopes. While having each pair of students perform
a blood pressure reading in a separate room to minimize
environmental noise is ideal, it is not always a practical
solution for many program coordinators. However, a separate room could be feasible if only used to conduct the
nal assessment of blood pressure technique. In the future, using standardized patients and comparing studentand clinician-measured blood pressure readings will be
considered.
Despite these areas for improvement, there were
a number of factors that contributed to the success of
the physical assessment module. Having one instructor
per group of 6 students appeared to be an appropriate
student-to-instructor ratio for supervising and evaluating
blood pressure technique. In addition, providing immediate feedback to the students and allocating 3 hours for
the workshop allowed students with enough time to practice and develop their technique.
REFERENCES
1. Rospond RM, Tice A, Tice B. Physical assessment for the
community pharmacist, part 2. Americas Pharm. 1999;121:47-53.
2. Pauley T, Marcrom R, Randolph R. Physical assessment in the
community pharmacy. Americas Pharm 1995;NS35(5):40-9.
3. Da Camara C, DElia R, Swanson L. Survey of physical
assessment course offerings in American colleges of pharmacy. Am J
Pharm Educ. 1996;60(4):343-347.
4. Spray JW, Parnapy SA. Teaching patient assessment skills to
doctor of pharmacy students: the TOPAS study. Am J Pharm Educ.
2007;(4):Article 64.
5. Longe RL. Teaching physical assessment to doctor of pharmacy
students. Am J Pharm Educ. 1995;59(2):151-155.
6. Simpson SH, Wilson B. Should pharmacists perform physical
assessments? Can J Hosp Pharm. 2007;60(4):271-272.
7. Barry AR, McCarthy L, Nelson CL, Pearson GJ. An evaluation of
teaching physical examination to pharmacists. Can Pharm J (Ott).
2012;145(4):174-180.
8. Association of Faculties of Pharmacy of Canada. Educational
outcomes for rst professional degree programs in pharmacy (Entryto-Practice Pharmacy Programs) in Canada. Vancouver (BC):
Association of Faculties of Pharmacy of Canada; 2010. https://www.
afpc.info/sites/default/les/AFPC%20Educational%20Outcomes.
pdf. Accessed September 3, 2014.
9. Medina MS, Plaza CM, Stowe CD, et al. Center for the
Advancement of Pharmacy Education educational outcomes 2013.
Am J Pharm Educ. 2013; in press.
10. Bolesta S, Trombetta DP, Longyhore DS. Pharmacist instruction
of physical assessment for pharmacy students. Am J Pharm Educ.
2011;75(2):Article 29.
11. Albano CB, Brown W. Integration of physical assessment within
a pathophysiology course for pharmacy. Am J Pharm Educ. 2012;
76(1):Article 14.
12. Sherman JJ, Riche DM, Stover KR. Physical assessment
experience in a problem-based learning course. Am J Pharm Educ.
2011;75(8):Article 156.
SUMMARY
A physical assessment skills module for third-year
pharmacy students at the University of Manitoba was successfully implemented and evaluated. Pharmacy student
knowledge of pharmacy assessment skills was evaluated,
the values between student-measured and machinemeasured blood pressure readings were compared, and
54

15. Grice GR, Wenger P, Brooks N, Berry TM. Comparison of
patient simulation methods used in a physical assessment course.
Am J Pharm Educ. 2013;77(4):Article77.
16. American Diagnostic Corporation (ADC). Semi-Automatic
Blood Pressure Monitor 6012 Instruction Manual. ADC. Hauppauge,
New York.
17. Bickley LS. Bates Guide to Physical Examination and History
Taking. 9th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2007.
13. Elliott KE, McCall KL, Fike DS, Polk J, Raehl C. Assessment of
manual blood pressure and heart rate measurement skills of pharmacy
students: a follow-up investigation. Am J Pharm Educ. 2008;72(3):
Article 60.
14. McCall KL, Raehl C, Nelson S, Haase K, Fike DS. Evaluation of
pharmacy students blood pressure and heart rate measurement skills
after completion of a patient assessment course. Am J Pharm Educ.
2007;71(1):Article 1.
55

Appendix 1. Physical Assessment of Vitals Laboratory Agenda
Instructions: By the end of the Practical Skills Workshop, students are expected to:
1. Complete and submit the Physical Assessment of Vitals Laboratory Assignment
2. Demonstrate the ability to perform a blood pressure measurement on a simulator arminstructor (Final Assessment)
Each pair of students should have:
(1) ONE stethoscope (unless student has brought his/her own)
(2) ONE manual aneroid sphygmomanometer
(3) ONE automated blood pressure monitor
(4) Alcohol swabs
Each room will have:
(1) ONE simulator arm
(2) ONE teaching stethoscope
(3) ONE measuring tape
(4) ONE tympanic thermometer
Expected timeframe to complete each component of the workshop:
Component
Expected time to complete
Blood pressure reading with an aneroid sphygmomanometer

Blood pressure reading with an automated BP machine
Heart rate or respiratory rate
Tympanic temperature
5 to 10 minutes
1 to 2 minutes
30 seconds to 1 minute
1 to 2 seconds
56

Student Self-Screening for Methicillin-Resistant Staphylococcus Aureus
(MRSA) Nasal Colonization in Hand Hygiene Education
Tia Lum, PharmDa Kristin Picardo, PhD,b Theresa Westbay, PhD,b Amber Barnello, BS,b
Lynn Fine, PhD,c Jill Lavigne, PhD, MPHa
a
Wegmans School of Pharmacy, St John Fisher College, Rochester, New York

Department of Biology, St. John Fisher College, Rochester, New York
c
University of Rochester Medical Center, Rochester, New York
b
Objective. To determine the feasibility and effectiveness of adding a hand hygiene exercise in selfscreening for Methicillin-Resistant Staphylococcus Aureus (MRSA) nasal colonization to a health care
delivery course for rst-year pharmacy (P1) students.
Design. About one month after students were trained in hand hygiene technique and indications,
faculty members demonstrated how to self-screen for MRSA nasal colonization. Students were then
asked to screen themselves during the required class time. Aggregated class results were shared and
compared to prevalence estimates for the general population and health care providers.
Assessment. The 71 students present in class on the day of the self-screening exercise chose to
participate. A survey comparing presecreening and postscreening responses indicated incremental
improvements in student knowledge and awareness of health care associated infections and motivation
to perform hand hygiene. On the written exam, student performance demonstrated improved knowledge compared to previous class years.
Conclusion. Self-screening for MRSA nasal colonization in a health care delivery course for P1
students increased students motivation to perform hand hygiene techniques and follow indications
promulgated by the World Health Organization.
Keywords: hand hygiene, health care associated infection, prevention, MRSA, screening, colonization
students have not tested the feasibility of student screening

or its impact on student knowledge and attitudes.11
In 2011, faculty members at the Wegmans School of
Pharmacy began training all newly matriculated P1 students to follow the World Health Organizations 8-step
method for hand hygiene using alcohol-based products.11
Students were successful in mastering the skill for hand
hygiene and reported greater motivation to perform hand
hygiene after training.12
To make students more aware of the likelihood of
MRSA exposure within their own classroom, we introduced a new active learning exercise in fall 2012: the
MRSA nasal colonization self-screening. The purpose
of this study was to determine the feasibility of having
students participate in this invasive exercise and to assess
the effects on student attitudes toward, knowledge of, and
motivation to perform hand hygiene.
INTRODUCTION
Methicillin-resistant Staphylococcus aureus (MRSA)
is a common cause of health care associated infections, and
the incidence of community-acquired infections has increased in recent years.1,2 Of the general population, about
1.5% are asymptomatic nasal carriers of MRSA, and the
prevalence among health care workers is about 3 times as
high (4.6%).3-6 The anterior nares serve as the main MRSA
reservoir, but transient hand carriage and subsequent transmission is possible. Implementation of proper hand hygiene
technique is effective in preventing person-to-person transmission of MRSA, and such programs have been associated
with reduced prevalence of the infection.8-10 Despite the
higher prevalence of the bacteria among health care
workers, health care providers are rarely required to undergo
screening for MRSA nasal colonization. Studies of the prevalence of MRSA carriage among health care professional
DESIGN
Corresponding Author: Jill Lavigne, Wegmans School of

Pharmacy, St. John Fisher College, Rochester, New York
14618. Tel: 585 385 5255. E-mail: jlavigne@sjfc.edu
Newly matriculated doctor of pharmacy (PharmD)

students received hand hygiene instruction within the
57

context of a required 3-credit hour, semester-long lecture
course titled Health Care Delivery. Hand hygiene instruction was a natural companion to the other introductory
content presented in the course, which spanned what it
means to be a health care professional, the various health
care professions, health care settings, public health, the
Triple Aim (lower cost, better health care quality, and
improved patient outcomes), and interdisciplinary, patientcentered care. By providing this instruction at the start of
the first semester of the P1 year and following it with
MRSA self-screening one month later, we hoped to provide students with a clinical skill that they could use
immediately and which, if performed regularly, could
protect them throughout their experiential education. This
study was reviewed and approved by the schools Institutional Review Board (IRB).
Approximately one month after the P1 students completed their hand hygiene examination and skills assessment (evaluated using a rubric), we introduced the MRSA
self-screening exercise during class. The flora sampling
and reporting of MRSA nasal colonization incidence were
designed to actively engage students in inquiry, reinforcing their interest and helping them make connections to
hand hygiene. Thus, pedagogical components of this exercise (ie, teaching technique, providing evidence to support its importance) engaged students in an investigation
that underscored the importance of hand hygiene.
Two faculty members from the biology department
began the class by briefly reviewing MRSA microbiology
and the history of contagion. This introduction consisted
of a review of the importance of hand hygiene based on an
18th century case and a condensed tutorial on MRSA
microbiology. Historical information regarding the history of hand washing and data demonstrating how hand
washing improved mortality rates in clinical settings were
presented to capture the students interest. The course
instructor then reviewed the prevalence of health careassociated and community-acquired MRSA and introduced the exercise as a way to better understand the ora
present in the classroom. At our institution, P1 students
remain in the same classroom through the school year
with instructors circulating through the room. Therefore,
the ora of the classroom may more closely resemble that
of the P1 students than classrooms in which many different students spend considerably less time.
Techniques to properly gather anterior nasal specimens using a sterile, calcium alginate swab moistened
with sterile saline were then described and demonstrated
by the biology faculty member and course instructor, who
collected their own samples in front of the class. The
faculty members drew a specimen from each naris and
transferred the first sample to a BBL CHROMagar MRSA
II screening agar plate (Becton, Dickinson and Company,

Sparks, MD), and the second sample to a mannitol salt
agar (Becton, Dickinson and Company, Sparks, MD)
plate. Plates were marked with a unique 6-digit identification number, which each participant was to record and
retain to ensure anonymity. No personal or other identifying information was attached to the plates. Faculty
members then distributed sample collection supplies to
each student and instructed the class to self-collect specimens. The supplies were acquired by the Department of
Biology for $15.44 per person. The introduction, demonstration, and student sampling took about 20 minutes.
Immediately following specimen collection, plates were
transferred to the laboratory and incubated according
to manufacturer specifications for the recommended
24-hour time period at 37C. After 24 hours, plates were
evaluated for growth. Any CHROMagar MRSA II plates
that had colony growth as indicated by a mauve-violet
color were identied as preliminary positive results suggestive of MRSA presence. Any MSA plates that demonstrated a color change from baseline pink to bright yellow,
which is indicative of a Staphylococcus species capable of
fermenting mannitol, were identied as positive for pathogenic Staphylococcus growth. Each set of positive results was cross-referenced according to the unique 6-digit
identier on the plates. If a participants samples corresponded to positive results on both CHROMagar MRSA
II and mannitol salt agar plates, further testing was performed. First, an isolated colony from the mannitol salt
agar plate was extracted and placed on a BBL Trypticase
Soy Agar with 5% Sheep Blood (TSA II; Becton, Dickinson
and Company, Sparks, MD) plate according to manufacturer
recommendations for a 24-hour incubation period at 37C.
Additionally, colony growth on TSA II plates exhibiting
beta hemolysis after incubation was sampled for Gram
staining. Results of the specimen plating exercise are represented in Figure 1.
Approximately 2 weeks later, culture results (identified
only by 6-digit plate numbers) were posted to Blackboard
where students could view their results and those of the rest of
the class. Although MRSA nasal colonization is not considered a condition requiring treatment, we recommended that
students with a positive result see their health care provider or
visit the campus wellness center. Health care providers have
the option of repeating the test and treating any colonization
with a regimen of mupirocin lotion applied topically by prescription, for example. During the next class period, we calculated the prevalence of MRSA nasal colonization in the
class and compared it to the prevalence among the general
population and among health care providers.
Two weeks after completing the screening exercise,
after the MRSA screening results were shared with the
58
Figure 1. Schematic representation of the self-screening process and data collection and analysis protocols.
class, students were asked to complete an anonymous

survey during class to assess their knowledge, attitudes,
and motivation regarding hand hygiene before and after
completing the MRSA instruction. To measure incremental improvement in motivation and attitudes to perform
hand hygiene after the MRSA exercise, many of the same
survey items that we had used earlier in the semester to
assess student knowledge of hand hygiene technique were
used on the survey instrument. The survey design was
based on our previously published work16,17 It included
17 questions with responses measured on a 4-point Likert
scale (15weak or disagree, 25fair or somewhat disagree,
35good or somewhat agree, 45very good or agree) and
each item had a prescreening and postscreening column in
which to record their score. Participation in the survey
was anonymous and voluntary. Analysis of students re-
sponses for prescreening vs postscreening was performed

using the Wilcoxon signed ranks test. All analyses were
conducted in SPSS, version 20, and repeated in STATA,
version 10 (College Station, Texas).
After completing the self-screening, reviewing the
results, and calculating and comparing MRSA nasal colonization prevalence, students were expected to be able to
meet several learning objectives incorporating the knowledge, application, and synthesis domains of Blooms Taxonomy and the human dimension, application, integration,
and foundational knowledge domains of Finks Taxonomy (Table 1). For example, collection of samples (application skill) allowed for rst-hand experience of the
nasal specimen collection process and for observation
of how the students, their classmates, and instructors
reacted to the experience (human dimension [learning
59

Table 1. Cognitive Domains of Learning Objectives According to Taxonomies of Learning for a Class Exercise on Self-Screening
for Methicillin-Resistant Staphylococcus Aureus
Learning Objective
Cognitive Domain in
Blooms Taxonomy14
Successfully obtain a viable sample of nasal

mucosa for testing.
Express an improvement in knowledge of the
importance of hand hygiene to patient health
and pharmacy practice.
Application
Application
Cognitive Domain in
Finks Taxonomy15
Human Dimension
Application
Integration
Human Dimension
Foundational
Knowledge
Integration
Human Dimension
Foundational
Knowledge
Express increased awareness of the impact of

hand hygiene on patient and provider health.
Application
Express the belief that self-screening for MRSA

colonization is an important exercise for
pharmacy students.
Knowledge
Human Dimension
Application
Express an improvement in motivation to

perform hand hygiene as indicated.
Synthesis
Integration
Human Dimension
Foundational Knowledge
about oneself and others]). Through the sharing of anonymous results, students gained foundational knowledge
about their personal ora and that of their classmates
(human dimension). Students had the opportunity to synthesize information and experience from this and previous hand hygiene curriculum components to further
enhance their knowledge and awareness of the importance of hand hygiene, as well as their motivation to perform proper hand hygiene.
statement: Screening for MRSA colonization is an important exercise for pharmacy students. All participants
responded positively to the following statements: Learning about hand hygiene with alcohol-based preparations
will help me to be a better pharmacist and This class
provided information that I can apply in practice.
Class performance questions on hand hygiene and
health care-associated infections on a written examination (worth 33% of the course grade) was compared to
performance of the P1 class in the previous year, which
had received the hand hygiene instruction but had not
completed the MRSA nasal colonization self-screening
exercise. For the test question, Health care associated

Matriculating P1 students in Fall 2012 were primarily Caucasian and between the ages of 18 and 24 (Table 2).
Of the 80 students in the P1 class, 71 were present in class
on the day of the self-screening exercise and all participated in the exercise. Of the 71 sets of plates, all samples
yielded colony growth. Three (4.2%) plates screened positive for MRSA colonization. The Fisher exact test was
used to determine if there was an association between the
classroom prevalence and the general population prevalence. The difference between the groups was not significant (2-tailed, p50.65), suggesting that the newly
matriculating P1 class prevalence resembled that of the
general population.
Twenty-six of the 71 students who had completed
the exercise completed the survey instrument. Students
scores on knowledge, awareness, and motivation to perform hand hygiene signicantly improved following completion of the MRSA self-screening exercise (Table 3).
Notably, 24 (92.3%) students responded positively to the
Table 2. Demographics of First-Year Pharmacy Class Who

Participated in an Exercise on Self-Screening for MethicillinResistant Staphylococcus Aureus (N580)
60
Variable
No. (%)
Male
Female
Age (years)
18-24
25-34
35-44
Race
Caucasian
African American
Asian
Asian Indian
No response
35 (43.8)
45 (56.3)
60 (75.0)
19 (23.8)
1 (1.3)
58
5
6
1
10
(72.5)
(6.3)
(7.5)
(1.3)
(12.5)

Table 3. First-Year Pharmacy Students Responses on a Pre/Post Screening Survey Regarding Hand Hygiene and Health Care
Associated Infections after Participating in a Self-screening Exercise to Detect Methicillin-Resistant Staphylococcus Aureus
(N526)
Survey Item
My ability to identify indications for hand hygiene (that
is, when to do hand hygiene).
My knowledge of how to do hand hygiene using
alcohol based solutions.
My ability to correctly perform hand hygiene with an
alcohol based solution.
My motivation to perform hand hygiene with an alcohol
based solution as indicated.
My awareness of the impact of hand hygiene on patient
health.
My awareness of the impact of hand hygiene on the
health of health care providers.
My knowledge of why hand hygiene with alcohol based
solutions is important in pharmacy practice.
My ability to perform hand hygiene as indicated while
practicing pharmacy.
My motivation to perform hand hygiene as indicated
while working in a pharmacy setting.
My awareness of the impact of hand hygiene in
pharmacy practice on patient health.
Screening for MRSA colonization is an important
exercise for pharmacy students.
It is important for pharmacy students to learn proper
hand hygiene technique using alcohol based
preparations.
Failure to perform hand hygiene is a major issue in
patient care.
Expert hand hygiene knowledge and skills will help
me be a better pharmacist.
Failure to perform hand hygiene with alcohol based
preparations as indicated is a major issue in
pharmacy practice.
Learning about hand hygiene with alcohol based
preparations will help me be a better pharmacist.
This class provided information that I can apply in
practice.
Positive
Responsesa
No. (%)
Difference
Post/Pre Screening
Scores, Mean (SD)
Pb
26 (100)
1.0 (0.7)
, 0.001
25 (96.2)
1.0 (1.0)
, 0.001
25 (96.2)
1 (0.96)
, 0.001
26 (100)
0.92 (0.80)
, 0.001
26 (100)
0.88 (0.82)
, 0.001
26 (100)
0.85 (0.88)
, 0.001
26 (100)
0.88 (0.95)
, 0.001
25 (96.2)
0.96 (1.03)
, 0.001
26 (100)
0.92 (0.93)
, 0.001
26 (100)
0.96 (0.96)
, 0.001
24 (92.3)
0.54 (0.86)
, 0.05
26 (100)
0.58 (0.90)
, 0.05
26 (100)
0.42 (0.76)
, 0.05
25 (100)
0.40 (0.71)
, 0.05
24 (92.3)
0.62 (0.75)
, 0.001
26 (100)
0.50 (0.76)
, 0.05
26 (100)
0.50 (0.71)
, 0.001
a
Responses were based on a Likert scale ( 15Weak, 25Fair, 35Good, 45Very Good) on which ratings of 3 or 4 were considered positive
responses.
b
As determined by Wilcoxon signed rank test.
DISCUSSION
infections are a rare cause of death, true or false? the

number of students selecting the correct answer was
signicantly higher in the class that had completed the
MRSA self-screening exercise (98.75% compared to
88.3% of the previous class with p,0.01). (The p-value
associated with the chi-square test for a difference in proportions is p,0.01.) Student performance on test items
specic to hand hygiene indications and technique were
not signicantly better than those of the previous class.
We assessed 3 broad questions in this study: (1) Is

MRSA nasal colonization self-screening feasible and acceptable to students in a required course completed prior
to clinical training? (2) Do students express an increased
knowledge and awareness of health care-associated infections and the importance of hand hygiene after selfscreening than after hand hygiene instruction alone? (3)
Was the class performance on test questions related to
61

health care-associated infections and hand hygiene better
than that of the previous years class, which received the
same instructions on hand hygiene technique and indications, but did not participate in the self-screening exercise?
The feasibility of adding the self-screening exercise
was determined by the number of students who successfully obtained nasal samples by inserting a cotton swab to
the very back of each side of their noses and swiping each
swab onto a culture plate, one at a time. We anticipated
that some students would feel uncomfortable sticking
a swab up their nose, particularly in front of classmates.
Others might not have wanted their samples tested and
reported, even though samples were anonymous. Nevertheless, all 71 students (100%) present in class completed
the exercise and every set of plates returned a viable result. In a relatively full classroom with faculty members
participating, it may have been difficult for students to
resist participation, particularly at this early stage in their
training.
We concluded that self-screening for MRSA nasal
colonization was feasible in this class of P1 students. The
cultures demonstrated a prevalence of MRSA nasal colonization among the students in the class that was similar
to that in the general population, and the pre/post screening survey responses suggested that this exercise resulted
in achievement of learning objectives.
A limitation of the study was the relatively low survey
response rate. Factors that may have contributed to the low
response rate may include lower class attendance on the
day the survey was administered and student fatigue. The
pre/post screening survey used after the self-screening for
MRSA nasal colonization was nearly identical to the pre/
post survey students completed at the end of the hand rubbing skills training with alcohol based preparations. The
instructor explained the purpose of the second survey, but
fatigue, lack of time between classes, and competing demands likely had a negative effect on the response rate.
We expected to see modest improvements in students rating of incremental learning after completing
the MRSA self-screening exercise but did not. Nevertheless, students reported signicant improvements across all
parameters, including improvements in their knowledge
of hand hygiene even though the MRSA self-screening
exercise did not address hand hygiene technique. Students
may have assumed that self-screening for MRSA nasal
colonization was a component of hand hygiene programs.
Alternatively, students may have responded positively
overall to the exercise and, therefore, tended to rate all
of the survey items positively. Class performance on a test
question related to health care-associated infections was
signicantly better than that of the previous class (which
had received the same hand hygiene training but not the
self-screening exercise), but scores on test questions related to hand hygiene were not higher than those of the
previous class.
SUMMARY
Self-screening for MRSA nasal colonization in a first
semester lecture course for P1 students appears to be
a feasible method for reinforcing motivation to perform
hand hygiene technique as promulgated by the World
Health Organization. Future studies should examine the
effects of such active learning on student hand hygiene
behaviors and on retention of learned practice upon entering the health care workforce.
REFERENCES
1. Chambers HF, DeLeo FR. Waves of resistance: Staphylococcus
aureus in the antibiotic era. Nat Rev Microbiol. 2009;7(9):629-641.
2. Methicillin-resistant Staphylococcus aureus (MRSA) Infections.
Centers for Disease Control and Prevention. http://www.cdc.gov/
mrsa/health care/index.html. Accessed February 19, 2014.
3. Albrich WC, Harbarth S. Health-care workers: source, vector, or
victim of MRSA? Lancet Infect Dis. 2008;8(5):289-301.
4. Jernigan J, Kallen A. Methicillin-resistant Staphylococcus aureus
(MRSA) infections. Activity C: ELC prevention collaboratives.
Division of health care quality promotion. Centers for disease control
and prevention. http://www.cdc.gov/HAI/pdfs/toolkits/MRSA_toolkit_
white_020910_v2.pdf. Accessed March 1 2013.
5. Elie-Turenne MC, Fernandes H, Mediavilla JR, et al. Prevalence
and characteristics of Staphylococcus aureus colonization among
health care professionals in an urban teaching hospital. Infect Control
Hosp Epidemiol. 2010;31(6):574-580.
6. Gorwitz RJ, Kruszon-Moran D, McAllister SK, et al. Changes in
the prevalence of nasal colonization with Staphylococcus aureus in
the United States, 2001-2004. J Infect Dis. 2008;197(9):1226-1234.
7. Pittet D, Hugonnet S, Harbarth S, et al. Effectiveness of a hospitalwide programme to improve compliance with hand hygiene. Lancet.
2000;356(9238):1307-1312.
8. Grayson ML, Russo PL, Cruickshank M, et al. Outcomes from the
rst 2 years of the Australian National Hand Hygiene Initiative. Med
J Aust. 2011;195(10):615-619.
9. Sroka S, Gastmeier P, Meyer E. Impact of alcohol hand-rub use on
methicillin-resistant Staphylococcus aureus: an analysis of the
literature. J Host Infect. 2010;74(3):204-211.
10. WHO guidelines on hand hygiene in health care. Geneva: World
Health Organization. 2009. http://whqlibdoc.who.int/publications/
2009/9789241597906_eng.pdf
11. Rohde RE, Rowder C, Patterson T, Redwine G, Vasquez B,
Carranco E. Methicillin resistant Staphylococcus aureus (MRSA): an
interim report of carriage and conversion rates in nursing students.
Clin Lab Sci. 2012;25(2):94-101.
12. Lavigne JE. Schwartzmeyer C. Implementing World Health
Organization standards for hand hygiene indications and methods using
alcohol-based preparations at the Wegmans School of Pharmacy.
International Forum on Quality & Safety in Health care. Paris, France.
April, 2012. http://sherpub.sjfc.edu/cgi/viewcontent.cgi?article5
1015&context5pharmacy_facpub&sei-redir51&referer5http%3A%
2F%2Fwww.bing.com%2Fsearch%3Fq%3Dlavigne%2Bhand%
2Bhygiene%2B2012%26qs%3Dn%26form%3DQBRE%26pq%
62

15. Fink LD. Creating Signicant Learning Experiences. An
Integrated Approach to Designing College Courses. San Francisco,
CA: Jossey-Bass, 2003.
16. Gilligan AM, Myers J, Nash JD, et al. Educating pharmacy
students to improve quality (EPIQ) in colleges and schools of
pharmacy. Am J Pharm Educ. 2012;76(6):Article 109.
17. Lavigne J. Implementing Educating Pharmacy Students and
Pharmacists to Improve Quality (EPIC) as a requirement at the
Wegmans School of Pharmacy. Curr Pharm Teach Learn. 2012;4
(3):212-216.
3Dlavigne%2Bhand%2Bhygiene%2B2012%26sc%3D0-21%26sp%
3D-1%26sk%3D%26cvid%3Defee68187d3d42869ac6fb3cf09a78d2#
search5%22lavigne%20hand%20hygiene%202012%22
13. Paule SM, Mehta M, Hacek DM, et al. Chromogenic media vs
real-time PCR for nasal surveillance of methicillin-resistant
Staphylococcus aureus: impact on detection of MRSA-positive
persons. Am J Clin Pathol. 2009;131(4):532-539.
14. Bloom BS, ed. Taxonomy of Educational Objectives. The
Classication of Educational Goals. Handbook I: Cognitive Domain.
New York, NY: McKay, 1956.
63

Using Curriculum Mapping to Engage Faculty Members in the Analysis
of a Pharmacy Program
Sheryl Zelenitsky, PharmD,a Lavern Vercaigne, PharmD,a Neal M. Davies, PhD,a Christine Davis,
PharmD,a Robert Renaud, PhD,b Cheryl Kristjanson, PhDa
a
b
Faculty of Pharmacy, University of Manitoba, Winnipeg, Manitoba, Canada

Faculty of Education, University of Manitoba, Winnipeg, Manitoba, Canada
Submitted January 31, 2014; accepted April 9, 2014; published September 15, 2014.
Objective. To develop a curriculum mapping process that supports continuous analysis and evidencebased decisions in a pharmacy program.
Design. A curriculum map based on the national educational outcomes for pharmacy programs was
created using conceptual frameworks grounded in cognitive learning and skill acquisition.
Assessment. The curriculum map was used to align the intended curriculum with the national educational outcomes and licensing examination blueprint. The leveling and sequencing of content showed
longitudinal progression of student learning and performance. There was good concordance between
the intended and learned curricula as validated by survey responses from employers and graduating
students.
Conclusion. The curriculum mapping process was efcient and effective in providing an evidencebased approach to the continuous quality improvement of a pharmacy program.
Keywords: curriculum mapping; learning objectives; educational outcomes; learning outcomes; competencies
intended curriculum. However, the question remains of

how to assess whether students have been given adequate learning opportunities and achieved the appropriate level of expertise. Faculty members at the University
of Manitoba developed an effective process for planning
and developing a pharmacy curriculum map that can be
used to analyze how well an intended curriculum aligns
with essential educational outcomes, and how effectively it is delivered or learned by students. The value
of curriculum mapping in engaging faculty members in
the continuous analysis and evidence-based decisions in
a pharmacy program is also demonstrated.
Curriculum mapping is generally used to evaluate an
existing or to create a new curriculum.3 It documents what
is taught (learning objectives and course content), how it
is taught (teaching methods and learning opportunities),
when it is taught (timetabling and sequencing), and what
is learned (learning outcomes and student assessment). It
also demonstrates the connections among the components
(eg, courses, discipline streams, years) of a program. Curriculum mapping can be used to demonstrate alignments
with essential educational outcomes or accreditation standards.4 Specically, it can identify gaps, redundancies, or
inconsistencies in content, learning opportunities, or student assessments.4-6 It can also guide and support faculty
members in making informed decisions regarding the
INTRODUCTION
As professional education programs, pharmacy curricula are subject to comprehensive reviews as required
by internal or external (eg, accreditation) processes. Given
the specialized knowledge and skills required for pharmacy practice, it is important to demonstrate that the
content and structure of a curriculum are appropriate.
The essential educational outcomes for pharmacy programs in Canada are outlined by The Association of
Faculties of Pharmacy of Canada (AFPC) as: care provider, communicator, collaborator, manager, advocate,
scholar, and professional.1 Each AFPC educational outcome is composed of elements and subelements such as:
1.2 Elicit and complete an assessment of required information to determine the patients medication-related
and other relevant health needs. The essential educational outcomes for pharmacy programs in the United
States are described by the Center for the Advancement
of Pharmacy Education (CAPE).2 Educational outcomes
guide pharmacy curricula in providing the learning
opportunities and setting the expectations that prepare
students for entry to practice. This is referred to as the
Corresponding author: Sheryl Zelenitsky, Faculty of
Pharmacy, University of Manitoba 750 McDermot Avenue,
Winnipeg, Manitoba, Canada R3E 0T5. Tel: 204-474-8414.
Fax: 204-474-7617. E-mail: zelenits@umanitoba.ca
64

required course or curricular change.7 Finally, curriculum
mapping can be used to analyze alignments between an
intended and learned curriculum.4,8
As described by Harden, curriculum mapping can
have numerous functions.9 For teaching faculty members,
a curriculum map can demonstrate how their course relates
to other courses and the overall curriculum. It can assist in
course design and decisions related to content and sequencing, teaching methods, and student assessment. For administrators, a curriculum map can inform program planning in
areas related to course evaluation, faculty development,
teaching assignments, and resource allocation. Finally,
for program evaluation, a curriculum map can provide
structure for informative inquiry, analysis, and continuous
quality improvement.3,4,9 It can measure how well the
intended curriculum aligns with the learned one, and by
extension, its impact on student learning and performance,
and ultimately, their professional practice.4,8,10
Curriculum mapping can be a complex process. Effective collaboration is essential, with pharmacy experts
providing the content and context and education experts
guiding and validating the process. Faculty engagement
encourages participation and reduces concerns related to
course ownership, teaching evaluations, or potential requirements for changes.3 Curriculum mapping needs to
be practical and productive, with data collection that is
purposeful and reliable. The end product needs to be informative and functional in a way that maintains faculty
interest and ownership of the process.11 Finally, the process requires support from school administration, effective leadership, and adequate resources.12
The primary goal of this study was to implement
an inclusive and comprehensive curriculum mapping
process that supported continuous analysis and evidence-based decisions in a pharmacy program. The key
objectives were: to (1) determine to what degree our
intended curriculum covered each of the AFPC educational outcomes, and (2) how our intended curriculum
aligned with what students actually learned.
(1) adopting an overly theoretical approach that does not

engage and maintain faculty support; (2) collecting inappropriate, inconsistent, or excessive data; and (3) using processes without sufcient focus on results and action.
The Curriculum Management Committee partnered
with University Teaching Services (later renamed the
Centre for the Advancement of Teaching and Learning),
which provided leadership and support for individuals
and programs at the University of Manitoba. The 1-year
process began with initial faculty discussions followed by
a series of seminars on the purpose, components, construction, and design of curriculum maps. Faculty members agreed on an approach based on mapping course
objectives to the AFPC educational outcome elements.
Faculty development sessions were conducted to review
the key components, appropriate language, and effective
format for course objectives. Further instruction was provided on linking course objectives to the relevant AFPC
educational outcome element(s).
The curriculum map was developed using conceptual frameworks grounded in the theories of cognitive
learning and skill acquisition. The former was incorporated using an educational framework called ICE (ideas,
connections, and extensions) that defined levels of learning
as: ideas (student has basic or fundamental knowledge),
connections (student uses basic concepts to make connections or form relationships), extensions (student synthesizes knowledge and uses concepts to extrapolate and
make informed decisions).13 As described by the developers, the framework. . . helps to clarify the characteristics
and markers that indicate where learners are along the
learning continuum and in doing so, enables teachers to
make instructional decisions that maximize learning.13
Teaching faculty members used the ICE framework to
assign their expectation of student learning for each AFPC
educational outcome element linked to a course objective.
Next, the process of acquiring skills and professional
attitudes was captured in a framework on the development
of expertise. Early research by Dreyfus and Dreyfus described the acquisition of skill through a series of stages,14
whereas later work by Ericsson stressed the importance of
deliberate practice in the process.15 Similar to the ICE levels
for student learning, we developed levels for student performance based on an expertise framework of: novice
(student has cognitive abilities limited to facts or does
not connect knowledge with pharmacy practice; student
requires extensive coaching or supervision in pharmacy
practice or simulation environments), functional (student
demonstrates ability to connect knowledge with pharmacy
practice; student requires minimal coaching or supervision
in pharmacy practice or simulation environments), competent (student demonstrates ability to independently and
DESIGN
Curriculum mapping was initiated by the Curriculum
Management Committee at the University of Manitobas
Faculty of Pharmacy as an ongoing process to facilitate
curriculum documentation, analysis, and continuous quality improvement. The schools 4-year degree program was
composed of 36 courses (140 credit hours) categorized into
general discipline streams of: pharmaceutical sciences,
clinical and applied sciences, and performance-based
courses and pharmacy practice (ie, experiential). Key
threats to successful curriculum mapping were identied
through consultations with local and national experts as:
65

consistently synthesize and extend knowledge to pharmacy
practice; student does not require supervision in pharmacy
practice or simulation environments). Teaching faculty
members used the NFC (novice, functional, competent)
framework to assign their expectation of student performance for each AFPC educational outcome element linked
to a course objective. The systematic approach in using
conceptual frameworks for cognitive learning and skill acquisition ensured that the learning objectives not only
aligned with the curriculum but also with the expectations
that teaching faculty members had for students. Importantly, the use of conceptual frameworks framed the acquisition of knowledge, skills, and professional attitudes in an
integrated process rather than in the mastery of separate
content areas or disciplines.
Teaching faculty members submitted information
for their individual courses, while course coordinators
collected and collated data for multi-instructor courses.
A data collection template (Table 1) was provided to link
each course objective to the relevant AFPC educational
outcome element(s) and to document the expected levels
of student learning (ICE) and performance (NFC). All
information was reviewed by the curriculum management
committee and an education expert.
Data management and analysis were conducted using Excel, whereas final versions of the curriculum map
were available in PowerPoint files that contained hyperlinks to more detailed information. The curriculum map
was available in 2 versions: an outcome-centered map that
followed the AFPC educational outcomes and a studentcentered map that followed progression through each
year of the program. Importantly, the curriculum map
captured data for every course at the level of each AFPC

educational outcome (and element). It provided robust
analytical capabilities using accessible software where
the weighting, learning levels, and performance levels
of each AFPC educational outcome (and element) could
be assessed within courses, discipline streams, years, or
the overall program. Although electronic and Web-based
curriculum management systems were available, we recognized important limitations related to inflexible platforms, specialized operating skills and costs.11,12 Thus, we
made the decision to use Microsoft Ofce software based on
its widespread access,familiarity, and cost effectiveness.

Assessing the Intended Curriculum
First, the curriculum map was used to characterize the
relative number of course objectives (weightings) linked to
each AFPC educational outcome. As summarized in Table 2,
the alignment of the curriculum with the AFPC educational
outcomes showed the highest weighting of course objectives
for care provider (35.0%), followed by scholar (27.0%), and
the lowest number for manager (4.3%). Next, the curriculum
map and weighting of course objectives were compared with
the national licensing examination constructed by the Pharmacy Examining Board of Canada (PEBC). In this case, the
weighting of course objectives were compared with the
weighting of professional competencies outlined in the examination blueprint (Table 2).16 The results showed reasonable concordance between the curriculum map and the
national licensing examination, with weightings of 35.0%
vs 38% for care provider, 27.0% vs 22% for scholar, and
15.3% vs 20% for communicator, respectively.
Table 1. Course Objectives Linked to Principal AFPC Educational Outcomes Along With Expected Learning and Performance
Levels
Course Objectives
At the completion of this course,
the student should be able to:
AFPC Educational
Outcome Achieved
Identify and connect relevant patient, clinical,

1.2 Determine patient needs
and laboratory data in patient assessment and
and desired outcomes
disease state management generally and for
1.3 Identify and prioritize
patient-specic scenarios (eg, patient
patient drug-related issues
demographics, medical conditions, co-morbidities, 1.5 Refer when appropriate
medications, allergies, clinical status,
laboratory results)
Compare and contrast (differentiate) therapeutic
alternatives considering issues related to patient
needs, clinical efcacy, potential adverse effects
and drug interactions
1.4 Develop therapeutic and

monitoring plans to achieve
optimal outcomes
2.2 Practice in a professional and
ethical manner, accountable to
the patient
66
Learning Level Performance Level

(ICE Ideas,
(NFC Novice,
Connections,
Functional,
Extensions)
Competent)
Connections
Functional
Connections
Functional
Connections
Competent
Connections
Functional
Connections
Functional

Table 2. Alignment of the Intended Curriculum With the Principal AFPCa Educational Outcomes and Weighting of Professional
Competencies in the PEBCb National Licensing Examination
AFPC Educational Outcomes for First
Degree Programs in Pharmacy (Entry-to-Practice)
in Canada
CARE PROVIDER
Use knowledge, skills, and professional judgement to
provide pharmaceutical care and to facilitate management
of patients medication and overall health needs. Elicit and
complete an assessment of required information to determine
the patients medication needs.
COMMUNICATOR
Communicate with diverse audiences, using a variety of
strategies that take into account the situation, intended
outcomes of the communication and the target audience.
ADVOCATE
Use their expertise and inuence to advance the health and
well-being of individual patients, communities, and
populations, and to support pharmacists professional roles.
COLLABORATOR
Work collaboratively with teams to provide effective,
quality health care and to fulll their professional
obligations to the community and society at large.
MANAGER
Use management skills in their daily practice to optimize
the care of patients, to ensure the safe and effective
distribution of medications, and to make efcient
use of health resources.
SCHOLAR
Have and can apply the core knowledge and skills required
to be a medication therapy expert, and are able to master,
generate, interpret, and disseminate pharmaceutical and
pharmacy practice knowledge.
PROFESSIONAL
Honor their roles as self-regulated professionals through
both individual patient care and fulllment of their
professional obligations to the profession, the community,
and society at large.
Faculty of Pharmacy
Course Objectives
[n = 790] (% weighting)
PEBC Licensing
Examination
Blueprint (% weighting)
35.0
38
15.3
20
7.7
4.3
27.0
22
10.5
10
Association of Faculties of Pharmacy of Canada

Pharmacy Examining Board of Canada
c
For comparison with PEBC competencies for Communication (20%), course objectives for Communicator (9.7%) and Advocate (5.6%)
were combined
d
For comparison with course objectives for Scholar (27%), PEBC competencies for Drug, Therapeutic and Practice Information (6%) and
Drug Distribution (16%) were combined.
b
The curriculum map was also used to characterize

the sequencing and leveling of student learning and performance through the program. An analysis of learning
levels by program year showed objectives at an introductory level (ideas) in 42% of cases in the first year, with
progression to an advanced level (extensions) in 77% of
cases in the final year (Figure 1). The most common
expected learning level was ideas in year 1 (42.0%), connections in year 2 (78.0%), and extensions in years 3 and 4
(53.0% and 76.5%, respectively). Similar progressions
were observed for the expected performance levels as students gained experience and expertise over time (Figure 2).
The most common expected performance level was novice
in year 1 (98.5%), functional in years 2 and 3 (48.0% and
67.8%, respectively) and competent in the nal year
(67.8%).
Validating the Learned Curriculum
The learned curriculum was assessed and validated
using employer surveys where participants rated their
67
Figure 1. Expected student learning level based on course objectives by program year
satisfaction with the performance and skills of recent

graduates (within 5 years) in areas related to the AFPC
educational outcomes. Overall, 71.0% of respondents
ranked the schools pharmacy graduates as very good or
excellent, 12.9% as good, and 16.1% as satisfactory. No
respondents rated the graduates as unsatisfactory. Employer feedback on specic educational outcomes is detailed in Table 3.
In addition, graduating student exit surveys and
focus groups were conducted where respondents rated
their abilities. As seen in Table 3, there was high concordance between the employer ratings and the self-perceptions of graduating students.
DISCUSSION
The curriculum mapping process was an efficient
and effective method of deconstructing and analyzing
the intended curriculum relative to the national educational outcomes for pharmacy programs and competencies for pharmacy licensure. The map showed an
68
Figure 2. Expected student performance level based on course objectives by program year
emphasis on care provider and scholar, which supported

the focus on pharmacy knowledge and professional practice. The map also demonstrated progression in the leveling and sequencing of student learning and performance
through the program. The process identified some important areas for improvement. For example, gaps in educational outcomes related to the role of manager were
triangulated with the light weighting of course objectives
on the curriculum map and low scores on both the employer

and graduating student exit surveys. The issue was investigated further in student focus groups that specified content
gaps related to business processes, human resources, and
drug plan procedures. This information was shared with the
Curriculum Management Committee, which investigated
further to determine where and at what level learning
objectives should be added to the curriculum.
69

Table 3. Validation of the Learned Curriculum as Demonstrated by the Results of Employer and Graduating Student Exit Surveys
AFPCa Educational Outcomes for First-Degree
Programs in Pharmacy in Canada
Employer Survey of
Recent Graduatesb
Graduating Student
Exit Surveyc
CARE PROVIDER
Use knowledge, skills, and professional
judgement to provide pharmaceutical
care and to facilitate management of
patients medication and overall health
needs. Elicit and complete an assessment
of required information to determine the
patients medication needs.
Novice, 6%
Functional, 63%
Skilled, 31%
Take a patient history (A 37%, SA 63%)

Recognize and prioritize drug-related
problems (A 61%, SA 37%)
Interpret relevant patient data
(A 63%, SA 38%)
Develop a therapeutic plan (A73%, SA 24%)
COMMUNICATOR
Communicate with diverse audiences,
using a variety of strategies that take into
account the situation, intended
outcomes of the communication, and the
target audience.
Novice, 16%,
Functional, 56%
Skilled, 25%
Communicate with health care team

(A 67%, SA 22%)
Communicate with patients (A 75%, SA 26%)
ADVOCATE
Use their expertise and inuence to advance
the health and well-being of individual
patients, communities, and populations,
and to support pharmacists professional roles.
Novice, 22%,
Functional, 50%
Skilled, 28%
Advocate for patients (A 65%, SA 33%)
COLLABORATOR
Work collaboratively with teams to provide
effective, quality health care and to fulll
their professional obligations to the community
and society at large.
Novice, 9%
Functional, 59%
Skilled, 28%
Work collaboratively (A 65%, SA 14%)
MANAGER
Use management skills in their daily practice
to optimize the care of patients, to ensure the
safe and effective distribution of medications,
and to make efcient use of health resources.
Novice, 41%
Functional, 38%
Skilled, 19%
Apply management principles

(DA 39%, A 55%)
Manage and maintain resources and
personnel (DA 46%, A 42%)
SCHOLAR
Have and can apply the core knowledge and
skills required to be a medication therapy expert,
and are able to master, generate, interpret, and
disseminate pharmaceutical and pharmacy
practice knowledge.
Novice, 6%,
Functional, 53%
Skilled, 38%
Use appropriate resources to access

information (A 61%, SA 33%)
Conduct systemic reviews and appraisals of
the literature (DA 33%, A 55%)
Incorporate evidence into decisions
(A 78%, SA 14%)
PROFESSIONAL
Honor their roles as self-regulated professionals
through both individual patient care and fulllment
of their professional obligations to the profession,
the community, and society at large.
Novice, 7%
Functional, 55%
Skilled, 39%
Include ethical issues into decisions

(A 78%, SA 22%)
Value honesty and integrity
(A 51%, SA 49%)
Safeguard confidentiality of patients
(A 37%, SA 63%)
AFPC is the Association of Faculties of Pharmacy of Canada.

Pharmacy employer survey conducted in 2013 (n536) where respondents rated their satisfaction with the performance and expertise of recent
graduates (within 5 years); where Novice required supervision to complete activities and knowledge base was inadequate in some areas;
Functional completed activities with minimal supervision and knowledge base was appropriate in most areas; and Skilled completed most
activities independently and knowledge base exceeded expectations for entry-to-practice.
c
Graduating student exit surveys and focus groups conducted in 2013 (n551) where respondents rated their performance abilities at the end of the
program, and where SA is strongly agree, A is agree and DA is disagree.
b
70

Our curriculum mapping process resulted in a number of positive outcomes for the school. It reinforced key
educational principles and introduced conceptual frameworks that provided a systematic approach and common
language for discussing, analyzing, and modifying the
curriculum. The inclusive approach resulted in participation from all teaching faculty members in providing the
necessary data to create a relevant and reliable curriculum
map. Information collected during the process was also
valuable for teaching faculty members to make coursespecific decisions related to the content, teaching
methods, and learning opportunities. For example, a survey of teaching faculty members identified concerns
related to teaching critical-thinking skills. A faculty
development session was conducted during which we
defined critical thinking and outlined the important skills
expected of students. These included the ability to define
questions, search and appraise relevant information,
make informed decisions, and deal with ambiguity. An
analysis of the curriculum map revealed that these skills
were covered with the exception of managing ambiguity.
This observation was validated in focus groups where students described being comfortable conducting literature
searches and appraisals but less confident dealing with
ambiguous or inconsistent information. As such, teaching
faculty members were given guidance on teaching methods
for developing critical-thinking and decision-making
skills.
The curriculum map was used to facilitate curriculum presentation and analysis in an accreditation review.
The curriculum mapping process also instilled an approach of inquiry and scholarship, which led to the creation of a formal program evaluation committee. This
extended the mandate to developing a formal process for
evaluating performance outcomes of the pharmacy program including admissions, the curriculum, teaching and
learning, and graduate performance. This work has fostered
close collaborations with committees and administrators of
other schools. Finally, the curriculum mapping experience
yielded broader institutional benefits. As the educational
principles and conceptual frameworks were relevant and
transferable, other professional programs adopted our approach to constructing curriculum maps. Our institution
also embarked on an interprofessional education and competency mapping process to align with entry-to-practice
standards of health care professionals in the local region.
Our experience confirmed some essentials for
successful curriculum mapping. Close collaboration of
experts from pharmacy and education ensured a process
that was relevant and grounded in solid principles. Faculty engagement and development improved the consistency and reliability of data used to construct the
curriculum map. Data collection and management was

facilitated by using common templates and familiar software programs. Finally, involvement and support from
administration reinforced the importance and value of
curriculum mapping.
One limitation of our original curriculum map was
that it lacked information related to teaching methods and
student assessment. Although these data were added to an
updated version, the information was only descriptive. As
such, we have embarked on a pilot project that uses the
curriculum map to determine how well student assessments (eg, examinations) align with the content, level,
and weighting of course objectives.
Another limitation was the need for more objective
external validation of alignment between the intended and
learned curricula. Although the overall performance of
graduates on the PEBC licensing examination is very good,
it is only a crude indicator of the effectiveness of the learned
curriculum. As such, we have engaged in collaborative research with the PEBC to allow more detailed analyses of our
graduates performance on each professional competency in
the licensing examination blueprint.
SUMMARY
A curriculum map embedded in solid educational
principles demonstrated how courses were integrated to
achieve an intended curriculum. It provided ongoing documentation and analysis as opposed to a snapshot description for periodic accreditation or other reviews. It
also demonstrated the longitudinal progression and scaffolding of student learning and performance through
a program. The curriculum map promoted and developed
shared responsibility for the curriculum and its mandate
of preparing students for pharmacy practice. This curriculum mapping process was efcient and effective in providing an evidence-based approach to continuous quality
improvement of a pharmacy program.
ACKNOWLEDGMENTS
The authors acknowledge the contributions of
Ms. Angela Tittle (Consultant, University Teaching Services, University of Manitoba) and Ms. Cheryl Lee (Undergraduate Program Administrator, Faculty of Pharmacy,
University of Manitoba).
REFERENCES
1. The Association of Faculties of Pharmacy of Canada (AFPC):
Educational Outcomes for rst professional degree programs in
pharmacy (entry-to-practice pharmacy programs) in Canada. http://
www.afpc.info/node/39. Accessed December 17, 2013.
2. The Advancement of Pharmacy Education (CAPE): CAPE
Educational Outcomes 2013. http://www.aacp.org/resources/
education/cape/Pages/default.aspx. Accessed March 20, 2014.
71

10. Mazurat R, Schonwetter D. Electronic curriculum mapping:
supporting competency-based dental education. J Can Dent Assoc.
2008;74(10):886-889.
11. Bell C, Ellaway R, Rhind S. Getting started with curriculum
mapping in a veterinary degree program. J Vet Med Educ. 2009;36
(1):100-106
12. Britton M, Letassy N, Medina MD, et al. A curriculum review
and mapping process supported by an electronic database system. Am
13. Fostaty Young S, Wilson R. Assessment and Learning: The ICE
Approach. Winnipeg, MB: Portage and Main Press; 2000: 9.
14. Dreyfus S, Dreyfus H. A ve stage model of the mental activities
involved in directed skill acquisition [monograph]. California
University Berkeley Operations Research Center; 1980. http://www.
dtic.mil/dtic/index.html. Accessed January 7, 2014.
15. Ericsson KA. The role of deliberate practice in the acquisition of
expert performance. In: The Cambridge Handbook of Expertise and
Expert Performance. New York: Cambridge University Press.
2006:683-703.
16. The Pharmacy Examining Board of Canada (PEBC): Pharmacist
qualifying exam blueprint. http://microsites.pebc.ca/EnglishPages/
QEX/QEXBlueprint.html. Accessed December 17, 2013.
3. Kopera-Frye K, Mahaffy J, Svare G. The map to curriculum

alignment and improvement. In: Wright A, Murray S, Wilson M, eds.
CELT: Collected Essays on Learning and Teaching. Windsor, ON:
Society for Teaching and Learning in Higher Education; 2008:8-14.
4. Kelley K, McAuley J, Wallace L, Frank S. Evaluation,
assessment, and outcomes in pharmacy education: the 2007 AACP
Institute - Curricular Mapping: Process and Product. Am J Pharm
Educ. 2008;72(5):Article 96.
5. Robley W, Whittle S, Murdoch-Eaton D. Mapping generic skills
curricula: a recommended methodology. J Further Higher Educ
2005;29(3):221-231.
6. Robley W, Whittle S, Murdoch-Eaton D. Mapping generic skills
curricula: outcomes and discussion. J Further Higher Educ. 2005;29
(4):321-330.
7. Uchiyama K, Radin J. Curriculum mapping in higher education:
a vehicle for collaboration. Innov Higher Educ. 2009;33:271-280.
8. Willett T, Marshall K, Broudo M, Clarke M. Its about TIME:
a general-purpose taxonomy of subjects in medical education. Med
Educ. 2008;42(4):432-438.
9. Harden R. AMEE Guide No.21: Curriculum mapping: a tool for
transparent and authentic teaching and learning. Med Teach. 2001;23
(2):123-137.
72

Improved Knowledge Retention Among Clinical Pharmacy Students
Using an Anthropology Classroom Assessment Technique
Heather P. Whitley, PharmD, BCPS, CDE,a,b Jason M. Parton, PhDc
a
Harrison School of Pharmacy, Auburn University, Auburn, Alabama

Montgomery Family Medicine Residency Program, Baptist Health System, Montgomery, Alabama
c
Culverhouse College of Commerce and Business Administration, The University of Alabama, Tuscaloosa, Alabama
b
Submitted January 13, 2014; accepted March 13, 2014; published September 15, 2014.
Objective. To adapt a classroom assessment technique (CAT) from an anthropology course to a diabetes module in a clinical pharmacy skills laboratory and to determine student knowledge retention
from baseline.
Design. Diabetes item stems, focused on module objectives, replaced anthropology terms. Answer
choices, coded to Blooms Taxonomy, were expanded to include higher-order thinking. Students
completed the online 5-item probe 4 times: prelaboratory lecture, postlaboratory, and at 6 months
and 12 months after laboratory. Statistical analyses utilized a single factor, repeated measures design
using rank transformations of means with a Mann-Whitney-Wilcoxon test.
Assessment. The CAT revealed a signicant increase in knowledge from prelaboratory compared to all
postlaboratory measurements (p,0.0001). Signicant knowledge retention was maintained with basic
terms, but declined with complex terms between 6 and 12 months.
Conclusion. The anthropology assessment tool was effectively adapted using Blooms Taxonomy as
a guide and, when used repeatedly, demonstrated knowledge retention. Minimal time was devoted to
application of the probe making it an easily adaptable CAT.
Keywords: Classroom assessment techniques; diabetes; skills lab; background knowledge probe; knowledge
retention; CAPE domains
learning process.2 Doing so would facilitate the ability to

use analyzed assessment data for immediate improvements in the educational modality rather than waiting until later time points, such as the nal evaluation, when
adjustments would only benet students in subsequent
semesters. Additionally, using frequent assessment techniques could demonstrate student learning changes by
comparing data from several time points. A systematic
and sequential administration of assessments will either
validate the educational modality or bring attention to
improvements needed relating to long-term learning
gains. Classroom assessment techniques (CATs) are brief
formative evaluations that are available in a large variety
of constructs. When administered at various time points,
they may be used to track student achievement at different
levels, while fostering instructor experimentation and innovation. Therefore, CATs could be used to meet the need
set forth by Standard No. 15.
CATs augment clearly defined teaching objectives
by allowing instructors to formatively determine what,
how much, and how well students are achieving learning
goals.3-5 Important characteristics of effective CATs
INTRODUCTION
In 2011, the Accreditation Council for Pharmacy
Education (ACPE) updated the Accreditation Standards
and Guidelines for the Professional Program in Pharmacy
Leading to the Doctor of Pharmacy Degree.1 Standard
No. 15, which addresses Assessment and Evaluation of
Student Learning and Curricular Effectiveness, denotes
that curricular evaluation should include varied formative and summative assessment methods that are systematically and sequentially administered to determine
students achievement at different levels and foster
experimentation and innovation.
Based on this guideline, educators might want to
consider implementing formative and summative assessment techniques in tandem with the course, module, or
lecture. These techniques should be incorporated frequently through a course, making assessment part of the
Corresponding Author: Heather P. Whitley, PharmD, BCPS,
CDE, Montgomery Family Residency Program, Baptist
Health, 4371 Narrow Lane Road, Suite 100, Montgomery, AL
36116. Tel: 334-280-7084; Fax: 334-613-3685. E-mail:
whitlhp@auburn.edu
73

include being learner-centered, teacher-directed, mutually benecial, formative, context-specic, and well integrated into the teaching and learning process.6 CATs
allow instructors to gage students knowledge or perceived knowledge of various topics before, during, or
after learning experiences. In addition to supporting instructional development, CATs also allow instructors to
focus valuable class time toward knowledge decient
areas rather than topics more comfortable for students.3,6
Due to the formative nature of CATs, educational adjustments can be made immediately rather than waiting until
the following semester. CATs also present an opportunity
for students to provide anonymous feedback about their
learning. Students particularly hesitant to ask questions
aloud in class can more comfortably communicate with
instructors by completing a CAT and may also learn that
other classmates share like concerns.3 Additionally, by
providing welcome feedback, students become more involved in their learning, motivated to successfully complete the course, and self-directed.3,4
CATs may be used to determine students prior
knowledge or ability to recall, apply, analyze, synthesize,
create, or critically consider material. The CATs most
commonly used to assess knowledge and recall are the
background knowledge probe, the one minute paper,
and the muddiest point. Instructors use the one minute
paper to allow students to respond in writing to the following two questions during the nal few minutes of class
time: What was the most important thing you learned
today in class? and What are you still confused about? 7
This technique allows instructors to compare course or
module objectives with students perceptions of important topics and their learning. Again, students respond in
writing at the conclusion of a lecture to the muddiest
point. This knowledge and recall CAT mirrors half of
the one minute paper by allowing students to respond regarding the most confusing aspect of education provided.
This helps the instructor identify points to better emphasize or explain prior to starting the next course lecture.3,6
Quite different from the one minute paper and muddiest point, the background knowledge probe CAT allows
instructors to easily collect and analyze student preparedness about a particular topic. Gathered information can
then be applied to determine the most appropriate starting
point and instruction level for a given lesson.3,6 Highlighting essential information when using a background
knowledge probe not only allows review of previously
presented material, but also provides direction for future
topics of study within a course. Background knowledge
probes require students to reect on current knowledge
and assess their understanding, often through either short
answer responses or selection of answer choices.3,6 They
are most commonly administered prior to beginning a new

course, lesson, or topic.6 However, they may also be administered immediately following an event to gain a rough
sense of improvement in knowledge base or familiarity.3,6
Although CAT examples are described in the literature, including the arena of Allied Health, most offer
anecdotal support of their impact rather than sufficient,
well-articulated detail to determine objective, outcomerelated changes.8-10 A study by Wise is one exception,
describing the process and impact of implementing the
muddiest point CAT in a 2-hour lecture-based course in
physical therapy for third-year students.11 Condentially
completed evaluations showed signicant improvements
in student perception of the course and instructor when
a CAT was administered throughout the semester compared to previous semesters, when no CAT was used.
Students responded that course assignments were reasonable and contributed to learning (p50.002), the course was
well organized (p50.007), and the instructor provided
students with opportunities for questions (p50.004).11
CATs have also been developed and implemented in
various doctor of pharmacy programs, although few studies describe specific outcomes. Van Amburgh and colleagues assessed the impact of various active-learning
techniques integrated into pharmacy classroom lectures,
including CATs, but did not specifically outline the benefits gained by the individual assessment measures.12
Other assessment review articles mention that CATs, especially the muddiest point, is often use by pharmacy
faculty, but again, few specics are provided.8,13,14
Bartlett and Morrow, however, provide a comprehensive
description of adapting and implementing the one minute
paper in a rst-year Biochemical Basics of Drugs and
Disease pharmacy course. They expanded the 2-question
CAT to include a third: What was the most interesting
fact you learned today? Course feedback gathered
through the mid-term evaluation and a student-completed
survey indicated that inclusion of the one minute paper
improved student-faculty relationships, student understanding of difcult material, and student likelihood of
asking questions in class (with women feeling more encouraged than men to ask questions in class) (p,0.01).15
Additionally, students believed the CAT was an effective
use of class time.15
While limited literature outlines the implementation
and assessment process of including a CAT in pharmacy
or other health-related disciplines, findings do tend to
point to improvements in either student perception of
the course, content, or instructor. The current study investigates the process of translating a background knowledge probe CAT from another discipline (anthropology6)
to a pharmacy-specic course. Additionally, it is the rst
74

assessment to objectify changes in student knowledge
retention over a 12-month interval using this type of background knowledge probe.
indicating varying degrees of familiarity, which were

nearly identical for each item. Prior to administering the
in-class, paper-based probe, students were informed that it
was not a test and would not be graded. The results were
used to improve student learning during the semester.6
(Table 1)
Because the background knowledge probe was adapted from a semester-length course to a brief module, the
number of items tested was decreased from 50 to 5. Tested
item topics were selected from and focused on module
objectives and were presented in order of appearance during the prelaboratory and laboratory sections and in increasing complexity. The wording of answer choices was
kept as closely as possible to the original anthropology
examples. To more accurately translate to applied aspects
of patient care, 1 additional option was added to the first 3
items and 2 additional options were added to the last 2
items. To accurately analyze the anthropology probe,
each answer choice was coded to Blooms Taxonomy to
indicate degree of higher order thinking (Table 1). The
rst answer choice (Have not heard of this) was not
assigned a level of Blooms Taxonomy, as it indicates
that no knowledge had been acquired on the topic. The
second and third answer choices were both coded as
Knowledge, while the fourth answer choice was coded
as Comprehension. The same method of coding answer
choices to Blooms Taxonomy was applied to the adapted
diabetes probe (Table 2). Two independent individuals,
an education assessment expert, and a course coordinator
skilled in assessment techniques assisted in the coding of
the answers. Answer choices were listed in order of increasing level of understanding. Four 4th-year pharmacy
students participated in cooperative inquiry to validate the
utility of the survey and to ensure accurate interpretation.
The 5-item probe was reformatted as a Vovici online
questionnaire (Vovici Corporation, Herndon, VA), in
which knowledge levels were ranked to represent participants knowledge of the diabetes term.
All third-year pharmacy students enrolled in the
mandatory clinical pharmacy skills laboratory during
the fall of 2011 and 2012 were e-mailed an informational
DESIGN
Third-year pharmacy students attended (or later
viewed the recording of) a 1-hour prelaboratory lecture
on type 1 diabetes at Auburn University in a large classroom setting. The off-campus instructor used a video conference system (Polycom) to present background content
to students. Whether students attended the real-time prelaboratory lecture or later watched the recorded presentation, they received identical content. Over the following 7
days, students completed a homework assignment of photographing their dinner, counting carbohydrates in the
meal, and calculating a dose of rapid-acting insulin based
on a given insulin-to-carbohydrate ratio. A week later
students attended a 2-hour clinical pharmacy skills laboratory (approximately 32 students per section), where
they participated in hands-on, active learning, case-based
education. The laboratory included 4 carbohydrate counting exercises, presentation of 4 meal photographs per
laboratory section with discussion, and 10 patient cases
requiring insulin dosing. Each exercise was conducted
through the think-pair-share method.
After completing the 1-hour prelaboratory and 2hour clinical pharmacy skills laboratory, students were
expected to have met the following learning outcome
objectives: (1) Comprehension: identify the foods most
likely to increase blood glucose; (2) Application: apply
carbohydrate counting to food labels; (3) Comprehension: describe the plate method; (4) Knowledge: define
basal-bolus insulin regimen; (5) Analysis: compare and
contrast basal-bolus regimens to pre-mixed insulin products; (6) Knowledge: dene the term insulin sensitivity
factor; (7) Knowledge: dene the term insulin-tocarbohydrate ratio. The above learning outcome objectives were used through the 2011 fall semester (2013
graduating class). The learning outcome objective regarding the plate method (objective 3) was removed from the
module in the fall of 2012 (2014 graduating class) and
inserted into a laboratory module occurring later in the fall
semester. Therefore, content regarding the plate method
was also removed from the prelaboratory and laboratory
sections of the 2012 fall module.
The original anthropology background knowledge
probe contained 50 items and was administered on the first
day of a semester-long course.6 Tested topic items were
recommended by prerequisite and lower-level course instructors. Each item consisted of a term (for example The
Weimar Republic, Senator Joseph McCarthy, or The
Golden Triangle), followed by 4 possible answer choices
Table 1. Anthropology Item with Coded Ranking

Classifications per Blooms Taxonomy
The Weimar Republic
Have
Have
Have
Have
never heard of this (N)

heard of it but dont really know what it means (K)
some idea what this means , but not too clear (K)
a clear idea what this means and can explain it (C)
None (N)
Knowledge (K)
Comprehension (C)
75

Table 2. Student Self-Reported Familiarity with Diabetes Terms Using the Background Knowledge Probe and Coded to Blooms
Taxonomya Number (%)
Term (Blooms Taxonomy)
Response Rate (n5281 number potential participants)
Carbohydrate
Have never heard of this term (N)
Have heard the term, but do not know what it means (K)
Have some idea what this is, but am not too clear (K)
Have a clear idea what this is and can list some examples (C)
Have a clear idea what this is, can list some examples,
and can explain how it relates to diabetes (AP)
Before
Prelab
Immediately
After Lab
6 Months
After Lab
12 Months
After Lab
194/(281-12)
(72.1)
230/(281-12)
(85.5)
203/281
(72.2)
221/281
(78.6)
1 (0.5)
0 (0)
12 (6.2)
75 (38.7)
106 (54.6)
0 (0)
0 (0)
0 (0)
11 (4.8)
219 (95.2)
0 (0)
0 (0)
0 (0)
8 (3.5)
195 (96.1)
2 (0.5)
0 (0)
0 (%)
15 (6.8)
204 (92.3)
Plate Method
Have never heard of this phrase (N)
Have some idea what this means, but am not too clear (K)
Have a clear idea what this means and can explain it (C)
Have a clear idea what this means, can explain it, and
could use it in patient care (AP)
89
32
44
16
11
(45.7)
(16.5)
(22.7)
(8.2)
(5.7)
4
10
34
39
142
(1.7)
(4.3)
(14.8)
(16.6)
(61.7)
0 (0)
0 (0)
15 (7.4)
34 (16.7)
152 (74.7)
2 (0.5)
0 (0)
16 (7.2)
44 (15.9)
159 (71.9)
Basal-Bolus Insulin Regimen

Have heard the term but, do not know what it means (K)
Have a clear idea what this means, can explain it, and can
develop a basal-bolus insulin regimen (S)
58
62
52
13
8
(25.9)
(31.6)
(26.6)
(6.7)
(4.1)
0 (0)
0 (0)
1 (0.4)
21 (9.1)
208 (90.4)
0 (0)
0 (0)
2 (0.6)
33 (16.3)
167 (62.3)
1 (0.5)
0 (0)
3 (1.4)
38 (17.2)
178 (80.5)
46
51
78
12
2
(23.7)
(26.3)
(40.2)
(6.2)
(1)
0 (0)
0 (0)
1 (0.4)
3 (1.3)
23 (10)
Insulin-to-Carbohydrate Ratio
can calculate it (AP)
Have a clear idea what this means, can explain it,
can calculate it, and could use it in patient care (E)
3 (1.5)
Insulin Sensitivity Factor or Correction Factor

can calculate it (AP)
Have a clear idea what this means, can explain it,
can calculate it, and could use it in patient care (E)
40
74
69
4
4
(20.6)
(38.1)
(35.6)
(2.1)
(2.1)
2 (1)
1
1
8
30
35
(0.5)
(0.5)
(3.5)
(14.8)
(17.2)
203 (88.3)
141 (69.5)
0 (0)
0 (0)
5 (2.2)
1 (0.4)
27 (11.7)
1 (0.5)
0 (0)
6 (2.6)
30 (14.8)
30 (14.8)
196 (85.2)
137 (67.5)
1
20
45
43
0 (0)
(0.5)
(9)
(20.4)
(19.5)
112 (50.7)
1
27
42
44
0 (0)
(0.5)
(12.2)
(19)
(15.9)
107 (48.4)
Data collected September 2011 through September 2013 for 2013 and 2014 graduating classes
N- None, K Knowledge, C Comprehension, AP Application, AN Analysis, S Synthesis, E Evaluation
letter two days prior to the prelaboratory lecture explaining the voluntary, anonymous nature of participation and
containing a link to the online 5-item questionnaire. The
letter explained that participation in the questionnaire

would not affect laboratory or course grades and would
be used for teaching purposes only, so using outside
76

resources to complete the questionnaire would not benefit
students. The questionnaire was closed and data analyzed
30 minutes before the prelaboratory class lecture. Results
were shared with students during the 1-hour lecture period. Students were e-mailed a second, third, and fourth
request to complete identical surveys immediately after
the 2-hour laboratory section, which occurred 1 week
later, at 6 months, and lastly at 12 months, respectively.
Students were allowed 10 days to complete the follow-up
questionnaires and were sent 3 reminder e-mails to improve response rates. The average response rate for each
familiarity statement was calculated for all 4 phases of
measurement.
Data for each familiarity statement consisted of ranking classifications based on Blooms Taxonomy. In order
to account for nonparametric trait outcomes, rank transformations were applied to the knowledge responses of
the 5 terms. Using these ranks, a single factor repeated
measures analysis was applied to examine studentreported familiarity and retention over the 4 time-point
measurements from the prelaboratory through 12-month
follow-up. SAS version 9.4 (SAS Institute, Cary, NC) was
used for all analyses. The project received Institutional
Review Board approval through exempt procedures at
Auburn University.
Additionally, loss of knowledge retention was most notable with more complex terms. Table 4 denotes change
in knowledge retention over time. Signicant knowledge
improvements were noted for all 5 terms from prelaboratory through the 12-month assessment point
(p,0.0001). This change in knowledge (per the negative
difference values) demonstrated a steeper improvement
with more complex terms possibly due to a lower
Blooms Taxonomy baseline starting point (Table 4 and
Figure 1). The most basic term, carbohydrate, shows no
signicant change in familiarity from postlaboratory
through 12 months (p51), indicating no signicant loss
of knowledge. On the other hand, the most complex terms
of insulin-to-carbohydrate ratio and insulin sensitivity
factor demonstrate a signicant decline in knowledge
retention (per the positive difference values) when the
assessment time points occurring after the prelaboratory
were compared to each other (p,0.0001). Knowledge
retention for the term basal-bolus displayed a more balanced change across the time points with a signicant
decline seen when comparing the 12-month assessment
to the postlaboratory (p50.017) and 6-month (p,0.0001)
assessment (Table 4).
Change in knowledge and familiarity with the term
plate method exhibited a uniquely different trend over
time. This was the only term with results indicating
knowledge continuing to increase from postlaboratory
through the 6-month follow-up and a non-signicant
knowledge decline from the 6-month assessment to the
12-month assessment. Removal of plate method content from the module during the 2011 fall semester and
placing it in a module later in the semester explained this
knowledge change. When the data for plate method was
separated based on year of collection, a stark difference
was noted in Wilcoxon scores at prelaboratory, postlaboratory, 6 months, and 12 months, with the 2013 graduating
class scoring 59.3, 248.2, 221.8, and 214.5, respectively,
and the 2014 class scoring 99.8, 229.3, 313.9, and 307,
respectively. The removal of plate method content from
the module allowed the term to function as a control by
comparing data between the two years. This demonstrated
the sensitivity of the background knowledge probe to
capture data differently when term-related education
was removed.
Collectively, this data indicated a significant knowledge gain occurred from the prelaboratory to postlaboratory assessment time points, but most dramatically for
more complex terms. Conversely, knowledge regression
was steeper with more complex terms, but was retained
with basic terms. Finally, the probe demonstrated sensitivity to change in knowledge when new information was
gained at later time points.

Responses from both graduating classes were combined for analysis to evaluate student learning. A total of
281 third-year pharmacy students were enrolled in the
2011 and 2012 fall course. Twelve student e-mails were
returned in the fall of 2011 during the prelaboratory and
immediate postlaboratory survey due to changes in the
universitys e-mail system. Response rates for each time
point of measurement were 72.1%, 85.5%, 72.2%, and
78.6%, respectively. Results from the student sample
showed a dramatic improvement in self-reported term
familiarity from prelaboratory to each time point assessment after the laboratory, including a shift to higher order
learning per the application of Blooms Taxonomy to the
background knowledge probe (Table 2). When changes
in familiarity were coded based on Blooms Taxonomy
using a Mann-Whitney-Wilcoxon test, signicant increases (p,0.0001) were noted for all 5 terms over the
12-month assessment period when compared to the prelaboratory measurement (Table 3). This shift was most
pronounced for more complex terms such as insulinto-carbohydrate ratio and insulin sensitivity factor,
which overlay each other in Figure 1, vs the most basic
term carbohydrate. The modest, but still signicant,
improvement from prelaboratory with carbohydrate
was likely due to a stronger baseline familiarity.
77

Table 3. Single Factor Repeated Measures for Background Knowledge Probe
Carbohydrate
PreLab
PostLab
6 Month
12 Month
Plate Method
PreLab
PostLab
6 Month
12 Month
Basal-Bolus
PreLab
PostLab
6 Month
12 Month
PreLab
PostLab
6 Month
12 Month
Insulin Sensitivity Factor
PreLab
PostLab
6 Month
12 Month
Wilcoxon Score
Standard Error
F Statistic
df
p Value
293.9
472.5
476.
459.6
9.5
8.9
9.5
9.1
88.1
,0.0001
155.3
472.2
534.4
521.3
11.6
10.8
11.6
11.1
240.8
,0.0001
121.7
539.6
513.1
505.8
8.5
7.9
8.4
8.1
568.9
,0.0001
121.1
588.8
523.7
445.3
9.9
9.2
9.8
9.4
454.5
,0.0001
125.5
587.8
522.7
445.4
10.3
9.6
10.2
9.7
413.2
,0.0001
DISCUSSION
Analysis, although it did reach Synthesis and

Evaluation.
The background knowledge probe presented here is
not a traditional multiple choice quiz with one correct
answer in addition to several distractors. Rather, through
the answer selection process, students could reflect and
consider their level of understanding and abilities. Furthermore, completing the background knowledge probe
periodically gave students a way to reflectively consider
improvements in their learning and abilities throughout
the 12-month assessment interval.
Unlike many studies analyzing knowledge retention,
the present investigation demonstrated a trend in knowledge enhancement at several time points. Other studies
demonstrate immediate increases in student understanding directly following a specific educational intervention16,17 or document knowledge or skill retention from
baseline to a sole second time point months after the educational intervention.18,19 Study designs that collect data
at only 2 time points limit the ability to evaluate knowledge retention trends, whereas assessing knowledge more
often over the same or longer intervals provides more
meaningful data regarding knowledge retention. Two
pharmacy student investigations assessed knowledge retention at 3 time points. Kopacek and colleagues aimed to
The highest level of understanding measured in the

anthropology probe, per Blooms Taxonomy, regardless
of actual achievement, was Comprehension (Table 1).
This limited assessment range may not accurately represent the true baseline level of understanding occurring at
higher orders. Secondly, as the anthropology knowledge
probe stood, it would be difcult to measure much depth
in understanding improvements for this same reason; thus
the utility of the original probe was further limited for
follow-up efforts to measure changes. While objectives
for the anthropology course were unknown, the objectives
for the diabetes laboratory pushed students toward Application and Analysis in addition to lower orders of
understanding. This further supported the importance of
altering the probe to better assess higher orders of understanding. For these reasons, it was essential during the
adaptation process of the probe from anthropology to
clinical pharmacy, to add more answer choices to better
measure higher orders of understanding. Furthermore, it
was recommended that instructors strategically map
learning activities, objectives, and learner knowledge
for CATs to Blooms Taxonomy to stimulate higher-order
thinking.9 While the adapted background knowledge
probe met this need, it failed to specically measure
78

Table 4. Difference between Wilcoxon Scores for Time Points of Measurement
Difference
(95% CL)
p Valuea
-178.6
-182.1
-165.7
-3.5
12.9
16.4
(-212.8, -144.3)
(-217.4, -146.8)
(-200.3, -131.2)
(-37.7, 30.6)
(-20.6, 46.3)
(-18.1, 50.9)
,0.0001
,0.0001
,0.0001
1.0000
1.0000
1.0000
PostLab
6 Month
12 Month
6 Month
12 Month
12 Month
-317.0
-379.2
-366.1
-62.2
-49.1
13.1
(-359.0, -274.9)
(-422.6, -335.8)
(-408.5, -323.7)
(-104.2, -20.3)
(-90.0, -8.2)
(-29.2, 55.4)
,0.0001
,0.0001
,0.0001
0.0006
0.05
1.0000
PostLab
6 Month
12 Month
6 Month
12 Month
12 Month
-417.9
-391.4
-384.1
26.5
33.8
7.3
(-448.4, -387.3)
(-422.9, -359.8)
(-415.0, -353.2)
(-4.0, 57.0)
(3.9, 63.6)
(-23.6, 38.1)
,0.0001
,0.0001
,0.0001
0.13
0.02
1.00
PreLab
PostLab
PreLab
6 Month
PreLab
12 Month
PostLab
6 Month
PostLab
12 Month
6 Month
12 Month
-467.7
-402.6
-324.2
65.1
143.5
78.4
(-503.5, -431.9)
(-439.5, -365.8)
(-360.3, -288.1)
(29.5, 100.7)
(108.7, 178.3)
(42.5, 114.3)
,0.0001
,0.0001
,0.0001
,0.0001
,0.0001
,0.0001
Insulin Sensitivity Factor

PostLab
6 Month
12 Month
6 Month
12 Month
12 Month
-462.3
-397.2
-319.9
65.1
142.3
77.3
(-499.4, -426)
(-435.4, -359.0)
(-357.3, -282.5)
(28.1, 102.0)
(106.2, 178.5)
(40.00, 114.5)
,0.0001
,0.0001
,0.0001
,0.0001
,0.0001
,0.0001
PreLab
PreLab
PreLab
PostLab
PostLab
6 Month
Carbohydrate
PostLab
6 Month
12 Month
6 Month
12 Month
12 Month
Plate Method
PreLab
PreLab
PreLab
PostLab
PostLab
6 Month
Basal-Bolus
PreLab
PreLab
PreLab
PostLab
PostLab
6 Month
PreLab
PreLab
PreLab
PostLab
PostLab
6 Month
a
p-values are based on the Bonferroni correction for multiple comparisons.
assess P2 pharmacy student retention of knowledge about

automated external debrillator use following a didactic
training and simulated experience using identical questionnaires administered at baseline, 3 weeks, and 4
months following the intervention.20 While signicant
improvements were noted in 2 out of 5 knowledge measures and all 6 performance measures at 3 weeks, the
statistical difference declined to zero knowledge and 2
performance measures by 4 months.20 The authors concluded that the intervention was not sufcient to improve
student knowledge at 4 months, and recommended
incorporating short refresher courses into the curriculum

to enhance knowledge retention.20 Morello and colleagues aimed to evaluate P1 pharmacy student condence and knowledge retention about diabetes self-care
education using a performance case-based knowledge test
administered at baseline, immediately after, and 9 months
after the educational intervention, which consisted of lectures, active learning assignments, and workshops.21
While no inferential statistical tests were performed,
the average overall percent correct on the knowledge
test nearly doubled from baseline (39.5%) to the test
79
Figure 1. Change in Familiarity Over the Two 12-Month Long Assessment Intervals.
administered immediately following the educational intervention (85%), but declined by 9 months postintervention (76%).21 Collectively, both studies indicated that
while knowledge improved as expected immediately following an educational intervention, retention declined
over time.20,21 The more extensive study design of the
present investigation, which used the background knowledge probe at 4 time points (3 following the educational
intervention), more fully illustrated that knowledge decline is not always stable or consistent throughout the time
frame (Figure 1). Certainly, additional investigations are
warranted in the realm of knowledge retention, but in our
study, knowledge did appear to decay at a steeper rate for
more complex terms.
Although the number of terms evaluated with the
adapted clinical pharmacy probe was decreased to 5, compared to the 50 terms evaluated with the anthropology
course, the probe addressed understanding for each key
concept outlined in the module objectives. Use of an
online survey was likely a more efficient use of technology
than the paper method used for the anthropology probe.
Lastly, providing the survey prior to the prelaboratory
lecture and after the laboratory module reserved class time
for course work.
There are 2 drawbacks to using this assessment. First
a considerable amount of time was put towards the
upfront development of the probe, which is comparable

to another investigation in which 71% of instructors
reported more preparation time was needed to develop
active learning teaching efforts.12 Through the translation
of the probe from an anthropology class, time was predominately devoted to the application of Blooms Taxonomy and efforts to expand ability to assess deeper levels
of thinking. If adapted for another clinical pharmacy
course, the primary effort would likely be devoted to selection of the key terms, resulting in a relatively short
development stage. Secondly, the probe provided subjective measurements, which may have varied from student
to student. However, the probe likely has reliable intrasubject variability, which would provide a reasonable prediction of true student change in familiarity.
There are several benefits to using this background
knowledge probe. After the translation process was complete, administering the probe resulted in little impact on
workload. The probe was formatted into an electronic
survey tool, which took minimal effort and would for
anyone using relatively simple and familiar software. It
was easy to implement because students completed the
assessment via an e-mail link. It was only 5 items long and
multiple choice, so it placed little burden on students who
chose to participate. Because it was electronically administered, analyzing results and returning them to students
80

was easy. Finally, because the probe was administered
prior to class, results were analyzed and reported to students during the prelaboratory lecture, which also took
little time. The lack of time required to administer this
probe is significant because when asked, most pharmacy
instructors elect not to incorporate active learning or assessment tools into didactic course work because their
belief that it is too time intensive and integrated at the
cost of lecture content outweighs their belief that it improves knowledge retention and student engagement.12
This concern expands beyond pharmacy faculty members to nursing faculty members, who also perceive it
as a barrier to implementing critical thinking strategies.22
Instead of conducting the probe as a preclass assignment, one could administer the assessment during the
beginning of class using electronic clickers, which provide instant responses. This method would likely not
detract much time from the lecture, possibly increase
the response rate (assuming all students attend the lecture), and potentially increase student satisfaction, attentiveness, and involvement.23 However, implementation
of an in-class probe utilizing clickers would only be
reasonable for the baseline time point and immediate
follow-up measurement.
Adding more time between completion of the probe
and the prelaboratory lecture could allow instructors to
adjust content to target knowledge gaps revealed in the
baseline results. Alternatively, instructors could use
baseline results to adjust laboratory (versus prelaboratory) content to target knowledge deficits. However,
course coordinators require faculty members to turn in
content for laboratories several weeks in advance, which
hinders acute adjustment of material. Still, the results
allow for content of future years to focus on past student
deficits in baseline knowledge, which in this case,
appeared to revolve around every term except for
carbohydrate.
education assessment techniques and Paul Jungnickel,

PhD, RPh, Associate Dean and Professor for Academic
and Student Affairs for his expertise in CAPE domain
interpretation and application.
REFERENCES
1. Accreditation Council for Pharmacy Education, Accreditation
Leading to the Doctor of Pharmacy Degree. 2011:39-40. https://
www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf. Accessed
July 11, 2013.
2. Zlatic TD. Abilities-based assessment within pharmacy education:
preparing students for practice of pharmaceutical care. In: Wilkin,
N.E., ed. Handbook for Pharmacy Educators: contemporary teaching
principles and strategies. New York: Pharmaceutical Products Press,
2000:5-27.
3. Mihram D. Classroom Assessment Techniques. University of
Southern California; Center for Excellence in Teaching. John
Hopkins Bloomberg School of Public Health. http://www.crlt.umich.
edu/sites/default/les/resource_les/
ClassroomAssessmentTechniquesHopkins.pdf. Accessed February
17, 2014.
4. Melland HI, Volden CM. Classroom assessment: linking teaching
and learning. J Nurs Education 1998;37(6):275-277.
5. Cross KP. Classroom research: implementing the scholarship of
teaching. Am J Pharm Educ. 1996;60:402-407.
6. Angelo TA, Cross KP. Classroom Assessment Techniques: A
Handbook for College Teachers. San Francisco, CA: Jossey-Bass, 1993.
7. Weaver RL, Cotrell HW. Mental aerobics: The half-sheet
response. Innovative Higher Educ. 1985;10(1):23-31.
8. Simpson-Beck V. Assessing classroom assessment techniques.
Active Learn High Educ. 2011;12(2):125-132.
9. Davidson JE. Preceptor use of classroom assessment techniques to
stimulate higher-order thinking in the clinical setting. J Contin Educ
Nurs. 2009;40(3):139-143.
10. Bowles DJ. Active learning strategies. . .Not for the birds! Int J
Nurs Educ Scholarsh. 2006;3(1):Article 22.
11. Wise HH. Student perceptions of a lecture-based course
compared with the same course utilizing a classroom assessment
technique. J Phys Ther Educ. 2004:18(1):75-79.
12. Van Amburgh JA, Devlin JW, Kirwin JL, Qualters DM. A tool
for measuring active learning in the classroom. Am J Pharm Educ.
2007;71(5):Article 85.
13. Anderson HM, Moore DL, Guadelupe A, Bird E. Student
learning outcomes assessment: a component of program assessment.
Am J Pharm Educ. 2005;69(2):Article 39.
14. Boyce EG. A guide for doctor of pharmacy program assessment.
Alexandria, VA: American Association of Colleges of Pharmacy;
2000. http://www.tsu.edu/academics/colleges__schools/
College_of_Pharmacy_and_Health_Sciences/assessment/pdf/
PharmDProgramAssessment.pdf. Accessed August 29, 2014.
15. Bartlett MB, Morrow KA. Method for assessing course
knowledge in a large classroom environment: an improved version of
the minute paper. Am J Pharm Educ. 2001;65:264-267.
16. Burkiewicz JS, Bruce SP, Weberski JA, Ritter JL, Sohn AH. Preand post-rotation assessment of pharmacy student learning. J Pharm
Teaching. 2005;12(2):83-96.
17. Brown BK, Watkins TA. Using formative content analysis to
improve learning outcomes in a pharmaceutical care course. J Pharm
Teaching. 2005;12(1):11-22.
SUMMARY
The adaptation of an anthropology background
knowledge probe was effectively translated to a clinical,
diabetes-focused, pharmacy skills laboratory using
Blooms Taxonomy as a guide. The probe showed improvements in and retention of student familiarity and
understanding of 5 diabetes related terms. This is the rst
objective assessment measuring knowledge retention using a CAT-designed background knowledge probe.
ACKNOWLEDGMENTS
Kristen Helms, PharmD, BCSP, Associate Clinical
Professor at Auburn University and Sharon McDonough,
PhD at University of Tennessee for their expertise in
81

21. Morello CM, Neighbors M, Luu L, Kobayashi S, Mutrux B,
Best BM. Impact of a rst-year student pharmacists diabetes
self-care education program. Am J Pharm Educ. 2013;77(10):
Article 215.
22. Shell R. Perceived barriers to teaching for critical thinking by
BSN nursing faculty. Nursing and Health Care Perspectives.
2001;22(6):286-291.
23. Graeff EC, Vail M, Maldonado A, Lund M, Galante S, Tataronis
G. Click it: Assessment of classroom response systems in physician
assistant education. J Allied Health. 2011;40(1):e1-5.
18. Hegener MA, Buring SM, Papas E. Impact of a required

pharmaceutical calculations course on mathematics ability
and knowledge retention. Am J Pharm Educ. 2013;77(6):Article
124.
19. Eley JG, Birnie C. Retention of compounding skills among
pharmacy students. Am J Pharm Educ. 2006;70(6):Article 132.
20. Kopacek KB, Dopp AL, Dopp JM, Vardeny O, Sims JJ.
Pharmacy students retention of knowledge and skills following
training in automated external debrillator use. Am J Pharm Educ.
2010;74(6):Article 109.
82

An Active Learning Complementary and Alternative Medicine
Session in a Self-Care Therapeutics Class
Melissa J. Mattison, PharmD, Eric C. Nemec, PharmD, BCPS
College of Pharmacy, Western New England University, Springfield, Massachusetts
Objective. To provide an interactive, non-supplement based complementary and alternative medicine

(CAM) session in a self-care therapeutics class and to evaluate the effect of the session on pharmacy
students perceptions and knowledge of CAM.
Design. Second professional year pharmacy students enrolled in a required 3-credit course titled Self-Care
Therapeutics participated in an active learning session on CAM. Students physically engaged in 5 separate
active learning CAM sessions including massage therapy, Tai Chi, yoga, progressive muscle relaxation, and
Reiki.
Assessment. Students were assessed on both knowledge and perception of CAM. Concept mastery was
assessed using a written examination and individual readiness assurance tests (iRAT) and team readiness
assurance tests (tRAT). Perception of CAM was measured using both a presession and a postsession
survey.
Conclusion. Participating in an intensive, active learning CAM session provided an opportunity to increase
students knowledge of CAM and an effective strategy for providing the learner with the experience to
better envision incorporation into patient therapies.
Keywords: Complementary and alternative medicine (CAM), active learning, self-care
practices.3-5 Many health care professionals perceive their

knowledge of CAM as limited and have little personal experience with it.6 Pharmacists typically rate their own
knowledge of CAM as inadequate and feel uncomfortable
answering patient questions related to it.7-10
Yet the 2002 National Health Interview Survey
reported 50% of all adults in the United States have used
some form of CAM.11 In 2007, adults in the United States
spent nearly $34 billion on complementary and alternative
medicine Thanks Good Catch products, including natural
products, classes, and visits to CAM providers.12 Between
2002 and 2007, increased use was seen among adults for
acupuncture, deep breathing exercises, massage therapy,
meditation, naturopathy, and yoga.13
Increased use of natural products and alternative
therapies could have a significant impact on the practice
of pharmacy in the United States, which would necessitate practitioners to be better versed in CAM and its
complementary use in Western medicine.11 For example, potential drug interactions with CAM could become
a concern because some patients might not disclose the
herbal medications they are taking. This in turn could
impact health and the health care system by resulting in
additional adverse outcomes. Pharmacists are the most
accessible health care professional, and if they need to
INTRODUCTION
Products and treatments labeled alternative medicine by Western medicine are actually predominant modalities of health care in traditional Asian cultures or
Eastern medicine.1 Over the years, many people have
been merging Eastern medicine with Western medicine,
integrating CAM therapies with conventional medical
treatments, but the former are not always embraced by
Western medicine.2 While healthy adults as well as ill
patients use CAM to maintain or improve health and reduce disease risk, they arent able to easily integrate it
with their traditional therapy because their health care
providers lack knowledge or awareness of it.2
Alternative medicine or therapies are not widely taught
in medical or pharmacy schools in the United States, nor are
they available at many American hospitals or health care
facilities.1 Studies surveying American pharmacy schools
concluded that while approximately 80% of schools offer
some form of CAM training in the curriculum, CAM education was primarily offered as electives and generally focused on natural products rather than the full range of CAM
Corresponding Author: Melissa J. Mattison, 1215 Wilbraham
Road, Springeld, MA 01119. Tel: 413-796-2428. Fax: 413796-2266. E-mail: mmattison@wne.edu
83

eld questions regarding CAM, they could positively
impact a patients health through counseling and an adequate CAM knowledge base.
Pharmacists perceive barriers to providing CAM guidance that include a lack of suitable training and deficiencies
in available information sources. Recognizing this knowledge deficit, pharmacists have proposed ways to overcome
these barriers, such as more extensive formal training.14 For
example, the Accreditation Council for Pharmacy Education (ACPE) 2007 Standards for Accreditation recommended didactic coursework in dietary supplements, alternative
medical treatments, evaluation of efcacy and safety of
CAM, and herbal-drug interactions, as well as exposure to
these products during advanced pharmacy practice experiences.15 Guideline 12 of the ACPE Standards focuses on
professional competencies to provide patient care based
on therapeutic principles and evidence-based data and to
promote health improvement, wellness, and disease prevention.15 Guideline 13.3 states that the college or school
curriculum should address issues related to a general broadening of perspective including communication skills, professionalism, critical thinking, problem-solving, health and
wellness, patient safety, teamwork, mathematical skills,
and information management. Moreover, the Center for
the Advancement of Pharmaceutical Education (CAPE)
Outcomes also suggests students be well versed in traditional and CAM modalities for their APPEs so they are able
to recommend prescription and nonprescription medications, dietary supplements, diet and nutrition guidance,
traditional nondrug therapies, and complementary and alternative therapies.16
Current pharmacy school offerings vary, from a small
amount of CAM coursework to electives, to more comprehensive coursework. In order for pharmacists to promote
health, wellness, and disease prevention among patients
increasingly inclined to use CAM, they need better formal
training and experience. Patients seek out CAM either as
therapy complementary to their traditional treatment or
as alternative therapy that would encompass modalities
other than traditional ones. Patients use CAM to promote
a healthy lifestyle, achieve wellness, and reduce disease
risk. Improving CAM offerings to students would not only
help to meet such patient needs, but would also directly
align with the needs addressed by the ACPE Standards
for Accreditation and the CAPE recommendations.15,16
sessions. During the CAM session, learners participated in

5 intensive, 15-minute physically active mini-sessions, each
incorporating a different CAM modality. Experts from our
institution and the community led learners through each
session in their respective specialties at separate CAM Stations. Specialties included massage therapy, Reiki, Tai Chi,
yoga, and progressive muscle relaxation (PMR). Learners
were expected to gain foundational knowledge, make connections between assigned readings and application, and
nally apply the knowledge to cases on their nal exam.
Session scheduling was based on practitioner availability
and the willingness of volunteer practitioners (as there was
no guaranteed honorarium, which may have inuenced the
representation of CAM modalities).
Prior to the interactive CAM session, learners were
offered a voluntary survey approved by the Institutional
Review Board. They were then assigned readings from
their textbook, the Handbook of Nonprescription Drugs,
and other materials provided by the various specialists.
They also completed a case-based, CAM individual readiness assurance test (iRAT). The iRAT was modied from
its traditional use of delivering it during class and was
completed outside of class to allow for more time to participate in the CAM sessions. Throughout the semester the
iRAT/tRAT (or team readiness assurance test) was given to
assess the weekly self-care topics to ensure that learners
were reading ahead and prepared for class. The survey
served to assess learners perceptions and knowledge about
their understanding of, experience with, and comfort with
incorporating CAM into a treatment. Additionally, the survey was meant to assess if learners would be more likely to
recommend CAM to an appropriate patient after the intensive learning experience.
Sixty-nine learners participated and were divided into
two sections, and further separated into groups of 6-7 to
facilitate interactive, small group learning experience. To
accommodate this, the session required more than the normal allotted time. Instructors were allowed to incorporate
one of the professional development hours that preceded
the Self-Care Therapeutics class into class time in order to
allow enough time for each CAM mini-session and give 1.5
hours to each of the 2 sections. Learners rotated through 15
minutes of experiential time in each CAM station and were
allotted 2 minutes to change stations. A facilitator helped
coordinate the rotations.
Before participating, students signed a waiver that
mitigated outside practitioner responsibility. Due to the
physical component (actually doing yoga, practicing Tai
Chi, etc), participation was optional. For learners unable or
unwilling to participate, an alternative written CAM exercise was designed to enhance learner knowledge. Two
learners opted for the written assignment. The alternative
DESIGN
An immersive CAM session was offered in the required
3-credit Self-Care Therapeutics class during the spring of the
second professional year. The instructional design of the
self-care class included a combination of case-based didactic learning, team-based activities, and active learning
84

assignment involved searching the primary literature for
quality, evidence-based documentation on the benefits of
CAM and which disease states may be impacted by CAM.
As the CAM session occurred at the end of the semester,
case-based questions were included on the final exam. The
questions related to the history of the modalities and their
application to patient cases, which illustrated the relevance
of CAM in Western society and the impact a pharmacist
could have on wellness and disease states.
Within 48 hours of completing each CAM station, students submitted a written reflection on the active learning
experience. At the beginning of the next class session (2 days
later), students were asked to complete a follow-up survey
and take a tRAT. Conducting the tRAT after the session
gave students more uninterrupted time with the experiential
part. Students were tested on the information in a case-based
format on the final exam.
limited to two or less separate modalities, specically massage therapy and/or yoga. The lack of awareness regarding
other Eastern or alternative medicine modalities commonly
employed by patients for health maintenance was unexpected. Yoga and massage are ubiquitous in Western culture
and used to some degree in Western medicine. The presession survey did not categorize if the lack of familiarity was
due to a knowledge gap preventing students from identifying
yoga and massage as CAM, or if students were genuinely
unaware of their existence.
Perceptions regarding the role of CAM in health care
changed after the intensive session (Table 2). Prior to the
session, 54.7% (n535) of participants responded that they
felt CAM did not t into the current health care model;
whereas after, 85.2% (n546) felt that it did (p,0.05). Almost all of the postsession survey respondents (96.2%,
n551) afrmed that participating in the CAM session
improved their understanding compared to just reading
assigned text. We believe that experiencing CAM rst hand
gave students a deeper understanding of its role and empathy for modalities they may not have fully understood in the
context of Western medicine. This was illustrated in the
postsession survey where 100% of respondents (n554) said
they would recommend CAM to a patient if their condition
might benet from it.
In the postsession survey, 94.3% (n550) of respondents said they beneted personally and 96.3% (n552)
beneted professionally from the experience. Positive feedback was provided by 98% (n569) of students, who described their changed perceptions and new enthusiasm for
a nontraditional classroom experience. This attitudinal scale
showed a deeper understanding and increased empathy for
CAM. Survey results also demonstrated a greater awareness
among students of various CAM modalities, with 89% of
them being able to list 4 or more modalities compared to
64% who were able to list 1 or 2 prior to the session.

Performance was assessed using presession iRATs and
postsession tRATs (Table 1). Postsession scores improved
signicantly, with the entire class receiving an A. High
stakes assessments and a comparison of questions based on
assigned readings and the interactive session indicated that
learners performed better on case-based application type
questions relating directly to their CAM session. Although
learners had assigned readings that covered historical aspects
of CAM, they did signicantly worse on this portion. For
example, 61.4% answered the Tai Chi history question correctly versus 92.8% who answered the Tai Chi case-based
application question correctly. We believe these assessments
illustrate the value and impact of interactive sessions on
learner performance.
The presession survey response rate was 93% (N567),
and the postsession survey response rate was 78% (n554).
Learners who opted out of the session did not complete the
survey. In the presession survey, 30% (n521) of learners
reported they were not familiar with any CAM modality.
After the session, only 9% (n56) reported being unfamiliar
with CAM (p50.02). For the purpose of the survey, authors
considered learners to be familiar with the concept if they
conrmed familiarity and demonstrated the ability to specify
CAM modalities in an open-ended follow-up question.
Of the 69.7% (n546) who reported familiarity or experience
with CAM, 64% (n529) reported their knowledge was
DISCUSSION
The intensive, interactive approach improved attitudes
regarding CAM and learner performance on assessment.
Other institutions that include CAM in their curriculum
might want to consider providing a similar session to improve knowledge about and attitudes toward CAM. Learners
reported that after the session they perceived reduced stress,
an improved sense of well being, and decreased anxiety.
These reflections allude to students ability to extrapolate
this personal benet to the professional setting and the benets their patients may derive from CAM. As one learner
stated, I found this a great tool. I think this was a way to help
make us aware of alternative medicine because our patients
will be using these techniques or may have questions about
them. Many of the learners vowed to begin a CAM practice
Table 1. Individual Readiness Assurance Test (iRAT)/Team

Readiness Assurance Test (tRAT) Grade Distribution
Letter Grade A (90%) B (80%) C (70%) F (,70%)
iRAT
tRAT
14
71
47
0
0
0
9
0
85

Table 2. Results of Classroom Survey Regarding Learner Knowledge and Perceptions of CAM
Question
Presession % Yes
(N=67)
Postsession % Yes
(n=54)
69.7 (n546)
90.7 (n549)
p50.02
52.2 (n535)
56.7 (n538)
77.8 (n542)
85.2 (n546)
p,0.05
p,0.05
Are you familiar with complementary and alternative

medicine (CAM)?
Does CAM t into our current health care model?
Does the pharmacist have a role in CAM?
on their own to help manage the stress of pharmacy school.

One learner reected great learning opportunity for us.
The practitioners who presented the CAM modalities were
amenable to teaching our classes, even without the promise
of an honorarium. They were pleased to have the opportunity
to present their respective discipline to pharmacy learners
and hoped to expand the learners understanding of CAM
and its role in health care today. They also reported leaving
the session enthusiastic and ready to incorporate new modalities into their own practices or revisit old ones.
A limitation to the CAM sessions/this study was the
availability of the practitioners. Acupuncture, for example,
was mentioned by several learners as something they
would benefit from experiencing. Though an acupuncturist
was scheduled, he was unable to attend on the actual day. In
the future we plan to incorporate additional modalities.
Further limitations include a decreased postsession survey
response rate, which prevented us from quantifying the
impact on all learners in the course.
3. Dutta AP, Daftary MN, Egba PA, Kang H. State of CAM

education in U.S. schools of pharmacy: results of a national survey. J
Am Pharm Assoc. 2003;43(6):81-83.
4. Shields KM. Natural product education in schools of pharmacy in
the United States. Am J Pharm Educ. 2003;67(10):43-48.
5. Mackowiak ED, Parikh A, Freely J. Herbal product education in
United States pharmacy schools: core or elective program? Am J
Pharm Educ. 2001;65(1):1-6.
6. Wong LY, Toh MP, Kong KH. Barriers to patient referral for
complementary and alternative medicines and its implications on
interventions. Complement Ther Med. 2010;18(3-4):135-142.
7. Dolder C, Lacro J, Dolder N, Gregory P. Pharmacists use of and
attitudes and beliefs about alternative medications. Am J Health-Syst
Pharm. 2003;60(13):1352-1357.
8. Koh HL, Teo HH, Ng HL. Pharmacists patterns of use,
knowledge, and attitudes toward complementary and alternative
medicine. J Altern Complement Med. 2003;9(1):51-63.
9. Harris IM, Richard RL, Rodriguez R, Choudary V. Attitudes
towards complementary and alternative medicine among pharmacy
faculty and students. Am J Pharm Educ. 2006;70(6):1-8.
10. Tiralongo E, Wallis M. Integrating complementary and
alternative medicine education into the pharmacy curriculum. Am J
Pharm Educ. 2008;72(4):1-9.
11. Evans E, Evans J. Changes in pharmacy students attitudes and
perceptions toward complementary and alternative medicine after
completion of a required course. Am J Pharm Educ. 2006; 70(5):1-7.
12. Nahin RL, Barnes PM, Stussman BJ, Bloom B. Costs of
complementary and alternative medicine (CAM) and frequency of visits to
CAM practitioners: United States, 2007. National health statistics reports;
no 18. Hyattsville, MD: National Center for Health Statistics. 2009. http://
www.cdc.gov/nchs/data/nhsr/nhsr018.pdf Accessed Feb 24, 2014.
13. Barnes PM, Bloom B, Nahin RL. Complementary and alternative
medicine use among adults and children: United States, 2007.
National health statistics reports; no 12. Hyattsville, MD: National
Center for Health Statistics. 2008. http://www.cdc.gov/nchs/data/
nhsr/nhsr012.pdf. Accessed Feb 24, 2014.
14. Semple SJ, Hotham E, Rao D, Martin K, Smith CA, Bloustien
GF. Community pharmacists in Australia: barriers to information
provision on complementary and alternative medicines. Pharm World
Sci. 2006;28(6):366-373.
15. Accreditation Council for Pharmacy Education. Accreditation
standards and guidelines for the professionals program in pharmacy
leading to the doctor of pharmacy, version 2.0. Feb 14, 2011.
Available from: http://www.acpe-accredit.org/pdf/
FinalS2007Guidelines2.0.pdf. Accessed July 1, 2013.
standards and guidelines for the professional program in pharmacy
leading to the doctor of pharmacy degree. 2007. http://www.
acpe-accredit.org/pdf/ACPE_Revised_PharmD_Standards_
Adopted_Jan152006.pdf. Accessed Aug 28, 2014
SUMMARY
CAM is an important therapeutic modality that can be
used in treatment; however pharmacists need appropriate
exposure to it in their education to accurately recommend it
or augment current therapy with it. The purpose of this
session was to improve assessment performance and attitudes regarding CAM by having learners experience firsthand what CAM encompasses. After participating in the
active learning session incorporating 5 different modalities
of CAM, students increased their knowledge and understanding of a variety of CAM, the role it plays the current
Western medical model, and the role pharmacists can play
in recommending CAM to patients. The interactive CAM
session was valuable and added a new dimension to the
learners educational experience.
REFERENCES
1. Dutta AP, Miederhoff PA, Pyles MA. Complementary and
alternative medicine education: students perspectives. Am J Pharm
Educ. 2003;67(2):1-7.
2. Ulbricht C, Chao W. Common complementary and alternative
medicine health systems. In: Krinsky DL, ed. Handbook of
Nonprescription Drugs. 17th ed. Washington, D.C.: American
Pharmacists Association; 2012:1007-1018.
86

A Multiyear Analysis of Team-Based Learning
in a Pharmacotherapeutics Course
June Felice Johnson, PharmD,a Edward Bell, PharmD,a Michelle Bottenberg, PharmD,a
Darla Eastman, PharmD,a Sarah Grady, PharmD,a Carrie Koenigsfeld, PharmD,a
Erik Maki, PharmD,a Kristin Meyer, PharmD,a Chuck Phillips, PharmD, PhD,a
Lori Schirmer, PharmDb*
a
Drake University College of Pharmacy and Health Sciences, Des Moines, Iowa
Fort Sanders Regional Medical Center, Knoxville, Tennessee
*
Author afliation at time of study was Drake University.
b
Objectives. To evaluate the impact of team-based learning (TBL) in a pharmacotherapeutics course on

pharmacy students ratings of faculty instructors and the course, and to assess students performance
after implementation of team-taught TBL.
Design. Teaching methodology in a pharmacotherapeutics course was changed from a lecture with recitation approach in 2 semesters of a 6 credit-hour course to a TBL framework in a 3-semester 31415 credit
hour course. The distribution of faculty of instruction was changed from 4 faculty members per week to 1
faculty per 1-credit-hour module. TBL consisted of preclass study preparation, readiness assurance (Individual Readiness Assessment Test and Group Readiness Assessment Test), and in-class application
exercises requiring simultaneous team responses.
Assessment. Retrospective analysis of student ratings of faculty and instructional methods was conducted
for the 2 years pre-TBL and 4 years during TBL. Final course grades were evaluated during the same time
period. Student ratings showed progressive improvements over 4 years after the introduction of team-based
learning. When aggregated, ratings in the excellent teacher category were unchanged with TBL compared
to pre-TBL. Improvements in faculty instructor approaches to teaching were noted during TBL. Group
grades were consistently higher than individual grades, and aggregate course grades were similar to those
prior to TBL implementation.
Conclusion. Implementation of TBL in a pharmacotherapeutics course series demonstrated the value of
team performance over individual performance, indicated positive student perceptions of teaching approaches by course faculty, and resulted in comparable student performance in nal course grades
compared to the previous teaching method.
Keywords: team-based learning, pharmacotherapeutics, student evaluations, faculty performance, student
performance
require colleges and schools of pharmacy to ensure that

students are prepared to problem-solve effectively as valued members of a patient-centered care team. The Accreditation Council for Pharmaceutical Education (ACPE)
Standards 2.0 emphasize that pharmacy programs graduate
student pharmacists who . . .can contribute to the care of
patients and to the profession by practicing with competence and condence in collaboration with other health
care providers.5 Team-based learning in the pharmacy
curriculum has been described as a pedagogy that can address this type of professional preparation.6-16
Team-based learning is a specific instructional
strategy and framework that uses intentionally formed
INTRODUCTION
Transformation of health care delivery models so
that they improve quality and reduce the cost of patientcentered care will require that pharmacists take new approaches to care coordination, team-based care, and
chronic disease management, and that they assume new
health delivery roles.1-4 These new collaborative models
Corresponding Author: June Felice Johnson, Professor of
Pharmacy Practice, Drake University College of Pharmacy
and Health Sciences, 2507 University Avenue, Des Moines,
Iowa, 50311. Tel: 515-271-1849. Fax: 515-271-4171. E-mail:
june.johnson@drake.edu
87

teams of learners to deepen student learning and develop
high-performing teams. The core principles of TBL are:
(1) learning groups must be formed and managed; (2)
students must be held accountable for the quality of both
their individual work and the groups work; (3) students
must receive frequent and timely feedback on their
learning; and (4) team assignments must promote both
learning of the content and development of the team.17
Team-based learning comprises 3 major steps: (1)
individual study and preparation; (2) readiness assurance,
completion of an individual readiness assessment test
(iRAT) in class, followed by the same test as a group,
and completion of the group readiness assessment test
(gRAT); and (3) application exercises in class where
teams work together to solve problems using information
gained from the previous 2 steps. Each student evaluates
the other members of the group through graded peer evaluations during the semester.
The TBL model has been used extensively in business in the 1970s and, more recently, in health sciences
education, with improved learning outcomes in colleges
and schools of pharmacy and other health professions.18-28
Reports of TBL in pharmacy education often use student
and faculty perception or performance to examine the
initial implementation of TBL. However, no reports have
used nationally standardized faculty evaluations of teaching using TBL, examined experiences over a longitudinal
implementation of TBL, or reported course outcomes in
a pharmacotherapeutics course series that used TBL.
More data are needed for faculty members to understand
the longitudinal impact of implementing TBL in a required team-taught course.
Medical educators have increasingly advocated for
active-learning strategies that allow students to apply
problem-solving skills and learn collaboratively. However, student performance and attitudes towards these
methods are often mixed, and comparisons are not possible because of the wide variation in teaching models
used.20,21,24,26,29
The Drake University College of Pharmacy and
Health Sciences (DUCPHS) practice faculty members
adopted TBL in 2009 during a college-wide curricular
revision in 2008 and in tandem with the release of ACPE
Standards 2007. This study describes a multi-year experience with TBL in a required pharmacotherapeutics
course series at DUCPHS. The primary objective of this
study was to evaluate the impact of implementing TBL on
faculty and course evaluations, and the secondary objective was to measure student performance before and after
curriculum revision.
We hypothesized that student evaluations of faculty
members and the pharmacotherapeutics course series
would remain relatively stable and measures of student

engagement with learning would increase with TBL. The
authors also hypothesized that students course performance
would not be negatively impacted and that team grades
(gRATs) would outperform individual grades (iRATs).
DESIGN
This project was approved as exempt by the Drake
University Institutional Review Board. At DUCPHS, the
previous model for classroom instruction of pharmacotherapeutics was the live lecture, which focused on delivering
content to the entire class, followed in the same week by
smaller case-based discussion group sessions. However,
faculty members were not satisfied with the students
engagement with the material, preclass preparation, or
postlecture retention of concepts. These assessments were
made informally and consistently between faculty members
over the years through direct observation of student participation in class, in discussions during annual therapeutics retreats, and during direct interaction with students
during their practice experiences. Workload for faculty
members teaching in the course was demanding, requiring both a content expert for the lecture component and
3 additional faculty facilitators to conduct recitation groups
each week. A comprehensive revision of the professional
curriculum at DUCPHS to meet ACPE Standards 2007
offered an opportunity to explore a new pedagogy that
would enhance student learning and strengthen development of high-performing teams. Concurrently, delivery
of the course was changed from a 2-semester fall-spring
sequence for third-year (P3) students to a 3-semester
sequence beginning in the spring for second-year (P2)
students and fall and spring for P3 students; this sequence
more evenly distributed course credit hours (Figure 1).
Course faculty members explored new teaching methods
and developed a set of criteria that any new method used
would have to meet: increase student accountability for
learning; increase active learning; develop life-long learning skills; increase retention of knowledge; increase contact with individual faculty members; and decrease the
number of faculty members involved in each semester by
creating 1-credit hour blocks (modules) for which each
faculty member was responsible.
The Michaelsen model for TBL was selected as an
active learning strategy because it best fit the needs of the
course in preparing students for professional roles, emphasized engaged learning and accountability for learning, and reinforced the value of strong team performance
on learning and problem solving.17 Another active learning
strategy considered was problem-based learning (PBL),
which was rst used in medical education in the 1960s.
This instructional strategy introduces a problem to the
88
Figure 1. Pharmacotherapeutics course structure before and during team-based learning
students at the beginning of an instructional module and

then requires students to self-discover answers progressively through repeated interaction with a faculty member. The DUCPHS practice faculty members decided that
PBL would be time-intensive and require more time
teaching outside of class, adding to the challenges of
work-life balance and faculty member success in scholarship and service. The effects of PBL on learning outcomes
were equivocal.29 Previous experience with a lecture plus
case-based discussion did not always meet faculty expectations for learning outcomes and was demanding of faculty teaching time. Based on the extensive literature on
TBL, efciency of course delivery, and potential for improved outcomes of both student learning and student
attitudes towards learning, the Michaelsen model for
TBL was selected. At the time, neither the students nor
the DUCPHS faculty had any previous experience with
TBL in the curriculum, though various active learning
strategies such as recitation groups and case-based discussion had been used. The change in course pedagogy
was approved by DUCPHS faculty members in 2008 and
preparation began for delivery of the revised course in
spring 2009.
All faculty members at DUCPHS attended a workshop 1 year prior to implementation; however, only the
pharmacy practice faculty members decided to fully
adopt this model in its entirety for the course. Pharmacotherapeutics faculty members held additional selfdirected, hands-on practice sessions prior to implementing
this pedagogy. A number of practice faculty members
joined the TBL Collaborative to access additional resources and for ongoing support from TBL experts around
the country.
The TBL course was taught live as a 3-semester sequence to the P2 and P3 classes of doctor of pharmacy
(PharmD) degree students on the university campus. The
number of hours spent in class reflected the course credit
hours: multiple modules equal to 1-credit hour were taught
each semester, and only one faculty expert taught a module
in one semester of the course (Figure 1). The rst TBL
offering was held in a traditional-style classroom located
within the college that was not conducive to group work,
so this room was subsequently updated to better support
engaged active learning. Twelve pharmacy practice faculty
members taught in the course series. All of the faculty
members maintained a clinical practice in an area of
89

expertise, had completed postgraduate residency training,
and had at least 1 advanced credential (eg, Board Certied
Pharmacotherapy Specialist).
progress, ie, I made small gains on this objective,

35moderate progress, ie, I made some gains on this objective, 45substantial progress, ie, I made large gains on this
objective, 55exceptional progress, ie, I made outstanding
gains on this objective.
The IDEA course evaluation system was used for measures of the primary study outcomes. This system has been
shown to be both valid and reliable for course-related outcomes and student progress. For evaluation, each cycle of
the course sequence was treated as a discrete unit for comparison. This produced 6 cohorts of data (2 years of the
course sequence prior to TBL and 4 years using TBL for
comparison.
Student ratings for 5 specific approaches to instruction were compared to support the assertion that fundamental teaching changes took place in the course sequence
(Table 1). Pre-TBL and TBL courses were compared using
Student t tests for these 5 independent activities. In
addition, course assignments (reading and nonreading
assignments) and student ratings for the difculty of subject matter were compared across the pre-TBL and TBL
courses using student t tests in an attempt to further support
changes in format rather than subject matter.
Student progress was analyzed using the 5 IDEA
course objectives identified by the faculty as being the
objectives they would focus on during the course sequence.
Because these course objectives were highly correlated and
expected to be simultaneously affected by TBL, multivariate analysis of variance (MANOVA) was used to evaluate
differences across the 6 study years. Although these ratings
consisted of ordinal data, graphical representation showed
adequate dispersion across the scale to warrant this parametric approach. When analyzing independent constructs
across the 6 years (such as excellent course ratings and
excellent teacher ratings), and when comparing a total

Student ratings of the course and faculty members
were used to support the main hypothesis for this study. A
Student Ratings of Instruction system, created by the nonprofit IDEA Center, which focused on student learning of
12 different objectives and factored out extraneous circumstances, was used to assess the course. The system
also enabled students to rate faculty instruction on 20
different items.30 Students completed these evaluations
at the end of each faculty members section during both
course iterations.
The IDEA Center system has been used by DUCPHS
since 2004 for course and faculty evaluations. These evaluations are completed either in class on paper or outside of
class online. Because of this history, both faculty members and students were comfortable with the system during the time TBL was introduced. Overall, the faculty
evaluation process remained the same both before and
after TBL implementation. Using the IDEA Center system, course coordinators selected the objectives believed
integral to the course. Prior to TBL implementation, faculty teaching in pharmacotherapeutics selected gaining
factual knowledge, learning fundamental principles,
learning to apply course material, and learning to analyze and critically evaluate ideas, arguments, and points
of view as key objectives for the course. For the TBL
sequence, key objectives were gaining factual knowledge, learning to apply course material, and acquiring skills in working with others as a member of a team.
Progress on these objectives was rated by the students on
a 5-point Likert scale (15no apparent progress, 25 slight
Table 1. Pharmacy Student Ratings of Changes in Instructor Teaching Approaches Pre-Team-based Learning (Pre-TBL) and
Team-based Learning (TBL) Implementation
Approach
Scheduled course work (class activities, tests, projects) in ways
that encouraged students to stay up-to-date in their work
Formed teams or discussion groups to facilitate learning
Involved students in hands-on projects such as research,
case studies, or real life activities
Asked students to help each other understand ideas or concepts
Gave projects, tests, or assignments that required original or
creative thinking
a
b
Pre-TBL,
Mean (SD)
TBL,
Mean (SD)a,b
4.0 (0.3)
4.4 (0.3)
3.5 (0.6)
3.9 (0.4)
4.8 (0.1)
4.3 (0.4)
3.7 (0.4)
2.8 (0.3)
4.4 (0.3)
3.2 (0.5)
Ratings based on Likert scale of 15hardly ever, 25occasionally, 35sometimes, 45frequently, 55almost always.
All signicant at 0.003 or less for student t test.
90

score across the 5 objectives, analysis of variance
(ANOVA) was used with conservative Scheffe multiple
comparison testing. SPSS (IBM Corp. Released 2012.
IBM SPSS Statistics for Macintosh, Version 21.0.
Armonk, NY: IBM Corp.) was used for all analyses with
an a priori alpha of 0.05.
Two student focus groups were conducted at the end
of the spring semester in 2009 and 2010. The focus groups
were conducted by an individual trained in qualitative
focus group analysis who met with 6 P2 students enrolled
in each spring semester. In response to themes extracted
from these focus groups, purposeful action was taken to
improve clarity of expectations, to improve connections
between objectives and study questions, and to enhance
details on module objectives. Student organizations subsequently provided leadership by presenting mock TBL
sessions to inform students on the process and benefits.
Minimal changes were made in the amount of reading and
preparation required for the course.
Peer evaluations were conducted each semester,
but reduced from 3 to 2, ensuring that students continued
to receive valuable feedback while minimizing time
burden. Manual tabulation was used initially. iPeer, version 3.08 (open-source software, University of British
Columbia), was implemented in 2012 to streamline the
process.31
Student performance during TBL was assessed by
comparing gRAT with iRAT performance. On average,
3 iRATs or gRATs were given in each module. Eighty
percent of the final course grades in TBL reflected individual performance, 15% reflected team performance,
and 5% reflected peer evaluations. In the course preTBL, 80% of the grade was based on individual performance on examinations, and 20% on recitation and peer
evaluations. Final course grades in the 2 years pre-TBL
were compared to those in the 4 years TBL was used.
Across the 6 years the course sequence was evaluated, the mean (SD) class size was 119611.8 (range
102-135) students. The IDEA Center student evaluation
response rates were reliable and averaged 55.6% (range
18-99%) with higher in-class paper response rates. Students perceptions of course format suggested that the
approaches to teaching the courses did, in fact, change
after TBL was implemented. Students in the TBL course
noted signicantly greater emphasis on 5 preselected
teaching approaches that were consistent with TBL
(Table 1). Students also noted that the amount of reading
increased in the TBL years while the amount of other
nonreading assignments decreased. Overall, the students
did not see a difference in the difculty of the subject
matter from pre-TBL to TBL implementation in the
course (Table 2).
Table 2. Analysis of Course Assignments and Subject

Difficulty Prior to and After Implementation of Team-based
Learning (TBL)
Course Materiala
Amount of reading
Amount of work in other
(non-reading) assignments
Difculty of subject matter
Pre-TBL
(SD)
TBL
(SD)
3.8 (0.2)
4.0 (0.2)
4.3 (0.3) ,0.001

3.8 (0.3)
0.001
4.3 (0.2)
4.3 (0.3)
0.69
Based on Likert scale of 15much less than most courses, 25less

than most courses, 35about average, 45more than most courses, and
55much more than most courses.
When viewing the cohort of instructors across the 6

years, there was no significant difference in aggregate
scores for the item, Overall, I rate this instructor an excellent teacher (F51.988, p50.09). Scores for the 6
cohorts ranged from 3.6 to 4.3 on a 5-point scale
(15denitely false, 25more false than true, 35in between, 45more true than false, 55denitely true). When
viewing the entire course, 1 year did show a signicant
decrease in student ratings for, Overall, I rate this course
as excellent (F58.93, p5,0.001). A Scheffe multiple
comparison test for the statement, Overall, I rate this
course as excellent found a mean of 4.1 in both preTBL years 1 and 2, and means of 3.5, 4.0, 4.4, and 4.3
in years 3 to 6, respectively, of TBL implementation.
While the third year was signicantly different from all
other years, the fth and sixth years demonstrated progressive improvement in responses compared to pre-TBL
or the rst 2 years of TBL implementation.
The MANOVA did reveal significant differences
between various years and student progress on IDEA
course objectives (Table 3). For gaining factual knowledge and learning to apply course material, year 3 had the
lowest progress ratings and was signicantly lower than
years 5 and 6. Both of these objectives were identied as
key objectives in both pre-TBL and TBL implementation
years. Similarly, the learning fundamental principles objective varied among class years. Again, year 3 (the rst
year of TBL) received the lowest progress ratings and was
statistically lower than years 5 and 6. The rst pre-TBL
year also received signicantly lower progress ratings
than year 6.
The IDEA objective of acquiring skills in working
with others as a member of a team was used as a course
focus for only the TBL years. As expected, this objective
showed different progress scores from pre-TBL to TBL
years. The lowest score on this objective was in year 1,
followed by year 2. The pre-TBL years showed significantly lower progress than all 4 TBL years. In addition,
the first year of TBL implementation (year 3) also showed
91

Table 3. Progress Scores for IDEA Course Objectives Across the Six-year Period
Objectivea
Pre-Team-based
Learning(SD)
Year 1
Year 2
Gaining factual knowledge (terminology,

4.1
classications, methods, trends)
Learning to apply course material
4.0
(to improve thinking, problem solving,
and decisions)
Learning fundamental principles,
3.9
generalizations, and theories
Acquiring skills in working with others
2.5
as a member of a team
Learning to analyze and critically evaluate 2.8
ideas, arguments, and points of view
Total:
17.3
Year 3
Team-based Learning (SD)

Implementation
Year 4
Year 5
Year 6
4.5 (0.2)b
(0.3)
4.3 (0.4)
4.0 (0.5)
4.3 (0.4)
4.5 (0.3)
(0.4)
4.1 (0.4)
3.8 (0.5)
4.2 (0.4)
4.4 (0.3)b
4.5 (0.2)b
(0.3)
3.9 (0.4)
3.6 (0.5)
4.1 (0.3)
4.3 (0.2)b
4.4 (0.2)b,c
(0.3)b,d
3.2 (0.4)c
3.6 (0.3)c,d
4.0 (0.4)c,d
3.8 (0.3)c,d
4.1 (0.3)b,c,d
(0.3)
2.8 (0.4)
2.8 (0.4)
3.1 (0.4)
2.9 (.40)
3.5 (0.3)b,c,d,e
(1.5)
18.3 (1.9)
17.8 (2.2)
19.7 (1.8)c
20.0 (1.5)c
21.0 (2.1)b,c,d
Based on Likert scale of: 15no apparent progress, 25slight progress (I made small gains on this objective), 35moderate progress (I made some
gains on this objective), 45substantial progress (I made large gains on this objective), and 55exceptional progress (I made outstanding gains on
this objective).
b
Signicantly different from year 3.
c
d
e
lower progress as compared to the last year of TBL (year

6). Finally, the pre-TBL courses focused on the IDEA
objective of learning to analyze and critically evaluate
ideas, arguments, and points of view. In a comparison
of the 6 years, year 6 had the highest progress rating. This
was significantly higher than all but year 4. When analyzing the overall measure of progress on the 5 objectives
(total score on the 5), ANOVA showed significant progress in the last 3 years using TBL. Study years 4, 5, and 6
showed higher total scores than year 1, while year 6 was
also significantly higher than years 2 and 3 (Table 3).
As postulated in the TBL model, and as the authors
hypothesized, students scored consistently better on their
gRATs in comparison to their iRATs each year the course
was offered (Table 4). Overall, nal course grades preTBL and during TBL were similar, though there was
a higher proportion (5.3%) of D grades in the rst course
in TBL year 1 (Table 5).
and progressive improvements over time. Ratings for faculty

members and the course as excellent teacher and excellent course were similar, with the exception of a decrease in
the rst year of TBL implementation. Because students had
no previous exposure to TBL before this course, unfamiliarity with this pedagogy likely contributed to lower ratings the
rst year. Additionally, all faculty members were new to
Table 4. Grade Summary of Individual Readiness Assessment
Test and Group Readiness Assessment Test Scores Over 4
Years of Team-based Learning Implementation in
a Pharmacotherapeutics Course Series
Assessment
Test
PHAR 190
iRAT
gRAT
PHAR 191
iRAT
gRAT
PHAR 192
iRAT
gRAT
DISCUSSION
The study findings supported our main hypothesis that
student evaluations would remain relatively stable and measures of student engagement with learning would increase
with TBL. Subjective IDEA Center student evaluations of
faculty members and the course did not change drastically
and improved in some areas when comparing pre-TBL to
TBL evaluations over multiple years. An overall rating of
the 5 course objectives showed improved student progress
Class
2011a
Class
2012
Class
2013
Class
2014
80
96.9
78.9
97.9
83.4
98.7
82.6
99.1
82.2
98.6
79.7
96.7
81.1
97.6
81.1
97.1
82.5
97.6
80.3
97.1
81.1
97.9
78.3
97.4
Graduation year of class enrolled in TBL PHAR 190 course in

spring 2 years previous.
Abbreviations: iRAT5 Individual Readiness Assessment Test,
gRAT5 Group Readiness Assessment Test. PHAR
1905Therapeutics I; PHAR 1915Therapeutics II; PHAR
1925Therapeutics III.
92

Table 5. Summary of Final Pharmacotherapeutics Course Series Grades Over Six Years Prior to and After Team-based Learning
Implementation
Final Course Grades
PHAR 190 N No. (%)
A
B
C
D
F
PHAR 191 N No. (%)
A
B
C
D
F
PHAR 192 N No. (%)
A
B
C
D
F
a
Pre-Team-based Learning
Year 1
Year 2
134
34 (25.4)
80 (59.7)
19 (14.2)
0
1 (0.7)
103
35 (34)
57 (55.3)
11 (10.7)
0
0
N/Aa
N/A
N/A
N/A
N/A
135
18 (13.3)
80 (59.3)
35 (25.9)
0
1 (0.7)
N/A
N/A
N/A
N/A
N/A
102
63 (61.8
31 (30.4)
8 (7.8)
0
0
Team-based Learning Implementation

Year 3
Year 4
Year 5
Year 6
114
(32.5)
(59.6)
(2.6)
(5.3)
0
114
36 (31.6)
72 (63.2)
5 (4.4)
1 (0.8)
0
N5113
33 (29.2)
70 (61.9)
9 (8)
1 (0.9)
0
37
68
3
6
110
(22.7)
(67.3)
(10.9)
(1.8)
0
103
39 (36.8)
66 (62.3)
1 (0.9)
0
0
109
15 (13.8)
77 (70.6)
14 (12.8)
3 (2.8)
0
25
74
12
2
125
(30.4)
(64)
(4.8)
(0.8)
0
120
49 (40.8)
66 (55)
5 (4.2)
0
0
123
46 (37.3)
63 (51.2)
12 (9.8)
2 (1.6)
0
38
80
6
1
128
(35.9)
(56.3)
(7)
(0.8)
0
129
52 (40.3)
67 (51.9)
10 (7.8)
0
0
130
37 (28.5)
75 (57.7)
16 (12.3)
2 (1.5)
0
46
72
9
1
N/A: Class not taught pre-team-based learning.
TBL, which also may have contributed to the rst TBL year
ratings. Over time, increased practice in the pedagogy and
constructive responses to student feedback resulted in an
improved course.
These findings were consistent with studies that
showed similar or higher student ratings over time when
comparing TBL to other learning methods.8,11,14,24 In
support of our secondary hypothesis, this study showed
improvement in team performance over individual performance and comparable course performance, which
was also consistent with previous studies.7,10,13,14
Lower-performing students beneted more from TBL,
or had observed no differences in performance between
case-based learning and TBL.22,25 The proportion of D
and F grades was low in both iterations of this course,
though a higher proportion of D grades occurred in the
rst year of TBL. Perhaps these students struggled more
in adapting to TBL. In subsequent TBL years, the proportion of low grades was comparable to pre-TBL; however, the small numbers of students in this category
preclude denite conclusions.
Although other studies have examined multi-year
experiences, to our knowledge this study was the first to
focus on IDEA Center teaching evaluations over time. In
addition, we used a pure TBL model instead of using
select components of TBL as many published hybrid
models have done.
Most of the pharmacotherapeutics course faculty
members were seasoned veterans, who had taught the
course for more than 5 years and had previously used

the more traditional model of lecture and case discussion. These factors likely contributed to faculty members ability to successfully adapt to the TBL model.
Involvement of faculty champions played a key role
in faculty and administrative buy-in, course development, and implementation. Adequate faculty preparation and practice sessions supported successful course
delivery.
Teaching circles of all faculty teaching in the course
per semester were formed and met regularly to discuss
ideas, share success stories, and brainstorm on strategies.
An annual faculty retreat allowed course faculty members
to review course statistics, ensure coordination of topics,
and establish common policies and procedures between
semesters. Summary themes from 2 formal focus group
sessions were shared and discussed at the retreats during
the initial years of TBL. College administration supported
the course innovation and provided reassurance that
course evaluation scores in the initial years would be
evaluated within the context of change. Academic pharmacy support for TBL implementation can also be found
in the most recent Center for the Advancement of Pharmaceutical Education outcomes, with specific emphasis
on problem solving and communication.32
This change in educational methodology was initially time intensive, requiring content revision, double
teaching for 1 year, and conversion from a 2- to 3semester model. Students may have felt they were in
93

a constant state of rsts and may have had difculty
keeping their minds open to the educational value of
change. The advance preparation was a shift in student
thinking. Students complained that they did not have as
much time to study for other courses. Faculty members
from other courses remarked that they felt TBL courses
pushed students to study less for their courses and this
concern is one of the reasons DUCPHS faculty members have not adopted TBL across the curriculum. Student emotions ran high the rst year in particular, and
written student comments on course evaluations were
extensive and mainly negative. These negative perceptions motivated course faculty members to spend more
time orienting students to the rationale and mechanics
of TBL during the course and to provide more extensive
guidance in the course syllabus, but also in other areas,
such as by working with student organizations in the
semester prior to TBL.
The first time the TBL class was offered, an existing
traditional-style classroom did not easily support group
work and could have contributed to some of the negative
feedback on TBL initially. Although major physical improvements were subsequently made to the classroom to
accommodate TBL, further improvements are needed to
optimize team interaction. Past use of detailed PowerPoint slides had to be avoided with the TBL format. For
faculty experts with previous experience using extensive
slide sets in lecture, reduction to 1 to 3 slides per application exercise question was challenging. Course faculty
members have agreed to follow mutually established
guidelines going forward, with the understanding that
flexibility will ensure students receive the necessary
training for success upon graduation.
Care Team: Learning From Effective Ambulatory Practices project.

Acad Med. 2013;88(12):1830-1834.
2. Nielsen M, Olayiwola N, Grundy P, Grumbach K. The PatientCentered Medical Homes Impact on Cost & Quality: An Annual
Update of the Evidence, 2012-2013. Patient Centered Primary Care
Collaborative. 2014. http://www.pcpcc.org/resource/medical-homesimpact-cost-quality. Accessed February 20, 2014.
3. Zarowitz BJ, Miller WA, Helling DK, et al. Optimal medication
therapy prescribing and management: Meeting patients needs in
an evolving health care system. Pharmacotherapy. 2010;30(11):
350e-359e.
4. Institute of Medicine, Committee on Health Professions Education
Summit. Health Professions Education: A Bridge to Quality.
Washington, DC: National Academies Press, 2003.
standards and guidelines for the professional program in pharmacy
leading to the doctor of pharmacy degree. Guidelines Version 2.0.
Chicago, Illinois. https://www.acpe-accredit.org/standards/default.
asp. Accessed June 3, 2014.
6. Allen RE, Copeland J, Franks AS, et al. Team-based learning in
US colleges and schools of pharmacy. Am J Pharm Educ. 2013;77(6):
Article 115.
7. Beatty SJ, Kelley KA, Metzger AH, et al. Team-based learning in
therapeutics workshop sessions. Am J Pharm Educ. 2009;73(6):
Article 100.
8. Conway SE, Johnson JL, Ripley TL. Integration of team-based
learning strategies into a cardiovascular module. Am J Pharm Educ.
2010;74(2):Article 35.
9. Franks AS. Enhancing team-based active learning through handson experience with nicotine replacement therapy. Am J Pharm Educ.
2013;77(6):Article 128.
10. Grady SE. Teachers Topics: Team-based learning in
pharmacotherapeutics. Am J Pharm Educ. 2011;75(7):Article 136.
11. Letassy NA, Fugate SE, Medina MS, et al. Using team-based
learning in an endocrine module taught across two campuses. Am
12. Persky AM, Pollack GM. A modied team-based learning
physiology course. Am J Pharm Educ. 2011;75(10): Article 204.
13. Persky AM. The impact of team-based learning on a foundational
pharmacokinetics course. Am J Pharm Educ. 2012;76(2):Article 31.
14. Zingone MM, Franks AS, Guirguis AB, et al. Comparing teambased and mixed active-learning methods in an ambulatory care
elective course. Am J Pharm Educ. 2010;74(9):Article 160.
15. Kolluru S, Roesch DM, de la Fuente AA. A multi-instructor
team-based, active-learning exercise to integrate basic and clinical
science content. Am J Pharm Educ. 2012;76(2):Article 33.
16. Bleske BE, Remington TL, Wells TD, et al. Team-based learning
to improve learning outcomes in a therapeutics course sequence. Am
17. Michaelsen LK, Parmalee DX, McMahon LL, eds. Team-Based
Learning for Health Professions Education. A Guide to Using Small
Groups for Improving Learning. Sterling, VA: Stylus Publishing
LLC; 2008.
18. Ofstad W, Brunner LJ. Reviews: Team-based learning in
pharmacy education. Am J Pharm Educ. 2013;77(4):Article 70.
19. Farland MZ, Sicat BL, Franks AS, et al. Best practices for
implementing team-based learning in pharmacy education. Am
20. Searle NS, Haidet P, Kelly PA, et al. Team learning in medical
education: Initial experiences at ten institutions. Acad Med. 2003
(Oct. Suppl.);78(10):S55-S58.
CONCLUSION
Faculty members of a team-taught pharmacotherapeutics course successfully transitioned to a TBL pedagogy that increased student accountability, reinforced the
value of teamwork, enhanced professional development,
and optimized faculty workload. Students evaluations of
the course using the IDEA Centers rating system showed
improved student progress with TBL, while aggregate
teacher ratings remained stable. Team performance on
gRATs exceeded individual performance on iRATs, and
overall course grades remained stable with TBL. Changes
in faculty member approaches to instruction with TBL
were viewed positively by students and enhanced learning in the course.
REFERENCES
1. Ladden MD, Bodenheimer T, Fishman NW, et al. The emerging
primary care workforce: Preliminary observations from The Primary
94

27. Fatmi M, Hartling L, Hillier T et al. The effectiveness of teambased learning on learning outcomes in health professions
education: BEME Guide No. 30. Med Teach. 2013;35(12):e1608e-1624.
28. Cheng CY, Liou SR, Tsai HM, et al. The effects of team-based
learning on learning behaviors in the maternal-child nursing course.
Nurse Educ Today. 2014;34(1):25-30.
29. Hartling L, Spooner C, Josvold LT, Oswald A. Problem-based
learning in pre-clinical medical education: 22 years of outcome
research. Medical Teach. 2010;32(1):28-35.
30. The IDEA Center. http://www.theideacenter.org. Accessed
November 1, 2013.
31. iPeer. http://ipeer.ctlt.ubc.ca/. Accessed November 5, 2013.
32. Medina MS, Plaza CM, Stowe CD, et al. Center for the
Advancement of Pharmacy Education 2013 Educational Outcomes.
Am J Pharm Educ. 2013;77(8):Article 162.
21. Thompson BM, Schneider VF, Haidet P, et al. Team-based

learning at ten medical schools: Two years later. Med Educ. 2007;
41(3):250-257.
22. Koles P, Nelson S, Stol A et al. Active learning in a Year 2
pathology curriculum. Med Educ. 2005;39(10):1045-55.
23. Clark MC, Nguyen HT, Bray C, et al. Team-based learning in an
undergraduate nursing course. J Nurs Educ. 2008;47(3):111-117.
24. Parmalee DX, DeStephen D, Borges NJ. Medical students
attitudes about team-based learning in a pre-clinical curriculum. Med
Educ Online. 2009;14(1):1-7.
25. Koles PG, Stol A, Borges NJ, et al. The impact of team-based
learning on medical students academic performance. Academic Med.
2010;85(11)1739-1745.
26. Bahramifarid N, Sutherland S, Jalali A. Investigating the
applications of team-based learning in medical education. Educ Med
J. 2012;4(2):e7-e12.
95
Instructions to Authors
Introduction
The Journal is devoted to providing a forum for communication of relevant information for pharmacy
educators and all others interested in the advancement of pharmacy education. To be considered for
publication, manuscripts must directly relate to pharmacy education and provide useful information for the
national or international audience of the Journal. If a submission has only local or regional relevance, its
usefulness to the majority of readers is limited and thus will not be accepted. To ensure that only accurate
and substantive articles are included, all manuscripts undergo a peer review process and editorial approval
prior to acceptance.
Manuscript Categories
Reviews. Reviews are comprehensive, well-referenced descriptive papers on teaching or research topics
directly related to entry-level and graduate or postgraduate education and training or skill development. The
Reviews section includes papers on the history of pharmacy education. These manuscripts should not
exceed 25 double-spaced pages for all components.
Research Articles. Research articles describe experimental or observational investigations that used
formal methods for data collection and reporting of results of studies directly related to pharmacy education.
The category does not include investigations of instructional methods or approaches. These manuscripts
should not exceed 25 double-spaced pages for all components.
Instructional Design and Assessment.
Instructional Design and Assessment papers describe

novel methods for professional and graduate student instruction (lectures, laboratories, practice
experiences, or courses), or informational manuscripts on programmatic and curriculum development.
These manuscripts should not exceed 25 double-spaced pages for all components. Instructional Design and
Assessment articles should conform to the IDEAS format as described in an updated article by Poirier et al.
[Poirier T, Crouch M, Hak E, MacKinnon G, Mehvar R, Monk-Tutor M. Updated guidelines for manuscripts
describing instructional design and assessment: the IDEAS format. Am J Pharm Educ. 2009;73(3):Article
55.] (http://www.ajpe.org/view.asp?art=aj730355&pdf=yes)
Teachers' Topics. Teachers' Topics are invited manuscripts from those named Teachers of the Year at
AACP member institutions. Teachers Topics papers follow the IDEAS format as described above. These
manuscripts should not exceed 25 double-spaced pages for all components.
Innovations in Teaching. Innovations in Teaching are invited manuscripts from recipients of the
AACP Innovations in Teaching Award. These manuscripts should not exceed 25 double-spaced pages for
all components.
Letters to the Editor. Letters to the Editor serve as a forum for the expression of ideas or for
commenting on matters of interest. It is also an avenue for critiquing or expanding on the information
presented in a previously published manuscript. Authors are required to identify themselves. The Editor
reserves the right to reject, shorten, excerpt, or edit letters for publication.
Book and Software Reviews. Book and Software Reviews are brief documents that provide the reader with
a clear understanding of content in a book or software program, as well as the product structure, scope, and
limitations. The reviewer should state the value or utility of the product for instruction, research, or other
academic activities.
AACP Reports and Minutes.
AACP Reports and AACP Minutes are summaries of activity of

officers, delegates, and committees within AACP.
Research Standards
For all manuscripts reporting on research involving human subjects (including educational and institutional
research), the author should indicate in the cover letter that the research has been reviewed and approved
by the appropriate human research or ethics review committee, or that it has been determined to be exempt
from such review. For research that has undergone such review and approval, a statement to that effect
should be included in the methods section. All survey research must meet criteria established by the
Journals Editorial Board. Please refer to the following publications for guidelines:
Draugalis JR, Plaza CM. Best practices for survey research reports revisited:
implications of target population, probability sampling, and response rate. Am J
Pharm Educ. 2009;73(8):Article 142.
(http://www.ajpe.org/view.asp?art=aj7308142&pdf=yes)
Draugalis JR, Coons SJ, Plaza CM. Best practices for survey research reports: a
synopsis for authors and reviewers. Am J Pharm Educ. 2008;72(1):Article 11.
(http://www.ajpe.org/view.asp?art=aj720111&pdf=yes)
Fincham JE.Response rates and responsiveness for surveys, standards, and the
Journal. Am J Pharm Educ. 2008;72(3):Article
43.(http://www.ajpe.org/view.asp?art=aj720243&pdf=yes)
Stylistic Considerations
The style specifications for the Journal must be followed. Below are general guidelines for manuscript
format and style. If in doubt about style, authors should refer to the American Medical Association (AMA)
Manual of Style, 10th ed, or consult a recent issue of the Journal. Text. The text should be scholarly,
readable, clear, and concise. Standard nomenclature should be used. Unfamiliar terms and acronyms
should be defined at first mention. Manuscripts that were prepared for oral presentation must be rewritten for
print. Authors of research papers are discouraged from writing excessively long introduction or discussion
sections. Word Style. Consult a current edition of Webster's dictionary for guidance on spelling,
ompounding, and word separation. Foreign words, not in general use, should be italicized. For proper use of
chemical and biochemical terms, mathematical equations, mathematical expressions, special symbols,
subscripts, superscripts, or Greek letters, please refer to the AMA Manual of Style.
Capitalization. The word "association" must be capitalized when referring to the American Association
of Colleges of Pharmacy. When the word "journal" is capitalized and italicized as Journal, it can refer only to
the American Journal of Pharmaceutical Education. In scientific writing, always capitalize the following:
major words in titles and headings of manuscripts, designators for tables, figures, and appendices (eg,
Appendix 1), eponyms (but not the noun that follows them, eg, Gram stain, Babinski sign), names of tests
(eg, Beck Depression Inventory), genus names of organisms (but not the name of species, varieties or
subspecies), acts of legislation (eg, Medicare), awards (eg, Nobel Prize), proprietary names (eg, Xerox
copier), the title of a person when followed by the person's name (eg, Chair John W. Jones), official names
of organizations and institutions (eg, Centers for Disease Control and Prevention), geographic places (eg,
United States of America), sociocultural designations (eg, Republicans, French people), and historical
events (eg, Vietnam War).
Abbreviations. In instances where repeated use of an organization or chemical name would become
awkward, an official or accepted abbreviation may be substituted. The abbreviation should be placed in
parentheses immediately following the first use of the name in the main body of the text. Abbreviations of
common pharmaceutical associations or organizations do not require periods or spaces between letters (eg,
AMA). Abbreviations of "eg," and "ie," and "et al" should not be separated by periods. The names of US
states and countries should be spelled out when they stand alone (eg, pharmacists throughout the
United States...). Do not use postal abbreviations for states in the text. The abbreviation US may be used
as a modifier only when it directly precedes the word it modifies (eg, US health policies). Otherwise, it should
be spelled out (eg, the population of the United States). The names of all other cities, states, provinces,
and countries should be spelled out when they occur within the text of the article. Refer to the AMA Manual
of Style for additional rules regarding abbreviations.
Numbers. Numbers must be written as Arabic numerals unless they occur at the beginning of a sentence,
in which case the number should be spelled out. The exception to this rule is when the number one is used
in isolation within the text and substituting an Arabic number would seem awkward (eg, there was only one
logical solution to the problem). A number containing a decimal must be styled as an Arabic number. All
fractions must be written as decimal equivalents.
Measurements. The metric system will be used for all measurements; however, conventional units
should be used instead of SI units. Do not use periods when abbreviating units of measure.
Reference numbers. Reference numbers cited in the text of an article should be superscript Arabic
numerals placed at the end of the sentence, outside the final period or other punctuation. Subsequent
citations to the same reference must be indicated by the same number originally asiged to that reference.
Do not place parentheses around reference numbers cited in text
.
Personal Communications.
If the reference source for information in an article is from a personal

communication, it should be referenced as such in parentheses immediately following the material to which
it pertains. A personal communication reference must include the persons first initial and last name, type of
communication, and date (eg, T.L. Johnson, e-mail, October 2, 2002). AJPE requires that you obtain and
provide to the Journal written permission from the person to use them as a reference.
Hypertext Links.
Authors may identify uniform resource locators (URLs) for Internet Web sites that
provide the reader with additional information on the topic addressed in the manuscript. Although URLs are
an important feature of electronic publishing, authors are encouraged to be selective in their choice of sites
to include. Do not include URLs for Web pages with newspaper or journal articles that will be removed or
archived to another Web page. Links to pharmaceutical manufacturers or other sources of product
information are acceptable; however, providing a URL to the reader should not be substituted for adequate
discussion within the manuscript itself. Only include links to sites that are accessible without a password.
Manuscript Organization
When submitting a manuscript in Editorial Manager, the manuscript document (usually a Word file) should
be arranged in the following order: title page, abstract, text, references, tables, figures, and appendices.
Editorial Manager allows authors to upload files with tables, figures, and appendices separately if that is
more convenient.
Title Page. The title page should have the following information: a concise title, name of each author,
terminal degree, academic/professional title and affiliation, and city and state where located. If an author has
relocated to another institution, please include his/her affiliation during the time the author worked on the
manuscript. At the lower left of the page, indicate the name of the corresponding author and provide his/her
mailing address, telephone number, facsimile number, and e-mail address. At the bottom of the title page,
indicate 3 to 5 keywords.
Abstract. Each manuscript must include an abstract of 100 to 150 words. For Research Articles,
the abstract should include a brief statement (1 to 3 sentences) for each of the following sections: Objective,
Methods, Results, and Conclusions. For Instructional Design and Assessment papers, the abstract should
include a statement for the following sections: Objective, Design, Assessment, and Conclusions. Each
section within the abstract should be flushed left, followed by a period and the statements summarizing that
section.
Main Body of Text. The use of subheadings to divide the text is encouraged. Primary headings should
be in bold upper case letters and should contain no more than 35 characters or spaces. Secondary
headings should be in bold title case and appear above the paragraph. Tertiary headings should be in bold
with only the initial letter capitalized. Tertiary headings should end with a period and should appear before
the beginning of the first sentence in that section. Designand Methods sections should be written in the past
tense voice. Authors should address in the Discussion or Conclusion how the work could be translated to
other institutions.
Acknowledgments. Any special funding received for research that is the subject of the manuscript
should be included under a section entitled "Acknowledgments" at the end of the text. If the authors wish to
thank colleagues or others who provided assistance with their research or manuscript preparation, those
acknowledgments also should be included under this section. Any statements concerning liability for the
content of the manuscript may be included here as well (eg, "the ideas expressed in this manuscript are
those of the author and in no way are intended to represent the position of"). Conflicts in funding issues
should be included.
References. References to a published source should be provided for all information in the manuscript
that contains dates, facts, or opinions other than those of the author. Authors are responsible for the
accuracy and completeness of all citations. References should be numbered consecutively in the order in
which the information contained in the referenced publication appears or is referred to in the manuscript. A
second abbreviated reference or use of the term "ibid" to refer to information cited in a previous reference
should not be cited. Instead, if information from the same source is referred to a second time in the
manuscript, the same reference number originally assigned to that source should be cited. For detailed
information on reference style, refer to the section on Stylistic Concerns. Each journal citation must include
the surnames and complete initials of all authors. For manuscripts with 7 or more authors, the first 3 authors
should be listed, followed by a comma and et al. The names of all periodicals cited must be abbreviated in
accordance with abbreviations adopted by the National Library of Medicine and used in Index Medicus. An
example and special instructions for specific types of references are provided below. For additional
guidance, please refer to the AMA Manual of Style, 10th ed, or to a recent issue of the Journal.
Journal articles. For references to journal articles, list the names of the authors beginning with the last
name of each author followed by his/her initials. After the authors' names, list the title of the article. The first
letter of the first word of the title is upper case type, but the remainder of the title should be in lower case
letters, except for the first letter of proper names. A period should be placed after the title. Next, provide the
properly abbreviated title of the journal in italics followed by a period. Please refer to the journal list on
PubMed for proper reference abbreviations (http://www.ncbi.nlm.nih.gov/journals). Finally, include the
following numerical information: year of publication followed by a semicolon, volume number, issue number
in parentheses, and a colon followed by page number(s). If the article does not appear on consecutive
pages, use a semicolon between each segment of pages (eg, 172-175;179-183;199.)
Example: Stratton TP, Cochran GA. A rural geriatric experience. Am J Pharm Educ. 1990;62:151-155.
Book. List the last name of the first author of the book, followed by the first and middle initial if given, just
as in a journal reference. The names of all authors must be listed. Place a period after the last author's
initials. Next, state the title of the book using standard rules for capitalization within titles. A period should be
placed at the end of the title. If more than one edition of the bookhas been published, then the edition
number must be given. The numerical form of the ordinal number should be used to indicate the edition
number (eg, 9th), followed by a space and "ed." Next, provide the city and state where the publisher is
located. Use postal abbreviations for states. A colon should separate the city and state from the name of the
publisher. The full name of the publisher should be given, followed by a semicolon. Next, provide the year of
publication followed by a period. If the reference is to a specific page(s) within the book, place a colon after
the year of publication, followed by the page or page numbers referenced.
Example: Martin AN. Physical Pharmacy. 4th ed. Philadelphia, PA: Lea & Febiger; 1993:268;270-273.
Chapter in a book.
To reference a single chapter in a book, first list the authors and state the title as
you would if citing a journal article. The chapter title should be followed by the word "In" followed by a colon.
Next, list the name(s) and initials of the editors of the book, followed by a comma and the abbreviation ed
or eds followed by a period. Next include the title of the book, location of the publisher, name of publisher,
year of publication, and page numbers (same format as for a reference to an entire book (see previous
example). Example: Lyon RA, Titeler M. Pharmacology and biochemistry of the 5-HT2 receptor. In: SandersBush E, ed. The Serotonin Receptors. Clifton, NJ: Humana Press;1989:59-88.
Thesis or dissertation. For references to theses or dissertations, place the title of the thesis or
dissertation in italics. Include the location of the institution, its name, and the year the thesis or dissertation
was completed. Example: Thorn MD. A Comparative Review of the Statistical and Research Quality of the
Medical and Pharmacy Literature [masters thesis]. Chapel Hill: University of North Carolina, 1982.
Online material. For references to journals, e-magazines, or other publications on the Internet, state
the names of the authors, title of the article, publication title, and volume and publication date in the same
format as you would for a journal reference. For references to other information, give the title of the Web
page, followed by the name of the organization or Web site that published the information. For all references
to online material, the author should include the uniform resource locator (URL) for the page of the Web site
referenced (eg, www.hcfa.gov/stats.htm), followed by a period. Finally, write Accessed followed by the
month, day, and year on which the information was obtained from the site, followed by a period.
Example: Healthy People 2010, Office of Disease Prevention and Health Promotion, U.S.
Department of Health and Human Services. http://health.gov/healthypeople. Accessed May 25,2010.
Unpublished works. For references to unpublished material, such as articles or abstracts presented at
professional meetings but not published, provide the name of the meeting where the article was presented.
If the abstract has been published, the published source should be cited.
Article in press. For references to information in books or articles that are currently in press,
provide all of the available information for the reference. In place of the year, volume, issue, and
page numbers, include In press.
Example: Adamcik B, Hurley S, Erramouspe J. Assessment of pharmacy students' critical thinking and
problem-solving abilities. Am J Pharm Educ. In press. Manuscripts that have been submitted to a publisher
or journal but have not been accepted for publication cannot be included in the reference section. To
attribute information in the text to an unpublished source, list the authors and date the manuscript was
completed, along with the words unpublished data.
Example: Similar results were achieved in a study of attrition rates in 2 Southeastern colleges of
pharmacy conducted by P.T. Jones (unpublished data, 2009).
Tables and Figures
Tables.should not duplicate information provided in the text. Instead, tables should be used to provide
additional information that illustrates or expands on a specific point the author wishes to make. Each table
should be self-explanatory and begin on a separate page in the document. Tables should be numbered
using Arabic numbers according to the order in which they are referred to in the text. Tables should be
created using Microsoft Word table formatting tools (do not use the tab key to form rows and columns of
data as tab information is lost when the document is processed by the publisher). The table number and
table title should be placed in the first row (merged to form a single cell) of the table. Data must be placed in
separate cells within the table to prevent text and numbers from shifting. Footnotes should be placed at the
bottom of the table inside a single row and ordered using superscript lower-case letters (beginning with a)
rather than footnote symbols. A superscript letter that refers the reader to the corresponding footnote should
be inserted in the table title or body of the table. Refer to a current issue of the Journal for examples of table
style.
Figures. Figures should be numbered using Arabic numbers, based on the order in which they are
presented in the text. Figure legends should be concise and self-explanatory. All illustrative materials for the
figures should be submitted as high-resolution gif or jpg files. The key to any symbols in a graph or chart
should be included as part of the illustration itself, rather than in the legend. Do not place a box around
graphs or other types of figures. If figures contain illustrations that have been published elsewhere, a letter
of permission to reprint from the original publisher must accompany the manuscript.
A graphic image embedded in a MS Word file has a resolution of 188 dpi or lower. As a result, when printed,
graphics and text within the graphic may look fuzzy. If possible, send graphics with a resolution of at least
300 dpi. Large and/or high-resolution graphic images saved as TIFF or EPS images should be uploaded to
Editorial Manager as separate files from the manuscript text (Word file). Use Arial font for any lettering within
the graphic images. Figures, symbols, lettering andnumbering should be clear and large enough to be
legible when reduced. The minimum font size that should be used within the figure is 10 pt and the
maximum is 14 pt. As a general rule, the final graphic should be no more than 6.5 inches in width, large
enough to span a single Journal page.
Manuscript Submission
Please submit your manuscript using AJPEs Editorial Manager online tracking system at
http://ajpe.edmgr.com. Log in using your username and password and then follow the step-by-step on-
screen instructions for uploading your files. If you do not know your username and password or need to
have an account created for you, please send an e-mail to ajpe@cop.sc.edu and someone will respond as
quickly as possible.
NOTE:
When you attempt to log in, you may get an alert message stating Editorial Manager requires
browser cookies (a tiny file placed on your computer so the site will remember who you are). Option 1: If
you're using Internet Explorer, go to Tools > Internet Options > Privacy. Move the setting to "low" (ie, accept
first-party cookies asking for personal information but no third-party cookies). Option 2: If you do not wish to
change your browser settings, simply click OK in response to the message, then log in twice (the first time it
will not work because there's no cookie to access, but the second time it will accept the information and
open Editorial Manager). If for any reason it is not possible for you to submit your paper using AJPEs
Editorial Manager site as outlined above, send your manuscript as an e-mail attachment to ajpe@cop.sc.edu
and someone will assist you.
Copyright Form. Manuscripts submitted to the Journal should be unpublished and not under
consideration elsewhere. Under the terms of the Copyright Revision Act of 1976 (Public Law 94-533) it is
necessary to have the rights of the authors transferred to the publisher in order to provide for the widest
possible dissemination of professional and scientific literature. The editorial office must receive a transfer of
copyright form before a manuscript can be published online. The author may mail the form at the time a
paper is submitted or wait until it is accepted for publication.
Formatting, Copyediting, and Proofing of Accepted Manuscripts
Copyediting Stage. Prior to publication, all manuscripts are copyedited for organization, style, and
clarity. Authors may be asked at this stage to reorganize a manuscript or shorten the text.
Proofing Stage. Authors will receive an e-mail with a link to online electronic galley proofs (eProof) of
their paper for review approximately 10 days prior to publication. Authors may send an e-mail with an
annotated PDF (corrections entered using Adobe Acrobat software) attached, or they may print out the
eProof, mark corrections on the copy, and fax only those pages with corrections to the journal office (fax:
803-777-3097). The Journal allows authors 2 business days to return eProofs.

American Journal of Pharmaceutical Education-7

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

American Journal of Pharmaceutical Education-7

Transféré par

Droits d'auteur :

Formats disponibles

Vol ume 78, Issue 7, 2014

Rufus A. Lyman, University of Nebraska 1937-1955

American Journal of Pharmaceutical Education

727 King Street, Alexandria, VA 22314

Copyright 2011, American Association of Colleges of Pharmacy, 1426 Prince Street,

Naser Alsharif, PhD

Deborah Harper-Brown, PharmD

Stuart Haines, PharmD

Cecilia Plaza, PharmD, PhD

Vol ume 78, Issue 7, 2014

Turning the World of Pharmacy Education into a Global Community

Complex Issues Affecting Student Pharmacist Debt

A Three-Year Study of the Impact of Instructor Attitude, Enthusiasm, and

Identifying Achievement Goals and Their Relationship to Academic

Self-Efficacy and Self-Esteem in Third-Year Pharmacy Students

Investigating the Relationship Between Pharmacy Students Achievement

INSTRUCTIONAL DESIGN AND ASSESSMENT

Long-term Effectiveness of Online Anaphylaxis Education for Pharmacists

A Physical Assessment Skills Module on Vital Signs

Student Self-Screening for Methicillin-Resistant Staphylococcus Aureus

Using Curriculum Mapping to Engage Faculty Members in the Analysis

Improved Knowledge Retention Among Clinical Pharmacy Students

An Active Learning Complementary and Alternative Medicine Session

A Multiyear Analysis of Team-Based Learning in a Pharmacotherapeutics Course 87-95

American Journal of Pharmaceutical Education 2014; 78 (7) Article 1.

Associate Editor, American Journal of Pharmaceutical Education

Barriers to delivering quality pharmacy education in

resources, pharmaceutical education expertise and experience, and research capacity.

Corresponding Author: Claire Anderson, PhD, BPharm,

American Journal of Pharmaceutical Education 2014; 78 (7) Article 1.

academics and scientists in collaborative research and

American Journal of Pharmaceutical Education 2014;78(7) Article 2.

College of Pharmacy, University of Kentucky, Lexington, Kentucky

higher education funding over the last several years. At first,

REDUCED STATE SUPPORT OF HIGHER

American Journal of Pharmaceutical Education 2014;78(7) Article 2.

STUDENT PERSONAL FINANCE

Students and their families must accept personal

INCREASED PERSONNEL FOR

American Journal of Pharmaceutical Education 2014;78(7) Article 2.

principles to students prior to graduation, but some students

Recent significant expansion of pharmacy education

American Journal of Pharmaceutical Education 2014;78(7) Article 2.

for College Affordability and Productivity. http://www.

American Journal of Pharmaceutical Education 2014;78(7) Article 3.

Key to this relationship is having instructors who can

American Journal of Pharmaceutical Education 2014;78(7) Article 3.

less than 0.05 was considered signicant. An exempt status

A convenience sample that included distance

The study involved 187 (39.6%) distance students

American Journal of Pharmaceutical Education 2014;78(7) Article 3.

Three Year Average

items, the distance students (96%) and campus students

Distance students responded more positively to

American Journal of Pharmaceutical Education 2014;78(7) Article 3.

American Journal of Pharmaceutical Education 2014;78(7) Article 3.

and providing different ways to learn content (Table 3).

GPA with teaching style and student intrinsic motivation.

American Journal of Pharmaceutical Education 2014;78(7) Article 3.

American Journal of Pharmaceutical Education 2014;78(7) Article 3.

22. I am highly motivated about the

and engaged in the class. However, it is difcult to get all of

more comfortable with e-mail and text communication