Final Canadian National Delphi Consensus Results What Are The Appropriate National Clinical

Pharmacy Key Performance Indicators (cpKPI) For

Canadian Hospital Pharmacists?

OlavoFernandesBScPhm,ACPR,PharmD,FCSHP
DirectorofPharmacy-Clinical,UniversityHealthNetwork,TorontoON
AssistantProfessor(Status)-LeslieDanFacultyofPharmacy
SeanK.Gorman,BSc(Pharm),ACPR,PharmD
RegionalCoordinator-ClinicalQualityandResearch,PharmacotherapeuticSpecialistCriticalCare
InteriorHealthAuthority,ClinicalAssociateProfessorFacultyofPharmaceuticalSciences,UBC
KentToombsBSc(Pharm),ACPR
ClinicalPharmacyManager,CapitalDistrictHealthAuthority,Halifax,NS

CanadianHospitalPharmacyLeadershipConference,June8,2013

Objectives
Tooutlinethekeyelementsofthenationalconsensusprocessin
developingclinicalpharmacykeyperformanceindicators(cpKPI)
forhospitalpharmacists
includingconsensusdefinition,selectioncriteriaforcpKPI,criticaltopic/
activityfociandpre-DelphicandidatecpKPI)

ToreportthefinalresultsoftherecentnationalDelphiconsensus
phasetoestablishafinalsuiteofcpKPI
Tosummarizethenextphasesandcommunicationplansinthe
nationalcpKPIprocess:
1.
2.
3.
4.
5.

exploringinterprofessional/externalstakeholderfeedback,
nationalinformationcapture/measurementsystems,
cpKPIknowledgetranslationkit
practicaldefinitionandmeasurementquestions
pan-Canadiancommunication/Manuscriptpublications/posters
2

Overall Goal of the National cpKPI Collaborative /

National Consensus Process
Todevelopacoresetofnationalclinical
pharmacyKPIforhospitalpharmacistsviaa
systematicnationalevidence-informed
consensusprocess

Key Performance Indicators (KPI)

Whatisit?
Quantifiablemeasuresthatreflectthe
criticalsuccessfactorsofanorganization 1
Quantitativemeasuresofquality
Whyisitimportant?
Elevateprofessionalaccountability&
transparency
Servetoimprovequalityofcare
1.DoucetteD,MillenB.ShouldKeyPerformanceIndicatorsforClinicalPharmacyServicesBe
Mandatory, Can J Hosp Pharm 2011;64(1):55-57.

Rationale for clinical pharmacy KPI (cpKPI)

GAP:currentlyNOestablishednationalor
internationalconsensusonwhatconstitutesa
KPIforclinicalpharmacyservices
Rationale:Toadvancepracticetowarddesired
evidence-informedpatientoutcomes
cpKPIwillservetobetterdefineminimum
standardsandpermitbenchmarkcomparisons
withinandbetweenorganizations
5

National cpKPI Collaborative

Definition of cpKPI
Fivepillars/characteristicsofcpKPI:
1.
2.
3.
4.
5.

Reflectadesiredqualitypracticeand
Ametricwithalinktodirectpatientcareand
Linktoevidenceofimpactonmeaningfulpatient
outcomesand
Apharmacy/pharmacistsensitivemetric
Feasibletomeasure

. Clinicalmetricwouldhavetofulfillall5pillarsto
qualifyasacandidatecpKPI

Hierarchy of Study Outcomes (AHRQ)

Level1:ClinicalandQoLoutcomes
Morbidity,mortality,adverseevents

Level2:Surrogateoutcomes
I.e.bloodglucose,bloodpressure,cholesterol

Level3:Measureablevariableswithanindirector
unestablishedconnectiontotargetoutcome
I.e.medicationdiseasestateknowledge

Level4:Indirectvariables
I.e.patientsatisfaction,potentialadverseevents

Should Align with Local Consensus or Guidelines for

Prioritization of Hospital Pharmacist Activities

6Domains
1. Pharmaceuticalcarepatientcareprocess
2. Operationalpatientcaresupportingactivities
3. Druginformation
4. Teaching/Education/Learning
5. Research
6. Service(clinicalandpharmacycommittees)
*ExtractedfromUHNPharmacistPyramid-PrioritizationofPharmacistActivitiesDraft

Information Gathering - Prior to

Consensus Building
Front-line
Staff/Leaders
CSHP2015/
CPhABlueprint

Literature:
1.Evidence
2.Process

Optimal
National
cpKPI
PeerHospital
BestPractices

Pharmacy
Leadership

Proposed Timeline
Pre-Delphi

Delphi

Post-Delphi

KPIWGformed
May2011

Information
Gathering

Delphi
Process
Dec12Mar13

Aug2011

CSHP
endorsed
concept

Survey
Development

Wearehere
Feb2013

Consensus
Meeting

Key National Process Milestones

1.
2.

NationalconsensusdefinitioncpKPI(Aug2011)
NationalCrudeInventoryofcandidatecpKPI/metrics(startedJan
2012)
NationalInformation-gatheringProcess:Workshops/Information
sessions-Frontlinefeedback(Feb2012-Nov2012)
OutcomeandProcessDebates/FinalizedEvidencesummarytables
(June-July2012)
AprioriconsensuscpKPIselectioncriteria(idealattributes)Slavik
11(FinalizedJuly2012)
KeycpKPICriticalActivity/TopicAreasDoucette8(Finalized
August2012)
FinalPre-DelphiCandidatecpKPIlist(October2012)
SelectionofNationalDelphiPanelmembers(November2012)
DelphiPanelProcessRound1-3(Dec21,2013-Mar82013)

3.
4.
5.
6.
7.
8.
9.

cpKPILiveMeeting(February5,2013)

11

Slavik -11- Consensus Criteria Ideal Attributes

Basedonhighqualityliteratureevidence
(e.g.Observationaldatavs.RCTvs.systematicreview)

Relevantimpactonclinicallyimportantoutcomes
(e.g.Surrogateversusclinicalendpoints,effectsizeofintervention)

Best-suitedtopharmacistsrole
(e.g.Identifiespharmacist-specificclinicalrolevs.GPvs.RN)

Attributabletodirectpatientcare
(e.g.Markerofclinicalintervention,notdistribution)

Specifictopharmaceuticalcareprocess
(e.g.Relatedtogenerally-acceptedPCprocesses)

Alignedwithprofessionalgoals,objectives,practices
(e.g.AccreditationCanadaROPs,standards,CSHPVision2015,etc.)
12

Slavik -11- Consensus Criteria Ideal Attributes

Accepteddisease-basedqualityindicator
(e.g.ACEIorBBforHF,VTEprophylaxisinhospitalizedpatients)

Feasibletomeasure
(e.g.Reliablemeasurementsystemscan/couldbeputinplace)

Efficienttomeasure
(E.g.Acceptabletimecommitment,useable)

Valuablequalitymeasure
(E.g.Prevalent,impactfulproblemwithpractical,proveninterventions)

Generalizability
(E.g.Versatileenoughtobeappliedinlarge,academicandsmallcommunity
sites)
13

Delphi panelist priority ranking of consensus

cpKPI selection criteria- Final Mar 2013

14

RCT Outcome Findings

Bringing the evidence all together

with
Gillespie
U et extrapolation
al. 2009- RCT
Bond et. al. (2007) Observational
Study
Clinical Pharmacy & Mortality

1. admissiondrughistories
2. medicalroundsparticipation
3. CPRteamparticipation
Kaboli PJ et al. (2006) Systematic Review

4. attendanceonpatientcare
rounds
5. patientinterviewsand
assessments
6. medicationreconciliation
7. dischargecounselling

(patientmedicationeducation)

8. follow-upafterdischarge

Integrated Intervention pharmaceutical care

Integrated Intervention
1. post-discharge hospital visits (ED +
readmissions)
2. emergency department visits
3. drug related readmissions
Makowsky MJ et al. 2009- RCT
1. overall quality score
2. 3 and 6 month all-cause readmission
(hospitalorEDvisitafterindex
hospitaladmission)
Chisholm-Burns MA et al 2010, systematic
review w/ focussed meta-analyses
HbA1c , LDL Cholesterol, Blood Pressure
Adverse Drug Events

Evidence Summary Tables

Discussion:specificgroupsuggestionstomodifyor
concurwiththefollowsections

StrengthsandLimitations
Application/Synthesis:HowdoesthisstudyinformthecpKPI
selectionprocess(methods,cpKPIselectioncriteria,and
candidatecpKPI)?
Whatarethepatterns(similaritiesanddifferences)compared
tootherkeypapers?

Purpose:August-usedtorefreshandfocusoutcome
evidenceforstreamlining;Nov-UsedbyDelphi
paneliststosupportrankinganddecisionmaking

16

Levels of Evidence

1. ObservationalStudies
2. SystematicReviews
3. RandomizedControlledTrials

PRACTICE QUESTION
Doespharmacist-ledcomprehensivepharmaceutical
carereducemorbidity(&othermeaningfulpatient
outcomes)forelderlyhospitalizedpatients?

18

A Comprehensive Pharmacist Intervention to

Reduce Morbidity in Patients 80 years or Older
Gillespie U et al. Arch Intern Med 2009; 169(9):894-900.

Objectives:
assesstheeffectivenessofinterventionsperformedbyward-based
pharmacistsonmorbidityandoveralluseof(secondary)hospitalcare

Design:
prospective,singlecentre,unblinded,randomizedcontroltrial
patient-unitofrandomization,centralcentre
Setting:2acuteinternalmedicinewards(universityteaching
hospital)inUppsala,Sweden
Duration:Oct2005-June2006
Patients:
Patients80yearsorolderadmittedto2acutecareinternalmedicine
wards
Writteninformedconsent
Samplesizecalculation:400patients

A Comprehensive Pharmacist Intervention to

Reduce Morbidity in Patients 80 years or Older
Gillespie U et al. Arch Intern Med 2009; 169(9):894-900.

Patientsrandomizedto:
intervention(comprehensivecarebyhospitalpharmacist)
Wardbasedclinicalpharmacists
1. comprehensivepatientinterview,BPMH,admissionmedication
reconciliation,
2. pharmaceuticalcaredrugreview(Cipollemethod)toidentifyand
resolveDTPs,physicianinterventionsondrugselection,
dosages,monitoring.
1. Addressed:indication,effectiveness,safetyandadherence
2. DTPsdiscussedonwardrounds
3. Patientsreceivededucationanddischargecounselling/
reconciliation
4. pharmacistdischargelettercommunicatedtoprimarycarephysiciansby
pharmacists
5. Followuptelephonecall2monthsafterdischarge

control:standardcarewithoutpharmacistinvolvementby
physiciansandnurses

Results: Major Outcomes

Gillespie U et al. Arch Intern Med 2009; 169(9):894-900.

PatientsEvaluated(n=368,182intervention/186control)
overa12monthperiod

Post-DischargeHospitalVisits(ED+readmission)
16%interventiongroup
(quotient1.88vs.2.24,95%CI0.72-0.99)

EmergencyDepartmentVisits:
47%interventiongroup
(quotient0.35vs.0.66,95%CI0.37-0.75)

DrugRelatedReadmissions:
80%interventiongroup
(quotient0.06vs.0.32,95%CI0.10-0.41)
Aside:BalancingMeasures-ReadmissionsAloneandMortality:
-Nosignificantdifference

Gillespie U et al. 2009- RCT

IntegratedInterventionpharmaceutical
Bringing the evidence all together
with extrapolation
careIntegratedIntervention

1. post-dischargehospitalvisits(ED+
Bond et. al. (2007) Observational
readmissions)
Study
2. emergencydepartmentvisits
Clinical Pharmacy & Mortality
3. drugrelatedreadmissions
1. admissiondrughistories
2. medicalroundsparticipation
Makowsky MJ et al. 2009- RCT
3. CPRteamparticipation
1.overallqualityscore
Kaboli PJ et al. (2006) Systematic Review

4. attendanceonpatientcare
rounds
5. patientinterviewsand
assessments
6. medicationreconciliation
7. dischargecounselling

(patientmedicationeducation)

8. follow-upafterdischarge

2.3and6monthall-causereadmission
(hospitalorEDvisitafterindex
hospitaladmission)
Chisholm-Burns MA et al 2010,
systematic review w/ focussed
meta-analyses
HbA1c,LDLCholesterol,Blood
Pressure
AdverseDrugEvents

Doucette 8- Consensus Critical Activity / Topic

Areas
1. PharmaceuticalCareIntegrated(DTPassessment/
careplan/monitoring)
2. MedicationReconciliation-BPMH/MedHistoryTaking
3. MedicationReconciliation-AdmissionReconciliation
4. MedicationReconciliation-DischargeReconciliation
5. Team(orPatient)Rounds
6. DischargePatientEducation/Counselling
7. PostDischargeFollow-Up
8. DiseaseorDrugSpecificBestPracticeQuality
Indicators
23

Doucette 8- Consensus Critical Activity / Topic

Areas

Dotvoting:20dotsperperson
Assignproportionately
Question:
WillmeasuringacpKPIinthiscritical
activitytopicareabeusefultoadvance
clinicalpharmacypracticetoimprovethe
qualityofpatientcare?
Semchuk-26DraftCandidateKPIlist
24

DEMOGRAPHICS OF cpKPI DELPHI

PANEL

25

Whatisyourpracticesetting(checkallthatapply)?

69%
(18)
Other=Administration,PharmacyAssociation,
Overseemultiplesites,RegionalHealthAuthority,
LongTermCareandRehabilitationCentre,District
healthauthoritywithtertiaryandcommunitypractice.

35%(9)
27%(7)

27%(7)

23%(6)
12%(3)

Teaching
hospital

Community
Tertiarycare hospital
hospital

Academia

Other

Clinic

Doyouworkprimarilywithpediatricsoradults?

Adults

88%(23)

12%(3)

Pediatrics

Howmanyyearsofexperiencedoyouhaveasa
licensedPharmacist?
65%
(17)

19%(5)

8%(2)

8%(2)

0%(0)
0-5years

6-10years

11-15years 16-20years

20+years

Whatisyoureducationalbackground?

Other:MBA,BSc(Pharmacology),EXTRA
Fellowship(CFHI)CertifiedHealthExecutive(CHE),
MBA,PostPharmDResidency,CertifiedGeriatric
Pharmacist

100%
(26)

54%(14)

54%(14)

23%(6)

BScPhm

PharmD

Residency
(ACPR)

Other

19%(5)

Masters
Degree

DOUCETTE 8 20 Dot Voting

RESULTS

30

Doucette 8- Consensus Critical Activity / Topic

Areas
1. PharmaceuticalCareIntegrated(DTPassessment/care
plan/monitoring)
2. MedicationReconciliation-BPMH/MedHistoryTaking
3. MedicationReconciliation-AdmissionReconciliation
4. MedicationReconciliation-DischargeReconciliation
5. Interprofessional(team)patientcarerounds
6. DischargePatientEducation/Counselling
7. PostDischargeFollow-Up
8. DiseaseorDrugSpecificBestPracticeQualityIndicators

. UsedtocreateSemchuk26candidatecpKPIlist
31

Grape Analogy: BUNDLES AND

CRITICAL ELEMENTS

Modified Delphi Process Methodology

ADelphitechniqueisastructuredprocesscommonlyused
todevelopconsensushealthcarequalityindicators
Itwasdevelopedtominimizeinfluencefrommorevocal
groupmembers,andutilizessurveysorquestionnaires
insteadofdiscussion.
frequentlyusedwithexpertpanelstogenerateconsensuson
healthcareissues

Toarriveatconsensus,amodifiedDelphitechniquewillbe
used.
Thismodifiedtechniqueisaniterativeprocessthatbuilds
consensususingthreeroundsofanonymouspanelistratingswitha
live/tconmeeting
34

Delphi Rounds
A. StandardizedOrientation

AudioPowerPoint+MandatoryPre-Reading

B. Round1

DemographicInformation;PanelistranksSemchuk26cpKPI,For
eachSlavik11andOverallRanking,SuggestnewcpKPI

C. Round2

ReviewR1aggregatesummary/reportcardforeachcpKPI
FrequencyGraphsSummary
Reviewanonymousqualitativecomments
Panelistre-ranksallcpKPI

D. LiveMeetingDebateandDiscussiontoinformindividualrankings

identifymeetotherpanelistsforthefirsttime

E. Round3
ReviewFeb5LiveMinutes,R2summaries(asabove),Final
Rankings
35

Delphi Rounds
1.
2.
3.
4.

IndividualcpKPIratings
OpportunitytosuggestadditionalcandidatecpKPI(round1only
toallowpanelfeedback)
RankingofpriorityofDoucette8CriticalActivitiesandSlavik
11SelectionCriteria
CombiningcpKPI,CreatingNewcpKPIbymodifyingworking(ie
cpKPI27,28,30)

. Thresholdforconsensusconsideration:

75%ofpanelistsassignaratingof7-9onthe9pointLikertscale
MAGICNUMBER=20

36

Round 1 Qualitative Panelist Discussion Themes

QualitativediscussionthemeswhilecomparingcpKPI
included:
1. Varyingdegreesofsensitivitytopharmacists
contribution
2. Varyingdegreesoffeasibilityofmeasurement
3. Varyingdegreesofgeneralizabilityacrosspractice
areas(i.e..psychiatry,surgery)aswellasacross
differenttypesofhospitals(i.e.urbanversusrural)
4. Inter-relationshipsbetween:medication
reconciliationcpKPIs;discharge/inpatient
counsellingcpKPIs

37

Round 1 3 New cpKPI Submitted by Panelists

1.

cpKPI#27:combinedpharmacistadmissionMedRec+BPMH
Number(orproportion)ofpatientswhoreceiveformal
documentedadmissionmedicationreconciliationbya
pharmacist(includesapharmacist-BPMHORpharmacistBPMH-reviewaspartofreconciliationaswellasresolutionof
identifieddiscrepancies).

2.

cpKPI#28:Proactivebundle;Number(orproportion)of
patientsreceivingproactivecomprehensive,directpatient
carebyapharmacistincollaborationwiththehealthcare
team(MakowskyCollaborateRCTBundle).

3.

cpKPI#29:TimeonWardCommitteddecentralizedclinical
pharmacisttimeperpatientdayperpatientservice.
38

Live Feb 5: Meeting Issues and Controversies

1. Howtooptimallyhandleprocessofcarevs.
disease/drug-specificindicators?

2.
3.
4.
5.

HighValueActionDTPresolvedasasubset

GrapeTheory:BundlesandCriticalElements
Numbervs.proportion
AprioriSuiteproperties
HighRiskvs.AllPatients

Final Delphi Results

Round3FinalRankings

8cpKPIhaveofficiallymetconsensus
6/8DoucetteCategoriesrepresentedwithcombos

Final 8: cpKPI Number and Description

Proportion of patients who receive formal documented

discharge medication reconciliation and resolution of
identified discrepancies by a pharmacist (#11)
Number (or proportion) of patients who receive formal
documented admission medication reconciliation by
a pharmacist (combined BPMH) (#27)
Number (or proportion) of patients for whom clinical
pharmacists have completed
(executed/implemented) a pharmaceutical care
plan (#27)
Number (or proportion) of pharmacists who actively
participate in interprofessional patient care rounds
to improve medication management

Final 8: cpKPI Number and Description

Number of total drug therapy problems (DTPs)

resolved by pharmacists
Number (or proportion) of patients receiving "proactive
comprehensive, direct patient care by a pharmacist in
collaboration with the health care team" (Makowsky
Collaborate RCT Proactive Bundle) (#28)
Number (or proportion) of hospitalized patients who receive
medication counselling by a pharmacist at discharge
Number (or proportion) of patients who have received in
person education from a pharmacist about their
disease(s) and medication(s) during their hospital stay

How do the final national clinical pharmacy key

performance indicators align with national consensus
selection criteria?

43

National cpKPI Collaborative

Next Steps
7Post-DelphiPhases
1.

cpKPIknowledgetranslationkit-practicalgettingstartedkit

2.

Final8:PracticalOutstandingQuestions

3.

Wording,outstandingquestions,practicaldefinitions,practicalmeasurementissues

Exploringexternalstakeholderfeedback

4.
5.
6.
7.

Final8cpKPI-specificmeasurementsummaries,background,7stepchange
managementframework

Interprofessional:physicians,nurses,MinistryofHealth,pharmacists-US,
UK,NZ,Aus,patients,hospitaladministrators

Nationalinformationcapturesystems/measurementsystems(apps)
Pan-CanadianCommunicationofFinalDelphiResults
2ManuscriptPublications/7ConferenceAbstracts
FormalPilotSites