Reflections on Defining and

Measuring Quality of Emergency

Care in Denmark

Philip D. Anderson, MD
Assistant Professor, Harvard Medical School
Department of Emergency Medicine
Beth Israel Deaconess Medical Center
Boston, Massachusetts USA
Overview
• EM-specific quality and performance
measurement

• The Balanced Scorecard approach

• Categories and Examples of Indicators

2
Progress towards a new model of
emergency care delivery in Denmark

• Specific plans from all Regions describing implementation of

recommendations from Sundhedsstyrelsen

• Fagomraadsbeskrivelsen for akutmedicin by DMS – education

based on this has started

• Many FAME enheder established at regional hospitals

• Agreements with primary sector and psychiatry for cooperation with

FAME enheder

• New national model for klinisk basisuddannelse – all nye læger

spend some time in FAM

3
Defining and Measuring Quality:
A Significant Challenge Worldwide

• Multi-dimensional nature of Quality

– Safe, Effective, Patient-centered, Timely, Efficient, Equitable
– Meaningful understanding requires multiple measures

• Complexity of emergency care delivery

– Wide range of patients, providers, processes

• Outcomes alone inadequate measures of quality

– Variability due to many factors
– Isolating effect of EM care from subsequent care
– Difficult to interpret in terms of what to fix
– Infrequent occurrence of bad outcomes = low statistical power

4
International Experience with Defining and
Measuring EM Quality
• EM quality measurement literature
– current concepts, strategies

• EM quality measurement
strategies in 4 countries
– Canada, UK, Australia, USA
– Wide range of EM specific
indicators
– “no one has a perfect strategy”

• Existing emergency healthcare

data gathering in Denmark
– Klinisk Epidemiologisk Afd.
Aarhus Universitet

5
Why is there an urgent need for EM
specific quality measurement tools?

• Historic national quality improvement initiative

– General goals, model for emergency care is clearly defined
– Many details of how to implement not clearly defined

• Individualized regional, local hospital strategies

– Increase potential for variability in implementation
– Some variability unavoidable to accommodate local circumstances
– Too much variability threatens larger quality goals

• Success depends on uniform standards

– What structure / process elements don’t want to compromise on?
– Incorporate these into quality standards

• Regions will be judged on success / failure of implementation

– Better to define own success criteria, rather than use someone else’s

6
What challenges do
Healthcare Leaders face?

• Clearly articulate a specific quality agenda for hospital-based

emergency care:
– What do we want to improve / change? (indicators)
– How will we measure success? (standards)

• Promote uniform development of FAM system:

– Organizational structures
– Clinical practice model
– FAM staffing, education and training

• Successfully balance multiple (competing) agendas:

– Quality / Safety / Satisfaction – Organizational / Operational
– Financial – Innovation / Sustainability

7
Unique Opportunities for
Emergency Medicine in Denmark
• Implement a world-class model of
emergency care delivery
– Strengths of Danish healthcare
system
– Build on international EM
experience

• Create a “best-practices”
framework for measuring quality,
impact of care
– Existing national quality tools
– National healthcare databases

8
Existing National Quality Tools:
• Den Danske Kvalitetsmodel (DKM):
– General accreditation model for all healthcare institutions
– Leadership and quality improvement tool
– Framework for developing standards

• Det Nationale Indikatorprojekt (NIP):

– Development, testing and implementation of healthcare quality
indicators
– Evidence-based,
– Diagnosis / condition-specific

These seem to be perfect; why is there a need for anything else?

9
 Potential limitations for use in
measuring emergency care quality?
• Early stages of development?
– Much EM specific content yet to be developed
– How long will it take to develop? When is a quality measurement tool needed?

• Too general?
– Do they contain the necessary detail to provide useful guidance for development
of emergency care system?

• Political dimensions?
– Committee-driven process w/ many stakeholders, many agendas
– Tend towards least controversial standards, maintain status quo
– Will they produce standards that push a necessary paradigm shift?

10
Existing emergency healthcare
data sources offer great potential:

• ”Vores overordnede konklusion er, at LPR kan anvendes

til en overordnet monitorering af akut området...”

• ”For at optimere monitorering af akutområdet bør den

nuværende registrering ændres og udvides til at omfatte
oplysning som tillader en bedre karakteristik af
indlæggelsesforløbene...”

Christensen, et al. Akutte indlæggelsesforløb og skadestuebesøg på hospitaler

i Region Midtjylland og Nordjylland 2003-2007, Klinisk Epidemiologisk Afd.,
Aarhus Universitet, 2009.

11
Electronic capture of administrative and
process data at departmental level

• Electronic time stamps for actions of interest

– Administrative data, demographics, patient
movements
– Computerized Provider Order Entry
– Test results, medications, interventions
• Automated reporting of benchmarking data
– Start simple : add more data elements over
time

12
 Landspatientregister
Regional EM Data
Regional administration
Reporting System
Local FAM administration

Health systems research

13
National Ambulatory Care
Reporting System (NACRS)
• The Canadian Institute for
Health Information (CIHI)
– an independent, not-for-profit
organization
– provides essential data and
analysis on Canada’s health
system and the health of
Canadians.
• The National Ambulatory Care
Reporting System (NACRS)
– contains data for all hospital-
based and community-based
ambulatory care:
– day surgery, outpatient clinics
and emergency departments.
• Individual ED patient visit level
data reporting
• 179 Emergency Departments
reporting in Ontario
• 82 data elements in 2009
version:
– Administrative
– Demographics
– Referral – Disposition**
– Chief complaint
– Acuity
– Time-motion data
– Therapeutic Interventions
**allows for accurate linking of “kontakter”
to reconstruct overall “patientforløb”
14
Charting the Course Forward
• Create EM specific quality measurement
tools
– Establish uniform standards
– Drive uniform development

• Develop national databases and IT tools

to meet emergency care data needs
– Clinical care, administration, research
– Input from relevant stakeholders is critical

• Balanced scorecard approach

– Align organizational strategy with
performance measurement
– Many perspectives and indicators to
consider

15
The Balanced Scorecard: a performance
measurement and strategic planning methodology
Kaplan and Norton. Harv Bus Rev, 1992;70(1):71-79

Priority is to Priority is to fulfill

Mission &
Financial generate profit for mission and satisfy
shareholders stakeholders Stakeholders

How do our Healthy finances a

Customer customers necessary condition Financial
perceive us? rather than ultimate goal

Processes at which we need

Internal to excel in order to satisfy our Internal
customers

Basic infrastructure to
Learning and Growth improve, create value and Learning and Growth
achieve mission

Not-for-profit (Healthcare)
For-profit organizations
organizations

16
Applying the Balanced Scorecard in
Healthcare Provider Organizations
Inamdar and Kaplan. J Healthc Manag. 2002;47(3):179-195

Study of 9 Provider Organizations Organizational Performance

• Integrated healthcare delivery systems improvement = 64%
• Academic medical centers • Volume of provided services
• Community hospitals
• Productivity
Benefit Themes: • Patient satisfaction
• Clarify and gain consensus on strategy • Utilization management
• Increase credibility of management
with board members
• Framework for executive decision Improved Financial Position =
making 76%
• Set priorities by identifying, • Cost reduction
rationalizing and aligning initiatives
• Revenue enhancement
• Link strategy with resource allocation
• Greater accountability
• Enabled learning and continuous
improvement

17
Ontario Hospital Association
Balanced Scorecard for ED Care
Hospital Report 2007: Emergency Department Care

• Publically financed healthcare system

• 25 indicators across 4 performance areas
• 124 participating hospital emergency departments across Ontario
• Voluntary participation: 109 (88%) 1 quadrant; 85 (69%) 4 quadrants
• “High-Performing” Hospitals identified for each quadrant

18
Balanced Scorecard for the
Dutch Health System
ten Asbroek at al. Int J Qual Health Care. 2004;16 Suppl 1:i65-i71

Perspectives and
indicator areas of Financial Perspective
balanced scorecard for • Health system costs
• Efficient use of resources
Dutch Health System • Financial viability

Internal Process Perspective

Consumer Perspective • Quality of healthcare delivery
• Effectiveness processes
• Patient safety • Concentration of care provision
• Patient satisfaction • Human resources (availability,
vacancies, satisfaction)

Innovation Perspective
• Funds for learning and growth
• Information infrastructure
• Innovative working environments
• Development and diffusion of
organizational innovations
• “Anticipate need for new
professionals for healthcare
delivery of tomorrow”
19
What system perspectives would
you prioritize?

?
? ?
?

Which indicators would you

choose?
20
 System Elements Related to Quality
Structure Process Outcome

“the resources we “what we do to “what happens to

use, and conditions patients in the patients as a result
under which, we process of delivering of our delivering care
deliver care” care to them”

“Good structures increase the likelihood of good processes, and

good processes increase the likelihood of good outcomes.”

Donabedian, JAMA, 1988

21
Categories & Characteristics of Indicators
Structure Process Outcome

• Material resources • Representative tasks • Health status

• Human resources • Representative • Patient satisfaction
• Organizational conditions
structures

• Indicators should be:

– Relevant (matter to stakeholders)
– Meaningful (can be influenced by healthcare system, room for improvement)
– Scientifically sound (validity, reliability)
– Evidence-based (causal linkage to desired outcomes)
– Measurable (clearly defined numerator, denominator, technically feasible to collect data)

• Indicators can change over time to reflect evolving quality agenda

22
 Structural Indicators of Emergency Care Quality

Structure Process Outcome

What characteristics of the • Material Resources

emergency care system affect the – Facilities
system’s ability to provide the – Equipment
– Financing
desired emergency care?
• Human Resources
– Type, number of staff
– Staff qualifications

• Organizational Structures
– FAM level
– Hospital level
– Regional level

23
Indicators Related to Material Resources
Structure
Material Resources:
Features of the FAM facilities that
describe how well suited it is to provide
efficient and effective emergency care
Does our FAM have the space / beds
that we need to care for our patients?
Do we have the access we need to key
functions to provide care effectively
Process Outcome
Examples:
FAM configuration:
 Contiguous clinical areas in FAM
 Admin/educ space adjacent to FAM
 Proximity to vagtlaege konsultation
Access to other hospital functions:
 x-ray, laboratory
 ICU, OR, cath lab
Hospital inpatient resources:
 Access to inpatient floor beds
 Access to inpatient ICU beds
24
 Indicators Related to Material Resources
Structure Process Outcome

Examples of Access related indicators:

• Access Block for ED patient, wait > 8 hours
– Percent of patients admitted, planned for admission but discharged,
transferred to another hospital or died in ED, whose total ED time
exceeded 8 hours
• Access Block for ED patients, wait > 4 hours
– Percent of mental health or critical care patients who wait greater than
4 hrs in the ED after the time of decision to admit them

Australasian Clinical Indicator Report: 2001-2007. ACHS, 2008.

25
Indicators Related to Human Resources
Structure
Human Resources:
Characteristics of the FAM staffing
that describe how well suited it is to
provide efficient and effective
emergency care
Do we have sufficient numbers of staff
to provide adequate clinical coverage?
Do our staff have the necessary
education and training to provide the
desired scope of care?
Process Outcome
Examples:
Adequate number of physicians for:
 24/7 clinical coverage?
 Administrative functions?
 Education (self, junior physicians)
Staff Qualifications:
 Percent of FAM physician staff with
recommended training in EM
 Percent of FAM nursing staff with
recommended training in EM
26
 Indicators Related to Organizational Structures
Structure Process
Organizational Structures:
• Characteristics of the organization,
policies and practices that:
– encourage delivery of cost effective,
high quality care
– support growth and development of
robust, sustainable FAM organization
• Attractive work environment
• Desirable career choice
• Professional recognition
Outcome
Examples:
System Integration:
 Use of Standardized Protocols
 Internal coordination of care
 External partnerships
Medical Informatics:
 Clinical data collection
 Use of clinical information technology
FAM Department Leadership:
 reports directly to hospital leadership
 able to hire own physician staff
Strategic Alignment of Incentives:
 appropriateness of inpatient admission
 employment, compensation models
27
 Indicators Related to Organizational Structures
Structure Process Outcome

Organizational Structures:

• Characteristics of the organization, policies and practices that:

– encourage delivery of cost effective, high quality care
– support growth and development of robust, sustainable FAM organization
• Attractive work environment
• Desirable career choice
• Professional recognition

28
Process Indicators of Emergency Care Quality
Structure Process Outcome

What did we do to the patient? • Representative tasks performed

How well was it done? in the FAM
– Diagnostics
Process measures ideally need
– Therapeutics
compelling evidence linking them to
– Others
desired outcomes to be valid

When hard evidence doesn’t exist, • Representative conditions seen

process measures can be based on in the FAM
expert consensus – Common problems
– Across spectrum of acuity

29
 Process Indicators of Emergency Care Quality
Structure Process Outcome

Representative tasks performed in the FAM:

• Triage • Emergency stabilization

• Focused history and physical exam • Diagnostic studies

• Determine diagnosis • Therapeutic interventions

• Pharmacotherapy • Observation and reassessment

• Consultation and disposition • Prevention and education

• Documentation

Thomas et al. Acad Emerg Med. 2008:15(8);776-779

30
 Process Indicators of Emergency Care Quality
Structure Process Outcome

Representative conditions encountered in FAM:

• Treated in most FAM
• Wide spectrum of age groups
• Represent different degrees of patient acuity
• Common reasons for seeking emergency care
• Evidence that “best practice clinical care” in FAM may have impact on patient
outcome or lead to enhanced clinical efficiency
• Rare conditions or where improving FAM care unlikely to change patient outcomes
should be excluded

Lindsay et al. Acad Emerg Med, 2002, 9(11):1131-1139.

31
 Process Indicators of Emergency Care Quality
Structure Process Outcome

Example of a set of Representative Conditions:

• Asthma
• Pneumonia
• Acute myocardial infarction
• Deep venous thrombosis / pulmonary embolus
• Chest pain
• Minor head trauma
• Ankle / foot trauma

Lindsay et al. Acad Emerg Med, 2002, 9(11):1131-1139.

32
 Process Indicators of Emergency Care Quality
Structure Process Outcome

Examples of disease specific process measures:

• Asthma
– Beta-agonist administration in all patients presenting to the ED with an
exacerbation of asthma (within 15 minutes of arrival in ED)
– Corticosteroid administration in all ED patients with asthma with:
• 1) moderate to severe exacerbations,
• 2) failure to respond promptly to inhaled beta-agonists,
• 3) admitted to hospital,
• 4) already on steroids at time of ED arrival
– Oral corticosteroids at discharge in all asthmatic patients who meet criteria to
receive steroids in the ED

Sullivan et al. Acad Emerg Med. 2007; 14:1182–1189

33
 Process Indicators of Emergency Care Quality
Structure Process Outcome

Examples of disease specific process measures:

• Acute myocardial infarction
– Timely ECG in all patients who present to ED with symptoms suggestive of
ACS (door-ECG time < 15 minutes)
– Delivery of aspirin / anti-platelet agent to all patients without contraindication
who present with symptoms suggestive of ACS
– Delivery of reperfusion therapy to all AMI patients who meet criteria for
reperfusion therapy
• door-to-needle time < 45 minutes for iv thrombolytic therapy
• door-to balloon time < 60 minutes for primary angioplasty

Sullivan et al. Acad Emerg Med. 2007; 14:1182–1189

34
Process Indicators of Emergency Care Quality
Structure Process Outcome

Examples of Laboratory Turnaround Time indicators:

• Urgent serum potassium
– result within 60 minutes, during normal working hours
– result within 60 minutes, out of hours
• Urgent haemoglobin
– result within 60 minutes, during normal working hours
– result within 60 minutes, out of hours

Australasian Clinical Indicator Report: 2001-2007. ACHS, 2008.

35
Outcome Indicators of Emergency Care Quality
Structure Process Outcome

What happened to the patient as a • Health Status

result of the care that was provided? – Morbidity
– Mortality (???)
To what extent can we expect changes – Disability
in FAM care delivery to change the
outcome? • Patient Satisfaction
– Overall Impressions
Need for risk adjustment of outcomes? – Communication
– Consideration
– Responsiveness

36
Outcome Indicators of Emergency Care Quality
Structure Process Outcome

Examples of Outcome Indicators:

• Proportion of Pneumonia Patients with Inpatient LOS ≤ 2 days
• Return Visit Rate for Asthma (≤ 24 hrs, 24-72 hrs)
• X-ray Rate for Ankle or Foot Injury Patients
• Return X-ray Rate for Ankle of Foot Injury Patients (≤ 7 days)

Hospital Report 2007: Emergency Department Care. Ontario Hospital Assoc.

37
Outcome Indicators of Emergency Care Quality
Structure Process Outcome

Examples of Outcome Indicators:

• Percentage of ED visits where Adverse Drug Event recorded
• Percentage of ED visits where Adverse Transfusion Event recorded

Australasian Clinical Indicator Report: 2001-2007. ACHS, 2008.

38
Outcome Indicators of Emergency Care Quality
Structure Process Outcome

Examples of Patient Satisfaction Indicators:

• Overall Impressions
– Patients’ assessments, overall, of their ED stay
• Communication
– Patients’ assessments of how well information was communicated to them or their family
during their ED stay
• Consideration
– Patients’ assessments of whether they were treated with respect and courtesy by doctors,
nurses and staff during their stays in the ED
• Responsiveness
– Patients’ assessments of the amount of time they waited to see doctors and nurses and
receive test results, assessments of pain management; assessments of team work; and
staff’s responsiveness to their needs

Hospital Report 2007: Emergency Department Care. Ontario Hospital Assoc.

39
 Putting it all together
• Excellent foundation with existing quality frameworks, metrics, data
gathering
– Danske Kvalitetsmodel (DKM), National Indikator Project (NIP)
– Patients Administrative Systemer (PAS)
– Landspatientregistret (LPR)

• Simplified overview of key perspectives, indicators to drive uniform

development
– Balanced scorecard

• Additional indicators to create a meaningful framework of measures:

– Wide range of examples from international experience
– Structural conditions that support development of effective FAM system
– Focus on patients, processes seen in the FAM

40