Human Factors Assessment of Devices

CBEB Presentation, October, 2012 Tony Easty, PhD PEng, CCE,

Centre for Global eHealth Innovation, Toronto, Canada

Video Clip Medication Error

How did we get this information?

! Recruit typical users as volunteers ! High fidelity simulation of a clinical environment ! Train participants before each experiment ! Look for replication of problems, saturation

Human Factors Engineering

Human Factors Engineering

! Aims to improve the design of devices, and systems to conform to user needs, capabilities, and limitations (both physical and cognitive) ! Users within the system boundary
patient environment device System boundary

users

Centre for Global eHealth Innovation Labs

! 15,000 square feet within University Health Network
! Includes testing and observation rooms ! Advanced remote observation via web-streaming and video conference

! Multi-tasking Simulation Laboratory

! Using theatre style props space can be transformed into range of real-world settings where health care is delivered

Centre for Global eHealth Innovation Labs

Why do Human Factors Evaluations?

! improved adoption of technology ! improved efficiency ! decreased training ! improved patient safety

Areas for Inclusion of Human Factors

!Medical Device and Healthcare IT Evaluations
! Clinical & technical reviews to short-list products ! Heuristic evaluation ! Low & high fidelity usability evaluation

!Iterative User-Centered Design

! For use during device development cycle

Our Methodologies
! Ethnographic Observation and Workflow Analysis ! Heuristic Analysis ! Low and High Fidelity Simulation

Ethnographic Observation and Workflow Analysis

! Conduct observational studies in the environment in which the technology is used ! Understand the limitations of the technology in the environment ! Understand the constraints the environment places on the use of the technology

Heuristic Analysis
! Seeking to discover ! Systematic rules-based review of a technology ! Typically performed by one or more trained human factors engineers

Nielsen-Shneiderman Heuristics*
! Consistency & standards ! Visibility of system state ! Match between system and world ! Minimalist ! Minimize memory load ! Informative feedback ! Flexibility and efficiency ! ! ! ! ! ! ! Good error messages Prevent errors Clear closure Reversible actions Use users language Users in control Help and documentation

*Zhang et al. Using usability heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics 2003;36:23-30.

Usability Testing
! Actual simulation of the use of a technology
! Low or high fidelity environment

! Can be performed in a real clinical environment or a lab ! Scripted series of encounters with a technology, observed and analyzed

Usability Testing (Low Fidelity)

Usability Testing (High Fidelity)

! 3 Anesthesia Information Management Systems tested ! High fidelity testing because of complicated systems and environment ! 17 anesthesiologists participated

High Fidelity Usability Testing

High Fidelity Usability Evaluation

! Representative end users perform realistic tasks in simulated real-world environments ! Environmental design considers ! Lighting ! Noise ! Space ! Scenarios take into consideration ! Multi-user interactions ! Interruptions

High Fidelity Usability Evaluation

! Participants are audio and video taped while verbalizing their thoughts ! Video is analyzed across participants for: ! Interface issues ! Training implications ! Patient Safety Implications

Evaluation Criteria
! Efficiency & Speed ! Task completion time ! Task accuracy & error frequency ! Number of requests for help ! Number of attempts to correct errors ! Impact & Severity of error (patient safety) ! Workload/Difficulty scale (1-7) ! User feedback & preferences

How Applicable?
!Technology
! Medical Devices, Healthcare IT Applications

!Processes
! How is the technology integrated into overall workflow?

!Communications
! Are these verbal, written, electronic?

!Environment
! Ambient noise, lighting, interruptions, layout and ergonomics

! ! ! Role of Human Factors in the ! Product Selection Process! ! ! - Electrosurgical Generators "

Simplified ESU Interface

Automatic External Defibrillators - Study Objective ! To identify the safety issues of 4 leading Automatic External Defibrillators (AEDs) ! Heuristic evaluation and high fidelity usability testing (8 participants)

AED

Usability Testing Example: Opening the Case

Usability Testing Example: Inadvertently Turning Device Off

Study Outcomes
! Purchase of the top AED from the usability testing ! Installation of AEDs in strategic areas across UHN and MSH

Some General Outcomes

! Reduction in errors with technologies selected with Human Factors evaluation ! Staff are empowered by Human Factors training and knowledge ! Industrial Collaborators release new device to enthusiastic user response ! Education can be targeted to identified problem areas of use

What Can We All Do?

! Observe how clinical users interact with devices (workflow analysis) ! Try your own heuristic analysis does the sequence of operations make sense to you? ! Target training on devices to address problem areas

Conclusions
! HF evaluation is a necessary process to help ensure a safer patient environment when selecting medical devices or IT systems for purchase ! Educational approaches can be developed based on outcomes, and general Human Factors training empowers staff ! Incorporating HF into iterative design process results in safer, easy to use products for the future

Thank you!
Contact: tony.easty@uhn.ca