Vous êtes sur la page 1sur 25

Applied Ergonomics 54 (2016) 218e242

Contents lists available at ScienceDirect

Applied Ergonomics
journal homepage: www.elsevier.com/locate/apergo

Review article

Work system barriers to patient, provider, and caregiver use of


personal health records: A systematic review
Morgan J. Thompson a, Jeremiah D. Reilly b, Rupa S. Valdez b, *
a
Psychology Department, The College of William and Mary, Williamsburg, VA 23187, USA
b
Department of Public Health Sciences, University of Virginia, P.O. Box 800717, Hospital West Complex, Charlottesville, VA 22908, USA

a r t i c l e i n f o a b s t r a c t

Article history: Objectives: This review applied a human factors/ergonomics (HF/E) paradigm to assess individual, work
Received 19 May 2015 system/unit, organization, and external environment factors generating barriers to patient, provider, and
Received in revised form informal caregiver personal health record (PHR) use.
16 October 2015
Methods: The literature search was conducted using five electronic databases for the timeframe January
Accepted 18 October 2015
Available online 15 January 2016
2000 to October 2013, resulting in 4865 citations. Two authors independently coded included articles
(n ¼ 60).
Results: Fifty-five, ten and five articles reported barriers to patient, provider and caregiver PHR use,
Keywords:
Systematic review
respectively. Barriers centered around 20 subfactors. The most frequently noted were needs, biases, be-
Personal health records liefs, and mood (n ¼ 35) and technology functions and features (n ¼ 32).
Human factors and ergonomics Conclusions: The HF/E paradigm was effective in framing the assessment of factors creating barriers to
PHR use. Design efforts should address literacy, interoperability, access to health information, and secure
messaging. A deeper understanding of the interactions between work systems and the role of organi-
zation and external environment factors is required.
© 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
2. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.2. Literature search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.3. Title screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.4. Abstract screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.5. Full-text screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.6. Data extraction and synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
3.1. Search results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
3.2. Study characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
3.3. HF/E paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
3.4. Patient work system barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
3.5. Provider work system barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
3.6. Caregiver work system barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
4.1. Application of HF/E paradigm to barrier classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
4.2. Application of the HF/E paradigm to caregivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
4.3. Representation of individuals in the work system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
4.4. Implications for PHR design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

* Corresponding author.
E-mail addresses: mjthompson@email.wm.edu (M.J. Thompson), jdr5bd@
virginia.edu (J.D. Reilly), rsv9d@virginia.edu (R.S. Valdez).

http://dx.doi.org/10.1016/j.apergo.2015.10.010
0003-6870/© 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 219

4.5. Designing interventions: attending to context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229


4.6. Limitations and future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

1. Introduction location (Wynia et al., 2011), and concerns regarding appropriate


reimbursement and liability (Wynia et al., 2011). Low caregiver usage
As clinician and patient roles evolve, responsibility for health rates are associated with caregiving responsibilities for a child who
management is shifting from clinician-governed to patient- has a nonsevere illness (Byczkowski et al., 2014), slow Internet
controlled (Wasson et al., 2012). The continuing development of connection (Tom et al., 2012), insurance type (Tom et al., 2012), and
consumer health information technology (IT), health IT designed for fear of discrimination from insurance companies (Weitzman et al.,
use by lay people, supports this change in roles by enabling patients 2012). Given the multi-dimensionality of these rationales, HF/E
to actively engage in health management both alone and in part- frameworks should be systematically applied to understand barriers
nership with formal healthcare providers (i.e., providers) (Center for to patient, provider, and caregiver consumer health IT use.
Advancing Health, 2010). This engagement through consumer Previous research in HF/E has focused on the impact of work
health IT has been accelerated by market forces such as the prolif- system factors on healthcare providers' clinical health IT use, such as
eration of consumer-facing mobile health applications EHR and computerized provider order entry (CPOE) (Holden et al.,
(PricewaterhouseCoopers LLP, 2012; Greenspun and Coughlin, 2012) 2013; Carayon and 2006; Fuji et al., 2008; Nazi, 2013; Urowitz
and has been promoted by the Meaningful Use initiative within the et al., 2012). These studies demonstrate that health IT use in pro-
United States (The Office of the National Coordinator for Health fessional settings is shaped not only by individual characteristics,
Information Technology (ONC), 2013a; Center for Medicare and but also by a wider range of work system factors. Multiple models of
Medicaid Services, 2015): Stage 2 of this initiative (The Office of the work system (Carayon and 2006) emphasize that system out-
the National Coordinator for Health Information Technology comes are influenced by interactions between users, tasks per-
(ONC), 2013a) incentivizes increased health information exchange, formed, tools used to accomplish these tasks, and the physical,
patient-controlled data, requirements for e-prescribing and incor- social, and organizational environments in which users are
porating lab results, and electronic transmission of patient care embedded (Holden et al., 2013). While work system models tradi-
summaries across multiple settings. The recently proposed Stage 3 tionally characterize the sociotechnical systems of professional
guidelines (Center for Medicare and Medicaid Services, 2015) work, more recent models translate this concept for patients and
extend the Stage 2 requirements for patient engagement by caregivers (Holden et al., 2013; National Research Council, 2011;
increasing the requirements regarding the proportion of a provider's Carayon et al., 2006). These models enable the concept of the
patients who have interacted with their health records (i.e., viewed, work system to be applied to healthcare work conducted by both
downloaded, or transmitted their health records) and used secure healthcare providers and lay people, facilitating the study of work
messaging through the electronic health record (EHR) (Center for system factors in the context of consumer health IT use. In this re-
Medicare and Medicaid Services, 2015). view, barriers are categorized using the HF/E paradigm developed
Patient engagement in health management, however, does not by Karsh and colleagues (Karsh et al., 2006). Although this paradigm
solely rely on actions taken by patients and providers. Rather, health targets patients and providers, we extend and apply it to caregivers.
management in home and community settings often depends upon This review focuses on a specific type of consumer health IT,
the active participation of other individuals (Valdez and Brennan, personal health records (PHRs), because Meaningful Use Stages 2
2015; Skeels, 2010), including informal caregivers (i.e., caregivers). and 3 as well as the Patient Engagement Framework promote
Caregiver as well as patient and provider interest in accessing and increased patient, provider, and caregiver PHR use within the
exchanging health information has been documented (Friction and United States. Though varyingly defined (Archer et al., 2011;
Davies, 2008). The Patient Engagement Framework (National Connecting for Health Personal Health Working Group, 2003; The
eHealth Collaborative and Healthcare Information and Office of the National Coordinator for Health Information
Management Systems Society Foundation, 2014), championed by Technology (ONC), 2013b), PHRs in the present study are
the Healthcare Information Management Systems Society (HIMSS) “Internet-based set[s] of tools that allow people to access and co-
and the National eHealth Collaborative (NeHC), outlines a vision for ordinate their lifelong health information and make appropriate
Meaningful Use Stages 3 and 4 to further support, through con- parts of it available to those who need it” (Connecting for Health
sumer health IT, the full range of individuals (e.g., caregivers, family, Personal Health Working Group, 2003), (p3). Common PHR func-
friends, clergy) involved in patients' health management. The need tionalities include the ability to access lab results, engage in secure
to attend to caregivers' perspectives has also been recently identi- messaging with providers, request prescription renewals, schedule
fied within the HF/E literature (Holden et al., 2013, 2015). appointments, authorize referrals, and view and update medication
Despite widespread development and promotion of consumer and allergy lists (Archer et al., 2011). Studies included here report
health IT, usage rates have remained relatively low (Valdez et al., on PHRs that vary in terms of available features and of organiza-
2015b). Proposed explanations for these low patient usage rates tions through which patients were enrolled (e.g., employer, gov-
include low numeracy and limited technology experience (Taha et al., ernment agency, primary care physician). In contrast to previous
2013), difficult login procedures (McInnes et al., 2013), limited family reviews of PHRs (Archer et al., 2011; Jabour and Jones, 2013;
support for using advanced technologies (Mayberry et al., 2011), and Amante et al., 2014), this review is systematic, restricted to
fears concerning the misuse of stored information (Tjora et al., 2005). empirical peer-reviewed articles, focused on work system barriers
Reasons for low provider usage rates include no prior email to use by patients, providers, and caregivers, and based on an HF/E
communication with patients (Crotty et al., 2013), medical practice paradigm.
220 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

2. Methods or telemedicine systems (unless this had been integrated into a PHR
and the article examined both components), (d) were introductions
2.1. Scope or prefaces to a special issue, conference proceedings, or book, (e)
were review articles, and/or (f) were not written in English.
Our review was restricted to studies in which users were asked to Two authors (MJT & RSV) independently coded 100 (2.5%) of the
engage with PHRs in the context of their everyday lives. Studies abstracts for eligibility. The first set of 50 abstracts was used to
focused solely on lab-based usability testing or system demonstra- finalize inclusion and exclusion criteria. The second set was used
tions were excluded. This review categorized all factors identified as for calculating inter-rater reliability. The resulting value for Cohen's
barriers in quantitative, mixed methods, and qualitative studies. kappa statistic (k¼.90) was interpreted as strong (McHugh, 2012).
Traditionally, the term “barrier” is defined as “a law, rule, problem, When discrepancies occurred, decisions were discussed and
etc., that makes something difficult or impossible” (Merriam- resolved through consensus-building. Decisions were documented
Webster). However, for the purposes of this review the term “bar- and subsequently used by the first author to include or exclude the
rier” has been extended to allow factors associated with decreased remaining abstracts.
PHR use such as individual characteristics (e.g., race, age) to be
conceptualized as barriers. The scope did not include conducting a 2.5. Full-text screening
meta-analysis to assess the degree to which each factor affected PHR
use. Preferred Reporting Items for Systematic Reviews and Meta- Articles excluded during the initial full-text screening (a) met
Analyses (PRISMA) (Moher et al., 2009) criteria were adopted to abstract exclusion criteria, (b) were opinion articles, (c) were
the extent possible; however, we could not adopt the standards in formative papers, (d) were not full papers, (e) described paper-
full. For example, due to the range of methodologies among included based systems, (f) focused on system architecture, and/or (g)
articles, we were unable to assess risk of bias in a consistent manner. were unavailable after exhausting the University of Virginia's li-
brary resources. Two authors (MJT & RSV) independently screened
2.2. Literature search 10% of the full-text articles (n ¼ 21) before engaging in decision
making similar to that described for abstract screening. The first
A health science librarian and a computer science librarian were author then completed the first round of full-text screening.
consulted in developing the search strategy, including the search A second round of full-text screening ensured that included
terms. A range synonymous with personal health record and bar- articles focused on PHRs implemented in participants' everyday
rier (see Table 1) was used to ensure that all related articles were lives. During this round, articles were excluded if the PHR was (a)
included in the initial screening. The search was performed using not used, (b) used only during a demonstration, and/or (c) not
five electronic databases during October 2013: PubMed, CINAHL implemented in participants' everyday lives. Two authors (MJT &
(excluding MEDLINE records), Engineering Village (Compendex and JDR) individually reviewed all articles during this round. Inter-rater
INSPEC), IEEE Xplor, and ACM Digital Library. To keep search stra- reliability was moderate, according to Cohen's kappa statistic
tegies consistent across databases, controlled vocabulary was not (McHugh, 2012) (k¼.67). Discrepancies were resolved through
used. All searches were limited to articles published between consensus-building and unresolved differences were referred to
January 2000 and October 2013. the senior author (RSV).

2.3. Title screening 2.6. Data extraction and synthesis

Title screening consisted of reviewing search results for dupli- Two authors (MJT&JDR) independently extracted the following:
cates and publication in peer-reviewed journals or conference
proceedings. Duplicates were screened for manually by the first  sample (patient, provider, or caregiver)
author and the RefWorks duplicate finder tool (ProQuest LLC, 2010).  sample size
Ulrichsweb was used to exclude non-peer-reviewed journals and  type of PHR (tethered or untethered)
conference proceedings. If a conference proceeding was not listed  instrumentation (survey, interview, focus group, advisory panel,
on Ulrichsweb, we determined whether it was peer-reviewed PHR system logs, and EHR patient data)
based on the call for participation.  study design (quantitative, qualitative, or mixed methods)
 study location (country)
2.4. Abstract screening  journal domain (as classified in Ulrichsweb)
 patient characteristics (health condition, race, age)
Abstracts considered eligible and included in full-text screening  provider characteristics (medical practice, age)
(a) mentioned PHRs and (b) related to patients', providers', and/or  caregiver characteristics (relationship to patient, race, age)
caregivers' experiences with PHRs. Excluded abstracts (a) focused  barriers to PHR use associated with each work system (patients,
solely on provider-facing electronic systems, (b) focused on tech- providers, and caregivers)
nology with functionality limited to symptom tracking without
integration into other PHR functions, (c) focused on self-management Because the studies reviewed implemented a range of

Table 1
Search terms.

Keywords for PHR Patient facing, personal health record, personal health records, patient internet portal, patient internet portals, patient portal, patient portals, patient
held record, patient held records, patient accessible electronic medical record, personal medical record, personal medical records, electronic patient
portal, electronic patient portals, electronic portal, personally controlled health record, personally controlled health records, electronic patient record,
electronic patient records, web-based personal health record, web-based personal health records, personal electronic health record, personal electronic
health records, web-based patient portal, web-based patient portals, patient web portal, patient web portals, computerized patient record,
computerized patient records, patient accessible electronic health record, and patient accessible electronic health records
Keywords for barrier*, adopt*, satisfaction, implement*, facilitat*, challenge*, accept*, dissem*, use, usefulness, useful, attitude*, belief*, advantage*, and
barrier disadvantage*
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 221

qualitative, descriptive, and mixed method approaches, statistical 3.2. Study characteristics
significance could not be reported for all barriers. However, for
studies reporting statistical significance, only statistically signifi- Included studies employed a range of approaches for under-
cant barriers were extracted for analysis. standing barriers to PHR use (see Appendix A). Forty-five articles
Karsh and colleagues' HF/E paradigm specifies four work system focused exclusively on patient work systems (McInnes et al., 2013;
levels (see Table 2) (Karsh et al., 2006). The original model refers to Tjora et al., 2005; Ancker et al., 2011; Burke et al., 2010; Day and Gu,
individual factors as patient and provider factors. However, because 2012; Goel et al., 2011; Gu and Day, 2013; Guy et al., 2012; Hess
we extended the model to caregivers, we refer to this work system et al., 2007; Kahn et al., 2010; Kim et al., 2009; Krist et al., 2011;
level as individual factors. Individual factors are characteristics of Lau et al., 2013a, 2013b, 2013c; Lober et al., 2006; Mayberry et al.,
the person (e.g., weight, personality). Work system/unit factors are 2011; Miller et al., 2007; Nagykaldi et al., 2012; Nielsen et al.,
tasks performed (e.g., searching for health information) and the 2012; Osborn et al., 2013; Sarkar et al., 2010, 2011; Schnipper et
tools and technology used to accomplish these tasks (e.g., search al., 2008; Tsai et al., 2012; Tuil et al., 2006; Vodicka et al., 2013;
engines). Organization factors are characteristics of the home and Wade-Vuturo et al., 2013; Wagner et al., 2012, 2010; Wang et al.,
community (patient- and caregiver-focused) or healthcare institu- 2004; Weingart et al., 2006; Wen et al., 2010; Wiljer et al., 2010;
tion (provider focused). External environment factors are charac- Zickmund et al., 2008; Zulman et al., 2011; Emani et al., 2012; Nazi,
teristics of an individual's outside environment (e.g., clinics 2010; Nazi et al., 2013; Wald et al., 2009; Tenforde et al., 2012;
[caregiver- and patient-focused] or external clinics [provider- Denton, 2001; Goldner et al., 2013; Gordon et al., 2012; Lin et al.,
focused]). As noted, organization and external environment fac- 2005), three exclusively on provider work systems (Crotty et al.,
tors depend on the population being assessed. 2013; Wynia et al., 2011; Fuji et al., 2008), two exclusively on
Data were entered into Microsoft Excel (Microsoft Office caregiver work systems (Byczkowski et al., 2014; Britto et al., 2013),
Professional, 2010) and analyzed using qualitative content anal- seven on patient and provider work systems (Nazi, 2013; Urowitz
ysis methods (Graneheim and Lundman, 2004; Sandelowski, 2010; et al., 2012; Wald et al., 2010; Earnest et al., 2004; Jung et al.,
Sandelowski, 2000; Hsieh and Shannon, 2005). Using the HF/E 2011; Do et al., 2011; Poon et al., 2007), two on patient and care-
paradigm as an overarching framework, barriers were coded both giver work systems (Tom et al., 2012; Weitzman et al., 2012), and
inductively and deductively (see Fig. 1). Deductive coding matched one on patient, provider, and caregiver work systems (Woods et al.,
barriers to existing elements of the framework; however, when a 2013). Sample sizes ranged from 10 to 100,617. A majority reported
quote did not fit an existing category, we created a new code (i.e., barriers to using tethered systems (n ¼ 47), were based in the
inductive coding). Simultaneous coding was used when content United States (n ¼ 50), and were classified in the medical sciences
extracted from an article reflected more than one factor and/or journal domain (n ¼ 54). Over half (n ¼ 37) used quantitative de-
subfactor. Two authors (MJT & JDR) independently coded all signs; the remaining studies used mixed methods (n ¼ 15) or
extracted data. Decisions were discussed and discrepancies qualitative designs (n ¼ 8). Data collection included a variety of
resolved through consensus-building. Unresolved differences were instruments, with most employing surveys (n ¼ 41).
referred to the senior author (RSV). Participant demographic characteristics varied across the studies
(see Appendix A). Of 28 reporting patient health conditions, diabetes
3. Results was the most frequent (n ¼ 14). Twenty-nine studies reported
patients' average age, with only two reporting an average age less
3.1. Search results than 40. Thirty-one reported patients' race, most frequently Cauca-
sian (n ¼ 31) or African American (n ¼ 16). Thirteen reported patient
The search, using PubMed, CINAHL, Engineering Village (Com- participants of Hispanic/Latino ethnicity. Five studies focusing on
pendex and INSPEC), IEEE Xplor, and ACM Digital Library, produced provider work systems reported the type of medical practice, with
a total of 4865 records. After title, abstract, and full-text screening, two relating to primary care. Three reported provider participants'
60 articles remained for data extraction and synthesis (see Fig. 2). average age (ranging from 41 to 51 years). Three studies focusing on

Table 2
HF/E paradigm factors and subfactors (Karsh et al., 2006).

Factor Subfactor

Individual factors Skills, knowledge, training, education


Size, weight, reach, strength
Age, gender, ethnicity, language
Needs, biases, beliefs, mood
Work system/unit factors Task demands, complexity, difficulty
Time and sequence demands
Availability of usable technology
Technology functions/features
Noise, temperature, lighting
Physical layout and geography
Organization factors Organizational policy/priorities
Organizational structure
Financial resources
Rewards structure
Management structure
Training provided
Staffing levels
Social norms and pressures
Social climate/culture
External environment Extra-organizational rules, standards, legislation enforcement
Industry social influence
Industry workforce characteristics
222 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Fig. 1. Coding example.

caregiver work systems reported the relationship between the barriers to using PHRs in at least one of the four HF/E paradigm
caregiver and patient, all of which were parents or guardians. Only levels: individual (n ¼ 46), work system/unit (n ¼ 39), organization
one study reported caregivers' race (primarily Caucasian) and only (n ¼ 20), and external environment factors (n ¼ 25). All examples
one reported caregivers' average age (36 years). provided in the Results section are either direct quotes from the
authors of an article or direct quotes from participants in qualitative
3.3. HF/E paradigm studies. Barriers within patient work systems were identified for
four subfactors related to individual factors (see Table 3 and
In total, work system barriers were classified under 19 of the 22 Appendix A). Patients' skills, employment, knowledge, training, and
subfactors described by Karsh and colleagues and were noted at all education, such as awareness of system features and computer lit-
levels of the work system. Barriers to both patient and provider use eracy were recognized as barriers to PHR use. Patients' de-
were categorized in 16 of the 22 subfactors, while barriers to mographics such as age, gender, ethnicity, language, and marital
caregiver use were categorized in eight. In addition to the sub- status were described as barriers. Barriers related to patients' needs,
factors identified from the HF/E paradigm, one new subfactor, biases, beliefs, and mood focused on perceived PHR value, degree of
behavior, was identified for patient, provider, and caregiver work need or existence of alternative technologies, and communication
system barriers. Thus, the total number of subfactors identified for preferences. Patients' behavior was acknowledged as a barrier,
each population was 17, 17, and 9, respectively. Fig. 3 lists the including health service utilization, practice of a healthy lifestyle,
subfactors identified for each population, illustrating subfactors and norms for technology use and communication.
unique to specific populations and those shared by multiple pop- Barriers within patient work systems were identified for five
ulations. None were identified as unique to caregivers. subfactors related to work system/unit factors (see Table 4 and
Appendix A). Barriers related to system task demands, complexity
3.4. Patient work system barriers and difficulty related to issues encountered when using the PHR,
such as unintuitive navigation and password recovery. Barriers to
A majority of articles (n ¼ 55) reported patient work system patient use related to time and sequence demands focused on
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 223

Fig. 2. Flow diagram for search results.

provider and system response times and the PHRs' fit with patients' where the patient lived and whether the patient used the system
everyday routines. Availability of usable technology including while hospitalized or after discharge.
Internet, computer, and smartphone access, was described as a Barriers within patient work systems were identified for five
barrier to patients' use. Technology functions and features was the subfactors related to organization factors (see Table 5 and Appendix
most frequently reported work system/unit subfactor and posed A). Barriers to patient use related to organizational policy/priorities
barriers to patients because of challenging content (e.g., difficult focused on whether members of their home and community (e.g.,
terminology), inadequate privacy and security settings, missing family, friends) would have access to patients' PHRs. Organizational
functionality, and limited technology platform options. Barriers structure, or home and community members' presence at doctors'
pertaining to physical layout and geography focused on the location appointments and in the household, was recognized as a barrier.

Fig. 3. Subfactors identified for each population.


224 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Table 3
Barriers to patient use related to individual factors.

Subfactor Definition Example

Patient: skills, employment, knowledge, Patients' PHR use reflected Compared to those with a college degree, those with a lower educational attainment were more
training, and education (n ¼ 22) individuals' understanding. likely to never have logged on (Sarkar et al., 2011).
Patient: age, gender, ethnicity, language, Patients' PHR use reflected There were marked race/ethnic differences in use, with African American (31%), Latino (34%),
and marital status (n ¼ 20) individuals' demographic groups. and Filipino (32%) participants least likely, and Asian (53%) and White (51%) participants most
likely to both request a password for the internet-based patient portal (a marker for internet
access and intent to use) and log on to the portal after requesting a password was completed
(an early marker for navigability once access is obtained) (Sarkar et al., 2011).
Patient: needs, biases, beliefs, and mood Patients' PHR use reflected There was a significant difference (p < 0.0001) in the number of medications prescribed by
(n ¼ 33) individuals' mindset. multiple sclerosis staff between PatientSite users (M ¼ 4.61, SD ¼ 2.57) and PatientSite non-
users (M ¼ 2.49, SD ¼ 2.10) (Nielsen et al., 2012).
Patient: behavior (n ¼ 24) Patients' PHR use reflected ‘When you use such Net-based systems that have nothing to do with your email account, you
individuals' actions. have to access it separately. And I read so much email for the rest of the day or do so many other
things, that to log on to check if I have had a response todaydI don't bother. Then, it is much
better to use an email account that I use on a daily basis.’ These problems have led many
patients to use ordinary email instead of MedAxess (Tjora et al., 2005).

Available financial resources including household income, insurance related to individual factors. Providers' skills, employment, knowl-
status, and willingness to pay for PHR service, were identified as edge, training, and education, including awareness of PHRs and
barriers. Staffing levels, or the availability of home and community provider specialty, were described as barriers to providers' PHR use.
members to assist patients with using PHRs, was categorized as a Within the subfactor of age, gender, ethnicity, language, and marital
barrier. Barriers related to social climate/culture focused on home status only gender was identified as a barrier to PHR use. Providers'
and community members' support, health status, and education. needs, biases, beliefs, and mood, such as perceived PHR value, degree
Barriers within patient work systems were identified for three to which an alternative technology solution was perceived as
subfactors related to external environment factors (see Table 6 and feasible or appropriate, and beliefs regarding patient self-care, were
Appendix A). Extra-organizational rules, standards, and legislation described as barriers. Behavior, including use of EHRs, degree of
enforcement, such as third party access to PHRs, PHRs' interopera- engagement with PHRs, and prior use of alternative forms of
bility with EHRs, and the ability/inability to contact multiple pro- technology, was also recognized as a barrier.
viders, were recognized as barriers. Patient barriers regarding Barriers within provider work systems were identified for five
industry social influences focused on patienteprovider relationships subfactors related to work system/unit factors (see Table 8 and
and the value of the system to providers. Industry workforce char- Appendix A). Barriers related to system task demands, complexity and
acteristics, such as the degree of involvement from the healthcare difficulty focused on concerns regarding patients' expectations and
team and issues related to provider work (e.g., difficulty responding issues encountered while using the PHR (e.g., cumbersome system
to patient inquiries, inconsistencies in provider notes, and un- interface and navigation). Time and sequence demands, such as time
availability of clinical support), also presented barriers to patients. allotted to patient care, workload, timing of information access, and
time required to use and learn the system, were identified as barriers.
The availability of usable technology, specifically the lack of patient-
3.5. Provider work system barriers accessible computers in waiting rooms, posed barriers to pro-
viders. Technology functions and features including accuracy of and
Ten articles reported provider-related barriers to the use of PHRs patient access to sensitive information, missing functions, privacy,
on at least one of the four HF/E paradigm levels: individual (n ¼ 7), awkward security procedures, and general technical issues, were
work system/unit (n ¼ 9), organization (n ¼ 6), and external envi- described as barriers. Barriers related to physical layout and geogra-
ronment factors (n ¼ 4) (see Table 7 and Appendix A). Barriers phy focused on the clinic setting (e.g., rural or urban settings).
within provider work systems were identified for four subfactors

Table 4
Barriers to patient use related to work system/unit factors.

Subfactor Definition Example

Patient: task demands, Patients' PHR use reflected individuals' ability Patients thought the terminology was sometimes difficult to understand and would require
complexity and difficulty to complete related work. help from the healthcare team. ‘[You] have to really read it closely and if you don't know
(n ¼ 22) much about terminology … it could be real confusing’ (Wagner et al., 2010).
Patient: time and sequence Patients' PHR use reflected the temporal Another participant expressed dissatisfaction after he sent a message to his provider about a
demands (n ¼ 14) length and order of completing related work. medication side effect, and he did not get a response within a reasonable time frame. The
consequences of this were threefold: (1) the participant adjusted his medication without
provider input, (2) the participant now relies on more traditional forms of communication
(e.g., a phone call or an office visit); and (3) the participant has been unsatisfied with his care
(Nazi, 2013).
Patient: availability of usable Patients' PHR use reflected individuals' access Some access concerns were identified through patient comments, such as: ‘Internet
technology (n ¼ 9) to electronic resources. Explorer as the only option for access is very restrictive;’ and ‘Not sure I'd be able to do [the
journal] at home [if I didn't have] a job with access to the internet’ (Wald et al., 2009).
Patient: technology functions Patients' PHR use reflected characteristics of More than one half of the respondents (62%) wanted to grant PHR access to their spouse or
and features (n ¼ 28) the electronic resources. partner [but are unable to do so], and a smaller percentage wanted to grant access to a child
(23%), other family member (15%), unrelated caregiver (7%), or friend or neighbor (2%)
(Zulman et al., 2011).
Patient: physical layout and Patients' PHR use reflected individuals' Utilisation patterns showed that users accessed their patient-accessible electronic health
geography (n ¼ 2) location. records significantly more often while the patients were in hospital (median of 6 logins)
than after discharge (median of 4 logins, p < 0.001) (Burke et al., 2010).
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 225

Table 5
Barriers to patient use related to organization factors.

Subfactor Definitions Example

Patient: organizational Patients' PHR use reflected established practices for For example, respondents were more interested in sharing access to medication lists,
policy/priorities home and community members' PHR use. appointment information, and laboratory and test results with their designee than patient-
(n ¼ 1) entered health information or communications with providers. Respondents were similarly
more interested in delegating prescription refill requests and appointment scheduling than
in having the designee communicate with their health care provider … Although
respondents tended to be most interested in sharing information with family members
(especially a spouse or partner), they expressed high levels of interest in allowing unrelated
caregivers to conduct activities in their PHR, such as requesting prescription refills or
scheduling appointments (Zulman et al., 2011).
Patient: organizational Patients' PHR use reflected home and community A common way participants learned about the medical center's PWP [patient web portal]
structure (n ¼ 2) members' structure and participation. was through a knowledgeable family member: ‘My daughter showed [the PWP] to me in my
doctor's office, on the computer in the waiting room. No one in the doctor's office ever
approached me about it. If it wasn't for my daughter, I wouldn't be a [PWP] user’ (Mayberry
et al., 2011).
Patient: financial Patients' PHR use reflected available monetary MHAV users were more likely than nonusers to be Caucasian/white, have higher incomes,
resources (n ¼ 13) resources within the home and community. and be privately insured (Osborn et al., 2013).
Patient: staffing levels Patients' PHR use reflected the availability of home The system was most frequently used on Thursdays (67%, 5387/8008), followed by Fridays
(n ¼ 2) and community members. (14%, 1098/8008), which coincided with the onsite availability of graduate nursing students.
Most (77%, 6174/8008) of the system use happened while assistance from graduate nursing
students or housing staff was available to the residents. On the other hand, 8% (677/8008) of
user activities occurred during off hours when the students or staff were not available (from
5:00 pm to 8:00 am weekdays and weekends) (Kim et al., 2009).
Patient: social climate/ Patients' PHR use reflected home and community Personal health record users were younger, more likely to have commercial insurance,
culture (n ¼ 5) members' values, experiences, and practices. identify as Caucasian, have higher household incomes, and live in a region with higher rates
of high school completion compared to MyChart non-users (Tenforde et al., 2012).

Table 6
Barriers to patient use related to external environment factors.

Subfactor Definition Example

Patient: extra-organizational rules, Patients' PHR use reflected policies However, some patients were uncertain about a potential misuse of information
standards, legislation enforcement surrounding the healthcare system. transmitted through MedAxess. Also, the fact that communication is logged and stored in a
(n ¼ 15) database made the situation quite different from that of, for instance, telephone
conversations. If such written communication is stored for a very long time, it is difficult to
foresee who will have access to the information in years to come (Tjora et al., 2005).
Patient: industry social influences Patients' PHR use reflected Fear of losing relationships: Some participants valued their choices of how to
(n ¼ 10) individuals' interactions with communicate, and were concerned that the portal might cause them to lose those choices
providers. (Zickmund et al., 2008).
Patient: industry workforce Patients' PHR use reflected aspects of Although communication seemed to occur primarily with a nurse, dietitian, or other AHP
characteristics (n ¼ 11) providers' work. via the portal, patients often wished their physician had taken more of an interest in the
program and had reviewed the information they had entered on the portal during their
clinic visits (this was also largely done by AHPs). Responses revealed that it would have
been beneficial if a health care provider had referred them to information in the Health
Library. The responses typically reflected a widespread notion that physicians were often
busy and may be unable to fulfill this role as much as they would have liked (Urowitz et al.,
2012).

Table 7
Barriers to provider use related to individual factors.

Subfactor Definition Example

Provider: skills, employment, Providers' PHR use reflected Many health care professionals reported general awareness of My HealtheVet but limited
knowledge, training, education individuals' understanding. familiarity with its features, with the exception of secure messaging. Health care professionals
(n ¼ 4) note that this lack of knowledge limits their ability to endorse patient use, or to integrate use of
My HealtheVet features within the clinical practice setting (Nazi, 2013).
Provider: age, gender, ethnicity, Providers' PHR use reflected A greater number of male physicians (32.9 percent of males vs. 20.4 percent of females) perceive
language, marital status (n ¼ 1) individuals' demographic groups. that none of their patients use PHRs (c2 ¼ 13.846, p ¼ 0.000). Yet, more male physicians than
female physicians reported using the patient's PHR information (6 percent vs. 2.8 percent,
c2 ¼ 3.780, p ¼ 0.052), having a member of their staff work with the patients and their PHRs (5.2
percent vs. 1.2 percent, c2 ¼ 7.537, p ¼ 0.006), and being capable of electronically integrating
PHR information into their own EHR (3.5 percent vs. 2.8 percent) (Fuji et al., 2008).
Provider: needs, biases, beliefs, mood Providers' PHR use reflected Providers commonly viewed patients' interactions with the portal positively and their own
(n ¼ 3) individuals' mindset. interaction negatively. Negative comments typically concerned time constraints and technical
barriers. There were instances where providers indicated that they believed the portal may be
more beneficial for patient self-education than for significant provider usage (Urowitz et al.,
2012).
Provider: behavior (n ¼ 5) Providers' PHR use reflected In general, health care professionals reported limited experiences with patient use (and their
individuals' actions. own use) of My HealtheVet health education resources, tools to support medication
reconciliation, and tools to support patient self-reported data (with some exceptions), often
using alternative tools and resources instead (Nazi, 2013).
226 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Table 8
Barriers to provider use related to work system/unit factors.

Subfactor Definition Example

Provider: task demands, Providers' PHR use reflected individuals' ability ‘The system is cumbersome and needs an interface that addresses the needs of patients and
complexity difficulty to complete related work. data entry requirements’ (Urowitz et al., 2012).
(n ¼ 2)
Provider: time and sequence Providers' PHR use reflected the temporal Providers believed that accessing patient information was time consuming and sometimes
demands (n ¼ 5) length and order of completing related work. redundant (e.g., due to manual data entry) (Urowitz et al., 2012).
Provider: availability of Providers' PHR use reflected individuals' access
Several factors have inhibited the My HealtheVet PHR adoption, use, and endorsement of
usable technology (n ¼ 2) to electronic resources. patient use: Lack of alignment with structures (e.g., lack of patient-accessible computers in
the clinic setting) (Urowitz et al., 2012).
Provider: technology Providers' PHR use reflected characteristics of Additional topics suggested [functionality to be included in the PHR] were social history
functions/features (n ¼ 8) the electronic resources. update (79%), information about recent appointments with other providers (59%), reason for
visit (34%), office questionnaires (31%), and additional screening questions (24%) (Wald et al.,
2010).
Provider: physical layout Providers' PHR use reflected individuals' The characteristics of users largely tracked those of physicians using electronic records in
and geography (n ¼ 2) location. general; for example, they were more often urban, in group practices or hospital employed,
and in noneprimary care specialties (Wynia et al., 2011).

Barriers within provider work systems were identified for six 3.6. Caregiver work system barriers
subfactors related to organization factors (see Table 9 and Appendix
A). Organizational policy/priorities, such as sensitive information per- Five articles reported caregiver-related barriers to the use of
missions and lack of alignment with provider workflow, were PHRs on at least one of the four HF/E paradigm levels: individual
recognized as barriers to providers' use. Barriers related to financial (n ¼ 4), work system/unit (n ¼ 4), organization (n ¼ 1), and external
resources focused on the cost-benefit of using the system. Staffing environment factors (n ¼ 1) (see Table 11 and Appendix A). Barriers
levels, or the availability of healthcare institution staff to work with within caregiver work systems were identified for three subfactors
patients on PHRs, was described as a barrier. Providers had concerns related to individual factors. Caregivers' skills, employment, knowl-
regarding the rewards structure and whether or not they would be edge, training, and education, including system awareness and
appropriately reimbursed for time spent reviewing PHRs. Inadequate experience using the Internet, were recognized as barriers to
training provided to healthcare institution staff and patients posed caregivers' use. Caregivers' needs, biases, beliefs, and mood,
barriers to providers. Barriers associated with social norms and pres- including the perceived system value, degree of need, comfort
sures included the healthcare institution staffs' technology use norms. sharing medical information on the Internet, and communication
Barriers within provider work systems were identified for two preferences, posed barriers. Caregivers' behavior included the
subfactors related to external environment factors (see Table 10 and likelihood of receiving information from other sources and was
Appendix A). Extra-organizational rules, standards, and legislation described as a barrier.
enforcement, such as patient access to provider entered data and Barriers within caregiver work systems were identified for four
provider liability for all information within a patient's PHR, posed subfactors related to work system/unit factors (see Table 12 and
barriers to providers' use. Barriers related to industry social in- Appendix A). Task demands, complexity and difficulty, such as the
fluences focused on the amount of time spent with patients and ease with which caregivers could access and use the system, were
patients' expectations and assumptions (e.g., providers actively recognized as barriers. Barriers related to time and sequence de-
monitoring patients' health status on the portal). mands focused on provider and system response times, Internet

Table 9
Barriers to provider use related to organization factors.

Subfactor Definition Example

Provider: organizational Providers' PHR use reflected established practices Several factors have inhibited the My HealtheVet PHR adoption, use, and endorsement of
policy/priorities (n ¼ 2) for healthcare institutions' PHR use. patient use: Lack of alignment with workflow (e.g., lack of integration with the primary
clinical information system), Lack of alignment with processes (e.g., barriers to information
flow) (Nazi, 2013).
Provider: financial Providers' PHR use reflected available monetary Each [participant] questioned whether the merits of the intervention would warrant the
resources (n ¼ 1) resources within the healthcare institution. resources spent on it (Earnest et al., 2004).
Provider: rewards Providers' PHR use reflected healthcare There was particular concern (greater than 70 percent) about both unintentional and
structure (n ¼ 2) institutions' incentives for PHR use. intentional data inaccuracies in these records. There were also concerns about privacy, lack of
reimbursement for time spent reviewing them, and liability for knowing all of the
information in a patient's personal health record (Wynia et al., 2011).
Provider: training Providers' PHR use reflected healthcare Provider responses revealed that neither they nor the majority of their patients were able to
provided (n ¼ 1) institutions' instruction in PHR use. use the portal easily. More training and improved portal usability testing were said to be
needed for the portal to be used more effectively. Issues with specific features such as the
display of health indicators and with reading weight and exercise values were mentioned
less by respondents (Urowitz et al., 2012).
Provider: staffing levels Providers' PHR use reflected the availability of 6.5 percent of users reported that a member of their staff works with patient PHRs. These
(n ¼ 1) healthcare institution staff. numbers were even lower for planners (3.9 percent) and nonplanners (1.2 percent) (Fuji
et al., 2008).
Provider: social norms Providers' PHR use reflected the healthcare Health care professionals often reported using alternative tools and resources. For example,
and pressures (n ¼ 1) institutions' technology use norms. although My HealtheVet provides a significant library of health education resources, health
care professionals already use alternative resources, such as subscription-based software
that is linked from within the primary clinical workflow system or resources retrieved from
the Internet, with little incentive to change. Health care professionals reported that they
increasingly use Internet resources easily found by search engines, and speculated that
patients do as well. As one health care provider said: ‘Why not just Google?’ (Nazi, 2013).
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 227

Table 10
Barriers to provider use related to external environment factors.

Subfactor Definition Example

Provider: extra-organizational rules, Providers' PHR use reflected the policies There was particular concern (greater than 70 percent) about both unintentional
standards, legislation enforcement surrounding the larger (beyond own) and intentional data inaccuracies in these records. There were also concerns about
(n ¼ 2) healthcare system. privacy, lack of reimbursement for time spent reviewing them, and liability for
knowing all of the information in a patient's personal health record (Wynia et al.,
2011).
Provider: industry social influences Providers' PHR use reflected individuals' The messaging system is great yet can be utilized negatively by patients increasing
(n ¼ 2) interactions with patients. workload on mydoctor.ca and decreasing time for other patient interactions in
office. The messaging system has also increased expectations from patients for
immediate response (Urowitz et al., 2012).

speed, and general time constraints. The availability of usable identified in this review. Three subfactors included in that paradigm
technology, including computer and Internet access, posed barriers were not relevant and one additional subfactor was inductively
to caregivers. Technology functions and features were described as derived. Size, weight, reach, and strength (individual factor), noise,
barriers due to the lack of specific functions or features and con- temperature, and lighting (work system/unit factor), and management
cerns such as accessing frightening information. structure (organization factor) were included in the HF/E paradigm
Barriers within caregiver work systems were identified for one but not recognized as barriers in this review. The salience of these
subfactor related to organization factors (see Table 13 and work system factors has also been identified in previous literature.
Appendix A). Financial resources, or the type of insurance caregivers For example, Smith and Sainfort (Smith and Sainfort,1989) described
have for their child, were recognized as barriers to caregivers. factors similar to size, weight, reach and strength, noise, temperature
Barriers within caregiver work systems were identified for one and lighting, and management structure as influential in work system
subfactor related to external environment factors (see Table 14 and models focusing on stress reduction. These factors were similarly
Appendix A). Extra-organizational rules, standards, and legislation specified within the SEIPS 2.0 model (Holden et al., 2013), which
enforcement, or the sharing of data with external parties (e.g., gov- focuses on provider, patient, and collaborative work systems. Addi-
ernment agencies, outside providers), posed barriers to caregivers. tional work is needed to empirically determine if these theoretically
identified work system factors serve as barriers to PHR use. Addi-
4. Discussion tionally, despite the applicability of the HF/E paradigm, one new
subfactor was identified, behavior, which has been previously
4.1. Application of HF/E paradigm to barrier classification described as a germane work system factor in other domains,
including stress reduction (Smith and Sainfort, 1989), computer se-
Overall, the HF/E paradigm was comprehensive of the barriers curity (Carayon, 2006), and self-care (Holden et al., 2015).

Table 11
Barriers to caregiver use related to individual factors.

Subfactor Definition Example

Caregiver: skills, knowledge, Caregivers' PHR use reflected Users (versus nonusers) were more likely to have commercial insurance for their child (97% vs. 73%,
training, education (n ¼ 2) individuals' understanding. P,.001) and have at least a 4-year college degree (74% vs. 50%, P < .001) (Tom et al., 2012).
Caregiver: needs, biases, beliefs, Caregivers' PHR use reflected Only 2 percent of the parents agreed that they sometimes saw information they wish they had not
mood (n ¼ 4) individuals' mindset. seen. On the contrary, 12 percent agreed that they sometimes saw information in the portal that
frightened them, and 11 percent reported that they sometimes see information that they would have
preferred to get directly from their provider (Byczkowski et al., 2014).
Caregiver: behavior (n ¼ 1) Caregivers' PHR use reflected Parents receive disease-related information through other avenues (n ¼ 3 of 15 nonusers)
individuals' actions. (Byczkowski et al., 2014).

Table 12
Barriers to caregiver use related to work system/unit factors.

Subfactor Definition Example

Caregiver: task demands, Caregivers' PHR use reflected individuals' Make it easier to access the web site and log-in (n ¼ 18 of 126 respondents) (Tom et al.,
complexity difficulty ability to complete related work. 2012).
(n ¼ 2)
Caregiver: time and Caregivers' PHR use reflected the temporal Faster information (e.g., quick email responses, updates from clinic visits) (n ¼ 9 of 126
sequence demands (n ¼ 2) length and order of completing related work. respondents) (Tom et al., 2012).
Caregiver: availability of Caregivers' PHR use reflected individuals' ‘I don't have access to the Internet,’ ‘No high-speed Internet,’ ‘I don't have a computer’ (Tom
usable technology (n ¼ 2) access to electronic resources. et al., 2012).
Caregiver: technology Caregivers' PHR use reflected characteristics of Statements illustrated challenges stemming from viewing newly revealed information that
functions/features (n ¼ 3) the electronic resources. had not previously been disclosed to patients. One participant, a wife of a patient, expressed
stress upon seeing an operative report; when asked if reading such notes was harmful, she
denied harm had ensued: ‘I would rather not have known. There was a lot of little things
they wrote, you know, step-by-step what had happened in his operation’ (Woods et al.,
2013).

Table 13
Barriers to caregiver use related to organization factors.

Subfactor Definition Example

Caregiver: financial Caregivers' PHR use reflected available monetary Users (versus nonusers) were more likely to have commercial insurance for their child (97% vs.
resources (n ¼ 1) resources within the home and community. 73%, P < .001) and have at least a 4-year college degree (74% vs. 50%, P < .001) (Tom et al., 2012).
228 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Table 14
Barriers to caregiver use related to external environment factors.

Subfactor Definition Example

Extra-organizational rules, Caregivers' PHR use reflected policies The most common objections to sharing data with an outside provider were relevance to
standards, legislation enforcement surrounding the healthcare system. patient care and the potential for discrimination by insurance companies (Weitzman et al.,
(n ¼ 1) 2012).

Overall, relatively limited attention to higher work system levels group has unique roles and is embedded in a different context.
was demonstrated. A majority of the articles found barriers related Consequently, work system models that explicitly represent the
to individual factors (n ¼ 51) and work system/unit factors (n ¼ 45); roles, contexts, and interactions of all actors jointly involved in the
however, less than half found barriers related to organization factors processes and outcomes of patient care are needed. This recom-
(n ¼ 26) and external environment factors (n ¼ 27). More specif- mendation contrasts with existing work system models which,
ically needs, biases, beliefs, and mood (individual factor) (n ¼ 35) and while theoretically enabling simultaneous representation of mul-
technology functions and features (work system/unit factor) (n ¼ 32) tiple individuals, do not distinguish between nor represent their
were the most frequently recognized subfactors. This may be partly overlapping vs. unique individual, work system/unit, organization,
due to the familiarity and widespread use of models such as the and external environment characteristics (Valdez et al. 2015). The
Technology Acceptance Model (TAM) (Holden and Karsh, 2010; phenomenon of overlapping work system factors existing at
Legris et al., 2003) and its derivatives, which emphasize the per- different levels is clearly illustrated here. For example, the degree of
sonetechnology interaction and focus less on broader contexts. value providers find in PHRs is an individual factor for provider
There is a clear need to apply HF/E models and approaches that work systems (i.e., needs, biases, beliefs, and mood), but an external
attend to higher levels of the work system. Identifying and un- environment factor for patient work systems (i.e., industry social
derstanding barriers at higher levels will require complementing influences). Thus, sophisticated representations of work systems
traditional lab-based and task-oriented HF/E approaches with field- that account for intersections between what may, with less fidelity,
based approaches focused on the context of use. This will likely be represented as multiple, separate work systems, are needed.
require HF/E professionals to include qualitative and mixed
methods approaches (Desnoyers, 2004; Hancock and Scalma, 2004; 4.4. Implications for PHR design
Hignett and Wilson, 2004; Moray, 2000; Carayon et al., 2015). Using
participatory design methods (Holden et al., 2013; Valdez et al., Although we did not conduct a meta-analysis to evaluate the
2015b) that engage designers in patients', caregivers', and pro- strength of the evidence, we did observe several repeatedly re-
viders' naturalistic environments of use and reveal their percep- ported barriers. For example, level of education, degree of health
tions of roles played by environmental factors (e.g., through and computer literacy, and access to technologies were recurring
interviews or focus groups) (Holden et al., 2015; Thompson and themes among barriers to patient and caregiver PHR use. According
Valdez, 2013; Ancker et al., 2015) will be imperative. to an Institute of Medicine (IOM) report (2009) (Institute of
Medicine, 2009), greater disparities exist among underserved
4.2. Application of the HF/E paradigm to caregivers populations, many of which experience these barriers. The National
Library of Medicine (NLM), the National Institute on Aging (NIA),
Although Karsh and colleagues' HF/E paradigm was originally and the Office of the National Coordinator for Health Information
formulated for patients and providers, we found it feasible to apply Technologies (ONC) have prioritized development of consumer
it to caregivers. As with patients and providers, it was possible to health IT aligned with laypersons' health management skills and
map all identified barriers, except individual behaviors, to elements health literacy (Alper et al., 2015; National Institute on Aging,
of the paradigm. However, few studies examined caregiver barriers, 2013e2016; National Institute on Aging, 2010e2013a, 2013b). To
despite growing recognition within the HF/E and medical infor- prevent the growth of health and healthcare disparities caused by
matics communities of the critical role these individuals play (Tjora barriers to use, HF/E professionals must engage directly with un-
et al., 2005; Skeels, 2010; Chappell and Reid, 2002). As a likely derserved and elderly populations, ensuring that the next genera-
consequence, barriers relevant to patients and providers centered tion of consumer health IT is accessible and usable across
on almost twice as many subfactors as those relevant to caregivers. demographics. Innovative solutions may be needed that focus not
As previously mentioned, the review found limited research only on access through individually owned devices and text-based
focusing on organization and external environment factors. This content, but also through community owned devices and voice-
general trend also held for caregivers; only two subfactors related based content (Wells et al., 2015).
to these phenomena were identified (see Fig. 3), despite evidence of Another recurring theme was fear of accessing unwanted or
the organizational and environmental challenges faced by care- frightening information (e.g., test results). To our knowledge, no
givers (Werner et al., 2012; Chen et al., 2015; Streid et al., 2014). As policies exist that protect patients and caregivers from seeing such
encouraged by the Patient Engagement Framework (National information. The absence of such policies may be partly due to a
eHealth Collaborative and Healthcare Information and focus on management and control of health information access by
Management Systems Society Foundation, 2014), future research others including providers, caregivers, and care partners (Sarkar
should include a focus on caregivers, specifically challenges they and Bates, 2014). However, although the intent of objective 2C of
face at the organization and environmental levels. the Federal Health IT Strategic Plan 2015e2020 (The Office of the
National Coordinator for Health Information Technology (ONC),
4.3. Representation of individuals in the work system 2014) is to protect the privacy and security of health information,
the strategies it contains may be relevant to ensuring individual
Sociotechnical systems, including those specific to healthcare, control over information access. For example, strategy 3 of objec-
comprise multiple individuals (Holden et al., 2013; Vincent, 2003). tive 2C states, “Support the development of policies, standards,
This review presents three distinct groups of individuals involved technology, guidance, and solutions to facilitate individuals' ability
in patient care (i.e., patients, providers, caregivers). Although all to manage, control, and authorize the disclosure of specific elec-
may benefit from PHR use, this review demonstrates that each tronic health information.” This strategy could be extended to
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 229

guidelines enabling individuals to manage, control, and authorize design (i.e., determining how to design technology to align with
their own access to specific electronic health information. The contextual factors) (Valdez et al., 2015a; Marquard and Zayas-Caban,
ability to tailor personal access preferences will be necessary given 2012; Moen and Brennan, 2005). Yet, as discussed during the recent
that other patients have expressed the desire for faster release (Do HFES annual meeting, we acknowledge that the scope of our pro-
et al., 2011) and greater accessibility (Krist et al., 2011) of test re- fession should be broadened to determine how to design and
sults. In other words, effort should be placed on allowing in- redesign these contextual environments. Current interventions to
dividuals to determine what information is presented in their PHRs increase provider PHR use within the United States include the
and when it is made available, although a minimum waiting period Centers for Medicare & Medicaid Services' (CMS) EHR Incentive
under certain circumstances may be advisable (Do et al., 2011). Program, which rewards healthcare providers who demonstrate
Moreover, because elements of the medical record may be difficult meaningful use (i.e., improved patient care outcomes through
for laypersons to understand, information access requires tools meeting core objectives of Meaningful Use Stages 1, 2 and 3) with
facilitating understanding and decision-making (Wagner et al., incentive payments (Center for Medicare and Medicaid Services,
2010; Do et al., 2011; Wiljer et al., 2010). 2015). HF/E professionals should collaborate on designing other
Patients, providers, and caregivers all commented on concerns interventions at higher work system levels to address barriers to
regarding secure messaging. Both patients and caregivers worried PHR use for providers, patients, and caregivers. For example, in-
that secure messaging could replace more personal modes of surance companies enact patient wellness programs, which incen-
communication (i.e., calling). Similarly, providers expressed con- tivize patients' participation in health-related activities such as
cerns that secure messaging would lead to increased workload and exercising or attending wellness checkups (Department of Health
lost face-to-face interaction with patients. Despite all three groups' and Human Services, 2013). Similar programs could address pa-
concerns, Meaningful Use Stage 2 (The Office of the National tient and caregiver barriers to PHR use.
Coordinator for Health Information Technology (ONC), 2013a)
measure 17 calls for patients' ability to electronically message their 4.6. Limitations and future research
providers. In design terms, this requires creating a solution to two
conflicting viewpoints. Patients' and caregivers' fear of losing per- The limitations of this review yield four directions for future
sonal modes of communication may be partly due to not under- research. First, included articles were limited to studies conducted
standing how or when to use secure messaging. Thus, providing in six countries, highlighting the need for work that explicitly ad-
tutorials (Urowitz et al., 2012; Do et al., 2011) through the PHR dresses organization and external environment barriers arising in
either via web link or an embedded video may help patients feel other national contexts. Second, the coding strategy used in this
more comfortable using another means to contact providers. review focused on assigning barriers to specific work system factors.
However, as the use of secure messaging increases (Cronin et al., Future research should build on this by studying interactions be-
2015), design solutions must also focus on lessening provider tween barriers within and across works systems. Third, this review
workload. One potential solution may be to implement a triage- is systematic in that specific criteria were used to determine article
based system that would route secure messages to appropriate inclusion; however, the authors did not assess the level of evidence
staff members (Katz et al., 2003). Another option is to limit the for the contribution of each work system factor to PHR use. This
number of characters allowed in messages, thus reducing the latter should be the focus of future inquiry. Lastly, although we
amount of text that must be read (Ye et al., 2010) and guiding the consulted with one health science librarian and one computer sci-
patient to use other forms of communication for in-depth inquiries. ence librarian to assist in our literature search, we acknowledge that
Some barriers encountered in this review are well recognized developing an exhaustive search strategy is challenging. In retro-
within the HF/E and medical informatics communities. One spect, additional terms such as “impediment” and “obstacle” should
frequently reported under work system/unit factors was that data be incorporated into future systematic reviews on this topic.
entry was a time consuming and difficult task with which some
patients needed assistance. This barrier may be addressed by tradi- 5. Conclusion
tional human factors methods (e.g., simplified user interface, ca-
pacity for multiple forms of data). These methods may be This review focused on patient, provider, and caregiver work
supplemented by training, such as through tutorials on data entry system barriers to PHR use and is timely given developments
and retrieval. At a higher level of the work system, barriers related to intended to accelerate PHR use such as the recent proposal of
system boundaries and lack of interoperability were also recurring Meaningful Use Stage 3 guidelines within the United States. The
themes. The need to address these barriers not only for PHRs, but also results highlight the need to address how organization and external
for EHRs, has been widely recognized within the medical informatics environment factors hinder PHR use, as well as how such barriers
community and prioritized as a topic of research and intervention can be overcome. Moreover, simultaneous attention to patients,
(The Office of the National Coordinator for Health Information providers, and caregivers lays the groundwork for developing new
Technology (ONC), 2014; Otte-Trojel et al., 2015; Brennan et al., 2015). work system models that explicitly account for the unique and
overlapping roles and contexts of distinct individuals within a
4.5. Designing interventions: attending to context sociotechnical system. Future design directions should address low
health literacy, access to unwanted health information, secure
Identifying barriers should serve as a foundation for devising messaging impacts on provider workflow, secure messaging appeal
design solutions, a key component of HF/E practice (Dul et al., 2012). to patients and caregivers, ease of data entry, and interoperability. A
This review demonstrates that constructing solutions to improve key next step is to synthesize the level of evidence for each type of
the experience of PHR use requires targeting barriers at all levels of PHR use barrier for each work system to guide intervention design.
the work system. This means not only targeting the individual, the
task performed, and the technology used, but also the larger envi- Acknowledgments
ronment in which these interactions are embedded (Holden et al.,
2013; National Research Council, 2011; Dul et al., 2012). Tradition- We would like to thank Kelly Near and Andrea Denton from the
ally, when attending to contextual factors, HF/E professionals have University of Virginia's Library System for guiding and assisting
focused on implications of these work system levels for technology with our literature search.
230 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Appendix A

Study characteristics of included articles.

Author, year Study name Sample Sample Type of Instrumentation Methodology Study Journal domain
(reference) size PHR location

Ancker Use of an electronic patient portal among Patients 74,368 Tethered EHR patient Quantitative United Medical sciences e Internal
et al., disadvantaged populations data; PHR use States medicine
2011 logs
Britto et al., Parents' perceptions of a patient portal for Caregivers 24 Tethered Interview Qualitative United Medical sciences e Pediatrics
2013 managing their child's chronic illness States
Burke et al., Transforming patient and family access to Patients 252 Tethered EHR patient Quantitative United Medical sciences e
2010 medical information: Utilization patterns of a data; PHR States Cardiovascular diseases
patient-accessible electronic health record system logs
Byczkowski Family perceptions of the usability and value Caregivers 530 Tethered Interview; Mixed United Medical sciences e Computer
et al, of chronic disease web-based patient portals survey methods States applications/Nurses and
2014 nursing
Crotty et al., Preparing residents for future practice: Report Providers 108 Tethered Survey, PHR use Quantitative United Medical sciences
2013 of a curriculum for electronic patient-doctor logs States
communication providers
Day and Gu, Influencing factors for adopting personal Patients 10 Tethered Interview, QualitativeNew Medical sciences e Computer
2012 health record (PHR) observation Zealand applications
Denton, Will patients use electronic personal health Patients 136 Untethered Survey Quantitative United Health facilities and
2001 records? Responses from a real-life experience States administration
Do et al., The military health system's personal health Patients; 250; 10 Untethered Survey, advisory Mixed United Medical sciences e Computer
2011 record pilot with Microsoft HealthVault and providers panels methods States applications
google health
Earnest Use of a patient-accessible electronic medical Patients; 107; 11 Tethered Survey, Mixed United Medical sciences e Computer
et al., record in a practice for congestive heart providers interview, focus methods States applications
2004 failure: Patient and physician experiences group
Emani et al., Patient perceptions of a personal health Patients 760 Tethered Survey Quantitative United Medical sciences e Computer
2012 record: A test of the diffusion of innovation States applicable
model
Fuji et al., Personal health record use by patients as Providers 955 Both Survey Quantitative United Medical sciences e Business
2008 perceived by ambulatory care physicians in States and economics e
Nebraska and South Dakota: A cross-sectional management
study
Goel et al., Disparities in enrollment and use of an Patients 7088 Tethered PHR use logs Quantitative United Medical sciences e Internal
2011 electronic patient portal States medicine
Goldner The intersection of gender and place in online Patients 7674 Not Survey Quantitative United Medical sciences;
et al., health activities Specified States Communications
2013
Gordon Processes and outcomes of developing a Patients 7080 Tethered Focus group, Mixed United Biology e Computer
et al., continuity of care document for use as a survey, PHR use methods States applications; Medical sciences
2012 personal health record by people living with logs e computer applications
HIV/AIDS in New York City
Gu and Day, Propensity of people with long-term Patients 10 Tethered Interview Qualitative New Medical sciences e computer
2013 conditions to use personal health records Zealand applications; Medical sciences
Guy et al., Evaluation of a web-based patient portal for Patients 99 Tethered Survey, focus Mixed Canada n/a
2012 chronic disease management group methods
Hess et al., Exploring challenges and potentials of Patients 39 Tethered Focus group, Mixed United Medical sciences e Computer
2007 personal health records in diabetes self- PHR use logs methods States applications
management: Implementation and initial
assessment
Jung et al., Who are portal users vs. early E-Visit Patients, 10,532 Tethered PHR use logs Quantitative United Medical sciences e Computer
2011 adopters? A preliminary analysis providers States applications
Kahn et al., Personal health records in a public hospital: Patients 136 Tethered PHR use logs, Quantitative United Medical sciences e Computer
2010 experience at the HIV/AIDS clinic at San survey States applications
Francisco General Hospital
Kim et al., Challenges to using an electronic personal Patient 70 Untethered Survey, PHR use Quantitative United Medical sciences e Computer
2009 health record by a low-income elderly logs States applications
population
Krist et al., Designing a patient-centered personal health Patients 7235 Both EHR patient Mixed United Medical sciences e Computer
2011 record to promote preventive care data, focus method States applications
group, survey,
interview
Lau et al., Consumers' online social Network topologies Patients 709 Untethered Survey, PHR use Quantitative Australia Medical sciences e computer
2013a and health behaviors logs applications; Medical sciences
Lau et al., Social and self-reflective use of a web-based Patients 709 Untethered Survey, PHR use Quantitative Australia Medical sciences e Computer
2013b personally controlled health management logs applications
system
Patients 709 Tethered Quantitative Australia
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 231

(continued )

Author, year Study name Sample Sample Type of Instrumentation Methodology Study Journal domain
(reference) size PHR location

Lau et al., Which bundles of features in a web-based Survey, PHR use Medical sciences e Computer
2013c personally controlled health management logs applications
system are associated with consumer help-
seeking behaviors for physical and emotional
well-being?
Lin et al., An internet-based patient-provider Patients 606; 14 Tethered Survey, PHR use Mixed United Medical sciences e Computer
2005 communication system: Randomized logs methods States applications
controlled trial
Lober et al., An internet-based patient-provider Patients 38 Untethered PHR use logs, Quantitative United Medical sciences e Computer
2006 communication system: Randomized observation States applications
controlled trial
Mayberry Bridging the digital divide in diabetes: Family Patients 75 Tethered Focus group, Mixed United Medical sciences e
et al., support and implications for health literacy survey, EHR methods States Endocrinology
2011 data
McInnes Development and evaluation of an internet Patients 14 Tethered Survey; Mixed United Medical sciences; Public
et al., and personal health record training program interview; focus methods States health and safety
2013 for low-income patients with HIV or hepatitis group
C
Miller et al., Determinants of personal health record use Patients 63,295 Tethered PHR use logs Quantitative United Health facilities and
2007 States administration
Nagykaldi Impact of a wellness portal on the delivery of Patients 538 Tethered Survey, PHR use Quantitative United Medical sciences
et al., patient-centered preventive care logs States
2012
Nazi, 2010 Veterans' voices: use of the American Patients 100,617 Tethered Interview, Mixed United Medical sciences e Computer
Customer Satisfaction Index (ACSI) Survey to survey methods States application
identify My HealtheVet personal health record
users' characteristics, needs, and preferences
Nazi, 2013 The personal health record paradox: Patients, 30 Tethered Survey, Mixed United Medical sciences e Computer
Healthcare professionals' perspectives and the providers Interview methods States application
information ecology of personal health record
systems in organizational and clinical settings
Nazi et al., Evaluating patient access to electronic health Patients 668 Tethered Survey Quantitative United Medical sciences; Public
2013 records results from a survey of veterans States health and safety
Nielsen Internet portal use in an academic multiple Patients 240 Tethered EHR patient Quantitative United Medical sciences e Computer
et al., sclerosis center data, PHR use States applications
2012 logs
Osborn Understanding patient portal use: Patients 75 Tethered Focus group, Mixed United Medical sciences e Computer
et al., Implications for medication management survey, EHR methods States applications
2013 patient data
Poon et al., Empowering patients to improve the quality Patients 2779 Tethered Survey Quantitative United Medical sciences e Computer
2007 of their care: design and implementation of a States applications, Medical sciences
shared health maintenance module in a US
integrated healthcare delivery network
Sarkar et al., The literacy divide: Health literacy and the use Patients 14,102 Tethered Survey, PHR use Quantitative United Medical sciences,
2010 of an internet- based patient portal in an logs States Communications
integrated health systemdresults from the
Diabetes Study of Northern California
(DISTANCE)
Sarkar et al., Social disparities in internet patient portal use Patients 14,102 Tethered Survey, PHR use Quantitative United Medical sciences e Computer
2011 in diabetes: evidence that the digital divide logs States applications
extends beyond access
Schnipper Design and implementation of a web-based Patients 5298 Tethered PHR use logs, Quantitative United Medical sciences e Computer
et al., patient portal linked to an electronic health survey States applications
2008 record designed to improve medication safety:
the Patient Gateway medications module
Tenforde The association between personal health Patients 10,746 Tethered EHR patient Quantitative United Medical sciences e Internal
et al., record use and diabetes quality measures data, PHR use States medicine
2012 logs
Tjora et al., Privacy vs. usability: A qualitative exploration Patients 15 Tethered Interview Qualitative Norway Medical sciences e Computer
2005 of patients' experiences with secure internet applications
communication with their general
practitioner
Tom et al., Integrated personal health records use: Patients, 256 Tethered Survey Quantitative United Medical sciences e Pediatrics
2012 association with parent-reported care caregivers States
experiences
Tsai et al., Use of the internet and an online personal Patients 7215 Tethered Survey Quantitative United Medical sciences e Computer
2012 health record system by US veterans: States applications
comparison of Veterans Affairs mental health
service users and other veterans nationally
Tuil et al., Patient-centered care: using online personal Patients 102 Tethered Questionnaire, Mixed The Medical sciences e
2006 medical records in IVF practice PHR use logs, methods Netherlands Endocrinology; Medical
interview sciences e Obstetrics and
(continued on next page)
232 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

(continued )

Author, year Study name Sample Sample Type of Instrumentation Methodology Study Journal domain
(reference) size PHR location

gynecology; Pharmacy and


pharmacology
Urowitz Improving diabetes management with a Patients; 17; 64 Tethered Survey; Qualitative Canada Medical sciences e Computer
et al., patient portal: a qualitative study of diabetes providers interview applications
2012 self-management portal
Vodicka Online access to doctors' notes: patient Patients 3874 Tethered Survey, PHR use Quantitative United Medical sciences e Computer
et al., concerns about privacy logs States applications
2013
Wade- Secure messaging and diabetes management: Patients 54 Tethered Focus group, Mixed United Medical sciences e Computer
Vuturo experiences and perspectives of patient portal survey methods States applications
et al., use
2013
Wagner Personal health records and hypertension Patients 443 Tethered EHR patient Quantitative United Medical sciences e Computer
et al., control: A randomized trial data, survey States applications
2012
Wagner Incorporating patient perspectives into the Patients 16 Untethered Interview Qualitative United Medical sciences; Business
et al., personal health record: implications for care States and Economics e
2010 and caring Management
Wald et al., Implementing practice-linked pre-visit Patients; 3979; Tethered Survey, EHR Quantitative United Medical sciences e Computer
2010 electronic journals in primary care: Patient providers 272 patient data States applications
and physician use and satisfaction
Wald et al., Survey analysis of patient experience using a Patients 126 Tethered Survey, PHR use Quantitative United Medical sciences e Computer
2009 practice-linked PHR for type 2 diabetes logs States applications
mellitus
Wang et al., Personal health information management Patients 61 Untethered PHR use logs Quantitative United Biology e Biotechnology;
2004 system and its application in referral States Medical sciences e Computer
management applications
Weingart Who uses the patient internet portal? The Patients 980 Tethered PHR use logs Quantitative United Medical sciences e Computer
et al., PatientSite experience States applications
2006
Weitzman Willingness to share personal health record Patients, 261 Untethered Survey Quantitative United Medical sciences e Computer
et al., data for care improvement and public health: caregivers States applications
2012 a survey of experienced personal health
record users
Wen et al., Consumers' perceptions about and use of the Patients 7674 Not Survey Quantitative United Medical sciences e Computer
2010 internet for personal health records and health Specified States applications
information exchange: Analysis of the 2007
Health Information National Trends Survey
Wiljer et al., The anxious wait: assessing the impact of Patients 311 Tethered Survey Quantitative Canada Medical sciences e Computer
2010 patient accessible EHRs for breast cancer applications
patients
Woods Patient experiences with full electronic access Patients, 30; 6 Tethered Focus group Qualitative United Medical sciences e Computer
et al., to health records and clinical notes through providers, States applications
2013 the My HealtheVet personal health record caregivers
pilot: Qualitative study
Wynia et al., Many physicians are willing to use patients' Providers 856 Not Survey Quantitative United Insurance; Public health and
2011 electronic personal health records, but doctors Specified States safety
differ by location, gender, and practice
Zickmund Interest in the use of computerized patient Patients 39 Tethered Focus Group Qualitative United Medical sciences e Internal
et al., portals: Role of the providerepatient States medicine
2008 relationship
Zulman Patient interest in sharing personal health Patients 18,471 Tethered Survey Quantitative United Medical sciences e Internal
et al., record information, a web-based survey States medicine
2011

Participant demographics for included studies focusing on patient work systems.

Citation Health condition Race Age

Ancker Hypertension e 16%, Hyperlipidemia 13%, Asthma e 16% Black, 19% Hispanic, 44% White, 6% other, 15% M ¼ 40
et al., 9%, Diabetes e 7%, Depression e 6%, Drug Abuse/ missing/unknown
2011 Dependence e 2.3%, Chronic hepatitis (B, C, or other) -
1.8%, Alcoholism e 1.5%, HIV/AIDS e 1.3%,
Burke Congenial cardiac disease 37.5% Hispanic, 35.2% White, 10.1% Black, 0.7% Asian, n/a
et al., 16.5% Other
2010
Day and One or more long term health condition n/a Range: 35e79
Gu,
2012
Spinal disorders n/a Range: 35e85
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 233

(continued )

Citation Health condition Race Age

Denton,
2001
Do et al., n/a n/a M ¼ 53.14
2011
Earnest Congestive heart failure 90% White, non-Hispanic M ¼ 56
et al.,
2004
Emani Asthma, congestive heart failure (CHF), hypertension, Innovators: 94% Caucasian, Other users: 90% Innovators: M ¼ 55.4, Other users: M ¼ 59.8,
et al., or diabetes Caucasian, Laggards: 94% Caucasian, Rejecters: 86% Laggards: M ¼ 59.0, Rejecters: M ¼ 58.3, Non-
2012 Caucasian, Non-adopters: 76% Caucasian adopters: M ¼ 61.7
Goel et al., n/a 49% White, 15% Black. 4% Latino, 2% Asian, 12% other, Median age ¼ 48
2011 19% missing
Goldner n/a Health seekers: 25.2% Non-White, 74.8% White; Health seekers: 29.6% 18e34, 30.9% 35e49,
et al., Internet users: 24.7% Non-White, 75.3% White 25.7% 50e64, 7.9% 65e74, 5.9% 75þ
2013 Internet users: 36.5% 18e34, 32.2% 35e49,
23.6% 50e64, 5.4% 65e74, 2.3% 75þ
Gordon HIV 38.1% African American/Black, 0.3% Asian, <0.1% Unique members: M ¼ 45.6
et al., Native Hawaiian/Pacific Islander, 0.3% Native
2012 American or Alaskan, 38.3% White/Caucasian, 23.0%
Other/not specified, 38.0% Hispanic/Latino, 39.0%
Non-Hispanic/Latino
Gu and One or more long term conditions including: prostate n/a Range: 35e79
Day, cancer, type 2 diabetes, asthma, heart condition, M ¼ 63.5
2013 essential tremor, benign prostate hypertrophy, high
cholesterol, alcoholism, vertigo, glaucoma, osteo-
arthritis, obesity, chornic abdominal pain
(endometriosis), and depression. They also reported
surgery, including bilateral hip replacement, ‘broken
knee cap’, sinus surgery, removal of gall bladder,
ovarian cystectomy and hysterectomy, thyroidectomy
and ‘heart burn surgery’.
Guy et al., Type 2 diabetes and prostate cancer n/a n/a
2012
Hess et al., Diabetes 28% Non-White M ¼ 54
2007
Jung et al., n/a eVisit users: 76.5% White eVisit: M ¼ 46.88, Range 21e78; Portal Only:
2011 Portal only users: 76.4% White M ¼ 51.73, Range 19e97
Kahn et al., HIV/AIDS 78% Caucasian, 15% Hispanic 37% < 40 years old
2010
Kim et al., n/a n/a PHIIMS users: M ¼ 63.1; Range:21e30 (2.9%),
2009 31e40 (2.9%), 41e50 (11.4%), 51e60 (20.0%), 61
e70 (38.6%), 71e80 (7.1%), 81e90 (10.0%), 91
e100 (7.1%)
Survey respondents: M ¼ 65.5
Krist et al., n/a n/a 56% 18e49, 36% 50e64, 8% 65e75
2011
Lau et al., n/a n/a n/a
2013a
Lau et al., n/a n/a n/a
2013b
Lau et al., n/a n/a M ¼ 25.2
2013c
Lin et al., n/a n/a Portal group M ¼ 52
2005 Control group M ¼ 50
Lober Many had chronic diseases n/a Range: 49e92
et al., M ¼ 69
2006
Mayberry Type 2 diabetes 65% White, 35% Non-white M ¼ 56.8
et al.,
2011
McInnes HIV (n ¼ 8) or hepatitis C (HCV) (n ¼ 6) 14.3% Hispanic, 57.1% White, 21.4% Black, 7.1% Other 14.3% 35e49, 71.4% 50e64, 14.3% 65e74
et al.,
2013
Miller n/a Users: 88% Caucasian, 6% African American, 1% Asian/ Users M ¼ 50.1
et al., Pacific Islander, 1% Hispanic/Latino, 2% Other Never users M ¼ 50.7
2007 Never users: 82% Caucasian, 11% African American, 2%
Asian Pacific Islander, 2% Hispanic/Latino, 2% Other
Nagykaldi n/a 82% White non-Hispanic, 8% Black non-Hispanic, 6% Range: 40e75
et al., Hispanic, 4% American Indian/Alaskan Native Control M ¼ 50.5
2012 Intervention M ¼ 54.6
Nazi et al., n/a n/a 9, 1% < 40, 7% 41e50, 26% 51e60, 45% 61e70,
2013 15% 71e80, 6% 81e90
Nazi, 2010 n/a n/a 1% 19e30, 4% 31e40, 11% 41e50, 34% 51e60,
34% 61e70, 16% 71þ
(continued on next page)
234 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

(continued )

Citation Health condition Race Age

Nielsen Clinically isolated syndrome (n ¼ 12), Relapsing- Users: 4% Non-White Users: 83.3% < 55, 16.7%>¼55
et al., remitting multiple sclerosis (n ¼ 118), Secondary Nonusers: 17.5% Non-White Nonusers: 61.7% < 55, 38.3%>¼55 M¼45.4
2012 progressive multiple sclerosis (n ¼ 17), Primary
progressive multiple sclerosis (n ¼ 7)
Osborn Type 2 diabetes 62.7% White, 33.3% African American/black M ¼ 56.9
et al.,
2013
Poon et al., n/a n/a M ¼ 47.4
2007
Sarkar Diabetes 21% AfricaneAmerican, 28% Non-Hispanic White, 14% 4% 30e39, 15% 40e49, 33% 50e59, 32% 60e69,
et al., Latino, 9% Asian American, 12% Filipino, 17% Other/ 17% 70þ
2010 mixed
Sarkar Diabetes 21% AfricaneAmerican, 28% Non-Hispanic White, 14% 4% 30e39, 15% 40e49, 33% 50e59, 32% 60e69,
et al., Latino, 9% Asian American, 12% Filipino, 17% Other/ 17% 70þ
2011 mixed
Schnipper n/a White 83% M ¼ 48
et al.,
2008
Tenforde Diabetes Mellitus Users: 84.0% Caucasian, 11.0% African American, 3.0% Users: M ¼ 59
et al., Other, 2% unknown, <1% Hispanic Non-Users: M ¼ 62
2012 Nonusers: 67.3% Caucasian, 28.6% African American,
2.0% Other, 2.0% Unknown, <1% Hispanic
Tjora et al., n/a n/a n/a
2005
Tom et al., Asthma 24%, congenital musculoskeletal White 70%, Black 5%, Asian/Pacific Islander 13%, Other n/a
2012 abnormalities 20%, congenital heart disease 18%, 13%; 5% Hispanic, 94% Non-Hispanic
inborn errors of metabolism 9%, cystic fibrosis 7%, and
hereditary and acquired helytic anemias 5%
Tsai et al., n/a Not enrolled: 87.30% White, 8.90% Black, 2.20% Not enrolled: M ¼ 62.23
2012 American Indian/Alaskan Native, 1.60% Asian/Pacific Enrolled but does not use mental health
Islander services: M ¼ 61.75
Enrolled but does not use mental health services: Enrolled and does use mental health services:
81.70% White, 14.56% Black, 2.93% American Indian/ M ¼ 53.45
Alaskan Native, 0.81% Asian/Pacific Islander
Enrolled and using mental health services: 74.60%
White, 18.97% Black, 4.50% American Indian/Alaskan
Native, 1.93% Asian/Pacific Islander
Tuil et al., n/a n/a M ¼ 34.44
2006
Urowitz Type 1 and 2 diabetes n/a n/a
et al.,
2012
Vodicka n/a 92.8% White, 52.6% >¼ 55 years old
et al., 7.2% Non-White 47.4% < 55 years old
2013
Wade- Type 2 diabetes 75.9% Caucasian/White M ¼ 57.1
Vuturo 20.4% African American/Black
et al.,
2013
Wagner Hypertension 50.2% White M ¼ 54.8
et al., 46.1% Black
2012 3.7% Other
Wagner Hypertension n/a n/a
et al.,
2010
Wald et al., n/a 87.1% White M ¼ 48.9
2010
Wald et al., Type 2 diabetes 93% White M ¼ 59.4
2009
Wang n/a n/a Range: 22e75
et al., M ¼ 45.7
2004
Weingart n/a Enrollees: 80% White Enrollees (20e81), Nonuser (21e92)
et al., Nonusers: 54% White Enrollees (M ¼ 42.9), Nonuser (M ¼ 52.9)
2006
Weitzman n/a 91.3% White, non-Hispanic n/a
et al.,
2012
Wen et al., Internet users (cancer survivor ¼ 5.8%) Internet Internet users: 74.5% Non-Hispanic White, 9.4% Internet users: 16.2% 18e24, 19.7% 25e34,
2010 nonusers (cancer survivor ¼ 9.4%) Hispanic, 9.5% Black/African American, 6.7% Other 22.0% 35e44, 19.9% 45e54, 13.7% 55e64%, 8.4%
Internet nonusers: 57.3% Non-Hispanic White, 21.1% 65þ
Hispanic, 15.8% Black/African American7.1%, Other Internet nonusers: 6.1% 18e24, 13.1% 25e34,
13.9% 35e44, 17.6% 45e54, 14.5% 55e64, 34.9%
65þ
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 235

(continued )

Citation Health condition Race Age

Wiljer Breast cancer n/a 75.9% < 60 years


et al., 24.1% > 60 years
2010
Woods n/a n/a Range: 49e82
et al.,
2013
Zickmund Diabetes 28% Non-White M ¼ 54
et al.,
2008
Zulman n/a n/a <50 y (n ¼ 1,836, 10.1%), 50e64 (n ¼ 9,206,
et al., 50.6%), 65 y (n ¼ 7,146, 39.3%)
2011

Note: we list ethnic/racial groups, ages, and health conditions of participants for studies focusing on patient work systems at the level of detail reported in the original
manuscripts.

Participant demographics for included studies focusing on provider work systems.

Citation Medical practice Age

Crotty et al., 2013 n/a n/a


Do et al., 2011 n/a n/a
Earnest et al., 2004 University of Colorado Hospital - cardiology n/a
Fuji et al., 2008 n/a M ¼ 50
Jung et al., 2011 n/a n/a
Nazi, 2013 Primary care: n ¼ 24 n/a
Specialty care: n ¼ 4
Both/other: n ¼ 2
Urowitz et al., 2012 Providers included a mix of general practitioners, nurses, nurse practitioners, dietitians, diabetes educators, and other clinical staff n/a
Wald et al., 2010 n/a M ¼ 41
Wynia et al., 2011 Primary care: 47.3%, Pediatrics: 8.4%, Obstetrics/gynecology: 6.0%, Psychiatry: 7.0%, Surgery: 11.7%, Other: 19.7% M ¼ 51.2

Participant demographics for included studies focusing on caregiver work systems.

Citation Relationship to Race Age


patient

Britto et al., 2013 Parent n/a n/a


Byczkowski et al., Parent White e 91% responders, 89% nonresponders AfricaneAmerican e 1% responders, 1% nonresponders Other e 9% n/a
2014 responders, 10% nonresponders
Tom et al., 2012 Parent/guardian n/a M ¼ 36
Weitzman et al., n/a n/a n/a
2012
Woods et al., 2013 n/a n/a n/a

Summary of articles reporting patient barriers related to individual factors.

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Ancker et al., 2011 X X X


Burke et al., 2010 X
Denton, 2001 X
Earnest et al., 2004 X X X
Emani et al., 2012 X X X X
Goel et al., 2011 X
Goldner et al., 2013 X X X
Gordon et al., 2012 X X
Gu and Day, 2013 X
(continued on next page)
236 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

(continued )

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Hess et al., 2007 X


Jung et al., 2011 X X X X
Kahn et al., 2010 X
Kim et al., 2009 X
Krist et al., 2011 X
Lau et al., 2013a X X
Lau et al., 2013b X X
Lau et al., 2013c X
Lin et al., 2005 X X
Lober et al., 2006 X X X
Mayberry et al., 2011 X
Miller et al., 2007 X X X X
Nagykaldi et al., X X X
2012
Nazi, 2010 X
Nazi, 2013 X X
Nielsen et al., 2012 X X X X
Osborn et al., 2013 X X X
Poon et al., 2007 X
Sarkar et al., 2010 X X X
Sarkar et al., 2011 X X X
Schnipper et al., X X
2008
Tenforde et al., 2012 X X X X
Tjora et al., 2005 X X
Tom et al., 2012 X
Tsai et al., 2012 X X
Tuil et al., 2006 X
Urowitz et al., 2012 X X
Vodicka et al., 2013 X X
Wade-Vuturo et al., X X
2013
Wagner et al., 2012 X X X
Wagner et al., 2010 X X
Wald et al., 2009 X
Weingart et al., 2006 X X X
Weitzman et al., X X X
2012
Wen et al., 2010 X X X X
Woods et al., 2013 X
Zickmund et al., X X
2008
Total 22 0 20 33 24

Summary of articles reporting patient barriers related to work system/unit factors.

Reference Task demands, complexity, Time and sequence Availability of usable Technology functions Noise, temperature, and Physical layout and
difficulty demands technology and features lighting geography

Ancker et al., 2011 X


Burke et al., 2010 X
Day and Gu, 2012 X X
Denton, 2001 X X
Do et al., 2011 X X
Earnest et al., 2004 X X
Emani et al., 2012 X X X
Goldner et al., X
2013
Gordon et al., 2012 X
Gu and Day, 2013 X X
Guy et al., 2012 X X X
Hess et al., 2007 X X X
Kahn et al., 2010 X X
Kim et al., 2009 X
Krist et al., 2011 X
Lober et al., 2006 X X
Mayberry et al., X
2011
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 237

(continued )

Reference Task demands, complexity, Time and sequence Availability of usable Technology functions Noise, temperature, and Physical layout and
difficulty demands technology and features lighting geography

McInnes et al., X X X
2013
Nazi, 2010 X X X X
Nazi, 2013
Nazi et al., 2013 X
Osborn et al., 2013 X
Poon et al., 2007 X X
Sarkar et al., 2010 X
Schnipper et al., X
2008
Tjora et al., 2005 X X X
Tuil et al., 2006 X
Urowitz et al., X X X X
2012
Vodicka et al., X
2013
Wade-Vuturo X
et al., 2013
Wagner et al., X X X
2010
Wald et al., 2010 X X
Wald et al., 2009 X X X X
Wang et al., 2004 X X X
Wen et al., 2010 X
Wiljer et al., 2010 X X
Woods et al., 2013 X X X
Zickmund et al., X X X
2008
Zulman et al., X
2011
Total 22 14 9 28 0 2

Summary of articles reporting patient barriers related to organization factors.

Reference Organizational policy/ Organizational Financial Rewards Management Training Staffing Social norms and Social climate/
priorities structure resources structure structure provided levels pressures culture

Ancker et al., 2011 X


Earnest et al., 2004 X
Emani et al., 2012 X
Goldner et al., 2013 X
Guy et al., 2012 X X
Jung et al., 2011 X
Kim et al., 2009 X
Lau et al., 2013a X
Lau et al., 2013b X
Lober et al., 2006 X
Lin et al., 2005 X
Mayberry et al., X X
2011
Miller et al., 2007 X
Nielsen et al., 2012 X
Osborn et al., 2013 X
Schnipper et al., X
2008
Tenforde et al., X X
2012
Tsai et al., 2012 X
Weingart et al., X
2006
Zulman et al., 2011 X
Total 1 2 13 0 0 0 2 0 5
238 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Summary of articles reporting patient barriers related to external environment factors.

Reference Extra-organizational rules, standards, legislation enforcement Industry social influences Industry workforce characteristics

Do et al., 2011 X
Guy et al., 2012 X X
Hess et al., 2007 X X
Kahn et al., 2010 X
Kim et al., 2009 X
Lober et al., 2006 X
Mayberry et al., 2011 X
Nazi, 2010 X
Nazi, 2013 X X X
Osborn et al., 2013 X
Poon et al., 2007 X
Tjora et al., 2005 X X X
Tuil et al., 2006 X
Urowitz et al., 2012 X
Wade-Vuturo et al., 2013 X X
Wagner et al., 2012 X X
Wagner et al., 2010 X
Wald et al., 2010 X X
Wald et al., 2009 X X
Wang et al., 2004 X
Weitzman et al., 2012 X
Wen et al., 2010 X
Woods et al., 2013 X X
Zickmund et al., 2008 X
Zulman et al., 2011 X
Total 15 10 11

Summary of articles reporting provider barriers related to individual factors.

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Crotty et al., 2013 X X


Earnest et al., 2004 X
Fuji et al., 2008 X X X
Jung et al., 2011 X
Nazi, 2013 X X X
Urowitz et al., 2012 X
Wynia et al., 2011 X X
Total 4 0 1 3 5

Summary of articles reporting provider barriers related to work system/unit factors.

Reference Task demands, complexity, Time and sequence Availability of usable Technology functions Noise, temperature, and Physical layout and
difficulty demands technology and features lighting geography

Crotty et al., 2013 X


Do et al., 2011 X X
Earnest et al., X X
2004
Fuji et al., 2008 X
Nazi, 2013 X X X X
Urowitz et al., X X X X
2012
Wald et al., 2010 X
Woods et al., 2013 X
Wynia et al., 2011 X X X
Total 2 5 2 8 0 2
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 239

Summary of articles reporting provider barriers related to organization factors.

Reference Organizational policy/ Organizational Financial Rewards Management Training Staffing Social norms and Social climate/
priorities structure resources structure structure provided levels pressures culture

Do et al., 2011 X
Earnest et al., X X
2004
Fuji et al., 2008 X
Nazi, 2013 X X
Urowitz et al., X
2012
Wynia et al., 2011 X
Total 2 0 1 2 0 1 1 1 0

Summary of articles reporting provider barriers related to external environment factors.

Reference Extra-organizational rules, standards, legislation enforcement Industry social influences Industry workforce characteristics

Crotty et al., 2013 X


Nazi, 2013 X
Urowitz et al., 2012 X
Wynia et al., 2011 X
Total 2 2 0

Summary of articles reporting caregiver barriers related to individual factors.

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Britto et al., 2013 X


Byczkowski et al., X X
2014
Tom et al., 2012 X X
Weitzman et al., 2012 X X
Total 2 0 0 4 1

Summary of articles reporting caregiver barriers related to work system/unit factors.

Reference Task demands, Time and sequence Availability of usable Technology functions Noise, temperature, Physical layout and
complexity, difficulty demands technology and features and lighting geography

Britto et al., 2013 X


Byczkowski et al., X X X X
2014
Tom et al., 2012 X X X
Woods et al., 2013 X
Total 2 2 2 3 0 0

Summary of articles reporting caregiver barriers related to organizational factors.

Reference Organizational policy/ Organizational Financial Rewards Management Training Staffing Social norms and Social climate/
priorities structure resources structure structure provided levels pressures culture

Tom et al., X
2012
Total 0 0 1 0 0 0 0 0 0

Summary of articles reporting caregiver barriers related to external environment factors.

Reference Extra-organizational rules, standards, legislation enforcement Industry social influences Industry workforce characteristics

Weitzman et al., 2012 X


Total 1 0 0
240 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

References for human factors/ergonomics: developing the discipline and profession. Er-
gonomics 55 (4), 377e395. http://dx.doi.org/10.1080/00140139.2012.661087.
Earnest, M.A., Ross, S.E., Wittevrongel, L., Moore, L.A., Lin, C.T., 2004 SepeOct. Use of
Alper, J., Rapporteur; Roundtable on Health Literacy; Board on Population Health
a patient-accessible electronic medical record in a practice for congestive heart
and Public Health Practice; Institute of Medicine; The National Academies of
failure: patient and physician experiences. J. Am. Med. Inf. Assoc. 11 (5),
Sciences, Engineering, and Medicine, 2015 Aug. Health Literacy: Past, Present,
410e417. PMID: 15187074.
and Future: Workshop Summary. The National Academies Press, Washington
Emani, S., Yamin, C.K., Peters, E., Karson, A.S., Lipsitz, S.R., Wald, J.S., et al., 2012 Nov
DC. PMID: 26290936.
5. Patient perceptions of a personal health record: a test of the diffusion of
Amante, D.J., Hogan, T.P., Pagoto, S.L., English, T.M., 2014 Nov. A systematic review of
innovation model. J. Med. Internet Res. 14 (6), e150. http://dx.doi.org/10.2196/
electronic portal usage among patients with diabetes. Diabetes Technol. Ther.
jmir.2278.
16 (11), 784e793. http://dx.doi.org/10.1089/dia.2014.0078.
n, Y., Rockoff, M.L., Hauser, D., Pichardo, M., Szerencsy, A., et al., Friction, J.R., Davies, D., 2008. Personal health records to improve health informa-
Ancker, J.S., Barro
tion exchange and patient safety (Internet). In: Kenriksen, K., Battles, J.,
2011 Oct. Use of an electronic patient portal among disadvantaged populations.
Keyes, M., Grady, M. (Eds.), Advances in Patient Safety: New Directions and
J. Gen. Intern. Med. 26 (10), 1117e1123. http://dx.doi.org/10.1007/s11606-011-
Alternative Approaches, vol. 4. Technology and Medication Safety, Rockville,
1749-y.
MD [cited 2015 Oct 16]. Available from: http://www.ahrq.gov/sites/default/files/
Ancker, J.S., Witteman, H.O., Hafeez, B., Provencher, T., de Graaf, M., Wei, E., 2015 Jun
wysiwyg/professionals/quality-patient-safety/patient-safety-resources/
4. The invisible work of personal health information management among
resources/advances-in-patient-safety-2/vol4/Advances-Fricton_21.pdf.
people with multiple chronic conditions: qualitative interview study among
Fuji, K.T., Galt, K.A., Serocca, A.B., 2008. Personal health record use by patients as
patients and providers. J. Med. Internet Res. 17 (6), 137e149. http://dx.doi.org/
perceived by ambulatory care physicians in Nebraska and South Dakota: a
10.2196/jmir.4381.
cross-sectional study. Perspect. Health Inf. Manag. 5, 15. PMID: 18927602.
Archer, N., Fevrier-Thomas, U., Lokker, C., McKibbon, K.A., Straus, S.E., 2011 JuleAug.
Goel, M.S., Brown, T.L., Williams, A., Hasnain-Wynia, R., Thompson, J.A., Baker, D.W.,
Personal health records: a scoping review. J. Am. Med. Inf. Assoc. 18 (4),
2011 Oct. Disparities in enrollment and use of an electronic patient portal. J.
515e522. http://dx.doi.org/10.1136/amiajnl-2011-000105.
Gen. Intern. Med. 26 (10), 1112e1116. http://dx.doi.org/10.1007/s11606-011-
Brennan, P., Valdez, R., Alexander, G., Arora, S., Bernstam, E., Edmunds, M., et al.,
1728-3.
2015 Apr. Patient-centered care, collaboration, communication, and coordina-
Goldner, M., Hale, T.M., Cotten, S.R., Stern, M.J., Drentea, P., 2013. The intersection of
tion: a report from AMIA's 2013 policy meeting. J. Am. Med. Inf. Assoc. 22 (e1),
gender and place in online health activities. J. Health Commun. 18 (10),
e2e6. http://dx.doi.org/10.1136/amiajnl-2014-003176.
1235e1255. http://dx.doi.org/10.1080/10810730.2013.778364.
Britto, M.T., Hesse, E.A., Kamdar, O.J., Munafo, J.K., 2013 Jul. Parents' perceptions of a
Gordon, P., Camhi, E., Hesse, R., Odlum, M., Schnall, R., Rodriquez, M., et al., 2012.
patient portal for managing their child's chronic illness. J. Pediatr. 163 (1),
Processes and outcomes of developing a continuity of care document for use as a
280e281. http://dx.doi.org/10.1016/j.jpeds.2013.02.041 e1e2.
personal health record by people living with HIV/AIDS in New York City. Int. J.
Burke, R.P., Rossi, A.F., Wilner, B.R., Hannan, R.L., Zabinsky, J.A., White, J.A., 2010 Oct.
Med. Inform. 81 (10), e63ee73. http://dx.doi.org/10.1016/j.ijmedinf.2012.06.004.
Transforming patient and family access to medical information: utilisation
Graneheim, U.H., Lundman, B., 2004 Feb. Qualitative content analysis in nursing
patterns of a patient-accessible electronic health record. Cardiol. Young 20 (5),
research: concepts, procedures, and measures to achieve trustworthiness.
477e484. http://dx.doi.org/10.1017/S1047951110000363.
Nurse Educ. Today 24 (2), 105e112. PMID: 14769454.
Byczkowski, T.L., Munafo, J.K., Britto, M.T., 2014 Jun. Family perceptions of the us-
Greenspun, H., Coughlin, S., 2012. mHealth in An MWorld: How Mobile Technology
ability and value of chronic disease web-based patient portals. Heal Inf. J. 20 (2),
Is Transforming Health Care (Internet). Deloitte Center for Health Solutions,
151e162. http://dx.doi.org/10.1177/1460458213489054.
Washington, DC [cited 2015 Oct 16]. Available from: http://www2.deloitte.com/
Carayon, P., 2006 Jul. Human factors of complex sociotechnical systems. Appl.
content/dam/Deloitte/us/Documents/life-sciences-health-care/us-lhsc-
Ergon. 37 (4), 525e535. PMID: 16756937.
mhealth-in-an-mworld-103014.pdf.
Carayon, P., Schoofs Hundt, A., Karsh, B.T., Gurses, A.P., Alvarado, C.J., Smith, M.,
Gu, Y., Day, K., 2013. Propensity of people with long-term conditions to use personal
et al., 2006 Dec. Work system design for patient safety: the SEIPS model. Qual.
health records. Stud. Health Technol. Inf. 188, 46e51. PMID: 23823287.
Saf. Health Care 15 (Suppl. 1), i50e8. PMID: 17142610.
Guy, S., Ratzki-leewing, A., Gwadry-Sridhar, F., 2012. Evaluation of a web-based
Carayon, P., Kianfar, S., Li, Y., Xie, A., Alyousef, B., Wooldridge, A., 2015 Nov.
patient portal for chronic disease management. Elect. Healthc. 91, 114e121.
A systematic review of mixed methods research on human factors and ergo-
http://dx.doi.org/10.1007/978-3-642-29262-0_17. Malaga, Spain: Springer Ber-
nomics in health care. Appl. Ergon. Elsevier Ltd. 51, 291e321. http://dx.doi.org/
lin Heidelberg.
10.1016/j.apergo.2015.06.001.
Hancock, P.A., Scalma, J.L., 2004 NoveDec. On the relevance of qualitative methods
Center for Advancing Health, 2010. A New Definition of Patient Engagement: What
for ergonomics. Theor. Issues Ergon. Sci. 5 (6), 499e506. http://dx.doi.org/
Is Engagement and Why Is it Important?, 2000 Florida Avenue, NW. Suite 210.
10.1080/14639220412331303391.
Washington, DC [cited 2015 Oct 16]. Available from: http://www.cfah.org/pdfs/
Hess, R., Bryce, C.L., Paone, S., Fischer, G., McTigue, K.M., Olshansky, E., et al., 2007
CFAH_Engagement_Behavior_Framework_current.pdf.
Oct. Exploring challenges and potentials of personal health records in diabetes
Center for Medicare and Medicaid Services, 2015 Mar 30. Electronic Health Record
self-management: implementation and initial assessment. Telemed. J. E Heal 13
Incentive Program Stage 3 (Internet) [cited 2015 Oct 16]. Available from:
(5), 509e517. PMID: 17999613.
https://federalregister.gov/a/2015-06685.
Hignett, S., Wilson, J.R., 2004. The role for qualitative methodology in ergonomics: a
Chappell, N.L., Reid, R.C., 2002 Dec. Burden and well-being among caregivers:
case study to explore theoretical issues. Theor. Issues Ergon. Sci. 5 (6), 473e493.
examining the distinction. Gerontologist 42 (6), 772e780. PMID: 12451158.
http://dx.doi.org/10.1080/14639220412331303382.
Chen, S.H., Huang, J.L., Yeh, K.W., Tsai, Y.F., 2015 Apr 14. The stress of caring for
Holden, R.J., Karsh, B., 2010 Feb. The technology acceptance model: its past and its
children with asthma: a qualitative study of primary caregivers. J. Nurs. Res.
future in health care. J. Biomed. Inf. 43 (1), 159e172. http://dx.doi.org/10.1016/
[Epub ahead of print]. PMID: 25875102.
j.jbi.2009.07.002.
Connecting for Health Personal Health Working Group, 2003. The Personal Health
Holden, R.J., Carayon, P., Gurses, A.P., Hoonakker, P., Hundt, A.S., Ozok, A.A., et al.,
Working Group: Final Report (Internet) [cited 2015 Oct 16]. Available from:
2013. SEIPS 2.0. A human factors framework for studying and improving the
http://www.policyarchive.org/collections/markle/index?section¼5&id¼15473.
work of healthcare professionals and patients. Ergonomics 56 (11), 1669e1686.
Cronin, R.M., Davis, S.E., Shenson, J.A., Chen, Q., Rosenbloom, S.T., Jackson, G.P., 2015
http://dx.doi.org/10.1080/00140139.2013.838643.
Apr 29. Growth of secure messaging through a patient portal as a form of
Holden, R.J., Schubert, C.C., Mickelson, R.S., 2015 Mar. The patient work system: an
outpatient interaction across clinical specialties. Appl. Clin. Inf. 6 (2), 288e304.
analysis of self-care performance barriers among elderly health failure patients
http://dx.doi.org/10.4338/ACI-2014-12-RA-0117.
and their informal caregivers. Appl. Ergon. 47, 133e150. http://dx.doi.org/
Crotty, B.H., Mostaghimi, A., Landon, B.E., 2013 Oct. Preparing residents for future
10.1016/j.apergo.2014.09.009.
practice: report of a curriculum for electronic patient-doctor communication.
Hsieh, H.F., Shannon, S.E., 2005 Nov. Three approaches to qualitative content
Postgrad. Med. J. 89 (1056), 554e559. http://dx.doi.org/10.1136/postgradmedj-
analysis. Qual. Health Res. 15 (9), 1277e1288. PMID: 16204405.
2012-131688.
Institute of Medicine, 2009. Health Literacy, eHealth, and Communication: Putting
Day, K., Gu, Y., 2012. Influencing factors for adopting personal health record (PHR).
the Consumer First: Workshop Summary. Institute of Medicine (US) Roundtable
Stud. Health Technol. Inf. 178, 39e44. PMID: 22797017.
on Health Literacy. National Acadamies Press, Washginton, DC. PMID:
Denton, I.C., 2001. Will patients use electronic personal health records? responses
20662120.
from a real-life experience. J. Healthc. Inf. Manag. 15 (3), 251e259.
Jabour, A., Jones, J., 2013. Facilitators and Barriers to Patients' Engagements with
Department of Health and Human Services, 2013 Jun 3. Incentives for nondis-
Personal Health Records: Systematic Review. Universal Access in Human-
criminatory wellness programs in group health plans (Internet). Fed. Regist. 78
computer Interaction Applications and Services for Quality of Life. Springer
(106), 33158e33175. Available from: http://www.gpo.gov/fdsys/pkg/FR-2013-
Berline Heidelberg, Las Vegas, NV, pp. 472e481. http://dx.doi.org/10.1007/978-
06-03/pdf/2013-12916.pdf.
3-642-39194-1_55.
Desnoyers, L., 2004. The role of qualitative methodology in ergonomics: a com-
Jung, C., Padman, R., Shevchik, G., Paone, S., 2011. Who are portal users vs. early e-
mentary. Theor. Issues Ergon. Sci. 5 (6), 495e498. http://dx.doi.org/10.1080/
Visit adopters? A preliminary analysis. AMIA Annu. Symp. Proc. 1070e1079.
14639220412331303373.
PMID: 22195168.
Do, N.V., Barnhill, R., Heermann-Do, K., Salzman, K.L., Gimbel, R.W., 2011. The
Kahn, J.S., Hilton, J.F., Van Nunnery, T., Leasure, S., Bryant, K.M., Hare, C.B., et al., 2010
military health system's personal health record pilot with microsoft Health-
MareApr. Personal health records in a public hospital: experience at the HIV/
Vault and google health. J. Am. Med. Inf. Assoc. 18 (2), 118e124. http://
AIDS clinic at San Francisco General Hospital. J. Am. Med. Inf. Assoc. 17 (2),
dx.doi.org/10.1136/jamia.2010.004671.
224e228. http://dx.doi.org/10.1136/jamia.2009.000315.
Dul, J., Bruder, R., Buckle, P., Carayon, P., Falzon, P., Marras, W., et al., 2012. A strategy
Karsh, B.T., Holden, R.J., Alper, S.J., Or, C.K., 2006 Dec. A human factors engineering
M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 241

paradigm for patient safety - designing to support the performnce of the health Committee on the Role of Human Factors in Home Health Care, Board on
care professional. Qual. Saf. Health Care 15 (Suppl. 1), i59e65. PMID: 17142611. Human-systems Integration, Division of Behavioral and Social Sciences and
Katz, S.J., Moyer, C.A., Cox, D.T., Stern, D.T., 2003 Sep. Effect of a triage-based E-mail Education. The National Academies Press, Washington, DC.
system on clinic resource use and patient and physician satisfaction in primary Nazi, K.M., 2010 MareApr. Veterans' voices: use of the American Customer Satis-
care: a randomized controlled trial. J. Gen. Intern. Med. 18 (9), 736e744. PMID: faction Index (ACSI) Survey to identify My HealtheVet personal health record
12950483. users' characteristics, needs, and preferences. J. Am. Med. Inf. Assoc. 17 (2),
Kim, E.-H., Stolyar, A., Lober, W.B., Herbaugh, A.L., Shinstrom, S.E., Zierler, B.K., et al., 203e211. http://dx.doi.org/10.1136/jamia.2009.000240.
2009 Oct 27. Challenges to using an electronic personal health record by a low- Nazi, K.M., 2013 April 4. The personal health record paradox: health care pro-
income elderly population. J. Med. Internet Res. 11 (4), e44. http://dx.doi.org/ fessionals' perspectives and the information ecology of personal health record
10.2196/jmir.1256. systems in organizational and clinical settings. J. Med. Internet Res. 15 (4), e70.
Krist, A.H., Peele, E., Woolf, S.H., Rothemich, S.F., Loomis, J.F., Longo, D.R., et al., 2011 http://dx.doi.org/10.2196/jmir.2443.
Nov 24. Designing a patient-centered personal health record to promote pre- Nazi, K.M., Hogan, T.P., Mcinnes, D.K., Woods, S.S., Graham, G., 2013 Mar. Evaluating
ventive care. BMC Med. Inf. Decis. Mak. 11, 73. http://dx.doi.org/10.1186/1472- patient access to electronic health records: results from a survey of veterans.
6947-11-73. Med. Care 51 (3 Suppl. 1), S52eS56. http://dx.doi.org/10.1097/
Lau, A.Y.S., Dunn, A.G., Mortimer, N., Gallagher, A., Proudfoot, J., Andrews, A., et al., MLR.0b013e31827808db.
2013a Sep 23. Social and self-reflective use of a web-based personally Nielsen, A.S., Halamka, J.D., Kinkel, R.P., Israel, B., 2012 JaneFeb. Internet portal use
controlled health management system. J. Med. Internet Res. 15 (9), e211. http:// in an academic multiple sclerosis center. J. Am. Med. Inf. Assoc. 19 (1), 128e133.
dx.doi.org/10.2196/jmir.2682. http://dx.doi.org/10.1136/amiajnl-2011-000177.
Lau, A., Dunn, A., Mortimer, N., Proudfoot, J., Andrews, A., Liaw, S.T., et al., 2013b. Osborn, C.Y., Mayberry, L.S., Wallston, K.A., Johnson, K.B., Elasy, T., 2013 Jul 3. Un-
Consumers' online social network topologies and health behaviours. Stud. derstanding patient portal use: implications for medication management.
Health Technol. Inf. 192, 77e81. PMID: 23920519. J. Med. Internet Res. 15 (7), e133. http://dx.doi.org/10.2196/jmir.2589.
Lau, A.Y., Proudfoot, J., Andrews, A., Liaw, S.T., Crimmins, J., Arguel, A., Coiera, E., Otte-Trojel, T., de Bont, A., Rundall, T.G., van de Klundert, J., 2015 Sep 2. What do we
2013c. Which bundles of features in a web-based personally controlled health know about developing patient portals? A systematic literature review. J. Am.
management system are associated with consumer help-seeking behaviors for Med. Inf. Assoc. http://dx.doi.org/10.1093/jamia/ocv114. Pii: ocv114.
physical and emotional well-being? J. Med. Internet Res. 15 (5), e79. http:// Poon, E.G., Wald, J., Schnipper, J.L., Grant, R., Gandhi, T.K., Volk, L.A., et al., 2007.
dx.doi.org/10.2196/jmir.2414. Empowering patients to improve the quality of their care: design and imple-
Legris, P., Ingham, J., Collerette, P., 2003 Jan. Why do people use information mentation of a shared health maintenance module in a US integrated health-
technology? A critical review of the technology acceptance model. Inf. Manag. care delivery network. Stud. Health Technol. Inf. 129 (Pt 2), 1002e1006. PMID:
40 (3), 191e204. http://dx.doi.org/10.1016/S0378-7206(01)00143-4. 17911866.
Lin, C.T., Wittevrongel, L., Moore, L., Beaty, B.L., Ross, S.E., 2005 Aug 5. An internet- PricewaterhouseCoopers LLP, 2012 Jun. Emerging MHealth: Paths for Growth
based patient-provider communication system: randomized controlled trial. (Internet) [cited 2015 Oct 15]. Available from: https://www.pwc.com/gx/en/
J. Med. Internet Res. 7 (4), e47. PMID: 16236699. healthcare/mhealth/assets/pwc-emerging-mhealth-exec-summary.pdf.
Lober, W.B., Zierler, B., Herbaugh, A., Shinstrom, S.E., Stolyar, A., Kim, E.H., et al., ProQuest LLC. RefWorks. 2010.
2006. Barriers to the use of a personal health record by an elderly population. Sandelowski, M., 2000 Aug. Whatever happened to qualitative description? Res.
AMIA Annu. Symp. Proc. 514e518. PMID: 17238394. Nurs. Health 23 (4), 334e340. PMID: 10940958.
Marquard, J.L., Zayas-Caban, T., 2012 JaneFeb. Transforming off-the-shelf consumer Sandelowski, M., 2010 Feb. What's in a name? Qualitative description revised. Res.
health informatics interventions: recommendations for their design, evaluation Nurs. Health 33 (1), 77e84. http://dx.doi.org/10.1002/nur.20362.
and redesign. J. Am. Med. Inf. Assoc. 19 (1), 137e142. http://dx.doi.org/10.1136/ Sarkar, U., Karter, A.J., Liu, J.Y., Schillinger, D., 2010. The literacy divide: health lit-
amiajnl-2011-000338. eracy and the use of an internet-based patient portal in an integrated health
Mayberry, L.S., Kripalani, S., Rothman, R.L., Osborn, C.Y., 2011 Oct. Bridging the system-results from the diabetes study of northern California. J. Health Com-
digital divide in diabetes: family support and implications for health literacy. mun. 15 (Suppl. 2), 183e196. http://dx.doi.org/10.1080/10810730.2010.499988.
Diabetes Technol. Ther. 13 (10), 1005e1012. http://dx.doi.org/10.1089/ Sarkar, U., Karter, A.J., Liu, J.Y., Adler, N.E., Nguyen, R., Lopez, A., et al., 2011 May 1.
dia.2011.0055. Social disparities in internet patient portal use in diabetes: evidence that the
McHugh, M.L., 2012. Interrater reliability: the kappa statistic. Biochem. Med. 22 (3), digital divide extends beyond access. J. Am. Med. Inf. Assoc. 18 (3), 318e321.
276e282. PMID: 23092060. http://dx.doi.org/10.1136/jamia.2010.006015.
McInnes, D.K., Solomon, J.L., Shimada, S.L., Petrakis, B.A., Bokhour, B.G., Asch, S.M., Sarkar, U., Bates, D.W., 2014 Jan 22e29. Care partners and online patient portals.
et al., 2013 Mar. Development and evaluation of an internet and personal health JAMA 311 (4), 357e358. http://dx.doi.org/10.1001/jama.2013.285825.
record training program for low-income patients with HIV or hepatitis C. Med. Schnipper, J.L., Ghandi, T.K., Wald, J.S., Grant, R.W., Poon, E.G., Volk, L.A., et al., 2008.
Care 51 (3 Suppl. 1), S62eS66. http://dx.doi.org/10.1097/ Design and implementation of a web-based patient portal linked to an elec-
MLR.0b013e31827808bf. tronic health record designed to improve medication safety: the patient
Merriam-Webster. Barrier (Internet). An Encyclopӕdia Britannica Company. (cited gateway medications module. Inf. Prim. Care 16 (2), 147e155. PMID: 18713531.
2015 Oct 16). Available from: http://www.merriam-webster.com/dictionary/ Skeels, M.M., 2010. Sharing by Design: Understanding and Supporting Personal
barrier. Health Information Sharing and Collaboration within Social Networks. Uni-
Microsoft Office Professional Plus 2010, 2010. Microsoft Excel, Redmond. versity of Washington, ISBN 978-1-1243-0841-8.
Miller, H., Vandenbosch, B., Ivanov, D., Black, P., 2007 Summer. Determinants of Smith, M.J., Sainfort, P.C., 1989 Jul. A balance theory of job design for stress
personal health record use: a large population study at cleveland clinic. reduction. Int. J. Ind. Ergon. 4 (1), 67e79. http://dx.doi.org/10.1016/0169-
J. Healthc. Inf. Manag. 21 (3), 44e48. PMID: 19195293. 8141(89)90051-6.
Moen, A., Brennan, P.F., 2005 NoveDec. Health@ Home: the work of health infor- Streid, J., Harding, R., Agupio, G., Dinat, N., Downing, J., Gwyther, L., et al., 2014 Mar.
mation management in the household (HIMH): implications for consumer Stressors and resources of caregivers of patients with incurable progressive
health informatics (CHI) innovations. J. Am. Med. Inf. Assoc. 12 (6), 648e656. illness in sub-Saharan Africa. Qual. Health Res. 24 (3), 317e328. http://
PMID: 16049230. dx.doi.org/10.1177/1049732314523682.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D., 2009 Aug. Preferred reporting items Taha, J., Czaja, S.J., Sharit, J., Morrow, D.G., 2013 Dec. Factors affecting usage of a
for systematic reviews and meta-analyses: the PRISMA statement. PRISMA personal health record (PHR) to manage health. Psychol. Aging 28 (4),
Group. Ann. Intern. Med. 151 (4), 264e269. W64. PMID: 19622511. 1124e1139. http://dx.doi.org/10.1037/a0033911.
Moray, N., 2000 Jul. Culture, politics and ergonomics. Ergonomics 43 (7), 858e868. Tenforde, M., Nowacki, A., Jain, A., Hickner, J., 2012 Apr. The association between
PMID: 10929822. personal health record use and diabetes quality measures. J. Gen. Intern. Med.
Nagykaldi, Z., Aspy, C.B., Chou, A., 2012 MareApr. Impact of a wellness portal on the 27 (4), 420e424. http://dx.doi.org/10.1007/s11606-011-1889-0.
delivery of patient-centered preventive care. J. Am. Board Fam. Med. 25 (2), The Office of the National Coordinator for Health Information Technology (ONC),
158e167. http://dx.doi.org/10.3122/jabfm.2012.02.110130. 2013a. How to Attain Meaningful Use (Internet) [cited 2015 Oct 16]. Available
National eHealth Collaborative and Healthcare Information and Management Sys- from: https://www.healthit.gov/providers-professionals/how-attain-
tems Society Foundation, 2014 Feb. The Patient Engagement Framework meaningful-use.
(Internet) [cited 2015 Oct 16]. Available from: http://www.himss.org/ The Office of the National Coordinator for Health Information Technology (ONC),
ResourceLibrary/genResourceDetailPDF.aspx?ItemNumber¼28305. 2013b. Personal Health Records: what Health Care Providers Need to Know
National Institute on Aging, 2010e2013a. Understanding and Promoting Health (Internet) [cited 2015 Oct 16]. Available from: http://www.healthit.gov/sites/
Literacy (RO1). PAR-10e133. (Internet). Available from: https://www.nia.nih. default/files/about-phrs-for-providers-011311.pdf.
gov/research/funding/2011/09/understanding-and-promoting-health-literacy- The Office of the National Coordinator for Health Information Technology (ONC),
r01. 2014. Federal Health it Strategic Plan 2015e2020 (Internet) [cited 2015 Oct 16].
National Institute on Aging, 2010e2013b. Understanding and Promoting Health Available from: https://www.healthit.gov/sites/default/files/federal-healthIT-
Literacy (R03) (Internet). PAR-10e134. Available from: https://www.nia.nih. strategic-plan-2014.pdf.
gov/research/funding/2011/09/understanding-and-promoting-health-literacy- Thompson, M., Valdez, R., 2013. Macroergonomic Factors of the Patient Work Sys-
r03. tem: the Case of mHealth Based Asthma Self-management Second Presentation
National Institute on Aging, 2013e2016. Understanding and Promoting Health Format. Summary.
Literacy (R21) (Internet). PAR-13e132. Available from: http://grants.nih.gov/ Tjora, A., Tran, T., Faxvaag, A., 2005 May 31. Privacy vs usability: a qualitative
grants/guide/pa-files/PAR-13-132.html. exploration of patients' experiences with secure internet communication with
National Research Council, 2011. Health Care Comes Home: the Human Factors. their general practitioner. J. Med. Internet Res. 7 (2), e15. PMID: 15998606.
242 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Tom, J.O., Mangione-Smith, R., Solomon, C., Grossman, D.C., 2012 Jul. Integrated 2010 SepeOct. Implementing practice-linked pre-visit electronic journals in
personal health record use: association with parent-reported care experiences. primary care: patient and physician use and satisfaction. J. Am. Med. Inf. Assoc.
Pediatrics 130 (1), 183e190. http://dx.doi.org/10.1542/peds.2011-1786. 17 (5), 502e506. http://dx.doi.org/10.1136/jamia.2009.001362.
Tsai, J., Rosenheck, R.A., Ave, C., 2012 NoveDec. Use of the internet and an online Wang, M., Lau, C., Matsen, F.A., Kim, Y., 2004 Sep. Personal health information
personal health record system by US veterans: comparison of Veterans Affairs management system and its application in referral management. IEEE Trans. Inf.
mental health service users and other veterans nationally. J. Am. Med. Inf. Technol. Biomed. 8 (3), 287e297. PMID: 15484434.
Assoc. 19 (6), 1089e1094. http://dx.doi.org/10.1136/amiajnl-2012-000971. Wasson, J.H., Forsberg, H.H., Lindblad, S., Mazowita, G., McQuillen, K., Nelson, E.C.,
Tuil, W.S., Hoopen, A.J., Braat, D.D.M., Robbe , P.F.D.V., Kremer, J.A.M., 2006 Nov. 2012 ApreJun. The medium is the (health) measure: patient engagement using
Patient-centred care: using online personal medical records in IVF practice. personal technologies. J. Ambul. Care Manag. 35 (2), 109e117. http://dx.doi.org/
Hum. Reprod. 21 (11), 2955e2959. PMID: 16982658. 10.1097/JAC.0b013e31824a235e.
Ulrichsweb: Global Serials Directory (Internet). (cited 2015 Oct 16). Available from: Weingart, S.N., Rind, D., Tofias, Z., Sands, D.Z., 2006 JaneFeb. Who uses the patient
http://ulrichsweb.serialssolutions.com/. internet portal? The PatientSite experience. J. Am. Med. Inf. Assoc. 13 (1),
Urowitz, S., Wiljer, D., Dupak, K., Kuehner, Z., Leonard, K., Lovrics, E., et al., 2012 Nov 91e95. PMID: 16221943.
30. Improving diabetes management with a patient portal: a qualitative study Weitzman, E.R., Kelemen, S., Kaci, L., Mandl, K.D., 2012 May 22. Willingness to share
of diabetes self-management portal. J. Med. Internet Res. 14 (6), e158. http:// personal health record data for care improvement and public health: a survey
dx.doi.org/10.2196/jmir.2265. of experienced personal health record users. BMC Med. Inf. Decis. Mak. 12, 39.
Valdez, R.S., Holden, R.J., Novak, L.L., Veinot, T.C., 2015a Apr. Technological infra- http://dx.doi.org/10.1186/1472-6947-12-39.
structure implications of the patient work framework. J. Am. Med. Inf. Assoc. 22 Wells, K., Vazquez-Otero, C., Rivera, M., Chaet, A., Proctor, S., Arroyo, G., et al., 2015.
(e1), e213e5. http://dx.doi.org/10.1093/jamia/ocu031. Assessing the acceptability of an embodied conversational agent-based com-
Valdez, R.S., Holden, R.J., Novak, L.L., Veinot, T.C., 2015b Jan. Transforming consumer puter application for hispanic women. J. Hisp. Health Care Int.
health informatics through a patient work framework: connecting patients to Wen, K.Y., Kreps, G., Zhu, F., Miller, S., 2010 Oct-Dec. Consumers' perceptions about
context. J. Am. Med. Inf. Assoc. 22 (1), 1e7. http://dx.doi.org/10.1136/amiajnl- and use of the internet for personal health records and health information
2014-002826. exchange: analysis of the 2007 Health Information National Trends Survey. J.
Valdez, R.S., Brennan, P.F., 2015 May. Exploring patients' health information Med. Internet. Res. 12 (6), e73. http://dx.doi.org/10.2196/jmir.1668.
communication practices with social network members as a foundation for Werner, P., Mittelman, M.S., Goldstein, D., 2012 Feb. Family stigma and caregiver
consumer health IT design. Int. J. Med. Inf. 84 (5), 363e374. http://dx.doi.org/ burden in Alzheimer's disease. Gerontologist 52 (1), 89e97. http://dx.doi.org/
10.1016/j.ijmedinf.2015.01.014. 10.1093/geront/gnr117.
Valdez, R., Holden, R., Carayon, P., 2015. Expanding representations of individuals Wiljer, D., Leonard, K.J., Urowitz, S., Apatu, E., Massey, C., Quartey, N.K., et al., 2010
and their interactions within work system models. International Ergonomics Sep 1. The anxious wait: assessing the impact of patient accessible EHRs for
Association, Melbourne, Australia. breast cancer patients. BMC Med. Inf. Decis. Mak. 10, 46. http://dx.doi.org/
Vincent, C., 2003 Mar 13. Understanding and responding to adverse events. N. Engl. 10.1186/1472-6947-10-46.
J. Med. 348 (11), 1051e1056. PMID: 12637617. Woods, S.S., Schwartz, E., Tuepker, A., Press, N.A., Nazi, K.M., Turvey, C.L., et al., 2013
Vodicka, E., Mejilla, R., Leveille, S.G., Ralston, J.D., Darer, J.D., Delbanco, T., et al., 2013 Mar 27. Patient experiences with full electronic access to health records and
Sep 26. Online access to doctors' notes: patient concerns about privacy. J. Med. clinical notes through the My HealtheVet Personal Health Record Pilot: quali-
Internet Res. 15 (9), e208. http://dx.doi.org/10.2196/jmir.2670. tative study. J. Med. Internet Res. 15 (3), e65. http://dx.doi.org/10.2196/
Wade-Vuturo, A., Mayberry, L.S., Osborn, C.Y., 2013 May. Secure messaging and jmir.2356.
diabetes management: experiences and perspectives of patient portal users. Wynia, M.K., Torres, G.W., Lemieux, J., 2011 Feb. Many physicians are willing to use
J. Am. Med. Inf. Assoc. 20 (3), 519e525. http://dx.doi.org/10.1136/amiajnl-2012- patients' electronic personal health records, but doctors differ by location,
001253. gender, and practice. Health Aff. 30 (2), 266e273. http://dx.doi.org/10.1377/
Wagner, P.J., Howard, S.M., Bentley, D.R., Seol, Y., 2010 Oct 1. Incorporating patient hlthaff.2010.0342.
perspectives into the personal health record: implications for care and caring. Ye, J., Rust, G., Fry-Johnson, Y., Strothers, H., 2010 Aug. E-mail in patient-provider
Perspect. Health Inf. Manag. 7, 1e. PMID: 21063546. communication: a systematic review. Patient Educ. Couns. 80 (2), 266e273.
Wagner, P.J., Dias, J., Howard, S., Kintziger, K.W., Hudson, M.F., Seol, Y., et al., 2012 http://dx.doi.org/10.1016/j.pec.2009.09.038.
JuleAug. Personal health records and hypertension control: a randomized trial. Zickmund, S.L., Hess, R., Bryce, C.L., Mctigue, K., Olshansky, E., Fitzgerald, K., et al.,
J. Am. Med. Inf. Assoc. 19 (4), 626e634. http://dx.doi.org/10.1136/amiajnl-2011- 2008 Jan. Interest in the use of computerized patient portals: role of the
000349. provider-patient relationship. J. Gen. Intern. Med. 23 (Suppl. 1), 20e26. http://
Wald, J.S., Grant, R.W., Schnipper, J.L., Gandhi, T.K., Poon, E.G., Alexandra, C., et al., dx.doi.org/10.1007/s11606-007-0273-6.
2009 Nov 14. Survey analysis of patient experience using a practice-linked PHR Zulman, D.M., Nazi, K.M., Turvey, C.L., Wagner, T.H., Woods, S.S., An, L.C., 2011 Dec
for type 2 diabetes mellitus. AMIA Annu. Symp. Proc. 2009, 678e682. PMID: 20. Patient interest in sharing personal health record information. Ann. Intern.
20351940. Med. 155 (12), 805e811. http://dx.doi.org/10.7326/0003-4819-155-12-
Wald, J.S., Businger, A., Gandhi, T.K., Grant, R.W., Poon, E.G., Schnipper, J.L., et al., 201112200-00002.