Usability and Perceived Usefulness of Personal Health Records For Preventive Health Care - A Case Study

Applied Ergonomics 45 (2014) 613e628
Contents lists available at ScienceDirect
Applied Ergonomics
journal homepage: www.elsevier.com/locate/apergo
Usability and perceived usefulness of personal health records for

preventive health care: A case study focusing on patients and primary
care providers perspectives
A. Ant Ozok a, *, Huijuan Wu a, 1, Melissa Garrido b, 2, Peter J. Pronovost c, 3,
Ayse P. Gurses c, 4
a
Department of Information Systems, University of Maryland Baltimore County, 1000 Hilltop Circle, Baltimore, MD 21250, USA
GRECC, James J Peters VA Medical Center, 130 W Kingsbridge Road, Bronx, NY, USA
Armstrong Institute for Patient Safety and Quality, Department of Anesthesiology and Critical Care Medicine, School of Medicine, The Johns Hopkins
University, 750 E. Pratt Street, 15th Floor, Baltimore, MD 21202, USA
b
c
a r t i c l e i n f o
a b s t r a c t
Article history:
Received 26 July 2011
Accepted 2 September 2013
Personal Health Records (PHR) are electronic applications for individuals to access, manage and share
their health information in a secure environment. The goal of this study was to evaluate the usefulness
and usability of a Web-based PHR technology aimed at improving preventive care, from both the patients and primary care providers perspectives. We conducted a multi-method descriptive study that
included direct observations, concurrent think-aloud, surveys, interviews and focus groups in a suburban
primary care clinic. Patients found the tailored health recommendations useful and the PHR easy to
understand and use. They also reported asking useful health-related questions to their physicians
because of using the system. Generally, care providers were interested in using the system due to its
useful content and impact on patient activation. Future successful systems should be better integrated
with hospital records; put more emphasis on system security; and offer more tailored health information
based on comprehensive health databases.
2013 Elsevier Ltd and The Ergonomics Society. All rights reserved.
Keywords:
Personal health records
Usefulness
Usability
1. Introduction
Each year, thousands of people suffer from potentially preventable illnesses such as heart attack, stroke and cancer. In addition to claiming lives and reducing individuals quality of life, these
illnesses also result in millions of dollars in health care costs
(Peterson and Dragon, 1998; Avorn and Shrank, 2008). Screening
can prevent a considerable amount of these illnesses and reduce
costs (Salkeld, 2006). The number of recommended preventive
screening tests has increased in the last decade, but patients
compliance with recommended health screening rates still remains
low (Manne et al., 2002; Weinberg et al., 2004). In 2003, the
* Corresponding author. Tel.: 1 410 455 8627; fax: 1 410 455 1073.
E-mail addresses: ozok@umbc.edu (A. Ant Ozok), huijuan.wu@umbc.edu
(H. Wu), melissa.garrido@mssm.edu (M. Garrido), ppronovo@jhmi.edu
(P.J. Pronovost), agurses1@jhmi.edu (A.P. Gurses).
1
Tel.: 1 455 8834; fax: 1 410 455 1073.
2
Tel.: 1 732 932 5230; fax: 1 732 932 1945.
3
Tel.: 1 410 637 6261.
4
Tel.: 1 410 637 4387.
seminal paper of McGlynn and colleagues indicated that patients

receive only 55% of the recommended preventive care based on a
survey of a random sample of 6712 adults in 12 metropolitan areas
in the US (McGlynn et al., 2003). Based on CDCs recent report, for
example, only 59% of patients in the US receive the recommended
colorectal cancer screening tests (Centers for Disease Control and
Prevention, 2012). These low rates may partially be due to a lack
of awareness by clinicians and patients, inadequate reinforcement
mechanisms for patients with regards to compliance with the
recommended preventive screening guidelines, limited access to
high quality of care, and nancial barriers (Mosca et al., 2006).
Patient-centric health information technologies (HIT) are
becoming increasingly popular (Perlin et al., 2004; Sunyaev et al.,
2010). To offer better care at lower costs, HIT needs to support
and facilitate patient-centered care rather than focusing on isolated
physician and clinician tasks (Walker and Carayon, 2009). If
designed well, HIT can educate patients on health care topics, give
reminders to ensure timely screening, and provide support mechanisms for patients to facilitate their preventive screening decisions
(Institute of Medicine Committee on Quality of Health Care in
0003-6870/$ e see front matter 2013 Elsevier Ltd and The Ergonomics Society. All rights reserved.
http://dx.doi.org/10.1016/j.apergo.2013.09.005
614
A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628
Perceived
usefulness
Attitude
Behavioral intention to
use (acceptance)
Actual use
Compliance with
preventive
screening
recommendations
and lifestyle
changes
Perceived
usability
Fig. 1. Conceptual framework guiding the study (adapted from the Technology Acceptance Model) (Davis, 1989; Davis et al., 1989).
America, 2001). Personal Health Records (PHR) are an HIT that can
play a signicant role in improving patients knowledge of and
attitudes toward their health care (Powsner, 1998; Brennan, 2000).
The Markle Foundation (2003) denes PHR as an electronic
application through which individuals can access, manage and
share their health information in a private, secure, and condential
environment. International Standards Organization (ISO) (2009)
indicated: The Personal Health Record of an individual is a repository of information considered by that individual to be relevant
to his or her health, wellness, development and welfare, and for
which that individual has primary control over the records
content.
PHR can improve health care, including preventive care, through
educating and engaging patients in their own health care (Krist and
Woolf, 2011; Tang et al., 2006), and as a result transforming them
into more activated patients (Von Korff et al., 1998; Lorig et al.,
1996). Patient activation aims to help patients develop the skills,
knowledge and motivation to become an effective participant in
their health (Hibbard et al., 2004, 2005). It has been linked to better
health outcomes and lower costs (Bodenheimer et al., 2002; Britto
and Wimberg, 2009). PHR can increase patient activation (Eriksen
and Ursin, 2004; Goth, 2008; Ueckert et al., 2003) and improve
communication between patients and caregivers (Ueckert et al.,
2003) (Eriksen and Ursin, 2004; Steele and Lo, 2009). These
studies make it clear that patients are more likely to make good
decisions and take actions to promote their health if they become
active participants of their own health care (Bourgeois et al., 2008;
Ueckert et al., 2003).
In the early stages of health information systems, PHR-like
systems were not always Web-based (Tang et al., 2006;
Srinivasan, 2006). While there are still various platforms supporting PHR systems, including free-standing/PC-based, universal serial
bus (USB)/portable storage-based, and mobile/smart phone-based
platforms, Web-based systems have so far shown their dominancy (Steele and Lo, 2009). Unlike Web-based health portals,
which in most cases consist of a collection of standard information
on health issues and illnesses and user-created discussion boards
(Weingart et al., 2006). PHR systems typically allow users to
manipulate their own health information, and therefore have the
potential to better satisfy their health information needs and
positively impact health behaviors (Malamateniou and
Vassilacopoulos, 2010). PHR can motivate patients to take the recommended preventive screening tests and to improve compliance
with drug and other therapies by providing them with either
tailored recommendations and education or electronic reminder
mechanisms (Palen and Aalokke, 2006; Krist et al., 2011). These
recommendations and reminders can lead to more informed patientecaregiver interactions and cost savings. PHR can help family
members manage the health information of specic groups such as
children, elderly, or terminally disabled patients as well

(Greenhalgh et al., 2010; Bourgeois et al., 2008).
PHR systems can also improve quality of health care by supporting care providers work. For example, they can help in closing
the health information gap between patients and providers by
making the episodic nature of care more continuous (Ball et al.,
2006) as well as facilitate patient education and shared decisionmaking.
Although PHR have great potential to improve health care (Tang
and Lansky, 2005; Scherger, 2005), in particular preventive care
(Kahn et al., 2009; Krist et al., 2011), to date there is limited evidence supporting a positive impact of PHR on health care from
patients as well as caregivers perspectives (Tang et al., 2006;
Zuckerman and Kim, 2009). A small number of studies identied
barriers to acceptance and widespread usage concerns about privacy, security and condentiality of health-related data, feasibility
of integration of PHR to electronic health records (EHR), accuracy
and completeness of data entered, and usability and usefulness of
PHR systems (Denton, 2001; Liu et al., 2011; Tang et al., 2006;
Hargreaves, 2010). For PHR to gain widespread use, it needs to be
accepted and adopted by both patients and care providers, as
similar technology acceptance and adoption issues were reported
in other areas (Venkatesh and Davis, 2000). It is therefore important to study factors that may potentially affect PHR acceptance
from both the patients and care providers viewpoints. This will
guide us on how to design and implement PHR that will meet the
needs of both stakeholders, and will increase the likelihood of
success.
There is a knowledge gap in the literature regarding patient and
care provider perceptions, attitudes and preferences toward any
type of HIT (Angst and Agarwal, 2009), in particular PHR (Baird
et al., 2011). A number of studies focused on usability and usefulness issues from both patient and provider perspectives
(Montague, 2010; Martin et al., 2011), but they focused on medical
device design, rather than HIT. Furthermore, previous studies
evaluated PHR from either patients (direct users of PHR) or providers (indirect users) perspective, which is only part of the picture. More work is needed to understand the use of PHR systems as
a tool that can complement the traditional care, as well as their
impact on patients, providers, organizations and health care systems based on human factors and ergonomics concepts and
methods (Nazi et al., 2009).
Based on the above factors and the well-known Technology
Acceptance Model (TAM) (Davis, 1989; Davis et al., 1989), we argue
that high-usability and usefulness of patient-centric HIT can be
valuable in improving patients compliance with the recommended
guidelines and medical outcomes. Fig. 1 presents the overarching
conceptual framework guiding our study, which was adapted from
the TAM. The framework indicates that a patients or care providers
attitude toward the PHR system can be determined by two constructs: perceived usefulness and perceived usability. Instead of the
construct perceived ease of use included in the original TAM
model, the modied framework uses the construct of perceived
usability. Due to the broader range of the usability term that includes ease of use as well as a number of other concepts (Nielsen,
1993), perceived usability was chosen as one of the main constructs. Nielsen (1993) denes a high-usability interface as an
interface with high learnability, efciency, memorability and
satisfaction, and one with a low number of errors. In the context of
this study, perceived usability refers to the degree to which using
this particular PHR system will be free of effort. We dene
perceived usefulness as the degree to which a patient or provider
believes that using the PHR would increase compliance with preventive screening and lifestyle changes. The conceptual framework
postulates that if patients and providers have a more positive
attitude toward the technologies they use (in terms of usefulness
and usability), their acceptance, intention to use, and eventually
their actual use of these technologies will also be higher.
The purpose of this study was to evaluate the value (i.e., usefulness and usability) of a specic patient-centered information
technology in improving awareness of (for patients) and compliance with (for both patients and care providers) preventive care
guidelines. Using a case study, we specically aimed at determining
patients and providers perceptions regarding (1) whether a PHR
system can be useful in improving patientecare provider interactions, preventive health screening, and compliance with
healthier lifestyle behaviors; and (2) the key usability factors of
such a system. For the specic technology, we used a Web-based,
secure system, MySafe-T.Net, developed as a PHR with the primary aims of (1) improving preventive care and preventive
screening rates and (2) changing patients lifestyles to improve
their health. MySafe-T.Nets secondary aim is to serve as a health
information repository.
Making the technology useful and easy to use for both patients
and care providers is a challenging goal, yet not exclusive to PHR
systems. Health care is lled with situations in which there are
different types of users for the same technology (Carayon et al.,
2010). Care is often provided in complex socio-technical environments (Buckle et al., 2006; Carayon, 2010). Hence, (re)designing
technologies based on human factors engineering, specically usability and usefulness, principles by taking into account the overlapping, complementary, and sometimes conicting needs of
multiple players is crucial if we want to improve overall systems
performance.
2. Methodology
2.1. Description of the MySafe-T.net system
MySafe-T.net was designed with the goal of preventing future
health conditions such as high blood pressure, obesity and cancer
and leading people to live a healthy life style by increasing patient
activation. MySafe-T.net is aimed at increasing patients compliance
with preventive screening tests (e.g., colorectal cancer screenings,
cholesterol screenings) and lifestyle changes (e.g., physical exercise,
eating habits) recommended by the US Preventive Services Task
Force based on an individuals prole including demographics,
personal and family health histories, lifestyle and health habits
(Shea et al., 1996). It can be used by people of various ages and
clinical conditions, including healthy users. The system provides
patients with an interactive environment to access their own health
records via a Web browser. During the initial use, the system asks
each patient to provide information on ve main categories
including demographics, personal health history, family history,
615
lifestyle, and health habits (Appendices A1, A2 and A3). It develops

an individual prole based on this information, and then applies
the U.S. Preventive Services Task Force (2007) preventive health
screening recommendations to this prole to produce prevention
and lifestyle change (e.g., frequency and intensity of physical exercise) recommendations tailored to each person based on their
individual risk factors (Appendix A4). MySafe-T.net also provides
individualized educational Web site links (Appendix A5) based on
each patients prole. The recommendations are updated when the
patient makes changes on their stored prole.
Using the system, the patient can print a letter for the care
provider and share this letter with their physician and other care
providers. This letter includes information about the patients demographics, own and family health histories, and the recommended preventive screening tests and lifestyle changes based on
their prole. This feature was designed with the goal of improving
the communication between patients and care providers and
increasing shared decision-making.
2.2. Study site and sample
The study was conducted at a Minnesota suburban primary care
clinic afliated with a nation-wide institution. We collected data
from both patients and care providers. The clinic coordinator contacted 36 consecutive patients who had made an appointment at
the clinic for an annual physical exam within a two-week window
in February 2008, described the study briey and gave the contact
information of the research assistant (MG) to those interested.
Twenty-nine patients contacted the research assistant, who provided detailed information about the study and checked each patients eligibility for participation. The inclusion criteria included
having scheduled an annual physical exam at the time of the study,
having a moderate level of experience with computers and the
Internet, and being 21 years of age or older. Of the 24 eligible patients, 22 agreed to participate. Two patients declined due to
scheduling issues. The assistant obtained consent from these patients on the phone and instructed them to arrive at the clinic 2 h
before their scheduled appointment. Each participant was paid $70
as an incentive at the end of their visit. Sixteen out of the 22
recruited patients participated in the follow-up phone interview
within two weeks after their clinic appointment. All study procedures were approved by the Institutional Review Boards of the
involved institutions.
We also recruited eight care providers (four out of six primary
care physicians and four out of ve medical assistants) employed in
the same clinic. Physicians were included in the study due to their
important roles in providing preventive care including recommending, ordering and interpreting the results of preventive care
screening, providing lifestyle change recommendations, and prescribing medications to patients. Medical assistants almost always
interact with the patients before the physicians on the day of the
annual physical exam, take medical histories including those
related to preventive care, record vital signs, prepare patients for
examinations, and convey information important for the exam to
physicians. The information included in the PHR overlaps signicantly with the information physicians and medical assistants need
from patients during the annual physical exam.
2.3. Study procedures
We used a multi-method approach (Johnson et al., 2007) for
answering the research questions. We collected data via observations, concurrent think-aloud method, two types of patient interviews (one immediately after the tasks and one follow-up
within 2 weeks of the annual exam), a patient survey, and two
616
Table 1
Summary of data collection methods, time of data collection, participant types, and
research questions addressed.
Data collection methods Time completed
Participant Research
type
questions
addressed
Concurrent thinkaloud method

while completing
the experimental
tasks provided by
researchers in
the PHR
PHR survey (PHRS)
Day of experiment
Patients
Day of experiment
Patients
Semi-structured
interviews
Follow-up phone
interviews
First focus
group
Second focus
group
PHR usability
from patients
perspective
PHR usability and

usefulness from
patients
perspective
Day of experiment
Patients
PHR usability and
usefulness from
patients
perspective
2 weeks after
Patients
PHR usability and
experiment
usefulness from
patients
perspective
PHR usefulness
Primary
3 days after all
and usability
care
patient data
collection completed physicians from care
providers
perspective
Medical
PHR usefulness
6 days after
assistants and usability
all patient data
from care
collection completion
providers
perspective
caregiver focus groups. Table 1 summarizes the data collection

methods along with their respective completion times, participant
types (patients or care providers) and research questions
addressed.
2.3.1. Data collection from patients

The assistant rst introduced herself and escorted the patient to
a room with computer and Internet access where she provided
information and instructions about the study. The patient was
asked to complete eight tasks detailed in Table 2 (sample screen
shots in Appendix A) and was instructed to think-aloud while
she/he was completing the tasks, known as the concurrent thinkaloud method (Van den Haak et al., 2003). This method allowed
us to examine in more depth the perceived usefulness and usability
issues concerning the system. After the patient completed entering
the required information to develop the personal prole (demographics, personal and family health history, medications and
recent screenings), the system provided tailored preventive
screening and lifestyle change recommendations. The patient was
then asked to review these recommendations and visit at least
three Web sites recommended by the system. It also automatically
printed the letter for the care provider which the assistant asked
them to share with their medical assistant and the primary care
physician at the beginning of their examination. During the
experimentation, the research assistant observed participants interactions with the system and took notes on automation surprises,
confusion over instructions, overlooked site functions, and other
potential problems. Next, patients were asked to complete a written survey and participate in an interview. They were then
informed that they could login and use MySafe-T.net any time
within the next two weeks to update their information, review the
recommendations, or receive new recommendations. The experimentation took about 1 h to complete. A follow-up phone interview
was conducted around the two-week mark.
2.3.2. The Personal Health Records Survey (PHRS)
In order to determine the overall user perceptions concerning
the usability and usefulness of the system and to complement the
ndings from the qualitative studies, we conducted a survey among
the patients. The Personal Health Records Survey (PHRS) is a paper-
Table 2
List of the experimental tasks and the instructions and additional information given to users.
Tasks
Users were instructed to:
Users were informed that:
Task 1: Entering Demographic Information
- Enter demographic information

- Save their information and move to the next task
Task 2: Entering Personal Health History
- Enter medication allergies and intolerances using

the corresponding Edit buttons and drop-down menus
- Enter past and current medical information
- Enter their prior surgical history (surgery type and date)
- Enter prior screening tests
- Save and move to next task
- Answer a number of questions for the system
- Classify their habits into one or more of the
health habits categories
- They can always modify their demographic

information later on by clicking on the Demographics
button from the main menu
- They can return to the main menu using the Back button
- They can repeat each procedure until they enter all
of their information
Task 3: Entering Lifestyle and Health Habits
Task 4: Entering Family History

Task 5: Viewing Ones Prole Summary
Task 6: Viewing Recommendations
Task 7: Printing the Letter for the Care

Provider
Task 8: Clicking on the Resources Link
- Enter information regarding the medical history of

family members (acute and chronic illnesses, surgeries)
- Read their summary and make any modications by
going back to the corresponding section
- Read health recommendations produced by the system
based on their previous entries and through the use
of some algorithms developed based on the U.S.
Preventive Services Task Force (USPSTF) guidelines
- Keep the printed letter and hand it to the medical

assistant and the physician at the beginning of
their examination
- Click on and review at least three of the external
Web page links recommended by the system based
on patients prole
- Health habits categories include smoking, alcohol use,

exercise and diet. Patient information on the habits is
used by the system to generate lifestyle
recommendations
- They can also add information about their distant
family members (e.g., uncles and aunts)
- Recommendations are tailored based on the

information patients provided
- For example, if a patient provided high cholesterol
values, specic courses of action for lowering
cholesterol would be recommended in this section
based on age, lifestyle and habits, medical history,
family history and the USPSTF guidelines
- The letter provides a summary of key clinical
information for each patient and preventive health
screenings recommended for each patient
- Patients can get additional information about a
particular health screening and/or lifestyle change
recommendation through these links.
617
Table 3
Description of the personal health records survey.
Section name
Constructs measured/section
content
Dimensions
Measures used/adapted from
Usefulness
Davis (1989)
0.92
Trust and security
Gefen (2000)
0.98
Impact on personal health

management
Denton (2001) and Pratt

et al. (2006)
0.96
0.91
10
0.76
Overall satisfaction
Denton (2001) and Pratt

et al. (2006)
Davis (1989)
Lin et al., 1997
Hackman and Oldham
(1980)
Benets (Duplicate questions

for internal reliability)
Ease of use/usability
0.83
G (part 1)
Workload
0.89
G (part 2)
Any other comments/suggestions?
Demographic information
Relevance of information received from the site

Usefulness of site recommendations
Level of trust on the site
Hesitation on entering condential information
to the site
- Willingness to use the site to collect healthrelated information
- Helpfulness of the site in following recommended
health screenings
- The sites contribution in maintaining a
healthy lifestyle
- Number of steps to complete the tasks
- Perceived number of mistakes made
- Willingness to continue using the site
- Advising others to use the site
- Using the site being a frustrating experience
- Mental demand/memory load: Thinking, deciding,
calculating, remembering, looking, searching, etc.
required to use the site
- Effort: The level of mental and physical demand
to complete the tasks on the site
Open-ended question on whether patients had any
other comments and/or suggestions regarding
the MySafe-T.net system
Age, gender, weekly computer usage, frequency, etc.
(Hart and Staveland,

1988; Human Performance
Research Group, 1997;
Lin et al., 1997
# of items
Cronbachs
alpha values
10
Survey Cronbachs Alpha Values: 0.90, 0.93, 0.81.
based survey aimed at determining patient perceptions regarding

MySafe-T.net. It measures eight major constructs presented in
Table 3 with corresponding dimensions, used measures, number of
items, and Cronbachs Alpha values. Thirty-eight of the 51 survey
items had seven-point Likert scales as response categories ranging
from 1 (strongly disagree) to 7 (strongly agree). Two items (on
mental demand and effort) had 10-point scales (Low to High), and
the rest consisted of demographic questions. The items that
measured usability were in part based on the framework developed
by Lin et al. (1997), and the perceived usefulness measures were
based on the framework by Davis (1989) and Davis et al. (1989).
2.3.3. Patient interviews
2.3.3.1. Face-to-face patient interviews in the clinic. Following the
survey, patients were asked to participate in a short semistructured interview conducted by the research assistant. The
interview guide (Appendix B) was developed based on the conceptual framework guiding the study (Fig. 1, Davis, 1989; Davis
et al., 1989; Lin et al., 1997). The goal of the interview was to pursue a one-on-one interaction with each patient to get more indepth information (compared to the survey) about the positive
and negative perceptions of usefulness and usability of the MySafeT.net and the underlying factors for these perceptions. The usability
questions were based on Nielsen (1993) and Shneiderman (1992)
and Lin et al. (1997), and the usefulness questions, including ease
of use and mental demand, were based on Daviss (1989) framework on perceived usefulness. All interviews were digitally recorded and transcribed.
2.3.3.2. Follow-up patient phone interviews. A follow-up phone
interview with each patient was conducted about two weeks after
the experiment. Sixteen of the 22 initial participants agreed to
participate in the phone interview; 3 could not be reached after 4
attempts to contact via phone, and the rest declined to attend due
to their busy schedules. During this interview, the research assistant inquired about (1) whether the patient used the system since
their physicians visit and why or why not; (2) how the physician
and the medical assistant reacted to the letter for the care provider; and (3) whether they found the system useful and usable/
easy to use in the long-term and why or why not. The assistant took
detailed notes during the interview.
2.3.4. Data collection from care providers
We conducted two focus groups with care providers after
completing data collection from the patients, one with primary care
physicians and one with medical assistants. All recruited care
providers provided care to one or more patients who participated
in the study, which allowed them to examine the letter for the care
provider that the patients shared with them as part of the study (at
least twice) before participating in the focus groups. The focus
group technique was used to allow moderated interactions between care providers and to give providers an opportunity to
enhance each others ideas and opinions concerning the system.
Both focus groups were moderated by a human factors engineering
expert (APG). The care providers did not use MySafe-T.net in
practice. However, conducting the focus groups after the patient
experiments ensured that providers had seen the letter for the
care provider for at least two patients. They were also provided
with a detailed demonstration of the system at the beginning of the
focus group by the moderator. The moderator asked questions using the focus group guide in Appendix C. The sessions were
recorded using a digital voice recorder and transcribed. Each session took under 1 h to complete.
2.3.5. Data analysis
2.3.5.1. Quantitative analysis. We limited our quantitative analysis
to descriptive statistics due to the limited sample size as well as the
supporting nature of the quantitative data. After data from the
surveys were manually entered, standard data checking and verication were performed, and descriptive statistics including means
and standard deviations were calculated.
2.3.5.2. Qualitative analysis. To identify patients and care providers perceptions about the value of MySafe-T.net, we analyzed
618
Table 4
Patient demographics and background.
<35
35e50
51e65
65
Male
Female
6 (27%)
8 (36%)
6 (27%)
2 (9%)
11 (50%)
11 (50%)
Highest education level
8th grade or less

Some high school
High school graduate
Some college or 2-year degree
4 year college
>4 year college
0
2
8
5
6
1
(0%)
(9%)
(36%)
(23%)
(27%)
(5%)
Computer usage frequency
Never
<1/2 h/week
1/2 to 1 h per week
1e3 h per week
3e7 h per week
>7 h/week
1
2
1
3
7
7
Internet usage frequency
Never
<1/2 h/week
1/2 to 1 h per week
1e3 h per week
3e7 h per week
>7 h/week
2
3
2
4
8
3
(9%)
(14%)
(9%)
(18%)
(36%)
(14%)
Types of health sites visited
Health news
3 (14%)
Health information
12 (55%)
Health online communities
1 (5%)
Other (5, 23%): E-mail, Facebook, shopping,
networking
Places the Internet is visited
At home
20 (91%)
At work
10 (45%)
In public places
2 (9%)
Dont have access
0
Other (2, 9%): At mom and dads, in the library
Very good (32%)
Good (59%)
Fair (5%)
Age
Gender
Rating of Ones General Health

Excellent (5%)
(5%)
(9%)
(5%)
(14%)
(32%)
(32%)
the data qualitatively via the following four methods: (1) as part of
the survey collected by one open-ended question, (2) semistructured face-to-face patient interviews immediately after the
experiment, (3) follow-up phone interviews with patients within
two weeks after the experiment in the clinic, and (4) care provider
focus groups. We used qualitative, directed content analysis using a
deductive reasoning approach supplemented by inductive
reasoning (Hsieh and Shannon, 2005). The overall analysis was
guided by the conceptual framework in Fig. 1 (deductive approach).
The top level categories in the coding structure were perceived
usefulness and perceived usability. Although we started our data
analysis with these two major categories, we allowed any additional top-level categories as well as sub-categories under each
major category to emerge from the data (inductive approach). The
coding structure was developed by two human factors engineers
(AAO, APG) using a consensus approach. The combined deductive
and inductive approaches ensured that we stayed focused on the
research questions of interest (perceived usefulness and usability of
PHR in improving compliance with preventive screening and life
style changes), while it allowed us to identify specic characteristics of this type of PHR systems important for user acceptance based
on the data (Hsieh and Shannon, 2005; Patton, 2002). Once the
classication scheme was developed, the data were coded by one
author (AAO) and then reviewed independently by the other author
(APG).
Table 5
Participating care provider demographics.
Demographic
information
Physicians
Medical
assistants
Age
<30
30e40
41e50
50
0
2
1
1
1
2
1
0
Gender
Male
Female
3
1
1
3
Years of experience
as a care provider
<1 year
1e5 years
5e10 years
10e15 years
15e20 years
0
1
2
0
1
1
2
1
0
0
Poor (0)
2.3.5.3. Proposed design characteristics. Based on the results obtained, all authors convened and developed proposed PHR design
characteristics.
3. Results
3.1. Participant demographics
The patient sample was gender-balanced (50e50) and mostly in
good or very good health (Table 4). The majority of the participating
patients accessed the Web on a regular basis, mostly from their
homes, with half of them accessing it more than 3 h per week.
Table 5 provides demographic information concerning the care
providers.
3.2. Descriptive statistics on elements of PHRS
Table 6 presents the descriptive statistics for all items in the
PHRS, except demographics questions, which are presented in
Table 4. The internal reliability of the survey was measured through
three pairs of duplicate questions. The Cronbachs Alpha values for
these pairs were 0.90, 0.93, and 0.81, which were acceptable
(Cronbach, 1990). Additionally, Cronbachs Alpha values were
calculated for each of the seven survey sections (Table 3). The coefcients were high, with four constructs having a value higher
than 0.90, and only one construct being slightly below 0.80 (Section
E), providing evidence for adequate reliability (Cronbach, 1990).
The highest scores from each section are discussed next in
descending order.
Patient perceptions regarding the usefulness of the system
(Section A) were mostly positive, with all seven questions scoring
higher than 4.9 on a 7-point Likert scale (1 representing strongly
disagree and 7 representing strongly agree). Patients indicated
that care providers could keep a better eye on their health (Item 4)
with the help of MySafe-T.net. They found the Resources page
(Item 7) and the system overall useful (Item 5), as well as the information on the system relevant (Item 1). Patients perceptions
concerning the usefulness of the Recommendations page (Item 6)
and usefulness of the recommendations provided by the system
(Item 2) were also rated highly. Patients also gave high marks to the
perceived security and trustworthiness of the site (Section B), as the
condence in entering condential information, the perceived
619
Table 6
Descriptive statistics for the rst forty items of the PHRS.
#
Description
Mean
S.D.
Description
Mean
S.D.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Information relevant
Recommendations useful
Learned on preventive screening
Clinicians awareness of my health
Overall usefulness
Recommendations page useful
Resources page useful
Willingness to enter condential info
Trusting the site
Finding site technically reliable
Use site for preventive scr. info
Informative conversation w/clinician
Following preventive screenings
Start/maintain lifestyle changes
More informed on health risks
More in charge of own health
More questions to physician
Consequences of noncompliance
Consequences of lifestyle
Benets of compliance
5.50
5.18
4.91
5.73
5.50
5.36
5.59
5.77
5.95
6.09
5.27
5.41
5.27
4.77
5.36
5.50
3.77
4.73
5.05
5.18
1.16
1.27
1.16
1.05
1.12
1.19
0.94
1.38
1.02
0.95
1.60
1.64
1.48
1.56
1.43
1.37
1.62
1.35
1.46
1.19
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
Benets of lifestyle
Enhanced communication
Ease of use of the site
A lot of mistakes made
Too many steps for some tasks
Easy login
Well-designed screens
Generally interesting site
Not difcult to understand
Understandable instructions
Can work fast on site
Consistent designs
Advise others to use site
Good use of time
Will continue using the site
Satised overall
Stressful to use
Frustrating to use
Mental demand
Effort
5.18
5.32
6.27
2.77
2.45
6.23
5.86
5.82
6.14
5.95
5.73
6.05
5.59
5.50
5.45
5.73
1.77
2.00
4.77
3.77
1.19
1.49
0.91
1.76
1.53
0.85
1.01
0.98
0.92
1.07
1.21
0.93
1.07
1.34
1.41
0.86
1.28
2.34
2.75
2.54
trustworthiness, and perceived technical reliability of my Safe-T.net

(Items 8, 9, 10) received high scores.
Section C focused on the perceived impact of the system on
personal health management. Patients welcomed the potential
help offered by the MySafe-T.net in enabling them to be more in
charge of their own health (Item 16) and in having more informative conversations with their physicians (Item 12). They felt more
informed regarding health risks due to family heritage through
MySafe-T.net (Item 15). Their interest in collecting information
from the system before their next physicians visit and the MySafeT.nets helpfulness in better complying with the recommended
preventive health screenings received more moderate scores (Items
11 and 13). Participants indicated the system helped them have a
better understanding of the consequences of not following a
healthy life style (Item 19). While still positive, scores were lower
regarding the systems helpfulness in starting and maintaining life
style changes (Item 14) and whether the system gave them a better
understanding of the consequences of not getting timely screenings
(Item 18). Results indicated the positive informational and educational characteristics of the system while scores concerning sharing
of this information and making it a more integral part in their interactions were lower. Patients were less willing to make radical
changes in their lifestyles based on what they learned from MySafeT.net (Item 14). Section D included three questions that were duplicates of previous questions for calculating the Cronbachs Alpha
coefcients (Table 3).
As part of Section E, patients found the system easy to use (Item
23) and the screens well-designed (Item 26). They had no difculty
understanding MySafe-T.nets content (Item 29) and found the
designs consistent (Item 32). Overall, perceived user performance
and satisfaction levels with the system were very high. The majority of the patients found screen instructions understandable
(Item 30), screens well-designed (Item 27), and the PHR interesting
(Item 28) and fast to work on (Item 31). Participants mostly did not
think they made a lot of mistakes (Item 24) and there were too
many task steps (Item 25). No signicant usability problems for
MySafe-T.net were detected.
Participants overall satisfaction with the system (Item 36) was
fairly high (about 82% of perfect score). They would recommend the
system to others (Item 33), thought the system was a good use of
their time (Item 34), and wished to continue using it (Item 35).
Mental demand (Item 39, 10- point scale) and overall perceived
effort to complete the tasks (Item 40, 10- point scale) produced
moderate scores, which indicate that the system requires a moderate level of effort.
3.3. Qualitative analysis results
3.3.1. Patients views on the MySafe-T.net system
Positive patient perceptions regarding the system were categorized as information tailored to individual patients, continuity of
care, patient activation and improved communication with care
providers. Negative perceptions and improvement suggestions
included use of medical terminology that cannot be understood by
lay people easily, need for more tailored and individualized information, and difculty in remembering personal and family medical
histories. Details are presented below.
3.3.1.1. Positive perceptions
3.3.1.1.1. Information tailored to individual patients. During the
interviews, 8 patients reported that they found the tailored and
informative nature of the system based on medical and family
history helpful and informative-more informative than generic
sites-as evidenced by the following quote.
I like this system because when you put in all your information,
then it takes you to specic Web sites that pertain to your issues
or disorders. It is more informative than other systems I have
seen because it is more personal. (Patient #4)
3.3.1.1.2. Continuity of care. Five patients indicated that MySafeT.net has the potential to improve patient safety by giving them
more control over their own health information and by functioning
as a reliable repository of preventive health-related information.
They can use the information in MySafe-T.net as they move across
different care systems and care providers, which can result in more
reliable and timely information transfer over the continuum of care.
Patients found it useful that the system can alert their physicians of
possible medicine interactions, prevent duplicate entries, and
provide another medium for communication with their care
providers.
3.3.1.1.3. Patient activation and improved communication with
care providers. Based on the information they learned from
MySafe-T.net, ten patients indicated they have additional questions
for their primary care physicians regarding some tests, and the
system allowed them to easily remember these questions. The
system is a nice place to start the doctors visit on (Participant #9).
620
It made communication with care providers easier, particularly due

to the tailored recommendations and patient education provided,
as exemplied by the following quote.
I should have a certain test done in ve years, but this system
told me, because of my family history, that three years would
probably be more appropriate for me. I want to talk to my doctor
about this. (Patient #17)
Other positive items identied by the patients included MySafeT.net (1) focusing on key information, nuts and bolts, avoiding
unnecessary details; (2) acting as a reminder for things they should
do such as eating better and exercising; (3) being useful because it
allows them to track their health record and see possible health
dangers; (4) being easy to use and self-explanatory.
3.3.1.2. Negative perceptions and improvement suggestions
3.3.1.2.1. More
tailored
and
individualized
information.
Three patients indicated the system could provide more specic
information tailored to patients conditions. For example, two patients had multiple results for a specic test and the system allowed
them to enter only one value:
[.] when I was in the preventive health screening test section,
I was only able to enter the results of my most recent cholesterol.[.] However, my cholesterol was higher couple years ago
but the system did not allow me to enter this information.
(Patient #14)
3.3.1.2.2. Use of medical terminology. While generally patients
indicated the system was easy to use, three patients reported not
understanding one or more of the terms used in the MySafe-T.net.
For example, one patient did not understand the term tubal ligation when completing her medical history, while another had a
hard time with the instructions for entering her medication information. Also, one patient indicated: I know I am allergic to eye
polytron hydroxy, but I did not know which category classied that.
(Patient #8).
3.3.1.2.3. Remembering personal and family medical history.
Two patients had some difculty remembering their own and
family medical histories in enough detail to enter into the system:
The matter is trying to remember what your background is,
what your familys background is. [.] It is a good tool as far as
[.] things you probably should know about and you can always
add them in later. (Patient #20)
Overall, interviews indicated that MySafe-T.net was perceived
by patients as a useful and easy to use system. To improve the
acceptance of such a system, emphasis should be given to providing
tailored information and recommendations to patients, and using a
language that can easily be understood by laypeople.
3.3.2. Prospects for long-term use by patients based on follow-up
interviews at the two-week mark
Of the 16 participants who agreed to participate in the follow-up
phone interview, nine indicated that they used the system in the
two weeks since their physical exam. All nine used it to access information and resources, and two used it to update their proles
and review the recommendations based on their updated proles.
In the interview, participants were presented six statements and
were asked to rate their agreement with those statements on a 7point Likert scale. Participants moderately agreed that using the
system allowed them to have a more informative conversation with
their physicians (Mean 4.1.4, Std. Dev. 1.88).
Three participants indicated that the physician and/or the
medical assistant found the printout potentially useful, with one
medical assistant indicating it would make things easier. Four

patients indicated they found the output useful during their
conversations. One patient and one physician indicated the preventive screening information was useful, and two indicated the
doctors became quite interested in how the printout was produced and asked for more information. One physician read
directly from the care provider letter and discussed each item with
the patient. One patient became interested in the electronic nature of the system and started using the clinics own electronic
charting system after their experience with MySafe-T.net. One
indicated MySafe-T.net should send optional monthly reminder email alerts for relling their medications, updating their information, and going to their upcoming laboratory tests and doctors
appointments.
There were few negative comments from patients in the followup interviews. One patient found the system irrelevant to their
health, one indicated the system was not accurate in its recommendations, and two indicated they were too young or too
healthy to use it. Another one indicated that it could be useful if it
could be shared between different hospitals. In spite of these few
negative comments, all but three patients who participated in the
follow-up interviews stated they would be willing to use the system on a regular basis.
3.3.3. Care providers views on the MySafe-T.net system
Overall, both the participating primary care physicians and the
medical assistants indicated that the system can improve quality
and efciency in primary care clinics. Physicians found several
functions of the system useful, including medical histories of patients and their families, tailored preventive screening recommendations, and medical education provided to patients, as
evidenced by the following statement:
I think that the history part they enter is nice to have. I can see
someone I didnt know walking in with one of these [letter for
the care provider], that would be really helpful. (Physician #3)
Both the physicians and medical assistants found the letter for
the care provider helpful as it provided preventive care screening
recommendations based on each patients condition. Two physicians noted that patients who used the system had more detailed
questions during the examination. Other positive points included
the time-saving aspect of the system, the system being a health
motivator in both motivating patients to learn more about their
health and to keep a healthy lifestyle, and being a limited but good
technology aid. (Physician #3)
Finally something to save me time. [.] I dont have to explain
them everything, I mean everything, [.] with this. (Physician #2)
Maybe they can go look up [the Internet] to help them get
informed and be healthy, good health motivator. (Physician #3)
Regarding how to further improve MySafe-T.net, they recommended including both generic and brand-name medications for
patients to enter, have pop-up boxes providing more information
on conditions, tests and medical terms, and automatically link to
some information on best practices for better patient education.
One physician indicated the system should connect to EMR (electronic medical records) automatically and bi-directionally. One
physician wanted to see the exact screening dates appearing in this
kind of EMR-integrated system.
It would be nice if they could . [be] potentially integrated with
any EMR. So, for both directions, the patient could still maintain
their own record . when they go from place to place. (Physician #2)

Table 7
Overview of the qualitative and quantitative ndings: characteristics of the MySafeT.net system perceived positively by its users and the recommendations for
improvement.
Characteristics desired and available
in MySafe-T.net
Usefulness-related characteristics
Suggestions/reminders for
lifestyle changes
Suggestions/reminders for early
screenings/tests
Informational web links
Information on specic health
conditions
Tracking of health records
Usability-related characteristics
Simple, parsimonious design
Characteristics desired and can be
improved/added in MySafe-T.net
Usefulness-related characteristics
More individualized and tailored
health information for patients
with different illnesses and
education levels
Data entry being less reliant
on ones memory
Usability-related characteristics
Simpler, easy-to-understand
language, clear explanation
of medical terminology
More structured and categorized
patient entries
Perceived security
More emphasis on the information
entered being secure and
condential
Patients
Clinicians
Qualitative
Quantitative
Qualitative
U
U
If it is connected, I could go over [the patients] screening dates

and tell them what [screening] they soon need. (Physician #3)
The suggestions were enhancement-focused and for how to
make [the system] better. (Physician #2). One negative comment
included the system being not for everyone and difcult.
(Physician #1) Table 7 summarizes our recommendations
regarding both the structural and content-related PHR design
characteristics based on this study, along with the information on
which part of the study those recommendations were deduced
from.
3.3.4. Proposed design characteristics
Table 8 presents a set of proposed design characteristics as a
result of our analyses of the qualitative and quantitative data obtained from patients and care providers. It should be noted that the
characteristics on Table 8 are not the result of rigorous testing, as
our study primarily aimed at understanding the usability and
usefulness issues of a PHR system based on a specic case study.
However, the data we collected allowed us to provide some insight
on the design features that are desired by patients and care providers, based on their needs, limitations and motivations. The
recommendations mostly focus on practical issues such as
personalized structure, exibility in the number of patient entries
and interoperability with other systems, as well as possible future
expansions of the system, such as creating push notications (in the
form of e-mails and other tools), and syncing with the patients
electronic medical records. The proposed design features can be
used as a starting point to develop PHR design guidelines produced
from patient and caregiver perspectives.
621
4. Discussion
We examined a particular Web-based PHR system for critical
factors concerning perceived usefulness and usability from both the
patients and providers perspectives in a case study. User perceptions regarding the usefulness of this PHR system were high, with
users showing interest in keeping their health records online and
using them to get educated on health care issues. Patients reported
that they were more aware of the relevant preventive health
screening tests and procedures after using the MySafe-T.net. They
found the tailored recommendations for preventive screening and
lifestyle changes helpful. Furthermore, both patients and care
providers generally found MySafe-T.net a good reminder of health
issues and a relevant information resource. Patients valued the
information that motivates them to get screenings and tests on
time, as well as having their health information recorded electronically and be easily accessible. Patients also pointed out
simplicity as a positive design factor. Furthermore, some patients
indicated that the PHR system contributed to improved communication with their health care providers, allowing them to have
more effective conversations and ask targeted questions. In addition, participating patients and care providers also expressed a
relatively high level of trust in the system with keeping their private information secure and condential. Although patients found
the system useful and usable, they also stated that they would feel
uncomfortable relying solely on the systems recommendations
regarding their preventive health care. Patients unwillingness to
make important health care decisions based on the information
they gathered from the site can be seen as one of the strengths of
the system, as the aim of MySafe-T.net is not to replace but complement primary care visits. Patients should consult their physicians before making critical decisions related to their health care.
Additionally, radical health behavior change is hard to achieve and
in general requires multi-level interventions on multiple behaviors
(Ory et al., 2002). The system is perceived as an informational tool
rather than a tool offering guidelines on health behaviors and
lifestyle choices. This view can guide researchers in developing
better ways to present health information.
In general, care providers found the system promising for
improving preventive care. They indicated that having it as a patients (incomplete) medical information resource is useful, and the
system provided some information they otherwise did not know
about the patients. They saw the letter for the care provider as a
time saver. While patient Internet portals are relatively common
(Weingart et al., 2006), the unique characteristics of this system
included providing more tailored, evidence-based content and a
higher level of interactivity. Overall, patients and care providers
showed substantial interest in adopting MySafe-T.net on a permanent basis.
The study also revealed several areas that can be focused on to
improve the usability and usefulness of such a system. While patients valued the information obtained from the system, they reported that the system would be more valuable with even more
tailored information. Furthermore, several patients found some of
the terminology confusing and unfamiliar, and did not like the
systems high reliance on ones memory. These problems are
difcult to overcome. One potential solution is to integrate the
system with the patients medical records, which is known to be
challenging from nancial, legal and technological perspectives
(Angst and Agarwal, 2009; Bourgeois et al., 2008; Baird et al., 2011).
Although care providers found the letter for the care provider
helpful, some of them also indicated that this feature may be not
suitable for every patient. Furthermore, care providers indicated
that some pieces of important information was lacking in the system such as the information on generic medications. An ideal
622
Table 8
Proposed design characteristics for a PHR system for preventive health care.
Proposed design characteristics based on patient responses
Qualitative results
Provide summary information on the letter to care provider, including highlights of patient history related to preventive
screening
Provide tailored and personalized information based on the medical history and the patients own input
Provide personalized, specic Web site recommendations on the patients issues and disorders
Support interoperability (compatibility) with other systems
Ensure consistency with other systems (not explicitly stated by patients and providers)
Design as a reliable repository of patient health information where the patient is in charge
Design as a virtual place to start the doctors visit
Present key, nuts and bolts information in the form of a practical reminder on eating better and exercising, as well as
keep track of vital statistics such as blood pressure and blood glucose levels
Use simple language, avoid medical terminology as much as possible
Provide exible number of entries by the patient, with entries being non-forced (not required to ll out)
Push important information created by the system to users (e.g., e-mail notications)
Proposed design characteristics based on provider responses

Present warning signs when patient deviates from recommended norms, consciously or not (such as if their eating
habits are bad, they do not do any exercise, or their blood pressure is too high)
Provide automatic, bi-directional syncing with patient EMRs
Design the system as a long-term tool, with one of the primary goals being patient education
Design as a time saver (for both patients and caregivers) and a health motivator
Provide a comprehensive database of medications (including generic medication names), illnesses, health statuses
system would therefore need a more comprehensive database of

medications, illnesses, health statuses and other information types.
Overall, using a multi-method approach and evaluating MySafeT.net from both the direct and indirect users perspectives provided
valuable and in-depth information regarding the positive and
negative design characteristics of such a system. A PHR system can
be useful for both patients and care providers and used by patients
on a regular basis as a reliable information resource and health
education tool. This nding is parallel to the ndings by Denton
(2001) and Pratt et al. (2006) who indicated that a wellpresented electronic environment can increase PHR use and
motivate patients to manage their health information. The ndings
support and build upon the health care literature by determining
the role PHR can play in making patients more active participants in
managing their own health. A usable and reliable Web-based system that presents relevant content on maintaining a healthy lifestyle, improving communication with caregivers, and emphasis on
preventive health screening can be adopted by patients. The key for
high adoption is to design a user-friendly and relevant system that
can provide tailored and comprehensive, yet easy to understand
recommendations to patients based on scientic evidence. We
focused on preventive care due to the literature indicating that
employing PHR may improve patient awareness of preventive care
guidelines and compliance rates with the recommended preventive
screening tests and other related measures. Human factors research
shows that for a technology to positively affect compliance and
performance, one of the rst requirements is for the technology to
be accepted as useful and usable (Karsh et al., 2006; Krist and
Woolf, 2011). A PHR that is perceived as useful and usable, in
turn, may result in improved health outcomes (Kahn et al., 2009). In
this study, however, we did not evaluate the impact of a PHR on
patients long-term behavior change (e.g., increased compliance
with recommended diets, increased medication adherence) and
outcome measures on improvements in preventive care (e.g.,
reduced morbidities). Future research should evaluate such impact.
The methodological contribution of our study mainly lies in the
multi-method approach and the patient-centric design of the
experiment. The quantitative methodologies allowed us to determine patients views of a PHR system with a focus on preventive
screening with regards to well-known constructs that affect technology acceptance (for example, Denton, 2001), while the qualitative methodologies allowed us to determine patients and
caregivers attitudes on the usefulness of a PHR system focused on
U
U
U
U
U
U
U
U
U
U
Quantitative results
U
U
U
U
U
U
U
U
improving preventive care and suggestions on how such systems

can be improved both content- and usability-wise in future designs.
Additionally, there are a limited number of studies in the literature
that evaluated patient-centric HIT so far using a human factors
engineering approach. Our study was aimed at understanding the
perceptions of usability and usefulness of PHR systems based on a
specic system from both the patients and caregivers perspectives. Findings from this case study can help researchers in operationalizing usefulness and usability in future PHR design and
evaluation studies. Human factors research needs to take multiple
users for the same technology/interface into consideration in many
health care settings (Carayon et al., 2010), and qualitative (Gurses
et al., 2009; Karsh et al., 2006) and quantitative (Miller et al.,
2006) methodologies may need to be used together for investigating different user groups perspectives on such systems.
Experiences with many of the technologies introduced into
health care, especially information technologies, have been mediocre at best (Karsh et al., 2010). Most of the health information
technologies have been developed under the assumption of a linear
and predictable nature of work in health care (Gurses et al., 2011).
However, health care is, by its nature, team-based, non-linear,
event-driven, complex, and full of deviations. Given the complex
and safety-critical nature of health care, more systematic and
science-based approaches (Leveson, 2001) that build upon strong
partnerships among product developers, human factors and usability engineers, clinicians, patients, and regulators are needed.
With the move from paper to electronic record environments, this
type of partnership is a must, not an option to improve quality and
safety of health care.
4.1. Limitations
This study had several limitations. First, we used a particular PHR
as a case study, and our ndings may not be generalizable to other
PHR systems. A wide variety of Web-based PHR systems are
currently available that may vary in the features they offer. Our study
focused on the usability and usefulness issues involving a specic
PHR. Due to the fragmented nature of currently available PHR designs, we did not fully assess how similar MySafe-T.nets features are
to those currently available in other PHR systems. Our study also did
not evaluate in-situ or long-term PHR usability or perceptions of
usefulness. For assessing more general perceptions of health consumers regarding PHR systems, different methods (e.g., a large-scale
survey of health care consumers who have used PHRs) will need to
be used with a larger sample size. Second, due to the exploratory
nature of our study, we did not evaluate the PHR system after it has
been used on a long-term basis by patients and care providers.
Rather, participants indicated their opinions concerning the system
after a structured examination of it (with a follow-up interview). It is
possible that participants may have additional and/or different
opinions after using the system on a long-term basis.
Third, our sample size may be viewed as relatively small for
collecting data via a questionnaire. However, we considered a
sample size of 22 as adequate due to the exploratory nature of the
study that focused on determining current general trends and attitudes toward a PHR system aimed at improving preventive care.
Additionally, we contacted thirty-six consecutive patients who
called the clinic for an annual physical exam appointment, resulting
in a fairly high response rate of 61%. The survey helped determine
the potential usefulness and usability issues concerning the system,
and complemented the data obtained through qualitative methods.
It should be noted, however, that a higher number of participants
that have a broader spectrum including different ailment severities
and types, as well as different age and education levels, could have
helped in identifying new design characteristics or rening the
existing ones for the Web-based PHR systems to make them
appealing to a broader population. With this sample size, participant diversity may not have been adequately captured to further
generalize our ndings. Also, there is a body of work focusing on
PHR adoption differences due to culture (Hartley, 2004; Fuji et al.,
2008; Urowitz et al., 2008; Kahn et al., 2009). Our study did not
consider the role of culture in PHR adoption. Additionally, while our
case study was based on a PHR system focused on preventive
health, we did not measure the impact of PHR on health outcomes.
Finally, 16 out of the 22 patients invited participated in the followup interviews, and it is possible that those who did not participate
in the follow-up may have different views about the system.
623
4.2. Future work

While the PHR system was found useful, patients indicated they
prefer an even more tailored system. The recommendations and
resources are useful, but sometimes rely on external links. An
agent-based approach, where intelligent programs would do
searches for relevant information in similar patterns to humans
(Luck and DInverno, 2001), may increase the accuracy of the recommendations and resources, and the information the site provides can be enhanced with more comprehensive databases that
can be integrated into the system. Moreover, a future system can
interface with the provider-based electronic health records of the
patients and combine the two information sources. PHR systems
may also expand to mobile environments, and the concept of
technology acceptance may go through some changes while being
adapted to health care environments (Holden and Karsh, 2010).
Acknowledgments
This study was funded by the Medical Industry Leadership
Institute of the University of Minnesota. Dr. Ayse P. Gurses was
supported in part by the Agency for Healthcare Research and
Quality K01 grant #HS018762. We would like to thank Dr. David
Moen and the participating primary care clinic care providers for
their support of the study. Supported by Dept. of Veterans Affairs,
Veterans Health Administration, Health Services Research and
Development Service (CDA 11-201/CDP 12-255). The views
expressed in this article are those of the authors and do not
necessarily reect the position or policy of the Department of
Veterans Affairs or the United States government.
Appendix A. Sample screen shots from the MySafe-T.net

system
A1. The past medical history screen.
624
A2. The cholesterol data entry screen.
A3. The family history screen.
A4. Updated screening recommendations based on risk factors.
A5. Link to the relevant web site is provided.
625
626
A6. Lifestyle recommendations.
Appendix B. Patient interview guide

(Usability questions are marked with [1] and usefulness questions are marked with [2].)
1 Do you nd the system easy to use? Why or why not? [1]
2 Would you use this system to learn more about and get
recommendations on preventive health screening? [2]
3 If you had control over the design of a perfect computer tool
to help you with your health care management, specically
related to preventive health screening and lifestyle choices,
what would it be like? What would be on it and why? What
issues are important to you that you would like the developers of these tools to consider? [1, 2]
5 You seemed to like ... [noted during session]... what did you
like about it? [1,2]
6 What were the major positive aspects of the system? [1,2]
7 What were the major difculties you had with this system?
[1,2]
8 In your opinion, how can this system be improved? [1,2]
9 If this system were available to you at home, would you use it
on a regular basis? Why or why not? [1,2]
10 Do you plan on asking your physician any questions based on
what you have learned for this web site? Yes No If yes what
are you planning to ask? [2]
11 What other suggestions do you have to improve this system?
[1,2]
Appendix C. Caregiver focus group guide
(Usability questions are marked with [1] and usefulness questions are marked with [2].)
- What are your thoughts about this Web site? Would you
recommend it to your patients? To your colleagues? Why or why
not? [2]
- Do you have any concerns about this Web site? If so, what? [1,2]
- How do you think this Web site can affect (1) patient activation
(2) patient-physician communication, (3) shared decisionmaking, (4) compliance with preventive screening, (5) changes
in lifestyle choices (e.g., regular exercise, healthy diet, smoking
cessation)? [2]
- If you had control over the design of a perfect information
tool for preventive health screening, what would it be
like? What would be on it and why? What issues are
important to you that you would like the developer to
consider? [1,2]
- What are your thoughts about the letter to the doctor functionality of the system? How do you think this letter can help
you? How would you like to modify it? [2]
- What is the process for informing patients about preventive
health screening? [2]
- What do you do to improve the lifestyle choices of the patients?
Do you think this Web site can help you? How? [2]
- If you had all the resources you needed, how would you
improve your current methods to discuss preventive
screening? [2]
- What would be most helpful for you at your practice to improve
your ability to introduce and discuss preventive health
screening with patients? [1,2]
- What would be most helpful for you at your practice to improve
your ability to introduce and discuss lifestyle choices with your
patients? [1,2]
- Do you feel it is feasible for your patients to use this Web site?
[1,2]
References
Angst, C.M., Agarwal, R., 2009. Adoption of electronic health records in the presence
of privacy concerns: the elaboration likelihood model and individual persuasion. MIS Quarter. 33 (2), 339e370.
Avorn, J., Shrank, W.H., 2008. A substantial cause of preventable illneess. Biomed. J.
335 (7650) (published online April 8, 2008).
Baird, A., North, F., Raghu, T.S., 2011. Personal health records (PHR) and the future of
the physicianepatient relationship. In: Proceedings of the 2011 ACM iConference, Seattle, WA, USA, February 8e11, 2011, pp. 281e288.
Ball, J., Smith, C., Bakalr, R., 2006. Personal health records: empowering consumers.
J. Healthc. Inform. Manage. 21 (1), 76e86.
Bodenheimer, T., Lorig, K., Holman, H., Grumbach, K., 2002. Patient selfmanagement of chronic disease in primary care. J. Am. Med. Assoc. 288 (19),
2469e2475.
Bourgeois, F., Taylor, P., Emans, S., Nigrin, D., Mandl, K., 2008. Whose personal
control? Creating private, personally controlled health records for pediatric and
adolescent patients. J. Am. Med. Inform. Assoc. 15 (6), 737e743.
Brennan, T.A., 2000. The Institute of Medicine report on medical errors e could it do
harm? (Sounding Board). N. Engl. J. Med. 324 (15).
Britto, M., Wimberg, J., 2009. Pediatric personal health records: current trends and
key challenges. Pediatrics 123 (Suppl. 2), S97eS99.
Buckle, P., Clarkson, P.J., Coleman, R., Ward, J., Anderson, J., 2006. Patient safety,
systems design and ergonomics. Appl. Ergon. 37 (4), 491e500.
Carayon, P., 2010. Human factors in patient safety as an innovation. Appl. Ergon. 41
(5), 657e665.
Carayon, P., Karsh, B., Cartmill, R., et al., July 2010. Incorporating Health IT into Workow
Redesign: Request for Information Summary Report. AHRQ publication no. 100074-EF. The Agency for Healthcare Research and Quality, Rockville, MD.
Cronbach, L., 1990. Essentials of Psychological Testing. Harper & Row Publishing,
New York, NY.
Davis, F., 1989. Perceived usefulness, perceived ease of use, and user acceptance of
information technology. MIS Quarter. 13 (3), 319e340.
Davis, F., Bagozzi, R., Warshaw, P., 1989. User acceptance of computer technology: a
comparison of two theoretical models. Manage. Sci. 35, 982e1003.
Denton, I.C., 2001. Will patients use electronic personal health records? Responses
from a real-life literature. J. Healthc. Inform. Manage. 15 (3), 251e259. Fall 2001.
Eriksen, H., Ursin, H., 2004. Subjective health complaints, sensitization, and sustained cognitive activation (stress). J. Psychom. Res. 56 (4), 445e448.
Fuji, K.T., Galt, K.A., Serocca, A.B., 2008. Personal health record use by patients as
perceived by ambulatory care physicians in Nebraska and South Dakota: a
cross-sectional study. Perspect. Health Inform. Manage./AHIMA, Am. Health
Inform. Manage. Assoc. 5.
Gefen, D., 2000. E-commerce: the role of familiarity and trust. Omega 28, 725e737.
Goth, G., 2008. Power to the patients. IEEE Distrib. Syst. Online 9, 1e4.
Greenhalgh, T., Hinder, S., Stramer, K., Bratan, T., Russell, J., 2010. Adoption, nonadoption, and abandonment of a personal electronic health record: case
study of HealthSpace. Br. Med. J. 341, c5814.
Gurses, A., Xiao, Y., Hu, P., 2009. User-designed information tools to support
communication and care coordination in a trauma hospital. J. Biomed. Inform.
42 (4), 667e677.
Gurses, A., Ozok, A., Pronovost, P., 2011. Time to accelerate integration of human
factors and ergonomics in patient safety. BMJ Qual. Saf. 21 (4), 347e351.
Hackman, J., Oldham, G., 1980. Work Redesign (Organization Development). Prentice Hall, Upper Saddle River, NJ.
Hargreaves, J., 2010. Will electronic personal health records benet providers and
patients in rural America? Telemed. e-Health 16 (2), 167e176.
Hartley, D., 2004. Rural health disparities, population health, and rural culture. Am.
J. Pub. Health 94 (10), 1675e1678.
Hibbard, J.H., Stockard, J., Mahoney, E.R., Tusler, M., 2004. Development of the patient activation measure (PAM): conceptualizing and measuring activation in
patients and consumers. Health Serv. Res. 39 (4p1), 1005e1026.
Hibbard, J.H., Mahoney, E.R., Stockard, J., Tusler, M., 2005. Development and testing
of a short form of the patient activation measure. Health Serv. Res. 40 (6), 1918e
1930.
Holden, R.J., Karsh, B.-T., 2010. The technology acceptance model: its past and its
future in health care. J. Biomed. Inform. 43, 159e172.
Hsieh, H., Shannon, S., 2005. Three approaches to qualitative content analysis. Qual.
Health Res. 15, 1277e1288.
Human Performance Research Group, 1997. NASA Task Load Index. NASA Ames
Research Center, Moffett Field, CA.
International Standards Organization (ISO), 2009. Personal Health Records: Denition, Scope and Context. Draft Technical Report. Geneva, 2009.
Johnson, R., Onwuegbuzie, A., Turner, L., 2007. Toward a denition of mixed
methods research. J. Mixed Methods Res. 1 (2), 112e133.
Kahn, J.S., Aulakh, V., Bosworth, A., 2009. What it takes: characteristics of the ideal
personal health record. Health Affairs 28 (2), 369e376.
Karsh, B., Hamilton Escoto, K., Beasley, J., Holden, R., 2006. Towards a theoretical to
medical error reporting system research and design. Appl. Ergon. 37 (3), 283e
295.
Karsh, B.T., Weinger, M.B., Abbott, P.A., Wears, R.L., 2010. Health information technology: fallacies and sober realities. J. Am. Med. Inform. Assoc. 17 (6), 617e623.
Krist, H., Woolf, S., 2011. A vision for patient-centered health information systems.
J. Am. Med. Assoc. 305 (3), 300e301.
627
Krist, A., Peele, E., Woolf, S., Rothemich, S., Loomis, J., Longo, D., Kuzel, A., 2011.
Designing a patient-centered personal health record to promote preventive
care. BMC Med. Inform. Decis. Mak. 11 (73), 1e11.
Leveson, N., 2001. Safeware: System Safety and Computers. Addison-Wesley,
Boston.
Lin, H., Choong, Y., Salvendy, G., 1997. A proposed index of usability: a method for
comparing the relative usability of different software systems. Behav. Inform.
Technol. 16 (4e5), 267e277.
Liu, L.S., Shih, P.C., Hayes, G.R., 2011. Barriers to the adoption and use of personal
health record systems. In: Proceedings of the 2011 ACM iConference, Seattle,
WA, USA, February 8e11, 2011, pp. 363e370.
Lorig, K.R., Sobel, D.S., Stewart, A.L., et al., 1996. Evidence suggesting that a chronic
disease self-management program can improve health status while reducing
hospitalization: a randomized trial. Med. Care 37 (1), 5e14.
Luck, M., DInverno, M., 2001. A conceptual framework for agent denition and
development. Comput. J. 44 (1), 1e20.
Malamateniou, V.K., Vassilacopoulos, G., 2010. An agent-based application of personal health record in homecare. In: ACM Proceedings of the 3rd International
Conference on Pervasive Technologies Related to Assistive Environments, June
23e25, 2010, Samos, Greece.
Manne, S., Markowitz, A., Winawer, S., Meropol, N., Haller, D., Rakowski, W., Babb, J.,
Jandorf, L., Jan. 2002. Correlates of colorectal cancer screening compliance and
stage of adoption among siblings of individuals with early onset colorectal
cancer. Health Psychol. 21 (1), 3e15.
Martin, J.L., Clarke, D.L., Morgan, S.P., Crowe, J.A., Murphy, E., 2011. A user-centered
approach to requirements elicitation in medical device development: a case
study from an industry perspective. Appl. Ergon. 43 (1), 184e190.
McGlynn, E., Asch, S., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., Kerr, A., 2003.
The quality of health care delivered to adults in the United States. N. Engl. J.
Med. 348, 2635e2645.
Miller, A., Engt, C., Tate, R., Yassi, A., 2006. Evaluation and effectiveness of portable
ceiling lifts in a new long-term care facility. Appl. Ergon. 37 (3), 377e385.
Montague, E., 2010. Validation of a trust in medical technology instrument. Appl.
Ergon. 41 (6), 812e821.
Morbidity and Mortality Weekly Report, Cancer Screening United States, 2010,
January 27, 2012. Centers for Disease Control and Prevention, pp. 41e45, 61(3).
Accessed online at: www.cdc.gov/mmwr/preview/mmwrhtml/mm6103a1.htm?
s_cidmm6103a1_w (accessed 21.06.12).
Mosca, L., Mochari, H., Christian, A., Berra, K., Taubert, K., Mills, T., Burdick, K.A.,
Simpson, S.L., 2006. National study of womens awareness, preventive action,
and barriers to cardiovascular health. Circulation 113, 525e534.
Nazi, K.M., Hogan, T.P., et al., 2009. Embracing a health services research perspective
on personal health records: lessons learned from the VA M HealtheVet system.
J. General Int. Med. 25 (Suppl. 1), 62e67.
Nielsen, J., 1993. Usability Engineering. Morgan-Kauffman, San Francisco, CA.
Ory, M.G., Jordan, P.J., Bazzarre, T., 2002. The Behavior Change Consortium: setting
the stage for a new century of health behavior-change research. Health Educ.
Res. 17.5, 500e511.
Palen, L., Aalokke, S., 2006. Of pill boxes and piano benches: home-made methods
for managing medication. In: Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, pp. 79e88.
Patton, M.Q., 2002. Qualitative Research and Methods, third ed. Sage Publications,
Thousand Oaks.
Perlin, J., et al., 2004. The Veterans health Administration: quality value, accountability, and information as transforming strategies for patient-centered care.
Am. J. Manage. Care 10, 828e836.
Peterson, A.M., Dragon, C.J., 1998. Improving medication adherence in patients
receiving home health. Home Healthc. Consult. 5 (9), 25e27.
Powsner, S.M., 1998. Opportunities for and challenges of computerization. Lancet
352 (9140), 1617e1622.
Pratt, W., Unruh, K., Civan, A., Skeels, M., 2006. Personal health information management. Commun. ACM Assoc. Comput. Machine. 49 (1), 51e55.
Salkeld, G., 2006. What are the benets of preventive health care. Healthc. Anal. 6
(2), 106e112.
Scherger, J.E., 2005. Primary care needs a new model of ofce practice. Bio-Med. J.
330 (7504), E358eE359.
Shea, S., DuMouchel, W., Bohamonde, L., 1996. A meta-analysis of 16 randomized
controlled trials to evaluate computer-based clinical reminder systems for
preventive care in the ambulatory setting. J. Am. Med. Inform. Assoc. 3 (6),
399e409.
Shneiderman, B., 1992. Designing the User Interface: Strategies for Effective HumaneComputer Interaction. Addison-Wesley, New York, NY.
Srinivasan, A., 2006. Keeping online personal records private: security and privacy
considerations for Web-based PHR systems. J. Am. Health Inform. Manage.
Assoc. 77 (1), 62e68.
Steele, R., Lo, A., 2009. Future personal health records as a Foundation for
Computational Health. Lecture Notes Comput. Sci. 5593, 719e733.
Sunyaev, A., Chornyi, D., Mauro, C., 2010. Evaluation framework for personal health
records: Microsoft HealthVault vs. Google Health. In: 43rd Hawaii International
Conference on System Sciences (HICSS), 5e8 Jan. 2010, Honolulu, HI, pp. 1e10.
Tang, P.C., Lansky, D., 2005. The missing link: bridging the patient-provider health
information gap. Health Affairs 24 (5), 1290e1295.
Tang, P.C., Ash, J.S., Bates, D.W., Overhage, J.M., Sands, D.Z., 2006. Personal health
records: denitions, benets, and strategies for overcoming barriers to adoption. J. Am. Med. Inform. Assoc. 13 (2), 121e126.
628
The Markle Foundation, 2003. Connecting for Health. The personal health working
group nal report, July 1, 2003.
U. S. Department of Health and Human Services, 2001. Healthy People 2010. Understanding and Improving Health. Government Printing Ofce, Washington,
DC: U.S.
Ueckert, F., Goerz, M., Ataian, M., Tessmann, S., Prokosch, H.-U., 2003. Empowerment of patients and communication with health care professionals through an
electronic health record. Int. J. Am. Med. Inform. Assoc. 70 (2), 99e108.
Urowitz, S., et al., 2008. Is Canada ready for patient accessible electronic health
records? A national scan. BMC Med. Inform. Decis. Mak. 8 (1), 33.
U.S. Preventive Services Task Force, 2007. Accessed online at: http://www.ahrq.gov/
CLINIC/uspstx.htm (accessed 18.06.08).
Van den Haak, M., De Jong, M., Schellens, P., 2003. Retrospective vs. concurrent
thinkaloud protocols: testing the usability of an online library catalogue.
Behav. Inform. Technol. 22 (5), 339e351.
Venkatesh, V., Davis, F., 2000. A theoretical extension of the technology acceptance
model: four longitudinal eld studies. Manage. Sci. 46 (2), 186e204.
Von Korff, M., Moore, J.E., Lorig, K., et al., 1998. A randomized trial of a lay personled self-management group intervention for back pain patients in primary care.
Spine 23 (23), 2608e2615.
Walker, J., Carayon, P., 2009. From tasks to processes: the case for changing health
information technology to improve health care. Health Affairs 28 (2), 467e477.
Weinberg, D.S., Turner, B.J., Wang, H., Myers, R.E., Miller, S., June 2004. A survey of
women regarding factors affecting colorectal cancer screening compliance.
Prevent. Med. 38 (6), 669e675.
Weingart, S., Rind, D., Toas, Z., Sands, D., 2006. Who uses the patient Internet
portal? The PatientSite experience. J. Am. Med. Inform. Assoc. 13, 91e95.
Zuckerman, A.E., Kim, G.R., 2009. Personal health records. Health Inform. 4, 293e
301.

Usability and Perceived Usefulness of Personal Health Records For Preventive Health Care - A Case Study

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Usability and Perceived Usefulness of Personal Health Records For Preventive Health Care - A Case Study

Transféré par

Droits d'auteur :

Formats disponibles

Applied Ergonomics 45 (2014) 613e628

Contents lists available at ScienceDirect

Usability and perceived usefulness of personal health records for

seminal paper of McGlynn and colleagues indicated that patients

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

children, elderly, or terminally disabled patients as well

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

lifestyle, and health habits (Appendices A1, A2 and A3). It develops

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

Concurrent thinkaloud method

PHR usability and

caregiver focus groups. Table 1 summarizes the data collection

2.3.1. Data collection from patients

Users were instructed to:

Users were informed that:

Task 1: Entering Demographic Information

- Enter demographic information

Task 2: Entering Personal Health History

- Enter medication allergies and intolerances using

- They can always modify their demographic

Task 3: Entering Lifestyle and Health Habits

Task 4: Entering Family History

Task 7: Printing the Letter for the Care

- Enter information regarding the medical history of

- Keep the printed letter and hand it to the medical

- Health habits categories include smoking, alcohol use,

- Recommendations are tailored based on the

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

Measures used/adapted from

Trust and security

Impact on personal health

Denton (2001) and Pratt

Denton (2001) and Pratt

Benets (Duplicate questions

Any other comments/suggestions?

Relevance of information received from the site

(Hart and Staveland,

Survey Cronbachs Alpha Values: 0.90, 0.93, 0.81.

based survey aimed at determining patient perceptions regarding

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

Highest education level

8th grade or less

Computer usage frequency

Internet usage frequency

Types of health sites visited

Places the Internet is visited

Very good (32%)

Rating of Ones General Health

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

trustworthiness, and perceived technical reliability of my Safe-T.net

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

It made communication with care providers easier, particularly due

medical assistant indicating it would make things easier. Four

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

If it is connected, I could go over [the patients] screening dates

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

Proposed design characteristics based on provider responses

system would therefore need a more comprehensive database of

improving preventive care and suggestions on how such systems

A. Ant Ozok et al. / Applied Ergonomics 45 (2014) 613e628

4.2. Future work

Appendix A. Sample screen shots from the MySafe-T.net

A1. The past medical history screen.