Copyright eContent Management Pty Ltd. International Journal of Multiple Research Approaches (2011) 5: 5263.

Beyond the ceiling effect: Using

a mixed methods approach to
measure patient satisfaction
SHARON ANDREW
Professor of Nursing, Department of Acute Care, Anglia Ruskin University, Chelmsford, Essex, UK;
Family and Community Health Research Group FaCH, Penrith, NSW, Australia

YENNA SALAMONSON
School of Nursing and Midwifery, University of Western Sydney; Family and Community Health
Research Group FaCH, Penrith, NSW, Australia

BRONWYN EVERETT
Faculty of Nursing, Midwifery & Health, University of Technology Sydney, Lindfield, NSW,
Australia; Family and Community Health Research Group FaCH, Penrith, NSW, Australia

ELIZABETH J HALCOMB
School of Nursing and Midwifery, University of Western Sydney; Family and Community Health
Research Group FaCH, Penrith, NSW, Australia

PATRICIA M DAVIDSON
Centre of Cardiovascular and Chronic Care, School of Nursing and Midwifery, University of
Technology Sydney and Curtin University, Sydney, NSW, Australia

ABSTRACT
This study reports patient satisfaction with the nursing care on a neurosurgical critical care unit. A
modified version of the Ludwig-Beymer Patient Satisfaction Questionnaire (PSQ-7) was used, and
included structured items and semi-structured interview questions. Data were collected from 149
patients. Participants rated their satisfaction as high (Mean = 25.14; SD = 2.96). The distribution of scores was skewed and every item demonstrated a ceiling effect. Principal component analysis
yielded a one-component solution which explained 48% of the variance.
NVivo was used to match PSQ-7 scores with qualitative data. Participants comments
were categorised as positive, negative or yes, but. Just over half the patients made positive
comments and 29% of patients in the low group made negative comments. Three categories: communication, nursing care delivery and the hospital environment emerged from the
qualitative data.
A mixed method approach enables researchers to move beyond the ceiling effect of quantitative measures of patient satisfaction and to gain a more meaningful explanation of patient
satisfaction.
Keywords: patient satisfaction, mixed methods, communication, patient safety, quality assurance

52

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

Volume 5, Issue 1, April 2011

Mixed methods approach to measure patient satisfaction

INTRODUCTION

s the consumer movement continues to

gain momentum, the patient perspective on
the quality of health care is increasingly being
recognised as an important factor in health reform
(Cheraghi-Sohi et al., 2008; Potiriadis et al., 2008;
Rowell & Polipnick, 2008). In order to respond
to patient preferences we need to understand their
levels of satisfaction with current service provision (Cheraghi-Sohi et al., 2006, 2008). Despite
the multitude of patient satisfaction instruments,
a consistent limitation of these scales has been the
ceiling effect, as patients tend to respond positively (Bradley, Plowright, Stewart, Valentine, &
Witthaus, 2007; Cappelleri, Gerber, Kourides,
& Gelfand, 2000). Hence, the sole reliance on
evaluating patient satisfaction using only quantitative measures may not be adequate as it may lack
sensitivity in determining variability or changes
in patient-reported quality of care. Importantly,
a purely quantitative approach does not provide
direction for quality improvement which is a key
reason for measuring patient satisfaction. Therefore
a mixed method approach combining quantitative
and qualitative data in the one study may assist in
better elucidating patients ratings of satisfaction
and identifying areas for practice changes.

BACKGROUND
Consumer satisfaction
Patient satisfaction is frequently used as an outcome measure when assessing the quality of
nursing or medical care. Understanding patients
satisfaction with health care delivery is important
in ensuring models of care are both acceptable and
appropriate within the particular context in which
they are being delivered (Redsell, Stokes, Jackson,
Hastings, & Baker, 2007). When patients are satisfied they are more likely to be compliant and
potentially experience improved clinical outcomes
(Donovan, 1995; Grogan, Conner, Norman,
Willits, & Porter, 2000; Hardy, West, & Hill,
1996). Additionally, those patients who are more
satisfied with their care are more likely to stay with

existing, rather than seeking alternative service

providers (Grogan et al., 2000; Hardy et al., 1996).
Whilst various definitions exist in the literature, in this study, satisfaction is defined as the
congruence between patients expectations of ideal
nursing care and their perception of the care they
actually receive (Merkouris, Papathanassoglou,
& Lemonidou, 2004). Numerous quantitative
patient satisfaction measures exist in the literature, for both general use and for specific areas of
practice (Greco, Powell, & Sweeney, 2003; Mead,
Bower, & Roland, 2008; Trout, Magnusson, &
Hedges, 2000). These instruments have been
criticised for their lack of psychometric testing
(Bradley, 1999; Cappelleri et al., 2000; Pouwer,
Snoek, & Heine, 1998). In addition, a ceiling effect is often observed in satisfaction scales
(Cappelleri et al., 2000) as patients consistently
tend to score their care in the mid-to-high range
for most items. As a result it is difficult for
researchers to use the data for comparative studies
and align with conceptually congruent constructs.
Mixed methods
Structured quantitative surveys are the more commonly used method for determining patient satisfaction with their care although there are those
who question their use:
Moreover it is unlikely that quantitative measures can be sophisticated enough to capture the
specific, often personal, issues that are important to people. (Dawood & Gallin, 2010, p. 27)
Some researchers, however, have used qualitative measures to understand patient satisfaction
(Dawood & Gallin, 2010; Hunt, 1999) or utilised a
mixed methods approach (Merkouris et al., 2004).
Mixed methods research involves the use of
qualitative and quantitative data in a single study
(Creswell & Plano Clark, 2007; Halcomb, Andrew,
& Brannen, 2009). Mixed methods designs have
been increasingly used in health research (Andrew
& Halcomb, 2006) and more specifically when
determining patient satisfaction (Merkouris
et al., 2004). Merkouris et al. (2004) conducted

Volume 5, Issue 1, April 2011 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

53

Sharon Andrew et al.

qualitative interviews concurrently with a quantitative survey and triangulated the results finding
that this gave a more complete understanding of
patient satisfaction than would have been obtained
by using survey or interview in isolation.
Completeness, holism, or comprehensiveness
are similar concepts and have been proposed as
reasons for conducting mixed methods research
(Andrew & Halcomb, 2006; Creswell & Plano
Clark, 2007; Greene, Caracelli, & Graham,
1989). There are many mixed methods research
designs (Creswell & Plano Clark, 2007; Kroll &
Neri, 2009). This paper reports the benefits gained
by using mixed methods data, in the context of an
action research project, to understand the ceiling
effects when measuring patient satisfaction.

METHODS
This study was part of a larger project using a
mixed methods action research approach. The
project monitored the impact on patient and
nurse satisfaction following a change in the model
of care on a neurosurgical critical care unit in a

large metropolitan hospital in New South Wales,

Australia. The study included data collection
prior to and following the change in model of
care. Preliminary statistical and qualitative analysis found no difference in the prepost data sets
and hence they are aggregated in this paper. The
qualitative and quantitative data were collected
concurrently and results were analysed separately
and then integrated using NVivo.
The aim of this paper is to present the patient
satisfaction findings including the quantitative
and qualitative data and to discuss how the integration of these data sets, using a mixed methods
approach, can elucidate a deeper understanding
of the concept of patient satisfaction.
Satisfaction was measured by a modified version (PSQ-7) of the Ludwig-Beymer et al. (1993)
patient satisfaction questionnaire (PSQ). The
PSQ contains two yes/no items about the hospital
which were not included in this study and seven
items (PSQ-7) measuring patients responses
to perceived care provided by nursing staff (see
Table 1), with responses ranging on a Likert scale

TABLE 1: PSQ-7 ITEMS SUMMARY DATA, FLOOR AND CEILING EFFECTS (N = 147)

54

Number

Patient satisfaction item

Mean (SD; range)

Floor effects
(% minimum
score of 0)

Ceiling effects
(% maximum
score of 100)

The nurses on this unit

are caring and attentive.

3.72 (0.56; 14)

78

The nursing staff responds to

your requests within a
reasonable length of time.

3.54 (0.66; 14)

63

The nurses are knowledgeable

about the care you require.

3.60 (0.62; 14)

67

The nurses provide sufficient

explanations about medications,
procedures, and routines.

3.59 (0.64; 14)

67

The nursing staff is courteous

to your family and visitors.

3.78 (0.46; 14)

80

Upon arrival to this unit, the

orientation and attention
you received were:

3.50 (0.68; 14)

59

Overall, the nursing care

you are receiving is:

3.65 (0.53; 14)

68

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

Volume 5, Issue 1, April 2011

Mixed methods approach to measure patient satisfaction

from 1 never to 4 always for the first five items

(Ludwig-Beymer et al., 1993), and from 1 poor to
4 excellent for the last two items. Items included,
for example: The nursing staff respond to your
requests within a reasonable length of time. The
questionnaire also included two open-ended items
where patients could comment about (nursing)
staff they found especially positive or negative
during their hospital stay or add any comments
they considered relevant to their (nursing) care.
While the PSQ-7 can be a self-report measure,
recognising the nature of the seriousness of the
health status of the patients involved in the study,
patients were first assessed for suitability by the
nurse unit manager. The questionnaire was then
administered by a nurse research assistant, trained
in the study protocol, who was independent of the
unit where the study was undertaken. The questions were read to patients who could indicate
their response on a laminated board and their
qualitative answers were transcribed verbatim by
the research assistant. Again, due to the patients
health status, the interviews were short including
the qualitative components which tended to comprise short statements in response to the questions. Demographic data collected included age,
gender, country of birth and language spoken at
home, while clinical data included length of stay
in the unit, reason for admission (elective or emergency), type and quantity of analgesia. Patients
informed consent was sought for participation in
the study. Patients who were physically unstable
or whose cognitive condition was impaired were
excluded from the study. Ethical approval for the
study was obtained from the relevant university
and health authority.

the KolmogorovSmirnov test, we explored the

PSQ-7 for normality of distributions. We used
a 30% cut-off (Kane, 2006) for acceptable floor
and ceiling effect. Descriptive statistics, principal
component analysis (PCA) and Cronbachs alpha
reliability were used to evaluate the PSQ-7. We
used the KaiserMeyerOlkin (KMO) measure
of 0.6 as the reference point to determine if the
correlations were adequate for conducting PCA
(Tabachnick & Fidell, 2007). To determine the
number of components to extract, the Kaisers criterion to retain components with eigenvalues >1
was used. Items that loaded more than 0.3 were
considered as significant factor loading. We computed Cronbachs alpha for PSQ-7, with a cut-off
of 0.7 used for acceptable reliability coefficients
(Nunnally & Bernstein, 1994). An item analysis
on how well each individual item related to other
items in the PSQ-7 was assessed using corrected
item-total correlation and Cronbachs alpha if
item deleted (Streiner & Norman, 2008). Itemtotal correlation values below 0.3 were considered
unacceptably low and not contributing sufficiently to the internal consistency of the scale and
subscale and hence deleted.
The qualitative data was imported into
NVivo, coded and grouped into small and then
larger related categories (Coffey & Atkinson,
1996). Attributes such as patients PSQ-7 scores
were then imported into NVivo for integration
with patients qualitative data. For comparison
of positive, negative, positive plus a qualifier
(termed yes, but) textual comments provided by
patients about their satisfaction with nursing care,
the PSQ-7 scores were grouped into tertiles (see
Figure 1).

Data analysis
Demographic and PSQ-7 data were analysed
using SPSS Version 17.0. We calculated the central tendency and distribution of scores, as well
as floor and ceiling effects for each item of the
PSQ-7. An aggregate score of the PSQ-7 was
calculated by summing all seven items giving
a potential score ranging from 7 to 28. Using

RESULTS
Demographic and clinical
characteristics
A total of 147 patients gave their consent to participate in the study. The mean age of the participants was 53.1 years (SD: 18.5, Range: 1688)
and 52% were male. Although just over one-third

Volume 5, Issue 1, April 2011 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

55

Sharon Andrew et al.

30

27.9

PSQ-7 scores
Mean: 25.14 (SD: 2.96, Range: 13-28)

25

PSQ-7 terle
low

Percent

20

up to 24

medium

25 to 27

high

more than 27
15.0

15

12.2

10

10.2

7.5
4.8 4.8

10.9

10

11

12

0.7 0.7 0.7

13

14

17

1.4 1.4

18

19

2.0

20

21

22

23

24

25

26

27

28

Total PSQ-7 Score

FIGURE 1: DISTRIBUTION OF PSQ-7 SCORES (N = 147)

(n = 57, 39%) of participants were born overseas,

the majority of participants (n = 123, 84%) spoke
only English at home. Sixty-four percent of the
participants were emergency admissions (n = 94),
and the average length of stay in the unit was
3.6 days (SD: 3.9). Qualitative responses were
obtained from 136 patients.
Floor and ceiling effects
The mean score of the PSQ-7 in this study sample was 25.14 (SD: 2.96, Range: 1328), with a
median of 26.00 and a highly skewed distribution (skewness: 1.35, kurtosis: 2.30). Applying
the 30% cut-off for floor and ceiling effect,
every item on the PSQ demonstrated a ceiling
effect with 5980% of participants selecting the
top score for each of the PSQ-7 items (Table 1).
This skewness in the distribution of the PSQ-7
scores is also depicted in Figure 1 with 27.9%
of participants having a maximum total PSQ-7
score of 28.
56

Principal components analysis (PCA)

The KaiserMeyerOlkin measure of sampling
adequacy was 0.80 which supported proceeding
with PCA. Using principal components extraction procedure with the criterion of an eigenvalue
greater than 1 to extract the number of components for the PSQ-7, the analysis yielded a onecomponent solution, with an eigenvalue of 3.34,
which accounted for 48% of the variance. All
seven items loaded onto this factor with factor
loadings ranging from 0.60 to 0.75, which were
all above the 0.3 factor loading threshold.
Internal consistency and inter-item
correlations
Cronbachs alpha coefficient for the one-component PSQ-7 was 0.81. In the item analysis, the
corrected item-total correlations of all seven items
were higher than 0.30. All alpha values were not
higher than 0.81 when the item was deleted,
hence all items of the PSQ-7 were retained.

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

Volume 5, Issue 1, April 2011

Mixed methods approach to measure patient satisfaction

Group comparisons of PSQ-7 and

qualitative comments
Some patients made either positive or negative comments about their care, however, many
prefaced their statement with a general positive
comment and followed this by a negative comment or one indicating some dissatisfaction, with
or suggestions for, change/improvement in care.
We called these the yes, but statements. For
example:
Nursing staff are very courteous and they care
for me very well, they are always on time and
you dont have to chase them up. [BUT] A
few are not friendly and need a little more
orientation so they give the best care to all
patients
Fifty-three percent of patients made positive comments only, 15% made negative comments and 33% made yes, but comments. As
expected the low PSQ-7 tertile group had the
lowest number of positive comments (25%) and
highest number of negative (29%) and yes, but
comments (46%) for all groups. Examples of
comments for each group are given in Table 2.
While no patients in the high group, and only
12% of the medium group made negative comments, 24% and 28% of the medium and high
groups, respectively, made yes, but comments.
The examples indicate the yes, but comments
have an underlying criticism about their care or
a suggestion for change/improvement to their
care and hence can be perceived as a negative
comment being prefaced with a positive comment. If the yes, but comments are combined
with the negative ones the qualitative comments
indicate that almost 50% of patients had some
experience that suggested dissatisfaction with
their care or hospitalisation experience indicating areas requiring improvement/or change. The
negative and the qualifier comment with the
yes, but statements were analysed to reveal what
patients viewed as impacting on their satisfaction with their care.

Qualitative findings
The qualitative data was grouped into three
categories: communication, nursing care delivery and the hospital environment.

Communication
This category concerned participants comments
about aspects of communication they perceived as
negative or requiring some improvement. In particular, participants recommended that nursing
staff introduced themselves before doing a task or
when looking after them for a shift:
One of them (nursing staff ) did not introduce herself when she did a blood test.
Just one gentleman nurse did not introduce
himself, did not talk to me last night.
Nurses also needed to communicate what were
routine or regular practices in the unit as these
may have been unfamiliar to patients. One practice, for example, was not leaving an ambulant
patient alone in the bathroom which may explain
why this participant said:
When I go to the toilet I need privacy. They
(nurses) dont listen to me, they still remain
standing beside me.
Participants also wanted more explanation
about what drugs they were receiving, or why
treatments were or were not performed. Some
participants wanted to speak to senior staff
or the nurse unit manager and some commented in general about the need for more
communication.

Nursing care delivery

Participants made comments about the delivery
of nursing care including nurses manner in giving care and how they responded to participant
requests (responding to requests).
Giving care: Nurses attitudes and manners
when delivering care were pivotal to participants

Volume 5, Issue 1, April 2011 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

57

58

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

28

32

Medium
(2527)

High (More
than 27)

12

Low
(Up to 24)

PSQ-7 tertile

Total positive: 72

They have been

always good to me.
They respond to my
requests straight away.
No negative aspects
(of nursing care).

Their communication is
excellent. Couldnt be
improved upon. Very
good about making you
feel unembarrassed if
something goes wrong.
Cant think of any way to
improve it. They go out
of their way to be helpful.

Theyre caring. They do

everything they can to
help you.

Comment example

Positive

On the weekends there

is a lack of nurses; not
enough time to take
care of me. Need more
nurses.

Total negative: 20

They are too slow to

respond to my pain.
They did not come
back even though
they told me that the
patients requirement
should be on their work
list. They need more
training.

14

Comment example

Negative

TABLE 2: TERTILE GROUPS WITH EXAMPLES OF POSITIVE, NEGATIVE AND YES-BUT COMMENTS

Most of the nurses are caring. They

are very helpful and meet your needs.
Just one nurse did not introduce herself, did not talk to me last night. It
took 15 minutes to wait for the nurse
caring for me. Need more nurses.
Theyre very hardworking.

12

Total Yes, but 44

They (nursing staff) are very nice.

Sometimes they take too long
to come (response to call bell).
They need more funding from the
government.

They saved my life and I appreciate

that. I have to do what they want me
to do and hear what I have to hear.
Some treatments are too painful.
When they are busy I have to wait
but normally they let me know.
They dont explain about the drugs
enough. I need the senior staff to
come to talk with me. That does not
happen. I need to know my rights.

22

11

Comment example

Yes, but

136

39

50

48

Total

Sharon Andrew et al.

Volume 5, Issue 1, April 2011

Mixed methods approach to measure patient satisfaction

satisfaction with their care. Participants mentioned nurses they perceived as rough and were
unhappy with nurses who seemed rushed in their
delivery of care:
They always rush their job, they should take
their time. They should be more relaxed.
Participants mentioned being criticised and
nurses as being not nice or grumpy nurses and
how it impacted on their perception of care:
Take the grumpy nurses away, theyre not suitable for nursing.
Nurses are very pleasant, nice people, not all
of them, but most of them. When nurses are
not nice to you then you are not satisfied.
Generally they are excellent.
Participants also recognised experienced and inexperienced nurses expressing a preference for the
former although some found it difficult to distinguish between hospital staff due to the confusion
with uniforms.
Responding to requests: Many participants comments were about having to wait for care or the
response of nurses to the call bell.
I press the button (nurse call bell), they
come in a few minutes. If they dont come they
are busy.
While others mentioned that sometimes I have
to wait for calls some were concerned the nurses
indicated that they would come back but this did
not occur:
Usually they answer the buzzer very quickly.
My bed was a mess last night and they cleaned
it for me they are very lovely people. One day
they forget to come back, but they said We
will be back soon.
Many participants commented on the need for
more nursing staff generally or for specific times
including night shift, weekends or morning
shifts:

At night shifts, they dont see and check on me

very often.

The hospital environment

While the study was concerned with patient satisfaction with their nursing care, patients also mentioned how the clinical environment impacted
on their comfort, including the hospital bed too
lumpy, machines too noisy, too much blood
pressure tests, blood tests, heating is not good,
too cold and corridor lighting.

DISCUSSION
Findings from the current study support previous
research that the ability of quantitative questionnaires used to assess patient satisfaction may be
limited by their lack of variability and sensitivity
(Biering, Becker, Calvin, & Grobe, 2006; Lin,
1996) and the presence of ceiling effects (Cappelleri
et al., 2000), resulting in highly skewed scores.
This was demonstrated in 5980% of patients
choosing the highest possible score for the items,
with almost one-third of patients having a maximum score of 28 for the seven-item instrument.
In the context of the current study, there are
several reasons which could account for the highly
positive scores. First, a number of social-psychological artefacts (Sitzia & Wood, 1997) have been
suggested to influence patient satisfaction scores.
These include social desirability response bias,
where patients report greater satisfaction than
they feel, believing positive responses will be more
acceptable to the researcher (Biering et al., 2006;
Hays & Ware, 1986; Sitzia & Wood, 1997). It has
also been suggested that patients may be reluctant to complain, for fear of unfavourable treatment (Ley, 1972), particularly when they are still
dependent on medical staff for treatment, thus
leading to another form of response bias (Stevens,
Reininga, Boss, & Van Horn, 2006). This may
have been particularly relevant in the context of
the current study, which took place in a high-dependency neurosurgical unit where patient acuity
was high.

Volume 5, Issue 1, April 2011 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

59

Sharon Andrew et al.

It could also be that the timing of the PSQ

resulted in more positive scores, with studies suggesting patients surveyed during their stay are more
likely to express greater satisfaction than those
surveyed post-discharge (Jensen, Ammentorp, &
Kofoed, 2010; Stevens et al., 2006). Again, given
that two-thirds of these patients had been emergency admissions, it could be that in focusing on
their present state of health (Kane, Maciejewski,
& Finch, 1997), their higher satisfaction scores
were a reflection of relief and gratitude for the
care received (Stevens et al., 2006).
Patient satisfaction scores could also have been
high because the response format was more likely
to elicit positive ratings. For example, neutrally
worded item stems using an excellentpoor scale
have been shown to elicit greater variance and less
skewness (Ware & Hays, 1988), and are thus proposed as being superior to items that are either
positively or negatively worded (Rubin, 1990).
Two of the items in the Ludwig-Beymer Patient
Satisfaction Instrument used in this study were
neutrally worded, one of which demonstrated
reduced ceiling effects (Item 6).
Although the extremely high satisfaction scores
were not totally unexpected, this level of satisfaction was not always reflected in the qualitative
comments, with 28% of patients in the high tertile group making what we have described as yes,
but comments. Although the issue of response
bias previously discussed should, theoretically,
equally apply to the qualitative comments, it
seems that when given the opportunity to provide feedback to open-ended questions, almost
half the sample of patients included comments
that indicated a level of dissatisfaction. It may be
that as suggested by Beiring et al. (2006), rather
than patient satisfaction and dissatisfaction being
at opposite ends of the same continuum, they
are, in fact, two different phenomena that require
two different definitions, and two kinds of instruments. Using our study as an example it may be
that a quantitative instrument is best used to measure satisfaction whereas a qualitative method is a
more informative measure of dissatisfaction.
60

In combining both quantitative and qualitative

data collection methods, it may be, however, that
this was unknowingly measuring two different
constructs. On the other hand the combination of
data collection methods may have led to a deeper
insight (Bazeley, 2002, 2009) than may have been
possible by using one method. Moreover, this integration is enhanced by the availability of computer
programs such as NVivo that can assist by adding
another dimension to the study findings (Andrew,
Salamonson & Halcomb, 2008; Bazeley, 2009).
Despite scoring all of the quantitative items
highly, the qualitative responses indicated some
dissatisfaction with areas of the participants care.
The qualitative categories identified in this study
were similar in aspects to those identified by others specifically communication, nursing care and
environment (Dawood & Gallin, 2010; Hunt,
1999; Merkouris et al., 2004). Communication
has been consistently viewed as one of the crucial factors influencing patient satisfaction (Hunt,
1999; Lee & Yom, 2007; McCabe, 2004; Oflaz
& Vural, 2010; Wagner & Bear, 2008). Similarly,
patients views about nurses response to the call
bell has been found to be a factor in patient satisfaction (Roszell, Jones, & Lynn, 2008). As
communication failures are strongly linked with
adverse events (Manno et al., 2006), monitoring
patients views are of particular importance.
Around half the participants made positive
comments about their care and the remaining percent made negative or yes, but statements indicating some level of dissatisfaction with an aspect of
care or had a suggestion for some change. As others
have documented, every interaction with a patient
is important and it takes only one encounter for
a patient to feel some dissatisfaction with their
nursing care (Merkouris et al., 2004). If we are to
improve nursing care then we need to understand
what we can to do achieve this outcome. Structured
satisfaction scales undertaken for meeting quality
assurance standards may meet their intended use
but without a qualitative component they are missing the opportunity to understand patients perceptions of their care and how it can be improved.

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

Volume 5, Issue 1, April 2011

Mixed methods approach to measure patient satisfaction

CONCLUSION
A mixed method approach to measuring patient
satisfaction enables researchers to move beyond
the ceiling effect of patient satisfaction scales and
to gain a more meaningful explanation of patient
satisfaction.

ACKNOWLEDGMENT
The authors would like to acknowledge funding support provided by the NSW Nurses and
Midwives Board for the conduct of this study.

References
Andrew, S., & Halcomb, E. J. (2006). Mixed
methods research is an effective method
of enquiry for community health research.
Contemporary Nurse, 23, 145153.
Andrew, S., Salamonson, Y., & Halcomb, E.
J. (2008). Integrating mixed methods data
analysis using NVivo: An example examining
attrition and persistence of nursing students.
International Journal of Multiple Research
Approaches, 2, 3643.
Bazeley, P. (2002). The evolution of a project
involving an integrated analysis of structured
qualitative and quantitative data: From N3 to
NVivo. International Journal of Social Research
Methodology, 5(3), 229243.
Bazeley, P. (2009). Analysing mixed methods data
(Chap. 6). In S. Andrew & E. J. Halcomb
(Eds.), Mixed methods research for nursing and
the health sciences (pp. 161180). Chichester,
West Sussex: Wiley-Blackwell.
Biering, P., Becker, H., Calvin, A., & Grobe,
S. J. (2006). Casting light on the concept of
patient satisfaction by studying the construct
validity and the sensitivity of a questionnaire.
International Journal of Health Care Quality
Assurance, 19(3), 246258.
Bradley, C. (1999). Diabetes treatment satisfaction
questionnaire: Change version for use alongside
status version provides appropriate solution
where ceiling effects occur. Diabetes Care, 22,
530532.
Bradley, C., Plowright, R., Stewart, J., Valentine,
J., & Witthaus, E. (2007). The diabetes treatment satisfaction questionnaire change version

(DTSQc) evaluated in insulin glargine trials

shows greater responsiveness to improvements
than the original DSTQ. Health and Quality of
Life Outcomes, 5(1), 57.
Cappelleri, J. C., Gerber, R. A., Kourides, I. A., &
Gelfand, R. A. (2000). Development and factor
analysis of a questionnaire to measure patient satisfaction with injected and inhaled insulin for type
1 diabetes. Diabetes Care, 23(12), 17991803.
Cheraghi-Sohi, S., Bower, P., Mead, N., McDonald,
R., Whalley, D., & Roland, M. (2006). What
are the key attributes of primary care for
patients? Building a conceptual map of patient
preferences. Health Expectations, 9(3), 275284.
Cheraghi-Sohi, S., Hole, A., Mead, N., McDonald,
R., Whalley, D., Bower, P., et al. (2008). What
patients want from a primary care consultation. A discrete choice experiment to identify
patients priorities. Annals of Family Medicine,
6(2), 107115.
Coffey, A., & Atkinson, P. (1996). Making sense
of qualitative data: Complementary research strategies. Thousand Oaks, CA: Sage.
Creswell, J. W., & Plano Clark, V. L. (2007).
Designing and conducting mixed methods research.
Thousand Oaks, CA: Sage.
Dawood, M., & Gallin, A. (2010). Using discovery
interviews to understand the patient experience.
Nursing Management, 17(1), 2631.
Donovan, J. L. (1995). Patient decision making.
The missing ingredient in compliance research.
International Journal of Technology Assessment in
Health Care, 11(3), 443455.
Greco, M., Powell, R., & Sweeney, K. (2003). The
improving practice questionnaire (IPQ): A practical tool for general practices seeking patient
views. Education for Primary Care, 14, 440448.
Greene, J. C., Caracelli, V. J., & Graham, W. F.
(1989). Toward a conceptual framework for
mixed-method evaluation designs. Educational
Evaluation and Policy Analysis, 11(3), 255274.
Grogan, S., Conner, M., Norman, P., Willits, D.,
& Porter, I. (2000). Validation of a questionnaire measuring patient satisfaction with general
practitioner services. Quality and Safety in
Health Care, 9(4), 210215.
Halcomb, E. J., Andrew, S., & Brannen, J. (2009).
Introduction to mixed methods research for
nursing and the health sciences (Chap. 1). In S.

Volume 5, Issue 1, April 2011 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

61

Sharon Andrew et al.

Andrew & E. J. Halcomb (Eds.), Mixed methods
for nursing and the health sciences (pp. 312).
Chichester: Wiley-Blackwell.
Hardy, G. E., West, M. A., & Hill, F. (1996).
Components and predictors of patient satisfaction. British Journal of Health Psychology, 1, 6585.
Hays, R. D., & Ware, J. E., Jr. (1986). My medical
care is better than yours. Social desirability and
patient satisfaction ratings. Medical Care, 24(6),
519524.
Hunt, J. M. (1999). The cardiac surgical patients
expectations and experiences of nursing care in
the intensive care unit. Australian Critical Care,
12(2), 4753.
Jensen, H. I., Ammentorp, J., & Kofoed, P. E.
(2010). User satisfaction is influenced by the
interval between a health care service and the
assessment of the service. Social Science and
Medicine, 70(12), 18821887.
Kane, R. L. (2006). Understanding health care
outcomes research (2nd ed.). Sudbury, MA: Jones
and Bartlett.
Kane, R. L., Maciejewski, M., & Finch, M. (1997).
The relationship of patient satisfaction with
care and clinical outcomes. Medical Care, 37(7),
714730.
Kroll, T., & Neri, M. (2009). Designs for mixed
methods research (Chap. 3). In S. Andrew &
E. J. Halcomb (Eds.), Mixed methods research
for nursing and the health sciences (pp. 312).
Chichester: Wiley-Blackwell.
Lee, M. A., & Yom, Y. H. (2007). A comparative
study of patients and nurses perceptions of
the quality of nursing services, satisfaction and
intent to revisit the hospital: A questionnaire
survey. International Journal of Nursing Studies,
44, 545555.
Ley, P. (1972). Complaints made by hospital staff
and patients: A review of the literature. Bulletin
of British Psychologists, 25, 115120.
Lin, C.-C. (1996). Patient satisfaction with nursing
care as an outcome variable: Dilemmas for nursing evaluation researchers. Journal of Professional
Nursing, 12(4), 207216.
Ludwig-Beymer, P., Ryan, C. J., Johnson, N. J.,
Hennessy, K. A., Gattuso, M. C., Epsom, R.,
et al. (1993). Using patient perceptions to
improve quality care. Journal of Nursing Care
Quality, 7(2), 4251.
62

Manno, M., Hogan, P., Heberlein, V., Nyakiti,

J., & Mee, C. L (2006) Patient safety survey
report. Nursing 36(5) 5464.
McCabe, C. (2004). Nurse-patient communication:
An exploration of patients experiences. Journal
of Clinical Nursing, 13, 4149.
Mead, N., Bower, P., & Roland, M. (2008). The
general practice assessment questionnaire
(GPAQ)-development and psychometric characteristics. BMC Family Practice, 9, 13.
Merkouris, A., Papathanassoglou, E. D. E., &
Lemonidou, C. (2004). Evaluation of patient
satisfaction with nursing care: Quantitative or
qualitative approach? International Journal of
Nursing Studies, 41(4), 355367.
Nunnally, J. C., & Bernstein, I. H. (1994).
Psychometric theory (3rd ed.). New York:
McGraw-Hill.
Oflaz, F., & Vural, H. (2010). The evaluation of
nurses and nursing activites through the perceptions of inpatients. International Nursing Review,
57, 232239.
Potiriadis, M., Chondros, P., Gilchrist, G., Hegarty,
K., Blashki, G., & Gunn, J. M. (2008). How do
Australian patients rate their general practitioner? A descriptive study using the general practice assessment questionnaire. Medical Journal of
Australia, 189(4), 215219.
Pouwer, F., Snoek, F., & Heine, R. (1998). Ceiling
effect reduces the validity of the diabetes treatment satisfaction questionnaire. Diabetes Care,
21(11), 2039.
Redsell, S., Stokes, T., Jackson, C., Hastings, A.,
& Baker, R. (2007). Patients accounts of the
differences in nurses and general practitioners roles in primary care. Journal of Advanced
Nursing, 57(2), 172180.
Roszell, S., Jones, C. B., & Lynn, M. R. (2008).
Call bell requests, call bell response time, and
patient satisfaction. Journal of Nursing Care
Quality, 24(1), 6975.
Rowell, R. M., & Polipnick, J. (2008). A pilot
mixed methods study of patient satisfaction
with chiropractic care for back pain. Journal
of Manipulative and Physiological Therapeutics,
13(8), 602610.
Rubin, H. R. (1990). Patient evaluations of hospital care: A review of the literature. Medical Care,
28(9 Suppl.), S3S9.

INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

Volume 5, Issue 1, April 2011

Mixed methods approach to measure patient satisfaction

Sitzia, J., & Wood, N. (1997). Patient satisfaction:
A review of issues and concepts. Social Science &
Medicine, 45(12), 18291843.
Stevens, M., Reininga, I. H. F., Boss, N. A. D., &
Van Horn, J. R. (2006). Patient satisfaction at
and after discharge. Effect of a time lag. Patient
Education and Counseling, 60(2), 241245.
Streiner, D. L., & Norman, G. R. (2008). Health
measurement scales: A practical guide to their
development and use (4th ed.). Oxford: Oxford
University Press.
Tabachnick, B. G., & Fidell, L. S. (2007). Using
multivariate statistics (5th ed.). Boston: Pearson/
Allyn & Bacon.

Trout, A., Magnusson, R., & Hedges, J. R.

(2000). Patient satisfaction investigations and
the emergency department: What does the
literature say? Academic Emergency Medicine,
7(6), 695709.
Wagner, D., & Bear, M. (2008). Patient satisfaction with nursing care: A concept analysis
within a nursing framework. Journal of Advanced
Nursing, 65(3), 692701.
Ware, J. E., Jr., & Hays, R. D. (1988). Methods for
measuring patient satisfaction with specific medical encounters. Medical Care, 26(4), 393402.
Received 28 June 2010

Accepted 24 August 2010

SPECIAL ISSUES FROM eCONTENT

Transformations in Health Care:
Privatisation, Corporatisation
and the Market

Advances in Complex Health Care: Nursing

Interventions
Edited by Wai-Tong Chien (Professor & Convener of Mental Health
Research Group School of Nursing, Faculty of Health and Social
Sciences The Hong Kong Polytechnic University,
Hong Kong SAR, China)
Co-edited by Sally Chan (Professor and Head, National University
of Singapore, Alice Lee Centre for Nursing Studies, Yong Loo Lin
School of Medicine, Singapore), David R Thompson (Professor,
Cardiovascular Research Centre, Australian Catholic University
Melbourne, Australia), Richard Gray (Professor of Research and
Director of Research Degree Programmes Faculty of Health,
University of East Anglia, Norwich, UK) and Yip Mei-Po (Research
Scientist, Asian Health Promotion Program University of Washington,
Seattle, WA, USA)
a special issue of Journal of Contemporary Nurse vol 40/1
ISBN 978-1-921729-51-5 December 2011 ii+126 pages
http://www.contemporarynurse.com/archives/vol/40/issue/1/marketing/

Advances in Contemporary Community and

Family Health Care (2nd edn)
Edited by Anne McMurray (Adjunct Professor, Research Centre for
Clinical and Community Practice, Innovation, Grifth University, Gold
Coast QLD, Australia) and Mark Hayter (Reader in Nursing and Head
of Taught Post Graduate Programmes, The University of Shefeld
School of Nursing and Midwifery, UK)
a special issue of Journal of Contemporary Nurse vol 39/1
ISBN 978-1-921729-28-7 August 2011 ii+126 pages
http://www.contemporarynurse.com/archives/vol/39/issue/1/marketing/

Advances in Contemporary Nurse Education

(2nd edn)
Edited by Debra Jackson (University of Western Sydney, Austra),
Roger Watson (University of Shefeld, UK) and
Tom Mason (University of Chester, UK),
a special issue of Journal of Contemporary Nurse vol 38/1
ISBN 978-1-921729-26-3 April 2011 ii+126 pages
http://www.contemporarynurse.com/archives/vol/38/issue/1/marketing/

Advances in Contemporary Indigenous Health

Care (2nd edn)
Edited by Kim Usher (James Cook University) and Rhonda Marriott
(Murdoch University)
a special issue of Journal of Contemporary Nurse vol 37/1
ISBN 978-1-921348-89-1 December 2010 126 pages
http://www.contemporarynurse.com/archives/vol/37/issue/1/marketing/

Edited by Kevin White (The Australian National

University), Fran Collyer (The University of Sydney) and
Jane Jones (Flinders University of South Australia)
a special issue of Health Sociology Review vol 20/3
ISBN 978-1-921729-04-1 September 2011 ii+110 pages
http://hsr.e-contentmanagement.com/archives/vol/20/issue/3/marketing//

Mental Health and Illness: Practice and

Service Issues
Edited by Pauline Savy, Anne-Maree Sawyer and
Katy Richmond (Faculty of Humanities and Social
Sciences La Trobe University, VIC, Australia)
a special issue of Health Sociology Review vol 20/2
ISBN 9781921348570 June 2011 ii+110 pages
http://hsr.e-contentmanagement.com/archives/vol/20/issue/2/marketing/

Mens Health
Edited by Gary Dowsett (La Trobe University), Victor
Minichiello (University of New England) and
John Scott (University of New England)
a special issue of Health Sociology Review vol 19/4
ISBN 978-1-921348-91-4 December 2010 ii+138 pages
http://hsr.e-contentmanagement.com/archives/vol/19/issue/4/marketing/

Lifestyle Science: Self-healing,

co-production and DIY
Edited by Rick Iedema (Centre for Health
Communication University of Technology Sydney,
Australia) and Irena C Veljanova (School of Social
Sciences University of Western Sydney, Australia)
a special issue of Health Sociology Review vol 21/3
ISBN 978-1-921729-36-2 September 2012 iv + 100 pages
http://hsr.e-contentmanagement.com/archives/vol/21/issue/3/call/

eContent Management Pty Ltd, PO Box 1027, Maleny QLD 4552, Australia
Tel.: +61-7-5435-2900; Fax. +61-7-5435-2911;
subscriptions@e-contentmanagement.com www.e-contentmanagement.com

Content

PTY

management LTD

Volume 5, Issue 1, April 2011 INTERNATIONAL JOURNAL OF MULTIPLE RESEARCH APPROACHES

63

Copyright of International Journal of Multiple Research Approaches is the property of eContent Management
Pty. Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder's express written permission. However, users may print, download, or email articles for
individual use.