See

discussions, stats, and author profiles for this publication at: http://www.researchgate.net/publication/225376376

What Makes a Quality Therapy? A Consideration

of Parsimony, Ease, and Efficiency
ARTICLE in BEHAVIOR THERAPY SEPTEMBER 2012
Impact Factor: 2.85 DOI: 10.1016/j.beth.2010.12.007 Source: PubMed

CITATIONS

DOWNLOADS

VIEWS

179

176

1 AUTHOR:
Jesse R Cougle
Florida State University
61 PUBLICATIONS 632 CITATIONS
SEE PROFILE

Available from: Jesse R Cougle

Retrieved on: 28 July 2015

Available online at www.sciencedirect.com

Behavior Therapy 43 (2012) 468 481

www.elsevier.com/locate/bt

What Makes a Quality Therapy? A Consideration of Parsimony,

Ease, and Efficiency
Jesse R. Cougle
Florida State University

Evaluations of psychotherapy have traditionally focused on

symptom reduction as the primary standard by which their
value is determined. This has contributed to the appearance
of equivalence between many therapies that may differ
considerably in complexity, feasibility, amount of homework and therapist contact required, expected cost, speed of
symptom decline, and transdiagnostic utility. In the current
paper, I make the case that these are fundamental features
related to quality that should be considered in psychotherapy development, randomized controlled trials, and dissemination efforts. Empirically supported treatments for
different disorders are evaluated based on these criteria,
and special consideration is given to cognitive-behavioral
treatments for anxiety disorders. Specific recommendations
for a quality-oriented clinical research agenda are also
provided.

Keywords: therapy efficiency; cognitive-behavioral therapy;

exposure therapy; cognitive restructuring; treatment dissemination

VARIOUS FORCES IN and outside the scientific

community have worked to focus increasing attention on identifying empirically supported interventions for psychiatric disorders. One of the products of
such efforts was the formation of the Task Force on
Promotion and Dissemination of Psychological Procedures of Division 12 (Clinical Psychology) of the

I would like to thank the anonymous reviewers for their helpful

comments on an earlier version of this manuscript.
Address correspondence to Jesse R. Cougle, Ph.D., Department
of Psychology, Florida State University, PO Box 3064301,
Tallahassee, FL 32306; e-mail: cougle@psy.fsu.edu.
0005-7894/43/468-481/$1.00/0
2011 Association for Behavioral and Cognitive Therapies. Published by
Elsevier Ltd. All rights reserved.

American Psychological Association (Chambless,

1995). This Task Force proposed several criteria
for identifying empirically supported treatments,
including comparison of the treatment to an
appropriate control group, the use of treatment
manuals, and independent replication of therapy
effects. They also classified treatments according to
the number of such criteria they met, giving the
designation of well established to those fulfilling
all criteria and the probably efficacious title to
those meeting fewer but not all criteria (Chambless et
al., 1998). The Task Force originally identified 25
treatments in 1995 (Chambless, 1995), though this
number grew to 71 in 1998 (Chambless et al., 1998).
By 2001, the group identified 108 well-established or
probably efficacious treatments for adults and 38 for
children (Chambless & Ollendick, 2001). A Division
12 Web site (www.PsychologicalTreatments.org)
currently keeps an updated list of treatments with
modest or strong research support.
Though the efforts of the Division 12 Task Force
are important and worthwhile, the high number of
empirically supported treatments identified is problematic for a few reasons. First, the preponderance
of therapies makes it difficult for educators and
practitioners to identify which interventions to
teach, learn, and utilize. It also raises issues of
quality control, as only those who are well trained
in specific approaches would be able to provide
adequate supervision. Therapist competency may
be negatively impacted by the scarcity of expert
supervisors available, as well as the varied criteria
for proficiency likely to exist across different
schools of therapy. Further, and perhaps most
important, this list leaves the impression of
equivalence between therapies that may have key
differences that would make one approach preferable to another.

what makes a quality therapy?

It would likely be asking too much for the Task
Force to further distinguish between treatments
with strong empirical support. Besides being
fraught with controversy, such an endeavor would
in many cases be going beyond the data. Indeed,
insofar as they are primarily focused on treatment
efficacy, published clinical trials offer little guidance
for this problem. Many of the empirically supported treatments listed do in general lead to
equivalent outcomes.
In the current essay, I hope to review as well as
present additional ways in which psychological
treatments might be evaluated, with the goal of
moving beyond the criterion of symptom reduction.
I propose that an overattentiveness to symptom
reduction has served to distract the field from
important ways in which the quality of therapy can
be improved and empirically supported treatments
can be distinguished. Instead, I argue for greater
attention to parsimony, ease, and efficiency in
psychotherapy. Different interventions will be
discussed in light of these criteria, with an emphasis
on cognitive-behavioral treatments for anxiety
disorders. Some consideration will also be given
to treatments for depression. In conclusion, I
present specific recommendations for a qualityoriented clinical research agenda.
I should note that these proposed criteria for
psychotherapy evaluation have a significant degree
of overlap and have been mentioned by various
researchers over the years. However, this is to my
knowledge the first attempt to formally specify
potential features of a quality therapy in a single
essay. My goal in doing so is to spur additional
research and discussion.

The Importance of Parsimony in Clinical

Interventions
Given equivalent outcomes between two different
therapies, the more parsimonious approach may be
the preferred approach. As I discuss below,
parsimony in therapy, or the number and simplicity
of therapeutic components, may have a direct
bearing on the amount of therapist training
required to successfully implement therapy, the
ease of treatment dissemination, treatment integrity, and the degree to which clients adhere to
treatment.

more parsimonious therapies may

require less clinical training and
will likely be easier to disseminate
Complex treatments will generally require a larger
amount of clinical training than simpler approaches. The less there is to teach, the less time,
effort, and money will be required to learn.

469

Resource constraints may be less of a concern for

doctoral students undergoing years of clinical
training. However, full-time therapists, in particular, will have difficulty leaving work and paying for
days or even weeks of clinical training and
supervision. Thus, the complexity of a treatment
will likely influence how easily it can be disseminated.
A treatment with fewer components will not
necessarily require less training. Cognitive restructuring as a specific treatment component is arguably more complex and may require more training
and supervision than relaxation, breathing retraining, or exposure therapy. One study demonstrated
some level of success in teaching exposure therapy
to nonclinicians using a 1-hour computerized
training course (Gega, Norman, & Marks, 2007).
Unfortunately, there is scant research that speaks to
the relative difficulty of training clinicians toward
adequate proficiency in different treatments.
Foa and colleagues (2005a) conducted one
particularly relevant study in which individuals
with PTSD were treated using prolonged exposure
with or without cognitive restructuring. The
treatments were delivered by doctoral-level clinicians from an academic research center and
master's-level community therapists with little
prior experience with cognitive-behavioral therapy
(CBT). All therapists were trained together in a 5day workshop for prolonged exposure and a
second 5-day workshop for cognitive restructuring.
No significant differences in outcome emerged
between treatment conditions or between academic
and community therapists. It is possible that the
extra 5-day training in cognitive restructuring led to
benefits from prolonged exposure that would not
have otherwise appeared. However, these findings
still suggest that prolonged exposure without
cognitive restructuring would be the preferred
treatment for PTSD, as significantly more clinical
training was required for the cognitive restructuring
component that demonstrated no additive value.
Though these therapies produced equivalent outcomes, the additional value conferred by greater
parsimony and less required training would tip the
scale in favor of prolonged exposure without
cognitive restructuring. 1
Similarly, component analyses of cognitive therapy for depression have found that behavioral
activation is just as effective as the full treatment
1

These conclusions are of course based on the design of this

study. That is, the investigators did not examine the efficacy of
cognitive restructuring with and without prolonged exposure. The
fact that only certain components are usually targeted for removal
is a limitation of dismantling studies.

470

cougle

package (Jacobson et al., 1996). Given the simplicity of this procedure and the extensive training
required to learn the additional components of
cognitive therapy, behavioral activation would
appear to be the preferred intervention for depression. Meta-analytic findings also provide encouraging evidence for the efficacy of behavioral
activation (Cuijpers, Van Straten, & Warmerdam,
2007; Mazzucchelli, Kane & Rees, 2009).

greater parsimony may lead to

greater treatment integrity
Among empirically supported interventions, more
parsimonious treatments may also be preferred
because they are likely to lead to greater treatment
integrity. Treatment integrity, or the degree to
which an intervention is administered as it was
intended (Yeaton & Sechrest, 1981), has been
implicated as an essential factor in predicting
therapy success, though there is conflicting evidence
regarding its importance. A very recent metaanalysis found no overall relationship between
treatment integrity and outcome (Webb, Derubeis,
& Barber, 2010), though moderator analyses found
integrity to predict outcome in studies of depression. Some have argued that the absence of
associations between integrity and outcome across
different studies may be due to limitations in the
way integrity is usually assessed (Perepletchikova &
Kazdin, 2005).
More complex treatments with a greater number
of components may require more materials, resources, and sessions than simpler approaches.
Given the limitations imposed by cost, time, client
attrition, and the number of procedures therapists
will be able to learn and implement effectively, there
should be significant benefit associated with the
most parsimonious treatment. Generally speaking,
the lower the number and the greater the simplicity
of treatment components, the greater the likelihood
that each component will be used and the treatment
will be implemented as intended.
One effectiveness study of group treatment for
depression compared professionals to paraprofessionals who were trained to administer CBT or a
nondirective mutual support therapy (Bright,
Baker, & Neimeyer, 1999). Overall, equivalent
outcomes were found between treatments and types
of therapists. However, among clients treated with
CBT, those treated by professionals had higher
rates of recovery than those treated by paraprofessionals. The authors speculated that the lower
efficacy shown by paraprofessionals administering
CBT may be due to the sophistication of techniques
required by this treatment. This study parallels
other clinical trials that have found greater therapist

experience to be associated with improved outcome

in CBT for depression (DeRubeis et al., 2005;
Jacobson & Hollon, 1996). Overall, this research
implicates potential difficulties in implementing
CBT that may not be present with simpler, less
structured approaches.

greater parsimony may lead to greater

client treatment adherence and
improved outcome
More complex treatments may lead to poorer client
treatment adherence than simpler approaches.
Some clients will have difficulty comprehending
various treatment components, and there will likely
be an inverse relationship between the number of
treatment components and overall treatment adherence. Thus, complex therapies may lead to
poorer outcomes than simpler approaches.
There has been very little research examining the
potential relationships between parsimony and
client treatment adherence and treatment outcome.
Intelligence has not been found to predict outcome
in individuals undergoing cognitive therapy
(Haaga, DeRubeis, Stewart, & Beck, 1991; Rizvi,
Vogt, & Resick, 2009); however, such studies may
be confounded by range restriction, behavioral
components of treatment, or skill level of clinicians.
Illiterate populations, older adults, and individuals
with intellectual disabilities are typically excluded
from clinical trials of CBT. Procedures such as
monitoring of automatic thoughts, identification of
cognitive distortions, and cognitive reappraisal may
have less utility for such individuals and in many
cases will be impossible to implement. One study
evaluated the cognitive-emotional skills of individuals with mild intellectual disabilities and noted
some difficulties these individuals had in understanding basic concepts (e.g., cognitive mediation)
essential for cognitive therapy (Dagnan, Chadwick,
& Proudlove, 2000).
Cognitive therapies usually include certain components that are simple enough for lower-functioning clients, and the apparent benefits of cognitive
therapy for such individuals may be due to the
effects of these simpler components. For example,
clients who may not comprehend or adhere well to
the full cognitive therapy package for depression
may still benefit from the simpler behavioral
activation component that is used throughout
treatment. In his review of cognitive therapy for
people with intellectual disabilities, Sturmey
(2004) noted that it was difficult to ascertain the
unique benefits of specific cognitive techniques for
this population because many interventions labeled
cognitive consisted mostly of behavioral components. He concluded that more work is needed

what makes a quality therapy?

examining specific cognitive techniques to help
determine their efficacy with this group.
Therapies for emotional disorders are usually
modified for use with children, older adults, and
populations with cognitive impairment. For example, it has been recommended that cognitive
therapy for depressed older adults involve fewer
goals and focus more on behavioral activation than
cognitive restructuring (Crowther, Scogin, &
Norton, 2010). Stripped-down or simplified adaptations of therapies are also recommended for
children (Friedberg & McClure, 2002) and for
clients with intellectual disabilities (Lindsay, Howells, & Pitcaithly, 1993). More parsimonious
interventions may require fewer adaptations for
these populations.
There are some data to suggest that greater
therapeutic complexity will negatively impact
therapy outcome. For example, McLean et al.
(2001) found that behavior therapy (exposure and
response prevention) was more effective than CBT
when delivered in a group format for obsessive
compulsive disorder (OCD). The authors speculated
that the group setting was not an optimal format
for administering a treatment as complex as CBT,
which in their study focused on six different
belief domains. The increased efficiency and costeffectiveness conferred by group behavior therapy no
doubt make it an especially attractive treatment
option for OCD.
Many interventions rely on knowledge and skills
that are accumulated over multiple therapy sessions
rather than specific techniques that can be quickly
implemented. For example, cognitive therapies
(e.g., Heimberg & Becker, 2002) typically involve
psychoeducation regarding thoughts and emotions,
identification and self-monitoring of automatic
thoughts, and identification of thinking errors, as
well as methods for challenging thinking errors.
Sporadic or limited session attendance may reduce
the likelihood that such procedures will have any
efficacy (Reardon, Cukrowicz, Reeves, & Joiner,
2002).

The Importance of Ease in

Clinical Interventions
The quality of a therapy may also be influenced by
the ease with which it is implemented and the
difficulties it imposes on both therapist and client. I
refer to ease here as including feasibility, aversiveness, number and duration of sessions, amount of
homework, and cost. Each of these interrelated
characteristics has been neglected to varying degrees
in psychotherapy evaluation, though I argue that
they are all important in assessing the advantages
and disadvantages of different treatments.

471

feasibility of therapy
Psychotherapies that produce equivalent outcomes
may differ considerably in feasibility of implementation. This issue is particularly relevant to anxiety
treatments that rely on extensive exposure exercises. Lack of resources may make various
treatment components difficult to use in typical
clinical settings. For example, recent CBT protocols for social anxiety disorder utilize video and
audio feedback of client performance along with
exposure exercises involving audiences and conversational partners (Clark et al., 2006). Many
clinical settings do not have the equipment
necessary for such exercises, and it is often difficult
to coordinate in-session exposure exercises involving an audience or conversational partner. Greater
disclosure of the degree to which therapies rely on
outside resources would help assess the feasibility
of different treatment protocols. More specifically,
anxiety researchers could report the average
amount and type of coordinated exposures per
client, which could include total audience or
conversational partner exposures, as well as any
out-of-session exposures (e.g., driving, in-home
exposures). Generally speaking, interventions that
require less effort to implement successfully will be
preferred.
Concerns of feasibility might make such methods
as role-playing and imaginal exposure more attractive options for in-session use. Group therapy
settings should make audience exposures much
easier to conduct. In addition, different Internet
sites may be helpful in facilitating exposure by
providing chat rooms for treating social anxiety or
combat-related audio recordings and video clips for
treating PTSD (e.g., http://www.ilovewavs.com/
Effects/War/War.htm).
aversiveness of therapy
The emotional distress required by therapies should
also be considered when evaluating their quality. I
say this with caution, understanding that many
disorders are maintained by fear or avoidance of
anxiety and distress (Hayes, Wilson, Gifford,
Follette, & Strosahl, 1996), and fear confrontation
via exposure therapy is an effective and relatively
parsimonious treatment approach. However, there
is a need for less aversive alternatives. Approximately 25% of individuals with OCD refuse
exposure and response prevention treatment
(Franklin & Foa, 1998) and up to 28% drop out
once treatment begins (Foa, Liebowitz, Kozak et al.,
2005b). Though reasons for exposure therapy
refusal have not been studied extensively, such
refusal is no doubt due in part to apprehension

472

cougle

regarding the intensity and distress it requires.

Similar hesitancies related to fear confrontation are
likely a major cause of the low rates of treatment
utilization among people with other anxiety disorders (Issakidis & Andrews, 2002).
A number of alternatives exist that could benefit
those who refuse exposure therapy. CBT generally
includes some elements of fear confrontation,
though in the form of behavioral experiments
involving brief exposure rather than repeated,
prolonged exposure (Clark et al., 1994). There is
some evidence to suggest CBT is less aversive to
clients than treatments involving repeated exposure. For example, research suggests that CBT
results in lower dropout rates than exposure and
response prevention in the treatment of OCD
(Abramowitz, Taylor, & McKay, 2005; Whittal,
Robichaud, Thordarson, & McLean, 2008),
though a meta-analysis of treatments for social
anxiety disorder found no differences in dropout
rates between behavior therapy and cognitive
therapy without exposure (Gould, Buckminster,
Pollack, Otto, & Yap, 1997). Researchers surveyed
852 psychologists and found that very few used
exposure for PTSD, and approximately half
thought the use of exposure would increase the
likelihood of client attrition (Becker, Zayfert, &
Anderson, 2004). However, a review of published
treatment studies for PTSD found no differences in
attrition rates between exposure therapy and
cognitive therapy (Hembree et al., 2003). 2
If cognitive techniques have benefits as less
aversive alternatives to exposure therapy, these
benefits must also be balanced against other issues
of quality mentioned in this essay (e.g., parsimony,
training requirements). Cognitive procedures may
also require more time to implement effectively. For
example, st, Alm, Brandberg, & Breitholtz (2001)
found that one session of exposure was as effective
as five sessions of cognitive restructuring (and five
sessions of exposure therapy) in the treatment of
claustrophobia; unfortunately, they did not include
a single-session cognitive restructuring condition
for adequate comparison. Treatment protocols
incorporating as little as 30 min of exposure have
demonstrated response rates of up to 94% for
claustrophobia (Powers, Smits, & Telch, 2004); to
my knowledge, no studies have demonstrated
2
Comparing rates of refusal or attrition between different
treatments is complicated by the fact that many clients are initially
told that there is a likelihood that they will be assigned to a
treatment involving fear confrontation. Many anxious clients also
seek out treatment with the understanding that it will involve
distressing assessments and interventions. Thus, rates of refusal and
attrition reported in clinical trials may underestimate problems
posed by distressing treatments.

comparable efficacy from cognitive restructuring

in as short an amount of time. Additionally, a metaanalysis of treatments for social anxiety disorder
found that exposure therapy produced larger effect
sizes than cognitive restructuring without exposure
(Gould et al., 1997).
Other noteworthy treatments that may be less
distressing than exposure therapy include applied
relaxation (st, 1987), which has demonstrated
efficacy in the treatment of various anxiety disorders. However, at least with regard to attrition, this
intervention appears to have no advantage over
CBT (Siev & Chambless, 2007). Preliminary
evidence also suggests interpersonal psychotherapy,
an additional intervention not requiring repeated
fear confrontation, is effective for the treatment of
PTSD (Bleiberg & Markowitz, 2005), though it has
not been researched extensively.
It is not yet clear whether cognitive therapy,
applied relaxation, and interpersonal psychotherapy are in fact less aversive than exposure therapy, at
least as measured by differences in refusal and
attrition rates. To the degree that each of these
alternative treatments requires effort on the part of
the client, they involve their own type of discomfort. The acceptability and effectiveness of these
therapies for clients who refuse exposure therapy
has thus far received little research attention. It is
possible that repeated exposure might be best
facilitated by first having clients engage in relaxation, cognitive restructuring, or other less distressing techniques, though the use of such techniques
may require sacrifices in therapy efficiency. This is
certainly a topic deserving of further research.
Further head-to-head comparisons of exposureand non-exposure-based treatments would also
help identify individuals who are at risk of refusal
or attrition who may be more likely to benefit from
either type of treatment or perhaps a more gradual
treatment approach. The assessment of aversiveness
through means other than refusal and attrition rates
would also be beneficial. This could be done
through asking various questions related to client
perceptions of treatment, including satisfaction
with therapy, perceptions of the amount of distress
or work required in therapy, and whether he or she
would use the treatment again or recommend it to a
friend. Such basic questions are commonly asked in
product surveys and would also be worth including
in randomized controlled trials.
Furthermore, given that enrollment in randomized controlled trials often involves some willingness
on the part of anxious individuals to confront their
fears (especially if there is some likelihood that they
would be assigned to exposure therapy), examination of treatment perception among non-treatment-

what makes a quality therapy?

seeking individuals with anxiety disorders may be
beneficial in identifying more acceptable alternatives to exposure-based treatments. Studies that do
or do not allow anxious individuals to choose from
among multiple psychotherapies, with varied rationales and degrees of aversiveness, might also be
helpful toward identifying factors that maximize
treatment utilization and minimize attrition.

treatment length
Treatment length is an important characteristic
related to each feature of therapy quality I have
proposed and is discussed at several points in this
essay. Meta-analyses of brief interventions have
generally found a dose relationship between number of therapy sessions and greater likelihood of
successful outcome (Howard, Kopta, Krause, &
Orlinsky, 1986). However, due to the constraints of
cost, time, and effort, fewer sessions are generally
preferred. Hansen, Lambert, and Forman (2002)
noted that clinical trials consisted of an average of
12.7 sessions and produced improvements in 57.6
67.2% of participants; however, their analysis of a
national database of more than 6,000 patients
revealed that the average number of sessions
attended by this sample was less than five and the
improvement rate was approximately 20%. Given
these real-world constraints, treatments with very
few components that can be quickly and effectively
implemented should have great value.
A balance between effort and effectiveness will be
an obvious consideration when attempting to
establish the optimal number of sessions. That is,
if 20 additional sessions contributed to small
decreases in symptoms or marginal increases in
response rates, the inclusion of these extra sessions
in a standard therapy package is probably unwarranted. Costbenefit analyses may lead researchers
to develop multitiered interventions that vary in
length and efficacy. Perhaps longer, more effective
interventions could be reserved for certain settings
that would allow for their full implementation.
Some investigators have varied the number of
sessions and found equivalent outcomes between
full and condensed treatments. For example, researchers have found a condensed 6-session cognitive therapy to be just as effective as the full 12session treatment for panic disorder (Clark et al.,
1999; Kenardy et al., 2003). This research is
important to consider when conducting new
randomized controlled trials, since it would suggest
that this 6-session CBT package is the preferred
treatment to which newer therapies should be
compared. Single-session interventions have also
shown impressive efficacy for both anxiety and
depression (Gawrysiak, Cristopher, & Hopko,

473

2009; Kunik et al., 2001; st et al., 2001).

Generally speaking, treatments that require more
sessions should be able to show improved quality in
some regard (e.g., greater efficacy, feasibility).
The literature on sudden gains might be helpful in
determining the optimal number of sessions for a
particular client (Tang & DeRubeis, 1999). Clients
who experience early, marked improvements may
require fewer sessions to achieve lasting recovery.
Research that examines predictors of sudden gains
or other factors related to chronicity and remission
may be informative in developing a treatment plan
that requires the least number of sessions. Randomized controlled trials could also be conducted that
compare fixed-session frequency to session frequency that is varied based on the experience of sudden gains or pretreatment predictors of remission/
chronicity.
Session length also varies between treatments,
generally lasting between 1 and 2 hours. Again,
given the constraints of time, cost, and effort,
shorter sessions will be preferred. A meta-analysis
by Abramowitz (1996) found that greater length of
therapist-assisted exposure was associated with
improved outcome in exposure and response prevention treatment for OCD. In contrast, van Minnen
and Foa (2006) recently found that PTSD treatment
using 30 min of imaginal exposure in each session was just as effective as treatment using 60 min
of exposure per session. Similar studies comparing
shorter and longer therapy sessions would go a long
way toward improving therapy quality. The burden
will be on investigators using longer sessions to
demonstrate that their treatments have additive value
over those with shorter sessions.

amount of required homework

Despite the fact that homework is quite integral to
many psychotherapies, there has been little effort to
quantify the amount of homework required by
various treatments and assess the effort such
homework requires. Available evidence suggests
both the quantity and quality of homework
completion is predictive of treatment outcome
(Kazantzis, Deane, & Ronan, 2000; Mausbach,
Moore, Roesch, Cardenas, & Patterson, 2010).
While Schmidt and Woolaway-Bickel (2000) found
clinician-rated quality and percentage of homework
assignments completed were associated with treatment outcome for CBT for panic disorder, client
reports of number of hours spent on homework was
not associated with outcome. Other researchers
examining homework compliance and behavior
therapy outcome for individuals with OCD or
panic disorder with agoraphobia found no relationship between therapy outcome and quantity or

474

cougle

quality of homework (Woods, Chambless, &

Steketee, 2002). Interestingly, these researchers
noted that their clients spent an average of
46 hours on homework throughout the course of
treatment. The absence of an association between
homework and outcome may be due to unreliability
of client self-report. Regardless, these findings
suggest that the amount of assigned homework
could be scaled back without harming treatment
efficacy. Clinical trials that experimentally vary the
amount of homework could do much to enhance
therapy quality. Generally speaking, if two different
treatments produce equivalent outcomes, the treatment requiring less and easier homework is
preferred. 3 Reducing the amount of assigned
homework may also reduce the risk of attrition.

cost of therapy
The expected cost of therapy is an obvious
consideration when evaluating its quality, and
many features of therapy discussed in this essay
have cost implications. Duration and number of
treatment sessions are important contributors. The
format in which therapy is delivered is also a factor,
with group treatments costing less to clients than
individual treatment (Otto, Pollack, & Maki,
2000). Computer-aided treatments (Craske et al.,
2009) will have great advantages in terms of costeffectiveness. Relatively parsimonious therapies
that can be effectively delivered by paraprofessionals with little clinical training will also help reduce therapy expense (and increase dissemination),
as paraprofessionals will likely charge less for therapy than doctoral-level psychologists (Christensen
& Jacobson, 1994). The administration of psychoeducation and other treatment components through
self-study modules can also help minimize therapist
contact and reduce cost; this was primarily how
Clark et al. (1999) condensed the aforementioned
12-session treatment for panic disorder to 6 sessions. In short, there are many opportunities for
reducing the cost of psychotherapy.

The Importance of Efficiency in

Clinical Interventions
One of the hallmarks of a quality therapy should be
its efficiency. In general, highly efficient therapies
will not include ineffective strategies and will seek
to maximize the speed of symptom decline. The
most efficient interventions should also demonstrate transdiagnostic utility.
3
On a related point, I could not identify any published
randomized controlled trial that compared the amount of homework required and completed between two different empirically
supported treatments.

speed of symptom reduction

A quality therapy will seek to maximize the speed at
which symptoms are reduced. The importance of
rapid symptom reduction lies in the impairment and
comorbidity associated with the symptoms themselves, the possibility of treatment dropout (resulting in a lower likelihood of ever benefiting from
treatment), the low number of sessions attended by
many clients, and of course the wishes of clients
to reduce symptoms as quickly as possible. The
expected speed of symptom reduction is no doubt
one reason why many individuals with psychiatric
disorders choose medication over psychotherapy,
and there is some evidence to suggest medications
reduce symptoms more rapidly (Heimberg et al.,
1998).
Speed of symptom reduction should be an
obvious consideration for treatment researchers,
though it is often neglected in outcome research.
Most studies assess and report symptoms only at
pre- and posttreatment. More frequent assessments
would help determine differences in speed of effects
between therapies as well as the optimal number
of recommended treatment sessions. Component
analyses would also help identify ineffective or
less effective treatment procedures that could be
replaced with faster, more efficient methods.
Research assessing rates and predictors of sudden
gains for particular treatments (Tang & DeRubeis,
1999) would also be relevant toward this aim.
transdiagnostic utility
Psychiatric disorders rarely occur alone (Brown,
Campbell, Lehman, Grisham, & Mancill, 2001);
thus, the quality of a therapy may be determined in
part by its transdiagnostic utility. Such utility can be
demonstrated in multiple ways, including the impact
of an intervention on comorbid conditions, the ease
with which components from an intervention can be
utilized or adapted for treating multiple disorders,
and whether an intervention comes from a theoretical
perspective broad enough to account for multiple
disorders (cf. Barlow, Allen, & Choate, 2004).
These features of therapy have several implications for efficiency. First, the degree to which
therapy impacts maintaining factors occurring
across disorders may affect overall treatment
outcome, as well as the total number of sessions
required. Second, therapists and clients will be able
to transition more easily between specific interventions for different disorders if these procedures are
very similar or come from similar theoretical
perspectives. This might allow them to consolidate
and build on therapeutic gains. Third, a protocol
that can be used for treating multiple disorders

what makes a quality therapy?

may require less overall training and less time spent
on assessment and case formulation for multiple
disorders. It might also allow for diagnostic
heterogeneity in a group therapy setting.
There is now an abundance of empirical support
for CBT for various disorders (Norton & Price,
2007). The explanatory power of cognitive theory
is no doubt a compelling reason to use CBT.
However, a potential weakness of CBT is the rise of
so many disorder-specific dysfunctional beliefs. For
example, recent cognitive-behavioral approaches to
OCD target beliefs related to inflated responsibility,
perfectionism, intolerance for uncertainty, overimportance of thoughts, control of thoughts, and
overestimation of threat (McLean et al., 2001).
Maladaptive beliefs about memory have also been
noted (Cougle, Salkovskis, & Wahl, 2007). For
PTSD, dysfunctional beliefs related to self-blame,
dangerousness of the world, and emotional reactions to the trauma have been proposed (Foa,
Ehlers, Clark, Tolin, & Orsillo, 1999), and cognitive processing therapy in particular focuses on
dysfunctional beliefs related to safety, trust, power
and control, self-esteem, self-intimacy, and intimacy with others (Resick & Schnicke, 1993). Targeting so many different disorder-specific beliefs may
negatively impact therapy efficiency. In general, the
lower the number of targeted maintaining factors,
the more efficiently they may be addressed. I should
note here that one of the attractive features of
acceptance and commitment therapy (ACT; Hayes
et al., 1996) is its emphasis on experiential
avoidance as a key maintaining factor occurring
across psychiatric disorders.
Transdiagnostic limitations can be found in other
therapies, as well. For example, interpersonal
psychotherapy has established efficacy for the
treatment of depression (Frank & Spanier, 1995),
but there is less support for its use with OCD or
panic disorder; thus, it may be less efficient to use
this treatment for depressed clients with certain
comorbid profiles. In addition, despite the fact that
prolonged exposure and eye movement desensitization and reprocessing (EMDR; Shapiro, 1995)
are equivalent in efficacy for PTSD (Seidler &
Wagner, 2006), the degree to which prolonged
exposure complements and overlaps with exposurebased treatments for other anxiety disorders makes
it preferable to EMDR, which represents a more
distinct trauma-oriented approach. 4
Recently, researchers have created treatment protocols for use with multiple disorders. A transdiag4

EMDR has also been heavily criticized by many for including

components with no incremental value (Herbert, Lilienfeld, & Lohr
et al., 2000).

475

nostic treatment for emotional disorders has been

developed that focuses on emotional awareness, cognitive restructuring, emotional and behavioral avoidance, and exposure (Ellard, Fairholme, Boisseau,
Farchione, & Barlow, 2010). Group therapy protocols have also been produced for the treatment
of anxiety disorders (see this issue for Norton, 2012this issue; and Schmidt, Buckner, Pusser, WoolawayBickel, & Preston, 2012-this issue). Whether such
interventions represent true advances in therapeutic
efficiency remains to be seen. Empirical evaluations
of these interventions will hopefully examine (a) how
they compare to existing disorder-specific treatments
in terms of efficacy and required training, (b) the
impact of such interventions on comorbidity,
(c) whether such treatments would be more useful
or efficient than existing disorder-specific treatments
implemented with an equivalent degree of flexibility,
and (d) what procedures in these interventions are
essential for producing symptom reduction.

Quality-Oriented Research Recommendations

I have hopefully covered many different issues
related to therapy quality that are worthy of further
research. In Table 1, I provide a summary of
research recommendations. I have devoted extensive discussion to parsimony in this essay, but there
is currently very little research that has tested its
potential significance. One way in which its importance can be evaluated is by examining the relative
ease and effectiveness with which different treatments and treatment components can be taught
to and implemented by community clinicians and
paraprofessionals. This would be extremely relevant to treatment dissemination efforts. The types
of randomized controlled training studies conducted by Strosahl, Hayes, Bergan, and Romano
(1998), who assigned therapists to receive ACTbased training or a control condition, would be
especially beneficial. Head-to-head comparisons of
training in different therapies, along with comprehensive assessments of client outcomes and therapist proficiencies, are needed to evaluate many of
the features of therapy mentioned in this essay.
Analysis of the decay of proficiency and treatment
component utilization over time would also be
important to incorporate into effectiveness studies.
In addition, mini-effectiveness studies in which
undergraduate research assistants, hospital personnel, or other easily accessible samples are randomly
assigned to training in different approaches to treat
spider fearful, blood phobic, or claustrophobic
individuals in single-session interventions would
address some of these research issues. Efforts
toward increasing parsimony will also be aided by
dismantling studies and component analyses. If

476

cougle

Table 1

Research Recommendations to Improve the Quality of Psychotherapies

Issues Related to Quality
of Therapy

Potential Means to Improve

Increasing Parsimony
- Compare the effectiveness of treatments that vary in complexity and number of components
- Examine the amount of training required to implement different treatments effectively
- Assess client comprehension of various components across different populations
- Test therapies using newly trained clinicians and paraprofessionals
- Conduct dismantling studies and component analyses; use findings from such studies to revise protocol
- Prioritize techniques that can be implemented quickly and do not rely on knowledge or skills
accumulated over multiple sessions
Increasing Ease
Increasing feasibility

- Develop and utilize interventions that are easy to implement in a standard community clinic setting
- Administer treatment through computers or other accessible formats (e.g., group, informal class)
- Identify highly accessible stimuli and situations for exposure therapy

Reducing aversiveness

Reducing treatment length

- Examine or develop less distressing alternatives to exposure therapy and test their efficacy for
individuals who refuse exposure-based treatment
- Develop instruments that fully assess client perception of different treatments and their aversiveness
- Test variations in the number of sessions and session length to help determine optimal quantities for
each
- Assess symptoms frequently to determine the optimal point at which therapy can be discontinued

Reducing the amount of

required homework
Reducing cost

- Examine predictors of sudden gains or early remission to identify whether therapy might be terminated
earlier for certain clients
- Quantify and compare the amount of homework assigned and completed for different treatments
- Test variations in the amount of assigned homework to help determine the most optimal quantity
- Develop treatments that can be administered in a group format
- Use computers and other technologies to aid in the implementation of therapy
- Develop treatments simple enough to be administered by paraprofessionals
- Use take-home readings to replace therapy time spent on psychoeducation and other procedures

Increasing Efficiency
Increasing speed of symptom
reduction

Increasing transdiagnostic
utility

- Use frequent symptom assessments and component analyses to assess the relative speed with which
different treatments and treatment components reduce symptoms
- If possible, frontload components in therapy that are most likely to have the quickest effects on
symptoms
- Develop interventions targeting factors thought to maintain multiple disorders
- Develop interventions that can be utilized or easily adapted for the treatment of multiple disorders
- Utilize interventions that build on or complement existing treatments targeting other disorders
- Compare the impact of different interventions on target problems and comorbid conditions

ineffective or nonessential components are identified through such studies, their removal could
improve therapy protocols. Researchers are likely
to see benefits from making treatments with
established efficacy simpler.
With regard to ease, it will be important for
researchers to develop interventions that are highly
feasible for implementing in a traditional community clinic setting. The extent to which procedures
with questionable feasibility (e.g., the use of
audience exposures) are used in a therapy should
be reported by treatment researchers, as this will
affect the generalizability of treatment outcome

findings. Further efforts should be made to quantify

the optimal number and length of sessions, as well
as the optimal amount of homework required for
effective treatment. Generally speaking, researchers
should attempt to produce highly effective treatments requiring less, easier homework and fewer,
shorter sessions. Research examining sudden gains
or predictors of remission/chronicity may be
informative in determining the optimal session
length for a particular client. In addition, as long
as such efforts do little or no damage to treatment
efficacy, researchers should seek to reduce the
discomfort associated with therapy or develop

477

63.8%
Moderate

High

6.8/15

5.8/12

None

Para-professionals
group

6
Treatment C

Treatment B

Low

High

40 hrs

10 hrs

12 hrs

20 hrs

35 hrs

25 hrs

24.2%

27.2%

Para-professionals
computer-assisted
group
25.4%
30 hrs
15 hrs
30 hrs
Moderate
4
Treatment A

Modest

Strong
65.3%

Strong
70.5%
Somewhat
7.3/12

Number of Session Hrs

Until 50% Symptom
Reduction/Total
Session Hrs
Amount of
Assigned
Homework
Amount of
Assigned
Exposure

Ease

Amount of
Training Required
to Implement
Effectively
Level of
Sophistication
Number of
Components

Parsimony

There has been growing debate over the value of

cognitive techniques in therapy (Longmore &
Worrell, 2007). While most if not all behavioral
interventions include some cognitive procedures
(e.g., discussion of thoughts, feared consequences),
evidence has accumulated that suggests the addition
of more focused cognitive procedures does not
confer any benefit over behavioral activation for
depression (Jacobson et al., 1996) and exposurebased interventions for PTSD (Foa et al., 2005a),
OCD (Whittal et al., 2008), and social anxiety
disorder (Feske & Chambless, 1995). While one
unpublished study found that cognitive restructuring plus exposure led to more favorable outcomes
than exposure alone for the treatment of panic
disorder (Margraf & Schneider, 1991), published
research has demonstrated equivalent outcomes
between CBT and interoceptive exposure (without
in vivo exposure) for this disorder (Arntz, 2002). A
recent meta-analysis also found applied relaxation,

Table 2

Should Cognitive Restructuring Be Part of a

Quality Therapy?

A Hypothetical Comparison of the Quality of Three Anxiety Disorder Treatments

Attrition
Rate

Evidence for
Cost-Effective
Delivery

Efficiency

Overlap With
Treatments
for Other
Disorders

Efficacy

alternative interventions that allow clients to

gradually work up to aversive procedures. More
detailed assessments of perceptions of various
treatment components in both treatment-seeking
and non-treatment-seeking samples would help
establish whether certain procedures may be
substituted for more distressing ones. Direct comparisons of therapies that vary in levels of
aversiveness may also assist researchers in identifying clients who are likely to benefit more from less
distressing treatments.
Therapy efficiency concerns should also lead
researchers to seek to maximize the speed at which
symptoms are reduced. Component analyses and
randomized controlled trials utilizing frequent
assessments (to compare rates of symptom decline
between treatments) would aid such efforts. Lastly,
researchers should develop treatments with high
transdiagnostic value. Ideally, such interventions
would impact maintaining factors appearing across
disorders and comorbid conditions that are not
necessarily the focus of treatment. They would also
include components that could be easily utilized or
adapted for treating multiple disorders.
In Table 2, I provide a hypothetical example of a
quality-oriented comparison of three anxiety disorder treatments. Some indices, including sophistication and amount of overlap with treatments for
other disorders, carry with them a greater degree
of subjective evaluation than others, though perhaps more objective rating systems could be developed that better evaluate treatments on these
characteristics.

Recovery Level of
Rate
Research
Support
(Div. 12)

what makes a quality therapy?

478

cougle

arguably a more parsimonious treatment, to be just

as effective as CBT for generalized anxiety disorder
(Siev & Chambless, 2007).
What is noteworthy about much of the research
demonstrating no incremental value from cognitive
techniques is that it has been carried out mostly by
prominent research teams with allegiances to CBT.
They typically used very skilled clinicians who are
highly proficient in CBT. If additive value from
cognitive procedures was not demonstrated by
these investigators, it is much less likely to be
found among less proficient community clinicians
who undergo less training and have less supervision. To my knowledge, there is no published
research that has found CBT to be more effective
than behaviorally oriented interventions when
implemented by newly trained community clinicians.
Behavioral activation and exposure-based interventions have great advantages over existing
cognitive-behavioral treatments in terms of parsimony. They are likely easier for clients to understand and for therapists to learn and utilize. They
can be quickly implemented and do not typically
rely on knowledge and skills accumulated over
multiple sessions. One study also found behavioral
activation to be more effective than CBT for severe
depression, though it produced similar outcomes to
CBT for less severe depression (Dimidjian et al.,
2006). In addition, exposure and response prevention appears to be more effective than CBT for
OCD when conducted in group format (McLean
et al., 2001). Very brief exposure therapy protocols
have shown remarkable effectiveness (e.g., Powers
et al., 2004), and it is possible that exposure-based
interventions lead to more rapid fear reduction than
cognitive restructuring; to date, this has received
little research attention. While some acknowledge
that the addition of cognitive techniques to
exposure-based interventions does not yield better
outcomes, they maintain that such techniques are
less aversive and have utility for this reason
(Heimberg & Ritter, 2008). However, evidence
for their substitutionary value is underwhelming,
with lower attrition rates revealed for CBT than
behavior therapy for OCD (Whittal et al., 2008)
but comparable rates of attrition between cognitive
interventions and behavior therapy for social
anxiety disorder (Gould et al., 1997) and PTSD
(Hembree et al., 2003). Given the rarity with which
therapists in the general population use empirically
supported treatments (e.g., Becker et al., 2004), it
would behoove researchers to attempt to develop
and disseminate effective interventions that are
most easily learned and implemented. Adding
nonessential, labor intensive cognitive procedures

to simple, effective interventions may hamper

efforts to ensure that therapists use empirically
supported treatments.
Given these treatment outcome findings and the
issues involved with administering more complex
interventions, one might ask whether we as a field
should be disseminating cognitive procedures.
However, it would be important here to distinguish
between different procedures. Behavioral experiments, for example, can be implemented more
quickly and efficiently than cognitive restructuring,
and there is some evidence to suggest that exposure
in the context of a behavioral experiment (i.e.,
identifying threats and evaluating them through
exposure) is more effective than exposure only
(McMillan & Lee, 2010). A recent clinical trial
found CBT that relied on several behavioral
experiments led to better outcomes in the treatment
of social anxiety disorder than exposure therapy
with applied relaxation (Clark et al., 2006). In
addition, metacognitive therapy, which uses a
combination of verbal strategies and behavioral
experiments, appears to be very effective for
generalized anxiety disorder (Wells et al., 2010), a
condition notoriously difficult to treat. It is possible
that cognitive restructuring has unique benefits for
this (highly cognitive) disorder. On the whole,
however, there are many reasons for the field to
carefully evaluate whether cognitive restructuring
specifically should be included as a treatment
component for most psychological disorders.

Balancing Efficacy and Various Features

of Therapy Quality
Several features of therapy quality mentioned in this
essay, such as number and length of sessions,
aversiveness, and cost, may at times conflict with
each other and with goals of symptom reduction.
For example, it is possible that less aversive
interventions may reduce symptoms at a slower
speed than interventions that begin with exposure.
Various forms of cost-effective interventions (e.g.,
treatments requiring fewer sessions, treatments
delivered via computer or by paraprofessionals)
may also be less effective than more expensive
interventions. Different features may be prioritized
on a case-by-case basis or based on the needs
associated with a particular practice. Less aversive
treatments that may also be less effective or slower
in producing symptom change may be warranted
for a client who refuses exposure therapy. Treatment enrollment and client retention should generally be prioritized over the need for immediate
symptom reduction. Shorter, higher-impact interventions may be warranted if a client is likely to
attend few sessions or show sporadic session

what makes a quality therapy?

attendance. Constraints of session attendance
should inform the approach used to maximize
symptom reduction. From a dissemination standpoint, these concerns may warrant training therapists in different techniques that would be
adaptable for use with clients at risk of refusal or
attrition and those likely to attend relatively few
sessions. Of course, such approaches should be
empirically informed. Additional research that tests
therapies that vary in levels of aversiveness and
duration would help address these issues.

Beyond Symptom Reduction

Over the last several years, psychotherapy research
has focused excessively on pre- to posttreatment
reduction in symptoms as the primary criterion
by which the quality of therapy is determined. This
emphasis is in some ways equivalent to a car
manufacturer being only concerned with whether
their product reliably delivers the driver from point
A to point B, while neglecting its safety, comfort,
speed, and cost. The efficacy of treatments for many
disorders seems to have peaked 20 to 30 years ago
(e.g., Whittal et al., 2008). Such stagnancy is
troubling to many researchers. However, if greater
effort is made toward examining the additional
features of therapy discussed in this essay, this
could lead to new and meaningful improvements in
the quality of our product.
References
Abramowitz, J. S. (1996). Variants of exposure and response
prevention in the treatment of obsessive-compulsive disorder:
A meta-analysis. Behavior Therapy, 27, 583600.
Abramowitz, J. S., Taylor, S., & McKay, D. (2005). Potentials
and limitations of cognitive treatments for obsessivecompulsive disorder. Cognitive Behaviour Therapy, 34,
140147.
Arntz, A. (2002). Cognitive therapy versus interoceptive
exposure as treatment of panic disorder without agoraphobia.
Behaviour Research and Therapy, 40, 325341.
Barlow, D. H., Allen, L. B., & Choate, M. L. (2004). Toward a
unified treatment for emotional disorders. Behavior Therapy,
35, 205230.
Becker, C. B., Zayfert, C., & Anderson, E. (2004). A survey of
psychologists attitudes towards and utilization of exposure
therapy for PTSD. Behaviour Research and Therapy, 42,
277292.
Bleiberg, K. L., & Markowitz, J. C. (2005). A pilot study of
interpersonal psychotherapy for posttraumatic stress disorder.
American Journal of Psychiatry, 162, 181183.
Bright, J. I., Baker, K. D., & Neimeyer, R. A. (1999).
Professional and paraprofessional group treatments for
depression: A comparison of cognitive-behavioral and
mutual support interventions. Journal of Consulting and
Clinical Psychology, 67, 491501.
Brown, T. A., Campbell, L. A., Lehman, C. L., Grisham, G. R.,
& Mancill, R. B. (2001). Current and lifetime comorbidity
of the DSM-IV anxiety and mood disorders in a large clinical
sample. Journal of Abnormal Psychology, 110, 585599.

479

Chambless, D. (1995). Training in and dissemination of

empirically-validated psychological treatments: Report and
recommendations. Clinical Psychologist, 48, 323.
Chambless, D., Baker, M. J., Baucom, D. H., Beutler, L. E.,
Calhoun, K. S., Crits-Christoph, P., et al. (1998). Update on
empirically validated therapies: II. Clinical Psychologist, 49,
518.
Chambless, D., & Ollendick, T. (2001). Empirically supported
psychological interventions: Controversies and evidence.
Annual Review of Psychology, 52, 685716.
Christensen, A., & Jacobson, N. S. (1994). Who (or what) can
do psychotherapy: The status and challenge of nonprofessional therapies. Psychological Science, 5, 814.
Clark, D. M., Ehlers, A., Hackmann, A., McManus, F., Fennell,
M., Grey, N., et al. (2006). Cognitive therapy versus
exposure and applied relaxation in social phobia: A
randomized controlled trial. Journal of Consulting and
Clinical Psychology, 74, 568578.
Clark, D. M., Salkovskis, P. M., Hackmann, A., Middleton, H.,
Anastasiades, P., & Gelder, M. (1994). Comparison of
cognitive therapy, applied relaxation and imipramine in the
treatment of panic disorder. British Journal of Psychiatry,
164, 759769.
Clark, D. M., Salkovskis, P. M., Hackmann, A., Wells, A.,
Ludgate, J., & Gelder, M. (1999). Brief cognitive therapy for
panic disorder: A randomised controlled trial. Journal of
Consulting and Clinical Psychology, 67, 583589.
Cougle, J. R., Salkovskis, P. M., & Wahl, K. (2007). Perception of
memory ability and confidence in recollections in obsessivecompulsive checking. Journal of Anxiety Disorders, 21,
118130.
Craske, M. G., Rose, R. D., Lang, A., Welch, S. S., CampbellSills, L., Sullivan, G., et al. (2009). Computer-assisted
delivery of cognitive behavioral therapy for anxiety disorders in primary-care settings. Depression and Anxiety, 26,
235242.
Crowther, M. R., Scogin, F., & Norton, M. J. (2010). Treating
the aged in rural communities: The application of cognitivebehavioral therapy for depression. Journal of Clinical
Psychology, 66, 502512.
Cuijpers, P., Van Straten, A., & Warmerdam, L. (2007).
Behavioral activation treatments of depression: A metaanalysis. Clinical Psychology Review, 27, 318326.
Dagnan, D., Chadwick, P., & Proudlove, J. (2000). Toward
assessment of suitability of people with mental retardation
for cognitive therapy. Cognitive Therapy and Research, 24,
627636.
DeRubeis, R. J., Hollon, S. D., Amsterdam, J. D., Shelton, R. C.,
Young, P. R., Salomon, R. M., et al. (2005). Cognitive therapy
vs. medications in the treatment of moderate to severe
depression. Archives of General Psychiatry, 62, 409416.
Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B.,
Kohlenberg, R. J., Addis, M. E., et al. (2006). Randomized
trial of behavioral activation, cognitive therapy, and
antidepressant medication in the acute treatment of adults
with major depression. Journal of Consulting and Clinical
Psychology, 74, 658670.
Ellard, K. K., Fairholme, C. P., Boisseau, C. L., Farchione, T. J.,
& Barlow, D. H. (2010). Unified protocol for the
transdiagnostic treatment of emotional disorders: Protocol
development and initial outcome data. Cognitive and
Behavioral Practice, 17, 88101.
Feske, U., & Chambless, D. L. (1995). Cognitive behavioral
versus exposure only treatment for social phobia: A metaanalysis. Behavior Therapy, 26, 695720.
Foa, E. B., Ehlers, A., Clark, D. M., Tolin, D. F., & Orsillo, S. M.
(1999). The Posttraumatic Cognitions Inventory (PTCI):

480

cougle

Development and validation. Psychological Assessment, 11,

303314.
Foa, E. B., Hembree, E. A., Cahill, S. P., Rauch, S. A. M., Riggs,
D. S., Feeny, N. C., et al. (2005a). Randomized trial of
prolonged exposure for posttraumatic stress disorder with
and without cognitive restructuring: Outcome at academic
and community clinics. Journal of Consulting and Clinical
Psychology, 73, 953964.
Foa, E. B., Liebowitz, M. R., Kozak, M. J., Davies, S., Campeas,
R., Franklin, M. E., Huppert, J. D., et al. (2005b). Treatment
of obsessive-compulsive disorder by exposure and ritual
prevention, clomipramine, and their combination: A randomized, placebo controlled trial. American Journal of
Psychiatry, 162, 151161.
Frank, E., & Spanier, C. (1995). Interpersonal psychotherapy for
depression: Overview, clinical efficacy, and future directions.
Clinical Psychology: Science and Practice, 2, 349369.
Franklin, M., & Foa, E. B. (1998). Cognitive-behavioral
treatments for obsessive compulsive disorder. In P. N. A. J.
Gorman (Ed.), A guide to treatments that work
(pp. 339357). New York: Oxford University Press.
Friedberg, R. D., & McClure, J. M. (2002). Clinical practice of
cognitive therapy with children and adolescents: The nuts
and bolts. New York: Guilford Press.
Gawrysiak, M., Cristopher, N., & Hopko, D. R. (2009).
Behavioral activation for moderately depressed university
students: Randomized controlled trial. Journal of Counseling Psychology, 56, 468475.
Gega, L., Norman, I., & Marks, I. (2007). Computer-aided vs.
tutor-delivered teaching of exposure therapy for phobia/panic:
A randomised controlled trial with pre-registration nursing
students. International Journal of Nursing Studies, 44,
397405.
Gould, R. A., Buckminster, S., Pollack, M. H., Otto, M. W., &
Yap, L. (1997). Cognitive-behavioral and pharmacological
treatment for social phobia: A meta-analysis. Clinical
Psychology: Science and Practice, 4, 291306.
Haaga, D. A. F., DeRubeis, R. J., Stewart, B. L., & Beck, A. T.
(1991). Relationship of intelligence with cognitive therapy
outcome. Behaviour Research and Therapy, 29, 277281.
Hansen, N. B., Lambert, M. J., & Forman, E. M. (2002). The
psychotherapy dose-response effect and its implications for
treatment delivery services. Clinical Psychology: Science and
Practice, 9, 329343.
Hayes, S. C., Wilson, K. G., Gifford, E. V., Follette, V. M., &
Strosahl, K. (1996). Experiential avoidance and behavioral
disorders: A functional dimensional approach to diagnosis
and treatment. Journal of Consulting and Clinical Psychology, 64, 11521168.
Heimberg, R. G., & Becker, R. E. (2002). Cognitive-behavioral
group therapy for social phobia: Basic mechanisms and
clinical strategies. New York: Guilford Press.
Heimberg, R. G., Liebowitz, M. R., Hope, D. A., Schneier, F. R.,
Holt, C. S., Welkowitz, L. A., et al. (1998). Cognitive
behavioral group therapy vs. phenelzine therapy for social
phobia: 12-week outcome. Archives of General Psychiatry,
55, 11331141.
Heimberg, R. G., & Ritter, M. R. (2008). Cognitive behavioral
therapy and acceptance and commitment therapy for the
anxiety disorders: Two approaches with much to offer.
Clinical Psychology: Science and Practice, 15, 296298.
Hembree, E. A., Foa, E. B., Dorfan, N. M., Street, G. P.,
Kowalski, J., & Tu, X. (2003). Do patients drop out
prematurely from exposure therapy for PTSD? Journal of
Traumatic Stress, 16, 555562.
Herbert, J. D., Lilienfeld, S. O., Lohr, J. M., Montgomery, R. W.,
O'Donohue, W. T., Rosen, G. M., et al. (2000). Science and

pseudoscience in the development of eye movement

desensitization and reprocessing: Implications for clinical
psychology. Clinical Psychology Review, 20, 945971.
Howard, K. I., Kopta, S. M., Krause, M. K., & Orlinsky, D. E.
(1986). The dose-effect relationship in psychotherapy.
American Psychologist, 41, 159164.
Issakidis, C., & Andrews, G. (2002). Service utilisation for
anxiety in an Australian community sample. Social Psychiatry and Psychiatric Empidemiology, 37, 153163.
Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E.,
Koerner, K., Gollan, J. K., et al. (1996). A component analysis
of cognitive-behavioral treatment for depression. Journal of
Consulting and Clinical Psychology, 64, 295304.
Jacobson, N. S., & Hollon, S. D. (1996). Prospects for
future comparisons between drugs and psychotherapy:
Lessons from the CBT-versus-pharmacotherapy exchange.
Journal of Consulting and Clinical Psychology, 64,
104108.
Kazantzis, N., Deane, F. P., & Ronan, K. R. (2000). Homework
assignments in cognitive and behavioral therapy: A metaanalysis. Clinical Psychology: Science and Practice, 7,
189202.
Kenardy, J. A., Dow, M. G. T., Johnston, D. W., Newman, M.
G., Thomson, A., & Taylor, C. B. (2003). A comparison of
delivery methods of cognitive-behavioral therapy for panic
disorder: An international multicenter trial. Journal of
Consulting and Clinical Psychology, 71, 10681075.
Kunik, M. E., Braun, U., Stanley, M. A., Wristers, K., Molinari,
V., Stoebner, D., & Orengo, C. A. (2001). One session
cognitive behavioral therapy for elderly patients with chronic
obstructive pulmonary disease. Psychological Medicine, 31,
717723.
Lindsay, W. R., Howells, L., & Pitcaithly, D. (1993). Cognitive
therapy for depression with individuals with intellectual
disabilities. British Journal of Medical Psychology, 66,
135141.
Longmore, R. J., & Worrell, M. (2007). Do we need to
challenge thoughts in cognitive behavior therapy? Clinical
Psychology Review, 27, 173187.
Margraf, J., & Schneider, S. (1991, November 26). Outcome
and active ingredients of cognitive-behavioural treatments
for panic disorder. Paper presented at the Annual Conference
of Association for Advancement of Behaviour Therapy,
New York, NY.
Mausbach, B. T., Moore, R., Roesch, S., Cardenas, V., &
Patterson, T. L. (2010). The relationship between homework compliance and therapy outcomes: An updated
meta-analysis. Cognitive Therapy and Research, 34(5),
429438.
Mazzucchelli, T., Kane, R., & Rees, C. (2009). Behavioral
activation treatments for depression in adults: A metaanalysis and review. Clinical Psychology: Science and
Practice, 16, 383411.
McLean, P. D., Whittal, M. L., Thordarson, D., Taylor, S.,
Sochting, I., Koch, W. J., et al. (2001). Cognitive versus
behavior therapy in the group treatment of obsessive
compulsive disorder. Journal of Consulting and Clinical
Psychology, 69, 205214.
McMillan, D., & Lee, R. (2010). A systematic review of
behavioral experiments vs. exposure alone in the treatment
of anxiety disorders: A case of exposure while wearing the
emperor's new clothes? Clinical Psychology Review, 30,
467478.
Norton, P. J. (2012). A randomized clinical trial of transdiagnostic cognitve-behavioral treatments for anxiety disorder
by comparison to relaxation training. Behavior Therapy,
43, 506517 (this issue).

what makes a quality therapy?

Norton, P. J., & Price, E. C. (2007). A meta-analytic review of
adult cognitive-behavioral treatment outcome across the
anxiety disorders. Journal of Nervous and Mental Disease,
195, 521531.
st, L. G. (1987). Applied relaxation: Description of a coping
technique and review of controlled studies. Behaviour
Research and Therapy, 25, 379409.
st, L. G., Alm, T., Brandberg, M., & Breitholtz, E. (2001).
One vs. five sessions of exposure and five sessions of
cognitive therapy in the treatment of claustrophobia.
Behaviour Research and Therapy, 39, 167183.
Otto, M. W., Pollack, M. H., & Maki, K. M. (2000).
Empirically supported treatments for panic disorder:
Costs, benefits, and stepped care. Journal of Consulting
and Clinical Psychology, 68, 556563.
Perepletchikova, R., & Kazdin, A. E. (2005). Treatment
integrity and therapeutic change: Issues and research
recommendations. Clinical Psychology: Science and Practice,
12, 365383.
Powers, M. B., Smits, J. A. J., & Telch, M. J. (2004).
Disentangling the effects of safety-behavior utilization and
safety-behavior availability during exposure-based treatment:
A placebo-controlled trial. Journal of Consulting and Clinical
Psychology, 72, 448454.
Reardon, M. L., Cukrowicz, K. C., Reeves, M. D., & JoinerJr.,
T. E. (2002). Duration and regularity of therapy attendance
as predictors of treatment outcome in an adult outpatient
population. Psychotherapy Research, 12, 273285.
Resick, P. A., & Schnicke, M. K. (1993). Cognitive processing
therapy for rape victims. Newbury Park, CA: Sage.
Rizvi, S. L., Vogt, D. S., & Resick, P. A. (2009). Cognitive and
affective predictors of treatment outcome in cognitive
processing therapy and prolonged exposure for posttraumatic stress disorder. Behaviour Research and Therapy, 47,
737743.
Schmidt, N. B., Buckner, J. D., Pusser, A., Woolaway-Bickel, K.,
& Preston, J. L. (2012). Randomized controlled trial of false
safety behavior elimination therapy: A unified cognitive
behavioral treatment for anxiety psychopathology. Behavior
Therapy, 43, 518532 (this issue).
Schmidt, N. B., & Woolaway-Bickel, K. (2000). The effects
of treatment compliance on outcome in cognitivebehavioral therapy for panic disorder: Quality versus
quantity. Journal of Consulting and Clinical Psychology,
68, 1318.
Seidler, G. H., & Wagner, F. E. (2006). Comparing the efficacy
of EMDR and trauma focused cognitive behavioural
therapy in the treatment of PTSD: A meta-analytic study.
Psychological Medicine, 36, 15151522.

481

Shapiro, F. (1995). Eye movement desensitization and reprocessing:

Basic principles, protocols and procedures. New York: Guilford
Press.
Siev, J., & Chambless, D. L. (2007). Specificity of treatment
effects: Cognitive therapy and relaxation for generalized
anxiety and panic disorders. Journal of Consulting and
Clinical Psychology, 75, 513522.
Strosahl, K. D., Hayes, S. C., Bergan, J., & Romano, P. (1998).
Does field based training in behavior therapy improve
clinical effectiveness? Evidence from the acceptance and
commitment therapy training project. Behavior Therapy,
29, 3564.
Sturmey, P. (2004). Cognitive therapy with people with
intellectual disabilities: A selective review and critique.
Clinical Psychology and Psychotherapy, 11, 223232.
Tang, T., & DeRubeis, R. J. (1999). Sudden gains and critical
sessions in cognitive-behavioral therapy for depression.
Journal of Consulting and Clinical Psychology, 67,
894904.
Van Minnen, A., & Foa, E. B. (2006). The effect of imaginal
exposure length on outcome of treatment for PTSD. Journal
of Traumatic Stress, 19, 427438.
Webb, C. A., Derubeis, R. J., & Barber, J. P. (2010). Therapist
adherence/competence and treatment outcome: A metaanalytic review. Journal of Consulting and Clinical Psychology,
78, 200211.
Wells, A., Welford, M., King, P., Papageorgiou, C., Wisely, J.,
& Mendel, E. (2010). A pilot randomized trial of
metacognitive therapy vs applied relaxation in the treatment
of adults with generalized anxiety disorder. Behaviour
Research and Therapy, 48, 429434.
Whittal, M., Robichaud, M., Thordarson, D., & McLean, P.
(2008). Group and individual treatment of obsessivecompulsive disorder using cognitive therapy and exposure
plus response prevention: A 2-year follow-up of two
randomized trials. Journal of Consulting and Clinical
Psychology, 76, 10031014.
Woods, C. M., Chambless, D. L., & Steketee, G. (2002).
Homework compliance and behavior therapy outcome for
panic with agoraphobia and obsessive compulsive disorder.
Cognitive Behaviour Therapy, 31, 8895.
Yeaton, W., & Sechrest, L. (1981). Critical dimensions in the
choice and maintenance of successful treatments: Strength,
integrity, and effectiveness. Journal of Consulting and
Clinical Psychology, 49, 156167.
R E C E I V E D : April 23, 2010
A C C E P T E D : December 14, 2010
Available online 25 May 2011