Vous êtes sur la page 1sur 7

CanJPsychiatry 2015;60(12):534–540

In Review

Evidence-Based Practice: Separating Science From


Pseudoscience

Catherine M Lee, PhD1; John Hunsley, PhD2


1
Professor, School of Psychology, University of Ottawa, Ottawa, Ontario.
Correspondence: School of Psychology, University of Ottawa, Ottawa, ON K1N 6N5; catherine.lee@uottawa.ca.
2
Professor, School of Psychology, University of Ottawa, Ottawa, Ontario.

Evidence-based practice (EBP) requires that clinicians be guided by the best available
Key Words: pseudoscience, evidence. In this article, we address the impact of science and pseudoscience on
evidence-based practice, psychotherapy in psychiatric practice. We describe the key principles of evidence-based
critical appraisal, heuristics and
intervention. We describe pseudoscience and provide illustrative examples of popular
biases
intervention practices that have not been abandoned, despite evidence that they are not
efficacious and may be harmful. We distinguish efficacy from effectiveness, and describe
Received July, revised, and
accepted September 2015.
modular approaches to treatment. Reasons for the persistence of practices that are not
evidence based are examined at both the individual and the professional system level.
Finally, we offer suggestions for the promotion of EBP through clinical practice guidelines,
modelling of scientific decision making, and training in core skills.

WWW

Pratique fondée sur des données probantes : séparer la science de


la pseudoscience
La pratique fondée sur des données probantes (PFDP) exige que les cliniciens soient
guidés par les meilleures données probantes. Dans cet article, nous abordons l’effet de la
science et de la pseudoscience sur la psychothérapie dans la pratique psychiatrique. Nous
décrivons les principaux principes d’une intervention fondée sur des données probantes.
Nous décrivons la pseudoscience et offrons des exemples illustrant les pratiques
d’intervention populaires qui n’ont pas été abandonnées, malgré qu’il soit prouvé qu’elles ne
sont pas efficaces et peuvent être nuisibles. Nous distinguons entre efficience et efficacité,
et décrivons les approches modulaires du traitement. Les raisons de la persistance des
pratiques qui ne sont pas fondées sur des données probantes sont examinées au niveau
individuel et professionnel du système. Enfin, nous offrons des suggestions pour la
promotion de la PFDP par des guides de pratique clinique, la modélisation de la prise de
décisions scientifiques, et la formation en compétences de base.

B urgeoning rates of mental disorder in most countries


(including Canada) and a global shortage of
psychiatrists1 require judicious allocation of resources to
psychiatry. We begin by discussing key aspects of the
EBP movement in mental health care. EBP involves initial
assessment, followed by the selection of a treatment,
address mental health problems.2 The promotion of EBP and ongoing monitoring to determine the usefulness of
ensures that health care yields maximum returns.3 The the treatment for the patient. In this article, we focus on
commitment to EBP in Canadian psychiatry was articulated psychotherapy, presenting examples that are evidence
in the submission made by the RCPSC to the Romanow based as well as those that are not (see online Supplemental
Commission on Health Care in 2002. Requirements for Materials for a review of issues in evidence-based
psychiatry residency education from the RCPSC and the assessment). Finally, we examine reasons for the reluctance
US Accreditation Council for Graduate Medical Education of many clinicians to embrace EBP and conclude with
include competence in the provision of EBP, although they constructive strategies to promote EBP.
do not address how training should be delivered or outcomes
should be measured.4 Evidence-Based Practice
With the broad professional context of EBP in mind, There is a long-standing emphasis on the importance of
we address the impact of science and pseudoscience on research evidence in psychiatry. Almost a century ago,

534 W La Revue canadienne de psychiatrie, vol 60, no 12, décembre 2015 www.LaRCP.ca
Evidence-Based Practice: Separating Science From Pseudoscience

Eugen Bleuler criticized the “autistic and undisciplined


thinking”5, p 1 in medicine that allowed unsubstantiated Clinical Implications
treatments to flourish. Bleuler suggested that the motivation • EBP is an essential safeguard against pseudoscience
to alleviate suffering, combined with limited knowledge, and potentially harmful assessment and treatment
methods.
created conditions in which physicians and patients are
fooled into believing that an ineffective treatment works. • Even interventions that appear to be plausible can be
ineffective or even iatrogenic.
There is no ethically defensible argument against the need
to offer treatments demonstrated to work for patients Limitations
with similar problems and compatible with their needs • It can be difficult to distinguish between science and
and preferences. Still, a thorny question remains: What some forms of pseudoscience.
constitutes evidence that an intervention works? • Research on the effectiveness of training psychiatrists in
EBP is still in its infancy.
One model for choosing psychotherapies is comparable to
that used to evaluate medications: establish methodological
criteria for evaluating studies, create lists of treatments
that meet these criteria, and promote these treatments. This The American Psychological Association Presidential
approach was adopted in the early 1990s, by task forces of the Task Force on Evidence-Based Practice18 defined EBP as
American Psychological Association’s Society of Clinical the integration of the best available research and clinical
Psychology, to generate lists of ESTs.6 Classification as an expertise within the context of client characteristics,
EST required sufficient research of high quality (multiple culture, values, and treatment preferences. The assertion
RCTs or single-participant designs), demonstrating efficacy that treatment should be informed by research but
of the intervention in clients with a specific condition. determined by other clinical information, client choice, and
the likely costs and benefits of treatments has sometimes
These initiatives underlined that efficacious psychosocial
been interpreted as suggesting that research evidence is the
treatments for many disorders across the lifespan exist
least important consideration in EBP.19 Additionally, the
(see also Roth and Fonagy,7 Fonagy et al,8,9 and Nathan
American Psychological Association’s statement was silent
and Gorman10–13). This encouraging news has neither been
on the need to ensure that clients’ views were based on
embraced by some clinicians as positively as one might
accurate rather than inaccurate information. Clients often
expect nor led to the widespread adoption of ESTs.14,15
request a treatment they have heard about from a friend or
Critics of ESTs have expressed concerns about the scientific
read about online, unaware that it is not backed by scientific
soundness of the endeavour and its potential negative
evidence. In such situations, clinicians bear a responsibility
impact on clinicians who do not offer ESTs.16 They noted
to educate their clients to ensure that their choices are
that a treatment with a positive, but limited, evidence base
informed by sound knowledge.
could fall short of EST criteria. The EST initiative was an
important step in the development and promotion of EBP, The Canadian Psychological Association Task Force on
but it is not synonymous with EBP. Originating in Canada Evidence-Based Practice of Psychological Treatments20
and the United Kingdom, evidence-based medicine is based asserted that EBP should rely, first and foremost, on peer-
on the premise that the application of empirical knowledge reviewed research. The task force recognized the potential
improves patient care.3 This movement spread to the United contribution of diverse methodologies to yield evidence
States17 and other countries. The term evidence based to guide practice, but established a hierarchy in which
entails the synthesis of information from a wide array the strongest weight is given to research that is based on
of sources to guide clinicians to select the best available data from the highest-quality research designs and that has
treatment options.17 Training in EBP encompasses attention been scrutinized by peer review. Clinicians are exhorted
to the content of the evidence base and to the process of to apply their knowledge of the best available research,
evidence-based decision making.4 Sources of information considering client characteristics, cultural backgrounds,
typically considered in EBP include systematic data, and treatment preferences. Further, they are expected to
clinical expertise, and patient preferences. continuously monitor the effects of treatment and to adjust
treatment when appropriate. The monitoring of treatment
Two North American psychological associations created task effects is a key element of EBP and has been shown to
forces to operationalize evidence-based psychotherapies. have a substantial positive impact on treatment outcomes.21
Although surveys indicate that mental health professionals
underuse patient-report measures when making decisions
Abbreviations
about clinical services,22,23 there are widely available sources
EBP evidence-based practice that summarize psychometrically sound instruments that
EST empirically supported treatment can be used for treatment planning and monitoring.24,25
RCT randomized controlled trial Additionally, recent initiatives in the United States have
RCPSC Royal College of Physicians and Surgeons of Canada led to the widespread availability of psychometrically
TFT Thought Field Therapy sound, patient-reported health status and social well-being
measures that can also be used for service delivery decisions

www.TheCJP.ca The Canadian Journal of Psychiatry, Vol 60, No 12, December 2015 W 535
In Review

(see Patient Reported Outcomes Measurement Information mean that one causes the other). Clearly, it is insufficient
System [PROMIS] Network26). to think scientifically only when conducting research: one
must think scientifically in all aspects of one’s professional
Science, Compared With Pseudoscience activities, including clinical services.
With the move toward scientifically supported practice, one
commonly sees monikers, such as scientifically proven or Evidence-Based Treatment
evidence-informed, attached to descriptions of treatments. Prior to the 1960s, few studies had evaluated psychotherapy
The cachet of scientific support is a potent marketing outcome, but, since the 1970s, thousands of studies have
tool. Lilienfeld et al27 viewed science and pseudoscience filled this gap. There are now evidence-based treatments
as existing along a continuum. Features of pseudoscience for many mental health disorders in children, adolescents,
include the following: and adults.9,12 However, less emphasis has been placed on
1) The overuse of ad hoc hypotheses to account for the need to desist from delivering interventions that are
negative research findings. inefficacious or harmful.

2) Avoidance of peer review. For example, media reports of natural and human-made
tragedies almost inevitably end with the statement that
3) Emphasis on confirmation rather than refutation. counsellors will be available on site to assist. Given the
4) Lack of connection with basic or applied research. woeful shortage of services for mental health problems, it
is striking that such services are commonly offered.35 The
5) Overreliance on anecdotal evidence.
rationale for postdisaster debriefing is intuitively appealing.
6) Reversed burden of proof in which proponents of If one could protect people from developing posttraumatic
a technique demand that critics refute claims of stress disorder by devoting a few hours to hearing their
treatment efficacy. stories, it would be time well invested. Nevertheless,
The greater the number of such features, the more likely a literature reviews do not support the efficacy of single
practice is based on pseudoscience rather than science. or multiple sessions of debriefing and suggest that it can
sometimes be harmful, as it may impede natural recovery
TFT, a treatment applied to mood, anxiety, and trauma- processes.36,37
related disorders, is a prime example of practice founded
on pseudoscience. TFT is based on the premise that bodily Another program based on popular belief is Scared
energy imbalances cause negative emotions. Treatment is Straight. Prison visits are designed to deter youth on a
purported to rectify imbalances by tapping on acupuncture trajectory toward delinquency by arousing fear about the
meridians.28 Virtually no peer-reviewed research supports life they could expect if they engaged in crime.38 Fuelled
this treatment rationale. With only methodologically weak by anecdotal reports suggesting high success rates, Scared
reports available in the literature, the so-called science cited Straight was widely implemented in the United States.
to support TFT is primarily anecdotal and does not rule Echoing the pattern found for debriefing, reviews of RCTs
out placebo effects.29–31 Despite these criticisms, the TFT provided no evidence that Scared Straight or other juvenile
website32 continues to advance unsupported claims about awareness programs are efficacious; to the contrary, they
TFT’s ability to cure almost any emotional problem. boost the odds of future offending.39

Clinicians may find it difficult to resist the pull of anecdotal An important distinction in treatment outcome research is
evidence, such as unsystematic clinical observations, in that between efficacy and effectiveness.40 Efficacy refers
decision making. Dramatic changes in patient functioning to evidence of treatment effects obtained in controlled
following an intense session may appear to provide research, whereas effectiveness refers to evidence of
more compelling evidence of a treatment’s impact than treatment effects as evaluated in the real world. Skeptics
do the results of RCTs or meta-analyses. Although of EBP argue that findings from efficacy trials, often
anecdotal evidence can inform hypotheses to be evaluated conducted in academic training centres, have limited
systematically, such evidence should not be equated with applicability to services delivered in the exigencies of real
scientific data. Without the controls afforded by scientific clinical practice, in which patients have multiple diagnoses
practices and scientific thinking, unsystematic clinical and severe problems.15 Nevertheless, Jensen-Doss and
observations can lead to erroneous conclusions about the Weisz41 found no evidence of poorer treatment effects in
value of a clinical procedure. Lilienfeld et al33,34 enumerated youth with multiple problems. Kazdin and Whitley42 found
over 2-dozen cognitive errors that can contribute to spurious that comorbidity was associated with greater change in
therapeutic effectiveness. These include naive realism (for young people with disruptive behaviour disorders who
example, “If I saw it, it must be true”), confirmation bias received evidence-based parent training or problem-solving
(looking only for evidence that supports an hypothesis treatments. These findings suggest that evidence-based
and failing to look for evidence that refutes it), illusory services are useful to people with multiple problems.
correlation (the co-occurrence of 2 events or characteristics Using a benchmarking strategy, in which results of efficacy
does not mean that they are linked), and illusory causation trials are used as a point of comparison, Hunsley and
(the co-occurrence of 2 events or characteristics does not Lee43 found that effectiveness studies of evidence-based

536 W La Revue canadienne de psychiatrie, vol 60, no 12, décembre 2015 www.LaRCP.ca
Evidence-Based Practice: Separating Science From Pseudoscience

treatments for adults reported comparable completion and How Can We Understand Persistence in
improvement rates to those obtained in efficacy trials. Offering Services That Are Not Evidence
A meta-analysis of effectiveness studies of cognitive- Based?
behavioural therapy for adult anxiety disorders similarly From the treatment of scurvy, to routine handwashing to
found that the mean treatment outcome in over 50 studies prevent infection, the history of health care is replete with
was consistent with results from efficacy studies.44 Lee et al45 examples of long delays between the demonstration that
examined treatment effectiveness studies in regular clinical a practice is efficacious and its adoption in routine care.50
settings for children and adolescents. Across studies, more Similarly, clinicians do not always abandon a mental
than 75% of patients completed the services. Improvement health service that is ineffective or harmful. To understand
rates for internalizing problems were comparable to those why clinicians remain loyal to unsupported or iatrogenic
reported in efficacy trials. There was greater variability practices and why they do not embrace efficacious practices,
in outcomes for parenting interventions for disruptive we must consider both individual and contextual variables.
behaviour problems, with several studies yielding superior Individual-level variables include challenges in keeping
results compared with the benchmark, and a smaller number up with the scientific literature, perceived difficulties in
accessing training, and attitudes toward life-long learning
yielding poorer results. Overall, benchmarking studies
in general and in EBP in particular.51 Contextual variables
indicate that evidence-based treatments for children and
include the culture of the profession,4 training in EBP, and
youth can be effective in routine practice settings.
institutional and professional support for EBP.52
Critics of EBP have raised concerns that the use of treatment
Keeping up with the literature requires clinicians to
manuals to guide services interferes with the therapeutic formulate a question related to patient care, seek the best
alliance. To test this hypothesis, Langer et al46 randomly research evidence that addresses the question, critically
assigned youth with internalizing disorders in community appraise the validity and applicability of research to the
clinics to receive manualized services or nonmanualized patient, apply the information, and systematically evaluate
usual care. Observer ratings revealed that early on in the treatment effects.53 Further, it requires recognition that the
process, youth receiving manualized services had a stronger evidence base is constantly evolving, thus a practice that is
therapeutic alliance than did youth receiving usual care. currently the best available may later become obsolete or
By the end of services, there were no significant group require modification. With the proliferation of efficacious
differences. Although this study requires replication, it is treatments, psychiatrists may feel overwhelmed, believing
noteworthy that there was no evidence that manualized that they require proficiency in numerous treatments to
services impaired the therapeutic relationship; instead, these serve their patients’ needs. Clinicians who are convinced
services were associated with a stronger early alliance. that their current practice is efficacious may not perceive
a need to devote energy and time to achieve competence
One pressing question in the delivery of EBP is to identify in new treatments. Clinicians may be most reluctant to
active treatment ingredients.47 Chorpita and Daleiden48 consider new interventions that are grounded in a different
analyzed the practice elements in 322 RCTs of treatments theoretical framework than that in which they were trained.
for children and adolescents. They identified constellations
The culture of a profession can influence clinicians’
of strategies consistently found to be helpful for specific openness to EBP. The antithesis of EBP is practice based on
problems. For example, for anxiety, treatments commonly tradition and authority (facetiously dubbed eminence-based
included modules for exposure, relaxation, cognitive practice54) in which recommendations are accepted because
interventions, modelling, and psychoeducation. The authors the person delivering them is regarded as an expert. Unless
recommended that clinicians individualize treatments by backed by high-quality evidence, the opinions of recognized
integrating treatment modules that best address a client’s experts are just that—opinions.55 An apprenticeship model
problems. of training provides fertile ground for eminence-based
The usefulness of a modular approach to the treatment for practice, in which the views of charismatic or authoritative
supervisors are adopted, regardless of their evidence base.
depression, anxiety, and conduct problems in people aged 7
Recent discussion in The Canadian Journal of Psychiatry
to 13 was examined by Weisz et al.49 Community clinicians
has examined the susceptibility of psychiatry to fads
were randomly assigned to 1 of the following: treatment as
that arouse enthusiasm but are later abandoned.56 Some
usual; an evidence-based treatment for anxiety, depression, commentators (for example, see Goldberg57) have suggested
or conduct problems; or a modularized approach in which that the tradition in psychiatry to convene expert panels to
clinicians flexibly delivered modules that integrated the 3 offer pronouncements about practice can be problematic.
evidence-based approaches as needed. Modular treatment Goldberg warned that psychiatrists must be aware of the
outperformed both usual care and a single problem-focused, risks of abandoning practices based on expert consensus
evidence-based treatment, suggesting that this approach rather than research evidence. The dialectic between the
holds promise for treatments delivered to youth. promotion of hope and certainty in helping professions

www.TheCJP.ca The Canadian Journal of Psychiatry, Vol 60, No 12, December 2015 W 537
In Review

and the need for skepticism in science is especially acute Clinical Practice Guidelines,61 Evidence-Based Behavioral
in psychiatry.56 Practice,62 and the Society for Clinical Psychology’s site on
Responsibility also rests with researchers to ensure their psychological treatments,63 and sites based in the United
findings are presented accurately. In a review of the Kingdom, including the National Institute for Health and
American Association for the Advancement of Science’s Care Excellence.64
online coverage of press releases in all areas of science, Despite the wealth of practice guidelines based on research
Yavchitz et al58 found an unwarranted emphasis on the syntheses, the extent to which clinicians access these
benefits of experimental treatments (such as focusing only resources is unknown. A survey65 found that psychologists
on findings showing a treatment effect) in 40% of journal were moderately knowledgeable about general research
abstracts, 47% of press releases, and 51% of news items databases, such as PsycINFO and MEDLINE, but were
based on press releases. In most instances, the spin in press much less knowledgeable about online databases that
releases and media reports originated in journal abstracts, provide integrated information on evidence-based health
suggesting that researchers themselves are responsible for and mental health services. Slightly less than one-half of
at least some of the overselling of findings. Gonon et al59 respondents reported feeling competent with at least one
examined media reports of influential studies on attention- evidence-based database. Clinicians will require training in
deficit hyperactivity disorder. Most findings reported in the use of databases to allow for their ongoing integration
newspapers during the 1990s were not replicated, with
into practice. We recommend that case presentations,
only 2 of the 10 most cited studies supported by later
which are routinely used for training and consultation,
investigations. Only one newspaper article reported that
should include information not only about the patient but
the findings stemming from one of the most commonly
also about how the clinician searched for and selected an
mentioned studies were not replicated.
intervention.66
The practising psychiatrist faces a daunting task in offering
EBP. The Draft CanMEDS 2015 Milestones Guide—May
Modelling of Scientific Reasoning in Clinical
201453 highlights that a physician must “integrate best
available evidence, contextualized to specific situations, Decision Making
into real-time decision-making.”p 38, 42, 46 The foundation In the apprenticeship model of training in psychiatry,
for critical appraisal skills is to be established in medical residents consolidate their didactic learning by applying
school, with demonstration of these skills by the time of knowledge in the clinical context. Peers and supervisors
transition to practice. The steps to acquisition of competence model how they generate hypotheses about patients, search
in formulating testable hypotheses about a patient, the literature for evidence, and monitor the usefulness of
consulting the relevant scientific literature and appraising their practice.4 The adoption of EBP requires a commitment
its relevance, and ongoing monitoring of the usefulness of to a humble stance in which testable hypotheses guide a
a practice are elaborated in broad brushstrokes. However, search for the best evidence, and ongoing monitoring
the Guide offers few specific suggestions, so it is unclear informs the usefulness of an intervention. In this model,
how training assists in the development of this competence. the capacity to generate questions, search the literature,
Nevertheless, psychiatrists require training that promotes a and critically appraise evidence is more important than is
scientific approach to decision making, access to research knowledge of the literature.
syntheses, and an environment supportive of EBP.
Training in Core Skills
Constructive Strategies to Promote Use of An increasing number of online tools have been developed
Evidence-Based Practices to assist in EBP training. The Psychotherapy Training
Given the rapid evolution of the knowledge base, psychiatric e-Resources based at McMaster University are designed
residents must not only master current knowledge but also for training and continuing education of mental health
learn strategies for continual questioning and updating professionals.67 The PracticeWise site68 offers a database
of knowledge in a process of life-long learning. Our on evidence-based services for children and youth, as
recommendations focus on strategies that create a context well as practice guides and tools for treatment plans and
favourable for EBP. tracking progress. Expertise in using scientifically sound
assessment tools, to both develop treatment plans and
Promoting Use of Clinical Practice Guidelines monitor treatment progress, is essential for true EBP.
The Draft CanMEDS 2015 Milestones Guide—May 201453 Because no clinician can expect to master all evidence-
specifies that by their transition to practice, physicians must based protocols for all disorders, there is growing interest in
be competent in identifying, selecting, and navigating pre- identifying core treatment skills that can be flexibly applied
appraised evidence. Psychiatrists have access to many high- to different problems. Although there was a time when
quality EBP databases, including the Canadian Psychiatric EBP was largely synonymous with cognitive-behavioural
Association Clinical Practice Guidelines,60 American therapy, interventions from other theoretical frameworks
sites, such as the American Psychiatric Association (for example, interpersonal psychotherapy and short-term

538 W La Revue canadienne de psychiatrie, vol 60, no 12, décembre 2015 www.LaRCP.ca
Evidence-Based Practice: Separating Science From Pseudoscience

psychodynamic therapies) are also efficacious for certain 16. Westen D, Novotny CM, Thompson-Brenner H. The empirical
status of empirically supported psychotherapies: assumptions,
conditions.69,70 findings, and reporting in controlled clinical trials. Psychol Bull.
2004;130:631–663.
Final Comment 17. Institute of Medicine. Crossing the quality chasm: a new health
system for the 21st century. Washington (DC): National Academy
The goal of psychiatric services is to provide the best Press; 2002.
treatment possible to people with mental disorders. Attending 18. American Psychological Association Presidential Task Force on
to the research evidence in planning these services and Evidence-Based Practice. Evidence-based practice in psychology.
Am Psychol. 2006;61:271–285.
systematically monitoring their impact, provides the optimal 19. Stuart RB, Lilienfeld SO. The evidence missing from evidence-
means of achieving this goal. For this goal to be realized, based practice. Am Psychol. 2007;62:613–614.
clinicians must recognize not only the value of disciplined 20. Dozois DJA, Mikail S, Alden LE, et al. The CPA presidential task
force on evidence-based practice of psychological treatments. Can
scientific inquiry but also that clinical judgment should be Psychol. 2014;55:153–160.
continually informed by such inquiry. By developing the 21. Lambert MJ, Shimokawa K. Collecting client feedback.
attitudes, values, and skills consistent with EBP principles, Psychotherapy (Chic). 2011;48:72–79.
people in training to provide psychiatric services will be 22. Bhugra D, Easter A, Mallaris Y, et al. Clinical decision
making in psychiatry by psychiatrists. Acta Psychiatr Scand.
well-placed to deliver the best services possible. 2011;124:403–411.
23. Ionita G, Fitzpatrick M. Bringing science to clinical practice:
a Canadian survey of psychological practice and usage of progress
Acknowledgements monitoring measures. Can J Psychol. 2014;55:187–196.
The authors have no conflicts of interest to report. 24. Antony MM, Barlow DH. Handbook of assessment and treatment
planning for psychological disorders. 2nd ed. New York (NY):
The Canadian Psychiatric Association proudly supports Guilford Press; 2010.
the In Review series by providing the authors with an 25. Hunsley J, Mash EJ. A guide to assessments that work. New York
honorarium. (NY): Oxford University Press; 2008.
26. Patient Reported Outcomes Measurement Information System
(PROMIS) Network. Dynamic tools to measure health outcomes
References from the patient perspective [Internet]. Chapel Hill (NC): PROMIS;
1. World Health Organization (WHO). 2014 mental health atlas 2011 [cited 2015 Aug 1]. Available from: http://www.nihpromis.org/
[Internet]. Geneva (CH): WHO; 2015 [cited 2015 Oct 6]. about/abouthome. National Institutes of Health–funded.
Available from: http://www.who.int/mental_health/evidence/atlas/ 27. Lilienfeld SO, Lynn SJ, Lohr JM. Science and pseudoscience in
mental_health_atlas_2014/en. clinical psychology: initial thoughts, reflections, and considerations.
2. Canadian Psychiatric Association (CPA) and Ontario Psychiatric In: Lilienfeld SO, Lynn SJ, Lohr JM, editors. Science and
pseudoscience in clinical psychology. 2nd ed. New York (NY):
Association (OPA). CPA and OPA welcome report on psychiatric
Guilford Press; 2015. p 1–16.
supply and practice patterns [Internet]. Ottawa (ON): CPA; 2014 Jul
[cited 2015 Sep 15]. Available from: http://www.cpa-apc.org/ 28. Callahan R. The impact of thought field therapy on heart rate
media.php?mid=2024. variability. J Clin Psychol. 2001;57:1153–1170.
3. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence-based 29. Gaudiano BA, Herbert JD. Can we really tap our problems away?
medicine: what it is and what it isn’t. Br Med J. 1996;312:71–72. A critical analysis of thought field therapy. The Skeptical Inquirer.
2000;24:29–33,36.
4. Agrawal S, Szatmari P, Hanson M. Teaching evidence-based
psychiatry: aligning the formal and hidden curricula. Acad 30. Lohr JM, Hooke W, Gist R, et al. Novel and controversial treatments
for trauma-related and stress disorders. In: Lilienfeld SO, Lynn SJ,
Psychiatry. 2008;32:470–474.
Lohr JM, editors. Science and pseudoscience in clinical psychology.
5. Stam J, Vermeulen MJ. Eugen Bleuler (1857–1939), an early New York (NY): Guilford Press; 2003. p 243–272.
pioneer of evidence based medicine. J Neurol Neurosurg Psychiatry.
31. Pignotti M. Callahan fails to meet the burden of proof for thought
2013;84:594–595.
field therapy claims. J Clin Psychol. 2005;61:251–255.
6. Chambless DL, Ollendick TH. Empirically supported psychological
32. Callahan Techniques, Ltd. Callahan Techniques: thought field
interventions: controversies and evidence. Annu Rev Psychol.
therapy [Internet]. La Quinta (CA): Callahan Techniques, Ltd; 2015
2001;52:685–716.
[cited 2015 Aug 1]. Available from: http://www.rogercallahan.com/
7. Roth A, Fonagy P. What works for whom? A critical review of index.php.
psychotherapy research. New York (NY): Guilford Press; 1996. 33. Lilienfeld SO, Ritschel LA, Lynn SJ, et al. Why many clinical
8. Fonagy P, Target M, Cottrell D, et al. What works for whom? psychologists are resistant to evidence-based practice: root causes
A critical review of treatments for children and adolescents. New and constructive remedies. Clin Psychol Rev. 2013;33:883–890.
York (NY): Guilford Press; 2002. 34. Lilienfeld SO, Ritschel LA, Lynn SJ, et al. Why ineffective
9. Fonagy P, Cottrell P, Phillips J, et al. What works for whom? psychotherapies appear to work: a taxonomy of causes of spurious
A critical review of treatments for children and adolescents. 2nd ed. therapeutic effectiveness. Perspect Psychol Sci. 2014;9:355–387.
New York (NY): Guilford; 2014. 35. North CS, Pfefferbaum B. Mental health response to community
10. Nathan P, Gorman JM. A guide to treatments that work. New York disasters: a systematic review. JAMA. 2013;310:507–518.
(NY): Oxford University Press; 1998. 36. Rose S, Bisson J, Churchill R, et al. Psychological debriefing for
11. Nathan P, Gorman JM. A guide to treatments that work. 2nd ed. New preventing posttraumatic stress disorder (PTSD). Cochrane Database
York (NY): Oxford University Press; 2002. Syst Rev. 2002;(2):CD000560.
12. Nathan PE, Gorman JM. A guide to treatments that work. 3rd ed. 37. Roberts NP, Kitchiner NJ, Kenardy J, et al. Multiple session early
New York (NY): Oxford University Press; 2007. psychological interventions for prevention of post-traumatic stress
13. Nathan PE, Gorman JM. A guide to treatments that work. 4th ed. disorder. Cochrane Database Syst Rev. 2009;(3):CD006869.
New York (NY): Oxford University Press; 2015. 38. Petrosino A, Turpin-Petrosino C, Buehler J. Scared Straight
14. von Ransom KM, Wallace LM, Stevenson A. Psychotherapies and other juvenile awareness programs for preventing juvenile
provided for eating disorders by community clinicians: infrequent delinquency. Sci Rev Ment Health Pract. 2005;4:48–54.
use of evidence-based treatment. Psychother Res. 2013;23:333–343. 39. Lilienfeld SO. Psychological treatments that cause harm. Perspect
15. Weissman MM, Verdeli H, Gameroff MJ, et al. National survey of Psychol Sci. 2007;2:53–70.
psychotherapy training in psychiatry, psychology, and social work. 40. Seligman MEP. The effectiveness of psychotherapy: the Consumer
Arch Gen Psychiatry. 2006;63:925–934. Reports study. Am Psychol. 1995;50:965–974.

www.TheCJP.ca The Canadian Journal of Psychiatry, Vol 60, No 12, December 2015 W 539
In Review

41. Jensen-Doss A, Weisz JR. Syndrome co-occurrence and treatment 58. Yavchitz A, Boutron I, Bafeta A, et al. Misrepresentation of
outcomes in youth mental health clinics. J Consult Clin Psychol. randomized controlled trials in press releases and news coverage:
2006;74:416–425. a cohort study. PLoS Med. 2012;9(9):e1001308.
42. Kazdin AE, Whitley MK. Comorbidity, case complexity, and effects 59. Gonon F, Konsman J-P, Cohen D, et al. Why most biomedical
of evidence-based treatments for children referred for disruptive findings echoed by newspapers turn out to be false: the
behavior. J Consult Clin Psychol. 2006;74:455–467. case of attention deficit hyperactivity disorder. PLoS ONE.
43. Hunsley J, Lee CM. Research-informed benchmarks for 2012;7(9):e44275.
psychological treatments: efficacy studies, effectiveness studies, and 60. Canadian Psychiatric Association (CPA). Clinical practice
beyond. Prof Psychol Res Pr. 2007;38:21–33. guidelines [Internet]. Ottawa (ON): CPA; 2001–2012 [cited 2015
44. Stewart RE, Chambless DL. Cognitive-behavioral therapy for Aug 1]. Available from: http://publications.cpa-apc.org/browse/
adult anxiety disorders in clinical practice: a meta-analysis of documents/67.
effectiveness studies. J Consult Clin Psychol. 2009;77:595–606. 61. American Psychiatric Association (APA). Clinical practice
45. Lee CM, Horvath C, Hunsley J. Does it work in the real-world? guidelines [Internet]. Arlington (VA): APA; 2015 [cited 2015 Oct
Effectiveness studies of treatments for psychosocial problems in 6]. Available from: http://www.psychiatry.org/psychiatrists/practice/
children and adolescents. Prof Psychol Res Pr. 2013;44:81–88. clinical-practice-guidelines.
46. Langer DA, McLeod BD, Weisz JR. Do treatment manuals 62. EBBP: Evidence-Based Behavioral-Practice [Internet]. Chicago
undermine youth–therapist alliance in community clinical practice? (IL): Bonnie Spring, Northwestern University; 2007 [cited 2015 Oct
J Consult Clin Psychol. 2011;79:427–432. 6]. Available from: http://www.ebbp.org.
47. Kazdin AE. Psychotherapy for children and adolescents. Ann Rev 63. Division 12: Society for Clinical Psychology. Research-supported
Psychol. 2003;54:253–276. psychological treatments [Internet]. Atlanta (GA): Division 12 of
48. Chorpita BF, Daleiden EL. Mapping evidence-based treatments for the American Psychological Association; 2013 [cited 2014 Oct 1].
children and adolescents: application of the distillation and matching Available from: http://www.div12.org/psychological-treatments.
model to 615 treatments from 322 randomized trials. J Consult Clin 64. National Institute for Health and Care Excellence (NICE).
Psychol. 2009;77:566–579. Guidance list [Internet]. London (GB): NICE; 2014 [cited
49. Weisz JR, Chorpita BF, Palinkas LA, et al. Testing standard and 2015 Oct 6]. Available from: http://www.nice.org.uk/guidance/
modular designs for psychotherapy treating depression, anxiety, and published?type=cg.
conduct problems in youth. Arch Gen Psychiatry. 2012;69:274–282. 65. Berke DM, Rozell CA, Hogan TP, et al. What clinical psychologists
50. Glouberman S. Knowledge transfer and the complex story of scurvy. know about evidence-based practice: familiarity with online
J Eval Clin Pract. 2009;15:553–557. resources and research methods. J Clin Psychol. 2011;67:329–339.
51. Gallo KP, Barlow DH. Factors involved in clinician adoption and 66. Edmunds JM, Beidel RS, Kendall PC. Dissemination and
nonadoption of evidence-based interventions in mental health. Clin implementation of evidence-based practices: training and
Psychol Sci Pr. 2012;19:93–106. consultation as implementation strategies. Clin Psychol Res Pr.
52. Beidas RS, Kendall PC. Training therapists in evidence-based 2013;20:152–165.
practice: a critical review of studies from a systems-contextual 67. McMaster University. Psychotherapy training e-resources [Internet].
perspective. Clin Psychol Sci Pr. 2010;17:1–30. Hamilton (ON): McMaster University; 2015 [cited 2015 Aug 1].
53. Frank JR, Snell LS, Sherbino J, et al. Draft CanMEDS 2015 Available from: https://pter.mcmaster.ca.
milestones guide—May 2014. Ottawa (ON): Royal College of 68. PracticeWise. What works in children’s mental health [Internet].
Physicians and Surgeons of Canada; 2014 May. Satellite Beach (FL): PracticeWise; 2005–2015 [cited 2014 Oct 1].
54. Isaacs D, Fitzgerald D. Seven alternatives to evidence-based Available from: https://www.practicewise.com.
medicine. BMJ. 1999;319:1618. 69. Cuijpers P, Berking M, Andersson G, et al. A meta-analysis of
55. Mullen EJ, Streiner DL. The evidence for and against evidence- cognitive-behavioural therapy for adults alone, and in comparison
based practice. Brief Treat Crisis Interv. 2004;4:111–1121. with other treatments. Can J Psychiatry. 2013;58:376–385.
56. Paris J. Why is psychiatry prone to fads? Can J Psychiatry. 70. Hunsley J, Elliott KP, Therrien Z. The efficacy and effectiveness of
2013;58:560–565. psychological treatments for mood, anxiety, and related disorders.
57. Goldberg D. Fads in psychiatry. Can J Psychiatry. 2013;58:553–554. Can Psychol. 2014;55:161–176.

540 W La Revue canadienne de psychiatrie, vol 60, no 12, décembre 2015 www.LaRCP.ca