Vous êtes sur la page 1sur 10

Research

Original Investigation

Brief Intervention for Problem Drug Use in Safety-Net


Primary Care Settings
A Randomized Clinical Trial
Peter Roy-Byrne, MD; Kristin Bumgardner, BS; Antoinette Krupski, PhD; Chris Dunn, PhD; Richard Ries, MD;
Dennis Donovan, PhD; Imara I. West, MPH; Charles Maynard, PhD; David C. Atkins, PhD; Meredith C. Graves, PhD;
Jutta M. Joesch, PhD; Gary A. Zarkin, PhD
Editorial page 488
IMPORTANCE Although brief intervention is effective for reducing problem alcohol use, few

Related article page 502

data exist on its effectiveness for reducing problem drug use, a common issue in
disadvantaged populations seeking care in safety-net medical settings (hospitals and
community health clinics serving low-income patients with limited or no insurance).

Supplemental content at
jama.com

OBJECTIVE To determine whether brief intervention improves drug use outcomes compared
with enhanced care as usual.
DESIGN, SETTING, AND PARTICIPANTS A randomized clinical trial with blinded assessments at
baseline and at 3, 6, 9, and 12 months conducted in 7 safety-net primary care clinics in
Washington State. Of 1621 eligible patients reporting any problem drug use in the past 90
days, 868 consented and were randomized between April 2009 and September 2012.
Follow-up participation was more than 87% at all points.
INTERVENTIONS Participants received a single brief intervention using motivational

interviewing, a handout and list of substance abuse resources, and an attempted 10-minute
telephone booster within 2 weeks (n = 435) or enhanced care as usual, which included a
handout and list of substance abuse resources (n = 433).
MAIN OUTCOMES AND MEASURES The primary outcomes were self-reported days of problem
drug use in the past 30 days and Addiction Severity IndexLite (ASI) Drug Use composite score.
Secondary outcomes were admission to substance abuse treatment; ASI composite scores for
medical, psychiatric, social, and legal domains; emergency department and inpatient hospital
admissions, arrests, mortality, and human immunodeficiency virus risk behavior.
RESULTS Mean days used of the most common problem drug at baseline were 14.40 (SD,
11.29) (brief intervention) and 13.25 (SD, 10.69) (enhanced care as usual); at 3 months
postintervention, means were 11.87 (SD, 12.13) (brief intervention) and 9.84 (SD, 10.64)
(enhanced care as usual) and not significantly different (difference in differences, = 0.89
[95% CI, 0.49 to 2.26]). Mean ASI Drug Use composite score at baseline was 0.11 (SD, 0.10)
(brief intervention) and 0.11 (SD, 0.10) (enhanced care as usual) and at 3 months was 0.10 (SD,
0.09) (brief intervention) and 0.09 (SD, 0.09) (enhanced care as usual) and not significantly
different (difference in differences, = 0.008 [95% CI, 0.006 to 0.021]). During the 12
months following intervention, no significant treatment differences were found for either
variable. No significant differences were found for secondary outcomes.
CONCLUSIONS AND RELEVANCE A one-time brief intervention with attempted telephone
booster had no effect on drug use in patients seen in safety-net primary care settings. This
finding suggests a need for caution in promoting widespread adoption of this intervention for
drug use in primary care.
TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT00877331

JAMA. 2014;312(5):492-501. doi:10.1001/jama.2014.7860


492

Author Affiliations: Department of


Psychiatry and Behavioral Sciences,
School of Medicine, University of
Washington, Seattle (Roy-Byrne,
Bumgardner, Krupski, Dunn, Ries,
Donovan, West, Atkins, Graves,
Joesch); Department of Health
Services, School of Public Health,
University of Washington, Seattle
(Maynard); King County Office of
Performance, Strategy and Budget,
Seattle, Washington (Joesch); RTI
International, Research Triangle Park,
North Carolina (Zarkin).
Corresponding Author: Peter RoyByrne, MD, University of Washington
School of Medicine, PO Box 359911,
325 Ninth Ave, Seattle, WA 98104
(roybyrne@uw.edu).
jama.com

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

Brief Intervention for Problem Drug Use

Original Investigation Research

ased on the established efficacy of brief (1-2 sessions)


interventions for hazardous alcohol use in patients seen
in medical settings,1,2 national dissemination programs of screening, brief intervention, and referral to treatment (SBIRT) for alcohol and drugs have been implemented on a widespread scale.3,4 This rapid adoption of brief
intervention for drugs other than alcohol has outstripped its
meager evidence base. Of the 3 randomized clinical trials of
brief intervention for drug abuse in nontreatment-seeking
individuals,5-8 only 2 were positive,7,8 and only 1 found an effect among US patients.8
Ambulatory primary care is an important setting to test the
effectiveness of brief intervention because of the large volume of patients seen in that setting.9 The majority of patients
with problem drug use in this setting are not accessing substance abuse treatment.4,10 Adding brief substance abuse interventions onsite in primary care is also consistent with newly
emerging integrated care models.11
Recent studies show that a disproportionate number of individuals with drug abuse or dependence are from lower socioeconomic strata12 and primarily receive care in public sector hospitals and community health clinics (ie, safety-net
medical settings that serve low-income patients with limited
or no insurance). Although the cost of untreated drug abuse
in this system is substantial (eg, mortality, crime, lost productivity, increased medical care),13 there is limited information
on these public healthrelevant outcomes for brief
intervention.14,15 Policy makers need such data to create traction to drive policy changes required to improve substance
abuse treatment.
The purpose of this study was to determine whether a brief
intervention using motivational interviewing would reduce problem drug use in safety-net primary care patients and increase admission to specialist substance abuse treatment. We also sought
to estimate the effects of the brief intervention on several public health outcomes directly related to problem drug use.

Methods
Design
A 2-group randomized clinical trial was conducted to examine the effects of a 1-time brief intervention using motivational interviewing with attempted telephone booster compared with enhanced care as usual. A hybrid efficacyeffectiveness design was chosen to enhance external validity
by using a brief intervention protocol similar to that used in
the Substance Abuse and Mental Health Services Administrationfunded Washington State SBIRT study16 and having the
intervention delivered by social workers already working with
patients in these clinics.17 The study was approved by an institutional review board and an independent data and safety
monitoring board. Further details about study design and rationale have been described.18

Participants
Participants were recruited between April 2009 and September 2012 from the waiting rooms of 7 safety-net (public hospital
jama.com

associated) primary care clinics in King County, Washington.


Inclusion criteria were age 18 years or older; self-reported use
of an illegal drug or nonprescribed medication (ie, problem
drug use) at least once in the 90 days before screening19; English-speaking and able to read and understand screening and
consent forms (sixth-grade literacy); currently receiving and
planning to continue receiving care in the clinic; and having
telephone or e-mail access to facilitate scheduling follow-up
assessments. Exclusion criteria were attendance in formal substance abuse treatment in the past month (excluding selfhelp groups such as Narcotics Anonymous); high risk of imminent suicide; life-threatening medical illness; severe
cognitive impairment; or active psychosis. All participants provided written informed consent and received compensation
in gift cards for completion of study assessments ($25 at baseline and 3 months, $30 at 6 months, $35 at 9 months, $40 at 12
months, and $10 for urine samples at each follow-up).

Randomization
After a brief baseline assessment, participants were randomized in a 1:1 ratio (Figure) to brief intervention or enhanced care
as usual using permuted blocks stratified by clinic and by 3 factors known to affect outcome: drug use severity20 (Drug Abuse
Screening Test [DAST-10] score 3 [range, 0-10; 10 indicates
greatest severity]; the DAST-10 instrument and interpretation guide are available in the eAppendix in the Supplement),
comorbid mental illness21 (Addiction Severity IndexLite [ASI]
Psychiatric Status composite score >0.38 [range, 0-1; 1 indicates greatest problem severity]), and readiness to change22
(goal of abstinence on Thoughts about Abstinence assessment). Varying-sized blocks were generated prior to enrollment, and allocation was concealed in sequentially numbered opaque envelopes opened by the research assistant at
randomization.23

Intervention
To maximize the possibility of an effect while maintaining realworld feasibility, brief interventionists were recruited from social workers in participating clinics (n = 11).24 The interventionists agreed to adhere to the study protocol, including
ongoing training and supervision, and to provide written informed consent (because data on their fidelity to the motivational interviewing model would be used for the study). Additional masters- and bachelors-level interventionists not
working in the clinics (n = 6) were later added to help meet recruitment goals. All interventionists were trained by 1 of the
original trainers for the Substance Abuse and Mental Health
Services Administrations national SBIRT initiative3 using a
group workshop followed by individual feedback on audiotaped role-plays with up to 5 standardized patients over the
course of 5 weeks.25 All interventionists met basic motivational interviewing proficiency training goals on at least 1 practice role-play.26
Participants randomized to the intervention condition were
to receive a single, approximately 30-minute brief intervention. The clinical tasks of the intervention protocol were to give
participants feedback about their drug use screening (DAST10) results, explore the pros and cons of drug use, increase parJAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

493

Research Original Investigation

Brief Intervention for Problem Drug Use

ticipant confidence in being able to change, and discuss options for change. When appropriate, interventionists used an
illustrated handout depicting the participants DAST-10 score
and its associated problem severity (low, intermediate, substantial/severe) to aid the discussion (the DAST-10 feedback
handout is available in the eAppendix in the Supplement) and
provided a list of substance abuse treatment resources. A motivational interviewing approach was used to perform these
tasks, with the hope that understanding and exploring ambivalence about change among vulnerable, underserved participants while eliciting their own arguments for change might
empower them and bridge cultural differences. Also, as opposed to a one size fits all intervention, this tailored approach allowed flexibility as to which or how many drugs to
target, as well as in how to guide the participant (eg, specialty
treatment, abstinence, harm reduction). The same interventionist attempted a follow-up telephone booster session within
2 weeks of the intervention. Interventions were audio recorded and scored by trained coders using the Motivational Interviewing Treatment Integrity (MITI) coding system. 27
Monthly group supervision and ongoing e-mailed feedback
from the trainer (25% of recorded brief interventions) were used
to maintain intervention fidelity. To maximize external validity, MITI scoring feedback was not given to interventionists for
training role-plays or for real-patient supervision.

Enhanced Care as Usual


Participants in the enhanced care as usual group received the
same illustrated handout depicting their DAST-10 drug problem severity score and list of substance abuse treatment resources. These were provided with only a quick introduction
by the research assistant to minimize intervention elements
in the control condition and resembled the notification and
referral strategy that might be implemented in high-quality
usual care.

Assessment Measures
The assessment battery, administered by trained research assistants, was brief to minimize any unintended intervention
effects17,28 and to maximize the chance of completing study
procedures around a primary care visit. The baseline interview assessed demographics, including self-reported race and
ethnicity; substance abuse severity (DAST-10)20; drug use in
the previous month, comorbid medical and mental illness, and
social and legal outcomes related to drug use (ASI)29; goal for
changing drug use (Thoughts about Abstinence assessment)22;
and the HIV Risk-taking Behavior Scale.30 Urine samples for
drug screens were also collected to track point prevalence of
actual drug use. All measures except the demographics and
DAST-10 were repeated at 3-, 6-, 9-, and 12-month follow-up
assessments conducted by research assistants blinded to randomization assignment. The last 12-month assessment was
completed on September 20, 2013.

Primary Outcomes
The primary outcome, prespecified on ClinicalTrials.gov, was
self-reported days of drug use during the past 30 days. The ASI
measures drug use for individual drugs but does not include
494

an omnibus category for number of days with any drug use.


As a proxy for this, present analyses focused on the most frequently used drug at baseline for each individual. This was
supplemented with the ASI Drug Use composite score (range,
0-1; 1 indicates greatest severity), an integrated measure of frequency of use and associated problems for all drugs used. For
both primary outcomes, measurement of drug use excluded
use of alcohol and prescription drugs when used as prescribed.

Secondary Outcomes
Medical, psychiatric, social, and legal subscales of the ASI were
used to assess functional correlates of drug use. The assessments were supplemented with Washington State administrative data, which included chemical dependency treatment
records, inpatient hospitalizations, state patrol arrest records, and death records. Emergency department and outpatient visits were also tracked using electronic medical record
data from the safety-net medical center where the study took
place.

Analysis
Acute treatment differences in primary outcomes (days used
during last 30 days and drug use composite) focused on preintervention to 3 months postintervention and used a difference in differences analysis strategy.31 Long-term treatment
differences during 12 months postintervention were analyzed using generalized estimating equations (GEEs).32 Because days of use was highly bimodal, the original count variable was divided into 4 categories of days of use (0, 1-10, 1120, 21-30) and analyzed using an ordinal GEE,33 whereas the
drug use composite score was analyzed with a gaussian GEE.
For both outcomes, alternative distributions (eg, negative binomial, log-normal) were examined, but all substantive conclusions were identical to those reported. Covariates included baseline value of the outcome, indicator variables for
the clinics where recruitment occurred, and baseline values
of the stratification variables used in randomization. Administrative data were analyzed via logistic regression, and ASI
scores for medical, psychiatric, social, and legal domains were
analyzed via linear regression. Prespecified treatment effect
modification of primary outcomes was examined for baseline stratification variables of drug use severity, comorbid psychiatric illness, and motivation to change drug use. As a post
hoc exploratory analysis, treatment effect modification of select secondary outcomes was also examined.
All analyses were intention-to-treat, with all available
data from participants analyzed by randomization group.
There were few missing data overall (87% of planned assessments completed), and GEE analyses made use of all available data. Significance was based on 2-sided P values <.05. A
priori power analyses concluded that a trial with an initial
sample of 1000 with 25% attrition (ie, 375 participants per
group) yielded 80% power to detect small treatment differences in drug use (defined by 2.5% reduction in drug use
frequency). All analyses were conducted using either SPSS
version 19.0 (IBM Corp), Stata version 13 (StataCorp), or R
version 3.0.2.34

JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

jama.com

Brief Intervention for Problem Drug Use

Original Investigation Research

Results
Sample Selection, Attrition, and Description
Of 1621 eligible participants, 868 (54%) were randomized
(Figure 1). Follow-up rates were greater than 87% at all 4 points
(3, 6, 9, and 12 months) and were similar in both groups. Participants without follow-ups (n = 45) were not significantly different from participants with follow-ups (n = 823) on baseline measures or by randomization group. Assessments
averaged 25 minutes, and the majority of assessments were
conducted in person (91%).
At baseline, the sample composition was similar in both
groups: middle aged, mostly men (70%), nonwhite (55%),
single (81%), and not working (91%); approximately 30%
were homeless at least 1 night in the past 90 days (Table 1).
In the year prior to enrollment, participants had substantial
medical comorbidity (mean, >6 medical conditions),35 with
notable percentages of hospitalization (25%) and emergency
department use (49%); 10% had received chemical dependency treatment (Table 2). Drug use was similar between
groups (mean, 13.82 [SD, 11.00] days of the most frequently
used drug) and was highly heterogeneous, with different
types and patterns of use evident according to the severity
of drug use measured by validated DAST-10 score categories
(eTable 1 and eTable 2 in the Supplement). Many participants also used alcohol, but because we had no data on
quantity of alcohol consumed per drinking day, determining
degree of hazardous alcohol use was not possible. Supplementing self-reported drug use with positive urine screen
results only slightly increased the proportion of individuals
classified as using particular drugs (eTable 3 in the Supplement). A majority of participants (56%) had some comorbid
mental illness, and a minority (37%) had the goal to try to
abstain from drugs. These 2 stratification variables were
also significantly associated with problem drug use severity
such that the most severe group was most likely to have a
goal of abstinence and also to have more mental illness.

Intervention Participation and Fidelity


Of the 435 participants randomized to the intervention, 423
(97%) received a brief intervention and 203 (47%) received a
booster call. Most participants (90%) received the brief intervention on the same day as the baseline assessment, but for
logistical reasons, the remaining 10% had to return to the clinic
for the intervention a median of 4 days after assessment. The
brief intervention averaged 27 minutes, and 86% of interventions were recorded. Mean interventionist scores for all audiorecorded brief interventions met basic proficiency cutoffs for
4 of 5 MITI summary scores, suggesting adequate but not expert proficiency.26 Over the course of the next year, a small
number of participants in both conditions (brief intervention: n = 26; enhanced care as usual: n = 34) received brief interventions outside the study via regular clinical services.

Main Outcomes
Figure 2 depicts days of drug use during the last 30 days and
drug use composite scores over time in the 2 groups. The
jama.com

treatment means in each plot are virtually identical, change


over time is minimal, and there is no evidence of an intervention effect. Mean days used of the most common problem
drug at baseline were 14.40 (SD, 11.29) (brief intervention)
and 13.25 (SD, 10.69) (enhanced care as usual) and at 3
months postintervention were 11.97 (SD, 12.13) (brief intervention) and 9.84 (SD, 10.64) (enhanced care as usual). Mean
ASI Drug Use composite score at baseline was 0.11 (SD, 0.10)
(brief intervention) and 0.11 (SD, 0.10) (enhanced care as
usual) and at 3 months was 0.10 (SD, 0.09) (brief intervention) and 0.09 (SD, 0.09) (enhanced care as usual). Acute
treatment differences based on difference in differences
revealed no significant differences in either days of drug use
( = 0.89 [95% CI, 0.49 to 2.26]) or ASI Drug Use composite
( = 0.008 [95% CI, 0.006 to 0.021]). Generalized estimating
equation analyses confirmed that there were no significant
treatment differences during 12 months postintervention in
either days of drug use (odds ratio, 1.20 [95% CI, 0.96 to 1.50])
or the ASI Drug Use composite ( = 0.005 [95% CI, 0.005 to
0.016]).
Table 2 reports the secondary outcome variables, including ASI scores for medical, psychiatric, employment, social, and
legal domains, proportion of individuals who accepted a referral to chemical dependency treatment and had an initial
treatment contact, and relevant public health outcomes (eg,
medical care use, arrests, deaths). There were no significant
intervention effects on any secondary outcomes, including admission to chemical dependency treatment.
Effect modification of primary drug use outcomes by baseline drug use severity, psychiatric comorbidity, and motivation to change revealed no significant moderation effects
(eTable 4 and eTable 5 in the Supplement). However, in exploratory effect modification analyses of secondary outcomes, drug use severity significantly moderated intervention effects on both admission to treatment and emergency
department use. Participants with high drug problem severity receiving the intervention were more likely to enter specialist drug treatment and more likely to reduce their use of
the emergency department (Table 3 and eTable 6 in the Supplement).

Discussion
In the overall sample, participants who received a brief
intervention using proficiency-level motivational interviewing showed no benefit across multiple domains, including
drug use frequency, admissions to drug treatment, and all
other secondary measures, including those of high public
health relevance. Both groups showed modest and similar
reductions in drug use frequency over the first 3 months
with no subsequent change, possibly suggesting a regression to the mean in both groups. Although our negative
findings are consistent with prior studies in treatmentseeking patients in the United States,5,6,8 they also may be a
product of our participant characteristics as well as the ways
in which the intervention was conducted and the outcomes
were evaluated.
JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

495

Research Original Investigation

Brief Intervention for Problem Drug Use

Figure 1. Participant Flow in the Trial of a Brief Intervention for Problem Drug Use
39 062 Patients approached for screening
28 725 Excluded
15 560 Excluded at approach (did not meet inclusion
criteria or met exclusion criteria)a
9952 Declined to participate
3213 Other (eg, left during approach)
10 337 Screened
8716 Excluded
8540 Excluded at screening (did not meet
inclusion criteria or met exclusion criteria)a
176 Other (eg, left during screening)
1621 Eligible
674 Excluded
520 Declined to participate
154 Other (eg, left during consent process)
947 Provided consent
79 Excluded
44 Excluded during baseline assessment
22 Formal chemical dependency treatment
9 Acute suicidality or psychosis
8 No drug use in past 3 mo
3 Not primary care patient
2 Insufficient contact information
32 Declined to participate
3 Other

868 Randomized

435 Randomized to receive brief intervention


plus booster call
423 Received brief intervention as
randomized
203 Received booster call as randomized
379
35
3
2
13
3

Completed 3-mo follow-up assessment


Could not be reached
Deaths
Incarcerated
Refused
Other

390
28
3
6
6

Completed 3-mo follow-up assessment


Could not be reached
Deaths
Incarcerated
Refused

380
26
3
5
17
4

Completed 6-mo follow-up assessment


Could not be reached
Cumulative deaths
Incarcerated
Refused
Other

387
28
4
6
8

Completed 6-mo follow-up assessment


Could not be reached
Cumulative deaths
Incarcerated
Refused

378
25
6
6
18
2

Completed 9-mo follow-up assessment


Could not be reached
Cumulative deaths
Incarcerated
Refused
Other

383
20
4
17
9

Completed 9-mo follow-up assessment


Could not be reached
Cumulative deaths
Incarcerated
Refused

385
12
10
3
1
22
2

Completed 12-mo follow-up assessment


Could not be reached
Cumulative deaths
Incarcerated
Physically/mentally unable
Refused
Other

392
12
7
12
1
9

Completed 12-mo follow-up assessment


Could not be reached
Cumulative deaths
Incarcerated
Physically/mentally unable
Refused

435 Included in primary analysis


426 Provided consent to access
administrative data

496

433 Randomized to receive enhanced care


as usual (control)
433 Received enhanced care as usual
as randomized

433 Included in primary analysis


422 Provided consent to access
administrative data

JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

Inclusion criteria were age 18 years


or older; self-reported use of an
illegal drug or nonprescribed
medication (ie, problem drug use) at
least once in the 90 days before
screening19; English-speaking and
able to read and understand
screening and consent forms
(sixth-grade literacy); currently
receiving and planning to continue
care in the clinic; and having
telephone or e-mail access to
facilitate scheduling follow-up
assessments. Exclusion criteria were
attendance in formal substance
abuse treatment in the past month
(excluding self-help groups such as
Narcotics Anonymous); high risk of
imminent suicide; life-threatening
medical illness; severe cognitive
impairment; or active psychosis.
Data for reasons of exclusion are not
available.

jama.com

Brief Intervention for Problem Drug Use

Original Investigation Research

Table 1. Baseline Characteristics of Randomized Participants (N=868)a


Participants, No. (%)
Overall
(N = 868)

Brief Intervention
(n = 435)

Enhanced Care
as Usual
(n = 433)

47.76 (10.89)

47.50 (11.29)

48.03 (10.48)

264 (30)

127 (29)

137 (32)

White

386 (45)

190 (44)

196 (45)

Black

320 (37)

157 (36)

163 (38)

Other

150 (18)

82 (19)

68 (16)

72 (9)

33 (9)

39 (10)

Characteristic
Demographics
Age, mean (SD), y
Women
Raceb

Hispanic
Marital status
Married/living with partner

161 (19)

80 (18)

81 (19)

Divorced/separated/widowed

348 (40)

167 (39)

181 (42)

Never married

357 (41)

188 (43)

169 (39)

Education
Some high school or less

166 (19)

91 (21)

75 (17)

High school graduate

254 (29)

125 (29)

129 (30)

Beyond high school

447 (52)

218 (50)

229 (53)

Employment status
Working
Unemployed/retired/in school/homemaker/other

78 (9)

33 (8)

45 (10)

238 (27)

120 (28)

118 (27)

Disabled and unable to work

551 (64)

282 (65)

269 (62)

Homeless in shelter or on street 1 night


in past 90 d

263 (30)

130 (30)

133 (31)

CDPS medical conditions, mean (SD)35,c

6.12 (3.56)

6.20 (3.54)

6.04 (3.58)

ASI Psychiatric Status composite score >0.3821,d

470 (54)

232 (53)

238 (55)

1 Mental illness ICD-9 diagnosis codec

478 (56)

248 (58)

230 (55)

Medical/psychiatric

Substance use
e

ASI days most frequently used drug, mean (SD)


ASI Drug Use composite score, mean (SD)d,e

13.82 (11.00)

14.40 (11.29)

13.25 (10.69)

0.11 (0.10)

0.11 (0.10)

0.11 (0.10)

ASI drug use, any in past 30 df


Marijuana

656 (76)

333 (77)

323 (75)

Stimulantsf

362 (42)

184 (42)

178 (41)

Cocaine

325 (37)

161 (37)

164 (38)

Amphetamines
Opiatesf
Heroin
Methadone and other opiates/analgesics
nonprescribed
Sedatives/hypnotics/tranquilizers
Other drugse,g

63 (7)

36 (8)

27 (6)

228 (26)

105 (24)

123 (28)

59 (7)

30 (7)

29 (7)

208 (24)

96 (22)

112 (26)

72 (8)

29 (7)

43 (10)

51 (6)

23 (5)

28 (6)

2 or more drugs used in past 30 de

389 (45)

188 (43)

201 (46)

Intravenous drug use in past 30 d

72 (8)

39 (9)

33 (8)

Low (score 1-2)

278 (32)

141 (32)

137 (32)

Intermediate (score 3-5)

328 (38)

169 (39)

159 (37)

Substantial/severe (score 6)

262 (30)

125 (29)

137 (32)

Goal of total abstinence from drugse,h

323 (37)

158 (36)

165 (38)

ASI Alcohol Use composite score, mean (SD)d

0.15 (0.20)

0.14 (0.20)

0.16 (0.20)

ASI alcohol use, any in past 30 d

598 (69)

276 (63)

322 (74)

Nicotine use, any in past 30 d

620 (72)

318 (73)

302 (70)

DAST-10 drug use severitye

jama.com

Abbreviations: ASI, Addiction


Severity Index; BI, brief intervention;
CD, chemical dependency; CDPS,
Chronic Illness and Disability
Payment System; DAST-10, Drug
Abuse Screening Test 10-item; ICD-9,
International Classification of
Diseases, Ninth Revision.
a

Missing values are not included in


this table.

Assessed by self-report using


National Institutes of Health
reporting categories for federally
funded clinical research.

Administrative data available for


848 participants in the 1-year
preintervention period.

ASI composite scores range from 0


to 1, with 1 indicating greatest
problem severity.

Excludes use of alcohol or nicotine.

ASI drug use groups reported are


not mutually exclusive.

Other drugs can include all other


abused medications (eg,
antihistamines, antidepressants) or
drugs of abuse (eg, hallucinogens,
inhalants) not included in the
existing categories.

From the Thoughts about


Abstinence measure, which is used
to assess ones goal for changing
drug use (no goal, controlled use,
occasional use, temporary
abstinence, total abstinence slip is
possible, total abstinence never use
again). The reported Goal of total
abstinence from drugs includes
Total abstinence, never use again
and Total abstinence, slip is
possible.

JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

497

Research Original Investigation

Brief Intervention for Problem Drug Use

Table 2. Select Secondary Outcomesa,b


1 Yearc

Baseline
Brief
Intervention
(n = 426)

Enhanced Care
as Usual
(n = 422)

Brief
Intervention
(n = 426)

Enhanced Care
as Usual
(n = 422)

Regression Coefficient
or Adjusted OR
(95% CI)d,e

P
Value

0.65 (0.32)

0.66 (0.35)

0.54 (0.35)

0.56 (0.36)

0.004 (0.050 to 0.042)d

.86

Psychiatric

0.37 (0.24)

0.39 (0.24)

0.31 (0.26)

0.32 (0.26)

0.004 (0.026 to 0.034)d

.79

Employmenti

0.79 (0.23)

0.79 (0.23)

0.78 (0.24)

0.78 (0.24)

0.006 (0.016 to 0.028)d

.58

Family/socialj

0.17 (0.22)

0.17 (0.22)

0.11 (0.18)

0.13 (0.20)

0.020 (0.046 to 0.006)d

.14

Legalk

0.06 (0.12)

0.07 (0.15)

0.04 (0.10)

0.04 (0.12)

0.000 (0.014 to 0.014)d

.95

Outcome
ASI composite scores, mean (SD)f
Medicalg
h

Public health-relevant measures, No. (%) with any


Washington State chemical
dependency treatment
admissionsl

37 (9)

53 (13)

60 (14)

57 (14)

1.16 (0.77 to 1.73)e

.48

Washington State inpatient


hospitalizationsm

113 (27)

108 (26)

106 (25)

98 (23)

1.09 (0.78 to 1.51)e

.62

Safety-net hospital
emergency department
visits

215 (50)

213 (50)

204 (48)

198 (47)

1.04 (0.76 to 2.06)e

.77

Safety-net outpatient visits

381 (89)

380 (90)

402 (94)

399 (95)

1.00 (0.53 to 1.88)e

.99

36 (8)

43 (10)

41 (10)

37 (9)

1.21 (0.74 to 1.98)e

.45

237 (55)

225 (53)

195 (51)

198 (51)

0.90 (0.66 to 1.25)e

.54

NA

NA

10 (2)

7 (2)

1.42 (0.54 to 3.78)e

.48

Felony or gross
misdemeanor arrests
HIV Risk-taking Behavior
Scale risk factor 1n
Death

Abbreviations: ASI, Addiction Severity IndexLite; ED, emergency department;


HIV, human immunodeficiency virus; NA, not applicable; OR, odds ratio.

ASI composite scores range from 0 to 1, with 1 indicating greatest problem


severity.

Data available for 848 participants unless otherwise indicated.

Data available for 776 participants.

With the exception of death, the preintervention value of the outcome was
used as a covariate in the model. For example, in assessing the association
between randomization and hospitalization within 1 year after intervention,
the proportion hospitalized in the 1 year before intervention was used as a
covariate.

Data available for 764 participants.

Data available for 771 participants.

Data available for 768 participants.

Data available for 748 participants.

Excludes detoxification services.

Results measured at 12-month assessment for ASI composite scores and


during the 1-year postintervention period for public health-relevant measures.

Regression coefficients by linear regression. A positive coefficient indicates


the intervention was associated with a higher score.

Adjusted ORs by logistic regression. Values greater than 1 indicate the


intervention was associated with higher odds of the outcome.

Data available through December 31, 2012; 86% had complete 1-year
information.

Data available for 764 participants.

Figure 2. Changes in Problem Drug Use Over Time


ASI drug use composite score during 12 mo
0.16

12

0.12

Mean (95% CI)

Mean (95% CI)

Drug use days in last 30 d for most frequently used drug


16

Brief intervention
Enhanced care as usual

0.8

0.4

0
0

12

Time Since Randomization, mo


No. of participants
Brief intervention
435
Enhanced care as usual 433

378
389

381
385

377
384

12

Time Since Randomization, mo


386
392

426
420

370
379

374
373

372
375

380
382

The Addiction Severity IndexLite (ASI) Drug Use composite score accounts for frequency of use and associated problems for all drugs used, excluding alcohol and
medications taken as prescribed (range, 0-1; 1 indicates greatest severity).

498

JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

jama.com

Brief Intervention for Problem Drug Use

Original Investigation Research

Table 3. Potential Effect Modification: Estimates for Select Secondary Outcomes by Baseline Drug Use Severitya
Admissions
Emergency Department

Chemical Dependency Treatment


(Estimated Likelihood of 1 Admission)
Brief
Intervention
(n = 426)

Enhanced Care
as Usual
(n = 422)

P
Value

0.04

0.09

.14

4/137

9/135

Intermediate (DAST-10
score 3-5)

0.13

0.13

No. experiencing
event/sample size
or mean (SD)c

20/167

19/156

0.26

0.16

36/122

29/131

Potential Effect
Modifier/Level

Estimated Likelihood of 1 Admission


Brief
Intervention
(n = 426)

Enhanced Care
as Usual
(n = 422)

P
Value
.86

Mean Number Among Admittedb


Brief
Intervention
(n = 204)

Enhanced Care
as Usual
(n = 198)

P
Value
.01

Baseline drug use severity


Low (DAST-10 score 1-2)
No. experiencing
event/sample size
or mean (SD)c

Substantial/severe
(DAST-10 score 6)
No. experiencing
event/sample size
or mean (SD)c

0.79

0.90

53/137

54/135

0.78

0.76

81/167

69/156

0.84

0.77

70/122

75/131

.95

.04

Outcome estimates apply to 1-year postintervention period, conditional on


baseline drug use severity and at mean values of preintervention level of
outcomes, baseline ASI Psychiatric composite score, and abstinence goal on
the Thoughts about Abstinence assessment at baseline.

Treatment differences for emergency department costs were also examined


and contained a number of extreme outliers. Nonparametric rank statistics

First and most important, drug use in the sample was


very heterogeneous, spanning casual marijuana use to
severe opiate and stimulant abuse. Although the DAST-10
score, which measures severity and is strongly related to
type of drug used, did not moderate intervention effects on
frequency of drug use, it did moderate both treatment
uptake and emergency department use in exploratory
analyses of secondary outcomes. Participants in the brief
intervention group with the most severe drug use problems,
with a greater pattern of stimulant and opiate abuse, were
more likely to pursue specialty treatment and had reduced
use of the emergency department relative to those in the
enhanced care as usual group. This is consistent with the 1
positive study of brief intervention for cocaine and opiate
users by Bernstein et al,7 although no effects were found on
treatment uptake. In contrast, no benefit of the intervention
was found for those in the lowest severity of drug use problems, predominantly marijuana users. Although legalization
of marijuana use in Washington State did not occur until
well after enrollment, the legalization climate may have
served to normalize marijuana use, often not associated
with work or personal problems, and reduce motivation for
change for this substance. Second, the sample was severely
socioeconomically disadvantaged. However, studies of alcohol brief intervention have not indicated that disadvantage/
poverty moderates brief intervention effect,36 and the study
by Bernstein et al drew participants from homeless and lowincome clinics. Third, the sample had a high rate of comorbid mental illness, which is known to be associated with
poorer substance use outcomes.37 These last 2 aspects of
jama.com

.90

1.44
1.65 (1.05)

2.80

2.93

2.94 (2.93)

2.83 (4.73)

1.93

3.60

2.63 (2.81)

4.03 (5.29)

.86

.01

stratified by levels of DAST-10 scores revealed no significant treatment


differences.

Abbreviations: ASI, Addiction Severity IndexLite; DAST-10, Drug Abuse


Screening Test 10-item.
a

.45

2.31
2.49 (2.33)

Unadjusted results measured over 1-year postintervention period for number


of participants experiencing 1 or more chemical dependency treatment
admissions, number of participants experiencing 1 or more emergency
department admissions, and mean number of emergency department
admissions among those with 1 or more admissions.

our population (poverty and mental illness), along with the


fact that patients in this study received compensation, may
limit the generalizability of our results.
Other factors may have influenced the effectiveness of the
brief intervention. First, the majority of participants had a
single brief intervention contact, with only 47% receiving a follow-up booster call. Although more frequent contact may increase efficacy,1 some studies have failed to show this.38 Second, the participants primary care physician did not deliver
the intervention, nor did a research-confirmed expert. Although the current use of a physician-extender to deliver the
intervention is consistent with extant integrated care models
and has been shown effective in other contexts,39 the physician might have more influence with a patient.15
Third, despite efforts to simplify the assessment battery,
all participants received up to 5 assessments, in which information about drug use was discussed, as well as DAST-10 feedback. It is possible that this may have inadvertently served to
produce an intervention effect in the comparison group.40
Fourth, measuring frequency but not quantity of drug use is a
limited measure of outcome. Similarly, our measure of alcohol use did not include quantity consumed, limiting our ability to use alcohol as either a predictor or outcome variable. Currently there is no gold standard for quantifying problem drug
use. Researchers measuring problem drug use outcomes must
find a way to measure quantity, as well as frequency, of use. It
is possible that our exploratory finding of reduced emergency department utilization could reflect changes in quantity but not frequency of drug use, although the finding may
also be spurious.
JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

499

Research Original Investigation

Brief Intervention for Problem Drug Use

Despite these limitations, further research to identify subgroups responsive to this intervention, as well as the role of
more intensive interventions, appears to be warranted. For example, targeting intervention efforts toward individuals with
severe drug abuse, many of whom use stimulants and opiates
and may be at higher risk of overdose and other harmful consequences, might increase the uptake of specialty treatment
and reduce emergency department utilization.

ARTICLE INFORMATION
Author Contributions: Dr Roy-Byrne had full
access to all of the data in the study and takes
responsibility for the integrity of the data and the
accuracy of the data analysis.
Study concept and design: Roy-Byrne, Bumgardner,
Krupski, Dunn, Ries, Donovan, Joesch.
Acquisition, analysis, or interpretation of data: All
authors.
Drafting of the manuscript: Roy-Byrne,
Bumgardner, Dunn, Ries, Atkins.
Critical revision of the manuscript for important
intellectual content: Roy-Byrne, Bumgardner,
Krupski, Ries, Donovan, West, Maynard, Atkins,
Graves, Joesch, Zarkin.
Statistical analysis: West, Maynard, Atkins, Graves,
Joesch, Zarkin.
Obtained funding: Roy-Byrne, Bumgardner, Krupski,
Dunn, Ries, Donovan, Joesch, Zarkin.
Administrative, technical, or material support:
Roy-Byrne, Bumgardner, Krupski.
Study supervision: Roy-Byrne, Bumgardner, Dunn,
Ries.
Conflict of Interest Disclosures: All authors have
completed and submitted the ICMJE Form for
Disclosure of Potential Conflicts of Interest. Dr
Roy-Byrne reported receiving financial support as
the editor in chief of Depression and Anxiety,
Journal Watch Psychiatry, and UpToDate Psychiatry
and receiving stock options for consultation to
Valant Medical Solutions (behavioral health
electronic medical record company), outside the
submitted work. Dr Ries reported receiving
financial support from Janssen Pharmaceuticals Inc,
Alkermes, and Reckitt Benckiser Pharamaceutical
Inc, outside the submitted work. No other authors
reported disclosures.
Funding/Support: This study was funded by
National Institute on Drug Abuse grant R01
DA026014 (Dr Roy-Byrne).
Role of the Sponsor: The National Institute on
Drug Abuse had no role in the design and conduct
of the study; the collection, management, analysis,
and interpretation of the data; the preparation,
review, or approval of the manuscript; or the
decision to submit the manuscript for publication.
Disclaimer: The views expressed reflect those of
the authors and do not necessarily reflect the
official views of the National Institute on Drug
Abuse or the National Institutes of Health.
Additional Contributions: We gratefully
acknowledge the participation and support of many
people without whom this study could never have
been completed. These include the patients and
the administrative and clinical staff, including
medical directors, of the primary care clinics that
participated in the study; the interventionists
receiving grant support who spent time to train and
gain proficiency in the motivational interviewing-

500

Conclusions
A one-time brief intervention with attempted telephone
booster call had no effect on drug use in patients seen in safetynet primary care settings. This finding suggests a need for caution in promoting widespread adoption of this intervention for
drug use in primary care.

enhanced brief intervention and then to provide


study interventions to participating clinic patients;
and the expert research staff employed by the
grant for study coordination, data collection
resulting in more than 87% follow-up participation,
and coding of intervention recordings. We also
gratefully acknowledge the expert oversight of our
data and safety monitoring board (Katharine A.
Bradley, MD, MPH [Group Health Research
Institute, Seattle, Washington]; John M. Roll, PhD
[Professor and Senior Vice Chancellor, Washington
State University, Spokane]; and Andrew J. Saxon,
MD [Director, Center of Excellence in Substance
Abuse Treatment and Education, VA Puget Sound
Health Care System; Professor, Department of
Psychiatry & Behavioral Sciences, University of
Washington, Seattle]), and the support of the late
Richard A. Denisco, MD, MPH, our program official
at the National Institute on Drug Abuse. Dr Denisco
and the members of the data and safety monitoring
board did not receive compensation from this
grant.
REFERENCES
1. Kaner EF, Beyer F, Dickinson HO, et al.
Effectiveness of brief alcohol interventions in
primary care populations. Cochrane Database Syst
Rev. 2007;(2):CD004148.
2. Bertholet N, Daeppen JB, Wietlisbach V, Fleming
M, Burnand B. Reduction of alcohol consumption
by brief alcohol intervention in primary care:
systematic review and meta-analysis. Arch Intern
Med. 2005;165(9):986-995.
3. Substance Abuse and Mental Health Services
Administration (SAMHSA). Screening, Brief
Intervention, Referral and Treatment (SBIRT).
SAMHSA website. http://beta.samhsa.gov/sbirt. April
2014. Accessed July 2, 2014.
4. Babor TF, McRee BG, Kassebaum PA, Grimaldi
PL, Ahmed K, Bray J. Screening, Brief Intervention,
and Referral to Treatment (SBIRT): toward a public
health approach to the management of substance
abuse. Subst Abus. 2007;28(3):7-30.
5. Otto C, Crackau B, Lhrmann I, et al. Brief
intervention in general hospital for problematic
prescription drug use: 12-month outcome. Drug
Alcohol Depend. 2009;105(3):221-226.
6. Zahradnik A, Otto C, Crackau B, et al.
Randomized controlled trial of a brief intervention
for problematic prescription drug use in
non-treatment-seeking patients. Addiction. 2009;
104(1):109-117.
7. Bernstein J, Bernstein E, Tassiopoulos K, Heeren
T, Levenson S, Hingson R. Brief motivational
intervention at a clinic visit reduces cocaine and
heroin use. Drug Alcohol Depend. 2005;77(1):49-59.
8. Humeniuk R, Ali R, Babor T, et al. A randomized
controlled trial of a brief intervention for illicit drugs
linked to the Alcohol, Smoking and Substance

Involvement Screening Test (ASSIST) in clients


recruited from primary health-care settings in four
countries. Addiction. 2012;107(5):957-966.
9. Saitz R, Alford DP, Bernstein J, Cheng DM, Samet
J, Palfai T. Screening and brief intervention for
unhealthy drug use in primary care settings:
randomized clinical trials are needed. J Addict Med.
2010;4(3):123-130.
10. Saitz R. Introduction: screening and brief
intervention. Subst Abus. 2007;28(3):1-2.
11. Chan YF, Huang H, Sieu N, Untzer J. Substance
screening and referral for substance abuse
treatment in an integrated mental health care
program. Psychiatr Serv. 2013;64(1):88-90.
12. Compton WM, Thomas YF, Stinson FS, Grant BF.
Prevalence, correlates, disability, and comorbidity
of DSM-IV drug abuse and dependence in the
United States: results from the national
epidemiologic survey on alcohol and related
conditions. Arch Gen Psychiatry. 2007;64(5):566576.
13. Wickizer TM, Campbell K, Krupski A, Stark K.
Employment outcomes among AFDC recipients
treated for substance abuse in Washington State.
Milbank Q. 2000;78(4):585-608.
14. Solberg LI, Maciosek MV, Edwards NM. Primary
care intervention to reduce alcohol misuse: ranking
its health impact and cost effectiveness. Am J Prev
Med. 2008;34(2):143-152.
15. Fleming MF, Barry KL, Manwell LB, Johnson K,
London R. Brief physician advice for problem
alcohol drinkers: a randomized controlled trial in
community-based primary care practices. JAMA.
1997;277(13):1039-1045.
16. Krupski A, Sears JM, Joesch JM, et al. Impact of
brief interventions and brief treatment on
admissions to chemical dependency treatment.
Drug Alcohol Depend. 2010;110(1-2):126-136.
17. Carroll KM, Rounsaville BJ. Bridging the gap:
a hybrid model to link efficacy and effectiveness
research in substance abuse treatment. Psychiatr
Serv. 2003;54(3):333-339.
18. Krupski A, Joesch JM, Dunn C, et al. Testing the
effects of brief intervention in primary care for
problem drug use in a randomized controlled trial:
rationale, design, and methods. Addict Sci Clin Pract.
2012;7(1):27.
19. Smith PC, Schmidt SM, Allensworth-Davies D,
Saitz R. A single-question screening test for drug
use in primary care. Arch Intern Med. 2010;170(13):
1155-1160.
20. Yudko E, Lozhkina O, Fouts A. A
comprehensive review of the psychometric
properties of the Drug Abuse Screening Test.
J Subst Abuse Treat. 2007;32(2):189-198.
21. Alterman AI, McLellan AT, Shifman RB. Do
substance abuse patients with more

JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

jama.com

Brief Intervention for Problem Drug Use

Original Investigation Research

psychopathology receive more treatment? J Nerv


Ment Dis. 1993;181(9):576-582.
22. Hall SM, Havassy BE, Wasserman DA.
Commitment to abstinence and acute stress in
relapse to alcohol, opiates, and nicotine. J Consult
Clin Psychol. 1990;58(2):175-181.
23. Doig GS, Simpson F. Randomization and
allocation concealment: a practical guide for
researchers. J Crit Care. 2005;20(2):187-193.
24. Roche AM, Freeman T. Brief interventions:
good in theory but weak in practice. Drug Alcohol Rev.
2004;23(1):11-18.
25. Baer JS, Rosengren DB, Dunn CW, Wells EA,
Ogle RL, Hartzler B. An evaluation of workshop
training in motivational interviewing for addiction
and mental health clinicians. Drug Alcohol Depend.
2004;73(1):99-106.
26. Moyers TB, Martin T, Manuel JK, Hendrickson
SM, Miller WR. Assessing competence in the use of
motivational interviewing. J Subst Abuse Treat.
2005;28(1):19-26.
27. Moyers TB, Martin T, Manuel JK, Miller WR,
Ernst D. Revised Global Scales: Motivational
Interview Treatment Integrity (MITI 3.1). University
of New Mexico Center on Alcoholism, Substance
Abuse, and Addictions (CASAA) website. http://casaa
.unm.edu/download/MITI3_1.pdf. January 22, 2010.
Accessed July 16, 2014.

28. Kypri K. Methodological issues in alcohol


screening and brief intervention research. Subst Abus.
2007;28(3):31-42.

outcome of, screening and brief interventions for


alcohol misuse: a review. Alcohol Alcohol. 2006;41
(5):540-545.

29. Cacciola JS, Alterman AI, McLellan AT, Lin YT,


Lynch KG. Initial evidence for the reliability and
validity of a Lite version of the Addiction Severity
Index. Drug Alcohol Depend. 2007;87(2-3):297-302.

37. Krupski A, Sears JM, Joesch JM, et al.


Self-reported alcohol and drug use six months after
brief intervention: do changes in reported use vary
by mental-health status? Addict Sci Clin Pract. 2012;
7(1):24.

30. Darke S, Hall W, Heather N, Ward J, Wodak A.


The reliability and validity of a scale to measure HIV
risk-taking behaviour among intravenous drug
users. AIDS. 1991;5(2):181-185.
31. Ashenfelter O, Card D. Using the longitudinal
structure of earnings to estimate the effects of
training programs. Rev Econ Stat. 1985;67:648-660.
32. Hardin J, Hildbe J. Generalized Estimating
Equations. 2nd ed. Boca Raton, FL: CRC Press; 2013.
33. Touloumis A, Agresti A, Kateri M. GEE for
multinomial responses using a local odds ratios
parameterization. Biometrics. 2013;69(3):633-640.
34. Team RCR. A Language and Environment for
Statistical Computing. Vienna, Austria: R Foundation
for Statistical Computing; 2013.

38. DOnofrio G, Fiellin DA, Pantalon MV, et al. A


brief intervention reduces hazardous and harmful
drinking in emergency department patients. Ann
Emerg Med. 2012;60(2):181-192.
39. Sullivan LE, Tetrault JM, Braithwaite RS, Turner
BJ, Fiellin DA. A meta-analysis of the efficacy of
nonphysician brief interventions for unhealthy
alcohol use: implications for the patient-centered
medical home. Am J Addict. 2011;20(4):343-356.
40. Neighbors C, Lewis MA, Atkins DC, et al.
Efficacy of web-based personalized normative
feedback: a two-year randomized controlled trial.
J Consult Clin Psychol. 2010;78(6):898-911.

35. Kronick R, Gilmer T, Dreyfus T, Lee L. Improving


health-based payment for Medicaid beneficiaries:
CDPS. Health Care Financ Rev. 2000;21(3):29-64.
36. Littlejohn C. Does socio-economic status
influence the acceptability of, attendance for, and

jama.com

JAMA August 6, 2014 Volume 312, Number 5

Copyright 2014 American Medical Association. All rights reserved.

Downloaded From: http://jama.jamanetwork.com/ by Miguel Herrera on 05/12/2016

501

Vous aimerez peut-être aussi