Understanding articles about treatment

Dan Merenstein, MD
Assistant Professor and Director of Research Programs, Family Medicine Georgetown University

Objectives
Explain benefits of RCTs Internal and External Validity Allocation Concealment Blinding Limitations

Probiotics
Live microorganisms when administered in adequate amounts confer a beneficial health effect on the host- WHO definition All yogurts have at least two probiotics (Lactobacillus bulgaricus and Streptococcus thermophilus)

Case
You are seeing a 34 y/o woman who has 3 kids and has heard a lot about these new miracle products called probiotics. She wants to know if it will keep her kids healthy and if so what specifically should she give them.

Study Methods to Answer This Question

Epidemiology: Countries that consume more probiotics have less episodes of ear infections Biology: LGG ointment increases T cells in rats Case report: It worked on one patient Case-series: It worked on a bunch of patients Randomized controlled trial: 1/2 get probiotic, 1/2 placebo. No one knows who until the end who took what

Randomization is the best protection against being mislead

Levels of Evidence (LOE)

Centre for Evidence-Based Medicine, Oxford
Expert Opinion: LOE = 5 Case Series: LOE = 4 Case Control: LOE = 3b RCT: LOE = 1b Systematic review with homogeneity = 1a

http://www.cebm.net/levels_of_evidence.asp

The value of randomization

32 controlled trials of anticoagulation in acute MI Results by type of study:
Relative Risk Reduction Historical control Randomized controlled trial Concealed Allocation 42% 31% 18% Case fatality rate 38.3% 19.6% 12.1%

Come Up Front for Snack

There are 10 drinks Some are regular yogurt with two starter cultures Others are identical except we add BB-12 at very high dose Taste, smell, compare next to each other Look at the bottles and consistency

Strawberry Yogurt
Can the research assistant tell which is the active or placebo-allocation and blinding Can participants tell which is active or placebo-blinding Is this something you can see yourself or patient drinking-external validity Is the amount necessary to consume reasonable-external validity What are the side effects and cost-external validity

Patient Oriented Evidence that Matters (POEM)

Disease Oriented Evidence (DOE) Generally are RCT and meta-analysis but dont have to be CAST Example

Validity
Internal validity: How well was the study done? Do the results reflect the truth? External validity: can I apply these results to MY patients?

Determining Validity
Read the methods section What is the primary outcome Study design flaws are common, but are they fatal?

Conducting a Study
Potential Subjects
Trial starts

Concealed Allocation

Actual Subjects

Randomization Blinding, etc

Was allocation assignment concealed?

Concealed allocation blinding
Blinding can occur without concealment and concealment can occur without blinding
UVA example- surfactant in the NICU

Allocation can be concealed in an unblinded study

PT vs surgery for knee DJD
Moseley JB, O'Malley K, Petersen NJ, et al. N Engl J Med 2002; 347:81-8. 347:81-8.

Was study double-blinded?

Did the patients know to which group they were assigned? Did the treating physician know? Triple blinding-Did investigators assessing outcomes know

Were intervention and control groups similar?

Table 1 of most studies Randomization is best way to avoid bias, though imbalances still can occur (especially if allocation was not concealed) Review table 1 for both clinical and statistical differences Small differences sometimes are important

Participant Demographics
No. Subjects Gender --Male --Female --Female Age --MEAN (sd) (sd) Hours/week in school Hours/ --Less than 15 hours --Less --15-20 hours --21-30 hours --More than 40 hours --31-40 hours Income --Less than $15,000 --Less --$15,000-$30,000 --$30,001-$50,000 --$50,001-$75,000 --$75,001-$100,000 --More than $100,000 13 ( 4.3%) 41 ( 13.7%) 28 ( 9.4%) 38 ( 12.7%) 55 ( 18.4%) 124 ( 41.5%) 29 ( 9.1%) 40 ( 12.6%) 26 ( 8.2%) 38 ( 11.9%) 43 ( 13.5%) 142 ( 44.7%) 6 ( 1.9%) 33 ( 10.5%) 62 ( 19.7%) 143 ( 45.5%) 70 ( 22.3%) 5 ( 1.5%) 44 ( 13.6%) 47 ( 14.5%) 159 ( 49.1%) 69 ( 21.3%) 4.86 (+1.12) 4.94 (+1.13) 157 ( 50.0%) 157 ( 50.0%) 172 ( 53.1%) 152 ( 46.9%) Active=314 Placebo=324

Are the study patients similar to yours?

Addresses generalizability of results to your practice (external validity) Examples:
Probiotic supplements Very high dose probiotics

Were all the patients properly accounted for at its conclusion?

Complete follow-up? Intention to treat analysis?
Patients are analyzed in the groups to which they are assigned Attempts to reflect real world clinical situations in which not all patients are compliant Watch when they compare compliers and exclude non-compliers
Compliant subjects always do better overall

Participant Flow Chart

745 Screened 681 Assessed for Eligibility 106 Not Eligible 638 Randomized

314 Intervention

324 Control

Nonfebrile Seizure Incidence

Diabetes Complications and Control Trial

Patients:
Type 1 diabetes, 13-39 years old No HTN, cholesterol or diabetic complications Willing to check BS QID, inject insulin 3-4 times/day Monthly visits for 6.5 years Twice weekly phone follow-up x 6.5 years No co-morbid conditions

Are these patients representative of type 2 diabetics seen in most offices?

ADA uses these results to support tight glucose control in type 2 DM

Ethical Dilemma
A nonrandomized study showed that MVI prevented neural tube defects Does treating neonates with high levels of oxygen prevent death Determination if PSA tests prevent prostate cancer death Determination if Pap Smears prevent cervical death

New PSA Studies

Two studies, total N=253,000 men Take home points
Seven fewer cancers for every 10,000 men screened and followed for 9 years

Example
One man is screened, + test leads to biopsy There is a 1 in 50 chance that he was spared death in next 10 years However 49/50 chance he was treated unnecessarily for a cancer that would have no impact on his life

Types of RCTs
Efficacy- ideal or laboratory circumstances Effectiveness-ordinary or real life situations Practical Clinical Trials-designed specifically to answer questions by decision makers. Generally have more than one outcome

RCT Shortcomings
External Validity Adherence Randomization Expense Bias

Evidence
OBJECTIVES: To determine whether parachutes are effective in preventing major trauma related to gravitational challenge. DESIGN: Systematic review of randomised controlled trials. DATA SOURCES: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists. STUDY SELECTION: Studies showing the effects of using a parachute during free fall. MAIN OUTCOME MEASURE: Death or major trauma, defined as an injury severity score > 15. RESULTS: We were unable to identify any randomised controlled trials of parachute intervention. CONCLUSIONS: As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

Summary
Determine Relevance
Is it a common POEM that will require you to change your practice?

Determine Validity
Proper allocation and blinding Intention-to-treat analysis Weigh fatal vs. nonfatal flaws Ideal RCT: study of patients similar to yours