Evidence Based Medicine

on
Therapy Study

Siti Setiati
 Outline

{ Reports of individual studies

{ Systematic reviews
{ Qualitative literature
{ Economic analysis
Reports of individual studies
Are the result of this individual study
valid?

1. Was the assignment of patients to treatment

randomized?
2. Was the randomization concealed ?
3. Were the groups similar at the start of the
trial ?
4. Was follow-up of patients sufficiently long and
complete
5. Were all patients analyzed in the groups to
which they were randomized
6. Were patients, clinicians and study personnel
kept blind to treatment
7. Were groups treated equally, apart from the
experimental therapy
1 - Was the assignment of patients to
treatment randomized?

{ The method section should tell how patients were

allocated to groups

{ Randomization balances the treatment groups for

prognostic factors

{ If the study wasnt randomize stop reading it & go

on to the next article in your search
2 - Was the randomization concealed ?
{ If allocation was concealed, the clinicians would be
unaware of which treatment the next patient would
receive & thus unable consciously or unconsciously,
to distort the balance between the groups being
compared
3 - Were the groups similar at the start of
the trial ?

{ If randomisation process worked, the group

should be similar
{ The result should have a table of baseline
characteristics
{ If not, there may be a description of group
similarity in the first paragraph of the results
{ In well-designed randomized trials, any
observed differences are due to chance
 4 - Was follow-up of patients sufficiently
long and complete

{ Follow-up of patients should

sufficiently long to see a
clinically important effect

{ Loss to follow-up should be

minimal-preferably less than
20%

{ Inclusion of flow diagrams

aims to enhance reporting
accuracy of trials
4 - Was follow-up of patients sufficiently
long and complete

{ The result section should say how many patients

were randomised & how many patients were
actually included in the analysis

{ Reason for losses to follow-up should be stated in

the analysis
5 - Were all patients analyzed in the groups to
which they were randomized

{ Patient should be analysed in the group to

which they were randomised intention
to treat analysis.
 6 - Were patients, clinicians and study personnel
kept blind to treatment

{ Ideally the study is double-blinded

{ If the outcome is critical (eg. death) then blinding is
less critical
{ If the outcome is subjective (eg.symptoms or
function) then blinding of the outcome assessor is
critical
{ Look in the method section to see if there is some
mention of masking of treatments
{ Themethod section should describe how the outcome
was assessed & whether the assessor/s were aware of
the patients treatment
7 - Were groups treated equally, apart from
the experimental therapy

{ Blinding of patients, clinical & study personnel can

prevent them from adding any additional treatments
(or co-intervention), apart from the experimental
treatment

{ Information about con-interventions can be found in

the methods and/or results section of an article
Are the valid results of this individual
study important ?

1. What is the magnitude of the treatment

effect ?
2. How precise is the estimate of the
treatment effect ?
1 - What is the magnitude of the treatment
effect ?
{ Requires assessment of the outcomes that included
in the study
{ Some trials will report surrogate outcomes
Ex: BMD as surrogate outcomes in study assessing
the effectiveness of osteoporosis therapy
{ Surrogates outcome can reduce sample size &
follow-up time.
{ Composite outcome also frequently seen in trial
 1 - What is the magnitude of the treatment
effect ?

Outcome Outcome
Present Absent

Treated a= b= EER(Experimental
/exposed Event Rate)
a/ (a+b)
Control / c= d= CER (Control
hot exposed Event Rate)
c / (c+d)

RRR (relative risk reduction) = (CER-EER) / CER

ARR (absolute risk reduction) = CER EER
NNT (number needed to treat) = 1/ARR
 1 - What is the magnitude of the treatment
effect ?
EER. The proportion of patients in the experimental treatment
group who are observed to experience the outcome of interest
CER. The proportion of patients in the control group who are
observed to experience the outcome of interest
ARR. The absolute arithmetic difference in rates of bad
outcomes between experimental and control participants in a
trial. (sometimes called the risk differences)
ARR 0 theres no difference between the two groups,
treatment had no effect
RRR. The proportional reduction in rates of bad outcomes
between experimental and control partici pants in a trial.
NNT. The number of patients who need to be treated with the
specified intervention to prevent one bad outcome or produce
one good outcome over the period of time specified in the
study.
 1 - What is the magnitude of the treatment
effect ?

Measures of effect size

Even rate= stroke (mean Relative Risk Absolute Number

follow-up 5 years) Reduction Risk needed to
(RRR) Reduction treat (NNT)
(ARR)
Control Experimental |CER EER|/ |CER EER| 1/ARR
event rate even rate CER
(CER) (EER)

MRC trial 5.7% 4.3% |5.7- {5.7- 1/1.4%=72

4.3|/5.7%=25 4.3%|=0.014
% or 1.4%

Hypothetical, 0.000057% 0.000043% |0.000057% - |0.000057% - 1/0.000014%

trivial case 0.000043%| / 0.000043%|= = 7142857
0.000057% = 0.000014%
25%
 1 - What is the magnitude of the treatment
effect ?
Intensive vs standard glucose control to prevent vacular events in
type 2 diabetes
Outcomes at Intensive Standard of RRR (95% CI) NNT (CI)
median (5 year) control (%) control (%)

Macro or 18 20 9% 2 to 16 56% 31 to 280

microvascular event

Macrovascular 10 11 6% 16t o 15 Not significance

event
Microvascular event 9.4 11 13% 3 to 22 69% 42 to 324

New or worsening 4.1 5.2 21% 7 to 33 94% 58 to 282

nephropathy
Macroalbuminuria 2.9 4.1 30% 15 to 42 83% 58 to 166

New onset -
microalbuminuria

Severe 2.7 1.5 85% 42 to 137 79% 49 to 162

hypoglycemia
2 - How precise is the estimate of the treatment
effect ?

{ Confidence interval (CI)

Provides the range of values that are likely to
include the true risk & quantifies the uncertainty in
measurement
{ In its absence p-value tells statistical significance
{ The smaller the number of patients in the study that
generated the NNT, the wider its CI
{ CI and significance test are closely related
{ A significant P value will correspond to95% CI
2 - How precise is the estimate of the treatment
effect ?
Example :
PROGRESS trial reported :
{ blood pressure lowering after stroke or TIA reduced the
absolute rates of ischemic stroke from 10%to 8% (RRR
24%; 95% CI 10 to 35)
95% confidence that the true RRR value lies between
10% and 35% with 25% being the most likely value
{ ARR is 2% (from 10% to 8%), for which we can calculate
95% CI from 1% to 3.5% (using the 10% control group
rate & 95% CI for RRR)
{ CI for the NNT = 1/0.01 to 1/0.035 = 100-29
Are the valid, important results of this
individual study applicable to our patient ?

1. Is our patient so different from those in the study

that its results cannot apply ?
2. Is the treatment feasible in our setting ?
3. What are our patients potential benefits and
harms from the therapy ?
4. What are our patients values and expectations for
both the outcome we are trying to prevent and the
treatment we are offering ?
Report of Systematic Review
Is the evidence fromthis systematic
review valid ?

1. Is this a systematic review of randomized trial ?

2. Does it describe a comprehensive and detailed
search for relevant trials ?
3. Were the individual studies assessed for validity ?
4. Were individual patient data (or aggregate data)
used in the analysis
Is the valid evidence from this
systematic review important ?

1. Are the results consistent across studies ?

2. What is the magnitude of the treatment
effect?
3. How precise is the treatment effect ?
Are the valid, important results of this
systematic review applicable to our
patient ?

1. Is our patient so different from those in the study

that its results cannot apply ?
2. Is the treatment feasible in our setting ?
3. What are our patients potential benefits and
harms from the therapy ?
4. What are our patients values and expectations for
both the outcome we are trying to prevent and the
adverse effects we may cause ?
Qualitative literature
 Is this evidence from a CDA valid ?
1. Were all important therapeutic alternatives
(including no treatment) and outcomes
included ?
2. Are the probabilities of the outcomes valid
and credible ?
3. Are the utilities of the outcomes valid and
credible ?
Is this valid evidence from a CDA
important ?

1. Did one course of action lead to

clinically important gains ?
2. Was the same course of action
preferred despite clinically sensible
changes in probabilities and
utilities ?
Is this valid and important evidence
from a CDA applicable to our patients ?

1. Do the probabilities in this CDA

apply to our patients ?
2. Can our patient state his/her
utilities in a stable, useable form ?
Reports of Economic Analysis
 Is this evidence from an economic
analysis valid ?

1. Are all well-defined courses of action compared ?

2. Does it provide a specified view from which the costs
and consequences are being viewed ?
3. Does it cite comprehensive evidence on the efficacy
of alternatives ?
4. Does it identify all the costs and consequences we
think it should and select credible and accurate
measures of them ?
5. Was the type of analysis appropriate for the question
posed ?
Is this valid evidence from an economic
analysis important ?

1. Are the resulting cost or cost/unit of

health gained clinically significant ?
2. Did the results of this economic analysis
change with sensible changes to cost and
effectiveness
Is this valid and important evidence from
an economic analysis applicable to our
patient ?

1. Do the costs in the economic analysis

apply in our setting ?
2. Are the treatments likely to be effective in
our setting ?