Vous êtes sur la page 1sur 10

COMMUNITY MEDICINE 5.

01
CRITICAL APPRAISAL
Dean M. Sosa & Assoc. | January 18, 2019
Transcriber group: 16B
INTRODUCTION WHAT IS EVIDENCE-BASED MEDICINE?
Alright fam! So this trans is a combination of the lecture Dean
Sosa gave plus a transcript of the two videos we were Evidence-based medicine is the integration of best evidence
supposed to watch. It’ll be divided into those three main parts. with clinical expertise and patient values. - Dave Sackett 2001
Good luck studying!

• PAGE 1 – DEAN SOSA LECTURE


• PAGE 2 - CRITICAL APPRAISAL OF THERAPY (VIDEO)
• PAGE 5 - CRITICAL APPRAISAL OF DIAGNOSIS (VIDEO)

PRACTICE OF MEDICINE BEFORE EBM

• Before Evidence-Based Medicine (EBM) thinking began to


impact on its structure, clinical practice relied on expert
advice — often driven by physiological reasoning and
individual clinicians’ experience.
• This emphasis resulted in significant gap between the
available evidence and the clinical practice.

HOW IMPORTANT IS EVIDENCE BASED MEDICINE PRACTICE?

• Applying the knowledge gained from large clinical trials to


patient care promotes consistency of treatment, and
optimal outcomes helps establish national standards of
patient care, and sets criteria to measure and reward Clinical Expertise > Research Evidence
performance-based medical practice.
• Implementation of EBM in the managed care setting
provides standards that have the potential to provide the
best medical care at the lowest cost.

AN EBM APPROACH TO EDUCATION

• 1990: Sackett’s “Just in Time” learning


• Evidence cart on ward rounds – 1995
• Looked up 2-3 questions per patient
• Took 15-90 seconds to find
• Change about 1/3 decisions
• Rounds took longer!
• *David Lawrence Sackett was an American-Canadian
medical doctor and a pioneer in evidence-based
medicine. He is known as one of the fathers of Evidence-
Based Medicine.
CRITICAL APPRAISAL

• Claims of effectiveness
• Claims of accuracy
• Claims on causation
• Claims on prognosis

1 of 10
FOUR STEPS IN EBM o PICO
o P: population
1. Formulate answerable question o I: intervention
2. Track down the best evidence o C: comparison/comparator
3. Critically appraise the evidence o O: outcome
4. Individualise, based clinical expertise and patient • Research
concerns • Retrieving medical literature

FORMULATION OF DIAGNOSIS QUESTION

Is Positron Emission Tomography (Test/intervention) good test


for coronary disease (outcome)?

Is PET (test/intervention) a more sensitive and specific test


(outcome) in diagnosing coronary artery disease (patient) as
compared to coronary angiography (comparison)?
Its practice requires:
• Asking
• Acquiring
• Appraising
• Applying
• Assessing

Up next is the “Critical Appraisal of RCT for students — • The highest level of evidence for testing effectiveness of a
therapeutic” video transcript. Feel free to watch the video treatment or a therapy is found in randomized controlled
while reading this trans. It’s only 28 min and 54 seconds long. double-blind studies and systematic reviews and meta-
Kaya mo yan! analyses
• Randomized Controlled Trial
EVIDENCE-BASED MEDICINE o A study in which participants are randomly
allocated to an experimental or comparison
• Defined as a systematic approach to the acquisition, group
appraisal, and application of research evidence to the o The experimental group gets an intervention
decisions in health care o The comparison group gets something different
• Involves three skills such that evidence should be: (either no intervention, a placebo, or a different
o acquired intervention from that received by the
o appraised experimental group)
o applied o Outcomes in each group are compared to
STEPS IN ACQUIRING EVIDENCE determine the effect of the intervention
o The purpose of randomization is to make the two
Asking a focused clinical question groups equal in which the only difference is the
• Most important because if you ask the wrong question, intervention
you will never find the right answer o Results of RCT’s can infer causality which can
• Perhaps the hardest skill to learn attribute differences in outcomes to the
• Should be well-stated in terms of the following variables: differences in the treatment
o PEO STEPS IN APPRAISING ARTICLES
o P: patient population with a certain disease or
condition APPRAISING DIRECTNESS
o E: exposure (or treatments) to be administered • How well does the PEO or PICO in the study (or research
to these patients question) correspond with your own PEO or PICO (clinical
▪ E1 or E – the experimental treatment question)
being evaluated (usually a new one) • Example:
o Among children 1-5 years old age (P), how
▪ E2 or C – the control treatment to which
effective is zinc supplementation (E1 or E)
E1 is being compared to
compared to placebo (E2 or C) in preventing
o O: outcome (or conditions) that the treatment is
acute diarrhea (O)?
intended to prevent

2 of 10
Research Question(s) Clinical Question(s) • Q3: Were baseline characteristics similar at the start of
the trial?
P o Compare baseline characteristics between
groups in terms of prognostic factors
E o Randomization is key
• Q4: Were patients blinded to treatment assignments?
O o Helps to avoid performance bias
o Blinding helps in the objective assessment of the
APPRAISING VALIDITY (INTERNAL VALIDITY) treatment response
• Done only after making sure that the clinical question o Patients may have the tendency to report
matches the research question symptoms more if they know they are receiving
• Assess the risk of bias and decide if the results are placebo, which will make the active drug seem
trustworthy better
o Selection bias — systematic differences between o Tip is to look for identical preparations
baseline characteristics of the groups that are • Q5: Were caregivers blinded to treatment assignments?
compared o Helps to avoid performance bias
o Performance bias — systematic differences o The goal is to provide equal treatments and
between groups in the care that is provided, or in behaviour to the experimental and control
exposure to factors other than the interventions groups other than the intervention itself
of interest • Q6: Were outcome assessors blinded to treatment
o Detection bias — systematic differences assignment?
between groups in how outcomes are o Helps to avoid detection bias
determined o The goal is to assess outcomes the same way for
o Attrition bias — systematic differences between both experimental and control groups
groups in withdrawals from a study o Objective judgment of outcomes is necessary
• Q7: Were all patients analysed in the groups to which
QUESTIONS ANSWERED BY APPRAISING VALIDITY they were originally randomized?
• Q1: Were patients randomly assigned to treatment o Helps to avoid attrition bias
groups? o Possible reasons why patients may not adhere to
o Randomization ensures that both groups are treatment groups
comparable ▪ Forgetting how to take drugs properly
o To check for randomization, look for the words ▪ Suffering from side effects of the drug
randomization, randomized, or randomly ▪ Refusing to go on for no apparent
assigned in the title, abstract or methodology reason
• Q2: Was allocation concealed? o Intention To Treat (ITT)
o Allocation sequence — random sequence by
▪ Once randomized, participants are
which patients are assigned to treatment groups
analysed in their assigned group
o Allocation concealment — measure taken to
regardless of:
ensure that the sequence is not altered
• Their status
knowingly or unknowingly
• Whether they were lost to
▪ Placement of treatment assignment in follow-up
concealed envelopes • Even if they never received
▪ Delegation of a third party to assign treatment, or
treatment (for example, a pharmacist) • Whether they crossed over to
▪ Use of objective computers the other treatment group
o Used to avoid selection bias ▪ Benefits:
▪ Groups should have similar baseline • Groups stay equal
characteristics • The power of the study is
▪ Achieved through random sequence maintained
generation and concealed allocation • An estimated of “real world”
▪ Bias is committed if a participant’s effectiveness
decision to provide consent or a • Q8: Was follow-up rate adequate?
recruiter’s decision to enroll a o Adequacy of follow-up refers to minimization of
participant is influenced by knowledge the number of patients who drop out of the
of which group a patient would be in if study, usually due to adverse events or
they participated (selection bias) dissatisfaction
o Helps to avoid attrition bias

3 of 10
o Study validity is threatened if the number of • Confidence interval (CI) provides range of possible values
participants lost to follow-up is high of the treatment effect.
o Performing sensitivity analyses would determine o Usually reported at 95% level of confidence
significant dropout rate
▪ 95% CI means “we are 95% sure that
o “5-and-20 rule of thumb”
the true effect of the treatment lies
▪ <5% loss to follow-up — little bias
within this range”
▪ 5-20% loss — small bias • In a study on warfarin treatment, the reported RRR is
▪ >20% — serious threat to validity 79% with a 95% confidence interval of 52% and 90%
(RRR: 79% (95% CI: 52%, 90%). What does that mean?
CLASSIFICATION SCHEME FOR BIAS
o That means that warfarin reduces risk of stroke
- See table 1 in patients with atrial fibrillation by 79% but the
actual RRR could range from 52-90%.
IF NO SIGNIFICANT BIAS HAS BEEN COMMITTED Interpreting Confidence Intervals (CI)
• When both ends of the CI are on the side of benefit, the
• Move on to the following steps: treatment is definitely beneficial or superior
o Appraising the results
o Ex. RR of 0.63 (95% CI: 0.53,0.73)
o Assessing applicability
o It means that the confidence interval lies on the
o Individualizing the results
side of benefit because it is less than 1, thus, the
APPRAISING RESULTS treatment is beneficial or superior

There are two questions that need to be considered: • When both ends of the CI are on the side of harm, the
1. How large was the treatment effect? treatment is definitely harmful or inferior
2. How precise was the estimate of the treatment o Ex. RR of 2.3 (95% CI: 1.5,3.1)
effect? o The 95% confidence interval is between 1.5-3.1
then treatment is definitely harmful.
Q1. How large was the treatment effect?
• There are two ways of expressing the results of article on
• When one end reflects important benefit and the other
effectiveness depending upon the outcome of variable —
dichotomous or continuous variable end reflect important harm, the study is inconclusive.
o Ex. RR of 0.98 (95% CI: 0.3,2.5)
Ways of expressing effectiveness (see table 2) o For example, we now present you the results of a
relative risk, remember that you interpret a
• For dichotomous variable, it can be expressed as: relative risk of <1 as beneficial treatment and >1
o Percentage/proportion (e.g. deaths per 100 as harmful. Now for the relative risk of 0.98, it is
patients) seemingly beneficial. However, when you take a
o Rate (e.g. deaths per 100 patients per year) look at the 95% confidence interval, the lower end
• Comparing results between two groups with a of 0.3 is beneficial while the higher end of 2.5 is
dichotomous variable can be expressed as: harm, that means that the study is inconclusive
o Proportion
▪ Relative Risk Reduction (RRR) • When one end reflects unimportant benefit and the
▪ Absolute Risk Reduction (ARR) other end reflects unimportant harm, then for all intents
▪ Relative Risk (RR) and purposes the two treatments being compared are
o Rate the same or equivalent.
▪ Hazard Ratio o Ex. RR of 0.98 (95% CI: 0.95,1.2)
• Continuous variables can be expressed as: o Again, our example is the relative risk, 0.98 < 1
o Mean that means treatment is beneficial but when you
▪ Mean difference can be used to take a look at the 95% confidence interval, the
compare results between two groups lower end is 0.95 < 1 while the higher end is 1.02
which is a little bit more than 1. This reflects
Measure of effectiveness for dichotomous outcome (see unimportant benefit for the lower confidence
table 3) interval and unimportant harm for the higher
confidence interval which means that the two
Q2. How precise was the estimate of the treatment effect?
treatments being compared are the same or
• RRR, ARR and RR are examples of point estimates
equivalent.

4 of 10
ASSESSING APPLICABILITY o Determine prognosis with unknown disease
• General objective: to apply the various rules of evidence
• After evaluating validity and analyzing the results of the
in deciding if the results of an article on a diagnostic test
trial, the next step is to decide if the results can be applied
are valid
to our own patients.
• Specific objectives: to discuss the rational for each
• In making decisions in applicability is best to use a
appraisal step pertaining to the validity of claims on
measure of effectiveness that is:
accuracy
o Stable
• To learn the definitions of, calculations for and
o Constant across different population.
interpretation of sensitivity, specificity, positive predictive
• Now we can be guided using SCRAPS
value, negative predictive value, likelihood ratio, pre-test
o S – Sex: consider physiologic, hormonal or
probability, post-test probability, pre-test odds, post-test
biochemical differences between sex that may
odds, diagnostic and therapeutic threshold
affect results of interventions
STEPS IN APPRAISING ARTICLES ON DIAGNOSIS
o C – Comorbidities: consider coexistent
conditions • Appraising directness
▪ Are there any coexistent conditions in • Appraising validity
the study that are not present in your • Appraising the results
patient that can affect the results of the • Assessing applicability
intervention? • Individualizing the results
o R – Race: consider racial differences ASKING A FOCUSED CLINICAL QUESTION ON THERAPY (PEO)
o A – Age: consider age differences • P = the patient population on whom the test might be
o P – Pathology: consider actual disease under done
study • E = the exposure (in this case, the test to be performed,
o S – Socioeconomic factor: usually studies are referred to as the index test)
conducted in an ideal condition which is difficult • O = the outcome (the condition that the test is supposed
to apply in everyday life to diagnose
INDIVIDUALIZING RESULTS ASKING A FOCUSED CLINICAL QUESTION ON DIAGNOSIS
Example: Among post-menopausal women presenting with a
• It is done when biologic and socioeconomic factors will
breast mass (P), how accurate is fine needle aspiration biopsy
not compromise effectiveness
(E) in establishing the presence or absence of breast cancer
• Studies report the effectiveness of a treatment in a
(O)?
population as a whole, the benefits, cost and risks varied
APPRAISING DIRECTNESS
greatly from patient to patient
• Variability affects the ARR to a greater extent than RRR • Evaluate how well the PEO in the study (research
due to patient’s baseline risk question) corresponds with your own PEO (clinical
• Number needed to TREAT to avoid ONE unfavourable question)
outcome Research Clinical
• 1/risk difference (ARR) = NNT (better) description for Questions Questions
clinical significance)
• Example: ARR is 10% P
• NNT = 1/0.10 = 10 (treat 10 patients to avoid ONE
E
unfavourable outcome
O
Whew! That’s the end of that! Up next is the “Critical Appraisal
of Diagnosis” video transcript. Feel free to watch the video and
• Main question: Does the study provide a direct enough
read along with the trans. It’s only 49 min and 55 seconds.
answer in terms of patients, examination or test used, and
Note that there is some overlap with the other video and this
disease/outcome diagnosed?
video. Anyway, enjoy!
• Evaluating directness: Does the study provide a direct
INTRODUCTION
enough answer to your clinical question in terms of
• Reasons to run diagnostic tests patients (P), examination or test (E) used and disease or
o Diagnose presence or absence of condition outcome (O) being diagnosed?
o Plan an intervention (surgery)
o Monitor response to therapy
o Estimate risk of future evens

5 of 10
• Why is it important? - Many times, the patient you are o What to look for: Check whether measures were
managing, the test you are about to use, or the outcome taken to ensure that the result of the index test
you are trying to prevent is not exactly the same as those was not an indication for doing the reference
studied by the authors of a paper. In this situation, you standard.
need to decide if you can use the study results at all. The • QUESTION 4: WAS “INTERPRETATION” OF THE INDEX
decision requires some expertise on the disease under TEST AND THE REFERENCE STANDARD INDEPENDENT?
question. o Appraising validity: Was “interpretation” of the
• What to look for: Seek the opinion of an expert (this might index test and the reference standard
be you), or your colleagues. independent?
APPRAISING VALIDITY o Why is it important: If in a study setting,
knowledge of the results of an earlier index test
• To make sure the odds are not stacked for or against a may influence interpretation of the reference
particular diagnostic test (bias), there are four criteria to standard, making the index test seem better than
assess the validity of studies on diagnostic accuracy. it really is.
• They are phrased as yes or no questions. (the more yes, o What to look for: Look for attempts to make sure
the better the study) interpretation of one test did not affect
• QUESTION 1: WAS THE REFERENCE STANDARD AN interpretation of the other.
ACCEPTABLE ONE?
o Appraising validity: Was the reference standard APPRAISING THE RESULTS
an acceptable one?
o Why is it important: A reference standard that is • QUESTION 1: WHAT WERE THE LIKELIHOOD RATIOS OF
accepted in the scientific community as the THE VARIOUS TEST RESULTS?
definition of disease is the yardstick with which o Appraising results: What were the likelihood
the performance of the test is measured ratios of the various test results?
o What to look for: Look for standard diagnostic o Why is it important: Likelihood ratios allow us to
test(s) used to ascertain presence of disease in convert pre-test to post-test probability, even if
the study population. tests have more than just 2 results (positive or
• QUESTION 2: WAS “DEFINITION” OF THE INDEX TEST negative).
AND THE REFERENCE STANDARD INDEPENDENT? o What to look for: If LR’s [likelihood ratios] are not
o Appraising validity: Was “definition” of the index reported, 2 x n [two by n] tables need to be
test and the reference standard independent? reconstructed, and calculations need to be made.
o Why is it important: If any of the criteria for the WHAT ARE THE RESULTS?
index test are part of the criteria for the • Are the likelihood ratios for the test results presented or
reference standard, we tend to observe a falsely data necessary for their calculation included?
high level of agreement between the two. This • Validity of a Clinical Test: Sensitivity (Sn) and Specificity
will result in an overestimate of the accuracy of (Sp)
the index test. • Sn: proportion of people with disease who have a positive
o What to look for: Look for the criteria used to test; positivity in disease
interpret the index test and the reference • Sp: proportion of people free of disease who have a
standard and make sure they do not overlap negative test; negativity in health
• QUESTION 3: WAS “PERFORMANCE” OF THE INDEX TEST SENSITIVITY (Sn) AND SPECIFICITY (Sp)
AND THE REFERENCE STANDARD INDEPENDENT? Reference Standard Total
o Appraising validity: Was “performance” of the Result
index test and the reference standard
independent? Test Result Present Absent
o Why is it important: If in a study setting the result A B
of the index test becomes the indication for the Positive A+B
reference standard, the agreement between the Negative C D C+D
index test and the reference standard will Total A+C B+D A+B+C+D
increase making the index test look better than it
actually is.
Sn = A/A+C
Sp = D/B+D
Accuracy = A+D/A+B+C+D

6 of 10
SENSITIVITY (Sn) AND SPECIFICITY(Sp) (+) LR = (A/A+C) / (B/B+D)
(-) LR = (C/A+C) / (D/B+D)
Reference Standard Total *It’s not important to remember the formulas for calculating
Result likelihood ratio so long as it is understood that likelihood ratio
is a measure of how much the likelihood of a disease changes
Test Result Present Absent given a test result
a=TP b=FP
• LRs indicate by how much a given diagnostic test result will
Positive a+b
raise or lower the pretest probability of the target
Negative c=FN d=TN c+d disorder
Total a+c b+d a+b+c+d • LR = 1: post-test probability is exactly the same as the
pretest probability
• LR > 1: increase the probability that the disease is present
Sn = TP/TP + FN (better at ruling IN disease — confirmatory tests)
Sp = TN/TN + FP • LR < 1: decrease the probability of the disease (better at
TP – True Positive, FP – False Positive
ruling OUT disease — screening tests)
TN – True Negative, FN – False Negative
• *there are rarely tests with BOTH strongly positive and
Utility of a Clinical Test: Predictive Values (PV)
strongly negative likelihood ratios
(+) PV: Proportion of those with a positive test who actually ASSESSING APPLICABILITY
have the disease as measured by the gold standard
• QUESTION 1: ARE THERE BIOLOGIC ISSUES THAT MAY
(-) PV: Proportion of those with a negative test who actually AFFECT ACCURACY OF THE TEST?
are free of the disease as measured by the gold standard o Assessing applicability: Are there biologic issues
PREDICTIVE VALUES that may affect accuracy of the test? (Consider
the influence of sex, comorbidity, race, age, and
Reference Standard Total pathology)
Result o Why is it important: There should be adequate
consideration of variables that may affect test
Test Result Present Absent
performance and utilization in our population of
A B
interest
Positive A+B
o What to look for: Look for characteristics of the
Negative C D C+D study population and assess if the authors
Total A+C B+D A+B+C+D attempted to isolate variables or made statistical
adjustments
(+) PV = A/A+B = Post-test Probability of disease • QUESTION 2: ARE TEHRE SOCIO-ECONOMIC ISSUES THAT
(-) PV = D/C+D = Post-test Probability of no disease MAY AFFECT ACCURACY OF THE TEST?
Prevalence = A+C/A+B+C+D = Pretest Probability of disease o Assessing applicability: Are there socioeconomic
Likelihood Ratio issues that may affect accuracy of the test?
• Expresses the odds that a given level of a diagnostic test o Why is it important: Social, cultural, and
result would be expected in a patient with the target
economic contexts that may potentially affect
disorder or disease
diagnostic test performance have to be taken
• The probability of a test result among patients with the
disease, divided by the probability of that test result into account particularly for interventions that
among patients without the disease. require socio0cultural sensitivity (e.g.
questionnaires and/or provider compliance)
o What to look for: Check if the tests done were
Reference Standard Total
Result locally validated, you may need to do validation
studies in your locale. Look for attempts to
Test Result Present Absent compute costs.
A B
Positive A+B • SCRAPS
Negative C D C+D o S-Sex – consider physiologic, hormonal, or
Total A+C B+D A+B+C+D biochemical differences between sex that may
affect results of the test
o C-comorbidities – consider co-existent
conditions

7 of 10
o R-race – consider racial differences • Therapeutic threshold: probability of disease above which
o A-age – consider age difference we are reasonably sure that we should stop testing and
o P-pathology: consider actual disease under study just get on with treatment
o S-socioeconomic factor — ideal condition • Testing range: between the two thresholds, the clinician
difficult to apply in everyday life is uncertain, so further tests are required
INDIVIDUALIZING THE RESULTS
SUMMARY
• Determine the impact that the test might have on your
patient’s probability of having the disease Articles on diagnostic tests are sometimes considered
• Convert the probability of the disease before the test (pre- difficult reading. By dissecting them one step at a time, reading
test probability) to the probability of the disease after the them becomes simple and straightforward.
test (post-test probability) First, appraise directness. Is the question asked
• Set the Diagnostic and Therapeutic Threshold important to you? Is the test something you are considering?
Is it available? Is it a feasible option? If not, maybe you should
PROBABILITY OF DISEASE GIVEN A TEST RESULT be looking for another article. If it does address an important
question for you, the next step is to appraise validity. Is the
Step How to do it reference standard acceptable? Were the two tests
independent in terms of definition, performance, and
1.Estimate the pre- Based on your history and PE, your interpretation? The answers to these questions will tell you
test probability as a clinical experience will give you a good whether the results are credible. If they’re not, why even
percentage estimate of the probability of disease. bother appraising the article? These two steps can actually
If the case is difficult, ask an expert save you from a lot of futile reading. If you decide to proceed
and interpret the results, look for likelihood ratios, or
2.Convert pre-test Probability information/data that might lead you to compute them
probability to odds Odds = -------------------- yourself. Before actually using these numbers yourself,
100-probability however, decide whether the information is applicable to your
setting: biologic, and socioeconomic factors should be
(Probability expressed as %) considered.
3.Multiply pre-test If everything is in order, directness, validity, results,
odds by the LR of the Post-test odds = pre-test odds x LR and applicability, you now have the information necessary to
test result to get the decide whether the test is something you’d like to request or
post-test odds (Probability expressed as %) if you are using them, interpret the results when you arrive.
Don’t become too engrossed with numbers. Sometimes it’s
4.Convert post-test Odds enough just to understand that likelihood ratios are a measure
odds back to post- Probability = ------------ x 100 of changes in disease probability. Test results with likelihood
test probability in per 1 + Odds ratios greater than 1 increase the disease probability. The
cent higher the likelihood ratio, the closer it can bring you into
(Probability expressed as %) confirming the disease. Test results with likelihood ratio less
than one decrease the disease probability. The lower the
likelihood ratio, the closer it can bring you to ruling out the
disease. Test results with likelihood ratio very close to one
have very little impact on your clinical or pretest estimates of
SETTING THE DIAGNOSTIC AND THERAPEUTIC THRESHOLD
disease probability.
When you have arrived at your post-test probability of
disease, you can make a clinical decision based on your
threshold of management and your patient’s preferences.
These are usually three choices: 1) stop testing and get on with
the treatment of the probable disease 2) stop testing and
reassure the patient the disease probability is low 3) do more
tests before you decide
*sorry bout the speaker, that’s there the entire video lol*
• Diagnostic threshold: the probability of disease below
which we are reasonably sure that we should stop testing
and just reassure the patient

8 of 10
COMMUNITY MEDICINE 5.01
CRITICAL APPRAISAL
Dean M. Sosa & Assoc. | January 18, 2019
Transcriber group: 16B

TABLE 1: Classification Scheme for Bias

Type of bias Description Relevant domain in the “risk of


bias” tool
Selection Systematic differences between baseline Sequence generation
characteristics of the groups that are compared
Allocation concealment
Performance Systematic differences between groups in the Blinding of participants and
care that is provided, or in exposure to factors personnel
other than the interventions of interest
Addressing other potential
threats to validity
Detection Systematic differences between groups in how Blinding of outcome assessors
outcomes are determined
Addressing other potential
threats to validity
Attrition Systematic difference between groups in Incomplete outcome data
withdrawals from a study

TABLE 2: Ways of Expressing Effectiveness

Summary of results Comparison of results between two


Outcome
within each group groups
Dichotomous (e.g. lived or Proportions (e.g. deaths Relative risk reduction, absolute risk
died, blood pressure per 100 patients) reduction, relative risk
controlled or not)
Rate (e.g. deaths per 100 Hazard ratio = rate in treatment/rate in
patients per year control
Continuous (e.g. blood Mean (e.g. mean blood Mean difference = mean in control —
pressure in mmHg, quality pressure) mean in treatment
of life in a scale of 1-10)

1 of 10
TABLE 3: Measure of Effectiveness for Dichotomous Outcome

Weight analogy Measure of effectiveness Interpretation


I lost 20 kg Weight difference I lost 2% of my risk Risk difference ARR (%)
(absolute weight = Wc – Wt Absolute Risk = Rc – Rt
>0%: Tx beneficial
reduction) Reduction (ARR) 0%: no effect of Tx
<0%: Tx harmful

I lost 25% of my Weight difference I lost 25% of my risk Risk difference / RRR (%)
weight (relative Original = (Wc-Wt) Relative Risk Reduction Original Risk
>0%: Tx beneficial
risk reduction) / Wc (RRR) 0%: no effect of Tx
(Rc – Rt) / Rc <0%: Tx harmful

I now weigh 75% New weight / My risk is now 0.75 of New risk / RR (decimal)
of what I weigh original weight what it was Original risk
<1: Tx beneficial
(relative risk) Relative Risk (RR) 1: no effect of Tx
Rt / Rc >1: Tx harmful

*Xc = Control; Xt = Treatment

Example
An RCT showed death rates of 4% in the treatment group and 5% in the placebo group. What would be
the RRR, ARR and RR?

RRR = (Rc – Rt) / Rc


(5 – 4) / 5 = 0.2
2. x 100% = 20% (answers should be in percentage/rate for RRR and ARR)

ARR = Rc – Rt
5% - 4% = 1%

RR = Rt / Rc
4 / 5 = 0.8

Interpretation
RRR: 20% > 0% =>Treatment is beneficial (it means that there is a 20% reduction of the risk of death in the
treatment group)
ARR: 1% > 0% =>Treatment is beneficial
RR: 0.8 < 1 => Treatment is beneficial

--END—
“Love is like an hourglass, with the heart filling up as the brain empties. – Jules Renard, French author”

Happy February fam! #ONE2021

REFERENCES:
• Recordings
• PPT photos
• Lecture notes
• Video lectures

2 of 10