EBM Presentation

EVIDENCE BASED MEDICINE
A new approach to clinical care and research
Developed and presented by

Judy Tarselli, RN
Dubai, UAE
Karachi, Pakistan
October 2003
Organized by NKF cyberNephrology

University of Alberta, Canada
www.cyberNephrology.org
Special thanks to our sponsors Janssen-Cilag

PROGRAM OUTLINE
1. Definition of EBM
2. Basic Steps
3. Trials, Studies and Reports
4. Pros, Cons and Limitations
5. EBM in Developing Countries
6. EBM Library
7. Advanced EBM
But first, a test…
WHAT IS THE BASIS OF YOUR
MEDICAL PRACTICE?
(Check all that apply)
A. Training, clinical experience and consultation

with other professionals
B. Convincing evidence (non-experimental) from

articles, case reports, product literature, etc.
C. Preferences of the patient
D. Active search of Randomized Controlled Trials,

Systematic Reviews, Meta-Analysis Reports
MEDICAL PRACTICE?
EXCELLLENT!
A. Training, clinical experience and consultation
with other professionals
B. Convincing evidence (non-experimental) from

articles, case reports, product literature, etc.

Systematic Reviews, Meta-Analysis Reports
BUT… Past knowledge and practice
might be outdated or inadequate
Graduate Medical School Practiced Physician

MEDICAL PRACTICE?
FANTASTIC!
A. Training, clinical experience and consultation with

other professionals
B. Convincing evidence (non-experimental) from articles,

case reports, product literature, etc.

Systematic Reviews, Meta-Analysis reports
BUT… This evidence may be biased, outdated,
incorrect, or not applicable to your patient
JOURNALS (1987 to present)
ARTICLES ADVERTISEMENTS
MEDICAL PRACTICE?
WONDERFUL!
A. Training, clinical experience and consultation with other
professionals

Mutual Respect +
Shared Goals =
C. Preferences of the patient Better Cooperation
and Compliance
The patient should be involved in
all important decisions
But this is NOT always an easy task!
And conflicts WILL occur!

No salt? I WON’T take that medicine…
Lose weight? The side effects are But doctor, I DO want
Forget it! INTOLERABLE! to have children!
Just give me a pill!
And conflicts WILL occur!

No salt? I WON’T take that medicine…
Lose weight? The side effects are But doctor, I DO want
Forget it! INTOLERABLE! to have children!
Just give me a pill!
Education about current alternatives and risks is often

needed… for both the Patient and the Doctor!
Wow… Yes, I’d like to try that I’ll discuss those risks
I never knew that high new medication! with my husband.
blood pressure could
be so dangerous at my
age!
Education about current alternatives and risks is often

needed… for both the Patient and the Doctor!
An important rule in Evidence Based Medicine…
It STARTS with the patient and ENDS with the patient.
The patient’s preferences MUST be considered!

WHAT IS THE BASIS OF YOUR MEDICAL
PRACTICE?
WOW!!! SUPERB!!!
professionals


In the practice of Evidence Based Medicine,
it is the physician’s duty to find the best and
most current information and apply it
judiciously for the benefit of the patient.
But… A practice based exclusively on science and math
is effective only if your patients are robots or clones!
Don’t forget to allow for individual human differences

and personal preferences!
MEDICAL PRACTICE?
If you checked all 4 items…

professionals


CONGRATULATIONS!
You are practicing
EVIDENCE BASED
MEDICINE!
professionals
B. Convincing evidence (non-experimental) from articles, case reports,

product literature, etc.
D. Active search of Randomized Controlled Trials, Systematic

Reviews, Meta-Analysis reports
1. Definition of EBM
2. Basic Steps
3. Trials, Studies and Reports
4. Pros, Cons and Limitations
5. EBM in Developing Countries
6. EBM Library
7. Advanced EBM
“What is Evidence Based Medicine?”
“And where did it come from?”
A BRIEF HISTORY
1980’s: McMasters University in Ontario, Canada
Dr. David Sackett and colleagues proposed Evidence
Based Medicine (EBM) as a new way of teaching, learning
and practicing medicine.
Dr. Sackett defines EBM as:

“…The conscientious, explicit, and judicious use
of current best evidence in making decisions
about the care of individual patients.”
Evidence Based Medicine
It is a change in the way physicians practice medicine, teach and
learn, and handle research.
Clinical practice: Based on the best current evidence

(not necessarily on how it’s always been done)
Patient Care: Compassionate, patient-oriented

(less authoritarian)
Learning & Teaching: Problem-based, problem-solving

more investigative, less know-it-all-by-yesterday
Research: More stringent approach, better proof criteria

(more demanding of proof, less room for error)
THREE MAJOR
COMPONENTS of
EBM PATIENT
Question
or
Problem
PHYSICIAN
INFORMATION
THE ADDED DETAILS
PATIENT
Values, Concerns Preferences,
Expectations
Life predicament
EBM
PHYSICIAN INFORMATION
Training & Experience Clinically relevant
Current Expertise Proven by research
Continued learning Best up-to-date
Demand for proof evidence
OPTIONAL
COMPONENTS
PATIENT
TO BE ADDED BY
THE PHYSICIAN Values, Preferences
Concerns, Expectations
Life predicament
HUMILITY CHARITY
Non-authoritarian EBM is not a
practice required practice
(yet)
EBM
PHYSICIAN
Training INFORMATION
Expertise ENTHUSIASM Clinically relevant
Continued Learning Challenge, Variety, Proven by research
Demand for proof Change Current, up to date
“Isn’t this the way “Aren’t these just the
we have always same old ingredients
practiced medicine?” tossed into a new
recipe?”
When am I supposed to find

the time to do that?
The basic steps of EBM
THE FIVE BASIC STEPS OF EBM
1. Clinical Question
Patient-focused, problem-oriented
2. Find Best Evidence
Literary Search
3. Critical Appraisal
Evaluate evidence for quality and usefulness
4. Apply the Evidence
Implement useful findings in clinical practice
5. Evaluate
The information, intervention, and EBM process
Literary Search
5. Evaluate
Literary Search
5. Evaluate
Literary Search
5. Evaluate
Literary Search
5. Evaluate
Literary Search
5. Evaluate
The Clinical Question
The FIRST step

The HARDEST step
The MOST IMPORTANT step!
FACT: We all have informational needs!
That is not a problem!

Problems arise
• if we fail to recognize those needs
• if we fail to bridge the information gap

• if we fail to ask the right questions
Asking good questions Hmmm… Is he
about to give me a
is a skill to be learned. BONUS?
Or is he about
to FIRE me?
Lee, exactly how
much time did you
spend on that big
project?
It will make life

easier for you...
And also for others

around you!
Lee, can you give me an Oh sure! I’ll have it
accounting of the extra time on your desk by
you spent on that project so tomorrow!
that I can charge it back to
the client?
A GOOD QUESTION…
• Is focused and relevant
• Provides clear
communication
• Clarifies your goal or need
• Will reduce the amount of
time needed to obtain the
answer
WHEN PRACTICING EBM, ACTUAL CASE SCENARIO
a good question must also: Large cauc male, age 40
2mo ago: Presented with classic
• Be specific
nephrotic syndrome, significant
Identify the problem, clarifiy
symptoms. Bx showed IgAN. Cr
the clinical issue
1.4, incr to 2 range, now 1.6
• Be answerable Tried prednisone 60mg qd -

through the literature tolerated poorly w/tremors and
depression.
• Contain multiple aspects Needs new regimen, but others
(patient, options, are aimed more at nephritic IgA
comparisons, etc) rather than nephrotic syndrome.
Suggestions?
It should NOT involve a
question of Personal Preference
or Local Concern.
THE EVIDENCE BASED RESPONSE
Posted on Nephrol 4/13/03
Respondant recommends cyclophosphamide and
prednisolone (assuming secondary causes excluded) - a
combination that allows for lower dose prednisolone…
“In the study below, proteinuria and renal

function improved on this combination:
Ballardie FW, Roberts IS. Controlled prospective

trial of prednisolone and cytotoxics in progressive
IgAN. J Am Soc Nephrol 2002 Jan….”
“I have patients on this regime who have

benefitted.”
Regards,
Dr. Paulose P. Thomas
Nephrologist - Belhoul Apollo Hospital, Dubai, UAE
BACKGROUND and FOREGROUND QUESTIONS
(all part of EBM)
FOREGROUND QUESTIONS
NEW POSSIBILITIES
INDEFINITE ANSWERS
“Where do we want to go,
and how else might
we get there?”
“Where are we now?

And which way are we headed?”
BASIC & CONCRETE
BACKGROUND QUESTIONS
STUDENT GRAD EXPERT

BACKGROUND QUESTIONS
BASIC & CONCRETE
1. QUESTION
• Who, What, Where, When, Why, How
2. VERB
• is, causes, does, treats, reduces, cures, prevents, affects
3. GENERAL KNOWLEDGE ABOUT DISORDER

clinical manifestations of disease, patient findings, differential
diagnosis, etiology, patient experience, comorbid condition,
screening and diagnostic tests, prognosis, therapy, risk factors,
etc.
STUDENT GRAD EXPERT
FOREGROUND QUESTIONS
NEW POSSIBILITIES
INDEFINITE ANSWERS
PT AND/OR PROBLEM Differential dx, Unusual presentation, uncertain

etiology, pt’s prior experience, comorbid conditions
INTERVENTION Exposure, test. Prognostic factor,

treatment, pt perception, etc.
COMPARISON INTERVENTION
OUTCOMES
STUDENT GRAD EXPERT

EBM QUESTION: Should include multiple factors
(Examples)
P PATIENT type of patient or population

Ex: 47 yr male w/DM2 and cellulitis toe, 25 yr female w/DVT and chest pain
E EXPOSURE environmental, personal, biological

Ex: TB, tobacco, drug, diet, pregnancy or menopause, MRSA, allergy
I INTERVENTION clinical intervention

Ex: medication, procedure, test, surgery, radiation, drug, vaccine
C COMPARISON compare alternative treatment

Ex: other prior, new or existing therapy
O OUTCOME clinical outcome of interest

Ex: Reduced death rate in 5 yrs, decreased infections, fewer hospitalizations
FRAMING THE QUESTION (Example: PICO)
ELEMENT PROMPTS THE QUESTION:
Patient How would I describe a group of patients similar to mine?

Intervention What main action am I considering?
Comparison What is/are the other options?
Outcome What do I (or the patient) want to happen (or not happen)?
Example:
P: In kids under age 12 with poorly controlled asthma on metered

dose inhaled steroids…
I: would the addition of salmetrol to the current therapy
C: compared to increasing the dose of current steroid
O: lead to better control of symptoms without increasing side effects?
CATEGORY OF QUESTION
MAJOR CATEGORIES
1. Diagnosis
2. Prognosis
3. Therapy/ Treatment PICO
4. Harm (iatrogenic, other) PEO
MISCELLANEOUS
• Quality of care
• Health economics
• Office Management
• Etc.
THE PATIENT’S QUESTIONS
Must be considered!
Often QUALITATIVE (not based on measureable outcomes)

Feelings, ideas, experiences, preferences, concerns, fears, beliefs,
ethnicity
Usually based on LIMITED BACKGROUND

Perception of problem
Self-diagnosis
Treatment wanted or needed
Alternatives (read, heard, considered, tried)
What is the patient hoping to avoid?
What benefits does the patient want or need most?
Etc.
QUANTITATIVE vs QUALITATIVE QUESTIONS
QUANTITATIVE: “Solid Evidence”

• Measurable answer or response
• Necessary for scientific study
• Necessary for the practice of EBM
QUALITATIVE: “Quality of Life”

• “Fuzzy” data - Impact on daily life, work, family, etc.
• May be very important and influential to decisions –
especially for the patient
• Creates added challenge or twist to practice of EBM
QUALY: QUALITY ADJUSTED LIFE YEAR
CHRONIC MEDIOCRITY
PERFECT HEALTH
10
10 9
9 8
8 7
7 6
6 5
5
4 4
3 3
2 2
1 1
2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10
DETERIORATING HEALTH 5-yrTREATMENT COMPARISON

9
8
7 10
6
5 8
Chemo&Rad
4 6
3 4 Radiation only
2
2 Chemo only
1
0 0
2 3 4 5 6 7 8 9 10 1 2 3 4 5
Literary Search
5. Evaluate
Find the Best Evidence
“The Literary Search”
HINT: If your desk looks like this, it’s probably the

LAST place you should start looking!
Find the Best Evidence
“The Literary Search”
The BEST EVIDENCE is:
External - from outside resources (researchers, experts)
Current – not out of date, most recent
High Quality - accurate, precise, effective, safe
Patient focused - applicable and appropriate for your individual

patient
FIVE STEPS TO FINDING THE BEST EVIDENCE
1. IDENTIFY NEEDS: What type of information is needed?

2. IDENTIFY RESOURCES: Types, Availability, Timeliness,Costs?
3. SEARCH & RETRIEVE: Use efficient strategies
4. REVIEW : Check quality and usefulness of info
5. INTERPRET: Help patient understand info, application
WHAT TYPE OF INFORMATION IS NEEDED?
WHAT CATEGORY IS THE QUESTION?
• Diagnosis
• Prognosis
• Therapy
• Harm
WHAT STUDY DESIGN FITS IT BEST?
There are MANY study designs!
EXPERIMENTAL TRIALS
(Answers questions of diagnosis or treatment)
Randomized Controlled Trials (RCTs)
Controlled studies
Blinded vs Open
ETC.
OBSERVATIONAL STUDIES
Descriptive reports
Retrospective studies
Cohort studies
Case Control
ETC.
EXAMPLE
Randomized Controlled Trials (RCT)

“Gold Standard” of research
Ideal experimental design - Best design for TREATMENT questions
Must identify objective of treatment

(Ex: cure, prevent complication, palliation, reassurance)
Still not always the right intervention for individual patient at that particular time and
place
What type of evidence best addresses the question, problem or issue?
CLINICAL PRACTICE APPROPRIATE DESIGN FOR CLINICAL RESEARCH
Diagnosis, Dx testing Cross-sectional study – not randomized trial
Prognosis Follow-up studies of patients evaluated at same early point of illness
Therapy, treatment RCT or Systematic review of multiple RCTs must be used

Avoid non-experimental approaches to avoid false conclusions about efficacy
Exceptions:
When treatment may be successful in an otherwise fatal condition
When no studies are available (rare conditions, new treatments, etc.)
Harm RCT, Cohort, Case-control
OTHER INFORMATIONAL
Explore hypothesis Qualitative research
History-taking Case control study
Individual trial & error n of 1 trial
Following clinical course Cohort study
Recordkeeping Systematic registry-based (computer supported) research
Quality of Care research Individual peer review, Process Evaluation
MISCELLANEOUS Basic Science, Genetics, Immunology, etc.

WHAT FORM OF INFORMATION?
Case report
Controlled Trial
Systematic review
Meta-analysis
Clinical guidelines
etc.
LITERARY SEARCH: NEXT STEP
IDENTIFY YOUR RESOURCES
Colleagues
Consultation, Discussion
(Caution: Response may be an outdated “This is what we do”)
Paper resources
books, reports, journals
Electronic databases
Health Literature Services

specialized librarians, staff
Review services, Abstract Services, etc.

SEARCH AND RETREIVE THE BEST EVIDENCE
Learn and Practice various SEARCH STRATEGIES:

• To find useful information quickly
• To eliminate irrelevant, inappropriate or weak information
Try to develop the habit of learning as you go;
Not just in lengthy formal sessions!
LITERARY SEARCH STRATEGY
ASK FOR HELP!
SPECIALIZED PERSONNEL
• track down information, textbooks, articles,
guidelines
• may provide electronic search support or training
EXAMPLES
• Medical Librarians
• Medical Informatics Specialists
• Specially trained staff member
LITERARY RESOURCES
• TEXTBOOKS (caution – most obsolete!)

• Traditional
• Evidence Based
• JOURNALS (may be outdated)
• REVIEW ARTICLES (summaries, abstracts)
• SYSTEMATIC REVIEWS (prepared in systematic, rigorous

manner) Ex: Cochrane Collection
• META-ANALYSIS
• CLINICAL PRACTICE GUIDELINES

Summarized and easily digestible information
ELECTRONIC RESOURCES, DATABASES, INTERNET
Bibliographic Database
Example: Medline, PubMed
Medical Information Services: Medscape, HDCN
Review Services
Subjective
Systematic Reviews
Meta-analysis
Examples:
• Cochrane,
• Best Evidence,
• Up to Date
MORE GREAT INTERNET RESOURCES
Websites
cyberNephrology, National Kidney Foundation. NIDDK,
American Heart Association, American Cancer Society.
National Institutes of Health, etc
Listserve Discussion Groups

CyberNephrology, C-span, etc.
Specialty Electronic Databases

Psyclit
CancerLit
CINAHL
(allied health and nursing journals)
Etc
OTHER RESOURCES
Tapes
Videos
CD-ROMs
Specialty seminars
Product information and comparisons

A closer look at some Internet Resources…
MEDLINE
WHAT IS IT?
Searchable database of medical information compiled by National Library of

Medicine in US 1966-present
Catalogs articles from approx 4000 world journals (of estimated 12-15k total)
SEARCH METHODS
Any word or words (title, abstract, content, author name, institution, etc.)
Medical Subject Heading (MeSH) terms
A restricted thesaurus of medical titles
Articles categorized by most specific possible MeSH heading

COST: FREE!
Or may subscribe to companies with specialized search strategies:
• Ovid Technologies (ovid)
• Silver Platter Information (WinSPIRS)
BENEFITS
Free
Vast database
LIMITATIONS
Not all articles are indexed on Medline (only 1/3 of approx 10 million!)
Much material listed and described on Medline can only be accessed through
journal article
MEDLINE: ELECTRONIC SEARCH STRATEGIES
Search through “Clinical Queries” service of PubMed

http://www.ncbi.nlm.nih.gov/clinical.html
Medical Subject Headings (MeSH)
Search filters
Search by a text word can supplement a MeSH search
Boolean search: “and”, “not”, etc.
To increase sensitivity
• use “explode” command
• avoid using subheadings
Online Tutorial is available!

COCHRANE LIBRARY
Cochrane Database of Systematic Reviews

-systematically compiled reviews of intervention
Cochrane Controlled Trials Register

-citations of controlled trials identified anywhere in the world
Cochrane Review Methodology Database

-methodological papers relating to systematic reviews
Etc.
BEST EVIDENCE
Electronic version of two publications:
• Evidence Based Medicine
• American College of Physicians Journal Club
Covers broad topics of information

Literary Search
5. Evaluate
CRITICAL APPRAISAL
Interpreting the evidence

• How to read a paper
• How to do the math
CRITICAL APPRAISAL
IMPORTANT!
You do NOT have to become a researcher,
epidemiologist, or statistician to practice EBM.
Focus on how to USE research reports –

not on how to generate them!
CRITICAL APPRAISAL
HOWEVER…
You must have a solid understanding of
basic research principles and
study designs in order to understand
and interpret the evidence!

TYPES OF STUDIES AND REPORTS
Randomized Controlled Trial - “The Gold Standard”

Systematic review
Meta-analysis
Retroactive vs Prospective
Incidence
Prevalence
Case Control
Cohort (Follow-up)
Cross-sectional
Ecologic
Longitudinal
Experimental
Blinded vs Open
Qualitative Screening
DETOUR
BASIC RESEARCH PRINCIPLES
STUDY DESIGNS
THE TIME FACTOR
When was the study done?
What was its duration?
In what time direction is it headed?
RETROSPECTIVE PROSPECTIVE
THE TIME FACTOR
When was the study done?

What year?
What technology? (ie: test, drug, equipment, procedure)
Any associated social factor or historical event?
THE TIME FACTOR
What was the Study Duration?
Was it an appropriate length of time for the

intended goal?
Limited time study or ongoing?
Was study completed? Stopped early?
In what direction is it headed?
“LOOKING BACK” “LOOKING FORWARD”

Historical Review or Future Results
Investigation The Great Unknown
PAST PRESENT FUTURE

In what direction is it headed?
PRO
•Lower risk of bias
PRO
•May provide good
CON:
direction for future study
May get faulty results based
“Hind Sight is 20/20”
on incomplete data or
CON: insignificant subgroups
•Prone to Bias
(Example of Error: Untreated
•A“Fishing Expedition” for hypertension unlikely to cause
positive results cardiac event in child, so treatment
PRESENT is unnecessary below age 18yrs)
“Was there a similar comparison group?”
No comparison group
All subjects receive Experimental Intervention
Experimental
Intervention
UNCONTROLLED STUDIES
NO EVENT
Experimental
Intervention
OUTCOME
EVENT
“Trial and Error?” or “Before & After?”

UNCONTROLLED STUDIES
Generally NOT accepted:
“Traditional Study Method”
Potentially Dangerous and Flawed
May produce strong results
Prone to BIAS!
“Trial and Error” “Before & After”
PROBLEMS BENEFITS
POSITIVE OUTCOME MAY BE DUE Can answer some questions
TO: about:
•Other factors
•likelihood of response
•Natural course of disease (some
get better, some don’t!) •adverse effect, etc.
•Spontaneous change of health
•Placebo Effect VERY PATIENT-SPECIFIC!
•Hawthorne Effect
NEGATIVE OUTCOME MAY BE ONLY OPTION

May be due to study treatment. Rare conditions
Could be disastrous! Previously unknown conditions
UNCONTROLLED TRIALS: “TRIAL AND ERROR”
GOOD!
Example#1 Resistant to
Cowpox and
SMALLPOX Smallpox
VACCINATION
James Phipps, (NO DISEASE
age 8 years OUTCOME)
SMALLPOX VACCINE
1. 1796: Edward Jenner inoculates 8yr-old James Phipps with cowpox virus
from a milkmaid’s hands.
Child develops illness, recovers.
2. Two weeks later, inoculates same child with smallpox virus.
Child survives, no illness.
(Centuries later, smallpox eradicated!)
n=1
UNCONTROLLED TRIALS: “TRIAL AND ERROR” n=1
Example #2 NO
OUTCOME
Drinks culture of
H.pylori
Dr. Marshall SEVERE
Microbiologist GASTRITIS
HELICOBACTER PYLORI - GASTRIC ULCERS

1982: Australian microbiologist Barry J. Marshall presents evidence showing a
possible infectious cause for gastric ulcers. Suggests they may be treatable with
antibiotics.
Findings are met with disinterest and disbelief by medical community. Lacks
support for further study.
5 years later: Prepares a broth of live organisms isolated from a gastric ulcer
patient and drinks it. Becomes violently ill, develops severe acute gastritis.
1990’s Antibiotics are used routinely to cure some gastric ulcers!
UNCONTROLLED TRIAL
RECOVERED
Experimental
Intervention
May represent the ONLY DIED

treatment option for a new or rare
disease
Present
FUTURE
STRONGLY PREFERRED! Reduces BIAS. Provides stronger results.
Experimental
Intervention
Control
Group
CONTROLLED STUDY
Only the TEST group receives the Experimental Intervention
ExperimentalI
ntervention
Control group may receive…
Nothing IMPORTANT
Placebo All other differences
should be minimized or
Observation only
eliminated to reduce
Other potential BIAS
Gold Standard
Treatment
RANDOMIZED CONTROLLED TRIAL (RCT)
“The Gold
Standard”
Experimental
Intervention
Control
Group
THE FIRST RANDOMIZED CONTROLLED TRIAL
By Sir Austin Bradford Hill
Streptomycin
(n=50)
(BLINDED)
Bedrest
(n=50)
1944 TUBERCULOSIS TREATMENT: Streptomycin vs Bedrest

OPEN vs BLINDED STUDIES
Experimental
Intervention
OPEN
Control
Group
OPEN vs BLINDED STUDIES
BLINDED
TRIAL
BLINDED
BLINDED
TRIAL
BLINDING
SINGLE BLINDED:
Pt unaware of what group s/he is in
DOUBLE BLINDED:
Pt and MD unaware
OPEN LABEL:
Everyone is aware
RANDOMIZED vs NON-RANDOMIZED TRIALS
Experimental
Intervention
How is this
group divided?
Control
Group
NON-RANDOMIZED
Experimental
Intervention
Assigned to
groups, usually
by the
researcher
Control
Group
Potential for RESEARCHER BIAS!

RANDOMIZED
Experimental
Intervention
Random method of
assignment used
Control
Group
Maximizes “sameness,” Eliminates BIAS!

(EXPERIMENTAL TRIAL)
Experimental
Intervention
“The Gold Standard”
Control
Group
Present
FUTURE
One patient, series of tests n=1
TRIAL SERIES FOR INDIVIDUAL PATIENT
GOOD GOOD
Experimental Experimental
Intervention Intervention
Trial of Medicine Trial of Medicine
1 2
Or placebo Or placebo
NO CHANGE NO CHANGE
OR BAD OR BAD
Why a TRIAL SERIES for one patient?
BENEFIT
Produces data most applicable to the individual patient
EXAMPLES:
Trial of different medications and/or placebo for child reported
to have ADHD symptoms that are not clinically apparent
Trial of different analgesics for patient with chronic pain from
a combination of diseases not previously studied
PATIENT PHYSICIAN
•Must be blinded •May need to be blinded (enlist help
•Must keep diary or complete of pharmacist!)
questionnaire •Must treat patient as usual in all
other respects
CROSSOVER TRIALS
ONE GROUP, MULTIPLE TESTS
(Best if participants are blinded)
A A
B B
ASSESS ASSESS
OUTCOMES #1 OUTCOMES #2
COMPARE OUTCOMES
CROSSOVER TRIALS PROS & CONS
Fewer participants needed than a RCT!
A A
B B
ASSESS ASSESS
All are in experimental group Lower costs

CROSSOVER TRIALS PROS & CONS
MUST HAVE SHORT CARRYOVER EFFECT
MUST HAVE SHORT WASHOUT EFFECT
A A
B B
ASSESS ASSESS
(OR WAIT A SUITABLY LONG WASHOUT TIME!)

CASE CONTROL
(“A LOOK BACK”)
RISK FACTOR?
(PAST) Present
CASE CONTROL
(“A LOOK BACK”)
HEALTHY
NEVER
SMOKED
RISK FACTOR
LUNG CANCER
SMOKER
(PAST) Present
CASE CONTROL
(“A LOOK BACK”)
NON-DIABETIC
NORMAL
WEIGHT
RISK FACTOR
DM TYPE II
OBESITY
Present
COHORT“FOLLOWUP DESIGN”
IS RISK
FACTOR
PRESENT?
(Exclude those
with outcome
already!) Future Outcome
TO INVESTIGATE ETIOLOGY
COHORT OR HYPOTHETICAL CAUSE
OF DISEASE/OUTCOME
IS RISK
FACTOR
PRESENT?
“FOLLOWUP DESIGN”
Present Future Outcome
COHORT EXAMPLE
RISK FACTOR
Hgb <9
DIALYSIS PATIENTS
Measures future outcome for

Present dialysis pts w/o treatment of anemia
CROSS SECTIONAL DESIGN
? Cause ? Risk factors
A look back
CROSS SECTIONAL DESIGN
OTHER CAUSES
RISK = SLEEP PRONE
INFANT
SIDS DEATHS DEATHS
Problems of looking back
NON-SIMILAR
CONDITIONS
Social VARIATION IN
Personal TREATMENT
Comorbid conditions OR METHOD
Other treatments
Etc.
CURRENT
GROUP OF
NO PATIENTS
Not usually
CONTROL
accepted by
OVER
medical journals
CONTROL
(accepted in
GROUP
popular press,
not reviewed)
RANDOMIZED & CONTROLLED TRIAL (RCT)
Experimental
Intervention
MAY BE
BLINDED
Control
Group
PROSPECTIVE
START WITH YOUR TARGET POPULATION
Set CRITERIA for

INCLUSION / EXCLUSION
This will determine:

ELIGIBILITY at the start
VALIDITY at the end
ELIMINATE THOSE WHO DO NOT MEET THE CRITERIA
NEXT: GATHER A SAMPLE GROUP
THE SAMPLE GROUP WILL:
•Represent the target population

•Meet the criteria for inclusion / exclusion
SIDE NOTES…
Study should be approved by an
Ethics Committee
Informed consent should be
obtained from study participants
SAMPLE GROUP MAY BE SUBDIVIDED FURTHER
STRATIFICATION
Divide into subgroups based on
important similar characteristics
RANDOMIZATION
Divide into sub-groups based on
unknown confounders
STRATIFICATION
“important similar characteristics”
Examples:
• Male or Female
• Age
• Stage of illness
• Prior illness or treatment
• Hospital vs Office groups
• Comorbid condition
• Etc.
EXAMPLE OF
STRATIFICATION
FEMALE
MALE
RANDOMIZATION
“unknown confounders”
Examples:
• Postal code
• Month of birth
• Random number
• Etc.
EXAMPLE OF
RANDOMIZATION
DX IN JANUARY-JUNE
DX IN JULY-DECEMBER
Next… Divide your sample group(s) into STUDY GROUPS
Experimental
Intervention
“Test Group”
Control
Group “Baseline Group”
Next… Divide your sample group(s) into STUDY GROUPS
“Test Group”
Experimental
Receives Experimental
“Baseline Group”
• Nothing
• Observation
Control • “Same” miscellaneous
Group intervention (non-
experimental)
• Placebo
• “Gold Standard” therapy -
especially if unethical to do
otherwise!
ASSIGN PATIENTS TO STUDY GROUPS
Experimental
Intervention
Use caution against bias!
Control
Group
Sample Group Study Groups

STUDY INVESTIGATOR
 usually assigns
patients to study
groups.
Experimental
Intervention  usually has a
personal preference
for the treatment or
patient
 might unconsciously
“work harder” to
Control make the study work
with non-preferred
Group candidates
= POTENTIAL FOR
BIAS
Experimental
Intervention
Use random
separation
and assignment!
Control
Group
Experimental
Intervention
Control
Group
Experimental
Intervention
Control
Group
Present
Proceed with study
FUTURE
Experimental
Intervention
EXPERIMENTAL EVENT
RATE (EER)
Control
Group CONTROL
EVENT RATE
(CER)
“The Gold
Standard”
Experimental
Intervention
EXPERIMENTAL EVENT
RATE (EER)
Control
Group CONTROL
EVENT RATE
(CER)
Present
FUTURE
Disadvantages of RCT
Expensive
large # pts needed
Prolonged recruitment and follow-up time needed
Funding difficult to obtain except w/support of
pharmaceutical companies (problematic!)
RETURN FROM DETOUR
Literary Search
5. Evaluate
CRITICAL APPRAISAL

EVALUATE WRITTEN EVIDENCE FOR…
Quality
Usefulness
…BY ASSESSING
Validity
Reliability
Relevance
Clinical importance
Critical Appraisal: VALIDITY
What was the original purpose of the study?
When was it prepared?
By whom?
• credentials?
• affiliations?
Sample population
Did the subjects represent an appropriate test group?
How were they selected?
Were controls used?
Were groups similar for important prognostic characteristics?
VALIDITY
How was the information gathered and processed?
Were groups treated equally except for trial therapy?
Were appropriate criteria used to measure results?
Were criteria applied rigorously?
Was the study completed?

(Or ended early for a specified reason?)
Did the study account for all test subjects?

Including subjects lost to follow-up?
Were ALL pts analyzed in their allocated groups?
(ie: INTENTION TO TREAT - not “completed treatment” analysis)
VALIDITY
Information
Does the paper support its claims?
Is the information accurately presented?
Does it represent the truth?
Results
Are the results believable?
To what degree of confidence?
Ex: Disagreement is not uncommon on angiograms, EKGs, radiographs, pathology, PAP
tests, etc.
VALIDITY
Comprehensiveness
Size: Was it large enough to yield credible results?
Thoroughness: Was it complete enough?
Duration: Was it long enough?
CRITICAL APPRAISAL: RELIABILITY
Do we trust the information and results?
1. APPROPRIATE TYPE OF STUDY
2. REPRODUCEABILITY
3. INTERPRETATION OF RESULTS
4. BIAS
RELIABILITY
APPROPRIATE TYPE OF STUDY
Was the type of study design used proper for the question?
Example:
RCT would be choice for questions on TREATMENT
RELIABILITY
Are the Measurements and Results reproducibile?
Different determinations may be caused by:
• Variation in measurement methods
• Different interpretation of results
• Lack of agreement
Example:
BP checks on same patient may vary. Are differences result of pt factor, examiner
factor, treatment factor, normal variance
Would the same results be obtained if patient is re-measured?

(with identical procedure)
• at another time?
• by another person?
Were any similar studies done?

• Was the information comparable?
• Did the results agree?
RELIABILITY
INTERPRETATION OF RESULTS
Is there consistency among researchers?
Different determinations may be caused by:

• Variation in measurement methods
• Different interpretation of results
• Lack of agreement
EXAMPLE:
BANFF CONFERENCE - Setting standards in Transplant Pathology
established by Kim Solez, MD
Were any new questions or controversies raised by the study?

RELIABILITY
IS THERE ANY EVIDENCE OF BIAS?

A dangerous pitfall!
• PATIENTS
• RESEARCHERS
PATIENT BIAS
Social Desirability Bias
Patient responds in the way Patient denies unhealthy

they perceive as correct behavior, gets misclassified
• to support MD Ex: Smoker vs Non-smoker

• to support a preconceived
notion (ie: foods vs ADD)
PATIENT BIAS
Hawthorne Effect
People act differently when Authors must take steps to

they know they are being reduce this bias by treating
watched. all equally!
Ex: Follow more careful diet Ex: Weigh all patients with same
when regular weigh-ins are frequency, even for group not on
scheduled special diet
RESEARCHER BIAS
Who sponsored or funded the study?
Personal gain or loss from results?

Affiliates
Special interests
Conflict of interest
Biased goal?
To satisfy editors and reviewers… rather than solve
real life clinical problems
RESEARCHER BIAS
Criteria bias?
Risk-avoidance by researchers
(will focus energy on topics that produce positive results)
Bias toward patients?

Sample selection criteria used (inclusive, exclusive)
Assignment to test group or control - Random? Blind?
RESEARCHER BIAS
Data collection methods used

• applied similarly to all subjects, including controls?
• starting point – prospective/retrospective, stage of patient?
• Was assessment blind?
Data analysis
• Were all potential subjects included in denominator or otherwise
accounted?
• Were they evaluated in originally designated group?
(INTENTION TO TREAT)
REDUCING OR ELIMINATING BIAS AND ERROR
CONDUCT BLIND STUDIES

• Single
• Double-blinded
USE INDEPENDENT OBSERVERS

• When doctor and/or patient can not be blinded, blinded IO measures outcome
• IO may even be unaware of study hypothesis
USE MULTIPLE OBSERVERS

Ex: Send subject slides to multiple pathologists for interpretation
ESTABLISH CLEAR STANDARDS

• Exact methods to use to reduce variation in technique among researchers
• Clear wording on surveys, etc
VALIDATING INSTRUMENTS
• Repeat screening to check for correct answers on surveys
• More frequent evaluations or surveys prevent guesstimates common to less
frequent evaluation
NEXT STEP IN CRITICAL APPRAISAL:
RELEVANCE
QUESTION: Is the report applicable to our…
Problem?
“Does it address the questions raised?”
Patient(s)?
“Will my patient respond like those in the study?”
Practice?
“Can it be done within my practice or circle?”
ARE THE STUDY PATIENTS
• Comparable within the study? (similar traits, age,

socioeconomic group, stage of illness, treatment, etc.)
• Comparable to your patient?
ARE THE STUDY PROFESSIONALS
• Comparable to you?
(general/specialist, primary care/teaching hospital, etc.)

NEXT STEP in CRITICAL APPRAISAL
CLINICAL IMPORTANCE
Information can be true and interesting in theory,

yet useless in clinical practice!
1. Is the information clinically important?
2. If yes, how important is it?
• study design - See: Hierarchy of Evidence
• weight of results
HEIERARCHY OF EVIDENCE
(value of study design to maximize wt, minimize bias)
1. Systematic Review of all relevant RCTs
2. At least one properly designed RCT
3. Trials and case studies
4. Well-designed Controlled Trial without Randomization
5. Well designed Cohort or Case Control Studies, preferably from >1
centre or group
6. Multiple Time series with or without intervention
7. (Exception: Dramatic results in uncontrolled trials, such as
introduction of PCN in the 1940s)
8. Opinions of respected authorities, based on
9. Clinical expertise
10. Descriptive studies
11. Reports of Expert Committees
Evaluation of RCT
Were all clinically appropriate outcomes measured?
Did an ethics committee approve the study?
Any statistically significant results also clinically significant?

Any significant adverse reactions?
Was follow-up procedural analysis identical?
Was continuous data analysis vs end of trial data used?
HOW TO DO THE MATH
Incidence
Prevalence
Statistical Formulas
+/- Predictive Values - Probability - The p value
Relative Risk
Risk Reduction
Odds Ratios
NNT (Number Needed to Treat) – Risk Reduction
Confidence Intervals
Sensitivity and Specificity
Regression Analysis
Subgroup Analysis
Health Status Evaluation
Health Economics
OUTCOMES: NOT “STUDY FAILURES”
ACCOUNT FOR ALL even if

•Non-compliant
•Lost to follow-up
OUTCOMES RELATE TO EVERYDAY CLINICAL PRACTICE, including…

• Deaths
• Poor compliance
• Wrong treatment received
• Lost to follow-up
• Etc.
Analyze as a member of the originally assigned group!
Analysis SHOULD BE BASED ON

• INTENTION TO TREAT
• NOT on “completed treatment” analysis
INCIDENCE & PREVALENCE
NEPHROL, a service of NKF cyberNephrology
7/10/03 10:17:12AM
Dear Nephrolers,
I would like to know how to calculate incidence and

prevalence of B and C virus in HD.
Thank you in advance.
Mario Cuba, MD
Servicio de Nefrologia
Hospital Lucia Iniguez Landin
Holguin, Cuba
INCIDENCE & PREVALENCE
Response from Michel Jadoul, MD

NEPHROL, a service of NKF cyberNephrology
Prevalence: total number of positive patients divided by total

number of patients: 20+/200=10%
Incidence: number of new positive cases/total number of

cases negative at start of period (e.g; year)/period (year?)
thus : for instance 2 new positive cases /100 negative cases
at start of year=2%/year.
M.Jadoul, M.D.
PREVALENCE
B
10 cases Hep B in 100 patients

B
10 / 100 = 0.10
B PREVALENCE = 10%
B
B
B
B B
B
B
INCIDENCE
B B
1 year period
B
90 HBsA(-) at start of study
B 5 new cases B
5 / 90 = 0.06
INCIDENCE = 6% per year B

B
B B
B B B
B
B B
B B RECALCULATED PREVALENCE
16 cases Hep B in 100 patients

B
16 / 100 = 0.16
B NEW PREVALENCE = 16% B
B
B
B B
B B B
B
B B
COMPARISON STUDIES: NEW DIAGNOSTIC TESTS
COMPARING A NEW TEST AGAINST THE GOLD STANDARD TEST
RESULTS OF GOLD STANDARD TEST
DISEASE
NO DISEASE
PRESENT
EXPERIMENTAL TEST
POSITIVE TRUE (+) FALSE (+)

a b
NEGATIVE FALSE (-) TRUE (-)

c d
ACCURACY OF TEST - COMPARE TO GOLD STANDARD
What is the usefulness of the test in various groups and subgroups of pts?
TESTS ARE RARELY 100% RESULTS OF GOLD STANDARD TEST

ACCURATE
THEY MUST BE COMPARED DISEASE NO DISEASE

AGAINST THE PRESENT PRESENT
GOLD STANDARD
a+c b+d
EXPERIMENTAL TEST
TEST
POSITIVE
TRUE (+) FALSE (+)
a+b a b
TEST
NEGATIVE FALSE (-) TRUE (-)
c+d
c d
TESTS ARE RARELY

RESULTS OF GOLD TOTALS
100% ACCURATE STANDARD TEST
THEY MUST BE
COMPARED AGAINST
DISEASE DISEASE NOT
THE GOLD STANDARD PRESENT PRESENT
TRUE (+) FALSE (+)

EXPERIMENTAL
TEST
POSITIVE a b a+b
TEST
TEST FALSE (-) TRUE (-)

NEGATIVE c d c+d
a+c b+d a+b+c+d

TOTALS
TESTS ARE RARELY

RESULTS OF GOLD TOTALS
100% ACCURATE STANDARD TEST
THEY MUST BE
COMPARED AGAINST
DISEASE DISEASE NOT
THE GOLD STANDARD PRESENT PRESENT
TRUE (+) FALSE (+)

EXPERIMENTAL
TEST
POSITIVE a b a+b
SENSITIVITY
TEST
TEST FALSE (-) TRUE (-)

NEGATIVE c d c+d
SPECIFICITY
a+c b+d a+b+c+d

TOTALS
SENSITIVITY AND SPECIFICITY
SENSITIVITY = PATIENT (+) TEST (+)

Probability that patient WITH disease
will have ABNORMAL result
(instead of False Negative)
SPECIFICITY = PATIENT (-) TEST (-)

Probability that patient WITHOUT disease
will have NORMAL result
(instead of False Positive)
OVERALL DISCRIMINATION OF TESTS

High SENSITIVITY = low number false negatives
High SPECIFICITY = low number of false positives
Best accuracy if both factors are close to 100%
SENSITIVITY = a / (a + c)
PATIENT (+) TEST (+)
SPECIFICITY = d / (b + d)
PATIENT (-) TEST (-)
POSITIVE PREDICTIVE VALUE = a / (a + b)

If pt tests (+), what is the likelihood s/he has the disease?
NEGATIVE PREDICTIVE VALUE = d / (c + d)

If pt tests (-), what is the likelihood s/he does NOT have the disease?
PREVALENCE = (a + c) / (a + b + c + d)
ACCURACY = (a + d) / (a + b + c + d)
Proportion of results that correctly identify pts with and without disease
(True + and True - as proportion of all results)
LIKELIHOOD RATIO = sensitivity / (1 - specificity)

How likely is it that + result accurately indicates disease, and - result no disease?
LIKELIHOOD RATIO (LR)
Because Sensitivity and Specificity are NOT always 100%
How likely is it that + result accurately indicates disease, and - result no disease?
LIKELIHOOD RATIO FOR A POSITIVE RESULT (LR+)

Probability of (+) result in diseased subject
divided by
Probability of (+) result in a healthy subject
- or worded differently -
Sensitivity
divided by
100% - Specificity
LIKELIHOOD RATIO FOR A NEGATIVE RESULT (LR-)

100% - Sensitivity
divided by
Specificity
DISCRIMINATION = ZERO IF LR = 1
EVALUATING STUDY RESULTS
Example:
Mortality rates in 4444 pts x 5.4 trial years:
11.5% Placebo
8.2% Medicine
RRR 29% (Relative Risk Reduction)

ARR 3.3% (Absolute Risk Reduction)
NNT 30 (Number needed to treat for 5.4
years to save 1 life
QALY = QUALITY ADJUSTED LIFE YEAR
QUALITY RATING
NOT specific for disease or treatment!
Value rating - subject to different values of patients, physician, community

• Patient-based
• Economy-based - cost-utility/cost-effectiveness analysis
Compares outcomes of conditions or intervention(s)

• State of health - vs -Time spent in it
QUALY: QUALITY ADJUSTED LIFE YEAR
CHRONIC MEDIOCRITY
PERFECT HEALTH
10
10 9
9 8
8 7
7 6
6 5
5
4 4
3 3
2 2
1 1
2 3 4 5 6 7 8 9 10 2 3 4 5 6 7 8 9 10
DETERIORATING HEALTH 5-yrTREATMENT COMPARISON

9
8
7 10
6
5 8
Chemo&Rad
4 6
3 4 Radiation only
2
2 Chemo only
1
0 0
2 3 4 5 6 7 8 9 10 1 2 3 4 5
TEST RESULTS: VARIABILITY
FACT: Study results may vary.
Group too small

May be discovered
Not representative or identified through
of larger group study
Etc.
Age, sex, race, condition, culture, etc.

Compliancy issues (patient and
physician!)
FACT: Variability may or may not be significant
Group too small

May be discovered
Etc.

physician!)
Obviously faulty studies should be eliminated.
Group too small

May be discovered
Etc.

physician!)
Some variability should be expected in the rest.
Group too small

May be discovered
Etc.

physician!)
Some factors are completely unexpected.
Group too small

May be discovered
Etc.

physician!)
The statistics allow us to distinguish between
and variability due to

PROBABILITY CHANCE
Statistically significant No statistical

significance
Results are measurable
and predictable Random, unpredictable
Affected by sample size
(1 in 20 is less convincing than
1 in 10,000)
WARNING
PROBABILITY should
NOT
be confused with
CHANCE!
PROBABILITY
A Study in Probability…
QUESTION #1:
What percentage of patients will develop diarrhea while
taking Antibiotic A?
QUESTION #2:
Will the results be the same, better or worse on
Antibiotic B?
QUESTION #1:
50 20 study
50 groups
50
50 50
50
50 50 pts each
50 50 50 study
50
50 50 50 50
1000 patients
50 50 50 total
50 50
QUESTION #1:
Conduct
NUMBER OF STUDIES
studies
Organize
results
PERCENTAGE OF PATIENTS WITH DIARRHEA

4 6 8 10 12 14 16%
IDENTIFY STATISTICALLY SIGNIFICANT RESULTS
NUMBER OF STUDIES
The Bell Curve

4 6 8 10 12 14 16%
Most COMMON result =
NUMBER OF STUDIES MODE

4 6 8 10 12 14 16%
The CENTER of distribution =
NUMBER OF STUDIES MEDIAN

4% 6 8 10 12 14 16%
DETERMINE RESULTS OF STUDY
MODE (most COMMON result) = 10%
MEDIAN (the CENTER of distribution) = 10%
NUMBER OF STUDIES

4 6 8 10 12 14 16%
CONCLUSION
“10% of patients will develop diarrhea while
taking Antibiotic A.”
NUMBER OF STUDIES

4 6 8 10 12 14 16%
Wait… What about the other 15 study groups?
You can’t just

NUMBER OF STUDIES
ignore them! Or can

you?

4 6 8 10 12 14 16
You can’t just

15 groups Or can
x 50 per group ignore them!
you?
= 750 patients
(75%!)

4 6 8 10 12 14 16
Studies must You can’t just

ignore them! Or can
account for
you?
ALL patients

4 6 8 10 12 14 16
You can’t just

NUMBER OF STUDIES
Results should ignore them! Or can

not be ignored, you?
but
STATISTICAL
SIGNIFICANCE
may be
questioned.

4 6 8 10 12 14 16
So, how is
STATISTICAL
SIGNIFICANCE
NUMBER OF STUDIES
determined?
My head is
starting to feel
heavy…

4 6 8 10 12 14 16
STATISTICAL SIGNIFICANCE
PROBABILITY is determined by it.
CHANCE is not related to it at all!
How is it determined?
Let’s use our study on “Antibiotic A” as the example
20 groups were tested.
Only 1 of 20 groups landed at each end of the Bell Curve….
WHY?
NUMBER OF STUDIES
SAMPLE
VARIATION?
or CHANCE?

4 6 8 10 12 14 16%
It is tempting to say,
“There is a 1 in 20 chance that other patients
will land in these categories.”
NUMBER OF STUDIES
But that
would NOT
be a correct
statement!

4 6 8 10 12 14 16%
It is tempting to say,
“There is a 1 in 20 chance that other patients
will land in these categories.”
NUMBER OF STUDIES
Why?
Because
CHANCE
can not be
used to
predict
future
results!

4 6 8 10 12 14 16%
CHANCE is based on RANDOM possibility.
Example: THE COIN TOSS
Coins tossed: 20
“Heads” 17 (85%)
“Tails” 3 (15%)
Statistical Significance: ZERO!
The next coin toss will still produce a random result!
Random results can not be used to calculate
Statistical Probability.
So instead of measuring “chance”…
“There is a 1 in 20 chance that patients will
land in one of these two categories.”
1 in 20 1 in 20
chance chance

4% 6 8 10 12 14 16%
We need to determine the PROBABILITY!
“There is a 5% probability that patients will
land in one of these two categories.”
1 in 20 5%
Translates into
chance probability

4% 6 8 10 12 14 16%
The results now look like this:
There is a There is a
5% probability 5% probability
that 4% of that 16%of
patients will patients will
develop develop
diarrhea on diarrhea on
Antibiotic A. Antibiotic A.

4% 6 8 10 12 14 16%
And now… Let’s abbreviate it some more!
p = 0.05
DECIMEL
1 in 20 Probability
chance = 5%
FRACTION PERCENTAGE

4% 6 8 10 12 14 16%
…by changing “% probability” to the “p value”
p = 0.05
DECIMEL
1 in 20 Probability
chance = 5%
FRACTION PERCENTAGE

4% 6 8 10 12 14 16%
The “p value” is statistically important!
p = 0.05
DECIMEL
1 in 20 Probability
chance = 5%
FRACTION PERCENTAGE

4% 6 8 10 12 14 16%
It determines statistical PROBABILITY.
p = 0.05 “p value”
1 in 20 Probability
“chance” = 5%
4% 6 8 10 12 14 16%
PROBABILITY vs CHANCE
…but PROBABILITY is
very important!
It tells us the likelihood that
something will happen.
NUMBER OF STUDIES
So, CHANCE
has ZERO
significance

4% 6 8 10 12 14 16%
OUR PROBABILITY STATEMENT
“There is a 5% probability that our study patients
will fall into either of these categories.”
p = 0.05 p = 0.05

4 6 8 10 12 14 16%
THAT IS A LOW PROBABILITY!
Anything less than 5% (p= 0.05)
MAY be due to chance.
p = 0.05 p = 0.05

4 6 8 10 12 14 16%
And anything less than 1% (p= 0.01)
is MOST LIKELY due to chance!
p = 0.01 p = 0.01
4 6 8 10 12 14 16%
Anything less than 5% (p= 0.05) MAY be due to chance.
Anything less than 1% (p= 0.01) is MOST LIKELY due to chance.
p = 0.05 p = 0.05
p = 0.01 p = 0.01
4 6 8 10 12 14 16%
But when compared to another study…
(p= 0.05) becomes VERY SIGNIFICANT
And (p= 0.01) becomes HIGHLY SIGNIFICANT!
4 6 8 10 12 14 16 18%
But when compared to another study…
4 6 8 10 12 14 16 18%
Antibiotic A
NUMBER OF STUDIES

4% 6 8 10 12 14 16 18%
Antibiotic B
NUMBER OF STUDIES

4% 6 8 10 12 14 16 18%
DIFFERENCE = STATISTICALLY SIGNIFICANT
4 6 8 10 12 14 16 18%
The P value
Measures Probability
How often is this finding expected to occur?
Determines Statistical Significance

What is the likelihood these findings are TRUE or FALSE?
Do the comparative findings show a significant difference?
Meaningful ranges
p >0.05 Not significant
p <0.05 Statistically SIGNIFICANT
p <0.01 HIGHLY SIGNIFICANT!
Does probability provide PROOF?

NO! We could be misled by it.
The sample size is very important when determining probability!
SAMPLE SIZE and PROBABILITY
EXAMPLE:
100 pieces of fruit are in a bin: APPLES and ORANGES
You close your eyes and pick 10 of them:
Question: Does your sample accurately

represent what is in the bin?
Answer: No!
Larger samples provide a closer approximation of the

populations they represent... But the only way to get
100% proof is to examine “all of the fruit in the bin!”
The P value
Meaningful ranges
p >0.05 Not significant
p <0.05 Statistically SIGNIFICANT*
p <0.01 HIGHLY SIGNIFICANT!**
*Significance only means that CHANCE is an unlikely explanation for the
results
LIMITATION
The p value determines LIKELIHOOD… Not proof!
CAUTION
Statistical significance does not necessarily imply any clinical
significance!
EXAMPLE: Looking through a pinhole will improve vision in most people… But would
this be an appropriate treatment for your myopic patients? (Key Topics in EBM )
MISCELLANEOUS STUFF
Literary Search
5. Evaluate
PATIENT
Question
or
Problem
(The Three Major Components of EBM)

PATIENT
PATIENT
PATIENT
“A METHODOLOGICAL
MINEFIELD”
PATIENT
“A
METHODOLOGICAL
Difficult time
Personal MINEFIELD”
priorities may understanding
conflict with background
yours information
PATIENT
Recognize:
Needs
Choices
Preferences
Values
Socioeconomic
concerns
CONFLICT?
SEPARATE THE ISSUES!
Respect the Help the patient to
Help the patient
personal to negotiate a understand and
priorities of the decision on interpret available
patient intervention, information
treatment
PATIENT
And then help the

patient pull it all
together again
PATIENT
KEY POINTS
PARADIGM SHIFT
OLD: Doctor had authority

(despite the pile of unread journals!)
NEW: Current Best Evidence leads medical practice

but it MUST be individually applied
THE INDIVIDUAL PATIENT
Every patient is different. Treat YOURS and not others
The “ideal” course of action is not necessarily best for THIS patient.
EBM + Psychosocial factors =

THIS patient should be advised to take
THIS therapy at
THIS point in time.
Literary Search
5. Evaluate
Evaluate
INFORMATION
Adequate resources?
Ease or Difficulty of finding and getting desired information?
Costs?
INTERVENTION
Patient response or acceptance?
Ease or Difficulty of Application?
Clinical outcomes?
EBM PROCESS
EFFECT ON PRACTICE
Will this particular experience change our thinking or practice?
SELF EVALUATION
How did we do? (Question, Search, Appraise, Apply)
How could we improve our own EBM performance?
EBM: PROS, CONS and LIMITATIONS
PROS
Clinicians update knowledge base routinely
Improved understanding of research methods
Physician becomes more critical in use of data
Increased confidence in management decisions
Increased computer literacy, data search technology
Better reading habits
Provides framework for group problem solving, team generated

practice
Transforms weakness or paucity of knowledge into positive change
OK to be uncertain
OK to be skeptical
OK to be flexible
Integrates medical education, research and clinical expertise
Can be learned by non-clinicians – other HCWs, patient groups,

purchasers, etc.
Allows us to keep up with our better-educated patients!

Increased contribution of junior MDs
Increased patient benefit
Better communication with patients re: rationale of management

decisions
Promotes better and more appropriate use of limited resources
May reduce costs or medical care or practice by eliminating outdated or

unnecessary factors
Can be learned at any stage of physician’s career

CONS
Time consuming
Information overload
Time to learn and practice
Time may be needed for team conferencing, planning and review
Takes $$$ to establish resource infrastructure – library, office, etc.
computers, peripherals
Internet costs
Programs, software information, CD-ROMS
Subscription costs – online and paper resources
May increase cost of care (but hopefully offset by elimination of

unnecessary medical interventions, tests, journals, etc. – plus save time
in getting proper intervention)
Online references made to unavailable journals or references
Exposes gaps in the evidence (but provides ideas for researchers!)

Requires computer skills (but can be done with minimal
computer literacy and skill)
May expose your current practice as obsolete or dangerous

(loss of authority and respect)
LIMITATIONS
Lack of evidence (shortage of studies)
Difficulty applying evidence to care of a particular patient
Barriers to the practice of high quality medicine

Lack of skills (search, appraise, etc.) (foster development of new
skills!)
Lack of time to learn and practice EBM (promotes lifelong learning thru
better focus)
Lack of physician resources for instant access to evidence (EBM has

worldwide applicability)
RESTRICTED AVAILABILITY OF LAB TESTS
NON-TEXTBOOK CASE
co morbidity, additional risk factors
AFFORDABILITY (MD & PT)“I can’t afford to practice EBM.”
Language barriers – available evidence may be unreadable, should be

included
Physician attitude: Can be the greatest limitation!
“It decreases the importance of my clinical expertise”

(that’s a necessary component!)
“It only applies to those involved in research.”

(promotes cooperation among multiple physicians)
“It ignores patient values and preferences.”
“It’s just another cookbook approach to medicine.”
“It’s a poorly disguised way to cut medical costs.”

(cost of care may actually increase)
“It’s a way to ration care and resources.”

(Provides better utilization of avail resources)
DISAGREEMENT
Pt’s comfort, choice, acceptance, values preferences

Vs MD’s recommendations
DOES RISK OR SIDE EFFECTS OF TREATMENT OUTWEIGHT THE

BENEFITS?
The unanswered question…
“DOES EBM REALLY MAKE A DIFFERENCE?”
Effect of practicing EBM on patient outcome is actually

unknown – no studies done
EBM good based on population studies:

(ie: Pts who rec’d ___ generally fare better than those who don’t)
EBM IN DEVELOPING COUNTRIES
LIMITED RESOURCES
May help to eliminate unnecessary or poor quality
screening tests (ie: resting EKG to screen for
CAD = high false negative and false positive
rates)
LIMITED DRUG REGULATION

Approval for drug marketing easy - promotes
insurgence of new drugs for questionable
indications, limited effectiveness, false claims,
inflated prices based on ad response (include
“more expensive is better”)
LIMITED CAPACITY FOR CME

Drug companies - may sponsor meetings that are little
more than captive marketing sessions or biased
education sessions (drug education vs promo)
Result may be push for more expensive, less effective

treatments (ie push for CCB’s over BB’s) - calc channel
blockers over Beta Blockers
LIMITED ACCESS TO LITERATURE DATABASES
Desktop computer with CD ROM reader and modem

($900)
Electricity
1 yr subscription to MedLine on CD ROM (?500)
Internet connection $25/mt
Convince administrators of expense:

Publicly cite how searches help with lectures, research
and patient care management decisions
Get equipment from drug companies

(usually strings attached)
LIMITED ACCESS TO ADEQUATE LIBRARY FACIILITIES
ALMOST INEVITABLE IN DEVELOPING COUNTRIES

Identify resources via search, but then unable to retrieve articles!
A top EBM practitioner (Philippines) recommends:

1. Top 3 medical libraries in your country
2. Multinational drug company libraries
3. Friends and colleagues - including in other countries
QUESTIONABLE APPLICABILITY OF ARTICLES

RETRIEVED
Article describes a treatment that worked in one country, but

seems impossible in yours
Check…
• Are there pathophysiologic differences?
• Will patient differences diminish the treatment response?
• Patient compliance issues?
• Provider compliance issues?
• Co-morbid conditions which will alter the benefits or risks?
OBSTACLES TO TEACHING OR LEARNING EBM
Your Hospital or Institution does not reimburse for time spent on

Continuing Medical Education programs
The standard 5-day workshop would be far too costly to provide or

attend!
Need to learn the basics - computer skills, etc.
TRY THESE!
Combine efforts to learn more and practice EBM with handful of
colleagues (small group learning)
Ask about basis for information provided by drug reps, medical supply
companies, etc. It will prompt them to provide you with on the spot
teaching and better information, too!
EBM LIBRARY
BASIC REQUIREMENTS
Convenient – easy access at point of contact with patient if
possible
Current – Up to date information
Electronic Database – Should be included

• Online
• CD-ROM
ELECTRONIC DATABASES
Evidence-Based Medicine Reviews (EBMR) – from Ovid

(ovid.com)
- combines Cochrane, Best evidence, Evidence Based
Mental health, EB Nursing, Cancerlit, healthstar, AIDSline,
Medline, and journal links (Described by one EBM specialist
as “the best”)
Cochrane Library – “Gold Standard” for systematic reviews
Best Evidence
Medline – world’s largest, free resource – over 10 million

references
PERSONNEL
Medical Librarian
Informatics Specialist
“We can learn a great deal about current best information

sources from librarians and other experts in medical
informatics, and should seek hands-on training from them
as an essential part of our clinical training.”
(ch 2 p29-30 – Blue circled 2)
PRINTED RESOURCES
TEXTBOOKS
most obsolete!
Some updated yearly, plus heavy references and scientific
evidence for support
Clinical Evidence (BMJ Publishing Group & ACP – 1999-

present)
Evidence-Based On Call (http//cebm.jr.ox.uk/eboc/eboc.html)
Up To Date (General medicine, CD format, Medline abstracts

used for evidence)
Scientific American Medicine – limited references from

Medline, Harrisons
JOURNALS
Traditional Journals
subject to author submissions
specialists need to read and evaluate
may subscribe to services that send articles of interest to
your specialty
timely, instant information at time of publication
Ex: NEJM, Clinical Nephrology, etc.
Evidence Based journals

selects best studies from multiple journals of interest,
summarizes best evidence
Good for use by generalists
Lag time from original publication: 3-6 months
Ex: Evidence Based Medicine, Evidence Based
Nursing, Evidence Based CV Medicine, etc.
SPECIAL RESOURCES
WHO Blue trunk
Hinari
PATIENT RESOURCES
Medical treatments www.nlm.nih.gov/medlineplus
Medical guidelines www.guideline.gov
THE NEXT LEVEL: ADVANCED EBM
SUMMARIZE AND STORE INFORMATION

Future reference
SHARING INFO
Local Colleagues
Author paper
TEACHING
New skills or treatments
EBM practices
OTHER APPLICATIONS
Care of the individual patient
Team protocols
Hospital or practice guidelines
EBM in Medical Education
Message to medical educators from Trisha Greenhalgh, MD,
co-author of Evidence Based Health Care Workbook:
“An important challenge for medical educators… is to

recognize that the competent student (and clinician!) is one
who knows how to cope with an immense and rapidly
changing body of knowledge and not one who excels in
recalling the traditional or memorizing the ephemeral.
The deans of medical and nursing schools must develop an

infrastructure that allows problem-based, self-directed
learning methods to develop within the didactic, lecture-
based curricula, which have seen no fundamental changes
for two centuries or more.”
ADVANCED EBM: ADVANCED APPLICATIONS
APPLY METHODS TO…

Care of the individual patient
Team protocols
Hospital or practice guidelines
Continued Learning: problem-based approach
Teaching
SELF-DIRECTED LEARNING
JAMA Series of User Guides
“Clinical Epidemiology: A Basic Science for Clinical

Medicine”
Week-long workshops
On-the-job learning (in your own practice)


Judy Tarselli, RN
Dubai, UAE
Karachi, Pakistan
October 2003

Special thanks to our sponsors at Janssen-Cilag of Dubai


Judy Tarselli, RN
Dubai, UAE
Karachi, Pakistan
October 2003

Special thanks to our sponsors at Janssen-Cilag of Dubai

EBM Presentation

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

EBM Presentation

Transféré par

Droits d'auteur :

Formats disponibles

EVIDENCE BASED MEDICINE

A new approach to clinical care and research

Developed and presented by

Organized by NKF cyberNephrology

Special thanks to our sponsors Janssen-Cilag

(Check all that apply)

A. Training, clinical experience and consultation

B. Convincing evidence (non-experimental) from

C. Preferences of the patient

D. Active search of Randomized Controlled Trials,

B. Convincing evidence (non-experimental) from

C. Preferences of the patient

D. Active search of Randomized Controlled Trials,

Graduate Medical School Practiced Physician

A. Training, clinical experience and consultation with

B. Convincing evidence (non-experimental) from articles,

C. Preferences of the patient

D. Active search of Randomized Controlled Trials,

JOURNALS (1987 to present)

B. Convincing evidence (non-experimental) from articles,

And conflicts WILL occur!

And conflicts WILL occur!

Education about current alternatives and risks is often

Education about current alternatives and risks is often

The patient’s preferences MUST be considered!

B. Convincing evidence (non-experimental) from articles,

C. Preferences of the patient

D. Active search of Randomized Controlled Trials,

Don’t forget to allow for individual human differences

If you checked all 4 items…

B. Convincing evidence (non-experimental) from articles,

C. Preferences of the patient

D. Active search of Randomized Controlled Trials,

B. Convincing evidence (non-experimental) from articles, case reports,

C. Preferences of the patient

D. Active search of Randomized Controlled Trials, Systematic

Dr. Sackett defines EBM as:

Clinical practice: Based on the best current evidence

Patient Care: Compassionate, patient-oriented

Learning & Teaching: Problem-based, problem-solving

Research: More stringent approach, better proof criteria

When am I supposed to find

The FIRST step

That is not a problem!

• if we fail to recognize those needs

• if we fail to bridge the information gap

It will make life

And also for others

• Be answerable Tried prednisone 60mg qd -

“In the study below, proteinuria and renal

Ballardie FW, Roberts IS. Controlled prospective

“I have patients on this regime who have

“Where are we now?

STUDENT GRAD EXPERT

3. GENERAL KNOWLEDGE ABOUT DISORDER

PT AND/OR PROBLEM Differential dx, Unusual presentation, uncertain

INTERVENTION Exposure, test. Prognostic factor,

STUDENT GRAD EXPERT

P PATIENT type of patient or population

E EXPOSURE environmental, personal, biological

I INTERVENTION clinical intervention

C COMPARISON compare alternative treatment

O OUTCOME clinical outcome of interest