Variability and Bias

Variability and Bias
PATIENT PROFILE
of shots is off center. This might occur, for example, if
the sight of the gun were bent. The precision is com A 45-year-old man began working as a
production p arable, but the result in D is systematically off target supervisor, and his
employer required that he undergo or biased. The results in target C are accurate, or
valid. a complete medical examination. His physician It is important to consider the
accuracy and precision learned that the patient's father had died of myocar of any
measurements made in the medical setting. In dial infarction at age 65. On physical
examination, the c linical medicine and m edical research, variability can patient was
moderately obese, and his blood pressure occur at a number of different levels (eg, at
the level of
the individual or the population) (Table 10-1). At each vealed no notable abnormalities.
The patient' s total, l evel, the variability inherent in the method of mea serum
cholesterol level ( nonfasting) was 242 mg/dL. V surement is important.
According to t he guidelines of the N ational Cho lesterol Education Program
(NCEP) , a total serum Variability within the Individual cholesterol concentration
greater than 2 40 mg/ dL is T he first level of concern is variability in the true an indication
ossible pharmacologic lowering o
for p f value of a person's characteristics over time.
This was serum cholesterol. A v alue of 2 00-239 m g/ d
L is con a source of concern for
the clinician in the Patient sidered borderline and should t rigger d ietary i nter Profile.
Some potential sources of individual variabil v ention, and a v alue l ess t han 2 00
mg/d L is considered i ty are listed in Table 10-2. Variation can occur be normal
cause of biological changes in an individual over time. B ased o n t he i nitial
cholesterol results, the p hysi T hese changes may (1) occur on a minute-to-minute cian
asked t he patient t o r eturn i n 2 weeks for f urther b asis (eg, heart rate), (2) follow a
reguiar diurnal pat testing. On repeat m
easurement, the t otal serum cho tern (eg, body
temperature), or (3) progress with nor). lesterol concentration was 198 mg/dL on a
al development (eg, height or weight); lipid profile. Table 10-1 lists several
fasting m
different factors When the variation within a subject is large, a single that could explain
the observed variability i n m easured measurement may not adequately represent
tal s erum cholesterol l evel. T
the "true" to he s ource of t his v ari s tatus of that individual.
By repeating a test, the physi ability in t he m easured t otal cholesterol l evel had i m
portant implications for h
ow the physician treated t his v alue and its variability. This m
ay also
provide the
clinician with information about variability or error due to the measurement technique. In the
Patient
Profile, different results were obtained when the total VARIABILITY IN MEDICAL
RESEARCH
serum cholesterol level was measured a second time
when the patient was fasting. It is unlikely, however, Difficulties in the interpretation of test
results of in that the fasting state alone could cause such a drop in dividual patients a
re
magnified when groups of pa total serum cholesterol concentration. Furthermore, it tients are
studied. The sources of variability in test is unlikely that the patient could have made the
kind of results and errors in medical research are discussed in dietary or other
alterations in 2 weeks that would lead this chapter. Appreciation of these i ssues is
important to the observed change in total serum cholesterol level. for the interpretation and
appropriate application of research findings in the clinical setting
Variability Related Variability in measurements can be either random or to Measurement
systematic. A schematic representation of random and Laboratory measurements
of total serum cholesterol systematic variation is shown in Figure 10-1. The level
are notorious for both variability and error. To de shots at the targets i n both A
and B are centered around termine which value--198 mg/dL or 242 mg/dL-is the middle,
but in A the shots are less scattered and closer to the truth, the physician in the Patient
Profile have less variability, or more precision. In targets C w
ould need to know whether both
measurements were and D t he scatter is similar, but in target D t he cluster obtained in the same
laboratory. For example, the first
141
142 / CHAPTER 10
Table 10-1. Levels of variability.
Levels
Features
Individual
Population
Individual variability Measurement variability Genetic variability between individuals Environmental
variability Measurement variability Manner of sampling
Size of sample Measurement variability
Sample
result may have been obtained from a desktop analyzer in the physician's office,
whereas the lipid profile may have been measured in a standardized laboratory. In
lity, the physician may not be able to discern readily which value is closer to the
re a
truth. This is one reason that programs with guidelines that support cutoff points for
clinical decision making, such as the NCEP, often rec ommend that elevated values be
confirmed by repeated measurements over time before treatment is instituted.
Figure 10-1. Schematic illustrations of increased random error (target B versus target A)
and systematic error (tar get D versus target C).
Variations within Populations
Just as there is variation in individuals, there is also variability in populations,
which can be considered the c umulative variability of individuals. Because
popula tions are made up o f individuals with different genetic constitutions
who are subject to different environmental influences, populations often
exhibit more variation than individuals. Physicians use knowledge about
variability in populations to define what is "normal" and "a bnormal.” The
physician in the Patient Profile c ould refer to population survey data to
learn that for 45-year-old males, a total serum cholesterol level of 200 mg/dL is
L is equivalent to the
close to the 50th percentile, and a con centration of 240 mg/d
75th per centile. Accordingly, t he patient generally falls in the upper half of the
population distribution of total serum cholesterol values. Assuming that the
measurement is correct, t his could be a result of genetic factors, envi ronmental
factors, or both.
hort studies such as the Framingham Heart Study, groups of subjects were
followed and compared according to their different levels of total serum
cholesterol and the as sociated frequency of death from myocardial infarction
or stroke. In these investig
ations, a higher level of total serum cholesterol was
associated with an increased risk of death from cardiovascular disease.
When investigators perform such studies, they cannot usually study the
entire population. Instead they study s
ubsets or samples of t he population. This
introduces an other source of v ariability-sampling v ariability--that is
important in medical research. Using a single sam ple of subjects to
represent the population is analogous to using a s ingle measurement to
characterize an indi vidual. Repeated samples from the population will give
different estimates of the true population values. Sampling variability is
illustrated in Figure 10–2. In the source population of 20 persons, there are five in
dividuals (25%) with total serum cholesterol values above 240 mg/dL. I n the three
different samples of five subjects drawn from the source population by c hance, the
proportion of individuals with total serum choles
Variability in Research Studies
It is worthwhile to ask how the clinician would know that a total serum cholesterol
value in the u pper end of the population distribution is disadvantageous. Are these
values really unhealthy? Answers may be found in stud ies that have linked the level
of total serum cholesterol with an increased risk of cardiovascular mortality. In co-
Table 10-2. Potential sources of variability in measurements of individuals.
Features
Sources of Variability Individual characteristics
Diurnal variation Changes related to f actors such as age, diet, and exercise Environmental f actors such
as season or temperature Poor calibration of the instrument Inherent lack of precision of the instrument
Misreading or misrecording information f rom the instrument by the technician
Measurement characteristics
VARIABILITY AND BIAS / 143
Source population
Study A (200 subjects)
305
180 174
Diet
215 305
276 195 215
Prevalence = 40%
H
Drug
170
233 276 146 195
5
10
15
Sample 1
Risk (percent)
205
188
Sample 2
295 146 220
190
Prevalence = 20%
162
Study B (2000 subjects)
Sample 3
228
295 170 164 248 162 220
Diet
Drug
219
219 164 190 188 233
Prevalence = 0%
228 250
15
5 10 Risk (percent)
Prevalence
= 25%
Figure 10-2. Schematic diagram of sampling variability The source population of 20 persons
has a 25% prevalence of hypercholesterolemia (elevated cholesterol values are pre sented in
bold). The three random samples of five persons yield prevalence estimates ranging from 0
to 40%,
Figure 10-3. The effect of sample size on precision of risk estimates. Point estimates
are shown as dashed vertical lines and 95% confidence intervals are shown as solid hor izontal lines.
In both studies, the 5-year risk of developing myocardial infarction was 9% among persons
receiving dietary therapy and 6% among persons treated with a cholesterol-lowering drug. In the
larger study, however, the 95% confidence intervals are narrower, and the difference in risk
between treatment groups is statistically significant.
terol values above 240 mg/dL ranges from 0 to 40%. Each of these small
samples presents a different pic ture of the source population. A larger
sample size would result in less variability and would more likely represent the source
population.
Variability can be important in other ways when two groups are compared in a
study. The goal of such stud les often is to determine whether a measurable differ ence
exists between the groups. When a research paper reports no statistically
significant difference between groups, the reader must ask the following
question: W as there actually no difference between the two treat ments, or was the
estimate of effect so imprecise that the investigator could not distinguish
differences be t ween the two groups (ie, a type II error)?
A graphic display of the results of two hypothetical studies of the same question
is presented in Figure 10-3. In each study, the investigators attempted to determine
whether a cholesterol-lowering drug had a favorable effect on the risk of developing
myocardial infarction. The measure of effect that was estimated in each study
was the risk ratio. Each study compared a group of pa tients randomly allocated
to receive the cholesterol lowering drug with a group chosen to receive
dietary modification alone. The researchers reached different conclusions. In the
study with the smaller sample, the report indicated that the drug had no
beneficial effect o n reducing the risk of developing myocardial infarc tion, when compared
with diet therapy. In the study
with the larger sample, the investigators concluded that the drug decreased the risk
of developing myocardial infarction, when compared with dietary management.
As shown in Figure 10–3, Study A had a small sam ple size, which resulted in
imprecise estimates (ie, wide confidence intervals) of the risk of developing
myocardial infarction in the two groups. Consequently, the two estimates
overlapped, and the statistical test was not capable of distinguishing between the
effects of the two treatments. The investigators concluded that there was no
difference in risk of developing myo cardial infarction between patients who received the
cholesterol-lowering drug and those who received di etary therapy. In Study B,
the investigator used a larger sample size yielding the same point estimates
of risk in the two groups but with much greater precision (ie, narrower confidence
intervals) in the estimate of the ef fects of the drug. With this gain in precision, the sta
tistical test was able to distinguish between the two groups, and the investigator
was able to infer correctly that the cholesterol-lowering drug was superior to di
etary therapy. Generally, the larger the sample size, the more precise the estimate of
effect and the smaller the detectable differences between groups. In studies with very large
sample sizes, small differences between groups may be judged to be statistically
significant but
144 / C
H
APTER 10
have little biological or clinical meaning. For example, An example of the kind of
difficulty that can occur a study of 20,000 subjects might have concluded that a when
study results are generalized is the criticism that 1% difference in risk of developing
myocardial infarc too many clinical studies focus on white males. One tion was
statistically significant. It is unlikely, however, such study is the Lipid Research
Clinics-Primary that a difference in risk this small would justify pro * Prevention Trial,
which demonstrated a significant re longed use of the cholesterol-lowering agent.
duction in cardiovascular mortality for white men aged 35-59 years with
hypercholesterolemia who were
placed on a cholesterol-lowering diet and medication. VALIDITY
Do the results also apply to women? Do t hey apply to
men of different ages, races, or with different, but still The c oncept o f validity c oncerns
the degree to w hich a bnormal, serum cholesterol levels? These questions a measurement
or study r eaches a c orrect conclusion. l ed to the suggestion that federally funded
research A measurement or study may lead to an incorrect (in- should include women,
minorities, and children in the valid) conclusion because of the effects of bias. The
study populations. variability seen with b ias is systematic or nonrandom and d istorts the
estimated effect. In Figure 10-1, the Bias amount of bias can be determined by the
degree to which Bias i s a systematic error in a s tudy t hat leads to a the shots are off
target in D. Unfortunately, in medical distortion o f the results. Bias, a threat to validity,
can research the truth (bull's-eye) may not be known, or occur in any research, but is of
particular concern in there may be no "gold standard” for comparison, observational
studies because the lack of randomiza Consequently, the degree of bias often is difficult to
de tion increases the chance that study groups will differ termine. Two different types
of validity, internal valid with respect to important characteristics. Bias often is i ty and
external validity, are described in this chapter. subdivided into different
categories, based on how bias
enters the study. The most common c lassification di Internal Validity.
alidity i s t he e
vides bias into three categories: Internal v xtent t o w
hich t he r esults o n
f , a
investigation accurately reflect t he t rue situation of ( 1) Selection bias V t he s t udy
population. I f the results are not valid in the (2) Information bias
study population, there is little reason to suspect that (3) Confounding. those results will
apply to other populations. Internal validity is defined by the boundaries of the study itself.
Although these categories overlap, this classification Therefore, a study is internally
valid if it provides a is useful because it provides the reader with a system true estimate
tic approach to evaluate bias. It should be
of effect, given the limits of the popula a
remembered tion studied. Measures that can be used to improve in that with the
exception of confounding, which can be ternal validity often involve restricting the type
of q uantitated, the evaluation of bias is subjective and in subjects and the
environment in which the study is per volves a judgment regarding the likelihood of
(1) the formed. These measures decrease the impact of factors presence of bias and
(2) its direction and potential mag extraneous to the question of interest.
nitude of effect on the results. Even though the magni
tude of bias cannot be quantified, often its influence on External Validity
the results of a study can be inferred. It is important to A result obtained in a tightly controlled
environ discern whether the suspected bias is likely to make an ment, however, may not be
applicable to more general association appear stronger or weaker than it really is.
situations. External v alidity is t he e
xtent to which the , Overestimation of a risk ratio for a
protective exposure results of a
study are
applicable t o ot her populations. and a
separate hazardous exposure is demonstrated External validity addresses the
following question: Do schematically in Figure 10-4. Underestimation of a risk these
results apply to other patients, such as patients r atio for a protective exposure and a
hazardous expo who are older, s icker, or less economically advantaged s ure is shown
in Figure 10–5. than subjects in the study?
External validity often is of particular interest to Selection Bias clinicians who must decide
if a research finding is ap A variety of procedures can be used to select sub plicable to
their clinical practice. Determining whether jects for a study. Usually, it is not possible to
include the results of a study can be generalized i nvolves a all individuals with a particular
disease or exposure in judgment regarding the following:
a study, so a sample of subjects must be chosen. The
procedures used for the selection of subjects depend on (1) The type of subjects included in
the investigation a number of factors, including (2) The type of patients seen by the clinician
(3) Whether there are clinically meaningful differ (1) The design of the investigation
ences between the study population and other (2) The setting of the study
populations.
(3) The disease and exposure of interest.
VA RIABILITY AND BIAS / 145
A. Protective exposure

Bias away from the null hypothesis
Study value
True value
Risk ratio
B. Hazardous exposure
Bias away from the null hypothesis
True
Study value
value
Risk ratio
Figure 10-4. Overestimation of a

risk ratio for (A) a protective exposure and (B)
a hazardous exposure.
A. Protective exposure

Bias toward the null hypothesis
True value
Study value
Risk ratio
B. Hazardous exposure

Bias toward the null hypothesis
Study value
True value
Risk ratio
Figure 10-5. Underestimation of a risk ratio for (A) a protective exposure
and (B) a hazardous exposure.
146 / CHAPTER 10
Often subjects are selected in a manner that is conve- nient for the investigator.
Under optimal circumstances, the method for inclusion of subjects leads to a
valid comparison that, in turn, yields correct information re garding a disease process or
treatment. The selection process itself, however, may increase or decrease the chance
that a relationship between the exposure and dis ease of interest will be detected. A
schematic diagram of the steps involved in recruiting and maintaining a study
population is shown in Figure 10-6. From this diagram, it is easy to see that
selection factors could lead to biased results at several different steps in the
process.
Some aspects of the selection of subjects lead pri marily to problems with the
generalization (extrapola tion) of the results (ie, external validity). Subjects must agree
to participate in a study, and this causes one of the most common problems. Volunteers
for a study may differ from individuals who do not volunteer in various characteristics such as
age, race, economic sta tus, education level, and sex. Moreover, volunteers may be
healthier than those who decline to participate. A study of a population limited to
individuals who are employed may also make it difficult to generalize the results,
because people who work are generally health ier than those who do not. A comparison
of health out comes between workers and the general population may show that the
workers have a more favorable out come simply because they are healthy enough
to be employed (the "h
ealthy worker" effect).
Referral of patients to clinical facilities can also lead to distorted study conclusions.
Selective referral patterns can be seen in the study of children with febrile seizures.
Febrile seizures are brief, generalized seizures that occur in conjunction with elevation
in temperature in children aged 6 months to 6 years. There is some disagreement
about whether these febrile convulsions are predictive of future seizures and other
unfavorable neurologic se quelae. Ellenberg and Nelson (1980) compared the re sults
of a number of studies on the long-term outcome of patients with febrile seizures.
Studies of geographi cally defined populations in which affected children were
followed, regardless of whether medical care was sought, consistently revealed a
relatively low rate of un favorable sequelae. Clinic-based studies tended to report a

high frequency of adverse outcomes. Accordingly, it was concluded that clinic-based
studies selectively in cluded children at the more severe end of the clinical
spectrum. The inferences that might be drawn regard ing the prognosis of a child
e very different based on whether a clinic-based or a
with febrile seizures might b
population-based sample was studied.
Other aspects of the selection process can diminish internal validity. I n a c linical t rial or
cohort s tudy, t he major p otential selection b ias is los s to f ollow- u
p. O
nce subjects are
enrolled in the study, they may de cide to discontinue participation. Certain types of sub
jects are more likely than others to drop out of a study. Furthermore, during the course
of the study some sub jects may die from causes other than the outcome of in
Total population
Sampling scheme
Sampled population
Exclusions
Inclusion
criteria
Eligible subjects
related to selection because the subject already was en rolled in the study. If the lost
subjects differ, however, in their risk of the outcome of interest, biased estimates of risk may
be obtained.
If the unrecognized early manifestations of the dis ease of interest cause exposed
persons to leave the study more or less frequently than unexposed persons, a distorted
conclusion might be reached. For example, in a randomized controlled trial of the
effects of using a cholesterol-lowering drug versus diet therapy on prevention of
myocardial infarctions, bias might be in troduced if drug-treated patients with coronary
insuffi ciency were more likely to develop side effects from treatment and withdrew from
participation, whereas pa tients with coronary insufficiency receiving dietary ther apy
remained in the study.
Selection bias is of particular importance in case control studies (see Chapter 9) in
which the investiga tor must select two study groups, cases and controls, in a setting in
which the exposure has already occurred. For example, it must be decided whether to
use exist ing (prevalent) cases who are available at the time of study, regardless of the
duration of their disease, or to limit eligibility to newly diagnosed (incident) cases. If the
risk factor of interest also is a prognostic factor, the
Subjects asked to
Nonparticipants
Informed
consent
Participants
Lost to follow-up
Complete study
Figure 10-6. Steps in the selection and maintenance of subjects in a study
Consider, for example, a case-control study of total serum cholesterol level as a risk factor
for developing
WE
myocardial infarction. Suppose that of patients who , more difficult for the investigator
to detect a real asso have a m
yocardial infarction those with very high total, ciation
between the exposure and the disease. This is s erum cholesterol levels are more
likely to die suddenly often referred to as a bias toward the null hypothesis or t han
those with lower serum cholesterol levels. Under-toward no association - these
circumstances, a comparison of patients surviv- "Differential
misclassification occurs when the mis ing myocardial infarction with controls
will underesti classification of one variable depends on the status of mate
the true association between elevation in total the other. In a
case-control
study, this type of misclas serum cholesterol level and risk of developing myocar
sification could occur if the information on exposure dial i nfarction.
status depends on whether the subject has the disease. Another p otential type of
selection bias can occur If a case with a myocardial infarction is more likely
to when a case-control study involves subjects who are overestimate the level of
dietary fat intake than a con h ospitalized. Patients with two medical conditions are
trol subject, a biased result may occur. In this instance, more likely to be
hospitalized than those with a single the bias would lead to an overestimate of the
etween dietary
relation disease. Thus, a hospital-based case-control study ship b
fat intake and risk of developing m
ight find a link between two diseases or
between an ! myocardial infarction. exposure and a disease when there is no
association be The difference between nondifferential and differ tween them in the
general population. This type of ential misclassification can be demonstrated by exam
bias, often called Berkson's bias, was demonstrated in ining the data in Figure 10-7.
Consider a case-control a study that showed that respiratory and bone diseases study
of the relationship between high-fat diets and were associated in a sample of
hospitalized patients but r isk of developing myocardial infarction in which the not in
the general population. Thus, in a hospital-based true odds ratio (OR) is 2.3. With
nondifferential mis study, an exposure such as cigarette smoking, which is
classification, the subjects did not r ecall the amount of correlated with respiratory
disease, may also appear to fatty foods eaten, but the errors in recall did not
depend occur together with bone disease because those dis on whether they
had a myocardial infarction. In this sit eases are related in hospitalized patients.
uation, 20% of both cases and controls who ate high
fat diets underreported fat intake. The resulting OR of Information Bias
2.0 was an underestimate of t he true O . On the other Information ( or
R
misclassification) b ias c an occur hand, if all the patients who had a myocardial
infarc when t here i s r andom or s ystematic inaccuracy i n t ion correctly recalled their
dietary fat exposure status, measurement. This can be visualized best in
epidemi but only 80% of the exposed controls correctly re ological studies that involve
dichotomous exposure ported their exposure, then differential misclassifica and
disease variables, such as elevated total serum cho tion would occur. This
type of misclassification can lesterol and myocardial infarction. Subjects are classi
result in either an underestimate or overestimate of the fied according to
whether they have had high total t rue OR. In this example, the investigator overesti
serum cholesterol levels and whether they have had a m ated the OR. m
yocardial
infarction. The investigator either can be Two common types of differential
information bias correct or incorrect, resulting in true-positive and true are often
referred to as recall bias and interviewer negative findings, as well as
false-positive and false bias. Recall bias results from differential ability of
negative classifications of subjects with respect to subjects to remember
previous activities and expo either exposure or disease.
sures. Patients who have a serious disease may search If the errors in classification of
exposure or disease their memory for an exposure in an attempt to explain status
are independent of the level of the other variable, or to understand why they acquired the
illness. Control then the misclassification is termed nondifferential. subjects, who
do not have the disease, may be less Nondifferential misclassification may occur in a
case likely to remember an exposure because it has less control study if the
subject's memory of exposure sta meaning and is less important for them. t us is
unrelated to whether the subject has the disease of When interviewers are
employed to determine ex interest. An example of nondifferential misclassifica
posures in case-control studies, results may be influ tion is sometimes referred to
as unacceptability bias. enced by how the interviewers collect information. If
Subjects may answer a question about the exposure they are aware of the
research hypothesis, the inter with a socially acceptable but sometimes inaccurate re
viewers intentionally or unintentionally may influ sponse, regardless of whether they
have the disease of ence the responses of the subjects. They may probe
interest. Consider a case-control study of myocardial more deeply for responses
from cases than from con infarction in which the exposure of interest is prior in
trols. If a dietary exposure is examined, the inter take of foods high in saturated
fats. Regardless of dis viewers may ask certain subjects specific questions ease
status, respondents may underreport intake of about particular food items.
Interviewers may also foods with high fat content because they think low-fat give
the subjects subtle clues by tone of voice or diets are more acceptable to the
investigator. In most in body language that suggest a preference for certain re
stances, when nondifferential misclassification occurs, sponses. Generally, it is
desirable to blind the inter it blurs differences between the study groups, making
it viewers to the research hypothesis under investigation.
-
-
-
148 / CHAPTER 10
Nondifferential misclassification
Truth:
Study:
Dietary fat
Low
Dietary fat High
Low
High
MI
60
I 40
48
52
No MI
No MI
68
OR = (60 x 60)/(40 x 40) = 2.3

OR = (48 x 68)/(52 x 32) = 2.0
Differential misclassification
Truth:
Study:
Dietary fat High
Low
Dietary fat High
Low
MI
60
L
40
60
. 40
No MI
No MI
32
68
OR = (60 x 60)/ ( 40 x 40) = 2.3

OR = (60 X 68)/( 40 x 32) = 3.2
Figure 10-7. Illustration of nondifferential and differential misclassification of exposure to
odds ratio.)
high-fat diets in a case-control study of myocardial infarction (MI). (OR =
In a case-control study, however, it may be difficult to blind the interviewers to the
disease status of cases and controls. Nevertheless, if the interviewers are not aware
of the exposure of primary interest, biased data collection still can be
minimized.
As a way to reduce misclassification and to improve accuracy of study
measurements, investigators in creasingly are using biological markers. As shown in
Table 10–3, these markers can measure many facets of disease and
exposure-or the relationship between the two. For example, biological markers
can measure
(3) Biologically effective dose ( biological m
arkers
can be used to measure the amount of a sub
stance that reaches the target sites) (4) Biological effect (biological markers can be
used to quantify a deleterious effect of a partic ular exposure).
(1) Susceptibility (biological markers can be used

to identify subjects with particularly high risk
due to a particular biological predisposition) (2) Internal dose (biological
markers can be used to
measure the amount of a chemical or other ex posure in the body)
Biological markers are used in most substantive areas of investigation, including
nutritional, cardiovascular, reproductive, cancer, and infectious disease epidemiol
ogy. Use of biological markers is important in observa tional studies for several
reasons. These markers are important methodologically because they can serve
to r educe misclassification by allowing more accurate as sessment of
exposure or disease status. Furthermore, they may allow the investigator to define
more homoge n eous disease categories or to identify susceptible sub jects,
so that the study can focus o
n specific subgroups.
Table 10-3. Uses of biological markers in epidemiology.

Application of Marker
Example To measure susceptibility
Those with high aryl hydrocarbon hydoxylase activity have higher risk of bladder
cancer To measure internal dose
Those with high serum carotene levels may have lower risk of lung cancer To measure biologically
effective dose Those with greater amounts o
f effective dose of polycyclic aromatic hydrocarbon
DNA adducts have experienced greater exposure and interaction of their DNA to
these carcinogenic hydrocarbonsa To easure biological effect
m
Higher levels of the ras oncogene product may be a preclinical marker of early
carcinogenic response
• Perera F et al: Biologic markers in risk assessment for environmental carcinogens. Environ
Health Persp 1991; 90:247.
and disease o f interest. Confounding can be demon strated by the following
hypothetical example. Suppose investigators undertake a case-control study of the
as sociation between high total serum cholesterol level and risk of developing
myocardial infarction. From the results of other studies, the researchers know that the
risk of myocardial infarction is associated with obesity, and that total cholesterol levels
also correlate with obe sity (see Figure 10–8). Suppose that in our hypothetical
case-control study, 36 of 60 patients with myocardial
Finally, biological markers can help provide insight into the underlying disease
process and pathogenesis.
The use of levels of serum dioxin to measure exposure V of men who worked with the herbicide
Agent Orange during the Vietnam W ar illustrates the use of a biologi cal marker to measure
internal dose. After the Vietnam War, concern arose about wartime exposures of
service men to Agent Orange, in part because of its contamina tion with the highly toxic
trace contaminant known as 2,3,7,8-tetrachlorodibenzo-p- dioxin (TCDD). Because of
this concern, Air Force researchers began epidemio logic studies to assess the health
effects among Air Force veterans associated with exposure to Agent Orange and
TCDD. Researchers initially used job descriptions to classify exposure to TCDD.
Later, after laboratory tech niques became available to measure minute concen
trations of TCDD within the blood, the researchers discovered that classification of
exposure based on job descriptions was associated with substantial misclassifi
cation. In subsequent studies, the more accurate serum) TCDD measurements were
used to assess exposures. V

Despite the importance of biological markers and the possibility that their use may
reduce information bias, they do not eliminate the possibility of systematic errors.
Although it may be diminished by employing a biological marker, misclassification
remains a possi bility. For example, marker instability and inter- or in traindividual
variability can contribute to measurement errors. Moreover, if required biological
specimens are collected after disease occurrence, as often happens in case-control
studies, the presence of disease in cases may affect the biological marker. This
possibility can make the biological marker particularly susceptible to d ifferential
misclassification and measurement error. Bias can even be created if the
investigator adjusts in a ppropriately for a factor that is caused by the exposure of
interest and is associated with the outcome.
Case-control studies of the relationship of B carotene and cancer illustrate the potential for
residual information bias. B-Carotene is a fat-soluble antioxi dant found in many fruits
and vegetables. It acts as a provitamin (vitamin A), protects against development of
cancer in animals, and may reduce the risk of devel oping cancer in humans. In a
case-control study of s erum levels of this antioxidant, differential misclassi fication could
create or accentuate a protective effect, if
cholesterol levels, and only 24 of 60 controls (40%) are discovered to have elevated
serum cholesterol levels. This would s
uggest that elevated t otal serum choles
terol levels are associated with an i ncreased risk of de v eloping myocardial
infarction.
When the observed association is examined sepa rately in obese and nonobese persons,
however, a dif ferent conclusion is reached. Among obese persons, 34 of 40 patients
with myocardial infarction (85%) and 18 of 20 controls (90%) are found to have
elevated total serum cholesterol levels. Among nonobese persons, 2 of 20 patients
with myocardial infarction (10%) and 6 of 40 controls (15%) have high total serum
cho lesterol levels. Thus in the case of both obese and nonobese individuals,
elevated total serum cholesterol levels are more common in controls than in
patients with myocardial infarction. Keep in mind that in the hypothetical study, obesity
was associated with myocar dial infarction, since 52 of 60 obese subjects (87%)
had elevated total serum cholesterol levels, and only 8 of 60 nonobese persons (13%)
had high total serum cho lesterol levels. Clearly, in this hypothetical example, the results
are confounded by the extraneous variable, obesity. The results are illustrated in Figure
10-9.
For a variable-in this case, obesity—to be con sidered a potential confounder, it must
satisfy two conditions:
(1) Association with the disease of interest in the

absence of exposure (2) Association with the exposure but not as a result
of being exposed.
Obesity
tus and a resulting lowering of ß-carotene levels. Although these biases are
somewhat speculative, the potential for bias in case-control studies is evident.
Thus, use of biological markers offers many advan tages, particularly an improved
assessment of exposure and a more homogeneous definition of disease. Never theless,
because use of these markers does not elimi nate the possibility of information bias,
caution in interpretation is still w
arranted.
Total cholesterol
Myocardial infarction
Confounding
Confounding refers t o the mixing of t he e
ffect o
fa
n ariable w
extraneous v ith t he
effects of t he e
xposure
tween total serum cholesterol level and r isk of developing myocardial infarction, with
confounding by obesity.
150 / CHAPTER 10
All subjects
....0000 00000000 00000000 0000

MI
ooo No MI

00000000 00000000 00000000 00000000 O
Obese
Nonobese ..00000 ...... ..... 00000000000000 . .00 1000000 0000000
OOOOOO
0000000
0000000 00000
000 MI
00
No MI
MI
No MI
Figure 10-9. Illustration of the relationship between total serum cholesterol level and risk
of developing myocardial in farction (MI), with confounding by obesity. Shaded circles
represent persons with elevated total serum cholesterol levels and unshaded circles represent persons
with normal total serum cholesterol levels.
SUMMARY
Because i t can be evaluated in the analysis of results, confounding differs from
selection bias and informa tion bias. The presence of confounding is demonstrated
by a change in the apparent strength of association be tween the exposure and the
disease of interest when the effects of e xtraneous variables are taken into account.
Confounding, which is not an all-or-none property of an extraneous variable, may occur to
different degrees in different studies.
Generally, the list o f potential confounders in a study is limited to established risk
factors for the dis ease of interest. There are two accepted methods for dealing
with potential confounders. The first is to con sider them in the design of the study
by matching on the potential confounder or by restricting the sample to limited
levels of the potential confounder. The other m ethod is to evaluate confounding in
the analysis by stratification, as demonstrated schematically in Figure 1 0-9, or by
using multivariate analysis techniques such as multiple logistic regression.
The goal of any epidemiologic study is to provide a valid conclusion. To accomplish this
objective, com plete attention must be given to all aspects of the study, from inception
to design and data collection and, fi nally, to analysis and reporting of results. It is
impor tant to remember that bias can be introduced at any of these stages, leading to
erroneous results. Thus, it is useful to look carefully for potential sources of bias and
to consider their possible impact. Çlinicians must judge whether results can be
generalized to their par ticular practice. Understanding the potential problems with
measurement and bias in medical research im proves the ability of physicians to
decide on appropri ate preventive a nd therapeutic strategies.
In this chapter, the topics of variability and system atic errors in epidemiologic
measurements are dis cussed, with illustrative examples that focus primarily on the
relationship between total serum cholesterol level and the risk of developing myocardial
infarction. A distinction is drawn between random variation, which is inversely related to
precision in measurement, and nonrandom or systematic error, which is related to a
distortion in measurement.
Variability can arise from (1) the subjects under study, (2) differences between
individuals, (3) the ap p
roach used to sample subjects, or (4) the measurement
process itself. Variability related to sampling is likely to diminish as the sample size
increases. With ex tremely large sample sizes, a very small difference in outcome
between study groups can be statistically sig nificant. Whether the magnitude of this
difference is sufficient to warrant a change in clinical practice is a separate, but equally
important, question.
Validity concerns the extent t o which the f indings o f a s tudy r eflect t ruth. Internal validity
relates to the ac curacy of study findings for the persons who are in vestigated.
External validity concerns the extent to which study findings accurately apply to
persons who are not studied.
Bias is defined as lack of validity. Conventionally, bias is classified into three major
types: selection bias, information (misclassification) bias, and confound ing. Selection
bias refers to the introduction of sys tematic errors into study results through the
manner in which study subjects are selected. Information bias re sults in
systematic errors in study findings that origi--
While in storage, the specimens have d

eteriorated, producing artificially low
homocysteine levels.
3. In a cohort study of hormone replacement therapy

and the risk of developing atherosclerotic coro nary artery disease, high socioeconomic
status is a ssociated with both use of hormone replacement therapy and the risk of
developing coronary artery disease.
4. In a case-control study of the relationship o

f
stressful life events and the occurrence of coro nary artery disease, the cases are more
likely than the controls to overreport stressful events.
nate in the approach to collecting information. Two kinds of information bias can exist.
Nondifferential misclassification occurs when errors in the informa tion about one
variable are unrelated to the status of an other variable. Differential misclassification, on
the other hand, occurs when errors in the information a bout one variable are affected
by the status of another variable.
Confounding is concerned with the mixing of the primary effect of interest with the effects of one
or more extraneous factors. In experimental studies, the problem of confounding is
reduced by randomiza tion, which tends to balance the study groups with re spect
to both known and unknown determinants of the outcome. In observational research,
however, s tudy groups may differ appreciably in factors that are (1) related to the risk
isease among unexposed persons and (2) are also associated with the exposure
of d
of i nterest, but not as a result of being exposed. The influence of these potential
confounders can be ad dressed in the study design (eg, through matching or
restrictive inclusion criteria) or in the analysis (eg, through stratification or
regression techniques). Only known confounders can be addressed in observational
research.
No study is immune from t he possibility of bias. The investigator must therefore
consider potential sources of bias when sampling subjects, collecting in formation,
analyzing results, and interpreting find ings. With planning and forethought, it is
possible to anticipate and avoid certain types of error and thus conduct a study that
leads to a convincing and valid c onclusion.
Questions 5-8: For each numbered situation below, select from the following lettered options
the most likely effect on the study findings. Each option can be used once, more than
once, or not at all.
A. Overestimation B. Underestimation C. No effect D. Cannot be determined
5. In a cohort study of the effects of birth asphyxia on
subsequent cognitive development, the tests of cognitive performance are
administered by inves tigators who do n
ot know whether the subjects had birth asphyxia.
6. In a cohort study of obesity and the risk of devel
oping non-insulin-dependent diabetes mellitus, the loss to follow-up i s greater for obese
compared to nonobese persons.
STUDY QUESTIONS
Questions 1-4: For each numbered situation below, select from the following
lettered options the type of error that would most likely result. Each option can be used
once, more than once, or not at all.
7. In a case-control study of ingestion o
f L-trypto
phan as a risk factor for developing eosinophilia myalgia syndrome (EMS), women
with EMS tend to give more false-positive reports of using L-tryp tophan than do women
without EMS.
A. Selection bias B. Nondifferential misclassification C. Differential misclassification D.
Confounding E. Ecologic fallacy F. Random error
8. Patients with a history of myocardial i nfarction
within the past 5 years are enrolled in a case control study of cigarette smoking as a
risk factor for developing a myocardial infarction. Among patients who have had a
myocardial infarction the case-fatality is higher for smokers than for n
on smokers.
1. In a case-control study of t he relationship between
a cholesterol lowering drug and the risk of devel oping breast cancer, control
subjects are sampled from participants in a health screening fair.
Questions 9–10: For each numbered situation below, s elect from the following lettered
options the most likely study design. Each option can be used once, more than once, or not
at all.
2. In a case-control study of the relationship of serum !
concentrations of homocysteine to risk of develop ing coronary heart disease, 20-year-old
stored frozen serum specimens are used to measure ho mocysteine levels among cases
and controls.
A. Case-control study B. Cohort study C. Randomized clinical trial D. E
cologic
(correlation) study
HAPTER 10
152 / C
9. Population consumption of sodium and population

prevalence of hypertension are associated across populations without
being linked in individual subjects.
10. The ability to assign by chance the use of a cho
lesterol-lowering agent to individual subjects re duces the likelihood of
confounding in a study of risk of developing myocardial infarction.
FURTHER READING
Rosenbaum PR: Discussing hidden bias in observational

studies. Ann Intern Med 1991;115:90.
REFERENCES
Patient Profile
Cleeman J, Lenfant C: The National Cholesterol
Education Program. JAMA 1998;280:2099. National Heart, Lung and Blood Institute:
Recommen
dations f or I mproving Cholesterol M easurements: A Report f rom the Laboratory
Standardization P anel of the National Cholesterol Education Program. US Department
of Health and Human Services. NIH Publication No. 90-2964, 1990.
Selection Bias
Ellenberg JH, Nelson KB: Sample selection and the nat
ural history of disease: S
tudies of febrile seizures.
JAMA 1980:243:1337. Roberts RS, Spitzer WO, Delmore T et al: Empirical
demonstration of Berkson's bias. J Chron Dis 1978; 31:119.
Variability Related to Measurement

Report of the National Cholesterol Education Program
Expert Panel on detection, evaluation and treatment of high blood cholesterol in
adults. Arch Intern Med 1988;148:36.
Information Bias
Coates RJ et al: Cancer risk in relation to serum copper
iologic M
ulka BS, Wilcosky TC, Griffith JD: B
levels. Cancer Res 1989:49:4353. H arkers i n
Epidemiology. Oxford University Press, 1990. Perera F et al: Biologic markers in risk
assessment for en
vironmental carcinogens. E
nviron Health Persp 1991; 90:247.
External Validity
Altman DG, Bland JM: Generalization and extrapolation.
BMJ 1998;317:409. Lipid Research Clinic Program: The Lipid Research
Clinic's Coronary Primary Prevention Trial results. 1. Reduction in incidence of coronary
heart disease. JAMA 1984,251:351.
Confounding
Weinberg CR: Toward a clearer definition of confound
ing. Am J Epidemiol 1993;137:1.
e-PIDEMIOLOGY
Patient Profile
http://www.nhlbi.nih.gov/guidelines/c holesterol/
ww.americanheart.org/Heart_and_Stroke_A_Z_
chol_scr.pdf http://w
Guide/chollev.html
External Validity
http://trochim.human.comell.edu/tutorial/ward/
external.htm
Variability in Medical Research

http://www.bmj.com/e
pidem/epid.4.shtml
Selection Bias
http:// www.facsnet.org/report_tools/guides_primers/
epidemiology/chap3.html#selection_bias
Validity
http:// www.usd.edu/-dstevens/ctrial.htm#ERROR
Information Biasa
http://www.facsnet.org/report_tools/g
uides_primers/
epidemiology/chap3.html#misclassification
Internal Validity
http://trochim.human.cornell.edu/kb/intval.htm
Confounding
http://www.facsnet.org/report_tools/guides_primers/
epidemiology/c hap3.html#confounding

Variability and Bias

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Variability and Bias

Transféré par

Droits d'auteur :

Formats disponibles

Variability ​and ​Bias

Table ​10​-​1​. ​Levels ​of ​variability​.

A​. ​Protective ​exposure

B​. ​Hazardous ​exposure

Bias ​away ​from ​the ​null ​hypothesis

Figure ​10​-​4​. ​Overestimation ​of a

A​. ​Protective ​exposure

B​. ​Hazardous ​exposure

148 ​/ ​CHAPTER ​10

OR ​= ​(​60 ​x ​60​)​/​(​40 ​x ​40​) ​= 2​.​3

OR ​= ​(​60 ​x ​60​)/​ (​ ​40 ​x ​40​) ​= ​2​.​3

(​1​) ​Susceptibility ​(​biological ​markers ​can ​be ​used

Table ​10​-​3​. ​Uses ​of ​biological ​markers ​in ​epidemiology​.

(​1​) ​Association ​with the ​disease ​of ​interest ​in ​the

.​.​.​.​000​0 ​00000000 ​00000000 ​0000

​ ooo ​No ​MI

While ​in ​storage​, ​the ​specimens ​have d

3​. ​In ​a ​cohort ​study ​of ​hormone ​replacement ​therapy

4​. ​In ​a ​case​-​control ​study ​of ​the ​relationship o

9​. ​Population ​consumption ​of ​sodium ​and ​population

Rosenbaum ​PR​: ​Discussing ​hidden ​bias ​in ​observational

Variability ​Rela​ted ​to ​Measurement

Variability in Medica​l Research

Vous aimerez peut-être aussi

Variability and Bias

Table 10-1. Levels of variability.

A. Protective exposure

B. Hazardous exposure

Bias away from the null hypothesis

Figure 10-4. Overestimation of a

A. Protective exposure

B. Hazardous exposure

148 / CHAPTER 10

OR = (60 x 60)/(40 x 40) = 2.3

OR = (60 x 60)/ ( 40 x 40) = 2.3

(1) Susceptibility (biological markers can be used

Table 10-3. Uses of biological markers in epidemiology.

(1) Association with the disease of interest in the

....0000 00000000 00000000 0000

ooo No MI

While in storage, the specimens have d

3. In a cohort study of hormone replacement therapy

4. In a case-control study of the relationship o

9. Population consumption of sodium and population

Rosenbaum PR: Discussing hidden bias in observational

Variability Related to Measurement

Variability in Medical Research