Pharmacoeconomics

A Primer for the

Pharmaceutical Industry

Alan Morrison
Albert 1. Wertheimer

Center for Pharmaceutical Health Services Research

Temple University School of Pharmacy

©Temple University, 2002.

 3

TABLE OF CONTENTS
INTRODUCTION 5
DEFINITIONS OF PHARMACOECONOMICS AND OUTCOMES
RESEARCH 6
TYPES OF PHARMACOECONOMIC EV ALUA TION 7
COST ANALySIS............................. 8
CostofCare 8
Cost of Illness and Burden of Illness 9
COST-OUTCOMES ANALySIS.... 10
Cost-Effectiveness JO
Cost-Utility 12
Cost-Minimization 12
Cost-Benefit................................. . .... 13
BASIC CONCEPTS 14
PHARMACOECONOMICS 14
Decision Analysis 14
Definition of Case-Effectiveness 16
Definitions of Costs 18
Perspectives 21
Time Horizon 22
Discounting 23
MODELING FRAMEWORKS 24
Steps in Decision Analysis 24
Influence Diagrams 34
Markov Models 36
CLINICAL EPIDEMIOLOGy 38
5TA TISTICAL ANALySIS 40
UTILITy 41
PSYCHOMETRlCS 42
VARIABLES IN OUTCOMES AND PHARMACOECONOMICS
...................................................................................................... 46
PATIENT OUTCOMES .46
Physiological 46
 4 5

Humanistic 46
Mortality and Morbidity 48 Introduction
ECONOMIC OUTCOMES................... 48 The question 'Does a medicine work?' is answered in a
Non-Monetary Costs 49 controlled clinical trial that is typically financed and designed
Monetary Costs 49
DA T A SOURCES 49
by a pharmaceutical company, carried out hy a contract
Literature Analvsis 50 research organization with the participation of members of
Meta-Analvsis 52 the health care professions and patient volunteers, and
Clinical Studies 55 watched over and vetted by an agency of the Federal
Administrative Databases 56
Financial Data 59
government. The randomized controlled clinical trial is a
ACCOUNTING FOR UNCERTAINTy 60 cornerstone of the pharmaceutical industry and the means by
DEFINITION OF ERROR 60 which society determines whether drugs should be marketed
PRINCIPAL SOURCES OF ERROR IN CUNICAL STUDIES 61 to the public. The randomized controlled trial does not.
REDUCING ERROR IN CLINICAL STUDIES......... 62
Selection Bias in Subjects Entering the Study 62
however, answer questions about the effects of the drug on
Selection Bias in Subjects after Entering the Study 65 the health of the population once the drug is marketed, nor on
PHARI\IACOECONOI\1IC STUDIES... 65 the financial consequences to the health care system of using
Sensitivity Analvsis .. .. 65
the drug. Two related disciplines have come into being to
EV ALUA TING PHARMACOECONOMIC STUDIES 68
REFERENCES 69 answer these questions: outcomes research and
pharmacoeconomics. Pharmacoeconomics, with which this
handbook is primarily concerned, is a subset of outcomes
research-that part that deals specifically with drugs as
opposed to health care services in general, and that includes
economic assessments. Pharmacoeconomics is a nexus for
ideas and methods from clinical epidemiology, economics,
decision analysis, psychometrics, statistics, and other
disciplines. Thus, what may appear to be simple accounting
rests on a complex set of ideas, some of which may be
standard practice in one discipline but of questionable
 6 7
validity from the viewpoint of another. Pharmacoeconomics
thus may be difficult for the non-specialist to understand and
viewed as controversial. Nevertheless, pharmacoeconornic
analysis has become a standard and sometimes required
element in the assessment of drugs.
The purpose of this handbook is to make pharmacoeconomics
accessible to the more general audience in the pharmaceutical
industry by explaining the concepts involved in a concise
form. This will help specialists communicate their work to
others and enable generalists both to understand
pharmacoeconomic studies and to assess their validity.
Definitions of Pharmacoeconomics
and Outcomes Research
Pharrnacoeconomics arises from a fusion of pharmacy and
economics. Economics is defined as "a social science
concerned chiefly with description and analysis of the
production, distribution, and consumption of goods and
services." In pharmacoeconornics the 'goods and services'
are pharmaceutical products and services, and it is the effects
of their consumption that is the focus of interest. Thus, we
Mern.un-Webstcrs Collegiate Dictionary, Tenth Edition.
may define pharmacoeconomics as: a social science
concerned with the description and analysis of the costs of
pharmaceutical products and services and their impact on
individuals, health care systems, and society.
Pharmacoeconomics is a subset of health economics. which
deals with health care services in general rather than being
restricted specifically to pharmaceuticals.
Because pharmacoeconornics is a social science suh-tantially
concerned with events in clinical practice, it overlaps with a
branch of medicine called outcomes research. OutCl)l11CS
research is the study of the clinical (e.g .. presence or disease').
economic, or humanistic (e.g., patient quality or life) end
results (,outcomes') of providing health care services.
Pharmacoeconornics is that subset of outcomes research that
deals with pharmaceuticals and includes economic outcomes.
Types of Pharmacoeconomic
Evaluation
There are essentially two kinds of health economic .malvsis:
cost analysis and cost -outcomes (or cost -conseq Lienee)
analysis. In cost analysis, only the costs of providing health
care products or services are considered, without regard to the
outcomes experienced by the patient or providers. In a cost-
outcomes analysis, the endpoint of the analysis is a ratio of
 8 9

the costs of providing health care and a measure of the etc.-and the dollar costs of providing care to a given patient
outcomes of the care. The main types of analysis are listed in population over a given time period. The outcomes resulting
Table 1. from the care are not considered.

Table 1. Common Pharmacoeconomic Methodologies

Cost of Illness and Burden of Illness
Method of Analysis Cost Measure Outcome Measure
A cost of illness analysis normally falls under the umbrella of
Cost Analysis
outcomes research rather than of pharmacoeconomics. In a
Cost-of-Care Currency N/A classical cost of illness analysis, the total cost that a particular
Cost-Outcomes Analysis disease imposes on society is expressed as a single dollar
amount. Included in the calculation might be the costs of
Cost -Effecti veness Currency Natural units (e.g .. life-years
saved)
providing care for the illness (including drug therapy), the
value of the lost productivity, and the monetary cost to
Cost-Utility Currency Quality-adjusted life years or society of premature death. Classical cost of illness analysis
other utility
has metamorphosed in recent years into the burden of illness
Cost-Benefit Currency Currency analysis, which in essence is the same thing except that the
Cost -Minimization Currency Natural units or utility emphasis is placed on the more tangible component costs
rather than on an aggregate dollar figure. Thus, the total
N/A, not assessed. direct medical costs of treating an illness, the number of
deaths, hospitalizations, lost work days, etc., are the variables
COST ANALYSIS of interest in a burden of illness analysis. The most infamous
misuse of cost of illness and burden of illness analysis is to be
found in the opening paragraph of many a medical economic
Cost of Care article, where future projections of the societal impact of the
disease in question are deli vered for their rhetorical effect.
A cost of care analysis is an enumeration of the health care
resources consumed-in this case drugs, pharmacy services,
 10 11

COST-OUTCOMES ANALYSIS with statins (i.e., the drug costs) versus the effects of statiu
treatment (expressed as percent reduction in LDL-C l. The
The different methodologies for cost-outcomes analysis are points represent different statins and different dosuge«. The
essentially similar in that the endpoint is a ratio of the costs line connecting those points representing the lowest cost at
and outcomes: they differ in the way the outcomes are any given effectiveness describes the 'efficient frontier.' In
expressed (Table 1). Figure I. t1uvastatin and atorvastatin are the only two statins
on the efficient frontier-t1uvastatin at lesser and atorvastatin
Cost-Effectiveness at greater effectiveness.

4000 r--
Cost-effectiveness analysis compares two (or more)
alternative treatments for a given condition in terms of their
L50
monetary costs per unit of effectiveness. The unit of
(jJ 3000 r-
•
effectiveness can be any 'natural" unit-e.g .. percent ::)
lowering of LDL-C. major coronary events. number of lives ~ •
u;
o
saved. or years of life saved. The units of cost (currency and o
OJ 2000 L
year) and effectiveness must be the same for the treatments ;:> .
o L-1C

compared. Cost-effectiveness analysis is used to decide ro • 520 .s-w

among two or more treatment options. The definition of
:J
C
••
P40
• ;".1C

'cost -effecti veness ' is discussed in more detail below.

C
« 1000
P2. ••
L20 •
1\2(i

F20·
"10
-- - --...--
S10
• Al0

F40

The Ejjicien t F roll tie r o

20 40 60
Figure 1_Annual Cost of Starin- versus Percent Reduction 111LDL-C Se,tltc"1 plll\
The cost-effectiveness ratio may be given as a single number,
using data of Hillcman et al. II The line connects drugs and do.sagl's that lie' on the
but it may be more illuminating to present cost-effectiveness 'efficient frontier' of least cost fur any degree of cffedl\ c·ne'."_ A. .uorv :1S\atIlL 1-.
fluvasratin: L. lov.istatin: S. <imvast.uin: and P. prava-t.uin. :\ I(). ,11,)l"\asl,111I\ lilll1~:
data graphically as a plot of costs versus effects. Figure 1
lete) .
.-.;110\\S a plot of the costs of treating hypercholesterolemia
 12
Cost-Utility
A cost-utility analysis is performed in the same way as a cost-
effectiveness analysis except that the unit of effectiveness is
quality-adjusted life years (QAL Ys) or another measure of
utility. Consider that the outcome of a treatment may be a
prolonged life but with a degree of disability, or a reduced
probability of disability without prolongation of life. The
value or 'utility' that individuals or society place on different
life outcomes can be quantified using a number of techniques.
Since the endpoint is in practice always expressed as cost per
quality-adjusted life-year saved, cost-utility analysis can, in
principle, be used to compare not just alternative therapies for
the same disease but therapies for different diseases, and
rankings of the cost-utilities can be drawn up. Such rankings
can be useful in selecting policies when, for example, a
government wants to choose among installing highway guard
rails, hiring additional food inspectors, or vaccinating seniors
for flu.
Cost-Minimization
A cost-minimization analysis is a cost-effectiveness analysis
in the special case in which the effectiveness of the
treatments is the same. Once the effectiveness (expressed in
whatever natural units are appropriate) has been determined
to be equivalent for the alternative treatments, it is not
13
considered further and the analysis focuses entirely on the
costs, with the aim of determining which treatment minimizes
costs. A cost-minimization analysis is, in effect. a cost-of-
care analysis in which alternative treatments are compared.
Unlike a true cost-of-care analysis, however, the outcomes
are taken into account and must be shown to be equivalent.
Cost-Benefit
Like cost-effectiveness analysis, cost-benefit analysis
compares the costs and outcomes of alternative therapies:
unlike cost-effectiveness analysis, however. the outcomes in
a cost-benefit analysis are expressed in monetary terms. For
example, the outcome of the treatment in question is first
expressed in terms of life-years saved or quality-adjusted life
years saved, and this is then translated into an equivalent
monetary amount-under the human capital approach, this
amount is the present value of a person's lifetime
productivity. Since both the costs and the effects of the
treatment are expressed in the same (monetary) units, they
can be directly compared. Any cost-benefit ratio of less than
1.0 is cost-beneficial. A ratio of 1:6 means that one receives
$6 of value for $1 of investment.
 15
14

The basis of decision analysis is the decision tree. figure 2

Basic Concepts illustrates the components of a decision tree: nodes (decision.
chance, and terminal) and branches. A series of chance nodes
PHARMACOECONOMICS and branches connect a decision node with terminal nodes.
which represent the outcomes of interest in the analysis. The
tree is structured from left to right. The tree in Figure '2
Decision Analysis begins with a decision node and two branches representing
alternative courses of action. i.e .. to use either a new drug or
Decision analysis provides the basic framework for cost- standard care to treat disease X. Both courses lead to a
effect: veness analysis, which is the most common type of chance node that diverges into branches representing the
ph.urnacoeconomic analysis. Decision analysis is a possible outcomes of survival or death following treatment
systematic. quantitative approach to assessing the relative These branches end in terminal nodes. representing the
value of one or more alternatives. outcomes of interest in this decision analysis. i.e .. life or
death. Chance nodes identify points at which t \\0 (or more)
possible events may occur. Which event will occur cannot be
Live
predicted with certainty. and so the chance nodes are
,
Use new' d111g.J
f "
n :::
..•
<] l·!.J.lr.,,~e
associated with a probability for each emergent branch. In
Use new d.lllg l' Die
----] [·I··d this case. the probability of survival following treatment with
r s t andard C ale ';,
(I
0.2
.... .!:';j

the new drug is 0.8 and the probability of death is 0.2; these
LIve probabilities must sum to unity and the branches exiting the
I
Use s t anderd Cale
(
J",
n "
•. _'
<] i·Jive chance node must exhaust the possible outcomes. Following
1 DIe ..
<. .] ['1.!:'. ad
standard care, the probabilities of surviving and dying are
0.5 ..••.
both 0.5.

Fi~urt' ~. Hvpothcticu! Decision Tree. The tree consists of branches (lines) and
In this explanatory example, it is easy to see that the new
nodes ;\ dccrxinu node (square I. chance nodes (circular i, and terminal nodes drug is superior to standard care in terms of the number of
tri.myular.
1"

surviving patients.
 16 17

Definition of Cost-Effectiveness lives. The incremental cost-effectiveness ratio is thus $700

Decision trees such as the hypothetical example shown in
Figure 2 are a basic step in cost-effectiveness analysis.
Suppose that in the example shown in Figure 2 the cost of
providing the new drug therapy to 100 patients was $1,000.
This includes the cost of the new drug and the cost the
physician's services for diagnosing the condition and
prescribing the treatment. Since 80 of the 100 patients given
the new drug lived. the cost-effectiveness ratio is $1,000
di vided by 80. or $12.S per life saved. This ratio is referred
to as the average cost-effectiveness ratio.
The cost-effectiveness ratio of interest is not the average cost-
effectiveness ratio but the incremental cost-effectiveness ratio
of the new drug relative to standard care. Suppose that, in the
divided by 30, or $23.33 per life saved.
The incremental cost-effecti veness, and not the average cost-
effectiveness, is calculated because there is always an
alternative to the new drug or whatever therapy is in question.
Even if the alternative is literally to do nothing. there are
associated costs and/ or effects. Suppose that, in the example
we are discussing, if literally nothing is done to treat 100
patients with disease X, then 70 patients die and 30
spontaneously recover and survive. There are no associated
medical costs because no treatment is provided. This is
illustrated in Figure 3.
Live
JA~
Use new dmg
r-----------<I
JI (I :3

example shown in Figure 2, the cost of providing standard 1 Die

0_]
J De~
care to 100 patients was $300. Standard care is thus less
Use new drug, standard Live
costly than the new drug, but also less effective. The ~A~
incremental cost-effectiveness of the new drug relative to care, or do nothini' Use standard care J I
(1.5

standard care is the difference in costs divided by the

1 Dle
1 De~
0.5
difference in effects.
Live
~A~
C /E = G!-Cb Do nothing J (1.3
fu-8J I I I
1 [He
~ De~
In this case, the difference in costs is $1000 minus $300, or (17

$700. and the difference in effects is 80 minus SO lives, or 30 Figure 3. Hypothetical DeCISIOn Tree with Three Alternauves [\,,'I1:t the' ailcT1LlliI

is literally to do nothing. the outcomes of interest accrue.

 18 19

Table 2. Definitions of Pharmacoeconornic Cosh

The incremental cost-effectiveness of the new drug relative to Term Definition

no treatment is:
Average cost Total cost divided bv the number of unrt , produced

($1,000 - $0)/(80 - 30) == $20.00 per life saved Direct cost The cost of the gc,ods and service thai art' ue-ed In
providing a treatment
Similarly, the incremental cost-effectiveness of standard
Incremental cost The increased cost of one treatment pr"t:r~lJn relative II'
therapy relative to no treatment is $IS.OO per life saved.
an alternative
Relative to no treatment. the standard therapy for disease X is
Indirect cost The value of the product i . it) Ill" rcsultin" Irorn :111
less effective but more cost-effective than the new drug.
illness

Intangible cost The value of psychosocial e1lech ,uch as p:l1n and

Definitions of Costs suffering.

l\larginal cost Change in total Cllst that results trorn the production (11
The definition" of some cost terms commonly used in an additional unit
pharmacocconornics are given in Table 2.
Mortality cost The cost incurred due to death
Economists distinguish average cost from marginal cost. Opportunity cost The value of all cpslS in an alternative lIse

EXAMPLE. The average cost of detecting a case of Overhead cost The coxt of pr()\'idin~ space. PU\\l'l. :ldlllll1i,tr:I\I\('
condition Y is the total cost of all the screening tests services, etc.

performed divided by the number of true positive cases of Production cost The total amount of resources used in producinu
Y detected, If the screen only uncovers 7S% of cases of something

condition Y and a program is applied to increase this to Productivity cost Same as indirect cost
85%, the cost of that program would be the marginal cost
of increasing the detection rate by 10%, cost-effectiveness, The incremental cost is the increased cU:-,l
of one health care program relative to an alternative. The
incremental cost differs from the marginal cost in that the
In practice the marginal cost is usually greater than the former relates to treatment alternatives while the latter refers
average cost. The distinction between average and to more of the same treatment.
incremental costs was discussed above under the definition of
 20 21

Total costs = Direct costs + Indirect c'(hh t· Il1t:ll1!Cihk L'(l.'h

The distinction between direct and indirect costs is
particularly relevant to cost-of-illness analysis. The direct
cost of an illness, such as asthma, to society is the cost of
providing all of the health care services to treat it, including
the costs of medicines, physician visits, emergency room
visits. and hospitalizations due to asthma. The indirect cost
to society is the value of the productivity lost when asthma
prevents people from working.
Most direct costs considered in phannacoeconomc analysis
are direct medical costs-the costs of physician visits.
hospitalizations. laboratory tests. drugs. and medical supplies
and equipment. Non-medical costs include a variety of out-
of-pocket expenses. such as transportation to health care
facilities. special foods, etc.
Direct costs = Direct medical COsts + Direct non-medical
Indirect costs. i.e .. productivity costs. arise from morbidity-
when people miss work (absenteeism) or are less productive
while at work (presenteeism)-and mortality.
Indirect costs = Morbidity costs + Mortality costs
In computing total costs, researchers may include not only the
direct and indirect costs but also intangible costs. IntangiblC'
costs include the value placed on pain and suffering.
Perspectives
Since costs are seen differently from different point« of vie'\\'.
the perspective of any pharmacocconomic ~lllaly..,is must be
explicitly stated. Society. health care insurers (payers!. and
health care providers all have different perspect: ve s on Cosh.
Furthermore. the organization of health care financing. and
therefore the appropriate cost perspective. \ urie-. from
country to country and within countries such as the United
States that have mixed systems. The usual pl'l\r)t~\.:ti\es III
cost-effectiveness anatvsis arc those of S(h:ii...'\\ .md those ul
the program.
EXAMPLE. A science writer with coronarx heart disease
costs
is hospitalized following an acute myocardial infarction
that subsequently proves fatal. From the perspective of
the science writer's health care insurer (the payer). the
cost of hospitalization is the amount or money paid to the
hospital under the terms of the health plan. From the
perspective of the provider (the hospital). the cost is the
true cost of providing the hospital services. From the
perspective of the science writer's employer. which (ill
this fictional example) entirely subsidizes its employees'
health plans. the cost is that part of the iI1S11!dllCepremium
designed to cover coronary heart disease. pillS the i ndircc:
 22
costs. i.e .. the cost of the productivity lost while the
science writer was incapacitated, and the cost of hiring a
replacement.
Time Horizon
The term 'time horizon' is used to specify a period of time
during which the outcomes of an analysis will be considered.
The time horizon could be expressed as a fixed number of
years (or months or weeks) or relative to study variables (e.g.,
patients' lifetimes, or the amount of time that patients were
enrollee! in a clinical trial). More precisely. the time horizon
is :1 point in the future up to which all costs and effects must
bc accounted for and bevond which evervthinz can be
o -' L
ignored.
EXAMPLE. A cost-effectiveness analysis of the ACE
inhibitor enalapril based on data from the SOL VD trials
considered two time horizons: a within-trial horizon, and
a patients' lifetimes' horizon,6 The within-trial horizon
referred to the actual time period of the SOL YO trial.
Because of staggered enrollment and deaths during the
trial, the durations of enrollment varied up to the
maximum duration of the trial (about five years), The
within-trial analysis relied on observed data of the
effectiveness of enalapril (cost data were inferred), The
23
lifetime analysis was a projection of events into the
future.
Discounting
If the time horizon of a pharrnacoeconomic analvsis is several
weeks or months, no adjustment for changes in costs over
time is required. If the time horizon is several Yl'ars.
however, then costs that are incurred at different time : mu-t
be brought to same reference time point.
The value of a dollar todav. is not what it \\;h ''--:1\.
. 1\\Cnt\
years ago. For example. the cost of a drug in ll)~) muxt he'
expressed in dollars for the current year (or the rdCTl'l1CC y\.'~tr
of the analysis) by increasing it according to the annual
inflation rate for pharmaceuticals, Conversely. the value of a
dollar twenty years from now will be less than it:-;present
value. In order to bring future costs to the same frame of
reference as present costs, they must be discounted.
EXAMPLE. The cost of a medical service i:-; ')1.0()O, The
medical service will be utilized five veal'S from now: what
is its present value? The formula for discounting prices
into the future is:
Cll
Cprl'SCll!- (1 + r )n
 24 25

where cpresentis the current cost, n is the number of years, c, is EXAMPLE. National survey data for 1994 indicate that
the cost n years from now, and r is the discount rate. If the 49% of all pregnancies were unintended: 54(7(, of the
lo
discount rate is 5% per annum, the present cost of the medical unintended pregnancies ended in abortion. About half
service IS: of the women who unintentionally became pregnant had
been using a regular method of contraception.
CprCSCIll = I ,(XX) Emergency contraception can prevent pregnancy if taken
(1 + 0.05)" = $784 within 72 hours of unprotected sex. We can explore the
consequences of a decision whether or not to use
emergency contraception using decision analysis. If
MODELING FRAMEWORKS emergency contraception is used. the probability of
pregnancy is reduced (but not eliminated). I r pregnancy
does occur, a predictable proportion of women will chose
Steps in Decision Analysis
to terminate the pregnancy. Some women who continue
While the process may be broken down in a number of their pregnancies will miscarry. For the sake of
different ways. we will follow previous authors and describe simplicity, we shall ignore the effects of nausea following
a decision analysis in terms of five steps.i' the Lise of emergency contraception. and complications
such as ectopic pregnancies.
I. Identify and bound the problem

2. Construct a decision tree I. Identify and bound the problem

3. Collect the information to fill the decision
The first step in a decision analysis is to identify the
tree
alternative courses of action. In the example we are using.
4. Analyze the decision tree the decision is whether or not to use emergency contraception
following unprotected sex. The consequences of this
5. Conduct a sensitivity analysis decision that interest us are the numbers of unwanted
pregnancies, or more specifically the number of pregnancy
terminations and live births that would he avoided through
 26 27

Lise of emergency contraception. The endpoints of the Following the use (or not) of emergency contraception,
analysis, therefore, are pregnancy terminations and live pregnancy mayor may not occur. A chance node reflecting
births. The time horizon will be limited to an episode of these alternative outcomes is added to each branch emanating
unprotected sex and its unintended consequences, i.e., nine from the decision node (Figure 5).
months. The perspective is that of society.

'l. Construct a decision tree

Plegn;:U:lt
Step 2, construction of a decision tree, makes the description
I
Use Eep
I. J
J
of the problem and its elements explicit. The tree begins with -1 Not pl'e:?~'l;mt
the decision node and branches representing the alternative U nprotected sex
courses of action. Here, the decision is to use O[ not to use Pregn;mt
emergency contraception following unprotected sex (Figure
-1-). I
Do not use ECP
"1
J
Not pl'epmLt
i.

Use Eep Figure 5. Partial Decision TrL": with Chance Nudes. Chance IWclL'., reflect the
likelihood of pregnancy following unprotected sn.
U nprotected sex
I
Do not use EC:P
If pregnancy occurs, some women opt for termination and
others to continue their pregnancy to term. While for an
individual woman this is a decision that must be made, from
F'ilurc 4. Partial dc'chlnn tree wilh decision node. The decision is whether or not to
lI",'l'n1L'ri,'cnc\ c{ln!r;llcplioll i'l)lIowing unprotected sex. [CPo cmer)lL'nc) the perspective of an observer of a population of women, a
':\ lIll1acl']JII vc 1''' I measurable proportion of women will chose one option over
the other. This proportion might vary according to the
 28 29

composition of the population of women and other factors. 3. Collect the information to fill the decision trc«
The node branching to either pregnancy termination or
continuation is thus a chance node. In the case of the decision tree in Figure 6. the information
sought is a probability value for each chance node. The
A certain proportion of women continuing their pregnancies
probability estimates are displayed beneath the branches of
undergo spontaneous abortion: this is also reflected in a
the decision tree, as seen in Figure 7.
chance node. The branches now in the model lead to the
endpoints that were decided on in Step I-pregnancy
termination, live birth, and no pregnancy. The last step in
creating the decision tree, therefore, is to add the terminal
lnduced .~bl:IIlll:':1
nodes (Figure 6). The decision tree describing the problem ; ~l J: c: .,~"
.::-Ir'
r'rr~\:illt
we identified and bounded in Step I is now complete. .;p'~;~.:'':..:\i'.·'I:';; ~,:<,!',l(!,
1! (I!~;
Conunue iJI':g1\."1r~:y
-..-..~,---
;,;.:.
--- .'1.
".' P,t ;;!l.:",,"J

Use ECP .; ~4 ~:':~!~tUdJ'" pl~~>llll::~'

1:' ',:,'}l~:::

h'""",,,~r---------..Sf~::'=
Induced abcruon

r
Terminancn
I ~ Not pre-gni.!.Itt
H))·:.·,'.1.,,:.'"'.;
N..:, pI>::?:nal\cy ! 09J1
! f Conunue pregnancv .~
tic< E·.:
I -- ~::ut F't~g!~v:~_
~npwttct~d :;f.:~ r-!!.':~~~~~
;.~~~~~~.
.._._...
_....._ _
.._.

" I
I~I~€:nl\1\t
--------
Lrve bu-l.

No pregnancy
l[0M' de I e,,::::~, L'"'": ;,"".~~~:;~
I
eM

!-); ',~';,t ,,!..!,," '.r

':.t_I_IJ.;,~~~~d ~~•~ ~jl

. _[' lu·1Ul •.d ebortror
71tgll-!int I,----.-- ------ -------- -- - Te IT:1.ll1.3.tIon L:-'."I;' 1:,u1i":

:
!I
~
r

"I 'j!;" Ed'

I----" I ,:;ponlar~o"s
t. onnnue preV'"'''' r--'--' .--.:~~
,I," 'C
N,) pp-gnoirl.CY LN:~I'!"~_='
(193125
. :':_. .. !':,:,pJ"-Pt<lll':'/

I
~ i ...•
Lon'",'", pre;;n""";!
Lrve buth

1 NOI pregTlanl
--_. __
._------._-- 1<0p"gw.ru:y

I:igure h. Complete Decision Tree. Complete tree: for the decision of whether to use
"1l1ergenc\ counaccpuon following unprotected sex Figure 7. Decision Tree with Probabilitie s. The decisi..n tree sht",n III th'_' 1'1'," \1'1:'

figure \ Figure h.l with probabilitie« added.

 31
30
2
o
2
:;
2
c
2
0
tr
rj
m
,'-'" ,---
en :-r
--
=
;:s
-:
8 5e.. ~
::0 co rn CT1 rn rn '7

In Figure 7, the probabilities of conception with and without

q
::.,:J
g~~p
c
~
,<
(/)
!) C) !) C)
g' G
'JJ

emergency contraception are taken from a clinical trial of

"U

g. § ~ , w
»
-
....J (l)
CL ~
(D
~ _ _ _ '1TO
;:;3
C)8 C
emergency contraception versus a control group without 00.. '-0 0 8 8
---.-.-----:::
-c ::: :=.: ::: :.; (--<
ex:; "--' --' ---' tv --.J ~ --J ,::I J
emergency contraception. The probabilities of induced and ;l'
~ ~ _. ..-
t-0
x'
--J
ex
-.J
'x;
--J
'JI 'JI 'Jl 'JI i r: J.
v; ~. 'Jl ',J! fJ\ 'J! ~ ~
....,..,
spontaneous abortion are obtained from state statistics." (A c

more extensive description of information sources is given ~-

-l::O
-, I[ H
.....,
I;:;
.:t;' g (l;;

under 'Data Sources.')

-;
2:.~ - -
,-,'

'J)
'-'

1Jl.
~
.......-
~
-
,::;,
~
::;,
-
;.:. II
'"
~
r:
J:

~ ~ •..• .j.... 0'. .j.... .j.... :J' .;

o
:;' ::i g- -'
-::0 '"'
"".
4. Analv;« the decision tree """' ""Tj

g'' '"0 ,-
~!2'-' 2-...•2.-
rro 'J:
c:: C :J G
J; ri
We will analyze the tree by calculating the probability of ::r-....]
n __ M
r. 12 § :3r;
reaching the outcome represented by each of the terminal
'f.

r (D
i::t ~
nodes. This is done by tracing the branches from each
::

--::::...,
r-, -
("101

rt
/.
~
---J
--..-1
-
t,-->
Jj
-
-....]
-...j
-
1...1 I::
'f
iv:
r,
r-
v ;;..
terminal node backwards to the beginning of the tree: the C '<
JC/) r-.
probabilities along these branches are multiplied together to ~
produce the probability of the outcome. These calculations
~5
;::;.
0
~'~'
can be performed using a spreadsheet. The spreadsheet has a ., I ~
o 0 G

row for each terminal node and a column for each chance
node plus a column for the calculated probability of the
-'0'
::0

.",o...,
~'

0
:::;8
_.;::;.
8 18 8 8
~
~
~
8
N WOW
:J'
-
0
~
0
0
.j....
o
.
~ ~ §
0'.
1'8 8 38 3::: 8
I
..

--J \:0 X
-
~
C
::
?

(l) (l) -....]

8 N W ~ ..p... '-J) o: 'J.,J -::J". ;;
outcome. r:ro VJ
0

\2 ,.....2
::0
C) """+-,
::0 v
() ..., V1 -i
The spreadsheet corresponding to the decision analysis ll1 '< C~~C:2;:J
21 ~~ ~ ~
fD
.')'0
;::' 7' c ~ ~ 3 r,
Figure 7 is shown in Table 3. The probability of an induced CT1s ~ (;;~ @ ~ ("';; .....• f"D
::J

abortion if emergency contraception is used (corresponding

no
. v
~ ..:
~
:::: JQ
:::
:::;
-'
(To
:::
~
•.... -
C
-'
JQ
- ~
~
:::
("i)
nQ co
:::; :: [ 5 g I~
to the top row of Table 3 and the uppermost branch line of
~ ..,
::0 (l)
n.......•
.., :j
~ in
~-<'
'Y
5" '<
r;
I~ [..o
l~' C2.
r: ::0

tv: c. :::; Ie..

Figure 7) is 0.075 x 0.46 = 0.0086. (Note that there are blank o
re,
~ :: ~r~ I
cells in the table where a particular chance node does not '-<'. ::

'J;
)~

2:
<
n
I
,

r~.
~
!

Iifili5'Kii35!ft3'1
 32 33

occur along the branch line.) The outcome for the second and 5. Conduct a sensitivitv . analvsis
.

fourth rows of Table 3 is the same ('No pregnancy') and the

probabilities are added together: the probability of no Any measurement should be expressed in terms of a point
pregnancy if emergency contraception is used is 0.0023 + estimate and an indication of its reliability. For instance. in
0.9813 = 0.9836. Similarly, the probability of 'No descriptive statistics a mean (point estimate) and range.
pregnancy" if emergency contraception is not used is: 0.0093 standard deviation, or 95'1t confidence interval rnav be
+ 0.9250 = 0.9343. The probability of an unwanted provided. The decision analysis described "bu\"l' has yielded
pregnancy (sum of pregnancy terminations and unplanned a point estimate of the number of unintended pregn.mcies
births) is 0.0164 if emergency contraception is usee! and prevented by emergency contraception. The reliability of
(U)657 if it is not (Table 3). such a point estimate is made difficult to calculate by the
(usually) large number of probabilities involved in the model.
The consequences. in terms of induced abortions and A point estimate was used for each of the probabilities in the
unplanned births. of the decision to LIse or not to use model, but of course there is a range of likcl , values for each
emergency' contraception for a hypothetical population of of the probabilities. Sensitivity analysis determines the effect
10.000 women are shown in Table 4. The use of emergency on the result of varying the probability estimates through th-
contraception v..ould prevent 259 induced abortions and 234 range of their possible or likely values. In a (111e\\ ay
unplanned births per 10.oon women who had had unprotected sensitivity analysis. the probabilities at each chance node in
sex. the decision tree are varied across their range of values one at
a time. This process determines the scnsitivirv of the results
Table 4. Outcomes of Decision to Use Emergency to changes in the assumptions in the moue! ~lnu can identi fy
Contraception per 10,000 Women the most critical assumptions in the model. i.c., those that
have the greatest effect on the results. The following is an
example of one-way sensitivity analysis. Two-way and
Induced Abortions Unplanned Births
three-way and other forms of sensitivity analysis are
EC 86 78 discussed below.
:\0 EC .\45 ."112
EXAMPLE. In our decision analysis of the use of
Difference --259 .. 2)..+ emergency contraception, the point estimate of the

EC. emergency contraception

 35
34

abortion. The decision, chance elements. and outcome are

probability of spontaneous abortion was 0.23. The range
presented as a square, circles, and a lozenge shape.
of values for this probability is 0.17-0.29.15 Inspection of
respectively.
the decision tree shows that changing the probability of
spontaneous abortion does not affect the number of
induced abortions but does affect the number of live
births. Substituting first the upper limit estimate (0.29)
and then the lower limit estimate (0.17) for the value 0.23 ECP
used in the initial calculation, we find that the difference
in the number of unplanned births (without emergency
Induced
contraception minus with emergency contraception)
abort 1011
varies between 216 and 252 (the point estimate was 234).
The spontaneous abortion rate, thus. does not critically
affect the reduction in the number of unplanned births
attributable to the use of emergency contraception.

Influence Diagrams

It is sometimes usefu I to draw an influence diagram before

constructing a detailed decision tree. An influence diagram
makes specific the decision to be taken, the outcome of
interest, and the chance clements that influence the outcome.
Figure 8 shows an influence diagram corresponding to the Figure 8. Influence Diagram. Diagram representing the CkCl,l\)11 clcrncnt and cli:l11cl'
clements influencing the outcome of unwanted birth rep. el1lcr~en(\ c\l[llrac'cl'till'
decision tree in Figure 7. The only outcome of interest in pill
Figure 8 is live births following unprotected sex (induced
abortion was also an endpoint in the decision analysis shown
in Figure 8 J. The outcome is affected by the chance
occurrences of pregnancy, induced abortion. and spontaneous
 36 37

Markov Models 'dead' state at the end of each cycle. Needless to ~~ly,
individuals cannot exit the dead state. Probubiluic-. must hI.'
The decision analysis shown in Figure 7 represents a single.
assigned for each transition, Since time is modeled Cis ,1
linear chain of C\l'nts tran"piring over ,I single time period, series of cycles of equal length. thl' probabilitie-, can hl'
SOl1ll' disl'asL's, however. progrl'ss gradually over a period of different at each cycle. so that thc , can he 111'Ii..kd"jWIH.il'nt ()11
Years. while the rixk of the outcome of interest. for instance.
the age of individual- entering the model. l\brK()\ IllUdl'ls ill
coronary death. increases with age, \1<lrKO\ analysi • is which probabilities arc time dependent arl' l,~t1kd \LtrK(I\
appropriate for such problems,
process models,
Markov analyses use tree diagrams similar to those used ill
simple decision analysis. However, the elements of the
problem arc first mapped out in a Markov diagram similar to
an influence diagram, Figure l) shows a Marko: diagram
representing. the progression or congestive heart failure, The
Markr» model consists of states (ovals) and transitions "Y1L·\·I,1I <,
(aITO\\S), III Figure <J. there are four stales: well, early stage
heart failure. late stage heart failure, and dead from heart
failure. where early and late stage heart failure are
~~

NYH:\.III'IV~'
»>:
~C-~~"

represented by New York Heart Association (N'{HA) class

Ill! and IIIIIV, respectively, Time i~ broken down into a
series of sequential periods or cycles: within each cycle, an Q
individual must be in one of the four states; transitions
between the states occur at the end of each cycle, Individuals
in the 'well" state can transition into either of the NYHA
CLISS states or remain in the well state (represented by an
dlT()\\ e.\iting from and circling buck into the well state), ( Fiplrc I). :'\lark\)\ Tr.msiuon Stale Dia;,:r:1Il11I>II':!li"lIl.' 111111
"i:l,:r<l111 h:ls,'J Oil Stlll!\ h; xau Hout cI,d:- [','I,illll'lkil\
I k,liI liulur. \I.II~'"
\YI1..I. ·1:1"1."1:111<111
1i:IIT h,','1!combined. as h:I\L'1\YIL\ eLlss,', III :llld 1\' j\;YII.\. 01'11York lle',l:1
Similarlv. individuals in the N't'H,-\-UII stall' (:111 remain in \ '\,sl,,'I,llillll,·l;hS 1,,1' hc.ut l.ulur ,'
that state. progre."s into the NYHA-IfI/I\/ state. or enter the
 38 39

CLINICAL EPIDEMIOLOGY study. the investigator merely observes events al1d dues 110t
interfere with them. There cannot be. therefore. :111
Pharmacocconomic studies usually require data both on costs 'observational trial' Second. the events heing studied m:ly
and cffecti veuess. The effecti veness data arc taken from have alreudv . occurred. in which case the study. is :1
epidemiological or medical research studies. The design of retrospective one. or may nut yet h:I\'C occurred. in which
every medical research study can be classified according to a case they will be studied as they happen. i.c .. prospcct ivc ly
t'C\\ fundamental mutual" exclusive dichotomies (see Table An observational study may be either rctroxpcctiv« ()1'
.5 l. prospective hut an experimental study can only be
prospective-unless. that is. the investigator possesses a time
machine in which he visits the past to perform his
intervention before returning to the present lo colk\'[ the
Table 5. Concepts in Cliniral Study Design results. Third. the observations ma~ refer to a poin: in time.
l-' Sru dv. De~i(r!l and are called cross-sectional. or to two or more points in
h

:,1:
time. and are then called longitudinal. All observational
In\-i.'ql~Jtl ~ I: Ii:,\: " l'('rlI11L':-:
II"
i study may be longitudinal or cross-sectional and in both case'>
In' I I: \" .: ~I ~ r: !1-_. .._.

Il P~-i"p ..:t-rL"i.'
may be either prospective or retrospective. An experimeutal
lim« I\T';'\.·(il\·~· !)rl)'IH.:Cll\'l' I\Ltr."p~·\·ti\t·
study, however, Gill only be longitudinal because. however
1.( Jfl:..,l.l! lI\..\[r:~d 1.< >Ih!,;!~illll1.1j \ ,I' I..1; 1.L:il"1.l\.hn"~
T: ~~L· :-J::~l'lilj..! I q"
brief in duration it might be. it is an analysis of CIUSC and
c ~.(''''...."~'~. !"]1. I n.i j j"';" . :" VI ! tl
i' \~"1.
effect-the 'cause' being the intervention being tested. and
the 'effect' being the clinical outcome being ohsenl'd-
which cannot by definition occur at the same point ill time.
Thus. a clinical trial is a prospective, longitudinal.
First. a study may be either observational or experimental. .A experimental study.
clinical trial is an experiment-a test of an intervention-s-in
which the investigator interferes with the normal course of
l'\l'Jlts. usually by providing certain people with a treatment
they would nut otherwise have received. In an observational
 40 41

STATISTICAL ANALYSIS UTILITY

Clu-.xical statislics is based on hypothesis testing. The The effectiveness of many medical treatments C;\I1 he
hypothesi" is made that the observations to be cxpluincd are expressed in terms of prolongation of life. c.g .. :\S the
the result purc l. of chancc-e-thix is the null hypothesis. A (average) number of years of life s(\\l'd. SUIl1\' trc.umcntx.
calculation is then made of the probahility that the however. may prevent a worsening ill the quality tll life
observations would arise under the null hypothesis and. if th.u without actually extending it. Similarly.;\ trc.umcm may
probability is below an arbitrary threshold (1110-;t often I in extend life hut with the presence of significant disahility that
2(). or O.(5). ihc null hypothesis is rejected. The results reduces the quality of life. These sitLutioll" ;11\' dc:alt with h ,
cannot be explained purely by chance and are said to he placing a value on the quality of life. i.c .. if'.; ui ilii .
"-;uti-;tic;tlly significant." The utility of normal health is given a value of I.n. while the
\otL' that the above procedure explores the role of random utility of not being alive is set at 0: a <tate of rcduc.d health
chance and in it-elf docs nothing to assl'S-; the role of has a value between () and 1.0. This utility (l') is multiplied
,,~sicmatic error {discussed belowi. which is often more by the number of years of life (Y) saved by till' trc.nment in
important than random error. Hypothesis testing is not order to arrive at the number of quality-adjusted life' years
parucularly useful in decision analysis. where we need to (QALYs).
KIlO\\ the prohahility of a certain event occurring (such as
QALY =Y x li
death from a myocardial infarction) under a certain set of
,-:lrCLlmst~lI1cl'" 1 such as when a patient has already had one A related concept is the disability-adjusted life year I. D""\L Y:
heart auack l. The calculation of such conditional The DALY was developed to quantify the burden t)f disease
probabilities i" referred to as Bayesian analyxis. To the non- and injury on societies (as in the Global Burden of Discasl'
preconditioned mind. the Bayesian approach may be more Studyl7) and represents the reduction in the 1111111hl'lof years
intuitive, if k"" conceptually sophisticated, than hypothesi" of life due to disease. weighted hy the quality (lr lif.. due in
tl' -;till g. the presence of the disease.

Utilitv values fur QALY culcul.u ions call he f(llind III th«
medical literature. A detailed description of' 11\1\1,ulility IS
 42 43

determined i~ beyond the scope of this primer. There arc and an a\erage score for the response s to all the items ill the
cxxcntially two way». First, one can determine people." s domain can be computed.
preferences Ior different health states by methods based on
EXAMPLE. A domain
in an instrument l'tlillains three
decision analysis using techniques such as the standard
items, each with the response options of ye~ or no, "cored
~amble or the time trade-off. Secondly, the utility of a
as I or 0, respectively. The maximum SC\ll'L' for the
particular -r.uc of health can be estimated using psychometric
domain is 3 and the minimum xcore (I lutcuucdi.u.'
<calc-. scores of I or 2 arc possible.

PSYCHOMETRICS

The rucu-.urcmcut of humanistic endpoints, ~Ud1 as quality or Table 6. The Elements of Ouextionn.urc-.
lite. in medicine is based on questionnaires. While a simple
Element Description
questionnaire might collect descriptive data xuch as the ---.--~--.------------------- ._._--- ._----
rcspondcni» gender. favorite color, ere. the questionnaires he III .:\ .;illt:k que-non or <t.ucrucrn 1);m.',J \\ u h ir-
r('.;pulls,' options
used in the health sciences arc grounded in psychometric
theory and arc used to quantify various dimensions of health: I )(lIIlai 11t Dimcnsion ] :\ "<'I1l'e'pt measured h) a ~f'(HII' \It' ucm-,

ihev arc referred to as 'instruments', Sc:llc I[eills l'l'I'I'CSCl1til1~ a c!\11ll;1I1l \,(llllhll]('d 1(' PI'l)dUl"';1
scure
The elements or an 'instrument' are listed in Table 6. A
Profile' SnC/;II different dO/ll;III1' "lspLI\ c.l ;1' '<'I);lr;lIl'
typica! instrument consists of a set of scales or 'domains.' A SCOI'C';

domain is designed to measure (\ particular 'construct' or

Index .'\11 :lggJ'cg:llt' 'l'(lIT PI' Sl'\ \'1'.11 L1Plil:IIII';
concept, such as social functioning or mental health. Each
domain typically consists of several questions (or 'items'), BailerI' Scvcral inxtnuucnts comluned [(I pJ'(I\'id" ;1

comprehensive underst.mcliug o! a lil.,c.!.,,' <1I

each item relating to a slightly different aspect of the intervention
construct. The options for response to each item might be a --_ .. -- ---."-~--------

simple yes or no. These responses might be scored as I or 0

.'\11IIITC A(,(,I):
 44 45

A more sensitive way to structure the responses is to provide an aggregate of the correlations :lJ11011gtill' different irem-. \)1'

more than t\VO options. For example. the items could he the scale.
phrased as statements and the response options could be the Another common test of reliability is called te,,[-retl'."[
following: 'strongly agree. agree, neutral. disagree. or reliability. It measures the extent to which the all'i\\ Ch ar,:
strongly disagree'. These five response options could be the same when the questionnaire is given to the 'ianlL' people
'ic\)rccl. for example. 4, :1. 2. I. and O. respectively. Items on two different (but c1usely spaced! occaxion-. II thl' "lurl':-'
with the responses structured in this way are known as Likert on the instrument are very different on the two ()l'l';lsiul1..,. the
xcalcx. wording of the questions should he reexamined.
EXAMPLE. A domain in an instrument contains three In addition to reliability, the validity of the inxtrumeru should
items. each with the response options structured as Likert be assessed. The distinction between reliabilitv - ~\I1dvulidit -.
scales with five options. scored from 0 to -1-. The call be seen if we think about the analogy of mc.isuring xkul l
maximum "core for the domain is 12 and the minimum diameter in order to assess intelligence. \\'e (wild measure
score 0 Intermediate scores from I to II are possible. skull diameter using a variety of different methods that might
The domains in the above 1\\0 examples are 'scale<, because vary in their accuracy and reproducibility. such as J \isll~lI
they generatt' a range of scores that measure the constructs assessment. t:l tape measure, or a CAT scan: th('''l' rncrhod.
represented by the domains. We have discussed multi-item vary in their reliability. No maucr how reliable the
scales, but a scale (or domain) might also contain only a measurement method. however. the skull diameter i" not :l
single item. valid way of estimating human intelligence because there is
no demonstrable relationship bet ween the two.
III the world of health-related questionnaires, psychometric
in-arumenr« are not simply designed and used: they must be There arc various approaches to the validity of psychomcuir
subjected to a series of tests to determine their reliability ami instruments. One common measure, construct v.iliditv.
validity. FiN. it is necessary to determine whether the assesses the relationship between the instrument and tilt'
di tferent items in a scale reliably measure a common construct it is designed to measure. Construct validitv is
construct. This measure of reliability is called internal determined hy comparing in-arument snm:?s «irh S(lille other
l'Ulhht\.'ncy. Internal consistency is computed by calculating measure of the construct.
 46 47

desires. humanistic endpoints are based on patients'

Variables in Outcomes and perceptions of their lives and are by definition subjective.
Pharmacoeconomics
Quality of Life
PATIENT OOTCOMES
The measurement of quality of life is an attempt to measure
the totality of a person's experience of life. including work.
Physiological
recreation. social activities. sex life. etc. In
pharmacoeconornics. we are usually interested in the impact
Patient outcomes may be expressed in terms of a of a person's health status on their quality of life. i.c .. their
physiological variable such as blood pressure. bone mineral health-related quality of lire. Standardized questionnaires
density. or serum concentration of cholesterol, etc. In a cost- measure either health-related quality of life in general or
effectiveness analysis of a cholesterol-lowering drug, for quality of life in the context of a specific disease. The
example, the patient endpoint might be the average Medical Outcomes Study Short Form 36. or SF:16. is a \\idel)
percentage reduction in serum cholesterol concentration. used generic health-related quality-of-life instrument. The
These kinds of variables can be measured objectively using SF36 has eight domains, including physical functioning.
medical instrumentation. However, such 'hard' endpoints are social functioning, role functioning, psychological distress,
criticized as not being directly experienced by patients and general health perceptions, bodily pain, vitality. and
thus not directly relevant to patients' feelings of well-being. psychological well-being. All example of a disease-specific
questionnaire is a migraine quality-of-life questionnaire that
measures migraine symptoms, work functioning. social
Humanistic
functioning, vi tality, and the migrai ne su ffcrer s fee Iings or
In outcomes research, endpoints more directly relevant to concerns about her or his migraine and its treatment."
patients' experience of their lives are often chosen. These
'humanistic' endpoints include quality of life and patient
satisfaction. while more relevant to patients' expressed
 48 49

Patient Preference and Satisfaction Non-Monetary Costs

The same psychometric techniques that are used to measure Direct non-monetary costs are expressed as the health c<lre
quality of life can be applied to patients' satisfaction with the resources consumed by a program or treatment alternative
health care they receive. Patients might be asked to rate the These typically are the costs of physician visits.
health care provider'< knowledge and skill. the quality of the hospitalizations, laboratory tests, drugs. and medical <upplies.
interpersonal care. their degree of trust in the provider, etc. The numbers of these events are counted. The indirect C()sh
may be expressed ill terms of the number of work-loss days,
or the number of days off school.
Mortality and Morbidity

In between the objective physiological endpoints and Monetary Costs

subjective humanistic endpoints arc mortality and morbidity.
Death is both relevant to patient's li yes and objectively
measurable. ~Iorbidity-a diagnosis. e.g .. heart failure, or
pneumonia, and the effects on the patient' s ability to
function-is also relevant to patients' lives and may be based
at least in part on objective data. Diagnoses, however, are
usually based in part on patients' symptoms, which are by
definition subjective.
ECONOMIC OUTCOMES
The true monetary custs of the health care n:soul,-'('''; lI:-,,:'d ~1IL'
not necessarily easy to determine and in P:',lclice "e\ \.Tal
surrogate measures are used. Charges by provider- m<l:- be
equated with cost. HO\\C\-er, these charges ,11\.' not
necessarily the same as the true cost to the provider Jill!
equivalent to the payments they actually receive. The cust of
meclical services and pharmaceuticals can be equated with the
actual reimbursements that the health care insurer paid to the
provider. List prices for drugs and fee schedules for medical
services are available.

The word 'cost' is taken to mean two different things in a

pharmacoeconomic analysis, i.e. the non-monetary costs. Data Sources
referring to the health care and other resources consumed,
and the monetary value of these resources. In a cost-effectiveness analysis. two kinds of data arc needed:
probability values for the chance nodes in the decision tree:

50
and an account of the health care resources used and the
monetary costs associated with the consequences (branches)
stemming from each chance node.
Suppose that a health care professional wishes to know the
answer to a certain question. such as: is drug X an effective
treatment for disease Y in population Z'? There are three
ways to find an answer: the first is to ask someone; the
second is to look the answer up in the literature: and the third
is to perform a study designed to find out the answer. All
three ways of finding answers are used in
phannacoeconomics.
The first method-asking someone who might know-is
institutionalized in the form of the augustly named Delphic
Panel, which is a panel or experts convened to provide their
collective opinion. The Delphic Panel is named after the
oracle of Delphi, famed in classical Greece for its cryptic
pronouncements that could be interpreted as prophetic only in
retrospect. Unless supported by an explicit marshalling of
factual data and analysis, mere opinion does not qualify as
evidence-based medicine. The second two methods arc
discussed below.
Literature Analysis
The U.S. state and federal governments make available basic
demographic information that is often needed in
51
pharmacoeconornic analyses-e.g., the death rate from all
causes by age and gender, death rates by cause, etc. The
Census Bureau publishes past and projected population
demographic data. The National Center for Health Statistics
also collects and makes available ongoing national survey
data addressing a variety of subjects. These include the
National Health Insurance Interview Survey, the Nauoual
Health and Nutrition Examination Survey. and the National
Health Care Surveys.
Much of the information required in a pharmacoeconornic
analysis is present in studies published in the medical
literature. There are several considerations in extracting data
from the literature: determining sources of information.
defining a search strategy, categorizing the studie • identified,
assessing the internal validity of individual reports, <.is~cssillg
the representativeness of the sample of studies idcnri Cicci. and
determining the external validity of the data, i.e.. whether the
data that is valid in the context of the published study is
applicable to the setting of the pharmacoeconomic analyxis.
Very often, several studies can be identified that yield range
of values for the variable of interest. There are two
approaches to this situation. First, if there are one or two
large, well designed studies (such as randomized controlled
trials designed and performed under FDA scrutiny J. it is
reasonable to consider this as 'best evidence' and disregard
larger numbers of smaller, statistically underpowered studies

or studies with less rigorous designs. Second. the results of
several studies can be combined in a meta-analysis or
systematic review.
Meta-Analysis
The term 'meta-analysis' is used in a number of different
ways but. in essence, it means the statistical pooling of data
from several studies. Meta-analysis has its origin in the
social sciences. A 'systematic review' is a meta-analysis
interpreted by the evidence-based medicine movement.
performed according to a predefined system or methodology
and requires conceptual homogeneity in the design and
endpoints of the studies being pooled. One of the most useful
sources for meta-analyses of trials of the efficacy of
treatments is the Cochrane Database of Systematic Reviews.
EXAMPLE. An example of a systematic review is
provided by an investigation of the effects of lipid-
lowering drugs in the primary prevention of coronary
heart diseasc.::~ The authors of the review included only
long-term randomized controlled trials with, as clinical
endpoint, the proportion of the patients who died from
coronary heart disease. Three trials of statins were
identified. in which about one thousand-to-seven
thousand patients were followed for up to about fi ve
years. The authors calculated a pooled odds ratio for the
52
53
effect of statins on coronary heart disease mortality, i.e.,
the odds of coronary heart disease death while taking
statins divided by the odds of coronary heart disease death
while taking placebo. This pooled odds ratio wus 0.65
with a 95% confidence interval of 0.48 to 0.89. indicating
that statins were an effective treatment. This study.
published in the British Medical Journal, is a good
example of a meta-analysis performed under the
procedures of a systematic review. The statistical pool i 11&
as was done with studies that were conceptually
It is homogenous in design, interventions. and endpoints.
Purists might object, however, that the studies \\ ere
heterogeneous in several ways-different statins \vere
administered to different populations during different time
periods-and therefore not eligible for statistical pooling.
The above is an example of a meta-analysis that is also tl
systematic review in the tradition of evidence-based
medicine. The following is an example of a meta-analysis
from the social sciences that diverges from evidence-based
medicine in several key ways.
EXAMPLE. A meta-analysis of interventions intended to
improve patient adherence was carried out as follows.:"
Both randomized trials and trials with non-randomly
assigned comparison groups were included in the meta-
 54
analytical pooling. Trials with a variety of different
interventions were pooled in a single category; e.g., the
category of 'behaviorally focused interventions' included
trials of skill building activities, contracting, drug
packaging, tailoring, rewards, and mail and telephone
reminders. Studies with different methods of measuring
adherence were pooled, e.g., pill counting and
prescription refilling. Study endpoints reported in a
variety of different ways (as proportions, as means, or in
terms of a derived statistic such as a chi squared test)
were pooled. These disparate results were pooled by
computing an 'effect size,' in this case the Pearson
correlation coefficient. r, where r represents the
association between the adherence intervention and the
adherence outcome on a scale of -1 to + 1. Since the
distributions of the association differed among different
interventions and outcome measures, the correlations
were 'normalized' to a standard normal distribution using
the Z transformation.
What is the difference between a systematic review and a
meta-analysis? These methods lie on a continuum, but the
LIseof a standardized 'effect size' might be taken as a
dividing line between them because it is symptomatic of
heterogeneity in the study outcomes. Various effect size
statistics have been devised: Hedges' g and Cohen's dare
commonly used, in addition to the Pearson correlation
55
coefficient. The practice of pooling disparate data ill meta-
analyses as in the above example partly explains why meta-
analysis has been seen as controversial. Some authors of
systematic reviews eschew statistical pooling of
heterogeneous data.
EXAMPLE. Haynes et a1. reported a systematic review of
randomized controlled trials of interventions designed to
help patients follow prescriptions for medications." Trials
were identified and data extracted from them using a
systematic procedure. However. the trials identified were
too disparate in clinical problems, interventions. measures
and reporting of adherence. and in methods of measuring
and computing the clinical outcomes to warrant statistical
pooling.
Clinical Studies
In extremis, when the required data are not available in the
literature, a dedicated study must be performed.
Observational studies are often undertaken to determine
costs. However, a clinical trial is never undertaken with the
sole purpose of determining cost-effectiveness. This is
because a cost-effectiveness analysis is only considered aft«:
effectiveness has first been demonstrated. However. cost
data may be collected prospectively in an elfccti vencss lor
efficacy) trial. which is then called a piggybad:. trial
 56
Administrative Databases
Medical information about patients, such as blood pressure,
temperature, severity of illness, etc., is usually recorded on
paper 'charts' and is conveniently accessible only for
individual patients or small groups of patients. An alternative
is to "follow the money." Every health care transaction that
takes place is documented using standard systems of
codification. with the ultimate purpose of obtaining payment.
This codified information can be used not only to track
payments for health care services, but to build a picture of the
health care services used by a patient or class of patients, and
to infer a picture of the courses of diseases and the effects of
the treatments.
The main types of data contained in administrative databases
are demographic. diagnostic, procedural, and pharmaceutical.
The demographic data is usually limited to the patient's age
and gender. The diagnosis the patient receives is entered as a
code using the International Classification of Diseases (ICD).
The principal diagnosis and a limited number of co-
morbidities are recorded by their lCD codes. The exact
nature of any procedure performed by a physician is
documented in terms of a CPT code, and each drug that is
prescribed is specified by its National Drug Code (NDC).
These data sets also include information on the service
provider: the physician's specialty, the setting (primary care,
outpatient. inpatient). geographic location. and. if inpatient.
57
the number of days of stay, etc. Each record contains a
patient identifier, a unique number that identifies the recipient
of the health care service.
Because of the fragmented nature of the health care system in
the United States, however, collating the different data sets
into one coherent whole may be problematic for any gi ven
population of patients. State Medicaid and Medicare data
sets contain comprehensive records for patients covered by
these insurance systems, but they only apply to eligible
indigent and elderly (65 years and over) patients.
respectively. Staff model H1·10s also may contain complete
data sets. In the case of third-party payment syslellls.
however, the data sets may be dispersed among different
payers. Pharmacy data sets may be maintained by Pharrnacv
Benefit Management companies (PBMs). while numerous
health care insurers may maintain hospital and primary care
claims data. Collations of private sector health care insurance
data sets can be purchased that offer a complete picture of the
encounters of hundreds of thousands of patients with the
health care system over defined time periods.
EXAMPLE 1: An HMO administrative database analysis.
The health care costs of peptic ulcers and bleeding
resulting from the prescription of non-steroidal anti-
inflammatory agents (NSAIDs) for arthritis have been
intensively investigated. Johnson et al. estimated the
incidence of inpatient and outpatient gastropathics. the
 58
"en ices provided to treat them. and the costs of those
services for cldcrlv members of an HMO located in the
north-wext I.' nited States. 1-1 The data for morbiditv and
health care resource use were obtained from four
automated dat~lbases maintained bv the HT\IO: an
outpatient pharmacy d.uabaxc. a hospital discharge
database. a Illl'mht'rship inforru.uion datab;l"e. and an
outpatient utiliz.n ion database. These four data seb were
linked Llsing the patient identifier tagging each record.
The pharmacy and hospital discharge dCltClhases contained
Information for every prescription and inpatient stay.
rl'spt'L'li\l'ly. that occurred. The outpatient database
represented ;1 random sample of outpatient encounters
absuactcd from paper medical record". Costs to the Ht\10
were estimated from the Medicare Cost Report. an
aggregate report of \.:()sts that includes direct medical and
overhead custs related to capital invexuncnt. general
administration. ere .. and which formed the basis for cost-
based Medicare reimbursements to the HMO. The result
of the study \\as that, for every dollar spent by the HMO
on NSAID therapy for the elderly, another 35 cents was
spent to treat the adverse gastmintcstinall,ffects of the
['.;SAIDs
EXAt\,1PLE 2: A private-pay, fee-for-service database
~lI1ahsis. The health care costs associated with the
treatment \)1' depression with different classes of
59
antidepressant drug were estimated ill ~I relruspL'cti\e
cohort study.23 The data source was a proprietary
database of medical and pharmacy claim-, cull.ucd trou:
numerous private. fee-for-sen'ice health care inxurcrx
covering employees of corporate America. lndividual-,
\\'110 had a new prescription for an antidcprcss:lnt dru~
and a diagnosis of depression within a lkfilll'd i imc Pl'!I\h.1
(the index period) were identified. and their health cure
resource use was tr;tcKed for the next six months. ,\
diagnosis of depression was indicated by :1 relevant ICD-l)
code in inpatient and outpatient records. -\Iltideprl's.-,alll
drugs were identified by their NDC codes. C\)SlS were
compared for antidepressant drug classl's llsillg all intent-
to-treat analysis. i.e .. the patient \\';" cl:lssifil'd ;t('lorclil1~
to the initial antidepressant drug hl' or shl' \\as prescribc.]
in the index period. regurdlevs of whether Ill' or sill'
subsequently switched to another cla.ss duriIlg tile J'llllu\\-
lip period.
Financial Data
Administrative data sets contain the dollar paymcui-, made lor
the health care resources used. These payments are not
necessarily the same as the charges made by the puvce
because of negotiated fee schedules, capitation. ere. Li-a
prices of health care goods and services are puhl: ..,hed h:-
State Medicare system". List prices for drLl!:2>'Clrl' puhl i-hcd
 60 61

as average wholesale prices and are also available as retail Similarly, the word "bia-.' docs not require ,I c(llhci(llh ur
prices. Data Oil employee remuneration. which might be unconscious human motivation to alter the results.
needed for an indirect costs analysis. are provided by the
EXAMPLE. Several randomly chosen 11LIIll,ln -ubjeci-,
Bureau of Labor Statistics.
were given the same dose of a drug. The response or thl'
subjects to the single dose varied greatly and a graph of
the drug response versus frequency descrihed a bvll curve.
Accounting for Uncertainty Among the factors causing this \ariahility were
genetically determined differences alllong subject» in their
DEFINITIO]'.; OF ERROR ability to absorb. metabolize. and c liminar.- the drug. and
in the interaction of the drug with its tissuc targcl. The
There are two hind" of error, namely random error and subjects varied greatly' ill their body ma-«. hl()od \ olumc.
'-.\"tel11atlc error. Random error is variabilitv - ill the result percent bodv fat. the Illetaholizillg ahi lit v (If their lih'!':'>.
.

Gtu"ed by random or unprcciicubk variability in the factors and in :1 host of other atuihutcx lh:tt 11lociificd 1111..' :!l'li()1l (I(

determining the result. Systematic error is a bias in the result the drug.
c,lused by nonrandom variability ill these factors.
To understand the distinction between the two kinds of error. PRINCIPAL SOURCES OF ERROR IN CU!\;J( '.\t
Imagine darts ihrov n at a dartboard.
v The random clustering
ot the darts around the bull'<-eye (which is the target) STUDIES
rl'presents random error. The tighter the clustering around
the bulls-cyc, the less random error there is. If the darts tend Although many types or hiax in the de"igll and conduct ut'
III cluster in the lower right quadrant of the board, for clinical studies have been described. Chalmcr-, h,IS pointed
example. there is a systematic en-or or bias in the thrower's out that most fall into three important cakgoril's: xe lccuon
bias at study entry, selection bias after study emr). ,1I1d bias
aim.
in a.'-;sessing outcomes-see Table 7. ~
In this context. the word 'error' means variability and does
not impl , that ~lprc\cntable mistake has been made.
 62 63

Table 7. Means of Accounting for Systematic Error by Study imbalance in the allocation of subjects may remain-e.g.,
Design more females than males may be assigned to treatment A than
Controlled Trial Comparative to treatment B-simply because of random error. but this
Error
Observational Study may be reduced by increasing the sample size. The process
of random allocation is blind to both perceivable and
Selection bias at Randomization Case matching. Propensity
sconng imperceivable differences between subjects, and this is its
study entry
principal virtue.
Selection bias after Intent-to-treat analysis Intent -to-treat analysis.
study entry multivariate analysis

Bias in assessing Observer blinding.

outcomes subject blinding
REDUCING ERROR IN CLINICAL STUDIES
The obscuring of the true result due to random error can be
lessened in two ways: by increasing the sample size; and by
reducing the variability in the sample. Approaches to
reducing systematic error are discussed below.
Selection Bias in Subjects Entering the Study
Randomization
In a controlled clinical trial. random allocation of subjects to
treatments eliminates systematic error at study entry. An
Case Matching
In a comparative observational study, there may be
considerable differences between the subjects who recei vcd
treatment A and those who recei ved treatment B. The
investigator can control for perceivable (but not
imperceivable) differences in subjects by balancing these
characteristics between the two comparative groups. This i.-.;
case matching.
EXAMPLE. It is the year 2002. Patients who happened to
receive treatment A at a clinic in the year 2000 were
predominantly female and younger, while those who
received treatment B were mostly male and older. The
investigator compares treatment A with treatment B by
picking, for example, a female in the 30-40 year age
category who received treatment A and a (harder-to-find)
female in the same age category who happened to have
received treatment B. The investigator does thesame for
 64
65

females in the 50-60 year age category and in the 70-80 Selection Bias in Subjects after Entering the Study
year age category, ere, and for males in the same age
categories. The investigator now has a group who received In a randomized controlled trial. subjects initially allocated to
treatment A that is evenly balanced by age and gender with receive (say) treatment A might subsequently leave their
a group who received treatment B, and he may proceed assigned
'- treatment group
'- and receive no treatment. an
with computation of the study outcome. unplanned treatment, or switch to treatment B. In an intent-
to-treat analysis, all subjects initially assigned to recci ve
treatment A are included in the analysis of outcomes of group
Propensity Scoring A. This ensures that the function of randomization-the
control for selection bias-is preserved. A selection bias
In a cast-control study, subjects who received treatment A
would be introduced if patients who left their assigned
and those who received treatment B are chosen
treatment group were not included in the analysis. since these
retrospectively to be matched according to age, gender. and
patients were subject to some form of selection. The same
any other relevant criteria that can he measured. Tn an
intent-to-treat approach can he applied to retrospective
analysis based on propensity scores, subjects in group A are
studies.
matched with those subjects in group B who have the same
propensity score. where the propensity score is the
probability that a subject will be assigned to group B rather PHARMACOECONOMIC STUDIES
than group A based on a composite of observable
determining characteristics. 13 The study population is then
divided into categories (usually guintiles) based on their Sensitivity Analysis
propensity score and within each guintile the outcomes of
individuals who received treatment A are compared with The sample decision analysis described earlier contained all
those received treatment B. example of one-way sensitivity analysis. One-way sensitivity
analysis is an example of simple sensitivity analysis. which is
one of several general approaches.
 66 67

Simple Sensitivitv Analysis
In simple sensitivity analysis, one study variable is varied
over the range of likely values, while all other variables are
held constant If the variables are independent, a series of
one-way sensitivities is informative, In two-way sensitivity
analysis, the effects of varying two variables simultaneously
are computed. Similarly, in three-way sensitivity analysis,
three variables are varied over their likely ranges of values at
the same time. Two- and three-way sensitivity analysis are
more appropriate when study variables interact. The
calculations are performed using a computer program and the
results displayed graphically.
Probabilistic Analvsis
probabilities are known, the computer can take these into
account (rather than assuming a uniform distribution within a
range of values). This type of probabilistic analysis is
referred to as a Monte Carlo simulation.
Best and Worst Case Scenarios
This form of sensitivity analysis is more formally called
analysis of extremes, A best-case estimate of cost-
effectiveness would combine the lower extreme estimate of
costs and the upper extreme estimate of effectiveness.
Similarly, a worst-case estimate would combine the upper
extreme of costs and the lower extreme of eftccti veness.
Compared to a Monte Carlo simulation, this type of analysis
is crude, but it is useful in some circumstances.

Instead of a performing several one-way or multi-way

analyses, it would be more desirable to perform an 'every- Threshold Analysis
way' analysis, i.e .. simultaneously vary all of the study
Threshold analysis is a modification of one-way sensitivity
variables throughout their ranges of likely values. A
analysis. Instead of a single variable being varied throughout
computer program can do this by starting with an imaginary
its range of likely values, it is varied with the purpose of
cohort of, say. 1,000 patients and running them one by one
finding the threshold at which the decision alternatives have
through the decision tree, randomly assigning a likely value
the same expected value. The threshold is also called the
for the probability at each chance node. A distribution of breakeven point.
outcome values for the entire 1,000 patients can be thus
calculated. and a mean and 95% confidence interval
computed. When distributions for the values of chance node

Evaluating Pharmacoeconomic
Studies
The established standard for assessing any published study is
that sufficient information should be presented for a peer
researcher to be able to repeat the study (and get the same
answer). in the case of a cost-effectiveness analysis, this
standard requires that the decision tree be presented or
completely described in the text, and that all study variables
and information sources be reported. Many published cost-
effectiveness analyses are black boxes that are difficult for
the average reader to assess Jet alone reproduce: in such
circumstances. the credibility of the researchers is important.
Guidelines or checklists can be useful aids in assessing
studies. Guidelines for evaluating pharmacoeconomic
have been presented by university researchers, I the
Pharmaceutical Research and Manufacturers of America."
and state governments.i In addition to these. criteria for
assessing the conduct and reporting of various kinds of
studies have been developed. The Users' Guides to the
Medical Literature series published in lAMA includes articles
on the assessment of economic analysis.i' 19 decision
analysis,2425 health-related quality of life,' outcomes
researc I1. an' systematic reviews. 20 There are a Iso
IX d .,
published criteria for the reporting of clinical trials
(CONSORT)lhand for writing manuscripts of different
types. -I'
68
69
References
I.
Economic analysis of health care technology. A report on principle-;
Task Force on Principles for Economic Allalvsis of Health Care
Technology. AIl111nlern Med 1995: 12:\11):61-7().
2. Pharmacoeconomn-e and Outcomes: Applications for Patient Care.
Kansas City, MO: American College of Clinical Pharmacy. 19%.
3. Chalmers L Hetherington J. Elbourne D. Ke irsc !'vIJNC Materials
and methods used in synthesizing evidence to evaluate the effects 01
care during pregnancy and childbirth. Ill: Chalmers l. Enkin M.
Keirse Mf N, eds. Ljjl'clil'c Care ill Pregnane, an.! Childbinh. sot.
Volume J: Pregnane-c. Nc\,' York. ]\;Y: Oxford l'lli\lT,ity Pres., Inc.
1989: 39-65.
<-1.
Clemens K. Townsend R. Luscombe F. i\ lauskopr}. Oxrerh.n: .-; .J.
studies Babula J. Methodological and conduct principle , lor
phannacocconomic research. Pharmaceurical Research and
Manufacturers of America. P/w rtnacoeco noniirs 1995: 8( 21: I h9-!-+.
5. Drummond MF, Richardson \\/S, O'Brien 8J. Levine i\l. Heyland D.
Users' guides to the medical literature. Xlll How to use all article Oil
economic analysis of clinical practice. A. Are the results of the study
vaJid? Evidence-Based Medicine \Vorking Group. JAil-fA. J 997:
277( 19): 1552-7
6. Glick H, Cook 1. Kinosian B et al. Costs and effects of enalapril
therapy in patients with symptomatic heart failure: An economic
analysis of the studies of the left ventricular dysfunction ISOL VD)
treatment trial. J Cardiac Failure 1995; 1371-9
7.
Guyatt GH. Naylor CD, Juniper E. Heylund DK . .I~It:'l·IlLc R. C()(Jk
OJ. Users' guides to the medical literature XlI. How l\l Lise article,
 70
71

about health-related quality of life. Evidence-Based Medicine

Working Group. lAMA 1997: 277(15): 1232-7.
8. Hartmaier SL, Santanello NC, Epstein RS, Silberstein SD.
Development of a brief 24-hour migraine-specific quality of life
questionnaire. Headache 1995: 35(6):320-9
9. Haynes RB. McKibbon KA, Kanani R. Systematic review of
randornised trials of interventions to assist patients to follow
prescriptions for medications. Lancet 1996: 348(9024):383-6.
10. Henshaw SK. Unintended pregnancy in the United States. Fam
Plann Perspect 1998; 30(1 ):24-9,46.
II Hilleman DE. Heineman SM, Fond PA. Pharmacoeconornic
assessment of IH.,.1G-CoA reductase inhibitor therapy: an analysis
based on the CURVES study. Pharmacotherapv 2000; 20(7):819-
1,
I~ International Committee of Medical Journal Editors. Uniform
Requirements for Manuscripts Submitted to Biomedical Journals
lAMA 1997: 277(1])927-34
13. Joffe MM, Rosenbaum PR. Invited commentary: propensity
Am 1£pidemioI1999; 150(4):327-33.
14. Johnson RE, Hornbrook MC, Hooker RS, Woodson GT, Shneidrnan
R. Analysis of the costs of NSAID-associated gastropathy.
Experience in a US health maintenance organisation.
Phil rmacoeconomics 1997; 12( 1):76-88.
Reporting Trials. lA/vIA 1998; 279( 18): 1489-91
l7. Murray CJ, Lopez AD. Global mortality, disability. and the
contribution of risk factors: Global Burden of Disease Study. LaIlCl:'T
1997; 349(9063):1436-42
] 8. Naylor CD, Guyatt GH. Users' guides to the medical literature X.
How to use an article reporting variations in the outcomes of health
services. The Evidence-Based Medicine Working Group. lAMA
1996: 275(7)554-8.
19. O'Brien BJ, Heyland D, Richardson WS, Levine M, Drummond 1\11-".
Users' guides to the medical literature. XIll. How to use all article (In
economic analysis of clinical practice B. What arc tht' results .uid
will they help me in caring for my patients') Evidcnce-Bnscd
Medicine Working Group. lA/I,fA 1997: 7.771 n II S(J:>-o.
20. Oxman AD, Cook DJ, Guyatt GH. Users' guides to the medical
literature. VI. How to use an overview Evidence-Based Medicine
Working Group. lAMA 1994: 272( 17) 1367-71.
21.
Petitti DB. Meta-Analysis, Decision Analysis, and Cost-
scores. Effectiveness Analysis Methods for Quantitatiti ve Synthesis In
Medicine. Monographs in Epidemiology and Biostatistics. Volume
24. New York: Oxford University Press. I CJ94.
22. Pignone M, Phillips C. Mulrow C Use or lipid lowering drugs for
primary prevention of coronary heart disease. meta-analysis or
randomised trials. BMl 2000: 321 (7267):983-0

Marciante KD, Gardner JS, Veenstra DL, Sullivan SD. Modeling the 23. Poret A W, Neslusan C, Ricci J-F, Wang S. Khan ZM, Kwong JW.
15.
cost and outcomes of pharmacist-prescribed emergency Retrospective analysis of the health care costs or bupropion
contraception. AI1I 1 Public Health 200]; 91(9): 1443-5. sustained release in comparison with other antidepressants. \ 'atue ill
Health 2001: 4(5):362-9.
16. Moher D. CONSORT: an evolving tool to help improve the quality
of reports of randomized controlled trials. Consolidated Standards of 24. Richardson \\/S. Detsky AS. U sers: guide" to the medical li ter.uure
VJl. How to use a clinical decision analysis. A. Are the results of the

•• ,,-- I!';!'·,:''-.f:
 72

study valid'! Evidence-Based Medicine Working Group. lAMA

1995: 273(] 6) 1292-5.

25. Richardson WS, Detsky AS. Users' guides to the medical literature.
VII. How to use a clinical decision analysis. B. What are the results
and will they help me in caring for my patients? Evidence Based
Medicine Working Group. lAMA 1995; 273(20):1610-3.

26. Rater DL Hall lA, Merisca R, Nordstrom B, Cretin D, Svarstad B.

Effect: veness of interventions to improve patient compliance: a
meta- analysis. Med Care 1998; 36(8): 1138-61.

27. Torrance GW, Blaker D, Detsky A et al . Canadian guidelines for

economic evaluation of pharmaceuticals. Canadian Collaborative
Workshop for Pharmacoeconomics. Pharmacoeconomics 1996;
9(61535-59

2R. van HULltSA, Wielink G, Bonsel G1, Rutten FF. Effects of ACE
inhibitors on heart failure in The Netherlands: a pharmacoeconornic
model. Pharmacocconomics 1993; 3(5):387-97.