Topic 1-9 - PolicyAnalysis-all Topics

Public Policy Analysis
Introduction to the Public Policy

Process
Policy Analysis in the PolicyMaking Process

Policy analysis is the activity of creating
knowledge of and in the policy-making process.

In creating knowledge of policy-making processes
policy analysts investigate the causes,
consequences, and performance of public policies
and programs.
Such knowledge remains incomplete, however,
unless it is made available to policymakers and the
public they are obligated to serve.
The Process of Policy Inquiry

This course provides a methodology for policy
analysis.
Methodology is a system of standards, rules and
procedures for creating, critically assessing, and
communicating policy-relevant knowledge.
Problem solving is a key element of the
methodology of policy analysis

Policy analysis is also a methodology for
formulating problems as a part of a search for
solutions.
The Methodology of Policy

Analysis
The methodology of policy analysis draws
from and integrates elements of multiple

disciplines
Political science
Sociology
Psychology
Economics
Philosophy

Analysis (contd.)
Policy analysis is partly descriptive, seeking
knowledge about causes and consequences

of public policies.
But, policy analysis is also normative,
creating and critiquing knowledge claims
about the value of public policies for past,
present,and future generations.

Analysis (contd.)
Policy-relevant knowledge involves choosing
among competing values such as health, wealth,

security, peace, justice, equality, and freedom.
To choose among competing values is not a technical
judgment, but a moral judgment. Thus, policy analysis
is a form of applied ethics.
Policy analysis also seeks to create knowledge that
improves the efficiency of choices among

alternative policies.

Analysis (contd.)
The methodology of policy analysis aims at
creating, critically assessing, and

communicating policy-relevant knowledge.

Analysis (contd.)
Knowledge in this context refers to
plausibly true beliefs, as distinguished from

beliefs that are certainly true, or even true
with a particular statistical probability
Statistical evidence performs only a
supplementary or reinforcing role.

Analysis (contd.)
In short, policy analysis has developed a
core of moderately coherent underlying

theories, a variety of methods that enjoy
reasonably broad assent among
practitioners, a tradition of criticism
directed at political, ideological, and ethical
issues raised by policy analysis, and
systematic as well as anecdotal evidence of
having improved the capacities of clients to
solve problems.

Analysis (contd.)
A key feature of research and analysis on
social problems over the past 40 or more

years is the growing recognition of
complexity.
The methodological core of policy analysis can
be broadly characterized as a form of critical
multiplism.

Analysis (contd.)
For critical multiplism, inductive
plausibility, not certainty, is the defining

characteristic of knowledge and a major
standard of success in policy inquiry.
The analyst does not enumerate supporting or
confirming cases but identifies, evaluates, and
eliminates competing explanations.

Analysis (contd.)
The other major standard is policy
relevance.
Knowledge that assists in formulating and
solving problems, as these problems are
experienced by policymakers and citizens on
whom the policies have an impact, including
the disenfranchised.

Analysis (contd.)
General guidelines for policy inquiry under
critical multiplism.
Multiple operationalism.
The use of multiple measures of policy constructs.
Multimethod research.
The use of multiple methods to observe policy processes and
outcomes.
Multiple analytic synthesis.

The synthesis and critical assessment of available analyses of
similar policies and programs.
Multivariate analysis.
The inclusion of multiple variables in policy models.

Analysis (contd.)
General guidelines for policy inquiry under
critical multiplism (contd.)
Multiple stakeholder analysis.

The investigation of interpretive frameworks and perspectives
of multiple policy stakeholders.
Multiple perspective analysis.

The incorporation into policy analysis of multiple perspectives
ethical, political, organizational, economic, social, cultural,
psychological, and technical.
Multimedia communications.
The use by policy analysts of multiple communications media
is essential for ensuring that knowledge is policy relevant.

Analysis (contd.)
Given typical time and financial constraints,
it is usually impossible to observe all of

these guidelines.
In addition, depending on the stage of the
policy process being analyzed, several
guidelines may not be relevant.
Multiple methods makes analysts less likely
to commit errors resulting from limited
perspectives.
Policy-Relevant Information
The methodology of policy analysis designed to
answer five questions:

What is the nature of the problem?
Policy problems.
What present and past policies have been established to

address the problem, and what are their outcomes?
Policy outcomes.
How valuable are these outcomes in solving the

problem?
Policy performance.
Policy-Relevant Information
The methodology of policy analysis
designed to answer five questions:

What policy alternatives are available to
address the problem, and what are their likely
future outcomes?
Policy futures.
What alternatives should be acted on to solve

the problems?
Policy actions.
Five types of policy relevant

information.
POLICY
PERFORMANCE
POLICY
OUTCOMES
POLICY
PROBLEMS
POLICY
ACTIONS
POLICY
FUTURES
Policy-Analytic Procedures
Methodology of policy analysis incorporates
standards, rules, and procedures.
The standards and rules govern the selection of

procedures and the interpretation of results.
The five procedures common to all human
problem solving:
Definition
Prediction
Prescription
Description
Evaluation
Five procedures as applied to policy analysis:
Problem structuring (definition)
Yields information about the conditions giving rise to a policy
problem.
Forecasting (prediction)
Supplies information about future consequences of acting on
policy alternatives, including doing nothing.
Recommendation (prescription)
Provides information about the relative value or worth of these
future consequences in solving or alleviating the problem.
Monitoring (description)
Yields information about the present or past
consequences of acting on policy alternatives.
Evaluation (evaluation)
Provides information about the value or worth of
these consequences in solving or alleviating the
problem.
The five procedures serve as the means of
organizing particular methods and techniques of

analysis.
Methods are general procedures for producing and
transforming policy relevant information.
Each of the methods is supported by several techniques.
The complete framework for problem-centered
policy analysis combines policy-relevant

information transformed by policy analytic
procedures.
Five policy-analytic procedures
Forecasting
Evaluation
Problem
Structuring
Problem
Structuring
Problem
Structuring
Problem
Structuring
Monitoring
Recommendation
Problem-centered policy analysis

POLICY
PERFORMANCE
Forecasting
Evaluation
Problem
Structuring
POLICY
OUTCOMES
POLICY
PROBLEMS
Problem
Structuring
Problem
Structuring
POLICY
FUTURES
Problem
Structuring
Monitoring
Recommendation
POLICY
ACTIONS
Phases of the policy-making

process
PHASE
CHARACTERISTICS
ILLUSTRATION
AGENDA SETTING
Elected and appointed officials place

problems on the public agenda. Many
problems are not acted on at all, while
others are addressed only after long
delays.
A state legislator and her cosponsor

prepare a bill that goes to the Health and
Welfare Committee for study and
approval. The bill stays in committee and
is not voted on.
POLICY
FORMULATION
Officials formulate alternative policies to

deal with a problem. Alternative policies
assume the form of executive orders, court
decisions, and legislative acts.
A state court considers the use of

standardized achievement tests such as the
SAT on grounds that the tests are biased
against women and minorities.
POLICY ADOPTION
A policy alternative is adopted with the

support of a legislative majority,
consensus among agency directors, or a
court decision.
In Roe v. Wade Supreme Court justices

reach a majority decision that women have
the right to terminate pregnancies through
abortion.
POLICY
IMPLEMENTATION
An adopted policy is carried out by

administrative units which mobilize
financial and human resources to comply
with the policy.
The city treasurer hires additional staff to

ensure compliance with a new law which
imposes taxes on hospitals that no longer
have tax-exempt status.
POLICY
ASSESSMENT
Auditing and accounting units in

government determine whether executive
agencies, legislatures, and courts are in
compliance with statutory requirements of
a policy and achieving its objectives.
The General Accounting Office monitors

social welfare programs such as
Temporary Assistance to Needy Families
(TANF) to determine the scope of welfare
fraud.
Policy analytic procedures and

phases of policy-making
Problem
Structuring
Forecasting
Recommendation
AGENDA SETTING
POLICY
FORMULATION
POLICY ADOPTION
Monitoring
POLICY
IMPLEMENTATION
Evaluation
POLICY
ASSESSMENT
Problem Structuring
Problem structuring can supply policy-relevant
knowledge that challenges the assumptions

underlying the definition of problems reaching the
policy-making process through agenda-setting.
Problem structuring can assist in discovering
hidden assumptions, diagnosing causes, mapping
possible objectives, synthesizing conflicting
views, and designing new policy options.
Forecasting
Forecasting can provide policy-relevant knowledge
about future states of affairs which are likely to occur

as a consequence of adopting alternatives, including
doing nothing, that are under consideration at the
phase of policy formulation.
Forecasting can examine plausible, potential, and
normatively valued futures, estimate the consequences
of existing and proposed policies, specify probable
future constraints on the achievement of objectives,
and estimate the political feasibility (support and
opposition) of different options.
Recommendation
Recommendation yields policy-relevant
knowledge about the benefits and costs of

alternatives the future consequences of which have
been estimated through forecasting, thus aiding
policymakers in the policy adoption phase.
Recommendation helps estimate levels of risk and
uncertainty, identify externalities and spillovers,
specify criteria for making choices, and assign
administrative responsibility for implementing
policies.
Monitoring
Monitoring provides policy-relevant knowledge
about the consequences of previously adopted

policies, thus assisting policy-makers in the policy
implementation phase.
Monitoring helps assess the degree of compliance,
discover unintended consequences of policies and
programs, identify implementational obstacles and
constraints, and locate sources of responsibility for
departures from policies.
Evaluation
Evaluation yields policy-relevant knowledge
about discrepancies between expected and actual

policy performance, thus assisting policymakers in
the policy assessment phase of the policy-making
process. Evaluation not only results in
conclusions about the extent to which problems
have been alleviated; it also may contribute to the
clarification and critique of values driving a
policy, aid in the adjustment or reformulation of
policies, and establish a basis for restructuring
problems.

The Functions of Policy Argument
Introduction
Policy argumentation is central to policy
analysis and the policy-making process.

The analysis and evaluation of policy
argumentation are central to the process of
critical thinking.
The Structure of Policy Arguments

A policy argument is the product of
argumentation, which is the process.

In real-life policy settings, arguments are complex
and prone to misunderstanding.
To minimize misunderstanding, we use the
structural model of argument developed by
Stephen Toulmin, which is designed to investigate
structures and processes of practical reasoning.

The conclusions of practical arguments are
always uncertain, as are the reasons and

evidence that lead to these conclusions.

Types of knowledge claims.
A knowledge claim is the conclusion of a
policy argument.
Three types of knowledge claims:
Designative: questions of fact. What are the
observed outcomes of a policy and why did they
occur?
Evaluative: questions of value. Was the policy
worthwhile?
Advocative: questions of right action. Which policy
should be adopted?

Types of knowledge claims (contd.).
Policy arguments contain six elements:
Information (I), Claim (C), Warrant (W), Backing (B), Rebuttal (R),
Qualifier (Q).
The first four of these elements are present in every policy

argument.
The claim C is the conclusion or output of an argument,
which is supported by policy-relevant information I, which
is the beginning or input of the argument.
The warrant W is the justification, or reason, for concluding
C from I.
The qualifier Q indicates that C has a given truth or
plausibility.

Example:
The Senator supports the privatization of the
federal highway system, which have significant
gains in efficiency and a reduction in taxes.
Considering that the privatization of public
services has been successful in other areas, this
is definitely a no brainer. Besides this is the
same conclusion as a panel of experts on
privatization.

The underdetermination of conclusions by
information.
Policy-relevant information does not fully determine
the conclusions of policy arguments. Information does
not speak for itself.
Identical information can and often does lead to
different conclusions, which we call policy claims to
emphasize the fallible and indeterminate character of
arguments.

Example: Policy-relevant information from the Coleman
Report Black students attending primarily black schools

had lower achievement test scores than black students
attending primarily white schools.
Designative claim and qualifier: Since schools in large
urban areas are primarily black, the hopes of blacks for
higher education achievement [simply] cannot be realized.
Evaluative claim and qualifier: The Coleman Report is
[clearly] a racist document based on ethnically biased
achievement tests.
Advocative claim and qualifier: [There is no question] that
a national policy of compulsory school busing ought to be
adopted to achieve integrated schools.

Warrants and rebuttals.
Although each of the claims about the Coleman
report begins with the same information, very
different conclusions are drawn.
Differences are due not to the information, but
to the role of the warrants in justifying
(plausibility or implausibility) the claims on the
basis of the information supplied.
Modes of Policy Argumentation

Modes of policy argumentation are the characteristic
routes followed by information as it is transformed into

policy claims.
The several different modes of argument involve reasoning
from authority, method, generalization, classification,
intuition, cause, sign, motivation, analogy, parallel case,
and ethics.
Each of the eleven modes of argument has a different type
of warrant, and multiple modes can be found in any policy
argument.
The warrants are the reasons offered by the proponent or
opponent of a policy to justify a claim, or inference, based
on the information supplied.

Authority
Reasoning from authority is based on warrants
having to do with the achieved or ascribed
statuses of producers of policy-relevant
information. For example, experts, insiders,
scientists, specialists, gurus, power brokers.
Footnotes and references are disguised
authoritative arguments.
Argumentation from Authority

Method
Reasoning from method is based on warrants
about the approved status of methods or
techniques used to produce information. The
focus is on the achieved or ascribed status or
"power" of procedures, rather than persons.
Examples include approved statistical,
econometric, qualitative, ethnographic, and
hermeneutic methods and techniques.
Argumentation from Method

Generalization
Reasoning from generalization is based on similarities
between samples and populations from which samples
are selected. Although samples can be random,
generalizations can also be based on qualitative
comparisons. The assumption is that what is true of
members of a sample will also be true of members of
the population not included in the sample. For example,
random samples of n 30 are taken to be representative
of the (unobserved and often unobservable) population
of elements from which the sample is drawn.
Argumentation from Generalization

Classification
Reasoning from classification has to do with
membership in a defined class. The reasoning is
that what is true of the class of persons or
events described in the warrant is also true of
individuals or groups which are members of the
class described in the information. An example
is the untenable ideological argument that
because a country has a socialist economy it
must be undemocratic, because all socialist
systems are undemocratic.
Argumentation from Classification

Cause
Reasoning from cause is about the activity of
generative powers ("causes") and their consequences
("effects"). For example, a claim may be made based on
general propositions, or laws, of economics that state
invariant relations between cause and effect. Other
causal claims are based on observing conditions that
must be satisfied to infer that a policy has a specified
effect. Most argumentation in the social and natural
sciences is based on reasoning from cause.
Argumentation from Cause
Argumentation from Cause

Sign
Reasoning from sign is based on signs, or indicators,
and their referents. The presence of a sign indicates the
presence of an event or condition, because the sign and
what it refers to occur together. Examples are indicators
of institutional performance such as "organizational
report cards" and "benchmarks," or indicators of
economic performance such as "leading economic
indicators." Signs are not causes, because causality
must satisfy additional requirements not expected of
signs.
Argumentation from Sign

Motivation
Reasoning from motivation is based on the
motivating power of goals, values, and inten
tions in shaping individual and collective
behavior. For example, a claim that citizens will
support the strict enforcement of pollution
standards might be based on reasoning that,
since citizens are motivated by the desire to
achieve the goal of clean air and water, they
will act to offer their support.
Argumentation from Motivation

Intuition
Reasoning from intuition is based on the
conscious or preconscious cognitive, emotional,
or spiritual states of producers of policyrelevant information. For example, the
awareness that an advisor has some special
insight, feeling, or "tacit knowledge" may serve
as a reason to accept his judgment.
Argumentation from Intuition

Analogy-Metaphor
Reasoning from analogies and metaphors is based on
similarities between relations found in a given case and
relations characteristic of a metaphor, analogy, or
allegory. For example, the claim that government
should "quarantine" a country by interdicting illegal
drugswith the illegal drugs seen as an "infectious
disease"is based on reasoning that, since quarantine
has been effective in cases of infectious diseases,
interdiction will be effective in the case of illegal drugs.
Argumentation from Analogy
Modes of Policy Argument

Parallel Case
Reasoning from parallel case is based on
similarities among two or more cases of policy
making. For example, a reason that a local
government should strictly enforce pollution
standards is that a parallel policy was
successfully implemented in a similar local
government elsewhere.
Argumentation from Parallel Case
Modes of Policy Argument

Ethics
Reasoning from ethics is based on the rightness or
wrongness, goodness or badness, of policies or their
consequences. For example, policy claims are
frequently based on moral principles stating the
conditions of a "just" or "good" society, or on ethical
norms prohibiting lying in public life. Moral principles
and ethical norms go beyond the values and norms of
particular individuals or groups. In public policy, many
arguments about economic benefits and costs involve
unstated or implicit moral and ethical reasoning.
Ethical Argumentation
Systems of Argumentation
Completeness. Elements of an argument should
comprise a genuine whole that encompasses all

appropriate considerations. For example, the
plausibility of arguments about the effects of a
policy depends on whether such arguments
encompass a full range of plausible rival
explanations similar in form and content to classes
of rival hypotheses (threats to validity) developed
in the tradition of quasi-experimentation.
Consonance. Elements of an argument
should be internally consistent and

compatible. For example, ethical arguments
concerning the justice or fairness of a policy
are plausible to the degree that they
incorporate a system of internally and
externally consistent ethical hypotheses.
Cohesiveness. Elements of an argument
should be operationally connected. For

example, the plausibility of an ethical
argument depends on whether responses to
several levels of descriptive and valuative
questions levels ranging from verification
and validation to vindication are
operationally linked.
Functional regularity. Elements of an argument
should conform to an expected pattern. For

example, statistical arguments that offer estimates
of parameters of unobserved (and often
unobservable) populations are plausible to the
degree that patterns in the sample and the
population from which it is drawn are functionally
regular or uniform, not irregular, based on sample
data and background knowledge.
Class Example: Excerpt from Bush

Speech on Iraq
It is clear that we need to change our strategy in Iraq. So
my national security team, military commanders, and

diplomats conducted a comprehensive review. We
consulted Members of Congress from both parties, allies
abroad, and distinguished outside experts. We benefited
from the thoughtful recommendations of the Iraq Study
Group a bipartisan panel led by former Secretary of State
James Baker and former Congressman Lee Hamilton. In
our discussions, we all agreed that there is no magic
formula for success in Iraq. And one message came
through loud and clear: Failure in Iraq would be a disaster
for the United States.

Speech on Iraq
The consequences of failure are clear: Radical Islamic
extremists would grow in strength and gain new recruits.

They would be in a better position to topple moderate
governments, create chaos in the region, and use oil
revenues to fund their ambitions. Iran would be
emboldened in its pursuit of nuclear weapons. Our enemies
would have a safe haven from which to plan and launch
attacks on the American people. On September the 11th,
2001, we saw what a refuge for extremists on the other
side of the world could bring to the streets of our own
cities. For the safety of our people, America must succeed
in Iraq.

Speech on Iraq
Many are concerned that the Iraqis are becoming too
dependent on the United States and therefore, our policy

should focus on protecting Iraq's borders and hunting
down al Qaeda. Their solution is to scale back America's
efforts in Baghdad or announce the phased withdrawal of
our combat forces. We carefully considered these
proposals. And we concluded that to step back now would
force a collapse of the Iraqi government, tear that country
apart, and result in mass killings on an unimaginable scale.
Such a scenario would result in our troops being forced to
stay in Iraq even longer, and confront an enemy that is
even more lethal.

Problem Structuring
Nature of Policy Problems

Problem structuring, which is a continuously
recurring phase of policy inquiry in which analysts

search among competing problem formulations of
different stakeholders, is no doubt the most
important activity performed by policy analysts.
It is so important because policy analysts seem to
fail more often because they solve the wrong
problem than because they get the wrong solution
to the right problem.

Beyond problem solving.
Policy analysis is a dynamic, multilevel process
in which methods of problem structuring take
priority over methods of problem solving (see
figure).
Priority of Problem Structuring

in Policy Analysis
Problem
Sensing
PROBLEM
SITUATION
Problem
Structuring
Problem
Dissolving
POLICY
PROBLEM
NO
Problem
Unsolving
RIGHT
PROBLEM?
YES
Problem
Solving
POLICY
SOLUTION
YES
POLICY
SOLUTION?
Problem
Resolving

Beyond problem solving.
Distinctions among problem-related processes.
Problem sensing versus problem structuring.
Problem structuring versus problem solving.
Problem resolving versus problem unsolving and problem
dissolving.
Characteristics of problems.
Interdependence of policy problems (policy messes).
Analytic versus holistic approaches.
Subjectivity of policy problems.

Artificiality of policy problems.
Dynamics of policy problems.

Characteristics of problems (contd.)
In short, systems of problems (messes) cannot be
decomposed into independent subsets without running
the risk of producing the right solution to the wrong
problem.
The whole is greater (qualitatively different) than the
sum of its parts.
A recognition of the interdependence, subjectivity,
artificiality, and dynamics of policy problems alerts us
to the possible unanticipated consequences that may
follow from policies based on the right solution to the
wrong problem.

Problems versus issues.
If policy problems are really systems of
problems, then policy issues are equally
complex.
Policy issues reflect not only conflict over
courses of actions, but over definitions of the
problem.

Major Issues
Secondary Issues
Functional Issues
Minor Issues

Major issues.
Those encountered at highest levels of government
within and between federal, state, and local
jurisdictions.
Involve questions of agency mission.
Secondary issues.
Located at the level of agency programs at the
federal, state, and local levels.
The setting of program priorities and the definition
of target groups and beneficiaries.

Functional issues.
Located at both the program and project levels.
Involve questions of budget, finance, and
procurement.
Minor issues.
Located at the level of specific projects.
Involve questions of personnel, staffing, employee
benefits, vacation times, working hours, and
standard operating procedures.

Strategic policies are policies where the
consequences are relatively irreversible.
Operational polices are policies where the
consequences are relatively reversible.

Three classes of policy problems.
Well-structured.
Prototype: completely computerized decision
problems.
Moderately structured.
Prototype: prisoners dilemma.
Ill-structured.
Prototype: most important problems.

STRUCTURE OF PROBLEM
Moderately
Structured
ELEMENT
Well structured
Ill Structured
Decision maker(s)
One or few
One or few
Many
Alternatives
Limited
Limited
Unlimited
Utilities (values)
Consensus
Consensus
Conflict
Outcomes
Certainty or risk
Uncertainty
Unknown
Probabilities
Calculable
Incalculable
Incalculable
Problem Structuring in Policy

Analysis
The requirements for solving ill-structured
problems demand that the analyst take an

active role in defining the problem.

Analysis
Creativity in problem structuring.
The product of the analysis is sufficiently novel that
most people could not or would not have arrived at the
same solution;
The process of analysis is sufficiently unconventional
that it involves the modification or rejection of previous
accepted ideas;
The process of analysis requires sufficiently high
motivation and persistence that analysis takes place
with high intensity or over long periods of time;

Analysis
Creativity in problem structuring (contd.).
The product of analysis is regarded as valuable
by analysts, policymakers, and other
stakeholders, since it provides an appropriate
solution to the problem; And.
The problem initially posed is so ambiguous,
value, and ill defined that part of the task is to
formulate the problem itself.

Analysis
Phases of problem structuring.
Problem search.
Problem definition.
Problem specification.
Problem sensing.

Analysis
METAPROBLEM
Problem
Definition
Problem
Search
PROBLEM
SITUATION
SUBSTANTIVE
PROBLEM
Problem
Sensing
Problem
Specification
FORMAL PROBLEM

Analysis
Errors of the third type (EIII).
How well do the substantive and formal problems
correspond to the original problem situation?
If most problem situations are messes, then models should
reflect complexity.
Types of errors.
Type I rejecting the null hypothesis when it is true.
Type II accepting the null hypothesis when it is false.
Type III solving the wrong problem.
Types of Policy Models

Policy models are simplified representations
of selected aspects of a problem situation

constructed for specific purposes.
By definition, they are artificial constructs.
Descriptive models.
The purpose of descriptive models is to explain
and/or predict the causes and consequences of
policy choices.
Used to monitor the outcomes of policy actions and
to forecast performance.
Types of Policy Models

Normative models.
Explain and predict, but also to provide rules
and recommendations for optimizing some
utility or value.
Example: compound interest.
S n (1 r ) S 0
n
Forms of Policy Models

Verbal models.
Expressed in everyday language.
The equivalent of substantive problems.
Limitation: the reasons for recommendations and
predictions may be hidden.
Symbolic models.
Use mathematical symbols to describe relationships
among key variables believed to characterize a
problem. The premises must be made explicit.
Y a bX

Table 1. Mean education and mean income
Birmingham, Alabama neighborhoods
50000
Adjusted income 2000
40000
30000
20000
10000
Rsq = 0.6369
0.0
.1
.2
.3
.4
.5
.6
.7
.8
Proportion with high school or higher
.9
1.0

Procedural models.
Represent dynamic relationships among
variables believed to characterize a policy
problem.
Example:
Decision tree.
Methods of Problem Structuring

Boundary analysis (used to estimate boundaries of
metaproblem).
Saturation sampling.
Elicitation of problem representations.
Boundary estimation.
Classificational analysis.
Uses logical division and logical classification.
Criteria.
Substantive relevance.
Exhaustiveness.
Disjointness.
Consistency.
Hierarchical distinctiveness.

Hierarchy analysis.
Possible clauses, plausible causes, and
actionable causes.
Uses same rules as classificational analysis.
Synectics.
Personal analogies.
Direct analogies.
Symbolic analogies.
Fantasy analogies.

Brainstorming.
Groups should be composed of knowledgeable
subjects.
Idea generation and idea evaluation should be kept
separate.
Atmosphere should be open and permissive.
Idea-evaluating should only begin after idea-generating
has ceased.
At the end of idea-evaluating, ideas should be
prioritized and incorporated into a proposal that
contains a conceptualization of the problem and its
potential solutions.

Multiple perspective analysis.
Technical perspective.
Organizational perspective.
Personal perspective.
Assumptional analysis.
Stakeholder identification.
Assumption surfacing.
Assumption challenging.
Assumption pooling.
Assumption synthesis.

Argumentation mapping.
Assessing probability and plausibility of policy
argument warrants.
Policy Restraint and Barriers to

Change
A rationale for stability.
Deliberation and public participation are at least as
important as rapid and efficient policymaking.
Rapid policy change can happen with the right
combination of political factors, but during normal
periods, minorities can block policy change and the
system preserves their right to do so.
Policy does not change rapidly because most of the
public does not support such change except under rare
circumstances.

Change
Fragmentation.
Fragmentation is a double-edge sword.
On the one hand, it requires multiple serial
majorities to promote change.
On the other hand, it offers multiple points of access
to the policy process.
Dimensions of fragmentation.
Separation of powers.
Division of powers (federalism).

Change
Table 2.2. The Balance of Power
Function
Congress
President
Courts
Legislative
Make laws
Recommend laws; veto

laws; make regulations
that have the force of
law.
Review laws to
determine legislative
intent, new
interpretations
Executive
Override vetoes;
legislative vetoes of
regulations
Enforce and implement

laws.
Review executive acts;

restrain executive
actions.
Judicial
Impeach judges and the

president, call witnesses
to hearings, set judicial
jurisdiction.
Pardon criminals,
nominate judges
Interpret laws.
Note: The primary function of each branch is indicated in the boxes with the diagonal lines.

Forecasting Policy Futures
Forecasting in Policy Analysis

Forecasting is a procedure for producing factual
information about future states of society on the

basis of prior information about policy problems.
Forms of forecasting.
Projection.
A forecast based on the extrapolation of current and historical
trends into the future.
Prediction.
A forecast based on explicit theoretical assumptions.
Conjecture.
A forecast based on informed or expert judgments about future
states of society.

Aims of forecasting.
Forecasts provide information about future
changes in policies and their consequences.
Forecasting permits greater control through
understanding past policies and their
consequences, implying that the future is
determined by the past.
Forecasting also enables us to shape the future
in an active manner, irrespective of what has
happened in the past.

Limitations of forecasting.
Forecast accuracy.
Recent simple forecasting models have had huge errors in
recent years (as much as 50%) in econometric forecasting.
Comparative yield.
Both simple and complex theoretical models have been no
more accurate than simple extrapolative models and informed
expert judgment.
Context.
Institutional (nonprofit more accurate than business and
government).
Temporal (long-term less accurate than short-term).
Historical (modern complexity reduces accuracy).
Assumption drag.

Types of futures.
Potential futures.
Future societal states that may occur.
Plausible futures.
Future states that, on the basis of assumptions about causation
in nature and society, are believed to be likely if policymakers
do not intervene to redirect the course of events.
Normative futures.
Potential and plausible futures which are consistent with an
analysts conception of future needs, values, and opportunities.
The specification of normative futures narrows the range of
potential and plausible futures, thus linking forecasts to
specific goals and objectives.

Table 1. Contrast between goals and objectives
CHARACTERISTIC
GOALS
OBJECTIVES
Broadly stated (. . . To
upgrade the quality of
health care)
Concrete (. . . Increase
the number of physicians
by 10 percent)
Definition of terms
Formal ( . . . The quality

of health care refers to
the accessibility of
medical services)
Operational ( . . . The
quality of health care
refers to the number of
physicians per 100,000
persons . . .)
Time period
Unspecified ( . . . In the
future)
Specified ( . . . In the
period 2006-2016)
Nonquantitative
(adequate health care
insurance)
Frequently quantitative
( . . . The number persons
covered per 1,000
persons)
Broadly defined ( . . .
People in need of care)
Specifically defined ( . . .
Families with incomes
below $19,000)
Specification of purposes
Measurement procedure
Treatment of target groups

Sources of goals, objectives, and alternatives.
Authority.
Insight.
Method.
Scientific theories.
Motivation.
Parallel case.
Analogy.
Ethical systems.
Approaches to Forecasting
Approaches to forecasting.
Decide what to forecast, or determine the object
of the forecast.
Decide how to make the forecast, or select one
or more bases for the forecast.
Choose techniques that are most appropriate for
the object and base selected.
Objects.
The object of a forecast is the point of reference
of a projection, prediction, or conjecture.
Objects of forecasting.
Consequences of existing policies.

Consequences of new policies.
Contents of new policies.
Behavior of policy stakeholders.
Bases.
The basis of a forecast is the set of assumptions
or data used to establish the plausibility of
estimates of consequences of existing or new
policies, the content of new policies, or the
behavior of policy stakeholders.
Bases (contd.).
Bases for forecasts.
Trend extrapolation (projection).
The extension into the future of trends observed in the
past.
Based on inductive logic.
Theoretical assumptions (prediction).
Systematically structured and empirically testable sets of
laws or propositions that make predictions about the
occurrence of one event based on another.
Based on deductive logic.
Informed judgments (conjecture).
Knowledge based on experience and insight, rather than
inductive or deductive reasoning.
Based on retroductive logic.
Methods and Techniques of

Forecasting
APPROACH
BASIS
APPROPRIATE
TECHNIQUES
PRODUCT
Extrapolative
forecasting
Trend extrapolation
Classical time-series analysis

Linear trend estimation
Exponential weighting
Data transformation
Catastrophe methodology
Projections
Theoretical forecasting
Theory
Theory mapping
Causal modeling
Regression analysis
Point and interval estimation
Correlational analysis
Predictions
Judgmental forecasting
Informed judgment
Conventional Delphi
Policy Delphi
Cross-impact analysis
Feasibility assessment
Conjectures

Forecasting
Classical time-series analysis.
Time series are made up four components.
Secular trend.
Seasonal variation.
Cyclical fluctuations.
Irregular movements.
Linear trend estimation.

Nonlinear time series.
Oscillations.
Cycles.
Growth curves.
Decline curves.
Catastrophes.
Time Series Example

Forecasting
Linear forecasting example.
Major disaster data set, 1953-2007.
Disaster Declarations = 9.851 + .803 * years; R2 = .592.
Where 1953 is set to zero.
1953 1971: Disaster Declarations = 13.516 + 0.340 * years;

R2 = .103.
1972 - 1988: Disaster Declarations = 71.419 - 1.551 * years;
R2 = .497.
1989 - 2007: Disaster Declarations = -9.368 + 1.281* years; R 2
= .329.
Projection for 2008 (55) using whole data set: 54.0.
Projection for 2008 (55) using post-1988 set: 61.1.

Forecasting
Theoretical forecasting.
Theory mapping.
Types of causal arguments.
Convergent arguments.
Divergent arguments.
Serial arguments.
Cyclic arguments.
Procedures for uncovering the structure of an argument.
Separate and number each assumption.
Underline the words that indicate claims (therefore,
thus, hence).
When specific words are omitted, but implied, supply the
appropriate logical indicators in brackets.
Arrange number assumptions and claims in an arrow
diagram that illustrates the causal argument or theory.

Forecasting
Example for theory mapping from Daniels
and Clark-Daniels.
http://www.csub.edu/~rdaniels/Pages%20from
%20DanielsClarkDanielsIJMED.pdf
.

Forecasting
Causal modeling and regression analysis.
The creation of theoretical models on the basis
of previous research and logic.
The testing of these models with causal analysis
methods such as path analysis and structural
equation modeling.
Example:
http://www.csub.edu/~rdaniels/Model%20of%20Pre
sidential%20Disaster%20Decision-Making.xls
.

Forecasting
Judgmental forecasting.
Delphi technique.
Principles.
Selective anonymity.
Iteration.
Informed multiple advocacy.
Polarized statistical response.
Structured conflict.
Computer conferencing.

Forecasting
Judgmental forecasting (contd.).
Delphi technique (contd.).
Steps.
Issue specification.
Selection of advocates.
Questionnaire design.
Forecasting items (probability of occurrence).
Issue items (ranking of importance).
Goal items (desirability and feasibility).
Option items (alternatives).
Analysis of first-round results.
Development of subsequent questionnaires.
Organization of group meetings.
Preparation of final report.

Forecasting
Delphi article.
http://www.csub.edu/~rdaniels/ch3b1.pdf.
The Public Policy-Making

Process
Official and Unofficial Actors and
Their Roles in Public Policy
Introduction
Political science traditions.
Institutionalism focus on texts of constitutions, laws,
and other written statements of policies and the
relationships between formal government institutions.
Behaviorism focus on political motivations of
individuals, acting singly and in groups, often through
polling, game theory, and statistical techniques.
Neo-institutionalism focus on organizations and
systems in which individuals interact and achieve
political and policy goals through explicit or implicit
rules and operating procedures.
Introduction
Main categories of actors in the policy
process.
Official actors statutory or constitutional
responsibilities.
Legislative, executive, and judiciary.
Unofficial actors participation with no

explicit legal authority.
Interest groups, media.
Legislatures
First listed branch in the federal and most
state constitutions.
Source of considerable research.
Primary function is lawmaking. Number of
bills and resolutions gives some idea of how

busy legislatures are.
Legislatures
Jurisdiction
House or
Assembly
Bills
House or
Assembly
Concurrent
Resolution
s
House or
Assembly
Joint
Resolution
s
Senate
Bills
Senate
Concurrent
Resolution
s
Senate
Joint
Resolution
s
U.S.
Congress
(20012002)
5,767
521
125
3,181
160
53
California
Legislature
(20012002)
3.283
264
66
2,277
114
56
Legislatures
Burden eased by staff.
Bills sifted by committee structure at both
the federal and state level.

Committee chairs wield significant power.
Most bills fail to move past their first
committee hurdles because they are largely
symbolic gestures.
Legislatures
Other critical functions performed by legislators
that affect public policy.
Casework activities to help constituents with

government agencies or to gain a privilege or benefit.
Supports reelection.
Oversight Monitor the implementation of public

policy.
Government Accountability Office www.gao.gov. Studies
public programs and makes recommendations to improve
efficiency, effectiveness, and accountability.
Public hearings.
Help understand issues.
Reveal shortcomings in current policies.
Make political capital.
Legislative Organization
California process.
http://www.leginfo.ca.gov/bil2lawx.html.
Most of the critical work on public policy is done
in committees, which review legislation, propose

and vote on amendments, and, in the end, decide
whether a bill will die at the committee level or be
elevated for consideration by the full body.
One of the most critical elements of legislative
organization is the organization on party lines.
Legislature Critiques of Public

Policy Process
Many people argue that legislatures are out
of touch with the people.

To understand why legislatures work as
they do, you need to understand two
elements of the legislature: the nature of the
members of the body and the organization
and nature of the branch itself.
Legislature Critiques of Public

Policy Process
The primary goal of the typical legislator is
reelection. Casework allows legislators to please

voters.
Home style and hill style.
Legislatures are decentralized institutions,
especially Congress.
Committees and subcommittees.
Decentralization and centralization of party leadership.
Issue networks and policy subsystems.
Legislatures Implications for

Policy Making
Decentralization and casework focus makes
complex and change-oriented legislation

difficult to pass.
The Executive Branch

For the sake of discussion, the executive branch
can be considered in two parts: the administration,

staff, and appointees; and the bureaucracy.
Advantages of an elected executive in the policy
process.
Veto power.
Unitary branch of government.
Media and public attention.
Informational advantage over the legislature.
The Executive Branch

Elected executive limitations.
Power to persuade.
The size of the Executive Office of the
President.
Elected executives focus on agenda-setting.
Administrative Agencies and

Bureaucrats
Characteristics of bureaucracy.
Fixed and official jurisdictional areas.
Hierarchical organization.
Written documentation.
Expert training of staff.
Career, full-time occupation.
Standard operating procedures.
Key complaints about bureaucracy.
Size.
Red tape.

Bureaucrats
What Do Government Agencies Do?
Government agencies provide services that are
uneconomical for the private sector (public
goods free-rider problem).
Public goods are indivisible and nonexclusive.
Complaints tend to focus on speed, efficiency,

and effectiveness of public service delivery.

Bureaucrats
Bureaucracy and the problem of accountability.
The key problem is the question of accountability.
Most public employees are appointed on merit, not
accountability to elected officials.
Early thinking focused on separation of politics and
administration.
Modern thinking: Agency decisions are political and in
the realm of administrative discretion.
Problem: no single, agreed-upon definition of the
public interest.
Administrative discretion: ability to make decisions
with minimal interference.
The Courts
The ability to interpret legislative and
executive actions: judicial review.

Courts are the weakest because their
authority rests on the legitimacy of the law
and their ability to argue their case.
Legislatures and executives initiate public
policy, while courts react to the practical
effects of such policies.
Unofficial Actors and Their Roles in

Public Policy
Individual citizens.
Low political participation.
Voting.
Other forms of participation: campaigning,
contacting, etc.
Despite this, citizens can be mobilized:

Recall election in California.
Generally speaking, individuals want the most

services for ourselves while paying the least
taxes for those services.

Public Policy
Interest groups.
Interest groups have been part of the political
scene since the founding.
Madison and the dangers of faction.
Since the 1960s the number of groups has

greatly expanded.
Transportation, mass communication, expansion of
government.
Few legal barriers to the creation of groups.

Public Policy
Interest groups.
The power of interest groups varies.
Knowledge, money, information.
Group size, peak associations.
Intensity, direct economic interest, ideological
commitment.
Social movements (combinations of interest
groups).

Public Policy
Types of interest groups.
Institutional versus membership groups.
Economic (private) versus public interest versus
ideological groups.
Benefits, free-rider problems.
Activities of interest groups.

Lobbying.
Campaign contributions.
Access (well-off).
Mass mobilization, protest, and litigation.
Riots and protest marches.

Public Policy
Political parties.
Functions.
Voting cues.
Transmission of political preferences.
Creation of packages of policy ideas.
Organization of the legislative branch.
Think tanks and other research organizations.

Brookings, Cato, Urban Institute, Rand, American
Enterprise Institute.
Ideological, scholarly, and methodological distinctions.

Public Policy
Communications media.
The news media are important actors in the policy
process.
Newspapers National versus regional versus local.
TV is the central news medium. Older population, networks;
younger population, cable news.
Entertainment programming can be equally important.
Movies, t.v., videogames.
Medias primary function in policy process is agendasetting. Media coverage correlates with institutional
attention.

Public Policy
Communications media.
News media are not just passive actors.
Interest try to arouse media focus.

Time and space constraints require discretion.
Profit-driven businesses.
Competitive biases of news gathering: dramatic and
narrative qualities of the story.

Public Policy
Subgovernments, issue networks, and domains.
Policy domain is the substantive area of policy over
which participants in policy-making compete and
compromise.
The political culture and legal environment influence
the domains.
Policy community inside the domain consists of the
actors actively involved in policy making in that
domain.
Iron triangles one way of organizing the policy community.
Issue networks may be more accurate description.

Public Policy
Subgovernments, issue networks, and domains.
Prying open policy networks (major corporate interests
usually dominate).
But, policy change is possible by prying open a
domain.
Focusing events.
Social movements and mobilization.
Exploiting the decentralization of American government.
Going public.

Recommending Policy Actions.
Recommendation in Policy
Analysis
The policy-analytic procedure of
recommendation enables analysts to

produce information about the likelihood
that future courses of action will result in
consequences that are valuable to some
individual, group, or society as a whole.
Analysis
The procedure of recommendation involves
the transformation of information about

policy futures into information about policy
actions that will result in valued outcomes.
Policy recommendations are normative
(advocative) rather than empirical
(descriptive) or evaluative.
Analysis
Characteristics of advocative claims.
Actionable: Advocative claims focus on actions that
make be taken to resolve a policy problem.
Prospective: Advocative claims occur prior to the time
that actions are taken.
Value laden: Advocative claims require both that
actions have the predicted consequences, but also that
those consequences have value for society.
Ethically complex: Advocative claims can be intrinsic
(valued as ends in themselves) or extrinsic (valued
because they will produce some other value).
Analysis
Simple model of choice.
Advocative claims are only possible when the
analyst is confronted by a situation of choice
between two or more alternatives.
Simple model.
The definition of a problem requiring action.
The comparison of consequences of two or more
alternatives to resolve the problem.
The recommendation of the alternative that will
result in a preferred outcome.
Analysis
Simple model of choice.
A1
O1
A2
O2
O1
>
O2
A1
Conditions for choice.

Single decision maker.
Certainty.
Immediacy of consequences.
Analysis
Complex model of choice.
Conditions.
Multiple stakeholders.
Uncertainty about outcomes.
The passage of time between actions and
consequences.
Results.
Intransitivity of choice.
Analysis
Forms of rationality.
Given the conditions of complex choice, there
are multiple forms of rationality.
Technical rationality.
Economic rationality.
Legal rationality.
Social rationality.
Substantive rationality.
Analysis
Rational-comprehensive theory.
An individual or collective decision-maker must
identify a policy problem on which there is a consensus
among all relevant stakeholders.
An individual or collective decision-maker must define
and consistently rank all goals and objectives whose
attainment would represent a resolution of the problem.
identify all policy alternatives that may contribute to
the attainment of each goal and objective.
Analysis
Rational-comprehensive theory.
forecast all consequences that will result from the
selection of each alternative.
compare each alternative in terms of its consequences
for the attainment of each goal and objective.
An individual or collective decision-maker must choose
that alternative that maximizes the attainment of
objectives.
Analysis
Disjointed incremental theory.
Consider only those objectives that differ incrementally
from the status quo.
Limit the number of consequences forecast for each
alternative.
Make mutual adjustments in goals and objectives, on
the one hand, and alternatives on the other.
Continuously reformulate problems and hence goals,
objectives, and alternatives in the course of acquiring
new information.

Monitoring Policy Outcomes
Introduction
The consequences of policy actions are
never fully known in advance and, for this

reason, it is essential to monitor policy
actions after they have occurred.
In fact, policy recommendations may be
viewed as hypotheses about the relationship
between policy actions and policy
outcomes: if action A is taken at time t1,
outcome O will result at time t2.
Monitoring in Policy Analysis

Monitoring is the policy-analytic procedure used
to produce information about the causes and

consequences of public policies.
Monitoring, since it permits analysts to describe
relationships between policy-program operations
and their outcomes, is the primary source of
knowledge about policy implementation.
Monitoring is primarily concerned about with
establishing factual premises about public policy.

Monitoring performs at least four major
functions:
Compliance: monitoring helps determine whether

the actions of program administrators, staff, and
other stakeholders are in compliance with standards
and procedures imposed by legislatures, regulatory
agencies, and professional bodies.
Auditing: monitoring helps determine whether
resources and services intended for certain target
groups and beneficiaries have actually reached
them.

Monitoring performs at least four major
functions:
Accounting: monitoring produces information

that is helpful in accounting for social and
economic changes that follow the
implementation of broad sets of public policies
and programs over time.
Explanation: monitoring also yields
information that helps to explain why the
outcomes of public polices and programs differ.

Sources of information.
Information must be:
Relevant.
Macronegative versus micropositive.
Reliable.
Observations precise and dependable.
Valid.
Information about policy outcomes should measure what
we think it is measuring.

Information on policy outcomes.
Historical Statistics of the United States
Current Opinion
Statistical Abstract of the United States
United States Census of Population by States
County and City Data Book
Congressional District Data Book
Census Use Study
National Opinion Research Center General Social

Survey
Social and Economic Characteristics of Students
Survey Research Center National Election Studies
Educational Attainment in the United States
National Clearinghouse for Mental Health

Information
Current Population Reports
National Clearinghouse for Drug Abuse Information
The Social and Economic Status of the Black

Population in the United States
National Criminal Justice Reference Service
Female Family Heads
Child Abuse and Neglect Clearinghouse Project
Monthly Labor Review
National Clearinghouse on Revenue Sharing
Handbook of Labor Statistics
Social Indicators
Congressional Quarterly
State Economic and Social Indicators
Law Digest

When information is not available from
existing sources, monitoring may be carried out
by some combination of questionnaires,
interviewing, field observation, and the use of
agency records.

Types of policy outcomes.
Consequences:
Policy outputs the goods, services, or resources received by
target groups and beneficiaries.
Policy impacts actual changes in behavior and attitudes that
result from policy outputs.
Populations:
Target groups individuals, communities, or organizations on
whom policies and programs are designed to have an effect.
Beneficiaries groups for whom the effects of policies are
beneficial or valuable.

Types of policy actions. Policy actions have
two major purposes.
Regulation actions designed to ensure

compliance with certain standards or
procedures.
Allocation actions that require inputs of
money, time, personnel, and equipment.
Regulative and allocative actions may have
consequences that are distributive or
redistributive.

Types of policy actions.
Policy inputs the resources time, money,
personnel, equipment, and supplies used to
produce outputs and impacts.
Policy processes the administrative,
organizational, and political activities and
attitudes that shape the transformation of policy
inputs into policy outputs and impacts.

POLICY ACTIONS
POLICY OUTCOMES
ISSUE AREA
Inputs
Processes
Outputs
Impacts
Criminal Justice.
Dollar
expenditures for
salaries,
equipment,
maintenance.
Illegal arrests as
percentages of
total arrests.
Criminals arrested
per 100,000
known crimes.
Criminals
convicted per
100,000 known
crimes.
Municipal
Services.
Dollar
expenditures for
sanitation workers
and equipment.
Morale among
workers.
Total residences
served.
Cleanliness of
streets.
Social welfare.
Number of social
workers.
Rapport with
welfare recipients.
Welfare cases per

social worker.
Standard of living
of dependent
children.

Definitions and indicators.
Variables versus constants.
Definitions.
Constitutive definitions gives meaning to words used to
describe variables by using synonymous words.
Provide no concrete rules or guidelines for actually
monitoring changes.
Operational definitions gives meaning to a variable by
specifying the operations necessary to experience and measure
it.
Indicators directly observable characteristics.
Approaches to Monitoring
APPROACH
TYPES OF
CONTROL
TYPE OF
INFORMATION
REQUIRED
Social systems accounting
Quantitative
Available and/or new

information
Social experimentation
Direct manipulations and

quantitative
New information
Social auditing
Quantitative and/or
qualitative
New information
Research and practice

synthesis
Quantitative and/or
qualitative
Available information
Approaches to Monitoring
MANIPULABLE ACTIONS
POLICY
INPUTS
POLICY
PROCESSES
CONTROLLED OUTCOMES
POLICY
OUTPUTS
POLICY
IMPACTS
In1
P1
O1
Im1
In2
.
.
.
Ini
P2
.
.
.
Pj
O2
.
.
.
Om
Im2
.
.
.
Imn
PC1
E1
SES 1
PC2
.
.
.
PCp
E2
.
.
.
Eq
SES 2
.
.
.
SES r
PRECONDITIONS
UNFORESEEN
EVENTS
UNMANIPULABLE CAUSES
SIDE-EFFECTS
AND
SPILLOVERS
UNCONTROLLED EFFECTS
Social Systems Accounting

An approach and set of methods that permit
analysts to monitor changes in objective

and subjective social conditions over time.

The major analytic element of social
systems accounting is the social indicator.

Statistics that measure social conditions and
changes therein over time for various segments
of a population. By social conditions, we mean
both the external (social and physical) and the
internal (subjective and perceptional) contexts
of human existence in a given society.
Social systems accounting

Representative social indicators.
AREA
INDICATOR
Health and illness
Persons in state mental hospitals
Public safety
Persons afraid to walk alone at night
Education
High school graduates aged 25 and older
Employment
Labor force participation by women
Income
Percent of population below the poverty line
Housing
Households living in substandard units
Leisure and recreation
Average annual paid vacation in manufacturing
Population
Actual and projected population
Government and politics
Quality of public administration
Social values and attitudes
Overall life satisfaction and alienation
Social mobility
Change from fathers occupation
Physical environment
Air pollution index
Science and technology
Scientific discoveries

Must assume that the reasons for indicator change
are related to policy actions. This is not always

the case.
Advantages.
Identify areas of insufficient information.

When indicators provide reliable information, possible
to modify policies.

Limitations.
Choice of indicators influenced by values of analysts.
Social indicators frequently do not cover manipulable
policy actions.
Most social indicators are based available data about
objective (rather than subjective) social conditions.
Social indicators provide little information about how
inputs are transformed into outcomes.
Coefficientsa
Breaks in
disaster patterns
1950 - 1971
Model
1
1972 - 1988
1989 - 2006
(Constant)
Number of years
(Constant)
Number of years
(Constant)
Number of years
Unstandardized
Coefficients
B
Std. Error
13.516
2.561
.340
.243
71.419
11.060
-1.551
.403
-6.745
22.133
1.218
.494
a. Dependent Variable: Total Disaster Declarations
Standardized
Coefficients
Beta
.322
-.705
.525
t
5.277
1.400
6.458
-3.849
-.305
2.465
Sig.
.000
.179
.000
.002
.764
.025
Social Experimentation
Use of social indicators often leads to
random innovation rather than systematic

change.
Social experimentation is the process of
systematically manipulating policy actions
in way that permits more or less precise
answers to questions about the sources of
change in policy outcomes.
Characteristics and procedures.
Direct control of experimental treatment.
Comparison (control) groups.
Random assignment.
Goal: internal validity the capacity of
experiments and quasi-experiments to make

valid causal inferences about the effects of
actions on outcomes.
Threats to internal validity.
History.
Maturation.
Instability.
Instrumentation and testing.
Mortality.
Selection.
Regression artifacts.
Social experimentation is weakest in the
area of external validity or generalizability.

Neglects policy processes.
Seattle-Denver Income Maintenance

Experiment
A social experiment is a field test of one or
more social programs or, to use the

phraseology of the natural sciences, a test of
one or more "treatments."
A social experiment is a field test in the
sense that families or individuals are
actually enrolled in a pilot social program
offering some type of special benefit or
service.

Experiment
It is "experimental" in the sense that
families or individuals are enrolled in each

of the tested programs on the basis of a
random assignment process, for example,
the flip of a coin.

Experiment
To draw conclusions about the effects of the
treatment, it is necessary to collect

information about the people who are
enrolled in each experimental program and
about those who receive no special
treatment (called the control group), and
then to compare them on the basis of the
collected information.

Experiment
The primary focus of SIME/DIME the
effect of income maintenance on work

effort or labor supply was a good subject
for experimentation .
An increase in unearned income or a
decrease in the effective wage rate were
hypothesized to lead to decreases in labor
supply.

Experiment
In the case of SIME/DIME, the experiment
was in fact designed to test the effect of two

different kinds of social programs on
participant work effort.
The two policies were a variety of cash
transfer or negative income tax programs
and several combinations of job counseling
and education or training subsidy programs.

Experiment
Any observed effects of the experiment was
interpreted as the differential effects of the

experimental treatment compared to
existing government programs.
Thus any observed experimental-control
differences in outcomes must be interpreted
as estimates of the effect of replacing the
early 1970s status quo with the
experimental programs.

Experiment
Table 4.
Labor Supply Response of Husbands:
A. Overall NIT response
(Percentage difference in annual hours worked)
Year
Sample Group
1
2
3
4
5
3-year sample
-1.6
-7.3
-7.3
-0.5
-0.2
5-year sample
-5.9 -12.2 -13.2 -13.6 -12.3
Total Sample
-3.1
-9
-9.3

Experiment
Table 5.
Labor Supply Response of Wives:
Year
Sample Group
1
2
3
4
5
-4
-16.5 -15.2 -2
+13.4
3-year sample
-15.1 -26.5 -21.6 -27.1 -24
5-year sample
-8.1 -20.1 -17.4
Total Sample

Experiment
Table 6.
Labor Supply Response of Female Heads:
Year
Sample Group
1
2
3
4
5
-5.5 -14.1 -21.6 -8.9
-7.7
3-year sample
-7.9 -15
-21.2 -28.3 -31.8
5-year sample
-6.3 -14.3 -21.4
Total Sample
Social Auditing
One of the limitations of social systems
accounting and social experimentation is

that both approaches neglect or
oversimplify policy processes.
Social Auditing
Social auditing explicitly monitors relations
among inputs, processes, outputs, and

impacts in an effort to trace policy inputs
from disbursement to final recipient.
Social auditing helps to determine whether
ineffective outcomes are the result of
inadequate inputs or processes that divert
resources or services from intended target
groups or beneficiaries.
Social Auditing
Processes monitored of two main types:
Resource diversion (from target or beneficiary
groups).
Resource transformation (changes in meaning
of policy actions from administrator to
recipient).
Social Audit Steps

The three phases of a social audit
Phase 1: Design and data collection
Clarify the strategic focus
Design instruments, pilot test
Collect information from households and key
informants in a panel of representative
communities
Social Audit Steps

Phase 2: Evidence-based dialogue and
analysis
Link household data with information from
public services
Analyze findings in a way that points to action
Take findings back to the communities for their
views about how to improve the situation
Bring community members into discussion of
evidence with service providers/planners.
Social Audit Steps

Phase 3: Socialization of evidence for
public accountability
Work-shopping
Communication strategy
Evidence-based training of planners and service
providers
Media training
Partnerships with civil society
Research and practice synthesis.

An approach to monitoring that involves the
systematic compilation, comparison, and

assessment of the results of past efforts to
implement public policies.
Two primary sources of available information.
Case studies of policy formulation and implementation.
Research reports that address relations among policy
actions and outcomes.
Research and practice synthesis.

Two methods.
Case survey method a set of procedures used
to identify and analyze factors that account for
variations in the adoption and implementation
of policies.
Requires case coding scheme, a set of categories
that capture key aspects of policy inputs, processes,
outputs, and impacts.
Research and practice synthesis

Two methods.
Research survey method a set of procedures
used to compare and appraise results of past
research on policy actions and outcomes.
Yields empirical generalizations about sources of
variation in policy outcomes, summary assessments
of the confidence researchers have in these
generalizations, and policy alternatives or action
guidelines that are implied in these generalizations.
Requires the construction of a format for extracting
information about research reports.

Advantages.
Comparatively efficient way to compile and
appraise an increasingly large body of cases
and research reports on policy implementation.
The case survey method is one among several
ways to uncover different dimensions of policy
processes that affect policy outcomes.
The case survey method is a good way to
examine both objective and subjective
conditions.

Limitations.
All related to reliability and validity of
information.
Research and Practice Synthesis:

Example
David H. Greenberg; Charles
Michalopoulos; Philip K. Robins.

Do Experimental and Nonexperimental
Evaluations Give Different Answers
about the Effectiveness of GovernmentFunded Training Programs?

Example
Abstract
This paper uses meta-analysis to investigate
whether random assignment (or experimental)

evaluations of voluntary government-funded
training programs for the disadvantaged have
produced different conclusions than
nonexperimental evaluations.
Information includes several hundred estimates
from 31 evaluations of 15 programs that operated
between 1964 and 1998.

Example
The results suggest that experimental and
nonexperimental evaluations yield similar

conclusions about the effectiveness of
training programs, but that estimates of
average effects for youth and possibly men
might have been larger in experimental
studies.
Program Evaluation
Source
Basic Guide to Program Evaluation
Written by
Carter McNamara, MBA, PhD, Authenticity Co
nsulting, LLC
. Copyright 1997-2006.
Adapted from the
Field Guide to Nonprofit Program Design, Mark
eting and Evaluation
.
A Brief Introduction ...

Note that the concept of program evaluation can
include a wide variety of methods to evaluate

many aspects of programs in nonprofit or forprofit organizations.
However, personnel do not have to be experts in
these topics to carry out a useful program
evaluation. The "20-80" rule applies here, that
20% of effort generates 80% of the needed results.
It's better to do what might turn out to be an
average effort at evaluation than to do no
evaluation at all.
Some Myths about Program

Evaluation
Many people believe evaluation is a useless
activity that generates lots of boring data

with useless conclusions.
More recently (especially as a result of
Michael Patton's development of utilizationfocused evaluation), evaluation has focused
on utility, relevance and practicality at least
as much as scientific validity.

Evaluation
Many people believe that evaluation is about
proving the success or failure of a program.

This myth assumes that success is implementing
the perfect program and never having to hear from
employees, customers or clients again -- the
program will now run itself perfectly.
This doesn't happen in real life.
Success is remaining open to continuing feedback
and adjusting the program accordingly.
Evaluation gives you this continuing feedback.

Evaluation
Many believe that evaluation is a highly unique and
complex process that occurs at a certain time in a certain

way, and almost always includes the use of outside
experts.
Many people believe they must completely understand
terms such as validity and reliability.
They don't have to.
They do have to consider what information they need to
make current decisions about program issues or needs.
And they have to be willing to commit to understanding
what is really going on.
So What is Program Evaluation?

First, we'll consider "what is a program?"
Typically, organizations work from their mission to
identify several overall goals which must be reached to

accomplish their mission.
In public agencies and nonprofits, each of these goals often
becomes a program.
Public and nonprofit programs are organized methods to
provide certain related services to constituents, e.g.,
clients, customers, patients, etc.
Programs must be evaluated to decide if the programs are
indeed useful to constituents.
So What is Program Evaluation?

Program evaluation is carefully collecting
information about a program or some aspect of a

program in order to make necessary decisions
about the program.
Don't worry about what type of evaluation you
need or are doing -- worry about what you need to
know to make the program decisions you need to
make, and worry about how you can accurately
collect and understand that information.
Where Program Evaluation is

Helpful
Understand, verify or increase the impact of
products or services on customers or clients.

Improve delivery mechanisms to be more
efficient and less costly.
Verify that you're doing what you think
you're doing.
Other Reasons to Conduct

Evaluations
Facilitate management's really thinking
about what their program is all about,

including its goals, how it meets it goals
and how it will know if it has met its goals
or not.
Produce data or verify results that can be
used for public relations and promoting
services in the community.
Other Reasons to Conduct

Evaluations
Produce valid comparisons between
programs to decide which should be

retained, e.g., in the face of pending budget
cuts.
Fully examine and describe effective
programs for duplication elsewhere.
Basic Ingredients: Organization

and Program(s)
You Need An Organization:
This may seem too obvious to discuss, but before an
organization embarks on evaluating a program, it
should have well established means to conduct itself as
an organization.
You Need Program(s):
To effectively conduct program evaluation, you should
first have programs. That is, you need a strong
impression of what your customers or clients actually
need.
Next, you need some effective methods to meet each of
those goals. These methods are usually in the form of
programs.
Programs
Inputs are the various resources needed to
run the program, e.g., money, facilities,

customers, clients, program staff, etc.
The process is how the program is carried
out.
Programs
The outputs are the units of service, e.g., number
of customers serviced, number of clients

counseled, children cared for, artistic pieces
produced, or members in the association.
Outcomes are the impacts on the customers or on
clients receiving services, e.g., increased mental
health, safe and secure development, richer artistic
appreciation and perspectives in life, increased
effectiveness among members, etc.
Planning Your Program

Evaluation
Depends on what information you need to
make your decisions and on your resources.

Your program evaluation plans depend on what
information you need to collect in order to make
major decisions.
But the more focused you are about what you want to
examine by the evaluation, the more efficient you can
be in your evaluation, the shorter the time it will take
you and ultimately the less it will cost you.
Key Considerations
For what purposes is the evaluation being
done, i.e., what do you want to be able to

decide as a result of the evaluation?
Who are the audiences for the information
from the evaluation, e.g., customers,
bankers, funders, board, management, staff,
customers, clients, citizens, etc.
Key Considerations
What kinds of information are needed to make the
decision you need to make and/or enlighten your

intended audiences, e.g., information to really
understand the process of the product or program
(its inputs, activities and outputs), the customers
or clients who experience the product or program,
strengths and weaknesses of the product or
program, benefits to customers or clients
(outcomes), how the product or program failed
and why, etc.
Key Considerations
From what sources should the information be
collected, e.g., employees, customers, clients,

groups of customers or clients and employees
together, program documentation, etc.
How can that information be collected in a
reasonable fashion, e.g., questionnaires,
interviews, examining documentation, observing
customers or employees, conducting focus groups
among customers or employees, etc.
Key Considerations
When is the information needed (so, by
when must it be collected)?

What resources are available to collect the
information?
Goals-Based Evaluation
1. How were the program goals (and objectives, is
applicable) established? Was the process

effective?
2. What is the status of the program's progress
toward achieving the goals?
3. Will the goals be achieved according to the
timelines specified in the program
implementation or operations plan? If not, then
why?
4. Do personnel have adequate resources (money,
equipment, facilities, training, etc.) to achieve the
goals?
Goals-Based Evaluation
5. How should priorities be changed to put more focus on
achieving the goals? (Depending on the context, this

question might be viewed as a program management
decision, more than an evaluation question.)
6. How should timelines be changed (be careful about
making these changes - know why efforts are behind
schedule before timelines are changed)?
7. How should goals be changed (be careful about making
these changes - know why efforts are not achieving the
goals before changing the goals)? Should any goals be
added or removed? Why?
8. How should goals be established in the future?
Process-Based Evaluations
1. On what basis do employees and/or the
customers decide that products or services

are needed?
2. What is required of employees in order to
deliver the product or services?
3. How are employees trained about how to
deliver the product or services?

4. How do customers or clients come into
the program?
5. What is required of customers or client?
6. How do employees select which products
or services will be provided to the customer
or client?
7. What is the general process that
customers or clients go through with the

product or program?
8. What do customers or clients consider to
be strengths of the program?
9. What do staff consider to be strengths of
the product or program?
10. What typical complaints are heard from
employees and/or customers?
11. What do employees and/or customers
recommend to improve the product or

program?
12. On what basis do employees and/or the
customer decide that the product or services
are no longer needed?
Outcome-Based Evaluations
Identify the major outcomes that you want to
examine or verify for the program under

evaluation.
Choose the outcomes that you want to examine,
prioritize the outcomes and, if your time and
resources are limited, pick the top two to four
most important outcomes to examine for now.
For each outcome, specify what observable
measures, or indicators, will suggest that you're
achieving that key outcome with your clients.
Outcome-Based Evaluations
Specify a "target" goal of clients, i.e., with
what number or percent of clients you

commit to achieving specific outcomes?
Identify what information is needed to show
these indicators.
Decide how can that information be
efficiently and realistically gathered.
Analyze and report the findings.
Methods to Collection Information

Questionnaires, surveys, checklists.
Interviews.
Documentation review.
Observation.
Focus groups.
Case studies.
Ethics: Informed Consent from

Program Participants
Note that if you plan to include in your evaluation,
the focus and reporting on personal information

about customers or clients participating in the
evaluation, then you should first gain their consent
to do so.
They should understand what you're doing with
them in the evaluation and how any information
associated with them will be reported.
Ethics: Informed Consent from

Program Participants
You should clearly convey terms of
confidentiality regarding access to

evaluation results.
They should have the right to participate or
not.
Have participants review and sign an
informed consent form.
Selecting Which Methods to Use

The overall goal in selecting evaluation method(s) is to get
the most useful information to key decision makers in the

most cost-effective and realistic fashion.
Note that, ideally, the evaluator uses a combination of
methods, for example, a questionnaire to quickly collect a
great deal of information from a lot of people, and then
interviews to get more in-depth information from certain
respondents to the questionnaires.
Perhaps case studies could then be used for more in-depth
analysis of unique and notable cases, e.g., those who
benefited or not from the program, those who quit the
program, etc.

1. What information is needed to make current
2.
3.
4.
5.
decisions about a product or program?

Of this information, how much can be collected
and analyzed in a low-cost and practical manner,
e.g., using questionnaires, surveys and checklists?
How accurate will the information be (reference
the above table for disadvantages of methods)?
Will the methods get all of the needed
information?
What additional methods should and could be used
if additional information is needed?

6. Will the information appear as credible to
decision makers, e.g., to funders or top

management?
7. Will the nature of the audience conform to the
methods, e.g., will they fill out questionnaires
carefully, engage in interviews or focus groups,
let you examine their documentations, etc.?
8. Who can administer the methods now or is
training required?
9. How can the information be analyzed?
Four Levels of Evaluation:

There are four levels of evaluation information
that can be gathered from clients, including

getting their:
1. reactions and feelings (feelings are often poor

indicators that your service made lasting impact)
2. learning (enhanced attitudes, perceptions or
knowledge)
3. changes in skills (applied the learning to enhance
behaviors)
4. effectiveness (improved performance because of
enhanced behaviors)
Four Levels of Evaluation:

Usually, the farther your evaluation
information gets down the list, the more

useful is your evaluation. Unfortunately, it
is quite difficult to reliably get information
about effectiveness. Still, information
about learning and skills is quite useful.
Analyzing and Interpreting

Information
Always start with your evaluation goals:
When analyzing data (whether from
questionnaires, interviews, focus groups, or
whatever), always start from review of your
evaluation goals, i.e., the reason you undertook
the evaluation in the first place. This will help
you organize your data and focus your analysis.

Information
Basic analysis of "quantitative" information (for
information other than commentary, e.g., ratings,

rankings, yes's, no's, etc.):
1. Make copies of your data and store the master copy
away. Use the copy for making edits, cutting and
pasting, etc.
2. Tabulate the information, i.e., add up the number of
ratings, rankings, yes's, no's for each question.
3. For ratings and rankings, consider computing a mean,
or average, for each question.
4. Consider conveying the range of answers, e.g., 20
people ranked "1", 30 ranked "2", and 20 people
ranked "3".

Information
Basic analysis of "qualitative" information
(respondents' verbal answers in interviews, focus

groups, or written commentary on
questionnaires):
1. Read through all the data.
2. Organize comments into similar categories.
3. Label the categories or themes, e.g., concerns,
suggestions, etc.
4. Attempt to identify patterns, or associations and causal
relationships in the themes.
5. Keep all commentary for several years after
completion in case needed for future reference.
Interpreting Information:
1. Attempt to put the information in perspective.
a)
b)
c)
d)
e)
f)
Compare results to what you expected;

Management or program staff
Common standards for your services
Original program goals (program evaluation)
Indications of accomplishing outcomes (outcomes evaluation)
Description of the program's experiences, strengths, weaknesses,
etc. (process evaluation).
2. Consider recommendations to help program staff improve
the program, conclusions about program operations or

meeting goals, etc.
3. Record conclusions and recommendations in a report
document, and associate interpretations to justify your
conclusions or recommendations.
Reporting Evaluation Results

1. The level and scope of content depends on
to whom the report is intended,

2. Be sure employees have a chance to
carefully review and discuss the report.
Translate recommendations to action plans,
including who is going to do what about the
program and by when.
Reporting Evaluation Results

3. Bankers or funders will likely require a report that
includes an executive summary (this is a summary of

conclusions and recommendations, not a listing of what
sections of information are in the report -- that's a table of
contents); description of the organization and the program
under evaluation; explanation of the evaluation goals,
methods, and analysis procedures; listing of conclusions
and recommendations; and any relevant attachments,
4. Be sure to record the evaluation plans and activities in an
evaluation plan which can be referenced when a similar
program evaluation is needed in the future.
Pitfalls to Avoid
1. Don't balk at evaluation because it seems far too
"scientific." It's not. Usually the first 20% of effort

will generate the first 80% of the plan, and this is
far better than nothing.
2. There is no "perfect" evaluation design. Don't
worry about the plan being perfect. It's far more
important to do something, than to wait until
every last detail has been tested.
Pitfalls to Avoid
3. Work hard to include some interviews in your evaluation
methods. Questionnaires don't capture "the story," and the

story is usually the most powerful depiction of the
benefits of your services.
4. Don't interview just the successes. You'll learn a great
deal about the program by understanding its failures,
dropouts, etc.
5. Don't throw away evaluation results once a report has
been generated. Results don't take up much room, and
they can provide precious information later when trying
to understand changes in the program.

Example
The results also suggest that variation
among nonexprimental estimates of

program effects is similar to variation
among experimental estimates for men and
youth, but not for women (for whom it
seems to be larger), although small sample
sizes make the estimated differences
somewhat imprecise for all three groups.

Example
The policy implications of the findings are
discussed.
2006 by the Association for Public Policy
Analysis and Management
Techniques for Monitoring

TABULAR
DISPLAYS
INDEX
NUMBERS
INTERRUPTED
TIMESERIES
ANALYSIS
CONTROLSERIES
ANALYSIS
REGRESSION DISCONTINUITY
ANALYSIS
APPROACH
GRAPHIC
DISPLAYS
Social systems
accounting
Social auditing
Social
experimentation
Research and
practice
synthesis
Analysis
Disjointed incremental theory.
Analyze and evaluate alternatives in a sequence of
steps, such that choices are continuously amended over
time, rather than made at a single point in time.
Continuously remedy existing social problems, rather
than solve problems completely at one point in time.
Share responsibilities for analysis and evaluation with
many groups in society, so that the process of making
choices is fragmented or disjointed.
Analysis
Arrows impossibility theorem.
It is impossible for democratic decision makers
in a democratic society to meet conditions of
the rational comprehensive model.
Individual choices cannot be aggregated through
majority voting procedures to create a collective
decision that will produce a single best solution for
all parties.
The Voters Paradox

COMMITTEE MEMBER
Brown (criterion: risk)
PREFERENCE
A (solar) preferred to B (coal)
B (coal) preferred to C (nuclear)
A (solar preferred to C (nuclear)
Jones (criterion: feasibility)
B preferred to C
C preferred to A
B preferred to A
Smith (criterion: efficiency)
C preferred to A
A preferred to B
C preferred to B
Majority (intransitive and cyclic)
A preferred to B
B preferred to C
C preferred to A
Analysis
Arrows impossibility theorem.
Reasonable conditions for democratic decision
procedures.
Nonrestriction of choices.
Nonperversity of collective choice.
Independence of irrelevant alternatives.
Citizens sovereignty.
Nondictatorship.
Analysis
Bounded rationality.
Decision makers engage in satisficing behavior
(identify courses of action that are good
enough.).
Consider the most evident alternatives that produce
a reasonable increase in benefits.
Rationality as constrained maximization.

Rational choice within the boundaries of
constraint.
Analysis
Criteria for policy recommendation.
Effectiveness does a given alternative result in the
achievement of a valued outcome (technical
rationality).
Efficiency the amount of effort needed to produce a
given level of effectiveness (economic rationality).
Adequacy the extent to which any given level of
effectiveness satisfies the needs, values, or
opportunities that gave rise to the problem.
Fixed costs and variable effectiveness (type I, maximize
effectiveness).
Fixed effectiveness and variable costs (type II, minimize
costs).
Variable costs and variable effectiveness (type III, efficiency).
Fixed costs and fixed effectiveness (type IV, do nothing).
Analysis
Criteria for policy recommendation.
Equity the distribution of effects and effort among
different groups in society (legal and social rationality).
Maximize individual welfare.

Protect minimum welfare.
Maximize net welfare.
Maximize redistributive welfare.
Responsiveness satisfies the needs, preferences, or

values of particular groups.
Appropriateness the value or worth of a programs
objectives and the tenability of assumptions underlying
these objectives.
Approaches to Recommendation
Public versus private choice.
Nature of public policy processes.
Numerous stakeholders with conflicting values.
Collective nature of public policy goals.

Multiple conflicting criteria for choice.
Nature of public goods.

Specific, collective, and quasi-collective goods.
Supply and demand (market mechanisms).

Discuss.
Public choice.
Problems with supply demand models of
public policy.
Multiple legitimate stakeholders.

Collective and quasi-collective goods.
Limited comparability of income measures.
Public responsibility for social costs and benefits.
Benefit-cost analysis.
Characteristics.
Measure all costs and benefits to society of a program
including intangibles.
Traditional benefit-cost analysis emphasizes economic
rationality: net benefits are greater than zero and higher than
alternative uses.
Traditional benefit-cost analysis uses the private marketplace
as the point of departure in recommending programs.
Social benefit-cost analysis also measures redistributional
benefits.
Types of costs and benefits.
Inside versus outside costs and benefits.
Tangible versus intangible costs and benefits.
Direct versus indirect costs and benefits.
Net efficiency versus redistributional benefits.
Tasks in benefit-cost analysis.
Problem structuring.
Specification of objectives.
Identification of alternative solutions.
Information search, analysis, and interpretation.
Identification of target groups and beneficiaries.
Estimation of costs and benefits.
Discounting of costs and benefits.
Estimation of risk and uncertainty.
Choice of decision criterion.
Recommendation.
Benefit-cost and Costeffectiveness Analysis

Benefit-cost analysis is an applied branch of
economics that attempts to assess service

programs by determining whether the total societal
welfare has increased (in aggregate more people
have been made better off) because of the project
or program.
Steps.
Determine the benefits of a proposed or existing
program and place a dollar value on those benefits.
Calculate the total costs of the program.
Compare the benefits and costs.

Simple steps pose real challenge, especially
estimating intangible benefits.

Procedure still useful in uncovering
assumptions and estimating value of
intangibles.

Cost-effectiveness analysis.
The major costing alternative to benefit-cost analysis.
Relates the cost of a given alternative to specific measures of
program objectives.
For example, dollar per life saved on various highway
safety programs.
Often the first step in a benefit-cost analysis.

Especially useful if analyst cannot quantify benefits,
but has fairly specific program objectives.
Key problem: situation where there are multiple
benefits. Results often very subjective.
2nd key problem: does not produce a bottom line
number.

A private sector analogy.
Benefit-cost analysis similar to financial analysis in
private sector.
Should the firm have done the project at all, i.E., Is the project
producing a satisfactory rate of return?
Public version: is the program a success, i.E., Has it improved
social welfare?
What other options are there for the use of the firms
resources?
Public version: should the program be continued when
weighed against alternative uses for the governments funds.

A private sector analogy.
Benefits and costs do not occur simultaneously
in either private or public sector.
R&D costs, marketing, capital investment, training.
Government not priced, thus benefits are more
broadly defined.
Alternative uses of resources (opportunity

costs).

Benefit-cost illustration.
Costs
Year
R&D
Capital
O&M
Total
Benefits
$1500
$2000
$0
$3500
$0
500
2000
2000
4500
3500
2000
2500
4500
5500
2000
3000
5000
6500
2000
3500
5500
8500
$10000
$11000
$23000
$24000
Totals
Year
$2000
Benefit-Costs
Present Value, Benefits-Costs (10%)
($3500)
($3500)
(1000)
(909)
1000
826
1500
1127
3000
2049
$1000
($407)
Totals

Formula for net present value.
y2
y2
y3
y3
yx
yx
B
C
B
C
B
C
NPV B y1 C y1
...
2
1 r
(1 r )
(1 r ) x 1
Formula sensitive to choice of rate of return.

Could also use return on investment.
Discount rate that would make the present value zero.

Continuing or not continuing the project.
Costs
Year
6
R&D
Capital
$500
O&M
Total
Benefits
$400
$4500
$5400
$9000
400
5000
5400
8500
400
6000
6400
8500
400
7500
7900
8000
10
400
8000
8400
7500
$2000
$31000
$33500
$41500
Totals
Year
$500
Benefit-Costs
Present Value, Benefits-Costs (10%)
$3600
3600
3100
2818
2100
1736
100
75
(900)
(615)
$8000
$7614
10
Totals

Total versus marginal benefits and costs.
When considering the overall profitability of a project,
an agency will consider the total costs involved in
getting the project started through its operations cycle.
But at any point in time, when an agency is continuing
or discontinuing a project or program, it only considers
the marginal costs or benefits.
Definitions.
Marginal cost: incremental cost of producing one more unit of
output.
Marginal benefit: incremental benefit of that one unit of
output.
Public sector usually does not do it on the basis of a

single unit, but what are the benefits being generated
now versus the costs now?

Public-private sector differences and
similarities.
Similarities alternative uses for funds, onetime costs, recurring costs, land, labor, capital.
Differences.
Distributional considerations.
Spillovers.
Framework for Analysis

Benefit
Indicator
Measure
Dollar value
Assumptions
Indicator
Measure
Dollar value
Assumptions
Real
Direct
Tangible
Intangible
Indirect
Tangible
Intangible
Cost
Real
Direct
Tangible
Intangible

Real benefits and costs versus transfers.
Real: net gains and losses to society.
Transfers: merely alter the distribution of
resources in society.

Direct and indirect benefits and costs.
Direct costs and benefits are closely related to
the primary objectives of the project.
Direct costs personnel, facilities, equipment,
material, project administration.
Indirect costs and benefits are byproducts,

multipliers, spillovers, or investment effects.
Indirect costs are unintended costs that result from
government action.
Indirect benefits might include benefits of space
exploration.

Tangible and intangible benefits and costs.
Tangible benefits and costs can be converted
readily into dollar figures.
Intangible benefits and costs are those things
that cannot be directly assigned an explicit
price.
Determining the geographic scope of
analysis.
Spillover effects may determine true

geographic jurisdiction.

(Agricultural Dam Example)
Real Benefits
Nature of Benefit/Cost
Direct
Tangible
Increased farm output

New supply of water
Intangible
Maintaining family farms
Indirect
Tangible
Reduced soil erosion
Intangible
Preservation of rural society
Real Costs
Direct
Tangible
Construction material, labor, operations and maintenance, direct program supervision by agency
Intangible
Loss of recreational value of land or river
Indirect
Tangible
Administrative overhead of government

Diversion of water and its effects
Increased salinity
Intangible
Loss of wilderness area
Transfers
Relative improvement of profit for farm implement industry
Measuring Benefits
Evaluation problem difficult for
government because of multiple benefits

and intangibles.
Measuring Benefits
Sources of data.
Existing records and statistics kept by agency.
Feedback from clients.
Ratings by trained observers.
Experience of other governments or private or
nonprofit corporations.
Special data gathering.
Whenever possible analyst should use
market value or willingness to pay.
Measuring Benefits
Valuing benefits.
Cost savings.
Time saved.
Lives saved.
Increased productivity or wages.
Recreational benefits.
Land values.
Alternatives to market prices.
See handout.
Measuring Costs
Cost categories.
One-time, fixed, or up-front costs.
Ongoing investment costs.
Recurring costs.
Compliance costs.
Mitigation measures.
Measuring Costs
Valuing indirect costs.
Flat overhead figure.
Does the project actually costs increased administrative
burden?
Costs to the private sector.

Valuing the use of capital.
Valuing the damage effects of government programs.
Other cost issues.
Sunk costs.
Interest costs.
Analysis of Benefits and Costs

Framework.
Retrospective.
Snapshot.
Prospective.

Importance of using present value.
Choice of an appropriate discount rate.
Private sector rate.
Low social discount rate.
Long-term treasury bill rates.
Adjustment for inflation.

Presenting the results.
Net present value (B-C).
Benefit cost ratio (B/C).
Return on investment (discount rate to reduce
present value to zero).
Appropriate perspective.
Costs and benefits vary across stakeholders.
May conduct analyses from several
perspectives.

Sensitivity analysis.
Use spreadsheet analysis to vary the assumptions.
Intangibles.
Relate intangibles to dollar results: if there are net
costs, do the intangible benefits overcome the deficit?
Particular problems.
Equity concerns (weighting of values).
Multiple causation and co-production problems.

Process
How to Deliver Oral Testimony
Based on a Written Statement
Introduction
Goal:
To speak authoritatively and to answer
questions responsively in public deliberation.
Objective:
Skill of writing speakable text, skill of speaking
easily from written text, and readiness to
answer anticipated and unanticipated questions.
Introduction
Scope:
Pinpointed topic pertinent to hearings purpose and the
witnesss role.
Product:
Two expected communication products:
Short oral summary.
Full written statement.
Strategy:
Confident and useful public testimony resulting from
advance preparation.

Process
How to Inform Policy Makers in a
Briefing Memo or Opinion Statement
Introduction
Goal:
Recognition of meaningful information in a mass of details
and representation highlighting the significance of
information for a user.
Objective:
Skills of distilling, listening, recording, observing,
evaluating sources, relating details to context, interpreting
details accurately in context, and selecting details according
to relevance; capability of stating informed opinion that is
aware of and responsive to other opinions.

Topic 1-9 - PolicyAnalysis-all Topics

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Topic 1-9 - PolicyAnalysis-all Topics

Transféré par

Droits d'auteur :

Formats disponibles

Public Policy Analysis

Introduction to the Public Policy

Policy Analysis in the PolicyMaking Process

knowledge of and in the policy-making process.

The Process of Policy Inquiry

Problem solving is a key element of the

methodology of policy analysis

The Methodology of Policy

from and integrates elements of multiple

The Methodology of Policy

knowledge about causes and consequences

The Methodology of Policy

among competing values such as health, wealth,

Policy analysis also seeks to create knowledge that

improves the efficiency of choices among

The Methodology of Policy

creating, critically assessing, and

The Methodology of Policy

plausibly true beliefs, as distinguished from

The Methodology of Policy

core of moderately coherent underlying

The Methodology of Policy

social problems over the past 40 or more

The Methodology of Policy

plausibility, not certainty, is the defining

The Methodology of Policy

The Methodology of Policy

Multiple analytic synthesis.

The Methodology of Policy

critical multiplism (contd.)

Multiple stakeholder analysis.

Multiple perspective analysis.

The Methodology of Policy

it is usually impossible to observe all of

answer five questions:

What present and past policies have been established to

How valuable are these outcomes in solving the

designed to answer five questions:

What alternatives should be acted on to solve

Five types of policy relevant

standards, rules, and procedures.

The standards and rules govern the selection of

The five procedures common to all human

organizing particular methods and techniques of

The complete framework for problem-centered

policy analysis combines policy-relevant

Five policy-analytic procedures

Problem-centered policy analysis

Phases of the policy-making

Elected and appointed officials place

A state legislator and her cosponsor

Officials formulate alternative policies to

A state court considers the use of

A policy alternative is adopted with the

In Roe v. Wade Supreme Court justices

An adopted policy is carried out by

The city treasurer hires additional staff to

Auditing and accounting units in

The General Accounting Office monitors

Policy analytic procedures and

knowledge that challenges the assumptions

about future states of affairs which are likely to occur

knowledge about the benefits and costs of