Approaches to Public

Policymaking, Policy Analysis

& Evaluation Research
Kathy Luckett
University of Cape Town

 http://www.academia.edu/6487086/POL

ICY_DEVELOPMENT_THEORETICAL_PERSPEC
TIVES_ON_PUBLIC_POLICY

Research Paradigms
Not rigid paradigmatic incommensurability
A map for navigating choppy waters around

policy analysis and evaluation methodological

debates
Post-positivist experimental, pragmatic
Interpretative constructionist, poststructuralist
Critical PAR, empowerment evaluation,CSH
Critical Realism theory-based evaluation

Post-positivist: Quasi-experimental
Popper, Campbell & Stanley (1963, 1966), Lasswell, Rossi,

Lipsey & Freeman

Based on methods of the natural sciences, statistical
measurement techniques
Social science can contribute to improved governance or
management
Establish cause & effect relations bet policy/ programme
objectives, inputs & interventions --------- outputs, outcomes &
impact
Human performance can be objectively measured tv. Efficiency
& effectiveness criteria
Evaluator: objective, neutral, rational
Problem: to secure internal validity of evaluation results
By 19702 disillusion set in, shift to quasi-experimental methods
(pre- & post tests, times series, comparison group designs)

Critique: Quasi-experimental models

Uses a model developed for closed systems for open

social systems
Adopts a flat ontology reality = regularities bet
observable, atomistic objects & events (ignores the
non-observable)
Causality = regularities bet variables within stat. sig.
samples
Claims about causation usually unclear and
unconvincing
Can only provide descriptions (for a few variables on
large populations), seldom explanation

Post-positivist: Pragmatic
(dominant model)
Developed from new public management , the

evaluative state wants practical, workable results,

useful for decision-making
Takes policy/ programme goals as focus of evaluation
Methods: a) open-ended case study (improvement)
e.g. Patton
b) closed-system sets up criteria & performance
indicators to measure performance & accountability
of individuals & institutions e.g. programme
accreditation
Sets up criteria and performance indicators to
measure performance & accountability by institutions
and individuals a closed system

Critique: Pragmatic Models

Assumes stable external environment
Difficult to set measurable objectives, criteria

& indicators for actual performance

Difficult to control variables in open soc
systems - possibility of rival explanations,
difficult to prove cause & effect
Ignores context & stakeholder meanings,
black box evaluation seldom diagnostic
Can be prescriptive, leading to conformity

Interpretive: Constructionist
1970s 80s linguistic turn, 1980s policy sociology:

meaning socially constructed, human action culturally

and discursively mediated rejection of naturalism
Vickers (1995) policymaking as communicative
activity for institutional regulation, a process of normsetting
Neo-institutional theory emphasises cognitive and
normative factors in policy adoption and
implementation
Guba & Lincoln (1989), 4th generation evaluation:
focus on subjective stakeholder meanings, values &
interests, evaluator as facilitator, truth as agreement,
evaluation useful to insiders

Critique: of Constructionist Models

Over-socialised, emphases subjectivity at

expense of structure, truth located in

subjectivities of respondents
Ignores systemic asymmetries of power
Inability to rise above context
Relativist ontology

Interpretive: Post-structuralist
Foucaults geneaology, Ball (1993), Gale (2000)
Discourse is socially constitutive, in dialectical relation to

practice sets up systems of power/ knowledge, norms &

values
Policy as political artefact as text & discourse with unequal
material & discursive effects that should be exposed
Policy has a normalising & regulatory role, sets up subject
positions that constrain ways of speaking & thinking
Technologization of language for institutional ends
How do certain discourses become dominant?
What discourses are at work when those who govern, govern?
How do they become institutionalised & supported legislatively,
professionally & financially?

Critique: Post-structuralist Models

Weak on method, selectivity of data, dominance of

researcher as interpreter, tendency to jump from data

to (preconceived) narrative
Quest to successfully link the micro and macro levels
of analysis difficult to achieve
All of social life gets reduced to discourse,(materiality
of the social world gets lost)
Knowledge reduced to conditions of its production
and interests of its producers (epistemological
relativity)

Critical: Emancipatory
Neo-Marxist insights, Frankfurt School (Habermas

empancipatory interest)
Critical policy analysis (the argumentative turn)
policy discourses construct social problems & policy
solutions, policymaking a form of argument to
persuade & manufacture consent
Challenge: how do discourses become
institutionalised & reflected in institutional practices?
Ulrich (1994) Critical systems heuristics: policy to be
normatively acceptable to those affected by it, value
clarification diff groups of stakeholders

Critical: Emancipatory
Developmental evaluation (Patton)
PAR
Empowerment evaluation (Fetterman 1996)
Transformative evaluation (Mertens 2005)
Development of evaluees, giving voice to the

silenced, inclusion of marginalised groups

affected by the results

Critique: Emancipatory Models

Utopian: the better argument is produced through

power not rational dialogue - all communication

already penetrated by power
Why should the involved (the powerful) bother to take
into account the views and concerns of the affected
(the powerless)?
Cannot work under conditions of coercion requires a
fully functioning public sphere
Needs to hold material conditions and structures as
contexts for vlaues & interests
Post-structuralists: consensus is neither possible nor
desirable

Critical Realist: Theory-based

Bhaskar (1978, 1998), Sayer (1992, 2000)
Reality is stratified empirical (experiences), actual

(events) & real (non-observable structures & causal

powers)
Holds tog ontological realism + epistemological
relativism
Both agency & structure have causal powers attend
to both (analytically separate)
Openess of the social world, plurality and
contingency of causality
Key to successful intervention = change of social
practice

Critical Realist: Theory-based

Pawson & Tilley (1997) Realist evaluation: what

works, how, for whom and under what conditions?

(builds in context & subjectivity)
Evaluator to make programme theory explicit & to
check it out with stakeholders: C + M = O
tests assumptions about causal relations & change
Tests goal realisation, but places in context of wider
social explanation
Evaluation can be cumulative middle range theories
Critique: Demanding to operationalise, timeconsuming

Conclusion
Be aware of tradition & model youre working

in - & of other possibilities

Complex nature of policy analysis &
evaluation justifies methodological pluralism
But dont use methods opportunistically,
select according to values, purpose of
evaluation, stage of the policy/programme
cycle & practical constraints
Think purpose (teleology), ontology,
epistemology 1st then methodology!