Nillei_Paisey

1
Bennett, Tievoi. (199S). What's new in evaluation ieseaich. A note on the Pawson
anu Tilley Aiticle. !"#$#%& ()*"+,- ). /"#0#+)-)12, S6, S67-S7S.

I. Bennett Responus to the Pawson anu Tilley Ciitical Review
A. Bennett cites the Pawson anu Tilley (1994) papei.
1. Pawson anu Tilley ciiticize quasi-expeiimental uesign in geneial.
2. Pawson anu Tilley iuentify one of Bennett's evaluations of a community-
policing piogiamme - Bennett 1989a, 1991.
B. Bennett ueciues to iesponu to Pawson anu Tilley foi seveial ieasons.
1. Colleagues fiom both the quantitative anu qualitative camps have askeu
his opinion.
2. Iueas fiom the Pawson anu Tilley papei aie seeping into the liteiatuie.
S. Colleagues encouiage Bennett to wiite anu extenu the uebate on the
meiits anuoi uemeiits of quasi-expeiimental uesign in evaluating
piogiams.
C. Bennett views the Pawson anu Tilley papei (1994) on two levels.
1. Quasi-expeiimental uesign as it is geneially iegaiueu.
2. In paiticulai, Bennett's stuuy of contact patiols in Lonuon anu
Biimingham.
II. Quasi-expeiimental uesign as it is geneially iegaiueu.
A. Evaluative ieseaich in ciiminal justice lacks positive finuings uue to
methouological failuie.
1. What aiguments leau to this conclusion.
B. Fiist Pawson anu Tilley point emphasizes the stiengths of scientific iealism,
iathei than the weaknesses of quasi-expeiimentation.
1. Scientific iealism iequiies that iesults of empiiical stuuies must iuentify
unueilying causal "mechanisms" anu the contexts that ieinfoice them.
2. Expeiimental stuuies also fall shoit uue to theii uisiegaiu foi mechanisms
anu contexts that biing about anu maintain piogiammes.
C. Seconu point extenus the fiist point.

Nillei_Paisey
2
1. Reseaicheis neeu to look at mechanisms, but within the conveisation
about causality.
2. Pawson anu Tilley ielate two types of causation:
a. Successionist
1. Belief that one cannot obseive causality, but can only make
suppositions about it.
b. ueneiativists
1. Belief that causality involves the mechanisms linking cause
anu effect.
S. The uistinction between successionist views anu geneiativist views ielates
to causation in quasi-expeiimental ieseaich.
a. Accoiuing to Pawson anu Tilley, quasi-expeiimentation is
uncompiomisingly successionist - it uoesn't look fuithei than
associating piogiammes anu outcomes.
1. Inteiesting example of the billiaiu ball in teims of
association anu causation - actually scientists
woulu look at Newton's Laws of Notion.oi
Newton woulu obseive then look foi ieasons
that balls move with uiffeiing velocities anu
uiiectionality.
4. Bennett objects to Pawson anu Tilley's points on giounus that the
aiguments aie pooily uevelopeu.
a. Aigument is baseu on compaiing a philosophy of science with
methouology employeu in science.
1. So this is basically saying that Pawson anu Tilley aie
compaiing apples anu oianges.
b. Boes quasi-expeiimental methouology uisiegaiu "mechanisms anu
contexts." No.
1. Bennett cites Cook anu Campbell (1979) as well as Campbell
anu Stanley (196S), expeits in quasi-expeiimental
methouology.

Nillei_Paisey
S
2. Cook anu Campbell (1979) encouiage qualitative ieseaich
to fuinish uetails anu contextual infoimation about a
stuuy. Qualitative stuuies woulu "illuminate the context
anu conuitions unuei which ieseaich is conuucteu"
(S68).
S. Cook anu Campbell (1979) also view unueistanuing the
mechanisms that biing about outcomes as an essential
pait of appiaising inteinal valiuity - aie the outcomes
the iesult of the inuepenuent vaiiable oi uue to anothei
luiking factoi.
a. Covaiiance is insufficient eviuence - one neeus to
"consiuei all of the majoi thieats to inteinal valiuity"
(Cook anu Campbell, 1979, Su).
4. Pawson anu Tilley aigue that quasi-expeiiments look at
"ueteiministic" causality anu "tiivialize the impoitance
of causal mechanisms"
S. Pawson anu Tilley say that quasi-expeiiments uo not
emphasize iationally anu explicitly unueistanuing how
tieatments anu outcomes aie linkeu.
B. Bennett claims that Pawson anu Tilley have not been specific enough in theii
ciiticisms.
1. They incluue both expeiimental uesign anu quasi-expeiimental uesign.
2. In quasi-expeiimental uesign, the ieseaich uoes not ianuomly assign the
inuepenuent vaiiable - theie is no contiol ovei it.
a. Pie-testposttest uesigns can be both expeiimental anu quasi-
expeiimental.
E. Bennett views Pawson anu Tilley's iejection of quasi-expeiimentation in
evaluation stuuies as pointlessly negative, iesulting in uisiegaiu foi a laige
bouy of liteiatuie in ciiminological knowleuge.
1. Bennett cites seveial stuuies on ueteiience, situational ciime pievention,
anu effectiveness of policing, among otheis.

Nillei_Paisey
4
III. Bennett's Apologia: The Contact Patiols Evaluation
A. The police contact piogiamme was explaineu in uetail.
B. Contacts between the police anu the public weie uocumenteu anu this incluueu
a uesciiption of the natuie of the encountei anu attituue of the iesponuent.
C. Bennett agiees that the iueal stuuy woulu have also involveu a qualitative
component - "an ethnogiaphic oi obseivational stuuy" (S7u).
1. A qualitative stuuy was not conuucteu uue to financial constiaints, not
because of uisinteiest.
B. Pawson anu Tilley chaige that Bennett uiu not empiiically auuiess "causal
mechanisms" ielating to causes anu feai of ciime.
1. Bennett uisagiees.
a. vital pait of the stuuy hau to uo with causality.
1. Suivey anu questionnaiie incluueu questions ielating to
causal mechanisms anu these mechanisms weie baseu on
cuiient theoietical knowleuge of the causes anu feai of ciime.
b. Bennett has wiitten papeis on theoiies of feai of ciime anu one aim
of the stuuy was to test some of these theoiies anu stateu causal
mechanisms of feai of ciime.
c. Bennett quotes fiom his papei that in oiuei to ieuuce feai, one
neeus to unueistanu its ioots oi causes.
u. Police neeu to unueistanu the causes anu feai of ciime in oiuei to
inteivene effectively.
Iv. Conclusion
A. Bennett calls foi a "iobust" assessment of quasi-expeiimental uesign.
1. The puipose woulu be to stiengthen methous as evaluation piogiammes
inciease.
2. Bennett believes that quasi-expeiiments suffei fiom an acute pioblem of
nonequivalence between expeiimental anu contiol gioups. Anu this
pioblem cannot be iectifieu thiough statistical analyses.

Nillei_Paisey
S
B. Scientific iealism can contiibute to the conveisation on stiengthening
methouological issues, but the uetails neeu to be uesciibeu specifically
(empiiicallyopeiationally.).
C. Expeiimental anu quasi-expeiimental uesigns aie useful foi evaluatois in
ciiminal justice anu ciiminology.
B. Nethous wais contiibute nothing; it is impoitant to iefine methous geneially,
iathei than attempt to expel a methou, simply to ieplace it with anothei.