Empirical Methodology and Legal Scholarship

Author(s): Jack Goldsmith and Adrian Vermeule

Source: The University of Chicago Law Review, Vol. 69, No. 1 (Winter, 2002), pp. 153-167
Published by: The University of Chicago Law Review
Stable URL: http://www.jstor.org/stable/1600351 .
Accessed: 20/08/2014 14:15

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

The University of Chicago Law Review is collaborating with JSTOR to digitize, preserve and extend access to
The University of Chicago Law Review.

http://www.jstor.org

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
 and Legal Scholarship
EmpiricalMethodology
JackGoldsmitht
AdrianVermeulett

In TheRulesofInference' ("Rules"),Lee Epsteinand GaryKing

arguethatall empirical and causalassertions in all legalscholarship-
includingdoctrinal, normative, and interpretive scholarship thatas-
sertssubsidiary empiricaland causalclaims-shouldfollowa unitary
approachto inference, one thatis groundedin thelogicof statistics.
Fromthatpremise, theyadvancea sweepingindictment oflegalschol-
arshipand its supporting claimingthateveryarticlein
institutions,
theirlargedatapool violatesat leastone ruleofinference2 andurging
a radicalreorientationoflegaleducation.3
At somelevelofgenerality, itis hardto disagreewiththespiritof
Epsteinand King'scomplaints. Scholarship generally, notonlyin law
or politicalscience,shouldgroundits empiricalassertionsin war-
rantedinferences fromsoundevidence,shouldadmitto causal and
empiricaluncertainty whereit exists,shouldavoid tendentiousness
andselectionbias,andshouldfollowthebeststatistical when
practices
makingstatistical claims.Lawyersand judges,and indeededucated
people generally,wouldbenefitfroma basicknowledgeof statistical
methods. Putting asidetheopportunity costsofstatistical training
(an
issueto whichwe returnbelow),greaterattention to therulesof in-
ference would,whenappropriate, improve legalscholarship.
We nonetheless rejectmuchofEpsteinand King'sindictment of
legalscholarship.We beginin PartI byidentifying a broaddomainof
legal scholarshipforwhichthe rulesof inference lack obviousrele-
vance.As is unsurprisingforoutsiders,EpsteinandKingoverlookthat
legal scholarshipfrequently pursuesdoctrinal, interpretive,and nor-
mativepurposesratherthanempiricalones.Legal scholarsoftenare
just playinga differentgamethanthe empiricists play,whichmeans
thatno amountofinsistence on theempiricists' rulescan indictlegal
scholarship- anymorethanstrict adherenceto therulesof baseball

t Professorof Law,The Universityof Chicago.

tt Professorof Law,The Universityof Chicago.
We thankDouglas Baird,RichardEpstein,JakeGersen,Kate Kraus,Daryl Levinson,Doug
Lichtman,Martha Nussbaum,Eric Posner,RichardPosner,Duncan Snidal,Cass Sunstein,Steve
Walt,and David Weisbachforcommentsand conversation.
1 Lee Epstein and GaryKing,The Rules of Inference,
69 U Chi L Rev 1 (2002).
2 Id at 11.
3 Id at 11433.
153

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
154 The University
of ChicagoLaw Review [69:153

supportsan indictment of cricket.Epsteinand Kingmissthispoint

becausetheirempiricalmethodology blindsthemto legal scholar-
ship'sinternalperspective. EpsteinandKing'sexternal perspective-if
valid-mightcause legalscholarsto see theirpractices in a newand
interestinglight.Butinmanydomainsoflegalscholarship, itprovides
no basisforpersuading insidersto acceptEpsteinand King'smeth-
odologicalcounsels.
Havingnarrowed thefieldofpotential relevance, we nextques-
tiontheprivileged statusofEpsteinandKing'srulesofinference even
within politicalscience,muchlessin otherdomains. and
Epstein King
givetheimpression thattheirarticleextendsuncontroversial methods
frompoliticalsciencetolaw.In fact,as we showinPartII, Epsteinand
King'sprescriptions are contested evenin theirowndiscipline. There
simply arenot"RulesofInference" inthesenseofuniversally agreed-
uponmethods ofempirical analysis.
Evengranting therelevanceand legitimacy oftherulesofinfer-
ence,PartIII contains a pragmatic critiqueofEpsteinandKing'spre-
If empirical
scriptions. research werecostless,EpsteinandKing'spre-
scriptionswouldbe sensible.But givenconstraints on time,informa-
and
tion,expertise, research funds,academics faceinevitable tradeoffs
betweenrigorand accuracy, on the one hand,and timeliness, rele-
vance,and utility, on theother.Thistradeoff is especiallypointedfor
disciplinesthat,likelaw,areprofessionally andpractically involvedin
thebusinessof courtsand othergovernmental institutionsthatmust
constantly reachdecisionsdespiteprofound empirical A
uncertainty.
warning signforEpsteinandKingis thattheirownarticlecontainsa
hostof embeddedempiricaland causal assertions thatviolatethe
rulesof inference. Thisfactis significant notbecauseit exposesEp-
steinand Kingas inconsistent or hypocritical,
butbecauseit under-
scoresthatscholarship mustnecessarily proceedon thebasisof un-
provenempirical assumptions ifitis toproceedatall.

I. RELEVANCE
Epsteinand Kinglacka normative accountoftheaimsof legal
scholarship,and the lack of suchan accountdeprivesthemof the
normative baselinetheyneedto groundtheircriticisms.Thereaderof
Epsteinand King's 133-pagearticlewill findalmostnothingthat
speaksto thesimplequestion, "Whatis legalscholarshipfor?"Much
ofthetimeEpsteinand Kingimplicitly assumethatlegalscholarship
oughtto relateto thelegalsystem in thesamewaythatpoliticalsci-
enceor government departments relateto thepoliticalsystem gener-
ally.On thisview,legalscholarsare,at present,
justpoliticalscientists
who (1) studya limitedsubsetof politicalinstitutions,
namelythose
thatproduceand enforcelegalrules,and (2) lacktheempirical and

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
2002] andLegalScholarship
Methodology
Empirical 155

methodological toolsthatreal politicalscientists possess.The needed

reforms willimprovethelegalacademy'smethodological skills,while
leavingthelegalacademyconfined to itspresentsubstantive domain.
Legal scholarsshouldbe junior-varsity politicalscientists -and, as we
discussin the nextPart,politicalscientists of a particularly narrow
stripe.
But thereis an entirelydifferent pictureof the aims of legal
scholarship. In a largedomain,it is nottruethatlegalscholarstryto
do justwhatpoliticalscientists do,onlyless successfully. Often,legal
scholarstryto do otherthings entirely,andat leastsometimes do them
ratherwell.Thoseotherthingsincludemakingdoctrinal, interpretive,
and normativearguments.4 The legal academysuppliesvocational
ratherthanscientific lawschoolsusuallyproducelawyers,
training; not
graduatestudents; and legalscholarsoftenwritein thelawyer'sstyle
ratherthanintheempiricist's becausetheyareparticipants in,notjust
students of,thelegalsystem's practices.
The substanceofmuchlegalscholarship is doctrinal,interpretive,
and normative. Subgenresin thiscategory includearticlesand books
thatattemptto reconcileor distinguish linesof precedentdisplaying
internal tensions; thatprovideconceptualanalysisoftheinternal logic
ofstatutes, cases,and othermaterials;and that provide novel readings
of canonicallegal sources.The bestlegal scholarship combinesthese
features, fittingconfusedcanonicalmaterialstogetherin a coherent
way and presenting the materialsin a normatively attractive light.
Workin thisveincontainsno empiricalclaimsin anyimportant or
contestable sense-at leastnotif"contestable" is definedbyreference
to theinternal consensusoflegalacademics. As we discussbelow,any
scholarlycommunity presupposessome degree of consensuson
sharedpremises, empiricalandotherwise.
Doctrinal,interpretive,and normative legal scholarship seeksto
persuade,whichmeansthatthelawyer'sstyleis oftenlargelyrhetori-
cal. Epsteinand Kingdescribelawyerly rhetoric as "politicalpersua-
in a tonesuggesting
sion,",5 thatEpsteinand Kingequate persuasion
withsophistry. ButEpsteinandKingoverlookthattherhetorical style
is a mixedbag ofviceand virtue.6 The viceis thatmuchlegalscholar-
shipis advocacyscholarship, and therefore rhetorical in thecondem-
natorysense:it is tendentious, sloppilyor evendeceptively reasoned,
and restsuponunsubstantiated factualclaimsor thesortofempirical
shibboleths thatcirculatein law schools(forexample,thatdisagree-

4 Thereare,of course,manyotheraspectsto legal scholarship,manyof whichare not

In theseshortcomments
touchedbyEpsteinandKing'scritique. we focusouranalysison these
threeaims.
5 EpsteinandKing,69 U ChiL Rev at9-10n 23 (citedinnote1).
6 See Richard
A. Posner,
OvercomingLaw 73-74,517-30 (Harvard1995).

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
156 The University
of ChicagoLaw Review [69:153

mentamongthejusticesharmstheSupremeCourt'spublicstanding).
The lastpointis,as we havesaid,thekerneloftruthin Epsteinand
King'sposition.
But thereis a sensein which"rhetorical" neednotbe a termof
condemnation. Rhetoricis oftena moreeffective tool ofpersuasion
thanis,forexample, regressionanalysis, becausesuccessful rhetoric is
accessible,vivid,andmixespositive, interpretive,andnormative claims
in richlytextured ways.Good legalscholarship is effectiveprecisely
becauseit displaystheserhetorical traits.At itsbest,persuasive legal
scholarship displaysa powerto elucidatethelogicof doctrinal prac-
ticesin enlightening ways,to engageaudiences, andto motivate legal
andsocialchange-a powerusuallylackinginthepallidoutputofsta-
tistical
empiricism.
EpsteinandKingbriefly nodto someofthesepointsin a buried
footnote,7buttherestoftheirarticleshowsthattheyhavefailedtoin-
ternalizetheirimplications. Theysaythatpersuasion andadvocacyare
appropriate to the courtroom, but that "the facultycommons"is a
placein which"thetruth, and notjusta particular versionofit,mat-
In PartII we questiontheirequationofvalidinference
ters."8 withsta-
inference.
tistical Butevenon itsownterms, EpsteinandKing'scom-
plaintelidesa critical thecontest
possibility: of"particularversions" of
truthventilatedby legal articlesthatare tendentious whentaken
separately may,at the systemic level,produceincreasingly accurate
approximations oftruth, as scholar-advocates theworkofop-
criticize
posingcamps.Thatpossibility transposes thestandard justificationfor
theadversary system fromthecourtroom to thefaculty commons, col-
lapsingoneoftheloadbearing wallsinEpsteinandKing'sconstruct.
The pointhereis not to defendan adversarial pictureof legal
scholarship.The pointis thatEpsteinand King'sdistinction restson
thesamefallacy as thesimplistic complaint thattheadversarial system
of adjudication, as opposedto theinquisitorial system, distorts facts.
True,lawyersin the courtroom shade factsand law to accordwith
theirdesiredoutcomes. But it is possible,and indeed,forbetteror
of
worse,itis thepremise ourlitigation system, thattheaggregate ef-
fectof individualtendentiousness is a fullyroundedpictureof the
truth.Thesameaggregate effect mayholdfortendentious legalschol-
arship.In boththeacademicandcourtroom settings,thereis a system-
leveljustificationforthecompetitive production ofevidencethatEp-
steinandKing'ssimplecritique assumesaway.
The consequenceofEpsteinand King'sfailureto appreciate the
doctrinal,interpretive, andnormative castofmuchlegalscholarship is

7 Epsteinand King,69 U Chi L Rev at 9-10 n 23 (citedin note 1).

8 Id at 10.

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
2002] Empirical andLegalScholarship
Methodology 157

thatan airofunreality,
ofMartiananthropology,
hoversoverthearti-
cle. Amar and Widawskyfamouslyargued that the Thirteenth
Amendmentprohibitsprivatelyinitiatedchildabuse as a formof
servitude."9
"slavery[or] involuntary Epsteinand King say thatthe
theory"surely... lends itselfto severaltestablepre-
Amar/Widawsky
dictions"and "observableimplications";'0 forexample,theyclaimthat
"iftheterm'slavery'is broadenoughto coverchildabuse,thenwe
mightexpectto findtracesofthatsentiment in thehistorical,
legisla-
tive,and judicialrecords.""Thereis a misstephere,akinto thesmall
butjarringmistakesthatspeakersof a secondlanguageconstantly
commit:Amar is a textualist, not an originalist per se, and would
probablythinksuchevidenceaboutthespecificintentof contempo-
raneousactorsmarginally relevantto thetaskofdeveloping a coher-
entaccountofthetext'simplicit structureanddeepcommitments.
But the largergaffeis to thinkthatworkslike Amarand Wi-
dawsky'sare making"empirical" claimsin anysenserecognizable to
politicalscientists.The claimthat"childabuse is slavery"is not a
statement offactlike"HydeParkis in Chicago."It is an interpretive
and normativeprogram-legalactorsshouldcountchildabuse as
slaveryfor purposesof enforcingthe Thirteenth Amendment
presentedin the formof an arresting metaphor.Legal scholarship
sometimes deployssuchmetaphors to disrupttheaudience'sassump-
tions,in the hopes of presenting old practicesin a new lightand
thereby stirringreform.Thisis an exampleof thesociallyproductive
powerof rhetoric to reorientperceptions.AftertheAmar/Widawsky
we notice,evenifwe chooseto reject,uncomfortable
article, analogies
betweenpractices thatpreviously seemedunconnected.
In a similarvein,Epsteinand Kingseemingly assumethroughout
thatstatements suchas "theFramersthought X" are testableclaimsof
factthatmightat leastin principle be quantified by counting quota-
tionsin Farrandor in theratification debates.More oftenin legallit-
erature, though,suchclaimsshouldbe takenas shorthand forcomplex
interpretive accountsof sourcesthathave attainedcanonicalstatus
withinthelegal culture.Epsteinand King'sfailureto recognizethis
pointcommits them,no doubtunwittingly, to a narrowandcontrover-
sial versionof specific-intention originalism, one now widelydis-
avowedbymostoriginalist academics,'2andcausesthemto equatena-
ivelycanonicaland noncanonicalsources.On Epstein and King's

9 Akhil Reed Amar and Daniel Widawsky,ChildAbuse as Slavery:A Thirteenth Amend-

mentResponseto DeShaney,105 Harv L Rev 1359 (1992).
10 Epstein and King,69 U Chi L Rev at 73 (cited in note 1).
11 Id.
12 See, forexample,Steven G. Calabresi and SaikrishnaB. Prakash,The President'sPower
to ExecutetheLaws, 104 Yale L J541,552 & n 35 (1994).

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
158 The University
of ChicagoLaw Review [69:153

methods, a statement byJamesMadisoncountsforno morethana

statement by othersignersof the Constitution, including suchun-
knownsas GunningBedfordof Delaware,RichardDobbs Spaight
fromNorthCarolina,orWilliamFew ofGeorgia."3 The difference be-
tweenMadisonand theunknowns is thesortof softcultural variable
thattheexternal perspective findsitespecially toappreciate.
difficult
For a finaland somewhatmorecomplicated example,consider
theempirical controversy overwhether thedeathpenalty hasa deter-
renteffect. Legal scholarscontinue to citeIsaac Ehrlich'sfamousbut
methodologically suspectstudy;Epsteinand Kinguse thisas an ex-
ampleof"thestaying powerofflawedanddiscredited legalstudies."
14
As good empiricists, however, Epsteinand King shouldask them-
selveswhether thereis an explanation (otherthantendentiousness or
stupidity) forlegal scholars'persistence in invoking Ehrlich'swork.
Dan Kahan,forexample,arguesthatlegalscholarsuse thelanguage
ofdeterrence inthehopethatitwillpresent an appearanceofneutral-
ityand dampenintractable conflict overthe philosophy of punish-
ment."5On thisview, noit is more important thatthe relevant studies
are inaccurate or outdatedthanit is important to a lawyerthatthe
phrase"hungjury"is an archaicandobscuremetaphor.
Butitmaybe askingtoomuchofexternal observers,notsituated
withinand saturatedby the debatesinternalto the legal discourse
aboutcapitalpunishment, to imagineevensuchan explanation. Our
point here is not to endorse tendentious distortionsof fact.
Our point
is simplyto callintoquestionEpsteinandKing'sresolutely externalist
approachto legalscholarship; theMartiananthropologist mayendup
beinga worse,ratherthanbetter, empiricist, becausetheMartianan-
thropologist's inability to participate in and identify empathetically
withthepractice oflegaldiscourse mayradically contracttherangeof
hypotheses thatseemplausiblecandidates fortesting.Thatpossibility
is centralto a long-running methodological debatein anthropology,
politicalscience,and manyotherdisciplines, betweenscholarsof Ep-
steinand King'sexternalist stripe, on theone hand,and hermeneutic
orinterpretive socialtheorists, on theother. Theinterpretive campar-
guesthatempiricists mustbecome,to theextentpossible, participants
whointerpret culturalpractices fromtheinside.'6By assuming away
theinterpretivists' viewat theoutset, EpsteinandKingcommit them-

13 See US Constsignatories.
14 Epsteinand King,69 U Chi L Rev at 17 n 42 (citedin note 1).
15 See Dan M. Kahan, TheSecretAmbition ofDeterrence, 113 Harv L Rev 413 (1999).
16 See, forexample,Daniel Little,Varieties of Social Explanation:An Introduction to the
Philosophyof Social Science (Westview1991); CliffordGeertz, The Interpretation of Cultures
(Basic 1973).

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
2002] Empirical
Methodology
andLegalScholarship 159

thatnarrowtheaudiencefor,and
assumptions
selvesto controversial
critique.
technical
therelevanceof,theirsupposedly

II. THE RULES OF INFERENCE?

This pointsto a different critiqueof Rules.Epsteinand King
complainthatlegalscholarship displays"littleawarenessof,muchless
compliancewith,therulesof inference thatguideempiricalresearch
in thesocialand naturalsciences."'7 A readerofRulesuninformed of
longstanding methodological debatesin socialsciencesmightbe for-
givenforthinking thatthereis a single,uncontroversial approachto
inference acrossdisciplinesthatis essentially Rulesnotonly
statistical.
assumesawaythe possibility of normative and interpretive political
and socialtheory, butalso,evenwithin non-interpretive as-
the camp,
sumesthehotlycontestedproposition thatstatistical methodsof in-
ferenceapplyequallyto large-number empiricalclaims,on the one
hand,andqualitative case-studywork,on theother.
Thisassumption requiresan extendeddefensenotprovidedhere
by Epsteinand King,forthereis a vigorouscontemporary debate
aboutthevalueofstatistical approachesto inference in small-number
case studies.We cannotdo betterhere thanto pointthe readerto
criticismsofthestatisticalapproach,advancedin thehighly respected
peer-reviewed journalsthatEpsteinand Kingpraise,by politicalsci-
entistswithimpressivemethodological credentials."'The essential
pointofthiscritiqueis thatthestatistical worldview does nottranslate
cleanlyto the single(or small-number) case study, wherea detailed
contextualanalysiscan often uncover causal and other explanatory
mechanisms correlation
thatstatistical cannotcapture.
For similarreasons,proponents ofthestatistical approachfindit
puzzling that certaincases receive disproportionate attention,bothin
politicalscienceand in law,becausethosecaseshaveattainedcanoni-
cal statusin therelevantacademiccommunity, or havebecomefocal
pointsaroundwhichacademiccommunities organizetheirdebates.
EpsteinandKinghavea tinear forthisphenomenon, as theircritique
of Sklansky'swork on the FourthAmendmentshows.19 Sklansky
claimsthattheSupremeCourt'ssearch-and-seizure jurisprudence has
recentlytakenan originalist turnand supports his claim by pointingto

17 Epstein and King,69 U Chi L Rev at 6 (cited in note 1) (emphasisadded).

18 See, forexample,TimothyJ.McKeown, Case Studiesand the StatisticalWorldview:Re-
viewof King,Keohane,and Verba'sDesigningSocial Inquiry:ScientificInferencein Qualitative
Research, 53 Intl Org 161 (1999); Mark I. Lichbach, ed, Review Symposium:The Qualitative-
QuantitativeDisputation:Gary King, Robert 0. Keohane,and Sidney Verba'sDesigning Social
Inquiry:ScientificInferencein QualitativeResearch,89 Am PolitSci Rev 454 (1995).
19 Epstein and King,69 U Chi L Rev at 32-34 (citedin note 1).

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
160 of ChicagoLaw Review
The University [69:153

a set of prominent recentcases.'oEpsteinand King complainthat

Sklansky examinesonlyelevenof thefifty-four cases theCourtde-
cidedintherelevant period,anddemandthathe demonstrate thathis
samplefairly represents thewholeset."Butthatcomplaint is aboutas
impressive as questioningtheempirical importance ofsubstantive due
processon thegroundthattheCourthassustained suchclaimsinonly
a fewareas-abortion, contraception, and similarcontexts-while re-
jectingit in many others. The legal and culturalsalienceof Roe v
Wade"faroutruns itsstatistical
significance.
A similarshortcoming ofEpsteinandKing'sworldview is appar-
entin theirinsistence on a statistical to
approach history. There are
manyexamplesofthisinRules.Considertheircriticism ofFisher'sar-
ticleon theriseof plea bargaining.3 Epsteinand KingchargeFisher
withfailingto demonstrate whathisdetailedcase studyofplea bar-
gaining in one tierofa Massachusetts county'sjudicialsystem reveals
abouttheriseofplea bargaining inAmerica.Justas in politicalsci-
ence,wherecase studiescan be valuablebecausetheyuncoverprevi-
ouslyunknown chainsofcausation,25 so tootheweavingofan interest-
ingand coherentnarrative in one jurisdiction can revealpreviously
unknownpatterns or explanations in broadercontexts. Presumably,
Fishercouldhave made thinobservations aboutmanyjurisdictions,
butinsteadhe choseto makethickobservations aboutone.The only
reasonforrejecting thelatterstrategy outofhandis thatEpsteinand
King wantto remakeall otherdisciplines in the imageof large-
numberstatistical empiricism. EpsteinandKing'sstatistical approach
to historywouldstartlehistorians trainedin theirdiscipline'squalita-
tiveandnarrative approachtoinference.

III. TRADEOFFS
We turn,finally,to a pragmatic critiqueof Epsteinand King's
project.Epstein and King's indictment of legal scholarshipis
comprehensive and severe.Theyclaimthat"everysingle[lawreview
articletheyread]violatesat leastone oftherules"ofinference.26
They
appearto concludethatall suchlegalscholarship is illegitimate
and-
becauseit mayaffect publicpolicy-pernicious.27The problemis that

20 David A. Sklansky, The FourthAmendment and CommonLaw, 100 Colum L Rev 1739,
1745-74(2000).
21 See Epsteinand King,69 U Chi L Rev at 3,32 (citedin note 1).
22 410 US 113 (1973).
23 GeorgeFisher,Plea Bargaining'sTriumph, 109Yale L J857 (2000).
24 Epsteinand King,69 U Chi L Rev at 101-02 (citedin note 1).
2s See McKeown,53 IntlOrg at 174 (citedin note 18).
26 Epsteinand King,69 U Chi L Rev at 17 (citedin note 1).
27 See, forexample,id ("Since [ ] everysingle[articleanalyzedhas] the potentialto find
[its] way into a courtcase, an administrative
proceeding,or a legislativehearing,we can only

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
2002] Empirical
Methodology
andLegalScholarship 161

the same indictment can be made of Rulesitself. Throughout Rules,

Epsteinand Kingfrequently makeor assumea varietyof causal or
descriptive empiricalclaimsthatviolatethe veryrulesof inference
thattheyurgethelegalcommunity to embrace.Thisis important not
becauseitevincesinconsistency or hypocrisy on theirpart.Rather,itis
important becauseit demonstrates a fundamental constraintofschol-
arshipthatEpsteinand Kingdiscount-namely, thetradeoffbetween
(certainconceptionsof) accuracyand rigor,on the one hand,and
timeliness andrelevance, on theother.
ConsiderEpsteinand King'sastounding claimthatlegalscholar-
shipis nota socialenterprise, and thatlegalscholarsdo notviewitas
a socialenterprise.> Epsteinand Kingmentionthree"telltalesignsof
[legal scholars']disinterest" in the social aspectsof scholarship-
nonadherence to Epsteinand King'sstrictcriteriaforreplication of
research, therelativelack of coauthoredscholarship, and therelative
absenceof peer-reviewed law journals.29How does one assess these
claims?Epsteinand Kingmightsimplybelievethatscholarship as a
socialenterprise is scholarshipcharacterized bythesethreecharacter-
istics.But thiswouldmake theirclaiman uninteresting definitional
stop,leavingthemopen to therejoinder thatlegalscholarsdefineso-
ciabilitydifferently.
The morecharitableinterpretation is thatEpsteinand Kingare
makinga causal empiricalclaim.Theyappear to believethatlegal
scholarsdo not take seriouslythe social aspectsof scholarship-
engaging theworkofotherscholars, buildingon it,criticizing
it,work-
ingthrough problems together -because theydo notattendto replica-
tionstandards, do not coauthor,and do not publishin peer-edited
journals.Or perhapstheyare makingtheoppositecausalclaim:legal
scholarsdo notengagein thesethreepracticesbecausetheybelieve
scholarship shouldproceedin isolation.It does notmatterwhichway
thepurported causalityruns,forEpsteinand Kingdo notprovidea
shredof empiricalevidenceforeithercausalclaim.Moreover,in the
courseof makingtheclaims,Epsteinand Kingcommita hostof sins
theydecry. Theycertainly havenotdonewhattheyinsistlegalscholars
do-namely,"maximiz[ing] theirvulnerability" by considering "rival
hypotheses" forthe practicestheybemoan.>They fail to consider
otherindiciaof scholarly sociability-workshops, co-teaching,draft-
swapping, written and oral comments, lunchconversations, symposia,
etc.,all of whichare in our experiencemorepresentin law schools

imaginethe seriousconsequencesforpublicpolicy(not to mentionforthe development

of
knowledge)thatmayhavealreadyresulted-orstillmayresult.").
28 Id at45.
29 Id at45-49.
30 Id at76.

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
162 The University
of ChicagoLaw Review [69:153

thanin politicalsciencedepartments.31 And theyreadevidenceofso-

ciabilityin legal scholarship uncharitably whentheyassert-again,
without theattention to therulesof inference thattheyclaimis so
crucial-thatextravagant footnoting and lengthy "thankyou" foot-
notesareevidenceoftheflabbiness, ratherthanthesociability, oflegal
scholarship.32Whyis coauthoring evidenceofsociability, whilethank-
ingcolleaguesforcomments is not?
Moreover, EpsteinandKingneverconsider whylawyers commit-
tedto legalscholarship as a socialenterprise might withgoodreason
avoidtheinstitutional practices thatEpsteinand Kingpraise.Forex-
ample,whilethereis muchto be saidforpeer-reviewed journals, there
arereasonsnotto favorthemas well.One has to do withthelengthy
delay-sometimesyears-betweenwriting and publication thatin-
heresin a peer-review system. Whatever valuethepeer-review system
bringsthusmustbe tradedoffagainstthecostsof untimeliness and
delay.It is justsuchcoststhathaveled physicists-scholars whoun-
doubtedlyshare Epstein and King's commitment to statistical
method-tojettisontheex antepeerreviewsystem forall purposes
buttenurereview, andinsteadto publishtheirworkonlineforimme-
diatecirculation,lettingthemarket sortoutqualityex post.33
Peer reviewsystems also raisetheconcernthatseniorscholars
favorlike-minded scholarship and chokeoffthechannelsofintellec-
tualchangeand development. Indeed,overone hundred politicalsci-
entists,manyofthemleadersintheirsubfields, recently signeda letter
complaining thattheeditorial boardoftheAmerican PoliticalScience
Review(the discipline's"flagship"publication34) is dominatedby
scholars "intensely focusedon technical methods at theexpenseofthe
great,substantive politicalquestionsthatactuallyintrigue" manypo-
Thisletterwenton to statethat"[i]tis veryunfortu-
liticalscientists.35
nate that deeplycommitted politicalscientistsgenuinelybelieve,
whether rightlyor wrongly, thattheycannotcriticize thestatusquo
safelywithout thecloakofanonymity."3 The concerns raisedbythese
politicalscientistshave specialsaliencein a discipline that,likelaw,

31 Even if we are wrongin thisconjecture,Epstein and King should have,by theirown

criteria,consideredand rejectedthese and otherpossible "telltalesigns"of sociabilityin legal
scholarshipbeforemakingsucha pointedclaim.
32 Epsteinand King,69 U Chi L Rev at 45,127 (citedin note 1).
33 See E-PrintArchive,available onlineat <xxx.lanl.gov>(visitedNov 19,2001).We thank
Kate Kraus forbringing thispracticeto our attention.
34 Epsteinand King,69 U Chi L Rev at 45 (citedin note 1).
35 See Rogers M. Smith,et al, An Open Letterto theAPSA Leadershipand Members,33
PS: Polit Sci and Polit 735, 735 (Dec 2000). For anothercomplaintabout thistrendin political
science,see JonathanCohn,IrrationalExuberance:WhenDid PoliticalScienceForgetaboutPoli-
tics?,New Republic25,31 (Oct 25,1999).
36 RogersM. Smith, et al,33 PS: Polt Sci and Politat 736 (citedin note35).

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
2002] andLegalScholarship
Methodology
Empirical 163

lacksa generalconsensusaboutthe meansand ends of scholarship,

and thatis ideologically charged.These concernsare attenuated in a
publication systemthatis,as in law,decentralized and notdominated
byscholarswithparticular commitments andagendas.
Rulesis filledwithmanyotherempiricalwhoppersand inferen-
tialbloopers,including: theflatlyerroneousclaimthatthereis nota
single"law reviewarticle[] devotedto improving, understanding, ex-
or adaptingtherulesofinference";37
plicating, theassertionthatlegal
scholarsdo not citerelevantliteratures (a claimat odds withtheir
complaintaboutlawyers'stringcitations, and,moreimportantly, one
supported, contrary to theirprinciples, onlybya stringcitationand a
fewexamplesthatreekofselectionbias);38 theunsubstantiated,ques-
tion-begging, and much-criticized claimwithinpoliticalsciencethat
"[o]therfieldsseem to benefitfromdevotingsustainedattention to
theirmethodological problems";39 and theamusingbutgroundless as-
sertionthat attentionto the rules of inferencewill increaselegal
scholars'readership.40Therearemanyotherexamples.
Epsteinand Kingmightrespondbyclaiming thattheircriticisms
oflegalscholarship are notmadeat thislevelof empirical detail.We
doubtthisis true,butitis impossible to tellfromRules.The problems
beginwithEpsteinand King's description of their"supplemental"
data setof"empirical articlesfromsixtoplaw reviews."941 We haveno
idea whether "empiricalarticles"includearticleswiththeword"em-
pirical"in the title(the criterion fortheinitialdata set),or articles
thatare"empirical" inthesensethattheyincludeany"evidenceabout
theworldbased on observation or experience."42Settingaside prob-
lemsin the description of the data pool,Epsteinand King"do not
tabulatetheparticular typesofmistakes" theyidentified,do notcode
theirdata (becauseitwastoo hardto do), and"summarize [their]sur-
vey ... without explicit references."43
These are unfortunateomissions
in an articlethatstressesoverand overagainthat"[r]esearch mustbe
replicable."4As EpsteinandKinginsist:"[A]nother researchershould
be able to understand, evaluate,buildon,and reproducetheresearch

37 Epsteinand King,69 U Chi L Rev at 11 (citedin note 1). To take one of many
ofwhichEpsteinandKingappearignorant,
counterexamples see GeorgeL. PriestandBenjamin
Klein, The Selectionof Disputesfor Litigation,13 J Legal Stud 1 (1984), whichargues thatset-
effects
tlement casesan unrepresentative
makethepooloflitigated sampleoflegalissues.
69 U ChiL Revat57 (citedinnote1).
38 See EpsteinandKing,
39 Id at 18.
40 See id at59.
41 Id at 16.
42 Idat2.
43 Id at 15 & n 36.
44 id at 38.
See,forexample,

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
164 The University
of ChicagoLaw Review [69:153

without anyadditionalinformationfromtheauthor."45 And yetEp-

steinandKingprovideno wayforotherscholarsto assesstheirclaims
aboutthesorrymethodological shapeof legal scholarship.Theydo
not,forexample, "explainwhytheymarshalparticular piecesof evi-
denceand neglectothers,"46therebymakingit impossible to "repro-
duce [theirstudy]without to theauthors."47
talking Thisis notto sug-
gestthatwe questiontheirmethodologicalrigor;theyhaveextraordi-
naryreputationsas rigorous
empirical ButEpsteinandKing
scientists.
... is whollyirrelevantto
vehementlywarnthat"appeal-to-authority
validinference.""
Wedo notbelievethatEpsteinandKing'sviolations oftheirown
rulesdemonstrate theinvalidity of theserules.Rather,we pointout
theseviolationsforthreereasons.First, doingso demonstrates thatin
anyworkofscholarship, someempirical claimsmustbe takenas given
so thatotherscan be questioned. In thenormalscienceofday-to-day
scholarlywork,an intellectualcommunity necessarilyindulgesinnu-
merablesharedassumptions-including sharedempiricalassump-
tions-thatdo notrequireempirical demonstration. Of course,schol-
arshipsometimes questionstheseassumptions, and sometimes an ex-
ternalperspectivecan helpshinelighton empirical premises thatare
unwarranted. But the incessantdemandforempiricalconfirmation
revealsincomprehension oftheassumptions demandedbytheinter-
nalperspectiveofanypractice.
Second,Epsteinand King'sviolations oftheirrulesofinference
showthatscholarship canbe valuableevenwhenitcontains problem-
aticempiricalassertions.To repeata pointwe madein theintroduc-
tion:we believethatthereis muchvaluablecriticism oflegalscholar-
shipinRules.Andwe haveno doubtthatthearticlewillhavea bene-
on legalscholarship,
ficialinfluence bothin introducing somelawyers
to therulesofinference, and in sparking a healthy debate.The point
hereis thatRulesis notrendered valuelessbecauseit containsmany
unfounded empiricalclaims.The same,we think, is trueof thelegal
scholarshipthatEpsteinandKingcriticize.49
Third,and mostimportantly, Epsteinand King'sviolationsof
theirownrulesilluminate important practicalconstraints on scholar-
ship thattheyignore.Because of limitedresources (in terms oftime,
researchfunds,information, expertise,etc.),thereis a tradeoff be-

45 Id.
46 Id at 8n21.
47 Id at 39.
48 Id at 34.
49 In thisconnection,it is perhaps worthnotingthat Rules never demonstrates that the
empiricalerrorsdiscoveredin legal articlescompletelydiminishthe value of the articles'argu-
ments.

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
2002] andLegalScholarship
Methodology
Empirical 165

tweenaccuracy, on theone hand,andtimeliness andrelevance, on the

other.Each department in a universityresolvesthesetradeoffs differ-
entlydependingon thenatureof thediscipline, itscustoms, and,im-
portantly,thedemandforthediscipline's scholarly outputfromout-
sideinstitutions.One reason(butnottheonlyone) whylegalscholars
sometimestradeaccuracyforrelevanceand timeliness is theirclose
connections to governmental institutions,
especiallycourts, thathave
to makedecisionsin theshorttermunderconditions ofempirical un-
certainty.
ConsiderEpsteinand King'sexampleofthefallacious jurystud-
ies thattheSupremeCourtcitedin a 1973decisionholdingthatsix-
personjuriessatisfy theSeventhAmendment's guaranteeofjurytrials
in civilcases.? WhatEpsteinand Kingoverlookis thattherewas a
case to be decidedthatturnedon whetherthe outputsof six-and
twelve-person juriesweresimilar. The threestudiestheycriticize were
presumably donein anticipation ofthisdecision.One consequenceof
Epsteinand King'sprescriptions is thatthesestudiesshouldnothave
beenpublished. Thismeansthat,in Epsteinand King'sworld,theSu-
premeCourtwouldhavebeenforcedto decidetheissuewithout any
scholarlydatawhatsoever. Thisis a situation thatgovernmental actors
faceall thetime.Theymustdecidean issuenowintheabsenceofem-
piricalstudiesthatsatisfyEpsteinandKing'srules.
EpsteinandKingseemcommitted to theviewthatsuchdecisions
are bettermadeon thebasisofno empirical studiesthanon thebasis
of sometimesflawedempiricalstudies.But Epsteinand King have
done noneoftheworkthat,on theirownterms, is neededto support
such a drasticpublic-policy prescription. Any such prescription is
questionable on boththecase levelandthesystemic level.On thecase
level,courtsmightpreferto navigatewithan inaccuratemap than
withno map at all. On thesystemic level,legalscholarship produces
timelybutsometimes methodologically unsophisticated studies, some
of whichare accurateand some of whichare not.A universalinsis-
tenceon EpsteinandKing'sversionofmethodological rigormightre-
quiremakingall studiesless timely, therebyeliminating studiesthat
are bothtimelyand accurate.Epsteinand Kinghave notshownthat
thecostsof methodological perfectionism are worthincurring in the
public-policyrealm.
Epsteinand Kingtryto sidestepthispointbyadvancing propos-
als forrestructuring the legal academythatwould facilitatemore
timelyand accurateempiricalstudies.But sincetheirproposalsdo
nothingto increasetheresourcesavailableto thelegal academy, the
proposalssimplytransposethetradeoff to theinstitutional level.Le-

50 Epstein and King,69 U Chi L Rev at 7 n 21 (citedin note 1).

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
166 TheUniversity
ofChicagoLaw Review [69:153

gal scholarswhospendyearsstudying econometrics and

andstatistics
generating data setsare legalscholarswhohaveforegone training in
clerkshipsandlawfirms andwhoenterteaching andscholarshipat an
olderage,havingreadfewerlaw cases.To be sure,recentyearshave
seen an increasing (but stillsmall) numberof empirically trained
scholarsenterthe legal academy.But it is simultaneously parochial
and imperialisticforEpsteinand Kingto assert,without demonstra-
tionaccording to therulesofinference, thattheopportunitycostsof
methodological training are identicalin bothlaw schoolsand Ph.D.
programs inpoliticalscience.

CONCLUSION
One motivation forRulesandsimilar politicalscienceworkcriti-
cal oflegalscholarship appearsto be concernaboutthelackofinflu-
ence of pertinent politicalscienceworkon legal scholarship.5" Al-
though we havenotamassedthedataneededtoassessthisclaim(nor,
bytheway,havethepoliticalscientists whoassertit),52we believeitis
misplaced. relatedtovoting,
Politicalscienceliterature apportionment,
politicalparties,
redistricting, thebudgetprocess, has
and legislatures
hadan enormous impactinthelegalworld.53 Leadingtheories oflegal
interpretationhavebeenheavilyinfluenced bypoliticalsciencework
in socialchoiceandpositivepoliticaltheory.54The sameis trueofthe
workof manyleadingconstitutional And variousinterna-
theorists.55
tionalrelationstheories frompoliticalsciencehavehad an enormous
impactoninternational lawscholarship duringthepastdecade.56
politicalscienceappearsto be enjoying
In short, greatsuccessin
thelegalacademy. A largepartofthissuccessis no doubtattributable
to insightsgarneredby politicalscience'sdistinctive methodological

51 See id at 16-17;Gerald N. Rosenberg,AcrosstheGreatDivide (betweenLaw and Politi-

cal Science),3 GreenBag 2d 267,272 (2000).
52 Epstein and King and the Rosenbergpiece cited in the previousfootnoteofferonly

anecdotalsupportfortheirclaims.
53 See, forexample,Samuel Issacharoff and RichardH. Pildes,Politicsas Markets:Partisan
Lockups of theDemocraticProcess,50 Stan L Rev 643 (1998); ElizabethGarrett,Rethinking the
StructuresofDecisionmakingin theFederalBudgetProcess,35 Harv Jon Legis 387 (1998).
54 See, forexample,WilliamN. Eskridge,Jr.and JohnFerejohn,The ArticleI, Section 7
Game, 80 GeorgetownL J 523 (1992); Frank H. Easterbrook,The Role of OriginalIntentin
StatutoryConstruction, 11 Harv JL & Pub Pol 59 (1988).
55 See, forexample,Cass R. Sunstein,FreeMarketsand Social Justice(Oxford1997);Mark
Tushnet,Foreword:The New Constitutional Orderand the Chasteningof ConstitutionalAspira-
tion,113 Harv L Rev 29 (1999); Bruce Ackerman,The New Separationof Powers,113 Harv L
Rev 633 (2000); Michael J.Klarman,Rethinking theCivilRightsand CivilLibertiesRevolutions,
82 Va L Rev 1 (1996).
56 See, for example,Anne-Marie Slaughter,Andrew S. Tulumello,and Stephan Wood,
InternationalLaw and InternationalRelationsTheory:A New Generationof Interdisciplinary
Scholarship,92 Am JIntlL 367 (1998).

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions
2002] Empirical
Methodology
andLegalScholarship 167

approaches.It does notfollow,

however,thatthelegalacademyshould
embraceEpsteinand King'snormative Politicalscien-
prescriptions.
tists'influencehas and willcontinueto come naturally-notwhole-
sale,through theimperialcommandof the"Rules of Inference," but
ratherpiecemeal,through leakageacrossdisciplinary
theintellectual
boundaries ofideasthatlawyers findpersuasive.

This content downloaded from 130.239.116.185 on Wed, 20 Aug 2014 14:15:07 PM

All use subject to JSTOR Terms and Conditions