Designofexperiments

FromWikipedia,thefreeencyclopedia

Thedesignofexperiments(DOE,DOX,orexperimentaldesign)is
thedesignofanytaskthataimstodescribeorexplainthevariationof
informationunderconditionsthatarehypothesizedtoreflectthe
variation.Thetermisgenerallyassociatedwithtrueexperimentsin
whichthedesignintroducesconditionsthatdirectlyaffectthevariation,
butmayalsorefertothedesignofquasiexperiments,inwhichnatural
conditionsthatinfluencethevariationareselectedforobservation.
Initssimplestform,anexperimentaimsatpredictingtheoutcomeby
Designofexperimentswithfull
introducingachangeofthepreconditions,whichisreflectedina
factorialdesign(left),response
variablecalledthepredictor.Thechangeinthepredictorisgenerally
surfacewithseconddegree
hypothesizedtoresultinachangeinthesecondvariable,hencecalled
polynomial(right)
theoutcomevariable.Experimentaldesigninvolvesnotonlythe
selectionofsuitablepredictorsandoutcomes,butplanningthedelivery
oftheexperimentunderstatisticallyoptimalconditionsgiventheconstraintsofavailableresources.
Mainconcernsinexperimentaldesignincludetheestablishmentofvalidity,reliability,andreplicability.For
example,theseconcernscanbepartiallyaddressedbycarefullychoosingthepredictor,reducingtheriskof
measurementerror,andensuringthatthedocumentationofthemethodissufficientlydetailed.Related
concernsincludeachievingappropriatelevelsofstatisticalpowerandsensitivity.
Correctlydesignedexperimentsadvanceknowledgeinthenaturalandsocialsciencesandengineering.Other
applicationsincludemarketingandpolicymaking.

Contents
1 History
1.1 Systematicclinicaltrials
1.2 Statisticalexperiments,followingCharlesS.Peirce
1.2.1 Randomizedexperiments
1.2.2 Optimaldesignsforregressionmodels
1.3 Sequencesofexperiments
2 Fisher'sprinciples
3 Example
4 Avoidingfalsepositives
5 Discussiontopicswhensettingupanexperimentaldesign
6 Causalattributions
7 Statisticalcontrol
8 ExperimentaldesignsafterFisher
9 Humanparticipantconstraints
10 Seealso
11 Notes
12 References
13 Externallinks

History
Systematicclinicaltrials

In1747,whileservingassurgeononHMSSalisbury,JamesLindcarriedoutasystematicclinicaltrialto
compareremediesforscurvy.[1]ThissystematicclinicaltrialconstitutesatypeofDOE.
Lindselected12menfromtheship,allsufferingfromscurvy.Lindlimitedhissubjectstomenwho"wereas
similarasIcouldhavethem,"thatis,heprovidedstrictentryrequirementstoreduceextraneousvariation.He
dividedthemintosixpairs,givingeachpairdifferentsupplementstotheirbasicdietfortwoweeks.The
treatmentswereallremediesthathadbeenproposed:
Aquartofcidereveryday.
Twentyfivegutts(drops)ofvitriol(sulphuricacid)threetimesadayuponanemptystomach.
Onehalfpintofseawatereveryday.
Amixtureofgarlic,mustard,andhorseradishinalumpthesizeofanutmeg.
Twospoonfulsofvinegarthreetimesaday.
Twoorangesandonelemoneveryday.
Thecitrustreatmentstoppedaftersixdayswhentheyranoutoffruit,butbythattimeonesailorwasfitfor
dutywhiletheotherhadalmostrecovered.Apartfromthat,onlygroupone(cider)showedsomeeffectofits
treatment.Theremainderofthecrewpresumablyservedasacontrol,butLinddidnotreportresultsfromany
control(untreated)group.

Statisticalexperiments,followingCharlesS.Peirce
AtheoryofstatisticalinferencewasdevelopedbyCharlesS.Peircein"IllustrationsoftheLogicofScience"
(18771878)and"ATheoryofProbableInference"(1883),twopublicationsthatemphasizedtheimportanceof
randomizationbasedinferenceinstatistics.
Randomizedexperiments
CharlesS.Peircerandomlyassignedvolunteerstoablinded,repeatedmeasuresdesigntoevaluatetheirability
todiscriminateweights.[2][3][4][5]Peirce'sexperimentinspiredotherresearchersinpsychologyandeducation,
whichdevelopedaresearchtraditionofrandomizedexperimentsinlaboratoriesandspecializedtextbooksin
the1800s.[2][3][4][5]
Optimaldesignsforregressionmodels
CharlesS.PeircealsocontributedthefirstEnglishlanguagepublicationonanoptimaldesignforregression
modelsin1876.[6]ApioneeringoptimaldesignforpolynomialregressionwassuggestedbyGergonnein1815.
In1918KirstineSmithpublishedoptimaldesignsforpolynomialsofdegreesix(andless).

Sequencesofexperiments
Theuseofasequenceofexperiments,wherethedesignofeachmaydependontheresultsofprevious
experiments,includingthepossibledecisiontostopexperimenting,iswithinthescopeofSequentialanalysis,a
fieldthatwaspioneered[7]byAbrahamWaldinthecontextofsequentialtestsofstatisticalhypotheses.[8]
HermanChernoffwroteanoverviewofoptimalsequentialdesigns,[9]whileadaptivedesignshavebeen
surveyedbyS.Zacks.[10]Onespecifictypeofsequentialdesignisthe"twoarmedbandit",generalizedtothe
multiarmedbandit,onwhichearlyworkwasdonebyHerbertRobbinsin1952.[11]

Fisher'sprinciples
AmethodologyfordesigningexperimentswasproposedbyRonaldFisher,inhisinnovativebooks:The
ArrangementofFieldExperiments(1926)andTheDesignofExperiments(1935).Muchofhispioneeringwork
dealtwithagriculturalapplicationsofstatisticalmethods.Asamundaneexample,hedescribedhowtotestthe

ladytastingteahypothesis,thatacertainladycoulddistinguishbyflavour
alonewhetherthemilkortheteawasfirstplacedinthecup.These
methodshavebeenbroadlyadaptedinthephysicalandsocialsciences,are
stillusedinagriculturalengineeringanddifferfromthedesignandanalysis
ofcomputerexperiments.
Comparison
Insomefieldsofstudyitisnotpossibletohaveindependent
measurementstoatraceablemetrologystandard.Comparisons
betweentreatmentsaremuchmorevaluableandareusually
preferable,andoftencomparedagainstascientificcontrolor
traditionaltreatmentthatactsasbaseline.
Randomization
Randomassignmentistheprocessofassigningindividualsat
RonaldFisher
randomtogroupsortodifferentgroupsinanexperiment,sothat
eachindividualofthepopulationhasthesamechanceofbecominga
participantinthestudy.Therandomassignmentofindividualsto
groups(orconditionswithinagroup)distinguishesarigorous,"true"experimentfromanobservational
studyor"quasiexperiment".[12]Thereisanextensivebodyofmathematicaltheorythatexploresthe
consequencesofmakingtheallocationofunitstotreatmentsbymeansofsomerandommechanismsuch
astablesofrandomnumbers,ortheuseofrandomizationdevicessuchasplayingcardsordice.
Assigningunitstotreatmentsatrandomtendstomitigateconfounding,whichmakeseffectsdueto
factorsotherthanthetreatmenttoappeartoresultfromthetreatment.Therisksassociatedwithrandom
allocation(suchashavingaseriousimbalanceinakeycharacteristicbetweenatreatmentgroupanda
controlgroup)arecalculableandhencecanbemanageddowntoanacceptablelevelbyusingenough
experimentalunits.However,ifthepopulationisdividedintoseveralsubpopulationsthatsomehow
differ,andtheresearchrequireseachsubpopulationtobeequalinsize,stratifiedsamplingcanbeused.
Inthatway,theunitsineachsubpopulationarerandomized,butnotthewholesample.Theresultsofan
experimentcanbegeneralizedreliablyfromtheexperimentalunitstoalargerstatisticalpopulationof
unitsonlyiftheexperimentalunitsarearandomsamplefromthelargerpopulationtheprobableerrorof
suchanextrapolationdependsonthesamplesize,amongotherthings.
Statisticalreplication
Measurementsareusuallysubjecttovariationandmeasurementuncertaintythustheyarerepeatedand
fullexperimentsarereplicatedtohelpidentifythesourcesofvariation,tobetterestimatethetrueeffects
oftreatments,tofurtherstrengthentheexperiment'sreliabilityandvalidity,andtoaddtotheexisting
knowledgeofthetopic.[13]However,certainconditionsmustbemetbeforethereplicationofthe
experimentiscommenced:theoriginalresearchquestionhasbeenpublishedinapeerreviewedjournal
orwidelycited,theresearcherisindependentoftheoriginalexperiment,theresearchermustfirsttryto
replicatetheoriginalfindingsusingtheoriginaldata,andthewriteupshouldstatethatthestudy
conductedisareplicationstudythattriedtofollowtheoriginalstudyasstrictlyaspossible.[14]
Blocking
Blockingisthearrangementofexperimentalunitsintogroups(blocks/lots)consistingofunitsthatare
similartooneanother.Blockingreducesknownbutirrelevantsourcesofvariationbetweenunitsandthus
allowsgreaterprecisionintheestimationofthesourceofvariationunderstudy.
Orthogonality
Orthogonalityconcernstheformsofcomparison(contrasts)thatcanbelegitimatelyandefficiently
carriedout.Contrastscanberepresentedbyvectorsandsetsoforthogonalcontrastsareuncorrelatedand
independentlydistributedifthedataarenormal.Becauseofthisindependence,eachorthogonaltreatment
providesdifferentinformationtotheothers.IfthereareTtreatmentsandT1orthogonalcontrasts,all
theinformationthatcanbecapturedfromtheexperimentisobtainablefromthesetofcontrasts.
Factorialexperiments

Useoffactorialexperimentsinsteadoftheonefactoratatime
method.Theseareefficientatevaluatingtheeffectsandpossible
interactionsofseveralfactors(independentvariables).Analysis
ofexperimentdesignisbuiltonthefoundationoftheanalysisof
variance,acollectionofmodelsthatpartitiontheobserved
varianceintocomponents,accordingtowhatfactorsthe
experimentmustestimateortest.

Example

Exampleoforthogonalfactorial
design

ThisexampleisattributedtoHaroldHotelling.[9]Itconveyssomeof
theflavorofthoseaspectsofthesubjectthatinvolvecombinatorial
designs.
Weightsofeightobjectsaremeasuredusingapanbalanceandsetof
standardweights.Eachweighingmeasurestheweightdifference
betweenobjectsintheleftpanvs.anyobjectsintherightpanby
addingcalibratedweightstothelighterpanuntilthebalanceisin
equilibrium.Eachmeasurementhasarandomerror.Theaverageerror
iszerothestandarddeviationsoftheprobabilitydistributionofthe
errorsisthesamenumberondifferentweighingserrorsondifferent
weighingsareindependent.Denotethetrueweightsby

Weconsidertwodifferentexperiments:
1.Weigheachobjectinonepan,withtheotherpanempty.LetXibethemeasuredweightoftheobject,fori
=1,...,8.
2.DotheeightweighingsaccordingtothefollowingscheduleandletYibethemeasureddifferencefori=
1,...,8:

Thentheestimatedvalueoftheweight1is

Similarestimatescanbefoundfortheweightsoftheotheritems.Forexample

Thequestionofdesignofexperimentsis:whichexperimentisbetter?
ThevarianceoftheestimateX1of1is2ifweusethefirstexperiment.Butifweusethesecondexperiment,
thevarianceoftheestimategivenaboveis2/8.Thusthesecondexperimentgivesus8timesasmuch
precisionfortheestimateofasingleitem,andestimatesallitemssimultaneously,withthesameprecision.
Whatthesecondexperimentachieveswitheightwouldrequire64weighingsiftheitemsareweighed
separately.However,notethattheestimatesfortheitemsobtainedinthesecondexperimenthaveerrorsthat
correlatewitheachother.
Manyproblemsofthedesignofexperimentsinvolvecombinatorialdesigns,asinthisexampleandothers.[15]

Avoidingfalsepositives
Falsepositiveconclusions,oftenresultingfromthepressuretopublishortheauthor'sownconfirmationbias,
areaninherenthazardinmanyfields.Agoodwaytopreventbiasespotentiallyleadingtofalsepositivesinthe
datacollectionphaseistouseadoubleblinddesign.Whenadoubleblinddesignisused,participantsare
randomlyassignedtoexperimentalgroupsbuttheresearcherisunawareofwhatparticipantsbelongtowhich
group.Therefore,theresearchercannotaffecttheparticipants'responsetotheintervention.Experimental
designswithundiscloseddegreesoffreedomareaproblem.[16]Thiscanleadtoconsciousorunconscious"p
hacking":tryingmultiplethingsuntilyougetthedesiredresult.Ittypicallyinvolvesthemanipulationperhaps
unconsciouslyoftheprocessofstatisticalanalysisandthedegreesoffreedomuntiltheyreturnafigurebelow
thep<.05levelofstatisticalsignificance.[17][18]Sothedesignoftheexperimentshouldincludeaclear
statementproposingtheanalysestobeundertaken.Phackingcanbepreventedbypreregisteringresearches,in
whichresearchershavetosendtheirdataanalysisplantothejournaltheywishtopublishtheirpaperinbefore
theyevenstarttheirdatacollection,sonodataminingispossible(https://osf.io).Anotherwaytopreventthisis
takingthedoubleblinddesigntothedataanalysisphase,wherethedataaresenttoadataanalystunrelatedto
theresearchwhoscramblesupthedatasothereisnowaytoknowwhichparticipantsbelongtobeforetheyare
potentiallytakenawayasoutliers.
Clearandcompletedocumentationoftheexperimentalmethodologyisalsoimportantinordertosupport
replicationofresults.[19]

Discussiontopicswhensettingupanexperimentaldesign
Anexperimentaldesignorrandomizedclinicaltrialrequirescarefulconsiderationofseveralfactorsbefore
actuallydoingtheexperiment.[20]Anexperimentaldesignisthelayingoutofadetailedexperimentalplanin
advanceofdoingtheexperiment.Someofthefollowingtopicshavealreadybeendiscussedintheprinciplesof
experimentaldesignsection:
1.Howmanyfactorsdoesthedesignhave?andarethelevelsofthesefactorsfixedorrandom?
2.Arecontrolconditionsneeded,andwhatshouldtheybe?

3.Manipulationchecksdidthemanipulationreallywork?
4.Whatarethebackgroundvariables?
5.Whatisthesamplesize.Howmanyunitsmustbecollectedfortheexperimenttobegeneralisableand
haveenoughpower?
6.Whatistherelevanceofinteractionsbetweenfactors?
7.Whatistheinfluenceofdelayedeffectsofsubstantivefactorsonoutcomes?
8.Howdoresponseshiftsaffectselfreportmeasures?
9.Howfeasibleisrepeatedadministrationofthesamemeasurementinstrumentstothesameunitsat
differentoccasions,withaposttestandfollowuptests?
10.Whataboutusingaproxypretest?
11.Aretherelurkingvariables?
12.Shouldtheclient/patient,researcheroreventheanalystofthedatabeblindtoconditions?
13.Whatisthefeasibilityofsubsequentapplicationofdifferentconditionstothesameunits?
14.Howmanyofeachcontrolandnoisefactorsshouldbetakenintoaccount?
Theindependentvariableofastudyoftenhasmanylevelsordifferentgroups.Inatrueexperiment,researchers
canhaveanexperimentalgroup,whichiswheretheirinterventiontestingthehypothesisisimplemented,anda
controlgroup,whichhasallthesameelementastheexperimentalgroup,withouttheinterventionalelement.
Thus,wheneverythingelseexceptforoneinterventionisheldconstant,researcherscancertifywithsome
certaintythatthisoneelementiswhatcausedtheobservedchange.Insomeinstances,havingacontrolgroupis
notethical.Thisissometimessolvedusingtwodifferentexperimentalgroups.Insomecases,independent
variablesarenotmanipulable,forexamplewhentestingthedifferencebetweentwogroupswhohavea
differentdisease,ortestingthedifferencebetweengenders(obviouslyvariablesthatwouldbehardorunethical
toassignparticipantsto).Inthesecases,aquasiexperimentaldesignmaybeused.

Causalattributions
Inthepureexperimentaldesign,theindependent(predictor)variableismanipulatedbytheresearcherthatis
everyparticipantoftheresearchischosenrandomlyfromthepopulation,andeachparticipantchosenis
assignedrandomlytoconditionsoftheindependentvariable.Onlywhenthisisdoneisitpossibletocertify
withhighprobabilitythatthereasonforthedifferencesintheoutcomevariablesarecausedbythedifferent
conditions.Therefore,researchersshouldchoosetheexperimentaldesignoverotherdesigntypeswhenever
possible.However,thenatureoftheindependentvariabledoesnotalwaysallowformanipulation.Inthose
cases,researchersmustbeawareofnotcertifyingaboutcausalattributionwhentheirdesigndoesn'tallowfor
it.Forexample,inobservationaldesigns,participantsarenotassignedrandomlytoconditions,andsoifthere
aredifferencesfoundinoutcomevariablesbetweenconditions,itislikelythatthereissomethingotherthanthe
differencesbetweentheconditionsthatcausesthedifferencesinoutcomes,thatisathirdvariable.Thesame
goesforstudieswithcorrelationaldesign.(Adr&Mellenbergh,2008).

Statisticalcontrol
Itisbestthataprocessbeinreasonablestatisticalcontrolpriortoconductingdesignedexperiments.Whenthis
isnotpossible,properblocking,replication,andrandomizationallowforthecarefulconductofdesigned
experiments.[21]Tocontrolfornuisancevariables,researchersinstitutecontrolchecksasadditionalmeasures.
Investigatorsshouldensurethatuncontrolledinfluences(e.g.,sourcecredibilityperception)donotskewthe
findingsofthestudy.Amanipulationcheckisoneexampleofacontrolcheck.Manipulationchecksallow
investigatorstoisolatethechiefvariablestostrengthensupportthatthesevariablesareoperatingasplanned.
Oneofthemostimportantrequirementsofexperimentalresearchdesignsisthenecessityofeliminatingthe
effectsofspurious,intervening,andantecedentvariables.Inthemostbasicmodel,cause(X)leadstoeffect
(Y).Buttherecouldbeathirdvariable(Z)thatinfluences(Y),andXmightnotbethetruecauseatall.Zis
saidtobeaspuriousvariableandmustbecontrolledfor.Thesameistrueforinterveningvariables(avariable
inbetweenthesupposedcause(X)andtheeffect(Y)),andantecedingvariables(avariablepriortothe

supposedcause(X)thatisthetruecause).Whenathirdvariableisinvolvedandhasnotbeencontrolledfor,the
relationissaidtobeazeroorderrelationship.Inmostpracticalapplicationsofexperimentalresearchdesigns
thereareseveralcauses(X1,X2,X3).Inmostdesigns,onlyoneofthesecausesismanipulatedatatime.

ExperimentaldesignsafterFisher
Someefficientdesignsforestimatingseveralmaineffectswerefoundindependentlyandinnearsuccessionby
RajChandraBoseandK.Kishenin1940attheIndianStatisticalInstitute,butremainedlittleknownuntilthe
PlackettBurmandesignswerepublishedinBiometrikain1946.Aboutthesametime,C.R.Raointroducedthe
conceptsoforthogonalarraysasexperimentaldesigns.Thisconceptplayedacentralroleinthedevelopmentof
TaguchimethodsbyGenichiTaguchi,whichtookplaceduringhisvisittoIndianStatisticalInstituteinearly
1950s.HismethodsweresuccessfullyappliedandadoptedbyJapaneseandIndianindustriesandsubsequently
werealsoembracedbyUSindustryalbeitwithsomereservations.
In1950,GertrudeMaryCoxandWilliamGemmellCochranpublishedthebookExperimentalDesigns,which
becamethemajorreferenceworkonthedesignofexperimentsforstatisticiansforyearsafterwards.
Developmentsofthetheoryoflinearmodelshaveencompassedandsurpassedthecasesthatconcernedearly
writers.Today,thetheoryrestsonadvancedtopicsinlinearalgebra,algebraandcombinatorics.
Aswithotherbranchesofstatistics,experimentaldesignispursuedusingbothfrequentistandBayesian
approaches:Inevaluatingstatisticalprocedureslikeexperimentaldesigns,frequentiststatisticsstudiesthe
samplingdistributionwhileBayesianstatisticsupdatesaprobabilitydistributionontheparameterspace.
SomeimportantcontributorstothefieldofexperimentaldesignsareC.S.Peirce,R.A.Fisher,F.Yates,C.R.
Rao,R.C.Bose,J.N.Srivastava,ShrikhandeS.S.,D.Raghavarao,W.G.Cochran,O.Kempthorne,W.T.
Federer,V.V.Fedorov,A.S.Hedayat,J.A.Nelder,R.A.Bailey,J.Kiefer,W.J.Studden,A.Pzman,F.
Pukelsheim,D.R.Cox,H.P.Wynn,A.C.Atkinson,G.E.P.BoxandG.Taguchi.ThetextbooksofD.
MontgomeryandR.Myershavereachedgenerationsofstudentsandpractitioners.[22][23][24]

Humanparticipantconstraints
Lawsandethicalconsiderationsprecludesomecarefullydesignedexperimentswithhumansubjects.Legal
constraintsaredependentonjurisdiction.Constraintsmayinvolveinstitutionalreviewboards,informed
consentandconfidentialityaffectingbothclinical(medical)trialsandbehavioralandsocialscience
experiments.[25]Inthefieldoftoxicology,forexample,experimentationisperformedonlaboratoryanimals
withthegoalofdefiningsafeexposurelimitsforhumans.[26]Balancingtheconstraintsareviewsfromthe
medicalfield.[27]Regardingtherandomizationofpatients,"...ifnooneknowswhichtherapyisbetter,thereis
noethicalimperativetouseonetherapyoranother."(p380)Regardingexperimentaldesign,"...itisclearlynot
ethicaltoplacesubjectsatrisktocollectdatainapoorlydesignedstudywhenthissituationcanbeeasily
avoided...".(p393)

Seealso
Adversarialcollaboration
Bayesianexperimentaldesign
Clinicaltrial
Computerexperiment
Controlvariable
Controllingforavariable
Experimetrics(econometricsrelated
experiments)
Factoranalysis

Firstinmanstudy
Glossaryofexperimentaldesign
Greyboxmodel
Industrialengineering
Instrumenteffect
Lawoflargenumbers
Manipulationchecks
Multifactordesignofexperimentssoftware
Probabilisticdesign

Protocol(naturalsciences)
Quasiexperimentaldesign
Randomizedblockdesign
Randomizedcontrolledtrial
Researchdesign
Robustparameterdesign

Supersaturateddesign
Surveysampling
Systemidentification
Taguchimethods

Notes
1.Dunn,Peter(January1997)."JamesLind(171694)ofEdinburghandthetreatmentofscurvy".ArchiveofDiseasein
ChildhoodFetalandNeonatalEdition.UnitedKingdom:BritishMedicalJournalPublishingGroup.76(1):6465.
doi:10.1136/fn.76.1.F64.PMC1720613 .PMID9059193.Retrieved20090117.
2.Peirce,CharlesSandersJastrow,Joseph(1885)."OnSmallDifferencesinSensation".MemoirsoftheNational
AcademyofSciences.3:7383.
3.Hacking,Ian(September1988)."Telepathy:OriginsofRandomizationinExperimentalDesign".Isis.79(3):427
451.doi:10.1086/354775.JSTOR234674.MR1013489.
4.StephenM.Stigler(November1992)."AHistoricalViewofStatisticalConceptsinPsychologyandEducational
Research".AmericanJournalofEducation.101(1):6070.doi:10.1086/444032.JSTOR1085417.
5.TrudyDehue(December1997)."Deception,Efficiency,andRandomGroups:PsychologyandtheGradual
OriginationoftheRandomGroupDesign".Isis.88(4):653673.doi:10.1086/383850.PMID9519574.
6.Peirce,C.S.(1876)."NoteontheTheoryoftheEconomyofResearch".CoastSurveyReport:197201.,actually
published1879,NOAAPDFEprint(http://docs.lib.noaa.gov/rescue/cgs/001_pdf/CSC0025.PDF#page=222).
ReprintedinCollectedPapers7,paragraphs139157,alsoinWritings4,pp.7278,andinPeirce,C.S.(July
August1967)."NoteontheTheoryoftheEconomyofResearch".OperationsResearch.15(4):643648.
doi:10.1287/opre.15.4.643.JSTOR168276.
7.Johnson,N.L.(1961)."Sequentialanalysis:asurvey."JournaloftheRoyalStatisticalSociety,SeriesA.Vol.124(3),
372411.(pages375376)
8.Wald,A.(1945)"SequentialTestsofStatisticalHypotheses",AnnalsofMathematicalStatistics,16(2),117186.
9.HermanChernoff,SequentialAnalysisandOptimalDesign,SIAMMonograph,1972.
10.Zacks,S.(1996)"AdaptiveDesignsforParametricModels".In:Ghosh,S.andRao,C.R.,(Eds)(1996)."Design
andAnalysisofExperiments,"HandbookofStatistics,Volume13.NorthHolland.ISBN0444820612.(pages151
180)
11.Robbins,H.(1952)."SomeAspectsoftheSequentialDesignofExperiments".BulletinoftheAmerican
MathematicalSociety.58(5):527535.doi:10.1090/S000299041952096208.
12.Creswell,J.W.(2008).Educationalresearch:Planning,conducting,andevaluatingquantitativeandqualitative
research(3rd).UpperSaddleRiver,NJ:PrenticeHall.2008,p.300.ISBN0136135501
13.Dr.Hani(2009)."Replicationstudy".Retrieved27October2011.
14.Burman,LeonardE.RobertW.ReedJamesAlm(2010)."Acallforreplicationstudies"(journalarticle).Public
FinanceReview.pp.787793.doi:10.1177/1091142110385210.Retrieved27October2011.
15.JackSifri(8December2014)."HowtoUseDesignofExperimentstoCreateRobustDesignsWithHighYield".
youtube.com.Retrieved20150211.
16.Simmons,JosephLeifNelsonUriSimonsohn(November2011)."FalsePositivePsychology:Undisclosed
FlexibilityinDataCollectionandAnalysisAllowsPresentingAnythingasSignificant".PsychologicalScience.
WashingtonDC:AssociationforPsychologicalScience.22(11):13591366.doi:10.1177/0956797611417632.
ISSN09567976.PMID22006061.Retrieved29January2012.
17."Science,TrustAndPsychologyInCrisis".KPLU.20140602.Retrieved20140612.
18."WhyStatisticallySignificantStudiesCanBeInsignificant".PacificStandard.20140604.Retrieved20140612.
19.ChrisChambers(20140610)."Physicsenvy:Do'hard'sciencesholdthesolutiontothereplicationcrisisin
psychology?".theguardian.com.Retrieved20140612.
20.Ader,Mellenberg&Hand(2008)"AdvisingonResearchMethods:Aconsultant'scompanion"
21.Bisgaard,S(2008)"MustaProcessbeinStatisticalControlbeforeConductingDesignedExperiments?",Quality
Engineering,ASQ,20(2),pp143176
22.Montgomery,Douglas(2013).Designandanalysisofexperiments(8thed.).Hoboken,NJ:JohnWiley&Sons,Inc.
ISBN9781118146927.
23.Walpole,RonaldE.Myers,RaymondH.Myers,SharonL.Ye,Keying(2007).Probability&statisticsfor
engineers&scientists(8ed.).UpperSaddleRiver,NJ:PearsonPrenticeHall.ISBN9780131877115.
24.Myers,RaymondH.Montgomery,DouglasC.Vining,G.GeoffreyRobinson,TimothyJ.(2010).Generalized
linearmodels:withapplicationsinengineeringandthesciences(2ed.).Hoboken,N.J:Wiley.ISBN978
0470454633.

25.Moore,DavidS.Notz,WilliamI.(2006).Statistics:conceptsandcontroversies(6thed.).NewYork:W.H.
Freeman.pp.Chapter7:Dataethics.ISBN9780716786368.
26.Ottoboni,M.Alice(1991).Thedosemakesthepoison:aplainlanguageguidetotoxicology(2nded.).NewYork,
N.Y:VanNostrandReinhold.ISBN0442006608.
27.Glantz,StantonA.(1992).Primerofbiostatistics(3rded.).ISBN0070235112.

References
Peirce,C.S.(18771878),"IllustrationsoftheLogicofScience"(series),PopularScienceMonthly,
vols.1213.Relevantindividualpapers:
(1878March),"TheDoctrineofChances",PopularScienceMonthly,v.12,Marchissue,pp.604
(https://books.google.com/books?id=ZKMVAAAAYAAJ&jtp=604)615.InternetArchiveEprint
(https://archive.org/stream/popscimonthly12yoummiss#page/612/mode/1up).
(1878April),"TheProbabilityofInduction",PopularScienceMonthly,v.12,pp.705(https://book
s.google.com/books?id=ZKMVAAAAYAAJ&jtp=705)718.InternetArchiveEprint(https://archiv
e.org/stream/popscimonthly12yoummiss#page/715/mode/1up).
(1878June),"TheOrderofNature",PopularScienceMonthly,v.13,pp.203(https://books.googl
e.com/books?id=u8sWAQAAIAAJ&jtp=203)217.InternetArchiveEprint(https://archive.org/stre
am/popularsciencemo13newy#page/203/mode/1up).
(1878August),"Deduction,Induction,andHypothesis",PopularScienceMonthly,v.13,pp.470
(https://books.google.com/books?id=u8sWAQAAIAAJ&jtp=470)482.InternetArchiveEprint(htt
ps://archive.org/stream/popularsciencemo13newy#page/470/mode/1up).
Peirce,C.S.(1883),"ATheoryofProbableInference",StudiesinLogic,pp.126181(https://book
s.google.com/books?id=V7oIAAAAQAAJ&pg=PA126),Little,Brown,andCompany.(Reprinted
1983,JohnBenjaminsPublishingCompany,ISBN9027232717)

Externallinks
Achapter(http://www.itl.nist.gov/div898/handbook/pri/section1/pri1.htm)froma"NIST/SEMATECH
HandbookonEngineeringStatistics"(http://www.itl.nist.gov/div898/handbook/)atNIST
BoxBehnkendesigns(http://www.itl.nist.gov/div898/handbook/pri/section3/pri3362.htm)froma
"NIST/SEMATECHHandbookonEngineeringStatistics"(http://www.itl.nist.gov/div898/handbook/)at
NIST
Retrievedfrom"https://en.wikipedia.org/w/index.php?title=Design_of_experiments&oldid=747864953"
Categories: Designofexperiments Statisticalmethods Statisticaltheory Industrialengineering
Systemsengineering Qualitycontrol Quality Quantitativeresearch Engineeringstatistics Experiments
Thispagewaslastmodifiedon4November2016,at20:55.
TextisavailableundertheCreativeCommonsAttributionShareAlikeLicenseadditionaltermsmay
apply.Byusingthissite,youagreetotheTermsofUseandPrivacyPolicy.Wikipediaisaregistered
trademarkoftheWikimediaFoundation,Inc.,anonprofitorganization.