Research Method, Zoltan

5HVHDUFK0HWKRGVLQ$SSOLHG/LQJXLVWLFV

>283WRFKHFN@

)RU2[IRUG$SSOLHG/LQJXLVWLFV
3XEOLVKHGLQWKLVVHULHV

%DFKPDQ)XQGDPHQWDO&RQVLGHUDWLRQVLQ/DQJXDJH7HVWLQJ
%DFKPDQDQG3DOPHU/DQJXDJH7HVWLQJLQ3UDFWLFH
%UXPILW,QGLYLGXDO)UHHGRPDQG/DQJXDJH7HDFKLQJ
%UXPILWDQG&DUWHU HGV /LWHUDWXUHDQG/DQJXDJH7HDFKLQJ
&DQDJDUDMDK5HVLVWLQJ/LQJXLVWLF,PSHULDOLVPLQ/DQJXDJH7HDFKLQJ
&RRN'LVFRXUVHDQG/LWHUDWXUH
&RRN/DQJXDJH3OD\/DQJXDJH/HDUQLQJ
&RRNDQG6HLGOKRIHU HGV 3ULQFLSOHDQG3UDFWLFHLQ$SSOLHG/LQJXLVWLFV
(OOLV6/$5HVHDUFKDQG/DQJXDJH7HDFKLQJ
(OOLV7DVNEDVHG/DQJXDJH/HDUQLQJDQG7HDFKLQJ
(OOLV7KH6WXG\RI6HFRQG/DQJXDJH$FTXLVLWLRQ
(OOLV8QGHUVWDQGLQJ6HFRQG/DQJXDJH$FTXLVLWLRQ
(OOLVDQG%DUNKXL]HQ$QDO\VLQJ/HDUQHU/DQJXDJH

)RWRVDQG1DVVDML HGV )RUPIRFXVHG,QVWUXFWLRQDQG7HDFKHU(GXFDWLRQ
+ROOLGD\7KH6WUXJJOHWR7HDFK(QJOLVKDVDQ,QWHUQDWLRQDO/DQJXDJH
+RZDWW$+LVWRU\RI(QJOLVK/DQJXDJH7HDFKLQJ
-HQNLQV7KH3KRQRORJ\RI(QJOLVKDVDQ,QWHUQDWLRQDO/DQJXDJH
.HUQ/LWHUDF\DQG/DQJXDJH7HDFKLQJ
.UDPVFK&RQWH[WDQG&XOWXUHLQ/DQJXDJH7HDFKLQJ
/DQWROI HG 6RFLRFXOWXUDO7KHRU\DQG6HFRQG/DQJXDJH/HDUQLQJ
/DQWROIDQG7KRUQH6RFLRFXOWXUDO7KHRU\DQGWKH*HQHVLVRI6HFRQG/DQJXDJH
'HYHORSPHQW
0HLQKRI/DQJXDJH/HDUQLQJLQWKH$JHRI6DWHOOLWH7HOHYLVLRQ
1DWWLQJHUDQG'H&DUULFR/H[LFDO3KUDVHVDQG/DQJXDJH7HDFKLQJ
3KLOOLSVRQ/LQJXLVWLF,PSHULDOLVP
6HLGOKRIHU HG &RQWURYHUVLHVLQ$SSOLHG/LQJXLVWLFV
6HOLJHUDQG6KRKDP\6HFRQG/DQJXDJH5HVHDUFK0HWKRGV
6NHKDQ$&RJQLWLYH$SSURDFKWR/DQJXDJH/HDUQLQJ
6WHUQ)XQGDPHQWDO&RQFHSWVRI/DQJXDJH7HDFKLQJ
6WHUQ HGV3$OOHQDQG%+DUOH\ ,VVXHVDQG2SWLRQVLQ/DQJXDJH7HDFKLQJ
7DURQHDQG<XOH)RFXVRQWKH/DQJXDJH/HDUQHU
:LGGRZVRQ$VSHFWVRI/DQJXDJH7HDFKLQJ
:LGGRZVRQ'HILQLQJ,VVXHVLQ(QJOLVK/DQJXDJH7HDFKLQJ
:LGGRZVRQ3UDFWLFDO6W\OLVWLFV
:LGGRZVRQ7HDFKLQJ/DQJXDJHDV&RPPXQLFDWLRQ

5HVHDUFK0HWKRGVLQ$SSOLHG/LQJXLVWLFV

4XDQWLWDWLYH4XDOLWDWLYHDQG0L[HG0HWKRGRORJLHV

=ROWiQ'|UQ\HL

2[IRUG8QLYHUVLW\3UHVV

>283WRFKHFN@

*UHDW&ODUHQGRQ6WUHHW2[IRUG2;'3

2[IRUG8QLYHUVLW\3UHVVLVDGHSDUWPHQWRIWKH8QLYHUVLW\RI2[IRUG
,WIXUWKHUVWKH8QLYHUVLW\¶VREMHFWLYHRIH[FHOOHQFHLQUHVHDUFKVFKRODUVKLS
DQGHGXFDWLRQE\SXEOLVKLQJZRUOGZLGHLQ

2[IRUG1HZ<RUN

$XFNODQG%DQJNRN%XHQRV$LUHV&DSH7RZQ&KHQQDL
'DUHV6DODDP'HOKL+RQJ.RQJ,VWDQEXO.DUDFKL.RONDWD
.XDOD/XPSXU0DGULG0HOERXUQH0H[LFR&LW\0XPEDL1DLUREL
6mR3DXOR6KDQJKDL6LQJDSRUH7DLSHL7RN\R7RURQWR

ZLWKDQDVVRFLDWHGFRPSDQ\LQ%HUOLQ

2[IRUGDQG2[IRUG(QJOLVKDUHUHJLVWHUHGWUDGHPDUNVRI
2[IRUG8QLYHUVLW\3UHVVLQWKH8.DQGLQFHUWDLQRWKHUFRXQWULHV


7KHPRUDOULJKWVRIWKHDXWKRUKDYHEHHQDVVHUWHG

'DWDEDVHULJKW2[IRUG8QLYHUVLW\3UHVV PDNHU

)LUVWSXEOLVKHG

1RXQDXWKRUL]HGSKRWRFRS\LQJ
$OOULJKWVUHVHUYHG1RSDUWRIWKLVSXEOLFDWLRQPD\EHUHSURGXFHG
VWRUHGLQDUHWULHYDOV\VWHPRUWUDQVPLWWHGLQDQ\IRUPRUE\DQ\
PHDQVZLWKRXWWKHSULRUSHUPLVVLRQLQZULWLQJRI2[IRUG8QLYHUVLW\
3UHVVRUDVH[SUHVVO\SHUPLWWHGE\ODZRUXQGHUWHUPVDJUHHG
ZLWKWKHDSSURSULDWHUHSURJUDSKLFVULJKWVRUJDQL]DWLRQ(QTXLULHV
FRQFHUQLQJUHSURGXFWLRQRXWVLGHWKHVFRSHRIWKHDERYHVKRXOG
EHVHQWWRWKH(/75LJKWV'HSDUWPHQW2[IRUG8QLYHUVLW\3UHVV
DWWKHDGGUHVVDERYH

<RXPXVWQRWFLUFXODWHWKLVERRNLQDQ\RWKHUELQGLQJRUFRYHU

DQG\RXPXVWLPSRVHWKLVVDPHFRQGLWLRQRQDQ\DFTXLUHU

$Q\ZHEVLWHVUHIHUUHGWRLQWKLVSXEOLFDWLRQDUHLQWKHSXEOLFGRPDLQDQG
WKHLUDGGUHVVHVDUHSURYLGHGE\2[IRUG8QLYHUVLW\3UHVVIRULQIRUPDWLRQRQO\
2[IRUG8QLYHUVLW\3UHVVGLVFODLPVDQ\UHVSRQVLELOLW\IRUWKHFRQWHQW

,6%1[[[[[[[>&+(&.@

3ULQWHGLQ

7DEOHRIFRQWHQWV
3UHIDFH
$FNQRZOHGJHPHQWV
3DUW.H\LVVXHVLQUHVHDUFKPHWKRGRORJ\
,QWURGXFWLRQ:KDWLVUHVHDUFKDQGKRZGRHVWKLVERRNDSSURDFKLW"
7KHJRRGUHVHDUFKHU
0\DSSURDFKWRUHVHDUFKPHWKRGRORJ\
7KHFRQWHQWRIWKLVERRN
)XUWKHUUHDGLQJ
4XDOLWDWLYHTXDQWLWDWLYHDQGPL[HGPHWKRGVUHVHDUFK
7KHTXDOLWDWLYH±TXDQWLWDWLYHGLVWLQFWLRQ
4XDQWLWDWLYHUHVHDUFK
4XDOLWDWLYHUHVHDUFK
0L[HGPHWKRGVUHVHDUFK
0\RZQSDUDGLJPDWLFVWDQFH
4XDOLW\FULWHULDUHVHDUFKHWKLFVDQGRWKHUUHVHDUFKLVVXHV
4XDOLW\FULWHULDIRUUHVHDUFK
5HVHDUFKHWKLFV
5HVHDUFKTXHVWLRQVDQGK\SRWKHVHV
2WKHUHVVHQWLDOVIRUODXQFKLQJDVWXG\SLORWVWXG\UHVHDUFKORJDQGGDWDPDQDJHPHQW
/RQJLWXGLQDOYHUVXVFURVVVHFWLRQDOUHVHDUFK
'HILQLWLRQDQGSXUSRVHRIORQJLWXGLQDOUHVHDUFK
/RQJLWXGLQDOUHVHDUFKLQDSSOLHGOLQJXLVWLFV
0DLQW\SHVRIORQJLWXGLQDOGHVLJQV
/RQJLWXGLQDOTXDOLWDWLYHUHVHDUFK
/RQJLWXGLQDOPL[HGGHVLJQV
&KRRVLQJDGHVLJQORQJLWXGLQDORUFURVVVHFWLRQDO"
4XDQWLWDWLYHORQJLWXGLQDOGDWDDQDO\VLV
3DUW'DWDFROOHFWLRQ
4XDQWLWDWLYHGDWDFROOHFWLRQ
6DPSOLQJLQTXDQWLWDWLYHUHVHDUFK
4XHVWLRQQDLUHVXUYH\V
([SHULPHQWDODQGTXDVLH[SHULPHQWDOVWXGLHV
&ROOHFWLQJTXDQWLWDWLYHGDWDYLDWKH,QWHUQHW
4XDOLWDWLYHGDWDFROOHFWLRQ
4XDOLWDWLYHGDWD
6DPSOLQJLQTXDOLWDWLYHUHVHDUFK
(WKQRJUDSK\
,QWHUYLHZV
)RFXVJURXSLQWHUYLHZV
,QWURVSHFWLYHPHWKRGV
&DVHVWXGLHV
'LDU\VWXGLHV
5HVHDUFKMRXUQDOV
0L[HGPHWKRGVUHVHDUFKSXUSRVHDQGGHVLJQ
7KHSXUSRVHRIPL[HGPHWKRGVUHVHDUFK
7KHFRPSDWLELOLW\RIGLIIHUHQWUHVHDUFKSDUDGLJPV
0DLQW\SHVRIPL[HGPHWKRGVGHVLJQV
:K\GRQ¶WSHRSOHPL[PHWKRGVPRUH"
&ODVVURRPUHVHDUFK
2ULJLQVDQGPDLQW\SHVRIFODVVURRPUHVHDUFK
&ODVVURRPREVHUYDWLRQ
0L[HGPHWKRGVFODVVURRPUHVHDUFK
'LIILFXOWLHVDQGFKDOOHQJHVLQFODVVURRPUHVHDUFK

$FWLRQUHVHDUFK
3DUW'DWDDQDO\VLV
4XDQWLWDWLYHGDWDDQDO\VLV
&RPSXWHUL]HGGDWDDQDO\VLVDQG6366
3UHSDULQJWKHGDWDIRUDQDO\VLV
'DWDUHGXFWLRQDQGUHOLDELOLW\DQDO\VLV
.H\VWDWLVWLFDOFRQFHSWV
'HVFULSWLYHVWDWLVWLFV
&RPSDULQJWZRJURXSVWWHVWV
&RPSDULQJPRUHWKDQWZRJURXSVDQDO\VLVRIYDULDQFH $129$
&RUUHODWLRQ
1RQSDUDPHWULFWHVWV
$GYDQFHGVWDWLVWLFDOSURFHGXUHV
4XDOLWDWLYHGDWDDQDO\VLV
0DLQSULQFLSOHVRITXDOLWDWLYHGDWDDQDO\VLV
4XDOLWDWLYHFRQWHQWDQDO\VLV
*URXQGHGWKHRU\
&RPSXWHUDLGHGTXDOLWDWLYHGDWDDQDO\VLV &$4'$6
'DWDDQDO\VLVLQPL[HGPHWKRGVUHVHDUFK
'DWDWUDQVIRUPDWLRQ
([WUHPHFDVHDQDO\VLV
7\SRORJ\FDWHJRU\GHYHORSPHQW
0XOWLSOHOHYHODQDO\VLV
&RPSXWHUDLGHGPL[HGPHWKRGVGDWDDQDO\VLV
3DUW5HSRUWLQJUHVHDUFKUHVXOWV
:ULWLQJDTXDQWLWDWLYHUHSRUW
7ZRIXQFWLRQVRIDFDGHPLFZULWLQJ
6W\OHPDQXDOV
7KHVWUXFWXUHRIWKHTXDQWLWDWLYHUHVHDUFKUHSRUW
5HDGHUIULHQGO\GDWDSUHVHQWDWLRQPHWKRGV
:ULWLQJTXDOLWDWLYHDQGPL[HGPHWKRGVUHSRUWV
:ULWLQJDTXDOLWDWLYHUHSRUW
:ULWLQJDPL[HGPHWKRGVUHSRUW
3DUW6XPPLQJXS
+RZWRFKRRVHWKHDSSURSULDWHUHVHDUFKPHWKRG
*HQHUDOUHFRPPHQGDWLRQ,$GRSWDSUDJPDWLFDSSURDFK
5HVHDUFKFRQWHQWFRQVLGHUDWLRQV
$XGLHQFHFRQVLGHUDWLRQV
3UDFWLFDOFRQVLGHUDWLRQV
3HUVRQDOFRQVLGHUDWLRQV
*HQHUDOUHFRPPHQGDWLRQ,,&RQVLGHUPL[LQJPHWKRGRORJLHV
)LQDOZRUGV
$IWHUZRUG
%LEOLRJUDSK\

3UHIDFH
:KHQ,VWDUWHGP\3K'UHVHDUFKLQWKHPLGV,ZDVXQDZDUHRIWKHIDFWWKDWWKHUH
ZDVDILHUFHµSDUDGLJPZDU¶UDJLQJDURXQGPHEHWZHHQWZRFDPSVRIUHVHDUFK
PHWKRGRORJLVWVWKHFRQVWUXFWLYLVWLQWHUSUHWLVWVZKRXVHGTXDOLWDWLYHPHWKRGVDQGWKH
SRVLWLYLVWHPSLULFLVWVZKRXVHGTXDQWLWDWLYHPHWKRGV+DG,NQRZQDERXWWKLVGLYLVLRQ
DWWKHWLPH,ZRXOGSUREDEO\KDYHEHHQLQFOLQHGLGHRORJLFDOO\WRMRLQWKH
FRQVWUXFWLYLVWLQWHUSUHWLVWFDPS+RZHYHUDVLWKDSSHQHGWKHPDLQUHVHDUFKWRRO
HPSOR\HGLQWKHDUHDRIP\LQWHUHVW²WKHVRFLDOSV\FKRORJ\RIVHFRQGODQJXDJH
DFTXLVLWLRQDQGDWWLWXGHVPRWLYDWLRQLQSDUWLFXODU²ZDVWKHµDWWLWXGHTXHVWLRQQDLUH¶%\
XVLQJWKLVLQVWUXPHQWIRUP\3K'UHVHDUFK,QRZUHDOL]HWKDW,ZDVLUUHVLVWLEO\
SURSHOOHGLQWRWKHPLGVWRIWKHSRVLWLYLVWFDPS
,ZDVVRLJQRUDQWRIWKHVHLVVXHVWKDWZKHQ,ZDVILUVWWROG²LQWKHV²WKDW,ZDV
DµSRVLWLYLVW¶,QHHGHGWRILQGRXWZKDWµSRVLWLYLVP¶DFWXDOO\PHDQW,OHDUQWWKDWLW
UHIHUUHGWRDVFLHQWLILFSDUDGLJPDQGZRUOGYLHZWKDWDVVXPHVWKHH[LVWHQFHRIDQ
REMHFWLYHDQGLQGHSHQGHQWVRFLDOUHDOLW\µRXWWKHUH¶WKDWFDQEHUHVHDUFKHGHPSLULFDOO\
ZLWKVWDQGDUGL]HGVFLHQWLILFLQVWUXPHQWV+RZHYHU,DOVRQRWLFHGWKDWWKHWHUP
µSRVLWLYLVW¶ZDVDOPRVWH[FOXVLYHO\XVHGE\SHRSOHZKRGLGQRWWKHPVHOYHVDOLJQZLWK
WKLVSDUDGLJPDQGWKDWWKHLUXVDJHRIWKHWHUPZDVGHILQLWHO\QRWSRVLWLYHLWVHHPHGWR
LQGLFDWHWKDWVRPHRQHZDVDELWWKLFNKHDGHGDQGKDGGHILQLWHO\ORVWWRXFK DQG
SUREDEO\DOVRKDGPDFKROHDQLQJV 7KLVZDVQRWJRRGQHZVDQGWRDGGWRP\
VRUURZVQRWRQO\ZDV,DSRVLWLYLVWEXWZKDWPDQ\GLGQRWNQRZ,ZDVDOVRDµSRVLWLYLVW
WUDLWRU¶LQWKHPLGV,VWDUWHGWRFRQGXFWWHQWDWLYHTXDOLWDWLYHUHVHDUFKWR
LQYHVWLJDWHWKHFRQFHSWVRIGHPRWLYDWLRQDQGJURXSG\QDPLFVLQ6/$DQGRYHUWKHSDVW
GHFDGHPRVWRIP\3K'VWXGHQWVKDYHEHHQHQJDJHGLQHLWKHUTXDOLWDWLYHRU²HYHQ
ZRUVH²FRPELQHGTXDOLWDWLYH±TXDQWLWDWLYHUHVHDUFK,ZDVFOHDUO\LQDPHVV
+RZHYHUP\FKHHUOHVVVWRU\VHHPVWRKDYHWDNHQDKDSSLHUWXUQUHFHQWO\,KDYH
OHDUQHGWKDW,QRZTXDOLI\WREHDµSUDJPDWLVW¶DVDUHVHDUFKHUWKDWLVDSURSRQHQWRID
UHVSHFWDEOHSKLORVRSKLFDODSSURDFKDQGWKHUHVHDUFKP\VWXGHQWVDQG,KDYHEHHQ
FRQGXFWLQJLVµPL[HGPHWKRGVUHVHDUFK¶GHHPHGFRPPHQGDEOHDQGKLJKO\VRXJKWDIWHU
LQVRPHFLUFOHV:KDWDIRUWXQDWHWLPHWREHZULWLQJWKLVERRN
,QWKHOLJKWRIWKHDERYH,FDQQRZDGPLWWKDWWKHDSSURDFKWKDWJXLGHGPHLQ
SUHSDULQJWKHPDQXVFULSWRIWKLVYROXPHKDVEHHQSUDJPDWLFLQHYHU\UHVSHFW0\
SUDFWLFDOGLOHPPDKDVEHHQWKLV5HVHDUFKLVDFRPSOH[PXOWLIDFHWHGDFWLYLW\VXFK
WKDWLWLVQRWHDV\WRSURYLGHQRYLFHUHVHDUFKHUVZLWKUHOHYDQWSURFHGXUDONQRZOHGJH

,ZDVSOHDVHGWRILQGRXWODWHUWKDW+DPPHUVOH\DQG$WNLQVRQ VKDUHGP\
SHUFHSWLRQDVWKH\FRQFOXGHGµ7RGD\WKHWHUPµSRVLWLYLVP¶KDVEHFRPHOLWWOHPRUH
WKDQDWHUPRIDEXVHDPRQJVRFLDOVFLHQWLVWVDQGDVDUHVXOWLWVPHDQLQJKDVEHFRPH
REVFXUHG¶

ZLWKRXWRYHUZKHOPLQJDQGWKXVGLVHPSRZHULQJWKHP)XUWKHUPRUHHYHQWKRXJK
DFTXLULQJUHVHDUFKH[SHUWLVHPD\ZHOOEHDOLIHORQJSURFHVVZHVLPSO\FDQQRWKROG
EDFNZLWKRXUILUVWLQYHVWLJDWLRQXQWLOZHKDYHOHDUQWHYHU\UHOHYDQWGHWDLO,QVWHDGZKDW
VHHPVWRPHWKHPRVWLPSRUWDQWWKLQJDERXWGRLQJUHVHDUFKLVWKDWZHJHWGRZQWRLW
µJHWRXUIHHWZHW¶DQGDVDUHVXOWJHWµKRRNHG¶RQZKDWLVDYHU\H[FLWLQJDFWLYLW\6R
KRZFDQUHVHDUFKPHWKRGRORJ\EHWDXJKWHIIHFWLYHO\DQGZKDWFDQZHGRWRSUHYHQW
\RXQJVFKRODUVIURPODXQFKLQJLQYHVWLJDWLRQVWKDWPD\YLRODWHNH\SULQFLSOHVRI
VFLHQWLILFLQTXLU\"
0\DQVZHUKDVEHHQWRDVVXPHWKDWWKHUHH[LVWVDEDVLFWKUHVKROGRIUHVHDUFK
H[SHUWLVHZKLFKRQFHDFKLHYHGDOORZVXVWRHPEDUNRQWKHSURFHVVRIGRLQJGHFHQW
UHVHDUFKWKDWZHZLOOQRWEHDVKDPHGRIZKHQORRNLQJEDFNDGHFDGHODWHU:HFDQILQG
DQLQWHUHVWLQJDQDORJ\WRWKLVLGHDLQWKHSV\FKRDQDO\WLFWKHRU\RIWKHV:LQQLFRWW
LQWURGXFHGWKHFRQFHSWRIWKHµJRRGHQRXJKPRWKHU¶ZKLFKZDVWKHQH[WHQGHG
WRSDUHQWLQJLQJHQHUDO7KHFRQFHSWRIWKHµJRRGHQRXJKSDUHQW¶VXJJHVWVWKDWLQRUGHU
WRSURGXFHSV\FKRORJLFDOKHDOWKLQWKHFKLOGZHGRQRWQHHGWREHSHUIHFWLQVWHDG
µJRRGHQRXJKSDUHQWLQJ¶RQO\UHTXLUHVWKHSDUHQWWRH[FHHGDFHUWDLQWKUHVKROGRI
TXDOLW\SDUHQWLQJ LQFOXGLQJHPSDWKLFXQGHUVWDQGLQJVRRWKLQJSURWHFWLRQDQGORYH WR
SURPRWHKHDOWK\GHYHORSPHQW %HWWOHKHLP ZLWKRXWQHFHVVDULO\KDYLQJWREHD
µ6XSHUPXP¶RUµ6XSHUGDG¶3HUVRQDOO\,KDYHDOZD\VIRXQGWKLVQRWLRQHQFRXUDJLQJ
DQGWKXVIROORZLQJWKHµJRRGHQRXJK¶DQDORJ\LWKDVEHHQP\DLPLQWKLVERRNWR
VXPPDUL]HWKRVHNH\FRPSRQHQWVRIUHVHDUFKPHWKRGRORJ\WKDWDUHQHFHVVDU\WR
EHFRPHDµJRRGHQRXJKUHVHDUFKHU¶
,VWURQJO\EHOLHYHWKDWWKHµJRRGHQRXJKUHVHDUFKHU¶QHHGVWRPDVWHUVRPH
NQRZOHGJHRIERWKTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKDVZHOODVZD\VRIFRPELQLQJ
WKHP,QWKH+DQGERRNRI0L[HG0HWKRGVLQ6RFLDODQG%HKDYLRXUDO5HVHDUFK
7DVKDNNRULDQG7HGGOLH F SRLQWRXWWKDWSHUKDSVWKHZRUVWUHVLGXHRIWKH
µSDUDGLJPZDU¶LQUHVHDUFKPHWKRGRORJ\KDVEHHQLWVLPSDFWRQVWXGHQWV$VWKH\
DUJXHPDQ\RIRXUVWXGHQWVDUHDVDFRQVHTXHQFHVXIIHULQJIURPDIUDFWXUHGµGXDO
OLQJXDOLVP¶ZKLFKUHSUHVHQWVDVSOLWSHUVRQDOLW\LQPHWKRGVRIVWXG\DQGZD\VRI
WKLQNLQJ$WDQ\JLYHQPRPHQWWKH\DUHDVNHGWREHHLWKHUTXDOLWDWLYHRUTXDQWLWDWLYH
DQGZKLOHLQHDFKPRGHWRIRUJHWWKDWWKHRWKHUH[LVWV7KLVLVFOHDUO\XQGHVLUDEOHDQG
OXFNLO\DSSOLHGOLQJXLVWVKDYHE\DQGODUJHVWHHUHGFOHDURIVXFKH[WUHPHSRVLWLRQV,
KRSHWKDWPRVWUHDGHUVRIWKLVERRNZLOODJUHHZLWKWKHQHHGWRHGXFDWHDQHZ
JHQHUDWLRQRIµJRRGHQRXJKUHVHDUFKHUV¶ZKRDUHVXIILFLHQWO\IDPLOLDUZLWKERWK
TXDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGVWREHDEOHWRXQGHUVWDQGDQGDSSUHFLDWHWKHUHVXOWV
FRPLQJIURPHDFKVFKRRODQGSHUKDSVHYHQWRYDU\WKHPHWKRGVDSSOLHGLQWKHLURZQ
SUDFWLFHDFFRUGLQJWRWKHLUSDUWLFXODUUHVHDUFKWRSLFTXHVWLRQ7KXV,DPLQIXOO
DJUHHPHQWZLWK'XII¶V FRQFOXVLRQWKDWµDJUHDWHUFROOHFWLYHDZDUHQHVVDQG
XQGHUVWDQGLQJ DQGLGHDOO\JHQXLQHDSSUHFLDWLRQ RIGLIIHUHQWUHVHDUFKPHWKRGVDQG
DUHDVRIVWXG\ZRXOGEHKHOSIXOWRWKHILHOGDWODUJH¶$FFRUGLQJO\WKHZULWLQJRIWKLV
ERRNKDVEHHQPRWLYDWHGE\WKHZLVKWRJREH\RQGSDUDGLJPDWLFFRPSDUWPHQWDOL]DWLRQ

E\KLJKOLJKWLQJWKHVWUHQJWKVRIERWKDSSURDFKHVDQGE\LQWURGXFLQJZD\VE\ZKLFK
WKHVHVWUHQJWKVFDQEHFRPELQHGLQDFRPSOHPHQWDU\PDQQHU
$FNQRZOHGJHPHQWV
:ULWLQJWKLVERRNKDVEHHQDQHQMR\DEOHMRXUQH\DQG,KDYHEHHQHQFRXUDJHGDQG
KHOSHGE\DQXPEHURISHRSOHDORQJWKHZD\)LUVWDQGIRUHPRVW,ZRXOGOLNHWR
H[SUHVVP\WKDQNVWR&ULVWLQD:KLWHFURVVIURP2[IRUG8QLYHUVLW\3UHVVZKRKDVEHHQ
PXFKPRUHWKDQPHUHO\DJRRGHGLWRU+HURQJRLQJVXSSRUWIRUWKHSURMHFWULJKWIURP
WKHEHJLQQLQJKDVEHHQLQYDOXDEOH6SHFLDOWKDQNVDUHGXHWR6LPRQ0XULVRQ%RZLH
ZKRKDVQRWRQO\GRQHDFDUHIXOMREFRS\HGLWLQJWKHWH[WEXWKDVDOVRSURYLGHGXVHIXO
VXJJHVWLRQVZLWKUHJDUGWRWKHFRQWHQWDQGWKHFODULW\RISUHVHQWDWLRQ3DWV\'XIIDQG
-HDQ0DUF'HZDHOHJDYHWKHLUEDFNLQJWRWKHERRNZKHQLWZDVVWLOOOLWWOHPRUHWKDQD
SURSRVDO²WKDQN\RXRQFHDJDLQDQG,GRKRSHWKDWLQWKHOLJKWRIWKHILQDOYHUVLRQ\RX
ZRQ¶WUHJUHW\RXUGHFLVLRQ
,KDYHEHHQIRUWXQDWHWRKDYHVHYHUDOIULHQGVDQGFROOHDJXHVZKRZHUHZLOOLQJWR
UHDGWKHZKROHRUYDULRXVSDUWVRIWKHPDQXVFULSWDQGJLYHPHGHWDLOHGVXJJHVWLRQVRQ
KRZWRLPSURYHWKHERRN0\ZDUPWKDQNVDUHGXHWR.DWK\&RQNOLQ-XGLW.RUPRV
0DJJLH.XEDQ\LRYD3DWV\/LJKWERZQDQG+HQU\:LGGRZVRQIRUWKHLUGHWDLOHGDQG
FRQVWUXFWLYHFRPPHQWV ZKLFKRIFRXUVHFDXVHGPHDORWRIH[WUDZRUN« ,KDYH
GRQHP\EHVWWRWDNHDVPDQ\RIWKHFRPPHQWVRQERDUGDVSRVVLEOHDQG,GREHOLHYH
WKDWWKHPDQXVFULSWKDVLPSURYHGFRQVLGHUDEO\DVDUHVXOW,QWKHHQGKRZHYHUWKLVLV
P\RZQSHUVRQDOWDNHRQUHVHDUFKPHWKRGRORJ\ZLWK²QRGRXEW²VHYHUDORPLVVLRQV
DQGIODZV/RRNLQJEDFNWKLVKDVEHHQDPXFKELJJHUSURMHFWWKDQ,RULJLQDOO\
DQWLFLSDWHGEXW,GRKRSHWKDW,PDQDJHGWRFRQYH\VRPHRIWKHIDVFLQDWLRQWKDW,IHOW
ZKLOHH[SORULQJZKDWPDNHVUHVHDUFKZRUN

3DUW.H\LVVXHVLQ
UHVHDUFKPHWKRGRORJ\
,QWURGXFWLRQ:KDWLVUHVHDUFKDQG
KRZGRHVWKLVERRNDSSURDFKLW"
:KHQVWXGHQWVILUVWHQFRXQWHUUHVHDUFKPHWKRGRORJ\LQWKHVRFLDOVFLHQFHVWKH\DUH
OLNHO\WRKHDUWHUPVVXFKDVµVWDWLVWLFV¶µVDPSOLQJ¶µUHOLDELOLW\¶DQGµYDOLGLW\¶ZKLFK
PDNHWKLVDUHDORRNYHU\FRPSOH[DQGWHFKQLFDODQGDOVRTXLWHLQWLPLGDWLQJ,QGHHGIRU
DQRUPDOSHUVRQWKHLPDJHRIDVRFLDOVFLHQWLVWVXEPLWWLQJWKHUHVXOWVRI/LNHUWVFDOH
LWHPVREWDLQHGIURPDVWUDWLILHGUDQGRPVDPSOHWRVWUXFWXUDOHTXDWLRQPRGHOOLQJLV
KDUGO\PRUHUHDOWKDQWKDWRIWKHVRPEUHORRNLQJQDWXUDOVFLHQWLVWLQDZKLWHFRDWGRLQJ
DOOVRUWVRIVWUDQJHWKLQJVLQDODERUDWRU\6XFKWHFKQLFDOWHUPVDQGLPDJHVDOOVXJJHVW
WKDWUHVHDUFKLVVRPHWKLQJWKDWRQO\H[SHUWVGRDQGRUGLQDU\KXPDQVVKRXOGVWHHUFOHDU
RI,QWKLVERRN,ZRXOGOLNHWRSUHVHQWDPRUHXVHUIULHQGO\IDFHWRIUHVHDUFKWU\LQJWR
VKRZWKDWZLWKDELWRIFDUHDQGORWVRIFRPPRQVHQVHDOORIXVFDQFRQGXFW
LQYHVWLJDWLRQVWKDWZLOO\LHOGYDOXDEOHUHVXOWV,GRKRSHWKDWE\WKHHQGRIWKLVERRN
PRVWUHDGHUVZLOODJUHHWKDWµ<RXFDQGRLWLI\RXUHDOO\ZDQWWR¶
6RZKDWLVUHVHDUFK",QWKHPRVWSURIRXQGVHQVHµUHVHDUFK¶VLPSO\PHDQVWU\LQJWR
ILQGDQVZHUVWRTXHVWLRQVDQDFWLYLW\HYHU\RQHRIXVGRHVDOOWKHWLPHWROHDUQPRUH
DERXWWKHZRUOGDURXQGXV/HWXVWDNHDYHU\VSHFLILFH[DPSOHVD\ZDQWLQJWRILQGD
PXVLF&'DWWKHORZHVWSRVVLEOHSULFH7RGRVRZHFDQJRWRVHYHUDOVKRSVWRFRPSDUH
SULFHVDQGQRZDGD\VZHFDQDOVRVHDUFKWKH,QWHUQHWIRUWKHEHVWSULFH7KURXJKRXWDOO
WKLVZKDWZHDUHGRLQJLVµUHVHDUFK¶1RWQHFHVVDULO\JRRGUHVHDUFK,DGPLWEHFDXVH
RXULQYHVWLJDWLRQZLOOSUREDEO\QRWEHFRPSUHKHQVLYH(YHQLIZHILQGZKDWVHHPVWR
EHDUHDOO\JRRGSULFHZHZLOOQRWNQRZZKHWKHUDVKRSRUDZHEVLWHWKDWZHKDYHQRW
YLVLWHGPLJKWKDYHRIIHUHGWKHVDPH&'IRUHYHQFKHDSHU7RGRJRRGUHVHDUFKZHQHHG
WREHV\VWHPDWLFVRWKDWE\WKHHQGZHFDQVWDQGE\RXUUHVXOWVZLWKFRQILGHQFH7KDWLV
UHVHDUFKLQWKHVFLHQWLILFVHQVHLVµWKHRUJDQL]HGV\VWHPDWLFVHDUFKIRUDQVZHUVWRWKH
TXHVWLRQVZHDVN¶ +DWFKDQG/D]DUDWRQ 2ULQVKRUWUHVHDUFKLVGLVFLSOLQHG
LQTXLU\
)ROORZLQJ%URZQ ZHFDQLGHQWLI\WZREDVLFZD\VRIILQGLQJDQVZHUVWR

TXHVWLRQV

%\ORRNLQJDWZKDWRWKHUSHRSOHKDYHVDLGDERXWDSDUWLFXODULVVXHWKLVLVXVXDOO\
FDOOHGµVHFRQGDU\¶µFRQFHSWXDO¶RUµOLEUDU\¶UHVHDUFKDQGLWLVDQHVVHQWLDOIRUPRI
LQTXLU\EHFDXVHLWZRXOGEHDZDVWHRIWLPHDQGHQHUJ\WRLJQRUHRWKHUSHRSOH¶V
ZRUNDQGµUHLQYHQWWKHZKHHO¶DJDLQDQGDJDLQ
%\FRQGXFWLQJRQH¶VRZQGDWDEDVHG RULQUHVHDUFKWHUPVµHPSLULFDO¶
LQYHVWLJDWLRQZKLFKLQYROYHVFROOHFWLQJVRPHVRUWRILQIRUPDWLRQ RUµGDWD¶ DQG
WKHQGUDZLQJVRPHFRQFOXVLRQIURPLW7KLVLVFDOOHGµSULPDU\ UHVHDUFK
7KLVERRNLVFRQFHUQHGZLWKWKHVHFRQGW\SHRIUHVHDUFKWKHFROOHFWLRQRIRQH¶VRZQ
GDWDSDUWLFXODUO\DVLWLVDSSOLHGWRWKHVWXG\RIODQJXDJHV%XWZK\FROOHFWRXURZQ
GDWD",VQ¶WLWQDwYHRUHYHQDUURJDQWWRWKLQNWKDWZHFDQFRQWULEXWHVRPHWKLQJQHZWR
WKHXQGHUVWDQGLQJRIKRZODQJXDJHVDUHDFTXLUHGRUXVHG"$IWHUDOOSHRSOHZKRDUHIDU
PRUHNQRZOHGJHDEOHWKDQXVPXVWKDYHVXUHO\DGGUHVVHGDOOWKHZRUWKZKLOHTXHVWLRQV
WREHDGGUHVVHGDQGPXVWKDYHGUDZQDOOWKHFRQFOXVLRQVWKDWFRXOGEHGUDZQ,ZRXOG
VWURQJO\DUJXHWKDWWKLVLVQRWWKHFDVHIRUDWOHDVWWZRUHDVRQV)LUVWQRWZRODQJXDJH
OHDUQLQJXVHVLWXDWLRQVDUHH[DFWO\WKHVDPHDQGWKHUHIRUHWKHJXLGHOLQHVWKDWZHFDQ
ILQGLQWKHOLWHUDWXUHUDUHO\SURYLGHWKHH[DFWDQVZHUVWRRQH¶VVSHFLILFTXHVWLRQV
6HFRQGWKHUHLVSURJUHVVLQUHVHDUFKWKDWLVQHZDGYDQFHVDUHFRQWLQXRXVO\PDGHE\
EXLOGLQJRQWKHUHVXOWVRISDVWUHVHDUFK
7KHUHLVDOVRDWKLUGUHDVRQZK\,FRQVLGHUGRLQJVRPHSULPDU\UHVHDUFKLPSRUWDQW
IRUHYHU\ODQJXDJHSURIHVVLRQDO6LPSO\SXWUHVHDUFKFDQEHDJUHDWGHDORIIXQDQGDQ
LQWHOOHFWXDOO\KLJKO\VDWLVI\LQJDFWLYLW\0DQ\SHRSOHHQMR\GRLQJFURVVZRUGVDQG
SX]]OHVDQGLWLVKHOSIXOWRORRNDWUHVHDUFKDVVRPHWKLQJVLPLODUEXWRQDODUJHUVFDOH
)XUWKHUPRUHQRWRQO\LVWKHSURFHVVRIDQVZHULQJTXHVWLRQVVROYLQJSX]]OHVRU
GLVFRYHULQJFDXVHVRIWHQH[FLWLQJDQGLOOXPLQDWLQJEXWLWFDQDOVREHRQHRIWKHPRVW
HIIHFWLYHIRUPVRISURIHVVLRQDOGHYHORSPHQWRQHWKDWFDQEHQHILWRWKHUSHRSOHDVZHOO
7KLVSRLQWKDVEHHQH[SUHVVHGYHU\FOHDUO\LQDUHFHQWERRNRQUHVHDUFKIRUODQJXDJH
WHDFKHUVE\0F.D\
)RUWHDFKHUVDSULPDU\UHDVRQIRUGRLQJUHVHDUFKLVWREHFRPHPRUHHIIHFWLYH
WHDFKHUV5HVHDUFKFRQWULEXWHVWRPRUHHIIHFWLYHWHDFKLQJQRWE\RIIHULQJ
GHILQLWLYHDQVZHUVWRSHGDJRJLFDOTXHVWLRQVEXWUDWKHUE\SURYLGLQJQHZLQVLJKWV
LQWRWKHWHDFKLQJDQGOHDUQLQJSURFHVV
7KHJRRGUHVHDUFKHU
:KDWNLQGRIFKDUDFWHULVWLFVGRZHQHHGWREHFRPHJRRGUHVHDUFKHUV"/HWPHVWDUWE\
VWDWLQJWKDWLWLVP\EHOLHIWKDWEHFRPLQJDJRRGUHVHDUFKHUGRHVQRWQHFHVVDULO\UHTXLUH
GHFDGHVRIJUXHOOLQJDSSUHQWLFHVKLSLQODERUDWRULHVPXVHXPDUFKLYHVRUWKHOLNH7KDW
LVUHVHDUFKLVQRWVRPHWKLQJWKDWFDQRQO\EHGRQHZHOOLIZHDUHUHDG\WRVDFULILFHD
JUHDWGHDORIRXUZRUNLQJOLYHVDQGWKXVµSD\WKHSULFH¶ 0DQ\DSSOLHGOLQJXLVWV
DFWXDOO\HQMR\OLIH :KLOHH[SHULHQFHDQGDFDGHPLFH[SHUWLVHGRXQGRXEWHGO\KHOS
WKH\DUHQRWWKHPDLQSUHUHTXLVLWHVWREHLQJDVXFFHVVIXOUHVHDUFKHU)RUH[DPSOHLWLV

QRWXQKHDUGRIWKDWXQLYHUVLW\VHPLQDUSDSHUVDUHSXEOLVKHGLQLQWHUQDWLRQDOMRXUQDOV
DQGPRUHWKDQRQHLPSRUWDQWVWXG\LQDSSOLHGOLQJXLVWLFVKDYHEHHQEDVHGRQDQ
DXWKRU¶V0$GLVVHUWDWLRQ7KXVLQRUGHUWREHDEOHWRSURGXFHJRRGUHVHDUFKUHVXOWVRQH
GRHVQRWQHFHVVDULO\KDYHWREHROGRURYHUO\VHDVRQHG,QP\YLHZWKHUHDUHIRXU
IXQGDPHQWDOIHDWXUHVRIDUHVHDUFKHUWKDWZLOOKHOSKLPKHUWRDFKLHYHH[FHOOHQFH
JHQXLQHFXULRVLW\DORWRIFRPPRQVHQVHJRRGLGHDVDQGVRPHWKLQJWKDWFDQEHEHVW
GHVFULEHGDVDFRPELQDWLRQRIGLVFLSOLQHUHOLDELOLW\DQGVRFLDOUHVSRQVLELOLW\/HWXV
WDNHWKHVHRQHE\RQH
2YHUWKH\HDUV,KDYHFRPHWREHOLHYHWKDWWKHSULPDU\FKDUDFWHULVWLFRIJRRG
UHVHDUFKHUVLVWKDWWKH\KDYHDJHQXLQHDQGVWURQJFXULRVLW\DERXWWKHLUWRSLF*RRG
UHVHDUFKHUVDUHDOZD\VDIWHUVRPHWKLQJWKDWWKH\ILQGLQWULJXLQJRUSX]]OLQJRUDERXW
ZKLFKWKH\KDYHDKXQFK RULQUHVHDUFKWHUPVDµK\SRWKHVLV¶ 6HULRXVUHVHDUFKZLOO
LQHYLWDEO\UHTXLUHVRPHKDUGZRUNDQGWKHRQO\ZD\WRPDLQWDLQRXUPRPHQWXPDQG
FUHDWLYLW\WKURXJKRXWWKHODERULRXVSDUWVLVWREHGULYHQE\RXUSDVVLRQIRUWKHWRSLF
7KLVLVZK\LWLVDGYLVDEOHIRUQRYLFHUHVHDUFKHUVWRFKRRVHDUHVHDUFKWRSLFWKDWWKH\
DUHJHQXLQHO\LQWHUHVWHGLQUDWKHUWKDQRQHWKDWVHHPVVHQVLEOHIURPDFDUHHUSRLQWRI
YLHZ
7KHVHFRQGLPSRUWDQWIHDWXUHRIDJRRGUHVHDUFKHULVFRPPRQVHQVH:KHQZH
EHFRPHLQWLPDWHO\LQYROYHGLQSXUVXLQJDQDFWLYLW\LWLVDOOWRRHDV\WRORVHWRXFKZLWK
UHDOLW\DQGDEDQGRQRQH¶VFODULW\RISXUSRVHWREHFRPHELDVHGDQGJUDGXDOO\JRDVWUD\
/RWVRIHODERUDWHFRVWO\DQGKLJKO\µVFLHQWLILF¶LQYHVWLJDWLRQVKDYHEHHQFRQGXFWHGRQ
WULYLDODQGIUDQNO\ERULQJWRSLFV$SSOLHGOLQJXLVWVDUHE\GHILQLWLRQHQJDJHGZLWKWKH
UHDOZRUOGDQG,KDYHIRXQGWKDWWKHEHVWUHVHDUFKHUVLQWKHILHOGWHQGWREHYHU\QRUPDO
SHRSOH7KH\KDYHDKLJKOHYHORIFRPPRQVHQVHWKDWKHOSVWRNHHSWKHLUIHHWILUPO\RQ
WKHJURXQG
7KHWKLUGDVSHFWRIVXFFHVVIXOUHVHDUFKWKDW,ZRXOGOLNHWRXQGHUOLQHLVKDYLQJJRRG
LGHDV,WVHHPVWRPHWKDWQRDPRXQWRIVRSKLVWLFDWHGUHVHDUFKGHVLJQRUFRPSOH[
DQDO\WLFDOWHFKQLTXHFDQEHDVXEVWLWXWHIRUFUHDWLYHWKLQNLQJWKDWLVJURXQGHGLQUHDOLW\
0DQ\RIWKHEHVWNQRZQVWXGLHVLQDSSOLHGOLQJXLVWLFVDUHQRWDWDOOFRPSOLFDWHGLQWHUPV
RIWKHLUUHVHDUFKPHWKRGRORJ\EXWDUHEDVHGRQIDLUO\VLPSOHEXWRULJLQDOLQVLJKWV
PDNLQJXVVLJKµ:K\GLGQ¶W,WKLQNRIWKLV«¶
)LQDOO\,VWURQJO\EHOLHYHWKDWDJRRGUHVHDUFKHUQHHGVWREHGLVFLSOLQHGDQG
UHVSRQVLEOH7KLVLVUHODWHGWRWKHV\VWHPDWLFQDWXUHRIUHVHDUFK'XULQJWKHFRXUVHRIDQ
LQYHVWLJDWLRQWKHUHDUHUHFXUULQJWHPSWDWLRQVWRFXWFRUQHUVWROHDYHRXWVRPHQHFHVVDU\
VWHSVDQGWRGUDZFRQFOXVLRQVWKDWDUHQRWIXOO\VXSSRUWHGE\WKHGDWD,WLVWKH
UHVHDUFKHU¶VGLVFLSOLQHWKDWNHHSVKLPKHURQWKHULJKWWUDFNDQGWKHODFNRIGLVFLSOLQH
RUFRQVLVWHQF\LVRQHRIWKHPRVWIUHTXHQWVRXUFHVRILQDGHTXDWHUHVHDUFK$QLPSRUWDQW
UHODWHGFKDUDFWHULVWLFKDVEHHQSRLQWHGRXWWRPHE\3DWV\'XII SHUVRQDO
FRPPXQLFDWLRQ $JRRGUHVHDUFKHUDOVRKDVDVHQVHRIVRFLDOUHVSRQVLELOLW\WKDWLV
DFFRXQWDELOLW\WRWKHILHOGDQGPRUHEURDGO\WRWKHZRUOG5HVHDUFKLVQRWGRQHIRULWV
RZQVDNHEXWWRJHQHUDWHNQRZOHGJHDQGWRIXUWKHURXUXQGHUVWDQGLQJ7KLVPHDQVWKDW

WKHJRRGUHVHDUFKHUQHHGVWROHDUQWRFRPPXQLFDWHKLVKHUILQGLQJVZLWKRWKHUVZKLFK
LVZK\3DUWRIWKLVERRNFRQFHUQVZULWLQJXSDQGUHSRUWLQJRXUUHVXOWV
0\DSSURDFKWRUHVHDUFKPHWKRGRORJ\
0\DSSURDFKWRUHVHDUFKPHWKRGRORJ\LVWZRIROG2QWKHRQHKDQG,FDQQRWUHODWHZHOO
WRUHVHDUFKWH[WVWKDWDUHWRRKHDY\RQGLVFXVVLQJWKHSKLORVRSKLFDOXQGHUSLQQLQJVRI
UHVHDUFKPHWKRGRORJ\$OWKRXJKEHLQJP\VHOIDUHVHDUFKHULQWRPRWLYDWLRQ,GRDFFHSW
WKDWRXUEHKDYLRXUV DQGSDUWLFXODUO\VXVWDLQHGDQGFRQVLVWHQWEHKDYLRXUVVXFKDV
UHVHDUFKHIIRUWV DUHJRYHUQHGE\DEVWUDFWSULQFLSOHV,JHWHDVLO\GLVRULHQWDWHGLQWKH
PLGVWRIGLVFXVVLQJUHVHDUFKDWVXFKDQDEVWUDFWOHYHODQGRIWHQILQGP\VHOIWKLQNLQJ
µ&DQ¶WZHMXVWJHWRQZLWKLW«"¶,ZDVUHOLHYHGWRILQGWKDW,DPQRWDORQHZLWKWKLV
IHHOLQJIRUH[DPSOHWDONLQJDERXWDOWHUQDWLYHPHWKRGVLQTXDOLWDWLYHUHVHDUFK0LOHV
DQG+XEHUPDQ VWDWHµ$WWLPHVLWVHHPVDVLIWKHFRPSHWLQJRIWHQSROHPLFDO
DUJXPHQWVRIGLIIHUHQWVFKRROVRIWKRXJKWDERXWKRZTXDOLWDWLYHUHVHDUFKVKRXOGEH
GRQHSURSHUO\XVHPRUHHQHUJ\WKDQWKHDFWXDOUHVHDUFKGRHV¶,DFFHSWKRZHYHUWKDW
WKLVTXHVWLRQLVQRWVRVLPSOHDQGWKHUHIRUHZHZLOOORRNDWWKHYDULRXVVWDQFHVLQPRUH
GHWDLOLQ&KDSWHUVDQG 6HHDOVRWKH3UHIDFHIRUDSHUVRQDODFFRXQW
7KHRWKHUVLGHRIWKHFRLQLVKRZHYHUWKDW,UHDOO\GLVOLNHEDGUHVHDUFK7KHUHIRUH,
ILUPO\EHOLHYHLQWKHLPSRUWDQFHRIEHFRPLQJIDPLOLDUZLWKWKHµUHVHDUFKORUH¶²WKDWLV
WKHSULQFLSOHVRIEHVWSUDFWLFH²WKDWSDVWUHVHDUFKHUVKDYHDFFXPXODWHGRYHUWKH
GHFDGHV7KHUHDUHVHYHUDOWHFKQLFDODVSHFWVRIUHVHDUFKWKDWDUHFHUWDLQO\ZRUWK
OHDUQLQJDERXWEHFDXVHWKHSLWIDOOVWKDWZHFDQIDOOLQWRLIZHIROORZDSXUHO\WULDODQG
HUURUPHWKRGFDQEHGHHSDQGFRVWO\LQWHUPVRIWLPHDQGHQHUJ\$OVRWKHUHDUHFHUWDLQ
NH\SULQFLSOHVRIUHVHDUFKPHWKRGRORJ\WKDWZHQHHGWREHDULQPLQGRUHOVHZHULVNRXU
UHVXOWVEHFRPLQJLUUHSDUDEO\IODZHG7KXV,EHOLHYHLQWKHXVHIXOQHVVRID
VWUDLJKWIRUZDUGµWRWKHSRLQW¶UHVHDUFKPHWKRGRORJ\WKDWRIIHUVDJUHDWGHDORISUDFWLFDO
KHOSDQGZKLFKDOVRVHWVFOHDUERXQGDULHVIRUUHVHDUFKHUV
7KHFRQWHQWRIWKLVERRN
5HVHDUFKPHWKRGRORJ\FRYHUVDYHU\EURDGUDQJHDQGWKHUHIRUHHYHU\RYHUYLHZLV
LQHYLWDEO\VHOHFWLYH:LWKRXWZDQWLQJWREHWRRWHFKQLFDODWWKLVVWDJH,IHHOWKDWLWLV
LPSRUWDQWWRGHVFULEHKHUHVRPHNH\DVSHFWVRIP\DSSURDFKLQVHOHFWLQJDQG
RUJDQL]LQJWKHFRQWHQW
4XDOLWDWLYHDQGTXDQWLWDWLYHGDWD
$VVWDWHGEHIRUHWKLVERRNIRFXVHVRQµSULPDU\UHVHDUFK¶ZKLFKLQYROYHVFROOHFWLQJ
RULJLQDOGDWDDQDO\VLQJLWDQGWKHQGUDZLQJRQH¶VRZQFRQFOXVLRQIURPWKHDQDO\VLV
$FFRUGLQJO\WKHNH\LVVXHLQWKLVERRNLVGDWD%URDGO\VSHDNLQJLQDSSOLHGOLQJXLVWLF
UHVHDUFKZHFDQILQGWKUHHPDLQW\SHVRISULPDU\GDWD

x 4XDQWLWDWLYHGDWDZKLFKLVPRVWFRPPRQO\H[SUHVVHGLQQXPEHUV IRUH[DPSOH
WKHVFRUHRIDODQJXDJHDSWLWXGHWHVWRUWKHQXPEHURIWLPHVDVWXGHQWYROXQWHHUVLQ
FODVV
x 4XDOLWDWLYHGDWDZKLFKXVXDOO\LQYROYHVUHFRUGHGVSRNHQGDWD IRUH[DPSOH
LQWHUYLHZGDWD WKDWLVWUDQVFULEHGWRWH[WXDOIRUPDVZHOODVZULWWHQ ILHOG QRWHV
DQGGRFXPHQWVRIYDULRXVVRUWV
x /DQJXDJHGDWDZKLFKLQYROYHVODQJXDJHVDPSOHVRIYDULRXVOHQJWKHOLFLWHGIURP
WKHUHVSRQGHQWSULPDULO\IRUWKHSXUSRVHRIODQJXDJHDQDO\VLV IRUH[DPSOHD
UHFRUGHGODQJXDJHWDVNRUDVROLFLWHGVWXGHQWHVVD\WKDWLVWRVXEPLWWHGWRGLVFRXUVH
DQDO\VLV :HVKRXOGQRWHKHUHWKDWWKLVW\SHRIODQJXDJHGDWDLVRIWHQVXEVXPHG
XQGHUTXDOLWDWLYHGDWDLQWKHOLWHUDWXUHEXW,IHHOWKDWLWLVXVHIXOWRVHSDUDWHWKHWZR
GDWDW\SHV
7KLVERRNZLOOIRFXVSULPDULO\RQWKHILUVWWZRW\SHVRIGDWDDQGZLOOQRWEH
FRQFHUQHGZLWKWKHWKLUGW\SHODQJXDJHGDWDXQOHVVLWLVVXEPLWWHGWRQRQOLQJXLVWLF
DQDO\VLV7KHFROOHFWLRQDQGDQDO\VLVRIODQJXDJHGDWDLVDKLJKO\VSHFLDOL]HGDSSOLHG
OLQJXLVWLFWDVNDQGVHYHUDOVXEGLVFLSOLQHV²IRUH[DPSOHODQJXDJHWHVWLQJDVVHVVPHQW
GLVFRXUVHDQDO\VLVFRQYHUVDWLRQDQDO\VLVDQGFRUSXVOLQJXLVWLFV²KDYHEHHQGHYHORSHG
WRKHOSWRFRQGXFWWKHSURFHVVLQJRIVXFKGDWDZLWKDJUHDWQXPEHURIJRRGRYHUYLHZV
SXEOLVKHGLQWKHILHOG IRUH[DPSOH$GROSKV$OGHUVRQ%DFKPDQDQG
3DOPHU%URZQ&DUWHUHWDO&KDXGURQ(OOLVDQG%DUNKXL]HQ
)XOFKHUDQG'DYLGVRQ0DUNHH1RUULVDQG2UWHJD6FKLIIULQHW
DO:LGGRZVRQ $OWKRXJK,ZLOOIURPWLPHWRWLPHUHIHUWRZRUNLQWKLV
DUHDWKHPDLQLQWHQWLRQRIWKLVERRNLVWRVXPPDUL]HWKHJHQHULFDVSHFWVRIZRUNLQJ
ZLWKTXDOLWDWLYHDQGTXDQWLWDWLYHGDWDZKLFKFDQWKHQEHWUDQVIHUUHGWREHXVHGLQVRPH
DVSHFWVRIODQJXDJHDQDO\VLV 7KLVLVDGPLWWHGO\DURXJKFODVVLILFDWLRQEHFDXVHWKHUH
DUHVHYHUDOµFURVVRYHUV¶EHWZHHQGDWDW\SHV²IRUH[DPSOHERWKTXDOLWDWLYHDQG
ODQJXDJHGDWDFDQEHWXUQHGTXDQWLWDWLYHUHODWLYHO\HDVLO\²EXWWKLVEURDGV\VWHPLV
UHPDUNDEO\UREXVWDQGKHOSIXOLQFODULI\LQJWKHIRXQGDWLRQVRIUHVHDUFK
4XDOLWDWLYHTXDQWLWDWLYHDQGPL[HGPHWKRGVUHVHDUFK
'XHWRWKHGLIIHUHQWQDWXUHRITXDOLWDWLYHDQGTXDQWLWDWLYHGDWDGLIIHUHQWPHWKRGVKDYH
EHHQGHYHORSHGLQWKHSDVWWRFROOHFWDQGDQDO\VHWKHPOHDGLQJWRWKHGLIIHUHQWLDWLRQ
EHWZHHQWZRUHVHDUFKSDUDGLJPVµTXDOLWDWLYHUHVHDUFK¶DQGµTXDQWLWDWLYHUHVHDUFK¶:H
VKRXOGQRWHKHUHWKDWWKHWHUPVTXDOLWDWLYHDQGTXDQWLWDWLYHDUHµRYHUVWDWHGELQDULHV¶
'XII DQGDORQJZLWKRWKHUV%URZQ DUJXHVWKDWDPRUHFRQVWUXFWLYH
DSSURDFKLVWRYLHZTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKDVDPDWWHURIGHJUHHVRUD
FRQWLQXXPUDWKHUWKDQDFOHDUFXWGLFKRWRP\+RZHYHUWKHVHFDWHJRULHVDUHXVHIXOWR
JHWXVVWDUWHGDQGZHZLOOHODERUDWHRQWKHPLQWKHQH[WFKDSWHU
$OWKRXJKWKHTXDOLWDWLYH±TXDQWLWDWLYHGLVWLQFWLRQGRHVVHSDUDWHWZRUHVHDUFK
DSSURDFKHV,GRQRWVHHTXDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGRORJLHVDVQHFHVVDULO\
PXWXDOO\H[FOXVLYHDQGWKHUHIRUHDVSHFLDOREMHFWLYHRIWKLVERRNLVWRLQWURGXFHUHDGHUV

WRDWKLUGUHVHDUFKDSSURDFKµPL[HGPHWKRGVUHVHDUFK¶7KLVLVDQHZDQGYLJRURXVO\
JURZLQJEUDQFKRIUHVHDUFKPHWKRGRORJ\LQYROYLQJWKHFRPELQHGXVHRITXDOLWDWLYH
DQGTXDQWLWDWLYHPHWKRGVZLWKWKHKRSHRIRIIHULQJWKHEHVWRIERWKZRUOGV7KXVLQWKLV
ERRN,ZLOOWU\WREUHDNGRZQWKHEDUULHUVEHWZHHQTXDOLWDWLYHDQGTXDQWLWDWLYH
PHWKRGRORJLHVDVPXFKDVSUDFWLFDODQGKLJKOLJKWWKHPDQLIROGZD\VRIFRPELQLQJ
WKHPWREHVWHIIHFW
&RQVHUYDWLYHDQGFDQRQLFDODSSURDFK
5HVHDUFKPHWKRGRORJ\KDVUHDFKHGVHYHUDOLQWHUHVWLQJFURVVURDGVLQWKHSDVWGHFDGH,Q
TXDQWLWDWLYHUHVHDUFKIRUH[DPSOHWKHUHLVDOLYHO\GHEDWHFRQFHUQLQJDSSOLFDWLRQVRI
FHUWDLQVWDWLVWLFVLQFOXGLQJVLJQLILFDQFHWHVWLQJDQGLWVDOWHUQDWLYHVDQGLQTXDOLWDWLYH
UHVHDUFKWKHUHKDYHEHHQVRPHKLJKSURILOHVSOLWVEHWZHHQYDULRXVVFKRROVLQ
HWKQRJUDSKLFUHVHDUFKDQGLQJURXQGHGWKHRU\6XFKGLYHUVLW\LVQRWXQLTXHWRWKHVH
VSHFLILFLVVXHVRUDUHDVEHFDXVHUHVHDUFKPHWKRGRORJ\FRQFHUQVWKHSULQFLSDOSURFHVV
RIJHWWLQJWRNQRZDQGFRPLQJWRWHUPVZLWKWKHZRUOGDURXQGXVGLIIHUHQWVFKRODUV
ZLWKGLIIHUHQWZRUOGYLHZVDUHOLNHO\WRDGYRFDWHGLIIHUHQWDSSURDFKHVUHJDUGLQJKRZWR
JRDERXWGRLQJUHVHDUFK$VDFRQVHTXHQFHGXULQJWKHSDVWIRXUGHFDGHVUHVHDUFK
PHWKRGRORJ\KDVEHFRPHDQDUHQDZKHUHGLYHUVHSKLORVRSKLFDODQGSROLWLFDOYLHZV
KDYHFODVKHGLQDVSHFWDFXODUPDQQHURYHUTXHVWLRQVVXFKDVµ:KDWLVWKHUHDOLW\DURXQG
XVOLNH"¶RUµ:KDWLVWKHVRFLDOUHVSRQVLELOLW\RIWKHUHVHDUFKHU"¶RUµ:KDWHYLGHQFHGR
ZHQHHGWRSUHVHQWWREHDEOHWRDUJXHDSRLQWFRQYLQFLQJO\"¶
,GRQRWWKLQNWKDWLWUHTXLUHVPXFKMXVWLILFDWLRQWKDWDEDVLFRYHUYLHZVXFKDVWKDW
RIIHUHGE\WKLVYROXPHFDQQRWUHSUHVHQWDOOWKHGLYHUVHVWDQFHVWKDWKDYHEHHQSXVKLQJ
WKHERXQGDULHVEXWLQVWHDGQHHGVWRIRFXVRQWKHFDQRQLFDODVSHFWVRIWKHDFFXPXODWHG
UHVHDUFKORUH7KHUHIRUHLQP\VXPPDULHV,ZLOOJLYHSULRULW\WRWKHUHVHDUFKSULQFLSOHV
DQGUHFRPPHQGDWLRQVWKDWKDYHVWRRGWKHWHVWRIWLPHZKLFKZLOOLQHYLWDEO\UHVXOWLQD
VRPHZKDWFRQVHUYDWLYHVODQWRIWKHSUHVHQWDWLRQRIWKHPDWHULDO
)XUWKHUUHDGLQJ
$SSOLHGOLQJXLVWVFDQILQGXVHIXOUHVRXUFHVLQUHVHDUFKPHWKRGRORJ\LQVHYHUDOGLIIHUHQW
DUHDVILUVWDQGIRUHPRVWWKHUHLVDKHDOWK\ERG\RIPHWKRGRORJ\OLWHUDWXUHVSHFLILFDOO\
IRFXVLQJRQDSSOLHGOLQJXLVWLFUHVHDUFK+RZHYHUEHFDXVHVRPXFKLQUHVHDUFK
PHWKRGRORJ\LVVXIILFLHQWO\JHQHULFWREHWUDQVIHUDEOHZHFDQDOVRILQGUHOHYDQW
LQIRUPDWLRQLQUHVHDUFKPHWKRGRORJ\WH[WVZULWWHQIRUVRFLDOVFLHQWLVWVLQJHQHUDODQG
IRUSV\FKRORJLFDODQGHGXFDWLRQDOUHVHDUFKHUVLQSDUWLFXODU7KXVUHDGHUVLQWHUHVWHGLQ
PRUHDGYDQFHGDVSHFWVRIUHVHDUFKLVVXHVDUHDGYLVHGWRFRQVXOWZRUNVIURPWKHVH
µIHHGHU¶GLVFLSOLQHVZKHUHWKHDFFXPXODWHGUHVHDUFKH[SHULHQFHFRQVLGHUDEO\H[FHHGV
WKDWLQDSSOLHGOLQJXLVWLFV4XDOLWDWLYHUHVHDUFKKDVWUDGLWLRQDOO\EHHQVWURQJZLWKLQ
VRFLRORJ\DQGDQWKURSRORJ\²UHVHDUFKWH[WVRULJLQDWLQJIURPWKHVHWUDGLWLRQVDUHRIWHQ
DLPHGDWWKHVRFLDOVFLHQFHVLQJHQHUDO4XDQWLWDWLYHUHVHDUFKDQGVWDWLVWLFVKDYH
WUDGLWLRQDOO\EHHQGULYHQE\SV\FKRORJLVWVDQGWKHILHOGRISV\FKRORJ\VXEVXPHVDVXE

GLVFLSOLQHVSHFLDOL]HGLQDVVHVVPHQWLVVXHVµSV\FKRPHWULFV¶(GXFDWLRQDOUHVHDUFKKDV
EHHQFUHDWLYHLQDGDSWLQJDZLGHUDQJHRIUHVHDUFKPHWKRGVXVHGLQWKHVRFLDOVFLHQFHV
WRWKHVSHFLILFDUHDRIVWXGHQWOHDUQLQJDQGWKHUHIRUHLWFDQRIIHUXQLTXHH[SHUWLVHLQ
VSHFLDOL]HGDUHDVVXFKDVHGXFDWLRQDOLQWHUYHQWLRQUHVHDUFKRUFODVVURRPREVHUYDWLRQ
,QWKHIROORZLQJ,ZLOOOLVWDQXPEHURIUHVHDUFKPHWKRGRORJ\WH[WVWKDW,KDYHIRXQG
XVHIXOLQP\RZQUHVHDUFKDQGLQZULWLQJWKLVERRN7KLVLVQRWDFRPSUHKHQVLYHOLVW
DQGFROOHDJXHVZLOOSUREDEO\LPPHGLDWHO\ILQGVRPHUHDOµJHPV¶WKDW,KDYH
UHJUHWWDEO\RPLWWHG
/LQJXLVWLFDQGDSSOLHGOLQJXLVWLFUHVRXUFHV
7KHIROORZLQJWLWOHVDUHJHQHUDOUHVHDUFKPHWKRGRORJ\WH[WVIRUDSSOLHGOLQJXLVWV )RU
UHFHQWUHYLHZVVHH/D]DUDWRQ/LVNLQ*DVSDUUR )RUZRUNVWKDWFRQFHUQ
QDUURZHUUHVHDUFKDUHDVVXFKDVVSHFLILFPHWKRGVSOHDVHVHHWKHUHIHUHQFHVLQWKH
UHOHYDQWFKDSWHUV
%DFKPDQ/)6WDWLVWLFDO$QDO\VHVIRU/DQJXDJH$VVHVVPHQW&DPEULGJH
&DPEULGJH8QLYHUVLW\3UHVV
%URZQ-'DQG765RGJHUV'RLQJ6HFRQG/DQJXDJH5HVHDUFK2[IRUG
+DWFK(0DQG$/D]DUDWRQ7KH5HVHDUFK0DQXDO'HVLJQDQG6WDWLVWLFVIRU
$SSOLHG/LQJXLVWLFV1HZ<RUN1HZEXU\+RXVH
-RKQVRQ'0$SSURDFKHVWR5HVHDUFKLQ6HFRQG/DQJXDJH/HDUQLQJ1HZ
<RUN/RQJPDQ
0DFNH\$DQG60*DVV6HFRQG/DQJXDJH5HVHDUFK0HWKRGRORJ\DQG
'HVLJQ0DKZDK1-/DZUHQFH(UOEDXP
0F'RQRXJK-DQG60F'RQRXJK5HVHDUFK0HWKRGVIRU(QJOLVK/DQJXDJH
7HDFKHUV/RQGRQ$UQROG
0F.D\6/5HVHDUFKLQJ6HFRQG/DQJXDJH&ODVVURRPV0DKZDK1-
/DZUHQFH(UOEDXP
0LOUR\/DQG0*RUGRQ6RFLROLQJXLVWLFV0HWKRGDQG,QWHUSUHWDWLRQ2[IRUG
%ODFNZHOO
1XQDQ'5HVHDUFK0HWKRGVLQ/DQJXDJH/HDUQLQJ&DPEULGJH&DPEULGJH
8QLYHUVLW\3UHVV
5LFKDUGV.4XDOLWDWLYH,QTXLU\LQ7(62/%DVLQJVWRNH(QJODQG3DOJUDYH
0DFPLOODQ

6HOLJHU+:DQG(6KRKDP\6HFRQG/DQJXDJH5HVHDUFK0HWKRGV2[IRUG
:UD\$.7URWWDQG$%ORRPHU3URMHFWVLQ/LQJXLVWLFV$3UDFWLFDO*XLGH
WR5HVHDUFKLQJ/DQJXDJH/RQGRQ$UQROG
$QQRWDWHGOLVWRIVHOHFWHGUHVHDUFKWH[WVIURPWKHVRFLDOVFLHQFHV
&UHVZHOO-:3ODQQLQJ&RQGXFWLQJDQG(YDOXDWLQJ4XDQWLWDWLYHDQG
4XDOLWDWLYH5HVHDUFK8SSHU6DGGOH5LYHU1-3HDUVRQ3UHQWLFH+DOO $EDODQFHG
FRYHUDJHRIHYHU\DVSHFWRIUHVHDUFKLQFOXGLQJPL[HGPHWKRGV
'HQ]LQ1.DQG<6/LQFROQ HGV 7KH6DJH+DQGERRNRI4XDOLWDWLYH

5HVHDUFKUG(GLWLRQ7KRXVDQG2DNV&$6DJH 7KLVLVRQHRIWKHNH\ZRUNVLQ
TXDOLWDWLYHUHVHDUFKDOUHDG\RQLWVWKLUGHGLWLRQ
+ROOLGD\$'RLQJDQG:ULWLQJ4XDOLWDWLYH5HVHDUFK/RQGRQ6DJH $ERRN
ZULWWHQE\DQDSSOLHGOLQJXLVWIRUDJHQHUDODXGLHQFHZLWKDVSHFLDOHPSKDVLVRQ
ZULWLQJTXDOLWDWLYHUHSRUWV
-RKQVRQ%DQG/&KULVWHQVHQ(GXFDWLRQ5HVHDUFK4XDQWLWDWLYH4XDOLWDWLYH
DQG0L[HG$SSURDFKHVQG(GLWLRQ%RVWRQ$OO\QDQG%DFRQ $FRPSUHKHQVLYH
DQGEDODQFHGUHVHDUFKPHWKRGVWH[WZLWKDQHGXFDWLRQDOIRFXV
0RUVH-0DQG/5LFKDUGV5HDGPH)LUVWIRUD8VHU V*XLGHWR4XDOLWDWLYH
5HVHDUFK7KRXVDQG2DNV&$6DJH ,GHDOIRUQRYLFHTXDOLWDWLYHUHVHDUFKHUV
3DOODQW-63666XUYLYDO0DQXDOQG(GLWLRQ0DLGHQKHDG(QJODQG2SHQ
8QLYHUVLW\3UHVVDQG0F*UDZ+LOO(GXFDWLRQ 7KHUHDUHPDQ\JRRG6366PDQXDOV
DURXQG²,KDYHIRXQGWKLVSDUWLFXODUO\FRPSUHKHQVLYHDQGXVHUIULHQGO\
3XQFK.),QWURGXFWLRQWR6RFLDO5HVHDUFKQG(GLWLRQ7KRXVDQG2DNV&$
6DJH 6KRUWWRWKHSRLQWDQGDFFHVVLEOHKLJKO\UHFRPPHQGHG
5LFKDUGV/+DQGOLQJ4XDOLWDWLYH'DWD$3UDFWLFDO*XLGH/RQGRQ6DJH $
UHIUHVKLQJO\GRZQWRHDUWKDQGSUDFWLFDOERRNZULWWHQE\RQHRIWKHGHYHORSHUVRI
WKH18' ,67DQG19LYRTXDOLWDWLYHGDWDDQDO\VLVVRIWZDUH
5REVRQ&5HDO:RUOG5HVHDUFK$5HVRXUFHIRU6RFLDO6FLHQWLVWVDQG
3UDFWLWLRQHU5HVHDUFKHUVQG(GLWLRQ2[IRUG%ODFNZHOO $FRPSUHKHQVLYHDQG
YHU\SUDFWLFDORYHUYLHZ²WKHUHLVVRPHWKLQJLQLWIRUHYHU\ERG\
6LOYHUPDQ''RLQJ4XDOLWDWLYH5HVHDUFKQG(GLWLRQ/RQGRQ6DJH
*HQHUDOO\DFFHSWHGWREHRQHRIWKHPRVWXVHIXOWLWOHVIRUQRYLFHTXDOLWDWLYH
UHVHDUFKHUV

7DEDFKQLFN%*DQG/6)LGHOO8VLQJ0XOWLYDULDWH6WDWLVWLFVWK(GLWLRQ
1HHGKDP+HLJKWV0$$OO\QDQG%DFRQ 7KHXOWLPDWHDXWKRULW\ZKHQLWFRPHVWR
FRPSOH[VWDWLVWLFDOLVVXHV
7DVKDNNRUL$DQG&7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ6RFLDO

DQG%HKDYLRUDO5HVHDUFK7KRXVDQGV2DNV&$6DJH $OO\RXZDQWWRNQRZDERXW
PL[HGPHWKRGVUHVHDUFK

4XDOLWDWLYHTXDQWLWDWLYHDQGPL[HG
PHWKRGVUHVHDUFK
2QHRIWKHPRVWJHQHUDODQGEHVWNQRZQGLVWLQFWLRQVLQUHVHDUFKPHWKRGRORJ\LVWKDW
EHWZHHQTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFK$V'DYLHV HPSKDVL]HVWKH
GLVWLQFWLRQVLJQLILHVPRUHWKDQPHUHO\XVLQJILJXUHVYHUVXVQRQTXDQWLWDWLYHGDWD VXFK
DVRSHQHQGHGLQWHUYLHZVRUQDWXUDOGDWD LQVWHDGWKHGLFKRWRP\UHIHUVWRVHYHUDO
WKLQJVDWWKHVDPHWLPHWKHJHQHUDOLGHRORJLFDORULHQWDWLRQXQGHUO\LQJWKHVWXG\WKH
PHWKRGRIGDWDFROOHFWLRQDSSOLHGWKHQDWXUHRIWKHFROOHFWHGGDWDDQGWKHPHWKRGRI
GDWDDQDO\VLVXVHGWRSURFHVVWKHGDWDDQGWRREWDLQUHVXOWV7KLVLVFOHDUO\DFRPSOH[
LVVXHVROHWXVVWDUWZLWKDSUHOLPLQDU\ZRUNLQJGHILQLWLRQIRUWKHWZRDSSURDFKHV
x 4XDQWLWDWLYHUHVHDUFKLQYROYHVGDWDFROOHFWLRQSURFHGXUHVWKDWUHVXOWSULPDULO\LQ
QXPHULFDOGDWDZKLFKLVWKHQDQDO\VHGSULPDULO\E\VWDWLVWLFDOPHWKRGV7\SLFDO
H[DPSOHVXUYH\UHVHDUFKXVLQJDTXHVWLRQQDLUHDQDO\VHGE\VWDWLVWLFDOVRIWZDUH
VXFKDV6366
x 4XDOLWDWLYHUHVHDUFKLQYROYHVGDWDFROOHFWLRQSURFHGXUHVWKDWUHVXOWSULPDULO\LQ
RSHQHQGHGQRQQXPHULFDOGDWDZKLFKLVWKHQDQDO\VHGSULPDULO\E\QRQVWDWLVWLFDO
PHWKRGV7\SLFDOH[DPSOHLQWHUYLHZUHVHDUFKZLWKWKHWUDQVFULEHGUHFRUGLQJV
DQDO\VHGE\TXDOLWDWLYHFRQWHQWDQDO\VLV
,PHQWLRQHGLQ&KDSWHUEULHIO\WKDWDOWKRXJKWKHWZRSDUDGLJPVUHSUHVHQWWZR
GLIIHUHQWDSSURDFKHVWRHPSLULFDOUHVHDUFKWKH\DUHQRWQHFHVVDULO\H[FOXVLYH7KHLU
SULQFLSOHGFRPELQDWLRQKDVOHGWRDQHPHUJLQJWKLUGUHVHDUFKDSSURDFK
x 0L[HGPHWKRGVUHVHDUFKLQYROYHVGLIIHUHQWFRPELQDWLRQVRITXDOLWDWLYHDQG
TXDQWLWDWLYHUHVHDUFKHLWKHUDWWKHGDWDFROOHFWLRQRUDWWKHDQDO\VLVOHYHOV7\SLFDO
H[DPSOH&RQVHFXWLYHDQGLQWHUUHODWHGTXHVWLRQQDLUHDQGLQWHUYLHZVWXGLHV
,ZLOOGLVFXVVWKHWKUHHDSSURDFKHVLQGHWDLOLQVHSDUDWHVHFWLRQVEHORZ LQ6HFWLRQV
± EXWOHWXVILUVWVSHQGVRPHWLPHH[DPLQLQJWKHTXDOLWDWLYH±TXDQWLWDWLYH
GLVWLQFWLRQWRXQGHUVWDQGZKDWOLHVDWWKHKHDUWRIWKLVPHWKRGRORJLFDOVFKLVP,Q
UHVHDUFKWH[WVLWKDVEHFRPHVWDQGDUGXVDJHWRUHIHUWRWKHWZRDSSURDFKHVDV48$/
DQG48$1ZKHQFRQWUDVWLQJWKHPDQG,ZLOOVRPHWLPHVIROORZWKLVSUDFWLFH
7KHTXDOLWDWLYH±TXDQWLWDWLYHGLVWLQFWLRQ
$OWKRXJKDWILUVWVLJKWWKHGLIIHUHQFHEHWZHHQTXDOLWDWLYHDQGTXDQWLWDWLYHGDWDUHVHDUFK
DSSHDUVWREHUHODWLYHO\VWUDLJKWIRUZDUGWKHGLVWLQFWLRQKDVEHHQWKHVRXUFHRIDJUHDW
GHDORIGLVFXVVLRQLQWKHSDVWDWHYHU\FRQFHLYDEOHOHYHORIDEVWUDFWLRQ:LWKRXW
GZHOOLQJRQWKLVLVVXHWRRORQJOHWPHRIIHUDWDVWHRIKRZWKLQJVFDQJHWYHU\
FRPSOLFDWHGZKHQZHVWDUWGLVFXVVLQJWKH48$/±48$1FRQWUDVW

7RVWDUWZLWKLVWKHUHUHDOO\VXFKDFRQWUDVW"$QGLIVRZKHUHH[DFWO\GRHVLWOLH"
5LFKDUGV IRUH[DPSOHSRLQWVRXWWKDWWKHQXPHULFDOYVQRQQXPHULFDO
GLVWLQFWLRQGRHVQRWJLYHXVFOHDUHQRXJKJXLGHOLQHVEHFDXVHTXDOLWDWLYHUHVHDUFKHUV
ZRXOGDOPRVWDOZD\VFROOHFWVRPHLQIRUPDWLRQLQQXPEHUV IRUH[DPSOHWKHDJHRIWKH
SDUWLFLSDQWV DQGVLPLODUO\TXDQWLWDWLYHUHVHDUFKHUVXVXDOO\DOVRFROOHFWVRPHQRQ
QXPHULFDOLQIRUPDWLRQ IRUH[DPSOHWKHJHQGHURUQDWLRQDOLW\RIWKHSDUWLFLSDQWV 6R
DVVKHFRQFOXGHVµTXDOLWDWLYHDQGTXDQWLWDWLYHGDWDGRQRWLQKDELWGLIIHUHQWZRUOGV
7KH\DUHGLIIHUHQWZD\VRIUHFRUGLQJREVHUYDWLRQVRIWKHVDPHZRUOG¶ S $UJXLQJ
LQDVLPLODUYHLQ0LOHVDQG+XEHUPDQ DVVHUWWKDWLQVRPHVHQVHDOOGDWDDUH
TXDOLWDWLYHEHFDXVHWKH\UHIHUWRµHVVHQFHVRISHRSOHREMHFWVDQGVLWXDWLRQV¶ S
VRPHWLPHVZHFRQYHUWRXUUDZH[SHULHQFHVRIWKHVRFLDOZRUOGLQWRZRUGV LH48$/
DWRWKHUWLPHVLQWRQXPEHUV LH48$1 7KHUHIRUH6DQGHORZVNL DFWXDOO\
FRQFOXGHVWKDWTXDOLWDWLYHUHVHDUFKLVQRWFOHDUO\GLVWLQJXLVKDEOHIURPTXDQWLWDWLYH
UHVHDUFKEHFDXVHWKHUHLVQRFRQVLVWHQWPDQQHULQZKLFKVXFKDFRPSDULVRQFDQEH
PDGH
(YHQWKRXJK,DJUHHWKDW48$/DQG48$1DUHQRWH[WUHPHVEXWUDWKHUIRUPD
FRQWLQXXPZHVWLOOWHQGWRFRPSDUHWKHPDOOWKHWLPH:K\LVWKDW",ZRXOGVXJJHVW
WKDWWKHDOPRVWLUUHVLVWLEOHXUJHWRFRQWUDVWTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKJRHV
EDFNWRWKUHHEDVLFVRXUFHVRIGLYLVLRQEHWZHHQWKHWZRDSSURDFKHV D DQLGHRORJLFDO
FRQWUDVW E DFRQWUDVWLQFDWHJRUL]DWLRQDQG F DFRQWUDVWLQWKHSHUFHSWLRQRI
LQGLYLGXDOGLYHUVLW\/HWXVORRNDWWKHVHFRQWUDVWVRQHE\RQH
,GHRORJLFDOGLIIHUHQFHV
$OWKRXJKVFKRODUVLQWKHVRFLDOVFLHQFHV IRUH[DPSOHLQVRFLRORJ\ KDYHEHHQXVLQJ
ERWKTXDOLWDWLYHOLNHDQGTXDQWLWDWLYHOLNHGDWDVLQFHWKHEHJLQQLQJRIWKHWKFHQWXU\
WKH48$/±48$1GLVWLQFWLRQRQO\HPHUJHGDIWHUQXPEHUEDVHGVWDWLVWLFDOUHVHDUFK
EHFDPHGRPLQDQWLQWKHPLGGOHRIWKHWKFHQWXU\DQGVRPHVFKRODUVVWDUWHGWR
FKDOOHQJHWKLVKHJHPRQ\IO\LQJWKHµTXDOLWDWLYH¶EDQQHU 7KHJHQHVLVRIWKHWZR
DSSURDFKHVZLOOEHGLVFXVVHGLQPRUHGHWDLOLQVHSDUDWHVHFWLRQVEHORZ 7KXVWKHWHUPV
µTXDOLWDWLYH¶DQGµTXDQWLWDWLYH¶ZHUHRULJLQDOO\LQWURGXFHGDVSDUWRIRUUDWKHUIRUWKH
SXUSRVHRIDQLGHRORJLFDOFRQIURQWDWLRQ,QDWKRXJKWIXODQDO\VLV6FKZDQGW
GHVFULEHVTXDOLWDWLYHLQTXLU\LQJHQHUDODVDµUHIRUPLVWPRYHPHQW¶XQLWLQJDZLGH
YDULHW\RIVFKRODUVZKRDSSHDUWRVKDUHYHU\OLWWOHLQFRPPRQH[FHSWWKHLUJHQHUDO
GLVWDVWHIRUWKHPDLQVWUHDPTXDQWLWDWLYHSDUDGLJP$VKHZULWHV
TXDOLWDWLYHLQTXLU\LVDµKRPH¶IRUDZLGHYDULHW\RIVFKRODUVZKRRIWHQDUH
VHULRXVO\DWRGGVZLWKRQHDQRWKHUEXWZKRVKDUHDJHQHUDOUHMHFWLRQRIWKHEOHQG
RIVFLHQWLVPIRXQGDWLRQDOLVWHSLVWHPRORJ\LQVWUXPHQWDOUHDVRQLQJDQGWKH
SKLORVRSKLFDODQWKURSRORJ\RIGLVHQJDJHPHQWWKDWKDVPDUNHGµPDLQVWUHDP¶
VRFLDOVFLHQFH S
+DYLQJEHHQFUHDWHGLQWKHVSLULWRIDQWDJRQLVPZHVKRXOGQRWEHVXUSULVHGWKDWWKH
WZRWHUPVDUHVWLOORIWHQXVHGWRUHSUHVHQWFRQWUDVWLQJYLHZVDERXWWKHZRUOGDURXQGXV

&RQWUDVWLQJFDWHJRUL]LQJFRGLQJSUDFWLFHV
2QHWKLQJWKDWLVFRPPRQWRHYHU\UHVHDUFKDSSURDFKLVWKDWWKHDOPRVWOLPLWOHVV
LQIRUPDWLRQREWDLQDEOHIURPWKHVRFLDOZRUOGDURXQGXVQHHGVWREHUHGXFHGWRPDNHLW
PDQDJHDEOH5HVHDUFKHUVW\SLFDOO\XVHµFDWHJRULHV¶RUµFRGHV¶WRVWUXFWXUHDQGVKDSH
WKLVLQIRUPDWLRQEXWWKLVLVZKHUHWKHVLPLODULWLHVEHWZHHQ48$/DQG48$1HQG:H
ILQGWKDWWKHQDWXUHRIWKHFDWHJRULHVDQGWKHFDWHJRUL]DWLRQSURFHVVLQ48$/DQG
48$1DUHYHU\GLIIHUHQW,QIDFW%D]HOH\ DUJXHVWKDWµ&RGHV²WKHZD\
WKH\DUHJHQHUDWHGZKDWWKH\VWDQGIRUDQGWKHZD\WKH\DUHXVHG²OLHDWWKHKHDUWRI
GLIIHUHQFHVEHWZHHQTXDQWLWDWLYHDQGTXDOLWDWLYHGDWDDQGDQDO\VLVWRROV¶
4XDQWLWDWLYHUHVHDUFKHUVGHILQHWKHYDULDEOHVWKH\ZRUNZLWKZHOOLQDGYDQFHDQG
DVVLJQDORJLFDOVFDOHRIYDOXHVWRWKHPZKLFKFDQEHH[SUHVVHGLQQXPEHUV7KXV
TXDQWLWDWLYHUHVHDUFKFDQVWDUWDUHVHDUFKSURMHFWZLWKSUHFLVHFRGLQJWDEOHVIRU
SURFHVVLQJWKHGDWD IRUH[DPSOHZLWKLQWKHµJHQGHU¶YDULDEOHµPDOH¶LVWREHDVVLJQHG
DQGµIHPDOH¶ 4XDOLWDWLYHUHVHDUFKHUVDOVRXVHFRGLQJH[WHQVLYHO\EXWWKH48$/
FDWHJRULHVDUHGLIIHUHQWLQWZRLPSRUWDQWZD\V)LUVWWKH\DUHQRWQXPHULFDOEXWYHUEDO
DPRXQWLQJWRVKRUWWH[WXDOODEHOV6HFRQGWKH\DUHXVXDOO\QRWGHWHUPLQHGDSULRULEXW
DUHOHIWRSHQDQGIOH[LEOHDVORQJDVSRVVLEOHWREHDEOHWRDFFRXQWIRUWKHVXEWOH
QXDQFHVRIPHDQLQJXQFRYHUHGGXULQJWKHSURFHVVRILQYHVWLJDWLRQ)RUH[DPSOHLIZH
ZDQWHGWRGUDZWKHERXQGDU\EHWZHHQWZRFRXQWULHVLQDQXQNQRZQWHUUDLQWKH48$1
DSSURDFKZRXOGEHWRWDNHWKHPDSDQGDIWHUGHILQLQJWKHVL]HGLVWULEXWLRQRIWKHWZR
FRXQWULHVGUDZVWUDLJKWOLQHVXVLQJDUXOHU,QFRQWUDVWWKH48$/DSSURDFKZRXOG
UHVLVWWKLVWRSGRZQGHFLVLRQPDNLQJEXWZRXOGH[SHFWWKHERXQGDULHVWRQDWXUDOO\
HPHUJHXVLQJWKHLQKHUHQWJHRJUDSKLFDOSURSHUWLHVRIWKHWHUUDLQ IRUH[DPSOHULYHUV
DQGPRXQWDLQULGJHV
'LIIHUHQWDSSURDFKHVWRLQGLYLGXDOGLYHUVLW\
0RVWGDWDFROOHFWHGLQWKHVRFLDOVFLHQFHVUHJDUGOHVVRIZKHWKHULWLV48$/RU48$1
LVUHODWHGWRSHRSOH²ZKDWWKH\GRZKDWWKH\DUHOLNHZKDWWKH\WKLQNRUEHOLHYHLQ
ZKDWWKH\SODQWRGRHWF%HFDXVHSHRSOHGLIIHUIURPHDFKRWKHULQWKHZD\WKH\
SHUFHLYHLQWHUSUHWDQGUHPHPEHUWKLQJVWKHLUDFFRXQWVZLOOVKRZFRQVLGHUDEOH
YDULDWLRQDFURVVLQGLYLGXDOV7KHSUREOHPLVWKDWQRPDWWHUKRZZHOOIXQGHGRXU
UHVHDUFKLVZHFDQQHYHUH[DPLQHDOOWKHSHRSOHZKRVHDQVZHUVZRXOGEHUHOHYDQWWR
RXUUHVHDUFKTXHVWLRQDQGWKHUHIRUHZHKDYHWRIDFHWKHIDFWWKDWWKHILQDOSLFWXUH
XQIROGLQJLQRXUUHVHDUFKZLOODOZD\VEHDIXQFWLRQRIZKRPZHKDYHVHOHFWHGWRREWDLQ
RXUGDWDIURP
%RWK48$/DQG48$1UHVHDUFKHUVDFNQRZOHGJHWKLVOLQNEHWZHHQWKHVSHFLILF
VDPSOHRISDUWLFLSDQWVH[DPLQHGDQGWKHUHVXOWVREWDLQHGE\WKHUHVHDUFKEXWWKHWZR
FDPSVFRQVLGHUWKHLVVXHLQDYHU\GLIIHUHQWOLJKW4XDQWLWDWLYHUHVHDUFKHUVUHJDUGWKH
VDPSOHUHODWHGYDULDWLRQDVDSUREOHPZKLFKQHHGVWREHIL[HG7KH48$1VROXWLRQLV
WRWDNHDODUJHHQRXJKVDPSOHLQZKLFKWKHLGLRV\QFUDWLFGLIIHUHQFHVDVVRFLDWHGZLWKWKH
SDUWLFXODULQGLYLGXDOVDUHLURQHGRXWE\WKHVDPSOHVL]HDQGWKHUHIRUHWKHSRROHGUHVXOWV

ODUJHO\UHIOHFWWKHFRPPRQDOLWLHVWKDWH[LVWLQWKHGDWD4XDOLWDWLYHUHVHDUFKHUVRQWKH
RWKHUKDQGTXHVWLRQWKHYDOXHRISUHSDULQJDQRYHUDOODYHUDJHGHVFULSWLRQRIDODUJHU
JURXSRISHRSOHEHFDXVHLQWKLVZD\ZHORVHWKHLQGLYLGXDOVWRULHV7KH\VHHWKLVDVDQ
XQGHVLUDEOHUHGXFWLRQSURFHVVEHFDXVHLQ48$/WHUPVWKHUHDOPHDQLQJOLHVZLWK
LQGLYLGXDOFDVHVZKRPDNHXSRXUZRUOG2IFRXUVHTXDOLWDWLYHUHVHDUFKHUVDUHQRW
REOLYLRXVWRWKHIDFWWKDWLQGLYLGXDOVDUHGLIIHUHQWEXWUDWKHUWKDQEHOLHYLQJLQDKLJKHU
OHYHOPHDQLQJWKDWFDQEHDUULYHGDWE\VXPPLQJXSLQGLYLGXDOFDVHVWKH\KROGWKDW
WKHUHDUHPXOWLSOHPHDQLQJVWRGLVFRYHU
7KXVTXDQWLWDWLYHUHVHDUFKHUVIROORZDµPHDQLQJLQWKHJHQHUDO¶VWUDWHJ\ZKHUHDV
TXDOLWDWLYHUHVHDUFKHUVFRQFHQWUDWHRQDQLQGHSWKXQGHUVWDQGLQJRIWKHµPHDQLQJLQWKH
SDUWLFXODU¶+RZHYHUWKHVWRU\GRHVQRWHQGKHUHEHFDXVHWKHµELJQXPEHU¶DSSURDFKRI
TXDQWLWDWLYHUHVHDUFKHUVKDVRIIHUHGDQDGGLWLRQDOERQXVIRU48$1GDWDDQDO\VLV
VWDWLVWLFV:HPXVWVWRSDQGH[DPLQHWKLVDELWPRUHEHIRUHZHPRYHRQ
6WDWLVWLFVYHUVXVUHVHDUFKHUVHQVLWLYLW\
2QFHTXDQWLWDWLYHUHVHDUFKHUVKDGJRQHGRZQWKHµPHDQLQJLQQXPEHUV¶SDWKD
ZHOFRPHERQXVHPHUJHG0DWKHPDWLFLDQVKDYHIRXQGWKDWLIZHKDYHDVXIILFLHQWO\ELJ
VDPSOHVL]HWKHFKDUDFWHULVWLFVRIWKHSHRSOHLQWKLVJURXSZLOODSSURDFKDYHU\VSHFLDO
SDWWHUQWHUPHGµQRUPDOGLVWULEXWLRQ¶7KLVPHDQVWKDWZLWKLQWKHVDPSOHDIHZSHRSOH
ZLOOGLVSOD\YHU\KLJKYDOXHVDIHZRWKHUVYHU\ORZRQHVZLWKWKHEXONRIWKHVDPSOH
FHQWUHGDURXQGWKHPLGGOHRUDYHUDJHUDQJH7KLVLVWKHDOOLPSRUWDQWµEHOOVKDSHG
FXUYH¶ VHHEHORZ DQGLWKDVEHHQIRXQGWKDWWKHJUHDWHUWKHVDPSOHWKHPRUHµQRUPDO¶
WKHGLVWULEXWLRQDQGWKHPRUHUHJXODUWKHFXUYHEHFRPHV )RUPRUHGHWDLOVVHH6HFWLRQ
:KDWPDNHVWKLVEHOOVKDSHGFXUYHVRLPSRUWDQWLVWKDWLWKDVXQLTXHSURSHUWLHV
XSRQZKLFKLWLVSRVVLEOHWREXLOGDZKROHUDQJHRIPDWKHPDWLFDOSURFHGXUHVWKDWKDYH
OHGWRWKHGHYHORSPHQWRIµVWDWLVWLFV¶

7KXVDGRSWLQJWKHµPHDQLQJLQQXPEHUV¶DSSURDFKKDVQRWRQO\RIIHUHGTXDQWLWDWLYH
UHVHDUFKHUVDZD\RXWRIWKHLQGLYLGXDOUHVSRQGHQWYDULDWLRQGLOHPPDPHQWLRQHGDERYH
EXWLWKDVDOVRSURYLGHGDQHODERUDWHVHWRIVWDWLVWLFDODQDO\WLFDOWRROVWRXVHWRDGG
V\VWHPDWLFLW\WRWKHGDWDDQDO\VLVSKDVHUDWKHUWKDQKDYLQJWRUHO\RQWKHUHVHDUFKHU¶V
VXEMHFWLYHLQWHUSUHWDWLRQV7KXVTXDQWLWDWLYHUHVHDUFKFRXOGHOLPLQDWHLQGLYLGXDO
YDULDELOLW\ERWKDWWKHGDWDFROOHFWLRQDQGWKHGDWDDQDO\VLVVWDJHV)RUPDQ\VFKRODUV
WKHPDMRUDWWUDFWLRQRI48$1LVWKLVV\VWHPDWLFµLQGLYLGXDOSURRI¶QDWXUHJRYHUQHGE\
SUHFLVHUXOHVDQGUHJXODWLRQVWKXVDSSUR[LPDWLQJWKHUHJXODULW\RIWKHQDWXUDOVFLHQFHV

,QFRQWUDVWWKHµPHDQLQJLQWKHSDUWLFXODU¶DSSURDFKRITXDOLWDWLYHUHVHDUFKKDVQRW
RIIHUHGDQ\ERQXVJLIWVIRUWKHDQDO\VLVSKDVHRITXDOLWDWLYHUHVHDUFK&RQVHTXHQWO\
DOWKRXJKTXDOLWDWLYHUHVHDUFKDOVRDSSOLHVYDULRXVGDWDDQDO\WLFDOSURFHGXUHVWRPDNH
WKHLQYHVWLJDWLRQVPRUHULJRURXVDQGV\VWHPDWLFDWWKHKHDUWRIDQ\TXDOLWDWLYHDQDO\VLV
LVVWLOOWKHUHVHDUFKHU¶VVXEMHFWLYHVHQVLWLYLW\WUDLQLQJDQGH[SHULHQFH7KXVZKLOHQR
RQHZRXOGGHQ\WKDWE\XVLQJTXDOLWDWLYHPHWKRGVZHFDQXQFRYHUVXEWOHPHDQLQJVWKDW
DUHLQHYLWDEO\ORVWLQTXDQWLWDWLYHUHVHDUFK48$/LVOLQNHGWRWZREDVLFVRXUFHVRI
YDULDWLRQDVVRFLDWHGZLWKWKHLQGLYLGXDOUHVSRQGHQWVDQGWKHLQGLYLGXDOUHVHDUFKHU)RU
PDQ\VFKRODUVWKHPDMRUDWWUDFWLRQRI48$/LVH[DFWO\WKLVVHQVLWLYLW\WRWKHLQGLYLGXDO
EXWZHFDQSHUKDSVVWDUWVHQVLQJDWWKLVSRLQWZKHUHVRPHRIWKHVWURQJHPRWLRQV
FKDUDFWHULVLQJWKH48$/±48$1GHEDWHRULJLQDWHLWLVDOOWRRHDV\WRSUHVHQWWKH
DERYHFRQWUDVWDVWKHDQWDJRQLVWLFILJKWEHWZHHQµFDOORXV¶YHUVXVµVHQVLWLYH¶RU
µV\VWHPDWLF¶YHUVXVµIX]]\¶DQGXOWLPDWHO\EHWZHHQµREMHFWLYH¶YHUVXVµVXEMHFWLYH¶
7KH48$/±48$1FRQWUDVWDQGWKHµSDUDGLJPZDU¶
7KHLQKHUHQW48$/±48$1GLIIHUHQFHVOHGWRFRQIOLFWVZKLFKHVFDODWHGLQWRDIXOO\
IOHGJHGµSDUDGLJPZDU¶LQWKHVDQGV7KLVFODVKZDVDOPRVWLQHYLWDEOHEHFDXVH
E\WKHQDOOWKHFRPSRQHQWVZHUHLQSODFHIRUDSURSHUVKRZGRZQEHWZHHQWKHWZR
PHWKRGRORJLFDOFDPSVWKHWHUPV48$/DQG48$1ZHUHRULJLQDOO\LQWURGXFHGWR
GHQRWHDQDQWDJRQLVWLFVWDQGSRLQWDQGDVZHKDYHVHHQLQWKHSUHYLRXVVHFWLRQVWKLV
LQLWLDOFRQIOLFWLQJVWDQFHZDVJLYHQVXEVWDQFHE\WKHFRQWUDVWLQJSDWWHUQVRIWKHWZR
UHVHDUFKSDUDGLJPVLQ
D FDWHJRUL]LQJWKHZRUOG 48$1SUHGHWHUPLQHGQXPHULFDOFDWHJRU\V\VWHP48$/
HPHUJHQWIOH[LEOHYHUEDOFRGLQJ
E SHUFHLYLQJLQGLYLGXDOGLYHUVLW\ 48$1XVLQJODUJHVDPSOHVWRLURQRXWDQ\
LQGLYLGXDOLGLRV\QFUDVLHV48$/IRFXVLQJRQWKHXQLTXHPHDQLQJFDUULHGE\
LQGLYLGXDORUJDQLVPV
F DQDO\VLQJGDWD 48$1UHO\LQJRQWKHIRUPDOL]HGV\VWHPRIVWDWLVWLFV48$/
UHO\LQJRQWKHUHVHDUFKHU¶VLQGLYLGXDOVHQVLWLYLW\
4XDQWLWDWLYHUHVHDUFKZDVVHHQWRRIIHUDVWUXFWXUHGDQGKLJKO\UHJXODWHGZD\RI
DFKLHYLQJDPDFURSHUVSHFWLYHRIWKHRYHUDUFKLQJWUHQGVLQWKHZRUOGZKHUHDV
TXDOLWDWLYHUHVHDUFKZDVSHUFHLYHGWRUHSUHVHQWDIOH[LEOHDQGKLJKO\FRQWH[WVHQVLWLYH
PLFURSHUVSHFWLYHRIWKHHYHU\GD\UHDOLWLHVRIWKHZRUOG,QWKHSDUDGLJPZDUWKLV
GLVWLQFWLRQZDVH[WHQGHGWRWZRGLIIHUHQWZRUOGYLHZVDQGWKHµSDUDGLJPZDUULRUV¶
FRQWHVWHGZKLFKOHYHORIDQDO\VLVSURYLGHGDPRUHYDOLGUHSUHVHQWDWLRQRIKXPDQOLIH
DQGWKHVRFLDOZRUOGLQJHQHUDO0DQ\LQWHOOHFWXDOO\FKDOOHQJLQJSRVLWLRQSDSHUVKDYH
EHHQZULWWHQDERXWWKHVHGLIIHUHQWRULHQWDWLRQVDUJXLQJWKDWDWWKHHQGRIWKHGD\
TXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKPHWKRGRORJLHVDUHURRWHGLQWZRVWULNLQJO\
GLIIHUHQWSDUDGLJPVDQGWKXVE\PL[LQJWKHPZHDUHOLNHO\WRORVHWKHLUYHU\HVVHQFH
7KHSRVLWLRQ,KDYHWDNHQLQWKLVERRNLVWKDWWKLVYLHZLVLQFRUUHFW

7KUHHSRVLWLRQVUHJDUGLQJWKH48$/±48$1GLIIHUHQFH
SXULVWVLWXDWLRQDOLVWDQGSUDJPDWLVW
$VZHKDYHVHHQLQWKHSUHYLRXVVHFWLRQWDNLQJWKHRULVLQJWRWKHOHYHORIDEVWUDFWLRQRI
GLIIHUHQWZRUOGYLHZVSDUDGLJPVDQGSHUVSHFWLYHVFDQORJLFDOO\OHDGWRSURSRVLQJZKDW
5RVVPDQDQG:LOVRQ FDOOHGDµSXULVW¶DSSURDFKWRUHVHDUFKPHWKRGRORJ\
DUJXLQJWKDWWKHTXDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGRORJLHVDUHPXWXDOO\H[FOXVLYH
,QWHUHVWLQJO\DOWKRXJKWKHUHLVQRVKRUWDJHRIFRQYLQFLQJLQWHOOHFWXDODUJXPHQWVWR
MXVWLI\SDUDGLJPLQFRPSDWLELOLW\PRVWUHVHDUFKHUVKDYHDFWXDOO\VWRSSHGVKRUWRI
FODLPLQJWKHLQHYLWDELOLW\RIWKLVFRQIOLFWDQGSDUWLFXODUO\LQWKHSDVWGHFDGHVFKRODUV
KDYHVWDUWHGWRORRNIRUVRPHVRUWRIDQLQWHUIDFHEHWZHHQWKHWZRUHVHDUFKWUDGLWLRQV
0LOHVDQG+XEHUPDQ ± IRUH[DPSOHSRLQWHGRXWWKDWµ,QHSLVWHPRORJLFDO
GHEDWHVLWLVWHPSWLQJWRRSHUDWHDWWKHSROHV%XWLQWKHDFWXDOSUDFWLFHRIHPSLULFDO
UHVHDUFKZHEHOLHYHWKDWDOORIXV²UHDOLVWVLQWHUSUHWLYLVWVFULWLFDOWKHRULVWV²DUHFORVHU
WRWKHFHQWUHZLWKPXOWLSOHRYHUODSV¶
,QGHHGLIZHVWRSWUHDWLQJ48$/DQG48$1UHVHDUFKLQDYHU\JHQHUDODQG
FRQWUDVWLQJPDQQHUDQGIRFXVRQWKHVSHFLILFUHVHDUFKLVVXHVDWKDQGZHILQGWKDW
FRQFUHWHUHVHDUFKWRSLFVYDU\JUHDWO\LQWKHH[WHQWWRZKLFKWKH\OHQGWKHPVHOYHVWR
PLFURRUPDFUROHYHODQDO\VLV7RWDNHDQH[DPSOHIURPP\RZQUHVHDUFKWKHFRQFHSW
RIµGHPRWLYDWLRQ¶DSSHDUVWREHRQHZKHUHDPLFUROHYHOTXDOLWDWLYHLQYHVWLJDWLRQFDQEH
EHQHILFLDOLQXQFRYHULQJWKHVXEWOHSHUVRQDOSURFHVVHVZKHUHE\RQH¶VHQWKXVLDVPLV
JUDGXDOO\GDPSHQHGE\DQXPEHURILQWHUQDODQGH[WHUQDOGHPRWLYDWLQJIDFWRUV
'|UQ\HL 2QWKHRWKHUKDQGWKHSURFHVVRIµODQJXDJHJOREDOL]DWLRQ¶FDQEH
LQYHVWLJDWHGSDUWLFXODUO\ZHOOIURPDTXDQWLWDWLYHPDFURSHUVSHFWLYHGHWHUPLQLQJIRU
H[DPSOHKRZ*OREDO(QJOLVKLPSDFWVWKHDFTXLVLWLRQDQGXVHRIORFDOODQJXDJHVLQ
YDULRXVVSHHFKFRPPXQLWLHV '|UQ\HL1pPHWKDQG&VL]pU 7KLVZRXOGVXJJHVW
WKDWERWKDSSURDFKHVKDYHYDOXHLIWKH\DUHDSSOLHGLQWKHDSSURSULDWHUHVHDUFK
FRQWH[W²DYLHZWKDWKDVEHHQRIWHQUHIHUUHGWRDVWKHµVLWXDWLRQDOLVW¶DSSURDFKWR
UHVHDUFKPHWKRGRORJ\ 6HH5RVVPDQDQG:LOVRQ
$OWKRXJKWKHVLWXDWLRQDOLVWYLHZDFFHSWVWKHVWUHQJWKVRIERWKUHVHDUFKWUDGLWLRQVLW
VWLOOUHSUHVHQWVDQµHLWKHURU¶DSSURDFK+RZHYHUZHGRQRWQHFHVVDULO\KDYHWRVWRS
KHUH:KLOHLWLVWUXHWKDWSDUWLFXODUUHVHDUFKTXHVWLRQVRUWRSLFVFDQEHPRUHQDWXUDOO\
OLQNHGWRHLWKHU48$/RU48$1PHWKRGVLQPRVWFDVHVZHFDQDOVRORRNDWWKHVDPH
UHVHDUFKTXHVWLRQIURPDQRWKHUDQJOHXVLQJWKHRWKHUDSSURDFKWKXVXQFRYHULQJQHZ
DVSHFWVRIWKHLVVXH)RUH[DPSOHZKHQFRQVLGHULQJVWXGHQWGHPRWLYDWLRQ²ZKLFK,
VXJJHVWHGDERYHFDQEHVXFFHVVIXOO\H[DPLQHGWKURXJKDTXDOLWDWLYHDSSURDFK²ZHFDQ
DOVRH[DPLQHKRZH[WHQVLYHWKLVSUREOHPLVLQRXUVFKRROVRUKRZPXFKLPSDFWLWKDV
RQVWXGHQWV¶OHDUQLQJDFKLHYHPHQWDQGWKHVHTXHVWLRQVFDQEHEHVWDGGUHVVHGWKURXJK
TXDQWLWDWLYHVWXGLHV$QGVLPLODUO\HYHQEURDGWUHQGVVXFKDVODQJXDJHJOREDOL]DWLRQ
FDQEHLQYHVWLJDWHGIURPDPLFURSHUVSHFWLYHE\DQDO\VLQJIRUH[DPSOHWKHGD\WRGD\
SURFHVVZKHUHE\ELOLQJXDOIDPLOLHVLQPXOWLFXOWXUDOHQYLURQPHQWVVKLIWWRZDUGVWKHXVH
RIRQHRUWKHRWKHUODQJXDJH7KLVLQGLFDWHVWKDWVRPHVRUWRIDQLQWHJUDWLRQRIWKHWZR

UHVHDUFKPHWKRGRORJLHVFDQEHEHQHILFLDOWRµFRUURERUDWH SURYLGHFRQYHUJHQFHLQ
ILQGLQJV HODERUDWH SURYLGHULFKQHVVDQGGHWDLO RULQLWLDWH RIIHUQHZLQWHUSUHWDWLRQV
ILQGLQJVIURPWKHRWKHUPHWKRG¶ 5RVVPDQDQG:LOVRQ 7KLVLVWKH
SUDJPDWLVWSRVLWLRQXQGHUO\LQJPL[HGPHWKRGVUHVHDUFKDQGDVVWDWHGLQWKH3UHIDFH
DQG&KDSWHULWLVP\SHUVRQDOEHOLHIWKDWPL[LQJPHWKRGVKDVJUHDWSRWHQWLDOLQPRVW
UHVHDUFKFRQWH[WV
4XDQWLWDWLYHUHVHDUFK
/HWXVVHWRXWRQDPRUHGHWDLOHGH[DPLQDWLRQRIWKHWKUHHUHVHDUFKDSSURDFKHV7KHIDFW
WKDW,EHJLQZLWKWKHDQDO\VLVRITXDQWLWDWLYHUHVHDUFKLVQRWWREHWDNHQDVDQLQGLFDWLRQ
RIDKLHUDUFKLFDORUGHU0\UHDVRQLVSXUHO\SUDJPDWLFEHFDXVHTXDOLWDWLYHUHVHDUFK
JDLQHGSDUDGLJPDWLFVWDWXVDVDUHDFWLRQDJDLQVWTXDQWLWDWLYHUHVHDUFKLWLVSUDFWLFDOWR
JHWWRNQRZTXDQWLWDWLYHUHVHDUFKILUVWDVDNLQGRIEDVHOLQH,ZLOOIROORZWKLVSUDFWLFH
WKURXJKRXWWKHZKROHERRN
%ULHIKLVWRULFDORYHUYLHZ
4XDQWLWDWLYHVRFLDOUHVHDUFKZDVRULJLQDOO\LQVSLUHGE\WKHVSHFWDFXODUSURJUHVVRIWKH
QDWXUDOVFLHQFHVLQWKHWKFHQWXU\DQGWKHUHIRUHHDUO\VRFLDOUHVHDUFKHUVVHWRXWWR
DGRSWZKDWZDVFDOOHGWKHµVFLHQWLILFPHWKRG¶LQWKHLULQYHVWLJDWLRQV7KLVPHWKRGKDG
EHHQHYROYLQJLQZHVWHUQWKLQNLQJVLQFHDERXWWKHPLGWKFHQWXU\ WKHSHULRGRIWKH
(QOLJKWHQPHQW WKURXJKWKHZRUNRISKLORVRSKHUVDQGVFKRODUVVXFKDV&RSHUQLFXV
%DFRQ*DOLOHL.HSOHU1HZWRQ'HVFDUWHV+XPH&RPWHDQG3HLUFH )RURYHUYLHZV
VHH*DUUDWWDQG/L-RUGDQ %URDGO\VSHDNLQJWKHVFLHQWLILFPHWKRG
SRVWXODWHVWKUHHNH\VWDJHVLQWKHUHVHDUFKSURFHVV D REVHUYLQJDSKHQRPHQRQRU
LGHQWLI\LQJDSUREOHP E JHQHUDWLQJDQLQLWLDOK\SRWKHVLVDQG F WHVWLQJWKH
K\SRWKHVLVE\FROOHFWLQJDQGDQDO\VLQJHPSLULFDOGDWDXVLQJVWDQGDUGL]HGSURFHGXUHV
2QFHWKHK\SRWKHVLVKDVEHHQVXFFHVVIXOO\WHVWHGDQGIXUWKHUYDOLGDWHGWKURXJK
UHSOLFDWLRQLWEHFRPHVDFFHSWHGDVDVFLHQWLILFWKHRU\RUODZ7KXVWKHVFLHQWLILF
PHWKRGRIIHUHGDWRROWRH[SORUHTXHVWLRQVLQDQµREMHFWLYH¶PDQQHUWU\LQJWRPLQLPL]H
WKHLQIOXHQFHRIDQ\UHVHDUFKHUELDVRUSUHMXGLFHWKHUHE\UHVXOWLQJLQZKDWVFKRODUV
EHOLHYHGZDVDQDFFXUDWHDQGUHOLDEOHGHVFULSWLRQRIWKHZRUOG
7KHVFLHQWLILFPHWKRGZDVFORVHO\DVVRFLDWHGZLWKQXPHULFDOYDOXHVDQGVWDWLVWLFV
DORQJWKHOLQHRI1REHOSUL]HZLQQHU/RUG5XWKHUIRUG¶VIDPRXVPD[LPWKDWDQ\
NQRZOHGJHWKDWRQHFDQQRWPHDVXUHQXPHULFDOO\µLVDSRRUVRUWRINQRZOHGJH¶%HLQJD
VFLHQWLVWZDVXOWLPDWHO\DVVRFLDWHGZLWKHPSLULFDOO\PHDVXULQJRQH¶VVXEMHFWPDWWHU
DQGSUHIHUDEO\FRQGXFWLQJH[SHULPHQWV7RVHUYHWKHPDWKHPDWLFDOQHHGVRIWKHQHZO\
HPHUJLQJVRFLDOVFLHQFHVVWDWLVWLFVEHFDPHDIXOO\IOHGJHGVXEGLVFLSOLQHRI
PDWKHPDWLFVE\WKHHQGRIWKHWKFHQWXU\7KHFRQWULEXWLRQRIRQHVFKRODULQ
SDUWLFXODU)UDQFLV*DOWRQZDVVLJQLILFDQWLQHVWDEOLVKLQJTXDQWLWDWLYHGDWDFROOHFWLRQ
DQGDQDO\WLFDOPHWKRGVLQSV\FKRORJ\DWWKHWXUQRIWKHWKFHQWXU\$PRQJVWRWKHU

WKLQJV*DOWRQLQLWLDWHGSV\FKRORJLFDOWHVWLQJLQWURGXFHGWKHXVHRITXHVWLRQQDLUHVDQG
FUHDWHGWKHVWDWLVWLFDOFRQFHSWVRIUHJUHVVLRQDQGFRUUHODWLRQ
7KHILUVWKDOIRIWKHWKFHQWXU\VDZPDMRUGHYHORSPHQWVERWKLQWKHVFLHQWLILF
PHWKRG PRVWQRWDEO\WKURXJKWKHZRUNRI.DUO3RSSHU DQGLQVWDWLVWLFV IRUH[DPSOH
E\6SHDUPDQ)LVKHU1H\PDQDQG3HDUVRQ OHDGLQJWRWKHLQFUHDVHGXVHRI
TXDQWLWDWLYHPHWKRGRORJ\DFURVVWKHZKROHUDQJHRIVRFLDOGLVFLSOLQHV$VDUHVXOWRI
WKLVSURJUHVVWKHVRFLDOVFLHQFHVDFKLHYHGPDWXULW\DQGHDUQHGWKHUHSXWDWLRQRIEHLQJ
DEOHWRVWXG\KXPDQEHLQJVµVFLHQWLILFDOO\¶ERWKDWWKHLQGLYLGXDODQGWKHVRFLHWDOOHYHOV
)XHOOHGE\WKHDGYDQFHVLQSV\FKRPHWULFV DVXEGLVFLSOLQHIRFXVLQJRQPHDVXUHPHQWLQ
SV\FKRORJ\ FODVVLFDOWHVWWKHRU\H[SHULPHQWDOGHVLJQVXUYH\UHVHDUFKTXHVWLRQQDLUH
WKHRU\DQGPXOWLYDULDWHVWDWLVWLFVWKHPLGGOHRIWKHWKFHQWXU\EHFDPHGRPLQDWHGE\
TXDQWLWDWLYHPHWKRGRORJ\LQWKHVRFLDOVFLHQFHV7KLVKHJHPRQ\RQO\VWDUWHGWRFKDQJH
LQWKHVDVDUHVXOWRIWKHFKDOOHQJHVRITXDOLWDWLYHUHVHDUFKOHDGLQJWRD
UHVWUXFWXULQJRIUHVHDUFKPHWKRGRORJ\&XUUHQWO\LQPDQ\DUHDVRIWKHVRFLDOVFLHQFHV
ZHFDQVHHDSHDFHIXOFRH[LVWHQFHRITXDQWLWDWLYHDQGTXDOLWDWLYHPHWKRGV
,QDSSOLHGOLQJXLVWLFVDFFRUGLQJWR/D]DUDWRQ WKHSHULRGEHWZHHQ±
VDZDVLJQLILFDQWLQFUHDVHRITXDQWLWDWLYHUHVHDUFKDUWLFOHVZKLFKZHQWKDQGLQKDQG
ZLWKWKHSXEOLFDWLRQRIVHYHUDOUHVHDUFKPHWKRGVWH[WVLQWKHVFXOPLQDWLQJLQ
+DWFKDQG/D]DUDWRQ¶V VHPLQDO5HVHDUFK0DQXDOWKLVSURYLGHGDYHU\GHWDLOHG
VXPPDU\RITXDQWLWDWLYHUHVHDUFKDQGVWDWLVWLFVULFKO\LOOXVWUDWHGZLWKSXEOLVKHG
TXDQWLWDWLYHVWXGLHV/D]DUDWRQ UHSRUWVRQDVXUYH\RIHPSLULFDOVWXGLHVWKDW
DSSHDUHGLQIRXUPDMRUDSSOLHGOLQJXLVWLFMRXUQDOVEHWZHHQDQGWKHUHVXOWV
VKRZWKDWDVPDQ\DVSHUFHQWRIWKHUHVHDUFKSDSHUVZHUHTXDQWLWDWLYH ZKLOHSHU
FHQWTXDOLWDWLYHDQGSHUFHQWPL[HGPHWKRGV ZKLFKOHG/D]DUDWRQWRFRQFOXGHWKDW
TXDQWLWDWLYHVWXGLHVµUHLJQVXSUHPH¶ S LQRXUILHOG
/D]DUDWRQ DOVRKLJKOLJKWHGDPDMRUFKDQJHWDNLQJSODFHLQUHVHDUFK
RULHQWDWLRQLQDSSOLHGOLQJXLVWLFVZKLOHLQWKHVWKHUHZDVDQµXQTXHVWLRQHG
UHOLDQFH¶RQTXDVLH[SHULPHQWDOVWXGLHVWKHSDVW\HDUVKDYHEURXJKWDORQJDEURDGHU
PXOWLGLVFLSOLQDU\SHUVSHFWLYHRQUHVHDUFKPHWKRGRORJ\ZLWKDQLQFUHDVLQJQXPEHURI
DOWHUQDWLYHRIWHQTXDOLWDWLYHGHVLJQVHPSOR\HG,QDQRYHUYLHZRIUHVHDUFKPHWKRGVLQ
WKHILHOG'XII DOVRKLJKOLJKWVWKHJURZLQJVRSKLVWLFDWLRQRITXDQWLWDWLYHVWXGLHV
LQWKHVERWKLQWHUPVRIWKHLUGHVLJQDQGWKHLUSV\FKRPHWULFUHILQHPHQWZKLFK
FRQILUPV/D]DUDWRQ¶V FRQFOXVLRQWKDWWKHUHKDVEHHQDµFRPLQJRIDJH¶RI
TXDQWLWDWLYHUHVHDUFKLQDSSOLHGOLQJXLVWLFV
0DLQFKDUDFWHULVWLFVRITXDQWLWDWLYHUHVHDUFK
$VZHVDZLQWKHSUHYLRXVVHFWLRQTXDQWLWDWLYHVRFLDOUHVHDUFKKDGJURZQRXWRIWKH
GHVLUHWRHPXODWHWKHµREMHFWLYH¶SURFHGXUHVLQWKHQDWXUDOVFLHQFHV+RZHYHUDORQJ
ZLWKPDQ\RWKHUV6KDYHOVRQDQG7RZQH SRLQWRXWWKDWHYHQWKRXJKVHYHUDO
DVSHFWVRIWKHµVFLHQWLILFPHWKRG¶DSSHDUWREHWUDQVIHUDEOHWRVRFLDOUHVHDUFKWKHUHDUH
DOVRVRPHIXQGDPHQWDOGLIIHUHQFHVEHWZHHQWKHQDWXUDODQGVRFLDOVFLHQFHV7KHPRVW

REYLRXVRIWKHVHLVWKDWXQOLNHDWRPVRUPROHFXOHVSHRSOHVKRZYDULDWLRQRYHUWLPH
DQGDFURVVVRFLDODQGFXOWXUDOFRQWH[WV7KH\DOVRGLVSOD\ZLWKLQLQGLYLGXDOYDULDWLRQ
DQGWKHUHIRUHHYHQLIWKH\DUHSODFHGXQGHUVLPLODUFRQGLWLRQVWKHLUUHDFWLRQZLOOYDU\
ZLGHO\ZKLFKLVVRPHWKLQJQDWXUDOVFLHQWLVWVZRUNLQJZLWKDWRPVDQGPROHFXOHVGRQRW
KDYHWRZRUU\DERXW '|UQ\HL 7KHUHIRUHZKLOHTXDQWLWDWLYHPHWKRGVLQWKH
VRFLDOVFLHQFHVE\DQGODUJHDOLJQZLWKWKHJHQHUDOSULQFLSOHVRIWKHµVFLHQWLILFPHWKRG¶
WKH\DOVRVKRZFHUWDLQGLVWLQFWLYHIHDWXUHV
6HFWLRQDOUHDG\OLVWHGVRPHRIWKHFKDUDFWHULVWLFIHDWXUHVRITXDQWLWDWLYHUHVHDUFK
7KHIROORZLQJVXPPDU\UHLWHUDWHVWKRVHDQGDGGVVRPHIXUWKHUFKDUDFWHULVWLFVWKDWKDYH
QRWEHHQPHQWLRQHGEHIRUH
x 8VLQJQXPEHUV7KHVLQJOHPRVWLPSRUWDQWIHDWXUHRITXDQWLWDWLYHUHVHDUFKLV
QDWXUDOO\WKDWLWLVFHQWUHGDURXQGQXPEHUV7KLVERWKRSHQVXSDUDQJHRI
SRVVLELOLWLHVDQGVHWVVRPHOLPLWDWLRQVIRUUHVHDUFKHUV1XPEHUVDUHSRZHUIXODV
DWWHVWHGWRE\WKHGLVFLSOLQHRIPDWKHPDWLFV<HWQXPEHUVDUHDOVRUDWKHUSRZHUOHVV
LQWKHPVHOYHVEHFDXVHLQUHVHDUFKFRQWH[WVWKH\GRQRWPHDQDQ\WKLQJZLWKRXW
FRQWH[WXDOµEDFNLQJ¶7KH\DUHIDFHOHVVDQGPHDQLQJOHVVXQOHVVZHVSHFLI\H[DFWO\
WKHFDWHJRU\WKDWZHXVHWKHVSHFLILFQXPEHUIRUDQGDOVRWKHGLIIHUHQWYDOXHV
ZLWKLQWKHYDULDEOH LHNQRZLQJZKDWµ¶RUµ¶PHDQLQDSDUWLFXODUFDWHJRU\
7KXVIRUQXPEHUVWRZRUNZHQHHGSUHFLVHGHILQLWLRQVRIWKHFRQWHQWDQGWKH
ERXQGDULHVRIWKHYDULDEOHVZHXVHDQGZHDOVRQHHGH[DFWGHVFULSWRUVIRUWKHUDQJH
RIYDOXHVWKDWDUHDOORZHGZLWKLQWKHYDULDEOH+RZHYHULIZHPDQDJHWRSURYLGH
DOOWKLVQXPEHUVGRZRUNDQGWKHGHYHORSPHQWRITXDQWLWDWLYHUHVHDUFKRYHUWKHODVW
FHQWXU\KDVEHHQIUDQNO\DVWRXQGLQJ7KHOHYHORIVRSKLVWLFDWLRQWKDWTXDQWLWDWLYH
GDWDDQDO\VLVKDVUHDFKHGLVDZHVRPHLQHYHU\VHQVHRIWKHZRUG
x $SULRULFDWHJRUL]DWLRQ%HFDXVHWKHXVHRIQXPEHUVDOUHDG\GRPLQDWHVWKHGDWD
FROOHFWLRQSKDVHWKHZRUNUHTXLUHGWRVSHFLI\WKHFDWHJRULHVDQGYDOXHVQHHGVWREH
GRQHSULRUWRWKHDFWXDOVWXG\ 6HHDOVR6HFWLRQ ,IIRUH[DPSOHUHVSRQGHQWV
DUHDVNHGWRHQFLUFOHILJXUHVLQDTXHVWLRQQDLUHLWHPWKH\KDYHWRNQRZH[DFWO\
ZKDWWKRVHILJXUHVUHSUHVHQWDQGLQRUGHUWRPDNHVXUHWKDWHDFKUHVSRQGHQWJLYHV
WKHLUQXPHULFDODQVZHUEDVHGRQWKHVDPHXQGHUVWDQGLQJWKHGHILQLWLRQVDQGYDOXH
GHVFULSWRUVQHHGWREHXQDPELJXRXV7RDFKLHYHWKLVWDNHVWLPHDQGHIIRUWDQG
DOWKRXJK DVZLOOEHGLVFXVVHGODWHU PRVWSKDVHVRITXDOLWDWLYHUHVHDUFKWHQGWREH
PRUHODERXULQWHQVLYHWKDQWKRVHRITXDQWLWDWLYHUHVHDUFKWKHSUHSDUDWLRQSKDVHLV
DQH[FHSWLRQ:KHUHDVLQDTXDOLWDWLYHLQWHUYLHZVWXG\RQHFDQVWDUWWKHILUVW
LQWHUYLHZVRRQDIWHUWKHLQVWLJDWLRQRIWKHSURMHFWLQDTXDQWLWDWLYHVWXG\VHYHUDO
ZHHNVDQGRIWHQPRQWKVRIPHWLFXORXVSUHSDUDWLRQDQGSLORWLQJDUHXVXDOO\QHHGHG
EHIRUHWKHILQDOL]HGLQVWUXPHQWFDQEHDGPLQLVWHUHG/XFNLO\DIWHUWKH
DGPLQLVWUDWLRQRIWKHLQVWUXPHQWWKLQJVVSHHGXSDQGHYHQLQDODUJHVFDOH
TXDQWLWDWLYHVWXG\LWLVQRWXQKHDUGRIWRKDYHSUHOLPLQDU\UHVXOWVZLWKLQDZHHN
DIWHUWKHGDWDKDVEHHQFROOHFWHG7KLVZRXOGEHLPSRVVLEOHLQTXDOLWDWLYHUHVHDUFK

x 9DULDEOHVUDWKHUWKDQFDVHV$VGLVFXVVHGLQ6HFWLRQTXDQWLWDWLYHUHVHDUFKHUV
DUHOHVVLQWHUHVWHGLQLQGLYLGXDOVWKDQLQWKHFRPPRQIHDWXUHVRIJURXSVRISHRSOH
7KHUHIRUHLQFRQWUDVWWRWKH48$/HPSKDVLVRQWKHLQGLYLGXDOFDVH48$1
UHVHDUFKLVFHQWUHGDURXQGWKHVWXG\RIYDULDEOHVWKDWFDSWXUHWKHVHFRPPRQ
IHDWXUHVDQGZKLFKDUHTXDQWLILHGE\FRXQWLQJVFDOLQJRUE\DVVLJQLQJYDOXHVWR
FDWHJRULFDOGDWD 6HH6HFWLRQVDQG $OOWKHYDULRXVTXDQWLWDWLYH
PHWKRGVDUHDLPHGDWLGHQWLI\LQJWKHUHODWLRQVKLSVEHWZHHQYDULDEOHVE\PHDVXULQJ
WKHPDQGRIWHQDOVRPDQLSXODWLQJWKHP DVLQH[SHULPHQWDOVWXGLHVVHH6HFWLRQ
+DPPHUVOH\DQG$WNLQVRQ UHJDUGWKHTXHVWIRUVSHFLI\LQJWKH
UHODWLRQVKLSVDPRQJVWYDULDEOHVDVWKHGHILQLQJIHDWXUHRITXDQWLWDWLYHVRFLDO
UHVHDUFK
x 6WDWLVWLFVDQGWKHODQJXDJHRIVWDWLVWLFV6HFWLRQGLVFXVVHGWKHVLJQLILFDQFHRI
VWDWLVWLFVLQTXDQWLWDWLYHUHVHDUFK7KLVLVXQGRXEWHGO\WKHPRVWVDOLHQW48$1
IHDWXUH²DVZHZLOOVHHLQ&KDSWHUVWDWLVWLFDODQDO\VHVFDQUDQJHIURPFDOFXODWLQJ
WKHDYHUDJH RUDVLWLVUHIHUUHGWRLQVWDWLVWLFVWKHµPHDQ¶ RIVHYHUDOILJXUHVRQD
SRFNHWFDOFXODWRUWRUXQQLQJFRPSOH[PXOWLYDULDWHDQDO\VHVRQDFRPSXWHU%HFDXVH
RIWKHFORVHOLQNRITXDQWLWDWLYHUHVHDUFKDQGVWDWLVWLFVPXFKRIWKHVWDWLVWLFDO
WHUPLQRORJ\KDVEHFRPHSDUWRIWKH48$1YRFDEXODU\DQGWKHUHVXOWLQJXQLTXH
48$1ODQJXDJHDGGVIXUWKHUSRZHUWRWKHTXDQWLWDWLYHSDUDGLJP1RZRQGHUWKDW
TXDOLWDWLYHUHVHDUFKHUVKDYHJRQHWRJUHDWOHQJWKVWRWU\DQGUHSODFHVRPHRIWKH
48$1UHVHDUFKWHUPLQRORJ\ZLWK48$/VSHFLILFODQJXDJH IRUH[DPSOHZKHQ
GHILQLQJWKH48$/TXDOLW\FULWHULD²VHH6HFWLRQ
x 6WDQGDUGL]HGSURFHGXUHVWRDVVHVVREMHFWLYHUHDOLW\6HFWLRQVDQG
KLJKOLJKWHGWKHJHQHUDO48$1DVSLUDWLRQRIHOLPLQDWLQJDQ\LQGLYLGXDOEDVHG
VXEMHFWLYLW\IURPWKHYDULRXVSKDVHVRIWKHUHVHDUFKSURFHVVE\GHYHORSLQJ
V\VWHPDWLFFDQRQVDQGUXOHVIRUHYHU\IDFHWRIGDWDFROOHFWLRQDQGDQDO\VLV
4XDQWLWDWLYHPHWKRGRORJ\KDVLQGHHGJRQHDORQJZD\WRZDUGVVWDQGDUGL]LQJ
UHVHDUFKSURFHGXUHVWRHQVXUHWKDWWKH\UHPDLQVWDEOHDFURVVLQYHVWLJDWRUVDQG
VXEMHFWV7KLVLQGHSHQGHQFHRILGLRV\QFUDWLFKXPDQYDULDELOLW\DQGELDVKDVEHHQ
HTXDWHGZLWKµREMHFWLYLW\¶E\TXDQWLWDWLYHUHVHDUFKHUVDQGWKHUHVXOWVDFFXPXODWHG
WKURXJKVXFKSURFHGXUHVDUHWKRXJKWWRGHVFULEHWKHREMHFWLYHUHDOLW\WKDWLVµRXW
WKHUH¶LQGHSHQGHQWRIWKHUHVHDUFKHU¶VVXEMHFWLYHSHUFHSWLRQV2IFRXUVHDV
%DFKPDQ D SRLQWVRXWWKLVVWDQFHLVEDVHGRQWKHDVVXPSWLRQWKDWWKHUH
DFWXDOO\H[LVWVDQREMHFWLYHUHDOLW\VRWKDWZKHQGLIIHUHQWUHVHDUFKHUVREVHUYHWKH
VDPHSKHQRPHQRQXVLQJVWDQGDUGL]HGPHDVXUHVWKHLUILQGLQJVZLOOVKRZ
DJUHHPHQWDQGFRQYHUJHQFH
x 4XHVWIRUJHQHUDOL]DELOLW\DQGXQLYHUVDOODZV1XPEHUVYDULDEOHVVWDQGDUGL]HG
SURFHGXUHVVWDWLVWLFVVFLHQWLILFUHDVRQLQJDUHDOOSDUWRIWKHXOWLPDWH48$1TXHVW
IRUIDFWVWKDWDUHJHQHUDOL]DEOHEH\RQGWKHSDUWLFXODUDQGDGGXSWRZLGHUDQJLQJ
LGHDOO\XQLYHUVDOODZV:KHWKHUVXFKODZVDFWXDOO\H[LVWZLWKUHJDUGWRWKHVRFLDO

EHKDYLRXURIKXPDQVDQGLIWKH\GRKRZXQLYHUVDOWKH\DUHDUHIXQGDPHQWDO
LGHRORJLFDOTXHVWLRQVWKDWJREH\RQGWKHVFRSHRIWKLVERRN
6WUHQJWKVDQGZHDNQHVVHVRITXDQWLWDWLYHUHVHDUFK
7KHVWUHQJWKVRITXDQWLWDWLYHUHVHDUFKDUHPDQLIROGDQGPRVWKDYHEHHQGLVFXVVHGLQWKH
SUHYLRXVVHFWLRQV48$1SURSRQHQWVXVXDOO\HPSKDVL]HWKDWDWLWVEHVWWKHTXDQWLWDWLYH
LQTXLU\LVV\VWHPDWLFULJRURXVIRFXVHGDQGWLJKWO\FRQWUROOHGLQYROYLQJSUHFLVH
PHDVXUHPHQWDQGSURGXFLQJUHOLDEOHDQGUHSOLFDEOHGDWDWKDWLVJHQHUDOL]DEOHWRRWKHU
FRQWH[WV7KHVWDWLVWLFDODQDO\WLFDODSSDUDWXVLVUHILQHGDQGIDUUHDFKLQJDQGLWDOVR
RIIHUVVRPHLQEXLOWTXDOLW\FKHFNVDQGLQGLFHV VXFKDVVWDWLVWLFDOVLJQLILFDQFH WKDW
KHOSUHDGHUVWRGHFLGHRQWKHYDOLGLW\RITXDQWLWDWLYHILQGLQJV)URPDSUDFWLFDO
SHUVSHFWLYHHYHQZLWKWKHORQJHUSUHSDUDWLRQSHULRGGLVFXVVHGHDUOLHUWKHUHVHDUFK
SURFHVVLVUHODWLYHO\TXLFNDQGRIIHUVJRRGYDOXHIRUPRQH\SDUWLFXODUO\EHFDXVHWKH
GDWDDQDO\VLVFDQEHGRQHXVLQJVWDWLVWLFDOFRPSXWHUVRIWZDUH)LQDOO\TXDQWLWDWLYH
ILQGLQJVWHQGWRHQMR\DXQLYHUVDOO\KLJKUHSXWDWLRQZLWKDOPRVWDQ\DXGLHQFHRU
VWDNHKROGHUJURXS
7KHGRZQVLGHRITXDQWLWDWLYHPHWKRGVLVWKDWWKH\DYHUDJHRXWUHVSRQVHVDFURVVWKH
ZKROHREVHUYHGJURXSRISDUWLFLSDQWVDQGE\ZRUNLQJZLWKFRQFHSWVRIDYHUDJHVLWLV
LPSRVVLEOHWRGRMXVWLFHWRWKHVXEMHFWLYHYDULHW\RIDQLQGLYLGXDOOLIH6LPLODUVFRUHV
FDQUHVXOWIURPTXLWHGLIIHUHQWXQGHUO\LQJSURFHVVHVDQGTXDQWLWDWLYHPHWKRGVDUH
JHQHUDOO\QRWYHU\VHQVLWLYHLQXQFRYHULQJWKHUHDVRQVIRUSDUWLFXODUREVHUYDWLRQVRUWKH
G\QDPLFVXQGHUO\LQJWKHH[DPLQHGVLWXDWLRQRUSKHQRPHQRQ7KDWLVWKHJHQHUDO
H[SORUDWRU\FDSDFLW\RITXDQWLWDWLYHUHVHDUFKLVUDWKHUOLPLWHG%HFDXVHRIWKHVH
VKRUWFRPLQJVTXDOLWDWLYHUHVHDUFKHUVRIWHQYLHZTXDQWLWDWLYHUHVHDUFKDVµRYHUO\
VLPSOLVWLFGHFRQWH[WXDOL]HGUHGXFWLRQLVWLQWHUPVRILWVJHQHUDOL]DWLRQVDQGIDLOLQJWR
FDSWXUHWKHPHDQLQJVWKDWDFWRUVDWWDFKWRWKHLUOLYHVDQGFLUFXPVWDQFHV¶ %UDQQHQ

4XDOLWDWLYHUHVHDUFK
'HVFULELQJTXDQWLWDWLYHUHVHDUFKKDVEHHQDUHODWLYHO\VWUDLJKWIRUZDUGWDVNEHFDXVH
WKHUHLVDJHQHUDODJUHHPHQWDPRQJVW48$1SUDFWLWLRQHUVDERXWWKHPDLQIHDWXUHVDQG
SULQFLSOHVRIWKHDSSURDFK7KLVLVQRWDWDOOWKHFDVHZLWK48$/UHVHDUFK,QDUHFHQW
RYHUYLHZRIWKHILHOGWZRRIWKHPRVWLQIOXHQWLDOTXDOLWDWLYHUHVHDUFKHUV'HQ]LQDQG
/LQFROQ D FRQFOXGHGWKDWµTXDOLWDWLYHUHVHDUFKLVGLIILFXOWWRGHILQHFOHDUO\,WKDV
QRWKHRU\RUSDUDGLJPWKDWLVGLVWLQFWO\LWVRZQ«1RUGRHVTXDOLWDWLYHUHVHDUFKKDYHD
GLVWLQFWVHWRIPHWKRGVRUSUDFWLFHVWKDWDUHHQWLUHO\LWVRZQ S± $QGODWHUWKH\
DGGHGµ4XDOLWDWLYHUHVHDUFKLVPDQ\WKLQJVWRPDQ\SHRSOH¶ S
'HQ]LQDQG/LQFROQ¶VYLHZLVQRWDWDOOH[DJJHUDWHGDQGLVVKDUHGWKURXJKRXWWKH
SURIHVVLRQ)RUH[DPSOHDQRWKHUZHOONQRZQSURSRQHQWRITXDOLWDWLYHUHVHDUFK
6LOYHUPDQ H[SUHVVHGDVLPLODUFRQFOXVLRQZKHQKHVWDWHGWKDWµWKHUHLVQR
DJUHHGGRFWULQHXQGHUO\LQJDOOTXDOLWDWLYHVRFLDOUHVHDUFK¶ S )XUWKHUPRUH

+ROOLGD\ KDVDGGHGWKDWµERXQGDULHVLQFXUUHQWTXDOLWDWLYHUHVHDUFKDUH
FUXPEOLQJDQGUHVHDUFKHUVDUHLQFUHDVLQJO\GRLQJZKDWHYHUWKH\FDQWRILQGRXWZKDW
WKH\ZDQWWRNQRZ¶$VVHHQHDUOLHUWKHODFNRIXQLIRUPLW\JRHVEDFNWRWKHJHQHVLVRI
WKHTXDOLWDWLYHDSSURDFKZKHQVFKRODUVRIGLYHUVHEHOLHIVXQLWHGXQGHUWKHTXDOLWDWLYH
ODEHOLQWKHLUILJKWDJDLQVWWKHTXDQWLWDWLYHSDUDGLJP
/XFNLO\WKHRYHUDOOSLFWXUHLVQRWDVJORRP\DQGIUDJPHQWHGDVWKHDERYHTXRWHV
ZRXOGVXJJHVW4XDOLWDWLYHUHVHDUFKLVLQIDFWDWKULYLQJGLVFLSOLQHDQGZKLOHLWLVWUXH
WKDWVRPHLVVXHVKDYHEHHQVXEMHFWWRDORWRIDQGVRPHWLPHVKHDWHGGLVFXVVLRQWKHUH
H[LVWVDFRUHVHWRIIHDWXUHVWKDWZRXOGXQLYHUVDOO\FKDUDFWHUL]HDSURSHUO\FRQGXFWHG
TXDOLWDWLYHVWXG\,QWKHQH[WVHFWLRQVZHDUHJRLQJWRORRNDWWKHVHFRUHDWWULEXWHV
5HVHDUFKWKDWFDQEHFRQVLGHUHGµTXDOLWDWLYH¶LQUHWURVSHFWKDVEHHQDURXQGIRUDERXWD
FHQWXU\LQWKHVRFLDOVFLHQFHV4XDOLWDWLYHOLNHPHWKRGVZHUHLQWURGXFHGLQWRVRFLRORJ\
DWWKHHQGRIWKHILUVWGHFDGHRIWKHWZHQWLHWKFHQWXU\WKURXJKWKHZRUNRIWKH&KLFDJR
6FKRROIRUWKHVWXG\RIKXPDQJURXSOLIHDQGGXULQJWKHILUVWWKLUGRIWKHFHQWXU\
DQWKURSRORJ\DOVRSURGXFHGVRPHVHPLQDOTXDOLWDWLYHVWXGLHVE\UHQRZQHGVFKRODUV
VXFKDV%RDVDQG0DOLQRZVNLGHILQLQJWKHRXWOLQHVRIWKHILHOGZRUNPHWKRG 'HQ]LQ
DQG/LQFROQD 7KXVWKHEDVLF48$/LGHDVDQGSULQFLSOHVDUHQRWQHZDWDOO\HW
WKHILUVWWH[WWKDWWULHGWRGHILQHµTXDOLWDWLYHPHWKRGRORJ\¶²*ODVHUDQG6WUDXVV¶V
7KH'LVFRYHU\RI*URXQGHG7KHRU\6WUDWHJLHVIRU4XDOLWDWLYH5HVHDUFK²GLGQRW
DSSHDUXQWLOWKHODWHVL[WLHV,QWKLVKLJKO\LQIOXHQWLDOERRNWKHDXWKRUVGHVFULEHGWKHXVH
RITXDOLWDWLYHSURFHGXUHVE\WKH48$/SLRQHHUVDVµQRQV\VWHPDWLFDQGQRQULJRURXV¶ S
DQGFRQWHQGHGWKDWHDUO\PRQRJUDSKVEDVHGRQTXDOLWDWLYHGDWDFRQVLVWHGRI
µOHQJWK\GHWDLOHGGHVFULSWLRQVZKLFKUHVXOWHGLQYHU\VPDOODPRXQWVRIWKHRU\LIDQ\¶
LELG
$IWHUWKHVDQGSDUWLFXODUO\DIWHU:RUOG:DU,,TXDQWLWDWLYHUHVHDUFK
PHWKRGRORJ\SURGXFHGVXEVWDQWLDODGYDQFHV VHH6HFWLRQ DQGTXDOLWDWLYHUHVHDUFK
ZDVUHOHJDWHGWRSUHOLPLQDU\H[SORUDWRU\ZRUNZKRVHUROHZDVVHHQWRSURYLGHWKH
µPRUHVHULRXV¶TXDQWLWDWLYHVWXGLHVZLWKDQDGHTXDWHVWDUWLQJSRLQW7KHPLGGOHRIWKH
WKFHQWXU\ZDVXQGRXEWHGO\GRPLQDWHGE\TXDQWLWDWLYHUHVHDUFKDQGWKHLQYDOXDEOH
PHULWRI*ODVHUDQG6WUDXVV¶V ERRNZDVWRRIIHUDYLDEOHFKDOOHQJHWRWKLV
KHJHPRQ\7KHVHDXWKRUVZHUHH[SOLFLWO\FRQFHUQHGZLWKWKHµV\VWHPDWL]DWLRQRIWKH
FROOHFWLRQFRGLQJDQGDQDO\VLVRITXDOLWDWLYHGDWDIRUWKHJHQHUDWLRQRIWKHRU\¶ S
VHHDOVRWKHGLVFXVVLRQRIµJURXQGHGWKHRU\¶LQ6HFWLRQ DQGIRUWKHILUVWWLPH
TXDOLWDWLYHO\LQFOLQHGUHVHDUFKHUVKDGKDGDQHODERUDWHWKHRUHWLFDOO\EDVHGPHWKRGRORJ\
DYDLODEOHWRWKHP4XDOLWDWLYHUHVHDUFKZDVRQWKHPDUFK
5HFHQW\HDUVKDYHVHHQDQH[SORVLRQRIWH[WVRQTXDOLWDWLYHPHWKRGVUHIOHFWLQJD
JURZLQJLQWHUHVWLQWKHDSSURDFKDFURVVDOOWKHGLVFLSOLQHVRIWKHVRFLDOVFLHQFHV6HDOH
HWDO IRUH[DPSOHH[DPLQHGWKHRXWSXWRIWKHPDLQSXEOLVKHURIPHWKRGRORJ\

WH[WV6DJH3XEOLFDWLRQVDQGIRXQGWKDWGXULQJWKHODVWGHFDGHWKHUHZDVDIRXUIROG
LQFUHDVHRISXEOLVKHGTXDOLWDWLYHPHWKRGVWH[WERRNV 1
,QDSSOLHGOLQJXLVWLFVWKHUHKDVEHHQDQLQFUHDVLQJYLVLELOLW\DQGDFFHSWDQFHRI
TXDOLWDWLYHUHVHDUFKVLQFHWKHPLGV 'XIILQSUHVV 7KLVLVUHODWHGWRWKHJURZLQJ
UHFRJQLWLRQWKDWDOPRVWHYHU\DVSHFWRIODQJXDJHDFTXLVLWLRQDQGXVHLVGHWHUPLQHGRU
VLJQLILFDQWO\VKDSHGE\VRFLDOFXOWXUDODQGVLWXDWLRQDOIDFWRUVDQGTXDOLWDWLYHUHVHDUFK
LVLGHDOIRUSURYLGLQJLQVLJKWVLQWRVXFKFRQWH[WXDOFRQGLWLRQVDQGLQIOXHQFHV
$FFRUGLQJO\DSSOLHGOLQJXLVWLFVKDVEHHQRIIHULQJDQLQFUHDVLQJO\OHYHOSOD\LQJILHOG
IRUERWK48$1DQG48$/DSSURDFKHV+DYLQJVDLGWKDWZHPXVWDOVRQRWHDVHULRXV
FRQFHUQLQWKLVUHVSHFWKLJKOLJKWHGE\/D]DUDWRQ QDPHO\WKDWWKHUHKDYHEHHQ
WRRIHZTXDOLWDWLYHVWXGLHVSXEOLVKHGLQWKHOHDGLQJDSSOLHGOLQJXLVWLFVMRXUQDOVZLWKWKH
SRVVLEOHH[FHSWLRQRI7(62/4XDUWHUO\)RUH[DPSOHWKHHGLWRURI7KH0RGHUQ
/DQJXDJH-RXUQDO6DOO\0DJQDQ UHSRUWHGWKDWDOWKRXJKWKHUHKDGEHHQDQ
µLQFUHDVHLQHWKQRJUDSKLFDQGFDVHVWXGLHVVXEPLWWHGIRUFRQVLGHUDWLRQWRWKHSRLQWWKDW
WKHLUQXPEHUVZHUHEHJLQQLQJWRDSSURDFKWKRVHRITXDQWLWDWLYHSLHFHV¶GXULQJWKH
±SHULRGRQO\SHUFHQWRIWKHUHVHDUFKDUWLFOHVLQKHUMRXUQDOZHUH
TXDOLWDWLYH DQGSHUFHQWXVHGDPL[HGPHWKRGRORJ\ 0DJQDQ ,WZRXOGEH
LQWHUHVWLQJWRFDUU\RXWDV\VWHPDWLFDQDO\VLVRIWKHUHDVRQVIRUWKHGLVFUHSDQF\EHWZHHQ
WKHVXEPLVVLRQDQGWKHSXEOLFDWLRQUDWHV
$OWKRXJKWKHIUHTXHQF\RISXEOLVKHG48$/VWXGLHVLVVWLOOUHODWLYHO\ORZWKHLPSDFW
RITXDOLWDWLYHUHVHDUFKLQDSSOLHGOLQJXLVWLFVRYHUWKHSDVWWKUHHGHFDGHVKDVEHHQ
SURIRXQG(DUO\FDVHVWXGLHVRIWKHVDQGVKDGDJURXQGEUHDNLQJHIIHFWRQRXU
XQGHUVWDQGLQJRI6/$DQGJHQHUDWHGPDQ\RIWKHSUHYDLOLQJSULQFLSOHVDQGPRGHOV
6HH6HFWLRQ :LWKUHJDUGWRFRQWHPSRUDU\UHVHDUFKZHILQGTXDOLWDWLYHVWXGLHV
IRFXVLQJRQWRSLFVDFURVVWKHZKROHUHVHDUFKVSHFWUXPHYHQLQFOXGLQJFRUHTXDQWLWDWLYH
DUHDVVXFKDVODQJXDJHWHVWLQJDQGVHYHUDONH\DUHDVRIDSSOLHGOLQJXLVWLFV IRU
H[DPSOHWKHVWXG\RIJHQGHUUDFHHWKQLFLW\DQGLGHQWLW\ DUHEHLQJGULYHQE\
TXDOLWDWLYHUHVHDUFK )RUDQRYHUYLHZRITXDOLWDWLYHLQTXLU\LQDSSOLHGOLQJXLVWLFVVHH
5LFKDUGV
0DLQFKDUDFWHULVWLFVRITXDOLWDWLYHUHVHDUFK
7KHUHVHDUFKPHWKRGRORJ\OLWHUDWXUHFRQWDLQVVHYHUDOGHWDLOHGVXPPDULHVRIWKHFRUH
IHDWXUHVRITXDOLWDWLYHLQTXLU\0DQ\RIWKHSRLQWVLQWKHGLIIHUHQWOLVWVRYHUODSEXWDV
PHQWLRQHGHDUOLHUWKHUHDUHDOVRVRPHFRQWHQWLRXVLVVXHV/HWXVORRNDWWKHPRVWRIWHQ
PHQWLRQHGDVSHFWV
x (PHUJHQWUHVHDUFKGHVLJQ,QGHVFULELQJWKHPDLQFKDUDFWHULVWLFVRITXDOLWDWLYH
UHVHDUFKPRVWUHVHDUFKWH[WVVWDUWZLWKKLJKOLJKWLQJLWVHPHUJHQWQDWXUH7KLV
PHDQVWKDWQRDVSHFWRIWKHUHVHDUFKGHVLJQLVWLJKWO\SUHILJXUHGDQGDVWXG\LVNHSW
RSHQDQGIOXLGVRWKDWLWFDQUHVSRQGLQDIOH[LEOHZD\WRQHZGHWDLOVRURSHQLQJV
WKDWPD\HPHUJHGXULQJWKHSURFHVVRILQYHVWLJDWLRQ7KLVIOH[LELOLW\HYHQDSSOLHVWR
WKHUHVHDUFKTXHVWLRQVZKLFKPD\HYROYHFKDQJHRUEHUHILQHGGXULQJWKHVWXG\²

VHH6HFWLRQ$QLPSRUWDQWDVSHFWRIWKLVHPHUJHQWQDWXUHLVWKHIDFWWKDW
LGHDOO\TXDOLWDWLYHUHVHDUFKHUVHQWHUWKHUHVHDUFKSURFHVVZLWKDFRPSOHWHO\RSHQ
PLQGDQGZLWKRXWVHWWLQJRXWWRWHVWSUHFRQFHLYHGK\SRWKHVHV7KLVPHDQVWKDWWKH
UHVHDUFKIRFXVLVQDUURZHGGRZQRQO\JUDGXDOO\DQGWKHDQDO\WLF
FDWHJRULHVFRQFHSWVDUHGHILQHGGXULQJUDWKHUWKDQSULRUWRWKHSURFHVVRIWKH
UHVHDUFK)RUH[DPSOHLQWKHLUVHPLQDOZRUN*ODVHUDQG6WUDXVV DFWLYHO\
HQFRXUDJHGTXDOLWDWLYHUHVHDUFKHUVWRLJQRUHWKHOLWHUDWXUHEHIRUHWKHLQYHVWLJDWLRQ
LQRUGHUWRDVVXUHWKDWµWKHHPHUJHQFHRIFDWHJRULHVZLOOQRWEHFRQWDPLQDWHGE\
FRQFHSWVPRUHVXLWHGWRGLIIHUHQWDUHDV¶ S 7KLVLVDKLJKO\FRQWURYHUVLDOLVVXH
DQGIRUWKLVUHDVRQZHFRPHEDFNWRLWLQDVHSDUDWHVHFWLRQEHORZ
x 7KHQDWXUHRITXDOLWDWLYHGDWD4XDOLWDWLYHUHVHDUFKZRUNVZLWKDZLGHUDQJHRIGDWD
LQFOXGLQJUHFRUGHGLQWHUYLHZVYDULRXVW\SHVRIWH[WV IRUH[DPSOHILHOGQRWHV
MRXUQDODQGGLDU\HQWULHVGRFXPHQWV DQGLPDJHV SKRWRVRUYLGHRV 'XULQJGDWD
SURFHVVLQJPRVWGDWDLVWUDQVIRUPHGLQWRDWH[WXDOIRUP IRUH[DPSOHLQWHUYLHZ
UHFRUGLQJVDUHWUDQVFULEHG EHFDXVHPRVWTXDOLWDWLYHGDWDDQDO\VLVLVGRQHZLWK
ZRUGV²VHH6HFWLRQ$OWKRXJKTXDOLWDWLYHGDWDLVQRWJDWKHUHGZLWKWKH
SXUSRVHRIEHLQJGLUHFWO\FRXQWHGRUPHDVXUHGLQDQREMHFWLYHZD\VXEVHTXHQW
DQDO\VLVFDQGHILQHFDWHJRULHVWKURXJKZKLFKFHUWDLQDVSHFWVRITXDOLWDWLYHGDWDFDQ
EHTXDQWLILHG²VHH6HFWLRQ%HFDXVHWKHFRPPRQREMHFWLYHRIDOOWKH
GLIIHUHQWW\SHVRITXDOLWDWLYHPHWKRGVLVWRPDNHVHQVHRIDVHWRI FXOWXUDORU
SHUVRQDO PHDQLQJVLQWKHREVHUYHGSKHQRPHQDLWLVLQGLVSHQVDEOHWKDWWKHGDWD
VKRXOGFDSWXUHULFKDQGFRPSOH[GHWDLOV7KHUHIRUHLQRUGHUWRDFKLHYHVXFKD
µWKLFN¶GHVFULSWLRQDOPRVWDQ\UHOHYDQWLQIRUPDWLRQFDQEHDGPLWWHGDV48$/GDWD
x 7KHFKDUDFWHULVWLFVRIWKHUHVHDUFKVHWWLQJ%HFDXVHRIWKH48$/REMHFWLYHWR
GHVFULEHVRFLDOSKHQRPHQDDVWKH\RFFXUQDWXUDOO\TXDOLWDWLYHUHVHDUFKWDNHVSODFH
LQWKHQDWXUDOVHWWLQJZLWKRXWDQ\DWWHPSWVWRPDQLSXODWHWKHVLWXDWLRQXQGHUVWXG\
,QRUGHUWRFDSWXUHDVXIILFLHQWOHYHORIGHWDLODERXWWKHQDWXUDOFRQWH[WVXFK
LQYHVWLJDWLRQVDUHXVXDOO\FRQGXFWHGWKURXJKDQLQWHQVHDQGSURORQJHGFRQWDFW
ZLWKRULPPHUVLRQLQWKHUHVHDUFKVHWWLQJ
x ,QVLGHUPHDQLQJ4XDOLWDWLYHUHVHDUFKLVFRQFHUQHGZLWKVXEMHFWLYHRSLQLRQV
H[SHULHQFHVDQGIHHOLQJVRILQGLYLGXDOVDQGWKXVWKHH[SOLFLWJRDORIUHVHDUFKLVWR
H[SORUHWKHSDUWLFLSDQWV¶YLHZVRIWKHVLWXDWLRQEHLQJVWXGLHG7KLVDSSURDFK
IROORZVIURPWKHZD\TXDOLWDWLYHUHVHDUFKHUVSHUFHLYHPHDQLQJLWLVDIXQGDPHQWDO
48$/SULQFLSOHWKDWKXPDQEHKDYLRXULVEDVHGXSRQPHDQLQJVZKLFKSHRSOH
DWWULEXWHWRDQGEULQJWRVLWXDWLRQV 3XQFK DQGLWLVRQO\WKHDFWXDO
SDUWLFLSDQWVWKHPVHOYHVZKRFDQUHYHDOWKHPHDQLQJVDQGLQWHUSUHWDWLRQVRIWKHLU
H[SHULHQFHVDQGDFWLRQV7KHUHIRUHTXDOLWDWLYHUHVHDUFKHUVVWULYHWRYLHZVRFLDO
SKHQRPHQDIURPWKHSHUVSHFWLYHVRIWKHµLQVLGHUV¶DQGWKHWHUPµLQVLGHU
SHUVSHFWLYH¶KDVDVSHFLDOSODFHLQWKHTXDOLWDWLYHFUHGR
x 6PDOOVDPSOHVL]H:HOOFRQGXFWHGTXDOLWDWLYHUHVHDUFKLVYHU\ODERXULQWHQVLYHDQG
WKHUHIRUHTXDOLWDWLYHVWXGLHVW\SLFDOO\XVHRIQHFHVVLW\PXFKVPDOOHUVDPSOHVRI

SDUWLFLSDQWVWKDQTXDQWLWDWLYHRQHV:HZLOOFRPHEDFNWRWKHTXHVWLRQRI
TXDOLWDWLYHVDPSOLQJLQ&KDSWHU 6HFWLRQ
x ,QWHUSUHWLYHDQDO\VLV4XDOLWDWLYHUHVHDUFKLVIXQGDPHQWDOO\LQWHUSUHWLYHZKLFK
PHDQVWKDWWKHUHVHDUFKRXWFRPHLVXOWLPDWHO\WKHSURGXFWRIWKHUHVHDUFKHU¶V
VXEMHFWLYHLQWHUSUHWDWLRQRIWKHGDWD6HYHUDODOWHUQDWLYHLQWHUSUHWDWLRQVDUHSRVVLEOH
IRUHDFKGDWDVHWDQGEHFDXVH48$/VWXGLHVXWLOL]HUHODWLYHO\OLPLWHGVWDQGDUGL]HG
LQVWUXPHQWDWLRQRUDQDO\WLFDOSURFHGXUHVLQWKHHQGLWLVWKHUHVHDUFKHUZKRZLOO
FKRRVHIURPWKHP$V0LOHVDQG+XEHUPDQ FRQFOXGHµ7KHUHVHDUFKHULV
HVVHQWLDOO\WKHPDLQ³PHDVXUHPHQWGHYLFH´LQWKHVWXG\¶$FFRUGLQJO\LQ
TXDOLWDWLYHUHVHDUFKWKHUHVHDUFKHU¶VRZQYDOXHVSHUVRQDOKLVWRU\DQGµSRVLWLRQ¶RQ
FKDUDFWHULVWLFVVXFKDVJHQGHUFXOWXUHFODVVDQGDJHEHFRPHLQWHJUDOSDUWRIWKH
LQTXLU\ +DYHUNDPS
7KHHPHUJHQWQRQHPHUJHQWGHEDWH
2QHRIWKHPRVWFRQWHQWLRXVLVVXHVDPRQJVWTXDOLWDWLYHUHVHDUFKHUVFRQFHUQVWKH
TXHVWLRQDVWRZKHWKHULQYHVWLJDWRUVQHHGWRHQWHUD48$/SURMHFWZLWKRQO\PLQLPDO
EDFNJURXQGNQRZOHGJHVRDVQRWWRµFRQWDPLQDWH¶WKHHPHUJHQWQDWXUHRIWKHVWXG\$V
TXRWHGHDUOLHU*ODVHUDQG6WUDXVV ZHUHH[SOLFLWDERXWWKLVUHTXLUHPHQWDQGLWKDV
EHFRPHRQHRIWKHPDLQWHQHWVRIWKHTXDOLWDWLYHLQTXLU\WKDWWKHUHVXOWVµHPHUJH¶
QDWXUDOO\ZLWKRXWDQ\ELDVHGLQWHUIHUHQFHRIWKHUHVHDUFKHU7KHUHVHDUFKHUWKHUHIRUH
QHHGVWRDGRSWDµWDEXODUDVD¶RULHQWDWLRQDQG*ODVHUDQG6WUDXVVSURSRVHGWKDWWKH
UHVHDUFKHU¶VµWKHRUHWLFDOVHQVLWLYLW\¶LVRQO\WRDSSHDUZKHQWKHGDWDKDVDOUHDG\EHHQ
FROOHFWHGDQGSDUWLDOO\DQDO\VHGVRWKDWWKHFRQFHSWVDQGK\SRWKHVHVWKDWKDYHHPHUJHG
IURPWKHGDWDFDQEHFRPELQHGZLWKH[LVWLQJNQRZOHGJH
6HYHUDOVFKRODUVKDYHTXHVWLRQHGWKHUHDOLW\RIWKLVSUHUHTXLVLWH7DVKDNNRULDQG
7HGGOLH E IRUH[DPSOHSRLQWHGRXWWKDWIHZVRFLDOVFLHQWLVWVZRXOGFRQGXFW
XQSODQQHGDQGXQVWUXFWXUHGUHVHDUFKZLWKQRRULHQWDWLRQRUSRLQWRIYLHZWRJXLGH
WKHP7RWKHFRQWUDU\PRVWHVWDEOLVKHGVRFLDOUHVHDUFKHUVKDYHH[WHQVLYHEDFNJURXQGV
LQWKHDUHDVWKDWWKH\DUHVWXG\LQJDQGWKHUHIRUHWKHµWDEXODUDVD¶DVVXPSWLRQLVVLPSO\
µQDwYH¶ S 0LOHVDQG+XEHUPDQ JRRQHVWHSIXUWKHUZKHQWKH\FODLPWKDWLW
LVWKHUHVHDUFKHUV¶EDFNJURXQGNQRZOHGJHWKDWKHOSVWKHPWRVHHDQGGHFLSKHUGHWDLOV
FRPSOH[LWLHVDQGVXEWOHWLHVDVZHOODVWRGHFLGHZKDWNLQGRITXHVWLRQVWRDVNRUZKLFK
LQFLGHQWVWRDWWHQGWRFORVHO\$VWKH\FRQFOXGHQRWWREHOHGE\RQH¶VFRQFHSWXDO
VWUHQJWKVFDQEHVHOIGHIHDWLQJZKLFKLVLQFRQWUDVWZLWK*ODVHUDQG6WUDXVV¶V
ZDUQLQJWKDWLIVFKRODUVFRPPLWWKHPVHOYHVH[FOXVLYHO\WRRQHVSHFLILFSUHFRQFHLYHG
WKHRU\WKH\EHFRPHµGRFWULQDLUH¶
6WUHQJWKVDQGZHDNQHVVHVRITXDOLWDWLYHUHVHDUFK
7KHUHLVQRVKRUWDJHLQWKHOLWHUDWXUHRIHODERUDWHGHVFULSWLRQVRIWKHPHULWVRI
TXDOLWDWLYHUHVHDUFKDQG,KDYHIRXQGWKHIROORZLQJSRLQWVSDUWLFXODUO\VLJQLILFDQW

x ([SORUDWRU\QDWXUH4XDOLWDWLYHUHVHDUFKKDVWUDGLWLRQDOO\EHHQVHHQDVDQHIIHFWLYH
ZD\RIH[SORULQJQHZXQFKDUWHGDUHDV,IYHU\OLWWOHLVNQRZQDERXWDSKHQRPHQRQ
WKHGHWDLOHGVWXG\RIDIHZFDVHVLVSDUWLFXODUO\DSSURSULDWHEHFDXVHLWGRHVQRWUHO\
RQSUHYLRXVOLWHUDWXUHRUSULRUHPSLULFDOILQGLQJV (LVHQKDUGW
x 0DNLQJVHQVHRIFRPSOH[LW\4XDOLWDWLYHPHWKRGVDUHXVHIXOIRUPDNLQJVHQVHRI
KLJKO\FRPSOH[VLWXDWLRQV,QVXFKFDVHVWKHUHLVDUHDOGDQJHUIRUUHVHDUFKHUVLQ
JHQHUDOWRSURGXFHUHGXFHGDQGVLPSOLILHGLQWHUSUHWDWLRQVWKDWGLVWRUWWKHELJJHU
SLFWXUHWKHSDUWLFLSDQWVHQVLWLYLW\RITXDOLWDWLYHUHVHDUFKLVYHU\KHOSIXOLQGHFLGLQJ
ZKDWDVSHFWVRIWKHGDWDUHTXLUHVSHFLDODWWHQWLRQEHFDXVHLWRIIHUVSULRULW\
JXLGHOLQHVWKDWDUHYDOLGDWHGE\WKHPDLQDFWRUVWKHPVHOYHV7KDWLVWKH
JURXQGHGQHVVRITXDOLWDWLYHUHVHDUFKKHOSVWRGLVWLQJXLVKUHDOSKHQRPHQDIURP
LQWHOOHFWXDOIDEULFDWLRQV
x $QVZHULQJµZK\¶TXHVWLRQV,WLVQRWXQFRPPRQLQTXDQWLWDWLYHVWXGLHVWRREWDLQ
VXUSULVLQJRUFRQWUDGLFWRU\UHVXOWVEXWLQPDQ\RIWKHVHFDVHVWKHFROOHFWHGGDWD
GRHVQRWRIIHUDQ\UHDOHQOLJKWHQPHQWDVWRWKHFDXVHV7KLVLVZKHQUHVHDUFKHUV
LQFOXGHDWWKHHQGRIWKHUHVHDUFKUHSRUWWKHZHOONQRZQVWDWHPHQWµIXUWKHUUHVHDUFK
LVQHHGHGWRXQGHUVWDQG«¶ PHDQLQJµZHKDYHQRLGHDZK\WKLVKDVRFFXUUHG«¶
,QFRQWUDVWWKHIOH[LEOHHPHUJHQWQDWXUHRIDTXDOLWDWLYHVWXG\DOORZVWKH
UHVHDUFKHUWRFRQGXFWWKHµIXUWKHUUHVHDUFK¶VWUDLJKWDZD\WKHUHE\UHDFKLQJDIXOOHU
XQGHUVWDQGLQJ
x %URDGHQLQJRXUXQGHUVWDQGLQJ7DONLQJDERXWWKHLVVXHRIJHQHUDOL]DELOLW\LQ
TXDOLWDWLYHUHVHDUFK'XII LQSUHVV HPSKDVL]HVWKDWLQVWHDGRIVHHNLQJD
JHQHUDOL]DEOHµFRUUHFWLQWHUSUHWDWLRQ¶TXDOLWDWLYHUHVHDUFKDLPVWREURDGHQWKH
UHSHUWRLUHRISRVVLEOHLQWHUSUHWDWLRQVRIKXPDQH[SHULHQFH7KXVWKHULFKGDWD
REWDLQHGDERXWWKHSDUWLFLSDQWV¶H[SHULHQFHFDQZLGHQWKHVFRSHRIRXU
XQGHUVWDQGLQJDQGFDQDGGGDWDGULYHQ UDWKHUWKDQVSHFXODWLYH GHSWKWRWKH
DQDO\VLVRIDSKHQRPHQRQ
x /RQJLWXGLQDOH[DPLQDWLRQRIG\QDPLFSKHQRPHQD,QWHUHVWLQJO\RQHRIWKHPDLQ
UHDVRQVZK\,KDYHFRPHWRDSSUHFLDWHDQGXVHTXDOLWDWLYHUHVHDUFKLVUDUHO\
PHQWLRQHGLQWKHOLWHUDWXUH,KDYHIRXQGWKDWTXDOLWDWLYHUHVHDUFKLVSDUWLFXODUO\
XVHIXOIRUWKHSXUSRVHRIORQJLWXGLQDOUHVHDUFK$VDUJXHGLQ&KDSWHU 6HFWLRQ
PRVWRIWKHSURFHVVHVVWXGLHGE\DSSOLHGOLQJXLVWVDUHG\QDPLFLQQDWXUHDQG
WKHUHIRUHZHZRXOGQHHGPDQ\PRUHORQJLWXGLQDOLQYHVWLJDWLRQVLQWKHILHOGWR
H[SORUHWKHVHTXHQWLDOSDWWHUQVDQGWKHFKDQJHVWKDWRFFXU4XDOLWDWLYHUHVHDUFK
RIIHUVDJRRGVWDUWLQJSRLQWLQWKLVUHVSHFW
x )OH[LELOLW\ZKHQWKLQJVJRZURQJ7KHUHLVDKRVWRIWKLQJVWKDWFDQJRZURQJZKLOH
GRLQJUHVHDUFKLQWKHILHOGSDUWLFXODUO\LIWKHUHVHDUFKVLWHLVZLWKLQDQHGXFDWLRQDO
LQVWLWXWLRQ 6HH6HFWLRQIRUWKHYDULRXVFKDOOHQJHVRIFODVVURRPUHVHDUFK ,IZH
XVHDSXUHO\TXDQWLWDWLYHUHVHDUFKGHVLJQVRPHRIWKHXQH[SHFWHGHYHQWVFDQUHQGHU
RXUVWXG\PHDQLQJOHVVZKHUHDVTXDOLWDWLYHPHWKRGVQRWRQO\DOORZXVWR

DFFRPPRGDWHWKHFKDQJHVEXWFDQDOVRHQDEOHXVWRFDSLWDOL]HRQWKHPDQGSURGXFH
H[FLWLQJUHVXOWV*KHUDUGLDQG7XUQHU UHSRUWRQDVWXG\E\/RZHZKLFK
H[DPLQHGDVHULHVRI48$1LQYHVWLJDWLRQVFDUULHGRXWE\YDULRXVGLVWLQJXLVKHG
VFKRODUV,WZDVIRXQGWKDWZLWKRQHH[FHSWLRQDOOWKHSURMHFWVUHDFKHGDSRLQWRI
GLVUXSWLRQZKHQWKHRULJLQDOSODQEURNHGRZQUHTXLULQJDFWLYLWLHVRIWKHRUHWLFDO
µSDWFKZRUNLQJ¶LQRUGHUWRUHSDLUWKHEUHDNGRZQDQGWRSUHVHQWDQDSSHDUDQFHRI
FRKHUHQFHLQWKHZRUN*KHUDUGLDQG7XUQHUFRQFOXGHGWKDWLIUHVHDUFKLV
UHFRJQL]HGWREHDMRXUQH\LQWRWKHXQNQRZQ LH48$/ UDWKHUWKDQDWDVNZKLFK
FDQEHIXOO\VSHFLILHGDQGSODQQHGLQDGYDQFH LH48$1 WKHQVXFKEUHDNGRZQV
ORRNOHVVVXUSULVLQJDQGFDQEHKDQGOHGZLWKLQWKHUHVHDUFKIUDPHZRUN
x 5LFKPDWHULDOIRUWKHUHVHDUFKUHSRUW2QHGLVKHDUWHQLQJDVSHFWRITXDQWLWDWLYH
VWXGLHVFDQEHZKHQWKHUHVXOWVRIVHYHUDOPRQWKVRIKDUGODERXUDUHVXPPDUL]HGLQ
RQHRUWZRVPDOOWDEOHV RIFRUUHODWLRQVIRUH[DPSOH *LOOKDP GHVFULEHV
WKLVVLWXDWLRQZHOOµ,IWKHEDVLFUHVHDUFKTXHVWLRQVDUHFRPSOH[ ZKHQDUHWKH\
QRW" WKHQ\RXUGDWDDUHJRLQJWRORRNSUHWW\WKLQDQGVXSHUILFLDO¶ S ,Q
FRQWUDVWTXDOLWDWLYHDFFRXQWVWKDWXVHWKHZRUGVDQGFDWHJRULHVRIWKHSDUWLFLSDQWV
PDNHLWPXFKHDVLHUWRSURGXFHDFRQYLQFLQJDQGYLYLGFDVHIRUDZLGHUDQJHRI
DXGLHQFHV
:HDNQHVVHV
,QWKHOLWHUDWXUHZHXVXDOO\ILQGWZRW\SHVRIFULWLFLVPVRITXDOLWDWLYHUHVHDUFK7KHILUVW
FRQVLVWVRITXDQWLWDWLYHO\PRWLYDWHGFRPSODLQWVDERXWFHUWDLQDVSHFWVRITXDOLWDWLYH
UHVHDUFKWKDWDUHGLIIHUHQWIURPTXDQWLWDWLYHUHVHDUFKEXWZKLFKTXDOLWDWLYHUHVHDUFKHUV
FRQVLGHUHLWKHUDVWUHQJWKRUDQRUPDOIHDWXUH7KHVHFRQGFRQWDLQVLVVXHVUDLVHGE\
TXDOLWDWLYHUHVHDUFKHUVWKHPVHOYHV/HWXVORRNDWILYHSDUWLFXODUO\VDOLHQWLVVXHV
x 6DPSOHVL]HDQGJHQHUDOL]DELOLW\7KHPRVWIUHTXHQWFULWLFLVPRIIHUHGE\
TXDQWLWDWLYHO\PLQGHGUHVHDUFKHUVFRQFHUQVWKHLGLRV\QFUDWLFQDWXUHRIWKHVPDOO
SDUWLFLSDQWVDPSOHVWKDWPRVWTXDOLWDWLYHVWXGLHVLQYHVWLJDWH7KLVTXHVWLRQZDV
DOUHDG\GLVFXVVHGLQ6HFWLRQZKHUH,DUJXHGWKDWWKHWZRSDUDGLJPVDSSURDFK
WKHTXHVWLRQRIJHQHUDOL]DELOLW\GLIIHUHQWO\+RZHYHUHYHQLIZHDFFHSWWKDWWKH
H[SORUDWLRQRISHUVRQDOPHDQLQJGRHVQRWUHTXLUHODUJHVDPSOHV'XII ZDUQV
XVWKDWDOWKRXJKWKHFRPPRQ48$/SUDFWLFHRIH[DPLQLQJµWHOOLQJ¶FDVHVPD\EH
YHU\KHOSIXOLQSURYLGLQJLQVLJKWVLQWRDSKHQRPHQRQWKHVSHFLILFFRQGLWLRQVRU
LQVLJKWVPD\QRWDSSO\EURDGO\WRRWKHUV<DWHV FDOOVWKLVLVVXHWKH
µSRWHQWLDORYHUUHDGLQJ¶RIWKHLQGLYLGXDOVWRULHV
x 5HVHDUFKHUUROH$QRWKHUFRQWHVWHGLVVXHFRQFHUQVWKHUROHSOD\HGE\WKHUHVHDUFKHU
LQDQDO\VLQJWKHGDWD$V0LOHVDQG+XEHUPDQ SXWLWµ7KHVWUHQJWKVRI
TXDOLWDWLYHGDWDUHVWYHU\FHQWUDOO\RQWKHFRPSHWHQFHZLWKZKLFKWKHLUDQDO\VLVLV
FDUULHGRXW¶4XDQWLWDWLYHUHVHDUFKHUVZRXOGOLNHWRVHHVRPHILUPVDIHJXDUGVWR
PDNHVXUHWKDWUHVXOWVDUHQRWLQIOXHQFHGE\WKHUHVHDUFKHU¶VSHUVRQDOELDVHVDQG
LGLRV\QFUDVLHV )RUPRUHGHWDLOVVHH6HFWLRQRQTXDOLW\FULWHULD

x /DFNRIPHWKRGRORJLFDOULJRXU)RUTXDQWLWDWLYHUHVHDUFKHUVZKRDUHXVHGWR
VWDQGDUGL]HGLQVWUXPHQWVDQGSURFHGXUHVDQGVWDWLVWLFDODQDO\WLFDOWHFKQLTXHV
TXDOLWDWLYHUHVHDUFKFDQHDVLO\DSSHDUXQSULQFLSOHGDQGµIX]]\¶,WLVQRWHZRUWK\
WKDWVLPLODUSRLQWVDUHDOVRPDGHZLWKLQWKHTXDOLWDWLYHFDPS)RUH[DPSOHDJURXS
RIUHVSHFWHGTXDOLWDWLYHVFKRODUV6HDOHHWDO DUJXHGDJDLQVWWKH
SRVWPRGHUQSRVLWLRQZLWKLQWKH48$/PRYHPHQWDVIROORZV
7KHVHDSSHDUWREHGULYHQE\DQDQWLPHWKRGRORJLFDOWHQRUWKDWSUHIHUVWKH
VXEVWDQFH UHVHDUFKWRSLFV WRWKHIRUP PHWKRGRORJ\ 6XFKDSHUVSHFWLYH
ERUQSDUWO\LQUHDFWLRQWRSRVLWLYLVPZDYHGDIODJRIWKHVXSHULRULW\RI
TXDOLWDWLYHUHVHDUFKWRVXUYH\VDQGH[SHULPHQWVDQGFRQVLGHUHG
PHWKRGRORJLFDOSULQFLSOHVLQFDSDEOHRIDFKLHYLQJDGHHSHUXQGHUVWDQGLQJRI
DIUDJPHQWHGDQGGLVORFDWHGFXOWXUH+RZHYHUWKLVUHVHDUFKVW\OHKDVQRW
DOZD\VPDLQWDLQHGLWVSURPLVHRIDFKLHYLQJDGHHSHUNLQGRIUHVHDUFK7KH
FRQVHTXHQFHVDUHWRRRIWHQH[SRVHGWRYLHZORZTXDOLW\TXDOLWDWLYH
UHVHDUFKDQGUHVHDUFKUHVXOWVWKDWDUHTXLWHVWHUHRW\SLFDODQGFORVHWR
FRPPRQVHQVH
:HPXVWQRWHWKRXJKWKDWWKHVHFRPPHQWVDERXWDODFNRIPHWKRGRORJLFDOULJRXU
RQO\DSSO\WRVRPHTXDOLWDWLYHVWUDQGVEHFDXVHWKHSDVWWZRGHFDGHVKDYHVHHQD
PDUNHGVKLIWWRZDUGVDSSO\LQJULJRURXVSURFHGXUHVLQ48$/VWXGLHV
x 7RRFRPSOH[RUWRRQDUURZWKHRULHV%HFDXVHTXDOLWDWLYHUHVHDUFKHUVKDYHQRUHDO
PHDQVRIDVVHVVLQJZKLFKRIWKHLUILQGLQJVDUHRIPRUHJHQHUDOLPSRUWDQFHDQG
ZKLFKDUHVLPSO\LGLRV\QFUDWLFWRDSDUWLFXODUFDVHHYHQ48$/VFKRODUV IRU
H[DPSOH(LVHQKDUGW SRLQWRXWWKDWWKHUHLVDUHDOGDQJHURIEXLOGLQJWRR
QDUURZWKHRULHVIURPWKHLQGLYLGXDOFDVHVVWXGLHG,QDVLPLODUZD\WKHLQWHQVLYH
XVHRIULFKGDWDFDQDOVR\LHOGDWKHRU\ZKLFKLVRYHUO\FRPSOH[
x 7LPHFRQVXPLQJDQGODERXULQWHQVLYH$ILQDOSRLQWZKLFKERWK48$/DQG48$1
VFKRODUVZRXOGDJUHHRQLVWKDW48$/UHVHDUFKSDUWLFXODUO\WKHSURFHVVLQJRI
48$/GDWDFDQEHUDWKHUWLPHFRQVXPLQJPRUHVRWKDQ48$1UHVHDUFK²DV
PHQWLRQHGHDUOLHULWLVSDUWO\WKLVIHDWXUHZKLFKH[SODLQVWKHUHODWLYHO\VPDOOVDPSOH
VL]HVXVHGLQ48$/LQTXLULHV
0L[HGPHWKRGVUHVHDUFK
5HVHDUFKHUVKDYHEHHQUHIHUULQJWRVWXGLHVWKDWFRPELQHTXDOLWDWLYHDQGTXDQWLWDWLYH
PHWKRGVXQGHUDYDULHW\RIQDPHVVXFKDVPXOWLWUDLWPXOWLPHWKRGUHVHDUFK
LQWHUUHODWLQJTXDOLWDWLYHDQGTXDQWLWDWLYHGDWDPHWKRGRORJLFDOWULDQJXODWLRQ
PXOWLPHWKRGRORJLFDOUHVHDUFKPL[HGPRGHOVWXGLHVDQGPL[HGPHWKRGVUHVHDUFK
&UHVZHOOHWDO ²DVLQGLFDWHGE\WKHWLWOHRIWKHUHFHQW+DQGERRNRI0L[HG
0HWKRGV5HVHDUFKWKHILHOGDSSHDUVWRKDYHVHWWOHGZLWKWKHODVWWHUP2YHUWKHSDVW
\HDUVPL[HGPHWKRGVUHVHDUFKKDVEHHQLQFUHDVLQJO\VHHQDVDWKLUGDSSURDFKLQ
UHVHDUFKPHWKRGRORJ\7KHPHWKRGKDVEHHQHQGRUVHGE\VRPHRIWKHPRVWLQIOXHQWLDO

PHWKRGRORJLVWVLQWKHVRFLDOVFLHQFHV0LOHVDQG+XEHUPDQ VXPPDUL]HG
WKHQHZHPHUJLQJ=HLWJHLVWZHOO
(QWHUWDLQPL[HGPRGHOV:HKDYHVRXJKWWRPDNHDYLUWXHRIDYRLGLQJ
SRODUL]DWLRQSROHPLFVDQGOLIHDWWKHH[WUHPHV4XDQWLWDWLYHDQGTXDOLWDWLYH
LQTXLU\FDQVXSSRUWDQGLQIRUPHDFKRWKHU1DUUDWLYHVDQGYDULDEOHGULYHQ
DQDO\VHVQHHGWRLQWHUSHQHWUDWHDQGLQIRUPHDFKRWKHU5HDOLVWVLGHDOLVWVDQG
FULWLFDOWKHRULVWVFDQGREHWWHUE\LQFRUSRUDWLQJRWKHULGHDVWKDQE\UHPDLQLQJ
SXUH7KLQNRILWDVK\EULGYLJRXU
/HWPHFLWHRQHPRUHLOOXVWUDWLYHH[WUDFWIURP6WUDXVVDQG&RUELQ¶V ERRN

4XDOLWDWLYHDQGTXDQWLWDWLYHIRUPVRIUHVHDUFKERWKKDYHUROHVWRSOD\LQ
WKHRULVLQJ7KHLVVXHLVQRWZKHWKHUWRXVHRQHIRUPRUDQRWKHUEXWUDWKHUKRZ
WKHVHPLJKWZRUNWRJHWKHUWRIRVWHUWKHGHYHORSPHQWRIWKHRU\$OWKRXJKPRVW
UHVHDUFKHUVWHQGWRXVHTXDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGVLQVXSSOHPHQWDU\RU
FRPSOHPHQWDU\IRUPVZKDWZHDUHDGYRFDWLQJLVDWUXHLQWHUSOD\EHWZHHQWKH
WZR7KHTXDOLWDWLYHVKRXOGGLUHFWWKHTXDQWLWDWLYHDQGWKHTXDQWLWDWLYHIHHGEDFN
LQWRWKHTXDOLWDWLYHLQDFLUFXODUEXWDWWKHVDPHWLPHHYROYLQJSURFHVVZLWKHDFK
PHWKRGFRQWULEXWLQJWRWKHWKHRU\LQZD\VWKDWRQO\HDFKFDQ
/HWXVQRZH[DPLQHZKHUHWKHLGHDRIPL[LQJPHWKRGRORJLHVKDVFRPHIURPDQG
ZKDWLWVPDLQSULQFLSOHVDUH
7KHSUDFWLFHRIFROOHFWLQJPXOWLSOHGDWDW\SHVGDWHVEDFNWRWKHHDUOLHVWVRFLDOVFLHQFH
UHVHDUFKDWWKHEHJLQQLQJRIWKHWKFHQWXU\DQGDV0D[ZHOODQG/RRPLV SRLQW
RXWWKHSUDFWLFHRIPL[LQJYHU\GLIIHUHQWUHVHDUFKDSSURDFKHV²IRUH[DPSOH
QDWXUDOLVWLFFRQWH[WXDOL]HGDQGLQGXFWLYHDSSURDFKHVZLWKH[SHULPHQWDOPDQLSXODWLRQ
DQGWKHRU\YHULILFDWLRQ²KDVDQHYHQORQJHUKLVWRU\LQGLVFLSOLQHVVXFKDVHWKRORJ\DQG
DQLPDOEHKDYLRXUSDODHRQWRORJ\DQGJHRORJ\<HWVLPLODUO\WRTXDOLWDWLYHUHVHDUFKDQ
H[SOLFLWGLVFXVVLRQRIPL[LQJPHWKRGRORJLHVKDGWRZDLWXQWLOWKHVHFRQGKDOIRIWKHWK
FHQWXU\IRU&DPSEHOODQG)LVNH¶V SURPRWLRQRIPXOWLWUDLWPXOWLPHWKRGUHVHDUFK
DVDZD\RIYDOLGDWLQJUHVHDUFKGHVLJQVE\VHSDUDWLQJWUDLWDQGPHWKRGHIIHFWV
$OWKRXJK&DPSEHOODQG)LVNHIRFXVHGRQFROOHFWLQJPXOWLSOHW\SHVRITXDQWLWDWLYHGDWD
RQO\WKHLUZRUNZDVLQVWUXPHQWDOLQJHQHUDOO\HQFRXUDJLQJWKHXVHRIPXOWLSOHPHWKRGV
DQGWKHFROOHFWLRQRIPXOWLSOHIRUPVRIGDWDLQDVLQJOHVWXG\ +DQVRQHWDO
7KHUHDOEUHDNWKURXJKLQFRPELQLQJTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKRFFXUUHG
LQWKHVZLWKWKHLQWURGXFWLRQRIWKHFRQFHSWRIµWULDQJXODWLRQ¶LQWRWKHVRFLDO
VFLHQFHV7KHWHUPZDVERUURZHGIURPQDYDOQDYLJDWLRQDQGODQGVXUYH\LQJZKHUHLW
UHIHUVWRDPHWKRGIRUGHWHUPLQLQJWKH\HWXQNQRZQSRVLWLRQRIDFHUWDLQVSDWLDOSRLQW
WKURXJKPHDVXUHPHQWRSHUDWLRQVIURPWZRNQRZQSRLQWV (U]EHUJHUDQG.HOOH
EXWLQVRFLDOUHVHDUFKLWEHFDPHV\QRQ\PRXVZLWKFRPELQLQJGDWDVRXUFHVWRVWXG\WKH
VDPHVRFLDOSKHQRPHQRQ,QKLVIDPRXVPRQRJUDSK7KH5HVHDUFK$FW'HQ]LQ

DGYRFDWHGWULDQJXODWLRQDVDZD\RIYDOLGDWLQJK\SRWKHVHVE\H[DPLQLQJWKHPWKURXJK
PXOWLSOHPHWKRGV$OWKRXJK'HQ]LQUHIHUUHGSULPDULO\WRPXOWLSOHTXDOLWDWLYHPHWKRGV
KHIRUPXODWHGZKDWEHFDPHWKHNH\WHQHWRIPL[HGPHWKRGVUHVHDUFKQDPHO\WKDW
PHWKRGRORJLFDOWULDQJXODWLRQFDQKHOSWRUHGXFHWKHLQKHUHQWZHDNQHVVHVRILQGLYLGXDO
PHWKRGVE\RIIVHWWLQJWKHPE\WKHVWUHQJWKRIDQRWKHUWKHUHE\PD[LPL]LQJERWKWKH
LQWHUQDODQGWKHH[WHUQDOYDOLGLW\RIUHVHDUFK 7KHVHWHUPVZLOOEHIXUWKHUGLVFXVVHGLQ
6HFWLRQ )RUH[DPSOH%UHZHUDQG+XQWHU LQWURGXFHGWKHLUSLRQHHULQJ
ERRNRQ0XOWLPHWKRG5HVHDUFKDVIROORZV
7KHVRFLDOVFLHQFHVDUHZHOONQRZQIRUGLVSXWHVEHWZHHQSURSRQHQWVRIGLIIHUHQW
PHWKRGVRUVW\OHVRIUHVHDUFK,QDVHQVHWKHVHPHWKRGRORJLFDOGHEDWHVDUHD
KHDOWK\VLJQ6FHSWLFLVPLVDQHVVHQWLDOSDUWRIVFLHQWLILFLQTXLU\DQGGLIIHUHQW
W\SHVRIPHWKRGVUHSUHVHQWLPSRUWDQWFULWLFDOSHUVSHFWLYHV(TXDOO\LPSRUWDQW
KRZHYHULVWKHIDFWWKDWGLIIHUHQWUHVHDUFKPHWKRGVRIIHUSRVVLEOHVROXWLRQVIRU
RQHDQRWKHU¶VSUREOHPV7KLVLVWKHFHQWUDOSUHPLVHRIWKLVERRN
$IWHUWKHSDUDGLJPZDUKDGORVWLWVHGJHLQWKHVDQGPL[HGPHWKRGV
UHVHDUFKHUVJDLQHGLGHRORJLFDOFRQILGHQFHE\GUDZLQJRQWKHSKLORVRSK\RISUDJPDWLVP
VHHIRUH[DPSOH&KHUU\KROPHV UHVHDUFKPHWKRGRORJ\WH[WVVWDUWHGWRLQFOXGH
FKDSWHUVRQFRPELQHGLQWHJUDWHGRUPL[HGPHWKRGV RQHSDUWLFXODUO\LQIOXHQWLDOZRUN
IURPWKHWLPHEHLQJ&UHVZHOO )ROORZLQJWKLVJURZLQJPRPHQWXPWZRKLJK
SURILOHSXEOLFDWLRQVE\7DVKDNNRULDQG7HGGOLH D ILQDOO\HVWDEOLVKHGPL[HG
PHWKRGVUHVHDUFKDVDOHJLWLPDWHIRUPRILQTXLU\LQWKHVRFLDOVFLHQFHV
,QDSSOLHGOLQJXLVWLFVZHILQGPDQ\VWXGLHVWKDWKDYHFRPELQHGPHWKRGRORJLHV
0DJQDQ IRUH[DPSOHUHSRUWVWKDWRYHUWKH±SHULRGSHUFHQWRIWKH
UHVHDUFKSDSHUVDSSHDULQJLQ7KH0RGHUQ/DQJXDJH-RXUQDOXVHGPL[HGPHWKRGV
ZKLFKLVUHODWLYHO\KLJKLIZHFRPSDUHLWWRWKHWRWDOQXPEHURITXDOLWDWLYHVWXGLHV
SHUFHQW +RZHYHUZHPXVWQRWHWKDWPRVWVWXGLHVLQZKLFKVRPHVRUWRIPHWKRG
PL[LQJKDVWDNHQSODFHKDYHQRWDFWXDOO\IRUHJURXQGHGWKHPL[HGPHWKRGVDSSURDFK
DQGKDUGO\DQ\SXEOLVKHGSDSHUVKDYHWUHDWHGPL[HGPHWKRGRORJ\LQDSULQFLSOHGZD\
&XUUHQWO\WKHUHLVDJHQHUDOFDOORQWKHSDUWRIDSSOLHGOLQJXLVWVRIERWK48$/DQG
48$1RULHQWDWLRQIRUPRUHHQJDJHPHQWLQWKLVSUDFWLFHDQG/D]DUDWRQ¶V
FRQFOXVLRQFDQEHVHHQDVUHSUHVHQWDWLYHµ,ZRXOGDOVRKRSHWKDWZHZRXOGVHHPRUH
VWXGLHVWKDWFRPELQHTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKPHWKRGVVLQFHHDFK
KLJKOLJKWV³UHDOLW\´LQDGLIIHUHQW\HWFRPSOHPHQWDU\ZD\¶
0DLQFKDUDFWHULVWLFVRIPL[HGPHWKRGVUHVHDUFK
$VWUDLJKWIRUZDUGZD\RIGHVFULELQJPL[HGPHWKRGVUHVHDUFKLVWRGHILQHLWDVVRPH
VRUWRIDFRPELQDWLRQRITXDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGVZLWKLQDVLQJOHUHVHDUFK
SURMHFW7KHVHWZRDSSURDFKHVKDYHDOUHDG\EHHQGHVFULEHGVHSDUDWHO\DERYHDQGVR
WKHUHLVQRQHHGWRUHLWHUDWHWKHLUPDLQIHDWXUHVWKHUHDOLVVXHLQPL[HGPHWKRGV
UHVHDUFKFRQFHUQVKRZWKH48$/±48$1FRPELQDWLRQWDNHVSODFHDQGVFKRODUVKDYH
SURSRVHGVHYHUDOYLDEOHGHVLJQRSWLRQVLQWKHOLWHUDWXUH7KHVHDUHGLVFXVVHGLQ&KDSWHU

LQGHWDLODVDSUHOLPLQDU\OHWPHVD\WKDWWKHYDULHW\RISRVVLEOHFRPELQDWLRQVLVULFK
JRLQJZHOOEH\RQGVLPSOHVHTXHQWLDODUUDQJHPHQWV LHDUHVHDUFKSKDVHLVIROORZHGE\
DVHFRQGSKDVHUHSUHVHQWLQJWKHRWKHUDSSURDFK )XUWKHUPRUHTXDOLWDWLYHDQG
TXDQWLWDWLYHSULQFLSOHVFDQDOVREHFRPELQHGDWWKHGDWDDQDO\VLVVWDJHE\µTXDQWLI\LQJ¶
RUµTXDOLWL]LQJ¶WKHGDWD 6HH6HFWLRQRQGDWDWUDQVIRUPDWLRQ
6WUHQJWKVDQGZHDNQHVVHVRIPL[HGPHWKRGVUHVHDUFK
$VDUHVXOWRIWKHJURZLQJSRSXODULW\RIPL[HGPHWKRGVUHVHDUFKVHYHUDODUJXPHQWV
KDYHEHHQSXWIRUZDUGDERXWWKHYDOXHRIPL[LQJPHWKRGV/HWXVKDYHDORRNDWWKH
PRVWLPSRUWDQWRQHV
x ,QFUHDVLQJWKHVWUHQJWKVZKLOHHOLPLQDWLQJWKHZHDNQHVVHV7KHPDLQDWWUDFWLRQRI
PL[HGPHWKRGVUHVHDUFKKDVEHHQWKHIDFWWKDWE\XVLQJERWK48$/DQG48$1
DSSURDFKHVUHVHDUFKHUVFDQEULQJRXWWKHEHVWRIERWKSDUDGLJPVWKHUHE\
FRPELQLQJTXDQWLWDWLYHDQGTXDOLWDWLYHUHVHDUFKVWUHQJWKV OLVWHGLQ6HFWLRQV
DQG 7KLVLVIXUWKHUDXJPHQWHGE\WKHSRWHQWLDOWKDWWKHVWUHQJWKVRIRQH
PHWKRGFDQEHXWLOL]HGWRRYHUFRPHWKHZHDNQHVVHVRIDQRWKHUPHWKRGXVHGLQWKH
VWXG\)RUH[DPSOHDVPHQWLRQHGHDUOLHU48$1UHVHDUFKHUVKDYHVHHQ48$/
UHVHDUFKDVEHLQJWRRFRQWH[WVSHFLILFDQGHPSOR\LQJXQUHSUHVHQWDWLYHVDPSOHV²LQ
DPL[HGPHWKRGVVWXG\WKHVDPSOLQJELDVFDQEHFDQFHOOHGRXWLIWKHVHOHFWLRQRI
WKHTXDOLWDWLYHSDUWLFLSDQWVLVEDVHGRQWKHUHVXOWVRIDQLQLWLDOUHSUHVHQWDWLYHVXUYH\
6HH6HFWLRQRQWKHPDLQW\SHVRIPL[HGPHWKRGVGHVLJQ 2QWKHRWKHUKDQG
48$/UHVHDUFKHUVXVXDOO\YLHZ48$1UHVHDUFKDVRYHUO\VLPSOLVWLF
GHFRQWH[WXDOL]HGDQGUHGXFWLRQLVWLQWHUPVRILWVJHQHUDOL]DWLRQVIDLOLQJWRFDSWXUH
WKHPHDQLQJVWKDWDFWRUVDWWDFKWRWKHLUOLYHVDQGFLUFXPVWDQFHV %UDQQHQ ²LQ
DPL[HGPHWKRGVVWXG\D48$1SKDVHFDQEHIROORZHGE\D48$/FRPSRQHQWWR
QHXWUDOL]HWKLVLVVXHE\DGGLQJGHSWKWRWKHTXDQWLWDWLYHUHVXOWVDQGWKHUHE\SXWWLQJ
IOHVKRQWKHERQHV
x 0XOWLOHYHODQDO\VLVRIFRPSOH[LVVXHV,WKDVEHHQVXJJHVWHGE\PDQ\WKDWZHFDQ
JDLQDEHWWHUXQGHUVWDQGLQJRIDFRPSOH[SKHQRPHQRQE\FRQYHUJLQJQXPHULF
WUHQGVIURPTXDQWLWDWLYHGDWDDQGVSHFLILFGHWDLOVIURPTXDOLWDWLYHGDWD:RUGVFDQ
EHXVHGWRDGGPHDQLQJWRQXPEHUVDQGQXPEHUVFDQEHXVHGWRDGGSUHFLVLRQWR
ZRUGV,WLVHDV\WRWKLQNRIVLWXDWLRQVLQDSSOLHGOLQJXLVWLFVZKHQZHDUHLQWHUHVWHG
DWWKHVDPHWLPHLQERWKWKHH[DFWQDWXUH LH48$/ DQGWKHGLVWULEXWLRQ LH
48$1 RIDSKHQRPHQRQ IRUH[DPSOHZK\GRVRPHWHHQDJHER\VFRQVLGHU
PRGHUQODQJXDJHOHDUQLQJµJLUOLVK¶DQGKRZH[WHQVLYHLVWKLVSHUFHSWLRQ" 0L[HG
PHWKRGVUHVHDUFKLVSDUWLFXODUO\DSSURSULDWHIRUVXFKPXOWLOHYHODQDO\VHVEHFDXVHLW
DOORZVLQYHVWLJDWRUVWRREWDLQGDWDDERXWERWKWKHLQGLYLGXDODQGWKHEURDGHU
VRFLHWDOFRQWH[W
x ,PSURYHGYDOLGLW\0L[HGPHWKRGVUHVHDUFKKDVDXQLTXHSRWHQWLDOWRSURGXFH
HYLGHQFHIRUWKHYDOLGLW\RIUHVHDUFKRXWFRPHVWKURXJKWKHFRQYHUJHQFHDQG
FRUURERUDWLRQRIWKHILQGLQJV 6HH&KDSWHUIRUDGHWDLOHGGLVFXVVLRQRIUHVHDUFK

YDOLGLW\ ,QGHHGLPSURYLQJWKHYDOLGLW\RIUHVHDUFKKDVEHHQDWWKHKHDUWRIWKH
QRWLRQRIWULDQJXODWLRQHYHUVLQFHLWVLQWURGXFWLRQLQWKHV&RUUHVSRQGLQJ
HYLGHQFHREWDLQHGWKURXJKPXOWLSOHPHWKRGVFDQDOVRLQFUHDVHWKH
JHQHUDOL]DELOLW\²WKDWLVH[WHUQDOYDOLGLW\²RIWKHUHVXOWV
x 5HDFKLQJPXOWLSOHDXGLHQFHV$ZHOFRPHEHQHILWRIFRPELQLQJ48$/DQG48$1
PHWKRGVLVWKDWWKHILQDOUHVXOWVDUHXVXDOO\DFFHSWDEOHIRUDODUJHUDXGLHQFHWKDQ
WKRVHRIDPRQRPHWKRGVWXG\ZRXOGEH$ZHOOH[HFXWHGPL[HGPHWKRGVVWXG\KDV
PXOWLSOHVHOOLQJSRLQWVDQGFDQRIIHUVRPHWKLQJWRHYHU\ERG\UHJDUGOHVVRIWKH
SDUDGLJPDWLFRULHQWDWLRQRIWKHSHUVRQ2IFRXUVHWKHUHLVDOVRWKHGDQJHUWKDWWKH
VWXG\PLJKWIDOOWKURXJKWKHµSDUDGLJPDWLFFUDFN¶DQGDOLHQDWHHYHU\ERG\EXWLQWKH
FXUUHQWVXSSRUWLYHFOLPDWHWKLVLVOHVVOLNHO\
:HDNQHVVHV
0L[LQJTXDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGVKDVFRPHWREHVHHQE\PDQ\DVD
IRUZDUGSRLQWLQJDQGSRWHQWLDOO\HQULFKLQJDSSURDFKEXWDV0DVRQ FDXWLRQVXV
WKHUHDVRQLQJRUORJLFEHKLQGVXFKDQDVVXPSWLRQLVQRWDOZD\VDVUHDGLO\H[SUHVVHGDV
LVWKHVHQWLPHQWLWVHOI+HVVH%LEHUDQG/HDY\ JRHYHQIXUWKHUZKHQWKH\
VXJJHVWWKDWWKHSRSXODUEHOLHIWKDWWKHVXPPD\EHJUHDWHUWKDQLWVSDUWVLVQRW
QHFHVVDULO\WUXH7KH\FLWHDQLQWHUYLHZZLWK-DQLFH0RUVHZKRZDUQVDERXWWKHGDQJHU
RIXVLQJPL[HGPHWKRGVUHVHDUFKDVDµVXEVWLWXWHIRUVKDUSFRQFHSWXDOWKLQNLQJDQG
LQVLJKWIXODQDO\VHV¶ S ,QGHHGLWZRXOGEHFOHDUO\FRXQWHUSURGXFWLYHWRDGRSWD
VWUDWHJ\ZKHUHE\µZKHQLQGRXEWPL[PHWKRGV«¶
+HVVH%LEHUDQG/HDY\ DOVRUDLVHWKHLVVXHRIKRZZHOOYHUVHGDQ\JLYHQ

UHVHDUFKHUFDQEHLQERWKW\SHVRIPHWKRGRORJ\ZKLFKOHDGVWRDFULWLFDOTXHVWLRQ&DQ
PRUHKDUPWKDQJRRGEHGRQHZKHQUHVHDUFKHUVDUHQRWDGHTXDWHO\WUDLQHGLQERWK
PHWKRGV"7KLVLVDUHDOLVWLFGDQJHUEHFDXVHWKHYDVWPDMRULW\RIUHVHDUFKHUVODFN
PHWKRGRORJLFDOVNLOOVWRKDQGOHERWK48$/DQG48$1GDWD$QGHYHQLIZHFDQ
H[SHFWWKLVVLWXDWLRQWRLPSURYHZLWKWKHJURZLQJDZDUHQHVVRIPL[HGPHWKRGV
UHVHDUFKWKHTXHVWLRQVWLOOUHPDLQV$SDUWIURPDUHODWLYHO\VPDOOQXPEHURIXQLTXH
PHWKRGRORJLFDOO\DPELGH[WURXVVSHFLPHQFDQZHDVVXPHWKDWWKHYLVLRQRID
PXOWLPHWKRGRORJLFDOO\VDYY\QHZEUHHGRIUHVHDUFKHUVLVUHDOLVWLF"
)LQDOO\0D[ZHOODQG/RRPLV KLJKOLJKWDIXUWKHULVVXHWKHGLYHUVLW\RIWKH

SRVVLEOHFRPELQDWLRQVRIGLIIHUHQWPHWKRGVZKLFKLVDVWKHVFKRODUVDUJXHIDUJUHDWHU
WKDQDQ\W\SRORJ\FDQDGHTXDWHO\HQFRPSDVV2QHFDQQRWKHOSZRQGHULQJZKHWKHU
WKHUHLVUHDOO\DSULQFLSOHGDSSURDFKWRJXLGLQJWKHYDULHW\RIFRPELQDWLRQVVRWKDWZH
GRQRWHQGXSZLWKDQµDQ\WKLQJJRHVDVORQJDV\RXPL[WKHP¶PHQWDOLW\:HZLOO
FRPHEDFNWRWKLVTXHVWLRQRIµSULQFLSOHGPL[LQJ¶ODWHULQWKLVERRN LQ6HFWLRQ
DQG&KDSWHU
0\RZQSDUDGLJPDWLFVWDQFH

,QDFFRUGDQFHZLWKP\RYHUDOOEHOLHIVDERXWUHVHDUFKPHWKRGRORJ\,WU\WRDVVXPHD
JHQXLQHO\XQELDVHGSRVLWLRQWKURXJKRXWWKLVERRNZLWKUHJDUGWRSDUDGLJPDWLF
SUHIHUHQFHVHPSKDVL]LQJZKHUHYHUSRVVLEOHSRVLWLYHDVSHFWVDQGSRWHQWLDOV+RZHYHU,
IHHOWKDWLWLVQHFHVVDU\DWWKLVSRLQWWREHDELWPRUHVSHFLILFDERXWP\RZQEDFNJURXQG
DQGUHVHDUFKRULHQWDWLRQ
$VDOUHDG\PHQWLRQHGLQWKH3UHIDFHPRVWFROOHDJXHVZRXOGSUREDEO\FRQVLGHUPHD
TXDQWLWDWLYHUHVHDUFKHUEHFDXVHP\EHVWNQRZQVWXGLHVLQYROYHWKHXVHRIUHVHDUFK
PHWKRGVDVVRFLDWHGZLWKTXDQWLWDWLYHVRFLDOSV\FKRORJ\ DWWLWXGHPRWLYDWLRQVXUYH\VLQ
SDUWLFXODU ,QGHHG,GRDSSUHFLDWHWKHHODERUDWHWHFKQLFDODSSDUDWXVLQYROYHGLQ
TXDQWLWDWLYHUHVHDUFKDQG,DFWXDOO\OLNHVWDWLVWLFV$WWKHVDPHWLPH,KDYHDOVR
H[SHULHQFHGDJDLQDQGDJDLQKRZPXFKULFKHUGDWDZHFDQREWDLQLQDZHOOFRQGXFWHG
DQGDQDO\VHGTXDOLWDWLYHVWXG\WKDQHYHQLQDODUJHVFDOHTXHVWLRQQDLUHVXUYH\7KXV
VWDUWLQJLQWKHPLGV,EHJDQWRLQFOXGHTXDOLWDWLYHFRPSRQHQWVLQP\UHVHDUFKDQG
,KDYHOHDUQWPXFKIURPWKHTXDOLWDWLYHGDWDP\DVVRFLDWHVDQG,KDYHFROOHFWHGDQG
DQDO\VHGRYHUWKH\HDUV
,GRDFFHSWWKDWFHUWDLQLVVXHVDUHEHVWUHVHDUFKHGXVLQJHLWKHU48$/RU48$1
PHWKRGVEXW,KDYHDOVRFRPHWREHOLHYHWKDWLQPRVWFDVHVDPL[HGPHWKRGVDSSURDFK
FDQRIIHUDGGLWLRQDOEHQHILWVIRUWKHXQGHUVWDQGLQJRIWKHSKHQRPHQRQLQTXHVWLRQ
7KHUHIRUHDWWKHHQGRIWKHUHVHDUFKPHWKRGRORJ\FKDSWHURIDERRN,KDYHZULWWHQRQ
/PRWLYDWLRQ '|UQ\HL ,LQFOXGHGDFRQFOXGLQJVHFWLRQFDOOHGµ7RZDUGVD
FRPELQHGXVHRITXDQWLWDWLYHDQGTXDOLWDWLYHVWXGLHV¶DQGRYHUWKHSDVWGHFDGH,KDYH
HQFRXUDJHGPRVWRIP\3K'VWXGHQWVWRWU\DQGLQWHJUDWH48$/DQG48$1PHWKRGV
LQYHVWLJDWLQJDUDQJHRIWRSLFVIURPDFFXOWXUDWLRQWRWHDFKHUPRWLYDWLRQ $OWKRXJKWKH
JHQHUDOO\KLJKTXDOLW\UHVHDUFKILQGLQJVP\VWXGHQWVSURGXFHGKDYHFRQILUPHGWRPH
WKHYLDELOLW\RIWKLVDSSURDFK,DPDZDUHRIWKHIDFWWKDWPRVWVFKRODUVDUHPRUH
QDWXUDOO\LQFOLQHGWRZDUGVHLWKHU48$/RU48$1UHVHDUFK VHH6HFWLRQRQ
SHUVRQDOFRQVLGHUDWLRQVLQPHWKRGVHOHFWLRQ DIDFWWKDW,VXVSHFWKDVWRGRZLWKRXU
FRJQLWLYHVW\OHV7KHUHIRUHLQFRQGXFWLQJPL[HGPHWKRGVVWXGLHV,VHHNWRFRRSHUDWH
ZLWKUHVHDUFKHUVZKRKDYHDTXDOLWDWLYHRULHQWDWLRQWRFRPSOHPHQWP\TXDQWLWDWLYH
EDFNJURXQG

4XDOLW\FULWHULDUHVHDUFKHWKLFVDQG
RWKHUUHVHDUFKLVVXHV
%HIRUHZHGLVFXVVWKHFROOHFWLRQDQGDQDO\VLVRIGDWDZHQHHGWRDGGUHVVDIHZEDVLF
UHVHDUFKLVVXHVWKDWKDYHDEHDULQJRQRXUPHWKRGRORJLFDOGHFLVLRQVUHJDUGOHVVRIRXU
SDUDGLJPDWLFFKRLFH)LUVWDQGIRUHPRVWFRPHWKHTXDOLW\FULWHULDIRUUHVHDUFKEHFDXVH
ZHFDQRQO\FODLPWKDWRXULQYHVWLJDWLRQLVLQGHHGDµGLVFLSOLQHG¶LQTXLU\LIZHFDQVHW
H[SOLFLWTXDOLW\VWDQGDUGVWRDFKLHYH7KHVHFRQGWRSLFWRFRYHULVUHVHDUFKHWKLFV
ZKLFKLVDFXULRXVO\QHJOHFWHGLVVXHLQPXFKDSSOLHGOLQJXLVWLFUHVHDUFKDQGRIWHQ
VXUIDFHVRQO\ZKHQUHVHDUFKHUVUHDOL]HWKDWWKH\QHHGWRSURGXFHVRPHµHWKLFDO
FOHDUDQFH¶IRUWKHLUVWXG\LIWKH\ZDQWWRVXEPLWWKHUHVXOWVWRREWDLQDSRVWJUDGXDWH
GHJUHHRUWREHSXEOLVKHGLQFHUWDLQMRXUQDOV)ROORZLQJWKHDQDO\VLVRIHWKLFDOLVVXHVZH
FRQVLGHUWKHUROHDQGVLJQLILFDQFHRIUHVHDUFKTXHVWLRQVDQGK\SRWKHVHV²WKHVHZLOOEH
IXUWKHUGLVFXVVHGLQ&KDSWHU7KHFKDSWHUFRQFOXGHVZLWKWKHGLVFXVVLRQRIWKUHH
WRSLFV²SLORWLQJWKHUHVHDUFKUHVHDUFKORJVDQGGDWDPDQDJHPHQW²WKDWDUHHVVHQWLDO
LVVXHVIRUODXQFKLQJDUHVHDUFKSURMHFWEXWZKLFKLQP\H[SHULHQFHGRQRWUHFHLYH
VXIILFLHQWDWWHQWLRQLQDSSOLHGOLQJXLVWLFUHVHDUFK
4XDOLW\FULWHULDIRUUHVHDUFK
µ9DOLGLW\¶LVDQRWKHUZRUGIRUWUXWK
6LOYHUPDQ
$VZHKDYHVHHQWKHEDVLFGHILQLWLRQRIVFLHQWLILFUHVHDUFKLVWKDWLWLVDµGLVFLSOLQHG¶
LQTXLU\DQGWKHUHIRUHRQHWKLQJUHVHDUFKFDQQRWDIIRUGLVWREHKDSKD]DUGRUODFNLQJ
ULJRXU$FFRUGLQJO\WKHUHLVDJHQHUDOFRQVHQVXVDPRQJVWUHVHDUFKHUVWKDWWKH\PXVW
FRQWLQXDOO\VWULYHWRDVVHVVDQGGRFXPHQWWKHOHJLWLPDF\RIWKHLUILQGLQJV²DIWHUDOO
VFKRODUVKDYHWRFRQYLQFHWKHLUDXGLHQFHVWKDWWKH\VKRXOGOLVWHQWRWKHPDQG
HYHQWXDOO\EHOLHYHWKHP
8QIRUWXQDWHO\JHQHUDODJUHHPHQWDERXWUHVHDUFKTXDOLW\LQVFKRODUO\FLUFOHVVWRSVDW
WKHUHFRJQLWLRQRILWVLPSRUWDQFHZKHQLWFRPHVWRVSHFLI\LQJWKHFRQFUHWHµTXDOLW\
FULWHULD¶WREHDSSOLHGWKHOLWHUDWXUHLVFKDUDFWHUL]HGE\DKRVWRISDUDOOHORUDOWHUQDWLYH
YLHZVDQGYHU\OLWWOHFRQVHQVXV7KHIUDJPHQWHGQDWXUHRIWKHGRPDLQLVZHOOUHIOHFWHG
E\WKHIDFWWKDWWKHUHGRHVQRWHYHQH[LVWDQ\XQLYHUVDOO\DFFHSWHGWHUPLQRORJ\WR
GHVFULEHTXDOLW\FULWHULDDQGWKHWHUPVWKDWDUHPRUHZLGHO\NQRZQ²µYDOLGLW\¶DQG
µUHOLDELOLW\¶LQSDUWLFXODU²DUHVXEMHFWWRRQJRLQJFULWLFLVPZLWKYDULRXVDXWKRUV
UHJXODUO\RIIHULQJGLVSDUDWHVHWVRIDOWHUQDWLYHV2IFRXUVHJLYHQWKHKXJHLPSRUWDQFH
RIUHVHDUFKTXDOLW\WKLVVLWXDWLRQLVQRWVXUSULVLQJ5HSUHVHQWDWLYHVRIGLIIHUHQWUHVHDUFK
WUDGLWLRQVXQGHUVWDQGDEO\HPSKDVL]HTXDOLW\SDUDPHWHUVWKDWZLOODOORZWKHW\SHRI
LQTXLU\WKH\SXUVXHWRFRPHRXWLQDJRRGOLJKW7KHSUREOHPLVWKDWWKHVFRSHRI

SRVVLEOHTXDOLW\FULWHULDLVUDWKHUZLGH²UDQJLQJIURPVWDWLVWLFDODQGPHWKRGRORJLFDO
LVVXHVWKURXJKUHDOZRUOGVLJQLILFDQFHDQGSUDFWLFDOYDOXHVWRWKHEHQHILWVWRWKH
UHVHDUFKSDUWLFLSDQWV²DQGVRPHSDUDPHWHUVWKDWVHHPWRGRRQHUHVHDUFKDSSURDFK
MXVWLFHGRQRWUHDOO\ZRUNZHOOZLWKRWKHUDSSURDFKHVWKHUHE\OHDGLQJWRWHQVLRQ
EHWZHHQFDPSVDQGWRIXUWKHUSUROLIHUDWLRQRITXDOLW\FULWHULD
,PHQWLRQHGLQ&KDSWHU 6HFWLRQ WKDWDQRYHUYLHZVXFKDVWKLVERRNQHHGVWR
EHUDWKHUFRQVHUYDWLYHLQLWVDSSURDFKLQWKLVVSLULW,ZLOOFHQWUHP\GLVFXVVLRQDURXQG
WKHWZREHVWNQRZQUHOHYDQWFRQFHSWVµYDOLGLW\¶DQGµUHOLDELOLW\¶DQGZLOOXVHWKHVH
ZLWKUHJDUGWRERWKTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFK:HPXVWQRWHKRZHYHUWKDW
ERWKWHUPVZHUHRULJLQDOO\LQWURGXFHGLQTXDQWLWDWLYHUHVHDUFKDQGWKHUHIRUHZKLOHWKH
VLJQLILFDQFHRIWKHWZRFRQFHSWVLVDQXQTXHVWLRQDEOHIDFWRIOLIHLQWKH48$1
SDUDGLJPPDQ\48$/UHVHDUFKHUVGHQ\WKHUHOHYDQFHRIµYDOLGLW\¶DQGµUHOLDELOLW\¶DV
GHILQHGLQTXDQWLWDWLYHWHUPV,QRUGHUWRLQWURGXFHTXDOLW\FULWHULDWKDWDUHPRUH
VXLWDEOHIRU48$/LQTXLULHVVHYHUDODOWHUQDWLYHWHUPVKDYHEHHQSURSRVHG9DOLGLW\KDV
EHHQUHIHUUHGWRDVµWUXVWZRUWKLQHVV¶µDXWKHQWLFLW\¶µFUHGLELOLW\¶µULJRXU¶DQG
µYHUDFLW\¶EXWQRQHRIWKHVHKDYHUHDFKHGDFRQVHQVXVDQGWKHWHUPLQRORJ\LQJHQHUDOLV
DKLJKO\GHEDWHGWRSLF7KHUHKDYHDOVREHHQDWWHPSWVWRPDWFK48$/DQG48$1
WHUPV IRUH[DPSOHH[WHUQDOYDOLGLW\ WUDQVIHUDELOLW\UHOLDELOLW\ GHSHQGDELOLW\ EXW
VXUHO\WKHZKROHUDWLRQDOHIRUGHYHORSLQJDSDUDOOHOWHUPLQRORJ\LVWKHFRQYLFWLRQWKDW
WKHUHDUHQRVWUDLJKWIRUZDUGSDUDOOHOVLQWKHWZRUHVHDUFKSDUDGLJPV,ZLOOGHVFULEHLQ
6HFWLRQWKHPRVWLQIOXHQWLDODOWHUQDWLYHYDOLGLW\W\SRORJ\SXWIRUZDUGE\/LQFROQ
DQG*XED EXW,ZLOOWKHQDUJXHLQIDYRXURI0D[ZHOO¶V ZHOONQRZQ
WD[RQRP\WKDWLVEXLOWDURXQGIDFHWVRITXDOLWDWLYHYDOLGLW\
4XDOLW\FULWHULDLQTXDQWLWDWLYHUHVHDUFK
$OWKRXJKWKHSUHYLRXVGLVFXVVLRQZLOOKDYHJLYHQWKHLPSUHVVLRQWKDWWKHWHUPLQRORJ\
XVHGWRGHVFULEHTXDQWLWDWLYHTXDOLW\VWDQGDUGVDQGFULWHULDLVUHODWLYHO\XQSUREOHPDWLF
WKLVLVQRWHQWLUHO\WKHFDVH7KHFRQFHSWRIµUHOLDELOLW\¶LVIDLUO\VWUDLJKWIRUZDUGEXW
ZKHQZHORRNDWµYDOLGLW\¶ZHILQGWZRSDUDOOHOV\VWHPVLQWKHTXDQWLWDWLYHOLWHUDWXUH²
RQHFHQWUHGDURXQGµFRQVWUXFWYDOLGLW\¶DQGLWVFRPSRQHQWVWKHRWKHUDURXQGWKH
µLQWHUQDOH[WHUQDOYDOLGLW\GLFKRWRP\¶²DQGVFKRODUVWHQGWREHVXUSULVLQJO\YDJXH
DERXWWKHUHODWLRQVKLSEHWZHHQWKHVHWZRV\VWHPV7KHXVXDOSUDFWLFHLVWKDWDZRUN
HLWKHUFRYHUVRQHRUWKHRWKHU7KLVGXDOLVWLFDSSURDFKLVGXHWRWKHIDFWWKDWZLWKLQWKH
TXDQWLWDWLYHSDUDGLJPPHDQLQJIXOQHVVKDVEHHQFRQFHSWXDOL]HGIURPWZRSHUVSHFWLYHV
UHVHDUFKGHVLJQDQGPHDVXUHPHQW 6HH%DFKPDQ/\QFK
5HVHDUFKYDOLGLW\FRQFHUQVWKHZKROHUHVHDUFKSURFHVVDQGIROORZLQJ&DPSEHOODQG
6WDQOH\ IRFXVHVRQWKHGLVWLQFWLRQRIµLQWHUQDOYDOLGLW\¶ZKLFKDGGUHVVHVWKH
VRXQGQHVVRIWKHUHVHDUFK LHZKHWKHUWKHRXWFRPHLVLQGHHGDIXQFWLRQRIWKHYDULRXV
YDULDEOHVDQGWUHDWPHQWIDFWRUVPHDVXUHG DQGµH[WHUQDOYDOLGLW\¶ZKLFKFRQFHUQVWKH
JHQHUDOL]DELOLW\RIWKHUHVXOWVEH\RQGWKHREVHUYHGVDPSOH

0HDVXUHPHQWYDOLGLW\UHIHUVWRWKHPHDQLQJIXOQHVVDQGDSSURSULDWHQHVVRIWKH
LQWHUSUHWDWLRQRIWKHYDULRXVWHVWVFRUHVRURWKHUDVVHVVPHQWSURFHGXUHRXWFRPHV$VZH
ZLOOVHHEHORZYDOLGLW\KHUHLVVHHQDVDXQLWDU\FRQFHSWH[SUHVVHGLQWHUPVRI
µFRQVWUXFWYDOLGLW\¶ZKLFKFDQEHIXUWKHUEURNHQGRZQWRYDULRXVIDFHWVVXFKDVFRQWHQW
RUFULWHULRQYDOLGLW\,WLVWKLVSHUVSHFWLYH JRLQJEDFNWR/DGR¶VZRUNRQWHVWLQJ WKDW
SURGXFHGWKHFODVVLFWHQHWWKDWDWHVWLVYDOLGLILWPHDVXUHVZKDWLWLVVXSSRVHGWR
PHDVXUHHYHQWKRXJKWKHFXUUHQWYLHZLVWKDWLWLVQHLWKHUWKHLQVWUXPHQWQRUWKHDFWXDO
VFRUHWKDWLVYDOLGEXWUDWKHUWKHLQWHUSUHWDWLRQRIWKHVFRUHZLWKUHJDUGWRDVSHFLILF
SRSXODWLRQ
7KXVWKHGLVFXVVLRQRITXDQWLWDWLYHTXDOLW\VWDQGDUGVLVEHVWGLYLGHGLQWRWKUHHSDUWV
D UHOLDELOLW\ E PHDVXUHPHQWYDOLGLW\DQG F UHVHDUFKYDOLGLW\
5HOLDELOLW\
7KHWHUPUHOLDELOLW\FRPHVIURPPHDVXUHPHQWWKHRU\DQGUHIHUVWRWKHµFRQVLVWHQFLHVRI
GDWDVFRUHVRUREVHUYDWLRQVREWDLQHGXVLQJHOLFLWDWLRQLQVWUXPHQWVZKLFKFDQLQFOXGHD
UDQJHRIWRROVIURPVWDQGDUGL]HGWHVWVDGPLQLVWHUHGLQHGXFDWLRQDOVHWWLQJVWRWDVNV
FRPSOHWHGE\SDUWLFLSDQWVLQDUHVHDUFKVWXG\¶ &KDOKRXE'HYLOOH ,QRWKHUZRUGV
UHOLDELOLW\LQGLFDWHVWKHH[WHQWWRZKLFKRXUPHDVXUHPHQWLQVWUXPHQWVDQGSURFHGXUHV
SURGXFHFRQVLVWHQWUHVXOWVLQDJLYHQSRSXODWLRQLQGLIIHUHQWFLUFXPVWDQFHV7KHYDULDWLRQRI
WKHFLUFXPVWDQFHVFDQLQYROYHGLIIHUHQFHVLQDGPLQLVWUDWLYHSURFHGXUHVFKDQJHVLQWHVW
WDNHUVRYHUWLPHGLIIHUHQFHVLQYDULRXVIRUPVRIWKHWHVWDQGGLIIHUHQFHVLQUDWHUV
%DFKPDQE ,IWKHVHYDULDWLRQVFDXVHLQFRQVLVWHQFLHVRUPHDVXUHPHQWHUURUWKHQ
RXUUHVXOWVDUHXQUHOLDEOH
,WLVLPSRUWDQWWRUHPHPEHUWKDWFRQWUDU\WRPXFKRIWKHXVDJHLQWKH
PHWKRGRORJLFDOOLWHUDWXUHLWLVQRWWKHWHVWRUWKHPHDVXULQJLQVWUXPHQWWKDWLVUHOLDEOHRU
XQUHOLDEOH5HOLDELOLW\LVDSURSHUW\RIWKHVFRUHVRQDWHVWIRUDSDUWLFXODUSRSXODWLRQRI
WHVWWDNHUV :LONLQVRQDQG7)6, DQG%DFKPDQ E UHPLQGVXVWKDW
DFFRUGLQJO\DOOWKHSURIHVVLRQDOLQWHUQDWLRQDOVWDQGDUGVUHTXLUHUHVHDUFKHUVWRHVWLPDWH
DQGUHSRUWWKHUHOLDELOLW\RIµHDFKWRWDOVFRUHVXEVFRUHRUFRPELQDWLRQRIVFRUHVWKDWLVWR
EHLQWHUSUHWHG¶ $(5$$3$DQG1&0( ,QWKHOLJKWRIWKLVLWLVVXUSULVLQJ
WKDWWKHYDVWPDMRULW\RITXDQWLWDWLYHVRFLDOVFLHQWLVWVGRQRWSURYLGHUHOLDELOLW\HVWLPDWHVIRU
WKHLURZQGDWD 2QZXHJEX]LHDQG/HHFK 7KLVLVSDUWO\GXHWRWKHIDOVHXQGHUVWDQGLQJ
WKDWUHOLDELOLW\LVDFKDUDFWHULVWLFRIWKHLQVWUXPHQWZKLFKZRXOGLPSO\WKDWLIZHXVHDQ
LQVWUXPHQWWKDWKDVEHHQGRFXPHQWHGWRSURGXFHUHOLDEOHVFRUHVEHIRUHZHGRQRWQHHGWR
ZRUU\DERXWHVWDEOLVKLQJUHOLDELOLW\LQRXUVDPSOHDJDLQ
%DFKPDQ E RIIHUVDGHWDLOHGGHVFULSWLRQRIWZRJHQHUDODSSURDFKHVZKHUHE\

FODVVLFWHVWWKHRU\SURYLGHVHVWLPDWHVRIUHOLDELOLW\ :HFDQFDOFXODWHWKHFRUUHODWLRQ
EHWZHHQWZRVHWVRIVFRUHVIRUH[DPSOHEHWZHHQWZRKDOYHVRIDWHVWRUWZRSDUDOOHO
IRUPVRUWZRGLIIHUHQWUDWHUV¶UDWLQJV :HFDQFDOFXODWHDVWDWLVWLFNQRZQDV
&URQEDFKDOSKD VHH6HFWLRQ ZKLFKLVEDVHGRQWKHYDULDQFHVRIWZRRUPRUH
VFRUHVDQGVHUYHVDVDQµLQWHUQDOFRQVLVWHQF\FRHIILFLHQW¶LQGLFDWLQJKRZWKHGLIIHUHQW

VFRUHVµKDQJWRJHWKHU¶ IRUH[DPSOHPRUHWKDQWZRUDWHUV¶VFRUHVRUVHYHUDOSDUDOOHO
TXHVWLRQQDLUHLWHPVLQDVFDOH
0HDVXUHPHQWYDOLGLW\
$VPHQWLRQHGHDUOLHUWKHFRQFHSWRIYDOLGLW\IURPDPHDVXUHPHQWSHUVSHFWLYHKDV
WUDGLWLRQDOO\EHHQVXPPDUL]HGE\WKHVLPSOHSKUDVHDWHVWLVYDOLGLILWPHDVXUHVZKDWLW
LVVXSSRVHGWRPHDVXUH+RZHYHUWKHVFLHQWLILFFRQFHSWXDOL]DWLRQRIPHDVXUHPHQW
YDOLGLW\KDVJRQHWKURXJKVRPHVLJQLILFDQWFKDQJHVRYHUWKHSDVWGHFDGHV$FFRUGLQJWR
&KDSHOOH¶V GHVFULSWLRQRIWKHGHYHORSPHQWRIWKHFRQFHSWLQDSSOLHGOLQJXLVWLFV
YDOLGLW\ZDVVHHQLQWKHVDVDFKDUDFWHULVWLFRIDODQJXDJHWHVW6HYHUDOW\SHVRI
YDOLGLW\ZHUHGLVWLQJXLVKHGµFULWHULRQYDOLGLW\¶ZDVGHILQHGE\WKHWHVW¶VFRUUHODWLRQ
ZLWKDQRWKHUVLPLODULQVWUXPHQWµFRQWHQWYDOLGLW\¶FRQFHUQHGH[SHUWMXGJHPHQWDERXW
WHVWFRQWHQWDQGµFRQVWUXFWYDOLGLW\¶VKRZHGKRZWKHWHVWUHVXOWVFRQIRUPHGWRDWKHRU\
RIZKLFKWKHWDUJHWFRQVWUXFWZDVDSDUW7KLVWUDGLWLRQDOFRQFHSWXDOL]DWLRQLVVWLOO
SHUYDVLYHWRGD\
,QWKHPDLQLQWHUQDWLRQDOJXLGHOLQHVIRUHGXFDWLRQDODQGSV\FKRORJLFDO
PHDVXUHPHQWVSRQVRUHGE\WKH$PHULFDQ(GXFDWLRQDO5HVHDUFK$VVRFLDWLRQWKH
$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQDQGWKH1DWLRQDO&RXQFLORQ0HDVXUHPHQWLQ
(GXFDWLRQ²WKH$(5$$3$1&0(6WDQGDUGVIRU(GXFDWLRQDODQG3V\FKRORJLFDO
7HVWLQJ $(5$$3$DQG1&0( ²UHSODFHGWKHIRUPHUGHILQLWLRQRIWKUHH
YDOLGLWLHVZLWKDXQLWDU\FRQFHSWµFRQVWUXFWYDOLGLW\¶7KLVFKDQJHZDVDQDWXUDO
FRQVHTXHQFHRIWKHVKLIWIURPVHHLQJYDOLGLW\DVDQDWWULEXWHRIWKHWHVWWRFRQVLGHULQJLW
WKHWUXWKIXOQHVVRIWKHLQWHUSUHWDWLRQRIWKHWHVWVFRUHV/\QFK VXPPDUL]HV
WKHQHZFRQFHSWLRQFOHDUO\µ:KHQH[DPLQLQJWKHYDOLGLW\RIDVVHVVPHQWLWLVLPSRUWDQW
WRUHPHPEHUWKDWYDOLGLW\LVDSURSHUW\RIWKHFRQFOXVLRQVLQWHUSUHWDWLRQVRULQIHUHQFHV
WKDWZHGUDZIURPWKHDVVHVVPHQWLQVWUXPHQWVDQGSURFHGXUHVQRWWKHLQVWUXPHQWVDQG
SURFHGXUHVWKHPVHOYHV¶)ROORZLQJWKHQHZDSSURDFKFRQWHQWDQGFULWHULRQUHODWHG
HYLGHQFHFDPHWREHVHHQDVFRQWULEXWRUVWRWKHRYHUDOOYDOLGLW\FRQVWUXFWDORQJZLWK
YDOLGLW\FRQVLGHUDWLRQVRIWKHFRQVHTXHQFHVRIVFRUHLQWHUSUHWDWLRQDQGXVHDQGHYHQ
UHOLDELOLW\ZDVVHHQDVRQHW\SHRIYDOLGLW\HYLGHQFH9DOLGLW\ZDVSRUWUD\HGQRZDVWKH
FRQFOXVLRQRIDFRPSOH[YDOLGLW\DUJXPHQWZKLFKXVHVYDULRXVVWDWLVWLFDODQGWKHRUHWLFDO
VRXUFHVDVHYLGHQFH
7KXVFRQVWUXFWYDOLGLW\LVQRZJHQHUDOO\DFFHSWHGDVDQXPEUHOODWHUPGHVFULELQJD
SURFHVVIRUWKHRU\YDOLGDWLRQWKDWVXEVXPHVVSHFLILFWHVWYDOLGDWLRQRSHUDWLRQV 6PLWK
7KHVHYDOLGDWLRQRSHUDWLRQVDUHFDUULHGRXWZLWKLQµYDOLGDWLRQVWXGLHV¶DQGFDQ
LQYROYHERWKTXDOLWDWLYHDQGTXDQWLWDWLYHHYLGHQFHGHSHQGLQJRQWKHVSHFLILFFODLPVSXW
IRUZDUGDQGFRXQWHUFODLPVUHMHFWHGLQRXUYDOLGLW\DUJXPHQW %DFKPDQE 7KHVH
DUJXPHQWVDUHQHYHUFRQWH[WIUHHDQG0F1DPDUD HPSKDVL]HVWKDWWHVWVFRUH
LQIHUHQFHVQHHGWREHUHYDOLGDWHGIRUHYHU\PDMRUFRQWH[WRIXVH
7RFRQFOXGHWKLVEULHIRYHUYLHZRIPHDVXUHPHQWYDOLGLW\KHUHLVDOLVWRIIRXUNH\
SRLQWVSURYLGHGE\%DFKPDQ E EDVHGRQ/LQQDQG*URQOXQG¶VZRUN

x 9DOLGLW\LVDTXDOLW\RIWKHLQWHUSUHWDWLRQVDQGQRWRIWKHWHVWRUWKHWHVWVFRUHV
x 3HUIHFWYDOLGLW\FDQQHYHUEHSURYHQ²WKHEHVWZHFDQGRLVSURYLGHHYLGHQFHWKDW
RXUYDOLGLW\DUJXPHQWLVPRUHSODXVLEOHWKDQRWKHUSRWHQWLDOFRPSHWLQJ
LQWHUSUHWDWLRQV
x 9DOLGLW\LVVSHFLILFWRDSDUWLFXODUVLWXDWLRQDQGLVQRWDXWRPDWLFDOO\WUDQVIHUDEOHWR
RWKHUV
x 9DOLGLW\LVDXQLWDU\FRQFHSWWKDWFDQEHVXSSRUWHGZLWKPDQ\GLIIHUHQWW\SHVRI
HYLGHQFH
5HVHDUFKYDOLGLW\
7KHVHFRQGW\SHRIYDOLGLW\µUHVHDUFKYDOLGLW\¶LVEURDGHUWKDQPHDVXUHPHQWYDOLGLW\DV
LWFRQFHUQVWKHRYHUDOOTXDOLW\RIWKHZKROHUHVHDUFKSURMHFWDQGPRUHVSHFLILFDOO\ D
WKHPHDQLQJIXOQHVVRIWKHLQWHUSUHWDWLRQVWKDWUHVHDUFKHUVPDNHRQWKHEDVLVRIWKHLU
REVHUYDWLRQVDQG E WKHH[WHQWWRZKLFKWKHVHLQWHUSUHWDWLRQVJHQHUDOL]HEH\RQGWKH
UHVHDUFKVWXG\ %DFKPDQD 7KHVHWZRYDOLGLW\DVSHFWVZHUHUHIHUUHGWRDV
LQWHUQDOYDOLGLW\DQGH[WHUQDOYDOLGLW\E\&DPSEHOODQG6WDQOH\ RYHUIRXU
GHFDGHVDJRDQGDOWKRXJKWKHUHKDYHEHHQDWWHPSWVWRILQHWXQHWKLVW\SRORJ\ IRU
H[DPSOHE\&RRNDQG&DPSEHOO WKHEURDGO\GLFKRWRPRXVV\VWHPKDVVWRRGWKH
WHVWRIWLPH
x $UHVHDUFKVWXG\RUH[SHULPHQWKDVµLQWHUQDOYDOLGLW\¶LIWKHRXWFRPHLVDIXQFWLRQ
RIWKHYDULDEOHVWKDWDUHPHDVXUHGFRQWUROOHGRUPDQLSXODWHGLQWKHVWXG\7KH
ILQGLQJVRIDVWXG\DUHLQWHUQDOO\LQYDOLGLIWKH\KDYHEHHQDIIHFWHGE\IDFWRUVRWKHU
WKDQWKRVHWKRXJKWWRKDYHFDXVHGWKHP
x µ([WHUQDOYDOLGLW\¶LVWKHH[WHQWWRZKLFKZHFDQJHQHUDOL]HRXUILQGLQJVWRDODUJHU
JURXSWRRWKHUFRQWH[WVRUWRGLIIHUHQWWLPHV$VWXG\LVH[WHUQDOO\LQYDOLGLIWKH
UHVXOWVDSSO\RQO\WRWKHXQLTXHVDPSOHRUVHWWLQJLQZKLFKWKH\ZHUHIRXQG
,QWKHTXDQWLWDWLYHSDUDGLJPUHVHDUFKYDOLGLW\LVGHPRQVWUDWHGE\UXOLQJRXWRU
SURYLGLQJHYLGHQFHDJDLQVWYDULRXVµWKUHDWV¶WRYDOLGLW\7KHVHWKUHDWVFRQFHUQ
XQLQWHQGHGIDFWRUVFLUFXPVWDQFHVIODZVRUHYHQWVWKDWFDQLQYDOLGDWHWKHUHVXOWV
,QWHUQDOYDOLGLW\WKUHDWVLQYROYHWKHXVHRILQDGHTXDWHSURFHGXUHVRULQVWUXPHQWVDQ\
XQH[SHFWHGSUREOHPVRFFXUULQJGXULQJWKHVWXG\RUDQ\XQFRQWUROOHGIDFWRUVWKDWFDQ
VLJQLILFDQWO\PRGLI\WKHUHVXOWV7KHPDLQWKUHDWWRH[WHUQDOYDOLGLW\LQH[SHULPHQWDO
VWXGLHVLQYROYHVDQ\VSHFLDOLQWHUDFWLRQEHWZHHQRXULQWHUYHQWLRQWUHDWPHQWDQGVRPH
FKDUDFWHULVWLFVRIWKHSDUWLFXODUJURXSRISDUWLFLSDQWVZKLFKFDXVHVWKHH[SHULPHQWWR
ZRUNRQO\LQRXUVWXG\ IRUH[DPSOHWKHH[SHULPHQWH[SORLWVDVSHFLDOIHDWXUHRIWKH
WUHDWPHQWJURXSWKDWLVXVXDOO\QRWH[LVWHQWLQRWKHUJURXSV ,QVXUYH\VWXGLHVH[WHUQDO
YDOLGLW\WKUHDWVFRQFHUQLQDGHTXDWHVDPSOLQJ WREHGLVFXVVHGLQ&KDSWHU
0DLQWKUHDWVWRUHVHDUFKYDOLGLW\

/HWXVKDYHDORRNDWWKHVL[PRVWVDOLHQWYDOLGLW\WKUHDWV,ZLOOQRWGLYLGHWKHPLQWR
LQWHUQDODQGH[WHUQDOFDWHJRULHVEHFDXVHDIODZLQWKHUHVHDUFKGHVLJQRIWHQDIIHFWVERWK
DVSHFWVRIUHVHDUFKYDOLGLW\
x 3DUWLFLSDQWPRUWDOLW\RUDWWULWLRQ,QVWXGLHVZKHUHZHFROOHFWGLIIHUHQWVHWVRIGDWD
IURPWKHSDUWLFLSDQWV IRUH[DPSOHSUHDQGSRVWWHVWRUPXOWLSOHWHVWVDQG
TXHVWLRQQDLUHV VXEMHFWGURSRXWLVDOZD\VDVHULRXVFRQFHUQ,WUHGXFHVWKHVL]HRI
WKHVDPSOHWKDWKDVDFRPSOHWHGDWDVHWDQGZKDWLVPRUHZRUU\LQJIURPWKH
SHUVSHFWLYHRIYDOLGLW\WKHGURSRXWPD\QRWEHUDQGRPEXWGLIIHUHQWLDO LH
SDUWLFLSDQWVZKRGURSRXWDUHGLIIHUHQWIURPWKRVHZKRVWD\ OHDYLQJWKHUHPDLQLQJ
JURXSZLWKGLVSURSRUWLRQDWHFKDUDFWHULVWLFV
x 7KH+DZWKRUQHHIIHFW7KHWHUPFRPHVIURPWKHQDPHRIDUHVHDUFKVLWH LQ
&KLFDJR ZKHUHWKLVHIIHFWZDVILUVWGRFXPHQWHGZKHQUHVHDUFKHUVLQYHVWLJDWLQJDQ
HOHFWULFFRPSDQ\IRXQGWKDWZRUNSURGXFWLRQLQFUHDVHGZKHQWKH\ZHUHSUHVHQW
UHJDUGOHVVRIWKHFRQGLWLRQVWKHZRUNHUVZHUHVXEMHFWHGWR7KHUHDVRQIRUVXFKDQ
LUUDWLRQDOHIIHFWLVWKDWSDUWLFLSDQWVSHUIRUPGLIIHUHQWO\ZKHQWKH\NQRZWKH\DUH
EHLQJVWXGLHG0HOORZHWDO IRXQGWKDWWKLVWKUHDWLVSDUWLFXODUO\VDOLHQW
LQDSSOLHGOLQJXLVWLFUHVHDUFKDVLWPD\EHµWKHVLQJOHPRVWVHULRXVWKUHDWWRVWXGLHV
RIVSRQWDQHRXVODQJXDJHXVH¶
x 3UDFWLFHHIIHFW,IDVWXG\LQYROYHVUHSHDWHGWHVWLQJRUUHSHDWHGWDVNV IRUH[DPSOH
LQDQH[SHULPHQWDORUORQJLWXGLQDOVWXG\ WKHSDUWLFLSDQWV¶SHUIRUPDQFHPD\
LPSURYHVLPSO\EHFDXVHWKH\DUHJDLQLQJH[SHULHQFHLQWDNLQJWKHSDUWLFXODUWHVWRU
SHUIRUPLQJWKHDVVHVVHGDFWLYLW\
x 0DWXUDWLRQ:KLOHQRWDFRQFHUQLQVKRUWVWXGLHVVXFKDVDRQHRIIVXUYH\
SDUWLFLSDQWPDWXUDWLRQ²WKDWLVSK\VLFDORUPHQWDOFKDQJHZLWKDJH²FDQSOD\D
PDMRUUROHLQORQJHUWHUPVWXGLHV,WLVLQHYLWDEOHWKDWVXEMHFWVFKDQJHLQWKHFRXUVH
RIDQH[SHULPHQWRUEHWZHHQUHSHDWHGDGPLQLVWUDWLRQRIDTXHVWLRQQDLUHWHVWGXHWR
WKHSDVVDJHRIWLPHSHUVH²WKHTXHVWLRQLVKRZPXFKWKLVQDWXUDOO\RFFXUULQJ
GHYHORSPHQWDOSURFHVVDIIHFWVWKHWDUJHWYDULDEOHVLQDVWXG\
x 3DUWLFLSDQWGHVLUHWRPHHWH[SHFWDWLRQ VRFLDOGHVLUDELOLW\ELDV 7KHSDUWLFLSDQWVRI
DVWXG\DUHRIWHQSURYLGHGZLWKFXHVWRWKHDQWLFLSDWHGUHVXOWVRIWKHSURMHFWDQGDV
DUHVXOWWKH\PD\EHJLQWRH[KLELWSHUIRUPDQFHWKDWWKH\EHOLHYHLVH[SHFWHGRI
WKHP$YDULDWLRQRIWKLVWKUHDWLVZKHQSDUWLFLSDQWVWU\WRPHHWVRFLDOH[SHFWDWLRQV
DQGRYHUUHSRUWGHVLUDEOHDWWLWXGHVDQGEHKDYLRXUVZKLOHXQGHUUHSRUWLQJWKRVHWKDW
DUHVRFLDOO\QRWUHVSHFWHG
x +LVWRU\(PSLULFDOUHVHDUFKGRHVQRWWDNHSODFHLQDYDFXXPDQGWKHUHIRUHZHPLJKW
EHVXEMHFWWRWKHHIIHFWVRIXQDQWLFLSDWHGHYHQWVZKLOHWKHVWXG\LVLQSURJUHVV 6HH
IRUH[DPSOH6HFWLRQRQWKHFKDOOHQJHVRIFODVVURRPUHVHDUFK 6XFKHYHQWVDUH
RXWVLGHWKHUHVHDUFKVWXG\\HWWKH\FDQDOWHUWKHSDUWLFLSDQWV¶SHUIRUPDQFH7KH
EHVWZHFDQGRDWWLPHVOLNHWKLVLVWRGRFXPHQWWKHLPSDFWRIWKHHYHQWVVRWKDW
ODWHUZHPD\QHXWUDOL]HLWE\XVLQJVRPHNLQGRIVWDWLVWLFDOFRQWURO

4XDOLW\FULWHULDLQTXDOLWDWLYHUHVHDUFK
7KHXVXDOVWDWHPHQWZHILQGLQWKHOLWHUDWXUHDERXWTXDOLW\LQ48$/UHVHDUFKLVWKDWLWLV
OHVVVWUDLJKWIRUZDUGWRGHILQHWKDQTXDOLW\LQ48$1UHVHDUFK )RUDQLQVLJKWIXO
DQDO\VLVIURPDQDSSOLHGOLQJXLVWLFSHUVSHFWLYHVHH/D]DUDWRQ :KLOHLQWKH
SUHYLRXVVHFWLRQVZHVDZWKDWHYHQTXDQWLWDWLYHUHVHDUFKFULWHULDDUHQRWFRPSOHWHO\
XQDPELJXRXVLWLVWUXHWKDWVHWWLQJH[SOLFLWTXDOLW\VWDQGDUGVLQWKHTXDOLWDWLYHSDUDGLJP
KDVEHHQSDUWLFXODUO\SUREOHPDWLF²DFODLPWKDWHYHQTXDOLWDWLYHUHVHDUFKHUVDJUHHZLWK
6DQGHORZVNLDQG%DUURVR VXPPDUL]HWKHVLWXDWLRQZHOO
2YHUWKHSDVW\HDUVUHDPVRIDUWLFOHVDQGERRNVKDYHEHHQZULWWHQRQWKH
VXEMHFWRITXDOLW\LQTXDOLWDWLYHUHVHDUFK$GGUHVVLQJVXFKFRQFHSWVDVUHOLDELOLW\
DQGULJRUYDOXHDQGYDOLGLW\DQGFULWHULDDQGFUHGLELOLW\VFKRODUVDFURVVWKH
SUDFWLFHDQGVRFLDOVFLHQFHGLVFLSOLQHVKDYHVRXJKWWRGHILQHZKDWDJRRGYDOLG
DQGRUWUXVWZRUWK\TXDOLWDWLYHVWXG\LVWRFKDUWWKHKLVWRU\RIDQGWRFDWHJRUL]H
HIIRUWVWRDFFRPSOLVKVXFKDGHILQLWLRQDQGWRGHVFULEHDQGFRGLI\WHFKQLTXHVIRU
ERWKHQVXULQJDQGUHFRJQL]LQJJRRGVWXGLHV<HWDIWHUDOORIWKLVHIIRUWZHVHHP
WREHQRFORVHUWRHVWDEOLVKLQJDFRQVHQVXVRQTXDOLW\FULWHULDRUHYHQDJUHHRQ
ZKHWKHULWLVDSSURSULDWHWRWU\WRHVWDEOLVKVXFKDFRQVHQVXV
2QHUHDVRQIRUWKHVHGLIILFXOWLHVOLHVLQWKHIDFWWKDWDOWKRXJKWKHWHUPVµYDOLGLW\¶DQG
µUHOLDELOLW\¶UHIHUWRHPSLULFDOUHVHDUFKLQJHQHUDOLQSUDFWLFHWKH\KDYHEHHQDVVRFLDWHG
ZLWKTXDQWLWDWLYHPHWKRGVDQGWKHLURSHUDWLRQDOL]DWLRQKDVWUDGLWLRQDOO\IROORZHG
TXDQWLWDWLYHSULQFLSOHV:HVDZLQ&KDSWHUWKDWDTXDOLWDWLYHVWXG\LVLQKHUHQWO\
VXEMHFWLYHLQWHUSUHWLYHDVZHOODVWLPHDQGFRQWH[WERXQGWKDWLVLQDTXDOLWDWLYH
LQTXLU\µWUXWK¶LVUHODWLYHDQGµIDFWV¶GHSHQGXSRQLQGLYLGXDOSHUFHSWLRQV 0RUVHDQG
5LFKDUGV IRUWKLVUHDVRQVHYHUDOUHVHDUFKHUVKDYHDUJXHGWKDWTXDOLWDWLYHUHVHDUFK
UHTXLUHVLWVRZQSURFHGXUHVIRUDWWDLQLQJYDOLGLW\WKDWDUHGLIIHUHQWIURPWKRVHXVHGLQ
TXDQWLWDWLYHDSSURDFKHV
7KHSUREOHPLVWKDWDOWKRXJKVHYHUDOW\SRORJLHVRIµTXDOLWDWLYHYDOLGLW\¶KDYHEHHQ
SXWIRUZDUGQRQHRIWKHPKDYHUHFHLYHGXQHTXLYRFDOVXSSRUWDQGWKHUHIRUHDWWKH
PRPHQWZHGRQRWVHHPWRKDYHDVWUDLJKWIRUZDUGDQGSRZHUIXODOWHUQDWLYHPHDQVRI
DVVXULQJTXDOLW\DQGWKXVFRQILUPLQJWKHOHJLWLPDF\RITXDOLWDWLYHUHVHDUFK,QIDFW
VRPHVFKRODUVKDYHHYHQDUJXHGWKDWTXDOLWDWLYHUHVHDUFKKDVVHYHUDOEXLOWLQFULWHULDIRU
SURGXFLQJYDOLGUHVXOWVE\GHILQLWLRQVXFKDVWKHµWKLFNGHVFULSWLRQ¶RIWKHWDUJHWHG
SKHQRPHQRQRUWKHIDFWWKDWWKHUHVXOWVDUHDUULYHGDWWKURXJKDQLWHUDWLYHSURFHVVRI
JRLQJEDFNDQGIRUWKEHWZHHQWKHGDWDDQGWKHDQDO\VLVXQWLODµJRRGQHVVRIILW¶LV
DFKLHYHG+RZHYHUDV/\QFK SRLQWVRXWWKLVDVVXPSWLRQRIDXWRPDWLF
TXDOLW\FRQWUROµZLOOQRWVDWLVI\PDQ\SHRSOH(YDOXDWLRQDXGLHQFHVH[SHFWH[SOLFLW
HYLGHQFHLQVXSSRUWRIYDOLGLW\¶,QGHHGWKHELJSUREOHPZLWKWKHµTXDOLWDWLYHUHVHDUFK
LVYDOLGE\GHILQLWLRQ¶DUJXPHQWLVWKDWDOWKRXJKLWFDQEHPDGHWRVRXQGFRQYLQFLQJ
QRWDOOTXDOLWDWLYHDFFRXQWVDUHHTXDOO\XVHIXOFUHGLEOHRUOHJLWLPDWHDQGLQPDQ\
FDVHVWKHGLIIHUHQFHEHWZHHQDPRUHDQGDOHVVWUXVWZRUWK\DFFRXQWRIDSKHQRPHQRQLV
QRWGXHWRWKHUHVHDUFKHU¶VGLIIHUHQWSHUVSHFWLYHEXWWRVRPHPHWKRGRORJLFDOIDFWRUV

ZKLFKGLVWRUWWKHUHVXOWVVRWKDWWKHVHGRQRWUHIOHFWWKHSKHQRPHQRQLQTXHVWLRQ
DFFXUDWHO\ 0RUVHDQG5LFKDUGV ,QVKRUWQRWHYHU\TXDOLWDWLYHVWXG\LVHTXDOO\
VWDEOHDQGFRUUHFWDQGWKHUHIRUHZHQHHGVWDQGDUGVWREHDEOHWRVLIWWKHZKHDWIURPWKH
FKDII/HWXVILUVWORRNDWWKHµFKDII¶WKDWLVVRPHNH\48$/TXDOLW\FRQFHUQVEHIRUH
ZHORRNDWVRPHUHVSRQVHVWRWKHTXDOLW\LVVXH
7KUHHEDVLFTXDOLW\FRQFHUQVLQTXDOLWDWLYHUHVHDUFK
4XDQWLWDWLYHUHVHDUFKHUVVRPHWLPHVFULWLFL]HWKHTXDOLWDWLYHSDUDGLJPIRUQRWIROORZLQJ
WKHSULQFLSOHVRIWKHµVFLHQWLILFPHWKRG¶ IRUH[DPSOHWKHREMHFWLYHDQGIRUPDOWHVWLQJ
RIK\SRWKHVHV RUKDYLQJWRRVPDOOVDPSOHVL]HVEXWWKHVHFRQFHUQVPLVVWKHSRLQWDV
WKH\LQHIIHFWVD\WKDWWKHSUREOHPZLWKTXDOLWDWLYHUHVHDUFKLVWKDWLWLVQRWTXDQWLWDWLYH
HQRXJK7KHUHDUHKRZHYHUFHUWDLQEDVLFTXDOLW\FRQFHUQVLQ48$/LQTXLULHVWKDWDUH
LQGHSHQGHQWRISDUDGLJPDWLFFRQVLGHUDWLRQV,KDYHIRXQGWKUHHVXFKLVVXHVSDUWLFXODUO\
VDOLHQW
,QVLSLGGDWD)RFXVLQJRQµLQGLYLGXDOPHDQLQJ¶GRHVQRWRIIHUDQ\SURFHGXUHVIRU
GHFLGLQJZKHWKHUWKHSDUWLFXODUPHDQLQJLVLQWHUHVWLQJHQRXJK VLQFHZHDUHQRWWR
MXGJHDUHVSRQGHQW¶VSHUVRQDOSHUFHSWLRQVDQGLQWHUSUHWDWLRQV DQGLILWLVQRW
VXIILFLHQWO\LQWHUHVWLQJWKHQQRPDWWHUKRZWUXWKIXOO\ZHUHIOHFWWKLVPHDQLQJLQWKH
DQDO\VLVZHZLOOREWDLQRQO\ORZTXDOLW\UHVXOWVWKDWDUHµTXLWHVWHUHRW\SLFDODQG
FORVHWRFRPPRQVHQVH¶ 6HDOHHWDO ,QRWKHUZRUGVWKHTXDOLW\RIWKH
DQDO\VLVLVGHSHQGHQWRQWKHTXDOLW\RIWKHRULJLQDOGDWDDQG,DPQRWVXUHZKHWKHULW
LVSRVVLEOHWRGHYHORSH[SOLFLWJXLGHOLQHVIRUMXGJLQJRQHVHWRIFRPSOH[
LGLRV\QFUDWLFPHDQLQJDVEHWWHUWKDQDQRWKHU$V6HDOHHWDOFRQFOXGHSDVWSUDFWLFH
LQTXDOLWDWLYHUHVHDUFKKDVQRWDOZD\VEHHQFRQYLQFLQJLQWKLVUHVSHFWDQGDOWKRXJK
WDNLQJWKHRUHWLFDOVDPSOLQJVHULRXVO\GRHVKHOS VHH6HFWLRQ QRTXDOLWDWLYH
VDPSOLQJSURFHGXUHFDQFRPSOHWHO\SUHYHQWWKHGRFXPHQWDWLRQRIWKHXQH[FLWLQJ
7KLVSUREOHPLVXQLTXHWR48$/LQTXLULHVEHFDXVH48$1UHVHDUFKDGGUHVVHVWKH
FRPPRQDOLW\IRXQGLQODUJHUVDPSOHVLQVWHDGRILQGLYLGXDOPHDQLQJ
4XDOLW\RIWKHUHVHDUFKHU0RUVHDQG5LFKDUGV DUHULJKWZKHQWKH\ZDUQXV

WKDWDQ\VWXG\LVRQO\DVJRRGDVWKHUHVHDUFKHUDQGLQDTXDOLWDWLYHVWXG\WKLVLVVXH
LVSDUWLFXODUO\SURPLQHQWEHFDXVHLQDZD\WKHUHVHDUFKHULVWKHLQVWUXPHQW²VHH
6HFWLRQ7KLVUDLVHVDVHULRXVTXHVWLRQ+RZFDQTXDOLW\FULWHULDDGGUHVVWKH
UHVHDUFKHU¶VVNLOOVWKDWDUHWRDODUJHH[WHQWUHVSRQVLEOHIRUHQVXULQJWKHTXDOLW\DQG
VFRSHRIWKHGDWDDQGWKHLQWHUSUHWDWLRQRIWKHUHVXOWV"$JDLQTXDQWLWDWLYHUHVHDUFK
GRHVQRWKDYHWRIDFHWKLVLVVXHEHFDXVHDJUHDWGHDORIWKHUHVHDUFKHU¶VUROHLV
JXLGHGE\VWDQGDUGL]HGSURFHGXUHV
$QHFGRWDOLVPDQGWKHODFNRITXDOLW\VDIHJXDUGV7KHILQDOTXDOLW\FRQFHUQKDVEHHQ
GHVFULEHGE\6LOYHUPDQ DVIROORZV
TXDOLWDWLYHUHVHDUFKHUVZLWKWKHLULQGHSWKDFFHVVWRVLQJOHFDVHVKDYHWR
RYHUFRPHDVSHFLDOWHPSWDWLRQ+RZDUHWKH\WRFRQYLQFHWKHPVHOYHV DQG
WKHLUDXGLHQFH WKDWWKHLUµILQGLQJV¶DUHJHQXLQHO\EDVHGRQFULWLFDO

LQYHVWLJDWLRQRIDOOWKHLUGDWDDQGGRQRWGHSHQGRQDIHZZHOOFKRVHQ
µH[DPSOHV¶"7KLVLVVRPHWLPHVNQRZQDVWKHSUREOHPRIDQHFGRWDOLVP
,QGHHGVSDFHOLPLWDWLRQVXVXDOO\GRQRWDOORZTXDOLWDWLYHUHVHDUFKHUVWRSURYLGH
PRUHWKDQDIHZH[HPSODU\LQVWDQFHVRIWKHGDWDWKDWKDVOHGWKHPWRWKHLUFRQFOXVLRQD
SUREOHPWKDWLVDJJUDYDWHGE\WKHIDFWWKDWVFKRODUVUDUHO\SURYLGHDQ\MXVWLILFDWLRQIRU
VHOHFWLQJWKHVSHFLILFVDPSOHH[WUDFWV LHGRQRWJLYHDQ\FULWHULDIRUµZLWKLQFDVH
VDPSOLQJ²VHH6HFWLRQ$VDUHVXOW0LOHVDQG+XEHUPDQ KDYH
FRQFOXGHGµ:HGRQRWUHDOO\VHHKRZWKHUHVHDUFKHUJRWIURPSDJHVRIILHOG
QRWHVWRWKHILQDOFRQFOXVLRQVDVVSULQNOHGZLWKYLYLGLOOXVWUDWLRQVDVWKH\PD\EH¶
7KHUHIRUHUHDGHUVRITXDOLWDWLYHVWXGLHVDUHXVXDOO\QRWLQDSRVLWLRQWRMXGJHWKH
V\VWHPDWLFLW\RIWKHDQDO\VLVOHWDORQHWRSURGXFHDQ\SRVVLEOHDOWHUQDWLYH
LQWHUSUHWDWLRQV$VDFRQVHTXHQFHLQWKHDEVHQFHRIDQ\LQEXLOWTXDOLW\VDIHJXDUGVLWLV
XQIRUWXQDWHO\WRRHDV\WRDEXVHWKHSURFHGXUHDQGSURGXFHDFRQYLQFLQJO\TXDOLWDWLYH
OLNHUHSRUWLQZKLFKWKHDXWKRUKDVFKRVHQDIHZTXRWDWLRQVIURPDPXFKODUJHUDQG
PRUHFRPSOH[GDWDEDVHWKDWVXSSRUWKLVKHUSUHFRQFHLYHGDUJXPHQW
5HOLDELOLW\
/HWXVVWDUWRXUGLVFXVVLRQRIVSHFLILF48$/TXDOLW\FULWHULDE\H[DPLQLQJWKHQRWLRQRI
µUHOLDELOLW\¶.LUNDQG0LOOHU SRLQWHGRXWWZRGHFDGHVDJRWKDWWKHPDLQWKUXVWRI
PHWKRGRORJLFDOGHYHORSPHQWLQTXDOLWDWLYHUHVHDUFKKDGEHHQWRZDUGVJUHDWHUYDOLGLW\
DQGUHOLDELOLW\KDGWKHUHIRUHEHHQVRPHZKDWRYHUORRNHG7KLVVLWXDWLRQKDVQRWFKDQJHG
JUHDWO\RYHUWKHSDVWWZRGHFDGHV
5HOLDELOLW\UHIHUVWRWKHµGHJUHHRIFRQVLVWHQF\ZLWKZKLFKLQVWDQFHVDUHDVVLJQHGWR
WKHVDPHFDWHJRU\E\GLIIHUHQWREVHUYHUVRUE\WKHVDPHREVHUYHURQGLIIHUHQW
RFFDVLRQV¶ 6LOYHUPDQ 7KHFRQFHSWRIFRQVLVWHQF\LVDOVRHPSKDVL]HGLQ
.LUNDQG0LOOHU¶V GHILQLWLRQRIUHOLDELOLW\LQILHOGZRUNDVWKHGHJUHHWR
ZKLFKµDQHWKQRJUDSKHUZRXOGH[SHFWWRREWDLQWKHIXQGLQJLIKHRUVKHWULHGDJDLQLQ
WKHVDPHZD\¶WKDWLVWKHGHJUHHWRZKLFKµWKHILQGLQJLVLQGHSHQGHQWRIDFFLGHQWDO
FLUFXPVWDQFHVRIWKHUHVHDUFK¶ S 0RUVHDQG5LFKDUGV¶V GHILQLWLRQ
VXPVXSWKHFRQVLVWHQF\LVVXHZHOOEXWDWWKHVDPHWLPHUHYHDOVZK\TXDOLWDWLYH
UHOLDELOLW\KDVEHHQSOD\HGGRZQLQWKHSDVWµUHOLDELOLW\UHTXLUHVWKDWWKHVDPHUHVXOWV
ZRXOGEHREWDLQHGLIWKHVWXG\ZHUHUHSOLFDWHG¶7KHSUREOHPLVWKDWUHSOLFDWLRQLVQRW
VRPHWKLQJWKDWLVHDV\WRDFKLHYHLQDUHVHDUFKSDUDGLJPZKHUHDQ\FRQFOXVLRQLVLQWKH
HQGMRLQWO\VKDSHGE\WKHUHVSRQGHQWV¶SHUVRQDODFFRXQWVDQGWKHUHVHDUFKHU¶V
VXEMHFWLYHLQWHUSUHWDWLRQRIWKHVHVWRULHV+DYLQJVDLGWKDWLWLVSRVVLEOHWRFRQGXFW
UHOLDELOLW\FKHFNVRIYDULRXVVXESURFHVVHVZLWKLQDTXDOLWDWLYHLQTXLU\IRUH[DPSOHRI
WKHFRGLQJRILQWHUYLHZWUDQVFULSWVE\DVNLQJDVHFRQGFRGHUWRFRGHVHSDUDWHO\D
VL]DEOHSDUWRIWKHWUDQVFULSW HLWKHUXVLQJWKHUHVHDUFKHU¶VFRGLQJWHPSODWHRUJHQHUDWLQJ
WKHFRGHVKLPKHUVHOI DQGWKHQUHYLHZLQJWKHSURSRUWLRQRIDJUHHPHQWVDQG
GLVDJUHHPHQWV
/LQFROQDQG*XED¶VWD[RQRP\RITXDOLW\FULWHULD

,QDVSLULWHGGHQLDORIWKHDOOHJDWLRQVWKDWTXDOLWDWLYHUHVHDUFKLVµVORSS\¶DQGTXDOLWDWLYH
UHVHDUFKHUVUHVSRQGLQGLVFULPLQDWHO\WRWKHµORXGHUEDQJVRUEULJKWHVWOLJKWV¶/LQFROQ
DQG*XED LQWURGXFHGWKHFRQFHSWRIµWUXVWZRUWKLQHVV¶DVTXDOLWDWLYHUHVHDUFKHUV¶
DQVZHUWRµYDOLGLW\¶7KH\SURSRVHGIRXUFRPSRQHQWVWRPDNHXSWUXVWZRUWKLQHVV
D µ&UHGLELOLW\¶RUWKHµWUXWKYDOXH¶RIDVWXG\ZKLFKLVWKHTXDOLWDWLYHFRXQWHUSDUWRI
µLQWHUQDOYDOLGLW\¶
E µ7UDQVIHUDELOLW\¶RUWKHµDSSOLFDELOLW\¶RIWKHUHVXOWVWRRWKHUFRQWH[WVZKLFKLVWKH
TXDOLWDWLYHFRXQWHUSDUWRIµH[WHUQDOYDOLGLW\¶
F µ'HSHQGDELOLW\¶RUWKHµFRQVLVWHQF\¶RIWKHILQGLQJVZKLFKLVWKHTXDOLWDWLYH
FRXQWHUSDUWRIµUHOLDELOLW\¶
G µ&RQILUPDELOLW\¶RUWKHQHXWUDOLW\RIWKHILQGLQJVZKLFKLVWKHTXDOLWDWLYH
FRXQWHUSDUWRIµREMHFWLYLW\¶
7KHVHWHUPVDUHVRPHWLPHVUHIHUUHGWRDVµSDUDOOHOFULWHULD¶EHFDXVHRIWKHLU
FRUUHVSRQGLQJTXDQWLWDWLYHFRXQWHUSDUWV$OWKRXJKWKHVHSDUDOOHOFULWHULDKDYHEHHQ
HPEUDFHGE\PDQ\TXDOLWDWLYHVFKRODUVWKH\KDYHDOVREHHQFULWLFL]HGRQVHYHUDO
JURXQGV²VHH0RUURZ0\SHUVRQDOFRQFHUQLVWKDWWKHSUROLIHUDWLRQRI
WHUPLQRORJ\LVOLNHO\WRPDNHWKHSLFWXUHPRUHFRQIXVLQJ,EHOLHYHWKDWLWLVSRVVLEOHWR
LGHQWLI\VWDQGDUGVIRUTXDOLWDWLYHUHVHDUFKE\LGHQWLI\LQJYDULRXVUHOHYDQWIDFHWVRIWKH
WUDGLWLRQDOFRQFHSWVRIYDOLGLW\DQGUHOLDELOLW\DSUDFWLFHIROORZHGE\0D[ZHOO
GHVFULEHGEHORZ
0D[ZHOO¶VWD[RQRP\RIYDOLGLW\LQTXDOLWDWLYHUHVHDUFK
,QWKHLQWURGXFWLRQRIKLVLQIOXHQWLDOW\SRORJ\RIYDOLGLW\LQTXDOLWDWLYHUHVHDUFK
0D[ZHOO H[SOLFLWO\VWDWHGWKDWKHGLGQRWWKLQNWKDW48$/DQG48$1
DSSURDFKHVWRYDOLGLW\ZHUHLQFRPSDWLEOH+HHYHQVXJJHVWHGWKDWKLVDQDO\VLVPLJKW
DOVRKDYHLPSOLFDWLRQVIRUWKHFRQFHSWRIYDOLGLW\LQTXDQWLWDWLYHDQGH[SHULPHQWDO
UHVHDUFK/HWXVH[DPLQHWKHILYHFRPSRQHQWVRIKLVSURSRVHGV\VWHP )RUDFULWLFDO
UHYLHZVHH:LQWHU
µ'HVFULSWLYHYDOLGLW\¶FRQFHUQVWKHIDFWXDODFFXUDF\RIWKHUHVHDUFKHU¶VDFFRXQW
0D[ZHOO UHJDUGVWKLVDVWKHSULPDU\DVSHFWRIYDOLGLW\EHFDXVHDOOWKHRWKHU
YDOLGLW\FDWHJRULHVDUHGHSHQGHQWRQLW,WUHIHUVWRZKDWWKHUHVHDUFKHUKLPKHUVHOI
KDVH[SHULHQFHGDQGDOVRWRµVHFRQGDU\¶DFFRXQWVRIWKLQJVWKDWFRXOGLQSULQFLSOH
KDYHEHHQREVHUYHGEXWWKDWZHUHLQIHUUHGIURPRWKHUGDWD2QHXVHIXOVWUDWHJ\IRU
HQVXULQJWKLVYDOLGLW\LVµLQYHVWLJDWRUWULDQJXODWLRQ¶WKDWLVXVLQJPXOWLSOH
LQYHVWLJDWRUVWRFROOHFWDQGLQWHUSUHWWKHGDWD
µ,QWHUSUHWLYHYDOLGLW\¶GHVFULSWLYHYDOLGLW\ZDVFDOOHGDSULPDU\YDOLGLW\GLPHQVLRQ
EHFDXVHLWXQGHUOLHVDOORWKHUYDOLGLW\DVSHFWVDQGQRWEHFDXVH0D[ZHOO
FRQVLGHUHGGHVFULSWLYHQHVVWKHPDLQFRQFHUQIRUTXDOLWDWLYHUHVHDUFK,QVWHDGKH
DUJXHGJRRGTXDOLWDWLYHUHVHDUFKIRFXVHVRQZKDWWKHYDULRXVWDQJLEOHHYHQWV
EHKDYLRXUVRUREMHFWVµPHDQ¶WRWKHSDUWLFLSDQWV,QWHUSUHWLYHYDOLGLW\WKHQIRFXVHV
RQWKHTXDOLW\RIWKHSRUWUD\DORIWKLVSDUWLFLSDQWSHUVSHFWLYH$QREYLRXVVWUDWHJ\WR

HQVXUHWKLVYDOLGLW\LVWRREWDLQSDUWLFLSDQWIHHGEDFNRUPHPEHUFKHFNLQJZKLFK
LQYROYHGLVFXVVLQJWKHILQGLQJVZLWKWKHSDUWLFLSDQWV )RUPRUHGHWDLOVVHH6HFWLRQ

µ7KHRUHWLFDOYDOLGLW\¶FRUUHVSRQGVWRVRPHH[WHQWWRWKHLQWHUQDOYDOLGLW\RIWKH
UHVHDUFKDVLWFRQFHUQVZKHWKHUWKHUHVHDUFKHU¶VDFFRXQWLQFOXGHVDQDSSURSULDWH
OHYHORIWKHRUHWLFDODEVWUDFWLRQDQGKRZZHOOWKLVWKHRU\H[SODLQVRUGHVFULEHVWKH
SKHQRPHQRQLQTXHVWLRQ
µ*HQHUDOL]DELOLW\¶LWLVLQWHUHVWLQJWKDWLQODEHOOLQJWKLVFDWHJRU\0D[ZHOO
GLGQRWXVHDSKUDVHFRQWDLQLQJWKHZRUGµYDOLGLW\¶HYHQWKRXJKWKHUHZRXOGKDYH
EHHQDQREYLRXVWHUPWRXVHH[WHUQDOYDOLGLW\7KLVLVEHFDXVHKHIXUWKHUGLYLGHG
µJHQHUDOL]DELOLW\¶LQWRµLQWHUQDOJHQHUDOL]DELOLW\¶DQGµH[WHUQDOJHQHUDOL]DELOLW\¶DQG
WKLVGLYLVLRQZRXOGQRWKDYHZRUNHGZLWKWKHWHUPµH[WHUQDOYDOLGLW\¶ LHZH
FDQQRWUHDOO\KDYHµLQWHUQDOH[WHUQDOYDOLGLW\¶ %RWKDVSHFWVRIJHQHUDOL]DELOLW\
UHIHUWRWKHH[WHQVLRQRIWKHDFFRXQWWRSHUVRQVWLPHVRUVHWWLQJVRWKHUWKDQWKRVH
GLUHFWO\VWXGLHGEXWµLQWHUQDOJHQHUDOL]DELOLW\¶FRQFHUQVJHQHUDOL]LQJZLWKLQWKH
FRPPXQLW\RULQVWLWXWLRQREVHUYHGZKHUHDVµH[WHUQDOJHQHUDOL]DELOLW\¶UHIHUVWR
JHQHUDOL]LQJWRRWKHUFRPPXQLWLHVRULQVWLWXWLRQV
'XII SRLQWVRXWWKDWPDQ\48$/UHVHDUFKHUVYLHZWKHWHUP
JHQHUDOL]DELOLW\VXVSLFLRXVO\EHFDXVHLWLVUHPLQLVFHQWRI48$1PHWKRGRORJ\LQ
ZKLFKWKHFDSDFLW\WRJHQHUDOL]HIURPWKHVDPSOHWRVRPHZLGHUSRSXODWLRQLVRQH
RIWKHNH\FRQFHUQV7KLVLVZKHUH0D[ZHOO¶V GLVWLQFWLRQEHWZHHQLQWHUQDO
DQGH[WHUQDOJHQHUDOL]DELOLW\LVHQOLJKWHQLQJKHDJUHHVWKDWJHQHUDOL]DELOLW\SOD\VD
GLIIHUHQWUROHLQ48$/UHVHDUFKWKDQLWGRHVLQ48$1UHVHDUFKDQGWKHUHIRUH
LQWHUQDOJHQHUDOL]DELOLW\LVIDUPRUHLPSRUWDQWIRUPRVWTXDOLWDWLYHUHVHDUFKHUVWKDQ
LVH[WHUQDOJHQHUDOL]DELOLW\+HIXUWKHUH[SODLQVWKDWJHQHUDOL]DWLRQLQTXDOLWDWLYH
UHVHDUFKXVXDOO\WDNHVSODFHWKURXJKWKHGHYHORSPHQWRIDWKHRU\GHULYHGIURPWKH
SDUWLFXODUSHUVRQVRUVLWXDWLRQVVWXGLHGZKLFKKHOSVWRPDNHVHQVHRIRWKHU
VLWXDWLRQV,QRWKHUZRUGVHYHQLIWKHSDUWLFXODUVRIDVWXG\GRQRWJHQHUDOL]HWKH
PDLQLGHDVDQGWKHSURFHVVREVHUYHGPLJKW7KLVLVZK\HYHQDVLQJOHVSHFLDOO\
VHOHFWHGFDVHFDQEHLOOXPLQDWLQJ$XVHIXOVWUDWHJ\WRH[DPLQHJHQHUDOL]DELOLW\
UHFRPPHQGHGE\'XIILVWRLQFOXGHLQWKHTXDOLWDWLYHDFFRXQWWKHSDUWLFLSDQWV¶RZQ
MXGJPHQWVDERXWWKHJHQHUDOL]DELOLW\RIWKHWDUJHWHGLVVXHSKHQRPHQRQ
µ(YDOXDWLYHYDOLGLW\¶UHIHUVWRWKHDVVHVVPHQWRIKRZWKHUHVHDUFKHUHYDOXDWHVWKH
SKHQRPHQRQVWXGLHG IRUH[DPSOHLQWHUPVRIXVHIXOQHVVSUDFWLFDELOLW\
GHVLUDELOLW\ WKDWLVKRZDFFXUDWHO\WKHUHVHDUFKDFFRXQWDVVLJQVYDOXHMXGJPHQWV
WRWKHSKHQRPHQRQ7KXVWKLVYDOLGLW\DVSHFWFRQFHUQVWKHLPSOLFLWRUH[SOLFLWXVH
RIDQHYDOXDWLRQIUDPHZRUN IRUH[DPSOHHWKLFDORUPRUDOMXGJHPHQWV LQD
TXDOLWDWLYHDFFRXQWH[DPLQLQJKRZWKHHYDOXDWLYHFODLPVILWWKHREVHUYHG
SKHQRPHQRQ(YDOXDWLYHYDOLGLW\LVJDLQLQJLPSRUWDQFHQRZDGD\VZLWKYDULRXV
µFULWLFDO¶WKHRULHVEHFRPLQJLQFUHDVLQJO\SURPLQHQWLQWKHVRFLDOVFLHQFHVDQGDOVR
LQDSSOLHGOLQJXLVWLFV

6WUDWHJLHVWRHQVXUHYDOLGLW\LQTXDOLWDWLYHUHVHDUFK
7KURXJKRXWWKLVFKDSWHU,KDYHEHHQXQFHUWDLQDERXWKRZPXFKVSDFHWRGHYRWHWR
GHVFULELQJWKHYDULRXVV\VWHPVRIYDOLGLW\DQGUHOLDELOLW\EHFDXVHWKHVHW\SRORJLHVDUH
DGPLWWHGO\QRWWRRSUDFWLFDOLQWKHPVHOYHV+RZHYHU,DJUHHZLWK0D[ZHOO¶V
DUJXPHQWWKDWVXFKW\SRORJLHVRIIHUDXVHIXOIUDPHZRUNIRUWKLQNLQJDERXWWKHQDWXUHRI
WKHWKUHDWVWRYDOLGLW\DQGWKHSRVVLEOHZD\VWKDWVSHFLILFWKUHDWVPLJKWEHDGGUHVVHG,Q
WKHIROORZLQJ,OLVWWKHPRVWFRPPRQVWUDWHJLHVXVHGWRHOLPLQDWHRUFRQWUROYDOLGLW\
WKUHDWVDQGWRJHQHUDWHWUXVWZRUWKLQHVV
%XLOGLQJXSDQLPDJHRIUHVHDUFKHULQWHJULW\
,WLVP\FRQYLFWLRQWKDWWKHPRVWLPSRUWDQWVWUDWHJ\WRHQVXUHWKHWUXVWZRUWKLQHVVRID
SURMHFWLVWRFUHDWHLQWKHDXGLHQFHDQLPDJHRIWKHUHVHDUFKHUDVDVFKRODUZLWK
SULQFLSOHGVWDQGDUGVDQGLQWHJULW\$WWKHHQGRIWKHGD\UHDGHUVZLOOGHFLGHZKHWKHU
WKH\KDYHFRQILGHQFHLQRQH¶VUHVHDUFKQRWE\WDNLQJVWRFNRIWKHYDULRXVYDOLGLW\
DUJXPHQWVEXWE\IRUPLQJDQRSLQLRQDERXWWKHLQYHVWLJDWRU¶VRYHUDOOUHVHDUFKLQWHJULW\
7KLVLPDJHRILQWHJULW\LVPDGHXSRIVHYHUDOVPDOOFRPSRQHQWVEXWWKHUHDUHFHUWDLQ
VWUDWHJLHVWKDWDUHSDUWLFXODUO\KHOSIXOLQVKRZLQJXSWKHUHVHDUFKHU¶VKLJKVWDQGDUGV
SURYLGHGRIFRXUVHWKRVHH[LVW
x /HDYLQJDQDXGLWWUDLO%\RIIHULQJDGHWDLOHGDQGUHIOHFWLYHDFFRXQWRIWKHVWHSV
WDNHQWRDFKLHYHWKHUHVXOWV²LQFOXGLQJWKHLWHUDWLYHPRYHVLQGDWDFROOHFWLRQDQG
DQDO\VLVWKHGHYHORSPHQWRIWKHFRGLQJIUDPHVDQGWKHHPHUJHQFHRIWKHPDLQ
WKHPHV²UHVHDUFKHUVFDQJHQHUDWHUHDGHUFRQILGHQFHLQWKHSULQFLSOHGZHOO
JURXQGHGDQGWKRURXJKQDWXUHRIWKHUHVHDUFKSURFHVV 7KLVTXHVWLRQLVIXUWKHU
GLVFXVVHGLQ6HFWLRQRQUHVHDUFKMRXUQDOVDQGLQ6HFWLRQRQZULWLQJXS
TXDOLWDWLYHUHSRUWV $V+ROOLGD\ VXPPDUL]HV
$VLQDOOUHVHDUFKDPDMRUDUHDRIDFFRXQWDELOLW\PXVWEHSURFHGXUHDQG
ZKHQWKHFKRLFHVDUHPRUHRSHQWKHSURFHGXUHPXVWEHPRUHWUDQVSDUHQWVR
WKDWWKHVFUXWLQL]HUVRIWKHUHVHDUFKFDQDVVHVVWKHDSSURSULDWHQHVVRIWKH
UHVHDUFKHU¶VFKRLFHV
x &RQWH[WXDOL]DWLRQDQGWKLFNGHVFULSWLRQ3UHVHQWLQJWKHILQGLQJVLQULFK
FRQWH[WXDOL]HGGHWDLOKHOSVWKHUHDGHUWRLGHQWLI\ZLWKWKHSURMHFWDQGWKXVFRPHRQ
ERDUG
x ,GHQWLI\LQJSRWHQWLDOUHVHDUFKHUELDV*LYHQWKHLPSRUWDQWUROHRIWKHUHVHDUFKHULQ
HYHU\VWDJHRIDTXDOLWDWLYHVWXG\LGHQWLI\LQJWKHUHVHDUFKHU¶VRZQELDVHVLV
REYLRXVO\DQLPSRUWDQWLVVXHLQDQLQTXLU\DQGLWDOVRFUHDWHVDQRSHQDQGKRQHVW
QDUUDWLYHWKDWZLOOUHVRQDWHZHOOZLWKWKHDXGLHQFH &UHVZHOO
x ([DPLQLQJRXWOLHUVH[WUHPHRUQHJDWLYHFDVHVDQGDOWHUQDWLYHH[SODQDWLRQV1R
UHVHDUFKVWXG\LVSHUIHFWDQGWKHUHDGHUVNQRZWKLV7KHUHIRUHH[SOLFLWO\SRLQWLQJ
RXWDQGGLVFXVVLQJDVSHFWVRIWKHVWXG\WKDWUXQFRXQWHUWRWKHILQDOFRQFOXVLRQLV
XVXDOO\QRWVHHQDVDZHDNQHVVEXWDGGVWRWKHFUHGLELOLW\RIWKHUHVHDUFKHU

6LPLODUO\JLYLQJDOWHUQDWLYHH[SODQDWLRQVDIDLUKHDULQJEHIRUHZHGLVPLVVWKHP
DOVRKHOSVWREXLOGFRQILGHQFHLQRXUUHVXOWV
9DOLGLW\UHOLDELOLW\FKHFNV
7KHSUHYLRXVVWUDWHJLHVGLGQRWDFWXDOO\DGGWRWKHYDOLGLW\RIWKHGDWDEXWRQO\KHOSHG
WRFRQYLQFHWKHDXGLHQFHWKDWWKHVWXG\ZDVYDOLG SURYLGHGLWZDV 7KHIROORZLQJ
VWUDWHJLHVDUHGLIIHUHQWLQWKDWWKH\LQYROYHVWHSVWDNHQE\WKHUHVHDUFKHUGXULQJWKH
SURMHFWWRHQVXUHYDOLGLW\DQGLWLVDSSURSULDWHWRVWDUWZLWKWKHSRSXODUWHFKQLTXHRI
EXLOGLQJLQWRWKHVWXG\YDULRXVYDOLGLW\FKHFNV
x 5HVSRQGHQWIHHGEDFN RUµUHVSRQGHQWYDOLGDWLRQ¶RUµPHPEHUFKHFNLQJ¶ %HFDXVHRI

WKHHPSKDVLVSODFHGLQTXDOLWDWLYHUHVHDUFKRQXQFRYHULQJSDUWLFLSDQWPHDQLQJLWLV
DQREYLRXVVWUDWHJ\WRLQYROYHWKHSDUWLFLSDQWVWKHPVHOYHVLQFRPPHQWLQJRQWKH
FRQFOXVLRQVRIWKHVWXG\7KH\FDQIRUH[DPSOHUHDGDQHDUO\GUDIWRIWKHUHVHDUFK
UHSRUWRUOLVWHQWRDSUHVHQWDWLRQRIVRPHWHQWDWLYHUHVXOWVRUWKHPHVDQGFDQWKHQ
H[SUHVVWKHLUYLHZVLQZKDWLVFDOOHGDµYDOLGDWLRQLQWHUYLHZ¶,IWKHUHLVDJUHHPHQW
EHWZHHQWKHUHVHDUFKHUDQGWKHSDUWLFLSDQWVWKHVWXG\¶VYDOLGLW\LVLQGHHG
UHLQIRUFHG+RZHYHUVHYHUDOVFKRODUVKDYHUDLVHGWKHTXHVWLRQRIKRZDQ\
GLVDJUHHPHQWVVKRXOGEHLQWHUSUHWHG,QWHUPVRIGHVFULSWLYHYDOLGLW\VXFKFKHFNV
DUHXQGRXEWHGO\XVHIXODQGDVZHKDYHVHHQLQWKHSUHYLRXVVHFWLRQUHVSRQGHQW
YDOLGDWLRQFDQDOVRHQKDQFHJHQHUDOL]DELOLW\%XWZKRPVKDOOZHOLVWHQWRZKHQ
WKHUHDUHFRQFHUQVDERXWWKHLQWHUSUHWLYHYDOLGLW\RIWKHUHVXOWV"(YHQWKRXJKWKH
SDUWLFLSDQWVDUHFOHDUO\WKHµLQVLGHUV¶WKHUHLVQRUHDVRQWRDVVXPHWKDWWKH\FDQ
LQWHUSUHWWKHLURZQH[SHULHQFHVRUFLUFXPVWDQFHVFRUUHFWO\²LQIDPLO\DUJXPHQWV
IRUH[DPSOHLQVLGHUVRIWHQKDYHFRQIOLFWLQJYLHZVDERXWWKHVDPHSKHQRPHQD
7KXVWKHEHVWZD\RIKDQGOLQJVXFKYDOLGLW\FKHFNVLVWRWUHDWWKHPDVIXUWKHUGDWD
WKDWFDQFRQWULEXWHWRWKHRYHUDOOYDOLGLW\DUJXPHQWDIWHUSURSHULQWHUSUHWDWLRQ
x 3HHUFKHFNLQJ4XDOLWDWLYHVWXGLHVRIWHQLQFOXGHUHOLDELOLW\FKHFNVSHUIRUPHGE\
SHHUV7KH\DOZD\VLQYROYHDVNLQJDFROOHDJXHWRSHUIRUPVRPHDVSHFWRIWKH
UHVHDUFKHU¶VUROH²XVXDOO\GHYHORSLQJRUWHVWLQJVRPHFRGLQJVFKHPHEXWWKH\FDQ
DOVRLQYROYHSHUIRUPLQJRWKHUDFWLYLWLHVVXFKDVFDUU\LQJRXWDQREVHUYDWLRQWDVN²
DQGWKHQFRPSDULQJWKHFRUUHVSRQGHQFHEHWZHHQWKHWZRVHWVRIRXWFRPHV7KLVLVD
YHU\XVHIXOVWUDWHJ\EHFDXVHHYHQORZFRUUHVSRQGHQFHFDQVHUYHDVXVHIXOIHHGEDFN
IRUWKHIXUWKHUFRXUVHRIWKHVWXG\EXWXQIRUWXQDWHO\LWLVRIWHQGLIILFXOWWRILQG
VRPHRQHZKRLVERWKFRPSHWHQWDQGUHDG\WRHQJDJHLQWKLVWLPHFRQVXPLQJ
DFWLYLW\
5HVHDUFKGHVLJQEDVHGVWUDWHJLHV
6WUDWHJLHVFRQFHUQLQJDVWXG\¶VUHVHDUFKGHVLJQFDQSURYLGHWKHPRVWFRQYLQFLQJ
HYLGHQFHDERXWWKHYDOLGLW\RIWKHUHVHDUFKDVWKH\DUHDQRUJDQLFSDUWRIWKHSURMHFW
UDWKHUWKDQEHLQJµDGGRQV¶,QDZD\WKHVHSUDFWLFHVDUHQRWQHFHVVDULO\µVWUDWHJLF
DFWLRQV¶EXWVLPSO\H[DPSOHVRIJRRGUHVHDUFKSUDFWLFH

x 0HWKRGDQGGDWDWULDQJXODWLRQ7KHFRQFHSWRIµWULDQJXODWLRQ¶LQYROYHVXVLQJ
PXOWLSOHPHWKRGVVRXUFHVRUSHUVSHFWLYHVLQDUHVHDUFKSURMHFW WREHGLVFXVVHGLQ
PRUHGHWDLOZKHQGHVFULELQJPL[HGPHWKRGVUHVHDUFKLQ6HFWLRQDQGDOVRLQ
&KDSWHU 7ULDQJXODWLRQKDVEHHQWUDGLWLRQDOO\VHHQDVRQHRIWKHPRVWHIILFLHQW
ZD\VRIUHGXFLQJWKHFKDQFHRIV\VWHPDWLFELDVLQDTXDOLWDWLYHVWXG\EHFDXVHLIZH
FRPHWRWKHVDPHFRQFOXVLRQDERXWDSKHQRPHQRQXVLQJDGLIIHUHQWGDWD
FROOHFWLRQDQDO\VLVPHWKRGRUDGLIIHUHQWSDUWLFLSDQWVDPSOHWKHFRQYHUJHQFHRIIHUV
VWURQJYDOLGLW\HYLGHQFH+RZHYHULWOHDYHVWKHVDPHTXHVWLRQRSHQDVWKHRQH
DOUHDG\UDLVHGZLWKUHJDUGWRSDUWLFLSDQWIHHGEDFNKRZVKDOOZHLQWHUSUHWDQ\
HPHUJLQJGLVDJUHHPHQWEHWZHHQWKHFRUUHVSRQGLQJUHVXOWV"
x 3URORQJHGHQJDJHPHQWDQGSHUVLVWHQWREVHUYDWLRQ5HVHDUFKGHVLJQVWKDWHPSKDVL]H
WKHTXDQWLW\RIHQJDJHPHQWZLWKWKHWDUJHWFRPPXQLW\SKHQRPHQRQFDUU\PRUH
IDFHYDOLGLW\0RVWSHRSOHZRXOGIRUH[DPSOHWUHDWDQDFFRXQWE\DQHWKQRJUDSKHU
ZKRKDVVSHQW\HDUVVWXG\LQJDFRPPXQLW\LQKHUHQWO\YDOLGHYHQWKRXJKWKLV
PD\QRWQHFHVVDULO\EHVR IRUH[DPSOHLIWKHREVHUYHUKDVµJRQHQDWLYH¶
x /RQJLWXGLQDOUHVHDUFKGHVLJQ'XII DUJXHVWKDWORQJLWXGLQDOVWXGLHVKDYHWKH
SRWHQWLDOWRLQFUHDVHWKHYDOLGLW\RIWKHLQIHUHQFHVWKDWFDQEHGUDZQIURPWKHP
EHFDXVHWKH\FDQUHYHDOYDULRXVGHYHORSPHQWDOSDWKZD\VDQGFDQDOVRGRFXPHQW
GLIIHUHQWW\SHVRILQWHUDFWLRQVRYHUWLPH/ORQJLWXGLQDOVWXGLHVDUHGHVFULEHGLQ
&KDSWHU
4XDOLW\FULWHULDLQPL[HGPHWKRGVUHVHDUFK
:HKDYHVHHQLQWKLVFKDSWHUWKDWTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKDUHDVVRFLDWHG
ZLWKVRPHZKDWGLIIHUHQWTXDOLW\FULWHULDDQGHYHQWKRXJKEURDGTXDOLW\GLPHQVLRQV
PLJKWEHGHVFULEHGXVLQJWKHXQLIRUPGLFKRWRP\RIµUHOLDELOLW\¶DQGµYDOLGLW\¶WKH
VSHFLILFGHWDLOVLQKRZWRRSHUDWLRQDOL]HWKHVHFRQFHSWVVKRZYDULDQFH6RZKDWVKDOO
ZHGRLQPL[HGPHWKRGVUHVHDUFK"+RZFDQZHVDWLVI\ERWK48$/DQG48$1TXDOLW\
FULWHULDLQDVLQJOHVWXG\"7KLVLVDQLPSRUWDQWTXHVWLRQEHFDXVHPL[HGPHWKRGV
UHVHDUFKLVDUHODWLYHO\QHZDSSURDFKDQGWKHUHIRUHUHVHDUFKHUVDGRSWLQJPL[HGGHVLJQV
VKRXOGEHSDUWLFXODUO\FDUHIXOWRGHIHQGWKHPHWKRGVWKH\DUHHPSOR\LQJ
7HGGOLHDQG7DVKDNNRUL SURSRVHGWKDWEHFDXVHPL[HGPHWKRGVUHVHDUFKLVWKH

WKLUGPDLQJHQHUDOUHVHDUFKDSSURDFKLWVKRXOGKDYHLWVRZQWHUPLQRORJ\IRUTXDOLW\
FULWHULD$VDUJXHGHDUOLHU,DPQRWFRQYLQFHGWKDWFUHDWLQJQHZWHFKQLFDOWHUPVZLOO
PDNHTXDOLW\VWDQGDUGVFOHDUHURUPRUHDFFHSWDEOHLQVWHDG,VXJJHVWWKDWZHFRQVLGHU
WKUHHVSHFLILFDVSHFWVRIWKHTXDOLW\RIPL[HGPHWKRGVUHVHDUFKVHSDUDWHO\ D WKH
UDWLRQDOHIRUPL[LQJPHWKRGVLQJHQHUDO E WKHUDWLRQDOHIRUWKHVSHFLILFPL[HGGHVLJQ
DSSOLHGLQFOXGLQJWKHFKRLFHRIWKHSDUWLFXODUPHWKRGVDQG F WKHTXDOLW\RIWKH
VSHFLILFPHWKRGVPDNLQJXSWKHVWXG\
7KHUDWLRQDOHIRUPL[LQJPHWKRGV

0L[HGPHWKRGVUHVHDUFKDVDQH[SOLFLWO\PDUNHGUHVHDUFKDSSURDFKLVVWLOOLQLWVLQIDQF\
DQGWKHUHIRUHVFKRODUVDSSO\LQJWKHSDUDGLJPQHHGWRMXVWLI\WKHLUFKRLFHFOHDUO\ 7RD
OHVVHUGHJUHHTXDOLWDWLYHUHVHDUFKHUVDOVRIDFHWKLVWDVNHVSHFLDOO\LQFRQWHQWDUHDVDQG
UHVHDUFKHQYLURQPHQWVZKHUHWKHUHLVVWLOODGRPLQDQFHRITXDQWLWDWLYHPHWKRGRORJ\
7KHJHQHUDODUJXPHQWVH[SODLQLQJZK\PL[HGPHWKRGVUHVHDUFKPLJKWEHXVHIXOZHUH
SUHVHQWHGLQ&KDSWHU 6HFWLRQ DQG&KDSWHUSURYLGHVIXUWKHUGHWDLOVDERXWWKH
SRVVLEOHFRPSDWLELOLW\RIWKH48$/DQG48$1SDUDGLJPV7KHLPSRUWDQWSRLQWWR
HPSKDVL]HLVWKDWDPL[HGPHWKRGVLQTXLU\RIIHUVDSRWHQWLDOO\PRUHFRPSUHKHQVLYH
PHDQVRIOHJLWLPL]LQJILQGLQJVWKDQGRHLWKHU48$/RU48$1PHWKRGVDORQHE\
DOORZLQJLQYHVWLJDWRUVWRDVVHVVLQIRUPDWLRQIURPERWKGDWDW\SHV:HVDZLQWKH
SUHYLRXVVHFWLRQWKDWWULDQJXODWLRQKDVWUDGLWLRQDOO\EHHQVHHQDVDZD\RIHQVXULQJ
UHVHDUFKYDOLGLW\DQGLWFDQEHDUJXHGWKDWWKHYDOLGLW\DUJXPHQWIRUPL[HGPHWKRGV
UHVHDUFKFDQFRPELQHWKHYDOLGLW\HYLGHQFHRIIHUHGE\WKH48$/DQGWKH48$1
FRPSRQHQWVVHSDUDWHO\
7KHµGHVLJQYDOLGLW\¶RIWKHVWXG\
$OWKRXJK,KDYHDUJXHGDERYHDJDLQVWLQWURGXFLQJQHZQRPHQFODWXUHIRUTXDOLW\
FULWHULDWKHWHUPµGHVLJQYDOLGLW\¶ 7HGGOLHDQG7DVKDNNRUL DSSHDUVWREHUHOHYDQW
DQGQHFHVVDU\EHFDXVHLWFRQFHUQVDQHZDVSHFWRILQWHUQDOYDOLGLW\VSHFLILFWRPL[HG
PHWKRGVUHVHDUFK,WUHIHUVWRWKHH[WHQWWRZKLFKWKH48$/DQG48$1FRPSRQHQWVRI
DPL[HGPHWKRGVVWXG\DUHFRPELQHGRULQWHJUDWHGLQDZD\WKDWWKHRYHUDOOGHVLJQ
GLVSOD\VFRPSOHPHQWDU\VWUHQJWKVDQGQRQRYHUODSSLQJZHDNQHVVHVRIWKHFRQVWLWXHQW
PHWKRGV 6HHDOVR%UHZHUDQG+XQWHU7DVKDNNRULDQG7HGGOLH
,QFODLPLQJJRRGGHVLJQYDOLGLW\UHVHDUFKHUVQHHGWRSUHVHQWHYLGHQFHIRUWZR
TXDOLW\DVSHFWVILUVWWKH\KDYHWRMXVWLI\WKHFKRLFHRIWKHVSHFLILFUHVHDUFKPHWKRGV
WKDWDUHFRPELQHGLQWKHVWXG\,QOLQHZLWKWKHSUDJPDWLFQDWXUHRIPL[HGPHWKRGV
UHVHDUFKWKHPDLQUDWLRQDOHKHUHLVOLNHO\WREHFHQWUHGDURXQGWKHPDWFKRIWKHUHVHDUFK
TXHVWLRQWRSLFZLWKWKHPHWKRGDUJXLQJIRUDµILWQHVVIRUSXUSRVH¶VHOHFWLRQ &KDSWHU
RIIHUVDVXPPDU\RIWKHFRQVLGHUDWLRQVWKDWDUHLQYROYHGLQFKRRVLQJDQDSSURSULDWH
PHWKRG 6HFRQGUHVHDUFKHUVQHHGWRGHPRQVWUDWHWKDWWKHPL[HGGHVLJQGLVSOD\V
HQKDQFHGYDOLGLW\UHODWLYHWRWKHFRQVWLWXHQWPHWKRGV&KDSWHUSUHVHQWVVHYHUDOPL[HG
PHWKRGVGHVLJQRSWLRQVZKRVHH[SOLFLWSXUSRVHLVWRRIIHUHQKDQFHGUHVHDUFK
FKDUDFWHULVWLFVLQWHUPVRIHLWKHULQWHUQDORUH[WHUQDOYDOLGLW\,QDGGLWLRQ&KDSWHU
GHVFULEHVVHYHUDOGDWDDQDO\WLFDOVHTXHQFHVZKRVHDLPLVWRDGGJUHDWHUOHJLWLPDF\WR
WKHUHVHDUFKHU¶VFRQFOXVLRQV IRUH[DPSOHVXEMHFWLQJµTXDQWLWL]HG¶GDWDWRVWDWLVWLFDO
DQDO\VLVLQDVWXG\ZKLFKZDVLQLWLDOO\EDVHGRQTXDOLWDWLYHGDWD
7KHTXDOLW\RIWKHVSHFLILFPHWKRGV
7KHVSHFLILFPHWKRGVWKDWDUHFRPELQHGLQPL[HGPHWKRGVUHVHDUFKDUH²REYLRXVO\²
HLWKHUTXDOLWDWLYHRUTXDQWLWDWLYHDQGWKHUHIRUHWKHTXDOLW\SULQFLSOHVGHVFULEHGHDUOLHULQ
WKLVFKDSWHUDSSO\WRWKHP6HYHUDOPL[HGPHWKRGVGHVLJQVGLVSOD\DGRPLQDQWPHWKRG
VHH&KDSWHU DQGLQVXFKFDVHVPRVWRIWKHHYLGHQFHLQFOXGHGLQWKHYDOLGLW\

DUJXPHQWZLOOQHHGWREHLQDFFRUGDQFHZLWKWKHTXDOLW\VWDQGDUGVRIWKHSDUWLFXODU
SDUDGLJP
5HVHDUFKHWKLFV
$Q\TXDOLWDWLYHUHVHDUFKHUZKRLVQRWDVOHHSSRQGHUVPRUDODQGHWKLFDO
TXHVWLRQV
0LOHVDQG+XEHUPDQ
6RFLDOUHVHDUFK²LQFOXGLQJUHVHDUFKLQHGXFDWLRQ²FRQFHUQVSHRSOH¶VOLYHVLQWKHVRFLDO
ZRUOGDQGWKHUHIRUHLWLQHYLWDEO\LQYROYHVHWKLFDOLVVXHV$V3XQFK SRLQWVRXW
VXFKLVVXHVDUHPRUHDFXWHLQ48$/WKDQLQ48$1DSSURDFKHVEHFDXVHTXDOLWDWLYH
UHVHDUFKRIWHQLQWUXGHVPRUHLQWRWKHKXPDQSULYDWHVSKHUHLWLVLQKHUHQWO\LQWHUHVWHGLQ
SHRSOH¶VSHUVRQDOYLHZVDQGRIWHQWDUJHWVVHQVLWLYHRULQWLPDWHPDWWHUV7KXVWKH
JURZLQJUHFRJQLWLRQDQGXWLOL]DWLRQRITXDOLWDWLYHPHWKRGVLQFRQWHPSRUDU\DSSOLHG
OLQJXLVWLFUHVHDUFKUDLVHVWKHµHWKLFDOVWDNHV¶:HKDYHWRDGGUHVVWKHLVVXHSDUWLFXODUO\
EHFDXVHDWWKHKHDUWRIWKHPDWWHUOLHVDEDVLFWHQVLRQRQWKHRQHKDQGZLWKRXU
UHVHDUFKHUKDWRQZHFDQQRWGHQ\WKHIDFWWKDWHWKLFDOLVVXHVDUHRIWHQDKLQGUDQFHWR
RXULQYHVWLJDWLRQDQGLQRXUSROLWLFDOO\FRUUHFWDJHHWKLFDOTXHVWLRQVFDQJHWVRRXWRI
SURSRUWLRQWKDWLWFDQEHFRPHYLUWXDOO\LPSRVVLEOHWRGRUHVHDUFKLQFHUWDLQFRQWH[WV2Q
WKHRWKHUKDQGDVKXPDQEHLQJVZLWKPRUDOSULQFLSOHVZHFDQQRWGHQ\HLWKHUWKDWWKHUH
LVPRUHWROLIHWKDQUHVHDUFKDQGLIWKHUHLVDSRVVLELOLW\IRUDFODVKEHWZHHQWKH
UHVHDUFKHU¶VDQGWKHSDUWLFLSDQWV¶LQWHUHVWVLWLVFOHDUZKHUHWKHSULRULWLHVVKRXOGOLH
+HVVH%LEHUDQG/HDY\ REVHUYHWKDWHWKLFDOGLVFXVVLRQVRIWHQUHPDLQ
GHWDFKHGRUPDUJLQDOL]HGIURPGLVFXVVLRQVRIWKHµUHDO¶UHVHDUFKSURMHFWDOPRVWDVDQ
DIWHUWKRXJKW:KLOHWKLVPLJKWEHGXHWRWKHIDFWWKDWUHVHDUFKSXEOLFDWLRQVDUHXVXDOO\
ZULWWHQIRUUHVHDUFKHUDXGLHQFHVDQGLWLVDVVXPHGWKDWFROOHDJXHVDUHDZDUHRIWKH
LVVXHVLQYROYHGLQDUHVHDUFKPHWKRGVWH[WZHQHHGWRDGGUHVVWKHYDULRXVHWKLFDO
GLOHPPDVWKDWZHIDFHZKHQZHHQJDJHLQUHDOZRUOGUHVHDUFK$VDSUHOLPLQDU\ZH
VKRXOGQRWHWKDWWKHLQWHUQDWLRQDOVFHQHGLVSOD\VFRQVLGHUDEOHGLYHUVLW\LQWHUPVRIWKH
HWKLFDODZDUHQHVVDFURVVFRXQWULHVDQGWKHOHJDOUHJXODWLRQRIUHVHDUFKHWKLFVDOVR
VKRZVJUHDWGLIIHUHQFHVZRUOGZLGH )RUPRUHGHWDLOVRQHWKLFVLQDSSOLHGOLQJXLVWLFV
HVSHFLDOO\IURPD1RUWK$PHULFDQSHUVSHFWLYHVHH'XIILQSUHVV0DFNH\DQG*DVV
<HWHYHQLQFRXQWULHVZKHUHUHVHDUFKHWKLFVKDYHQRWEHHQIRUPDOO\UHJXODWHG
WKHUHLVDJURZLQJDZDUHQHVVRIYDULRXVGDWDSURWHFWLRQLVVXHVZKLFKORJLFDOO\OHDGWR
WKHUHFRQVLGHUDWLRQRIKRZKXPDQUHVHDUFKLVGHDOWZLWK7KHUHIRUH,DQWLFLSDWHWKDW
ZLWKLQWKHQH[WGHFDGHPRUHDQGPRUHFRXQWULHVZLOOWDNHWKH1RUWK$PHULFDQH[DPSOH
LQVHWWLQJXSDVWULFWIUDPHZRUNIRUUHVHDUFKHWKLFVWKDWLVPRUHSDUWLFLSDQWWKDQ
UHVHDUFKHUIULHQGO\
6RPHNH\HWKLFDOGLOHPPDVDQGLVVXHV

7KHILUVWHWKLFDOGLOHPPDWRDGGUHVVLVKRZVHULRXVO\ZHVKRXOGWDNHWKHYDULRXVHWKLFDO
LVVXHVLQHGXFDWLRQDOFRQWH[WV0RVWUHVHDUFKPHWKRGRORJ\ERRNVUHLWHUDWHEURDG
UHFRPPHQGDWLRQVRIHWKLFDOFRQGXFWWKDWKDYHEHHQSUHSDUHGIRUWKHVRFLDOVFLHQFHV
LQFOXGLQJSV\FKRORJ\ RUIRUPHGLFDOUHVHDUFKDQGLQWKHVHEURDGFRQWH[WVWKHUHLV
LQGHHGDGDQJHUWKDWPLVJXLGHGRUH[SORLWLYHUHVHDUFKFDQFDXVHUHDOKDUP2QWKHRWKHU
KDQGVRPHUHVHDUFKHUVVXFKDV-RKQVRQDQG&KULVWHQVHQ HPSKDVL]HWKDW
HGXFDWLRQDOUHVHDUFKVKRXOGQRWEHWUHDWHGVLPLODUO\WRRWKHUVRFLDOGLVFLSOLQHV
)RUWXQDWHO\VWXGLHVFRQGXFWHGE\HGXFDWLRQDOUHVHDUFKHUVVHOGRPLIHYHUUXQWKH
ULVNRILQIOLFWLQJVXFKVHYHUHPHQWDODQGSK\VLFDOKDUPRQSDUWLFLSDQWV,QIDFW
HGXFDWLRQDOUHVHDUFKKDVKLVWRULFDOO\HQJDJHGLQUHVHDUFKWKDWLPSRVHVHLWKHU
PLQLPDORUQRULVNWRWKHSDUWLFLSDQWVDQGKDVHQMR\HGDVSHFLDOVWDWXVZLWK
UHVSHFWWRIRUPDOHWKLFDORYHUVLJKW
%HFDXVHWKLVVSHFLDOVWDWXVKDVEHHQDFNQRZOHGJHGE\$PHULFDQODZPDNHUVDODUJH
SRUWLRQRIHGXFDWLRQDOUHVHDUFK²IRUH[DPSOHUHVHDUFKRQLQVWUXFWLRQDOVWUDWHJLHV
FXUULFXODRUFODVVURRPPDQDJHPHQW²KDVEHHQVLQJOHGRXWIRUH[HPSWVWDWXVLQWKH
86$LQWKH&RGHRI)HGHUDO5HJXODWLRQVIRUWKH3URWHFWLRQRI+XPDQ6XEMHFWV
-RKQVRQDQG&KULVWHQVHQ 7KLVGRHVQRWPHDQKRZHYHUWKDWHWKLFDOSULQFLSOHV
GRQRWDSSO\WRHGXFDWLRQDOUHVHDUFKRUWKDWZHFDQVDIHO\LJQRUHWKHZKROHLVVXHDV
-RKQVRQDQG&KULVWHQVHQ SRLQWRXWFHUWDLQUHVHDUFKSUDFWLFHVHVSHFLDOO\
TXDOLWDWLYHRQHVLQFOXGHHOHPHQWVWKDWµPXGG\WKHHWKLFDOZDWHUV¶ S DQGZDUUDQW
FDUHIXOFRQVLGHUDWLRQ6RPHH[DPSOHVRIVXFKVHQVLWLYHDVSHFWVRIUHVHDUFKDUH
x 7KHDPRXQWRIVKDUHGLQIRUPDWLRQ$VZHZLOOVHHEHORZLQ6HFWLRQRQHRIWKH
EDVLFGLOHPPDVRIUHVHDUFKHUVLVWRGHFLGHKRZPXFKLQIRUPDWLRQVKRXOGEHVKDUHG
ZLWKWKHSDUWLFLSDQWVDERXWWKHUHVHDUFKVRDVQRWWRFDXVHDQ\UHVSRQVHELDVRUHYHQ
QRQSDUWLFLSDWLRQ,Q&RKHQHWDO¶V ZRUGVµ:KDWLVWKHSURSHUEDODQFH
EHWZHHQWKHLQWHUHVWVRIVFLHQFHDQGWKHWKRXJKWIXOKXPDQHWUHDWPHQWRISHRSOH
ZKRLQQRFHQWO\SURYLGHWKHGDWD"¶
x 5HODWLRQVKLSV4XDOLWDWLYHVWXGLHVFDQRIWHQUHVXOWLQDQLQWLPDWHUHODWLRQVKLS
EHWZHHQUHVHDUFKHUVDQGSDUWLFLSDQWVZLWKWKHIRUPHUWU\LQJWRHVWDEOLVKUDSSRUW
DQGHPSDWK\WRJDLQDFFHVVWRWKHSDUWLFLSDQWV¶OLYHVDQGVWRULHV7KLVUHODWLRQDO
FKDUDFWHURITXDOLWDWLYHUHVHDUFKUDLVHVJHQHUDOHWKLFDOTXHVWLRQVDERXWWKHOLPLWVRI
FORVHQHVVDQGLQWLPDF\ZLWKWKHUHVSRQGHQWV²5\HQ IRUH[DPSOHGLVFXVVHV
WKHFRQWURYHUVLDOLVVXHRIIOLUWLQJZLWK DGXOW SDUWLFLSDQWV²DQGWKHUHLVDFRQFUHWH
GLOHPPDDERXWKRZWRHQGDUHVHDUFKSURMHFWZLWKRXWOHDYLQJWKHSDUWLFLSDQWV
IHHOLQJWKDWWKH\ZHUHPHUHO\XVHG
x 'DWDFROOHFWLRQPHWKRGV&HUWDLQPHWKRGVWKDWUHPRYHWKHSDUWLFLSDQWVIURPWKHLU
QRUPDODFWLYLWLHVDQGLQYROYHIRUH[DPSOHRQHWRRQHFRQWDFWFDQIDOOXQGHUWKH
FRQILQHVRIFKLOGSURWHFWLRQOHJLVODWLRQLQVHYHUDOFRXQWULHV

x $QRQ\PLW\$EDVLFGLOHPPDLQHGXFDWLRQDOUHVHDUFKFRQFHUQVWKHIDFWWKDWDOWKRXJK
LGHDOO\RXUSDUWLFLSDQWVVKRXOGUHPDLQDQRQ\PRXVZHRIWHQQHHGWRLGHQWLI\WKH
UHVSRQGHQWVWREHDEOHWRPDWFKWKHLUSHUIRUPDQFHVRQYDULRXVLQVWUXPHQWVRUWDVNV
x +DQGOLQJWKHFROOHFWHGGDWD7KHYLGHRFDPHUDLQSDUWLFXODULVDYHU\SXEOLFH\HDQG
LWDOORZVWKHLGHQWLILFDWLRQRIWKHSDUWLFLSDQWVORQJDIWHUWKHUHVHDUFKKDVHQGHG
$XGLRUHFRUGLQJVFDQVLPLODUO\EHDWKUHDWWRDQRQ\PLW\
x 2ZQHUVKLSRIWKHGDWD:KRµRZQV¶WKHFROOHFWHGGDWD"'RHVWKHUHVHDUFKHUDVNV
5LFKDUGV KDYHFRPSOHWHFRQWUROLQHGLWLQJRUUHOHDVLQJLQIRUPDWLRQ
REVHUYLQJRIFRXUVHSULYDF\FRQVWUDLQWV "
x 6HQVLWLYHLQIRUPDWLRQ,QGHHSLQWHUYLHZVSDUWLFLSDQWVFDQUHYHDOVHQVLWLYH
LQIRUPDWLRQWKDWLVQRWUHODWHGWRWKHJRDORIWKHVWXG\ IRUH[DPSOHSRLQWLQJWR
DEXVHRUFULPLQDODFWLYLW\ +RZGRZHUHDFW"
x 7HVWLQJ$OWKRXJKWKLVERRNGRHVQRWFRYHUWHVWLQJZHQHHGWRQRWHWKDWWKHPLVXVH
RIWHVWVFRUHVFDUULHVUHDOGDQJHUV
6XFKGLOHPPDVDQGLVVXHVVKRZWKDWHYHQLQHGXFDWLRQDOUHVHDUFKZHQHHGWRKDYH
DQHWKLFDOIUDPHZRUNLQSODFHDQHHGZKLFKLVDOVRDXJPHQWHGE\OHJDOUHTXLUHPHQWVLQ
VHYHUDOFRXQWULHV/HWXVFRQVLGHUWKHPDLQFRPSRQHQWVRIVXFKDIUDPHZRUNLQVRPH
GHWDLO
/HJDOFRQWH[W
,QPDQ\FRXQWULHVREVHUYLQJHWKLFDOSULQFLSOHVLVHQIRUFHGE\OHJDODQGLQVWLWXWLRQDO
UHTXLUHPHQWV,QWKH86IRUH[DPSOHUHVHDUFKHUVKDYHWRVXEPLWDGHWDLOHGUHVHDUFK
SODQIRUDSSURYDOWRDQ,QVWLWXWLRQDO5HYLHZ%RDUGSULRUWRVWDUWLQJWKHLULQYHVWLJDWLRQV
LQRUGHUWRFRPSO\ZLWKIHGHUDOUHJXODWLRQVWKDWSURYLGHSURWHFWLRQDJDLQVWKXPDQULJKWV
YLRODWLRQV7KHVHUHJXODWLRQVDOVRDSSO\WRSRVWJUDGXDWH 0$RU3K' UHVHDUFKDQG
RQO\LQH[FHSWLRQDOFLUFXPVWDQFHVZLOO*UDGXDWH6FKRROVDFFHSWDWKHVLVRUGLVVHUWDWLRQ
ZLWKRXWVRPHVRUWRIµKXPDQVXEMHFWV¶DSSURYDO7KHUHDUHHODERUDWHSURFHGXUHVWRJR
WKURXJKLQFOXGLQJILOOLQJLQYDULRXVIRUPVRIIHUHGE\HDFKXQLYHUVLW\¶VUHYLHZERDUG
6HH'XIILQSUHVV0DFNH\DQG*DVV
/HJDOIUDPHZRUNVDQGUHJXODWLRQVFRQFHUQLQJUHVHDUFKHWKLFVDUHXVXDOO\OHVV
GHYHORSHGLQFRXQWULHVRXWVLGH1RUWK$PHULFD DOWKRXJKDVPHQWLRQHGDERYHWKH
LQWHUQDWLRQDOWHQGHQF\LVWRDSSUR[LPDWHWKH86H[DPSOH ,QWKH8.DV:LOHVHWDO
VXPPDUL]HWKH%ULWLVKµ'DWD3URWHFWLRQ$FW¶UHJXODWHVWKHQHHGWRHQVXUH
FRQVHQWWRFROOHFWLQJGDWD H[FHSWLQYHU\VSHFLILFFLUFXPVWDQFHV DVZHOODVWKHXVHWR
ZKLFKGDWDLVSXW,QDGGLWLRQVRFLDOUHVHDUFKLVLQFUHDVLQJO\EHLQJVXEMHFWHGWRHWKLFDO
UHYLHZIURPLQVWLWXWLRQDOUHVHDUFKHWKLFVFRPPLWWHHVWRHQVXUHWKHHWKLFDOVFUXWLQ\RI
UHVHDUFKLQYROYLQJKXPDQVXEMHFWVDQGXQGHUJRLQJHWKLFDOUHYLHZLVVHWWREHFRPHD
UHTXLUHPHQWRIIXQGLQJERGLHVLQWKHIXWXUH
5HVHDUFKHULQWHJULW\

&XULRXVO\IHZUHVHDUFKPHWKRGVWH[WVHODERUDWHRQWKHDUHDRIUHVHDUFKHWKLFVWKDWLVLQ
P\YLHZFHQWUDOWRDQ\LQYHVWLJDWLRQWKHUHVHDUFKHU¶VLQWHJULW\,Q6HFWLRQZH
WRXFKHGEULHIO\XSRQWKHVLJQLILFDQFHRIWKHUHVHDUFKHU¶VUHOLDELOLW\DQGDFFRXQWDELOLW\
WRWKHILHOGDQG,EHOLHYHWKDWDWWKHKHDUWRIUHVHDUFKHWKLFVOLHVWKHPRUDOFKDUDFWHURI
WKHUHVHDUFKHU,QIDFWWKHWHUPµHWKLFV¶GHULYHVIURPWKH*UHHNZRUGµHWKRV¶ZKLFK
PHDQVFKDUDFWHUDQGDOWKRXJKLQRXUPRGHUQZRUOGHWKLFDOSULQFLSOHVFDQHDVLO\EH
HTXDWHGVLPSO\ZLWKµFRPSO\LQJZLWKODZV¶WKLVLVQRWPHUHO\DOHJDOLVWLFLVVXHEXW
FRQFHUQVEDVLFKXPDQKRQHVW\DQGWUXVW7KLVLVIXOO\UHFRJQL]HGE\WKH(WKLFDO
6WDQGDUGVRIWKH$PHULFDQ(GXFDWLRQDO5HVHDUFK$VVRFLDWLRQ $(5$ ZKLFK
VWDUWVRXWZLWKDVHWRIµJXLGLQJVWDQGDUGV¶GHVFULELQJWKHUHVHDUFKHUV¶JHQHUDO
UHVSRQVLELOLWLHVWRWKHILHOG7KHVHLQFOXGHWKHIROORZLQJSRLQWV
x (GXFDWLRQDOUHVHDUFKHUVPXVWQRWIDEULFDWHIDOVLI\RUPLVUHSUHVHQWDXWKRUVKLS
HYLGHQFHGDWDILQGLQJVRUFRQFOXVLRQV
x (GXFDWLRQDOUHVHDUFKHUVPXVWQRWNQRZLQJO\RUQHJOLJHQWO\XVHWKHLUSURIHVVLRQDO
UROHVIRUIUDXGXOHQWSXUSRVHV
x (GXFDWLRQDOUHVHDUFKHUVVKRXOGDWWHPSWWRUHSRUWWKHLUILQGLQJVWRDOOUHOHYDQW
VWDNHKROGHUVDQGVKRXOGUHIUDLQIURPNHHSLQJVHFUHWRUVHOHFWLYHO\FRPPXQLFDWLQJ
WKHLUILQGLQJV
7KHUHDUHPDQ\JUH\DUHDVLQUHVHDUFKZKHUHQRWHYHQWKHPRVWGHWDLOHGUHJXODWLRQV
FDQSUHYHQWVRPHXQGHVLUDEOHVKRUWFXWVRUPDQLSXODWLRQ$OWKRXJKWKHJHQHUDOWHQRURI
WKLVERRNLVPHDQWWREHKLJKO\SUDFWLFDODQGSUDJPDWLFWKLVLVDQDUHDZKHUHZHPXVW
ILUPO\GUDZWKHOLQH
3URWHFWLRQIURPKDUPDQGDFKLHYLQJDQHTXLWDEOHFRVW
EHQHILWEDODQFH
7KHSULPDU\SULQFLSOHRIUHVHDUFKHWKLFVLVWKDWQRPHQWDORUSK\VLFDOKDUPVKRXOG
FRPHWRWKHUHVSRQGHQWVDVDUHVXOWRIWKHLUSDUWLFLSDWLRQLQWKHLQYHVWLJDWLRQ7KLV
SULQFLSOHRYHUULGHVDOORWKHUFRQVLGHUDWLRQV+DYHUNDPS SRLQWVRXWWKDWZKLOHWKH
SULPDU\PHFKDQLVPVIRUSURWHFWLQJSDUWLFLSDQWVLQTXDQWLWDWLYHUHVHDUFKLQYROYHDFOHDU
GLYLVLRQRIUROHVDQGWKHPLQLPL]DWLRQRIFRQWDFWEHWZHHQUHVHDUFKHUDQGSDUWLFLSDQWV
WKHVHPHFKDQLVPVDUHLQFRPSDWLEOHZLWKWKHKLJKO\UHODWLRQDOFKDUDFWHURIPRVW
TXDOLWDWLYHUHVHDUFK7KXVLQTXDOLWDWLYHDQGPL[HGPHWKRGVDSSURDFKHVWKHUHLVDQ
LQFUHDVHGOLNHOLKRRGIRUWKHRFFXUUHQFHRIµHWKLFDOO\UHOHYDQWPRPHQWV¶$WWKLVSRLQW
ZHQHHGWRUHLWHUDWHWKDWWKHDEXVHRIWHVWVFRUHVSRVHVDVLPLODUO\UHOHYDQWHWKLFDOLVVXH
LQTXDQWLWDWLYHUHVHDUFK6KRKDP\ IRUH[DPSOHUHSRUWVDQRFFDVLRQZKHQ
HGXFDWLRQDODXWKRULWLHVXVHGWKHDFKLHYHPHQWWHVWUHVXOWVRIDVWXG\FRDXWKRUHGE\KHU
WRMXVWLI\PHDVXUHVDJDLQVWPLQRULW\VWXGHQWVLQDZD\ZKLFKGLGQRWIROORZIURPWKH
LQYHVWLJDWLRQ7KHUHIRUHVKHZDUQVXVWKDWUHVHDUFKHUVPXVWQRWYLHZWKHLUWDVNVDV
FRPSOHWHZKHQWKHUHVHDUFKUHSRUWRUWKHDUWLFOHLVSXEOLVKHGEXWPXVWµIROORZWKHXVHV
DQGPLVXVHV RIWKHLUUHVHDUFKUHVXOWV¶ S

1RWRQO\PXVWZHSUHYHQWRXULQYHVWLJDWLRQIURPFDXVLQJDQ\KDUPEXWZHQHHGWR
WU\DOVRWRPDNHVXUHWKDWWKHSDUWLFLSDQWVEHQHILWIURPRXUUHVHDUFKLQVRPHZD\:H
VKRXOGQHYHUIRUJHWWKDWE\VSHQGLQJWLPHDQGHQHUJ\KHOSLQJXVWKH\DUHGRLQJXVD
IDYRXUDQGLWLVRXUUHVSRQVLELOLW\WRWU\WRPDNHWKHFRVWEHQHILWEDODQFHDVHTXLWDEOHDV
SRVVLEOH8QIRUWXQDWHO\LWLVDOOWRRFRPPRQWRVHHDµVODVKDQGEXUQ¶UHVHDUFKVWUDWHJ\
ZKHUHE\LQYHVWLJDWRUVXVHWKHLUSDUWLFLSDQWVZLWKRXWRIIHULQJDQ\WKLQJLQUHWXUQDQGDV
VRRQDVWKHGDWDKDVEHHQJDWKHUHGWKH\GLVDSSHDU,QVRPHFDVHVVD\LQJDZDUPDQG
VDOLHQWµWKDQN\RX¶PD\EHHQRXJKLQVRPHRWKHUFDVHVZHFDQRIIHUWKHUHVSRQGHQWVRU
WKHSDUWLFLSDWLQJWHDFKHUVVFKRROVVRPHIHHGEDFNRQWKHUHVXOWVLIQHHGHG6RPH
UHVHDUFKHUVJRHYHQIXUWKHUE\DOVRRIIHULQJZRUNVKRSVRUOLWWOHJLIWV7KHUHDUHPDQ\
NLQGJHVWXUHVWKDWFDQZRUN²LWUHDOO\LVWKHWKRXJKWWKDWFRXQWV$QGLWLVVRLPSRUWDQW
WRREVHUYH-'%URZQ¶V ZDUQLQJµ5HPHPEHULI\RXPDNHDSURPLVHWR
VHQGWKHPVRPHWKLQJ\RXUHDOO\PXVWUHPHPEHUWRGRLW¶
3ULYDF\FRQILGHQWLDOLW\DQRQ\PLW\DQGGDWDVWRUDJH
,WLVDEDVLFHWKLFDOSULQFLSOHWKDWWKHUHVSRQGHQW¶VULJKWWRSULYDF\VKRXOGDOZD\VEH
UHVSHFWHGDQGWKDWUHVSRQGHQWVDUHZLWKLQWKHLUULJKWVWRUHIXVHWRDQVZHUTXHVWLRQVRUWR
ZLWKGUDZIURPWKHVWXG\FRPSOHWHO\ZLWKRXWRIIHULQJDQ\H[SODQDWLRQ,WLVDOVRWKH
SDUWLFLSDQWV¶ULJKWWRUHPDLQDQRQ\PRXVDQGLIWKHSDUWLFLSDQW¶VLGHQWLW\LVNQRZQWR
WKHUHVHDUFKJURXSLWLVWKHUHVHDUFKHU¶VPRUDODQGSURIHVVLRQDO DQGLQVRPHFRQWH[WV
OHJDO REOLJDWLRQWRPDLQWDLQWKHOHYHORIFRQILGHQWLDOLW\WKDWZDVSURPLVHGDWWKHRQVHW
7KHVHVWUDLJKWIRUZDUGSULQFLSOHVKDYHDQXPEHURILPSOLFDWLRQV
x :HPXVWPDNHVXUHWKDWZHGRQRWSURPLVHDKLJKHUGHJUHHRIFRQILGHQWLDOLW\WKDQ
ZKDWZHFDQDFKLHYHDQGWKDWWKHJXDUDQWHHVRIFRQILGHQWLDOLW\DUHFDUULHGRXWIXOO\
x 7KHULJKWWRFRQILGHQWLDOLW\VKRXOGDOZD\VEHUHVSHFWHGZKHQQRFOHDU
XQGHUVWDQGLQJWRWKHFRQWUDU\KDVEHHQUHDFKHG
x :HPXVWPDNHVXUH²HVSHFLDOO\ZLWKUHFRUGHGWUDQVFULEHGGDWD²WKDWWKH
UHVSRQGHQWVDUHQRWWUDFHDEOHRULGHQWLILDEOH
$WWKLVSRLQWZHQHHGWRUHDOL]HWKDWTXDOLWDWLYHGDWDFDQLQFOXGHVXFKLQWLPDWH
SHUVRQDOGHWDLOVRIWKHSDUWLFLSDQWV¶OLIHDQGHQYLURQPHQWWKDWQRQWUDFHDELOLW\LVYHU\
KDUGRUDOPRVWLPSRVVLEOHWRDFKLHYHHYHQLQDUHDVRQDEO\GLVJXLVHGDFFRXQW,QVXFK
FDVHVWKHJHQHUDOUHFRPPHQGDWLRQLQWKHOLWHUDWXUHLVWKDWZHVKRXOGDOWHUVRPHNH\
HOHPHQWVRIWKHGHVFULSWLRQEXWWKLVFDQDFWXDOO\JRFRXQWHUWRWKHFRUHVWULYLQJRI
TXDOLWDWLYHUHVHDUFKWRFDSWXUHSDUWLFXODULW\DQGFRQWH[W7KHUHLVQREODQNHWVROXWLRQ
KHUH²HDFKLQGLYLGXDOFDVHVKRXOGEHFRQVLGHUHGVHSDUDWHO\VKRZLQJVHQVLWLYLW\DQG
H[HUFLVLQJFDXWLRQ
2QHSDUWLFXODUWKUHDWWRFRQILGHQWLDOLW\LVSRVHGE\WKHVWRUDJHRIGDWDSDUWLFXODUO\
DXGLRDQGYLGHRUHFRUGLQJDVZHOODVWKHLUWUDQVFULSWV+RZFDQZHHQVXUH
FRQILGHQWLDOLW\±\HDUVGRZQWKHOLQH"2QFHWKHVWXG\LVRYHUWKHUHLVDGDQJHULQ
DFDGHPLFFRQWH[WVIRUVWRUDJHGLVFLSOLQHWRVODFNHQRUIRUORVLQJFRXQWRIWKHQXPEHURI
FRSLHVPDGHZKLFKFDQUHVXOWLQWKHGDWDIDOOLQJLQWRXQDXWKRUL]HGKDQGV7KLVLVVXHLV

DXJPHQWHGLIWKHUHVHDUFKWHDPVSOLWVXSRULIWKHRZQHUVKLSRIWKHGDWDKDVQRWEHHQ
FOHDUO\VSHFLILHGULJKWIURPWKHVWDUW7KHEHVWZD\WRSUHYHQWWKHDEXVHRIGDWDVWRUDJH
LVWRGHVWUR\WKHGDWDDIWHUDZKLOH
,QIRUPHGFRQVHQWDQGWKHLVVXHRIGHFHSWLRQ
7KHPRVWVDOLHQWDQGPRVWRIWHQGLVFXVVHGDVSHFWRIUHVHDUFKHWKLFVLVWKHLVVXHRI
LQIRUPHGFRQVHQW )RUUHOHYDQWJXLGHOLQHVGHYHORSHGIRUDSSOLHGOLQJXLVWLFVVHHWKH
µ,QIRUPDWLRQIRU&RQWULEXWRUV¶VHFWLRQRIWKHMRXUQDO7(62/4XDUWHUO\ ,IWKHUHDUH
UHJXODWLRQVJRYHUQLQJHWKLFDOSUDFWLFHLQDFRXQWU\RULQVWLWXWLRQREWDLQLQJZULWWHQ
FRQVHQWIURPWKHSDUWLFLSDQWVLVOLNHO\WREHWKHILUVWLWHPRQWKHDJHQGD,QWKH86IRU
H[DPSOHIHGHUDOUHJXODWLRQVQRWRQO\UHTXLUHZULWWHQFRQVHQWIURPWKHSDUWLFLSDQWVEXW
DOVRUHTXLUHLQIRUPHGFRQVHQWEHIRUHDUHVHDUFKHUFDQXVHDQLQGLYLGXDO¶VH[LVWLQJ
UHFRUGVIRUUHVHDUFKSXUSRVHV -RKQVRQDQG&KULVWHQVHQ :KLOHQRERG\ZRXOG
TXHVWLRQWKHVLJQLILFDQFHRIWKHIDFWWKDWUHVHDUFKUHVSRQGHQWVVKRXOGEHZLOOLQJ
SDUWLFLSDQWVJDLQLQJLQIRUPHGFRQVHQWIURPSRWHQWLDOVWXG\SDUWLFLSDQWVLVIDUIURP
EHLQJDVWUDLJKWIRUZDUGLVVXH7KHUHLVTXLWHDELWRIFRQWURYHUV\DERXW D KRZ
µLQIRUPHG¶WKHFRQVHQWVKRXOGEH²WKDWLVKRZPXFKLQIRUPDWLRQGRZHQHHGWRVKDUH
ZLWKWKHUHVSRQGHQWVEHIRUHDVNLQJWKHPWRSDUWLFLSDWHDQG E LQZKDWIRUPVKRXOGWKH
FRQVHQWEHJLYHQ7KHVHDUHLPSRUWDQWTXHVWLRQVWRFRQVLGHUSULRUWRWKHUHVHDUFK
EHFDXVHGHFLVLRQVDERXWWKHVW\OHVDQGPHDQVRIREWDLQLQJLQIRUPHGFRQVHQWKDYHDQ
LPSDFWRQWKHW\SHRISHRSOHZKRDUHOLNHO\WRDJUHHWRSDUWLFLSDWH :LOHVHWDO
+RZµLQIRUPHG¶VKRXOGWKHFRQVHQWEH"2IFRXUVHZKHQZHDVNWKLVTXHVWLRQZKDW
ZHUHDOO\PHDQLVµKRZOLWWOHLQIRUPDWLRQLVHQRXJKWRVKDUHLQRUGHUWRUHPDLQHWKLFDO"¶
7KHUHOXFWDQFHWRUHYHDOWRRPXFKDERXWWKHQDWXUHRIRXUUHVHDUFKLVDSUDJPDWLFRQH
EHFDXVHFHUWDLQLQIRUPDWLRQFDQLQIOXHQFHRUELDVWKHSDUWLFLSDQWV¶UHVSRQVHVDQGPD\
HYHQPDNHWKHPZLWKGUDZIURPWKHVWXG\6RZHQHHGWRDFKLHYHDWUDGHRIIKHUHDQG
WKHUHFRPPHQGDWLRQVFRQFHUQLQJWKHPLQLPDOOHYHORILQIRUPHGQHVVIRXQGLQWKH
OLWHUDWXUHYDU\LQH[FHSWLRQDOFLUFXPVWDQFHVHYHQGHFHSWLRQFDQEHDFFHSWDEOHLIFHUWDLQ
UXOHVDUHREVHUYHG²VHHEHORZ,QDFFRUGDQFHZLWKWKH$(5$ JXLGHOLQHV,
EHOLHYHWKDWLQDUHVHDUFKVWXG\WKHSDUWLFLSDQWV RUWKHLUJXDUGLDQV²VHHEHORZ KDYH
WKHULJKWWREHLQIRUPHGDERXWWKHIROORZLQJSRLQWV
x $VPXFKDERXWWKHDLPVRIWKHLQYHVWLJDWLRQDVSRVVLEOHDQGWKHSXUSRVHIRUZKLFK
KHGDWDZLOOEHXVHG
x 7KHWDVNVWKHSDUWLFLSDQWVZLOOEHH[SHFWHGWRSHUIRUPGXULQJWKHVWXG\
x 7KHSRVVLEOHULVNVDQGWKHSRWHQWLDOFRQVHTXHQFHVRISDUWLFLSDWLQJLQWKHUHVHDUFK
x 7KHH[WHQWWRZKLFKDQVZHUVZLOOEHKHOGFRQILGHQWLDO
x 7KHEDVLFULJKWRIWKHSDUWLFLSDQWVWRZLWKGUDZIURPWKHVWXG\DWDQ\SRLQW
'HFHSWLRQ
,WGRHVQRWUHTXLUHPXFKMXVWLILFDWLRQWKDWVRPHWLPHVUHVHDUFKHUVFDQQRWSURYLGHIXOO
GLVFORVXUHRIWKHQDWXUHDQGSXUSRVHRIWKHVWXG\ZLWKRXWFDXVLQJSDUWLFLSDQWELDVRU
HYHQLQYDOLGDWLQJWKHVWXG\DQGLQVRPH UDUH FDVHVWKHUHVHDUFKHUQHHGVQRWRQO\WR

ZLWKKROGVRPHLQIRUPDWLRQEXWWRDFWLYHO\PLVOHDGWKHSDUWLFLSDQWV$QHWKQRJUDSKLF
VWXG\RIDFULPLQDOVXEFXOWXUHZRXOGEHDJRRGH[DPSOHEXWHYHQLQIDUOHVVH[WUHPH
FDVHVFHUWDLQW\SHRILQIRUPDWLRQPD\LQIOXHQFHWKHUHVXOWV$FFRUGLQJO\DOWKRXJKWKH
$(5$ JXLGHOLQHVHPSKDVL]HWKHVLJQLILFDQFHRIKRQHVW\WKH\GRQRWIRUELGDQ\
HQJDJHPHQWLQGHFHSWLRQLILWLVDEVROXWHO\QHFHVVDU\SURYLGHGLWLVPLQLPL]HGDQGDIWHU
WKHVWXG\WKHSDUWLFLSDQWVDUHEULHIHGDERXWWKHUHDVRQVIRUWKHGHFHSWLRQ,QDGGLWLRQLW
JRHVZLWKRXWVD\LQJWKDWWKHGHFHSWLRQPXVWQRWDGYHUVHO\DIIHFWWKHZHOIDUHRIWKH
SDUWLFLSDQWV:KLOHPRVWUHVHDUFKHUVZRXOGDFFHSWWKHVHJHQHUDOSULQFLSOHVWKHUHLV
FRQVLGHUDEOHGHEDWHLQWKHVRFLDOVFLHQFHOLWHUDWXUHDERXWWKHDFFHSWDEOHOHYHORIDFWXDO
GHFHLW
)RUPVRIFRQVHQWDQGWKHFRQVHQWIRUPV
7KHUHDUHWZREDVLFIRUPVRIFRQVHQWDFWLYHDQGSDVVLYHµ$FWLYH¶FRQVHQWLQYROYHV
FRQVHQWLQJWRSDUWLFLSDWHLQDUHVHDUFKVWXG\E\VLJQLQJDFRQVHQWIRUPZKHUHDV
µSDVVLYH¶FRQVHQWLQYROYHVQRWRSWLQJRXWRUQRWREMHFWLQJWRWKHVWXG\7KHPRVW
FRPPRQH[DPSOHRIWKHODWWHULVWRVHQGDFRQVHQWIRUPWRWKHSDUHQWVRIVFKRRO
FKLOGUHQDQGDVNWKHPWRUHWXUQLWRQO\LIWKH\GRQRWZDQWWKHLUFKLOGWRSDUWLFLSDWHLQ
WKHUHVHDUFK,WLVFOHDUWKDWREWDLQLQJDFWLYHFRQVHQWLVPRUHH[SOLFLWLQHQVXULQJWKDWWKH
SDUWLFLSDQWVNQRZWKHLUULJKWVDQGLWDOVRSURWHFWVWKHUHVHDUFKHUIURPDQ\ODWHU
DFFXVDWLRQVEXWLQFHUWDLQW\SHVRIHGXFDWLRQDOUHVHDUFKLWPD\QRWEHQHFHVVDU\RU
EHQHILFLDODUHTXHVWIRUFRQVHQWLQVXFKDIRUPDOL]HGDPDQQHUFDQEHRIISXWWLQJRU
FDQUDLVHXQGXHVXVSLFLRQWKDWVRPHWKLQJLVQRWTXLWHULJKWDERXWWKHVWXG\WKHUHE\
GLVFRXUDJLQJSHRSOHIURPSDUWLFLSDWLQJ,QDGGLWLRQVLJQHGFRQVHQWLVPHDQLQJOHVVIRU
VRPHFXOWXUDORUDJHJURXSV7KHUHIRUHZKLOH,ZRXOGDVNIRUDFWLYHFRQVHQWLQD
TXDOLWDWLYHLQWHUYLHZVWXG\,ZRXOGFRQVLGHURPLWWLQJLWDQGREWDLQLQJRQO\SDVVLYH
FRQVHQW²LIWKLVGRHVQRWFRQWUDGLFWDQ\OHJDOUHJXODWLRQVLQWKHSDUWLFXODUFRQWH[W²LQ
DQDQRQ\PRXVTXHVWLRQQDLUHVXUYH\
$ZULWWHQFRQVHQWIRUPXVXDOO\FRQWDLQVWKHIROORZLQJGHWDLOV &RKHQHWDO
&UHVZHOO-RKQVRQDQG&KULVWHQVHQ
x DIDLUH[SODQDWLRQRIWKHSXUSRVHRIWKHUHVHDUFKDQGWKHSURFHGXUHVWREHIROORZHG
x DGHVFULSWLRQRIDQ\ULVNVRUGLVFRPIRUWVDVZHOODVEHQHILWVWKHSDUWLFLSDQWPD\
HQFRXQWHUUHFHLYH
x DVWDWHPHQWRIWKHH[WHQWWRZKLFKWKHUHVXOWVZLOOEHNHSWFRQILGHQWLDO
x DVWDWHPHQWLQGLFDWLQJWKDWSDUWLFLSDWLRQLVYROXQWDU\DQGWKHSDUWLFLSDQWFDQ
ZLWKGUDZDQGUHIXVHWRSDUWLFLSDWHDWDQ\WLPHZLWKQRSHQDOW\
x DQRIIHUWRDQVZHUDQ\TXHVWLRQVFRQFHUQLQJWKHSURFHGXUHVDQG RSWLRQDOO\ WR
UHFHLYHDFRS\RIWKHUHVXOWV
x VLJQDWXUHVRIERWKWKHSDUWLFLSDQWDQGWKHUHVHDUFKHUDJUHHLQJWRWKHVHSURYLVLRQV
$GGLWLRQDOFRQVHQWIURPWHDFKHUVDQGSDUHQWV
0DQ\LIQRWPRVWHGXFDWLRQDOVWXGLHVDUHFRQGXFWHGZLWKLQVFKRROVRURWKHU
HGXFDWLRQDOLQVWLWXWHVDQGWKHUHIRUHWKHUHVHDUFKHUPD\QHHGWRREWDLQDGGLWLRQDO

FRQVHQWIURPYDULRXVDXWKRULW\ILJXUHVVXFKDVWHDFKHUVKHDGWHDFKHUVRUSULQFLSDOV,Q
VRPHFRXQWULHVFRQVHQWPXVWEHVRXJKWHYHQIURPWKHORFDOHGXFDWLRQDXWKRULWLHV
$GGLWLRQDOFRQVHQWPXVWDOVREHREWDLQHGZKHQWKHUHVHDUFKLVWDUJHWLQJPLQRUVZKR
FDQQRWEHUHJDUGHGDVIXOO\PDWXUHRUEHLQJRQHTXDOWHUPVZLWKWKHUHVHDUFKHU &RKHQ
HWDO 7KH\PD\QRWEHLQDSRVLWLRQWRUHSUHVHQWWKHPVHOYHVDSSURSULDWHO\ZKLFK
PHDQVWKDWVRPHRQHHOVHQHHGVWREHFRQVXOWHG7KHTXHVWLRQKHUHLVWRGHFLGHZKRKDV
VXIILFLHQWDXWKRULW\LQVXFKFDVHVWKHOHJDOJXDUGLDQ IRUH[DPSOHSDUHQW WKH
FKLOGUHQ¶VWHDFKHU V RUERWK,QWKLVUHVSHFWWKHH[LVWLQJOHJDODQGHWKLFDOUHVHDUFK
IUDPHZRUNVGLIIHUJUHDWO\DFURVVFRXQWULHV,WLVP\YLHZWKDWXQOHVVWKHUHH[LVWOHJDO
UHTXLUHPHQWVVWDWLQJRWKHUZLVHDQGLIWKHUHVHDUFKLVQHLWKHUDLPHGDWVHQVLWLYH
LQIRUPDWLRQQRULQYROYHVH[WHQVLYHSDUWLFLSDQWHQJDJHPHQW IRUH[DPSOHDUHODWLYHO\
QHXWUDODQRQ\PRXVTXHVWLRQQDLUH SHUPLVVLRQWRFRQGXFWWKHVWXG\FDQEHJUDQWHGE\
WKHFKLOGUHQ¶VWHDFKHUV7HDFKHUVDUHXVXDOO\DZDUHRIWKHVLJQLILFDQFHRIOHJDOPDWWHUV
DQGWKHUHIRUHLIWKH\KDYHDQ\GRXEWVDERXWZKRVKRXOGDXWKRUL]HWKHSURMHFWWKH\ZLOO
VHHNDGYLFH$QGHYHQLISDUHQWDOSHUPLVVLRQIRUWKHUHVHDUFKLVQHHGHG,ZRXOGIDYRXU
SDVVLYHFRQVHQWZKHUHE\SDUHQWVDUHDGYLVHGDERXWWKHSURSRVHGUHVHDUFKDQGSDUHQWDO
SHUPLVVLRQZLOOEHDVVXPHGXQOHVVWKHSDUHQWVREMHFWEHIRUHWKHSURSRVHGVWDUWLQJGDWH
&RQFOXGLQJUHPDUNVRQUHVHDUFKHWKLFV
0\SHUVRQDOFRQFOXVLRQDERXWUHVHDUFKHWKLFVLVWKDWLWVKRXOGEHWDNHQPRUHVHULRXVO\
WKDQPDQ\UHVHDUFKHUVGRDQGDWWKHVDPHWLPHLWVKRXOGEHWDNHQOHVVVHULRXVO\WKDQ
VRPHOHJLVODWRUVGR3RVVLEO\EHFDXVHDSSOLHGOLQJXLVWLFUHVHDUFKGRHVQRWXVXDOO\SRVH
DQ\REYLRXVWKUHDWWRWKHSDUWLFLSDQWVUHVHDUFKHUVLQRXUILHOGRIWHQLJQRUHWKH
VLJQLILFDQFHRIHWKLFDOLVVXHVDQGUHVHDUFKHWKLFVDUHRQO\PHQWLRQHGZLWKLQWKHFRQWH[W
RIOHJDOUHTXLUHPHQWVWRFRPSO\ZLWK7KHSUREOHPZLWKWKLVSUDFWLFHLVWKDWLWGRHVQRW
SURYLGHDILUPHWKLFDOIUDPHZRUNWKDWZHFDQUHO\RQZKHQDQHWKLFDOO\GLIILFXOW
VLWXDWLRQDULVHV,KRSHWKDWWKHSUHYLRXVGLVFXVVLRQKDVVKRZQFOHDUO\WKDWUHVHDUFK
SDUWLFLSDQWVQHHGVDIHJXDUGVDQGUHVHDUFKHUVQHHGLQIRUPHGDQVZHUVWRHWKLFDO
GLOHPPDV
2QWKHRWKHUKDQG,DJUHHZLWKWKRVH IRUH[DPSOH-RKQVRQDQG&KULVWHQVHQ
ZKRHPSKDVL]HWKDWLQHGXFDWLRQDOUHVHDUFKZHVKRXOGQRWVLPSO\LPSRUWHWKLFDO
JXLGHOLQHVIURPRWKHUILHOGV IRUH[DPSOHSV\FKRORJ\RUPHGLFDOUHVHDUFK ZKHUHWKH
HWKLFDOVWDNHVDUHPXFKKLJKHU,IDSSOLHGPHFKDQLFDOO\HWKLFDOFRGHVDQGUHJXODWLRQV
FDQHDVLO\EHFRPHWRRUHVWULFWLYHLURQLFDOO\LQWKHILHOGRIDSSOLHGOLQJXLVWLFVWKHPRVW
SUHVVLQJHWKLFDOLVVXH,DPDZDUHRILVWKHLQFUHDVLQJGLIILFXOW\RIFRQGXFWLQJUHVHDUFK
LQFHUWDLQFRQWH[WVGXHWRWKHEXUJHRQLQJFRQVWUDLQWVRIµHWKLFDOFRUUHFWQHVV¶6HDOHHW
DO DUHULJKWZKHQWKH\FRQFOXGHµ3URIHVVLRQDOHWKLFDOFRGHVIRUUHVHDUFKHUV
DUHWRRRIWHQFRQVWLWXWHGDVDUPFKDLUFULWLFLVPGLVWDQFHGIURPWKHQHHGVRIWKHUHVHDUFK
SUDFWLFH¶:KDWZHQHHGLVDFRQWH[WXDOL]HGDQGIOH[LEOHDSSURDFKWRHWKLFDOGHFLVLRQ
PDNLQJUHO\LQJPRUHRQWKHUHVHDUFKHUV¶SURIHVVLRQDOUHIOH[LYLW\DQGLQWHJULW\LQ
PDLQWDLQLQJKLJKVWDQGDUGV

5HVHDUFKTXHVWLRQVDQGK\SRWKHVHV
«PDQ\VFLHQWLVWVRZHWKHLUUHQRZQOHVVWRWKHLUDELOLW\WRVROYHSUREOHPVWKDQWR
WKHLUFDSDFLW\WRVHOHFWLQVLJKWIXOTXHVWLRQVIRULQYHVWLJDWLRQ«
6KDYHOVRQDQG7RZQH
0RVWUHVHDUFKWH[WVVXJJHVWWKDWWKHSURSHUZD\WRGRUHVHDUFKLQYROYHVILUVWJHQHUDWLQJ
RQHRUPRUHUHVHDUFKTXHVWLRQVDQGWKHQFKRRVLQJWKHGHVLJQWKHPHWKRGDQGWKH
LQVWUXPHQWVWKDWDOORZWKHUHVHDUFKHUWRILQGDQVZHUVWRWKHVHTXHVWLRQV)RUH[DPSOH
LQGLVFXVVLQJWKHSULQFLSOHVRIVFLHQWLILFUHVHDUFKLQHGXFDWLRQ6KDYHOVRQDQG7RZQH
H[SOLFLWO\VWDWHWKDWµZHKDYHWREHWUXHWRRXUDGPRQLWLRQWKDWWKHUHVHDUFK
TXHVWLRQGULYHVWKHGHVLJQQRWYLFHYHUVD¶:KLOHWKLVLVDORJLFDOSRVLWLRQPDQ\QRYLFH
UHVHDUFKHUVZLOODWWHVWWRWKHIDFWWKDWLQUHDOLW\WKHUHVHDUFKSURFHVVLVQRWVR
VWUDLJKWIRUZDUGDQGWKHµUHVHDUFKTXHVWLRQILUVW¶SULQFLSOHOHDYHVVHYHUDOTXHVWLRQV
RSHQ)RUH[DPSOHLQDQLQYHVWLJDWLRQWKDWLVSULPDULO\H[SORUDWRU\KRZFDQZHVWDUW
RXWE\SURGXFLQJDVSHFLILFUHVHDUFKTXHVWLRQZKHQZHNQRZUHODWLYHO\OLWWOHRIWKH
WRSLFZKLFKLVH[DFWO\ZK\ZHZDQWWRUHVHDUFKLQWRLW"+RZLVWKHUHVHDUFKTXHVWLRQ
UHODWHGWRRWKHUIUHTXHQWO\PHQWLRQHGWHUPVVXFKDVµUHVHDUFKWRSLF¶RUµUHVHDUFK
K\SRWKHVLV¶":KDWDERXWTXDOLWDWLYHUHVHDUFKLQZKLFKWKHH[DFWWKHPHDQGDQJOHRIWKH
LQYHVWLJDWLRQRIWHQHPHUJHRQO\GXULQJWKHSURMHFW"$QGILQDOO\ZKDWLVWKHRSWLPDO
UHVHDUFKTXHVWLRQOLNH"/HWXVEULHIO\GLVFXVVWKHVHLVVXHVIRUDPRUHGHWDLOHGDQGYHU\
SUDFWLFDOGLVFXVVLRQRIWKHSODQQLQJVWDJHRIDUHVHDUFKSURMHFWIURPWKHJHQHUDWLRQRI
LQLWLDOLGHDVWRQDUURZLQJGRZQWKHIRFXVVRWKDWWKHSURMHFWLVGRDEOHVHH5LFKDUGV
&KDSWHU
5HVHDUFKWRSLFUHVHDUFKSXUSRVHDQGUHVHDUFKTXHVWLRQV
(YHU\LQYHVWLJDWLRQKDVDVWDUWLQJSRLQWDQGXQOHVVZHDGRSWVRPHRQHHOVH¶V IRU
H[DPSOHWKHVXSHUYLVRU¶V GHVLJQLGHDRUMRLQDQRQJRLQJLQYHVWLJDWLRQDVDFR
UHVHDUFKHUWKLVVWDUWLQJSRLQWLVDEURDGµUHVHDUFKWRSLF¶,QLWLDOO\WKLVLVRQO\DJHQHUDO
DQGUDWKHUYDJXHDUHDRIUHVHDUFKLQWHUHVW IRUH[DPSOHDSUREOHPRUDQLQWULJXLQJ
SKHQRPHQRQWKDWZHKDYHQRWLFHG DQGLQRUGHUWREHDEOHWRVSHFLILFDOO\DGGUHVVLWZH
QHHGWRGHYHORSLWLQWRDµUHVHDUFKSXUSRVH¶7KHUHVHDUFKSXUSRVHLVDUHODWLYHO\VKRUW
VWDWHPHQW QRWPRUHWKDQDSDUDJUDSK WKDWGHVFULEHVWKHREMHFWLYHRIWKHSODQQHGVWXG\
H[SODLQLQJZK\WKHLQYHVWLJDWLRQLVXQGHUWDNHQDQGZKDWLWVSRWHQWLDOVLJQLILFDQFHLV
7KHQH[WVWHSWRZDUGVQDUURZLQJGRZQDQGµRSHUDWLRQDOL]LQJ¶WKHUHVHDUFKSXUSRVH
LVWRIRUPXODWHVSHFLILFµUHVHDUFKTXHVWLRQV¶7KHLUUROHLVWRWUDQVODWHWKHUHVHDUFK
SXUSRVHLQWRVSHFLILFTXHVWLRQVWKDWWKHSODQQHGVWXG\ZLOODLPWRDQVZHU%HLQJDEOHWR
SURGXFHJRRGUHVHDUFKTXHVWLRQVLQGLFDWHVWKDWWKHUHVHDUFKHULVUHDG\WRHPEDUNRQWKH
DFWXDOLQYHVWLJDWLRQ ZKHUHDVYDJXHRUPLVVLQJUHVHDUFKTXHVWLRQVXVXDOO\VHQGDODUP
VLJQDOVWRDVXSHUYLVRURUUHIHUHH DQGJRRGUHVHDUFKTXHVWLRQVDOVRSURYLGHRULHQWDWLRQ
WRWKHUHVHDUFKPHWKRGRORJ\WKDWFDQEHVWDFKLHYHWKHUHVHDUFKSXUSRVH²VHH&KDSWHU
:HVKRXOGQRWHKHUHWKDWDV-RKQVRQDQG&KULVWHQVHQ SRLQWRXWEHFDXVHLQ

VRPHLQYHVWLJDWLRQVWKHUHVHDUFKTXHVWLRQZRXOGEHWRDJUHDWH[WHQWDUHVWDWHPHQWRI
WKHSXUSRVHRIWKHVWXG\VRPHVFKRODUVDFWXDOO\RPLWLWIURPWKHILQDOUHVHDUFKUHSRUW
:KDWLVDJRRGUHVHDUFKTXHVWLRQOLNH"$VZHZLOOVHHLQWKHQH[WVHFWLRQWKHUHLVD
GLIIHUHQFHEHWZHHQHIIHFWLYHUHVHDUFKTXHVWLRQVLQ48$/DQG48$1VWXGLHVEXWLQ
JHQHUDOJRRGUHVHDUFKTXHVWLRQVQHHGWRDGGUHVVLQWHUHVWLQJLVVXHV:KHQZHKDYH
WKRXJKWRIVRPHSRVVLEOHTXHVWLRQVZHKDYHWRDVNRXUVHOYHV$UHWKH\ZRUWKDVNLQJ
DQGPRUHLPSRUWDQWO\DQVZHULQJ",EHOLHYHWKDWRQHWKLQJZHPXVWWU\DQGDYRLGDWDOO
FRVWLVWRµUXQLQWRD³VRZKDW´UHVSRQVHWRRXUUHVHDUFK¶ 0DFNH\DQG*DVV
7KHUHIRUH,DPLQDJUHHPHQWZLWK*DOOHWDO¶V FRQFOXVLRQWKDWµ7KH
LPDJLQDWLRQDQGLQVLJKWWKDWJRHVLQWRGHILQLQJWKHUHVHDUFKSUREOHPXVXDOO\GHWHUPLQHV
WKHXOWLPDWHYDOXHRIDUHVHDUFKVWXG\PRUHWKDQDQ\RWKHUIDFWRU¶
:KHUHFDQZHUHFHLYHWKHLQVSLUDWLRQIRUJRRGUHVHDUFKWRSLFVDQGTXHVWLRQV",QP\
H[SHULHQFHPRVWUHVHDUFKWRSLFVRULJLQDWHIURPDFRPELQDWLRQRIUHDGLQJWKHOLWHUDWXUH
DQGRQH¶VSHUVRQDOKLVWRU\)URPWLPHWRWLPHVRPHWKLQJZHUHDGµULQJVDEHOO¶DQGZH
UHDOL]HWKDWWKHWRSLFFRXOGEHIXUWKHUSXUVXHGRQWKHEDVLVRIUHOHYDQWH[SHULHQFHVZH
KDGLQWKHSDVW,QDGGLWLRQWKH&RQFOXVLRQVHFWLRQRIPRVWUHVHDUFKDUWLFOHV VHH
6HFWLRQ WHQGVWRFRQWDLQVXJJHVWLRQVIRUIXUWKHUUHVHDUFKDQGGLVFXVVLRQVZLWK
IULHQGVFROOHDJXHVDQGVWXGHQWVFDQDOVREHYHU\KHOSIXOLQGUDZLQJRXUDWWHQWLRQWR
SRWHQWLDOO\IUXLWIXOLVVXHV,I\RXGHFLGHWRNHHSDUHVHDUFKMRXUQDO VHH6HFWLRQ WKH
UHIOHFWLRQVUHFRUGHGLQWKHMRXUQDOPD\FRQWULEXWHWRUHILQLQJWKHUHVHDUFKTXHVWLRQVRI
DQRQJRLQJLQYHVWLJDWLRQDVZHOODVJHQHUDWLQJLGHDVIRUIXWXUHUHVHDUFK
6RPHWLPHVZKHQZHKDYHDKXQFKRUDOHDUQHGJXHVVDERXWWKHSRVVLEOHUHVXOWVRI
RXUSODQQHGLQYHVWLJDWLRQZHFDQQDUURZGRZQWKHUHVHDUFKSXUSRVHLQWRDFWXDO
µUHVHDUFKK\SRWKHVHV¶LQVWHDGRIRUEHVLGHVTXHVWLRQV7KHVHK\SRWKHVHVDUHVWDWHPHQWV
WKDWIRUPXODWHVSHFLILFSUHGLFWLRQVDERXWWKHRXWFRPHVDQGWKHHPSLULFDOUHVXOWVZLOO
HLWKHUFRQILUPRUUHIXWHWKHVH,IJHQHUDWLQJUHVHDUFKK\SRWKHVHVLVSRVVLEOHLQDVWXG\
WKH\DUHYHU\ZHOFRPHEHFDXVHWKH\DUHIXOO\LQOLQHZLWKWKHSULQFLSOHVRIWKH
µVFLHQWLILFPHWKRG¶²VHH6HFWLRQ
1XQDQ PDNHVDXVHIXOUHFRPPHQGDWLRQDERXWKRZWRRSHUDWLRQDOL]HDQ
LQYHVWLJDWLRQ+HKDVIRXQGWKDWDVNLQJSRVWJUDGXDWHVWXGHQWVWRFUHDWHDµUHVHDUFK
RXWOLQH¶FDQJUHDWO\IDFLOLWDWHWKHSODQQLQJDQGILQHWXQLQJRIDSURMHFW6XFKDQRXWOLQH
FRQVLVWVRIDVHULHVRIKHDGLQJVDQGVWXGHQWVDUHWRDGGDVKRUWVWDWHPHQWDIWHUHDFKWKH
KHDGLQJVFDQEHVHOHFWHGLQDIOH[LEOHZD\EXWWKH\ZRXOGQRUPDOO\LQFOXGHWKHJHQHUDO
UHVHDUFKDUHDUHVHDUFKSXUSRVHUHVHDUFKTXHVWLRQK\SRWKHVLVDVZHOODVGDWDFROOHFWLRQ
PHWKRGW\SHRIGDWDSDUWLFLSDQWVW\SHRIGDWDDQDO\VLVDQGUHVRXUFHVUHTXLUHG
3DUDGLJPDWLFGLIIHUHQFHVLQIRUPXODWLQJWKHUHVHDUFK
TXHVWLRQV
4XDOLWDWLYHDQGTXDQWLWDWLYHVWXGLHVGLIIHUFRQVLGHUDEO\LQWHUPVRIKRZWKHSXUSRVHRI
WKHLQYHVWLJDWLRQLVVSHFLILHGDQGKRZLWLVEURNHQGRZQLQWRVSHFLILFUHVHDUFK
TXHVWLRQV,QTXDQWLWDWLYHVWXGLHVLWLVJHQHUDOO\WUXHWKDWWKHPRUHVSHFLILFWKHUHVHDUFK

SXUSRVHTXHVWLRQWKHEHWWHU7KXVJRRGTXDQWLWDWLYHSXUSRVHVWDWHPHQWVRIWHQLGHQWLI\
WKHWDUJHWYDULDEOHVDQGWKHFDXVDORUGHVFULSWLYHUHODWLRQVKLSEHWZHHQWKHPWKDWLVWREH
H[DPLQHG7KHUHVHDUFKTXHVWLRQVWKHQVSHFLI\FRQFUHWHPHWKRGRORJLFDOSURFHGXUHV
DQGUHVHDUFKK\SRWKHVHVDUHDOVRGUDZQXSFRQWDLQLQJWKHUHVHDUFKHU¶VSUHGLFWLRQV
2QHFKDUDFWHULVWLFIHDWXUHRITXDOLWDWLYHVWXGLHVLVWKHLUHPHUJHQWQDWXUHDQG
WKHUHIRUH48$/UHVHDUFKSXUSRVHVDQGTXHVWLRQVDUHRIWHQLQHYLWDEO\YDJXHUWKDQWKHLU
48$1FRXQWHUSDUWV,QVWHDGRIGHVFULELQJDVSHFLILFLVVXHRUSUREOHPWKHUHVHDUFK
SXUSRVHRIWHQFRQWDLQVRQO\WKHVSHFLILFDWLRQRIDVLWXDWHGSKHQRPHQRQRUDFHQWUDOLGHD
WKDWZLOOEHH[SORUHGZLWKWKHDLPRIGHYHORSLQJQHZLQVLJKWVDQGSRVVLEO\IRUPLQJD
WKHRU\LQWKHHQG,QDFFRUGDQFHZLWKWKLVTXDOLWDWLYHUHVHDUFKTXHVWLRQVWHQGWREH
EURDGHUWKDQTXDQWLWDWLYHRQHVRIWHQIRFXVLQJRQWKHELJSLFWXUHRUWKHPDLQSURFHVVHV
WKDWDUHWKRXJKWWRVKDSHWKHWDUJHWSKHQRPHQRQ²XVXDOO\LWLVQRWSRVVLEOHWREHPRUH
VSHFLILFDWWKLVVWDJHZLWKRXWOLPLWLQJWKHLQTXLU\DQGWKHUHIRUHLQYHVWLJDWRUV
HPSKDVL]HWKHH[SORUDWRU\QDWXUHRIWKHVWXG\LQVWHDG
0L[HGPHWKRGVVWXGLHVSUHVHQWDFKDOOHQJHLQWHUPVRIVSHFLI\LQJWKHUHVHDUFK
SXUSRVHDQGTXHVWLRQVEHFDXVHWKH\UHTXLUHVHHPLQJO\FRQWUDGLFWRU\DSSURDFKHV EHLQJ
VSHFLILFDQGYDJXHEURDGDWWKHVDPHWLPH $JRRGVWUDWHJ\LQVXFKVWXGLHVLVWRVWDUW
ZLWKDQRYHUDOOSXUSRVHVWDWHPHQWIROORZHGE\DUDWLRQDOHIRUWKHVSHFLILFPL[HGGHVLJQ
DSSOLHGLQFOXGLQJWKHFKRLFHRIWKHSDUWLFXODUPHWKRGVDQGFRQFOXGHGE\VHSDUDWH
UHVHDUFKREMHFWLYHVDQGTXHVWLRQVIRUWKHGLIIHUHQWUHVHDUFKFRPSRQHQWV&UHVZHOO
PHQWLRQVWKDWRQHXVHIXOPRGHOIRXQGLQPL[HGPHWKRGVVWXGLHVLQYROYHV
PRYLQJWKHVSHFLILFUHVHDUFKTXHVWLRQVWRWKHLQWURGXFWLRQRIHDFKSKDVHLQWKHVWXG\
UDWKHUWKDQSUHVHQWLQJWKHPDOODWWKHEHJLQQLQJ
2WKHUHVVHQWLDOVIRUODXQFKLQJDVWXG\SLORWVWXG\
UHVHDUFKORJDQGGDWDPDQDJHPHQW
7RFRQFOXGHWKLVFKDSWHUOHWPHDGGUHVVWKUHHHVVHQWLDOLVVXHVWREHFRQVLGHUHGEHIRUH
ODXQFKLQJDQLQYHVWLJDWLRQ D SLORWLQJWKHLQVWUXPHQWVDQGWKHYDULRXVUHVHDUFK
SURFHGXUHV E NHHSLQJDSUHFLVHUHVHDUFKORJDQG F PDQDJLQJDQGVWRULQJGDWD
SURIHVVLRQDOO\
3LORWLQJWKHUHVHDUFK
7KHJLVWRIWKLVVHFWLRQFDQEHVXPPDUL]HGLQDVKRUWVHQWHQFHDOZD\VSLORW\RXU
UHVHDUFKLQVWUXPHQWVDQGSURFHGXUHVEHIRUHODXQFKLQJ\RXUSURMHFW-XVWOLNHWKHDWUH
SHUIRUPDQFHVDUHVHDUFKVWXG\DOVRQHHGVDGUHVVUHKHDUVDOWRHQVXUHWKHKLJKTXDOLW\
LQWHUPVRIUHOLDELOLW\DQGYDOLGLW\ RIWKHRXWFRPHVLQWKHVSHFLILFFRQWH[W$QGZKLOH
WKLVWHQHWLVXQLYHUVDOO\HQGRUVHGE\UHVHDUFKPHWKRGRORJLVWVP\SHUVRQDOH[SHULHQFH
LVWKDWDOOWRRRIWHQSLORWLQJHLWKHUGRHVQRWKDSSHQRULVVXSHUILFLDO7KHUHDUHDWOHDVW
WKUHHUHDVRQVIRUWKLVRPLVVLRQILUVWUHVHDUFKHUVPD\VLPSO\QRWUHDOL]HWKH
VLJQLILFDQFHRIWKHSLORWLQJSKDVH6HFRQGLQYHVWLJDWRUVDWWKHSUHSDUDWLRQVWDJHDUH
XVXDOO\HDJHUWRJHWGRZQWRGDWDFROOHFWLRQDQGVHHVRPHUHVXOWV7KLUGWKHSLORWLQJ

SKDVHPD\QRWKDYHEHHQVFKHGXOHGLQWKHWLPHIUDPHRIWKHSURMHFWEHFDXVHWKH
UHVHDUFKHUVGLGQRWUHDOL]HWKDWZKHQSURSHUO\GRQHWKLVSKDVHFDQWDNHVHYHUDOZHHNV
WRFRPSOHWH
3LORWLQJLVPRUHLPSRUWDQWLQTXDQWLWDWLYHVWXGLHVWKDQLQTXDOLWDWLYHRQHVEHFDXVH
TXDQWLWDWLYHVWXGLHVUHO\RQWKHSV\FKRPHWULFSURSHUWLHVRIWKHUHVHDUFKLQVWUXPHQWV,Q
DTXHVWLRQQDLUHVXUYH\IRUH[DPSOHLIZHKDYHQRWFRYHUHGDYDULDEOHE\VXIILFLHQW
TXHVWLRQQDLUHLWHPVWKHQWKLVYDULDEOHVLPSO\FDQQRWHPHUJHLQWKHUHVXOWVQRPDWWHU
KRZLPSRUWDQWLWPD\EHLQWKHSDUWLFXODUFRQWH[W:HZLOOFRPHEDFNWRWKHVSHFLILF
VWHSZLVHSURFHVVRISLORWLQJTXHVWLRQQDLUHVLQ&KDSWHU LQ6HFWLRQ EXWDVD
SUHOLPLQDU\OHWPHTXRWHWZRRIWKHEHVWNQRZQH[SHUWVRITXHVWLRQQDLUHGHVLJQ
6XGPDQDQG%UDGEXUQ µLI\RXGRQRWKDYHWKHUHVRXUFHVWRSLORWWHVW\RXU
TXHVWLRQQDLUHGRQ¶WGRWKHVWXG\¶7KXVSLORWLQJLVDQHVVHQWLDOSDUWRITXDQWLWDWLYH
UHVHDUFKDQGDQ\DWWHPSWWRVKRUWFXWWKHSLORWLQJVWDJHZLOOVHULRXVO\MHRSDUGL]HWKH
SV\FKRPHWULFTXDOLW\RIWKHVWXG\)XUWKHUPRUHP\SHUVRQDOH[SHULHQFHLVWKDWE\
SDWLHQWO\JRLQJWKURXJKWKHSLORWLQJSURFHGXUHVZHFDQDYRLGDJUHDWGHDORIIUXVWUDWLRQ
DQGSRVVLEOHH[WUDZRUNODWHURQ
,QTXDOLWDWLYHVWXGLHVDV5LFKDUGV SRLQWVRXWWKHUHLVQRUPDOO\QRUHDO
SLORWLQJVWDJHLQZKLFKWKHUHVHDUFKµWRROV¶DUHWHVWHG7KLVGRHVQRWPHDQWKDWWKHUHLV
QRXVHLQWU\LQJRXWFHUWDLQWHFKQLTXHV IRUH[DPSOHLQWHUYLHZLQJVNLOOV EXW48$/
SLORWLQJGLIIHUVIURP48$1SLORWLQJLQWKDWZHGRQRWKDYHWRGLVFDUGWKHREWDLQHGGDWD
DIWHUWKHVHµWULDOUXQV¶EXWFDQXVHLWIRUWKHILQDODQDO\VLV7KHVHHDUO\UHVSRQVHVPD\
GLIIHUIURPWKHRQHVFRPLQJIURPODWHUVWDJHVEXWGLIIHUHQFHVDUHµIRRGIRUDQDO\VLVQRW
DSUREOHPIRUFRQVLVWHQF\¶ S
5HVHDUFKORJ
3DVWUHVHDUFKH[SHULHQFHKDVFRQYLQFHGPHWKDWLWLVHVVHQWLDOWRVWDUWDIRUPDOµUHVHDUFK
ORJ¶ULJKWDWWKHVWDUWRIDUHVHDUFKSURMHFW'DWDFROOHFWLRQDQGDQDO\VLVUHTXLUHXVWR
PDNHGHFLVLRQVRQDQRQJRLQJEDVLVDQGXQOHVVWKHVHDUHSURSHUO\GRFXPHQWHGZHDUH
OLNHO\WRVRRQIRUJHWRUPL[XSVRPHRIWKHGHWDLOV$VZLWKDQ\UHDOORJERRNDOOWKH
HQWULHVVKRXOGEHSURSHUO\GDWHGDQGWKHFRQVHFXWLYHSDJHVRIWKHORJERRNVKRXOGEH
QXPEHUHGDQGNHSWWRJHWKHULQDIROGHU$OWHUQDWLYHO\ZHFDQNHHSDQHOHFWURQLFORJ
ILOHLQRXUFRPSXWHUWKDWZHUHJXODUO\XSGDWH6XFKDORJERRNZLOOQRWRQO\KHOSWRVRUW
RXWDQ\HPHUJLQJFRQIXVLRQEXWZLOODOVRFRQWDLQLQYDOXDEOHUHFRUGHGLQIRUPDWLRQWKDW
LVUHDGLO\XVDEOHGXULQJWKHZULWLQJXSVWDJH,QGHHGµUHVHDUFKMRXUQDOV¶²WKDWLV
H[WHQGHGUHVHDUFKORJVWKDWDOVRFRQWDLQPHPRVDQGSHUVRQDOQRWHV²FDQEHDQ
LPSRUWDQWVRXUFHRIGDWDLQTXDOLWDWLYHUHVHDUFK²VHH6HFWLRQ,QTXDQWLWDWLYHVWXGLHV
WKHORJKDVPRUHRIDQRUJDQL]LQJDQGDUFKLYLQJUROHFRQWDLQLQJDZLGHYDULHW\RI
LQIRUPDWLRQVXFKDVGDWHVUHFRUGVRIPHHWLQJVH[DFWGHWDLOVRIWKHSLORWLQJDQGGDWD
FROOHFWLRQSKDVHV ZKHQ"ZKR"ZKDW" DQ\HPHUJLQJSUREOHPVUHFRUGVRIZKHUHDQG
LQZKDWIRUPWKHGDWDLVVWRUHGDQGILQDOO\DOOWKHUHOHYDQWGHWDLOVRIWKHGDWDDQDO\VLV
SURFHVVLQFOXGLQJUHFRUGVRIWKHVWDWLVWLFDOSURFHGXUHVWULHGRXW HYHQWKHGHDGHQGV DV

ZHOODVWKHPDLQILQGLQJVDQGWKHQDPHVDQGORFDWLRQVRIWKHFRPSXWHUILOHVZKLFK
FRQWDLQWKHVXSSRUWLQJVWDWLVWLFV7KHORJERRNPLJKWHYHQFRYHUWKHZULWLQJXSSKDVH
NHHSLQJGHWDLOVRIWKHYDULRXVGUDIWVDQGPHDQVRIEDFNLQJWKHPXS7KLV
RUJDQL]LQJDUFKLYLQJUROHLVYLWDOEHFDXVHQXPHULFDOGHWDLOVFDQJHWHDVLO\PL[HGXSDQG
DWVXFKWLPHVDGHWDLOHGUHFRUGWRJREDFNWRIRURULHQWDWLRQEHFRPHVOLWHUDOO\
LQYDOXDEOH6RVWDUWORJJLQJ²QRZ
7HFKQLTXHVWRPDQDJHDQGVWRUHGDWDUHFRUGV
7KHVLJQLILFDQFHRIJRRGVWUDWHJLHVWRPDQDJHDQGVWRUHWKHGDWDLVWZRIROGFRQFHUQLQJ
ERWKORJLVWLFDQGHWKLFDOLVVXHV)LUVWLWLVDFKDOOHQJLQJWDVNWRGHYLVHDVWRULQJDQG
ODEHOOLQJV\VWHPZKLFKZLOOVWRSXVIURPORVLQJRUPL[LQJXSGDWDDQGZKLFKZLOO
SURYLGHDQHIILFLHQWZD\RIORFDWLQJDQGDFFHVVLQJLQIRUPDWLRQ5HVHDUFKHUVQHHGWR
PDNHDGHWDLOHGOLVWRIDOOWKHGDWDVRXUFHVLQDUHVHDUFKORJ VHHSUHYLRXVVHFWLRQ
RUJDQL]HGDFFRUGLQJWRNH\SDUDPHWHUV IRUH[DPSOHW\SHRIGDWDSODFHRIGDWD
JDWKHULQJ DQGWKHQQHHGWRNHHSWKHVHUHFRUGVXSWRGDWH$XGLRWDSHVYLGHRFDVVHWWHV
DQGDQ\LOOXVWUDWLYHGRFXPHQWVQHHGWREHSURSHUO\ODEHOOHGDQGVWRUHGLQDVHFXUHSODFH
DQGZHDOVRQHHGDWUDQVSDUHQWV\VWHPIRUNHHSLQJUHOHYDQWFRPSXWHUILOHV
3DUWLFXODUO\LQTXDOLWDWLYHUHVHDUFKZHFDQTXLFNO\JDWKHUPDVVHVRIUDWKHU
µXQRUGHUO\¶GDWDUDQJLQJIURPILHOGQRWHVWRGRFXPHQWVDQGRIFRXUVHWUDQVFULSWV,WLV
GLIILFXOWWRLPDJLQHQRZKRZZHXVHGWRPDQDJHVWRUDJHEHIRUHSHUVRQDOFRPSXWHUV
EHFDPHZLGHVSUHDGFRQWHPSRUDU\GDWDDQDO\VLVVRIWZDUHQRWRQO\KHOSVWRSURFHVVWKH
GDWDEXWDOVRSURYLGHVH[FHOOHQWVWRUDJHIDFLOLWLHV,QDQ\FDVHLWLVLPSHUDWLYHWRIROORZ
'DYLGVRQ¶V WZRODZVIRUKDQGOLQJFRPSXWHUGDWDµ/DZ%DFNLWXS/DZ
'RLWQRZ¶,WLVDOVRXVHIXOWREDFNXSDXGLRDQGYLGHRUHFRUGLQJV
7KHVHFRQGDVSHFWRIGDWDVWRUDJHKDVDOUHDG\EHHQWRXFKHGXSRQLQ6HFWLRQ
ZKHQGLVFXVVLQJWKHLVVXHRIDQRQ\PLW\0RUVHDQG5LFKDUGV DGYLVHXVWKDW
ZKHQSDUWLFLSDQWDQRQ\PLW\LVUHTXLUHGLWLVLPSRUWDQWWKDWZHFUHDWHDQGFRQVLVWHQWO\
XVHDV\VWHPRISVHXGRQ\PVRUFRGHVERWKLQWKHWUDQVFULSWVDQGLQWKHODEHOOLQJV\VWHP
7KHOLVWRIOLQNLQJQDPHVDQGFRGHVVKRXOGEHNHSWLQDVDIHSODFHDORQJZLWKPDWHULDOV
WKDWZRXOGPDNHWKHLGHQWLILFDWLRQRIWKHSDUWLFLSDQWVSRVVLEOH IRUH[DPSOHDXWKHQWLF
GRFXPHQWVYLGHRUHFRUGLQJVHWF
,QVXPLQRUGHUWREHDEOHWRUHPHPEHUZKHUHWKLQJVDUHDQGLQZKDWIRUPRUZKDW
FHUWDLQILOHQDPHVPHDQZHQHHGWRGHYLVHDGDWDPDQDJLQJVWRULQJV\VWHPULJKWIURP
WKHEHJLQQLQJ,WLVDOOWRRHDV\WREHDELWVODSGDVKZKHQZHDUHLQIXOOVZLQJRIGDWD
FROOHFWLRQEXWH[SHULHQFHVXJJHVWVWKDWZLWKRXWDIRROSURRIV\VWHPWKLQJVZLOO
LQHYLWDEO\HQGXSLQSODFHVZKHUHWKH\DUHQRWVXSSRVHGWREH5HVHDUFKLVGLVFLSOLQHG
LQTXLU\DQGWKHQHHGWREHGLVFLSOLQHGPXVWDOVRDSSO\WRWKHDUHDRIGDWDPDQDJHPHQW

/RQJLWXGLQDOYHUVXVFURVVVHFWLRQDO
UHVHDUFK
)RUPRVWUHVHDUFKHUVORQJLWXGLQDOUHVHDUFKLVVWLOODQXQH[SORUHGODQG
IDVFLQDWLQJEXWGDQJHURXV
5XVSLQL±
6HYHUDOKLJKSURILOHVWUDQGVLQDSSOLHGOLQJXLVWLFVIRFXVRQGHYHORSPHQWDODVSHFWVRIWKH
LQGLYLGXDO IRUH[DPSOHRQWKHGHYHORSPHQWRI/OLWHUDF\RU/SURILFLHQF\ DQGWKH
VWXG\RIVRFLDOFKDQJHLVDOVRDSURPLQHQWDUHDLQWKHILHOG7KLVVDOLHQFHRI
GHYHORSPHQWDQGFKDQJHKLJKOLJKWVWKHVLJQLILFDQFHRIµORQJLWXGLQDOUHVHDUFK¶WKDWLV
WKHRQJRLQJH[DPLQDWLRQRISHRSOHRUSKHQRPHQDRYHUWLPH,QGHHG$EEXKODQG
0DFNH\ LQSUHVV UHJDUGORQJLWXGLQDOUHVHDUFKDVRQHRIWKHPRVWSURPLVLQJUHVHDUFK
GLUHFWLRQVIRUDSSOLHGOLQJXLVWLFVWRWDNH*LYHQWKLVLPSRUWDQFHLWLVUDWKHUVXUSULVLQJ
KRZOLWWOHORQJLWXGLQDOUHVHDUFKZHILQGLQWKHDSSOLHGOLQJXLVWLFVOLWHUDWXUHDQGHYHQWKH
DYDLODEOHPHWKRGRORJ\WH[WVKDYHXVXDOO\OLWWOHWRVD\DERXWWKHVXEMHFW7KLVZDUUDQWV
GHYRWLQJDVHSDUDWHFKDSWHUWRWKHWRSLF
7KHFRXQWHUSDUWRIORQJLWXGLQDOUHVHDUFKLVµFURVVVHFWLRQDOUHVHDUFK¶ZKLFKUHIHUV
WRDVQDSVKRWOLNHDQDO\VLVRIWKHWDUJHWSKHQRPHQRQDWRQHSDUWLFXODUSRLQWLQWLPH
IRFXVLQJRQDVLQJOHWLPHLQWHUYDO,WDOORZVXVWRHVWDEOLVKUHODWLRQVKLSVEHWZHHQ
YDULDEOHVDQGWRILQGRXWDERXWWKHSDUWLFLSDQWV¶WKRXJKWVDWWLWXGHVHPRWLRQVDVZHOODV
YDULRXVFRJQLWLYHDQGSHUVRQDOLW\WUDLWV7KHHSLWRPHRIWKHFURVVVHFWLRQDOVWXG\LVWKH
QDWLRQDOFHQVXVLQZKLFKDZLGHUDQJHRIGDWDLVFROOHFWHGIURPWKHZKROHSRSXODWLRQ
ZLWKLQDYHU\QDUURZWLPHSHULRG &RKHQHWDO 2IFRXUVHHYHQFHQVXVGDWDFDQ
EHWXUQHGORQJLWXGLQDOE\OLQNLQJLQIRUPDWLRQJDWKHUHGLQWZRFRQVHFXWLYHFHQVXVHV
&RQWHPSRUDU\DSSOLHGOLQJXLVWLFUHVHDUFKLVGRPLQDWHGE\FURVVVHFWLRQDOVWXGLHV
ZKLFKLVDOVRWUXHRIHGXFDWLRQDOUHVHDUFKLQJHQHUDO DQGWKHUHIRUHWKHHPSKDVLVLQWKLV
FKDSWHUZLOOEHRQWKHOHVVIDPLOLDUDSSURDFKORQJLWXGLQDOUHVHDUFK%\GHVFULELQJLWV
PDLQW\SHVDQGE\H[SORULQJLWVSRWHQWLDOEHQHILWVUHODWLYHWRFURVVVHFWLRQDOVWXGLHV,
ZRXOGOLNHWRSUHVHQWDVWURQJFDVHIRUWKHQHHGWRH[SHULPHQWPRUHZLWKORQJLWXGLQDO
GHVLJQVLQLQYHVWLJDWLRQV
'HILQLWLRQDQGSXUSRVHRIORQJLWXGLQDOUHVHDUFK
/RQJLWXGLQDOUHVHDUFKLVDUDWKHULPSUHFLVHWHUPWKDWUHIHUVWRDIDPLO\RIPHWKRGVWKDW
VKDUHRQHWKLQJLQFRPPRQLQIRUPDWLRQLVJDWKHUHGDERXWWKHWDUJHWRIWKHUHVHDUFK
ZKLFKFDQLQFOXGHDZLGHUDQJHRIXQLWVVXFKDVSHRSOHKRXVHKROGVLQVWLWXWLRQV
QDWLRQVRUFRQFHSWXDOLVVXHV GXULQJDVHULHVRISRLQWVLQWLPH:KLOHWKLVPD\VHHPOLNH
DIDLUO\VWUDLJKWIRUZDUGGHILQLWLRQLWLVLQDFFXUDWHLQWKDWZHPD\KDYHDVLQJOHGDWD
JDWKHULQJVHVVLRQWKDWZLOO\LHOGORQJLWXGLQDOLQIRUPDWLRQDERXWFKDQJHRYHUWLPH IRU

H[DPSOHDQLQWHUYLHZIRFXVLQJRQDSHUVRQ¶VOLIHKLVWRU\ DQGZHPD\DOVRKDYHD
VHULHVRIGHHSLQWHUYLHZVWKDWODVWIRUVHYHUDOZHHNVRUPRQWKVDQG\HWGRQRWWDUJHW
FKDQJHEXWUDWKHUVRPHRWKHUFRPSOH[DVSHFWRIWKHLQWHUYLHZHH¶VSHUFHSWLRQDQG
LQWHUSUHWDWLRQRIKLVKHUVRFLDOHQYLURQPHQW$FFRUGLQJO\VHYHUDOVFKRODUVSRLQWRXW
WKDWWKHUHLVQRWQHFHVVDULO\DRQHWRRQHFRUUHVSRQGHQFHEHWZHHQWKHGHVLJQRIDVWXG\
DQGWKHW\SHRIGDWDFROOHFWHG 0HQDUG7DULV
7KXVORQJLWXGLQDOUHVHDUFKPXVWEHGHILQHGLQWHUPVRIERWKWKHGDWDDQGWKHGHVLJQ
WKDWDUHXVHGLQWKHUHVHDUFK$FFRUGLQJWR0HQDUG DORQJLWXGLQDOLQYHVWLJDWLRQ
LVUHVHDUFKLQZKLFK D GDWDDUHFROOHFWHGIRUWZRRUPRUHGLVWLQFWWLPHSHULRGV E WKH
VXEMHFWVRUFDVHVDQDO\VHGDUHWKHVDPHRUDUHFRPSDUDEOH LHGUDZQIURPWKHVDPH
SRSXODWLRQ IURPRQHSHULRGWRWKHQH[WDQG F WKHDQDO\VLVLQYROYHVVRPHFRPSDULVRQ
RIGDWDEHWZHHQSHULRGV$VKHFRQFOXGHVµDVDEDUHPLQLPXPDQ\WUXO\ORQJLWXGLQDO
GHVLJQZRXOGSHUPLWWKHPHDVXUHPHQWRIGLIIHUHQFHVRUFKDQJHLQDYDULDEOHIURPRQH
SHULRGWRDQRWKHU¶ S
/RQJLWXGLQDOUHVHDUFKVHUYHVWZRSULPDU\SXUSRVHVWRGHVFULEHSDWWHUQVRIFKDQJH
DQGWRH[SODLQFDXVDOUHODWLRQVKLSV$OWKRXJKWKHVHWZRDLPVDUHLQWHUUHODWHGWKH\GR
QRWDOZD\VFRLQFLGHEHFDXVHZHPD\REWDLQSUHFLVHLQIRUPDWLRQDERXWWKHWHPSRUDO
RUGHURIHYHQWVZLWKRXWGHWHFWLQJDQ\FDXVDOFRQQHFWLRQEHWZHHQWKHVHHYHQWV$V7DULV
FRQFOXGHVLQKLVDFFHVVLEOHLQWURGXFWLRQWRWKHVXEMHFWLQRUGHUWREHDEOHWR
LGHQWLI\FDXVH±HIIHFWOLQNVZHQHHGDQH[SOLFLWWKHRU\WKDWH[SODLQVWKHFDXVDO
SURFHVVHVXQGHUO\LQJWKHWHPSRUDOREVHUYDWLRQV
:HVKRXOGQRWHDWWKLVSRLQWWKDWDOWKRXJKORQJLWXGLQDOUHVHDUFKLVUDWKHU
XQGHUXWLOL]HGLQRXUILHOGLWLVTXLWHHVWDEOLVKHGLQVRPHRWKHUGLVFLSOLQHVZLWKLQWKH
VRFLDOVFLHQFHV)RUH[DPSOHLQHFRQRPLFVWKHXVHRIORQJLWXGLQDOGDWDLVHVVHQWLDOIRU
H[DPLQLQJYDULRXVDVSHFWVRIHFRQRPLFFKDQJH IRUH[DPSOHWKHFKDQJHRIWKHUDWHRI
LQIODWLRQRUWKH*URVV'RPHVWLF3URGXFWLQGH[ DQGUHOHYDQWHFRQRPLFLQIRUPDWLRQLV
JDWKHUHGDWYHU\IUHTXHQWDQGUHJXODULQWHUYDOV7KHUHDUHDOVRVHYHUDORQJRLQJODUJH
VFDOHµKRXVHKROGVXUYH\V¶LQPDQ\FRXQWULHVGRFXPHQWLQJFKDQJHVLQSDWWHUQVRIVRFLDO
EHKDYLRXUDFURVVQDWLRQDOO\UHSUHVHQWDWLYHVDPSOHVXVXDOO\RQD\HDUO\EDVLV,Q
FRQWUDVWLQWKHW\SLFDOSUDFWLFHRIORQJLWXGLQDOUHVHDUFKLQHGXFDWLRQDQGDSSOLHG
OLQJXLVWLFVWKHXVXDOQXPEHURIGDWDFROOHFWLRQSRLQWV DOVRFDOOHGWKHCSKDVHV RU
CZDYHV RIWKHVWXG\ LVPXFKVPDOOHURIWHQRQO\WZRRUWKUHHZLWKWKHDPRXQWRIWLPH
EHWZHHQWKHZDYHVEHLQJDQ\WKLQJIURPDIHZZHHNVWRDVPXFKDVDGHFDGH
)LQDOO\LWLVDOVRLPSRUWDQWWRSRLQWRXWWKDWORQJLWXGLQDOUHVHDUFKKDVWUDGLWLRQDOO\
EHHQDVVRFLDWHGZLWKWKHTXDQWLWDWLYHSDUDGLJPDLPLQJDWSURYLGLQJVWDWLVWLFDOSLFWXUHV
RIZLGHUVRFLDOWUHQGV$V1HDOHDQG)ORZHUGHZ H[SODLQLQVXFKVWXGLHV
VRFLHWLHVDUHVWXGLHGIURPWKHVRFLDOVWUXFWXUHµGRZQZDUGV¶UDWKHUWKDQIURPWKH
LQGLYLGXDORUSHUVRQDOQHWZRUNµXSZDUGV¶$FFRUGLQJWRWKHVHDXWKRUVWKLV
PHWKRGRORJ\WKHQRIIHUV

DELUG¶VH\HYLHZRIVRFLDOOLIHWKDWLVSDQRUDPLFLQVFRSHEXWODFNLQJLQDQ\
GHWDLO,WLVDQHSLFPRYLHRIIHULQJDJUDQGQDUUDWLYHDQGKHUHLQOLHVLWVYDOXHEXW
LWLVDPRYLHLQZKLFKWKHLQWULFDFLHVRIWKHSORWDQGWKHIOXLGWZLVWVDQGWXUQVRI
WKHLQGLYLGXDOVWRU\OLQHVDUHKLGGHQIURPYLHZ S
5HFHQWO\KRZHYHUWKHUHKDVEHHQDPRYHLQWKHVRFLDOVFLHQFHVWRPDNH
ORQJLWXGLQDOTXDOLWDWLYHUHVHDUFKPRUHSURPLQHQW7KHGLVWLQFWLYHQHVVRIVXFKVWXGLHVLV
JLYHQE\WKHLQWHUSOD\RIWKHWHPSRUDODQGFXOWXUDOGLPHQVLRQVRIVRFLDOOLIHRIIHULQJD
µERWWRPXS¶XQGHUVWDQGLQJRIKRZSHRSOHPRYHWKURXJKWLPHDQGFUDIWWKHWUDQVLWLRQ
SURFHVVHV7KXVWKHTXDOLWDWLYHPRYLHRIIHUVXVµD³FORVHXS´RIWKHIDEULFRIUHDOOLYHV
DVRSSRVHGWRWKHTXDQWLWDWLYH³ORQJVKRW´7KHIRFXVLVRQWKHSORWDQGGHWDLOHGVWRU\
OLQHVRIWKHNH\DFWRUVUDWKHUWKDQWKHJUDQGYLVWDVRIWKHHSLFSLFWXUH¶ 1HDOHDQG
)ORZHUGHZ 6HFWLRQGLVFXVVHVWKLVTXDOLWDWLYHPRYHPHQWLQVRPHGHWDLO
DQG6HFWLRQKLJKOLJKWVWKHIDFWWKDWORQJLWXGLQDOUHVHDUFKDOVROHQGVLWVHOIWRPL[HG
PHWKRGVVWXGLHV
/RQJLWXGLQDOUHVHDUFKLQDSSOLHGOLQJXLVWLFV
,QDQLQVLJKWIXOVWXG\ZULWWHQDGHFDGHDJR0HOORZHWDO¶V DUJXHGIRUFHIXOO\
WKDWWUDQVLWLRQWKHRULHVIRFXVLQJRQWKHGHYHORSPHQWDOFRXUVHRIODQJXDJHRYHUWLPHDUH
DFHQWUDODVSHFWRI6/$UHVHDUFKDQGWKDWWKHLUHIIHFWLYHVWXG\UHTXLUHVORQJLWXGLQDO
UHVHDUFKPHWKRGVVXFKDVDWLPHVHULHVGHVLJQ0HOORZHWDO¶VFRQFOXVLRQZDV
UHLWHUDWHGLQDUHFHQWRYHUYLHZRIORQJLWXGLQDOUHVHDUFKLQ6/$E\2UWHJDDQG,EHUUL
6KHD ZKRDVVHUWHGWKDWEHFDXVHODQJXDJHOHDUQLQJKDSSHQVWKURXJKDQGRYHU
WLPHPDQ\LIQRWDOOWRSLFVDERXW/OHDUQLQJWKDW6/$UHVHDUFKHUVLQYHVWLJDWHFDQEH
PRVWPHDQLQJIXOO\LQWHUSUHWHGRQO\ZLWKLQDIXOOORQJLWXGLQDOSHUVSHFWLYH$VWKH\
FRQFOXGHGµ8OWLPDWHO\LWLVWKURXJKFXPXODWLYHORQJLWXGLQDOILQGLQJVWKDWWKH6/$
UHVHDUFKFRPPXQLW\ZRXOGEHDEOHWRFRQWULEXWHPHDQLQJIXOFKDUDFWHUL]DWLRQVRIWKH
JUDGXDOSURFHVVRIDWWDLQLQJDGYDQFHGVHFRQGODQJXDJHDQGOLWHUDF\FRPSHWHQFLHV
DFURVVYDULRXVFRQWH[WV¶ S +RZHYHUWKHWKHRUHWLFDOFHQWUDOLW\RIWLPHLQ6/$
UHVHDUFKDVWKHDXWKRUVSRLQWHGRXWLVLQVWDUNFRQWUDVWZLWKWKHIDFWWKDWGLVFXVVLRQV
DERXWORQJLWXGLQDOUHVHDUFKDUHVFDUFHLQRXUILHOG
$OWKRXJK,DPLQDJUHHPHQWZLWK2UWHJDDQG,EHUUL6KHD¶V FRQFOXVLRQWKDW
WKHUHLVDGHDUWKRISULQFLSOHGDQGSURIHVVLRQDOO\H[HFXWHGORQJLWXGLQDOVWXGLHVLQ
DSSOLHGOLQJXLVWLFUHVHDUFKLIZHORRNDWWKHILHOGPRUHFORVHO\WKHWLPHHOHPHQWLV
DFWXDOO\SUHVHQWLQPDQ\DUHDV:HILQGIRUH[DPSOHVHYHUDOSRVWVWUXFWXUDOLVW
TXDOLWDWLYHLQYHVWLJDWLRQVXVXDOO\IUDPHGZLWKLQVRFLRFXOWXUDORUODQJXDJHVRFLDOL]DWLRQ
WKHRU\WKDWGLVSOD\ORQJLWXGLQDOHWKQRJUDSKLFGHVLJQV IRUH[DPSOH3DYOHQNR LQ
IDFWERWKHWKQRJUDSK\DQGFDVHVWXG\UHVHDUFKHPSKDVL]HSURORQJHGHQJDJHPHQWZLWK
WKHSDUWLFLSDQWV VHH6HFWLRQVDQG PDNLQJWKHPLQKHUHQWO\ORQJLWXGLQDO
+RZHYHU2UWHJDDQG,EHUUL6KHDDUHULJKWLQSRLQWLQJRXWWKDWPDQ\RIWKHVWXGLHVERUQ
LQWKLVYHLQDUHQRWFHQWUHGDURXQGH[SOLFLWO\IRUPXODWHGORQJLWXGLQDOUHVHDUFKJRDOV$
VLPLODUVLWXDWLRQH[LVWVLQTXDQWLWDWLYHUHVHDUFKEHFDXVHWKHFRPPRQUHVHDUFKW\SH

DFFRUGLQJWR2UWHJDDQG,EHUUL6KHDLVWKHVPDOOVFDOHGHVFULSWLYHTXDQWLWDWLYHGHVLJQ
ZLWKRXWHPSOR\LQJDQ\LQIHUHQWLDOVWDWLVWLFV7KXVZHKDYHPDQ\µORQJLWXGLQDOOLNH¶
LQYHVWLJDWLRQVLQERWKSDUDGLJPVWKDWDUHVLPSO\QRWGHVLJQHGFRQGXFWHGDQGDQDO\VHG
DFFRUGLQJWRPRGHUQORQJLWXGLQDOSULQFLSOHV$VZHZLOOVHHODWHULQWKLVFKDSWHU LQ
6HFWLRQVDQG WKLVLVSDUWO\FDXVHGE\WKHXQGHQLDEOHIDFWWKDWWRGRORQJLWXGLQDO
GDWDDQDO\VLVIXOOMXVWLFHLVQRHDV\WDVN
3HUKDSVWKHPRVWVXFFHVVIXODUHDRIORQJLWXGLQDOUHVHDUFKLQDSSOLHGOLQJXLVWLFVKDV
EHHQWKHLQYHVWLJDWLRQRI/LQVWUXFWLRQDOHIIHFWLYHQHVVXVLQJH[SHULPHQWDODQG PRUH
RIWHQ TXDVLH[SHULPHQWDOGHVLJQV²VHH6HFWLRQ+RZHYHUDV2UWHJDDQG,EHUUL
6KHD VXJJHVWHYHQLQWKLVDUHDWKHSLFWXUHLVQRWHQWLUHO\URV\EHFDXVH
UHVHDUFKHUVGRQRWVHHPWRWDNHDGYDQWDJHRIWLPHVHULHVGHVLJQVWKDWFRXOGHQKDQFHWKH
HIIHFWLYHQHVVRIVXFKVWXGLHV )RUDGLVFXVVLRQVHH6HFWLRQ 7KHSULQFLSOHVRIWKLV
GHVLJQDQGLWVUHOHYDQFHWR6/$ZHUHODLGRXWE\0HOORZHWDO DGHFDGHDJRDQG
\HWLQWKHLUUHYLHZRIWKHOLWHUDWXUH2UWHJDDQG,EHUUL6KHDZHUHDEOHWRLGHQWLI\RQO\
RQHUHFHQWVWXG\DGRSWLQJLW
0DLQW\SHVRIORQJLWXGLQDOGHVLJQV
7KHFODVVLFORQJLWXGLQDOGHVLJQLQYROYHVWDNLQJDJURXSRISHRSOHDQGIROORZLQJWKHLU
GHYHORSPHQWE\PXOWLSOHLQYHVWLJDWLRQVRYHUDSHULRGRIWLPH7KLVGHVLJQLVXVXDOO\
ODEHOOHGDµSDQHOVWXG\¶EXWLWLVQRWWKHRQO\ZD\RIJDWKHULQJHPSLULFDOGDWDDERXW
FKDQJH7KHUHDUHDVPDQ\DVWKUHHRWKHUZHOONQRZQORQJLWXGLQDODSSURDFKHVDQGLQ
PDQ\FDVHVWKH\PD\EHPRUHDSSURSULDWH DQGFKHDSHU WKDQWKHSDQHOVWXG\/HWXV
ORRNDWWKHIRXUPDLQGHVLJQW\SHVLQVRPHGHWDLO$OWKRXJKWKHXQLWVRIPHDVXUHPHQWLQ
ORQJLWXGLQDOVWXGLHVFDQEHLQVWLWXWLRQVSHRSOHVRUQDWLRQVORQJLWXGLQDOUHVHDUFKLV
W\SLFDOO\WDUJHWLQJSHRSOHDQGLQWKHIROORZLQJGHVFULSWLRQV,ZLOOUHIHURQO\WRKXPDQ
VXEMHFWV
3URVSHFWLYHORQJLWXGLQDOVWXGLHVRUµSDQHOVWXGLHV¶
7KHGHVLJQW\SHLQZKLFKVXFFHVVLYHPHDVXUHVDUHWDNHQDWGLIIHUHQWSRLQWVLQWLPHIURP
WKHVDPHUHVSRQGHQWVKDVEHHQFDOOHGGLIIHUHQWQDPHVIRUH[DPSOHµSURVSHFWLYH
ORQJLWXGLQDOVWXG\¶µIROORZXSVWXG\¶µFRKRUWVWXG\¶RUµSDQHOVWXG\¶ZLWKWKHODWWHU
EHFRPLQJWKHVWDQGDUGODEHO7KHWHUPZDVRULJLQDOO\FRLQHGLQWKH86LQWKHVE\
VRFLRORJLVW3DXO/D]DUVIHOGZKHQKHZDVVWXG\LQJWKHORQJWHUPPDUNHWLQJHIIHFWVRI
UDGLRDGYHUWLVLQJ7KHPDLQUHDVRQIRUWKHSRSXODULW\RIWKHSDQHOVWXG\LVWKHIDFWWKDW
LWDOORZVXVWRFROOHFWLQIRUPDWLRQDERXWFKDQJHDWWKHPLFUROHYHODVLWUHDOO\KDSSHQV
7RLOOXVWUDWHWKHLPSRUWDQFHRIWKLVOHWXVFRQVLGHUWZRVXFFHVVLYHRSLQLRQSROOV
UHJDUGLQJWZRFRPSHWLQJSROLWLFDOSDUWLHV$DQG%ZKLFKH[DPLQH²XQOLNHDSDQHO
VWXG\²GLIIHUHQWVDPSOHVRIWKHSRSXODWLRQ(YHQLIWKHUHVXOWVVKRZWKDWWKHSRSXODULW\
RI3DUW\$LQFUHDVHGRYHUWLPHZHFDQQRWWHOOIURPWKHGDWDZKHWKHUWKLVZDVDXQLIRUP
VKLIWIURP3DUW\%WR3DUW$RUZKHWKHUVRPHSHRSOHDFWXDOO\FKDQJHGIURP3DUW\$WR
3DUW\%EXWWKH\ZHUHRXWQXPEHUHGE\RWKHUVZKRFKDQJHGIURP3DUW\%WR3DUW\$

7KXVVXFFHVVLYHRSLQLRQSROOVZKLFKDUHWHFKQLFDOO\VSHDNLQJµUHSHDWHGVXUYH\V¶ WR
EHGLVFXVVHGEHORZ FDSWXUHRQO\µQHWFKDQJH¶WKDWLVWKHQHWHIIHFWRIDOOWKHFKDQJHV
)LUHEDXJK ZKHUHDVSURSHUSDQHOGHVLJQVHQDEOHUHVHDUFKHUVWRREVHUYHDOOWKH
VSHFLILFFKDQJHVDFURVVWLPH LHLWZRXOGFDSWXUHWKHDFWXDOUDWHRISDUW\VZLWFKLQJ
DPRQJLQGLYLGXDOVLQRXUH[DPSOH 7KXVSDQHOVWXGLHVRIIHUDSRZHUIXO
QRQH[SHULPHQWDOPHWKRGIRUH[DPLQLQJGHYHORSPHQWDQGFDXVDOLW\
8QIRUWXQDWHO\SDQHOVWXGLHVDUHUDWKHUH[SHQVLYHDQGWLPHFRQVXPLQJWRUXQDQG
WKH\DOVRUHTXLUHDFRPPLWWHGUHVHDUFKWHDPPDLQWDLQHGDFURVV\HDUV,QDGGLWLRQWKHVH
GHVLJQVDOVRVXIIHUIURPWZRVHULRXVWKUHDWVWRWKHLUYDOLGLW\µDWWULWLRQ¶DQGµSDQHO
FRQGLWLRQLQJ¶
x $WWULWLRQ7KHXVXDOSDWWHUQRISDUWLFLSDWLRQLQDORQJWHUPSDQHOVWXG\LVWKDWDQ
LQFUHDVLQJQXPEHURISDUWLFLSDQWVGURSRXWRIWKHSDQHOLQWKHVXFFHVVLYHZDYHV
7KHUHFDQEHPXOWLSOHUHDVRQVIRUWKLVDWWULWLRQ RUµPRUWDOLW\¶ WKHUHPD\EHIRU
H[DPSOHORJLVWLFRQHV QRQDYDLODELOLW\FKDQJLQJDGGUHVVRUWHOHSKRQHQXPEHU
HWF RUSDQHOPHPEHUVPD\EHFRPHLOORUVLPSO\XQZLOOLQJWRFRQWLQXHEHFDXVHRI
DODFNRIWLPHRUORVVRILQWHUHVW6XFKDWWULWLRQLVFXPXODWLYHEHFDXVHRQFHVRPHRQH
KDVPLVVHGDGDWDFROOHFWLRQZDYHWKDWSHUVRQLVORVWIRUWKHUHPDLQGHURIWKHVWXG\
7DULV 7KXVLQDSDQHOVWXG\WKHVDPSOHZLWKDIXOOGDWDVHWFDQGHFUHDVH
VLJQLILFDQWO\DQGPRUHZRUU\LQJIURPDYDOLGLW\SHUVSHFWLYHWKHUHGXFWLRQPD\
QRWEHUDQGRP7KDWLVLWPD\EHFHUWDLQW\SHVRISDUWLFLSDQWVZKRJLYHXSHDUO\RU
WRWXUQLWURXQGZKRVWLFNWRWKHVWXG\WRWKHHQG7KHILQDOVDPSOHPD\WKHUHIRUH
RIIHUDELDVHGUHSUHVHQWDWLRQRIWKHRULJLQDOVDPSOHDQGWKHSRSXODWLRQ:LWKSURSHU
WUDFLQJDQGVWD\LQJLQWRXFKVWUDWHJLHV UDQJLQJIURPVHQGLQJ&KULVWPDVFDUGVDQG
UHVHDUFKXSGDWHVWRUHJXODUWHOHSKRQHFRQWDFW WKHUDWHRIQRQUHVSRQVHFDQEH
UHGXFHGEXWWKHVHHIIRUWVUHTXLUHVXEVWDQWLDOKXPDQDQGILQDQFLDOUHVRXUFHV )RUD
GHWDLOHGDFFRXQWVHH7KRPVRQDQG+ROODQG ,QHGXFDWLRQUHVHDUFKSDQHO
VWXGLHVFDQEHSDUWLFXODUO\GLIILFXOWWRFRQGXFWEHFDXVHRIWKHIUHTXHQWFKDQJHVLQ
FODVVDQGVWDIIFRPSRVLWLRQDVZHOODVWHDFKLQJPHWKRGRORJ\
x 3DQHOFRQGLWLRQLQJ7KHVHFRQGSUREOHPFRQFHUQVWKHIDFWWKDWZKHQDJURXSRI
SHRSOHWDNHSDUWLQDORQJLWXGLQDOVWXG\WKHUHLVDUHDOGDQJHUWKDWWKHUHJXODU
PHHWLQJVWKDWWKHSDUWLFLSDWLRQLQYROYHVDQGWKHNQRZOHGJHRIEHLQJSDUWRIWKH
VWXG\FDQDOWHUWKHSDQHOPHPEHUV¶EHKDYLRXUDQGUHVSRQVHV7KH\PD\IRU
H[DPSOHORVHWKHLULQKLELWLRQDERXWWKHGDWDFROOHFWLRQIRUPDWRUWKHH[SHULHQFHRI
KDYLQJWRIRFXVRQFHUWDLQLVVXHVLQWKHLUOLYHVUDLVHVWKHLUDZDUHQHVVRIRU
VHQVLWLYLW\WRWKHVHLVVXHV$OWHUQDWLYHO\WKH\PD\DOVREHKDYHGLIIHUHQWO\EHFDXVH
WKH\ZDQWWRSOHDVHWKHUHVHDUFKHUVZKRPWKH\DUHJHWWLQJWRNQRZEHWWHUDQG
EHWWHU7KXVWKHUHVXOWLQJµSDQHOFRQGLWLRQLQJ¶HIIHFWFDQEHVHHQDVDFRPELQDWLRQ
RIµSUDFWLFHHIIHFW¶DQGWKHµ+DZWKRUQHHIIHFW¶²VHH6HFWLRQ
5HSHDWHGFURVVVHFWLRQDOVWXGLHVRUµWUHQGVWXGLHV¶

$SRSXODUZD\RIREWDLQLQJLQIRUPDWLRQDERXWFKDQJHLVE\DGPLQLVWHULQJUHSHDWHG
TXHVWLRQQDLUHVXUYH\VWRGLIIHUHQWVDPSOHVRIUHVSRQGHQWV,IWKHVXEVHTXHQWZDYHV
H[DPLQHVDPSOHVWKDWDUHUHSUHVHQWDWLYHRIWKHVDPHSRSXODWLRQWKHQWKHUHVXOWVFDQEH
VHHQWRFDUU\ORQJLWXGLQDOLQIRUPDWLRQDWWKHDJJUHJDWHOHYHO LHIRUWKHZKROHJURXS
UDWKHUWKDQIRUWKHLQGLYLGXDOV 6XFKUHSHDWHGFURVVVHFWLRQDOVWXGLHVDUHXVXDOO\FDOOHG
µWUHQGVWXGLHV¶DQGLIRXUUHVHDUFKWRSLFFRQFHUQVPDFUROHYHODVSHFWVRIVRFLDOFKDQJH
IRUH[DPSOHORRNLQJDWDZKROHFRPPXQLW\¶VHYROYLQJEHOLHIVRUSUDFWLFHV WKLVOHYHO
RIDQDO\VLVLVDSSURSULDWH)XUWKHUPRUHWKHGHVLJQDOVRPDNHVLWSRVVLEOHWRLQYHVWLJDWH
DQGFRPSDUHFKDQJHVLQYDULRXVVXEVDPSOHV IRUH[DPSOHPDOHVDQGIHPDOHVHWKQLF
JURXSVHWF
7UHQGVWXGLHVDUHQRWPHUHO\µVHFRQGEHVW¶ORQJLWXGLQDOVWXGLHV²DOWKRXJKLQGLYLGXDO
GHYHORSPHQWFDQQRWEHWUDFHGLQWKHPWKH\KDYHFHUWDLQDGYDQWDJHVRYHUSDQHOVWXGLHV
)LUVWRIDOOWKH\DUHXVXDOO\FKHDSHUDQGHDVLHUWRDUUDQJHDQGFRQGXFWZKLFKLVSDUWO\
GXHWRWKHIDFWWKDWWKHUHVSRQGHQWVFDQUHPDLQDQRQ\PRXVDQGSDUWO\WKDWRUJDQL]LQJD
QHZVDPSOHWHQGVWREHVLPSOHUWKDQWUDFNLQJGRZQWKHSDUWLFLSDQWVRIWKHSUHYLRXV
VXUYH\$IXUWKHUDGYDQWDJHLVWKDWWUHQGVWXGLHVGRQRWVXIIHUIURPDWWULWLRQRU
FRQQGLWLRQLQJDQGHDFKZDYHFDQEHPDGHUHSUHVHQWDWLYHRIWKHSRSXODWLRQ7KHUHLVD
GRZQVLGHKRZHYHUWKHPDFURSHUVSHFWLYHZKLFKLVFKDUDFWHULVWLFRIWKHLUGHVLJQLVQRW
SDUWLFXODUO\VXLWHGWRUHVROYHLVVXHVRIFDXVDORUGHURUWRVWXG\GHYHORSPHQWDOSDWWHUQV
7DULV
$IXUWKHUSUREOHPPLJKWEHFDXVHGE\WKHIDFWWKDWLQRUGHUWRHQVXUHWKH
FRPSDUDELOLW\RIWKHPHDVXUHPHQWVDFURVVWLPHZHQHHGWRXVHWKHVDPHTXHVWLRQQDLUH
LQDOOWKHZDYHVRIWKHVWXG\+RZHYHUHGXFDWLRQDOV\VWHPVDQGFXUULFXODDUHUDUHO\
VWDWLFIRUORQJSHULRGVDQGWKHUHIRUHWKHIXUWKHUWKHVWXG\JHWVIURPWKHLQLWLDOSKDVHWKH
PRUHOLNHO\LWEHFRPHVWKDWVRPHSDUWVRIWKHTXHVWLRQQDLUHHPSOR\HGZLOOQRWDGGUHVV
WKHWDUJHWLVVXHVDSSURSULDWHO\RUWKDWVRPHQHZO\HPHUJLQJDUHDVZLOOEHQHJOHFWHG
.HHYHV :KHQZHLQLWLDOO\GHVLJQWKHLQVWUXPHQWWREHXVHGZHPD\QRWNQRZ
ZKDWWUHQGVZHDUHJRLQJWRGHWHFWDQGWKHUHIRUHWKHTXHVWLRQQDLUHPD\QRWEHIXOO\
VXLWDEOHIRUDQDO\VLQJWKHVSHFLILFIDFWRUVDVVRFLDWHGZLWKLW²\HWZHFDQQRWFKDQJHWKH
LQVWUXPHQWZLWKRXWFRPSURPLVLQJLWVORQJLWXGLQDOFDSDFLW\
5HWURVSHFWLYHORQJLWXGLQDOVWXGLHV
$PDMRUGHWHUUHQWRIERWKSDQHODQGWUHQGVWXGLHVLVWKHIDFWWKDWZHKDYHWRZDLWD
FRQVLGHUDEOHSHULRGRIWLPHEHIRUHZHFDQVHHUHVXOWVµ5HWURVSHFWLYHORQJLWXGLQDO
VWXGLHV¶RIIHUDZD\DURXQGWKLVGHOD\DOWKRXJKQRWZLWKRXWDFRVW5HWURVSHFWLYH
ORQJLWXGLQDOGDWDDUHJDWKHUHGGXULQJDVLQJOHLQYHVWLJDWLRQLQZKLFKUHVSRQGHQWVDUH
DVNHGWRWKLQNEDFNDQGDQVZHUTXHVWLRQVDERXWWKHSDVW7KLVPD\VRXQGOLNHD
VWUDLJKWIRUZDUGLGHDWKDWFDQVDYHXVDORWRIWLPHDQGPRQH\EXWXQIRUWXQDWHO\SDVW
UHWURVSHFWLYHUHVHDUFKKDVUHYHDOHGWKDWWKHTXDOLW\RIWKHUHFROOHFWHGGDWDFDQEHYHU\
XQHYHQDVPXFKDVSHUFHQWRIWKHUHVSRQVHVPD\EHLQFRUUHFWRULQDFFXUDWHLQVRPH
ZD\ 7DULV 7KHUHWURVSHFWLYHDFFRXQWVFDQEHVLPSOLILHGRUVHOHFWLYHZLWKVRPH

LPSRUWDQWGHWDLOVRPLWWHGVXSSUHVVHGRUVLPSO\ZURQJ)XUWKHUPRUHE\ORRNLQJDWWKH
SDVWWKURXJKWLQWHGJODVVHV ZKLFKZHRIWHQGR WKHUHFRQVWUXFWLRQRIWKHSDUWLFLSDQWV¶
H[SHULHQFHVPLJKWEHGLVWRUWHGDQGSDVWIHHOLQJVDQGHYHQWVPLJKWEHUHLQWHUSUHWHGWR
PDWFKVXEVHTXHQWHYHQWVRUWKHUHVSRQGHQWV¶FXUUHQWSHUFHSWLRQVRUVLPSO\WRILWVRPH
FRKHUHQWVWRU\OLQH$VDUHVXOWUHWURVSHFWLYHGDWDWHQGVWREHPRUHXQUHOLDEOHWKDQ
SURVSHFWLYHGDWDDQGLWLVRIWHQGLIILFXOWWRVHSDUDWHUHDOIURPSHUFHLYHGRUSXWDWLYH
FDXVHV &RKHQHWDO 2QWKHRWKHUKDQGLIRXUVWXG\IRFXVHVRQDUHODWLYHO\VKRUW
SHULRG ZHHNVRUPRQWKVUDWKHUWKDQ\HDUV DUHWURVSHFWLYHGHVLJQPD\EHDSSURSULDWH
HVSHFLDOO\LIWKHGDWDFRQFHUQVSULPDULO\HYHQWVRUEHKDYLRXUUDWKHUWKDQDWWLWXGHVRU
EHOLHIV 5XVSLQL
6LPXOWDQHRXVFURVVVHFWLRQDOVWXGLHV
7KHIRXUWKGHVLJQLQWURGXFHGKHUHWKHµVLPXOWDQHRXVFURVVVHFWLRQDOVWXG\¶LVRQO\
SDUWLDOO\ORQJLWXGLQDOEHFDXVHLWGRHVQRWLQYROYHWKHH[DPLQDWLRQRIFKDQJHRYHUWLPH
EXWUDWKHUDFURVVDJHJURXSV:LWKLQWKLVGHVLJQDFURVVVHFWLRQDOVXUYH\LVFRQGXFWHG
ZLWKGLIIHUHQWDJHJURXSVVDPSOHG IRUH[DPSOHLQYHVWLJDWLQJIRXUGLIIHUHQW\HDUVLQD
VFKRRO 7DULV SRLQWVRXWWKDWLQHIIHFWPRVWVXUYH\VPLJKWEHFRQVLGHUHGDQ
H[DPSOHRIWKLVGHVLJQVLQFHQHDUO\DOOLQYHVWLJDWLRQVJDWKHUVRPHLQIRUPDWLRQDERXWWKH
UHVSRQGHQWV¶DJH+RZHYHULQDVLPXOWDQHRXVFURVVVHFWLRQDOVWXG\WKHUHVSRQGHQWV¶
DJHLVWKHNH\VDPSOLQJYDULDEOHZKHUHDVLQµRUGLQDU\¶FURVVVHFWLRQDOGHVLJQVDJHLV
MXVWDQRWKHUYDULDEOHWREHFRQWUROOHG
7KLVGHVLJQLVVWUDLJKWIRUZDUGDQGHFRQRPLFDODQGEHFDXVHLW\LHOGVGDWDDERXW
FKDQJHVDFURVVDJHJURXSVLWFDQEHXVHGWRH[DPLQHGHYHORSPHQWDOLVVXHV2QH
SUREOHPKRZHYHULVWKDWLWPHDVXUHVGLIIHUHQWFRKRUWVDQGWKHREVHUYHGFKDQJHVPD\
QRWEHGXHWRWKHDJHGLIIHUHQFHEXWWRWKHVSHFLDOH[SHULHQFHVRIDFRKRUW ZKLFKDUH
XVXDOO\UHIHUUHGWRDVWKHµFRKRUWHIIHFWV¶ )XUWKHUPRUHVLPLODUWRWUHQGVWXGLHVZH
QHHGWRDGPLQLVWHUWKHVDPHTXHVWLRQQDLUHWRDOOWKHGLIIHUHQWVXEJURXSVZKLFKPD\
PHDQWKDWZHKDYHWRGHYLVHJHQHULFLWHPVWKDWFDQQRWEHWRRDJHVSHFLILF
2WKHUORQJLWXGLQDOGHVLJQV
7KHIRXUEDVLFGHVLJQW\SHVPHQWLRQHGDERYHKDYHEHHQFRPELQHGLQVRPHLQJHQLRXV
ZD\VWRUHPHG\VRPHRIWKHZHDNQHVVHVLQKHUHQWWRWKHVSHFLILFDSSURDFKHV 5XVSLQL
,QµURWDWLQJSDQHOV¶DFHUWDLQSURSRUWLRQRIWKHVDPSOHLVUHSODFHGDWHDFKZDYH
WRFRUUHFWFRPSRVLWLRQGLVWRUWLRQVGXHWRDWWULWLRQRUWRPDWFKWKHVDPSOHWRWKH
FKDQJLQJSRSXODWLRQVXFKVXUYH\VWKXVFRPELQHWKHIHDWXUHVRIERWKSDQHODQG
UHSHDWHGFURVVVHFWLRQDOVWXGLHVµ6SOLWSDQHOV¶DOVRFRPELQHORQJLWXGLQDODQGFURVV
VHFWLRQDOGHVLJQVE\LQFOXGLQJDµFODVVLF¶SDQHOZKLFKLVDFFRPSDQLHGE\DQDGGLWLRQDO
URWDWLQJVDPSOHVXUYH\HGDORQJVLGH7KLVH[WUDJURXSLVLQYHVWLJDWHGRQO\RQFHDQGLV
WKHUHIRUHQRWH[SRVHGWRSDQHOFRQGLWLRQLQJµ/LQNHG¶RUµDGPLQLVWUDWLYHSDQHOV¶DUH
FUHDWHGDUELWUDULO\IURPH[LVWLQJGDWDZKLFKKDVQRWEHHQRULJLQDOO\FROOHFWHGIRU
ORQJLWXGLQDOSXUSRVHVEXWZKLFKFRQWDLQLGHQWLILFDWLRQFRGHV XVXDOO\QDPHV

DFFRPSDQLHGE\GDWHVDQGSODFHVRIELUWK WKDWDOORZWKHSDUWLFLSDQWVWREHPDWFKHG
ZLWKRWKHUGDWDREWDLQHGIURPWKHP IRUH[DPSOHFHQVXVGDWD
$IXUWKHUWHPSRUDOGHVLJQLVWKHµFRKRUWVWXG\¶ZKLFKLVVRPHZKHUHLQEHWZHHQD
SDQHODQGDWUHQGVWXG\,WLQYROYHVWKHRQJRLQJLQYHVWLJDWLRQRIDELUWKFRKRUW LHWKH
VHWRISHRSOHZKRZHUHERUQLQWKHVDPH\HDU EXWLQVWHDGRIH[DPLQLQJHDFKPHPEHU
RIWKHVDPSOHLQHDFKZDYH DVWKHSDQHOVWXG\GRHV DVHULHVRIUHSHDWHGFURVV
VHFWLRQDOVXUYH\VDUHDGPLQLVWHUHGWRVHOHFWLYHPHPEHUVHDFKWLPH&RKRUWVWXGLHVFDQ
DOVRLQFRUSRUDWHUHWURVSHFWLYHHOHPHQWV
&ROOLQV GHVFULEHVDQLQWHUHVWLQJPHWKRGRIREWDLQLQJSURVSHFWLYHOLNHGDWDLQ

OHVVWLPH,QWKHµDFFHOHUDWHGORQJLWXGLQDOGHVLJQ PXOWLSOHFRKRUWVRIGLIIHUHQWDJHVDUH
REVHUYHGORQJLWXGLQDOO\IRUDVKRUWHUSHULRGRIWLPH7KHLGHDLVWKDWLIWKHFRKRUWVDUH
VOLJKWO\RYHUODSSLQJZHFDQVWDWLVWLFDOO\FRPELQHWKHFRKRUWVDQGHVWLPDWHDVLQJOH
JURZWKWUDMHFWRU\H[WHQGLQJIURPWKH\RXQJHVWDJHREVHUYHGWRWKHROGHVW
([SHULPHQWDOUHVHDUFK VHH6HFWLRQ LVDOVRDW\SHRIDSDQHOVWXG\LQZKLFKSDUW
RIWKHVDPSOHLVH[SRVHGWRVRPHWUHDWPHQWLQWHUYHQWLRQPDQLSXODWLRQRUVRPH
SODQQHGOHDUQLQJH[SHULHQFH7KHVWXG\VHHNVWRILQGRXWZKHWKHURUQRWWKHVH
SDUWLFLSDQWVH[SHULHQFHGDFKDQJHWKDWZDVGLIIHUHQWLQGLUHFWLRQRUPDJQLWXGHIURPDQ\
FKDQJHH[SHULHQFHGE\WKHUHVWRIWKHVDPSOHZKRGLGQRWUHFHLYHDQ\WUHDWPHQW
,QGHHG-RKQVRQDQG&KULVWHQVHQ DUJXHWKDWZKHQWKHUHVHDUFKHU¶VREMHFWLYHLVWR
VWXG\FDXVHDQGHIIHFWDSDQHOVWXG\ZLWKVRPHVRUWRIPDQLSXODWLRQFDQEHVHHQDVD
PRUHSRZHUIXOYHUVLRQRIWKHRUGLQDU\SDQHOGHVLJQ2QHZD\RIDFKLHYLQJWKLVLVE\
DSSO\LQJDQµLQWHUUXSWHGWLPHVHULHVGHVLJQ¶$V-RKQVRQDQG&KULVWHQVHQGHVFULEHLQ
WKLVGHVLJQDVLQJOHJURXSRISDUWLFLSDQWVLVSUHWHVWHGDQXPEHURIWLPHVEHIRUHWKH
WUHDWPHQW LHGXULQJWKHµEDVHOLQHSKDVH¶ DQGWKHQPXOWLSOHSRVWWHVWPHDVXUHVDUH
WDNHQDWYDULRXVSRLQWVGXULQJWKHWUHDWPHQW,IWKHUHLVDWUHDWPHQWHIIHFWLWZLOOEH
UHIOHFWHGE\WKHGLIIHULQJSDWWHUQVRIWKHSUHWHVWYHUVXVWKHSRVWWHVWUHVSRQVHV7KLV
GLVFRQWLQXLW\FRXOGEHPDQLIHVWHGE\DFKDQJHLQWKHOHYHORIWKHSUHWHVWDQGSRVWWHVW
UHVSRQVHV IRUH[DPSOHDFKDQJHLQWKHVORSHRUGLUHFWLRQRIWKHHYROYLQJVFRUHV
)LQDOO\µGLDU\VWXGLHV¶GHVFULEHGLQ6HFWLRQDUHDOVRLQKHUHQWO\ORQJLWXGLQDODV
WKH\FRQWDLQGDWDUHFRUGHGRYHUWLPH
/RQJLWXGLQDOTXDOLWDWLYHUHVHDUFK
4XDOLWDWLYHUHVHDUFKVWXGLHVKDYHRIWHQHPSOR\HGORQJLWXGLQDOHOHPHQWVLQWKHSDVWIRU
H[DPSOHZKHQLQIRUPDQWVZHUHUHLQWHUYLHZHGUHVHDUFKVLWHVUHYLVLWHGRUPRUH
JHQHUDOO\WKURXJKWKHUHVHDUFKHU¶VVXVWDLQHGHQJDJHPHQWZLWKWKHSURMHFW+RZHYHUDV
7KRPVRQHWDO H[SODLQLQWKHLQWURGXFWLRQWRDVSHFLDOLVVXHRIWKH,QWHUQDWLRQDO
-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\µORQJLWXGLQDOTXDOLWDWLYHUHVHDUFK¶LVD
GLVWLQFWQHZPHWKRGRORJLFDOVWUDQGEHFDXVHLWLQFOXGHVDIHDWXUHGWHPSRUDOFRPSRQHQW
LQWKHUHVHDUFKGHVLJQWRPDNHFKDQJHDFHQWUDOIRFXVRIDQDO\WLFDWWHQWLRQ7KXVWKH
DXWKRUVFRQFOXGHVXFKVWXGLHVDUHORQJLWXGLQDOE\ERWKGHVLJQDQGGHIDXOW

,QSULQFLSOHGHVLJQLQJORQJLWXGLQDOTXDOLWDWLYHVWXGLHVLVUHODWLYHO\HDV\$OOZH
QHHGWRGRLVWDNHRQHRIWKHTXDQWLWDWLYHORQJLWXGLQDOGHVLJQVGHVFULEHGHDUOLHUDQG
UHSODFHWKH48$1GDWDFROOHFWLRQFRPSRQHQWZLWKD48$/PHWKRG IRUH[DPSOHWKH
TXHVWLRQQDLUHVXUYH\ZLWKDTXDOLWDWLYHLQWHUYLHZ 2IFRXUVHFURVVVHFWLRQDO
UHSUHVHQWDWLYHQHVVLVXQOLNHO\WREHDFKLHYHGEHFDXVHRIWKHLQHYLWDEO\VPDOOHUVDPSOH
VL]HVWKDWFKDUDFWHUL]HTXDOLWDWLYHUHVHDUFK VHH6HFWLRQ EXWTXDOLWDWLYHSDQHORU
TXDOLWDWLYHUHWURVSHFWLYHVWXGLHVDUHVWUDLJKWIRUZDUGWRGHYLVH6XFKLQYHVWLJDWLRQVRIIHU
JUHDWSRWHQWLDOE\SODFLQJDSDUWLFLSDQW¶VDFFRXQWVDQGLQWHUSUHWDWLRQVLQDWHPSRUDO
FRQWH[WDQGUHDGLQJWKHUHVSRQVHVREWDLQHGDWGLIIHUHQWSKDVHVRIWKHLQYHVWLJDWLRQ
DJDLQVWHDFKRWKHUDFRPSOH[DQGYDULHGSLFWXUHFDQEHEXLOWXSWKDWFXWVDFURVVPRRGV
DQGOLIHVWDJHV 0F/HRG 7KHLQGLYLGXDO¶VVWRU\FDQEHXQGHUVWRRGLQWHUPVRI
LQGLYLGXDOL]HGQRWLRQVRIWXUQLQJSRLQWVRUGHILQLQJPRPHQWVOLNHDPRYLHZLWKDQ
LQWULFDWHSORWWKDWLVDWUXHUHIOHFWLRQRIWKHWZLVWVDQGWXUQVRIWKHLQGLYLGXDOVWRU\OLQH
1HDOHDQG)ORZHUGHZ
,IWKLVLVVRZK\KDYHWKHUHEHHQVRIHZH[SOLFLWO\ORQJLWXGLQDOTXDOLWDWLYH
LQYHVWLJDWLRQVWRGDWH" )RUH[FHSWLRQVWKRXJKVHHORQJLWXGLQDOFDVHVWXGLHVLQ6HFWLRQ
%HVLGHVWKHIDFWRUVWKDWZRUNJHQHUDOO\DJDLQVWERWKTXDOLWDWLYHVWXGLHVDQG
ORQJLWXGLQDOUHVHDUFKWKHUHDUHWZRVSHFLILFGHWHUUHQWV D WKHFRPSOH[LW\RITXDOLWDWLYH
GDWDDQDO\VLVDQG E WKHLQFUHDVHGLPSDFWRISDQHOFRQGLWLRQLQJ
7KHFRPSOH[LW\RITXDOLWDWLYHGDWDDQDO\VLV
$VGHVFULEHGLQ&KDSWHUZHOOFRQGXFWHGTXDOLWDWLYHGDWDDQDO\VLVFDQEHKLJKO\
FRPSOH[HYHQZLWKRXWDGGLQJDWHPSRUDOGLPHQVLRQWRLW,QDGHWDLOHGDQGHQOLJKWHQLQJ
GHVFULSWLRQRIWKHDQDO\WLFDOSURFHGXUHVRIDODUJHVFDOHORQJLWXGLQDOVWXG\7KRPVRQ
DQG+ROODQG DGPLWWKDWDGGLQJWRWKHV\QFKURQLFGLPHQVLRQWKHWHPSRUDO
HOHPHQWPDGHWKHGDWDPDQDJHPHQWDQGDQDO\VLVKLJKO\ODERXULQWHQVLYHWRWKHSRLQW
WKDWLWVHHPHGDWWLPHVµLQWLPLGDWLQJ¶,QRUGHUWRGRMXVWLFHWRERWKWKHEUHDGWKDQGWKH
GHSWKRIWKHGDWDLQGLYLGXDOQDUUDWLYHDQDO\VHVZHUHXQGHUWDNHQE\WKHLQWHUYLHZHUV
VKRUWO\DIWHUHDFKLQWHUYLHZDQGDWWKHHQGRIHDFKURXQGRILQWHUYLHZLQJDµVXPPDU\
QDUUDWLYHDQDO\VLV¶ZDVZULWWHQ,QDGGLWLRQDIWHUWKHWKLUGZDYHRILQWHUYLHZVWKH
UHVHDUFKHUVGUHZWRJHWKHUWKHWKUHHQDUUDWLYHDQDO\VHVIRUHDFKSDUWLFLSDQWWRSURGXFHD
µFDVHSURILOH¶WUDFLQJFKDQJHVDQGFRQWLQXLWLHVLQWKHLUQDUUDWLYHVRYHUWLPHWKHUHE\
DGGLQJDSURPLQHQWGLDFKURQLFDQJOH7KHILQDOLQWHUSUHWDWLRQVZHUHGUDZQIURPWKH
V\QWKHVLVRIWKHVXPPDU\QDUUDWLYHDQDO\VHVDQGWKHFDVHSURILOHV$VWKHDXWKRUV
DFNQRZOHGJHNHHSLQJXSWKLVZULWLQJVFKHGXOHZDVDVWUXJJOH
$FFRUGLQJWR7KRPVRQDQG+ROODQG RQJRLQJORQJLWXGLQDOLQYHVWLJDWLRQVDOVR
UDLVHWKHTXHVWLRQRIZKHQH[DFWO\LWLVDSSURSULDWHWRVWDUWPDNLQJLQWHUSUHWDWLRQVDQG
ZULWLQJXSWKHUHVXOWV$IWHUDOOWKHUHLVQRUHDOFORVXUHRIDQDO\VLVDQGWKHQH[WURXQG
RIGDWDFDQVXEVWDQWLDOO\PRGLI\WKHLQWHUSUHWDWLRQV7KLVRSHQHQGHGQDWXUHRIWKH
UHVHDUFKFKDOOHQJHVWKHDXWKRULW\DQGVWDELOLW\RIWKHLQWHUSUHWDWLRQVDQGWKHDXWKRUV
UHSRUWHGWKDWE\H[DPLQLQJWKHUHVHDUFKHUV¶FRQWHPSRUDQHRXVREVHUYDWLRQVWKH\GLG

LQGHHGREVHUYHVKLIWVLQWKHLUSURYLVLRQDOLQWHUSUHWDWLRQV$V7KRPVRQDQG+ROODQG
FRQFOXGHGWKLVDEVHQFHRIDQDO\WLFFORVXUHWXUQHGRXWWREHWKHPRVWFKDOOHQJLQJDVSHFW
RIWKHLUDQDO\VLVEHFDXVHWKHQHZURXQGVRIGDWDDOZD\VWKUHDWHQHGWRUHQGHUWKH
SUHYLRXVLQWHUSUHWDWLRQVUHGXQGDQW
,QFUHDVHGSDQHOFRQGLWLRQLQJ
7KHSUREOHPDWLFHIIHFWVRISDQHOFRQGLWLRQLQJKDYHDOUHDG\EHHQPHQWLRQHGZLWKUHJDUG
WRTXDQWLWDWLYHSDQHOVWXGLHV+RZHYHUWKHJUDYLW\RIWKLVLVVXHLVPXFKELJJHULQ
TXDOLWDWLYHVWXGLHVEHFDXVHRIWKHPRUHLQWLPDWHUHODWLRQVKLSEHWZHHQUHVHDUFKHUDQG
UHVSRQGHQW$ORQJLWXGLQDOTXDOLWDWLYHVWXG\LQYROYHVµMRXUQH\LQJWRJHWKHU¶DQG
7KRPVRQDQG+ROODQG SRLQWRXWWKDWLQWKHLUSURMHFWWKH\EHFDPHLQFUHDVLQJO\
DZDUHRIWKHLPSDFWRIWKLVUHODWLRQVKLSQRWRQO\RQWKHUHVSRQGHQWEXWDOVRRQWKH
UHVHDUFKHU7KHQRUPDWLYHHIIHFWVRIWKHUHSHDWLQWHUYLHZSURFHGXUHLVWKXVDYHU\
VHULRXVWKUHDWWRWKHYDOLGLW\RIWKHUHVHDUFK
/RQJLWXGLQDOPL[HGGHVLJQV
:HVDZLQ6HFWLRQWKDWLQRUGHUWRLQFUHDVHWKHVWUHQJWKVDQGUHGXFHWKH
ZHDNQHVVHVRIFHUWDLQORQJLWXGLQDOLQTXLU\W\SHVYDULRXVFUHDWLYHORQJLWXGLQDOGHVLJQV
FRPELQHDVSHFWVRIFURVVVHFWLRQDODQGSDQHOVWXGLHVWKXVLQYROYLQJDQH[WHQVLYHDQG
DQLQWHQVLYHUHVHDUFKFRPSRQHQW7KHH[WHQVLYHHOHPHQWLVW\SLFDOO\TXDQWLWDWLYHDQG
WKHLQWHQVLYHFRPSRQHQWOHQGVLWVHOIWRTXDOLWDWLYHUHVHDUFKDFFRUGLQJO\1HDOHDQG
)ORZHUGHZ DUJXHWKDWWKH48$/DQG48$1WUDGLWLRQVRIORQJLWXGLQDOUHVHDUFK
DUHFRPSOHPHQWDU\:KLOH,EHOLHYHWKDWWKLVLVWUXHLQWKHRU\FXUUHQWO\WKHUHLVD
PDUNHGLPEDODQFHLQWKHVRFLDOVFLHQFHVEHWZHHQWKHUHODWLYHZHLJKWJLYHQWRWKHVHWZR
SDUDGLJPVZLWKTXDQWLWDWLYHVWXGLHVEHLQJGRPLQDQW+RZHYHUEHFDXVHORQJLWXGLQDO
VWXGLHVDUHLQKHUHQWO\FRQFHUQHGERWKZLWKWKHPLFURDQGPDFUROHYHOVRIGHYHORSPHQW
DQGFKDQJH IRUH[DPSOHLQGLYLGXDOJURZWKDQGFRPPXQLW\FKDQJH WKHPL[LQJRI
DSSURDFKHVLVWKHRUHWLFDOO\ZDUUDQWHG,QDGGLWLRQWKHVKHHUFRPSOH[LW\RIYDULRXV
G\QDPLFFKDQJHSDWWHUQVDOVRVXJJHVWVWKDWDFRPELQDWLRQRITXDOLWDWLYHDQG
TXDQWLWDWLYHPHWKRGVPLJKWEHDSSURSULDWHWRGRDORQJLWXGLQDODQDO\VLVIXOOMXVWLFH)RU
WKHVHUHDVRQVLQWKHLURYHUYLHZRIORQJLWXGLQDO6/$UHVHDUFK2UWHJDDQG,EHUUL6KHD
HQFRXUDJHGORQJLWXGLQDOUHVHDUFKWKDWFDSLWDOL]HVRQWKHVWUHQJWKVRIPL[HG
PHWKRGVGHVLJQV
&KRRVLQJDGHVLJQORQJLWXGLQDORUFURVVVHFWLRQDO"
:KLFKW\SHRIDQDO\VLVVKRXOGZHSUHIHUORQJLWXGLQDORUFURVVVHFWLRQDO"*LYHQWKH
SUHSRQGHUDQFHRIFURVVVHFWLRQDOUHVHDUFKLQDSSOLHGOLQJXLVWLFV DQGLQWKHVRFLDO
VFLHQFHVLQJHQHUDO WKHIDFWLVWKDWPRVWUHVHDUFKHUVWHQGWRGHFLGHDJDLQVWD
ORQJLWXGLQDODSSURDFK)URPDSUDFWLFDODQGHFRQRPLFSRLQWRIYLHZLWLVHDV\WRVHH
ZK\WKLVKDSSHQVORQJLWXGLQDOVWXGLHVWDNHDORQJWLPHWRSURGXFHDQ\PHDQLQJIXO
UHVXOWVDQGWKHUHIRUHWKH\PD\QRWEHDSSURSULDWHIRUUHVHDUFKVWXGHQWVRUDFDGHPLFVRQ

IL[HGFRQWUDFWV(YHQVFKRODUVZKRVHSRVLWLRQVDUHVWDEOHDUHXVXDOO\XQGHUSUHVVXUHWR
SURGXFHVKRUWWHUPUHVXOWVLQWKHµSXEOLVKRUSHULVK¶VSLULWRIRXUWLPH,WGRHVQRWKHOS
HLWKHUWKDWORQJLWXGLQDOSURMHFWVZRXOGQHHGORQJWHUPIXQGLQJWRNHHSWKHUHVHDUFK
WHDPVLQSODFHZKHUHDVIHZUHVHDUFKJUDQWVZRUOGZLGHFRYHUSHULRGVORQJHUWKDWRQHRU
WZR\HDUV7KXVJLYHQWKHPXOWLSOHSUHVVXUHVDQGFRPPLWPHQWVPRVWDFDGHPLFVDUH
W\SLFDOO\XQGHUDVZHOODVWKHODFNRIDQ\VDOLHQWLQFHQWLYHVDQGVXSSRUWVWUXFWXUHV
QRERG\FDQUHDOO\EHEODPHGIRUWDNLQJWKHµHDVLHU¶ZD\RXWDQGFRQGXFWLQJDFURVV
VHFWLRQDOVWXG\$IWHUDOORQHFDQDOVRSURGXFHYDOXDEOHUHVXOWVLQV\QFKURQLF
LQYHVWLJDWLRQVDQGE\GRLQJVRRQHFDQSRVVLEO\KDYHDUHVHDUFKUHSRUWFRPSOHWHGDQG
DFFHSWHGIRUSXEOLFDWLRQZLWKLQD\HDU
%HVLGHVWKHLVVXHRIWKHGXUDWLRQRIWKHSURMHFWFURVVVHFWLRQDOUHVHDUFKRIIHUVVRPH
IXUWKHUDGYDQWDJHV,WLVXVXDOO\HDVLHUWRUHFUXLWUHVSRQGHQWVRQDRQHRIIEDVLVDQG
VXFKGHVLJQVGRQRWUHTXLUHWKHFRQVLGHUDEOHFRVWDQGHQHUJ\WRNHHSLQWRXFKZLWKWKH
SDUWLFLSDQWVRYHUDVXVWDLQHGSHULRG:HGRQRWKDYHWRZRUU\DERXWSDQHOFRQGLWLRQLQJ
RUDWWULWLRQHIIHFWVHLWKHUDQGWKHDQDO\VLVRIFURVVVHFWLRQDOGDWDLVXVXDOO\OHVV
GHPDQGLQJWKDQORQJLWXGLQDOGDWDDQDO\VLV)LQDOO\LQDFURVVVHFWLRQDOVWXG\ZHDUH
OHVVH[SRVHGWRWKHGHWULPHQWDOLPSDFWRIXQIRUHVHHQH[WHUQDOHYHQWVWKDWDUHEH\RQG
RXUFRQWURO7KXVDV5XVSLQL VXPPDUL]HVORQJLWXGLQDOUHVHDUFKFDQEH
ULJKWIXOO\VHHQDVDULVN\µOX[XU\¶WKDWPDQ\RIXVFDQQRWDIIRUG
0\JHQHUDOFRQFHUQZLWKWKHDERYHDUJXPHQWVLVWKDWHYHQWKRXJKUXQQLQJDVHYHUDO
\HDUORQJSDQHOVWXG\ZLWKDVXEVWDQWLDOVDPSOHPD\LQGHHGEHEH\RQGWKHPHDQVRI
PRVWUHVHDUFKHUVWKHUHDUHVHYHUDOPHDQLQJIXOORQJLWXGLQDOHOHPHQWVWKDWFDQEHDGGHG
WRDUHVHDUFKGHVLJQWKDWGRQRWUHTXLUHKXJHLQYHVWPHQWVRIWLPHRUHQHUJ\)RU
H[DPSOHUHWURVSHFWLYHDQGVLPXOWDQHRXVORQJLWXGLQDOVWXGLHVUHTXLUHRQO\DVLQJOHZDYH
RIGDWDFROOHFWLRQWUHQGVWXGLHVFDQSURGXFHPHDQLQJIXOUHVXOWVDOUHDG\DIWHUWKHILUVW
SKDVH EHFDXVHWKHQWKH\FDQEHVHHQDVDQµRUGLQDU\¶FURVVVHFWLRQDOVXUYH\ DQGHYHQ
SDQHOVWXGLHVGRQRWQHFHVVDULO\KDYHWRWDNHDJHV²LQDSSOLHGOLQJXLVWLFVLWPLJKWEH
KLJKO\LQWHUHVWLQJWRDQDO\VHWKHPLFURSURFHVVHVWKDWRFFXUGXULQJVD\DZHHN
ODQJXDJHFRXUVH,QVKRUW,KDYHWKHIHHOLQJWKDWZKHQLWFRPHVWRORQJLWXGLQDO
UHVHDUFKZKHUHWKHUHLVDZLOOWKHUHLVDZD\6RWKHUHDOTXHVWLRQLVWKLVZKHQLVLW
PRUHDSSURSULDWHWRXVHDORQJLWXGLQDOWKDQDFURVVVHFWLRQDOGHVLJQ"
,QKLVLQIOXHQWLDOERRNRQORQJLWXGLQDOUHVHDUFK0HQDUG DUJXHVWKDW
ORQJLWXGLQDOUHVHDUFKVKRXOGEHVHHQDVWKHGHIDXOWZKHQZHLQWHQGWRH[DPLQHDQ\
G\QDPLFSURFHVVHVLQWKHVRFLDOVFLHQFHV6XFKG\QDPLFSURFHVVHVDUHREYLRXVO\
LQYROYHGLQKXPDQOHDUQLQJJURZWKRUVRFLDOFKDQJHEXWWKH\FDQDOVREHDVVRFLDWHG
ZLWKYDULRXVLQWHUDFWLRQVRIGLIIHUHQWOHYHOVRIDQLVVXH IRUH[DPSOHPLFURRUPDFUR
RURIGLIIHUHQWW\SHVRIYDULDEOHV IRUH[DPSOHOHDUQHUWUDLWVDQGOHDUQLQJWDVN
FKDUDFWHULVWLFV 7KHWUXWKLVWKDWWKHUHDUHIHZ LIDQ\ DUHDVLQDSSOLHGOLQJXLVWLFVWKDW
GRQRWFRQWDLQVRPHG\QDPLFHOHPHQWV7KLVZRXOGVXJJHVWWKDWLQPDQ\FDVHVD
ORQJLWXGLQDOGHVLJQZRXOGEHPRUHVXLWDEOHIRUDQLQYHVWLJDWLRQWKDQDFURVVVHFWLRQDO

RQHDQGZLWKVRPHUHVHDUFKWRSLFVDGGLQJDWHPSRUDOGLPHQVLRQWRWKHVWXG\ZRXOGEH
LQGLVSHQVDEOH
&URVVVHFWLRQDOUHVHDUFKUHPDLQVXVHIXOWRGHVFULEHYDULDEOHVDQGSDWWHUQVRI
UHODWLRQVKLSVDVWKH\H[LVWDWDSDUWLFXODUWLPH$OWKRXJKVXFKGHVLJQVDUHXVXDOO\
LQDSSURSULDWHIRUHVWDEOLVKLQJFDXVH±HIIHFWUHODWLRQVKLSVVRSKLVWLFDWHGPXOWLYDULDWH
VWDWLVWLFDOSURFHGXUHV DQGHVSHFLDOO\VWUXFWXUDOHTXDWLRQPRGHOOLQJVHH6HFWLRQ
FDQH[WHQGWKHVFRSHRIWKHSRVVLEOHUHVXOWVREWDLQHGIURPWKHDQDO\VLVRIFURVV
VHFWLRQDOGDWDDQGFDQHYHQWHVWWKHRUHWLFDOPRGHOVWKDWFRQWDLQGLUHFWLRQDOSDWKV
EHWZHHQWKHYDULDEOHV)XUWKHUPRUHFURVVVHFWLRQDOGDWDLVDOVRXVHGWRFRPSDUH
YDULRXVJURXSVRISHRSOH IRUH[DPSOHPHQDQGZRPHQRULQWHUPHGLDWHDQGDGYDQFHG
OHDUQHUV DQGLWFDQHYHQFRPSDUHDJHJURXSVDGHTXDWHO\7KXVLWZRXOGEHDQ
H[DJJHUDWLRQWRFODLPWKDWFURVVVHFWLRQDOUHVHDUFKLVPHUHO\DVHFRQGEHVWDOWHUQDWLYH
WRORQJLWXGLQDOGHVLJQV<HW0HQDUG LVULJKWZKHQKHFRQFOXGHVWKDW
µORQJLWXGLQDOUHVHDUFKFDQLQSULQFLSOHGRPXFKWKDWFURVVVHFWLRQDOUHVHDUFKFDQQRW
EXWWKDWWKHUHLVOLWWOHRUQRWKLQJWKDWFURVVVHFWLRQDOUHVHDUFKFDQLQSULQFLSOHGRWKDW
ORQJLWXGLQDOUHVHDUFKFDQQRW¶
4XDQWLWDWLYHORQJLWXGLQDOGDWDDQDO\VLV
)RUPRVWUHVHDUFKHUVWKHPDLQGHWHUUHQWRITXDQWLWDWLYHORQJLWXGLQDOUHVHDUFKLVOLNHO\WR
EHWKHFRPSOH[LW\RIWKHGDWDDQDO\VLVUHTXLUHG&KDQJHFDQKDSSHQLQYHU\GLIIHUHQW
IRUPVDQGZHQHHGGLIIHUHQWDQDO\WLFDOSURFHGXUHVWRH[DPLQHJUDGXDOOLQHDUFKDQJH
DQGFXUYLOLQHDUFKDQJHOHWDORQHSDWWHUQVWKDWDUHPRUHFRPSOH[WKDQWKHVH6RPH
WKHRUHWLFDOPRGHOVIRUH[DPSOHSRVWXODWHWKDWWKHUHLVDGLVFRQWLQXLW\LQFRQWLQXRXV
FKDQJHLQRWKHUZRUGVWKHUHLVDGLVWLQFWFKDQJHSRLQWDWZKLFKJURZWKDFFHOHUDWHV
VORZVGRZQRUOHYHOVRII²VHH&ROOLQV$QGZKDWDERXWJURZWKSDWWHUQVLQZKLFK
WZR RUPRUH FKDQJLQJYDULDEOHVLQWHUIHUHZLWKHDFKRWKHULQDV\VWHPDWLFZD\ IRU
H[DPSOHXQHPSOR\PHQWDQGLQIODWLRQRUWRJLYHDQDSSOLHGOLQJXLVWLFH[DPSOH/
SURILFLHQF\DQGDWWLWXGHVWRZDUGVLQWHJUDWLRQDFFXOWXUDWLRQ ",QGHHGWKHYDULHW\RI
SDWWHUQVRIORQJLWXGLQDOFKDQJHLVZLGHDQGIRUH[DPSOHLQWKHLUVWXG\RIWLPHVHULHV
GHVLJQVLQ6/$UHVHDUFK0HOORZHWDO RXWOLQHSRVVLEOHGLIIHUHQWOHDUQLQJ
FXUYHVRILQGLYLGXDOVLQUHVSRQVHWRODQJXDJHLQVWUXFWLRQ7KXVLQRUGHUWRGR
TXDQWLWDWLYHORQJLWXGLQDOGDWDMXVWLFHZHQHHGWRPDWFKDWKHRUHWLFDOPRGHORIFKDQJH
ZLWKDFRUUHVSRQGLQJVWDWLVWLFDOPRGHO$V2UWHJDDQG,EHUUL6KHD UHSRUWWKLVLV
DQDUHDZKHUHDSSOLHGOLQJXLVWVKDYHE\DQGODUJHIDOOHQVKRUWRIWKHPDUNEHFDXVHWKH\
KDYHW\SLFDOO\XVHGGHVFULSWLYHVWDWLVWLFVDQGYLVXDOGLVSOD\VWRDQDO\VHORQJLWXGLQDOGDWD
LQVWHDGRIDSSURSULDWHVWDWLVWLFDOPRGHOV
5HJUHWWDEO\FXUUHQWO\WKHUHLVQRJHQHULFVWDWLVWLFDOSURFHGXUHIRUDQDO\VLQJ
ORQJLWXGLQDOFKDQJHDQGLIZHKDYHPRUHWKDQWZRZDYHVRIGDWDFROOHFWLRQZHFDQQRW
DYRLGXVLQJKLJKO\DGYDQFHGVWDWLVWLFDOWHFKQLTXHVVXFKDVUHSHDWHGPHDVXUHVDQDO\VLV
ORJOLQHDUDQDO\VLVRUWLPHVHULHVDQDO\VLVWRQDPHWKHPRVWZHOONQRZQPHWKRGV )RUD
UHFHQWRYHUYLHZVHH&ROOLQV $IXUWKHUGLIILFXOW\LVSRVHGE\WKHIDFWWKDWWKH

PRVWFXWWLQJHGJHVWDWLVWLFDOSURFHGXUHVDUHQRWLQFOXGHGLQWKHSRSXODUVWDWLVWLFDO
SDFNDJHVVXFKDV6366RU6$6EXWKDYHWREHSXUFKDVHGVHSDUDWHO\,QVSLWHRIWKHVH
GLIILFXOWLHVLQWKHLURYHUYLHZRIORQJLWXGLQDOUHVHDUFKLQ6/$2UWHJDDQG,EHUUL6KHD
KLJKOLJKWWKHQHHGIRUDSSOLHGOLQJXLVWVWREHFRPHIDPLOLDUZLWKPRGHUQ
ORQJLWXGLQDODQDO\WLFDOSURFHGXUHV$VWKH\FRQFOXGH
,QWKHIXWXUHPRUH6/$UHVHDUFKHUVPD\VHHNWRPDSWKH/GHYHORSPHQWDOSDWK
RIDFRKRUWWUDQVLWLQJWKURXJKFXUULFXODUDQGLQVWUXFWLRQDOWLPHZLWKNH\WXUQLQJ
SRLQWVGLIIHUHQWSKDVHVRIVWDJQDWLRQDQGGLIIHUHQWO\SDFHGGHYHORSPHQWDQG
QRQOLQHDUSURJUHVV,IWKLVLVWKHFDVHDQGPRUHODUJHVL]HORQJLWXGLQDO
TXDQWLWDWLYHVWXGLHVDUHFRQGXFWHGLQ6/$LWZLOOEHLPSRUWDQWWRWUDLQRXUVHOYHV
LQWKHXVHRIVWDWLVWLFDODQDO\WLFDORSWLRQVWKDWDUHDYDLODEOHVSHFLILFDOO\IRUXVH
ZLWKORQJLWXGLQDOGHVLJQVDQGGDWD S
7KHGHVFULSWLRQRIWKHVHVSHFLDOL]HGVWDWLVWLFDORSWLRQVLVEH\RQGWKHVFRSHRIWKLV
ERRNVRKHUH,RIIHURQO\DJHQHUDORYHUYLHZIRURULHQWDWLRQ0HQDUG SURYLGHVD
URXJKFODVVLILFDWLRQRIPHWKRGVRIORQJLWXGLQDOGDWDDQDO\VLVDORQJWZRGLPHQVLRQVWKH
QXPEHURIFDVHVLQRXUVDPSOHDQGWKHQXPEHURIWLPHSHULRGVWKHVDPSOHKDVEHHQ
DVVHVVHGDW7KLVVLPSOHV\VWHPRIIHUVDJRRGVWDUWLQJSRLQWLQGHFLGLQJZKDWNLQGRI
DQDO\VLVZHQHHG/HWXVH[DPLQHWKHWZRGHVLJQRSWLRQVWKDWDUHPRVWUHOHYDQWWR
DSSOLHGOLQJXLVWVZKRZRXOGOLNHWRODXQFKDORQJLWXGLQDOLQYHVWLJDWLRQ
x 7LPHVHULHVDQDO\VLVH[DPLQHVRQO\DIHZFDVHV RIWHQHYHQDVLQJOHFDVH EXWPDQ\
DVVHVVPHQWSRLQWV XVXDOO\RYHU DQGWKHSULPDU\LQWHUHVWLVLQILQGLQJSDWWHUQV
RIFKDQJHLQLQGLYLGXDOFDVHVDQGJHQHUDOL]LQJWKHILQGLQJVDFURVVWLPHSHULRGV
UDWKHUWKDQDFURVVFDVHV 6HHWKHGHVFULSWLRQRIµLQWHUUXSWHGWLPHVHULHVDQDO\VLV¶LQ
6HFWLRQ 6XFKGHVLJQVDUHSDUWLFXODUO\VXLWHGWRDQDO\VLQJODQJXDJH
GHYHORSPHQW 0HOORZHWDO DQG6366RIIHUVVHYHUDORSWLRQVIRUFUHDWLQJDQG
H[DPLQLQJWLPHVHULHVPRGHOV $1$/<=(ĺ7,0(6(5,(6 LQFOXGLQJ
µ$5,0$¶DQDO\VLV $1$/<=(ĺ7,0(6(5,(6ĺ$5,0$ ZKLFKKDVEHFRPH
LQFUHDVLQJO\SRSXODULQWKHVRFLDOVFLHQFHV 0HQDUG
x 7KHRSSRVLWHRIWLPHVHULHVGHVLJQLVDFRPPRQGHVLJQLQDSSOLHGOLQJXLVWLF
UHVHDUFKWKDWLQYROYHVDODUJHVDPSOH 1! DQGIHZGDWDSHULRGV SUHIHUDEO\
QRWPRUHWKDQWKUHH 'DWDGHULYHGIURPVXFKGHVLJQVFDQEHDQDO\VHGE\PHDQVRI
µUHSHDWHGPHDVXUHVDQDO\VLV¶ZKLFKLVVXSSRUWHGE\6366 $1$/<=(ĺ
*(1(5$//,1($502'(/ĺ5(3($7('0($685(6 7KHXVHRI
µVWUXFWXUDOHTXDWLRQPRGHOOLQJ¶ 6(0VHH RIIHUVLQFUHDVHGIOH[LELOLW\LQ
DQDO\VLQJWKHUHVXOWVRIUHSHDWHGPHDVXUHVGHVLJQV RULQ6(0WHUPVµODWHQWJURZWK
FXUYHGPRGHOV¶ DVLWFDQKDQGOHPRUHWKDQ XSWRDERXW WLPHSHULRGV
$OWKRXJKHYHQWKHEDVLFORQJLWXGLQDOSURFHGXUHVGHVFULEHGDERYHPD\VHHPWRR
WHFKQLFDOIRUPDQ\UHVHDUFKHUVORQJLWXGLQDOGDWDDQDO\VLVLVQRWDOOEDGQHZV7DULV
SRLQWVRXWWKDWRQHRIWKHJUHDWYLUWXHVRIUHSHDWHGPHDVXUHVGDWDLVWKDWLWFDQ
FRQYHQLHQWO\EHGLVSOD\HGLQHDV\WRLQWHUSUHWSORWVRQZKLFKWLPHUHSUHVHQWHGRQWKH

KRUL]RQWDOD[LVDQGWKHWDUJHWYDULDEOHRQWKHYHUWLFDOD[LV6XFKSORWVRUOLQHGLDJUDPV
SURYLGHDFOHDUYLVXDOUHSUHVHQWDWLRQRIWKHXQGHUO\LQJWUHQGVDQGKHOSWRJHQHUDWH
VSHFLILFK\SRWKHVHVWKDWFDQWKHQEHWHVWHG:LWKVXFKVSHFLILFLGHDVLQPLQGLWPLJKWEH
HDVLHUWRLQYLWHH[SHUWKHOSWRUXQWKHQHFHVVDU\VWDWLVWLFV

3DUW'DWDFROOHFWLRQ
4XDQWLWDWLYHGDWDFROOHFWLRQ
4XDQWLWDWLYHGDWDFDQEHREWDLQHGLQDQXPEHURIZD\V7KHPRVWFRPPRQLQVWUXPHQW
XVHGIRUWKLVSXUSRVHLVWKHWHVWZKLFKKDVVHYHUDOW\SHV IRUH[DPSOHODQJXDJHWHVWVRU
SV\FKRORJLFDOWHVWVVXFKDVDSWLWXGHWHVWVRUSHUVRQDOLW\EDWWHULHV $VPHQWLRQHGLQ
&KDSWHUODQJXDJHWHVWLQJLVDFRPSOH[DQGKLJKO\VSHFLDOL]HGLVVXHZLWKDQH[WHQVLYH
OLWHUDWXUHDQGWKHUHIRUHLWZRXOGJREH\RQGWKHVFRSHRIWKLVERRNWRFRYHULW$VHFRQG
IRUPRISURGXFLQJTXDQWLWDWLYHGDWDLVWRPHDVXUHVRPHSKHQRPHQRQREMHFWLYHO\E\
FRQWUROOHGPHDQV IRUH[DPSOHDVVHVVLQJUHVSRQVHWLPHRUEHKDYLRXUDOIUHTXHQF\ DQG
ZHFDQDOVRTXDQWLI\GDWDWKDWZDVRULJLQDOO\FROOHFWHGLQDQRQTXDQWLWDWLYHZD\ 7KLV
ZLOOEHGLVFXVVHGLQ&KDSWHUZKHQZHORRNDWPL[HGPHWKRGVGDWDDQDO\VLV
)LQDOO\DIUHTXHQWPHWKRGRIFROOHFWLQJTXDQWLWDWLYHGDWDLVWKURXJKFRQGXFWLQJD
VXUYH\XVLQJVRPHVRUWRIDTXHVWLRQQDLUH,QGHHGEHVLGHVODQJXDJHWHVWV
TXHVWLRQQDLUHVDUHWKHPRVWFRPPRQGDWDFROOHFWLRQLQVWUXPHQWVLQDSSOLHGOLQJXLVWLFV
DQGWKHUHIRUHWKHEXONRIWKLVFKDSWHUZLOOEHGHYRWHGWRWKHLUGHVFULSWLRQ
,QRUGHUWRDFKLHYHDEURDGXQGHUVWDQGLQJRITXDQWLWDWLYHPHWKRGVZHQHHGWRFRYHU
WKUHHIXUWKHUWRSLFVEHVLGHVWKHTXHVWLRQQDLUHVXUYH\7KHILUVWRQHLVWKHGLIIHUHQWZD\V
RIVDPSOLQJSDUWLFLSDQWVWRHQVXUHWKDWWKHUHVXOWVDUHJHQHUDOL]DEOH7KHVHFRQGWRSLF
FRQFHUQVH[SHULPHQWDODQGTXDVLH[SHULPHQWDOVWXGLHVZKLFKUHSUHVHQWDVSHFLDO
UHVHDUFKGHVLJQRIGDWDFROOHFWLRQDQGSDUWLFLSDQWPDQLSXODWLRQWKDWKDVEHHQ
GHYHORSHG²IROORZLQJWKHPRGHORIH[SHULPHQWVLQWKHQDWXUDOVFLHQFHV²WRSURGXFHD
PHWKRGRIHVWDEOLVKLQJFDXVH±HIIHFWUHODWLRQVKLSV)LQDOO\WKHFKDSWHUZLOOFRQFOXGH
ZLWKDVHFWLRQDGGUHVVLQJWKHQHZDQGLQFUHDVLQJO\SRSXODUSKHQRPHQRQRIFROOHFWLQJ
GDWDYLDWKH,QWHUQHW
6DPSOLQJLQTXDQWLWDWLYHUHVHDUFK
7KHPRVWIUHTXHQWTXHVWLRQDVNHGE\QRYLFHUHVHDUFKHUVEHIRUHVWDUWLQJDTXDQWLWDWLYH
LQYHVWLJDWLRQLVµ+RZPDQ\SHRSOHGR,QHHGWRLQFOXGHLQP\VWXG\"¶,QPHDVXUHPHQW
WHUPVWKLVTXHVWLRQFDQEHUHSKUDVHGDVµ+RZODUJHVKRXOGP\VDPSOHEH"¶$QGWKH
VHFRQGTXHVWLRQWRIROORZLVXVXDOO\µ:KDWVRUWRISHRSOHVKDOO,VHOHFW"¶²LQRWKHU
ZRUGVµ:KRVKDOOP\VDPSOHFRQVLVWRI"¶7KHVHTXHVWLRQVUHIOHFWWKHUHFRJQLWLRQWKDW
LQHPSLULFDOUHVHDUFKLVVXHVFRQFHUQLQJWKHSDUWLFLSDQWVDPSOHFDQIXQGDPHQWDOO\
GHWHUPLQHWKHVXFFHVVRIDVWXG\)XUWKHUPRUHLWLVDOVRFOHDUWKDWLQTXDQWLWDWLYH
UHVHDUFKVDPSOLQJGHFLVLRQVPXVWEHWDNHQHDUO\LQWKHRYHUDOOSODQQLQJSURFHVV
EHFDXVHWKH\ZLOOFRQVLGHUDEO\DIIHFWWKHQHFHVVDU\LQLWLDODUUDQJHPHQWVWKHWLPLQJDQG

VFKHGXOLQJRIWKHSURMHFWDVZHOODVWKHYDULRXVFRVWVLQYROYHG/HWXVVWDUWRXU
GLVFXVVLRQE\GHILQLQJWKHWKUHHFRQFHSWVWKDWOLHDWWKHKHDUWRITXDQWLWDWLYHVDPSOLQJ
VDPSOHSRSXODWLRQDQGWKHQRWLRQRIUHSUHVHQWDWLYHQHVV
6DPSOHSRSXODWLRQDQGUHSUHVHQWDWLYHQHVV
7KHVDPSOHLVWKHJURXSRISDUWLFLSDQWVZKRPWKHUHVHDUFKHUDFWXDOO\H[DPLQHVLQDQ
HPSLULFDOLQYHVWLJDWLRQDQGWKHSRSXODWLRQLVWKHJURXSRISHRSOHZKRPWKHVWXG\LV
DERXW)RUH[DPSOHWKHSRSXODWLRQLQDVWXG\PLJKWEH()/OHDUQHUVLQ7DLZDQHVH
VHFRQGDU\VFKRROVDQGWKHDFWXDOVDPSOHPLJKWLQYROYHWKUHH7DLZDQHVHVHFRQGDU\
FODVVHV7KDWLVWKHWDUJHWSRSXODWLRQRIDVWXG\FRQVLVWVRIDOOWKHSHRSOHWRZKRPWKH
VXUYH\¶VILQGLQJVDUHWREHDSSOLHGRUJHQHUDOL]HG
$JRRGVDPSOHLVYHU\VLPLODUWRWKHWDUJHWSRSXODWLRQLQLWVPRVWLPSRUWDQWJHQHUDO
FKDUDFWHULVWLFV IRUH[DPSOHDJHJHQGHUHWKQLFLW\HGXFDWLRQDOEDFNJURXQGDFDGHPLF
FDSDELOLW\VRFLDOFODVVRUVRFLRHFRQRPLFVWDWXV DVZHOODVDOOWKHPRUHVSHFLILF
IHDWXUHVWKDWDUHNQRZQWREHUHODWHGWRWKHYDULDEOHVWKDWWKHVWXG\IRFXVHVRQ IRU
H[DPSOH/OHDUQLQJEDFNJURXQGRUWKHDPRXQWDQGW\SHRI/LQVWUXFWLRQUHFHLYHG
7KDWLVWKHVDPSOHLVDVXEVHWRIWKHSRSXODWLRQWKHVDPSOHEHLQJUHSUHVHQWDWLYHRIWKH
ZKROHSRSXODWLRQ7KHLVVXHRIUHSUHVHQWDWLYHQHVVLVFUXFLDOEHFDXVHDV0LOUR\DQG
*RUGRQ DORQJZLWKPDQ\RWKHUVFKRODUVSRLQWRXWWKHVWUHQJWKRIWKH
FRQFOXVLRQVZHFDQGUDZIURPWKHUHVXOWVREWDLQHGIURPDVHOHFWHGVPDOOJURXSGHSHQGV
RQKRZDFFXUDWHO\WKHSDUWLFXODUVDPSOHUHSUHVHQWVWKHODUJHUSRSXODWLRQ
:K\GRQ¶WZHLQFOXGHHYHU\PHPEHURIWKHSRSXODWLRQLQDVWXG\"7KLVLVDYDOLG
TXHVWLRQDQGLQGHHGWKHUHLVRQHSDUWLFXODUVXUYH\W\SHZKLFKGRHVMXVWWKDWWKH
µFHQVXV¶,QPRVWRWKHUFDVHVKRZHYHULQYHVWLJDWLQJWKHZKROHSRSXODWLRQLVQRW
QHFHVVDU\DQGZRXOGLQIDFWEHDZDVWHRIUHVRXUFHV%\DGRSWLQJDSSURSULDWHVDPSOLQJ
SURFHGXUHVWRVHOHFWDVPDOOHUQXPEHURISHRSOHWREHLQYHVWLJDWHGZHFDQVDYHD
FRQVLGHUDEOHDPRXQWRIWLPHFRVWDQGHIIRUWDQGFDQVWLOOFRPHXSZLWKDFFXUDWH
UHVXOWV²RSLQLRQSROOVIRUH[DPSOHVXFFHHGLQSURYLGLQJQDWLRQDOSURMHFWLRQVEDVHGRQ
DVIHZDV±UHVSRQGHQWV7KHNH\TXHVWLRQWKHQLVZKDWH[DFWO\ZHPHDQE\
µDSSURSULDWHVDPSOLQJSURFHGXUHV¶"
6DPSOLQJSURFHGXUHV
%URDGO\VSHDNLQJVDPSOLQJVWUDWHJLHVFDQEHGLYLGHGLQWRWZRJURXSV D VFLHQWLILFDOO\
VRXQGµSUREDELOLW\VDPSOLQJ¶ZKLFKLQYROYHVFRPSOH[DQGH[SHQVLYHSURFHGXUHVWKDW
DUHXVXDOO\ZHOOEH\RQGWKHPHDQVRIDSSOLHGOLQJXLVWVDQG E µQRQSUREDELOLW\
VDPSOLQJ¶ZKLFKFRQVLVWVRIDQXPEHURIVWUDWHJLHVWKDWWU\WRDFKLHYHDWUDGHRIIWKDW
LVDUHDVRQDEO\UHSUHVHQWDWLYHVDPSOHXVLQJUHVRXUFHVWKDWDUHZLWKLQWKHPHDQVRIWKH
RUGLQDU\UHVHDUFKHU
3UREDELOLW\VDPSOLQJ

3UREDELOLW\VDPSOLQJLVDJHQHULFWHUPXVHGIRUDQXPEHURIVFLHQWLILFSURFHGXUHVWKH
PRVWLPSRUWDQWRIZKLFKDUHWKHIROORZLQJ
x 5DQGRPVDPSOLQJ7KHNH\FRPSRQHQWRISUREDELOLW\VDPSOLQJLVµUDQGRP
VDPSOLQJ¶7KLVLQYROYHVVHOHFWLQJPHPEHUVRIWKHSRSXODWLRQWREHLQFOXGHGLQWKH
VDPSOHRQDFRPSOHWHO\UDQGRPEDVLVDELWOLNHGUDZLQJQXPEHUVIURPDKDW IRU
H[DPSOHE\QXPEHULQJHDFKPHPEHUDQGWKHQDVNLQJWKHFRPSXWHUWRJHQHUDWH
UDQGRPQXPEHUV 7KHDVVXPSWLRQXQGHUO\LQJWKLVSURFHGXUHLVWKDWWKHVHOHFWLRQLV
EDVHGHQWLUHO\RQSUREDELOLW\DQGFKDQFHWKXVPLQLPL]LQJWKHHIIHFWVRIDQ\
H[WUDQHRXVRUVXEMHFWLYHIDFWRUV$VDUHVXOWDVXIILFLHQWO\ODUJHVDPSOHVKRXOG
FRQWDLQVXEMHFWVZLWKFKDUDFWHULVWLFVVLPLODUWRWKHSRSXODWLRQDVDZKROH$OWKRXJK
WKLVLVUDUHO\IXOO\DFKLHYHGWKHUXOHRIWKXPELVWKDWUDQGRPVDPSOHVDUHDOPRVW
DOZD\VPRUHUHSUHVHQWDWLYHWKDQQRQUDQGRPVDPSOHV
x 6WUDWLILHGUDQGRPVDPSOLQJ&RPELQLQJUDQGRPVDPSOLQJZLWKVRPHIRUPRI
UDWLRQDOJURXSLQJLVDSDUWLFXODUO\HIIHFWLYHPHWKRGIRUUHVHDUFKZLWKDVSHFLILF
IRFXV,QµVWUDWLILHGUDQGRPVDPSOLQJ¶WKHSRSXODWLRQLVGLYLGHGLQWRJURXSVRU
µVWUDWD¶DQGDUDQGRPVDPSOHRIDSURSRUWLRQDWHVL]HLVVHOHFWHGIURPHDFKJURXS
7KXVLIZHZDQWWRDSSO\WKLVVWUDWHJ\ILUVWZHQHHGWRLGHQWLI\DQXPEHURI
SDUDPHWHUVRIWKHZLGHUSRSXODWLRQWKDWDUHLPSRUWDQWIURPWKHSRLQWRIYLHZRIWKH
UHVHDUFKLQDµVDPSOLQJIUDPH¶²DQREYLRXVH[DPSOHZRXOGEHDGLYLVLRQRIPDOHV
DQGIHPDOHV²DQGWKHQVHOHFWSDUWLFLSDQWVIRUHDFKFDWHJRU\RQDUDQGRPEDVLV$
VWUDWLILHGUDQGRPVDPSOHLVWKHUHIRUHDFRPELQDWLRQRIUDQGRPL]DWLRQDQG
FDWHJRUL]DWLRQ,QVWXGLHVIROORZLQJWKLVPHWKRGWKHSRSXODWLRQLVXVXDOO\VWUDWLILHG
RQPRUHWKDQRQHYDULDEOHDQGUDQGRPVDPSOHVDUHVHOHFWHGIURPDOOWKHJURXSV
GHILQHGE\WKHLQWHUVHFWLRQVRIWKHYDULRXVVWUDWD IRUH[DPSOHZHZRXOGVDPSOH
IHPDOHOHDUQHUVRI6SDQLVKDJHG±ZKRDWWHQGDSDUWLFXODUW\SHRI
LQVWUXFWLRQDOSURJUDPPHLQDSDUWLFXODUORFDWLRQ
x 6\VWHPDWLFVDPSOLQJ,QDQRQ\PRXVVXUYH\VLWFDQEHGLIILFXOWWRPDNHDUDQGRP
VHOHFWLRQEHFDXVHZHPD\KDYHQRPHDQVRILGHQWLI\LQJWKHSDUWLFLSDQWVLQDGYDQFH
DQGWKXVWKHLUQDPHVFDQQRWEHµSXWLQWKHKDW¶ &RKHQHWDO $XVHIXO
WHFKQLFDOVKRUWFXWLVLQVXFKFDVHVWRDSSO\µV\VWHPDWLFVDPSOLQJ¶ZKLFKLQYROYHV
VHOHFWLQJHYHU\QWKPHPEHURIWKHWDUJHWJURXS
x &OXVWHUVDPSOLQJ2QHZD\RIPDNLQJUDQGRPVDPSOLQJPRUHSUDFWLFDOHVSHFLDOO\
ZKHQWKHWDUJHWSRSXODWLRQLVZLGHO\GLVSHUVHGLVWRUDQGRPO\VHOHFWVRPHODUJHU
JURXSLQJVRUXQLWVRIWKHSRSXODWLRQV IRUH[DPSOHVFKRROV DQGWKHQH[DPLQHDOO
WKHVWXGHQWVLQWKRVHVHOHFWHGXQLWV
,WLVFOHDUIURPWKHVHEULHIGHVFULSWLRQVWKDWVHOHFWLQJDWUXO\UHSUHVHQWDWLYHVDPSOHLV
DSDLQVWDNLQJDQGFRVWO\SURFHVVDQGVHYHUDOKLJKO\WHFKQLFDOPRQRJUDSKVKDYHEHHQ
ZULWWHQDERXWWKHWRSLF IRUH[DPSOH&RFKUDQ/HY\DQG/HPHVKRZ
+RZHYHULWQHHGVWREHUHLWHUDWHGWKDWLQPRVWDSSOLHGOLQJXLVWLFUHVHDUFKLWLV

XQUHDOLVWLFRUVLPSO\QRWIHDVLEOHWRDLPIRUSHUIHFWUHSUHVHQWDWLYHQHVVLQWKH
SV\FKRPHWULFVHQVH
1RQSUREDELOLW\VDPSOLQJ
0RVWDFWXDOUHVHDUFKLQDSSOLHGOLQJXLVWLFVHPSOR\VµQRQSUREDELOLW\VDPSOHV¶,Q
TXDOLWDWLYHUHVHDUFKVXFKSXUSRVLYHQRQUHSUHVHQWDWLYHVDPSOHVPD\QRWEHVHHQDVD
SUREOHP VHH6HFWLRQ EXWLQTXDQWLWDWLYHUHVHDUFKZKLFKDOZD\VDLPVDW
UHSUHVHQWDWLYHQHVVQRQSUREDELOLW\VDPSOHVDUHUHJDUGHGDVOHVVWKDQSHUIHFW
FRPSURPLVHVWKDWUHDOLW\IRUFHVXSRQWKHUHVHDUFKHU:HFDQGLVWLQJXLVKWKUHHPDLQ
QRQSUREDELOLVWLFVDPSOLQJVWUDWHJLHV
x 4XRWDVDPSOLQJDQGGLPHQVLRQDOVDPSOLQJµ4XRWDVDPSOLQJ¶LVVLPLODUWR
SURSRUWLRQDOVWUDWLILHGUDQGRPVDPSOLQJZLWKRXWWKHµUDQGRP¶HOHPHQW7KDWLVZH
VWDUWRIIZLWKDVDPSOLQJIUDPHDQGWKHQGHWHUPLQHWKHPDLQSURSRUWLRQVRIWKH
VXEJURXSVGHILQHGE\WKHSDUDPHWHUVLQFOXGHGLQWKHIUDPH7KHDFWXDOVDPSOH
WKHQLVVHOHFWHGLQDZD\DVWRUHIOHFWWKHVHSURSRUWLRQVEXWZLWKLQWKHZHLJKWHG
VXEJURXSVQRUDQGRPVDPSOLQJLVXVHGEXWUDWKHUWKHUHVHDUFKHUPHHWVWKHTXRWDV
E\VHOHFWLQJSDUWLFLSDQWVKHVKHFDQKDYHDFFHVVWR)RUH[DPSOHLIWKHVDPSOLQJ
IUDPHLQDVWXG\RIODQJXDJHOHDUQHUVVSHFLILHVWKDWSHUFHQWRIWKH
SDUWLFLSDQWVVKRXOGFRPHIURPELOLQJXDODQGWKHRWKHUSHUFHQWIURPPRQROLQJXDO
IDPLOLHVWKHUHVHDUFKHUQHHGVWRUHFUXLWSDUWLFLSDQWVIURPHDFKJURXSEXWWKH
VHOHFWLRQGRHVQRWKDYHWREHUDQGRPµ'LPHQVLRQDOVDPSOLQJ¶LVDYDULDWLRQRI
TXRWDVDPSOLQJ7KHUHVHDUFKHUPDNHVVXUHWKDWDWOHDVWRQHUHSUHVHQWDWLYHRIHYHU\
FRPELQDWLRQRIWKHYDULRXVSDUDPHWHUVLQWKHVDPSOLQJIUDPHLVLQFOXGHGLQWKH
VDPSOH
x 6QRZEDOOVDPSOLQJ7KLVLQYROYHVDµFKDLQUHDFWLRQ¶ZKHUHE\WKHUHVHDUFKHU
LGHQWLILHVDIHZSHRSOHZKRPHHWWKHFULWHULDRIWKHSDUWLFXODUVWXG\DQGWKHQDVNV
WKHVHSDUWLFLSDQWVWRLGHQWLI\IXUWKHUDSSURSULDWHPHPEHUVRIWKHSRSXODWLRQ7KLV
WHFKQLTXHLVXVHIXOZKHQVWXG\LQJJURXSVZKRVHPHPEHUVKLSLVQRWUHDGLO\
LGHQWLILDEOH IRUH[DPSOHWHHQDJHJDQJPHPEHUV RUZKHQDFFHVVWRVXLWDEOHJURXS
PHPEHUVLVGLIILFXOWIRUVRPHUHDVRQ
x &RQYHQLHQFHRURSSRUWXQLW\VDPSOLQJ7KHPRVWFRPPRQVDPSOHW\SHLQ/
UHVHDUFKLVWKHµFRQYHQLHQFH¶RUµRSSRUWXQLW\VDPSOH¶ZKHUHDQLPSRUWDQWFULWHULRQ
RIVDPSOHVHOHFWLRQLVWKHFRQYHQLHQFHRIWKHUHVHDUFKHUPHPEHUVRIWKHWDUJHW
SRSXODWLRQDUHVHOHFWHGIRUWKHSXUSRVHRIWKHVWXG\LIWKH\PHHWFHUWDLQSUDFWLFDO
FULWHULDVXFKDVJHRJUDSKLFDOSUR[LPLW\DYDLODELOLW\DWDFHUWDLQWLPHHDV\
DFFHVVLELOLW\RUWKHZLOOLQJQHVVWRYROXQWHHU&DSWLYHDXGLHQFHVVXFKDVVWXGHQWVLQ
WKHUHVHDUFKHU¶VRZQLQVWLWXWLRQDUHSULPHH[DPSOHVRIFRQYHQLHQFHVDPSOHV7REH
IDLUFRQYHQLHQFHVDPSOHVDUHUDUHO\FRPSOHWHO\FRQYHQLHQFHEDVHGEXWDUHXVXDOO\
SDUWLDOO\SXUSRVHIXOZKLFKPHDQVWKDWEHVLGHVWKHUHODWLYHHDVHRIDFFHVVLELOLW\
SDUWLFLSDQWVDOVRKDYHWRSRVVHVVFHUWDLQNH\FKDUDFWHULVWLFVWKDWDUHUHODWHGWRWKH

SXUSRVHRIWKHLQYHVWLJDWLRQ7KXVFRQYHQLHQFHVDPSOLQJRIWHQFRQVWLWXWHVDOHVV
FRQWUROOHGYHUVLRQRIWKHTXRWDVDPSOLQJVWUDWHJ\GHVFULEHGDERYH
1RPDWWHUKRZSULQFLSOHGDQRQSUREDELOLW\VDPSOHVWULYHVWREHWKHH[WHQWRI
JHQHUDOL]DELOLW\LQWKLVW\SHRIVDPSOHLVRIWHQQHJOLJLEOH,WLVWKHUHIRUHVXUSULVLQJWKDW
WKHPDMRULW\RIHPSLULFDOUHVHDUFKLQWKHVRFLDOVFLHQFHVLVQRWEDVHGRQUDQGRP
VDPSOHV+RZHYHU.HPSHUHWDO¶V ± FRQFOXVLRQLVYHU\WUXH
6DPSOLQJLVVXHVDUHLQKHUHQWO\SUDFWLFDO6FKRODUO\GHFLVLRQVPD\EHGULYHQLQ
SDUWE\WKHRUHWLFDOFRQFHUQVEXWLWLVLQVDPSOLQJSHUKDSVPRUHWKDQDQ\ZKHUH
HOVHLQUHVHDUFKWKDWWKHRU\PHHWVWKHKDUGUHDOLWLHVRIWLPHDQGUHVRXUFHV«
6DPSOLQJLVVXHVDOPRVWLQYDULDEO\IRUFHSUDJPDWLFFKRLFHV
,QDQ\FDVHEHFDXVHRIWKHFRPSURPLVHGQDWXUHRIQRQSUREDELOLW\VDPSOLQJZH
QHHGWRGHVFULEHLQVXIILFLHQWGHWDLOWKHOLPLWDWLRQVRIVXFKVDPSOHVZKHQZHUHSRUWWKH
UHVXOWVZKLOHDOVRKLJKOLJKWLQJWKHFKDUDFWHULVWLFVWKDWWKHSDUWLFXODUVDPSOHVKDUHVZLWK
WKHGHILQHGWDUJHWSRSXODWLRQ,QDVLPLODUYHLQZHDOVRKDYHWREHSDUWLFXODUO\FDUHIXO
DERXWWKHFODLPVZHPDNHDERXWWKHPRUHJHQHUDOUHOHYDQFHRIRXUILQGLQJV
+RZODUJHVKRXOGWKHVDPSOHEH"
:KHQUHVHDUFKHUVDVNWKHTXHVWLRQµ+RZODUJHVKRXOGWKHVDPSOHEH"¶ZKDWWKH\UHDOO\
PHDQLVµ+RZVPDOODVDPSOHFDQ,JHWDZD\ZLWK"¶7KHUHIRUHWKHRIWHQTXRWHGµWKH
ODUJHUWKHEHWWHU¶SULQFLSOHLVVLQJXODUO\XQKHOSIXOIRUWKHP8QIRUWXQDWHO\WKHUHDUHQR
KDUGDQGIDVWUXOHVLQVHWWLQJWKHRSWLPDOVDPSOHVL]HWKHILQDODQVZHUWRWKHµKRZ
ODUJHVPDOO"¶TXHVWLRQVKRXOGEHWKHRXWFRPHRIWKHUHVHDUFKHUFRQVLGHULQJVHYHUDO
EURDGJXLGHOLQHV
x 5XOHVRIWKXPE,QWKHVXUYH\UHVHDUFKOLWHUDWXUHDUDQJHRIEHWZHHQSHUFHQWWR
SHUFHQWRIWKHSRSXODWLRQLVXVXDOO\PHQWLRQHGDVWKHPDJLFVDPSOLQJIUDFWLRQ
ZLWKDPLQLPXPRIDERXWSDUWLFLSDQWV+RZHYHUWKHPRUHVFLHQWLILFWKH
VDPSOLQJSURFHGXUHVWKHVPDOOHUWKHVDPSOHVL]HFDQEHZKLFKLVZK\RSLQLRQ
SROOVFDQSURGXFHDFFXUDWHSUHGLFWLRQVIURPVDPSOHVDVVPDOODVSHUFHQWRIWKH
SRSXODWLRQ7KHIROORZLQJURXJKHVWLPDWHVRIVDPSOHVL]HVIRUVSHFLILFW\SHVRI
TXDQWLWDWLYHPHWKRGVKDYHDOVREHHQDJUHHGRQE\VHYHUDOVFKRODUVFRUUHODWLRQDO
UHVHDUFK²DWOHDVWSDUWLFLSDQWVFRPSDUDWLYHDQGH[SHULPHQWDOSURFHGXUHV²DW
OHDVWSDUWLFLSDQWVLQHDFKJURXSIDFWRUDQDO\WLFDQGRWKHUPXOWLYDULDWH
SURFHGXUHV²DWOHDVWSDUWLFLSDQWV
x 6WDWLVWLFDOFRQVLGHUDWLRQ$EDVLFUHTXLUHPHQWLQTXDQWLWDWLYHUHVHDUFKLVWKDWWKH
VDPSOHVKRXOGKDYHDµQRUPDOGLVWULEXWLRQ¶DQG+DWFKDQG/D]DUDWRQ DUJXHV
WKDWWRDFKLHYHWKLVWKHVDPSOHQHHGVWRLQFOXGHRUPRUHSHRSOH+RZHYHU+DWFK
DQG/D]DUDWRQDOVRHPSKDVL]HWKDWVPDOOHUVDPSOHVL]HVFDQEHFRPSHQVDWHGIRUE\
XVLQJFHUWDLQVSHFLDOVWDWLVWLFDOSURFHGXUHV IRUH[DPSOHQRQSDUDPHWULFWHVWV²VHH
6HFWLRQ

x 6DPSOHFRPSRVLWLRQ$IXUWKHUFRQVLGHUDWLRQLVZKHWKHUWKHUHDUHDQ\GLVWLQFW
VXEJURXSVZLWKLQWKHVDPSOHWKDWPD\EHH[SHFWHGWREHKDYHGLIIHUHQWO\IURPWKH
RWKHUV,IZHFDQLGHQWLI\VXFKVXEJURXSVLQDGYDQFH IRUH[DPSOHLQPRVW/
VWXGLHVRIVFKRROFKLOGUHQJLUOVKDYHEHHQIRXQGWRSHUIRUPGLIIHUHQWO\IURPER\V
ZHVKRXOGVHWWKHVDPSOHVL]HVRWKDWWKHPLQLPXPVL]HDSSOLHVWRWKHVPDOOHVW
VXEJURXSLQWKHVDPSOH
x 6DIHW\PDUJLQ:KHQVHWWLQJWKHILQDOVDPSOHVL]HLWLVDGYLVDEOHWROHDYHDGHFHQW
µPDUJLQ¶WRSURYLGHIRUXQIRUHVHHQRUXQSODQQHGFLUFXPVWDQFHV)RUH[DPSOHVRPH
SDUWLFLSDQWVDUHOLNHO\WRGURSRXWRIDWOHDVWVRPHSKDVHVRIWKHSURMHFWVRPH
TXHVWLRQQDLUHVZLOODOZD\VKDYHWREHGLVTXDOLILHGIRURQHUHDVRQRUDQRWKHUDQG
ZHPD\DOVRGHWHFWXQH[SHFWHGVXEJURXSVWKDWQHHGWREHWUHDWHGVHSDUDWHO\
x 5HYHUVHDSSURDFK%HFDXVHVWDWLVWLFDOVLJQLILFDQFH VHH6HFWLRQ GHSHQGVRQ

WKHVDPSOHVL]HRXUSULQFLSOHFRQFHUQVKRXOGEHWRVDPSOHHQRXJKOHDUQHUVIRUWKH
H[SHFWHGUHVXOWVWREHDEOHWRUHDFKVWDWLVWLFDOVLJQLILFDQFH7KDWLVZHFDQWDNHD
µUHYHUVHDSSURDFK¶ILUVWZHDSSUR[LPDWHWKHH[SHFWHGPDJQLWXGHRUSRZHURIWKH
H[SHFWHGUHVXOWVDQGWKHQGHWHUPLQHWKHVDPSOHVL]HWKDWLVQHFHVVDU\WRGHWHFWWKLV
HIIHFWLILWDFWXDOO\H[LVWVLQWKHSRSXODWLRQ)RUH[DPSOHDWDSVLJQLILFDQFH
OHYHODQH[SHFWHGFRUUHODWLRQRIUHTXLUHVDWOHDVWSDUWLFLSDQWV 7KHVHILJXUHV
FDQEHORRNHGXSLQFRUUHODWLRQDOWDEOHVDYDLODEOHLQPRVWWH[WVRQVWDWLVWLFVVHHIRU
H[DPSOHKWWSZZZXZHDFXNKOVVOODVVWDWLVWLFVLQOLQJXLVWLFV$SSHQL[SGI
5HVHDUFKHUVFDQDOVRXVHDPRUHFRPSOH[SURFHGXUHµSRZHUDQDO\VLV¶WRGHFLGHRQ
WKHQHFHVVDU\VDPSOHVL]H )RUDIUHH,QWHUQHWEDVHGSRZHUDQDO\VLVVRIWZDUHVHH
%XFKQHUHWDO
7KHSUREOHPRIUHVSRQGHQWVHOIVHOHFWLRQ
7RFRQFOXGHWKHGLVFXVVLRQRIWKHYDULRXVVDPSOLQJLVVXHVOHWPHKLJKOLJKWDSRWHQWLDO
SLWIDOOWKDWPLJKWSXWWKHYDOLGLW\RIDQLQYHVWLJDWLRQDWULVNWKHSUREOHPRISDUWLFLSDQW
VHOIVHOHFWLRQ7KLVUHIHUVWRFDVHVZKHQIRUYDULRXVUHDVRQVWKHDFWXDOFRPSRVLWLRQRI
WKHVDPSOHLVQRWRQO\WKHIXQFWLRQRIVRPHV\VWHPDWLFVHOHFWLRQSURFHVVEXWDOVRRI
IDFWRUVUHODWHGWRWKHUHVSRQGHQWV¶RZQZLOOLQJQHVVWRSDUWLFLSDWH3UREOHPVFDQDULVH
IRUH[DPSOHZKHQ
x UHVHDUFKHUVLQYLWHYROXQWHHUVWRWDNHSDUWLQDVWXG\ RFFDVLRQDOO\HYHQRIIHULQJ
PRQH\WRFRPSHQVDWHIRUWKHLUWLPH
x WKHGHVLJQDOORZVIRUDKLJKGHJUHHRIGURSRXW RUµPRUWDOLW\¶ LQZKLFKFDVH
SDUWLFLSDQWVVHOIVHOHFWWKHPVHOYHVRXWRIWKHVDPSOH
x SDUWLFLSDQWVDUHIUHHWRFKRRVHZKHWKHUWKH\SDUWLFLSDWHLQDVWXG\RUQRW IRU
H[DPSOHLQSRVWDOTXHVWLRQQDLUHVXUYH\V
6HOIVHOHFWLRQLVLQHYLWDEOHWRVRPHH[WHQWEHFDXVHIHZLQYHVWLJDWLRQVFDQEHPDGH
FRPSXOVRU\KRZHYHULQVRPHFDVHV²IRUH[DPSOHLQWKHH[DPSOHVDERYH²LWFDQ
UHDFKVXFKDGHJUHHWKDWWKHUHLVDJRRGFKDQFHWKDWWKHUHVXOWLQJVDPSOHZLOOQRWEH
VLPLODUWRWKHWDUJHWSRSXODWLRQ)RUH[DPSOHYROXQWHHUVPD\EHGLIIHUHQWIURPQRQ

YROXQWHHUVLQWKHLUDSWLWXGHPRWLYDWLRQRUVRPHRWKHUEDVLFFKDUDFWHULVWLFVDQG
GURSRXWVPD\DOVRVKDUHVRPHFRPPRQIHDWXUHVWKDWZLOOEHXQGHUUHSUHVHQWHGLQWKH
VDPSOHZLWKWKHLUGHSDUWXUH IRUH[DPSOHGURSRXWVPD\EHPRUHXQPRWLYDWHGWKDQWKHLU
SHHUVDQGWKHUHIRUHWKHLUGHSDUWXUHPLJKWPDNHWKHUHPDLQLQJSDUWLFLSDQWV¶JHQHUDOOHYHO
RIPRWLYDWLRQXQQDWXUDOO\KLJK &RQVHTXHQWO\WKHVDPSOHPD\ORVHLWVUHSUHVHQWDWLYH
FKDUDFWHUZKLFKRIFRXUVHZRXOGSUHYHQWDQ\PHDQLQJIXOJHQHUDOL]DELOLW\%URZQ
H[SODLQVWKLVZHOOZLWKUHJDUGWRTXHVWLRQQDLUHVXUYH\V
7KHSUREOHPLVWKDWWKHW\SHVRIUHVSRQGHQWVZKRUHWXUQTXHVWLRQQDLUHVPD\EHD
VSHFLILFW\SHRIµHDJHUEHDYHU¶RUµJXQJKR¶UHVSRQGHQW7KXVWKHUHVXOWVRIWKH
VXUYH\FDQRQO\EHJHQHUDOL]HGWRµHDJHUEHDYHU¶RUµJXQJKR¶SHRSOHLQWKH
SRSXODWLRQUDWKHUWKDQWRWKHHQWLUHSRSXODWLRQ
4XHVWLRQQDLUHVXUYH\V
6XUYH\VWXGLHVDLPDWGHVFULELQJWKHFKDUDFWHULVWLFVRIDSRSXODWLRQE\H[DPLQLQJD
VDPSOHRIWKDWJURXS$OWKRXJKVXUYH\GDWDFDQEHFROOHFWHGE\PHDQVRIVWUXFWXUHG
LQWHUYLHZV IRUH[DPSOHLQPDUNHWUHVHDUFKRURSLQLRQSROOV WKHPDLQGDWDFROOHFWLRQ
PHWKRGLQVXUYH\VLVWKHXVHRITXHVWLRQQDLUHVLQWKLVVHFWLRQ,IRFXVRQO\RQWKLV
RSWLRQ7KHUHVXOWVRIDTXHVWLRQQDLUHVXUYH\DUHW\SLFDOO\TXDQWLWDWLYHDOWKRXJKWKH
LQVWUXPHQWPD\DOVRFRQWDLQVRPHRSHQHQGHGTXHVWLRQVWKDWZLOOUHTXLUHDTXDOLWDWLYH
DQDO\VLV7KHPDLQPHWKRGRORJLFDOLVVXHVFRQFHUQLQJVXUYH\VDUH D KRZWRVDPSOHWKH
SDUWLFLSDQWVZKLFKKDVEHHQGLVFXVVHGDERYHDQG E KRZWRGHVLJQDQGDGPLQLVWHUWKH
UHVHDUFKWRROWKHµVHOIUHSRUWTXHVWLRQQDLUH¶
$VVWDWHGLQ&KDSWHUWKHHVVHQFHRIVFLHQWLILFUHVHDUFKLVWU\LQJWRILQGDQVZHUVWR
TXHVWLRQVLQDV\VWHPDWLFDQGGLVFLSOLQHGPDQQHUDQGLWLVWKHUHIRUHQRZRQGHUWKDWWKH
TXHVWLRQQDLUHKDVEHFRPHRQHRIWKHPRVWSRSXODUUHVHDUFKLQVWUXPHQWVDSSOLHGLQWKH
VRFLDOVFLHQFHV7KHSRSXODULW\RITXHVWLRQQDLUHVLVGXHWRWKHIDFWWKDWWKH\DUH
UHODWLYHO\HDV\WRFRQVWUXFWH[WUHPHO\YHUVDWLOHDQGXQLTXHO\FDSDEOHRIJDWKHULQJD
ODUJHDPRXQWRILQIRUPDWLRQTXLFNO\LQDIRUPWKDWLVUHDGLO\SURFHVVLEOH,QGHHGWKH
IUHTXHQF\RIXVHRIVHOIFRPSOHWHGTXHVWLRQQDLUHVDVDUHVHDUFKWRROLQDSSOLHG
OLQJXLVWLFVLVVXUSDVVHGRQO\E\WKDWRIODQJXDJHSURILFLHQF\WHVWV )RUUHYLHZVVHH
%DNHULQSUHVV'|UQ\HL <HWLQVSLWHRIWKHZLGHDSSOLFDWLRQRITXHVWLRQQDLUHVLQ
RXUILHOGWKHUHGRHVQRWVHHPWREHVXIILFLHQWDZDUHQHVVLQWKHSURIHVVLRQDERXWWKH
WKHRU\RITXHVWLRQQDLUHGHVLJQDQGSURFHVVLQJWKHXVXDOSHUFHSWLRQLVWKDWDQ\ERG\
ZLWKDELWRIFRPPRQVHQVHDQGJRRGZRUGSURFHVVLQJVRIWZDUHFDQFRQVWUXFWDILQH
TXHVWLRQQDLUH8QIRUWXQDWHO\WKLVSHUFHSWLRQLVQRWWUXH-XVWDVLQHYHU\GD\OLIHZKHUH
QRWHYHU\TXHVWLRQHOLFLWVWKHULJKWDQVZHULWLVDOOWRRFRPPRQLQVFLHQWLILF
LQYHVWLJDWLRQVWRFRPHDFURVVTXHVWLRQQDLUHVWKDWIDLO,QIDFW,EHOLHYHWKDWPRVW
TXHVWLRQQDLUHVXVHGLQDSSOLHGOLQJXLVWLFUHVHDUFKDUHVRPHZKDWDGKRFLQVWUXPHQWV
DQGTXHVWLRQQDLUHVWKDW\LHOGVFRUHVZLWKVXIILFLHQW DQGZHOOGRFXPHQWHG UHOLDELOLW\
DQGYDOLGLW\DUHQRWWKDWHDV\WRFRPHE\LQRXUILHOG6ROHWXVVHHKRZTXHVWLRQQDLUH
WKHRU\FDQKHOSXVWRGHYLVHEHWWHULQVWUXPHQWV

:KDWDUHTXHVWLRQQDLUHVDQGZKDWGRWKH\PHDVXUH"
$OWKRXJKWKHWHUPµTXHVWLRQQDLUH¶LVRQHWKDWPRVWRIXVDUHIDPLOLDUZLWKLWLVQRWD
VWUDLJKWIRUZDUGWDVNWRSURYLGHDSUHFLVHGHILQLWLRQRILW7RVWDUWZLWKWKHWHUPLVSDUWO\
DPLVQRPHUEHFDXVHPDQ\TXHVWLRQQDLUHVGRQRWFRQWDLQDQ\UHDOTXHVWLRQVWKDWHQG
ZLWKDTXHVWLRQPDUN,QGHHGTXHVWLRQQDLUHVDUHDOVRRIWHQUHIHUUHGWRXQGHUGLIIHUHQW
QDPHVVXFKDVµLQYHQWRULHV¶µIRUPV¶µRSLQQLRQDLUHV¶µWHVWV¶µEDWWHULHV¶µFKHFNOLVWV¶
µVFDOHV¶µVXUYH\V¶µVFKHGXOHV¶µVWXGLHV¶µSURILOHV¶µLQGH[HVLQGLFDWRUV¶RUHYHQ
VLPSO\µVKHHWV¶
6HFRQGHYHQLQWKHUHVHDUFKFRPPXQLW\WKHJHQHUDOUXEULFRIµTXHVWLRQQDLUH¶KDV
EHHQXVHGLQDWOHDVWWZREURDGVHQVHV D ,QWHUYLHZVFKHGXOHVJXLGHV²GHVFULEHGLQ
GHWDLOLQ6HFWLRQDQG E VHOIDGPLQLVWHUHGSHQFLODQGSDSHUTXHVWLRQQDLUHV,Q
WKLVFKDSWHU,FRYHUWKLVVHFRQGW\SHRITXHVWLRQQDLUHGHILQLQJLWDVµDQ\ZULWWHQ
LQVWUXPHQWVWKDWSUHVHQWUHVSRQGHQWVZLWKDVHULHVRITXHVWLRQVRUVWDWHPHQWVWRZKLFK
WKH\DUHWRUHDFWHLWKHUE\ZULWLQJRXWWKHLUDQVZHUVRUVHOHFWLQJIURPDPRQJH[LVWLQJ
DQVZHUV¶ %URZQ
:KDWGRTXHVWLRQQDLUHVPHDVXUH"%URDGO\VSHDNLQJTXHVWLRQQDLUHVFDQ\LHOGWKUHH
W\SHVRIGDWDDERXWWKHUHVSRQGHQW
)DFWXDOTXHVWLRQVZKLFKDUHXVHGWRILQGRXWFHUWDLQIDFWVDERXWWKHUHVSRQGHQWVVXFK
DVGHPRJUDSKLFFKDUDFWHULVWLFV IRUH[DPSOHDJHJHQGHUDQGUDFH UHVLGHQWLDO
ORFDWLRQPDULWDODQGVRFLRHFRQRPLFVWDWXVOHYHORIHGXFDWLRQRFFXSDWLRQ
ODQJXDJHOHDUQLQJKLVWRU\DPRXQWRIWLPHVSHQWLQDQ/HQYLURQPHQWHWF
%HKDYLRXUDOTXHVWLRQVZKLFKDUHXVHGWRILQGRXWZKDWWKHUHVSRQGHQWVDUHGRLQJRU
KDYHGRQHLQWKHSDVWIRFXVLQJRQDFWLRQVOLIHVW\OHVKDELWVDQGSHUVRQDOKLVWRU\
$WWLWXGLQDOTXHVWLRQVZKLFKDUHXVHGWRILQGRXWZKDWSHRSOHWKLQNFRYHULQJDWWLWXGHV
RSLQLRQVEHOLHIVLQWHUHVWVDQGYDOXHV
:HVKRXOGQRWHDWWKLVSRLQWWKDWDOWKRXJKTXHVWLRQQDLUHVDUHRIWHQYHU\VLPLODUWR
ZULWWHQWHVWVWKHUHLVDEDVLFGLIIHUHQFHEHWZHHQWKHWZRLQVWUXPHQWW\SHV$WHVWWDNHVD
VDPSOHRIWKHUHVSRQGHQW¶VEHKDYLRXUNQRZOHGJHIRUWKHSXUSRVHRIHYDOXDWLQJWKH
LQGLYLGXDO¶VPRUHJHQHUDOXQGHUO\LQJFRPSHWHQFHDELOLWLHVVNLOOV IRUH[DPSOHRYHUDOO
/SURILFLHQF\ 7KXVDWHVWPHDVXUHVKRZZHOOVRPHRQHFDQGRVRPHWKLQJ,Q
FRQWUDVWTXHVWLRQQDLUHLWHPVGRQRWKDYHJRRGRUEDGDQVZHUVWKH\HOLFLWLQIRUPDWLRQ
DERXWWKHUHVSRQGHQWVLQDQRQHYDOXDWLYHPDQQHUZLWKRXWJDXJLQJWKHLUSHUIRUPDQFH
DJDLQVWDVHWRIFULWHULD7KXVDOWKRXJKVRPHFRPPHUFLDOO\DYDLODEOHTXHVWLRQQDLUHVDUH
DFWXDOO\FDOOHGµWHVWV¶WKHVHDUHQRWWHVWVLQWKHVDPHVHQVHDVDFKLHYHPHQWRUDSWLWXGH
WHVWV
,QDVLPLODUYHLQµSURGXFWLRQTXHVWLRQQDLUHV¶²RUDVWKH\KDYHEHHQWUDGLWLRQDOO\
EHHQFDOOHGµGLVFRXUVHFRPSOHWLRQWDVNV¶ '&7V ²DUHQRWTXHVWLRQQDLUHVSURSHUHLWKHU
7KHVHSHQFLODQGSDSHULQVWUXPHQWVFRPPRQO\XVHGLQLQWHUODQJXDJHSUDJPDWLFV
UHVHDUFKUHTXLUHWKHLQIRUPDQWWRSURGXFHVRPHVRUWRIDXWKHQWLFODQJXDJHGDWDDVD

UHVSRQVHWRVLWXDWLRQDOSURPSWV7KXVWKH\DUHVWUXFWXUHGODQJXDJHHOLFLWDWLRQ
LQVWUXPHQWVDQGDVVXFKWKH\VDPSOHWKHUHVSRQGHQW¶VFRPSHWHQFHLQSHUIRUPLQJ
FHUWDLQWDVNVZKLFKPDNHVWKHPVLPLODUWRODQJXDJHWHVWV
0XOWLLWHPVFDOHV
2QHFHQWUDOLVVXHDERXWTXHVWLRQQDLUHGHVLJQFRQFHUQVKRZWKHLWHPVWREHUHVSRQGHGWR
DUHDFWXDOO\ZRUGHG:KHQLWFRPHVWRDVVHVVLQJQRQIDFWXDOPDWWHUVVXFKDVWKH
UHVSRQGHQWV¶DWWLWXGHVEHOLHIVDQGRWKHUSHUVRQDORUPHQWDOYDULDEOHVWKHDFWXDO
ZRUGLQJRIWKHLWHPVFDQDVVXPHDQXQH[SHFWHGLPSRUWDQFH0LQRUGLIIHUHQFHVLQKRZ
DTXHVWLRQLVIRUPXODWHGDQGIUDPHGFDQRIWHQSURGXFHUDGLFDOO\GLIIHUHQWOHYHOVRI
DJUHHPHQWRUGLVDJUHHPHQW)RUH[DPSOH&RQYHUVHDQG3UHVVHU UHSRUWRQD
VXUYH\LQZKLFKVLPSO\FKDQJLQJµIRUELG¶WRµQRWDOORZ¶LQWKHZRUGLQJRIWKHLWHPµ'R
\RXWKLQNWKH8QLWHG6WDWHVVKRXOG>IRUELGQRWDOORZ@SXEOLFVSHHFKHVDJDLQVW
GHPRFUDF\"¶SURGXFHGGUDVWLFDOO\GLIIHUHQWUHVSRQVHV6LJQLILFDQWO\PRUHSHRSOHZHUH
ZLOOLQJWRµQRWDOORZ¶VSHHFKHVDJDLQVWGHPRFUDF\WKDQZHUHZLOOLQJWRµIRUELG¶WKHP
HYHQWKRXJKµDOORZ¶DQGµIRUELG¶DUHH[DFWORJLFDORSSRVLWHV*LYHQWKDWLQWKLVH[DPSOH
RQO\RQHZRUGZDVFKDQJHGDQGWKDWWKHDOWHUQDWLYHYHUVLRQKDGDQDOPRVWLGHQWLFDO
PHDQLQJWKLVLVDJRRGLOOXVWUDWLRQWKDWLWHPZRUGLQJLQJHQHUDOKDVDVXEVWDQWLDOLPSDFW
RQWKHUHVSRQVHV
6RKRZFDQZHGHDOZLWKWKHVHHPLQJO\XQSUHGLFWDEOHLPSDFWRILWHPZRUGLQJ"'R
ZHKDYHWRFRQFOXGHWKDWTXHVWLRQQDLUHVVLPSO\FDQQRWDFKLHYHWKHNLQGRIDFFXUDF\
WKDWLVQHHGHGIRUVFLHQWLILFPHDVXUHPHQWSXUSRVHV":HZRXOGKDYHWRLIPHDVXUHPHQW
WKHRUHWLFLDQV²DQGSDUWLFXODUO\$PHULFDQSV\FKRORJLVW5HQVLV/LNHUWLQKLV3K'
GLVVHUWDWLRQLQ²KDGQRWGLVFRYHUHGDQLQJHQLRXVZD\RIJHWWLQJDURXQGWKH
SUREOHPE\XVLQJµPXOWLLWHPVFDOHV¶7KHVHVFDOHVUHIHUWRDFOXVWHURIVHYHUDO
GLIIHUHQWO\ZRUGHGLWHPVWKDWIRFXVRQWKHVDPHWDUJHW7KHLWHPVFRUHVIRUWKHVLPLODU
TXHVWLRQVDUHVXPPHGUHVXOWLQJLQDWRWDOVFDOHVFRUHDQGWKHXQGHUO\LQJDVVXPSWLRQLV
WKDWDQ\LGLRV\QFUDWLFLQWHUSUHWDWLRQRIDQLWHPZLOOEHDYHUDJHGRXWGXULQJWKH
VXPPDWLRQRIWKHLWHPVFRUHV,QRWKHUZRUGVLIZHXVHPXOWLLWHPVFDOHVµQR
LQGLYLGXDOLWHPFDUULHVDQH[FHVVLYHORDGDQGDQLQFRQVLVWHQWUHVSRQVHWRRQHLWHP
ZRXOGFDXVHOLPLWHGGDPDJH¶ 6NHKDQ %\ZD\RILOOXVWUDWLRQ7DEOH
FRQWDLQVIRXULWHPVEHORQJLQJWRWKHµ6DWLDWLRQFRQWURO¶VFDOHLQ7VHQJHWDO¶V
6HOI5HJXODWRU\&DSDFLW\LQ9RFDEXODU\/HDUQLQJTXHVWLRQQDLUHFRQFHUQLQJ
HOLPLQDWLQJERUHGRPDQGDGGLQJH[WUDDWWUDFWLRQWRYRFDEXODU\OHDUQLQJWDVNV

6$7,$7,21&21752/
x 2QFHWKHQRYHOW\RIOHDUQLQJYRFDEXODU\LVJRQH,HDVLO\EHFRPHLPSDWLHQWZLWKLW
>5(9(56('&2',1*@

x 'XULQJWKHSURFHVVRIOHDUQLQJYRFDEXODU\,IHHOVDWLVILHGZLWKWKHZD\V,HOLPLQDWH
ERUHGRP
x 'XULQJWKHSURFHVVRIOHDUQLQJYRFDEXODU\,DPFRQILGHQWWKDW,FDQRYHUFRPHDQ\
VHQVHRIERUHGRP
x :KHQIHHOLQJERUHGZLWKOHDUQLQJYRFDEXODU\,NQRZKRZWRUHJXODWHP\PRRGLQ
RUGHUWRLQYLJRUDWHWKHOHDUQLQJSURFHVV
7DEOH7KHIRXULWHPVPDNLQJXSWKHµ6DWLDWLRQFRQWURO¶VFDOHLQ7VHQJHWDO¶V
6HOI5HJXODWRU\&DSDFLW\LQ9RFDEXODU\/HDUQLQJTXHVWLRQQDLUH
7KXVPXOWLLWHPVFDOHVPD[LPL]HWKHVWDEOHFRPSRQHQWWKDWWKHLWHPVVKDUHDQG
UHGXFHWKHH[WUDQHRXVLQIOXHQFHVXQLTXHWRWKHLQGLYLGXDOLWHPV$FFRUGLQJO\WKHUHLVD
JHQHUDOFRQVHQVXVDPRQJVXUYH\VSHFLDOLVWVWKDWPRUHWKDQRQHLWHP XVXDOO\± LV
QHHGHGWRDGGUHVVHDFKLGHQWLILHGFRQWHQWDUHDDOODLPHGDWWKHVDPHWDUJHWEXWGUDZLQJ
XSRQVOLJKWO\GLIIHUHQWDVSHFWVRILW$QREYLRXVSRLQWWKDWLVQHYHUWKHOHVVRIWHQ
RYHUORRNHGE\QRYLFHUHVHDUFKHUVLVWKDWLQWKHILQDOYHUVLRQRIWKHTXHVWLRQQDLUHWKH
LWHPVRIWKHYDULRXVPXOWLLWHPVFDOHVKDYHWREHPL[HGXSWRFUHDWHDVHQVHRIYDULHW\
DQGWRSUHYHQWUHVSRQGHQWVIURPVLPSO\UHSHDWLQJSUHYLRXVDQVZHUV
:ULWLQJTXHVWLRQQDLUHLWHPV
7KHW\SLFDOTXHVWLRQQDLUHLVDKLJKO\VWUXFWXUHGGDWDFROOHFWLRQLQVWUXPHQWZLWKPRVW
LWHPVHLWKHUDVNLQJDERXWYHU\VSHFLILFSLHFHVRILQIRUPDWLRQ IRUH[DPSOHRQH¶V
DGGUHVVRUIRRGSUHIHUHQFH RUJLYLQJYDULRXVUHVSRQVHRSWLRQVIRUWKHUHVSRQGHQWWR
FKRRVHIURPIRUH[DPSOHE\WLFNLQJDER[RUFLUFOLQJWKHPRVWDSSURSULDWHRSWLRQ7KLV
PDNHVTXHVWLRQQDLUHGDWDSDUWLFXODUO\VXLWHGIRUTXDQWLWDWLYHVWDWLVWLFDODQDO\VLV,WLVRI
FRXUVHSRVVLEOHWRGHYLVHDTXHVWLRQQDLUHWKDWLVPDGHXSRIRSHQHQGHGLWHPV IRU
H[DPSOHµ3OHDVHGHVFULEH\RXUGUHDPVIRUWKHIXWXUH«¶ WKHUHE\SURYLGLQJGDWDWKDW
LVTXDOLWDWLYHDQGH[SORUDWRU\LQQDWXUHEXWWKLVSUDFWLFHLVXVXDOO\GLVFRXUDJHGE\
WKHRUHWLFLDQV7KHSUREOHPZLWKTXHVWLRQQDLUHVIURPDTXDOLWDWLYHSHUVSHFWLYHLVWKDW
WKH\LQKHUHQWO\LQYROYHDVRPHZKDWVXSHUILFLDODQGUHODWLYHO\EULHIHQJDJHPHQWZLWKWKH
WRSLFRQWKHSDUWRIWKHUHVSRQGHQW7KHUHIRUHQRPDWWHUKRZFUHDWLYHO\ZHIRUPXODWH
WKHLWHPVWKH\DUHXQOLNHO\WR\LHOGWKHNLQGRIULFKDQGVHQVLWLYHGHVFULSWLRQRIHYHQWV
DQGSDUWLFLSDQWSHUVSHFWLYHVWKDWTXDOLWDWLYHLQWHUSUHWDWLRQVDUHJURXQGHGLQ,IZHDUH
VHHNLQJORQJDQGGHWDLOHGSHUVRQDODFFRXQWVRWKHUUHVHDUFKPHWKRGVVXFKDVLQWHUYLHZV
VHH6HFWLRQ DUHOLNHO\WREHPRUHVXLWDEOHIRURXUSXUSRVH5REVRQ KDV
VXPPDUL]HGWKLVSRLQWZHOOµ7KHGHVLUHWRXVHRSHQHQGHGTXHVWLRQVDSSHDUVWREH
DOPRVWXQLYHUVDOLQQRYLFHUHVHDUFKHUVEXWLVXVXDOO\UDSLGO\H[WLQJXLVKHGZLWK
H[SHULHQFH¶
7KXVPRVWSURIHVVLRQDOTXHVWLRQQDLUHVDUHSULPDULO\PDGHXSRIµFORVHGHQGHG¶
LWHPVZKLFKGRQRWUHTXLUHWKHUHVSRQGHQWVWRSURGXFHDQ\IUHHZULWLQJLQVWHDG
UHVSRQGHQWVDUHWRFKRRVHRQHRIWKHJLYHQDOWHUQDWLYHV UHJDUGOHVVRIZKHWKHUWKHLU
SUHIHUUHGDQVZHULVDPRQJWKHP 7KHVHOHFWHGUHVSRQVHRSWLRQVFDQWKHQHDVLO\EH
QXPHULFDOO\FRGHGDQGHQWHUHGLQWRDFRPSXWHUGDWDEDVH+DYLQJVDLGWKDWPRVW

TXHVWLRQQDLUHVGRFRQWDLQFHUWDLQSDUWLDOO\RSHQHQGHGLWHPVDQGWKHVHZLOOEH
VXPPDUL]HGDIWHUDQRYHUYLHZRIWKHPRVWFRPPRQFORVHGHQGHGLWHPIRUPDWV )RUD
PRUHGHWDLOHGGHVFULSWLRQRIWKHLWHPW\SHVZLWKLOOXVWUDWLRQVVHH'|UQ\HL
/LNHUWVFDOHV
7KHPRVWIDPRXVW\SHRIFORVHGHQGHGLWHPVLVXQGRXEWHGO\WKHµ/LNHUWVFDOH¶ QDPHG
DIWHULWVLQYHQWRU ZKLFKFRQVLVWVRIDFKDUDFWHULVWLFVWDWHPHQWDQGUHVSRQGHQWVDUH
DVNHGWRLQGLFDWHWKHH[WHQWWRZKLFKWKH\µDJUHH¶RUµGLVDJUHH¶ZLWKLWE\PDUNLQJ IRU
H[DPSOHFLUFOLQJ RQHRIWKHUHVSRQVHVUDQJLQJIURPµVWURQJO\DJUHH¶WRµVWURQJO\
GLVDJUHH¶)RUH[DPSOH
+XQJDULDQVDUHJHQXLQHO\QLFHSHRSOH
6WURQJO\$JUHH1HLWKHUDJUHH'LVDJUHH6WURQJO\
DJUHHQRUGLVDJUHHGLVDJUHH
$IWHUWKHLWHPKDVEHHQDGPLQLVWHUHGHDFKUHVSRQVHRSWLRQLVDVVLJQHGDQXPEHUIRU
VFRULQJSXUSRVHV IRUH[DPSOHµVWURQJO\DJUHH¶ «µVWURQJO\GLVDJUHH¶ DQGWKH
VFRUHVIRUWKHLWHPVDGGUHVVLQJWKHVDPHWDUJHWDUHVXPPHGXSRUDYHUDJHG
6HPDQWLFGLIIHUHQWLDOVFDOHV
$QRWKHUIUHTXHQWO\DSSOLHGZD\RIHOLFLWLQJDJUDGXDWHGUHVSRQVHLVWKHµVHPDQWLF
GLIIHUHQWLDOVFDOH¶7KLVWHFKQLTXHLVSRSXODUEHFDXVHE\XVLQJLWUHVHDUFKHUVFDQDYRLG
ZULWLQJVWDWHPHQWV ZKLFKLVQRWDOZD\VHDV\ LQVWHDGUHVSRQGHQWVDUHDVNHGWR
LQGLFDWHWKHLUDQVZHUVE\PDUNLQJDFRQWLQXXP ZLWKDWLFNRUDQµ;¶ EHWZHHQWZR
ELSRODUDGMHFWLYHVDWWKHH[WUHPHV)RUH[DPSOH
5HVHDUFKPHWKRGRORJ\WH[WVDUH
GLIILFXOWBBBBBBBBBBBBBBBB;BBBBHDV\
XVHOHVVBBBB;BBBBBBBBBBBBBBBBXVHIXO
6HPDQWLFGLIIHUHQWLDOVFDOHVDUHVLPLODUWR/LNHUWVFDOHVLQWKDWVHYHUDOLWHPVDUHXVHG
WRHYDOXDWHWKHVDPHWDUJHWDQGPXOWLLWHPVFRUHVDUHFRPSXWHGE\VXPPLQJXSWKH
LQGLYLGXDOLWHPVFRUHV$QLPSRUWDQWWHFKQLFDOSRLQWFRQFHUQLQJWKHFRQVWUXFWLRQRI
VXFKELSRODUVFDOHVLVWKDWWKHSRVLWLRQRIWKHµQHJDWLYH¶DQGµSRVLWLYH¶SROHVLIWKH\FDQ
EHGHVLJQDWHGDVVXFKVKRXOGEHYDULHG LHWKHSRVLWLYHSROHVKRXOGDOWHUQDWHEHWZHHQ
EHLQJRQWKHULJKWDQGWKHOHIWKDQGVLGHV WRDYRLGVXSHUILFLDOUHVSRQGLQJRUDSRVLWLRQ
UHVSRQVHVHW $LNHQ
1XPHULFDOUDWLQJVFDOHV
µ1XPHULFDOUDWLQJVFDOHV¶LQYROYHµJLYLQJVRPDQ\PDUNVRXWRIVRPDQ\¶WKDWLV
DVVLJQLQJRQHRIVHYHUDOQXPEHUV ZKLFKFRUUHVSRQGWRDVHULHVRIRUGHUHGFDWHJRULHV
WRGHVFULEHDIHDWXUHRIWKHWDUJHW:HXVHWKLVWHFKQLTXHLQHYHU\GD\OLIHZKHQZHVD\
IRUH[DPSOHWKDWRQDVFDOHIURPRQHWRILYHVRPHWKLQJ IRUH[DPSOHDILOP ZDVZRUWK
WKUHHRUIRXU7KHSRSXODULW\RIWKLVVFDOLQJWHFKQLTXHLVGXHWRWKHIDFWWKDWWKHUDWLQJ

FRQWLQXXPFDQUHIHUWRDZLGHUDQJHRIDGMHFWLYHV IRUH[DPSOHH[FHOOHQWoSRRU
FRQVFLHQWLRXVoVODSGDVK RUDGYHUEV IRUH[DPSOHDOZD\VoQHYHU LQIDFW
QXPHULFDOUDWLQJVFDQHDVLO\EHWXUQHGLQWRVHPDQWLFGLIIHUHQWLDOVFDOHVDQGYLFHYHUVD
2WKHUFORVHGHQGHGLWHPW\SHV
'HSHQGLQJRQWKHSXUSRVHDQGWKHWRSLFRIWKHTXHVWLRQQDLUHDVZHOODVRQWKHDJHDQG
RWKHUFKDUDFWHULVWLFVRIWKHUHVSRQGHQWVTXHVWLRQQDLUHGHVLJQHUVKDYHDOVRXVHGD
YDULHW\RIRWKHUFORVHGHQGHGLWHPW\SHVWKHPRVWFRPPRQRIZKLFKDUHDVIROORZV
x 7UXH±IDOVHLWHPV:KLOHJHQHUDOO\LWLVWUXHWKDWWKHPRUHUHVSRQVHRSWLRQVDQLWHP
FRQWDLQVWKHPRUHDFFXUDWHHYDOXDWLRQLW\LHOGVWKHUHPLJKWEHFDVHVZKHQRQO\D
SRODUL]HG\HV±QRGHFLVLRQFDQEHFRQVLGHUHGUHOLDEOH)RUH[DPSOHOLWWOHFKLOGUHQ
DUHVRPHWLPHVVHHQDVLQFDSDEOHRISURYLGLQJPRUHHODERUDWHUDWLQJVDQGVRPH
SHUVRQDOLW\WHVWLWHPVDOVRIROORZDWUXH±IDOVHUDWLQJWRHQVXUHUHOLDELOLW\LQGRPDLQV
ZKHUHWKHUHVSRQGHQWPD\QRWEHDEOHWRSURSHUO\HYDOXDWHWKHGHJUHHWRZKLFKD
SDUWLFXODUIHDWXUHLVSUHVHQWWUXHRUQRW7KHSUREOHPZLWKWKLVIRUFHGFKRLFHIRUPDW
LVWKDWLWFDQVLPSOLI\WKLQJVWRRPXFKUHVXOWLQJLQKLJKO\UHGXFHGDQGRFFDVLRQDOO\
HYHQGLVWRUWHGGDWD
x 0XOWLSOHFKRLFHLWHPV0RVWDSSOLHGOLQJXLVWVDUHIDPLOLDUZLWKWKLVIRUPDWEHFDXVH
RILWVSRSXODULW\LQVWDQGDUGL]HG/SURILFLHQF\WHVWLQJ,QTXHVWLRQQDLUHVWKH\DUH
RIWHQXVHGZKHQDVNLQJDERXWSHUVRQDOLQIRUPDWLRQVXFKDVWKHOHYHORIHGXFDWLRQ
RIWKHUHVSRQGHQWV
x 5DQNRUGHULWHPV,WLVDFRPPRQKXPDQPHQWDODFWLYLW\WRUDQNRUGHUSHRSOH
REMHFWVRUHYHQDEVWUDFWFRQFHSWVDFFRUGLQJWRVRPHFULWHULRQDQGUDQNRUGHULWHPV
LQTXHVWLRQQDLUHVFDSLWDOL]HRQRXUIDPLOLDULW\ZLWKWKLVSURFHVV$VWKHQDPH
VXJJHVWVWKHVHLWHPVFRQWDLQVRPHVRUWRIDOLVWDQGUHVSRQGHQWVDUHDVNHGWRRUGHU
WKHLWHPVE\DVVLJQLQJDQXPEHUWRWKHPDFFRUGLQJWRWKHLUSUHIHUHQFHV$VKRUWOLVW
RIVD\LWHPVFDQWKHQEHTXDQWLILHGE\DVVLJQLQJSRLQWVWRWKHWRSUDQNHG
RSWLRQWRWKHPLGGOHDQGWRWKHORZHVWUDQNHGLWHP
2SHQHQGHGTXHVWLRQV
µ2SHQHQGHGTXHVWLRQV¶LQFOXGHLWHPVZKHUHWKHDFWXDOTXHVWLRQLVQRWIROORZHGE\
UHVSRQVHRSWLRQVIRUWKHUHVSRQGHQWWRFKRRVHIURPEXWUDWKHUE\VRPHEODQNVSDFH IRU
H[DPSOHGRWWHGOLQHV IRUWKHUHVSRQGHQWWRILOOLQ$VPHQWLRQHGHDUOLHUTXHVWLRQQDLUHV
DUHQRWSDUWLFXODUO\VXLWHGIRUWUXO\TXDOLWDWLYHH[SORUDWRU\UHVHDUFKEXWVRPHRSHQ
HQGHGTXHVWLRQVFDQVWLOOKDYHPHULW%\SHUPLWWLQJJUHDWHUIUHHGRPRIH[SUHVVLRQ
RSHQIRUPDWLWHPVFDQSURYLGHDIDUJUHDWHUµULFKQHVV¶WKDQIXOO\TXDQWLWDWLYHGDWD7KH
RSHQUHVSRQVHVFDQRIIHUJUDSKLFH[DPSOHVLOOXVWUDWLYHTXRWHVDQGFDQDOVROHDGXVWR
LGHQWLI\LVVXHVQRWSUHYLRXVO\DQWLFLSDWHG)XUWKHUPRUHVRPHWLPHVZHQHHGRSHQHQGHG
LWHPVIRUWKHVLPSOHUHDVRQWKDWZHGRQRWNQRZWKHUDQJHRISRVVLEOHDQVZHUVDQG
WKHUHIRUHFDQQRWSURYLGHSUHSUHSDUHGUHVSRQVHFDWHJRULHV

,QP\H[SHULHQFHRSHQHQGHGTXHVWLRQVZRUNSDUWLFXODUO\ZHOOLIWKH\DUHQRW
FRPSOHWHO\RSHQEXWFRQWDLQFHUWDLQJXLGDQFHDVLOOXVWUDWHGE\WKHIROORZLQJIRXU
TXHVWLRQW\SHV
x 6SHFLILFRSHQTXHVWLRQVDVNDERXWFRQFUHWHSLHFHVRILQIRUPDWLRQVXFKDVIDFWV
DERXWWKHUHVSRQGHQWSDVWDFWLYLWLHVRUSUHIHUHQFHV
x &ODULILFDWLRQTXHVWLRQVFDQEHDWWDFKHGWRDQVZHUVWKDWKDYHVRPHVSHFLDO
LPSRUWDQFHDQGVXFKTXHVWLRQVDUHDOVRDSSURSULDWHDIWHUWKHµ2WKHU¶FDWHJRU\LQD
PXOWLSOHFKRLFHLWHP IRUH[DPSOHE\VWDWLQJµ3OHDVHVSHFLI\«¶
x 6HQWHQFHFRPSOHWLRQZKHUHDQXQILQLVKHGVHQWHQFHEHJLQQLQJLVSUHVHQWHGIRUWKH
UHVSRQGHQWVWRFRPSOHWH IRUH[DPSOHµ7KHWKLQJ,OLNHGPRVWDERXWWKHFRXUVHLV
«¶ 7KLVFDQHOLFLWDPRUHPHDQLQJIXODQVZHUWKDQDVLPSOHTXHVWLRQ,KDYH
VXFFHVVIXOO\XVHGWKLVWHFKQLTXHRQYDULRXVIHHGEDFNIRUPV
x 6KRUWDQVZHUTXHVWLRQVDUHGLIIHUHQWIURPµHVVD\TXHVWLRQV¶ ZKLFKDUHQRW
UHFRPPHQGHGLQRUGLQDU\TXHVWLRQQDLUHVDQGWKHUHIRUHZLOOQRWEHGLVFXVVHG LQWKDW
WKH\DUHZRUGHGLQVXFKDIRFXVHGZD\WKDWWKHTXHVWLRQFDQEHDQVZHUHG
VXFFLQFWO\ZLWKDµVKRUWDQVZHU¶²WKLVLVXVXDOO\PRUHWKDQDSKUDVHDQGOHVVWKDQD
SDUDJUDSK
5XOHVDERXWLWHPZRUGLQJ
2YHUWKHSDVW\HDUVVXUYH\UHVHDUFKHUVKDYHDFFXPXODWHGDFRQVLGHUDEOHERG\RI
NQRZOHGJHDQGH[SHULHQFHDERXWZKDWPDNHVDTXHVWLRQQDLUHLWHPZRUNDQGZKDWWKH
SRWHQWLDOSLWIDOOVDUH+RZHYHULWQHHGVWREHSRLQWHGRXWWKDWLWHPGHVLJQLVQRWD
SHUFHQWVFLHQWLILFDFWLYLW\EHFDXVHLQRUGHUWRZULWHJRRGLWHPVRQHDOVRQHHGVDFHUWDLQ
DPRXQWRIFUHDWLYLW\DQGORWVRIFRPPRQVHQVH,QWKHDEVHQFHRIKDUGDQGIDVWUXOHVLW
LVWKHULJRURXVSLORWLQJRIWKHTXHVWLRQQDLUHWKDWZLOOSURGXFHJRRGSV\FKRPHWULF
SURSHUWLHV²VHH6HFWLRQ+HUHLVDOLVWRISUDFWLFDODGYLFHRQLWHPZRUGLQJ
FRQFHUQLQJFODULW\DQGDFFHVVLELOLW\
$LPIRUVKRUWDQGVLPSOHLWHPV:KHQHYHUSRVVLEOHTXHVWLRQQDLUHLWHPVVKRXOGEH
VKRUWUDUHO\H[FHHGLQJZRUGV7KH\VKRXOGSUHIHUDEO\EHZULWWHQLQVLPSOH
VHQWHQFHVUDWKHUWKDQFRPSRXQGRUFRPSOH[VHQWHQFHVDQGHDFKVKRXOGFRQWDLQ
RQO\RQHFRPSOHWHWKRXJKW
8VHVLPSOHDQGQDWXUDOODQJXDJH$VDUXOHLQTXHVWLRQQDLUHLWHPVZHVKRXOGDOZD\V
FKRRVHWKHVLPSOHVWZD\WRVD\VRPHWKLQJ,WHPVQHHGWREHNHSWFOHDUDQGGLUHFW
ZLWKRXWDQ\DFURQ\PVDEEUHYLDWLRQVFROORTXLDOLVPVSURYHUEVMDUJRQRUWHFKQLFDO
WHUPV:HVKRXOGWU\WRVSHDNWKHµFRPPRQODQJXDJH¶²WKHEHVWLWHPVDUHWKHRQHV
WKDWVRXQGOLNHEHLQJWDNHQIURPDQDFWXDOLQWHUYLHZ
$YRLGDPELJXRXVRUORDGHGZRUGVDQGVHQWHQFHV,WJRHVZLWKRXWVD\LQJWKDWDQ\
HOHPHQWVWKDWPLJKWPDNHWKHODQJXDJHRIWKHLWHPVXQFOHDURUDPELJXRXVQHHGWR
EHDYRLGHG7KHPRVWQRWRULRXVRIVXFKHOHPHQWVDUH

1RQVSHFLILFDGMHFWLYHVRUDGYHUEV IRUH[DPSOHµJRRG¶µHDV\¶µPDQ\¶
µVRPHWLPHV¶µRIWHQ¶
,WHPVFRQWDLQLQJXQLYHUVDOVVXFKDVµDOO¶µQRQH¶µQHYHU¶
0RGLI\LQJZRUGVVXFKDVµRQO\¶µMXVW¶µPHUHO\¶
:RUGVKDYLQJPRUHWKDQRQHPHDQLQJ
/RDGHGZRUGV IRUH[DPSOHµGHPRFUDWLF¶µPRGHUQ¶µQDWXUDO¶µIUHH¶HWF
EHFDXVHWKH\PD\HOLFLWDQHPRWLRQDOUHDFWLRQWKDWPD\ELDVWKHDQVZHU
,WLVDOVRREYLRXVWKDWORDGHGTXHVWLRQVVXFKDVµ,VQ¶WLWUHDVRQDEOHWRVXSSRVH
WKDW«"¶RUµ'RQ¶W\RXEHOLHYHWKDW«"¶DUHOLNHO\WRELDVWKHUHVSRQGHQW
WRZDUGVJLYLQJDGHVLUHGDQVZHUDQGVKRXOGEHUHSKUDVHGLQDQHXWUDOZD\
$YRLGQHJDWLYHFRQVWUXFWLRQV,WHPVWKDWFRQWDLQDQHJDWLYHFRQVWUXFWLRQ LHLQFOXGLQJ
µQR¶RUµQRW¶ DUHGHFHSWLYHEHFDXVHDOWKRXJKWKH\UHDGVDWLVIDFWRULO\UHVSRQGLQJWR
WKHPFDQEHSUREOHPDWLF)RUH[DPSOHZKDWGRHVDQHJDWLYHUHVSRQVHWRDQHJDWLYH
TXHVWLRQPHDQ",QRUGHUWRDYRLGDQ\SRVVLEOHGLIILFXOWLHVWKHEHVWVROXWLRQLVWR
DYRLGWKHXVHRIQHJDWLYHVDOWRJHWKHU,QPRVWFDVHVQHJDWLYHLWHPVFDQEHUHVWDWHG
LQDSRVLWLYHZD\E\XVLQJYHUEVRUDGMHFWLYHVWKDWH[SUHVVWKHRSSRVLWHPHDQLQJ
IRUH[DPSOHµGLVOLNH¶LQVWHDGRIµQRWOLNH¶
$YRLGGRXEOHEDUUHOOHGTXHVWLRQVµ'RXEOHEDUUHOOHG¶TXHVWLRQVDUHWKRVHWKDWDVNWZR
RUPRUH TXHVWLRQVLQRQHZKLOHH[SHFWLQJDVLQJOHDQVZHU)RUH[DPSOHWKH
TXHVWLRQµ+RZDUH\RXUSDUHQWV"¶DVNVDERXWRQH¶VPRWKHUDQGIDWKHUDQGFDQQRW
EHDQVZHUHGVLPSO\LIRQHRIWKHPLVZHOODQGWKHRWKHUXQZHOO,QGHHGTXHVWLRQV
GHDOLQJZLWKSOXUDOLVPV FKLOGUHQVWXGHQWV RIWHQ\LHOGGRXEOHEDUUHOOHGTXHVWLRQV
EXWFRPSRXQGTXHVWLRQVDOVRRIWHQIDOOLQWRWKLVFDWHJRU\ IRUH[DPSOHµ'R\RX
DOZD\VZULWH\RXUKRPHZRUNDQGGRLWWKRURXJKO\"¶ :LWKGRXEOHEDUUHOOHG
TXHVWLRQVHYHQLIUHVSRQGHQWVGRSURYLGHDQDQVZHUWKHUHLVQRZD\RINQRZLQJ
ZKLFKSDUWRIWKHTXHVWLRQWKHDQVZHUUHVSRQGHGWR
$YRLGLWHPVWKDWDUHOLNHO\WREHDQVZHUHGWKHVDPHZD\E\HYHU\ERG\,QUDWLQJVFDOHV
ZHVKRXOGDYRLGVWDWHPHQWVWKDWDUHOLNHO\WREHHQGRUVHGE\DOPRVWHYHU\RQHRU
DOPRVWQRRQH,QPRVWFDVHVWKHVHLWHPVDUHQRWLQIRUPDWLYHDQGWKH\DUHFHUWDLQO\
GLIILFXOWLIQRWLPSRVVLEOHWRSURFHVVVWDWLVWLFDOO\ EHFDXVHRILQVXIILFLHQWYDULDQFH
,QFOXGHERWKSRVLWLYHO\DQGQHJDWLYHO\ZRUGHGLWHPV,QRUGHUWRDYRLGDUHVSRQVHVHWLQ
ZKLFKWKHUHVSRQGHQWVPDUNRQO\RQHVLGHRIDUDWLQJVFDOHLWLVZRUWKLQFOXGLQJLQ
WKHTXHVWLRQQDLUHERWKSRVLWLYHO\DQGQHJDWLYHO\ZRUGHGLWHPV:HVKRXOGQRWH
KRZHYHUWKDWLWLVDOOWRRHDV\WRIDOOLQWRWKHWUDSRIWU\LQJWRFUHDWHDQHJDWLYHO\
ZRUGHGLWHPE\XVLQJVRPHVRUWRIDQHJDWLYHFRQVWUXFWLRQ IRUH[DPSOHµGRQ¶W¶
ZKLFKKDVEHHQSUHYLRXVO\ZDUQHGDJDLQVW,QVWHDGQHJDWLYHO\ZRUGHGLWHPVVKRXOG
IRFXVRQQHJDWLYHUDWKHUWKDQSRVLWLYHDVSHFWVRIWKHWDUJHW IRUH[DPSOHLQVWHDGRI
ZULWLQJµ,GRQ¶WHQMR\OHDUQLQJ(QJOLVK¶ZHFDQZULWHµ/HDUQLQJ(QJOLVKLVDEXUGHQ
IRUPH¶
7KHIRUPDWRIWKHTXHVWLRQQDLUH

0DLQSDUWV
4XHVWLRQQDLUHVKDYHDIDLUO\VWDQGDUGFRPSRQHQWVWUXFWXUHWKDWFRQVLVWVRIWKHIROORZLQJ
HOHPHQWV
x 7LWOH-XVWOLNHDQ\RWKHUSLHFHRIZULWLQJDTXHVWLRQQDLUHVKRXOGKDYHDWLWOHWR
LGHQWLI\WKHGRPDLQRIWKHLQYHVWLJDWLRQWRSURYLGHWKHUHVSRQGHQWZLWKLQLWLDO
RULHQWDWLRQDQGWRDFWLYDWHUHOHYDQWEDFNJURXQGNQRZOHGJHDQGFRQWHQW
H[SHFWDWLRQV
x *HQHUDOLQWURGXFWLRQ7KHµRSHQLQJJUHHWLQJ¶XVXDOO\GHVFULEHVWKHSXUSRVHRIWKH
VWXG\DQGWKHRUJDQL]DWLRQFRQGXFWLQJVSRQVRULQJLW)XUWKHULPSRUWDQWIXQFWLRQVRI
WKLVVHFWLRQLQYROYHHPSKDVL]LQJWKDWWKHUHDUHQRULJKWRUZURQJDQVZHUV
SURPLVLQJFRQILGHQWLDOLW\RUDQRQ\PLW\DQGUHTXHVWLQJKRQHVWDQVZHUVDQGVD\LQJ
µWKDQN\RX¶
x 6SHFLILFLQVWUXFWLRQV7KHVHH[SODLQDQGGHPRQVWUDWH ZLWKH[DPSOHV KRZ
UHVSRQGHQWVVKRXOGJRDERXWDQVZHULQJWKHTXHVWLRQV
x 4XHVWLRQQDLUHLWHPV7KHVHFRQVWLWXWHWKHPDLQERG\RIWKHTXHVWLRQQDLUH7KH\
QHHGWREHYHU\FOHDUO\VHSDUDWHGIURPWKHLQVWUXFWLRQV7KLVLVZKHUHGLIIHUHQW
W\SHIDFHVDQGIRQWVW\OHVDUHXVHIXO
x $GGLWLRQDOLQIRUPDWLRQ$WWKHHQGRIWKHTXHVWLRQQDLUHZHPD\LQFOXGHDFRQWDFW
QDPH IRUH[DPSOHWKHUHVHDUFKHU¶VRUDQDGPLQLVWUDWRU¶V ZLWKDWHOHSKRQHQXPEHU
RUDGGUHVVDQGVRPHH[SOLFLWHQFRXUDJHPHQWWRJHWLQWRXFKLIWKHUHDUHDQ\
TXHVWLRQV,WLVDQLFHJHVWXUH XQIRUWXQDWHO\WRRUDUHO\XVHG WRLQFOXGHDEULHIQRWH
SURPLVLQJWRVHQGWKHUHVSRQGHQWDVXPPDU\RIWKHILQGLQJVLILQWHUHVWHG
6RPHWLPHVTXHVWLRQQDLUHVFDQDOVRHQGZLWKDQLQYLWDWLRQWRYROXQWHHUIRUDIROORZ
XSLQWHUYLHZ
x )LQDOµWKDQN\RX¶,WLVVXUSULVLQJKRZRIWHQWKLVLVRPLWWHG
/HQJWK
+RZORQJLVWKHRSWLPDOOHQJWKRIDTXHVWLRQQDLUH",WGHSHQGVRQKRZLPSRUWDQWWKH
WRSLFRIWKHTXHVWLRQQDLUHLVIRUWKHUHVSRQGHQW,IZHIHHOYHU\VWURQJO\DERXW
VRPHWKLQJZHDUHXVXDOO\ZLOOLQJWRVSHQGVHYHUDOKRXUVDQVZHULQJTXHVWLRQV
+RZHYHUPRVWTXHVWLRQQDLUHVLQDSSOLHGOLQJXLVWLFVFRQFHUQWRSLFVWKDWKDYHDORZ
VDOLHQFHIURPWKHUHVSRQGHQWV¶SHUVSHFWLYHDQGLQVXFKFDVHVWKHRSWLPDOOHQJWKLV
UDWKHUVKRUW0RVWUHVHDUFKHUVDJUHHWKDWDQ\WKLQJWKDWLVPRUHWKDQ±SDJHVORQJDQG
UHTXLUHVRYHUKDOIDQKRXUWRFRPSOHWHPD\EHFRQVLGHUHGWRRPXFKRIDQLPSRVLWLRQ
$VDSULQFLSOH,KDYHDOZD\VWULHGWRVWD\ZLWKLQDIRXUSDJHOLPLW,WLVUHPDUNDEOHKRZ
PDQ\LWHPVFDQEHLQFOXGHGZLWKLQIRXUZHOOGHVLJQHGSDJHVDQG,KDYHDOVRIRXQGWKDW
DTXHVWLRQQDLUHRISDJHVGRHVQRWWHQGWRH[FHHGWKHPLQXWHFRPSOHWLRQOLPLW
/D\RXW

,WLVP\H[SHULHQFHWKDWWKHOD\RXWRIWKHTXHVWLRQQDLUHLVIUHTXHQWO\RYHUORRNHGDVDQ
LPSRUWDQWDVSHFWRIWKHGHYHORSPHQWRIWKHLQVWUXPHQW7KLVLVDPLVWDNHRYHUWKHSDVW
\HDUV,KDYHLQFUHDVLQJO\REVHUYHGWKDWSURGXFLQJDQDWWUDFWLYHDQGSURIHVVLRQDO
GHVLJQLVKDOIWKHEDWWOHLQPRWLYDWLQJUHVSRQGHQWVWRSURGXFHUHOLDEOHDQGYDOLGGDWD
$IWHUDOODVZLOOEHGLVFXVVHGLQPRUHGHWDLOLQ6HFWLRQ RQDGPLQLVWHULQJWKH
TXHVWLRQQDLUH SHRSOHXVXDOO\GRQRWPLQGH[SUHVVLQJWKHLURSLQLRQVDQGDQVZHULQJ
TXHVWLRQVDVORQJDVWKH\WKLQNWKDWWKHVXUYH\LVVHULRXVDQGSURIHVVLRQDOO\FRQGXFWHG
7KUHHSRLQWVLQSDUWLFXODUDUHZRUWKEHDULQJLQPLQG
x %RRNOHWIRUPDW1RWRQO\GRHVWKHTXHVWLRQQDLUHKDYHWREHVKRUWEXWLWDOVRKDVWR
ORRNVKRUW,KDYHIRXQGWKDWWKHIRUPDWWKDWIHHOVPRVWFRPSDFWLVWKDWRIDERRNOHW
IRUH[DPSOHDQ$VKHHWIROGHGLQWRWZRWRPDNHDIRXUSDJH$ERRNOHW 7KLV
IRUPDWDOVRPDNHVLWHDV\WRUHDGDQGWRWXUQSDJHV DQGZKDWLVMXVWDVLPSRUWDQWLW
DOVRSUHYHQWVORVWSDJHV«
x $SSURSULDWHGHQVLW\:LWKUHJDUGWRKRZPXFKPDWHULDOZHSXWRQDSDJHD
FRPSURPLVHQHHGVWREHDFKLHYHG2QWKHRQHKDQGZHZDQWWRPDNHWKHSDJHVIXOO
EHFDXVHUHVSRQGHQWVDUHPXFKPRUHZLOOLQJWRILOOLQDWZRSDJHUDWKHUWKDQDIRXU
SDJHTXHVWLRQQDLUHHYHQLIWKHWZRLQVWUXPHQWVKDYHH[DFWO\WKHVDPHQXPEHURI
LWHPV2QWKHRWKHUKDQGZHPXVWQRWPDNHWKHSDJHVORRNFURZGHG IRUH[DPSOH
E\HFRQRPL]LQJRQWKHVSDFHVVHSDUDWLQJGLIIHUHQWVHFWLRQVRIWKHTXHVWLRQQDLUH
(IIHFWLYHZD\VRIDFKLHYLQJWKLVWUDGHRIILQYROYHUHGXFLQJWKHPDUJLQVXVLQJD
VSDFHHFRQRPLFDOIRQWDQGXWLOL]LQJWKHZKROHZLGWKRIWKHSDJHIRUH[DPSOHE\
SULQWLQJWKHUHVSRQVHRSWLRQVQH[WWRHDFKTXHVWLRQUDWKHUWKDQEHORZLW
x 6HTXHQFHPDUNLQJ,QRUPDOO\PDUNHDFKPDLQVHFWLRQRIWKHTXHVWLRQQDLUHZLWK
5RPDQQXPEHUVHDFKTXHVWLRQZLWKFRQVHFXWLYH$UDEILJXUHVDQGWKHQOHWWHUDOO
WKHVXESDUWVRIDTXHVWLRQDVDUHVXOW,PD\KDYH4XHVWLRQDRUGZLWKLQ
6HFWLRQ,RU,,,7KLVFUHDWHVDVHQVHRIVWUXFWXUHGQHVV,WLVDOVREHQHILFLDOWR
LQFOXGHDSKUDVHVXFKDVµ&RQWLQXHGRQEDFN¶DWWKHERWWRPRIWKHILUVWVLGHRID
VKHHWWKDWLVSULQWHGRQERWKVLGHV
,WHPVHTXHQFH
2QFHDOOWKHLWHPVWREHLQFOXGHGLQWKHTXHVWLRQQDLUHKDYHEHHQZULWWHQRUFROOHFWHG
ZHQHHGWRGHFLGHRQWKHLURUGHU,WHPVHTXHQFHLVDVLJQLILFDQWIDFWRUEHFDXVHWKH
FRQWH[WRIDTXHVWLRQFDQKDYHDQLPSDFWRQLWVLQWHUSUHWDWLRQDQGWKHUHVSRQVHJLYHQWR
LW7KHUHDUHIRXUSULQFLSOHVLQSDUWLFXODUWKDWZHQHHGWREHDULQPLQG
0L[LQJXSWKHVFDOHV7KHLWHPVIURPGLIIHUHQWVFDOHVQHHGWREHPL[HGXSDVPXFKDV
SRVVLEOHWRFUHDWHDVHQVHRIYDULHW\DQGWRSUHYHQWUHVSRQGHQWVIURPVLPSO\
UHSHDWLQJSUHYLRXVDQVZHUV
2SHQLQJTXHVWLRQV7RVHWWKHULJKWWRQHWKHRSHQLQJTXHVWLRQVQHHGWREHLQWHUHVWLQJ
UHODWLYHO\VLPSOH\HWDWWKHVDPHWLPHIRFXVHGRQLPSRUWDQWDQGVDOLHQWDVSHFWV:H
QHHGWREHFDUHIXOQRWWRIRUFHWKHUHVSRQGHQWVWRWDNHDIXQGDPHQWDOGHFLVLRQDW

VXFKDQHDUO\VWDJHEHFDXVHWKDWZRXOGDIIHFWDOOWKHVXEVHTXHQWDQVZHUV6RWKH
LQLWLDOTXHVWLRQVQHHGWREHUHODWLYHO\PLOGRUQHXWUDO
)DFWXDO RUµSHUVRQDO¶RUµFODVVLILFDWLRQ¶ TXHVWLRQV$V2SSHQKHLP FRQFOXGHV

QRYLFHUHVHDUFKHUVW\SLFDOO\VWDUWWRGHVLJQDTXHVWLRQQDLUHE\SXWWLQJDUDWKHU
IRUELGGLQJVHWRITXHVWLRQVDWWKHWRSRIDEODQNVKHHWRISDSHUDVNLQJIRUQDPH
DGGUHVVPDULWDOVWDWXVQXPEHURIFKLOGUHQUHOLJLRQDQGVRRQ7KHVH
SHUVRQDOFODVVLILFDWLRQTXHVWLRQVUHVHPEOHWKHPDQ\EXUHDXFUDWLFIRUPVZHKDYHWR
ILOOLQDQGDUHEHVWOHIWDWWKHHQGRIWKHTXHVWLRQQDLUH$OVRLQPDQ\FXOWXUHVLVVXHV
OLNHDJHOHYHORIHGXFDWLRQRUPDULWDOVWDWXVDUHFRQVLGHUHGSHUVRQDODQGSULYDWH
PDWWHUVDQGLIZHDVNWKHPQHDUWKHEHJLQQLQJRIWKHTXHVWLRQQDLUHWKH\PLJKW
FUHDWHVRPHUHVLVWDQFHLQWKHUHVSRQGHQWVRULQFDVHVZKHUHUHVSRQGHQWVDUHDVNHG
WRSURYLGHWKHLUQDPHWKLVPLJKWUHPLQGWKHPRIWKHQRQDQRQ\PRXVQDWXUHRIWKH
VXUYH\ZKLFKLQWXUQPD\LQKLELWVRPHRIWKHLUDQVZHUV
2SHQHQGHGTXHVWLRQVDWWKHHQG,IZHLQFOXGHUHDORSHQHQGHGTXHVWLRQVWKDWUHTXLUH
VXEVWDQWLDODQGFUHDWLYHZULWLQJLWLVSUHIHUDEOHWRSODFHWKHPQHDUWKHHQGUDWKHU
WKDQDWWKHEHJLQQLQJRIWKHTXHVWLRQQDLUH,QWKLVZD\WKHRWKHULWHPVZLOOQRWEH
DIIHFWHGE\WKHSRWHQWLDOQHJDWLYHFRQVHTXHQFHVRIWKHRSHQHQGHGTXHVWLRQ IRU
H[DPSOHWKHUHTXLUHGZRUNFDQSXWVRPHSHRSOHRIIRWKHUVPLJKWJHWERJJHGGRZQ
DQGVSHQGPRVWRIWKHDYDLODEOHWLPHDQGPHQWDOHQHUJ\DJRQL]LQJRYHUZKDWWKH\
VKRXOGZULWH ,QDGGLWLRQVRPHSHRSOHILQGLWSV\FKRORJLFDOO\PRUHDFFHSWDEOHWR
SXWLQWKHQHFHVVDU\ZRUNLIWKH\KDYHDOUHDG\LQYHVWHGLQWKHTXHVWLRQQDLUHDQGLI
WKH\NQRZWKDWWKLVLVWKHILQDOWDVN
'HYHORSLQJDQGSLORWLQJWKHTXHVWLRQQDLUH
7KHJHQHUDOLPSRUWDQFHRISLORWLQJUHVHDUFKWRROVDQGSURFHGXUHVZDVDOUHDG\
KLJKOLJKWHGLQ6HFWLRQ%HFDXVHDVZHKDYHVHHQHDUOLHULQTXHVWLRQQDLUHVVR
PXFKGHSHQGVRQWKHDFWXDOZRUGLQJRIWKHLWHPVDQLQWHJUDOSDUWRITXHVWLRQQDLUH
FRQVWUXFWLRQLVµILHOGWHVWLQJ¶WKDWLVSLORWLQJWKHTXHVWLRQQDLUHDWYDULRXVVWDJHVRILWV
GHYHORSPHQWRQDVDPSOHRISHRSOHZKRDUHVLPLODUWRWKHWDUJHWVDPSOHIRUZKLFKWKH
LQVWUXPHQWKDVEHHQGHVLJQHG7KHGHYHORSLQJDQGSLORWLQJRIDTXHVWLRQQDLUHLVD
VWHSZLVHSURFHVV
'UDZLQJXSDQLWHPSRRO7KHILUVWVWHSLVWROHWRXULPDJLQDWLRQJRIUHHDQGFUHDWHDV
PDQ\SRWHQWLDOLWHPVIRUHDFKVFDOHDVZHFDQWKLQNRI²WKLVFROOHFWLRQLVUHIHUUHGWR
DVWKHµLWHPSRRO¶,QGRLQJVRZHFDQGUDZLQVSLUDWLRQLGHDVIURPWZRVRXUFHV D
TXDOLWDWLYHH[SORUDWRU\GDWDJDWKHUHGLQLQWHUYLHZV RQHWRRQHRUIRFXVJURXS RU
VWXGHQWHVVD\VIRFXVLQJRQWKHFRQWHQWRIWKHTXHVWLRQQDLUHDQG E
HVWDEOLVKHGSXEOLVKHGTXHVWLRQQDLUHVLQWKHDUHD ERUURZLQJTXHVWLRQQDLUHLWHPVLV
DQDFFHSWDEOHSUDFWLFHLIWKHVRXUFHVDUHSURSHUO\DFNQRZOHGJHG
,QLWLDOSLORWLQJRIWKHLWHPSRRO,QRUGHUWRUHGXFHWKHODUJHOLVWRITXHVWLRQVLQWKHLWHP
SRROWRWKHLQWHQGHGILQDOQXPEHULWLVXVHIXOWRILUVWDVN±WUXVWHGDQGKHOSIXO
FROOHDJXHVRUIULHQGVWRJRWKURXJKWKHLWHPVDQGSURYLGHIHHGEDFN

)LQDOSLORWLQJ GUHVVUHKHDUVDO %DVHGRQWKHIHHGEDFNUHFHLYHGIURPWKHLQLWLDOSLORW
JURXSZHFDQQRUPDOO\SXWWRJHWKHUDQHDUILQDOYHUVLRQRIWKHTXHVWLRQQDLUHWKDW
µIHHOV¶VDWLVIDFWRU\DQGWKDWGRHVQRWKDYHDQ\REYLRXVJOLWFKHV+RZHYHUZHVWLOO
GRQRWNQRZKRZWKHLWHPVZLOOZRUNLQDFWXDOSUDFWLFHWKDWLVZKHWKHUWKH
UHVSRQGHQWVZLOOUHSO\WRWKHLWHPVLQWKHPDQQHULQWHQGHGE\WKHTXHVWLRQQDLUH
GHVLJQHUV7KHUHLVRQO\RQHZD\WRILQGRXWE\DGPLQLVWHULQJWKHTXHVWLRQQDLUHWR
DJURXSRIDERXWUHVSRQGHQWVZKRDUHLQHYHU\ZD\VLPLODUWRWKHWDUJHW
SRSXODWLRQWKHLQVWUXPHQWZDVGHVLJQHGIRU
,WHPDQDO\VLV7KHDQVZHUVRIWKHSLORWJURXSDUHVXEPLWWHGWRVWDWLVWLFDODQDO\VHVWR
ILQHWXQHDQGILQDOL]HWKHTXHVWLRQQDLUH7KHSURFHGXUHXVXDOO\LQYROYHVFKHFNLQJ
WKUHHDVSHFWVRIWKHUHVSRQVHSDWWHUQ D PLVVLQJUHVSRQVHVDQGSRVVLEOHVLJQVWKDW
WKHLQVWUXFWLRQVZHUHQRWXQGHUVWRRGFRUUHFWO\ E WKHUDQJHRIWKHUHVSRQVHV
HOLFLWHGE\HDFKLWHP:HVKRXOGH[FOXGHLWHPVWKDWDUHHQGRUVHGE\DOPRVW
HYHU\RQHRUE\DOPRVWQRRQHEHFDXVHWKH\DUHGLIILFXOWLIQRWLPSRVVLEOHWRSURFHVV
VWDWLVWLFDOO\ VLQFHVWDWLVWLFDOSURFHGXUHVUHTXLUHDFHUWDLQDPRXQWRIYDULDWLRQLQWKH
VFRUHV DQG F WKHLQWHUQDOFRQVLVWHQF\RIPXOWLLWHPVFDOHV0XOWLLWHPVFDOHVDUH
RQO\HIIHFWLYHLIWKHLWHPVZLWKLQDVFDOHZRUNWRJHWKHULQDKRPRJHQHRXVPDQQHU
WKDWLVLIWKH\PHDVXUHWKHVDPHWDUJHWDUHD,QSV\FKRPHWULFWHUPVWKLVPHDQVWKDW
HDFKLWHPRQDVFDOHVKRXOGFRUUHODWHZLWKWKHRWKHULWHPVDQGZLWKWKHWRWDOVFDOH
VFRUHZKLFKKDVEHHQUHIHUUHGWRDV/LNHUW¶VFULWHULRQRIµLQWHUQDOFRQVLVWHQF\¶
6WDWLVWLFDOSDFNDJHVOLNH6366 VHH6HFWLRQ RIIHUDYHU\XVHIXOSURFHGXUH
µUHOLDELOLW\DQDO\VLV¶ZKLFKSURYLGHVDVWUDLJKWIRUZDUGWHFKQLTXHWRH[FOXGHLWHPV
WKDWGRQRWZRUN VHH6HFWLRQ DQGWRVHOHFWWKHEHVWLWHPVXSWRWKH
SUHGHWHUPLQHGOHQJWKRIWKHLQVWUXPHQW
3RVWKRFLWHPDQDO\VLV$IWHUWKHDGPLQLVWUDWLRQRIWKHILQDOTXHVWLRQQDLUHUHVHDUFKHUV
XVXDOO\FRQGXFWDILQDOLWHPDQDO\VLVIROORZLQJWKHVDPHSURFHGXUHVDVGHVFULEHG
DERYHWRVFUHHQRXWDQ\LWHPVWKDWKDYHQRWZRUNHGSURSHUO\
$GPLQLVWHULQJWKHTXHVWLRQQDLUH
2QHDUHDLQZKLFKDTXHVWLRQQDLUHVWXG\FDQJRYHU\ZURQJFRQFHUQVWKHSURFHGXUHV
XVHGWRDGPLQLVWHUWKHTXHVWLRQQDLUH6WUDQJHO\HQRXJKWKLVDVSHFWRIVXUYH\UHVHDUFK
KDVKDUGO\HYHUEHHQGLVFXVVHGLQWKH/OLWHUDWXUH²TXHVWLRQQDLUHDGPLQLVWUDWLRQLV
RIWHQFRQVLGHUHGDPHUHWHFKQLFDOLVVXHUHOHJDWHGWRWKHGLVFUHWLRQRIWKHUHVHDUFK
DVVLVWDQWV7KLVLVPLVWDNHQWKHUHLVDPSOHHYLGHQFHLQWKHPHDVXUHPHQWOLWHUDWXUHWKDW
TXHVWLRQQDLUHDGPLQLVWUDWLRQSURFHGXUHVSOD\DVLJQLILFDQWUROHLQDIIHFWLQJWKHTXDOLW\
RIWKHHOLFLWHGUHVSRQVHV
,QVRFLDOUHVHDUFKWKHPRVWFRPPRQIRUPRIDGPLQLVWHULQJTXHVWLRQQDLUHVLVE\PDLO
EXWHGXFDWLRQDOUHVHDUFKLVGLIIHUHQWLQWKLVUHVSHFWEHFDXVHDGPLQLVWUDWLRQE\KDQGLV
W\SLFDOO\PRUHSURPLQHQWWKDQSRVWDOVXUYH\V,QDSSOLHGOLQJXLVWLFUHVHDUFKJURXS
DGPLQLVWUDWLRQLVWKHPRVWFRPPRQPHWKRGRIKDYLQJTXHVWLRQQDLUHVFRPSOHWHGSDUWO\
EHFDXVHWKHW\SLFDOWDUJHWVRIWKHVXUYH\VDUHODQJXDJHOHDUQHUVVWXG\LQJZLWKLQ

LQVWLWXWLRQDOFRQWH[WVDQGLWLVRIWHQSRVVLEOHWRDUUDQJHWRDGPLQLVWHUWKHLQVWUXPHQWWR
WKHPZKLOHWKH\DUHDVVHPEOHGWRJHWKHUIRUH[DPSOHDVSDUWRIDOHVVRQ,QWKLVIRUPDW
DIHZTXHVWLRQQDLUHDGPLQLVWUDWRUVFDQFROOHFWDYHU\ODUJHQXPEHURIGDWDZLWKLQD
UHODWLYHO\VKRUWWLPH
:K\ZRXOGWKHUHVSRQGHQWVWDNHWKHVXUYH\VHULRXVO\ZKHQWKH\KDYHXVXDOO\
QRWKLQJWRJDLQIURPSDUWLFLSDWLQJLQWKHUHVHDUFK"6FKRROFKLOGUHQDUHRIWHQZLOOLQJWR
ZRUNKDUGRQDWDVNVLPSO\EHFDXVHLWLVDVVLJQHGWRWKHPDQGLQP\H[SHULHQFHSHRSOH
LQJHQHUDOGRQRWPLQGH[SUHVVLQJWKHLURSLQLRQVDQGDQVZHULQJTXHVWLRQVDVORQJDV
WKH\WKLQNWKDWWKHVXUYH\LVUHODWHGWRDZRUWK\FDXVHDQGWKDWWKHLURSLQLRQPDWWHUV
7KXVLIZHWDNHVXIILFLHQWFDUHSODQQLQJDQGH[HFXWLQJWKHDGPLQLVWUDWLRQSURFHVVZH
FDQVXFFHVVIXOO\EXLOGRQWKLVKXPDQFKDUDFWHULVWLFDQGFDQVHFXUHWKHFRRSHUDWLRQRI
RXULQIRUPDQWV7KHIROORZLQJVWUDWHJLHVKDYHEHHQIRXQGHIIHFWLYHLQDFKLHYLQJWKLV
REMHFWLYH
$GYDQFHQRWLFH$QQRXQFLQJWKHTXHVWLRQQDLUHDIHZGD\VLQDGYDQFHDQGVHQGLQJHDFK
SDUWLFLSDQWDSULQWHGOHDIOHWWKDWLQYLWHVWKHLUSDUWLFLSDWLRQH[SODLQVWKHSXUSRVHDQG
QDWXUHRIWKHTXHVWLRQQDLUHDQGRIIHUVVRPHVDPSOHLWHPVLVDQHIIHFWLYHPHWKRGRI
JHQHUDWLQJDSRVLWLYHFOLPDWHIRUWKHDGPLQLVWUDWLRQDQGRIUDLVLQJWKHµSURIHVVLRQDO¶
IHHORIWKHVXUYH\
$WWLWXGHVFRQYH\HGE\WHDFKHUVSDUHQWVDQGRWKHUDXWKRULW\ILJXUHV3DUWLFLSDQWVDUH
TXLFNWRSLFNXSWKHLUVXSHULRUV¶ IRUH[DPSOHWHDFKHUV¶RUERVVHV¶ RUSDUHQWV¶
DWWLWXGHWRZDUGVWKHVXUYH\DQGRQO\DFTXLHVFHLIWKHPHVVDJHWKH\UHFHLYHLV
SRVLWLYH,WLVWKHUHIRUHDQLPSHUDWLYHWRZLQWKHVXSSRUWRIDOOWKHVHDXWKRULW\
ILJXUHVLQDGYDQFH
5HVSHFWDEOHVSRQVRUVKLS,IZHUHSUHVHQWDQRUJDQL]DWLRQWKDWLVHVWHHPHGKLJKO\E\WKH
UHVSRQGHQWVWKHSRVLWLYHUHSXWDWLRQLVOLNHO\WREHSURMHFWHGRQWRWKHVXUYH\
7KHEHKDYLRXURIWKHVXUYH\DGPLQLVWUDWRU7KHDGPLQLVWUDWRUVRIWKHTXHVWLRQQDLUHDUH
LQPDQ\ZD\VLGHQWLILHGZLWKWKHZKROHVXUYH\DQGWKHUHIRUHHYHU\WKLQJDERXW
WKHPPDWWHUVWKHLUFORWKHVVKRXOGEHEXVLQHVVOLNHEXWFHUWDLQO\QRWPRUHIRUPDO
WKDQZKDWLVW\SLFDOLQWKHJLYHQHQYLURQPHQWWKHZD\WKH\LQWURGXFHWKHPVHOYHVLV
LPSRUWDQWIULHQGOLQHVVLVLPSHUDWLYHDQGVPLOLQJXVXDOO\EUHDNVWKHLFHHIIHFWLYHO\
WKHLURYHUDOOFRQGXFWVKRXOGEHSURIHVVLRQDOWRUHSUHVHQWWKHVHULRXVFKDUDFWHURIWKH
VXUYH\ZLWKRXWEHLQJVWLIIDQGXQQDWXUDO
$GPLQLVWUDWRUDWWLWXGHV$FUXFLDODVSHFWRIWKHVXUYH\DGPLQLVWUDWRUV¶EHKDYLRXULVWKDW
LWVKRXOGH[KLELWNHHQLQYROYHPHQWLQWKHSURMHFWDQGVKRZDQREYLRXVLQWHUHVWLQ
WKHRXWFRPH
&RPPXQLFDWLQJWKHSXUSRVHDQGVLJQLILFDQFHRIWKHVXUYH\$QLPSRUWDQWHOHPHQWLQ
VHOOLQJWKHVXUYH\WRWKHSDUWLFLSDQWVLVFRPPXQLFDWLQJWRWKHPWKHSXUSRVHRIWKH
VXUYH\DQGFRQYH\LQJWRWKHPWKHSRWHQWLDOVLJQLILFDQFHRIWKHUHVXOWV7KH
LQWURGXFWRU\VSHHFKRIWKHTXHVWLRQQDLUHDGPLQLVWUDWRUQHHGVWREHFDUHIXOO\
GHVLJQHGWRFRYHUWKHIROORZLQJSRLQWVJUHHWLQJDQGWKDQNLQJWKHSXUSRVHRIWKH

VXUYH\DQGLWVSRWHQWLDOXVHIXOQHVVZK\WKHSDUWLFXODUSDUWLFLSDQWVKDYHEHHQ
VHOHFWHGDVVXUDQFHRIFRQILGHQWLDOLW\DQRQ\PLW\WKHGXUDWLRQRIFRPSOHWLQJWKH
TXHVWLRQQDLUHµ$Q\TXHVWLRQV"¶ILQDOWKDQN\RX
6WUHQJWKVDQGZHDNQHVVHVRITXHVWLRQQDLUHV
7KHPDLQDWWUDFWLRQRITXHVWLRQQDLUHVLVWKHLUHIILFLHQF\LQWHUPVRIUHVHDUFKHUWLPHDQG
HIIRUWDQGILQDQFLDOUHVRXUFHV%\DGPLQLVWHULQJDTXHVWLRQQDLUHWRDJURXSRISHRSOH
RQHFDQFROOHFWDKXJHDPRXQWRILQIRUPDWLRQLQOHVVWKDQDQKRXUDQGWKHSHUVRQDO
LQYHVWPHQWUHTXLUHGZLOOEHDIUDFWLRQRIZKDWZRXOGKDYHEHHQQHHGHGIRUVD\
LQWHUYLHZLQJWKHVDPHQXPEHURISHRSOH)XUWKHUPRUHLIWKHTXHVWLRQQDLUHLVZHOO
FRQVWUXFWHGSURFHVVLQJWKHGDWDFDQDOVREHIDVWDQGUHODWLYHO\VWUDLJKWIRUZDUG
HVSHFLDOO\E\XVLQJVRPHFRPSXWHUVRIWZDUH4XHVWLRQQDLUHVDUHDOVRYHU\YHUVDWLOH
ZKLFKPHDQVWKDWWKH\FDQEHXVHGVXFFHVVIXOO\ZLWKDYDULHW\RISHRSOHLQDYDULHW\RI
VLWXDWLRQVWDUJHWLQJDYDULHW\RIWRSLFV5HVSRQGHQWVXVXDOO\GRQRWPLQGWKHSURFHVVRI
ILOOLQJLQTXHVWLRQQDLUHVDQGWKHVXUYH\PHWKRGFDQRIIHUWKHPDQRQ\PLW\LIQHHGHG$V
DUHVXOWRIWKHVHVWUHQJWKVWKHYDVWPDMRULW\RIUHVHDUFKSURMHFWVLQWKHEHKDYLRXUDODQG
VRFLDOVFLHQFHVLQYROYHDWRQHVWDJHRUDQRWKHUFROOHFWLQJVRPHVRUWRITXHVWLRQQDLUH
GDWD
2QWKHRWKHUKDQGTXHVWLRQQDLUHVDOVRKDYHVRPHVHULRXVOLPLWDWLRQVDQGLWLVDOOWRR
HDV\WRSURGXFHXQUHOLDEOHDQGLQYDOLGGDWDE\PHDQVRIDQLOOFRQVWUXFWHG
TXHVWLRQQDLUH,QIDFWDV*LOOKDP SRLQWVRXWLQUHVHDUFKPHWKRGRORJ\µQR
VLQJOHPHWKRGKDVEHHQVRPXFKDEXVHG¶7KHZHDNHVWDVSHFWRITXHVWLRQQDLUHVLVWKH
IDFWWKDWWKHLWHPVQHHGWREHVXIILFLHQWO\VLPSOHDQGVWUDLJKWIRUZDUGWREHXQGHUVWRRG
E\HYHU\ERG\7KXVWKLVPHWKRGLVXQVXLWDEOHIRUSURELQJGHHSO\LQWRDQLVVXH 0RVHU
DQG.DOWRQ DQGLWXVXDOO\UHVXOWVLQUDWKHUVXSHUILFLDOGDWD7KHQHFHVVDU\
VLPSOLFLW\RIWKHTXHVWLRQVLVIXUWKHUDXJPHQWHGE\WKHIDFWWKDWWKHDPRXQWRIWLPH
UHVSRQGHQWVWHQGWREHZLOOLQJWRVSHQGZRUNLQJRQDTXHVWLRQQDLUHLVUDWKHUVKRUW
ZKLFKDJDLQOLPLWVWKHGHSWKRIWKHLQYHVWLJDWLRQ7KXVTXHVWLRQQDLUHVXUYH\VXVXDOO\
SURYLGHDUDWKHUµWKLQ¶GHVFULSWLRQRIWKHWDUJHWSKHQRPHQD2WKHULVVXHVLQFOXGH
SRVVLEOHUHVSRQGHQWOLWHUDF\SUREOHPVDQGWKHVRFLDOGHVLUDELOLW\ELDV²VHH6HFWLRQ

([SHULPHQWDODQGTXDVLH[SHULPHQWDOVWXGLHV
+DYLQJWDONHGDERXWRQHRIWKHPRVWSRSXODUTXDQWLWDWLYHGDWDFROOHFWLRQWRROVWKH
TXHVWLRQQDLUHOHWXVQRZWXUQWRDVSHFLDOTXDQWLWDWLYHGDWDFROOHFWLRQGHVLJQWKH
H[SHULPHQWDOVWXG\ZKLFKPDQ\SHRSOHZRXOGFODLPWRUHSUHVHQWTXDQWLWDWLYHUHVHDUFK
DWLWVPRVWVFLHQWLILFEHFDXVHLWFDQHVWDEOLVKXQDPELJXRXVFDXVH±HIIHFWUHODWLRQVKLSV
0DQ\UHVHDUFKVWXGLHVLQDSSOLHGOLQJXLVWLFVDUHLQWHQGHGWRXQFRYHUFDXVDOOLQNVE\
DQVZHULQJTXHVWLRQVVXFKDVµ:KDW¶VWKHUHDVRQIRU«"¶µ:KDWKDSSHQVLIZKHQ«"¶
DQGµ:KDWHIIHFWGRHVVRPHWKLQJKDYHRQ«"¶+RZHYHUWRHVWDEOLVKILUPFDXVH±HIIHFW
UHODWLRQVKLSVLVVXUSULVLQJO\GLIILFXOWEHFDXVHLQUHDOOLIHQRWKLQJKDSSHQVLQLVRODWLRQ

DQGLWLVKDUGWRGLVHQWDQJOHWKHLQWHUIHUHQFHVRIYDULRXVUHODWHGIDFWRUV)RUH[DPSOHLI
ZHZDQWWRFRPSDUHWKHHIIHFWVRIXVLQJWZRGLIIHUHQWFRXUVHERRNVLQODQJXDJH
WHDFKLQJKRZGRZHPHDVXUHWKHLULPSDFW",IZHVLPSO\FRPSDUHWZROHDUQHUJURXSV
ZKLFKXVHWKHWZRFRXUVHERRNVDQGILQGVRPHPDUNHGGLIIHUHQFHVEHWZHHQWKHLU
OHDUQLQJDFKLHYHPHQWFDQZHUHDOO\FODLPWKDWWKLVGLIIHUHQFHZDVGXHWRWKHGLIIHUHQFH
LQWKHFRXUVHERRNWKH\XVHG"1RWQHFHVVDULO\EHFDXVHWKHUHFDQEHDKRVWRIRWKHU
IDFWRUVFRQWULEXWLQJWRWKHGLIIHUHQFHIRUH[DPSOHWKHFRPSRVLWLRQRIWKHOHDUQHUVLQ
WKHWZRJURXSVRUWKHGLIIHUHQWOHDUQLQJHQYLURQPHQWV DQGSRVVLEO\WKHGLIIHUHQW
WHDFKHUV ,QGHHGLQWKHLUVHPLQDOZRUN&DPSEHOODQG6WDQOH\ SRLQWRXWWKDW
XQOHVVZHDSSO\VSHFLDOUHVHDUFKGHVLJQVZHKDYHQRZD\RIGLVFULPLQDWLQJDPRQJ
QXPHURXVSRVVLEOHDOWHUQDWLYHH[SODQDWLRQVIRUVXFKRXWFRPHV
7KHDQVZHUWRWKHFDXVH±HIIHFWGLOHPPDKDVEHHQSURYLGHGE\DVLPSOHEXW
LQJHQLRXVPHWKRGRORJLFDOLGHDWKDWKDVEHHQODEHOOHGµH[SHULPHQWDOGHVLJQ¶ILUVWWDNH
DJURXSRIOHDUQHUVDQGGRVRPHWKLQJVSHFLDOZLWKWRWKHPZKLOHPHDVXULQJWKHLU
SURJUHVV7KHQFRPSDUHWKHLUUHVXOWVZLWKGDWDREWDLQHGIURPDQRWKHUJURXSWKDWLV
VLPLODULQHYHU\UHVSHFWWRWKHILUVWJURXSH[FHSWIRUWKHIDFWWKDWLWGLGQRWUHFHLYHWKH
VSHFLDOWUHDWPHQW,IWKHUHLVDQ\GLVFUHSDQF\LQWKHUHVXOWVRIWKHWZRJURXSVWKHVHFDQ
EHDWWULEXWHGWRWKHRQO\GLIIHUHQFHEHWZHHQWKHPWKHWUHDWPHQWYDULDEOH7KXVWKH
FRPPRQIHDWXUHRIH[SHULPHQWDOGHVLJQVLVWKHIDFWWKDWFHUWDLQFRQVFLRXVO\
PDQLSXODWHGSURFHVVHVWDNHSODFHLQDWLJKWO\FRQWUROOHGHQYLURQPHQWLQZKLFKRQO\WKH
WDUJHWYDULDEOHVDUHYDULHGZKLOHRWKHUVDUHNHSWFRQVWDQW -RKQVRQDQG&KULVWHQVHQ

$W\SLFDOH[SHULPHQWDOGHVLJQZRXOGEHDQLQWHUYHQWLRQVWXG\ZKLFKFRQWDLQVDW
OHDVWWZRJURXSVWKHµWUHDWPHQW¶RUµH[SHULPHQWDOJURXS¶ZKLFKUHFHLYHVWKHWUHDWPHQW
RUZKLFKLVH[SRVHGWRVRPHVSHFLDOFRQGLWLRQVDQGWKHµFRQWUROJURXS¶ZKRVHUROHLV
WRSURYLGHDEDVHOLQHIRUFRPSDULVRQ)URPDWKHRUHWLFDOSHUVSHFWLYHWKHXOWLPDWH
FKDOOHQJHLVWRILQGDZD\RIPDNLQJWKHFRQWUROJURXSDVVLPLODUWRWKHWUHDWPHQWJURXS
DVSRVVLEOH$V&RRNDQG&DPSEHOO VXPPDUL]HRQHRIWKHJUHDWEUHDNWKURXJKV
LQH[SHULPHQWDOGHVLJQZDVWKHUHDOL]DWLRQWKDWZLWKVXIILFLHQWSDUWLFLSDQWVWKHUDQGRP
DVVLJQPHQWRIWKHSDUWLFLSDQWVWRH[SHULPHQWDODQGFRQWUROJURXSVFDQSURYLGHDZD\RI
PDNLQJWKHDYHUDJHSDUWLFLSDQWLQRQHJURXSFRPSDUDEOHWRWKHDYHUDJHSDUWLFLSDQWLQ
WKHRWKHUJURXSEHIRUHWKHWUHDWPHQWLVDSSOLHG
:KLOHWKHH[SHULPHQWDOJURXSLVUHFHLYLQJWKHWUHDWPHQWWKHFRQWUROJURXSLVDOVR
UHFHLYLQJLQVWUXFWLRQZKLFKLVXVXDOO\WKHVWDQGDUGWKDWLVWKHXQPDUNHGIRUPRI
WXLWLRQLQWKHSDUWLFXODUOHDUQLQJFRQWH[W3URJUHVVLVPHDVXUHGE\DGPLQLVWHULQJSUH
WHVWVEHIRUHWKHLQWHUYHQWLRQDQGSRVWWHVWVDIWHUWKHWUHDWPHQWKDVEHHQFRPSOHWHG7KH
FRPSDULVRQRIWKHWZRJURXSVLQVXFKDµSUHWHVW±SRVWWHVWFRQWUROJURXSGHVLJQ¶LV
FDUULHGRXWE\XVLQJVWDWLVWLFDOSURFHGXUHVWKDWZLOOEHGHVFULEHGLQ6HFWLRQEHORZ
7KXVWKHPDLQGLIIHUHQFHEHWZHHQH[SHULPHQWDOGHVLJQVDQGVXUYH\UHVHDUFKOLHVLQ
WKHIDFWWKDWLQWKHIRUPHUWKHUHVHDUFKHUGRHVQRWRQO\ORRNDWWKHUHODWLRQVEHWZHHQ
GLIIHUHQWREVHUYHGYDULDEOHVEXWDFWXDOO\DOWHUVRQH RUPRUH YDULDEOHDQGGHWHUPLQHV

WKHHIIHFWVRIWKLVFKDQJHRQRWKHUYDULDEOHV7KHDGGLWLRQRIDFRQWUROJURXSWKHQ
DOORZVXVWRLVRODWHWKHVSHFLILFHIIHFWRIWKHWDUJHWYDULDEOHLQDQXQHTXLYRFDOPDQQHU
ZKLFKLVZK\PDQ\VFKRODUVFRQVLGHUH[SHULPHQWDOGHVLJQWKHRSWLPDOPRGHOIRU
ULJRURXVUHVHDUFK
8QIRUWXQDWHO\LQHGXFDWLRQDOFRQWH[WVWUXHH[SHULPHQWDOGHVLJQVZLWKUDQGRPJURXS
DVVLJQPHQWVDUHYHU\UDUHO\IHDVLEOHDQGWKHUHIRUHWKHFRPPRQPHWKRGDSSOLHGXVHV
LQWDFWFODVVJURXSV7KLVGHVLJQLVREYLRXVO\DOHVVWKDQSHUIHFWFRPSURPLVHDQGKDV
EHHQFDOOHGDFFRUGLQJO\WKHµTXDVLH[SHULPHQWDOGHVLJQ¶%HIRUHZHORRNDWWKH
FKDUDFWHULVWLFVRIVXFKVWXGLHVOHWPHEULHIO\ZDUQDJDLQVWDQLQDGHTXDWHYDULDWLRQRI
WKHWUXHH[SHULPHQWDOGHVLJQWKDWVKRXOGEHDYRLGHGLISRVVLEOHWKHµRQHJURXSSUH
WHVW±SRVWWHVWGHVLJQ¶:KHQZDQWLQJWRWU\RXWWKHLPSDFWRIDVSHFLDOWUHDWPHQW IRU
H[DPSOHVWUDWHJ\WUDLQLQJ QRYLFHUHVHDUFKHUVRIWHQVSHFXODWHWKDWLWLVHQRXJKWRWDNH
DOHDUQHUJURXSDGPLQLVWHUWKHWUHDWPHQWDQGLIWKHGLIIHUHQFHEHWZHHQWKHSUHWHVWDQG
WKHSRVWWHVWLVVLJQLILFDQWWKHQWKHWUHDWPHQWKDVREYLRXVO\ZRUNHG7KHSUREOHPLV
WKDWHYHQLIWKHVSHFXODWLRQLVFRUUHFWWKHGHVLJQFDQQRWH[FOXGHRWKHUSRVVLEOHVRXUFHV
RIFKDQJHVXFKDVWKHPDWXUDWLRQHIIHFWWKHSUDFWLFHHIIHFWRUWKH+DZWKRUQHHIIHFW²
VHH6HFWLRQ7KHUHIRUH,ZRXOGVXJJHVWWKDWUHVHDUFKHUVDOZD\VLQFOXGHDFRQWURO
JURXSLQWKHLUH[SHULPHQWDOGHVLJQV
4XDVLH[SHULPHQWDOGHVLJQ
,QPRVWHGXFDWLRQDOVHWWLQJVUDQGRPDVVLJQPHQWRIVWXGHQWVE\WKHUHVHDUFKHULVUDUHO\
SRVVLEOHDQGWKHUHIRUHUHVHDUFKHUVRIWHQKDYHWRUHVRUWWRDµTXDVLH[SHULPHQWDOGHVLJQ¶
4XDVLH[SHULPHQWVDUHVLPLODUWRWUXHH[SHULPHQWVLQHYHU\UHVSHFWH[FHSWWKDWWKH\GR
QRWXVHUDQGRPDVVLJQPHQWWRFUHDWHWKHFRPSDULVRQVIURPZKLFKWUHDWPHQWFDXVHG
FKDQJHLVLQIHUUHG &RRNDQG&DPSEHOO %HFDXVHRIWKHSUDFWLFDOFRQVWUDLQWV
ZRUNLQJZLWKµQRQHTXLYDOHQWJURXSV¶KDVEHFRPHDQDFFHSWHGUHVHDUFKPHWKRGRORJ\LQ
ILHOGVWXGLHVZKHUHUDQGRPL]DWLRQLVLPSRVVLEOHRULPSUDFWLFDO+RZHYHULQVXFKFDVHV
ZHFDQQRWUHO\RQWKHQHDWDQGDXWRPDWLFZD\WKHWUXHH[SHULPHQWGHDOVZLWKYDULRXV
WKUHDWVWRYDOLGLW\EXWKDYHWRGHDOZLWKWKHVHWKUHDWVRXUVHOYHV,QSUDFWLFDOWHUPVLQ
RUGHUWREHDEOHWRPDNHFDXVDOFODLPVEDVHGRQDTXDVLH[SHULPHQWDOVWXG\WKHHIIHFWV
RIWKHLQLWLDOJURXSGLIIHUHQFHVQHHGWREHWDNHQLQWRDFFRXQW
,QDPHWDDQDO\VLVFRPSDULQJIXOOH[SHULPHQWDODQGTXDVLH[SHULPHQWDOGHVLJQV
+HLQVPDQDQG6KDGLVK IRXQGWKDWLIWKHWZRUHVHDUFKPHWKRGVZHUHHTXDOO\ZHOO
GHVLJQHGWKH\\LHOGHGFRPSDUDEOHUHVXOWV+RZHYHUWKHDXWKRUVDOVRSRLQWHGRXWWKDWLW
LVQRHDV\WDVNWRGHVLJQDTXDVLH[SHULPHQWDOVWXG\DVZHOODVDQH[SHULPHQWDOVWXG\
7KHUHVXOWVRIWKHPHWDDQDO\VLVSRLQWHGWRWZRVSHFLILFZD\VRILPSURYLQJWKHGHVLJQ
RITXDVLH[SHULPHQWDOVWXGLHV D DYRLGLQJDQ\VLWXDWLRQVZKHUHE\WKHVWXGHQWVVHOI
VHOHFWWKHPVHOYHV IRUH[DPSOHYROXQWHHU WREHLQWKHWUHDWPHQWJURXSDQG E
QLQLPL]LQJSUHWHVWGLIIHUHQFHVEHWZHHQWKHWUHDWPHQWDQGWKHFRQWUROJURXSVDVPXFK
DVSRVVLEOH7KHUHDUHWZRIUHTXHQWO\XVHGPHWKRGVIRUDFKLHYLQJWKLVODWWHUJRDO

0DWFKLQJSDUWLFLSDQWVLQWKHWUHDWPHQWDQGFRQWUROJURXSV7KHPRVWFRPPRQPDWFKLQJ
SURFHGXUHLQYROYHVHTXDWLQJWKHFRQWURODQGWUHDWPHQWJURXSVRQDFDVHE\FDVH
EDVLVRQRQHRUPRUHYDULDEOHV,IZHNQRZWKDWVRPHOHDUQHUFKDUDFWHULVWLFVDUH
OLNHO\WRKDYHDQLPSDFWRQWKHWDUJHWYDULDEOHZHDUHH[DPLQLQJLQWKHVWXG\ LH
WKHµGHSHQGHQWYDULDEOH¶ ZHILUVWQHHGWRGHWHUPLQHRUPHDVXUHWKHSDUWLFXODU
LQGLYLGXDOGLIIHUHQFHYDULDEOHV IRUH[DPSOHVH[RU,4 DQGWKHQLGHQWLI\
SDUWLFLSDQWVLQWKHWZRFRPSDULVRQJURXSVZLWKYHU\VLPLODUSDUDPHWHUV IRU
H[DPSOHDJLUOZLWKDQ,4RIDURXQGLQERWKJURXSV ,QDTXDVLH[SHULPHQWDO
VWXG\ZHDUHXQOLNHO\WREHDEOHDFKLHYHSHUIHFWPDWFKLQJHYHQLIZHRPLWVRPH
SDUWLFLSDQWVZKRGRQRWKDYHDFORVHFRXQWHUSDUWEXWWKHUHVXOWDQWJURXS
FRPSRVLWLRQVZLOOEHFRQVLGHUDEO\PRUHFRPSDWLEOHWKDQWKHLQLWLDORQHZLWKRXW
PDWFKLQJ
8VLQJDQDO\VLVRIFRYDULDQFH $1&29$ $1&29$RIIHUVDVWDWLVWLFDOPHWKRGIRU
DGMXVWLQJWKHSRVWWHVWVFRUHVIRUDQ\SUHWHVWGLIIHUHQFHVLQRWKHUZRUGVZHFDQ
VWDWLVWLFDOO\VFUHHQWKHXQZDQWHGHIIHFWVRXWRIWKHRXWFRPHPHDVXUH 7KH
SURFHGXUHLVGHVFULEHGLQ6HFWLRQ
7KHUHLVQRGRXEWWKDWDTXDVLH[SHULPHQWDOGHVLJQOHDYHVDVWXG\PRUHYXOQHUDEOHWR
WKUHDWVWRYDOLGLW\WKDQDIXOOH[SHULPHQWDOGHVLJQEXWDV-RKQVRQDQG&KULVWHQVHQ
HPSKDVL]HMXVWEHFDXVHDWKUHDWLVSRVVLEOHLWGRHVQRWPHDQWKDWLWLV
QHFHVVDULO\SODXVLEOH)XUWKHUPRUHZHFDQVLJQLILFDQWO\UHGXFHWKLVSODXVLELOLW\E\
DSSO\LQJWKHDERYHWHFKQLTXHV$VDUHVXOWLWLVJHQHUDOO\DFFHSWHGWKDWSURSHUO\
GHVLJQHGDQGH[HFXWHGTXDVLH[SHULPHQWDOVWXGLHV\LHOGVFLHQWLILFDOO\FUHGLEOHUHVXOWV
$QDO\VLQJWKHUHVXOWVRIH[SHULPHQWDODQGTXDVL
H[SHULPHQWDOVWXGLHV
'DWDREWDLQHGIURPDVWXG\ZLWKDµSUHWHVW±SRVWWHVWFRQWUROJURXSGHVLJQ¶FDQEH
DQDO\VHGLQWZRZD\V7KHVLPSOHUPHWKRGLQYROYHVILUVWFRPSXWLQJµJDLQVFRUHV¶
VHSDUDWHO\LQWKHWUHDWPHQWDQGWKHFRQWUROJURXSVE\VXEWUDFWLQJWKHSUHWHVWVFRUHV
IURPWKHSRVWWHVWVFRUHVDQGWKHQFRPSDULQJWKHVHJDLQVFRUHVE\XVLQJWWHVWVRU
µDQDO\VLVRIYDULDQFH¶ $129$ VHH6HFWLRQVDQG WRVHHZKHWKHUWKHJDLQLQ
WKHWUHDWPHQWFRQGLWLRQZDVVLJQLILFDQWO\ELJJHUWKDQWKDWLQWKHRUGLQDU\ FRQWURO
FRQGLWLRQ
$VOLJKWO\PRUHFRPSOLFDWHGPHWKRGLVXVLQJ$1&29$LQZKLFKZHFRPSDUHWKH
SRVWWHVWVFRUHVE\FRQWUROOLQJIRUWKHSUHWHVWVFRUHV²VHH6HFWLRQ%RWKPHWKRGV
DUHDFFHSWDEOHDQGXVHGLQFRQWHPSRUDU\VWXGLHVEXWWKHUHLVDJURZLQJUHFRJQLWLRQWKDW
$1&29$RIIHUVPRUHSUHFLVHUHVXOWV -RKQVRQDQG&KULVWHQVHQ IRUDWOHDVWWZR
UHDVRQVILUVWVHYHUDO EXWQRWDOO PHWKRGRORJLVWVFODLPWKDWJDLQVFRUHVDUHQRW
VXIILFLHQWO\UHOLDEOHDVWKH\DUHV\VWHPDWLFDOO\UHODWHGWRDQ\UDQGRPHUURURI
PHDVXUHPHQW 6HH0HQDUG 6HFRQGHVSHFLDOO\LQTXDVLH[SHULPHQWDOVWXGLHV
$1&29$KHOSVWRUHGXFHWKHLQLWLDOJURXSGLIIHUHQFHV DVGHVFULEHGDERYH

([SHULPHQWDOVWXGLHVLQHGXFDWLRQDODQGDSSOLHGOLQJXLVWLF
UHVHDUFK
,QHGXFDWLRQDOUHVHDUFKH[SHULPHQWDOVWXGLHVKDYHW\SLFDOO\EHHQFDOOHGµLQWHUYHQWLRQ
UHVHDUFK¶DQGDV+VLHKHWDO GHVFULEHVXFKLQYHVWLJDWLRQVKDYHEHHQHYLGHQW
VLQFHWKHEHJLQQLQJRIWKHWKFHQWXU\7KHPHWKRGZDVUHLQYLJRUDWHGLQWKHVE\
WKHSXEOLFDWLRQRI&DPSEHOODQG6WDQOH\¶V FODVVLFWH[WRQH[SHULPHQWDOUHVHDUFK
2YHUWKHSDVWWZRGHFDGHVKRZHYHUWKHXVHRIH[SHULPHQWDOPHWKRGRORJ\LQ
HGXFDWLRQDOUHVHDUFKKDVEHHQEHRQWKHGHFOLQH,QDVXUYH\RIUHVHDUFKVWXGLHV
SXEOLVKHGLQIRXUOHDGLQJHGXFDWLRQDOSV\FKRORJLFDOMRXUQDOVDVZHOODVWKHIODJVKLS
MRXUQDORI$PHULFDQHGXFDWLRQDOUHVHDUFKWKH$PHULFDQ(GXFDWLRQDO5HVHDUFK-RXUQDO
$(5- +VLHKHWDOIRXQGWKDWEHWZHHQDQGWKHSURSRUWLRQRILQWHUYHQWLRQ
VWXGLHVGURSSHGIURPSHUFHQW HGXFDWLRQDOSV\FKRORJ\ DQGSHUFHQW $(5- WR
SHUFHQWDQGSHUFHQWUHVSHFWLYHO\)XUWKHUPRUHDVWKHDXWKRUVUHSRUWQRWRQO\
KDVHGXFDWLRQDOLQWHUYHQWLRQUHVHDUFKEHHQGHFUHDVLQJLQTXDQWLW\EXWWKHUHKDVDOVR
EHHQDGHFOLQHLQTXDOLW\ZLWKWKHLQWHUYHQWLRQVEHFRPLQJVKRUWHU)RUH[DPSOHLQWKH
IRXUHGXFDWLRQDOSV\FKRORJ\MRXUQDOVLQSHUFHQWRIWKHLQWHUYHQWLRQVODVWHG
PRUHWKDQGD\ZKHUHDVWKHVDPHILJXUHLQZDVRQO\SHUFHQW2QHLPSRUWDQW
UHDVRQIRUWKLVGHFOLQHLVWKHLQFUHDVHGXVHRIVWUXFWXUDOHTXDWLRQPRGHOOLQJ VHH6HFWLRQ
ZKLFKPDNHVLWSRVVLEOHWRPDNHµTXDVLFDXVDO¶FODLPVDERXWRXWFRPHVEDVHG
RQQRQH[SHULPHQWDOFRUUHODWLRQDOUHVHDUFK7KXVE\XVLQJWKLVSURFHGXUHUHVHDUFKHUV
FDQVRPHWLPHV EXWQRWDOZD\V FLUFXPYHQWWKHODERULRXVDQGOHQJWK\SURFHGXUHRI
UXQQLQJDIXOOH[SHULPHQWDOVWXG\
,QDSSOLHGOLQJXLVWLFVWKHUHZDVDVWHDG\VWUHDPRILQWHUYHQWLRQVWXGLHVLQWKHV
DVSDUWRIWKHµPHWKRGVFRPSDULVRQVWXGLHV¶LQFODVVURRPUHVHDUFK VHH6HFWLRQ EXW
RYHUWKHVXEVHTXHQWGHFDGHVH[SHULPHQWVEHFDPHOHVVSRSXODUIRUDWOHDVWWZRUHDVRQV
D PDQ\RIWKHWRSLFVDSSOLHGOLQJXLVWVDUHLQWHUHVWHGLQDUHQRWGLUHFWO\UHODWHGWR
µWUHDWPHQW¶RUµLQWHUYHQWLRQ¶WKDWLVWKH\GRQRWHDVLO\OHQGWKHPVHOYHVWRPDQLSXODWLRQ
IRUH[DPSOHJHQGHUGLIIHUHQFHVSHUVRQDOLW\WUDLWVHWKQROLQJXLVWLFYDULDWLRQ DQG E
H[SHULPHQWDOUHVHDUFKLVUDWKHUQDUURZLQVFRSHDVRQO\RQHRUDIHZYDULDEOHVFDQEH
DOWHUHGDWDWLPH2QWKHRWKHUKDQGW\SLFDODSSOLHGOLQJXLVWLFYHQXHVVXFKDVODQJXDJH
FODVVURRPVDUHFRPSOH[HQYLURQPHQWVZKHUHPDQ\IDFWRUVRSHUDWHVLPXOWDQHRXVO\DQG
VLJQLILFDQWFKDQJHVFDQRIWHQRQO\EHDFKLHYHGLIVHYHUDOYDULDEOHVZRUNLQFRQFHUWRULQ
VSHFLDOFRPELQDWLRQV$QH[SHULPHQWDOGHVLJQWDUJHWLQJRQHRUWZRYDULDEOHVLV
LQDGHTXDWHWRDGGUHVVVXFKPXOWLYDULDWHSDWWHUQV
:KLOHWKHVHOLPLWDWLRQVDUHYDOLGOHWPHDOVRHPSKDVL]HWKDWLQPDQ\VLWXDWLRQV
H[SHULPHQWDOVWXGLHVZRXOGEHIHDVLEOH IRUH[DPSOHLQVWXGLHVWKDWORRNDWWKHLPSDFW
RIDQ\ODQJXDJHUHODWHGSURFHVVHV DQGXQGRXEWHGO\VXSHULRUWRWKHOHVVODERXU
LQWHQVLYHFRUUHODWLRQDORUVXUYH\VWXGLHVWKDWDUHXVXDOO\FRQGXFWHG$ZHOFRPHUHFHQW
WUHQGLQWKLVGLUHFWLRQKDVEHHQWKHHPHUJHQFHRIORQJLWXGLQDOLQYHVWLJDWLRQVRI/
LQVWUXFWLRQDOHIIHFWLYHQHVVIRFXVLQJRQVSHFLILFLQVWUXFWLRQDOWHFKQLTXHVVXFKDV
FRUUHFWLYHIHHGEDFN²DV2UWHJDDQG,EHUUL6KHD VXPPDUL]HWKHTXDOLW\RIWKH

TXDVLH[SHULPHQWDOGHVLJQWKDWLVW\SLFDOO\HPSOR\HGLQWKHVHVWXGLHVKDVVKRZQD
FRQVLGHUDEOHLPSURYHPHQWLQUHFHQW\HDUVIHDWXULQJORQJHUWUHDWPHQWSHULRGVDQG
LPPHGLDWHDVZHOODVGHOD\HGSRVWWHVWV
6WUHQJWKVDQGZHDNQHVVHVRIH[SHULPHQWDODQGTXDVL
H[SHULPHQWDOGHVLJQV
7KHPDLQVWUHQJWKRIWKHH[SHULPHQWDOGHVLJQLVWKDWLWLVWKHEHVWPHWKRG²DQGVRPH
ZRXOGFODLPWKHRQO\IXOO\FRPSHOOLQJPHWKRG²RIHVWDEOLVKLQJFDXVH±HIIHFW
UHODWLRQVKLSVDQGHYDOXDWLQJHGXFDWLRQDOLQQRYDWLRQV$IXOOµSUHWHVWSRVWWHVWFRQWURO
JURXSGHVLJQ¶GRHVDQH[FHOOHQWMREDWFRQWUROOLQJIRUWKHYDULRXVWKUHDWVWRWKHLQWHUQDO
YDOLGLW\RIWKHH[SHULPHQW+RZHYHUDV&ODUNH SRLQWVRXWWKHSULFHWKDWKDVWR
EHSDLGIRUGRLQJWKLVFDQVRPHWLPHVEHKLJK+LVPDLQFRQFHUQLVWKDWLQRUGHUWR
FRQWURODOOWKHYDULDEOHVWLJKWO\ZHPD\HQGXSZLWKDUWLILFLDOIUDPHZRUNVLQODERUDWRU\
FRQGLWLRQVZKLFKUHGXFHWKHH[WHUQDOYDOLGLW\ LHWKHJHQHUDOL]DELOLW\ RIWKHVWXG\,Q
RWKHUZRUGVH[SHULPHQWDOVWXGLHVFDQVDFULILFHH[WHUQDOYDOLGLW\IRUHQKDQFHGLQWHUQDO
YDOLGLW\ZKLFKLVRQHUHDVRQZK\WKHPHULWVRIWKHLUXVHLQHGXFDWLRQKDYHEHHQ
VHULRXVO\TXHVWLRQHG 6HH6KDYHOVRQDQG7RZQH $QGHYHQSURSHUO\FRQGXFWHG
H[SHULPHQWDOVWXGLHVFDQEHDIIHFWHGE\WKH+DZWKRUQHHIIHFWLQZKLFKWKHRXWFRPHRI
DQRWKHUZLVHVROLGH[SHULPHQWLVQRWFDXVHGE\WKHWUHDWPHQWYDULDEOHEXWE\WKHIDFW
WKDWDWUHDWPHQWKDVEHHQDSSOLHGUHJDUGOHVVRILWVQDWXUH²VHH6HFWLRQ
,QTXDVLH[SHULPHQWDOVWXGLHVZHGRQRWKDYHWRZRUU\VRPXFKDERXWUHGXFHG
H[WHUQDOYDOLGLW\EHFDXVHWKHVHLQYHVWLJDWLRQVWDNHSODFHLQDXWKHQWLFOHDUQLQJ
HQYLURQPHQWVXVLQJJHQXLQHFODVVJURXSVEXWVXFKGHVLJQVRSHQXSWKHVWXG\WRD
QXPEHURIQHZWKUHDWVGXHWRWKHLQHTXDOLW\RIWKHLQLWLDOWUHDWPHQWDQGFRQWUROJURXSV
7KLVSUREOHPLVUHIHUUHGWRDVWKHµVHOHFWLRQELDV¶DQGLWLQYROYHVRXWFRPHGLIIHUHQFHV
WKDWDUHQRWGXHWRWKHWUHDWPHQWEXWDUHDUWHIDFWVRIWKHGLIIHUHQFHVLQSUHH[LVWLQJ
FKDUDFWHULVWLFVRIWKHJURXSVEHLQJFRPSDUHG 6HH/DU]HOHUHHWDO $VDUHVXOWRI
WKLVELDVVXFKQRQUDQGRPL]HGGHVLJQVDUHOHVVHIIHFWLYHLQHOLPLQDWLQJFRPSHWLQJ
SODXVLEOHK\SRWKHVHVZLWKWKHVDPHDXWKRULW\DVDWUXHH[SHULPHQW0F'RQRXJKDQG
0F'RQRXJK DOVRGUDZDWWHQWLRQWRWKHIDFWWKDWLQPRVWODQJXDJHLQVWUXFWLRQDO
VLWXDWLRQVWKHOLVWRISRVVLEOHFRQIRXQGLQJYDULDEOHVLVVRODUJHWKDWDFKLHYLQJ
VDWLVIDFWRU\FRQWUROLVDUHDOFKDOOHQJH:KLOHWKLVLVXQGRXEWHGO\VRZHOOGHVLJQHGDQG
H[HFXWHGTXDVLH[SHULPHQWDOVWXGLHVWKDWFRQWUROIRUWKHHIIHFWVRIWKHPDLQLQLWLDOJURXS
GLIIHUHQFHVDUHXVXDOO\VHHQWRGHOLYHUVFLHQWLILFDOO\FUHGLEOHUHVXOWV
$ILQDOSUREOHPUDUHO\PHQWLRQHGLQUHVHDUFKWH[WVLVWKDWHYHQWKRXJKLQWHUYHQWLRQ
VWXGLHVDUHUDWKHUODERXULQWHQVLYHWKH\FDQH[DPLQHRQO\RQHRUWZRWDUJHWYDULDEOHVDW
DWLPH,QHGXFDWLRQDOUHVHDUFKZHRIWHQKDYHDZKROHUDQJHRISRWHQWLDOLQGHSHQGHQW
YDULDEOHVWRWHVW IRUH[DPSOHWKHLPSDFWRIYDULRXVLQVWUXFWLRQDOFRQGLWLRQVDQG
VWUDWHJLHV DQGLQVXFKFDVHVH[SHULPHQWDORUTXDVLH[SHULPHQWDOGHVLJQVDUHVLPSO\QRW
IHDVLEOH

&ROOHFWLQJTXDQWLWDWLYHGDWDYLDWKH,QWHUQHW
:LWKWKHJURZLQJXVHRIWKH,QWHUQHWLWZDVLQHYLWDEOHWKDWUHVHDUFKHUVZRXOGVWDUW
FROOHFWLQJGDWDE\PHDQVRIZHEEDVHGSURFHGXUHV,QGHHGJLYHQWKHLQFUHDVLQJO\
DYDLODEOHDQGFRQWLQXRXVO\LPSURYLQJFRPSXWHUKDUGZDUHDQGVRIWZDUHLWLVEHFRPLQJ
UHODWLYHO\HDV\WRVHWXSDQ,QWHUQHWVXUYH\RUH[SHULPHQWDQGDZHEEDVHGVWXG\RIIHUV
VRPHWHPSWLQJEHQHILWV IRUUHYLHZVLQFOXGLQJLQIRUPDWLRQRQIUHHVRIWZDUHVHH
%LUQEDXP)R[HWDO
x 5HGXFHGFRVWV6LQFHPRVWXQLYHUVLWLHVRUUHVHDUFKLQVWLWXWHVKDYHWKHQHFHVVDU\
IDFLOLWLHVVHWWLQJXSDQ,QWHUQHWEDVHGSURMHFWLVQRWPRUHH[SHQVLYHWKDQLQLWLDWLQJ
WUDGLWLRQDOUHVHDUFKZKHUHDVWKHUXQQLQJFRVWVDUHVLJQLILFDQWO\ORZHU VHHEHORZ
x &RQYHQLHQFHRIDGPLQLVWUDWLRQ7KHPDLQDWWUDFWLRQRIZHEEDVHGUHVHDUFKLVWKDWLW
GRHVQRWUHTXLUHDGPLQLVWUDWLRQRIWKHPDWHULDOVLQSHUVRQRQFHWKHUHFUXLWPHQW
SRVWLQJKDGEHHQPDGHDGPLQLVWUDWLRQLVVHOIUXQQLQJ
x $XWRPDWLFFRGLQJ8VLQJDVRFDOOHGµ&*,VFULSW¶WRUHFRUGUHVSRQVHVWRGLVFPDNHV
WKHFRGLQJDQGUHFRUGLQJRIWKHDQVZHUVDXWRPDWLFZKLFKFRXSOHGZLWKVHOI
UXQQLQJDGPLQLVWUDWLRQLVDUHDOERQXV
x +LJKOHYHORIDQRQ\PLW\7KHSHUFHSWLRQLVWKDWZHEEDVHGUHVHDUFKLVWUXO\
DQRQ\PRXVZKLFKHQKDQFHVWKHOHYHORIKRQHVW\+RZHYHUEHFDXVHWKHPDFKLQH
QXPEHURIDVXEPLVVLRQLVWUDFHDEOHDQGWKHVHFXULW\RIWKHGDWDGXULQJWUDQVLWRYHU
WKH,QWHUQHWFDQQRWEHJXDUDQWHHGE\WKHUHVHDUFKHULWLVLQSULQFLSOHSRVVLEOHWR
LGHQWLI\UHVSRQGHQWVE\DXWKRULWLHV
x ,QWHUQDWLRQDODFFHVV7KH,QWHUQHWGRHVQRWKDYHERXQGDULHVDQGWKHUHIRUHVFKRODUV
FDQUHDFKPXFKODUJHUDQGPRUHGLYHUVHVDPSOHVZRUOGZLGHWKDQZRXOGKDYHEHHQ
SRVVLEOHEHIRUH7KLVLVJRRGQHZVIRUFURVVFXOWXUDOUHVHDUFKHYHQWKRXJKWKH
ODQJXDJHEDUULHUVDUHVWLOOWKHUH
x $FFHVVWRVSHFLDOL]HGSRSXODWLRQV:HEEDVHGUHFUXLWLQJDOORZVDFFHVVWRVPDOO
VFDWWHUHGRUVSHFLDOL]HGSRSXODWLRQVZKLFKZRXOGRWKHUZLVHEHGLIILFXOWWRUHDFK
(QDPRXUHGRIWKHVHEHQHILWVDQGSRVVLELOLWLHVDQLQFUHDVLQJQXPEHURILQYHVWLJDWRUV
KDVDOUHDG\EHJXQH[SHULPHQWLQJZLWKZHEEDVHGUHVHDUFKDQGLWVHHPVWKDWWKHLQLWLDO
H[SHULHQFHVDQGWKHSXEOLFDFFHSWDQFHRIWKHILQGLQJVKDYHEHHQVXIILFLHQWO\SRVLWLYH
IRUWKLVPHWKRGWREHFRPHHVWDEOLVKHG,WLVQRWKDUGWRVHHWKDWZHEEDVHGUHVHDUFKZLOO
SOD\DPDMRUUROHLQIXWXUHUHVHDUFKPHWKRGRORJ\:KLOHWKLVLVXQGHUVWDQGDEOHZH
PXVWDOVRFRQVLGHUWKHOLPLWDWLRQVRIWKLVDSSURDFKWREHDEOHWRGHFLGHZKHWKHULWLV
DSSURSULDWHIRURXUSXUSRVHV7ZRSDUWLFXODUO\LPSRUWDQWDUHDVLQWKLVUHVSHFWFRQFHUQ
WHFKQLFDODQGVDPSOLQJLVVXHV
7HFKQLFDOLVVXHV
7KHQHZVWDQGDUGRIµ+\SHUWH[WPDUNXSODQJXDJH¶ +70/ LQWURGXFHGLQDOORZV
DSHUVRQYLHZLQJDZHESDJHWRHDVLO\VHQGGDWDEDFNWRDGHVLJQDWHGVHUYHUZKLFKFDQ

SURFHVVFRGHDQGVDYHWKHGDWDHOHFWURQLFDOO\ %LUQEDXP +RZHYHU,QWHUQHW
XVHUVKDYHGLIIHUHQWFRPSXWHUVV\VWHPVEURZVHUVDQGPRQLWRUVVRWKHDFWXDOVWLPXOXV
UHFHLYHGPD\GLIIHUTXLWHDELWIURPZKDWWKHLQYHVWLJDWRUKDVLQWHQGHG)RUWKLVUHDVRQ
LQDVXUYH\RQVHOIKDUP)R[HWDO GHFLGHGWRHPSOR\DVLQJOH+70/SDJH
RQO\$VWKHDXWKRUVH[SODLQZKLOHLWPD\KDYHEHHQSRVVLEOHWRLPSURYHWKHXVHU
LQWHUIDFHWKURXJKWKHXVHRIRWKHUWHFKQRORJLHV IRUH[DPSOH-DYD6FULSWDQGYDULRXV
RWKHUSOXJLQV LWZDVIHOWWKDWWKHEHQHILWVZRXOGKDYHEHHQRXWZHLJKHGE\WKH
SRWHQWLDODGYHUVHHIIHFWVRIWKHTXHVWLRQQDLUHQRWEHLQJDYDLODEOHLQWKHVDPHIRUPDWWR
DVZLGHDSRSXODWLRQ&XUUHQWO\SDUWLFLSDWLRQLQPRUHFRPSOH[ZHEEDVHGVXUYH\VDQG
H[SHULPHQWVLVOLPLWHGE\WHFKQLFDOLVVXHVVXFKDVFRQQHFWLRQVSHHGWRWKH,QWHUQHWDQG
TXDOLW\RILQVWDOOHGVRIWZDUHEXWJLYHQWKHVSHHGRISURJUHVVLQWKHVHDUHDVVXFK
UHVWULFWLRQVDUHOLNHO\WREHWHPSRUDU\
6DPSOLQJLVVXHV
7KHPRVWDFXWHSUREOHPDVVRFLDWHGZLWK,QWHUQHWEDVHGUHVHDUFKLVWKDWLWLVQRWSRVVLEOH
WRDSSO\DQ\V\VWHPDWLFSXUSRVLYHVDPSOLQJVWUDWHJ\:KDWQRUPDOO\KDSSHQVLQVXFK
UHVHDUFKLVWKDWRQFHWKHLQVWUXPHQW W\SLFDOO\EXWQRWDOZD\VDVXUYH\ KDVEHHQ
FRQVWUXFWHGDQGWHVWHGWKHLQYHVWLJDWRUVFRQWDFWYDULRXV,QWHUQHWGLVFXVVLRQJURXSV
EXOOHWLQERDUGVDQGOLVWVDQGRULQLWLDWHVRPHVRUWRIVQRZEDOOVDPSOLQJE\HPDLOLQJ
SRWHQWLDOSDUWLFLSDQWVDQGWKHQKRSHIRUDVL]HDEOHVDPSOH7KLVLVREYLRXVO\IDUIURP
EHLQJV\VWHPDWLFEXWEHIRUHZHGHFLGHWKDWWKLVODFNRIFRQWURORYHUZKRZLOO
HYHQWXDOO\SDUWLFLSDWHLQWKHVWXG\VKRXOGGLVTXDOLI\VXFKSURMHFWVIURPWKHFDWHJRU\RI
VFLHQWLILFLQTXLU\ZHVKRXOGUHFDOOWKDWQRQSUREDELOLW\VDPSOLQJ DQGHVSHFLDOO\
FRQYHQLHQFHVDPSOLQJ LVWKHPRVWFRPPRQVDPSOLQJµVWUDWHJ\¶HYHQLQQRQZHEEDVHG
UHVHDUFK²VHH6HFWLRQ6RWKHPDLQSUREOHPGRHVQRWQHFHVVDULO\FRQFHUQWKH
XQSULQFLSOHGVHOHFWLRQSURFHGXUHVEXWUDWKHUWKHIDFWWKDWWKHDFWXDOVDPSOHWKDW
FRPSOHWHVWKHZHEEDVHGVXUYH\RUH[SHULPHQWPD\EHPXFKPRUHKHWHURJHQHRXVWKDQ
LQWUDGLWLRQDOUHVHDUFKFRQVLVWLQJWRWDOO\RIVHOIVHOHFWHGSDUWLFLSDQWV$VDUHVXOWHYHQ
LIZHKDYHWKRXVDQGVRIUHVSRQVHVLWPD\EHYHU\GLIILFXOWWRGHFLGHKRZWRJHQHUDOL]H
WKHILQGLQJV
:KLOHWKHUHDUHQRNQRZQVROXWLRQVWRWKHVDPSOLQJSUREOHP%LUQEDXP
PHQWLRQVWZRVWUDWHJLHVWKDWPLJKWRIIHUDWOHDVWDSDUWLDOVROXWLRQ,QWKHILUVWDSSURDFK
ZHDQDO\VHWKHUHVHDUFKTXHVWLRQVHSDUDWHO\ZLWKLQHDFKVXEVWUDWXPRIWKHVDPSOH IRU
H[DPSOHDJHHGXFDWLRQJHQGHUDQGRWKHUGHPRJUDSKLFYDULDEOHV ,IWKHVDPH
FRQFOXVLRQVDUHUHDFKHGLQHDFKVXEJURXSWKLVPLJKWOHQGVRPHH[WHUQDOYDOLGLW\WRWKH
UHVXOWV7KHVHFRQGDSSURDFKDOVRLQYROYHVWKHFRPSDULVRQRIVXEVDPSOHVEXWWKLVWLPH
WKHLQYHVWLJDWRUVFRPSDUHWKHZHEEDVHGUHVXOWVZLWKWKHRXWFRPHVRIDVLPLODUQRQ
ZHEEDVHGVXUYH\RUH[SHULPHQW$JDLQWKHFRQYHUJHQFHRIWKHILQGLQJVFDQKHOSWR
YDOLGDWHWKHUHVXOWV)LQDOO\ZHFDQDOVRLPDJLQHFRPELQHGVWXGLHVLQZKLFKDVDPSOH
RIUHVSRQGHQWVLVDSSURDFKHGLQVRPHWUDGLWLRQDOZD\EXWLQVWHDGRIDGPLQLVWHULQJ
VRPHUHVHDUFKLQVWUXPHQWWRWKHPWKH\DUHDVNHGWRORJRQDQGFRPSOHWHWKHVXUYH\RU

H[SHULPHQWRQOLQH7KLVFDQKDSSHQDWKRPH ZKLFKZRXOGEHEDVLFDOO\WKHVDPHDVD
WDNHDZD\TXHVWLRQQDLUH RULQPRUHFRQWUROOHGFLUFXPVWDQFHVVXFKDVDFRPSXWHUODE

4XDOLWDWLYHGDWDFROOHFWLRQ
4XDOLWDWLYHGDWDDUHVH[\
0LOHVDQG+XEHUPDQ
4XDQWLWDWLYHLQTXLU\FDQEHGLYLGHGLQWRWZRGLVWLQFWSKDVHV²GDWDFROOHFWLRQDQGGDWD
DQDO\VLV²TXLWHHDVLO\EHFDXVHWKH\W\SLFDOO\IROORZHDFKRWKHULQDOLQHDUPDQQHU7KLV
LVQRWWKHFDVHZLWKTXDOLWDWLYHUHVHDUFKEHFDXVHDVZHVDZLQ&KDSWHULQDFFRUGDQFH
ZLWKWKHIOH[LEOHDQGHPHUJHQWQDWXUHRIWKHTXDOLWDWLYHUHVHDUFKSURFHVVTXDOLWDWLYH
GDWDFROOHFWLRQDQGDQDO\VLVDUHRIWHQFLUFXODUDQGIUHTXHQWO\RYHUODS)XUWKHUPRUHLWLV
VRPHWLPHVSUREOHPDWLFWRGHFLGHZKHWKHUDFHUWDLQTXDOLWDWLYHPHWKRGUHIHUVSULPDULO\
WRGDWDFROOHFWLRQGDWDDQDO\VLVRUDXQLTXHFRPELQHGGHVLJQ²JURXQGHGWKHRU\DQG
WKHFDVHVWXG\DUHJRRGH[DPSOHVRIWKLVDPELJXLW\,QGHHGLWLVIDLUWRVD\WKDW
UHVHDUFKPHWKRGRORJ\WH[WVLQJHQHUDOKDYHEHHQLQFRQVLVWHQWLQKRZWKH\KDYHEURNHQ
GRZQWKHEURDGXPEUHOODWHUPµTXDOLWDWLYHUHVHDUFK¶LQWRVSHFLILFGDWDFROOHFWLRQDQG
GDWDDQDO\WLFDOPHWKRGVWHFKQLTXHVDQGGHVLJQV
,QWKLVFKDSWHU,ZLOOODUJHO\GLVUHJDUGWKHYDULRXVWHUPLQRORJLFDOLQFRQVLVWHQFLHV
FRQFHUQLQJWKHGLVWLQFWLRQVRIPHWKRGYVGHVLJQDQGGDWDFROOHFWLRQYVGDWDDQDO\VLV
DQGZLOOFRYHUDOOWKHLPSRUWDQWTXDOLWDWLYHSURFHGXUHVWKDWOHDGWRWKHJHQHUDWLRQRID
GDWDVHWLQFOXGLQJHWKQRJUDSK\RQHWRRQHLQWHUYLHZVIRFXVJURXSLQWHUYLHZV
LQWURVSHFWLRQFDVHVWXGLHVGLDU\VWXGLHVDQGUHVHDUFKMRXUQDOV*URXQGHGWKHRU\ZLOOEH
GLVFXVVHGLQ&KDSWHU ZKLFKIRFXVHVRQTXDOLWDWLYHGDWDDQDO\VLV EHFDXVHLWVPRVW
VDOLHQWDVSHFWLQFRQWHPSRUDU\UHVHDUFKLVWKHVSHFLDOWKUHHSKDVHFRGLQJV\VWHPLW
RIIHUV%HIRUHH[DPLQLQJWKHVSHFLILFGDWDFROOHFWLRQSURFHGXUHVOHWXVILUVWORRNDWWZR
EDVLFLVVXHVWKHFKDUDFWHULVWLFVRITXDOLWDWLYHGDWDLQJHQHUDODQGWKHGHVFULSWLRQRI
TXDOLWDWLYHVDPSOLQJ
4XDOLWDWLYHGDWD
:HVDZLQ&KDSWHUWKDWDOWKRXJKTXDOLWDWLYHGDWDFDQFRPHIURPPDQ\VRXUFHVLWLV
XVXDOO\WUDQVIRUPHGLQWRWH[WXDOIRUP VHHDOVR6HFWLRQ UHVXOWLQJLQKXQGUHGV LI
QRWWKRXVDQGV RISDJHVRIWUDQVFULSWVDQGILHOGQRWHV,EHOLHYHLWLVLPSRUWDQWWR
KLJKOLJKWDWWKHEHJLQQLQJRIWKLVFKDSWHUWZRPDLQFKDUDFWHULVWLFVRIDW\SLFDO
TXDOLWDWLYHGDWDVHWLWVWHQGHQF\WREHFRPHLQFUHDVLQJO\ORQJDQGLWVUDWKHUXQIRFXVHG
DQGKHWHURJHQHRXVQDWXUH7RSXWLWEURDGO\TXDOLWDWLYHGDWDWHQGVWREHEXON\DQG
PHVV\
,QTXDOLWDWLYHUHVHDUFKWKHUHDUHQRH[SOLFLWUHVWULFWLRQVRQZKDWFDQEHFRQVLGHUHG
µGDWD¶DQGDV5LFKDUGV FRQFOXGHVWKHUHVHDUFKHULQDTXDOLWDWLYHSURMHFWRIWHQ
VWDUWVRXWE\WUHDWLQJHYHU\WKLQJDURXQGDWRSLFDVSRWHQWLDOGDWD7KXVTXDOLWDWLYHGDWD
H[SDQGVTXLFNO\DQGQRYLFHUHVHDUFKHUVRIWHQILQGWKDWWKHUHDOFKDOOHQJHLVQRWWR
JHQHUDWHHQRXJKGDWDEXWUDWKHUWRJHQHUDWHXVHIXOGDWD,QIDFWDVHULRXVSUREOHPLQ

TXDOLWDWLYHUHVHDUFKFDQEHWRRPXFKGDWDZKLFKLVDXJPHQWHGE\WKHIDFWWKDW
µTXDOLWDWLYHGDWDDUHPHVV\UHFRUGV¶ S XVXDOO\FRQVLVWLQJRIDPL[WXUHRIILHOG
QRWHVWUDQVFULSWVRIYDULRXVUHFRUGLQJVDVZHOODVGRFXPHQWVRIDGLYHUVHQDWXUHDQG
OHQJWK7KLVGRHVQRWPHDQWKDWTXDOLWDWLYHGDWDFDQQRWSURGXFHYDOXDEOHUHVXOWVEXWLW
GRHVPHDQWKDWSURFHVVLQJVXFKVL]HDEOHDQGKHWHURJHQHRXVGDWDVHWVFDQLQYROYHDORWRI
ZRUN7KHFDVHRIDUHFHQWUHVHDUFKVWXGHQWRIPLQHLVQRWDWDOOXQXVXDORQDWKUHH
ZHHNGDWDFROOHFWLRQILHOGWULSEDFNWRKHUKRPHFRXQWU\VKHJDWKHUHGVRPXFKGDWDWKDW
VKHVSHQWWKHQH[WHLJKWPRQWKVWUDQVODWLQJWUDQVFULELQJDQGSUHOLPLQDULO\FRGLQJLW
ZKLFKFXWGDQJHURXVO\LQWRKHUUHPDLQLQJWLPHIRUIROORZXSGDWDFROOHFWLRQDQG
VHFRQGRUGHUDQDO\VLV7KXVDVRPHZKDWXQIRFXVHGH[SORUDWLRQRIWKHVRFLDOZRUOGFDQ
EHLQ6LOYHUPDQ¶V ZRUGVµDUHFLSHIRUGLVDVWHU¶,QVXFKFDVHVHYHU\WKLQJ
ORRNVLPSRUWDQWDQGLWPD\EHRYHURSWLPLVWLFWRH[SHFWWKDWWKHWUXWKZLOOPLUDFXORXVO\
HPHUJHIURPWKHWRQVRIGDWDFROOHFWHG
7KHRWKHUVLGHRIWKHFRLQLVWKDWDOWKRXJKZHQHHGWREHFDUHIXOWREHVHOHFWLYH
GXULQJGDWDFROOHFWLRQZHPXVWQRWIRUJHWHLWKHUWKDWTXDOLWDWLYHUHVHDUFKLVE\
GHILQLWLRQOHVVV\VWHPDWLFDQGVWDQGDUGL]HGLQLWVGDWDFROOHFWLRQDSSURDFKWKDQ
TXDQWLWDWLYHUHVHDUFKDQGWKDWWKHPHVVLQHVVRIWKHULFKGDWDWKDWZHDUHDLPLQJIRULV
RIWHQPHUHO\DUHIOHFWLRQRIWKHFRPSOH[UHDOOLIHVLWXDWLRQVWKDWWKHGDWDFRQFHUQV,Q
RUGHUWRWDSLQWRFRUHSDUWLFLSDQWPHDQLQJVZHRIWHQQHHGWRIROORZPXOWLSOHVWUDWHJLHV
LQWRPXOWLSOHGLUHFWLRQVDWPXOWLSOHOHYHOV$FFRUGLQJO\WKHGHVLJQRIPRVWTXDOLWDWLYH
GDWDFROOHFWLRQLVIOXLGDQGRSHQHQGHGDQGUHVHDUFKHUVDUHQRWUHTXLUHGWRSODQDOO
HOHPHQWVRIWKHSURMHFWDWWKHRXWVHW:KLOHWKLVLVLQOLQHZLWKWKHGLVFRYHU\RULHQWHG
FKDUDFWHURITXDOLWDWLYHLQTXLU\5LFKDUGV DOVRVWUHVVHVWKDWSRVVLEO\WKHPRVW
FRPPRQFDXVHRISUREOHPVLQTXDOLWDWLYHGDWDFROOHFWLRQLVWKHODFNRIDQ\SODQVIRU
GDWDUHGXFWLRQ$VVKHDUJXHVµ1RYLFHUHVHDUFKHUVFDQYHU\HDVLO\IUDPHWKHVFDOHRID
SURMHFWLQWHUPVRIKRZZLGHO\WKH\PLJKWQHHGWRVSUHDGWKHLUQHWUDWKHUWKDQKRZ
ZLGHUHDOLVWLFDOO\WKHVSUHDGFDQEH¶ S
6DPSOLQJLQTXDOLWDWLYHUHVHDUFK
:HVDZLQ6HFWLRQWKDW48$1DQG48$/UHVHDUFKGLIIHUJUHDWO\LQKRZWKH\
DSSURDFKSDUWLFLSDQWVDPSOLQJ,QTXDQWLWDWLYHVWXGLHVWKHSULQFLSOHLVVWUDLJKWIRUZDUG
:HQHHGDVL]HDEOHVDPSOHWREHDEOHWRLURQRXWLGLRV\QFUDWLFLQGLYLGXDOGLIIHUHQFHV
4XDOLWDWLYHUHVHDUFKRQWKHRWKHUKDQGIRFXVHVRQGHVFULELQJXQGHUVWDQGLQJDQG
FODULI\LQJDKXPDQH[SHULHQFHDQGWKHUHIRUHTXDOLWDWLYHVWXGLHVDUHGLUHFWHGDW
GHVFULELQJWKHDVSHFWVWKDWPDNHXSDQLGLRV\QFUDWLFH[SHULHQFHUDWKHUWKDQGHWHUPLQLQJ
WKHPRVWOLNHO\RUPHDQH[SHULHQFHZLWKLQDJURXS 3RONLQJKRUQH $FFRUGLQJO\
DWOHDVWLQWKHRU\TXDOLWDWLYHLQTXLU\LVQRWFRQFHUQHGZLWKKRZUHSUHVHQWDWLYHWKH
UHVSRQGHQWVDPSOHLVRUKRZWKHH[SHULHQFHLVGLVWULEXWHGLQWKHSRSXODWLRQ,QVWHDGWKH
PDLQJRDORIVDPSOLQJLVWRILQGLQGLYLGXDOVZKRFDQSURYLGHULFKDQGYDULHGLQVLJKWV
LQWRWKHSKHQRPHQRQXQGHULQYHVWLJDWLRQVRDVWRPD[LPL]HZKDWZHFDQOHDUQ7KLV
JRDOLVEHVWDFKLHYHGE\PHDQVRIVRPHVRUWRIµSXUSRVHIXO¶RUµSXUSRVLYH¶VDPSOLQJ

3XUSRVLYHDQGWKHRUHWLFDOVDPSOLQJ
%HFDXVHWKHUHLVDOZD\VDOLPLWWRKRZPDQ\UHVSRQGHQWVZHFDQFRQWDFWRUKRZPDQ\
VLWHVZHFDQYLVLWZHKDYHWRPDNHVRPHSULQFLSOHGGHFLVLRQVRQKRZWRVHOHFWRXU
UHVSRQGHQWVDWZKLFKSRLQWWRDGGDGGLWLRQDOSDUWLFLSDQWVWRWKHVDPSOHDQGZKHQWR
VWRSJDWKHULQJPRUHGDWD(YHQZKHQRXUVDPSOHFRQVLVWVRIRQHFDVHRQO\ZHQHHGWR
VHOHFWWKHDVSHFWVRIWKDWFDVHWKDWZHZLOOIRFXVRQ 6HHµZLWKLQFDVHVDPSOLQJ¶LQ
6HFWLRQ 7KHUHIRUHDTXDOLWDWLYHVWXG\PXVWKDYHDVDPSOLQJSODQGHVFULELQJWKH
VDPSOLQJSDUDPHWHUV SDUWLFLSDQWVVHWWLQJVHYHQWVSURFHVVHV DQGWKLVSODQVKRXOGOLQH
XSZLWKWKHSXUSRVHVRIWKHVWXG\3XQFK VXJJHVWVWKDWLILWLVQRWFOHDUWRXV
ZKLFKFDVHVDVSHFWVRULQFLGHQWVWRVWXG\LWLVXVXDOO\ZRUWKGHYRWLQJPRUHZRUNWR
GHYHORSLQJWKHLQLWLDOUHVHDUFKTXHVWLRQV
,QWKHLULQIOXHQWLDOZRUNRQJURXQGHGWKHRU\*ODVHUDQG6WUDXVV VSRNHDERXW

µWKHRUHWLFDOVDPSOLQJ¶KLJKOLJKWLQJWKHIDFWWKDWVDPSOLQJVKRXOGEHDIOH[LEOH
RQJRLQJHYROYLQJSURFHVVRIVHOHFWLQJVXFFHVVLYHUHVSRQGHQWVRUVLWHVGLUHFWHGE\RXU
HDUOLHUGLVFRYHULHVVRWKDWWKHHPHUJLQJLGHDVDQGWKHRUHWLFDOFRQFHSWVFDQEHWHVWHGDQG
IXUWKHUUHILQHG6LOYHUPDQ H[SODLQVWKDWWKHWHUPµWKHRUHWLFDOVDPSOLQJ¶KDVEHHQ
WUDQVIHUUHGIURPJURXQGHGWKHRU\WRTXDOLWDWLYHUHVHDUFKLQJHQHUDODQGLQFXUUHQW
SUDFWLFHLWLVW\SLFDOO\XVHGV\QRQ\PRXVO\ZLWKµSXUSRVLYH¶VDPSOLQJ
,WHUDWLRQVDWXUDWLRQDQGVDPSOHVL]H
5HVHDUFKHUVDUHLQDJUHHPHQWWKDWWKHSDUWLFLSDQWVHOHFWLRQSURFHVVVKRXOGUHPDLQRSHQ
LQDTXDOLWDWLYHVWXG\DVORQJDVSRVVLEOHVRWKDWDIWHULQLWLDODFFRXQWVDUHJDWKHUHGDQG
DQDO\VHGDGGLWLRQDOSDUWLFLSDQWVFDQEHDGGHGZKRFDQILOOJDSVLQWKHLQLWLDO
GHVFULSWLRQRUFDQH[SDQGRUHYHQFKDOOHQJHLW7KLVF\FOLFDOSURFHVVRIPRYLQJEDFN
DQGIRUWKEHWZHHQGDWDFROOHFWLRQDQGDQDO\VLVLVRIWHQUHIHUUHGWRDVµLWHUDWLRQ¶
$OWKRXJKLWHUDWLRQLVDNH\SURFHVVLQTXDOLWDWLYHVDPSOLQJLWFDQQRWJRRQIRUHYHU
:KHQGRZHVWRSLW"7KHUHDUHQRULJLGJXLGHOLQHVEXWVFKRODUVDJUHHWKDWLGHDOO\WKH
LWHUDWLYHSURFHVVVKRXOGJRRQXQWLOZHUHDFKVDWXUDWLRQ*ODVHUDQG6WUDXVV
GHILQHGWKLVDVWKHSRLQWZKHQDGGLWLRQDOGDWDGRHVQRWVHHPWRGHYHORSWKHFRQFHSWV
DQ\IXUWKHUEXWVLPSO\UHSHDWVZKDWSUHYLRXVLQIRUPDQWVKDYHDOUHDG\UHYHDOHG,QRWKHU
ZRUGVVDWXUDWLRQLVWKHSRLQWZKHQWKHUHVHDUFKHUEHFRPHVµHPSLULFDOO\FRQILGHQW¶ S
WKDWKHVKHKDVDOOWKHGDWDQHHGHGWRDQVZHUWKHUHVHDUFKTXHVWLRQ,QSUDFWLFH
KRZHYHUUHVHDUFKHUVXVXDOO\GHFLGHZKHQWRVWRSDGGLQJFDVHVWRDVWXG\EDVHGRQD
FRPELQDWLRQRIWKHRUHWLFDOVDWXUDWLRQDQGSUDJPDWLFFRQVLGHUDWLRQVVXFKDVDYDLODEOH
WLPHDQGPRQH\ (LVHQKDUGW
)LQDOO\OHWXVDGGUHVVWKHIXQGDPHQWDOTXHVWLRQKRZELJVKRXOGWKHVDPSOHVL]HEH
LQDTXDOLWDWLYHVWXG\"$IWHUDOOZHFDQWDONDERXWWKHIOH[LELOLW\RIWKHSURFHVVRI
LWHUDWLRQEXWLQUHDOLW\PRVWUHVHDUFKHUVQHHGWRPDNHVRPHVRUWRIDQLQLWLDOSODQDERXW
WKHVDPSOHVL]HWRVFKHGXOHWKHLULQYHVWLJDWLRQ7KHSUDJPDWLFDQVZHULVWKDWLQP\
H[SHULHQFHDQLQWHUYLHZVWXG\ZLWKDQLQLWLDOVDPSOHVL]HRI±PLJKWZRUNZHOO
8VLQJFRPSXWHUDLGHGGDWDDQDO\VLVZHFDQLQFUHDVHWKHVDPSOHVL]HWRDVPDQ\DV

DOWKRXJKWKDWZRXOGSUREDEO\EHSXVKLQJWKHOLPLWVDQGZRXOGEHEDUHO\PDQDJHDEOH
IRUDVLQJOHUHVHDUFKHUVXFKDVDSRVWJUDGXDWHVWXGHQW7KHSRLQWWKRXJKLVWKDWDZHOO
GHVLJQHGTXDOLWDWLYHVWXG\XVXDOO\UHTXLUHVDUHODWLYHO\VPDOOQXPEHURIUHVSRQGHQWVWR
\LHOGWKHVDWXUDWHGDQGULFKGDWDWKDWLVQHHGHGWRXQGHUVWDQGHYHQVXEWOHPHDQLQJVLQ
WKHSKHQRPHQRQXQGHUIRFXV
6SHFLILFVDPSOLQJVWUDWHJLHV
3XUSRVLYHVDPSOLQJFDQIROORZDQXPEHURIGLIIHUHQWVWUDWHJLHVGHSHQGLQJRQWKH
UHVHDUFKWRSLFDQGVHWWLQJ:KHQGHVLJQLQJWKHVDPSOLQJSODQZHDOVRQHHGWRWDNHLQWR
DFFRXQWIHDVLELOLW\LVVXHV LQWHUPVRIWLPHPRQH\UHVSRQGHQWDYDLODELOLW\ DQG²ZKDW
LVRIWHQLJQRUHG²VDWXUDWLRQFRQVLGHUDWLRQV7KHPRUHFRKHVLYHKRPRJHQHRXVWKH
VDPSOHWKHIDVWHUWKHVDWXUDWLRQEXWDWWKHVDPHWLPHWKHQDUURZHUWKHVFRSHRIWKH
SURMHFW7KHIROORZLQJOLVWRIWKHPRVWFRPPRQTXDOLWDWLYHVDPSOLQJVWUDWHJLHVRIIHUVD
VHQVHRIKRZZLGHZHQHHGWRWKURZWKHQHWWREHDEOHWRUHDFKVDWXUDWLRQLQGLIIHUHQW
UHVHDUFKDUHDV$VDSUHOLPLQDU\OHWPHHPSKDVL]HWKDWLWLVYHU\LPSRUWDQWWRPDNHWKH
VDPSOLQJVWUDWHJ\H[SOLFLWULJKWIURPWKHVWDUWVRWKDWZHFDQFRQYH\WKHXQGHUO\LQJ
ORJLFWRWKHUHDGHUVRIWKHILQDOUHVHDUFKUHSRUW
5HODWLYHO\TXLFNVDWXUDWLRQFDQEHDFKLHYHGE\WKHIROORZLQJWKUHHLQWHUUHODWHG
VDPSOLQJVWUDWHJLHVEHFDXVHWKH\DOODLPDWVHOHFWLQJSDUWLFLSDQWVZKRDUHVLPLODULQ
VRPHUHVSHFW
x +RPRJHQHRXVVDPSOLQJ7KHUHVHDUFKHUVHOHFWVSDUWLFLSDQWVIURPDSDUWLFXODU
VXEJURXSZKRVKDUHVRPHLPSRUWDQWH[SHULHQFHUHOHYDQWWRRXUVWXG\ IRUH[DPSOH
WKH\KDYHSDUWLFLSDWHGLQDVWXG\DEURDGSURJUDPPH ,QWKLVZD\WKLVVWUDWHJ\
DOORZVXVWRFRQGXFWDQLQGHSWKDQDO\VLVWRLGHQWLI\FRPPRQSDWWHUQVLQDJURXS
ZLWKVLPLODUFKDUDFWHULVWLFV
x 7\SLFDOVDPSOLQJ7KHUHVHDUFKHUVHOHFWVSDUWLFLSDQWVZKRVHH[SHULHQFHLVW\SLFDO
ZLWKUHJDUGWRWKHUHVHDUFKIRFXV IRUH[DPSOHWKH\DOOVWXG\DIRUHLJQODQJXDJHDV
DVFKRROVXEMHFWDWDQLQWHUPHGLDWHOHYHOZLWKPRGHUDWHVXFFHVV 7KLVVWUDWHJ\
DVVXPHVWKDWZHKDYHDSURILOHRIWKHWDUJHWHGDWWULEXWHVSRVVHVVHGE\DQµDYHUDJH¶
OHDUQHU$OWKRXJKZHFDQQRWJHQHUDOL]HIURPWKHUHVXOWVEHFDXVHZHFDQQRWFODLP
WKDWHYHU\ERG\ZLOOKDYHWKHVDPHH[SHULHQFHZHFDQOLVWWKHW\SLFDORUQRUPDO
IHDWXUHVRIWKHH[SHULHQFH
x &ULWHULRQVDPSOLQJ7KHUHVHDUFKHUVHOHFWVSDUWLFLSDQWVZKRPHHWVRPHVSHFLILF
SUHGHWHUPLQHGFULWHULD IRUH[DPSOHFRPSDQ\H[HFXWLYHVZKRIDLOHGDQLPSRUWDQW
ODQJXDJHH[DP
:HFDQDOVRJDLQYDOXDEOHLQVLJKWLQWRDQLVVXHLIUDWKHUWKDQVHOHFWLQJW\SLFDO
SDUWLFLSDQWVZHLQWHQWLRQDOO\ORRNDWWKHZKROHUDQJHRISRVVLEOHUHVSRQVHVLQFOXGLQJ
YHU\VSHFLDOFDVHV:KLOHWKLVDSSURDFKFHUWDLQO\LQFUHDVHVWKHVFRSHRIWKHDQDO\VLV
'XII PHQWLRQVWKDWLWDOVRFDUULHVWKHSRWHQWLDOSUREOHPRISODFLQJµRYHU
HPSKDVLVRQSRVVLEO\DW\SLFDOFULWLFDOH[WUHPHLGHDOXQLTXHRUSDWKRORJLFDOFDVHV

UDWKHUWKDQW\SLFDORUUHSUHVHQWDWLYHFDVHV¶7KHIROORZLQJWKUHHVWUDWHJLHVRYHUODSWR
VRPHH[WHQW
x 0D[LPXPYDULDWLRQVDPSOLQJ7KHUHVHDUFKHUVHOHFWVFDVHVZLWKPDUNHGO\GLIIHUHQW
IRUPVRIH[SHULHQFH IRUH[DPSOH/OHDUQHUVIURPDOOGHYHORSPHQWDOOHYHOV 7KLV
SURFHVVZLOODOORZXVWRH[SORUHWKHYDULDWLRQZLWKLQWKHUHVSRQGHQWVDQGLWZLOODOVR
XQGHUVFRUHDQ\FRPPRQDOLWLHVWKDWZHILQGLIDSDWWHUQKROGVDFURVVWKHVDPSOHG
GLYHUVLW\ZHFDQDVVXPHWKDWLWLVUHDVRQDEO\VWDEOH
x ([WUHPHRUGHYLDQWFDVHVDPSOLQJ)ROORZLQJWKHVDPHORJLFDVPD[LPXPYDULDWLRQ
VDPSOLQJWKHUHVHDUFKHUVHOHFWVWKHPRVWH[WUHPHFDVHV IRUH[DPSOHWKHPRVW
PRWLYDWHGDQGGHPRWLYDWHGOHDUQHUV 2QWKHRQHKDQGWKLVDOORZVXVWRILQGWKH
OLPLWVRIWKHH[SHULHQFHRQWKHRWKHUKDQGLIHYHQVXFKFDVHVVKDUHFRPPRQ
HOHPHQWVWKH\DUHOLNHO\WREHUHDOFRUHFRPSRQHQWVRIWKHH[SHULHQFH
x &ULWLFDOFDVHVDPSOLQJ7KHUHVHDUFKHUGHOLEHUDWHO\WDUJHWVFDVHVZKLFKRIIHUD
GUDPDWLFRUIXOOUHSUHVHQWDWLRQRIWKHSKHQRPHQRQHLWKHUE\WKHLULQWHQVLW\RUE\
WKHLUXQLTXHQHVV IRUH[DPSOHLQDODQJXDJHDWWULWLRQVWXG\H[DPLQLQJSHRSOHZKR
KDYHFRPSOHWHO\IRUJRWWHQDQ/WKH\XVHGWRVSHDN 7KHLUFDVHPD\EHWDNHQDV
WKHPRVWVDOLHQWRUFRPSUHKHQVLYHPDQLIHVWDWLRQRIWKHSKHQRPHQRQXQGHU
VFUXWLQ\LQVXFKVLWXDWLRQVUHVHDUFKHUVDUHQRWRQO\LQWHUHVWHGLQZKDWWKH\ILQGEXW
DOVRLQZKDWWKH\GRQRWEHFDXVHVRPHWKLQJWKDWGRHVQRWRFFXULQVXFKVDOLHQW
FDVHVLVXQOLNHO\WRKDSSHQHOVHZKHUH
$VLWRIWHQKDSSHQVWKHPRVWSUDFWLFDODQGIHDVLEOH DQGRIFRXUVHPRVWFRPPRQ
VDPSOLQJVWUDWHJLHVDUHWKHOHDVWVRXQGIURPDWKHRUHWLFDOSRLQWRIYLHZ7KUHHVXFK
µOHVVSULQFLSOHG¶VWUDWHJLHVDUHSDUWLFXODUO\ZHOONQRZQ
x 6QRZEDOORUFKDLQVDPSOLQJ7KHVWDUWLQJSRLQWRIWKLVVWUDWHJ\LVDSULQFLSOHGOLVWRI
NH\UHVSRQGHQWVZKRDUHWKHQDVNHGWRUHFUXLWIXUWKHUSDUWLFLSDQWVZKRDUHVLPLODU
WRWKHPLQVRPHUHVSHFWFHQWUDOWRWKHLQYHVWLJDWLRQ7KLVFKDLQUHDFWLRQFDQUHDFK
IDUZKLFKLVLGHDOLQVLWXDWLRQVZKHUHWKHH[SHULHQFHLQTXHVWLRQLVUDUH
x 2SSRUWXQLVWLFVDPSOLQJ7KLVLVDQXQSODQQHGDQGSRWHQWLDOO\KDSKD]DUGSURFHGXUH
LQWKHVHQVHWKDWLWLVIROORZHGRQWKHVSXURIWKHPRPHQWZKLOHZRUNLQJLQWKH
ILHOGWKHUHVHDUFKHUVRPHWLPHVFRPHVDFURVVUHVSRQGHQWVZKRDUHµWRRJRRGWR
PLVV¶DQGDGHFLVLRQWRLQFOXGHWKHPLVPDGHRQWKHVSRW7KHSUREOHPLVWKDWWKH\
DUHQRWDOZD\VH[DFWO\ZKDWLVQHHGHG\HWWKHLUVHOHFWLRQLVYHU\PXFKLQOLQHZLWK
WKHHPHUJHQWQDWXUHRIWKHTXDOLWDWLYHLQTXLU\
x &RQYHQLHQFHVDPSOLQJ7KLVLVWKHOHDVWGHVLUDEOHEXWWKHPRVWFRPPRQVDPSOLQJ
VWUDWHJ\DWOHDVWDWWKHSRVWJUDGXDWHUHVHDUFKOHYHO,WLVQRWSXUSRVLYHEXWODUJHO\
SUDFWLFDOWKHUHVHDUFKHUXVHVWKRVHZKRDUHDYDLODEOH2IFRXUVHLQDQLGHDOZRUOG
QRERG\ZRXOGHPSOR\DFRQYHQLHQFHVDPSOHEXWUHVHDUFK DQGSDUWLFXODUO\
SRVWJUDGXDWHUHVHDUFK DOOWRRRIWHQKDSSHQVLQOHVVWKDQLGHDOFLUFXPVWDQFHVXQGHU
FRQVLGHUDEOHWLPHRUILQDQFLDOFRQVWUDLQWV2QHUHGHHPLQJIHDWXUHRIWKLVVDPSOLQJ
VWUDWHJ\LVWKDWLWXVXDOO\UHVXOWVLQZLOOLQJSDUWLFLSDQWVZKLFKLVDSUHUHTXLVLWHWR

KDYLQJDULFKGDWDVHW2QWKHRWKHUKDQGVDWXUDWLRQPD\QRWKDSSHQDWDOO7KXV
WKLVVWUDWHJ\PD\VDYHWLPHPRQH\DQGHIIRUWEXWDWWKHH[SHQVHRIFUHGLELOLW\
0LOHVDQG+XEHUPDQ
:LWKLQFDVHVDPSOLQJ
$OWKRXJKTXDOLWDWLYHUHVHDUFKHUVDUHEHFRPLQJDZDUHRIWKHLPSRUWDQFHRISULQFLSOHG
SXUSRVLYHVDPSOLQJWKHSURFHVVLVXVXDOO\DSSOLHGRQO\WRVHOHFWLQJUHVSRQGHQWVWKDWLV
WRµEHWZHHQFDVHVDPSOLQJ¶,QVRPHTXDOLWDWLYHPHWKRGV IRUH[DPSOHHWKQRJUDSK\RU
FDVHVWXG\ ZHQHHGWRH[WHQGRXUVDPSOLQJSODQWRDOVRLQFOXGHµZLWKLQFDVHVDPSOLQJ¶
LHVHOHFWLQJGDWDIURPWKHSRWHQWLDOO\DYDLODEOHGDWDSRROFRQFHUQLQJDSDUWLFLSDQW
EHFDXVHZHKDYHWRPDNHUHJXODUGHFLVLRQVDERXWZKHQDQGKRZWRFROOHFWGDWDIURPD
SDUWLFXODUUHVSRQGHQWZKDWDVSHFWVRIWKHFDVHWRGLUHFWRXUDWWHQWLRQWRDQGZKLFK
DFWLYLWLHVORFDWLRQVRUHYHQWVWRIRFXVRQ,QWKHVHGHFLVLRQVZHQHHGWREHMXVWDV
SXUSRVLYHZLWKRXUFKRLFHVDVZLWKWKHVHOHFWLRQRILQIRUPDQWV
(WKQRJUDSK\
$JRRGVWDUWLQJSRLQWLQRXUH[SORUDWLRQRIWKHVSHFLILFTXDOLWDWLYHPHWKRGVRIGLVFRYHU\
LVHWKQRJUDSK\EHFDXVHWKLVDSSURDFKHPERGLHVLQPDQ\ZD\VWKHHVVHQFHRIWKH
TXDOLWDWLYHLQTXLU\,QIDFWµHWKQRJUDSK\¶KDVIUHTXHQWO\EHHQXVHGDVDV\QRQ\PIRU
µTXDOLWDWLYHUHVHDUFK¶DQG+DPPHUVOH\DQG$WNLQVRQ IRUH[DPSOHVWDUWHG
WKHLULQIOXHQWLDOPRQRJUDSKRQHWKQRJUDSK\E\VWDWLQJµ)RUWKHSXUSRVHVRIWKLVERRN
ZHVKDOOLQWHUSUHWWKHWHUPµHWKQRJUDSK\¶LQDOLEHUDOZD\QRWZRUU\LQJPXFKDERXW
ZKDWGRHVDQGGRHVQRWFRXQWDVH[DPSOHVRILW¶7KH\WKHQDGGHGµWKHERXQGDULHV
DURXQGHWKQRJUDSK\DUHQHFHVVDULO\XQFOHDU,QSDUWLFXODUZHZRXOGQRWZDQWWRPDNH
DQ\KDUGDQGIDVWGLVWLQFWLRQEHWZHHQHWKQRJUDSK\DQGRWKHUVRUWVRITXDOLWDWLYH
LQTXLU\¶ S :HVKRXOGQRWHKRZHYHU'XII¶V LQSUHVV ZDUQLQJWKDWHTXDWLQJ
HWKQRJUDSK\ZLWKTXDOLWDWLYHUHVHDUFKLQJHQHUDOFRQIXVHVWKHSLFWXUHEHFDXVHFHUWDLQ
TXDOLWDWLYHPHWKRGVVXFKDVWKHFDVHVWXG\RIWHQGLVSOD\FKDUDFWHULVWLFVWKDWDUH
GLIIHUHQWIURPWKRVHRIHWKQRJUDSK\
2ULJLQDWLQJLQFXOWXUDODQWKURSRORJ\HWKQRJUDSKLFUHVHDUFKDLPVDWGHVFULELQJDQG
DQDO\VLQJWKHSUDFWLFHVDQGEHOLHIVRIFXOWXUHVµ&XOWXUH¶LVQRWOLPLWHGWRHWKQLFJURXSV
EXWFDQEHUHODWHGWRDQ\µERXQGHGXQLWV¶ +DUNODX VXFKDVRUJDQL]DWLRQV
SURJUDPPHVDQGHYHQGLVWLQFWFRPPXQLWLHV7KXVZHFDQWDONDERXWWKHHWKQRJUDSK\
RIWKHODQJXDJHFODVVURRPRUWKHHWKQRJUDSKLFDQDO\VLVRIVSHFLILFVFKRROVRURWKHU
ODQJXDJHOHDUQLQJFRQWH[WV7KHFODVVLFLPDJHRIWKHHWKQRJUDSKHULVDUHVHDUFKHUZKR
HQWHUVLQWRWKHFRPPXQLW\DQGEHFRPHVLPPHUVHGLQLWVFXOWXUHIRUH[DPSOHE\OLYLQJ
DPRQJµQDWLYHV¶RQDUHPRWHLVODQGIRUVHYHUDO\HDUV)RUWKLVUHDVRQHWKQRJUDSK\KDV
EHHQFULWLFL]HGLQWKHSDVWIRUUHSUHVHQWLQJDFRORQLDOLVWDWWLWXGHEXWVLQFHZHDUH
ORRNLQJDWWKHHGXFDWLRQDODSSOLFDWLRQVRIWKHPHWKRGZHGRQRWKDYHWRUHYLHZWKLV
GHEDWHKHUH1HLWKHUZLOOZHH[DPLQHWKHFXUUHQWGLYLGHLQYDULRXVVFKRROVDQGVW\OHVRI
HWKQRJUDSKLFWKHRU\SDUWO\SUHFLSLWDWHGE\SRVWPRGHUQLVP 6HH+DUNODX

7KHPDLQJRDORIPRVWHWKQRJUDSKLFUHVHDUFKLVWRSURYLGHDµWKLFNGHVFULSWLRQ¶RI
WKHWDUJHWFXOWXUHWKDWLVDQDUUDWLYHWKDWGHVFULEHVULFKO\DQGLQJUHDWGHWDLOWKHGDLO\
OLIHRIWKHFRPPXQLW\DVZHOODVWKHFXOWXUDOPHDQLQJVDQGEHOLHIVWKHSDUWLFLSDQWV
DWWDFKWRWKHLUDFWLYLWLHVHYHQWVDQGEHKDYLRXUV)RUWKLVSXUSRVHHWKQRJUDSK\XVHVDQ
HFOHFWLFUDQJHRIGDWDFROOHFWLRQWHFKQLTXHVLQFOXGLQJSDUWLFLSDQWDQGQRQSDUWLFLSDQW
REVHUYDWLRQ VHH6HFWLRQ LQWHUYLHZLQJ 6HFWLRQ DQGWKHHWKQRJUDSKHU¶VRZQ
GLDU\ZLWKILHOGQRWHVDQGMRXUQDOHQWULHV 6HFWLRQ 7KHVHGDWDVRXUFHVDUHIXUWKHU
VXSSOHPHQWHGE\ILOPRUDXGLRUHFRUGLQJVDVZHOODVDXWKHQWLFGRFXPHQWVDQGSK\VLFDO
DUWHIDFWVDQGHWKQRJUDSKHUVPD\HYHQXVHVWUXFWXUHGTXHVWLRQQDLUHVWKDWKDYHEHHQ
GHYHORSHGGXULQJWKHFRXUVHRIWKHILHOGZRUN
$SSOLHGOLQJXLVWLFVDVDILHOGKDVDQLQKHUHQWLQWHUHVWLQLQWHUFXOWXUDOFRPPXQLFDWLRQ
DQGWKHUHIRUHHWKQRJUDSKLFUHVHDUFKKDVEHHQHPEUDFHGE\VFKRODUVZKRORRNDW
ODQJXDJHOHDUQLQJDVDSURIRXQGO\VRFLDOSUDFWLFHDQGVHHµVHFRQGODQJXDJHOHDUQLQJ¶
µVHFRQGFXOWXUHOHDUQLQJ¶DQGµODQJXDJHVRFLDOL]DWLRQ¶DVLQH[WULFDEO\ERXQG IRU
H[DPSOH5REHUWVHWDO6FKLHIIHOLQDQG2FKV ,QDGGLWLRQEHFDXVHRIWKH
LQFUHDVLQJO\VLWXDWHGQDWXUHRIPXFKUHFHQW6/$UHVHDUFKHWKQRJUDSK\KDVDOVREHHQ
XWLOL]HGIRUWKHFRQWH[WXDOL]HGDQDO\VLVRIFODVVURRPGLVFRXUVHDQGVFKRROOHDUQLQJ IRU
H[DPSOH'XII5DPSWRQYDQ/LHU:DWVRQ*HJHRIRUUHFHQW
RYHUYLHZVVHH+DUNODXDQG7RRKH\LQSUHVV 7KXVDV'XII VXPPDUL]HV
HWKQRJUDSK\KDVEHHQFRQVLVWHQWO\JDLQLQJSURPLQHQFHZLWKLQDSSOLHGOLQJXLVWLFVVLQFH
WKHDSSHDUDQFHRI:DWVRQ*HJHR¶V LQIOXHQWLDODUWLFOHRQWKHWRSLFDQGWKLV
LPSRUWDQFHLVZHOOUHIOHFWHGE\WKHIDFWWKDWWKHMRXUQDO7(62/4XDUWHUO\KDVSXEOLVKHG
VHSDUDWHµ &ULWLFDO (WKQRJUDSK\*XLGHOLQHV¶ &KDSHOOHDQG'XII IRULWV
FRQWULEXWRUV7RRKH\ LQSUHVV DOVRFRQILUPVWKDWWKHQXPEHURIHWKQRJUDSKLFODQJXDJH
HGXFDWLRQVWXGLHVKDVGUDPDWLFDOO\LQFUHDVHGRYHUWKHSDVW\HDUVZLWKPXFKRIWKH
ZRUNH[DPLQLQJVSHFLILFOLQNDJHVDPRQJQHJRWLDWLRQVRILGHQWLWLHVSUDFWLFHVDQG
ODQJXDJHOHDUQLQJERWKLQORFDOFRQWH[WVDQGRQDZLGHUVRFLHWDOVFDOH )RUGHWDLOVVHH
6HFWLRQGLVFXVVLQJFDVHVWXGLHVVLQFHFDVHVWXG\UHVHDUFKRIWHQXWLOL]HV
HWKQRJUDSKLFPHWKRGRORJ\
0DLQIHDWXUHVRIDQHWKQRJUDSKLFVWXG\
$FFRUGLQJWR+DUNODX DKDOOPDUNRIFODVVLFHWKQRJUDSKLFUHVHDUFKLVWKDWLW
LQYROYHVILUVWKDQGµSDUWLFLSDQWREVHUYDWLRQ¶LQDQDWXUDOVHWWLQJDQGPRVWVWXGLHVWKDW
IUDPHWKHPVHOYHVDVµHWKQRJUDSKLF¶LQFOXGHVRPHGHJUHHRIWKLVPHWKRG+RZHYHUZH
VDZWKDWHWKQRJUDSKHUVDOVRXWLOL]HVHYHUDORWKHUGDWDFROOHFWLRQWHFKQLTXHVDQGLQIDFW
+DUNODXSRLQWVRXWWKDWPXOWLSOHGDWDVRXUFHVDUHXVXDOO\FRQVLGHUHGGHVLUDEOH6RLIWKH
PHWKRGRIGDWDFROOHFWLRQLVQRWDNH\GHWHUPLQDQWRIWKHHWKQRJUDSKLFDSSURDFKZKDW
DUHWKHIHDWXUHVWKDWGHILQHLW"7KHIROORZLQJWKUHHSRLQWVDUHHPSKDVL]HGPRVW
IUHTXHQWO\LQWKHOLWHUDWXUHWKH\ZLOOQHHGOLWWOHHODERUDWLRQKHUHEHFDXVHWKH\KDYHEHHQ
GHVFULEHGHDUOLHUZKHQGLVFXVVLQJWKHTXDOLWDWLYHDSSURDFKLQJHQHUDO 6HH6HFWLRQ

x )RFXVLQJRQSDUWLFLSDQWPHDQLQJ,QHWKQRJUDSKLFVWXGLHVWKHSDUWLFLSDQWV¶
VXEMHFWLYHLQWHUSUHWDWLRQRIWKHLURZQEHKDYLRXUVDQGFXVWRPVLVVHHQDVFUXFLDOWR
XQGHUVWDQGLQJWKHVSHFLILFFXOWXUH7KHUHIRUHDFHQWUDODVSHFWRIHWKQRJUDSK\LVWR
ILQGZD\VRIORRNLQJDWHYHQWVWKURXJKWKHH\HVRIDQLQVLGHU
x 3URORQJHGHQJDJHPHQWLQWKHQDWXUDOVHWWLQJ7KHVXEWOHWLHVRIWKHSDUWLFLSDQWV¶
PHDQLQJ DQGPRVWRIWHQPXOWLSOHPHDQLQJV FDQQRWEHXQFRYHUHGXQOHVVWKH
UHVHDUFKHULPPHUVHVKLPKHUVHOILQWKHFXOWXUHDQGVSHQGVDQH[WHQGHGSHULRG
OLYLQJWKHUHREVHUYLQJWKHSDUWLFLSDQWVDQGFROOHFWLQJGDWD7KHUHIRUHDPLQLPXP
VWD\RI±PRQWKVLVXVXDOO\UHFRPPHQGHGWRDFKLHYHWKHQHFHVVDU\SURORQJHG
HQJDJHPHQW
x (PHUJHQWQDWXUH%HFDXVHWKHHWKQRJUDSKHULVHQWHULQJDQHZFXOWXUHWKHH[DFW
IRFXVRIWKHUHVHDUFKZLOOHYROYHFRQWH[WXDOO\DQGµHPHUJH¶LQVLWXRQO\DIWHUVRPH
ILHOGZRUNKDVEHHQGRQH
0DLQSKDVHVRIDQHWKQRJUDSKLFVWXG\
$QHWKQRJUDSKLFVWXG\LQYROYHVDFRPSOH[SURFHVVRIµJHWWLQJLQ¶DQGµFRPLQJRXW¶
ZKLFKFDQEHGHVFULEHGDVDVHTXHQFHRIIRXUUHODWLYHO\GLVWLQFWSKDVHV 0RUVHDQG
5LFKDUGVVHHDOVR5LFKDUGV
x 7KHILUVWSKDVHLQYROYHVHQWHULQJZKDWLVIRUWKHUHVHDUFKHUDVWUDQJHHQYLURQPHQW
7KHHWKQRJUDSKHUQHHGVWRQHJRWLDWHWKHHQWU\ZLWKWKHJDWHNHHSHUVDQGWKHQILQGD
ZD\UHDVRQRUDGHTXDWHUROHIRUILWWLQJLQ,Q/UHVHDUFKJDWHNHHSHUVDUHOLNHO\WR
EHKHDGWHDFKHUVVFKRROPDQDJHUVSULQFLSDOVYDULRXVHGXFDWLRQRIILFLDOVZKRDV
5LFKDUGV SRLQWVRXWDOOKDYHµWKHLURZQSDUWLFXODUD[HVWRJULQGDQG
WHUULWRULHVWRSURWHFW¶7KHUHIRUHWKLVLVDUDWKHUGHOLFDWHSKDVHWKHUHVHDUFKHULV
XQGHUVWDQGDEO\VRPHZKDWORVWDQGGRHVQRWXQGHUVWDQGWKHVHWWLQJRUWKH
SDUWLFLSDQWVDVDFRQVHTXHQFHRIZKLFKGDWDFROOHFWLRQLQWKLVSKDVHODUJHO\
LQYROYHVµPDSSLQJWKHWHUUDLQ¶GHFLGLQJZKR¶VZKRDQGLQJHQHUDONHHSLQJD
GLDU\ZLWKILHOGQRWHV
x 7KHVHFRQGSKDVHLVHDVLHULQPDQ\ZD\VWKDQWKHILUVWEHFDXVHE\QRZWKHLFHKDV
EHHQEURNHQDQGWKHUHVHDUFKHUKDVEHFRPHIDPLOLDUZLWKWKHSDUWLFLSDQWVDQGWKH
URXWLQHVLQWKHVHWWLQJ1RQSDUWLFLSDQWREVHUYDWLRQLVQRZLQIXOOVZLQJDQGWKH
FULWLFDOWDVNRIILQGLQJNH\LQIRUPDQWVEHJLQVDORQJZLWKFRQGXFWLQJLQLWLDO
LQWHUYLHZVZLWKWKHP$WWKLVVWDJHWKHHWKQRJUDSKHUDOVRQHHGVWRVWDUWDQDO\VLQJ
WKHSUHOLPLQDU\GDWDWRGHYHORSVRPHLQLWLDOLGHDVDQGFRQFHSWV
x 7KHWKLUGSKDVHLVWKHPRVWSURGXFWLYHUHVHDUFKSKDVH$FFXOWXUDWLRQKDVEHHQ
FRPSOHWHGDQGWKHUHVHDUFKHUKDVEHHQDFFHSWHGDQGQRZIHHOVµDWKRPH¶LQWKH
VHWWLQJ7KLVDOORZVKLPKHUWRHPSOR\DYDULHW\RIWHFKQLTXHVWRFROOHFW
LQFUHDVLQJO\IRFXVHGGDWDZKLFKLVXVHGWRYHULI\LQLWLDOK\SRWKHVHVDQGLGHDVDQG
WRGHYHORSEURDGHUWKHRUHWLFDOFRQFHSWV'DWDDQDO\VLVDWWKLVVWDJHLVFKDUDFWHUL]HG

E\µSURJUHVVLYHIRFXVLQJ¶LQYROYLQJVLIWLQJVRUWLQJDQGUHYLHZLQJWKHGDWD &RKHQ
HWDO
x 7KHILQDOSKDVHLVWKHQHFHVVDU\ZLWKGUDZDO7KLVPD\EHDQHPRWLRQDOO\WD[LQJ
FORVXUHSKDVHHYRNLQJDIHHOLQJRIORVVDQGWKHHWKQRJUDSKHUQHHGVWRPDNHVXUH
WKDWKHVKHGLVHQJDJHVIURPWKHILHOGLQDZD\WKDWEULQJVDVOLWWOHGLVUXSWLRQWRWKH
JURXSRUVLWXDWLRQDVSRVVLEOH7KHIRFXVLVRQGDWDDQDO\VLVDQGDGGLWLRQDOGDWDLV
FROOHFWHGRQO\WRILOOJDSVWRUHVROYHDPELJXLWLHVDQGWRYDOLGDWHSUHYLRXVILQGLQJV
6WUHQJWKVDQGZHDNQHVVHVRIHWKQRJUDSKLFUHVHDUFK
7KHHWKQRJUDSKLFDSSURDFKLVSDUWLFXODUO\XVHIXOIRUH[SORULQJXQFKDUWHGWHUULWRULHVDQG
XQGHUVWDQGLQJVRFLDOSURFHVVHVIURPWKHSDUWLFLSDQWV¶SHUVSHFWLYH,WLVDQH[FHOOHQWZD\
RIµFURVVLQJFXOWXUHV¶DQGJDLQLQJLQVLJKWLQWRWKHOLIHRIRUJDQL]DWLRQVLQVWLWXWLRQVDQG
FRPPXQLWLHV,QVKRUWHWKQRJUDSK\LVLGHDOIRUJHQHUDWLQJLQLWLDOK\SRWKHVHVDERXW
VRPHWKLQJWRWDOO\XQNQRZQ+RUQEHUJHU DOVRKLJKOLJKWVWKHFDSDFLW\RI
HWKQRJUDSK\WRWDNHDKROLVWLFYLHZDQGIRFXVRQWKHZKROHSLFWXUHWKDWµOHDYHVQRWKLQJ
XQDFFRXQWHGIRUDQGWKDWUHYHDOVWKHLQWHUUHODWHGQHVVRIDOOWKHFRPSRQHQWSDUWV¶$V
VKHJRHVRQWRDUJXH
7KHYDOXHKHUHLVWKDWWKHDSSURDFKDOORZVLQGHHGLWLVWKHYHU\HVVHQFHRIWKH
DSSURDFKWRHQVXUHFRPSDULVRQDQGFRQWUDVWEHWZHHQZKDWSHRSOHVD\DQGZKDW
SHRSOHGRLQDJLYHQFRQWH[WDQGDFURVVFRQWH[WVLQRUGHUWRDUULYHDWDIXOOHU
UHSUHVHQWDWLRQRIZKDWLVJRLQJRQ,WLVQRWHQRXJKIRUHWKQRJUDSKHUVWRDVN
WHDFKHUVDERXWWKHLUFRPPXQLFDWLYHDSSURDFKWR(6/WHDFKLQJWKH\PXVWDOVR
REVHUYHLWLQDFWLRQ,WLVQRWHQRXJKWRVXUPLVHWKDWVWXGHQWSDUWLFLSDWLRQSDWWHUQV
LQFODVVDUHGLIIHUHQWIURPWKRVHWKHFKLOGUHQH[SHULHQFHLQKRPHVRFLDOL]DWLRQ
HWKQRJUDSKHUVVHHNWRREVHUYHERWKFRQWH[WVIRUWKHPVHOYHV,WLVE\FRPSDULQJ
DQGFRQWUDVWLQJWKHVHGLPHQVLRQVWKDWDUHDOLVWLFDQGPXOWLOD\HUHGGHVFULSWLRQFDQ
EHJLQWRHPHUJH SS±
7KHPDLQGUDZEDFNRIWKHDSSURDFKLVWKDWWKHQHHGIRUSURORQJHGHQJDJHPHQWZLWK
WKHSDUWLFLSDQWVLQWKHLUQDWXUDOVHWWLQJUHTXLUHVDQH[WHQVLYHWLPHLQYHVWPHQWWKDWIHZ
DFDGHPLFUHVHDUFKHUVFDQDIIRUG+RUQEHUJHU DOVRPHQWLRQVDIXUWKHUOLPLWDWLRQ
WKHµLQVLGHURXWVLGHUGLOHPPD¶ZKLFKFRQFHUQVWKHGLIILFXOW\RIVWULNLQJDEDODQFH
EHWZHHQLQVLGHUDQGRXWVLGHUSHUVSHFWLYHV$VVKHDUJXHVWKLVWHQVLRQVXUIDFHVLQ
VHYHUDOJXLVHV IRUH[DPSOHLQWKHH[WHQWRIUHVHDUFKHUSDUWLFLSDWLRQYHUVXV
QRQSDUWLFLSDQWREVHUYDWLRQDQGFORVHQHVVYHUVXVGHWDFKPHQW DQGLWLVSDUWLFXODUO\DFXWH
LQHWKQRJUDSKLFUHVHDUFKE\WHDFKHUVLQWKHLURZQFODVVURRPVDQGE\PLQRULW\
UHVHDUFKHUVLQWKHLURZQFRPPXQLWLHV:LGGRZVRQ SHUVRQDOFRPPXQLFDWLRQ DGGVWKDW
WKHLQVLGHURXWVLGHUGLOHPPDLVIXUWKHUPDQLIHVWHGLQUHSRUWLQJWKHUHVHDUFKUHVXOWVWR
H[WHUQDODXGLHQFHVEHFDXVHWKLVSURFHVVLQKHUHQWO\LQYROYHVSUHVHQWLQJLQVLGHU
SHUVSHFWLYHVLQRXWVLGHUWHUPVDQGWKLVWUDQVODWLRQLVOLNHO\WRLQYROYHDOWHUDWLRQV
)LQDOO\+DUNODX GUDZVDWWHQWLRQWRDSHFXOLDULW\FRQFHUQLQJHWKQRJUDSK\LQ

DSSOLHGOLQJXLVWLFVQDPHO\WKDWHWKQRJUDSKLFZRUNUHPDLQVODUJHO\OLPLWHGWRZKLWH

DQJORSKRQHUHVHDUFKHUVLQ(QJOLVKVSHDNLQJFRXQWULHVZLWK(QJOLVKUHPDLQLQJWKH
WDUJHWODQJXDJHLQWKHYDVWPDMRULW\RIVWXGLHV:KLOHWKLVPD\LQGHHGEHWUXHRIWKH
FXUUHQWSUDFWLFHWKHVLWXDWLRQLVOLNHO\WRFKDQJHLQWKHIXWXUHEHFDXVHHWKQRJUDSKLF
UHVHDUFKLVLQWKHSURFHVVRIJDLQLQJIXUWKHUJURXQGLQDSSOLHGOLQJXLVWLFVERWKLQWHUPV
RIVFRSHDQGTXDQWLW\)XUWKHUPRUHHWKQRJUDSK\KDVDOVREHHQSOD\LQJDQLPSRUWDQW
UROHLQLQIOXHQFLQJVHYHUDODVSHFWVRITXDOLWDWLYHUHVHDUFKLQJHQHUDOSURGXFLQJYDULRXV
IUXLWIXOµTXDVLHWKQRJUDSKLF¶DSSURDFKHV
,QWHUYLHZV
7KHVHFRQGTXDOLWDWLYHPHWKRGRILQTXLU\WREHGHVFULEHGLQWKLVFKDSWHULVFRQGXFWLQJ
LQWHUYLHZV,QWHUYLHZLQJLVDIUHTXHQWSDUWRIWKHVRFLDOOLIHVXUURXQGLQJPRVWRIXV:H
FDQKHDULQWHUYLHZVRQWKHUDGLRZDWFKSHRSOHEHLQJLQWHUYLHZHGRQ79DQGZH
RXUVHOYHVRIWHQSDUWLFLSDWHLQLQWHUYLHZVRIYDULRXVW\SHVHLWKHUDVLQWHUYLHZHUVRU
LQWHUYLHZHHV$V0LOOHUDQG&UDEWUHH SRLQWRXWWKHLQWHUYLHZJHQUHZLWKLWVWXUQ
WDNLQJFRQYHQWLRQVDQGH[SHFWDWLRQVIRUSDUWLFLSDQWUROHVHWLTXHWWHVDQGHYHQOLQJXLVWLF
SKUDVHVLVXVXDOO\VKDUHGFXOWXUDONQRZOHGJH,WLVH[DFWO\EHFDXVHLQWHUYLHZLQJLVD
NQRZQFRPPXQLFDWLRQURXWLQHWKDWWKHPHWKRGZRUNVVRZHOODVDYHUVDWLOHUHVHDUFK
LQVWUXPHQW²LQIDFWDOWKRXJKWKHUHLVDUDQJHRITXDOLWDWLYHUHVHDUFKWHFKQLTXHV
DYDLODEOHIRUUHVHDUFKHUVWKHLQWHUYLHZLVWKHPRVWRIWHQXVHGPHWKRGLQTXDOLWDWLYH
LQTXLULHV,WLVUHJXODUO\DSSOLHGLQDYDULHW\RIDSSOLHGOLQJXLVWLFFRQWH[WVIRUGLYHUVH
SXUSRVHV 6HHIRUH[DPSOH%ORFN5XELR5LFKDUGV
7KHUHDUHGLIIHUHQWW\SHVRILQWHUYLHZVWKHKLJKO\VWUXFWXUHGYHUVLRQVKDUHVPDQ\
VLPLODULWLHVZLWKTXDQWLWDWLYHZULWWHQTXHVWLRQQDLUHVDQGWKHUHIRUHLWZLOOEHGLVFXVVHG
KHUHRQO\YHU\EULHIO\7KHW\SLFDOTXDOLWDWLYHLQWHUYLHZLVDRQHWRRQHµSURIHVVLRQDO
FRQYHUVDWLRQ¶ .YDOH WKDWKDVDVWUXFWXUHDQGDSXUSRVHµWRREWDLQGHVFULSWLRQV
RIWKHOLIHZRUOGRIWKHLQWHUYLHZHHZLWKUHVSHFWWRLQWHUSUHWLQJWKHPHDQLQJRIWKH
GHVFULEHGSKHQRPHQD¶ SS± ²WKLVVHFWLRQZLOOFRQFHUQSULPDULO\WKLVLQWHUYLHZ
W\SH,QVXEVHTXHQWVHFWLRQV 6HFWLRQVDQG ZHZLOODOVRORRNDWWZR
VSHFLDOL]HGLQWHUYLHZLQJWHFKQLTXHVµIRFXVJURXSLQWHUYLHZV¶ZKLFKLQYROYHDJURXS
IRUPDWDQGµUHWURVSHFWLYHLQWHUYLHZV¶ZKLFKFRPHXQGHUWKHXPEUHOODWHUPRI
µLQWURVSHFWLYHPHWKRGV¶
$IWHUUHYLHZLQJWKHPDLQLQWHUYLHZW\SHVWKLVVHFWLRQZLOOIRFXVODUJHO\RQWKHWZR
SUDFWLFDODVSHFWVRIFRQGXFWLQJLQWHUYLHZUHVHDUFK D KRZWRSUHSDUHWKHµLQWHUYLHZ
JXLGH¶ LHWKHOLVWRITXHVWLRQVWRXVHGXULQJWKHLQWHUYLHZ DQG E KRZWRFRQGXFWWKH
DFWXDOLQWHUYLHZ )RUIXUWKHUGLVFXVVLRQVVHH.YDOH3DWWRQ5LFKDUGV

0DLQW\SHVRILQWHUYLHZV
2QHWRRQHLQWHUYLHZVFDQEHGLYLGHGLQWRGLIIHUHQWW\SHVDFFRUGLQJWRWKHGHJUHHRI
VWUXFWXUHLQWKHSURFHVVDQGZKHWKHUWKHUHDUHVLQJOHRUPXOWLSOHLQWHUYLHZVHVVLRQV/HW
XVVWDUWZLWKWKHODWWHUOHVVRIWHQPHQWLRQHGDVSHFW

6LQJOHRUPXOWLSOHVHVVLRQV
7KHW\SLFDOTXDOLWDWLYHLQWHUYLHZLVDRQHRIIHYHQWODVWLQJDERXW±PLQXWHV
+RZHYHUDV3RONLQJKRUQH DUJXHVRQHVKRWLQWHUYLHZVDUHUDUHO\DEOHWR
SURGXFHWKHIXOODQGULFKGHVFULSWLRQVQHFHVVDU\IRUZRUWKZKLOHILQGLQJV'UDZLQJRQ
6HLGPDQ¶VZRUNKHUHFRPPHQGVWKDWUHVHDUFKHUVDGPLQLVWHUDVHTXHQFHRIWKUHH
LQWHUYLHZVZLWKWKHVDPHSDUWLFLSDQWWRREWDLQVXIILFLHQWGHSWKDQGEUHDGWK7KHILUVW
LQWHUYLHZEUHDNVWKHLFHDQGGHYHORSVUDSSRUWZKLOHDOVRSURYLGLQJDTXLFNVZHHSRI
WKHDUHDVWREHLQYHVWLJDWHGODWHU7KHLQWHUYDOEHWZHHQWKHILUVWDQGWKHVHFRQG
LQWHUYLHZVDOORZVWKHLQWHUYLHZHUWRSUHSDUHDPRUHPDGHWRPHDVXUHLQWHUYLHZJXLGH
IRUWKHVHFRQGVHVVLRQDQGLWDOVRRIIHUVWKHLQWHUYLHZHHWKHFKDQFHWRWKLQNPRUHGHHSO\
DERXWWKHH[SHULHQFH$VDUHVXOWWKHVHFRQGLQWHUYLHZLVPRUHIRFXVHGWKDQWKHILUVW
)LQDOO\KDYLQJDQDO\VHGWKHWUDQVFULSWVRIWKHILUVWWZRVHVVLRQVLQWKHWKLUGLQWHUYLHZ
WKHUHVHDUFKHUFDQDVNDQ\µPRSXS¶RUIROORZXSTXHVWLRQVWRILOOLQDQGWRFODULI\WKH
DFFRXQW
,WPXVWEHQRWHGWKDWWKHPXOWLSOHVHVVLRQIRUPDWRXWOLQHGKHUHLVQRWWKHVDPHDVD
ORQJLWXGLQDOLQWHUYLHZVWXG\ VHH6HFWLRQ EHFDXVHWKHSXUSRVHRIWKHWKUHHVHVVLRQV
LVQRWWRGRFXPHQWWHPSRUDOFKDQJHVEXWWRDUULYHDWDIXOODFFRXQW,QDORQJLWXGLQDO
LQWHUYLHZVWXG\WKHPXOWLSOHVHVVLRQVZRXOGQHHGWREHRUJDQL]HGGLIIHUHQWO\ZLWKWKH
ILUVWRQHRUWZRLQWHUYLHZVFUHDWLQJWKHEDVHOLQHNQRZOHGJHDQGWKHVXEVHTXHQW
UHJXODUO\RFFXUULQJLQWHUYLHZVIRFXVLQJRQKRZDQGZK\WKHSDUWLFXODUSKHQRPHQRQ
XQGHUVWXG\FKDQJHV
6WUXFWXUHGLQWHUYLHZV
7KHVHFRQGPDLQFDWHJRUL]LQJSULQFLSOHRILQWHUYLHZVLVWKHGHJUHHRIVWUXFWXUHLQWKHP
ZLWKRQHH[WUHPHEHLQJWKHµVWUXFWXUHGLQWHUYLHZ¶,QWKLVIRUPDWWKHUHVHDUFKHUFORVHO\
IROORZVDSUHSUHSDUHGHODERUDWHµLQWHUYLHZVFKHGXOHJXLGH¶ZKLFKFRQWDLQVDOLVWRI
TXHVWLRQVWREHFRYHUHGFORVHO\ZLWKHYHU\LQWHUYLHZHHDQGWKHHOLFLWHGLQIRUPDWLRQ
VKDUHVPDQ\RIWKHDGYDQWDJHV IRUH[DPSOHFRPSDUDELOLW\DFURVVSDUWLFLSDQWV DQG
GLVDGYDQWDJHV IRUH[DPSOHOLPLWHGULFKQHVV RITXHVWLRQQDLUHGDWD6XFKWLJKWO\
FRQWUROOHGLQWHUYLHZVHQVXUHWKDWWKHLQWHUYLHZHHIRFXVHVRQWKHWDUJHWWRSLFDUHDDQG
WKDWWKHLQWHUYLHZFRYHUVDZHOOGHILQHGGRPDLQZKLFKPDNHVWKHDQVZHUVFRPSDUDEOH
DFURVVGLIIHUHQWUHVSRQGHQWV7KHRWKHUVLGHRIWKHFRLQLVKRZHYHUWKDWLQDVWUXFWXUHG
LQWHUYLHZWKHUHLVJHQHUDOO\OLWWOHURRPIRUYDULDWLRQRUVSRQWDQHLW\LQWKHUHVSRQVHV
EHFDXVHWKHLQWHUYLHZHULVWRUHFRUGWKHUHVSRQVHVDFFRUGLQJWRDFRGLQJVFKHPH7KHUH
LVDOVRYHU\OLWWOHIOH[LELOLW\LQWKHZD\TXHVWLRQVDUHDVNHGEHFDXVHE\DGRSWLQJD
VWDQGDUGL]HGIRUPDWLWLVKRSHGWKDWQRWKLQJZLOOEHOHIWWRFKDQFH7KLVLQWHUYLHZW\SH
LVDSSURSULDWHZKHQWKHUHVHDUFKHULVDZDUHRIZKDWKHVKHGRHVQRWNQRZDQGFDQ
IUDPHTXHVWLRQVWKDWZLOO\LHOGWKHQHHGHGDQVZHUV7KDWLVVWUXFWXUHGLQWHUYLHZVDUH
XVHGLQVLWXDWLRQVZKHUHDZULWWHQTXHVWLRQQDLUHZRXOGLQWKHRU\EHDGHTXDWHH[FHSW
WKDWIRUVRPHUHDVRQWKHZULWWHQIRUPDWLVQRWIHDVLEOH IRUH[DPSOHEHFDXVHRIWKHORZ
OHYHORIOLWHUDF\DPRQJVWWKHSDUWLFLSDQWVRUWKHQHHGIRUWLJKWHUFRQWURODVLQVRPH
PDUNHWUHVHDUFKVXUYH\VRURSLQLRQSROOV

8QVWUXFWXUHGLQWHUYLHZV
7KHRWKHUH[WUHPHWKHµXQVWUXFWXUHGLQWHUYLHZ¶ VRPHWLPHVDOVRUHIHUUHGWRDVWKH
µHWKQRJUDSKLFLQWHUYLHZ¶ DOORZVPD[LPXPIOH[LELOLW\WRIROORZWKHLQWHUYLHZHHLQ
XQSUHGLFWDEOHGLUHFWLRQVZLWKRQO\PLQLPDOLQWHUIHUHQFHIURPWKHUHVHDUFKDJHQGD7KH
LQWHQWLRQLVWRFUHDWHDUHOD[HGDWPRVSKHUHLQZKLFKWKHUHVSRQGHQWPD\UHYHDOPRUH
WKDQKHVKHZRXOGLQIRUPDOFRQWH[WVZLWKWKHLQWHUYLHZHUDVVXPLQJDOLVWHQLQJUROH
1RGHWDLOHGLQWHUYLHZJXLGHLVSUHSDUHGLQDGYDQFHDOWKRXJKWKHUHVHDUFKHUXVXDOO\
WKLQNVRIDIHZ RQHWRVL[ RSHQLQJTXHVWLRQV VRPHWLPHVFDOOHGµJUDQGWRXU¶TXHVWLRQV
WRHOLFLWWKHLQWHUYLHZHH¶VVWRU\'XULQJWKHLQWHUYLHZWKHUHVHDUFKHUPD\DVNDQ
RFFDVLRQDOTXHVWLRQIRUFODULILFDWLRQDQGPD\JLYHUHLQIRUFHPHQWIHHGEDFNDVDQ\JRRG
FRPPXQLFDWLRQSDUWQHUZRXOGWRNHHSWKHLQWHUYLHZPRYLQJEXWLQWHUUXSWLRQVDUHNHSW
WRDPLQLPXP
,WLVHDV\WRVHHWKDWIRUDQXQVWUXFWXUHGLQWHUYLHZWREHVXFFHVVIXOLWLVLQGLVSHQVDEOH
WKDWWKHLQWHUYLHZHUHVWDEOLVKHVYHU\JRRGUDSSRUWZLWKWKHLQWHUYLHZHH7KLVNLQGRI
LQWHUYLHZLVPRVWDSSURSULDWHZKHQDVWXG\IRFXVHVRQWKHGHHSPHDQLQJRISDUWLFXODU
SKHQRPHQDRUZKHQVRPHSHUVRQDOKLVWRULFDODFFRXQWRIKRZDSDUWLFXODUSKHQRPHQRQ
KDVGHYHORSHGLVUHTXLUHG,QGHSWKLQWHUYLHZVFDQDOVREHXVHGZKHUHH[SORUDWRU\ZRUN
LVUHTXLUHGEHIRUHDPRUHIRFXVHG IRUH[DPSOHTXDQWLWDWLYH VWXG\FDQEHFDUULHGRXW
6HPLVWUXFWXUHGLQWHUYLHZV
,QDSSOLHGOLQJXLVWLFUHVHDUFKPRVWLQWHUYLHZVFRQGXFWHGEHORQJWRWKHµVHPLVWUXFWXUHG
LQWHUYLHZ¶W\SHZKLFKRIIHUVDFRPSURPLVHEHWZHHQWKHWZRH[WUHPHV$OWKRXJKWKHUH
LVDVHWRISUHSUHSDUHGJXLGLQJTXHVWLRQVDQGSURPSWVWKHIRUPDWLVRSHQHQGHGDQG
WKHLQWHUYLHZHHLVHQFRXUDJHGWRHODERUDWHRQWKHLVVXHVUDLVHGLQDQH[SORUDWRU\
PDQQHU,QRWKHUZRUGVWKHLQWHUYLHZHUSURYLGHVJXLGDQFHDQGGLUHFWLRQ KHQFHWKHµ
VWUXFWXUHG¶SDUWLQWKHQDPH EXWLVDOVRNHHQWRIROORZXSLQWHUHVWLQJGHYHORSPHQWVDQG
WROHWWKHLQWHUYLHZHHHODERUDWHRQFHUWDLQLVVXHV KHQFHWKHµVHPL¶SDUW %HFDXVHRI
WKHJUHDWSRSXODULW\RIWKLVLQWHUYLHZIRUPDWPRVWRIWKHIROORZLQJUHFRPPHQGDWLRQV
RQTXHVWLRQZRUGLQJDQGLQWHUYLHZFRQGXFWZLOOEHJHDUHGDWVHPLVWUXFWXUHGLQWHUYLHZV
LQSDUWLFXODU
7KHVHPLVWUXFWXUHGLQWHUYLHZLVVXLWDEOHIRUFDVHVZKHQWKHUHVHDUFKHUKDVDJRRG
HQRXJKRYHUYLHZRIWKHSKHQRPHQRQRUGRPDLQLQTXHVWLRQDQGLVDEOHWRGHYHORS
EURDGTXHVWLRQVDERXWWKHWRSLFLQDGYDQFHEXWGRHVQRWZDQWWRXVHUHDG\PDGH
UHVSRQVHFDWHJRULHVWKDWZRXOGOLPLWWKHGHSWKDQGEUHDGWKRIWKHUHVSRQGHQW¶VVWRU\
7KLVIRUPDWWKHUHIRUHQHHGVDQµLQWHUYLHZJXLGH¶ZKLFKKDVWREHPDGHDQGSLORWHGLQ
DGYDQFH8VXDOO\WKHLQWHUYLHZHUZLOODVNWKHVDPHTXHVWLRQVRIDOORIWKHSDUWLFLSDQWV
DOWKRXJKQRWQHFHVVDULO\LQWKHVDPHRUGHURUZRUGLQJDQGZRXOGVXSSOHPHQWWKHPDLQ
TXHVWLRQVZLWKYDULRXVSUREHV²VHHEHORZ
3UHSDULQJIRUWKHLQWHUYLHZDQGGHVLJQLQJWKHLQWHUYLHZ
JXLGH

7KHFRPSOHWHLQWHUYLHZSURFHVVLQYROYHVDVHULHVRIFDUHIXOO\GHVLJQHGVWHSVZLWKWKH
SUHSDUDWLRQVWDUWLQJZHOOEHIRUHWKHILUVWLQWHUYLHZVHVVLRQ$IWHUWKHLQLWLDOVDPSOLQJ
SODQKDVEHHQILQDOL]HGDQGHWKLFDOLVVXHVVXFKDVLQIRUPHGFRQVHQWKDYHEHHQ
FRQVLGHUHGWKHUHVHDUFKHUQHHGVWRSUHSDUHDGHWDLOHGLQWHUYLHZJXLGHZKLFKZLOOVHUYH
DVWKHPDLQUHVHDUFKLQVWUXPHQW$OWKRXJKZHPD\EHOLHYHWKDWWKLVLVDVWUDLJKWIRUZDUG
DQGTXLFNMRE²DIWHUDOODOOZHQHHGWRGRLVMRWGRZQDIHZUHOHYDQWTXHVWLRQV²DJRRG
LQWHUYLHZJXLGHUHTXLUHVFDUHIXOSODQQLQJIROORZHGE\VRPHSLORWLQJ$IHZWULDOUXQV
FDQHQVXUHWKDWWKHTXHVWLRQVHOLFLWVXIILFLHQWO\ULFKGDWDDQGGRQRWGRPLQDWHWKHIORZ
RIWKHFRQYHUVDWLRQ
7KHPDLQIXQFWLRQRIWKHLQWHUYLHZJXLGH RUµLQWHUYLHZVFKHGXOHSURWRFRO¶ LVWR
KHOSWKHLQWHUYLHZHULQDQXPEHURIDUHDV D E\HQVXULQJWKDWWKHGRPDLQLVSURSHUO\
FRYHUHGDQGQRWKLQJLPSRUWDQWLVOHIWRXWE\DFFLGHQW E E\VXJJHVWLQJDSSURSULDWH
TXHVWLRQZRUGLQJV F E\RIIHULQJDOLVWRIXVHIXOSUREHTXHVWLRQVWREHXVHGLIQHHGHG
G E\RIIHULQJDWHPSODWHIRUWKHRSHQLQJVWDWHPHQWDQG H E\OLVWLQJVRPHFRPPHQWV
WREHDULQPLQG,WPLJKWEHDGYLVDEOHWRFRPELQHWKLVJXLGHZLWKDQµLQWHUYLHZORJ¶DQG
WKXVOHDYHVSDFHLQLWIRUUHFRUGLQJWKHGHWDLOVRIWKHLQWHUYLHZ IRUH[DPSOHSDUWLFLSDQW
VHWWLQJOHQJWK DVZHOODVIRUWKHLQWHUYLHZHU¶VFRPPHQWVDQGQRWHV$V0F&UDFNHQ
SRLQWVRXWWKHXVHRIDQLQWHUYLHZJXLGHLVVRPHWLPHVUHJDUGHGDVD
GLVFUHWLRQDU\PDWWHULQTXDOLWDWLYHLQWHUYLHZVWKLVKHDUJXHVLVZURQJEHFDXVHGXHWR
WKHGHPDQGLQJREMHFWLYHVZHZDQWWRDFKLHYHDQGWKHPXOWLSOHIDFWRUVZHKDYHWRDWWHQG
WRGXULQJWKHLQWHUYLHZWKHLQWHUYLHZJXLGHLVLQGLVSHQVDEOH,QGHHGDVDQ\ERG\ZKR
KDVGRQHLQWHUYLHZLQJZLOOFRQILUPWKHLQWHUYLHZHUFDQGRZLWKHYHU\ELWRIKHOSGXULQJ
WKHLQWHUYLHZLQJSURFHVVDQGWKHLQWHUYLHZJXLGHRIIHUVXVWKHEHVWSRVVLEOHKHOSLQWKLV
UHVSHFW
4XHVWLRQW\SHVDQGZRUGLQJLVVXHV
7KHUHDUHDYDULHW\RITXHVWLRQVZHFDQLQFOXGHLQWKHLQWHUYLHZJXLGHEXWZHQHHGWR
EHDULQPLQGWKDWWKHVHRQO\SURYLGHWKHIUDPHZRUNDQGWKHUHDOPHDQLQJLVVRRIWHQ
XQFRYHUHGWKURXJKH[SORUDWRU\DQGXQVWUXFWXUHGUHVSRQVHVWKDWGHYLDWHIURPWKH
LQWHUYLHZVFKHGXOH
x 7KHILUVWIHZTXHVWLRQV7KHVHDUHSDUWLFXODUO\LPSRUWDQWLQDQLQWHUYLHZQRWVR
PXFKIURPWKHFRQWHQWSRLQWRIYLHZEXWUDWKHUEHFDXVHWKH\VHWWKHWRQHDQGFUHDWH
LQLWLDOUDSSRUW,IWKHLQWHUYLHZHHVIHHOWKDWWKH\FDQGRWKHPVHOYHVMXVWLFHZKHQ
DQVZHULQJWKHVHLQLWLDOTXHVWLRQVWKLVZLOOPDNHWKHPIHHOFRPSHWHQWKHOSWKHPWR
UHOD[DQGFRQVHTXHQWO\HQFRXUDJHWKHPWRRSHQXS7KLVLVZK\UHVHDUFKHUVRIWHQ
VWDUWZLWKHDV\SHUVRQDORUIDFWXDOTXHVWLRQV IRUH[DPSOHDERXWWKHUHVSRQGHQW¶V
IDPLO\RUMRE 7KHTXDOLW\RIWKHVXEVHTXHQWUHVSRQVHVZLOOWRDODUJHH[WHQW
GHSHQGRQWKHFOLPDWHRIWUXVWWKDWZHFUHDWHLQWKLVLQLWLDOLFHEUHDNLQJSHULRG
x &RQWHQWTXHVWLRQV3DWWRQ SRLQWVRXWWKDWRQDQ\JLYHQWRSLFLWLVSRVVLEOHWR
DVNDQ\RIVL[PDLQW\SHVRITXHVWLRQIRFXVLQJRQ D H[SHULHQFHVDQGEHKDYLRXUV
E RSLQLRQVDQGYDOXHV F IHHOLQJV G NQRZOHGJH H VHQVRU\LQIRUPDWLRQ LH

ZKDWVRPHRQHKDVVHHQKHDUGWDVWHGVPHOOHGHWFDQGHYHQZKDWWKHLQWHUYLHZHU
ZRXOGKDYHVHHQRUKHDUGLIKHVKHKDGEHHQLQDSDUWLFXODUSODFH I EDFNJURXQG
RUGHPRJUDSKLFLQIRUPDWLRQ7KHVHVL[FDWHJRULHVFRQFHUQGLIIHUHQWDVSHFWVRIWKH
SDUWLFLSDQW¶VRYHUDOOYLHZH[SHULHQFHRIWKHSKHQRPHQRQDQGWKHUHIRUHZHFDQJHWD
URXQGHGSLFWXUHE\LQFOXGLQJLQRXULQWHUYLHZJXLGHTXHVWLRQVWKDWWDSLQWRHDFK
GLPHQVLRQ
x 3UREHV7KHHPHUJHQWQDWXUHRITXDOLWDWLYHLQWHUYLHZGDWDFDQEHHQKDQFHGE\XVLQJ
YDULRXVSUREHVWKDWXVHZKDWWKHLQWHUYLHZHHKDVVDLGDVDVWDUWLQJSRLQWWRJR
IXUWKHUDQGWRLQFUHDVHWKHULFKQHVVDQGGHSWKRIWKHUHVSRQVHV3UREHVPD\LQFOXGH
GHWDLORULHQWHGDQGFODULILFDWLRQTXHVWLRQVEXWDWHFKQLTXHRIWHQXVHGLQSHUVRQ
FHQWUHGSV\FKRWKHUDS\LVWRVLPSO\WDNHDVDOLHQWFRQWHQWZRUGXVHGE\WKH
UHVSRQGHQWDQGDVNWRHODERUDWH µ<RXKDYHXVHGWKHZRUGµIUHHGRP¶WZLFH²ZKDW
H[DFWO\GRHVLWPHDQWR\RXGR\RXPHDQE\WKDW«"¶ 3DWWRQ DOVRPHQWLRQV
DQLQWHUHVWLQJSUREHWKHµFRQWUDVWSUREH¶ZKLFKDVNVDERXWKRZDSDUWLFXODU
H[SHULHQFHIHHOLQJDFWLRQWHUPFRPSDUHVWRVRPHRWKHUVLPLODUFRQFHSW
x 7KHILQDOFORVLQJTXHVWLRQ7KLVSHUPLWVWKHLQWHUYLHZHHWRKDYHWKHILQDOVD\
6HYHUDOVFKRODUVKDYHQRWHGLQWKHOLWHUDWXUHWKHULFKQHVVRIWKHGDWDWKDWVLPSOH
FORVLQJTXHVWLRQVVXFKDVWKHIROORZLQJRQHVFDQ\LHOGµ,VWKHUHDQ\WKLQJHOVH\RX
ZRXOGOLNHWRDGG"¶RUµ:KDWVKRXOG,KDYHDVNHG\RXWKDW,GLGQ¶WWKLQNWRDVN"¶
:LWKUHJDUGWRWKHZRUGLQJRIWKHTXHVWLRQVDVNHGLQDTXDOLWDWLYHLQWHUYLHZWKH
OLWHUDWXUHFRQWDLQVDJUHDWGHDORIDGYLFH6RPHRIWKLVDGYLFHLVVHOIHYLGHQW IRU
H[DPSOHµ'RQ¶WXVHZRUGVWKDWWKHLQWHUYLHZHHGRHVQRWXQGHUVWDQG¶ EXWVRPHRWKHU
VXJJHVWLRQVDUHWUXO\SUDFWLFDODQGXVHIXO²3DWWRQ IRUH[DPSOHRIIHUV
SDUWLFXODUO\GHWDLOHGJXLGHOLQHV6LPLODUWRWKHZRUGLQJRIZULWWHQTXHVWLRQQDLUHLWHPV
VHH6HFWLRQ WKHUHDUHVRPHUXOHVRIWKXPEDERXWKRZWRIRUPXODWHRXU
TXHVWLRQV7ZRSDUWLFXODUO\LPSRUWDQWUXOHVDUHWKDWZHVKRXOGDYRLG D OHDGLQJ
TXHVWLRQV µ,WZDVIUXVWUDWLQJZDVQ¶WLW«"¶ DQG E ORDGHGRUDPELJXRXVZRUGVDQG
MDUJRQ,QJHQHUDOVKRUWDQGUHODWLYHO\VLPSOHTXHVWLRQVWKDWFRQWDLQRQO\RQHLGHDZRUN
EHVW8VLQJZRUGVWKDWPDNHVHQVHWRWKHLQWHUYLHZHHDQGUHIOHFWKLVKHUZRUOGYLHZKHOS
WRFRQQHFWWRWKHUHVSRQGHQWDQGLPSURYHWKHTXDOLW\RIWKHLQWHUYLHZGDWD
&RQGXFWLQJWKHLQWHUYLHZ
:KLOHLWLVWUXHWKDWSUDFWLFHXVXDOO\PDNHVXVPRUHUHOD[HGDQGSROLVKHGLQWHUYLHZHUV,
KDYHIRXQGWKDWVLPSO\EHLQJIDPLOLDUZLWKFHUWDLQNH\LVVXHVUHJDUGLQJLQWHUYLHZ
FRQGXFWLVDOUHDG\KHOSIXODQGWKHUHDUHVHYHUDOWHFKQLTXHVWKDWZHFDQLPSOHPHQWULJKW
IURPWKHVWDUWZLWKJRRGHIIHFW6ROHWXVH[DPLQHNH\FRPSRQHQWVRIWKHLQWHUYLHZLQJ
SURFHVV 6HHDOVR5LFKDUGV&KDSWHUIRUDYHU\SUDFWLFDODFFRXQWDLPHGDW
DSSOLHGOLQJXLVWVDOVRLQFOXGLQJVXJJHVWLRQVRQKRZWRGHYHORSLQWHUYLHZLQJVNLOOV
5HFRUGLQJWKHLQWHUYLHZ

7KHUHLVJHQHUDODJUHHPHQWLQWKHOLWHUDWXUHWKDWLIZHZDQWWRXVHWKHFRQWHQWRIDVHPL
VWUXFWXUHGRUXQVWUXFWXUHGLQWHUYLHZDVUHVHDUFKGDWDZHQHHGWRUHFRUGLW²WDNLQJ
QRWHVLVVLPSO\QRWHQRXJKDVZHDUHXQOLNHO\WREHDEOHWRFDWFKDOOWKHGHWDLOVRIWKH
QXDQFHVRISHUVRQDOPHDQLQJIXUWKHUPRUHQRWHWDNLQJDOVRGLVUXSWVWKHLQWHUYLHZLQJ
SURFHVV+RZHYHUZHPXVWEHDZDUHRIWKHIDFWWKDWPDQ\SHRSOHGRQRWOLNHWREH
UHFRUGHGDQGWKHUHIRUHZHPXVWGLVFXVVWKLVDVSHFWZLWKWKHLQWHUYLHZHHLQDGYDQFH
5HFRUGLQJKDVDWHFKQLFDODQGDWKHRUHWLFDODVSHFW7KHWHFKQLFDOLVVWUDLJKWIRUZDUG
ZHPXVWPDNHVXUHWKDWWKHWHFKQRORJ\ZRUNVVRWKDWZHHQGXSZLWKJRRGTXDOLW\
UHFRUGLQJV$OWKRXJKWKLVPD\VRXQGVHOIHYLGHQWLWLVDPD]LQJKRZIUHTXHQWO\
UHVHDUFKHUVGRQRWHQGXSZLWKJRRGTXDOLW\UHFRUGLQJV7KHLPSRUWDQFHRIWKLV
WHFKQLFDODVSHFWLVUHIOHFWHGE\WKHIDFWWKDWLQWKHDSSHQGL[RIKLVPRQRJUDSKRQ
/LQJXLVWLF$QWKURSRORJ\'XUDQWL GHYRWHGSDJHVWRGLVFXVVLQJUHFRUGLQJ
LVVXHVUDQJLQJIURPKRZWRSRVLWLRQWKHPLFURSKRQHWRKRZWRODEHOWKHFDVVHWWHV
7KHEHVWSUHFDXWLRQLVWRWDNH0XUSK\¶V/DZYHU\VHULRXVO\µ,ILWFDQJRZURQJLW
ZLOO¶6RLISRVVLEOHZHVKRXOGKDYHDVSDUHUHFRUGHUZLWKXVDOZD\VXVHQHZEDWWHULHV
DQGFDVVHWWHVLIDSSOLFDEOH DQGWHVWWKDWWKHHTXLSPHQWZRUNVDWWKHEHJLQQLQJRIWKH
LQWHUYLHZE\GRLQJDWULDOUHFRUGLQJ2QHQRWRULRXVSUREOHPZLWKUHFRUGLQJLVWKDW
XQOHVVZHODEHOWKHUHFRUGHGFDVVHWWHVGLVFVGLJLWDOILOHVLPPHGLDWHO\DQGSUHFLVHO\WKH\
ZLOOJHWPL[HGXS%HFDXVHZHDUHOLNHO\WRHQGXSZLWKDORWRIWKHVHUHFRUGLQJVLWLV
LPSRUWDQWWRNHHSDSUHFLVHORJRIWKHLUFRQWHQWULJKWIURPWKHVWDUW 6HHDOVR6HFWLRQV
DQGRQUHVHDUFKORJVDQGGDWDVWRUDJH 'XUDQWL DOVRUHFRPPHQGV
WKDWDIWHUZHKDYHILQLVKHGUHFRUGLQJZHPDNHFRSLHVRIWKHRULJLQDOWDSHVIRUOLVWHQLQJ
DQGWUDQVFULELQJ²LQGLJLWDOWHUPVWKLVZRXOGPHDQPDNLQJEDFNXSILOHV
7KHWKHRUHWLFDOLVVXHDERXWUHFRUGLQJFRQFHUQVWKHIDFWWKDWE\GRLQJDXGLR
UHFRUGLQJZHLQHYLWDEO\ORVHVRPHLQIRUPDWLRQIRUH[DPSOHQRQYHUEDOFXHVVXFKDVH\H
PRYHPHQWVIDFLDOH[SUHVVLRQVRUJHVWXUHV7KLVZRXOGVXJJHVWWKDWDYLGHRUHFRUGLQJLV
DOZD\VSUHIHUDEOHEXWPDQ\UHVHDUFKHUVGRQRWDFFHSWWKLVFODLP$OWKRXJKYLGHRGDWDLV
REYLRXVO\ULFKHUWKDQDXGLRUHFRUGLQJVWKHYLGHRUHFRUGLQJSURFHVVLVPXFKPRUH
GLIILFXOWDQGREWUXVLYHWKDQVZLWFKLQJRQDVPDOOGLFWDSKRQHDQGDQDO\VLQJYLGHRGDWD
LVQRWDQHDV\WDVNHLWKHU 6HH6HFWLRQ 6RP\UHFRPPHQGDWLRQLVWKDWZH
VKRXOGXVHYLGHRGDWDIRURXUUHVHDUFKRQO\LIZHUHDOO\QHHGLW
6WDUWLQJWKHLQWHUYLHZ
7KHILUVWIHZPLQXWHVRIWKHLQWHUYLHZVHVVLRQDUHLPSRUWDQWEHFDXVHWKLVLVWKHWLPH
ZKHQZHVHWWKHWRQHFOLPDWHRIWKHLQWHUYLHZDQGµVHOORXUVHOYHV¶WKDWLVHVWDEOLVKRXU
FUHGHQWLDOVDQGPDNHRXUVHOYHVDFFHSWHG:HPXVWGHPRQVWUDWHWKDWZHDUHUHDOO\
LQWHUHVWHGLQZKDWWKHLQWHUYLHZHHKDVJRWWRVD\DQGDOVRWKDWZHDUHDUHDVRQDEO\QLFH
DQGQRQWKUHDWHQLQJSHUVRQ$V0F&UDFNHQ FRQFOXGHVLWLVEHWWHUWRDSSHDU
µVOLJKWO\GLPDQGWRRDJUHHDEOHWKDQWRJLYHDQ\VLJQRIDFULWLFDORUVDUGRQLFDWWLWXGH¶
$VKHIXUWKHUDUJXHVDQHIIHFWLYHPDQQHURISUHVHQWLQJRXUVHOYHVLVVWULNLQJDEDODQFH
EHWZHHQIRUPDOLW\DQGLQIRUPDOLW\DFHUWDLQIRUPDOLW\LQGUHVVDQGVW\OHLVXVHIXO

EHFDXVHLWFUHDWHVDµVHULRXV¶DQGµSURIHVVLRQDO¶LPDJH DIWHUDOOµUHDO¶VFLHQWLVWVZHDU
ZKLWHFRDWVDQGXVH/DWLQZRUGVWKDWIHZSHRSOHFDQXQGHUVWDQG DQGLWDOVRLQGLFDWHV
WKDWWKHLQYHVWLJDWRUFDQEHWUXVWHGWRPDLQWDLQWKHFRQILGHQWLDOLW\WKDWKDVEHHQ
SURPLVHGWKHUHVSRQGHQW2QWKHRWKHUKDQGDFHUWDLQDPRXQWRILQIRUPDOLW\LVXVHIXO
EHFDXVHLWUHDVVXUHVWKHUHVSRQGHQWWKDWZHDUHDOVRLPSHUIHFWKXPDQEHLQJVQRW
LQGLIIHUHQWWRWKHFRPSOH[LWLHVDQGGLIILFXOWLHVRIWKHUHVSRQGHQW¶VOLIH
%HIRUHVWDUWLQJWKHUHFRUGLQJZHQHHGWRH[SODLQDJDLQWKHUHDVRQIRUWKH
LQWHUYLHZ²XQGHUVWDQGLQJWKHSXUSRVHRIWKHTXHVWLRQVZLOOLQFUHDVHWKHPRWLYDWLRQRI
WKHLQWHUYLHZHHWRUHVSRQGRSHQO\DQGLQGHWDLO:HVKRXOGDOVRVXPPDUL]HEULHIO\ZKDW
ZLOOKDSSHQWRWKHLQWHUYLHZGDWDDQGLWPD\EHZRUWKUHDVVXULQJWKHUHVSRQGHQWDJDLQ
RQWKHLVVXHRIFRQILGHQWLDOLW\$OUHDG\DWWKLVVWDJHZHQHHGWRIDFLOLWDWHDUHOD[HGQRQ
WKUHDWHQLQJDWPRVSKHUHLQZKLFKWKHLQWHUYLHZHHFDQIHHOFRPIRUWDEOHWRH[SUHVV
KLPKHUVHOI6RPHVPDOOWDONLVDOVRKHOSIXOWREXLOGUDSSRUWZLWKWKHLQWHUYLHZHH:KHQ
ZHWKLQNWKDWWKHLQWHUYLHZHHLVDWHDVHZHVKRXOGDVNLILWLVDOOULJKWWRVZLWFKRQWKH
UHFRUGHUDQGWRWHVWWKDWLWZRUNV7KHQJHWRQZLWKWKHILUVWTXHVWLRQV²DVGHVFULEHGLQ
WKHSUHYLRXVVHFWLRQ 6HFWLRQ WKHVHVKRXOGEHVXFKWKDWWKH\FDQEHHDVLO\
DQVZHUHG
&RQGXFWLQJWKHLQWHUYLHZ
$JRRGTXDOLWDWLYHLQWHUYLHZKDVWZRNH\IHDWXUHV D LWIORZVQDWXUDOO\ZLWKWKH
YDULRXVSDUWVFRQQHFWLQJVHDPOHVVO\:HPXVWUHPHPEHUWKDWZHDUHWKHUHSULPDULO\WR
OLVWHQ DQGQRWVSHDN ²WKLVLVLQIDFWWKHILUVWRI5REVRQ¶V JHQHUDO
UHFRPPHQGDWLRQVRQLQWHUYLHZLQJ SUHVHQWHGLQ7DEOH :HPXVWOHWWKHLQWHUYLHZHH
GLFWDWHWKHSDFHZLWKRXWEHLQJUXVKHGRULQWHUUXSWHG(YHQLIWKHUHLVDVLOHQFHZHQHHG
WREHSDWLHQWDQGUHVLVWVWHSSLQJLQWRRTXLFNO\ZLWKDQHZTXHVWLRQDQG E LWLVULFKLQ
GHWDLO$FFRUGLQJO\5LFKDUGV SRLQWVRXWWKDWDJROGHQUXOHIRUDOO
LQWHUYLHZLQJLVµ$OZD\VVHHNWKHSDUWLFXODU¶,WLVFOHDUWKDWPHHWLQJWKLVVHFRQG
UHTXLUHPHQWFDQFRPHDWWKHH[SHQVHRIWKHILUVWEHFDXVHDWWLPHVZHPD\QHHGWR
LQWHUUXSWWKHQDWXUDOIORZRIDQDFFRXQW SDUWLFXODUO\LILWGULIWVLQWRUDPEOLQJPRGH DQG
IRFXVRQVSHFLILFGHWDLOV7KLVLVDQDUHDZKHUHWKHVNLOIXOXVHRIYDULRXVSUREHVFDQ
PDNHDUHDOGLIIHUHQFH,ZLOOFRPHEDFNWRWKLVTXHVWLRQEHORZEXWILUVWZHQHHGWR
DGGUHVVDQLVVXHWKDWLVFHQWUDOWRWKHLQWHUYLHZHU¶VFRQGXFWWKHQHXWUDOLW\RIWKH
UHVHDUFKHU
,WKDVEHHQDNH\SULQFLSOHLQLQWHUYLHZPHWKRGRORJ\WKDWWKHLQWHUYLHZHUVKRXOGWU\
WREHQHXWUDOZLWKRXWLPSRVLQJDQ\SHUVRQDOELDV%XWZKDWGRHVWKLVQHXWUDOLW\PHDQLQ
SUDFWLFDOWHUPV",EHOLHYHWKDWLWLQYROYHVFUHDWLQJDSSURSULDWHVSDFHIRUWKHLQWHUYLHZHHV
WRVKDUHWKHLUH[SHULHQFHZLWKXVIUHHO\UHJDUGOHVVRIDQ\VRFLDOPRUDORUSROLWLFDO
FRQWHQW7KHSRLQWWRVWUHVVKHUHLVWKDWVLPSO\QRWH[SUHVVLQJDQ\SHUVRQDOELDV
RXUVHOYHVPD\QRWEHHQRXJKWRFUHDWHWKLVQHXWUDOVSDFHEHFDXVHUHVSRQGHQWVZLOOEH
LQHYLWDEO\LQIOXHQFHGE\ZKDWLVXVXDOO\WHUPHGWKHµVRFLDOGHVLUDELOLW\ELDV¶ 6HH
6HFWLRQ 7KDWLVWKH\PD\IHHOWKDWVRPHUHVSRQVHZRXOGPHHWZLWKGLVDSSURYDO
QRWQHFHVVDULO\EHFDXVHZHKDYHJLYHQDQ\LQGLFDWLRQWKDWWKLVPLJKWEHVREXWEHFDXVH

WKHUHVSRQVHFODVKHVZLWKVRPHVRFLDOFRQYHQWLRQVRUQRUPV,QHYLWDEO\UHVSRQGHQWV
HQWHUWKHLQWHUYLHZVHVVLRQZLWKVRPHLGHDVRIZKDWPD\FRQVWLWXWHSUHIHUUHGDQG
GLVSUHIHUUHGUHVSRQVHVDQGLIZHGRQRWGHDOZLWKWKLVLVVXHKHDGRQZHPD\HQGXS
ZLWKDQHDWVHOIFHQVRUHGDQGUDWKHUVWHULOHQDUUDWLYH$IWHUDOODV2SSHQKHLP
ZDUQVXVHYHQIDFWXDOTXHVWLRQVFDQEHORDGHGZLWKGHVLUDELOLW\LVVXHVSHRSOHPLJKW
FODLPIRUH[DPSOHWKDWWKH\UHDGPRUHWKDQWKH\GREDWKHPRUHRIWHQWKDQLVWUXHRU
VSHQGPRUHWLPHZLWKWKHLUFKLOGUHQWKDQDFWXDOO\KDSSHQV7KHWUXO\QHXWUDOLQWHUYLHZ
VSDFHHQFRXUDJHVWKHVKDULQJRIHYHQWKHVRFLDOO\OHVVWKDQGHVLUDEOH
+RZFDQVXFKQHXWUDOLW\EHDFKLHYHG"5DSSRUWLVLQGLVSHQVDEOHDQGQRWSUHVHQWLQJ
RXUVHOYHVLQDSHUIHFWOLJKWLVDOVRKHOSIXO7KHUHDUHDOVRVRPHSUDFWLFDOWHFKQLTXHVWR
PLWLJDWHWKHXQGHVLUDEOH VHHDOVR'|UQ\HL VXFKDVZRUGLQJWKHTXHVWLRQLQDZD\
WKDWLWVXJJHVWVWKDWWKHEHKDYLRXULVUDWKHUFRPPRQ IRUH[DPSOHµ(YHQWKHPRVW
FRQVFLHQWLRXVWHDFKHUVVRPHWLPHV«¶ XVLQJDXWKRULW\WRPDNHWKHVHQVLWLYH
LVVXHEHKDYLRXUDSSHDUWREHMXVWLILDEOH IRUH[DPSOHµ0DQ\UHVHDUFKHUVQRZWKLQNWKDW
«¶ ,QFOXGLQJUHDVRQVWKDWH[SODLQWKHEHKDYLRXU IRUH[DPSOHµ'RHV\RXUEXV\
VFKHGXOHVRPHWLPHVSUHYHQW\RXIURP«"¶ RUVLPSO\DGRSWLQJDFDVXDODSSURDFK IRU
H[DPSOHµ'LG\RXKDSSHQWR«"¶
,WPXVWEHQRWHGWKDWWKHUHLVDGHEDWHLQWKHTXDOLWDWLYHOLWHUDWXUHFRQFHUQLQJWKH
QHXWUDOLW\LVVXH IRUDUHFHQWDQDO\VLVVHH.YDOH DQGIRUH[DPSOH)RQWDQDDQG
)UH\ DGYRFDWHµHPSDWKHWLFLQWHUYLHZLQJ¶LQZKLFKWKHLQWHUYLHZHUWDNHVDVWDQFH
7KHDXWKRUVDUJXHWKDWWKHJRDORIVFLHQWLILFQHXWUDOLW\LQDQLQWHUYLHZVHVVLRQLVµODUJHO\
P\WKLFDO¶ S EHFDXVHLQWHUYLHZLQJLVQRWPHUHO\WKHQHXWUDOH[FKDQJHRIDVNLQJ
TXHVWLRQVDQGJHWWLQJDQVZHUVEXWUDWKHUDFRFRQVWUXFWHGVRFLDOH[FKDQJHLQZKLFK
WDNLQJDVWDQFHEHFRPHVXQDYRLGDEOH7KLVHPSDWKLFVWDQFHLVXQGRXEWHGO\KHOSIXOLQ
GHDOLQJZLWKVHQVLWLYHWRSLFVDQGHOLFLWLQJPRUHKRQHVW\EHFDXVHWKHLQWHUYLHZHULVVHHQ
DVDQDOO\EXW,DPQRWTXLWHVXUHKRZRQHQHHGVWRKDQGOHDQLQWHUYLHZLIRQH¶V
SHUVRQDOVWDQFHGRHVQRWIXOO\FRLQFLGH RUHYHQFODVKHV ZLWKWKDWRIWKHLQWHUYLHZHH
2EYLRXVO\VRPHGHOLFDWHEDODQFLQJDFWLVQHHGHGKHUHEHWZHHQQRQMXGJHPHQWDO
QHXWUDOLW\DQGHPSDWKHWLFXQGHUVWDQGLQJDQGDSSURYDO
x /LVWHQPRUHWKDQ\RXVSHDN0RVWLQWHUYLHZHUVWDONWRRPXFK7KHLQWHUYLHZLVQRWD
SODWIRUPIRUWKHLQWHUYLHZHU¶VSHUVRQDOH[SHULHQFHVDQGRSLQLRQV
x 3XWTXHVWLRQVLQDVWUDLJKWIRUZDUGFOHDUDQGQRQWKUHDWHQLQJZD\,ISHRSOHDUHFRQIXVHG
RUGHIHQVLYH\RXZLOOQRWJHWWKHLQIRUPDWLRQ\RXVHHN
x (OLPLQDWHFXHVZKLFKOHDGLQWHUYLHZHHVWRUHVSRQGLQDSDUWLFXODUZD\ IRUH[DPSOHµ$UH
\RXDJDLQVWVLQ"¶ 0DQ\LQWHUYLHZHHVZLOOVHHNWRSOHDVHWKHLQWHUYLHZHUE\JLYLQJµFRUUHFW¶
UHVSRQVHV
x (QMR\LW RUDWOHDVWORRNDVWKRXJK\RXGR 'RQ¶WJLYHWKHPHVVDJHWKDW\RXDUHERUHGRU

VFDUHG9DU\\RXUYRLFHDQGIDFLDOH[SUHVVLRQ

7DEOH5REVRQ¶V JHQHUDODGYLFHIRULQWHUYLHZHUV

2QFHZHKDYHVXFFHVVIXOO\HVWDEOLVKHGWKHIORZRIWKHLQWHUYLHZZHQHHGWRVXVWDLQ
LWLQDQXQREWUXVLYHZD\7KHUHDUHDQXPEHURIXVHIXOWHFKQLTXHVWRNHHSWKHLQWHUYLHZ
RQWUDFN
x &DUU\RQIHHGEDFN%DFNFKDQQHOOLQJVLJQDOV IRUH[DPSOHQRGVµHKKXK¶QRLVHV
RQHZRUGXWWHUDQFHVOLNHµ\HDK¶ DUHRIIHUHGE\DQ\V\PSDWKHWLFOLVWHQHUDVSDUWRI
QRUPDOFRPPXQLFDWLRQDQGWKHLQWHUYLHZHUPXVWDSSHDUWREHDSDUWLFXODUO\
V\PSDWKHWLFOLVWHQHU0LOOHUDQG&UDEWUHH DOVRKLJKOLJKWWKHLPSRUWDQWUROHRI
VPDOOJHVWXUHVVXFKDVWKHµDWWHQWLYHOHDQ¶DQGWKHµH\HEURZIODVK¶DQGZHFDQDGG
KHUHWKHµV\PSDWKHWLFVPLOH¶
x 5HLQIRUFHPHQWIHHGEDFN-XVWDVLQDQRUPDOFRQYHUVDWLRQZHQHHGWRSURYLGHIURP
WLPHWRWLPHUHLQIRUFHPHQWLQGLFDWLQJWKDWZHDUHSOHDVHGZLWKWKHZD\WKH
LQWHUYLHZLVJRLQJDQGFRQILUPLQJWKDWWKHLQWHUYLHZHH¶VHIIRUWVDUHZRUWKZKLOH
(YHU\ERG\OLNHVWREHSUDLVHG
x 1HJDWLYHUHLQIRUFHPHQW:KDWVKDOOZHGRZKHQWKHLQWHUYLHZLVQRWJRLQJDVZHOO
DVZHWKLQNLWFRXOG"$IWHUDOOSHRSOHRIWHQOLNHWRWDONDQGJRRIIDWDWDQJHQWLQD
ZD\ZKLFKLVQRWµVSRQWDQHRXVO\HPHUJLQJSHUVRQDOPHDQLQJ¶EXWPHUHO\ZDIIOLQJ
,IZHJLYHUHJXODUVKRUWUHLQIRUFHPHQWIHHGEDFN IRUH[DPSOHQRGGLQJXKKXKV
WKHQVLPSO\ZLWKKROGLQJWKHVHDQGUHSODFLQJWKHPZLWKDQLQWHUMHFWHGTXHVWLRQRUD
VKLIWLQRXUSRVLWLRQPLJKWEHHQRXJKRIDQXGJHWRJHWEDFNRQWUDFN$OWHUQDWLYHO\
DV3DWWRQ SRLQWVRXWVRPHFOHYHUSKUDVHVPLJKWEHXVHGIRULQRIIHQVLYH
LQWHUUXSWLRQDQGUHIRFXVLQJ IRUH[DPSOHµ/HWPHVWRS\RXKHUHIRUDPRPHQWDQG
JREDFNWRZKDW\RXVDLGHDUOLHUWRPDNHVXUHWKDW,XQGHUVWRRG\RXZHOO¶
x (QFRXUDJLQJHODERUDWLRQ,QFDVHVZKHQWKHLQWHUYLHZHHLVQRWYHU\IRUWKFRPLQJ
DERXWDFHUWDLQWRSLFZHFDQXVHYDULRXVSUREHV GLVFXVVHGLQ6HFWLRQ
LQFOXGLQJµVLOHQWSUREHV¶WKDWLQYROYHUHPDLQLQJTXLHWWRLQGLFDWHWKDWZHDUH
ZDLWLQJIRUPRUH0RUHGLUHFWFDOOVIRUHODERUDWLRQLQFOXGHWKHµHFKRSURPSW¶
UHSHDWLQJWKHODVWZRUGVSRNHQE\WKHUHVSRQGHQW ORZLQIHUHQFHSDUDSKUDVLQJRU
UHIOHFWLYHVXPPDU\DQGFODULILFDWLRQTXHVWLRQV$VPHQWLRQHGHDUOLHUDFRPPRQ
WHFKQLTXHLVWRWDNHDVDOLHQWFRQWHQWZRUGXVHGE\WKHUHVSRQGHQWDQGUHSHDWLWZLWK
DQLQWHUURJDWLYHWRQHRUDVNWKHLQWHUYLHZHHWRHODERUDWHRQLW
x $WWHQWLRQIRFXVLQJGHYLFHV,QDORQJLQWHUYLHZZHQHHGIURPWLPHWRWLPHWR
UHNLQGOHWKHUHVSRQGHQW¶VLQWHUHVWDQGUHIRFXVWKHUHVSRQVHV$QDWWHQWLRQJHWWLQJ
FRPPHQWWKDWSUHFHGHVWKHTXHVWLRQFDQDFKLHYHWKLVLWPD\FRQFHUQWKHTXHVWLRQ¶V
LPSRUWDQFHGLIILFXOW\RUDQ\RWKHUFKDUDFWHULVWLFV µ&DQ,DVN\RXQRZDERXWDQ
LVVXHWKDWLVYHU\LPSRUWDQWEXWQRWHDV\WRWDONDERXW«¶ ,WLVDOVRXVHIXOWRKDYH
WUDQVLWLRQDQQRXQFHPHQWVDQGLQWURGXFWRU\VWDWHPHQWVEHIRUHDQHZLVVXHLV
DGGUHVVHGEHFDXVHWKLVDGYDQFHQRWLFHZLOOKHOSWRFUHDWHWKHDSSURSULDWHPLQGVHWRU
VFKHPD

(QGLQJWKHLQWHUYLHZ
:HFDQVLJQDOWKDWWKHLQWHUYLHZLVQHDULQJWKHHQGE\XVLQJSUHFORVLQJPRYHVVXFKDV
VXPPDUL]LQJRUUHFDSSLQJWKHPDLQSRLQWVRIWKHGLVFXVVLRQ7KHVHDOVRKDYHD
VLJQLILFDQFHFRQWHQWZLVHEHFDXVHWKH\DOORZWKHLQWHUYLHZHHWRFRUUHFWDQ\WKLQJWKDW
ZHPD\KDYHPLVXQGHUVWRRGDQGWRPDNHDQ\DGGLWLRQDOSRLQWV,QIDFWDVGLVFXVVHGLQ
FRQQHFWLRQWRWKHLQWHUYLHZJXLGH 6HFWLRQ ZHPD\ZLVKWRJLYHWKHUHVSRQGHQW
DQH[SOLFLWRSSRUWXQLW\WRPDNHDQ\FRPPHQWVWKDWPLJKWEHUHOHYDQWLPSRUWDQWEXW
ZKLFKKDYHQRWEHHQFRYHUHGLQWKHUHVWRIWKHLQWHUYLHZ µ,KDYHQRIXUWKHUTXHVWLRQV
'R\RXKDYHDQ\WKLQJPRUH\RXZDQWWREULQJXSRUDVNDERXWEHIRUHZHILQLVKWKH
LQWHUYLHZ"¶.YDOH :KHWKHUZHGRWKLVRUQRWZHQHHGWREHFDUHIXOQRWWR
HQGWKHLQWHUYLHZRQDGLIILFXOWWRSLF²MXVWOLNHLQRUDOODQJXDJHH[DPVZHQHHGWR
LQFOXGHDILQDOZLQGLQJGRZQSKDVHE\VWHHULQJWKHLQWHUYLHZWRZDUGVSRVLWLYH
H[SHULHQFHV.YDOHFDOOVWKLVIXQFWLRQµGHEULHILQJ¶DQGFRQVLGHUVLWYLWDOJLYHQWKDWWKH
LQWHUYLHZHHKDVRSHQHGKLPKHUVHOIXSDQGPD\KDYHGLVFORVHGGHHSO\SHUVRQDO
LQIRUPDWLRQ7KHUHIRUHLQWKLVILQDOSKDVHZHQHHGWRUHH[SUHVVRXUJUDWHIXOQHVVDQG
UHVSHFWDQGGLVFXVVWKHZD\VE\ZKLFKWKHPDWHULDOZLOOEHXVHGDQGWKHORJLVWLFVRI
KRZWRNHHSLQWRXFKLQWKHIXWXUH
6WUHQJWKVDQGZHDNQHVVHVRILQWHUYLHZV
7KHLQWHUYLHZLVDQDWXUDODQGVRFLDOO\DFFHSWDEOHZD\RIFROOHFWLQJLQIRUPDWLRQWKDW
PRVWSHRSOHIHHOFRPIRUWDEOHZLWKDQGZKLFKFDQEHXVHGLQDYDULHW\RIVLWXDWLRQVDQG
IRFXVLQJRQGLYHUVHWRSLFVWR\LHOGLQGHSWKGDWD7KHLQWHUYLHZHU¶VSUHVHQFHDOORZVIRU
IOH[LEOHDSSURDFKHVSURELQJLQWRDQ\HPHUJLQJQHZLVVXHZKLOHWKHLQWHUYLHZJXLGH
KHOSVWRPDLQWDLQDV\VWHPDWLFFRYHUDJHRIWKHGRPDLQ%HFDXVHRIWKHKLJKVRFLDO
SURILOHRILQWHUYLHZLQJPRVWRIXVZLOOKDYHVHYHUDOJRRGLQWHUYLHZHUPRGHOVLQRXU
PLQGVDQGWKHUHIRUHHYHQEHJLQQLQJUHVHDUFKHUVDUHOLNHO\WREHDEOHWRREWDLQULFKGDWD
LQWKHLUYHU\ILUVWLQWHUYLHZV
7KHPDLQZHDNQHVVRIWKHLQWHUYLHZLVWKDWLWLVWLPHFRQVXPLQJWRVHWXSDQG
FRQGXFWDQGWKDWLWUHTXLUHVJRRGFRPPXQLFDWLRQVNLOOVRQWKHSDUWRIWKHLQWHUYLHZHU
ZKLFKQRWDOORIXVKDYHQDWXUDOO\%HFDXVHWKHLQWHUYLHZIRUPDWGRHVQRWDOORZIRU
DQRQ\PLW\WKHUHLVDFKDQFHWKDWWKHUHVSRQGHQWZLOOWU\WRGLVSOD\KLPKHUVHOILQD
EHWWHUWKDQUHDOOLJKW,QWHUYLHZHHVFDQDOVREHWRRVK\DQGLQDUWLFXODWHWRSURGXFH
VXIILFLHQWGDWDRUDWWKHRWKHUH[WUHPHWKH\FDQEHWRRYHUERVHJHQHUDWLQJDORWRIOHVV
WKDQXVHIXOGDWD
)RFXVJURXSLQWHUYLHZV
)RFXVJURXSLQWHUYLHZVDUHVRPHWLPHVWUHDWHGDVDVXEW\SHRILQWHUYLHZLQJEXWEHFDXVH
ERWKWKHIRUPDWDQGWKHLQWHUYLHZHU¶VUROHDUHFRQVLGHUDEO\GLIIHUHQWLQWKHLQWHUYLHZLQJ
SURFHVV,ILQGLWPRUHDSSURSULDWHWRNHHSWKHWZRPHWKRGVVHSDUDWH)RFXVJURXS
LQWHUYLHZV²DVWKHQDPHVXJJHVWV²LQYROYHDJURXSIRUPDWZKHUHE\DQLQWHUYLHZHU
UHFRUGVWKHUHVSRQVHVRIDVPDOOJURXS XVXDOO\±PHPEHUV 7KLVLVREYLRXVO\DQ

HFRQRPLFDOZD\WRJDWKHUDUHODWLYHO\ODUJHDPRXQWRITXDOLWDWLYHGDWDDQGWKHUHIRUH
IRFXVJURXSVDUHXVHGIRUDYDULHW\RISXUSRVHVLQPDQ\GLIIHUHQWILHOGV7KHQDPH
RULJLQDOO\FRPHVIURPPDUNHWUHVHDUFKEXWQRZWKHWHUPVµIRFXVJURXSLQWHUYLHZ¶DQG
µJURXSLQWHUYLHZ¶DUHXVHGLQWHUFKDQJHDEO\7KHPHWKRGKDVEHFRPHIDPLOLDUWRWKH
ODUJHUSXEOLFZKHQSROLWLFDOSDUWLHVKDYHVWDUWHGWRXVHLWWRJDXJHYRWHUUHDFWLRQWR
VRPHSODQQHGSROLFLHVDQGJURXSLQWHUYLHZVDOVRIHDWXUHIUHTXHQWO\RQ79
7KHIRFXVJURXSIRUPDWLVEDVHGRQWKHFROOHFWLYHH[SHULHQFHRIJURXS
EUDLQVWRUPLQJWKDWLVSDUWLFLSDQWVWKLQNLQJWRJHWKHULQVSLULQJDQGFKDOOHQJLQJHDFK
RWKHUDQGUHDFWLQJWRWKHHPHUJLQJLVVXHVDQGSRLQWV7KLVZLWKLQJURXSLQWHUDFWLRQFDQ
\LHOGKLJKTXDOLW\GDWDDVLWFDQFUHDWHDV\QHUJLVWLFHQYLURQPHQWWKDWUHVXOWVLQDGHHS
DQGLQVLJKWIXOGLVFXVVLRQ7KHIRUPDWDOVRDOORZVIRUYDULRXVGHJUHHVRIVWUXFWXUH
GHSHQGLQJRQWKHH[WHQWWRZKLFKWKHUHVHDUFKHUUHOLHVRQDQLQWHUYLHZJXLGHSURWRFRO
UDWKHUWKDQJLYLQJWKHSDUWLFLSDQWVIUHHGRPWRGLVFXVVVRPHEURDGWRSLFV-XVWOLNHZLWK
RQHWRRQHLQWHUYLHZVWKHVHPLVWUXFWXUHGW\SHRIIRFXVJURXSLVWKHPRVWFRPPRQ
IRUPDWEHFDXVHLWLQFOXGHVERWKRSHQDQGFORVHGHQGHGTXHVWLRQVSRVHGE\WKH
UHVHDUFKHU
&KDUDFWHULVWLFVRIWKHIRFXVJURXSV
7KHVL]HRIDIRFXVJURXSUDQJHVEHWZHHQ± VRPHWLPHV SHRSOH)HZHUWKDQ
SHRSOHZRXOGOLPLWWKHSRWHQWLDORIWKHµFROOHFWLYHZLVGRP¶ZKHUHDVWRRODUJHDVL]H
PDNHVLWGLIILFXOWIRUHYHU\RQHWRSDUWLFLSDWH:KHQGHVLJQLQJDIRFXVJURXSVWXG\WKH
WZRNH\WHFKQLFDOTXHVWLRQVWRGHFLGHDUH D ZKHWKHUWRKDYHKRPRJHQHRXVRU
KHWHURJHQHRXVSHRSOHLQDJURXSDQG E KRZPDQ\JURXSVWRKDYH
x &RPSRVLWLRQ$OWKRXJKLQOLQHZLWKWKHSULQFLSOHVRIPD[LPXPYDULDWLRQVDPSOLQJ
VHH6HFWLRQ KHWHURJHQHRXVVDPSOHVFRQVLVWLQJRIGLVVLPLODUSHRSOHFRXOGLQ
WKHRU\EHXVHIXOLQSURYLGLQJYDULHGDQGULFKGDWDWKDWFRYHUVDOODQJOHVLWKDVEHHQ
IRXQGWKDWWKHG\QDPLFVRIWKHIRFXVJURXSZRUNVEHWWHUZLWKKRPRJHQHRXV
VDPSOHV7KHUHIRUHLQRUGHUWRREWDLQDZLGHUDQJHRILQIRUPDWLRQWKHXVXDO
VWUDWHJ\LVWRKDYHVHYHUDOJURXSVZKLFKDVDZKROHDUHGLIIHUHQWIURPHDFKEXW
HDFKRIZKLFKLVPDGHXSRIVLPLODUSHRSOHWKLVLVXVXDOO\UHIHUUHGWRDV
µVHJPHQWDWLRQ¶DQGLWLQYROYHVZLWKLQJURXSKRPRJHQHLW\DQGLQWHUJURXS
KHWHURJHQHLW\LQWKHVDPSOH
x 1XPEHURUSDUDOOHOIRFXVJURXSV7KHVWDQGDUGSUDFWLFHLVWRUXQVHYHUDOIRFXV
JURXSVLQDQ\UHVHDUFKSURMHFW,QWKLVZD\ZHFDQPLWLJDWHDQ\LGLRV\QFUDWLFUHVXOWV
WKDWRFFXUEHFDXVHRIVRPHXQH[SHFWHGLQWHUQDORUH[WHUQDOIDFWRUWKDWDIIHFWVWKH
G\QDPLFVRIDJURXS7KXVLQRUGHUWRDFKLHYHDGHTXDWHEUHDGWKDQGGHSWKRI
LQIRUPDWLRQLWLVXVXDOO\UHFRPPHQGHGWKDWDSURMHFWLQYROYH±JURXSVDVD
PLQLPXPZLWKDIHZPRUHLISRVVLEOH
&RQGXFWLQJWKHIRFXVJURXSLQWHUYLHZ

,QIRFXVJURXSLQWHUYLHZVWKHLQWHUYLHZHULVXVXDOO\UHIHUUHGWRDVWKHµPRGHUDWRU¶DQG
WKLVVSHFLDOQDPHUHIOHFWVWKHIDFWWKDWWKHUHVHDUFKHUV¶UROHGLIIHUVIURPWKDWLQRQHWR
RQHLQWHUYLHZV$OWKRXJKWKH\VWLOOQHHGWRDVNTXHVWLRQVGXULQJWKHVHVVLRQWKH\QHHG
WRIXQFWLRQPRUHDVIDFLOLWDWRUVRIWKHGLVFXVVLRQWKDQDVLQWHUYLHZHUVLQWKHWUDGLWLRQDO
VHQVH%HFDXVHWKHG\QDPLFRIWKHIRFXVJURXSLVRQHRIWKHXQLTXHIHDWXUHVRIWKLV
PHWKRGWKHUHVHDUFKHU¶VUROHLQHYLWDEO\LQYROYHVVRPHJURXSOHDGHUVKLSIXQFWLRQV
LQFOXGLQJPDNLQJVXUHWKDWQRERG\GRPLQDWHVWKHIORRUDQGWKDWHYHQWKHVK\HU
SDUWLFLSDQWVKDYHDFKDQFHWRH[SUHVVWKHLUYLHZV,QDGGLWLRQPRGHUDWRUVQHHGWR
SUHYHQWDQ\GRPLQDWLQJDQGLQKLELWLQJJURXSRSLQLRQ²RUµJURXSWKLQN¶²IURP
HPHUJLQJE\DFWLYHO\HQFRXUDJLQJJURXSPHPEHUVWRWKLQNFULWLFDOO\ '|UQ\HLDQG
0XUSKH\ ,WLVEHFDXVHRIWKHVHPXOWLSOHUHVSRQVLELOLWLHVWKDWUHVHDUFK
PHWKRGRORJLVWVXVXDOO\DJUHHWKDWDIRFXVJURXSLVRQO\DVJRRGDVLWVPRGHUDWRU
7KXVIRFXVJURXSPRGHUDWLRQFDQEHDFKDOOHQJLQJWDVNSDUWLFXODUO\LQWKHOLJKWRI
WKHIDFWWKDWDIRFXVJURXSLQWHUYLHZFDQODVWDVORQJDVKRXUV DOWKRXJKWKHXVXDO
OHQJWKLVEHWZHHQ±KRXUV 7KHSURFHVVVWDUWVZLWKDQLQWURGXFWRU\SKDVHLQZKLFK
WKHPRGHUDWRUZHOFRPHVWKHSDUWLFLSDQWVRXWOLQHVWKHSXUSRVHRIWKHGLVFXVVLRQDQGVHWV
WKHSDUDPHWHUVRIWKHLQWHUYLHZLQWHUPVRIOHQJWKDQGFRQILGHQWLDOLW\5HVHDUFKHUV
VKRXOGDWWKLVSRLQWDOVRVSHQGVRPHWLPHH[SODLQLQJZK\WKH\UHFRUGWKHLQWHUYLHZDQG
ZKDWVRUWRIWHFKQLFDOLVVXHVWKLVUDLVHVLQDJURXSGLVFXVVLRQ SDUWLFXODUO\WDONLQJRQHDW
DWLPH )LQDOO\LWLVLPSRUWDQWWRHPSKDVL]HWKDWWKHGLVFXVVLRQLVDERXWSHUVRQDOYLHZV
DQGH[SHULHQFHVDQGWKHUHIRUHWKHUHDUHQRULJKWRUZURQJDQVZHUV
7KHDFWXDOGLVFXVVLRQIROORZVEURDGO\WKHLQWHUYLHZJXLGHEXWHYHQDVHPL
VWUXFWXUHGJXLGHGRHVQRWXVXDOO\FRQWDLQPRUHWKDQ±EURDGRSHQHQGHGTXHVWLRQV
DFFRPSDQLHGE\DIHZFORVHGHQGHGTXHVWLRQV$IWHUDOOWKHVWUHQJWKRIWKLVIRUPDWLV
WKHGLVFXVVLRQWKDWHPHUJHVDERXWDEURDGWRSLF7KHPRGHUDWRUFDQVWHHUWKHGLVFXVVLRQ
E\XVLQJSUREHVDQGERG\ODQJXDJHDQGJHVWXULQJDUHHIIHFWLYHGHYLFHVWRFRQWUROWKH
IORZDQGNHHSLQJWKHJURXSIRFXVHG3DUWLFXODUFDUHPXVWEHWDNHQWRDOVRDOORZVRFLDOO\
OHVVGHVLUDEOHYLHZVWREHYRLFHGEHFDXVHLQDJURXSVHVVLRQUHVSRQGHQWVPD\EHPRUH
UHOXFWDQWWRVKDUHGLVSUHIHUUHGDQVZHUVWKDQLQDRQHWRRQHLQWHUYLHZ
,QWKHFRQFOXGLQJSKDVHWKHPRGHUDWRUQHHGVWRDVNLIWKHUHDUHDQ\LVVXHVRU
FRQFHUQVWKDWUHTXLUHIXUWKHUGLVFXVVLRQRUKDYHQRW\HWEHHQDGGUHVVHG%HFDXVHRIWKH
JURXSQDWXUHRIWKHVHVVLRQZHDOVRQHHGWRLQFOXGHDVKRUWZLQGLQJGRZQSKDVHDQG
VRPHSRVLWLYHIHHGEDFNVRWKDWQRERG\OHDYHVWKHVHVVLRQEHLQJGLVVDWLVILHGZLWK
WKHPVHOYHVRUZLWKWKHVRFLDOLPDJHWKH\PD\KDYHSURMHFWHG
6WUHQJWKVDQGZHDNQHVVHVRIIRFXVJURXSLQWHUYLHZLQJ
)RFXVJURXSVDUHKLJKO\YHUVDWLOHDVPHQWLRQHGHDUOLHUWKH\FDQEHXVHGLQDZLGH
UDQJHRIDUHDVIURPPDUNHWUHVHDUFKWRSROLWLFDORSLQLRQSURELQJDQGLQHGXFDWLRQDO
FRQWH[WVWKH\DUHDOVRRIWHQXVHGIRUSURJUDPPHHYDOXDWLRQWRDVVHVVWKHHIIHFWLYHQHVV
RIDSDUWLFXODUFRXUVHWRXQGHUVWDQGZKDWZDVRUZDVQRWZRUNLQJDQGZK\3HRSOHGR
QRWXVXDOO\PLQGSDUWLFLSDWLQJLQIRFXVJURXSV²LQIDFWWKH\WHQGWRILQGWKHVHVVLRQV

HQMR\DEOHDQGVWLPXODWLQJ²DQGWKHLQWHUYLHZVW\SLFDOO\\LHOGULFKGDWD7KHVHPHULWV
FRXSOHGZLWKWKHTXLFNWXUQDURXQGPDNHWKHPHWKRGSRSXODU
%HFDXVHRIWKHIOH[LEOHDQGLQIRUPDWLRQULFKQDWXUHRIWKHPHWKRGIRFXVJURXSVDUH
RIWHQXVHGLQPL[HGPHWKRGVUHVHDUFK$OWKRXJKWKH\FDQEHXVHGDVDVWDQGDORQH
PHWKRGRILQTXLU\WKLVLVQRWWKDWFRPPRQH[FHSWIRUVRPHDUHDVZKHUHIRFXVJURXSV
DUHZHOOHVWDEOLVKHG PRVWQRWDEO\PDUNHWUHVHDUFK EHFDXVHUHVHDUFKHUVRIWHQIHHOWKDW
WKH\KDYHPRUHFRQWURORYHUWKHFRQWHQWDQGFDQHOLFLWGHHSHUDQGPRUHµXQHGLWHG¶
SHUVRQDOPHDQLQJLQRQHWRRQHLQWHUYLHZV,QDSSOLHGOLQJXLVWLFUHVHDUFKWKH\KDYH
EHHQZLGHO\XVHGIRUJHQHUDWLQJLGHDVWRLQIRUPWKHGHYHORSPHQWRITXHVWLRQQDLUHVDQG
VXEVHTXHQWGHHSLQWHUYLHZV
7KHGRZQVLGHRIIRFXVJURXSLQWHUYLHZVLVWKDWWKH\QHHGTXLWHDELWRISUHSDUDWLRQ
WRVHWXSDQGWRGRWKHPZHOOWKHPRGHUDWRUQHHGVWREHDEOHWRFDUU\RXWVHYHUDO
IXQFWLRQVVLPXOWDQHRXVO\7KHUHLVDOVRPRUHQHHGWRLPSURYLVHEHFDXVHWKHQXPEHURI
TXHVWLRQVWREHDVNHGIROORZLQJWKHLQWHUYLHZJXLGHLVUHODWLYHO\ORZDQGDORWRIWKH
FRQWHQWHPHUJHVIURPZHOOPDQDJHGJURXSGLVFXVVLRQIDFLOLWDWHGE\SUREHTXHVWLRQV
:LWKUHJDUGWRWKHFRQWHQWRIWKHHOLFLWHGPDWHULDO6PLWKVRQ PHQWLRQVWZR
SRVVLEOHOLPLWDWLRQVWKHWHQGHQF\IRUFHUWDLQW\SHVRIVRFLDOO\DFFHSWDEOHRSLQLRQWR
HPHUJHDQGIRUFHUWDLQW\SHVRISDUWLFLSDQWWRGRPLQDWHWKHUHVHDUFKSURFHVV)LQDOO\DQ
LPSRUWDQWWHFKQLFDOSRLQWLVWKDWWUDQVFULELQJVXFKDQLQWHUYLHZFDQEHUDWKHUGLIILFXOW
EHFDXVHRIWKHQXPEHURISHRSOH LHYRLFHV LQYROYHG)RUWKLVUHDVRQUHVHDUFKHUV
VRPHWLPHVDOVRSUHSDUHDYLGHRUHFRUGLQJRIWKHLQWHUYLHZV²DQFLOODU\WR
DXGLRWDSLQJ²VRWKDWWKH\FDQLGHQWLI\ZKRLVVSHDNLQJDWDQ\RQHWLPH
,QWURVSHFWLYHPHWKRGV
(YHUVLQFHWKHEHJLQQLQJVRISV\FKRORJLFDOUHVHDUFKDWWKHHQGRIWKHWKFHQWXU\
SV\FKRORJLVWVKDYHEHHQWU\LQJWRILQGZD\VRIREWDLQLQJLQIRUPDWLRQDERXW
XQREVHUYDEOHPHQWDOSURFHVVHVVXFKDVWKRXJKWVIHHOLQJVDQGPRWLYHV2QHREYLRXV
VRXUFHRILQIRUPDWLRQDERXWWKHVHSURFHVVHVLVWKHLQGLYLGXDOKLPKHUVHOIDQGWKH
YDULRXVZD\VRIHOLFLWLQJVHOIUHIOHFWLRQVIURPUHVSRQGHQWVLVXVXDOO\UHIHUUHGWRXQGHU
WKHXPEUHOODWHUPµLQWURVSHFWLYHPHWKRG¶,WVXEVXPHVVHYHUDOGLIIHUHQWDSSURDFKHVWKDW
DOODLPDWKHOSLQJWKHUHVSRQGHQWVWRYRFDOL]HZKDWLVZDVJRLQJWKURXJKWKHLUPLQGV
ZKHQPDNLQJDMXGJHPHQWVROYLQJDSUREOHPRUSHUIRUPLQJDWDVN7KHGDWDJHQHUDWHG
E\WKLVPHWKRGRORJ\LVFDOOHGDµYHUEDOUHSRUW¶RUµYHUEDOSURWRFRO¶DQGWKHUHIRUH
LQWURVSHFWLYHPHWKRGVDUHDOVRUHIHUUHGWRDVµYHUEDOUHSRUWLQJ¶RUµSURWRFRODQDO\VLV¶
:HPXVWQRWHKHUHWKDWWKHUHLVVRPHLQFRQVLVWHQF\LQWKHWHUPLQRORJ\LQWKHDUHD
EHFDXVHVRPHWLPHVµLQWURVSHFWLRQ¶LVXVHGDVDKLJKHURUGHUWHUPWKDWLQFOXGHVDQ\IRUP
RIVHOIUHSRUWVXFKDVGLDULHVLQWHUYLHZVDQGHYHQVXUYH\VDQGZKHQWKHWHUPLVDSSOLHG
VREURDGO\YHUEDOUHSRUWLQJLVVHHQRQO\DVDVXEVHW 6HH*DVVDQG0DFNH\
+RZHYHUW\SLFDOO\ZKHQZHWDONDERXWLQWURVSHFWLYHPHWKRGVZHXVXDOO\LPSO\WZR
VSHFLILFWHFKQLTXHVµWKLQNDORXG¶DQGµUHWURVSHFWLYHUHSRUWVLQWHUYLHZV¶WKHODWWHUDOVR
FDOOHGµVWLPXODWHGUHFDOO¶7KHPDLQGLIIHUHQFHEHWZHHQWKHVHWZRW\SHVRILQWURVSHFWLRQ

OLHVLQWKHWLPLQJWKHWKLQNDORXGWHFKQLTXHLVDSSOLHGUHDOWLPHFRQFXUUHQWO\WRWKH
H[DPLQHGWDVNSURFHVVZKHUHDVWKHUHWURVSHFWLYHLQWHUYLHZDVWKHQDPHVXJJHVWV
KDSSHQVDIWHUWKHWDVNSURFHVVKDVEHHQFRPSOHWHG
$V*DVVDQG0DFNH\ H[SODLQWKHDVVXPSWLRQXQGHUO\LQJLQWURVSHFWLRQLVWKDW
LWLVSRVVLEOHWRREVHUYHLQWHUQDOSURFHVVHVWKDWLVZKDWLVJRLQJRQLQRQH¶V
FRQVFLRXVQHVVLQPXFKWKHVDPHZD\DVRQHFDQREVHUYHH[WHUQDOUHDOZRUOGHYHQWV2I
FRXUVHWKLVµREVHUYDWLRQ¶UHTXLUHVWKHDFWLYHFRRSHUDWLRQRIWKHSHUVRQZKRVHWKRXJKW
SURFHVVHVZHDUHH[DPLQLQJDQGDVXEVHTXHQWDVVXPSWLRQLVWKHUHIRUHWKDWKXPDQV
KDYHDFFHVVWRWKHLULQWHUQDOWKRXJKWSURFHVVHVDWVRPHOHYHODQGFDQYHUEDOL]HWKHP
(ULFVVRQ VHHDOVR(ULFVVRQDQG6LPRQ REVHUYHVWKDWLQWURVSHFWLRQKDV
EHHQSUHVHQWLQZHVWHUQWKRXJKWIRUDORQJWLPHEXWDSSHDUHGDVDPHWKRGRIVFLHQWLILF
LQTXLU\DWWKHHQGRIWKHWKFHQWXU\ZKHQSV\FKRORJ\ILUVWHPHUJHGDVDVFLHQWLILF
GLVFLSOLQH,QIDFWHDUO\SV\FKRORJLVWVUHOLHGKHDYLO\RQLQWURVSHFWLYHDQDO\VLVRI
WKRXJKWVDQGPHPRULHVEXWWKHPHWKRGVRRQORVWIDYRXUEHFDXVHLWFDPHWREHUHJDUGHG
DVXQUHOLDEOHDQGUHDFWLYH LHLQWHUIHULQJZLWKWKHDFWXDOWKRXJKWSURFHVVHVUDWKHUWKDQ
SXUHO\UHSRUWLQJWKHP ,WZDVRQO\DIWHUFRJQLWLYHSV\FKRORJ\JUDGXDOO\UHSODFHGWKH
GRPLQDQFHRIEHKDYLRXULVPLQWKHVWKDWYHUEDOUHSRUWLQJHVSHFLDOO\WKLQNDORXG
WHFKQLTXHVZHUHUHLQWURGXFHGLQWRVFLHQWLILFUHVHDUFKDQGWKHPHWKRGRORJ\ZDV
V\VWHPDWLFDOO\UHILQHG
%HFDXVHRIWKHVLJQLILFDQFHRIYDULRXVPHQWDORSHUDWLRQVLQSURGXFLQJODQJXDJH
LQWURVSHFWLYHPHWKRGVKDYHEHHQVHHQWRKDYHFRQVLGHUDEOHUHOHYDQFHWRDSSOLHG
OLQJXLVWLFVIRUVHYHUDOGHFDGHV 6HH) UFKDQG.DVSHU .RUPRV SRLQWV
RXWWKDWWKHPHWKRGLVSDUWLFXODUO\LPSRUWDQWLQVHFRQGODQJXDJHUHVHDUFKEHFDXVHLWFDQ
KHOSWRXQFRYHUWKHFRJQLWLYHDQGSV\FKROLQJXLVWLFSURFHVVHVXQGHUO\LQJODQJXDJH
SHUIRUPDQFH%HFDXVHLQWURVSHFWLYHPHWKRGVFDQEHXVHGWRVXSSOHPHQWDOPRVWDQ\
RWKHUUHVHDUFKPHWKRG,EHOLHYHWKHUHLVDODUJHVFRSHIRULQFUHDVLQJLWVDSSOLFDWLRQLQ
RXUILHOG
7KLQNDORXGWHFKQLTXH
2QHRIWKHPDLQSURSRQHQWVRILQWURVSHFWLYHPHWKRGVLQSV\FKRORJ\(ULFVVRQ
H[SODLQVWKDWWKHFORVHVWFRQQHFWLRQEHWZHHQWKLQNLQJSURFHVVHVDQGYHUEDOUHSRUWVDUH
IRXQGZKHQSDUWLFLSDQWVDUHLQVWUXFWHGWRYHUEDOL]HWKHLURQJRLQJWKRXJKWVZKLOH
IRFXVLQJRQDWDVN7KLVWHFKQLTXHKDVFRPHWREHNQRZQDVµWKLQNDORXG¶DQGLW
LQYROYHVWKHFRQFXUUHQWYRFDOL]DWLRQRIRQH¶VµLQQHUVSHHFK¶ZLWKRXWRIIHULQJDQ\
DQDO\VLVRUH[SODQDWLRQ7KXVUHVSRQGHQWVDUHDVNHGWRYHUEDOL]HRQO\WKHWKRXJKWVWKDW
HQWHUWKHLUDWWHQWLRQZKLOHVWLOOLQWKHUHVSRQGHQW¶VVKRUWWHUPPHPRU\,QWKLVZD\WKH
SURFHGXUHGRHVQRWDOWHUWKHVHTXHQFHRIWKRXJKWVPHGLDWLQJWKHFRPSOHWLRQRIDWDVN
DQGDFFRUGLQJWR(ULFVVRQFDQWKHUHIRUHEHDFFHSWHGDVYDOLGGDWDRQWKLQNLQJ7KH
UHVXOWLQJYHUEDOSURWRFROLVUHFRUGHGDQGWKHQDQDO\VHG
,WLVFOHDUWKDWSURYLGLQJWKLQNDORXGFRPPHQWDU\LVQRWDQDWXUDOSURFHVVDQG
WKHUHIRUHSDUWLFLSDQWVQHHGSUHFLVHLQVWUXFWLRQVDQGVRPHWUDLQLQJEHIRUHWKH\FDQEH

H[SHFWHGWRSURGXFHXVHIXOGDWD7KH\QHHGWREHWROGWRIRFXVRQWKHLUWDVN
SHUIRUPDQFHUDWKHUWKDQRQWKHµWKLQNDORXG¶DQGWKH\XVXDOO\QHHGWREHUHPLQGHGWR
NHHSRQWDONLQJZKLOHFDUU\LQJRXWDQDFWLYLW\ IRUH[DPSOHµ:KDWPDGH\RXGRWKDW"¶
RUµ:KDWDUH\RXWKLQNLQJDERXWQRZ"¶ ,QWHUPVRIWUDLQLQJWKHUHVHDUFKHUQHHGVILUVW
WRGHPRQVWUDWHWKHSURFHGXUHVRQDVLPLODUWDVN RUVKRZDYLGHRWKDWPRGHOVWKH
UHTXLUHGEHKDYLRXU DQGWKHQUHVSRQGHQWVDUHJLYHQWULDOVRQZDUPXSWDVNVXQWLOWKH\
JHWWKHIHHORIUHSRUWLQJZLWKRXWH[SODLQLQJRUMXVWLI\LQJ 6HH&RKHQ *DVVDQG
0DFNH\ SRLQWRXWWKDWLWLVVWLOODQXQUHVROYHGLVVXHKRZVXFKWUDLQLQJDIIHFWVWKH
YDOLGLW\RIWKHYHUEDOUHSRUWEHFDXVHYHUEDOL]DWLRQVFDQEHLQIOXHQFHGE\WKHSUHFHGLQJ
LQSXW LHFRQIRXQGLQJYDULDEOHVFDQEHLQWURGXFHG
5HWURVSHFWLYHLQWHUYLHZRUVWLPXODWHGUHFDOO
,QµUHWURVSHFWLRQ¶WKHUHVSRQGHQWVYHUEDOL]HWKHLUWKRXJKWVDIWHUWKH\KDYHSHUIRUPHGD
WDVNRUPHQWDORSHUDWLRQ,QVXFKFDVHVWKHUHOHYDQWLQIRUPDWLRQQHHGVWREHUHWULHYHG
IURPORQJWHUPPHPRU\DQGWKXVWKHYDOLGLW\RIUHWURVSHFWLYHSURWRFROVGHSHQGVRQWKH
WLPHLQWHUYDOEHWZHHQWKHRFFXUUHQFHRIDWKRXJKWDQGLWVYHUEDOUHSRUW(ULFVVRQ
FRQFOXGHVWKDWIRUWDVNVZLWKUHODWLYHO\VKRUWUHVSRQVHODWHQFLHV OHVVWKDQ
VHFRQGV VXEMHFWVDUHDEOHWRSURGXFHDFFXUDWHUHFROOHFWLRQVEXWIRUFRJQLWLYH
SURFHVVHVRIORQJHUGXUDWLRQWKHGLIILFXOW\RIDFFXUDWHUHFDOORISULRUWKRXJKWV
LQFUHDVHV$OWKRXJK0DFNH\DQG*DVV UHSRUWVRPHLPPHGLDWHUHFDOOVWXGLHVLQ
DSSOLHGOLQJXLVWLFUHVHDUFKWKH\SRLQWRXWWKDWWKLVW\SHRIUHWURVSHFWLRQLVGLIILFXOWWR
LPSOHPHQWLQDUHDVVXFKDVLQWHUDFWLRQUHVHDUFK,QVWHDGWKH\UHFRPPHQGDQRWKHUW\SH
RIUHWURVSHFWLRQWKDWLVRIWHQUHIHUUHGWRDVµVWLPXODWHGUHFDOO¶RU µUHWURVSHFWLYH
LQWHUYLHZ¶
6WLPXODWHGUHFDOOWDNHVSODFHVRPHWLPHDIWHUWKHRFFXUUHQFHRIWKHWDUJHWHGWKRXJKW
SURFHVVHVDQGLQRUGHUWRKHOSWKHUHVSRQGHQWVWRUHWULHYHWKHLUUHOHYDQWWKRXJKWVVRPH
VRUWRIVWLPXOXVLVXVHGDVDVXSSRUWIRUWKHUHFDOO KHQFHWKHWHUPµVWLPXODWHGUHFDOO¶
VXFKDVZDWFKLQJWKHUHVSRQGHQW¶VRZQWDVNSHUIRUPDQFHRQYLGHROLVWHQLQJWRD
UHFRUGLQJRIZKDWWKHSHUVRQKDVVDLGRUVKRZLQJWKHSHUVRQDZULWWHQZRUNWKDWKHVKH
KDVSURGXFHG7KXVWKHXQGHUO\LQJLGHDLVWKDWVRPHWDQJLEOH YLVXDORUDXUDO UHPLQGHU
RIDQHYHQWZLOOVWLPXODWHUHFDOOWRDQH[WHQWWKDWWKHUHVSRQGHQWVFDQUHWULHYHDQGWKHQ
YHUEDOL]HZKDWZDVJRLQJRQLQWKHLUPLQGVGXULQJWKHHYHQW *DVVDQG0DFNH\
:KLOHWKHTXDOLW\RIWKHUHFDOOLQHYLWDEO\VXIIHUVZLWKWKHWLPHODSVHUHJDUGOHVVRIWKH
QDWXUHRIWKHVWLPXOXVSURPSW(ULFVVRQ DOVRKLJKOLJKWVDPHULWRIWKLVDSSURDFK
LWLVWKHOHDVWUHDFWLYHRIDOOWKHLQWURVSHFWLYHWHFKQLTXHVEHFDXVHWKHWDUJHWHGWKRXJKW
SURFHVVHVDUHQRWDIIHFWHGE\WKHSURFHGXUHLQDQ\ZD\SDUWLFXODUO\LIZHGRQRWHYHQ
WHOOWKHUHVSRQGHQWVGXULQJWKHWDVNWKDWZHZRXOGOLNHWRDVNWKHPWRJLYHDUXQQLQJ
FRPPHQWDU\DIWHUZDUGV
+RZFDQZHLPSURYHWKHTXDOLW\RIWKHUHWURVSHFWLYHGDWD"%DVHGRQ*DVVDQG
0DFNH\ 0DFNH\DQG*DVV WKHVWXGLHVLQ) UFKDQG.DVSHU DV
ZHOODVP\RZQH[SHULHQFHWKHIROORZLQJUHFRPPHQGDWLRQVFDQEHPDGH

x :HQHHGWRWU\DQGNHHSWKHLQWHUYDOEHWZHHQWKHWDVNDQGWKHUHWURVSHFWLYH
LQWHUYLHZDVVKRUWDVSRVVLEOH2IWHQ IRUH[DPSOHLQVWUDWHJ\UHVHDUFK WKH
UHVHDUFKHUILUVWQHHGVWRWUDQVFULEHDQGDQDO\VHWKHUHVSRQGHQW¶VVSHHFKWRPDNHWKH
UHWURVSHFWLYHVHVVLRQUHDOO\PHDQLQJIXOEHFDXVHRQO\VXFKDWKRURXJKHQJDJHPHQW
ZLWKWKHWH[WFDQUHYHDOFHUWDLQVXEWOHLVVXHVWKDWUHTXLUHFODULILFDWLRQKRZHYHU
HYHQLQVXFKFDVHVWKHWLPHODSVHVKRXOGQRWH[FHHGWZRGD\V DQGVKRXOG
SUHIHUDEO\EHOHVVWKDQKRXUV
x 7KHULFKHUWKHFRQWH[WXDOLQIRUPDWLRQDQGVWLPXOXVDERXWWKHWDUJHWHYHQWWKH
EHWWHU7KHUHIRUHOLVWHQLQJWRDUHFRUGLQJLVEHWWHUWKDQPHUHO\ORRNLQJDWWKH
WUDQVFULSWDQGZDWFKLQJDYLGHRUHFRUGLQJLVVXSHULRUWROLVWHQLQJWRDQDXGLR
UHFRUGLQJ
x :HVKRXOGRQO\HQFRXUDJHWKHUHFDOORIGLUHFWO\UHWULHYDEOHLQIRUPDWLRQ IRU
H[DPSOHµ:KDWZHUH\RXWKLQNLQJRI"¶ UDWKHUWKDQH[SODQDWLRQVRULQWHUSUHWDWLRQV
x ,ISRVVLEOHWKHUHVSRQGHQWVVKRXOGQRWEHLQIRUPHGRIWKHVXEVHTXHQWUHWURVSHFWLYH
LQWHUYLHZ RUWKHH[DFWGHWDLOVRIZKDWLWZLOOLQYROYH EHIRUHWKHFRPSOHWLRQRIWKH
WDVNVRWKDWWKHIRUHNQRZOHGJHGRHVQRWDIIHFWWKHLUSHUIRUPDQFH
x *DVVDQG0DFNH\ SRLQWRXWWKDWVWLPXODWHGUHFDOOGRHVQRWUHTXLUHH[WHQVLYH
SDUWLFLSDQWWUDLQLQJEHFDXVHVLPSOHLQVWUXFWLRQVDQGDGLUHFWPRGHODUHRIWHQ
HQRXJK:HQHHGWREHFDUHIXOQRWWRFXHWKHUHVSRQGHQWVLQWRDQ\DVSHFWVRIWKH
VWXG\WKDWDUHXQQHFHVVDU\DQGPLJKWDIIHFWWKHLUUHVSRQVHV
x 'XULQJWKHUHWURVSHFWLYHLQWHUYLHZZHVKRXOGWU\DQGLQYROYHWKHUHVSRQGHQWVDV
PXFKDVSRVVLEOHLQYROXQWHHULQJGDWD IRUH[DPSOHDVNWKHPWRVWRSWKHFDVVHWWH
UHFRUGHUZKHQWKH\UHPHPEHUVRPHWKLQJWKDWPLJKWEHXVHIXO DQGHYHQZKHQZH
KLJKOLJKWSDUWVWRFRPPHQWRQ IRUH[DPSOHE\VWRSSLQJWKHSOD\LQJRIWKHWDSHDW
FULWLFDOSRLQWV ZHVKRXOGDYRLGOHDGLQJTXHVWLRQVRUDQ\RWKHUUHVHDUFKHU
LQWHUIHUHQFH
x ,ISRVVLEOHWKHUHWURVSHFWLYHLQWHUYLHZVKRXOGEHLQWKHUHVSRQGHQW¶V/ RULQWKH
ODQJXDJHWKHUHVSRQGHQWFKRRVHV
x 7KHZKROHUHWURVSHFWLYHSURFHGXUHQHHGVWREHSLORWHG*DVVDQG0DFNH\
RIIHUFRPSOHWHVDPSOHSURWRFROVIRUVWLPXODWHGUHFDOOSURFHGXUHVLQRUDOLQWHUDFWLRQ
VHWWLQJV
$QDO\VLQJLQWURVSHFWLYHGDWD
-XVWEHFDXVHLQWURVSHFWLYHUHSRUWVFRPHµVWUDLJKWIURPWKHKRUVH¶VPRXWK¶WKH\FDQQRW
EHWDNHQDVWKHXOWLPDWHUHYHODWLRQVDERXWWKRXJKWSURFHVVHV5DWKHUDV.DVSHU
HPSKDVL]HVWKH\UHSUHVHQWLQIRUPDWLRQDERXWZKDWLVJRLQJWKURXJKVRPHRQH¶VPLQG
ZKHQDGGUHVVLQJDWDVNRULVVXHDQGWKHXQGHUO\LQJFRJQLWLYHSURFHVVHVQHHGWREH
LQIHUUHGIURPWKHVHYHUEDOUHSRUWVMXVWDVZLWKRWKHUW\SHVRIGDWD,QRWKHUZRUGV
YHUEDOUHSRUWVPXVWEHVHHQRQO\DVRQHYDOXDEOHVRXUFHRIGDWDWKDWQHHGVWREH
VXEMHFWHGWRTXDOLWDWLYHGDWDDQDO\VLVVLPLODUWRDOOWKHRWKHUTXDOLWDWLYHGDWDW\SHV 6HH

&KDSWHU 6ZDLQ IXUWKHUDUJXHVWKDWYHUEDOSURWRFROVQRWRQO\UHSRUWFRJQLWLYH
SURFHVVHVEXWKDYHWKHSRZHUWRLQIOXHQFHFRJQLWLRQDVZHOODQGDOWKRXJKJLYHQWKH
LQWHQVLYHSDUWLFLSDQWHQJDJHPHQWUHTXLUHGE\PRVWTXDOLWDWLYHWHFKQLTXHVLWLVQRW
XQLTXHWRKDYHWKHWKUHDWRIVRPHVRUWRIDQLQWHUDFWLRQEHWZHHQUHVSRQGHQWGDWDDQG
GHVLJQSDUDPHWHUVLQJHQHUDO IRUH[DPSOHµUHVHDUFKHUHIIHFW¶µSUDFWLFHHIIHFW¶RU
µSDQHOFRQGLWLRQLQJ¶VHH6HFWLRQ ZHQHHGWREHDULQPLQGWKDWLQWURVSHFWLYHGDWD
LVYHU\VHQVLWLYHWRWKLVEHFDXVHWKHDUWLFXODWLRQRIWKRXJKWVFDQHDVLO\ DQG6ZDLQ
DUJXHVLQHYLWDEO\ PHGLDWHGHYHORSPHQW7KLVDGGVIXUWKHUZHLJKWWRWKHIDFWWKDWZH
QHHGWRH[HUFLVHFDXWLRQZKHQLQWHUSUHWLQJLQWURVSHFWLYHGDWD
6WUHQJWKVDQGZHDNQHVVHVRILQWURVSHFWLYHPHWKRGV
7KHPDMRUDGYDQWDJHRIWKHXVHRILQWURVSHFWLYHPHWKRGVLVREYLRXVE\DSSO\LQJWKHP
ZHFDQSRWHQWLDOO\JDLQDFFHVVWRPHQWDOSURFHVVHVWKDWDUHFHQWUDOIRUH[DPSOHWR
ODQJXDJHSURFHVVLQJDQGSURGXFWLRQEXWZKLFKDUHLQDFFHVVLEOHE\DQ\RWKHUPHDQV
9HUEDOUHSRUWVDUHDOVRYHUVDWLOHDVWKH\FDQEHXVHGLQDYDULHW\RIGRPDLQV²
UHWURVSHFWLYHLQWHUYLHZVFDQXWLOL]HGLYHUVHVWLPXOLIURPYLGHRUHFRUGLQJVWRZULWWHQ
SURGXFWVDQGHYHQWHVWRUTXHVWLRQQDLUHUHVSRQVHV,QWURVSHFWLRQFDQEHFRPELQHGZLWK
PRVWRWKHUUHVHDUFKPHWKRGV VHH6HFWLRQ DQGLWKDVEHHQJHQHUDOO\IRXQGWR
JUHDWO\HQKDQFHWKHULFKQHVVRIWKHGDWDREWDLQHGDQGFDQDOVRLQFUHDVHWRDFRQVLGHUDEOH
H[WHQWWKHUHOLDELOLW\RIWKHGDWDDQDO\VLV .RUPRV
$OWKRXJKWKHUHOLDELOLW\DQGWKHYDOLGLW\RILQWURVSHFWLYHPHDVXUHVKDVEHHQ
TXHVWLRQHGLQSV\FKRORJ\LWDSSHDUVWKDWSURSHUO\GHVLJQHGDQGFRQGXFWHG
LQWURVSHFWLYHVWXGLHVPHHWWKHUHTXLUHPHQWVRIVFLHQWLILFUHVHDUFK 6HH&RKHQ
(ULFVVRQ*DVVDQG0DFNH\ $V(ULFVVRQFRQFOXGHVWKHWKHRUHWLFDODQG
PHWKRGRORJLFDOTXHVWLRQVUDLVHGDERXWYHUEDOUHSRUWVKDYHQHYHUFDVWGRXEWRQSHRSOH¶V
DELOLW\WRUHFDOOSDUWRIWKHLUWKRXJKWVHTXHQFHVDQGDOOPDMRUWKHRUHWLFDOIUDPHZRUNV
FRQFHUQHGZLWKWKLQNLQJKDYHDGYRFDWHGWKHXVHRIYHUEDOO\UHSRUWHGVHTXHQFHVRI
WKRXJKWV
2QWKHQHJDWLYHVLGHDV&RKHQ VXPPDUL]HVFULWLFVRILQWURVSHFWLYHPHWKRGV
QRWHWKDWPXFKRIFRJQLWLYHSURFHVVLQJLVLQDFFHVVLEOHEHFDXVHLWLVXQFRQVFLRXVDQG
HYHQFHUWDLQFRQVFLRXVSURFHVVHVFDQEHDUJXHGWREHWRRFRPSOH[WREHFDSWXUHGLQ
YHUEDOSURWRFROV)XUWKHUPRUHYHUEDOUHSRUWVDUHQRWLPPXQHWRWKHVRFLDOGHVLUDELOLW\
ELDVDQGWKHUHLVDGDQJHUWKDWWKHEDFNJURXQGNQRZOHGJHRUµIRONSV\FKRORJ\¶RIWKH
UHVSRQGHQWPLJKWFRQWDPLQDWHWKHGDWD,QDGGLWLRQLQVWLPXODWHGUHFDOOVWXGLHVWKHUHLV
DQLQHYLWDEOHLQIRUPDWLRQORVVGXHWRWKHWLPHODSVHEHWZHHQWKHWDVNDQGWKH
UHWURVSHFWLYHLQWHUYLHZ
2QHRIWKHPRVWVHULRXVFRQFHUQVDERXWLQWURVSHFWLYHGDWDLVWKHUHDFWLYHHIIHFWVRI
SURGXFLQJYHUEDOUHSRUWV7KHSURFHGXUHFDQLQWHUIHUHZLWKWDVNFRPSOHWLRQDQGWKRXJKW
SURFHVVHVDQGLQDVWXG\E\6WUDWPDQDQG+DPS/\RQV RQWKHUHDFWLYLW\RI
WKLQNDORXGVLQZULWLQJWKHUHVHDUFKHUVIRXQGVRPH EXWQRWFRQFOXVLYH HYLGHQFHWKDW
WKLQNDORXGSURWRFROVLQIOXHQFHGWKHFRUUHFWLRQRIRUJDQL]DWLRQDOOHYHOHUURUVDVZHOODV

WKHDPRXQWDQGNLQGRIPLFURVWUXFWXUDOPHDQLQJFKDQJHV,QDGGLWLRQ,KDYHDOUHDG\
PHQWLRQHG6ZDLQ¶V FRQFHUQWKDWYHUEDOUHSRUWVDUHQRWQHXWUDOµEUDLQGXPSV¶EXW
UDWKHUSURFHVVHVRIFRPSUHKHQGLQJDQGUHVKDSLQJH[SHULHQFH
&DVHVWXGLHV
7KHµFDVHVWXG\¶²DVWKHWHUPVXJJHVWV²LVWKHVWXG\RIWKHµSDUWLFXODULW\DQG
FRPSOH[LW\RIDVLQJOHFDVH¶ 6WDNH[L :KDWLVDµFDVH¶"&DVHVDUHSULPDULO\
SHRSOHEXWUHVHDUFKHUVFDQDOVRH[SORUHLQGHSWKDSURJUDPPHDQLQVWLWXWLRQDQ
RUJDQL]DWLRQRUDFRPPXQLW\,QIDFWDOPRVWDQ\WKLQJFDQVHUYHDVDFDVHDVORQJDVLW
FRQVWLWXWHVDVLQJOHHQWLW\ZLWKFOHDUO\GHILQHGERXQGDULHV5HVHDUFKVWXGLHVVRPHWLPHV
GHVFULEHDVHULHVRIµPXOWLSOHFDVHV¶WKLVLVILQHDVORQJDVHDFKLQGLYLGXDOFDVHLV
FRQVLGHUHGWKHWDUJHWRIDVHSDUDWHVWXG\)RUH[DPSOH'XII LQSUHVV GHVFULEHVKHU
HWKQRJUDSKLFFDVHVWXG\RIELOLQJXDOHGXFDWLRQLQ+XQJDU\DVKDYLQJDWOHDVWWKUHH
OHYHOVWKHILUVWOHYHOZDVRQHFDVHWKHFRXQWU\LWVHOIWKHVHFRQGOHYHOLQYROYHGWKUHH
FDVHVWKHWKUHHVFKRROVVWXGLHGDQGWKHWKLUGOHYHOLQFOXGHGDVPDQ\DVHLJKWFDVHVWKH
WHDFKHUSDUWLFLSDQWV
+RZDUHWKHVHOHFWHGFDVHVVWXGLHG"&DVHVWXG\UHVHDUFKHUVXVXDOO\FRPELQHD
YDULHW\RIGDWDFROOHFWLRQPHWKRGVVXFKDVLQWHUYLHZVREVHUYDWLRQDQGGRFXPHQW
DUFKLYHV$OWKRXJKFDVHVWXGLHVDUHW\SLFDOO\GLVFXVVHGXQGHUWKHODEHORITXDOLWDWLYH
UHVHDUFK EHFDXVHDVLQJOHFDVHFDQQRWEHUHSUHVHQWDWLYHRIDSRSXODWLRQ DFWXDOFDVH
VWXGLHVRIWHQLQFOXGHTXDQWLWDWLYHGDWDFROOHFWLRQLQVWUXPHQWVDVZHOOVXFKDV
TXHVWLRQQDLUHV 9HUVFKXUHQ 7KXVWKHFDVHVWXG\LVQRWDVSHFLILFWHFKQLTXHEXW
UDWKHUDPHWKRGRIFROOHFWLQJDQGRUJDQL]LQJGDWDVRDVWRPD[LPL]HRXUXQGHUVWDQGLQJ
RIWKHXQLWDU\FKDUDFWHURIWKHVRFLDOEHLQJRUREMHFWVWXGLHG,QPDQ\ZD\VLWLVWKH
XOWLPDWHTXDOLWDWLYHPHWKRGIRFXVLQJRQWKHµ3DUWLFXODU2QH¶
0DLQIHDWXUHVRIDFDVHVWXG\
6WDNH GLVWLQJXLVKHVWKUHHW\SHVRIFDVHVWXG\ D WKHµLQWULQVLFFDVHVWXG\¶
LVXQGHUWDNHQWRXQGHUVWDQGWKHLQWULJXLQJQDWXUHRIDSDUWLFXODUFDVH7KDWLVWKHFDVHLV
RILQWHUHVWQRWEHFDXVHLWLOOXVWUDWHVVRPHWKLQJRUUHSUHVHQWVRWKHUFDVHVEXWEHFDXVHRI
LWVRZQYDOXHRUVSHFLDOLW\ E WKHµLQVWUXPHQWDOFDVHVWXG\¶LVLQWHQGHGWRSURYLGH
LQVLJKWLQWRDZLGHULVVXHZKLOHWKHDFWXDOFDVHLVRIVHFRQGDU\LQWHUHVWLWIDFLOLWDWHVRXU
XQGHUVWDQGLQJRIVRPHWKLQJHOVHDQG F WKHµPXOWLSOHRUFROOHFWLYHFDVHVWXG\¶ZKHUH
WKHUHLVHYHQOHVVLQWHUHVWLQRQHSDUWLFXODUFDVHDQGDQXPEHURIFDVHVDUHVWXGLHG
MRLQWO\LQRUGHUWRLQYHVWLJDWHDSKHQRPHQRQRUJHQHUDOFRQGLWLRQ7KXVDPXOWLSOHFDVH
VWXG\LVLQHIIHFWDQLQVWUXPHQWDOFDVHVWXG\H[WHQGHGWRVHYHUDOFDVHV$VRQHRIWKH
OHDGLQJFDVHVWXG\UHVHDUFKHUVLQDSSOLHGOLQJXLVWLFV'XII GHVFULEHVPRVWRI
KHUVWXGHQWVFRQGXFWFDVHVWXGLHVZLWK±IRFDOSDUWLFLSDQWVLQRQHRUPRUHVLWHVDQG
WKLVPXOWLSOHFDVHVWXG\IRUPDWFDQEHVHHQDVIDLUO\W\SLFDO'XIIH[SODLQVWKDW
FKRRVLQJFDVHVLQLWLDOO\PHDQVWKDWHYHQLIWKHUHLVDWWULWLRQDPRQJWKHSDUWLFLSDQWV
DQGWKHUHXVXDOO\LV WKHUHZLOOOLNHO\EH±FDVHVUHPDLQLQJ

%HFDXVHRIWKHGHWDLOHGLQIRUPDWLRQZHZLVKWRJDWKHUDERXWWKHFDVHUHVHDUFKHUV
XVXDOO\VSHQGDQH[WHQGHGSHULRGRIWLPHH[DPLQLQJWKHFDVHLQLWVQDWXUDO
VXUURXQGLQJV7KHUHIRUHFDVHVWXGLHVDUHRIWHQDWOHDVWSDUWLDOO\ORQJLWXGLQDOLQQDWXUH
'XULQJWKLVSHULRG²DVPHQWLRQHGDERYH²DOPRVWDQ\W\SHRIUHVHDUFKPHWKRGFDQEH
XVHGWKDWFDQ\LHOGFDVHVSHFLILFGDWD(VSHFLDOO\LIWKHFDVHLVQRWDQLQGLYLGXDOEXW
VD\DQLQVWLWXWLRQWKLVFDQOHDGWRDQRYHUDEXQGDQFHRIGDWDDQGLQRUGHUWRUHPDLQRQ
WKHWRSRIDOOWKHGHWDLOVLWPD\EHXVHIXOWRKDYHDµGDWDJDWKHULQJSODQ¶ZKLFKGHILQHV
WKHFDVHWKHUHVHDUFKTXHVWLRQVWKHGDWDVRXUFHVDQGWKHDOORFDWLRQRIWLPH
7KHJHQHUDOL]DELOLW\LVVXH
7KHPDLQUHVHUYDWLRQDERXWFDVHVWXGLHVSDUWLFXODUO\LIWKH\FRQFHUQLQGLYLGXDOVUDWKHU
WKDQVRFLDOREMHFWVLVWKHLUJHQHUDOL]DELOLW\+RZFDQNQRZOHGJHFRPLQJIURPRQH
LQHYLWDEO\LGLRV\QFUDWLFVRXUFHEHRIZLGHUUHOHYDQFH",Q6HFWLRQZHVDZWKDW
TXDOLWDWLYHUHVHDUFKLVQRWRYHUO\FRQFHUQHGZLWKJHQHUDOL]DELOLW\DVORQJDVWKH
UHVHDUFKHUEHOLHYHVWKDWWKHVSHFLILFLQGLYLGXDOPHDQLQJREWDLQHGIURPWKHVDPSOHLV
LQVLJKWIXODQGHQOLJKWHQLQJ<HWWKHUHLVDGPLWWHGO\DJUHDWSV\FKRORJLFDOGLIIHUHQFH
EHWZHHQWUXVWLQJDVPDOOVFDOHTXDOLWDWLYHVWXG\ UHJDUGOHVVRIWKHVDPSOHVL]H DQGD
VLQJOHFDVHVWXG\$VDUHVXOWWKLVLVVXHKDVEHHQWKHVXEMHFWRIKHDWHGGLVFXVVLRQVLQ
WKHOLWHUDWXUHDQGZDUUDQWVIXUWKHUDWWHQWLRQKHUHWRR
$VZHVDZDERYHLQLQWULQVLFFDVHVWXGLHVWKHFDVHUHSUHVHQWVVRPHXQLTXHRUQRW
\HWXQGHUVWRRGIHDWXUHZKLFKPLJKWH[SODLQQRWRQO\WRWKHUHVHDUFKHUEXWDOVRWRWKH
UHVHDUFKDXGLHQFHWKHUHOHYDQFHRIWKHFDVHWRRXUJHQHUDOXQGHUVWDQGLQJRIDZLGHU
GRPDLQ)RUH[DPSOHIHZSHRSOHZRXOGTXHVWLRQWKDWDFDVHVWXG\RIDµODQJXDJH
VDYDQW¶ DQLQGLYLGXDOZLWKOHVVWKDQDYHUDJHPHQWDOVNLOOVEXWZLWKYHU\VSHFLDO
ODQJXDJHDELOLWLHV RUDXQLTXHO\JLIWHGODQJXDJHOHDUQHUZRXOGEHDZRUWKZKLOHDQGD
ZLGHO\VRXJKWDIWHULQYHVWLJDWLRQLQDSSOLHGOLQJXLVWLFV0XOWLSOHFDVHVWXGLHVDJDLQ
KDYHVDWLVIDFWRU\IDFHYDOLGLW\EHFDXVHRIWKHLUFRPSDUDWLYHQDWXUH 6HHIRUH[DPSOH
:RQJ)LOOPRUHZKLFKKDVEHHQDSDUWLFXODUO\LQIOXHQWLDOVWXG\LQDSSOLHG
OLQJXLVWLFVUHJDUGOHVVRIWKHIDFWWKDWLWRQO\LQYROYHGILYHFDVHV 7KHUHDOLVVXHWKHQLV
ZKDW6WDNH KDVWHUPHGWKHLQVWUXPHQWDOFDVHVWXG\ZKHUHZHH[DPLQHD
FDVHWRJDLQLQVLJKWVLQWRDPRUHJHQHUDOPDWWHU7KHUHDUHWZRSRLQWVWRFRQVLGHUKHUH
x $QDO\WLFJHQHUDOL]DWLRQ'XII LQSUHVV SRLQWVRXWWKDWDOWKRXJKWKHFRQFHSWRI
µJHQHUDOL]DWLRQ¶LVXVXDOO\XQGHUVWRRGDVJHQHUDOL]LQJWRSRSXODWLRQVLWFDQDOVR
UHIHUWRWKHJHQHUDOL]DWLRQWRWKHRUHWLFDOPRGHOVUHVXOWLQJLQµDQDO\WLF
JHQHUDOL]DWLRQ¶7KHHDUO\FDVHVWXGLHVLQDSSOLHGOLQJXLVWLFV VHH6HFWLRQ
EHORZ PDQ\RIZKLFKIDOOXQGHUWKHLQVWUXPHQWDOFDWHJRU\UHSUHVHQWWKLVDSSURDFK
ZHOOEHFDXVHWKH\OHGWRWKHIRUPXODWLRQRIVHYHUDOWKHRUHWLFDOSULQFLSOHVDQG
PRGHOVWKDWDUHVWLOOFRQVLGHUHGUHOHYDQWWRGD\
x 3XUSRVLYHVDPSOLQJ,QLQVWUXPHQWDOFDVHVWXGLHVDNH\LVVXHXQGHUO\LQJWKHEURDGHU
UHOHYDQFHRIWKHLQYHVWLJDWLRQLVWKHVDPSOLQJVWUDWHJ\WKDWUHVXOWVLQWKHVHOHFWLRQRI
WKHSDUWLFXODUFDVH,QGHHG'XII LQSUHVV FRQILUPVWKDWFDVHVHOHFWLRQDQG

VDPSOLQJDUHµDPRQJWKHPRVWFUXFLDOFRQVLGHUDWLRQVLQFDVHVWXG\UHVHDUFK¶
6HFWLRQSUHVHQWVYDULRXVTXDOLWDWLYHVDPSOLQJVWUDWHJLHVWKDWIDOOXQGHUWKH
EURDGHUFDWHJRU\RIµSXUSRVLYHVDPSOLQJ¶,ZRXOGDUJXHWKDWLIFRQGXFWHGZHOO
VHYHUDORIWKHVHVWUDWHJLHV²VXFKDVW\SLFDOFULWHULRQH[WUHPHGHYLDQWDQGFULWLFDO
FDVHVDPSOLQJ²ZLOOOHDGWRFDVHVZKRVHVWXG\FDQKDYHDORWWRRIIHUWRWKHZLGHU
UHVHDUFKFRPPXQLW\SDUWLFXODUO\LQWHUPVRIQHZFRQFHSWXDOL]DWLRQVDQGQRYHO
SURSRVLWLRQV7KLVLVHVSHFLDOO\WUXHDV3XQFK HPSKDVL]HVLQDUHDVZKHUH
RXUNQRZOHGJHLVVKDOORZIUDJPHQWDU\LQFRPSOHWHRUQRQH[LVWHQW
7KXVFDVHVWXGLHVIROORZLQJSXUSRVLYHVDPSOLQJDQGFRPELQHGZLWKDQDO\WLF
JHQHUDOL]DWLRQFDQRIIHU DQGKDYHSURYHGWREHDEOHWRRIIHU DVYDOLGUHVXOWVDVDQ\
RWKHUUHVHDUFKPHWKRG2QWKHRWKHUKDQG3XQFK DOVRHPSKDVL]HVWKDWWKH
SRWHQWLDOYDOXHRILQGHSWKFDVHVWXGLHVGRHVQRWPHDQWKDWZHVKRXOGQRWEHFULWLFDORI
VRPHVWDQGDORQHVWXGLHVWKDWDUHWRRGHVFULSWLYHRULQVXIILFLHQWO\LQWHJUDWHGLQWRWKH
EURDGHUVWXG\RIWKHVXEMHFWPDWWHURUZKLFKFODLPPRUHIURPWKHLUILQGLQJVWKDQWKH
GDWDFDQEHDU
&DVHVWXGLHVLQDSSOLHGOLQJXLVWLFUHVHDUFK
7ZRUHFHQWUHYLHZVRIFDVHVWXGLHVE\'XII LQSUHVV DQGYDQ/LHU SURYLGH
FRQYLQFLQJHYLGHQFHWKDWWKHFDVHVWXG\DSSURDFKKDVEHHQSURGXFWLYHDQGKLJKO\
LQIOXHQWLDOLQDSSOLHGOLQJXLVWLFV,QIDFWDVERWKVFKRODUVSRLQWRXWWKHIRXQGDWLRQVRI
WKHZKROHGLVFLSOLQHZHUHEDVHGRQWKHUHVXOWVRIWKHILUVWZDYHRIFDVHVWXGLHVRI
ODQJXDJHOHDUQHUVLQWKHVDQGV E\UHVHDUFKHUVVXFKDV+DWFK6FKPLGW
6FKXPDQQDQG:RQJ)LOOPRUH 7KHVHVWXGLHVDUHVWLOOZLGHO\FLWHGLQWKHOLWHUDWXUH
EHFDXVHWKH\KHOSHGWRVKDSHRXUFROOHFWLYHWKLQNLQJLQVXEVWDQWLDOZD\V
&DVHVWXGLHVGLGQRWORVHWKHLUVLJQLILFDQFHDIWHUWKHIRXQGDWLRQVKDGEHHQODLGDQG
DSSOLHGOLQJXLVWLFUHVHDUFKHQWHUHGLQWRDQHZSKDVHRIILQHWXQLQJWKHEURDG
REVHUYDWLRQVPDGHE\WKHILUVWJHQHUDWLRQVFKRODUV$VGHVFULEHGLQ&KDSWHU
TXDOLWDWLYHUHVHDUFKLVXQLTXHO\FDSDEOHRIGRFXPHQWLQJDQGDQDO\VLQJWKHVLWXDWHG
FRQWH[WXDOLQIOXHQFHVRQODQJXDJHDFTXLVLWLRQDQGXVHDVZHOODVWKHVXEWOHYDULDWLRQV
LQOHDUQHUDQGWHDFKHULGHQWLWLHVWKDWHPHUJHGXULQJWKHODQJXDJHOHDUQLQJWHDFKLQJ
SURFHVV&DVHVWXG\UHVHDUFKHUV IRUH[DPSOH'XII+DQ/DP0F.D\1RUWRQ
6DUDQJL7RRKH\DQGFROOHDJXHV KDYHHPEUDFHGDZLGHUDQJHRIUHODWHGWRSLFVIURP
UDFHDQGJHQGHUWRFRPPXQLW\PHPEHUVKLSDQGVRFLDOVWDWXVE\H[SORLWLQJWKH
LQKHUHQWO\ORQJLWXGLQDOQDWXUHRIFDVHVWXG\PHWKRGRORJ\WKH\VXFFHHGHGLQJHQHUDWLQJ
DQHZXQGHUVWDQGLQJRIKRZ6/$LVH[SHULHQFHGDQGSHUFHLYHGE\WKHSDUWLFLSDQWVDQG
ZKDWHIIHFWVWKHLUDJHQF\KDVRQWKHLUVSHFLILFODQJXDJHGHYHORSPHQW9DQ/LHU
SRLQWVRXWWKDWDQDUHDZKLFKLVFXUUHQWO\PXFKLQQHHGRIFDVHVWXG\UHVHDUFKLVWKHUROH
RIWHFKQRORJ\LQ6/$IRUH[DPSOH&$//
7RLOOXVWUDWHWKHYDULHW\RIFDVHVWXG\UHVHDUFKOHWPHOLVWVRPHW\SLFDOSDUWLFLSDQWV
RIWKHVHVWXGLHV 'XIILQSUHVV LQIDQWVFKLOGUHQLQPRQROLQJXDODQGELOLQJXDO
KRPHVVFKRROVPLQRULW\VWXGHQWVDGROHVFHQWDQGDGXOWLPPLJUDQWVVWXG\DEURDG

VWXGHQWVDGXOWVOHDUQLQJDQDGGLWLRQDOODQJXDJHRUORVLQJDQH[LVWLQJODQJXDJH
H[FHSWLRQDOOHDUQHUVHWF7KHGRPDLQVDGGUHVVHGKDYHDOVREHHQIDUUDQJLQJDV
GHPRQVWUDWHGE\WKHIROORZLQJVHOHFWLRQRISURPLQHQWWRSLFVFRYHUHGE\'XII LQ
SUHVV FKLOGODQJXDJHDFTXLVLWLRQELOLQJXDOLVPELOLQJXDOIDPLOLHVELFXOWXUDOLVP
ODQJXDJHORVVGHYHORSPHQWDORUGHULGHQWLW\LQYHVWPHQWJHQGHUIRVVLOL]DWLRQ
SUDJPDWLFGHYHORSPHQWODQJXDJHVRFLDOL]DWLRQYLUWXDOGLVFRXUVHFRPPXQLWLHVWHDFKHU
DJHQF\HWF7KXVLWLVREYLRXVWKDWFDVHVWXG\PHWKRGRORJ\KDVEHHQVXLWDEOHWREH
XWLOL]HGLQUHODWLRQWRGLYHUVHFRQWH[WVDQGWRSLFVWKURXJKRXWWKHHYROXWLRQRIWKHILHOG
RIDSSOLHGOLQJXLVWLFVXSWRWKHPRVWFXUUHQWUHVHDUFKLQWHUHVWV
6WUHQJWKVDQGZHDNQHVVHVRIFDVHVWXGLHV
7KHFDVHVWXG\LVDQH[FHOOHQWPHWKRGIRUREWDLQLQJDWKLFNGHVFULSWLRQRIDFRPSOH[
VRFLDOLVVXHHPEHGGHGZLWKLQDFXOWXUDOFRQWH[W,WRIIHUVULFKDQGLQGHSWKLQVLJKWVWKDW
QRRWKHUPHWKRGFDQ\LHOGDOORZLQJUHVHDUFKHUVWRH[DPLQHKRZDQLQWULFDWHVHWRI
FLUFXPVWDQFHVFRPHWRJHWKHUDQGLQWHUDFWLQVKDSLQJWKHVRFLDOZRUOGDURXQGXV:KHQ
GRQHZHOODV'XII LQSUHVV FRQFOXGHVFDVHVWXGLHVGLVSOD\DKLJKGHJUHHRI
FRPSOHWHQHVVGHSWKRIDQDO\VLVDQGUHDGDELOLW\DQGWKH\DUHHIIHFWLYHLQJHQHUDWLQJ
QHZK\SRWKHVHVPRGHOVDQGXQGHUVWDQGLQJVDERXWWKHWDUJHWSKHQRPHQD7KXVWKH
PHWKRGLVKLJKO\UHFRPPHQGHGIRUH[SORULQJXQFKDUWHGWHUULWRULHVRUPDNLQJVHQVHRID
SDUWLFXODUO\SUREOHPDWLFUHVHDUFKDUHDDQGLWFDQSURYLGHDQXQSDUDOOHOHGXQGHUVWDQGLQJ
RIORQJLWXGLQDOSURFHVVHV$VYDQ/LHU VXPPDUL]HVµ&DVHVWXG\UHVHDUFK
KDVEHFRPHDNH\PHWKRGIRUUHVHDUFKLQJFKDQJHVLQFRPSOH[SKHQRPHQDRYHUWLPH
0DQ\RIWKHSURFHVVHVLQYHVWLJDWHGLQFDVHVWXGLHVFDQQRWEHDGHTXDWHO\UHVHDUFKHGLQ
DQ\RIWKHRWKHUFRPPRQUHVHDUFKPHWKRGV¶
,QDGGLWLRQZKDWLVSDUWLFXODUO\QRWHZRUWK\IURPWKHSHUVSHFWLYHRIWKLVERRNWKH
FDVHVWXG\LVLGHDOO\VXLWHGIRUEHLQJFRPELQHGZLWKRWKHUUHVHDUFKDSSURDFKHV IRU
H[DPSOHDVXEVHTXHQWVXUYH\ LQPL[HGPHWKRGVVWXGLHV,QGHHG'XII LQSUHVV
UHSRUWVWKDWFDVHVWXGLHVKDYHEHHQLQFUHDVLQJO\XVHGLQPL[HGPHWKRGVVWXGLHVVXFKDV
SURJUDPPHHYDOXDWLRQV
:LWKUHJDUGWRLWVZHDNQHVVHV'XII LQSUHVV SRLQWVRXWWKDWFDVHVWXG\
PHWKRGRORJ\LVRIWHQFRQWUDVWHGQHJDWLYHO\ZLWKODUJHVFDOHH[SHULPHQWDOPHWKRGV
ZLWKWKHVWUHQJWKVRIRQHDSSURDFKEHLQJWKHZHDNQHVVHVRIWKHRWKHU7KLVFRQWUDVWLV
LQDSSURSULDWHDQGUDWKHUXQIDLUEHFDXVHWKHWZRW\SHVRIPHWKRGRORJLHVDUHLQWHQGHGWR
DFKLHYHGLIIHUHQWJRDOV<HWKDYLQJDVLQJOHFDVHSUHVHQWVFHUWDLQOLPLWDWLRQVDQGZLWK
WKHFDVHVWXG\EHLQJDSURWRW\SHRITXDOLWDWLYHUHVHDUFKPDQ\RIWKHSRWHQWLDO
VKRUWFRPLQJVRIWKHTXDOLWDWLYHDSSURDFKPHQWLRQHGLQ&KDSWHU 6HFWLRQ FRXOG
EHOLVWHGKHUH0\SHUVRQDOIHHOLQJLVWKDWEHFDXVHRIWKHKHLJKWHQHGYXOQHUDELOLW\RI
WKLVPHWKRGLQWHUPVRILGLRV\QFUDWLFXQSUHGLFWDELOLW\DQGDXGLHQFHFULWLFDOLW\LQPRVW
FDVHVLWPD\EHZRUWKXVLQJ D PXOWLSOHFDVHGHVLJQVRU E FDVHVWXGLHVLQVRPH
FRPELQDWLRQZLWKRWKHUPHWKRGV

'LDU\VWXGLHV
'LDU\ZULWLQJLVDSHUYDVLYHQDUUDWLYHIRUP
0F'RQRXJKDQG0F'RQRXJK
'LDULHVKDYHEHHQXVHGIRUKXQGUHGVRI\HDUVWRUHFRUGWKHHYHQWVRISHRSOH VHYHU\GD\
OLYHVEXWWKHGLDU\VWXG\DVDGDWDFROOHFWLRQPHWKRGKDVEHHQHPSOR\HGE\VRFLDO
UHVHDUFKHUVRQO\VLQFHWKHV'LDU\PHWKRGVZHUHLQLWLDWHGLQWKHILHOGRI
SV\FKRORJ\WRVWXG\HPRWLRQVDQGPRRGVDFURVVVLWXDWLRQVLQGDLO\H[SHULHQFHDQGWKH\
KDYHDOVREHHQXVHGLQIDPLO\SV\FKRORJ\WRREWDLQGDWDDERXWLQWLPDWHDVSHFWVRIWKH
OLIHRIFRXSOHV /DXUHQFHDXDQG%ROJHU ,QJHQHUDOGLDULHVRIIHUWKHRSSRUWXQLW\
WRLQYHVWLJDWHVRFLDOSV\FKRORJLFDODQGSK\VLRORJLFDOSURFHVVHVZLWKLQHYHU\GD\
VLWXDWLRQVDQGDV%ROJHUHWDO VXPPDUL]HDVNLQJUHVHDUFKSDUWLFLSDQWVWRNHHS
UHJXODUUHFRUGVRIFHUWDLQDVSHFWVRIWKHLUGDLO\OLYHVDOORZVWKHUHVHDUFKHUWRFDSWXUHWKH
SDUWLFXODUVRIH[SHULHQFHLQDZD\WKDWLVQRWSRVVLEOHXVLQJRWKHUPHWKRGV
,WLVLPSRUWDQWWRQRWHDWWKLVSRLQWWKDWWKHWHUPµGLDU\VWXGLHV¶XVXDOO\UHIHUVWRGDWD
REWDLQHGIURPµVROLFLWHGGLDULHV¶RQO\WKDWLVIURPDFFRXQWVSURGXFHGVSHFLILFDOO\DW
WKHUHVHDUFKHU¶VUHTXHVWE\DQLQIRUPDQWRULQIRUPDQWV %HOO 2IFRXUVHSHUVRQDO
GLDULHVPD\DOVRFRQWDLQGDWDWKDWLVUHOHYDQWIRUVRPHUHVHDUFKSXUSRVHVEXWXVLQJVXFK
GDWDUDLVHVUHVHDUFKHWKLFDODQGYDOLGLW\TXHVWLRQVDQGLQDQ\FDVHILQGLQJVXFK
VSRQWDQHRXVGLDULHVLVXVXDOO\QRWSUDFWLFDOIRUPRVWUHVHDUFKVWXGLHV6ROLFLWHGGLDULHV
RIIHUWKHSRVVLELOLW\WRFDSWXUHWKHDXWRELRJUDSKLFDODVSHFWRISULYDWHGLDULHVLQDPRUH
V\VWHPDWLFDQGFRQWUROOHGZD\WKHUHE\OHWWLQJµSHRSOHEHKHDUGRQWKHLURZQWHUPV¶ S

,QDSSOLHGOLQJXLVWLFVGLDULHVKDYHEHHQXVHGVLQFHWKHEHJLQQLQJRIWKHVWR
REWDLQSHUVRQDODFFRXQWVRIWKHH[SHULHQFHRIODQJXDJHOHDUQLQJERWKE\OHDUQHUV
WKHPVHOYHVDQGE\SDUHQWVGRFXPHQWLQJWKHLU PDLQO\ELOLQJXDO FKLOGUHQ¶V/
GHYHORSPHQWDQGGLDU\VWXGLHVKDYHDOVREHHQXVHGLQWHDFKHUHGXFDWLRQSURJUDPPHV
)RURYHUYLHZVVHHIRUH[DPSOH%DLOH\'XIILQSUHVV0F'RQRXJKDQG
0F'RQRXJK&KDSWHU
5HFRUGLQJGLDU\HQWULHV
'LDU\VWXGLHVKDYHRIWHQEHHQFODVVLILHGLQWRWKUHHFDWHJRULHVGHSHQGLQJRQZKHQWKH
GLDU\HQWULHVDUHWREHPDGHµLQWHUYDOµµVLJQDO¶DQGµHYHQWFRQWLQJHQW¶GHVLJQV
%ROJHUHWDO 7KHLQWHUYDOFRQWLQJHQWGHVLJQUHTXLUHVSDUWLFLSDQWVWRUHSRUWRQ
WKHLUH[SHULHQFHVDWUHJXODUSUHGHWHUPLQHGLQWHUYDOV IRUH[DPSOHHYHU\DIWHUQRRQDW
SP 6LJQDOFRQWLQJHQWGHVLJQVUHO\RQVRPHVLJQDOOLQJGHYLFHVXFKDVDSDJHUD
SURJUDPPHGZULVWZDWFKRUDSKRQHFDOOWRSURPSWSDUWLFLSDQWVWRSURYLGHGLDU\UHSRUWV
7KLVGHVLJQLVRIWHQXVHGZKHQVWXG\LQJPRPHQWDU\H[SHULHQFHVRISHRSOHVXFKDV
SV\FKRORJLFDOVWDWHV IRUH[DPSOHKDSSLQHVVRUVWUHVV (YHQWFRQWLQJHQWVWXGLHV

UHTXLUHSDUWLFLSDQWVWRSURYLGHDVHOIUHSRUWHDFKWLPHDVSHFLILFHYHQW VXFKDVD
PHHWLQJZLWKDQ/VSHDNHU RFFXUV
7KHPRVWFRPPRQO\XVHGDSSURDFKLQGLDU\VWXGLHVLVJLYLQJWKHUHVSRQGHQWV
WUDGLWLRQDOSDSHUDQGSHQFLOGLDULHV7KHHQWULHVFDQLQYROYHILOOLQJLQDVKRUW
TXHVWLRQQDLUHRUHQWHULQJDVKRUWYHUEDODFFRXQW7KHIRUPHULVLQIDFWDUHSHDWHG
PHDVXUHVTXDQWLWDWLYHTXHVWLRQQDLUHVWXG\DQGZLOOQRWEHGLVFXVVHGKHUH2YHUWKHODVW
GHFDGHSDSHUDQGSHQFLOGLDULHVKDYHEHHQJUDGXDOO\UHSODFHGE\HOHFWURQLFGDWD
FROOHFWLRQPHWKRGVRIYDULRXVIRUPVXWLOL]LQJKDQGKHOGFRPSXWHUVHTXLSSHGZLWK
VSHFLDOVRIWZDUH )RUDGHWDLOHGWHFKQLFDOLQWURGXFWLRQWKDWDOVRLQFOXGHVVRIWZDUH
GHVFULSWLRQVVHH%ROJHUHWDO 7KHVHDOORZIRUVSHFLILFVLJQDOOLQJDQGDOVR
SURYLGHWLPHDQGGDWHVWDPSVIRUWKHUHVSRQVHV
6WUHQJWKVRIGLDU\VWXGLHV
'LDULHVRIIHUDUDQJHRIVSHFLDOIHDWXUHVWKDWDUHGLIILFXOWRULPSRVVLEOHWRUHSOLFDWH
XVLQJRWKHUGDWDFROOHFWLRQPHWKRGV7KHPRVWLPSRUWDQWRQHVDUHDVIROORZV
x 'LDULHVDOORZWKHUHVHDUFKHUDQXQREWUXVLYHZD\RIWDSSLQJLQWRDUHDVRISHRSOH V
OLYHVWKDWPD\RWKHUZLVHEHLQDFFHVVLEOH *LEVRQ :HKDYHVHHQLQ&KDSWHU
WKDWTXDOLWDWLYHUHVHDUFKLQYROYHVWKHVWXG\RISHRSOHLQWKHLUQDWXUDOFRQWH[WVZLWK
DVOLWWOHREWUXVLRQDVSRVVLEOH²GLDU\VWXGLHVFDQVDWLVI\WKLVUHTXLUHPHQWWRDODUJH
H[WHQW
x $IXUWKHUHOHPHQWRIWKHTXDOLWDWLYHLQTXLU\LVWRWU\DQGHOLFLWWKHSDUWLFLSDQWV¶RZQ
GHVFULSWLRQVDQGLQWHUSUHWDWLRQVRIHYHQWVDQGEHKDYLRXUV²LQGLDU\VWXGLHVWKH
SDUWLFLSDQWVLQHYLWDEO\EHFRPHFRUHVHDUFKHUVDVWKH\NHHSUHFRUGVRIWKHLURZQ
IHHOLQJVWKRXJKWVRUDFWLYLWLHV'LDU\GDWDLVE\GHILQLWLRQDQLQVLGHUDFFRXQW
x 'LDU\PHWKRGVHQDEOHWKHUHVHDUFKHUWRVWXG\WLPHUHODWHGHYROXWLRQRUIOXFWXDWLRQ
ZLWKLQLQGLYLGXDOVE\FROOHFWLQJGDWDRQPDQ\RFFDVLRQVIURPWKHVDPHLQGLYLGXDOV
7KXVGLDU\VWXGLHVDUHDSSURSULDWHIRUORRNLQJDWWHPSRUDOYDULDWLRQLQG\QDPLF
SURFHVVHVLQYHVWLJDWLQJIRUH[DPSOHKRZSHRSOHFKDQJHRUUHVSRQGWRFHUWDLQ
VWLPXOL'LDU\VWXGLHVDUHPRUHVHQVLWLYHWRVXFKTXHVWLRQVWKDQPDQ\RWKHU
ORQJLWXGLQDOGHVLJQV VHH6HFWLRQ EHFDXVHWKH\W\SLFDOO\LQYROYHPRUHIUHTXHQW
GDWDUHFRUGLQJWKDWFDQFDSWXUHFKDQJHVZLWKLQFUHDVHGILGHOLW\
x 'LDU\VWXGLHVFDQSURYLGHRQJRLQJEDFNJURXQGLQIRUPDWLRQWKDWFDQKHOSUHVROYH
DPELJXLW\UHJDUGLQJFDXVDOGLUHFWLRQEHWZHHQYDULDEOHV,QDSSOLHGOLQJXLVWLFV
6FKPLGWDQG)URWD UHSRUWHGIRUH[DPSOHWKDWGLDU\HQWULHVKHOSHGWKHPWR
FRQILUPWKHH[LVWHQFHRIFHUWDLQDFTXLVLWLRQDORUGHUV
x 'LDU\VWXGLHVRIIHUDVHOIUHSRUWIRUPDWWKDWUHGXFHVLQDFFXUDFLHVVWHPPLQJIURP
QRWUHPHPEHULQJVRPHWKLQJFRUUHFWO\EHFDXVHZKHQZULWLQJWKHLUHQWULHV
SDUWLFLSDQWVUHFDOOUHFHQWUDWKHUWKDQGLVWDQWHYHQWV YDQ(HUGHHWDO
:HDNQHVVHVRIGLDU\VWXGLHV

,IGLDU\VWXGLHVKDYHDVPXFKSRWHQWLDODVGHVFULEHGLQWKHSUHYLRXVVHFWLRQZK\GRQ¶W
ZHXVHWKHPPRUH"7KHUHDUHDWOHDVWWZRPDLQUHDVRQVIRUWKHGHDUWKRIGLDU\VWXGLHVLQ
DSSOLHGOLQJXLVWLFV2QHLVVLPSO\WKDWWKDWWKLVPHWKRGLVUDWKHUQRYHODQGWKHUHIRUHLWLV
QRW\HWFRYHUHGLQVWDQGDUGPHWKRGRORJ\FRXUVHVDQGWH[WV7KXVPDQ\VFKRODUVDUH
VLPSO\QRWIDPLOLDUZLWKLWVXIILFLHQWO\WRFRQVLGHULWDUHDOLVWLFRSWLRQZKHQSUHSDULQJD
UHVHDUFKGHVLJQ7KHRWKHUUHDVRQLVWKDWWKHPHWKRGKDVVRPHVHULRXVZHDNQHVVHV
ZKLFKPDNHUHVHDUFKHUVWKLQNWZLFHEHIRUHDWWHPSWLQJWRXVHLW/HWXVKDYHDORRNDW
WKHVH
x 7KHILUVWDQGPRVWREYLRXVOLPLWDWLRQRIGLDU\VWXGLHVLVWKDWWKHLQIRUPDQWQHHGVWR
EHQRWRQO\OLWHUDWHEXWDFWXDOO\FRPIRUWDEOHDWZULWLQJGLDU\HQWULHV$V*LEVRQ
SRLQWVRXWRQHZD\RIRYHUFRPLQJWKHSUREOHPRILOOLWHUDF\ZRXOGEHWR
DOORZWKHLQIRUPDQWWRNHHSDQDXGLRRUYLGHRGLDU\
x 7KHVHFRQGVRXUFHRIZHDNQHVVLVWKDWDGLDU\VWXG\LVYHU\GHPDQGLQJRQWKHSDUW
RIWKHLQIRUPDQW)LUVWRIDOOLWUHTXLUHVDGHWDLOHGWUDLQLQJVHVVLRQWRHQVXUHWKDW
SDUWLFLSDQWVIXOO\XQGHUVWDQGWKHSURWRFRO6HFRQGWRSURGXFHUHJXODUKLJKTXDOLW\
GDWDGLDU\VWXGLHVPXVWDFKLHYHDOHYHORISDUWLFLSDQWFRPPLWPHQWDQGGHGLFDWLRQ
UDUHO\UHTXLUHGLQRWKHUW\SHVRIUHVHDUFK %ROJHUHWDO 5RVVLWHU¶V
H[SHULHQFHIRUH[DPSOHDSSHDUVWREHTXLWHW\SLFDOµ6WXGHQWVZHUHDOVRDVNHGWR
FRPSOHWHOHDUQLQJMRXUQDOVDQGPRWLYDWLRQJUDSKVLQDGGLWLRQWRWKHLUDVVLJQHG
KRPHZRUNEXWDVWKHUHZDVQRLQFHQWLYHIRUWKHPWRGRVRPDQ\GLGQRW¶2QH
ZD\RIDGGUHVVLQJWKHSUREOHPLVWRUHGXFHWKHWLPHQHFHVVDU\WRXVHWKHGLDU\
LQVWUXPHQWEXWWKLVLVREYLRXVO\DWWKHH[SHQVHRIWKHULFKQHVVRIWKHUHVXOWLQJ
GDWDVHW
x $IXUWKHUVHHPLQJO\WULYLDOEXWLQDFWXDOSUDFWLFHUDWKHUVHULRXVVKRUWFRPLQJRI
GLDU\VWXGLHVLVWKDWWKH\DUHYXOQHUDEOHWRKRQHVWIRUJHWIXOQHVVZKHUHSDUWLFLSDQWV
IDLOWRUHPHPEHUWKHVFKHGXOHGUHVSRQVHWLPHVRUIDLOWRKDYHWKHGLDULHVDWKDQG
%ROJHUHWDO DOWHUQDWLYHO\WKH\PD\EHWRRWLUHGRUVLPSO\QRWLQWKHPRRG
WRSUHSDUHWKHLUVFKHGXOHGHQWU\7KLVRIFRXUVHFDQGHIHDWWKHPDLQEHQHILWRI
GLDULHVWKHLUDELOLW\WRREWDLQDFFXUDWHUHDOWLPHLQIRUPDWLRQ
x ,WKDVDOVREHHQIRXQGWKDWWKHOHQJWKDQGGHSWKRIGLDU\HQWULHVVKRZFRQVLGHUDEOH
YDULDWLRQ*LEVRQ IRUH[DPSOHUHYLHZVVWXGLHVWKDWUHSRUWDFRQVLVWHQW
GHFOLQHLQWKHUHFRXQWLQJRIHYHQWVZLWKWKHSDVVDJHRIWLPH$OVRDQGIRUREYLRXV
UHDVRQVWKHUHLVDGHFOLQHLQWKHGLDU\HQWULHVGXULQJVWUHVVIXOHYHQWVHYHQWKRXJK
IURPWKHUHVHDUFKHU¶VSHUVSHFWLYHWKHVHHYHQWVPD\EHRISDUWLFXODULQWHUHVW
3UDFWLFDOVXJJHVWLRQVWRIDFLOLWDWHGLDU\VWXGLHV
'HVSLWHDOOWKHSUREOHPVWKH\PLJKWEHDVVRFLDWHGZLWKGLDU\VWXGLHVFRXOGEHXVHGWR
JRRGHIIHFWPRUHRIWHQLQDSSOLHGOLQJXLVWLFUHVHDUFK7KHUHIRUHOHWPHPDNHVRPH
SUDFWLFDOSRLQWVWRIDFLOLWDWHWKLVSUDFWLFH

x 7KHPXOWLSOHEHQHILWVRIGLDU\VWXGLHVZRXOGZDUUDQWLQPDQ\FDVHVDWOHDVWDQ
DWWHPSWWRLPSOHPHQWDGLDU\VWXG\,KDYHVHHQHQFRXUDJLQJH[DPSOHVRI
VXFFHVVIXOGLDU\VWXGLHVLQHGXFDWLRQDOFRQWH[WVZKHUHWKHUHVHDUFKHUVZHUHLQ
UHJXODUWRXFKZLWKWKHLQIRUPDQWV IRUH[DPSOHWKH\RUWKHLUUHVHDUFK
DVVLVWDQWVDVVRFLDWHVZHUHWHDFKLQJWKHUHVSRQGHQWV
x 5HVHDUFKHUVKDYHWULHGGLIIHUHQWZD\VRILQFUHDVLQJWKHLQIRUPDQWV¶PRWLYDWLRQWR
SHUVLVWZLWKWKHVWXG\DQGPDNHKLJKTXDOLW\HQWULHVDWWKHVFKHGXOHGWLPH 6HH
%ROJHUHWDO*LEVRQ )LUVWWKHSURFHGXUHVFDQEHPDGHDVFRQYHQLHQW
DQGXVHUIULHQGO\DVSRVVLEOHZLWKPDNLQJSRUWDEOHSRFNHWVL]HGLDULHVDQG
SUHSULQWWKHGDWHVDQGWLPHVRIH[SHFWHGUHVSRQVHVRQWRWKHGLDU\VKHHWVLQRUGHUWR
NHHSSDUWLFLSDQWVRQWUDFN,QFHUWDLQVWXGLHVWKHGLDU\FDQEHIXUWKHUVWUXFWXUHGE\
DGGLQJIRUH[DPSOHNH\TXHVWLRQV²:KHQ":KHUH":KDW":KR"²RQWKHGLDU\
SDJHV6HFRQGDUHJXODUJHQWOHFKHFNXSDQGLIQHHGHGµQXGJH¶E\DUHVHDUFK
DVVLVWDQWZLWKZKRPWKHLQIRUPDQWKDVGHYHORSHGJRRGUDSSRUWKDVEHHQIRXQGWR
EHEHQHILFLDO7KLUGLQVRPHLQVWDQFHVUHVHDUFKHUVKDYHXVHGUHZDUGVDVDQ
LQFHQWLYHWRERRVWFRPSOLDQFH²WKLVFDQHLWKHUEHPRQH\RUVRPHJLIWWRNHQ
x )LQDOO\HYHQLIWKHUHDUHJDSVLQWKHGLDU\HQWULHVWKHVHFDQEHILOOHGLQE\IROORZ
XSLQWHUYLHZV,QWKLVFDVHWKHGLDU\HQWULHVKDYHDVLPLODUIXQFWLRQWRWKHZULWWHQRU
DXGLWRU\UHFDOOSURPSWVXVHGLQVWLPXODWHGUHFDOOPHWKRGRORJ\ 6HH6HFWLRQ
5HVHDUFKMRXUQDOV
2QHRIWKHEHVWZD\VWREHJLQWRGHILQHZKDWLQWHUHVWV\RXDQG\RXUWKRXJKWV
DERXWZKDW\RXDOUHDG\NQRZLVZLWKDUHVHDUFKMRXUQDO
+DWFKDQG/D]DUDWRQ
5HVHDUFKMRXUQDOVDUHGLDULHVNHSWE\WKHUHVHDUFKHUVWKHPVHOYHVGXULQJWKHFRXUVHRID
UHVHDUFKSURMHFWUDWKHUWKDQE\WKHSDUWLFLSDQWVRIDVWXG\FRQFHUQLQJWKHWRSLFRIWKH
LQYHVWLJDWLRQ ZKLFKZDVGLVFXVVHGLQWKHSUHYLRXVVHFWLRQ WRPDNHWKLVGLVWLQFWLRQ
FOHDU,ZLOOUHIHUWRWKHVHUHVHDUFKHUQRWHVDVµMRXUQDOV¶HYHQWKRXJKLQWKHOLWHUDWXUHWKH
WHUPVµGLDU\¶DQGµMRXUQDO¶DUHRIWHQXVHGV\QRQ\PRXVO\,WLVDFRPPRQ
UHFRPPHQGDWLRQLQVRFLDOUHVHDUFKWKDWWKHUHVHDUFKHUVKRXOGNHHSDMRXUQDODQGZH
ILQGDPHQWLRQRIWKHXVHRIWKHDXWKRU¶VUHVHDUFKMRXUQDOLQPDQ\UHVHDUFKSXEOLFDWLRQV
)RUDUHFHQWRYHUYLHZRIXVLQJUHVHDUFKMRXUQDOVLQHGXFDWLRQDOUHVHDUFKVHH$OWULFKWHU
DQG+ROO\ +RZHYHUDSDUWIURPHWKQRJUDSKLFVWXGLHVZKHUHILHOGQRWHVDUHDQ
LQWHJUDOSDUWRIWKHLQTXLU\TXDOLWDWLYHLQYHVWLJDWLRQVW\SLFDOO\SURYLGHIHZGHWDLOVRQ
WKHQDWXUHRIWKHMRXUQDOGDWDWKDWLVZKDWVRUWRILQIRUPDWLRQZDVUHFRUGHGDQGKRZ
,QIDFWDV+DPPHUVOH\DQG$WNLQVRQ SRLQWRXWHYHQHWKQRJUDSKLFILHOGQRWHV
DUHFKDUDFWHUL]HGE\µUHODWLYHLQYLVLELOLW\¶ S DQGP\SHUVRQDOH[SHULHQFHLVWKDW
QRYLFHUHVHDUFKHUVZKRGRQRWVSHFLDOL]HLQHWKQRJUDSK\DUHRIWHQXQDZDUHRIWKHIDFW
WKDWLIWKH\NHHSWKHLUUHVHDUFKMRXUQDOµRUGHUO\¶E\XVLQJVRPHVWDQGDUGL]HGIRUPDW
WKHQLWFDQEHHQWHUHGDVDYDOXDEOHVRXUFHRIGDWD

,WZDVVDLGLQ&KDSWHUWKDWLQTXDOLWDWLYHUHVHDUFKDOPRVWDQ\WKLQJFDQEH
SHUFHLYHGDVSRWHQWLDOGDWDDQGWKHUHLVQRUHDVRQZK\WKHUHVHDUFKHU¶VILHOGQRWHVUHDO
WLPHFRPPHQWVPHPRVDQGDQQRWDWLRQVZRXOGEHH[FHSWLRQV3HUVRQDODJHQF\LVDQ
LPSRUWDQWSDUWRITXDOLWDWLYHLQTXLULHVDQGWKHµPHWDGDWD¶JHQHUDWHGE\WKHUHVHDUFKHU
RIIHUVYDOXDEOHLQVLJKWVLQWRWKHSURMHFW,Q6HFWLRQZHGLVFXVVHGWKHLPSRUWDQFH
RIWKHµUHVHDUFKORJ¶EXWZKHUHDVLQTXDQWLWDWLYHVWXGLHVVXFKDORJSOD\VODUJHO\DQ
DGPLQLVWUDWLYHUROHWKHORJWUDLORIDTXDOLWDWLYHVWXG\FDQEHFRPHDQLPSRUWDQWSDUWRI
WKHGDWDVHW$V+DPPHUVOH\DQG$WNLQVRQ ± FRPPHQWµ7KHFRQVWUXFWLRQ
RIDQDO\WLFQRWHV«FRQVWLWXWHVSUHFLVHO\WKHVRUWRILQWHUQDOGLDORJXHRUWKLQNLQJDORXG
WKDWLVWKHHVVHQFHRIUHIOH[LYHHWKQRJUDSK\¶$QGWKHYHKLFOHWRWUDQVIRUPSULYDWH
NQRZOHGJHE\UHIOHFWLRQDQGDQDO\VLVLQWRSRWHQWLDOSXEOLFNQRZOHGJHLVWKH
UHVHDUFKHU¶VMRXUQDO6LOYHUPDQ DOVRUHFRPPHQGVNHHSLQJDUHVHDUFK
MRXUQDODVLQWKLVµ\RXFDQVKRZ\RXUUHDGHUVWKHGHYHORSPHQWRI\RXUWKLQNLQJKHOS
\RXURZQUHIOHFWLRQLPSURYH\RXUWLPHPDQDJHPHQWDQGSURYLGHLGHDVIRUWKHIXWXUH
GLUHFWLRQRI\RXUZRUN¶
5LFKDUGV HPSKDVL]HVDIXUWKHUVLJQLILFDQFHRIDQDSSURSULDWHO\PDLQWDLQHG
µORJWUDLO¶6KHDUJXHVWKDWLQRUGHUWRPDNHFODLPVFUHGLEOHTXDOLWDWLYHUHVHDUFKHUVQHHG
WRDFFRXQWIRUHDFKVWHSLQWKHSURMHFWDQGGRFXPHQWZKHUHWKHLGHDVDQGWKHRULHVFDPH
IURP7KXVVXFKDFFRXQWVDUHFHQWUDOLQIHQGLQJRIIFKDOOHQJHVWRYDOLGLW\DQG
UHOLDELOLW\LQTXDOLWDWLYHUHVHDUFK 6HHDOVRWKHGLVFXVVLRQRQµDXGLWWUDLOV¶LQ6HFWLRQ

'HVSLWHWKHVHREYLRXVDGYDQWDJHVIHZUHVHDUFKHUVDFWXDOO\PDLQWDLQDV\VWHPDWLF
UHVHDUFKMRXUQDO2QHUHDVRQIRUWKLVPD\EHVLPSO\QRWUHDOL]LQJWKHVLJQLILFDQFHRIWKH
MRXUQDOEXW'XII LQSUHVV SRLQWVRXWDVHFRQGSRVVLEOHUHDVRQQDPHO\WKDWNHHSLQJD
V\VWHPDWLFDFFRXQWRIRQH¶VUHVHDUFKDFWLYLWLHVDQGUHIOHFWLRQVWDNHVFRQVLGHUDEOH
GLVFLSOLQHµHVSHFLDOO\ZKHQMXJJOLQJPDQ\NLQGVRIWDVNV LQFOXGLQJGDWDFROOHFWLRQDQG
DQDO\VLV DWRQFH¶<HWVKHUHFRPPHQGVKDYLQJDUHVHDUFKMRXUQDOEHFDXVHLWQRWRQO\
KHOSVLQUHPHPEHULQJLPSRUWDQWGHWDLOVODWHURQEXWµMRXUQDONHHSLQJEHFRPHVSDUWRI
WKHDQDO\VLVDQGLQWHUSUHWDWLRQSURFHVVLWVHOIDVUHVHDUFKHUVVWDUWWRPXOORYHUQHZGDWD
DQGWKHPHV¶6KHFLWHV6FKPLGW¶VIDPRXVUHVHDUFKMRXUQDO 6FKPLGWDQG)URWD DV
DJRRGH[DPSOHRIKRZWKHMRXUQDOFDQEHFRPHDµSODWIRUPIRUFRQFHSWXDOL]LQJ
QRWLFLQJDUWLFXODWLQJRUWHVWLQJRXWQHZK\SRWKHVHVRULGHDV¶
+RZWRNHHSUHVHDUFKUHFRUGV
6FLHQWLILFGDWDQHHGVWRPHHWFHUWDLQUHTXLUHPHQWVRIUHOLDELOLW\DQGYDOLGLW\DQG
WKHUHIRUHLIZHGHFLGHWRXVHRXUUHVHDUFKMRXUQDODVDGDWDVRXUFHZHQHHGWRSD\
DWWHQWLRQWRVRPHWHFKQLFDOGHWDLOV,IZHNHHSDSHQFLODQGSDSHUMRXUQDOWKHPLQLPDO
UHTXLUHPHQWLVWRKDYHDSURSHUO\SUHERXQGIROGHUZLWKHDFKSDJHQXPEHUHGDQGHDFK
HQWU\GDWHGEXWVRPHUHVHDUFKHUVUHFRPPHQGDPRUHIRUPDOL]HGDSSURDFK6LOYHUPDQ
IRUH[DPSOHRXWOLQHVDQRUJDQL]DWLRQIUDPHZRUNFRQVLVWLQJRIIRXUFDWHJRULHV
2EVHUYDWLRQQRWHVDERXWH[SHULHQFHV PHWKRGRORJLFDOQRWHVDERXWKRZDQGZKDW

NLQGRIGDWDZDVFROOHFWHG WKHRUHWLFDOQRWHVGHVFULELQJKXQFKHVK\SRWKHVHVDQG
LGHDVDQG SHUVRQDOQRWHVFRQWDLQLQJIHHOLQJVWDWHPHQWV FRQFHUQLQJIRUH[DPSOH
VDWLVIDFWLRQVXUSULVHVKRFNHWF DQGRWKHUVXEMHFWLYHFRPPHQWV&U\HU
UHFRPPHQGVWKDWUHVHDUFKVWXGHQWVVKRXOGUHFRUGLQWKHLUORJERRNVDQGMRXUQDOVWKH
IROORZLQJ
x ZKDW\RXGRDQGZKHUHKRZZKHQDQGZK\\RXGRLW
x ZKDW\RXUHDG
x ZKDWGDWD\RXFROOHFWDQGKRZ\RXSURFHVVLW
x DQ\RXWFRPHVRIWKHGDWDDQDO\VLV
x SDUWLFXODUDFKLHYHPHQWVGHDGHQGVDQGVXUSULVHV
x ZKDW\RXWKLQNRUIHHODERXWZKDWLVKDSSHQLQJDQGDQ\LGHDVWKDWPD\EHUHOHYDQW
IRU\RXUUHVHDUFK
x DQ\WKLQJHOVHWKDWLVLQIOXHQFLQJ\RX
:LWKWKHLQFUHDVLQJDYDLODELOLW\DQGXVHRISHUVRQDODQGHVSHFLDOO\SRUWDEOH
FRPSXWHUVPRUHDQGPRUHVFKRODUVWUDQVIHUVRPHRIWKHWUDGLWLRQDOSDSHUDQGSHQFLO
UHFRUGVWRHOHFWURQLFILOHV4XDOLWDWLYHGDWDDQDO\VLVVRIWZDUHIXOO\VXSSRUWVVXFK
HOHFWURQLFORJJLQJDQGPHPRLQJ VHH5LFKDUGVDQG6HFWLRQLQWKLVYROXPH
DQGWKH\PDNHLWHDV\WRLQWHJUDWHWKHHQWULHVRIWKHHOHFWURQLFUHVHDUFKMRXUQDOZLWKWKH
UHVWRIWKHWUDQVFULEHGGDWD,QWKLVZD\RXURZQUHIOHFWLRQVDQGLGHDVFDQEHFRPHPRUH
WKDQDQDO\WLFWRROV²WKH\FDQEHFRPHGDWDWKDWFDQEHVXEPLWWHGWRIXUWKHUDQDO\VLV
$OWULFKWHUDQG+ROO\ HPSKDVL]HWKDWDQLPSRUWDQWDVSHFWRIWKHUHVHDUFK
MRXUQDODVDGDWDVRXUFHLVWRLQFOXGHµGHVFULSWLYHVHTXHQFHV¶VXFKDVDFFRXQWVRI
DFWLYLWLHVDQGHYHQWVRUUHFRQVWUXFWLRQVRIGLDORJXHV,QVXFKDFFRXQWVWKHDXWKRUV
VWUHVVGHWDLOVDUHPRUHLPSRUWDQWWKDQVXPPDULHVDQGZHQHHGWRIRFXVRQWKH
SDUWLFXODUUDWKHUWKDQWKHJHQHUDO([DFWTXRWHV PDUNHGZLWKTXRWDWLRQPDUNV DUH
ZHOFRPHSDUWLFXODUO\ZRUGVDQGSKUDVHVWKDWDUHW\SLFDORIDSHUVRQ$OWULFKWHUDQG
+ROO\DOVRPDNHVRPHSUDFWLFDOSRLQWVRQWKHEDVLVRIZKLFKWKHIROORZLQJVL[MRXUQDO
ZULWLQJVWUDWHJLHVFDQEHIRUPXODWHG
:ULWHUHJXODUO\
3HUVLVWWKURXJKWKHLQLWLDOGLIILFXOWSHULRGWKDWRFFXUVWRPDQ\QHZZULWHUV
&RQVLGHUWKHMRXUQDOSULYDWHVRWKDW\RXGRQ¶WKDYHWRFHQVRU\RXUVHOIRUZRUU\
DERXWVW\OHDQGSXQFWXDWLRQ \RXFDQGHFLGHDERXWGLVFORVLQJVRPHSDUWVODWHU
,QWURGXFHDUHJXODUVWUXFWXUHDQGIRUPDWLQ\RXUHQWULHV
,QFOXGHUHOHYDQWLOOXVWUDWLYHPDWHULDOVVXFKDVSKRWRVQRWHVGRFXPHQWVOHVVRQ
SODQVVWXGHQWZRUNVHWF
)URPWLPHWRWLPHGRDSURYLVLRQDODQDO\VLVRIWKHMRXUQDOHQWULHVWRFKHFNZKHWKHU
WKHEDODQFHDQGDPRXQWRIWKHGLIIHUHQWW\SHRIQRWHVLVULJKW

0L[HGPHWKRGVUHVHDUFKSXUSRVH
DQGGHVLJQ
«LWLVLPSRUWDQWWRQRWHWKDWZHVHHQRFKDVPEHWZHHQTXDOLWDWLYHDQG
TXDQWLWDWLYHWHFKQLTXHV,WLVRXUH[SHULHQFHWKDWPDQ\TXDOLWDWLYHSURMHFWV
LQYROYHFRXQWLQJDWVRPHVWDJHDQGPDQ\TXHVWLRQVDUHEHVWDQVZHUHGE\
TXDQWLILFDWLRQ
0RUVHDQG5LFKDUGV
$PL[HGPHWKRGVVWXG\LQYROYHVWKHFROOHFWLRQRUDQDO\VLVRIERWKTXDQWLWDWLYHDQG
TXDOLWDWLYHGDWDLQDVLQJOHVWXG\ZLWKVRPHDWWHPSWVWRLQWHJUDWHWKHWZRDSSURDFKHVDW
RQHRUPRUHVWDJHVRIWKHUHVHDUFKSURFHVV,QRWKHUZRUGVPL[HGPHWKRGVUHVHDUFK
LQYROYHVWKHPL[LQJRITXDQWLWDWLYHDQGTXDOLWDWLYHUHVHDUFKPHWKRGVRUSDUDGLJP
FKDUDFWHULVWLFV -RKQVRQDQG&KULVWHQVHQ $VPHQWLRQHGLQ6HFWLRQYDULRXV
ODEHOVKDYHEHHQXVHGWRGHVFULEHWKLVPHWKRGRORJ\²PXOWLWUDLWPXOWLPHWKRGUHVHDUFK
LQWHUUHODWLQJTXDOLWDWLYHDQGTXDQWLWDWLYHGDWDPHWKRGRORJLFDOWULDQJXODWLRQ
PXOWLPHWKRGRORJLFDOUHVHDUFKPL[HGPRGHOVWXGLHVDQGPL[HGPHWKRGVUHVHDUFK²ZLWK
µPL[HGPHWKRGVUHVHDUFK¶EHFRPLQJWKHZLGHO\DFFHSWHGVWDQGDUGWHUP
:HVDZLQ&KDSWHUWKDWDIWHUWKHµSDUDGLJPZDU¶LQWKHVDQGVZKLFK
HPSKDVL]HGWKHLGHRORJLFDOGLIIHUHQFHVEHWZHHQTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKD
SHULRGRIUHFRQFLOLDWLRQIROORZHGDVDUHVXOWRIZKLFK48$/DQG48$1UHVHDUFKHUV
KDYHQRWRQO\VWDUWHGWRDFFHSWHDFKRWKHUEXWKDYHHYHQEHJXQWRLQWHJUDWHWKHWZR
DSSURDFKHV,QIDFWDV6DQGHORZVNL FRQFOXGHVPL[HGPHWKRGVUHVHDUFKKDVE\
QRZEHFRPHVRPHZKDWRIDIDGDZD\WREHPHWKRGRORJLFDOO\IDVKLRQDEOHDQGWR
H[HUFLVHPHWKRGRORJLFDOHFXPHQLFLVP,WKDVLWVHQWKXVLDVWVDQGIRUH[DPSOH
2QZXHJEX]LHDQG/HHFK JRDVIDUDVWRFODLPWKDWµ0RQRPHWKRGUHVHDUFK
LVWKHELJJHVWWKUHDWWRWKHDGYDQFHPHQWRIWKHVRFLDOVFLHQFHV¶6XFKFRPPLWWHG
EHOLHYHUVLQPL[HGPHWKRGRORJ\QRWRQO\YLHZWKHXVHRIPXOWLSOHSHUVSHFWLYHV
WKHRULHVDQGUHVHDUFKPHWKRGVDVDVWUHQJWKLQHGXFDWLRQDOUHVHDUFKEXWWKH\ZRXOG
FODLPWKDWPL[HGPHWKRGVVWXGLHVFDQEHVXSHULRUWRLQYHVWLJDWLRQVSURGXFHGE\HLWKHU
TXDQWLWDWLYHRUTXDOLWDWLYHUHVHDUFKDORQH7KLVLVDGPLWWHGO\DQH[WUHPHYLHZEXWWKHUH
LVQRZDPSOHHYLGHQFHLQWKHOLWHUDWXUHWKDWFRPELQLQJPHWKRGVFDQRSHQXSIUXLWIXO
QHZDYHQXHVIRUUHVHDUFKLQWKHVRFLDOVFLHQFHV
,QRUGHUWRSUHVHQWDFRQYLQFLQJFDVHIRUPL[HGPHWKRGVUHVHDUFKWKUHHEDVLF
TXHVWLRQVQHHGWREHDQVZHUHG :K\GRZHZDQWWRPL[PHWKRGV" ,VQ¶WWKHUHD
GDQJHUWKDWWKHSDUDGLJPDWLFFROOLVLRQFDQFHOVRXWDQ\SRWHQWLDOEHQHILWV" :KDWDUH
WKHEHVWZD\VRIPL[LQJPHWKRGV"(DFKRIWKHVHTXHVWLRQVZLOOEHDGGUHVVHGLQD
VHSDUDWHVHFWLRQDQGWKHFKDSWHULVFRQFOXGHGE\H[DPLQLQJDILQDODGGRQTXHVWLRQ

7KHSXUSRVHRIPL[HGPHWKRGVUHVHDUFK
$FFRUGLQJWR6DQGHORZVNL WKHUHDUHWZRPDLQDQGVRPHZKDWFRQIOLFWLQJ
SXUSRVHVIRUFRPELQLQJPHWKRGV D WRDFKLHYHDIXOOHUXQGHUVWDQGLQJRIDWDUJHW
SKHQRPHQRQDQG E WRYHULI\RQHVHWRIILQGLQJVDJDLQVWWKHRWKHU,QWKHILUVWLQVWDQFH
WKHJRDOLVWRDFKLHYHDQHODERUDWHDQGFRPSUHKHQVLYHXQGHUVWDQGLQJRIDFRPSOH[
PDWWHUORRNLQJDWLWIURPGLIIHUHQWDQJOHV7KHVHFRQGSXUSRVHLVWKHWUDGLWLRQDOJRDORI
WULDQJXODWLRQQDPHO\WRYDOLGDWHRQH¶VFRQFOXVLRQE\SUHVHQWLQJFRQYHUJLQJUHVXOWV
REWDLQHGWKURXJKGLIIHUHQWPHWKRGV )RUDGHWDLOHGGLVFXVVLRQRIWKHGLIIHUHQWIXQFWLRQV
ZLWKDFWXDOH[DPSOHVIURPHGXFDWLRQDOUHVHDUFKVHH+DPPRQG :HFDQDGGD
YHU\SUDFWLFDOWKLUGSXUSRVHWRWKHOLVWRIWKHPDLQUDWLRQDOHVIRUPL[LQJWRUHDFK
DXGLHQFHVWKDWZRXOGQRWEHV\PSDWKHWLFWRRQHRIWKHDSSURDFKHVLIDSSOLHGDORQH/HW
XVKDYHDORRNDWWKHVHWKUHHREMHFWLYHVLQPRUHGHWDLO
([SDQGLQJWKHXQGHUVWDQGLQJRIDFRPSOH[LVVXH
$FFRUGLQJWR0HUWHQV PL[HGPHWKRGVKDYHSDUWLFXODUYDOXHZKHQZHZDQWWR
H[DPLQHDQLVVXHWKDWLVHPEHGGHGLQDFRPSOH[HGXFDWLRQDORUVRFLDOFRQWH[W6KH
DUJXHVWKDWPDQ\UHVHDUFKHUVKDYHXVHGPL[HGPHWKRGVEHFDXVHLWVHHPHGLQWXLWLYHO\
REYLRXVWRWKHPWKDWFRPELQLQJDQGLQFUHDVLQJWKHQXPEHURIUHVHDUFKVWUDWHJLHVXVHG
ZLWKLQDSDUWLFXODUSURMHFWZRXOGEURDGHQWKHVFRSHRIWKHLQYHVWLJDWLRQDQGHQULFKWKH
VFKRODUV¶DELOLW\WRGUDZFRQFOXVLRQVDERXWWKHSUREOHPXQGHUVWXG\,QDQLPSRUWDQW
SDSHULQ*UHHQH&DUDFHOOLDQG*UDKDPOLVWHGIRXUVSHFLILFIXQFWLRQVE\ZKLFK
PL[HGPHWKRGVUHVHDUFKFDQSURGXFHDIXOOHUSLFWXUHRIDQLVVXH
x &RPSOHPHQWDULW\IXQFWLRQ4XDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGVDUHXVHGWR
PHDVXUHRYHUODSSLQJEXWDOVRGLIIHUHQWIDFHWVRIDSKHQRPHQRQ\LHOGLQJDQ
HQULFKHGXQGHUVWDQGLQJE\LOOXVWUDWLQJFODULI\LQJRUHODERUDWLQJRQFHUWDLQDVSHFWV
7KHDVVXPSWLRQWKHUHIRUHLVWKDWVXSSOHPHQWDU\ILQGLQJVFDQSURGXFHDIXOOHU
SRUWUDLWRIWKHVRFLDOZRUOGVLPLODUO\WRSLHFHVRIDMLJVDZSX]]OHZKHQSXWWRJHWKHU
LQWKHFRUUHFWZD\ (U]EHUJHUDQG.HOOH 7KLVLGHDXQGHUOLHVWKHRIWHQ
PHQWLRQHGFRQFHSWLRQDERXWWKH48$/±48$1µGLYLVLRQRIODERXU¶ IRUH[DPSOH
0F&UDFNHQ ZKHUHE\TXDOLWDWLYHUHVHDUFKLVXVHGWRH[SORUHDQHZ
SKHQRPHQRQDQGGHYHORSDQLQLWLDOK\SRWKHVLVZKLFKLVWKHQWHVWHGLQWHUPVRIWKH
EUHDGWKRILWVGLVWULEXWLRQLQWKHSRSXODWLRQE\DTXDQWLWDWLYHPHWKRG IRUH[DPSOH
DVXUYH\
x 'HYHORSPHQWIXQFWLRQ4XDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGVDUHXVHGVHTXHQWLDOO\
VRWKDWWKHUHVXOWVRIWKHILUVWPHWKRGLQIRUPWKHGHYHORSPHQWRIWKHVHFRQG)RU
H[DPSOHDIRFXVJURXSLQWHUYLHZLVXVHGWRGHYHORSLWHPVIRUDTXDQWLWDWLYH
TXHVWLRQQDLUHµ'HYHORSPHQW¶FDQDOVREHXQGHUVWRRGKHUHWRLQIRUPVDPSOLQJ
GHFLVLRQV IRUH[DPSOHDTXHVWLRQQDLUHLVXVHGWRVHOHFWSDUWLFLSDQWVIRUDIROORZXS
LQWHUYLHZVWXG\

x ,QLWLDWLRQIXQFWLRQ5HVXOWVREWDLQHGE\PXOWLSOHPHWKRGVGRQRWDOZD\VSURGXFH
FRUURERUDWLQJRUFRPSOHPHQWDU\UHVXOWVKRZHYHUGLYHUJHQWUHVXOWVFDQDOVREH
LOOXPLQDWLQJ7KHUHIRUHUHVHDUFKHUVPD\LQWHQWLRQDOO\XWLOL]HYDULHGPHWKRGVWR
JHQHUDWHGLVFUHSDQFLHVSDUDGR[HVRUFRQWUDGLFWLRQVZKLFKDUHPHDQWWREH
SURYRFDWLYHWKURXJKWKHUHFDVWLQJRITXHVWLRQVOHDGLQJKRSHIXOO\WRQHZ
SHUVSHFWLYHV
x ([SDQVLRQIXQFWLRQ,WLVDIUHTXHQWGHVLUHRIUHVHDUFKHUVWRH[SDQGWKHVFRSHDQG
EUHDGWKRIDVWXG\E\LQFOXGLQJPXOWLSOHFRPSRQHQWV)RUH[DPSOHTXDOLWDWLYH
PHWKRGVFDQEHXVHGWRH[SORUHWKHSURFHVVHVRIDFHUWDLQLQVWUXFWLRQDOSURJUDPPH
DQGTXDQWLWDWLYHPHWKRGVWRDVVHVVWKHSURJUDPPHRXWFRPHV
&RUURERUDWLQJILQGLQJVWKURXJKµWULDQJXODWLRQ¶
,Q6HFWLRQZHVDZWKDWWKHLQWURGXFWLRQRIµWULDQJXODWLRQ¶LQWKHVRFLDOVFLHQFHV
SOD\HGDNH\UROHLQLQLWLDWLQJWKHFRPELQHGXVHRITXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFK
LQWKHV7UDQVIRUPLQJ&DPSEHOODQG)LVNH¶V RULJLQDOFRQFHSWLQWRD
EURDGHUUHVHDUFKIUDPHZRUN'HQ]LQ DSSOLHGWKHWHUPWRUHIHUWRWKHJHQHUDWLRQ
RIPXOWLSOHSHUVSHFWLYHVRQDSKHQRPHQRQE\XVLQJDYDULHW\RIGDWDVRXUFHV
LQYHVWLJDWRUVWKHRULHVRUUHVHDUFKPHWKRGVZLWKWKHSXUSRVHRIFRUURERUDWLQJDQRYHUDOO
LQWHUSUHWDWLRQ6LQFHWKHQWULDQJXODWLRQKDVEHHQVHHQDVDQHIIHFWLYHVWUDWHJ\WRHQVXUH
UHVHDUFKYDOLGLW\LIDILQGLQJVXUYLYHVDVHULHVRIWHVWVZLWKGLIIHUHQWPHWKRGVLWFDQEH
UHJDUGHGDVPRUHYDOLGWKDQDK\SRWKHVLVWHVWHGRQO\ZLWKWKHKHOSRIDVLQJOHPHWKRG
(U]EHUJHUDQG.HOOH
,WLVLPSRUWDQWWRQRWHDWWKLVSRLQWWKDWDOWKRXJKLWLVLQGHHGUHDVVXULQJZKHQD
FRQFOXVLRQLVFRQILUPHGIURPDGLIIHUHQWYDQWDJHSRLQWWKHUHDOTXHVWLRQLVKRZZH
VKRXOGLQWHUSUHWDQ\GLYHUJHQFHLQWKHWULDQJXODWHGILQGLQJV([SORULQJWKHFRQIOLFWLQJ
UHVXOWVFDQOHDGWRHQKDQFHGXQGHUVWDQGLQJHVSHFLDOO\LILWDOVRLQYROYHVIXUWKHUGDWD
FROOHFWLRQEXWWKLVSURFHVVLVDOUHDG\QRWSDUWRIµWULDQJXODWLRQ¶LQWKHVWULFWVHQVHEXW
EHFRPHVDQH[DPSOHRIWKHµLQLWLDWLRQIXQFWLRQ¶GLVFXVVHGDERYH
:HVKRXOGQRWHIXUWKHUWKDWZKHUHDVLQWKHWHFKQLFDOVHQVHµWULDQJXODWLRQ¶RQO\UHIHUV
WRµYDOLGDWLRQWKURXJKFRQYHUJHQFH¶LWKDVEHHQXVHGYHU\EURDGO\LQUHVHDUFK
PHWKRGRORJ\WH[WVRIWHQDVDV\QRQ\PIRUPL[LQJPHWKRGV7KXVµWULDQJXODWLRQ¶KDV
EHFRPHDSRSXODUXPEUHOODWHUPZLWKGLIIHUHQWSRVVLEOHPHDQLQJVWKDWFDQEHUHODWHGWR
DYDULHW\RISUREOHPVZKLFKPDGH7DVKDNNRULDQG7HGGOLH F DUJXHDJDLQVWWKH
XVHRIWKHWHUPE\PL[HGPHWKRGVUHVHDUFKHUV )RUDUHFHQWRYHUYLHZRIWKH
WULDQJXODWLRQLVVXHVHH0RUDQ(OOLVHWDO
5HDFKLQJPXOWLSOHDXGLHQFHV
,PHQWLRQHGDOUHDG\LQ6HFWLRQWKDWEHFDXVHRIWKHFRPELQDWLRQRIPHWKRGVLQ
PL[HGPHWKRGVUHVHDUFKWKHILQDOUHVXOWVFDQEHPRUHSDODWDEOHIRUFHUWDLQDXGLHQFHV
WKDQWKHRXWFRPHVRIDPRQRPHWKRGVWXG\7RSXWLWEURDGO\DZHOOH[HFXWHGPL[HG
PHWKRGVVWXG\KDVPXOWLSOHVHOOLQJSRLQWV)RUH[DPSOHLID3K'GLVVHUWDWLRQSDQHO

FRQVLVWVSULPDULO\RIVFKRODUVZRUNLQJLQWKHTXDQWLWDWLYHSDUDGLJPWKH\ZLOOEHEHWWHU
DEOHWRUHODWHWRDTXDOLWDWLYHVWXG\LILWLVDFFRPSDQLHGE\VRPHTXDQWLWDWLYH
FRPSRQHQW7KLVLVQRWVLPSO\DFDVHRIYDOLGDWLRQWKURXJKWULDQJXODWLRQDVGHVFULEHGLQ
WKHSUHYLRXVVHFWLRQEXWUDWKHUJHQHUDWLQJDQRYHUDOOOHYHORIWUXVWZRUWKLQHVVIRUWKH
UHVHDUFKHU7RGGHWDO DOVRPDLQWDLQWKDWPDNLQJILQGLQJVPRUHSUHVHQWDEOHWKLV
ZD\DOVRLQFUHDVHVWKHFKDQFHVRIJHWWLQJWKHUHVXOWVRIODUJHO\TXDOLWDWLYHZRUN
SXEOLVKHGLQDPRUHWUDGLWLRQDOO\TXDQWLWDWLYHMRXUQDO 6HHDOVR6HFWLRQRQWKH
µ&KDOOHQJHVDQGDGYDQWDJHVRIPL[HGPHWKRGVUHSRUWV¶
7KHFRPSDWLELOLW\RIGLIIHUHQWUHVHDUFKSDUDGLJPV
«PXFKUHVHDUFKLQWKHµUHDOZRUOG¶GRHVQRWILWLQWRQHDWFDWHJRUL]DWLRQVRI
µTXDOLWDWLYH¶DQGµTXDQWLWDWLYH¶DQGDOVRGRHVQRWDSSHDUWREHWRRFRQFHUQHG
ZLWKWKHHSLVWHPRORJLFDOLVVXHVWKDWVRH[HUFLVHVRPHFRPPHQWDWRUV
+DUGHQDQG7KRPDV
%HIRUHORRNLQJDWWKHPDLQW\SHVRIPL[HGPHWKRGVGHVLJQVOHWXVDGGUHVVWKHNH\LVVXH
XQGHUO\LQJPL[HGPHWKRGVUHVHDUFKFDQGLIIHUHQWIRUPVRIGDWDDQGNQRZOHGJHDERXW
WKHZRUOGEHLQWHJUDWHG"$IWHUDOOZHVDZLQ&KDSWHUWKDWTXDQWLWDWLYHUHVHDUFKHUV
DVSLUHWRREMHFWLYLW\DQGXQLYHUVDOWUXWKVZKLOHTXDOLWDWLYHDGYRFDWHVHPSKDVL]HWKH
µLQWHUSUHWLYHYDOXHODGHQFRQWH[WXDODQGFRQWLQJHQWQDWXUHRIVRFLDONQRZOHGJH¶
*UHHQHHWDO :HUHWKHµSDUDGLJPZDUULRUV¶HQJDJLQJLQWKHSDUDGLJPZDU
RIWKHVDQGVDOORYHUUHDFWLQJ"2ULVWKHUHVRPHWUXWKLQWKHFODLPWKDW
GLIIHULQJZRUOGYLHZVDQGFRQWUDVWLQJH[SODQDWRU\ORJLFVFDQRQO\EHPL[HG
VXSHUILFLDOO\UHVXOWLQJLQDQHFOHFWLFPLVKPDVK"
,EHOLHYHWKDWWKHUHLVQRFRUUHFWDQVZHULIZHDVNWKHTXHVWLRQVWKLVZD\EHFDXVH
HYHU\WKLQJGHSHQGVRQZKDWH[DFWO\ZHDUHDIWHULQRXUUHVHDUFK'LIIHUHQWVFKRODUV
ORRNDWWKHZRUOGWKURXJKGLIIHUHQWOHQVHVUHJDUGGLIIHUHQWWKLQJVDVLPSRUWDQWWRNQRZ
DQGH[SUHVVWKHPVHOYHVEHVWZLWKLQGLIIHUHQWUHVHDUFKSDUDGLJPV<HWWKLVPXOWL
FRORXUHGUHVHDUFKVFHQHLVQRWWREHPLVWDNHQIRUDQµDQ\WKLQJJRHV¶GLVSRVLWLRQWKH
FUXFLDOSRLQWLVWKDWRQH¶VZRUOGYLHZUHVHDUFKPHWKRGRORJ\DQGQDWXUHRI
LQWHUSUHWDWLRQVKRXOGEHFRQVLVWHQW,IZHDFKLHYHWKLVFRQVLVWHQF\ZHFDQSURGXFHKLJK
TXDOLW\UHVHDUFKUHVXOWVERWKLQTXDOLWDWLYHDQGTXDQWLWDWLYHZD\V²DQGPL[HGPHWKRGV
UHVHDUFKKDVEHHQDGYRFDWHGDVDWKLUGSULQFLSOHGDSSURDFKWRSXUVXLQJWKLVREMHFWLYH
7KLVKDVEHHQFOHDUO\H[SUHVVHGE\-RKQVRQDQG2QZXHJEX]LH ± LQD
SRVLWLRQSDSHUHQWLWOHGµ0L[HGPHWKRGVUHVHDUFK$UHVHDUFKSDUDGLJPZKRVHWLPHKDV
FRPH¶
2XUSXUSRVHLQZULWLQJWKLVDUWLFOHLVWRSUHVHQWPL[HGPHWKRGVUHVHDUFKDVWKH
WKLUGUHVHDUFKSDUDGLJPLQHGXFDWLRQDOUHVHDUFK:HKRSHWKHILHOGZLOOPRYH
EH\RQGTXDQWLWDWLYHYHUVXVTXDOLWDWLYHUHVHDUFKDUJXPHQWVEHFDXVHDVUHFRJQL]HG
E\PL[HGPHWKRGVUHVHDUFKERWKTXDQWLWDWLYHDQGTXDOLWDWLYHUHVHDUFKDUH
LPSRUWDQWDQGXVHIXO7KHJRDORIPL[HGPHWKRGVUHVHDUFKLVQRWWRUHSODFHHLWKHU

RIWKHVHDSSURDFKHVEXWUDWKHUWRGUDZIURPWKHVWUHQJWKVDQGPLQLPL]HWKH
ZHDNQHVVHVRIERWKLQVLQJOHUHVHDUFKVWXGLHVDQGDFURVVVWXGLHV,I\RXYLVXDOLVH
DFRQWLQXXPZLWKTXDOLWDWLYHUHVHDUFKDQFKRUHGDWRQHSROHDQGTXDQWLWDWLYH
UHVHDUFKDQFKRUHGDWWKHRWKHUPL[HGPHWKRGVUHVHDUFKFRYHUVWKHODUJHVHWRI
SRLQWVLQWKHPLGGOHDUHD,IRQHSUHIHUVWRWKLQNFDWHJRULFDOO\PL[HGPHWKRGV
UHVHDUFKVLWVLQDQHZWKLUGFKDLUZLWKTXDOLWDWLYHUHVHDUFKVLWWLQJRQWKHOHIWVLGH
DQGTXDQWLWDWLYHUHVHDUFKVLWWLQJRQWKHULJKWVLGH
+RZFDQZHDFKLHYHFRQVLVWHQF\LQDQDSSURDFKWKDWLVEDVHGRQPL[LQJXSYHU\
GLIIHUHQWFRPSRQHQWV"7KHNH\SURFHVVLVµSULQFLSOHGPL[LQJ¶0L[LQJPHWKRGVLVQRW
JRRGSHUVHDQGLQIDFWLWFDQEHDGLVWUDFWLRQWRVHULRXVUHVHDUFK+RZHYHUFHUWDLQ
PL[WXUHVFDQFRPELQHGLIIHUHQWPHWKRGVLQVXFKDZD\WKDWWKHLUVWUHQJWKVDJJUHJDWH
WKHUHE\PDNLQJWKHVXPJUHDWHUWKDQWKHSDUWV7KLVµDGGLWLYHPL[LQJ¶LVDWWKHKHDUWRI
PL[HGPHWKRGVUHVHDUFKZKLFKLVZK\-RKQVRQDQG7XUQHU FRQFOXGHGWKDWWKH
IXQGDPHQWDOSULQFLSOHRIPL[HGPHWKRGVUHVHDUFKLVWKDWUHVHDUFKHUVVKRXOGFROOHFW
PXOWLSOHGDWDXVLQJGLIIHUHQWVWUDWHJLHVDSSURDFKHVDQGPHWKRGVLQVXFKDZD\WKDWWKH
UHVXOWLQJPL[WXUHRUFRPELQDWLRQLVOLNHO\WRUHVXOWLQFRPSOHPHQWDU\VWUHQJWKVDQG
QRQRYHUODSSLQJZHDNQHVVHV
,QDFFRUGDQFHZLWKWKHSUDJPDWLFIRXQGDWLRQRIPL[HGPHWKRGVUHVHDUFKWKHPL[LQJ
SURFHVVLVFHQWUHGDURXQGWKHSXUSRVHRIWKHLQYHVWLJDWLRQWKDWLVWKHUHVHDUFKWRSLFRU
TXHVWLRQ-RKQVRQDQG2QZXHJEX]LH DUJXHWKDWWRGD\¶VUHVHDUFKVFHQHLV
EHFRPLQJLQFUHDVLQJO\LQWHUGLVFLSOLQDU\DQGFRPSOH[DVDUHVXOWRIZKLFKPDQ\
UHVHDUFKHUVIHHOWKHQHHGWRFRPSOHPHQWRQHPHWKRGZLWKDQRWKHU%UDQQHQ DGGV
DIXUWKHUUHVHDUFKGLPHQVLRQWKDWZRUNVLQIDYRXURIWKHWUDQVFHQGHQFHRISDUDGLJPV
WKHJURZLQJGHVLUHRIVRFLDOUHVHDUFKHUVWROLQNPLFURDQGPDFUROHYHODQDO\VHV,QWKH
SDVWVRFLDOVFLHQWLVWVH[SORULQJWKHUHODWLRQVKLSEHWZHHQWKHLQGLYLGXDODQGWKH
VXUURXQGLQJVRFLDOZRUOGKDYHW\SLFDOO\DGRSWHGRQHRIWZRSHUVSHFWLYHVDQ
LQGLYLGXDOLVWLFRUDVRFLHWDO
x ,QWKHµLQGLYLGXDOLVWLFSHUVSHFWLYH¶WKHFRPSOH[LW\RIWKHVRFLDOHQYLURQPHQWLV
RQO\LPSRUWDQWLQDVPXFKDVLWLVUHIOHFWHGLQWKHLQGLYLGXDO¶VPHQWDOSURFHVVHVDQG
WKHUHVXOWLQJDWWLWXGHVEHOLHIVDQGYDOXHVWKDWLVWKLVSHUVSHFWLYHYLHZVWKHVRFLDO
ZRUOGWKURXJKWKHLQGLYLGXDO¶VH\HV IRUH[DPSOHLQVRFLDOFRJQLWLRQWKHRU\
x 7KHµVRFLHWDOSHUVSHFWLYH¶IRFXVHVRQEURDGVRFLDOSURFHVVHVDQGPDFURFRQWH[WXDO
IDFWRUVVXFKDVVRFLRFXOWXUDOQRUPVLQWHUJURXSUHODWLRQVDFFXOWXUDWLRQDVVLPLODWLRQ
SURFHVVHVDQGLQWHUHWKQLFFRQIOLFWVIURPWKLVSHUVSHFWLYHWKHLQGLYLGXDO¶V
EHKDYLRXULVVHHQWREHGHWHUPLQHGE\WKHPRUHSRZHUIXOIRUFHVDWODUJH IRU
H[DPSOHLQVRFLDOLGHQWLW\WKHRU\
7KHWHQVLRQEHWZHHQWKHWZRSHUVSHFWLYHVKDVEHHQRQHRIWKHPRVWEDVLFGLOHPPDV
LQVRFLDOUHVHDUFKDQGRYHUWKHSDVWGHFDGHVWKHUHKDYHEHHQFDOOVIRUGHYHORSLQJ
LQWHJUDWLYHWKHRUHWLFDOIUDPHZRUNVWKDWFDQFUHDWHOLQNVEHWZHHQWKHWZRSDUDGLJPVWR
XQGHUVWDQGLQGLYLGXDOVLQVRFLHW\ )RUDQRYHUYLHZVHH$EUDPVDQG+RJJ

0L[HGPHWKRGVUHVHDUFKVHHPVLGHDOO\VXLWHGWRRSHUDWLRQDOL]HWKHWKHRUHWLFDO
WUDQVFHQGHQFHRIWKHPLFURDQGPDFURSHUVSHFWLYHVDWWKHUHVHDUFKOHYHO
7RFRQFOXGHOHWXVUHWXUQWRWKHVWDUWLQJTXHVWLRQRIWKLVVHFWLRQFDQGLIIHUHQW
UHVHDUFKSDUDGLJPVEHLQWHJUDWHG"7KHJHQHUDODQVZHUHPHUJLQJLQWKHILHOGLVµ\HV¶DQG
RIWHQWRYHU\JRRGHIIHFW*UHHQHDQG&DUDFHOOL HYHQFODLPWKDWLQDFWXDO
UHVHDUFKSDUDGLJPVGRQRWPDWWHUWKDWPXFKDQGWKHUHIRUHWKHSKLORVRSKLFDO
GLVFXVVLRQVDERXWSDUDGLJPFRPSDWLELOLW\KDYHOLWWOHUHOHYDQFHWRHPSLULFDOUHVHDUFK
µDSSOLHGVRFLDOLQTXLUHUVDSSHDUWRJURXQGLQTXLU\GHFLVLRQVSULPDULO\LQWKHQDWXUHRI
WKHSKHQRPHQDEHLQJLQYHVWLJDWHGDQGWKHFRQWH[WVLQZKLFKWKHVWXGLHVDUHFRQGXFWHG
,QTXLU\GHFLVLRQVDUHUDUHO\LIHYHUFRQVFLRXVO\URRWHGLQSKLORVRSKLFDODVVXPSWLRQVRU
EHOLHIV¶ S )URPWKLVSHUVSHFWLYHPL[HGPHWKRGVUHVHDUFKRIIHUVUHVHDUFKHUVWKH
DGYDQWDJHRIEHLQJDEOHWRFKRRVHIURPWKHIXOOUHSHUWRLUHRIPHWKRGRORJLFDORSWLRQV
SURGXFLQJDVDUHVXOWPDQ\GLIIHUHQWNLQGVRIFUHDWLYHPL[HV:HDUHJRLQJWRVXUYH\
WKHVHQH[W
0DLQW\SHVRIPL[HGPHWKRGVGHVLJQV
6HYHUDOFRPSOH[WD[RQRPLHVKDYHEHHQSURSRVHGWRFRYHUDOOWKHSRVVLELOLWLHVZKHUHE\
GLIIHUHQWFRPSRQHQWVRITXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKFDQEHLQWHJUDWHGLQWRD
VLQJOHVWXG\ )RUUHYLHZVVHH&UHVZHOO&UHVZHOOHWDO-RKQVRQDQG
2QZXHJEX]LH0D[ZHOODQG/RRPLV :KLOHVXFKW\SRORJLHVFDQEH
EHQHILFLDOLQRUJDQL]LQJDQGODEHOOLQJYDULHGSUDFWLFHVDQGWKXVFRQYH\LQJDVHQVHRI
ULJRXU &UHVZHOOHWDO LWLVQRWFOHDUKRZIDULWLVZRUWKSXUVXLQJVXFKD
IRUPDOL]HGW\SRORJLFDODSSURDFK7KHSUREOHPLVWZRIROG2QWKHRQHKDQGWKHDFWXDO
RUSRWHQWLDOGLYHUVLW\LQPL[HGPHWKRGVVWXGLHVLVIDUJUHDWHUWKDQDQ\W\SRORJ\FDQ
DGHTXDWHO\HQFRPSDVVDQGWKHUHIRUHQRWHYHQWKHPRVWGHWDLOHGW\SRORJLHVDUH
H[KDXVWLYH 0D[ZHOODQG/RRPLV7HGGOLHDQG7DVKDNNRUL 2QWKHRWKHU
KDQGWKHW\SRORJLFDODSSURDFKVHHPVWRKDYHGHYHORSHGDOLIHRILWVRZQGHWDFKHG
IURPWKHDFWXDOUHVHDUFKSUDFWLFHDOWKRXJKZHFDQWKLQNRIDUDQJHRIVRSKLVWLFDWHG
PL[HGPRGHOV%U\PDQ¶V ODUJHVFDOHUHYLHZRIWKHUHVHDUFKOLWHUDWXUH
GHPRQVWUDWHVWKDWLQDFWXDOUHVHDUFKIDUIHZHUFRPELQDWLRQVKDYHEHHQXVHGZLWKWKH
FRPELQDWLRQRIWZRPHWKRGVLQSDUWLFXODU²TXHVWLRQQDLUHVDQGVHPLVWUXFWXUHG
LQWHUYLHZV²GRPLQDWLQJ
,QWKHOLJKWRIWKHDERYHWKHEXONRIWKHIROORZLQJPDWHULDOGHVFULELQJYDULRXV
UHVHDUFKGHVLJQVZLOOEHRUJDQL]HGDURXQGDQµH[HPSODUEDVHG¶W\SRORJ\WKDW
HPSKDVL]HVWKHPRVWIUHTXHQWPHWKRGFRPELQDWLRQV$OWKRXJKWKLVFKDSWHUIRFXVHVRQ
GDWDFROOHFWLRQDQG&KDSWHURQGDWDDQDO\VLVWKHPDWHULDOLQWKHWZRFKDSWHUV
RYHUODSVVRPHZKDWEHFDXVHPL[HGGHVLJQVRIWHQGLVSOD\FRPELQDWLRQVDWYDULRXVOHYHOV
DWWKHVDPHWLPH
7\SRORJLFDORUJDQL]DWLRQ

$FFRUGLQJWR7DVKDNNRULDQG7HGGOLH¶V F VXPPDU\WKHWHUPLQRORJ\
FXUUHQWO\GHVFULELQJPL[HGPHWKRGVGHVLJQVLVµFKDRWLF¶,QRUGHUWRFUHDWHVRPHRUGHU
DQXPEHURIGLIIHUHQWW\SRORJLHVKDYHEHHQSURSRVHGFHQWUHGDURXQGYDULRXV
RUJDQL]LQJSULQFLSOHV7KHWZRPRVWZLGHO\DFFHSWHGW\SRORJLFDOSULQFLSOHVKDYHEHHQ
WKHVHTXHQFHDQGWKHGRPLQDQFHRIWKHPHWKRGFRQVWLWXHQWVDQGVFKRODUVKDYHDOVR
GHYHORSHGDVLPSOHV\PEROV\VWHPWRSURGXFHDVWUDLJKWIRUZDUGYLVXDOUHSUHVHQWDWLRQ
IRUH[DPSOH-RKQVRQDQG&KULVWHQVHQ
x µ48$/¶RUµTXDO¶VWDQGIRUTXDOLWDWLYHUHVHDUFK
x µ48$1¶RUµTXDQ¶VWDQGIRUTXDQWLWDWLYHUHVHDUFK
x &DSLWDOOHWWHUVGHQRWHSULRULW\RULQFUHDVHGZHLJKW
x /RZHUFDVHOHWWHUVGHQRWHORZHUSULRULW\RUZHLJKW
x $SOXVVLJQ UHSUHVHQWVDFRQFXUUHQWFROOHFWLRQRIGDWD
x $QDUURZ o UHSUHVHQWVDVHTXHQWLDOFROOHFWLRQRIGDWD
7KXVIRUH[DPSOHµTXDOo48$1¶LQGLFDWHVDVWXG\WKDWFRQVLVWVRIWZRSKDVHV
ZLWKWKHVHFRQGTXDQWLWDWLYHSKDVHGRPLQDWLQJWKLVV\PEROFRPELQDWLRQZRXOG
GHVFULEHIRUH[DPSOHDTXHVWLRQQDLUHVXUYH\ZKHUHLQLWLDOIRFXVJURXSLQWHUYLHZVZHUH
XVHGWRIDFLOLWDWHWKHGHYHORSPHQWRIWKHLQVWUXPHQW
,IDVWXG\KDVRQO\WZRFRPSRQHQWVDTXDOLWDWLYHDQGDTXDQWLWDWLYHERWKWKH
VHTXHQFHDQGWKHGRPLQDQFHGLPHQVLRQVKDYHWKUHHFDWHJRULHV TXDOLWDWLYHILUVW
TXDQWLWDWLYHILUVWRUFRQFXUUHQWDQGTXDOLWDWLYHGRPLQDQWTXDQWLWDWLYHGRPLQDQWRU
HTXDOVWDWXV UHVXOWLQJLQQLQHSRVVLEOHFRPELQDWLRQV
48$/48$1
48$/TXDQ
48$1TXDO
48$/o48$1
48$1o48$/
48$/oTXDQ
TXDOo48$1
48$1oTXDO
TXDQo48$/
7KLVOLVWEHFRPHVPRUHH[WHQVLYHLIZHWDNHPRUHWKDQWZRSRVVLEOHUHVHDUFKSKDVHV
LQWRFRQVLGHUDWLRQDQGWKHSRVVLEOHGHVLJQW\SHVSUROLIHUDWHDOPRVWXQVWRSSDEO\LIZH
DGGIXUWKHURUJDQL]LQJFDWHJRULHVIROORZLQJWKHVXJJHVWLRQVRIVHYHUDOVFKRODUV3XQFK
IRUH[DPSOHDOVRUHFRPPHQGHGFRQVLGHULQJZKHWKHURUQRWWKHPHWKRGV
LQIOXHQFHWKHRSHUDWLRQDOL]DWLRQRIHDFKRWKHU&UHVZHOO &UHVZHOOHWDO
GLYLGHGWKHIXQFWLRQRILQWHJUDWLRQLQWRWKUHHFODVVHVWULDQJXODWLRQH[SODQDWLRQRU
H[SORUDWLRQ7DVKDNNRULDQG7HGGOLH KLJKOLJKWHGWKHLPSRUWDQFHRIGHFLGLQJDW
ZKDWVWDJH V LQWKHUHVHDUFKSURFHVVWKHLQWHJUDWLRQRFFXUVUHVHDUFKREMHFWLYHV
IRUPXODWLRQGDWDFROOHFWLRQGDWDDQDO\VLVRUGDWDLQWHUSUHWDWLRQ7KHVHDUHDOO
LPSRUWDQWFODVVLILFDWLRQGLPHQVLRQVEXWLIZHLQFOXGHDOOWKHLUSHUPXWDWLRQVLQD

W\SRORJ\LWZLOOEHFRPHIDUWRRODUJHWRVHUYHDQ\SUDFWLFDOSXUSRVH$WWKHVDPHWLPH
DV-RKQVRQDQG2QZXHJEX]LH FRQFOXGHWKLVYDULHW\LOOXVWUDWHVWKHIDFWWKDW
PL[HGPHWKRGVUHVHDUFKWUXO\RSHQVXSDQH[FLWLQJDQGDOPRVWXQOLPLWHGSRWHQWLDOIRU
IXWXUHUHVHDUFK
([HPSODUEDVHGW\SRORJ\
$VVHHQDERYHWKHW\SRORJLFDODSSURDFKWHQGVWREHUDWKHUDEVWUDFWDQGDOWKRXJKLWLV
SRVVLEOHWRWKLQNRIFRQFUHWHH[DPSOHVIRUPRVWRIWKHGLIIHUHQWFDWHJRULHVVXFK
WKHRUHWLFDOO\FRQFHLYHGW\SRORJLHVDUHPRUHXVHIXOIRUWKHSXUSRVHRISRVWKRF
FODVVLILFDWLRQRIVWXGLHVWKDQIRUGHVLJQLQJQHZUHVHDUFKSURMHFWV7KHUHIRUHLQWKH
IROORZLQJ,ZLOOIROORZDPRUHSUDJPDWLFDSSURDFKLQSUHVHQWLQJYDULRXVPL[HGGHVLJQV
WKDWDUHRUJDQL]HGDURXQGWKHDFWXDOGDWDFROOHFWLRQPHWKRGVDSSOLHGDQGWKHPDLQ
IXQFWLRQVRIWKHPL[LQJ/LVWLQJGHVLJQW\SHVLQWKLVZD\KDVDSRVLWLYHDQGDQHJDWLYH
VLGHRQWKHSRVLWLYHVLGHLWXVHVGHVFULSWLYHODEHOVWRIDFLOLWDWHXQGHUVWDQGLQJ IRU
H[DPSOHµ4XHVWLRQQDLUHVXUYH\IDFLOLWDWHGE\SUHFHGLQJLQWHUYLHZ¶LQVWHDGRIµTXDOo
48$1¶ 2QWKHQHJDWLYHVLGHWKHOLVWLVVHOHFWLYHUDWKHUWKDQFRPSUHKHQVLYHWKDWLVLW
RQO\LQFOXGHVWKHPRVWSURPLQHQWEDVLFFRPELQDWLRQV,EHOLHYHWKDWVXFKDVHOHFWLYH
DSSURDFKLVXVHIXOWRIDPLOLDUL]HUHVHDUFKHUVZLWKWKHFRQFHSWRIPL[LQJVRWKDWWKH\FDQ
WKHQSURGXFHWKHLURZQPDGHWRPHDVXUHYDULDWLRQV
4XHVWLRQQDLUHVXUYH\ZLWKIROORZXSLQWHUYLHZRUUHWURVSHFWLRQ 48$1
oTXDO
:HVDZLQ&KDSWHUWKDWDOWKRXJKWKHTXHVWLRQQDLUHVXUYH\LVDYHUVDWLOHWHFKQLTXH
WKDWDOORZVXVWRFROOHFWDODUJHDPRXQWRIGDWDLQDUHODWLYHO\VKRUWWLPHLWDOVRVXIIHUV
IURPDQLQKHUHQWZHDNQHVVWKHUHVSRQGHQWV¶HQJDJHPHQWWHQGVWREHUDWKHUVKDOORZDQG
WKHUHIRUHZHFDQQRWH[SORUHFRPSOH[PHDQLQJGLUHFWO\ZLWKWKLVWHFKQLTXH7KHXVHRI
VRSKLVWLFDWHGVWDWLVWLFDOSURFHGXUHVDOORZVXVWRH[DPLQHWKHLQWHUUHODWLRQVKLSRIWKH
YDULDEOHVPHDVXUHGEXWLIZHILQGVRPHXQH[SHFWHGUHVXOWV DQGWKHUHDUHDOZD\VVRPH
XQH[SHFWHGUHVXOWV ZHFDQQRWXVXDOO\LQWHUSUHWWKRVHRQWKHEDVLVRIWKHTXHVWLRQQDLUH
GDWD$QGHYHQLIDQREVHUYHGUHODWLRQVKLSPDNHVVHQVHWKHTXHVWLRQQDLUHGDWDXVXDOO\
UHYHDOVOLWWOHDERXWWKHH[DFWQDWXUHRIWKHUHODWLRQVKLS$GGLQJDVXEVHTXHQWTXDOLWDWLYH
FRPSRQHQWWRWKHVWXG\FDQUHPHG\WKLVZHDNQHVV,QDIROORZXSLQWHUYLHZ HLWKHULQ
DQLQGLYLGXDORUJURXSIRUPDW ZHFDQDVNWKHUHVSRQGHQWVWRH[SODLQRULOOXVWUDWHWKH
REWDLQHGSDWWHUQVWKHUHE\DGGLQJIOHVKWRWKHERQHV,QDVWXG\LQYHVWLJDWLQJWKHHIIHFW
RIFODVVVL]HGLIIHUHQFHVRQFODVVURRPSURFHVVHV%ODWFKIRUG H[SUHVVHGWKLV
FRPSOHPHQWDU\IXQFWLRQYHU\FOHDUO\
7KHUHPD\EHUHVHDUFKTXHVWLRQVEHVWDGGUHVVHGZLWKMXVWRQHWHFKQLTXHEXW
WKHUHDUHWLPHVDQGZHZRXOGFLWHRXUVWXG\RIUHODWLRQVEHWZHHQFODVVVL]HDQG
WHDFKLQJDVRQHVXFKZKHQUHVXOWVIURPWKHLQWHJUDWLRQRIGLIIHUHQWWHFKQLTXHV
DGGXSWRPRUHWKDQWKHVXPRIWKHSDUWV7KLVLVVHHQIRUH[DPSOHLQWKHZD\
WKDWQXPHULFDOHYLGHQFHRQWKHZD\VPDOOFODVVHVOHDGVWRPRUHLQGLYLGXDO
DWWHQWLRQZDVVWUHQJWKHQHGDQGPDGHPRUHFRPSOHWHE\GHWDLOHGLQIRUPDWLRQRQ

KRZWKLVZDVH[SUHVVHGLQSDUWLFXODUFODVVURRPV FDVHVWXGLHV DQGSHUFHLYHGE\
LQGLYLGXDOWHDFKHUV TXHVWLRQQDLUHV
&UHVZHOOHWDO ODEHOOHGWKLVFRPELQDWLRQDµVHTXHQWLDOH[SODQDWRU\GHVLJQ¶

ZKLFKH[SUHVVHVLWVUDWLRQDOHZHOO7KLVLVDVWUDLJKWIRUZDUGGHVLJQWKDWLVHDV\WR
LPSOHPHQWDQGDQDO\VH\HWZKLFKHQULFKHVWKHILQDOILQGLQJVFRQVLGHUDEO\,WFDQ
EHQHILWDOPRVWHYHU\TXDQWLWDWLYHVWXG\
$QLPSRUWDQWYDULDWLRQRIWKLVGHVLJQWKDWKLJKOLJKWVLWVH[SODQDWRU\QDWXUHLQYROYHV
FRQGXFWLQJDµUHWURVSHFWLYHLQWHUYLHZ¶ VHH6HFWLRQ ZLWKVRPHRIWKHVXUYH\
SDUWLFLSDQWVXVLQJWKHUHVSRQGHQWV¶RZQVXUYH\UHVSRQVHVDVWKHUHWURVSHFWLYHSURPSWV
IRUIXUWKHURSHQHQGHGUHIOHFWLRQDERXWZKDWWKH\UHDOO\PHDQW7KLVGHVLJQSDWWHUQFDQ
DOVREHXVHGIRUYDOLGDWLQJWHVWUHVXOWVZLWKDQHZO\GHYHORSHGWHVW
4XHVWLRQQDLUHVXUYH\IDFLOLWDWHGE\SUHFHGLQJLQWHUYLHZ TXDOo48$1
$IUHTXHQWO\UHFRPPHQGHGSURFHGXUHIRUGHVLJQLQJDQHZTXHVWLRQQDLUHLQYROYHV
FRQGXFWLQJDVPDOOVFDOHH[SORUDWRU\TXDOLWDWLYHVWXG\ILUVW XVXDOO\IRFXVJURXS
LQWHUYLHZVEXWRQHWRRQHLQWHUYLHZVFDQDOVRVHUYHWKHSXUSRVH WRSURYLGHEDFNJURXQG
LQIRUPDWLRQRQWKHFRQWH[WWRLGHQWLI\RUQDUURZGRZQWKHIRFXVRIWKHSRVVLEOH
YDULDEOHVDQGWRDFWDVDYDOXDEOHVRXUFHRILGHDVIRUSUHSDULQJWKHLWHPSRROIRUWKH
SXUSRVHRITXHVWLRQQDLUHVFDOHFRQVWUXFWLRQ6XFKDGHVLJQLVHIIHFWLYHLQLPSURYLQJWKH
FRQWHQWUHSUHVHQWDWLRQRIWKHVXUYH\DQGWKXVWKHLQWHUQDOYDOLGLW\RIWKHVWXG\,WLV
URXWLQHO\XVHGZKHQDUHVHDUFKHULVEXLOGLQJDQHZLQVWUXPHQW
,QWHUYLHZVWXG\ZLWKIROORZXSTXHVWLRQQDLUHVXUYH\ 48$/oTXDQ
$EURDGJHQHUDOL]DWLRQLQUHVHDUFKPHWKRGRORJ\WKDWLVE\DQGODUJHWUXHLVWKDWRQHRI
WKHVWUHQJWKVRITXDOLWDWLYHUHVHDUFKLVLWVH[SORUDWRU\QDWXUHDOORZLQJXVWRJDLQQHZ
LQVLJKWVDQGIRUPXODWHQHZWKHRULHV+RZHYHUEHFDXVHRIWKHQRQUHSUHVHQWDWLYHQHVVRI
WKHW\SLFDOVDPSOHVTXDOLWDWLYHGDWDFDQQRWLQIRUPXVDERXWKRZZLGHO\ZKDWLV
GLVFRYHUHGH[LVWVLQWKHUHVWRIWKHZRUOG²H[DPLQLQJWKHGLVWULEXWLRQRIDSKHQRPHQRQ
LQDSRSXODWLRQLVDW\SLFDOTXDQWLWDWLYHREMHFWLYH&RPELQLQJDTXDOLWDWLYHLQWHUYLHZ
VWXG\ZLWKDIROORZXSVXUYH\FDQRIIHUWKHEHVWRIERWKZRUOGVDVWKHTXHVWLRQQDLUH
FDQVSHFLILFDOO\WDUJHWWKHLVVXHVXQFRYHUHGLQWKHILUVWSKDVHRIWKHUHVHDUFKDQG
LQYHVWLJDWHWKHJHQHUDOL]DELOLW\RIWKHQHZK\SRWKHVHVLQZLGHUSRSXODWLRQV
$OWHUQDWLYHO\RULQDGGLWLRQWRWKLVWKHTXHVWLRQQDLUHFDQDOVREHXVHGWRWHVWFHUWDLQ
HOHPHQWVRIWKHWKHRU\HPHUJLQJIURPWKHTXDOLWDWLYHSKDVH
,QWHUYLHZVWXG\IDFLOLWDWHGE\SUHFHGLQJTXHVWLRQQDLUHVXUYH\ TXDQo
48$/
,WZDVSRLQWHGRXWLQ&KDSWHU 6HFWLRQ WKDWDQDUHDZKHUHTXDOLWDWLYHUHVHDUFK
VKRZVYXOQHUDELOLW\LVWKHXVXDOO\VPDOOVDPSOHVL]HVRIWKHUHVSRQGHQWVH[DPLQHG2QH
ZD\RIGHDOLQJZLWKWKLVLVVXHLVWRDSSO\SXUSRVLYHVDPSOLQJ VHH6HFWLRQ DQGWKLV
SURFHGXUHFDQEHPDGHPRUHSULQFLSOHGLIZHLQFOXGHDQLQLWLDOTXHVWLRQQDLUHLQWKH
VWXG\ZKRVHUROHLVWRKHOSWRVHOHFWWKHSDUWLFLSDQWVIRUWKHVXEVHTXHQWTXDOLWDWLYH

SKDVHV\VWHPDWLFDOO\7KXVDV'XII LQSUHVV SRLQWVRXWWKHGHVLJQFDQKHOSWR
HVWDEOLVKWKHUHSUHVHQWDWLYHQHVVRIWKHFDVHVSUHVHQWHG7KHVWUHQJWKRIWKLVGHVLJQLVLWV
IOH[LELOLW\EHFDXVHLWFDQEHXVHGIRUPRVWWKHRUHWLFDOVDPSOLQJSXUSRVHV IRUH[DPSOH
WRFKRRVHH[WUHPHRUW\SLFDOFDVHVRUWRKLJKOLJKWLQGLYLGXDOVZLWKFHUWDLQWUDLWV
+RZHYHUWKHGHVLJQDOVRKDVRQHSRVVLEOHGUDZEDFNLWGRHVQRWZRUNLIWKHLQLWLDO
TXHVWLRQQDLUHLVDQRQ\PRXVEHFDXVHWKHQZHFDQQRWLGHQWLI\WKHDSSURSULDWHVXUYH\
SDUWLFLSDQWV
&RQFXUUHQWFRPELQDWLRQVRITXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFK
48$/TXDO48$1TXDQ
6RIDUZHKDYHORRNHGDWVHTXHQWLDOGHVLJQVLQZKLFKDGRPLQDQWSKDVHZDV
DFFRPSDQLHGE\DVHFRQGDU\FRPSRQHQWWRHQKDQFHFHUWDLQVWUHQJWKVRUUHGXFHFHUWDLQ
ZHDNQHVVHVRIWKHPDLQPHWKRG$QDOWHUQDWLYHWRVHTXHQWLDOGHVLJQVLVWKHYDULHW\RI
µFRQFXUUHQWGHVLJQV¶LQZKLFKZHXVHWZRPHWKRGVLQDVHSDUDWHDQGSDUDOOHOPDQQHU
LHWKH\GRQRWLQIOXHQFHWKHRSHUDWLRQDOL]DWLRQRIHDFKRWKHU DQGWKHUHVXOWVDUH
LQWHJUDWHGLQWKHLQWHUSUHWDWLRQSKDVH7KHPDLQSXUSRVHRIWKLVGHVLJQLVWREURDGHQWKH
UHVHDUFKSHUVSHFWLYHDQGWKXVSURYLGHDJHQHUDOSLFWXUHRUWRWHVWKRZWKHGLIIHUHQW
ILQGLQJVFRPSOHPHQWRUFRUURERUDWHHDFKRWKHU'HSHQGLQJRQKRZPXFKZHLJKWLV
DVVLJQHGWRWKHFRQWULEXWLRQRIWKHYDULRXVPHWKRGVDVWXG\FDQEH D 48$/TXDQ
ZKLFKLVIUHTXHQWLQFDVHVWXGLHVZKHUHWKHSULPDULO\TXDOLWDWLYHGDWDFDQEH
VXSSOHPHQWHGE\TXHVWLRQQDLUHRUWHVWLQJGDWD E 48$1TXDOZKLFKLVXVHIXOIRU
H[DPSOHWRGHVFULEHDQDVSHFWRIDTXDQWLWDWLYHVWXG\WKDWFDQQRWEHTXDQWLILHGRUWR
HPEHGDTXDOLWDWLYHFRPSRQHQWZLWKLQDODUJHUSULPDULO\TXDQWLWDWLYHVWXG\VXFKDVD
SURJUDPPHHYDOXDWLRQDQG F 48$/48$1ZKLFKLVHPSOR\HGIRUH[DPSOHLQD
WUDGLWLRQDOWULDQJXODWLRQGHVLJQFRQGXFWHGIRUYDOLGDWLRQSXUSRVHV
,QJHQHUDOFRQFXUUHQWGHVLJQVDUHLQYDOXDEOHZKHQZHH[DPLQHDSKHQRPHQRQWKDW
KDVVHYHUDOOHYHOV)RUH[DPSOHWKHLPSDFWRIWHDFKHUFRJQLWLRQFDQEHVWXGLHGDWWKH
WHDFKHU¶VOHYHOXVLQJLQWHUYLHZVDQGDWWKHFODVV LHVWXGHQWV¶ OHYHOXVLQJ
TXHVWLRQQDLUHV&RQFXUUHQWGHVLJQVDUHDOVRXVHIXOIRUFRPELQLQJPLFURDQGPDFUR
SHUVSHFWLYHVIRUH[DPSOHTXDQWLWDWLYHUHVHDUFKFDQWDSODUJHVFDOHWUHQGVLQVRFLDOOLIH
ZKLOHTXDOLWDWLYHUHVHDUFKFDQSURYLGHDPLFURDQDO\VLVRIKRZWKHEURDGWUHQGVDIIHFW
RUDUHSHUFHLYHGE\WKHLQGLYLGXDOV
([SHULPHQWVZLWKSDUDOOHOLQWHUYLHZV 48$1TXDO
7KHPL[HGPHWKRGVSDWWHUQVGHVFULEHGDERYHKDYHLQYROYHGWKHXVHRIWKHWZRPRVW
FRPPRQUHVHDUFKPHWKRGVWKHTXHVWLRQQDLUHVXUYH\DQGWKHLQWHUYLHZVWXG\DVWKHNH\
UHVHDUFKFRPSRQHQWVEXWPHWKRGPL[LQJLVQRWFRQILQHGWRWKHVH-RKQVRQDQG
&KULVWHQVHQ IRUH[DPSOHSRLQWRXWWKDWZHFDQVRPHWLPHVLPSURYHH[SHULPHQWV
HYHQIXUWKHUE\FRQGXFWLQJLQWHUYLHZV RQHWRRQHRUIRFXVJURXSV WRJHWDWWKH
UHVHDUFKSDUWLFLSDQWV¶SHUVSHFWLYHVDQGPHDQLQJVWKDWOLHEHKLQGWKHH[SHULPHQWDO
UHVHDUFKILQGLQJV,QIDFWEHFDXVHH[SHULPHQWVLQYROYHIHDWXUHGSURFHVVHOHPHQWV LH
WKHGHYHORSPHQWRIWKHSDUWLFLSDQWVDVDUHVXOWRIWKHWUHDWPHQW LQFOXGLQJDTXDOLWDWLYH

SKDVHWRH[SORUHWKHQDWXUHRIVXFKSURFHVVHVLVDQDWXUDODQGSRWHQWLDOO\KLJKO\IUXLWIXO
GHVLJQWKDWFDQJUHDWO\HQKDQFHWKHVWXG\¶VLQWHUQDOYDOLGLW\
/RQJLWXGLQDOVWXG\ZLWKPL[HGPHWKRGVFRPSRQHQWV 48$148$/
7KHH[SHULPHQWDOGHVLJQGHVFULEHGDERYHLVDOUHDG\DQH[DPSOHRIDORQJLWXGLQDOVWXG\
DQGZHFDQFRQFOXGHLQJHQHUDOWKDWORQJLWXGLQDOGHVLJQVOHQGWKHPVHOYHVWRYDULRXV
FRPELQDWLRQVRITXDOLWDWLYHDQGTXDQWLWDWLYHPHWKRGVDWGLIIHUHQWVWDJHVRIWKHSURMHFW
7KLVKDVEHHQH[SODLQHGLQPRUHGHWDLOLQ&KDSWHU 6HFWLRQ
&RPELQLQJVHOIUHSRUWDQGREVHUYDWLRQDOGDWD 48$/48$1
7ZRIXQGDPHQWDOZD\VRIJDLQLQJLQIRUPDWLRQDERXWSHRSOHDUH D WKURXJKVHOI
UHSRUWWKDWLVWKURXJKWKHLQGLYLGXDOV¶RZQDFFRXQWV IRUH[DPSOHLQLQWHUYLHZVRU
TXHVWLRQQDLUHV DQG E WKURXJKH[WHUQDOREVHUYDWLRQRIWKHLQGLYLGXDO $WKLUGZD\
LQYROYHVDVVHVVLQJWKHSHUIRUPDQFHRIWKHLQGLYLGXDORQDQLQVWUXPHQWRUDWDVNEXWWKLV
IDOOVXQGHUWKHDXVSLFHVRIWHVWLQJDQGLVWKHUHIRUHRXWVLGHWKHVFRSHRIWKLVERRN %RWK
PHWKRGVKDYHDGYDQWDJHVDQGGLVDGYDQWDJHVDQGFRPELQLQJWKHPRIIHUVWKHFODVVLF
PHULWRIPL[HGPHWKRGVUHVHDUFKQDPHO\WRLQFUHDVHWKHVWUHQJWKVDQGPLWLJDWHWKH
ZHDNQHVVHVRIWKHVWXG\,QGHHGFODVVURRPREVHUYDWLRQVWXGLHVDUHRIWHQDFFRPSDQLHG
E\YDULRXVVHOIUHSRUWPHDVXUHVZLWKUHVHDUFKHUVEHLQJLQWHUHVWHGLQERWKWKH
FRQYHUJHQWDQGGLYHUJHQWUHVXOWV )RUPRUHGHWDLOVRQPL[HGPHWKRGVFODVVURRP
UHVHDUFKVHH6HFWLRQ
7KHOLVWRIWKHPRVWFRPPRQFRPELQDWLRQVRI48$/DQG48$1PHWKRGVLQWKH
SUHYLRXVVHFWLRQKDVSURYLGHG ,KRSH FRQYLQFLQJHYLGHQFHWKDWPHWKRGPL[LQJFDQEH
EHQHILFLDODQGRIWHQUHTXLUHVUHODWLYHO\OLWWOHLQYHVWPHQWRQWKHSDUWRIWKHUHVHDUFKHU
$OVRWKHYLDEOHH[DPSOHVPHQWLRQHGZLOOKDYHVXFFHHGHG DJDLQ,KRSH LQGHIXVLQJ
WKHSKLORVRSKLFDOFRQFHUQWKDWE\PL[LQJGLIIHUHQWZRUOGYLHZVZHPLJKWHQGXSLQ
µSDUDGLJPDWLFYDFXXP¶RULQDQLUUHVROYDEOHµSDUDGLJPDWLFVFKLVP¶0L[HGPHWKRGV
UHVHDUFKFDQEHVHHQLQPDQ\VLWXDWLRQVDVDQDWWUDFWLYHRSWLRQDQGDVH[SODLQHGHDUOLHU
WKHDSSURDFKKDVUHFHQWO\DFTXLUHGDJRRGUHSXWDWLRQ<HWZHGRQRWVHHPDQ\
SXEOLVKHGVWXGLHVWKDWIHDWXUHDQ\SULQFLSOHGXVHRIPHWKRGPL[LQJ6RZK\GRQ¶W
SHRSOHPL[PHWKRGVPRUH"
$OWKRXJKWKLVLVDFRPSOH[LVVXHZLWKXQGRXEWHGO\VHYHUDOH[SODQDWLRQV,EHOLHYH
WKDWWKHUHDUHWZRPDLQIDFWRUVDWWKHKHDUWRILWDODFNRIVXIILFLHQWNQRZOHGJHDERXW
PHWKRGPL[LQJDQGDODFNRIH[SHUWLVHWRLPSOHPHQWDPL[HGGHVLJQ7KHILUVWIDFWRULV
UHODWLYHO\HDV\WRGHDOZLWKPL[HGPHWKRGVUHVHDUFKLVDQHZSKHQRPHQRQRQWKH
UHVHDUFKSDOHWWHDQGWKHUHIRUHLWLVOLNHO\WRWDNHDZKLOHIRULWWREHIXOO\LQWHJUDWHGLQWR
PHWKRGRORJ\WH[WVDQGFRXUVHV+RZHYHURQWKHEDVLVRIWKHUDWHRILWVJURZLQJ
SRSXODULW\LWLVVDIHWRFRQFOXGHWKDWWKHIXOOHPDQFLSDWLRQRIWKHDSSURDFKLVQRWWRRIDU
DZD\%XWZKHQWKLVKDSSHQVZLOOSHRSOHDFWXDOO\XVHLW":LOOHVWDEOLVKHGUHVHDUFKHUV
FKDQJHWKHLUPRQRPHWKRGRORJLFDOVWDQFHDQGZLOO\RXQJVFKRODUVHPEUDFHWKH

FKDOOHQJHVRIPHWKRGVPL[LQJ",WLVQRWHDV\WRDQVZHUWKHVHTXHVWLRQVEHFDXVHPL[HG
PHWKRGVUHVHDUFKKDVDVHYHUHµVKRUWFRPLQJ¶LWUHTXLUHVWKHFRPSHWHQWKDQGOLQJRI
ERWKTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFK/HWXVH[DPLQHWKLVTXHVWLRQ
0RVWHVWDEOLVKHGUHVHDUFKHUVDUHQRWHTXDOO\ZHOOYHUVHGLQERWK48$/DQG48$1
PHWKRGRORJLHVDQGWKLVLVQRWVLPSO\GXHWRLQVXIILFLHQWRULPEDODQFHGUHVHDUFK
WUDLQLQJ$V,ZLOOIXUWKHUGLVFXVVLQ&KDSWHU,EHOLHYHWKDWPRVWUHVHDUFKHUVKDYHD
QDWXUDOLQFOLQDWLRQWRZDUGVHLWKHUTXDOLWDWLYHRUTXDQWLWDWLYHUHVHDUFKZKLFKLVPRVW
SUREDEO\UHODWHGWRGLIIHUHQFHVLQWKHLUFRJQLWLYHVW\OHVDQGFHUWDLQSHUVRQDOLW\WUDLWV
7DNLQJP\RZQFDVHDOWKRXJK,JHQXLQHO\DSSUHFLDWHTXDOLWDWLYHO\RULHQWHGFROOHDJXHV¶
VNLOOVLQWHDVLQJPHDQLQJIXOSDWWHUQVRXWRIPDVVHVRIUDWKHUIOXLGDQGPHVV\GDWDP\
DWWUDFWLRQWRZHOOVWUXFWXUHGV\VWHPVFOHDUFXWERXQGDULHVVWDQGDUGL]HGSURFHGXUHVDQG
VWDWLVWLFDODQDO\VHVPDNHPHPRUHQDWXUDOO\DTXDQWLWDWLYHUHVHDUFKHU6LPLODUO\WKHUH
DUHPDQ\SHRSOHZKRZRXOGIHHOWKDWWKH\DUHOHVVDEOHWRGRWKHPVHOYHVMXVWLFHXVLQJ
RQHDSSURDFKWKDQWKHRWKHU,QDGGLWLRQLWUHTXLUHVFRQVLGHUDEOHHIIRUWWRVWXG\D
SKHQRPHQRQZLWKWZR RUPRUH VHSDUDWHPHWKRGVDQGLQWKHOLJKWRIDOOWKLVLWLV
XQGHUVWDQGDEOHWKDWPDQ\ LIQRWPRVW UHVHDUFKHUVPD\SUHIHUWRUHPDLQRQVDIH
PRQRPHWKRGRORJLFDOJURXQGV
$ORQJZLWKRWKHUV IRUH[DPSOH+DQVRQHWDO-RKQVRQDQG&KULVWHQVHQ
,EHOLHYHWKDWWKHUHDOSRWHQWLDOIRUWKHLPSOHPHQWDWLRQRIPL[HGPHWKRGV
UHVHDUFKOLHVLQZRUNLQJLQWHDPVWKDWFRQVLVWRIPHPEHUVZLWKGLIIHUHQWUHVHDUFK
RULHQWDWLRQV'XII LVULJKWZKHQVKHFRQFOXGHVWKDWµ&RPELQLQJWKHH[SHUWLVH
RIDSSOLHGOLQJXLVWVHVSRXVLQJGLIIHUHQWUHVHDUFKSDUDGLJPVLQFRPSOHPHQWDU\W\SHVRI
DQDO\VLVRIWKHVDPHSKHQRPHQRQZRXOGDOVR\LHOGULFKHUDQDO\VHVRIFRPSOH[LVVXHV¶
$VHYLGHQFHGE\WKHQXPEHURIFRDXWKRUHGSXEOLFDWLRQVLQDSSOLHGOLQJXLVWLFV
FROODERUDWLYHUHVHDUFKLVZLGHVSUHDGVHWWLQJXSVXFKµPL[HG¶WHDPVLVWKHUHIRUHQRWDW
DOOLQFRQFHLYDEOH,WPD\ZHOOEHWKHFDVHWKDWSDUWRIWKHUHDVRQIRUWKHLQVXIILFLHQW
FRRSHUDWLRQEHWZHHQTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKHUVLQWKHSDVWKDVEHHQWKH
ODFNRIDQ\REYLRXVEOXHSULQWVIRUVXFKFROODERUDWLRQ0L[HGPHWKRGVGHVLJQVQRZ
RIIHUDWWUDFWLYHDFWLRQSODQVWKDWFDQIDFLOLWDWHWKHIRUPDWLRQRIWHDPV*UHHQHHWDO
SURYLGHDGHWDLOHGGHVFULSWLRQRIKRZVXFKWHDPZRUNZDVLQLWLDWHGDQG
VWUXFWXUHGDVSDUWRIWKHµ+DUYDUG)DPLO\5HVHDUFK3URMHFW¶DVWKHDXWKRUVFRQFOXGHE\
PL[LQJPHWKRGVWKH\DUULYHGDWDXQLTXHFRPSOH[DQGQXDQFHGXQGHUVWDQGLQJRIWKHLU
UHVHDUFKWRSLF WKHUHODWLRQVKLSEHWZHHQORZLQFRPHPRWKHUV¶HPSOR\PHQWDQGWKHLU
IDPLO\HGXFDWLRQDOLQYROYHPHQW DQGWKH\µDOVRDUULYHGDWDQDSSUHFLDWLRQRIPL[LQJ
PHWKRGVDVDFKDOOHQJLQJEXWHPLQHQWO\ZRUWKZKLOHDSSURDFK¶ S
)LQDOO\WKHUHPD\EHDWKLUGUHDVRQIRUWKHSDXFLW\RIPL[HGPHWKRGVVWXGLHV7KH
SXEOLFDWLRQSUHVVXUHVRIWKHFXUUHQWUHVHDUFKFOLPDWHSURPRWHSLHFHPHDOSXEOLFDWLRQVLQ
JHQHUDODQGWKHUHIRUHHYHQZKHQDSURMHFWLQYROYHVPHWKRGVPL[LQJWKHDXWKRUVPLJKW
FKRRVHWRWU\DQGSXEOLVKWKHGLIIHUHQWSKDVHVVHSDUDWHO\ -XGLW.RUPRVSHUVRQDO
FRPPXQLFDWLRQ

&ODVVURRPUHVHDUFK
&ODVVURRPUHVHDUFKLVDEURDGXPEUHOODWHUPIRUHPSLULFDOLQYHVWLJDWLRQVWKDWXVHWKH
FODVVURRPDVWKHPDLQUHVHDUFKVLWH7KXVWKHWHUPFRQFHUQVDQ\VWXG\WKDWH[DPLQHV
KRZWHDFKLQJDQGOHDUQLQJWDNHVSODFHLQFRQWH[W$OWKRXJKJLYHQWKHYDULHW\RI
SRVVLEOHWHDFKLQJVSDFHV IRUH[DPSOHVHPLQDUURRPVODQJXDJHODEVFRPSXWHUURRPV
OHFWXUHWKHDWUHV LWPD\QRWEHDEVROXWHO\VWUDLJKWIRUZDUGWRGHILQHZKDWDµFODVVURRP¶
LVWKHEHVWWKLQJLVWRUHO\RQRXUFRPPRQVHQVHDQGLQFOXGHDQ\SK\VLFDOVSDFHLQ
ZKLFKVFKHGXOHGWHDFKLQJWDNHVSODFH,QDUHFHQWRYHUYLHZRIWKHILHOG1XQDQ
KDVSRLQWHGRXWWKDWVXFKDGHILQLWLRQKDVUHFHQWO\FRPHXQGHUFKDOOHQJHEHFDXVHRIWKH
UDSLGVSUHDGRILQIRUPDWLRQWHFKQRORJ\LQHGXFDWLRQDQGWKHVXEVHTXHQWGHYHORSPHQWRI
µYLUWXDO¶FODVVURRPVEXWLQWKLVFKDSWHU,GRQRWH[WHQGWKHQRWLRQRIFODVVURRPUHVHDUFK
WR,QWHUQHWEDVHGOHDUQLQJHQYLURQPHQWV
:K\GRZHQHHGDVHSDUDWHFKDSWHURQFODVVURRPUHVHDUFK"$IWHUDOO,DPLQ
DJUHHPHQWZLWK0DFNH\DQG*DVV ZKRSRLQWRXWWKDWDOWKRXJKFODVVURRPV
FRQVWLWXWHDGLVWLQFWFRQWH[WIRUUHVHDUFKPDQ\RIWKHPHWKRGRORJLFDOSUDFWLFHVDSSOLHG
LQUHVHDUFKLQJFODVVURRPVDUHQRWXQLTXHWRFODVVURRPVHWWLQJV<HWWKHVHDXWKRUVDOVR
GHYRWHDZKROHFKDSWHUWRWKHWRSLFIRUWKHVDPHUHDVRQDV,GRWKHFODVVURRP²DQG
PRVWRIWHQWKHIRUHLJQVHFRQGODQJXDJHFODVVURRP²LVDSULPDU\UHVHDUFKVLWHLQ
DSSOLHGOLQJXLVWLFLQYHVWLJDWLRQVDQGWKHXQLTXHIHDWXUHVRIWKLVFRQWH[WKDYHDVWURQJ
EHDULQJRQWKHZD\ZHFDQFRQGXFWUHVHDUFKLQLW7KLVGRHVQRWPHDQWKDWHYHU\WKLQJ
DERXWFODVVURRPUHVHDUFKLVGLVWLQFWLYHDVZHZLOOVHHFODVVURRPUHVHDUFKHUVXVH
VHYHUDOPHWKRGVWKDWKDYHDOUHDG\EHHQGHVFULEHG IRUH[DPSOHVXUYH\VTXDVL
H[SHULPHQWVHWKQRJUDSKLFUHVHDUFK <HWWKHUHDUHWKUHHXQLTXHDVSHFWVRIFRQGXFWLQJ
UHVHDUFKLQDFODVVURRPFRQWH[WWKDWUHTXLUHVSHFLDODWWHQWLRQ
7KHUHVHDUFKPHWKRGRIFODVVURRPREVHUYDWLRQZKLFKLVDKLJKO\GHYHORSHGGDWD
FROOHFWLRQDSSURDFKW\SLFDORIH[DPLQLQJOHDUQLQJHQYLURQPHQWV
7KHSURPLQHQFHWKDWPRVWFODVVURRPUHVHDUFKHUVSODFHRQFRQGXFWLQJPL[HG
PHWKRGVUHVHDUFKWREHDEOHWRXQGHUVWDQGWKHLQWULFDWHWDSHVWU\RIFODVVURRPHYHQWV
,DUJXHWKURXJKRXWWKLVERRNWKDWLQPDQ\UHVHDUFKVLWXDWLRQVFRPELQLQJ
TXDQWLWDWLYHDQGTXDOLWDWLYHPHWKRGRORJLHVFDQEHEHQHILFLDODQGVHYHUDOVFKRODUV
FODLPWKDWLQFODVVURRPUHVHDUFKPL[LQJPHWKRGVLVLQGLVSHQVDEOH
'RLQJKLJKTXDOLW\FODVVURRPUHVHDUFKSRVHVVSHFLDOFKDOOHQJHV$OWKRXJKGRLQJ
JRRGUHVHDUFKLVQHYHUHDV\LWDSSHDUVWKDWWKHLQKHUHQWGLIILFXOWLHVRIFODVVURRP
LQYHVWLJDWLRQVDUHDVDOLHQWIHDWXUHRIWKLVSDUWLFXODUUHVHDUFKW\SHXVXDOO\
XQGHUSOD\HGE\UHVHDUFKUHSRUWVDQGPHWKRGRORJ\WH[WV6FKDFKWHUDQG*DVV
YLLL VXPPDUL]HWKLVSRLQWVRZHOO
5HSRUWVRIUHVHDUFKSURMHFWVPDNHLWDOOORRNVRVLPSOH«7KHUHLVQR
LQGLFDWLRQRIWKHEORRGVZHDWDQGWHDUVWKDWJRLQWRJHWWLQJSHUPLVVLRQWR

XQGHUWDNHWKHSURMHFWWKDWJRLQWRDFWXDOGDWDFROOHFWLRQWKDWJRLQWR
WUDQVFULSWLRQDQGVRIRUWK
7KHVLWXDWLRQDFFRUGLQJWR6FKDFKWHUDQG*DVV LVDELWOLNHJRLQJWRWKH

WKHDWUHWKHSHUIRUPDQFHDOOORRNVVRHDV\DQGVRSURIHVVLRQDOEXWZHUDUHO\WKLQNDERXW
ZKDWKDVJRQHRQEHKLQGWKHVFHQHVGXULQJWKHSUHSDUDWLRQSKDVH7DONLQJDERXWKHU
FODVVURRPEDVHG3K'ZRUN5RVVLWHU DOVRHPSKDVL]HGWKDWQRQHRIWKHUHVHDUFK
PDQXDOVVKHKDGUHYLHZHGSURYLGHGDWUXO\FRPSUHKHQVLYHWUHDWPHQWRIWKHPDQ\
FRPSOH[LWLHVRIWKHUHVHDUFKSURFHVVVKHFDOOHGIRUUHVHDUFKWH[WVWRUHIOHFWWKHUHDOLWLHV
RIWKHFRQWH[WVLQZKLFKWKHUHVHDUFKZDVFRQGXFWHGEHWWHU'XIIDQG(DUO\ DJUHH
WKDWWKHGLVPLVVDORUGRZQSOD\LQJRIGLIILFXOWLHVDVVRFLDWHGZLWKWKHDFWXDOFRQGXFWRI
FODVVURRPUHVHDUFKFRQVWLWXWHVDµGLVVHUYLFHWRRWKHUUHVHDUFKHUVDQGSDUWLFXODUO\WKRVH
ZLWKOHVVH[SHULHQFH¶ S DQGWKH\HQFRXUDJHPRUHRSHQGLVFXVVLRQRIUHVHDUFK
SUREOHPVWKDWPD\LPSHGHSURGXFWLYHFODVVURRPUHVHDUFK
,QWKHIROORZLQJ,ZLOOGLVFXVVWKHWKUHHLVVXHVRXWOLQHGDERYHDQGFRQFOXGHWKH
FKDSWHUZLWKDVHFWLRQRQDVSHFLDODQGVRPHZKDWFRQWURYHUVLDOW\SHRIFODVVURRP
UHVHDUFKWKDWLVXVXDOO\UHIHUUHGWRDVµDFWLRQUHVHDUFK¶/HWXVILUVWKRZHYHUVWDUWRXU
H[SORUDWLRQZLWKDVKRUWRYHUYLHZRIWKHRULJLQVDQGWKHPDLQW\SHVRIFODVVURRP
UHVHDUFK
2ULJLQVDQGPDLQW\SHVRIFODVVURRPUHVHDUFK
$V:UDJJ GHVFULEHVHDUO\FODVVURRPUHVHDUFKZDVLQLWLDWHGLQWKHVDQG
VLQWKH8QLWHG6WDWHVLQYHVWLJDWLQJWKHHIIHFWLYHQHVVRIWHDFKHUEHKDYLRXUDQG
WHDFKHUWDONDQGSURGXFLQJUDWKHUµWHQGHQWLRXV¶YDOXHVWDWHPHQWVDERXWWKHYDULRXV
WHDFKHUDFWV0RGHUQFODVVURRPUHVHDUFKVWDUWHGLQHDUQHVWPXFKODWHULQWKHVDV
SDUWRIWHDFKHUWUDLQLQJFRXUVHVZKHQWUDLQHUVUHDOL]HGWKDWWKH\QHHGHGDSSURSULDWH
REVHUYDWLRQLQVWUXPHQWVDQGTXDOLW\VWDQGDUGVWREHDEOHWRHYDOXDWHHIIHFWLYHWHDFKLQJ
7KHDUHDZDVJLYHQIXUWKHULPSHWXVLQWKHVE\WKHVRFDOOHGµPHWKRGVFRPSDULVRQ
VWXGLHV¶ 1XQDQ ZKLFKLQYROYHGODUJHVFDOHLQYHVWLJDWLRQVWKDWFRPSDUHGYDULRXV
WHDFKLQJPHWKRGVIRUH[DPSOHWKHDXGLROLQJXDOPHWKRGWRPRUHWUDGLWLRQDODSSURDFKHV
LQWKHODQJXDJHWHDFKLQJDUHD7KHPRYHIURPWHDFKHUWUDLQLQJWRPRUHEDVLFUHVHDUFK
UHVXOWHGLQDUHILQHPHQWRIWKHLQVWUXPHQWVXVHG $OOZULJKWDQG%DLOH\
REVHUYDWLRQVFKHGXOHVVFKHPHVWKDWLQLWLDOO\FRQWDLQHGUDWKHUEURDGDQGSUHVFULSWLYH
FDWHJRULHVZHUHUHSODFHGE\PRUHHODERUDWHFKHFNOLVWVWKDWVHUYHGGHVFULSWLYHSXUSRVHV
DQGDQXPEHURIVWDQGDUGL]HGREVHUYDWLRQVFKHPHVZHUHSXEOLVKHGWKHPRVWIDPRXV
EHLQJ)ODQGHUV¶,QWHUDFWLRQ$QDO\VLV&DWHJRULHV ),$& VFKHPH LQFOXGHGIRUH[DPSOH
LQWKH$SSHQGL[RI$OOZULJKWDQG%DLOH\
&RQWHPSRUDU\FODVVURRPUHVHDUFKERWKLQHGXFDWLRQDOSV\FKRORJ\DQGLQDSSOLHG
OLQJXLVWLFVLVVWULYLQJIRUDVLWXDWHGXQGHUVWDQGLQJRIOHDUQLQJGRFXPHQWLQJDQG
DQDO\VLQJWKHG\QDPLFLQWHUSOD\RIYDULRXVFODVVURRPSURFHVVHVDQGFRQGLWLRQV
FRQWULEXWLQJWRYDULDWLRQLQOHDUQLQJRXWFRPHV$V/LJKWERZQ VXPPDUL]HV
LQKHUFRPSUHKHQVLYHRYHUYLHZRIFODVVURRP6/$UHVHDUFKWKHYDULRXVFODVVURRP

UHVHDUFKSURMHFWVVKDUHWKHXQLI\LQJGHVLUHWRµLGHQWLI\DQGEHWWHUXQGHUVWDQGWKHUROHVRI
WKHGLIIHUHQWSDUWLFLSDQWVLQFODVVURRPLQWHUDFWLRQWKHLPSDFWWKDWFHUWDLQW\SHRI
LQVWUXFWLRQPD\KDYHRQ)/6/OHDUQLQJDQGWKHIDFWRUVZKLFKSURPRWHRULQKLELW
OHDUQLQJ¶)RUWKLVSXUSRVHLQYHVWLJDWRUVHPSOR\WKHZKROHUHSHUWRLUHRIDYDLODEOH
UHVHDUFKPHWKRGRORJLFDOWHFKQLTXHVZKLOHVWUXFWXUHGFODVVURRPREVHUYDWLRQKDV
PDLQWDLQHGDSURPLQHQWSRVLWLRQLQFODVVURRPUHVHDUFKLQIRUPDWLRQDERXWFODVVURRPVLV
DOVRJDWKHUHGE\VHOIUHSRUWWHFKQLTXHVVXFKDVVXUYH\VLQWHUYLHZVDQGGLDU\VWXGLHV,Q
DGGLWLRQWKHUROHRIVSHFLILFLQVWUXFWLRQDOIHDWXUHVDQGSUDFWLFHVLVRIWHQH[DPLQHGE\
PHDQVRIFRQVFLRXVPDQLSXODWLRQZLWKLQTXDVLH[SHULPHQWDOGHVLJQV$SURPLQHQWDUHD
RIFODVVURRPUHVHDUFKLQYROYHVHWKQRJUDSKLFVWXGLHVZKLFKWUHDWFODVVURRPVDV
VXEFXOWXUHVZKRVHFRPSOH[LWLHVQHHGWREHXQFRYHUHGWKURXJKVXVWDLQHGTXDOLWDWLYH
HQJDJHPHQWEDVHGRQLQVLGHUDFFRXQWV YDQ/LHU 7KHVHODWWHUPHWKRGVKDYH
EHHQGHVFULEHGLQ&KDSWHUVDQGDQGZLOOQRWEHIXUWKHUGLVFXVVHGKHUH$ILQDO
GRPDLQLQFODVVURRPUHVHDUFKLQYROYHVWKHGLVFRXUVHDQDO\VLVRIFODVVURRPLQWHUDFWLRQ
EXWDVGHVFULEHGLQ&KDSWHUODQJXDJHDQDO\VLVIDOOVRXWVLGHWKHVFRSHRIWKLVERRN
)RUDUHYLHZVHHIRUH[DPSOHWKHVSHFLDOLVVXHRI$SSOLHG/LQJXLVWLFVHGLWHGE\
=XHQJOHUDQG0RUL
&ODVVURRPREVHUYDWLRQ
&ODVVURRPVDUHH[FHSWLRQDOO\EXV\SODFHVVRREVHUYHUVQHHGWREHRQWKHLUWRHV
:UDJJ
%HVLGHVDVNLQJTXHVWLRQVREVHUYLQJWKHZRUOGDURXQGXVLVWKHRWKHUEDVLFKXPDQ
DFWLYLW\WKDWDOORIXVKDYHEHHQLQYROYHGLQVLQFHEDE\KRRGWROHDUQDQGJDLQ
XQGHUVWDQGLQJ)URPDUHVHDUFKSHUVSHFWLYHREVHUYDWLRQLVIXQGDPHQWDOO\GLIIHUHQW
IURPTXHVWLRQLQJEHFDXVHLWSURYLGHVGLUHFWLQIRUPDWLRQUDWKHUWKDQVHOIUHSRUW
DFFRXQWVDQGWKXVLWLVRQHRIWKHWKUHHEDVLFGDWDVRXUFHVIRUHPSLULFDOUHVHDUFK ZLWK
WHVWLQJEHLQJWKHWKLUG $OWKRXJKREVHUYDWLRQLVDQLQWHJUDOSDUWRIWKHUHVHDUFK
UHSHUWRLUHRIHWKQRJUDSK\WKHW\SLFDOW\SHRIFODVVURRPREVHUYDWLRQWKDWWKLVVHFWLRQ
IRFXVHVRQLVQRWLQWHQGHGWRSURYLGHDIXOOHWKQRJUDSKLFDFFRXQWEXWIRFXVHVRQGHWDLOV
RIVSHFLILFDUHDVIRUWKLVUHDVRQ3ROLR FDOOHGWKLVW\SHRIUHVHDUFK
µQRQHWKQRJUDSKLFQRQH[SHULPHQWDOUHVHDUFK¶
7RRUJDQL]HWKHPDQ\GLIIHUHQWZD\VLQZKLFKZHFDQREVHUYHFODVVURRPVWZR
GLFKRWRPLHVDUHXVXDOO\RIIHUHGµSDUWLFLSDQW¶YHUVXVµQRQSDUWLFLSDQWREVHUYDWLRQ¶DQG
µVWUXFWXUHG¶YHUVXVµXQVWUXFWXUHGREVHUYDWLRQ¶7KHSDUWLFLSDQWREVHUYHUEHFRPHVDIXOO
PHPEHURIWKHJURXSWDNLQJSDUWLQDOOWKHDFWLYLWLHV7KLVLVWKHXVXDOIRUPRI
REVHUYDWLRQLQHWKQRJUDSKLFVWXGLHV,QFODVVURRPREVHUYDWLRQKRZHYHUWKHUHVHDUFKHU
LVXVXDOO\QRWRURQO\PLQLPDOO\LQYROYHGLQWKHVHWWLQJDQGWKHUHIRUHKHVKHFDQEH
GHVFULEHGDVDµQRQSDUWLFLSDQWREVHUYHU¶:KLOHWKLVGLVWLQFWLRQPD\EHKHOSIXOLQVRPH
FDVHV0RUVHDQG5LFKDUGV ZDUQXVWKDWLWVLPSOLILHVWKHP\ULDGZD\VRI
ZDWFKLQJDQGOLVWHQLQJDVWKH\DUJXHQRREVHUYHULVHQWLUHO\DSDUWLFLSDQWDQGLWLV

LPSRVVLEOHWRREVHUYHLQDOPRVWHYHU\QRQH[SHULPHQWDOVLWXDWLRQZLWKRXWVRPH
SDUWLFLSDWLRQ
7KHµVWUXFWXUHGXQVWUXFWXUHG¶GLVWLQFWLRQLVVLPLODUWRWKHµTXDQWLWDWLYHTXDOLWDWLYH¶
GLVWLQFWLRQLQREVHUYDWLRQDOWHUPV+LJKO\VWUXFWXUHGREVHUYDWLRQLQYROYHVJRLQJLQWR
WKHFODVVURRPZLWKDVSHFLILFIRFXVDQGZLWKFRQFUHWHREVHUYDWLRQFDWHJRULHV 48$1
ZKHUHDVXQVWUXFWXUHGREVHUYDWLRQLVOHVVFOHDURQZKDWLWLVORRNLQJIRUDQGWKH
UHVHDUFKHUQHHGVWRREVHUYHILUVWZKDWLVWDNLQJSODFHEHIRUHGHFLGLQJRQLWVVLJQLILFDQFH
IRUWKHUHVHDUFK 48$/ &RKHQHWDO 7KHIRUPHUDSSURDFKLQYROYHV
FRPSOHWLQJDQREVHUYDWLRQVFKHPHZKLOHWKHODWWHUFRPSOHWLQJQDUUDWLYHILHOGQRWHV
RIWHQVXSSOHPHQWHGE\PDSVRUGLDJUDPV7KLVLVRIFRXUVHDFRQWLQXXPDQGLQ
SUDFWLFHXVXDOO\VRPHFRPELQDWLRQRIWKHWZRDSSURDFKHVWDNHVSODFH
,QWKHIROORZLQJ,ZLOOIRFXVSULPDULO\RQVWUXFWXUHGTXDQWLWDWLYHREVHUYDWLRQWKDW
XWLOL]HVµREVHUYDWLRQVFKHPHV¶EHFDXVHWKLVLVWKHXQLTXHGDWDFROOHFWLRQPHWKRG
DVVRFLDWHGZLWKFODVVURRPUHVHDUFK+RZHYHUOHWPHVWUHVVDVDSUHOLPLQDU\WKDW
UHJDUGOHVVRIKRZVRSKLVWLFDWHGDQREVHUYDWLRQSURWRFROPLJKWEHLWZLOOIDLOWRWHOOWKH
ZKROHVWRU\RIFODVVURRPOLIH$V$OOZULJKWDQG%DLOH\ ZDUQHGXVVWUXFWXUHG
µFORVHG¶WHFKQLTXHVPD\HDVLO\PLVVWKHLQVLJKWVWKDWFRXOGEHSURYLGHGE\WKH
SDUWLFLSDQWVWKHPVHOYHV7RFDSWXUHVXFKLQVLJKWVZHQHHGWRFRPELQHVWUXFWXUHG
REVHUYDWLRQZLWKDOWHUQDWLYHIRUPVRIGDWDFROOHFWLRQE\PHDQVRIPL[HGPHWKRGV
UHVHDUFKZKLFKZLOOEHGHVFULEHGLQ6HFWLRQ
2EVHUYDWLRQVFKHPHV
,IZHGHFLGHWRXVHDVWUXFWXUHGµREVHUYDWLRQVFKHPH¶ DOVRFDOOHGµREVHUYDWLRQDO
VFKHGXOH¶RUµSURWRFRO¶ ZHFDQFKRRVHIURPDQXPEHURIUHDGLO\DYDLODEOHLQVWUXPHQWV
EXWLQPRVWFDVHVWKH\ZLOOQHHGWREHDGDSWHGWRRXUVSHFLILFUHVHDUFKIRFXVDQG
FODVVURRPVLWXDWLRQ$OOZULJKWDQG%DLOH\ SURYLGHH[DPSOHVRIVHYHQGLIIHUHQW
SXEOLVKHGFRGLQJVFKHPHVDQGXVHIXOLQIRUPDWLRQRQH[LVWLQJVFKHPHVFDQDOVREH
IRXQGLQ&KDXGURQ DQG0DFNH\DQG*DVV
7KHPDLQSULQFLSOHRIREVHUYDWLRQVFKHPHVLV²VLPLODUWRTXHVWLRQQDLUHV²WRKDYHD
UDQJHRIV\VWHPDWLFFDWHJRULHVZKLFKDOORZWKHREVHUYHUWRUHFRUGHYHQWVTXLFNO\E\
XVLQJWDOO\PDUNV7KHUHLVXVXDOO\QRWLPHGXULQJWKHREVHUYDWLRQWRHQWHUOHQJWK\
RSHQHQGHGFRPPHQWVVRWKHREVHUYDWLRQVFKHPHQHHGVWREHFDUHIXOO\SLORWHGLQ
DGYDQFH7KHUHDUHWZRPDLQPHWKRGVIRUUHFRUGLQJHYHQWV
x ,QµHYHQWVDPSOLQJ¶ZHHQWHUDWDOO\PDUNDJDLQVWDFDWHJRU\HYHU\WLPHLWRFFXUV
IRUH[DPSOHHYHU\WLPHWKHWHDFKHUDVNVDTXHVWLRQ 7KXVWKLVPHWKRGSURYLGHV
DQDFFXUDWHGHVFULSWLRQRIWKHWRWDOIUHTXHQF\RIWKHHYHQWVRUSURFHGXUHVREVHUYHG
x ,QµWLPHVDPSOLQJ¶DFDWHJRU\LVUHFRUGHGDWDIL[HGLQWHUYDOPRVWRIWHQHYHU\
VHFRQGVRUPLQXWH7KHUHVHDUFKHUHLWKHUQRWHVZKDWLVKDSSHQLQJRQWKHVWURNHRI
WKHLQWHUYDORUFKDUWVZKDWKDVKDSSHQHGGXULQJWKHSUHFHGLQJLQWHUYDO7KXVWLPH
VDPSOLQJGRHVQRWSURYLGHDQH[KDXVWLYHUHFRUGRIWKHRFFXUUHQFHRIDQHYHQWEXW

UDWKHUJLYHVDFKURQRORJLFDOUHSUHVHQWDWLRQRIWKHIORZRIWKHZKROHFODVVWKDWLV
WKHGLVWULEXWLRQRIWKHSDUWLFXODUSKHQRPHQRQWKURXJKRXWWKHFODVV
:KDWHYHUWKHH[DFWFRGLQJFRQYHQWLRQHPSOR\HGWKHILQDOVFRUHVDUHXVXDOO\
REWDLQHGE\DGGLQJXSWKHWDOO\PDUNVIRUHDFKFDWHJRU\DQGVWDQGDUGL]LQJWKHVXPVIRU
WLPHWRFRPSHQVDWHIRUXQHTXDOOHVVRQOHQJWKV IRUH[DPSOHE\GLYLGLQJWKHVXPE\WKH
WRWDOOHQJWKRIWKHOHVVRQLQPLQXWHVDQGPXOWLSO\LQJLWE\ :HVKRXOGQRWHKHUH
WKDWREVHUYDWLRQVFKHPHVPD\DOVRFRQWDLQµUDWLQJVFDOHV¶ RIWHQµVHPDQWLFGLIIHUHQWLDO
VFDOHV¶²VHH6HFWLRQ IRUWKHUHVHDUFKHUWRPDNHVRPHRYHUDOOMXGJHPHQWVDERXW
VRPHDVSHFWVRIWKHFODVVREVHUYHGHLWKHUGXULQJRULPPHGLDWHO\DIWHUWKHREVHUYDWLRQ
:KDWNLQGRIFDWHJRULHVFDQDQREVHUYDWLRQVFKHPHFRQWDLQ"%HFDXVHFODVVURRP
SURFHVVHVDUHH[WUHPHO\YDULHGDQGLQYROYHGLIIHUHQWDQGFRQWLQXRXVO\FKDQJLQJ
FRPELQDWLRQVRISDUWLFLSDQWVWKHUDQJHRIFDWHJRULHVXVHGLQWKHSDVWKDVEHHQEURDG
IURPDVSHFWVRIWHDFKHUWDONDQGERG\ODQJXDJHWKURXJKWDVNRUJDQL]DWLRQWRVWXGHQW
EHKDYLRXUVVXFKDVYROXQWHHULQJ7KHRQO\FULWHULRQLVWKDWWKHFDWHJRU\VKRXOGUHIHUWR
DQREVHUYDEOHSKHQRPHQRQLQRQHRIWZRZD\V
x $µORZLQIHUHQFH¶FDWHJRU\LVVRVWUDLJKWIRUZDUGWKDWHYHQLQUHDOWLPHFRGLQJ LH
RQJRLQJFRGLQJGXULQJREVHUYDWLRQ WKHREVHUYHUFDQUHDFKDOPRVWSHUIHFW
UHOLDELOLW\LQUHFRUGLQJLQVWDQFHVRILW IRUH[DPSOHWHDFKHUZULWLQJRQWKHERDUG
x $µKLJKLQIHUHQFH¶FDWHJRU\LVEDVHGRQZKDWWKHREVHUYHUFDQVHHEXWLWDOVR
UHTXLUHVVRPHMXGJHPHQWDERXWWKHIXQFWLRQRUPHDQLQJRIWKHREVHUYHGEHKDYLRXU
IRUH[DPSOHW\SHRIIHHGEDFNRUSUDLVH :LWKVXFKFDWHJRULHVFRQVLVWHQW
UHFRUGLQJFDQQRWEHDFKLHYHGZLWKRXWVRPHWUDLQLQJ
0DFNH\DQG*DVV SRLQWRXWWKDWDOWKRXJKFODVVURRPREVHUYDWLRQVFKHPHV
YDU\FRQVLGHUDEO\ZHFDQLGHQWLI\VRPHFRPPRQHOHPHQWVZKLFKVHHPWREHFHQWUDOWR
WKHGHVFULSWLRQRIFODVVURRPSURFHVVHVUHJDUGOHVVRIWKHVSHFLILFIRFXVRIWKHVWXG\
6XFKFRPPRQFDWHJRULHVLQFOXGHWKHµJURXSLQJIRUPDW¶RIWKHSDUWLFLSDQWV IRU
H[DPSOHLQGLYLGXDOSDLURUJURXSZRUN µFRQWHQWWRSLF¶RIWKHWDVNOHVVRQPDLQ
FKDUDFWHULVWLFVRIWKHµLQWHUDFWLRQ¶ IRUH[DPSOHZKRLQLWLDWHGLWRUZKDWODQJXDJHLV
XVHG DQGWKHWDUJHWHGµOLQJXLVWLFDUHD¶%\ZD\RILOOXVWUDWLRQOHWPHGHVFULEHWKUHH
UDWKHUGLIIHUHQWVFKHPHV*HUWUXGH0RVNRZLW]¶V)OLQW6SDGDDQG)U|KOLFK¶V&2/7DQG
*XLOORWHDX[DQG'|UQ\HL¶V02/7
7KH)OLQW )RUHLJQ/DQJXDJH,QWHUDFWLRQ$QDO\VLV LV0RVNRZLW]¶V UHSULQWHG
LQ&KDXGURQDQG$OOZULJKWDQG%DLOH\ PRGLILFDWLRQRI)ODQGHUV¶IDPRXV
,QWHUDFWLRQ$QDO\VLV&DWHJRULHV ),$6 VFKHPHIRUXVHLQVHFRQGODQJXDJHVWXGLHV7KH
)OLQWUHTXLUHVUHDOWLPHFRGLQJLQWKUHHVHFRQGLQWHUYDOVDFFRUGLQJWRFDWHJRULHV
7KHVHDUHGLYLGHGLQWRWKUHHJURXSV D 7HDFKHUWDON FDWHJRULHVUHODWHGWRµLQGLUHFW
LQIOXHQFH¶VXFKDVµSUDLVHV¶RUµMRNHV¶DQGFDWHJRULHVUHODWHGWRµGLUHFWLQIOXHQFH¶
VXFKDVµJLYHVGLUHFWLRQ¶RUµFULWLFL]HVVWXGHQWUHVSRQVH¶ E 6WXGHQWWDON FDWHJRULHV
µVWXGHQWUHVSRQVHVSHFLILF¶µVWXGHQWUHVSRQVHFKRUDO¶DQGµVWXGHQWUHVSRQVHRSHQ

HQGHGRUVWXGHQWLQLWLDWHG¶ DQG F DPLVFHOODQHRXVFDWHJRU\FRQWDLQLQJFDWHJRULHV
VXFKDVµVLOHQFH¶µFRQIXVLRQZRUNRULHQWHG¶DQGµODXJKWHU¶
&2/7 &RPPXQLFDWLRQ2ULHQWDWLRQRI/DQJXDJH7HDFKLQJ6SDGDDQG)U|KOLFK
ZDVGHYHORSHGDVSDUWRIDODUJHVFDOHLQYHVWLJDWLRQRIFRPPXQLFDWLYH
FRPSHWHQFHDQGODQJXDJHWHDFKLQJDWWKH2QWDULR,QVWLWXWHRI6WXGLHVLQ(GXFDWLRQ
7RURQWR&DQDGD 7KHVFKHPHLVGLYLGHGLQWRWZRSDUWV3DUW$IRFXVHVRQFODVVURRP
HYHQWVDQGUHTXLUHVUHDOWLPHFRGLQJDWRQHPLQXWHLQWHUYDOVXVLQJDYHU\GHWDLOHGJULG
ZLWKFDWHJRU\FROXPQV VHH)LJXUH 7KHVHFRQFHUQYDULRXVDVSHFWVRIDFWLYLW\
W\SHSDUWLFLSDQWRUJDQL]DWLRQFRQWHQWVWXGHQWPRGDOLW\ LHODQJXDJHVNLOOVLQYROYHG
DQGPDWHULDOV2UJDQL]HGLQWRFDWHJRULHV3DUW%LVGHVLJQHGIRUQRQUHDOWLPH
SRVWKRFDQDO\VLVRIWKHFRPPXQLFDWLYHODQJXDJHIHDWXUHVRIHDFKDFWLYLW\ EDVHGRQ
WDSHUHFRUGLQJV 8VXDOO\RQO\FHUWDLQDFWLYLWLHVVHOHFWHGRQWKHEDVLVRI3DUW$DUH
VXEMHFWHGWRWKHYHU\GHWDLOHGPLFUROHYHODQDO\VLVRI3DUW%'XHWRLWVHODERUDWH
FDWHJRU\V\VWHPDQGWKHKLJKSURILOHUHVHDUFKLWKDVEHHQXVHGLQ&2/7KDVEHHQ
KLJKO\LQIOXHQWLDOLQ/FODVVURRPUHVHDUFK

&RPPXQLFDWLYH2ULHQWDWLRQRI /DQJXDJH7HDFKLQJ &2/7 3DUW$
6&+22/
7($&+(5 *5$'(6 '$7(
68%-(&7 /(66216 2%6(59(5
&RO
3$57,&25*$1,=$7,21 &217(17 678'(17 0$7(5,$/6
7,0( $&7,9,7,(6 723,&
&ODVV *URXS&RPE 0$1 /$1*8$*( 27+(5723,&6 &21752/ 02'$/,7< 7\SH 8VH
1$552: /,0,7(' %52$' 7H[W
7VF
6VF
&KRUDO
6DPH
'LIIHUHQW
,QGLYLGXDO
*U,QG
3URFHGXUH
'LVFLSOLQH
)RUP
)XQFWLRQ
'LVFRXUVH
6RFLROLQJ
&ODVVURRP
6WHUHRW\S
3HUV%LR
2WKHU
3HUVRQDO
5RXW6RF
)DP&RP
6FKRRO7
2WKHU
$EVWUDFW
3HUV5HI
,PDJLQDWLRQ
:RUOG7
2WKHU
7HDFKHU
7HDFKHU6WXG
6WXGHQW
/LVWHQLQJ
6SHDNLQJ
5HDGLQJ
:ULWLQJ
2WKHU
0LQLPDO
([WHQGHG
$XGLR
9LVXDO
3HGDJRJLF
6HPL3HGDJRJ
1RQ3HGDJRJ
+LJK&RQWURO
6HPL&RQWURO
0LQL&RQWURO

)LJXUH3DUW$RI6SDGDDQG)U|KOLFK¶V &RPPXQLFDWLRQ2ULHQWDWLRQRI/DQJXDJH7HDFKLQJREVHUYDWLRQVFKHPH

02/7 0RWLYDWLRQ2ULHQWDWLRQLQ/DQJXDJH7HDFKLQJ*XLOORWHDX[DQG'|UQ\HLLQ
SUHVV KDVEHHQGHYHORSHGPRGHOOHGRQ&2/7EXWIRFXVLQJRQDVSHFWVRIWKHWHDFKHU¶V
PRWLYDWLRQDOSUDFWLFHDQGWKHVWXGHQWV¶PRWLYDWHGEHKDYLRXULQODQJXDJHFODVVHV7KH
UHDOWLPHLQWHUYDOVDPSOLQJIRUPDWLVVLPLODUWR3DUW$RIWKH&2/7VFKHPHDQGVHYHUDO
RIWKHFDWHJRULHVLWFRQWDLQVFRLQFLGHZLWKWKRVHLQ&2/77KHFDWHJRULHVWKDW
GHVFULEHWKHWHDFKHU¶VPRWLYDWLRQDOSUDFWLFHIROORZ'|UQ\HL¶V V\VWHPRI
PRWLYDWLRQDOVWUDWHJLHV)LJXUHSURYLGHVDVDPSOHH[WUDFWIURPWKHVFKHPH

*HQHUDWLQJPDLQWDLQLQJDQGSURWHFWLQJVLWXDWLRQVSHFLILF
WDVNPRWLYDWLRQ 0RWLYDWHG
(QFRXUDJLQJ EHKDYLRXU
SRVLWLYH
7HDFKHUGLVFRXUVH $FWLYLW\GHVLJQ UHWURVSHFWLYH
6WDWLQJWKHFRPPXQLFDWLYHSXUSRVHXWLOLW\
VHOI
(OLFLWDWLRQRIVHOISHHUFRUUHFWLRQVHVVLRQ
HYDOXDWLRQ
FUHDWLYHLQWHUHVWLQJIDQWDV\HOHPHQW
0LQXWH
(DJHUYROXQWHHULQJ DWOHDVW
3DUWLFLSDWLQJ !RIWKHFODVV
$URXVLQJFXULRVLW\RUDWWHQWLRQ
3URPRWHVLQVWUXPHQWDOYDOXHV
3URPRWLQJLQWHJUDWLYHYDOXHV
$WWHQWLRQ !RIWKHFODVV
3URFHVVIHHGEDFNVHVVLRQ
LQWHOOHFWXDOFKDOOHQJH
3URPRWLQJFRRSHUDWLRQ
WDQJLEOHWDVNSURGXFW
(VWDEOLVKLQJUHOHYDQFH
5HIHUHQWLDO4XHVWLRQV
3URPRWLQJDXWRQRP\
WHDPFRPSHWLWLRQ
WDQJLEOHUHZDUG
SHUVRQDOL]DWLRQ
(IIHFWLYHSUDLVH
&ODVVDSSODXVH
*URXSZRUN
6LJQSRVWLQJ
6FDIIROGLQJ
RIDFWLYLW\
3DLUZRUN

)LJXUH([WUDFWIURPWKH02/7REVHUYDWLRQVFKHPH PRWLYDWLRQVSHFLILFFDWHJRULHVRQO\
*XLOORWHDX[DQG'|UQ\HLLQSUHVV
9LGHRWDSLQJFODVVURRPV
9LGHRUHFRUGLQJKDVSURYLGHGDWHFKQRORJ\WKDWPLJKWEHFRQVLGHUHGLGHDOIRU
FODVVURRPUHVHDUFKDVLWFDQUHSODFHWKHQHHGIRUUHDOWLPHFRGLQJ,QGHHGQRZDGD\V
HYHQPRVWSRVWJUDGXDWHVWXGHQWVFDQDUUDQJHWRERUURZJRRGTXDOLW\HTXLSPHQWIRU
WKHLUUHVHDUFKLQREVHUYDWLRQDOVHWWLQJVDQGPDQ\XQLYHUVLWLHVDQGUHVHDUFKFHQWUHVKDYH
QRWRQO\JRWYLGHRFDPHUDVDYDLODEOHIRUUHVHDUFKSXUSRVHVEXWFDQHYHQSURYLGH
WHFKQLFLDQVWRKHOS5HJUHWWDEO\LQFOXGLQJYLGHRUHFRUGLQJLQRXUUHVHDUFKGHVLJQGRHV
QRWHOLPLQDWHDOOWKHGLIILFXOWLHVDVVRFLDWHGZLWKFODVVURRPREVHUYDWLRQEHFDXVH
YLGHRWDSLQJDVDFODVVURRPUHVHDUFKWRROLVIDUIURPEHLQJSHUIHFW$V0DFNH\DQG*DVV
SRLQWRXWLQDODERUDWRU\VHWWLQJYLGHRUHFRUGLQJFDQEHDUHODWLYHO\

VWUDLJKWIRUZDUGPDWWHUEXWLQFODVVURRPVLWFDQSUHVHQWDXQLTXHVHWRISUREOHPV
UDQJLQJIURPWHFKQLFDOLVVXHVVXFKDVXVLQJPLFURSKRQHVWKDWSLFNXSHYHU\GHWDLO
YLGHRHTXLSPHQWLVXVXDOO\VWURQJHURQLPDJHWKDQVRXQG WRHWKLFDOLVVXHVRIKRZWR
DUUDQJHVWXGHQWVLIVRPHRIWKHPKDYHQRWFRQVHQWHGWREHYLGHRWDSHG
,QDGHWDLOHGDQDO\VLVRIWKHGLIIHUHQFHVEHWZHHQWKHµDQDO\VWH\HV¶DQGµFDPHUD
H\HV¶=XHQJOHUHWDO DUJXHWKDWZKLOHWHFKQRORJ\FDQHQKDQFHRXUDELOLW\WRVHH
FHUWDLQWKLQJVLWFDQDOVRFUHDWHµOLWHUDODQGILJXUDWLYHEOLQGVSRWV¶ S 7KH
UHVHDUFKHUVKDYHKLJKOLJKWHGWZRSUREOHPVLQSDUWLFXODU
x /LWHUDOEOLQGVSRWV$IL[HGFDPHUDFDQRQO\VHHZKDWLWLVSRLQWLQJDWDQGXVXDOO\
ZHFDQQRWEDFNWKHFDPHUDXSIDUHQRXJKWRFDSWXUHWKHHQWLUHFODVVDQGWKH
LQVWUXFWRU=XHQJOHUDQGKHUFROOHDJXHV ILUVWWULHGWRFRPSHQVDWHIRUWKLVE\
XVLQJWKHµSDQDQGVFDQ¶WHFKQLTXHRIIHUHGE\DPRYDEOHFDPHUDEXWWKLVVLPSO\
GLGQRWZRUNIRUWZRUHDVRQVILUVWWKHFDPHUDZDVDOZD\VODJJLQJEHKLQGWKH
DFWLRQEHFDXVHRIWKHXQSUHGLFWDEOHQDWXUHRIFODVVURRPHYHQWVDQGVHFRQGWKH
WHFKQLTXHLQWURGXFHGDVWURQJVXEMHFWLYHFRPSRQHQWEHFDXVHWKHFDPHUDRSHUDWRU
ZDVIRUFHGWRFRQWLQXDOO\PDNHLQVWDQWDQHRXVGHFLVLRQVDERXWZKRRUZKDWZDV
ZRUWK\RIWKHFDPHUD¶VµSUHFLRXVIRFXV¶7KHVROXWLRQWKDWWKHWHDPDUULYHGDW
LQYROYHGXVLQJWZRFDPHUDVDPRYLQJFDPHUDZLWKDQRSHUDWRULQWKHEDFN
FRYHULQJWKHWHDFKHUDQGDVWDWLFFDPHUDLQWKHIURQWFDSWXULQJWKHIDFHVRIWKH
VWXGHQWV:KLOHWKLVDUUDQJHPHQWSURGXFHGDVDWLVIDFWRU\YLGHRFRYHUDJHWKHQHHG
IRUH[WUDHTXLSPHQWFRQVLGHUDEO\LQFUHDVHGWKHUHVHDUFKFRVWVDQGWKHRYHUDOO
XSKHDYDOLQWKHFODVVURRP(YHQWKHVXEVHTXHQWDQDO\VLVRIWKHWZRUHFRUGHG
SDUDOOHODFFRXQWVZDVQRWVWUDLJKWIRUZDUGWKHEHVWVROXWLRQWKH\IRXQGZDVWKH
µSLFWXUHLQSLFWXUH¶WHFKQLTXHIRUFRPELQLQJYLGHRLPDJHVWKH\V\QFKURQL]HGWKH
WZRWDSHVDQGWKHWHDFKHU¶VLPDJHZDVVXSHULPSRVHGLQWKHFRUQHURIWKHODUJHU
VWDWLFLPDJHRIWKHFODVVURRP
x 'LVWUDFWLRQFDXVHGE\WKHFDPHUD(YHQLQRXUDJHZKHQYLGHRFDPHUDVDUH
FRPPRQWKHSURFHVVRIYLGHRWDSLQJLQWKHFODVVURRPPD\GLVWUDFWWKHSDUWLFLSDQWV
DQGPD\HOLFLWRXWRIWKHRUGLQDU\EHKDYLRXURQWKHSDUWRIERWKWKHWHDFKHUDQGWKH
VWXGHQWV=XHQJOHUHWDO IRXQGWKDWHYHQLQWKHLUORQJLWXGLQDOVWXG\ZKHUHLW
ZDVKRSHGWKHSDUWLFLSDQWVZRXOGEHVRIDPLOLDUZLWKWKHSURFHGXUHVWKDWWKH\
ZRXOGIRUJHWDERXWWKHSUHVHQFHRIWKHUHVHDUFKHUVWKHUHZHUHUHJXODUUHIHUHQFHVLQ
WKHGDWDWRWKHIDFWWKDWWKHSDUWLFLSDQWVZHUHEHLQJYLGHRWDSHGDQG'XII LQSUHVV
UHSRUWVDVLPLODUH[SHULHQFH,QGHHGLQDQH[SHULPHQWDOVWXG\RQWKHLPSDFWRI
DQ[LHW\RQ6/$E\0DF,QW\UHDQG*DUGQHU DYLGHRFDPHUDZDV
VXFFHVVIXOO\XVHGWRLQWHQWLRQDOO\JHQHUDWHWKHDQ[LHW\FRQGLWLRQDPRQJVWWKH
SDUWLFLSDQWV<HWDV=XHQJOHUHWDODUJXHDODUJHSDUWRIFODVVURRPLQWHUDFWLRQDQG
EHKDYLRXULVEH\RQGWKHDELOLW\RIPRVWSHRSOHWRDOWHUVLJQLILFDQWO\IRUH[WHQGHG
SHULRGVRIWLPHDQGWKHUHIRUHLWLVRQO\RQHRIIUHFRUGLQJVWKDWDUHSDUWLFXODUO\
YXOQHUDEOHWRREWDLQLQJGLVWRUWHGDQGXQQDWXUDOGDWD

,QVSLWHRIWKHDERYH DQGRWKHU VKRUWFRPLQJV=XHQJOHUDQGKHUFROOHDJXHV
FRQFOXGHGWKDWWKHDGYDQWDJHVRIYLGHRUHFRUGLQJDUHFRPSHOOLQJ7KH\OLNHQHGWKH
FXUUHQWVLWXDWLRQWRZKHQWKHPLFURVFRSHZDVLQWURGXFHGWRWKHOLIHVFLHQFHVFDXVLQJD
UHYROXWLRQLQELRORJ\E\PDNLQJYLVLEOHSUHYLRXVO\XQVHHQGHWDLOV,QDVLPLODUYHLQWKH
YLGHRFDQKHOSXVXQFRYHUWKHVXEWOHUHDOLW\RIFODVVURRPOLIH)XUWKHUPRUHYLGHR
UHFRUGLQJVFDQDOVREHXVHGDVDKHOSIXODQFLOODU\WRDXGLRWDSLQJ0XULVRQ%RZLH
SHUVRQDOFRPPXQLFDWLRQ SRLQWVRXWWKHXVHIXOQHVVRIYLGHRLQGLVDPELJXDWLQJDXGLR
UHFRUGLQJVSDUWLFXODUO\LQLGHQWLI\LQJZKRLVVSHDNLQJDWDQ\RQHWLPH,QVXFKFDVHVD
IL[HGSRVLWLRQYLGHRFDPHUDLVVHWXSWRWDNHDQRYHUYLHZRIWKHFODVVJURXSDQGWKH
UHFRUGLQJLVXVHGVROHO\WRIDFLOLWDWHWKHWUDQVFULSWLRQSURFHVV2QFHWKHDXGLR
WUDQVFULSWLRQKDVEHHQFRPSOHWHGWKHYLGHRUHFRUGLQJFDQEHGLVFDUGHGZKLFKKHOSVWR
HOLPLQDWHVRPHRIWKHHWKLFDOSUREOHPVDVVRFLDWHGZLWKYLGHR

6WUHQJWKVDQGZHDNQHVVHVRIREVHUYDWLRQDOGDWD
7KHPDLQPHULWRIREVHUYDWLRQDOGDWDLVWKDWLWDOORZVUHVHDUFKHUVWRVHHGLUHFWO\ZKDW
SHRSOHGRZLWKRXWKDYLQJWRUHO\RQZKDWWKH\VD\WKH\GR7KHUHIRUHVXFKGDWDFDQ
SURYLGHDPRUHREMHFWLYHDFFRXQWRIHYHQWVDQGEHKDYLRXUVWKDQVHFRQGKDQGVHOI
UHSRUWGDWD2EVHUYDWLRQLVDOVRYHUVDWLOHDQGFDQEHXVHGZLWKSDUWLFLSDQWVZLWKZHDN
YHUEDOVNLOOV,WLVLQYDOXDEOHIRUSURYLGLQJGHVFULSWLYHFRQWH[WXDOLQIRUPDWLRQDERXWWKH
VHWWLQJRIWKHWDUJHWHGSKHQRPHQRQ
$GGLQJVWUXFWXUHWRREVHUYDWLRQE\PHDQVRIXVLQJREVHUYDWLRQVFKHPHVPDNHVWKH
SURFHVVPRUHUHOLDEOHDQGSURGXFHVUHVXOWVWKDWDUHFRPSDUDEOHDFURVVFODVVURRPVDQG
RYHUWLPH6WUXFWXUHGREVHUYDWLRQDOJXLGHOLQHVPDNHWKHIRUPLGDEOHWDVNRI
GRFXPHQWLQJWKHFRPSOH[LW\RIFODVVURRPUHDOLW\GRDEOHDQGKHOSWRIRFXVRQFHUWDLQ
NH\HYHQWVDQGSKHQRPHQD7KXVFRGLQJVFKHPHVLQWURGXFHV\VWHPDWLFLW\LQWRWKH
UHVHDUFKSURFHVV0RUHRYHUSURFHVVLQJVWUXFWXUHGREVHUYDWLRQDOGDWDLVUHODWLYHO\
VWUDLJKWIRUZDUGDQGFDQEHIXUWKHUDQDO\VHGE\PHDQVRIVWDWLVWLFDOSURFHGXUHV
2QWKHQHJDWLYHVLGHZHPXVWVWDUWZLWKWKHREYLRXVQDPHO\WKDWRQO\REVHUYDEOH
SKHQRPHQDFDQEHREVHUYHGZKHUHDVLQDSSOLHGOLQJXLVWLFVVRPDQ\RIWKHNH\
YDULDEOHVDQGSURFHVVHVWKDWUHVHDUFKHUVLQYHVWLJDWHDUHPHQWDODQGWKXVXQREVHUYDEOH
$VHFRQGSUREOHPLVWKDWUHFRUGLQJDSKHQRPHQRQGRHVQRWQHFHVVDULO\OHDGWR
XQGHUVWDQGLQJWKHUHDVRQVZK\LWKDVKDSSHQHGSDUWLFXODUO\ZKHQORZLQIHUHQFH
FDWHJRULHVDUHXVHG$PRUHWHFKQLFDOLVVXHWKDWKDVDOUHDG\EHHQGLVFXVVHGHDUOLHUZLWK
UHJDUGWRYLGHRWDSLQJDOVRDSSOLHVWRREVHUYDWLRQVLQJHQHUDOQDPHO\WKDWWKHSUHVHQFH
RIWKHLQYHVWLJDWRUFDQDIIHFWDQGELDVWKHSDUWLFLSDQWV¶EHKDYLRXU,QGHHGWKHTXDOLW\RI
REVHUYDWLRQDOGDWDLVGHSHQGHQWRQWKHVNLOOZLWKZKLFKWKHUHVHDUFKHUFRQGXFWVWKH
REVHUYDWLRQ
)LQDOO\DVHULRXVFRQFHUQZLWKVWUXFWXUHGREVHUYDWLRQLVWKDWUHJDUGOHVVRIWKH
FRGLQJFRQYHQWLRQDSSOLHGLWLQYROYHVDUHGXFWLRQRIWKHFRPSOH[LW\RIWKHREVHUYHG
VLWXDWLRQDQGE\IRFXVLQJRQWKHWDUJHWFDWHJRULHVWKHREVHUYHUFDQPLVVVRPHPRUH

LPSRUWDQWIHDWXUHV+LJKO\VWUXFWXUHGVFKHPHVDOVRVKDUHWKHJHQHUDOZHDNQHVVRI
TXDQWLWDWLYHPHDVXUHVQDPHO\WKDWWKHH[DPLQHGFDWHJRULHVDUHSUHFRQFHLYHGDQGWKH
LQVWUXPHQWLVQRWVHQVLWLYHWRFRQWH[WVSHFLILFHPHUJHQWLQIRUPDWLRQ
0L[HGPHWKRGVFODVVURRPUHVHDUFK
7KHFODVVURRPLVDKLJKO\FRPSOH[HQYLURQPHQWDQGWKHUHIRUHDV7XUQHUDQG0H\HU
VXPPDUL]HLWKDVEHHQYDULRXVO\VWXGLHGLQWHUPVRIWKHµEHOLHIVJRDOVYDOXHV
SHUFHSWLRQVEHKDYLRXUVFODVVURRPPDQDJHPHQWVRFLDOUHODWLRQVSK\VLFDOVSDFHDQG
VRFLDOHPRWLRQDODQGHYDOXDWLYHFOLPDWHVWKDWFRQWULEXWHWRWKHSDUWLFLSDQWV¶
XQGHUVWDQGLQJRIWKHFODVVURRP¶ S ,WLVFRPPRQWRGLVWLQJXLVKDWOHDVWWZREURDG
GLPHQVLRQVRIWKHFODVVURRPHQYLURQPHQWWKHµLQVWUXFWLRQDOFRQWH[W¶ZKLFKFRQFHUQV
WKHLQIOXHQFHVRIWKHWHDFKHUVWXGHQWVFXUULFXOXPOHDUQLQJWDVNVDQGWHDFKLQJPHWKRG
DPRQJVWRWKHUWKLQJVDQGWKHµVRFLDOFRQWH[W¶ZKLFKLVUHODWHGWRWKHIDFWWKDWWKH
FODVVURRPLVDOVRWKHPDLQVRFLDODUHQDIRUVWXGHQWVRIIHULQJGHHSO\LQWHQVLYHSHUVRQDO
H[SHULHQFHVVXFKDVIULHQGVKLSORYHRULGHQWLW\IRUPDWLRQ7KHVHWZRFRQWH[WVDUH
LQWHUGHSHQGHQWDQGDOVRLQWHUDFWZLWKWKHFRPSOH[SURFHVVRIOHDUQLQJUHVXOWLQJLQ
SOHQW\RIIRRGIRUWKRXJKWIRUVHYHUDOGLVFLSOLQHVDQGUHVHDUFKGLUHFWLRQVZLWKLQWKH
EURDGDUHDVRIHGXFDWLRQSV\FKRORJ\VRFLRORJ\DQGDQWKURSRORJ\
,DUJXHGLQ6HFWLRQWKDWWKHXQGHUVWDQGLQJRIWKHRSHUDWLRQRIFRPSOH[
HQYLURQPHQWV²VXFKDVFODVVURRPV²OHQGVLWVHOIWRPL[HGPHWKRGVUHVHDUFKEHFDXVH
FRPELQLQJVHYHUDOUHVHDUFKVWUDWHJLHVFDQEURDGHQWKHVFRSHRIWKHLQYHVWLJDWLRQDQG
HQULFKWKHUHVHDUFKHU¶VDELOLW\WRGUDZFRQFOXVLRQV7KLVUHFRJQLWLRQKDVEHHQHFKRHG
E\PDQ\FODVVURRPUHVHDUFKHUVLQWKHSDVW)RUH[DPSOH$OOZULJKWDQG%DLOH\
VWDWHG
)URPDOOWKDWZHKDYHVDLGRQWKHWRSLFLWVKRXOGEHFOHDUWKDWZHVHHPRVWYDOXH
LQLQYHVWLJDWLRQVWKDWFRPELQHREMHFWLYHDQGVXEMHFWLYHHOHPHQWVWKDWTXDQWLI\
RQO\ZKDWFDQEHXVHIXOO\TXDQWLILHGDQGWKDWXWLOL]HTXDOLWDWLYHGDWDFROOHFWLRQ
DQGDQDO\VLVSURFHGXUHVZKHUHYHUWKH\DUHDSSURSULDWH
0DFNH\DQG*DVV DOVRHPSKDVL]HWKHQHFHVVLW\RIµXVLQJPXOWLSOHPHWKRGV
DQGWHFKQLTXHV¶ S LQFODVVURRPUHVHDUFKDQGDFFRUGLQJWR1XQDQ
µ&ODVVURRPUHVHDUFKHUVDSSHDUWREHLQFUHDVLQJO\UHOXFWDQWWRUHVWULFWWKHPVHOYHVWRD
VLQJOHGDWDFROOHFWLRQWHFKQLTXHRUHYHQDVLQJOHUHVHDUFKSDUDGLJP¶,QDVLPLODUYHLQ
&KDXGURQ EHJLQVKLVGHWDLOHGUHYLHZRIFODVVURRPUHVHDUFKE\VWDWLQJ
,QWKHVXUYH\RI/FODVVURRPUHVHDUFKWKDWIROORZVWKHUHDGHUZLOOUHFRJQL]HD
FRQWLQXRXVJLYHDQGWDNHEHWZHHQWKHVXFFHVVHVDQGIDLOXUHVRITXDQWLWDWLYHRU
TXDOLWDWLYHDSSURDFKHVWRSRUWUD\DQGH[SODLQDGHTXDWHO\WKHSURFHVVHVDQGSURGXFWV
RIFODVVURRPLQWHUDFWLRQ
:HVKRXOGQRWHKRZHYHUWKDWLQDFWXDOSUDFWLFHPL[LQJPHWKRGVLQFODVVURRP
UHVHDUFKPD\QRWEHDQXQDPELJXRXVSURFHVVDQGWKLVKDVEHHQH[SOLFLWO\H[SUHVVHGLQ
WKHVHOIUHIOHFWLRQVRIWZRKLJKO\H[SHULHQFHGFODVVURRPHWKQRJUDSKHUV'XIIDQG(DUO\

µ«GHVSLWHSUHVFULSWLRQVDQGDVVXUDQFHVWRWKHFRQWUDU\«ERWKRXUVWXGLHV
IRXQGLWFKDOOHQJLQJWRDWWHPSWWRVWULNHDPHDQLQJIXOEDODQFHEHWZHHQTXDQWLWDWLYHDQG
TXDOLWDWLYHDSSURDFKHVWRVWXG\LQJFODVVURRPSKHQRPHQD¶
,VWDUWHGWKLVVHFWLRQUHIHUULQJWRDVWXG\E\HGXFDWLRQDOSV\FKRORJLVWV7XUQHUDQG
0H\HU OHWXVUHWXUQWRWKHLUDFFRXQWWRVHHKRZWKH\WULHGWRGHDOZLWKWKH
48$/±48$1WHQVLRQ
:HKDYHDOORZHGRXUGDWDDQGWKHFRQWUDGLFWLRQVDPRQJRXUGLIIHUHQWPHWKRGVWR
LQIRUPRXUDQDO\VHVDQGZHKDYHFKDQJHGRXUFRGLQJFDWHJRULHVDQGUHYLVLWHG
FRQFOXVLRQVEDVHGRQLQGXFWLYHUHDVRQLQJ$VDUHVXOWZHKDYHGHYHORSHGQHZ
DQGPRUHFRQWH[WXDOL]HGXQGHUVWDQGLQJVRIWKHWKHRULHVXVHGWRIUDPHRXU
TXHVWLRQV
,QWKHHQGWKHIUXLWIXOQHVVRIWKLVLWHUDWLYHSURFHVVKDVOHG7XUQHUDQG0H\HU
WRFDOOIRUPRUHPXOWLPHWKRGDSSURDFKHVWRDFKLHYLQJDGHVFULSWLRQDQGH[SODQDWLRQRI
ZKDWLVKDSSHQLQJLQFODVVURRPV$VWKH\FRQFOXGHGPHWKRGRORJLHVGLUHFWHGDWWKH
PHDVXUHPHQWRIFODVVURRPYDULDEOHVLQHGXFDWLRQDOSV\FKRORJ\KDYHEHHQPRVWO\
GHGXFWLYHDQGTXDQWLWDWLYHZLWKOLWWOHH[SORUDWLRQRIWKHKRZDQGZK\RIOHDUQLQJEDVHG
RQWKHLUH[SHULHQFHGLVFHUQLQJZKDWWKHYDULRXVFRQVWUXFWVPHDQLQDSDUWLFXODUVHWWLQJ
QHFHVVLWDWHVTXDOLWDWLYHPHWKRGVWKDWFDQXQFRYHUSDUWLFLSDQWLQWHUSUHWDWLRQV²DQ
DUJXPHQWWKDWKDVEHHQVWDWHGPRUHWKDQRQFHLQWKLVERRN
'LIILFXOWLHVDQGFKDOOHQJHVLQFODVVURRPUHVHDUFK
7KHUHLVQRGRXEWLQP\PLQGWKDW0U0XUSK\ IURPµ0XUSK\¶V/DZV¶ ZDVRULJLQDOO\
DFODVVURRPUHVHDUFKHUDQGLWZDVWKLVH[SHULHQFHZKLFKRULJLQDOO\JHQHUDWHGKLVUDWKHU
JORRP\ZRUOGYLHZ,QGHHG,IXOO\VKDUH5RVVLWHU¶V H[SHULHQFH
)DFHGZLWKGHYHORSLQJFRQVWUDLQWVUHODWHGWRQRQHTXLYDOHQWJURXSVVWXGHQWDQG
WHDFKHUSDUWLFLSDQWVGDWDFROOHFWLRQGDWDDQDO\VLVWDVNGLIIHUHQFHVDQGHWKLFDO
FRQVLGHUDWLRQVWKHWHPSWDWLRQIRUPDQ\FODVVURRPRULHQWHGUHVHDUFKHUVLQP\
SRVLWLRQPLJKWEHWRFXUWDLORUHYHQDEDQGRQWKHLUVWXG\,PDLQWDLQKRZHYHU
WKDWZKDWDUHRIWHQSHUFHLYHGDVSUREOHPVE\UHVHDUFKHUVDUHLQIDFWWKHGDLO\
UHDOLWLHVRIWKHFRQWH[WVLQZKLFKPRVWWHDFKHUVSUDFWLVH7KHOLPLWDWLRQVLQWKHVH
UHVHDUFKVHWWLQJVPD\IUXVWUDWHLQYHVWLJDWRUV«WKH\DUHKRZHYHUSDUWDQGSDUFHO
RIWKHFODVVURRPFRQWH[W
7KXVWKHPDLQSXUSRVHRIWKHIROORZLQJFDWDORJXHRIGLIILFXOWLHVDQGFKDOOHQJHVLV
WRIRUHZDUQUHVHDUFKHUVDQGWRUHDVVXUHWKHPWKDWWKHSUREOHPVWKH\PLJKWHQFRXQWHULQ
FODVVURRPUHVHDUFKGRQRWQHFHVVDULO\FRQVWLWXWHHYLGHQFHRIWKHLUWRWDOLQDGHTXDF\
UHTXLULQJDFDUHHUFKDQJH6ROHWXVORRNDWWHQSDUWLFXODUO\VDOLHQWWURXEOHVSRWV²
UHJUHWWDEO\WKLVOLVWPD\QRWEHH[KDXVWLYH
0HHWLQJGLIIHUHQWQHHGVDQGVWDQGDUGV3HUKDSVWKHJUHDWHVWFKDOOHQJHRIFRQGXFWLQJ
UHVHDUFKLQFODVVURRPVLVDV3LFD SRLQWVRXWWKHIDFWWKDWWKHUHVHDUFKHU
PXVWGHDOZLWKVWDQGDUGVIRUPHWKRGRORJLFDOULJRXULQDFRPSOH[HGXFDWLRQDO

VHWWLQJZLWKSDUWLFLSDQWVZKRKDYHLPSRUWDQWZRUNWRGRDQGKDYHUHVSRQVLELOLWLHV
DQGJRDOVWKDWPD\QRWEHFRPSDWLEOHZLWKWKRVHRIWKHUHVHDUFKHU$VVKHDVNVµ,V
LWSRVVLEOHWRGHYHORSDPHWKRGRORJ\WKDWFDQVDWLVI\WKHQHHGVDQGH[SHFWDWLRQVRI
OHDUQHUVWHDFKHUVDQGUHVHDUFKHUV"¶7KHKRQHVWDQVZHULVWKDWLWKDVEHHQGRQHLQ
WKHSDVWEXWLWLVQRWDOZD\VHDV\DQGFDQVRPHWLPHVWXUQRXWWREHRXWULJKW
XQGRDEOH
)OXLGLW\RIWKHVWXGHQWERG\)RUULJRURXVUHVHDUFKZHQHHGDZHOOGHILQHG
SDUWLFLSDQWVDPSOH7KLVVHHPLQJO\EDVLFFRQGLWLRQFDQEHVXUSULVLQJO\GLIILFXOWWR
DFKLHYHLQVFKRROVHWWLQJVLIWKHVWXG\LQYROYHVPRUHWKDQDRQHRIIFURVVVHFWLRQDO
VXUYH\5RVVLWHU IRUH[DPSOHJLYHVDGHWDLOHGDFFRXQWRIKRZKHUTXDVL
H[SHULPHQWDOGHVLJQZDVGLVUXSWHGE\VWXGHQWDWWULWLRQQHZFRPHUVVWXGHQWV
UHSHDWLQJDFRXUVHDVZHOODVSDUWLFLSDQWVZLWKGUDZLQJIURPWKHSURMHFWIRUYDULRXV
UHDVRQV7KHVHFKDQJHVQRWRQO\UHGXFHGKHUVDPSOHZLWKDFRPSOHWHGDWDVHW LH
ZLWKRXWDQ\PLVVLQJYDOXHV EXWDOVRDIIHFWHGWKHFODVVURRPG\QDPLFVRIWKH
SDUWLFLSDWLQJJURXSVDQGFDXVHGFRQVLGHUDEOHWXUPRLO'XIIDQG(DUO\ DOVR
UHSRUWYHU\VLPLODUH[SHULHQFHV
7LPHFRQVXPLQJQDWXUH5HVHDUFKFDUULHGRXWLQFODVVURRPVWDNHVXSDORWRIWLPH
QRWMXVWEHFDXVHRIWKHGDWDFROOHFWLRQSURFHGXUHV ZKLFKDUHSUHGLFWDEOH EXW
EHFDXVHRIWKHQHFHVVLW\RIPHHWLQJVFKRRODGPLQLVWUDWRUVWHDFKHUVVWXGHQWVDQG
VRPHWLPHVHYHQWKHSDUHQWVUHJXODUO\WRVHOOWKHSURMHFWWRWKHPDQGWRNHHSWKHP
HQJDJHG)XUWKHUPRUHWKHUHVHDUFKHU¶VMRELVQRWRYHUHYHQDIWHUWKHILHOGZRUNKDV
EHHQFRPSOHWHGEHFDXVHZHQHHGWRUHSRUWEDFNWRWKHSDUWLFLSDQWVZLWKWKHUHVXOWV
,QGHHG6SDGD LVULJKWZKHQVKHFRQFOXGHVWKDWµWKHWLPHFRQVXPLQJ
QDWXUHRIFODVVURRPUHVHDUFKLVQRWDPLQRUIDFWRU¶
:RUNLQJZLWKWHDFKHUV&ODVVURRPUHVHDUFKW\SLFDOO\LQYROYHVZRUNLQJZLWKD
QXPEHURIWHDFKHUVIRUDQH[WHQGHGSHULRG7KLVFDQEHWD[LQJWHDFKHUVFDQEHYHU\
EXV\DQGVWUHVVHGRXWDQGWKH\KDYHWKHLURZQGLVWLQFWLYHEHOLHIVDQGVW\OHVDVZHOO
DVSURIHVVLRQDODQGSHUVRQDODJHQGDV7KXVLWPD\QRWEHHDV\WREULQJWKHPRQ
ERDUGDQGLWLVDUHDOFKDOOHQJHLQDOPRVWHYHU\FDVHWRNHHSXSWKHLUFRPPLWPHQW
7KLVLVXQGHUVWDQGDEOHEHFDXVHOHW¶VIDFHLWWKHUHVHDUFKSURMHFWLVRIWHQDPDMRU
QXLVDQFHIRUWKHFODVVURRPSUDFWLWLRQHUVDQGLWPD\UHTXLUHJUHDWVRFLDOVNLOOVWR
NHHSWKHPPRWLYDWHG )RUDGHWDLOHGDFFRXQWRIKRZWRDFKLHYHWKLVVHH6SDGDHW
DO 5RXQGV VXPPDUL]HVWKHFRQIOLFWLQJVLWXDWLRQZHOO
5HVHDUFKHUVRIWHQGHVFULEHWKHWHDFKHUVWKH\VWXG\DVXQFRRSHUDWLYH
XQKHOSIXOLQIOH[LEOHXQWUXVWLQJDQGHYHQMXVWSODLQREVWUXFWLRQLVW+RZLVLW
WKDWUHVHDUFKHUVDUHVRPHWLPHVVHHQE\WKHVHVDPHWHDFKHUVDVEHLQJ
GHPDQGLQJLQWUXVLYHXQKHOSIXOLQIOH[LEOHDQGGRJPDWLF"
,WGRHVQRWKHOSHLWKHUWKDWWKHWXUQRYHURIWHDFKHUVLQDOPRVWHYHU\W\SHRI
VFKRROV\VWHPLVKLJKDQGHYHQLIWKHWHDFKHUVGRQRWDFWXDOO\OHDYHWKHVFKRROWKH
IUHTXHQWUHRUJDQL]DWLRQVWKDWFKDUDFWHUL]HWKHHGXFDWLRQDOV\VWHPRIPDQ\FRXQWULHV

QRZDGD\VPD\DOVRGLVUXSWWKHFRQWLQXLW\DQGGHVLJQRIWKHSURMHFW,QDGGLWLRQD
VSHFLDOVRXUFHRIGLIILFXOW\DULVHVLIZHQHHGWRLQYROYHWKHWHDFKHUVWRGHOLYHUVRPH
LQVWUXFWLRQDOWUHDWPHQWIRUXVLQWKHLUFODVVHV(YHQLIZHSUHSDUHFRPSOHWHSDFNDJHV
RIWHDFKLQJPDWHULDOVZLWKGHWDLOHGJXLGHOLQHVWRWU\WRHQVXUHDXQLIRUP
LPSOHPHQWDWLRQRIWKHPDWHULDOVWKLVPD\QRWQHFHVVDULO\KDSSHQDQGµWKHIDFW
UHPDLQVWKDWZHFDQQHYHUNQRZHYHU\WKLQJWKDWWDNHVSODFHLQDFODVVURRPGXULQJ
DQLQVWUXFWLRQDOLQWHUYHQWLRQ¶ 6SDGD
:RUNLQJZLWKVWXGHQWV(YHQLIZHKDYHWKHIXOOVXSSRUWDQGFRRSHUDWLRQRIWKH
WHDFKHUVWKLVPD\QRWPHDQWKDWDOOWKHVWXGHQWVZLOOIROORZVXLW/HDUQHUVPD\KDYH
YDU\LQJQDWLYHODQJXDJHVDQGFXOWXUHVSURILFLHQF\OHYHOVOHDUQLQJVW\OHV
PRWLYDWLRQVDQGDWWLWXGHVDQGWKHG\QDPLFVRIWKHGLIIHUHQWSDUWLFLSDWLQJOHDUQHU
JURXSVPD\DOVREHGLVVLPLODU,QIDFWLWLVKLJKO\XQOLNHO\WKDWHYHU\VWXGHQWZLOO
GRKLVKHUEHVWIRUDSURMHFWLQZKLFKWKH\KDYHOLWWOHLQWHUHVWDQGZKLFKKDVQR
GLUHFWEHDULQJRQWKHLUVFKRROJUDGHV,QDODUJHVFDOHWDVNEDVHGVWXG\IRUH[DPSOH
ZHKDYHIRXQG '|UQ\HLDQG.RUPRV WKDWWKHUHVXOWVRQO\VWDUWHGWRPDNH
VHQVHRQFHZHVHSDUDWHGWKHVWXGHQWVZKRGLGDQGZKRGLGQRWWDNHWKHWDVNV
VHULRXVO\'XIIDQG(DUO\ DOVRUHSRUWHGWKDWWHVWUHVXOWVWKDWVHHPHGLUUHJXODU
VXJJHVWLQJIRUH[DPSOHFKHDWLQJRUDSDWK\ KDGWREHDQDO\VHGVHSDUDWHO\RU
GURSSHGLQWKHLUVWXG\
8QH[SHFWHGHYHQWVDQGLQWHUUXSWLRQV$OPRVWHYHU\FODVVURRPUHVHDUFKHUKDVDVWRU\
DERXWDUULYLQJDWWKHVFKRRORQµ'GD\¶ZLWKDOOWKHWHFKQLFDOHTXLSPHQWRQO\WRILQG
RXWWKDWWKHVWXGHQWVDUHDZD\RQDFODVVWULSRUVNLLQJKROLGD\ZKLFKQRERG\KDV
UHPHPEHUHGWRWHOOWKHPDERXW2¶&RQQRUDQG6KDUNH\¶V H[SHULHQFHLV
VDGO\XQLYHUVDO
$OWKRXJKZHDQWLFLSDWHGWKDWVLFNGD\VVQRZGD\VVFKRROKROLGD\VDQG
SDUHQWWHDFKHUFRQIHUHQFHVZRXOGFURZGRXUVFKHGXOHVZHGLGQRWDQG
SHUKDSVFRXOGQRWKDYHIRUHVHHQWKHVHULHVRIZHOOPHDQLQJHYHQWVWKDW
GLVUXSWHGRXUSURFHVVDQGSURMHFW7KHVHHYHQWVRIWHQFDOOHGZLWKLQDZHHN
RUHYHQDGD\¶VQRWLFHLQFOXGHGSROLWLFLDQV¶YLVLWVVFKRROZLGHPHHWLQJV
GLVWULFW(62/PHHWLQJVJXHVWVSHDNHUVDQGSURIHVVLRQDOGHYHORSPHQW
ZRUNVKRSVWKDWWHDFKHUVZHUHHQFRXUDJHGWRDWWHQG
2EWUXVLYHUHVHDUFKHUHIIHFW2QHVRXUFHRISUREOHPVKDVJRWWRGRZLWKXVWKH
UHVHDUFKHUVZHKDYHDOUHDG\PHQWLRQHGWKHQHJDWLYHLPSDFWRIWKHµREVHUYHU
HIIHFW¶DQGUHJDUGOHVVRIKRZORZDSURILOHZHVWULYHWRNHHSZHPXVWIDFHLW
FODVVURRPUHVHDUFKHUVDUHLQWUXGHUVZKRDUHLQHYLWDEO\REWUXVLYH,WLVDUHDO
FKDOOHQJHLQPRVWVLWXDWLRQVWRILQGZD\VRIPLQLPL]LQJWKHLQWUXVLRQVRWKDW
FODVVURRPHYHQWVDUHDVQDWXUDODQGXQVWDJHGDVSRVVLEOHZKLOHZHDUHSUHVHQW
ZKLFKRIFRXUVHLVWKHSUHUHTXLVLWHIRUREWDLQLQJYDOLGGDWD
(WKLFDOFRQVLGHUDWLRQV,Q&KDSWHUZHWDONHGDERXWYDULRXVHWKLFDOLVVXHVDQGIRU
PDQ\DSSOLHGOLQJXLVWVLWLVGXULQJFODVVURRPUHVHDUFKWKDWVXFKFRQFHUQVDFWXDOO\

EHFRPHWDQJLEOH7KHGLVUXSWLRQVPHQWLRQHGDERYHPD\QRWRQO\DIIHFWQHJDWLYHO\
WKHSXULW\RIRXUGDWDEXWPRUHJHQHUDOO\WKHZKROHOHDUQLQJSURFHVV1RWRQO\
PD\VWXGHQWVQRWEHQHILWIURPSDUWLFLSDWLQJLQRXUUHVHDUFKEXWLQIDFWWKHSURMHFW
PD\KDYHKDUPIXOHIIHFWVRQWKHLUGHYHORSPHQW,QDGGLWLRQWKHUHLVWKHTXHVWLRQRI
SULYDF\DQGFRQILGHQWLDOLW\'XIIDQG(DUO\ DUHULJKWZKHQWKH\SRLQWRXW
WKDWWKHFRPPRQDGYLFHRIGLVJXLVLQJWKHLGHQWLW\RIRXUSDUWLFLSDQWVLVQRWYHU\
XVHIXOLQVLWXDWLRQVZKHUHWKHUHLVXVXDOO\MXVWRQHSULQFLSDODQGPD\EHRQO\RQHRU
WZRODQJXDJHWHDFKHUVLQWKHVFKRROWKDWLVRXUUHVHDUFKVLWH+RZFDQZHUHSRUW
GDWDIURPDERXWWKHPZLWKRXWLWEHLQJREYLRXVZKRZHDUHWDONLQJDERXW"
7HFKQLFDOGLIILFXOWLHV0XUSK\¶VXQLYHUVDOODZVWDWHVWKDWLIVRPHWKLQJFDQJRZURQJ
LWZLOO,FDQRQO\DGGWRWKLVWKDWFODVVURRPHTXLSPHQWZLOOIDLOSDUWLFXODUO\DWWKRVH
WLPHVZKHQZHKDYHIRUJRWWHQWREULQJVSDUHHTXLSPHQW%XWHYHQLIWKHHOHFWULFDO
GHYLFHVGRZRUNWKHYRLFHVLQDUHFRUGLQJFDQEHGURZQHGE\HQYLURQPHQWDOQRLVHV
IURPVRXUFHVVXFKDVFRUULGRUVWUDIILFDQGSOD\JURXQGV 5RVVLWHU
0XOWLVLWHGHVLJQ)LQDOO\'XIIDQG(DUO\ GUDZDWWHQWLRQWRWKHSRVVLEOH

GUDZEDFNVRIPXOWLVLWHGHVLJQVWKDWLVVWXGLHVWKDWLQFOXGHVHYHUDOUHVHDUFKVLWHVIRU
WKHVDNHRISDUWLFLSDQWYDULHW\7KHVHUHVHDUFKHUVKDYHIRXQGWKDWWKHGLVDGYDQWDJHV
DVVRFLDWHGZLWKWKHLQDELOLW\WRPDLQWDLQDSUHVHQFHDWDQ\RQHVLWHRYHUDORQJ
SHULRGRIWLPHDQGWKXVKDYLQJWRFRQGXFWWKHUHVHDUFKIURPDIDURUE\SUR[\PD\
H[FHHGWKHEHQHILWVRIKDYLQJDZLGHUUDQJHRIVLWXDWLRQVDQGSRRORISDUWLFLSDQWV
$FWLRQUHVHDUFK
,WLVGLIILFXOWWRGHFLGHKRZWRSUHVHQWµDFWLRQUHVHDUFK¶RQWKHRQHKDQGLQDQLGHDO
ZRUOGLWRXJKWWREHDYLDEOHUHVHDUFKDUHDZLWKDSRZHUIXOLPSDFW2QWKHRWKHUKDQG
DOWKRXJKLWLVXQGRXEWHGO\DQREOHLGHDLWMXVWGRHVQRWVHHPWRZRUNLQSUDFWLFH
UHJDUGOHVVRIFRQWLQXRXVHIIRUWVWRNHHSLWJRLQJHYHUVLQFHWKHQRWLRQZDVILUVW
LQWURGXFHGE\VRFLDOSV\FKRORJLVW.XUW/HYLQLQWKHPLGV )RUDQLQIRUPDWLYH
KLVWRULFDORYHUYLHZVHH%XUQV /HWXVH[DPLQHWKLVHQLJPD
'HILQLWLRQ
µ$FWLRQUHVHDUFK¶LVDJHQHULFWHUPIRUDIDPLO\RIUHODWHGPHWKRGVWKDWVKDUHVRPH
LPSRUWDQWFRPPRQSULQFLSOHV7KHPRVWLPSRUWDQWWHQHWFRQFHUQVWKHFORVHOLQN
EHWZHHQUHVHDUFKDQGWHDFKLQJDVZHOODVWKHUHVHDUFKHUDQGWKHWHDFKHUDFWLRQUHVHDUFK
LVFRQGXFWHGE\RULQFRRSHUDWLRQZLWKWHDFKHUVIRUWKHSXUSRVHRIJDLQLQJDEHWWHU
XQGHUVWDQGLQJRIWKHLUHGXFDWLRQDOHQYLURQPHQWDQGLPSURYLQJWKHHIIHFWLYHQHVVRI
WKHLUWHDFKLQJ7KXVWKHHQKDQFHPHQWRISUDFWLFHDQGWKHLQWURGXFWLRQRIFKDQJHLQWR
WKHVRFLDOHQWHUSULVHDUHFHQWUDOFKDUDFWHULVWLFVRIDFWLRQUHVHDUFK %XUQV
7UDGLWLRQDOO\WKHWHDFKHU±UHVHDUFKHUOLQNZDVWDNHQVRVHULRXVO\LQWKLVDUHDWKDW
RQO\UHVHDUFKGRQHE\WKHWHDFKHUKLPKHUVHOIZDVFRQVLGHUHGDFWLRQUHVHDUFKSURSHU
+RZHYHUDIWHULWZDVUHDOL]HGWKDWLWLVRIWHQXQUHDOLVWLFWRH[SHFWWHDFKHUVWRKDYHWKH
H[SHUWLVHWRFRQGXFWULJRURXVUHVHDUFKVFKRODUVVWDUWHGWRHPSKDVL]HWKHFROODERUDWLRQ

EHWZHHQWHDFKHUVDQGUHVHDUFKHUV%XUQV H[SODLQVWKDWWKLVFROODERUDWLRQFDQWDNH
VHYHUDOIRUPVIURPWKHUHVHDUFKHURZQLQJWKHSURMHFWDQGFRRSWLQJDSDUWLFLSDWLQJ
WHDFKHUWRUHDOFROODERUDWLRQZKHUHUHVHDUFKHUVDQGWHDFKHUVSDUWLFLSDWHHTXDOO\LQWKH
UHVHDUFKDJHQGD
3UREOHPDWLFLVVXHV
7KHUHLVRQHELJSUREOHPZLWKDFWLRQUHVHDUFKWKHUHLVWRROLWWOHRILW3XEOLVKHGVWXGLHV
RIDFWLRQUHVHDUFKLQDSSOLHGOLQJXLVWLFVDUHVPDOOLQQXPEHUDQGDVIDUDVP\SHUVRQDO
H[SHULHQFHLVFRQFHUQHG,DPVWLOOWRPHHWDWHDFKHUZKRKDVEHHQYROXQWDULO\LQYROYHG
LQDQDFWLRQUHVHDUFKSURMHFW6RLURQLFDOO\HYHQWKRXJKRQHRIWKHVWDWHGJRDOVRI
DFWLRQUHVHDUFKKDVEHHQWRµGHPRFUDWL]H¶UHVHDUFKWKDWLVWRRSSRVHDµSURIHVVLRQDO
H[SHUWPRGHO¶DQGDYRLGWKHSLWIDOOVRIµWKH³WRSGRZQWHFKQRORJ\WUDQVIHUPRGHO´RI
DFDGHPLFLQWHUYHQWLRQSROLF\IRUPDWLRQDQGSROLF\LPSOHPHQWDWLRQ¶ 'DYLG
WKHPRYHPHQWLVUDWKHUµWRSGRZQ¶LWVHOIZLWKSULPDULO\UHVHDUFKHUVWU\LQJWRSURPRWH
LW,QWHUHVWLQJO\%DUWHOV SURYLGHVVRPHHYLGHQFHWKDWHYHQDSSOLHGOLQJXLVWV
WKHPVHOYHVIDLOWRFRQGXFWDFWLRQUHVHDUFKLQWKHLURZQWHDFKLQJZKLFKVXJJHVWVWKH
SRVVLELOLW\RIGRXEOHVWDQGDUGV
7KHUHDUHDWOHDVWWKUHHUHDVRQVIRUDFWLRQUHVHDUFKQRWEHFRPLQJPRUHSURPLQHQW
7HDFKHUVXVXDOO\ODFN D WKHWLPH E WKHLQFHQWLYHVDQG F WKHH[SHUWLVHRU
SURIHVVLRQDOVXSSRUWWRJHWPHDQLQJIXOO\HQJDJHGZLWKUHVHDUFK/HWXVVWDUWZLWKWKH
WLPHLVVXH/DQJXDJHWHDFKHUVXVXDOO\KDYHDYHU\KLJKZRUNORDGDQGWKH\RIWHQ
FRPSODLQRIQRWKDYLQJVXIILFLHQWWLPHHYHQWRSUHSDUHIRUWKHLUFODVVHVOHWDORQHWRGR
H[WUDUHVHDUFK,WLVQRWHZRUWK\WKDWHYHQRQHRIWKHRULJLQDOSURPRWHUVRIDFWLRQ
UHVHDUFK'LFN$OOZULJKWVWDUWHGWRKDYHVHULRXVGRXEWVDERXWWKHYLDELOLW\RIDKLJK
SURILOHSURMHFWWKDWKHKDGLQLWLDWHGLQ%UD]LO
7KHFODVVURRPEDVHG6/$UHVHDUFKSURMHFWZDVFOHDUO\WDNLQJXSIDUWRRPXFK
VWDIIWLPHWREHZRUWKSXUVXLQJDQGLWZDVDOVRUHTXLULQJVWDIIWROHDUQUHVHDUFK
VNLOOVWKDWZHUHQRWOLNHO\WREHKHOSIXOLQWKHLUOLYHVDVWHDFKHUV6RLWZDV
KHDYLO\SDUDVLWLFXSRQWKHLUQRUPDOZRUNLQJOLYHVUDWKHUWKDQVXSSRUWLYHRIWKHP
«
$OOZULJKW
$QGODWHU$OOZULJKW DGGHGµ/RRNLQJDW$FWLRQ5HVHDUFKLQDFWLRQLQ
YDULRXVSDUWVRIWKHZRUOGDWWKDWWLPHLWVHHPHGWRPHLWKDGWKHVDPHSRWHQWLDOWROHDG
WREXUQRXWDVP\DFDGHPLFPRGHORIUHVHDUFK¶6XFKDSHVVLPLVWLFYLHZKDVEHHQ
HFKRHGE\PDQ\DXWKRUVZRUNLQJLQRWKHUSDUWVRIWKHZRUOG IRUH[DPSOH2¶&RQQRU
DQG6KDUNH\ WDONLQJDERXW&DQDGLDQ(6/FRQWH[WV5RVVLWHU IRU
H[DPSOHFRQFOXGHG
7KHPDMRULW\RIWHDFKHUVKDYHPLQLPDOWLPHIRUFRXUVHSUHSDUDWLRQDQGUHIOHFWLRQ
DQGHYHQOHVVRSSRUWXQLW\WRUHDGWKHMRXUQDOVRUUHVHDUFKPDQXDOVWRZKLFKWKH\
PLJKWKDYHDFFHVV:LWKRXWDFOHDUXQGHUVWDQGLQJRIWKHUHVHDUFKSURFHVVDQG

UHOHYDQWOLWHUDWXUHKRZHYHUDVSLULQJWHDFKHUUHVHDUFKHUVFKDQFHZDVWLQJWKHLU
WLPHDQGHQHUJ\RQIUXLWOHVVHQGHDYRXUV
$OWKRXJKFODVVURRPSUDFWLWLRQHUVWHQGWREHYHU\EXV\LIQRWRYHUZRUNHGP\
H[SHULHQFHLVWKDWPDQ\RIWKHPFRXOGEHPRWLYDWHGWRHQJDJHLQVRPHPHDQLQJIXO
H[SORUDWLRQDFWLYLW\HYHQVSHQGLQJVRPHRIWKHLUIUHHWLPHRQLWLIWKHUHZDVVRPH
LQVWLWXWLRQDOLQFHQWLYHIRULWLQWKHIRUPRIIRUH[DPSOHRIILFLDOUHFRJQLWLRQILQDQFLDO
UHZDUGRUUHOHDVHWLPH7KLVKRZHYHUKDUGO\HYHUKDSSHQVDQGHYHQLQFRQWH[WVZKHUH
WHDFKHUVDUHSURYLGHGZLWKLQVHUYLFHWUDLQLQJWKLVWHQGVWRLQYROYHOHFWXUHVRU
ZRUNVKRSVUDWKHUWKDQVRPHIRUPRIDFWLRQUHVHDUFK
)LQDOO\DV1XQDQ VXPPDUL]HVWKHDYHUDJHFODVVURRPSUDFWLWLRQHUPD\QRW
KDYHWKHUHVHDUFKNQRZOHGJHDQGVNLOOVWRFRQGXFWDQLQTXLU\LQDZD\WKDWZRXOGJXDUG
LWDJDLQVWWKUHDWVWRUHOLDELOLW\DQGYDOLGLW\7KHGDQJHULVWKDWHYHQLIWHDFKHUVGHFLGHWR
LQLWLDWHDQDFWLRQUHVHDUFKSURMHFWWKHFKDQFHVDUHWKDWZLWKOLWWOHEDFNJURXQG
NQRZOHGJHDQGLQVXIILFLHQWUHVHDUFKH[SHUWLVHWKH\ZLOOSURGXFHTXHVWLRQDEOHRUWULYLDO
UHVXOWV,GHDOO\WKH\ZRXOGQHHGWKHVXSSRUWRIDWUDLQHGVXSHUYLVRURUDWOHDVWDFFHVVWR
VRPHVRUWRIDµUHVHDUFKFOLQLF¶EXWUHJUHWWDEO\LQPRVWFRQWH[WVWKHVHDUHOHVVWKDQ
UHDOLVWLFZLVKHV
&RQFOXGLQJZRUGVDERXWDFWLRQUHVHDUFK
$V,VDLGDWWKHEHJLQQLQJRIWKLVVHFWLRQLWLVGLIILFXOWWRGHFLGHKRZWRSLWFKWKH
VXPPDU\RIDUHVHDUFKPRYHPHQWWKDWLVKLJKO\FRPPHQGDEOHEXWZKLFK,GRQRW
FRQVLGHUYLDEOHIRUWKHWLPHEHLQJ,IXOO\DJUHHZLWK%XUQV¶ FODLPWKDW
DFWLRQUHVHDUFKµRIIHUVDPHDQVIRUWHDFKHUVWREHFRPHDJHQWVUDWKHUWKDQUHFLSLHQWVRI
NQRZOHGJHDERXWVHFRQGODQJXDJHWHDFKLQJDQGOHDUQLQJDQGWKXVWRFRQWULEXWHWRZDUG
WKHEXLOGLQJRIHGXFDWLRQDOWKHRULHVRISUDFWLFH¶,FDQDOVRVHHZK\DFWLRQUHVHDUFKFDQ
HYRNHLGHRORJ\GULYHQSDVVLRQZHOOH[SUHVVHGE\&URRNHV
6RORQJDVUHVHDUFKLVRQO\SUHVHQWHGDVVRPHWKLQJWKDWRWKHUSHRSOH²QRW
WHDFKHUV²GRDQGVRORQJDVLWVHHPVWRWHDFKHUVWKDWUHVHDUFKUHSRUWVPXVW
QHFHVVDULO\EHZULWWHQLQDODQJXDJHWKH\GRQRWUHDGRUVSHDNZHZLOOEH
DFFRPPRGDWLQJWKHH[SORLWDWLYHSUHVVXUHVRIWKHLQVWLWXWLRQVWHDFKHUVZRUNLQ
$FWLRQDJDLQVWVXFKSUHVVXUHVFDQWDNHPDQ\IRUPV7KHFRQGXFWLQJRIDFWLRQ
UHVHDUFKDVDPHDQVRIFULWLFDOUHIOHFWLRQRQWHDFKLQJDQGRQWKHVRFLRSROLWLFDO
FRQWH[WLQZKLFKWHDFKHUVILQGWKHPVHOYHVKDVWKHSRWHQWLDOWREHDPDMRU
FRPSRQHQWLQWKHFRQWLQXLQJVWUXJJOHWRLPSURYH6/>VHFRQGODQJXDJH@WHDFKLQJ
+RZHYHULWVHHPVWREHWKHFDVHWKDWDFWLRQUHVHDUFKLVQRWIXOILOOLQJWKHVHKRSHV
EHFDXVHLQWKHFXUUHQWXQVXSSRUWLYHWHDFKLQJFOLPDWHLWLVVLPSO\QRWKDSSHQLQJ
H[WHQVLYHO\HQRXJKWRKDYHDQLPSDFW$OOZULJKWKDVFRPHWRWHUPVZLWKWKLVIDFWDQG
PRYHGRQWRVXJJHVWDPRUHWHDFKHUIULHQGO\YHUVLRQRIDFWLRQUHVHDUFKµH[SORUDWRU\
SUDFWLFH¶ 6HH$OOZULJKW$OOZULJKWDQG%DLOH\ 7KHSXUSRVHRIWKLV
DSSURDFKLVVLPLODUWRDFWLRQUHVHDUFKLQWKDWLWRIIHUVZD\VIRUWHDFKHUVWRXQGHUVWDQG
FODVVURRPHYHQWVEXWLQVWHDGRIULJRURXVUHVHDUFKPHWKRGVLWXVHVDVHTXHQFHRI

UHIOHFWLYHSHGDJRJLFDOSUDFWLFHV IRUH[DPSOHH[SHULPHQWLQJZLWKFODVVURRP
SURFHGXUHVDQGLQWHUSUHWLQJWKHRXWFRPHV WRDFKLHYHWKLVSXUSRVH$VVXFKLWIDOOV
RXWVLGHWKHUHPLWRIWKLVERRN
,QFRQFOXVLRQ,EHOLHYHWKDWLWLVIDLUWRPDNHWKHIROORZLQJWKUHHFODLPV D PDQ\LI
QRWPRVWODQJXDJHWHDFKHUVZRXOGOLNHWRJDLQDPRUHWKRURXJKXQGHUVWDQGLQJRIWKH
WHDFKLQJDQGOHDUQLQJSURFHVVDVZHOODVWKHYDULRXVFODVVURRPHYHQWV E RQHPDLQ
SXUSRVHRIDSSOLHGOLQJXLVWLFUHVHDUFKLVWRSURYLGHDQVZHUVWRTXHVWLRQVWKDWFRQFHUQ
WKHVHLVVXHVDQG F FXUUHQWO\WKHUHLVDZLGHJDSEHWZHHQWHDFKHUVDQGUHVHDUFKHUVLQ
PRVWFRXQWULHVZKLFKQHHGVWREHEULGJHG$V/LJKWERZQ VXPPDUL]HVLWLV
HVVHQWLDOIRUUHVHDUFKHUVµWRHQWHULQWRGLDORJXHZLWKFODVVURRPWHDFKHUV²QRWRQO\VR
WKDWWHDFKHUVFDQNQRZZKDWUHVHDUFKHUVDUHVD\LQJEXWDOVRVRWKDWUHVHDUFKHUVFDQ
KHDUZKDWWHDFKHUVDUHVD\LQJ¶
7KHTXHVWLRQWKHQLVKRZWRPDNHWKHWHDFKHU±UHVHDUFKHUOLQNYLDEOHDQGDFWLYH
6LPSO\FRQVLGHULQJWHDFKHUVDVSDVVLYHUHFLSLHQWVRIUHVHDUFKHUNQRZOHGJHKDVQRWEHHQ
VXFFHVVIXOLQWKHSDVWEXWQHLWKHUKDVWKHRWKHUH[WUHPHQDPHO\WU\LQJWRJHWWHDFKHUV
WRJHQHUDWHWKHLURZQNQRZOHGJHDQGWKHRULHV7KHFKDOOHQJHLVWRILQGVRPHGRDEOH
IRUPRIWHDFKHU±UHVHDUFKHUSDUWQHUVKLSEHWZHHQWKHVHWZRH[WUHPHVWKDWLVHPEUDFHG
DQGVXSSRUWHG E\HGXFDWLRQDOPDQDJHUVDQGZKLFKWHDFKHUVZLOOQRWFRQVLGHUPHUHO\
DQDGGLWLRQDOEXUGHQRQWKHLUDOUHDG\EXV\GDLO\OLYHV7KHPRVWREYLRXVSODFHWRVWDUW
ZRXOGEHLQSUHVHUYLFHDQGLQVHUYLFHWHDFKHUWUDLQLQJFRXUVHVDQGDV%XUQV
DQG0F'RQRXJK VXPPDUL]HWKHUHDUHLQGHHGVRPHKRSHIXOLQLWLDWLYHVLQWKHVH
DUHDV7KHDFWLRQUHVHDUFKSURJUDPPHGHVFULEHGE\0F'RQRXJKDPRQJVWSRVWJUDGXDWH
7HDFKLQJ$VVLVWDQWVDSSHDUVWREHSDUWLFXODUO\XVHIXODQGYLDEOH
/HWPHILQLVKWKHGLVFXVVLRQRIDFWLRQUHVHDUFKRQDSRVLWLYHQRWHZLWKDQH[WUDFW
IURP5LFKDUGV WKDWLVDWWKHKHDUWRIDQ\RSWLPLVWLFEHOLHIVWKDWWHDFKHUVFDQ
EHLQWHJUDWHGVRPHKRZLQWRWKHUHVHDUFKSURFHVV
0RVW(62/>LH()/(6/@WHDFKHUVDUHQDWXUDOUHVHDUFKHUV:H¶UHXVHGWR
ZRUNLQJRXWWKHQHHGVRIRXUVWXGHQWVHYDOXDWLQJWKHHIIHFWVRISDUWLFXODU
DSSURDFKHVVSRWWLQJWKLQJVWKDWZRUNRUGRQ¶WZRUNDQGDGMXVWLQJRXUWHDFKLQJ
DFFRUGLQJO\9HU\IHZWHDFKHUVDSSURDFKWKHLUZRUNPHFKDQLFDOO\DQGQHDUO\DOO
RIXVUHIOHFWRQZKDWZHGRLQWKHFODVVURRP

3DUW'DWDDQDO\VLV
4XDQWLWDWLYHGDWDDQDO\VLV
+DYLQJFROOHFWHGTXDQWLWDWLYHGDWDWKHQH[WVWHSLVWRDQDO\VHLWXVLQJDVHWRI
PDWKHPDWLFDOSURFHGXUHVFDOOHGµVWDWLVWLFV¶7KHVHUDQJHIURPVLPSOHGHVFULSWLYH
VWDWLVWLFVVXFKDVFDOFXODWLQJWKHPHDQ LHWKHDYHUDJH RIWKHVFRUHVLQDJURXSWR
FRPSOH[PXOWLYDULDWHSURFHGXUHVVXFKDVVWUXFWXUDOHTXDWLRQPRGHOOLQJ7KHVHOHFWLRQ
RIWKHH[DFWSURFHGXUHZLOOGHSHQGRQWKHUHVHDUFKTXHVWLRQDQGWKHW\SHRIGDWD
FROOHFWHGDQGRQFHWKHDQDO\VLVLVFRPSOHWHGWKHUHVXOWVZLOOSURYLGHLQIRUPDWLRQRQWKH
EDVLVRIZKLFKZHFDQDQVZHUWKHUHVHDUFKTXHVWLRQVDQGDFFHSWRUUHMHFWWKHUHVHDUFK
K\SRWKHVHV:HVDZLQ&KDSWHUWKDWTXDQWLWDWLYHDQDO\VLVLVPRUHVWUDLJKWIRUZDUGLQ
WKLVUHVSHFWWKDQTXDOLWDWLYHDQDO\VLVEHFDXVHKHUHWKHUHDUHZHOOGHILQHGSURFHGXUHV
JXLGHGE\XQLYHUVDOO\DFFHSWHGFDQRQVWRDGGUHVVUHVHDUFKLVVXHVDQGWKHFRPSXWHUZLOO
GRPRVWRIWKHGHWDLOGPDWKHPDWLFDOZRUNIRUXVSURGXFLQJUHODWLYHO\VWUDLJKWIRUZDUG
UHVXOWV
0RVWRIWKLVFKDSWHUZLOOEHGHYRWHGWRGHVFULELQJWKHPDLQVWDWLVWLFDOSURFHGXUHV
XVHGLQDSSOLHGOLQJXLVWLFV+RZHYHUWKHUHDUHWZRSUHUHTXLVLWHVIRUJHWWLQJGRZQWR
GDWDDQDO\VLVWKDWQHHGWREHGLVFXVVHGILUVW ZHQHHGWRVHOHFWDQGOHDUQWRXVHD
VWDWLVWLFDOSURJUDPWRUXQWKHVWDWLVWLFDOSURFHGXUHVDQG WKHGDWDQHHGWREHSUHSDUHG
IRUDQDO\VLV
&RPSXWHUL]HGGDWDDQDO\VLVDQG6366
,QWKHHDUO\GD\VPRVWVWDWLVWLFVZHUHFDOFXODWHGPDQXDOO\RQSDSHUILOOLQJUHDPVRI
SDSHUZLWKHTXDWLRQVZLWKRQO\VLPSOHFDOFXODWRUVWRKHOSZLWKEDVLFDULWKPHWLF7KLV
VLWXDWLRQKDVFRPSOHWHO\FKDQJHG1RZDGD\VWKHPDWKHPDWLFDODVSHFWVRITXDQWLWDWLYH
GDWDDQDO\VLVDUHODUJHO\OHIWWRWKHFRPSXWHUDQGVHYHUDOVWDWLVWLFDOSURJUDPVKDYHEHHQ
GHYHORSHGWRGLUHFWWKHFRPSXWHU¶VZRUN7KHVHVRIWZDUHSDFNDJHVKDYHEHFRPH
LQFUHDVLQJO\XVHUIULHQGO\WRWKHH[WHQWWKDWUHVHDUFKHUVGRQRWQHHGWRKDYHDQ\
PDWKHPDWLFDOWUDLQLQJWRXVHWKHPDQGPRVWRIWKHSURFHGXUHVDUHIXOO\LQWHUDFWLYH LH
ZHGRQRWHYHQKDYHWRZULWHFRPPDQGVEXWRQO\FKRRVHRSWLRQVWKDWVXLWRXUSXUSRVHV
IURPUHDG\PDGHPHQXV :KDWZHGRQHHGLVDQXQGHUVWDQGLQJRIWKHSXUSRVHRIWKH
YDULRXVVWDWLVWLFDOSURFHGXUHVVRWKDWZHFDQVHOHFWWKHDSSURSULDWHSURFHGXUHIRU
DQDO\VLQJWKHGDWDDQGWKHQLQVWUXFWWKHFRPSXWHUWRUXQWKHDQDO\VLV
7KHUHDUHPDQ\VWDWLVWLFDOSDFNDJHVRQWKHPDUNHWDQGHYHQVSUHDGVKHHWVVXFKDV
0LFURVRIW¶V([FHOFDQGRDIDLUDPRXQWRIVWDWLVWLFV+RZHYHUP\UHFRPPHQGDWLRQLV

WKDWQRYLFHUHVHDUFKHUVVWDUWZRUNLQJZLWKRQHRIWKHPDMRUVRIWZDUHSDFNDJHVULJKW
IURPWKHEHJLQQLQJ&RQVLGHUDEOHHIIRUWVKDYHEHHQPDGHWRPDNHWKHVHFRPSXWHU
SURJUDPVXVHUIULHQGO\VRWKHLUXVHLVKDUGO\PRUHGLIILFXOWWKDQWKDWRIWKHVPDOOHU
SDFNDJHVDQGWKHUDQJHRIDQDO\VHVWKH\DUHFDSDEOHRIUXQQLQJLVLQFRPSDUDEO\
VXSHULRU7KHTXHVWLRQWKHQLVZKLFKSURJUDPVKDOOZHFKRRVH"7KHVRIWZDUHSDFNDJH
PRVWFRPPRQO\XVHGLQDSSOLHGOLQJXLVWLFDQGHGXFDWLRQDOUHVHDUFKLVµ6366¶
6WDWLVWLFDO3DFNDJHIRUWKH6RFLDO6FLHQFHV SURGXFHGE\6366,QFLQ&KLFDJR86$
DQGLWPDNHVVHQVHWRDGRSWWKLVSURJUDP7KHUHIRUHDOOWKHVSHFLILFDGYLFHRQ
VWDWLVWLFDODSSOLFDWLRQVLQWKLVFKDSWHUZLOOFRQFHUQ6366,WVKRXOGEHDEOHWRSHUIRUP
DOOWKHVWDWLVWLFDORSHUDWLRQVWKDWRQHQHHGVDWWKLVVWDJHDQGLI\RXUUHVHDUFKUHTXLUHV
IXUWKHUPRUHVSHFLDOL]HGVWDWLVWLFDODQDO\VHVDWDODWHUVWDJH IRUH[DPSOHVWUXFWXUDO
HTXDWLRQPRGHOOLQJ \RXFDQILQGDSSURSULDWHVRIWZDUHWKDWLVFRPSDWLEOHZLWK6366
6366LVQRWDEHJLQQHUV¶VRIWZDUHSDFNDJH\HWLWLVDSSURSULDWHIRUSHRSOHZKRKDYH
OLWWOHH[SHULHQFHLQVWDWLVWLFVDQGSURJUDPPLQJEHFDXVHRILWVXVHUIULHQGO\LQWHUDFWLYH
IHDWXUHV,WKDVEHHQDURXQGIRURYHU\HDUVDQGWKHUHIRUHWKHFXUUHQWYHUVLRQ DWWKH
WLPHRIZULWLQJWKLV9HUVLRQ LVKLJKO\UHILQHGDQGE\DQGODUJHEXJIUHH6366LV
HDV\WRLQVWDOODQGVWDUW,I\RXGHFLGHWRXVHLWLQ\RXUUHVHDUFK\RXZLOOQHHGVRPHVRUW
RIDJXLGHRUPDQXDOWRKHOSWRUXQWKHVRIWZDUHDOWKRXJKRQHFDQJRDORQJZD\E\
VLPSO\UHO\LQJRQWKHVRIWZDUH¶VRZQµ+HOS¶V\VWHP)XUWKHUPRUHEHFDXVH6366LV
EHLQJWDXJKWDWVRPDQ\XQLYHUVLWLHVHYHQDTXLFNVHDUFKRQWKH,QWHUQHWZLOOUHVXOWLQD
QXPEHURIYHU\XVHIXORQOLQHWXWRULDOVDQGXVHUIULHQGO\RYHUYLHZV%HFDXVHRIWKH
ZLGHVSUHDGXVHRI6366PRVWRIWKHVHULRXVDFDGHPLFSXEOLVKHUVRIIHUWKHLURZQµ+RZ
WR«¶PDQXDOV²,ZRXOGVXJJHVWFKRRVLQJRQHZKLFK D GRHVQRWLQFOXGHDQ\
PDWKHPDWLFDOHTXDWLRQV E DOVRRIIHUVVWDWLVWLFDOH[SODQDWLRQVEHVLGHVWKHLQVWUXFWLRQV
RQKRZWRUXQWKHVRIWZDUH F KDVVWHSE\VWHSJXLGHOLQHVDQGH[WHQVLYHLOOXVWUDWLRQV
DQG G RIIHUVWHPSODWHVRIKRZWRSUHVHQWWKHUHVXOWV ,KDYHIRXQG3DOODQWYHU\
XVHIXO
3UHSDULQJWKHGDWDIRUDQDO\VLV
$V%URZQ ULJKWO\SRLQWVRXWKDYLQJFROOHFWHGRXUGDWDLVKDOIWKHEDWWOHDQG
QRZZHPXVWDGGUHVVWKHRWKHUKDOIWKHSURFHVVLQJRIWKHVWDFNVRIFRPSOHWHG
TXHVWLRQQDLUHVRUWHVWVRUPXOWLSOHSDJHVRIRXUQRWHVRIYDULRXVTXDQWLWDWLYHVFRUHV
'DYLGVRQ L[ UHPDUNVWKDWLWLVVXUSULVLQJKRZIHZRIWKHVWDWLVWLFVWH[WVGHVFULEH
WKHµSDLQIXOH[SHULHQFHRIJRLQJIURPDVWDFNRIGLVRUJDQL]HGKDUGFRS\WRRQOLQHGDWD
WKDWDUHWUXVWZRUWK\¶7KLVLVTXLWHWUXHVROHWXVVSHQGVRPHWLPHJRLQJWKURXJKWKH
LQLWLDOSURFHGXUHVWKDWDUHXVXDOO\UHIHUUHGWRDVµGDWDSUHSDUDWLRQ¶
7KHILUVWSULQFLSOHRISUHSDULQJRXUGDWDIRUDQDO\VLVLVWKDWTXDQWLWDWLYHGDWDQHHGWR
EHVWRUHGLQDFRPSXWHUILOH7KLVUHTXLUHVWKHV\VWHPDWLFFRGLQJRIWKHGDWDFUHDWLQJD
GDWDILOHDQGWKHQLQSXWWLQJWKHFRGHGGDWD2QFHZHKDYHRXUGDWDRQOLQHZHQHHGWR
VFUHHQFOHDQDQGSRVVLEO\PDQLSXODWHLWEHIRUHLWLVUHDG\IRUDQDO\VLV

&RGLQJTXDQWLWDWLYHGDWD
4XDQWLWDWLYHGDWDDQDO\VLVVRIWZDUHKDQGOHVGDWDLQDQXPHULFDOUDWKHUWKDQDQ
DOSKDEHWLFIRUPDQGHYHQWKRXJK6366DOORZVWKHVWRUDJHRILQIRUPDWLRQUHFRUGHGDV
OHWWHUV IRUH[DPSOHZRUGVRUSURSHUQDPHV WKHSURFHGXUHVWKDWDUHDYDLODEOHIRU
KDQGOLQJVXFKWH[WXDOGDWDDUHOLPLWHGFRPSDUHGWRWKHYDVWDUVHQDORIVWDWLVWLFDO
WHFKQLTXHVWREHXVHGZLWKQXPHULFDOUHVSRQVHV7KHUHIRUHWKHILUVWVWHSRIGDWD
SURFHVVLQJLQYROYHVFRQYHUWLQJWKHUHVSRQGHQWV¶DQVZHUVWRQXPEHUVE\PHDQVRI
µFRGLQJSURFHGXUHV¶%HFDXVHQXPEHUVDUHPHDQLQJOHVVLQWKHPVHOYHVDQGDUHDOOWRR
HDV\WRPL[XSDPDMRUHOHPHQWRIWKHFRGLQJSKDVHLVWRGHILQHHDFKYDULDEOHDQGWKHQ
WRFRPSLOHFRGLQJVSHFLILFDWLRQVIRUHYHU\SRVVLEOHµYDOXH¶WKDWWKHSDUWLFXODUYDULDEOH
FDQWDNH :HVKRXOGQRWHKHUHWKDWµYDOXH¶LVDWHFKQLFDOWHUPXVHGLQVWDWLVWLFV
UHIHUULQJWRWKHQXPEHUVDVVLJQHGWRWKHUHVSRQVHRSWLRQVRIWKHYDULDEOH )RUH[DPSOH
JHQGHUGDWDLVXVXDOO\ODEHOOHGµVH[¶DQGLWFDQWDNHWZRQXPHULFDOYDOXHVµPDOH¶LV
XVXDOO\FRGHGµ¶DQGµIHPDOH¶µ¶
:LWKQXPHULFDOYDULDEOHVVXFKDVWHVWVFRUHVWKHFRGLQJLVVLPSOHEHFDXVHWKHYDOXH
UDQJHRIWKHYDULDEOHZLOOEHWKHVDPHDVWKHSRVVLEOHVFRUHVIRUWKHWHVW IRUH[DPSOH
WKHYDOXHRIWKHYDULDEOHµ72()/¶IRUDSHUVRQLVVD\ :LWKFORVHGHQGHG
TXHVWLRQQDLUHLWHPVVXFKDV/LNHUWVFDOHV VHH6HFWLRQ WKHFRGLQJIUDPHLV
VLPLODUO\VWUDLJKWIRUZDUGHDFKSUHGHWHUPLQHGUHVSRQVHRSWLRQLVDVVLJQHGDQXPEHU
IRUH[DPSOHµVWURQJO\GLVDJUHH¶ µGLVDJUHH¶ µQHXWUDO¶ µDJUHH¶
µVWURQJO\DJUHH¶ )RUVLPSOHRSHQHQGHGTXHVWLRQQDLUHLWHPV IRUH[DPSOHD
EDFNJURXQGLQIRUPDWLRQTXHVWLRQVXFKDVµ:KDWIRUHLJQODQJXDJHVKDYH\RXOHDUQHGLQ
WKHSDVW"¶ WKHFRGLQJIUDPHLVPRUHFRPSOH[EHFDXVHLWFDQKDYHPDQ\FDWHJRULHV IRU
H[DPSOH*HUPDQ )UHQFK HWF LQIDFWDVPDQ\DVWKHQXPEHURIWKHGLIIHUHQW
DQVZHUVLQDOOWKHTXHVWLRQQDLUHV7KXVZLWKVXFKLWHPVWKHFRGLQJIUDPHLV
FRQWLQXRXVO\H[WHQGHGGXULQJWKHSURFHVVLQJRIWKHGDWDZLWKHYHU\QHZODQJXDJH
PHQWLRQHGE\WKHUHVSRQGHQWVEHLQJDVVLJQHGDQHZQXPEHU
7KHFRGLQJRIRWKHURSHQHQGHGTXHVWLRQQDLUHLWHPVWKDWHOLFLWPRUHGLYHUVHRU
ORQJHUUHVSRQVHVKRZHYHUPD\JREH\RQGWKHPHFKDQLFDOFRQYHUVLRQRIDFDWHJRU\
LQWRDQXPEHUDQGPD\UHTXLUHDFHUWDLQDPRXQWRIVXEMHFWLYHLQWHUSUHWDWLRQDQG
VXPPDU\RQWKHSDUWRIWKHFRGHU)RUH[DPSOHZLWKDTXHVWLRQVXFKDVµ:KDWLV\RXU
IDYRXULWHOHLVXUHDFWLYLW\"¶WKHWDVNLVWRFRQGHQVHWKHGLYHUVHLQIRUPDWLRQFRQWDLQHGLQ
WKHUHVSRQVHVLQWRDOLPLWHGQXPEHURIFDWHJRULHVRQJRLQJGHFLVLRQVZLOOQHHGWREH
PDGHDERXWZKHWKHUWRODEHOWZRVLPLODUEXWQRWFRPSOHWHO\LGHQWLFDOUHVSRQVHV IRU
H[DPSOHµZDONLQJWKHGRJ¶DQGµJRLQJIRUDZDON¶ DVWKHVDPHRUZKHWKHUWRPDUNWKH
GLIIHUHQFHVRPHKRZ
,QWKHSDVWUHVHDUFKHUVKDYHKDGWRFRPSLOHDµFRGHERRN¶WKDWLQFOXGHGWKHFRGLQJ
IUDPHVIRUHDFKYDULDEOHLQWKHLULQYHVWLJDWLRQ+RZHYHUWKHGDWDILOHVLQWKHODWHVW
YHUVLRQVRI6366RIIHUVXFKFRQYHQLHQWIDFLOLWLHVWRLQFOXGHGHWDLOHGGHVFULSWLRQVRIDOO
WKHYDULDEOHVDQGYDOXHVWKDWPDQ\UHVHDUFKHUVQRZDGD\VVLPSO\LQFRUSRUDWHWKH
FRGHERRNLQWRWKHGDWDILOH7KLVSUDFWLFHLVIXUWKHUVXSSRUWHGE\WKHIDFWWKDWVWDUWLQJ

ZLWK9HUVLRQYDULDEOHQDPHVFDQEHDVORQJDVFKDUDFWHUVDOORZLQJIRU
PHDQLQJIXOGHVFULSWRUV ZKHUHDVLQWKHHDUOLHUYHUVLRQVFKDUDFWHUVZHUHWKHPD[LPXP
ZKLFKRIWHQIRUFHGXVWRSURGXFHVWUDQJHDEEUHYLDWLRQVIRUWKHYDULDEOHQDPHV
,QSXWWLQJWKHGDWD
(QWHULQJWKHGDWDLQWRDFRPSXWHUILOHUHTXLUHVWKUHHVWHSV &UHDWLQJWKHGDWDILOH
GHILQLQJWKHFRGLQJIUDPHVIRUWKHYDULDEOHVDQG NH\LQJLQWKHGDWD
&UHDWLQJDQGQDPLQJWKHGDWDILOH
&UHDWLQJDQHZGDWDILOHLVHDV\LQPRGHUQVWDWLVWLFDOVRIWZDUHVXFKDV6366+RZHYHU
WKHQDWXUHRIGDWDILOHVLVVXFKWKDWZHUHJXODUO\PDNHFKDQJHVLQWKHPGXULQJWKH
SURFHVVRIGDWDDQDO\VLV IRUH[DPSOHUHFRGHYDULDEOHVRUFRPSXWHQHZFRPSRVLWH
YDULDEOHV DQGRQHRIWKHUHFXUULQJSUREOHPV,KDYHHQFRXQWHUHGZLWKQRYLFH DQGQRW
VRQRYLFH UHVHDUFKHUVLVWKDWPXOWLSOHYHUVLRQVRIWKHGDWDILOHVDUHVWRUHGQH[WWRHDFK
RWKHUZLWKWKHILOHQDPHVEHFRPLQJFRQIXVLQJDQGWKHILOHVJHWWLQJPL[HGXS
)XUWKHUPRUHLQPRVWVWXGLHVZHDUHOLNHO\WRUHWXUQWRWKHGDWDVHWDWODWHUVWDJHVDQG
ZLWKRXWDIRROSURRIQDPLQJODEHOOLQJV\VWHPDQGDSURSHUO\PDLQWDLQHGUHVHDUFKORJ
VHH6HFWLRQ LWPLJKWEHDGDXQWLQJWDVNWRVRUWRXWZKLFKILOHLVZKDW7KHUHIRUH
LWLVZRUWKDGRSWLQJWKHIROORZLQJILOLQJSURFHVV WKRXJKSOHDVHFRQVXOWDQ6366JXLGH
RQKRZH[DFWO\WRFDUU\RXWWKHIROORZLQJVWHSV
*LYHWKHGDWDILOHDVLPSOHJHQHULFQDPH IRUH[DPSOHµ0RWLYDWLRQBGDWD¶ WRZKLFK

6366ZLOODGGWKHH[WHQVLRQµVDY¶ZKHQHYHU\RXPDNHDQ\FKDQJHVLQWKHGDWDILOH
VDYHWKHPRGLILHGILOHXQGHUDQHZYHUVLRQQDPH IRUH[DPSOH
µ0RWLYDWLRQBGDWDVDY¶µ0RWLYDWLRQBGDWDVDY¶HWF 0\UXOHRIWKXPELVWKDWQR
FKDQJHLQWKHGDWDILOHVKRXOGEHVDYHGXQGHUWKHROGILOHQDPHEHFDXVHWKLVZRXOG
PDNHLWYHU\GLIILFXOW LIQRWLPSRVVLEOH DWDODWHUVWDJHWRWUDFHEDFNZKDWZHKDYH
GRQH
7KHILUVWYHUVLRQRIWKHGDWDILOH µ0RWLYDWLRQBGDWDVDY¶ VKRXOGFRQWDLQWKHUDZ
GDWDZLWKRXWDQ\PRGLILFDWLRQV
:KHQZHPDNHDQ\FKDQJHLQWKHGDWDILOHZHQHHGWRKDYHDSHUPDQHQWUHFRUGRI
WKLVFKDQJHLQWKHIRUPRIDµFRPPDQGILOH¶ZKLFKLQ6366LVFDOOHGDµV\QWD[ILOH¶
DQGFDUULHVWKHH[WHQVLRQRIµVSV¶6XFKV\QWD[ILOHVFDQEHIRUPHGLQWKHLQWHUDFWLYH
PRGHE\XVLQJWKHµ3$67(¶FRPPDQG
1DPHWKHV\QWD[ILOHWKDWFUHDWHVDQHZYHUVLRQRIWKHGDWDILOHLQDZD\WKDWLW
FRUUHVSRQGVWRWKHQHZO\FUHDWHGGDWDILOH¶VQDPH)RUH[DPSOHWKHILOHWKDWFUHDWHV
µ0RWLYDWLRQBGDWDVDY¶VKRXOGEHQDPHGµ0RWLYDWLRQBGDWDVSV¶
,I\RXIROORZWKLVSURFHGXUHFRQVLVWHQWO\\RXZLOODOZD\VNQRZZKLFKLVWKHODWHVW
YHUVLRQRIWKHGDWDILOH²WKHRQHZLWKWKHKLJKHVWYHUVLRQQXPEHU²DQG\RXFDQ
DOZD\VWUDFHEDFNZKDWKDVFKDQJHGIURPRQHYHUVLRQWRWKHRWKHUE\ORRNLQJDWWKH
VHULHVRIFRUUHVSRQGLQJV\QWD[ILOHV7DEOHRIIHUVDQH[DPSOHRIWKHSURFHVV

,QLWLDOGDWDILOHFRQWDLQLQJWKHUDZGDWD 0RWLYDWLRQBGDWDVDY
6\QWD[ILOHFUHDWLQJDQHZYHUVLRQRIWKHGDWDILOH 0RWLYDWLRQBGDWDVSV
([DPSOHRI0RWLYDWLRQBGDWDVSV
*(7),/( &?0\'RFXPHQWV?0RWLYDWLRQBGDWDVDY
5(&2'(TXHVWTXHVWTXHVW
6$9(287),/( &?0\'RFXPHQWV?0RWLYDWLRQBGDWDVDY
&2035(66('
7KLVV\QWD[ILOHILUVWRSHQVWKHODVWYHUVLRQRIWKHGDWDILOHWKHQUHFRGHVWKHYDOXHVRIWKUHH
YDULDEOHV²TXHVWTXHVWDQGTXHVW²DQGILQDOO\VDYHVWKHQHZPRGLILHGGDWDILOHXQGHU
WKHFRQVHFXWLYHYHUVLRQQXPEHU
7DEOH([DPSOHRIFUHDWLQJDQHZYHUVLRQRIDQ6366GDWDILOH
'HILQLQJWKHFRGLQJIUDPHV
:KHQWKHGDWDILOHKDVEHHQFUHDWHGDQGSURSHUO\QDPHGZHQHHGWRGHFLGHRQWKH
µFRGLQJIUDPHV¶IRUDOOWKHYDULDEOHV7KH6366'DWD(GLWRUVFUHHQKDVDµGDWD¶DQGD
µYDULDEOH¶YLHZ LQGLFDWHGLQWKHERWWRPOHIWFRUQHU ,QWKHµYDULDEOH¶YLHZZHPXVW
ILUVWVSHFLI\WKHQDPHRIHDFKYDULDEOH WKHEHVWWKLQJLVWRFKRRVHRQHZRUGXSWR
FKDUDFWHUVIRUHDFKYDULDEOHEXWWKHVKRUWHUWKHQDPHWKHEHWWHU DQGZHPD\DOVRZDQW
WRFUHDWHDµYDULDEOHODEHO¶ WKLVLVRSWLRQDO WRDFFRPSDQ\HDFKYDULDEOHQDPH²WKLVLVD
ORQJHUGHILQLWLRQRUGHVFULSWLRQ IRUH[DPSOHWKHZKROHTXHVWLRQQDLUHLWHPWKDW
JHQHUDWHGWKHSDUWLFXODUYDULDEOH ,ZRXOGVXJJHVWWKDWDIWHUVSHFLI\LQJWKHYDULDEOHVZH
FUHDWHµYDOXHODEHOV¶IRUHDFKYDULDEOH WKLVDJDLQLVRSWLRQDO WRGHVFULEHWKHSRVVLEOH
QXPEHUVWKDWWKHYDULDEOHFDQWDNHDQGZKDWHDFKQXPEHUVWDQGVIRU3URSHUODEHOOLQJ
ZLOOKHOSXVWRUHPHPEHUWKHGHWDLOVZKHQZHFRPHEDFNWRWKHGDWDILOHDWDODWHUVWDJH
.H\LQJLQWKHGDWD
.H\LQJLQWKHGDWDLVDUDWKHUWHGLRXVDQGWLPHFRQVXPLQJSURFHVV$OORIXVLQYROYHGLQ
TXDQWLWDWLYHUHVHDUFKKDYHVSHQWFRXQWOHVVKRXUVLQIURQWRIDFRPSXWHUVFUHHQW\SLQJ
VHHPLQJO\HQGOHVVURZVRIILJXUHV+RZHYHUERULQJDQGPHFKDQLFDOWKLVMREPD\EHLW
LVHVVHQWLDOWKDWZHPDLQWDLQFRQFHQWUDWLRQEHFDXVHHYHU\PLVW\SHGQXPEHUZLOOEHD
FRQWDPLQDWLRQRIWKHGDWDVHW,QDJUHHPHQWZLWK%URZQ ,KDYHIRXQGWKDWRQH
ZD\RIPDNLQJWKHWDVNPRUHLQWHUHVWLQJDQGPRUHDFFXUDWHLVWRZRUNZLWKDSDUWQHU
WDNLQJWXUQVDWGLFWDWLQJDQGW\SLQJ7KH6366'DWD(GLWRUVFUHHQRIIHUVDQXPEHURI
XVHUIULHQGO\VKRUWFXWVDQGRSWLRQV IRUH[DPSOHFXWWLQJDQGSDVWLQJ WRKHOSWKHGDWD
HQWU\SURFHVV²SOHDVHFRQVXOWWKHPDQXDO

:KHQZHFRPHDFURVVPLVVLQJGDWD²IRUH[DPSOHZHGRQRWKDYHWKHSURILFLHQF\
WHVWVFRUHIRUDSHUVRQEHFDXVHKHVKHZDVDEVHQWZKHQWKHWHVWZDVDGPLQLVWHUHG²WKH
EHVWRSWLRQLVWROHDYHWKHSDUWLFXODUFHOOHPSW\ UDWKHUWKDQIROORZLQJWKHWUDGLWLRQDO
SUDFWLFHRIW\SLQJLQDVHWQXPEHUVXFKDVµ¶ 7KLVHQVXUHVWKDWZHGRQRWPL[XSUHDO
YDOXHVZLWKPLVVLQJGDWDYDOXHVE\DFFLGHQWDQGWKHUHZLOOEHRSSRUWXQLWLHVODWHUIRU
HDFKVWDWLVWLFDODQDO\VLVWRGHFLGHKRZWRKDQGOHWKHPLVVLQJGDWD
'DWDVFUHHQLQJDQGFOHDQLQJ
,QP\H[SHULHQFHWKHLQLWLDOGDWDILOHZLOODOZD\VFRQWDLQPLVWDNHV6RPHRIWKHVHDUH
WKHUHVXOWRIKXPDQHUURURFFXUULQJGXULQJWKHGDWDHQWU\SKDVH IRUH[DPSOHW\SLQJWKH
ZURQJQXPEHU DQGVRPHDUHPLVWDNHVPDGHE\WKHUHVSRQGHQWVZKHQILOOLQJLQWKH
TXHVWLRQQDLUHµ'DWDVFUHHQLQJDQGFOHDQLQJ¶LQYROYHVVSRWWLQJDQGFRUUHFWLQJDVPDQ\
RIWKHVHHUURUVDQGLQDFFXUDFLHVDVSRVVLEOHEHIRUHVWDUWLQJWKHDFWXDODQDO\VHV7KLVLV
DQLQGLVSHQVDEOHSKDVHRISUHSDULQJWKHGDWDEHFDXVHVRPHPLVWDNHVFDQFRPSOHWHO\
GLVWRUWRXUUHVXOWV7KHPDLQVFUHHQLQJWHFKQLTXHVDUHDVIROORZV
x &RUUHFWLQJLPSRVVLEOHGDWD0RVWLWHPVKDYHDVSHFLILFUDQJHGHWHUPLQHGE\WKH
JLYHQUHVSRQVHRSWLRQVRUE\WKHLQKHUHQWORJLFRIWKHLWHP$TXLFNIUHTXHQF\
OLVWLQJRIDOOWKHLWHPV LQ6366$1$/<=(ĺ'(6&5,37,9(67$7,67,&6ĺ
)5(48(1&,(6 FDQH[SRVHRXWRIUDQJHYDOXHVIRUH[DPSOHZLWKDVL[SRLQW
/LNHUWVFDOHDYDOXHRILVREYLRXVO\LQFRUUHFWDQGLIVRPHRQH¶VDJHLVHQWHUHGDV
ZHFDQDOVRVXVSHFWKXPDQHUURU2QFHVXFKLPSRVVLEOHYDOXHVKDYHEHHQ
GHWHFWHGZHQHHGWRFKHFNWKHKDUGFRS\RIWKHSDUWLFXODUTXHVWLRQQDLUHRUUHFRUG
VKHHWDQGWKHQHQWHUWKHFRUUHFWYDOXHV
x &RUUHFWLQJLQFRUUHFWO\HQWHUHGYDOXHVWKDWFRQIRUPWRWKHSHUPLVVLEOHUDQJH,WLV
HDV\WRGHWHFWDQGFRUUHFWDYDOXHRIZLWKDVL[SRLQW/LNHUWVFDOH%XWZKDWDERXW
DW\SLQJHUURULQWKHVDPHVFDOHZKHQµ¶KDVEHHQUHFRUGHGLQVWHDGRIµ¶"7KH
RQO\ZD\RIGHWHFWLQJVXFKDPLVWDNHLVE\PHDQVRIDYHU\ODERULRXVSURFHGXUH
ZKHUHE\WKHHQWLUHGDWDEDQNLVUHHQWHUHGLQDVHFRQGGDWDILOHDQGWKHQWKHWZR
GDWDILOHV ZKLFKRXJKWWREHLGHQWLFDO DUHFRPSXWHUFKHFNHGIRUFRUUHVSRQGHQFH
ZLWKHDFKRWKHU
x &RUUHFWLQJFRQWUDGLFWLQJGDWD6RPHTXHVWLRQQDLUHVKDYHµURXWHG¶LWHPVZKLFK
PHDQVWKDWVRPHTXHVWLRQVDUHWREHDQVZHUHGRQO\LIWKHUHVSRQGHQWJDYHD
SDUWLFXODUDQVZHUWRDSUHYLRXVTXHVWLRQ)RUH[DPSOHLIDODQJXDJHOHDUQHUJLYHVD
QHJDWLYHDQVZHUWRWKHTXHVWLRQµ+DYH\RXHYHUVWD\HGLQWKH/HQYLURQPHQWIRU
DQH[WHQGHGSHULRG"¶DQGWKHQDQVZHUVµPRQWKV¶WRWKHVXEVHTXHQWTXHVWLRQµ,I
VRIRUKRZORQJ"¶VRPHWKLQJLVREYLRXVO\ZURQJ'HSHQGLQJRQWKHW\SHRI
TXHVWLRQVDVNHGVHYHUDORWKHUORJLFDOFKHFNVDUHDOVRFRQFHLYDEOH,QDQ\FDVH
ZKHQVXFKLQFRQVLVWHQFLHVDUHIRXQGDFORVHULQVSHFWLRQRIWKHTXHVWLRQQDLUHFDQ
XVXDOO\KHOSWRUHPHG\WKHVHEXWVRPHWLPHVWKHRQO\ZD\RIJHWWLQJULGRIWKH
FRQWDPLQDWLRQLVWRHOLPLQDWHERWKSDUWVRIWKHFRQWUDGLFWLQJRULOORJLFDO
FRPELQDWLRQ

'HDOLQJZLWKRXWOLHUV
7KHGDWDFKHFNPD\KLJKOLJKWYDOXHVWKDWDUHLQFRQVLVWHQWZLWKWKHUHVWRIWKHGDWDVHW
IRUH[DPSOHEHFDXVHWKH\DUHZD\RXWRIWKHXVXDOUDQJH IRUH[DPSOHWKHVL]HRIWKH
VSHHFKSURGXFHGE\DODQJXDJHOHDUQHULQDWDVNLVWZLFHDVELJDVWKHVHFRQGODUJHVW
VFRUH 7KHVHVXVSLFLRXVYDOXHVDUHUHIHUUHGWRDVRXWOLHUVDQGWKH\DUHRIWHQWKHUHVXOW
RILQFRUUHFWGDWDHQWU\7KHSUREOHPZLWKRXWOLHUVLVWKDWWKH\FDQVHULRXVO\GLVWRUWWKH
VWDWLVWLFDOUHVXOWVE\KDYLQJDQXQGXHLPSDFWRQWKHRXWFRPH7KHUHIRUHDQLPSRUWDQW
DVSHFWRIGDWDVFUHHQLQJFOHDQLQJLVWRFKHFNWKHGDWDIRURXWOLHUVDQGWKHQGHDOZLWKWKH
LGHQWLILHGH[WUHPHFDVHV LIWKHUHDUHDQ\ $YLVXDOZD\RIVSRWWLQJWKHVHLVE\XVLQJ
µVFDWWHUSORWV¶ *5$3+6ĺ6&$77(5'27 7KH(;3/25(SURFHGXUHLQ6366
$1$/<=(ĺ'(6&5,37,9(67$7,67,&6ĺ(;3/25( LVSDUWLFXODUO\VXLWDEOH
IRUFKHFNLQJIRURXWOLHUVEHFDXVHLWQRWRQO\SURYLGHVDYDULHW\RISORWV IRUH[DPSOH
KLVWRJUDPVDQGER[SORWV EXWDOVRGLVSOD\VH[WUHPHYDOXHV WKHILYHODUJHVWDQGWKHILYH
VPDOOHVWYDOXHV IRUHDFKYDULDEOH)LJXUHSUHVHQWVRXWSXWVRIWKH(;3/25(
SURFHGXUH E\WLFNLQJµ2XWOLHUV¶LQWKHµ6WDWLVWLFV¶RSWLRQ RIWKHQXPEHURIZRUGV/
OHDUQHUVXVHGLQDFRPPXQLFDWLYHWDVN,QWKHER[SORWDQ\VFRUHWKDW6366FRQVLGHUVDQ
RXWOLHUDSSHDUVDVDOLWWOHFLUFOHZLWKLWVFDVHQXPEHUZULWWHQQH[WWRLW,QRXUH[DPSOH
&DVHKDVDQH[WUHPHYDOXHDQGWKH([WUHPH9DOXHVWDEOHLQGHHGFRQILUPVLQGHHG
WKDWWKLVVWXGHQWXVHGPRUHWKDQSHUFHQWPRUHZRUGVLQKHUSHUIRUPDQFHWKDQWKH
VHFRQGPRVWWDONDWLYHVSHDNHU

([WUHPH9DOXHV

&DVH1XPEHU 9DOXH
ZRUG +LJKHVW

/RZHVW

ZRUG
)LJXUH2XWSXWRIWKHµ2XWOLHU¶RSWLRQRIWKH6366(;3/25(SURFHGXUH([WUHPHYDOXHV
DQGER[SORW
+RZVKDOOZHµGHDO¶ZLWKRXWOLHUV")LUVWZHQHHGWRFKHFNLQWKHRULJLQDO
TXHVWLRQQDLUHRUGDWDUHFRUGZKHWKHUWKHH[WUHPHYDOXHLVVLPSO\WKHUHVXOWRIDPLVWDNH
ZHPDGHZKHQHQWHULQJWKHGDWD,IVRLWFDQEHHDVLO\FRUUHFWHG+RZHYHURXWOLHUVFDQ
DOVRLQGLFDWHDQRXWRIWKHRUGLQDU\EXWWUXHUHVSRQVHUHVSRQGHQWFDUHOHVVQHVVDQGHYHQ
LQWHQWLRQDOVLOOLQHVVRQWKHSDUWRIWKHSDUWLFLSDQWV ZKLFKGRHVKDSSHQZLWKVRPH
SDUWLFLSDQWVVXFKDVWHHQDJHNLGV ,IDFORVHDQDO\VLVRIWKHUHVSRQVHSDWWHUQVLQWKH
SDUWLFXODUTXHVWLRQQDLUHUHFRUGSRLQWVWRRQHRIWKHODWWHURSWLRQVZHVKRXOGFRQVLGHU
GHOHWLQJWKHVSXULRXVLQIRUPDWLRQVRWKDWLWGRHVQRWELDVWKHUHVXOWV,IZHFDQQRWPDNH

DQXQDPELJXRXVGHFLVLRQDVWRZKHWKHUWKHRXWOLHUUHIOHFWVDWUXHRUDQXQUHDOFDVHZH
PD\FRQGXFWWKHPDLQDQDO\VHVWZLFHZLWKDQGZLWKRXWWKHRXWOLHUDQGVHHLIWKH
RXWFRPHVZLOOSURYLGHDFOXHDERXWKRZWRKDQGOHWKHRXWOLHU
'DWDPDQLSXODWLRQ
µ'DWDPDQLSXODWLRQ¶LQYROYHVPDNLQJFKDQJHVLQWKHGDWDVHWSULRUWRWKHDQDO\VHVLQ
RUGHUWRPDNHLWPRUHDSSURSULDWHIRUFHUWDLQVWDWLVWLFDOSURFHGXUHVLWGRHVQRWLQYROYH
ELDVLQJWKHUHVXOWVRQHZD\RUDQRWKHU$WWKLVVWDJHWKHUHDUHWKUHHSDUWLFXODULVVXHVWR
DWWHQGWR D KDQGOLQJPLVVLQJGDWD E UHFRGLQJFHUWDLQYDOXHVDQG F FRQVLGHULQJ
VWDQGDUGL]LQJWKHGDWD
+DQGOLQJPLVVLQJGDWD
0LVVLQJGDWDDUHDQXLVDQFHIRUDWOHDVWWZRUHDVRQV)LUVWLWLVQRWDOZD\VFOHDUZKHWKHU
WKHODFNRIDQ\UHVSRQVHLVPHDQLQJIXORUQRWWKDWLVZKHWKHUWKHRPLVVLRQZDV
GHOLEHUDWHRUPHUHO\DQDFFLGHQW6HFRQGIRUWKHSXUSRVHRIFHUWDLQVWDWLVWLFDO
SURFHGXUHVDVLQJOHPLVVLQJYDOXHFDQLQYDOLGDWHDQRWKHUZLVHFRPSOHWHGDWDVHWIRUD
SHUVRQ)RUH[DPSOHLQPXOWLYDULDWHVWDWLVWLFVZKHQPDQ\YDULDEOHVDUHH[DPLQHGDWWKH
VDPHWLPHLWFDQEHDEDVLFUHTXLUHPHQWWRKDYHYDOLGYDOXHVIRUHYHU\YDULDEOHIRUD
SHUVRQRUHOVHWKHSHUVRQZLOOEHDXWRPDWLFDOO\H[FOXGHGIURPWKHDQDO\VHV*LYHQWKDW
UHJUHWWDEO\LWLVTXLWHFRPPRQWRKDYHDIHZPLVVLQJYDOXHVLQHYHU\TXHVWLRQQDLUHZH
FDQHQGXSORVLQJDVPXFKDVKDOIRIRXUVDPSOHWKLVZD\ZKLFKLVFOHDUO\XQGHVLUDEOH
,QVXFKFDVHVWKHSURJUDPPLJKWRIIHUVRPHZD\VRILQSXWWLQJWKHPLVVLQJGDWDWKDWDUH
XQOLNHO\WRFKDQJHWKHUHVXOWVIRUH[DPSOHE\LQFOXGLQJLWHPPHDQVRUPD[LPXP
OLNHOLKRRGHVWLPDWHV/XFNLO\VHYHUDOVWDWLVWLFDOSURFHGXUHVDOORZIRUDFKRLFHEHWZHHQ
µOLVWZLVHGHOHWLRQ¶DQGµSDLUZLVHGHOHWLRQ¶RIFDVHVZLWKPLVVLQJYDOXHV7KHIRUPHU
UHIHUVWRWKHµKDUGOLQH¶VROXWLRQZKHUHE\RQHPLVVLQJYDOXHGHOHWHVDZKROHFDVHIURP
DOOWKHDQDO\VHVHYHQLIVRPHRIWKHDYDLODEOHGDWDFRXOGEHXVHGIRUFHUWDLQFDOFXODWLRQV
7KHODWWHURSWLRQ²ZKLFKZRXOGEHP\UHFRPPHQGDWLRQIRUPRVWFDVHV²UHIHUVWRWKH
WHPSRUDU\GHOHWLRQRIDFDVHIURPDQDQDO\VLVRQO\ZKHQVSHFLILFVWDWLVWLFVDUH
FRPSXWHGWKDWZRXOGLQYROYHWKHSDUWLFXODUPLVVLQJYDOXHZKLOHLQFOXGLQJWKHFDVHLQ
DOOWKHRWKHUDQDO\VHVIRUZKLFKLWKDVVXIILFLHQWLQIRUPDWLRQ
5HFRGLQJQHJDWLYHO\ZRUGHGYDOXHV
,QRUGHUWRDYRLGDUHVSRQVHVHWLQTXHVWLRQQDLUHVZKHUHE\WKHUHVSRQGHQWVPDUNRQO\
RQHVLGHRIDUDWLQJVFDOHTXHVWLRQQDLUHGHVLJQHUVRIWHQLQFOXGHERWKSRVLWLYHO\DQG
QHJDWLYHO\ZRUGHGLWHPV WKLVPD\DOVRUHGXFHWKHKDUPIXOHIIHFWVRIWKHµDFTXLHVFHQFH
ELDV¶ IRUH[DPSOHDVFDOHWKDWWDUJHWVSRVLWLYHDWWLWXGHVWRZDUGVWKH/FODVVPD\
LQFOXGHWKHLWHPµ,ILQGP\ODQJXDJHFODVVHVERULQJ¶,IZHKDYHVXFKQHJDWLYHO\
ZRUGHGLWHPVZHPXVWQRWIRUJHWWRUHYHUVHWKHVFRULQJIRUWKHVHEHIRUHLQFOXGLQJWKHP
LQPXOWLLWHPVFDOHV 75$16)250ĺ5(&2'(ĺ,1726$0(9$5,$%/(6VHH
7DEOHIRUDQH[DPSOH 7KLVPD\VRXQGOLNHDQREYLRXVDQGWULYLDOUHFRPPHQGDWLRQ

EXWXQOHVV\RXPDNHWKHUHFRGLQJRIVXFKVFRUHVDFRPSXOVRU\VWHSLQWKHGDWD
SUHSDUDWLRQSKDVHLWLVDOOWRRHDV\WRIRUJHWDERXWLW
6WDQGDUGL]LQJWKHGDWD
:KHQZHXVHSRROHGUHVXOWVIURPYDULRXVVXEVDPSOHV IRUH[DPSOHVHYHUDOVFKRROVRU
FODVVJURXSV RQHZD\RIFRQWUROOLQJIRUWKHKHWHURJHQHRXVQDWXUHRIWKHVXEJURXSVLV
WRXVHVWDQGDUGL]HGVFRUHV7KHVWDQGDUGL]DWLRQRIUDZVFRUHVLQYROYHVWKHFRQYHUVLRQ
RIWKHGLVWULEXWLRQZLWKLQDVDPSOHLQDZD\WKDWWKHPHDQZLOOEHDQGWKHVWDQGDUG
GHYLDWLRQ )RUDQH[SODQDWLRQRIWKHVHWHUPVVHH6HFWLRQRQGHVFULSWLYH
VWDWLVWLFV 7KXVVWDQGDUGVFRUHVH[SUHVVKRZPXFKHDFKUDZYDOXHLVGLIIHUHQWIURPWKH
JURXSPHDQDQGE\HTXDOL]LQJWKHPHDQVVFRUHVREWDLQHGIURPGLIIHUHQWVXEVDPSOHV
DUHUHDGLO\FRPSDUDEOH6XFKDWUDQVIRUPDWLRQLVSHUPLVVLEOHZLWKFRUUHODWLRQEDVHG
DQDO\VHV IRUH[DPSOHFRUUHODWLRQDQDO\VLVIDFWRUDQDO\VLVDQGVWUXFWXUDOHTXDWLRQ
PRGHOOLQJ EHFDXVHZKHQZHFRPSXWHFRUUHODWLRQVZHFDQFDUU\RXWFHUWDLQPHFKDQLFDO
FRQYHUVLRQVRIWKHUDZVFRUHVZLWKRXWWKHVHDIIHFWLQJWKHUHVXOWLQJFRHIILFLHQWV)RUD
GHWDLOHGDUJXPHQWLQIDYRXURIVWDQGDUGL]LQJKHWHURJHQHRXVTXHVWLRQQDLUHGDWDVHH
*DUGQHU ± ZKRDOVRSURYLGHVDK\SRWKHWLFDOLOOXVWUDWLRQLQZKLFKD
PRWLYDWLRQPHDVXUHVKRZVVLJQLILFDQWSRVLWLYHFRUUHODWLRQZLWKOHDUQLQJDFKLHYHPHQWLQ
WZRVFKRROFODVVHVZKHQFRPSXWHGVHSDUDWHO\EXWWKHVDPHFRUUHODWLRQEHFRPHVQRQ
VLJQLILFDQWZKHQWKHGDWDDUHSRROHGZLWKRXWVWDQGDUGL]DWLRQ6WDQGDUGVFRUHVFDQEH
FRPSXWHGLQ6366E\WLFNLQJWKHµ6DYHVWDQGDUGL]HGYDOXHVDVYDULDEOHV¶ER[LQ
'(6&5,37,9(6 $1$/<=(ĺ'(6&5,37,9(67$7,67,&6ĺ'(6&5,37,9(6
'DWDUHGXFWLRQDQGUHOLDELOLW\DQDO\VLV
µ'DWDUHGXFWLRQ¶LVVRPHZKHUHEHWZHHQGDWDPDQLSXODWLRQDQGGDWDDQDO\VLVLQWKH
VHQVHWKDWDOWKRXJKWKLVSURFHGXUHDOUHDG\DIIHFWVWKHUHVXOWVLWVLPSDFWLVNHSWPLQLPDO
DQGWKHPDLQREMHFWLYHRIWKHSURFHGXUHLVWRSUHSDUHWKHGDWDIRUDQDO\VLVE\UHGXFLQJ
WKHQXPEHURIYDULDEOHVWRDPDQDJHDEOHVL]H(YHU\LQLWLDOTXDQWLWDWLYHGDWDVHWKDV
PDQ\PRUHYDULDEOHVWKDQQHFHVVDU\)RUH[DPSOHDZHOOGHVLJQHGTXHVWLRQQDLUH
FRQWDLQVPXOWLSOHLWHPVIRFXVLQJRQHDFKFRQWHQWDUHD VHH6HFWLRQ DQGWKHUHIRUH
WKHSDUDOOHOLWHPVQHHGWREHVXPPHGXSLQµPXOWLLWHPVFDOHV¶IRUWKHSXUSRVHRI
DQDO\VLV7KXVGDWDUHGXFWLRQLQYROYHVFUHDWLQJIHZHUEXWEURDGHUYDULDEOHVWKDWFDUU\
DOPRVWDVPXFKLQIRUPDWLRQDVWKHRULJLQDOYDULDEOHV7KHPRVWFRPPRQSURFHGXUHWR
DFKLHYHWKLVLVWRFDOFXODWHWKHPHDQRIWKHSDUDOOHORULQWHUUHODWHGLWHPV 75$16)250
ĺ&20387( ,QRUGHUWRGHFLGHZKLFKLWHPVKDQJWRJHWKHUDQGKRZPXFKZHFDQ
DSSO\DXVHIXOSURFHGXUHµIDFWRUDQDO\VLV¶ 6HFWLRQ
,WGRHVQRWUHTXLUHPXFKMXVWLILFDWLRQWKDWFRPSRVLWHYDULDEOHVVXFKDVPXOWLLWHP
VFDOHVDUHRQO\HIIHFWLYHLIWKHFRQVWLWXHQWLWHPVWKDWPDNHXSWKHYDULDEOHZRUNWRJHWKHU
LQDKRPRJHQHRXVPDQQHUWKDWLVLIWKH\PHDVXUHWKHVDPHWDUJHWDUHD,Q
SV\FKRPHWULFWHUPVWKLVPHDQVWKDWHDFKLWHPRQDPXOWLLWHPVFDOHVKRXOGFRUUHODWH
ZLWKWKHRWKHULWHPVDQGZLWKWKHWRWDOVFDOHVFRUHZKLFKKDVEHHQUHIHUUHGWRDV
/LNHUW¶VFULWHULRQRIµLQWHUQDOFRQVLVWHQF\¶,QGHHGµLQWHUQDOFRQVLVWHQF\UHOLDELOLW\¶LV

JHQHUDOO\VHHQDVWKHSV\FKRPHWULFSUHUHTXLVLWHIRUDQ\PXOWLLWHPVFDOHLQD
TXHVWLRQQDLUHWKDWLVWREHXVHGDVDUHVHDUFKLQVWUXPHQW6XUSULVLQJO\WKHLQWHUQDO
FRQVLVWHQF\UHOLDELOLW\RIDVFDOHGRHVQRWRQO\GHSHQGRQWKHLQWHUQDOFRQVLVWHQF\RIWKH
LWHPVEXWDOVRRQWKHQXPEHURILWHPVPDNLQJXSWKHVFDOH7KDWLVLWLVHDVLHUWR
DFKLHYHDSSURSULDWHLQWHUQDOFRQVLVWHQF\UHOLDELOLW\ZLWKLWHPVWKDQZLWK7KLVRI
FRXUVHPDNHVJRRGVHQVHLQDVKRUWVFDOHFRQVLVWLQJRI±LWHPVDGLYHUJHQWLWHPFDQ
PDNHPXFKPRUHRIDGLVUXSWLRQWKDQLQDVFDOHRIDQGWKHUHIRUHVKRUWVFDOHVQHHGWR
GLVSOD\PRUHHYLGHQFHRIKRPRJHQHLW\WKDQORQJRQHVLIWKH\DUHWREHVHHQDV
WUXVWZRUWK\
,QWHUQDOFRQVLVWHQF\UHOLDELOLW\LVPHDVXUHGE\WKH&URQEDFK$OSKDFRHIILFLHQW
QDPHGDIWHULWVLQWURGXFHU/-&URQEDFK 7KLVLVDILJXUHUDQJLQJEHWZHHQDQG
DOWKRXJKLQH[WUHPHFDVHV²IRUH[DPSOHZLWKYHU\VPDOOVDPSOHVDQGZLWKLWHPVWKDW
PHDVXUHGLIIHUHQWWKLQJV²LWFDQDOVREHQHJDWLYH DQGLILWSURYHVWREHYHU\ORZ
HLWKHUWKHSDUWLFXODUVFDOHLVWRRVKRUWRUWKHLWHPVKDYHYHU\OLWWOHLQFRPPRQ,QWHUQDO
FRQVLVWHQF\HVWLPDWHVIRUZHOOGHYHORSHGVFDOHVFRQWDLQLQJDERXWLWHPVRXJKWWR
DSSURDFK%HFDXVHRIWKHFRPSOH[LW\RIWKHVHFRQGODQJXDJHDFTXLVLWLRQSURFHVV
/UHVHDUFKHUVW\SLFDOO\ZDQWWRPHDVXUHPDQ\GLIIHUHQWDUHDVLQRQHTXHVWLRQQDLUHDQG
WKHUHIRUHFDQQRWXVHYHU\ORQJVFDOHV RUWKHFRPSOHWLRQRIWKHTXHVWLRQQDLUHZRXOG
WDNHVHYHUDOKRXUV 7KLVPHDQVWKDWVRPHZKDWORZHU&URQEDFK$OSKDFRHIILFLHQWVDUH
WREHH[SHFWHGEXWHYHQZLWKVKRUWVFDOHVRILWHPVZHVKRXOGDLPDWUHOLDELOLW\
FRHIILFLHQWVLQH[FHVVRILIWKH&URQEDFK$OSKDRIDVFDOHGRHVQRWUHDFKWKLV
VKRXOGVRXQGZDUQLQJEHOOV
+RZGRZHREWDLQ&URQEDFK$OSKDFRHIILFLHQWV"6366PDNHVLWUHODWLYHO\HDV\WR
FRQGXFWUHOLDELOLW\DQDO\VLV7KHµ5HOLDELOLW\¶SURFHGXUHRI6366 $1$/<6,6ĺ
6&$/(ĺ5(/,$%,/,7<$1$/<6,6 QRWRQO\SURYLGHVWKH&URQEDFK$OSKDIRUDQ\
JLYHQVFDOHEXWE\FKRRVLQJDVWDWLVWLFVRSWLRQLWDOVRFRPSXWHVZKDWWKHDOSKD
FRHIILFLHQWZRXOGEHLIDSDUWLFXODULWHPZHUHGHOHWHGIURPWKHVFDOH%\ORRNLQJDWWKH
OLVWRIWKHVHµZRXOGEH¶DOSKDVIRUHDFKLWHPZHFDQLPPHGLDWHO\VHHZKLFKLWHP
UHGXFHVWKHLQWHUQDOFRQVLVWHQF\RIWKHVFDOHDQGVKRXOGWKHUHIRUHEHFRQVLGHUHGIRU
RPLVVLRQ
.H\VWDWLVWLFDOFRQFHSWV
%HIRUHZHVWDUWH[SORULQJWKHUDQJHRIVWDWLVWLFDOWHFKQLTXHVDWRXUGLVSRVDOWKHUHDUH
FHUWDLQNH\VWDWLVWLFDOFRQFHSWVWKDWZHQHHGWRXQGHUVWDQGWREHDEOHWRFKRRVHWKHULJKW
SURFHGXUHDQGLQWHUSUHWWKHUHVXOWVFRUUHFWO\,QWKLVVHFWLRQ,ZLOOGLVFXVVWKHPDLQW\SHV
RITXDQWLWDWLYHGDWDWKHLPSRUWDQFHRIDQRUPDOGLVWULEXWLRQRIWKHGDWDWKHGLVWLQFWLRQ
EHWZHHQGHVFULSWLYHDQGLQIHUHQWLDOVWDWLVWLFVDQGWKHQRWLRQVRIVWDWLVWLFDOVLJQLILFDQFH
FRQILGHQFHLQWHUYDODQGHIIHFWVL]H
0DLQW\SHVRITXDQWLWDWLYHGDWD

$OWKRXJKTXDQWLWDWLYHUHVHDUFKKDVVHYHUDOSRVVLEOHGDWDVRXUFHVWKHILQDOQXPHULFDOO\
FRGHGGDWDLVRQO\RIWKUHHW\SHVµQRPLQDO¶ RUµFDWHJRULFDO¶ µRUGLQDO¶DQGµLQWHUYDO¶
7KHGLIIHUHQFHEHWZHHQWKHWKUHHPDLQW\SHVOLHVLQWKHSUHFLVLRQRIWKHPHDVXUHPHQW
DQGWKLVGLVWLQFWLRQGHWHUPLQHVWKHW\SHRIVWDWLVWLFDOSURFHGXUHVWKDWZHFDQXVHWR
DQDO\VHWKHGDWD
x µ1RPLQDO¶RUµFDWHJRULFDO¶GDWDLVWKHOHDVWSUHFLVHGDWDW\SH,WLVDVVRFLDWHGZLWK
YDULDEOHVWKDWKDYHQRQXPHULFDOYDOXHVVXFKDVJHQGHURUUDFH+HUHWKHDVVLJQHG
YDOXHVDUHFRPSOHWHO\DUELWUDU\IRUH[DPSOHDVDOUHDG\PHQWLRQHGIRUWKHJHQGHU
YDULDEOHPDOHLVXVXDOO\FRGHGµ¶DQGIHPDOHµ¶ZKLFKGRHVQRWLQGLFDWHDQ\
GLIIHUHQFHLQVL]HRUVDOLHQFH
x µ2UGLQDO¶GDWDLQYROYHVUDQNHGQXPEHUV)RUH[DPSOHDPXOWLSOHFKRLFHLWHPZLWK
RSWLRQVVXFKDVµQHYHU¶µRQFHDGD\¶µRQFHDZHHN¶DQGµRQFHDPRQWK¶ZLOO
SURGXFHRUGLQDOGDWDEHFDXVHWKHDQVZHUVFDQEHSODFHGRQDµIUHTXHQF\¶
FRQWLQXXPEXWWKHYDOXHVGRQRWFRUUHVSRQGWRDQ\UHJXODUPHDVXUHPHQWRQDVFDOH
DVZRXOGµRQFHDGD\¶µWZLFHDGD\¶µWKUHHWLPHVDGD\¶HWF
x µ,QWHUYDO¶GDWDLVWKHPRVWSUHFLVHW\SHLWFDQEHVHHQDVRUGLQDOGDWDLQZKLFKWKH
YDULRXVYDOXHVDUHDWDQHTXDOGLVWDQFH²RULQWHUYDOV²IURPHDFKRWKHURQD
FRQWLQXXP7HVWVFRUHVIRUH[DPSOHXVXDOO\SURYLGHDVHULHVRIYDOXHVZKLFK
FRUUHVSRQGWRHTXDOGLIIHUHQFHVLQWKHGHJUHHVL]HRIWKHYDULDEOHPHDVXUHG
7KHVHSDUDWLRQRIWKHVHWKUHHW\SHVRIPHDVXUHVEHFRPHVLPSRUWDQWZKHQZHVHOHFW
WKHVWDWLVWLFDOWHFKQLTXHVWREHXVHGZLWKRXUGDWD7KHELJGLYLGLQJOLQHLVEHWZHHQ
µSDUDPHWULFSURFHGXUHV¶ZKLFKUHTXLUHLQWHUYDOGDWDDQGµQRQSDUDPHWULFSURFHGXUHV¶
ZKLFKFDQEHDSSOLHGWRRUGLQDODQGHYHQQRPLQDOGDWD :HZLOOFRPHEDFNWRWKLV
TXHVWLRQLQ6HFWLRQ
1RUPDOGLVWULEXWLRQRIWKHGDWD
$VHFRQGFKDUDFWHULVWLFRIWKHGDWDWKDWKDVDVSHFLDOVWDWLVWLFDOVLJQLILFDQFHLVZKHWKHULW
LVµQRUPDOO\GLVWULEXWHG¶7KLVPHDQVWKDWLIZHSORWWKHGDWDZHJHWDV\PPHWULFDOEHOO
VKDSHGFXUYHZKLFKKDVWKHJUHDWHVWIUHTXHQF\RIVFRUHVLQWKHPLGGOHZLWKVPDOOHU
IUHTXHQFLHVWRZDUGVWKHH[WUHPHVMXVWOLNHWKHLOOXVWUDWLRQEHORZ,QSUDFWLFDOWHUPVD
QRUPDOGLVWULEXWLRQPHDQVWKDWVRPHRIWKHYDOXHVDUHORZVRPHKLJKDQGWKHEXONRI
WKHYDOXHVDUHFHQWUHGDURXQGWKHPLGGOHWKDWLVDURXQGWKHPHDQ

,QWKHSUHYLRXVVHFWLRQ,PHQWLRQHGWKDWµSDUDPHWULF¶WHVWVUHTXLUHLQWHUYDOGDWDDQG
QRZZHFDQDGGWKDWLQRUGHUWREHDEOHWRXVHSDUDPHWULFSURFHGXUHVWKHLQWHUYDOGDWD
DOVRQHHGVWREHQRUPDOO\GLVWULEXWHG7KLVLVDQLPSRUWDQWDVVXPSWLRQRIWKHGDWD
EHFDXVHPRVWRIWKHVWDWLVWLFDOSURFHGXUHVGLVFXVVHGLQWKLVFKDSWHUDUHSDUDPHWULF )RU
QRQSDUDPHWULFWHVWVVHH6HFWLRQ )RUWXQDWHO\QRUPDOLW\GRHVQRWKDYHWREH
SHUIHFWEHFDXVHPRVWSURFHGXUHVZRUNZHOOZLWKGDWDWKDWLVRQO\DSSUR[LPDWHO\
QRUPDOO\GLVWULEXWHG
+RZFDQZHGHFLGHZKHWKHURXUGDWDLVQRUPDOO\GLVWULEXWHG"7KH(;3/25(
SURFHGXUHRI6366 $1$/<=(ĺ'(6&5,37,9(67$7,67,&6ĺ(;3/25(
RIIHUVDVSHFLDOµ1RUPDOLW\SORWVZLWKWHVWV¶IXQFWLRQXQGHUWKHµ3ORWV¶RSWLRQSURGXFLQJ
YDULRXVSORWVDVZHOODVWKHUHVXOWVRIWKHµ.ROPRJRURY6PLUQRY¶VWDWLVWLFZKLFKLVD
WHVWRIQRUPDOLW\7KLVODWWHUZRUNVZHOOZLWKVPDOOHUVDPSOHVEXWLWFDQEHWRRVHQVLWLYH
ZLWKODUJHVDPSOHV²KRZHYHUZLWKVDPSOHVRUPRUHFDVHVWKHLPSDFWRIDQ\
GHYLDWLRQVIURPQRUPDOLW\GLPLQLVKHVDQGZLWKVXFKODUJHGDWDVHWVH[DPLQLQJWKHYLVXDO
DSSHDUDQFHRIWKHGLVWULEXWLRQLVXVXDOO\VXIILFLHQW 7DEDFKQLFNDQG)LGHOO
'HVFULSWLYHYHUVXVLQIHUHQWLDOVWDWLVWLFV
6WDWLVWLFVFDQEHGLYLGHGLQWRWZRSULQFLSDODUHDVµGHVFULSWLYHVWDWLVWLFV¶DQGµLQIHUHQWLDO
VWDWLVWLFV¶7KLVGLVWLQFWLRQOLHVDWWKHKHDUWRIVWDWLVWLFVDQGIDLOXUHWRXQGHUVWDQG
VWDWLVWLFVLVRIWHQFDXVHGE\WKHIDFWWKDWLQVXIILFLHQWDWWHQWLRQKDVEHHQSDLGWRWKLV
GLIIHUHQFH'HVFULSWLYHVWDWLVWLFVDUHXVHGWRVXPPDUL]HVHWVRIQXPHULFDOGDWDLQRUGHU
WRFRQVHUYHWLPHDQGVSDFH,WLVREYLRXVWKDWSURYLGLQJWKHPHDQDQGWKHUDQJH LH
PLQLPXPDQGPD[LPXPYDOXHV RIDYDULDEOHLVDPRUHSURIHVVLRQDOZD\RIGHVFULELQJ
WKHUHVSRQGHQWV¶DQVZHUVWKDQOLVWLQJDOOWKHVFRUHVWKDWZHKDYHREWDLQHG$QGLIZH
DOVRLQFOXGHWKHµVWDQGDUGGHYLDWLRQ¶RIWKHUHVXOWV ZKLFKDVZHZLOOVHHLQ6HFWLRQ
RQGHVFULSWLYHVWDWLVWLFVLVDQLQGH[RIWKHDYHUDJHGLVSDULW\DPRQJWKHVFRUHV ZHKDYH
DFKLHYHGDZHOOURXQGHGGHVFULSWLRQRIWKHVFRUHVWKDWZRXOGVDWLVI\PRVWSXUSRVHV
7KXVGHVFULSWLYHVWDWLVWLFVRIIHUDWLG\ZD\RISUHVHQWLQJWKHGDWDZHKDYH7KH
LPSRUWDQWWKLQJKRZHYHULVWRQRWHWKDWWKHVHVWDWLVWLFVGRQRWDOORZGUDZLQJDQ\
JHQHUDOFRQFOXVLRQVWKDWZRXOGJREH\RQGWKHVDPSOH,QSUDFWLFHWKLVPHDQVWKDWZKHQ
ZHUHSRUWGHVFULSWLYHUHVXOWVZHRXJKWWRVWDUWHYHU\VHQWHQFHZKLFKGHVFULEHV
GHVFULSWLYHIHDWXUHVZLWKµ,QP\VDPSOH«¶,IZHZDQWWRVD\VRPHWKLQJDERXWSRVVLEOH
JHQHUDOOHVVRQVWKDWPD\EHGUDZQIURPRXUVWXG\²ZKLFKLVZKDWZHXVXDOO\GRZKHQ
ZHFRQGXFWUHVHDUFK²ZHQHHGWRFRPSXWHLQIHUHQWLDOVWDWLVWLFV%URDGO\VSHDNLQJ
LQIHUHQWLDOVWDWLVWLFVDUHWKHVDPHDVGHVFULSWLYHVWDWLVWLFVH[FHSWWKDWWKHFRPSXWHUDOVR
WHVWVZKHWKHUWKHUHVXOWVWKDWZHREVHUYHGLQRXUVDPSOH IRUH[DPSOHGLIIHUHQFHVRU
FRUUHODWLRQV DUHSRZHUIXOHQRXJKWRJHQHUDOL]HWRWKHZKROHSRSXODWLRQ,WPDNHV
LQWXLWLYHVHQVHWKDWZKDWZHILQGQHHGVWRSDVVDFHUWDLQWKUHVKROGRIPDJQLWXGHWREHRI
JHQHUDOVLJQLILFDQFH²LQIHUHQWLDOVWDWLVWLFVEDVLFDOO\HYDOXDWHVWKHUHVXOWVDJDLQVWVXFKD
WKUHVKROG

'HVFULSWLYHVWDWLVWLFVDUHXVHIXOIRUH[DPSOHWRGHVFULEHWKHDFKLHYHPHQWRID
SDUWLFXODUFODVVRIOHDUQHUV:KDWKDSSHQVKRZHYHULIZHQRWLFHWKDWVD\WKH/
OHDUQLQJDFKLHYHPHQWRIER\VDQGJLUOVVKRZVDUHPDUNDEOHGLIIHUHQFHLQRXUVDPSOH
ZLWKJLUOVRXWSHUIRUPLQJER\V ZKLFKLVRIWHQWKHFDVH "&DQZHGUDZWKHLQIHUHQFHWKDW
JLUOVDUHEHWWHUODQJXDJHOHDUQHUV"1R%DVHGRQGHVFULSWLYHVWDWLVWLFVDOOZHFDQVD\LV
WKDWLQWKLVFODVVJLUOVGLGEHWWHUWKDQER\V,QRUGHUWRYHQWXUHDQ\JHQHUDOL]DWLRQ
FRQFHUQLQJWKHZLGHUSRSXODWLRQDQGQRWMXVWWKHSDUWLFXODUVDPSOHZHQHHGWRVKRZ
WKDWWKHGLIIHUHQFHEHWZHHQJLUOVDQGER\VLVVLJQLILFDQWLQWKHVWDWLVWLFDOVHQVH7R
DFKLHYHWKLVDVGLVFXVVHGDERYHZHQHHGWRHPSOR\LQIHUHQWLDOVWDWLVWLFDOSURFHGXUHV
6WDWLVWLFDOVLJQLILFDQFH
7KHPDLQFRQFHUQRILQIHUHQWLDOVWDWLVWLFVKDVWUDGLWLRQDOO\EHHQWKHWHVWLQJRIµVWDWLVWLFDO
VLJQLILFDQFH¶6WDWLVWLFDOVLJQLILFDQFHGHQRWHVZKHWKHUDSDUWLFXODUUHVXOWREVHUYHGLQD
VDPSOHLVµWUXH¶IRUWKHZKROHSRSXODWLRQDQGLVWKHUHIRUHJHQHUDOL]DEOH,IDUHVXOWLV
QRQVLJQLILFDQWWKLVPHDQVWKDWZHFDQQRWEHFHUWDLQWKDWLWGLGQRWRFFXULQWKH
SDUWLFXODUVDPSOHRQO\EHFDXVHRIFKDQFH IRUH[DPSOHEHFDXVHRIWKHXQLTXH
FRPSRVLWLRQRIWKHOHDUQHUVH[DPLQHG ,QRWKHUZRUGVHYHQLIZHIHHOWKDWWKH
SDUWLFXODUGHVFULSWLYHGDWDUHYHDOVVRPHWUXHDQGPRUHJHQHUDOWHQGHQFLHVLIWKHUHVXOWV
DUHQRWVLJQLILFDQWZHFDQQRWH[FOXGHWKHSRVVLELOLW\RIDPHUHFRLQFLGHQFH)RUWKLV
UHDVRQVWDWLVWLFDOO\QRQVLJQLILFDQWUHVXOWVPXVWEHLJQRUHGLQUHVHDUFKVWXGLHV
2QHLPSRUWDQWIHDWXUHRIVWDWLVWLFDOVLJQLILFDQFHLVWKDWLWLVWKHIXQFWLRQRIQRWRQO\
WKHPDJQLWXGHRIWKHUHVXOWEXWDOVRWKHVL]HRIWKHVDPSOHLQYHVWLJDWHG,WLVHDV\WRVHH
ZK\WKLVLVVRLIZHDVVHVVVD\PLOOLRQVRISHRSOHHYHQDUHODWLYHO\ZHDNWHQGHQF\
FDQEHUHJDUGHGDVW\SLFDORIWKHZKROHSRSXODWLRQZKHUHDVZLWKRQO\DKDQGIXORI
SHRSOHZHFDQQRWEHFHUWDLQHYHQDERXWIDUVWURQJHUWHQGHQFLHV7KHUHIRUHWKH
FRPSXWHUWDNHVWKHFRPELQHGHIIHFWRIPDJQLWXGHDQGVDPSOHVL]HLQWRDFFRXQWZKHQLW
FDOFXODWHVWKHVLJQLILFDQFHRIDUHVXOW,IDSDUWLFXODUUHVXOWLVPDUNHGDVVLJQLILFDQWZH
FDQXWWHUDVLJKRIUHOLHIDVWKLVPHDQVWKDWWKHREVHUYHGSKHQRPHQRQUHSUHVHQWVD
VLJQLILFDQWGHSDUWXUHIURPZKDWPLJKWEHH[SHFWHGE\FKDQFHDORQH7KDWLVLWFDQEH
DVVXPHGWREHUHDODQGZHFDQUHSRUWLW+RZHYHULIDSDUWLFXODUUHVXOWLVQRW
VLJQLILFDQWZHPXVWQRWUHSRUWLW IRUH[DPSOHLIWKHPHDQVFRUHVRIWZRJURXSVDUHQRW
VLJQLILFDQWO\GLIIHUHQWZHPXVWQRWVD\WKDWRQHLVODUJHUWKDQWKHRWKHUHYHQLIWKH
GHVFULSWLYHVWDWLVWLFVGRVKRZVRPHGLIIHUHQFH
6LJQLILFDQFHLVPHDVXUHGE\DSUREDELOLW\FRHIILFLHQW S ZKLFKFDQUDQJHIURPWR
$SILJXUHRIVD\PHDQVWKDWWKHREWDLQHGUHVXOWPLJKWEHGXHWRSXUHFKDQFH
LQSHUFHQWRIWKHFDVHV,QVRFLDOVFLHQFHVZHW\SLFDOO\FRQVLGHUDUHVXOWVLJQLILFDQWLI
SWKDWLVLIWKHSUREDELOLW\RIWKHUHVXOWQRWEHLQJUHDOEXWRQO\GXHWRFKDQFH
IRUH[DPSOHVDPSOLQJSHFXOLDULW\ LVOHVVWKHQSHUFHQW LHILYHWLPHVRXWRID
KXQGUHG 6LJQLILFDQFHLVW\SLFDOO\PDUNHGLQWDEOHVDVIROORZV

LQGLFDWHVS
LQGLFDWHVS
LQGLFDWHVS
,WLVLPSRUWDQWWRVWUHVVKHUHWKDWVWDWLVWLFDOVLJQLILFDQFHLVQRWWKHILQDODQVZHU
EHFDXVHHYHQWKRXJKDUHVXOWPD\EHVWDWLVWLFDOO\VLJQLILFDQW LHUHOLDEOHDQGWUXHWRWKH
ODUJHUSRSXODWLRQ LWPD\QRWEHYDOXDEOHµ6LJQLILFDQFH¶LQWKHVWDWLVWLFDOVHQVHRQO\
PHDQVµSUREDEO\WUXH¶UDWKHUWKDQµLPSRUWDQW¶7RH[DPLQHWKHLPSRUWDQFHRIDQ
REVHUYHGVFRUHZHQHHGWRFRQVLGHUWKHQRWLRQVRIµFRQILGHQFHLQWHUYDOV¶DQGµHIIHFW
VL]H¶²VHHEHORZ$QGILQDOO\DWHFKQLFDOSRLQW,VXJJHVWWKDWUHVHDUFKHUVDOZD\VVHW
WKHFRPSXWHUWRFRPSXWHµWZRWDLOHG¶VLJQLILFDQFH ZKLFKLVWKHGHIDXOWLQ6366
ZKLFKLVDSSURSULDWHZKHQZHDUHQRWVXUHDERXWZKHWKHUWKHUHVXOWVZLOOEHQHJDWLYHRU
SRVLWLYH LHLQWWHVWV²VHH6HFWLRQ²ZHDUHQRWVXUHZKLFKRIWKHFRPSDUHGPHDQV
LVODUJHUDQGLQFRUUHODWLRQZHGRQRWNQRZZKHWKHUWKHFRHIILFLHQWZLOOEHSRVLWLYHRU
QHJDWLYH ,QP\H[SHULHQFHRQO\YHU\UDUHO\FDQZHSUHGLFWZLWKFHUWDLQW\WKHGLUHFWLRQ
RIWKHUHVXOWVLQDGYDQFHLQZKLFKFDVHµRQHWDLOHG¶VLJQLILFDQFH ZKLFKLVHDVLHUWR
DFKLHYH ZRXOGEHVXIILFLHQW
&RQILGHQFHLQWHUYDOV
:KLOHµVWDWLVWLFDOVLJQLILFDQFH¶RQO\JLYHVXVD\HVQRDQVZHUDERXWDUHVXOWWKH
µFRQILGHQFHLQWHUYDO¶LVPRUHLQIRUPDWLYHEHFDXVHLWSURYLGHVDUDQJHRIVFRUHV
EHWZHHQWZRERXQGDULHV WKDWFDQEHFRQVLGHUHGVLJQLILFDQWIRUWKHSRSXODWLRQ,QWKH
VRFLDOVFLHQFHVWKHXVXDOFRQILGHQFHOHYHOLVSHUFHQWDQGDSHUFHQWFRQILGHQFH
LQWHUYDORIDPHDQVFRUHFDQEHLQWHUSUHWHGDVDUDQJHRIVFRUHVWKDWZLOOFRQWDLQWKH
SRSXODWLRQPHDQZLWKDSUREDELOLW\,IWKHPHDQRIRXUVSHFLILFVDPSOHIDOOVZLWKLQ
WKLVUDQJHWKHQLWFDQEHFRQVLGHUHGVLJQLILFDQW6366DOZD\VVXSSOHPHQWVWHVWVRI
VWDWLVWLFDOVLJQLILFDQFHZLWKFRQILGHQFHLQWHUYDOVDQGPDQ\UHVHDUFKHUVKDYHUHFHQWO\
UHFRPPHQGHGWKDWZHUHSRUWVXFKLQWHUYDOVPRUHRIWHQLQSXEOLVKHGVWXGLHV7KH
3XEOLFDWLRQ0DQXDORIWKH$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQ $3$ ZKLFK
LVJHQHUDOO\VHHQDVWKHPRVWLPSRUWDQWVRXUFHRIJXLGHOLQHVRQKRZWRSUHVHQWRXU
ILQGLQJVLQDSSOLHGOLQJXLVWLFV VHH6HFWLRQRQVW\OHPDQXDOV IRUH[DPSOHVWDWHV
7KHUHSRUWLQJRIFRQILGHQFHLQWHUYDOV«FDQEHDQH[WUHPHO\HIIHFWLYHZD\RI
UHSRUWLQJUHVXOWV%HFDXVHFRQILGHQFHLQWHUYDOVFRPELQHLQIRUPDWLRQRQORFDWLRQ
DQGSUHFLVLRQDQGFDQRIWHQEHGLUHFWO\XVHGWRLQIHUVLJQLILFDQFHOHYHOVWKH\DUH
LQJHQHUDOWKHEHVWUHSRUWLQJVWUDWHJ\7KHXVHRIFRQILGHQFHLQWHUYDOVLV
WKHUHIRUHVWURQJO\UHFRPPHQGHG
6366PDNHVWKHUHSRUWLQJRIFRQILGHQFHLQWHUYDOVHDV\EHFDXVHLWSURYLGHVWKHPDV
SDUWRIWKHGHIDXOWRXWSXWIRUPRVWSURFHGXUHV,QVSLWHRIWKLVWKHXVXDOSUDFWLFHLQERWK
DSSOLHGOLQJXLVWLFVDQGHGXFDWLRQDOSV\FKRORJ\KDVEHHQWRLQGLFDWHVWDWLVWLFDO
VLJQLILFDQFHRQO\ZLWKRXWSUHVHQWLQJWKHDFWXDOFRQILGHQFHLQWHUYDOV
(IIHFWVL]H

:K\GRZHQHHGWRERWKHUDERXWµHIIHFWVL]H¶ RUDVLWLVDOVRFDOOHGµVWUHQJWKRI
DVVRFLDWLRQ¶ LIDUHVXOWLVVWDWLVWLFDOO\VLJQLILFDQW"7KHUHDVRQLVWKDWµVWDWLVWLFDO
VLJQLILFDQFH¶RQO\PHDQVWKDWDQREVHUYHGSKHQRPHQRQLVPRVWSUREDEO\WUXHLQWKH
SRSXODWLRQ DQGQRWMXVWLQRXUVDPSOH EXWZKDWLVWUXHPD\QRWQHFHVVDULO\EH
LPSRUWDQW)RUH[DPSOHZLWKYHU\ODUJHVDPSOHVL]HVHYHQWLQ\GLIIHUHQFHVEHWZHHQ
WZRJURXSVPLJKWUHDFKVWDWLVWLFDOVLJQLILFDQFHHYHQWKRXJKWKH\PD\KDYHQRSUDFWLFDO
RUWKHRUHWLFDOLPSRUWDQFHZKDWVRHYHU7KLVUDLVHVDEURDGHULVVXHRIWKHRYHUDOOYDOXHRI
VLJQLILFDQFHWHVWLQJDQGLQGHHGWKHSDVWGHFDGHKDVVHHQDKHDWHGGHEDWHRYHUWKLVLVVXH
)RUDUHFHQWUHYLHZVHH%DOOXHUNDHWDO $VDUHVXOWRIWKLVFRQWURYHUV\WKH
$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQ $3$ FRQYHQHGDFRPPLWWHHFDOOHGWKHµ7DVN
)RUFHRQ6WDWLVWLFDO,QIHUHQFH¶ 7)6, ZKLFKFRQFOXGHGWKDWUHVHDUFKHUVVKRXOGDOZD\V
SUHVHQWHIIHFWVL]HVIRUSULPDU\RXWFRPHV :LONLQVRQDQG7)6,
7KXVHIIHFWVL]HVQHHGWREHFRPSXWHGLQRUGHUWRSURYLGHLQIRUPDWLRQDERXWWKH
PDJQLWXGHRIDQREVHUYHGSKHQRPHQRQDQGGRLQJVRZLOODOVRHQDEOHXVWRFRPSDUH
WKHUHVXOWVUHSRUWHGLQGLIIHUHQWVWXGLHVEHFDXVHHIIHFWVL]HLQGLFHVDUHWUDQVIRUPDWLRQV
RQWRDFRPPRQVFDOH,QGHHG:LONLQVRQDQG7)6, HPSKDVL]HGWKDWVFKRODUV
VKRXOGUHSRUWDQGLQWHUSUHWHIIHFWVL]HVLQWKHFRQWH[WRISUHYLRXVO\UHSRUWHGHIIHFWVDV
WKLVZRXOGPDNHLWSRVVLEOHWRHYDOXDWHWKHVWDELOLW\RIUHVXOWVDFURVVVDPSOHVGHVLJQV
DQGDQDO\VHV+DYLQJVDLGWKLVZHPXVWDOVRUHDOL]HWKHPDLQVKRUWFRPLQJRIHIIHFW
VL]HQDPHO\WKDWWKHUHDUHQRXQLYHUVDOO\DFFHSWHGDQGVWUDLJKWIRUZDUGLQGLFHVWR
GHVFULEHLW,QVWHDGWKHFDOFXODWLRQGLIIHUVIRUYDULRXVVWDWLVWLFDOWHVWVDQGHYHQZLWK
UHJDUGWRRQHWHVWZHVRPHWLPHVILQGDOWHUQDWLYHDQGFRPSHWLQJHIIHFWVL]HLQGLFDWRUV
)RUDGHWDLOHGRYHUYLHZVHH*ULVVRPDQG.LP 7KHSUREOHPRIGLYHUVLW\LVZHOO
LOOXVWUDWHGE\WKHIROORZLQJGHVFULSWLRQIURPWKH$3$3XEOLFDWLRQ0DQXDO ±
ZKLFKLVOLNHO\WRJRRYHUWKHKHDGVRIPRVWUHVHDUFKHUVZKRDUHQRWVWDWLVWLFV
H[SHUWVWKHPVHOYHV
<RXFDQHVWLPDWHWKHPDJQLWXGHRIWKHHIIHFWRUWKHVWUHQJWKRIWKHUHODWLRQVKLS
ZLWKDQXPEHURIFRPPRQHIIHFWVL]HHVWLPDWHVLQFOXGLQJ EXWQRWOLPLWHGWR U
ȘȦ5ĳ&UDPHU¶V9.HQGDOO¶V:&RKHQ¶VGDQGț*RRGPDQ.UXVNDO¶VȜ
DQGȖ-DFREVRQDQG7UXD[¶VDQG.HQGDOO¶VSURSRVHGPHDVXUHVRIFOLQLFDO
VLJQLILFDQFHDQGWKHPXOWLYDULDWH5R\¶VĬDQGWKH3LOODL%DUWOHWW9
+RZPXFKWRJHQHUDOL]H
$VPHQWLRQHGHDUOLHUPRUHWKDQRQFHRQHRIWKHNH\LVVXHVLQTXDQWLWDWLYHGDWDDQDO\VLV
LVWRGHFLGHZKHWKHURUQRWZHFDQJHQHUDOL]HRXUUHVXOWVWRRWKHUFRQWH[WVDQGLIVR
KRZPXFK7KHVWDQGDUGDGYLFHZHILQGLQVWDWLVWLFVWH[WVXVXDOO\DPRXQWVWRWKHEURDG
DQGUDWKHUXQKHOSIXOJXLGHOLQH'RQRWRYHUJHQHUDOL]H+RZHYHUUHVHDUFKLQPRVW
FDVHVLVDOODERXWWKHQHHGWRSURGXFHJHQHUDOL]DEOHILQGLQJV$IWHUDOOZLWKWKHSRVVLEOH
H[FHSWLRQRIµDFWLRQUHVHDUFK¶ VHH6HFWLRQ DSSOLHGOLQJXLVWVUDUHO\LQYHVWLJDWHD
VDPSOHZLWKWKHVROHSXUSRVHRIZDQWLQJWRNQRZPRUHRQO\DERXWWKHSDUWLFXODUSHRSOH
XQGHULQYHVWLJDWLRQ,QVWHDGZKDWZHQRUPDOO\ZDQWWRGRLQTXDQWLWDWLYHUHVHDUFKLV

ILQGRXWPRUHDERXWWKHSRSXODWLRQWKDWLVDERXWDOOWKHVLPLODUSHRSOHLQWKHZRUOG6R
WKHUHDOTXHVWLRQLVZKDWH[DFWO\µRYHU¶PHDQVLQWKHµ'RQRWRYHUJHQHUDOL]H¶ZDUQLQJ
,WZRXOGDJDLQEHHDV\WRJLYHDOHVVWKDQXVHIXOWKRXJKWHFKQLFDOO\FRUUHFW
GHILQLWLRQRIµRYHUJHQHUDOL]DWLRQ¶QDPHO\WKDWLWRFFXUVZKHQZHJHQHUDOL]HWKH
ILQGLQJVWRDSRSXODWLRQWKDWRXUVDPSOHLVQRWUHSUHVHQWDWLYHRI 6HH6HFWLRQ
7KLVVWDWHVLQHIIHFWWKDWLIZHH[DPLQHVD\SULPDU\VFKRROSXSLOVZHVKRXOGQRW
JHQHUDOL]HRXUILQGLQJVWRVHFRQGDU\RUXQLYHUVLW\VWXGHQWV7KHUHLVQRTXHVWLRQDERXW
WKHYDOLGLW\RIWKLVFODLPDQG\HWLWDYRLGVWKHFUX[RIWKHSUREOHPZKLFKLVWKDWLIZH
ZHUHWRREVHUYHWKLVJXLGHOLQHWRRFORVHO\IHZ LIDQ\ VWXGLHVLQWKHHGXFDWLRQDO
SV\FKRORJLFDOOLWHUDWXUHFRXOGVSHDNDERXWµVWXGHQWV¶LQJHQHUDO,WLVFOHDUWKDWKDUGO\
DQ\LQYHVWLJDWLRQLVVXIILFLHQWO\ODUJHVFDOHWRLQFOXGHUHSUHVHQWDWLYHVRIHYHU\PDLQDJH
JURXSHWKQLFLW\VFKRROW\SHDQGVXEMHFWPDWWHULQDVLQJOHVWXG\ MXVWWROLVWIRXUNH\
IDFWRUV ²\HWWKHGLVFXVVLRQVRIWKHILQGLQJVDUHUDUHO\UHVWULFWHGWRWKHSDUWLFXODU
VXESRSXODWLRQLQTXHVWLRQ
+DYLQJVDLGWKLV,VWLOOEHOLHYHWKDWZHVKRXOGEHZDUHRIRYHUJHQHUDOL]DWLRQVEXWLQ
WKHDEVHQFHRIKDUGDQGIDVWUXOHVDERXWZKDWFRQVWLWXWHVµRYHU¶JHQHUDOL]DWLRQZHQHHG
WRVWULYHWRILQGDGHOLFDWHEDODQFHEHWZHHQWKHIROORZLQJWZRFRQVLGHUDWLRQVRQWKHRQH
KDQGZHPD\ZLVKWREHDEOHWRVD\VRPHWKLQJRIDEURDGHUUHOHYDQFH VLQFHLWPD\
VHYHUHO\UHGXFHRXUDXGLHQFHLIZHOLPLWRXUGLVFXVVLRQWRYHU\VSHFLILFVXEJURXSV 2Q
WKHRWKHUKDQGELJFODLPVFDQXVXDOO\EHPDGHRQO\RQWKHEDVLVRIELJVWXGLHV+DYLQJ
VDLGWKLVVRPHFODVVLFVWXGLHVLQWKHUHVHDUFKOLWHUDWXUHGLGFRQILQHWKHLUIRFXVWR
H[WUHPHO\OLPLWHGWDUJHWLVVXHVDQGVRPHIDPRXVELJFODLPVZHUHLQGHHGPDGHEDVHG
RQVPDOOVWXGLHV
6RWKHRQO\FRQFOXVLRQ,FDQRIIHUKHUHLVWKDWUHVHDUFKHUVQHHGWRH[HUFLVHJUHDW
FDXWLRQZKHQSLWFKLQJWKHOHYHORIJHQHUDOL]DWLRQLQWKHLUUHVHDUFKUHSRUWV/D]DUDWRQ
ZDUQVXVWKDWXVLQJKLJKSRZHUHGSDUDPHWULFSURFHGXUHVPD\HDVLO\WHPSW
UHVHDUFKHUVWRRYHUJHQHUDOL]HWKHLUUHVXOWVDQGWRPDNHJUDQGFODLPVUHJDUGLQJWKHLU
ILQGLQJVµFODLPVWKDWIDUH[FHHGZKDWLVSHUPLWWHGE\WKHLUPHWKRGRORJLFDO
XQGHUSLQQLQJV¶ 7DURQHFLWHGE\/D]DUDWRQ 2QWKHRWKHUKDQGDORQJ
ZLWKWKHµ7DVN)RUFHRQ6WDWLVWLFDO,QIHUHQFH¶RIWKH$PHULFDQ3V\FKRORJLFDO
$VVRFLDWLRQ :LONLQVRQDQG7)6, ,ZRXOGHQFRXUDJHUHVHDUFKHUVQRWWREH
DIUDLGµWRH[WHQG\RXULQWHUSUHWDWLRQVWRDJHQHUDOFODVVRUSRSXODWLRQLI\RXKDYH
UHDVRQVWRDVVXPHWKDW\RXUUHVXOWVDSSO\¶
'HVFULSWLYHVWDWLVWLFV
$VPHQWLRQHGDERYHGHVFULSWLYHVWDWLVWLFVKHOSXVVXPPDUL]HILQGLQJVE\GHVFULELQJ
JHQHUDOWHQGHQFLHVLQWKHGDWDDQGWKHRYHUDOOVSUHDGRIWKHVFRUHV LHKRZYDULHGWKH
VFRUHVDUH 6XFKVWDWLVWLFVDUHLQGLVSHQVDEOHZKHQZHVKDUHRXUUHVXOWV IRUH[DPSOH
ZKHQZHGHVFULEHWKHSDUWLFLSDQWVLQWKHPHWKRGRORJ\VHFWLRQRIDSDSHU DQGWKH\DOVR
IRUPWKHEDVLVRIIXUWKHULQIHUHQWLDOVWDWLVWLFV7KHWZRPDLQFDWHJRULHVRIGHVFULSWLYH
VWDWLVWLFVDUHWKHµPHDVXUHVRIFHQWUDOWHQGHQF\¶DQGWKHµPHDVXUHVRIYDULDELOLW\¶

0HDVXUHVRIFHQWUDOWHQGHQF\GHVFULEHWKHGDWDVHWZLWKDVLQJOHQXPHULFDOYDOXH
7KHWKUHHPRVWFRPPRQO\XVHGPHDVXUHVDUHWKHIROORZLQJ
x µ0HDQ¶ZKLFKLVWKHDYHUDJHRIWKHVFRUHV7KLVLVWKHPRVWFRPPRQGHVFULSWLYH
PHDVXUHEHFDXVHLWWDNHVLQWRDFFRXQWDOOWKHVFRUHVEXWLWKDVDGLVDGYDQWDJH
QDPHO\WKDWH[WUHPHVFRUHVVNHZLWFRQVLGHUDEO\
x µ0HGLDQ¶ZKLFKLVWKHµILIWLHWKSHUFHQWLOH¶WKDWLVWKHPLGGOHSRLQWLQDVHWRI
VFRUHVWKDWKDYHEHHQDUUDQJHGLQDUDQNRUGHU7KLVPHDVXUHLVQRWVHQVLWLYHWR
H[WUHPHVEXWLWVDFWXDOYDOXHLVRQO\GHSHQGHQWRQWKHPLGGOHVFRUHV
x µ0RGH¶ZKLFKLVWKHPRVWIUHTXHQWO\RFFXUULQJVFRUH7KLVLVDVWUDLJKWIRUZDUG
PHDVXUHEXWLWFDQRQO\SURYLGHDJRRGHVWLPDWHRIWKHFHQWUDOWHQGHQF\RIWKH
GLVWULEXWLRQLIWKHUHLVDYDOXHWKDWLQGHHGVWDQGVRXWZLWKLWVIUHTXHQF\
µ0HDVXUHVRIYDULDELOLW\¶SURYLGHLQGLFHVRIKRZGLVSHUVHGRUYDULHGWKHVFRUHVDUH
LQDGDWDVHW7KH\LQFOXGHWKHIROORZLQJPHDVXUHV
x µ5DQJH¶ZKLFKLVWKHGLIIHUHQFHEHWZHHQWKHKLJKHVWDQGWKHORZHVWVFRUH
x µ9DULDQFH¶DQGLWVVTXDUHURRWWKHµVWDQGDUGGHYLDWLRQ¶ZKLFKDUHLQGLFDWRUVRIWKH
DYHUDJHGLVWDQFHRIWKHVFRUHVIURPWKHPHDQ7KH\DUHKLJKLIWKHVDPSOHLV
KHWHURJHQHRXVDQGFRQWDLQVH[WUHPHVFRUHVZKHUHDVWKH\DUHORZLQD
KRPRJHQHRXVVDPSOHZLWKDOOWKHVFRUHVFOXVWHUHGDURXQGWKHPHDQ
'HVFULSWLYHVWDWLVWLFVFDQEHREWDLQHGE\PRUHWKDQRQH6366SURFHGXUHWKHPRVW
FRPPRQO\XVHGRQHVDUH)5(48(1&,(6DQG'(6&5,37,9(6,QUHVHDUFKDUWLFOHV
DQGWKHVHVGLVVHUWDWLRQVVXFKVWDWLVWLFVDUHDOPRVWDOZD\VUHSRUWHGLQWDEOHV7DEOH
RIIHUVDW\SLFDOVDPSOHIROORZLQJWKH$3$IRUPDW ZKLFKGRHVQRWFRQWDLQDQ\YHUWLFDO
OLQHV $VFDQEHVHHQWKHEDVLFVWDWLVWLFVWKDWDUHXVXDOO\SURYLGHGDUHWKHPHDQ 0
WKHVWDQGDUGGHYLDWLRQ 6' DQGWKHQXPEHURISDUWLFLSDQWV Q 3OHDVHQRWHWKDWLQ
WDEOHVZHXVXDOO\RPLWµ¶IURPWKHEHJLQQLQJRIILJXUHVVPDOOHUWKDQ LHZHZULWH
LQVWHDGRI
/DQJXDJH$Q[LHW\6FRUHVRI+XQJDULDQ6HFRQGDU\6FKRRO/HDUQHUVRI)LQQLVK%URNHQ'RZQE\
3URILFLHQF\/HYHO
/DQJXDJHSURILFLHQF\OHYHO
%HJLQQHU ,QWHUPHGLDWH $GYDQFHG
6H[ 0 6' Q 0 6' Q 0 6' Q
)HPDOH

0DOH
7DEOH6DPSOHWDEOHUHSRUWLQJGHVFULSWLYHVWDWLVWLFV
&RPSDULQJWZRJURXSVWWHVWV
&RPSDULQJYDULRXVJURXSVRISHRSOHLVWKHPRVWFRPPRQVWDWLVWLFDOSURFHGXUHLQ
DSSOLHGOLQJXLVWLFUHVHDUFK /D]DUDWRQ ,QVWDWLVWLFVWKHUHDUHGLIIHUHQWPHWKRGV
DYDLODEOHIRUVXFKFRPSDULVRQVGHSHQGLQJRQWKHQXPEHURIJURXSVZHZLVKWRDQDO\VH
,IZHZDQWWRFRPSDUHWZRJURXSV IRUH[DPSOHPHQDQGZRPHQ ZHFRPSXWHDµW
WHVW¶ZKLFKLVWKHVXEMHFWRIWKLVVHFWLRQ7KHSURFHGXUHWREHDSSOLHGZLWKPRUHWKDQ
WZRJURXSVLVWKHµDQDO\VLVRIYDULDQFH¶ $129$ ZKLFKZLOOEHGLVFXVVHGLQ6HFWLRQ

,IZHWDNHDQ\WZRVHWVRIVFRUHVZHDUHERXQGWRILQGVRPHGLIIHUHQFHLQWKHUDZ
VFRUHVEXWZHFDQQRWDXWRPDWLFDOO\DVVXPHWKDWWKHREVHUYHGGLIIHUHQFHUHIOHFWVDQ\
µUHDO¶GLIIHUHQFHWKXVZHQHHGWWHVWVWDWLVWLFVWRFKHFNZKHWKHUZHKDYHJRWD
JHQHUDOL]DEOHUHVXOWRUZKHWKHUWKHVFRUHLVOLNHO\WREHPHUHO\DQDUWHIDFWRIUDQGRP
YDULDWLRQ7KHUHDUHWZRPDLQW\SHVRIWWHVWV
x µ,QGHSHQGHQWVDPSOHVWWHVWV¶DUHIRUUHVHDUFKGHVLJQVZKHUHZHDUHFRPSDULQJWKH
UHVXOWVRIJURXSVWKDWDUHLQGHSHQGHQWRIHDFKRWKHU IRUH[DPSOH&ODVVDQG&ODVV

x µ3DLUHGVDPSOHVWWHVWV¶ DOVRNQRZQDVµPDWFKHGWWHVWV¶µPDWFKHGSDLUVWWHVWV¶RU
µSDLUVWWHVWV¶ DUHIRUUHVHDUFKGHVLJQVZKHUHZHZDQWWRFRPSDUHWZRVHWVRI
VFRUHV LHWZRYDULDEOHV REWDLQHGIURPWKHVDPHJURXS IRUH[DPSOHWKHOHDUQHUV¶
FRXUVHJUDGHVLQKLVWRU\DQG(QJOLVK RUZKHQWKHVDPHSDUWLFLSDQWVDUHPHDVXUHG
PRUHWKDQRQFH IRUH[DPSOHWHVWVFRUHVEHIRUHDQGDIWHUDFRXUVH 7KDWLVWKLV
SURFHGXUHH[DPLQHVGLIIHUHQWUHVXOWVREWDLQHGIURPWKHVDPHJURXS
%RWKW\SHVDUHVLPLODULQWKDWZHWHVWZKHWKHUWKHGLIIHUHQFHEHWZHHQWZRVHWVRI
VFRUHVLVELJHQRXJKWRUHDFKVWDWLVWLFDOVLJQLILFDQFH+RZHYHUEHFDXVHWKHILUVWW\SH
LQYROYHVWZRVHSDUDWHJURXSVRISDUWLFLSDQWVZKLOHWKHVHFRQGLQYROYHVRQO\RQHJURXS
6366XVHVGLIIHUHQWPDWKHPDWLFDOFRPSXWDWLRQVIRUWKHPWKHUHIRUHZKHQZHUXQDW
WHVWZHKDYHWRVHOHFWLQDGYDQFHZKLFKSURFHGXUHLVUHOHYDQWWRXV
,OOXVWUDWLRQ
6366SURYLGHVDVDPSOHGDWDVHWIRUSUDFWLFHFRQWDLQLQJGDWDIURPWKH*HQHUDO86
6RFLDO6XUYH\2QHYDULDEOHLQWKLVVXUYH\LQYROYHVWKHOHYHORIµKDSSLQHVV¶H[SHULHQFHG
E\WKHUHVSRQGHQWV/HWXVH[DPLQHZKHWKHUWKHUHZDVDVLJQLILFDQWGLIIHUHQFHLQWKH86
LQEHWZHHQPHQDQGZRPHQLQWKLVUHVSHFW7RGRVRZHQHHGWRFRPSXWHDQ
LQGHSHQGHQWVDPSOHVWWHVW7KHµV\QWD[ILOH¶ $1$/<=(ĺ&203$5(0($16ĺ
,1'(3(1'(176$03/(677(673$67( FRQWDLQVWKHIROORZLQJVLPSOH
FRPPDQGV

77(67
*52836 VH[
0,66,1* $1$/<6,6
9$5,$%/(6 KDSS\
&5,7(5,$ &,
7DEOHSUHVHQWVWKH6366RXWSXW

77HVW
*URXS6WDWLVWLFV
6WG(UURU
VH[ 1 0HDQ 6WG'HYLDWLRQ 0HDQ
KDSS\

,QGHSHQGHQW6DPSOHV7HVW
/HYHQH V7HVWIRU
(TXDOLW\RI9DULDQFHV WWHVWIRU(TXDOLW\RI0HDQV
&RQILGHQFH
,QWHUYDORIWKH
0HDQ 6WG(UURU 'LIIHUHQFH
) 6LJ W GI 6LJ WDLOHG 'LIIHUHQFH 'LIIHUHQFH /RZHU 8SSHU
KDSS\ (TXDOYDULDQFHV

DVVXPHG
(TXDOYDULDQFHV

QRWDVVXPHG

7DEOH6DPSOH6366RXWSXWUHSRUWLQJWWHVWVWDWLVWLFV
,QWKHILUVWWDEOHZHFDQVHHWKDWPHQ LQGLFDWHGZLWKµ¶LQWKHµVH[¶YDULDEOH
GLVSOD\DVRPHZKDWORZHUPHDQVFRUHIRUKDSSLQHVVWKDQZRPHQ YV DQGWKH
NH\TXHVWLRQLVZKHWKHUWKLVGLIIHUHQFHUHDFKHVVWDWLVWLFDOVLJQLILFDQFH%HFDXVHWKH
SUREDELOLW\ILJXUH²PDUNHGDVµ6LJ¶LQWKHVHFRQGWDEOH²LVVPDOOHUWKDQ LWLV
WKHGLIIHUHQFHFDQEHFRQVLGHUHGWUXH 1RWHWKDWLQWKHVHFRQGWDEOHEHFDXVHWKH
/HYHQH¶VWHVWGRHVQRWVKRZDQ\VLJQLILFDQWGLIIHUHQFHZHFDQDVVXPHHTXDOYDULDQFH
LQWKHWZRJURXSVDQGWKHUHIRUHWKHILUVWOLQHRIWKHUHVXOWVLVUHOHYDQW,IWKH/HYHQHWHVW
VKRZHGDVLJQLILFDQWGLIIHUHQFH²LHLILWZDVVPDOOHUWKDQ²WKLVZRXOGLQGLFDWH
WKDWHTXDOYDULDQFHFDQQRWEHDVVXPHGDQGWKHQZHZRXOGQHHGWRFRQVLGHUWKHILJXUHV
LQWKHERWWRPURZ :HVKRXOGQRWHKHUHWKDWLQWKLVSDUWLFXODUGDWDVHWWKHFRGLQJZDV
VXFKWKDWWKHORZHUWKHVFRUHWKHKLJKHUWKHOHYHORIKDSSLQHVVDQGWKHUHIRUHZHFDQ
FRQFOXGHWKDWLQ$PHULFDQPHQZHUHUHSRUWHGWREHKDSSLHUWKDQWKHLUIHPDOH
FRXQWHUSDUWV
7KHPRVWFRPPRQHIIHFWVL]HLQGLFDWRUIRULQGHSHQGHQWVDPSOHVWWHVWVLVµHWD
VTXDUHG¶ LWFDQEHLQWHUSUHWHGDVWKHSHUFHQWDJHRIWKHYDULDQFHLQWKHWDUJHWYDULDEOH
H[SODLQHGE\WKHJURXSLQJYDULDEOH 8QIRUWXQDWHO\6366GRHVQRWSURYLGHWKLVILJXUH
EXWLWLVHDV\WRFRPSXWHXVLQJWKHIROORZLQJIRUPXOD 3DOODQW1UHIHUVWRWKHVL]H
RIWKHJURXSV
BBBBBWBBBBB
W 11±

,QRXUFXUUHQWH[DPSOHHWDVTXDUHGLV7KHXVXDOLQWHUSUHWDWLRQRIHWDVTXDUHGLV
WKDW VPDOOHIIHFW PRGHUDWHHIIHFWDQG ODUJHHIIHFWZKLFKPHDQVWKDW
RXUHIIHFWVL]HLVYHU\VPDOOLQGLFDWLQJWKDWHYHQWKRXJKWKHGLIIHUHQFHRIKDSSLQHVVLV
VLJQLILFDQWLWVPDJQLWXGHLVLJQRUDEOH6RLQHIIHFW$PHULFDQPHQDUHQRWKDSSLHUWKDQ
$PHULFDQZRPHQ
5HSRUWLQJWWHVWUHVXOWV
,QUHVHDUFKDUWLFOHVZHW\SLFDOO\ILQGWKDWWWHVWUHVXOWVDUHUHSRUWHGHPEHGGHGZLWKLQWKH
WH[WUDWKHUWKHQSUHVHQWHGLQDVHSDUDWHWDEOH7KHIROORZLQJH[WUDFWSUHVHQWVDW\SLFDO
UHVXOWVXPPDU\LWVKRZVWKDWZHQHHGWRSURYLGHWKHPHDQVDQGVWDQGDUGGHYLDWLRQV
WKHVHDUHVRPHWLPHVSUHVHQWHGLQDWDEOH WKHWYDOXHWKHGHJUHHRIIUHHGRPLQ
EUDFNHWVWKHSUREDELOLW\ILJXUHDQGWKHHIIHFWVL]H 7KHµGHJUHHRIIUHHGRP¶LVD
VWDWLVWLFDOILJXUHWKDWURXJKO\UHIOHFWVWKHVDPSOHVL]HDOWKRXJKLWGRHVQRWFRLQFLGHZLWK
LWH[DFWO\EHFDXVHLQWKLVFDVHWKHQXPEHURIJURXSVKDVEHHQVXEWUDFWHGIURPLW
:HKDYHFDUULHGRXWDQLQGHSHQGHQWVDPSOHVWWHVWWRFRPSDUHWKHKDSSLQHVVVFRUHV
IRU$PHULFDQPHQDQGZRPHQ7KHUHZDVDVLJQLILFDQWGLIIHUHQFHLQVFRUHVIRUPHQ
0 6' DQGZRPHQ 0 6' W SEXW
WKHPDJQLWXGHRIWKHGLIIHUHQFHLQWKHPHDQVZDVYHU\VPDOO HWDVTXDUHG
ZLWKVH[H[SODLQLQJRQO\SHUFHQWRIWKHYDULDQFHLQKDSSLQHVV
,IZHKDYHDVHULHVRIWWHVWVWDWLVWLFVWKDWZRXOGPDNHUHSRUWLQJWKHPLQWKHWH[W
FRQIXVLQJZHFDQLQFOXGHWKHPLQDWDEOH7DEOHSUHVHQWVSDLUHGVDPSOHVWWHVW
VWDWLVWLFVIRUWKHWDEOHZHQHHGWRSURYLGHWKHWRWDOQXPEHURISDUWLFLSDQWV Q WKH
PHDQVRIWKHWZRYDULDEOHVFRPSDUHG 0 WKHVWDQGDUGGHYLDWLRQV 6' WKHGHJUHHVRI
IUHHGRP G WKHWYDOXHDQGWKHHIIHFWVL]H,QRUGHUWRVDYHVSDFHLQWDEOHVZHGRQRW
XVXDOO\SUHVHQWVHSDUDWHSUREDELOLW\YDOXHVEXWLQVWHDGLQGLFDWHWKHOHYHORIVLJQLILFDQFH
E\DGGLQJDVWHULVNVWRWKHWYDOXHVWKDWKDYHEHHQIRXQGVWDWLVWLFDOO\VLJQLILFDQW7KHVH
DVWHULVNVDUHDOZD\VH[SODLQHGDWWKHERWWRPRIWKHWDEOH DVLQ7DEOH
3DLUHG6DPSOHV77HVWVRIWKH6WXGHQWV¶3HUIRUPDQFHRQ'LIIHUHQW/LVWHQLQJDQG5HDGLQJ7HVWV 1
(IIHFW
0 6' G W VL]HD
/LVWHQLQJFRPSUHKHQVLRQ
7HVW
7HVW

5HDGLQJFRPSUHKHQVLRQ
7HVW
7HVW
S
D
(WDVTXDUHG
7DEOH6DPSOHWDEOHUHSRUWLQJWWHVWVWDWLVWLFV
&RPSDULQJPRUHWKDQWZRJURXSVDQDO\VLVRI
YDULDQFH $129$
µ$QDO\VLVRIYDULDQFH¶ PRUHSUHFLVHO\µRQHZD\DQDO\VLVRIYDULDQFH¶XVXDOO\UHIHUUHG
WRDVµ$129$¶ LVYHU\VLPLODUWRWKHWWHVWEXWLQWKLVFDVHZHDVVHVVWKHVLJQLILFDQFH
RIWKHGLIIHUHQFHVLQWKHPHDQVRIPRUHWKDQWZRJURXSV IRUH[DPSOHWKHPHDQ
DFKLHYHPHQWRIWKUHHGLIIHUHQWFODVVJURXSV ZHFDQDOVRUXQ$129$ZLWKRQO\WZR
JURXSVLQZKLFKFDVHWKHUHVXOWZLOOEHWKHVDPHDVFRUUHVSRQGLQJWWHVWVWDWLVWLFV,QD
VXUYH\RIHPSLULFDOVWXGLHVWKDWDSSHDUHGLQIRXUPDMRUDSSOLHGOLQJXLVWLFMRXUQDOV
EHWZHHQDQG/D]DUDWRQ IRXQGWKDW$129$ZDVWKHPRVWIUHTXHQWO\
XVHGVWDWLVWLFDOSURFHGXUHDFFRXQWLQJIRURYHUSHUFHQWRIWKHDQDO\VHVUHSRUWHGLQ
WKHDUWLFOHV$VVKHFRQFOXGHGµRQHLPSOLFDWLRQRIWKLVILQGLQJLVWKDWLIDSSOLHG
OLQJXLVWVDUHWROHDUQWRFDUU\RXWDQGRUWRLQWHUSUHWMXVWRQHVWDWLVWLFDOSURFHGXUHWKDW
SURFHGXUHVKRXOGEH$129$¶ S 7RGR$129$ZHQHHGWZRYDULDEOHV
x $µGHSHQGHQWYDULDEOH¶ZKLFKLVWKHWDUJHWYDULDEOHWREHFRPSDUHGVXFKDVVFKRRO
DFKLHYHPHQW
x $QµLQGHSHQGHQWYDULDEOH¶ FDOOHGLQ6366WKHµIDFWRU¶ ZKLFKLVWKHJURXSLQJ
YDULDEOHWKDWKDVDVPDQ\YDOXHVDVWKHQXPEHURIJURXSVZHZDQWWRFRPSDUH)RU
H[DPSOHZHPD\KDYHDYDULDEOHFDOOHGµFODVVJURXS¶ZLWKWKUHHYDOXHV DQG
UHSUHVHQWLQJWKUHHFODVVHVZKLFKZLOODOORZXVWRUXQ$129$WRFRPSDUHWKHVH
FODVVHV
$129$SURGXFHVPXOWLSOHFRPSDULVRQVLQWZRVWHSV)LUVWDQ)YDOXHLVFRPSXWHG
DQGFKHFNHGIRUVLJQLILFDQFH,IWKHYDOXHLVVLJQLILFDQWLWPHDQVWKDWWKHUHLVDWOHDVW
RQHVLJQLILFDQWGLIIHUHQFHDPRQJVWWKHJURXSPHDQV+RZHYHUEHFDXVHZLWKPRUHWKDQ
WZRJURXSVZHKDYHPRUHWKDQRQHFRQWUDVW LHZLWKWKUHHJURXSV²$%DQG&²ZH
KDYHWKUHHFRQWUDVWVEHWZHHQ$DQG%%DQG&RU&DQG$ ZHQHHGDVHFRQGVWHSWR
GHWHUPLQHZKLFKFRQWUDVW V LVDUHVLJQLILFDQW)RUWKLVSXUSRVHZHFRPSXWHDSRVWKRF
WHVWZKLFKLVRIIHUHGE\6366DVDQRSWLRQIRU$129$XQGHUµ3267+2&¶6HYHUDO
W\SHVRISRVWKRFWHVWVDUHOLVWHGW\SLFDOO\GHOLYHULQJDOPRVWH[DFWO\WKHVDPHUHVXOWV

7KHPRVWFRPPRQO\XVHGRQHVDUH/6'61.DQG7XNH\ ,WLVLPSRUWDQWWRUHLWHUDWH
WKDWZHFRQGXFWSRVWKRFWHVWVLIDQGRQO\LIWKHLQLWLDO)YDOXHIRUWKH$129$ZDV
VLJQLILFDQW
,OOXVWUDWLRQ
/HWXVXVHDJDLQWKH*HQHUDO866RFLDO6XUYH\,IZHORRNDWWKH5$&(YDULDEOH
UHIHUULQJWRWKHHWKQLFEDFNJURXQGRIWKHUHVSRQGHQWV ZHFDQVHHWKDWWKHUHVSRQGHQWV
ZHUHGLYLGHGLQWRWKUHHFDWHJRULHVZKLWH EODFN DQGRWKHU /HWXVH[DPLQH
ZKHWKHUWKHWKUHHUDFHW\SHVGLIIHULQWHUPVRIWKHLUKDSSLQHVV
7KHµV\QWD[ILOH¶ $1$/<=(ĺ&203$5(0($16ĺ21(:$<
$129$3$67( LVMXVWDVVWUDLJKWIRUZDUGDVZLWKWKHWWHVWV
21(:$<
KDSS\%<UDFH
67$7,67,&6'(6&5,37,9(6
0,66,1*$1$/<6,6
3267+2& 61.$/3+$

7KH6366RXWSXWLVSULQWHGEHORZLQ7DEOH$VWKHSUREDELOLW\ILJXUH PDUNHGDV
µ6LJ¶LQWKHVHFRQGWDEOH VKRZVWKH)YDOXHLVVWDWLVWLFDOO\VLJQLILFDQWDWWKHS
OHYHOZKLFKLVLPSUHVVLYH7KHSRVWKRFWHVW 61. UHYHDOVWKDWWKHGLIIHUHQFHOLHV
EHWZHHQZKLWHSHRSOHDQGWKHRWKHUUDFHV DVWKH\DUHSUHVHQWHGLQWZRGLIIHUHQW
FROXPQV ZKLWHSHRSOHDUHVLJQLILFDQWO\KDSSLHUWKDQPHPEHUVRIWKHQRQZKLWHUDFHV
EODFNDQGRWKHU ZKHUHDVWKHODWWHUWZRJURXSVGRQRWGLIIHUIURPHDFKRWKHU
VLJQLILFDQWO\DQGWKHUHIRUHPXVWEHFRQVLGHUHGHTXDO :HVKRXOGQRWHWKDWWKHFRGLQJ
ZDVVXFKWKDWWKHORZHUWKHVFRUHWKHKLJKHUWKHOHYHORIKDSSLQHVV
2QHZD\
'HVFULSWLYHV
KDSS\
&RQILGHQFH,QWHUYDOIRU
0HDQ
1 0HDQ 6WG'HYLDWLRQ 6WG(UURU /RZHU%RXQG 8SSHU%RXQG 0LQLPXP 0D[LPXP

7RWDO

$129$
KDSS\
6XPRI
6TXDUHV GI 0HDQ6TXDUH ) 6LJ
%HWZHHQ*URXSV
:LWKLQ*URXSV
7RWDO

3RVW+RF7HVWV

+RPRJHQHRXV6XEVHWV
KDSS\
DE
6WXGHQW1HZPDQ.HXOV
6XEVHWIRUDOSKD
UDFH 1

6LJ
0HDQVIRUJURXSVLQKRPRJHQHRXVVXEVHWVDUHGLVSOD\HG
D 8VHV+DUPRQLF0HDQ6DPSOH6L]H
E 7KHJURXSVL]HVDUHXQHTXDO7KHKDUPRQLFPHDQ
RIWKHJURXSVL]HVLVXVHG7\SH,HUURUOHYHOVDUH
QRWJXDUDQWHHG

7DEOH6DPSOH6366RXWSXWUHSRUWLQJ$129$VWDWLVWLFV

/LNHZLWKWKHWWHVWWKHPRVWFRPPRQHIIHFWVL]HLQGLFDWRUIRU$129$LVµHWD
VTXDUHG¶ LWFDQEHLQWHUSUHWHGDVWKHSHUFHQWDJHRIWKHYDULDQFHLQWKHWDUJHWYDULDEOH
H[SODLQHGE\WKHJURXSLQJYDULDEOH 8QIRUWXQDWHO\6366GRHVQRWSURYLGHWKLVILJXUH
EXWLWLVHDV\WRFRPSXWHXVLQJWKHIROORZLQJIRUPXOD 3DOODQW
VXPRIVTXDUHVEHWZHHQJURXSV
WRWDOVXPRIVTXDUHV
7KHµVXPRIVTXDUHV¶ILJXUHVFDQEHIRXQGLQWKH$129$RXWSXWWDEOHLQRXU
FXUUHQWH[DPSOHHWDVTXDUHGLV7KHXVXDOLQWHUSUHWDWLRQRIHWDVTXDUHGLVWKDW
VPDOOHIIHFW PRGHUDWHHIIHFWDQG ODUJHHIIHFWZKLFKPHDQVWKDWRXU
HIIHFWVL]HLVUDWKHUVPDOOLQGLFDWLQJWKDWHYHQWKRXJKWKHGLIIHUHQFHRIKDSSLQHVVLV
VLJQLILFDQWLWVPDJQLWXGHLVWRRVPDOOWREHUHDOO\PHDQLQJIXO
5HSRUWLQJ$129$UHVXOWV
6LPLODUWRWWHVWVWDWLVWLFV$129$UHVXOWVDUHW\SLFDOO\UHSRUWHGHPEHGGHGZLWKLQWKH
WH[WLQUHVHDUFKDUWLFOHV7KHIROORZLQJVDPSOHVKRZVWKDWZHQHHGWRSURYLGHWKHPHDQV
DQGVWDQGDUGGHYLDWLRQVWKH)YDOXHWKHGHJUHHRIIUHHGRPDQGWKHVDPSOHVL]HLQ
EUDFNHWVDVZHOODVWKHSUREDELOLW\ILJXUHDQGWKHHIIHFWVL]H,IWKHUHLVDVLJQLILFDQW
GLIIHUHQFHWKHQZHDOVRQHHGWRSUHVHQWWKHUHVXOWVRIDSRVWKRFWHVW3OHDVHQRWHWKDWLQ
WKHIROORZLQJH[DPSOHWKHGHVFULSWLYHVWDWLVWLFVDUHHPEHGGHGLQWKHWH[WEXWIRUWKH
VDNHRIFODULW\GHVFULSWLYHLQIRUPDWLRQLVRIWHQSUHVHQWHGLQDVHSDUDWHWDEOH
$RQHZD\DQDO\VLVRIYDULDQFHLQGLFDWHGWKDWWKHUHZDVDVLJQLILFDQWGLIIHUHQFH
LQKDSSLQHVVDPRQJVWZKLWHSHRSOH 0 6' EODFNSHRSOH 0

6' DQGRWKHUUDFHV 0 6' ) S
7KHHIIHFWVL]HZDVVPDOO HWDVTXDUHG 61.SRVWKRFWHVWV
VKRZHGWKDWZKLWHSHRSOHZHUHVLJQLILFDQWO\KDSSLHUWKDQPHPEHUVRIWKHQRQ
ZKLWHUDFHV EODFNDQGRWKHU SZKHUHDVWKHODWWHUWZRJURXSVGLGQRW
GLIIHUIURPHDFKRWKHUVLJQLILFDQWO\
,IZHKDYHPXOWLSOHFRPSDULVRQVZHFDQUHSRUWWKHUHVXOWVLQDWDEOH7DEOH
SUHVHQWVVDPSOH$129$VWDWLVWLFVLQFOXGLQJGHVFULSWLYHVWDWLVWLFVIRUWKHVXEVDPSOHV
FRPSDUHG
&RPSDULVRQRI$IIHFWLYH/HDUQHU9DULDEOHV$FURVV'ZHOOLQJ7\SHV
0 6'
9LOODJH 7RZQ &LW\ (IIHFW

Q Q Q ) VL]HD

$WWLWXGHVWRZDUGV
WRXULVWV
6HOIFRQILGHQFH

/DQJXDJHOHDUQLQJ
PRWLYDWLRQ
S S
D
(WDVTXDUHG
7DEOH6DPSOHWDEOHUHSRUWLQJ$129$VWDWLVWLFV
$QDO\VLVRIFRYDULDQFH $1&29$
7KHPDLQTXHVWLRQWKDWWKHDQDO\VLVRIFRYDULDQFH $1&29$ SURFHGXUHDQVZHUVLVWKH
VDPHDVZLWKWKHWWHVWDQG$129$DUHWKHPHDQGLIIHUHQFHVREVHUYHGEHWZHHQ
GLIIHUHQWJURXSVVLJQLILFDQWWKDWLVWUXHIRUWKHSRSXODWLRQ"$1&29$KRZHYHU
H[WHQGV$129$E\DGGLQJDIXUWKHUHOHPHQWEHIRUHFRPSDULQJWKHPHDQVLW
VWDWLVWLFDOO\UHPRYHVFHUWDLQREVFXULQJRUFRQIRXQGLQJHIIHFWVIRUH[DPSOHFHUWDLQ
JURXSGLIIHUHQFHVWKDWDUHUHODWHGWRWKHWDUJHW LHGHSHQGHQW YDULDEOH)RUH[DPSOHLI
ZHZDQWWRFRPSDUHWKH/FRXUVHDFKLHYHPHQWRIWZRFODVVJURXSVLWPD\SURYLGHD
IDLUHUUHVXOWLIZHUHPRYHVWDWLVWLFDOO\WKHODQJXDJHDSWLWXGHGLIIHUHQFHEHWZHHQWKH
FODVVHV7KXVZKHQZHZDQWWRFRPSDUHDYDULDEOHLQWZRRUPRUHJURXSVDQGVXVSHFW
WKDWWKHJURXSVDUHQRWVLPLODULQWHUPVRIVRPHLPSRUWDQWEDFNJURXQGYDULDEOHWKDWLV

OLNHO\WRDIIHFWWKHWDUJHWYDULDEOHZHFDQVSHFLI\WKLVEDFNJURXQGYDULDEOHDVD
µFRYDULDWH¶DQGWKHQUXQ$1&29$WRWHVWWKHJURXSGLIIHUHQFHZKLOHFRQWUROOLQJIRU
WKLVFRYDULDWH,QRWKHUZRUGVE\UHPRYLQJWKHHIIHFWVRIFHUWDLQEDFNJURXQGYDULDEOHV
$1&29$LQHIIHFWSURGXFHVDPRUHOHYHOSOD\LQJILHOGIRUWKHSXUSRVHRIFRPSDULVRQV
$VSHFLDOFDVHRIWKHXVHRI$1&29$RFFXUVLQTXDVLH[SHULPHQWDOGHVLJQV VHH
6HFWLRQ ZKHQZHFRPSDUHWKHSRVWWHVWVFRUHVRIWKHFRQWURODQGWKHWUHDWPHQW
JURXSVZKLOHFRQWUROOLQJIRUWKHSUHWHVWVFRUHVDVWKHFRYDULDWH,QWKLVFDVHLIZHILQG
DQ\VLJQLILFDQWGLIIHUHQFHEHWZHHQWKHSRVWWHVWVFRUHVWKRVHZLOOEHUHODWHGWRHYHQWV
WKDWWRRNSODFHDIWHUWKHSUHWHVWEHFDXVHWKHSUHH[LVWLQJGLIIHUHQFHVRIWKHWZRJURXSV
KDYHEHHQUHPRYHGE\FRQWUROOLQJIRUWKHSUHWHVWVFRUHV
$1&29$FDQEHIRXQGLQ6366XQGHUµ*HQHUDO/LQHDU0RGHOV¶ $1$/<6,6ĺ
*(1(5$//,1($502'(/ĺ81,9$5,$7( $OWKRXJKWKHSURFHGXUHLVQRW
SDUWLFXODUO\GLIILFXOWWRUXQLQ6366VSDFHOLPLWDWLRQVGRQRWDOORZPHWRSURYLGH
LQVWUXFWLRQVKHUH²SOHDVHUHIHUWRDQ6366JXLGHIRUWKHH[DFWGHWDLOV7KHJRRGQHZV
DERXWWKH6366RXWSXWLVWKDWQH[WWRWKHVLJQLILFDQFHYDOXHZHILQGWKHµSDUWLDOHWD
VTXDUHG¶LQGH[ZKLFKLVWKHHIIHFWVL]HVRZHGRQRWQHHGWRFDOFXODWHWKLVPDQXDOO\
&RUUHODWLRQ
7KHWZRFRUHSXUSRVHVRIVWDWLVWLFVDUHWRORRNDWWKHGLIIHUHQFHEHWZHHQYDULDEOHV²
GLVFXVVHGLQ6HFWLRQVDQG²DQGWRH[DPLQHWKHUHODWLRQVKLSEHWZHHQYDULDEOHV
7KHVWDWLVWLFDOSURFHGXUHWRDFKLHYHWKHVHFRQGSXUSRVHLVFDOOHGµFRUUHODWLRQDQDO\VLV¶
DQGLWDOORZVXVWRORRNDWWZRYDULDEOHVDQGHYDOXDWHWKHVWUHQJWKDQGGLUHFWLRQRIWKHLU
UHODWLRQVKLSRUDVVRFLDWLRQZLWKHDFKRWKHU7RGRVRZHFRPSXWHµFRUUHODWLRQ
FRHIILFLHQWV¶EHWZHHQWKHWZRYDULDEOHVZKLFKFDQUDQJHEHWZHHQWR
x $KLJKFRHIILFLHQWPHDQVDVWURQJUHODWLRQVKLS LHLIDQLQGLYLGXDOVFRUHVKLJK
RQRQHYDULDEOHKHVKHLVDOVROLNHO\WRVFRUHKLJKRQWKHRWKHU IRUH[DPSOH
VWXGHQWV¶ODQJXDJHDSWLWXGHVFRUHVDQGIRUHLJQODQJXDJHJUDGHVWHQGWRGLVSOD\D
KLJKSRVLWLYHFRUUHODWLRQ
x $FRHIILFLHQWRIVXJJHVWVQRUHODWLRQVKLSEHWZHHQWKHWZRYDULDEOHVIRU
H[DPSOHVWXGHQWV¶DWWLWXGHVWRZDUGVFKRFRODWHDQGWKHQXPEHURI
EURWKHUVVLVWHUVWKH\KDYHDUHOLNHO\WRKDYHQRVLJQLILFDQWFRUUHODWLRQV
x 1HJDWLYHFRUUHODWLRQFRHIILFLHQWVVXJJHVWLQYHUVHUHODWLRQVKLSVIRUH[DPSOHWKH
QXPEHURIGD\VVWXGHQWVPLVVVFKRROLVOLNHO\WRKDYHDQHJDWLYHFRUUHODWLRQZLWK
WKHLURYHUDOOVFKRRODFKLHYHPHQW
2IFRXUVHVLPLODUWRWWHVWDQG$129$VWDWLVWLFVWKHFRUUHODWLRQFRHIILFLHQWVQHHG
WREHVWDWLVWLFDOO\VLJQLILFDQWWRLQGLFDWHWUXHVFRUHV
6WUHQJWKRIWKHFRUUHODWLRQ
7RJLYHDQLQGLFDWLRQRIWKHVWUHQJWKRIWKHFRUUHODWLRQFRHIILFLHQWLQDSSOLHGOLQJXLVWLFV
UHVHDUFK\RXFDQILQGPHDQLQJIXOFRUUHODWLRQVRIDVORZDV± IRUH[DPSOH

EHWZHHQPRWLYDWLRQDQGDFKLHYHPHQW DQGLIWZRWHVWVFRUUHODWHZLWKHDFKRWKHULQWKH
RUGHURIZHFDQVD\WKDWWKH\PHDVXUHPRUHRUOHVVWKHVDPHWKLQJ$OWKRXJK
QHJDWLYHFRUUHODWLRQVDUHSUHFHGHGE\DQHJDWLYHVLJQWKLVRQO\LQGLFDWHVWKHLUGLUHFWLRQ
QRWWKHLUVWUHQJWK7KHFRHIILFLHQWVRIU DQGU DUHHTXDOO\VWURQJ$WWKLV
SRLQWZHPXVWQRWHWKDWFRUUHODWLRQFRHIILFLHQWVRQO\GHVFULEHµOLQHDUUHODWLRQVKLSV¶WKDW
LVDVVRFLDWLRQVEHWZHHQWZRYDULDEOHVWKDWIROORZWKHµLIZHVFRUHKLJKRQRQHZHDUH
OLNHO\WRVFRUHKLJKRQWKHRWKHU¶SULQFLSOH7KXVFRUUHODWLRQLVQRWDSSURSULDWHWRDVVHVV
µFXUYLOLQHDUUHODWLRQVKLSV¶WKDWLV8VKDSHGGLVWULEXWLRQVRIVFRUHV IRUH[DPSOHWKH
UHODWLRQVKLSEHWZHHQHDWLQJFKRFRODWHDQGHQMR\PHQWZKLFKIRUPRVWSHRSOHLQFUHDVHV
XSWRDOHYHOEXWDIWHUDZKLOHFDQWXUQLQWRVOLJKWQDXVHD
7RPDNHWKHDFWXDOVWUHQJWKRIDFRUUHODWLRQPRUHWDQJLEOHLWLVFRPPRQWRVTXDUH
WKHYDOXHRIWKHFRUUHODWLRQFRHIILFLHQWEHFDXVHWKHUHVXOWWKXVREWDLQHGUHSUHVHQWVWKH
SURSRUWLRQRIWKHYDULDQFHVKDUHGE\WKHWZRYDULDEOHV LHWKHYDULDWLRQLQWKHVFRUHRQ
RQHYDULDEOHZKLFKLVDFFRXQWHGIRUE\WKHRWKHU )RUH[DPSOHDFRUUHODWLRQRI
PHDQVWKDWSHUFHQWRIWKHYDULDQFHLQWKHVFRUHVLVH[SODLQHGE\WKHUHODWLRQVKLS
EHWZHHQWKHWZRYDULDEOHVUDWKHUWKDQE\FKDQFHRUE\VRPHRWKHUFDXVH V 7KLVILJXUH
LVVHHQDVWKHµHIIHFWVL]H¶RIFRUUHODWLRQ
0DLQW\SHVRIFRUUHODWLRQ
7KHIROORZLQJDUHWKHIRXUPDLQW\SHVRIFRUUHODWLRQXVHGLQDSSOLHGOLQJXLVWLFVUHVHDUFK
DILIWKQRQSDUDPHWULFYHUVLRQZLOOEHLQWURGXFHGLQ6HFWLRQ
x µ3HDUVRQSURGXFW±PRPHQWFRUUHODWLRQ¶LVWKHVWDQGDUGW\SHFRPSXWHGEHWZHHQWZR
FRQWLQXRXVYDULDEOHV:KHQZHWDONDERXWµFRUUHODWLRQ¶LQJHQHUDOWKLVLVZKDWZH
XVXDOO\PHDQ7KH3HDUVRQSURGXFWPRPHQWFRUUHODWLRQFRHIILFLHQWLVV\PEROL]HG
E\WKHORZHUFDVHOHWWHUµU¶ SURQRXQFHGµUKR¶
x µ3RLQWELVHULDOFRUUHODWLRQ¶DQGµSKLFRHIILFLHQWV¶&RUUHODWLRQFRHIILFLHQWVFDQEH
FDOFXODWHGZKHQRQHRUERWKYDULDEOHVDUHGLFKRWRPRXV LHKDYHRQO\WZRYDOXHV
IRUH[DPSOHµJHQGHU¶ IRUWXQDWHO\ZHGRQRWQHHGWRZRUU\DERXWWKLVEHFDXVH
6366DXWRPDWLFDOO\DGMXVWVWKHFDOFXODWLRQDFFRUGLQJO\
x µ3DUWLDOFRUUHODWLRQ¶LVDYHU\XVHIXOWHFKQLTXHDOORZLQJXVWRH[DPLQHWKH
UHODWLRQVKLSEHWZHHQWZRYDULDEOHVDIWHUUHPRYLQJWKHFRUUHODWLRQZKLFKLVGXHWR
WKHLUPXWXDODVVRFLDWLRQZLWKDWKLUGYDULDEOH IRUH[DPSOHDEDFNJURXQGYDULDEOH
VXFKDVWKHOHDUQHUV¶LQWHOOLJHQFHDSWLWXGHZKLFKFDQHDVLO\PRGLI\WKHVFRUHVZKHQ
FRPSXWLQJFRUUHODWLRQVEHWZHHQPRWLYDWLRQDQGVD\DFKLHYHPHQW 7KLVLVLQD
ZD\WKHFRUUHODWLRQDOFRXQWHUSDUWRI$1&29$
x µ0XOWLSOHFRUUHODWLRQ¶LVDWHFKQLTXHWRFRPSXWHWKHFRUUHODWLRQEHWZHHQRQH
YDULDEOHDQGDJURXSRIYDULDEOHV7KLVW\SHRIFRUUHODWLRQFDQEHXVHGIRUH[DPSOH
ZKHQZHZDQWWRVHHWKHFRPSRVLWHHIIHFWVRIDUDQJHRIOHDUQHUFKDUDFWHULVWLFV IRU
H[DPSOHDSWLWXGHPRWLYDWLRQFUHDWLYLW\DQ[LHW\HWF RQDFKLHYHPHQW,WLV
XVXDOO\V\PEROL]HGE\DQXSSHUFDVHµ5¶

7KHLVVXHRIFDXVH±HIIHFWUHODWLRQV
%HIRUHZHGLVFXVVYDULRXVWHFKQLFDODVSHFWVRIFRUUHODWLRQDQDO\VLVOHWPHKLJKOLJKWWKH
JUHDWHVWVKRUWFRPLQJRIFRUUHODWLRQDOUHVHDUFKQDPHO\WKDWLWFDQQRWLGHQWLI\FDXVHDQG
HIIHFW:KHQWZRYDULDEOHVVKRZVLJQLILFDQWSRVLWLYHFRUUHODWLRQZHFDQQRWFODLPWKDW
WKLVLVEHFDXVHRQHFDXVHVRULQIOXHQFHVWKHRWKHU$OOZHFDQVD\LVWKDWWKH\DUH
LQWHUUHODWHGVLQFHWKHKLJKHURQHYDULDEOHJHWVWKHKLJKHUWKHRWKHULVOLNHO\WREH,W
FDQIRUH[DPSOHEHWKHFDVHWKDWDQREVHUYHGVLJQLILFDQWDVVRFLDWLRQEHWZHHQWZR
YDULDEOHVLVRQO\FDXVHGE\WKHLUUHODWLRQVKLSZLWKDWKLUGYDULDEOHIRUH[DPSOH
SK\VLFDOVWUHQJWKDQGODQJXDJHOHDUQLQJPRWLYDWLRQDUHH[SHFWHGWRVKRZDVLJQLILFDQW
QHJDWLYHFRUUHODWLRQQRWEHFDXVHWKHWZRYDULDEOHVKDYHDQ\WKLQJWRGRZLWKHDFKRWKHU
EXWEHFDXVHWKH\DUHERWKUHODWHGWRWKHOHDUQHU¶VVH[ER\VWHQGWREHVWURQJHUDQGJLUOV
WHQGWREHPRUHPRWLYDWHGODQJXDJHOHDUQHUV:HPXVWWKHUHIRUHH[HUFLVHJUHDWFDXWLRQ
ZKHQUHSRUWLQJFRUUHODWLRQDOUHVXOWV2QWKHRWKHUKDQGEHFDXVHWKHUHPD\EH DQG
RIWHQLV DFDXVDOUHODWLRQVKLSEHWZHHQWZRYDULDEOHVWKDWDUHLQWHUFRUUHODWHGFRUUHODWLRQ
DQDO\VLVFDQVXJJHVWGLUHFWLRQVIRUIROORZXSH[SHULPHQWDOUHVHDUFK
5XQQLQJFRUUHODWLRQDQDO\VLVLQ6366
5XQQLQJµRUGLQDU\¶FRUUHODWLRQ LHµ3HDUVRQSURGXFWPRPHQWFRUUHODWLRQ¶ LV
VWUDLJKWIRUZDUG$1$/<=(ĺ&255(/$7(ĺ%,9$5,$7(:HQHHGQRWZRUU\
DERXWYDULDEOHVZKLFKKDYHWZRYDOXHVRQO\²6366DXWRPDWLFDOO\DGMXVWVWKH
FDOFXODWLRQVDFFRUGLQJO\%HIRUHSHUIRUPLQJDFRUUHODWLRQDQDO\VLV,KDYHIRXQGWKDWLW
LVXVHIXOWRSURGXFHDYLVXDOUHSUHVHQWDWLRQRIWKHGDWDGLVWULEXWLRQE\JHQHUDWLQJD
VFDWWHUSORW *5$3+6ĺ6&$77(5'27 )LJXUHSUHVHQWVDVDPSOHVFDWWHUSORWRI
WKHGLVWULEXWLRQRIWKHQXPEHURIZRUGVDQGWXUQVSURGXFHGE\OHDUQHUVRI(QJOLVKLQ
DQ/FRPPXQLFDWLYHWDVN$OWKRXJKWKHUHDUHVRPHREYLRXVRXWOLHUV ZKLFKZHPLJKW
ZDQWWRµGHDOZLWK¶²VHH6HFWLRQ ZHFDQGHWHFWDµFLJDUVKDSHG¶FOXVWHUZKLFK
VXJJHVWVDUHDVRQDEO\VWURQJFRUUHODWLRQ LQGHHGU ,IWKHPDLQFOXVWHURIWKH
SRLQWVIRUPVDFXUYHGOLQHWKHQXVLQJFRUUHODWLRQLVQRWDSSURSULDWHEHFDXVHFRUUHODWLRQ
DVVXPHVDOLQHDUUHODWLRQVKLS

WXUQ

ZRUG

)LJXUH6DPSOHVFDWWHUSORW
3DUWLDOFRUUHODWLRQLVVLPLODUO\VWUDLJKWIRUZDUG$1$/<=(ĺ&255(/$7(ĺ
3$57,$/$OOZHKDYHWRGRKHUHLVWRVSHFLI\DQDGGLWLRQDOYDULDEOHWKDWVKRXOGEH
FRQWUROOHGIRU
0XOWLSOHFRUUHODWLRQFDQEHFRPSXWHGXVLQJWKHµ5HJUHVVLRQ¶FRPPDQG $1$/<=(
ĺ5(*5(66,21 ZKHUHPXOWLSOHFRUUHODWLRQVDUHSURGXFHGDVSDUWRIWKHGHIDXOW
RXWSXW0XOWLSOHFRUUHODWLRQLVFDOFXODWHGEHWZHHQRQHYDULDEOHDQGDJURXSRI
YDULDEOHVLQWKH5HJUHVVLRQGLDORJXHER[VSHFLI\WKHVLQJOHYDULDEOHDVWKHµ'HSHQGHQW¶
DQGHQWHUWKHJURXSRIYDULDEOHVLQWKHµ,QGHSHQGHQW V ¶ER[7KHµPHWKRG¶VKRXOGEH
µHQWHU¶ZKLFKLVWKHGHIDXOW,QWKHRXWSXWWKHPXOWLSOHFRUUHODWLRQFRHIILFLHQWµ5¶FDQ
EHIRXQGLQWKHµ0RGHOVXPPDU\¶WDEOHDQGWKHDPRXQWRIVKDUHGYDULDQFHLVSURYLGHG
XQGHUµ56TXDUH¶LQWKHVDPHWDEOH7KHVLJQLILFDQFHRI5FDQEHIRXQGLQWKHQH[W
WDEOHODEHOOHGµ$129$¶
5HSRUWLQJFRUUHODWLRQV
,QUHVHDUFKDUWLFOHVFRUUHODWLRQVDUHW\SLFDOO\UHSRUWHGLQFRUUHODWLRQWDEOHVRUDVWKH\DUH
XVXDOO\UHIHUUHGWRµFRUUHODWLRQPDWULFHV¶7KHVHRIIHUDGLVSOD\RIKHFRUUHODWLRQ
FRHIILFLHQWVIRUDOOYDULDEOHVLQWKHVWXG\7DEOHSUHVHQWVDW\SLFDOH[DPSOH3OHDVH
QRWHWKDWDOOWKHDEEUHYLDWLRQVXVHGLQWKHWDEOHDUHH[SODLQHGLQWKHµ1RWH¶IROORZLQJWKH
WDEOH
&RUUHODWLRQV%HWZHHQ0RWLYDWLRQDO2ULHQWDWLRQVDQG$FKLHYHPHQW0HDVXUHV
0RWLYDWLRQDORULHQWDWLRQ
$FKLHYHPHQWPHDVXUH ,QWULQVLFPRWLYDWLRQ ([WULQVLFPRWLYDWLRQ

*3$²2YHUDOO
*3$²/DQJXDJHDUWV
*3$²0DWKHPDWLFV
&$76²2YHUDOO
&$76²5HDGLQJ
&$76²0DWKHPDWLFV
1RWH*3$ JUDGHSRLQWDYHUDJH&$76 &DOLIRUQLD$FKLHYHPHQW7HVW
S S S
7DEOH6DPSOHFRUUHODWLRQPDWUL[
&RUUHODWLRQFRHIILFLHQWVFDQDOVREHUHSRUWHGHPEHGGHGLQWKHWH[W+HUHLVDQ
H[DPSOHµ$VRQHZRXOGH[SHFWIURPWKHH[WHQVLYHOLWHUDWXUHGRFXPHQWLQJWKHEHQHILWV
RILQWULQVLFPRWLYDWLRQWKHUHZDVDVLJQLILFDQWSRVLWLYHFRUUHODWLRQEHWZHHQRYHUDOO
*3$>JUDGHSRLQWDYHUDJH@DQGLQWULQVLFPRWLYDWLRQ U S ¶,QVRPH
MRXUQDOVQREUDFNHWVDUHXVHGIRUSUHVHQWLQJWKHILJXUHVDVLQWKHIROORZLQJH[DPSOH
µ$QDQDO\VLVXVLQJ3HDUVRQ VFRUUHODWLRQFRHIILFLHQWVXSSRUWHGWKHK\SRWKHVL]HG
UHODWLRQVKLSEHWZHHQ*3$DQGLQWULQVLFPRWLYDWLRQU S¶
1RQSDUDPHWULFWHVWV
,Q6HFWLRQZHVDZWKDWWKHUHDUHGLIIHUHQWW\SHVRITXDQWLWDWLYHGDWDµLQWHUYDO¶
µRUGLQDO¶DQGµQRPLQDO¶,QWHUYDOGDWDSURYLGHVWKHPRVWµSUHFLVH¶LQIRUPDWLRQDQGVXFK
GDWD²SURYLGHGLWLVUHDVRQDEO\QRUPDOO\GLVWULEXWHG VHH6HFWLRQ ²FDQEH
SURFHVVHGE\SDUDPHWULFWHVWVVXFKDV$129$RU3HDUVRQFRUUHODWLRQ,IZHKDYHOHVV
SUHFLVHRUGLQDOGDWDRUFDWHJRULFDO LHQRPLQDO GDWDRULIWKHGDWDLVQRWQRUPDOO\
GLVWULEXWHGSDUDPHWULFWHVWVDUHQRWDSSURSULDWHDQGZHQHHGWRXVHQRQSDUDPHWULF
SURFHGXUHV6366RIIHUVDZLGHUDQJHRIVXFKWHVWVZLWKHYHU\EDVLFSDUDPHWULF
SURFHGXUHKDYLQJDQRQSDUDPHWULFDOWHUQDWLYH
%HIRUH,GHVFULEHVRPHFRQFUHWHSURFHGXUHVLWLVLPSRUWDQWWRUHDOL]HWKDWQRQ
SDUDPHWULFWHVWVFDQDOVREHXVHGZLWKLQWHUYDOGDWDZKHQZHGRQRWIHHOFRQILGHQWDERXW
WKHµHTXDOLQWHUYDOQHVV¶RIWKHPHDVXUHPHQWRUWKHQRUPDOGLVWULEXWLRQRIWKHGDWD7KLV
KRZHYHUUDLVHVDTXHVWLRQLIQRQSDUDPHWULFWHVWVFDQEHVHOHFWHGIRUQRPLQDORUGLQDO
DQGHYHQLQWHUYDOGDWDZLWKRXWDQ\ZRUULHVDERXWWKHQRUPDOLW\RIWKHGLVWULEXWLRQZK\
GRQ¶WZHVLPSO\RSWIRUWKHPDOOWKHWLPH"7KHUHDVRQLVWKDWZHZDQWWRXVHWKHPRVW

SRZHUIXOSURFHGXUHWKDWZHFDQILQGWRWHVWRXUK\SRWKHVHV µ3RZHU¶LVDWHFKQLFDOWHUP
LQVWDWLVWLFVFRQFHUQLQJWKHSUREDELOLW\RIREWDLQLQJVWDWLVWLFDOO\VLJQLILFDQWUHVXOWV
3DUDPHWULFWHVWVXWLOL]HWKHPRVWLQIRUPDWLRQVRWKH\DUHPRUHSRZHUIXOWKDQWKHLUQRQ
SDUDPHWULFFRXQWHUSDUWV,QHIIHFWWKLVPHDQVWKDWZKHQZHFRPSDUHIRUH[DPSOHWZR
JURXSPHDQVZLWKDSDUDPHWULFWHVWVXFKDVWKHWWHVWZHDUHPRUHOLNHO\WRJHW
VLJQLILFDQWUHVXOWVWKDQLIZHFRPSDUHWKHVDPHPHDQVZLWKDQRQSDUDPHWULFWHVW
7KHUHIRUHSDUDPHWULFWHVWVVKRXOGEHRXUILUVWFKRLFHLIZHIHHOFRQILGHQWDERXWWKH
PHDVXUHPHQWRIWKHYDULDEOHVDQGWKHGLVWULEXWLRQRIWKHVFRUHV3OHDVHQRWHKRZHYHU
WKDWQRQSDUDPHWULFWHVWVDUHQRWQHFHVVDULO\VHFRQGEHVWLQWKLVUHVSHFW,QDUHFHQW
VWXG\IRUH[DPSOHZHKDGGRXEWVDERXWWKHLQWHUYDOQHVVRIWKHVFDOHVDQGWKHUHIRUH
UHSODFHGWKHSDUDPHWULF 3HDUVRQ FRUUHODWLRQVZLWKQRQSDUDPHWULFRQHV 6SHDUPDQ
FRUUHODWLRQV²VHHEHORZ :HZHUHVXUSULVHGWRVHHWKDWLQVRPHFDVHVWKHQHZ
FRHIILFLHQWVZHUHDFWXDOO\KLJKHU
7KHPRVWFRPPRQO\XVHGQRQSDUDPHWULFWHVWLQDSSOLHGOLQJXLVWLFVLVWKHµ&KL
VTXDUHWHVW¶$IWHUGHVFULELQJLW,ZLOOJLYHDEULHIRYHUYLHZRIVRPHRWKHUXVHIXOQRQ
SDUDPHWULFRSWLRQV
&KLVTXDUHWHVW
7KH&KLVTXDUHSURFHGXUHLVYDOXDEOHEHFDXVHLWLVRQHRIWKHIHZSURFHGXUHVWKDWFDQ
GHDOZLWKQRPLQDOGDWD5HFDOOWKDWQRPLQDOGDWDFRQFHUQIDFWVWKDWFDQEHVRUWHGLQWR
YDULRXVFDWHJRULHV 7\SLFDOH[DPSOHVRIQRPLQDOGDWDDUHµ/EDFNJURXQG¶VRUWHGLQWR
0DQGDULQ+XQJDULDQ.RUHDQHWFRUµVH[¶VRUWHGDVPDOHDQGIHPDOH /HWXVWDNHD
OHDUQHUJURXSRIVWXGHQWV JLUOVDQGER\V ZKRKDYHSDUWLFLSDWHGLQDEDOORWE\
YRWLQJµ\HV¶RUµQR¶7KHIUHTXHQF\RIWKHYRWHVEURNHQGRZQE\VH[FDQEHSXWLQWRD
[JULG

%R\V *LUOV
<HV
1R

/RRNLQJDWWKHILJXUHVZHFDQQRWKHOSZRQGHULQJZKHWKHUWKHVWXGHQWV¶JHQGHU
LQIOXHQFHGWKHLUFKRLFHEHFDXVHWKHUHVHHPWREHYHU\IHZµ\HV¶YRWHVJLYHQE\ER\V
7KH&KLVTXDUHSURFHGXUHFDQWHOOXVZKHWKHUWKHUHZDVDVLJQLILFDQWUHODWLRQVKLS
EHWZHHQYRWLQJRULHQWDWLRQDQGJHQGHUDQGLIWKHUHZDVDQ\ZHFDQILQGRXWKRZWKH
ILJXUHVLQHDFKFHOODUHGLIIHUHQWIURPWKHH[SHFWHGILJXUH7KH&KLVTXDUHWHVWLV
RIIHUHGDVDµ6WDWLVWLFV¶RSWLRQRI&52667$%6 $1$/<=(ĺ'(6&5,37,9(
67$7,67,&6ĺ&52667$%6ĺ67$7,67,&6 LQDGGLWLRQWRVHOHFWLQJ&KLVTXDUH

DOVRVHOHFWERWKWKHµH[SHFWHG¶DQGµREVHUYHGFRXQWV¶DQGWKHµXQVWDQGDUGL]HGUHVLGXDO¶
LQWKHµ&HOOV¶RSWLRQ
7KHUHLVDOVRDVLPSOLILHGIRUPRIWKH&KLVTXDUHSURFHGXUHZKLFKLVXVHIXOZKHQZH
ZDQWWRVHHZKHWKHUWKHYDOXHVIURPDYDULDEOHDUHVLJQLILFDQWO\GLIIHUHQWIURPH[SHFWHG
YDOXHVLIZHNQRZWKHVHYDOXHV7KHPRVWFRPPRQH[DPSOHLVWRWHVWZKHWKHUWKH
JHQGHUGLVWULEXWLRQRIRXUVDPSOHLVVLJQLILFDQWO\GLIIHUHQWIURPWKHSRSXODWLRQEHFDXVH
ZHNQRZWKDWLQWKHSRSXODWLRQWKHQXPEHURIPDOHVDQGIHPDOHVLVURXJKO\HTXDO7KLV
W\SHRI&KLVTXDUHSURFHGXUHLVUXQE\VHOHFWLQJ$1$/<=(ĺ1213$5$0(75,&
7(676ĺ&+,648$5(
,OOXVWUDWLRQDQGUHSRUWLQJWKHUHVXOWV
7RLOOXVWUDWHWKHILUVW DQGPRUHFRPPRQ W\SHRI&KLVTXDUHWHVWOHWXVXVHDJDLQWKH
*HQHUDO866RFLDO6XUYH\GDWDVHW WKDWZHKDYHDOUHDG\XVHGLQ6HFWLRQV
DQG DQGH[DPLQHZKHWKHUWKHVDPSOLQJRIVH[DQGUDFHZDVLQGHSHQGHQWRIHDFK
RWKHURUZKHWKHUWKHUHZDVDV\VWHPDWLFUHODWLRQVKLSEHWZHHQWKHWZRFDWHJRULHV LH
PRUHPDOHVRUIHPDOHVZHUHVHOHFWHGIURPDSDUWLFXODUUDFH ²REYLRXVO\WKLVODWWHUFDVH
ZRXOGQRWEHGHVLUDEOHEHFDXVHLWZRXOGDPRXQWWRZKDWLVDJHQGHUELDV7DEOH
SUHVHQWVWKH6366RXWSXW

VH[ UDFH&URVVWDEXODWLRQ
UDFH
7RWDO
VH[ &RXQW
([SHFWHG&RXQW
5HVLGXDO
&RXQW
([SHFWHG&RXQW
5HVLGXDO
7RWDO &RXQW
([SHFWHG&RXQW

&KL6TXDUH7HVWV
$V\PS6LJ
9DOXH GI VLGHG
3HDUVRQ&KL6TXDUH D
/LNHOLKRRG5DWLR
/LQHDUE\/LQHDU

$VVRFLDWLRQ
1RI9DOLG&DVHV
D FHOOV KDYHH[SHFWHGFRXQWOHVVWKDQ7KH
PLQLPXPH[SHFWHGFRXQWLV

7DEOH6DPSOH6366RXWSXWUHSRUWLQJ&KLVTXDUHVWDWLVWLFV

,QWKHRXWSXWILUVWZHQHHGWRORRNDWWKHVLJQLILFDQFHRIWKH3HDUVRQ&KLVTXDUH
YDOXH,ILWLVVLJQLILFDQWDWWKHSOHYHOZHFDQFODLPWKDWWKHWZRYDULDEOHVDUHQRW
LQGHSHQGHQWEXWDUHUHODWHG,IVRWKHµ5HVLGXDO¶GLIIHUHQFHEHWZHHQWKHREVHUYHGDQG
WKHH[SHFWHGFRXQWVZLOOWHOOXVZKHUHWKHJUHDWHVWLPSDFWRFFXUUHG,QRXUSDUWLFXODU
H[DPSOHWKH3HDUVRQ&KLVTXDUHYDOXHLVQRWVLJQLILFDQWZKLFKPHDQVWKDWWKH
SURSRUWLRQRIPHQDQGZRPHQLQWKHHWKQLFJURXSVDPSOHVLVQRWVLJQLILFDQWO\GLIIHUHQW
IURPHDFKRWKHU ZKLFKLVJRRGQHZVIURPWKHVXUYH\¶VSRLQWRIYLHZ ,QGHHGWKH
UHVLGXDOVLQWKHµVH[ UDFHFURVVWDEXODWLRQ¶WDEOHDUHUHODWLYHO\ORZFRPSDUHGWRWKH
VDPSOHVL]H
:HFRXOGUHSRUWWKHVHUHVXOWVDVIROORZVµ$[&KLVTXDUHDQDO\VLVUHYHDOHGWKDW
WKHUHZDVQRVLJQLILFDQWUHODWLRQVKLSEHWZHHQWKHJHQGHUDQGWKHHWKQLFEDFNJURXQGRI
WKHVDPSOHGSDUWLFLSDQWV[ S ¶
1RQSDUDPHWULFHTXLYDOHQWVRINH\SDUDPHWULFWHVWV
:LWKRXWJRLQJLQWRGHWDLOOHWPHOLVWVRPHRWKHUXVHIXOQRQSDUDPHWULFWHVWV7KH\DUH
VWUDLJKWIRUZDUGWRUXQDQGWKHLUUHVXOWVDUHWREHLQWHUSUHWHGLQWKHVDPHZD\DVWKRVHRI
WKHFRUUHVSRQGLQJSDUDPHWULFWHVWV
x 6SHDUPDQ¶VUDQNRUGHUFRUUHODWLRQ7KLVFRUUHODWLRQFRHIILFLHQWLVREWDLQHGE\
VLPSO\WLFNLQJWKHµ6SHDUPDQ¶ER[LQWKH3HDUVRQFRUUHODWLRQGLDORJXHER[ 6HH
6HFWLRQ $VWKHQDPHVXJJHVWVWKHFRUUHODWLRQLVEDVHGRQWKHUDQNVRIWKH
GDWD LQDUDQNRUGHU UDWKHUWKDQWKHDFWXDOYDOXHV,WLVDSSURSULDWHIRURUGLQDOGDWD
RUIRULQWHUYDOGDWDWKDWGRQRWVDWLVI\WKHGLVWULEXWLRQQRUPDOLW\RUWKHHTXDO
LQWHUYDOQHVVDVVXPSWLRQV,IZHFRPSDUHWKHWZRW\SHVRIFRUUHODWLRQVFRPSXWHGIRU
WKHVDPHYDULDEOHVZHFDQVHHWKDWWKH\DUHYHU\VLPLODUEXWWKH6SHDUPDQ
FRHIILFLHQW²DVH[SHFWHG²LVXVXDOO\OHVVµSRZHUIXO¶WKDQWKH3HDUVRQRQHWKDWLVLW
LVORZHUDQGWKHDFFRPSDQ\LQJVLJQLILFDQFHILJXUHLVKLJKHU
x 0DQQ:KLWQH\8WHVW7KLVLVWKHQRQSDUDPHWULFDOWHUQDWLYHWRWKHLQGHSHQGHQW
VDPSOHVWWHVW $1$/<=(ĺ1213$5$0(75,&7(676ĺ,1'(3(1'(17
6$03/(6ĺ0$11:+,71(<8
x :LOFR[RQVLJQHGUDQNWHVW7KLVLVWKHQRQSDUDPHWULFDOWHUQDWLYHWRWKHSDLUHG
VDPSOHVWWHVW $1$/<=(ĺ1213$5$0(75,&7(676ĺ5(/$7('
6$03/(6ĺ:,/&2;21
x .UXVNDO:DOOLVWHVW7KLVLVWKHQRQSDUDPHWULFDOWHUQDWLYHWRRQHZD\$129$
$1$/<=(ĺ1213$5$0(75,&7(676ĺ.,1'(3(1'(176$03/(6
ĺ.586.$/:$//,6+
$GYDQFHGVWDWLVWLFDOSURFHGXUHV
7KHSUHYLRXVVHFWLRQVKDYHFRYHUHGWKHEDVLFDQGPRVWRIWHQXVHGVWDWLVWLFDO
SURFHGXUHV,QWKLVILQDOVHFWLRQ,ZLOOLQWURGXFHPRUHDGYDQFHGWHFKQLTXHVµ$GYDQFHG¶
LQPRVWFDVHVGRHVQRWPHDQWKDWRQHQHFHVVDULO\QHHGVPRUHPDWKHPDWLFDONQRZOHGJH

WRUXQWKHVHWHVWV,QIDFWWKHILUVWSURFHGXUHWREHLQWURGXFHG²µWZRZD\$129$¶²
RQO\GLIIHUVIURPWKHRSHUDWLRQVGLVFXVVHGEHIRUHLQWKDWWKHUHDUHVOLJKWO\PRUHRSWLRQV
WRVHOHFWIURPLQWKHIDPLOLDULQWHUDFWLYH6366GLDORJXHER[
7KHQH[WWZRSURFHGXUHVWREHSUHVHQWHGµIDFWRUDQDO\VLV¶DQGµFOXVWHUDQDO\VLV¶DUH
H[SORUDWRU\WHFKQLTXHVDQGWKHUHIRUHWDNLQJGLIIHUHQWRSWLRQVPLJKWOHDGWRVRPHZKDW
GLIIHUHQWRXWFRPHV7KXVWKHVHWHFKQLTXHVUHTXLUHDWKRURXJKXQGHUVWDQGLQJRIWKH
JXLGLQJSULQFLSOHV EXWQRWWKHPDWKHPDWLFV )LQDOO\,EULHIO\LQWURGXFHWZR
SURFHGXUHV²µVWUXFWXUDOHTXDWLRQPRGHOOLQJ¶DQGµPHWDDQDO\VLV¶²WKDWKDYHEHFRPH
YHU\SRSXODURYHUWKHSDVWGHFDGH7KH\DUHFRPSOH[LQVHYHUDOZD\VDQGWRXVHWKHP
RQHQHHGVVSHFLDOVRIWZDUH7KHLUGHVFULSWLRQKHUHZLOORQO\RIIHUDVXPPDU\WRKHOSWR
XQGHUVWDQGDUWLFOHVXVLQJWKHSURFHGXUHV
$OWKRXJKDGYDQFHGVWDWLVWLFDOSURFHGXUHVDSSHDUWRRIIHUH[WHQGHGDQDO\WLFDO
RSSRUWXQLWLHVLWPD\EHZRUWKFLWLQJKHUHWKHDGYLFHRIWKHµ7DVN)RUFHRQ6WDWLVWLFDO
,QIHUHQFH¶RIWKH$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQWRH[HUFLVHFDXWLRQQRWWR
RYHUFRPSOLFDWHWKHDQDO\VLV :LONLQVRQDQG7)6,
7KHHQRUPRXVYDULHW\RIPRGHUQTXDQWLWDWLYHPHWKRGVOHDYHVUHVHDUFKHUVZLWK
WKHQRQWULYLDOWDVNRIPDWFKLQJDQDO\VLVDQGGHVLJQWRWKHUHVHDUFKTXHVWLRQ
$OWKRXJKFRPSOH[GHVLJQVDQGVWDWHRIWKHDUWPHWKRGVDUHVRPHWLPHVQHFHVVDU\
WRDGGUHVVUHVHDUFKTXHVWLRQVHIIHFWLYHO\VLPSOHUFODVVLFDODSSURDFKHVRIWHQFDQ
SURYLGHHOHJDQWDQGVXIILFLHQWDQVZHUVWRLPSRUWDQWTXHVWLRQV'RQRWFKRRVHDQ
DQDO\WLFPHWKRGWRLPSUHVV\RXUUHDGHUVRUWRGHIOHFWFULWLFLVP,IWKH
DVVXPSWLRQVDQGVWUHQJWKRIDVLPSOHUPHWKRGDUHUHDVRQDEOHIRU\RXUGDWDDQG
UHVHDUFKSUREOHPXVHLW
7ZRZD\$129$
µ7ZRZD\$129$¶LVVLPLODUWRRQHZD\$129$ZKLFKZHKDYHGLVFXVVHGLQ
6HFWLRQH[FHSWIRURQHGLIIHUHQFHLQVWHDGRIRQHLQGHSHQGHQWRUJURXSLQJ
YDULDEOHZHLQFOXGHWZRVXFKIDFWRUV)RUH[DPSOHLIZHKDYHDGDWDVHWRIVFKRRO
FKLOGUHQFRPLQJIURPWKUHHGLIIHUHQWFODVVHVZHZRXOGXVHDRQHZD\$129$WR
FRPSDUHWKHLUUHVXOWV%XWLIZHNQRZWKDWWKHUHLVDQRWKHUUHOHYDQWIDFWRUWKDWDOVR
GLYLGHVWKHVHVWXGHQWVLQWRRWKHUJURXSV IRUH[DPSOHWKHLUVH[RUWKHLUOHYHORI/
SURILFLHQF\ DQGZHZDQWWRLQFOXGHWKLVIDFWRULQWKHSDUDGLJPDVZHOOWKHQZHQHHGWR
UXQDWZRZD\$129$
0DWKHPDWLFDOO\WKHWZRZD\$129$LVPRUHFRPSOH[WKDQLWVRQHZD\
FRXQWHUSDUWEHFDXVHKHUHLQVWHDGRIRQH)VFRUHZHQHHGWKUHHRQHIRUHDFKRIWKH
LQGHSHQGHQWYDULDEOHVDQGRQHIRUWKHLULQWHUDFWLRQ7KHEHVWZD\WRXQGHUVWDQGWKLVLV
E\VKRZLQJDQH[DPSOH
,QDODUJHVFDOHORQJLWXGLQDOVXUYH\RI+XQJDULDQVFKRROFKLOGUHQ¶VDWWLWXGHV
WRZDUGVYDULRXVIRUHLJQODQJXDJHVZHKDYHIRXQG '|UQ\HL&VL]pUDQG1pPHWK
WKDWWKHFKLOGUHQ¶VDWWLWXGHVWRZDUGV*HUPDQGURSSHGVLJQLILFDQWO\EHWZHHQ WKH

ILUVWZDYHRIGDWDFROOHFWLRQ DQG WKHODVWZDYHRIWKHGDWDFROOHFWLRQ :HDOVR
QRWLFHGWKDWVWXGHQWVZKRZHUHVWXG\LQJ*HUPDQDWVFKRROWHQGHGWRKDYHPRUHSRVLWLYH
DWWLWXGHVWRZDUGVWKHODQJXDJHWKDQWKRVHWKDWZHUHVWXG\LQJRWKHUODQJXDJHVDWVFKRRO
UHJDUGOHVVRIZKHWKHU*HUPDQZDVWKHLURZQFKRLFHRUZKHWKHUWKH\RQO\WRRNLW
EHFDXVHWKLVZDVWKHRQO\/RIIHUHGE\WKHVFKRRO:HZDQWHGWRFRPELQHWKHVHWZR
REVHUYDWLRQVDQGZDQWHGWRWHVWZKHWKHUWKHDWWLWXGHVRIDFWLYHOHDUQHUVRI*HUPDQ
GURSSHGOHVVRYHUWKH\HDUSHULRGWKDQWKHDWWLWXGHVRIQRQOHDUQHUVRI*HUPDQ7KLV
ZDVDW\SLFDOFDVHIRUDWZRZD\$129$ZLWKµ$WWLWXGHVWRZDUGV*HUPDQ¶EHLQJWKH
GHSHQGHQWDQGµ7LPH¶DQGµ6FKRRO¶EHLQJWKHLQGHSHQGHQWYDULDEOHV
7ZRZD\$129$FDQEHUXQLQ6366E\VHOHFWLQJ*(1(5$//,1($502'(/6
DQGWKHQ81,9$5,$7(7KHLQGHSHQGHQWJURXSLQJYDULDEOHVDUHFDOOHGµ)L[HG
IDFWRUV¶7DEOHSUHVHQWVWKH6366RXWSXWZHUHFHLYHG:HDUHLQWHUHVWHGLQOLQHV±
LQWKHWDEOH/LQHWHOOVXVWKDWWKHUHLVLQGHHGDKLJKO\VLJQLILFDQWVFKRROHIIHFW ZKLFK
ZHNQHZ /LQHWHOOVXVWKDWWKHUHLVLQGHHGDKLJKO\VLJQLILFDQWWLPHHIIHFW ZKLFKZH
DOVRNQHZ /LQHUHSRUWVRQWKHLQWHUDFWLRQRIWKHWZRYDULDEOHVDVWKH\LPSDFWHGWKH
*HUPDQUHODWHGDWWLWXGHV7KHKLJKO\VLJQLILFDQW)YDOXHLQGLFDWHVWKDWWKHUHLVDµWUXH¶
FRPELQHGHIIHFW

7HVWVRI%HWZHHQ6XEMHFWV(IIHFWV
'HSHQGHQW9DULDEOHDWWJHU
7\SH,,,6XP
6RXUFH RI6TXDUHV GI 0HDQ6TXDUH ) 6LJ
&RUUHFWHG0RGHO D
,QWHUFHSW
VFKRRO
WLPH
VFKRRO WLPH
(UURU
7RWDO
&RUUHFWHG7RWDO
D 56TXDUHG $GMXVWHG56TXDUHG

7DEOH6DPSOH6366RXWSXWUHSRUWLQJWZRZD\$129$VWDWLVWLFV

2IWHQWKHEHVWZD\RILQWHUSUHWLQJVXFKLQWHUDFWLRQVLVE\DJUDSK$QRSWLRQRIWKH
8QLYDULDWHSURFHGXUHRI6366LVWRSURYLGHDµSURILOHSORW¶ XQGHUWKHµ3ORW¶RSWLRQ
7KHOLQHGLDJUDPLQ)LJXUHZDVSUHSDUHGEDVHGRQWKLVRXWSXW,IWKHUHKDGEHHQQR
VLJQLILFDQWLQWHUDFWLRQEHWZHHQWKHWZRLQGHSHQGHQWYDULDEOHVWKHQZHZRXOGKDYHWZR
URXJKO\SDUDOOHOOLQHVLQ)LJXUH,QWKLVSDUWLFXODUFDVHWKHLQWHUDFWLRQLVVLJQLILFDQW
DQGZHFDQHDVLO\VHHWKDWWKLVVLJQLILFDQFHLVFDXVHGE\WKHIDFWWKDWWKHDWWLWXGHVRI
DFWLYH*HUPDQOHDUQHUVGLGQRWGHFUHDVHDVVKDUSO\DVWKDWRIWKHQRQOHDUQHUV

)LJXUH6DPSOHGLDJUDPGHVFULELQJWKHFKDQJHLQ/OHDUQHUV¶GLVSRVLWLRQWRZDUGVWKH/
RYHUWLPH
5HVHDUFKHUVZKRDUHQRWIDPLOLDUZLWKWKHSURFHGXUHPD\ZDQWWRVWDUWZLWK
LQGHSHQGHQWYDULDEOHVWKDWKDYHRQO\WZRYDOXHVHDFK DVLQ)LJXUH EHFDXVHWKH
LQWHUSUHWDWLRQRIWKHUHVXOWVLVHDVLHUWKLVZD\:LWKPRUHWKDQWZRYDOXHVZHQHHGWR
UXQSRVWKRFWHVWV VLPLODUO\WR$129$ ²WKHVHDUHUHDGLO\DYDLODEOHLQ6366:HFDQ
JHWWKHHIIHFWVL]HE\FOLFNLQJRQµ(VWLPDWHVRIHIIHFWVL]H¶ZKLFKLVDQRSWLRQWKH
HIIHFWVL]HLQGH[µSDUWLDOHWDVTXDUHG¶FDQEHLQWHUSUHWHGLQWKHVDPHZD\DVLQDRQH
ZD\$129$
)DFWRUDQDO\VLV
µ)DFWRUDQDO\VLV¶LVUDWKHUFRPSOH[PDWKHPDWLFDOO\EXWIDLUO\VWUDLJKWIRUZDUG
FRQFHSWXDOO\,QRUGHUWRXQFRYHUWKHODWHQWVWUXFWXUHWKDWXQGHUOLHVODUJHGDWDVHWVLW
UHGXFHVWKHQXPEHURIYDULDEOHVVXEPLWWHGWRWKHDQDO\VLVWRDIHZYDOXHVWKDWZLOOVWLOO
FRQWDLQPRVWRIWKHLQIRUPDWLRQIRXQGLQWKHRULJLQDOYDULDEOHV7KHRXWFRPHRIWKH
SURFHGXUHLVDVPDOOVHWRIXQGHUO\LQJGLPHQVLRQVUHIHUUHGWRDVµIDFWRUV¶RU
µFRPSRQHQWV¶)RUH[DPSOHLIZHKDYHFROOHFWHGYDULRXVSLHFHVRILQIRUPDWLRQDERXWD
JURXSRIFKLOGUHQFRQFHUQLQJWKHLU/SURILFLHQF\ IRUH[DPSOHWHVWVFRUHVFRXUVH
DFKLHYHPHQWSHUFHLYHGFRPSHWHQFH DQGWKHLUSK\VLFDOVNLOOV IRUH[DPSOHUXQQLQJ
VSHHG3K\VLFDO(GXFDWLRQJUDGHVRPHVWUHQJWKLQGH[ IDFWRUDQDO\VLVLVOLNHO\WR
SURGXFHWZRIDFWRUVFRUUHVSRQGLQJWRWKHWZRDUHDV
$VSDUWRIWKHIDFWRUDQDO\VLVRXWSXW6366SURGXFHVDWDEOH²WKHµIDFWRUPDWUL[¶²
ZKLFKFRQWDLQVWKHFRUUHODWLRQVEHWZHHQWKHREWDLQHGIDFWRUVDQGWKHRULJLQDOYDULDEOHV
WKDWWKH\KDYHEHHQH[WUDFWHGIURP7KHVHFRUUHODWLRQVDUHWKHµIDFWRUORDGLQJV¶DQGWKH\
VKRZWKHH[WHQWWRZKLFKHDFKRIWKHRULJLQDOYDULDEOHVKDVFRQWULEXWHGWRWKHUHVXOWDQW

IDFWRUV7KXVIDFWRUDQDO\WLFDOVWXGLHVH[SORLWWKHµSDWWHUQILQGLQJ¶FDSDFLW\RIWKH
SURFHGXUHE\VDPSOLQJDZLGHUDQJHRILWHPVDQGWKHQH[DPLQLQJWKHLUUHODWLRQVKLSV
ZLWKFRPPRQXQGHUO\LQJWKHPHV%HFDXVHIDFWRUDQDO\VLVLVXVHIXOLQPDNLQJODUJH
GDWDVHWVPRUHPDQDJHDEOHWKHSURFHGXUHLVRIWHQXVHGDVDSUHSDUDWRU\VWHSLQGDWD
SURFHVVLQJEHIRUHFRQGXFWLQJIXUWKHUDQDO\VHV
)DFWRUDQDO\VLVLVHDV\WRUXQRQ6366 $1$/<=(ĺ'$7$5('8&7,21ĺ
)$&725 7KHUHDUHKRZHYHUIRXULVVXHVWRFRQVLGHU
,WHPVVXEPLWWHGWRWKHDQDO\VLV$FRPPRQPLVWDNHPDGHZKHQXVLQJIDFWRU
DQDO\VLVLVWREHOLHYHWKDWLIZHVXEPLWDZLGHHQRXJKUDQJHRILWHPVUHODWHGWRD
EURDGWDUJHWGRPDLQWRIDFWRUDQDO\VLVWKHILQDOIDFWRUVWUXFWXUHZLOOSURYLGHDIDLU
UHSUHVHQWDWLRQRIWKHPDLQGLPHQVLRQVXQGHUO\LQJWKHGRPDLQ7KLVLVLQFRUUHFW7KH
ILQDOIDFWRUVWUXFWXUHZLOORQO\UHSUHVHQWWKHLWHPVVXEPLWWHGWRWKHDQDO\VLV,IZH
RULJLQDOO\GLGQRWLQFOXGHLQRXUTXHVWLRQQDLUHLWHPVFRQFHUQLQJDNH\IHDWXUHRIWKH
GRPDLQWKDWSDUWLFXODUGRPDLQFRPSRQHQWKDVQRFKDQFHWRHPHUJHDVDVWDWLVWLFDO
IDFWRU7KDWLVRXUVHOHFWLRQRILWHPVIXQGDPHQWDOO\GHWHUPLQHVWKHILQDOIDFWRU
VWUXFWXUH
6HOHFWLQJWKHW\SHRIIDFWRUDQDO\VLV H[WUDFWLRQDQGURWDWLRQ )DFWRUDQDO\VLV
HPSOR\VDVWHSZLVHSURFHGXUH7KHILUVWVWHSLQYROYHVµIDFWRUH[WUDFWLRQ¶WKDWLV
FRQGHQVLQJWKHYDULDEOHVLQWRDUHODWLYHO\VPDOOQXPEHURIIDFWRUV7KHVHFRQGVWHS
LVµIDFWRUURWDWLRQ¶WKDWLVPDNLQJWKHIDFWRUVHDVLHUWRLQWHUSUHWE\HPSOR\LQJ
PDWKHPDWLFDOWHFKQLTXHV7KHUHDUHVHYHUDODOWHUQDWLYHPHWKRGVIRUERWKSURFHGXUHV
DQGWKHUHIRUHZHPXVWPDNHDGHFLVLRQDVWRZKLFKPHWKRGWRHPSOR\,QSUDFWLFH
WKHQHFHVVDU\GHFLVLRQVFDQEHUHVWULFWHGWRFKRRVLQJEHWZHHQWZRH[WUDFWLRQ
PHWKRGVµSULQFLSDOFRPSRQHQWDQDO\VLV¶DQGµPD[LPXPOLNHOLKRRGDQDO\VLV¶DQG
WZRURWDWLRQPHWKRGVµRUWKRJRQDO YDULPD[ ¶RUµREOLTXH REOLPLQ URWDWLRQ¶7KHVH
FDQEHFRPELQHGLQIRXUGLIIHUHQWZD\VDQGLQWKHHGXFDWLRQDOSV\FKRORJLFDODQG
DSSOLHGOLQJXLVWLFOLWHUDWXUHZHFDQILQGH[DPSOHVRIDOOIRXUFRPELQDWLRQV0\
SHUVRQDOSUHIHUHQFHLVIRUDPD[LPXPOLNHOLKRRGDQDO\VLVZLWKREOLTXHURWDWLRQEXW
LIWKHUHLVDVWURQJPXOWLGLPHQVLRQDOVWUXFWXUHXQGHUO\LQJWKHGDWDDQ\PHWKRGZLOO
GHWHFWLWDQGWKHUHZLOOEHRQO\PLQRUYDULDWLRQLQWKHLQGLYLGXDOORDGLQJV7KLV
PHDQVWKDWLIWKHYDULRXVPHWKRGVSURGXFHHVVHQWLDOO\WKHVDPHUHVXOWVZHFDQEH
DVVXUHGWKDWWKHIDFWRUVWUXFWXUHLVYDOLG+RZHYHULIWKHGDWDLVVXFKWKDWGLIIHUHQW
DSSURDFKHVOHDGWRYHU\GLIIHUHQWVROXWLRQVWKLVVKRXOGEHDVRXUFHRIFRQFHUQ
'HWHUPLQLQJWKHQXPEHURIIDFWRUVWREHH[WUDFWHG7KHPRVWGLIILFXOWLVVXHLQIDFWRU
DQDO\VLVLVWRGHFLGHRQWKHQXPEHURIIDFWRUVWREHH[WUDFWHG6366ZLOOSURYLGHD
UHDG\PDGHDQVZHUEXWLWLVQRWDQREMHFWLYHVROXWLRQEXWMXVWDµGHIDXOW¶WKDWKDV
EHHQDUWLILFLDOO\VHW7KHW\SLFDOGHIDXOWSULQFLSOHLVWKDWWKHH[WUDFWLRQSURFHVV
VKRXOGVWRSDWIDFWRUVZKLFKZRXOGH[SODLQOHVVYDULDQFHWKDQHDFKLQGLYLGXDOLWHP
VXEPLWWHGWRWKHDQDO\VLV LHIDFWRUVZKRVHµHLJHQYDOXH¶LVVPDOOHUWKDQ EHFDXVH
WKHQLQHIIHFWWKHIDFWRULVQRWEHWWHUWKDQDVLQJOHLWHP7KLVLVKRZHYHUQRW
QHFHVVDULO\WKHEHVWVROXWLRQVRPHWLPHVIHZHUVRPHWLPHVPRUHIDFWRUVPD\OHDGWR

DEHWWHUUHVXOW7KHUHLVQRSHUIHFWZD\RIVHWWLQJWKHRSWLPDOIDFWRUQXPEHUEXW
WKHUHDUHVHYHUDOXVHIXOLQGLFDWRUVIRUUHVHDUFKHUVWRWDNHLQWRDFFRXQW
x &DWWHOO¶V VFUHHWHVW%\SORWWLQJWKHYDULDQFHDVVRFLDWHGZLWKHDFKIDFWRU

ZKLFKWKHFRPSXWHUGRHVIRUXV ZHREWDLQDYLVXDOUHSUHVHQWDWLRQRIWKHVWHHS
GHFUHDVH²WKHµVORSH¶²DQGWKHJUDGXDOWUDLOLQJRII²WKHµVFUHH¶²RIWKHIDFWRUV
$FFRUGLQJWRWKHVFUHHWHVWRQO\WKHIDFWRUVRQWKHµVORSH¶VKRXOGEHFRQVLGHUHG
)LJXUHLOOXVWUDWHVWKHVFUHHSORWRIWKHHLJHQYDOXHV ZKLFKDUHLQGLFHVRIWKH
DPRXQWRIYDULDQFHHDFKIDFWRUH[SODLQV REWDLQHGE\WLFNLQJµ6FUHHSORW¶XQGHU
WKH(;75$&7,21RSWLRQ²LQWKLVSDUWLFXODUFDVHZHKDYHGHFLGHGWKDWILYH
IDFWRUVRIIHUHGWKHEHVWVROXWLRQ

)LJXUH6DPSOH6366RXWSXWRIWKHVFUHHSORWRIIDFWRUHLJHQYDOXHV
x $LPLQJIRUDVLPSOHVWUXFWXUHWKDWLVFKRRVLQJDIDFWRUVROXWLRQLQZKLFKHDFK
YDULDEOHKDVVDOLHQWORDGLQJV LHORDGLQJVDERYH RQO\RQRQHIDFWRU LH
WKHUHDUHQRFURVVORDGLQJV DQGHDFKIDFWRUUHFHLYHVVDOLHQWORDGLQJVIURPDW
OHDVWWZRYDULDEOHV ZKLFKLVXVXDOO\GHVLUDEOHWRKDYHZHOOGHILQHGIDFWRUV
7DEOHSUHVHQWVWKHSDWWHUQPDWUL[DFFRPSDQ\LQJWKHILYHIDFWRUVROXWLRQ
UHSUHVHQWHGLQ)LJXUH7KHVROXWLRQLVLQGHHGVDWLVIDFWRU\EXWSOHDVHQRWH
WKDWVXFKFOHDUIDFWRUPDWULFHVFDQRQO\EHDFKLHYHGLIZHVHOHFWZLWKLQWKH
237,216WKHµ6RUWHGE\VL]H¶DQGµ6XSSUHVVDEVROXWHYDOXHVOHVVWKDQ¶
RSWLRQV7KHILUVWUHRUJDQL]HVWKHGLVSOD\RUGHURIWKHIDFWRUVDFFRUGLQJWRWKH
IDFWRUJURXSLQJWKHVHFRQGGHOHWHVDOOWKHIDFWRUORDGLQJVWKDWDUHWRRVPDOOWREH
VLJQLILFDQWDQGZRXOGRQO\FRQIXVHWKHSLFWXUH

7DEOH6DPSOH6366RXWSXWRIDIDFWRUPDWUL[
,GHQWLI\LQJDQGQDPLQJIDFWRUV7KHILQDOVRXUFHRIGLIILFXOW\LVWKDWHYHQLIZH
REWDLQDVWUDLJKWIRUZDUGIDFWRUVROXWLRQZLWKDFOHDUFXWVHWRIQRQRYHUODSSLQJ
IDFWRUVLWFDQEHUDWKHUGLIILFXOWWRLQWHUSUHWDQGQDPHWKHLQGLYLGXDOIDFWRUV
%HFDXVHHDFKIDFWRULVGHWHUPLQHGE\WKHYDULDEOHVWKDWKDYHWKHKLJKHVWORDGLQJVRQ
LWWKHWDVNLQLQWHUSUHWLQJDIDFWRULVWRLGHQWLI\WKHFRPPRQIHDWXUHVDPRQJWKHVH
YDULDEOHVWKDWLVWRXQGHUVWDQGWKHWKHPHWKDWEURXJKWWKHFOXVWHURIYDULDEOHV
WRJHWKHU7KHQDPHRIWKHIDFWRUWKHQVKRXOGUHIOHFWWKLVFRPPRQWKHPHDVFORVHO\
DVSRVVLEOH,QVRPHFDVHVODEHOOLQJDIDFWRULVIDLUO\VWUDLJKWIRUZDUGEXWLQFDVHV
ZKHUHWKHIDFWRUUHFHLYHVVDOLHQWORDGLQJVIURPVHHPLQJO\GLIIHUHQWYDULDEOHVWKH
ILQDOIDFWRUODEHOFDQUHIOHFWWKHUHVHDUFKHU¶VVXEMHFWLYLW\DQGWKXVFDQEHKLJKO\
FRQWHVWDEOH,QGHSHQGHQWODEHOOLQJDQGVXEVHTXHQWGLVFXVVLRQE\PRUHWKDQRQH
SHUVRQLVDQREYLRXVZD\WRUHGXFHWKHSHUVRQDOELDVHV
&OXVWHUDQDO\VLV
7KHREMHFWLYHRIµFOXVWHUDQDO\VLV¶LVWRLGHQWLI\FHUWDLQKRPRJHQRXVVXEJURXSV²RU
FOXVWHUV²RISDUWLFLSDQWVZLWKLQDJLYHQVDPSOHZKRVKDUHVLPLODUFRPELQDWLRQVRI
FKDUDFWHULVWLFV7KHUHDUHWZRPDLQW\SHVRIFOXVWHUDQDO\VLVµKLHUDUFKLFDO¶DQGµQRQ
KLHUDUFKLFDOFOXVWHULQJ¶,QKLHUDUFKLFDOFOXVWHULQJWKHILUVWVWHSLQYROYHVWKHGHILQLWLRQ
RIHDFKVDPSOHPHPEHUDVDQLQGLYLGXDOFOXVWHU6XEVHTXHQWVWHSVPHUJHWKHFORVHVW
FOXVWHUVXQWLORQHVLQJOHFOXVWHUFRQWDLQLQJDOOVDPSOHPHPEHUVLVDUULYHGDW1RQ

KLHUDUFKLFDOFOXVWHULQJIROORZVDGLIIHUHQWSDWK'XULQJWKHSURFHVVVDPSOHPHPEHUV
DUHSXWLQWRDSUHGHILQHGQXPEHURIFOXVWHUV$VDILUVWVWHSWKHVWDWLVWLFDOSURJUDPWDNHV
WKHILUVW1PHPEHUVRIWKHVDPSOH 1HTXDOVWKHQXPEHURIFOXVWHUVGHILQHGSULRUWRWKH
DQDO\VLV DQGGHILQHVWKHVHDVWKHFHQWUHVRI1FOXVWHUV)ROORZLQJWKLVWKHZKROH
GDWDVHWLVSDUWLWLRQHGLQWR1FOXVWHUVE\DVVLJQLQJVDPSOHPHPEHUVWRWKHSUHGHILQHG
FOXVWHUFHQWUHWKDWWKH\DUHFORVHVWWR)LQDOO\RQWKHEDVLVRIWKHSRVLWLRQRIWKHFOXVWHU
PHPEHUVQHZFHQWUHVDUHLGHQWLILHGDQGVDPSOHPHPEHUVDUHUHJURXSHGDFFRUGLQJWR
WKHVHQHZFHQWUHV7KHSURFHGXUHLVUHSHDWHGXQWLOWKHFHQWUHVEHFRPHVWDEOHWKDWLV
XQWLOWKH\VKRZQRFKDQJHDIWHUIXUWKHUUHJURXSLQJ
%RWKKLHUDUFKLFDODQGQRQKLHUDUFKLFDOFOXVWHULQJWHFKQLTXHVKDYHWKHLUDGYDQWDJHV
DQGGLVDGYDQWDJHV2QWKHRQHKDQGKLHUDUFKLFDOFOXVWHULQJLVGLIILFXOWWRDSSO\LIWKH
VDPSOHVL]HLVWRRODUJH2QWKHRWKHUKDQGWKHUHVXOWVRIQRQKLHUDUFKLFDOFOXVWHULQJDUH
KLJKO\GHSHQGHQWRQWKHLQLWLDOFOXVWHUFHQWUHV7RDYRLGWKHVHOLPLWDWLRQVFOXVWHULQJLV
RIWHQGRQHLQWZRVWDJHVILUVWKLHUDUFKLFDOFOXVWHULQJLVFDUULHGRXWRQDVPDOOHU
VXEVDPSOH%DVHGRQWKLVILUVWVWHSWKHQXPEHURIFOXVWHUVDQGWKHLUSRVLWLRQV LHWKH
LQLWLDOFOXVWHUFHQWUHV DUHGHILQHGDQGVXEVHTXHQWO\QRQKLHUDUFKLFDOFOXVWHULQJLVUXQ
RQWKHZKROHVDPSOHE\LQSXWWLQJWKHFOXVWHUFHQWUHVGHILQHGSUHYLRXVO\DQGWKH
SURFHGXUHRIQRQKLHUDUFKLFDOFOXVWHULQJLVLWHUDWHGXQWLOVWDEOHFOXVWHUFHQWUHVDUH
UHFHLYHG6366RIIHUVERWKKLHUDUFKLFDOFOXVWHULQJ $1$/<=(ĺ&/$66,)<ĺ
+,(5$5&+,&$/&/867(5 DQGQRQKLHUDUFKLFDOFOXVWHULQJ $1$/<=(ĺ
&/$66,)<ĺ.0($16&/867(5 ,Q9HUVLRQ6366DOVRLQWURGXFHGD
SURFHGXUHWKH\ODEHOOHGµ7ZR6WHS&OXVWHU$QDO\VLV¶ZKLFKRIIHUVDFRPELQHGPHWKRG
)RUVSHFLILFGHWDLOVRQKRZWRUXQFOXVWHUDQDO\VLVSOHDVHFRQVXOWDQ6366JXLGH
,WLVLPSRUWDQWWRSRLQWRXWWKDWFOXVWHUDQDO\VLVLVDQH[SORUDWRU\UDWKHUWKDQD
WKHRU\WKHQUHVHDUFKK\SRWKHVLVWHVWLQJWHFKQLTXH .RMLF6DERDQG/LJKWERZQ
ZKHUHE\WKHFRPSXWHUXQFRYHUVYDULRXVJURXSLQJSDWWHUQVEDVHGRQWKHPDWKHPDWLFDO
FRQILJXUDWLRQVIRXQGLQWKHOHDUQHUGDWD*URXSLQJFDQEHEDVHGRQGLIIHUHQWSULQFLSOHV
DQGFXUUHQWO\FOXVWHUDQDO\VLVODFNVDQXQGHUO\LQJERG\RIXQLYHUVDOO\HQGRUVHG
VWDWLVWLFDOWKHRU\7KXVUHVHDUFKHUVQHHGWRPDNHVHYHUDOVRPHZKDWDUELWUDU\GHFLVLRQV
UHODWLQJWRWKHPHWKRGRIFDOFXODWLRQDQGWKHVHZLOOKDYHDVWURQJLQIOXHQFHRQWKH
UHVXOWVRIWKHFODVVLILFDWLRQ,QGHHGPXOWLSOHFOXVWHUVROXWLRQVDUHXVXDOO\SRVVLEOHIURP
DVLQJOHGDWDVHW7KXVLQRUGHUWRXVHWKHWHFKQLTXHPHDQLQJIXOO\UHVHDUFKHUVQHHGWR
EHFDUHIXOWRNHHSWKHDQDO\VLVRQVRXQGWKHRUHWLFDOIRRWLQJDQGWRVXEVWDQWLDWHWKH
HPHUJLQJFOXVWHUJURXSVE\PHDQVRIYDULRXVYDOLGDWLQJSURFHGXUHV
2QHSURFHGXUHWKDWLVSDUWLFXODUO\DSSURSULDWHWRVXEVWDQWLDWHUHVXOWVLQHGXFDWLRQDO
FRQWH[WVLQYROYHVXVLQJDQH[WHUQDOFULWHULRQYDULDEOHWRIXQFWLRQDVDQLQGHSHQGHQW
LQGLFDWRURIFOXVWHUJURXSGLIIHUHQFHV%URDGO\VSHDNLQJLIWKHLGHQWLILHGVXEJURXSV
SHUIRUPVLJQLILFDQWO\GLIIHUHQWO\RQWKHJLYHQFULWHULRQPHDVXUHWKLVLQGLFDWHVWKDWWKH
FODVVLILFDWLRQUHVXOWHGLQPHDQLQJIXOO\GLIIHUHQWJURXSLQJVDQGWKXVFRQILUPVWKH
YDOLGLW\RIWKHSDUWLFXODUJURXSLQJVROXWLRQ:HFDQXVHDVLPSOH$129$WRFRPSDUH
WKHFULWHULRQYDULDEOHDFURVVWKHFOXVWHUJURXSV

6WUXFWXUDOHTXDWLRQPRGHOOLQJ
6LPLODUWRIDFWRUDQDO\VLVµVWUXFWXUDOHTXDWLRQPRGHOOLQJ¶ 6(0 LVXVHGWRLQWHUSUHW
WKHUHODWLRQVKLSDPRQJVHYHUDOYDULDEOHVZLWKLQDVLQJOHIUDPHZRUN,WVELJDGYDQWDJH
RYHUIDFWRUDQDO\VLVLVWKDWDQ6(0PRGHODOVRLQFOXGHVGLUHFWLRQDOSDWKVEHWZHHQWKH
YDULDEOHVDQGQRWMXVWLQIRUPDWLRQDERXWKRZWKHYDULDEOHVKDQJWRJHWKHU7KXV6(0
PDNHVLWSRVVLEOHWRWHVWFDXVH±HIIHFWUHODWLRQVKLSVEDVHGRQFRUUHODWLRQDOGDWD ZKLFK
DVZHKDYHVHHQHDUOLHUFRUUHODWLRQDQDO\VLVFDQQRWSURYLGH ZKLFKPDNHV6(0D
SRZHUIXODQDO\WLFDOWRRODVLWFRPELQHVWKHYHUVDWLOLW\RIFRUUHODWLRQDQDO\VLVDQGWKH
FDXVDOYDOLGLW\RIH[SHULPHQWDOUHVHDUFK7KHVWUXFWXUDOUHODWLRQVKLSVXQGHU
LQYHVWLJDWLRQDUHWHVWHGE\PHDQVRIDVHULHVRIUHJUHVVLRQHTXDWLRQV KHQFHWKHZRUGV
µVWUXFWXUDOHTXDWLRQ¶LQWKHQDPH WKHZRUGµPRGHOOLQJ¶LQWKHQDPHVLPSO\VXJJHVWV
WKDWWKHVWUXFWXUDOUHODWLRQVFDQEHPRGHOOHGSLFWRULDOO\)LJXUHFRQWDLQVDW\SLFDO
6(0PRGHO7KHODUJHFLUFOHVLQGLFDWHODWHQWWKHRUHWLFDOFRPSRQHQWVZKLFKDUH
GHILQHGE\WKHUHDOREVHUYHGPHDVXUHV LQWKLVFDVHFRPSRVLWHTXHVWLRQQDLUHVFDOHV LQ
WKHUHFWDQJXODUER[HV7KHVPDOOFLUFOHVLQGLFDWHHUURUYDULDEOHVWKDWZHVKRXOGLJQRUHDW
WKLVVWDJH DQGWKH\DUHRIWHQOHIWRXWLQ6(0RXWSXWILJXUHV 7KHDUURZVLQGLFDWHWKH
OLQNEHWZHHQWKHYDULDEOHVDQGWKHFRHIILFLHQWVQH[WWRWKHDUURZVFDQEHLQWHUSUHWHGLQ
WKHVDPHZD\DVFRUUHODWLRQFRHIILFLHQWV

)LJXUH6DPSOH6(0VWUXFWXUDOPRGHO
7RVWDUWDSSO\LQJ6(0WRGDWDUHVHDUFKHUVQHHGDQH[SOLFLWO\VWDWHGWKHRUHWLFDO
PRGHOLQZKLFKWKHPDLQYDULDEOHVDUHTXDQWLILHGDQGWKHLUGLUHFWLRQDOUHODWLRQVKLSLV
FOHDUO\VWDWHG7KH6(0SURFHGXUHLVWKHQXVHGWRFRQILUPRUUHMHFWWKHYDOLGLW\RIWKLV
PRGHO²WKXV6(0LVQRWDQH[SORUDWRU\EXWDFRQILUPDWRU\WHFKQLTXHDOWKRXJKLWLV
FDSDEOHRIVXJJHVWLQJFHUWDLQDGMXVWPHQWVWRWKHPRGHOWHVWHGE\SURYLGLQJ
µPRGLILFDWLRQLQGLFHV¶:KHQVHWWLQJXSDPRGHOUHVHDUFKHUVQHHGWRGRWZRWKLQJV

'HVFULEHWKHUHODWLRQVKLSEHWZHHQWKHPHDVXUHGYDULDEOHVDQGWKHK\SRWKHVL]HG
ODWHQWYDULDEOHV IRUH[DPSOHVSHFLI\WKDWWKHWKHRUHWLFDOFRQVWUXFWRIµVHOI
FRQILGHQFH¶LVPHDVXUHGE\VD\WKUHHYDULDEOHV/XVHDQ[LHW\SHUFHLYHGVHOI
FRPSHWHQFHDQGDVHOIUHSRUWFRQILGHQFHVFDOH WKLVUHVXOWVLQDPHDVXUHPHQW
PRGHO)RUFRPSOH[VWXGLHVWKDWFRYHUDZLGHUDQJHRIYDULDEOHVZHXVXDOO\KDYH
PRUHWKDQRQHPHDVXUHPHQWPRGHO
3RVLWWKHFDVXDOOLQNVEHWZHHQDOOWKHODWHQW WKHRUHWLFDO YDULDEOHVXQGHUVWXG\WKLV
UHVXOWVLQDIXOOVWUXFWXUDOPRGHOZKLFKLVLQHIIHFWDFRPELQDWLRQRIDOOWKH
PHDVXUHPHQWPRGHOV
6(0WKHQWHVWVWKHDGHTXDF\RIERWKPRGHOW\SHVSURYLGHVJRRGQHVVRIILW
PHDVXUHVIRUWKHIXOO LHFRPELQHG VWUXFWXUDOPRGHODQGSURGXFHVPRGLILFDWLRQ
LQGLFHVIRUWKHSXUSRVHRILPSURYHPHQW$VPHQWLRQHGHDUOLHUWRUXQ6(0ZHQHHG
VSHFLDOVRIWZDUH&XUUHQWO\/,65(/$026DQG(46DUHWKHWKUHHPRVWSRSXODU
VWDWLVWLFDOSDFNDJHVXVHGIRUFDUU\LQJRXW6(0ZLWK6366VXSSRUWLQJ$026
$VHYHU\DQDO\WLFDOWHFKQLTXH6(0KDVFHUWDLQOLPLWDWLRQVRQHRIZKLFKLQ
SDUWLFXODUQHHGVWREHERUQLQPLQG6(0SURYLGHVVHYHUDOLQGLFHVWRVKRZKRZ
VDWLVIDFWRU\WKHILWRIWKHILQDOPRGHOLVDQGWKHVHFDQEHXVHGWRFRPSDUHDOWHUQDWLYH
PRGHOVDQGWRUHMHFWLOOILWWLQJPRGHOV+RZHYHUHYHQDVROXWLRQZLWKDQDGHTXDWHILWLV
RQO\RQHRIPDQ\WKDWPLJKWILWWKHGDWDHTXDOO\ZHOO7KXV6(0LVQRWWKHµEHDOODQG
HQGDOOWRUHVHDUFKHQGHDYRXUVFRQFHUQHGZLWKLQIHUULQJFDXVDWLRQIURPFRUUHODWLRQDO
GDWD¶ *DUGQHU ,WGRHVQRWLGHQWLI\FDXVDWLRQEXWRQO\LQIRUPVWKH
UHVHDUFKHUDVWRZKHWKHUDK\SRWKHVL]HGFDXVH±HIIHFWUHODWLRQVKLSLVFRQFHLYDEOHEDVHG
RQWKHWRWDODPRXQWRIGDWD
0HWDDQDO\VLV
µ0HWDDQDO\VLV¶DVDVWDWLVWLFDOWHUPZDVLQWURGXFHGLQE\*HQH*ODVVWRGHVFULEH
DV\VWHPDWLFDSSURDFKWRUHYLHZLQJDQGV\QWKHVL]LQJDODUJHQXPEHURISXEOLVKHG
UHVHDUFKVWXGLHVRQDWRSLF$VKHGHILQHGLWµ0HWDDQDO\VLVUHIHUVWRWKHDQDO\VLVRIWKH
DQDO\VHV«,WFRQQRWHVDULJRURXVDOWHUQDWLYHWRWKHFDVXDOQDUUDWLYHGLVFXVVLRQVRI
UHVHDUFKVWXGLHVZKLFKW\SLI\RXUDWWHPSWVWRPDNHVHQVHRIWKHUDSLGO\H[SDQGLQJ
UHVHDUFKOLWHUDWXUH¶ S 7KXVPHWDDQDO\VLVFDQEHVHHQDVDµTXDQWLWDWLYHOLWHUDWXUH
UHYLHZ¶,WDLPVDWGHWHUPLQLQJDVXPPDU\HVWLPDWHRIWKHµHIIHFWVL]HV¶UHSRUWHGLQWKH
YDULRXVVWXGLHVWKXVSURGXFLQJDFRPELQHGVXSHURUGLQDWHUHVXOWWKDWV\QWKHVL]HVWKH
LQGLYLGXDOVWXGLHV,Q6HFWLRQZHVDZWKDWHIIHFWVL]HSURYLGHVLQIRUPDWLRQDERXW
WKHPDJQLWXGHRIDSKHQRPHQRQREVHUYHGLQDQLQYHVWLJDWLRQDQGWKDWLWLVDOVRXVHIXOLQ
HQDEOLQJXVWRFRPSDUHWKHUHVXOWVUHSRUWHGLQGLIIHUHQWVWXGLHVEHFDXVHHIIHFWVL]H
LQGLFDWRUVDUHWUDQVIRUPDWLRQVRQWRDFRPPRQVFDOH0HWDDQDO\VLVFDSLWDOL]HVRQWKLV
FRQWH[WLQGHSHQGHQWQDWXUHRIWKHVHLQGLFHV7KHSURFHGXUHFRQVLVWVRIVHYHUDOVWHSV
&RQGXFWDOLWHUDWXUHVHDUFKWRLGHQWLI\DQH[WHQVLYHVDPSOHRIUHOHYDQWUHVHDUFK
VWXGLHV6FKRODUVW\SLFDOO\XVHHOHFWURQLFGDWDEDVHV VXFKDV(5,&RUWKH/LQJXLVWLFV
DQG/DQJXDJH%HKDYLRXU$EVWUDFWV IRUWKLVSXUSRVH

6FUHHQRXWDOOWKHVWXGLHVWKDWGRQRWPHHWFHUWDLQZHOOGHILQHGUHVHDUFKFULWHULDRU
ZKLFKGRQRWFRQWDLQFHUWDLQQHFHVVDU\WHFKQLFDOGHWDLOV
,GHQWLI\DQDSSURSULDWHHIIHFWVL]HLQGLFDWRULQHDFKVWXG\6LQFHWKH$3$
*XLGHOLQHVKDYHHQFRXUDJHGUHVHDUFKHUVWRUHSRUWHIIHFWVL]HVDQGWKH7(62/
4XDUWHUO\TXDQWLWDWLYHJXLGHOLQHVDOVRUHTXLUHWKHUHSRUWLQJRIHIIHFWVL]H,QVSLWHRI
WKLVDV1RUULVDQG2UWHJD KDYHIRXQGPDQ\HPSLULFDOVWXGLHVSXEOLVKHGLQ
DSSOLHGOLQJXLVWLFMRXUQDOVVWLOOIDLOWRSURYLGHHIIHFWVL]HLQGLFHVDQGWKHUHIRUH
UHVHDUFKHUVGRLQJDPHWDDQDO\VLVQHHGWREHSUHSDUHGWRFRQGXFWVHFRQGDU\
DQDO\VHVRIWKHSULPDU\UHVHDUFKGDWDWRGHULYHDSSURSULDWHHIIHFWVL]HLQGLFHVIURP
WKHSXEOLVKHGGHWDLOV 6HH1RUULVDQG2UWHJD¶VVWXG\IRUDQXPEHURIXVHIXO
IRUPXODHWREHXVHG
7KHILQDOWDVNLVWRFDOFXODWHWKHVXPPDU\HIIHFW7KHUHDUHVHYHUDOZD\VRIGRLQJ
WKLV SOHDVHFRQVXOWDVWDWLVWLFVJXLGH DQGDQ,QWHUQHWVHDUFKZLOODOVRORFDWHVHYHUDO
IUHHFDOFXODWRUVRIWZDUHIRUH[DPSOH0HWD6WDWE\5XGQHU*ODVV(YDUWWDQG
(PHU\ 6HHWKHZZZHGUHVRUJPHWDZHEVLWHIRUWKLVDQGRWKHUVRIWZDUH
7KXVPHWDDQDO\VLVRIIHUVDV\VWHPDWLFZD\RIRUJDQL]LQJDQGV\QWKHVL]LQJWKH
UDSLGO\H[SDQGLQJUHVHDUFKOLWHUDWXUH1HYHUWKHOHVVWKHTXDOLW\RIWKHDQDO\VLV
XOWLPDWHO\GHSHQGVRQWKHTXDOLW\RIWKHXQGHUO\LQJVWXGLHV

4XDOLWDWLYHGDWDDQDO\VLV
:RUNLQJZLWKQDWXUDOO\RFFXUULQJGDWDLVLQHYLWDEO\DPHVV\HQWHUSULVH«
/HXQJ+DUULVDQG5DPSWRQ
:HVDZLQ&KDSWHUWKDWTXDOLWDWLYHUHVHDUFKLVIDUIURPEHLQJDXQLIRUPDSSURDFKEXW
LVFKDUDFWHUL]HGE\GLYHUVLW\$V3XQFK SRLQWVRXWSHUKDSVQRZKHUHLVWKDW
GLYHUVLW\PRUHDSSDUHQWWKDQLQDSSURDFKHVWRWKHDQDO\VLVRITXDOLWDWLYHGDWD,QGHHG
WKHWHUPµTXDOLWDWLYHGDWDDQDO\VLV¶LVXVHGE\YDULRXVVFKRODUVWRGHQRWHGLIIHUHQW
DFWLYLWLHVIURPLPDJLQDWLYHDQGDUWIXOVSHFXODWLRQWRIROORZLQJZHOOGHILQHGDQDO\WLFDO
PRYHVIURPGHGXFWLYHFDWHJRUL]DWLRQWRLQGXFWLYHSDWWHUQILQGLQJ,QVRPHZD\VWKH
VPDOOHVWFRPPRQGHQRPLQDWRUDPRQJVWWKHYDULRXVDSSURDFKHVLVWKHUHMHFWLRQRIWKH
XVHRITXDQWLWDWLYHVWDWLVWLFDOWHFKQLTXHV%XWRIFRXUVHWKLVLVH[DFWO\DWWKHKHDUWRI
TXDOLWDWLYHGDWDDQDO\VLVWRGHYHORSDQGIROORZFHUWDLQSULQFLSOHGDQDO\WLFDOVHTXHQFHV
ZLWKRXWEHLQJWLHGE\WKHFRQVWUDLQWVRIWKHSURFHGXUHVDQGVDFULILFLQJWKHUHVHDUFKHU¶V
FUHDWLYHOLEHUW\RILQWHUSUHWDWLRQ,QRWKHUZRUGVWRµ>V@HHNIRUPDOL]DWLRQDQGGLVWUXVWLW¶
0LOHVDQG+XEHUPDQ
+DYLQJVDLGWKHDERYHZKHQZHJREH\RQGWKHSKLORVRSKLFDOGLVFXVVLRQVDQGORRN
DWZKDWDFWXDOO\KDSSHQVZKHQZHVLWGRZQLQIURQWRIWKHTXDOLWDWLYHGDWDDQGVWDUW
WHDVLQJRXWWKHKLGGHQPHDQLQJIURPLWZHDFWXDOO\ILQGPDQ\FRPPRQDOLWLHVDQGDV
ZLOOEHVKRZQODWHUDJUHDWGHDORIWKHDQDO\WLFDOSURFHVVLVPDGHXSRIJHQHULF
PHWKRGLQGHSHQGHQWSURFHGXUHV$GPLWWHGO\WKHUHH[LVWFHUWDLQGLIIHUHQFHVLQSUDFWLFH
EXWHYHQWKHVHFDQEHVXPPDUL]HGLQWHUPVRIDIHZNH\LVVXHVDQGGLOHPPDV7KXV
ZLWKLQWKHGLYHUJHQFHRITXDOLWDWLYHDQDO\VLVWKHUHLVDJUHDWGHDORIFRQYHUJHQFHLQKRZ
VFKRODUVµWDPHWKHGDWD¶ 5LFKDUGV DQGWU\WRPDNHVHQVHRILW )RUDYHU\XVHIXO
GHVFULSWLRQIURPDVHFRQGODQJXDJHSHUVSHFWLYHVHH5LFKDUGV&KDSWHU
,QWKHIROORZLQJ,ILUVWGLVFXVVIRXUSULQFLSOHVRITXDOLWDWLYHGDWDDQDO\VLVWKDW,
FRQVLGHUFHQWUDODQGWKHQRXWOLQHWKHPDLQGDWDDQDO\WLFDODSSURDFKHVGHYHORSHGRYHU
WKHSDVWWKUHHGHFDGHV2IWKHVHRQHDSSURDFK²µJURXQGHGWKHRU\¶²ZLOOEHH[DPLQHG
LQPRUHGHWDLODIWHUSUHVHQWLQJWKHJHQHULFSKDVHVRITXDOLWDWLYHGDWDDQDO\VLV7KH
FKDSWHULVFRQFOXGHGE\ORRNLQJDWKRZFRPSXWHUVFDQKHOSXVZLWKKDQGOLQJWKHGDWD
0DLQSULQFLSOHVRITXDOLWDWLYHGDWDDQDO\VLV
7KRXVDQGVRISDJHVKDYHEHHQZULWWHQDERXWWKHPDLQSULQFLSOHVRITXDOLWDWLYHGDWD
DQDO\VLVZKLFKUHIOHFWVWKHFRPSOH[LW\RIWKHTXHVWLRQ/HWXVILUVWORRNDWIRXUJHQHUDO
LVVXHVWKDWDUHFHQWUDOWRWKHXQGHUVWDQGLQJRIWKHSURFHVV D 7KHODQJXDJHEDVHG
QDWXUHRIWKHDQDO\VLV E WKHLWHUDWLYHSURFHVV F WKHWHQVLRQEHWZHHQIRUPDOL]DWLRQ
DQGLQWXLWLRQDQG G WKHWHQVLRQEHWZHHQEHLQJPHWKRGRORJ\ERXQGDQGPHWKRGRORJ\
LQGHSHQGHQW

/DQJXDJHEDVHGDQDO\VLV
:HVDZLQ&KDSWHUWKDWPRVWTXDOLWDWLYHGDWDLVWUDQVIRUPHGLQWRDWH[WXDOIRUP²IRU
H[DPSOHLQWHUYLHZWUDQVFULSWLRQV²DQGWKHDQDO\VLVLVGRQHSULPDULO\ZLWKZRUGV
7KXVTXDOLWDWLYHGDWDDQDO\VLVLVLQKHUHQWO\DODQJXDJHEDVHGDQDO\VLV ,QSULQFLSOHLW
LVDOVRSRVVLEOHWRZRUNZLWKLPDJHVEXWWKLVLVIDUOHVVSURPLQHQWWKDQWKHODQJXDJH
EDVHGDQDO\VLVDQGWKHUHIRUHZLOOQRWEHGLVFXVVHGLQWKLVFKDSWHU :HPXVWUHDOL]HWKDW
WKHODQJXDJHVSHFLILFQDWXUHRITXDOLWDWLYHDQDO\VLVDFWXDOO\IDYRXUVDSSOLHGOLQJXLVWV
EHFDXVHGLVFRXUVHDQDO\WLFDOWHFKQLTXHV LQFOXGLQJFRQYHUVDWLRQDQDO\VLV DUHSDUWRI
RXUFRUHGLVFLSOLQH$VGLVFXVVHGLQ&KDSWHUWKLVERRNGRHVQRWFRYHUVXFKOLQJXLVWLF
DSSURDFKHVEHFDXVHWKHDSSOLHGOLQJXLVWLFVOLWHUDWXUHFRQWDLQVDULFKYDULHW\RI
VSHFLDOL]HGWH[WVLQWKHVHDUHDV
,WHUDWLYHSURFHVV
7KHUHLVQRSDUWLFXODUPRPHQWZKHQGDWDDQDO\VLVEHJLQV
6WDNH
4XDQWLWDWLYHDQDO\VLVLVRUGHUO\LQDOLQHDUPDQQHUEDVHGRQWKHNQRZOHGJHRIWKH
UHOHYDQWOLWHUDWXUHZHGHVLJQWKHUHVHDUFKLQVWUXPHQWVWKHQFROOHFWSURFHVVDQGDQDO\VH
WKHGDWDDQGILQDOO\ZULWHXSWKHUHVXOWV(DFKVWHSLVZHOOGHILQHGDQGEXLOGVRQWKH
SUHYLRXVRQH,QFRQWUDVWTXDOLWDWLYHUHVHDUFKLVLWHUDWLYHXVLQJDQRQOLQHDUµ]LJ]DJ¶
SDWWHUQZHPRYHEDFNDQGIRUWKEHWZHHQGDWDFROOHFWLRQGDWDDQDO\VLVDQGGDWD
LQWHUSUHWDWLRQGHSHQGLQJRQWKHHPHUJHQWUHVXOWV)RUH[DPSOHLWLVQRWDWDOOXQXVXDOWR
GHFLGHZHOOLQWRWKHDQDO\WLFDOSKDVHWKDWZHQHHGWRFROOHFWVRPHPRUHGDWDRID
VSHFLILFVRUWRUWRJREDFNWRWKHRULJLQDOGDWDWUDQVFULSWVWRUHFRGHWKHWH[WDFFRUGLQJWR
VRPHQHZO\FRQFHLYHGFDWHJRULHV7KDWLVXQOLNHLQ48$1UHVHDUFKZHGRQRWQHHGDQ
H[WHQVLYHLQLWLDOGDWDVHWWRVWDUW48$/GDWDDQDO\VLVEHFDXVHDV+HVVH%LEHUDQG/HDY\
SRLQWRXWµ$OLWWOHELWRIGDWDFDQJRDORQJZD\LQJDWKHULQJPHDQLQJ¶,Q
IDFWWKHVHVFKRODUVZDUQXVDJDLQVWEHLQJWHPSWHGWRJDWKHUWRRPXFKGDWDLQLWLDOO\
EHFDXVHWKLVPD\GLVWUDFWXVIURPUHIOHFWLQJRQWKHGHWDLOV²DFRPPRQPLVWDNHZLWK
QRYLFHUHVHDUFKHUV 6HHDOVR6HFWLRQ
$NH\WHUPLQTXDOLWDWLYHGDWDDQDO\VLVLVµVDWXUDWLRQ¶:HKDYHDOUHDG\XVHGLWZLWK
UHJDUGWRTXDOLWDWLYHVDPSOLQJUHIHUULQJWRWKHSRLQWZKHQIXUWKHUGDWDGRHVQRWVHHPWR
DGGQHZLQIRUPDWLRQDQGLQGDWDDQDO\VLVWKHWHUPFDUULHVWKHVDPHPHDQLQJQDPHO\
WKDWWKHLWHUDWLYHSURFHVVVWRSVSURGXFLQJQHZWRSLFVLGHDVDQGFDWHJRULHVDQGWKH
SURMHFWµOHYHOVRII¶ 5LFKDUGV 7KLVLVWKHSRLQWZKHQWKHUHVHDUFKHUJDLQVDVHQVH
RIZKDWLVUHOHYDQWDQGZKDWIDOOVRXWVLGHWKHVFRSHRIWKHLQYHVWLJDWLRQ2IFRXUVHDV
5LFKDUGVVWUHVVHVVKDOORZGDWDDQGHVSHFLDOO\ZKHQZHDUHQRWDVNLQJTXHVWLRQVWKDW
JREH\RQGWKHVXUIDFHPHDQLQJFDQVDWXUDWHDWDOHYHORIWULYLDOLW\²VDWXUDWLRQRQO\
LQGLFDWHVWKDWZHKDYHFRYHUHGWKHEUHDGWKRIRXUGDWDZLWKWKHOHYHORIDQDO\VLVXVHG
6XEMHFWLYHLQWXLWLRQYHUVXVIRUPDOL]DWLRQ

:HVDZLQ6HFWLRQWKDWWKHLQLWLDOUDZTXDOLWDWLYHGDWDVHWLVOLNHO\WREHµPHVV\¶LI
QRWFKDRWLF7KHFKDOOHQJHIRUWKHDQDO\VWLVWREULQJVRPHLQVLJKWIXORUGHUWRWKH
PXOWLSOHDFFRXQWVRIKXPDQVWRULHVDQGSUDFWLFHVFROOHFWHGRUREVHUYHG7KLVLVWKH
SRLQWZKHQZHFDQUHVRUWWRWZRIXQGDPHQWDOO\GLIIHUHQWDQDO\WLFDODSSURDFKHV:HFDQ
HLWKHUUHO\RQRXUVXEMHFWLYHLQWXLWLRQWRILQGDFUHDWLYHZD\RXWRIWKHPD]HRUZHFDQ
IROORZIRUPDOL]HGDQDO\WLFDOSURFHGXUHVWKDWZLOOKHOSXVWRXQFRYHUWKHKLGGHQ
PHDQLQJLQDV\VWHPDWLFVWHSE\VWHSSURFHVV%RWKDSSURDFKHVKDYHFRQVLGHUDEOH
VXSSRUWDPRQJVWTXDOLWDWLYHUHVHDUFKHUV
7KHPDLQDUJXPHQWLQIDYRXURIDSULPDULO\LQWXLWLYHDSSURDFKKDVWRGRZLWKWKH
LQKHUHQWLPSRUWDQFHDWWDFKHGWRWKHVXEMHFWLYHDQGUHIOH[LYHLQYROYHPHQWRIWKH
UHVHDUFKHULQWKHDQDO\VLV VHH6HFWLRQ DQGWKHQHHGWRPDLQWDLQDIOXLGDQG
FUHDWLYHDQDO\WLFDOSRVLWLRQWKDWLVQRWFRQVWUDLQHGE\SURFHGXUDOWUDGLWLRQVDQGWKDW
DOORZVQHZWKHRULHVWRHPHUJHIUHHO\,QIDFWRQHRIWKHPDLQFULWLFLVPVOHYHOOHGDW
TXDQWLWDWLYHUHVHDUFKHUVE\WKHLUTXDOLWDWLYHFROOHDJXHVLVWKHULJLGLW\RIWKHDQDO\VLV,Q
FRQWUDVWTXDOLWDWLYHDQDO\VLVQHHGVWREHIOH[LEOHGDWDOHGDQG²DFRPPRQWHUPXVHG
E\VFKRODUVLQWKHLQWXLWLYHFDPS²µDUWIXO¶
:KLOHWKHQHHGIRUIOH[LELOLW\LVQRWTXHVWLRQHGE\DQ\TXDOLWDWLYHUHVHDUFKHUWKHUHLV
QRDJUHHPHQWDVWRZKDWH[WHQWWKLVµDUWIXOIOXLGLW\¶WROHUDWHVVWUXFWXUHLQWKHDQDO\WLFDO
SURFHVV'RZHDOZD\VVWDUWIURPVFUDWFKZKHQZHIDFHDQHZGDWDVHWRUFDQZHGUDZ
RQPHWKRGRORJLFDONQRZOHGJHLQDSSO\LQJFHUWDLQHVWDEOLVKHGWHFKQLTXHVIRUWKH
SXUSRVHRIUHDFKLQJFRPSOH[KLGGHQPHDQLQJ"7KHPRVWLQIOXHQWLDOSURSRQHQWVRID
PRUHVWUXFWXUHGDSSURDFKWRGDWDDQDO\VLVKDYHEHHQ0LOHVDQG+XEHUPDQ ZKR
PDLQWDLQWKDWWKHODFNRIDFFHSWHGDQDO\WLFDOFRQYHQWLRQVLVQRWDVWUHQJWKEXWD
ZHDNQHVVRITXDOLWDWLYHGDWDDQDO\VLVEHFDXVHWKHLQWXLWLYHUHIOH[LYHQHVVGRHVQRWLQ
LWVHOISURYLGHDQ\VDIHJXDUGVDJDLQVWVLPSO\EHLQJZURQJ,QDGGLWLRQDV0LOHVDQG
+XEHUPDQDUJXHWKHUHDUHFHUWDLQWDVNVWKDWQHHGWREHFDUULHGRXWUHJDUGOHVVRIRXU
DSSURDFKWKH\KLJKOLJKWWKUHHVXFKRSHUDWLRQVµGDWDUHGXFWLRQ¶µGDWDGLVSOD\¶DQG
µGDWDLQWHUSUHWDWLRQ¶$VWKHVHDXWKRUVFODLPRQO\E\XVLQJFHUWDLQV\VWHPDWLF²WKDWLV
IRUPDOL]HG²SURFHGXUHVLQDWUDQVSDUHQWPDQQHUWRDFFRPSOLVKWKHVHWDVNVFDQZH
FRQYLQFHRXUDXGLHQFHVWKDWRXUFRQFOXVLRQVDUHYDOLG 6HHDOVR6HFWLRQRQ
µTXDOLW\FULWHULDLQTXDOLWDWLYHUHVHDUFK¶
7KXVWKHEDVLFFKDOOHQJHRITXDOLWDWLYHGDWDDQDO\VLVLVWRDFKLHYHµULJRURXV
IOH[LELOLW\¶RUµGLVFLSOLQHGDUWIXOQHVV¶E\DSSO\LQJSURFHGXUHVDQGIUDPHZRUNVWKDWDUH
FRQGXFLYHWRJHQHUDWLQJQHZLQVLJKWVUDWKHUWKDQDFWLQJDVDFRQVWUDLQLQJPRXOGWKDW
GRHVQRWOHWQHZUHVXOWVHPHUJH7KHYDULRXVTXDOLWDWLYHGDWDDQDO\WLFDOPHWKRGVGLIIHU
LQWKHZD\VE\ZKLFKWKH\WU\WRVWDQGDUGL]HWKHSURFHVVZKLOHPDLQWDLQLQJWKH
UHVHDUFKHU¶VVXEMHFWLYHIUHHGRP
*HQHULFDQDO\WLFDOPRYHVYHUVXVVSHFLILFPHWKRGRORJLHV
$ILQDODUHDWKDWLVFHQWUDOWRWKHXQGHUVWDQGLQJRITXDOLWDWLYHGDWDDQDO\VLVLVWKH
WHQVLRQEHWZHHQDGYRFDWLQJWKHXVHRIDVSHFLILFDQGFRKHUHQWPHWKRGRORJ\VXFKDV

JURXQGHGWKHRU\ VHH6HFWLRQ RUHPSKDVL]LQJJHQHUDODQGJHQHULFDQDO\WLFDO
PRYHVVXFKDVµFRGLQJ¶DQGµPHPRLQJ¶ VHH6HFWLRQVDQG :HILQG
VWURQJYLHZVIURPERWKVLGHVH[SUHVVHGLQWKHOLWHUDWXUHEXWLWLVP\LPSUHVVLRQWKDWWKH
DFWXDOSUDFWLFHRIVWXGLHVSXEOLVKHGLQUHVHDUFKMRXUQDOVWHQGVWRIDYRXUWKHODWWHU
DSSURDFKDQGZKHQVFKRODUVGRQRWZLVKWRDIILOLDWHWKHPVHOYHVWRRFORVHO\ZLWKD
VSHFLILFPHWKRGRORJ\WKH\RIWHQXVHWKHEURDGWHUPRIµTXDOLWDWLYHFRQWHQWDQDO\VLV¶WR
FKDUDFWHUL]HWKHFROOHFWLRQRIJHQHULFTXDOLWDWLYHDQDO\WLFDOPRYHVWKDWDUHDSSOLHGWR
HVWDEOLVKSDWWHUQVLQWKHGDWD,QWKHQH[WVHFWLRQ ,GHVFULEHLQGHWDLOWKHPRVW
FRPPRQµSDWWHUQILQGLQJ¶PRYHVRIWKLVJHQHULFDSSURDFKDQGLQWKHVXEVHTXHQW
VHFWLRQ ,SURYLGHDFRXQWHUSRLQWE\SUHVHQWLQJWKHVSHFLILFPHWKRGRORJ\RI
JURXQGHGWKHRU\
4XDOLWDWLYHFRQWHQWDQDO\VLV
&RQWHQWDQDO\VLVKDVUHFHQWO\EHFRPHFORVHO\DVVRFLDWHGZLWKTXDOLWDWLYHUHVHDUFKDQG
ZHFDQWKHUHIRUHHDVLO\IRUJHWWKDWLWDFWXDOO\RULJLQDWHVIURPDTXDQWLWDWLYHDQDO\WLFDO
PHWKRGRIH[DPLQLQJZULWWHQWH[WVWKDWLQYROYHVWKHFRXQWLQJRILQVWDQFHVRIZRUGV
SKUDVHVRUJUDPPDWLFDOVWUXFWXUHVWKDWIDOOLQWRVSHFLILFFDWHJRULHV%HFDXVHTXDOLWDWLYH
GDWDLVW\SLFDOO\WH[WXDO VHH6HFWLRQ 48$1FRQWHQWDQDO\VLVKDVEHHQ
WUDQVIHUUHGWRWKHGRPDLQRI48$/UHVHDUFKZLWKRQHIXQGDPHQWDOFKDQJHXQOLNHWKHLU
SUHFRQFHLYHGTXDQWLWDWLYHFRXQWHUSDUWVWKHTXDOLWDWLYHFDWHJRULHVXVHGLQFRQWHQW
DQDO\VLVDUHQRWSUHGHWHUPLQHGEXWDUHGHULYHGLQGXFWLYHO\IURPWKHGDWDDQDO\VHG
$QRWKHUZD\RIGLVWLQJXLVKLQJTXDQWLWDWLYHDQGTXDOLWDWLYHFRQWHQWDQDO\VHVLVE\
UHIHUULQJWRWKHIRUPHUDVµPDQLIHVWOHYHODQDO\VLV¶EHFDXVHLWLVDQREMHFWLYHDQG
GHVFULSWLYHDFFRXQWRIWKHVXUIDFHPHDQLQJRIWKHGDWDDQGWKHODWWHUDVµODWHQWOHYHO
DQDO\VLV¶EHFDXVHLWFRQFHUQVDVHFRQGOHYHOLQWHUSUHWLYHDQDO\VLVRIWKHXQGHUO\LQJ
GHHSHUPHDQLQJRIWKHGDWD
,QWKLVVHFWLRQ,FRQFHQWUDWHRQODWHQWFRQWHQWDQDO\VLV7KLVW\SHRIDQDO\VLVIROORZV
WKHYHU\JHQHUDOL]HGVHTXHQFHRIFRGLQJIRUWKHPHVORRNLQJIRUSDWWHUQVPDNLQJ
LQWHUSUHWDWLRQVDQGEXLOGLQJWKHRU\ (OOLVDQG%DUNKXL]HQ 7RWKLVZHFDQDGGDQ
LQLWLDOSKDVHRIµWUDQVFULELQJWKHGDWD¶EHFDXVHPXFK WKRXJKQRWDOO TXDOLWDWLYHGDWD
FRQVLVWVRIUHFRUGHGVSRNHQGDWDWKDWQHHGVWREHWUDQVFULEHGDQGDVZHZLOOVHH
WXUQLQJUHFRUGLQJVLQWRWUDQVFULSWVDOUHDG\FRQWDLQVLQWHUSUHWLYHHOHPHQWV7KXV,ZLOO
FHQWUHWKHIROORZLQJGLVFXVVLRQDURXQGIRXUSKDVHVRIWKHDQDO\WLFDOSURFHVV D
WUDQVFULELQJWKHGDWD E SUHFRGLQJDQGFRGLQJ F JURZLQJLGHDV²PHPRVYLJQHWWHV
SURILOHVDQGRWKHUIRUPVRIGDWDGLVSOD\DQG G LQWHUSUHWLQJWKHGDWDDQGGUDZLQJ
FRQFOXVLRQV
7UDQVFULELQJWKHGDWD
7KHILUVWVWHSLQGDWDDQDO\VLVLVWRWUDQVIRUPWKHUHFRUGLQJVLQWRDWH[WXDOIRUP 2I
FRXUVHLQFDVHVZKHQZHXVHZULWWHQGDWDVXFKDVGRFXPHQWVFRUUHVSRQGHQFHRUGLDU\
HQWLUHVWKLVSKDVHPD\QRWEHDSSOLFDEOH 7KLVLVDWLPHFRQVXPLQJSURFHVV

SDUWLFXODUO\LIWKHWH[WDOVRQHHGVWREHWUDQVODWHG ²GHSHQGLQJRQWKHTXDOLW\RIWKH
UHFRUGLQJWUDQVFULELQJDRQHKRXULQWHUYLHZFDQWDNHDVPXFKDV±KRXUVEXWZLWK
YHU\ILQHWXQHGWUDQVFULSWLRQVLQZKLFKIRUH[DPSOHSDXVHVQHHGWREHPHDVXUHGLWFDQ
WDNHDVORQJDVKRXUV 'XIILQSUHVV $OWKRXJKLWLVSRVVLEOHWRZRUNZLWKWDSHVRQO\
LHZLWKRXWDWUDQVFULEHGYHUVLRQZKLFKLVXVXDOO\UHIHUUHGWRDVµWDSHDQDO\VLV¶ RU
ZLWKRQO\SDUWLDOWUDQVFULSWLRQVZHVKRXOGUHVRUWWRWKHVHODERXUVDYLQJVWUDWHJLHVRQO\
LQVSHFLDOFDVHVDQGWKHGHIDXOWSUDFWLFHQHHGVWREHWKHSUHSDUDWLRQRIIXOOZULWWHQ
WUDQVFULSWVRIRXUUHFRUGHGGDWD )RUDPRUHGHWDLOHGGLVFXVVLRQVHHQH[WVHFWLRQ
7KHRQO\JRRGWKLQJWRVD\DERXWWKHWUDQVFULSWLRQSURFHVVLVWKDWLWDOORZVXVWRJHW
WRNQRZRXUGDWDWKRURXJKO\RWKHUZLVHLWLVXVXDOO\DIDUWRRORQJDQGOHVVWKDQ
HQMR\DEOHSURFHVV WRVD\WKHOHDVW ,WLVZRUWKEX\LQJRUERUURZLQJDWUDQVFULELQJ
PDFKLQHWKDWKDVDIRRWSHGDOVSHHGFRQWUROVDQGHDUSKRQHVEHFDXVHLWFDQVSHHGXS
RXUZRUNFRQVLGHUDEO\DOWHUQDWLYHO\LWPLJKWEHZRUWKGRLQJWKHRULJLQDOUHFRUGLQJ
GLJLWDOO\EHFDXVHWKHUHLVKLJKTXDOLW\VRIWZDUHDYDLODEOHWRIDFLOLWDWHWKHWUDQVFULSWLRQ
SURFHVVRIGLJLWDOVRXQGILOHV8QIRUWXQDWHO\PLQGQXPELQJWHGLRXVQHVVLVQRWWKH
JUHDWHVWSUREOHPZLWKWUDQVFULSWLRQV²WKHUHDOFRQFHUQLVWKHORVVRILQIRUPDWLRQ
WKURXJKWKHSURFHVV1RPDWWHUKRZDFFXUDWHDQGHODERUDWHDWUDQVFULSWLVLWZLOOQHYHU
FDSWXUHWKHUHDOLW\RIWKHUHFRUGHGVLWXDWLRQ7UDQVFULSWLRQVDUHLQ0LOOHUDQG&UDEWUHH¶V
ZRUGVµIUR]HQLQWHUSUHWLYHFRQVWUXFWV¶
7KHPRVWREYLRXVDUHDRIORVVLVWKHQRQYHUEDODVSHFWVRIWKHRULJLQDO
FRPPXQLFDWLRQVLWXDWLRQVXFKDVWKHERG\ODQJXDJHRIWKHUHVSRQGHQWV IRUH[DPSOH
IDFLDOH[SUHVVLRQVJHVWXUHVRUH\HPRYHPHQW ²JLYHQWKDWµDFWLRQVVSHDNORXGHUWKDQ
ZRUGV¶ZULWWHQWUDQVFULSWLRQVDUHVHULRXVO\LPSRYHULVKHGLQWKLVUHVSHFW,WLVDOVR
SUREOHPDWLFWRGHFLGHZKDWWRGRZLWKVXSUDVHJPHQWDOVVXFKDVVWUHVVDQGLQWRQDWLRQ
DQGSDUDOLQJXLVWLFIDFWRUVVXFKDVDFRXVWLFVRXQGV IRUH[DPSOHJUXQWV RUQRQYRFDO
QRLVHV IRUH[DPSOHKLVVHV $WKLUGUHFXUULQJLVVXHLVKRZWRUHSUHVHQWVSHHFKWKDWLV
µLPSHUIHFW¶LQVRPHZD\VIRUH[DPSOHIDOVHVWDUWVZRUGUHSHWLWLRQVWDPPHULQJRU
ODQJXDJHPLVWDNHV
)RUOLQJXLVWLFLQYHVWLJDWLRQVVXFKDVGLVFRXUVHRUFRQYHUVDWLRQDQDO\VLVLWLV
REYLRXVO\LPSRUWDQWWRLQFOXGHDVPDQ\GHWDLOVDVSRVVLEOHLQWKHWUDQVFULSWVDQG
UHFHQWO\WKHXVHRIYLGHRUHFRUGLQJVKDVEHHQLQFUHDVLQJO\DGYRFDWHGWRVXSSOHPHQW
ZULWWHQFRUSRUD 6HHIRUH[DPSOH$GROSKV 2QWKHRWKHUKDQGLIZHDUH
LQWHUHVWHGLQWKHFRQWHQWUDWKHUWKDQWKHIRUPRIWKHYHUEDOGDWDZHFDQGHFLGHWRHGLW
RXWDQ\OLQJXLVWLFVXUIDFHSKHQRPHQDEXWZHDUHQRWDGYLVHGWRPDNHDQ\FRQWHQW
VHOHFWLRQHGLWLQJDWWKLVVWDJHEHFDXVHZHVLPSO\FDQQRWNQRZZKDWPLJKWWXUQRXWWREH
LPSRUWDQW 6HH6HFWLRQ 7KHUHVSRQGHQW¶VHPRWLRQDORYHUWRQHVFDQEHFUXFLDO
ZLWKUHJDUGWRWKHUHDOPHDQLQJRIWKHPHVVDJHDQGIRUWKLVUHDVRQUHOHYDQWLQWRQDWLRQDO
FRQWRXUVDQGHPSKDVHVDUHRIWHQLQGLFDWHGLQWKHVFULSWE\XVLQJSXQFWXDWLRQPDUNV
XSSHUFDVHOHWWHULQJXQGHUOLQLQJRUHPEROGHQLQJ
7KHGHVFULSWLRQVRIDUPD\KDYHJLYHQWKHLPSUHVVLRQWKDWWUDQVFULSWLRQLVPHUHO\D
WHFKQLFDOPDWWHU+RZHYHUWKHLVVXHLVPRUHFRPSOH[WKDQWKDWEHFDXVHHYHU\

WUDQVFULSWLRQFRQYHQWLRQZHXVHZLOODIIHFWWKHLQWHUSUHWDWLRQRIWKHGDWD )RU
GLVFXVVLRQVVHHIRUH[DPSOH'XIILQSUHVV5REHUWV ,WLVZHOONQRZQWRDSSOLHG
OLQJXLVWVWKDWDQDUURZWUDQVFULSWLRQRIRUDOGLVFRXUVHRIWHQDSSHDUVLQFRKHUHQWDQG
XQVRSKLVWLFDWHG+RZHYHUWKHSUREOHPLVPRUHJHQHUDODQGFRQFHUQVWKHIDFWWKDW
VSRNHQDQGZULWWHQODQJXDJHDUHVWUXFWXUHGGLIIHUHQWO\WKHUHIRUHLQRUGHUWRWU\DQG
FUHDWHWKHµIHHO¶RIWKHRUDOFRPPXQLFDWLRQLQZULWLQJZHQHHGWRDSSO\FHUWDLQZULWLQJ
VWUDWHJLHV IRUH[DPSOHSROLVKLQJWKHWH[WXVLQJYDULHGSXQFWXDWLRQPDUNVGLYLGLQJWKH
VSHHFKLQWRVHQWHQFHV WKDWZLOOIDFLOLWDWHWKHLQWHQGHGNLQGRIUHDGLQJ
7KXVZHVKRXOGDOZD\VEHPLQGIXORIWKHIDFWWKDWXVLQJGLIIHUHQWWUDQVFULELQJ
FRQYHQWLRQVWRSURFHVVWKHVDPHUHFRUGLQJFDQSURGXFHYHU\GLIIHUHQWHIIHFWVLQWKH
UHDGHU7KLVLVZK\/DSDGDW DORQJZLWKPDQ\RWKHUVHPSKDVL]HVWKDWD
WUDQVFULSWLVDQLQWHUSUHWLYHµUHWHOOLQJ¶RIWKHRULJLQDOFRPPXQLFDWLRQ2UDV5REHUWV
SXWVLWµWUDQVFULEHUVEULQJWKHLURZQODQJXDJHLGHRORJ\WRWKHWDVN,QRWKHU
ZRUGVDOOWUDQVFULSWLRQLVUHSUHVHQWDWLRQDQGWKHUHLVQRQDWXUDORUREMHFWLYHZD\LQ
ZKLFKWDONFDQEHZULWWHQ¶5REHUWVWKHQJRHVRQWRKLJKOLJKWWKHµUHWHOOLQJ¶DVSHFWRI
WUDQVFULSWLRQE\HPSKDVL]LQJWKDWHYHU\GHFLVLRQDERXWKRZWRWUDQVFULEHWHOOVDVWRU\
WKHTXHVWLRQVKHDVNVLVZKRVHVWRU\DQGIRUZKDWSXUSRVH"7KHVWDQGDUGWUDQVFULSWLRQ
SUDFWLFHLQDSSOLHGOLQJXLVWLFUHVHDUFKRQ6/$KDVEHHQWRWU\DQGSURYLGHDQHODERUDWH
UHSUHVHQWDWLRQRIYDULRXVLQWHUODQJXDJHFKDUDFWHULVWLFVLQWKHUHVSRQGHQWV¶VSHHFKEXW
WKLVKDVRIWHQEHHQDWWKHH[SHQVHRIIRUHJURXQGLQJWKHUHVSRQGHQWV¶PHVVDJHDERXW
WKHLULGHQWLWLHV
$IWHUDOOWKHFRQVLGHUDWLRQVSUHVHQWHGDERYHOHWPHDGGUHVVWZRSUDFWLFDOTXHVWLRQV
WKDWHYHU\TXDOLWDWLYHUHVHDUFKHUPXVWIDFHDWRQHSRLQW
:KLFKWUDQVFULSWLRQFRQYHQWLRQVKDOOZHXVH"
8QIRUWXQDWHO\WKHUHLVQRµSHUIHFW¶WUDQVFULSWLRQFRQYHQWLRQWKDWZHFRXOGDGRSW
DXWRPDWLFDOO\7KHUHDUHVHYHUDOZLGHO\XVHGWUDQVFULSWLRQVFKHPHVDURXQGDQG,
ZRXOGVXJJHVWIROORZLQJDSULQFLSOHGµSLFNDQGPL[¶SURFHGXUHWRVHOHFWLGHDVIURP
WKHVH )RUGHWDLOHGGHVFULSWLRQVRIVRPHFRQYHQWLRQVVHH$SSHQGL[%LQ6FKLIIULQ
±DQG&KDSWHULQ:UD\HWDO± ,IWKHUHLVDJRRG
HQRXJKUHDVRQLWPD\DOVREHDSSURSULDWHWRLQYHQWLQGLYLGXDOL]HGWUDQVFULSWLRQ
UXOHVDQGIRUPDWVWKDWILWRXUUHVHDUFKSXUSRVH /DSDGDW 7KXV,IXOO\DJUHH
ZLWK5REHUWV ZKHQVKHVXJJHVWVWKDWWUDQVFULEHUVKDYHWRXVHRUGHYHORSD
WUDQVFULSWLRQV\VWHPWKDWFDQEHVWUHSUHVHQWWKHLQWHUDFWLRQVWKH\KDYHUHFRUGHG,Q
GRLQJVRZHQHHGWRPDQDJHWKHWHQVLRQEHWZHHQDFFXUDF\UHDGDELOLW\DQGWKH
µSROLWLFVRIUHSUHVHQWDWLRQ¶5REHUWVPDNHVFHUWDLQSUDFWLFDOUHFRPPHQGDWLRQVWR
WKDWHIIHFW
x 8VHVWDQGDUGRUWKRJUDSK\LISRVVLEOHHYHQZKHQWUDQVFULELQJDQRQVWDQGDUG
YDULHW\WRDYRLGVWLJPDWLVDWLRQDQGWRHYRNHWKHQDWXUDOQHVVDQGUHDGDELOLW\
RIWKHGLVFRXUVH,IDSSURSULDWHGLVFXVVZLWKWKHLQIRUPDQWVKRZWKH\ZLVK
WKHIHDWXUHVRIWKHLUVSHHFKWREHUHSUHVHQWHG

x 7U\WRILQGZD\VRIFRQWH[WXDOL]LQJHYRNLQJWKHVSHDNHUV¶YRLFHV
x &RQVLGHUXVLQJDOD\HUHGDSSURDFKWRWUDQVFULSWLRQRIIHULQJGLIIHUHQW
YHUVLRQV IRUH[DPSOHPRUHHWKQRJUDSKLFDQGPRUHILQHJUDLQHGV\VWHPV
WRJLYHGLIIHUHQWUHDGLQJV
,IWUDQVFULSWLRQLVDVLQWHUSUHWLYHDVDUJXHGDERYHGRHVWKLVPHDQWKDWWKHUHVHDUFKHU
KLPKHUVHOIPXVWWUDQVFULEHUDWKHUWKDQKLUHDUHVHDUFKDVVLVWDQW"
5HDOLVWLFDOO\EHFDXVHRIWKHGHPDQGLQJDQGWLPHFRQVXPLQJQDWXUHRIWKHSURFHVV
ZHRIWHQQHHGKHOSLQGRLQJWKHWUDQVFULSWLRQV+RZHYHUHYHQLIZHDUHDVVLVWHGE\
DWUDQVFULEHUZHVKRXOGVWD\DVFORVHWRWKHGDWDDVSRVVLEOH7KLVZRXOGLQYROYH
SUHSDULQJGHWDLOHGJXLGHOLQHV µWUDQVFULSWLRQSURWRFROV¶ KDYLQJUHJXODUGHEULHILQJ
GLVFXVVLRQVDQGFKHFNLQJDWOHDVWVRPHRIWKHWUDQVFULSWV7KHWUDQVFULSWLRQ
SURWRFROVKRXOGHPSKDVL]HWKHQHHGIRUFRQILGHQWLDOLW\DVZHOODVVSHFLI\WKHH[WHQW
WRZKLFKWKHWUDQVFULEHULVWRµWLG\XS¶WKHWH[WDQGZKDWVSHFLDOV\PEROVWRXVH IRU
H[DPSOHIRULOOHJLEOHSDUWVSDXVHVHPSKDVLV
7DSHDQDO\VLVDQGSDUWLDOWUDQVFULSWLRQ
,EHOLHYHWKDWLQRUGHUWRFRQGXFWDIXOO\IOHGJHGTXDOLWDWLYHLQYHVWLJDWLRQZHQHHGWR
KDYHIXOOWUDQVFULSWVRIWKHUHFRUGLQJVWKDWZHZDQWWRDQDO\VH+RZHYHULQFHUWDLQ
W\SHVRIPL[HGPHWKRGVUHVHDUFK SDUWLFXODUO\IRUTXDOo48$1DQG48$1oTXDO
VHH6HFWLRQ ZKHUHWKHTXDOLWDWLYHFRPSRQHQWLVRIVHFRQGDU\LPSRUWDQFHDQGLV
PDLQO\LQWHQGHGWRSURYLGHDGGLWLRQDOLOOXVWUDWLRQRUFODULILFDWLRQLWPD\QRWEH
HVVHQWLDOWRWUDQVFULEHHYHU\LQWHUYLHZDQGLQVWHDGZHFDQFDUU\RXWDWDSHDQDO\VLV
7KLVVLPSO\PHDQVWDNLQJQRWHVZKLOHOLVWHQLQJWRWKHUHFRUGLQJVSRVVLEO\PDUNLQJ
SDUWVRIWKHGDWD IRUH[DPSOHWKHFRXQWHUSRVLWLRQRIWKHFDVVHWWHUHFRUGHURULWVGLJLWDO
HTXLYDOHQW WKDWZDUUDQWPRUHHODERUDWHVXEVHTXHQWDQDO\VLV5HVHDUFKPHWKRGRORJ\LV
RIWHQDEDODQFLQJDFWEHWZHHQJRDOVDQGUHVRXUFHVDQGLQFHUWDLQVLWXDWLRQVWDSHDQDO\VLV
FDQEHMXVWLILHGDVDJRRGFRPSURPLVHEHFDXVHWKHSDUWLFXODUGDWDPD\QRWGHVHUYHWKH
KXJHLQYHVWPHQWRIDIXOOWUDQVFULSW
,QWKHVSLULWRIWKHDERYHDQRWKHUSRVVLEOHFRPSURPLVHLVWRSUHSDUHDSDUWLDO
WUDQVFULSWLRQRIWKHVHFWLRQVWKDWVHHPLPSRUWDQW7KHREYLRXVGUDZEDFNRIWKLVPHWKRG
LVWKDWZHQHHGWRPDNHLPSRUWDQWGHFLVLRQVDVWRZKDWWRLQFOXGHDQGH[FOXGHUHODWLYHO\
HDUO\LQWKHDQDO\WLFDOSURMHFWEXWWKHSURFHGXUHOHDYHVRSHQWKHRSWLRQWRH[SDQGWKH
WUDQVFULSWLRQODWHU2QHZD\RILPSURYLQJWKLVPHWKRGLVE\KDYLQJQRWHVRIWKHNH\
SRLQWVRIWKHZKROHUHFRUGLQJLQGLFDWLQJZKLFKSDUWVKDYHQRWEHHQWUDQVFULEHG
&RPSXWHUDVVLVWHGWUDQVFULSWLRQ
$UNVH\DQG.QLJKW PHQWLRQDQLQWHUHVWLQJWHFKQLFDOLQQRYDWLRQ²FRPSXWHU
DVVLVWHGWUDQVFULSWLRQ²WKDWVRPHUHVHDUFKHUVPLJKWILQGXVHIXO5HFHQWVRIWZDUH
GHYHORSPHQWVKDYHSURGXFHGLQFUHDVLQJO\DFFXUDWHYRLFHUHFRJQLWLRQSURJUDPVWKDW
HQDEOHWKHUHVHDUFKHUWRµGLFWDWH¶WH[WVWRWKHFRPSXWHUGLUHFWO\LQWRWKHZRUGSURFHVVRU
$OOZHQHHGWRGRLVOLVWHQWRWKHUHFRUGLQJWKURXJKKHDGSKRQHVDQGWKHQUHSHDWZKDW

ZHKHDUELWE\ELW8QIRUWXQDWHO\HYHQZLWKH[SHULHQFHGµGLFWDWRUV¶WKHUHZLOOEH
PLVWDNHVDQGFRUUHFWLQJWKHVHPD\VWLOOWDNHVRPHWLPHDOWKRXJKOHVVWKDQWUDQVFULELQJ
WKHWDSHVRXUVHOYHV$UNVH\DQG.QLJKWEHOLHYHWKDWZLWKWKHLQFUHDVLQJVRSKLVWLFDWLRQ
RIWKHVHSURJUDPVFRPSXWHUDVVLVWHGWUDQVFULSWLRQLVOLNHO\WREHFRPHDSUHIHUUHGZD\
RIWUDQVFULELQJUHFRUGHGGDWD,QGHHG'XII LQSUHVV UHSRUWVWKDWVHYHUDORIKHUVWXGHQWV
KDYHVXFFHVVIXOO\DSSOLHGWKLVPHWKRG
7UDQVFULELQJYLGHRWDSHV
7KHDGYDQWDJHRIYLGHRUHFRUGLQJLVWKDWLWFDQFDSWXUHLQIRUPDWLRQWKDWZRXOGJHWORVW
E\DXGLRUHFRUGLQJ7KLVLVWUXHEXWKRZGRZHDFWXDOO\WUDQVFULEHWKLVDGGLWLRQDO
LQIRUPDWLRQ":HGRQRWKDYHWKHHTXLYDOHQWRIµZULWLQJ¶ZKHQLWFRPHVWRGHVFULELQJ
ERG\ODQJXDJHDQGFRQWH[WXDOLQIRUPDWLRQ)XUWKHUPRUHLIZULWLQJFDQFDUU\WRRPDQ\
VXEMHFWLYHHOHPHQWV DVZHKDYHDUJXHGDERYH UHSUHVHQWLQJDQGLQWHUSUHWLQJERG\
ODQJXDJHLVERXQGWREHHYHQPRUHVHOHFWLYHDQGRSHQWRGLVSXWH$FRPPRQSUDFWLFHLQ
YLGHRWUDQVFULSWLRQVLVWRLQFOXGHDFROXPQRIUXQQLQJFRPPHQWDU\QH[WWRWKHYHUEDO
WUDQVFULSWEXWWKLVZLOOPXOWLSO\WKHFRVWVRIWKHWUDQVFULSWLRQSURFHVV$FFRUGLQJO\
$UNVH\DQG.QLJKW RQO\UHFRPPHQGWKHWUDQVFULSWLRQRIYLGHRWDSHVLQ
H[FHSWLRQDOFDVHVDQGZLWKDVSHFLILFIRFXV7KLVRIFRXUVHUDLVHVWKHTXHVWLRQRIGRLQJ
YLGHRUHFRUGLQJVDWDOOEHFDXVHLWLVP\LPSUHVVLRQWKDWXQWUDQVFULEHGWDSHVZLOORQO\
UDUHO\EHUHYLVLWHGDWODWHUVWDJHVRIDUHVHDUFKSURMHFW
3UHFRGLQJDQGFRGLQJ
0RVWUHVHDUFKPHWKRGVWH[WVZRXOGFRQILUPWKDWUHJDUGOHVVRIWKHVSHFLILFPHWKRGRORJ\
IROORZHGTXDOLWDWLYHGDWDDQDO\VLVLQYDULDEO\VWDUWVZLWKFRGLQJ<HWWKLVVWDWHPHQWLV
RQO\SDUWLDOO\WUXHEHFDXVHXVXDOO\DFRQVLGHUDEOHDPRXQWRIDQDO\VLVKDVDOUHDG\WDNHQ
SODFHZKHQZHEHJLQWKHDFWXDOFRGLQJSURFHVV6WDNH GHILQHVDQDO\VLVDVD
µPDWWHURIJLYLQJPHDQLQJWRILUVWLPSUHVVLRQVDVZHOODVWRILQDOFRPSLODWLRQV¶DQG
PDNLQJVHQVHRIRXUILUVWLPSUHVVLRQVLVDFUXFLDOSUHFRGLQJPRYH,WLQYROYHVUHDGLQJ
DQGUHUHDGLQJWKHWUDQVFULSWVUHIOHFWLQJRQWKHPDQGQRWLQJGRZQRXUWKRXJKWVLQ
MRXUQDOHQWULHV 6HFWLRQ DQGPHPRV 6HFWLRQ 7KHVHSUHFRGLQJUHIOHFWLRQV
VKDSHRXUWKLQNLQJDERXWWKHGDWDDQGLQIOXHQFHWKHZD\ZHZLOOJRDERXWFRGLQJLW
5LFKDUGV VXPPDUL]HVWKLVTXHVWLRQDVIROORZV
7KHUHLVQRDOWHUQDWLYHWRUHDGLQJDQGUHIOHFWLQJRQHDFKGDWDUHFRUGDQGWKH
VRRQHUWKHEHWWHU7KLVLVSXUSRVLYHUHDGLQJ$LPWRTXHVWLRQWKHUHFRUGDQGDGG
WRLWWRFRPPHQWRQLWDQGWRORRNIRULGHDVWKDWOHDG\RXXSIURPWKHSDUWLFXODU
WH[WWRWKHPHVLQWKHSURMHFW
7KXVWKHSURFHVVRIµPHHWLQJWKHGDWDPHDQLQJIXOO\¶GXULQJWKHSUHFRGLQJSKDVHLV
DQLQGLVSHQVDEOHSUHSDUDWRU\PRYH$VD3K'VWXGHQWVWDWHG
,I,KDGVWDUWHGZLWKFRGLQJVWUDLJKWDZD\,WKLQN,ZRXOGKDYHMXVWJRWORVWLQWKH
GDWDDQGSHUKDSVZRXOGKDYHHQGHGXSZLWKWKRXVDQGVRIFRGHVWKDW,ZRXOGKDYH

QHYHUEHHQDEOHWRLQWHJUDWHLQWRDPHDQLQJIXOIUDPHZRUN 0DJJLH.XEDQ\LRYD
SHUVRQDOFRPPXQLFDWLRQ
$WRQHSRLQWKRZHYHUSUHFRGLQJGHOLEHUDWLRQQHHGVWRJLYHZD\WRDPRUHIRUPDO
DQGVWUXFWXUHGFRGLQJSURFHVV7KLVFDQEHFRPSDUHGWRSDFNDJLQJYHJHWDEOHVLQD
VXSHUPDUNHWWKDWLVWXUQLQJWKHµPHVV\¶IDUPSURGXFWVRILUUHJXODUVKDSHVW\SHVDQG
VL]HVLQWRGLVWLQFWVRUWHGDQGQHDWO\ZUDSSHGXSSDUFHOVWKDWFDQEHSODFHGRQVKHOYHV
RUVWRUHGSLOHGXS$IWHUDOODµFRGH¶LVVLPSO\DODEHODWWDFKHGWRDFKXQNRIWH[W
LQWHQGHGWRPDNHWKHSDUWLFXODUSLHFHRILQIRUPDWLRQPDQDJHDEOHDQGPDOOHDEOH2I
FRXUVHMXVWOLNHODEHOVFRGHVFDQEHXVHGIRUPDQ\SXUSRVHVIURPSXUHO\GHVFULSWLYH
IRUH[DPSOHµ1RWWLQJKDP¶RQDSKRWR WRPRUHDEVWUDFW IRUH[DPSOHµ7KH%LJ
0RPHQW¶RQDZHGGLQJSKRWR 7KLVLVZK\DV0RUVHDQG5LFKDUGV SRLQWRXW
GLIIHUHQWUHVHDUFKHUVPHDQVHYHUDOGLIIHUHQWWKLQJVZKHQWKH\XVHWKHWHUPµFRGLQJ¶<HW
DOOWKHTXDOLWDWLYHFRGLQJWHFKQLTXHVDUHDLPHGDWUHGXFLQJRUVLPSOLI\LQJWKHGDWDZKLOH
KLJKOLJKWLQJVSHFLDOIHDWXUHVRIFHUWDLQGDWDVHJPHQWVLQRUGHUWROLQNWKHPWREURDGHU
WRSLFVRUFRQFHSWV
,QDFWXDOSUDFWLFHFRGLQJLQYROYHVKLJKOLJKWLQJH[WUDFWVRIWKHWUDQVFULEHGGDWDDQG
ODEHOOLQJWKHVHLQDZD\WKDWWKH\FDQEHHDVLO\LGHQWLILHGUHWULHYHGRUJURXSHG
7UDGLWLRQDOO\ZHXVHGWRXVHKDUGFRS\SULQWRXWVRIWKHWH[WVDQGPDUNHGWKHVHZLWK
PXOWLFRORXUHGKLJKOLJKWHUVEXWQRZDGD\VFRGLQJLVLQFUHDVLQJO\GRQHHOHFWURQLFDOO\
6HH6HFWLRQ :KLFKHYHUWHFKQLTXHZHDSSO\FRGLQJVSHFLILHVODQJXDJHFKXQNV
WKDWFDQUDQJHLQOHQJWKIURPDSKUDVHWRVHYHUDOSDUDJUDSKV
:HVDZLQ6HFWLRQWKDWTXDOLWDWLYHDQDO\VLVLVDQLWHUDWLYHSURFHVVDQGWKLVLV
SDUWLFXODUO\REYLRXVLQWKHSURFHVVRIFRGLQJ5HVHDUFKHUVXVXDOO\FRGHDQGUHFRGHD
WH[WVHYHUDOWLPHVZLWKWKHLQLWLDOXVXDOO\GHVFULSWLYHDQGORZLQIHUHQFHFRGHV
JUDGXDOO\EHLQJUHSODFHGRUVXSSOHPHQWHGE\KLJKHURUGHUµSDWWHUQFRGHV¶ IRUH[DPSOH
DQH[WUDFWRULJLQDOO\FRGHGµORZVDODU\¶PD\EHUHFRGHGODWHUDVµVRXUFHRI
GHPRWLYDWLRQ¶ $VDUHVXOWRIUHYLVLWLQJWKHGDWDDQXPEHURIWLPHVVRPHVDOLHQW
FRQWHQWFDWHJRULHVHPHUJHOLQNHGWRYDULRXVGDWDVHJPHQWV)XUWKHUFRGLQJWKHQ
DJJUHJDWHVWKHVHVHJPHQWVVRWKDWZHFDQZRUNZLWKWKHPWRJHWKHUµJDLQLQJDQHZFXW
RQWKHGDWD¶ 5LFKDUGV
%HIRUHGHVFULELQJWKHFRGLQJSURFHVVLQPRUHGHWDLOOHWPHSRLQWRXWWKDWJURXQGHG
WKHRU\ZKLFKLVLQWURGXFHGLQ6HFWLRQFRQWDLQVDQHODERUDWHWKUHHWLHUFRGLQJ
FRPSRQHQWZKLFKKDVDIIHFWHGTXDOLWDWLYHGDWDDQDO\VLVLQJHQHUDO7KXVWKHUHDUH
PDQ\VLPLODULWLHVEHWZHHQWKHJHQHULFSURFHGXUHVGHVFULEHGEHORZDQGWKHVSHFLILF
FRGLQJPHWKRGRORJ\VXJJHVWHGE\JURXQGHGWKHRU\
,QLWLDOFRGLQJ
+RZVKDOOZHEHJLQFRGLQJ"+HUHLVDW\SLFDOUHFLSHILUVWFKRRVHDWH[WDQGUHDGLW
WKURXJKVHYHUDOWLPHVWRREWDLQDJHQHUDOVHQVHRIWKHGDWD7KHQWDNHDKLJKOLJKWHU LI
\RXDUHZRUNLQJZLWKKDUGFRS\ DQGVWDUWUHDGLQJWKHWH[WIURPWKHEHJLQQLQJ:KHQ
\RXFRPHWRDSDVVDJHWKDWLVUHOHYDQWWR\RXUWRSLFVLPSO\KLJKOLJKWLWDQGDGGDQ

LQIRUPDWLYHODEHORQWKHPDUJLQ,WIROORZVIURPWKHVSLULWRITXDOLWDWLYHUHVHDUFKWKDWDW
WKLVVWDJH\RXVKRXOGKLJKOLJKWDQ\LQWHUHVWLQJORRNLQJSDVVDJHHYHQLILWLVQRWGLUHFWO\
OLQNHGWR\RXULPPHGLDWHIRFXVDUHD7KLVLVKRZQHZLQVLJKWVFDQHPHUJH
5HJDUGLQJWKHFKDUDFWHULVWLFVRIWKHODEHOVXVHGIRUFRGLQJFODULW\LVWKHPRVW
LPSRUWDQWIHDWXUHWRDLPIRUEHFDXVHLWGHIHDWVWKHZKROHSXUSRVHRIFRGLQJLIWKH
PHDQLQJRIDFRGHLVQRWLPPHGLDWHO\WUDQVSDUHQW6RPHUHVHDUFKHUVIDYRXUXVLQJVRPH
NH\ZRUGVIURPWKHDFWXDOSDVVDJHWRPDNHWKHSUHOLPLQDU\FRGHVPRUHDXWKHQWLF
ZKLFKLVFDOOHGµLQYLYR¶FRGLQJLQJURXQGHGWKHRU\ ,WLVYLWDOWKDWZHSURGXFHH[SOLFLW
GHVFULSWLRQVRIWKHFRGHV²)LJXUHSUHVHQWVDGLDORJXHER[RIWKHGDWDDQDO\WLFDO
VRIWZDUH19LYR VHH6HFWLRQ WKDWDOORZVIRUWKHSUHFLVHUHFRUGLQJRIFRGH
SURSHUWLHV$OVR/\QFK UHFRPPHQGVWKDWUHVHDUFKHUVWU\WRDUUDQJHDQH[WHUQDO
FRGHFKHFNWKDWLVDVNVRPHRQHWRORRNDWDSRUWLRQRIWKHGDWDDQGUHFRGHLWXVLQJWKH
OLVWRIFRGHVDOUHDG\GHYHORSHGDVZHOODVSRVVLEO\LQWURGXFLQJQHZRQHV%RWKWKH
VLPLODULWLHVDQGGLVFUHSDQFLHVFDQEHHQOLJKWHQLQJOHDGLQJWRDUHYLVLRQRIWKHRULJLQDO
FRGHV

)LJXUH6DPSOH19LYRGLDORJXHER[IRUUHFRUGLQJFRGHSURSHUWLHV
6HFRQGOHYHOFRGLQJ
(YHU\TXDOLWDWLYHDQDO\WLFDOPHWKRGFRQWDLQVDVHFRQGOHYHOFRGLQJSURFHVVEHFDXVHLQ
PRVWLQYHVWLJDWLRQVZHZDQWWRJREH\RQGDPHUHGHVFULSWLYHODEHOOLQJRIWKHUHOHYDQW
GDWDVHJPHQWV%HIRUHORQJZHZLOOKDYHLGHDVWKDWJREH\RQGWKHVSHFLILFDFFRXQWZH
UHDGDQGZHDUHDOVROLNHO\WRQRWLFHSDWWHUQVWKDWHPHUJHDFURVVWKHLQGLYLGXDO
DFFRXQWV )RUPRUHGHWDLOVRQWKHVHSDWWHUQVVHHµD[LDOFRGLQJ¶LQ6HFWLRQ
EHORZ 6HFRQGOHYHOFRGLQJLQWHQGVWRFDSWXUHWKHVHPRUHDEVWUDFWFRPPRQDOLWLHV
2QHZD\RIODXQFKLQJVHFRQGOHYHOFRGLQJLVWRJRWKURXJKVHYHUDOUHVSRQGHQWV¶
DFFRXQWVDQGOLVWDOOWKHFRGHVWKDWZHKDYHLGHQWLILHGLQWKHP7KHUHZLOOLQHYLWDEO\EH
VRPHVLPLODURUFORVHO\UHODWHGFDWHJRULHVZKLFKFDQEHFOXVWHUHGWRJHWKHUXQGHUD

EURDGHUODEHO$WWKLVSRLQWZHQHHGWRORRNDWDOOWKHVSHFLILFH[WUDFWVWKDWDUHOLQNHGWR
WKHQHZO\IRUPHGEURDGHUFDWHJRU\WRGHFLGHZKHWKHUWKHQHZODEHODSSOLHVWRDOORI
WKHPRUZKHWKHUVRPHPD\QHHGWREHUHFRGHG,IWKHPDMRULW\RIWKHH[WUDFWVILWWKH
QHZV\VWHPWKLVFDQEHVHHQDVDVLJQRIWKHYDOLGLW\RIWKHFRGH2QFHZHKDYH
ILQDOL]HGWKHUHYLVHGOLVWRIFRGHVZHPD\ZDQWWRJREDFNWRWKHRULJLQDOWUDQVFULSWV
DQGUHFRGHWKHPDFFRUGLQJWRWKHQHZFDWHJRULHV,QVRPHVWXGLHVWKLVSURFHVVLV
LWHUDWHGPRUHWKDQRQFH
$QRWKHUXVHIXOSURFHVVLQVHFRQGOHYHOFRGLQJLVWRSURGXFHDKLHUDUFK\RIFRGHVLQ
WKHIRUPRIDWUHHGLDJUDP)LJXUHSUHVHQWVSDUWRIVXFKDµWUHH¶SURGXFHGE\WKH
GDWDDQDO\VLVVRIWZDUH19LYRGHVFULELQJWKHKLHUDUFK\RIFRGHVUHODWHGWRµLQWULQVLF
PRWLYDWLRQ¶3OD\LQJDURXQGZLWKVXFKFRGHVWUXFWXUHVLVLQLWVHOIDQHIIHFWLYHDQDO\WLFDO
PRYHEHFDXVHLWKHOSVWRFODULI\KRZWKHFDWHJRULHVDUHUHODWHGWRHDFKRWKHU

)LJXUH6DPSOH19LYRRXWSXWRIFDWHJRU\FRGHVRUJDQL]HGLQWRDµWUHH¶
8VLQJDWHPSODWHRIFRGHV
&UDEWUHHDQG0LOOHU GHVFULEHDFRGLQJPHWKRGWKDWWKH\FDOOHGWKHµWHPSODWH
RUJDQL]LQJVW\OH¶7KLVLVDQLQWHUHVWLQJYDULDWLRQRQWKHVWDQGDUGFRGLQJSURFHGXUHVLQ
WKDWLWGRHVQRWHPSKDVL]HWKHHPHUJHQWQDWXUHRIWKHFRGHVEXWWRWKHFRQWUDU\VWDUWV
RXWZLWKDWHPSODWHRIFRGHV7KXVDVLQTXDQWLWDWLYHUHVHDUFKWKHILUVWVWHSRIGDWD
DQDO\VLVLQYROYHVSUHSDULQJDWHPSODWHRUFRGHPDQXDODQGWKHWUDQVFULEHGWH[WVDUH
FRGHGXVLQJWKLVSUHGHWHUPLQHGWHPSODWH2EYLRXVO\WKHWHPSODWHPHWKRGFDQRQO\EH
XVHGLIWKHUHLVVXIILFLHQWEDFNJURXQGLQIRUPDWLRQRQWKHWRSLFWREHDEOHWRGHILQHWKH
WHPSODWHFDWHJRULHVDOWKRXJKWKHDXWKRUVDOVRDVVHUWWKDWWKHWHPSODWHFDQEHSUHSDUHG
DVDUHVXOWRISUHOLPLQDU\VFDQQLQJRIWKHGDWD)XUWKHUPRUHWKHSURFHVVFDQEHPDGH
PRUHµTXDOLWDWLYH¶LIZHDOORZIRUUHYLVLRQRUILQHWXQLQJRIWKHWHPSODWHDWVRPHSRLQW
LQWKHDQDO\WLFDOSURFHVV
$OWKRXJKXVLQJSUHFRQFHLYHGFDWHJRULHVVHHPVWREHDWRGGVZLWKFHUWDLQTXDOLWDWLYH
SULQFLSOHVLQHIIHFWIHZLIDQ\UHVHDUFKHUVVWDUWGDWDDQDO\VLVZLWKQRLQLWLDOLGHDVDQG

ELDVHV VHH6HFWLRQRQWKHHPHUJHQWQRQHPHUJHQWGHEDWH DQGKDYLQJDVSHFLILF
WHPSODWHRIFRGHVKHOSVWRFDUU\RXWWKHLQLWLDOFRGLQJRIDODUJHYROXPHRIWH[WLQD
IRFXVHGDQGWLPHHIILFLHQWPDQQHUFUHDWLQJOLQNVEHWZHHQH[WUDFWVIURPGLIIHUHQW
DFFRXQWVHDUOLHULQWKHSURFHVV0LOHVDQG+XEHUPDQ DOVRXQGHUVFRUHWKH
XVHIXOQHVVRIH[SORULQJERWKKRZWRDUULYHDWDQDO\WLFFDWHJRULHVGHGXFWLYHO\ LH
EULQJLQJFRGHVWRWKHGDWD DQGJHWWLQJJUDGXDOO\WRWKHPLQGXFWLYHO\ LHILQGLQJWKHP
LQWKHGDWD :HVKRXOGQRWHWKRXJKWKDWWKLVLVDFRQWHQWLRXVLVVXHLQTXDOLWDWLYH
UHVHDUFK
,QWHUHVWLQJO\RQHDGYDQWDJHRIWKHWHPSODWHPHWKRGWKDW&UDEWUHHDQG0LOOHU
KLJKOLJKWLVWKDWLWFDQPDNHWKHDQDO\VLVPRUHDFFHSWDEOHIRUWKRVHVFHSWLFDORI
TXDOLWDWLYHUHVHDUFK$VWKH\FRQFOXGH
7KHLQWHUSUHWDWLRQRIDPRXQWDLQRILQIRUPDWLRQULFKSXUSRVHIXOO\VDPSOHG
TXDOLWDWLYHWH[WVFDQHDVLO\DSSHDULQVXUPRXQWDEOHDQGTXL[RWLFHVSHFLDOO\WR
UHVHDUFKHUVSURILFLHQWLQTXDQWLWDWLYHPHWKRGV7KLVLVUHDVRQHQRXJKWRSDXVH
DQGEULHIO\WUHPEOH+RZHYHUWKHWHPSODWHRUJDQL]LQJVW\OHGHVFULEHGLQWKLV
FKDSWHULVRQHVSHFLILFZD\IRUTXDQWLWDWLYHO\WUDLQHG«UHVHDUFKHUVWRWDNHWKH
ILUVWVWHSLQWRTXDOLWDWLYHDQDO\VLV S
*URZLQJLGHDVPHPRVYLJQHWWHVLQWHUYLHZSURILOHVDQG
IRUPVRIGDWDGLVSOD\
&RGLQJLVXQGRXEWHGO\DNH\SURFHVVLQTXDOLWDWLYHFRQWHQWDQDO\VLVEXWPRVW
UHVHDUFKHUVDJUHHWKDWLWVKRXOGEHDFFRPSDQLHGE\RWKHUHVVHQWLDODQDO\WLFDOWRROVWKDW
FDQKHOSWRµJURZWKHLGHDV¶DQGWRGHYHORSWKHPLQWRWKHILQDOPDLQWKHPH V RIWKH
VWXG\7KHVHWRROVLQFOXGHSUHSDULQJPHPRVYLJQHWWHVLQWHUYLHZSURILOHVDQGYDULRXV
IRUPVRIGDWDGLVSOD\DOORIZKLFKDUHVHFRQGDU\ LHGHULYHG GDWDWKDWWKHUHVHDUFKHU
SURGXFHVDVDPHDQVRIVWUXFWXUHGUHIOHFWLRQ
7KHPRVWLPSRUWDQWRIDOOWKHVHDQDO\WLFDOWRROVLVZULWLQJµPHPRV¶ RUµPHPRLQJ¶
5HVHDUFKHUVYDU\LQWKHH[WHQWWRZKLFKWKH\XVHPHPRLQJGXULQJGDWDDQDO\VLVEXWLWLV
QRWXQXVXDOWRKHDUFODLPVWKDWµ5HDODQDO\VLVLVLQWKHPHPRZULWLQJ¶7KHSURFHVVRI
FRGLQJDFWLYHO\HQJDJHVDQDO\VWVZLWKWKHGDWDDQGGXULQJWKLVSURFHVVWKH\DUHWRPDNH
QRWHVRIDOOWKHWKRXJKWVDQGLGHDVWKDWFRPHWRPLQG7KHVHQRWHVDUHXVXDOO\UHIHUUHG
WRDVµDQDO\WLFPHPRV¶²WKH\DUHLQYDOXDEOHLQIDFLOLWDWLQJVHFRQGOHYHOFRGLQJDQGDUH
DOVROLNHO\WRFRQWDLQWKHHPEU\RVRIVRPHRIWKHPDLQFRQFOXVLRQVWREHGUDZQIURP
WKHVWXG\7KXVPHPRVDUHLQHIIHFWH[SORUDWLRQVRILGHDVKXQFKHVDQGWKRXJKWVDERXW
WKHFRGHVRULQ/\QFK¶V ZRUGVWKH\DUHµZRUNLQJLGHDVZKLFKPD\RUPD\
QRWSDQRXWLQWKHIXOOQHVVRI\RXUDQDO\VLV¶7KH\FDQEHDVVKRUWDVDVHQWHQFHRUDV
ORQJDVVHYHUDOSDUDJUDSKVDQGWKH\EHFRPHSDUWRIWKHRYHUDOOGDWDVHW7KH\DUH
HVSHFLDOO\LPSRUWDQWLQWHDPSURMHFWVWRNHHSHDFKLQGLYLGXDOUHVHDUFKHULQWKHORRSRI
FROOHFWLYHWKLQNLQJ
)LJXUHSUHVHQWVDVDPSOHDQDO\WLFPHPR LQWKHIRUPRIDQ19LYRVFUHHQVKRW
WKDWGLVFXVVHVWKHGLIIHUHQFHEHWZHHQWZRUHODWHGFRGHV RUDV19LYRFDOOVWKHP

µQRGHV¶ 7KHUHVHDUFKHUZKRSURGXFHGWKHPHPRFRPPHQWVEHORZWKDWWKLVSDUWLFXODU
PHPRWULJJHUHGRIIDFKDLQRIWKRXJKWVWKDWHYHQWXDOO\OHGWRWKHFRQFHSWXDOL]DWLRQRI
RQHRIWKHPDLQWKHPHVRIKHUILQDOUHVHDUFKUHSRUW 0DJJLH.XEDQ\LRYDSHUVRQDO
FRPPXQLFDWLRQ
7KLVPHPRLQIDFWRSHQHGXSDZKROHQHZWKHPHDQGRQUHUHDGLQJWKHGDWD
UHFRUGVDQGP\HDUO\DQQRWDWLRQV,GLVFRYHUHGQXPHURXVFOXHVLQGLFDWLQJ
VXSHUILFLDOHQGRUVHPHQWRIWKHFRXUVHFRQWHQWE\WKHSDUWLFLSDQWV,EHJDQWRVHH
WKDWPDQ\RIWKHVHFOXHVSRLQWHGWRZKDW,ODWHUVDZDVDQLPSRUWDQWWKHPH
HPHUJLQJIURPWKHGDWDWKHUHVHDUFKSDUWLFLSDQWV¶GHVLUHWROLYHXSWRWKH
UHVHDUFKHU¶VH[SHFWDWLRQV7KLVWXUQHGRXWWREHDQLPSRUWDQWSRLQWHUWRZDUGVWKH
UHVHDUFKSDUWLFLSDQWV¶PRWLYDWLRQWRMRLQWKHUHVHDUFKSURMHFWZKLFKLQWXUQ
DSSHDUHGWRKDYHFUXFLDOLPSOLFDWLRQVIRUWHDFKHUFKDQJH

)LJXUH6DPSOH19LYRRXWSXWRIDQDQDO\WLFPHPR
,QDGGLWLRQWRZULWLQJDQDO\WLFPHPRVPDQ\TXDOLWDWLYHUHVHDUFKHUVSUHSDUH
µYLJQHWWHV¶ZKLFKDUHVKRUWQDUUDWLYHVWKDWSURYLGHIRFXVHGGHVFULSWLRQVRIHYHQWVRU
SDUWLFLSDQWH[SHULHQFHV8QOLNHPHPRVYLJQHWWHVDUHQRWDQDO\WLFDOEXWVWRU\OLNH
RIIHULQJDYLYLGGHVFULSWLRQRIVRPHWKLQJWKDWLVVHHQDVW\SLFDORUUHSUHVHQWDWLYH
$YHU\XVHIXOPHWKRGRIRSHQLQJXSLQWHUYLHZGDWDWRIXUWKHUDQDO\VLVDQG
LQWHUSUHWDWLRQKDVEHHQVXJJHVWHGE\6HLGPDQ +HSURSRVHGWKHFUDIWLQJRI
µLQWHUYLHZSURILOHV¶ZKLFKDUHPRUHVXEVWDQWLDOVXPPDULHVRISDUWLFLSDQWDFFRXQWVWKDQ
YLJQHWWHV$VSHFLDOIHDWXUHRIDQLQWHUYLHZSURILOHLVWKDWLWLVSULPDULO\DFRPSLODWLRQ
RIWKHLQWHUYLHZHH¶VRZQZRUGVXVLQJWKHILUVWSHUVRQYRLFHRIWKHSDUWLFLSDQWZLWK
RQO\PLQLPDOWUDQVLWLRQDODGGLWLRQVDQGFODULILFDWLRQVE\WKHUHVHDUFKHU ZKLFKVKRXOG
EHZHOOPDUNHGLQDSSHDUDQFH 7KHVHSURILOHVVHUYHWKHVDPHIXQFWLRQDV
µFRQWDFWGRFXPHQWVXPPDU\VKHHWV¶GHVFULEHGE\'XII LQSUHVV ZKLFKFRQWDLQVKRUW

VXPPDULHVRIDQREVHUYDWLRQLQWHUYLHZRUGRFXPHQWKLJKOLJKWLQJWKHPRVWVDOLHQW
SRLQWVDQGWKHPHV
6RIDUDOOWKHDQDO\WLFDOWRROVGHVFULEHGDVSRVVLEOHPHWKRGVIRUGHYHORSLQJWKH
UHVHDUFKHU¶VWKLQNLQJKDYHEHHQYHUEDOLQQDWXUH+RZHYHUSUHVHQWLQJDQGVXPPDUL]LQJ
GDWDYLVXDOO\LVDOVRSRVVLEOHDQGLQGHHG0LOHVDQG+XEHUPDQ GHYRWHDJUHDW
GHDORIWKHLUVXPPDU\RITXDOLWDWLYHGDWDDQDO\VLVWRGHVFULELQJPHWKRGVRIµGDWD
GLVSOD\¶GHILQLQJDµGLVSOD\¶DVµDQRUJDQL]HGFRPSUHVVHGDVVHPEO\RILQIRUPDWLRQ
WKDWSHUPLWVFRQFOXVLRQGUDZLQJDQGDFWLRQ¶ S :LWKLQWKHFDWHJRU\RIGLVSOD\WKH\
GLVFXVVVHYHUDOW\SHVRIPDWULFHVJUDSKVFKDUWVDQGQHWZRUNVWKDWDOORUJDQL]HDQG
SUHVHQWLQIRUPDWLRQLQDYLVXDOO\DFFHVVLEOHPDQQHULQDSSURSULDWHDQGVLPSOLILHG
µJHVWDOWVRUHDVLO\XQGHUVWRRGFRQILJXUDWLRQV¶ S 6XFKVFKHPDWLFUHSUHVHQWDWLRQV
FDQWDNHDZLGHYDULHW\RIIRUPVIURPWUDGLWLRQDOWDEOHVZLWKUHJXODUURZVDQGFROXPQV
WRIORZFKDUWVDQGYDULRXVYLVXDOµFDWHJRU\PDSV¶ )LJXUHSUHVHQWVDQH[DPSOHRI
WKHODWWHU

)LJXUH6DPSOHFRPSXWHUVFUHHQVKRWGLVSOD\LQJDFDWHJRU\PDS
7KHLPSRUWDQFHRIGDWDGLVSOD\OLHVLQWKHIDFWWKDWWKHYDULRXVYLVXDOVXPPDULHVDUH
SRZHUIXOWRROVWRKHOSWKHUHVHDUFKHUGUDZYDOLGFRQFOXVLRQVDV0LOHVDQG+XEHUPDQ
FRQFOXGHWKHµGLFWXP³<RXDUHZKDW\RXHDW´PLJKWEHWUDQVSRVHGWR³<RX
NQRZZKDW\RXGLVSOD\´¶2QZXHJEX]LHDQG/HHFK DOVRSRLQWRXWWKDWLWLV
SRVVLEOHIRUWKHGDWDGLVSOD\WREHVRFRPSHOOLQJWKDWGDWDLQWHUSUHWDWLRQFDQ
LPPHGLDWHO\EHJLQZLWKRXWUHTXLULQJDQ\IXUWKHUDQDO\WLFDORSHUDWLRQV
,QWHUSUHWLQJWKHGDWDDQGGUDZLQJFRQFOXVLRQV

7KHILQDOJHQHULFSURFHVVRITXDOLWDWLYHGDWDDQDO\VLVLVLQWHUSUHWLQJWKHGDWDDQG
GUDZLQJFRQFOXVLRQV6WULFWO\VSHDNLQJGDWDLQWHUSUHWDWLRQKDSSHQVQRWRQO\QHDUWKH
HQGRIDSURMHFWUHVHDUFKHUVVWDUWWHQWDWLYHLQWHUSUHWLQJDVHDUO\DVWKHLQLWLDOFRGLQJ
VWDJHZKHQWKH\SUHSDUHPHPRVLQIDFWWKHZKROHSURFHVVRITXDOLWDWLYHGDWDDQDO\VLV
LVJHDUHGWRRXUEHFRPLQJPRUHDQGPRUHIDPLOLDUZLWKWKHGDWDDQGGHYHORSLQJ
LQFUHDVLQJO\DEVWUDFWDQDO\WLFDOLQVLJKWVLQWRWKHXQGHUO\LQJPHDQLQJV7KXVDVZLWKVR
PDQ\FRPSRQHQWVRITXDOLWDWLYHUHVHDUFKGDWDLQWHUSUHWDWLRQLVDOVRDQLWHUDWLYH
SURFHVV<HWLWLVXVHIXOWRGLVWLQJXLVKWKHILQDOVWDJHVRIWKLVSURFHVVIURPWKHHDUOLHU
RQJRLQJLQWHUSUHWDWLRQEHFDXVHWKLVLVZKHUHZHKDYHWRVHOHFWWKHRYHUDUFKLQJWKHPH
RUWKHPHVWKDWWKHZULWHXSZLOOEHFHQWUHGDURXQG,QRWKHUZRUGVWKLVLVZKHUHWKH
SURFHVVLVWXUQHGLQWRDSURGXFWLQWKHIRUPRIWKHILQDOFRQFOXVLRQV
:KDWDUHWKHVHILQDOFRQFOXVLRQVOLNH"7RVWDUWZLWKWKH\QDWXUDOO\EXLOGRQWKH
YDULRXVLQWHULPVXPPDULHVDQGLQWHUSUHWDWLRQV IRUH[DPSOHPHPRVYLJQHWWHVSURILOHV
DQGGLVSOD\V DVZHOODVWKHKLJKHUOHYHOFRGLQJFDWHJRULHVWKDWWKHUHVHDUFKHUKDV
GHYHORSHG,QTXDOLWDWLYHUHVHDUFKWKHPDLQWKHPHVJURZRUJDQLFDOO\RXWRIWKH
IRXQGDWLRQVWKDWZHUHODLGGXULQJWKHDQDO\WLFDOSURFHVVDQGWKHUHIRUHWKHSURFHVVRI
GUDZLQJWKHILQDOFRQFOXVLRQVLQYROYHVWDNLQJVWRFNRIZKDWZHKDYHJRWDSSUDLVLQJWKH
JHQHUDWHGSDWWHUQVDQGLQVLJKWVDQGILQDOO\VHOHFWLQJDOLPLWHGQXPEHURIPDLQWKHPHV
RUVWRU\OLQHVWRHODERUDWHRQ6HOHFWLRQLVEDVHGRQWKHVDOLHQFHRIWKHSDUWLFXODU
FRQFHSWSURFHVVDQGLWVUHODWLRQVKLSZLWKRWKHULPSRUWDQWFDWHJRULHVLQWKHGRPDLQ
LGHDOO\WKHPDLQWKHPH V VKRXOGVHUYHDVDIRFXVRUOHQVWKURXJKZKLFKWKHZKROH
GRPDLQFDQEHSUHVHQWHG
5LFKDUGV ZDUQVXVWKDWWKHSURPLVHRIDVLQJOHVWRU\OLQHRUDµFRUHWKHPH¶
WKDWEULQJVHYHU\WKLQJWRJHWKHUFDQEHVHGXFWLYHDWWKLVVWDJH\HWLWFDUULHVWKHGDQJHURI
HOLPLQDWLQJWKHVXEWOHW\RIPHDQLQJVWKDWZHKDYHZRUNHGVRKDUGWRXQFRYHU$VVKH
DUJXHVLWPD\RIWHQEHLPSRVVLEOHWRSURGXFHDXQLWDU\DFFRXQWZLWKRXWLWEHFRPLQJ
WULYLDOZKHUHDVH[SODLQLQJGLYHUJHQFHVDQGWKHUHDVRQVZK\WKHUHLVQRELJSLFWXUHPD\
UHVXOWLQDSRZHUIXOXQGHUVWDQGLQJRIWKHVLWXDWLRQ,QRWKHUZRUGVGUDZLQJWKHILQDO
FRQFOXVLRQLVDGHOLFDWHEDODQFLQJDFWEHWZHHQWU\LQJWRVD\VRPHWKLQJRIRYHUDUFKLQJ
VLJQLILFDQFHZKLOHDWWKHVDPHWLPHSUHVHUYLQJWKHLQWULFDF\RIVLWXDWHGPXOWLSOH
PHDQLQJV
*URXQGHGWKHRU\
,I\RXGRQ¶WNQRZZKDW>TXDOLWDWLYH@PHWKRG\RXDUHXVLQJLWLVKLJKO\XQOLNHO\WR
EHJURXQGHGWKHRU\
0RUVHDQG5LFKDUGV
7KHWHUPµJURXQGHGWKHRU\¶LVVRFRPPRQLQWKHTXDOLWDWLYHUHVHDUFKOLWHUDWXUHWKDWLWLV
RIWHQWDNHQDVDV\QRQ\PIRUTXDOLWDWLYHUHVHDUFK:KLOHJURXQGHGWKHRU\LVQRWWKH
VDPHDVTXDOLWDWLYHUHVHDUFKLQJHQHUDOLWLVUHYHDOLQJWRORRNLQWRWKHUHDVRQIRUWKLV
FRQIXVLRQ)LUVWRIDOOWKHERXQGDULHVRIWKHWHUPDUHUDWKHUIOXLGSDUWLFXODUO\ZKHQLW

FRPHVWRDFWXDODSSOLFDWLRQV$VZHZLOOVHHLQ6HFWLRQGHVSLWHLWVQDPH
µJURXQGHGWKHRU\¶LVQRWDWKHRU\DWDOOEXWUDWKHUDTXDOLWDWLYHUHVHDUFKPHWKRG<HW
WKLVLVVWLOOQRWTXLWHFRUUHFWEHFDXVHLQVSLWHRIWKHIDFWWKDWJURXQGHGWKHRU\LVFODLPHG
WRDSSO\WRWKHZKROHUHVHDUFKSURFHVVIURPSDUWLFLSDQWVDPSOLQJDQGGDWDFROOHFWLRQWR
GDWDDQDO\VLV&KDUPD] SRLQWVRXWWKDWUHVHDUFKHUVFRPPRQO\XVHWKHWHUPWR
PHDQRQO\DVSHFLILFPRGHRIGDWDDQDO\VLV 6HHDOVR'H\IRUDVLPLODUYLHZ $V
VKHVXPPDUL]HVµ(VVHQWLDOO\JURXQGHGWKHRU\PHWKRGVDUHDVHWRIIOH[LEOHDQDO\WLF
JXLGHOLQHVWKDWHQDEOHUHVHDUFKHUVWRIRFXVWKHLUGDWDFROOHFWLRQDQGWREXLOGLQGXFWLYH
PLGGOHUDQJHWKHRULHVWKURXJKVXFFHVVLYHOHYHOVRIGDWDDQDO\VLVDQGFRQFHSWXDO
GHYHORSPHQW¶ S )RUWKLVUHDVRQ,GLVFXVVJURXQGHGWKHRU\LQWKLVFKDSWHUUDWKHU
WKDQLQ&KDSWHU
*LYHQWKDWJURXQGHGWKHRU\LVE\DQGODUJHVHHQDVDVSHFLILFDQDO\WLFDSSURDFKLWLV
HYHQPRUHFXULRXVWKDWLWVSRSXODULW\ DVUHIOHFWHGLQFLWDWLRQVLQUHVHDUFKWH[WV LVVXFK
WKDWLWFDQHDVLO\EHVHHQDVDFRPPRQPHWKRGRORJLFDOIUDPHZRUNIRUTXDOLWDWLYH
UHVHDUFKLQJHQHUDO $UNVH\DQG.QLJKW 3DUWRIWKHH[SODQDWLRQOLHVLQWKHIDFW
WKDWDVZHVDZLQ6HFWLRQJURXQGHGWKHRU\RIIHUHGWKHILUVWFRQVFLRXVDWWHPSWWR
EUHDNWKHKHJHPRQ\RITXDQWLWDWLYHUHVHDUFKLWRIIHUHGTXDOLWDWLYHO\LQFOLQHG
UHVHDUFKHUVDWKHRUHWLFDOO\EDVHGHODERUDWHPHWKRGRORJ\7KXVZLWKLQWKHFRQWH[WRI
WKHHDUO\SDUDGLJPZDUVJURXQGHGWKHRU\ZDVWKHKHDY\DUWLOOHU\WKDWTXDOLWDWLYH
UHVHDUFKHUVHPSOR\HGWRFODLPOHJLWLPDF\$V&KDUPD] SXWVLWµ)RUPDQ\
UHVHDUFKHUVJURXQGHGWKHRU\PHWKRGVSURYLGHGDWHPSODWHIRUGRLQJTXDOLWDWLYH
UHVHDUFKVWDPSHGZLWKSRVLWLYLVWDSSURYDO¶6KHJRHVRQWRDUJXHWKDWHYHQWKRXJK
UHVHDUFKHUVGLGQRWDOZD\VXQGHUVWDQGJURXQGHGWKHRU\PHWKRGVDQGUDUHO\IROORZHG
WKHPIXOO\WKH\ZLGHO\FLWHGDQGDFFODLPHGWKHVHPHWKRGVEHFDXVHWKH\MXVWLILHGWKHLU
SUHYLRXVO\LPSOLFLWSUDFWLFH
*URXQGHGWKHRU\KDVWKXVEHFRPHDEDQQHUXQGHUZKLFKTXDOLWDWLYHUHVHDUFKHUV
FRXOGMRLQIRUFHVDQGWKLVH[SODLQVLWVSHUYDVLYHLQIOXHQFHRQTXDOLWDWLYHUHVHDUFK
PHWKRGRORJ\LQJHQHUDOZLWKDOPRVWHYHU\TXDOLWDWLYHUHVHDUFKHUXVLQJ²NQRZLQJO\RU
XQNQRZLQJO\²VRPHRILWVHOHPHQWV,WKDVEHFRPHSDUWRIWKHPDLQVWUHDPDQGPDQ\RI
LWVRULJLQDOO\XQLTXHPHWKRGRORJLFDODVSHFWVKDYHQRZEHFRPHFRUHLVVXHVLQ
TXDOLWDWLYHUHVHDUFK:HVKRXOGEHFDUHIXOKRZHYHUQRWWRFDOORXUUHVHDUFKµJURXQGHG
WKHRU\¶XQOHVVDWOHDVWWZREDVLFFULWHULDDUHPHW D 7KHGDWDDQDO\VLVIROORZVWKH
VSHFLILFVHTXHQWLDOFRGLQJV\VWHPDGYRFDWHGE\JURXQGHGWKHRU\ VHHEHORZ DQG E
WKHDQDO\VLVSURGXFHVVRPHWKHRU\DVDQRXWFRPHRIWKHLQYHVWLJDWLRQ,QWKHIROORZLQJ
DIWHUSURYLGLQJDEULHIKLVWRULFDORYHUYLHZ,H[DPLQHWKHVHWZRFULWHULD
7KHJHQHVLVRIJURXQGHGWKHRU\JRHVEDFNWRWKHVZKHQWZR$PHULFDQ
VRFLRORJLVWV%DUQH\*ODVHUDQG$QVHOP6WUDXVVVWDUWHGWRH[SHULPHQWZLWKDQHZ
TXDOLWDWLYHPHWKRGLQWKHLULQYHVWLJDWLRQRILQWHUDFWLRQVEHWZHHQKHDOWKFDUH
SURIHVVLRQDOVDQGWHUPLQDOSDWLHQWVLQKRVSLWDOV,QWKH\SXEOLVKHGDERRNZKLFK

GHWDLOHGWKHPHWKRGVRIWKHLUE\WKDWWLPHKLJKO\DFFODLPHGµG\LQJVWXGLHV¶HQWLWOHG
µ7KH'LVFRYHU\RI*URXQGHG7KHRU\6WUDWHJLHVIRU4XDOLWDWLYH5HVHDUFK¶$V
PHQWLRQHGDERYHJURXQGHGWKHRU\ZDVWKHILUVWTXDOLWDWLYHDSSURDFKWKDWKDGEHHQ
VXIILFLHQWO\HODERUDWHDQGSURFHGXUDOO\ULJRURXVHQRXJKWRZLWKVWDQGWKHFULWLFLVPVRI
TXDQWLWDWLYHVFKRODUV7KHPHWKRGJUHZWKHUHIRUHIURPLWVRULJLQDOVRFLRORJLFDOURRWV
WREHDGRSWHGZLGHO\E\UHVHDUFKHUVLQDYDULHW\RIILHOGVLQWKHVRFLDOVFLHQFHV
LQFOXGLQJHGXFDWLRQDQGSV\FKRORJ\,WLVXQGRXEWHGO\RQHRIWKHPRVWLQIOXHQWLDO
SDUDGLJPVLQTXDOLWDWLYHUHVHDUFKWRGD\
,URQLFDOO\LWLVWKHPDLQVWUHQJWKRIJURXQGHGWKHRU\WKHGHWDLOHGSURFHGXUDO
JXLGHOLQHVIRUGDWDDQDO\VLVWKDWOHGWRDULIWEHWZHHQWKHWZRRULJLQDWRUVRIWKHPHWKRG
,Q*ODVHUFULWLFL]HGDERRN6WUDXVVKDGZULWWHQZLWKDQRWKHUFRDXWKRU-XOLHW
&RUELQIRUDGYRFDWLQJWKHXVHRISUHFRQFHLYHGLGHDVDQGWKXVIRUFLQJUHVHDUFKLQWRD
VHWIUDPHZRUNZLWKRXWDOORZLQJQHZWKHRULHVWRHPHUJH²DFODVVLFH[DPSOHRIWKH
WHQVLRQEHWZHHQVXEMHFWLYHLQWXLWLRQYHUVXVIRUPDOL]DWLRQGHVFULEHGLQ6HFWLRQ
7KHGLYLVLRQEHWZHHQ*ODVHUDQG6WUDXVVVSUHDGWRWKHTXDOLWDWLYHUHVHDUFKFRPPXQLW\
LQJHQHUDOZLWKUHVHDUFKHUVRIWHQEHLQJH[SHFWHGWRDOLJQWKHPVHOYHVZLWKRQHRIWKH
WZRFRPSHWLQJYHUVLRQVRIJURXQGHGWKHRU\
/DWHUIXUWKHUFKDVPVDSSHDUHGLQJURXQGHGWKHRU\ VHHIRUH[DPSOH&KDUPD]
DQGDVDUHVXOWQRZDGD\VZHFDQQRWWDONDERXWDVLQJOHXQLILHGJURXQGHGWKHRU\
PHWKRGRORJ\ 'H\ 6HYHUDOYDULDWLRQVRIWKHPHWKRGDUHSUDFWLVHGDQGDVDOUHDG\
GLVFXVVHGDODUJHQXPEHURIUHVHDUFKHUVXWLOL]HHOHPHQWVRIWKHPHWKRG SULPDULO\WKH
FRGLQJSURFHGXUHV WRFDUU\RXWLQYHVWLJDWLRQVWKDWPD\RUPD\QRWEHWUXHJURXQGHG
WKHRU\VWXGLHV )DVVLQJHU
µ7KHRU\¶LQJURXQGHGWKHRU\
*URXQGHGWKHRU\LVQRWDWKHRU\VRZKHUHGRHVWKHµWKHRU\¶FRPSRQHQWRIWKHODEHO
RULJLQDWH",WFRPHVIURPWKHH[SOLFLWJRDORIWKHPHWKRGWRGHYHORSDQHZWKHRU\7KLV
DLPWRSURGXFHQHZWKHRUHWLFDORIIHULQJVHPSKDVL]HGDJDLQDQGDJDLQLQWKHJURXQGHG
WKHRU\OLWHUDWXUHZDVVHWE\*ODVHUDQG6WUDXVV DVDUHDFWLRQWRWKHWKHRU\
YHULILFDWLRQSUDFWLFHWKDWGRPLQDWHGTXDQWLWDWLYHUHVHDUFKLQWKHPLGGOHRIWKHWK
FHQWXU\7KXVWKHODEHOZDVLQWHQGHGWRSRVLWLRQWKHDSSURDFKDVLQGXFWLYHLQFRQWUDVW
WRGHGXFWLYHDQGWKHWHUPµJURXQGHG¶LQGLFDWHGWKDWWKHQHZWKHRUHWLFDOLQVLJKWVZHUH
WREHJHQHUDWHGRQWKHEDVLVRIHPSLULFDOGDWD
7KHREYLRXVTXHVWLRQDWWKLVSRLQWLVZKDWH[DFWO\µWKHRU\¶PHDQVLQJURXQGHG
WKHRU\WHUPV:HFDQUHVWDVVXUHGWKDW*ODVHUDQG6WUDXVVGLGQRWPHDQE\WKHWHUPDQ\
µJUDQGWKHRULHV¶VXFKDVWKH7KHRU\RI5HODWLYLW\EXWUDWKHUDQDEVWUDFWH[SODQDWLRQRID
SURFHVVDERXWDVXEVWDQWLYHWRSLFJURXQGHGLQWKHGDWD &UHVZHOO 7KHPDLQSRLQW
DERXWµWKHRU\¶LVWKDWUHVHDUFKHUVVKRXOGJREH\RQGPHUHO\GHVFULELQJRUFDWHJRUL]LQJ
WKHWDUJHWSKHQRPHQRQDQGVKRXOGRIIHUVRPHEDVLFXQGHUVWDQGLQJRIWKHSULQFLSOHV
UHODWLRQVKLSVFDXVHVDQGRUPRWLYHVXQGHUO\LQJLW)DVVLQJHU SRLQWVRXWWKDW
WKHUHLVFRQVLGHUDEOHFRQWURYHUV\DERXWZKDWVRUWRISURGXFWFDQOHJLWLPDWHO\FODLPWR

EHDµWKHRU\¶SURSHUEXWLIWKHRXWFRPHRIRXUUHVHDUFKDUWLFXODWHVDFRKHUHQW
FRQWH[WXDOL]HGH[SODQDWLRQ UDWKHUWKDQPHUHO\DFRQWH[WXDOGHVFULSWLRQ RIDQLVVXH
SRVVLEO\DOVRRXWOLQLQJD WHQWDWLYH PRGHORUIUDPHZRUNJURXQGHGWKHRULVWVZRXOGEH
OLNHO\WRDFFHSWWKDWDVDYDOLGWKHRUHWLFDORIIHULQJ:KLOHGHILQLQJµWKHRU\¶LQVXFKOHVV
WKDQJUDQGDPDQQHUPDNHVVHQVH,EHOLHYHWKDW6HLGPDQ DOVRKDVDSRLQWZKHQ
KHVWDWHVWKDWWKHIUHTXHQWXVHRIµWKHRU\¶E\JURXQGHGWKHRULVWVKDVVHUYHGWRLQIODWHWKH
WHUPWRWKHSRLQWWKDWLWKDVORVWVRPHRILWVXVHIXOQHVV
&RGLQJLQJURXQGHGWKHRU\
7KHEHVWNQRZQDVSHFWRIJURXQGHGWKHRU\LVWKHGLVWLQFWLRQVEHWZHHQGLIIHUHQWSKDVHV
RIFRGLQJWKHGDWD 6HH6WUDXVVDQG&RUELQ :HVDZLQ6HFWLRQWKDW
FRGLQJLQTXDOLWDWLYHUHVHDUFKLVDPXOWLOHYHOSURFHGXUHDQGJURXQGHGWKHRU\GHVFULEHV
DORJLFDOWKUHHOHYHOV\VWHPILUVWZHEUHDNXSWKHGDWDLQWRFKXQNVDQGDVVLJQ
FRQFHSWXDOFDWHJRULHVWRWKHGDWDVHJPHQWV µRSHQFRGLQJ¶ 6HFRQGZHLGHQWLI\
LQWHUUHODWLRQVKLSVEHWZHHQWKHVHFDWHJRULHV µD[LDOFRGLQJ¶ 7KLUGZHH[SODLQWKHVH
UHODWLRQVKLSVDWDKLJKHUOHYHORIDEVWUDFWLRQ µVHOHFWLYHFRGLQJ¶ 7KXVWKHSURFHVV
DSSHDUVWREHVHTXHQWLDOPRYLQJIURPWKHGHVFULSWLYHWRWKHDEVWUDFWEXWJLYHQWKH
LQKHUHQWO\LWHUDWLYHQDWXUHRITXDOLWDWLYHUHVHDUFKLWVKRXOGFRPHDVQRVXUSULVHWKDWWKH
WKUHHSKDVHVRFFXUUHFXUVLYHO\
2SHQFRGLQJ
µ2SHQFRGLQJ¶FRQVWLWXWHVWKHILUVWOHYHORIFRQFHSWXDODQDO\VLVRIWKHGDWD7KHWH[WXDO
GDWDLVµEURNHQRSHQ¶LQWRFKXQNVZKRVHOHQJWKXVXDOO\YDULHVEHWZHHQDORQJSKUDVHD
OLQHDVHQWHQFHRUHYHQDVKRUWSDUDJUDSK(DFKRIWKHVHVHJPHQWVLVDVVLJQHGD
FDWHJRU\ODEHOEXWLQOLQHZLWKWKHWKHRU\EXLOGLQJHPSKDVLVRIWKHPHWKRGWKH
HPSKDVLVDOUHDG\DWWKLVVWDJHLVRQVWLPXODWLQJQHZLGHDVDQGWKHUHIRUHWKHFDWHJRULHV
DUHDEVWUDFWDQGFRQFHSWXDOUDWKHUWKDQGHVFULSWLYH7KLVSUDFWLFHLVGLIIHUHQWIURPRWKHU
IRUPVRITXDOLWDWLYHDQDO\VLVZKLFKXVXDOO\VWDUWRXWZLWKGHVFULSWLYHRULQWHUSUHWLYH
FDWHJRULHVLQRSHQFRGLQJWKHPDLQTXHVWLRQVWKHUHVHDUFKHUDVNVDERXWHDFKVHJPHQW
DUHRIWKHIROORZLQJW\SH:KDWLVWKLVSLHFHRIGDWDDQH[DPSOHRI":KDWLVJRLQJRQ"
:KDWSULQFLSOHVXQGHUOLHWKHVHDFWLRQVVWDWHPHQWV":KDWGRWKH\DFWXDOO\PHDQ"7KXV
HDFKGDWDVHJPHQWLVWKRURXJKO\µLQWHUURJDWHG¶EHIRUHDVVLJQLQJPHDQLQJWRLWUHVXOWLQJ
LQZKDWZHFDQFDOODQDEVWUDFWLQJSURFHVVIURPGDWDWRILUVWRUGHUFRQFHSWV
$[LDO RUWKHRUHWLFDO FRGLQJ

µ$[LDOFRGLQJ¶H[WHQGVWKHDEVWUDFWLQJSURFHVVIURPILUVWRUGHUFRQFHSWVWRKLJKHURUGHU
FRQFHSWV7KHUHVHDUFKHUPDNHVFRQQHFWLRQVEHWZHHQFDWHJRULHVWKHUHE\DWWHPSWLQJWR
LQWHJUDWHWKHPDQGJURXSWKHPLQWRPRUHHQFRPSDVVLQJFRQFHSWVWKDWVXEVXPHVHYHUDO
VXEFDWHJRULHV7KHZRUGµD[LDO¶ZDVXVHGE\6WUDXVVDQG&RUELQ WRVLJQLI\WKDWD
PHWDSKRULFDOD[LVLVSODFHGWKURXJKWKHGDWDWRUHODWHWKHFDWHJRULHVWRHDFKRWKHU
*ODVHU XVHGWKHPRUHJHQHUDOWHUPµWKHRUHWLFDOFRGLQJ¶WRGHVFULEHWKLVVWDJH
7KHUHODWLRQVKLSVEHWZHHQWKHFDWHJRULHVFDQEHPDQLIROGUHIHUULQJIRUH[DPSOHWR

FDXVDOFRQGLWLRQVFRQVHTXHQFHVDQGVLPLODULWLHVDVZHOODVFRQWH[WXDOSURFHGXUDORU
VWUDWHJLFLQWHUGHSHQGHQFH7KHVHUHODWLRQDOSURSRVLWLRQVRIWHQDSSHDUILUVWLQWKH
UHVHDUFKHU¶VPHPRV²PHPRLQJLVDQLQWHJUDOSDUWRIJURXQGHGWKHRU\DQDO\VLVDQGLV
GRQHLQWKHVDPHZD\DVGHVFULEHGLQ6HFWLRQ²EXWVRPHWLPHVWKHLQIRUPDQWV
WKHPVHOYHVZLOODOHUWWKHUHVHDUFKHUWRWKHH[LVWHQFHRIDSDWWHUQ (OOLVDQG%DUNKXL]HQ

:HVKRXOGQRWHWKDWE\IRUPLQJVXFKUHODWLRQVKLSVZHDUHDOUHDG\EHJLQQLQJWR
KLJKOLJKWDQGSRVLWLRQFHUWDLQFDWHJRULHVLQWKHFHQWUHRIWKHFRGLQJSURFHVV²WKLVZLOO
IRUPWKHEDVLVRIWKHWKLUGµVHOHFWLYHFRGLQJ¶SKDVH&UHVZHOO VXJJHVWVWKDWDW
WKLVVWDJHZHPLJKWUHWXUQWRGDWDFROOHFWLRQRUUHDQDO\VHRXUH[LVWLQJGDWDLQRUGHUWR
H[SOLFDWHRUUHILQHWKHQHZO\HVWDEOLVKHGFRQQHFWLRQVEHWZHHQWKHFDWHJRULHV
GHOLQHDWLQJWKHQDWXUHRIWKHUHODWLRQVKLSVEHWZHHQWKHP
6HOHFWLYHFRGLQJ
µ6HOHFWLYHFRGLQJ¶LVWKHILQDOVWDJHLQJURXQGHGWKHRU\DQDO\VLVLQZKLFKDVWKHWHUP
VXJJHVWVZHQHHGWRVHOHFWDµFRUHFDWHJRU\¶ RUDV5LFKDUGVSXWVLWµH[SODQDWRU\
FRQFHSW¶ WRFRQFHQWUDWHRQLQWKHUHVWRIWKHDQDO\VLVDQGWKHZULWLQJXSRIWKHVWXG\
7KLVFRUHFDWHJRU\ZLOOEHWKHFHQWUHSLHFHRIWKHSURSRVHGQHZWKHRU\%HFDXVH
XOWLPDWHO\ZHZDQWWRLQWHJUDWHPXFKRIRXUSURFHVVHGGDWDWKHFHQWUDOFDWHJRU\WKHPH
QHHGVWREHRIDVXIILFLHQWO\KLJKOHYHORIDEVWUDFWLRQWREHDEOHWRVXEVXPHRWKHU
FDWHJRULHV7KHPRVWFRPPRQPHWKRGRIVHOHFWLQJWKHFHQWUDOFRQFHSWLVWRIXUWKHU
HODERUDWHRQDQLGHDWKDWKDVDOUHDG\EHHQUHFRUGHGLQPHPRVDQGZKLFKKDVDOVREHHQ
H[SORUHGGXULQJWKHD[LDOFRGLQJVWDJH2EYLRXVO\WKHFHQWUDOWKHPHQHHGVWREHFHQWUDO
ERWKIURPRXURZQDQGWKHSDUWLFLSDQWV¶SHUVSHFWLYHEHFDXVHZHQHHGVXFKDIRFDO
SRLQWWREHDEOHWREULQJWRJHWKHUWKHRWKHUFDWHJRULHVLQDFRKHUHQWPDQQHU$OWKRXJK
WKLVILQDODQDO\WLFDOVWDJHGLVSOD\VJUHDWYDULDELOLW\DFURVVVWXGLHVWKHRXWFRPHLV
W\SLFDOO\DµFRUHVWRU\¶ZLWKDVLQJOHVWRU\OLQH²WKDWLVWKHSURSRVHGµWKHRU\¶7KLV
WKHRU\LVDOVRHYDOXDWHGDJDLQVWWKHH[LVWLQJOLWHUDWXUHWRHQULFKXQGHUVWDQGLQJDQG
H[SODQDWRU\SRZHU )DVVLQJHU
6WUHQJWKVDQGZHDNQHVVHVRIJURXQGHGWKHRU\
$PDMRUVWUHQJWKRIJURXQGHGWKHRU\LVWKDWWKHPHWKRGSURYLGHVWRROVIRUSURYLGLQJDQ
LQGHSWKDQDO\VLVRIDSKHQRPHQRQ$V&KDUPD] FRQFOXGHVµWKHIRFXVHG
LQTXLU\RIJURXQGHGWKHRU\ZLWKLWVSURJUHVVLYHLQGXFWLYHDQDO\VLVPRYHVWKHZRUN
WKHRUHWLFDOO\DQGFRYHUVPRUHHPSLULFDOREVHUYDWLRQVWKDQRWKHUDSSURDFKHV¶7KH
PHWKRGLVSDUWLFXODUO\DSSURSULDWHLQJHQHUDWLQJWKHRUHWLFDONQRZOHGJHLQDUHDVZKHUH
OLWWOHLVNQRZQDERXWDSKHQRPHQRQ7KHV\VWHPDWLFDQGGHWDLOHGSURFHGXUHVDVVRFLDWHG
ZLWKWKHYDULRXVJURXQGHGWKHRU\DSSURDFKHV SDUWLFXODUO\ZLWKWKH6WUDXVVDQG&RUELQ
YHUVLRQ PDNHLWUHODWLYHO\HDV\IRUQRYLFHUHVHDUFKHUVWRDGRSWWKHPHWKRGDQG
WKHDYDLODELOLW\RIWKHVHSURFHGXUHVDOVRIDFLOLWDWHWKHWUDQVLWLRQIURPDTXDQWLWDWLYH
UHVHDUFKWUDGLWLRQWRDPRUHQDWXUDOLVWLFSDUDGLJP

2QWKHRWKHUKDQGWRGRJURXQGHGWKHRU\DQDO\VLVZHOOLVDGHPDQGLQJDQGODERXU
LQWHQVLYHWDVNUHTXLULQJKLJKO\GHYHORSHGDQDO\WLFDODQGFRQFHSWXDOVNLOOV,QP\
H[SHULHQFHWKHUHLVDUHDOGDQJHUIRUJRLQJUHOLJLRXVO\WKURXJKWKHWLPHFRQVXPLQJ
SURFHVVRIWKHFRGLQJVHTXHQFHVLQDQLWHUDWLYHPDQQHUDQGVWLOOHQGLQJXSZLWKQR
µWKHRU\¶RUDUDWKHUWULYLDORQH7KHGLIILFXOW\RIWKHJURXQGHGWKHRU\DSSURDFKFDQEH
HDVHGE\XWLOL]LQJFRPSXWHUVLQWKHDQDO\VLV²DVGLVFXVVHGLQWKHQH[WVHFWLRQWKHUHDUH
VHYHUDOFRPSXWHUSURJUDPVWKDWKDYHEHHQVSHFLILFDOO\GHVLJQHGWREHXVHGLQJURXQGHG
WKHRU\DQDO\VLV
&RPSXWHUDLGHGTXDOLWDWLYHGDWDDQDO\VLV
&$4'$6
,QWKHILQDOVHFWLRQRIWKLVFKDSWHU,ORRNDWKRZFRPSXWHUVFDQEHXWLOL]HGLQTXDOLWDWLYH
UHVHDUFK,QWKHSUHYLRXVFKDSWHURQTXDQWLWDWLYHGDWDDQDO\VLVWKHLQWURGXFWLRQRID
VWDWLVWLFDOVRIWZDUH 6366 ZDVGRQHLQWKHILUVWVHFWLRQ DQGWKHYHU\GLIIHUHQW
SRVLWLRQLQJRIWKHFRPSXWHUVHFWLRQVLQWKH48$1DQG48$/FKDSWHUVLVWHOOLQJ
:KHUHDVWKHXVHRIFRPSXWHUVE\TXDQWLWDWLYHUHVHDUFKHUVKDVEHHQVWDQGDUGSUDFWLFHIRU
VHYHUDOGHFDGHVFRPSXWHUDLGHGTXDOLWDWLYHGDWDDQDO\VLV &$4'$6 VRIWZDUHKDV
EHHQDUHODWLYHO\UHFHQWSKHQRPHQRQ+RZHYHUSURJUHVVERWKLQFRPSXWHUOLWHUDF\
DPRQJVWUHVHDUFKHUVDQGLQWKHTXDOLW\RI&$4'$6VRIWZDUHKDVEHHQVRUDSLGRYHUWKH
SDVWGHFDGHWKDWIHZHUDQGIHZHUPHWKRGRORJLVWVZRXOGTXHVWLRQ5LFKDUGV¶V
FODLPWKDWZRUNLQJZLWKRXWFRPSXWHUVLVVLPSO\QRORQJHUDPHUHRSWLRQIRUTXDOLWDWLYH
UHVHDUFKHUVZKHWKHUZHOLNHLWRUQRWZHKDYHWRDFFHSWWKDWFRPSXWHUVKDYHEHFRPH
DQLQWHJUDOSDUWRITXDOLWDWLYHGDWDDQDO\VLV7KLVKDVEHHQUHFRJQL]HGLQWKH8.E\WKH
OHDGLQJUHVHDUFKIXQGLQJDJHQF\LQWKHVRFLDOVFLHQFHVWKH(FRQRPLFDQG6RFLDO
5HVHDUFK&RXQFLO (65& ZKHQWKH\DGYLVHGLQWKHLUUHYLVHGJXLGHOLQHVIRUWKH
WUDLQLQJRIJUDGXDWHVWXGHQWVWKDWVWXGHQWVVKRXOGKDYHVNLOOLQWKHXVHRIVXFKVRIWZDUH
SDFNDJHV
$VDUHVXOWRIWKLVFKDQJLQJSHUVSHFWLYHFXUUHQWGLVFXVVLRQVRI&$4'$6QRORQJHU
QHHGWRDGGUHVVWKHTXHVWLRQVWKDWXVHGWREHWKHFUX[RIWKHPDWWHUIRUSRWHQWLDOXVHUV
QDPHO\µ:LOOWKHVRIWZDUHVDYHPHWLPH"¶RUµ,VLWZRUWKWKHHIIRUWRIOHDUQLQJWRXVH
LW"¶,QVWHDGWKHQHZTXHVWLRQVWRDVNDUHµ:KDWFDQWKHVRIWZDUHRIIHUTXDOLWDWLYH
UHVHDUFKHUV"¶DQGµ+RZFDQ,PDNHWKHPRVWRILWIRUWKHSXUSRVHRIP\DQDO\VLV"¶
:KDWGRHVFRPSXWHUVRIWZDUHRIIHUTXDOLWDWLYH
UHVHDUFKHUV"
7KHPRVWLPSRUWDQWSRLQWWRVWUHVVDERXW&$4'$6VRIWZDUHLVWKDWWKHVHSURJUDPVGR
QRWGRDQ\UHDODQDO\VLVIRUXVUDWKHUWKH\KDYHEHHQGHVLJQHGWRKHOSUHVHDUFKHUVZLWK
WKHPHFKDQLVDWLRQRIWKHFOHULFDODVSHFWVRIGDWDPDQDJHPHQW$Q\ERG\ZKRKDVGRQH
TXDOLWDWLYHGDWDDQDO\VLVNQRZVWKDWVRUWLQJRUJDQL]LQJDQGDUFKLYLQJKXQGUHGVRI
SDJHVRIUDWKHUPHVV\LQWHUYLHZWUDQVFULSWVFDQEHDORJLVWLFDOQLJKWPDUHZLWKWKH
UHVHDUFKHUWU\LQJWRORFDWHXVHIXOTXRWHVKLGGHQLQWKHPLGGOHRIWKLFNSLOHVDQGFXWWLQJ

RXWH[WUDFWVWREHVWXFNRQVPDOOPHPRFDUGV7KLVLVZKHUH&$4'$6RIIHUV
LQYDOXDEOHDVVLVWDQFHEXW²OHWPHUHLWHUDWH²WKLVDVVLVWDQFHLVE\DQGODUJHQRWUHODWHG
WRWKHDFWXDODQDO\VLVDVSHFWRIWKHUHVHDUFK)RUWKLVUHDVRQRQHRIWKHOHDGLQJH[SHUWV
RQ&$4'$6.HOOH DFWXDOO\VXJJHVWVWKDWLWZRXOGEHPRUHDSSURSULDWHWR
UHIHUWRWKHVHSURJUDPVDVVRIWZDUHIRUµGDWDDGPLQLVWUDWLRQDQGDUFKLYLQJ¶UDWKHUWKDQ
DVWRROVIRUµGDWDDQDO\VLV¶7KDWLVDV6pURU VXPPDUL]HVLQKLVUHFHQWUHYLHZRI
&$4'$6IRUDSSOLHGOLQJXLVWVZKDWFRPSXWHUVFDQRIIHULVWRUHSODFHWKHWUDGLWLRQDO
WRROVRITXDOLWDWLYHGDWDDQDO\VLV²SDSHUSHQVFLVVRUVDQGKLJKOLJKWHU²ZLWKWRROVRI
WKHGLJLWDODJHVFUHHQPRXVHDQGNH\ERDUG,QWKLVVHQVH&$4'$6VRIWZDUHLVYHU\
GLIIHUHQWIURPWKHVWDWLVWLFDOSDFNDJHVXVHGLQTXDQWLWDWLYHUHVHDUFKEHFDXVHWKHODWWHUGR
FDUU\RXWSDUWRIWKHDQDO\VLV )RUDQH[FHOOHQWVXPPDU\RIWKHIXQFWLRQVRI&$4'$6
ZULWWHQE\RQHRIWKHIRUHPRVWVRIWZDUHGHYHORSHUVVHH5LFKDUGV
,QUHWURVSHFWLWZDVLQHYLWDEOHWKDWFRPSXWHUVVKRXOGEHFRPHXWLOL]HGLQTXDOLWDWLYH
UHVHDUFKVLQFHWKH\DUHYHU\JRRGDWWZRWKLQJVWKDWDUHFUXFLDOIRU48$/GDWD
DQDO\VLVILUVWFRPSXWHUVFDQVWRUHODUJHDPRXQWVRIGDWDDQGJLYHQWKDWPRVW
GRFXPHQWV LQFOXGLQJWUDQVFULSWVRIUHFRUGLQJV DUHQRZDGD\VSURGXFHGHOHFWURQLFDOO\
LWZRXOGEHDQDFKURQLVWLFWRSULQWWKRVHRXWILUVWDQGWKHQVWRUHWKHPDVKDUGFRSLHV
6HFRQGKLJKOLJKWLQJH[WUDFWVLQHOHFWURQLFWH[WVLVDVLPSOHDQGEDVLFZRUGSURFHVVLQJ
WDVNDQGLWKDSSHQVWRFRLQFLGHZLWKDNH\FRPSRQHQWRIWKHTXDOLWDWLYHFRGLQJSURFHVV
7KXVRQFHWKHWHFKQRORJ\ZDVDYDLODEOHLWGLGQRWWDNHORQJWRGHYHORSWKHILUVW
VRIWZDUHSURJUDPVWKDWDOORZHGUHVHDUFKHUVWRDWWDFKODEHOVWRWKHKLJKOLJKWHGWH[W
VHJPHQWVVRWKDWWKHVHFRXOGEHUHWULHYHG²DQG&$4'$6ZDVRQWKHPDUFK
$IWHUWKLVLQLWLDOVWHSDERXW\HDUVDJRVFKRODUVUDSLGO\UHDOL]HGWKDWWKH
DGYDQWDJHVRI&$4'$6ZHQWEH\RQGWKHFRGLQJVWRULQJDQGUHWULHYLQJIXQFWLRQVLW
RIIHUHGVHYHUDORWKHUZRUGSURFHVVLQJVNLOOVWKDWDOORIXVZHUHZLOOLQJO\RUXQZLOOLQJO\
DFTXLULQJLQWKHVDQGV IRUH[DPSOHHGLWLQJDQGVHDUFKLQJWH[WVZULWLQJDQG
VDYLQJVKRUWPHPRVSDUDOOHODFFHVVLQJPXOWLSOHWH[WV FRXOGDOVREHQHILWTXDOLWDWLYH
GDWDDQDO\VLV,WEHFDPHLQFUHDVLQJO\FOHDUWKDWPDQ\DVSHFWVRITXDOLWDWLYHGDWDDQDO\VLV
OHQGWKHPVHOYHVWRFRPSXWHUL]DWLRQDQGWKHUHIROORZHGDUDSLGH[SDQVLRQLQWKH
QXPEHUFDSDFLW\IOH[LELOLW\DQGSRSXODULW\RIFRPSXWHUSURJUDPVDYDLODEOHIRUERWK
TXDOLWDWLYHGDWDDQDO\VLVDQGTXDQWLWDWLYHFRQWHQWDQDO\VLVRIWH[WV %D]HOH\
&XUUHQWGHYHORSPHQWVLQ&$4'$6VRIWZDUHLQYROYHWU\LQJWRH[WHQGWKHSURJUDPV¶
FDSDELOLWLHVEH\RQGPHUHO\IXQFWLRQLQJDVDFOHULFDOWRROVRWKDWWKH\FDQDOVREHXVHGLQ
WKHDFWXDODQDO\WLFDOSURFHVV&RQWHQWDQDO\VLVSURJUDPVFDQVHDUFKIRUDQGFRXQWNH\
GRPDLQVSHFLILFZRUGVDQGSKUDVHVDQGFDQHYHQH[DPLQHWKHFRRFFXUUHQFHRIFHUWDLQ
WHUPV7KH\FDQSHUIRUPµDXWRPDWHG¶FRGLQJWKURXJKWH[WVHDUFKDQGFDQSURGXFH
IUHTXHQF\WDEOHVRIWDUJHWXQLWVWKDWFDQEHH[SRUWHGLQDILOHIRUPDWWRDVWDWLVWLFDO
SDFNDJHIRUIROORZXSDQDO\VLV
)XUWKHUVRSKLVWLFDWLRQKDVEHHQUHDFKHGE\SURJUDPVWKDWDOVRIDFLOLWDWHFRGHEDVHG
WKHRU\EXLOGLQJIXQFWLRQVE\DOORZLQJUHVHDUFKHUVWRDQDO\VHDQGYLVXDOO\GLVSOD\
UHODWLRQVKLSVEHWZHHQWKHLGHQWLILHGFRGHVDQGRUJDQL]HWKHPLQWRKLHUDUFKLFDOO\

RUGHUHGµWUHHV¶,QWKLVZD\FRGHVPD\EHUHODWHGWRVXSHURUGLQDWHWKHPHVDQG
FDWHJRULHVZKLFKLVDFRQVFLRXVDWWHPSWWRPDNHWKHSURJUDPVDSSURSULDWHIRUPXOWL
OHYHODQDO\VLVVXFKDVWKHRQHLQYROYHGLQJURXQGHGWKHRU\$V6pURU GHVFULEHV
WKHVRIWZDUH4XDOUXVHYHQXVHVFRPSXWDWLRQDOVWUDWHJLHVWRVXJJHVWQHZFRGHVEDVHGRQ
WKHXVHU¶VSUHYLRXVFRGLQJVWUDWHJLHVDQGSDWWHUQVIRXQGLQWKHGDWD0LOHVDQG
+XEHUPDQ SUHVHQWHGDQH[WHQVLYHUDQJHRIGLIIHUHQWGDWDGLVSOD\PHWKRGVIRU
WKHSXUSRVHRIGHYHORSLQJFRQFHSWVDQGWKHRULHVDQGZHFDQH[SHFWQHZFRPSXWHU
VRIWZDUHWRSURYLGHPRUHDQGPRUHKHOSLQSURGXFLQJVXFKGLVSOD\V,QVXP
GHYHORSPHQWLVIDVWDQGZHDUHOLNHO\WRVHHVHYHUDOQHZEUHDNWKURXJKVZLWKLQWKHQH[W
GHFDGH
:KLFKVRIWZDUHWRFKRRVH"
&XUUHQWO\&$4'$6VRIWZDUHGHYHORSPHQWLVDWDQH[SDQVLRQVWDJHZLWKDQXPEHURI
SDFNDJHVFRPSHWLQJIRUWKHKLJKO\OXFUDWLYHGRPLQDQWSRVLWLRQ ZKLFK6366KDV
PDQDJHGWRWDNHLQWKHTXDQWLWDWLYHPDUNHW 7KHPRVWIUHTXHQWO\PHQWLRQHGVRIWZDUH
QDPHVLQWKHOLWHUDWXUHLQFOXGH18' ,67DQGLWVVXFFHVVRU19LYR7KH(WKQRJUDSK
$WODVWLZLQ0$;DQGLWVVXFFHVVRU0$;TGD4XDOUXV4'$0LQHUDQG
+\SHU5(6($5&+²6pURU DQG/HZLQVDQG6LOYHU LQSUHVV RIIHUJRRG
RYHUYLHZVRIWKHYDULRXVSURJUDPV IRUDFRQGHQVHGVXPPDU\VHH
KWWSFDTGDVVRFVXUUH\DFXN DQGDVLPSOH,QWHUQHWVHDUFKZLOODOVRORFDWHWKH
SURJUDPV¶KRPHSDJHVDVZHOODVYDULRXVUHYLHZVLWHV0\SHUVRQDOIHHOLQJLVWKDWRQH
RIWKHPRVWVHULRXV&$4'$6VRIWZDUHGHYHORSPHQWFRPSDQLHVLQWKHILHOGLVWKH
$XVWUDOLDQ465>4XDOLWDWLYH6ROXWLRQVDQG5HVHDUFK@,QWHUQDWLRQDOZKLFKSURGXFHG
18' ,67LQWKHHDUO\VDQGWKHQIXUWKHUGHYHORSHGLWLQWR19LYR7KHSRSXODULW\
RI19LYRLVRQWKHLQFUHDVHDQGWKHVHQLRUVRIWZDUHGHYHORSHUVLQWKHFRPSDQ\KDYH
DOVREHHQFRQWULEXWLQJWRWKHRUHWLFDODGYDQFHVLQTXDOLWDWLYHUHVHDUFKPHWKRGRORJ\ IRU
H[DPSOH%D]HOH\5LFKDUGV
0HWKRGRORJLFDOPHULWVRI&$4'$6
,EHOLHYHWKDW&$4'$6LVQRWVRPHWKLQJWKDWZHVLPSO\KDYHWRDFFHSWEHFDXVHRIWKH
GLJLWDODJHZHOLYHLQ,DPFRQYLQFHGWKDWDVZLWKZRUGSURFHVVRUVWKHUHZLOOEHDWLPH
ZKHQZHZLOOQRWEHDEOHWRLPDJLQHKRZZHXVHGWRPDQDJHZLWKRXWTXDOLWDWLYHGDWD
DQDO\VLVVRIWZDUH7KHPHWKRGRORJLFDOPHULWVRI&$4'$6DUHPDQLIROG
x 6SHHGRIKDQGOLQJODUJHYROXPHVRIGDWD7KHFRPSXWHUL]DWLRQRIFXPEHUVRPH
FOHULFDOWDVNVUHVXOWVLQFRQVLGHUDEOHJDLQVLQHIILFLHQF\7KLVIUHHVXSUHVHDUFKHU
WLPHDQGKHOSVWRDYRLGGDWDRYHUORDG,QDGGLWLRQWKHIDFWWKDWFRPSXWHUVFDQ
KDQGOHODUJHYROXPHVRIGDWDYHU\TXLFNO\PHDQVWKDWIRUHYHU\TXHVWLRQRU
RSHUDWLRQZHFDQVHDUFKWKHZKROHGDWDEDVHDQGQRWMXVWWKHPRVWREYLRXVRUPRVW
DFFHVVLEOHIROGHUV
x $OPRVWXQOLPLWHGVWRUDJHFDSDFLW\DQGHDV\LQGH[LQJRSWLRQV$UFKLYLQJDOOWKH
YDULRXVGDWDW\SHVFDQEHGRQHLQDVLQJOHVWRUDJHV\VWHPDQGWKHGDWDFDQEH

SUHVHUYHGLQGHILQLWHO\)XUWKHUPRUHFRPSXWHUVRIWZDUHRIIHUVHDV\WRXVHLQGH[LQJ
V\VWHPVIRULGHQWLI\LQJDQGUHWULHYLQJDQ\GDWDWKDWZHKDYHSURFHVVHG
x 0RUHVHQVLWLYHVHFRQGOHYHOFRGLQJ+DPHUVOH\DQG$WNLQVRQ SRLQWRXWWKDW

FRPSXWHUVRIWZDUHDOORZVIRUWKHPXOWLSOHRYHUODSSLQJFRGLQJRIVHJPHQWVZKLFK
PHDQVWKDWZHFDQUHWULHYHYDULRXVFRPELQDWLRQVRIWKHFRGHGWUDQVFULSWH[WUDFWVLQ
DQDSSUR[LPDWLRQRI%RROHDQDOJHEUD IRUH[DPSOHUHWULHYHDOOWKHµ;¶VDQGµ<¶V
RUUHWULHYHDOOWKHµ;¶VEXWQRWWKHµ<¶V 7KLVREYLRXVO\IDFLOLWDWHVD[LDOFRGLQJ VHH
6HFWLRQ DQGVRGRHVWKHIDFWWKDWZHFDQDFFHVVDQGUHYLHZDOOLGHQWLFDOO\
FRGHGGDWDVHJPHQWVZLWKFRQVLGHUDEOHVSHHG
x 3RWHQWLDOIRUQHZDQDO\WLFDOVWUDWHJLHV&$4'$6DOORZVUHVHDUFKHUVWRH[SHULPHQW
ZLWKWKHGDWDIUHHO\ZLWKRXWWKHSK\VLFDODQGWLPHFRQVWUDLQWVRIKDQGOLQJKDUGWR
PDQDJHSLOHVRIDFWXDOWUDQVFULSWV)XUWKHUPRUHWKHUHFHQWO\GHYHORSHGDQDO\WLFDO
WRROVFDQLQVSLUHRUHQDEOHFRPSOHWHO\QHZDQDO\WLFDOVWUDWHJLHV
x ,PSURYHPHQWRIULJRXU7KHFRPSXWHUL]HGSURFHGXUHVDUHOLNHO\WRFUHDWHPRUH
ULJRXULQKDQGOLQJWKHGDWDWKDQWKHVRPHZKDWLPSUHVVLRQLVWLFDQGQRQWUDQVSDUHQW
ZD\WKDWKDVEHHQW\SLFDORIPDQ\VRFDOOHGµTXDOLWDWLYH¶VWXGLHV
x (DV\JHQHUDWLRQRIDQDXGLWWUDLO:HVDZLQVHFWLRQWKDWRQHHIIHFWLYHZD\RI
JHQHUDWLQJFRQILGHQFHLQTXDOLWDWLYHLQTXLULHVLVWROHDYHDQµDXGLWWUDLO¶WKDWDWWHVWV
WRWKHSULQFLSOHGZHOOJURXQGHGDQGWKRURXJKQDWXUHRIWKHUHVHDUFKSURFHVV
LQFOXGLQJWKHLWHUDWLYHPRYHVLQGDWDFROOHFWLRQDQGDQDO\VLVWKHGHYHORSPHQWRI
WKHFRGLQJIUDPHVDQGWKHHPHUJHQFHRIWKHPDLQWKHPHV6XFKDUHFRUGFDQEH
HDVLO\JHQHUDWHGWKURXJK&$4'$6E\VLPSO\VDYLQJVRPHLPSRUWDQWVFUHHQVKRWV
RULQWHULPILOHV )RUGHWDLOVVHH6HFWLRQ
x ,QFUHDVHGOHJLWLPDF\RITXDOLWDWLYHUHVHDUFK,QP\H[SHULHQFHTXDQWLWDWLYHO\
RULHQWHGUHVHDUFKHUVUHJDUGWKHXVHRIFRPSXWHUL]HGSURFHGXUHVDVVRPHVRUWRID
JXDUDQWHHRIWKHTXDOLW\RIWKHSURMHFWDQGWKHUHIRUH&$4'$6FDQEHXVHGDVDQ
DUJXPHQWWRFODLPOHJLWLPDF\RITXDOLWDWLYHLQTXLULHV
x 3RVVLEOHDSSOLFDWLRQLQPL[HGPHWKRGVUHVHDUFK)LQDOO\EHFDXVH&$4'$6
VRIWZDUHFDQEHXVHGWRTXDQWLI\FHUWDLQDVSHFWVRIWKHTXDOLWDWLYHGDWDVHW VHH
6HFWLRQ PDNLQJLWHDV\WROLQN48$/DQG48$1GDWDFRQFHUQLQJD
SDUWLFXODUSDUWLFLSDQWLWOHQGVLWVHOIWREHDSSOLHGLQPL[HGPHWKRGVUHVHDUFK
3RVVLEOHGDQJHUVRI&$4'$6
,NQRZDVHQLRUFROOHDJXHLQRXUILHOGZKRVWLOOZULWHVPRVWRIKLVDUWLFOHVDQGERRNVE\
KDQGVLWWLQJLQDFDIpEHFDXVHKHILQGVWKDWXVLQJDZRUGSURFHVVRUGLVWRUWVKLV
WKRXJKWV²WKHUHLVQRGRXEWWKDWWKHDFWXDOPHWKRGRIZULWLQJRUWKHPHWKRGRIDQDO\VLV
LQRXUFDVHDIIHFWVWKHILQDORXWFRPH,WLVDOVRREYLRXVWKDWPRVWVFKRODUVLQWKHSDVW
KDYHEHHQDEOHWRDGMXVWWRWKHFKDQJLQJWHFKQLFDOFRQGLWLRQVDQGVXFFHVVIXODFDGHPLF
WUHDWLVHVKDYHEHHQZULWWHQERWKE\KDQGDQGXVLQJW\SHZULWHUVRUZRUGSURFHVVRUV)RU
WKLVUHDVRQ,GRQRWEHOLHYHWKDWWKHLQHYLWDEOHVSUHDGRI&$4'$6ZLOOUHVXOWLQDQ

RYHUDOOGHWHULRUDWLRQRITXDOLWDWLYHUHVHDUFKRXWFRPHVWKDWKDVEHHQIHDUHGE\VRPH ,Q
IDFWSHUVRQDOO\,DPRSWLPLVWLFDQGEHOLHYHWKDWDQLPSURYHPHQWLVPRUHOLNHO\ 7KLV
GRHVQRWPHDQWKRXJKWKDWZHGRQRWKDYHWRORRNRXWIRUSRWHQWLDOGDQJHUVWKDWWKH
XVHRI&$4'$6FDUULHV,QWKLVILQDOVHFWLRQ,ZLOOGUDZDWWHQWLRQWRVRPHLVVXHVWKDW,
VHHDVSRWHQWLDOO\SUREOHPDWLF
x 7HFKQRORJLFDOWKLQNLQJ7KHPRVWSURIRXQGFRQFHUQDERXW&$4'$6LVUHODWHGWR
WKHVKLIWLWFDXVHVWRZDUGVWHFKQLTXHVDQGWHFKQRORJ\ZKLFKPD\EHVHHQDV
DQWDJRQLVWLFWRWKHLQWXLWLYHDQGFUHDWLYHSURFHVVRILQWHUSUHWDWLRQWKDWIRUPVWKH
EDVLVRIJRRGTXDOLWDWLYHUHVHDUFK1RPDWWHUKRZFRPSOH[PXOWLOHYHOQHVWHG
VXSHUFRGLQJV\VWHPZHXVHLWZLOOQRWUHSODFHWKHUHVHDUFKHU¶VWKRXJKWIXO
GHOLEHUDWLRQDQGZLOOQRWFRPSHQVDWHIRUWKHODFNRIDQ\RULJLQDOLGHDV,GRQRW
WKLQNWKDWKLWHFKµILVKLQJH[SHGLWLRQV¶ZLOOHYHUEHIUXLWIXOLQTXDOLWDWLYHUHVHDUFK
x 7KHFRGLQJWUDS5HODWHGWRWKHSUHYLRXVSRLQW5LFKDUGV ZDUQVXVRIWKH
GDQJHURIRYHUGRLQJFRGLQJHVSHFLDOO\GHVFULSWLYHRUWRSLFFRGLQJZKHQZHDUH
XQFHUWDLQDERXWZKDWHOVHWRGRDQGQRWKHRU\VHHPVWREHHPHUJLQJIURPWKHGDWD
$VVKHDUJXHVPRUHDQGPRUHFDWHJRULHVDUHXQOLNHO\WRVROYHDQ\WKLQJEXWZLOO
DFWXDOO\SUHYHQWDQ\LGHDVIURPHPHUJLQJUHVXOWLQJLQDµFRGLQJWUDS¶WKDWPD\
µGHOD\RUHYHQGHVWUR\DSURMHFW S
x ,QGLUHFWWKHRUHWLFDOLQIOXHQFH'XII LQSUHVV HPSKDVL]HVWKDWWKHLQFUHDVLQJO\
DGYDQFHGIDFLOLWLHVRI&$4'$6VRIWZDUHWKDWJREH\RQGPHUHO\RIIHULQJFOHULFDO
KHOSDUHQRWWKHRU\QHXWUDOEXWKDYHµLPSOLFLWDVVXPSWLRQVDERXWWKHQDWXUHRIGDWD
IRUH[DPSOHKLHUDUFKLFDOFRQFHSWXDOUHODWLRQVKLSV WKDWPLJKWOHDGUHVHDUFKHUVWR
YLHZGDWDGLIIHUHQWO\WKDQLIWKH\H[DPLQHGLWZLWKRXWWKHFRQVWUDLQWVRIDQ
HVWDEOLVKHGSURJUDP¶
x &RGLQJRQVFUHHQYVFRGLQJRQSDSHU,KDYHIRXQGWKDW,FDQQHYHUGRDVWKRURXJK
DMRESURRIUHDGLQJDWH[WRQVFUHHQDVRQDSDSHUFRS\DQG6HLGPDQ IRXQG
WKHVDPHWKLQJZLWKUHJDUGWRFRGLQJ$VKHDUJXHVWKHUHLVDVLJQLILFDQWGLIIHUHQFH
EHWZHHQZKDWRQHVHHVLQDWH[WSUHVHQWHGRQSDSHUDQGWKHVDPHWH[WVKRZQRQ
VFUHHQDQGWKHUHIRUHKHDFWXDOO\UHFRPPHQGVWKDWZHZRUNZLWKDSDSHUFRS\ILUVW
DQGWUDQVIHUWKHFRGHVWRWKHFRPSXWHUODWHU,VLPSO\GRQRWNQRZKRZJHQHUDOWKLV
GLIIHUHQWLDOLPSDFWRIWKHPHGLXPRIVFUHHQDQGSDSHULVDPRQJVWVFKRODUVDQG
WKHUHIRUHKRZVHULRXVO\ZHVKRXOGWDNHWKLVLVVXH
x 'HFRQWH[WXDOL]HGFRGLQJ,IZHXVHKDUGFRSLHVRIWKHWUDQVFULSWVIRUDQDO\VLV
FRGLQJLVE\GHILQLWLRQFRQWH[WXDOL]HGZLWKLQWKHERG\RIWKHSDUWLFXODUWH[W
+RZHYHUDV<DWHV KDVH[SHULHQFHGWKHHOHFWURQLFFRGLQJSRVVLELOLWLHVWHQG
WRHQFRXUDJHLQFUHDVLQJO\ILQHFDWHJRUL]DWLRQVZLWKWKHUHVHDUFKHUZRUNLQJZLWK
UHWULHYHGFKXQNVWKDWDUHQRORQJHUUHDGLQFRQWH[W7KHGDQJHULVREYLRXV%\
IRFXVLQJRQWKHWUHHVZHFDQORVHVLJKWRIWKHIRUHVW
x 'DQJHURIORVLQJWKHGDWD+DUGFRSLHVDUHFXPEHUVRPHEXWDUHDWOHDVWVROLG
ZKHUHDVHOHFWURQLFILOHVFDQEHZRUU\LQJO\YXOQHUDEOHDQGWUDQVLHQWDOPRVW

HYHU\ERG\NQRZVRIDFDVHZKHQVRPHRQHKDVDFFLGHQWDOO\GHOHWHGVRPHLPSRUWDQW
GRFXPHQWVRUKDVQRWEDFNHGXSKLVKHUGDWDSURSHUO\DQGDIWHUDFRPSXWHUFUDVK
VHYHUDOPRQWKV¶KDUGZRUNKDVGLVDSSHDUHG
x &ROOHFWLQJWRRPXFKGDWD%HFDXVHRIWKHHDVHRIKDQGOLQJFOHULFDOWDVNVWKURXJK
&$4'$6WKHUHPD\EHDGDQJHURIRYHUGRLQJWKHGDWDFROOHFWLRQSKDVHDQGDV
5LFKDUGV ULJKWO\FRQFOXGHVµ%XONGDWDHVSHFLDOO\EXONGDWDWKDWDUHQRW
\HWKDQGOHGDWDOOFDQFUHDWHDPDVVLYHEDUULHUWRWKLQNLQJ¶
x ,QVXIILFLHQWVXSSRUW6pURU PHQWLRQVDSUREOHPWKDWP\VWXGHQWVDQG,KDYH
DOVRH[SHULHQFHGQDPHO\WKDWHYHQWRGD\UHODWLYHO\IHZSHRSOHFDQDFWXDOO\XVH
&$4'$6VRIWZDUHZHOOZKLFKPDNHVLWGLIILFXOWIRUQRYLFHUHVHDUFKHUVWRILQG
VRPHRQHWRDVNIRUKHOS:LWKUHJDUGWRDVSHFLILFWHFKQLFDOTXHVWLRQRQHRIP\
UHVHDUFKVWXGHQWVHYHQKDGWRFRQWDFWWKHFRPSDQ\WKDWSURGXFHGWKHSDUWLFXODU
VRIWZDUHZHXVHG 19LYR WRUHFHLYHDGYLFH2QHVROXWLRQWRWKLVGLIILFXOW\LVWR
MRLQRQHRIWKHVHYHUDO,QWHUQHWIRUXPV²WKH465IRUXP
KWWSIRUXPVTVULQWHUQDWLRQDOFRP IRUH[DPSOHRIIHUVµTXHVWLRQDQGDQVZHU¶DQG
GLVFXVVLRQVHFWLRQV

'DWDDQDO\VLVLQPL[HGPHWKRGV
UHVHDUFK
1HLWKHUTXDQWLWDWLYHQRUTXDOLWDWLYHDUHPRQROLWKLFLWVEXWSHUPHDEOHGRPDLQVLQ
ZKLFKPDQ\VW\OHVKDYHHPHUJHG&RPSOH[LWLHVLQLQGXFWLYHDQGGHGXFWLYH
LWHUDWLYHVWUDWHJLHVDERXQGDQGWKHUHLVDUHQHZDORILQWHUHVWLQERWKDQGUDWKHU
WKDQPRQRHLWKHURUVRIDUDVPHWKRGVDQGPHWKRGRORJ\DUHFRQFHUQHG
*UHHQDQG3UHVWRQ
7KHPRVWFRPPRQSHUFHSWLRQRIPL[HGPHWKRGVUHVHDUFKLVWKDWLWLVDPRGXODUSURFHVV
LQZKLFKTXDOLWDWLYHDQGTXDQWLWDWLYHFRPSRQHQWVDUHFDUULHGRXWHLWKHUFRQFXUUHQWO\RU
VHTXHQWLDOO\$OWKRXJKWKLVSHUFHSWLRQLVE\DQGODUJHWUXHLWDOVRVXJJHVWVWKDWWKH
DQDO\VLVRIWKHGDWDVKRXOGSURFHHGLQGHSHQGHQWO\IRUWKH48$1DQG48$/SKDVHV
DQGPL[LQJVKRXOGRFFXURQO\DWWKHILQDOLQWHUSUHWDWLRQVWDJH7KLVFRQFOXVLRQLVRQO\
SDUWLDOO\WUXH:KLOHLWLVSHUIHFWO\OHJLWLPDWHWROHDYHPL[LQJWRWKHYHU\ILQDOVWDJHVRI
WKHSURMHFWVHYHUDOVFKRODUV IRUH[DPSOH&DUDFHOOLDQG*UHHQ2QZXHJEX]LHDQG
7HGGOLH KDYHDUJXHGWKDWZHFDQDOVRVWDUWLQWHJUDWLQJWKHGDWDDWWKHDQDO\VLV
VWDJHUHVXOWLQJLQZKDWFDQEHFDOOHGµPL[HGPHWKRGVGDWDDQDO\VLV¶,QWKLVFKDSWHU,
SUHVHQWVSHFLILFPL[HGPHWKRGVDQDO\WLFDOVWUDWHJLHVWKDWDWWHPSWWRJHWPRUHRXWRIWKH
GDWDWKDQZRXOGEHSRVVLEOHE\XVLQJ48$/DQG48$1WHFKQLTXHVVHSDUDWHO\
$VDSUHOLPLQDU\OHWPHDGGUHVVDNH\TXHVWLRQ*LYHQWKHIXQGDPHQWDOO\GLIIHUHQW
QDWXUHRI48$/DQG48$1GDWDDQDO\VLV²ZLWKWKHIRUPHUEHLQJODUJHO\LWHUDWLYHDQG
LQGXFWLYHDQGWKHODWWHUEHLQJVHTXHQWLDODQGGHGXFWLYH²LVLWUHDOLVWLFWRDWWHPSWWR
LQWHJUDWHWKHWZRSURFHGXUHV"7KHDQVZHULVWZRIROGLQPDQ\FDVHVLWPD\EHEHWWHUWR
NHHSWKHDQDO\VHVVHSDUDWHDQGRQO\PL[WKH48$/DQG48$1UHVXOWVDWDODWHVWDJHWR
LOOXPLQDWHRUFRUURERUDWHHDFKRWKHU+RZHYHULQVRPHFDVHVWKHLQGXFWLYHGHGXFWLYH
GLVWLQFWLRQLVWRREODFNDQGZKLWHDQGGRHVQRWUHIOHFWZKDWLVUHDOO\JRLQJRQDWWKH
DQDO\WLFDOOHYHO)RUH[DPSOH%D]HOH\ SRLQWVRXWWKDWLQFHUWDLQSURFHGXUHVWKH
EOXUULQJRIWKHERXQGDULHVEHWZHHQ48$/DQG48$1GDWDDQGDQDO\VLVEHFRPHV
PDUNHGO\HYLGHQW6KHPHQWLRQVH[SORUDWRU\IDFWRUDQDO\VLVDVDQH[DPSOH VHH6HFWLRQ
ZKHUHE\DUULYLQJDWDILQDOVROXWLRQLQWHUPVRIH[WUDFWLRQDQGURWDWLRQPHWKRGV
DQGGHWHUPLQLQJWKHQXPEHURIIDFWRUVLQSDUWLFXODULVWKHUHVXOWRIDODUJHO\LQGXFWLYH
SURFHVVGUDZLQJKHDYLO\RQWKHWKHRUHWLFDODQGFRQWH[WXDONQRZOHGJHDQGLQWHUSUHWLYH
DELOLW\RIWKHUHVHDUFKHU7KHVDPHZRXOGDSSO\WRFOXVWHUDQDO\VLV 6HFWLRQ DV
ZHOODQGZHFDQDOVRILQGLQVWDQFHVLQTXDOLWDWLYHUHVHDUFKZKHUHDPRUHGHGXFWLYH
DSSURDFKLVSXUVXHGZLWKWKHWHPSODWHPHWKRGRIFRGLQJ 6HFWLRQ EHLQJDSULPH
H[DPSOH7KXVWKHW\SLFDODVVRFLDWLRQRI48$/UHVHDUFKZLWKDQLQGXFWLYHORJLFRI
LQTXLU\DQG48$1UHVHDUFKZLWKK\SRWKHWLFGHGXFWLRQFDQRIWHQEHUHYHUVHGLQ
SUDFWLFHPDNLQJVRPHW\SHVRI48$/DQG48$1DQDO\VLVJHQXLQHO\FRPSDWLEOHZLWK
HDFKRWKHUVKDULQJDVWKH\GRVRPHXQGHUO\LQJSULQFLSOHV

7KHPRVWFRPPRQLQWHJUDWHGDQDO\WLFDOVWUDWHJ\LVµGDWDWUDQVIRUPDWLRQ¶ZKHUHE\
TXDOLWDWLYHGDWDLVWXUQHGLQWRTXDQWLWDWLYHDQGYLFHYHUVDVR,VWDUWWKHGLVFXVVLRQE\
DGGUHVVLQJWKLVLVVXH7KLVLVIROORZHGE\GHVFULELQJWKUHHRWKHUEURDGDQDO\WLFDO
VWUDWHJLHVµH[WUHPHFDVHDQDO\VLV¶ZKHUHE\RXWOLHUVWKDWDUHLGHQWLILHGLQRQHPHWKRG
DUHIXUWKHUH[DPLQHGXVLQJWKHRWKHUµW\SRORJ\FDWHJRU\GHYHORSPHQW¶ZKHUHE\ZH
GHILQHVXEVWDQWLYHFDWHJRULHVLQRQHPHWKRGDQGXVHWKHVHWRDQDO\VHWKHGDWDIROORZLQJ
WKHRWKHUPHWKRGDQGµPXOWLSOHOHYHODQDO\VLV¶LQZKLFKZHJDLQDGGLWLRQDONQRZOHGJH
RIDVSHFLILFVXEVHWRISDUWLFLSDQWVE\DQDO\VLQJWKHLUUHVSRQVHVLQWKHOLJKWRIGDWD
JDWKHUHGIURPDODUJHUVDPSOHWKDWWKHVHSDUWLFLSDQWVZHUHSDUWRI,WPXVWEHQRWHG
KHUHWKRXJKWKDWWKHWHUPLQRORJ\XVHGLQWKHOLWHUDWXUHWRUHIHUWRWKHVHWHFKQLTXHVLV
UDWKHUPL[HGUHIOHFWLQJWKHXQHVWDEOLVKHGFKDUDFWHURIWKHGRPDLQ7KHFKDSWHULV
FRQFOXGHGZLWKDEULHIGLVFXVVLRQRIKRZPL[HGPHWKRGVDQDO\VLVFDQEHFRPSXWHUL]HG
'DWDWUDQVIRUPDWLRQ
7KHPRVWREYLRXVPHDQVE\ZKLFKTXDOLWDWLYHDQGTXDQWLWDWLYHGDWDFDQEHLQWHJUDWHG
GXULQJDQDO\VLVLVWRWUDQVIRUPRQHGDWDW\SHLQWRWKHRWKHUWKXVDOORZLQJIRUVWDWLVWLFDO
RUWKHPDWLFDQDO\VLVRIERWKGDWDW\SHVWRJHWKHU7KLVSURFHGXUHLVXVXDOO\UHIHUUHGWRDV
µGDWDWUDQVIRUPDWLRQ¶ DOWKRXJK*UHHQHHWDOFDOOVLWµFURVVRYHUWUDFNDQDO\VLV¶
DQGLWVILUVWSURSRQHQWVZHUH0LOHVDQG+XEHUPDQ DQG7DVKDNNRULDQG7HGGOLH
'HSHQGLQJRQWKHGLUHFWLRQRIWKHWUDQVIRUPDWLRQZHFDQWDONDERXW
µTXDQWLWL]LQJGDWD¶WKDWLVWXUQLQJ48$/GDWDLQWR48$1DQGµTXDOLWL]LQJGDWD¶WKDW
LVWXUQLQJ48$1GDWDLQWR48$/7KHIRUPHULQSDUWLFXODUKDVEHHQFRPPRQLQ
UHVHDUFKLQWKHVRFLDOVFLHQFHV
µ4XDQWLWL]LQJ¶GDWD
7KHWHFKQLTXHRIµTXDQWLWL]LQJ¶ZDVILUVWPHQWLRQHGE\0LOHVDQG+XEHUPDQ
WKH\FDOOHGLWµTXDQWLVLQJ¶ EXWLQDFWXDOSUDFWLFHLWKDVEHHQFRPPRQO\XVHGE\
TXDOLWDWLYHUHVHDUFKHUVZKRSURGXFHQXPHULFDOWDEXODWLRQVRIFHUWDLQDVSHFWVRIWKHLU
GDWD7KHWHUPZDVLQWURGXFHGE\7DVKDNNRULDQG7HGGOLH ZKRFRQVLGHUHGLWD
NH\RSHUDWLRQLQPL[HGPHWKRGVGDWDDQDO\VLV
4XDQWLWL]LQJLQYROYHVFRQYHUWLQJTXDOLWDWLYHGDWDLQWRQXPHULFDOFRGHVWKDWFDQEH
IXUWKHUSURFHVVHGVWDWLVWLFDOO\7KXVFHUWDLQVDOLHQWTXDOLWDWLYHWKHPHVDUHQXPHULFDOO\
UHSUHVHQWHGHLWKHULQVFRUHV IRUH[DPSOHIUHTXHQF\VFRUHVRIKRZPDQ\WLPHVWKH
WKHPHLVPHQWLRQHGLQWKHVDPSOHRULQDVLQJOHLQGLYLGXDO¶VUHVSRQVHV RUVFDOHV IRU
H[DPSOHHYDOXDWLQJWKHLQWHQVLW\RUGHJUHHRUZHLJKWRIDQDUUDWLYHDFFRXQWVXFKDVDQ
LQWHUYLHZHH¶VGHVFULSWLRQRIKLVKHUDWWLWXGHVWRZDUGVVD\WKHOHDUQLQJHQYLURQPHQW $
UHOLDEOHZD\RISURGXFLQJVFDOHVFRUHVRIWKHODWWHUW\SHLVWRDVNDSDQHORIH[SHUW
MXGJHVWRUDWHWKHGDWDRQDFRQWLQXXPDQGWKHQWDNHWKHPHDQUDWLQJDVWKHVFDOHVFRUH
$OWKRXJKWHFKQLFDOO\VSHDNLQJDQ\TXDOLWDWLYHWKHPHFDQEHTXDQWLWL]HGE\VLPSO\
DVVLJQLQJRUWRLWGHSHQGLQJZKHWKHULWDSSHDUVLQDVSHFLILFGDWDVHWRUQRWDOUHDG\
0LOHVDQG+XEHUPDQ ZDUQHGXVWKDWTXDQWLWL]LQJGDWDµLVQRWDQRSHUDWLRQWREH

WDNHQOLJKWO\¶ S &RXQWLQJLPSURYHVWKHTXDOLW\RIWKHDQDO\VLVRQO\LQFDVHV
ZKHQWKHQXPHULFUHVXOWVDUHXVHGSURSHUO\)RUH[DPSOHWKHFRPPRQSURFHGXUHRI
FRXQWLQJKRZPDQ\RIRXUSDUWLFLSDQWVPHQWLRQDFHUWDLQWRSLFDQGWKHQFRPSDULQJWKH
UHVXOWVVD\LQJWKDW7RSLF;LVPRUHLPSRUWDQWWKDQ7RSLF<EHFDXVHPRUHUHVSRQGHQWV
PHQWLRQHGLWLVXVXDOO\HUURQHRXVEHFDXVHDTXDOLWDWLYHVDPSOHLVQRWPHDQWWREH
UHSUHVHQWDWLYHDQGWKHUHIRUHVXESURSRUWLRQVRILWGRQRWUHIOHFWDQ\UHDOWHQGHQFLHV
+RZHYHULIZHEHOLHYHWKDWRXUVDPSOLQJKDVSURGXFHGDJURXSRISDUWLFLSDQWVZKR
PD\EHVHHQDVUHSUHVHQWDWLYHRIDFHUWDLQSRSXODWLRQWKHQZHFDQPDNHFRPSDULVRQV
EXWZHVKRXOGWHVWWKHVLJQLILFDQFHRIWKHUHVXOWVVWDWLVWLFDOO\IRUH[DPSOHE\HPSOR\LQJ
DWWHVWRUD&KLVTXDUHWHVW,IZHREWDLQDVWDWLVWLFDOO\VLJQLILFDQWUHVXOWWKDWZRXOGEHD
SRZHUIXOFRQILUPDWLRQRIWKHYDOLGLW\RIRXULQIHUHQFH
2QFHZHKDYHTXDQWLWL]HGVRPHGDWDZHFDQH[SRUWWKHQXPHULFDOFRGHVLQWRD
VWDWLVWLFDOSDFNDJHVRWKDWLWFDQEHVXEPLWWHGWRYDULRXVTXDQWLWDWLYHDQDO\VHVUHODWLQJ
LWWRRWKHUTXDQWLWDWLYHPHDVXUHV,QPRVWFDVHVQRQSDUDPHWULFWHVWV IRUH[DPSOH&KL
VTXDUHWHVWRUUDQNRUGHUFRUUHODWLRQVHH6HFWLRQ ZLOOEHPRUHDSSURSULDWHIRUWKH
VWDWLVWLFDODQDO\VLVWKDQSDUDPHWULFVWDWLVWLFVEHFDXVHRIWKHOLPLWHGVDPSOHVL]HWKDWPD\
QRWEHQRUPDOO\GLVWULEXWHGRUEHFDXVHWKHVFDOHVFRUHVZHKDYHIRUPHGDUHRUGLQDO
UDWKHUWKDQLQWHUYDO 6HH6HFWLRQ
2QZXHJEX]LHDQG/HHFK UHFRPPHQGDQLQWHUHVWLQJEXWVRPHZKDW
FRQWURYHUVLDODSSURDFKWRSURFHVVLQJTXDQWLWL]HGTXDOLWDWLYHGDWDWKH\DUJXHWKDWLIZH
WXUQHDFKFRGLQJFDWHJRU\LQWRDQXPHULFDOVFRUHDQGWKHQIDFWRUDQDO\VHWKHUHVXOWV VHH
6HFWLRQ ZHPD\REWDLQµPHWDWKHPHV¶WKDWVXEVXPHWKHRULJLQDOWKHPHVWKHUHE\
GHVFULELQJWKHUHODWLRQVKLSDPRQJWKHVHWKHPHV7KHSUREOHPZLWKWKLVPHWKRGLVWKDW
IDFWRUDQDO\VLVUHTXLUHVDUHODWLYHO\ODUJHVDPSOHVL]HSUHIHUDEO\RYHUSDUWLFLSDQWV
ZKLFKLVRQO\UDUHO\DFKLHYHGLQTXDOLWDWLYHVWXGLHVGXHWRWKHKXJHZRUNORDGLQYROYHG
2QWKHRWKHUKDQGDVPHQWLRQHGHDUOLHUWKHXQGHUO\LQJORJLFRIIDFWRUDQDO\VLVLV
ODUJHO\LQGXFWLYHDQGWKHUHIRUHVXFKDQDQDO\VLVZRXOGILWWKHRYHUDOOWKUXVWRID
TXDOLWDWLYHVWXG\:HFDQDOVRFRQVLGHUVXEPLWWLQJTXDQWLWL]HGGDWDWRFOXVWHUDQDO\VLV
WRLGHQWLI\VXEJURXSVLQWKHVDPSOHZKRVKDUHVLPLODUFRPELQDWLRQVRIVRPHRIWKHLU
YLHZVRUFKDUDFWHULVWLFV
µ4XDOLWL]LQJ¶GDWD
µ4XDOLWL]LQJ¶GDWDWKDWLVLQWHUSUHWLQJTXDQWLWDWLYHGDWDZLWKLQDTXDOLWDWLYHFRQWH[WLV
OHVVFRPPRQLQSXEOLVKHGUHVHDUFKEHFDXVHTXDQWLWDWLYHO\RULHQWDWHGUHVHDUFKHUVUDUHO\
IHHOWKHQHHGWRDGGDTXDOLWDWLYHFRPSRQHQWWRWKHLUVWXGLHV+RZHYHUWKLVDSSURDFKLV
IUHTXHQWLVVPDOOVFDOHVWXGHQWHVVD\VDQGGLVVHUWDWLRQVZKHQVWXGHQWVFROOHFW
TXDQWLWDWLYH W\SLFDOO\TXHVWLRQQDLUH GDWDEXWWKHQGHFLGHWRDQDO\VHWKHVFRUHVQRQ
TXDQWLWDWLYHO\XVXDOO\EHFDXVHWKH\FDQQRWKDQGOHVWDWLVWLFV7KLVSUDFWLFHLVLQPRVW
FDVHVDQLQDSSURSULDWHXVHRIWKHTXDOLWL]LQJSURFHVV
7KHWZRPRVWFRPPRQ DQGOHJLWLPDWH XVHVRITXDOLWL]HGGDWDDUH D XWLOL]LQJ

TXDQWLWDWLYHEDFNJURXQGLQIRUPDWLRQLQDTXDOLWDWLYHDQDO\VLVDQG E µQDUUDWLYHSURILOH

IRUPDWLRQ¶$W\SLFDOH[DPSOHRIWKHILUVWZRXOGLQYROYHWKHUHVHDUFKHUKDQGLQJRXWD
VKRUWEDFNJURXQGTXHVWLRQQDLUH PHDVXULQJIRUH[DPSOHGHPRJUDSKLFLQIRUPDWLRQ
VXFKDVDJHDQGJHQGHU WRDOOKLVKHULQWHUYLHZHHVDQGWKHQLQWHJUDWLQJWKHDQVZHUVLQWR
DTXDOLWDWLYHDFFRXQWWRLQIRUPWKHLQWHUSUHWDWLRQRIWKHGDWD7KHDSSURDFKFDQEH
H[WHQGHGE\LQFOXGLQJRWKHUVRUWVRITXHVWLRQVDERXWDWWLWXGHVEHOLHIVKDELWVHWF$
UHWURVSHFWLYHLQWHUYLHZ VHH6HFWLRQVDQG IRFXVLQJRQTXHVWLRQQDLUHVFRUHV
LQHIIHFWSURYLGHVLPPHGLDWHTXDOLWL]LQJRIWKHTXHVWLRQQDLUHUHVXOWV
µ1DUUDWLYHSURILOHV¶RIFHUWDLQSDUWLFLSDQWVEDVHGRQTXDQWLWDWLYHLQIRUPDWLRQDUH
VLPLODUWRLQFOXGLQJTXDQWLWDWLYHEDFNJURXQGLQIRUPDWLRQLQWKHVWXG\EXWWKHSXUSRVH
RIWKLVSURFHVVLVQRWVRPXFKWKHFRQWH[WXDOL]DWLRQRIWKHUHVSRQGHQWV¶DFFRXQW LH
H[DPLQLQJWKHGDWDLQWKHOLJKWRIWKHDGGLWLRQDOTXDQWLWDWLYHLQIRUPDWLRQ DVWRGHYHORS
DIXOOHUXQGHUVWDQGLQJRIDSDUWLFXODUSHUVRQDJURXSRUDQLQVWLWXWLRQ IRUH[DPSOHLQD
FDVHVWXG\ 3URILOHVDUHDQDFFHVVLEOHDQGSRZHUIXOPHDQVRIFRPPXQLFDWLQJPHDQLQJ
DQGE\LQFOXGLQJTXDOLWL]HGVXPPDULHVRIVHYHUDOTXDQWLWDWLYHSLHFHVRILQIRUPDWLRQZH
FDQPDNHWKHSURILOHPRUHFRQYLQFLQJDQGYDOLG
'DWDPHUJLQJ
$VSHFLDOFDVHRIGDWDWUDQVIRUPDWLRQLVZKHQZHQRWRQO\WXUQRQHGDWDW\SHLQWRWKH
RWKHUEXWPHUJHWRJHWKHUGLIIHUHQWW\SHVRIGDWD E\ILUVWFRQYHUWLQJRQHW\SH DQGWKXV
FUHDWHDQHZYDULDEOH)RUH[DPSOHLQRUGHUWRGHILQHµPRQLWRURYHUXVHUV¶ .UDVKHQ
WKDWLVODQJXDJHOHDUQHUVZKRZRUU\DERXWDQGUHSDLUWKHLU/XWWHUDQFHVWRR
PXFKZHFDQXVHDOHDUQLQJVW\OHTXHVWLRQQDLUHIRFXVLQJRQDQDO\WLFVW\OH 48$1
DQGZHFDQFRPELQHWKLVPHDVXUHZLWKWKHOHDUQHUVVHOIUHSRUWVDERXWKRZWKH\
SHUFHLYHDQGKDQGOHPLVWDNHV 48$/ 7KHFRPELQDWLRQFDQIRUPWKHEDVLVRID
FRPSRVLWHTXDQWLWDWLYHPHDVXUHRUDW\SRORJ\²VHH
([WUHPHFDVHDQDO\VLV
:HKDYHVHHQLQ6HFWLRQWKDWµRXWOLHUV¶WKDWLVH[WUHPHYDOXHVWKDWDUH
LQFRQVLVWHQWZLWKWKHUHVWRIWKHGDWDVHWFDQFDXVHKDYRFLQVWDWLVWLFV+RZHYHUEHFDXVH
RIWKHLURXWRIWKHRUGLQDU\IHDWXUHVRXWOLHUSDUWLFLSDQWVFDQDOVREHVHHQDVSDUWLFXODUO\
LQWHUHVWLQJFDVHVZKRVHDQDO\VLVPD\VKHGOLJKWRQEURDGHULVVXHVµ([WUHPHFDVH
DQDO\VLV¶LVDLPHGDWH[DPLQLQJWKHYDOXHRIVXFKXQXVXDOFDVHVE\LGHQWLI\LQJWKHPE\
RQHPHWKRGDQGIXUWKHUH[DPLQLQJWKHPXVLQJWKHRWKHU7KHXVXDOSUDFWLFHLQYROYHV
LGHQWLI\LQJRXWOLHUVLQTXDQWLWDWLYHUHVHDUFKDQGWKHQVXEPLWWLQJWKHPWRIXUWKHUPRUH
LQWHQVLYHTXDOLWDWLYHDQDO\VLVWRXQGHUVWDQGWKHUHDVRQIRUWKHGHYLDQFH+RZHYHUZH
FDQDOVRFRQFHLYHRIDQDQDO\WLFDOVHTXHQFHZKHUHE\ZHEHFRPHDZDUHRIDQXQXVXDO
SDUWLFLSDQWGXULQJWKHTXDOLWDWLYHDQDO\VLVDQGWKHQH[DPLQHKRZWKHTXDQWLWDWLYHGDWD
DYDLODEOHDERXWWKHSDUWLFXODUUHVSRQGHQWIXUWKHUVRXUXQGHUVWDQGLQJ
7\SRORJ\FDWHJRU\GHYHORSPHQW

,QµW\SRORJ\FDWHJRU\GHYHORSPHQW¶ZHDQDO\VHRQHGDWDW\SHDQGHVWDEOLVKVRPH
VXEVWDQWLYHFDWHJRULHVRUWKHPHV7KHVHFDWHJRULHVDUHWKHQDSSOLHGLQWKHDQDO\VLVRI
WKHRWKHUW\SHRIGDWD:HFDQXVHWKHFDWHJRULHVWRGLYLGHWKHVDPSOHLQWRVXEVDPSOHV
WKDWLVFODVVLI\LQGLYLGXDOVLQWRGLIIHUHQWW\SHVZKLFKZRXOGEHDFDVHRIµW\SRORJ\
GHYHORSPHQW¶,IWKHVRUWLQJFDWHJRU\ZDVRULJLQDOO\TXDOLWDWLYHWKHQZHFRXOGVHH
ZKHWKHUWKHJURXSLQJFDQEHVXEVWDQWLDWHGE\WKHTXDQWLWDWLYHGDWDDQGZHFDQDOVR
LGHQWLI\WKRVHYDULDEOHVWKDWGLVFULPLQDWHZHOOEHWZHHQWKHWZRJURXSV)RUH[DPSOH
EDVHGRQWKHTXDOLWDWLYHGDWDZHPLJKWLGHQWLI\WZRW\SHVRIOHDUQHUVWKRVHZKRSD\
PRUHDWWHQWLRQWRFRPPXQLFDWLYHIOXHQF\WKDQJUDPPDWLFDODFFXUDF\DQGYLFHYHUVD,Q
WKHVXEVHTXHQWTXDQWLWDWLYHDQDO\VLVZHFDQVXEVWDQWLDWHWKLVILQGLQJE\FRPSDULQJWKH
ODQJXDJHSURILFLHQF\WHVWVFRUHVRIWKHOHDUQHUVLQWKHWZRJURXSVDQGZHFDQDOVR
H[DPLQHZKHWKHUWKHWZRJURXSVVKRZDQ\GLIIHUHQFHRQRWKHUYDULDEOHVIRUH[DPSOH
ODQJXDJHDSWLWXGHRUOHDUQLQJVW\OH
,IWKHVRUWLQJFDWHJRU\LVTXDQWLWDWLYHDQGLWZDVGHULYHGIURPDVWDWLVWLFDOO\
VLJQLILFDQWJURXSGLIIHUHQFHWKLVZRXOGDOORZXVWRWUDQVIHUWKHJURXSLQJSULQFLSOHWR
WKHDQDO\VLVRIWKHTXDOLWDWLYHGDWDDQGWRFRPSDUHWKHVXEJURXSVWKDWDUHIRUPHGXVLQJ
WKHVRUWLQJFDWHJRU\²VRPHWKLQJZKLFKZRXOGQRUPDOO\QRWEHOHJLWLPDWHLQTXDOLWDWLYH
UHVHDUFK)RUH[DPSOHLIZHLGHQWLILHGWZRJURXSVRIOHDUQHUVUHSUHVHQWLQJWZRW\SHVRI
PRWLYDWLRQZHFRXOGH[DPLQHKRZWKLVGLVWLQFWLRQLVUHIOHFWHGLQWKHLUTXDOLWDWLYH
DFFRXQWV
&DUDFHOOLDQG*UHHQ IXUWKHUPHQWLRQWKDWDQDGGLWLRQDOLPSRUWDQWIHDWXUHRI

WKHW\SRORJ\GHYHORSPHQWVWUDWHJ\LVLWVSRWHQWLDOIRULWHUDWLRQ$VWKH\DUJXHD
W\SRORJ\FRXOGEHFUHDWHGIURPRQHGDWDW\SHDQGDSSOLHGWRWKHDQDO\VLVRIWKHRWKHU
GDWDW\SHWKHUHVXOWVRIZKLFKFRXOGLQWXUQEHXVHGWRUHILQHDQGHODERUDWHWKH
W\SRORJ\7KLVLPSURYHGW\SRORJ\FRXOGWKHQEHUHDSSOLHGLQVXEVHTXHQWDQDO\VHVRI
HLWKHUGDWDW\SHDQGVRIRUWKIXUWKHUH[SOLFDWLQJWKHLQLWLDODQDO\VHV
$YDULDWLRQRIWKHW\SRORJ\GHYHORSPHQWVWUDWHJ\LVµFDWHJRU\GHYHORSPHQW¶,QWKLV
FDVHTXDQWLWDWLYHGDWDLVXVHGWRLGHQWLI\VDOLHQWYDULDEOHVUHOHYDQWWRWKHWDUJHW
SKHQRPHQRQDQGWKHVHFDWHJRULHVDUHXVHGVXEVHTXHQWO\IRUFRGLQJWKHTXDOLWDWLYH
GDWD7KLVVWUDWHJ\FRXOGZRUNYHU\ZHOOZLWKWKHµWHPSODWHDSSURDFK¶RIFRGLQJ
GHVFULEHGLQWKHSUHYLRXVFKDSWHU 6HFWLRQ
0XOWLSOHOHYHODQDO\VLV
µ0XOWLSOHOHYHODQDO\VLV¶LVUHOHYDQWLIZHKDYHTXDQWLWDWLYH IRUH[DPSOHVXUYH\ GDWD
DERXWDODUJHUJURXSDQGWKHQVXEPLWDVXEVDPSOHRIWKLVJURXSWRIXUWKHULQWHQVLYH
TXDOLWDWLYHLQYHVWLJDWLRQ IRUH[DPSOHE\PHDQVRIDQLQWHUYLHZVWXG\ ,QWKLVZD\ZH
KDYHWZROHYHOVRIHPEHGGHGGDWDZKLFKZHFDQLQWHJUDWHWRJRRGHIIHFW)RUH[DPSOH
ZHFDQXVHWKHODUJHUVDPSOHDVDQRUPJURXSDJDLQVWZKLFKWKHVXEVDPSOH¶V
FKDUDFWHULVWLFVDQGSHUIRUPDQFHUHVXOWVFDQEHHYDOXDWHG,Q'|UQ\HLHWDO ZH
IROORZHGWKLVVWUDWHJ\E\LQWHUSUHWLQJYDULRXVDSWLWXGHDQGPRWLYDWLRQVFRUHVRI
SDUWLFLSDQWVRIDQLQWHUYLHZVWXG\LQWKHOLJKWRIWKHUHVXOWVRIDODUJHUVXUYH\RI

VWXGHQWV7RPDNHWKHTXDQWLWDWLYHUHVXOWVPRUHPHDQLQJIXOLQWKLVTXDOLWDWLYHVWXG\ZH
FRPSXWHGVWDQGDUGL]HGVFRUHV VHH6HFWLRQ ZKLFKLQGLFDWHGKRZRXU
LQWHUYLHZHHV¶UHVXOWVGLIIHUHGIURPWKHPHDQVFRUHVREWDLQHGLQWKHODUJHUJURXS7KXV
PXOWLSOHOHYHODQDO\VLVFRPELQHVTXDOLWDWLYHUHVHDUFKZLWKTXDQWLWDWLYHPHDVXUHVRI
SRSXODWLRQV7KLVVWUDWHJ\FDQEHXVHGHYHQZKHQWKHTXDOLWDWLYHFRPSRQHQWLVWKH
VWXG\RIDVLQJOHFDVHWRPDNHODUJHUFODLPVRIJHQHUDOL]DELOLW\
&RPSXWHUDLGHGPL[HGPHWKRGVGDWDDQDO\VLV
'DWDDQDO\VLVVRIWZDUHKDVWUDGLWLRQDOO\IRFXVHGRQHLWKHUTXDQWLWDWLYHRUTXDOLWDWLYH
UHVXOWV+RZHYHUDV%D]HOH\ GHVFULEHVWKHERXQGDULHVEHWZHHQQXPHULFDOO\DQG
WH[WXDOO\EDVHGUHVHDUFKDUHEHFRPLQJOHVVGLVWLQFWDQGUHFHQWYHUVLRQVRIVHYHUDO
VRIWZDUHSDFNDJHVKDYHRSHQHGXSQHZSRVVLELOLWLHVIRUZRUNLQJZLWKPL[HGGDWDW\SHV
0RVWTXDOLWDWLYHGDWDDQDO\VLVSURJUDPVFDQQRZLPSRUWDQGLQFRUSRUDWHTXDQWLWDWLYH
GDWDLQWRDTXDOLWDWLYHDQDO\VLVDQGPDQ\VXSSRUWTXDQWLWL]LQJTXDOLWDWLYHGDWDDQG
H[SRUWLQJLWWRDVWDWLVWLFDOSDFNDJH%D]HOH\FDOOVWKHVHSURFHGXUHVWKHµIXVLQJ¶RI
DQDO\VLVDQGWKLVWHUPH[SUHVVHVZHOOWKHQHZTXDOLW\RIPL[HGPHWKRGDQDO\VLVWKDW
FRPSXWHUDLGHGDQDO\VLVFDQRIIHU
6WDWLVWLFDOSDFNDJHVKDYHDOVRUHVSRQGHGWRXVHUGHPDQGDQGHYHQ6366KDV
UHOHDVHGDWH[WDQDO\WLFDODGGRQPRGXOHµ7H[W6PDUW¶WRIDFLOLWDWHWKHSURFHVVLQJRI
UHVSRQVHVWRRSHQHQGHGLWHPVLQTXHVWLRQQDLUHV7KHZHOONQRZQTXHVWLRQQDLUHVXUYH\
VRIWZDUH6SKLQ[6XUYH\KDVDOVRGHYHORSHGDµ/H[LFD¶YHUVLRQZKLFKRIIHUVDGYDQFHG
WH[WDQDO\VLVIHDWXUHVWKDWLQFOXGHFHUWDLQTXDOLWDWLYHFRGLQJIXQFWLRQV%RWKVRIWZDUH
SURJUDPVVXSSRUWWKHLQWHJUDWHGVWDWLVWLFDODQDO\VLVRIWKHQHZO\IRUPXODWHGTXDQWLWL]HG
YDULDEOHVDQGWKHH[LVWLQJTXDQWLWDWLYHGDWDVHW

3DUW5HSRUWLQJUHVHDUFK
UHVXOWV
:ULWLQJDTXDQWLWDWLYHUHSRUW
$FDGHPLFZULWLQJLVDQ³DUHQDIRUVWUXJJOH´LQZKLFKVWXGHQWVDQGUHVHDUFKHUV
FDQILQGLWKDUGWRDFKLHYHSHUVRQDOSRZHUDQGYRLFH
+ROOLGD\
+DYLQJFROOHFWHGDQGDQDO\VHGWKHGDWDWKHUHVHDUFKHU¶VMRELVIDUIURPEHLQJFRPSOHWH
EHFDXVHWKHUHVXOWVKDYHWREHZULWWHQXSDQGGLVVHPLQDWHG5HVHDUFKLVLQKHUHQWO\D
VRFLDODFWLYLW\DQGDV2UWHJD ULJKWO\SRLQWVRXWµ7KHYDOXHRIUHVHDUFKLVWR
EHMXGJHGE\LWVVRFLDOXWLOLW\¶$VVKHDUJXHVDQ\UHVHDUFKILHOGLQWKHVRFLDOVFLHQFHV
LQFOXGLQJDSSOLHGOLQJXLVWLFVKDVDVLWVXOWLPDWHJRDOWKHLPSURYHPHQWRIKXPDQOLIH
DQGWKHUHIRUHWKHYDOXHRIDSSOLHGOLQJXLVWLFVUHVHDUFKVKRXOGEHMXGJHGQRWRQO\E\WKH
PHWKRGRORJLFDOULJRXULWGLVSOD\VEXWDOVRµXOWLPDWHO\RQWKHEDVLVRILWVSRWHQWLDOIRU
SRVLWLYHLPSDFWRQVRFLHWDODQGHGXFDWLRQDOSUREOHPV¶ S *LYHQWKDWWKHµDSSOLHG¶
QDWXUHRIRXUILHOGLVHYHQPDUNHGLQLWVQDPH,EHOLHYHWKDW2UWHJD¶VPHVVDJH
UHVRQDWHVZLWKPDQ\VFKRODUV
,IZHDJUHHRQWKHLPSRUWDQFHRIWKHVRFLDOYDOXHRIUHVHDUFKZHPXVWWDNHWKHQH[W
ORJLFDOVWHSDQGDFFHSWWKDWVKDULQJUHVHDUFKUHVXOWVZLWKWKHZLGHUFRPPXQLW\LVD
FHQWUDOSDUWRIDUHVHDUFKHU¶VMREGHVFULSWLRQ7KDWLVFRPPXQLFDWLQJZLWKRXUDXGLHQFH
LVDPDMRUIDFHWRIDUHVHDUFKHU¶VUHVSRQVLELOLWLHV,WLVIRUWKLVUHDVRQWKDWWKH(WKLFDO
6WDQGDUGVRIWKH$PHULFDQ(GXFDWLRQDO5HVHDUFK$VVRFLDWLRQ $(5$ LQFOXGH
DPRQJVWWKHUHVHDUFKHUV¶UHVSRQVLELOLWLHVWRWKHILHOGWKDWUHVHDUFKHUVVKRXOGµDWWHPSWWR
UHSRUWWKHLUILQGLQJVWRDOOUHOHYDQWVWDNHKROGHUV¶ *XLGLQJ6WDQGDUGV, DQGWKDWWKHLU
µUHSRUWVWRWKHSXEOLFVKRXOGEHZULWWHQVWUDLJKWIRUZDUGO\WRFRPPXQLFDWHWKHSUDFWLFDO
VLJQLILFDQFHIRUSROLF\¶ *XLGLQJ6WDQGDUGV, 7KLVODWWHUUHFRPPHQGDWLRQLVODWHU
UHSHDWHGµ(GXFDWLRQDOUHVHDUFKHUVVKRXOGFRPPXQLFDWHWKHLUILQGLQJVDQGWKHSUDFWLFDO
VLJQLILFDQFHRIWKHLUUHVHDUFKLQFOHDUVWUDLJKWIRUZDUGDQGDSSURSULDWHODQJXDJHWR
UHOHYDQWUHVHDUFKSRSXODWLRQVLQVWLWXWLRQDOUHSUHVHQWDWLYHVDQGRWKHUVWDNHKROGHUV¶
*XLGLQJ6WDQGDUGV,,
7KLVDQGWKHQH[WFKDSWHUGLVFXVVYDULRXVDVSHFWVRIWKHDFDGHPLFZULWLQJSURFHVVLQ
TXDQWLWDWLYHTXDOLWDWLYHDQGPL[HGPHWKRGVSURMHFWV6SDFHFRQVWUDLQWVSUHYHQWWKLV
ERRNIURPRIIHULQJGHWDLOHGJXLGHOLQHVRQKRZWRDFKLHYHJRRGDFDGHPLFZULWLQJLQ

JHQHUDOEXWIRUWXQDWHO\WKHUHDUHPDQ\XVHIXOWH[WVIRUWKLVSXUSRVH,QVWHDGRXUIRFXV
ZLOOEHRQKLJKOLJKWLQJWKRVHIDFHWVRIDFDGHPLFZULWLQJWKDWDUHVSHFLILFDOO\DVVRFLDWHG
ZLWKWKHSDUWLFXODUUHVHDUFKDSSURDFKZHKDYHDGRSWHGWKDWLVZKLFKVKRZ
SDUDGLJPDWLFYDULDWLRQ)RUH[DPSOHWKHUHZLOOQRWEHPXFKGLVFXVVLRQRQXVLQJ
UHJLVWHUDSSURSULDWHYRFDEXODU\EXWZHZLOODQDO\VHLQVRPHGHWDLOWKHGLIIHUHQWUROHRI
WKHOLWHUDWXUHUHYLHZLQTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFK
7ZRIXQFWLRQVRIDFDGHPLFZULWLQJ
7KHPRVWREYLRXVIXQFWLRQRIDFDGHPLFZULWLQJLVWRFRPPXQLFDWHRXUUHVXOWVWRDQ
DXGLHQFH$FFRUGLQJO\UHVHDUFKHUVQHHGWRGRWKHLUEHVWWRPDNHWKHLUUHSRUWV
DFFHVVLEOH7KHIROORZLQJWKUHHSULQFLSOHVKDYHJXLGHGP\RZQZULWLQJRYHUWKH\HDUV
x 5HDGHUIULHQGOLQHVV$Q\ZULWLQJ²LQFOXGLQJTXDQWLWDWLYHUHVHDUFKUHSRUWV²VKRXOG
EHXOWLPDWHO\JXLGHGE\WKHSULQFLSOHRIUHDGHUIULHQGOLQHVVDQGZULWHUVQHHGWR
PDNHDFRQVFLRXVHIIRUWWRSUHVHQWWKHFRQWHQWLQDZD\WKDWPD[LPL]HVWKH
OLNHOLKRRGRIVXFFHVVIXOFRPSUHKHQVLRQLQWKHUHDGHUV7KHUHIRUHLWLVP\SHUVRQDO
EHOLHIWKDWZKHQHYHUZHKDYHDGLOHPPDDERXWKRZWRSUHVHQWIRUPDWRUIRUPXODWH
VRPHWKLQJDXGLHQFHVHQVLWLYLW\VKRXOGEHRXUPDLQJXLGLQJSULQFLSOH
x $FFHVVLEOHODQJXDJHDQGVW\OH7KHUHLVDP\WKDPRQJVWQRYLFHUHVHDUFKHUVWKDWDQ\
VHULRXVDFDGHPLFZULWLQJLVE\GHILQLWLRQUDWKHUFRQYROXWHGODQJXDJHZLVH
$OWKRXJKWKHUHLVLQGHHGDWHQGHQF\LQDFDGHPLDWRWU\DQGPDNHRXUZRUNDSSHDU
PRUHVFKRODUO\E\RYHUXVLQJMDUJRQDQGXVLQJFRPSOH[SKUDVHVZKHUHVLPSOHRQHV
ZRXOGGRZHVKRXOGQRWHWKDWVRPHRIWKHPRVWUHQRZQHGVFKRODUVLQWKHVRFLDO
VFLHQFHVDFWXDOO\ZULWHYHU\FOHDUO\$OWKRXJKVRPHWLPHVZHGRQHHGWRH[SUHVV
QXDQFHVLQDVXEWOHZD\P\SHUVRQDOIHHOLQJLVWKDWPRVWFDVHVRILPSHQHWUDEOH
ODQJXDJHDFWXDOO\UHIOHFWZULWHUXQFHUWDLQWLHV
x 7HOOLQJDVWRU\7HOOLQJDVWRU\LVE\IDUWKHPRVWHIIHFWLYHZD\RIJHWWLQJWKURXJKWR
DXGLHQFHVDQGLIWKHFRQWHQWRIDSDUWLFXODUZRUNLVFHQWUHGDURXQGDFRKHUHQWDQG
LQWHUHVWLQJVWRU\OLQHHYHQFRPSOH[PDWHULDOFDQEHPDGHHQMR\DEOHDQGGLJHVWLEOH
,ZDVSOHDVHGWRILQGVXSSRUWIRUDYLYLGSHUVRQDOL]HGIRUPRISUHVHQWDWLRQLQWKH
VW\OHPDQXDORIWKH$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQZKLFKLVDFROOHFWLRQRI
VRPHRIWKHPRVWVFKRODUO\UXOHVLQDFDGHPLFZULWLQJ
$OWKRXJKVFLHQWLILFZULWLQJGLIIHUVLQIRUPIURPOLWHUDU\ZULWLQJLWQHHGQRW
DQGVKRXOGQRWODFNVW\OHRUEHGXOO,QGHVFULELQJ\RXUUHVHDUFKSUHVHQWWKH
LGHDVDQGILQGLQJVGLUHFWO\EXWDLPIRUDQLQWHUHVWLQJDQGFRPSHOOLQJ
PDQQHUWKDWUHIOHFWV\RXULQYROYHPHQWZLWKWKHSUREOHP $3$
%HVLGHVEHLQJDFRPPXQLFDWLYHDFWDFDGHPLFZULWLQJDOVRSOD\VDVHFRQGLPSRUWDQW
UROHLWLVSDUWRIWKHSURFHVVRILQTXLU\:HKDYHVHHQLQ&KDSWHU 6HFWLRQ
WKDWZULWLQJPHPRVLVDQDQDO\WLFDOWRROLQJURZLQJGLVWLOOLQJDQGVWUXFWXULQJRXULGHDV
DQG,KDYHIRXQGWKDWWKHVDPHWKLQJLVWUXHDERXWDFDGHPLFZULWLQJRQDEURDGHUOHYHO
RQO\E\ZULWLQJUHJXODUO\FDQZHEULQJRXUUHVHDUFKLGHDVWRPDWXULW\)RUWKLVUHDVRQLI

ZHZDQWWREHJRRGUHVHDUFKHUVZHQHHGWRDFFHSWWKHIDFWWKDWEHLQJDZULWHULVD
UHJXODUSDUWRIRXUSURIHVVLRQDOLGHQWLW\,QGHHGDV&UHVZHOO VXPPDUL]HV
µ:ULWLQJH[LVWVLQDOOSKDVHVRIWKHFUHDWLYHSURFHVVRISODQQLQJDQGLQFRQGXFWLQJ
UHVHDUFK¶
6W\OHPDQXDOV
$FKDUDFWHULVWLFIHDWXUHRITXDQWLWDWLYHUHVHDUFKLVWKHGHVLUHIRUVWDQGDUGL]DWLRQDWHYHU\
OHYHODQGDFFRUGLQJO\WKHSUHSDUDWLRQRIDTXDQWLWDWLYHUHVHDUFKUHSRUWLVJRYHUQHGE\
WKHJXLGLQJSULQFLSOHVDQGUHJXODWLRQVRIGHWDLOHGµVW\OHPDQXDOV¶$VW\OHPDQXDO RU
µVW\OHJXLGH¶ LVLQWHQGHGWRHQVXUHWKHFRQVLVWHQF\RIHYHU\IDFHWRIDFDGHPLFZULWLQJ
IURPWKHOHYHORISXQFWXDWLRQWRWKHRUJDQL]LQJVWUXFWXUHRIWKHPDQXVFULSW7KHUHDUH
VHYHUDOFRPSHWLQJVW\OHPDQXDOVDYDLODEOHDQGMRXUQDOVDQGERRNSXEOLVKHUVDOZD\V
UHTXLUHDXWKRUVWRVXEPLWWKHLUZRUNIROORZLQJRQHRIWKHVHPDQXDOV7KHPRVW
SURPLQHQWGLIIHUHQFHVEHWZHHQWKHGLIIHUHQWVW\OHPDQXDOVLQFOXGHWKHZD\ZHFLWH
UHIHUHQFHVLQWKHWH[W LHWKHIRUPDWRIWKHLQWH[WFLWDWLRQV DQGSUHSDUHWKHILQDOOLVWRI
UHIHUHQFHVLQWKHELEOLRJUDSK\ LHWKHIRUPDWRIWKHHQGRIWH[WUHIHUHQFHVIRUPRUH
GHWDLOVVHHWKHQH[WVHFWLRQ :HDOVRILQGVRPHGLVSDULW\LQWKHIRUPDWRI
KHDGLQJVVXEKHDGLQJVILJXUHVDQGWDEOHVDVZHOODVLQWKHXVHRIFHUWDLQV\PEROV
DEEUHYLDWLRQVDQGRWKHUVSHFLDOODQJXDJHDVSHFWV
%HFDXVHLWPD\EHWLPHFRQVXPLQJWRUHIRUPDWWKHFRPSOHWHGZRUNLQWKHODVW
PLQXWHLWLVDGYLVDEOHWRDGRSWDVW\OHPDQXDOHDUO\LQWKHZULWLQJSURFHVVDQGWKHQ
VWLFNWRLWWRWKHHQG7KHPRVWFRPPRQO\XVHGVW\OHPDQXDOLQRXUILHOGLVWKHRQH
RIIHUHGE\WKH$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQLWLVSXEOLVKHGLQDERRNIRUPDW
DQGWKHPRVWUHFHQWHGLWLRQRIWKHµ$3$VW\OH¶ DVLVXVXDOO\UHIHUUHGWR LVWKHILIWK
HGLWLRQSXEOLVKHGLQ $3$ 0DQ\DSSOLHGOLQJXLVWLFVMRXUQDOVVXFKDV
/DQJXDJH/HDUQLQJ7KH0RGHUQ/DQJXDJH-RXUQDO7(62/4XDUWHUO\DQG6WXGLHVLQ
6HFRQG/DQJXDJH$FTXLVLWLRQIROORZWKLVVW\OH:LWKUHJDUGWRERRNSXEOLVKHUVVRPH
KDYHDGRSWHGWKH$3$VW\OHZKHUHDVRWKHUVKDYHGHYHORSHGWKHLURZQVW\OH
UHTXLUHPHQWVRYHUWKH\HDUV7KHSXEOLVKHURIWKHFXUUHQWERRN2[IRUG8QLYHUVLW\
3UHVVIRUH[DPSOHKDVLWVRZQVW\OHPDQXDODQGDOOWKHERRNVDQGMRXUQDOVSXEOLVKHGE\
WKLVSXEOLVKHU IRUH[DPSOH$SSOLHG/LQJXLVWLFVDQG(/7-RXUQDO IROORZWKLV
$XVHIXOIHDWXUHRIWKH$3$VW\OHPDQXDOLVWKDWLWQRWRQO\RIIHUVIRUPDW
SUHVFULSWLRQVEXWDOVRSURYLGHVHODERUDWHJXLGHOLQHVRQKRZWRVWUXFWXUHUHVHDUFKUHSRUWV
DQGKRZWRSUHVHQWDQGRUJDQL]HWKHFRQWHQWRIWKHYDULRXVSDUWV,QWKHIROORZLQJ
GLVFXVVLRQRIWKHVWUXFWXUHRIWKHTXDQWLWDWLYHUHVHDUFKUHSRUW,ZLOOGUDZRQWKH
SULQFLSOHVRIWKH$3$VW\OH $3$±
7KHVWUXFWXUHRIWKHTXDQWLWDWLYHUHVHDUFKUHSRUW
5HJDUGOHVVRIWKHOHQJWKRIDTXDQWLWDWLYHUHVHDUFKUHSRUW IRUH[DPSOHMRXUQDODUWLFOHRU
GLVVHUWDWLRQWKHVLV ZULWLQJXSWKHUHVXOWVLVUHODWLYHO\VWUDLJKWIRUZDUGEHFDXVHWKHUHLVD
ZHOOHVWDEOLVKHGUHSRUWLQJIRUPDWZLWKFRQYHQWLRQDOPRGHOVDQGVWUXFWXUHVWRJXLGHXV

7KHVSHFLILFGHWDLOVRIKRZWRSUHVHQWRXUILQGLQJVDUHXVXDOO\SURYLGHGE\WKHIRUXPWR
ZKLFKZHZLVKWRVXEPLWWKHILQDOUHSRUW7KHVHYDULRXVIRUPDWWLQJUHTXLUHPHQWVDUH
REOLJDWRU\WRIROORZDQGWKH\PD\LQLWLDOO\DSSHDUDVDFRQVWUDLQWLQWU\LQJWR
FRPPXQLFDWHRXUPHVVDJH+RZHYHU,KDYHFRPHWRDSSUHFLDWHWKHTXDQWLWDWLYHUHSRUW
IRUPDWDVEHLQJORJLFDODQGUHDGHUIULHQGO\ZLWKHYHU\WKLQJLQLWEHLQJJHDUHGDWWKH
DFFHVVLEOHSUHVHQWDWLRQRIWKHHPSLULFDOUHVXOWV,QWKHIROORZLQJ,GHVFULEHWKHPDLQ
SDUWVRIWKHTXDQWLWDWLYHUHVHDUFKUHSRUWDQGUHDGHUVFDQDOVRUHIHUWRWKHµ4XDQWLWDWLYH
5HVHDUFK*XLGHOLQHV¶SXEOLVKHGE\WKHMRXUQDO7(62/4XDUWHUO\ &KDSHOOHDQG'XII

µ)URQWPDWWHU¶
7KHµIURQWPDWWHU¶LQFOXGHVWKHWLWOHWKHDEVWUDFWDQGLQVRPHZRUNVWKHWDEOHRI
FRQWHQWV7KHLPSRUWDQFHRIWKHVHUDWKHUWHFKQLFDOSDUWVLVWKDWWKH\SURYLGHWKHLQLWLDO
FRQWDFWSRLQWEHWZHHQRXUZRUNDQGWKHDXGLHQFH
x 7LWOH$JRRGWLWOHSHUIRUPVWZRIXQFWLRQV2QWKHRQHKDQGLWVXPPDUL]HVWKHPDLQ
LGHDRIWKHZRUNGHVFULELQJWKHWRSLFDQGWKHDFWXDOYDULDEOHVLVVXHVXQGHU
LQYHVWLJDWLRQ7KXVLWVKRXOGEHLQIRUPDWLYHSDUWLFXODUO\EHFDXVHLQHOHFWURQLF
DEVWUDFWLQJDQGLQIRUPDWLRQVHUYLFHVVXFKDVWKH/LQJXLVWLFDQG/DQJXDJH
%HKDYLRXU$EVWUDFWV //%$ RUWKH(5,& (GXFDWLRQDO5HVRXUFHV,QIRUPDWLRQ
&HQWHU GDWDEDVHWLWOHVHDUFKHVDUHYHU\FRPPRQ2QWKHRWKHUKDQGDJRRGWLWOH²
VLPLODUWRLWVFRXQWHUSDUWVLQFUHDWLYHZULWLQJ²QHHGVDOVRWREHµFDWFK\¶WKDWLV
DWWUDFWWKHUHDGHU¶VDWWHQWLRQ)RUWKLVUHDVRQVRPHDXWKRUVLQFOXGHFDSWLYDWLQJ
SKUDVHVLQWKHLUWLWOHV)LQGLQJDEDODQFHEHWZHHQWKHWZRDVSHFWVLVGLIILFXOWDQG,
ZRXOGUHFRPPHQGDPRUHFRQVHUYDWLYHDSSURDFKIDYRXULQJDQLQIRUPDWLYHWLWOH
RYHUDZLWW\RQH+RZHYHU,PXVWDGPLWWKDWFKRRVLQJDUHDOO\JRRGWLWOH²IRU
H[DPSOH5HDO0HQ'RQ¶W&ROOHFW6RIW'DWDE\*KHUDUGLDQG7XUQHU IRUD
SDSHURQWKH48$/48$1SDUDGLJPGLVWLQFWLRQ²FDQFRQWULEXWHFRQVLGHUDEO\WR
WKHSHUFHSWLRQDQGWKHFDUHHURIWKHSDUWLFXODUZRUN
x $EVWUDFW7KHDEVWUDFWLVDKXJHO\LPSRUWDQWSDUDJUDSKDWWKHEHJLQQLQJRIUHVHDUFK
DUWLFOHVZKLFKIXUWKHUSXUVXHVWKHµLQIRUPDQGDWWUDFW¶IXQFWLRQRIWKHWLWOH7KH
$3$3XEOLFDWLRQ0DQXDO $3$ ULJKWO\SRLQWVRXW
0RVWSHRSOHZLOOKDYHWKHLUILUVWFRQWDFWZLWKDQDUWLFOHE\VHHLQJMXVWWKH
DEVWUDFWXVXDOO\RQDFRPSXWHUVFUHHQZLWKVHYHUDORWKHUDEVWUDFWVDVWKH\DUH
GRLQJDOLWHUDWXUHVHDUFKWKURXJKDQHOHFWURQLFDEVWUDFWUHWULHYDOV\VWHP
5HDGHUVIUHTXHQWO\GHFLGHRQWKHEDVLVRIWKHDEVWUDFWZKHWKHUWRUHDGWKH
HQWLUHDUWLFOH
7KHDEVWUDFWQHHGVWRVXPPDUL]HWKHUHVHDUFKSUREOHPDQGMXVWLI\ZK\LWLVZRUWK
VWXG\LQJ,WDOVRQHHGVWRSURYLGHVRPHLQIRUPDWLRQDERXWWKHPHWKRGRORJ\RIWKH
VWXG\DQGDERXWWKHPRVWLPSRUWDQWUHVXOWVDQGLPSOLFDWLRQV7KHUHLVXVXDOO\DVWULFW

ZRUGOLPLWIRUDEVWUDFWV ZRUGVLVFRPPRQ VRWKHFKDOOHQJHLVWRVD\DV
PXFKDVSRVVLEOHLQDVIHZZRUGVDVSRVVLEOH
x 7DEOHRIFRQWHQWV)RUUHSRUWVWKDWDUHORQJHUWKDQDUWLFOHVZHDOVRQHHGDWDEOHRI
FRQWHQWV6LOYHUPDQ HPSKDVL]HVWKDWDWDEOHRIFRQWHQWVLVQRWPHUHO\D
WULYLDOWHFKQLFDOPDWWHU,WLVRQHRIWKHILUVWWKLQJVDUHDGHURIDERRNRU
GLVVHUWDWLRQWKHVLVORRNVDWDQGWKXVLWFRQYH\VDQLPSRUWDQWPDVVDJHDERXWWKH
ORJLFDODQGVRXQGQDWXUHRIWKHPDWHULDO
,QWURGXFWLRQOLWHUDWXUHUHYLHZDQGUHVHDUFKTXHVWLRQV
,QWKH,QWURGXFWLRQZHUHLWHUDWHRQFHDJDLQZKDWZHVDLGLQWKHWLWOHDQGWKHDEVWUDFWEXW
WKLVWLPHDWPRUHOHQJWK7KLVSDUWSUHVHQWVWKHVSHFLILFWRSLFSUREOHPLVVXHXQGHUVWXG\
DQGDGGUHVVHVWKHIROORZLQJTXHVWLRQV:KDWLVWKHWRSLFDQGZK\LVLWLPSRUWDQW"+RZ
GRHVWKHUHVHDUFKDSSURDFKRIWKHSDSHUUHODWHWRWKHLVVXHLQTXHVWLRQ":KDWDUHWKH
WKHRUHWLFDOLPSOLFDWLRQVRIWKHVWXG\DQGKRZGRHVWKHVWXG\UHODWHWRSUHYLRXVZRUNLQ
WKHDUHD"7KDWLVWKLVVHFWLRQLVVWLOOSDUWRIWKHµVHOOLQJDQGLQIRUPLQJ¶SKDVHRIWKH
UHSRUW,QORQJHUZRUNVVXFKDVGLVVHUWDWLRQVDQGWKHVHVWKH,QWURGXFWLRQDOVR
VXPPDUL]HVWKHVWUXFWXUHRIWKHZRUNE\EULHIO\RXWOLQLQJDQGMXVWLI\LQJWKHFRQWHQWRI
WKHVXEVHTXHQWFKDSWHUV,QDUWLFOHVL]HGUHSRUWVWKH,QWURGXFWLRQLVVRPHWLPHVPHUJHG
ZLWKWKHOLWHUDWXUHUHYLHZ
$OWKRXJKHYHU\ERG\ZRXOGDJUHHWKDWDSURSHUUHVHDUFKUHSRUWVKRXOGFRQWDLQD
UHYLHZRIWKHOLWHUDWXUHWKHUHLVXVXDOO\TXLWHDELWRIFRQIXVLRQDERXWZKDWVRUWRI
OLWHUDWXUHVXPPDU\LVQHHGHG,EHOLHYHWKDWSDUWRIWKHSUREOHPVWHPVIURPWKH
DPELJXRXVUROHRIWKHUROHRIWKHOLWHUDWXUHUHYLHZDVLWFDQDFFRPSOLVKDWOHDVWIRXU
GLIIHUHQWIXQFWLRQV D ,WFDQDFWDVWKHµPDSRIWKHWHUUDLQ¶SURYLGLQJDFRPSUHKHQVLYH
DQGRIWHQKLVWRULFDORYHUYLHZRIWKHERRNVDUWLFOHVDQGRWKHUGRFXPHQWVGHVFULELQJWKH
EURDGHUGRPDLQXQGHULQYHVWLJDWLRQ E ,WFDQSURYLGHWKHVSHFLILFWKHRUHWLFDO
EDFNJURXQGRIWKHHPSLULFDOLQYHVWLJDWLRQUHSRUWHGMXVWLI\LQJWKHQHHGIRULWWKXV
IRFXVLQJSULPDULO\RQZRUNVWKDWKDYHDFRQFUHWHEHDULQJRQWKHUHVHDUFKGHVLJQRUWKH
LQWHUSUHWDWLRQRIWKHUHVXOWV F ,WFDQEHXVHGWRPDUNRXWWKHLQWHOOHFWXDO
VWUDQGSRVLWLRQWKDWWKHDXWKRULVDOLJQLQJZLWKDQGWRJHQHUDWHWUXVWZRUWKLQHVVLQWKH
DXGLHQFHE\VKRZLQJWKDWWKDWWKHDXWKRULVNQRZOHGJHDEOHLQWKHDUHD G )LQDOO\DQG
UHODWHGWRWKHSUHYLRXVSRLQWLQXQLYHUVLW\DVVLJQPHQWV IRUH[DPSOHGLVVHUWDWLRQVDQG
WKHVHV LWLVDOVRWRSURYHWKDWWKHDXWKRUKDVGRQHKLVKHUµKRPHZRUN¶DQGKDVEHFRPH
IDPLOLDUZLWKDZLGHUDQJHRIUHOHYDQWWKHRUHWLFDODQGUHVHDUFKDSSURDFKHV$JRRG
OLWHUDWXUHUHYLHZGLVSOD\VDELWRIDOOWKHIRXUIXQFWLRQVEXWERRNVXVXDOO\SD\VSHFLDO
DWWHQWLRQWRWKHILUVWRQH PDSSLQJWKHWHUUDLQ ZKLOHDUWLFOHVWRWKHVHFRQG SURYLGLQJ
VSHFLILFWKHRUHWLFDOEDFNJURXQG
:HVKRXOGQRWHKHUHWKDWLQPDQ\UHVHDUFKDUWLFOHVWKHOLWHUDWXUHUHYLHZFDQEHTXLWH
VKRUWRIWHQRQO\DERXWSHUFHQWRIWKHWRWDOOHQJWKRUHYHQOHVV DOWKRXJKLQVRPH
FDVHVLWFDQEHFRQVLGHUDEO\ORQJHUWDNLQJXSKDOIRIWKHZKROHSDSHU 7KLVLVGXHWR
WKHVSDFHFRQVWUDLQWVRIMRXUQDOVZLWKHGLWRUVZDQWLQJDXWKRUVWRSODFHWKHHPSKDVLVRQ

WKHQHZILQGLQJVUHSRUWHGLQWKHDUWLFOH,QIDFWWKH$3$JXLGHOLQHV VWDWH
µ'LVFXVVWKHOLWHUDWXUHEXWGRQRWLQFOXGHDQH[KDXVWLYHKLVWRULFDOUHYLHZ$VVXPHWKDW
WKHUHDGHULVNQRZOHGJHDEOHDERXWWKHILHOGIRUZKLFK\RXDUHZULWLQJDQGGRHVQRW
UHTXLUHDFRPSOHWHGLJHVW¶
,GLGQRWPHQWLRQLQWKHDERYHGHVFULSWLRQDIXQFWLRQWKDWLVVRRIWHQFLWHGDVWKH
PDLQUROHRIWKHOLWHUDWXUHUHYLHZWRSURYLGHDµFULWLFDODQDO\VLV¶RIWKHOLWHUDWXUH,GR
QRWWKLQNWKDWWKLVLVDSDUWLFXODUO\KHOSIXOVSHFLILFDWLRQEHFDXVHLWWHQGVWREHXVHGWR
LPSO\DQXPEHURIWKLQJVWKDWVWUDQJHO\HQRXJKGRQRWUHDOO\LQFOXGHEHLQJµFULWLFDO¶LQ
WKHVWULFWVHQVH,QIDFWLWLVUDUHDQGUDWKHUPDUNHGWRILQGDOLWHUDWXUHUHYLHZLQD
SXEOLVKHGZRUNWKDWLVRYHUO\µFULWLFDO¶LQWKHµFULWLFL]LQJ¶VHQVHDQGWKH$3$
JXLGHOLQHV H[SOLFLWO\VWDWHWKDWGLIIHUHQFHVVKRXOGEHSUHVHQWHGLQD
µSURIHVVLRQDOQRQFRPEDWLYHPDQQHU¶:KHQZHWDONWRVWXGHQWVDERXWWKHQHHGWREH
µFULWLFDO¶ZKDWZHUHDOO\PHDQLVWKDW D WKHUHVKRXOGEHDSHUVRQDODQJOHSULRULWL]DWLRQ
RUVWRU\OLQH VHH6HFWLRQ XQGHUO\LQJWKHLURYHUYLHZRIWKHOLWHUDWXUHDQG E WKH\
VKRXOGQRWKLGHEHKLQGDXWKRULWLHV DQGLQFOXGHWRRPDQ\OLWHUDOFLWDWLRQV EXWVKRXOG
DOVRPDNHWKHLURZQYRLFHVKHDUG
7KHLPSRUWDQFHRIUHVHDUFKTXHVWLRQVDQGK\SRWKHVHVKDVEHHQGLVFXVVHGLQ&KDSWHU
6HFWLRQ DQGOLVWLQJWKHVHTXHVWLRQVK\SRWKHVHVLVDFRPSXOVRU\FRPSRQHQWLQD
TXDQWLWDWLYHUHVHDUFKUHSRUW7KHEHVWSODFHWRGRVRLVXVXDOO\DWWKHHQGRIWKH
OLWHUDWXUHUHYLHZEXWLQORQJHUZRUNVWKHZULWHUPD\LQFOXGHDVHSDUDWH VKRUW VHFWLRQ
EHWZHHQWKHOLWHUDWXUHUHYLHZDQGWKHµ0HWKRG¶SDUWWRGHVFULEHWKHUHVHDUFKGHVLJQDQG
WKHJXLGLQJTXHVWLRQV
,QWHUQHWVHDUFKHVDQGHOHFWURQLFGDWDEDVHV
7KHXVHRI,QWHUQHWVHDUFKHVDQGHOHFWURQLFGDWDEDVHVLQSUHSDULQJWKHOLWHUDWXUHUHYLHZ
LVEHFRPLQJDQDFXWHLVVXHLQPDQ\SRVWJUDGXDWHSURJUDPPHVVROHWXVORRNDWWKLV
TXHVWLRQEULHIO\,QWKHFXUUHQWGLJLWDODJHLWLVLQFUHDVLQJO\QDWXUDOWRFRQGXFWDVHDUFK
XVLQJDQ,QWHUQHWVHDUFKHQJLQHVXFKDV*RRJOHZKHQHYHUZHQHHGVRPHLQIRUPDWLRQ
+RZHYHUWKLVSUDFWLFHVKRXOGQRWEHDXWRPDWLFDOO\WUDQVIHUUHGWRRXUVHDUFKIRU
UHOHYDQWOLWHUDWXUH7KHUHLVDORWRIPDWHULDORQWKH,QWHUQHWWKDWPD\ORRNUHOHYDQWEXWLV
DFWXDOO\RITXHVWLRQDEOHTXDOLW\5HVHDUFKHUVW\SLFDOO\ZDQWWRSXEOLVKWKHLUUHVXOWVLQ
ERRNVDQGMRXUQDOVEHFDXVHRQO\IRUVXFKSXEOLFDWLRQVGRWKH\JHWRIILFLDOUHFRJQLWLRQ
+RZHYHURQFHDZRUNKDVEHHQSXEOLVKHGLQDQHVWDEOLVKHGIRUXPFRS\ULJKWLVVXHV
XVXDOO\SUHYHQWLWIURPEHLQJPDGHIUHHO\DYDLODEOHRQWKH,QWHUQHW7RILQGTXDOLW\
SDSHUVUHODWHGWRRXUUHVHDUFKWRSLFZHQHHGWRVHDUFKDFDGHPLFGDWDEDVHVVXFKDVWKH
/LQJXLVWLFVDQG/DQJXDJH%HKDYLRXU$EVWUDFWVWKH0/$,QWHUQDWLRQDO
%LEOLRJUDSK\(5,&RU3V\FK,QIR7KHVHDUHDOVR,QWHUQHWEDVHGEXWDUHSDVVZRUG
SURWHFWHG²WKH\FDQEHUHDFKHGIRUH[DPSOHWKURXJKXQLYHUVLW\OLEUDULHV,WLVP\
SHUVRQDOYLHZWKDWRQO\LQWKHPRVWH[FHSWLRQDOFDVHVVKRXOGZHLQFOXGH,QWHUQHW
VRXUFHVLQWKHOLWHUDWXUHUHYLHZ²RIFRXUVHWKLVUHVWULFWLRQGRHVQRWDSSO\WRDUWLFOHV
IURPOHJLWLPDWH,QWHUQHWEDVHGHOHFWURQLFMRXUQDOVDQGQHLWKHUGRHVLWFRQFHUQ

SUHYLRXVO\SXEOLVKHGSDSHUVWKDWWKHLUDXWKRUKDVPDGHDYDLODEOHRQWKH,QWHUQHWIRUWKH
SXEOLF
0HWKRG
7KHµ0HWKRG¶VHFWLRQGHVFULEHVWKHWHFKQLFDOGHWDLOVRIKRZWKHLQYHVWLJDWLRQZDV
FRQGXFWHG2QO\RQWKHEDVLVRIWKLVLQIRUPDWLRQFDQUHDGHUVHYDOXDWHWKH
DSSURSULDWHQHVVRIWKHUHVHDUFKGHVLJQDQGWKHUHIRUHWKLVVHFWLRQLVFUXFLDOLQ
FRQYLQFLQJWKHDXGLHQFHDERXWWKHFUHGLELOLW\RIWKHUHVXOWV,WLVLPSRUWDQWWKDWWKLVSDUW
EHFRQFLVHDQGWRWKHSRLQWVRWKDWLWFDQVHUYHDVDµUHIHUHQFHVHFWLRQ¶LIWKHUHDGHU
ZDQWVWRORRNXSVRPHGHWDLOV7KHUHIRUHLQWKH0HWKRGVHFWLRQZHVKRXOGQRWUHIHUWR
RWKHUSHRSOH¶VZRUNEXWIRFXVHQWLUHO\RQRXURZQVWXG\
)RUWKHVDNHRIUHDGHUIULHQGOLQHVVWKH0HWKRGVHFWLRQLVDOZD\VIXUWKHUGLYLGHGLQWR
ODEHOOHGVXEVHFWLRQV7KHFRQYHQWLRQDOVXEKHDGLQJVLQFOXGHµ3DUWLFLSDQWV¶
µ,QVWUXPHQWV0DWHULDOV$SSDUDWXV¶µ3URFHGXUHV¶DQGVRPHWLPHVµ'DWDDQDO\VLV¶DQG
µ9DULDEOHVLQWKHVWXG\¶
x 3DUWLFLSDQWV2EYLRXVO\WKHDSSURSULDWHGHVFULSWLRQRIWKHUHVHDUFKVDPSOHLV
FULWLFDOWRTXDQWLWDWLYHUHVHDUFKEHFDXVHRQO\LQWKHOLJKWRIWKLVFDQZHGHFLGH
ZKHWKHUWKHJHQHUDOL]DWLRQRIWKHILQGLQJVLVOHJLWLPDWH7KHH[DFWGHWDLOVWREH
VXSSOLHGGHSHQGRQWKHIRFXVRIWKHVWXG\EXWQRUPDOO\LQFOXGHDVDPLQLPXPWKH
VDPSOHVL]H SRVVLEO\DFFRPSDQLHGE\VRPHMXVWLILFDWLRQRIWKHVDPSOLQJVWUDWHJ\
DQGWKHWRWDOQXPEHURIDOOWKHHOLJLEOHSHRSOH WKHSDUWLFLSDQWV¶DJHJHQGHUDQG
HWKQLFLW\DVZHOODVDQ\UHOHYDQWJURXSLQJYDULDEOH IRUH[DPSOHQXPEHURIFRXUVHV
RUFODVVHVWKH\FRPHIURP ,IWKHVWXG\FRQFHUQVODQJXDJHOHDUQHUVZHQHHGWR
VSHFLI\WKHLUOHYHORI/SURILFLHQF\/OHDUQLQJKLVWRU\/WHDFKLQJLQVWLWXWLRQ LI
DSSOLFDEOH DQGWKHW\SHRIWXLWLRQUHFHLYHG7\SLFDORSWLRQDOEDFNJURXQGYDULDEOHV
LQFOXGHWKHSDUWLFLSDQWV¶JHQHUDORUODQJXDJHDSWLWXGH RUDFDGHPLFDELOLW\
VRFLRHFRQRPLFVWDWXVRFFXSDWLRQRU LIWKHSDUWLFLSDQWVDUHVWXGHQWV DUHDVRI
VSHFLDOL]DWLRQ/FODVVVL]H/WHDFKLQJPDWHULDOVXVHGDQGWKHDPRXQWRIWLPH
VSHQWLQDQ/KRVWHQYLURQPHQW
x ,QVWUXPHQWV0DWHULDOV$SSDUDWXV,QTXDQWLWDWLYHVWXGLHVWKHGHVLJQDQGWKHTXDOLW\
RIWKHUHVHDUFKLQVWUXPHQWV W\SLFDOO\WHVWVDQGTXHVWLRQQDLUHV GHWHUPLQHERWKWKH
W\SHDQGWKHNLQGRIGDWDWKDWFDQEHFROOHFWHG7KHUHIRUHZHQRWRQO\KDYHWR
GHVFULEHWKHLQVWUXPHQWDFFXUDWHO\EXWKDYHDOVRWRJLYHDUDWLRQDOHIRUWKHVSHFLILF
FRQWHQWDUHDVFRYHUHGDQGMXVWLI\OHDYLQJRXWVRPHSRWHQWLDOO\LPSRUWDQWDUHDV:H
DOVRQHHGWRVXSSO\GHWDLOVDERXWWKHSLORWLQJRIWKHLQVWUXPHQWDQGSDUWLFXODUO\
ZKHQVHQVLWLYHLWHPVDUHLQFOXGHGKRZDQRQ\PLW\DQGFRQILGHQWLDOO\ZHUH
KDQGOHG5HOLDELOLW\DQGYDOLGLW\GDWDFDQEHLQWURGXFHGKHUHRUODWHULQWKH
µ5HVXOWV¶VHFWLRQ7KHDFWXDOLQVWUXPHQWLVRIWHQLQFOXGHGLQDQDSSHQGL[
x 3URFHGXUH8QGHUWKLVVXEKHDGLQJZHGHVFULEHWKHSURFHGXUHVZHIROORZHGGXULQJ
WKHGDWDFROOHFWLRQSKDVHZLWKDVSHFLILFIRFXVRQKRZWKHLQVWUXPHQWVZHUH
DGPLQLVWHUHGVXPPDUL]LQJWKHFRQGLWLRQVXQGHUZKLFKWKHPHDVXUHPHQWVZHUH

WDNHQ IRUH[DPSOHIRUPDWWLPHSODFHDQGSHUVRQQHOZKRFROOHFWHGWKHGDWD
:LONLQVRQDQG7)6, 7KLVLVZKHUHZHFDQWDONDERXWWUDQVODWLRQLVVXHVLI
WKHLQVWUXPHQWZDVLQWKHSDUWLFLSDQWV¶/DVZHOODVDQ\FRPSOLFDWLRQVRU
XQH[SHFWHGHYHQWVGXULQJGDWDFROOHFWLRQ
x 'DWDDQDO\VLV$OWKRXJKWKH$3$JXLGHOLQHVGRQRWKLJKOLJKWµGDWDDQDO\VLV¶DVDQ
REOLJDWRU\VHFWLRQPDQ\VWXGLHVLQFOXGHLWEHFDXVHVXPPDUL]LQJWKHGDWD
SURFHVVLQJSURFHGXUHVLQDGYDQFHLVDUHDGHUIULHQGO\ZD\RISUHSDULQJWKH
DXGLHQFHIRUZKDWLVWRFRPHHVSHFLDOO\LIVRPHFRPSOH[DQDO\WLFDOWHFKQLTXHLV
XVHG7KLVVHFWLRQLVDOVRXVHIXOWRGLVFXVVSRVVLEOHSUREOHPLVVXHVVXFKDVPLVVLQJ
GDWDDWWULWLRQDQGQRQUHVSRQVHDQGKRZWKHVHZHUHKDQGOHG,WLVLPSRUWDQWWRQRWH
KRZHYHUWKDWWKH'DWDDQDO\VLVVHFWLRQLVW\SLFDOO\VKRUWDQGGRHVQRWFRQWDLQDQ\
UHDOUHVXOWVRQO\WKHOLVWLQJRIWKHVWDWLVWLFDOSURFHGXUHVDSSOLHG
x 9DULDEOHVLQWKHVWXG\:KHQWKHGDWDDQDO\VLVFRQFHUQVDODUJHQXPEHURIYDULDEOHV
DQGHVSHFLDOO\ZKHQVRPHRIWKHYDULDEOHVKDYHEHHQGHULYHGIURPRWKHUV IRU
H[DPSOHFRPSRVLWHVFRUHV ZHPD\IHHOWKDWWKHFODULW\RIWKHUHSRUWUHTXLUHV
OLVWLQJDOOWKHYDULDEOHVXVHGLQWKHVWXG\GHVFULELQJKRZWKH\ZHUHGHULYHGRU
RSHUDWLRQDOL]HG
,QVXPWKHUHDUHFHUWDLQFRPSXOVRU\VXEVHFWLRQVRIWKH0HWKRGVHFWLRQDQGZHFDQ
DGGWRWKHVHRWKHUVHFWLRQVLIZHGHFLGHWKDWWKH\DUHHVVHQWLDORUXVHIXOIRU
FRPSUHKHQGLQJDQGSRVVLEO\UHSOLFDWLQJWKHVWXG\
5HVXOWVDQGGLVFXVVLRQ
3UHVHQWLQJDQGLQWHUSUHWLQJWKHILQGLQJVFDQEHGRQHHLWKHULQRQHFRPELQHGRULQWZR
VHSDUDWHVHFWLRQVGHSHQGLQJRQZKHWKHUZHSUHIHUGHVFULELQJRXUUHVXOWVILUVWDQGWKHQ
HYDOXDWLQJLQWHUSUHWLQJWKHPRUGRLQJWKHWZRWDVNVLQDQRQJRLQJDQGFRPELQHG
PDQQHU,IWKHUHLVDORWRIQXPHULFDOGDWDWRSUHVHQWEHIRUHEHLQJDEOHWRGUDZ
LQIHUHQFHVLWPD\EHEHWWHUWRUHOHJDWHWKHVHLQWRDQLQGHSHQGHQW5HVXOWVVHFWLRQWREH
IROORZHGE\DPRUHFRQFHSWXDO'LVFXVVLRQ,QPRVWRWKHUFDVHV,KDYHIRXQGWKDWWKH
FRPELQDWLRQRIWKHWZRVHFWLRQVZRUNVEHWWHUEHFDXVHLWDOORZVIRUDQRQJRLQJ
FRPPHQWDU\DQGWKHRUHWLFDOIUDPLQJ
:LWKORQJHUZRUNVVXFKDVGLVVHUWDWLRQVWKHUHDUHVRPDQ\GLIIHUHQWW\SHVRIUHVXOWV
WRUHSRUWWKDWWKHPRVWUHDGHUIULHQGO\ZD\RIGHDOLQJZLWKWKHPLVWRGLYLGHWKH
µ5HVXOWVDQGGLVFXVVLRQ¶SDUWLQWRWKHPDWLFDOO\ODEHOOHGFKDSWHUVHDFKFRQWDLQLQJ
FRPELQHGUHVXOWVDQGGLVFXVVLRQ,QDJUHHPHQWZLWKWKH$3$JXLGHOLQHVOHWPHGUDZ
DWWHQWLRQWRWZRDVSHFWVRIWKHGLVFXVVLRQSURFHVVWKDWDUHRIWHQRPLWWHGKLJKOLJKWLQJ
DQ\WKHRUHWLFDOFRQVHTXHQFHVRIWKHUHVXOWVDQGH[DPLQLQJWKHYDOLGLW\RIWKH
FRQFOXVLRQ V
$V6DQGHORZVNL H[SODLQVLQTXDQWLWDWLYHUHVHDUFKWKHDSSHDOWRQXPEHUV
JLYHVVWXGLHVWKHLUUKHWRULFDOSRZHUDQGWKHUHIRUHWKH5HVXOWVDQG'LVFXVVLRQVHFWLRQLV
FHQWUHGDURXQGVWDWLVWLFV<HWLWLVLPSRUWDQWWRVWUHVVWKDWWKLVGRHVQRWPHDQWKDWWKLV

VHFWLRQQHHGVWREHRYHUO\WHFKQLFDOXVLQJVWDWLVWLFDOµSRZHUODQJXDJH¶DQGDVPDQ\
FRPSOH[UHVXOWVWDEOHVDVSRVVLEOH/D]DUDWRQ ± H[SUHVVHVWKLVYHU\
FOHDUO\
:KDWDUHWKHLPSOLFDWLRQVRIVRPXFKSXEOLVKHGDSSOLHGOLQJXLVWLFVUHVHDUFK
HPSOR\LQJIDLUO\VRSKLVWLFDWHGDQDO\WLFDOSURFHGXUHV"2QHUHVXOWLVWKDWDJUHDW
GHDORIWKHUHVHDUFKEHFRPHVREVFXUHIRUDOOEXWWKHPRVWVWDWLVWLFDOO\OLWHUDWH
DPRQJXV3DJHDIWHUSDJHRI$129$WDEOHVLQDUHVXOWVVHFWLRQFKDOOHQJHVHYHQ
WKHPRVWGHWHUPLQHGUHDGHUZKRPD\VWLOOJLYHXSDQGKHDGULJKWIRUWKH
GLVFXVVLRQVHFWLRQ
,QFRQWUDVWVRPHRIWKHEHVWVWDWLVWLFLDQVZULWHYHU\FOHDUO\HPSKDVL]LQJWKHLQWHUQDO
ORJLFUDWKHUWKDQWKHPDWKHPDWLFDOSURFHGXUDOFRPSOH[LW\RIWKHDQDO\VHV$OVRZH
VKRXOGXVHLISRVVLEOHUHDGHUIULHQGO\GDWDSUHVHQWDWLRQPHWKRGVVXFKDVWDEOHVDQG
ILJXUHV VHH6HFWLRQ DQGLIZHGRVRZHPXVWPDNHVXUHWKDWZHGRQRWUHSHDW
YHUEDWLPWKHLQIRUPDWLRQWKDWKDVDOUHDG\EHHQSUHVHQWHGLQWDEOHVDQGILJXUHV
&RQFOXVLRQ
1RWHYHU\UHVHDUFKDUWLFOHKDVDµ&RQFOXVLRQ¶EHFDXVHLIZHKDYHVHSDUDWHµ5HVXOWV¶DQG
µ'LVFXVVLRQ¶VHFWLRQVWKHODWWHUFDQFRQFOXGHWKHZKROHSDSHU<HW,ZRXOGJHQHUDOO\
UHFRPPHQGWKHLQFOXVLRQRIDVKRUWFRQFOXGLQJVHFWLRQEHFDXVHWKLVLVDKLJKO\UHDGHU
IULHQGO\RSSRUWXQLW\WRVXPPDUL]HRXUPDLQILQGLQJVLQQRQWHFKQLFDOODQJXDJH$V
0F.D\ VXPVXSWKH&RQFOXVLRQFDQDOVRLQFOXGHDYDULHW\RIRWKHUPRYHVLQ
DGGLWLRQWRWKHVXPPDU\RIWKHPDMRUILQGLQJVVXFKDVDFDOOIRUIXUWKHUUHVHDUFKRQ
TXHVWLRQVUDLVHGE\WKHLQYHVWLJDWLRQVRPHGLVFXVVLRQRIWKHOLPLWDWLRQVRIWKHVWXG\DQ
H[SODQDWLRQRIWKHSHGDJRJLFDOLPSOLFDWLRQVRIWKHUHVXOWVDQGDVWDWHPHQWRIWKH
RYHUDOOVLJQLILFDQFHRIWKHWRSLFDGGUHVVHGE\WKHVWXG\+RZHYHUZHVKRXOGDOVRQRWH
WKHUHFRPPHQGDWLRQVRIWKHµ7DVN)RUFHRQ6WDWLVWLFDO,QIHUHQFH¶RIWKH$PHULFDQ
3V\FKRORJLFDO$VVRFLDWLRQ :LONLQVRQDQG7)6, DERXWWKHFRQWHQWRI
FRQFOXVLRQV
6SHFXODWLRQPD\EHDSSURSULDWHEXWXVHLWVSDULQJO\DQGH[SOLFLWO\1RWHWKH
VKRUWFRPLQJVRI\RXUVWXG\5HPHPEHUKRZHYHUWKDWDFNQRZOHGJLQJ
OLPLWDWLRQVLVIRUWKHSXUSRVHRITXDOLI\LQJUHVXOWVDQGDYRLGLQJSLWIDOOVLQIXWXUH
UHVHDUFK&RQIHVVLRQVKRXOGQRWKDYHWKHJRDORIGLVDUPLQJFULWLFLVP
5HFRPPHQGDWLRQVIRUIXWXUHUHVHDUFKVKRXOGEHWKRXJKWIXODQGJURXQGHGLQ
SUHVHQWDQGSUHYLRXVILQGLQJV*UDWXLWRXVVXJJHVWLRQV µIXUWKHUUHVHDUFKQHHGVWR
EHGRQH«¶ ZDVWHVSDFH
,QDQ\FDVHDV0F.D\ VXJJHVWVWKH&RQFOXVLRQVKRXOGHQGRQDµVWURQJ
QRWH¶ZLWKVRPHJHQHUDOVWDWHPHQWWKDWOHDYHVWKHUHDGHUZLWKDµSRVLWLYHVHQVHRIWKH
DUWLFOH¶ S
5HIHUHQFHVRU%LEOLRJUDSK\

7KHUHDUHWZREDVLFUXOHVZLWKUHJDUGWRWKHILQDOµ5HIHUHQFHV¶OLVW DOVRFDOOHG
µ%LEOLRJUDSK\¶E\VRPHVW\OHPDQXDOV )LUVWDOOWKHFLWDWLRQVLQWKHUHSRUWPXVWDSSHDU
LQWKHUHIHUHQFHOLVWDQGDOOUHIHUHQFHVLQWKHOLVWPXVWEHFLWHGLQWKHWH[W6HFRQGWKH
IRUPDWUHTXLUHPHQWVRIWKHSUHVHQWDWLRQVW\OHUHTXLUHGE\WKHSDUWLFXODUSXEOLFDWLRQ
PXVWEHDGKHUHGWRULJRURXVO\,KDYHIRXQGWKDWWKHTXDOLW\DQGDFFXUDF\RIWKH
UHIHUHQFHOLVWDQGWKHZD\LWLVSUHVHQWHGLQDPDQXVFULSWLVDQLPSRUWDQWLQGH[RIWKH
TXDOLW\RIWKHZKROHZRUN
5HVHDUFKHUVZLWKDSXEOLFDWLRQKLVWRU\XVXDOO\KDYHUDWKHUXQSOHDVDQWPHPRULHVRI
WKHORQJKRXUV DQGVRPHWLPHVGD\V WKH\KDYHVSHQWLQWKHSDVWUHIRUPDWWLQJ
UHIHUHQFHVZKHQWKH\ZDQWHGWRXVHWKHPLQDQRWKHUSXEOLFDWLRQWKDWUHTXLUHGD
GLIIHUHQWVW\OH)RUWXQDWHO\WKHVHGD\VPLJKWEHRYHUEHFDXVHUHFHQWFRPSXWHU
VRIWZDUHVXFKDV(QG1RWHFDQKHOSXVWRVWRUHDOORXUUHIHUHQFHVLQDJHQHULFIRUPWREH
SULQWHGRXWLQDQ\IRUPDWZHSOHDVH7KLVLVVRPHWKLQJHYHU\QRYLFHUHVHDUFKHUVKRXOG
ORRNLQWR LI\RXKDYHQRWGRQHVRDOUHDG\
$SSHQGL[
$QDSSHQGL[LVKHOSIXOLIWKHGHWDLOHGGHVFULSWLRQRIFHUWDLQUHOHYDQWPDWHULDOLV
GLVWUDFWLQJLQRULQDSSURSULDWHWRWKHPDLQERG\RIWKHZRUN)RUH[DPSOHVRPH
H[WHQVLYHVWDWLVWLFDOWDEOHVFDQEHUHOHJDWHGWRWKHDSSHQGL[EXWRQO\LIWKH\GRQRW
FRQWDLQNH\UHVXOWVVXSSRUWLQJWKHPDLQDUJXPHQWV²LIWKH\GRWKH\VKRXOGEHOHIWLQ
WKHPDLQERG\RIWKHWH[WUHJDUGOHVVRIWKHLUVL]H7KH$SSHQGL[LVDOVRWKHSODFHWR
LQFOXGHFRSLHVRIWKHLQVWUXPHQWVWUDQVFULSWVDQGRWKHUVXSSOHPHQWDU\PDWHULDOVIRU
UHDGHUVWRKDYHDEHWWHULGHDRIZKDWH[DFWO\ZHQWRQVRPHUHVHDUFKMRXUQDOV
VSHFLILFDOO\UHTXLUHWKHDXWKRUVWRKDYHDQDSSHQGL[ZLWKDOORIWKHLUH[SHULPHQWDO
LWHPV:LWKGLVVHUWDWLRQVDQGWKHVHV,WHQGWRHQFRXUDJHP\VWXGHQWVWRLQFOXGHDOO
UHOHYDQWPDWHULDOLQWKH$SSHQGLFHVVRWKDWWKH\GRQRWORVHWKHPRYHUWKH\HDUV
5HDGHUIULHQGO\GDWDSUHVHQWDWLRQPHWKRGV
7KHUHDUHWZRVSHFLILFUHDVRQVZK\ZHPD\ZDQWWRLQFOXGHYDULRXVUHDGHUIULHQGO\
GDWDSUHVHQWDWLRQPHWKRGVLQTXDQWLWDWLYHUHVHDUFKUHSRUWV)LUVWIRUPDQ\SHRSOH
QXPEHUVDQGVWDWLVWLFVDUHUDWKHUGU\GXOODQGGLIILFXOWWRFRPSUHKHQG6HFRQG
TXDQWLWDWLYHVWXGLHVFDQRIWHQSURGXFHFRPSOH[GDWDWKDWQHHGVWREHSUHVHQWHGLQDQ
HIIHFWLYHDQGGLJHVWLEOHZD\WRPDLQWDLQFODULW\7KHPRVWUHDGHUIULHQGO\SUHVHQWDWLRQ
PHWKRGVDUHYLVXDORQHV²ILJXUHV UHSUHVHQWLQJLQWULFDWHUHODWLRQVKLSV DQGWDEOHV
SURYLGLQJFRPSOH[GDWDDQGVHWVRIDQDO\VHV ²ZKHUHE\ZHFDQKDYHDQRYHUYLHZRID
JUHDWGHDORIWKHUHVXOWVDWDJODQFH$V:LONLQVRQDQGWKH7)6, SRLQWRXW
EHFDXVHLQGLYLGXDOVKDYHGLIIHUHQWSUHIHUHQFHVIRUSURFHVVLQJFRPSOH[LQIRUPDWLRQLW
RIWHQKHOSVWRSURYLGHERWKILJXUHVDQGWDEOHV2QWKHRWKHUKDQGWKH$3$JXLGHOLQHV
FDXWLRQXVQRWWRXVHWDEOHVIRUGDWDWKDWFDQEHHDVLO\SUHVHQWHGLQDIHZVHQWHQFHVLQ
WKHWH[WSDUWO\EHFDXVHWKH\WDNHXSPRUHVSDFHDQGDUHPRUHH[SHQVLYHWRSULQW

:KHQZHXVHILJXUHVDQGWDEOHVZHPXVWQXPEHUWKHPDQGJLYHWKHPDEULHI
H[SODQDWRU\WLWOH7KHQZHQHHGWRPHQWLRQHYHU\RQHRIWKHPLQWKHWH[WUHIHUULQJWKH
UHDGHUWRWKHH[DFWQXPEHU IRUH[DPSOHµVHH)LJXUH¶ UDWKHUWKDQVD\LQJVRPHWKLQJ
WRWKHHIIHFWWKDWµDVWKHWDEOHEHORZVKRZV¶
)LJXUHV
)LJXUHVDUHPHWKRGVRIYLVXDOLVLQJYDULRXVFKDUDFWHULVWLFVRIWKHGDWD,QUHVHDUFK
UHSRUWVWZRW\SHVRIILJXUHVLQSDUWLFXODUDUHXVHGFKDUWVGLDJUDPVDQGVFKHPDWLF
UHSUHVHQWDWLRQV
µ&KDUWV¶RUDVWKH\DUHDOVRFDOOHGµGLDJUDPV¶RIIHUDXVHIXOZD\RIGHVFULELQJWKH
VL]HVWUHQJWKRIYDULDEOHVLQUHODWLRQWRHDFKRWKHUµ%DUFKDUWV¶DQGµOLQHFKDUWV¶XVHD
YHUWLFDO<D[LVDQGDKRUL]RQWDO;D[LVWRSUHVHQWGDWD VHH)LJXUHVDQG 7KH
YHUWLFDOD[LVXVXDOO\UHSUHVHQWVWKHXQLWRIPHDVXUHPHQW RUGHSHQGHQWYDULDEOH DQGWKH
KRUL]RQWDOD[LVWKHJURXSLQJ RULQGHSHQGHQW YDULDEOH V 7KHVHFKDUWVDUHYHU\IOH[LEOH
LQWHUPVRIWKHW\SHRIGDWDWKH\FDQGLVSOD\DQGWKH\FDQHIIHFWLYHO\GHPRQVWUDWH
FRPSDULVRQVRUFKDQJHVRYHUWLPHLQDZD\WKDWLVHDV\WRLQWHUSUHW

)LJXUH6DPSOHEDUFKDUWGLDJUDP

)LJXUH6DPSOHOLQHFKDUWGLDJUDP

µ6FKHPDWLFUHSUHVHQWDWLRQV¶RIIHUDXVHIXOZD\WRGHVFULEHFRPSOH[UHODWLRQVKLSV
EHWZHHQPXOWLSOHYDULDEOHVDQGW\SLFDOO\XWLOL]HYDULRXVER[HVDQGDUURZV VHH)LJXUH
7KH\FDQEHXVHGIRUH[DPSOHWRGHVFULEHWKHEOXHSULQWRIPHQWDOSURFHVVHVRU
WKHFRPSRQHQWLDOVWUXFWXUHRIPXOWLOHYHOFRQVWUXFWV

)LJXUH6DPSOHVFKHPDWLFUHSUHVHQWDWLRQ
7DEOHV
7DEOHVDUHXVHGWRVXPPDUL]HODUJHDPRXQWVRIGDWDDERXWWKHUHVSRQGHQWVDQGWKHLU
UHVSRQVHVDQGWRSUHVHQWWKHUHVXOWVRIVWDWLVWLFDODQDO\VHV7KH\DUHW\SLFDOO\PDGHXS
RIURZVDQGFROXPQVRIQXPEHUVHDFKPDUNHGZLWKKHDGLQJVDQGVXEKHDGLQJV7KH\
FDQSURYLGHDPRUHDFFXUDWHDQGULFKHUGHVFULSWLRQWKDQILJXUHVEXWWKH\DUHOHVV
GLJHVWLEOH7DEOHVDUHWKHUHIRUHPRUHDSSURSULDWHIRUDUWLFOHVLQDFDGHPLFMRXUQDOVWKDQ
IRUOHFWXUHVWRQRQVSHFLDOLVWDXGLHQFHV6WDWLVWLFDOUHVXOWVDUHRIWHQSUHVHQWHGLQWDEOHV
DQG&KDSWHUFRQWDLQVVHYHUDOH[DPSOHVRIWKHVH
7KHUHDUHWZRSRLQWVWKDWQHHGWREHVWUHVVHGDERXWWDEOHV)LUVWDVDOUHDG\
PHQWLRQHGLQ6HFWLRQRQµ5HVXOWVDQGGLVFXVVLRQ¶LIZHSUHVHQWVWDWLVWLFVLQ
WDEOHVZHVKRXOGQRWUHSHDWWKHPLQWKHWH[WH[FHSWIRUKLJKOLJKWLQJVRPHSDUWLFXODUO\
QRWHZRUWK\UHVXOWV6HFRQGZHVKRXOGQRWHWKDWVWDWLVWLFVWDEOHVKDYHFHUWDLQFDQRQLFDO
IRUPV DVLOOXVWUDWHGWKURXJKRXW&KDSWHU ERWKLQFRQWHQW LHZKDWLQIRUPDWLRQWR
LQFOXGH DQGIRUPDW IRUH[DPSOHXVXDOO\ZHGRQRWXVHYHUWLFDOOLQHV 7KHVHQHHGWR
EHREVHUYHGFORVHO\ZKLFKPHDQVWKDWVLPSO\LPSRUWLQJDWDEOHIURP6366LQWRWKH
PDQXVFULSWLVOLNHO\WREHLQDSSURSULDWH+RZHYHUFRQFHUQVIRUUHDGHUIULHQGOLQHVV
VSDFHFRQVLGHUDWLRQVDQGWKHSDUWLFXODUSXEOLFDWLRQ¶VVW\OHUHTXLUHPHQWVPD\RYHUZULWH
WKLVVWDQGDUGIRUP7KHEHVWWKLQJLVWRXVHSXEOLVKHGH[DPSOHVRIDVLPLODUQDWXUHWKDW
IROORZWKHVDPHVW\OHPDQXDODVPRGHOV

:ULWLQJTXDOLWDWLYHDQGPL[HG
PHWKRGVUHSRUWV
$OWKRXJKWKHSUHYLRXVFKDSWHUFRYHUHGWKHZULWLQJRIDUHVHDUFKUHSRUWSULPDULO\IURPD
TXDQWLWDWLYHSHUVSHFWLYHDJUHDWGHDORIZKDWZDVVDLGWKHUHDOVRDSSOLHVWRTXDOLWDWLYH
RUPL[HGPHWKRGVVWXGLHV7KLVLVSDUWO\EHFDXVHVRPHRIWKHDFDGHPLFZULWLQJ
SULQFLSOHVDUHXQLYHUVDODQGSDUWO\EHFDXVHLQPDQ\ZD\VWKHTXDQWLWDWLYHZULWLQJ
IRUPDWFDQEHVHHQDVWKHGHIDXOWLQWKHVRFLDOVFLHQFHV²DQH[DPSOHRIWKLVLVWKHIDFW
WKDWWKH$3$VW\OHUHVHDUFKUHSRUWVWUXFWXUHLVEHLQJDGRSWHGZLWKPRGLILFDWLRQVE\DQ
LQFUHDVLQJQXPEHURITXDOLWDWLYHUHVHDUFKHUV VHHEHORZ ,QRUGHUWRDYRLGDQ\
RYHUODSVWKHIRFXVLQWKLVFKDSWHUZLOOEHRQKRZTXDOLWDWLYHDQGPL[HGPHWKRGV
UHSRUWVGLIIHUIURPTXDQWLWDWLYHUHSRUWVZLWKRXWUHSHDWLQJWKHFRPPRQIHDWXUHV IRU
H[DPSOHWKHFKDUDFWHULVWLFVRIWKHWLWOHWKHDEVWUDFWRUWKHUHIHUHQFHV
:ULWLQJDTXDOLWDWLYHUHSRUW
$QLQWHUYLHZUHSRUWVKRXOGLGHDOO\EHDEOHWROLYHXSWRDUWLVWLFGHPDQGVRI
H[SUHVVLRQDVZHOODVWRWKHFURVVH[DPLQDWLRQRIWKHFRXUWURRP
.YDOH
7KHUHLVDZHDOWKRIOLWHUDWXUHDYDLODEOHRIIHULQJJXLGHOLQHVIRUZULWLQJXSTXDOLWDWLYH
UHVHDUFK7KHIDFWWKDWERWKDXWKRUVDQGSXEOLVKHUVIHHOWKDWZHQHHGSOHQW\RIVXFK
DGYLFHLVDQDFFXUDWHUHIOHFWLRQRIWKHSUREOHPDWLFQDWXUHRISUHSDULQJDTXDOLWDWLYH
UHSRUW:HVDZLQWKHSUHYLRXVFKDSWHUWKDWZULWLQJXSDTXDQWLWDWLYHVWXG\LVUHODWLYHO\
VWUDLJKWIRUZDUGZLWKYDULRXVWHPSODWHVDQGFRQYHQWLRQVJXLGLQJRXUWDVN,QFRQWUDVW
VLPLODUWRDOORWKHUDVSHFWVRITXDOLWDWLYHUHVHDUFKTXDOLWDWLYHUHVHDUFKZULWLQJLV
FKDUDFWHUL]HGE\PRUHIUHHGRPGLYHUVLW\DQGGLVDJUHHPHQWWKDQLWVPRQROLWKLF
TXDQWLWDWLYHFRXQWHUSDUW7KLVLVZK\'DYLHV IRUH[DPSOHVWDWHVWKDWLQKHUYLHZ
WKHEHVWZD\WROHDUQKRZTXDOLWDWLYHUHVHDUFKILQGLQJVDUHUHSRUWHGLVWRUHDGMRXUQDO
DUWLFOHVDQGERRNFKDSWHUVWKDWPRGHOWKLVSDUWLFXODUJHQUH:KLOH,DJUHHWKDWIROORZLQJ
PRGHOVLVDQHIIHFWLYHZD\RIEHLQJVRFLDOL]HGLQWRWKHUROHRIWKHTXDOLWDWLYHZULWHULQ
WKLVFKDSWHU,VXPPDUL]HDIHZSULQFLSOHVDQGJXLGHOLQHVWKDWFDQIDFLOLWDWHDQGVSHHGXS
WKLVVRFLDOL]DWLRQSURFHVV
0DLQGLIIHUHQFHVEHWZHHQTXDOLWDWLYHDQGTXDQWLWDWLYH
UHVHDUFKUHSRUWV
:HGRQRWKDYHWRFRQGXFWDWKRURXJKUHYLHZRIDFDGHPLFSDSHUVWRUHDOL]HWKDW
TXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKUHSRUWVFDQEHYHU\GLIIHUHQWERWKLQWKHLU
RUJDQL]DWLRQDQGWKHLUVW\OHDQGHYHQLQWKHLUOHQJWK$OWKRXJKZKROHERRNVKDYHEHHQ

ZULWWHQRQWKLVWRSLF,ZRXOGVXJJHVWWKDWDJUHDWGHDORIWKLVGLIIHUHQFHFDQEH
H[SODLQHGRQWKHEDVLVRIWKHIROORZLQJVL[SRLQWV
'UDZLQJRQSUHYLRXVZULWLQJ
,Q&KDSWHU 6HFWLRQ ,DUJXHGWKDWEHLQJDZULWHULVDQLQWULQVLFSDUWRID
UHVHDUFKHU¶VSURIHVVLRQDOLGHQWLW\DQGRQO\E\UHJXODUO\ZULWLQJFDQZHWDNHRXU
UHVHDUFKLGHDVWRPDWXULW\7KLVLVLQFUHDVLQJO\VRLQTXDOLWDWLYHUHVHDUFKZKHUHZULWLQJ
LVVHHQDVDQLQGLVSHQVDEOHPHDQVRILQTXLU\4XDOLWDWLYHUHVHDUFKHUVVWDUWZULWLQJ
PHPRVYLJQHWWHVDQGRWKHUIRUPVRIVXPPDULHV VHH6HFWLRQ DVZHOODV
SHUVRQDOUHVHDUFKMRXUQDOV 6HFWLRQ VRRQDIWHUEHJLQQLQJWKHLUSURMHFWVDQGNHHS
ZULWLQJVWHDGLO\DQGUHJXODUO\ZKLOHWKH\UHFRUGDQGFODULI\WKHLUXQGHUVWDQGLQJRIWKH
GDWD7KXVDV6DQGHORZVNL SRLQWVRXWLQTXDOLWDWLYHUHVHDUFKWKHZULWHXSLV
FRQFHLYHGOHVVDVDQHQGSURGXFWRILQTXLU\WKDQDQLQTXLU\LQWKHPDNLQJEHFDXVH
ZULWLQJWRUHSRUWILQGLQJVW\SLFDOO\OHDGVUHVHDUFKHUVWRWKHJHQHUDWLRQRIPRUHLGHDV
DQGVXEVHTXHQWO\PRUHZULWLQJ
2IFRXUVHWKHSURFHVVFDQQRWJRRQIRUHYHUDQGDWRQHSRLQWZHQHHGWRZUDSLWXS
DQGSURGXFHDILQDOUHSRUW+RZHYHUE\WKDWWLPHZHKDYHDFFXPXODWHGDJUHDWGHDORI
FRPSOHWHGZULWLQJLQYDULRXVIRUPVDQGWKHUHIRUHWKHPDLQIXQFWLRQRIWKHILQDOZULWLQJ
XSSURFHVVLVWRDODUJHH[WHQWWRWKUHDGWKHVHWRJHWKHULQWRDFRKHUHQWRUGHU,QIDFWLWLV
GLIILFXOWWRLPDJLQHKRZDQ\JRRGTXDOLWDWLYHUHSRUWFRXOGHPHUJHZLWKRXWGUDZLQJ
H[WHQVLYHO\RQRXUSUHYLRXVZULWLQJV
1RIL[HGIRUPDWVRUWHPSODWHV
+RZGRZHµWKUHDG¶WKHYDULRXVELWVDQGSLHFHVWRJHWKHU"8QIRUWXQDWHO\IRUVRPHDQG
IRUWXQDWHO\IRURWKHUV GHSHQGLQJRQRQH¶VSUHIHUHQFH LQTXDOLWDWLYHUHSRUWLQJWKHUHDUH
QRIL[HGIRUPDWVRUWHPSODWHV RULQ0LOHVDQG+XEHUPDQ¶VZRUGVQR
µVKDUHGFDQRQV¶RIKRZRXUVWXGLHVVKRXOGEHUHSRUWHG 7KLVTXDOLW\KDVDGHILQLWH
SRVWPRGHUQIHHODERXWLW DQGVRPHVFKRODUVZRXOGHTXDWHTXDOLWDWLYHZULWLQJZLWK
SRVWPRGHUQZULWLQJVHH+HVVH%LEHUDQG/HDY\ EXWLQIDFWLWGRHVQRWRULJLQDWH
ZLWKSRVWPRGHUQLVPEXWUDWKHUZLWKWKHLWHUDWLYHHPHUJHQWQDWXUHRIWKHTXDOLWDWLYH
LQTXLU\WKDWSUHFHGHVSRVWPRGHUQLVP7KHFDQRQLFDOWHPSODWHRIWKHTXDQWLWDWLYH
UHVHDUFKUHSRUWLVWKHFU\VWDOOL]DWLRQRIGHFDGHVRIH[SHULHQFHRQKRZEHVWWRSUHVHQWWKH
HQGSURGXFWRIDOLQHDUSURFHVV2QWKHRWKHUKDQGDTXDOLWDWLYHLQYHVWLJDWLRQLVRIWHQ
PDUNHGO\QRQOLQHDUDQGWKHUHIRUHLPSRVLQJWKHWHPSODWHGHVFULEHGLQWKHODVWFKDSWHU
RQLWLVVHHQE\PDQ\DVFRXQWHUSURGXFWLYH&RQVHTXHQWO\RYHUWKHSDVWWKUHHGHFDGHV
VHYHUDODOWHUQDWLYHRIWHQQRQFRQYHQWLRQDOJHQUHVKDYHEHHQH[SHULPHQWHGZLWKLQ
RUGHUWRIXOO\FRPPXQLFDWHWKHILQGLQJVRITXDOLWDWLYHVWXGLHVWKHVHLQFOXGHOLWHUDU\
PRGHVVXFKDVQRYHOVDQGSRHPVDVZHOODVGUDPDDQGGDQFH 6DQGHORZVNLDQG
%DUURVR 7KHTXHVWLRQRIKRZIDUZHFDQRUVKRXOGGHYLDWHIURPWKHJHQHULF
QRUPVRIDFDGHPLFZULWLQJWKDWRULJLQDWHZLWKDGLIIHUHQW LHTXDQWLWDWLYH DSSURDFKLV
DKLJKO\FRQWHVWHGLVVXHHYHQLQTXDOLWDWLYHFLUFOHVDQGZHZLOOFRPHEDFNWRLWLQ
6HFWLRQEHORZ

:RUGVUDWKHUWKDQQXPEHUVXVHGDVHYLGHQFH
$FRPPRQIHDWXUHRIDQ\NLQGRISULPDU\UHVHDUFKLVWKDWHPSLULFDOGDWDLVXVHGWR
VXSSRUWDUJXPHQWVDQGFODLPV,QTXDQWLWDWLYHUHVHDUFKWKLVHYLGHQFHLVDOPRVWHQWLUHO\
QXPEHUEDVHGVWDWLVWLFV$OWKRXJKQXPEHUVFDQDOVREHXVHGLQTXDOLWDWLYHUHVHDUFKWKH
GRPLQDQWIRUPRI48$/HYLGHQFHLQYROYHVH[WUDFWVIURPZRUGEDVHGQDUUDWLYH
DFFRXQWVRIHLWKHUWKHUHVSRQGHQWVRUWKHUHVHDUFKHUKLPKHUVHOI IRUH[DPSOHILHOG
QRWHVRUPHPRV 7KXVFUHGHQFHLVDFKLHYHGE\LOOXVWUDWLQJDQGYDOLGDWLQJWKH
UHVHDUFKHU¶VLQIHUHQFHVZLWKDUHSUHVHQWDWLYHUDQJHRIORZLQIHUHQFHGHVFULSWRUVWKDW
EULQJWKHSDUWLFXODUVLWXDWLRQRUSKHQRPHQRQDOLYH7KHDLPLVWRSURYLGHDULFKDQG
YLYLGGHVFULSWLRQVRWKDWµWKHUHDGHUFDQYLFDULRXVO\H[SHULHQFHZKDWLWLVOLNHWREHLQ
WKHVDPHVLWXDWLRQDVWKHUHVHDUFKSDUWLFLSDQWV¶ -RKQVRQDQG&KULVWHQVHQ
*UHDWHUOHQJWK
2QHJRRGWKLQJDERXWQXPEHUVLVWKDWWKH\FDQEHVXPPDUL]HGLQDVPDOOVSDFH²ZH
FDQRIWHQSUHVHQWDOOWKHPDLQILQGLQJVRIDVWXG\LQWKUHHRUIRXUWDEOHV,QFRQWUDVW
ZRUGVDUHUDWKHUµZRUG\¶DQGWRSURYLGHDVXIILFLHQWQXPEHURITXRWHVWDNHVXSDORWRI
VSDFHPDNLQJTXDOLWDWLYHUHSRUWVXVXDOO\FRQVLGHUDEO\ORQJHUWKDQWKHLUTXDQWLWDWLYH
FRXQWHUSDUWV²VHH0DJQDQ*DOOHWDO HVWLPDWHVWKDW48$/UHSRUWVWHQGWR
EHDOPRVWWZLFHDVORQJDV48$1UHSRUWV7KXVTXDOLWDWLYHUHVHDUFKHUVDUHFRQVWDQWO\
EHWZHHQDURFNDQGDKDUGSODFHRQWKHRQHKDQGWKH\QHHGWRSURYLGHULFKDQG
SHUVXDVLYHQDUUDWLYHVXSSRUWZLWKLQWKHIL[HGZRUGOLPLWVWKDWDUHLPSRVHGE\DOPRVWDOO
W\SHVRISXEOLFDWLRQVDQGRQWKHRWKHUKDQGHYHQE\VWUHWFKLQJWKHOLPLWV DQGWKH
HGLWRUV¶JRRGZLOO WRWKHPD[LPXPRQO\DWLQ\SRUWLRQRIWKHUDZGDWDFDQPDNHLWLQWR
WKHILQDOUHSRUW
7HOOLQJDVWRU\
$VVWDWHGLQWKHSUHYLRXVFKDSWHU 6HFWLRQ ,DPYHU\PXFKLQIDYRXURIKDYLQJD
OXFLGVWRU\OLQHLQDQ\ZULWLQJLQFOXGLQJUHVHDUFKUHSRUWVDQGWKHTXDQWLWDWLYHIRUPDW
PDNHVLWUHODWLYHO\HDV\WREHFRKHUHQW$IWHUDOOWKHVWUXFWXUHLWVHOISUHVHQWVDQ
RYHUDUFKLQJSURFHVVWKHWDUJHWLVVXHLVLQWURGXFHGDQGJURXQGHGLQWKHOLWHUDWXUH
UHYLHZWKHQVSHFLILFUHVHDUFKTXHVWLRQVRUK\SRWKHVHVDUHSUHVHQWHGWRZKLFKFOHDUFXW
DQVZHUVDUHJLYHQXVLQJFRQFLVHVWDWLVWLFDOVXSSRUWDQGILQDOO\WKHILQGLQJVDUH
VXPPDUL]HG2QHFDQKDUGO\JRZURQJE\DGKHULQJWRWKLVWHPSODWH,QFRQWUDVW
TXDOLWDWLYHDFFRXQWVDUHORQJHUDQGFRQWDLQIDUULFKHUGHWDLOVDUHEDVHGRQDQLWHUDWLYH
DQGUHFXUVLYHGDWDFROOHFWLRQDQDO\VLVSURFHVVDQGRIWHQGHVFULEHPXOWLSOHPHDQLQJV
7KHRQO\ZD\WRSUHVHQWWKLVZHOOLVE\EHFRPLQJJRRGVWRU\WHOOHUVµ:KHQZHFDQQRW
HQJDJHRWKHUVWRUHDGRXUVWRULHV²RXUFRPSOHWHGDQGFRPSOHWHDFFRXQWV²WKHQRXU
HIIRUWVDWGHVFULSWLYHUHVHDUFKDUHIRUQDXJKW¶ :ROFRWW 7KXVWKHVXFFHVVRI
UHSRUWLQJRXUTXDOLWDWLYHILQGLQJVGHSHQGVWRDODUJHH[WHQWRQKRZPXFKZHPDQDJHWR
ZUDSXSRXUPHVVDJHLQDQDEVRUELQJVWRU\
5HIOH[LYLW\VW\OHDQGWRQH

$OWKRXJKTXDQWLWDWLYHUHVHDUFKPLJKWKDYHWKHHGJHRYHUTXDOLWDWLYHUHSRUWVLQWHUPVRI
VXJJHVWLQJDQREYLRXVVWRU\OLQHTXDOLWDWLYHUHVHDUFKFRPSHQVDWHVIRUWKLVE\
DOORZLQJ²LQIDFWHQFRXUDJLQJ²ULFKGHWDLODVZHOODVDSHUVRQDOVW\OHDQGDQLQIRUPDO
WRQH,DUJXHGLQWKHSUHYLRXVFKDSWHUWKDWDTXDQWLWDWLYHUHSRUWGRHVQRWQHHGWREH
LPSHUVRQDODQGIRUPDO VHH6HFWLRQ \HWWKHIDFWUHPDLQVWKDWTXDQWLWDWLYH
UHVHDUFKHUVRIWHQIROORZWKHVW\OHRIWKHVWXGLHVFRPPRQLQWKHQDWXUDOVFLHQFHVLQ
ZKLFKPRVWRIWKHPHVVDJHLVFRQYH\HGE\ILJXUHVDQGIDFWVUDWKHUWKDQE\WKHZULWHU¶V
RZQYRLFH,QFRQWUDVWEHFDXVHWKHUHVXOWVRIDTXDOLWDWLYHVWXG\DUHXVXDOO\VHHQDVD
FRFRQVWUXFWHGSURGXFWRIWKHSDUWLFLSDQWV¶DQGWKHUHVHDUFKHU¶VSHUFHSWLRQVWKH
TXDOLWDWLYHJHQUHLQYROYHVUHIOH[LYLW\WKDWLVLQFOXGLQJDGLVFXVVLRQRIWKHZULWHU¶V
ELDVHVYDOXHVDQGDVVXPSWLRQVLQWKHWH[W$VDFRQFUHWHUHVXOWRIWKLVTXDOLWDWLYH
UHVHDUFKHUVWHQGWRXVHWKHILUVWSHUVRQVLQJXODUZKHQWDONLQJDERXWWKHPVHOYHVUDWKHU
WKHQWKHLPSHUVRQDOµWKHUHVHDUFKHU¶ ZKLFKLVDFWXDOO\EHFRPLQJRXWGDWHGHYHQLQ
TXDQWLWDWLYHUHSRUWV
7KXVZLWKWKHHPSKDVLVRQUHIOH[LYLW\DQGUHVHDUFKHULQYROYHPHQWTXDOLWDWLYH
UHVHDUFKRIIHUVZULWHUVWKHIUHHGRPWRKDYHWKHLURZQDVZHOODVWKHLUSDUWLFLSDQWV¶
YRLFHVKHDUGZKLFKFDQEHWXUQHGLQWRDSRZHUIXOSUHVHQWDWLRQWRRO*LYHQWKDWWKHXVH
RIH[SUHVVLYHODQJXDJHWHQGVWREHDOVRKLJKO\SUL]HGDPRQJTXDOLWDWLYHUHVHDUFKHUV
6DQGHORZVNL WKHTXDOLWDWLYHZULWHUKDVDOOWKHLQJUHGLHQWVDWKLVKHUGLVSRVDOWR
SURGXFHDYLYLGGHWDLOULFKGUDPDWL]HGVWRU\ZKLFKFDQEHDIDUEHWWHUUHDGWKDQD
UHSRUWRIWKHUHVXOWVRIDPXOWLSOHDQDO\VLVRIYDULDQFH$FFRUGLQJO\+ROOLGD\
WDONVDERXWWKHHPHUJHQFHRIDµQHZWKLQNLQJHVSHFLDOO\ZLWKLQSURJUHVVLYH
TXDOLWDWLYHUHVHDUFKZKLFKSURYLGHVVFRSHIRUUHVHDUFKHUVDVZULWHUVWRXVHWKH
FRQYHQWLRQVWRHVWDEOLVKDVWURQJSHUVRQDOSUHVHQFHLQWKHJHQUH¶
7KHVWUXFWXUHRIDTXDOLWDWLYHUHSRUW
9DULDELOLW\DQGFUHDWLYHIUHHGRPDVZHOODVWKHFRQVFLRXVHIIRUWWRDYRLGULJLGIRUPXODH
DUHKDOOPDUNVRITXDOLWDWLYHLQTXLULHVDWHYHU\SKDVHRIWKHUHVHDUFKSURFHVVLQFOXGLQJ
WKHSUHSDUDWLRQRIWKHUHVHDUFKUHSRUW+RZHYHULIZHORRNDWWKLVFORVHO\ZHILQGWKDW
WKHIUHHGRPRIH[SUHVVLRQLVQRWXQOLPLWHGEHFDXVHWKHUHDUHFHUWDLQDVSHFWVRIWKH
UHVHDUFKSURMHFWWKDWPXVWEHFRYHUHGLQDUHSRUWUHJDUGOHVVRIWKHVSHFLILFVWUXFWXUHZH
XVH7KHVHµFRPSXOVRU\¶SDUWVLQFOXGHWKHLQWURGXFWLRQDQGMXVWLILFDWLRQRIWKHWRSLFWKH
GHVFULSWLRQRIWKHUHVHDUFKPHWKRGRORJ\ GDWDFROOHFWLRQDQGDQDO\VLV WKHSUHVHQWDWLRQ
RIWKHILQGLQJVDQGKRZWKH\DUHUHODWHGWRRWKHUVFKRODUV¶ILQGLQJVLQWKHILHOGDQG
ILQDOO\VRPHVRUWRIDFRQFOXVLRQ4XLWHIUDQNO\WKHVHFRPSRQHQWVDUHUDWKHUVLPLODUWR
WKHPDLQSDUWVRIWKHTXDQWLWDWLYHUHSRUWGHVFULEHGLQWKHSUHYLRXVFKDSWHUDQGHYHQ
WKRXJKTXDOLWDWLYHVWXGLHVPD\QRWXVHWKHFRQYHQWLRQDOLVHGVXEKHDGLQJVRIWKH
TXDQWLWDWLYHUHVHDUFKUHSRUWDQGPD\YDU\WKHRUGHURIWKHFRPSRQHQWV SDUWLFXODUO\WKH
SODFHRIWKHOLWHUDWXUHUHYLHZLVIOH[LEOH LWLVIDLUWRVD\WKDWDTXDOLWDWLYHUHSRUWLQ
HIIHFWFRYHUVWKHVDPHJURXQGDVLWVTXDQWLWDWLYHFRXQWHUSDUW

2IFRXUVHFRYHULQJWKHVDPHJURXQGGRHVQRWPHDQWKDWZHKDYHWRGRLWLQWKH
VDPHZD\DQGWKHUHIRUHWKHPDLQTXHVWLRQFRQFHUQLQJTXDOLWDWLYHZULWLQJLVKRZPXFK
YDULDWLRQDQGDWZKLFKOHYHOLVGHVLUDEOH/HWPHVKDUHDWWKLVVWDJHP\SHUVRQDOYLHZ
$ORQJZLWKDQXPEHURIUHVHDUFKHUV IRUH[DPSOH-RKQVRQDQG&KULVWHQVHQ
0RUURZ ,ZLOODGYRFDWHEHORZDUDWKHUFRQVHUYDWLYHVWDQFHVXJJHVWLQJWKDWZH
DGRSWDVOLJKWO\PRGLILHGYHUVLRQRIWKH$3$VW\OHTXDQWLWDWLYHVWUXFWXULQJIRUPDWIRU
TXDOLWDWLYHUHSRUWVWRWKHH[WHQWWKDWZHHYHQXVHPRVWRIWKHVXEKHDGLQJVUHFRPPHQGHG
E\WKH$3$VW\OHPDQXDO7KHUHDVRQVIRUWKLVDUHWZRIROG
0\ILUVWUHDVRQLVDGPLWWHGO\SUDJPDWLF5HVHDUFKHUVGRQRWH[LVWLQDYDFXXPDQG
WKHUHIRUHWKH\QHHGWREHDZDUHRIWKHQRUPDWLYHLQIOXHQFHVRIWKHUHVHDUFK
HQYLURQPHQWWKH\ZRUNLQ7KXV,ZRXOGQRWUHFRPPHQGXVLQJXQWUDGLWLRQDO
IRUPDWVVLPSO\EHFDXVH,KDYHQHYHUKHDUGRIVD\DSRHPEHLQJDFFHSWHGIRUD
SRVWJUDGXDWHGLVVHUWDWLRQWKHVLVLQDSSOLHGOLQJXLVWLFVDQGVLPLODUO\DUHVHDUFKHU
DQGHVSHFLDOO\DQRQHVWDEOLVKHGUHVHDUFKHU ZLOOVWDQGOLWWOHFKDQFHRISXEOLVKLQJ
KLVKHUUHVXOWVLQDPDLQVWUHDPIRUXPLIKHVKHGRHVQRWFRQIRUPWRWKHSUHYDLOLQJ
FRQYHQWLRQV
7KHVHFRQGUHDVRQKDVWRGRZLWKWKHZD\µFUHDWLYHIUHHGRP¶LVFRQFHSWXDOL]HG
ZLWKLQWKHUHVHDUFKZULWLQJFRQWH[W7KHFHQWUDOTXHVWLRQLVZKHWKHUIUHHGRPFDQEH
H[HUFLVHGZLWKLQDIL[HGIUDPHZRUNRUZKHWKHUWKHFRQYHQWLRQDOUHVHDUFKUHSRUW
JHQUHLVWRRUHVWULFWLYHWRDOORZWKHH[SUHVVLRQRIFHUWDLQPHDQLQJ7KHSDLQWHU
9LQFHQWYDQ*RJKLVVDLGWRKDYHFRPSODLQHGRQFHDERXWWKHGLIILFXOWVLWXDWLRQRI
PRGHUQDUWLVWVLQWKHVHQVHWKDWWKHUHZHUHQRERXQGDULHVDQ\PRUHIRUWKHPWR
VWUHWFKZKHUHDVLQWKHSDVWYHU\VXEWOHPHDQLQJFRXOGEHH[SUHVVHGE\DUWLVWV
FRPSO\LQJZLWKDQGDWWKHVDPHWLPHIORXWLQJWKHQRUPDWLYHH[SHFWDWLRQVRIWKHLU
DJH,QDVLPLODUYHLQ,EHOLHYHWKDWDODFNRIERXQGDULHVPD\KDYHDGHWULPHQWDO
HIIHFWDQGDGKHULQJWRDV\VWHPFDQDFWXDOO\EHOLEHUDWLQJ
,ZRXOGVXJJHVWWKHUHIRUHWKDWE\DGRSWLQJWKHPRVWUHFRJQL]DEOHDFDGHPLF
RUJDQL]LQJV\VWHPWKH$3$VWUXFWXULQJIRUPDWTXDOLWDWLYHUHVHDUFKHUVFDQFUHDWHD
FHUWDLQDPRXQWRIOHHZD\IRUWKHPVHOYHVEHFDXVHWKHUHDGHU¶VIDPLOLDULW\ZLWKWKHVH
FRQYHQWLRQVKHOSVWRFDUU\VXVWDLQDQGDQFKRUDQ\XQFRQYHQWLRQDOFRQWHQWHOHPHQWV
VXFKDVDQRWDOZD\VHQWLUHO\DFDGHPLFVW\OH)XUWKHUPRUHJHWWLQJULGRIWUDGLWLRQDO
IRUPDWFRQYHQWLRQVDQGVHWWLQJRXWWRUHSRUWRXUILQGLQJVLQDPRUHOLWHUDU\IRUPVXFKDV
DSRHPIRUH[DPSOHZRXOGEHIHOWDVLQKLELWLYHE\PDQ\TXDOLWDWLYHZULWHUVHVSHFLDOO\
E\WKRVHZKRDUHQHZWRWKHILHOG,DPLQDJUHHPHQWZLWK5LFKDUGV WKDW
KDYLQJQRWHPSODWHWRILOOLQµLVRIWHQDPDMRUREVWDFOHWRZULWLQJ¶DQG,GREHOLHYHWKDW
WKHVDPHLGHDKDVPRWLYDWHG5LFKDUGVRQDQG6W3LHUUH ZKHQWKH\
VXJJHVWHGWKDWLQFRQVLGHULQJZULWLQJIRUPDWVUHVHDUFKHUVVKRXOGVWDUWZLWKDµMRXUQDO
DUWLFOHWKDWH[HPSOLILHVWKHPDLQVWUHDPZULWLQJFRQYHQWLRQVRI\RXUGLVFLSOLQH¶
/HWXVQRZORRNDWWKRVHDVSHFWVRIWKH$3$VW\OHTXDQWLWDWLYHUHVHDUFKUHSRUWWKDW
PD\UHTXLUHVRPHPRGLILFDWLRQWREHDEOHWRDFFRPPRGDWHWKHGHVFULSWLRQRID

TXDOLWDWLYHLQTXLU\ )RUDOLVWRIHODERUDWHVXJJHVWLRQVVHHWKHDSSHQGL[RI0RUURZ

,QWURGXFWLRQOLWHUDWXUHUHYLHZDQGUHVHDUFKTXHVWLRQV
7KHUROHRIWKH,QWURGXFWLRQDUHVLPLODULQTXDOLWDWLYHDQGTXDQWLWDWLYHVWXGLHVH[FHSW
IRURQHDUHDWKHOLWHUDWXUHUHYLHZ7KHTXDQWLWDWLYHORJLFUHTXLUHVVXEVWDQWLDOLQLWLDO
FRYHUDJHRIWKHOLWHUDWXUHEHFDXVHWKHUHVHDUFKTXHVWLRQVWKHUHVHDUFKGHVLJQDQGWKH
LQVWUXPHQWVDUHDOOMXVWLILHGE\DSULRULWKHRUHWLFDONQRZOHGJH4XDOLWDWLYHVWXGLHVRQ
WKHRWKHUKDQGDUHRIWHQH[SORUDWRU\DQGRSHQHQGHGDQGHYHQWKRXJKIHZTXDOLWDWLYH
UHVHDUFKHUVZRXOGDGRSWDµWDEXODUDVD¶RULHQWDWLRQGHWDLOHGEDFNJURXQGNQRZOHGJHDW
WKHEHJLQQLQJVWDJHLVQRWVHHQDVHVVHQWLDOLQJURXQGHGWKHRU\IRUH[DPSOHWKH
H[LVWLQJOLWHUDWXUHLVRQO\WREHVXUYH\HGLQWKHILQDODQDO\WLFDOVWDJH LHLQVHOHFWLYH
FRGLQJVHH6HFWLRQ WRHYDOXDWHWKHHPHUJLQJILQGLQJVRUµWKHRULHV¶DJDLQVWLW
+DYLQJVDLGWKDW,VWLOOIHHOWKDWHYHQDQH[SORUDWRU\HPHUJHQWTXDOLWDWLYHVWXG\QHHGV
DQLQLWLDOSHUKDSVOHVVVXEVWDQWLDOOLWHUDWXUHUHYLHZWRSURYLGHDWKHRUHWLFDOFRQWH[WIRU
WKHLQYHVWLJDWLRQDQGWRMXVWLI\WKHUHVHDUFKWRSLFEXWLWPLJKWDOVREHOHJLWLPDWHWR
LQWURGXFHIXUWKHUOLWHUDWXUHODWHULQWKH'LVFXVVLRQVHFWLRQ)LQDOO\WKHUHLVDQRWKHUDUHD
LQWKH,QWURGXFWLRQZKHUH48$/DQG48$1UHSRUWVGLIIHUWKHFKDUDFWHULVWLFVRIWKH
UHVHDUFKTXHVWLRQV²WKLVZDVGLVFXVVHGLQ6HFWLRQ
0HWKRG
7KHPDLQIXQFWLRQRIWKH0HWKRGVHFWLRQLVWKHVDPHDVLQWKH$3$VW\OH²QDPHO\WR
SURYLGHWKHWHFKQLFDOLQIRUPDWLRQWKDWLVQHFHVVDU\IRUWKHHYDOXDWLRQRIWKHVXEVHTXHQW
ILQGLQJVDQGWRVHUYHDVDUHIHUHQFHVHFWLRQ²EXWWKHGLIIHUHQWQDWXUHRITXDOLWDWLYHDQG
TXDQWLWDWLYHPHWKRGRORJLHVZDUUDQWFHUWDLQXVHIXODGGLWLRQVDQGPRGLILFDWLRQV 0RUURZ

x %HIRUHLQWURGXFLQJWKHSDUWLFLSDQWVLWLVXVHIXOWRLQFOXGHDVHFWLRQRQµ5HVHDUFK
DSSURDFKDQGGHVLJQ¶GHVFULELQJWKHDVVXPSWLRQVDQGSULQFLSOHVXQGHUSLQQLQJWKH
UHVHDUFKDQGWKHW\SHRITXDOLWDWLYHPHWKRGRORJ\DSSOLHGLQWKHVWXG\
x %HFDXVHRIWKHVLWXDWHGQDWXUHRITXDOLWDWLYHUHVHDUFKWKHGHVFULSWLRQRIWKH
SDUWLFLSDQWVQHHGVWREHSDUWLFXODUO\ULFKDQGWKHVDPSOLQJVWUDWHJ\PXVWEH
H[SOLFLWO\H[SODLQHG
x 0RUURZ VXJJHVWVDXVHIXOPRGLILFDWLRQZKHUHE\ZHPHUJHWKHLQVWUXPHQWV

DQGSURFHGXUHVVHFWLRQVXQGHUWKHKHDGLQJRIµ6RXUFHVRIGDWD¶VRWKDWWKHYDULRXV
LWHUDWLYHVWHSVWKDWKDYHEHHQWDNHQFDQEHEHWWHUGHVFULEHGDQGH[SODLQHG
x $QLPSRUWDQWDVSHFWRIWKHµ'DWDDQDO\VLV¶LVWRGHVFULEHKRZWKHUHFRUGLQJVZHUH
WUDQVFULEHGLQFOXGLQJWKHPDLQIHDWXUHVRIWKHµWUDQVFULSWLRQSURWRFRO¶²VHH6HFWLRQ
7KLVVHFWLRQPLJKWDOVREHWKHSODFHWRDGGUHVVWKHµUHVHDUFKHUDV
LQVWUXPHQW¶LVVXHIRFXVLQJRQFHUWDLQFKDUDFWHULVWLFVRIWKHUHVHDUFKHU IRUH[DPSOH
ELDVHVDQGGLVFLSOLQDU\EDFNJURXQG WKDWPD\KDYHDIIHFWHGWKHDQDO\VLV:HDOVR
QHHGWRSURYLGHLQIRUPDWLRQRQWKHVSHFLILFDSSURDFKRIGDWDDQDO\VLVDSSOLHG

LQFOXGLQJWKHQDWXUHRIWKHYDULRXVFRGLQJSKDVHV$FRPPRQVKRUWFRPLQJRI
TXDOLWDWLYHVWXGLHVLVWKDWLWLVQRWWUDQVSDUHQWKRZWKHLQIHUHQFHVZHUHGHULYHGIURP
WKHUDZGDWD+XEHUPDQDQG0LOHV [ IRUH[DPSOHSRLQWRXWWKDW48$/
0HWKRGVHFWLRQVDUHVRPHWLPHVQRPRUHWKDQµVKRSSLQJOLVWVRIGHYLFHVFRPPRQO\
FLWHGLQWKHOLWHUDWXUH«DVRUWRIEX]]ZRUGPHWKRGRORJ\ZLWKVFDQWGHPRQVWUDWLRQ
RIWKHSURFHVVHVLQYROYHGLQDQDO\]LQJSDUWLFXODUVHWVRIGDWD¶7RFUHDWHUHDO
WUDQVSDUHQF\ZHQHHGWRVXSSO\NH\HOHPHQWVRIDQµDXGLWWUDLO¶:LWKWKHXVHRI
FRPSXWHUDLGHGTXDOLWDWLYHGDWDDQDO\VLV &$4'$6VHH6HFWLRQ WKLV
SRZHUIXOYDOLGDWLRQVWUDWHJ\KDVEHFRPHUHODWLYHO\HDV\WRFDUU\RXW VHHEHORZ
x (OHFWURQLFDXGLWWUDLO%ULQJHUHWDO SURYLGHGHWDLOHGJXLGHOLQHVRQWKHNLQG
RILOOXVWUDWLRQVWKDWZRUNEHVWLQRXWOLQLQJWKHDXGLWWUDLO7KHVHLQFOXGHVFUHHQ
SULQWVRIFRGLQJFDWHJRULHVSHUVRQDOMRXUQDOSDJHVDQGH[DPSOHVIURPRSHQDQG
D[LDOFRGLQJLQDJURXQGHGWKHRU\DSSURDFK 6FUHHQSULQWVFDQEHHDVLO\FUHDWHGE\
SUHVVLQJWKHµ3ULQW6FUHHQ¶NH\RQWKHNH\ERDUGDQGWKHQSDVWLQJWKHVWDWLFSLFWXUH
WKHFRPSXWHUKDVWDNHQRIWKHFXUUHQWVFUHHQLQWRWKHZRUGSURFHVVRUGRFXPHQW ,Q
DGGLWLRQH[DPSOHVRITXRWHVIRUHDFKPDLQFRGLQJFDWHJRU\SURYLGHSRZHUIXO
LOOXVWUDWLRQVDQG%ULQJHUHWDODOVRVXJJHVWVXSSO\LQJDWDEOHWKDWGLVSOD\VWKH
QXPEHUDQGWKHW\SHVRIPHPRVXVHGDQGIRUZKDWSXUSRVH7KHSDUWLFXODU
&$4'$6VRIWZDUHWKHVHDXWKRUVXVHG19LYRKDVVSHFLDOµUHSRUWLQJ¶IXQFWLRQVWR
SURYLGHFDWHJRU\DQGGRFXPHQWOLVWLQJVDVZHOODVPRUHVSHFLILFLQIRUPDWLRQVXFK
DVIRUH[DPSOHKRZPXFKRIHDFKGRFXPHQWKDGEHHQFRGHGWKHVHFDQDOVREH
XVHGHIIHFWLYHO\WRGHVFULEHWKHGHYHORSPHQWRIWKHRYHUDOOSURMHFW
x )LQDOO\6HFWLRQOLVWHGDQXPEHURISUDFWLFDOVWUDWHJLHVWRHQVXUHYDOLGLW\LQ
TXDOLWDWLYHUHVHDUFK7KHGDWDDQDO\VLVVHFWLRQVKRXOGDOVRFRQWDLQVRPHGLVFXVVLRQ
RIWKHXVHRIVXFKVWUDWHJLHVLQRUGHUWRJHQHUDWHWUXVWZRUWKLQHVVDQGHOLPLQDWHRU
FRQWUROYDOLGLW\WKUHDWV
5HVXOWVDQGGLVFXVVLRQ
$VH[SODLQHGLQWKHSUHYLRXVFKDSWHUWKH5HVXOWVVHFWLRQPD\RUPD\QRWEHPHUJHG
ZLWKWKH'LVFXVVLRQVHFWLRQ0RUURZ UHFRPPHQGVWKDWWKHWZRVHFWLRQVEHNHSW
VHSDUDWHLQTXDOLWDWLYHUHVHDUFKDUWLFOHVWRHPSKDVL]HWKHGLVWLQFWLRQEHWZHHQULJRURXV
GDWDEDVHGLQIHUHQFHVDQGWKHUHVHDUFKHU¶VRZQFRQFOXVLRQVDERXWWKHILQGLQJV,Q
ORQJHUZRUNVVXFKDVSRVWJUDGXDWHWKHVHVGLVVHUWDWLRQVKRZHYHULWPLJKWDGGWRWKH
FODULW\RIWKHSUHVHQWDWLRQRIWKHILQGLQJVLIZHGLYLGHWKH5HVXOWVDQG'LVFXVVLRQLQWR
WKHPDWLFFKDSWHUVLQZKLFKWKHYDULRXVSKDVHVRIWKHGDWDDQDO\VLVDUHGHVFULEHG
WRJHWKHU,WDOVRDGGVWRWKHDFFHVVLELOLW\RIWKHGLVFXVVLRQ ERWKLQDUWLFOHVDQGLQ
WKHVHVGLVVHUWDWLRQV LIZHEUHDNGRZQWKHWH[WLQWRVXEVHFWLRQVZLWKWKHLURZQ
VXEKHDGLQJV7KHVHVXEKHDGLQJVDFWDVUHDGHUIULHQGO\VLJQSRVWV $ERXWWKH
LPSRUWDQFHRIVLJQSRVWLQJVHHDOVR6HFWLRQ
%HVLGHVWKHFRKHUHQWVWRU\OLQH ZKLFKKDVEHHQGLVFXVVHGLQ6HFWLRQ WKH
RWKHUNH\GHWHUPLQDQWRIWKHHIIHFWLYHQHVVRISUHVHQWLQJRXUILQGLQJVLVKRZZHVXFFHHG

LQNHHSLQJWKHEDODQFHEHWZHHQRXURZQLQWHUSUHWLYHFRPPHQWDU\DQGWKHVXSSRUWLQJ
HYLGHQFHLQWKHIRUPRITXRWHVIURPSDUWLFLSDQWVRUILHOGQRWHV 0RUURZ *RRG
TXDOLWDWLYHUHSRUWVGLVSOD\DVPRRWKO\IORZLQJQDWXUDOUK\WKPRIWH[WDQGTXRWHV²WRR
PDQ\TXRWHVGDPSHQWKHUHVHDUFKHU¶VYRLFHDQGPDNHKLVKHUDUJXPHQWGLIILFXOWWR
IROORZZKHUHDVWRRIHZTXRWHVPD\SURYLGHLQVXIILFLHQWVXSSRUWDVNLQJWKHUHDGHULQ
HIIHFWWRUHO\KHDYLO\RQWKHUHVHDUFKHU¶VZRUG0RUVHDQG5LFKDUGV HPSKDVL]H
WKDWZHVKRXOGDGGVRPHFRQWH[WXDOLQIRUPDWLRQWRWKHTXRWDWLRQVFRQFHUQLQJWKHVRFLDO
VLWXDWLRQDQGWKHFRPPXQLFDWLYHFRQWH[W7KHFKDOOHQJHLVWRDFKLHYHDWKLFN
GHVFULSWLRQZLWKRXWWKHVWXG\EHLQJRYHUO\GHVFULSWLYHEXWUDWKHUDQDO\WLFDODVDZKROH
&DQZHHGLWWKHTXRWDWLRQVLIQHHGEH"<HVEXWZLWKRXWGLVWRUWLQJRU
PLVUHSUHVHQWLQJWKHPHDQLQJZHQHHGDOVRWRIROORZFHUWDLQFRQYHQWLRQVPRVW
LPSRUWDQWO\LQGLFDWLQJ IRUH[DPSOHZLWKWKUHHGRWV ZKHUHVRPHWKLQJKDVEHHQOHIWRXW
EHFDXVHLWZDVUHGXQGDQWEULHIHUTXRWDWLRQVWHQGWREHPRUHSRZHUIXOWKDQOHQJWK\
PRQRORJXHV 0RUVHDQG5LFKDUGV $QLPSRUWDQWSXUSRVHRIWKHHGLWLQJLVWR
HQVXUHWKHDQRQ\PLW\RIWKHSDUWLFLSDQWVZKLFKPD\UHTXLUHPRUHWKDQVLPSO\
FKDQJLQJWKHQDPHV:LWKQRQQDWLYHUHVSRQGHQWVZHPD\DOVRHGLWPLQRUOLQJXLVWLF
LQDFFXUDFLHVWRIDFLOLWDWHUHDGLQJEXWLIZHGHFLGHWRGRVRZHVKRXOGQRWHWKLVFOHDUO\
DWWKHILUVWRFFXUUHQFHRULQWKH0HWKRGVHFWLRQ
,IZHGHFLGHWRKDYHDVHSDUDWH'LVFXVVLRQVHFWLRQWKLVLVWKHSODFHWRSXOOWKH
YDULRXVWKUHDGVWRJHWKHUDQGRIIHUELJSLFWXUHH[SODQDWLRQV6RPHTXDOLWDWLYHVWXGLHV
SUHVHQWDQDGGLWLRQDOUHYLHZRIWKHOLWHUDWXUHDWWKLVSRLQWWRHYDOXDWHWKHLQWHUSUHWDWLRQV
LQWKHOLJKWRIRWKHUVFKRODUV¶ILQGLQJV
&RQFOXVLRQ
2SLQLRQVDERXWWKHQHHGIRUD&RQFOXVLRQDQGZKDWLWVKRXOGFRQWDLQYDU\6LOYHUPDQ
IRUH[DPSOHDUJXHVIHUYHQWO\WKDWµ\RXVKRXOGQHYHUZULWHDVXPPDU\DV
\RXUFRQFOXGLQJFKDSWHU«UHVLVWWKHWHPSWDWLRQVRIDILQDOGRZQKLOOIUHHZKHHO¶$VKH
H[SODLQVE\WKHQRQHVKRXOGKDYHVDLGHYHU\WKLQJLQDFU\VWDOFOHDUPDQQHUDQG
WKHUHIRUHLIDUHLWHUDWLRQRIWKHILQGLQJVLVQHFHVVDU\WKHQVRPHWKLQJDERXWWKHSUHYLRXV
WH[WLVZURQJ,QVWHDGRIDVXPPDU\WKHUHIRUHKHVXJJHVWVWKDWWKH&RQFOXVLRQVKRXOG
EHXVHGWRUHIOHFWRQWKHRYHUDOOSURJUHVVLRQRIWKHVWXG\IURPWKHRULJLQDOUHVHDUFK
TXHVWLRQWRGLVFXVVDQ\OLPLWDWLRQVDQGIXWXUHUHVHDUFKSRVVLELOLWLHVDQGWR
FRPPXQLFDWHERWKWKHWKHRUHWLFDOVLJQLILFDQFHRIWKHUHVHDUFKDQGLWVSUDFWLFDO
LPSOLFDWLRQVIRUSROLF\DQGSUDFWLFH:KLOH,IXOO\DJUHHZLWKWKHVHSRLQWV,IHHOWKDWLW
PD\DOVREHXVHIXOWRUHFDSLWXODWHWKHPDLQUHVXOWVLQQRQWHFKQLFDOODQJXDJHVRPH
UHDGHUVDUHOLNHO\WRJRIURPWKH,QWURGXFWLRQVWUDLJKWWRWKH&RQFOXVLRQDQGZLOORQO\
UHDGWKHUHVWRIWKHVWXG\LIWKHILQDOFODLPVDURXVHWKHLULQWHUHVW
4XDOLWDWLYHUHSRUWVWKDWGRQ¶WZRUN
,WVVKRXOGEHFOHDUIURPWKHSUHYLRXVVHFWLRQVWKDWSUHSDULQJDQHIIHFWLYHTXDOLWDWLYH
UHSRUWLVQRVLPSOHEXVLQHVVDQGWKHUHDUHPDQ\DUHDVZKHUHWKHZULWLQJFDQJRZURQJ

7RKLJKOLJKWVRPHW\SLFDOSUREOHPVRXUFHVOHWXVORRNDWVRPHH[DPSOHVRIWKHKRZ
QRWWRGRLWNLQG
3VHXGRTXDOLWDWLYHVWXGLHV2QHRIWKHLQKHUHQWZHDNQHVVHVRIWKHTXDOLWDWLYHUHSRUW
LVWKDWLWLVUHODWLYHO\HDV\WRµDEXVH¶DQGFDOODVWXG\µTXDOLWDWLYH¶HYHQWKRXJKLWLVQRW
EDVHGRQDWKHRUHWLFDOO\VRXQGHPSLULFDOO\JURXQGHGV\VWHPDWLFDQDO\VLVEXWRQO\
IHDWXUHVDIHZDQHFGRWHVDQGYLJQHWWHV DQGSHUKDSVVRPHKDQGSLFNHGLQWHUYLHZ
H[WUDFWV $FFRUGLQJWR'XII LQSUHVV WKHUHDUHXQIRUWXQDWHO\WRRPDQ\RIVXFKQRQ
VFLHQWLILFUHSRUWVDQGWKHVHFDQGLVFUHGLWWKHVFKRODUO\LPDJHRIWKHTXDOLWDWLYHLQTXLU\
LQJHQHUDO 6HHDOVR6HFWLRQRQWKHGDQJHURIDQHFGRWDOLVPDQGWKHODFNRITXDOLW\
VDIHJXDUGV
/RQJDQGERULQJ:HKDYHVHHQWKDWTXDOLWDWLYHUHSRUWVWHQGWREHORQJQDUUDWLYHV
ZLWKRXWXVXDOO\KDYLQJWDEOHVRURWKHUNLQGVRIVXPPDULHVWKDWKHOSWKHUHDGHUGLJHVW
TXDQWLWDWLYHUHSRUWV$V5LFKDUGVRQDQG6W3LHUUH SRLQWRXWTXDOLWDWLYHZRUN
FDQQRWEHVFDQQHGEXWPXVWEHUHDGIXOO\EHFDXVHLWFDUULHVLWVPHDQLQJLQLWVHQWLUHWH[W
7KHUHIRUHZHKDYHDPDMRUSUREOHPLIWKHWH[WLVERULQJ5LFKDUGVRQ¶VSHUVRQDOVWRU\
LOOXVWUDWHVWKLVSRLQWZHOO
$GHFDGHDJRLQWKHILUVWHGLWLRQRIWKLV+DQGERRN>RI4XDOLWDWLYH5HVHDUFK@,
FRQIHVVHGWKDWIRU\HDUV,KDG\DZQHGP\ZD\WKURXJKQXPHURXVVXSSRVHGO\
H[HPSODU\TXDOLWDWLYHVWXGLHV&RXQWOHVVQXPEHUVRIWH[WVKDG,DEDQGRQHGKDOI
UHDGKDOIVFDQQHG,ZRXOGRUGHUDQHZERRNZLWKJUHDWDQWLFLSDWLRQ²WKHWRSLF
ZDVRQH,ZDVLQWHUHVWHGLQWKHDXWKRUZDVVRPHRQH,ZDQWHGWRUHDG²RQO\WR
ILQGWKHWH[WERULQJ,QµFRPLQJRXW¶WRFROOHDJXHVDQGVWXGHQWVDERXWP\VHFUHW
GLVSOHDVXUHZLWKPXFKRITXDOLWDWLYHZULWLQJ,IRXQGDFRPPXQLW\RIOLNH
PLQGHGGLVFRQWHQWV8QGHUJUDGXDWHVJUDGXDWHVDQGFROOHDJXHVDOLNHVDLGWKDW
WKH\IRXQGPXFKRITXDOLWDWLYHZULWLQJWREH²\HV²ERULQJ 5LFKDUGVRQDQG6W
3LHUUH
$FDGHPLFVDUHZHOOHGXFDWHGDQGVSHQGDJUHDWGHDORIWKHLUWLPHLQDFDGHPLDDQG
WKHUHIRUHWKH\PD\QRWUHDOL]HWKDWZULWLQJLQDQDFDGHPLFVW\OHKDVEHFRPHVHFRQG
QDWXUHWRWKHP7KHSUREOHPLVWKDWWHOOLQJDVWRU\LQDIODYRXUOHVVDFDGHPLFGLDOHFW
GRHVQRWZRUNDQGLWZRUNVHYHQOHVVLQTXDOLWDWLYHQDUUDWLYHVWKDQLWGRHVLQ
TXDQWLWDWLYHUHSRUWV
1RVLJQSRVWLQJ4XDOLWDWLYHDFFRXQWVXVXDOO\GHVFULEHFRPSOH[DQGRIWHQPXOWLSOH
PHDQLQJVIRFXVLQJFORVHO\RQVHYHUDOSDUWLFLSDQWV H[FHSWIRUVLQJOHFDVHVWXGLHV
DOWHUQDWLQJSDUWLFLSDQWYRLFHVZLWKWKHZULWHU¶VRZQ$OOWKLVUHVXOWVLQWH[WVRILQWULFDWH
WDSHVWU\:KLOHVXFKFRPSOH[WH[WXUHVFDQEHXVHGWRJRRGHIIHFWWKH\DOVRFDUU\WKH
GDQJHURIWKHUHDGHUJHWWLQJORVWDQGFRQIXVHGE\WKHP7KHUHIRUHSURSHUVLJQSRVWLQJLV
FULWLFDOIRUHIIHFWLYHTXDOLWDWLYHUHSRUWV7KHVLJQSRVWVPD\LQFOXGHDQLQWURGXFWRU\
RYHUYLHZRIWKHVWUXFWXUHYDULRXVKHDGLQJVDQGVXEKHDGLQJVHQGRIVHFWLRQVXPPDULHV
DVZHOODVWKHZULWHU¶VUHFXUULQJFRPPHQWDU\ DVLQDJXLGHGWRXU RQZKHUHZHDUHLQ

WKHVWRU\KRZWKHSDUWLFXODUSDUWUHODWHVWRZKDWKDVDOUHDG\EHHQVDLGDQGZKDWLV
FRPLQJQH[W
,QDFFHVVLEOHODQJXDJH'XII PHQWLRQVDIXUWKHUSUREOHPVRXUFHWKDWPLJKW

XQGHUPLQHWKHHIIHFWLYHQHVVRIDTXDOLWDWLYHUHSRUWXVLQJLQDFFHVVLEOHODQJXDJH
,IUHDGHUVDUHXQDEOHWRHDVLO\FRPSUHKHQGWKHUHVXOWVDQGXQGHUVWDQGWKHFKDLQV
RIUHDVRQLQJWKHQHZNQRZOHGJHRULQVLJKWVZLOOQRWµWUDYHO¶RUEHWUDQVIHUDEOHWR
QHZVHWWLQJV,QVWHDGWKHILQGLQJVPD\RQO\UHVRQDWHZLWKDVPDOOQXPEHURI
OLNHPLQGHGUHVHDUFKHUVDQGWKXVPD\KDYHOHVVLPSDFWRQWKHILHOGWKDQLQWHQGHG
RUGHVHUYHG S
2QWKHRWKHUKDQG'XII JRHVRQWRH[SODLQWKDWTXDOLWDWLYHUHVHDUFKDOVR
RIIHUVSRWHQWLDOEHQHILWVLQWKLVUHVSHFWEHFDXVHLIGRQHZHOODTXDOLWDWLYHUHSRUWFDQ
SURYLGHFRQFUHWHVLWXDWHGH[DPSOHVRIDEVWUDFWSKHQRPHQDDQGFDQWKXVFRPPXQLFDWH
FRPSOH[PHVVDJHVFOHDUO\
0RUHµDGYDQFHG¶IDLOXUHV%HVLGHVWKHSUREOHPVZLWKVW\OHVWUXFWXUHDQGODQJXDJH
GHVFULEHGDERYHWKHUHDUHVRPHPRUHµDGYDQFHG¶ZD\VE\ZKLFKDTXDOLWDWLYHUHSRUW
FDQJRZURQJ5LFKDUGV OLVWVWKUHHVXFKSDWWHUQV
x µ3DWFKZRUNTXLOW¶ZKHUHE\WKHUHSRUWLVPDGHXSRIDFROOHFWLRQRIORQJTXRWDWLRQV
VWLWFKHGWRJHWKHUE\IX]]\RUVXSHUILFLDOSDUWLDOVXPPDULHV7KLVLVLQGHHGD
FRPPRQPLVWDNHHVSHFLDOO\E\QRYLFHUHVHDUFKHUVEHFDXVHWKHµOHWWKHGDWDVSHDN
IRULWVHOI¶DWWLWXGHXVXDOO\FRYHUVXQFHUWDLQWLHVRUDODFNRIIRFXVRUDODFNRIDQ
RULJLQDOPHVVDJH
x µ,OOXPLQDWHGGHVFULSWLRQ¶ZKLFKRIIHUVDULFKDQGYLYLGDFFRXQWRIWKHVLWXDWLRQ
WKXVRIIHULQJDQHQWKUDOOLQJSUHVHQWDWLRQ:KDWLVPLVVLQJLVWKHLQWHJUDWLRQRIWKH
GDWDLQWRDFRKHUHQWDUJXPHQW
x µ/HDSRIIDLWK¶LQZKLFKWKHZULWHUXVHVDQDFFRXQWVXSSRUWHGZLWKFRQYLQFLQJ
HYLGHQFHDVDVSULQJERDUGWRVXGGHQO\ODXQFKLQWRPRUDORULGHRORJLFDODVVHUWLRQV
WKDWGRQRWIROORZGLUHFWO\IURPWKHGDWD%HFDXVHTXDOLWDWLYHLQTXLULHVDUHRIWHQ
PRWLYDWHGE\LGHRORJLFDOFRPPLWPHQWVRUVWULYLQJVIRUVRFLDOMXVWLFHVXFKDOHDS
FDQEHXQGHUVWDQGDEOHEXWLWFDQHDVLO\IRUJRWKHEDVLFSULQFLSOHRIUHVHDUFKUHSRUWV
QDPHO\WKDWWKHDUJXPHQWVVKRXOGEHILUPO\JURXQGHGLQWKHGDWD
:ULWLQJDPL[HGPHWKRGVUHSRUW
:HKDYHVHHQLQWKLVFKDSWHUWKDW48$/DQG48$1UHVHDUFKUHSRUWVGLIIHULQPDQ\
UHVSHFWVDQGHYHQDPRQJVWTXDOLWDWLYHUHSRUWVZHILQGDJUHDWGHDORIGLYHUJHQFH
:KHUHGRHVWKLVOHDYHXVZKHQZHDUHWRZULWHXSWKHUHVXOWVRIDPL[HGPHWKRGVVWXG\"
7KHDQVZHULVDELWOLNHWKHµJODVVLVKDOIIXOOHPSW\¶PHWDSKRUEHFDXVHZHFDQWDNH
ERWKDSRVLWLYHDQGDQHJDWLYHYLHZRQWKHFKDOOHQJHVRIPL[HGPHWKRGVZULWLQJ,QWKH
IROORZLQJ,ILUVWGLVFXVVERWKWKHFKDOOHQJHVDQGWKHDGYDQWDJHVRIVXFKDSURMHFWDQG

WKHQLQWURGXFHWKUHHNH\SULQFLSOHVRIZULWLQJPL[HGPHWKRGVUHSRUWV)LQDOO\,SUHVHQW
DQXPEHURIVSHFLILFUHFRPPHQGDWLRQVWRIDFLOLWDWHWKHZULWLQJSURFHVV
&KDOOHQJHVDQGDGYDQWDJHVRIPL[HGPHWKRGVUHSRUWV
7KHGLIIHUHQFHEHWZHHQ48$/DQG48$1UHVHDUFKUHSRUWVLVPRUHWKDQDWHFKQLFDO
RQHDQGFRYHUVERWKVWUXFWXUDODQGWHUPLQRORJLFDOLVVXHV7KHWZRW\SHVRIUHSRUWZDQW
WRDSSHDOWRWKHLUDXGLHQFHVLQGLIIHUHQWZD\VEHFDXVHWKHLUDXGLHQFHVWHQGWRILQG
GLIIHUHQWWKLQJVDSSHDOLQJ 6DQGHORZVNL $QGJLYHQWKDWIHZVFKRODUVLQDSSOLHG
OLQJXLVWLFV RULQWKHVRFLDOVFLHQFHVLQJHQHUDO DUHSDUDGLJPDWLFDOO\µELOLQJXDO¶RXU
JUHDWHVWFKDOOHQJHZKHQZULWLQJDPL[HGPHWKRGVUHSRUWLVWKDWVRPHRIWKHPDWHULDO
ZLOOE\GHILQLWLRQEHUHDGDQGHYDOXDWHGE\DSRWHQWLDOO\XQV\PSDWKHWLF RUHYHQ
KRVWLOH DXGLHQFH48$/DQG48$1UHVHDUFKHUVWHQGWRILQGGLIIHUHQWDUJXPHQWVDQG
HYLGHQFHFRQYLQFLQJDQGDSSO\GLIIHUHQWTXDOLW\FULWHULDLQMXGJLQJWKHYDOXHRID
VWXG\²VHH&KDSWHULQVKRUWDV6DQGHORZVNLFRQFOXGHVWKH\LQWHUDFWZLWKWKHWH[WV
GLIIHUHQWO\,QWKHOLJKWRIWKLVLVLWDQLOOXVLRQWRH[SHFWWKDWUHDGHUVZLOODSSUHFLDWHRXU
FODLPWKDWE\PL[LQJPHWKRGVZHFDQDFKLHYHDEHWWHUXQGHUVWDQGLQJRIRXUWRSLFWKDQ
E\XVLQJDSXUHPHWKRG"
7KHUHLVDOVRVRPHWHQVLRQDWWKHDFWXDOOHYHORIZULWLQJEHFDXVHLQDPL[HGPHWKRGV
UHSRUWZHQHHGWRXVHERWKZRUGVDQGQXPEHUVWRVXSSRUWRXULQWHUSUHWDWLRQVDQG
DOWKRXJKWKHVHGRQRWH[FOXGHHDFKRWKHUWRSUHVHQWWKHPLQDFRQYLQFLQJPDQQHUDQG
WRMXVWLI\WKHPHDQLQJLQIHUUHGIURPWKHPUHTXLUHVDGLIIHUHQWIUDPLQJDQGIRUPDWWLQJ
DSSURDFK,WPD\LQGHHGEHWKHFDVHWKDWWKHGLIIHUHQWSUHVHQWDWLRQDOPRGHVVLW
DZNZDUGO\WRJHWKHURQWKHSDJH %UDQQHQ ²WKHGLVVLPLODULW\EHWZHHQWKH
SHUVRQDOSUHVHQFHDQGYRLFHRIWKHZULWHULQ48$/RQWKHRQHKDQGDQGWKHLUDEVHQFH
LQ48$1WH[WVRQWKHRWKHULOOXVWUDWHVWKLVSRLQWZHOO,QJHQHUDO48$/DQG48$1
ZULWLQJKDYHDGLIIHUHQWµORRNDQGIHHO¶ 6DQGHORZVNL WKLVUDLVHVWKHTXHVWLRQDV
WRZKHWKHUDµK\EULG¶UHSRUWFDQSRVVLEO\KDYHWKHVDPHSRZHUDVLWVPRQRPHWKRG
FRXQWHUSDUWV
$WKLUGSUREOHPFRQFHUQVDYHU\SUDFWLFDO\HWIXQGDPHQWDOSRLQWWKHOHQJWKRI
PL[HGUHVHDUFKVWXGLHV$GLIILFXOWLVVXHZLWKTXDOLWDWLYHVWXGLHV DOUHDG\PHQWLRQHG
FRQFHUQVWKHIDFWWKDWEHFDXVHRIWKHQHHGIRUFRQWH[WXDOL]DWLRQDQGWKHXVHRITXRWHV
UDWKHUWKDQILJXUHVDVVXSSRUWLQJHYLGHQFHWKH\DUHLQKHUHQWO\ORQJHUWKDQTXDQWLWDWLYH
UHVHDUFKSDSHUV+RZHYHUPL[HGUHVHDUFKVWXGLHVLQYROYHERWKD48$/DQG48$1
FRPSRQHQW VRPHWLPHVPRUHWKDQRQHSKDVHRIHDFK DQGWKHUHIRUHUHTXLUHHYHQPRUH
VSDFHWKDQWKHLUPRQRPHWKRGFRXQWHUSDUWV:LWKPRVWMRXUQDOVKDYLQJVWULFWSDJH
OLPLWVWKHOHQJWKLVVXHFDQEHFRPHDVHULRXVSUREOHPZKHQUHVHDUFKHUVZRUNLQJLQD
PL[HGPHWKRGVSDUDGLJPDWWHPSWWRSXEOLVKWKHLUUHSRUWV -RKQVRQDQG&KULVWHQVHQ

2QWKHSRVLWLYHVLGHDOWKRXJKPL[LQJPHWKRGRORJLHVXQGHQLDEO\SRVHVFKDOOHQJHV
XVLQJFRPELQHGPHWKRGVFDQDOVREHWXUQHGLQWRDQDGYDQWDJH$IWHUDOOPRQRPHWKRG
VWXGLHVLQWKHSDVWKDYHW\SLFDOO\UHDFKHGWKHLUSDUDGLJPDWLFDOO\VHOHFWHGDXGLHQFHV

ZKHUHDVDVDUJXHGLQ6HFWLRQZLWKPL[HGPHWKRGVVWXGLHVZHKDYHIRUWKHILUVW
WLPHDQRSSRUWXQLW\WRUHDFKWKHZKROHUHVHDUFKVSHFWUXPEHFDXVHVXFKUHSRUWVKDYH
PRUHµVHOOLQJSRLQWV¶7RGGHWDO DUJXHWKDWZHFDQXVHWKHIDPLOLDUFRPSRQHQW
RIWKHUHVHDUFKWRZLQRYHUKRVWLOHDXGLHQFHVEHFDXVHIRUH[DPSOH48$1UHVXOWVZLOO
DSSHDUPRUHSUHVHQWDEOHWRD48$/DXGLHQFHLIWKH\DUHLOOXPLQDWHGE\VHQVLWLYH
48$/GDWD7KDWLVWKH48$/GDWDFDQµDXWKRUL]H¶WKH48$1UHVXOWVDQGYLFHYHUVD
,QWKHVDPHYHLQPL[HGPHWKRGVVWXGLHVFDQDOVRRSHQXSDZLGHUUDQJHRISXEOLFDWLRQ
YHQXHVIRUXVEHFDXVHERWKTXDOLWDWLYHO\DQGTXDQWLWDWLYHO\RULHQWHGMRXUQDOVZLOO
FRQVLGHUPL[HGPHWKRGVVXEPLVVLRQVDQGWKHSDUDGLJPDWLFELOLQJXDOLVPRIPL[HG
PHWKRGVUHVHDUFKHUVPD\DOVRHQDEOHWKHPWRJHWWKURXJKWRDZLGHUDXGLHQFHLQ
SUHVHQWDWLRQVDQGSXEOLFOHFWXUHV %UDQQHQ
.H\SULQFLSOHVRIZULWLQJPL[HGPHWKRGVUHSRUWV
%HIRUHDGGUHVVLQJVSHFLILFLVVXHVWRIDFLOLWDWHWKHZULWLQJRIPL[HGPHWKRGVVWXGLHVOHW
XVORRNDWWKUHHEDVLFSULQFLSOHVWKDWPLJKWJXLGHRXUDSSURDFK,EHOLHYHWKDWZULWLQJDQ
HIIHFWLYHPL[HGPHWKRGVUHSRUWUHTXLUHVJRRGEDODQFLQJVNLOOVDQGWKHUHIRUHWKH
IROORZLQJSULQFLSOHVDUHUHODWHGWRWKHZULWHU¶VVHQVLWLYLW\WRWKHSRVVLEOHLPSDFWRIWKUHH
NH\IDFWRUVRQWKHDSSUDLVDORIWKHVWXG\WKHDXGLHQFHWKHUHVHDUFKGHVLJQDQGWKH
UHVXOWV
x $XGLHQFHVHQVLWLYLW\7KHILUVWSULQFLSOHFDQEHVXPPDUL]HGLQRQHVHQWHQFH,I\RX
NQRZWKHSDUDGLJPDWLFRULHQWDWLRQRIWKHYDULRXVGHFLVLRQPDNHUVDQGVWDNHKROGHUV
IRUH[DPSOH3K'SDQHOH[DPLQHUVMRXUQDOERRNHGLWRUVRUFRQIHUHQFHDXGLHQFHV
JUDQWDSSOLFDWLRQUHYLHZHUV ZKRPLJKWEHDWWKHUHFHLYLQJHQGRI\RXUUHSRUW
HPSKDVL]HWKHDVSHFWRIWKHVWXG\WKH\DUHPRUHIDPLOLDUZLWKDQGIUDPHWKHRYHUDOO
LQWHUSUHWDWLRQLQWKHODQJXDJHDQGSUHVHQWDWLRQVW\OHRIWKHSDUWLFXODUDSSURDFK2U
DV-RKQVRQDQG&KULVWHQVHQ KDYHSXWLWµNQRZ\RXUDXGLHQFHDQGZULWH
LQDPDQQHUWKDWFOHDUO\FRPPXQLFDWHVWRWKDWDXGLHQFH¶7KLVGRHVQRWPHDQWKDW
ZHQHHGWRFRPSURPLVHRUFKDQJHRUGXPEGRZQRXUPHVVDJHUDWKHUZHFDQORRN
DWWKLVSURFHVVDVDWUDQVODWLRQH[HUFLVH
,IZHGHFLGHWKDWWKHPDLQDXGLHQFHLVOLNHO\WREHPL[HG6DQGHORZVNL
HPSKDVL]HVWKDWZHQHHGWRSHUPLWERWKTXDOLWDWLYHDQGTXDQWLWDWLYHUHDGHUVµDFFHVV¶
S WRRXUZRUN7KXVZULWLQJPL[HGPHWKRGVUHSRUWVUHTXLUHVDQ
XQGHUVWDQGLQJRIWKHSDUDGLJPDWLFGLIIHUHQFHVRIWKHPDLQFKDUDFWHULVWLFVRI
FRQYLQFLQJZULWHXSVLQERWKWKH48$/DQG48$1WUDGLWLRQVDQGZLWKPL[HG
DXGLHQFHVWKDWDUHE\GHILQLWLRQLQVXIILFLHQWO\YHUVHGLQERWKDSSURDFKHVZHVKRXOG
QRWWDNHKLJKO\VSHFLDOL]HGTXDQWLWDWLYHRUTXDOLWDWLYHWHUPVIRUJUDQWHG,QVWHDGZH
RXJKWWRSURYLGHDFFHVVLEOHGHILQLWLRQVDQGH[SODQDWLRQVWKHUHE\µHGXFDWLQJ¶RXU
DXGLHQFH ZKLFKLVLQFLGHQWDOO\ZKDWMXGJHVDQGODZ\HUVGRLQFRXUWWULDOVJLYHQ
WKDWPRVWMXU\PHPEHUVKDYHQROHJDOWUDLQLQJ
x 5HVHDUFKGHVLJQVHQVLWLYLW\7KHSDUDGLJPDWLFIUDPLQJRIRXUUHSRUWDOVRGHSHQGVRQ
RXUUHVHDUFKGHVLJQ7KHUHLVDQDWXUDOWHQGHQF\WRZULWHXSWKHTXDOLWDWLYHSDUWRI

WKHVWXG\LQDTXDOLWDWLYHPDQQHUDQGWKHTXDQWLWDWLYHSDUWLQDTXDQWLWDWLYHPDQQHU
DQGGRWKHPL[LQJDWWKHLQWHUSUHWDWLRQGLVFXVVLRQVWDJHVRQO\7KLVPD\EH
DSSURSULDWHIRUDµWULDQJXODWLRQ¶VWXG\ZKHUHWKHREMHFWLYHLVWRFRUURERUDWHILQGLQJV
DVDPHDQVRIYDOLGDWLRQ VHH6HFWLRQ EXWOHVVDSSURSULDWHIRUDVWXG\ZKHUH
PL[LQJDOVRRFFXUVDWWKHDQDO\VLVVWDJH,QWHJUDWHGDQDO\VHVQHHGWREHZULWWHQXS
LQDPRUHLQWHJUDWHGPDQQHU6HTXHQWLDOVWXGLHVLQZKLFKRQHUHVHDUFKSKDVHEXLOGV
RQDQRWKHUSUHVHQW\HWDQRWKHUVLWXDWLRQZLWKUHJDUGWRIUDPLQJWKHUHSRUWEHFDXVH
KHUHZHDOVRQHHGWRFRQVLGHUZKHWKHURQHRIWKHUHVHDUFKFRPSRQHQWVKDVJRWD
GRPLQDQWVWDWXV,QVXPWKHGHVLJQRIRXUVWXG\DOVRGLFWDWHVWRVRPHH[WHQWWKH
ZD\WKHUHSRUWFDQEHZULWWHQXSWREHVWHIIHFW
x 5HVXOWVHQVLWLYLW\)LQDOO\OHWPHPHQWLRQDYHU\SUDJPDWLFSULQFLSOH'HFLGLQJRQ
WKHEHVWSUHVHQWDWLRQDSSURDFKZLOODOVRGHSHQGRQZKDWVRUWRIUHVXOWVZHKDYH,I
RXUPRVWLPSUHVVLYHUHVXOWVDUHTXDQWLWDWLYHLWPDNHVVHQVHWRIUDPHWKHZKROH
UHSRUWLQDZD\VRDVWRKLJKOLJKWWKRVHZKHUHDVLIWKHTXDOLWDWLYHSKDVHKDV
JHQHUDWHGVRPHWUXO\UHPDUNDEOHLQVLJKWVDQGZHKDYHVRPHH[SUHVVLYHTXRWHVWR
EDFNWKHPXSWKHQWKLVZRXOGZDUUDQWDPRUHTXDOLWDWLYHSUHVHQWDWLRQIRUPDW
6SHFLILFUHFRPPHQGDWLRQVRQKRZWRZULWHPL[HG
PHWKRGVUHSRUWV
7RFRQFOXGHWKLVFKDSWHUOHWXVORRNDWVRPHVSHFLILFPL[HGPHWKRGVSUHVHQWDWLRQ
WHFKQLTXHVDQGLVVXHV$OWKRXJKLWLVDQRSWLRQIRUUHVHDUFKHUVWRZULWHVHSDUDWHUHSRUWV
IRUWKHTXDQWLWDWLYHDQGTXDOLWDWLYHSKDVHVRIWKHLUVWXGLHVWKDWDUHRQO\VXSHUILFLDOO\
OLQNHG,IHDUWKDWLQPDQ\FDVHVWKLV QRWXQFRPPRQ SUDFWLFHPD\DFWXDOO\µVKRUW
FKDQJH¶RXUZRUN7KHUHIRUHP\ILUVWUHFRPPHQGDWLRQLVWKDWLIDQLQYHVWLJDWLRQ
TXDOLILHVWREHODEHOOHGDµPL[HGPHWKRGVVWXG\¶ZHVKRXOGFRQVFLRXVO\IUDPHRXU
UHSRUWDVVXFK,QIDFW+DQVRQHWDO VSHFLILFDOO\UHFRPPHQGWKDWUHVHDUFKHUV
XVHWKHSKUDVHµPL[HGPHWKRGV¶LQWKHWLWOHRIWKHLUVWXGLHV
7KHIROORZLQJSRLQWVDUHLQWHQGHGWRKHOSWRDFKLHYHDSULQFLSOHGSUHVHQWDWLRQRI
PHWKRGRORJ\PL[LQJ+RZHYHUDVDSUHOLPLQDU\ZHFDQVD\WKDWDPL[HGPHWKRGV
UHVHDUFKUHSRUWFRQWDLQVWKHVDPHEDVLFIHDWXUHVDVGRPRVWPRQRPHWKRGUHSRUWVWKHUH
QHHGVWREHVRPHVRUWRIDQLQWURGXFWLRQDUHYLHZRIWKHUHOHYDQWOLWHUDWXUHDQGD
GLVFXVVLRQRIWKHPHWKRGRORJ\HPSOR\HGIROORZHGE\WKHSUHVHQWDWLRQDQGDQDO\VLVRI
WKHUHVXOWVDQGDFRQFOXVLRQ
x &RQVFLRXVDFFHQWXDWLRQRIPHWKRGRORJ\PL[LQJ%HFDXVHWKHPL[HGPHWKRGV
DSSURDFKLVVWLOODQHPHUJLQJILHOG+DQVRQHWDO UHFRPPHQGWKDW
UHVHDUFKHUVH[SODLQLQWKHLQWURGXFWLRQWKHORJLFWKHSURJUHVVLRQDQGWKHUDWLRQDOH
IRUWKHLUVWXG\²WKLVFDQLQIDFWVHUYHDVDµVXSHURUGLQDWHVWRU\OLQH¶²DQGLQFOXGH
H[SOLFLWSXUSRVHVWDWHPHQWVDQGUHVHDUFKTXHVWLRQVWKDWVSHFLI\WKHTXDQWLWDWLYHDQG
TXDOLWDWLYHDVSHFWVRIWKHLQYHVWLJDWLRQ%HFDXVHXOWLPDWHO\WKHYDOXHRIDPL[HG
PHWKRGVVWXG\GHSHQGVRQLWVFDSDFLW\WRSURGXFHDPRUHFRPSUHKHQVLYHDQVZHUWR
WKHUHVHDUFKTXHVWLRQWKDQDSXUHPHWKRGDORQHZRXOGLWLVZRUWKHODERUDWLQJRQ

WKLVLVVXHDWWKLVSRLQW6XFKH[SODQDWLRQVZLOOQRWRQO\OHQGFUHGHQFHWRWKHVWXG\
EXWZLOODOVRKRSHIXOO\KHOSWRHOLPLQDWHDQ\LQLWLDORULHQWDWLRQDOELDVLQWKHUHDGHU
x 8VLQJDPRGLILHG$3$VW\OHWHPSODWH6LPLODUO\WRP\IRUPDWUHFRPPHQGDWLRQIRU
TXDOLWDWLYHUHSRUWVLQ6HFWLRQ,ZRXOGVXJJHVWWKDWPL[HGPHWKRGV
UHVHDUFKHUVDSSO\WKH$3$VW\OHWHPSODWHIRUWKHRUJDQL]DWLRQRIWKHPDLQSDUWVRI
WKHLUUHSRUW8VLQJWKHIDPLOLDUVXEKHDGLQJVZLOOKHOSUHDGHUVWRRULHQWDWH
WKHPVHOYHVLQWKHRYHUDOOGHVLJQZKLFKLVERXQGWREHPRUHFRPSOH[WKDQWKDWRID
PRQRPHWKRGVWXG\EHFDXVHRIWKHPXOWLSOHUHVHDUFKSKDVHV:HPD\RUJDQL]HWKH
PDWHULDOE\UHVHDUFKTXHVWLRQRUE\UHVHDUFKSDUDGLJP,QWKHSV\FKRORJLFDO
OLWHUDWXUHLWLVQRWXQFRPPRQWRUHSRUWPXOWLSOHLQYHVWLJDWLRQLQDVLQJOHUHVHDUFK
DUWLFOHDQGWKHUHWKH\DUHXVXDOO\RUJDQL]HGE\QXPEHU IRUH[DPSOH6WXG\
6WXG\HWF
x /LWHUDWXUHUHYLHZ4XDOLWDWLYHDQGTXDQWLWDWLYHVWXGLHVFDQGLIIHUFRQVLGHUDEO\LQWKH
SODFHDQGUROHWKH\DVVLJQWRWKHOLWHUDWXUHUHYLHZ VHH6HFWLRQ 7KHUHIRUH
GHSHQGLQJRQWKHVHTXHQFLQJRIWKHSDUWLFXODUPL[HGPHWKRGVVWXG\ZHPD\GLYLGH
WKHOLWHUDWXUHUHYLHZLQWRVHYHUDOSDUWVZLWKDQLQLWLDORYHUYLHZLQWKH,QWURGXFWLRQ
DQGIXUWKHUPRUHVSHFLILFGLVFXVVLRQVODWHUZKHUHUHOHYDQW)RUWKHVDNHRI
WUDQVSDUHQF\ZHPD\VLJQSRVWWKLVGLYLVLRQE\OHWWLQJUHDGHUVNQRZZKDWWRH[SHFW
ODWHU
x 0HWKRGV%HFDXVHRILWVUHDGHUIULHQGOLQHVV,DPLQIDYRXURIDFRQFLVHDQGWRWKH
SRLQWPHWKRGRORJ\VHFWLRQLQDQ\UHVHDUFKUHSRUWDQGEUHDNLQJLWGRZQWRIDPLOLDU
VXEVHFWLRQV 3DUWLFLSDQWV,QVWUXPHQWV3URFHGXUHVHWF LQFUHDVHVLWVXVHIXOQHVVIRU
UHIHUHQFHSXUSRVHV LHZKHQVRPHRQHZDQWVWRORRNXSVRPHIDFWXDOLQIRUPDWLRQ
+RZHYHUZHQHHGWRGHYLDWHRQRQHSRLQWIURPWKHSXUHTXDQWLWDWLYHGHVLJQ
QDPHO\WKHLQFOXVLRQRIDµ0L[HGUHVHDUFKGHVLJQ¶VHFWLRQSRVVLEO\DWWKH
EHJLQQLQJRIWKH0HWKRGVSDUWLQZKLFKZHVSHFLI\WKHW\SHRIGHVLJQDSSOLHG
x &RQWH[WXDOL]DWLRQ%HFDXVHRIWKHVLWXDWHGQDWXUHRITXDOLWDWLYHUHVHDUFK48$/
VWXGLHVDUHKLJKO\FRQWH[WXDOL]HGZKHUHDV48$1VWXGLHVRIWHQUHTXLUHOLWWOH
FRQWH[WXDOL]DWLRQEH\RQGVXPPDUL]LQJWKHPDLQVDPSOHFKDUDFWHULVWLFV-RKQVRQ
DQG&KULVWHQVHQ UHFRPPHQGWKDWHYHQZKHQWKHTXDQWLWDWLYHSKDVHLV
GRPLQDQWZHVKRXOGDOZD\VFRQWH[WXDOL]HWKHUHSRUWV7KLVZLOOPDNHLWHDVLHUWR
UHODWHWKH48$/DQG48$1FRPSRQHQWVWRHDFKRWKHUDQGFRPPXQLFDWLQJ
FRQWH[WFKDUDFWHULVWLFVLVDOVRQHFHVVDU\IRUWKHUHDGHUWRDSSUDLVHWKHYDOLGLW\RI
PL[LQJDQGWKHJHQHUDOL]DELOLW\RIWKHUHVXOWV
x 7RQHDQGVW\OH7KHILQDODUHDWRFRQVLGHUZLWKUHJDUGWRSRVVLEOHWHQVLRQVEHWZHHQ
48$/DQG48$1SUHVHQWDWLRQUHTXLUHPHQWVFRQFHUQVWKHRYHUDOOWRQHDQGVW\OHRI
WKHZULWLQJ$VDUJXHGHDUOLHUDQHIIHFWLYHTXDOLWDWLYHUHSRUWRIWHQUHDGVOLNHD
YLYLGGHWDLOULFKGUDPDWL]HGVWRU\ZKHUHDVTXDQWLWDWLYHVWXGLHVWHQGWREHPRUH
GHWDFKHGIRUPDODQGPDWWHURIIDFWGHYRLGRIµOLWHUDU\IULOOV¶ 6DQGHORZVNL
0L[HGPHWKRGVUHVHDUFKHUVQHHGWRVWULNHDEDODQFHEHWZHHQWKHWZRW\SHVRI

ZULWLQJJXLGHGE\WKHLUµDXGLHQFHVHQVLWLYLW\¶GHVFULEHGLQWKHSUHYLRXVVHFWLRQ,Q
DGGLWLRQDVDOUHDG\PHQWLRQHGQRPDWWHUZKDWEDODQFHZHLQWHQGWRDFKLHYHLQD
PL[HGPHWKRGVVWXG\ZHFDQQRWUHO\RQWKHDXWKRUL]LQJIRUFHRIVSHFLDOL]HG
µSRZHUWHUPLQRORJ\¶EHFDXVHWKDWZRXOGLQHYLWDEO\DOLHQDWHSDUWRIRXUDXGLHQFH

3DUW6XPPLQJXS
+RZWRFKRRVHWKHDSSURSULDWH
UHVHDUFKPHWKRG
<RXUUHVHDUFKDSSURDFKFDQEHDVIOH[LEOHDV\RXZLVK<RXPD\LQIDFWZDQWWR
WDNHDPXOWLPHWKRGDSSURDFKLQ\RXUVHDUFKIRUDQVZHUV7KHUHLVQHYHUDµRQH
DQGRQO\RQH¶ZD\WRFDUU\RXWDSURMHFW
+DWFKDQG/D]DUDWRQ
+DYLQJFRYHUHGWKHPRVWLPSRUWDQWUHVHDUFKLVVXHVDQGWKHPDLQSULQFLSOHVRIGDWD
FROOHFWLRQDQGDQDO\VLVLWLVWLPHWRDGGUHVVRQHRIWKHPRVWIXQGDPHQWDOTXHVWLRQVLQ
UHVHDUFKPHWKRGRORJ\+RZGRZHGHFLGHZKLFKUHVHDUFKPHWKRGWRFKRVHIRURXU
SODQQHGLQYHVWLJDWLRQ"5HJUHWWDEO\WKHUHLVQRHDV\DQGXQLYHUVDOO\YDOLGDQVZHUWRWKLV
TXHVWLRQEHFDXVHGLIIHUHQWSHRSOHFKRRVHWKHLUUHVHDUFKPHWKRGVIRUGLIIHUHQWUHDVRQV
,QRUGHUWRSURYLGHVRPHSUDFWLFDOJXLGHOLQHVIRUPHWKRGVHOHFWLRQLQWKLVFRQFOXGLQJ
FKDSWHU,ZLOORIIHUWZRJHQHUDOUHFRPPHQGDWLRQVDQGVXPPDUL]HDQXPEHURI
FRQVLGHUDWLRQVWKDWPLJKWDIIHFWRXUFKRLFH
*HQHUDOUHFRPPHQGDWLRQ,$GRSWDSUDJPDWLF
DSSURDFK
,KDYHDUJXHGWKURXJKRXWWKLVERRNLQIDYRXURIDSUDJPDWLFDSSURDFKWRGHYHORSLQJRXU
UHVHDUFKGHVLJQWKLVLQYROYHVLQWKHEURDGHVWVHQVHPDLQWDLQLQJDQRSHQDQGIOH[LEOH
IUDPHRIPLQGDQGUHPDLQLQJDVIUHHDVSRVVLEOHRISDUDGLJPDWLFGRJPDV$WWKHHQGRI
WKHGD\UHVHDUFKLVQRWDSKLORVRSKLFDOH[HUFLVHEXWDQDWWHPSWWRILQGDQVZHUVWR
TXHVWLRQVDQGMXVWDVZHFDQJRDERXWWKLVLQRXUHYHU\GD\OLIHLQPDQ\GLIIHUHQWZD\V
WKHYDULHGSDOHWWHRIUHVHDUFKPHWKRGRORJ\LVFOHDUHYLGHQFHIRUWKHSRVVLELOLW\RI
GLYHUVHVROXWLRQVLQWKHVFLHQWLILFLQTXLU\$FFRUGLQJO\P\ILUVWJHQHUDO
UHFRPPHQGDWLRQLVLQGHHGYHU\JHQHUDO$GRSWDSUDJPDWLFDSSURDFKDQGIHHOIUHHWR
FKRRVHWKHUHVHDUFKPHWKRGWKDW\RXWKLQNZLOOZRUNEHVWLQ\RXULQTXLU\$VLPLODU
RSLQLRQZDVHFKRHGE\6LOYHUPDQ ZKHQKHFRQFOXGHGWKDWµLWLVVHQVLEOHWR
PDNHSUDJPDWLFFKRLFHVEHWZHHQUHVHDUFKPHWKRGRORJLHVDFFRUGLQJWR\RXUUHVHDUFK
SUREOHPDQGPRGHO¶7KHUHVWRIWKLVFKDSWHUZLOORIIHUDUDQJHRIGLIIHUHQWSHUVSHFWLYHV
RQKRZWRPDNHVXFKSUDJPDWLFFKRLFHV

5HVHDUFKFRQWHQWFRQVLGHUDWLRQV
*UHHQHDQG&DUDFHOOL SRLQWRXWWKDWFRQWUDU\WRZKDWZHPLJKWH[SHFWDQGZKDW
LVRIWHQVWDWHGLQWKHOLWHUDWXUHVRFLDOUHVHDUFKHUVDSSHDUWRJURXQGLQTXLU\GHFLVLRQV
SULPDULO\LQWKHQDWXUHRIWKHSKHQRPHQDEHLQJLQYHVWLJDWHGDQGWKHFRQWH[WVLQZKLFK
WKHVWXGLHVDUHFRQGXFWHGUDWKHUWKDQLQPRUHDEVWUDFWSDUDGLJPFRQFHUQV/HWXVVWDUW
H[DPLQLQJWKHVHVSHFLILFµJURXQGHG¶FRQVLGHUDWLRQVE\ORRNLQJDWWKHHIIHFWVRIWKH
UHVHDUFKFRQWHQW,QWKLVUHVSHFWWKUHHLVVXHVLQSDUWLFXODUDUHUHOHYDQW D WKHUHVHDUFK
TXHVWLRQWRSLF E ZKHWKHUWKHLQYHVWLJDWLRQLVH[SORUDWRU\RUH[SODQDWRU\DQG F WKH
H[LVWLQJUHVHDUFKWUDGLWLRQVLQWKHVSHFLILFDUHD
x 5HVHDUFKTXHVWLRQDQGWRSLF,QWKHLULQIOXHQWLDOERRNRQPL[HGPHWKRGVUHVHDUFK
7DVKDNNRULDQG7HGGOLH LQFOXGHDZKROHVHFWLRQHQWLWOHGµ7KHGLFWDWRUVKLSRI
WKHUHVHDUFKTXHVWLRQ¶,QWKLVWKH\FODLPWKDWLWLVWKHUHVHDUFKTXHVWLRQWKDWRXJKW
WRGHWHUPLQHHYHU\DVSHFWRIWKHGHVLJQRIWKHVWXG\DQGIRUPRVWUHVHDUFKHUV
PHWKRGLVVHFRQGDU\WRWKHUHVHDUFKTXHVWLRQLWVHOILQWKHVHQVHWKDWWKH\DUHUHDG\
WRDGGUHVVWKHLUUHVHDUFKTXHVWLRQVZLWKDQ\PHWKRGRORJLFDOWRRODYDLODEOHXVLQJ
WKHSUDJPDWLVWFUHGRRIµZKDWZRUNV¶,QRWKHUZRUGVµ7KHEHVWPHWKRGLVWKHRQH
WKDWDQVZHUVWKHUHVHDUFKTXHVWLRQ V PRVWHIILFLHQWO\¶ S $OWKRXJK,WHQGWR
DJUHHZLWKWKHWKUXVWRIWKLVDUJXPHQWLWDOVRUDLVHVDKDUGTXHVWLRQ+RZGRZH
NQRZLQDGYDQFHZKDWZLOOZRUN",QVRPHFDVHVWKHUHPD\EHDQREYLRXVPDWFK
EHWZHHQRXUUHVHDUFKTXHVWLRQDQGDPHWKRGRILQTXLU\EXWLQRWKHU DQG,EHOLHYH
PRVW FDVHVVHYHUDODSSURDFKHVPLJKWEHVXLWDEOHIRUH[SORULQJDSUREOHP$WWLPHV
OLNHWKLVWKHµFKRRVHWKHPHWKRGWKDWEHVWVXLWVWKHUHVHDUFKTXHVWLRQ¶SULQFLSOH
GRHVQRWRIIHUPXFKKHOSDQGRXUFKRLFHQHHGVWREHLQIRUPHGE\RWKHU
FRQVLGHUDWLRQV
x ([SORUDWRU\SXUSRVH$QREYLRXVVLWXDWLRQZKHQEDVLQJWKHPHWKRGVHOHFWLRQRQRXU
VSHFLILFUHVHDUFKTXHVWLRQFDQQRWZRUNLVZKHQZHGRQRWKDYHDVSHFLILFUHVHDUFK
TXHVWLRQ7KLVKDSSHQVIRUH[DPSOHZKHQOLWWOHLVNQRZQDERXWWKHWDUJHWSUREOHP
RUSKHQRPHQRQEHFDXVHLWKDVQRWEHHQVWXGLHGLQGHWDLOEHIRUHRUEHFDXVHSUHYLRXV
VWXGLHVKDYHJHQHUDWHGLQDGHTXDWHH[SODQDWLRQV6XFKVLWXDWLRQVQHFHVVLWDWH
H[SORUDWRU\LQYHVWLJDWLRQVWKDWKHOSXVWRPDSWKHWHUUDLQILUVWDQGILQHWXQHRXU
VSHFLILFUHVHDUFKDQJOHODWHULQWKHSURMHFW,Q0RUVHDQG5LFKDUGV¶
ZRUGVµ3XWEOXQWO\LI\RXGRQ¶WNQRZZKDW\RXDUHOLNHO\WRILQG\RXUSURMHFW
UHTXLUHVPHWKRGVWKDWZLOODOORZ\RXWROHDUQZKDWWKHTXHVWLRQLVIURPWKHGDWD¶
$VGLVFXVVHGLQ&KDSWHU 6HFWLRQ VXFKH[SORUDWRU\SXUSRVHVDUHXVXDOO\
DVVRFLDWHGZLWKDTXDOLWDWLYHPHWKRGRORJ\
x ([LVWLQJUHVHDUFKWUDGLWLRQLQWKHDUHD,QVRPHUHVHDUFKDUHDVWKHUHDUHFHUWDLQ
GRPLQDQWPHWKRGVRILQTXLU\)RUH[DPSOHDV,H[SODLQHGLQWKH3UHIDFHZKHQ,
VWDUWHGWRLQYHVWLJDWHODQJXDJHOHDUQLQJPRWLYDWLRQDVSDUWRIP\3K'VWXGLHVWKH
DYDLODEOHUHVHDUFKZDVDOPRVWHQWLUHO\TXDQWLWDWLYHXWLOL]LQJVXUYH\PHWKRGRORJ\
,IZHFDQLGHQWLI\VXFKDPHWKRGRORJLFDOVDOLHQFHDVVRFLDWHGZLWKRXUUHVHDUFK
SUREOHPLWPDNHVVHQVHWRVWDUWRXUSURMHFWE\IROORZLQJWKHSDUWLFXODUWUDGLWLRQ,Q

WKLVZD\ZHFDQUHFHLYHJXLGDQFHLQWKHOLWHUDWXUHDQGRXUUHVXOWVZLOOEHGLUHFWO\
FRPSDUDEOHZLWKRWKHUVFKRODUV¶ILQGLQJV/DWHURIFRXUVHZHPLJKWGHFLGHWR
EUHDNDZD\DQGWU\DQGVKHGQHZOLJKWRQWKHDUHDE\DGRSWLQJQHZUHVHDUFK
PHWKRGV
$XGLHQFHFRQVLGHUDWLRQV
,KDYHDUJXHGPRUHWKDQRQFHLQWKLVERRNWKDWUHVHDUFKLVDVRFLDODFWLYLW\ZKHUHE\
UHVHDUFKHUVQHHGWRVKDUHWKHLUILQGLQJVZLWKWKHZLGHUVFKRODUO\FRPPXQLW\
$FFRUGLQJO\ZHQHHGWREHVHQVLWLYHWRWKHDXGLHQFHVWRZKRPZHUHSRUWRXUUHVXOWV
ZKHWKHUWKH\EHXQLYHUVLW\FRPPLWWHHVGLVVHUWDWLRQWKHVLVH[DPLQHUVMRXUQDOHGLWRUV
FRQIHUHQFHDWWHQGHHVRUFROOHDJXHVLQWKHILHOG &UHVZHOO 6HOHFWLQJDUHVHDUFK
PHWKRGLQDZD\WKDWWDNHVLQWRFRQVLGHUDWLRQWKHSUHIHUHQFHVDQGH[SHFWDWLRQVRIRXU
WDUJHWHGDXGLHQFHLVLQOLQHZLWKVXFKDXGLHQFHFRQVLGHUDWLRQV+RZHYHUWKHUHLV
DGPLWWHGO\DILQHOLQHEHWZHHQVXFKSUDJPDWLFDXGLHQFHVHQVLWLYLW\DQGZKDW7RGGHWDO
FDOOµLOOHJLWLPDWHPRWLYHV¶IRUFKRRVLQJDPHWKRGEDVHGRQSROLWLFDOUHDVRQV
7KHVHPD\LQYROYHIRUH[DPSOHWU\LQJWRILWWKHPHWKRGZLWKWKHPRVWSURILWDEOH
UHVHDUFKSDUDGLJPZLWKLQWKHUHVHDUFKHU¶VSURIHVVLRQDOHQYLURQPHQWLQRUGHUWRDFKLHYH
DGYDQFHPHQW
3UDFWLFDOFRQVLGHUDWLRQV
,QP\H[SHULHQFHHYHQLIZHWU\WREHSULQFLSOHGLQRXUVHOHFWLRQRIDUHVHDUFKPHWKRG
WKHILQDOFKRLFHLVOLNHO\WREHDWUDGHRIIEHWZHHQWKHRUHWLFDODQGSUDFWLFDO
FRQVLGHUDWLRQV,WLVP\LPSUHVVLRQWKDWUHVHDUFKHUVDUHRIWHQDVKDPHGRIWKH
FRPSURPLVHVWKDWWKH\QHHGWRPDNHQRWUHDOL]LQJWKDWPDNLQJFRPSURPLVHVLVSDUWDQG
SDUFHORIEHLQJDUHVHDUFKHU7KUHHSUDFWLFDOFRQVLGHUDWLRQVDUHSDUWLFXODUO\UHOHYDQWWR
QRYLFHUHVHDUFKHUVZKHQWKH\DUHWRFKRVHDPHWKRGIRUWKHLULQYHVWLJDWLRQ
x 6XSHUYLVRUDQGRWKHUDYDLODEOHVXSSRUW(YHU\UHVHDUFKHUQHHGVDGYLFHDQGKHOS
IURPWLPHWRWLPHDQGIRUOHVVH[SHULHQFHGLQYHVWLJDWRUVHVWDEOLVKLQJVRPHVXSSRUW
OLQNLVYLWDO7KHUHIRUHRQHLPSRUWDQWFRQVLGHUDWLRQLQVHOHFWLQJWKHPHWKRGWREH
DSSOLHGLVWRPDNHLWFRPSDWLEOHZLWKWKHH[SHUWLVHRUUHVHDUFKSUHIHUHQFHRISHRSOH
DURXQGXVZKRPD\EHDEOHWRRIIHUDVVLVWDQFH
x $YDLODEOHUHVRXUFHV5HVHDUFKPHWKRGVYDU\LQWKHDPRXQWRIWLPHDQGPRQH\WKH\
UHTXLUHIRUVXFFHVVIXOFRPSOHWLRQ7KHUHIRUHFRVWHIIHFWLYHQHVVDQGFRVWEHQHILW
FDOFXODWLRQVDOZD\VQHHGWREHVHULRXVO\FRQVLGHUHG4XDQWLWDWLYHPHWKRGVDUHRIWHQ
VHOHFWHGEHFDXVHWKH\WHQGWREHOHVVWLPHFRQVXPLQJWRFRQGXFWWKDQTXDOLWDWLYH
VWXGLHV IRUH[DPSOHFRPSDUHDTXHVWLRQQDLUHVXUYH\WRDQLQWHUYLHZVWXG\ EXW
QRYLFHUHVHDUFKHUVRIWHQIRUJHWWKDWWKHVHPHWKRGVUHTXLUHSV\FKRPHWULFDOO\VRXQG
LQVWUXPHQWVDQGVXIILFLHQWVWDWLVWLFDOH[SHUWLVHWRSURFHVVWKHUHVXOWV
x $YDLODEOHVDPSOH7KHIHDVLELOLW\RIXVLQJDSDUWLFXODUPHWKRGJUHDWO\GHSHQGVRQ
WKHNLQGRISDUWLFLSDQWVDPSOHWKDWWKHUHVHDUFKHUFDQKDYHDFFHVVWR5HDFKLQJWKH

PRVWDSSURSULDWHUHVSRQGHQWVFDQEHSUHYHQWHGE\DYDULHW\RIWHFKQLFDOILQDQFLDO
ORJLVWLFRUWLPLQJIDFWRUV7KHUHIRUHUHVHDUFKHUVRIWHQIROORZDUDWKHURSSRUWXQLVWLF
EXWOHJLWLPDWH DSSURDFK,IWKH\UHDOL]HWKDWWKH\DUHLQDSRVLWLRQWRH[DPLQHD
VSHFLILFJURXSRISHRSOHZKRFRXOGSURYLGHVRPHXQLTXHRUSDUWLFXODUO\KLJK
TXDOLW\GDWDWKH\PD\WDLORUWKHLUPHWKRG DQGVRPHWLPHVHYHQWKHLUVSHFLILF
UHVHDUFKWRSLF WRWKHFKDUDFWHULVWLFVRIWKLVDYDLODEOHVDPSOLQJRSSRUWXQLW\
3HUVRQDOFRQVLGHUDWLRQV
$WDQ(GLWRULDO$GYLVRU\%RDUGPHHWLQJRI7(62/4XDUWHUO\LQWKHODWHV,KDGDQ
LQWHUHVWLQJFRQYHUVDWLRQZLWK.DWK\'DYLHV 8QLYHUVLW\RI+DZDLL DERXWWKH
GLIIHUHQFHVEHWZHHQTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFK.DWK\VDLGVRPHWKLQJWRWKH
HIIHFWWKDWTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKHUVWHQGWREHµGLIIHUHQWSHRSOH¶WKDWLV
WKH\GLIIHULQVRPHYLWDO UHVHDUFKUHODWHG SHUVRQDOFKDUDFWHULVWLFV7KHPRUH,KDYH
WKRXJKWDERXWWKLVRYHUWKH\HDUVWKHPRUHLPSRUWDQW,KDYHFRPHWRFRQVLGHUWKLV
SRLQW,WVHHPVWREHWKHFDVHWKDWVRPHWLPHVRXUUHVHDUFKRULHQWDWLRQLVQRWVRPXFK
UHODWHGWRSKLORVRSKLFDORUUHVHDUFKVSHFLILFFRQVLGHUDWLRQVDVWRZKRZHDUHDQGZKDW
ZHDUHOLNH/HWXVORRNDWWKLVTXHVWLRQLQPRUHGHWDLO
3HUVRQDOVW\OHDQGUHVHDUFKRULHQWDWLRQ
(U]EHUJHUDQG.HOOH VXPPDUL]HWKHVWDQGDUGUHVHDUFKPHWKRGRORJLFDO
SRVLWLRQUHJDUGLQJWKHUHODWLRQVKLSEHWZHHQSHUVRQDOSUHIHUHQFHVDQGPHWKRGFKRLFH
YHU\FOHDUO\
7KHVHOHFWLRQRIDGHTXDWHPHWKRGVVKRXOGQRWEHPDGHPDLQO\RQWKHEDVLVRI
V\PSDWKLHVWRZDUGDFHUWDLQPHWKRGRORJLFDOFDPSRUVFKRRO0HWKRGVDUHWRROV
IRUWKHDQVZHULQJRIWHVWTXHVWLRQVDQGQRWYLFHYHUVD&RQVHTXHQWO\GHFLVLRQV
DERXWWKHDPHWKRGVVKRXOGQRWEHPDGHEHIRUHWKHUHVHDUFKTXHVWLRQVDUH
IRUPXODWHG
:KLOH,DFFHSWWKDWWKLVLVDUDWLRQDOSRVLWLRQWRWDNH,GRKDYHGRXEWVDERXWZKHWKHU
LWLVUHDOLVWLFWRH[SHFWVFKRODUVWRJRDJDLQVWWKHLUSHUVRQDOWHQGHQFLHVDQGVW\OH,VWDWHG
LQ6HFWLRQWKDWLQP\RZQGD\WRGD\UHVHDUFKDFWLYLWLHV,DPPRUHLQFOLQHGWRZDUGV
DTXDQWLWDWLYHDSSURDFKWKDQDTXDOLWDWLYHRQHVLPSO\EHFDXVH,IHHO,DPEHWWHUDWLWDQG
,HQMR\LWPRUH7KLVLVQRWXQOLNHELOLQJXDOLVP$OWKRXJKZHFDQEHYHU\IOXHQWLQWZR
ODQJXDJHVWKHUHDUHYHU\IHZ LIDQ\ FRPSOHWHO\EDODQFHGELOLQJXDOV²GHSHQGLQJRQ
WKHVSHFLILFODQJXDJHIXQFWLRQVLWXDWLRQFRQFHUQHGPRVWELOLQJXDOVVKRZVRPHODQJXDJH
SUHIHUHQFH,GREHOLHYHWKDWUHVHDUFKHUVVKRXOGVWULYHWREHFRPHSDUDGLJPDWLFDOO\
ELOLQJXDO RUWULOLQJXDO \HW,ZRXOGH[SHFWWKDWPRVWRIXVDUHDZDUHRIZKLFKRXUILUVW
DQGZKLFKWKHDGGLWLRQDOUHVHDUFKODQJXDJHLV3XQFK GHVFULEHVWKLV
SKHQRPHQRQDVIROORZV
6RPHSHRSOHSUHIHUIRUDYDULHW\RIUHDVRQVRQHDSSURDFKRYHUWKHRWKHU7KLV
PD\LQYROYHSDUDGLJPDQGSKLORVRSKLFDOLVVXHVEXWWKHUHLVRIWHQDOVRD
VXEFRQVFLRXVWHPSHUDPHQWIDFWRUDWZRUN7KLVYLVXDOO\FRPHVGRZQWRGLIIHUHQW

LPDJHVDERXWZKDWJRRGSLHFHRIUHVHDUFKORRNVOLNH6WXGHQWVZLOORIWHQH[SUHVV
WKDWE\VD\LQJWKDWWKH\µOLNHWKLVZD\EHWWHU¶
$QDWXUDOO\TXDQWLWDWLYHUHVHDUFKHUILQGVLWH[FLWLQJDQGLQWULQVLFDOO\UHZDUGLQJWR
GHYHORSQHDWDQGRUGHUO\V\VWHPVRIYDULDEOHVZLWKILUPERXQGDULHVWRH[SORUHVWDWLVWLFV
WRWKHH[WHQWWKDWH[DPLQLQJDVWDWLVWLFDOWDEOHLVDSOHDVXUDEOHH[SHULHQFH DQGLQ
JHQHUDOWRXQFRYHUUHJXODULWLHVWKDWVHHPWRJRYHUQFHUWDLQDVSHFWVRIRXUOLYHV,Q
FRQWUDVWVFKRODUVZLWKDTXDOLWDWLYHRULHQWDWLRQUHYHOLQEHLQJLPPHUVHGLQWKHµPHVV\¶
DQGDPELJXRXV²RIWHQHYHQFRQIXVLQJ²UHDOLW\RIVRFLDOOLIHDQGKDYHWKHJLIWRI
QRWLFLQJSDWWHUQVDQGWHDVLQJRXWXQGHUO\LQJPHDQLQJVLQWKLVµQRLV\¶UHVHDUFK
HQYLURQPHQWWKH\DUHOHVVLQWHUHVWHGLQEURDGWHQGHQFLHVWKDQLQWKHVXEWOHSHUFHSWLRQV
LQIOXHQFHVDQGFRQGLWLRQVWKDWVKDSHRXULGLRV\QFUDWLFSHUVRQDOL]HGZRUOGV,DPQRW
DZDUHRIDQ\FRJQLWLYHVW\OHVUHVHDUFKLQWRSDUDGLJPDWLFSUHIHUHQFHVEXWLWVHHPVOLNHO\
WKDWRXUVW\OHFKDUDFWHULVWLFV IRUH[DPSOHJOREDOYVSDUWLFXODULQGXFWLYHYVGHGXFWLYH
RUV\QWKHWLFYVDQDO\WLF KDYHDVWURQJEHDULQJRQWKLVPDWWHU
3HUVRQDOWUDLQLQJDQGH[SHULHQFH
(YHQLIZHDFFHSWWKDWPDQ\UHVHDUFKHUVKDYHEDVLFRULHQWDWLRQDOSUHIHUHQFHVWKH
TXHVWLRQLVZKHWKHUWKHVHDUHLQQDWHRUOHDUQHG,VXVSHFWWKDWOLNHLQVRPDQ\FDVHV
ZKHQWKHµQDWXUHRUQXUWXUH¶TXHVWLRQDULVHVLQWKHVRFLDOVFLHQFHVWKHDQVZHULVPL[HG
:KLOH,EHOLHYHWKDWFHUWDLQVW\OLVWLFRUWHPSHUDPHQWDOLQGLYLGXDOGLIIHUHQFHIDFWRUVPD\
LQIOXHQFHRXUPHWKRGRORJLFDOFKRLFHV,DOVRFRQVLGHUWKHUHVHDUFKHU¶VSHUVRQDOWUDLQLQJ
DQGUHVHDUFKH[SHULHQFHFHQWUDOLQWKLVUHVSHFW&UHVZHOO ULJKWO\SRLQWVRXWWKDW
VRPHRQHWUDLQHGLQWHFKQLFDOVFLHQWLILFZULWLQJDQGVWDWLVWLFV DQGZKRLVDOVRIDPLOLDU
ZLWKTXDQWLWDWLYHMRXUQDOV ZRXOGPRVWOLNHO\FKRRVHTXDQWLWDWLYHPHWKRGRORJ\2QWKH
RWKHUKDQGWKHTXDOLWDWLYHDSSURDFKLQFRUSRUDWHVPRUHRIDOLWHUDU\IRUPRIZULWLQJ²
DQGZHFDQDGGFUHDWLYHZULWLQJDQGVWRU\WHOOLQJ²DVZHOODVH[SHULHQFHLQFRQGXFWLQJ
RSHQHQGHGLQWHUYLHZVDQGREVHUYDWLRQV
2XUPHWKRGRORJLFDOXSEULQJLQJFDQIUDPHRXUWKLQNLQJLQWZRZD\V)LUVWRQFHZH
KDYHLGHQWLILHGDUHVHDUFKTXHVWLRQZHDUHPRUHOLNHO\WRHVWDEOLVKDZRUNLQJSODQ
WRZDUGVDQVZHULQJLWE\XVLQJPHWKRGVZKRVHFDSDELOLWLHVZHDUHZHOODFTXDLQWHGZLWK
7KHVHFRQGURXWHLVPRUHVXEWOHRXUWUDLQLQJDQGSDVWH[SHULHQFHLQXVLQJFHUWDLQ
PHWKRGVQRWRQO\DFWDVWRROVEXWWKH\PD\DOVRVKDSHWKHZD\ZHORRNDWUHVHDUFK
SUREOHPVLQJHQHUDO7KXVLWPD\EHWKHFDVH DQG,EHOLHYHLWRIWHQLV WKDWWKHZD\ZH
VSHFLI\WKHUHVHDUFKTXHVWLRQVLVDOUHDG\GHWHUPLQHGE\RXULQLWLDOPHWKRGRORJLFDO
RULHQWDWLRQ1RUWRQ3HLUFH VWUHVVHVWKDWWKHRU\LPSOLFLWO\RUH[SOLFLWO\LQIRUPV
WKHTXHVWLRQVUHVHDUFKHUVDVNZKLFKLQWXUQLQIOXHQFHWKHUHVHDUFKPHWKRGVVHOHFWHG
:KLOHWKLVLVXQGRXEWHGO\WUXHWKHVHFRQVLGHUDWLRQVVXJJHVWWKDWRXUH[SHULHQWLDO
NQRZOHGJHDQGVNLOOVFDQµWLQWRXUOHQVHV¶RYHUDQGDERYHRXUWKHRUHWLFDORULHQWDWLRQ
DQGDFWXDOO\EHWKHVWDUWLQJSRLQWLQWKHPHWKRGVHOHFWLRQFKDLQ/D]DUDWRQ
HYHQVXJJHVWVWKDWWKHDSSHDORUµVH[LQHVV¶RIFHUWDLQKLJKSRZHUHGVWDWLVWLFDO

_SURFHGXUHVPD\OHDGRQHWRDVNUHVHDUFKTXHVWLRQVZKRVHDQVZHUVUHTXLUHWKHXVHRI
WKHSDUWLFXODUSURFHGXUHZKLFKVKHVD\VLVDQH[DPSOHRIWKHµZDJWKHGRJ¶V\QGURPH
'LVLOOXVLRQPHQWZLWKDQDSSURDFK
)LQDOO\ZHPLJKWEHPRYHGWRZDUGVDQDSSURDFKIRUQHJDWLYHUHDVRQVWKDWLVEHFDXVH
ZHGLVOLNHRUDUHGLVLOOXVLRQHGZLWKWKHDOWHUQDWLYHPHWKRGRORJ\)RUH[DPSOHP\RZQ
ILUVWHQFRXQWHUZLWKTXDOLWDWLYHPHWKRGVZDVPRWLYDWHGE\WKHUHDOL]DWLRQRIWKH
OLPLWDWLRQVRITXDQWLWDWLYHWHFKQLTXHVLQH[SORULQJQHZGLPHQVLRQVRI/PRWLYDWLRQ
P\PDLQUHVHDUFKDUHD ,QGHHGDFFRUGLQJWR0RUVHDQG5LFKDUGV TXDQWLWDWLYH
UHVHDUFKHUVRIWHQVWDUWZRUNLQJTXDOLWDWLYHO\EHFDXVHWKH\UHFRJQL]HWKDWWKHVWDWLVWLFDO
DQDO\VHVRIWKHLUVXUYH\GDWDGLGQRWVHHPWRILWZKDWUHVSRQGHQWVLQVLPLODUVLWXDWLRQV
VDLGRUZKDWWKH\ZURWHLQWKHLUDQVZHUVWRRSHQHQGHGTXHVWLRQV
7U\LQJRXWDQHZDSSURDFKEHFDXVHRIDQHDUOLHUGLVLOOXVLRQPHQWZLWKDQDOWHUQDWLYH
RQHLVDOHJLWLPDWHUHDVRQIRUPHWKRGFKRLFHEXWWKLVLVQRWWKHFDVHLIDV6LOYHUPDQ
GHVFULEHVZHWXUQWRTXDOLWDWLYHUHVHDUFKRQO\EHFDXVHZHFDQQRWGRVWDWLVWLFV
DQGFDQQRWEHERWKHUHGWROHDUQ0RUVHDQG5LFKDUGV DOVRXQGHUVFRUHWKHULVNVLQ
SURMHFWVZKHUHVRPHRQHGHFLGHVWRDSSO\TXDOLWDWLYHPHWKRGVIRUWKHZURQJUHDVRQDQG
EHVLGHVWKHFRPPRQLVVXHRIWU\LQJWRDYRLGVWDWLVWLFVWKH\PHQWLRQDQRWKHULQWHUHVWLQJ
PLVFRQFHSWLRQKHOGE\VRPHVFKRODUVQDPHO\WKDWTXDQWLWDWLYHUHVHDUFKLV
PRUDOO\HWKLFDOO\LQIHULRUWRWKHPRUHµKXPDQLVWLF¶TXDOLWDWLYHPHWKRGV$VWKH\SRLQW
RXWTXDOLWDWLYHWHFKQLTXHVFDQEHµPRVWLQYDVLYHLQWUXVLYHDQGPRUDOO\FKDOOHQJLQJ
WKHRQO\JRRGUHDVRQDUHVHDUFKHUVKRXOGFRQVLGHUXVLQJWKHPLVWKDWWKHUHVHDUFK
SUREOHPUHTXLUHVWKHP¶ S
*HQHUDOUHFRPPHQGDWLRQ,,&RQVLGHUPL[LQJ
PHWKRGRORJLHV
7KHVHFRQGJHQHUDOUHFRPPHQGDWLRQ,ZRXOGOLNHWRPDNHLVWKDWLWLVZRUWK
FRQVLGHULQJDSSO\LQJDPL[HGPHWKRGVUHVHDUFKGHVLJQLQHYHU\VLWXDWLRQ(YHQLIWKHUH
LVDJRRGUHDVRQIRURSWLQJIRUDVSHFLILFPHWKRGRORJ\DGGLQJDVHFRQGDU\FRPSRQHQW
IURPDQRWKHUDSSURDFKFDQEHKLJKO\LQIRUPDWLYH,KDYHFRPHWRVHHWKHWUXWKLQWKH
IROORZLQJDVVHUWLRQE\*UHHQHDQG&DUDFHOOL µ,QWKHRU\PL[HGPHWKRGV
LQTXLU\FDQEHDPHDQVIRUH[SORULQJGLIIHUHQFHVDIRUXPIRUGLDORJXHRUDQ
RSSRUWXQLW\WREHWWHUXQGHUVWDQGGLIIHUHQWZD\VRIVHHLQJNQRZLQJDQGYDOXLQJ¶7KLV
PHVVDJHLVUHPLQLVFHQWRI0LOHVDQG+XEHUPDQ¶V ZDUQLQJDERXWWKHGDQJHUV
RIIDOOLQJLQWRDPRQRPHWKRGRORJLFDOµGHIDXOW¶PRGHFKDUDFWHUL]HGE\DQ
µXQTXHVWLRQLQJSDUWLVDQIUDPHRIPLQG¶$QGOHWPHDOVRUHLWHUDWHKHUH/D]DUDWRQ¶V
FRQFOXVLRQWKDWKDVDOUHDG\EHHQFLWHGµ)LQDOO\,ZRXOGDOVRKRSHWKDWZH
ZRXOGVHHPRUHVWXGLHVWKDWFRPELQHTXDOLWDWLYHDQGTXDQWLWDWLYHUHVHDUFKPHWKRGV
VLQFHHDFKKLJKOLJKWV³UHDOLW\´LQDGLIIHUHQW\HWFRPSOHPHQWDU\ZD\¶
)LQDOZRUGV

0\XOWLPDWHUHFRPPHQGDWLRQIRUPHWKRGVHOHFWLRQZRXOGEHIRUUHVHDUFKHUVWRVLPSO\
FKRRVHWKHPHWKRGWKDWWKH\DUHPRVWFRPIRUWDEOHZLWK,ILUPO\EHOLHYHWKDWZHFDQ
ILQGRXWYDOXDEOHWKLQJVDERXWDOPRVWDQ\WRSLFE\XVLQJUHVHDUFKVWUDWHJLHVDVVRFLDWHG
ZLWKDQ\RIWKHWKUHHPDLQPHWKRGRORJLFDOSDUDGLJPVGHVFULEHGLQWKLVERRN7KDWLV
SHUVRQDOO\,GRQRWWKLQNWKDWWKHPHWKRGLWVHOILVWKHGHFLVLYHIDFWRULQUHVHDUFK²LWLV
ZKDWZHGRZLWKLQWKHPHWKRG7KHUHDOFKDOOHQJHPD\QRWEHWRILQGWKH
SDUDGLJPDWLFDOO\EHVWZD\RIH[DPLQLQJDWRSLFRILQWHUHVWEXWUDWKHUWRPDLQWDLQ
SHUVRQDOIUHVKQHVVDQGFUHDWLYLW\ZKLOHSXUVXLQJDGLVFLSOLQHGDQGV\VWHPDWLFDSSURDFK
,DUJXHGLQ&KDSWHUWKDWWKHIRXUIXQGDPHQWDOIHDWXUHVWKDWFDQKHOSDUHVHDUFKHUWR
DFKLHYHH[FHOOHQFHDUHFXULRVLW\FRPPRQVHQVHJRRGLGHDVDQGDPL[WXUHRI
GLVFLSOLQHUHOLDELOLW\DQGDFFRXQWDELOLW\0HWKRGRORJLFDOH[SHUWLVHLQLWVHOIZLOOQRW
PDNHXVJRRGUHVHDUFKHUVDOWKRXJKLWZLOOKHOSXVWRDYRLGGRLQJEDGUHVHDUFK
7KHUHLVRIFRXUVHDGHFLVLRQPDNLQJOHYHOWKDWLVEH\RQGUHVHDUFKPHWKRGRORJLFDO
FRQVLGHUDWLRQVWKHUHVHDUFKHU¶VPRUDOSROLWLFDOVWDQFHDQGLQWHJULW\/RXUGHV2UWHJD
KDVVXPPDUL]HGWKLVVRZHOOWKDW,DPKDSS\WRJLYHWKHILQDOVD\WRKHU
,KDYHDUJXHGWKDWLQWKHXOWLPDWHDQDO\VLVLWLVQRWWKHPHWKRGVRUWKH
HSLVWHPRORJLHVWKDWMXVWLI\WKHOHJLWLPDF\DQGTXDOLW\RIKXPDQUHVHDUFKEXWWKH
PRUDOSROLWLFDOSXUSRVHVWKDWJXLGHVXVWDLQHGUHVHDUFKHIIRUWV$QGLWLVRXU
UHVSRQVLELOLW\DVUHVHDUFKHUVWRFKRRVHRXUYDOXHFRPPLWPHQWVDVLWLVWRFKRRVH
RXUWKHRUHWLFDOIUDPHZRUNVDQGPHWKRGRORJLHVDQGOHWWKRVHYDOXHVLQIRUPWKH
GHVLJQFRQGXFWDQGGLVVHPLQDWLRQRIRXUVWXGLHVDQGRIWKHUHVHDUFKSURJUDPV
WKH\LQVWDQWLDWH

$IWHUZRUG
:KLOH,ZDVLQWKHSURFHVVRIZULWLQJWKLVERRN,QRWLFHGDWRQHSRLQWZKDWVKRXOGKDYH
EHHQREYLRXVWRPHDORQJWLPHEHIRUHQDPHO\WKDWZKDW,ZDVGRLQJFRUUHVSRQGHG
DOPRVWSHUIHFWO\ZLWKWKHSURFHVVRIFRQGXFWLQJTXDOLWDWLYHUHVHDUFK8VLQJD
FRPELQDWLRQRIµW\SLFDOVDPSOLQJ¶DQGµFULWHULRQVDPSOLQJ¶WKHSDUWLFLSDQWVRIP\ERRN
SURMHFWZHUHWKHYDULRXVDXWKRUVRIWKHPHWKRGRORJ\WH[WVWKDW,KDGUHDG7KHVDPSOLQJ
SURFHGXUHZDVµSXUSRVLYHWKHRUHWLFDO¶,VWRSSHGFROOHFWLQJGDWDIRUDQ\VSHFLILFWRSLF
ZKHQVDWXUDWLRQZDVUHDFKHGDQG,ORRNHGIRUQHZGDWD LHOLWHUDWXUH WRH[DPLQH
IUHVKO\HPHUJLQJLVVXHVHYHQDWWKHYHU\ILQDOVWDJHVRISUHSDULQJWKHPDQXVFULSW
)XUWKHUPRUHWKLVKDVXQGRXEWHGO\EHHQDQH[WHQGHGHQJDJHPHQWZLWKWKHGDWD²IRU
RYHUHLJKWPRQWKV,ZDVLPPHUVHGLQWKHWRSLFGD\DQGQLJKW
7KHWRWDOGDWDVHW²WKDWLVWKHWRWDODPRXQWRIOLWHUDWXUH,KDYHUHDG²DPRXQWHGWR
WHQVRIWKRXVDQGVRISDJHVRIWH[WDQGLQRUGHUWRSURFHVVWKHVH,XVHGDµWHPSODWH
DSSURDFK¶RIFRGLQJ,VWDUWHGRXWZLWKDEURDGWHPSODWH WKHFKDSWHUKHDGLQJVDQGVRPH
VXEKHDGLQJV DQGWKHFRGLQJSURFHVVLQYROYHGVHOHFWLQJRYHUDWKRXVDQGTXRWDWLRQV
IURPWKHYDULRXVWH[WVXQGHUWKHVHKHDGLQJV,VFDQQHGWKHVHH[WUDFWVZLWKP\
LQYDOXDEOH DQGKLJKO\UHFRPPHQGHG JDGJHWWKHµ&3HQ¶ZKLFKVDYHGPHWKH
ODERULRXVWDVNRIWUDQVFULSWLRQ LHFRS\LQJ 'XULQJWKHFRGLQJSURFHVVWKHWHPSODWH
LHWKHVWUXFWXUHRIWKHERRN ZDVILQHWXQHGDQGPRGLILHGVHYHUDOWLPHVDQG,NHSW
JRLQJEDFNDQGUHFRGLQJWKHGDWDDFFRUGLQJO\
$OWKRXJKWKHPDLQWKHPHRIP\ERRN²WKHFRPSOHPHQWDU\YDOXHVRITXDOLWDWLYHDQG
TXDQWLWDWLYHUHVHDUFKDQGWKHSURPLVLQJSRVVLELOLW\RIFRPELQLQJWKHPWREHVWHIIHFW
WKURXJKPL[HGPHWKRGVUHVHDUFK²KDGEHHQGHYHORSLQJLQP\PLQGIRUVHYHUDO\HDUV
WKHILQDOFRQWHQWDQGWKHVSHFLILFDQJOHRIVHYHUDOFKDSWHUVDFWXDOO\HPHUJHGGXULQJWKH
DQDO\VLVZULWLQJXSVWDJH,QGHHGWKHWDEOHRIFRQWHQWVVXEPLWWHGWR2[IRUG8QLYHUVLW\
3UHVVDVSDUWRIWKHERRNSURSRVDORULJLQDOO\LQFOXGHGRQO\FKDSWHUVZKLFKLQFUHDVHG
LQP\ILUVWGHWDLOHGWHPSODWHWRDQGVHWWOHGLQWKHILQDOYHUVLRQWR$QGLI,
FRPSDUHWKHVXEKHDGLQJVLQWKHSURSRVDODQGWKHILQDOYHUVLRQLWLVLQSODFHVDVLI,ZDV
ORRNLQJDWWZRGLIIHUHQWERRNV
'XULQJWKHGDWDDQDO\VLVSURFHVV,NHSWZULWLQJH[WHQVLYHPHPRVDQG,DOVRVWDUWHGWR
SUHSDUHDILUVWGUDIWRIWKHPDQXVFULSWHDUO\RQLQWKHSURMHFWEHFDXVH,IRXQGWKDWLQ
OLQHZLWKTXDOLWDWLYHSULQFLSOHVP\XQGHUVWDQGLQJGHYHORSHGVLJQLILFDQWO\GXULQJWKH
ZULWLQJSURFHVV3UHSDULQJWKHILQDOPDQXVFULSWLQYROYHGGUDIWLQJUHGUDIWLQJDQG
UHYLVLQJWKHWH[WWKXVDOORZLQJWKHPDWHULDOWRGHYHORSLWVRZQVKDSHDQGFKDUDFWHU,
H[SHULHQFHGGLUHFWO\WKHQHYHUHQGLQJQDWXUHRIWKHLWHUDWLYHSURFHVVRIWKHTXDOLWDWLYH
LQTXLU\²KDGLWQRWEHHQIRUWKHSXEOLVKHU¶VGHDGOLQH,ZRXOGEHVWLOOUHILQLQJWKH
QDUUDWLYH
$V,ORRNEDFN,ILQGWKDWP\ZULWLQJVW\OHZDVDOVRFRQVLVWHQWZLWKTXDOLWDWLYH
SULQFLSOHV6WDUWLQJLQWKH3UHIDFH,VSHFLILHGP\LGHRORJLFDOVWDQGSRLQWDQG

WKURXJKRXWWKHERRN,DOORZHGP\YRLFHWREHKHDUG+RZHYHU,GLGQRWZDQWWKLVWRJR
DWWKHH[SHQVHRIRWKHUSHRSOH¶VYRLFHVDQG,WKHUHIRUHLQFOXGHGPDQ\OLWHUDOTXRWDWLRQV
WRFRQYH\µLQVLGHU¶YLHZV,DOVRVKDUHGSHUVRQDOIHHOLQJVDQGHYHQFHUWDLQGRXEWVDW
VRPHSRLQWVEXW,GLGVWULYHIRUDEDODQFHGKDQGOLQJRIWKHGDWD$VDUHVXOWWKLVLVD
ERRNZKLFKSUHVHQWVP\SHUVRQDOWDNHRQUHVHDUFKPHWKRGRORJ\EXWLQIRUPHGE\
SDUWLFLSDQWPHDQLQJWKURXJKRXW,QVXP,UHDOL]HGWKDWZKDW,ZDVGRLQJZDV
TXDOLWDWLYHUHVHDUFKSURSHU 8QIRUWXQDWHO\WKLVHQGLQJUHPLQGVPHVRPXFKRIWKHMRNH
RIWKHOLWWOHER\ZKRFODLPHGWKDWKHVSRNH)UHQFK:KHQKLVIULHQGVDVNHGKLPWRVD\
VRPHWKLQJLQ)UHQFKKHVDLGµ-D¶µ%XWWKDW¶V*HUPDQ¶VDLGKLVIULHQGVµ:RZ¶
H[FODLPHGWKHOLWWOHER\µ,GLGQ¶WNQRZ,FRXOGVSHDN*HUPDQWRR«¶

%LEOLRJUDSK\
$EEXKO5DQG$0DFNH\,QSUHVVµ6HFRQGODQJXDJHUHVHDUFKPHWKRGV¶LQ..LQJ
DQG1+RUQEHUJHU HGV (QF\FORSHGLDRI/DQJXDJHDQG(GXFDWLRQ9RO
5HVHDUFK0HWKRGV1HZ<RUN6SULQJHU
$EUDPV'DQG0$+RJJ HGV 6RFLDO,GHQWLW\DQG6RFLDO&RJQLWLRQ

2[IRUG%ODFNZHOO
$GROSKV6,QWURGXFLQJ(OHFWURQLF7H[W$QDO\VLV$3UDFWLFDO*XLGHIRU
/DQJXDJHDQG/LWHUDU\6WXGLHV/RQGRQ5RXWOHGJH
$(5$µ(WKLFDO6WDQGDUGV¶KWWSZZZDHUDQHWDERXWDHUD"LG
$(5$$3$DQG1&0(6WDQGDUGVIRU(GXFDWLRQDODQG3V\FKRORJLFDO
7HVWLQJ:DVKLQJWRQ'&$XWKRU
$LNHQ/5DWLQJ6FDOHVDQG&KHFNOLVWV(YDOXDWLQJ%HKDYLRU3HUVRQDOLW\DQG
$WWLWXGHV1HZ<RUN-RKQ:LOH\
$OGHUVRQ&-'LDJQRVLQJ)RUHLJQ/DQJXDJH3URILFLHQF\7KH,QWHUIDFH
EHWZHHQ$VVHVVPHQWDQG/HDUQLQJ/RQGRQ&RQWLQXXP
$OOZULJKW'µ'HYHORSLQJSULQFLSOHVIRUSUDFWLWLRQHUUHVHDUFKWKHFDVHRI
H[SORUDWRU\SUDFWLFH¶0RGHUQ/DQJXDJH-RXUQDO±
$OOZULJKW'DQG.%DLOH\)RFXVRQWKH/DQJXDJH&ODVVURRP$Q
,QWURGXFWLRQWR/DQJXDJH&ODVVURRP5HVHDUFKIRU/DQJXDJH7HDFKHUV1HZ<RUN
$OWULFKWHU+DQG0/+ROO\µ5HVHDUFKGLDULHV¶LQ%6RPHNKDQG&/HZLQ
HGV 5HVHDUFK0HWKRGVLQWKH6RFLDO6FLHQFHV/RQGRQ6DJH
$3$3XEOLFDWLRQ0DQXDORIWKH$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQWK
(GLWLRQ:DVKLQJWRQ'&$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQ
$UNVH\+DQG3.QLJKW,QWHUYLHZLQJIRU6RFLDO6FLHQWLVWV/RQGRQ6DJH
%DFKPDQ/)Dµ/LQNLQJREVHUYDWLRQVWRLQWHUSUHWDWLRQVDQGXVHVLQ7(62/
UHVHDUFK¶7(62/4XDUWHUO\±
%DFKPDQ/)E6WDWLVWLFDO$QDO\VHVIRU/DQJXDJH$VVHVVPHQW&DPEULGJH

%DFKPDQ/)µ*HQHUDOL]DELOLW\DMRXUQH\LQWRWKHQDWXUHRIHPSLULFDOUHVHDUFK
LQDSSOLHGOLQJXLVWLFV¶LQ0&KDOKRXE'HYLOOH&$&KDSHOOHDQG3'XII HGV
,QIHUHQFHDQG*HQHUDOL]DELOLW\LQ$SSOLHG/LQJXLVWLFV0XOWLSOH3HUVSHFWLYHV
$PVWHUGDP-RKQ%HQMDPLQV
%DFKPDQ/)DQG$63DOPHU/DQJXDJH7HVWLQJLQ3UDFWLFH2[IRUG
%DLOH\.0µ7KHXVHRIGLDU\VWXGLHVLQWHDFKHUHGXFDWLRQSURJUDPPHV¶LQ-
&5LFKDUGVDQG'1XQDQ HGV 6HFRQG/DQJXDJH7HDFKHU(GXFDWLRQ&DPEULGJH
%DNHU&,QSUHVVµ6XUYH\PHWKRGVLQUHVHDUFKLQJODQJXDJHDQGHGXFDWLRQ¶LQ..LQJ
DQG1+RUQEHUJHU HGV (QF\FORSHGLDRI/DQJXDJHDQG(GXFDWLRQ9RO
5HVHDUFK0HWKRGV1HZ<RUN6SULQJHU
%DOOXHUND1-*zPH]DQG'+LGDOJRµ7KHFRQWURYHUV\RYHUQXOO
K\SRWKHVLVVLJQLILFDQFHWHVWLQJUHYLVLWHG¶0HWKRGRORJ\±
%DUWHOV1µ3URIHVVLRQDOSUHSDUDWLRQDQGDFWLRQUHVHDUFKRQO\IRUODQJXDJH
WHDFKHUV"¶7(62/4XDUWHUO\±
%D]HOH\3µ&RPSXWHUL]HGGDWDDQDO\VLVIRUPL[HGPHWKRGVUHVHDUFK¶LQ$
7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ6RFLDODQG
%HKDYLRUDO5HVHDUFK7KRXVDQGV2DNV&DOLI6DJH
%HOO/µ3XEOLFDQGSULYDWHPHDQLQJLQGLDULHV5HVHDUFKLQJIDPLO\DQG
FKLOGFDUH¶LQ$%U\PDQDQG5*%XUJHVV HGV 4XDOLWDWLYH5HVHDUFK9RO
/RQGRQ6DJH
%HWWHOKHLP% HG $*RRG(QRXJK3DUHQW/RQGRQ7KDPHVDQG+XGVRQ
%LUQEDXP0+µ+XPDQUHVHDUFKDQGGDWDFROOHFWLRQYLDWKH,QWHUQHW¶$QQXDO
5HYLHZRI3V\FKRORJ\±
%ODWFKIRUG3µ$PXOWLPHWKRGDSSURDFKWRWKHVWXG\RIVFKRROFODVVVL]H
GLIIHUHQFHV¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\±
%ORFN'µ3UREOHPDWL]LQJLQWHUYLHZGDWDYRLFHVLQWKHPLQG¶VPDFKLQH"¶
7(62/4XDUWHUO\±
%ROJHU1$'DYLVDQG(5DIDHOLµ'LDU\PHWKRGV&DSWXULQJOLIHDVLWLV
OLYHG¶$QQXDO5HYLHZRI3V\FKRORJ\±
%UDQQHQ-0L[HG0HWKRGV5HVHDUFK$'LVFXVVLRQ3DSHU6RXWKDPSWRQ
(65&1DWLRQDO&HQWUHIRU5HVHDUFK0HWKRGV $YDLODEOHRQOLQH

KWWSZZZQFUPDFXNSXEOLFDWLRQVGRFXPHQWV0HWKRGV5HYLHZ3DSHU1&50
SGI
%UHZHU-DQG$+XQWHU0XOWLPHWKRG5HVHDUFK$6\QWKHVLVRI6W\OHV
1HZEXU\3DUN&DOLI6DJH
%ULQJHU-'/+-RKQVWRQDQG&+%UDFNHQULGJHµ0D[LPL]LQJ
WUDQVSDUHQF\LQDGRFWRUDOWKHVLV7KHFRPSOH[LWLHVRIZULWLQJDERXWWKHXVHRI
465 19,92ZLWKLQDJURXQGHGWKHRU\VWXG\¶4XDOLWDWLYH5HVHDUFK±
%URZQ-'8QGHUVWDQGLQJ5HVHDUFKLQ6HFRQG/DQJXDJH/HDUQLQJ
&DPEULGJH&DPEULGJH8QLYHUVLW\3UHVV
%URZQ-'8VLQJ6XUYH\VLQ/DQJXDJH3URJUDPV&DPEULGJH&DPEULGJH
8QLYHUVLW\3UHVV
%URZQ-'µ5HVHDUFKPHWKRGVIRUDSSOLHGOLQJXLVWLFVVFRSHFKDUDFWHULVWLFV
DQGVWDQGDUGV¶LQ$'DYLVDQG&(OGHU HGV 7KH+DQGERRNRI$SSOLHG
/LQJXLVWLFV2[IRUG%ODFNZHOO
%URZQ-'7HVWLQJLQ/DQJXDJH3URJUDPV$&RPSUHKHQVLYH*XLGHWR
(QJOLVK/DQJXDJH$VVHVVPHQW1HZ<RUN0F*UDZ+LOO
%URZQ-'DQG765RGJHUV'RLQJ6HFRQG/DQJXDJH5HVHDUFK2[IRUG
%U\PDQ$µ,QWHJUDWLQJTXDQWLWDWLYHDQGTXDOLWDWLYHUHVHDUFK+RZLVLWGRQH"¶
4XDOLWDWLYH5HVHDUFK±
%XFKQHU$((UGIHOGHUDQG))DXOµ+RZWR8VH* 3RZHU¶
KWWSZZZSV\FKRXQLGXHVVHOGRUIGHDDSSURMHFWVJSRZHU
KRZBWRBXVHBJSRZHUKWPO
%XUQV$µ$FWLRQUHVHDUFK¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ6HFRQG

/DQJXDJH7HDFKLQJDQG/HDUQLQJ0DKZDK1-/DZUHQFH(UOEDXP
&DPSEHOO'7DQG-&6WDQOH\µ([SHULPHQWDODQGTXDVLH[SHULPHQWDO
GHVLJQVIRUUHVHDUFKRQWHDFKLQJ¶LQ1*DJH HG +DQGERRNRI5HVHDUFKRQ
7HDFKLQJ&KLFDJR5DQG0F1DOO\
&DPSEHOO'7DQG6:)LVNHµ&RQYHUJHQWDQGGLVFULPLQDQWYDOLGDWLRQE\
WKHPXOWLWUDLWPXOWLPHWKRGPDWUL[¶3V\FKRORJLFDO%XOOHWLQ±
&DUDFHOOL9-DQG-&*UHHQHµ'DWDDQDO\VLVVWUDWHJLHVIRUPL[HGPHWKRG
HYDOXDWLRQGHVLJQV¶(GXFDWLRQDO(YDOXDWLRQDQG3ROLF\$QDO\VLV±

&DUWHU5$*RGGDUG'5HDK.6DQJHUDQG0%RZULQJ:RUNLQJZLWK
7H[WV$&RUH,QWURGXFWLRQWR/DQJXDJH$QDO\VLV/RQGRQ5RXWOHGJH
&KDOKRXE'HYLOOH0µ'UDZLQJWKHOLQHWKHJHQHUDOL]DELOLW\DQGOLPLWDWLRQVRI
UHVHDUFKLQDSSOLHGOLQJXLVWLFV¶LQ0&KDOKRXE'HYLOOH&$&KDSHOOHDQG3'XII
HGV ,QIHUHQFHDQG*HQHUDOL]DELOLW\LQ$SSOLHG/LQJXLVWLFV0XOWLSOH3HUVSHFWLYHV
&KDSHOOH&$µ9DOLGLW\LQODQJXDJHDVVHVVPHQW¶$QQXDO5HYLHZRI$SSOLHG
/LQJXLVWLFV±
&KDSHOOH&$DQG3'XII HGV µ6RPHJXLGHOLQHVIRUFRQGXFWLQJTXDQWLWDWLYH

DQGTXDOLWDWLYHUHVHDUFKLQ7(62/¶7(62/4XDUWHUO\±
&KDUPD].µ*URXQGHGWKHRU\LQWKHVWFHQWXU\$SSOLFDWLRQVIRUDGYDQFLQJ
VRFLDOMXVWLFHVWXGLHV¶LQ1.'HQ]LQDQG<6/LQFROQ HGV 7KH6DJH+DQGERRN
RI4XDOLWDWLYH5HVHDUFKUG(GLWLRQ7KRXVDQG2DNV&DOLI6DJH
&KDXGURQ&6HFRQG/DQJXDJH&ODVVURRPV5HVHDUFKRQ7HDFKLQJDQG
/HDUQLQJ1HZ<RUN&DPEULGJH8QLYHUVLW\3UHVV
&KDXGURQ&µ'DWDFROOHFWLRQLQ6/$UHVHDUFK¶LQ&-'RXJKW\DQG0+
/RQJ HGV 7KH+DQGERRNRI6HFRQG/DQJXDJH$FTXLVLWLRQ2[IRUG%ODFNZHOO
&KHUU\KROPHV&+µ1RWHVRQSUDJPDWLVPDQGVFLHQWLILFUHDOLVP¶(GXFDWLRQDO
5HVHDUFKHU±
&ODUNH''µ³6WUXFWXUHGMXGJHPHQWPHWKRGV´7KHEHVWRIERWKZRUOGV"¶LQ=
7RGG%1HUOLFK60F.HRZQDQG''&ODUNH HGV 0L[LQJ0HWKRGVLQ
3V\FKRORJ\7KH,QWHJUDWLRQRI4XDOLWDWLYHDQG4XDQWLWDWLYH0HWKRGVLQ7KHRU\DQG
3UDFWLFH+RYH(DVW6XVVH[3V\FKRORJ\3UHVV
&RFKUDQ:*6DPSOLQJ7HFKQLTXHVUG(GLWLRQ1HZ<RUN:LOH\
&RKHQ$'6WUDWHJLHVLQ/HDUQLQJDQG8VLQJD6HFRQG/DQJXDJH+DUORZ
/RQJPDQ
&RKHQ//0DQLRQDQG.0RUULVRQ5HVHDUFK0HWKRGVLQ(GXFDWLRQWK
(GLWLRQ/RQGRQ5RXWOHGJH)DOPHU
&ROOLQV/0µ$QDO\VLVRIORQJLWXGLQDOGDWDWKHLQWHJUDWLRQRIWKHRUHWLFDO
PRGHOWHPSRUDOGHVLJQDQGVWDWLVWLFDOPRGHO¶$QQXDO5HYLHZRI3V\FKRORJ\
±
&RQYHUVH-0DQG63UHVVHU6XUYH\4XHVWLRQV+DQGFUDIWLQJWKH
6WDQGDUGL]HG4XHVWLRQQDLUH1HZEXU\3DUN&DOLI6DJH

&RRN7'DQG'7&DPSEHOO4XDVL([SHULPHQWDWLRQ'HVLJQDQG$QDO\VLV
,VVXHVIRU)LHOG6HWWLQJV&KLFDJR5DQG0F1DOO\
&UDEWUHH%)DQG:/0LOOHUµ8VLQJFRGHVDQGFRGHPDQXDOV¶LQ%)
&UDEWUHHDQG:/0LOOHU HGV 'RLQJ4XDOLWDWLYH5HVHDUFK/RQGRQ6DJH
&UHVZHOO-:5HVHDUFK'HVLJQ4XDOLWDWLYHDQG4XDQWLWDWLYH$SSURDFKHV
7KRXVDQG2DNV&DOLI6DJH
&UHVZHOO-:5HVHDUFK'HVLJQ4XDOLWDWLYH4XDQWLWDWLYHDQG0L[HG0HWKRGV
$SSURDFKHV7KRXVDQG2DNV&DOLI6DJH
&UHVZHOO-:3ODQQLQJ&RQGXFWLQJDQG(YDOXDWLQJ4XDQWLWDWLYHDQG
4XDOLWDWLYH5HVHDUFK8SSHU6DGGOH5LYHU1-3HDUVRQ3UHQWLFH+DOO
&UHVZHOO-:9/3ODQR&ODUN0/*XWPDQQDQG:(+DQVRQ
µ$GYDQFHGPL[HGPHWKRGVUHVHDUFKGHVLJQV¶LQ$7DVKDNNRULDQG&7HGGOLH HGV
+DQGERRNRI0L[HG0HWKRGVLQ6RFLDODQG%HKDYLRUDO5HVHDUFK7KRXVDQGV2DNV
&DOLI6DJH
&URRNHV*µ$FWLRQUHVHDUFKIRUVHFRQGODQJXDJHWHDFKHUVJRLQJEH\RQG
WHDFKHUUHVHDUFK¶$SSOLHG/LQJXLVWLFV±
&U\HU37KH5HVHDUFK6WXGHQW¶V*XLGHWR6XFFHVVQG(GLWLRQ%XFNLQJKDP
2SHQ8QLYHUVLW\3UHVV
'DYLG0µ3UREOHPVRISDUWLFLSDWLRQWKHOLPLWVRIDFWLRQUHVHDUFK¶
,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\±
'DYLGVRQ)3ULQFLSOHVRI6WDWLVWLFDO'DWD+DQGOLQJ7KRXVDQG2DNV&DOLI
6DJH
'DYLHV.$µ4XDOLWDWLYHWKHRU\DQGPHWKRGVLQDSSOLHGOLQJXLVWLFVUHVHDUFK¶
7(62/4XDUWHUO\
'HQ]LQ1.7KH5HVHDUFK$FW$7KHRUHWLFDO,QWURGXFWLRQWR6RFLRORJLFDO
0HWKRGV(QJOHZRRG&OLIIV1-3UHQWLFH+DOO
'HQ]LQ1.DQG<6/LQFROQDµ,QWURGXFWLRQ7KHGLVFLSOLQHDQGSUDFWLFHRI
TXDOLWDWLYHUHVHDUFK¶LQ1.'HQ]LQDQG<6/LQFROQ HGV 7KH6DJH+DQGERRN
RI4XDOLWDWLYH5HVHDUFKUG(GLWLRQ7KRXVDQG2DNV&DOLI6DJH
'HQ]LQ1.DQG<6/LQFROQ HGV E7KH6DJH+DQGERRNRI4XDOLWDWLYH

5HVHDUFKUG(GLWLRQ7KRXVDQG2DNV&DOLI6DJH

'H\,µ*URXQGHGWKHRU\¶LQ&6HDOH**RER-)*XEULXPDQG'6LOYHUPDQ
HGV 4XDOLWDWLYH5HVHDUFK3UDFWLFH/RQGRQ6DJH
'|UQ\HL=7HDFKLQJDQG5HVHDUFKLQJ0RWLYDWLRQ+DUORZ/RQJPDQ
'|UQ\HL=4XHVWLRQQDLUHVLQ6HFRQG/DQJXDJH5HVHDUFK&RQVWUXFWLRQ
$GPLQLVWUDWLRQDQG3URFHVVLQJ0DKZDK1-/DZUHQFH(UOEDXP
'|UQ\HL=7KH3V\FKRORJ\RIWKH/DQJXDJH/HDUQHU,QGLYLGXDO'LIIHUHQFHVLQ
6HFRQG/DQJXDJH$FTXLVLWLRQ0DKZDK1-/DZUHQFH(UOEDXP
'|UQ\HL=DQG-.RUPRVµ7KHUROHRILQGLYLGXDODQGVRFLDOYDULDEOHVLQRUDO
WDVNSHUIRUPDQFH¶/DQJXDJH7HDFKLQJ5HVHDUFK±
'|UQ\HL=DQG70XUSKH\*URXS'\QDPLFVLQWKH/DQJXDJH&ODVVURRP
&DPEULGJH&DPEULGJH8QLYHUVLW\3UHVV
'|UQ\HL=.&VL]pUDQG11pPHWK0RWLYDWLRQ/DQJXDJH$WWLWXGHVDQG
*OREDOLVDWLRQ$+XQJDULDQ3HUVSHFWLYH&OHYHGRQ(QJODQG0XOWLOLQJXDO0DWWHUV
'|UQ\HL=9'XURZDQG.=DKUDQµ,QGLYLGXDOGLIIHUHQFHVDQGWKHLUHIIHFW
RQIRUPXODLFVHTXHQFHDFTXLVLWLRQ¶LQ16FKPLWW HG )RUPXODLF6HTXHQFHV
'XII3µ5HVHDUFKDSSURDFKHVLQDSSOLHGOLQJXLVWLFV¶LQ5%.DSODQ HG 7KH

2[IRUG+DQGERRNRI$SSOLHG/LQJXLVWLFV1HZ<RUN2[IRUG8QLYHUVLW\3UHVV
'XII3µ%H\RQGJHQHUDOL]DELOLW\FRQWH[WXDOL]DWLRQFRPSOH[LW\DQGFUHGLELOLW\
LQDSSOLHGOLQJXLVWLFVUHVHDUFK¶LQ0&KDOKRXE'HYLOOH&$&KDSHOOHDQG3'XII
HGV ,QIHUHQFHDQG*HQHUDOL]DELOLW\LQ$SSOLHG/LQJXLVWLFV0XOWLSOH3HUVSHFWLYHV
'XII3,QSUHVV&DVH6WXG\5HVHDUFKLQ$SSOLHG/LQJXLVWLFV0DKZDK1-
/DZUHQFH(UOEDXP
'XII3DQG0(DUO\µ3UREOHPDWLFVRIFODVVURRPUHVHDUFKDFURVVVRFLRSROLWLFDO
FRQWH[WV¶LQ-6FKDFKWHUDQG6*DVV HGV 6HFRQG/DQJXDJH&ODVVURRP5HVHDUFK
,VVXHVDQG2SSRUWXQLWLHV0DKZDK1-/DZUHQFH(UOEDXP
'XUDQWL$/LQJXLVWLF$QWKURSRORJ\&DPEULGJH&DPEULGJH8QLYHUVLW\3UHVV
(LVHQKDUGW.0µ%XLOGLQJWKHRULHVIURPFDVHVWXG\UHVHDUFK¶$FDGHP\RI
0DQDJHPHQW5HYLHZ±
(OOLV5DQG*%DUNKXL]HQ$QDO\VLQJ/HDUQHU/DQJXDJH2[IRUG2[IRUG
8QLYHUVLW\3UHVV

(ULFVVRQ.$µ7RZDUGVDSURFHGXUHIRUHOLFLWLQJYHUEDOH[SUHVVLRQRIQRQ
YHUEDOH[SHULHQFHZLWKRXWUHDFWLYLW\LQWHUSUHWLQJWKHYHUEDORYHUVKDGRZLQJHIIHFW
ZLWKLQWKHWKHRUHWLFDOIUDPHZRUNIRUSURWRFRODQDO\VLV¶$SSOLHG&RJQLWLYH
3V\FKRORJ\±
(ULFVVRQ.$DQG+$6LPRQµ9HUEDOUHSRUWVRQWKLQNLQJ¶LQ&) UFKDQG
*.DVSHU HGV ,QWURVSHFWLRQLQ6HFRQG/DQJXDJH5HVHDUFK&OHYHGRQ(QJODQG
0XOWLOLQJXDO0DWWHUV
(U]EHUJHU&DQG8.HOOHµ0DNLQJLQIHUHQFHVLQPL[HGPHWKRGVWKHUXOHVRI
LQWHJUDWLRQ¶LQ$7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ
6RFLDODQG%HKDYLRUDO5HVHDUFK7KRXVDQGV2DNV&DOLI6DJH
) UFK&DQG*.DVSHU HGV ,QWURVSHFWLRQLQ6HFRQG/DQJXDJH5HVHDUFK

&OHYHGRQ0XOWLOLQJXDO0DWWHUV
)DVVLQJHU5(µ3DUDGLJPVSUD[LVSUREOHPVDQGSURPLVH*URXQGHGWKHRU\LQ
FRXQVHOLQJSV\FKRORJ\UHVHDUFK¶-RXUQDORI&RXQVHOLQJ3V\FKRORJ\±
)RQWDQD$DQG-+)UH\µ7KHLQWHUYLHZ¶LQ1.'HQ]LQDQG<6/LQFROQ
HGV 7KH6DJH+DQGERRNRI4XDOLWDWLYH5HVHDUFKUG(GLWLRQ7KRXVDQG2DNV
&DOLI6DJH
)R[-&0XUUD\DQG$:DUPµ&RQGXFWLQJUHVHDUFKXVLQJZHEEDVHG
TXHVWLRQQDLUHVSUDFWLFDOPHWKRGRORJLFDODQGHWKLFDOFRQVLGHUDWLRQV¶,QWHUQDWLRQDO
-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\±
)XOFKHU*DQG)'DYLGVRQ/DQJXDJH7HVWLQJDQG$VVHVVPHQW/RQGRQ
5RXWOHGJH
*DOO0'-3*DOODQG:5%RUJ(GXFDWLRQDO5HVHDUFK$Q,QWURGXFWLRQ
WK(GLWLRQ%RVWRQ0DVV3HDUVRQ
*DUGQHU5&6RFLDO3V\FKRORJ\DQG6HFRQG/DQJXDJH/HDUQLQJ7KH5ROHRI
$WWLWXGHVDQG0RWLYDWLRQ/RQGRQ(GZDUG$UQROG
*DUUDWW'DQG</Lµ7KHIRXQGDWLRQVRIH[SHULPHQWDOHPSLULFDOUHVHDUFK
PHWKRGV¶LQ%6RPHNKDQG&/HZLQ HGV 5HVHDUFK0HWKRGVLQWKH6RFLDO
6FLHQFHV/RQGRQ6DJH
*DVV60DQG$0DFNH\6WLPXODWHG5HFDOO0HWKRGRORJ\LQ6HFRQG
/DQJXDJH5HVHDUFK0DKZDK1-/DZUHQFH(UOEDXP
*KHUDUGL6DQG%7XUQHUµ5HDOPHQGRQ¶WFROOHFWVRIWGDWD¶LQ$%U\PDQ
HG 4XDOLWDWLYH5HVHDUFK9RO/RQGRQ6DJH

*LEVRQ9µ$QDQDO\VLVRIWKHXVHRIGLDULHVDVDGDWDFROOHFWLRQPHWKRG¶1XUVH
5HVHDUFKHU±
*LOOKDP%'HYHORSLQJD4XHVWLRQQDLUH/RQGRQ&RQWLQXXP
*ODVHU%*%DVLFVRI*URXQGHG7KHRU\$QDO\VLV(PHUJHQFHYV)RUFLQJ0LOO
9DOOH\&DOLI6RFLRORJ\3UHVV
*ODVHU%*DQG$/6WUDXVV7KH'LVFRYHU\RI*URXQGHG7KHRU\6WUDWHJLHV
IRU4XDOLWDWLYH5HVHDUFK1HZ<RUN$OGLQH
*ODVV*9µ3ULPDU\VHFRQGDU\DQGPHWDDQDO\VLVRIUHVHDUFK¶(GXFDWLRQDO
5HVHDUFK±
*UHHQ$DQG-3UHVWRQµ(GLWRULDO6SHDNLQJLQWRQJXHV'LYHUVLW\LQPL[HG
PHWKRGVUHVHDUFK¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\±

*UHHQH-&DQG9-&DUDFHOOLµ0DNLQJSDUDGLJPDWLFVHQVHRIPL[HG
PHWKRGVSUDFWLFH¶LQ$7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI0L[HG
0HWKRGVLQ6RFLDODQG%HKDYLRUDO5HVHDUFK7KRXVDQGV2DNV&DOLI6DJH
*UHHQH-&+.UHLGHUDQG(0D\HUµ&RPELQLQJTXDOLWDWLYHDQG
TXDQWLWDWLYHPHWKRGVLQVRFLDOLQTXLU\¶LQ%6RPHNKDQG&/HZLQ HGV 5HVHDUFK
0HWKRGVLQWKH6RFLDO6FLHQFHV/RQGRQ6DJH
*UHHQH-&9-&DUDFHOOLDQG:)*UDKDPµ7RZDUGDFRQFHSWXDO
IUDPHZRUNIRUPL[HGPHWKRGHYDOXDWLRQGHVLJQV¶(GXFDWLRQDO(YDOXDWLRQDQG
3ROLF\$QDO\VLV±
*ULVVRP5-DQG--.LP(IIHFW6L]HVIRU5HVHDUFK$%URDG3UDFWLFDO
$SSURDFK0DKZDK1-/DZUHQFH(UOEDXP
*XLOORWHDX[0-DQG='|UQ\HL,QSUHVVµ0RWLYDWLQJODQJXDJHOHDUQHUVD
FODVVURRPRULHQWHGLQYHVWLJDWLRQRIWKHHIIHFWVRIPRWLYDWLRQDOVWUDWHJLHVRQVWXGHQW
PRWLYDWLRQ¶7(62/4XDUWHUO\
+DPPHUVOH\0DQG3$WNLQVRQ(WKQRJUDSK\3ULQFLSOHVDQG3UDFWLFHQG
(GLWLRQ/RQGRQ5RXWOHGJH
+DPPRQG&µ7KHZLGHUEHQHILWVRIDGXOWOHDUQLQJ$QLOOXVWUDWLRQRIWKH
DGYDQWDJHVRIPXOWLPHWKRGUHVHDUFK¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK
0HWKRGRORJ\±

+DQVRQ:(-:&UHVZHOO9/3ODQR&ODUN.63HWVNDDQG-'&UHVZHOO
µ0L[HGPHWKRGVUHVHDUFKGHVLJQVLQFRXQVHOLQJSV\FKRORJ\¶-RXUQDORI
&RXQVHOLQJ3V\FKRORJ\±
+DUGHQ$DQG-7KRPDVµ0HWKRGRORJLFDOLVVXHVLQFRPELQLQJGLYHUVHVWXG\
W\SHVLQV\VWHPDWLFUHYLHZV¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\
±
+DUNODX/µ(WKQRJUDSK\DQGHWKQRJUDSKLFUHVHDUFKRQVHFRQGODQJXDJH
WHDFKLQJDQGOHDUQLQJ¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ6HFRQG
+DWFK(0DQG$/D]DUDWRQ7KH5HVHDUFK0DQXDO'HVLJQDQG6WDWLVWLFV
IRU$SSOLHG/LQJXLVWLFV1HZ<RUN1HZEXU\+RXVH
+DYHUNDPS%(µ(WKLFDOSHUVSHFWLYHVRQTXDOLWDWLYHUHVHDUFKLQDSSOLHG
SV\FKRORJ\¶-RXUQDORI&RXQVHOLQJ3V\FKRORJ\±
+HLQVPDQ'7DQG:56KDGLVKµ$VVLJQPHQWPHWKRGVLQH[SHULPHQWDWLRQ
ZKHQGRQRQUDQGRPL]HGH[SHULPHQWVDSSUR[LPDWHDQVZHUVIURPUDQGRPL]HG
H[SHULPHQWV"¶3V\FKRORJLFDO0HWKRGV±
+HVVH%LEHU16DQG3/HDY\7KH3UDFWLFHRI4XDOLWDWLYH5HVHDUFK
+ROOLGD\$'RLQJDQG:ULWLQJ4XDOLWDWLYH5HVHDUFK/RQGRQ6DJH
+ROOLGD\$µ,VVXHVRIYDOLGLW\LQSURJUHVVLYHSDUDGLJPVRITXDOLWDWLYHUHVHDUFK¶
7(62/4XDUWHUO\±
+RUQEHUJHU1µ(WKQRJUDSK\¶7(62/4XDUWHUO\±
+VLHK37$FHH:+&KXQJ<3+VLHK+.LP*'7KRPDV-<RXDQG-
//HYRQµ,VHGXFDWLRQDOLQWHUYHQWLRQUHVHDUFKRQWKHGHFOLQH"¶-RXUQDORI
(GXFDWLRQDO3V\FKRORJ\±
+XEHUPDQ$0DQG0%0LOHVµ,QWURGXFWLRQ¶LQ$0+XEHUPDQDQG0
%0LOHV HGV 7KH4XDOLWDWLYH5HVHDUFK&RPSDQLRQ7KRXVDQG2DNV&DOLI6DJH
-RKQVRQ'0$SSURDFKHVWR5HVHDUFKLQ6HFRQG/DQJXDJH/HDUQLQJ1HZ
<RUN/RQJPDQ
-RKQVRQ5%DQG/&KULVWHQVHQ(GXFDWLRQ5HVHDUFK4XDQWLWDWLYH
4XDOLWDWLYHDQG0L[HG$SSURDFKHVQG(GLWLRQ%RVWRQ$OO\QDQG%DFRQ

-RKQVRQ5%DQG$-2QZXHJEX]LHµ0L[HGPHWKRGVUHVHDUFKDUHVHDUFK
SDUDGLJPZKRVHWLPHKDVFRPH¶(GXFDWLRQDO5HVHDUFKHU±
-RKQVRQ5%DQG/$7XUQHUµ'DWDFROOHFWLRQVWUDWHJLHVLQPL[HGPHWKRGV
UHVHDUFK¶LQ$7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ
-RUGDQ*7KHRU\&RQVWUXFWLRQLQ6HFRQG/DQJXDJH$FTXLVLWLRQ$PVWHUGDP
-RKQ%HQMDPLQV
.DVSHU*µ$QDO\VLQJYHUEDOSURWRFROV¶7(62/4XDUWHUO\±
.HHYHV-3µ/RQJLWXGLQDOUHVHDUFKPHWKRGV¶LQ7+XVpQDQG71
3RVWOHWKZDLWH HGV 7KH,QWHUQDWLRQDO(QF\FORSHGLDRI(GXFDWLRQ9ROQG
(GLWLRQ2[IRUG3HUJDPRQ
.HOOH8µ&RPSXWHUDVVLVWHGTXDOLWDWLYHGDWDDQDO\VLV¶LQ&6HDOH**RER-)
*XEULXPDQG'6LOYHUPDQ HGV 4XDOLWDWLYH5HVHDUFK3UDFWLFH/RQGRQ6DJH
.HPSHU($66WULQJILHOGDQG&7HGGOLHµ0L[HGPHWKRGVVDPSOLQJ
VWUDWHJLHVLQVRFLDOUHVHDUFK¶LQ$7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI
0L[HG0HWKRGVLQ6RFLDODQG%HKDYLRUDO5HVHDUFK7KRXVDQGV2DNV&DOLI6DJH
.LUN-DQG0/0LOOHU5HOLDELOLW\DQG9DOLGLW\LQ4XDOLWDWLYH5HVHDUFK
1HZEXU\3DUN&DOLI6DJH
.RMLF6DER,DQG30/LJKWERZQµ6WXGHQWV DSSURDFKHVWRYRFDEXODU\
OHDUQLQJDQGWKHLUUHODWLRQVKLSWRVXFFHVV¶0RGHUQ/DQJXDJH-RXUQDO±
.RUPRV-µ9HUEDOUHSRUWVLQ/VSHHFKSURGXFWLRQUHVHDUFK¶7(62/4XDUWHUO\
±
.UDVKHQ6µ,QGLYLGXDOYDULDWLRQLQWKHXVHRIWKHPRQLWRU¶LQ:5LWFKLH HG
6HFRQG/DQJXDJH$FTXLVLWLRQ5HVHDUFK1HZ<RUN$FDGHPLF3UHVV
.YDOH6,QWHUYLHZV$Q,QWURGXFWLRQWR4XDOLWDWLYH5HVHDUFK,QWHUYLHZLQJ
.YDOH6µ'RPLQDQFHWKURXJKLQWHUYLHZVDQGGLDORJXHV¶4XDOLWDWLYH,QTXLU\
±
/DSDGDW-&µ3UREOHPDWL]LQJWUDQVFULSWLRQSXUSRVHSDUDGLJPDQGTXDOLW\¶

/DU]HOHUH5(%5.XKQDQG%-RKQVRQµ7KHLQWHUYHQWLRQVHOHFWLRQELDV
$QXQGHUUHFRJQL]HGFRQIRXQGLQLQWHUYHQWLRQUHVHDUFK¶3V\FKRORJLFDO%XOOHWLQ
±
/DXUHQFHDX-3DQG1%ROJHUµ8VLQJGLDU\PHWKRGVWRVWXG\PDULWDODQG
IDPLO\SURFHVVHV¶-RXUQDORI)DPLO\3V\FKRORJ\±
/D]DUDWRQ$µ&XUUHQWWUHQGVLQUHVHDUFKPHWKRGRORJ\DQGVWDWLVWLFVLQDSSOLHG
OLQJXLVWLFV¶7(62/4XDUWHUO\±
/D]DUDWRQ$µ(YDOXDWLYHFULWHULDIRUTXDOLWDWLYHUHVHDUFKLQDSSOLHGOLQJXLVWLFV
:KRVHFULWHULDDQGZKRVHUHVHDUFK"¶0RGHUQ/DQJXDJH-RXUQDO±
/D]DUDWRQ$µ4XDQWLWDWLYHUHVHDUFKPHWKRGV¶LQ(+LQNHO HG +DQGERRNRI

5HVHDUFKLQ6HFRQG/DQJXDJH7HDFKLQJDQG/HDUQLQJ0DKZDK1-/DZUHQFH
(UOEDXP
/HXQJ&5+DUULVDQG%5DPSWRQµ/LYLQJZLWKLQHOHJDQFHLQTXDOLWDWLYH
UHVHDUFKRQWDVNEDVHGOHDUQLQJ¶LQ%1RUWRQDQG.7RRKH\ HGV &ULWLFDO
3HGDJRJLHVDQG/DQJXDJH/HDUQLQJ1HZ<RUN&DPEULGJH8QLYHUVLW\3UHVV
/HY\36DQG6/HPHVKRZ6DPSOLQJRI3RSXODWLRQV0HWKRGVDQG
$SSOLFDWLRQVUG(GLWLRQ1HZ<RUN-RKQ:LOH\DPG6RQV
/HZLQV$DQG&6LOYHU,QSUHVV8VLQJ6RIWZDUHLQ4XDOLWDWLYH5HVHDUFK$6WHSE\
6WHS*XLGH/RQGRQ6DJH
/LJKWERZQ30µ&ODVVURRP6/$UHVHDUFKDQGVHFRQGODQJXDJHWHDFKLQJ¶
$SSOLHG/LQJXLVWLFV±
/LQFROQ<6DQG(**XED1DWXUDOLVWLF(QTXLU\1HZEXU\3DUN&DOLI
6DJH
/LVNLQ*DVSDUUR-( HG µ)URPWKH$VVRFLDWH(GLWRU0/-5HYLHZV

3UHVHQWLQJWKH6SHFLDO,VVXH5HYLHZV¶0RGHUQ/DQJXDJH-RXUQDO±
/\QFK%./DQJXDJH$VVHVVPHQWDQG3URJUDPPH(YDOXDWLRQ(GLQEXUJK
(GLQEXUJK8QLYHUVLW\3UHVV
0DF,QW\UH3'DQG5&*DUGQHUµ7KHVXEWOHHIIHFWVRIODQJXDJHDQ[LHW\RQ
FRJQLWLYHSURFHVVLQJLQWKHVHFRQGODQJXDJH¶/DQJXDJH/HDUQLQJ±
0DFNH\$DQG60*DVV6HFRQG/DQJXDJH5HVHDUFK0HWKRGRORJ\DQG
'HVLJQ0DKZDK1-/DZUHQFH(UOEDXP

0DJQDQ6µ)URPWKH(GLWRU0/-SROLFLHVDQGSUDFWLFHVWKHLUHYROXWLRQIURP
WRWKH\HDU¶0RGHUQ/DQJXDJH-RXUQDO
0DJQDQ6µ)URPWKH(GLWRUWKH0/-WXUQVLQDGLJLWDODJH¶0RGHUQ
/DQJXDJH-RXUQDO±
0DUNHH1µ5HRSHQLQJWKHUHVHDUFKDJHQGDUHVSHFLI\LQJPRWLYDWLRQDVD
ORFDOO\RFFDVLRQHGSKHQRPHQRQ¶3DSHUSUHVHQWHGDWWKHDQQXDOFRQIHUHQFHRIWKH
$PHULFDQ$VVRFLDWLRQRI$SSOLHG/LQJXLVWLFV $$$/ 6W/RXLV0LVV86$
0DVRQ-µ0L[LQJPHWKRGVLQDTXDOLWDWLYHO\GULYHQZD\¶4XDOLWDWLYH5HVHDUFK
±
0D[ZHOO-$µ8QGHUVWDQGLQJWKHYDOLGLW\LQTXDOLWDWLYHUHVHDUFK¶+DUYDUG
(GXFDWLRQDO5HYLHZ±
0D[ZHOO-$DQG'0/RRPLVµ0L[HGPHWKRGVGHVLJQDQDOWHUQDWLYH
DSSURDFK¶LQ$7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ
0F&UDFNHQ*7KH/RQJ,QWHUYLHZ1HZEXU\3DUN&DOLI6DJH
0F'RQRXJK-DQG60F'RQRXJK5HVHDUFK0HWKRGVIRU(QJOLVK/DQJXDJH
7HDFKHUV/RQGRQ$UQROG
0F'RQRXJK.µ$FWLRQUHVHDUFKDQGWKHSURIHVVLRQDOGHYHORSPHQWRIJUDGXDWH
WHDFKLQJDVVLVWDQWV¶0RGHUQ/DQJXDJH-RXUQDO±
0F.D\6/5HVHDUFKLQJ6HFRQG/DQJXDJH&ODVVURRPV0DKZDK1-
/DZUHQFH(UOEDXP
0F/HRG-µ:K\ZHLQWHUYLHZQRZUHIOH[LYLW\DQGSHUVSHFWLYHLQD
ORQJLWXGLQDOVWXG\¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\
±
0F1DPDUD7µ9DOLGLW\DQGYDOXHVLQIHUHQFHVDQGJHQHUDOL]DELOLW\LQODQJXDJH
WHVWLQJ¶LQ0&KDOKRXE'HYLOOH&$&KDSHOOHDQG3'XII HGV ,QIHUHQFHDQG
*HQHUDOL]DELOLW\LQ$SSOLHG/LQJXLVWLFV0XOWLSOH3HUVSHFWLYHV$PVWHUGDP-RKQ
%HQMDPLQV
0HOORZ-'.5HHGHUDQG()RUVWHUµ8VLQJWLPHVHULHVUHVHDUFKGHVLJQV
WRLQYHVWLJDWHWKHHIIHFWVRILQVWUXFWLRQRQ6/$¶6WXGLHVLQ6HFRQG/DQJXDJH
$FTXLVLWLRQ±
0HQDUG6/RQJLWXGLQDO5HVHDUFKQG(GLWLRQ7KRXVDQG2DNV&DOLI6DJH

0HUWHQV'05HVHDUFKDQG(YDOXDWLRQLQ(GXFDWLRQDQG3V\FKRORJ\
,QWHJUDWLQJ'LYHUVLW\ZLWK4XDQWLWDWLYH4XDOLWDWLYHDQG0L[HG0HWKRGVQG
(GLWLRQ7KRXVDQG2DNV&DOLI6DJH
0LOHV0%DQG$0+XEHUPDQ4XDOLWDWLYH'DWD$QDO\VLVQG(GLWLRQ
0LOOHU:/DQG%)&UDEWUHHµ'HSWKLQWHUYLHZLQJ¶LQ%)&UDEWUHHDQG:
/0LOOHU HGV 'RLQJ4XDOLWDWLYH5HVHDUFK/RQGRQ6DJH
0LOUR\/DQG0*RUGRQ6RFLROLQJXLVWLFV0HWKRGDQG,QWHUSUHWDWLRQ
2[IRUG%ODFNZHOO
0RUDQ(OOLV-9'$OH[DQGHU$&URQLQ0'LFNLQVRQ-)LHOGLQJ-6OHQH\
DQG+7KRPDV 7ULDQJXODWLRQDQGLQWHJUDWLRQ3URFHVVHVFODLPVDQG
LPSOLFDWLRQV 4XDOLWDWLYH5HVHDUFK±
0RUURZ6/µ4XDOLW\DQGWUXVWZRUWKLQHVVLQTXDOLWDWLYHUHVHDUFKLQFRXQVHOLQJ
SV\FKRORJ\¶-RXUQDORI&RXQVHOLQJ3V\FKRORJ\±
0RUVH-0DQG/5LFKDUGV5HDGPH)LUVWIRUD8VHU¶V*XLGHWR4XDOLWDWLYH
5HVHDUFK7KRXVDQG2DNV&DOLI6DJH
0RVHU&$DQG*.DOWRQ6XUYH\0HWKRGVLQ6RFLDO,QYHVWLJDWLRQ/RQGRQ
+HLQHPDQQ
0RVNRZLW]*µ,QWHUDFWLRQDQDO\VLV$QHZPRGHUQODQJXDJHIRUVXSHUYLVRUV¶)RUHLJQ
/DQJXDJH$QQDOV±
1HDOH%DQG-)ORZHUGHZµ7LPHWH[WXUHDQGFKLOGKRRGWKHFRQWRXUVRI
ORQJLWXGLQDOTXDOLWDWLYHUHVHDUFK¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK
0HWKRGRORJ\
1RUULV-DQG/2UWHJDµ(IIHFWLYHQHVVRI/LQVWUXFWLRQ$UHVHDUFKV\QWKHVLV
DQGTXDQWLWDWLYHPHWDDQDO\VLV¶/DQJXDJH/HDUQLQJ±
1RUULV-DQG/2UWHJDµ'HILQLQJDQGPHDVXULQJ6/$¶LQ&-'RXJKW\DQG
0+/RQJ HGV 7KH+DQGERRNRI6HFRQG/DQJXDJH$FTXLVLWLRQ2[IRUG
%ODFNZHOO
1RUWRQ3HLUFH%µ7KHWKHRU\RIPHWKRGRORJ\LQTXDOLWDWLYHUHVHDUFK¶7(62/
4XDUWHUO\±
1XQDQ'5HVHDUFK0HWKRGVLQ/DQJXDJH/HDUQLQJ&DPEULGJH&DPEULGJH
8QLYHUVLW\3UHVV

1XQDQ'µ&ODVVURRPUHVHDUFK¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ
6HFRQG/DQJXDJH7HDFKLQJDQG/HDUQLQJ0DKZDK1-/DZUHQFH(UOEDXP
2¶&RQQRU$DQG-6KDUNH\µ'HILQLQJWKHSURFHVVRIWHDFKHUUHVHDUFKHU
FROODERUDWLRQ¶7(62/4XDUWHUO\±
2QZXHJEX]LH$-DQG&7HGGOLHµ$IUDPHZRUNIRUDQDO\]LQJGDWDLQPL[HG
PHWKRGVUHVHDUFK¶LQ$7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI0L[HG
2QZXHJEX]LH$-DQG1//HHFKµ2QEHFRPLQJDSUDJPDWLFUHVHDUFKHUWKH
LPSRUWDQFHRIFRPELQLQJTXDQWLWDWLYHDQGTXDOLWDWLYHUHVHDUFKPHWKRGRORJLHV¶
2SSHQKHLP$14XHVWLRQQDLUH'HVLJQ,QWHUYLHZLQJDQG$WWLWXGH
0HDVXUHPHQW1HZ(GLWLRQ/RQGRQ3LQWHU
2UWHJD/µ0HWKRGRORJ\HSLVWHPRORJ\DQGHWKLFVLQLQVWUXFWHG6/$UHVHDUFK
$QLQWURGXFWLRQ¶0RGHUQ/DQJXDJH-RXUQDO±
2UWHJD/DQG*,EHUUL6KHDµ/RQJLWXGLQDOUHVHDUFKLQVHFRQGODQJXDJH
DFTXLVLWLRQ5HFHQWWUHQGVDQGIXWXUHGLUHFWLRQV¶$QQXDO5HYLHZRI$SSOLHG
/LQJXLVWLFV±
3DOODQW-63666XUYLYDO0DQXDOQG(GLWLRQ0DLGHQKHDG(QJODQG2SHQ
8QLYHUVLW\3UHVVDQG0F*UDZ+LOO(GXFDWLRQ
3DWWRQ044XDOLWDWLYH5HVHDUFKDQG(YDOXDWLRQ0HWKRGVUG(GLWLRQ
3DYOHQNR$µ3RVWVWUXFWXUDOLVWDSSURDFKHVWRWKHVWXG\RIVRFLDOIDFWRUVLQ
VHFRQGODQJXDJHOHDUQLQJDQGXVH¶LQ9&RRN HG 3RUWUDLWVRIWKH/8VHU
&OHYHGRQ0XOWLOLQJXDO0DWWHUV
3LFD7µ&ODVVURRPOHDUQLQJWHDFKLQJDQGUHVHDUFKDWDVNEDVHGSHUVSHFWLYH¶
0RGHUQ/DQJXDJH-RXUQDO±
3ROLR&*µ,VVXHVDQGSUREOHPVLQUHSRUWLQJFODVVURRPUHVHDUFK¶LQ-
6FKDFKWHUDQG6*DVV HGV 6HFRQG/DQJXDJH&ODVVURRP5HVHDUFK,VVXHVDQG
2SSRUWXQLWLHV0DKZDK1-/DZUHQFH(UOEDXP
3RONLQJKRUQH'(µ/DQJXDJHDQGPHDQLQJ'DWDFROOHFWLRQLQTXDOLWDWLYH
UHVHDUFK¶-RXUQDORI&RXQVHOLQJ3V\FKRORJ\±
3XQFK.),QWURGXFWLRQWR6RFLDO5HVHDUFKQG(GLWLRQ7KRXVDQG2DNV
&DOLI6DJH

5DPSWRQ%&URVVLQJ/DQJXDJHDQG(WKQLFLW\DPRQJ$GROHVFHQWV+DUORZ
/RQJPDQ
5LFKDUGV.4XDOLWDWLYH,QTXLU\LQ7(62/%DVLQJVWRNH3DOJUDYH0DFPLOODQ
5LFKDUGV/+DQGOLQJ4XDOLWDWLYH'DWD$3UDFWLFDO*XLGH/RQGRQ6DJH
5LFKDUGVRQ/DQG($6W3LHUUHµ:ULWLQJDPHWKRGRILQTXLU\¶LQ1.
'HQ]LQDQG<6/LQFROQ HGV 7KH6DJH+DQGERRNRI4XDOLWDWLYH5HVHDUFKUG
(GLWLRQ7KRXVDQG2DNV&DOLI6DJH
5REHUWV&µ7UDQVFULELQJWDONLVVXHVRIUHSUHVHQWDWLRQ¶7(62/4XDUWHUO\
±
5REHUWV&0%\UDP$%DUUR6-RUGDQDQG%6WUHHW/DQJXDJH
/HDUQHUVDV(WKQRJUDSKHUV&OHYHGRQ0XOWLOLQJXDO0DWWHUV
3UDFWLWLRQHU5HVHDUFKHUV2[IRUG%ODFNZHOO
3UDFWLWLRQHU5HVHDUFKHUVQG(GLWLRQ2[IRUG%ODFNZHOO
5RVVLWHU0-µ7KHFKDOOHQJHVRIFODVVURRPEDVHG6/$UHVHDUFK¶$SSOLHG
/DQJXDJH/HDUQLQJ±
5RVVPDQ*%DQG%/:LOVRQµ1XPEHUVDQGZRUGVFRPELQLQJTXDQWLWDWLYH
DQGTXDOLWDWLYHPHWKRGVLQDVLQJOHODUJHVFDOHHYDOXDWLRQVWXG\¶(YDOXDWLRQ5HYLHZ
±
5RXQGV3/µ7KHFODVVURRPEDVHGUHVHDUFKHUDVILHOGZRUNHUVWUDQJHUVLQD
VWUDQJHODQG¶LQ-6FKDFKWHUDQG6*DVV HGV 6HFRQG/DQJXDJH&ODVVURRP
5HVHDUFK,VVXHVDQG2SSRUWXQLWLHV0DKZDK1-/DZUHQFH(UOEDXP
5XELR2*µ(WKQRJUDSKLFLQWHUYLHZPHWKRGVLQUHVHDUFKLQJODQJXDJHDQG
HGXFDWLRQ¶LQ1+RUQEHUJHUDQG'&RUVRQ HGV (QF\FORSHGLDRI/DQJXDJHDQG
(GXFDWLRQ9RO5HVHDUFKPHWKRGVLQODQJXDJHDQGHGXFDWLRQ'RUGUHFKW
1HWKHUODQGV.OXZHU
5XVSLQL(,QWURGXFWLRQWR/RQJLWXGLQDO5HVHDUFK/RQGRQ5RXWOHGJH
5\HQ$µ(WKLFDOLVVXHV¶LQ&6HDOH**RER-)*XEULXPDQG'6LOYHUPDQ
HGV 4XDOLWDWLYH5HVHDUFK3UDFWLFH/RQGRQ6DJH

6DQGHORZVNL0µ7DEOHVRIWDEOHDX["7KHFKDOOHQJHVRIZULWLQJDQGUHDGLQJ
PL[HGPHWKRGVVWXGLHV¶LQ$7DVKDNNRULDQG&7HGGOLH HGV +DQGERRNRI0L[HG
6DQGHORZVNL0DQG-%DUURVRµ5HDGLQJTXDOLWDWLYHVWXGLHV¶,QWHUQDWLRQDO
-RXUQDORI4XDOLWDWLYH0HWKRGV RQOLQHMRXUQDO $UWLFOH
6FKDFKWHU-DQG6*DVVµ,QWURGXFWLRQ¶LQ-6FKDFKWHUDQG6*DVV HGV
6HFRQG/DQJXDJH&ODVVURRP5HVHDUFK,VVXHVDQG2SSRUWXQLWLHV0DKZDK1-
/DZUHQFH(UOEDXP
6FKLHIIOLQ%DQG(2FKV HGV /DQJXDJH6RFLDOL]DWLRQDFURVV&XOWXUHV1HZ

<RUN&DPEULGJH8QLYHUVLW\3UHVV
6FKLIIULQ'$SSURDFKHVWR'LVFRXUVH2[IRUG%ODFNZHOO
6FKLIIULQ''7DQQHQDQG+(+DPLOWRQ HGV 7KH+DQGERRNRI

'LVFRXUVH$QDO\VLV2[IRUG%ODFNZHOO
6FKPLGW5DQG6)1)URWDµ'HYHORSLQJEDVLFFRQYHUVDWLRQDODELOLW\LQD
VHFRQGODQJXDJHDFDVHVWXG\RIDQDGXOWOHDUQHURI3RUWXJXHVH¶LQ5'D\ HG
7DONLQJWR/HDUQ$&DVH6WXG\RIDQ$GXOW/HDUQHURI3RUWXJXHVH5RZOH\0DVV
1HZEXU\+RXVH
6FKZDQGW7$µ7KUHHHSLVWHPRORJLFDOVWDQFHVIRUTXDOLWDWLYHLQTXLU\¶LQ1.
'HQ]LQDQG<6/LQFROQ HGV +DQGERRNRI4XDOLWDWLYH5HVHDUFKQG(GLWLRQ
6HDOH&**RER-)*XEULXPDQG'6LOYHUPDQµ,QWURGXFWLRQ,QVLGH
TXDOLWDWLYHUHVHDUFK¶LQ&6HDOH**RER-)*XEULXPDQG'6LOYHUPDQ HGV
4XDOLWDWLYH5HVHDUFK3UDFWLFH/RQGRQ6DJH
6HLGPDQ,,QWHUYLHZLQJDV4XDOLWDWLYH5HVHDUFK$*XLGHIRU5HVHDUFKHUVLQ
(GXFDWLRQDQGWKH6RFLDO6FLHQFHVQG(GLWLRQ1HZ<RUN7HDFKHUV&ROOHJH3UHVV
6HOLJHU+:DQG(6KRKDP\6HFRQG/DQJXDJH5HVHDUFK0HWKRGV2[IRUG
6pURU-µ&RPSXWHUVDQGTXDOLWDWLYHGDWDDQDO\VLV3DSHUSHQVDQGKLJKOLJKWHUV
YVVFUHHQPRXVHDQGNH\ERDUG¶7(62/4XDUWHUO\±
6KDYHOVRQ5-DQG/7RZQH6FLHQWLILF5HVHDUFKLQ(GXFDWLRQ:DVKLQJWRQ
'&1DWLRQDO$FDGHPLHV3UHVV
6KRKDP\(µ5HIOHFWLRQVRQUHVHDUFKJXLGHOLQHVFDWHJRULHVDQGUHVSRQVLELOLW\¶
7(62/4XDUWHUO\±

6LOYHUPDQ'µ7KHORJLFVRITXDOLWDWLYHUHVHDUFK¶LQ*0LOOHUDQG5'LQJZDOO
HGV &RQWH[WDQG0HWKRGLQ4XDOLWDWLYH5HVHDUFK/RQGRQ6DJH
6LOYHUPDQ',QWHUSUHWLQJ4XDOLWDWLYH'DWD0HWKRGVIRU$QDO\VLQJ7DON7H[W
DQG,QWHUDFWLRQQG(GLWLRQ/RQGRQ6DJH
6LOYHUPDQ''RLQJ4XDOLWDWLYH5HVHDUFKQG(GLWLRQ/RQGRQ6DJH
6NHKDQ3,QGLYLGXDO'LIIHUHQFHVLQ6HFRQG/DQJXDJH/HDUQLQJ/RQGRQ
(GZDUG$UQROG
6PLWK*7µ2QFRQVWUXFWYDOLGLW\,VVXHVRIPHWKRGDQGPHDVXUHPHQW¶
3V\FKRORJLFDO$VVHVVPHQW±
6PLWKVRQ-µ8VLQJDQGDQDO\VLQJIRFXVJURXSVOLPLWDWLRQVDQGSRVVLELOLWLHV¶
6SDGD1µ&RQGLWLRQVDQGFKDOOHQJHVLQGHYHORSLQJVFKRROEDVHG6/$UHVHDUFK
SURJUDPV¶0RGHUQ/DQJXDJH-RXUQDO±
6SDGD1DQG0)U|KOLFK&ROW2EVHUYDWLRQ6FKHPH6\GQH\0DFTXDULH
8QLYHUVLW\1DWLRQDO&HQWUHIRU(QJOLVK/DQJXDJH7HDFKLQJDQG5HVHDUFK
6SDGD1/5DQWDDQG30/LJKWERZQµ:RUNLQJZLWKWHDFKHUVLQVHFRQG
ODQJXDJHDFTXLVLWLRQUHVHDUFK¶LQ-6FKDFKWHUDQG6*DVV HGV 6HFRQG/DQJXDJH
&ODVVURRP5HVHDUFK,VVXHVDQG2SSRUWXQLWLHV0DKZDK1-/DZUHQFH(UOEDXP
6WDNH5(7KH$UWRI&DVH6WXG\5HVHDUFK7KRXVDQG2DNV&DOLI6DJH
6WDNH5(µ4XDOLWDWLYHFDVHVWXGLHV¶LQ1.'HQ]LQDQG<6/LQFROQ HGV
7KH6DJH+DQGERRNRI4XDOLWDWLYH5HVHDUFKUG(GLWLRQ7KRXVDQG2DNV&DOLI
6DJH
6WUDWPDQ-)DQG/+DPS/\RQV 5HDFWLYLW\LQFRQFXUUHQWWKLQNDORXG
SURWRFROV LQ36PDJRULQVN\ HG 6SHDNLQJ$ERXW:ULWLQJ5HIOHFWLRQVRQ
5HVHDUFK0HWKRGRORJ\7KRXVDQG2DNV&DOLI6DJH
6WUDXVV$/DQG-&RUELQ%DVLFVRI4XDOLWDWLYH5HVHDUFK7HFKQLTXHVDQG
3URFHGXUHVIRU'HYHORSLQJ*URXQGHG7KHRU\QG(GLWLRQ7KRXVDQG2DNV&DOLI
6DJH
6XGPDQ6DQG10%UDGEXUQ$VNLQJ4XHVWLRQV6DQ)UDQFLVFR&DOLI
-RVVH\%DVV
6ZDLQ0µ9HUEDOSURWRFROVZKDWGRHVLWPHDQIRUUHVHDUFKWRXVHVSHDNLQJDVD
GDWDFROOHFWLRQWRRO"¶LQ0&KDOKRXE'HYLOOH&$&KDSHOOHDQG3'XII HGV

,QIHUHQFHDQG*HQHUDOL]DELOLW\LQ$SSOLHG/LQJXLVWLFV0XOWLSOH3HUVSHFWLYHV
7DEDFKQLFN%*DQG/6)LGHOO8VLQJ0XOWLYDULDWH6WDWLVWLFVWK(GLWLRQ
1HHGKDP+HLJKWV0DVV$OO\QDQG%DFRQ
7DULV7$3ULPHULQ/RQJLWXGLQDO'DWD$QDO\VLV7KRXVDQG2DNV6DJH
7DVKDNNRUL$DQG&7HGGOLH0L[HG0HWKRGRORJ\&RPELQLQJ4XDOLWDWLYH
DQG4XDQWLWDWLYH$SSURDFKHV7KRXVDQG2DNV&DOLI6DJH
7DVKDNNRUL$DQG&7HGGOLH HGV D+DQGERRNRI0L[HG0HWKRGVLQ6RFLDO

DQG%HKDYLRUDO5HVHDUFK7KRXVDQGV2DNV&DOLI6DJH
7DVKDNNRUL$DQG&7HGGOLHEµ,VVXHVDQGGLOHPPDVLQWHDFKLQJUHVHDUFK
PHWKRGVFRXUVHVLQVRFLDODQGEHKDYLRUDOVFLHQFHV86SHUVSHFWLYH¶,QWHUQDWLRQDO
-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\±
7DVKDNNRUL$DQG&7HGGOLHFµ7KHSDVWDQGIXWXUHRIPL[HGPHWKRGV
UHVHDUFK)URPGDWDWULDQJXODWLRQWRPL[HGPRGHOGHVLJQV¶LQ$7DVKDNNRULDQG&
7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ6RFLDODQG%HKDYLRUDO5HVHDUFK
7KRXVDQGV2DNV&DOLI6DJH
7HGGOLH&DQG$7DVKDNNRULµ0DMRULVVXHVDQGFRQWURYHUVLHVLQWKHXVHRI
PL[HGPHWKRGVLQWKHVRFLDODQGEHKDYLRUDOVFLHQFHV¶LQ$7DVKDNNRULDQG&
7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ6RFLDODQG%HKDYLRUDO5HVHDUFK
7KRXVDQGV2DNV&DOLI6DJH
7KRPVRQ5DQG-+ROODQGµ+LQGVLJKWIRUHVLJKWDQGLQVLJKWWKHFKDOOHQJHVRI
ORQJLWXGLQDOTXDOLWDWLYHUHVHDUFK¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK
0HWKRGRORJ\±
7KRPVRQ5/3OXPULGJHDQG-+ROODQGµ/RQJLWXGLQDOTXDOLWDWLYHUHVHDUFK
$GHYHORSLQJPHWKRGRORJ\¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\
±
7RGG=%1HUOLFKDQG60F.HRZQµ,QWURGXFWLRQ¶LQ=7RGG%1HUOLFK
60F.HRZQDQG''&ODUNH HGV 0L[LQJ0HWKRGVLQ3V\FKRORJ\7KH
,QWHJUDWLRQRI4XDOLWDWLYHDQG4XDQWLWDWLYH0HWKRGVLQ7KHRU\DQG3UDFWLFH+RYH
(DVW6XVVH[3V\FKRORJ\3UHVV
7RRKH\.,QSUHVVµ(WKQRJUDSK\DQGODQJXDJHHGXFDWLRQ¶LQ..LQJDQG1
+RUQEHUJHU HGV (QF\FORSHGLDRI/DQJXDJHDQG(GXFDWLRQ9RO5HVHDUFK
PHWKRGV1HZ<RUN6SULQJHU

7VHQJ:7='|UQ\HLDQG16FKPLWWµ$QHZDSSURDFKWRDVVHVVLQJ
VWUDWHJLFOHDUQLQJ7KHFDVHRIVHOIUHJXODWLRQLQYRFDEXODU\DFTXLVLWLRQ¶$SSOLHG
/LQJXLVWLFV±
7XUQHU-&DQG'.0H\HUµ6WXG\LQJDQGXQGHUVWDQGLQJWKHLQVWUXFWLRQDO
FRQWH[WVRIFODVVURRPV8VLQJRXUSDVWWRIRUJHRXUIXWXUH¶(GXFDWLRQDO
3V\FKRORJLVW±
YDQ(HUGH:'+ROPDQDQG37RWWHUGHOOµ6SHFLDOVHFWLRQHGLWRULDO¶
-RXUQDORI2FFXSDWLRQDODQG2UJDQL]DWLRQDO3V\FKRORJ\±
YDQ/LHU/7KH&ODVVURRPDQGWKH/DQJXDJH/HDUQHU(WKQRJUDSK\DQG
6HFRQG/DQJXDJH5HVHDUFK/RQGRQ/RQJPDQ
YDQ/LHU/µ&DVHVWXG\¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ6HFRQG

9HUVFKXUHQ3-0µ&DVHVWXG\DVDUHVHDUFKVWUDWHJ\VRPHDPELJXLWLHVDQG
RSSRUWXQLWLHV¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK0HWKRGRORJ\±
:DWVRQ*HJHR.$µ(WKQRJUDSK\LQ(6/GHILQLQJWKHHVVHQWLDOV¶7(62/
4XDUWHUO\±
:DWVRQ*HJHR.$µ&ODVVURRPHWKQRJUDSK\¶LQ1+RUQEHUJHUDQG'&RUVRQ
HGV (QF\FORSHGLDRI/DQJXDJHDQG(GXFDWLRQ9RO5HVHDUFKPHWKRGVLQ
ODQJXDJHDQGHGXFDWLRQ'RUGUHFKW1HWKHUODQGV.OXZHU
:LGGRZVRQ+*7H[W&RQWH[W3UHWH[W&ULWLFDO,VVXHVLQ'LVFRXUVH$QDO\VLV
2[IRUG%ODFNZHOO
:LOHV56+HDWK*&URZDQG9&KDUOHV,QIRUPHG&RQVHQWLQ6RFLDO
5HVHDUFK$/LWHUDWXUH5HYLHZ6RXWKDPSWRQ(65&1DWLRQDO&HQWUHIRU5HVHDUFK
0HWKRGV
:LONLQVRQ/DQG7)6,µ6WDWLVWLFDOPHWKRGVLQSV\FKRORJ\MRXUQDOV*XLGHOLQHV
DQGH[SODQDWLRQV¶$PHULFDQ3V\FKRORJLVW±
:LQQLFRWW':7KH0DWXUDWLRQDO3URFHVVDQGWKH)DFLOLWDWLQJ(QYLURQPHQW
/RQGRQ+RJDUWK3UHVV
:LQWHU*µ$FRPSDUDWLYHGLVFXVVLRQRIWKHQRWLRQRI³YDOLGLW\´LQTXDOLWDWLYHDQG
TXDQWLWDWLYHUHVHDUFK¶4XDOLWDWLYH5HSRUWDQG
:ROFRWW+)7UDQVIRUPLQJ4XDOLWDWLYH'DWD'HVFULSWLRQ$QDO\VLVDQG
,QWHUSUHWDWLRQ7KRXVDQG2DNV&DOLI6DJH

:RQJ)LOOPRUH/µ,QGLYLGXDOGLIIHUHQFHVLQVHFRQGODQJXDJHDFTXLVLWLRQ¶LQ&
-)LOOPRUH:6<:DQJDQG'.HPSOHU HGV ,QGLYLGXDO'LIIHUHQFHVLQ
/DQJXDJH$ELOLW\DQG/DQJXDJH%HKDYLRU1HZ<RUN$FDGHPLF3UHVV
:UDJJ(&,QWURGXFWLRQWR&ODVVURRP2EVHUYDWLRQQG(GLWLRQ/RQGRQ
5RXWOHGJH)DOPHU
:UD\$.7URWWDQG$%ORRPHU3URMHFWVLQ/LQJXLVWLFV$3UDFWLFDO*XLGH
WR5HVHDUFKLQJ/DQJXDJH/RQGRQ$UQROG
<DWHV/µ,QWHUSUHWLYHFODLPVDQGPHWKRGRORJLFDOZDUUDQWLQVPDOOQXPEHU
TXDOLWDWLYHORQJLWXGLQDOUHVHDUFK¶,QWHUQDWLRQDO-RXUQDORI6RFLDO5HVHDUFK
0HWKRGRORJ\±
=XHQJOHU-DQG-0RULµ0LFURDQDO\VHVRIFODVVURRPGLVFRXUVHDFULWLFDO
FRQVLGHUDWLRQRIPHWKRG²,QWURGXFWLRQWRWKH6SHFLDO,VVXH¶$SSOLHG/LQJXLVWLFV
±
=XHQJOHU-&)RUGDQG&)DVVQDFKWµ$QDO\VWH\HVDQGFDPHUDH\HV
WKHRUHWLFDODQGWHFKQRORJLFDOFRQVLGHUDWLRQVLQ³VHHLQJ´WKHGHWDLOVRIFODVVURRP
LQWHUDFWLRQ¶$OEDQ\1<6FKRRORI(GXFDWLRQ8QLYHUVLW\DW$OEDQ\

Research Method, Zoltan

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Research Method, Zoltan

Transféré par

Droits d'auteur :

Formats disponibles

5HVHDUFK0HWKRGVLQ$SSOLHG/LQJXLVWLFV

)ROORZLQJ%URZQ  ZHFDQLGHQWLI\WZREDVLFZD\VRIILQGLQJDQVZHUVWR

'HQ]LQ1.DQG<6/LQFROQ HGV 7KH6DJH+DQGERRNRI4XDOLWDWLYH

7DVKDNNRUL$DQG&7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ6RFLDO

/HWPHFLWHRQHPRUHLOOXVWUDWLYHH[WUDFWIURP6WUDXVVDQG&RUELQ¶V  ERRN

+HVVH%LEHUDQG/HDY\  DOVRUDLVHWKHLVVXHRIKRZZHOOYHUVHGDQ\JLYHQ

)LQDOO\0D[ZHOODQG/RRPLV  KLJKOLJKWDIXUWKHULVVXHWKHGLYHUVLW\RIWKH

%DFKPDQ E RIIHUVDGHWDLOHGGHVFULSWLRQRIWZRJHQHUDODSSURDFKHVZKHUHE\

 4XDOLW\RIWKHUHVHDUFKHU0RUVHDQG5LFKDUGV  DUHULJKWZKHQWKH\ZDUQXV

x 5HVSRQGHQWIHHGEDFN RUµUHVSRQGHQWYDOLGDWLRQ¶RUµPHPEHUFKHFNLQJ¶ %HFDXVHRI

7HGGOLHDQG7DVKDNNRUL  SURSRVHGWKDWEHFDXVHPL[HGPHWKRGVUHVHDUFKLVWKH

&ROOLQV  GHVFULEHVDQLQWHUHVWLQJPHWKRGRIREWDLQLQJSURVSHFWLYHOLNHGDWDLQ

x 5HYHUVHDSSURDFK%HFDXVHVWDWLVWLFDOVLJQLILFDQFH VHH6HFWLRQ GHSHQGVRQ

)DFWXDO RUµSHUVRQDO¶RUµFODVVLILFDWLRQ¶ TXHVWLRQV$V2SSHQKHLP  FRQFOXGHV

,QWKHLULQIOXHQWLDOZRUNRQJURXQGHGWKHRU\*ODVHUDQG6WUDXVV  VSRNHDERXW

)LQDOO\+DUNODX  GUDZVDWWHQWLRQWRDSHFXOLDULW\FRQFHUQLQJHWKQRJUDSK\LQ

x (QMR\LW RUDWOHDVWORRNDVWKRXJK\RXGR 'RQ¶WJLYHWKHPHVVDJHWKDW\RXDUHERUHGRU

&UHVZHOOHWDO  ODEHOOHGWKLVFRPELQDWLRQDµVHTXHQWLDOH[SODQDWRU\GHVLJQ¶

7KHVLWXDWLRQDFFRUGLQJWR6FKDFKWHUDQG*DVV  LVDELWOLNHJRLQJWRWKH

 0XOWLVLWHGHVLJQ)LQDOO\'XIIDQG(DUO\  GUDZDWWHQWLRQWRWKHSRVVLEOH

 *LYHWKHGDWDILOHDVLPSOHJHQHULFQDPH IRUH[DPSOHµ0RWLYDWLRQBGDWD¶ WRZKLFK

%HJLQQHU ,QWHUPHGLDWH $GYDQFHG

6H[ 0 6' Q 0 6' Q 0 6' Q

/LVWHQLQJFRPSUHKHQVLRQ   

7HVW  

7HVW  

7HVW  

7HVW  

9LOODJH 7RZQ &LW\ (IIHFW

6HOIFRQILGHQFH        

   

$FKLHYHPHQWPHDVXUH ,QWULQVLFPRWLYDWLRQ ([WULQVLFPRWLYDWLRQ

*3$²/DQJXDJHDUWV  

*3$²0DWKHPDWLFV  

&$76²2YHUDOO  

&$76²5HDGLQJ  

&$76²0DWKHPDWLFV  

1RWH*3$ JUDGHSRLQWDYHUDJH&$76 &DOLIRUQLD$FKLHYHPHQW7HVW

S S S

x &DWWHOO¶V  VFUHHWHVW%\SORWWLQJWKHYDULDQFHDVVRFLDWHGZLWKHDFKIDFWRU

$[LDO RUWKHRUHWLFDO FRGLQJ

x 0RUHVHQVLWLYHVHFRQGOHYHOFRGLQJ+DPHUVOH\DQG$WNLQVRQ  SRLQWRXWWKDW

7KHWZRPRVWFRPPRQ DQGOHJLWLPDWH XVHVRITXDOLWL]HGGDWDDUH D XWLOL]LQJ

&DUDFHOOLDQG*UHHQ  IXUWKHUPHQWLRQWKDWDQDGGLWLRQDOLPSRUWDQWIHDWXUHRI

x 0RUURZ  VXJJHVWVDXVHIXOPRGLILFDWLRQZKHUHE\ZHPHUJHWKHLQVWUXPHQWV

,QDFFHVVLEOHODQJXDJH'XII  PHQWLRQVDIXUWKHUSUREOHPVRXUFHWKDWPLJKW

$EUDPV'DQG0$+RJJ HGV 6RFLDO,GHQWLW\DQG6RFLDO&RJQLWLRQ

%HWWHOKHLP% HG $*RRG(QRXJK3DUHQW/RQGRQ7KDPHVDQG+XGVRQ

%XUQV$µ$FWLRQUHVHDUFK¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ6HFRQG

&KDSHOOH&$DQG3'XII HGV µ6RPHJXLGHOLQHVIRUFRQGXFWLQJTXDQWLWDWLYH

'HQ]LQ1.DQG<6/LQFROQ HGV E7KH6DJH+DQGERRNRI4XDOLWDWLYH

'XII3µ5HVHDUFKDSSURDFKHVLQDSSOLHGOLQJXLVWLFV¶LQ5%.DSODQ HG 7KH

) UFK&DQG*.DVSHU HGV ,QWURVSHFWLRQLQ6HFRQG/DQJXDJH5HVHDUFK

/D]DUDWRQ$µ4XDQWLWDWLYHUHVHDUFKPHWKRGV¶LQ(+LQNHO HG +DQGERRNRI

/LVNLQ*DVSDUUR-( HG µ)URPWKH$VVRFLDWH(GLWRU0/-5HYLHZV

6FKLHIIOLQ%DQG(2FKV HGV /DQJXDJH6RFLDOL]DWLRQDFURVV&XOWXUHV1HZ

6FKLIIULQ''7DQQHQDQG+(+DPLOWRQ HGV 7KH+DQGERRNRI

7DVKDNNRUL$DQG&7HGGOLH HGV D+DQGERRNRI0L[HG0HWKRGVLQ6RFLDO

YDQ/LHU/µ&DVHVWXG\¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ6HFRQG

Vous aimerez peut-être aussi

)ROORZLQJ%URZQ ZHFDQLGHQWLI\WZREDVLFZD\VRIILQGLQJDQVZHUVWR

'HQ]LQ1.DQG<6/LQFROQ HGV 7KH6DJH+DQGERRNRI4XDOLWDWLYH

7DVKDNNRUL$DQG&7HGGOLH HGV +DQGERRNRI0L[HG0HWKRGVLQ6RFLDO

/HWPHFLWHRQHPRUHLOOXVWUDWLYHH[WUDFWIURP6WUDXVVDQG&RUELQ¶V ERRN

+HVVH%LEHUDQG/HDY\ DOVRUDLVHWKHLVVXHRIKRZZHOOYHUVHGDQ\JLYHQ

)LQDOO\0D[ZHOODQG/RRPLV KLJKOLJKWDIXUWKHULVVXHWKHGLYHUVLW\RIWKH

%DFKPDQ E RIIHUVDGHWDLOHGGHVFULSWLRQRIWZRJHQHUDODSSURDFKHVZKHUHE\

4XDOLW\RIWKHUHVHDUFKHU0RUVHDQG5LFKDUGV DUHULJKWZKHQWKH\ZDUQXV

7HGGOLHDQG7DVKDNNRUL SURSRVHGWKDWEHFDXVHPL[HGPHWKRGVUHVHDUFKLVWKH

&ROOLQV GHVFULEHVDQLQWHUHVWLQJPHWKRGRIREWDLQLQJSURVSHFWLYHOLNHGDWDLQ

x 5HYHUVHDSSURDFK%HFDXVHVWDWLVWLFDOVLJQLILFDQFH VHH6HFWLRQ GHSHQGVRQ

)DFWXDO RUµSHUVRQDO¶RUµFODVVLILFDWLRQ¶ TXHVWLRQV$V2SSHQKHLP FRQFOXGHV

,QWKHLULQIOXHQWLDOZRUNRQJURXQGHGWKHRU\*ODVHUDQG6WUDXVV VSRNHDERXW

)LQDOO\+DUNODX GUDZVDWWHQWLRQWRDSHFXOLDULW\FRQFHUQLQJHWKQRJUDSK\LQ

x (QMR\LW RUDWOHDVWORRNDVWKRXJK\RXGR 'RQ¶WJLYHWKHPHVVDJHWKDW\RXDUHERUHGRU

&UHVZHOOHWDO ODEHOOHGWKLVFRPELQDWLRQDµVHTXHQWLDOH[SODQDWRU\GHVLJQ¶

7KHVLWXDWLRQDFFRUGLQJWR6FKDFKWHUDQG*DVV LVDELWOLNHJRLQJWRWKH

0XOWLVLWHGHVLJQ)LQDOO\'XIIDQG(DUO\ GUDZDWWHQWLRQWRWKHSRVVLEOH

*LYHWKHGDWDILOHDVLPSOHJHQHULFQDPH IRUH[DPSOHµ0RWLYDWLRQBGDWD¶ WRZKLFK

/LVWHQLQJFRPSUHKHQVLRQ

7HVW

7HVW

7HVW

7HVW

6HOIFRQILGHQFH

*3$²/DQJXDJHDUWV

*3$²0DWKHPDWLFV

&$76²2YHUDOO

&$76²5HDGLQJ

&$76²0DWKHPDWLFV

1RWH*3$ JUDGHSRLQWDYHUDJH&$76 &DOLIRUQLD$FKLHYHPHQW7HVW

S S S

x &DWWHOO¶V VFUHHWHVW%\SORWWLQJWKHYDULDQFHDVVRFLDWHGZLWKHDFKIDFWRU

x 0RUHVHQVLWLYHVHFRQGOHYHOFRGLQJ+DPHUVOH\DQG$WNLQVRQ SRLQWRXWWKDW

&DUDFHOOLDQG*UHHQ IXUWKHUPHQWLRQWKDWDQDGGLWLRQDOLPSRUWDQWIHDWXUHRI

x 0RUURZ VXJJHVWVDXVHIXOPRGLILFDWLRQZKHUHE\ZHPHUJHWKHLQVWUXPHQWV

,QDFFHVVLEOHODQJXDJH'XII PHQWLRQVDIXUWKHUSUREOHPVRXUFHWKDWPLJKW

$EUDPV'DQG0$+RJJ HGV 6RFLDO,GHQWLW\DQG6RFLDO&RJQLWLRQ

%HWWHOKHLP% HG $*RRG(QRXJK3DUHQW/RQGRQ7KDPHVDQG+XGVRQ

%XUQV$µ$FWLRQUHVHDUFK¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ6HFRQG

&KDSHOOH&$DQG3'XII HGV µ6RPHJXLGHOLQHVIRUFRQGXFWLQJTXDQWLWDWLYH

'HQ]LQ1.DQG<6/LQFROQ HGV E7KH6DJH+DQGERRNRI4XDOLWDWLYH

'XII3µ5HVHDUFKDSSURDFKHVLQDSSOLHGOLQJXLVWLFV¶LQ5%.DSODQ HG 7KH

) UFK&DQG*.DVSHU HGV ,QWURVSHFWLRQLQ6HFRQG/DQJXDJH5HVHDUFK

/D]DUDWRQ$µ4XDQWLWDWLYHUHVHDUFKPHWKRGV¶LQ(+LQNHO HG +DQGERRNRI

/LVNLQ*DVSDUUR-( HG µ)URPWKH$VVRFLDWH(GLWRU0/-5HYLHZV

6FKLHIIOLQ%DQG(2FKV HGV /DQJXDJH6RFLDOL]DWLRQDFURVV&XOWXUHV1HZ

6FKLIIULQ''7DQQHQDQG+(+DPLOWRQ HGV 7KH+DQGERRNRI

7DVKDNNRUL$DQG&7HGGOLH HGV D+DQGERRNRI0L[HG0HWKRGVLQ6RFLDO

YDQ/LHU/µ&DVHVWXG\¶LQ(+LQNHO HG +DQGERRNRI5HVHDUFKLQ6HFRQG