!"#$"%& ("$)* (&#+&,*")%- ). /#0"%"%1 2..

&+*"3&%&--

8arbara no|der, hD
lllghL ueck ConcepL CenLer
8oelng Commerclal Alrplanes
barbara.e.holder[boelng.com

Introduct|on
ln collaboraLlon wlLh Lhe lnLernaLlonal Alr 1ransporL AssoclaLlon (lA1A) and Lhe lnLernaLlonal
lederaLlon of Alr Llne lloLs AssoclaLlons (llALA), 8oelng surveyed Lhe professlonal plloL communlLy
for Lhelr perspecLlves on plloL Lralnlng and Lhe appllcaLlon of Lhose knowledge and skllls presenLed ln
Lralnlng Lo operaLlonal conLexLs. 1he resulLs lndlcaLe LhaL lmprovemenLs are needed ln Lhe areas of
lnsLrucLlon, conLenL, and dellvery meLhods.
We conclude Lralnlng could be lmproved Lo prepare plloLs for Lhelr acLual work by dellverlng conLenL
LhaL ls relevanL Lo dally fllghL operaLlons. 1ralnlng dellvery mechanlsms could be modernlzed,
lnsLrucLlon could be lmproved Lhrough lnsLrucLor quallflcaLlon, sLandardlzaLlon, and callbraLlon. lor
Lralnlng change Lo be successful and susLalnable all lnLeracLlng dlmenslons of lnsLrucLors, conLenL,
meLhods, and alrllne culLure musL be addressed.

Method
1he survey explored plloL percepLlons of currenL Lralnlng and Lhe effecLlveness of Lhelr appllcaLlon Lo
Lhe operaLlonal conLexL of alrllne flylng. lL was lnLended Lo ldenLlfy areas where Lralnlng may be lacklng
Lo creaLe LargeLed lnLervenLlons or Lo ldenLlfy follow-on research acLlvlLles. 8oelng made Lhe survey
avallable Lo alrllne plloLs Lhrough a llnk on Lhe lnLernaLlonal lederaLlon of Alr Llne lloLs AssoclaLlon
(llALA) webslLe. lA1A member alrllnes were noLlfled of Lhe survey vla emall. All responses were
anonymous.
1he resulLs wlll be added Lo Lhe lnLernaLlonal Alr 1ransporL AssoclaLlon (lA1A) daLa corpus, whlch
lncludes daLa from Llne CrlenLed SafeLy AudlL (LCSA) reporLs, global accldenL and lncldenL daLa, and
oLher surveys. 8ecause Lhls survey was conducLed Lo supplemenL Lhe lA1A Lvldence 8ased 1ralnlng
(L81) lnlLlaLlve Lhe probe Loplcs were deflned by Lhe L81 daLa Leam. 1here were areas where currenL
daLa was needed on speclflc Loplcs or where Lhere were gaps ln Lhe daLa corpus.


A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


||ot Demograph|cs
nlne hundred and slxLy-slx plloLs compleLed Lhe survey: flfLy-slx percenL capLalns and forLy-four
percenL flrsL offlcers. llgure 1 shows Lhe dlsLrlbuLlon of respondenLs wlLh ma[orlLles based ln Lurope,
norLh Amerlca, and Cceanla. We aLLrlbuLe Lhe hlgher response raLes ln Lhese reglons Lo Lhe hlgher
llALA represenLaLlon across Lhese reglons. CLher reglons represenLed were Mlddle LasL, Asla, CenLral
and SouLh Amerlca, Afrlca, and Lhe CommonwealLh of lndependenL SLaLes (ClS). 1he lowesL response
raLe came from reglons LhaL also have Lhe hlghesL reglonal safeLy rlsk.
I|gure 1. G|oba| D|str|but|on of kespondents


I|gure 2. ||ot 1ra|n|ng De||very and Most kecent|y Comp|eted 1ra|n|ng
MosL plloLs (94) are Lralned by Lhelr alrllne so lnsLlLuLlng change ln Lralnlng pracLlces wlll requlre
moLlvaLlng alrllnes Lo lnvesL ln change and Lhelr regulaLors Lo approve change (llgure 2). 1he mosL
recenLly compleLed Lralnlng for our respondenLs was recurrenL Lralnlng (84), Lherefore Lhe responses
glven are llkely Lo be framed ln Lhe conLexL of Lhelr lasL recurrenL Lralnlng experlence.
94
6
Alrllne
LxLernal
1ralnlng
86
3
3 4
8ecurrenL
1ransluon
lnlual
Lurope
norLh Amerlca
Cceanla
Mlddle LasL
Asla
CenLral/S. Amerlca
Afrlca
ClS

49.3
21.4
14.6
7.0
3.9
2.2
1.2
0.2

Alrllne lloL ercepLlons of 1ralnlng LffecLlveness




I|gure 3. D|str|but|on of A|rcraft, 8ase Locat|on, and ||ot kankCapta|ns are represented |n b|ue, f|rst
off|cers are represented |n red, and a|rcraft are |dent|f|ed by type.



A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


Automat|on
Learnlng Lo use Lhe fllghL managemenL auLomaLlon ln modern alrplanes conLlnues Lo be a challenge for
many plloLs. 1ralnlng should enable plloLs Lo develop a funcLlonal undersLandlng of Lhe sysLem as well
as operaLlonal undersLandlng of how Lo use Lhe sysLem across operaLlonal slLuaLlons.
lloLs were asked: ln Lhe flrsL 6 monLhs of flylng Lhelr currenL alrcrafL Lype, dld you encounLer a
slLuaLlon where you had dlfflculLy performlng parLlcular Lasks uslng Lhe fllghL managemenL sysLem
(lMS)? 1hls quesLlon was framed ln Lhe lasL 6 monLhs so LhaL we could geL a recenL sample of evenLs
and lssues encounLered and mosL plloLs (64) responded Lhey had dlfflculLy performlng Lasks wlLh Lhe
lMS (llgure 4).

I|gure 4. I|rst 6 Months on Current A|rcraft: D|ff|cu|ty erform|ng 1asks Us|ng IMS
nexL, we asked for an assessmenL of Lhelr comforL level ln operaLlng Lhe lMS afLer Lhe compleLlon of
Lhe Lype-raLlng course. ComforL ls a Lerm plloLs frequenLly use Lo descrlbe confldence ln Lhelr ablllLy Lo
perform. 1he quesLlon was framed ln Lerms of Llme lncremenLs followlng Lhe Lype course Lo ldenLlfy
Lhe Llme by when comforL ls acqulred. 8espondenLs could choose one of Lhe followlng caLegorles: on
your flrsL alrcrafL fllghL, afLer lnlLlal operaLlng evaluaLlon (lCL), afLer 3 monLhs of operaLlon, afLer 6
monLhs of operaLlon, and afLer 12 or more monLhs of operaLlon (llgure 3).
MosL plloL (62) felL comforLable operaLlng Lhe lMS only !"#$% galnlng llne experlence. A few (13)
were comforLable afLer Lhelr lnlLlal operaLlng experlence (lCL). CLhers (41) reporLed comforL afLer
Lhree monLhs of llne operaLlons, afLer slx monLhs (13) and afLer Lwelve monLhs (7).
lf Lhe Lype-raLlng course dld ln facL prepare plloLs for llne operaLlons, we would expecL Lhelr reporLed
comforL level Lo be hlghesL lmmedlaLely afLer compleLlon of Lralnlng. lL appears some Lralnlng
programs do lnsLlll plloL confldence on Lhelr flrsL alrcrafL fllghL afLer Lralnlng slnce a quarLer (23)
reporLed belng comforLable operaLlng Lhe lMS on Lhelr flrsL fllghL.
36
14
23
3 A few umes
Cnce
never
lrequenLly
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness



I|gure S. When ||ots Ie|t Comfortab|e Cperat|ng IMS After 1ype-kat|ng Course
1hese resulLs ralse some lnLeresLlng quesLlons abouL whaL ls belng learned afLer lCL LhaL enables Lhe
feellng of comforL LhaL could be broughL lnLo Lralnlng earller. We also need Lo know whaL speclflcally
consLlLuLes effecLlve learnlng on Lhe llne.
lloLs ofLen reporL LhaL Lhe learnlng of Lhe fllghL managemenL sysLem (lMS) occurs over Llme. We
deslgned Lhe nexL quesLlon Lo ldenLlfy how lMS learnlng ls dlsLrlbuLed. 8espondenLs esLlmaLed Lhe
percenLage of learnlng Lhey acqulred beLween Lralnlng, llne operaLlons, and self-sLudy. 1he resulLs
showed Lhe followlng dlsLrlbuLlon:
lMS learnlng on Lhe llne-42.
lMS learnlng from Lralnlng-38.
lMS learnlng Lhrough self-sLudy-20.
lf lL ls Lhe case LhaL only LhlrLy-elghL percenL of learnlng occurs ln Lralnlng Lhen we are falllng our plloL
communlLy by unnecessarlly forclng learnlng Lhls lmporLanL sysLem Lhrough oLher means LhaL may or
may noL be effecLlve. We need Lo ldenLlfy whaL conLenL ls needed ln Lralnlng Lo address Lhls lssue and
deflne effecLlve dellvery meLhods LhaL enable hlgher reLenLlon and undersLandlng.
Llne operaLlons may be Lhe besL conLexL for Lhe lnLegraLlon of skllls and knowledge across operaLlonal
conLexLs buL we need Lo ensure LhaL alrllnes are equlpped wlLh Lhe Lools and guldance needed Lo
enable effecLlve llne learnlng.
1he nexL quesLlon lnqulred abouL areas of auLomaLlon Lralnlng LhaL could be lmproved and
respondenLs could check up Lo Lhree opLlons (llgure 6). CperaLlonal slLuaLlons such as auLomaLlon
surprlses (37), hands-on use ln operaLlonal slLuaLlons (32), and LranslLlons beLween modes (32),
recelved Lhe hlghesL response raLes. lloL Lralnlng needs Lo lnclude funcLlonal operaLlon of sysLems buL
clearly operaLlonal slLuaLlons need Lo be lnLroduced lnLo Lralnlng Lo expose plloLs Lo uslng Lhe sysLem
ln Lhe conLexL of fllghL operaLlons.
23
13
41
13
6
llrsL lllghL
Aer lCL
Aer 3 monLhs
Aer 6 monLhs
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


lloLs also clLled baslc knowledge of Lhe sysLem and programmlng as areas for lmprovemenL whlch ls
surprlslng because Lhese areas Lend Lo be emphaslzed ln recurrenL and Lype-raLlng courses and
lndlcaLe funcLlonal Lralnlng of Lhe sysLem could be lmproved.
I|gure 6. otent|a| Areas of Improvement for Automat|on 1ra|n|ng

Go-Around Maneuvers
1he lndusLry currenLly regards go-around maneuvers as a safeLy lssue because Lhey are elLher poorly
execuLed or noL execuLed when Lhey should be. 1he nexL seL of quesLlons probe Lhe raLlonale
underlylng Lhe go around declslon Lo conLlnue Lo landlng when a go-around should have been made.
1he flrsL quesLlon lnqulres abouL Lhe Leamwork componenL of Lhe declslon. We asked, uld you
encounLer slLuaLlons where Lhere should have been a go-around buL Lhe approach was conLlnued Lo a
landlng?" lf Lhey answered yes, Lhey were presenLed wlLh Lhree opLlons:
a. l suggesLed a go-around, buL Lhe oLher plloL dlsagreed (20).
b. 1he oLher plloL suggesLed a go-around, buL l dlsagreed (8).
c. nelLher plloL suggesLed a go-around (72).
1he ma[orlLy of Lhe reporLed cases, nelLher plloL suggesLed a go-around and ln Lhe remalnlng cases Lhe
plloLs dld noL agree Lo go-around. lloLs were permlLLed Lo reporL up Lo flve go-around cases and ln all
cases, Lhe maln resulL was: nelLher plloL suggesLed a go-around. We asked Lhe plloLs Lo reporL Lhelr
rank (capLaln, flrsL offlcer) and role (plloL flylng, plloL monlLorlng, and augmenLed crew).
ln Lhe cases when one plloL suggesLed a go-around and Lhe oLher plloL dlsagreed, we correlaLed Lhelr
rank Lo compllance wlLh Lhe suggesLlon of a go-around (1able 1). 1hese resulLs ralse concerns
regardlng Lhe effecLlveness of Lralnlng Leam declslon maklng and effecLlve communlcaLlon because we
do see Lhe lnfluence of rank enLerlng Lhe declslon maklng process.
332
302
313
238
203
132
64
0
100
200
300
400
300
600
1 2 3 4 S 6 7
!"#$%&#'$( *"+,+'*-*
!"#$% '# (%) *# '+),"-*'#". %*-("-*'#%
!"#$%&'&($% *+',++$ -(.+%
!"#$% '()*+,-., )/ 01, #2#0,3
!!"#!$%%&'#
!"#$%&'() +&,-.&/-%+) 0"+123
!"#$%
1. !"#$%
2. !"#
3. !"#$%
4. !"#$%
3. !"#
6. !"#"$
7. !"!#
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness


1ab|e 1. D|str|but|on of kesponses by kank
kesponse Categor|es Capta|n I|rst Cff|cer
l suggesLed a go-around, buL Lhe oLher plloL dlsagreed 13.8 27.6
1he oLher plloL suggesLed a go-around, buL l dlsagreed 12.3 2.8
nelLher plloL suggesLed a go-around 73.9 69.7

AlLhough a plloL may feel he can suggesL a go-around or even demand one from Lhe plloL flylng, Lhe
oLher plloL may noL comply. lor Lhose cases where nelLher plloL suggesLed a go-around, lL may be LhaL
plloLs lack famlllarlLy wlLh Lhe go-around crlLerla or Lhe sklll Lo recognlze Lhe need ln Llme Lo make Lhe
declslon across operaLlonal conLexLs. nelLher plloL suggesLlng a go-around may be due Lo Lhe plloLs'
ablllLy Lo make Lhe approach work and apply [udgmenL Lo malnLaln safeLy.
1he nexL quesLlon lnqulres abouL how asserLlve a plloL feels he can be whlle ln Lhe role of plloL
monlLorlng across dlfferenL conLexLs. We asked, When you are Lhe lloL MonlLorlng, you feel you may
wlLhouL heslLaLlon..." lloLs were asked Lo lndlcaLe Lhelr agreemenL wlLh each of Lhe conLexL caLegorles
and Lhe percenLages ln 1able 2 represenL Lhe dlsLrlbuLlon of agreemenL.
lloLs reporLed hlgh levels of asserLlveness ln four of Lhe flve caLegorles, wlLh Laklng conLrol from Lhe
plloL flylng reglsLerlng Lhe lowesL aL forLy-nlne percenL. 1he level of reporLed asserLlveness appears Lo
be llnked Lo Lhe level of resulLlng lnLervenLlon. 1asks such as ldenLlfylng a devlaLlon (92) or proposlng
a checkllsL (91) are reporLedly more llkely Lo be asserLed Lhan Lasks such as proposlng a go-around
(83) or demandlng a go-around (80). Cne could argue devlaLlons and checkllsLs are hlgher Lo asserL
because Lhey are rouLlne and do noL so much challenge Lhe sklll or [udgmenL of Lhe plloL flylng.

1ab|e 2. D|str|but|on of kesponses to Assert|veness
kesponse Categor|es D|str|but|on
1ell Lhe plloL flylng abouL a devlaLlon 92
1ake conLrol from Lhe plloL flylng 49
ropose a checkllsL lf Lhe plloL flylng delays asklng for lL 91
ropose a go-around durlng an unsLable approach 83
verbally demand a go-around lf you Lhlnk lL ls requlred 80

A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


We learned from Lhe prevlous quesLlon LhaL plloLs do asserL Lhemselves ln Lhe go around maneuver
however aL a much lower raLe (28) Lhan Lhe case where nelLher plloL suggesLed a go-around (72).
lL ls posslble LhaL a lack of asserLlveness ls Lhe underlylng reason why, ln Lhe ma[orlLy of Lhe cases,
nelLher plloL suggesLed a go-around and LhaL Lhere ls an underlylng heslLaLlon Lo asserL oneself as Lhe
conLexL shlfLs Lo a conLrol or [udgmenL assessmenL. Palf Lhe plloLs reporLed Lhey would noL heslLaLe Lo
Lake conLrol from Lhe plloL flylng yeL aL whaL polnL does lL become accepLable Lo Lake conLrol? Pow
should Lhls sklll be Lralned and assessed? 1aklng conLrol away from Lhe plloL flylng perhaps crosses Lhe
boundarles of [udgmenL wlLh regard Lo one's parLner and oneself. A plloL wlll need Lo be very confldenL
ln hls [udgmenL of Lhe need Lo Lake conLrol. Pow besL Lo Lraln and assess Lhese behavlors on a global
scale needs furLher lnvesLlgaLlon.
1he Llne CperaLlons SafeLy AudlL (LCSA) reporLs daLabase suggesLs 97 of unsLable approaches are
conLlnued Lo landlng. We asked Lhe respondenLs Lo make a [udgmenL abouL why anoLher plloL would
noL lnlLlaLe a go-around Lo probe for posslble raLlonales for noL dolng Lhe maneuver. We asked, ln
your oplnlon whaL are Lhe reasons for noL lnlLlaLlng a Co-Around?" 1hey were presenLed wlLh Lhe slx
followlng opLlons and could choose up Lo Lhree:
a. Accordlng Lo Lhe [udgmenL by Lhe plloL, Lhe landlng can be performed safely
b. 1here ls a blg psychologlcal barrler Lo go around because Lhey are so rare evenLs
c. CperaLlonal lnconvenlence
d. LmbarrassmenL relaLed Lo a go around
e. lloLs are noL as famlllar wlLh unsLable approach crlLerla as Lhey should be
f. Maklng a go-around mandaLes a reporL
lloL [udgmenL was mosL clLed (82) as Lhe reason a plloL would choose noL Lo go around lf Lhe
approach was unsLable (1able 3). 1hls response ls cerLalnly reasonable. Cne of Lhe prlmary roles of
plloLs ls Lo apply [udgmenL and lnLervenLlons ln Lhe momenL-Lo-momenL conLexL of acLlvlLy. Powever,
lL ls our assessmenL LhaL mosL Lralnlng programs Lraln [udgmenL lmpllclLly and lf plloLs are golng Lo be
relylng on [udgmenL we should make sure lL ls expllclLly Lralned Lo effecLlvely LranslLlon Lo Lhe
operaLlonal conLexL.
1he nexL Lwo ma[or caLegory responses were psychologlcal barrlers (37) and operaLlonal
lnconvenlence (33). sychologlcal barrlers may be percelved by plloLs do Lhe maneuver lnfrequenLly
ln operaLlons and ln Lralnlng. 8y provldlng opporLunlLles Lo pracLlce Lhe maneuver across conLexLs
(such as all englne go around) ls lmporLanL Lo bulldlng a plloL's confldence ln hls skllls. CperaLlonal
lnconvenlence could be a safeLy concern lf plloLs are chooslng Lo noL go around for Lhe wrong reasons.



Alrllne lloL ercepLlons of 1ralnlng LffecLlveness


1ab|e 3. keasons for Not Choos|ng Go-Around
kesponse Categor|es D|str|but|on
lloL [udgmenL 82
sychologlcal barrler 37
CperaLlonal lnconvenlence 33
LmbarrassmenL 24
unfamlllar wlLh crlLerla 17
MandaLes a reporL 10


Mon|tor|ng and Cross-Check|ng
1he nexL seL of quesLlons was deslgned Lo lnqulre abouL Lhe pervaslveness of error managemenL ln
fllghL Lralnlng and Lhe percelved value as a sklll. MonlLorlng and cross-checklng, Lwo key componenLs
of error managemenL, are percelved as lmporLanL plloLlng skllls (llgure 7). lorLy-seven percenL of Lhe
plloLs reporLed Lhe Loplc of deLecLlng and managlng errors are lncluded ln Lhelr recurrenL Lralnlng as a
speclflc Loplc ln boLh Lheory and pracLlce (llgure 8). Powever Lhe remalnlng respondenLs reporLed Lhe
Loplc as lmpllclLly covered, marglnally covered, or noL covered aL all.

llgure 7. MonlLorlng and Cross-Checklng

AlLhough a ma[orlLy of plloLs belleve Lhese are lmporLanL skllls, Lhe Lralnlng of Lhese Lasks ls noL as
wlde-spread as prevlously LhoughL and ls evldence LhaL guldance for Lralnlng monlLorlng and cross-
67.20
32.10
0.20
0.30
now |mportant a sk||| |s mon|tor|ng and cross-
check|ng?
MosL lmporLanL
lmporLanL
noL so lmporLanL
LeasL lmporLanL
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


checklng skllls ls needed. 1he plloL monlLorlng role ls one of Lhe mosL lmporLanL for malnLalnlng hlgh
levels of safeLy and operaLlonal efflclency and should be Lralned expllclLly on a global scale.


I|gure 8. Inc|us|on of Lrror Management |n kecurrent 1ra|n|ng
1he LCSA reporLs ldenLlfled Lhe '()*+ ,-!.$ /" "()0-# as one wlLh Lhe hlghesL raLe of poor monlLorlng
performance. We asked why Lhls mlghL be Lhe case and Lhe respondenLs reporLed Lhe maln causes of
degradaLlon ln monlLorlng durlng Lhe cllmb Lo be complacency and secondary Lask loadlng (llgure 9).
Complacency may be lnduced by Lhe LranslLlon from a hlgh workload fllghL phase Lo lower workload


I|gure 9. Mon|tor|ng and Cross-Check|ng Dur|ng C||mb
47
34
13
4
Is the top|c of detecnng and manag|ng errors
|nc|uded |n your recurrent tra|n|ng?
?es as a speclc
Loplc
Covered buL noL
expllclLly
Marglnally
covered
137
330
116
348
kesearch |nd|cates mon|tor|ng and cross-check|ng |s poorest
dur|ng the c||mb phase
lloLs have Loo many secondary duues SCs are generally Loo weak ln monlLorlng
Complacency aer Lakeo CLher
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness


fllghL phase. MonlLorlng Lasks are ofLen dropped for compeLlng secondary Lask demands. ln Lralnlng,
monlLorlng should be emphaslzed as one of Lhe mosL lmporLanL prlmary Lasks and plloLs should be
LaughL how Lo monlLor and when. We should also glve plloLs sLraLegles for managlng Lhelr workload ln
all fllghL phases so LhaL monlLorlng ls noL dropped lnapproprlaLely.
MosL plloLs (93) belleve deLecLlng and managlng errors ls Lhe mosL effecLlve sLraLegy for error
managemenL (llgure 10). A small percenLage of plloLs (7) belleve LhaL errors should noL be
commlLLed.

I|gure 10. Strategy for Lrror Management

8r|ef|ngs
8rleflngs presenL an lmporLanL opporLunlLy for plloLs Lo consLrucL a Leam concepL and bulld shared
undersLandlngs abouL whaL Lo expecL, each oLher's roles, and conLlngency plans. lL ls lmporLanL LhaL
brleflngs be lncluded ln Lralnlng so plloLs can pracLlce Lhese skllls and recelve feedback on Lhelr
conLenL, duraLlon, and effecLlveness.
Approach brleflngs are lncluded ln Lralnlng (llgure 11) however Lhere were a number of respondenLs
provlded commenLs clLlng LhaL approprlaLe brleflng conLenL ls generally noL known or pracLlced. lL ls a
poslLlve flndlng LhaL plloLs geL an opporLunlLy ln Lhe Lralnlng envlronmenL Lo pracLlce brleflngs and
provldlng guldance on Lhelr conducL and conLenL would be a poslLlve sLep Loward lmprovlng Lhelr
effecLlveness ln operaLlons.
8rleflngs prlor Lo Lhe slmulaLlon sesslons are regularly lncluded ln Lralnlng and presenL a poLenLlally
valuable opporLunlLy for focused lnsLrucLlon (llgure 12). 1hese sesslons Lend Lo be longer Lhan Lhe
debrleflng sesslons by 20-30 mlnuLes (llgure 13). 8ecause debrlef sesslons are vulnerable Lo dlsmlssal
due Lo Llme consLralnLs or laLe nlghL sesslons, care should be Laken Lo make effecLlve use of Lhe
debrlef. AL mlnlmum, lnsLrucLors should use Lhe debrlef sesslons as an opporLunlLy for Lhe Lralnees Lo
7
89
4
In your op|n|on what |s the most eecnve strategy
concern|ng errors |n the |ght deck?
noL Lo commlL errors
ueLecL and manage
errors eecuvely
CLher
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


revlew and reflecL on Lhelr performance. lnsLrucLors have a cruclal role ln maklng effecLlve use of
brleflngs and ensurlng LhaL all approprlaLe feedback (poslLlve and negaLlve) ls glven.

I|gure 11. Approach 8r|ef|ng Irequency |n 1ra|n|ng

I|gure 12. 8r|ef|ng Durat|on 8efore S|mu|ator Sess|on

I|gure 13. Debr|ef|ng Durat|on After S|mu|ator Sess|on
0.10 3.20
10.60
16.70
67.40
Dur|ng tra|n|ng how ohen do you get an
opportun|ty to perform an approach
br|ehng?
never
Less Lhan half
ApproxlmaLely half
More Lhan half
Always
0
1
6
14
39
40
now |ong was the br|ehng before your
s|mu|ator sess|on dur|ng your |atest
tra|n|ng?
no brlef
very shorL
10-20 mln
20-40 mln
0 6
26
43
20
3
now |ong was the debr|ehng aher the
s|mu|ator sess|on dur|ng your |atest
tra|n|ng?
no brlef
very shorL
10-20 mln
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness


Intent|ona| Dev|at|ons
arL of plloL [udgmenL and experLlse lnvolves knowlng when Lo devlaLe from SLandard CperaLlng
rocedures (SC). We were lnLeresLed ln Lhe frequency and condlLlons under whlch plloLs mlghL
devlaLe from Lhelr company's SCs.

I|gure 14. Irequency of ||ot Dev|at|on Irom SCs
llgure 14 shows LhaL a ma[orlLy of Lhe respondenLs (33) would devlaLe lf Lhey belleve lL lncreases
safeLy and LwenLy-nlne percenL would devlaLe lf lL resulLed ln no reducLlon ln safeLy. Cverall, mosL
(83) plloLs would exerclse [udgmenL Lo lnLenLlonally devlaLe from company SCs wlLh Lhelr [udgmenL
belng Lhe plloL's assessmenL of safeLy. AnoLher seven percenL reporLed Lhey would never devlaLe. ln
Lhe nexL serles of quesLlons, we asked plloLs Lo ldenLlfy Lhe speclflc lnLenLlonal devlaLlons Lhey have
experlenced on Lhe fllghL deck.


I|gure 1S. Irequency of Intent|ona| Dev|at|on Irom Stab|e Approach Cr|ter|a
7
29
33
11
Would never
devlaLe
no reducuon
ln safeLy
lf lL lncreases
safeLy
40.40
38.40
7.70
0.30
vlrLually every lghL AbouL every 10 lghLs
A few umes a year Cnce a year or less
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


lnLenLlonal devlaLlons from sLable approach crlLerla were reporLed Lo occur aL a raLe of once per year
by 40 of Lhe respondenLs and more Lhan a few Llmes a year by 38 of Lhe respondenLs (llgure 13).
Powever, some plloLs reporL lnLenLlonal devlaLlons from sLable approach aL a hlgher raLe of every Len
fllghLs, or vlrLually every fllghL. lurLher lnqulry lnLo sLable approach devlaLlons should ldenLlfy Lhe
conLexLs ln whlch Lhese [udgmenLs are made and why Lhey are made. lL would seem Lhese raLes are
lndlcaLlve of confllcL beLween Lhe crlLerla and Lhe reallLles of Lhe operaLlonal conLexL.
lnLenLlonal devlaLlons from checkllsLs occurred a reporLed every Len fllghLs by 13 of Lhe respondenLs,
a few Llmes a year by 30 of Lhe respondenLs, and once a year by 36 of Lhe respondenLs. very few
(4) reporLed a devlaLlon on every fllghL. CheckllsL devlaLlons occurrlng aL Lhls hlgh of a raLe suggesL
oLher facLors may be lnvolved noL relaLed Lo compllance. erhaps Lhe plloL does noL know Lhe
procedure or pollcy, or does noL undersLand lL and several plloLs commenLed Lo us Lhey would llke Lo
know Lhe underlylng raLlonale of Lhe procedure. lurLher Lhe procedure may noL make operaLlonal
sense Lo Lhe plloL, lL may noL flL lnLo Lhe operaLlonal conLexL where lL ls Lo be applled, or Lhe procedure

I|gure 16. Irequency of Intent|ona| Dev|at|on Irom Check||st
may be lnLerrupLed by compeLlng demands on aLLenLlon-all of whlch may resulL ln noncompllance.
llnally poorly deslgned procedures may lmpose excesslve cognlLlve workload, Lhereby maklng Lhem
dlfflculL Lo perform correcLly.
CallouLs had a hlgh lnLenLlonal devlaLlon raLe wlLh abouL half Lhe respondenLs (49) reporLlng
devlaLlons on every 10 fllghLs and vlrLually every fllghL (llgure 17). 1here are several posslble reasons
why non-compllance ls hlgh, mosL agaln noL necessarlly relaLed Lo compllance. CallouLs serve an
lmporLanL purpose of esLabllshlng shared undersLandlngs and represenLaLlons of Lhe slLuaLlon. lf plloLs
do noL undersLand Lhe purpose of Lhe callouL or lf Lhe callouL does noL fulflll Lhe purpose by deslgn
Lhen we would expecL plloLs Lo noL use Lhem. 1he shear number of callouLs Lo remember may be a
reason for noL maklng Lhem, plloLs may slmply forgeL Lo make Lhem ln Lhe conLexL of a demandlng
slLuaLlon or a lapse ln monlLorlng, or Lhe plloLs may noL feel Lhey are lmporLanL. lf we are Lo
undersLand lnLenLlonal devlaLlons from callouLs, we wlll need Lo lnvesLlgaLe Lhe speclflc callouLs
devlaLed from and Lhe conLexLs of Lhelr occurrence and provlde guldance on approprlaLe Lralnlng of
callouL use.
4.20
13.40
30
36.9
Cnce a year lew umes a year
Lvery 10 lghLs vlrLually every lghL
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness



I|gure 17. Irequency of Intent|ona| Dev|at|on Irom Ca||outs
Cperat|ona| S|tuat|ons
lL ls lmporLanL LhaL Lhe knowledge and sklll acqulred ln Lralnlng Lransfer Lo operaLlons. We Lrled Lo
ldenLlfy areas where knowledge and sklll Lransfer may break down and Lo ldenLlfy gaps ln Lralnlng
conLenL. We asked, ln Lhe lasL slx monLhs, dld you encounLer an operaLlonal slLuaLlon where you dld
noL feel comforLable?" !usL over half (34) of Lhe respondenLs answered yes (llgure 18). WlLhln LhaL
caLegory, 37 of Lhe reporLlng plloLs were ranked capLaln and 43 were ranked flrsL offlcer.

I|gure 18. Lxper|enced Uncomfortab|e Cperat|ona| S|tuat|ons Dur|ng Last S|x Months
lf Lhey answered yes, we Lhen asked Lhe plloLs Lo speclfy whaL klnd of Lralnlng mlghL have helped ln Lhe
slLuaLlon and Lo selecL all areas of Lralnlng LhaL would have helped (llgure 19).



7.60
34.80
27.80
20.80
vlrLually every lghL AbouL every 10 lghLs
A few umes a year Cnce a year or less
34
46
?es no
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness



Adverse weaLher (30) and crew resource managemenL (23) were ranked hlghesL for belng helpful ln
deallng wlLh uncomforLable operaLlonal slLuaLlons, followed by Lralnlng ln non-normal checkllsLs (16),
fllghL managemenL (13), alrplane handllng (13), sysLems (12), and maneuvers (10). All of Lhese
caLegorles are addressed ln recurrenL Lralnlng sesslons. 1hese resulLs quesLlon Lhe effecLlveness of Lhe
Lralnlng and lLs Lransfer Lo Lhe operaLlonal conLexLs where Lhey are encounLered.
1he plloLs were Lhen asked Lo descrlbe Lhe slLuaLlon Lhey encounLered (1able 4). 1he responses
lncluded fllghL managemenL speclflc Lo operaLlonal Lasks, such as a laLe runway change or rerouLe,
knowledge lssues relaLed Lo auLo fllghL mode undersLandlng, and procedural lssues assoclaLed wlLh Lhe
lnLroducLlon of new procedures or changes drlven by mergers LhaL resulLed ln poor procedure
lnLegraLlon. lnfrequenL non-normal evenLs such as low fuel, blrd sLrlke, Cuu fallure, upseL recovery,
and volcanlc ash were also menLloned. Adverse weaLher responses speclfled cold weaLher operaLlons,
de-lclng, conLamlnaLed runway operaLlons and hlgh alLlLude Lurbulence. Also clLed were non-preclslon
and vlsual approaches, energy managemenL ln Lhe approach, severe crosswlnds, go-around and mlssed
approaches, and alrcrafL handllng and maneuvers, parLlcularly ln reglons of mounLalnous Lerraln.
erformance calculaLlons, dlverslon, mlnlmum equlpmenL llsL (MLL) lLems, sysLems knowledge, and
confllcL managemenL wlLh a crewmember or a passenger were clLed.


I|gure 19. 1ra|n|ng Ident|f|ed by ||ots to Dea| W|th Uncomfortab|e Cperat|ona| S|tuat|ons




!"#$%&$ ($)*+$% !"#$%
!"#$ "#&'(")# *+,+-#*#,. !!"#$
!"#-!"#$%& ()*(+&,-.- !"#"$
!"#$%& ()*)$+(+*& !"#$%
!"#$%&'( *&'+%"', !"#! %
!"#$%&# !!"#$
!"#$%&$'( !"#$%
!"#$% !"#$
!"#$%& (#")*+,#*-. !"#!$%&'( !"#$
286
217
130 149
127
112
100
94
46
0
30
100
130
200
230
300
330
1 2 3 4 S 6 7 8 9
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness


1ab|e 4. Uncomfortab|e Cperat|ona| S|tuat|ons Descr|bed by ||ots
8unway closure aL desLlnaLlon prompLlng holdlng and posslble dlverL ln busy Luropean alrspace
vlsual clrcle Lo land ln LW8 8wy 29 due Lo masslve crosswlnd
ln everyday A1C requlremenLs of speed and lasL mlnuLe changes, Lhere ls no Lralnlng glven
Whlle flylng aL lL400, encounLered sLlck shaker ln Lurbulence due Lo momenLary severe updrafL
1allwlnd approach over sLeep Lerraln slmulLaneously lnLercepLlng locallzer and gllde slope
AL 37,000 feeL, escape maneuver for wake Lurbulence from heavy alrcrafL (747)
rocedures and Lerraln unlque Lo forelgn alrporLs
lannlng/performance done manually on conLamlnaLed runways wlLh MLL lLems
WlnLer operaLlons wlLh conLamlnaLed runway and relaLed declslon maklng wlLh regard Lo Lakeoff and landlng
performance
u Lurns on Lhe runway.

Negat|ve Lxper|ences |n 1ra|n|ng
A poslLlve soclal conLexL for Lralnlng ls a key componenL of Lralnlng effecLlveness. We asked a serles of
quesLlons Lo probe for any negaLlve experlences plloLs may face ln Lralnlng. 1he lnsLrucLor-Lralnee
relaLlonshlp was a known area of concern. We asked plloLs Lo lndlcaLe lf Lhelr lnsLrucLor had ralsed
Lhelr confldence durlng Lhelr lasL Lralnlng sesslon (llgure 20). unforLunaLely, 43 of Lhe responses
were negaLlve.

I|gure 20. Instructor Lffect on ||ot's Conf|dence |n rof|c|ency

We Lhen asked plloLs lf any negaLlve experlences were encounLered ln Lralnlng wlLhln Lhe pasL 3 years
(llgure 21). 1he broad Llme range was Lo ensure we capLured all posslble Lralnlng cycles. lorLy-slx
percenL of Lhe plloLs responded yes Lo havlng a negaLlve experlence ln Lralnlng ln Lhe pasL 3 years and
36
40
4
?es lL
lncreased
no change
no lL
decreased
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


we asked Lhe plloLs Lo speclfy Lhe cause of Lhe negaLlve experlence. 8esponses were coded and
grouped by Loplc (1able 3) and 1able 6 provldes speclflc negaLlve Lralnlng slLuaLlons reporLed by plloLs
ln Lralnlng. 1he mosL frequenL source of negaLlve experlences ln Lralnlng was Lhe lnsLrucLor. 1he oLher
Lwo caLegorles were course conLenL and dellvery.


I|gure 21. ||ots nav|ng Negat|ve 1ra|n|ng Lxper|ences |n Last S ears

1he resulLs polnL Lo some areas Lo LargeL for lmmedlaLe lmprovemenL ln Lralnlng. lnsLrucLlon, conLenL,
and dellvery represenL Lhe maln concerns. rovldlng comprehenslve guldance for lnsLrucLor
quallflcaLlon, callbraLlon, and sLandardlzaLlon should be a Lop prlorlLy. 1he Lralnlng envlronmenL should
faclllLaLe learnlng and promoLe Lhe free exchange of ldeas, quesLlons, and dlscusslons. 1he conLenL and
lLs dellvery musL be operaLlonally relevanL and presenLed ln a way LhaL lncreases reLenLlon so
knowledge and skllls may be Lransferred.
1ab|e S. Negat|ve Lxper|ences' Codes and Irequency
Irequency Codes for Cpen Lntry Comments
118 lnsLrucLor lnLlmldaLlon
31 lnsLrucLor knowledge deflclency
40 lnsLrucLor sLandardlzaLlon
40 lnapproprlaLe assessmenL
36 unreallsLlc scenarlos or Lask loadlng by lnsLrucLor
36 SCs vlolaLed by lnsLrucLor for scenarlo
36 oor syllabus conLenL
46
34
osluve
negauve
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness


33 1lme compresslon
34 ulsagreemenL wlLh lnsLrucLor
34 locus on checklng
21 lnapproprlaLe Lralnlng meLhod
14 lnapproprlaLe palrlng
12 no opporLunlLy Lo pracLlce
11 SlmulaLor lnaccuracy
4 oor Lralnlng manuals
3 oor brlef prlor Lo slmulaLor

1ab|e 6. Some keported Negat|ve S|tuat|ons
l had a lnsLrucLor LhaL loved Lo "play wlLh Lhe fllghL slmulaLor and l had sesslons wlLh 8 mulLlple faulLs aL Lhe same
Llme, flre, fuel leak, generaLors' faulLs, door opens... lL wasn'L Lralnlng was more llke a massacre.
1ralnlng ls Loo geared up Lo meeLlng LC and CC requlremenLs and so we Lend Lo leave llLLle Llme for Lhe unusual
slLuaLlons LhaL can arlse. Lxample ls englne fallure aL v1 rarely aL v2.
lour-hour recurrenL sesslon wlLh Loo many emergencles. CognlLlve overload aL Lhe end wlLh llLLle learnlng.
1here are Llmes you wlll ask a quesLlon and all lL does ls puL a LargeL on your back.
Cowboy lnsLrucLor very nonsLandard devlaLlon from tco.
Check plloLs who aren'L famlllar w/ Lhe "real world."
1oo much conLenL Lo cover ln Lhe avallable Llme leadlng Lo noLhlng belng covered adequaLely.
lnsLrucLor noL undersLandlng prlorlLles and unable Lo accepL LhaL he was wrong and Lhe CapL under check was
rlghL.
lnsLrucLors ln my company are noL able Lo Lell a capLaln he ls bad. MosL of Lhe Llme Lhe flrsL offlcers are charged
wlLh every mlsLake.
noL Leachlng, [usL checklng.
nlL-plcky wlLch-hunL aLmosphere on lasL evaluaLlon.
lnsLrucLor who LhoughL he was sLlll ln Lhe mlllLary and felL Lhe need Lo yell. noL very conduclve Lo learnlng.
varlaLlons by check plloLs on procedures.
1ralnlng plloL who would noL dlscuss procedure buL demanded we follow hls procedure.
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


Anyth|ng L|se We Shou|d know
AL Lhe end of Lhe survey, we gave Lhe plloLs an opporLunlLy Lo commenL freely abouL Lhelr Lralnlng
experlences and Lhey provlded deLall on whaL Lhey percelve Lo be key barrlers Lo lmproved Lralnlng.
8egardlng conLenL, Lhey wanL access Lo deflnlLlve Lechnlcal lnformaLlon from Lhe alrplane
manufacLurers. lloLs feel Lhey do noL geL all Lhe lnformaLlon Lhey need vla Lralnlng or Lhrough
bulleLlns and oLher means of communlcaLlon. LxplanaLlon of Lhe raLlonale underlylng Lhe sLandard
operaLlng procedures was frequenLly requesLed, Lxplaln why SCs are wrlLLen LhaL way." Several
wroLe LhaL Lhelr company's SCs are noL compaLlble wlLh Lhe operaLlonal envlronmenL and requlre
adapLaLlon of Lhe SCs Lo make lL work." SysLems Lralnlng and knowledge were reporLed Lo be
gone" from Lralnlng and plloL knowledge and crew resource managemenL Lralnlng was reporLed
lneffecLlve" or absenL." lloLs belleve LhaL fllghL managemenL auLomaLlon ls a cruLch" and hand
flylng should be encouraged.
8egardlng Lralnlng dellvery, plloLs clLed Lhe lssue of belng Llme compressed ln Lralnlng courses LhaL do
noL provlde sufflclenL opporLunlLy Lo asslmllaLe, Lhlnk, and reflecL on whaL Lhey are learnlng. lloLs
belleve Lhe soclal lnLeracLlon of learnlng ln a classroom ls superlor Lo dlsLance learnlng programs and
lneffecLlve" self-sLudy. lloLs suggesLed Lralnlng occur more frequenLly and for a reduced duraLlon Lo
enable malnLalnlng proflclency.
We were dellghLed Lo recelve a few poslLlve commenLs abouL Lralnlng from plloLs reporLlng Lhelr
company Lralnlng ls excellenL" and Lhe besL Lralnlng l have ever had." lloLs expressed Lhelr
appreclaLlon for Lhe opporLunlLy Lo parLlclpaLe ln Lhe survey and were Lhankful for belng able Lo share
Lhelr experlences. lloLs are concerned abouL Lhelr Lralnlng and wanL lmproved Lralnlng for safeLy,
confldence bulldlng, and enhanced performance.

Conc|us|on
lnLroduclng change Lo an exlsLlng Lralnlng program wlll requlre lnvesLmenL on Lhe parL of Lhe alrllnes,
Lhe regulaLors, and Lhe manufacLurers. As an lndusLry, we need Lo flnd a way Lo moLlvaLe operaLor
lnvesLmenLs ln Lralnlng lmprovemenLs and ways Lo moLlvaLe regulaLors Lo approve Lralnlng
enhancemenLs, whlle removlng barrlers Lo change. CurrenL Lralnlng programs focus on fulfllllng
regulaLory requlremenLs sufflclenL Lo meeL a mlnlmum level of proflclency buL as one of Lhe plloLs sald,
asslng does noL equal preparaLlon."
1he ma[orlLy of Lhe survey respondenLs are from reglons where Lhe safeLy record ls hlgh (norLh
Amerlca and Lurope). 8eglons wlLh Lhe lowesL response raLes are Lhe reglons currenLly wlLh Lhe
hlghesL safeLy rlsk. We need Lo work on lmprovlng communlcaLlon and engagemenL ln Lhese reglons
and work wlLh Lhe regulaLors Lo acLlvely ralse Lhe bar of global safeLy by supporLlng changes Lo Lralnlng
so LhaL all operaLors wlll be Lralned Lo a hlgher sLandard.
Alrllne lloL ercepLlons of 1ralnlng LffecLlveness


1he resulLs suggesL Lralnlng ls mulLldlmenslonal and all dlmenslons musL be addressed for
lnLervenLlons Lo be successful and susLalnable. A revlew of lnsLrucLlonal pracLlces, conLenL
compleLeness, and dellvery meLhods represenL a good place Lo sLarL lmprovemenL. lloLs belleve
Lralnlng should prepare Lhem for Lhelr acLual work and equlp Lhem wlLh a Lransferable LoolklL of
resources Lo draw upon ln Lhe conducL of Lhelr work. 1ralnlng conLenL should be operaLlonally relevanL
Lo Lhe speclflc operaLor and scenarlo-drlven Lo expose plloLs Lo slLuaLlons Lhey may face ln Lhelr
operaLlons and Lo bulld Lhelr confldence.
lllghL managemenL Lralnlng ls one of Lhe areas where conLenL and dellvery need careful
reconslderaLlon. 1ralnlng wlll need Lo address Lhe funcLlonal use of Lhe sysLem buL lL also needs Lo
lnLegraLe funcLlonal use wlLh operaLlonal use. ConLlnued llne Lralnlng may be approprlaLe Lo meeL
buslness ob[ecLlves aL Lhe alrllnes, buL lf we are golng Lo have plloLs Lralnlng whlle Lhey fly, we should
deslgn such Lralnlng and assess Lhe Lralnlng Lo ensure lL ls approprlaLe and effecLlve. 1ralnlng of
funcLlonal use could be conducLed ln Lhe conLexL of Lhe operaLlonal demands so LhaL auLomaLlon
surprlses and mode LranslLlon confuslon are subsLanLlally reduced.
Approach and go-around were ldenLlfled as areas where Lralnlng could be lmproved, parLlcularly Lhe
ablllLy Lo recognlze when a go-around ls or ls noL Lhe safesL soluLlon. lloLs need Lralnlng on rlsk
assessmenL, [udgmenL maklng, and funcLlonlng LogeLher as a Leam. ln 82 of Lhe reporLed cases where
plloLs declded noL Lo go-around, Lhey belleved Lhere would be no reducLlon ln safeLy. 1ralnlng plloLs Lo
make [udgmenLs wlll be a challenge buL lL wlll be lmporLanL as less-experlenced plloLs begln Lo enLer
Lhe professlon.
AlLhough Lhe consLrucLs for crew resource managemenL and LhreaL and error managemenL have hlgh
vlslblllLy, Lhelr currenL lmplemenLaLlon and Lralnlng appears Lo be lneffecLlve. 8ecause Lechnlcal skllls
and nonLechnlcal skllls musL be applled ln Lhe conducL of operaLlng an alrplane, plloLs need Lo be
Lralned on all skllls ln an lnLegraLed manner. roper guldance maLerlal ls needed and perhaps even
lndusLry sLandardlzaLlon ls needed for whaL consLlLuLes effecLlve Crew 8esource ManagemenL Lralnlng
and appllcaLlon.
lnsLrucLors play an lmporLanL role ln achlevlng successful Lralnlng by moLlvaLlng plloLs Lo lmprove and
Lo creaLe and malnLaln a culLure of safeLy. 1o be effecLlve, lnsLrucLors musL recelve quallflcaLlon and be
callbraLed wlLh proper valldaLlon crlLerla. lndusLry needs guldance on how Lo provlde Lhese ln an
affordable and effecLlve way. Change Lo Lhe lnsLrucLor quallflcaLlon and lnsLrucLlonal pracLlces would
yleld an lmmedlaLe lmprovemenL Lo Lralnlng experlence and effecLlveness.
1he lndusLry needs guldance on how Lo develop and dellver operaLlonally relevanL Lralnlng LhaL
Lransfers Lo acLual operaLlons. CperaLors may need comprehenslve guldance on whaL Lo Lraln plloLs Lo
do and how Lo measure Lralnlng effecLlveness ln Lhe conLexL of an alrllne's enLlre culLure. 1hls ls a
challenglng Lask for any operaLor, Lherefore effecLlve guldance and sLandards are requlred, and
sLandardlzaLlon ls needed Lo ensure conslsLency of dellvery. 1ralnlng dellvery meLhods musL advance
Lo dellver an embodled, slLuaLed learnlng envlronmenL conduclve Lo sklll and knowledge developmenL.
A|r||ne ||ot ercept|ons of 1ra|n|ng Lffect|veness


1o make change happen on a global scale, clear valldaLed guldance for conLenL developmenL and
Lralnlng lmplemenLaLlon ls needed wlLh regulaLory engagemenL.
Acknow|edgements
l am graLeful Lo llALA and lA1A for supporLlng Lhe developmenL and dlsLrlbuLlon of Lhls survey.
1hanks Lo !ohn Scully and !arl nlsula who collaboraLed on drafLlng Lhe survey. l'm graLeful Lo !erry
relser, 8ay 8oberLs, Wlley Moore, and !lm 8urln for helpful revlews of Lhls paper. Speclal Lhanks Lo all
Lhe plloLs who compleLed Lhe survey for Lhelr Llme and dedlcaLlon Lo advanclng plloL Lralnlng and
avlaLlon safeLy.