Haskins, C. (2010). A Commander's View of Outcomes-Based Training & Education. In: Riccio, G., Diedrich, F.

, & Cortes, M. (Eds.). An

Initiative in Outcomes-Based Training and Education: Implications for an Integrated Approach to Values-Based Requirements (Appendix C).
Fort Meade, MD: U.S. Army Asymmetric Warfare Group. [Cover art by Wordle.net represents word frequency in text.]

340

Appendix C: Command Climate

Appendix C: A Commanders View of Outcomes-Based Training and Education
COL Casey Haskins
United States Military Academy
Summary

Outcomes-based training and education (OBTE) is a different system for training and for training
management. By way of analogy, OBTE is to training what mission command is to operations:
trainers are given requirements but not directed how they must achieve them. They are then held
accountable for the results.
Whereas standard Army training methods seek to teach Soldiers and leaders how to apply
approved, doctrinal solutions to particular problems, OBTE seeks instead to teach them how to
frame problems and solve them, focusing on the results rather than the methods used to obtain
them. It is thus designed to create thinking, adaptive Soldiers and leaders who are capable of
applying what they know to solve problems they have previously not encountered.
Definition
OBTE is an approach to planning, managing, and delivering training and education. It results in
the attainment of a set of holistic, observable, and measurable skills and behavioral traits
(outcomes) in individuals and units. It does so by requiring a thorough understanding of the
underlying principles and increasing mastery of fundamentals, gained while progressing through
a series of increasingly challenging scenarios. These scenarios at all times require the trainer and
student to think and solve problems in context.
Description
OBTE relies heavily on the trainers growing expertise and ability and much less on scripts,
standard procedures, or external controls. It standardizes training by the results or outcomes. In
other words, success is judged when the student or unit demonstrates they can solve a new
problem to an acceptable level using their newly trained skills or knowledge.
It does not standardize training by the process (every student will first do A, then do B, then
advance to C, and finally qualify on D) or by inputs (each student will fire three rounds, spend 2.5
hours on this topic, watch these slides, do three iterations of this drill, etc.) It recognizes that
many skills and situations have more than one acceptable way of solving the problem and does
not mandate one particular solution. Nor does it specify one way to teach a particular skill or to
learn it.
OBTE explicitly focuses at every step on developing such intangible traits as critical thinking,
judgment, problem solving, initiative, and accountabilityin both the instructors and the
students.
Though there are similarities, Outcomes-Based Training and Education are not just new words for
old concepts. Outcomes are not merely terminal learning objectives. Measures of effectiveness
are not merely task standards. Many senior leaders, upon hearing of OBTE, claim theyve always
done it. This is a common misperception. The vast majority have not.

Asymmetric Warfare Group

341

Haskins

Finally, OBTE requires no more resources than the standard Army training model. While it will
require a new method for allotting resources, since inputs (number of rounds, hours, miles, etc.)
no longer drive the training, a standard package of resources will still suffice, so long as the
trainers have more flexibility about how to use them.
Elements of OBTE.
Developing the Outcomes
This is the first step and arguably the most important. It requires clear thinking, and to succeed it
requires ownership by the leaders who will have to achieve them. The outcomes should be
simply stated in clear language. Ideally they are broad rather than detailed, and they are holistic.
In other words, they are not meant to stand alone, but together. That is important because it
prevents a leader from conducting training that might further one outcome while hindering or
even going backward in others. In the case of a unit, the outcomes are a statement of what the
unit should be able to do and they should describe its values and behavior. In the case of a
school, they specify what each graduate should be and be able to do.
Each outcome is then expanded one level into measures of effectiveness (MOE). These MOE
answer the question what does success in this outcome look like? Taken together, the MOE
define success. Furthermore, each MOE should be measurable or at least observable, so that they
form not just the basis for the training program, but also the basis for the assessment.
Developing the Training Plan
OBTE is most similar to standard training in this step. However, there are two main differences.
The training plan is not the point around which everything hinges; the outcomes are. So if
something isnt working, the plan changes. Traditionally, units have tended to do this well, but
schools were much more rigid. Using OBTE, the POI in a school will change from one cycle to
the next as leaders assess success or failure at achieving outcomes.
The other way in which OBTE differs from standard Army training plans is that trainers receive
more latitude. Less is directed by higher headquarters, but the plan correspondingly builds in
accountability. For instance, if land navigation is part of a school, the plan might build in several
days with little direction, available to junior leaders, and resourced with training areas and
transportation. There would be no expectation that each leader would use the time the same way.
A description of the types of navigation problems each student would be expected to solve on the
final day should provide sufficient guidance for trained leaders to teach their Soldiers; those
Soldiers performance will then show clearly how well each leader did and allow the chain of
command to hold the leaders accountable.

Asymmetric Warfare Group

Appendix C: Command Climate

342

Conducting Training
This is the area in which OBTE most differs from standard Army training. The differences stem
mostly from an attempt to make training better fit human nature, rather than working against it as
Army training often has. It is also geared to fit better with the ambiguity and complexity of
todays fights.

Far more flexibility is granted to both teachers and students, with far fewer external
controls. This requires an investment in training leaders (both on the skills and on how to
teach) but an overwhelming majority of both leaders and Soldiers respond
enthusiastically and perform at a much higher level.

Training emphasizes principles rather than checklists, procedures, or standards. While

there are certainly minimum standards for what an acceptable performance is, they are
rarely discussed with the students, who therefore tend to strive to do their best.

Training emphasizes the why. Traditional Army training emphasizes the what and
how but too often neglects the why. For instance, every Soldier knows how to low
crawl, but surprisingly few can explain when it might make sense to do so. They
therefore avoid doing it at all or they tend to do it in inappropriate circumstances. OBTE
builds in the why from the beginning, and reinforces it at every step by requiring the
student to solve problems using principles and newly acquired knowledge and skills.

As students solve problems, and learn by doing, they are required to figure as much of it
out for themselves as they are capable of. This guided self-discovery, in which the
leader helps the student only through the bits of a problem that he cant quite figure out
for himself, helps the student to more thoroughly understand the concepts and to connect
them to their context. While it may take a bit longer to learn, the learning is deeper and
retained longer. Students in OBTE are conditioned from the beginning to think and to
solve problems. In much of traditional Army training, by contrast, students are told what
to do, whether or not they understand why. They are therefore being conditioned not to
think and not to solve problems, but rather to follow sequences and procedures. This
conditioning causes them difficulty when they are forced to adapt to new and unexpected
circumstances. Not only are they less experienced at solving problems, but they are
conditioned not to try.

Problems in OBTE are realistic and dont divorce the skill from its context. To take an
example, treating a casualty in combat is different from treating one in a classroom or on
a range. This situation changes the medical response. For instance, the need to move the
casualty to a less vulnerable location for further treatment not only changes the sequence
of treatment steps in controlling bleeding (a tourniquet is immediate rather than after
trying direct pressure and elevation) but also requires the Soldier to balance medical
treatment with security, control of the unit, and accounting for personnel. Together, this
results in a fundamentally different learning experience from traditional training that
isolates each task and trains them one at a time.
(Note: Don Vandergriffs Adaptive Learning Model is essentially a classroom variant of
this. Students are challenged with tactical decision exercises of increasing difficulty, so
that as they learn to solve complex tactical problems, they do so in realistically
challenging scenarios, requiring difficult decisions, tradeoffs, and follow-through.)

Asymmetric Warfare Group

343

Haskins

Stress is reversed. Normally in an Army school, the student will experience a high level
of stress at the beginning. Whether induced intentionally or inadvertently through the
unfamiliarity of the situation, that stress usually interferes in mastering the fundamentals.
The student is more focused on avoiding trouble than in mastering the skills. OBTE, in
contrast, begins training with little or no stress. Treating the students as adults, trainers
explain skills thoroughly, breaking them down into their fundamentals, making sure
students understand principles and the why behind each skill, but without talking down
to them. On the other hand, as the student begins to master the basics, OBTE presents
him with increasingly difficult problems to solve with his new skills, even while he
struggles to master them. This results in increasing stress as the training progresses,
which helps the student to cement his skills and understanding, helps him to gain
confidence as he succeeds at handling more challenging problems, and helps him learn to
manage high levels of stress as he performsall with little or no yelling or other artificial
stressors. Students in traditional Army schools, by contrast, tend to experience decreased
stress as they adjust to the environment and get more comfortable. (Ranger students later
in the course require more yelling or bigger threats to induce a similar response to early
days.) This reduced stress results in incomplete learning, as the student often fails to
lock in the skills in a realistic combat context.

OBTE aims for a much higher level of mastery of individual and team skills. The
tyranny of throughput and external controls means that traditional Army training (task,
conditions, standards) results in everyone meeting minimum standards, but often
prevents Soldiers from achieving as much as they are capable of, and often penalizes
initiative and new approaches. This is certainly true in schools, but also often true in
units. The results are unfortunate but predictable. The level of individual expertise in
many units is too low to enable a unit to excel in combat situations. Worse, the
traditional approach can foster a climate where Soldiers and even leaders tend to wait to
be told what to do next, rather than exercising judgment and initiative.

Mistakes in OBTE are treated as opportunities for learning to occur. Students are
encouraged to try things for themselves and to learn from their mistakes. The leaders
role is to make sure that the students analyze why something went wrong and to draw
reasonable lessons from the experience. (Of course the leader is expected to exercise
judgment and not allow mistakes that will result in catastrophe.) Too often, Army
training treats mistakes as things to be avoided or nipped in the bud. Leaders are
rewarded for avoiding mistakes or correcting them immediately and are rarely
encouraged to allow mistakes to unfold so that their Soldiers can learn from them.

Accountability is built into OBTE. Leaders are given both responsibility and the
authority they need to conduct the training their own way. They can then be held
accountable for the results, which are usually clear. But if trainers are required to train a
task in a certain way, as is often the case in Army training, it is difficult to blame them if
their Soldiers dont learn. In OBTE, students too learn accountability. Required to solve
problems, and given the authority to try different solutions, they learn to accept
responsibility for their actions and decisions. Results matter. In almost every case,
students come to see accountability as an enabler, allowing them to solve problems their
own way.

Asymmetric Warfare Group

344

Appendix C: Command Climate

How Training is Assessed

Soldiers are given problems to solve and then watched as they solve them. Those problems
should be unfamiliar to them, but which they could be reasonably expected to solve given their
level of skills and experience. If the problems are properly designed, the trainers who are
observing them will not only see clearly whether the students have mastered the skills, but will
see whether they truly understand why things are done, and will gain insight into such intangibles
as initiative, judgment, and accountability. An added benefit is that the assessment itself is good
training for the students being assessed.
Conclusion
In an Army of motivated volunteers, expected to succeed in difficult and rapidly evolving
battlefields, OBTE is a better way to prepare. It aligns more closely with the way people learn.
While results are preliminary, the evidence is clear that it results in superior mastery of skills,
better retention, higher levels of confidence, and improved judgment, initiative, and
accountability.
On the other hand, implementing it poses challenges. It relies much more on leaders abilities,
and so requires investment in better preparing leaders. They need to have the skills, they need to
be able to explain why things are done the way they are, and they need to be able to teach. It
requires the right command climate. Strict insistence on uniformity and standardization will stop
it dead, since OBTE requires both leaders and students to experiment with different solutions. It
requires commanders to create conditions that allow it, including neutralizing external agency
vetoes, and removing rules and processes that prevent initiative. Finally, it will require a different
method of allocating resources to training, and more flexibility in using them, since resources are
currently matched to tasks being trained rather than to skills attained. Nevertheless, it can be
done, and when done it works very well.

Asymmetric Warfare Group

ix

Riccio, Diedrich, & Cortes (Eds.)

TABLE OF CONTENTS
.

page
Prologue: A Programmatic View of the Inquiry into Outcomes-Based Training & Education.......1
Historicity of our Research on OBTE ..........................................................................................1
The Approach and Lessons Learned from the Research..............................................................3
Documentation of the Research ...................................................................................................4
Section I. Development of Stakeholder Requirements for OBTE..............................................6
Chapter 1. Preparation for Full Spectrum Operations ......................................................................7
1.1 Requirements of Full Spectrum Operations ...........................................................................8
1.2 Outcomes-Based Training and Education (OBTE)..............................................................10
1.2.1 Exemplar of OBTE: Combat Applications Training Course........................................11
1.2.2 OBTE as a Multifaceted Instructional System .............................................................12
1.3 An Appraisal of Instruction with Respect to OBTE ............................................................13
1.3.1 A Systems Engineering Framework for Integration and Development of OBTE ........13
1.3.2 Preparation for Validation and Verification .................................................................14
1.4 References ............................................................................................................................17
Chapter 2. Formative Measures for Instructors ..............................................................................20
2.1 Development of Formative Measures ..................................................................................20
2.1.1 The COMPASS Methodology ......................................................................................20
2.1.2 Development of Measures for OBTE ...........................................................................21
2.2 Description of Formative Measures .....................................................................................21
2.2.1 Results of the COMPASS Process................................................................................21
2.2.2 Elaboration on the Description of Measures.................................................................23
2.3 OBTE Performance Measures: Planning for Training.........................................................23
2.3.1 Define Outcomes ..........................................................................................................23
2.3.2 Create a Positive Learning Environment ......................................................................25
2.3.3 Create the Parameters of Learning................................................................................27
2.4 OBTE Performance Indicators: Training Execution............................................................28
2.4.1 Communicate the Parameters of Learning....................................................................28
2.4.2 Training Emphasizes Broad Combat or Mission Success ............................................29
2.4.3 Customize Instruction When Possible Based on Constraints/Conditions ....................31
2.4.4 Facilitates Learning of Concepts ..................................................................................32
2.4.5 Creates a positive learning environment.......................................................................34
2.4.6 Instructors Utilize Measures of Effectiveness & Self-Evaluation ................................36
2.4.7 Uses scenarios to facilitate learning..............................................................................38
2.4.8 Instructors exhibit intangible attributes in own actions ................................................40
2.4.9 Hotwashes and Mini-AAR............................................................................................42
2.5 Uses of the Measures ...........................................................................................................43
2.5.1 Formative Measures for Instructors ..............................................................................44
2.5.2 Quality Assurance and Instructor Education ................................................................44
2.5.3 Continuous Improvement of Assessments....................................................................45
2.5.4 Program Evaluation and Organizational Change..........................................................46
2.6 References ............................................................................................................................46

Asymmetric Warfare Group

Table of Contents

Chapter 3. Principles and Practices of Outcomes Based Training & Education............................50

3.1 Multifaceted Inquiry.............................................................................................................50
3.1.1 Interaction with Progenitors of OBTE..........................................................................51
3.1.2 AWG Documents on OBTE .........................................................................................52
3.1.3 Collaborative Reflection on Participant Observation in CATC ...................................52
3.1.4 Interaction with Stakeholders .......................................................................................53
3.2 Essential Characteristics of OBTE.......................................................................................53
3.2.1 The Meaning of Developmental is a Critical Difference..............................................53
3.2.2 The Definition of Outcomes is a Critical Difference....................................................56
3.2.3 The Emphasis on Values and Causally Potent Intangibles is a Critical Difference .....58
3.2.4 The Meaning of Experience is a Critical Difference ....................................................61
3.2.5 The Emphasis on Instructor-Student Interactions is a Critical Difference ...................62
3.2.6 The Emphasis on Learning to Learn is a Critical Difference .......................................63
3.2.7 The Emphasis on Collaborative Design and Development is a Critical Difference.....65
3.3 Toward a Grounded Theory for OBTE ................................................................................66
3.3.1 Need for an Integrated Interdisciplinary Framework ...................................................66
3.3.2 Formative Measures of Instructor Behavior as Evolving Best Practices of OBTE......67
3.4 Emerging Best Practices in OBTE for a Community-Centered Environment.....................68
3.4.1 Leadership and Enculturation of Soldiers.....................................................................68
3.4.2 Robust and Adaptable Plan...........................................................................................70
3.4.3 Instructors as Role Models ...........................................................................................70
3.4.4 Collaborative Identification of Outcomes and Measures .............................................71
3.5 Emerging Best Practices in OBTE for a Knowledge-Centered Environment .....................71
3.5.1 Integrated Understanding of Basic Soldier Skills in Full Spectrum Operations ..........72
3.5.2 Task Relevance of Planned Instructional Events..........................................................72
3.5.3 Reveal Operational Relevance of Training...................................................................73
3.5.4 Incorporate Stress into Instructional Events .................................................................73
3.5.5 Identify General Lessons Learned and Extrapolate to New Situations ........................74
3.6 Emerging Best Practices in OBTE for an Assessment-Centered Environment ...................74
3.6.1 Collaborative Reflection and Problem Solving ............................................................75
3.6.2 Communication.............................................................................................................75
3.6.3 Nature and Extent of Guidance.....................................................................................76
3.6.4 Establish a Pervasive Mindset of Collaborative Reflection..........................................76
3.7 Emerging Best Practices in OBTE for a Learner-Centered Environment ...........................77
3.7.1 Soldier Motivation and Development of Intangibles....................................................77
3.7.2 Plan for Development of the Individual .......................................................................78
3.7.3 Get Students to Take Ownership ..................................................................................78
3.7.4 Collaborative Reflection as a Means to Develop Self Efficacy....................................79
3.8 References ............................................................................................................................79
Chapter 4. Grounded Theory for Values-Based Training & Education .........................................86
4.1 Exploration of Holistic and Functionalistic Underpinnings for OBTE ...............................86
4.1.1 Fundamental Units of Analysis.....................................................................................87
4.1.2 Nested Time Scales and Adaptability ...........................................................................88
4.1.3 Adaptability and Ambiguity .........................................................................................90
4.1.4 Mechanistic Analogies and Predominant Experimental Paradigms .............................92
4.2 Three Pillars for the Scientific Foundation of OBTE ..........................................................93
4.2.1 Ecological Psychology..................................................................................................93
4.2.2 Self-Efficacy Theory.....................................................................................................97
4.2.3 Positive psychology ......................................................................................................98
4.3 A More Integrated Scientific Infrastructure .......................................................................101
Asymmetric Warfare Group

xi

Riccio, Diedrich, & Cortes (Eds.)

4.3.1 Self Determination Theory .........................................................................................101
4.3.2 Situated Learning Theory ...........................................................................................103
4.3.3 Existential Psychology................................................................................................105
4.4 Building on the Scientific Infrastructure for OBTE...........................................................109
4.4.1 Triadic Frameworks ....................................................................................................109
4.4.2 Further Development ..................................................................................................112
4.5 References ..........................................................................................................................112

Chapter 5. Passion and Reason in Values-Based Learning & Development ...............................118

5.1 The Nested Self ..................................................................................................................118
5.1.1 An Alternative to Individual versus Collective ..........................................................118
5.1.2 Cognition and Reality .................................................................................................119
5.2 Conscious Experience and the Dynamics of Thinking ......................................................122
5.3 Emotion, Information, and Engagement ............................................................................125
5.3.1 Ecological Perspective on Emotion ............................................................................125
5.3.2 Emotion as Engagement .............................................................................................126
5.3.3 Implications for Training and Education ....................................................................129
5.4 Emotion, Decision-Making, and Inter-Temporal Choice...................................................129
5.4.1 Toward a More Integrated Theory..............................................................................129
5.4.2 Emotion and Decision-Making ...................................................................................130
5.4.3 Emotion and Nested Time Scales ...............................................................................131
5.4.4 Neuroeconomics and Inter-Temporal Reasoning .......................................................132
5.5.5 Inter-Temporal Reasoning and Adaptive Dynamical Systems...................................133
5.5 Beyond Science ..................................................................................................................134
5.5.1 Existentialism..............................................................................................................134
5.5.2 The Soldier-Scholar as an Emergent Property of a Collective Pursuit.......................135
5.6 References ..........................................................................................................................137
Section II. Verification and Validation of OBTE as a Service System ..................................142
Chapter 6. Initial Impressions of Participation in CATC .............................................................143
6.1 Methods..............................................................................................................................143
6.1.1 Participants..................................................................................................................143
6.1.2 Procedure ....................................................................................................................143
6.1.3 Analyses......................................................................................................................144
6.2 Results ................................................................................................................................144
6.3 Implications for Service System Development: Peer Review ...........................................146
6.4 References ..........................................................................................................................147
Chapter 7. Local Development of Measures of Effectiveness .....................................................149
7.1 What do Instructors Believe Soldiers Should Learn in Initial Entry Training? .................149
7.2 Measure Development Process ..........................................................................................150
7.3 What do OBTE-Trained DS Believe is Important to Assess in BRM/ARM? ...................151
7.4 Implications........................................................................................................................156
7.5 Conclusions ........................................................................................................................158
7.6 References ..........................................................................................................................159

Asymmetric Warfare Group

Table of Contents

xii

Chapter 8. Observations of Behavior and Communication in Rifle Marksmanship Training .....160

8.1 Methods..............................................................................................................................160
8.1.1 Participants..................................................................................................................160
8.1.2 Procedure ....................................................................................................................160
8.1.3 Analyses......................................................................................................................161
8.2 Results ................................................................................................................................163
8.2.1 Behavior of DS ...........................................................................................................163
8.2.2 Behavior and Performance of Privates .......................................................................165
8.2.3 Patterns of Communication ........................................................................................168
8.2.4 Potential Influence of Instructor Behavior on Performance of Privates .....................170
8.3 Implications for Service System Development..................................................................171
8.3.1 Verification of OBTE .................................................................................................171
8.3.2 Validation of OBTE....................................................................................................172
8.4 References ..........................................................................................................................173
Chapter 9. Impact on Rifle Marksmanship Training....................................................................174
9.1 Behavioral Data Collection During Basic Rifle Marksmanship ........................................174
9.1.1 Method ........................................................................................................................174
9.1.2 Assessment..................................................................................................................175
9.1.3 Results An Overview ...............................................................................................177
9.1.4 Evidence for Influence of OBTE ................................................................................178
9.1.5 Behavior of Drill Sergeants after Exposure to OBTE ................................................180
9.1.6 Behavior of Privates....................................................................................................182
9.1.7 Patterns of Communication ........................................................................................186
9.1.8 Summary .....................................................................................................................186
9.2 Attitudes Toward an OBTE in Basic Training...................................................................187
9.2.1 Method ........................................................................................................................187
9.2.2 Results.........................................................................................................................187
9.4 References ..........................................................................................................................191
Chapter 10. Influence of CATC in an Operational Setting ..........................................................192
10.1 Methods............................................................................................................................192
10.1.1 Participants................................................................................................................192
10.1.2 Procedure ..................................................................................................................192
10.1.3 Analyses....................................................................................................................193
10.2 Results ..............................................................................................................................193
10.2.1 Downstream Impact on Marksmanship ....................................................................193
10.2.2 Downstream Impact on Training in the Units ..........................................................194
10.2.3 Downstream Impact on Self Efficacy .......................................................................195
10.3 Implications for Service System Development: Validation.............................................196
10.4 References ........................................................................................................................197
Chapter 11. Implications for Service System Development.........................................................198
11.1 Lessons Learned about Transfer of OBTE.......................................................................198
11.2 Implications for Service System Development................................................................199
11.2.1 Further Development and Analysis of Stakeholder Requirements for OBTE..........199
11.2.2 Further Development of OBTE as a Service System ...............................................199
11.2.3 Further Verification and Validation of OBTE ..........................................................201
11.3 References ........................................................................................................................203

Asymmetric Warfare Group

xiii

Riccio, Diedrich, & Cortes (Eds.)

Section III. Further Development of OBTE as a Service System ..........................................206

Chapter 12. Development of General Measures for Students ......................................................207
12.1 Intent ................................................................................................................................207
12.2 Performance Measure Development Process...................................................................207
12.2.1 Phase One: Define Performance Indicators (PI).......................................................207
12.2.2 Phase Two: Translate PI into performance measures ...............................................208
12.2.3 Phase Three: Measure refinement.............................................................................208
12.2.4 Phase Four: Retranslation of Measures ....................................................................208
12.3 Product of Measure Development....................................................................................209
12.3.1 Learner Perception of the Instructor and Course ......................................................209
12.3.2 Learner Engagement .................................................................................................211
12.3.3 Student Relationship with Teacher ...........................................................................212
12.3.4 Student Results .........................................................................................................214
12.3.5 Self-Report Measures ...............................................................................................216
12.4 Conclusion........................................................................................................................217
12.5 References ........................................................................................................................217
Chapter 13. Adapting OBTE in a Classroom Environment .........................................................219
13.1 Intent ................................................................................................................................219
13.2 Observing OBTE in the Classroom Environment............................................................219
13.2.1. Participants...............................................................................................................219
13.2.2. Procedure .................................................................................................................220
13.2.3. Measures ..................................................................................................................220
13.3 Utility of OBTE Measures in a Classroom Environment ................................................220
13.3.1 Generality of Measures .............................................................................................220
13.3.2. Implications for Improvement of Measures.............................................................221
13.3.3 Implications for improvement of course design .......................................................222
13.4 Use of 360 Reviews for Collaborative Reflection..........................................................223
13.4.1 The Role of a 360 Review in OBTE .......................................................................223
13.4.2 Narrative of a Participant Observer ..........................................................................225
13.5 Learning, cognitive load and motivation..........................................................................228
13.5.1 The NASA Task Load Index as a subjective measure of student workload.............228
13.5.2 Results.......................................................................................................................229
13.5.3 Implications ..............................................................................................................230
13.6 Conclusions ......................................................................................................................230
13.7 References ........................................................................................................................231
Chapter 14. Organizational Climate and Creation of Durable Change ........................................233
14.1 The Need ..........................................................................................................................233
14.2 Initial Indications of Possible Resistance to Change .......................................................234
14.3 Models and Considerations for Sustainable Change........................................................235
14.3.1 The Change Transition Period ..................................................................................235
14.3.2 Organizational Culture..............................................................................................237
14.3.3 Clarity of Mission and Shared Understanding..........................................................237
14.3.4 Relevant Observations During the Current Investigation.........................................238
14.3.5 Organizational Support and Incentives.....................................................................238
14.4 Conclusions ......................................................................................................................239
14.5 References ........................................................................................................................239

Asymmetric Warfare Group

Table of Contents

xiv

Chapter 15. Five ways OBTE can enable the Army Leader Development Strategy....................242
15.1 Background ......................................................................................................................242
15.2 An Emerging Consensus ..................................................................................................244
15.2.1 What Part to Balance?...............................................................................................244
15.2.2 Improving Training, by Design ................................................................................245
15.2.3 Increased Use of dL and Dependence on Self-Development ...................................246
15.2.4 Future Orientation, Unknown Requirements............................................................247
15.2.5 The Quality Instructor Challenge .............................................................................247
15.2.6 Purpose and Design are Key .....................................................................................248
15.2.7 A Natural Advantage ................................................................................................249
15.2.8 Task Specialization or Generalized Competency .....................................................249
15.3 Conclusion........................................................................................................................251
15.4 References ........................................................................................................................252
Epilogue. Integration of Leader Development, Education, Training, and Self-Development .....254
Toward Values-Based Standards for Army Doctrinal Requirements ......................................254
Nested Standards and Quality Assurance.................................................................................256
Needs and Opportunities for Staff & Faculty Development ....................................................259
A Role for Science and Measurement .................................................................................259
Toward Best Practices in Instructor Education....................................................................260
Critical Considerations for Further Scientific Investigation ....................................................263
The Necessity of Long-Term Studies ..................................................................................263
False Dichotomy of Objective-Subjective ...........................................................................264
Clarity About What Is Evaluated.........................................................................................265
Next Steps ............................................................................................................................266
References ................................................................................................................................268
Section IV. Appendices...............................................................................................................270
Appendix A. OBTE Principles & Practices: Instructor Measures................................................271
A.1 Genesis of Formative Measures for Instructors ................................................................271
A.2 Principles of Outcomes-Based Training & Education ......................................................272
A.3 Guide to Using Measures of Instructor Behavior..............................................................276
A.4 Complete Menu of Instructor Measures............................................................................279
Appendix B. OBTE Principles & Practices: Student Measures ...................................................318
B.1 Guide to Using Measures of Student Behavior .................................................................318
B.2 Complete Menu of Student Measures ...............................................................................319
Appendix C: A Commanders View of Outcomes-Based Training and Education .....................340
Summary ..................................................................................................................................340
Definition .............................................................................................................................340
Description...........................................................................................................................340
Elements of OBTE. ..................................................................................................................341
Developing the Outcomes....................................................................................................341
Developing the Training Plan ..............................................................................................341
Conducting Training ............................................................................................................342
How Training is Assessed....................................................................................................344
Conclusion................................................................................................................................344

Asymmetric Warfare Group

xv

Riccio, Diedrich, & Cortes (Eds.)

Appendix D: Warrior Ethos..........................................................................................................345

Analysis of the Concept and Initial Development of Applications..........................................345
Current Understanding of Warrior Ethos.............................................................................345
Purpose.................................................................................................................................348
Approach..............................................................................................................................348
Expansion of the Definition of Warrior Ethos.....................................................................348
The Tenets of Warrior Ethos ...............................................................................................349
Clarifying the Definition of Warrior Ethos..........................................................................351
Warrior Attributes Derived from the Tenets of Warrior Ethos ...........................................353
References ................................................................................................................................355
Supplementary Work Product from Warrior Ethos Project .....................................................355
Appendix E: Indicators of Warrior Ethos.....................................................................................356
Methods....................................................................................................................................356
Participants...........................................................................................................................356
Instruments and Facilities ....................................................................................................356
Procedure .............................................................................................................................356
Results ......................................................................................................................................358
Qualitative Findings.............................................................................................................358
Quantitative Findings...........................................................................................................358
Discussion ................................................................................................................................359

Asymmetric Warfare Group

Evolution of the investigation as reflected in the chapters of this monograph.