Epilogue. Integration of Leadership, Education, Training, and Self-Development

Riccio, G. (2010). Integration of Leadership, Education, Training, and Self-Development. In: Riccio, G., Diedrich, F.
, & Cortes, M. (Eds.). An

Initiative in Outcomes-Based Training and Education: Implications for an Integrated Approach to Values-Based Requirements (Epilogue). Fort
Meade, MD: U.S. Army Asymmetric Warfare Group. [Cover art by Wordle.net represents word frequency in text.]
Epilogue 254
Asymmetric Warfare Group
Epilogue. Integration of Leadership, Education, Training, and Self-Development

Gary Riccio
The Wexford Group International

The purpose of this final chapter is to summarize the topics of discussion that, by the end of the
AWGs scientific engagement in OBTE, were most urgent and vibrant with respect to
implications for social and programmatic decision-making about Army training, education, and
leader development.

Toward Values-Based Standards for Army Doctrinal Requirements

To date, the proponents and participants in OBTE have stressed that we should consider all the
outcomes of specific learning events that are relevant to preparation of Soldiers for Full Spectrum
Operations (FSO), not just the outcomes that are intended or explicit in a Program of Instruction
or Training Support Package. This has led us to the conclusion that OBTE provides an
overarching vision for training, education, and leader development that makes all these endeavors
relevant to FSO and to operational commanders in theater (see Chapters 3, 4, and 5; see Figure 1).
The principles and practices of OBTE follow, arguably in a deductive fashion, from this vision of
relevance. While not especially surprising theoretically, at least in hindsight, this conclusion has
important practical implications. Most importantly, it suggests that OBTE is both a manifestation
and response to existing Army doctrine. The cascading implications are outlined below.

Figure 1. Triadic balance and equilibrium among intangible personal attributes
and the associated scaffolding for capabilities of the human dimension in Full
Spectrum Operations and competence in trans-extremis environments (e.g.,
transitions between strategically significant combat and noncombat operations).
See Chapter 3, section 3.2.3.

255 Riccio

The current notion of OBTE as vision for learning and development suggests that we can
approach the relationship between immediate learning objectives and long-term developmental
outcomes from the opposite direction, in a sort of reverse planning from what the Army expects
of its Soldiers coming out of programs of training and education. The intangibles such as
confidence, initiative, and accountability help bridge the gap between behavior that is
immediately observable and behavior that is meaningful and efficacious in FSO. But what are the
ultimate outcomes? Our current belief is that OBTE does not suggest a fundamentally new set of
concepts to be added to doctrine. Given the vision and the purpose of preparing for FSO,
outcomes should be derived from doctrinally accepted concepts such as Army Values and
Warrior Ethos or at least they should be traced to them.

One of the strengths of OBTE is that its implementation can be verified. OBTE thus can help
translate existing doctrinal requirements such as Warrior Ethos and Army Values into practices
that actually are observable and around which programs of instruction can be designed and
developed. By showing that intangibles can be observed through their behavioral manifestations
(a common proposition in the physical, biological, and behavioral sciences) and that measures
can be developed for observable behavior, we also can claim that standards can be identified for
such concepts. The logical conclusion of this line of argument is that a rigorous scientific
approach to development, verification, and validation of OBTE provides a way to identify
standards for value-based requirements in existing doctrine for which there currently are no
measures or standards.

A more axiomatic formulation of this line of argument is:

(1) The most fundamental aspects of doctrine are the requirements that are explicit or implicit in
Army Manuals, Regulations and associated documents. Concepts like Army Values and
Warrior Ethos are among the most common requirements stated explicitly in Army doctrine.

(2) The Army is a standards-based organization but there currently are no doctrinally codified
standards for some of the most important requirements in Army doctrine (e.g., as Army
Values, Warrior Ethos).

(3) Verifiability and validity are two universally accepted criteria in capabilities development and
quality assurance that all standards should address directly. Verifiability means that one can
observe something tangible that relates directly to the standard (e.g., one can verify that one
is doing what one says one is doing or should be doing). Validity refers to the relevance of
the standard to the requirement (e.g., does training to the standard help one meet the
requirement).

(4) The validity of all standards (e.g., current marksmanship qualification standards) is intangible
with respect to some of the most important requirements in the Army (i.e., Army Values,
Warrior Ethos). The relationship is inductive not deductive. Nevertheless, not all standards
and inductive relationships are created equal. One should develop, refine, and utilize
standards for which confidence in validity is greatest (i.e., for which evidence about validity
has the most depth and breadth).

(5) To date, OBTE is the only approach to training, education, and leader development that has
generated measurespotential standardsfor some of the most important requirements in
the Army (e.g., Army Values, Warrior Ethos). While there is evidence for the validity of
these potential standards, the more important point is that OBTE has shown that such
standards can be developed in principle.
Epilogue 256

(6) Tasks, conditions, and standards are particular solutions to particular requirements. The most
desirable solutions are ones that can evolve though refinement, upgrades, or other adaptations
to a changing context for the associated requirements over the life cycle of the solution.
Verification and validation fosters continued viability and relevance of a solution by
providing feedback about what to sustain and what to improve.

(7) OBTE arguably is the only approach to training, education, and leader development that has
generated a methodology for integrated verification and validation.

The application of OBTE to doctrine embodies a broader conceptualization of standards, one that
connotes a quality of something that is attainable, not just standardization and evaluation with
respect to norms. The implications about standards and their employment reveal another way that
OBTE transcends particular instructional methods. Irrespective of the methods utilized in training
and education, there is a need for further development of standards for values-based requirements
in current Army doctrine. The validity of such standards cannot depend critically on any
particular methodology even though, in particular situations or under particular conditions, some
instructional methodologies may be more effective than others in achieving values-based
standards.

Nested Standards and Quality Assurance

The elaboration and refinement of values-based standards provides critical guidance to instructors
and their chain of command but it is not sufficient. More guidance is needed with respect to how
such standards can be achieved. Without clarity about the means to the ends, recognition of gaps
or shortfalls in achieving standards will not be actionable; there will not be a way to assure that
achievement of standards will be sustained let alone to identify ways to improve instruction with
respect to the achievement of standards. In this respect, the most important contribution of the
scientific engagement in OBTE to date has been the development of formative measures for
instructors (e.g., Chapter 2, Appendix A) and the scientifically based approach to verification and
validation with which the measures are associated. The measures are the only way to verify that
OBTE is being practiced as it has been defined and thus to validate that it can be effective. They
provide a rigorous framework for the development of values-based standards that are achievable.

More generally, the most important realizations about OBTE are that there are requirements in
Army doctrine for which solutions are undefined or unverifiable, that measurable standards can
be developed for them, that values-based standards need not displace existing knowledge-based
or skill-based standards, that the achievability of values-based standards can be verified with
formative measures for instructors, and that achievement of values-based standards need not
require additional resources for training and education. The logical implication of these
realizations is that multiple standards of different kinds should be addressed in every learning
event. Note that the implication is not that values-based standards are merely a higher level
quantitatively on measures associated with existing standards. They are standards of a
fundamentally different kind.

The formative measures developed for OBTE (Appendices A and B) provide a menu from which
instructors, and other stakeholders with an interest in quality assurance, can select a manageable
number of measures for planning, executing, and learning lessons from a particular learning
event. None of these measures precludes assessment with respect to knowledge-based or skill-
based standards, although they certainly can inform priorities and tradeoffs with respect to an
exhaustive list of tasks, conditions and standards that are not practicable (Appendix C). The
257 Riccio

evidence from this investigation supports these claims, in essence, that multiple measures of
different kind can be used in the same learning event and that values-based measures in particular
can inform decision-making in planning and executing instruction (e.g., Chapters 8, 9, 13). This
realization offers great promise for packaging training and education in more efficient ways (e.g.,
Perry & McEnery, 2009). OBTE, and specifically the associated formative measures, provide
clues about how to link tasks to achieve multiple goals at the same time. Thus, in addition to
showing that values-based standards dont impose an additional resource load, formative
measures for OBTE may even show a path to achieving more with fewer resources.

It also is clear that there is much to be learned about the efficient and effective use of multiple
measures. Importantly, given the foundation provided by this investigation, increasing clarity on
such critical issues now is largely a matter of paying attention to the most important outcomes of
training and education whether intended or not, focusing collective priorities in a program of
instruction within a command intent that addresses the most important outcomes, and
documenting progress (Chapter 11; Appendix C). It is noteworthy that the value of these actions,
and the clarity they bring, is not limited to programs that adhere to the principles and practices of
OBTE. The principles and practices of OBTE will, however, bring greater value insofar as they
were developed pointedly to address values-based outcomes (Chapters 1-5).

Key constructs in OBTE are intangibles such as confidence, initiative and accountability. They
are assumed to have causal potency with respect to the developmental outcomes that motivated
OBTE. The trustworthiness of this assumption is revealed by the extensive dialectical reflection
and analysis we conducted during measure development, field-based observation, and the
development of grounded theory for the practical wisdom of Soldiers as leaders and instructors
(e.g., Chapters 2-5). The intangibles emphasized by OBTE relate directly to values-based
doctrinal requirements such Army Values and Warrior Ethos (see e.g., Chapter 3; Appendix D).
The identification and promulgation of values-based standards will require additional attention to
outcomes that are influenced by these intangibles (cf., Bandura, 1997; see Figure 1). There is a
need to identify and refine the understanding of such outcomes as effects on students that can
transfer to different situations including performance in Full Spectrum Operations (cf., Leibrecht,
Wampler, & Pleban, 2009) if not to other aspects of a Soldiers life (see Chapter 4, section 4.3.2;
Chapter 5, section 5.2).

We recommend that development of values-based standards consider the outcomes defined or
refined locally in collaborative decision-making among the instructor cadre within the local
commanders intent (Chapter 3, section 3.2.7; Chapter 14; Appendix C). Such outcomes and
associated measures currently are in the early stages of decentralized development across various
programs (Haskins, 2009; Perry & McEnery, 2009). Outcomes can be addressed systematically
across programs and within a time scale that is short relative to the refinement of doctrine.
Doctrine can assimilate this localized innovation and translate it into guidelines that stimulate and
focus innovation rather than discourage it. Within such guidelines, instructors and their chains of
command are in the best position to identify observable and achievable outcomes with the most
relevance to stakeholders for their programs of instruction. Collaborative identification of
outcomes is a means for instructors to develop and direct internal sources of motivation for good
instruction. It is a means for the instructional organization to operate coherently with respect to
the commanders intent and to achieve a command climate within which unnecessary unintended
friction is readily identified and eliminated.

Epilogue 258

Figure 2. Programmatic view of the development and employment of values-based
standards as well as associated instructional principles, practices, and measures (i.e.,
OBTE). Solid lines represent direct programmatic influences. Dotted lines connote
informal or indirect programmatic influences. Heavy solid lines connote direct
psychological causes and effects of instruction on students. Compare with Figure 8 in
Chapter 3.

To summarize key points about multiple standards:

Standards for values or ethos can be addressed in the same learning event as existing
standards.

Tasks, conditions, and standards can be designed to allow multiple standards to be
addressed in the same learning event, conditions permitting.

Addressing multiple standards in the same learning event suggests the potential to
accomplish more with the same resources or less.

Standards should reflect outcomes that are developed in decentralized innovation across
programs of instruction.

259 Riccio

Needs and Opportunities for Staff & Faculty Development

A Role for Science and Measurement

Several promising directions for instructor education in OBTE are outlined below. While the
recommendations are based on established practices, they are nuanced in the context of OBTE.
The nuances derive from the intent to be directive about process while not being directive about
content (see e.g., Chapter 3, section 3.2.7; Chapter 14; Appendix C), to foster flexibility and
adaptability in instructional planning and execution, and to strike a balance between the science
of teaching and the art of teaching. The subtlety we seek was captured well by William James in
his talks to teachers over a century ago:

The teachers of this country, one must say, have its future in their hands. The
earnestness which they at present show in striving to enlighten and strengthen
themselves is an index of the nations probabilities of advance in all ideal
directions [p. 3] You make a great, a very great mistake, if you think that
psychology, being the science of the mind's laws, is something from which you
can deduce definite programs and schemes and methods of instruction for
immediate school-room use. Psychology is a science, and teaching is an art; and
sciences never generate arts directly out of themselves. An intermediate inventive
mind must make that application, by using its originality [p. 7] A science only
lays down lines within which the rules of the art must fall, laws which the
follower of the art must not transgress; but what particular thing he
shall positively do within those lines is left exclusively to his own genius [p. 8]...
many diverse methods of teaching may well agree with psychological laws [p.
9] To know psychology, therefore, is absolutely no guarantee that we shall be
good teachers. To advance that result we must have an additional endowment
altogether, a happy tact and ingenuity to tell us what definite things to say and do
when that pupil is before us. That ingenuity in meeting and pursuing the pupil,
that tact for the concrete situation, though they are the alpha and omega of the
teachers art, are things to which psychology cannot help us in the least [p. 9]
We know in advance, if we are psychologists, that certain methods will be
wrong, so our psychology saves us from mistakes. It makes us, moreover, more
clear as to what we are about. We gain confidence in respect to any method
which we are using as soon as we believe it has theory as well as practice at its
back. [p. 11] (James, 1899/1907).

A vast amount of pedagogically relevant research has been conducted in the century since
Jamess remarks but his advice is still relevant. Our advantage today is that psychology and other
academic disciplines are in a better position to help instructors become more keenly aware of
where certain methods will be wrong, to save us from mistakes, and to give us confidence in
respect to any method which we are using as soon as we believe it has theory as well as practice
at its back. We were mindful of Jamess advice from the beginning of the investigation, and it
thoroughly infused our program of research. In general, through our interactions with the
progenitors and stakeholders of OBTE, we diligently sought to identify the tact and ingenuity to
tell us what definite things to say and do when that pupil is before us. Concurrently, we
identified relevant science and scholarship that lays down lines within which the rules of the art
must fall, laws which the follower of the art must not transgress (see e.g., Chapters 2 and 3).

Without something like the formative measures for instructors developed in this investigation, for
example, there is no principled basis for instructor education with respect to values-based
Epilogue 260
requirements in current Army doctrine. The student measures that were added toward the end of
the scientific engagement (Chapter 12; Appendix B) also can provide formative feedback to
instructors and to students about the efficacy of instructor behavior. They are measures that show
whether the instructor is having some effect on the student and what some of those immediate
effects are. However important, they are merely indicators (i.e., efficient causes) that outcomes
are likely to be influenced by instructors, especially if high scores are obtained on the instructor
measures (see Chapters 8 and 9). These can be dependent variables in a study of OBTE but they
should not be confused with outcomes. With regard to outcomes emphasized by OBTE, they are
intervening variables. The grounded theory outlined in Chapters 3-5 provides a foundation for
further research into measures that provide a more complete picture of the impact instructors and
their programs of instruction can have on individual learning and development with respect to
values-based outcomes.

Toward Best Practices in Instructor Education

OBTE illuminates both needs and opportunities for development of staff and faculty associated
with Army training, education, and leader development. It is important to note that none of these
actionable implications are uniquely required by OBTE. They apply to any approach to training,
education, and leader development, in principle, and they apply to current practices in these areas.
The need for increased resources often is associated with OBTE merely because practitioners
have a keener awareness about the best use of resources. They have greater motivation to achieve
more or at least they have a more coherently directed motivation. The insights of OBTE may
simply stimulate reconsideration about the distribution of resources. It may also reveal that an
increase in resources devoted to training and education, such as instructor education, can have a
disproportionately high increase in return on the investment. Again, this is not to be confused
with a requirement of OBTE for more resources.

Field-based training. The AWGs Combat Applications Training Course (CATC) is an effective
way to familiarize instructors with OBTE (see Chapter 6). Such opportunities for experiential
learning are an important element of instructor education but CATC is not sufficient to ensure
that instructors will understand the general principles of the new approach. Instructors should be
familiarized with the application of OBTE to different skill sets or knowledge sets (see Chapter
11). We recommend use of formative measures (see e.g., Chapter 2) in such courses to assure
quality and for collaborative reflection with participants in the course. The measures would help
student-instructors appreciate the ways in which the general principles and practices of OBTE are
manifested in particular instructional strategies and events. They would help see beyond the
particulars of the learning experience to reveal the deep structure of learning and instruction.

Seminars and workshops. The AWG has provided seminars on a regular basis to stakeholders in
OBTE (AWG, 2008a,b, 2009). These events have been very useful in helping instructors and
their chain of command come to understand the vision, purpose, principles, and intended
outcomes of OBTE even though the events are only several hours in duration. We prefer a
workshop format with two half-day periods on consecutive days given that discussion about
OBTE generally simulates subsequent reflection with noteworthy personal implications. We
expect that workshops will be more successful when they are coupled with experiential learning
(e.g., CATC). This assumption is consistent with our informal observations during this
investigation.

We have recommended that workshops include multiple sessions in which the topic of discussion
is the experience of instructors (Sidman, Riccio, Semmens, et al., 2009). Toward that end, it
would be valuable to have a corpus of narrative vignettes that provide concrete examples of an
261 Riccio

instructors thinking and behavior. The vignettes could describe instructional events at a level of
detail that provides enough substance and enough left unstated to stimulate meaningful
collaborative reflection on a time scale commensurate with the time scale of the events described
in the vignettes. The formative measures for OBTE (e.g., Chapter 2) provide an excellent
framework both for generating such a corpus and for discussing the vignettes. In a sense,
instructional vignettes can provide a way for the student-instructor to get inside the head of a
more experienced instructor as conditions are observed, as in-stride decisions are made, and as
consequences unfold. We believe this would help demystify good instruction. It also would have
the added value of showing student-instructors how to use formative measures for quality
assurance and self-development.

Collaborative decision-making. We have stressed the importance of participation of instructional
cadre and their chain of command in making decisions about outcomes and associated measures
of interest in their own programs (Chapter 3; section 3.2.7; Chapters 7 and 14; Appendix C).
These collaborative discussions have the added value of helping individual instructors develop a
broader and deeper understanding of OBTE. It is another way of learning from peers and
contributing to that learning. It provides more direct connections between general theory and
immediate practice.

Multimedia products. An advantage of a field-based course is that a student-instructor can see
examples of good instruction. One of the limitations of this experiential learning is that it can be
difficult to differentiate the essential from the incidental, to appreciate the general principles and
practices in the particulars in which they are manifested. Exposure to a variety of instructors and
applications of OBTE can be a powerful and effective method to help students generalize and to
learn more deeply. Video records would be a simple, low-cost, and highly accessible method for
student-instructors to be exposed and re-exposed, on demand, to a variety of instantiations of
OBTE. Moreover, established methods in multimedia instruction could blend aspects of field-
based training and seminars. We have developed multimedia materials for instructor education
that utilized formative measures for values-based requirements to guide viewers through video
segments of instructional events (Bruny, Riccio, Sidman, Darowski, & Diedrich, 2006; cf.,
Straus, Shanley, Burns, Waite, & Crowley, 2009).

There is added value to multimedia methods of instructor education that address a recognized
need. Student-instructors and their leadership sometimes express skepticism that they are capable
of the kind or level of instruction demonstrated by master instructors or by individuals with
extraordinary credentials as Soldiers. While OBTE or any instruction in the Army can benefit
from extraordinary experience and expertise in Soldiering, we believe that OBTE does not require
such levels of mastery. Common instructors have the potential to improve their instruction with
respect to the principles and practices of OBTE (e.g., Chapters 8 and 9). Nevertheless, it would be
valuable for student-instructors to be exposed to instructors like me in situations like mine. This
need not be limited to instructors. Multimedia materials also can highlight the role of an effective
Brigade or Battalion commander, a tipping-point First Sergeant or Company commander, or other
influential leaders.

Distributed collaboration. Collaborative decision-making and peer-to-peer sharing of lessons
learned need not be limited to forums in which all participants are physically co-located (Chapter
11). There will be increasing opportunities for web-based social networking for the foreseeable
future. Such capabilities should be leveraged for formal or ad hoc instructor education. One of the
current needs in this context is finding outside experts who can help relative novices find the right
information in the right amount and in the right amount of time. More generally, there is a need to
help individuals share their respective expertise in the context of common or convergent interests.
Epilogue 262
Internet portals that facilitate such decentralized ad hoc task organization are a promising
potential solution to this need (Mikroyannidis, 2007; Riccio, Lerario, et al., 2006). Web-based
utilities that enable user generated content and its distribution are transforming use of the web
as well as the perception of what the web affords. This can lower barriers to entry into
communities of practice and thus extending the opportunities to engage in and benefit from
volunteerism (McKee, 2007; Putnam, 2001; Thoits, & Hewitt, 2001).
\
Textbooks, pamphlets, and pocket guides. Textbooks are how novice instructors outside the
military teach with little or no background in teaching and even sometimes with little background
in the subject matter. One generally can use a text however one wants but at least it provides a
framework and reference for one's local and momentary initiative. If the textbooks dont help
instructors do this, they are not chosen or used. In any case, the authors get useful feedback about
the value of their contributions. This market-based solution does not necessarily require monetary
gain by the authors and may not even require monetary transactions. Other motives and
transactions can connect volunteers with people who value their special expertise. Recognition,
inclusion, and opportunities for impact can be powerful motivators (Maslow, 1943, 1968).
Content generated by volunteers would require review and perhaps editing by TRADOC but this
demands fewer resources than generating content. With regard to content for instructor education,
reviewing and editing would be simplified significantly by the formative measures developed in
this investigation (see Chapters 2, 7, 12).

In the context of OBTE, textbooks need not and should not tell instructors what to do. They
should provide guidance about how to become a more effective instructor. We believe that an
excellent source of guidance would be a corpus of narrative vignettes that provide concrete
examples of an instructors thinking and behavior as suggested above in the context of workshops
for instructor education. In any case, formative measures for instructor behavior are a valuable
framework within which to seek, provide, and interpret guidance about instruction. Given that
such guidance is not a script for everything an instructor does in a course, it can be concise.
Mnemonics also can be produced on pocket cards for instructors who have been familiarized with
the guidance represented in the cards. Pocket cards reflect the value of guidance that can be
accessed in situ. This is especially important for novices who may not have fully assimilated
lessons learned or retained knowledge of best practices or who may be overloaded during an
instructional event (Freeman, Jason, Aten, et al., 2008).

Portable electronic aids. Elaborate frameworks and context-dependent guidance can be
extremely valuable in situ. Hand-held electronic devices are becoming increasingly common aids
to performance in everyday tasks as well as on the job. As such, they are a natural extension to
pocket cards that are at once ubiquitous and excessive in the Institutional Army and the
Operational Army. Electronic aids are interactive, thus, they allow the presentation of information
to be as simple or as complex as needed. Clever design of menus and the underlying structure
also can help a user find information without knowing exactly what they are looking for. More
generally, use of an electronic aid and the information it provides can be context dependent.

Moreover, read-write capabilities enable hand-held electronic devices also to replace the green
book that is a common tool for collecting notes and observations in the field, especially during
instructional events. The capability to record noteworthy events within a framework of formative
measures can facilitate both instruction and learning. It can greatly facilitate collaborative
reflection either in formal After-Action Reviews or in ad hoc groups after a noteworthy event (see
Chapter 3). Such devices also offer the added value of facilitating the capture and sharing of
lessons learned with peers and stakeholders (see e.g., Riccio, dEchert, Lerario, et al., 2006).

263 Riccio

Critical Considerations for Further Scientific Investigation

The Necessity of Long-Term Studies

In our view, all the debate about OBTE reduces to lack of agreement about the impact of training
and education, in general, irrespective of approach. By way of analogy, any competition that
would pit OBTE against some other approach would be impossible to initiate until all parties
agree on the rules of the game. The various parties in the debate about OBTE are not even close
to being on the same page about what outcomes matter. In fact, the use of the term outcomes
across the Army, and more broadly in education, has become so muddled it may be wise to
consider dropping the term altogether (see Chapter 3). In any case, in the terminology of
empirical science, the most important and difficult questions about OBTE involve the choice of
dependent variables. If we can achieve some agreement on what effects to measure, then we can
tackle the tough decisions about independent variables (i.e., interventions, treatments) or
comparisons (e.g., demographics and characteristics of samples, groups, situations).

OBTE emphasizes long-term developmental outcomes consistent with values-based requirements
in existing Army doctrine. Scientific inquiry can play an important role in identifying behavioral
manifestations (i.e., outcomes) of the values developed in Soldiers as well as antecedents that
reflect progress in the development of such values. Short-term investigations such the ones
reported in this manuscript are valuable but they are not sufficient for overarching programmatic
decisions about OBTE. The value of the empirical work summarized herein is in revealing
proximate effects of any approach to training and education and, more importantly, in verifying
the implementation of any approach (Chapter 1). It cannot, by definition, reveal long-term effects
such as inculcation of values described in Army doctrine. Long-term studies are warranted if not
required if science is to help the Army meet its existing requirements. Moreover, theory
development is necessary to explore connections between short-term and long-term effects (see
e.g., Chapters 3-5). Without theory, it is difficult to take any empirical investigation seriously,
and it certainly is impossible to draw any actionable conclusions based on the evidence. The
broader and deeper the theory, the more confidence one can be about interpretations of the
evidence, and the more responsible one can be in making recommendations based on the
evidence. The seriousness of the tasks currently faced by Soldiers in FSO deserves theory that is
commensurate in depth and breadth.

OBTE claims that, when there is singular focus on a task and prescribed conditions associated
exclusively with a knowledge-based or skill-based standard, there is an unintended reliance on
short-term memorization and conditioning. This may result in a reasonable level of performance
in the learning event but our claim is that it also results in an unintended negative impact on long-
term outcomes. In particular, it can be counterproductive with respect to attributes and values
necessary to perform in ambiguous situations or across the variations implicit in Full Spectrum
Operations (FSO).

OBTE claims that, when leadership at all levels seeks to create a command climate that strongly
encourages the development of values and attributes emphasized by OBTE, the instructors will
naturally begin to identify opportunities to achieve the knowledge-based or skill-based standard
established for performance while also developing a Soldier with respect to long-term outcomes.
Further, we argue that this will lead to the discovery of new and better ways to instruct with
respect to short-term objectives and long-term outcomes within current resource constraints; that
is, it will lead to a more efficient use of resources. Such adaptability and ingenuity is exactly what
is needed in FSO.

Epilogue 264
In essence, OBTE claims that the Army is already conducting outcomes based training and
education, in that there is always long-term outcomes from every event or encounter with a
student. A singular focus on knowledge-based or skill-based standards, however, can lead to
negative outcomes. These are empirical questions, and this is where science can help. These
issues suggest that further research would do well to include social psychological and even
econometric perspectives (see Chapters 4 and 5) more deeply than we were able to address in the
current investigation which initially drew mostly on industrial-organizational psychology and the
experimental psychology associated with human perception and performance, cognition and
learning, and human development (see Chapter 3). Further research should be re-conceptualized
at a higher level in terms of inter-temporal risk-benefit or cost-benefit relationships, for example:

Benefit: What is the impact of any program of training and education on outcomes that
matter for mission effectiveness in Full Spectrum Operations?

Risk: Does OBTE have a positive or negative effect with respect to near-term learning
objectives in a particular program of instruction?

Risk-Benefit Analysis: Does impact of OBTE on outcomes justify additional risk, if any,
to near-term learning objectives?

Cost: Does OBTE require additional costs for materiel or non-materiel resources or
redistribution of such resources to maintain overall cost?

Cost-Benefit Analysis: Does impact of OBTE on outcome justify additional cost or
redistribution of costs, if any, of its adoption?

Note that risk-benefit and cost-benefit analyses are relevant only if there is a tradeoff, that is, if
outcomes come at a risk relative to near-term learning objectives or at cost in additional resources
used. With improved instructional design informed by experience with OBTE, we believe that
there will be no tradeoff. This seems to be the case at sites where instructor cadre and their
leadership have collaboratively re-designed their courses around OBTE (Haskins, 2009; Perry &
McEnery, 2009; Appendix C).

False Dichotomy of Objective-Subjective

Measures and standards that relate to long-term developmental outcomes will look different from
measures typically used to assess performance with respect to knowledge-based and skill-based
standards. There is some concern about subjectivity in measures associated with values-based
standards and, in particular, there are some assumptions about unreliability relative to objective
measures associated with more common standards. As a synonym for reliability and validity,
objectivity is a worthy pursuit and one that is convergent with rigorous subjectivity, especially
inter-subjectivity (Bridge, 2000; Dienes & Perner, 1999; Neisser & Fivush, 1994; Reed, 1996;
Searle (1997). The dichotomy is largely an illusion, however, for a variety of reasons.

Consider standards being applied in training today that are assumed to be objective. For any
standard, there are numerous ways to get to that standard that are not specified or precluded in the
associated regulations. Some of the unspecified ways of approaching a standard would be
interpreted as cheating, others are not so clear because of judgment on the spot when conditions
are not as assumed in the standard. In either case, the result depends critically on subjectivity. In
addition to this practical constraint on objectivity, there may be legitimate questions about the
putative validity of the standard with respect to the operational task to which it ostensibly is
265 Riccio

relevant. Potential violations of the assumption about validity generally are an even more serious
challenge to the common connotations of objectivity. The only way out of this conundrum is to
trust instructors and evaluators in using judgment (subjectivity), to establish a command climate
that motivates and fosters trustworthiness (shared values), to provide instructors and evaluators
with instruments and guidelines that are trustworthy (rigorous) without precluding judgment, and
to engage continually in meaningful collaborative reflection (inter-subjective validity).

When focused on nave objectivity (assessments that could be done by a novice), measures of
performance can pervert the purpose of training and education. Without the currency and
relevance (that novices are incapable of considering) mere verifiability and repeatability are
insufficient to assure development of the Soldier. Novice assessments tend to focus on specific
aspects of the nominal training and education process rather than on the various outcomes,
especially intangible outcomes, that the process influences. Novices can be expected to depend
upon these factors and to operate so as to maximize the arbitrarily observable aspects of their
instruction and minimize the role of their own expertise as a Soldier and instructor. On the other
hand, if instructors focus on intangible outcomes, their experience as a Soldier and instructor is a
critical foundation for reliable assessment of good training. Leaders are familiar with assessing
intangible attributes. The systems for evaluating Officers and Non-Commissioned Officers
depend on the ability of experienced raters to make judgments about competencies, skills, values
and other qualitative characteristics of an individual (AWG, 2009). This is no less valid for
assessing training and education.

The behavioral and social sciences have developed a plethora of methods that attempt to strike a
balance between objectivity and subjectivity, that reflect both considerations rather than an
assumption that one has to choose between them, and thus that provide sophisticated and
trustworthy perspectives on verifiability and validity (Chapters 4, 5, and 11). There is no reason
for programmatic decision makers to ruminate over these problems of measurement or, worse, to
make suboptimal and even counterproductive decisions because of an assumption that the
problems are insolvable. Science provides plenty of options for moving forward, especially when
it is embedded in a broader scholarly context (Chapters 3-5; see also, Flyvbjerg, 2001; Godfrey-
Smith, 2003; Schrim & Caterino, 2006).

Clarity About What Is Evaluated

We have developed measures that reflect best practices of OBTE and, most importantly, are
verifiable in practice. Without this, there is no way of knowing whether, or the extent to which,
OBTE actually is implemented; and without such verifiability, investigating the results in a
program of instruction is meaningless. No scientific conclusions can be drawn nor can any
improvements be suggested without verification of instructional practices. Documents describing
a program of instruction, a training support package, or any other kind of script for a learning
event do not satisfy this criterion. Measures are required so that scientists and evaluators can
verify what is occurring on the ground at the same time measures of effectiveness are being
applied and data on learning are being collected.

The focus of OBTE is on the instructor because instructors are the means through which the
organization has influence on the students. Measurement of instructor behavior is necessary for
this reason and, while it may be sufficient for scientific conclusions, it may not be sufficient to
come to conclusions that are actionable operationally. The reason is that there are organizational
factors that influence what an instructor can do (see Chapters 14 and 15; Appendix C). There is a
need to understand best practices within an instructional organization, including but not limited to
the command climate, with a level of theoretical grounding and practical detail commensurate
Epilogue 266
with the formative measures we have developed for instructor behavior. The thinking that
scientists would do to figure out what to measure within an organization is the very same thinking
that a programmatic stakeholder would have to do to reexamine and reprioritize activities and
objectives within a program. A provocative implication is that it would be useful to create a
forum in which scientists, leaders in programs of instruction, and active duty commanders or their
representatives could engage in collaborative reflection about the principles, practices, and
desired outcomes of training and education. The collaboration in such a forum could look very
much like the COMPASS process used to develop formative measures for OBTE (see e.g.,
Chapter 2).

The return on the investment in science to support programmatic decision-making is greatest if
the scientific assessment yields information that either can improve the capability or refine the
requirements that motivated the capability. One of the most important contributions of the
scientific inquiry into OBTE to date is to argue for a reorientation toward the common
competencies and attributes of Soldiers that matter to active duty units involved in Full Spectrum
Operations, to strive for a fuller and actionable interpretation of requirements for training and
education, and to close any associated gap between the Institutional Army and Operational Army.
Another important contribution is to show how best practices of instruction can become more
readily identifiable based on this reorientation. Together, this allows for systematic scientific
inquiry and a broader scholarly dialog among scientists and nonscientists that can have the
following results:

Stimulate debate and facilitate decision making about instructional methodologies
Identify and facilitate a coherent approach to training, education, and leader development
Provide framework for curriculum-level design and development
Provide framework for continuous and rigorous quality improvement

Next Steps

Further research should vigorously pursue development of (a) standards for the values-based
outcomes of interest to OBTE and Army doctrine; (b) measures that relate to various aspects and
stages of individual progress with respect to values-based outcomes; (c) theory and empirical
methodologies that reveal inter-temporal relationships between immediate observable effects and
long-term outcomes; and (d) an approach to verification and validation that brings quality
assurance into a tight correspondence with staff and faculty development within a comprehensive
approach to program evaluation.

This broad program of research would not be possible without the grounded theory we developed
over the course of this investigation (see e.g., Chapters 3, 4 and 5). These scientific underpinnings
are just a beginning for serious scholarship into OBTE and, more generally, into the critical
interrelationships among training, education, self-development, and leader development. At the
same time, this foundation is sufficient to initiate empirical research and to being to weave
together otherwise disparate sources of empirical evidence needed to support programmatic
decision-making in the institutional Army (cf., Flyvbjerg, 2001; Godfrey-Smith, 2003; Schrim &
Caterino, 2006; see also Chapter 11, section 11.2.3).

Figure 3 shows ways in which the scientific underpinnings for OBTE can guide the development
of values-based standards that can be verified and validated. This should be viewed as a road
map. The specific measures have yet to be identified and applied. It is important to note, however,
that these scientific underpinnings for OBTE are sources of measures that have been scientifically
validated and employed to come to a deeper understanding of situated behavior, situated
267 Riccio

experience, and situated meaning. That is, they preclude having to start from scratch and develop
new measures for all the important activities, decisions, and processes that are critical in OBTE
and, more generally, in the institutional Army. Our collaborative inquiry into the practical
wisdom as instructors and leaders has given us a good start. The integrative work should be
continued so that programmatic decision-makers can have the information they need to have the
greatest impact on preparation of Soldiers for FSO (see Chapters 11 and 15).

Figure 3. Sources of potential measures to support analysis, design, development,
implementation and evaluation of instruction that reflects a programmatic view of the
development and employment of values-based standards. Compare with Figure 2 in this
Epilogue and with Figure 8 in Chapter 3.
Epilogue 268
References

Asymmetric Warfare Group (2009, March). US Army Asymmetric Warfare Group workshop on
Outcomes-based Training and Education, Applied Physics Laboratory, Johns Hopkins
University, Laurel MD.
Bandura, A (1997). Self-efficacy: The exercise of control. New York, NY: Freeman and
Company.
Bridge, G. (2000). Rationality, ethics, and space: on situated universalism and the self-interested
acknowledgement of difference. Environment and Planning D: Society and Space 2000, 18,
pages 519-535.
Bruny, T., Riccio, G., Sidman, J., Darowski, A., & Diedrich, F. (2006). Enhancing warrior ethos
in initial entry training. Proceedings of the 50th Annual Meeting of the Human Factors and
Ergonomics Society, San Francisco, CA.
Darwin, M. (2008a). Asymmetric Warfare Group combat applications training course (CATC)
senior leader discussion (Briefing, January 23, 2008). Fort Meade, MD: Asymmetric Warfare
Group.
Darwin, M. (2008b). Outcomes-based training and education: fostering adaptability in full
spectrum operations (Briefing, December 2008). Fort Meade, MD: Asymmetric Warfare
Group.
Dienes, Z. & Perner, J. (1999) A theory of implicit and explicit knowledge. Behavioral and Brain
Sciences 22(5), 735-808
Freeman, J., Jason, J., Aten, T., Diedrich, F., Cooke, N., Winner, J., Rowe. L., & Riccio, G.
(2008). Shared Interpretation of Commander's Intent (SICI). Final Report to the Army
Research Institute for the Behavior and Social Sciences, contract number W74V8H-06-C-
0004.
Flyvbjerg, B. (2001). Making social science matter: Why social inquiry fails and how it can
succeed again. Cambridge, UK: Cambridge University.
Godfrey-Smith, P. (2003). Theory and reality. Chicago, IL: University of Chicago Press.
Haskins, C. (2009, March). Development of outcomes based training. Presentation at the US
Army Asymmetric Warfare Group workshop on Outcomes-based Training and Education,
Applied Physics Laboratory, Johns Hopkins University, Laurel MD.
James, W. (1907). Talks to teachers on psychology: and to students on some of life's ideals. New
York, NY: Henry Holt and Company. (Original work published 1899)
Leibrecht, B., Wampler, R., & Pleban, R. (2009). Methodology for evaluating transfer of learning
from the U.S. Armys advanced leaders course. Research Product 2009-05. Washington, DC:
U.S. Army Research Institute for the Behavioral and Social Sciences.
Maslow, A. (1943). A theory of human motivation. Psychological Review, 50, 370-396.
Maslow, A. (1968). Toward a psychology of being. New York: Van Nostrand.
McKee, J. (2007). The new breed: Understanding and equipping the 21st century volunteer.
Loveland, CO: Group Publishing.Mikroyannidis, A. (2007). Toward a social semantic web.
Computer, November 2007, 113-115. Los Alamitos, CA: IEEE Computer Society.
Neisser, U., & Fivush, R. (Ed.) (1994). The remembering self: construction and accuracy in the
self narrative. Cambridge, UK: Cambridge University Press.
Perry, R. & McEnery, K. (2009). Army reconnaissance course: Defining the aim point for
reconnaissance leader training. Armor, July-August, 14-20.
Putnam, R. (2001). Bowling alone: The collapse and revival of American community. New York:
Simon & Schuster.
Reed, E. (1996). The necessity of experience. New Haven: Yale University.
Riccio, G., dEchert, B.C., Lerario, M., Pound, D., Bruny, T., & Diedrich, F. (2006). Enhancing
Joint Task Force Cognitive Leadership Skills. Report to the Army Research Institute for the
269 Riccio

Behavioral and Social Sciences, contract number Army contract no. W74V8H-06-P-0186.
Vienna, VA: The Wexford Group International.
Riccio, G., Lerario, M., Cornell dEchert, B., Pound, D., Bruny, T., & Diedrich, F. (2006).
Training a Joint and expeditionary mindset. Report to the Army Research Institute for the
Behavioral and Social Sciences, contract number W74V8H-06-P-0189. Vienna, VA: The
Wexford Group International.
Schrim, S. & Caterino, B. (2006). Making political science matter: debating knowledge,
research, and method. New York: New York University.
Searle, J. (1997). Construction of social reality. New York: Free Press.
Sidman, J., Riccio, G., Semmens, R., Geyer, A., Dean, C., & Diedrich, F. (2009). Reshaping
Army institutional training: Current training. Final Report to the Army Research Institute for
the Behavior and Social Sciences, contract number W74V8H-04-D-0047 DO 0010.
Straus, S., Shanley, M., Burns, R., Waite, A., & Crowley, J. (2009). Improving the Armys
assessment of interactive multimedia instruction courseware. Santa Monica, CA: RAND.
Thoits, P.A., & Hewitt, L.N. (2001). Volunteer work and well-being. Journal of Health and
Social Behavior, 42, 115131.

ix Riccio, Diedrich, & Cortes (Eds.)
TABLE OF CONTENTS
.
page

Prologue: A Programmatic View of the Inquiry into Outcomes-Based Training & Education.......1
Historicity of our Research on OBTE..........................................................................................1
The Approach and Lessons Learned from the Research..............................................................3
Documentation of the Research ...................................................................................................4

Section I. Development of Stakeholder Requirements for OBTE..............................................6

Chapter 1. Preparation for Full Spectrum Operations ......................................................................7
1.1 Requirements of Full Spectrum Operations...........................................................................8
1.2 Outcomes-Based Training and Education (OBTE)..............................................................10
1.2.1 Exemplar of OBTE: Combat Applications Training Course........................................11
1.2.2 OBTE as a Multifaceted Instructional System .............................................................12
1.3 An Appraisal of Instruction with Respect to OBTE ............................................................13
1.3.1 A Systems Engineering Framework for Integration and Development of OBTE........13
1.3.2 Preparation for Validation and Verification .................................................................14
1.4 References ............................................................................................................................17

Chapter 2. Formative Measures for Instructors ..............................................................................20
2.1 Development of Formative Measures ..................................................................................20
2.1.1 The COMPASS Methodology......................................................................................20
2.1.2 Development of Measures for OBTE...........................................................................21
2.2 Description of Formative Measures .....................................................................................21
2.2.1 Results of the COMPASS Process................................................................................21
2.2.2 Elaboration on the Description of Measures.................................................................23
2.3 OBTE Performance Measures: Planning for Training.........................................................23
2.3.1 Define Outcomes ..........................................................................................................23
2.3.2 Create a Positive Learning Environment ......................................................................25
2.3.3 Create the Parameters of Learning................................................................................27
2.4 OBTE Performance Indicators: Training Execution............................................................28
2.4.1 Communicate the Parameters of Learning....................................................................28
2.4.2 Training Emphasizes Broad Combat or Mission Success ............................................29
2.4.3 Customize Instruction When Possible Based on Constraints/Conditions ....................31
2.4.4 Facilitates Learning of Concepts ..................................................................................32
2.4.5 Creates a positive learning environment.......................................................................34
2.4.6 Instructors Utilize Measures of Effectiveness & Self-Evaluation................................36
2.4.7 Uses scenarios to facilitate learning..............................................................................38
2.4.8 Instructors exhibit intangible attributes in own actions ................................................40
2.4.9 Hotwashes and Mini-AAR............................................................................................42
2.5 Uses of the Measures ...........................................................................................................43
2.5.1 Formative Measures for Instructors..............................................................................44
2.5.2 Quality Assurance and Instructor Education ................................................................44
2.5.3 Continuous Improvement of Assessments....................................................................45
2.5.4 Program Evaluation and Organizational Change..........................................................46
2.6 References ............................................................................................................................46

Table of Contents x
Chapter 3. Principles and Practices of Outcomes Based Training & Education............................50
3.1 Multifaceted Inquiry.............................................................................................................50
3.1.1 Interaction with Progenitors of OBTE..........................................................................51
3.1.2 AWG Documents on OBTE .........................................................................................52
3.1.3 Collaborative Reflection on Participant Observation in CATC ...................................52
3.1.4 Interaction with Stakeholders .......................................................................................53
3.2 Essential Characteristics of OBTE.......................................................................................53
3.2.1 The Meaning of Developmental is a Critical Difference..............................................53
3.2.2 The Definition of Outcomes is a Critical Difference....................................................56
3.2.3 The Emphasis on Values and Causally Potent Intangibles is a Critical Difference .....58
3.2.4 The Meaning of Experience is a Critical Difference ....................................................61
3.2.5 The Emphasis on Instructor-Student Interactions is a Critical Difference ...................62
3.2.6 The Emphasis on Learning to Learn is a Critical Difference .......................................63
3.2.7 The Emphasis on Collaborative Design and Development is a Critical Difference.....65
3.3 Toward a Grounded Theory for OBTE................................................................................66
3.3.1 Need for an Integrated Interdisciplinary Framework ...................................................66
3.3.2 Formative Measures of Instructor Behavior as Evolving Best Practices of OBTE......67
3.4 Emerging Best Practices in OBTE for a Community-Centered Environment.....................68
3.4.1 Leadership and Enculturation of Soldiers.....................................................................68
3.4.2 Robust and Adaptable Plan...........................................................................................70
3.4.3 Instructors as Role Models ...........................................................................................70
3.4.4 Collaborative Identification of Outcomes and Measures .............................................71
3.5 Emerging Best Practices in OBTE for a Knowledge-Centered Environment .....................71
3.5.1 Integrated Understanding of Basic Soldier Skills in Full Spectrum Operations ..........72
3.5.2 Task Relevance of Planned Instructional Events..........................................................72
3.5.3 Reveal Operational Relevance of Training...................................................................73
3.5.4 Incorporate Stress into Instructional Events .................................................................73
3.5.5 Identify General Lessons Learned and Extrapolate to New Situations ........................74
3.6 Emerging Best Practices in OBTE for an Assessment-Centered Environment ...................74
3.6.1 Collaborative Reflection and Problem Solving ............................................................75
3.6.2 Communication.............................................................................................................75
3.6.3 Nature and Extent of Guidance.....................................................................................76
3.6.4 Establish a Pervasive Mindset of Collaborative Reflection..........................................76
3.7 Emerging Best Practices in OBTE for a Learner-Centered Environment ...........................77
3.7.1 Soldier Motivation and Development of Intangibles....................................................77
3.7.2 Plan for Development of the Individual .......................................................................78
3.7.3 Get Students to Take Ownership ..................................................................................78
3.7.4 Collaborative Reflection as a Means to Develop Self Efficacy....................................79
3.8 References ............................................................................................................................79

Chapter 4. Grounded Theory for Values-Based Training & Education.........................................86
4.1 Exploration of Holistic and Functionalistic Underpinnings for OBTE ...............................86
4.1.1 Fundamental Units of Analysis.....................................................................................87
4.1.2 Nested Time Scales and Adaptability...........................................................................88
4.1.3 Adaptability and Ambiguity .........................................................................................90
4.1.4 Mechanistic Analogies and Predominant Experimental Paradigms .............................92
4.2 Three Pillars for the Scientific Foundation of OBTE ..........................................................93
4.2.1 Ecological Psychology..................................................................................................93
4.2.2 Self-Efficacy Theory.....................................................................................................97
4.2.3 Positive psychology ......................................................................................................98
4.3 A More Integrated Scientific Infrastructure.......................................................................101
xi Riccio, Diedrich, & Cortes (Eds.)
4.3.1 Self Determination Theory .........................................................................................101
4.3.2 Situated Learning Theory ...........................................................................................103
4.3.3 Existential Psychology................................................................................................105
4.4 Building on the Scientific Infrastructure for OBTE...........................................................109
4.4.1 Triadic Frameworks....................................................................................................109
4.4.2 Further Development ..................................................................................................112
4.5 References ..........................................................................................................................112

Chapter 5. Passion and Reason in Values-Based Learning & Development ...............................118
5.1 The Nested Self ..................................................................................................................118
5.1.1 An Alternative to Individual versus Collective ..........................................................118
5.1.2 Cognition and Reality .................................................................................................119
5.2 Conscious Experience and the Dynamics of Thinking ......................................................122
5.3 Emotion, Information, and Engagement ............................................................................125
5.3.1 Ecological Perspective on Emotion ............................................................................125
5.3.2 Emotion as Engagement .............................................................................................126
5.3.3 Implications for Training and Education....................................................................129
5.4 Emotion, Decision-Making, and Inter-Temporal Choice...................................................129
5.4.1 Toward a More Integrated Theory..............................................................................129
5.4.2 Emotion and Decision-Making...................................................................................130
5.4.3 Emotion and Nested Time Scales ...............................................................................131
5.4.4 Neuroeconomics and Inter-Temporal Reasoning .......................................................132
5.5.5 Inter-Temporal Reasoning and Adaptive Dynamical Systems...................................133
5.5 Beyond Science..................................................................................................................134
5.5.1 Existentialism..............................................................................................................134
5.5.2 The Soldier-Scholar as an Emergent Property of a Collective Pursuit.......................135
5.6 References ..........................................................................................................................137

Section II. Verification and Validation of OBTE as a Service System..................................142

Chapter 6. Initial Impressions of Participation in CATC.............................................................143
6.1 Methods..............................................................................................................................143
6.1.1 Participants..................................................................................................................143
6.1.2 Procedure ....................................................................................................................143
6.1.3 Analyses......................................................................................................................144
6.2 Results ................................................................................................................................144
6.3 Implications for Service System Development: Peer Review ...........................................146
6.4 References ..........................................................................................................................147

Chapter 7. Local Development of Measures of Effectiveness .....................................................149
7.1 What do Instructors Believe Soldiers Should Learn in Initial Entry Training?.................149
7.2 Measure Development Process ..........................................................................................150
7.3 What do OBTE-Trained DS Believe is Important to Assess in BRM/ARM? ...................151
7.4 Implications........................................................................................................................156
7.5 Conclusions ........................................................................................................................158
7.6 References ..........................................................................................................................159

Table of Contents xii
Chapter 8. Observations of Behavior and Communication in Rifle Marksmanship Training .....160
8.1 Methods..............................................................................................................................160
8.1.1 Participants..................................................................................................................160
8.1.2 Procedure ....................................................................................................................160
8.1.3 Analyses......................................................................................................................161
8.2 Results ................................................................................................................................163
8.2.1 Behavior of DS ...........................................................................................................163
8.2.2 Behavior and Performance of Privates .......................................................................165
8.2.3 Patterns of Communication ........................................................................................168
8.2.4 Potential Influence of Instructor Behavior on Performance of Privates.....................170
8.3 Implications for Service System Development..................................................................171
8.3.1 Verification of OBTE .................................................................................................171
8.3.2 Validation of OBTE....................................................................................................172
8.4 References ..........................................................................................................................173

Chapter 9. Impact on Rifle Marksmanship Training....................................................................174
9.1 Behavioral Data Collection During Basic Rifle Marksmanship........................................174
9.1.1 Method........................................................................................................................174
9.1.2 Assessment..................................................................................................................175
9.1.3 Results An Overview...............................................................................................177
9.1.4 Evidence for Influence of OBTE................................................................................178
9.1.5 Behavior of Drill Sergeants after Exposure to OBTE ................................................180
9.1.6 Behavior of Privates....................................................................................................182
9.1.7 Patterns of Communication ........................................................................................186
9.1.8 Summary.....................................................................................................................186
9.2 Attitudes Toward an OBTE in Basic Training...................................................................187
9.2.1 Method........................................................................................................................187
9.2.2 Results.........................................................................................................................187
9.4 References ..........................................................................................................................191

Chapter 10. Influence of CATC in an Operational Setting ..........................................................192
10.1 Methods............................................................................................................................192
10.1.1 Participants................................................................................................................192
10.1.2 Procedure ..................................................................................................................192
10.1.3 Analyses....................................................................................................................193
10.2 Results ..............................................................................................................................193
10.2.1 Downstream Impact on Marksmanship ....................................................................193
10.2.2 Downstream Impact on Training in the Units ..........................................................194
10.2.3 Downstream Impact on Self Efficacy.......................................................................195
10.3 Implications for Service System Development: Validation.............................................196
10.4 References ........................................................................................................................197

Chapter 11. Implications for Service System Development.........................................................198
11.1 Lessons Learned about Transfer of OBTE.......................................................................198
11.2 Implications for Service System Development................................................................199
11.2.1 Further Development and Analysis of Stakeholder Requirements for OBTE..........199
11.2.2 Further Development of OBTE as a Service System ...............................................199
11.2.3 Further Verification and Validation of OBTE..........................................................201
11.3 References ........................................................................................................................203

xiii Riccio, Diedrich, & Cortes (Eds.)
Section III. Further Development of OBTE as a Service System ..........................................206

Chapter 12. Development of General Measures for Students ......................................................207
12.1 Intent ................................................................................................................................207
12.2 Performance Measure Development Process...................................................................207
12.2.1 Phase One: Define Performance Indicators (PI).......................................................207
12.2.2 Phase Two: Translate PI into performance measures...............................................208
12.2.3 Phase Three: Measure refinement.............................................................................208
12.2.4 Phase Four: Retranslation of Measures ....................................................................208
12.3 Product of Measure Development....................................................................................209
12.3.1 Learner Perception of the Instructor and Course ......................................................209
12.3.2 Learner Engagement .................................................................................................211
12.3.3 Student Relationship with Teacher ...........................................................................212
12.3.4 Student Results .........................................................................................................214
12.3.5 Self-Report Measures ...............................................................................................216
12.4 Conclusion........................................................................................................................217
12.5 References ........................................................................................................................217

Chapter 13. Adapting OBTE in a Classroom Environment .........................................................219
13.1 Intent ................................................................................................................................219
13.2 Observing OBTE in the Classroom Environment............................................................219
13.2.1. Participants...............................................................................................................219
13.2.2. Procedure .................................................................................................................220
13.2.3. Measures ..................................................................................................................220
13.3 Utility of OBTE Measures in a Classroom Environment ................................................220
13.3.1 Generality of Measures.............................................................................................220
13.3.2. Implications for Improvement of Measures.............................................................221
13.3.3 Implications for improvement of course design .......................................................222
13.4 Use of 360 Reviews for Collaborative Reflection..........................................................223
13.4.1 The Role of a 360 Review in OBTE.......................................................................223
13.4.2 Narrative of a Participant Observer ..........................................................................225
13.5 Learning, cognitive load and motivation..........................................................................228
13.5.1 The NASA Task Load Index as a subjective measure of student workload.............228
13.5.2 Results.......................................................................................................................229
13.5.3 Implications ..............................................................................................................230
13.6 Conclusions ......................................................................................................................230
13.7 References ........................................................................................................................231

Chapter 14. Organizational Climate and Creation of Durable Change ........................................233
14.1 The Need ..........................................................................................................................233
14.2 Initial Indications of Possible Resistance to Change .......................................................234
14.3 Models and Considerations for Sustainable Change........................................................235
14.3.1 The Change Transition Period ..................................................................................235
14.3.2 Organizational Culture..............................................................................................237
14.3.3 Clarity of Mission and Shared Understanding..........................................................237
14.3.4 Relevant Observations During the Current Investigation.........................................238
14.3.5 Organizational Support and Incentives.....................................................................238
14.4 Conclusions ......................................................................................................................239
14.5 References ........................................................................................................................239
Table of Contents xiv
Chapter 15. Five ways OBTE can enable the Army Leader Development Strategy....................242
15.1 Background ......................................................................................................................242
15.2 An Emerging Consensus ..................................................................................................244
15.2.1 What Part to Balance?...............................................................................................244
15.2.2 Improving Training, by Design ................................................................................245
15.2.3 Increased Use of dL and Dependence on Self-Development ...................................246
15.2.4 Future Orientation, Unknown Requirements............................................................247
15.2.5 The Quality Instructor Challenge .............................................................................247
15.2.6 Purpose and Design are Key.....................................................................................248
15.2.7 A Natural Advantage ................................................................................................249
15.2.8 Task Specialization or Generalized Competency .....................................................249
15.3 Conclusion........................................................................................................................251
15.4 References ........................................................................................................................252

Epilogue. Integration of Leader Development, Education, Training, and Self-Development .....254
Toward Values-Based Standards for Army Doctrinal Requirements ......................................254
Nested Standards and Quality Assurance.................................................................................256
Needs and Opportunities for Staff & Faculty Development ....................................................259
A Role for Science and Measurement .................................................................................259
Toward Best Practices in Instructor Education....................................................................260
Critical Considerations for Further Scientific Investigation ....................................................263
The Necessity of Long-Term Studies ..................................................................................263
False Dichotomy of Objective-Subjective...........................................................................264
Clarity About What Is Evaluated.........................................................................................265
Next Steps ............................................................................................................................266
References ................................................................................................................................268

Section IV. Appendices...............................................................................................................270

Appendix A. OBTE Principles & Practices: Instructor Measures................................................271
A.1 Genesis of Formative Measures for Instructors ................................................................271
A.2 Principles of Outcomes-Based Training & Education ......................................................272
A.3 Guide to Using Measures of Instructor Behavior..............................................................276
A.4 Complete Menu of Instructor Measures............................................................................279

Appendix B. OBTE Principles & Practices: Student Measures ...................................................318
B.1 Guide to Using Measures of Student Behavior .................................................................318
B.2 Complete Menu of Student Measures ...............................................................................319

Appendix C: A Commanders View of Outcomes-Based Training and Education.....................340
Summary ..................................................................................................................................340
Definition.............................................................................................................................340
Description...........................................................................................................................340
Elements of OBTE. ..................................................................................................................341
Developing the Outcomes....................................................................................................341
Developing the Training Plan..............................................................................................341
Conducting Training............................................................................................................342
How Training is Assessed....................................................................................................344
Conclusion................................................................................................................................344
xv Riccio, Diedrich, & Cortes (Eds.)
Appendix D: Warrior Ethos..........................................................................................................345
Analysis of the Concept and Initial Development of Applications..........................................345
Current Understanding of Warrior Ethos.............................................................................345
Purpose.................................................................................................................................348
Approach..............................................................................................................................348
Expansion of the Definition of Warrior Ethos.....................................................................348
The Tenets of Warrior Ethos ...............................................................................................349
Clarifying the Definition of Warrior Ethos..........................................................................351
Warrior Attributes Derived from the Tenets of Warrior Ethos ...........................................353
References ................................................................................................................................355
Supplementary Work Product from Warrior Ethos Project .....................................................355

Appendix E: Indicators of Warrior Ethos.....................................................................................356
Methods....................................................................................................................................356
Participants...........................................................................................................................356
Instruments and Facilities ....................................................................................................356
Procedure .............................................................................................................................356
Results ......................................................................................................................................358
Qualitative Findings.............................................................................................................358
Quantitative Findings...........................................................................................................358
Discussion ................................................................................................................................359

!"#$%&'#( #* &+, '(",-&'./&'#( /- 0,*$,1&,2 '( &+, 1+/3&,0- #* &+'- 4#(#.0/3+5

Epilogue. Integration of Leadership, Education, Training, and Self-Development

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Epilogue. Integration of Leadership, Education, Training, and Self-Development

Transféré par

Droits d'auteur :

Formats disponibles

Riccio, G. (2010). Integration of Leadership, Education, Training, and Self-Development. In: Riccio, G., Diedrich, F.

, & Cortes, M. (Eds.). An

Vous aimerez peut-être aussi