Académique Documents
Professionnel Documents
Culture Documents
Capella University
June 2012
UMI 3517084
Copyright 2012 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
UMI 3517084
Copyright 2012 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
Abstract
The focal goal of this research was to extend the empirical effort on business
simulations as a form of experiential learning by providing the first empirical analysis of
business acumen and knowledge application skills. Disruptions in technology are
providing more opportunities to improve the simulation gaming learning experience and
a number of pedagogical innovations are beginning to emerge which will drive the way in
which business simulations are used in the future. The purpose of this quantitative,
experimentally-based research study was to investigate the use of online business
simulations as a disruptive technology by measuring the change in participants business
knowledge and business acumen compared to traditional corporate training. A sample of
65 participants was randomly selected from a company population of 720 employees and
managers. This quantitative based research study demonstrated the disruptive nature of
online business simulations when it comes to gains in business knowledge by measuring
a 2.55 standard deviation difference in the normalized gains between traditional training
and business simulation training. Baseline tests against a control group and traditional
training group using MANCOVA to account for multiple variables and covariates imply
that online business simulations enhance both business knowledge and business acumen
on a staggering scale and over a very short period of time.
iv
Dedication
This work is dedicated to my wonderful wife and mother of our five young
children. Thank you, Natalie for all of your love and support over these last 15 years. I
would have never left the private equity world and gone into academia without your faith
in me and our purpose in life together. If nothing else, I have learned from this long
journey that true teaching is leadership and most teaching is bad leadership. Thank you
for your example and personal leadership in our own family.
iii
Acknowledgments
Someone once said that defeat is bitter, but only if you swallow. I would like to
acknowledge the support of Dr. Mary Whitman who believed in me and on more than
one occasion, kept me from swallowing that bitter pill. I would also like to acknowledge
my dissertation committee members who have served as great examples to me and as a
source of personal inspiration. I hope that my life can be a small reflection to my own
students of the great principles that they have taught and, more importantly, lived.
iv
Table of Contents
Acknowledgments
iv
List of Tables
ix
List of Figures
CHAPTER 1. INTRODUCTION
Background to the Study
Specific Variables
Rationale
Research Questions
10
Null Hypotheses
11
12
14
Definition of Terms
15
18
19
21
24
28
30
31
33
Knowledge Application
35
36
38
Reflection
41
43
46
48
50
55
58
CHAPTER 3. METHODOLOGY
Philosophy and Justification
65
66
68
69
vi
Research Design
70
70
71
Sample
75
Setting
80
79
82
84
Field Testing
85
85
Data Collection
86
Variables
87
Data Analysis
87
92
Ethical Considerations
97
CHAPTER 4. RESULTS
Training for a Semiconductor Company
101
Participant Demographics
102
Normality Testing
103
ANOVA Analysis
109
MANCOVA Analysis
111
vii
Correlation Analysis
112
Comparison of Means
114
Hypothesis Testing
115
Alternative Hypotheses
117
118
118
119
Summary
121
124
Research Questions
124
Disruptive Innovation
129
130
Practical Implications
131
135
Conclusion
137
REFERENCES
140
152
157
viii
List of Tables
Table 1. Summary of Variables Used in This Study
21
33
69
104
105
106
107
108
109
112
113
114
115
120
123
ix
List of Figures
Figure 1. Conceptual Framework
20
23
56
57
68
72
74
81
86
103
Figure 11. Normal Q-Q Plots and Boxplot for Change in Business Knowledge
107
110
111
CHAPTER 1. INTRODUCTION
There has been a great divide between the gaming community and business
educators over the past twenty five years (Anderson & Lawton, 2009; Gosen &
Washbush, 2005; Wolfe, 1985). The gamers have dismissed educational simulations as
boring and irrelevant while business management educators have dismissed gaming and
simulations as trivial and pedagogically unproven (Aldrich, 2009a, xxi). Both appear to
be right and yet both may have missed an opportunity which lies within an engaging
business simulation and its potential impact on the world of management education
(Anderson & Lawton, 2009). Many business professionals have argued that the K-12 and
higher education systems are failing, myopically trapped in a nineteenth-century world of
learning by knowing, while the twenty-first-century world requires the judgment and
skill of learning by doing (Aldrich, 2009b, p. 12). Disruptions in technology are
providing more opportunities to improve the simulation gaming learning experience and
a number of pedagogical innovations are beginning to emerge which will drive the way in
which business simulations are used in the future (Faria, Hutchison, & Wellington., 2009,
p. 485). One of the major challenges with research in this field is that nobody has shown
definitively that simulation training works in the business world any better than
traditional instruction through workbooks or lectures (Davies, 2003, p. 36). The purpose
of this quantitative, experimentally-based research study was to investigate the use of
online business simulations as a disruptive technology by measuring the change in
participants business knowledge and business acumen compared to traditional corporate
training.
1
Wolfe (1990) identified this problem over 20 years ago, yet the gap still exists.
Researchers continue to use self-assessments rather than more suitable tools because selfassessments are much easier to employ. As a consequence, studies on the educational
merits of simulations often are measuring the affective domain, not the cognitive domain
they purport to measure (Anderson & Lawton, 2009, p. 197). Using perceptions tends to
be advantageous to those who wish to claim the superiority of simulations over
alternative pedagogies because simulations almost invariably are rated positively by
students. The downside of using perceptions is that evidence based on perceptions often
is dismissed by scholars because it lacks suitable rigor. However, studies that attempt to
go beyond perceptions to more objective measures of learning more often than not use
tools best suited for measuring lower levels of learning on Blooms taxonomy (Anderson
& Lawton, 2009, p. 209). The decisions required to effectively run a business simulation
often tap analytical, synthesis, and application skills of Blooms taxonomy (Bloom,
1956). This has led some researchers to believe that using simulations is a powerful and
disruptive form of learning because it is taking place at the higher levels of Blooms
taxonomy (Smith, 2006).
Disruptions often bring significant changes to an industry and these disruptions
create opportunities for those organizations willing to adopt and champion disruptive
technology (Smith, 2006). Christensen (1997) has highlighted the sometimes devastating
impact in the corporate environment of what he refers to as disruptive innovations (p. 41).
Successful, well managed firms that dominate their markets have sometimes gone into a
sharp decline or even collapsed when a new technology disrupts the pattern of their
market segment. Other firms and organizations, however, have handled such transitions
3
problem thus becomes how do business management educators significantly increase the
level of both business knowledge and business acumen for learners today?
Because of the open-ended content of most business simulations, participants can
have a hard time articulating what they have learned during their game-play experience
(Anderson & Lawton, 2007). For example, there are no well-conducted studies that
actually investigate the learning effects of business simulations on both learners
knowledge application skill and business acumen (Anderson & Lawton, 2009, p. 209). In
fact, almost all studies have focused on the students attitudes and perceptions of their
experience with the simulation (Aldrich, 2009, p. 220). The real gap in the research and
literature is the use of business simulations in the world of adult learning or andragogical
approach to learning (Anderson & Lawton, 2009). There is a need to determine whether
simulations are a disruptive innovation because the holy grail of research within
management education is to empirically demonstrate that andragogical techniques lead to
better learning outcomes (Knowles, Holton & Swanson, 2005, p. 235). Recent advances
in computer simulations allow potential business leaders to practice and rehearse these
business management skills in a context rich environment removed from real life, thus
allowing a participant the opportunity to strengthen their skills in an atmosphere of safety
(Sidor, 2008). Using computer based simulations, participants have the ability to review
business management skills and potentially modify their real world application skills.
This forms the following studys primary research question: Do online business
simulations provide an increase in knowledge or business acumen for participants, which
is on the order of a disruptive innovation?
Specific Variables
Participants in the simulation group ran an online business simulation, titled
experimental groups (case study and business simulation) responded to prompt questions
during the three day training period. In an attempt to investigate these research questions,
this study also looked into which of the following variables contributed most to
participants business simulation performance:
(a) Number of years of industry experience
(b) Level of education (i.e., undergraduate, graduate)
(c) Area of expertise in the company
(d) Level of autonomy/ work control at the company
(e) Previous business simulation experience
Rationale
The following topics have been addressed by some scholars as fertile areas for
research: Do participants improve their grasp of interrelationships among the various
functions of business (marketing, finance, production, etc.) as a result of participating in a
simulation? Are the interpersonal skills of participants improved through participating in
a simulation? Do participants in simulations really develop a greater appreciation for the
difficulty of implementing what may, on the surface, appear to be rather straightforward
business concepts? Are business simulations really effective devices for integrating
participants into business programs, and are they effective at improving retention rates?
(Anderson & Lawton, 2009, p. 212).
Based on these research questions, both practical and theoretical reasons were
present for conducting this study. From a practical perspective, knowing the benefits of
engaging students in reflection activities may be very helpful in order to improve
9
business acumen and knowledge application skills for the participants. Understanding
the impact that this experiential activity can have on knowledge application and
simulation performance will provide information that is valuable to both designers and
administrators of future simulations. Furthermore, if the types of prompt questions used
in this study; (a) strategy questions for business acumen and (b) financial and/or
accounting questions for knowledge application, turn out to be effective in promoting
participants knowledge and simulation performance, then simulation designers and
business instructors will have a basis from which to make informed decisions when it
comes to designing a better learning experience. On a theoretical level, this study
integrated a number of the precepts of experiential learning, reflective learning, and
online simulations in order to provide information on the factors that promote
participants knowledge. This combination of instructional techniques clearly
demonstrated the impact of this teaching innovation. Past research has shown that a two
standard deviation gain in a pre/post-test of interactive-engagement compared to
traditional instruction may place this innovation on the order of disruptive scale for
education (Hake, 1998).
Research Questions
This research attempted to address the following research questions:
Primary Research Question:
1. Do online business simulations provide an increase in business knowledge or
business acumen for participants, which is on the order of a disruptive
innovation?
Secondary Research Questions:
10
Null Hypotheses
Ho1. There is NO difference across experimental groups for business knowledge
after adjusting for previous simulation experience and autonomy/ work control.
Ho2. There is NO difference across experimental groups for business acumen
after adjusting for previous simulation experience and autonomy/ work control.
Ho3. NO correlation exists between participants business acumen and business
knowledge.
Ho4. Participants who engage in an online business simulation will NOT
demonstrate higher business knowledge after experiencing the business
simulation.
11
13
suddenly become a real threat. While business simulations have been recognized as a
separate pedagogy for instruction, there is little empirical evidence in the research
literature on the magnitude of this innovation when it comes to measuring the increase in
business knowledge or business acumen (Faria et al., 2009, p. 485).
Accordingly, this research contributed to both the management education and
disruptive innovation literature. By investigating the order of magnitude impact that the
business simulation had on participants based on the pre-test and post-test analysis, it was
possible to determine whether online business simulations are simply a sustaining
innovation for educators or truly a disruptive technology that will change how managers
and business leaders learn in the future.
Definition of Terms
The following definitions were used in this study:
Assessment
The art and science of testing individuals to determine what they have learned or,
as is more often the case, what they have not learned (Grne-Yanoff & Weirich, 2010).
Business Simulation
Computer-based role-playing game which makes use of high fidelity simulated
environments and involves decision-making in the research & development (R&D),
marketing, operations, human resources (HR), and finance departments of a company
(Grne-Yanoff & Weirich, 2010).
15
Business Strategy
This is a particular game-play-strategy or group of unique business decisions
which are in complete alignment in order for participants to achieve the highest possible
score on simulation performance (Grne-Yanoff & Weirich, 2010).
Disruptive Innovation
A term coined by Clayton Christensen and used in business and technology
literature to describe innovations which improve a product or service for non-consumers
in ways that the market does not expect. This is typically done by lowering price or
designing for a different set of consumers. Few technologies are intrinsically disruptive
or sustaining in character; however, it is the strategy or business model behind the
technology that it enables, which creates the disruptive impact (Christensen & Raynor,
2003).
Experiential Learning
A learning model which begins with the experience, followed by reflection,
discussion, analysis, and evaluation of the experience (Albert, 1970).
Game Based Learning
A learning method which combines educational content and elements of computer
games (Aldrich, 2009a).
Knowledge Application
The process of selecting appropriate business knowledge suitable to the challenge
at hand, and making connections between selected business knowledge and specific
strategies (Sarin & McDermott, 2003).
16
17
Sustaining Innovation
A sustaining innovation allows for increases in performance at typically higher
costs in the same market without effecting non-consumers from other markets.
Sustaining innovations allow a given technology to continue to improve in their own
market, but they do not directly impact other markets (Christensen & Raynor, 2003).
PROBLEM
Are Online Business Simulations a Sustaining
or Disruptive Innovation?
DEPENDENT VARIABLES
- Business Simulation Score
RQ4
H1-H4
RQ1
in Business
Acumen
in Business
Knowledge
INDEPENDENT VARIABLES
- Online Business Simulation
- Traditional Instruction
RQ3
RQ5
H6
H7
Level of Autonomy in
the Company
H8
RQ2
H3
RQ4
H2-H5
DEMOGRAPHIC VARIABLES
(Impact Performance)
WHY?
RQ6
RQ7
Reflection Activity:
1. Level of Engagement
2. Ability to Apply Learning
3. Peer Instruction
4. Blooms Taxonomy
Simulation
Experience
H1-H2
Autonomy/
W-Control
H8
Industry
Experience
Education
Level
H9
H10
Figure 1. Conceptual Framework. The research questions and hypotheses in this figure
tie out with the dependent, independent, and demographic variables in order to define the
problem in this study.
Characteristics of this experimental design required rigorous management of
experimental variables and conditions by direct control/manipulation or through
randomization (Isaac & Michael, 1997). The independent variables in this study were
participation in an online business simulation or participation in traditional instruction.
There was also a control group which did not participate in either. The primary
dependent variables were the change in business knowledge and change in business
20
acumen based on the pre/post-test for each group. The intent of this study was to make
ten predictions about the relationships among the variables identified in the null
Hypotheses. A summary of all variables is found in the table below (Table 1).
Table 1.
Summary of Variables Used in This Study
Variable
Pedagogical Nature of Training
Change in Business Knowledge
Change in Business Acumen
Knowledge Application Score
Level of Autonomy at Company
Simulation Experience
Area of Expertise
Industry Experience
Level of Education
Type of Variable
Experimental Treatment
Dependent
Dependent
Dependent
Independent Mediating
(Continuous Covariate)
Independent Moderator
(Dichotomous Covariate)
Descriptive
Descriptive
Descriptive
Measurement
Independent
Comp-XM
Comp-XM
Foundation
Work Control (Dwyer)
Level
Ordinal
Ratio
Ratio
Ratio
Ratio
Survey Item
Nominal
Survey Item
Survey Item
Survey Item
Nominal
Ordinal
Ordinal
implications. The study concludes with recommendations for future research based on
the results from the experiment.
22
able to reflect on the experience, (3) the learner must possess and use analytical skills to
conceptualize the experience, and (4) the learner must possess decision-making and
problem solving skills in order to use the new ideas gained from the experience. This
theory offers a fundamentally different view of the learning process from that of the
behavioral theories of learning based on an empirical epistemology or the more implicit
theories of learning that underlie traditional educational methods (Kolb, 1984).
Christensens (1997) The Innovators Dilemma laid the foundational research
which helped to explain why successful companies and institutions often fail to invest
aggressively in disruptive technologies. When applying this theory to higher education,
Christensen (2008) states that the more student-centric classrooms become, the more
demand there will be for new technologies. Christensens research also suggests ways to
identify innovations which are about to disrupt entire industries. The primary method for
identifying these innovations is that they begin as products or services which are much
simpler and cheaper than the existing competition. Also, these innovations generally
promise much lower margins and very little profit. The secondary method for
recognizing disruptive technologies is based on the fact that they are typically
commercialized in new and/ or insignificant markets (Archer, 1999). The third way that
disruptive innovations are recognized is based on the perspective of the leading firms
most profitable customers. Typically, these high-end customers, who are willing to pay
much higher prices, dont want or cant use such inferior products or services. In almost
every case, a disruptive innovation is initially embraced by customers who mean very
little to the industry leaders in that particular market. The great irony of this theory of
disruptive innovation is that those companies which are the best at listening to and
27
serving their most profitable and most successful customers are rarely able to build a case
for investing in disruptive technologies. When they finally do, it is often too late .
Christensens (1997) research highlights the fact that disruptive innovations in the
past were technologically straightforward and did not rely on a specific business model.
In fact, Christensen argues that many of the early disruptions in the computer industry
consisted of off-the-shelf components put together in a product architecture that was
simpler than prior approaches. These innovations typically offered less of what
customers in established markets wanted because they were much lower in product
performance. These disruptive innovations also began in emerging markets and
consequently, offered a different package of attributes which were considered
unimportant to the industry leaders. Christensen (2008) argues that through experiential
learning in a virtual environment, assessmentthe art and science of testing individuals to
determine what they have learnedcan be revolutionized . One of the potential areas for
this revolution in the virtual learning environment is online business simulations.
and as a subset of games (McGrenere, 1996). One of the more complex definitions is
proposed by Szczurek (1982). Szczurek referred to such educational tools as an
instructional method based on a simplified model or representation of a physical or social
reality in which participants compete for certain outcomes according to an established set
of rules or constraints. The competition in simulations is often against some specified
standard, where participants can work as individuals or cooperate as a team.
The origin of the business simulation dates back to 1955. In that year, the Rand
Corporation developed an exercise called Monopologs (Jackson, 1959). Monopologs
required its participants to perform as inventory managers in a simulated Air Force
supply system, thus providing decision-making experience without the risks associated
with the consequences of a wrong decision. The Air Force continued the use of
Monopologs for many years and reported it to be a highly successful training device.
One of the first practical and most successful business simulations for the masses was
Top Management by the American Management Association (AMA) in 1956. It was
used in numerous management seminars (Meier, Newell, & Pazer, 1969). Additionally,
the consulting firm of McKinsey and Company developed the Business Management
Game in 1957 for use in its management seminars (Andlinger, 1958) and the University
of Washington became the first university classroom user of a business game when a
simulation developed by Scheiber was used in a business policy course in 1957 (Watson,
1981).
The increased usage of business simulations can be measured in several different
ways. First, the number of simulations available in the market has increased dramatically
over the last ten years. Second, the number of organizations and journals devoted to
29
business games and simulations has also risen significantly (Faria, 2009). An email
survey of 14,497 business faculty members across all disciplines at American Association
of Collegiate Schools of Business member schools led to only 1,085 respondents. Of
those who responded, 30.6% were current business simulation users, 17.1% were former
simulation users and 52.3% had never used a business simulation in their coursework. In
earlier work, Faria (2004) had estimated that 95% of AACSB schools (The Association to
Advance College Schools of Business) and 86% of all business schools in the United
States were using business simulation games. Surprisingly, business simulations showed
the highest use in business policy and marketing areas.
learning and attitudinal variables. However, there were convergent findings when the
task of designing a simulation before playing it, either re-designing an existing game or
constructing one of their own (Cherryholmes, 1966, p. 7). Based on finding from
Cherryholmes research, a design opportunity will be provided for in this study, where
participants will be able to choose the financial outcomes and ratios that are most
important to their chosen strategy while running the business simulation.
the field of experiential gaming has made little progress. Regarding educational
effectiveness, a variety of suggestions for significantly improving the contribution of
simulations has been given. Examples include clarifying learning objectives (Bredemeier
& Greenblat, 1981), providing more conceptual background on the subject prior to the
simulation activity (Druckman & Robinson, 1998), creating time for reflection on the
events and getting feedback (Mclaughlan & Kircpatrick, 2005), and providing
participants with conceptual maps and graphics that reflect the simulations purpose
(Druckman & Ebner, 2008).
In contrast to just game playing, actual simulation design contributes to analysis
by identifying critical elements (roles, goals, resources, and rules) leading to new
analytical questions (Ebner & Efron, 2005). Attention to the design process remains a
strong focus, as evidenced by the simulation-building exercise featured at the 2007
International Simulation and Gaming Association conference at Nijmegen in the
Netherlands (Durckman & Ebner, 2008). These observations suggest the hypothesis that
simulation designers learn more about the concepts being simulated than do simulation
role-players. Crookal expounded on this hypothesis when he stated the key features of
the design process: (a) Design is concrete you can touch the results; (b) it is creative
you develop an object, and (c) it is involving you develop understanding in a passionate
and intimate way (1995, p.161). When participants have the ability to make changes
within the simulation and design their own experience, the learning about relations
between different concepts goes up significantly and approaches synthesis (Greenblat,
1998). In the case of an online business simulation, the participants in this research study
will have the opportunity to design their own performance metrics for their own company
32
and specific industry of competitors. This interactive engagement with the simulation
may lead to higher levels of motivation, participant involvement and commitment.
Table 2
Instructional Methods Used by U.S. Organizations (Galvin, 2003)
Percentage of Respondents
Often or Always
Never or Seldom
Instructional Method
Instructor-led classroom
Self-study, Web based
Performance Support
Public seminars
Case studies
Role-play
Non-computer-based games, simulations
Self-study, non-computer based
Virtual classroom with instructor
91
44
44
42
40
35
25
23
21
9
6
3
33
9
56
56
58
60
65
75
77
79
91
94
97
Despite the low percentage of business simulations used in 2003, Summers (2004)
outlines three specific advantages to using computer-based simulations in corporate
training which will allow it to grow exponentially in the future. These three advantages
are; (a) specific knowledge, (b) learning on demand and (c) lower costs (Summers, 2004,
p. 226). Summers also argued that many new technology companies are introducing
business simulations to corporations based on the benefits of learning on demand (p.
228). This research poses the question: Will the new technology companies come to
dominate or even replace the old? If so, this would be a case of creative destruction, a
concept posed by Schumpeter (1911/1989).
Alternatively, the new technology companies which introduce these simulations
to corporate training programs may simply increase the number and variety of products.
When the theory of disruptive innovation (Christensen, 1997) is cast on this dilemma, the
outcome depends on whether the new technology is superior and whether the new
technology companies can consolidate the industry (Summers, 2004, p. 232). Online
business simulations have rapidly penetrated business schools; however, the superiority
of such experiential learning methods has not been proven in the corporate training arena.
A simple increase in the number and variety of training methods for management
education would indicate a sustaining innovation for the industry, while a significant
consolidation of the corporate training industry would indicate a disruptive innovation
(Christensen, 2008).
34
Knowledge Application
Application of theory is an on-going issue in higher education (Falkenberg,
Russell and Ricker, 2000). The fact that most of what participants learn is intended for
application to problem situations in real life is indicative of the importance of knowledge
application as a learning objective, especially in the field of management education.
Bloom, Engleheart, Furst, Hill, and Drathwohl (1959) developed a system that classified
learning into six levels. These levels arranged in a hierarchical order to reflect
progressively higher levels of learning. They are, in ascending order: basic knowledge,
comprehension, application, analysis, objective synthesis, and objective evaluation.
Bloom (1956) showed the components of knowledge application in the problemsolving process of answering questions. The process involves six steps, in ascending
order: restructuring and classifying situations, selection of abstraction suitable to problem
type, the use of abstractions to solve a problem, and solution to a problem. This process
shows that in order to solve a problem through knowledge application, there are certain
steps to be followed. Learning how to apply knowledge means learning how to follow
these steps effectively. Bloom (1956) distinguished knowledge application from
knowledge comprehension by saying that a demonstration of comprehension shows that a
student can use the abstraction when its use is specified; while a demonstration of
application shows that he/she will use it correctly, given an appropriate situation in which
no mode of solution is specified.
Bloom (1956) stated that comprehending an abstraction does not certify that the
individual will be able to apply it correctly. Thus, participants need practice in applying
their knowledge to real-world problems in order to make their knowledge more useful for
35
real-world decision making rather than remain inert. In this study, knowledge
application in a business simulation refers to the process of selecting appropriate
business knowledge suitable to the problem at hand, and making connections between
selected business knowledge and business strategies to solve a complex problem. Agyris
and Shon (1974) argued that the only way for organizational effectiveness to increase
over time was through individuals learning from experience. Online business simulations
build upon this concept because the application of learning is based almost completely on
the participants experiences. Agryis and Shon (1974) stated that these types of
experiences provide insights during the learning process and allow for specific
knowledge application tools to be developed leading to consistent learning outcomes .
studies, and textbooks, and had not had enough opportunities to practice applying their
knowledge to real-world problem-solving either in a specific situation or in a simulated
environment. As a result, their knowledge was not sufficiently conditioned to relevant
application conditions, and remained inert.
Training students to apply their knowledge requires very different methods of
instruction than training them to memorize information or understand relationships within
a business context (Reigeluth & Moore, 1999). In business education, the most
commonly used instructional methods are those of linear formats, such as
lecture/textbook format and case analysis. These linear formats can be more efficient
than the experiential learning method for communicating a large number of concepts to a
large number of students. However, these formats do not do enough to encourage
creativity, problem solving, decision making, risk taking, and knowledge application. In
addition, most knowledge that business school students acquire from their lectures and
textbooks is what Anderson (1985) called the declarative knowledge: various business
concepts and principles are taught in a declarative form rather than an experiential form.
Therefore, when students learn business knowledge, they often treat new information as
facts to be learned rather than knowledge to be used. As a result, many business school
students have difficulties in using their knowledge as a tool in their business decision
making (Mandl et al., 1992). Since their knowledge is not based on actual experience, it
often remains inert.
Not all business knowledge needs to be taught in a procedural form. However,
most of those management-related concepts and principles consist of knowledge about
how to do things, which is what Anderson (1985) called the procedural knowledge.
37
application, analysis, and synthesis of knowledge (Anderson & Lawton, 1997). In fact,
researchers in business education have frequently used Blooms Taxonomy as a
framework for guiding their thinking on the areas where simulations are likely to have the
greatest impact on learning (Bloom et al., 1959). Since business simulations require
participants to act in the role of managers, it would seem likely that, if business
simulations excel in any area they would be strong in application (Anderson & Lawton,
2002). However, objective evidence for business simulation effectiveness at these higher
levels of Blooms taxonomy has been lacking.
Measuring the higher levels of Blooms taxonomy of learning objectives has
proven to be a difficult task. A lack of reliable and valid instruments has hindered
attempts to measure the learning occurring at the higher levels of Blooms taxonomy
(Anderson & Lawton, 1995). Thus, while Blooms taxonomy provides a useful
framework for the purpose of establishing learning objectives, the framework has not
been as helpful for assessing student learning, especially in the context of business
simulations. However, Anderson and Lawton (2002) conducted a study that has been
directed exclusively at the efficacy of simulations as a pedagogy for learning about the
application of business concepts. The premise underlying their research was that if
simulation performance does reflect learning associated with analysis and application,
those students who apply the concepts that are critical to the discipline should outperform
those who do not apply the knowledge presented in the course. The results of their study
showed a significant relationship between the application of concepts presented in a basic
marketing course and performance on a marketing simulation game. The greater the
39
performance in a business simulation is not the result of luck or random guesses and that
a business simulation rewards intelligent, planned decision-making practices.
Reflection
The word reflection appears frequently in the literature relating to experiential
learning. Boud et al. (1985) defined reflection as an important human ability in which
people recapture their experience, think about it and evaluate it. Dewey (1910) pointed
out that all genuine education comes through experience and that reflection can assist in
this process. The importance of reflection goes back to Deweys early writing, but there
has been increased interest in researching and using reflective processes in adult teaching
in the last twenty years (Salmon, 2001).
Bruce (2001) stated that reflection is described as contemplating the results of a
given experience within the overall context of the impact on the individual. Boud, Keogh
and Walker (1985) argued that only when this reflective process leads to a significant
change in behavior, can it be called reflective learning. Costa and Garmston (1994)
stated that reflective learning is the ability to mentally wander through a recent personal
experience. This mental process of reflection includes the following; (a) drawing forth
cognitive and emotional information from visual, auditory, kinesthetic, and tactile
sources, (b) linking information to previous learning, (c) comparing the results that were
anticipated and intended with the results that were achieved, (d) searching for effects and
finding connections among causal factors, (e) acting on and processing the information
by analyzing, synthesizing, and evaluating, (f) applying learning to contexts beyond the
41
one in which it was learned and making commitments to plans of action, and (g) the
metacognitive process of thinking about thinking.
Some researchers argue that reflection is essentially an independent activity. An
online business simulation allows for ideal period of reflection in between rounds of
company decisions. This period of reflection or debrief allows participants to ask critical
questions about their company performance and challenge their own basic assumptions
about strategy and execution (Davies, 2003; Foreman, 2004). Other researchers stress the
importance of collaboration with others in terms of the reflection process (Rose, 1992).
Lin (1999) argued that students reflection can be enhanced in a reflective social
discourse, and defined reflective thinking as actively monitoring, evaluating, and
modifying ones thinking and comparing it to both expert models and peers.
Reflection on experience is based upon the metacognitive theory developed by
Flavell (1987), who argued that becoming aware of oneself as a learner allows the student
to reflect, monitor, and revise the process and products of his own learning. The term
metacognition itself emerged from the early work of Flavell who referred to it as
knowledge concerning ones own cognitive process and products or anything related to
them (Flavell, 1976).
J. Biggs (1985) discussed the role of metacognition in learning, utilizing the term
metalearning to define the application of metacognition to student learning. More
particularly, he also defined metalearning as students awareness of their learning and
control over their strategy selection and employment. According to J. Biggs (1988), a
metalearner is one who is aware of their motives, task demands and personal cognitive
resources and exerts control over strategies used. J. Biggs (1988) also stated that these
42
reflections invite thinking about thinking which helps participants to make meaning out
of recent events and experiences. In this way, helping students develop abilities to
monitor and revise their own strategies and uses of resources may enable them to
improve general learning expertise that can be used in a wide variety of settings (Brown,
Campione, & Day, 1981; Scardamalia & Bereiter, 1991). Furthermore, by monitoring
effectiveness of ones own learning and uses of resources, participants may be able to see
the need to pursue a new level of learning and understanding (Bransford & Nitsch, 1978;
Chi, Bassok, Lews, Reimann, & Glaser, 1989). Online business simulations allow for
this self-awareness where participants can escape the nineteenth-century world of
learning by knowing and begin to compete in the twenty-first-century world which
requires the judgement and skill of learning by doing (Aldrich, 2009b, p. 12).
Disruptions in technology are providing more opportunities to improve the simulation
gaming learning experience and a number of pedagogical innovations are beginning to
emerge which will drive the way in which business simulations are used in the future
(Faria et al., 2009, p. 485). The underlying idea of metacognitive reflection is to enable
students to develop the ability to learn how to learn (Vye, Schwartz, Bransford, Barron,
and Zech, 1998).
public activity principally designed to improve future action. However, reflection-inaction is the contemplation of the experience itself, and is supposed to guide further
action for the rest of the experience (Bruce, 2001). Bruce also argued that this type of
reflection involves the necessary gathering and analysis of the data which is very actionoriented in that learners because they must use various tools to convert the data gathered
into useful information (2001). For example, in learning with a business simulation,
reflection-on-action involves stepping back from running the simulation and reflecting on
the different strategies employed after the simulation is over. When the simulation is
over, the student may take time to reflect on what strategies were employed, how they
worked, and their outcome.
In contrast, reflection-in-action, as the term suggests, occurs in the midst of
action. It is described as the consideration of the action being undertaken, and based on a
rapid interpretation of the situation, where rapid decisions are required. Bruce (2001)
stated that this type of reflection involves incorporating the data and information gathered
from the recent experience into immediate action. For example, in learning with a
business simulation, reflection-in-action involves stepping back from playing the game
and reflecting on ones current strategies while a student is still running the simulation.
Thus, in the midst of playing the simulation, a student may take a moment to reflect on
what strategies are being employed, how they are working, and their potential outcome.
Ertmer and Newby (1996) extended Schons (1983, 1987) notions of reflectionon-action and reflection-in-action to include reflection-for-action, which involves
empowering reflective thinking skills to evaluate the results of ones own learning
efforts. Unlike reflection-on-action, this type of reflection directs attention towards a
45
future orientation and considers a more directive global perspective than simply taking
immediate action. Bruce (2001) states that the learner in this realm seeks a mental
correlation between the long-term desired goal and the intermediate action steps required
to meet that goal. For example, in learning with a business simulation, reflection-foraction involves crosschecking current strategies with the desired goal, and then revising
these strategies based on the results for reflection-on-action. Thus, reflection-for-action
is more future-oriented than reflection-on-action. Reflection-on-action is simply stepping
back from a round of the simulation and reflecting on current business results. However,
reflection-for-action goes one step further and involves revising the current strategies
employed in order to achieve the desired goal more effectively.
business simulation performance (Adkins, 2004; Bruce, 2001; Davies, 2003). These
intelligent or open simulations present the user with a complex and dynamic business
problem that a learner cannot master simply by trial and error (Foreman, 2004). The
format of the simulation based on ethics, teamwork, innovation, leadership, motivation,
and conflict management require a period of reflection for the learner to understand what
has just happened (Adkins, 2004; Foreman, 2004).
One research study conducted an experiment which attempted to measure
participants reflection-in-action, reflection-on-action, and reflection-for-action during an
undergraduate business course while the participants were using a business simulation
(Bruce, 2001). The design of this particular experiment focused on capturing the
reflective behaviors demonstrated by the participants. Measurement of these behaviors
involved assessing the quality and quantity of time spent by participants of the business
simulation in reflecting on the different aspects of the experience. Participants involved
in the simulation were asked to record the amount of time and the quality of that time
spent on different types of decision-making activities. This experiment made several
measurements of the identified constructs and data values were collected as the
participants made quarterly marketing decisions within the simulation. The other source
of data for this research study came from the financial and operational reports which are
generated based on each round of the marketing simulation (Bruce, 2001). Three
hypotheses were tested in this study: (a) reflection-in-action time will decrease over the
length of the simulation experience, (b) reflection-on-action will decrease over the
simulation experience, and (c) reflection-for-action will remain constant during the
simulation experience.
47
Bruce (2001) also stated that based on the specific observations from this study,
the teams which outperformed their peers demonstrated improved decision making as
evidenced by decreased time spent in both reflection-in-action and reflection-on-action.
The teams, which did not perform well, were able to improve their financial measures
(i.e., net income) but at a much slower rate. For these teams which did not perform as
well, a pattern of decreased reflection time was not evident. Bruce (2001) argued that
based on these findings there was a strong correlation between reflection and actual
learning with a business simulation. However, further study is required to investigate
what kind of role reflection plays in learning with simulations and how to utilize
participants reflection to make learning with business simulations more effective.
including their emotions during the game. Special emphasis is placed upon situations
which occurred during the simulation that had a significant effect upon the results and
outcome, which the participants failed to detect in their own analysis. Lederman (1992)
stated the primary purpose and rational for debriefing was to (1) help participants gain an
understanding of what has happened, (2) find out what the participants have learned, and
(3) to test these outcomes against the instructors learning objective. Lederman makes it
clear that the purpose of the debrief is NOT to tell the participants that they have learned
what the instructor wanted them to learn. The process is much more discovery based as
the instructor wants to find out what was learned and determine what the implications are
for both the participants and the instructor (Lederman, 1992).
Over forty years ago, Harry (1971) stated that debriefing is a sine qua non for
learning in a simulation, and stated that a post-game discussion is necessary for
maximum effectiveness of any simulation. A few years later Gillespie (1973) observed
that simulations were not self-teaching and needed a good debriefing session to assist
students in reflecting on their behavior and the purpose of the simulation. More recently,
Thatcher (1990) stated that debriefing helps participants to reflect upon their learning and
in doing so come to recognize what they have learned.
In the military, debriefing is known as the after action review, when the players
climb out of their simulated tanks, cockpits, or command tents and discuss the battle
(Presnsky, 2001). The role of such an after-simulation reflective processing is to help the
participants highlight and generalize the various lessons learned so that they can later
apply them to future rounds of the simulation.
49
perception of senior management will influence how they respond. For example, if
senior management views the disruption as a threat, managers tend to overreact and
commit too many resources too quickly. However, if senior management sees the
disruption as an opportunity, they are likely to commit insufficient resources to its
development which does not allow for a successful launch. Gilbert and Bower (2002)
argued that thinking of disruptive innovations in such definitive terms threat or
opportunity is dangerous. The authors claimed that the most successful companies
frame the challenge differently at different times: When resources are being allocated,
senior management should see the disruptive innovation as a threat. However, when the
hard strategic work of discovering and responding to new markets begins, the disruptive
innovation should be treated as an opportunity (Gilbert & Bower, 2002).
Christensen (2003) stated that disruptive technologies are not always disruptive to
customers and the process can occur over a long period of time. These types of
innovations are often difficult to recognize and it is often entirely rational for existing
organizations to completely ignore disruptive innovations. The tendency of managers
and leaders to ignore these disruptions is based on how poorly theses new innovations
compare against existing technologies or products. Also, the niche markets available for
a disruptive innovation are often very small compared to the market for the established
technology (Christensen & Raynor, 2003). In the case of business simulations, it is clear
that many educators have dismissed this innovation as irrelevant (Aldrich, 2009b).
It is common for disruptive technologies to be very subtle when they first enter
the market and show very little threat to competing technologies (Christensen & Raynor,
2003). Examples include digital photography, online banking, IP/Internet telephony,
51
mini-mill steel plants, and distance education. The subtle nature of these technologies
showed them initially penetrating a very different market than the competing firms;
however, over time, the technology became better and better, eventually replacing
existing technologies (Christensen, 1997). In a similar way, it appears that business
simulations have been slowly integrated into some University curriculum and corporate
training programs over time with some disruption; however, it is not clear that business
simulations on their own are a disruptive technology.
For the last 100 years, corporate and university education has been delivered via
the lecture method. This method has been modernized recently with the use of both hard
and soft technologies while still relying heavily on lecture. Archer (1999) states that the
hard aspect of this technology consists of a physical classroom while the soft aspect
of the lecture method consists of the learning model framework devised by each
individual instructor. Various sustaining technologies or innovations have refined and
improved lectures with better presentation software and effective methods of collecting
feedback from participants. Online business simulations are a dramatic contrast to this
standard product in that they eliminate the same place feature of the lecture method,
and in many forms of business simulations the same time feature as well.
Demonstrating one of the key features of a disruptive innovation, these online business
simulations may also eliminate the lecturer.
This function could be readily divided among subject matter experts, instructional
designers, and course section tutors (Archer, Garrison & Anderson, 1999). Clearly,
business simulations are a discontinuity in corporate education. These same authors
stated that discontinuity with previous practice is not the defining feature of disruptive
52
innovations (1999, p. 19). Some sustaining technologies have also been discontinuous;
however, there was no major disruption to the industry. Thus, it is necessary to consider
whether online business simulations meet the description of a disruptive technology
based on the following: (1) the technology results in a product or service that has lower
performance relative to the competition and is less profitable to the producer, (2) the
technology is first commercialized in emerging or insignificant markets, and (3) the
leading institutions most profitable customers do not want these products or services due
to lower performance characteristics and consequently these products or services are
initially embraced by the least profitable customers in a market (Christensen, 1997).
1. Are online business simulations easier to implement and lower cost than the
lecture method, and do they promise lower margins? Yes, it is cheaper, if not
necessarily simpler. It is possible for corporate employees to enroll in a business
simulation without ever stepping foot onto a training campus or entering into an
expensive seminar. Also, the cost to a run a business simulation is only a fraction of
normal seminar instruction. For example, a three credit hour business class will typically
cost around $500 per credit hour for a total of $1,500, while access to a robust business
simulation is less than $250. In terms of the non-monetary reward system, creating and
teaching a business simulation is unlikely to earn a corporate trainer the prestige that the
same amount of time invested in research and publication would garner.
2.
markets? Yes, online business simulations have addressed small groups of learners
peripheral to the central concern of corporate consulting firms and large universities. The
53
biggest foothold market for business simulations has been found in small junior level
colleges and distance education companies, which focus on nontraditional adult learners.
3. Is it the case that corporate consulting firms most profitable customers
generally do not want, and indeed initially cannot use, products based on business
simulations? Yes, few consulting firms profitable customers (i.e., Fortune 100
Companies) initially accessed online business simulations. Kirby and Garrison (1990)
stated that when it comes to corporate training, many corporate policies which allow for
new types of training are enforced by such gatekeepers as human resource executives.
The initial lack of business simulations at the corporate training level may be due to
limited knowledge of human resource managers, rather than these most profitable
customers not wanting them. Whatever the cause of the initial lack of penetration of
business simulations into this segment of the market, the situation is changing very
rapidly with the recent introduction of new online and distance training programs.
Considering the criteria listed above, online business simulations seem to
correspond relatively well to Christensens description of disruptive technology
(Christensen, 1997). The challenge is determining whether these simulations are truly
disruptive innovations by measuring the effect of these business simulations on
participants today. As business simulations gain a foothold market and evolve over time,
there appears to be three areas of emphasis in order for this technology to become truly
disruptive; (a) there must be stronger pedagogical support for business simulations, (b)
the use of reflection and observation within these simulations needs to significantly
increase, and (c) empirical data on the impact of business simulations needs to be
collected as a form of experiential learning (i.e., increase in business acumen and
54
knowledge application skills). This research study was designed to help evaluate these
areas of emphasis and answer the main question: Do online business simulations provide
an increase in business knowledge or business acumen, which is on the order of a
disruptive innovation?
There is strong evidence to support the idea that business simulations have been
slowly moving up the value chain of management education. Davies (2003) argued that
computer simulations provide numerous advantages which can be applied across multiple
industries. Modern day business simulations provide very innovative ways of using
existing technology to foster communication and group collaboration. Improvements in
the delivery of these technologies will allow business simulations to move across
industries by providing significant gains in learning that are lower cost, higher
performance, and eventually more accessible than existing products (Davies, 2003). The
impact of disruptive technological change on an industry can be shown with both the
Product Performance
upper and lower band of performance for the market (see Figure 3).
Market-Years
Figure 3. The Impact of Disruptive Innovation. This graph indicates the difference
between sustaining technologies and disruptive technologies relative to performance at
the high end and low end of the market (Christensen, 1997).
When disruptive technologies enter a market, they not only offer a value
proposition that is attractive to low-end consumers, but these disruptions also allow for
56
Product Performance
cannot dissipate the force of the innovation as it disrupts the entire industry (p. 10).
OneSAF
CCIT
OLIVE
Americas
Army
Spearhead
SIMNET
Market-Years
57
58
simulations are a potential answer based on the game feel and sense of achievement
obtained from such simulations. Under no circumstances should externally imposed
negative consequences follow for nonparticipation. Knowles himself advocated:
Although it acknowledges that adults will respond to some external motivators a better
job, salary increase, and the like-the andragogical model predicates that the more potent
motivators are internal-self-esteem, recognition, better quality of life, greater selfconfidence, self-actualization, and the like (1984, p. 12). Future researchers in
management education should examine situations such as noncredit continuing education
programs where the great majority of the learners who want to be there, are motivated to
learn the material because it is intrinsically useful to them and are inclined to see the
learning activity as inherently valuable and not as a means to an end.
The second strategy for this kind of research focuses on adult status. Rather than
mixing adult and traditional students as Clark (1991), McMasters (1996), and
Strawbridge (1995) do, future management education studies should avoid college
settings if the various groups being compared are partly comprised of traditional college
students. In keeping with Knowles (1980) view that andragogy is for adults, traditional
college students and adult students should not be mixed within a comparison group. If a
college setting is used and traditional students are part of the study, it is very desirable to
have four groups, including an adult andragogy and an adult pedagogy group. For future
management education research, adult should refer to learners who have assumed the
social and culturally-defined roles characteristic of adulthood and who perceive
themselves to be adult. Rachal suggests that if those qualities are not ascertainable,
learners who are age 25 or older would be regarded as adult irrelevant of social
60
62
The sixth strategy for future survey work involves appropriate adult learning
environments. According to Knowles (1984), future andragogy studies involving
management education should make every attempt to insure that both the physical and
psychological environments are congruent as possible for adult learning settings.
Physical logistics are critical to the room arrangement as it allows for break out
sessions with smaller groups and creature comforts where a sense of collaboration can
be nurtured. Two memorable and symbolic gestures from Malcolm Knowles were that
(a) his learners call him by his first name and (b) his commentary on papers was done
with a green (never a red) pen (Knowles & Hulda, 1973). From the researchers
perspective, these environmental qualities are perhaps the most difficult to quantify.
Such atmospheric elements are often the result of some constellation of unique
characteristics of the facilitator, for example, friendliness, confidence, content
knowledge, charisma, empathy, humor, expressiveness, enthusiasm, body language,
fairness, respect, kindness, and understanding (Rachal, 1995). Such characteristics are
the Mona Lisas Smile of the instructor of management education. However, the
andragogy researcher should be able to recognize the right atmosphere when he or she
sees it.
The final survey research method recommended from this literature review
involves random assignment of participants. The reality that a group of management
education learners will fall within the range of college undergraduates to senior
executives involved in corporate training is unlikely. Saying that, the idea of in situ
groups where we find learners with similar backgrounds and experiences are the norm
and should be considered acceptable. Secondly, the issue of whether a single instructor
63
conducts both treatments (Anaemena, 1986) or separate instructors each conduct one
treatment (Clark, 1991) is not nearly as clear. One facilitator for both treatments (i.e.,
traditional corporate seminar versus online business simulation) helps assure that the
personality variables do not confuse the outcome based on the assumption that the
instructor does not present himself as Dr. Jekyll for one group of students and Mr. Hyde
for another group of students (Rachal, 2002). However, a single facilitator for both
treatments may exhibit some bias in favor of one treatment, or be inexperienced in one
treatment simply based on their background and experience. By contrast, the use of two
instructors, one for each treatment, invites inevitable differences in personality, rigor, or
experience which might easily be more important than the instructors teaching methods
when it comes to outcome measures such as achievement and especially satisfaction
(Rachal, 2002). Consequently, it is recommended that only one facilitator be used for
both treatments. Rachal (2002) points out that two facilitators may be used in the
research study, but it is critical that both instructors be matched as closely as possible in
terms of such qualities as background, experience, content knowledge and former
teaching evaluations. Other criteria appropriate to experimental and quasi-experimental
research should also be followed: an adequate number of participants (based on statistical
bias), equal and appropriate treatment duration, informed consent, and comparability of
groups (Donavant, 2009).
64
CHAPTER 3. METHODOLOGY
In this chapter, the research design for this study is explained in detail, including a
full justification for each of its elements. After a review of the studys research
questions, this process will begin with a defense of the choice of a quantitative approach,
including its underlying philosophical assumptions. The experimental approach itself
will be comprehensively described. The sample will next be defined, along with the
process of randomly selecting from the sample, the characteristics of the sample, and the
impact on validity and reliability. Next, this chapter will review the measurement
instruments which will be used to collect data. The data collection process will be
described in detail, followed by a stepwise description of the planned data analysis
procedure. With the research design then fully defined, this chapter will next turn to a
discussion of related validity and reliability concerns. Finally, this chapter will end with
a full consideration of ethical concerns related to the study, including steps which will be
taken to minimize specific risks and protect the participants.
Based on the gaps identified in the existing literature in chapter two, the purpose
of this quantitative, experimentally-based research study is to investigate the use of
online business simulations as a disruptive technology by measuring the change in
participants business knowledge and business acumen compared to traditional corporate
training. This experimental research allowed for a direct comparison between three
groups and identified a causal relationship between online business simulations and
experiential learning as a possible source of disruption. This study therefore connects
experiential learning theory from psychology with the developing field of disruptive
65
innovation, in order to support the business case for significantly higher levels of adult
learning through the use of online business simulations.
In order to examine the effectiveness of engaging adult learners in an online
business simulation, this experimental design study focused on the change in business
knowledge application skills of participants based on exposure to the simulation. This
research study evaluated the gains in business knowledge and business acumen to help
determine whether online business simulations are a sustaining or disruptive innovation
in comparison to more traditional corporate training methods. This study further
attempted to identify and understand the impact of online business simulations on worker
knowledge based on individual backgrounds, work experience, level of education and
tenure with the company. These research priorities were considered with respect to two
potential moderating variables, including level of autonomy in the company and previous
simulation experience. Previous simulation experience was a dichotomous covariate
variable in this studys multivariate analysis of covariance (MANCOVA) analysis, while
perceived autonomy (due to different levels of management) will be controlled for as a
continuous covariate.
Primary Research Question
This forms the studys primary research question:
after the intervention. This not only served to further validate the experimental design,
but also allowed investigation of the relationship between business knowledge and
business acumen. While the primary research question deals with the potential level of
disruption that comes from experiential learning, the secondary research questions are
relational:
Two of the dependent variables considered in the primary research question for this
studychange in business knowledge and change in business acumenwere analyzed
using multivariate analysis of covariance (MANCOVA) analysis. MANCOVA is
defined as an extension of analysis of covariance (ANCOVA) methods which are
designed to cover cases where there is more than one dependent variable and where these
two dependent variables cannot be combined (Swanson & Holton, 2005, p. 135). This is
similar to multiple analysis of variance (MANOVA); however, MANCOVA also allows
for control of the covariates (Cooper, 2008). Covariates are usually the variables in
experimental design which are not controlled by the experiment. However, these
covariates still have an effect on the dependent variables. In this research study, the
covariates include previous simulation experience (dichotomous covariate) and work
control (continuous covariate). The relationship between business knowledge application
skills and online business simulation performance, the second research question, was
investigated using correlation analysis. Finally, the differences between experimental
groups (case study, business simulation, and control) were analyzed using a simple means
67
test. The details of the research design and data analysis procedure will be discussed in
later sections of this chapter. Based on the use of these statistical procedures, the
research hypotheses are formed. Figure 5 displays the form of the linear model which
was used to investigate the studys primary research question.
Y = mX + b
Where:
Figure 5. General Linear Model for Primary Research Question. This equation
describes the general linear model where Y is the vector composed of dependent
variables and X is the design matrix of explanatory variables.
The specific hypotheses examined in this study are presented in Table 3 as both
null and alternative hypotheses. Hypothesis one reflects the primary research question as
to whether online business simulations provide gains in business knowledge which are on
the order of a disruptive innovation.
68
Table 3
Null and Alternative Hypotheses
Null Hypothesis
There is NO difference across
experimental groups for business
knowledge after adjusting for previous
simulation experience and autonomy/
work control.
H2 There is NO difference across
experimental groups for business
acumen after adjusting for previous
simulation experience and
autonomy/work control.
H3 NO correlation exists between
participants business acumen and
business knowledge.
H4 Participants who engage in an online
business simulation will NOT
demonstrate higher business
knowledge after experiencing the
online business simulation.
H5 Participants who engage in an online
business simulation will NOT
demonstrate higher business acumen
after experiencing online the business
simulation.
H6 There is NO difference between the
simulation group, the case study
group, and the control group based on
their business knowledge.
H7 There is NO difference between the
simulation group, the case study
group, and the control group based on
their business acumen.
H8 NO correlation exists between
participants business simulation
performance and their level of
autonomy in the company.
H9 NO correlation exists between
participants business simulation
performance and their years of
industry experience.
H10 NO correlation exists between
participants business simulation
performance and their level of
education.
H1
69
Alternative Hypothesis
There is a difference across experimental
groups for business knowledge after
adjusting for previous simulation
experience and autonomy/ work control.
There is a difference across experimental
groups for business acumen after
adjusting for previous simulation
experience and autonomy/work control.
A correlation exists between participants
business acumen and business
knowledge.
Participants who engage in an online
business simulation will demonstrate
higher business knowledge after
experiencing the online business
simulation.
Participants who engage in an online
business simulation will demonstrate
higher business acumen after
experiencing the online business
simulation.
There is a difference between the
simulation group, the case study group,
and the control group based on their
business knowledge.
There is a difference between the
simulation group, the case study group,
and the control group based on their
business acumen.
A correlation exists between participants
business simulation performance and
their level of autonomy/work control in
the company.
A correlation exists between participants
business simulation performance and
their years of industry experience.
A correlation exists between participants
business simulation performance and
their level of education.
Research Design
In Chapter 2, a need was identified for empirical research into the potential
disruptive nature of online business simulations when compared with more traditional
instruction in the field of management education (Segon & Booth, 2009). Based on this
research, this study employed an experimental design and pre/post-experiment survey.
Justification of Methodology
This study utilized a quantitative experimental research methodology.
Quantitative techniques are particularly strong at studying large groups of people and
making generalizations from the ample being studied to broader groups beyond the
sample (Swanson & Holton, 2005). Cohen (1994) asserted that quantitative techniques
must be used sensibly and be heavily informed by judgment (p. 1002). A quantitative
approach is typically employed when the research goal is to study relationships between
established variables in an objective manner (Creswell, 2009). This is the case for this
research, which will examine relationships between variables. Research has shown that
participants perceive online business simulations as an effective training tool (Burns et
al., 1990; Faria, 2004; Gosen et al., 2004). However, all of these studies relied upon
perceptions and self-reports of learning rather than more objective measures (Anderson &
Lawton, 2009, p. 211). There is a dearth of studies employing experimental designs with
control groups and almost no experimental studies exist that compare learning outcomes
under alternative pedagogies.
70
Wolfe (1990) identified this problem over 20 years ago, yet the gap still exists.
Researchers continue to use self-assessments rather than more suitable tools because they
are much easier to employ. As a consequence, studies on the educational merits of
simulations often are measuring the affective domain, not the cognitive domain they
purport to measure (Anderson & Lawton, 2009, p. 197). This has led some researchers to
believe that using simulations is a powerful and disruptive form of learning because it is
taking place at the higher levels of Blooms taxonomy (Smith, 2006). While continued
exploration of online business simulations as an abstract phenomenon is needed, the goal
of this research work is not explorative, but rather consequential. Quantitative
approaches are most appropriate when the target population is large and there is a desire
for generalization beyond the studys sample, the ability to predict future results, or the
capacity to credibly establish causation (Swanson & Holton, 2005). Quantitative
techniques are particular well suited for experimental methods where the environment is
well controlled against outside influences. While this limits generalizability, it also
maximizes internal validity (Creswell, 2009).
71
Population
Random
Sample
Treatment
Measure
Traditional Instruction
Control Group
Change in
Business Knowledge
(Theory)
Change in
Business Acumen
(Application)
Figure 6. Research Design Schematic. This schematic identifies the random sampling
from a company population for three experimental groups (traditional instruction, online
business simulation instruction, and the control group.
72
execution. On the third day, each participant was allowed to take post-test assessment in
order to evaluate the change in business knowledge and business acumen.
The Online Business Simulation group was introduced to the simulation and then
broken into smaller teams of 5 members each. These individual teams were allowed
several rehearsal rounds of the simulation where they could practice making decisions
for the company. Over the next two days, these teams competed with each other for
market share and highest level of stock price as they implemented specific strategies and
tactics that they had decided upon as a team. Each team ran a total of 5 rounds,
simulating 5 years of running a company. In between each round, a debrief meeting was
held where participants could ask questions and learn more about the competitive nature
of the industry. The first two days of training allowed participants to make decisions in
five different rounds, simulating five years of running the company. On the third day,
each participant in the online business simulation training group was allowed to take the
73
Prompt Questions
Strategy Only
Traditional
Instruction
HBS Review
Article
Discussion
Case Study
Discussion
Prompt Questions
Strategy & Application
Business
Simulation
Pre
- Simulation
Survey &
Knowledge Test
Practice
Rounds
Round 3
Round 4
Round 5
Post -Simulation
Survey &
Knowledge Test
No Reflection
Activity
A pre/post -test control group design will be used as the choice of research design for this study.
Participants will be randomly assigned to the Case Study, Business Simulation, and Control Group
Figure 7. Overview of the Research Design. A pre/post-test control group was used as the choice of experimental research
design for the study. Participants were randomly assigned to the Traditional Instruction (Case Study) Group, the Online Business
Simulation Group, and the Control Group.
74
The Control Group did not have any intervention and was simply allowed to take
the pre-test on the first day of training and the assessment on the last day of training.
This research design is unique in the business simulation literature. A significant
amount of quantitative and empirical research was collected which can be analyzed for
both comparisons across experimental groups and measured to determine the significance
in the change of business knowledge and business acumen based on the type of training
given in a corporate setting. Previously to this study, all empirical research had relied on
the participants perception of their learning experience (Williams, 2011). Utilizing an
experimental method which allowed for the clear establishment of increased learning
through different pedagogies, this research design clearly demonstrated the gains in
business knowledge and business acumen based on modern day corporate training. This
empirically based research study is a significant contribution to the understanding of the
magnitude of disruption that online business simulations can have on management
education today.
Sample
The most common method to calculate appropriate sample size is based on
calculating the necessary sample in order to achieve a desired power, effect size, and
level of statistical significance (Lenth, 2001). Power can be simply described as the
ability of a statistical test to detect a particular effect size. Lewis-Beck (1994) also
describes power as the probability of correctly rejecting a false null hypothesis given that
an effect size is present in the sample (p. 1080). Effect size is defined as an estimate of
the relationship between two variables. It can take a number of forms, but it is frequently
75
expressed as a standardized ratio between the magnitude of an observed effect and the
standard deviation (Fields, 2009). Selecting the appropriate effect size of interest prior to
conducting a scientific study can be challenging. The best method is to use prior research
as a guide (Lenth, 2001). Using the G3 Power software tool, (Faul et al., 2009) a power
analysis showed that a sample as small as 14 for three treatment groups will be able to
detect a MANCOVA main effect size of one standard deviation, with a power of 80%
and a statistical significance of 95%. At 90% power, a sample size of 17 is required
while 80% power is commonly accepted in social science research (Fields, 2009). These
calculations indicate that this research study with 20 to 23 participants per experimental
group was sufficient. It was determined that even if the effect size had fallen as low as
0.5 standard deviations, a sample size of 20 would be able to reject the null hypothesis at
95% significance with 90% power.
The population of interest for this study was managers and employees for a large
semiconductor manufacturing company. This experiment sought out a specific causeand-effect relationship between the type of pedagogical instruction offered and resulting
change in business knowledge and business acumen from the training. A more than two
standard deviation increase in the mean scores indicates a source of disruptive
innovation. However, this study is careful to state that such results are not able to make
statistical generalizations beyond the sample (Cooper & Schindler, 2008). The primary
goal of this study was to investigate the use of online business simulations as a disruptive
technology by measuring the change in business knowledge and business acumen
compared to traditional corporate training. Focusing on this critical population insured
that any conclusions, while limited in generalizability, will have the greatest possible
76
value. The focus on a large manufacturing company with random sampling allowed for a
population consisting of employed professionals, knowledge workers, and managers who
are typical of corporations seeking quality operations training today.
In order to access the targeted population, this researcher contacted the Vice
President of Operations for a large semiconductor company and requested permission to
provide corporate training for approximately 75 members of the senior management
team. Preliminary approval for this research study was received from the company in
advance. Once approval was received through the IRB approval process, a meeting was
held with senior management at the company to discuss the purpose and structure of the
research study. The corporate headquarters for this semiconductor company were
provided with an executive summary of the results as well as a presentation of
information and resources that could aid strategic plans for training senior management in
the future. Based on an initial target of 25 participants per group, a total of 65
participants were randomly selected and found available time to participate in the study.
This total of 65 is out of a population of 720 employees and managers.
This study employed a randomized cluster sampling technique in order to select
participants from the organization. Cluster sampling may result in less statistical
precision, but it is more economical than simple random sampling for large populations
(Cooper & Schindler, 2008, p. 392). Within the population of interest for this study,
departments within the semiconductor company form natural clusters (e.g., research &
development, production, finance, and marketing). Subgroups were created based on the
background and experience of each participant and participants were then randomly
selected for one of three experimental groups. Based on the parameters for each
77
participant it was possible to ensure that all three experimental groups were similar in
background, experience, and autonomy. Employing three groups with 20 to 23
participants per group was sufficient to achieve the desired level of statistical power.
The population of clusters for this study is all of the employees and managers of
the semiconductor company. According to its business profile, there are 14,307
personnel employed by this company around the world. From this population, a list of
potential participants were be randomly generated by the human resource department at
one manufacturing site. Random assignment of these participants to each experimental
group was necessary in order to insure the sample group was as homogenous as possible,
and therefore comparable with regard to the studys dependent variables. Issac and
Michael (1997) stated that the best solution for true experimental design is random
assignment of subjects to groups and random assignment of experimental treatments to
groups. Utilizing a randomization technique with a control group forms the classic R-OX-O true experimental design, maximizing internal validity and relative efficiency
(Creswell, 2009). All three groups were administered both a pre-test and post-test, but
the treatment was only provided to two of experimental groups (e.g., traditional
instruction and online business simulation).
Setting
A number of studies exist which have looked at the educational effectiveness of
business simulations in corporate settings (Burns, Gentry, & Wolfe, 1990; Faria, 2001;
Gosen & Washbush, 2004). However, none of these studies attempted to empirically
measure the change in business acumen and knowledge application skills of the
78
participants across different experimental groups (Sidor, 2008). This study was unique in
that it investigated the effects of engaging participants in different pedagogies of learning
and different methods of reflection as participants step back from the case study or
business simulation and consider what they have just learned in order to apply this new
knowledge to the post-test assessment. When it comes to investigating the magnitude of
learning based on different pedagogies as a possible disruptive innovation to management
education, this study was the first in its field.
While this experimental study added to the body of knowledge on experiential
learning, online business simulations, and disruptive innovation, there were also benefits
for the semiconductor company and participants involved in this study. The use of online
business simulations in the corporate training and management education is quickly
growing (Williams, 2011). However, this type of training is typically very expensive.
Fortune 500 companies are willing to pay as much as $50,000 per training session for top
managers and employees to attend. In the case of this research study, the semiconductor
company paid a small fraction of the normal cost. Participants in this study gained
significant insights into their business and how to better manage their manufacturing
facilities. This study confirmed that adults working in a corporate environment prefer
engagement and the opportunity to experience concrete action as the basis for learning,
rather than being lectured in a classroom (Segon & Booth, 2009). The senior
management team of the semiconductor company perceived this as a great learning
opportunity for their employees and the company as a whole.
79
80
assessment tool has been used by over 50,000 MBA students and is used as an instrument
for online computer usage, locus of control, and literacy (Prince, 2004).
With the exception of two demographic factors, all of the variables included in
this study were measured by survey instruments designed for their specific uses.
Previous simulation experience was considered a potential moderator in the casual
relationship between business knowledge and business acumen. Simulation experience
was measured by a single question on the post-experiment survey, with a dichotomous
choice: (1) yes or (2) no. In addition, data was collected on industry experience, purely
as a descriptive variable to help understand the composition of the sample. Industry
experience was divided into ten ordinal ranked categories, each representing five years:
81
(1) 1 to 4, (2) 5 to 9, (3) 10 to 14, (4) 15 to 19, (5) 20 to 24, (6) 25 to 30, (7) 31 to 34, (8)
35 to 40, (9) 41 to 44, and (10) 45 years or more. These variables and survey items are
common and have been used in previous studies dealing with simulations (Segon &
Booth, 2009; Williams, 2011).
Statement and to the Balance Sheet. Specific questions from the Board of Directors are
asked at the end of each year, which gives an opportunity for the participant to study the
financial statement and provide answers based on their knowledge (i.e., knowledge
application skills). The simulation also tracks the overall quality of the decisions
throughout the simulation which completes a Balanced Scorecard at the end of each year.
It is possible for a participant to score well on the balanced scorecard once or twice;
however, it is very difficult to maintain a good score throughout the duration of the
simulation without adhering to solid business strategy (i.e., business acumen).
Field Testing
Because this studys survey instruments were derived from previously validated
items, a distinct field test was not included in this research study.
(Smith, 2011). The unique two-part Comp-XM examination process uses a business
simulation with a built-in balanced scorecard, followed by a series of board query
questions that are specific to the results of the answers provided in the prior simulation
round. The overall score for performance on the simulation is based on four components;
financial, internal business process, customer, learning & growth of the company. Each
of these areas are evaluated and totaled for a final score (see Figure 9).
85
Figure 9. Foundation Business Simulation Performance Score. This table lists the
four primary areas of the balanced scorecard based on the participants decisions in
finance, internal business process, customer, and learning/growth for the workforce.
Data Collection
This research was conducted at a large semiconductor company which has
multiple manufacturing facilities around the world. One site was selected with a total of
720 employees and managers and randomly selected participants met in a conference
room at the facility where corporate training takes place. Data was collected from
participants in two ways. The first was through management decision software based on
Variables
Characteristics of this experimental design required rigorous management of
experimental variables and conditions by direct control/manipulation or through
randomization (Isaac & Michael, 1997). The independent variables are participation in
an online business simulation or participation in traditional instruction. There was also a
control group which did not participate in either. The dependent variables are the change
in business knowledge and change in business acumen based on the pre/post-test for each
group. The level of autonomy of each participant is an independent mediating variable
which helps to identify employees from managers with more responsibility. The intent of
this study was to make 10 predictions about the relationships among the variables
identified in the null Hypotheses. A summary of all variables is found in Table 1 (see
page 21).
results from Comp-XM which measure both business knowledge and business acumen.
Each survey was later marked with the number identifying individual assessment results,
along with the type of pedagogical instruction that was given over the three day period.
These survey results were then be transferred into a secure database manually.
Data Analysis
This studys primary research question and objective was focused on two
dependent variables: business knowledge and business acumen. These dependent
87
acumen scores were entered. For Dwyer and Gansters (1991) measure on the level of
autonomy or work control for each participant, a simple arithmetic mean is calculated.
The final performance simulation score based on the balanced scorecard for the virtual
company will also be entered for the online business simulation experimental group.
Data analysis was conducted with version 20.0 of the SPSS software. After the
data was tabulated, the data analysis process began with a thorough graphical analysis.
88
This allowed for the quick identification of any spurious data, yield information about the
distribution of the data set, and gave some guidance as to how the investigation should
proceed (Norusis, 2008). The next phase of data analysis was descriptive analysis,
starting with the calculation of univariate statistics for each variable. The distribution of
each continuous variable was then examined by calculating skewnees and kurtosis. This
analysis also served to highlight any outliers required further investigation (Norusis,
2008). Cronbachs alpha values were calculated for all scaled factors in the survey as a
check of acceptable internal consistency and reliability.
Analysis of variance methods, such as MANCOVA, are often considered an
extension of regression techniques which can be conceptualized as a multiple linear
regression with multiple dependent variables. Consequently, MANCOVA shares many
of the same assumptions as other general linear models (Norusis, 2008). These
assumptions were examined for the following; (1) independence of observations, (2)
random sampling from the population, (3) independence of the covariates and treatment
effects, (4) multivariate normality, (5) homogeneity of covariance matrices, and (6)
homogeneity of covariate regression slopes (Fields, 2009).
Independence of observations, random sampling, and independence of the
covariates across treatments are a function of the experimental procedure (Norusis,
2008). These assumptions can be ensured by random selection of experimental groups
and by utilizing a meticulously consistent experimental process. To check the
assumption of the independence of covariates across treatments, a simple analysis of
variance was run across the experimental groups in which the covariates act as the
dependent outcome variables. Because the main effects of these ANOVA analysis were
89
not significant, it was assumed that the covariates are roughly equal across the
experimental groups (Fields, 2009).
Univariate normality was verified through normality plots of the variables and by
conducting normality tests (e.g., Kolmogorov-Smirnov and Shaprio-Wilk). Fortunately,
analysis of variance is not very sensitive to normality. Because the data are not
extremely non-normally distributed and the sample sizes were not particularly small,
more than 17 each, it was considered safe to proceed with the analysis (Norisus, 2008).
The assumption of homogeneity of covariance matrices means that not only
should the variances between each experimental group be roughly equal for each
dependent variable, but also that the correlation between dependent variables is equal
from group to group. To check this assumption, first Levenes test was conducted on
each of the dependent variables in order to insure there is univariate equality of variances
between groups. Second, Boxs test was used to compare the variance-covariance
matrices for each group (Fields, 2009).
Finally, the assumption of homogeneity of covariate regression slopes was
investigated. MANCOVA relies on the assumption that the relationship between each
dependent variable and the covariates is consistent across groups. The assumption was
tested by calculating the significance of the interaction effects between the covariates and
the primary independent variable. If this interaction is statistically significant, the
assumption of homogeneity of covariate regression slopes is suspect (Fields, 2009).
Once the assumptions were verified, the MANCOVA analysis began. This
research study was tested by examining the significance of the F statistics for the
independent variable and for each covariate in the ANOVA tables. A significance level
90
below the critical value of 0.05 allowed for the null hypothesis to be rejected.
Confidence intervals and contrast matrices were also built to further evaluate statistical
results. The original research method allowed for other statistical methods to be
employed. If the assumptions of MANCOVA were substantially violated, techniques to
transform the data were available (Fields, 2009). In addition, robust methods which are
less susceptible to the requirements of traditional MANCOVA analysis, such as the
Munzel-Brunner method, were also available (Wilcox, 2005).
The second research question involving the degree of the relationship between
knowledge application skills and online business simulation performance was considered
using correlation analysis. The magnitude and direction of the Pearson correlation
coefficients served to describe the relationship between the variables. If the Pearson
correlation coefficients were found to be statistically significant, with critical values
below 0.05, then the null hypothesis was rejected.
The third research question involved the change in knowledge application skills
when comparing traditional corporate training with the use of online business
simulations. This investigation was intended to provide additional evidence that the
change in business knowledge and business acumen over the same period of time is
significantly different based on the type of pedagogical training that is given to adult
learners. This portion of the analysis was accomplished with simple comparisons of
means and T-tests. Business knowledge and business acumen levels were compared
across the three different experimental groups. Statistically significant results, with a
critical value below 0.05, indicated manipulation of the independent treatment variable
was related to changes in participants overall level of business knowledge and acumen.
91
was addressed by randomizing which participants were selected for the control and
experimental groups (Creswell, 2009). The use of a classic R-O-X-O experimental
design with random assignment of groups is an effective method to control the risks of
history, maturation, selection, and statistical regression (Cooper & Schindler, 2008).
Treatment diffusion occurs when participants in the control and treatment groups
speak with one another. While it was impossible to prevent participants from
communicating, this risk was minimized by conducting the experiments over a short time
frame (3 days), and by asking the participants to refrain from discussing their experiences
until the results are revealed. Compensatory demoralization or rivalry is risked when the
treatment and control groups receive different benefits. After the study was complete, the
researcher returned to the participants classrooms to reveal and discuss the results,
reestablishing equality. Pre-testing was not a threat to this study since participants were
allowed to participate in the experiment once, and the survey was administered at the end
of the experimental activity (Creswell, 2009).
The final threat to internal validity cited by Creswell (2009) is instrumentation.
The Capsim Comp-XM instrument was used to assess business knowledge and business
acumen. Quantitative methods were confined to only that which is measurable and
controllable (Malterud, 2001), which has been an area of much debate in business
simulation literature (Sidor, 2008). Engineered quantitative approaches may strip context
and meaning from phenomena in an attempt to study them objectively. It is important,
particularly with abstract subject matter, to insure the research design does not impose
meaning on observations rather than simply observe them (Swanson, 2005). Steps which
93
have been taken by the developers of the measurement instruments which were used in
this research to insure validity and reliability were previously discussed in this chapter.
The second class of validity threats is dangers to external validity. Laboratory
experiments maximize internal validity because they have the best chance of controlling
extraneous variables. However, the price of maximum internal validity is
generalizability (Echambadi et al., 2006). The artificiality of experimental settings risks
outcomes which may not exist in natural surroundings (Cooper & Schindler, 2008). The
simulated workplace which is being created for the purposes of the experiment is very
similar to the actual workplace for these managers and employees in the semiconductor
industry. Any claims made about the results of the research should be restricted to
groups which have similar characteristics to the experimental sample (Creswell, 2009).
Since the researcher had to directly interact with the participants during the experimental
activity in order to give instructions, another risk to external validity is experimenter
effects. Conscious or unconscious acts by the researcher could influence results (Gay &
Airasian, 2002). In order to minimize this risk, instructions were carefully scripted and
contact with the participants was limited.
Reliability is a matter of consistency or repeatability of measurements (Creswell,
2009). Low reliability can dramatically impact a researchers ability to uncover theorized
relationships between variables. The most commonly used indicator of reliability is
Cronbachs alpha value, which indicates the degree of internal consistency between
measurements. With survey research, internal consistency means items draw on the same
construct and have homogeneous results. In social science, Cronbach alpha values
exceeding 0.70 are generally accepted as sufficient for research (Norusis, 2008). The
94
Capsim instruments used in this research study were analyzed using psychometric
measures which the computing environment has used for many years (Prince, 2004).
Kay (1990) developed a 10-item instrument to assess computer locus of control that was
used by Prince to evaluate the Capstone business simulation. In both studies, the
Chronbach alpha was above 0.8 with Kays analysis reporting an alpha = 0.87 and
Princes study reporting an alpha = 0.88 (Prince, 2004).
There are several sources of reliability and validity threats specific to surveys
which must be considered. Survey methods rely on a communications process to capture
participant feelings and attitudes. A breakdown in this process could occur due to the
interviewer, the measurement instrument, or the participant (Creswell, 2009). With selfadministered surveys, such as in this study, the potential influence of the interviewer is
minimized. The validity and reliability of each of the measurement instruments which
was used by this study has been previously established. Error generated by the
participant, however, can come from either a lack of willingness to respond or from
mistakes in how a participant responds (Cooper & Schindler, 2008).
The reliability and validity of survey instruments is dependent on participants
who are willing to provide accurate and complete responses to the questions posed to
them. This can be a challenge, particularly for certain sensitive subject matters such as
religion, finances, or health. Participants may choose not to respond to a particular
questionnaire item at all, creating non-response bias (Cooper & Schindler, 2008). To
moderate this risk, the survey which was used in this study intentionally avoided such
sensitive topics as income, even though this could be theorized as a potential moderator
variable. In addition, the participants were assured of the strict confidentiality of their
95
responses in order to help allay any fears. Steps were taken to handle any non-responses
which may occur were outlined in the data analysis section of this chapter.
Unfortunately, surveys similar to Dwyer and Gansters (1991) work control which
utilize Likert-type scales do not allow for interpretation and offer very limited richness of
data in comparison to interviews or observations in natural settings (Creswell, 2009).
This makes errors from participants, or response bias, more difficult to detect. Response
bias can result from lack of personal experience with the subject matter, poor
instructions, or even social desirability bias, where participants modify responses in a
deliberate attempt to provide more socially acceptable results (Cooper & Schindler,
2008). In this study, the participants were asked to report on their experiences during the
experimental activity, so there is little risk of response bias due to lack of personal
knowledge. Participants were all educated professionals, but instructions were carefully
crafted for both the experiment and the survey in order to minimize any confusion. When
any confusion came up, the researcher was careful to share the same clarifying instruction
with all participants. Social desirability bias was controlled in part because the
participants did not initially know the true purposes of the study.
In addition, assurances
of strict confidentiality should diminish participants desires to provide less than truthful
responses in order to preserve their reputation or standing.
In this study, the reliability of the experimental productivity measurements also
has to be considered. Common method variance is variation that can be attributed to the
measurement method instead of the experimental treatment (Creswell, 2009).
Researchers must be concerned with both the stability and equivalence of the
measurement techniques. Stability refers to the ability to secure consistent results when
96
measuring the same person repeatedly. Equivalence has to do with error introduced by
different observers or different samples (Cooper & Schindler, 2008).
As experimental data was collected, several steps were necessary in order to
maximize reliability. To help ensure measurement stability, instructions were carefully
scripted and all interaction with the researcher were limited. This ensured consistency of
delivery from group to group and minimizes potential sources of bias. Uniformity
between the groups was strictly maintained in terms of experimental setting, timing, and
even tone. The experimental activity during the online business simulation involved
group work, so factors such as competence may have affected productivity and
performance.
Ethical Considerations
This study did not contain any inherent level of risks above the minimum for
participants. Ethical standards were maintained through the organizations ethics
committee and normal University IRB approval. Permissions to access the targeted
participants were sought by respective human resource and ethics committees, which
function as corporate versions of institutional research boards. All participants were
informed that their participation in the study was voluntary and that they could opt out
any time. Participants were asked to indicate their informed consent at the beginning of
the survey and pre-test. Participants were also informed that any results published or
shared with their employers would only include aggregated data that would not permit
the identification of specific individuals, workgroups, or organizations.
97
One item of particular concern that may arise from a research design with
different experimental groups is the use of deception, or at least the temporary
withholding of information. While some scholars believe deception should never occur
during a research study, others suggest it may be necessary in order to prevent bias or to
protect confidential information. Based on the research objective in this research study,
there was no need to disguise or conceal the true purpose of the study. All three
experimental groups were trained separately and the goodwill of the participants was
maintained at all times.
Another item of potential concern in this research design was the use of a sample
comprised of full-time employees and managers who were being assessed on strategy and
execution skills that they use in their day to day operations of running a semiconductor
manufacturing facility. These employees may be considered a vulnerable population due
to the fact that management has a position of authority over them and because they fear
that lack of participation may have some impact on their employment (Cooper &
Schindler, 2008). However, to help relieve these concerns, employees and managers at
the site were clearly informed that participation was completely voluntary and would not
impact their employment.
Participant confidentiality was protected through such measures as not collecting
personal identification information, such as names, but rather identifying each respondent
with an assigned code. All responses were further matched to the participant code only.
All data was collected by the online simulation database and solely accessed by this
researcher. Data will be electronically safeguarded with a username and password. Data
will be stored, through password protection, for the requisite seven years, at which time
98
they will be destroyed by electronic wiping of the hard drive and external devices.
Approval letters for both the simulation and semiconductor companies has been attached
in the appendix. While risks were minimal, they were balanced through research gains
and by fully debriefing participants after the studys completion.
99
CHAPTER 4. RESULTS
This chapter presents the results of the statistical analysis employed in this study.
The purpose of this experimentally-based research study was to investigate the use of
online business simulations as a disruptive technology by measuring the change in
participants business knowledge and business acumen compared to traditional corporate
training. The primary research question focuses on whether online business simulations
provide gains in business knowledge or business acumen, which is on the order of a
disruptive innovation. Initially, the training provided to the participants is discussed as
well as the online survey which was used to gather participant demographics. Next, the
reliability of individual variables used in the research study is analysed. After these
preliminary analyses are discussed, this chapter next turns to the testing of the alternative
hypotheses of this research study. Hypotheses were tested by examining the significance
of the F statistics for the independent variable and for each covariate in the ANOVA
tables. A significance level below the critical value of 0.05 allowed for the null
hypothesis to be rejected. Confidence intervals and contrast matrices were also built to
further evaluate the statistical results.
Supporting research questions involving the degree of the relationship between
knowledge application skills and online business simulation performance was evaluated
using Pearson correlation analysis. If the Pearson correlation coefficients were found to
be statistically significant, with critical values below 0.05, then the null hypothesis was
rejected. The hypothesis testing related to each of the supporting research questions is
presented and this chapter ends with a summary of the results.
100
business simulation training, there are four major decision areas which tie out with the
actual departments within the semiconductor company.
Participant Demographics
An online survey was used to capture information on the background, experience
and education for each participant just prior to the training. The information from these
survey items was used as descriptive variables in the analysis and included the following;
gender, level of education, years of industry experience, area of expertise, prior
simulation experience and level of autonomy or work control within the company. For
the demographic item of gender, 86.2% of the participants were male (N = 56), 58.5%
had earned a Bachelors degree (N = 38), 50.8% had more than 15 years of industry
experience (N = 33), 43.1% were in the production department (N = 28), 50.8% had no
prior simulation experience (N = 33), and 52.3% self-reported a moderate level of
autonomy in their current employment position (N = 34). This demographic information
tied out with the overall population having a majority of male employees and managers
with 15-20 years of industry experience who are involved in manufacturing operations.
The mean number of level of education was 16.3 years (including high school)
while the mean level of industry experience for participants was 16.5 years. Out of the
four major departments located at the manufacturing facility, 43.1% were from
operations (N = 28), 10.8% were from human resources (N = 7), 16.9% were from
marketing (N = 11), and 29.2% were from finance (N = 19). The continuous covariate,
level of autonomy, was based on a survey given to all participants using an instrument
developed by Dwyer and Ganster (1991). This survey instrument consists of twenty-two
102
Likert-scaled items where each participant is asked to indicate their level of individual
responsibility within the organization. On a scale of one to five the mean was 3.12 with a
standard deviation of 0.839 (N = 65) as shown in Figure 10.
\
Figure 10. Demographic Variable: Level of Autonomy. This normality plot shows the
level of autonomy for all participants in their current position within the company. This
demographic variable was treated as a continuous covariate in the MANCOVA analysis.
Normality Testing
Several different statistical tests were performed to determine the normality of the
aggregate data set. First, Shapiro-Wilk statistics were generated on each of the variable
scales. Next, histograms with superimposed normal curves and Q-Q plots were generated
for each scale. Skewness and kurtosis scores were also generated for each scale. Table 4
103
shows descriptive statistics for each of the demographic variables: years of education,
years of industry experience, and autonomy.
Table 4
Descriptive Statistics for Survey Items (N = 65)
Valid
Missing
Mean
Std. Deviation
Skewness
Kurtosis
Minimum
Maximum
Years of
Education
Years of Industry
Experience
Prior Simulation
Experience
Autonomy
65
0
16.25
1.36
1.20
3.89
14.00
22.00
65
0
16.49
6.25
0.38
-0.07
6.00
35.00
65
0
0.49
0.50
0.03
-2.06
0.00
1.00
65
0
3.12
0.84
-0.40
0.64
1.00
5.00
The histogram and Q-Q plot examination indicated a normal distribution for most
variables. This visual examination was confirmed by the skewness scores for these
variables: years of education = 1.20, years of industry experience = 0.38, prior
simulation experience = 0.03, and autonomy =-0.40. Both a visual examination of the
histograms and the generated kurtosis scores reveal minimal kurtosis for years of industry
experience (-0.07) but significant kurtosis for years of education (3.89).
A similar statistical test was performed for the independent variable, type of
pedagogical training, and for all dependent variables; change in business knowledge,
change in business acumen, and final knowledge application score. The histogram and
Q-Q plot examination indicated a normal distribution for all dependent variables. This
visual examination was confirmed by the skewness scores: type of training = 0.09,
104
change in business knowledge = 1.08, change in business acumen = 1.83, and knowledge
application = -0.96 as shown in Table 5. Both a visual examination of the histograms and
the generated kurtosis scores reveal minimal kurtosis for change in business knowledge
(0.00), but significant kurtosis for change in business acumen (3.99).
Table 5
Descriptive Statistics for Dependent Variables (N = 65)
Pedagogical
Training
N
Valid
Missing
Mean
Std. Deviation
Skewness
Kurtosis
Minimum
Maximum
65
0
1.95
0.82
0.09
-1.50
1.00
3.00
Change in
Business
Knowledge
65
0
18.15
26.50
1.08
0.00
-14.70
80.00
Change in
Business Acumen
Knowledge
Application
65
0
9.92
19.14
1.83
3.99
-21.40
80.00
65
0
739.97
126.83
-0.96
1.33
325.00
954.00
0.000) was achieved for change in business knowledge and change in business acumen;
however, the knowledge application score was not considered significant (p = 0.625).
Table 6
Test of Homogeneity of Variances for Dependent Variables
Levene
Statistic
df1
df2
Sig.
Change in Business
Knowledge
13.218
62
0.000
Change in Business
Acumen
15.019
62
0.000
Knowledge
Application Score
0.474
62
0.625
Dependent Variables
106
Table 7.
Kolmogorov-Smirnova
Shapiro-Wilk
Experimental
Treatment
Statistic
df
Sig.
Statistic
df
Sig.
0.111
23
.200*
0.969
23
0.675
0.146
22
.200*
0.927
22
0.106
0.142
20
.200*
0.925
20
0.124
Figure 11. Normal Q-Q Plots and Boxplot for Change in Business Knowledge.
Univariate normality was identified in the Q-Q plots and Boxplot for change in business
knowledge based on the type of experimental treatment for each group.
107
Kolmogorov-Smirnova
Shapiro-Wilk
Experimental
Treatment
Statistic
df
Sig.
Statistic
df
Sig.
0.177
23
0.060
0.920
23
0.067
0.213
22
0.010
0.864
22
0.006
0.117
20
.200*
0.961
20
0.560
across groups. The assumption can be tested by calculating the significance of the
interaction effects between the covariates and the primary independent variable. If this
interaction is statistically significant, the assumption of homogeneity of covariate
regression slopes is suspect (Fields, 2009).
This statistical analysis was performed using analysis of covariance (ANCOVA).
The extraneous variables, or covariates, identified in this study were the participants
prior experience with business simulations and autonomy or level of work control within
the company. The ANCOVA results show the continuous covariate (level of autonomy)
with no real significance (p = 0.710); however, a very high significance (p = 0.001) was
calculated for the dichotomous covariate (previous simulation experience). The
interaction is statistically significant for prior simulation experience which may indicate
that the assumption of homogeneity for this regression slope is suspect.
ANOVA Analysis
An analysis of variance (ANOVA) was performed for the pre/post-test for each
experimental group in order to determine the level of statistical significance between
groups. Based on this analysis, the post-test results for both business knowledge and
business acumen were determined to be statistically significant (p = 0.000).
109
Table 9
Experimental
Treatment
Sum of
Squares
df
Mean
Square
Mean
Square
Sig.
Pre-Test on
Business
Knowledge
Between Groups
262
131.1
0.854
0.431
Within Groups
9512
62
153.4
Total
9775
64
Post-Test on
Business
Knowledge
Between Groups
30389
Within Groups
4188
62
Total
34577
64
Pre-Test on
Business
Acumen
Between Groups
1491
745.3
Within Groups
12908
62
208.2
Total
14398
64
Post-Test on
Business
Acumen
Between Groups
18237
9118.4
Within Groups
6498
62
104.8
Total
24735
64
15194.4 224.938
0.000
67.5
3.580
0.034
86.996
0.000
The pre and post-test mean for business knowledge and business acumen were also
calculated for all three experimental treatments, resulting in 30.4 for the control group,
37.4 for traditional training and 80.2 for simulation training as shown in Figure 12.
The mean post-test score for business acumen was 36.7 for the control group, 40.7 for
traditional training and 74.8 for simulation training as shown below in Figure 13.
MANCOVA Analysis
The primary research question for this study was tested by examining the
significance of the F statistics for the independent variable and for each of the covariates
in the ANOVA tables. A significance level below the critical value of 0.05 allowed for
the null hypothesis to be rejected. Confidence intervals and contrast matrices were also
built to further evaluate the statistical results. This analysis showed a high level of
significance for experimental treatment (p = 0.000) and prior simulation experience (p =
0.002). Level of autonomy for each participant does not appear to be significant with a
significance of 0.511 and an observed power of 0.207. The tests of between subject
111
effects for experimental treatment was shown to have a significance of p = 0.000 and an
observed power of 1.000, 0.999, and 0.983 for change in knowledge, change in acumen
and knowledge application, respectively. Autonomy does not appear to have a high
significance; however simulation experience is significant for knowledge application as
shown in Table 10.
Table 10
Training
Mean
Square
Sig.
Observed
Power
27,978
13988.8
95.458
0.000
1.000
7,125
3562.3
15.383
0.000
0.999
229,773
114886.5
10.303
0.000
0.983
247
247.3
1.688
0.199
0.248
0.6
0.003
0.958
0.050
10,170
10170.1
0.912
0.343
0.156
14
13.6
0.093
0.761
0.060
64
63.6
0.274
0.602
0.081
156,161
156160.8
14.004
0.000
0.957
Dependent Variables
Knowledge Application
Change in Business Knowledge
Autonomy Change in Business Acumen
Knowledge Application
Change in Business Knowledge
Prior
Simulation Change in Business Acumen
Experience Knowledge Application
Correlation Analysis
The secondary research questions involving the degree of the relationship
between knowledge application skills and online business simulation performance was
considered using correlation analysis. The magnitude and direction of the Pearson
correlation coefficients served to describe the relationship between variables. This
analysis was performed by initially looking at the scatterplots based on experimental
112
Table 11
Type of
Training
Area of
Expertise
.811**
0.000
65
.596**
0.000
65
.347**
0.005
65
1
-.372**
0.002
65
-0.233
0.062
65
0.016
0.898
65
-.426**
0.000
65
1
65
-.426**
0.000
65
.252*
0.043
65
-0.046
0.717
65
65
-0.120
0.341
65
0.105
0.407
65
Industry
Level of
Experience Education
(Years)
(Years)
.302*
-0.090
0.014
0.477
65
65
.283*
-0.085
0.022
0.502
65
65
0.088
0.099
0.487
0.434
65
65
.252*
-0.046
0.043
0.717
65
65
-0.120
0.105
0.341
0.407
65
65
-0.140
1
0.267
65
65
-0.140
1
0.267
65
65
113
Comparison of Means
The supporting research questions involve measuring the change in learning when
comparing the control group (group 1) directly against traditional corporate training
(group 2) and online business simulations (group 3). This investigation was intended to
determine whether there was any additional evidence that the change in business
knowledge and change in business acumen was significantly different based on the type
of pedagogical training that was given to adult learners. This portion of the analysis was
accomplished with simple comparison of means and T-tests. The mean values and
standard deviation for each of these variables is shown in Table 12.
Table 12
Dependent
Variable
Change in
Business
Knowledge
Change in
Business
Acumen
Knowledge
Application
Experimental
Treatment
1
2
3
1
2
3
1
2
3
N
23
22
20
23
22
20
23
22
20
Mean
-0.290
5.550
53.210
-0.530
4.623
27.76
670.1
781.3
774.9
Std.
Deviation
8.910
6.502
18.362
4.664
9.129
24.85
139.9
104.6
101.4
The change in business knowledge and business acumen for the control group was
relatively small at -0.290 and -0.530, respectively. However, change in business
knowledge and business acumen was slightly higher for the traditional group at 5.550 and
114
4.623, respectively. Overall change in business knowledge and business acumen for
simulation training was significant at 53.210 and 27.76, respectively.
An independent samples test for equality of means was also performed in order to
determine the level of significance for each of these dependent variables where equal
variances are not assumed. The level of significance, mean difference, and standard error
of difference was calculated for change in business knowledge, change in business
acumen and knowledge application. Change in business knowledge showed a significant
value (p = 0.000) for both Levenes test and the t-test. Change in business acumen also
showed a significant alue (p = 0.001) for both tests; however, knowledge application was
not considered significant Levenes Test (p = 0.935) or the t-test (p = 0.841).
Table 13
Dependent
Variable
Change in
Business
Knowledge
Change in
Business
Acumen
Knowledge
Application
19.477
0.000
Not Assumed
Assumed
12.612
0.001
Not Assumed
Assumed
0.007
0.935
Not Assumed
df
-11.423 40.0
0.000
-47.660
4.172
-10.998 23.3
0.000
-47.660
4.334
-4.080
40.0
0.000
-23.137
5.672
-3.930
23.6
0.001
-23.137
5.887
0.202
40.0
0.841
6.423
31.852
0.202
39.8
0.841
6.423
31.805
Hypothesis Testing
Given the preceding discussion of the underlying characteristics of the data for
this study, the next step of analysis is the testing of the hypotheses. These hypotheses
115
were constructed via predicted relationships between one independent variable, three
dependent variables, two covariates, and three descriptive variables. The one
independent variable is type of pedagogical training. The three dependent variables are
change in business knowledge, change in business acumen, and knowledge application
skill based on the training that was introduced to each participant. The two covariates are
level autonomy within the company (continuous covariate) and previous simulation
experience (dichotomous covariate). The three descriptive variables are area of expertise,
industry experience, and level of education for each participant.
The three dependent variables were based on a pre/post-test which was given to
all participants. The change in business knowledge and change in business acumen was
based on the difference between the pre-test and post-tests (see Appendix A), while the
knowledge application score was based on the final Comp-XM score for each
participant. The continuous covariate, Level of Autonomy, was based on a survey given
to all participants using an instrument developed by Dwyer and Ganster (1991) as shown
in Appendix B. This survey instrument consists of twenty-two Likert-scaled items where
each participant is asked to indicate the extent to which each item is an accurate or
inaccurate description of their job by circling a number in front of each statement. The
dichotomous covariate, Simulation Experience, was also based on a survey where each
participant simply identified whether they had ever used a business simulation before this
training. The three descriptive variables regarding expertise, industry experience and
level of education were also captured through an online survey prior to the training.
Level of education was based on number of years of education including high school.
Industry experience was also self-reported by each participant as total number of years of
116
work experience in the semiconductor industry. Finally, expertise was based on their job
position and organized into four major departments; operations, marketing, finance, and
human resources.
Alternative Hypotheses
The alternative hypotheses are reiterated as follows:
H1a. There is a difference across experimental groups for business knowledge
after adjusting for previous simulation experience and autonomy/work control.
H2a. There is a difference across experimental groups for business acumen after
adjusting for previous simulation experience and autonomy/work control.
H3a. A correlation exists between participants business acumen and business
knowledge.
H4a. Participants who engage in an online business simulation will demonstrate
higher business knowledge after experiencing the business simulation.
H5a. Participants who engage in an online business simulation will demonstrate
higher business acumen after experiencing the business simulation.
H6a. There is a difference between the simulation group, the case study group,
and the control group based on their change in business knowledge.
H7a. There is a difference between the simulation group, the case study group,
and the control group based on their change in business acumen.
H8a. A correlation exists between participants business simulation performance
and their level of autonomy/ work control in the company.
H9a. A correlation exists between participants business simulation performance
and their years of industry experience.
H10a. A correlation exists between participants business simulation performance
and their level of education.
117
dev). In sharp contrast, the online business simulation group (N = 20) achieved an
average gain in business knowledge <g>S-BK = 53.21 18.36 (std dev.). This is a 2.55
standard deviation difference in the normalized gains between traditional and simulation
training.
In the case of business acumen, the gains were also significant at <g>T-BA =
4.62 9.13 (std dev.) while the online business simulation group achieved an average
gain of <g>S-BA = 27.76 24.85 (std dev.) which is almost one full standard deviation
difference in the normalized gains between traditional and simulation training. These
results demonstrate that there is a significant difference across experimental groups for
business acumen and business knowledge, allowing for both null hypotheses to be
rejected. Conclusions from these findings are discussed fully in Chapter 5.
simulation scores (H3) and if so, to determine how this change in knowledge application
skills compares across experimental groups (H6 and H7). Based on the analysis of
variance (ANOVA), the post-test results for both business knowledge and business
acumen were determined to be statistically significant (p = 0.000). Also, a strong
correlation was found between business knowledge and business acumen for all three
experimental groups based on the Pearson correlation coefficients. This analysis allowed
for the null hypothesis to be rejected for all three hypotheses. Conclusions from these
findings are also discussed in Chapter 5.
119
ANOVA can be used as a follow-up technique to determine the extent means differ
between groups and determine the level of significance for each variable.
Table 14
Analysis of Variance Between Groups
Variable
Between Groups
Within Groups
Total
Previous
Between Groups
Simulation
Within Groups
Experience Total
Change in
Between Groups
Business
Within Groups
Knowledge Total
Change in
Between Groups
Business
Within Groups
Acumen
Total
Knowledge Between Groups
Application Within Groups
Score
Total
Between Groups
Area of
Within Groups
Expertise
Total
Industry
Between Groups
Experience Within Groups
(Years)
Total
Between Groups
Education
Within Groups
Level (Years)
Total
Between Groups
Gender
Within Groups
Total
Level of
Autonomy
Sum of
Squares
0.0
45.0
45.0
3.7
12.6
16.2
35895.0
9040.8
44935.7
9492.4
13957.9
23450.3
174046.4
855515.5
1029561.9
21.1
87.1
108.2
158.9
2339.3
2498.2
0.4
117.7
118.1
0.2
7.5
7.8
120
df
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
2
62
64
Mean
Square
0.02
0.73
1.84
0.20
F
0.033
Sig.
0.967
9.079
0.000
17947.49 123.081
145.82
0.000
4746.22
225.13
21.082
0.000
87023.21
13798.64
6.307
0.003
10.55
1.40
7.511
0.001
79.46
37.73
2.106
0.130
0.19
1.90
0.098
0.907
0.11
0.12
0.916
0.405
This analysis shows a high level of statistical significance between groups for the
dichotomous covariate, previous simulation experience: F65 = 9.08, p = 0.000; change in
business knowledge: F65 = 123.08, p = 0.000; change in business acumen: F65 = 21.08, p
= 0.000, knowledge application score: F65 = 6.31, p = 0.003 and area of expertise: F65 =
7.51, p = 0.001. However, years of industry experience showed only slight significance:
F65 = 2.10, p = 0.130; while level of education: F65 = 0.098, p = 0.907 and level of
autonomy within the organization: F65 = 0.333, p = 0.967 showed very little significance
in this analysis. Based on the fact that these last three variables failed to achieve
statistical significance (at p = < 0.05), the null hypothesis was accepted for H8, H9, and
H10. Conclusions from these findings are discussed fully in Chapter 5.
Summary
This chapter presented the statistical analysis and interpretation of the quantitative
data collected during this study. A sample population of employees and managers at a
semiconductor manufacturing facility were exposed to different types of corporate
training in order to investigate the use of online business simulations by measuring the
change in participants business knowledge and business acumen. Descriptive statistics
of the demographics of this sample were discussed. The reliability of individual variables
used in the research were analyzed and univariate normality was verified. The
assumption of homogeneity of covariance matrices was also checked using Levenes and
Boxs tests. Finally, the assumption of homogeneity of covariate regression slopes was
investigated so that a multiple analysis of covariance (MANCOVA) could be completed.
121
An analysis of the data indicated that seven of the ten alternative hypotheses were
fully supported. No significant relationships were found between the dependent variable
of business simulation performance and participants individual level of autonomy, years
of industry experience or level of education. The results of the hypotheses test and the
previously mentioned descriptive statistics for the sample demographics are used in
Chapter 5 to answer the primary and secondary research questions as well as the
supporting research questions.
Exploratory analysis of the data indicated that some of the dependent variables
(e.g., change in business knowledge and knowledge application) are significantly
impacted by the type of pedagogical training given to participants. The results of this
exploratory analysis provide rich material as a basis for examining the research questions
and insight into the use of online business simulations as a disruptive technology.
122
Table 15
Summary of Hypothesis Testing
Null Hypothesis
Result
H01
Rejected
H02
Rejected
H03
Rejected
H04
Rejected
H05
Rejected
H06
There is NO difference between the simulation group, the case study group and
the control group based on their business knowledge.
Rejected
H07
There is NO difference between the simulation group, the case study group and
the control group based on their business acumen.
Rejected
H08
Accepted
H09
Accepted
H10
Accepted
123
Research Questions
Primary Research Question
The primary research question was as follows:
1. Do online business simulations provide an increase in business knowledge or
business acumen for participants, which is on the order of a disruptive
innovation?
Two alternative hypotheses were tested in this study in order to answer this first
research question. Change in business knowledge and change in business acumen were
hypothesized to show a difference across experimental groups after adjusting for three
descriptive variables and two covariates. The three dependent variables were change in
business knowledge, change in business acumen, and knowledge application skill based
124
on the type of training that was introduced to each participant. The three descriptive
variables were area of expertise, industry experience, and level of education. The two
covariates were level of autonomy within the organization (continuous covariate) and
previous simulation experience (dichotomous covariate).
(N = 22) which was based on lecture and the case method achieved an average gain in
business acumen <g>T-BA = 4.62 9.13 (std dev.) while the online business simulation
group achieved an average gain of <g>S-BA = 27.76 24.85 (std dev.). This gain is
almost one full standard deviation difference in the normalized gains between traditional
and simulation training. These results demonstrate that there is a difference across
experimental groups for business acumen and business knowledge allowing for both null
hypotheses to be rejected. However, it is not clear that this can be classified as a
disruptive innovation for change in business acumen, because the gain was not near a
2.00 standard deviation difference in the normalized gains between teaching pedagogies.
Part of this may be due to the wide spread standard deviation at 24.85% for business
acumen compared to smaller spread of 18.36% for business knowledge. Further research
could be done in this area to determine if a focus on change in business acumen through
application is what actually leads to such a dramatic increase in business knowledge.
Three alternative hypotheses were tested in this study in order to answer these
secondary research questions. The intent of these research questions were to explore
whether a correlation exists between participants business simulations and if so,
126
determine how this change in knowledge application skills compares across experimental
groups. Based on the analysis of variance (ANOVA), the post-test results for both
business knowledge and business acumen were determined to be statistically significant
(p = 0.000). These results indicate a strong correlation between a participants individual
level of business knowledge (what they know) and their individual level of business
acumen (what they do). Also, a strong correlation was found between the change in
business knowledge and change in business acumen for all three experimental groups
based on the Pearson correlation coefficients. This analysis allowed for the null
hypothesis to be rejected for all three hypotheses and these conclusions shed light on
theoretical debate about the usefulness of online business simulations. These results are
in line with researchers who defend the disruptive nature of business simulations and the
positive influence of promoting training benefits provided by new technologies
(Anderson & Lawton, 2006; Cantoni & McLoughlin, 2004; Hernandez et al., 2010).
127
Five alternative hypotheses were tested in this study in order to answer these
supporting research questions. The intent of these supporting research questions was to
investigate whether traditional training or online business simulations lead to a significant
increase in business knowledge or business acumen and also investigate whether any
correlation exists between simulation performance and autonomy, industry experience, or
level of education. The comparison of means for gains in business knowledge and
business acumen across experimental groups shows over a two standard deviation
difference in the normalized gains for business knowledge and almost one full standard
deviation difference in normalized gains for business acumen, allowing for two of these
five null hypothesis to be rejected.
This analysis showed a high level of statistical significance between groups for
previous simulation experience: F65 = 9.08, p = 0.000; change in business knowledge: F65
= 123.08, p = 0.000; change in business acumen: F65 = 21.08, p = 0.000, knowledge
application score: F65 = 6.31, p = 0.003 and area of expertise: F65 = 7.51, p = 0.001.
However, years of industry experience showed only slight significance: F65 = 2.10, p =
0.130; while level of education: F65 = 0.098, p = 0.907 and level of autonomy within the
organization: F65 = 0.333, p = 0.967 showed very little significance in this analysis.
Based on the fact that these last three variables failed to achieve statistical significance (at
p = < 0.05), the null hypothesis was accepted that no correlation exists between
participants business simulation performance and their level of autonomy, years of
industry experience or level of education. In summary, the background or work
experience of the participant is not relevant to their gains in learning.
128
Disruptive Innovation
This research supports the theory of disruptive innovation (Christensen, 1997)
that a new technology within management education, such as online business
simulations, can come into non-consumer markets where successful implementation is
not dependent on the participants background or work experience. This study also
indicates that the use of online business simulations could be used with success in other
markets of non-consumption where expensive simulation training is typically limited.
The gains in business knowledge and business acumen which resulted from this research
study were based on managers and employees at relatively low positions of autonomy
within the organization. One of the key identifying characteristics of disruptive
innovations is that they begin in markets which are considered insignificant or emerging
relative to competing technologies. In the case of online business simulations today,
most are used at very high levels of executive management within a corporation or at
elite business schools.
The results from this research study indicate that significant gains in business
knowledge can be gained for employees and managers at all levels within an
organization. This study therefore connects experiential learning theory from psychology
(Lewin, 1935) with the developing field of disruptive innovation (Christensen, 2008), in
order to support the business case for significantly higher levels of adult learning through
the use of online business simulations. The latest research today on business simulation
games still focuses on faculty perception of simulations instead of actual empirical data
(Tanner et al., 2012). Based on these perceptions, there are still arguments over whether
online business simulations are effective teaching tools or simply window dressing.
129
Practical Implications
The future of online business simulations shows promise as a disruptive
innovation versus a sustaining innovation. Based on the characteristics of a disruptive
technology, this type of teaching pedagogy has a bright future in emerging or
insignificant markets. One of these emerging markets today is online education. In fact,
the use of business simulations in online education and the impact on business knowledge
seem to correspond relatively well to Christensens (1997) description of disruptive
technology. The challenge in the past has been determining whether these simulations
are truly disruptive innovations by measuring the effect of these business simulations on
participants today. Based on this research and the gains shown in business knowledge
and business acumen relative to traditional education, a strong case can be made for the
use of online business simulations as a disruptive innovation.
As online business simulations gain a foothold market and evolve over time, there
appear to be three areas of emphasis in order for this technology to become truly
disruptive; (a) there must be stronger pedagogical support for business simulations, (b)
the use of reflection and observation within these simulations needs to significantly
increase, and (c) empirical data on the impact of business simulations needs to be
collected as a form of experiential learning. This empirical-based research study helps
move management education forward in at least two areas of emphasis. Empirical data
from this study can be used to encourage stronger pedagogical support for online
business simulations and encourage focus on current non-consumer markets.
This research study helped to fill an important gap in the literature based on the
fact that most business classes are taught today with one main pedagogy and almost no
131
experimental studies exist that compare learning outcomes under alternative pedagogies
(Anderson & Lawton, 2009, p. 208). Todays learning environments in both corporate
training programs and business management classes mandate greater levels of participant
engagement, due mostly to shorter attention spans and the need for increased interaction
and stimulation within a structured environment (Tanner et al., 2012). Unfortunately,
most research today on interactive engagement tools, such as online business simulations,
is based on qualitative analysis via the perceptionof faculty and instructors. For
example, when evaluating knowledge application, the following Likert Statements from
the Clute Institute (2012) are often being used:
The simulation helps participants understand and integrate previous business
concepts in ways that enable them to apply the concepts in the future because they
1. are helpful in applying concepts that are taught in my discipline.
2. are effective in creating a learning context where students are willing to
open their minds to the course subject matter.
3. are effective in getting students to apply lessons learned in my course.
4. make learning more enjoyable.
5. make learning the material in my course easier to understand.
6. make the material in my course easier to understand.
7. provide an educational experience where students can learn about interfunctional coordination within a business.
8. enable students to experience competition within a marketplace.
These long-winded questions with ambiguous ratings on a scale of 1 to 5 do not get to the
heart of what is happening with actual learning gains inside of the simulation. At best,
132
this type of research gives a sense of how faculty and participants feel about their
learning; however, there is no empirical evidence to support this analysis!
Based on a recent survey of 1,586 marketing management faculty in the United
States (out of which only 107 responded) less than half of the faculty felt that simulations
inspired them to be more creative or helped them to connect with their students (Tanner
et al., 2012). With respect to creativity, only 33.9% of faculty agreed or strongly agreed
that simulations inspire them to be more creative in class. Similar results were found in
the area of connecting with students, where only 45.9% of faculty agreed or strongly
agreed that simulations inspired them to connect with their students. This focus on
inspiration and creativity on the part of faculty has very little to do with learning gains,
which is what business simulations are designed to do. This same study went on to great
lengths in order to draw on the most recent survey research for business simulations
(Buzzetoo-More et al., 2009; Goorha & Mohan, 2010; Wilson & Gerber, 2008) which
were also faculty perception based in order to point out that while simulations are
engaging, faculty do not believe that they are holistically more effective than cases,
service learning, or in-class discussions (Tanner et al., 2012).
When considering the disruptive nature of this technology, it is not surprising that
faculty and instructors specifically say that they feel less valued and less creative. This is
because the innate nature of the business simulation and the ability of the participant to
reflect on what they are learning actually replace the instructor as the source of
knowledge. Here it becomes clear that not only are researchers using the wrong tools to
measure the impact of business simulations, they are surveying the wrong people with the
wrong instruments and methods!
133
classrooms. Based on the findings in this research and the importance of real empirical
evidence, it is the opinion of this researcher that the last thing that needs further analysis
is additional perception of business simulations. Other future research that has been
identified is the possible recency effect associated with simulations. It has been argued
that methodologies like case studies have existed much longer than simulations and it
would be interesting to determine if there is a recency effect in order to explain the
positive perception of simulations among students and participants. There is some
suspicion here that these same researchers are not interested in empirical data that would
allow simulations to stand on their own as an effective learning tool.
In light of these recommendations by other researchers in this field and the
valuable data gained from this research study, it is recommended that future research with
business simulations be conducted in the area where business management education is
beginning to show the biggest impact on transformational learning online education
(Garrison, 2004).
Conclusion
This study set out to investigate online business simulations as a form of
experiential learning by providing the first empirical analysis of business acumen and
knowledge application skills. This quantitative based research study demonstrated the
disruptive nature of online business simulations when it comes to gains in business
knowledge by measuring a 2.55 standard deviation difference in the normalized gains
between traditional training and business simulation training. Baseline tests against a
control group and traditional training group using MANCOVA to account for multiple
variables and covariates imply that online business simulations enhance both business
knowledge and business acumen on a staggering scale and in a very short period of time.
Previously to this study, almost all empirical research regarding business
simulations had relied on the participants perception of their learning experience (Tanner
et al., 2012 & Williams, 2011). This research study was unique in the business
simulation literature in that a significant amount of quantitative data was collected and
analyzed in order to measure the change of participants business knowledge and
business acumen based on the type of training given in a corporate setting. These
research results strongly suggest that the adoption of online business simulations in
corporate training environments can (1) increase business knowledge of employees and
managers at all levels within a company and (2) provide a rate of increase in business
knowledge and acumen which is well beyond that obtained by traditional practice.
As a potential disruptive technology, online business simulations have not
received the empirical research they deserve. Recent studies focused on the effectiveness
of business simulations have been based on perceptions of business management faculty.
137
Not surprisingly, these same faculty memberswho could quickly be replaced by such
technologyhave responded in surveys that simulations limit their creativity and ability
to connect with participants. It is important that the correct research methods be used to
measure the individuals who are actually impacted by the simulation and capture the
actual learning gains. One recent study of simulations (Xu & Yang, 2010) which actually
used empirical research methods focused on mental models as learning outcomes for
simulation participants instead of changes in business knowledge or business acumen.
This study hypothesized that the social interaction among group members and the level of
psychological safety led to higher learning gains, not necessarily the pedagogical nature
of the business simulation instruction. When it came to measures in this study, a Likerttype scale was used and once again the measurement was perception of the participants
learning experience instead of actual gains in learning.
Clearly, the disruptive nature of online business simulations will not be found in
the current consumer market of elite business schools or executive management programs
for large corporations. This technology has the low-cost and scalable structure to impact
non-consumer markets where successful implementation is not dependent on the
participants background or work experience. It is very possible that someday in the near
future, large numbers of employees at all levels within an organization will be able to
benefit from the use of online business simulations. The self-paced and interactive nature
of such simulations will allow participants to teach one another and reflect on what they
have learned through learning by doing rather than learning by knowing. This
disruptive form of learning may continue to be called gaming or it may translate into
something much more appealing for the common knowledge worker, actually becoming.
138
The great divide between the gaming community and management educators is
quickly coming to a close. In the past, gamers have dismissed educational simulations as
boring and irrelevant, while business management educators have dismissed gaming and
simulations as trivial and pedagogically unproven (Aldrich, 2009a). This research study
points out a significant opportunity which lies within an engaging business simulation
and its potential impact on the world of management education. As a truly disruptive
technology, online business simulations have the potential to dramatically impact
emerging markets such as online education with new and more powerful ways of
teaching and learning.
139
REFERENCES
Adkins, S. S. (2003). Radical learning technology happening now. Training and
Development, 57(11), 65-73.
Albert, W. (1970). Participative education and the inevitable revolution. Journal of
Creative Behavior, 4(4), 234-282.
Aldrich, C. (2009a). The complete guide to simulations and serious games: How the most
valuable content will be created in the age beyond Gutenberg to Google. San
Francisco, CA: Pfeiffer.
Aldrich, C. (2009b). Virtual worlds, simulations, and games for education: A unifying
view. Innovate 5 (5). Retrieved August 1, 2010 from http://faculty.chasshttp:
//www.innovateonline.info/index.php?view=article&id=727.
Anderson, J. R. (1985). Cognitive psychology and its implications (2nd ed.). New York:
Freeman.
Anderson, P. H., & Lawton, L. (1990). The relationship between financial performance
and other measures of learning on a simulation exercise. Developments in
Business Simulations and Experiential Exercises, 17(1), 6-10.
Anderson, P. H., & Lawton, L. (1995). The problem of determining and individualized
simulations validity as an assessment tool. Developments in Business Simulations
and Experiential Exercises, 22(1), 43-48.
Anderson, P. H., & Lawton, L. (1997). Demonstrating the learning effectiveness of
simulations: Where we are and where we need to go. Developments in Business
Simulations and Experiential Exercises, 34(1), 43-50.
Anderson, P. H., & Lawton, L. (2002). Is simulation performance related to application?
An exploratory study. Developments in Business Simulations and Experiential
Exercises, 29(1), 108-113.
Anderson, P. H., & Lawton, L. (2006). The relationship between students success on a
simulation exercise and their perception of its effectiveness as a pbl problem.
Developments in Business Simulations and Experiential Exercises, 33(1), 41-47.
Anderson, P. H., & Lawton, L. (2007). Simulation performance and its effectiveness as a
pbl problem: A follow-up study. Developments in Business Simulations and
Experiential Learning, 34(1), 43-50.
140
Anderson, P. H., & Lawton, L. (2009). Business simulations and cognitive learning:
Developments, desires, and future directions. Simulation & Gaming, 41(6), 193216.
Anderson, P. H., & Woodhouse, R. H (1984). The perceived relationship between
pedagogies and attaining objectives in the business policy course. Developments
in Business Simulations and Experiential Exercises, 11(1), 152-156.
Andlinger, G. R. (1958). Business games play-one. Harvard Business Review, 28(3),
115-125.
Archer, W., Garrison, R., & Anderson, T. (1999). Adopting disruptive technologies in
traditional universities: Continuing education as an incubator for innovation.
Canadian Journal of University Continuing Education, 25(1), 13-44. Retrieved
from Business Source Complete database.
Argyris, C., & Schon, D. A. (1974). Organizational learning: A theory of action
perspective, reading. Oxford, England: Addison-Wesley.
Bailey, D. A. (1990). Developing self-awareness through simulation and gaming. Journal
of Management Development, 9(2), 38-42.
Bergemann, L. H., & Queen, T. R. (1987). The aftermath: Treating traumatic stress is
crucial. Corrections Today, 49(5), 100-104.
Biggs, J. (1985). The role of metalearning in study processes. British Journal of
Educational Psychology, 55(1), 185-212.
Biggs, J. (1988). The role of metaleraning in enhancing learning. Australian Journal of
Education, 32(2), 127-138.
Biggs, W. (1990). Introduction to computerized business management simulations. Guide
to Business Gaming and Experiential Learning. (pp. 23-35). London, England:
Nichols Publishing.
Bloom, B. S. (1956). Taxonomy of educational objectives: Handbook/cognitive domain.
New York: McKay.
Bloom, B. S., Englehart, M. D., Furst, E. J., Hill, W. H., & Drathwohl, D. R. (1956).
Taxonomy of educational goals. Handbook 1: Cognitive domain. New York:
McKay.
Blythe, S. E., & Gosenpud, J. J. (1981). A relative evaluation of experiential and
simulation learning in terms of perceptions of effected changes in students.
Developments in Business Simulations and Experiential Exercises, 8(1), 145-148.
141
Boud, D., Keogh, R., & Walker, D. (1985). Reflection, turning experience into learning.
London, England: Nichols Publishing.
Bransford, J. D., & Nitsch, K. E. (1978). Coming to understand things we could not
previously understand. Speech and Language in the Laboratory, School and
Clinic (pp. 267-307). Cambridge, MA: MIT Press.
Bredemeier, M. E., & Greenblat, C. S. (1981). The educational effectiveness of
simulation games: A synthesis of findings. Simulation & Games, 12(1), 307-332.
Brown, A., Campione, J., and Day, J. (1981). Learning to learn: On training students to
learn from texts. Educational Researcher, 10, 14-21.
Bruce, M. (2001). Reflecting on reflection: An examination of learning and assessing
outcomes. Simulation and Gaming, 28, 8-12.
Burns, A. C., Gentry, J. W., & Wolfe, J. (1990). A cornucopia of considerations in
evaluating the effectiveness of experiential pedagogies. In J. W. Gentry (Ed.),
Guide to business gaming and experiential learning (pp. 253-278). East
Brunswick, NJ: Nichols/GP Publishing.
Buzzetto-More, N., & Mitchell, B. (2009). Student performance and perceptions in a
web-based competitive computer simulation. Interdisciplinary journal of elearning and learning objects, 5(1), 73-89.
Cantoni, L., & McLoughlin, C. (2004). Different thinking hats: The continuouslyevolving role of the instructor in E-problem based learning. Proceedings of EdMedia 2004: World Conference on Educational Multimedia, Hypermedia &
Telecommunications, (Chesapeake, VA: AACE), 2864-2870.
Chapman, K. J., & Sorge, C. L. (1999). Can a simulation help achieve course objectives?
An exploratory study investigating the differences among instructional tools.
Journal of Education of Business, 74(1), 225-230.
Cherryholmes, C. (1966). Some current research on effectiveness of educational
simulations: Implications for alternative strategies. American Behavioral
Scientist, 10(1), 4-7.
Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Selfexplanations: How students study and use examples in learning to solve problems.
Cognitive Science, 13, 145-182.
Christensen, C. M. (1997). The innovators dilemma: When new technologies cause great
firms to fail. Boston: Harvard Business School Press.
142
Christensen, C. M. (2008). Disrupting class: How disruptive innovation will change the
way the world learns. New York: McGraw Hill.
Christensen, C. M., & Raynor, M. E. (2003). The innovators solution: Creating and
sustaining successful growth. Boston: Harvard Business School Press.
144
Fields, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: Sage
Publications.
Flavell, J. H. (1976). Metacognitive aspects of problem solving. The Nature of
Intelligence (pp. 231-235). Hillsdale, NJ: Erlbaum.
Flavell, J. H. (1987). Speculations about the nature and development of meta-cognition.
Meta-cognition, Motivation and Understanding (pp. 21-29). Hillsdale, NJ:
Erlbaum.
Foreman, J. (2004). Game based learning: How to delight and instruct in the 21st century.
Educause Review, 39(5), 50-66.
Hake, R. R. (1998). Interactive engagement versus traditional methods: A six-thousandstudent survey of mechanics test data for introductory physics courses. American
Journal of Physics, 66(1), 64-74.
Harry, L. (1971). Using simulation in the classroom. Feedback on Instructional
Simulation Systems, 127-134.
Hernandez, A. B., Gorjup, M. T., & Cascon, R. (2010). The role of the instructor in
business games: A comparison of face-to-face and online instruction.
International Journal of Training & Development, 14(3), 169-179.
Heyman, M. (1982). What are simulation games? Environmental Education Report,
10(5), 2-5.
Isaac, S. & Michael, W. (1997). Handbook in research and evaluation. San Diego, CA:
EDITS.
Jackson, J. R. (1959). UCLA executive decision game. Proceedings of the National
Symposium on Management. University of Kansas, VI-9-VI-15.
Jacobs, J. W., & Dempsey, J. V. (1993). Simulation and gaming: Fidelity, feedback, and
motivation. Interactive Instruction and Feedback (pp. 192-227). Englewood Hills,
New Jersey: Educational Technology Publications.
Kayes, D. C. (2002). Experiential learning and its critics: Preserving the role of
experience in management learning and education. Academy of Management
Learning and Education, 1(2), 137-149.
Kirby, D., & Garrison, R. D. (1990). Graduate distance education: A question of
communication? Canadian Journal of University Continuing Education, 16(1),
25-36.
Knowles, M.S. (1978). The adult learner: A neglected species, 2nd ed. Houston: Gulf.
Knowles, M. S. (1980). The modern practice of adult education: From pedagogy to
andragogy. Englewood Cliffs, NJ: Cambridge.
Knowles, M.S. (1984). Andragogy in action. San Francisco: Jossey-Bass.
Knowles, M. S. (1989). The making of an adult educator. San Francisco: Jossey-Bass.
146
Knowles, M.S., Holton, E.F., & Swanson, R.A. (2005). The adult learner: the definitive
classic in adult education and human resource development. Sixth edition.
Boston: Elsevier.
Knowles, M. S. & Hulda, F. (1973). Introduction to group dynamics. Chicago: Follett.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and
development. New Jersey: Prentice Hall.
Lederman, L. C. (1984). Debriefing: A critical re-examination of the post experience
analytic process with implications for its effective use. Simulation and Games, 15,
415-431.
Lederman, L. C. (1992). Debriefing: Toward a systematic assessment of theory and
practice. Simulation and Gaming, 23, 145-159.
Lefebvre, J. R. (1997). The battle for change. Mortgage Banking, 57(4), 1-7.
Lenth, R. V. (2001). Some practical guidelines for effective sample size determination.
The American Statistician, 55(3), 187-193.
Lewin, K. (1935). A dynamic theory of personality. New York, NY: McGraw-Hill.
Lewin, K. (1947). Frontiers in group dynamics. Human Relations, 1(2), 143-153.
Lewis-Beck, M. S., Bryan, A., & Lia, T. F. (2004). The Sage encyclopedia of social
science research methods. Thousand Oaks, CA: Sage Publications.
Lin, X. D., Hmelo, C., Kinzer, C., & Secules, T. (1999). Designing technology to support
reflection. Educational Technology Research and Development, 47(3), 43-62.
Mandl, H., Gruber, H., & Renkl, A. (1992). Problems of knowledge utilization in the
development of expertise. Flexibility in Training and Vocational Education (pp.
291-305).
McGrenere, J. L. (1996). Design: educational electronic multi-player games: A Literature
review. Technical Report 96-12. Department of Computer Science, University of
British Columbia.
Mclaughlan, R. G., & Kirkpatrick, D. (2005). Online text-based role play-simulation: The
challenges ahead. Simtect, Sydney: Simulation Industry Association of Australia.
147
Rafii, F., & Kampas, P. J. (2002). How to identify your enemies before they destroy you.
Harvard Business Review, 80(11), 115-123.
Raths, J. (1987). Enhancing understanding through debriefing. Educational Leadership,
45(2), 24-27.
Reigeluth, C. M., & Moore, J. A. (1999). Cognitive education and the cognitive domain.
Instructional-design theories and models (2nd ed). Hillsdale, NJ: Eribaum.
Rose, A. (1992). Framing our experience: Research notes on reflective practice. Adult
Learning, 3(4), 5.
Ruben, B. D. (1999). Simulations, games, and experience-based learning: The quest for a
new paradigm for teaching and learning. Simulation & Gaming, 30(4), 498-505.
Rubin, A., & Babbie, E. (2005). Research methods for social work (5th ed.). Belomont,
CA: Thomson.
Salmon, W. (2001). Reflections of a bashful Bayesian: A reply to peter lipton. In
explanation: theoretical approaches and applications. Kluwer Academic
Publishers.
Sandefer, J. (2009). Foreword to: the complete guide to simulations and serious games:
How the most valuable content will be created in the age beyond Gutenberg to
Google. San Francisco, CA: Pfeiffer.
Sandefer, J. (2007). Yales Taliban man and other tales. Academic Questions, 20(2),
113-121. doi: 10.1007/s12129-007-9003-0.
Sarin, S., & McDermott, C. (2003). The effect of team leader characteristics on learning,
knowledge application, and performance of cross-functional new product
development teams. Decision Science, 34(4), 707-739.
Scardamalia, M., & Beretier, C. (1991). Higher levels of agency for children in
knowledge building: A challenge for the design of new knowledge medial.
Journal of the Learning Sciences, 1, 37-68.
Scherpereel, C. M. (2005). Changing mental models: Business simulation exercises.
Simulation & Gaming, 36(5), 388-403.
Schon, D. (1983). The reflective practitioner: How professional think in action. New
York: Basic Books.
Schon, D. (1987). Educating the reflective practitioner: Toward a new design for
teaching and learning in the professions. San Francisco: Jossey Bass.
149
Schumpeter, J. A. (1989). The theory of economic development (R. Opie, Trans.), New
Brunswick, NJ: Translation. (Original work published 1911).
Segon, M., & Booth, C. (2009). Enhancing management competence through business
simulations. Journal of Business & Policy Research, 4(2), 111-123.
Sidor, S. M. (2008). The impact of computer based simulation training on leadership
development (Doctoral dissertation). Retrieved from ProQuest Dissertations and
Theses database. (AAI 3277804)
Smith, R. (2006). Technology disruption in the simulation industry. Journal of Defense
Modeling and Simulation, 1(3), 3-10.
Strawbridge, W. G. (1995). The effectiveness of andragogical instruction as compared
with traditional instruction in introductory philosophy courses (Doctoral
dissertation, University of Southern Mississippi, 1994). Dissertation Abstracts
International, 55(11), 3387A.
Summers, G. J. (2004). Todays business simulation industry. Skillful product
development. Simulation & Gaming, 35(2), 208-241.
Swanson, R. A. & Holton, E. F. III. (2005). Research in organizations: Foundations and
methods of inquiry. San Francisco, CA: Berrett-Koehler.
Szczurek, M. (1982). Meta-analysis of simulation games effectiveness for cognitive
learning. Dissertations Abstracts International, 43(4), 1031A.
Tanner, J.R., Stewart, G., Totaro, M.W., & Hargrave, M. (2012). Business simulation
games: Effective teaching tools or window dressing? American Journal of
Business Education, 5(2), 115-128.
Thatcher, C. (1990). Promoting learning through games and simulation. Simulation and
Gaming. 21(3), 262-273.
Vye, N., Schwartz, D. L., Bransford, J. D., Barron, B. J., & Zech, L. (1998). SMART
environments that support monitoring, reflection and revision. Metacognition in
Educational Theory and Practice, 305-347. Hillsdale, NJ: Erlbaum Associates.
Watson, H. J. (1981). Computer Simulation in Business. New York: John Wiley & Sons.
Wilcox, R. R. (2005). Introduction to robust estimation and hypothesis testing (2nd ed.).
Burlington, MA: Elsevier.
150
151
2.
Review the FastTrack to determine Ferriss current strategy. Where will they
seek a competitive advantage?
From the following list, select the top five sources of competitive advantage
that Ferris would be most likely to pursue.
[ ] Seek high automation levels
[ ] Reduce labor costs through training and recruitment
[ ] Seek high plant utilization, even if it risks occasional small stockouts
[ ] Seek the lowest price in their target market while maintaining a
competitive contribution margin
[ ] Reduce cost of goods through TQM initiatives
[ ] Accept lower plant utilization and higher capacities to insure sufficient
capacity is available to meet demand
[ ] Increase demand through TQM initiatives
[ ] Seek excellent product designs, high awareness, and high accessibility
[ ] Offer attractive credit terms
[ ] Add additional products
152
3.
4.
Chester currently has $1,027 (000) in cash and management has decided to
issue stocks and bonds worth an additional $20,000 (000). Assuming that
cash from operations will be the same for each of the following activities,
which activity exposes this company to the most risk of being issued an
emergency loan?
A.
B.
C.
D.
5.
8,400
9,130
6,350
14,000
18,260
153
6.
Americ is a product of the Andrews Company. The sales forecast for Americ
is 1,500 units. Americ wants to have an extra 10% of units on hand above
and beyond their forecast in case sales are better than expected. (They would
risk the possibility of excess inventory carrying charges rather than risk lost
profits on a stock out.) Taking current inventory into account, what will
Andrews Production After Adjustment have to be in order to have a 10%
reserve of units available for sale?
A.
B.
C.
D.
7.
1,650 units
1,620 units
1,630 units
1,480 units
8.
Baker
Feast
Eyring
Fast
154
9.
10.
$20,510
$37,910
$43,679
$23,169
155
11.
12.
Last year the Chester company increased their equity. In 2014 their equity
was $28,363. Last year (2015) it increased to $37,673.
What are causes of change in equity? Check all that apply.
[
[
[
[
[
[
[
[
]
]
]
]
]
]
]
]
Profits of $13,455
Depreciation of -$4,185
Issue and retirement of stock
Dividend payment of $1,562.
Change in inventory of -$832.
An accounts payable change of $392.
An investment in plant and equipment of $5,050.
A change in cash of -$1,873.
156
7. How much control do you have over scheduling and duration of your rest
breaks?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
8. How much control do you have over when you come to work and leave?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
9. How much control do you have over when you take vacation or days off?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
10. How much are you able to predict what the results of your decisions you
make on the job will be?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
11. How much are you able to decorate, rearrange, or personalize your work
area?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
12. How much can you control the physical condition of your work station
(lighting, temperature)?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
13. How much control do you have over how you do your work?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
14. How much can you control when and how much you have to interact with
others at work?
1
2
3
4
5
Very Little
Little
Moderate
Much
Very Much
Amount
158