To:

From:
Celina
Date:
Subject:

Jennie Chiu
Lee Williams, Mary Ellen Potter, Grace Hwang, Shelby Ware, Rachel Han,
Abboud
October 27, 2016
Undercover Detectives: Playtesting-Based Prototype Revision

Following three rounds of playtesting (one session conducted internally by the game design
group itself and two sessions during class with educational and gaming experts), the Undercover
Detectives team has highlighted five key areas of the game to be revised: 1) the intuitiveness and
clarity of the physical materials, 2) the integration of dramatic elements into gameplay, 3) the
expansion of challenges and behavioral restrictions, 4) the feasibility and utility of the scoring
sessions, and 5) modifications for classrooms inclusive of students requiring specialized
instruction.
Physical Materials
Problems
During each round of playtesting, users expressed confusion and cognitive overload when
dealing with the physical materials provided in the detective kits handed out to all players. The
kits included a detailed rules and procedures list, two writing prompts on individual sheets of
paper, stickers, a mission card, and a personal score card. Players were observed sheafing
through the papers, skipping/missing important instructional points due to efforts to organize and
comprehend the various materials extended to them all at once. The progression of materials as
presented in the kits was unintuitive and out of sync with the teachers script of instruction.
Moreover, the titles and descriptions on the writing prompts/mission cards was not clear nor very
noticeable; as a result, players often mixed up their papers and experienced more confusion and
frustration. Finally, given the somewhat complex nature of the games central mechanisms,
playtesters felt that an older population may be more desirable as the ideal users of our game.
Solutions
To reduce the seemingly overwhelming barrage of information and materials presented all at
once, we decided upon several revisions. To begin, we will remove the Rules/Procedures
document from the detective kits. The teacher provides all of this information to the players, and
the presence of the document serves only to confuse the players. We also plan to combine the
writing prompts onto one piece of paper, front and back. The entire layout of the writing prompts
will be redesigned, with large, clear titles and very intuitive directions. The notes section will
be clearly demarcated, with additional lines for writing. The entirety of the language used
throughout the game will be revised to reflect a 6th grade vocabulary. Finally, materials will be
organized in the order in which they will be mentioned by the teacher in order to avoid further
confusion over the flow of the game.
Initially, our target population was upper-elementary school students such as fourth and fifth
graders, but in light of playtester feedback and preliminary research, we have decided instead to
focus on sixth graders for this game. Barbara Kaiser (Challenging

Behavior in Young Children,

2012) notes that as young people undergo puberty, their social-emotional capacities often suffer
developmental setbacks. Our game, then, could be used as a lever to combat this trend.
From the outset, our project goal has not been to create a standalone game to be played in one
single session, but to build an engaging and long-term game to develop important social and
writing skills in young people. This motivation undergirds our reasoning to embrace the
teacher-as-facilitator role and maintain our commitment to the longevity of the game, despite its
initial learning curve. Moreover, Clark et al. (2015) offer data in support of the effectiveness of
educational gameplay being spread over multiple shorter sessions as opposed to one long session
of play. Thus, we aimed to simplify instruction and process rather than alter the games
overarching structure.
Dramatic Elements
Problems
Playtesters noted that certain dramatic elements (namely, music during the writing and elaborate
mission descriptions) distracted from the main learning objectives and hindered player
performance. In other areas, the dramatic elements did not fully develop the games premise and
thus the game felt less like a fun experience and more like an in-school activity to our users.

Solutions
Rather than require music to be played both during the introduction and while players are
writing, we will offer a music selection as an option to teachers to be played during the
introduction (if they so choose). This fits into James Paul Gees notion of customization as a tool
for engaging games (2005).
We also plan to further build our premise in the aesthetics of the game. This will include
revamping the detective kits to look more professional and realistic as well as to offer detective
apparel to players as they progress through the ranks (e.g. magnifying glass, badge, hat). Tracy
Fullerton notes that premise is needed to contextualize the formal elements of gameplay to create
an entertaining and engaging game (Game Design Workshop, 2008).
The mission statements which provide the direct instruction for our central game mechanism
(prompting the detective to interview and empathize with her partner) will be simplified in that
we will remove some of the dramatic language in favor of simpler, more explicit directions. Our
reasoning is backed by the meta-analysis conducted by Clark et al. (2015) in addition to
corresponding feedback from playtesters. They found intense narrative for more complex
gameplay to be detrimental to the effectiveness of educational games. Given the inherently
complex nature of our key game mechanic, we have decided to pull some of the narrative out of
the key mechanics.
Challenges and Behavioral Restrictions
Problems
Some of the word/behavior restrictions for the interview section of the game were vague and
confusing, as reported by playtesters. On the whole, though, this is where playtesters had the
most fun (both by our observation and their self-report). Rather than an overt deficiency, these

observations give rise to an implicit issue with the game: we have not yet fully explored the
potential for fun gameplay as a result of the challenges imposed by our restrictions. Many
players were bored with level 1 because it did not include any restrictions, and thus no
challenges.
Solutions
We have developed and expanded upon the volume of restrictions we will utilize as well as the
type of restrictions we will utilize. These restrictions now include what types of sentences one is
allowed to use while interview, what words one may say or not say, how one may move his/her
body, and so on. We will include some restrictions in level 1 (nothing too difficult) in order to
ensure those players still reach what Gee calls a state of pleasant frustration. Our rationale is
buttressed by a desire to keep gameplay interesting and challenging, tying closely into Gees
concepts of well-ordered problems and cycles of expertise.
Furthermore, including varying restrictions with creative obstacles will build a conceptual open
space for play, allowing free movement within a rigid structure, (Fullerton, 2008). As a result,
players will experience emotional responses to the various levels of difficulty and will become
more engaged.
Scoring Sessions
Problems
The scoring process was vague and generally disengaging for most players. Most players seemed
confused as to how they could make a significant judgment on the matter. Voting class-wide is
both time- and teacher-intensive. The one guiding question - How much does the second piece
of writing sound like the first? - for voting is unclear, leading to somewhat arbitrary voting (and
thus assessment) methods.
Solutions
To deal with the issue of time, we will restructure voting such that instead of the entire class
voting on the strength of one persons undercover writing, it is a five person student panel,
chosen randomly each week. The student panel will evaluate each undercover piece in response
to 5 specific criteria: content similarity, vocabulary similarity, complete sentences/thoughts,
emotions conveyed, overall style. Each of these five criteria will be distilled into five easily
understandable questions for sixth graders to answer with Yes, Maybe/A Little, or No (points: 2,
1, 0, respectively). The panels consensus, not the scores assigned by individual panelists, will be
reported out to the class as a whole. Voting will remain the same in that one-third of the pairs
will be chosen for evaluation.
Our rationale for this overhaul of the scoring process is supported by the Conceptual Assessment
Framework described by Groff, et al. in Better Learning in Games (2015). Our learning
objectives (content model) and the related tasks (task model) are both robust, but our previous
methods of interpreting the evidence provided by the task model were vague and somewhat
arbitrary. With a clear rubric laid out along five specific criteria, we will be equipped to
understand what the written responses mean for the players and how that information can be
used to further gameplay.

Additionally, this procedure will decrease the overall amount of time spent on the voting, and it
will promote a sense of agency for the students (Gee, 2005).
One issue which was not necessarily lacking in our scoring structure, yet which was
recommended we explore is the integration of our game with Virginia Standards of Learning
(SOLs). Our interview and writing processes naturally align with several of the dimensions of
sixth grade Communication and Writing objectives (English Standards of Learning Curriculum
Framework, Board of Education, Commonwealth of Virginia, 2010). We will alter some of the
language within our prompts to reflect the associated SOL objectives in order to market our
game to teachers not only as an interesting alternative to traditional instruction, but as a fun
method to teach necessary SOL competencies.
Modifications for Inclusive Classrooms
Problems
As of the second round of in-class playtesting, our game did not make any provisions for an
inclusive classroom, especially regarding English Language Learners (ELL students), students
with autism spectrum disorders, or nonverbal students, among others. Students with learning
disabilities will struggle with the two key aspects of our game: writing (limited working
memory) and empathy. Students on the spectrum will have trouble with empathy and
communication.
Solutions
We will develop a buddy system in which students who are unable to participate in the
traditional sense will instead be paired with able students who might work with them to
accomplish similar game objectives. Students who are unable to engage in the writing task will
be evaluated according to an alternative rubric for scoring that is available to the teacher and
which can be used in conjunction with their buddys help. Finally, the teacher will be encouraged
to model how one should go about the interview process and begin the practice of empathy
before gameplay begins.
Most of our rationale at this point is derived from an interview with Marie Black, a graduate
student studying special education with a focus on autism spectrum disorders. We aim to delve
more deeply into the literature of this issue during our research-based prototype iteration.
Our particular project goal of working to foster greater empathy in our young players similarly
drives these decisions. As in many other aspects of education, those with whom it may be the
most difficult to engage and who might experience the greatest challenges in these tasks are most
likely those who stand to benefit the most. With this in mind, we look to design a workable
alternative pathway of participation for students with some sort of barrier to the traditional
gameplay.
Conclusion
These two to three rounds of playtesting have engendered some major overhauls of our initial
prototype along with some lesser tweaks of the more central game components. As we move

forward we plan to work these revisions into another operable prototype in order to secure more
feedback and continue the process of iteration.