Questions Arising From The Assessment of EFL Narrative Writing

Questions arising from the
assessment of EFL narrative writing

YongYi
Introduction
I had been a well-regarded English teacher in a university in China

before Ispent a year in Australia as a visiting scholar, followed by a total
of five years studying for MA and PhD degrees in the United Kingdom.
However, Ifound that Iwas not well prepared for the English language
Iencountered on my travels. This was a curiosity at first. Irealized
that this was not simply what is often referred to as cross-cultural
communication issues; there was more to it than simply learning about
the customs and mores of another peoplegroup.
What Ibelieve Iwas experiencing was that, despite using the same
language framework, communication and understanding was difficult
because we told different stories in different ways. Ibegan to reflect
on the nature of English teaching in China and how well it serves our
students, especially if they are to be employed in English speaking
countries or work with native English speakers in China or elsewhere
in the globalized world. It seemed to me that we needed to consider
revising our approach away from focusing on technical forms of
English towards a more pragmatic, evolved, and relevant engagement
with the language as it is used in different cultures that use English
natively. The majority of everyday communication basically consists of
engagement with an ongoing narrative: we are story-bound creatures, as
expressed so well by Hardy (1968: 5):
70
ELT Journal Volume 67/1 January 2013; doi:10.1093/elt/ccs062
The Author 2012. Published by Oxford University Press; all rights reserved.
Advance Access publication November 20, 2012
Downloaded from http://eltj.oxfordjournals.org/ at Universidad Autnoma de Tlaxcala on March 23, 2015
This article questions how narrative writing is assessed, seeking to understand

what we test, what we value, and why. It uses a single anomalous case that
arose in the course of my recent PhD thesis to highlight the issues, asking
if sufficient attention is being given to the value of emotional content in a
piece of writing in comparison to its technical merit. In doing so, the nature
and purpose of teaching narrative writing as a subject in EFL is questioned.
Given the diversity of English-speaking cultures, has too little attention
been given to the importance of narrative writing as a key subject area? Is
there unrealized potential for integrating narrative writing into language
acquisition? Anew dialogue between academics and practitioners is called
for to develop new approaches in understanding and enhancing our concepts
of narrative writing.
For we dream in narrative, day-dream in narrative, remember,

anticipate, hope, despair, believe, doubt, plan, revise, criticize,
construct, gossip, learn, hate, and love by narrative. In order really
to live, we make up stories about ourselves and others, about the
personal as well as the social, past and future.
Hardys reminder that narrative is an indispensable part of our daily life
led me to speculate that it perhaps should be a more important element
in EFL teaching and learning than it currently is.
The curious case of

Number20
To assess students pre- and post-test written narratives, four assessors

were engaged: two native speakers of English (D1 and D2) and two
native speakers of Chinese (Q and Y) who taught English in a Chinese
university. They were asked to rate the narratives by applying the
following five-element criteria:
reader involvement
imagery provoking/stimulating
communicative effectiveness
fluency
intrinsic value.
These criteria were not conventional, as Iwas attempting to find

a way to differentiate writing quality in line with criteria related to
development through visualization techniques.
The results proved inconclusive in terms of differentiating the test
narratives. At one level, therefore, this test section was devoid of useful
data. There was, however, one piece of work in the post-test set for the
Experimental group that presented a hugely anomalous result (see
Appendix A). All the assessors marked it as the highest in the whole
tranche, and the score was significantly higher than this participant
received in her pre-test piece (see Appendix B). As has been noted in
other situations throughout history, it is the anomaly, the exception,
that has provided people with the stimulus to explore more deeply.
Thus, the post-test piece by Number 20 became one of the most
curious and stimulating outcomes of my research.
In order to explore the assessors responses to this piece, Ineeded to
identify the issues at stake. There were multiple variables in the matrix;
most significantly and obviously Iwondered if the result stemmed from
problems in the rating criteria, the assessors use of the criteria, or was
it simply a response to the nature of the topic? Thus, Iformulated two
follow-up questions for the assessors to discover more about how they
conducted their assessment of Number 20s contributions:

Questions arising from the assessment of EFL narrative writing
71
The perspective Ioutline stemmed from issues that Iencountered in

my doctoral research (Yi 2011) concerning the effects of visualization
training on intermediate Chinese EFL learners narrative writing. In
this research, Iinvolved two groups of participants: the Experimental
group which was given developmental training in narrative writing with
specific training in visualization, and the Control group who received
developmental training without the visualization elements.
1 In what way(s) is the second piece better than the first? Please
comment.
2 Please feel free to add any other comment that you consider may be
of value.
I purposely kept the questions very open so that the assessors could say
whatever they had in mind when recording their thoughts about the
process. This was mainly to explore the assessors internal processes:
their subjective reality. Iwas not seeking to justify or critique this but
to see where it led me in terms of developing my conceptual thinking
about narrative writing. The following section is a summary of the
assessors responses.
Assessor D1 emphasized the fact that the technical use of words still
has to be good enough not to detract from the story, but that they
[the words] are very much secondary in importance to the intrinsic
power and value of the story itself. He placed strong emphasis on the
relationship between writer and reader in co-creating the story and its
meaning through the power of shared and individual imagery:
I found Number 20s first piece to be quite visual and vaguely
interesting, nominally engaging, but it was not fluent and full of
grating language errors. The emotional content and intrinsic value
did not raise itself up high enough to compensate for the frustration
in reading between the lines that was necessary to follow the story.
In contrast, the second piece, objectively, could be said to be little
more fluent or imagery-stimulating, but its emotional content was
deep, sincere and raw and the power of the story transcended the
technical aspects, drawing the reader in and almost making one gasp
with the intrinsic value and emotionality of the piece: it was not just
good, it was searingly, heart-rendingly beautiful. Ihad to fight to
try and find a place to be objective and mark down the fluency and
communicative effectiveness, as Ijust wanted to give it full marks
for making me feel so much.
Assessor D2 questioned whether Number 20had someone in the
family background with a fair competence in English, raising the
question of how exposure to competent English usage outside the
learning environment affects storytelling ability.
It is not easy to pin down five criteria for assessing a short piece of
writing. They cant be mutually exclusive, for certain. The criteria
reader involving and communicative effectiveness are both good,
but tend to overlap. Imagery provoking is difficult, as some students
tend to concentrate on emotions without much reference to things
seen and heard. Would reactions or inner feelings cover the
ground? Fluency is a good requirement to sort out the A-students
from the rest. Command of a natural English style is needed,
allowing the reader undivided attention.
Assessor Qs responses were written in Chinese and the following is a
close translation, which was confirmed by Q as being accurate.
72 YongYi
Responses to
follow-up questions
In the second piece the student expressed her emotions more clearly
and made it easier for the reader to be engaged in the whole piece of
writing and to share the sadness as well as to understand the father in
the end. It is more fluent and therefore the effect on communication
is better. The grammatical mistakes did not affect the meaning or the
conveyance of emotions.
In the following sections, Iexplore the context of EFL narrative writing

and its assessment and introduce the wider issues that arose for me.
Issues in the
assessment of
narrative writing
The issue of how to assess narrative writing has long been with us
(for example Yancey 1999). The fundamental teaching and learning
question has been characterized as a question of head or heart, but it
is subtler than that simple dichotomy and the assessment issues are
more complex than the traditional polarities of analytic or holistic.
Simply put, there is a problem in distinguishing good narrative
writing from poor narrative writing (Hamp-Lyons 1995). Some even
express frustration in this regard, for example Huot (1996: 449),
who commented that Many composition teachers and scholars feel
frustrated by, or cut off from, otherwise uninterested in the subject of
writing assessment .
The two traditional ways to evaluate writing, referred to as holistic and
analytic, may be defined as follows: according to Khalil (1989: 362),
holistic rating is based on the evaluators general impression of the
piece of writing, while analytic rating makes use of a set of criteria
preselected by either the evaluators or the researcher. There is little
consistency in the approach to narrative writing assessment criteria,
but the following references offer four indicative examples of differing
rubrics.
In developing assessment rubrics to train teachers in their assessment
of elementary school childrens narrative writing, Gearhart and Wolf
(1994) used the following principal headings:
theme
setting
character
plot
communication.
These highlight the constituent elements of narrative writing as

required for their purposes. Although focused upon elementary school

73
Assessor Ys responses were directed towards the technical content of

the writing. She attached importance to vividness of description, as
well as to the simplicity and fluency of the language used. Although
Assessor Y thought the first piece of writing lacked both intrinsic value
and fluency, she thought that the student followed some technique in
writing the narrative in the second piece. After carefully analysing the
whole of the second piece of writing, Assessor Y reached the conclusion
that it is a complete story with beginning, peak and ending and that the
reader can feel the true love through the language.
childrens approach to writing narrative, Ifound that their emphasis

on communication was relevant and useful for my study. They made a
strong case for communication being the core aim of narrative writing,
associated with the transmission of meaning, emphasizing that they
felt a need to upset the applecart of traditional notions about writing
assessment, where convention is more important than communication
(ibid.: 68). This supports the general direction of my belief that
assessment of narrative writing needs to focus on meaning, context,
and effective communication over simple reliance upon linguistic
competence.
the writers personality is expressed; confidence and feeling are

apparent;
individual, powerful commitment to the topic is obvious;
connection to audience and purpose is excellent;
writing evokes strong emotion in the reader.
The High Plains assessment rubric thus differs from others in that, in
addition to conventional areas such as grammar and spelling, it involves
the writers personality, feelings, and connection to the audience.
The inclusion of rating criteria concerning the stimulation of readers
emotions encourages the expression and assessment of communicative
competence in broader ways that meet more effectively the demands of
a twenty-first century globalized world.
Discussion
In my wide experience of rating students writing, no matter what

rubrics or criteria for assessment are designed beforehand, assessors
appear to be more inclined to mark according to their holistic, overall
response to the writing. According to my observations and dialogue
with teaching colleagues, it is the same throughout the process of
assessing students writing. Sometimes teachers/assessors might even
work backwards, using the framework in such a way as to justify the
marks awarded according to their original, intuitive, or experienced
responses. In order to take account of this phenomenon, it is necessary
to address the issue as it is, not as we might like it to be or assume
it has been. The time assessors have to report is also a factor and
might predispose towards holistic assessment, citing experience
as the justification. The assessment of narrative writing has to meet
justifiable criteria for consistency in order to ensure the students work
is assessed fairly, but the qualitative nature of the assessment needs
74 YongYi
After my work was completed, Idiscovered the High Plains Regional

Technology in Education Consortiums (2010) six-trait model designed
to contribute to the quality of narrative writing through an assessment
rubric, which supports the Gearhart and Wolf approach. This comprises
focus on topic (ideas and content); sequencing (organization); adding
personality (voice); word choice; sentence structure (fluency); and
grammar and spelling (conventions). Within this, for instance, the
element ideas and content states that effective and appropriate details
create a vivid picture showing knowledge and insight, which attempts
to blend holistic and analytical concerns. Similarly, with the criterion
voice, the rubric incorporates such elements as
acknowledgement. The development of an appropriate scheme needs

to be considered anew. Good narrative tends to appeal to criteria that
are not easily defined, but are inherently subjective, and thus resistant
to categorization. Phenomenologically, assessors derive a significant
proportion of their assessment using their instinct, responding to
their heart, especially in the face of an emotionally powerful story. But
is this kind of assessment reliable? In the learning and development
context, is it desirable? Is the emphasis upon effect right to hold over
technique?
To illustrate this, Ire-present the following assessor comments made in

response to Number 20s work:
t he reader can feel the true love through the language (Assessor Y);
grammatical mistakes did not affect the meaning or the conveyance
of emotions (Assessor Q);
the power of the story transcended the technical aspects (Assessor
D1).
These comments appear to support the approach engendered by the

Gearhart and Wolf and High Plains Consortium rubrics, as previously
described. By its very nature, narrative writing is subjective. Its
purpose is to appeal to readers emotions and sensitivities; to convey
information of course, but not to just do that. Therefore, my questions
are about the evolving nature of narrative and just what we are seeking
from it. In the past, the tendency was for a technical approach, such
as that reported in Zamel (1982), which points out that many studies
had sought to prove the efficacy of one grammar over another (thus
perpetuating the belief that a better pedagogical approach, particularly
one that focused on usage, structure, or correct form, would improve
writing), concluding that this approach, while relevant, was inadequate.
Johnson and VanBrackle (2012) show that previous research has
examined reactions by raters to errors in narrative writing pieces. They
reported that Connors and Lunsford (1988) conducted an historical
study on rater reactions to errors. They concluded that educators in
the early twentieth century were more concerned about errors than
contemporary educators due to the predominance of process writing
pedagogies. Paradoxically, they found that while contemporary educators
claim to be less error driven, they, in fact, mark errors more than they
comment on other aspects of essays (for example development or

75
In the context of these questions, it seems that a methodology

representing a new approach needs to be developed, based on a
pragmatic reflection of the need for engagement with the skill of telling
stories that really mean something and telling them well. This is the
reason for my wanting to develop narrative writing in EFL specifically.
For me, storytelling is the universal form of communication that
promotes contextual understanding of a language in dynamic use and
thus it becomes a potential force in language acquisition, not just
an exercise in technical competence. Would a new form of dialogue
between teaching practitioners and research academics be a useful
development?
Concluding thoughts
The curious case of Number 20 indicated that grammatical accuracy

does not necessarily affect the impact of narrative writing on readers.
Although it was a singular, anomalous result acquired tangentially from
other work, it points to findings and discussions elsewhere and opens a
valid area of enquiry.
Rarely will researchers want to take much account of single instances
if we assume a general empirical approach to development, but it is my
contention that we may gain much from the anomalous if we learn to
ask the right questions. My focus is on analysing what happened and
why, and what it might mean in terms of liberating the deeper potential
inherent in effective narratives to express meaningful, emotionally
intelligent writing, in balance with legitimate requirements for
grammatical accuracy and appropriate vocabulary, as expressed clearly
in another of Assessor D1s comments:
narrative writing is evidence of the power of a story to transcend its
component writing elements and calls into question purely analytical
approaches. The value of a narrative lies in its ability to communicate
meaning holistically; and that the meaning should be a blend of what
the writer found meaningful and what the writer wants to offer to
stimulate the readers own meaning.
In commenting on what to teach and how to assess, D1 adds:
Thus the job of the teacher is to teach effective, engaging, socially
valuable communication with meaning: not to produce an assembly
line of perfect grammar that means nothing much.
Is the job of the assessor to value the right elements using head and
heart, engaged emotionally and tuned to hear the story, and not just to
appreciate the apostrophe rule and correctly hanging indents? Ithink so,
but it would be useful for researchers and practitioners engaged in the
work of teaching and assessing EFL narrative writing to share practice.
Without any methodological restrictions, it would be useful to generate
a wide discussion on, for instance, how narrative writing is valued for
76 YongYi
content); Ispeculate that this is because it is easier to mark in this way

under time pressures, but Ineed to test this view through engagement
with others. Most significantly, Connors and Lunsford contend that
definitions of errors and views of what constitutes a severe error change
over time and that teachers are influenced by current trends. There is,
in other words, no absolute standard for the judgment of writing. This
again emphasizes the questions concerning what it is that we seek
and why; this is a dynamic, fluid field of work and, one may argue,
a field that is changing more rapidly now than ever before. Whether
the advancement of new criteria will help to address the concerns
about meaningless subjectivity for the objectivists or the concerns
of subjectivists about false objectivity or not, may depend on the
potential for gaining agreement on progress towards large-scale practical
application. But to do this, we return once again to the question of just
what narrative writing is hoping to achieve in an EFL context.
the significant contribution it makes to literacy and language acquisition

and how we can develop teaching and assessment to address a dynamic,
relevant, and culturally appropriate understanding of the place of
narrative. This focus on stimulating engagement and conversation
between academics and practitioners is to fundamentally address the
question of how we can best prepare our students to interact with
confidence in the diversity of English-speaking environments.
Final revised version received August 2012
Appendix A
Number 20s posttest writing
Khalil, A. 1989. A study of cohesion and

coherence in Arab EFL college students writing.
System 17/3: 35971.
Yancey, K.B. 1999. Looking back as we look
forward: historicizing writing assessment.
College Composition and Communication 50/3:
483503.
Yi, Y. 2011. The effects of visualization training
on intermediate Chinese EFL learners narrative
writing. Unpublished PhD thesis, Leeds
Metropolitan University, UK.
Zamel, V. 1982. Writing: the process of
discovering meaning. TESOL Quarterly 16/2:
195209.
The author
Yong Yi has been involved in EFL teaching for
many years in Qufu Normal University, China.
She holds a PhD in ELT from Leeds Metropolitan
University, UK. Her research interests include
EFL teaching, second language acquisition, and
using visualization to develop language skills.
She is also currently involved in a large-scale
project comparing translations of the Analects of
Confucius.
Email: yiyong01@gmail.com
Five years ago Ilost the most close friend all my life. My dear sister, a
lovely, diligent, clever and kind-hearted girl, left us and never got back.
Thats disaster of the whole family.
In those days, tears just running down whenever Isaw the bed we shared,
the books she had read, the dolls we played and the pictures she had
drawn. Icouldnt figure out why she left me all of a sudden. Icouldnt
accept it. So did my mother. She seemed to be older and heart broken.
But Icouldnt understand my father. He just kept doing all the things
in the house and arrange all the things left by the disaster. He talked
little to us. He just kept doing all the time. Didnt he love his little
daughter? No, that cant be true. Im sure. But why he seemed to be so
indifferent?
77
References
Connors, R. and A.Lunsford. 1988. Frequency of
formal errors in current college writing, or Ma
and Pa Kettle do research. College Composition and
Communication 39/4: 395409.
Gearhart, M. and S.A.Wolf. 1994. Engaging
teachers in assessment of their students narrative
writing: the role of subject matter knowledge.
Assessing Writing 1/1: 6790.
Hamp-Lyons, L. 1995. Research on the rating
process. Rating non-native writing: the trouble
with holistic scoring. TESOL Journal 29/4:
75962.
Hardy, B. 1968. Toward a poetics of fiction.
NOVEL: AForum on Fiction 2/1: 514.
High Plains Regional Technology in Education
Consortium. 2010. 6 trait writing model: narrative
writing rubric - hero story. Available at http://
www.fm.coe.uh.edu/resources/language_arts/
narrrubric.pdf (accessed on 10 October 2011).
Huot, B. 1996. Toward a new theory of
writing assessment. College Composition and
Communication 47/4: 54966.
Johnson, D. and L. VanBrackle. 2012. Linguistic
discrimination in writing assessment: how raters
react to African American errors, ESL errors,
and standard English errors on a state-mandated
writing exam. Assessing Writing 17/1: 3554.
Then about one month later. It was a dark dusk, mother asked me to
find father and had our supper. Iwalked to our new house, Ithought
he should be working there. When Iwas about 50 meters from the door
of our new house, a gloomy sound, which sounded like the roar of a
tiger, came to me. Iwas just scared. Ididnt know what it was. Ihad
never heard of a sound like that before. Ifound that the sound seemed
to be controlled not to let others hearit.
He didnt want us to know it. Imustnt let him know what Isaw him
crying. So Igently walked out and when Iwas about 50 meter away.
Icalled, Dad, where are you? Time for supper.
After a little while, he answered, OK, Ill be back right away.
After that answer, tears came out again. Ijust cant help myself.
When he got back Isaw gentle smile on his face. Ijust smiled back.
But Iwas crying deep in myheart!
His precious tears just taught me what true love is! Dear dad, Ilove you.
Twenty-one years has passed since Iwas born into the colourful world.
Appendix B
Number 20s pre-test Ihave experienced so many happy days. But that day was the happiest
day in mylife.
writing
It was July 26th. Iplanned to come back to school by train from
Beijing. Another boy who was expressing his true love to me wanted to
send me back. Ihad tried to refuse his advice. But he had tried all the
ways to let me know how much he lovedme.
That night, Iwas in the train ahead of the time. But he was slowed
down by the heavy traffic. He sent me news that he almost cried out.
He had done a lot for it, for that day. But Idont know anything. Itold
him not to worry. If he couldnt catch the train, he could go back home.
He wouldnt lose anything. Thats better forhim.
However, he said that he wouldnt give up. Ididnt know how he got on
the train at last, But hedid.
What Ididnt know at all was that it was July 7th in Chinese old way of
showing time. He brought me 99 roses with him. There was so many
people that the way was full of people. You couldnt get out to the W.C.
If you want to move, you have to ask a lot of people to move their feet.
There was no more space for yourfeet.
He was not in the same railway carriage. He must get through all the
people in the carriages. Icouldnt imagine how he could do it. Iwas not
sure how many people would try to stop him. He didnt have to dothat.
He sent me the news with only a few words: JUST WAIT FORME.
78 YongYi
I just stopped my steps, Ididnt know what was going on in the house.
But my father must be in the house. So Icalmed myself down and
walked very gently into door. Isaw a man sitting in the corner. There
was no light Icouldnt see him clearly. But Iwas sure he was my father.
He was crying with his head burying into his arms. Icouldnt believeit.
Then about one hour later. He brought the roses to me. My eyes were
all wet. Ididnt know how to express my feeling. But Iwas moved
completely. All the people around smiled at us. That moment Iknew
Iwas the happiest people in the world. That day, July 7th was the
happiest day in my life.
79

Questions Arising From The Assessment of EFL Narrative Writing

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Questions Arising From The Assessment of EFL Narrative Writing

Transféré par

Droits d'auteur :

Formats disponibles

Questions arising from the

assessment of EFL narrative writing

I had been a well-regarded English teacher in a university in China

ELT Journal Volume 67/1 January 2013; doi:10.1093/elt/ccs062

Downloaded from http://eltj.oxfordjournals.org/ at Universidad Autnoma de Tlaxcala on March 23, 2015

This article questions how narrative writing is assessed, seeking to understand

For we dream in narrative, day-dream in narrative, remember,

The curious case of

To assess students pre- and post-test written narratives, four assessors

These criteria were not conventional, as Iwas attempting to find