LANCE NOE

ASSESSMENT IN TESOL

TASK 2

ASSESSMENT IN TESOL TASK 2

Part A
Context:
The students are at the university level in an Asian country. Their level is upper intermediate and
they are studying English to pass the TOEFL test. The class is a requirement for graduation. Although the
course is not a test preparation class, the speaking class has become centered on the new TOEFL speaking
test. This assessment is a midyear assessment based on target criteria. It is not a summative assessment
but does count toward their final grade.

Purpose:
The purpose of this test is to see if the students have actually acquired the speaking skills in
order to pass certain aspects of the new TOEFL speaking test. The major stakeholders in this situation
are the students themselves. The teacher also has a stake in the process but this is limited due to the
fact that the test is not within the universitys jurisdiction. The parents that are providing the education
money for the university student would also have a stake, though minor, in the assessment.

Procedure:
I chose this particular task due to the fact, after researching university jobs, requirements and
course work that this is a new push in the Asian university level classes and I know very little about it. I
decide to choose speaking in particular because it interests me and I think it is a major weak point in the
Asian educational system when it comes to the study of English. The test will be an extensive speaking
type assessment and will use picture- cued story telling. The following items will be scored in the process
of the assessment

Pronunciation

Do they speak the words correctly and clearly

Do they use an amalgamation of English and their native tongue? (E.g. busUH, Mac a
donna all duh suh)

Do the mumble or swallow the endings of words

Can they be heard?

Are their words intelligible?

Intonation

Do they sound like a robot when they speak?

1

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

Do they speak with expression?

Do they use sentence clues like commas, periods, and emphasis to enhance their
speech?

Stress

Are they putting the right stress on words?

Do they break up words that are one word? (E.g. to morrow,

Do they combine words to make one word? (Want to = wanna)

Do questions go up at the end?

Cohesion Does the story make sense

Do they use pauses or thought words (umm, uh, suck in air sound, like, so)
excessively?

Does the solution they give go with the question they are given?

Does the solution make sense?

Does the solution fit the problem?

Do they give a solution to the problem/issue or only repeat the question with out any
real answers or opinions?

These particular items were picked based on the following basis: 1. they are included in the TOEFL test. 2.
They reflect important speaking issues that arise in the EFL environment. 3. Are easy for the examiner to
test and score for and 4. They target specific mechanics of speaking production skills.

Outline:
See appendix

Process:
The test will be given in the classroom over a series of weeks. Students will be asked to sign up for
a time slot in 25-minute increments. The test will be done individually in the classroom, office or study
room in the library. The student will be given written instructions and scoring criteria beforehand and on
the test day will be given verbal instructions. The students will be audio taped so that the instructor may
go back and review and give scores to the students.
The assessment will be based on a photograph or picture. There will be 3 different pictures the
students can talk about. All the pictures will be at the same level of difficulty and chosen by the teacher.
Two sheets will be filled out by the teacher, the initially interview with comments and things to listen for
on the tape and the official score sheet based on the taped assessment. All students will receive their
score sheet at the end along with a grade/score. This will allow them to see where their problems lie with
this particular assessment. This is not a summative assessment, therefore they have room to grow and
2

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

improve.

Criteria:
Following the official TOEFL iBT scores / criteria, independent research and personal experiences
in my own class and their issues, I have created the overall criteria for assessment and scoring. A
breakdown of specifics was then structured so that the test giver and the students would know exactly
how the test was to be scored. A breakdown of these scores can be found in the appendix. The criteria
were based on a score of 1 4, 4 being the highest and 1 being the lowest. (See Appendix)

Part B
Design of the procedure:
As was stated in class, speaking assessment is indeed rather difficult. I was under the
impression; wrongly I found out, that speaking assessment was just measuring pronunciation,
articulation, possibly pacing and understanding. Throughout the course of my research I have had to
tackle many issues and misconceptions on my part.
When I began this assessment I assumed I would simply follow the guidelines set up and
followed by the TOEFL exam. Since many of the students I will be teaching are ultimately trying to
pass this exam, it seemed a logical place to start. However, after reading about the exam I have
discovered it is not as clear-cut as it seems. First, there was the issue of what type of assessment do I
do, imitative, intensive, responsive, interactive, or extensive (Brown 2004)? Do I use speeches, picture

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

cues, opinion, or any number of other activities? Then there was how many do I assess? How do I
check for a correct assessment? Do I use a checklist, a chart, read aloud, instruction or create something
totally new? Time constraints also needed to be addressed. Not to mention the basics of what class will
I be teaching? What level are they? And what perhaps does the assessment hold? These are some of the
issues I will be addressing in this part of the task to help you understand my decisions in making this
speaking assessment.
First, I needed to decide what class I would be teaching. I did some research on university
teaching jobs on the Internet (Sperling 2007, (Foley 2007) and (AEAC 2007) and discovered that many
of them share a similar class structure; they all are conversation and speaking centered classes. Then I
looked at what that meant for each school. This was a bit tricky. Many of the schools curriculums were
not available online and I simply had to imagine what happens in the class. After speaking to a friend
who has been teaching overseas in the university realm for many years, I learned that the majority of
the classes are beginners and/or high intermediate / advanced. For some reason there are very few low
intermediate classes. The majority of the students in the beginning classes are students who must take
the class to graduate and have no intentions of furthering their English skills beyond the class. So I
decided to go with the high intermediate class. The majority of these students need English for the
work field, to pass the TOEFL test or because they intend to continue their education in a foreign
country. Since the bulk of these students will be taking the TOEFL test and will need speaking skills I
chose them. I did toy with the idea of lowering the bar and making it easier for me by trying to find a
4

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

simple speaking test for beginners but in time abandoned the idea.
I then had to research the TOEFL test itself. This was easy to find and lots of research has been
done on it, however, because the TOEFL speaking test is new, there is not a lot of information on its
effectiveness and quality. I had to turn a lot of my research from TOEFL to IELTS and general speaking
assessment. I thought about using the IELTS instead of the TOEFL but I decided to go with TOEFL for
one reason, in most East Asian countries, TOEFL is more popular and I know very little about it.
The TOEFL test, I discovered, covered many of the speaking types imitative, intensive,
responsive, interactive, and extensive, (Brown 2004) so I had to choose which one I wanted to use. At
first, I thought I would pick the last thing the TOEFL assessed, giving your opinion. I figured I would
utilize this since it seemed to be a more summative assessment. However, when I began to research the
different types of extensive style assessments, I realized it was not the only one that could be used as a
summative assessment. This led me into whether or not I wanted to do summative assessment more
about that decision later. I eventually selected the picture-cued assessment style based on the fact that it
can be used for high and low students. Therefore, in the future it will be more useful for me to have
some background knowledge about it, in case I choose to use it in a beginners class. Pictures are
ideally suited for eliciting oral language from students. For this reason, they can be included in the oral
assessment of individual students (SLIS 2001). I ultimately decided to use an extensive style because
according to Brown (2004) the opportunity for oral interaction from the listeners is either highly
limited (perhaps to nonverbal responses) or ruled out altogether. This way I could give little to no help
5

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

to the students and I would actually be assessing their speaking.

One of the challenges facing second language teachers is how to

provide students with opportunities to show what they can do with the
target language. Assessments must be created that tie into the
curriculum and to classroom practice. Assessment tasks must generate
language samples with enough depth and breadth so that teachers can
make judgments as to how students are doing and provide them with
meaningful feedback on their performance. (Tarone 2005)
I did question, a great deal, about whether I should choose different pictures for each student or
the same picture? As well as what kind of picture? How validity and reliability played into the choice
of pictures? I ultimately decided on different pictures. I assumed this would make it more valid though
less reliable. I assumed that if I use different pictures the students would not have the opportunity to tell
other students what the picture was and therefore, help ensure more reliability. This does raise fairness
issues. What if one picture is more difficult than another? I thought about having low, medium, and
high pictures but was unsure how I would base my decision on who got what picture. Eventually I tried
to choose pictures that I felt were around the same difficulty. Stimulus materials should require students
to demonstrate skills as a speaker or listener and clearly distinguish speaking and listening from
reading and writing (Backlund 1994).
As per our conversation, I thought about using a reading prompt or some other elicitation
technique but decided to still use the pictures because they can be used for low and high as well as
children. I am not sure what my future holds and I assumed this would be the most appropriate for all

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

age and level groups.

I came to the decision to make the assessment a midyear assessment because I was not sure if I
am capable, at this point, as an assessor to give a summative grade. By doing it mid-year I could see if
my scores were indeed correct in some way with what I thought about the students abilities and if I
could figure out how to help the students and teach myself to be better. I would of course use a
summative assessment at the end for their final grade and this could then be used as a barometer to see
if I had actually achieved my goal. Also, after speaking to my colleague, I was informed that many of
the students demanded a midyear grade as did the school itself and the only way to do that for a
conversation class was to have a conversation or speaking exercise.
Once I had decided to use a picture-cued assessment task, the level and style I then need to
outline what items I would be assessing. This was quite possibly the hardest of all the task and I
changed my mind many times right up until the end. However, clarity of goals is a major stumbling
block to many assessment programs (Backlund 1994). I knew I would include the following items:
pronunciation, intonation, stress, and cohesion but was unsure what else would be proper to include.
The longest assessment I had included the above plus, vocabulary, grammar, sentence connectors,
(First, second, finally) time relatives (then, and then so..), relevance of content, completeness of
content, and past tense verbs. This seemed way too much to assess at one time and being a beginner
myself, it would be hard and possibly not accurate. I tried to look at the most important aspects of
speech and choose from them. Eventually, I resolved on this criteria, pronunciation, intonation, stress,
7

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

vocabulary, past tense verbs and cohesion. I felt that they are the most essential and, quite frankly, the
easiest to score. I also wanted not only essential speaking skills (pronunciation, stress, intonation) but
also grammar skills which are vital to comprehension (vocabulary, past tense verbs, and cohesion). It is
nevertheless clear that the assessment of communicative skills is a complex task, made all the more
complex because of the general lack of agreement and clarity about what aspects of performance are
relevant (Brown 2007). After our chat in class I saw your point about taking out cohesion as it is not a
part of assessing production skills. However, I still feel it is an important part of what needs to be
assessed for this level and environment, especially since it is part of every high stakes test on the
market and this class is to help the students to succeed on that test. Therefore, rightly or wrongly, I kept
it in. I did take out past tense verbs and vocabulary, however, to ease the amount of things I was testing.
As it stands now, the assessment will be pronunciation, intonation, stress and cohesion.
Another issue was time. Since I will be holding meetings throughout the week and possibly not
at my scheduled lecture time, the use of that time is a big factor. I have to consider when the students
can come, should I only have assessments during class time and how long would that take? In the end, I
decided to go with trying to get them done within a week by setting up a sign up sheet and having
students fill in when they could come. I would need to extend my office hours to accommodate this and
work around the students schedules. I also needed to think about how much time I would give them to
prepare and actually speak. According to the TOEFL test a picture description is allotted the following
time frame: You will have 30 seconds to prepare your response. Then you will have 45 seconds to
8

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

speak about the picture (ETS 2007). I decided to give them a bit longer. I will allow the student to
prepare for 2 minutes and speak for 1 and half minutes. They will not be penalized for taking less time.
I did this because I assumed it will be their first or one of the first times they will ever take a test
like this and I wanted them to feel more comfortable and not rushed. Also, I know they will make
mistakes so they need time to recover or enhance their answers. However, Brown (2004) states, the
longer the stretch of language, the more possibility for error and therefore the more difficult it becomes
to assign a point system to the text. In such a case it, it may be imperative it score only the criterion of
the task. I have given each student a 15-minute time slot for his or her presentation. This will include a
small interview, which will not be graded but comments will be taken, at the beginning to talk about
where they stand in class and how they feel they are doing, instructions about the assessment, the
assessment itself and an exit interview.
Many of the research articles I read discussed the importance of assessing body language as
well as the students actual speaking. I did not include it on my assessment for the basic reason that it
will not be included on the TOEFL, IELTS or any other high stakes test I could find. Also, I am not
entirely sure how to assess this. I am intrigued by it and would like to research it more in the future.
Rating scales was my last big hurdle. Scoring is the key assessment challenge according to
Brown (2004). How do I make a score and score sheet? I narrowed it down to two types I would like to
use, the checklist and a score. I decided, though it was much more difficult to achieve, a score based
criteria. Analytic Rubric: An analytic rubric separates the different components and scores them
9

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

individually (SLIS 2001). I chose this because 1) It is the way the high stakes test are done and 2) It
would be more of a challenge to see if I could actually accomplish it. Looking at high stakes test (ETS
2007; IELTS 2007) helped actually writing the scales but at the end of the day the language used was
my own.

Evaluation:
I think my assessment, as a whole is ok. I certainly do not feel it is going to change the world of
assessment, as we know by any means. I learned a tremendous amount I didn't know and I am sure I
will be able to use it in the future. I also know I have a lot to learn about before I can make a good
assessment for speaking, however, I also know I am in good company as even the experts have a
difficult time doing it too.
My worries:
Did I rely too much on the parameters of the high stakes test? Should I have made it broader
and less geared toward one goal?
Are my criteria clear, correct and appropriate?
Should I have used one photo instead of a series and are the pictures I decided to use level
appropriate and consistent?
Is using a picture instead of a read aloud or another type of elicitation technique a mistake?
Do I have the skills to be a good listeners and/or interlocutor do carry out a good assessment?

10

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

I worry that doing a midyear instead of summative assessment might be an error. I just could
not decide so I went with midyear and tried to justify it the best I could.
Is choosing to assess grammatical speech as well as mechanics a bad or great idea. I know I
wanted to but just not sure it is correct.
Am I giving the students too much or not enough time for this task? I know the high stakes test
give them less time but I assumed this is the first time many of them have done anything like
this and I wanted to give them time to succeed.
I worry that many of the researchers discussed body language but I chose not to include it.
If I were to give this test today, would it be a complete and abysmal failure or succeed in some
way that I could grow and learn from?

11

References

APPENDIX
Criteria
Assessment sheet
Assessment Pictures
(Example: only one set)

LANCE NOE

ASSESSMENT IN TESOL

Subject

Pronunciation

Intonation

TASK 2

Consistent pronunciation
difficulties.
Considerable listener
effort. Hard to hear.
Ending and median
sounds are consistently
dropped or
unintelligible. Speech is
choppy. Excessive use of
verbal pauses.

Unintelligible.

Speech is mechanical and

Little to no expression

No expression or

times but some varied through

lacks expression and

or varied in speech.

varied. Sounds like

emphasis to draw points to

the use of pauses and emphasis.

variety. Major problems

Sounds like a robot.

a robot. Use no

specific items with in the

Minor problems with re-telling

with re-telling the story

Uses little pauses or

pauses or

speech. Uses native like

the story fluidly.

with fluidity and

emphasis.

expression.

Speech is clear and

Speech is usually clear and

Speech is basically

understandable. The endings

understandable. Some endings

intelligible. Many problems

and median sounds of words

and median sounds are dropped,

with pronunciation of

and phrases are heard

misspoken or unclear. Though

words, which causes the

distinctly. Only minor

there may be some problems

listener more effort.

difficulties with pronunciation,

with pronunciation, which cause

Problems with articulation

which do not affect the

the listeners, to make more

and inaudibility.

overall intelligibility of the

effort, it does not affect the

speech. Speaking is loud and

overall intelligibility. Some points

can be heard. Only minor

the speaking is hard to hear.

pauses.

Only minor pauses.

Variety of expression in the

Speech is a little mechanical at

speech. Uses pause and

expression with re-telling the

Speaker makes
little to no admit
to be understood
or heard.

expression. Boring.

story.

Stress

Stress is consistent and

Stresses problems are minor and

Increase in stress

Major issues with

No stresses used

accurate. Uses appropriate

the speaker uses correct

problems. Stresses are not

consistency and stress

when speaking.

questions sentence stress at

sentence stress at the end of

consistent. The ends of

appropriateness.

Often breaks up

the end sentences. Does not

sentences accurately most of

questions words have minor

Questions sentences

words and

break up words or combine

the time. Sometimes stresses

problems.

sometimes sound like

combines words to

words. Use the correct stress

the wrong syllable on larger

telling sentences.

create new words.

when use difficult words.

words. Minor combinations and

Combines words and

Questions sound

14

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

breaking up of words and

breaks up longer words.

phrases.

Cohesion

the same as
sentences.

The story is intelligible and

The story is intelligible and

The story is not always

The story goes astray in The story cannot

makes sense. Recounts all the

makes sense with some minor

clear, some problems with

many parts. Only major

be understood. It

minor and major details in the

flaws. Recounts some minor and

intelligibility. Recounts very

details are used to re-

lacks creativity and

story. The descriptions are

most major details in the story.

review minor details and

tell the story. Very

interest. The story

creative and accurate. Pauses

Descriptions are creative and

stays focused on the major

little creativity is used.

told does not match

and thought words are held to

accurate with minor issues. Few

details. Consistent pauses

Frequent pauses and

the pictures given.

a minimum. The story is

pauses and thought words. The

and thought words are used

thought words are used. Cannot be followed.

interesting and precise. The

story is interesting but could be

throughout. The story is not

The story is boring and

re-telling is perfect.

improved.

exciting but has good points.

hard to follow.

15

Comments

ASSESSMENT IN TESOL

Cohesion

Stress

Intonation

Pronunciation

Student

LANCE NOE
TASK 2

16

LANCE NOE

ASSESSMENT IN TESOL

TASK 2

17