Assessing Grammar

Christine
Alice
Stacey
Jenny
Ann

Test Evaluation
Summary Teaching
Evaluation of an Existing
Test

Five characteristics to measure

communicative grammar
1.
2.
3.
4.
5.

The test must provide more context than

only a single sentence.
The test taker should understand what
the communicative purpose of the task is.
He or she should also know who the
intended audience is.
He or she must have to focus on meaning
and not form to answer correctly.
Recognize is not sufficient. The test taker
must be able to produce grammatical
responses.
By: P.R. Dickins

Therefore

Have the test taker say or write

something (5) of discourse length (1)
in order to perform some
communicative task (2) for a known
audience (3), and what is said or
written must make sense (4).
Assessing grammar can only be
tested as part of a test of writing or
speaking?

Important Principle: What are we trying to

Measure?
English grammar is chiefly a system of syntax that
decides the order and patterns in which words are
arranged into sentences. Close 1982

Separated but closely related to assessing writing and

assessing speaking.
Oral performance
Writing samples

Accuracy and fluency

Count the errors and rate their severity

Focus on Grammar only

Meaning is a separated matter

Integrated tests
Discrete point tests

Traditional Paper and Pencil Test

NRT: TOEFL
CRT: Classroom tests
Types of Questions:

Complete sentences
Sentence combining
Paragraph editing
Correct topic sentence task
Multiple Choice
Fill in the blank
True/False
Combination

Alternative Assessment

Games
Class observation
Online activities
Portfolio

Games

Design to restrict the language need

Usually these games are defined by
their focus on
The Use of Language
Limited options for communication
Accuracy

Split Sentences

Write out some sentences, and then

cut each sentence in half. Hand out
these pieces to the students, and ask
them to find the matching half among
the other students.
Students abilities in combing
sentences
Grammar knowledge to all forms of
the sentences

Examples

If you eat that

If you touch the dog
If you steal my
boyfriend
If you go out now
If you dont leave
If you dont book a
ticket

youll be sick
itll bite you
Ill never speak
to you again
youll get soaked
Ill call the police
youll be lucky to
get a seat

Sentences from pictures

Hand out a sheet of pictures. Ask

students to come out sentences from
the pictures. Or ask students to tell
a stories.
Enough grammar knowledge to make
sentences by students.
Accuracy

Memory Test

Give limited time for students to see

the picture. Ask them to describe the
picture without seeing it.
Testing students ability of making
sentences.(enough grammar
knowledge)
Accuracy
The transformation of students
sentences

Picture Dictation

A student try to describe the picture

to other students who havent seen
it.
Testing if a student can make
sentences which can help him/her
communicate well.
Accuracy

Strip Story

Make students into groups and

assign them a text from a strip story.
Ask them to find the order and
resolve the problem from the story.
See if students can find the time
order from the tenses of the
sentences.
Accuracy

Miming an action

Ask students to role play in a limited

situation.
Through acting, it can test students
ability of thinking of lines. (which
may contains different tenses.)
Accuracy

Growing Stories

Story-building activities
Students ability of making sentences
with different tenses

Questionnaires

Turn what have taught in class into a

questionnaire. Get students to
survey each other. (does not contain
fully-written-out questions.)
It can see if students totally
understand what have learned in
class. Also it can see if students can
use it well.

Classroom Observation

Classroom activities
Classroom communication: Elicited
dialogues
Written exercise

Online Activities

In multimedia teaching times, teachers can use

Internet as a way of assessing students ability of
grammar
On-line quizzes help students to have more
chances of practices
On-line tests also provide clear explanation after
finishing the quizzes
Students can surf on these grammar website to
do practice
Teachers can ask students to do these exercise
and to keep record for themselves. Finally report
back for one of the reference of assessment

Online activity : websites

http://
www.ccc.commnet.edu/grammar/index.htm
Ex1. Irregular verbs quiz
http://www.ccc.commnet.edu/cgi-shl/par2_quiz.pl
/irregular_quiz.htm
Grammar Assessment http://
www.prenhall.com/grammarassessment/
Self-study grammar quizzes
http://a4esl.org/q/h/grammar.html
ESL Quizzes Center
http://www.pacificnet.net/%7Esperling/quiz/gra
mmar

Portfolio
Definition:a purposeful collection of student
work that tells the viewer about the student
Works should be organized by chronological
order and category teacher can evaluate
the childs achievements
Works including:
1. Traditional paper and pencil tests
2. Classroom observation
3. Online activities
4. Written assignments

Written assignments

Give students some assignments to

practice at home
Ask students using learned grammar
in class to write journal
Term paper throughout one semester
including several drafts we can see
students progress in grammar.

Advantages of portfolio

Why portfolio is the best way to

assess?
Strengthening students learning
Enhancing the teachers role
Improving processes

Test Evaluation of An Existing Test

Section 2 of the Computer-Based Test

of TOEFL

General Background Information

1. Title: Section 2 of TOEFL-Structure and Written
Expression
2. Author: Education Testing Service (ETS)

Test Evaluation of An Existing Test

3. Purpose of decision :
Section 2 measures the ability to recognize

language that is appropriate for standard

written English.

Test Evaluation of An Existing Test

4. Date of Application:
a. 1962, the formation of a National
Council on the TOEFL
b. July 1998 (Most parts of the world)
c. October 2000 (Asia, including
Taiwan)

Practical Orientation
1.

Ease of administration:
a. Computers needed
b. Convenience

2.

Price: expensive ($130)

3.

Quality of items: clarity

Practical Orientation
4.

Ease of scoring:
a.
b.
c.

5.

Photo score reporting

Computerized scoring (0~30)
Cumulative scoring

Ease of interpretation: transmission

speed.

Practical Orientation
6.

7.

Test construction:
a. Computer-adaptive Testing
b. Complete an incomplete sentence
c. Identify one unacceptable answer form
four underlined words.
Timing: 20-25 questions/ 15-20 minutes
(including pretest questions)

Description of the Computer-Based TOEFL Test www.ets.org

Test Evaluation of An Existing Test

Model Test

Analysis of TOEFL Grammar Test

8 sentence patterns

Basic Sentence & Compound Sentence

Adjective Clauses

Participial Phrases

Appositive

Noun Clauses

Adverbs Clauses

Prepositional Phrases

Expletives

Analysis of TOEFL Grammar Test

15 Answering
Techniques

1.

Word Form ....20 %

9.

Articles .5 %

2.

Word Choice ....

... 14 %

10.

Comparatives and

3.

Verbs ....11 %

4.

Parallelism ......9 %

11.

Conjunctions 4 %

5.

Pronouns ...7 %

12.

Redundancy.1 %

13.

Negative Words 2 %

14.

Word Order .4 %

6.

7.

8.

Singular/Plural Nouns ...

.....6 %

Superlative sentence4 %

Verbal (Infinitives, Gerunds, 15. Misplaced Modifiers ...1 %

and Participles) .....6 %
Prepositions ....6%

Total .100 %

Analysis of TOEFL Grammar Test

Reliabilities:

describes the tendency of a set of scores to

be ordered identically on two or more tests, and it can be estimated
by a variety of statistical procedures.

Table 1. Reliabilities and Standard Errors of Measurement (SEM)

July 1998 June 1999
Reliability

SEM

Listening

0.98

2.76

Structure/Writing

0.88

4.89

Reading

0.88

2.73

Total Score

0.95

10.8

Description of the Computer-Based TOEFL Test www.ets.org

Analysis of TOEFL Grammar Test

Validity

1.

------pure lead, the lead ore is mined, then melted, and

finally refined.
(A) Obtaining
(C) To obtain

2.

3.

(B) Being Obtained

(D) It is obtained

Louisa May Alcott is mainly noted for her very popular

A
B
novel, Little Women, whom she published in 1868.
C
D
Some hangars, buildings used to hold large aircraft, are
A
very tall that rain occasionally falls from clouds that form
B
C
along the ceilings. ( )
D

Analysis of TOEFL Grammar Test

Strength:
Accurate and
objective scores
Careful monitoring
of test
Commitment to
international
education
Expert advisory
committees

Highly skilled test

developers
Longstanding
reliability and
consistency
Ongoing test
improvements
Standardized delivery
procedures

Description of the Computer-Based

TOEFL Test www.ets.org

Analysis of TOEFL Grammar Test

Conclusion

No other test in the world is as reliable as

a standard for measuring a nonnative
speakers ability to use English at the
university level.

Description of the Computer-Based TOEFL Test www.ets.org

END