A FRAMEWORK FOR EVALUATION

RIAS WITA SURYANI / 16178070

1. INTRODUCTION

Evaluation is an activity that includes the collection of relevant information,

interpretation of that information, and making decisions about teaching and learning process.
In the past, discussions about second language evaluation, particularly when testing involved,
have often focused on making decisions about students, their placement, promotion,
advancement, and certification. Certainly, these are important reasons for doing second
language evaluation, but they are not the only ones. In fact, the majority of decisions teachers
make concern instruction-decisions about how and when to teach particular objectives; about
the instructional needs of individuals or groups of students; about the appropriateness of
instructional objectives and plans, and so on. Even decisions about students often call for
choices regarding instruction. For example, decisions to admit particular students to a class or
to promote students to the next level affect the composition of the classroom and may alter
instruction plans for that class. Much discussion about evaluation has also focused on
assessment of student achievement. This makes sense if the primary reason for evaluation is
to make choices regarding students. Decisions about instruction, however, require more than
data on student achievement. They require information about students needs, goals,
preferences, and attitudes towards school and learning.

2. THE CONTEXT AND STRATEGY OF CLASSROOM BASED EVALUATION

The Context of Classroom Based Evaluation

Evaluation in teaching English language is a process of collecting, analyzing and

interpreting information about teaching and learning in order to make informed decisions that
enhance student achievement and the success of educational programs. Evaluation is a
process that includes basic components:

a. Articulating the purpose of the educational system.

b. Identifying and collecting relevant information.
c. Analyzing and interpreting information for learners.
d. Classroom management or classroom decision making.

1
 According to Gensee and Upshur (1996) classroom assessment and evaluation is
concerned primarily with improving instruction so that student learning is enhanced.
Classroom teachers in educational system, more than anyone else, are actively and
continuously involved in assessment and evaluation. Students can also be active participants
in assessing their own achievements and in planning how they will study and learn the second
language. The context of classroom assessment and evaluation can be summarized in the
Figure 1.

As figure 1 shown, (a practical method of measuring based on experience) any

instruction consists of three components; the purposes identify the objectives of instruction.
The plans describe the means of attaining those objectives. And practices are what actually
takes place in the classroom. Gensee and Upshur (1996) also discuss other factors which,
strictly speaking, are not part of classroom instruction themselves, nevertheless, can have a
significant effect on second language teaching and learning. They refer to these additional
factors as input factors. Thus, we can say that context of classroom based evaluation has
four aspects (purposes, plans, practices, and input factors).

The Strategy of Classroom Based Evaluation

Gensee and Upshur (1996) argue that evaluation involve comparison. More specifically,
decisions that result from assessment are arrived at by making comparisons between various
components of instruction and the larger instructional context (including input factors,
purposes, plans, practices, and outcomes) and then taking action to reduce mismatches
between the components so that the desired outcome or match is achieved. If there is no
mismatch, then instruction can proceed without changing anything. Another way of viewing

2
classroom assessment and evaluation requires that you look for potential problems and decide
on actions to resolve them. Problems take the form of mismatches, inconsistencies between
what is actually happening or is likely to happen on the one hand and what you would like to
happen on the other. Mismatches indicate that there is a potential problem; decisions about
changes that will eliminate or reduce the problems. It can be summarized in figure 2 A
strategy for classroom based evaluation.

3. STEPS IN DESIGNING AN EFFECTIVE TEST AS CLASSROOM LANGUAGE

ASSESSMENT

Determining the Purposes of a Test

The first and most important step in designing classroom assessment is to step back and
consider the overall purposes of the exercise that students perform. The purpose of
assessment refer to as test usefulness.

Designing Clear, Unambiguous Objectives

Sometimes teacher give tests simply because its Friday of the third week of the course,
and after hasty glances at the chapter covered during those three weeks, they dash off some
test items so that students will have something to do during the class. This is no way to
approach a test. Instead, begin by taking a careful look at everything that you think your
students should know or be able to do based on the material that the students are
responsible for. In other words, examine the objective for the unit you are testing. Objectives

3
that are stated in terms of overt performance by students. Thus, an objectives that states
students will learn tag Questions or simply names the grammatical focus Tag Question is
not test cable. Thus, the important thing is to determine appropriate objectives.

Drawing up Test Specifications

Test specifications for classroom use can be a simple and practical outline of your test.
Test specifications are much more format and detailed. The specifications will simply
comprise (a) a broad outline of the test (b) what skills you will test, (c) what the items will
look like.

Devising Test Tasks

It is important to note that test development is not always a clear, linear process. In
reality, test design usually involves a number of loops as your discover problems and other
shortcomings. When design final exam, it is important to consider the age of students.

Administering the Test

When design the test, it based on considered purposes, objectives, and test
specification. Then, make sure that the actual administration of the test accomplished.

4. THE FRAMEWORK OF TEST METHOD FACETS

Testing Environment

Test takers might be expected to perform differently under differing environmental

conditions. Testing environment includes 4 facets (aspects):

Familiarity of the place and equipment used in administering the test (familiar place:
less threatening equipment: pencil versus computer)
The personnel involved in the test: a superior, a peer, or a subordinate)
The time of testing: early in the day, just after a heavy noon meal
Physical conditions: noise, temperature, humidity, seating arrangement, lighting)

Test Rubric

4
Test rubric specifies how test takers are expected to proceed in taking the test. Test rubric
includes 3 main facets:

Test organization: Tests consists of different parts (i.e. salience of parts - sequence of
parts - relative importance of parts)
Time allocation: The amount of time allocated for the test or its parts is likely to affect
test performance.
Instructions: Performance depends on how well the test conditions are understood.
Facets of instruction includes: (i.e. language channel - specification of procedures &
tasks - explicitness of criteria for correctness)

Input

It related to the information contained in a given test task, to which the test taker is expected
to respond. Input includes 2 main facets:

Input format
- Channel of presentation: aural, visual (e.g. listening comprehension test)
- Mode of presentation: Input will be in receptive mode
- Form of presentation: language (e.g. reading passage), nonlanguage (e.g.
pictures), both (e.g. a passage with tables)
- Vehicle of presentation: live vs. canned human input (e.g. tape recording)
- Language of presentation: native, target, both
- Identification of the problem: specific (e.g. correcting errors underlined) general
(e.g. identifiying and correcting an unspecified number of errors)
- Degree of speededness: Input may be perceived as speeded if test taker perceives
speed as a factor in performance. The perception of speededness vary from one
test-taker to another.
Nature of language input: When the form of the input is language, that language can
be characterized by its
- Length: Not a critical facet, but longer language samples include more potential
effects in terms of
- Propositional content: Vocabulary (frequency, specialization), degree of
contextualization (embedded/ reduced), distribution of new information
(compact/diffuse), type of information (concrete/abstract, positive/negative,
factual/counterfactual), topic, genre
- Organizational characteristics: grammar, cohesion, rhetorical organization
- Pragmatic characteristic: illocutionary force, sociolinguistic characteristics
(dialect or variety)

5
Expected Response

Format of expected response

- Channel: aural, visual (Listening comprehension test: marking the correct choices
on multiple-choice questions / responding speaking
- Mode: productive
- Type of response: selected (multiple choice tests), constructed (production of a
language sample interview tests of communicative proficiency)
- Form of expected response: language, non-language (making a mark, drawing a
picture), both
- Language of response: native, target, both
Nature of language
- Length: Not a critical facet, but longer language samples include more potential
effects in terms of
- Propositional content: Vocabulary (frequency, specialization: technical registers,
slangs), degree of contextualization (embedded/ reduced), distribution of new
information (compact/diffuse), type of information (concrete/abstract,
positive/negative, factual/counter-factual), topic, genre
- Organizational characteristics: grammar, cohesion, rhetorical organization
- Pragmatic characteristic: illocutionary force, sociolinguistic characteristics
(dialect or variety, register)

Restrictions on response

- Channel: variety of conditions (face-to-face, in quiet rooms, phone conversations)

- Format: restricted: selection/identification response types; unrestricted:

composition test

- Organizational characteristics: grammatical forms (multiple choice vocabulary

item: test-taker deals with the meanings of specific words), organization of
discourse (following a specific rhetorical pattern: comparing & contrasting,
arguing for & against)

- Propositional and illocutionary characteristics: Language tests restrict

illocutionary force of responses: Picture description tests: to describe what is
given in a picture

6
 - Time or length of response: In testing, situations, administrative considerations
place additional restrictions on time and length.

The Relationship between Input and Response

Reciprocal input and response

- Can be defined as the use of language by one individual to produce an effect in
another individual through the reduction of uncertainty with knowledge of results.
- One individual - another individual: a sender and a receiver need to be involved.
- To produce an effect: indicates that the language use has a communicative goal,
or illocutionary intention.
- Reduction of uncertainty: characterizes the means by which the communicative
goal is effected, that is, by means of a change in information at the disposal of the
receiver.
- There is an interaction and a feedback: Ex. a well conducted oral interview

Nonreciprocal input and response

- Nonreciprocal language use: there is no interaction between language users.
- There is no feedback: Ex. reading a book.

Adaptive input and response

- The relationship between input and response is adaptive if the input is influenced
by the response, but without the feedback that characterizes a reciprocal
relationship.

- An individual taking an adaptively administered multiple-choice test, for

example, typically begins with an item that is of medium difficulty.

5. CONCLUSION

An effective, goal-oriented, teaching-learning sequence contains clearly understood

objectives, productive classroom activities, and a sufficient amount of feedback to make
students aware of the strengths and weaknesses of their performances. Feedback and
evaluation are inseparably related to both instructional objectives and classroom learning
activities and are indispensable elements in the learning process. Classroom assessment and
evaluation is like a feedback. Loop-assessment activities are motivated and shaped by
instructional purposes, plans and practices in the classroom and decisions that arise from the

7
results of these activities in turn lead to reshaping of these instructional purposes, plans and
practices. Effective classroom assessment and evaluation requires an understanding of the
role of evaluation in planning and delivering instruction. Finally, an effective classroom
assessment and evaluation calls on teachers to become agents of change in their classrooms
actively using the results of assessment to modify and improve the learning environments
they create.

REFERENCES

Bachman, Lyle F. 1990. Fundamental Consideration in Language Testing. Oxford: Oxford

University Press.

Brown, H. Douglas and Priyanvada Abywickrama. 2010. Language Assessment: Principles

and Classroom Practices. NY: Pearson Education.

Genesee, Fred and Upshur, John A. 2002. Classroom-based Evaluation in Second Language
Education. Cambridge: Cambridge University Press.