SUMMARY #26

DEASTIKA BAYUNING SUDJASMARA

140247
8

ASSESSMENT AND RESEARCH

TERMINOLOGY AND CONCEPTS
The term assessment refers to a variety of ways of collecting information on a learner's
language ability or achievement. Although testing and assessment are often used
interchangeably, the latter is an umbrella term encompassing measurement instruments
administered on a 'one-off basis such as tests, as well as qualitative methods of
monitoring and recording student learning such as observation, simulations or project
work. Assessment is also distinguished from evaluation which is concerned with the
overall language program and not just with what individual students have learnt.
Proficiency assessment refers to the assessment of general language abilities acquired
by the learner independent of a course of study. On the other hand, assessment of
achievement aims to establish what a student has learned in relation to a particular
course or curriculum (thus frequently carried out by the teacher).
Assessment carried out by teachers during the learning process with the aim of using the
results to improve instruction is known as formative assessment. Assessment at the end
of a course, term or school year - often for purposes of providing aggregated information
on program outcomes to educational authorities - is referred to as summative
assessment.
The interpretation of assessment results may be norm-referenced or criterionreferenced. Norm-referenced assessment ranks learners in relation to each other.
Criterion-referencing occurs when learners' performance is described in relation to an
explicitly stated standard. The two key requirements for any assessment are that it
should be valid and reliable. In the field of language assessment, a distinction is made
between three types of validity:

Construct validity: the extent to which the content of the test/assessment reflects
current theoretical understandings of the skill(s) being assessed;
Content validity: whether it represents an adequate sample of ability; and
Criterion-related validity: the extent to which the results correlate with other
independent measures of ability.

PURPOSES
Assessment is carried out to collect information on learners' language proficiency and/or
achievement that can be used by the stakeholders in language learning program for
various purposes. These purposes include:

Selection: e.g. to determine whether learners have sufficient language proficiency

to be able to undertake tertiary study;
Certification: e.g. to provide people with a statement of their language ability for
employment purposes;
Accountability: e.g. to provide educational funding authorities with evidence that
intended learning outcomes have been achieved and to justify expenditure;
Diagnosis: e.g. to identify learners' strengths and weaknesses;

SUMMARY #26
DEASTIKA BAYUNING SUDJASMARA

140247
8

Instructional decision-making: e.g. to decide what material to present next or what

to revise;
Motivation: e.g. to encourage learners to study harder.

Background
In a series of research studies, John Oiler and his colleagues found strong relationships
between testers' performance in integrative tests and in the sub-components of various
other
test batteries testing other language skills, such as writing and speaking. On the basis of
these
findings, Oiler hypothesized that there was a single general proficiency factor which
underlay
test
performance. This became known as the 'unitary competence hypothesis' (Oiler 1976;
Oiler
and
Hinofotis 1980). However, in the face of critiques of the methodology used in the studies,
Oiler
(1983: 353) subsequently modified the hypothesis, acknowledging that language
proficiency
was
made up of multiple components. It is now generally accepted that a single test of
overall
ability,
such as a cloze passage, does not give an accurate picture of an individual's proficiency
and
that
a
range of different assessment procedures are necessary (Cohen 1994: 196).
Many language tests and assessments used nowadays often contain tasks which
resemble the kinds of language-use situations that test takers would encounter in using
the language for communicative purposes in everyday life. The kinds of tasks used in
communicative assessments of proficiency and achievement thus typically include
activities such as oral interviews, listening to and reading extracts from
the media and various kinds of 'authentic' writing tasks which reflect real-life demands
(for
a
range of examples, see Weir 1990, 1993).
Research
There has been an enormous amount of research activity in language assessment in
recent
years;
for a comprehensive overview, see Clapham and Corson (1997). This volume contains
state-of-heart surveys of a wide range of current issues in language assessment. A
summary of trends in language testing is also provided by Douglas (1995), while
Shohamy (1995) discusses the particular issues and problems involved in the
assessment of language performance. Hamayan (1995) describes a variety of
assessment procedures not involving the use of formal tests. Kunnan (1997) categorizes
over a hundred language testing research studies in terms of the framework for
language test validation proposed by Messick (1980). Current developments and
research in the assessment of the skills of listening, speaking, reading and writing,
respectively, are provided by Brindley (1998b), Turner (1998), Perkins (1998) and Kroll
(1998).

SUMMARY #26
DEASTIKA BAYUNING SUDJASMARA

140247
8

The brief review of research below focuses on two important issues in language
assessment:
(1) the key question of how to define language ability and (2) self-assessment of
language
ability.
Among the many important research topics that would merit attention in a longer review
include
the relationship between test-taker characteristics and test performance (e.g. Kunnan
1997), testtaker strategies (e.g. Storey 1998), test-taker discourse (e.g. O'Loughlin
1997),
factors
influencing
task difficulty (e.g. Fulcher 1996a), rater and interviewer behavior (e.g. Weigle 1994;
McNamara
1996; Morton et al. 1997) and applications of measurement theory to test analysis (e.g.
Lynch
and
McNamara 1998). The important question of the impact of assessment and testing on
teaching
and learning (known as washback) is also beginning to receive a good deal of attention
in
the
language assessment literature (Alderson and Wall 1993) as are issues of ethics and
fairness
(Hamp-Lyons 1998)
RESEARCH INTO THE NATURE OF COMMUNICATIVE LANGUAGE ABILITY
Many assessment specialists would argue that the fundamental issue in language
assessment
is
that
of construct validity. In other words, as Spolsky (1985) asks: 'What does it mean to know
how
to
use a language?' To answer this question, it is necessary to describe the nature of the
abilities
being
assessed; this is known as construct definition. Thus, if we are developing a test of
'speaking',
we
need to be able to specify what we mean by 'speaking ability', i.e. what its components
are
and
how these components are drawn on by different kinds of speaking tasks.
In recent years, two approaches to construct definition have been adopted (McNamara
1996). The first approach focuses on compiling detailed specifications of the features of
target language performances which learners have to carry out, often on the basis of an
analysis of communicative needs (Shohamy 1995). These features form the criteria for
assessment and are built into assessment instruments such as proficiency rating scales;
e.g. see the well-known scale used by the American Council on the Teaching of Foreign
Languages (ACTFL 1986) to assess language ability of foreign language teachers.
The second approach, rather than starting with an analysis of the language that the
learner needs to use, employs a theoretical model of language ability as a basis for
constructing tests and assessment tools; see Canale and Swain 1980; Bachman 1990
(the latter updated by Bachman and Palmer 1996). Such models provide a detailed and

SUMMARY #26
DEASTIKA BAYUNING SUDJASMARA

140247
8

explicit framework for describing the types of abilities involved in communicative

language use and have been drawn on in a number of language test construction and
validation projects (e.g. Harley et al. 1990; Bachman et al. 1995; McKay 1995; Milanovic
et al. 1996; Chalhoub-Deville et al. 1997).
RESEARCH INTO SELF-ASSESSMENT
There has been a considerable growth of interest in the use of self-assessment with
language learners in various educational settings (Oscarson 1997). Proponents have
argued that participating in self assessment can assist learners to become skilled judges
of their own strengths and weaknesses and to set realistic goals for themselves, thus
developing their capacity to become self-directed (Dickinson 1987; Oscarson 1997).
Research suggests that with training, learners are capable of self-assessing their
language ability with reasonable accuracy (Blanche and Merino 1989).
Research into the use of self-assessment has provided a number of insights that can
usefully inform language teaching practice. First, evidence suggests that the concept of
self-assessment may be quite unfamiliar and threatening to many learners since it alters
traditional teacher-learner relationships (Blue 1994; Heron 1988). Guidance in the use of
self-assessment techniques is therefore crucial (Cram 1995; Dickinson 1987). Second,
the ability of learners to self-assess accurately appears to be related to the transparency
of the instruments used. Some research studies suggest that learners find it easier to
say what they cannot do, or what they have difficulty doing, than what they can do
(Bachman
and
Palmer
1989;
Ready-Morfitt
1991).
This
finding
has
implications for the way in which self-assessment scales are worded. Third, learners
seem
to
be
able to assess their abilities more accurately when the self-assessment statements are
couched
in
specific terms and are closely related to their personal experience (Oscarson 1997; Ross
1998).
Finally, some evidence suggests that cultural factors affect learners' willingness to selfassess
as
well as the accuracy of these assessments (Blue 1994; von Elek 1985).