Vol 24 1

TABLE OF CONTENTS
TESOL QUARTERLY Volume 24, Number 1 ❑ Spring 1990
A Journal for Teachers of English to Speakers of Other Languages

and of Standard English as a Second Dialect
Editor
SANDRA SILBERSTEIN, University of Washington
Review Editor
HEIDI RIGGENBACH, University of Washington
Brief Reports and Summaries Editor
GAIL WEINSTEIN-SHR, Temple University
Assistant Editor
LINDA STOLFI, University of Washington
Editorial Assistant
MAUREEN P. PHILLIPS, University of Washington
Editorial Advisory Board
Elsa Auerbach Elliot L. Judd
University of Massachusetts at Boston University of Illinois at Chicago
Margje S. Berns Ilona Leki
Purdue University University of Tennessee
James D. Brown
University of Hawaii at Manoa Mary McGroarty
Ruth Larimer Cathcart Northern Arizona University
Monterey Institute of International Studies James W. Tollefson
Graham Crookes University of Washington
University of Hawaii at Manoa Leo van Lier
Patricia A. Dunkel Monterey Institute of International Studies
The Pennsylvania State University Lise Winer
Joan Eisterhold Carson Southern Illinois University
Georgia State University
Liz Hamp-Lyons Vivian Zamel
University of Michigan University of Massachusetts at Boston
Additional Readers
Patricia Byrd, Mark A. Clarke, Richard R. Day, Joanne Devine, Catherine Doughty, Karl Krahnke,
Nora Lewis, Judith Oster, William Harshbarger, Barbara Johnstone, Robert B. Kaplan,
Martha C. Pennington, Pat Rigg, Terry Santos, Andrew F. Siegel, Barry P. Taylor
Credits
Advertising arranged by Helen Kornblum, TESOL Central Office, Alexandria, Virginia
Typesetting, printing, and binding by Pantagraph Printing, Bloomington, Illinois
Design by Chuck Thayer Advertising, San Francisco, California
VOLUMES MENU
TESOL QUARTERLY
CONTENTS
To print, select PDF page

nos. in parentheses
ARTICLES
What is the Role of Language Planning in
Post-Apartheid South Africa? 9 (10-22)
Carol M. Eastman
A Context-Adaptive Model of Program Evaluation 23 (24-43)
Brian K. Lynch
Managing the Complexity of Revising Across Languages 43 (44-61)
Chris Hall
Another Turn in the Conversation:
What does Cloze Measure? 61 (62-84)
Jon Jonz
REVIEWS
Alien Winds: The Reeducation of America’s Indochinese Refugees 85
James W. Tollefson
Reviewed by Elsa Auerbach
Academic Writing: Techniques and Tasks 91
Ilona Leki
Reviewed by Mark Dunning
The Learner-Centred Curriculum: A Study in
Second Language Teaching 95
David Nunan
Reviewed by Pat Rigg
BRIEF REPORTS AND SUMMARIES

Writing-Proficiency Tests and Remediation:
Some Cultural Differences 99
Roger M. Thompson
THE FORUM
Comments on Bronwyn Norton Peirce’s
“Toward a Pedagogy of Possibility in the
Teaching of English Internationally:
People’s English in South Africa” 103
A Reader Reacts . . .
Betty Lou Dubois
The Author Responds . . .
Bronwyn Norton Peirce
Comments on Alan Beretta’s
“Attention to Form or Meaning?
Error Treatment in the Bangalore Project” 112
N. S. Prabhu
The Author Responds . . .
Alan Beretta
Information for Contributors 117

Editorial Policy
General Information for Authors
Publications Received 121
Publications Available from the TESOL Central Office 125
TESOL Membership Application 152
TESOL QUARTERLY
TESOL QUARTERLY
Editor's Note
Readers will note that we have replaced the (monthly) cover date with
a seasonal designation. This was a difficult decision for TESOL and the
TESOL Quarterly editorial staff. We recognize with concern that this
change may lend an unwanted appearance of Northern Hemisphere
provincialism to the journal, and we regret the incongruity these labels
may create for readers in the Southern Hemisphere. Unfortunately, this
proved the only scheme that satisfies both our need for flexibility in the
production process, and the bibliographic and accounting constraints of
subscribing institutions. We trust that our colleagues in the Southern
Hemisphere will overlook this bibliographic artifact and credit our good
will.
I am fortunate that Heidi Riggenbach, Assistant Professor of English at the
University of Washington, has accepted my invitation to serve as Review
Editor. With more than 10 years of experience in the field of TESOL,
Heidi has worked as an instructor and researcher in the United States,
China, and Malaysia, and has authored numerous publications in the areas
of language use, oral skills, and grammar.
In This Issue
The titles of two articles in this issue are framed as questions,

highlighting the fact that each of the articles described below addresses
current questions in applied linguistics. The first contribution responds to
an especially timely issue. The paper is a distillation of remarks delivered
in February 1990 at a conference designed to respond to the African
National Congress’ identification of language as a planning issue for post-
apartheid South Africa. Other papers speak to issues raised with regard to
program evaluation, first and second language revision practices, and
cloze testing.
5
• Carol Eastman asks what the role of language planning can and should
be in post-apartheid South Africa. Drawing on research in political and
sociolinguistics, Eastman notes that language use reflects speakers’
multiple identities, and their attitudes toward the languages in
question. Expecting that the future of South Africa will be “strongly
involved with English—variously realized and regionalized,” she
argues that “it would seem prudent to use sociolinguistic insights to
support [these] language practices.” Eastman advocates “a politically
minimal approach to language planning” supporting currently
developing language attitudes and practices.
• Brian Lynch contributes to the applied linguistics literature on program
evaluation. He uses examples from the REST Project at the University
of Guadalajara in Mexico to describe the context-adaptive model for
program evaluation. Responding to the need for a comprehensive
evaluative framework, the model consists of a series of seven steps
“designed to guide the program evaluator through consideration of the
issues, information, and design elements necessary for a thorough
evaluation.” The model is distinguished by its flexibility, allowing
“effective adaptation and refinement as it is implemented in a variety
of settings.
• Chris Hall investigates the relationship between first and second
language revision practices. He examines the writing processes of four
advanced ESL students, each of whom wrote two argumentative
essays in the native language and two in English. Hall analyzes the type
and purpose of revisions made during successive stages of the writing
process. The results reveal striking similarities between the
characteristics of L1 and L2 revisions and the stages at which they were
initiated. Differences are also noted “suggesting that while proficient
writers are capable of transferring their revision processes across
languages, they are also capable of adapting some of those processes”
to problems imposed by writing in a second language.
• Jon Jonz’ study addresses a controversy in cloze testing. At issue is
what the cloze procedure measures; that is, what information is
required to supply a correct response to a cloze item. Some researchers
have argued that cloze is a relatively local affair, drawing primarily on
low-level syntax. In contrast, Jonz finds that the standard fixed-ratio
cloze procedure draws heavily on knowledge beyond local syntax. His
analysis of eight cloze passages uses a system that estimates the
quantity of text required to cue the closure of each blank, and notes the
linguistic category of the deleted word. The kinds of language
knowledge required to complete cloze items were virtually the same
for each passage examined.
6 TESOL QUARTERLY
Also in this issue:
● Reviews: Elsa Auerbach reviews James Tollefson’s critical examination
of the U.S. overseas refugee education program in Alien Winds: The
Reeducation of America’s Indochinese Refugees; Mark Dunning
reviews Ilona Leki’s Academic Writing: Techniques and Tasks; and Pat
Rigg reviews David Nunan’s The Learner-Centred Curriculum.
● Brief Reports and Summaries: Roger Thompson reports cultural
differences in student responses to a required writing-proficiency test.
● The Forum: Betty Lou Dubois’ commentary on Bronwyn Peirce’s
recent TESOL Quarterly article, “Toward a Pedagogy of Possibility in
the Teaching of English Internationally: People’s English in South
Africa,” is followed by a response by the author; Alan Beretta
contributes a short explanation for his decision not to respond to
comments by N. S. Prabhu on Beretta’s recent TESOL Quarterly
article, “Attention to Form or Meaning? Error Treatment in the
Bangalore Project.”
IN THIS ISSUE 7
TESOL QUARTERLY, Vol. 24, No. 1, Spring 1990
What is the Role of Language

Planning in Post-Apartheid
South Africa?
CAROL M. EASTMAN
University of Washington
The applied branch of linguistics known as language planning

(LP) is traditionally seen as a way to provide governments with
policy-level guidance regarding the choice of official and national
languages. This paper examines LP as a concern of both political
and sociolinguists in the context of a post-apartheid South Africa.
For this context, I suggest a politically minimal approach to
language planning.
The African National Congress (ANC) has identified language as

a planning issue for post-apartheid South Africa. In anticipation of
that fact, a colloquium was held recently (January 30-February 2,
1990) at the University of the Witwatersrand in Johannesburg. The
colloquium received selective support from the Union of
Democratic University Staff Associations (UDUSA) so that scholars
familiar with language problems throughout Africa might come
together to share ideas.l This article is a distillation of my remarks
at that meeting.
On the last day of the colloquium, proceedings were interrupted
by the announcement that the ban on both the ANC and SACP
(South African Communist Party) was being lifted and that a num-
ber of political prisoners were being released. This act was followed
during the coming week by the release of Nelson Mandela from 27
years of imprisonment in Cape Town. At the risk of understate-
ment, it was a remarkably exciting time to be there.
My remarks at the colloquium, for the most part, were restrict-
ed to an overview of the branch of applied linguistics related to
language planning (LP) or language policy. I focused on research that
1 In keeping with guidelines accompanying the UDUSA statement of selective support for
the colloquium, I stated my opposition to apartheid and my active promotion of a “non-
racial democratic South Africa” (UDUSA Congress resolution, 1989).
9
had been done to date on the African continent. As most readers
probably know, the term language planning generally refers to
efforts in a sociopolitical context to solve language problems,
preferably on a long-term basis, by studying the process of
language change. LP focuses on the differential allocation of
languages and language varieties in a community speech economy
context. Traditionally, planners are wielders of political and
economic power in a state, nation, or nationalizing entity. The
question addressed here is what the role of language planning can
and should be in post-apartheid South Africa.
LANGUAGE PLANNING BEFORE LP

Prior to the advent of language planning as an applied branch of
sociolinguistics, language planning nonetheless took place. In fact,
one of the earliest LP debates developed among missionaries in
South Africa over the issue of orthographies for Xhosa and Zulu.
Similarly in East Africa, missionaries made the decision that the
Zanzibar dialect of Swahili should become the national language
rather than the dialect spoken in Mombasa (Eastman, 1983). Such
decisions have had ramifications for the way these languages are
perceived today, both structurally and politically.
Of course missionaries are not the only “lay” planners afoot in
Africa. Any decision regarding the language of instruction amounts
to a language plan, often made by the nonlinguist. Literacy
instruction, for example, is a language planning endeavor. In recent
years, in an apartheid context of “separate but equal” public
education, English made sense, and educators chose it as the
language of education.
Along with language planning by educators and missionaries,
some situations have simply evolved. Janson (cited in Romaine,
1988) reports that in urban areas of South Africa a new, mixed Bantu
creole is emerging in black townships in reaction to the urban use of
Afrikaans:
The enforced move of many traditional black urban dwellers into black
ghetto townships like Soweto has resulted in a shift away from Afrikaans
in the direction of local Bantu languages. . . . This new language, called
Taal or Tsotsi Taal, is popular because it is not understandable to white
Afrikaners. (p. 194)
Employers, too, may be involved in language planning. In the
mid-1980s in Southwest Africa/Namibia, suggestions were made to
implement an “English for Mining Purposes” in an effort to create a
situation of English diglossia, where two varieties of English would
10 TESOL QUARTERLY
exist in a hierarchical relationship, i.e., a special low-prestige version
of English was developed for use in the mines. This represents just
one of many forms of English for Special Purposes that have been
proposed for various situations in the world.
APPLIED LINGUISTICS AND LANGUAGE PLANNING

Patterns of language use throughout Africa indicate that language
planning should not be up to missionaries, educators, employers, or
politicians. I would argue that language-policy decisions should be
made in the context of available research in the linguistically
oriented social sciences. This paper focuses on research contribu-
tions in linguistics, particularly those made by political and
sociolinguists.
The political linguist examines language policy at the level of the
nation or the state, while the sociolinguist looks at policies affecting
interpersonal linguistic interaction. In a paper about language
conflict in Kenya, David Laitin and I (1989) asked which
perspective offers a better understanding of the future of language
in that country. Further, we explored the issue of how public policy
could enhance language welfare. We found that each perspective
provides a partial answer. Here, I attempt to see how knowledge
about language planning from these two perspectives might be of
use in the context of an emerging post-apartheid South Africa.
LANGUAGE IN POST-APARTHEID SOUTH AFRICA

LP for the Political Linguist
Linguistically oriented political scientists such as Laponce analyze
the linguistic role of the state, considering it as important as the
study of state politics and economics. Linguistic cleavages are
universal obstacles to communication. Laponce (1987) argues that
there are either personal or territorial solutions available to
politicians: “Personal solutions seek to establish language as a right
equally as portable as the right to vote or the right of religious
expression” (p. 165). Personal solutions are expressed as, e.g., the
right to speech, to ethnic identity, to education, etc. “Territorial
solutions stem from the principle that languages in contact should
be separated as much as possible by means of fixed frontiers that
give a feeling of security” (pp. 172-173).
Laponce is an advocate of territoriality, and provides an example
of success in what he calls the Belgian model. The guiding principle
of territoriality is the separation of languages in contact. In Belgium,
LANGUAGE PLANNING IN POST-APARTHEID SOUTH AFRICA 11

where there are French and Dutch regions, regulations dictate that
40% of public service officials are to be francophones, 40% to be
Dutch speakers, and the remaining 20% to be an equal number of
French-dominant and Dutch-dominant bilingual. What actually
happens varies from ministry to ministry, ranging from a split in the
Ministry of National Education, to situations in which a ministry
director will be surrounded by like speakers, with linguistic
dominance changing according to the ethnicity of the occupants of
various offices. This latter situation is seen by Laponce to confirm
“a normal consequence of bilingual administration,” i.e., one
language establishes itself as dominant (p. 181). Where language
mingling is discouraged, as in both the Ministry of National
Education (with parallel unilingual networks) and the Ministry of
the Interior (with unilingual regional offices), equal use of the two
languages is relatively ensured.
Interestingly, as an attempt at language maintenance. this Belgian
model was tried and failed in South Africa; Laponce (1987) reports
that
a system of mandatory alternation between the two official languages
was introduced for internal communication within some ministries—one
month English, the next month Afrikaans, and so on in sequence. The
experiment was abandoned when it was realized that officials were
frequently withholding replies and initiatives until their preferred
language was the language of the day. In the absence of strict rules, the
South African practice reverted to what it had been before the
experiment: The language used for internal communication was the
dominant language of the office or the ministry involved, though
endeavour was made to answer in the language in which the question,
note, or memorandum was couched. In actual fact, Afrikaans is the
dominant language in almost 90 per cent of [governmental] internal
communication. (p. 181)
By Laponce’s territorial model, English and Afrikaans were not
sufficiently separated at the level of state administration, to the
detriment of English use. Interestingly in the black homelands,
however, the very lack of adherence to the idea of language
separation appears to have led to the emergence of English. And,
under apartheid, the use of the vernaculars at the level of state
administration has not been part of the picture. It appears, then, that
the territorial model cannot completely explain current language
use in South Africa (e.g., in the case of the homelands), or provide
direction for language policy (e.g., with respect to the use of
vernaculars).
Nonetheless, for political linguists like Laponce, bi- and
multilingual state-level solutions to language problems have a
12 TESOL QUARTERLY
number of advantages. To avoid choosing a single language means
not having to play favorites or give the advantage to a neighbor.
Further, according to Laponce (1987), in situations such as South
Africa and Belgium,
demography and democracy have overturned the ethnic hierarchy; the
subordinate ethnic group displaced the dominant group before the latter
managed to establish its language. The bilingualism that facilitates the
rise of the subordinate ethnic group, and makes the loss of status more
acceptable to the ethnic group on the decline, is admirably suited to
these historic changes of position. (p. 191)
For the political linguist, personal solutions to state language
problems are too messy.
Political linguists often talk about state rationalization of language
policy. This is the situation when the national context makes it nec-
essary for citizens to use a national language. To exemplify, consid-
er a rural dweller with no interest in the government language.
Nevertheless, government policies make it difficult for the rural
person to receive goods and services without some form of accomo-
dation to the government’s policy. The rural person, in effect, has to
pay the “transaction costs” necessary here; thus the government
"wins"—"rationalization" has taken place in the interest of center/
periphery communication (Laitin, 1989, p. 24). Here, the political-
linguistic explanation focuses on the relative strength of the state in
which rationalization occurs in contrast to states where it does not.
From the perspective of occupational mobility, Laitin (1989) sees
essentially three language situations in modern Africa: those in
which people use three (plus or minus one) languages (citizens are
multilingual, with some people knowing as many as four languages,
while others may know only two); those in which people use two
languages (citizens are bilingual, retaining their vernaculars while
relying on an international language as a lingua franca); and those in
which people use one language (citizens use a single national/
official language that serves as a lingua franca).
Laitin (1989) sees pressure toward a two-language formula in an
independent South Africa and Namibia. The linguae francae
Fanagalo and Afrikaans are both ideologically unacceptable
candidates to become official languages:
The likelihood is that, in both countries, English would become the
language of international and intra-elite communication, while
Afrikaans, Zulu, Oviwambo, etc., will become languages consolidated in
separate regions. In the cities, the principle of personality (choose the
mother tongue school of your identification) could be promoted. This
would consolidate the two-language outcome. (p. 46)

In South Africa and Namibia, as a result of group and regional
identities, Laponce’s principle of territoriality (separation of
languages) describes the current situation. State rationalization has
not taken place. As African states become independent, they
commonly do so with both Ministries of Education and state
bureaucracies already in place prior to the consolidation of a
political class. Thus, the new political class is unlikely to be able to
force rationalization. For Laitin, this explains the possibility of the
three (plus or minus one)- and two-language outcomes over state
rationalization.
Political linguists, then, can tell us a good deal about language use
at the level of the state. From the perspective of political linguistics,
the future of the African continent appears to be a multilingual one.
In South Africa, however, from a sociolinguistic standpoint, that
future will be seen to be strongly involved with English—variously
realized and regionalized. While political-linguistic analysis can
describe state-level language situations, a troubling question arises:
Is there any need for state-level language policy?
LP for the Sociolinguist

As Laitin (1989) has observed, state-level language outcomes are
not socially neutral. Nonetheless, the policies that affect and
produce them are often devised in a vacuum. The sociolinguist
approaches language problems from quite a different vantage point
than the political linguist. The sociolinguist works from the
perspective of the speech community rather than the state. Within a
speech community, people share “rules for the conduct and
interpretation of speech” and rules for the interpretation of at least
one “linguistic variety” (Hymes, 1972, p. 52). The sociolinguist looks
at the entire repertoire of language use within a speech community.
Unlike states, speech communities cannot be seen as employing
separate linguistic systems for distinct purposes. Instead, people
interact in a speech community using varieties of language(s)
according to the various situations in which they find themselves,
and with regard to their own gender, class, and socioeconomic
status and that of the people with whom they are interacting.
Sociolinguistic solutions to language problems take the form of
explaining and accounting for appropriate language use in different
situations. Where the political linguist presumes to tell officials from
the perspective of the state what language ought to be the medium
of instruction in Soweto schools, the sociolinguist can tell them what
languages students will be using when they talk to each other and
14 TESOL QUARTERLY
when they talk to teachers. Where the political linguist’s interest in
language planning is the solution of language problems, the
sociolinguist’s chief concern is accounting for the functional
allocation of language use. As Myers-Scotton has found in much of
her work (see, for example, Myers-Scotton, 1988), patterns of
language use reveal that people switch from one language to
another in order to negotiate positions of power. Her work (and that
of others interested in this codeswitching phenomenon) demon-
strates that, despite legal mandates, speakers’ language choices
reveal their multiple identities.
Some sociolinguists focus on language attitudes as reflected in
language use. Almost 20 years ago, Heinz Kloss conducted a
sociolinguistic case study of South Africa. His specific concern was
language in education. Citing 1970 census data in his 1974 study,
Kloss (1978) found that 3.5 million people spoke Afrikaans as a first
language, and 1.5 million spoke English; of those 5 million, 1 million
were bilingual. There were 4 million first language speakers of Zulu
and 4 million of Xhosa. Afrikaans had become a second language
for large numbers of both whites and nonwhites, including first
language speakers of Tamil, Hindi, and other Asian languages.
English and Afrikaans were both official languages, though in the
black homelands various local languages served an official role.
At the time of the Kloss study, the now controversial Mother
Tongue Policy (MTP) was in place, according to which all people
were to be schooled in their first language. This adherence to the
principle of personality led to a situation in which those who were
poor and unable to go beyond primary grades were cut off from
access to science and technology, to political and economic power,
and to jobs. Though conforming to the UNESCO mandate in The
Use of Vernacular Languages in Education: The Report of the
UNESCO Meeting of Specialists, 1951 (cited in Fishman, 1968),
such a policy has had dubious educational benefits for most
speakers of Bantu languages, whether in South or East Africa. (See
Eastman, 1983, for a discussion of this UNESCO policy.) The only
exception to the MTP, prior to the Soweto uprising (June 16, 1976),
was for Indians educated in state schools in English. While a
number of Bantu languages such as Xhosa, Zulu, and Sotho have
gradually produced a rather sizeable body of literature, only
Afrikaans and English are used in science and technology at this
time. Indeed, Afrikaans is remarkable as an indigenous language,
having attained full university status.
The current context favors the use of English. Although it has
been reported that there are 80,000 blacks who speak Afrikaans as
a first language (Kotze, 1987), the shift to English as a first language,

documented by Kloss (1978), probably remains: Urban blacks,
“better off economically and culturally, more literate and more
vocal than their rural ‘fellow ethnics’” (p. 29), evidence a marked
shift to English. It is interesting that these Bantu speakers had been
exposed to first language instruction the longest! Due both to
educational policy and to neglect on the part of Afrikaners, Asians
(Indians) show rather dramatic amounts of shift to English as a
second language.
Not surprisingly, by the time of the Kloss study, substantial
Bantu-speaker opposition had developed to first language instruc-
tion. Increasingly, English and Afrikaans were being used in what
were known in the 1950s as “dual second-language medium schools”
(Kloss, 1978, p. 42). First language Bantu speakers were expected to
learn both English and Afrikaans, with the result that each was
learned poorly; by the 1960s, school districts began to choose one or
the other for use in school as a second language. Most, but not all,
Bantu-speaking area schools selected English, creating an acute
shortage of English-language school teachers. This, coupled with a
desire to introduce English at ever lower grades, made for a quite
chaotic situation, leading one English school official interviewed by
Kloss to conjecture that Afrikaans might, in the long run, be the
winner “since the present pressure to expand the teaching of English
in all directions might lead to the creation of a low-quality variety of
Bantu English which would be inferior to the quite acceptable
Afrikaans the selfsame Bantu would be likely to use” (p. 46).
By the late 1970s, English had become the leading “other tongue”
in most Bantu-speaking areas. The demands for English-only
instruction by Soweto students during the 1976 uprising shows how
far English had come. As a direct result of Soweto, it became
possible for schools to opt for English as the sole medium of
instruction above primary levels (with Afrikaans relegated to the
status of a language of study) (Kloss, 1978). The dual medium
school became legislative history. People learn to speak, and like to
learn, in a language they like.
Harlech-Jones (1987) emphasizes the importance of language
attitudes by citing Mackey’s (1984) observation that “the success or
failure of a programme of bilingual education depends enormously
on public enthusiasm and support” (p. 72). Similarly, Schumann
(1978) feels that the most powerful reason for learning a second
language is to integrate culturally with its speakers. Tom Reese and
I (1981) argue that individuals and groups may have powerful
associations with languages not initially part of their repertoire. I
would contend further that speakers’ language proficiency is not as
important to them as is the attitude that they “have” a language and
16 TESOL QUARTERLY
that, to some extent at least, it is theirs (see also Paulston, 1987).
With regard to the three dominant languages in South Africa
(Afrikaans, English, and Zulu), Meerkotter (1987) believes,
It has become evident that English will become the lingua franca . . . in
spite of the fact that more citizens presently use Afrikaans than . . .
English. The difference between the two languages lies in the fact that
more people are willing to learn and speak English than Afrikaans
because the latter tongue is associated largely with an establishment
which is not perceived as a symbol representing an open, free, and
united democratic country. (p. 141)
Zulu and other indigenous Bantu languages do not have the
political, social, and economic status of English. In sum, it is
language attitudes, in part based on language status, that have
brought English to the fore.
CONCLUSION
Given the primacy of language status, and the relationship
between status and perceived occupational mobility, it should be
clear from the preceding discussion that language planning that
adheres mainly to state-level factors can miss the mark. We saw that
the principle of territoriality provides political solutions that are
straightforward but not necessarily relevant throughout the polity to
questions of choice. The principle of personality can lead to choices
that will be unpopular: People do not necessarily want to be
educated in their first language if that language has no cachet in the
broader political/economic context.
Harlech-Jones (1987) demonstrated that the macro-aspects of
educational policy are always designed in the interest of a particular
political community. He argues that language policy should “in
effect, be a ‘bottom-up’ design” (p. 75). One way to carry out such
bottom-up design is to begin with language attitudes. Languages
that represent the interests of people and that will work in schools
and government offices are a matter of language attitude.
It appeared to David Laitin and me in our 1989 article on
language in Kenya that both the political science macro- and the
sociolinguistic micro-theories capture part of the reality:
To the extent that the Kenyan state exists, English will remain the
language of elite mobility, the vernaculars will get local funding and
support as media of instruction at lower levels of education, and Swahili
will get (from party elites) strong moral but weak economic support. To
the extent that the Kenyan state is weak and divided, its vibrant cities
will develop a Kenyanized Swahili and perhaps a new vernacular which

will express the language style of Kenyan urban society, and will be used
increasingly by writers to reflect and challenge this new culture. (p. 67)
A coherent language policy for Kenya would require a degree of
state centralization that we thought would be ruthless with regard to
the desires of the population. Such a policy would also offend our
own democratic sensibilities. The same would presumably be the
case in South Africa.
When language planning is left to the politicians and guided by
macro-level theorizing, the vernaculars suffer. A political
perspective tends to emphasize the wider picture at the expense of
attending to patterns of language use exhibited by individuals.
The applied sociolinguist can tell the political linguist that state
policies can have absolutely no effect on linguistic repertoires unless
those state policies have, as their primary goal, social change.
Language change follows social change, and social change occurs
despite government policies, which generally support the status
quo. In South Africa, English has become the language people like
best following the social changes accompanying political events of
recent years. Even if people speak other first languages at home, for
the most part they want to go to school in English and perceive that
the economic future of South Africa requires knowledge of English.
It would seem prudent to use sociolinguistic insights to support
current language practices. In post-apartheid South Africa, this
would mean to opt for English as the medium of instruction while
at the same time encouraging first languages in homes, neighbor-
hoods, and regional activities. Functioning politically, official
languages and languages used in the schools will be part of one’s
national identity regardless of an individual’s level of proficiency.
Functioning socially, a first language will remain part of one’s ethnic
identity whether or not it is used at all or is used only in certain
domains. It is in the interest of society not to inhibit linguistic and
cultural ethnic associations. Indeed, “all teachers of minority
children, whether they are members of the ethnic group or not,
need a working understanding of communicative competence and
its implications for the classroom” (Paulston, 1987, p. 31). That is,
they need to know when students are behaving appropriately in
terms of their own language and culture as they are acquiring the
language and culture of the classroom, and the teachers need to
know how to teach in such a language-contact situation.
Now that English appears to have become the language with the
most status and prestige in South Africa, it is hard not to support the
plea of Hartshorne (1987) for the state to “give urgent and
immediate attention” to an English-medium education system for
18 TESOL QUARTERLY
all South African children (p. 103). This, in effect, would support
popular language attitudes. In an earlier paper, again with regard to
Kenya, I term institutional diglossia the practice of monolingual
formal instruction with other first language encouragement
(Eastman, 1989). In regions where there are numbers of first
language speakers of one of the Bantu languages or Afrikaans,
literacy would be attained and instruction would take place in
English. However, discussion of the ideas presented could be
encouraged in the first languages and regional languages; a lingua
franca form of English might even be used during less formal, yet
structured, situations. Even for those whose first language is
English, the discussion aspect of such education would take place in
a local dialect or style of English. Everyone would be educated to
be literate and learned in English, while encouraged to maintain
their home language.
Louw (1989) emphasizes the need to recognize the oral nature of
many of South Africa’s languages, promoting them regionally and
nationally by means of nonprint media such as radio and television.
To some extent, such a view essentially confirms what already takes
place. The language of the school and the language of the home
differ for everyone. For the first language speaker of English, the
vocabulary used, the very fact of literacy, and the relatively formal
style of the classroom represent something quite different from the
language used by the pupil before entering school. Where the
English-as-a-first-language speaker becomes diglossic upon
entering school, learning another variety of English, the speakers of
Afrikaans or Zulu as first languages become situationally bilingual
once they leave home. They learn English for school, and keep
Afrikaans or Zulu for family, home, and regional affairs.
Much of the research done in Africa with a focus on language
planning points to the fact that the Mother Tongue Principle has
continued to be supported throughout the continent (see Chick &
Seneque, 1987). The idea that the transition between home and
school is eased by instruction in the child’s first language has long
gone unchallenged. The evidence of language attitudes in South
Africa and elsewhere in Africa indicates that extralinguistic factors
having to do with the perceived socioeconomic mobility associated
with a language may have as much, or more, to do with how
successful a language may be, either as a medium of instruction or
in any other official capacity.
Here we return to the initial question: Does language planning
have a role in South Africa? From a sociolinguistic perspective, we
know that people will “spontaneously determine what is going to be
used” (Meerkotter, 1987, p. 136). It would seem that the most

prudent course will be to avoid state-level policies that cannot be
responsive to such spontaneity.
THE AUTHOR
Professor Carol M. Eastman is Chair-designate of the Anthropology Department at
the University of Washington, and a member of the Board of Directors of the
African Studies Association. She has been studying language planning in Africa for
25 years. Her books include Aspects of Language and Culture (Chandler & Sharp,
1975/1990), Linguistic Theory and Language Description (Lippincott, 1978), and
Language Planning: An Introduction (Chandler& Sharp, 1983).
REFERENCES
Chick, K., & Seneque, M. (1987). The role of the applied linguist in
language planning: The medium of instruction problem in Kwazulu,
Natal. In D. N. Young (Ed.), Language: Planning and medium i n
education (pp. 120-135). Cape Town: The Language Education Unit
and SAALA.
Eastman, C. M. (1983). Language planning: An introduction. Novato, CA:
Chandler & Sharp.
Eastman, C. M. (1989). Dissociation: A unified language policy outcome
for Kenya. Manuscript submitted for publication.
Eastman, C. M., & Reese, T. (1981). Associated language: How language
and ethnicity are related. General Linguistics, 21 (2), 109-116.
Fishman, J. (Ed.), (1968). Readings in the sociology of language. T h e
Hague: Mouton.
Harlech-Jones, B. (1987). Implementing language policy decisions in
education: What do we know and what don’t we know? In D. N. Young
(Ed.), Language: Planning and medium in education (pp. 69-81). Cape
Town: The Language Education Unit and SAALA.
Hartshorne, K. B. (1987). Language policy in African education in South
Africa 1910-1985, with particular reference to the issue of medium of
instruction. In D. N. Young (Ed.), Language: Planning and medium in
education (pp. 82-106). Cape Town: The Language Education Unit and
SAALA.
Hymes, D. (1972). Marvels of the interaction of language and social life. In
J. J. Gumperz & D. Hymes (Eds.), Directions in sociolinguistics: The
ethnography of communication (pp. 35-71). New York: Holt, Rinehart &
Winston.
Kloss, H. (1978). Problems of language policy in South Africa [Special
issue]. Ethnos, 16.
20 TESOL QUARTERLY
Kotze, E. F. (1987). A black perspective on Afrikaans. In D. N. Young
(Ed.), Language: Planning and medium in education (pp. 169-183).
Cape Town: The Language Education Unit and SAALA.
Laitin, D. D. (1989). A political perspective on language repertoires in
Africa. Unpublished manuscript.
Laitin, D. D., & Eastman, C. M. (1989). Language conflict: Transactions
and games in Kenya. Cultural Anthropology, 4 (l), 51-72.
Laponce, J. A. (1987). Languages and their territories (A. Martin-Sperry,
Trans.). Toronto: University of Toronto Press.
Louw, E. (1989). Issues raised by Alexander’s Language Policy and
National Unity in South Africa/Azania. Language Projects’ Review,
4 (3), 12.
Mackey, W. (1984). Bilingual education and its social implications. In J.
Edwards (Ed.), Linguistic minorities, policies, and pluralism (pp. 151-178).
London: Academic Press.
Meerkotter, D. B. (1987). The struggle for liberation and the position of
English in South Africa. In D. N. Young (Ed.), Language: Planning and
medium in education (pp. 136-145). Cape Town: The Language
Education Unit and SAALA.
Myers-Scotton, C. (1988). Codeswitching as indexical of social negotia-
tions. In M. Heller (Ed.), Codeswitching: Anthropological and
sociolinguistic perspectives. Berlin: Mouton de Gruyter.
Paulston, C. B. (1987). Linguistic consequences of ethnicity and
nationalism in multilingual settings. In D. N. Young (Ed.), Language:
Planning and medium in education (pp. 12-57). Cape Town: The
Language Education Unit and SAALA.
Romaine, S. (1988). Pidgin and creole languages. London: Longman
Linguistics Library.
Schumann, J. H. (1978). The acculturation model for second-language
acquisition. In R. C. Gingras (Ed.), Second language acquisition and
foreign language teaching (pp. 27-50). Arlington, VA: Center for
Applied Linguistics.
Young, D. N. (Ed.), (1987). Language: Planning and medium in education.
Cape Town: The Language Education Unit and SAALA.

A Context-Adaptive Model for

Program Evaluation
BRIAN K. LYNCH
University of California, Los Angeles
The literature on the evaluation of language teaching programs has

focused almost entirely on specific issues of methodology and
measurement. This article presents a generalized model for ESL
program evaluation. The context-adaptive model consists of a
series of seven steps designed to guide the program evaluator
through consideration of the issues, information, and design
elements necessary for a thorough evaluation. These steps are
illustrated with examples from the evaluation of the Reading
English for Science and Technology (REST) Project at the
University of Guadalajara, Mexico. The model is intended to be
flexible, lending itself to effective adaptation and refinement as it
is implemented in a variety of ESL/EFL contexts.
Evaluation, the systematic attempt to examine what happens in,

and as a result of, language programs, typically serves as the basis
for judgments and decisions about these programs. Unfortunately,
the applied linguistics literature on program evaluation has focused
primarily on specific issues concerning evaluation methods (Long,
1983, 1984; Beretta, 1986a, 1986c; Elley, 1989) and the assessment of
student achievement (Henning, 1982; Beretta, 1986b; Bachman,
1989; Hudson, 1989). Less frequently discussed has been the issue of
a generalized evaluation model that addresses the full range of
concerns of language teaching programs.
Initial attempts to formulate a comprehensive framework for
evaluating language teaching programs have lacked the detail
necessary to be optimally useful (Robertson, 1982), or have been
tied to a particular evaluation purpose such as decision making
(Alkin & Wooley, 1970) or curriculum design and development
(Brown, 1989a). In fact, it may not be feasible to formulate a single
model capable of serving the vast array of potential evaluation
contexts. The purpose of this article is to outline a model for
program evaluation that is able to adapt to a variety of, if not all,
settings.
23
The context-adaptive model represents a refinement of the model
initially outlined in Lynch (1988). The distinguishing feature of this
model is its flexibility in responding to the range of contextual
constraints that program evaluation can encounter. This flexibility
derives from a series of steps and considerations that will vary from
context to context, in terms of the information they reveal and the
evaluation design ultimately chosen. Where possible, these steps
will be illustrated with examples from the evaluation of the Reading
English for Science and Technology (REST) Project at the Univer-
sity of Guadalajara, Mexico. (For a more complete discussion of this
particular evaluation, see Lynch, 1986,1987, 1988.)
THE MODEL
The series of general steps for the context-adaptive model
(Figure 1) are the following:
1. Establish the audience(s) and goals for the evaluation.
2. Develop a context inventory and determine which dimensions
are important in light of the goals and audience for the
evaluation.
3. Develop a preliminary thematic framework based on the issues
that are central to the particular context.
4. Develop a data collection design/system based on the audience
and goals and on the context inventory, and that is focused by the
thematic framework.
5. Collect the data and revise Steps 3 and 4 as necessary; possibly
elaborate Step 2.
6. Analyze the data and revise Steps 3 and 4 as necessary.
7. Formulate the evaluation report.
Rather than being rigidly linear, this model is meant to be
iterative, with the results of certain steps necessitating a return to
earlier ones for changes in conceptualization.
Step 1: Audience and Goals

The first step is to determine the audience and the goals for the
evaluation. In determining the audience, it is necessary to identify
not only the persons or organizations requesting that the evaluation
be done, but also the range of persons who will actually request or
see the evaluation report(s) in one form or another. The audience
may be a funding agency for the program, administrators of the
institution in which the program is being carried out, the program
24 TESOL QUARTERLY
FIGURE 1
The Context-Adaptive Model for Program Evaluation
staff, program developers or researchers from other settings, or a

combination of these audience types.
A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 25

In the case of the REST Project evaluation, there were multiple
audiences. The host institution, the University of Guadalajara
(UdeG), and its collaborator on the project, the University of
California at Los Angeles (UCLA), were the primary audiences. In
addition, the project staff, while closely involved in data collection
for the evaluation, were also considered an important audience.
The evaluation of a program can have far-reaching effects.
Awareness of the range of persons and institutions having an interest
in the evaluation results can assure that the evaluation report
provides relevant information to all those concerned. Related to this
identification of audience, then, the first step involves the determi-
nation of the evaluation’s goals, or the reasons for conducting the
evaluation. Different audiences have different and, usually, quite
specific reasons for requesting or being interested in the evaluation
of a program. Those reasons translate into how the information
from the evaluation will be used. These uses, in turn, are important
indicators of the types of evidence or data that will be necessary or
preferred.
The multiple audiences for the REST Project evaluation entailed
a combination of goals. The evaluation was being carried out to
inform the UdeG and UCLA of the success or failure of the
program in terms of student achievement at the end of its first year
and again at the end of the second year. These audiences were
primarily concerned with whether the program was proving to be
worth its cost. The same audiences were also interested in an
evaluation that would make recommendations for improving the
program. Accordingly, the REST Project staff used information
from the evaluation to better understand the processes of the
program and to then make changes in the curriculum that would
increase its effectiveness.
Step 2: Context Inventory

Once the goals and the audience for the evaluation have been
specified, a context inventory is developed. This inventory is the
key step in the context-adaptive model. Its development follows
Burstein and Guiton’s (1984) suggestion that evaluation emphasize a
conceptualization of the essential phenomena or features of the
educational program. In this model, the context inventory surveys
features characteristic of language teaching programs (e.g., students
and teachers; cf. Robertson, 1982) as well as questions fundamental
to the evaluation process (e.g., the availability of a comparison
group).
26 TESOL QUARTERLY
The dimensions of the context inventory are the following:
1. Availability of a comparison group
2. Availability of reliable and valid measures of language skills
3. Availability of various types of evaluation expertise (statistical
analysis, naturalistic research)
4. The timing for the evaluation
5. The selection process of students into the program
6. The program students
7. The program staff (especially their availability, competence,
and attitude toward the evaluation)
8. The size and intensity of the program (the number of students,
classrooms, proficiency/course levels)
9. The instructional materials and resources available to the
program
10. The perspective and purpose of the program
11. The social and political climate surrounding the program
Comparison group. Where available, the comparison group
provides the evaluator information that can be used to determine if
the observed effects are the result of the program, or if they might
have occurred even in the absence of that particular program. If the
same or similar effects are observed in the comparison group and in
the program being evaluated, it would be difficult to argue that the
program itself caused those effects. If available for the evaluation,
the comparison group should be characterized in the same fashion
as the program group within the other 10 dimensions of the context
inventory. It is often difficult to find the time and resources to
obtain this information in proper depth. However, to the degree
that the comparison group students, their program, and program
staff can be described in detail, the comparison and the evaluation
will be strengthened.
There are different types of comparison groups that can be
selected for the purposes of program evaluation. At one extreme we
find the classical control group; these students are similar to those in
the program group except that they receive no instruction in the
language being taught in the program. This is the least useful type
of comparison group; if the program achieves greater success than
the control group, it can be said only that the program is better than
nothing. However, this type of comparison does allow the
evaluation to provide judgments concerning possible negative
effects of the program, and to rule out the possibility of maturation

alone as the explanation for student gain (Fitz-Gibbon & Morris,
1987). At the other extreme, we find the comparison group that is
receiving a different and clearly contrasting method of instruction
in the same language being taught in the program. Between these
two extremes are comparison groups receiving instruction that may
have certain aspects similar to those of the program being
evaluated.
Ultimately, for any specific instance, the importance of this
dimension will be clarified as a part of the context-adaptive nature
of the model, that is, in its interaction with other context inventory
dimensions and steps in the model. In the case where no comparison
group is available, for example, the types of quantitative analyses
that can be done are limited; this will affect steps 4 and 6 of the
model. However, the lack of a comparison group need not
necessarily be seen as a weakness of the evaluation. Some
evaluation researchers argue that the control group design may not
be worth the expense and effort required. Cronbach and Associates
(1980) assert that the actual effect of a program can be accurately
assessed with a pretest and posttest design (without control group)
complemented by in-depth interviews with the program partici-
pants. Where limited time and resources force a choice between the
use of a comparison group and a more thorough description of the
program itself, the evaluation’s audience and goals should guide the
decision.
Reliable and valid measures. For this dimension, available language

tests should be specified in terms of their content and scope. For
example, the available tests may be norm-referenced (NR) or
criterion-referenced (CR). NR tests are designed to compare or
rank students, and are generally used for proficiency and placement
purposes. CR tests measure students’ performance with respect to a
specific set of (language) criteria, and are generally used for
achievement and diagnostic purposes. (For a more detailed
discussion of the differences between NR and CR measures, see
Hudson & Lynch, 1984; Brown, 1989b.) Depending on the
curricular goals of the program, as described in other dimensions of
the inventory, and on the goals of the evaluation, one type of test or
another will be preferred. These tests need to be further examined
for their potential sensitivity to changes brought about by
instruction, and if there is a comparison group, the possibility of the
tests favoring one group over another needs to be addressed (cf.
Beretta, 1986b). Addressing this dimension may result in the need
for new tests to be developed for the evaluation. Other measures
28 TESOL QUARTERLY
that may need to be identified or developed include background-
variable questionnaires, interview formats, and observation sched-
ules.
Evaluation expertise. This dimension requires access to various
kinds of evaluation expertise. Most evaluations, in order to be as
thorough as possible, will benefit from a combination of
quantitative and qualitative research methods and techniques
(Lynch, 1988; Brown, 1989a). Thus, the availability of persons
capable of doing statistical analysis, test development and analysis,
participant and nonparticipant observation, and interviewing
should be determined.
Timing. The total span of time required and available for the
evaluation must be determined in advance. In addition, time frames
need to be established for the academic calendar of the program,
the frequency of specific evaluation activities, instructional
sequences in the program, and the total number of person-hours
available for the entire evaluation.
Selection process. In most cases, the process of selecting students for
the program will be nonrandom; that is, the program students will
represent an intact group or will have been selected according to
certain criteria. In order to determine the degree to which selection
is responsible for the observed program effects (e.g., student
achievement), this process must be described with as much
accuracy as possible. To enhance accuracy, information should be
gathered from several different sources. This will help guard
against the possibility that any one person contacted to describe the
selection process may do so in such a way as to, intentionally or
unintentionally, leave out certain key variables that define the
program students as being unique or different from other students.
Students. To provide more information for understanding the
interaction between the program and the students, and for making
comparisons with other programs or groups of students, a detailed
description of the students is invaluable. Factors such as native
language background, culture, and country of origin are important
considerations. Other features to consider for this dimension are the
student’s age, sex, socioeconomic status, previous education,
previous academic achievement, and previous experience with the
language and culture being taught in the program.
Some relevant features will require special instruments in order to
be described, such as individual learning styles (e.g., inductive vs.
deductive, field dependent vs. field independent, visual vs. aural).
Other features useful to describe are self-concept (both in general

and in terms of ability in the language being taught), foreign
language aptitude, perceived need for the language instruction
offered by the program, and attitude toward that language and its
culture.
Staff. A description of this dimension includes the number of staff
personnel; their job descriptions; their professional qualifications as
teachers, curriculum and materials developers, administrators, etc.
Also useful is background information similar to that obtained for
the students. In the case of in-house evaluations, it is also important
to assess the availability of the staff for participation in the data-
gathering stages of the evaluation. This assessment includes the
number of hours each person would be able to devote to such
activities as interviews, observations, journal keeping, questionnaire
response, and test administration. This can be done through
informal or formal interviews and/or questionnaires prior to
beginning the evaluation.
Size and intensity. The program size is defined by the number of
students in the program, the organization of the program, and the
number of classrooms. The organization of the program, in this
case, refers to the distribution of courses (e.g., separate courses for
different language skills vs. an integrated skills approach) or
different language proficiency levels in the program. The intensity
of the program is defined in terms of the number of instructional
hours per week, the length and organization (e.g., semesters or
quarters) of the instructional year, and the length of time needed to
complete the entire instructional sequence of the program.
Instructional materials and resources. An accurate assessment of
what a program is doing and how well it is succeeding requires
information concerning the available instructional materials and
other resources. These include textbooks, other printed instructional
material, reference materials, audiovisual and other technological
aids, human resources, and basic office supplies.
Perspective and purpose. Perspective, as intended for this
dimension, refers to notions, beliefs, and assumptions concerning
the nature of language and the process of language learning.
Purpose refers to the curricular goals of the program. Depending on
the context, perspective may need to be described from several
potentially different points of view. For example, in the REST
Project, the perspective of the program staff was that certain
language skills (in this case, reading skills) could be isolated and
mastered. The foreign language teaching center of the host
30 TESOL QUARTERLY
institution operated from the perspective that successful language
learning required a more holistic, integrated skills approach.
Finally, the perspective of the REST students was that in order to
learn to read in a foreign language they needed instruction in the
spoken language as well.
Within this dimension, purpose is described in terms of the
overtly stated curricular goals for the program. These goals
naturally tend to reflect the perspective of the program developers.
However, the most complete description will be based not only on
documented goals but also on what various participants in the
program see as the goals of the program. This may uncover
conflicting perspectives, as discussed in relation to the REST
Project, above.
Social and political climate. The 11th and final dimension of the
context inventory concerns the social and political climate
surrounding the program. A description of this dimension entails an
assessment of how the program is perceived by the surrounding
academic community, if one is present, as well as by the larger
social community in which the program is located. This involves not
just the program and its goals per se, but the language being taught
and its associated culture.
In the case of ESL and EFL programs, the teaching of language
raises serious political and moral questions (Judd, 1984; Gaies,
1987). Within the ESL context, issues of cultural and linguistic
pluralism versus access to English as a means of socioeconomic
advancement arise (Gaies, 1987). Within the EFL context, the need
for English as the language of science and technology can be seen as
a positive means for effecting economic growth in developing
countries, or as a negative means for preserving the status quo of
dependency on U.S. technology (Judd, 1984). Further, the
presentation of language necessarily includes certain cultural values,
which can lead to accusations of cultural imperialism.
Once information has been gathered on each of these 11
dimensions the context inventory can be used, along with
information about the goals and audience for the evaluation, to
guide and inform the subsequent steps. In part, the adaptive nature
of this model lies in the fact that, depending on the goals and
audience, information on some of the dimensions need not be
gathered. In addition, budgetary considerations may limit what is
possible during the context inventory step. Whatever its form, the
inventory becomes the backbone of the evaluation. Where time and
resources limit the amount of information that can be gathered, the

context inventory remains important as an early indicator of the
limits of the particular evaluation. Furthermore, as will be seen in
the description of the subsequent evaluation steps, it will inform
decisions concerning data collection and analysis for each
evaluation context.
Step 3: Preliminary Thematic Framework

Each evaluation context presents a unique set of issues and
themes. During the first two steps of the context-adaptive model,
these issues and themes begin to emerge, leading to the
development of a preliminary thematic framework. Information
from the following dimensions of the context inventory are of
particular importance in developing a program-specific thematic
framework: the selection process, the program students, the size
and intensity of the program, the perspective and purpose, the
status of the language, and the social and political climate.
Depending on the context, the program staff may also be an
important dimension for developing the thematic framework. From
the information these dimensions provide, the most salient themes
and issues present at the beginning of the evaluation are selected.
This preliminary framework provides a conceptualization of the
program that will help guide the data collection and analysis phases
of the evaluation.
As an example, the following represents the preliminary thematic
framework developed for the REST Project evaluation:
1. Instructional focus on reading only
2. Instructional focus on reading skills
3. Use of authentic materials
4. Classrooms and class size
5. Use of Spanish versus English for instruction
6. Use of a “modified adjunct model” approach
7. Classroom research
8. Student levels of English proficiency upon entering the program
These themes and issues emerged after examination of meeting
notes and correspondence, and consideration of classroom
observations and conversations with program staff, students, and
UdeG and UCLA officials, during the early stages of the REST
Project evaluation.
32 TESOL QUARTERLY
Step 4: Data Collection Design/System
The results of the first three steps—establishing the audience and
goals for the evaluation, developing the context inventory, and
organizing the preliminary thematic framework—structure the
evaluation task in terms of specific questions that will need to be
and are capable of being answered. The next step is the selection of
the types of data and methods for data collection that will best
answer those questions. After identifying the desired types of data,
the context inventory can be useful in determining which of these
types will be the most feasible to collect. For example, discovering
a negative attitude on the part of program staff or students toward
the evaluation can alert the evaluator to constraints on data
collection. The context inventory can also indicate when relevant
information will be difficult to obtain. Finally, the context inventory
dimension of available evaluation expertise further narrows the
types of data and data collection methods that will be possible.
For example, in the evaluation of the REST Project, the audience
and goals required a combination of an assessment of student
achievement at the end of the first year and a descriptive, process-
oriented account of the program. The types of data that were
required, therefore, were both quantitative (e.g., test scores to
measure student achievement) and qualitative (e.g., observation
notes and interview transcripts to describe the program processes).
Given the expertise of the project staff and the availability of
consultation with faculty from UCLA, several types of quantitative
and qualitative data could be collected. As an example, the REST
teacher/researchers contributed to the collection of qualitative data
by keeping daily journals that described the events occurring in
their classrooms and the materials they used. This provided
important documentation of the developing curriculum and
invaluable information for the evaluation. On the other hand, data
that had been identified as important for some of the quantitative
analyses, such as student achievement scores prior to participation
in the project, were not available due to UdeG regulations.
The interaction of Steps 1, 2, and 3 in the formulation of the data
collection design/system is further illustrated in Figure 2.
Step 5: Data Collection

The iterative nature of the context-adaptive model is critical at
this step. As the data are being collected, and some of the data will
have already been collected during the context inventory step, it
may be necessary to revise the data collection system of Step 4, as

FIGURE 2
Formulation of the Data Collection Design/System
The Interaction of Steps 1,2, and 3
CONTEXT A SUMMATIVE REPORT
STEP 1
Audience and Goals
Evaluation needs to provide empirical evidence of program worth/success; inform
decisions on whether to continue the program or not; produce a summative report.
STEP 2
Context Inventory
34 TESOL QUARTERLY
FIGURE 2— continued
Formulation of the Data Collection Design/System
The Interaction of Steps 1,2, and 3
well as to elaborate the context inventory and to revise or elaborate

the preliminary thematic framework of Step 3. It is also possible
that new sources of important data will be discovered and that new
themes may present themselves.
A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 3.5

Step 6: Data Analysis
Quantitative analysis. As noted in Figure 2, in contexts where
reliable and valid measures of the language skills being taught are
unavailable, quantitative analysis will be limited in terms of the
claims or conclusions that can be made. Where the evaluation ex-
pertise permits, it would still be recommended that a variety of
quantitative analyses be carried out. Measures that are low in
validity, for example those that measure general language proficien-
cy when the program is focused on specific language skills, can still
be of some use in evaluating general language achievement
outcomes, assuming the measures are reliable. When the measures
are low in reliability, the judgments concerning student achieve-
ment must, of course, be tentatively offered.
Where time and resources allow, a combination of appropriate
techniques should be employed. A multiple analysis strategy
can strengthen the evaluation by avoiding the possibility of bias asso-
ciated with any particular technique. The following were found use-
ful in the REST Project evaluation. They should not be construed as
the only or best tests available for all instances of program evaluation,
They did, however, prove useful in this circumstance.1 (For a more
complete discussion of these techniques, see Lynch, 1988.)
The chi-square test is easily calculated by hand from a fairly
simple formula. While chi-square analysis cannot be used to make
causal claims about the program, it can be used to demonstrate the
strength of relationship between the program and achievement (i.e.,
if the frequency of program students with high postinstructional
achievement is greater than the frequency of comparison group
students with high postinstructional achievement, a significant chi-
square statistic indicates a strong relationship between the program
and student achievement).
Analysis of covariance (Kirk, 1982) can be used to compare the
posttest scores of program students with the comparison group,
while controlling for preexisting differences between the groups.
Although it can be difficult to verify all of the assumptions of this
statistical model, qualified claims about the effect of the program
on student achievement can be made.
Standardized change-score analysis (Kenney, 1975) compares the
correlation between group membership (program or comparison
group) and pretest scores with the correlation between group
membership and posttest scores. The difference between these
1 Standard tests useful in some studies but not used in the REST Project evaluation include
one- and two-step t-tests and exploratory data analysis (Ott, Larson, & Mendenhall, 1963).
36 TESOL QUARTERLY
correlations can be examined for statistical significance (Guilford &
Fruchter, 1978), with a larger posttest-group membership cor-
relation (and a larger posttest mean for the program group) indicat-
ing that the program has had a positive effect on student achieve-
ment.
Effect size analysis (Stallings, Needels, & Staybrook, 1979) can be
calculated from the average gain scores (from pretest to posttest)
and standard deviations (of gain scores) for the program and
comparison groups using a relatively simple formula. It yields an
estimate that indicates whether the program had a strong,
moderate, insignificant, or negative effect on student achievement.
Value-added analysis (Bryk, Strenio, & Weisberg, 1980) makes
use of regression analysis. It estimates the degree of student
achievement beyond that expected due to natural growth (i.e., in
the absence of the program). Value-added analysis can also be used
without a comparison group.
In addition to these techniques, growth-referenced measurement
(Henning, 1982) can be used where CR tests of the language skills
and curriculum subcomponents exist. This analysis can pinpoint
specific elements of the curriculum that are successful or are in need
of change.
In contexts where data exist that can characterize the selection
process, selection-bias modeling (Cain, 1975) can be used as an ad-
ditional analysis. This technique removes the effect of student selec-
tion from the estimate of program effect on student achievement.
Qualitative analysis. The analysis of qualitative data is an iterative
process of data reduction, interpretation, and return to the original
data. In the process, the preliminary thematic framework may go
through further revisions. For example, the following was the
thematic framework ultimately formulated for the evaluation of the
REST Project (in contrast to the preliminary framework presented
in the discussion of Step 3):
1. Focus on explicit grammar versus reading skills
2. Use of English versus Spanish in the classroom
3. Team teaching
4. Student group work in class
5. Student-teacher relations
6. Use of the adjunct model and authentic materials
7. Class size
8. Attendance
9. Teacher preparation

10. Student attitude and motivation
11. Changes in class level and teaching assignment
12. Text and task difficulty
13. Teacher perception of student ability
14. Homework
15. Use of introspection/retrospection with students
16. Grading system
In addition to making the themes in the original thematic
framework more specific, several new themes were discovered.
This is a natural result of the qualitative approach to evaluation,
which allows program issues and effects to emerge from the data-
gathering and analysis stages. In the process of interviewing the
students, for example, student-teacher relations, attendance, and
student attitude and motivation were identified as themes that
needed to be investigated.
After all the data were collected, they were reduced within the
thematic framework through the use of display matrices (Miles &
Huberman, 1984). These matrices presented the qualitative data in
a spatial format that was compressed and systematic, making it
easier for the evaluator to search for patterns and relationships in
the data. One example from the REST Project evaluation was the
Site Dynamics Matrix, in which rows listed particular dilemmas or
problems in the program, and columns described the problem, the
underlying theme(s), how the problem was accommodated, and
any resolution or change that occurred.
The refinement of the matrices required reexamination of the
original data. For example, one dilemma that emerged during the
analysis of the qualitative data was that the program was not what
the students had expected it to be. It was necessary to go back to
data from the interviews and open-ended questionnaires in order to
find actual student expressions of their expectations and program
staff expressions of reaction to this dilemma. Finally, the matrices
were interpreted and conclusions drawn. This also required a return
to the original data in order to look for rival explanations. (For a
complete description of this process, see Lynch, 1988.)
Step 7: Evaluation Report

The goals and audience, as well as the context inventory, will
influence the form of the final evaluation report, which can range
from formal written reports to informal oral ones. At this stage, the
social and political climate dimension of the context inventory
38 TESOL QUARTERLY
should be carefully assessed in light of the results. A thorough
evaluation, especially one that incorporates qualitative research
techniques, can present data and conclusions that are socially and
politically sensitive. This is an aspect of evaluaton that must be
considered with extreme care.
The evaluator may find that some conclusions and the evidence
on which they are based are best omitted or modified in the final
report. This is not so much out of the fear that “the truth” will not
set well with individuals or groups, but out of concern that the
conclusions and their supporting evidence be understood and
interpreted as intended by the evaluator. If a conclusion, stated in a
particular way, has the potential to inflame the social and political
passions of the audience for the evaluation, the evaluator must
realize that the intended message may become distorted and
misinterpreted.
CONCLUSION
This presentation of the context-adaptive model for the
evaluation of ESL programs is intended as an elaboration of a
generalized model that can be refined through its application in a
variety of language teaching contexts. I have presented each of the
seven steps in a general and open-ended fashion to underscore the
model’s flexibility and adaptability in different contexts. At the
same time, specific examples from the REST Project should
provide program evaluators the practical guidance to execute a
clear and productive evaluation.
The strongest approach to evaluation is one that combines as
many methods, qualitative and quantitative, as are appropriate to
the particular evaluation context. The context-adaptive model
provides a framework that encourages this multiple-strategy
approach. This iterative framework leads program evaluators
through a set of considerations that can adapt the evaluation to a
variety of specific program settings. The use of this model in
future evaluations will lead to its further refinement and increased
usefulness for ESL and other language-teaching program contexts.

THE AUTHOR
Brian Lynch is an Assistant Adjunct Professor in the Department of TESL/Applied
Linguistics at the University of California, Los Angeles, where he also serves as
Academic Director of ESL Service Courses. His primary research interests are
program evaluation, language testing, and English for specific purposes.
ACKNOWLEDGMENT
This article is a revised version of a paper presented at the 23rd Annual TESOL
Convention in San Antonio, TX, March 1989. The author wishes to thank an
anonymous TESOL Quarterly reviewer for helpful comments on an earlier draft of
this article.
REFERENCES
Alkin, M. C., & Wooley, D. C. (1970). A framework for the evaluation of

TESOL programs (Working Paper No. 4). Los Angeles: University of
California, Center for the Study of Evaluation.
Bachman, L. F. (1989). The development and use of criterion-referenced
tests of language ability in language program evaluation. In R. K.
Johnson (Ed.), The second language curriculum (pp. 242-258).
Cambridge: Cambridge University Press.
Beretta, A. (1986a). Toward a methodology of ESL program evaluation.
TESOL Quarterly, 20 (1), 144-155.
Beretta, A. (1986b). Program-fair language teaching evaluation. TESOL
Quarterly, 20 (3), 431-445.
Beretta, A. (1986 c). A case for field-experimentation in program
evaluation. Language Learning, 36 (3), 295-309.
Brown, J. D. (1989a). Language program evaluation: A synthesis of
existing possibilities. In R. K. Johnson (Ed.), The second language
curriculum (pp. 222-241). Cambridge: Cambridge University Press.
Brown, J. D. (1989b). Improving ESL placement tests from two
perspectives. TESOL Quarterly, 23 (l), 65-85.
Bryk, A. S., Strenio, J. F., & Weisberg, H. I. (1980). A method for
estimating treatment effects when individuals are growing. Journal of
Educational Statistics, 5 (l), 5-34.
Burstein, L., & Guiton, G. W. (1984). Methodological perspectives on
documenting program impact. Advances in Special Education, 4, 21-42.
Cain, G. G. (1975). Regression and selection models to improve
nonexperimental comparisons. In C. A. Bennett & A. A. Lumsdaine
(Eds.), Evaluation and experiment: Some critical issues in assessing
social problems (pp. 297-317). New York: Academic Press.
40 TESOL QUARTERLY
Cronbach, L. J., & Associates. (1980). Toward a reform of program
evaluation: Aims, methods, and institutional arrangements. S a n
Francisco: Jossey-Bass.
Elley, W. B. (1989). Tailoring the evaluation to fit the context. In R. K.
Fitz-Gibbon, C. T., & Morris, L. L. (1987). How to design a program
evaluation. Beverly Hills, CA: Sage.
Gaies, S. J. (1987). Research in TESOL: Romance, precision, and reality.
TESOL Newsletter, 21 (2), 21-23.
Guilford, J. P., & Fruchter, B. (1978). Fundamental statistics in psychology
and education (6th ed.). New York: McGraw-Hill.
Henning, G. (1982). Growth-referenced evaluation of foreign language
instructional programs. TESOL Quarterly, 16 (4), 467-477.
Hudson, T. D. (1989). Mastery decisions in program evaluation. In R. K.
Hudson, T. D., & Lynch, B. K. (1984). A criterion-referenced measurement
approach to ESL achievement testing. Language Testing, 1 (2), 171-201.
Joint Committee on Standards for Educational Evaluation. (1981).
Standards for evaluations of educational programs, projects, and
materials. New York: McGraw-Hill.
Judd, E. L. (1984). TESOL as a political act: A moral question. In R. A.
Orem & B. P. Taylor (Eds.), On TESOL ’83 (pp. 265-273). Washington,
DC: TESOL.
Kenney, D. A. (1975). A quasi-experimental approach to assessing
treatment effects in the nonequivalent control group design.
Psychological Bulletin, 82 (3), 229-234.
Kirk, R. E. (1982). Experimental design (2nd ed.). Belmont, CA: Brooks/
Cole.
Long, M. H. (1983). Inside the black box: Methodological issues in
classroom research on language learning. In H. W. Seliger & M. H. Long
(Eds.), Classroom oriented research in second language acquisition (pp.
3-35). Rowley, MA: Newbury House.
Long, M. H. (1984). Process and product in ESL program evaluation.
TESOL Quarterly, 18 (3), 409-425.
Lynch, B. K. (1986). Evaluating a program inside and out. Paper presented
at the 20th annual TESOL Convention, Anaheim, CA.
Lynch, B. K. (1987). Evaluating a program with qualitative data. Paper
presented at the 21st annual TESOL Convention, Miami, FL.
Lynch, B. K. (1988). Toward a context-adaptive model for the evaluation
of language teaching programs (Doctoral dissertation, University of
California, Los Angeles, 1987). Dissertation Abstracts International, 48,
2264A.

Miles, M. B., & Huberman, A. M. (1984). Qualitative data analysis: A
sourcebook of new methods. Beverly Hills, CA: Sage.
Ott, L., Larson, R. F., & Mendenhall, W. (1983). Statistics: A tool for the
social sciences (3rd ed.). Boston: Duxbury Press.
Robertson, D. L. (1982). Toward a model for the evaluation of programs
in English as a second language. In Y. Krachru & J. R. Cowan (Eds.),
TESL Studies (Vol. 5, pp. 155-171). Urbana-Champaign, IL: University
of Illinois Press.
Stallings, J., Needels, M., & Staybrook, N. (1979). How to change the
process of teaching basic reading skills (Research Report, May, 1979).
Menlo Park, CA: Standard Research Institute.
42 TESOL QUARTERLY
Managing the Complexity of

Revising Across Languages
CHRIS HALL
Wright State University
Although previous research in ESL composition suggests a link

between writing in a first and second language, few studies have
investigated this relationship in the context of the revising process.
This article examines revision in controlled L1 and L2 writing
tasks. Four advanced ESL writers with differing first language
backgrounds wrote two argumentative essays in their native
languages and two in English. Revisions were then analyzed for
specific discourse and linguistic features. The results, for the most
part, indicate striking similarities across languages. However,
some differences are noted, suggesting that while proficient
writers are capable of transferring their revision processes across
languages, they are also capable of adapting some of those
processes to new problems imposed by a second language.
Until quite recently, process-oriented research in ESL writing

was dominated by studies that sought to identify those strategies
and behaviors second language writers share and do not share with
their first language counterparts (Raimes, 1985). Such comparisons
between primary and nonprimary speakers of English are, of
course, valuable if ESL researchers and teachers are to determine
the relevance of first language pedagogy for second language
writers. Yet they do not represent the only direction for
comparative studies with consequences for theory and practice.
Researchers are now beginning to examine ESL writers’ behaviors
and strategies in both their first and second languages, a much
needed area of investigation that Zamel (1984) has claimed may
have implications for the notion that composing processes are
universal and transitive across languages. While most of these cross-
language studies have focused on general surveys of composing
processes (Chelala, 1981) or concentrated on text planning (Lay,
1982, 1983; Jones & Tetroe, 1987), to date, only one significant study
has been conducted on revising (Gaskill, 1987). Yet “revision,” as
43
Bartlett (1982) points out, “seems to be an essential component of
virtually every attempt to construct a model of the writing process”
(p. 345). The aim of the present study, therefore, is to expand
current research on revising processes across languages. It provides
baseline data on proficient second language writers’ L1 and L2
revising behaviors and strategies, a group for which little
information is now available.
BACKGROUND
Experienced L1 Revisers and their L2 Counterparts
Of particular relevance to this study is the research on revising
processes of proficient writers. Several important observations
about these writers’ perceptions and applications of revising have
emerged from L1 research. First of all, proficient writers assess the
value of revising differently than do inexperienced writers. For
experienced writers, revising can encompass the entire writing task,
from initial planning to final drafting (Faigley, Cherry, Jolliffe, &
Skinner, 1985). There is often a sense that revision is a process of
discovering meaning in a text (Murray, 1978; Sommers, 1980) and
that it shapes not only the form and content of a text but also its
voice (Murray, 1978; Sommers, 1980). In contrast, novice writers
often relegate revising to cosmetic adjustments of the text and to
proofreading for grammar and mechanics, citing the formulaic
language of textbooks as reasons for making these alterations
(Beach, 1976; Sommers, 1980).
Second, there is evidence that experienced and novice writers
employ strikingly different revising strategies. Some studies have
noted the linguistic and discourse units these two groups address. In
an early comparative study by Stallard (1974), for example, the
revisions of a group of good writers and a randomly selected group
revealed that, while single word changes dominated the revisions of
both groups, the good writers also initiated more multiple-word and
paragraph changes. In Sommers’ (1980) study, experienced adult
writers tended to change entire sentences to convey new meanings,
while her inexperienced student writers concentrated on single
words that affected mechanics and grammar.
The points at which revisions are initiated and their frequencies
also have been noted by researchers. Monahan (1984) found that his
competent writers revised more extensively from the first to the
final draft, in contrast to his basic writers who viewed the shaping
and reshaping of ideas a laborious task, frequently reserving
revision for the final draft. Bridwell (1980) discovered that her
better writers initiated a significantly higher number of revisions
44 TESOL QUARTERLY
while reviewing the first draft than did her poorer writers. Faigley
and Witte (1981) also noted that, while most revisions occurred
during the first and final drafts for all their groups, the advanced
student writers made a noticeable number of revisions between
drafts.
With so few studies of proficient second language writers
available, no composite picture of their revising processes can be
claimed. Yet there are some data on proficient second language
revisers that corroborates findings in first language studies. Zamel
(1983), for example, devoting part of her case studies of skilled and
unskilled ESL writers to a discussion of revising, found that her
skilled writers attended to more global units when revising. She
noted instances of paragraph reordering in her skilled writers’ texts,
along with a greater concern for refining and adding sentences. In
another study, Heuring (1980) examined the revisions of a group of
five ESL writers, ranked as less skilled to more skilled. No patterns
of revision were found among more skilled writers. Rather, these
writers tended to demonstrate individual approaches to revision.
L2 Research on Transfer in Writing

The notion that writing ability can be transferred between
languages has been suggested as early as the 1940s by Dunkel
(1948), who argued that general language proficiency is “probably
interlinguistic” (p. 90) and encompasses a person’s knowledge of
writing. Later investigations by Stendahl (1972) and Linnarud
(1977) also have suggested a correlation between knowledge in the
first language and writing skill in the second language. Recently, the
idea that L1 and L2 writing processes are interrelated has gained
prominence among process-oriented researchers and prompted a
series of investigations that have attempted to examine writers and
their L1 and L2 texts.
Chelala (1981) was the first to examine this transfer effect in
composing. She studied two Spanish-speaking subjects as they
composed two Spanish and two English texts. From her
observations, Chelala noted that certain comparable L1 and L2
strategies had a salutary effect on her writers’ texts, specifically,
taking notes, using cohesive devices, and revising to match a text
with a particular meaning. Edelsky (1982), studying the writing of
nine Spanish-English bilingual children, also found evidence that
certain writing skills in one language, such as knowledge of spelling
and manipulation of style, assisted her young writers when
composing in the other. In Lay’s (1982, 1983) investigations of
Chinese students writing in English, she noted that her writers
REVISING ACROSS LANGUAGES 45

tended “to translate key words into the first language to get a
stronger impression and association of ideas for the essay” (Lay,
1982, p. 406). More changes seemed to produce better organized
and detailed essays. These alterations revealed a strategy in which
semantic knowledge in the first language assisted the writer’s ability
to generate more precise and, in certain cases, elaborate structures
in the second language.
Perhaps the two most ambitious studies of transfer across
languages were undertaken by Jones and Tetroe (1987) and Gaskill
(1987). Jones and Tetroe (1987) studied planning. Comparing
compose-aloud data collected from six Spanish-speaking subjects
writing in Spanish and English, they concluded that the subjects’
methods of planning transferred across languages, especially at the
level of abstract planning. Proficiency, however, affected the
transfer, reducing the number but not the quality of planning
episodes. In his study of transfer, Gaskill (1987) focused on revising.
Four Spanish-speaking subjects, classified as either less or more
proficient, participated in his study. Gaskill analyzed their revisions
using Faigley and Witte’s (1981, 1984) taxonomy. From his analysis,
Gaskill found that surface changes (that is, changes affecting the
grammar and mechanics of a sentence or changes preserving the
sentence’s original meaning) dominated the L1 and L2 revisions.
Furthermore, the highest concentration of L1 and L2 revisions
occurred during the actual drafting of texts.
Since previous L1 and L2 investigations have indicated that
proficient writers demonstrate quite pronounced revising processes
and that these processes are likely to remain intact across languages,
this descriptive study attempts to identify the linguistic and
discourse features of revisions across languages in the texts of
proficient second language writers. Data were collected from a
series of four argumentative assignments, two in the subjects’ native
languages and two in English.
METHOD
Subjects
Four advanced ESL writers from the University of Wyoming
participated in the study. The first subject, Anna, is a 21-year-old
Polish woman who studied English in her native country for eight
years. Anton, the next subject, is a 38-year-old native of the French-
speaking region of Switzerland. He began studying English 12 years
ago after moving to the United States. The third subject, Bjorn, is a
23-year-old Norwegian who started his English studies in the fourth
46 TESOL QUARTERLY
grade. The last subject, Wei, is a citizen of the People’s Republic of
China. She is in her early 30s and began her English studies while
attending a Chinese university five years ago.
Although these four second language writers have differing
linguistic, cultural, and educational backgrounds, they are bound
together by their writing abilities in English. On the basis of three
criteria, all were classified as advanced ESL writers. First, each
subject had fulfilled the college English requirements in first-year
writing for nonnative speakers, or demonstrated sufficient writing
skill to be exempted from these requirements. Second, writing
samples gathered at the beginning of the study were evaluated
using the Jacobs, Zinkgraf, Wormuth, Harfiel, and Hughey (1981)
ESL Composition Profile. All of the subjects scored within the 83-
100 range, placing them in the advanced category of ESL writers.
Third, teachers familiar with the subjects’ performances as college
writers were interviewed about their levels of writing competence,
opportunities to write for a college audience, and types of writing
assignments. The interviews further indicated that these subjects
should be designated advanced ESL writers.
Writing Tasks
The four writing tasks used in this study were designed to elicit
argumentative texts. Topics for the writing assignments focused on
academic life and were drawn from interviews which probed the
subjects’ schooling and writing backgrounds.
In the construction of the writing assignments, a consistent format
was used which included a set of simple instructions and two topics,
worded in the form of a debate proposition. The following example,
taken from the first L2 writing task, illustrates this basic format:
Choose one of the topics below and write an essay in which you
explain your position for or against your chosen topic.
1. A basic core curriculum in the liberal arts should be instituted
for undergraduates.
2. Teaching assistantships should be restricted to native speakers
of English.
For each writing task, the same set of instructions was used, and
for the L2 and first L1 writing tasks, the same selection of topics was
used for each subject. For the second L1 writing task, however, it
was necessary to design two separate writing assignments, one for
Anna and Wei and the other for Anton and Bjorn. The lack of shared

experiences between these two groups precluded the construction
of common assignments for all the L1 writing tasks.
For the L1 writing tasks, the assignments were translated into the
subjects’ native languages by experienced language teachers. These
teachers were given samples of the L2 writing assignments as
models, and asked to translate the topics to be used in the L1
assignments. They were instructed to check their translations for
comparability in wording and structure with the L2 assignments.
A specific writer/audience relationship was defined for all the
writing assignments. This relationship was described in terms of
Applebee’s (1984) category, “Teacher: as part of an instructional
dialogue” (p. 13), in which the teacher is perceived as a participant
in a dialogue with the writer and not as an examiner of the writer’s
skill or information. This type of writer/audience relationship was
discussed with each subject before the start of the writing tasks.
While the structure of the writing tasks allowed subjects to choose
between two topics, there was surprising consistency among
subjects in the choice of topics. For the two L2 tasks, all of the
subjects chose to write on “A basic core curriculum in the liberal arts
should be instituted for all university students” and “A pass/fail
grading system should be instituted for all required general
education courses.” In the first L1 writing task, the proposition “A
student/faculty committee should be instituted for the hiring and
firing of teachers” was chosen by the subjects. For the two separate
assignments in the last L1 writing task, Anna and Wei wrote on
“More women should be allowed to study in the university.” Anton
and Bjorn selected “More funding for the education of university
students should come from private companies and industries.”
Writing Sessions
For each writing task, two 90-minute writing sessions were
individually scheduled. During these sessions, first and final drafts
were planned, composed, and revised. At the beginning of each
task, the writing assignment was given to the subject. At the end of
this session, the draft, with revisions, was collected and photo-
copied. A period of 48 hours then elapsed before the start of the
second session. For the next session, the draft was returned to the
subject, who was then free to make any additional revisions deemed
necessary or to proceed immediately to the final draft. After the
final draft was finished, the material was collected. As soon as a
writing session ended, the researcher, with the help of the subject,
identified and numbered each revision.
All writing sessions were held in a room equipped with a video
48 TESOL QUARTERLY
camera. Recordings of each writing session in this study served a
limited purpose, which was to identify significant pauses in the
writing process. A significant pause was designated as any
interruption lasting longer than five seconds in the transcribing of a
draft.
Analysis of Revisions
The linguistic analysis used in this study was based on research
conducted by Sommers (1980), Bridwell (1980), and Monahan
(1984). This classification scheme is, in large part, based on
linguistic and discourse units that have been consistently identified
in previous studies of revision. The scheme included four general
categories and their subcategories. The first category was called
stages of revision. After observing the writing processes of the
subjects in this study, I identified four subcategories of revision.
Stages 1 and 2 included revisions made during the first 90-minute
writing session. Stage 1 revisions consisted of changes to notes or
outlines made before the first draft was committed to paper. All
revisions made as the first draft was being composed were
designated Stage 2. Stages 3 and 4 included revisions made during
the second 90-minute writing session. At the beginning of the
second writing session, the subjects’ first drafts were returned to
them. Any revisions made to these texts were designated Stage 3.
Stage 4 revisions included those revisions made during the actual
writing of the final draft.
The next category, termed level of revision, dealt with seven
linguistic or manuscript units: (a) word, (b) phrase, (c) clause, (d)
sentence, (e) paragraph, (f) global, and (g) surface. Global units
were defined as encompassing more than one paragraph. Surface
units consisted of certain features outside the range of the other
units and included marginal notations and manuscript conventions
such as spacing, indentation, centering of titles, margins,
capitalizations, and editing symbols.
Type of revision constituted the third category and included the
following linguistic operation most often identified with the writing
process: (a) addition, (b) deletion, (c) substitution, (d) reordering,
and (e) consolidation. Table 1 illustrates these five types of revision.
The final category, purpose of revision, focused on three
subcategories associated with written discourse: (a) informational
(b) grammatical/mechanical, and (c) cosmetic. These three
categories are illustrated in Table 2.
To demonstrate how revisions were classified, the following

TABLE 1
Selected Examples of the Five Revision Types
TABLE 2
Selected Examples of the Four Revision Purposes
sentence from Anna’s first paper in English has been transcribed,

with revisions in brackets:
Sometimes it happens that while taking a course [in a field not
connected with one’s major], he finds out that actually he would be
50 TESOL QUARTERLY
There are two revisions in this passage. The first one, the insertion
of the prepositional phrase “in a field not connected with one’s
major,” was intended to add clarifying information about the noun
“course.” The second one, the substitution of the preposition “to”
for “with,” corrected a grammatical error.
All revisions in the L1 texts were classified with the help of native-
speaker consultants. These consultants made initial judgments,
which were confirmed after translating the revisions and discussing
them with me. For the L2 texts, revisions were classified by me and
an experienced ESL instructor. Discrepancies were resolved by
mutual consent after discussion.
RESULTS
General L1 and L2 Differences
A simple count of the total number of words in the four subjects’
first and second drafts indicated that more words were generated in
the L1 drafts than in the L2 drafts (9,191 L1 words versus 8,452 L2
words). The mean number of words in the final L1 and L2 drafts
were 625 and 607 respectively. Inspection of Table 3 reveals that the
lowest number of words generated in a final L1 draft was 410 and
the highest was 736. For the L2 writing samples the lowest count in
a final draft was 504 and the highest was 683.
Pauses while composing the drafts also revealed L1 and L2
differences. The average number of L2 pauses while a text was
being drafted (Stages 2 and 4) was 41, compared to 28 pauses
during the L1 drafting stages. Furthermore, these more numerous
pauses consumed a greater amount of time, on the average 40
minutes per actual L2 writing time compared to 23 minutes for L1
writing time.
TABLE 3
Number of Words in Final L1 and L2 Drafts
There was also a noticeable difference in the number of L1 and

L2 revisions. A total of 734 revisions were identified in the L1 and

L2 writing samples. Table 4 lists the distribution of these revisions
by subjects. Wei, who characterized herself as a careful and precise
reviser for style, had the most first language revisions with 87.
Anton, who frequently described himself as a haphazard reviser,
showed the most second language revisions with 120. In general, as
Table 4 reveals, L2 revisions noticeably exceeded LI revisions by a
substantial margin (447 L2 revisions as compared to 287 L1
revisions).
TABLE 4
Revisions for Each Subject and the Rate of Revision
Per 100 Words of Texta
Simple Frequencies
a Revisions divided by the total number of words generated by a subject in either L1 or L2

drafts and then multiplied by 100. Findings are in parentheses.
In addition, a comparison of the frequencies of revision per 100

words generated both in first and second drafts revealed that, on
the average, 5.29 L2 revisions were made in contrast to 3.12 L1
revisions. For all the subjects in this study, the rate of second
language revisions was conspicuously higher than that for first
language revisions. Second language revisions, in short, were almost
twice as numerous and came almost twice as frequently as first
language revisions, indicating that a significantly greater burden
was placed on the subjects’ ability to revise their English texts.
Linguistic and Discourse Features

As Table 5 indicates, an analysis of the linguistic and discourse
features of the subjects’ revisions reveal some striking similarities
across languages. The levels of each revision (that is, word, phrase,
clause, sentence, paragraph, or global) indicated that 62% (177) of
L1 revisions and 59% (262) of L2 revisions focused on single words.
The next most frequent level was the phrase, with 29% (83) of the
total L1 revisions and 26% (120) of the total L2 revisions. All other
levels accounted for 9% of the L1 and 15% of the L2 revisions. The
52 TESOL QUARTERLY
findings show parallels with those of first language researchers such
as Sommers (1980) and Bridwell (1980), that student writers tend to
focus on the subunits of sentences.
TABLE 5
Revisions by Level, Type, and Purpose: Simple Frequencies, Rate of Revision
per 100 Words , and Percentage of Total Number of Revisions
When the types of linguistic processes were analyzed, substitu-

tions dominated the operations. There were 156 L1 substitutions or
54% of the total in the first language writing samples, and 201 L2
substitutions or 45% of the total. L1 and L2 substitutions together
accounted for 48% of the total revision operations. Addition came
next in the L1 writing samples (21% or a count of 76), but deletion
(23% or a count of 142) was the second most prevalent operation:
This represented one of the few prominent divergences across
languages.
Few researchers in ESL writing have commented on the role of
revision when composing. However, in first language studies,
Sommers (1980) has noted that experienced writers were more
inclined to add material to their texts, in contrast to first-year

college students, who deleted or substituted material. Bridwell
(1980) describes a “stuttering effect”: Students reworked certain
words and phrases with a combination of deletions, additions, and
substitutions. In the present study there are also examples that could
be characterized as stuttering. Bjorn, in particular, observes that
choosing the right word or phrase to convey his meaning frequently
required repeated attempts. In addition, he notes that some
revisions grew from alterations of single words to more ambitious
reworkings of entire sentences:
I’d think of the most important word in a sentence as a noun and start
writing with this word in my mind. Halfway through the sentence, I’d
decide that it wasn’t right. So I thought of another one. But this word
was a verb. I’d have to go back and start the sentence all over again. If
that word couldn’t get it right, again I started over.
Instances of recurring revisions, such as Bjorn describes, which
are triggered by a single word, were infrequent. Only 14 cases were
identified in the L1 and L2 texts (6 in the L1 texts and 8 in the L2
texts). However, they suggest a rather sophisticated revising
technique, one that requires the manipulation of semantic-syntactic
relationships.
One effective L2 strategy was found associated with recursive
substitutions in the writings and observations of Anna. Occasionally,
she would underscore a particular word or phrase with a wavy line
to flag it for future review. Returning to the flagged item at a later
time, she would either insert some other item in its place or scratch
out the wavy line. Later, she explained that she used this device
when she could not recall the exact English word she wanted. She
would therefore put down the first word that approximated the
meaning she was struggling for, then continue writing until at some
later pause in her writing, the word would “magically come into my
mind.” Presumably, quickly flagging the problem would prevent
her from losing the stream of thought that was guiding her forward
progress. As the words of her text continued to unfold before her
eyes, or at a pause that allowed her to consider what she had written
from the problem word to the pause, the word that had eluded her
would become apparent from the context of what she had written.
This strategy, which seems well-suited to a second language
context, allowed Anna to discover an elusive lexical item. Lay (1982,
1983) discusses a strategy of code-switching in Chinese writers’
English compositions that is not entirely unlike Anna’s. In Lay’s
study, her subjects would often use Chinese characters when the
English word or words eluded them. These processes of switching,
therefore, also served as a heuristic tool.
54 TESOL QUARTERLY
In the analysis of revision purposes, the data revealed that
revisions affecting the information were the most numerous in both
languages: They accounted for 52% (149) of the L1 revisions and 51%
(230) of the L2 revisions. An appreciable number of these
informational revisions also displayed similarities with Faigley and
Witte’s (1981) meaning-preserving changes, that is, changes that do
not alter the basic meaning of a sentence but rather paraphrase it.
Grammatical/mechanical revisions were next, representing 38%
(109) of the L1 revisions and 42% (186) of the L2 revisions. Across
languages, however, these grammatical/mechanical revisions were
noticeably different. Most of the L1 revisions dealt with the
mechanics of spelling, On the other hand, the L2 revisions were
definitely grammatical in nature—corrections of prepositions,
articles, or inflections. Few of these grammatical revisions,
however, encompassed more than a single word, and often they
came immediately after some revision of the information. Cosmetic
revisions, at 7% (60) of the combined L1 and L2 revisions, were least
frequent in the subjects’ texts and were most often simple rewrites
of illegible handwriting.
Revision Stages
Like Gaskill (1987), I found that over half the L1 and L2 revisions
occurred during the actual drafting of texts (Stages 2 and 4). No
patterns of linguistic and discourse categories were associated with
these drafting stages; a variety of revision levels, types, and
purposes appeared in each stage.
As in the drafting stages, no pattern was associated with the
between-draft stage (Stage 3). Indeed, the behavior of the subjects
appeared erratic. Only one subject, Anna, attempted to revise
during this stage in the L1 writing assignments. The others, while
sometimes rereading what they had written, went directly to the
final in-progress draft (Stage 4). In the L2 samples, Anna and Wei
consistently reread and revised their texts between drafts, most
often concentrating on the information. Anton and Bjorn reread and
revised between drafts only on the first paper, Anton concentrating
on spelling errors and Bjorn dividing his revisions between
grammatical and informational purposes.
Perhaps the most remarkable feature discovered about the L2
versus the L1 revision stages concerned the predraft stage. Although
none of the four subjects constructed predraft plans or notes for
their L1 texts, Anna, Bjorn, and Wei made a conscientious effort to
sketch out plans or notes for their L2 texts. In addition, these three
writers made revisions in the process. Anna recorded the most

predraft revisions with 39; Bjorn had 8; and Wei produced 1.
Although the number of revisions was small, word-level changes
clearly predominated. Among revision types (Table 1), substitu-
tions predominated; among revision purposes (Table 2), informa-
tional changes comprised the majority. Comments by Anna
revealed something of the motives for her predraft plans. She
confided that the notes she made prior to writing an English text
helped her “test the words I think are most important to the paper.”
This involved seeing how a particular word and sentence fit
together to make an important argument. Once she started writing
her initial draft, however, she rarely referred to these notes. They
were not intended to be a “road plan” for her paper, as she
commented. Bjorn and Wei exhibited similar behavior; however,
their predraft sketches looked more like simple outlines. Yet once
they were constructed, no attempt was made to refer to them
during the construction of the initial drafts. These predraft notes
and their revisions served a special but limited purpose in their
writing: They were rehearsals for later textual statements.
DISCUSSION
The results of this study reveal some striking similarities between
L1 and L2 revisions with regard both to the linguistic and discourse
features of the changes and to the stages at which the changes were
initiated. They suggest that an advanced ESL writer is capable of
utilizing a single system of revision across languages. It appears that
this system is initially shaped in the first language and subsequently
transferred to the second language. It would be interesting to
determine whether the system draws on knowledge gained in the
second language, indicating that the process of transfer is
bidirectional and interactive.
In spite of these results, nevertheless, some differences need to be
addressed. First of all, L2 revisions were more time consuming and
numerous, suggesting that composition in a second language places
a far greater burden on revision while managing the complexity of
text production.
Furthermore, a few revising strategies were unique to the second
language. There were more revising and reviewing episodes.
Recursiveness took on an additional function in the second language
as the writers grappled with the semantics of words and the
structures of sentences. During the process, they would either flag
problematic words or list options, then move on, later returning
with new insights. These insights sometimes prompted changes to
entire sentences. Similar behavior has been noted by Raimes (1985)
56 TESOL QUARTERLY
in her study of unskilled ESL writers and by Zamel (1983) in her
study of advanced ESL writers.
Finally, a new function was ascribed to planning. Although plans
were sketched before the start of a draft and revised in the process,
they served the rather limited purpose of rendering a mental
scheme visual so that key ideas and words that formed the gist of
important textual statements could more efficiently be shaped and
reworked.
These differences indicate that revising in a second language is
not simply a mirror image of that process in the first language. The
system appears to be more flexible. Indeed, Flower, Hays, Carey,
Schriver, and Stratman (1986) have argued that “revision, then, is a
strategic action, adapted to the necessities of the task” (p. 19). This
adaptability, though, extends beyond the scope of the first
language. Writers can adapt not only to the problems imposed by
different texts but also to the problems imposed by different
languages.
IMPLICATIONS FOR THE CLASSROOM
This research suggests several implications for the ESL class-
room. First, revision plays a role in shaping both the semantics and
syntax of sentences in a text; this function deserves special
consideration in second language instruction. For the writers in this
study, this problem alone consumed most of the energy and time
they dedicated to revising. Indeed, testimonies by these writers
suggest that not only the meanings of words but also knowledge of
how these words shape the structure of sentences are crucial second
language skills. Vocabulary and structure, then, complemented
each other; instructors should consider ways of teaching them in
conjunction.
Second, revision cannot be appreciated or nurtured if it is
divorced from ESL writers’ own texts; their texts contain complex
plans for the discourse, and manifest writers’ individual struggles in
mastering the language. Furthermore, the findings in this study
seem to indicate that, at times, second language writers must
literally see the stages of a revision. Often, the advanced language
users in this study detected, categorized, and eventually corrected
the dissonance in language and discourse as their texts unfolded
before their eyes. Consequently, it appears that the ability to revise
develops and improves when ESL writers confront problems in
their own writing. Teaching students prescriptive revision, then, is
less effective than helping them individualize their revising
processes. For the second language writer, it is a skill that can be
adapted to the text and to the text’s language.

Third, the findings in this study should prompt a reconsideration
of the debate surrounding product-oriented instruction versus
process-oriented instruction. Our enthusiasm for the process
classroom, which emphasizes writing, then talking about writing,
and finally rewriting, has tended to devalue those instructional tools
assumed to focus solely on the product. In particular, outlining has
been dismissed as a straightjacket on composing. This belief is due
to a suspicion that fixing plans on paper at the start of a writing task
obstructs the more productive process of working ideas through in
multiple drafts. However, drafting and outlining are not mutually
exclusive. Indeed, viewed in the context of this study at least, a form
of outlining may have a salutary effect on particular writing tasks,
e.g., argumentation. Witte (1987) contends that “some writers
appear capable of constructing the substance of their arguments
before they finish reading the assignment sheet” (p. 419). Perhaps
there are ESL writers who must take this process one step further:
They must transfer their arguments to paper to facilitate what must
be a rather complex mental activity that includes setting goals,
evaluating them, and revising wording and structure. Furthermore,
this tool for managing complexity does not have to be limited to one
stage of the composition process. Spack (1985) has made this rather
liberating observation about outlining:
To outline today means not [only] to organize ideas before writing, but
to organize ideas after generating them by means of thinking and
writing—a complex, recursive series of mental and scriptural procedures
writers undergo each time they confront a writing assignment. (p. 398)
As writing teachers we must reexamine the complete repertoire
of instructional methods, integrating whichever procedures are
found to assist writing processes in a second language.
- .
ACKNOWLEDGMENTS
This research, conducted at the University of Wyoming, is based on a PhD
dissertation from the University of New Mexico under the direction of Dr. Lynn
Diane Beane.
THE AUTHOR
Chris Hall is Assistant Professor in the Department of English Language and
Literatures at Wright State University where he teaches linguistics and ESL
composition. His research interests include text linguistics, ESL composition, and
computers in writing.
58 TESOL QUARTERLY
REFERENCES
Applebee, A. N. (1984). Contexts for learning to write: Studies of
secondary school instruction. Norwood, NJ: Ablex.
Bartlett, E. J. (1982). Learning to revise: Some component processes. In M.
Nystand (Ed.), What writers know: The language and structure of
written discourse (pp. 345-363). New York: Academic Press.
Beach, R. (1976). Self-evaluation strategies of extensive revisers and non-
revisers. College Composition and Communication, 27, 111-119.
Bridwell, L. S. (1980). Revising strategies in twelfth grade students’
transactional writing. Research in the Teaching of English, 14, 197-222.
Chelala, S. I. (1981). The composing process of two Spanish speakers and
the coherence of their texts: A case study. Unpublished doctoral
dissertation, New York University, New York.
Dunkel, H. B. (1948). Second-language learning. Boston Ginn.
Edelsky, C. (1982). Writing in a bilingual program: The relation of L1 to L2
texts. TESOL Quarterly, 16, 211-228.
Faigley, L., Cherry, R. D., Jolliffe, D. A., & Skinner, A. M. (1985).
Assessing writers’ knowledge of processes of composing. Norwood, NJ:
Ablex.
Faigley, L., & Witte, S. P. (1981). Analyzing revision. College Composition
and Communication, 31, 21-32.
Faigley, L., & Witte, S. P. (1984). Measuring the effects of revisions on text
structure. In R. Beach & L. S. Bridwell (Eds.), New Directions in
Composition Research (pp. 95-108). New York: The Guilford Press.
Flower, L., Hays, J. R., Carey, L., Schriver, K., & Stratman, J. (1986).
Detection, diagnosis, and the strategies of revision. College Composition
and Communication, 37, 16-55.
Gaskill, W. (1987). Revising in Spanish and English as a second language:
A process-oriented study of composition. Unpublished doctoral
dissertation, University of California, Los Angeles.
Heuring, D. L. (1984, March). Revision strategies of ESL writers: Five case
studies. Paper presented at the 18th Annual TESOL Convention,
Houston, TX.
Jacobs, H. L., Zinkgraf, S. A., Wormuth, D. R., Harfiel, V. F., & Hughey,
J. B. (1981). Testing ESL composition: A practical approach. Rowley,
MA: Newbury House.
Jones, S., & Tetroe, J. (1987). Composing in a second language. In A.
Matsuhashi (Ed.), Writing in real time: Modelling production processes
(pp. 34-57). New York: Longman.
Lay, N. D. (1982). Composing processes of adult ESL learners: A case
study. TESOL Quarterly, 16, 406.
Lay, N. D. (1983). Code-switching in the composing processes of adult
ESL learners, Unpublished manuscript.
Linnarud, M. (1977). Some aspects of style in the source and the target
language. In J. Fisiak (Ed.), Papers and studies in contrastive linguistics
(pp. 85-94). Arlington, VA: Center for Applied Linguistics.

Monahan, B. D. (1984). Revision strategies of basic and competent writers
as they write for different audiences. Research in the Teaching of
English, 18, 288-301
Murray, D. (1978). Internal revision: A process of discovery. In C. R.
Cooper & L. Odell (Eds.), Research on composing (pp. 85-103).
Urbana, IL: National Council of Teachers of English.
Raimes, A. (1985). What unskilled ESL students do as they write: A
classroom study. TESOL Quarterly, 19, 229-257.
Sommers, N. (1980). Revision strategies of student writers and experi-
enced writers. College Composition and Communication, 31, 148-156.
Spack, R. (1985). Comments on Joy Reid’s “The radical outliner and the
radical brainstormer A perspective on composing processes.” TESOL
Quarterly, 19, 396-398.
Stallard, C. K. (1974). An analysis of the writing behavior of good student
writers. Research in the Teaching of English, 8, 206-218.
Stendahl, C. (1972). Proficiency in Swedish and English as shown by
Swedish university students of English. Moderna Sprak, LXVl, 117-123
Witte, S. P. (1987). Pre-text and composing. College Composition and
Communication, 38, 397-425.
Zamel, V. (1983). The composing processes of advanced ESL students: Six
case studies. TESOL Quarterly, 17, 165-187.
Zamel, V. (1984). In search of the key: Research and practice in
composition. In J. Handscombe, R. A. Orem, & B. P. Taylor (Eds. ), On
TESOL ’83 (pp. 195-207). Washington, DC: TESOL.
60 TESOL QUARTERLY
Another Turn in the Conversation:

What Does Cloze Measure?
JOHN JONZ
East Texas State University
This study addresses a controversy in cloze testing. At issue is

whether the cloze procedure measures comprehension that ranges
beyond the context immediately surrounding a cloze deletion.
Eight cloze passages published over the past 15 years were
analyzed, using a system that (a) estimates the quantity of text
required to cue closure of any one blank (Bachman, 1985) and (b)
considers the linguistic category of the deleted word. The research
reported here demonstrates that across the cloze tests considered,
the standard fixed-ratio cloze procedure has a high level of
sensitivity to intersentential ties and lexical selections, and that the
kinds of language knowledge required to complete cloze tests is
virtually the same from one test to the next. The implication of
these findings is that the fixed-ratio cloze procedure is far from
erratic in its selection of item types. This study suggests that, for
deriving tests of language comprehension, the cloze procedure
produces tests that are generally consistent in the ways they
measure the language knowledge of examinees.
This paper contributes to what is known about cloze procedure,

the practice of measuring language proficiency or language com-
prehension by requiring examinees to restore words that have been
removed from otherwise normal text. The study is intended to help
resolve the controversy as to whether the cloze procedure measures
comprehension that ranges beyond the context immediately
surrounding a cloze deletion.
Some researchers have found cloze to be insensitive to such long-
range constraint and, in addition, have found it to be unreliable and
erratic (e.g., Alderson, 1979, 1980; Kibby, 1980; Klein-Braley, 1983;
Leys, Fielding, Herman, & Pearson, 1983; Markam, 1985; Porter,
1978, 1983; Shanahan & Kamil, 1982, 1983; Shanahan, Kamil, &
Tobin, 1982). Others find cloze to be stable, reliable, and sensitive
to comprehension processes at various levels (e.g., Bachman, 1982,
1985; Brown, 1983; Chavez-Oller, Chihara, Weaver, & Oller, 1985;
61
Chihara, Oller, Weaver, & Chavez-Oller, 1977; Cziko, 1978;
Gamarra & Jonz, 1987; Jonz, 1987; McKenna & Layton, 1986; Oller,
1975; Rankin & Thomas, 1980; Taylor, 1957). The research reported
here supports this latter position by demonstrating (a) that the
standard fixed-ratio cloze procedure has a high level of sensitivity to
intersentential ties and lexical selections across the eight cloze tests
considered, and (b) that the kinds of language knowledge (clause
structure, interclausal relationships, intersentential ties, lexis)
required to complete cloze tests is virtually the same from one test
to the next.
In its broadest terms, the controversy among researchers re-
garding the capacity of cloze tests to measure long-range constraint
on response involves disagreement over the content of cloze tests
and the comparability of that content across tests. Sharp differences
exist about what cloze tests measure and about how nearly alike any
two cloze tests might be.
For example, Alderson (1979, 1980) contends that for any fixed-
ratio cloze procedure (wherein every nth word is removed), the
clausal and textual functions of deleted words will not necessarily
be similar to the functions of words deleted from any other cloze
test. According to this point of view, no two cloze tests can be
assumed to measure the same range of constraint on comprehen-
sion: Since no two texts have the same aggregation of lexical items,
syntactic patterns, discourse characteristics, and rhetorical traits,
every cloze test must necessarily have distinct content and must
place a unique demand on the examinee. Beyond this undesirable
variability across tests, it is contended, the act of restoring words to
context involves mostly “lower-order” linguistic ability, essentially
clause-level syntactic processing, and does not reflect the activation
or application of higher-order cognitive processes (Alderson, 1979).
In this view, the fixed-ratio cloze procedure is assumed to create
syntax-bound tests that vary dramatically from one another in
content and must be studied individually for evidence of reliability
(Klein-Braley, 1983).
On the other hand, other researchers (e.g., Chavez-Oller,
Chihara, Weaver, & Oller, 1985) point to an underlying similarity
among texts and among the language comprehension processes of
the people who produce and use them. Comprehension results from
the interaction between texts and people; its components are not
infinitely variant, regardless of differences among texts and among
people. The cloze procedure, it is asserted, challenges universal
processing mechanisms (see discussion in Jonz, 1987) at all levels
from word recognition through concept building; therefore,
responding to cloze tests must necessarily involve a great deal of
62 TESOL QUARTERLY
higher-order language processing. Any variation in cloze test scores
that might be attributed to superficial variation across texts is more
apparent than real. Since a cloze procedure measures the interaction
between text and language user, variation in cloze test scores simply
reflects variation in this interaction (Jonz, 1987).
Two recent studies by Bachman (1982, 1985) are particularly
pertinent to the resolution of this controversy. In the 1982 study,
Bachman concludes that cloze test scores reflect a general language
proficiency factor along with three specific traits: a syntactic
(clause-level) trait, a cohesive (interclausal and intersentential) trait,
and a strategic (semantic) trait (drawing on the contrast between
short-range “tactical” issues and longer-range “strategic” issues).
Significantly, Bachman demonstrates that cloze scores reflect not
merely lower-order phrase-processing, but complex skills ranging
along a hierarchy of lower- to higher-order human language
processing capacities.
The 1982 study is limited, however, by the fact that, before
constructing his cloze test, Bachman first predicted the distribution
of processing constraint (item types) that the scores on a cloze test
should reflect. Then, using a rational (i.e., not fixed-ratio) method,
he carefully deleted selected words (see Appendix B). Bachman did
not necessarily allow the deleted words to reflect whatever
constraint might have derived inherently from the text. His
conclusions, therefore, cannot be directly applied to standard fixed-
ratio cloze tests. Bachman’s (1982) study allows only the conclusion
that a cloze test can be made to reflect the designer’s assumptions
about human comprehension processes and the structure of
language within linguistic texts.
The 1982 study does not address whether Bachman’s assumptions
about language and human processing capacity would be sustained
in a standard fixed-ratio (i.e., random deletion) cloze test. Bachman
is fully aware of this limitation; he concludes his study with the
following suggestion:
It will be necessary to examine the internal structure of cloze passages

with random deletions [italics added]. Identifying the types of deletions
which occur in the passages of the numerous studies which have used ran-
dom deletion, and analyzing the patterns of responses to these deletions
through factor analytical procedures (both exploratory and confirmatory)
would appear to be a promising starting point. (pp. 66-67)
In the second of his recent studies, Bachman (1985) revises his

earlier analytical system to reflect the quantity of text that one
might hypothesize is required to cue the successful closure of any
WHAT DOES CLOZE MEASURE? 63

blank. This newer approach involves four classifications of item
types: (1) within clause; (2) across clause, within sentence; (3) across
sentence, within text; and (4) extratextual (p. 539). For this study,
Bachman created several cloze tests from a single passage of
otherwise normal English-language prose (see Appendix B). He
designed one rational-deletion cloze test that emphasized deletions
from the “cohesive” (cf. Bachman, 1982) categories: categories 2
(across clauses) and 3 (across sentences). He then made three
versions of a fixed-ratio test by deleting every 11th word, starting
with the 39th, 40th, and 41st words in the passage. His purpose in
creating these three versions of the every-11th-word cloze test was
to determine whether the distribution of item types was the same
across the different versions of the test; he concluded that across
versions there were no systematic differences in the relative
frequency of item types (cf. Jonz, 1988 for a study that draws the
same conclusion).
After confirming this consistency across cloze tests created from
the same passage, Bachman went on to hypothesize that the same
relative frequency of the four item-types discovered in this study
would be characteristic of fixed-ratio cloze tests generally. Al-
though this hypothesis was not a central feature of Bachman’s study,
it is well worth pursuing.
Indeed, the questions that Bachman has raised in these two
studies seem central to a general understanding of cloze tests.
Moreover, the discoveries that might be made in seeking answers to
his questions could be helpful to anyone interested in language
testing broadly construed and in the processes involved in the
comprehension of linguistic texts. The research goals of the study
reported here are therefore related directly to Bachman’s two
studies: First, we shall be interested in discovering whether the
types of deletions made on standard fixed-ratio cloze tests are
similar from one test to the next (cf. Bachman, 1982). Additionally,
we will want to determine what classification system will best
account for whatever deletion patterns exist. That is, would a three-
part system (Bachman, 1982), a four-part system (Bachman, 1985),
or indeed some other classification system be most helpful?
METHOD
Materials
To pursue these questions, I selected eight cloze passages (repro-
duced in Appendix B) published over the past 15 years: two from
Bachman (1982, 1985); two from Chavez-Oller et al. (1985); two
from Gamarra and Jonz (1987); and two from Jonz (1975, 1987).
64 TESOL QUARTERLY
Although not a random sample of all cloze tests ever published,
this selection was appropriate to the task: to investigate the degree
to which cloze tests are similar to, or distinct from, one another. The
sample is large, readily available, and representative of a wide
variety of topics, text types, deletion rates, and passage lengths. It
should be noted that I created two of these tests and assisted in the
design of two others; however, I had no involvement with the other
four and assume the use of this selection from the current literature
is not unreasonably biased.
Text topics included (1) the transmission of disease; (2) the rise of
automatic control systems; (3) the problems college students might
encounter in leaving home for the first time; (4) a visit to relatives
abroad; (5) a famous psychological experiment; (6) the characteris-
tics and habits of woodpeckers; (7) the Latin American counterpart
to April Fools’ Day; and (8) an excerpt from Robert Hutchins’ well-
known account of his own undergraduate education. Text types
comprised technical description and exposition, personal narratives,
a chronological report, an excerpt from an encyclopedia entry, and
a folk tale. Fixed-ratio deletion rates varied: For each text, words
were removed from every nth spot, where n = 6, 7, 11, or 15. One
passage from Bachman (1982), though representing a rational
deletion pattern (i.e., a variable deletion rate), was also analyzed.
Finally, the lengths of the tests varied substantially: Two contained
30 deleted items, one contained 48, three contained 50, one
contained 56, and one contained 65.
Though a nonrandom procedure for selecting tests represents a
limitation on the generalizability of the results reported in this
study, I am confident that the passages I have analyzed are
representative of a substantial proportion of all cloze tests. Readers
will want to consider carefully the variety outlined among the test
passages, and evaluate my findings in that context.
Design
To discover the degree of similarity or dissimilarity among the

passages, I first applied Bachman’s (1985) four-category analytical
approach to each of the passages and then compared the
frequencies of deletion types in the passages by applying the chi-
square statistic. This nonparametric statistic (see, for example,
Seigel, 1956 and the sources cited therein) is the one best suited to
the task of determining the comparability of the distribution of
frequencies in a contingency table of nominal data. Moreover, the
assumptions regarding the data to which the chi-square statistic is

applied are fewer and far weaker than are those associated with
parametric tests (such as t-tests, for example).
With regard to using chi-square, three precautions need to be
observed, the first two of which are particularly important: (a) The
categorizations should be independently derived; (b) the expected
(or hypothetical) frequency for any categorization for any one
passage should be reasonably large; and (c) no observation should
be simultaneously assigned two or more categorizations.
With respect to the first of these assumptions, it would seem clear
that my categorization of any one cloze deletion was only very
rarely dependent upon my categorization of another. For example,
my categorization of items #24 and #26 in the passage published in
Bachman, 1985 (see Table 1, p. 68) could be seen as a violation of
this assumption of independence of observation. These cases were
very rare in the 349 categorizations that were required in analyzing
the seven fixed-ratio passages (379 categorizations when I include
the rational-deletion passage from Bachman, 1982) and I am
confident that this assumption for using the chi-square statistic has
not been significantly violated.
With respect to the second assumption, that of reasonably large
expected frequencies, only one of the comparisons that I report
below might be considered suspect: the four-category comparison
(see Table 2, p. 69). Of the 28 cells in this comparison, 7 (25%) have
expected frequencies below 5. The normally cited criterion is that
no more than 20% of the cells have expectations this small. Along
with my nonrandom procedure for selecting passages, the reader
will want to bear this problem in mind while evaluating this report.
To interpret the chi-square statistic, I decided that if the
probability associated with the computed chi-square fell below a
liberally chosen level, p <.10, then I would have concluded that the
distribution of deletion types was significantly dissimilar across
passages.
Beyond this, however, one is generally able to assess the degree of
similarity across frequency distributions by evaluating the chi-
square p-value. As the size of the chi-square statistic becomes
smaller (and consequently the p-value becomes larger, approaching
unity [1.0] ), the more similar are the frequency distributions under
examination (see Bachman, 1982 or Jonz, 1988 for further discus-
sion). For example, notice that the p-value reported in Table 1 is
.005, indicating that the two frequency distributions are highly
significantly different; that is, my way of labeling item types is
substantially distinct from Bachman’s. In contrast, notice that the
probability associated with the chi-square statistic in Table 2 (.971)
66 TESOL QUARTERLY
falls just .03 short of unity, indicating that the distribution of item
types in each of the seven cited passages is remarkably similar to all
of the others.
My research design, then, included three steps: (a) Each cloze
deletion was assigned to one of Bachman’s (1985) four categories on
each of the eight tests (a total of 379 deletions); (b) a contingency
table was arranged and the chi-square statistic computed; and (c)
the computed chi-square was evaluated in terms of its probability
level and its magnitude in relation to its associated degrees of
freedom. All computations were made using the CROSSTABS
routine in the Statistical Package for the Social Sciences (SPSSX,
Release 3.0 for IBM VM/CMS).
Procedure
The flowchart in Figure 1 reflects my initial operationalization
of Bachman’s (1985) system for categorizing cloze deletions.
FIGURE 1
One Interpretation of Bachman’s (1985) Categories
Unfortunately, my application of this analytical procedure pro-
duced a number of problems.
I had determined that the prudent course of action would be to
analyze the passage from Bachman’s 1985 paper first. Since he had
published his analysis of this passage, I could easily compare my
analysis to his, thereby confirming my intuitive grasp of his
categorization system.
TABLE 1
Two Analyses of Bachman (1985)
68 TESOL QUARTERLY
I could not replicate Bachman’s (1985) analysis as it appears in the
appendix to his paper. For 14 cloze deletions out of 30, my analysis
did not match his (see Table 1). This unexpected development led
me to document very carefully the decisions that I made as I
attempted to categorize the deletions in each of the passages.
RESULTS
Applying my four-part categorization decisions to the seven
fixed-ratio passages, I discovered that the pattern of constraint on
responses was impressively similar across cloze passages, x 2 (18,
N = 349) = 8.47, p = .971. Table 2 summarizes the frequency with
TABLE 2
Four Constraint Levels Compared
which each category of constraint was encountered for each

passage (the item-by-item analyses of each of these passages appear
in Appendix B). Even with the addition of an analysis of the passage
from Bachman (1982), which did not employ a fixed-ratio deletion
pattern, the fit among passages remains quite good, x 2 (21,
N = 379) = 14.38, p = .853. Recall, however, that for these four-
category analyses, one of the assumptions for applying the chi-
square statistic is violated: The percentage of cells with expected

frequencies below five exceeds 20%, if only by a few percentage
points.
This initial stability of my analytical approach, along with the
need to find a chi-square solution in which the assumptions were not
violated, encouraged me to elaborate slightly on the four-category
approach. The first step was to make it more explicit in its account
of short-range constraint. I divided the first category (the “within
clause” category) into two subcategories: (a) clause-level syntax
(Category l-S) and (b) clause-level lexis (Category l-L). (Appendix
A includes an outline of these categorizations as they evolved during
the study.)
An additional advantage in doing this was that I might uncover
evidence that cloze tests measure higher-order, conceptual
processing to a significant extent, as opposed to being focused on
the application of lower-level syntactic knowledge. I assumed that
lexical choices represent higher-order knowledge, and I wanted to
segregate such locally constrained higher-order choices from
choices associated essentially with syntax.
As is clear from the data displayed in Table 3, for only one fixed-
ratio passage of the seven do the syntactic items exceed the local
TABLE 3
Five Constraint Levels Compared
70 TESOL QUARTERLY
lexical items in number. Moreover, in applying this slightly
elaborated five-category system, I found once again that each
passage had essentially the same pattern of constraint as did the
others, x 2 (24, N = 349) = 13.14, p = .964. Additionally, for this
more elaborate analysis, the number of cells with small expected
frequencies dropped below 20%.
Moreover, taking my lead from Bachman’s (1982) findings, when I
hypothesized just three sources of constraint, a similar, if not quite as
remarkable, consistency across passages was revealed. To arrive at a
three-category analysis, I combined my lexical category (l-L) (see
Appendix A) with Bachman’s (1985) “extratextual” category to
approximate a “strategic” trait (I call this category “local/distant
Iexis”). Like Bachman (1982), I treated interclausal and intersentential
categories as one to reflect a “cohesive” trait. I then allowed the
sentence structure (clause-level syntax) category to stand on its own
to represent a “syntactic” trait, notwithstanding the obvious
“grammatical” content of some “cohesive” items. The application of
this categorization system also yields a substantial fit among the
passages, x 2 (12, N = 349) = 6.83, p = .869 (see Table 4). Further-
more, this analysis has no cells with expected frequencies below five.
TABLE 4
Three Constraint Levels Compared

The implication of these findings is that, at least for the passages
studied, the fixed-ratio cloze procedure is far from erratic in its
selection of item types. This is very encouraging. It suggests that the
cloze procedure for deriving tests of language comprehension, a
procedure so uncomplicated that it can be accomplished easily by
anyone with a photocopier and a bottle of correction fluid,
produces tests that generally challenge the language knowledge of
an examinee in very similar ways.
DISCUSSION
This study demonstrates that, for the passages analyzed, con-
straints on cloze response derive, in order of frequency, from (a)
lexical selections (41.8%), (b) textual cohesion (34.1%), and lastly, (c)
syntax (24.1%). (See Table 4.) These findings support the contention
that the cloze procedure is sensitive to constraints on response
ranging well beyond the level of local syntax. This study supports
previous results with regard to the cohesive content of cloze tests
reported, for example, in Brown (1983), Chavez-Oller et al. (1985),
and Jonz (1987). In general, it sustains the procloze position outlined
at the beginning of this article.
However, two important issues remain that merit continued
attention from researchers. First, my three-category analysis varies
from Bachman’s (1982) three-category analysis for about 25% of the
items. Although this is a much better match, x 2 (2, N = 60) = 1.74,
p = .419, than between my four-category analysis and Bachman’s
(see Figure 1), discrepancies still exist. Additionally, in informal
settings, other skilled linguistic analysts have produced slightly
different analyses than either mine or Bachman’s. Rather than
assuming a simple instability in the analytical approach (variation in
category definition and data coding), I take these discrepancies to
indicate that the constraints on response for any cloze item might, in
fact, vary in principled ways from one person to the next. This
possibility merits careful investigation. We must assume that there
are various mental routes to comprehension; perhaps the responses
to cloze tests reflect a principled variation in the application of
communicative rules.
Second, the near congruence of content across passages that I
cited above for the three-, four-, and five-level analyses is also
characteristic of several of the possible two-category solutions. For
example, when I compare “cohesion” (sentence- and text-level
categories combined) to the combination of all other categories
across the seven fixed-ratio passages, the fit remains substantial,
72 TESOL QUARTERLY
x 2 (6, N = 349) = 1.72, p = .944, Likewise, when I compare
“extratextual” to all other categories combined, the fit is similarly
complete, x 2 (6, N = 349) = 2.65, p = .851. Even the comparison
(suggested by Mauranen, 1988) between “micro” constraint
(categories l-S, l-L, and 2) and “macro” constraint (categories 3 and
4) yields a very impressive fit, x 2 (6, N = 349) = 2.56, p = .861.
Perhaps a two-trait description of cloze procedure is possible or
even desirable.
That such could be the case is evident in Lee’s (1985) recent factor
analytical work. For three different cloze passages, Lee observed
that the general factors were “bipolar” (p. 139). He concluded that
constraint on cloze response might derive from just two sources:
relative “closeness” as opposed to relative “openness” as regards
possible responses. To get a sense of what this means, compare the
two items below.
1. The man in moon
2. The Secretary of State is today flying to to discuss peace
prospects.
The first item obviously allows far less variation in response than
does the second, and it could therefore be characterized as more
“closed.” It is quite possible that on further examination the two-
category solutions that I have mentioned above could amplify Lee’s
findings.
These converging lines of research (Bachman, 1982, 1985; Lee,
1985; and the present study) could be reconciled in a future study.
Data should be collected and factor analyzed to ascertain whether
cloze procedure scores might best be accounted for in a model that
both reflects the dichotomous nature of the two-category analyses
and respects three specific traits (“sentence structure,” “vocabu-
lary, “ “cohesion” [both “lexical” and “syntactic”]) or four traits
(“within clause,“ “within sentence, “ “across sentences,” “extratex-
tual”) or even five (“local syntax,” “local lexis,” “cohesion within
sentence, “ “cohesion across sentences,” “extratextual”).
Furthermore, in future studies we might assume (on the basis of
Jonz, 1987) that the comprehension processes of nonnative speakers
rely more heavily on textual cohesion (data-driven vs. knowledge-
based processing) than do those of native speakers. Future studies
should hypothesize that the constraints on native-speaker cloze
responses would vary from those on nonnative-speaker responses.

ACKNOWLEDGMENTS
This article is a revised and expanded version of a paper (“Constraint on Cloze
Response”) presented at the 22nd Annual TESOL Convention in Chicago, IL,
March 1988. A report of the pilot project was presented at the TEXTESOL annual
meeting in Houston, TX, October 1986.
The project reported here was supported in part by a grant-in-aid of research,
number 1501-9722, from the Graduate School at East Texas State University.
I am indebted to two anonymous TESOL Quarterly reviewers for the careful
reading that each gave to the original manuscript. Their insightful recommenda-
tions have improved the paper enormously. Any remaining infelicities, of course,
are my own.
THE AUTHOR
John Jonz, Professor of Literature and Languages, East Texas State University,
teaches courses in linguistics and language pedagogy. His current research interests
include the interactions among language proficiency, features of text, and
comprehension processes.
REFERENCES
Alderson, J. C. (1979). The cloze procedure and proficiency in English as
a foreign language. TESOL Quarterly, 13, 219-223.
Alderson, J. C. (1980). Native and non-native speaker performance on
cloze tests. Language Learning, 30, 59-76.
Bachman, L. F. (1982). The trait structure of cloze test scores. TESOL
Quarterly, 16, 61-70.
Bachman, L. F. (1985). Performance on cloze tests with fixed-ratio and
rational deletions. TESOL Quarterly, 19, 535-556.
Barlow, G. (Ed.). (1970). Leyendas Latinoamericanas. Skokie, IL: National
Textbook. (Translated from the traditional folktale “Pobre Inocente”)
Brown, J. D. (1983). A closer look at cloze: Validity and reliability. In J. W.
Oller, Jr. (Ed.), Issues in language testing research (pp. 237-250).
Rowley, MA: Newbury House.
Brown, T. H., & Gross, J. T. (Eds.). (1980). Dimensions: Essays for
composition. Cambridge, MA: Winthrop.
Chavez-Oller, M. A., Chihara, T., Weaver, K. A., & Oller, J. W., Jr. (1985).
When are cloze items sensitive to constraints across sentences? Language
Learning, 35, 181-206.
Chihara, T., Oller, J., Weaver, K., & Chavez-Oller, M. A. (1977). Are cloze
items sensitive to constraints across sentences? Language Learning, 27,
63-73.
Cziko, G. A. (1978). Differences in first- and second-language reading:
The use of syntactic, semantic, and discourse constraints. Canadian
Modern Language Review, 34, 473-489.
74 TESOL QUARTERLY
Gamarra, A., & Jonz, J. (1987). Cloze procedure and the sequence of
text. In J. E. Readence & R. S. Baldwin (Eds.), Research in literacy:
Merging perspectives (pp. 17-24). Rochester, NY: National Reading
Conference.
Gregg, J. Y. (1985). Communication and culture. Belmont, CA:
Wadsworth.
Grolier Society. (1956). Our common birds V: Woodpeckers, humming-
birds, swifts, whippoorwills, and nighthawks. In The book of popular
science (Vol. 8, p. 3343). New York: Author.
Hutchins, R. (1941). Education for freedom. Baton Rouge: Louisiana State
University Press.
Jonz, J. (1975). The values hierarchies of three groups of Spanish-English
multilingual adolescents. Unpublished doctoral dissertation, University
of New Mexico.
Jonz, J. (1987). Textual cohesion and second language comprehension.
Language Learning, 37, 409-438.
Jonz, J. (1988, October). An intratextual analysis of cloze. Paper presented
at the meeting of the Linguistic Association of the Southwest,
Albuquerque, NM.
Kibby, M. W. (1980). Intersentential processes in reading comprehension.
Journal of Reading Behavior, 12, 299-312.
K1ein-Braley, C. (1983). A cloze is a cloze is a question. In J. W. Oller, Jr.
(Ed.), Issues in language testing research (pp. 218-228). Rowley, MA:
Newbury House.
Lee, Y. P. (1985). Investigating the validity of the cloze score. In Y. P. Lee,
A. Fok, R. Lord, & G. Low (Eds.), New directions in language testing
(pp. 137-147). Oxford: Pergamon press.
Leys, M., Fielding, L., Herman, P., & Pearson, P. D. (1983). Does cloze
measure intersentence comprehension? A modified replication of
Shanahan, Kamil, and Tobin. In J. A. Niles & L. A. Harris (Eds.),
Searches for meaning in reading/language processing and instruction
(pp. 111-114). Rochester, NY: National Reading Conference.
Markam, P. L. (1985). The rational deletion cloze and global comprehen-
sion in German. Language Learning, 35, 423-430.
Mauranen, A. (1988, March). The semantic cloze: An approach to testing
EFL reading. Paper presented at the 22nd Annual TESOL Convention,
Chicago, IL.
McKenna, M. C., & Layton, K. (1986, December). Cloze and intersenten-
tial comprehension: A direct comparison. Paper presented at the
National Reading Conference, Austin, TX.
Oller, J. W., Jr. (1975). Cloze, discourse and approximations to English. In
M. K. Burt & H. Dulay (Eds.), New directions in second language
teaching and bilingual education: On TESOL ’75 (pp. 345-355).
Washington, DC: TESOL.

Porter, D. (1978). Cloze procedure and equivalence. Language Learning,
28, 333-341.
Porter, D. (1983). The effect of quantity of context on the ability to make
linguistic predictions: A flaw in a measure of general proficiency. In A.
Hughes & D. Porter (Eds.), Current developments in language testing
(pp. 63-74). London: Academic Press.
Praninskas, J. (1959). Rapid review of English grammar. Englewood
Cliffs, NJ: Prentice Hall.
Rankin, E. F., & Thomas, S. (1980). Contextual constraints and the
construct validity of the cloze procedure. In M. L. Kamil & A. J. Moe
(Eds.), Perspectives on reading: Research and instruction (pp. 47-55).
Washington, DC: National Reading Conference.
Seigel, S. (1956). Nonparametric statistics for the behavioral sciences. New
York: McGraw-Hill.
Shanahan, T., & Kamil, M. L. (1982). The sensitivity of cloze to passage
organization. In J. A. Niles & L. A. Harris (Eds.), New inquiries in
reading research and instruction (pp. 204-208). Rochester, NY: National
Reading Conference.
Shanahan, T., & Kamil, M. L. (1983). A further comparison of sensitivity of
cloze and recall to passage organization. In J. A. Niles & L. A. Harris
(Eds.), Searches for meaning in reading/language processing and
instruction (pp. 123-128). Rochester, NY: National Reading Conference.
Shanahan, T., Kamil, M. L., & Tobin, A. W. (1982). Cloze as a measure of
intersentential comprehension. Reading Research Quarterly, 17, 229-255.
Taylor, W. L. (1957). “Cloze” readability scores as indices of individual
differences in comprehension and aptitude. Journal of Applied
Psychology, 41, 19-26.
Tustin, A. (1953). Feedback. In Scientific American reader (p. 528). New
York: Simon & Schuster.
Wright, A. W., Barrett, R. P., & Van Syoc, W. B. (1968). Let’s learn English:
Intermediate course. New York: American Book.
APPENDIX A
Categorization Key for Cloze Items
Within Clause (Syntax)
l-S-a = determiner w/o text antecedent
l-S-b = conjunction below clause level
l-S-c = role-marking preposition
I-S-d = relative word in headless clause
l-S-e = complementizer
l-S-f = auxiliary
l-S-g = negation
l-S-h = copula BE & other intensive relationships
l-S-i = syntactic there or syntactic it + BE
l-S-j = syntactic elements in cleft or pseudocleft
76 TESOL QUARTERLY
Within Clause (Lexis)
l-L-a = referential determiner or deictic w/i clause
l-L-b = pre- & postdeterminers
l-L-c = pronoun (coreferent w/i clause)
l-L-d = indefinite preform (including you as indefinite)
l-L-e = preposition other than role-marker
l-L-f = collocation w/i clause
l-L-g = reiteration w/i clause
l-L-h = multipart item
l-L-i = qualifier in AdjP & AdvP
Across Clause, Within Sentence
2-a = referential determiner or deictic across clauses
2-b = conjunction at clause level
2-c = pronoun (coreferent across clauses)
2-d = relative word in headed clause
2-e = collocation across clauses
2-f = reiteration across clauses
2-g = subordinator
2-h = it of Extraposition
Across Sentence, Within Text
3-a = referential determiner or deictic across sentences
3-b = conjunction between sentences
3-c = pronoun (coreferent across sentences)
3-d = collocation across sentences
3-e = reiteration across sentences
3-f = textual discourse marker
Extratextual
4-a = speech-role pronoun w/o textual antecedent
4-b = lexical item with little or no local clue
APPENDIX B
Cloze Passages
Bachman (1982)
The discovery of the disease, malaria, began in 1680 when a French physician described a
malarial parasite obtained from the blood of one of his patients. Italian investigators later
demonstrated (1) THAT [l-S-e] the disease could be transmitted (2) FROM [l-L-e] human
to human by infected blood, (3) AND [2-b] in the 1980’s British and Italian scientists
suggested (4) THE [l-S-a] type of mosquito named anopheles as the transmitter of the
disease. By 1900 (5) IT [2-h] had been established that (6) THIS [3-a] theory was correct, by
demonstrating that the (7) DISEASE [3-e] was acquired only from the bite of (8) AN [l-S-
a] infected anopheles mosquito, (9) AND [2-b] that persons protected from (10) THIS [2-a]
mosquito did not contract the disease even in regions where (11) MALARIA [3-e] was rife.
The biological cycle of (12) THE [3-a] parasite has now been described in sufficient detail to
explain (13) WHY [l-S-d] quinine was an effective remedy, why (14) THE [3-a] bite of an
infected mosquito (15) DID [l-S-f] not transmit the disease until several days (16) AFTER [l-
L-e] the mosquito had become infected, and why (17) OTHER [l-L-h] kinds of mosquitoes
than the (18) ANOPHELES [3-e] did not transmit the disease.
(19) THERE [l-S-i] are four subtypes of the malarial parasite, (20) WHICH [2-d] is known
as plasmodium, (21) EACH [l-L-h] of which has its own characteristic pattern of biological
changes as (22) IT [2-c] passes from its (23) PRIMARY [l-L-f] host, the mosquito, to its

secondary (24) HOST [l-L-g], man. The parasite undergoes sexual reproduction in the
mosquito’s stomach (25) WALLS [3-e]. After a period of days, spores (26) OR [l-S-b] seeds
produced in the walls of the mosquito’s stomach enter (27) ITS [2-c] salivary glands and are
injected, along with saliva, into the bodies of future victims. (28) IT [2-h] is only when this
occurs that the (29) INSECT [3-d] becomes capable of spreading the (30) PARASITE [3-e].
Bachman (1985)
The science of automatic control depends on certain common principles by which an
organism, machine, or system regulates itself. Many historical developments up to the present
day have helped to identify these principles.
For hundreds of years there were (1) MANY [l-L-b] examples of automatic control
systems, but no connections were recognized (2) AMONG [l-L-e] them. A very early ex-
ample was a device on windmills (3) DESIGNED [l-L-h] to keep their sails facing into the
wind. It consisted (4) SIMPLY [4-b] of a miniature windmill which rotated the whole mill to
(5) FACE [3-e] in any direction. The small mill was at right angles (6) TO [l-L-h] the main
one, and whenever the latter faced in the (7) WRONG [2-e] direction, the wind caught the
small mill’s sails and rotated (8) THE [2-a] main mill to the correct position. Other automatic
control mechanisms (9) WERE [l-S-f] invented with the development of steam power: first
the engine (10) GOVERNOR [l-L-f], and then the steering engine controller, which operated
a ship’s (11) RUDDER [l-L-f] in correspondence with the helm. These mechanisms and a
few (12) OTHERS [l-L-f] constituted the achievement of the science of automatic control,
up (13) TO [l-L-h] about 50 years ago. In the past five decades, however, (14) RAPID [4-b]
technological development has created numerous urgent and complex problems. The (15)
SOLUTIONS [l-I-f] to these problems have given birth to new families of (16)
AUTOMATIC [3-e] control devices. For example, chemical plants needed controls for both
(17) TEMPERATURE [4-b] and flow; homes needed control for complex heating and
cooling (18) SYSTEMS [3-e]; radios required control circuits which would guarantee the
accuracy of (19) SIGNALS [2-e].
Historically, then, the modern science of automatic control has been (20) AIDED [3-d] by
related advances in many fields. It now seems surprising (21) TO [l-S-e] recall that the
relationships among these developments were not originally (22) RECOGNIZED [3-e]. Yet
we now know that automatic control and regulating systems (23) DEPEND [l-L-h] on
common principles which are found in both nature and (24) HUMAN [3-d] affairs.
Indeed, studies of modem and old automatic control systems (25) GIVE [l-L-h] us new
insight into a wide variety of natural and (26) HUMAN [3-e] phenomena. The results of these
studies have been very helpful (27) IN [l-L-h] understanding how a person is able to walk
upright, how (28) THE [l-S-a] human heart beats, why our economic system suffers from
slumps (29) AND [l-S-b] booms, and why the rabbit population m parts of Canada (30)
REGULARLY [4-b] fluctuates between scarcity and abundance.
Note. Original source: Tustin (1953), p. 528.
Chavez-Oller et al.-A (1985)

Joe is a freshman, and he (1) IS [l-S-f] having all the problems that most (2) FRESHMEN
[2-f] have. As a matter of fact, his (3) PROBLEMS [3-e] started before he even left home. (4)
HE [3-c] had to do a lot of (5) THINGS [4-b] he didn’t like to do (6) JUST [4-b] because he
was going to go (7) AWAY [l-L-h] to college. He had his eyes (8) EXAMINED [l-L-h] and
he had his cavities filled, (9) ALTHOUGH [2-g] he hates to go to a (10) DENTIST [2-e], and
he got his watch fixed (11) BY [1-S-C] a neighborhood jeweler. Then, at his (12) MOTHER’S
[4-b] suggestion, he had his father’s tailor (13) MEASURE [l-L-f] him for a suit. He didn’t
(14) HAVE [l-L-h] a suit made, though, because his (15) FATHER [3-d] wouldn’t let him
order one. “You’re (16) STILL [4-b] growing, son,” he said. “You’re growing (17) SO [l-L-h]
fast that you’d outgrow a suit (18) IN [l-L-h] no time. Buy yourself a pair (19) OF [l-S-h]
78 TESOL QUARTERLY
slacks and a sports jacket. Klein’s (20) HAS [4-b] such a large selection that I’m (21) SURE [l-
L-h] you will find something you like (22) THERE [2-a].” Joe’s father always suggested
Klein’s for (23) CLOTHES [3-d].
Joe went to Klein’s in order (24) TO [l-S-e] please his father, but he didn’t (25) FIND [3-
e] anything that he liked there so (26) HE [2-c] went to another store to buy (27) THE [3-a]
slacks. He took them out of (28) THE [l-S-a] box as soon as he got (29) HOME [3-d] so that
his father wouldn’t notice (30) WHERE [l-S-d] they came from.
When Joe was (31) ALL [l-L-h] ready to leave for school, his (32) MOTHER [3-e]
suggested that he visit all his (33) RELATIVES [3-e]. “What do you want me to (34) DO [3-
d] that for?” he asked, and she (35) ANSWERED [2-e], “To say good-bye.” She made him
(36) GO [4-b] to see his cousins in Bellevue (37) AND [l-S-b] his Uncle Ned in Plaintown and
(38) HIS [1-L-c] Great-Aunt Lizzie who lives in (39) THE [l-S-a] southern part of the state.
He (40) DIDN’T [l-S-f/h] want to visit all those people, (41) BUT [2-b] he did it anyway
because of (42) HIS [1-L-c] mother’s insistence.
On the day that (43) HE [3-c] left for college, his sister helped (44) HIM [1-L-c] pack his
clothes. She let him (45) BORROW [2-e] her suitcase because he didn’t have (46) ONE [2-c]
of his own. When everything was (47) ALL [l-L-h] ready, he got his father to (48) DRIVE [l-
L-f] him to the station, and the (49) WHOLE [l-L-b] family went along. Of course, his (50)
MOTHER [2-e] insisted on kissing him good-bye in (51) SPITE [l-L-h] of his
embarrassment. As soon as (52) THE [l-S-a] train pulled into the station, Joe (53) JUMPED
[l-L-h] on and hurriedly found his seat. (54) BY [l-L-h] the time it pulled out, he (55) WAS
[l-S-f] already contemplating his new life away (56) FROM [l-L-h] home.
Note. Original source: Praninskas (1959), p. 217.
Chavez-Oller et al-B (1985)

Nicholas Rizos was not a tourist; (1) HE [2-c] was in Athens to (2) WORK [4-b]. He had
arrived from America (3) THE [l-S-a] day before on a Greek cargo (4) SHIP [l-L-h]. During
his last year in (5) HIGH [l-L-h] school, his uncle had invited (6) HIM [1-L-c] to spend a year
in (7) GREECE [3-d] and to help him in (8) HIS [l-L-c] garage. Nicholas accepted the
invitation (9) BECAUSE [2-g] he wanted to become a (10) MECHANIC [3-d]; he thought
that the (11) WORK [3-e] would be good experience for (12) HIM [2-c]. He would also have
an (13) OPPORTUNITY [l-L-h] to learn more about the (14) COUNTRY [3-d] where his
parents were born.
(15) FOR [l-L-e] several months he had studied (16) THE [l-S-a] Greek language at night
school (17) IN [l-L-e] his home town. He wanted (18) TO [l-S-e] speak it as well as (19)
POSSIBLE [l-L-h] and to be able to (20) READ [2-e] signs, at least, when he (21) ARRIVED
[4-b]; but now he wished he (22) COULD [l-S-f] have practiced it more with the (23)
SAILORS [l-L-f] on the ship.
That first (24) MORNING [l-L-f], Nicholas woke up and looked (25) AROUND [l-L-h] at
the unfamiliar room. Everything (26) WAS [l-S-h] strange to him. From his (27) WINDOW
[3-d], he could see the Acropolis (28) AGAINST [l-L-e] the bright blue sky. Then (29) HE [3-
C] remembered; he was in Athens! (30) HOW [l-L-h] happy he was to be (31) THERE [l-S-
g]!
He got dressed quickly and (32) JOINED [4-b] his aunt and uncle in (33) THE [l-S-a]
kitchen. They seemed pleased when (34) HE [3-c] said “Good morning” in their (35) OWN
[4-b] language, but it was difficult (36) FOR [l-S-e] him to continue. They knew (37) VERY
[l-L-i] little English, and at first (36) HE [2-c] was afraid to try his (39) GREEK [3-e]. Before
long, however, they were (40) ALL [1-L-c] talking and laughing together.
After (41) BREAKFAST [3-d], Nicholas and his uncle left (42) TO [l-S-e] go to work. They
walked (43) RATHER [l-L-i] fast because his nucle was (44) IN [l-L-h] a hurry; in fact, he
(45) WAS [l-S-h] later than usual that morning. (46) NICHOLAS [3-e] would have liked to
walk (47) MORE [l-L-i] slowly in order to enjoy (48) THE [l-S-a] unfamiliar sights along the
way.
Note. Original source Wright, Barrett, and Van Syoc (1968), pp. 327-328.

Gamarra and Jonz-A (1987)
Psychologists Robert Rosenthal and Leonore Jacobson (1) PERFORMED [l-L-f] their
initial experiments testing the validity (2) OF [l-S-h] the self-fulfilling prophecy with
laboratory rats. (3) THEY [3-c] found that expectation really influences achievement. (4)
THE [l-S-a] rats believed to be bright did, (5) IN [l-L-h] fact, run their mazes better than (6)
THE [l-S-a] rats labeled dull, even though there (7) WAS [l-S-i] initially no difference in
ability between (8) THE [l-S-a] sets of rats.
The psychologists hypothesized (9) THAT [l-S-e] the self-fulfilling prophecy would
operate in (10) THE [l-S-a] classroom as well as in the (11) LABORATORY [3-e]. They
believed that the teachers' expectations about (12) THEIR [1-L-c] students would affect the
students’ achievement. (13) IT [2-h] was their hypothesis, for instance, that (14) IF [2-g]
teachers were told that certain children (15) WERE [l-S-h] “bright,” the teachers would form
higher (16) EXPECTATIONS [3-e] about them. These higher expectations would be (17)
COMMUNICATED [l-L-h] to the students in various ways (18) AND [2-b] the students
would, in fact, do (19) BETTER [2-e] work. To validate the hypothesis, it (20) WAS [l-S-h]
essential that the teachers’ expectations be (21) FORMED [l-L-h] solely by what they were
told (22) ABOUT [l-L-h] the students.
The psychologists devised an (23) EXPERIMENT [3-e] to test their prediction about the
(24) IMPACT [4-b] of self-fulfilling prophecy. The experiment began (25) IN [l-L-e] 1964 in
a California elementary school. (26) FIRST [3-f], a number of selected children were (27)
GIVEN [l-L-h] a test by the experimenters. The (28) EXPERIMENTERS [3-e] then told the
teachers that certain (29) CHILDREN [3-e] had tested well and could be (30) EXPECTED
[3-e] to make good academic progress. In (31) FACT [l-L-h], these ‘bight” children had not
done (32) BETTER [3-e] on the tests; all the children (33) HAD [l-S-f] done about the same.
Thus higher (34) EXPECTATIONS [3-e] for these children existed only in (35) THEIR [l-L-
C] teachers’ minds.
Later in the year (36) THE [3-a] teachers were asked to describe the (37) PERSONALI-
TIES [4-b] and potential success of their (38) STUDENTS [3-d]. They described the children
who had (39) BEEN [l-S-f] labeled “bright” in very positive ways (40) AND [l-S-b] rated
their chances for future success (41) VERY [l-L-i] good. All the children were tested (42)
FOUR [4-b] different times during the next two (43) YEARS [4-b]. The children who had
been labeled (44) BRIGHT [3-e] and rated highly by their teachers (45) DID [l-S-f], in fact,
show greater intellectual gains (46) THAN [l-L-h] the others. First- and second-grade
children (47) MADE [3-d] the most significant intellectual gains. Perhaps (48) THIS [3-a]
means that younger children are more (49) EASILY [4-b] influenced by teacher expectations
than older (50) ONES [2-c].
Note. Original source Gregg (1985), pp. 112-113.
Gamarra and Jonz-B (1987)

A bird watcher, in the course (1) OF [l-S-h] a day’s rambling, almost always finds (2)
SEVERAL [l-L-b] species of woodpeckers. Sometimes they reveal (3) THEIR [1-L-c]
whereabouts by piercing call notes, sometimes (4) BY [l-L-g] their steady hammering on tree
trunks. (5) WHEN [2-g] the watcher finally catches sight of (6) A [l-S-a] woodpecker, it is
generally spiraling its (7) WAY [l-L-h] up a tree trunk or along (8) A [l-S-a] branch. Every
so often the bird (9) BRACES [4-b] itself and digs for a grub (10) BENEATH [l-L-e] the
bark. When it reaches the (11) END [4-b] of its ascent, it flies off (12) TO [l-L-e] another
tree. In flying, it makes (13) SEVERAL [2-f] quick wing beats, pauses, makes several (14)
MORE [2-e] beats, pauses again and so on. (15) THIS [3-a] results in a characteristic
undulating, or (16) WAVY [l-L-f], flight pattern.
Members of the woodpecker (17) FAMILY [3-d] (Picidae) are found in almost all (18)
PLACES [3-d] where there are trees. They dwell (19) NOT [l-L-h] only in forests and woods
but (20) ALSO [l-L-h] in city parks and rural areas. (21) THE [l-S-a] woodpeckers known
as flickers are often (22) SEEN [3-d] on lawns and in gardens, diligently (23) SEARCHING
[3-d] for ants and grasshoppers. As a (24) RULE [l-L-h], woodpeckers are solitary birds;
family groups (25) BREAK [l-L-h] up after the young are able (26) TO [l-S-e] care for
80 TESOL QUARTERLY
themselves. The Picidae do (27) NOT [l-S-g] migrate great distances as do many (28)
OTHER [l-L-b] kinds of birds. Certain species live (29) THE [l-S-a] year round in the north
temperate (30) AREAS [3-d] of the world. Downy and hairy (31) WOODPECKERS [3-e] are
often seen on the coldest (32) DAYS [l-L-h] of the winter accompanying groups of
nuthatches (33) AND [l-S-b] chickadees in their search for food.
(34) THERE [l-S-i] are something like forty-five different species (35) AND [l-S-b]
subspecies of woodpeckers in North America. (36) MANY [l-L-b] other species of the family
are (37) SCATTERED [3-d] throughout the temperate and tropical parts (38) OF [l-S-h] the
globe. There are no woodpeckers, (39) HOWEVER [3-f], on the island of Madagascar and
(40) THE [l-S-a] continent of Australia.
Woodpeckers are highly (41) SPECIALIZED [4-b] for their tree-climbing and grub-
hunting activities. (42) THEIR [3-c] feet are equipped with (43) SHARP [4-b], curved claws.
Two toes on each (44) FOOT [3-e] are directed forward, while the other (45) TWO [2-f]
point to the rear, thus making (46) AN [l-S-a] effective pincer for grasping the bark (47) OF
[l-S-h] trees. (Three-toed woodpeckers have only one (48) HIND [3-d] toe on each foot.)
The feathers (49) OF [l-S-h] the tail are stiff and end (50) IN [l-L-h] sharp spines.
Note. Original source: Grolier Society (1956), p. 3343.
Jonz (1975)
Nearly three centuries ago, a Spaniard of good character named don Ramiro lived with his
wife in the royal and very noble city of Tunja. This gentleman was a stern man, a very stern
man. He was also a very proud man. According to him, he belonged to one of the most noble
and distinguished families from Spain. In addition, he also spoke in vague terms of a relative
of his who was the Viceroy of New Granada, a very old Columbian name.
Being as stern and proud as he was, don Ramiro was always the target of practical jokes on
the 28th of December. For four years, at 12:05 the morning of the 28th, mischievous boys in
his neighborhood had made it a practice to pound loudly on his door. Then when don Ramiro
opened his bedroom window, the laughing boys would yell:
‘Simpleton!’
The first year that they played (1) THIS [3-a] joke on him, don Ramiro just (2) LAUGHED
[Z-e]; the second year, he gave them (3) A [l-S-a] long lecture about courtesy; the third (4)
YEAR [2-f] he threw a pot full of (5) BEAUTIFUL [l-L-f] and rare flowers at them, and (6)
THE [l-S-a] fourth year he threatened to turn (7) THEM [2-c] over to the police for their (8)
MISBEHAVIOR [l-L-f].
The fifth year, don Ramiro decided (9) TO [l-S-e] put an end to the foolishness (10)
ONCE [l-L-h] and for all. The morning of (11) THE [l-S-a] 27th of December, this fine
Spanish (12) GENTLEMAN [3-e] took a huge gun from (13) ITS [1-L-c] cabinet. Carrying
it as though he (14) WERE [l-S-h] a soldier, don Ramiro marched through (15) THE [l-S-a]
streets of his neighborhood. To all (16) THE [l-S-a] boys he came across, he said:
‘(17) TOMORROW [3-d] I’m going to shoot this gun (18) AT [l-L-e] anyone who comes
to my door.’
(19) THE [3-a] boys, who were good but just a little mischievous, listened to him
courteously (20) WITHOUT [l-L-e] saying a word.
The evening of (21) THE [l-S-a] 27th of December, don Ramiro was (22) TIRED [2-e] and
his voice was hoarse from (23) HAVING [l-S-f] told everyone about his gun.
‘Tonight, (24) AFTER [l-L-e] four years of putting up with (25) DISTURBANCES [3-d],
I’m going to sleep in peace,’ (26) DON [3-e] Ramiro told his wife. ‘All the (27) BOYS [3-e]
should now be very frightened.’
His (28) LOVING [4-b] wife smiled. ‘Nobody is as smart (29) AS [l-L-h] my husband,’ she
said affectionately.
‘Thank (30) YOU [l-L-h], light of my life. Even if (31) IT’S [l-S-i] only nine o’clock by the
cathedral clock, I’m going right to bed.’
And (32) DON [3-e] Ramiro went to the bedroom where (33) HE [2-c] got ready for a
peaceful night.

(34) BUT [3-b] around midnight, don Ramiro heard a (35) LOUD [l-L-f] pounding at the
door: rap! rap! (36) RAP [l-L-g]!
Jumping from his bed, he grabbed (37) HIS [1-L-c] gun and threw open his bedroom (38)
WINDOW [3-d]. By the light of the moon, (39) DON [3-e] Ramiro could see a young man
(40) DRESSED [l-L-h] as a soldier waiting in front (41) OF [l-L-h] his house. Next to the
young (42) MAN [3-.] was a magnificent white horse.
‘Am (43) I [l-L-f] speaking with the distinguished don Ramiro (44) QUESADA [4-b]
Vasquez de la Vega?’ asked the (45) YOUNG [3-e] man in a very courteous, yet (46)
AUTHORITATIVE [l-L-f] tone of voice.
‘Yes, indeed you (47) ARE [l-S-f],’ replied don Ramiro, thinking he must (48) BE [l-S-f]
speaking with at least a Captain, (49) BUT [l-S-b] certainly no less than a Sergeant.
‘(50) A [3-a] relative of yours has the distinguished (51) POSITION [l-L-f] of Viceroy of
New Granada, isn’t (52) THAT [2-a] so?’
Don Ramiro was taken by (53) SURPRISE [l-L-h].
‘Well . well . ...’ he muttered, not knowing (54) WHETHER [l-S-e] to tell the truth or
not.
‘(55) VERY [l-L-h] well, sir, here is a letter (56) FOR [1-S-.] you; without a doubt a very
important (57) LETTER [2-f] at that.’
‘O. K., O. K., wait just (58) A [l-L-h] minute, please!’
Saying this, don Ramiro (59) TOSSED [l-L-f] his gun on the bed and (60) THEN [l-S-b]
changed from his nightshirt to an (61) OUTFIT [4-b] more suitable for the occasion. As (62)
QUICK [l-L-h] as a flash, he ran down (63) THE [l-S-a] stairs and opened the door.
‘Your (64) LETTER [3-.], sir, and please excnse the inconvenience (65) AT [1-L-.] this late
hour.’
Without awaiting a reply, the young man snapped a military salute, ran to his horse,
jumped on, and in the wink of an eye disappeared into the shadows of the night.
‘Who do you suppose wrote me this letter?’ thought don Ramiro. ‘What a huge envelope
and what elegant printing! It might even be from the lawyer in Spain telling me that I’ve
inherited my aunt’s fortune!’
Barely able to control himself, he tore open the envelope, pulled out the letter, a very brief
letter, and read these words:
‘Poor Simpleton!’
Note. Original source: Barlow (1970), pp. 6-12.
Jonz (1987)
I was born in the usual way forty-three years ago and brought up in a way that was not
unusual for persons born at that time. We had morning prayers with a Bible reading every
day. We went to church twice on Sunday. The result of the first is that I was amazed three
weeks ago when in a class I was teaching I found a senior at the University of Chicago who
had never heard of Joshua. The result of the second is that it is very hard for me to go to
church now and that I find myself singing, humming or moaning third-rate hymns like ‘Blest
Be the Tie That Binds’ while shaving, while waiting on the platform to make a speech, or in
other moments of abstraction or crisis.
We had at that time many advantages that have been denied to college students (1) IN [l-
L-e] recent years, but that may be restored to their successors. We had no radios, (2) AND
[l-S-b] for all practical purposes no automobiles, no movies, and no slick-paper magazines.
We had (3) TO [l-L-h] entertain ourselves. We could not by turning a small knob or paying
a small (4) FEE [l-L-f] get somebody else to do it for us. It never occurred to us that (5)
UNLESS [2-g] we could go somewhere or do something our lives were empty. We had
nowhere (6) TO [l-S-e] go and no way to get there. Our recreations were limited to two:
reading (7) AND [l-S-b] physical exercise.
You will notice that the circumstances under which I was brought up (8) GAVE [4-b] me
some knowledge of_one great book, the Bible, and the habit of reading. (9) THE [3-a] habit
of physical exercise I was fortunately forced to abandon at an early date. (10) YOU [3-c] will
notice, too, that the educational system had nothing to do with any of (11) THESE [3-a]
82 TESOL QUARTERLY
accomplishments or habits. I do not remember that I ever thought about being educated (12)
AT [l-L-h] all. I thought of getting through school. This, as I recall it, was a (13) BUSINESS
[4-b] of passing examinations and meeting requirements, all of which were meaningless to me
but (14) PRESUMABLY [4b] had some meaning to those who had me in their power. I have
no (15) DOUBT [l-L-h] that the Latin and Greek I studied did me good. All I can say (16)
IS [l-S-h] that I was not aware of it at the time. Nor did I have (17) ANY [l-L-b] idea of the
particular kind of good it was intended to do me. Since (18) I [3-c] had got the habit of
reading at home, I was perfectly willing to read (19) ANYTHING [l-L-d] anybody gave me.
Apart from a few plays of Shakespeare nobody gave me anything (20) GOOD [4-b] to read
until I was a sophomore in college. Then I was allowed to (21) EXAMINE [3-d] the grammar
and philology of the Apology of Socrates in a Greek course. And (22) SINCE [Z-g] I had had
an unusual amount of German, I was permitted to study Faust.
(23) MY [3-c] father once happened to remark to me that he had never liked mathematics.
Since (24) I [3-c] admired my father very much, it became a point of honor with me not (25)
TO [l-S-e] like mathematics either. I finally squeezed through Solid Geometry. But when, at
the age (26) OF [l-S-h] sixteen, I entered Oberlin College, I found that the authorities felt that
one hard (27) COURSE [3-e] was all that anybody ought to be asked to carry. You could take
either mathematics (2.3) OR [l-S-b] Greek. Of course if you took Greek you were allowed to
drop Latin. I (29) DID [l-S-f] not hesitate a moment. Languages were pie for me. It would
have been unfilial (30) TO [l-S-e] take mathematics. I took Greek and have never seen a
mathematics book since. I (31) HAVE [l-S-f] been permitted to glory in the possession of an
unmathematical mind.
My scientific attainments (32) WERE [l-S-h] of the same order. I had a course in physics
in prep school. (33) EVERY [2-f] Oberlin student had to take one course in science, because
every Oberlin student had (34) TO [l-L-h] take one course in everything—everything, that is,
except Greek and mathematics. After (35) I [3-c] had blown up all the retorts in the chemistry
laboratory doing the Marsh test (36) FOR [l-S-i] arsenic, the chemistry teacher was glad to
give me a passing grade and let (37) ME [1-L-c] go.
My philosophical attainments were such as may be derived from a ten weeks’ (38)
COURSE [3-d] in the History of Philosophy. I do not remember anything about the course
except (39) THAT [l-L-h] the book was green and that it contained pictures of Plato and
Aristotle. I (40) LEARNED [3-d] later that the pictures were wholly imaginary
representations of these writers. I have some (41) REASON [l-L-h] to believe that the
contents of the books bore the same relation to their (42) DOCTRINES [4-b].
So I arrived at the age of eighteen and the end of my sophomore (43) YEAR [l-L-h]. My
formal education had given me no understanding of science, mathematics, or philosophy. It
(44) HAD [l-S-f] added almost nothing to my knowledge of literature. I had some facility
with languages, (45) BUT [2-b] today I cannot read Greek or Latin except by guesswork.
What is perhaps (46) MORE [l-L-i] important, I had no idea what I was doing or why. My
father was (47) A [l-S-a] minister and a professor. The sons of ministers and the sons of
professors were (48) SUPPOSED [l-L-h] to go to college. College was a lot of courses. You
toiled your way (49) THROUGH [l-L-e] those which were required and for the rest
wandered around taking those that seemed (50) MOST [l-L-i] entertaining. The days of the
week and the hours of the day at which courses were offered were perhaps the most
important factor in determining the student’s course of study.
Note. Original source: Hutchins (1941); reprinted in Brown and Gross (Eds.), (1960), pp. 135-
140.

REVIEWS
The TESOL Quarterly welcomes evaluative reviews of publications relevant to
TESOL professionals. In addition to textbooks and reference materials, these
include computer and video software, testing instruments, and other forms of
nonprint materials.
Alien Winds: The Reeducation of

America’s Indochinese Refugees
James W. Tollefson. New York: Praeger, 1989. Pp. xvii+ 204.
American personnel working in the Philippine Refugee

Processing Center (PRPC) no longer live in buildings made of
asbestos: After some protest, virtually all of them have been moved
to asbestos-free concrete dormitories. But most of their Filipino
counterparts and all of the camps’ thousands of Indochinese
refugees continue to live in asbestos housing despite massive
documentation of its hazards, its pervasiveness, and recommenda-
tions for its removal.
This is one of the many astounding facts presented by James Tol-
lefson in Alien Winds: The Reeducation of America’s Indochinese
Refugees. There are more: that refugees can be imprisoned without
trials or hearings; that some staff members use the threat of reporting
poor classroom attendance to extort money, goods, or sexual favors;
that raw sewage runs freely through living quarters and contaminates
drinking water during the rainy season; that inadequate water
supplies lead refugees to collect asbestos-contaminated rainwater for
drinking as it runs off the roofs. But Alien Winds is far more than an
expose of the conditions of camp life It is a critical examination of
the role of the processing centers in socializing refugees for life in
America. As Tollefson shows, these facts are significant because they
are surface manifestations of the pervasive and institutionalized
subordination to which refugees are subjected in American-run
refugee camps. In this book, unique in its deeply probing
examination of the resettlement process, Tollefson contends that
resettlement functions to prepare refugees for assimilation into the
American socioeconomic structure at its lowest echelons.
While the book’s relevance cuts across disciplines, integrating
historical, economic, sociological, and moral perspectives, it is
particularly important for ESL educators. Alien Winds analyzes
the central role that education plays in this process; it peels away,
layer by layer, the historical, institutional, socioeconomic, and
85
ideological contexts in which language acquisition for refugees
occurs. It offers ESL teachers keen insight into their adult students’
lived histories—the realities behind and beyond the personal stories
they share with us.
Sociologist Milton Gordon (1978) describes assimilation as having
two components. The first, which he calls behavioral assimilation,
involves acquisition by immigrant groups of the basic habits,
attitudes, and lifestyle of the “host” culture. The second, which he
calls structural assimilation, involves initiation of immigrant groups
into the organizations, institutional activities, and economic and
civic life of the society. In Alien Winds, Tollefson examines the
interplay of these two aspects of assimilation through American
policy toward refugee resettlement from the mid-1950s to the
present. He argues that even as this policy explicitly promotes an
ideology of behavioral assimilation (namely, that it is necessary to
become Americanized to gain access to the culture and that
becoming Americanized means embracing an ethic of self-reliance,
individualism, and cultural conformity), it implicitly implements an
agenda of structural assimilation which, by creating relationships of
dependency, preparing students for entry-level jobs, and denying
access to the tools of upward mobility, ultimately takes the form of
economic exclusion. Thus, in the name of promoting self-
sufficiency, the reeducation program is structured to destroy the
most basic force for refugees’ survival—the force of community—
for the purpose of meeting structural needs of the economy.
Tollefson’s claim that the education of refugees acts as a tool for
social control may come as a shock to ESL educators concerned
primarily with linguistic, pedagogical, or psychological aspects of
language acquisition; however, it is very much in keeping with
current sociological and historical perspectives on American
education. Sociologists have long claimed that education plays a
central role in the assimilation process, functioning as "a deliberate,
systematic and sustained effort to transmit, evoke or acquire
knowledge, attitudes, values or sensibilities . . ." (Cremin, 1977,
p. 134). Historians have viewed schooling as a central mechanism in
carrying out first, cultural homogenization and then, the economic
integration functions of assimilation (Weiss, 1982). While the
primary focus of educational reform in the 19th century was to
create civic consensus (using the school to combat political, social,
and cultural diversity), with the rise of industrialism, “The
underlying reasons for [educational] reform slowly changed from
an active concern for consensus of values to economic functional-
ism” (Apple & King, 1977, p. 346). In particular, economic
functionalism came to mean the kinds of stratification necessitated
by capitalist development.
86 TESOL QUARTERLY
Thus, educational historians argue that “schools were part of and
reflected the values and concerns of a class-oriented society
dominated and manipulated by a socioeconomic elite. Concerned
primarily with its own self-interest, this elite used the schools to
inculcate values equated with social control rather than social
progress and mobility” (Weiss, 1982, p. xv). Curriculum theorists
call this implicit teaching of social and economic norms,
expectations, and values the “hidden curriculum” (see, for example,
Jackson, 1968; Apple & King, 1977); a growing body of research
documents how this hidden curriculum functions in the classroom
of American public schools to perpetuate social and economic
stratification (Anyon, 1980; Apple & King, 1977; Apple, 1979).
Alien Winds extends this analysis to a new domain, revealing
what has been obscured in the hidden curriculum of refugee
education, rendering the connections among politics, economics,
and curriculum visible and concrete. The author argues that
reeducation encompasses far more than the transmission of
knowledge and skills in the classroom; it entails the promulgation of
attitudes, values, and expectations through its structures,
regulations, administrative organization, curriculum content, and
processes.
Tollefson offers extensive documentation of the underpinnings of
this process; he bases this analysis on his own experience in the
PRPC from 1983 to 1986, as well as on a wide range of primary and
secondary sources. Alien Winds starts with the author’s arrival at the
PRPC and his search for Hung Diep, a seventeen-year-old friend of
a Vietnamese girl from Seattle. Part of the power of this book
derives from Tollefson’s integration of this personal, human
dimension throughout; he reminds us of the real people behind his
analysis by interspersing within the text their stories, words, and
excerpts from their writings.
The first three chapters of Alien Winds situate resettlement in a
historical context. Chapter 1 juxtaposes the experience of individual
refugees with the magnitude of the refugee crisis, focusing on three
refugees’ arrival at the PRPC, as well as on the vast movements of
refugees in Southeast Asia that led to the establishment of the
centers. Tollefson makes the important point that this humanitarian
effort must be seen in light of political exigencies: Even the way that
“refugee” is defined by the U.S. government depends on foreign
and domestic policy considerations as much as on the actual status
of the people involved.
Chapter 2 reviews the historical roots of the refugee crisis in
Southeast Asia since the 1950s. While Tollefson describes what to
well-fed, working Americans may seem unimaginable—the vast
REVIEWS 87
and absolute destruction of any means of self-support for millions
of people—he simultaneously presents documentation of the U.S.
role in contributing to the situation: For example, Tollefson informs
us that the U.S. dropped 50% more tonnage of bombs on Cambodia
than was dropped on Japan in WWII. His argument, therefore, is
that U.S. humanitarian aid must be seen in light of U.S.
responsibility for creating the very conditions that now give rise to
the need for aid.
In Chapter 3, Tollefson shifts this historical analysis to the
American front, looking at what “becoming American” has signified
for immigrants since the turn of the century. He examines the
ideological foundation for today’s refugee programs, arguing that
current approaches to refugee education (e.g., competency-based
education) have their roots in a century-old philosophy whose
cornerstone is the notion that “to live here, you must be like us.”
Tollefson argues that this philosophy, which teaches that hard work
and self-sufficiency will inevitably lead to material well-being and
happiness, in fact serves to undermine immigrants’ cultural
traditions, communities, and independence; at the same time, by
locating the responsibility for “success” with individuals, it provides
an ideological justification for maintaining their economic
subordination. The chapter includes a fascinating, detailed
examination of the texts and teaching methods of the Americaniza-
tion movement, reflecting both the economic and political forces
that shaped it and, as such, providing a political history of today’s
English Only movement.
Chapter 4 examines exactly how the current educational program
in the camps extends this philosophy through the content and
processes of curriculum development and classroom practice. It
looks at how curriculum decisions are made, showing that the
process is both hierarchical and subjective, informed more by
employers’ needs for a productive and docile workforce than
refugees’ needs for survival or economic mobility. Perhaps the most
powerful contribution of this chapter is a section in which
Tollefson, with detailed examples from materials and descriptions
of classroom interaction, carefully unpacks 13 assumptions about
refugee education. He argues, “Taken together, [these] assumptions
suggest that the underlying goal is to replace the refugees’ ties to
culture and community with an isolated [and illusory] struggle for
individual material wealth’ (p. 85).
Chapter 5 examines the institutional context of the reeducation
program, focusing on the relationship between private resettlement
agencies and the U.S. government, particularly in their bureaucratic
structures and educational implications. Tollefson contends that
88 TESOL QUARTERLY
although the administration of the processing centers is nominally
independent, its virtually complete dependence on the State
Department for funding ensures government control. As a result,
policy decisions are made in Washington, far from those most
affected by them, shaped more by domestic political considerations
than by refugee needs or educational factors. At the processing
centers themselves, an elaborate bureaucratic system separates staff
members according to job classification, nationality, and sex,
ensuring hierarchical control: Most Americans, for example, earn
approximately 10 times more than their Filipino counterparts.
Tollefson argues that because of the concern with maintaining
hierarchical control and continued funding, the administration has
done little to involve ESL professionals in program development
and has hired relatively untrained, inexperienced staff, a practice
resulting in significant consequences for the quality of instruction.
Chapter 6 demonstrates that the bases of claims made for the
success of the program may be illusory. Tollefson critiques the
findings of evaluation studies which contend that overseas language
training reduces refugee unemployment and welfare rates, leading
to “successful resettlement.” He analyzes refugee employment
patterns in the U. S., citing evidence that refugee employment is
determined more by prior education than by English proficiency,
and that welfare dependency is more a myth than a reality. The
chapter exposes the mechanisms of slotting refugees into entry-level
jobs (including Office of Refugee Resettlement regulations
prohibiting both language and job skills training when minimum-
wage employment and academic ESL are available). Tollefson
concludes that the training programs are designed more to convince
refugees that they must accept minimum-wage jobs than to ensure
successful resettlement.
Chapter 7 outlines the physical conditions and regulations of
camp life that work to reinforce refugees’ sense of dependency and
powerlessness. Here Tollefson examines how refugees’ daily lives
are controlled through the systems of work credits and sanctions, as
well as through the deprivation resulting from inadequate housing,
food, water, and sanitation facilities. The example of asbestos
housing is documented in detail to illustrate the role of conscious
decision making in maintaining these conditions.
In Chapter 8, Tollefson examines the importance of understand-
ing the role of the processing centers in developing our shared
memory of Vietnam. He argues that the way in which resettlement
is portrayed is essentially a battle for control of history: To continue
to view refugees as victims of communism and as dependent
REVIEWS 89
foreigners being treated in a humanitarian way by a generous
government is to deny U.S. responsibility for creating the
conditions that led to “the refugee problem” and for subsequent
actions that maintain refugee dependency. The alternative,
Tollefson argues, is to recognize refugees’ suffering and contribu-
tions, to hear their stories and, most important, to acknowledge the
role of the U.S. government. The book closes on a positive note,
first by outlining recommendations for change, second by
describing one refugee camp which provides an alternative model,
and third by returning to Hung, who, four years later, has just won
a victory defending himself in traffic court. This final anecdote
serves to remind us of the courage and strength with which refugees
have faced the forces described in this book.
For ESL educators, Alien Winds does not end with the last
chapter. It raises issues that mirror those we face in working with
refugees here, posing questions that we need to confront in our own
policy and practice. Where do our own approaches to curriculum
development fit in this historical context? What is the hidden
curriculum implicit in our methods and materials? For what kinds
of jobs do our classes prepare students and what view of
assimilation do they project? To what extent have we become
unwitting accomplices in a process of producing cheap labor? And,
most important, what can we do about these issues?
Clearly these questions are risky and the answers they elicit may
be even riskier. There is no doubt that Alien Winds will generate
strong reaction: There will be attempts to discredit it precisely
because so much is at stake. It raises questions about some of the
central policy-making professional institutions (like the Center for
Applied Linguistics), the most powerful funding agencies for adult
education in the U.S. (like the Office of Refugee Resettlement), and
the largest ESOL programs in the world.
Thousands of well-intentioned people have been involved in
resettlement programs, some of whom will no doubt rise to the
defense of these programs, charging subjectivity and bias in Alien
Winds. Tollefson does indeed have a strong perspective, but this
perspective is no stronger than that of the policy makers and
curriculum developers he challenges: The difference is that
Tollefson’s point of view is explicit while theirs, as he shows, is
implicit. Ironically, however, it is Tollefson’s attempt to counter
charges of subjectivity that forms one of the weaknesses of the
book: In his effort to provide irrefutable evidence for his claims, he
presents so much documentation that it sometimes proves
cumbersome, interfering with the clarity of the argument. His logic
is also occasionally difficult to follow, for example, when he cites
90 TESOL QUARTERLY
the shift to focusing on homebound skills as evidence for the failure
of refugee education programs. Finally, in some instances,
Tollefson does not distinguish clearly enough between policies and
individuals, an oversight that may result in misunderstanding by
some of the dedicated people who have given years of their lives
working in the camps.
Despite these drawbacks, Alien Winds is a powerful and eye-
opening account of the role of the PRPCS in preparing refugees for
life in the United States. It should be required reading for all ESL
educators.
REFERENCES
Anyon, J. (1980). Social class and the hidden curriculum of work. Journal
of Education, 162, 67-92.
Apple, M. (1979). Ideology and curriculum. London: Routledge & Kegan
Paul.
Apple, M., & King, N. (1977). What do schools teach? Curriculum Inquiry,
6 (4), 341-358.
Cremin, L. (1977). Traditions of American education. New York: Basic
Books.
Gordon, M. (1978). Human nature, class, and ethnicity. New York: Oxford
University Press.
Jackson, D. P. (1968). Life in classrooms. New York: Holt, Rinehart &
Winston.
Weiss, B. (Ed.). (1982). Introduction. American education and the
European immigrant, 1840-1940 (pp. xi-xxviii). Urbana: University of
Illinois Press.
ELSA AUERBACH
University of Massachusetts/Boston
Academic Writing: Techniques and Tasks

Ilona Leki. New York: St. Martin’s Press, 1989. Student’s Book,
pp. xxiv + 327; Instructor’s Manual (in Instructor’s Edition), pp. 32.
Academic Writing: Techniques and Tasks by Ilona Leki is a

writing textbook for the advanced ESL student who is college-
bound. Leki bases the text on the assumption that writing is a
process through which meaning is discovered (Zamel, 1982), and
acknowledges the connection between reading and writing. The
book comprises three interrelated parts: writing processes,
application of writing processes to academic writing, and readings
REVIEWS 91
(Appendix A). The numerous exercises and assignments in the text
are intended to provide the writing teacher with alternatives from
which to select. According to Leki, the material can be well-covered
in only 45 hours of instruction, an estimate which seems low
considering the density and length of the text. What follows is a
short sketch of each of the three parts of the book.
In Part I, Leki takes the learner through each step of the writing
process, i.e., formulating ideas, writing a draft, focusing on the main
ideas, revising, and editing. All phases are amply discussed and
student writing samples are used to exemplify those steps. For
instance, the book introduces three forms of revision: adding text,
deleting or substituting text, and moving text around. Then the
learner sees writing samples (produced by other students) both
before and after revision. By answering questions, such as “What
additions did the student make in the second draft?” (p. 128), the
learner discovers through comparison of the samples how each was
revised. In addition, the author provides a comprehensive question-
guide for revision so that the student writer can go about this
difficult, and oft-neglected, part of the writing process.
The extensive and impressive Chapter 2 of Part I, concerning the
generation of ideas, merits special attention. To spark the creative
development of ideas, the student is introduced to these invention
strategies: freewriting, listing, wh-questions, clusters, looping,
cubing, and outlining. Again, student writing samples exemplify
what is meant by the labels used to identify strategies. For instance,
the learner reads a rationale for a description of “listing,” and is
presented with an example (abbreviated here):
Student Example
SUNDAY NIGHT, MASSEY HALL LOBBY
Noisy: everybody returning from weekends
Crowded
Parents, boyfriends
Floor wet and white, snow
Coke machine noisy
R.A. at front desk bored, answers phone
Two guys playing Pac-Man
Others waiting
One guy on the phone for a long time
Two others waiting to call
Couples sitting in lobby, laugh, talk,
forbidden to go upstairs
92 TESOL QUARTERLY
The learner then sees the final draft of the composition the student
eventually produced based on this list. Questions are deftly posed to
make the learner see the connections between the list and the final
product. Finally, the learner is asked to develop a list for a new
topic: First Impressions of the Community in Which You Now Live
in the United States. This list is filed away for use at a later time as
the basis for a composition.
Besides invention strategies, Leki also presents journal writing as
a means of generating ideas. She presents 10 worthwhile procedural
suggestions with rationales, e.g., “Write your journal on loose-leaf
paper. By using loose-leaf, you can add entries to your journal even
when you do not have your notebook with you” (p. 7). Examples of
student journals are shown, reinforcing Leki’s notion that a journal,
in this sense, is not a blow-by-blow account of a student’s daily
activities. The practice of journal writing does not receive cursory
attention as so often happens in ESL writing books; journal topic
suggestions recur throughout the book to help the writer prepare for
the “Writing Assignment” in each chapter.
Although Part I follows a process approach to writing, it does
retain some of the more traditional elements. These include
discussion of theses and topic sentences, the generation of topic
sentences for model paragraphs, and use of logical connectors as a
means of achieving coherence in writing. (It should be noted that,
although the book presents the use of logical connectors, it does not
offer extensive practice for its advanced ESL audience.) Leki also
presents a schematic representation of a typical academic essay.
Faithful to the process approach, Leki cautions that her schematic
representation “should not be thought of as a formula for good
essays” (Instructor’s Manual, p. 13), and “. . . no rule of English . . .
says a statement of the main idea must appear near the beginning of
a paper” (p. 72),
Part II emphasizes the academic applications of the writing
principles introduced earlier. Whereas the academic essay is dealt
with in general terms in Part I, it is fully treated in many of its
rhetorical forms in this part of the book along with other academic
writing tasks. These forms and tasks include the following:
summarizing, analyzing issues, responding to written arguments,
arguing from written material, and taking essay examinations. In
addition to the rhetorical modes and academic tasks already cited,
the author treats both the paraphrasing of supporting material, and
the documentation of sources using the new style of the Modern
Language Association. One noticeable difference in these chapters
is the use of professional writing samples as the basis for models and
exercises.
REVIEWS 93
Those chapters in Part II that focus on objective writing,
responding to an argument, and writing and arguing from written
material, follow the same procedural format. For example, in
Chapter 13, “Responding to Written Arguments,” the student
initially receives journal suggestions to prepare for the “Writing
Assignment,” and then learns that “academic writing often requires
responding to someone else’s writing” (p. 209). Before reading two
essays that discuss the question of whether or not “gifted” children
should be given preferential treatment in school, the learner
engages in a whole-group prereading discussion of the issue. By
answering questions based on the articles, the learner summarizes
the arguments and analyzes the techniques each writer uses to
defend his or her position. Finally, the student writer responds to
one of the essays by either agreeing or disagreeing. Leki does not
allow the student to dash off an argumentative essay; she sets up the
assignment in such a way that the writer must go through each step
of the writing process, interacting with others at appropriate points
and armed with some organizational advice, i.e., “You might use the
following overall structure for your discussion . . .“ (p. 219).
Recognizing the connection between reading and writing, the
author includes an appendix which contains readings “loosely
linked to the Writing Assignments” (p. 262). In all, there are 12
engaging selections. Each reading is preceded by questions
designed to activate the reader’s prior knowledge of the subject so
as to make the passage more accessible. Following the text, a
variety of questions focus the reader’s attention on main points and
help the reader to draw parallels between the content of the passage
and the reader’s personal experience, knowledge of the world, and
feelings. Also, journal writing tasks are provided that are related to
the selections. In summary, this appendix should enable the student
to summon the ideas necessary to tackle the “Writing Assignments.”
Academic Writing is a welcome addition to the shelf of ESL
writing textbooks for advanced learners. Students should enjoy
working through the writing process with the guidance of student
writing samples, which they will find relevant and which reflect
writing proficiencies of students at this level. The chapters dealing
with invention strategies, revision, and editing deserve special
notice since they are informative and exhaustive, not often the case
in ESL writing textbooks. The sections devoted to academic writing
activities make those tasks less daunting because they are carried
out by following the steps of the writing process, rendered
understandable and accessible to the student in Part I. In short, this
is a first-rate writing textbook for the advanced ESL student.
94 TESOL QUARTERLY
REFERENCE
Zamel, V. (1982). Writing: The process of discovering meaning. TESOL
Quarterly, 16 (2), 195-209.
MARK DUNNING
LaGuardia Community College, New York
The Learner-Centred Curriculum: A Study in Second Lan-

guage Teaching Cambridge Applied Linguistics Series).
David Nunan. Cambridge: Cambridge University Press, 1988.
Pp. xii + 196.
This volume from Australia will make an important contribution

wherever educators struggle to meet ever-changing student needs.
As an example, in the U.S. today thousands of teachers in adult
ESOL classes are scrambling to adjust their curricula to the
demands of the government’s amnesty program, which requires not
only language instruction, but also instruction in U.S. history and
government. The Learner-Centred Curriculum would benefit these
educators as they confront this curricular challenge. The signifi-
cance of this book is not restricted to Australia and the U.S.; it has
a place wherever teachers, students, and program planners try to
develop and improve their curricula.
The author addresses two major questions: What is curriculum?
How can it be learner-centered? Nunan’s answers are based in part
on his synthesis of current research and theorizing, and in part on a
number of studies carried out with Australia’s Adult Migrant
Education Program (AMEP), which includes more than 1,500
ESOL educators and more than 120,000 adult students.
In traditional education, curriculum has meant a set of goals
established by “experts.” These goals are often implemented by sets
of prescribed materials and activities. The role of the teacher, then,
is to carry out the prescribed activities using the prescribed
materials, and to use prescribed tests for evaluation of both students
and teacher. This view of curriculum, like so much else in
education, has come to us from industry, specifically the factory.
When the factory is a model for schooling, students and teachers
have little or no say in the school’s goals, or in the materials they use,
the activities they perform, or the tests by which they are evaluated.
A learner-centered curriculum, on the other hand, derives its
goals, its implementing materials and activities, and its evaluation
from the students and teachers themselves. In this approach to education,
REVIEWS 95
one needs a great deal of information about the learners. Programs
that serve similar populations over a period of time can gather data
on current students to use as the foundation for planning future
curricula. Some of the necessary information is obvious: One needs
to know the students’ ages, previous education, nationalities, em-
ployment situations, and so on. Other information includes the
students’ expressed goals, for example, the following (adapted from
Nunan, p. 62):
I would like to
- talk with my child’s teacher.
- talk with my neighbor.
- read the newspaper.
- fill out forms.
- attend university.
These goals are translated into communicative tasks; the tasks
then are placed in contexts with topics, settings, interlocutors, etc.,
determined by the different goals of different groups of learners.
The tasks in their contexts may be sequenced according to difficulty
(admittedly sometimes an intuitive decision), except when the
students are in an open entry program, where sequencing is almost
impossible; or they may be arranged in order of importance to the
students. One of the main differences between this curriculum and
a functional syllabus is that, in the former, the functions come from
the students.
The book is organized into 10 chapters. In each, Nunan summa-
rizes the most pertinent recent articles and research, and discusses
possible applications. In Chapter 1, the author establishes both the
questions to be addressed and the means of addressing them.
Chapter 2 presents theoretical and philosophical perspectives,
focusing especially on the distinction between product- and
process-oriented approaches. Nunan merges the two here.
Chapter 3 compares content-centered and student-centered ap-
proaches to language curriculum development, referring to theories
of adult learning and of communicative teaching that support the
student-centered approach. This chapter also reports three studies
with AMEP teachers that indicate their attitudes toward planning
curriculum and toward communicative language teaching. This
focus on teacher attitudes reflects Nunan’s emphasis on the central
role of teachers in curricular planning.
In Chapter 4, the author discusses initial planning processes, in-
cluding approaches to student placement. What sort of information
needs to be gathered about the students? How is that information
96 TESOL QUARTERLY
used to place them? In the U. S., many ESOL placements are made
on the basis of English proficiency only (as measured by the
TOEFL or some other instrument), but Nunan lists many other
factors that should be considered. Among these are the following:
(a) the students’ short- and long-term goals in the language; (b) the
students’ previous education; (c) the time the students have been in
the English-speaking country; and (d) the time they intend to
remain. Teachers who have faced a “beginners” class in which some
students were Vietnamese professionals and others were Hmong
farmers, illiterate in their own language, recognize the necessity for
considering something other than oral proficiency as a basis for
placement.
Chapter 5, “Planning Content,” deals with the fundamental issues
of (a) describing general goals in specific terms along with
performance objectives; (b) defining the role of grammar; and (c)
deciding how the language instruction is to be arranged in a graded
sequence. Nunan lists several aspects of the learner, the task, and
the text that should be considered in making these decisions.
Learner factors, for example, include confidence, motivation, prior
learning experience, learning pace, observed ability in language
skills, cultural knowledge or awareness, and linguistic knowledge.
In Chapter 6, Nunan considers methodology from a communica-
tive perspective. This chapter also addresses an important dilemma:
How can we reconcile teachers’ preferences for a communicative
and learner-centered approach if learners prefer a less communica-
tive, more traditional approach? Nunan documents the likelihood
of such conflicts. Drawing from a study (Willing, 1985) in which
over 500 AMEP students ranked different class activities, Nunan
asked 60 AMEP teachers to rank the same activities: Although there
was some agreement, teachers’ and students’ rankings were at
opposite poles with respect to pair work and student self-discovery
of errors.
Chapter 7 discusses resources, including those from authentic and
community sources. Nunan advocates the use of authentic materials
by controlling the difficulty of the task, rather than controlling the
difficulty of the materials. He cautions teachers who want to leave
the classroom in order to use the community as resource that careful
preparation is required, reminding them that many students do not
regard field trips as valid language learning activities.
In Chapters 8 and 9, the author addresses the issue of evaluation,
not only of students, but also of teachers, of activities, and of
program effectiveness. Nunan suggests techniques for student and
teacher self-evaluation.
REVIEWS 97
Chapter 10 looks to the future, considering ways of discovering
and documenting student goals and abilities at different stages, and
suggesting what teachers need in the way of training and support if
they are to be responsible for developing the curriculum. Nunan
underscores the significance of the task at hand:
The most urgent need is for the profession to adopt a more rigorous
approach to the planning, implementation and evaluation of the
curriculum. . . . [and] to develop a more rigorously-formulated and
empirically-based approach to language proficiency. (p. 175)
The Learner-Centred Curriculum offers an important contribution
to this end.
REFERENCE
Willing, K. (1985). Learning styles in adult migrant education. Sydney:
New South Wales Adult Migrant Education Service.
PAT RIGG
American Language and Literacy
98 TESOL QUARTERLY
BRIEF REPORTS AND SUMMARIES
The TESOL Quarterly invites readers to submit short reports and updates on their
work. These summaries may address any areas of interest to Quarterly readers.
Authors’ addresses are printed with these reports to enable interested readers to
contact the authors for more details.
Edited by GAIL WEINSTEIN-SHR

Temple University
Writing-Proficiency Tests and Remediation:

Some Cultural Differences
ROGER M. THOMPSON
University of Florida
Increasingly, universities are establishing writing-proficiency standards

for their students, be they native or nonnative speakers of English. The
state board of education in the state of Florida, for example, requires that
all students, no matter their national origin, pass the same holistically
graded essay exam before they can receive an associate degree from a
public community college or be accepted as a junior in a state-supported
university. In addition, all students must fulfill what is known as the
Gordon Rule by completing 12 hours of courses with a heavy emphasis on
composition before they have completed 60 undergraduate hours. These
requirements also apply to students transferring from out-of-state
universities and from private schools.
How do international students cope with the requirement that they
compete with native speakers of English in writing? In order to find out,
the academic records of international students accepted into the lower
division of the University of Florida from August 1984 to August 1987 were
examined. The results show some interesting differences among
nationalities.
METHOD
For the period under consideration, the undergraduate international
students at the University of Florida came mainly from Latin America, the
Middle East, and Southeast Asia. The academic records of 79 students with
the following distribution were examined: Latin American students from
Colombia, Honduras, Panama, and Venezuela; Middle Eastern students
from Iran, Lebanon, Syria, Oman, Kuwait, and Jordan; Southeast Asian
students from Indonesia, Malaysia, Taiwan, Thailand, Singapore, and
Hong Kong.
99
For each student, several questions were investigated. What were their
scores on the state-mandated essay exam and how many times did they
take the exam before passing? What were their grade point averages after
two years of undergraduate work? Did they take a composition course
such as Expository and Argumentative Writing, Writing about Literature,
Writing for International Students, or Technical Writing before they took
the essay exam? (They are not required to do so.) Or did they fulfill the
Gordon Rule requirement with other popular, approved courses that may
require extensive writing but do not focus on developing writing skills,
e.g., Introduction to Music Listening, Human Sexuality, or Survey of
Architectural History? What were their average grades in courses that
fulfill the Gordon Rule?
RESULTS
Of the 79 students who entered the university during this period, 11 were
suspended from the university before they took the required exam. Table
1 shows the results for the 68 who wrote the required essay. The essay
scores are aligned with general grade point averages after two years of
study, the average grade in Gordon Rule courses, and the percentage who
enrolled in at least one writing course that focused on composition skills
before they wrote the required essay.
The statewide proficiency exam is graded holistically by two
experienced English teachers who read the essay rapidly and then rank it
on a scale of 1 to 4 with 1 being the lowest score. The essay passes if the
essay is rated at 2 or above by both readers. The ratings for these two
readings are then combined for a final score on a scale of 2 to 8. Since a
combined score of 4 passes, papers with a borderline score of 3 are read by
a third rater to determine a final score of 2 or 4; no one can receive 3 as a
final score.
DISCUSSION
On average, nearly one-third of the international students failed the
required essay on the first attempt. However, nearly half of the students
from Southeast Asia failed the first attempt. Students who failed shared
certain characteristics no matter where they were from. Generally they
had a low grade point average between C and C+ for all academic work
attempted and a C average for Gordon Rule courses. Usually they had not
taken a course that focused on writing instruction before they took the
exam. If they had taken a writing course, it was not the special composition
course for nonnative speakers of English that is offered by the University
of Florida.
Two other interesting features characterize those who failed. First,
nearly two-thirds were transfer students from out-of-state colleges and
universities or from private colleges in Florida. In fact half of the transfer
students failed the composition section on the first attempt. This suggests
100 TESOL QUARTERLY

TABLE 1
International Students: Essay Scores and Academic Performance
that the Florida practice of requiring all undergraduates, whether native or

nonnative speakers of English, to complete 12 hours of courses requiring
extensive writing is beneficial. Private colleges and other state university
systems do not have such requirements, potentially rendering international
students second-class citizens with relaxed requirements that do not
emphasize the writing of lengthy essays. Because everyone must pass the
same holistically graded exam in Florida, international students cannot be
ignored in writing classes. Presumably, as a result, their written English is
better.
The other interesting characteristic concerns how those who failed
prepare to retake the exam. As a rule, only one-third seek remedial
instruction in composition. However, there are regional differences.
Although the students from Southeast Asia had the highest initial failure
rate, they were the most likely to enroll in a composition course be-
fore the retake. More than 70% enrolled in such a course, most often,
BRIEF REPORTS AND SUMMARIES 101

composition for international students. Students from the Middle East
were the least likely, with only one student enrolling in a composition
course. Perhaps as a result, the students from Southeast Asia were the most
likely to pass the test with only one retake. Students from the Middle East
who had failed initially averaged two or three retakes. For all nationalities,
only two students who failed the exam the first time have not been able to
pass the exam after several attempts.
In summary, the research indicates cultural patterns that require further
investigation. First, the university does not seem to be meeting the
academic needs of Latin American students. These students seem to be less
academically prepared than their international counterparts. Of the 11
students who were suspended for academic reasons before they were
juniors, seven were from Latin America; they alone were suspended
because of poor grades. The three students from the Middle East and the
one from Southeast Asia who were suspended had either failed to fulfill
the Gordon Rule or had not sat for the required essay exam. When the
Latin American students who were suspended for academic reasons are
grouped with those who failed the essay exam on the first try, this group
comprises nearly half of the undergraduate students admitted to the
university from these Latin American countries.
Another pattern seems to be the academic success of students who seek
remediation. Students who sought writing instruction after failing the
exam for the first time were generally able to pass the exam on the next
attempt. This suggests that students who did not adopt this strategy
(perhaps because it was culturally unfamiliar) might well benefit from
explicit discussion of how to seek available academic help after initial
failure.
Finally, if we assume that holistic essays test writing skills fundamental
to university study, the success of Florida-educated students is suggestive.
TESOL programs that seek to prepare students to study in North
American universities should emphasize instruction in writing full-length
essays.
Author’s Address: Department of English, University of Florida, Gainesville, FL

32611
102 TESOL QUARTERLY

THE FORUM
The TESOL Quarterly invites commentary on current trends or practices in the
TESOL profession. It also welcomes responses or rebuttals to any articles or
remarks published here in The Forum or elsewhere in the Quarterly.
Comments on Bronwyn Norton Peirce’s

“Toward a Pedagogy of Possibility in the
Teaching of English Internationally:
People’s English in South Africa”
BETTY LOU DUBOIS

New Mexico State University
No matter how you slice it, no matter how you trap it (“pedagogy
of possibility,” “ poststructuralist theory of language”), Bronwyn
Peirce’s article in a recent TESOL Quarterly (Vol. 23, No. 3,
September 1989) still retails the same old linguistic determinism, the
same old relativism, the same old strong form, to use the Fishman
(1960) systematization, of the Sapir-Whorf hypothesis (in Carroll,
1956). In Peirce’s own words, “If we teach English in a way that
promotes a student’s uncritical integration into society, students will
lack the tools to question the predetermined roles established for
them by that society” (p. 407). If current nonpossibilist English
pedagogues indeed prevent the questioning of social roles, how to
account for the Chinese students interviewed on the spot for U. S.
media who spoke out—in English—against conditions which led to
the massacre in Tiananmen Square or those who spoke out against
them in the United States? How to account for black South Africans
who daily speak out and act out against a brutal, repressive regime?
If thought is indeed constrained by language teaching, how to
account for the current uprising all over Communist Europe? If we
teach students to say, “I want to be happy,” what keeps them from
saying, “I want to be free”? My own experience, admittedly limited,
has been that students bring ideas to the language classroom and
seek ways to express them, not that my teaching determines their
ideas.
103
If irony is defined as unintended incongruence, then the
placement of Peirce’s work in the same issue of the Quarterly as
Davies’ article, “Is International English an Interlanguage?” is a
masterpiece of irony, unless former editor Stephen Gaies
deliberately juxtaposed the two to stimulate discussion. Davies
provides the antidote to Peirce in his discussion of relativism (the
equation of language and thought), which is, as he notes, an idea
“both seductive and solipsist” (p. 451), but which must be rejected
by common sense—and, I might add, by a lack of objective
supporting evidence, no matter how hard its exponents have tried to
test this extremely appealing claim.
Finally, there is the question of fostering communicative
competence in a social order perceived to be unjust. Perhaps it does
happen that former or current ESL materials in South Africa teach
blacks to lower their gaze to say “Baas,” although I doubt it. No ESL
teacher of my acquaintance—again, limited—would accede to that
outrage. A more serious question here is our role in such a situation.
To believe that the impetus for social change should come from
outside the system seems to me merely a well-intentioned
neocolonialism, another expression of the white man’s burden.
I assume that Peirce, Davies, and I, in common with all other
readers of the TESOL Quarterly, are united in a desire for
improvement of the situation of the World’s oppressed. Where
Davies, it seems, and I part company with Peirce is over the latter’s
belief that current second language pedagogy can deter English
learners from assessing the political conditions under which they
live.
REFERENCES
Carroll, J. B. (Ed.). (1956). Language, thought, and reality: Selected
writings of Benjamin Lee Whorf. Cambridge: M.I.T. Press.
Davies, A. (1989). Is international English an interlanguage? TESOL
Quarterly, 23 (3), 447-467.
Fishman, J. (1960). A systematization of the Whorfian hypothesis.
Behavioral Sciences, 5, 323-339.
104 TESOL QUARTERLY

The Author Responds. . .
BRONWYN NORTON PEIRCE

Ontario Institute for Studies in Education
Dubois’ comments on my article are both incisive and

provocative. If, as Dubois asserts, a poststructuralist theory of
language is no more than the “same old strong form” of the Sapir-
Whorf hypothesis, it should be dismissed as irrelevant. Equally, if a
pedagogy of possibility is nothing more than “well-intentioned
neocolonialism,” it should be dismissed as outmoded. It is my
contention, however, that neither of these equivalence relations
apply. In order to support my argument, it is necessary for me (a)
to give a more comprehensive account of a poststructuralist theory
of language and its relevance to TESOL and (b) to demonstrate that
a pedagogy of possibility is not an atavistic theory but has
contemporary relevance for the TESOL profession internationally.
Since any discussion of educational linguistic theory should address
the issues that teachers have to confront in the daily practice of
teaching, I will develop my argument with reference to a recent
debate in one of my English as a Second Language (ESL)
classrooms in Toronto, Canada.
In the wake of the December 1989 killing of 14 female
engineering students by a young male in Montreal, Canada, my
students were struggling to make sense of the shocking event. It
soon became apparent that the ways in which different students
attempted to interpret and, more important, name the event created
conflict in the class. Some students understood the event as an act of
hatred against women, and named it an antifeminist massacre;
others understood the event as symptomatic of generic militance in
our society, and named it an act of violence; still others understood
the event as the isolated act of an insane individual, and named it an
act of madness. As the debate continued, two questions came
immediately to mind: First, why was the act of naming such an
important one for my students? Second, how was I, as the teacher,
to respond appropriately to the ideas that my students had brought
to class? In addressing these two questions—questions that arise
directly from classroom practice—I can begin to respond to Dubois’
questions about the nature of language and the role of the language
teacher in the classroom.
The first issue I wish to address is why the naming of the Montreal
killings created conflict in the class. For my students, the act of
naming was what Pennycook (1989) calls “interested.” By this, I
THE FORUM 105

mean that each group of students had different interests and
investments at stake when they struggled for the ascendancy of
their respective positions. If the view was legitimated that the
massacre was an antifeminist act, the Montreal killings would be
constituted within the discourse of gender relations. Students might
then be expected to address the role of women in society and what
courses of action might be appropriate in dealing with their
oppression.
If the view was legitimated that the massacre was an act of
militancy, the killings would be constituted within the discourse of
social violence. Students might then be expected to address the
causes of violence in society, the role of the media in generating
violence, and appropriate measures to control violence.
If the view was legitimated that the massacre was an act of
insanity, the killings would be constituted within the discourse of
mental health. Students might then be expected to address the way
society should deal with mental illness, and what rights mentally ill
people can legitimately claim. The struggle to name the Montreal
killings was not simply an attempt to express a variety of
viewpoints, all of which had equal claim to validity. It was a
struggle by different groups to situate the event within a framework
that would legitimate some views and invalidate others. It is best
understood with reference to the location of the students in
particular discourses with conflicting claims to truth and power.
Thus the struggle to name the Montreal killings was a political act—
not only in terms of the way the students understood the event, but
also in terms of the way such a naming might influence private and
public responses to the event. And such a naming is constituted in
language and by language.
As a language teacher, I need to account theoretically for my
students’ conflicting attempts to name the Montreal killings. Do I
draw the conclusion that language determines thought, and validate
the Whorfian hypothesis? Certainly, my classroom experience
indicates that there is a complex relationship between language and
thought, but Whorfianism does not take into account the notion of
human agency—the attempt by individuals to challenge and change
the signifying practices in their societies. Nor does Whorfianism
account for the struggles that exist within one language group to
appropriate the meaning that is to be assigned to particular signs
and symbols in their language. Whorfianism is too deterministic and
simplistic to account for the struggle in my classroom. In order to
appreciate how poststructuralism can convincingly account for this
struggle, it is useful to examine other structuralist theories of
language, since poststructuralism draws on and extends these
theories.
106 TESOL QUARTERLY

Saussure (1959), the founder of structuralism, argues that
language is an arbitrary system of differences in which elements
gain their meanings only from their relation to all other elements,
For structuralists, the building blocks of language structure are signs
that comprise the signifier (or sound-image) and the signified (the
concept or meaning). Saussure argues that neither the signifier nor
signified preexists the other and that the link between them is
arbitrary. For structuralists, it is the linguistic system itself that
guarantees the meaning of signs, and Saussure asserts that each
linguistic community has its own set of signifying practices that give
value to the signs in the language. Thus a structuralist conception of
language would enable my students to understand the subtle
difference between the concept of a “massacre” as opposed to the
concepts of “murder” or “genocide.” Significantly, however, it
would not account for the conflicting struggle over the meaning that
could be attributed to the sign /massacre/ with reference to the
particular space/time location of the Montreal killings. Poststructur-
alism, in contrast, has great explanatory power. Consider the words
of Terdiman (1985):
A sign is invested with the competing meanings of social groups and

refuses to remain static as an immediate consequence of such conflict
inscribed in its use. Conflict is thus as characteristic of the semiotic realm
as of the social. . . . Engaged with the realities of power, human
communities use words not in contemplation but in competition. Such
struggles are never equal ones. The facts of domination, of control, are
inscribed in the signs available for use by all members of a social
formation. (pp. 36-38)
The massacre took on a different meaning for different students,
and this meaning arose not only from the tragic events of December
6, 1989, but also from the way individual students took up positions
with respect to the event. In other words, the meaning of the
massacre was not only a product of a linguistic system that
differentiates signs in relation to one another, albeit in a social
context, but is located in the intersection between the event, the
sign, and an individual’s investment in the event and the sign. For
structuralists, the sign /massacre/ may have, in a sense, an idealized
meaning, but this does not account for the way it is taken up in
particular struggles for power and control. In poststructuralist
terms, students took up different subject positions in relation to the
massacre, and their struggle to make sense of the killing was
reflected in their respective attempts to name the killings—to
ascribe meaning to the sign /massacre/ within the social context of
the Montreal killings. There was a struggle amongst students to
THE FORUM 107

define the terms within which the Montreal killings could be
understood—a struggle that a deterministic theory could never
account for.
Thus poststructuralist theory is not equivalent to the old strong
form of the Sapir-Whorf hypothesis. Like Whorf (in Carroll, 1956)–
and Saussure—poststructuralists recognize that the signifying
practices that are available in a society influence the way a society
makes sense of the world. Few linguistic theorists would dispute this
claim. However, poststructuralists argue that every society has
different investments in these signifying practices and that they are
frequently inscribed in unequal relations of power. Unlike both
Whorf and Saussure, poststructuralists argue that linguistic communi-
ties are not homogeneous, consensual spaces in which language use is
predictable and conventional. For poststructuralists, linguistic com-
munities are perceived to be heterogeneous arenas in which language
is implicated in the struggle over meaning, access, and power.
It was for this reason that I was drawn to poststructuralism in my
attempt to account for the struggle for People’s English in South
Africa. To interpret People’s English as a dialect of international
English would do the movement a gross injustice; People’s English
is not only a language, it is a struggle to appropriate English in the
interests of democracy in South Africa. Thus the naming of People’s
English is a political act because it represents a challenge to the
current status of English in South Africa, in which control of the
language, access to the language, and teaching of the language are
entrenched within apartheid structures. I believe Davies himself
(referenced by Dubois) hints at the sociopolitical nature of language
when he states:
[English as an international language] is the story of a group or groups
of people forming themselves through interaction into a part-time
speech community because of their need to negotiate meanings across
political frontiers. Those meanings are public ones, but the very nature
of such interaction, its inertia as it were, is for the public meanings to
extend into the private and personal because that is what language use
brings about. Then that causes conflict in terms of language identity.
(p. 464)
To summarize, poststructuralism represents an attempt to
account for struggles over the meaning to be invested in signifying
practices in a society—of which language is the quintessential
example. It is not equivalent to the same old strong form of the
Sapir-Whorf hypothesis. It is relevant to teachers of English because
it provides new insights into the nature of language, and because it
helps to account precisely for the actions referenced by Dubois: the
Chinese students who spoke out in English against the Tiananmen
108 TESOL QUARTERLY

massacre; black South Africans who speak out daily against
apartheid. Poststructuralist theory, however, accounts for even
more than this. It accounts for the students who spoke out in
Mandarin against the Tiananmen massacre; it accounts for white
people who speak out daily against apartheid. The struggle to
challenge and change the dominant signifying practices in a society
is not only a linguistic struggle; it is a profoundly political one.
I now wish to turn to the second dilemma that confronted me when
my students debated the Montreal killings: How do I, as a teacher,
respond to the ideas that my students have brought to class? At what
point does current theory of communicative competence prove
inadequate, and what is the appeal of a pedagogy of possibility? Like
Dubois, I was trained to believe that my role as a teacher is to help
students express the ideas that they bring to the language classroom and
not to determine the way my students should think. In practice,
however, the contention that teaching can be “neutral” is problematic. If
one of my students had said to me—with respect to the Montreal
killings—that “women’s rights have gone too far,” any response would
have had political overtones. Ignoring the comment, deflecting the
comment, agreeing with the comment, opposing the comment, would
have been political in that each would have served to either affirm or
negate different views in the classroom. The issue for me is not whether
our role is a political one or not, but how we theorize this political role.
Thus the question that I raised in my paper with regard to current
English pedagogy was theoretically rather than empirically
motivated: Can we teach students to be communicatively
competent without asking ourselves what communicative compe-
tence means in a particular time and place? In other words:
Communicative competence for what? My position is that
communicative competence should not be idealized and removed
from the social context in which English is taught. Teachers of
English are trained that we should teach our students to use the
English language in “appropriate” ways, but what does it mean to
speak English appropriately in South Africa, China, or North
America? What role should English teachers play if appropriate
language perpetuates unequal relations of power between people in
a particular society? Clearly, no English teacher would teach
students to lower their eyes and say “Baas” in South Africa. But if
those students referred to an elderly black man as a “boy,” should
the teacher challenge the students to reflect on this statement? If the
teacher challenged the students, would she or he be accused of
attempting to “determine” the students’ ideas? Access to the
linguistic “rules of use” in South African society would not resolve
THE FORUM 109

this issue for the language teacher. What is required is a theory of
pedagogical practice—a pedagogy of possibility (Simon, 1987).
Like Simon, I support the position that a desirable pedagogical
goal is to expand possibilities for students, with a view to creating a
more just and compassionate society. Tolerance of racism, sexism,
and elitism, for example, are incompatible with such a project. But
a pedagogy of possibility does not dictate how teachers should
exercise their authority; it does not prescribe “politically correct”
rules of speech and behavior. If a student refers to an elderly man
as a “boy,” it would be problematic to simply correct the reference
by pointing out that such usage is inappropriate given the man’s age.
Rather, the teacher might take the opportunity to explore the
grounds on which such a naming is constituted, and whose interests
such a naming serves. The discussion might open up possibilities for
exploring how the form “boy” within this particular context
implicates the English speaker in social relations of power. If we as
English teachers wish to help our students to gain control over the
language that we teach, we need to alert students to the current
terrains of struggle that characterize the language and into which
the students enter as they learn the language. This is the agenda of
a pedagogy of possibility in the teaching of English internationally.
This is not the same as saying that “current second language
pedagogy can deter English learners from assessing the political
conditions under which they live.” A pedagogy of possibility seeks
to raise awareness about the way language is implicated in the
legitimation of and challenge to these very political conditions. Such
an exploration will enhance students’ understanding of the nature of
language and its signifying power in the society. It is a pedagogy
that opens up exciting possibilities for both teachers and learners of
English.
To summarize, a pedagogy of possibility is not well-intentioned
neocolonialism. It is a pedagogical theory that seeks to establish a
principled basis for the exercise of a teacher’s authority—whether
the teacher is located in North America, Britain, or China. (Montreal
and Toronto, for example, are not isolated corners of the neo-
colonialist world.) It is also a theory that raises awareness about the
force of language as a signifying practice in society. It is relevant to
teachers of English internationally because we are implicated in the
continual growth and influence of the English language—a
language that is caught up in many competing and conflicting
discourses. It is important, therefore, that we scrutinize the theories
that inform our teaching. As Terdiman argues, signs are invested
with competing meanings and refuse to remain static within the
burly-burly of social discourse. My position represents a challenge
110 TESOL QUARTERLY

to the meaning that is to be invested in the sign /communicative
competence/ within the context of the teaching of English
internationally.
I would like to conclude by drawing the reader’s attention to the
nature of the debate that has arisen between Dubois and me
because it exemplifies par excellence the political nature of
language and the power of poststructuralist theory to account for it.
When I wrote my article, my intention as the author was twofold:
first, to explore the adequacy of a theory of communicative
competence, given its application at a particular place and time;
second, to account for a dynamic pedagogical movement in South
Africa. What I said is that language is not neutral, and that People’s
English illustrates how the teaching of English can be undertaken as
a pedagogy that expands possibilities for students. What Dubois
understood was that my conception of language (poststructuralist)
was “Whorfian,” and that a pedagogy of possibility was “well-
intentioned neocolonialism. ” Where does meaning “really” lie? Can
meaning be located in what I as the speaker meant, in the words
that I used, or in Dubois’ interpretation? Poststructuralists would
argue that meaning shifts from one context to another at different
times and in different places. Meaning is not “owned” by the
speaker/writer, by the linguistic system, or by the hearer/reader; it
is a product of speaker, sign, and hearer, all of which are enmeshed
in time, place, and society. This is not necessarily a consensual
space: Witness Dubois’ attempts to dismiss poststructuralism as
“Whorfian” and a pedagogy of possibility as “neocolonialist.”
Poststructuralism and a pedagogy of possibility represent
challenges to Dubois’ conception of language and the role of the
English language teacher internationally. Dubois has chosen to
undermine these theories by renaming them “Whorfian” and
“neocolonialist” respectively. Such politically charged language
(convincingly accounted for by poststructuralist theory) is intended
to promote the view that poststructuralism and a pedagogy of
possibility are irrelevant and outmoded. This is irony indeed.
REFERENCES
Carroll, J. B. (Ed.). (1956). Language, thought, and reality: Selected
writings of Benjamin Lee Whorf. Cambridge: M.I.T. Press.
THE FORUM 111

Davies, A. (1989). Is international English an interlanguage? TESOL
Quarterly, 23 (3), 447-467.
Pennycook, A. (1989). The concept of method, interested knowledge, and
the politics of language teaching. TESOL Quarterly, 23 (4), 589-618.
Saussure, F. de (1959). Course in general linguistics (W. Baskin, Trans.).
New York: McGraw-Hill.
Simon, R. (1987). Empowerment as a pedagogy of possibility. Language
Arts, 64, 370-383.
Terdiman; R. (1985). Discourse/counter discourse. New York: Cornell
University Press.
Comments on Alan Beretta’s

“Attention to Form or Meaning?
Error Treatment in the Bangalore Project”
A Reader Reacts. . .
N. S. Prabhu
National University of Singapore
In his recent article in the TESOL Quarterly (Vol. 23, No. 2, June
1989), Alan Beretta presents an interesting investigation of a
distinction I tried to make between incidental correction a n d
systematic correction of learners’ errors. In the process, however, he
makes certain statements which seem to me to have misleading
implications. I therefore offer the following comments.
Beretta states (pp. 288-289), “It should be made clear that the 21
lessons analyzed in this investigation are simply those that were
recorded and made available to the author.” He points out that “of
the 21 lessons, 18 were taught by the Director and a close associate,”
that the data “did not include lessons given by regular teachers,”
and that “some of the recordings were shorter than others or
incomplete’’—points that he repeats elsewhere in the paper. This
carries the implication that the project developers (myself, in
particular) offered for Beretta’s study data that were unrepresenta-
tive and possibly biased.
The fact is that the 21 lessons (or parts thereof) were all recorded
by different people for different purposes, and none of the
recording was intended for use in Beretta’s study. Five of those
lessons (Q to U in Beretta’s Table 1) were not even a part of the
project in India: They were taught in Sri Lanka, as an illustration of
112 TESOL QUARTERLY

task-based teaching, by Teacher X and myself who visited that
country to conduct some teachers’ seminars. About 12 other lessons
(A to J, and, I think, L and M as well, in Beretta’s Table 1) were
recorded (and selected for recording) by D. J. Carroll, largely for
use in his own Ph.D. study. The remaining four were recordings
made by Teacher X and myself, of our own teaching, for our own
use. When, about a year after the conclusion of teaching on the
project, Beretta asked about the availability of recorded lessons for
his own use, I pointed out to him the existence of these various
recordings.
This is not of course to suggest that Beretta should not have used
these data for his study of error treatment, or that he should not
have drawn attention to their possible unrepresentative nature. I do
think, however, that he might have pointed out that the data were
an ad hoc collection of recordings that became available by chance,
instead of stating that they were “recorded and made available” to
him.
More unfortunately, the lessons recorded by Carroll (about half
of Beretta’s data) happen to be unrepresentative in a way not ac-
knowledged (or not realized) by Beretta. As I have pointed out
elsewhere (Prabhu, 1987), students in Indian secondary schools
must take public examinations at certain stages; these examinations
demand summaries of set texts, letter writing, and some focused
work on grammar and vocabulary. Consequently, whenever a class
that was part of the project came close to taking a public
examination, the project team had to decide whether to modify its
teaching in light of the approaching examination (thus deviating
from the project’s principles) or to stop project teaching for that
class (thus sending it back to traditional teaching). With one of the
classes (at the Malleswaram school; see Prabhu, 1987), the decision
was to continue but modify the teaching; with the others, the
decision was to stop project teaching. The class to which Carroll
had access was the one that received such modified teaching:
Set texts were summarized orally by the teacher in the form of
lecturettes; the writing of summaries and letters was practised;
some sentence-manipulation exercises were worked on; and, as
Beretta notes, students were asked to focus on and correct lan-
guage errors in samples of their own writing. This is why the
lessons recorded by Carroll show so much work based on the
lecturette and letter writing. Such work was not discussed in the
Regional Institute of English (RIE) newsletters on the project, nor
was it included in the task types listed in Prabhu (1987), simply
because this deviation from the project’s principles was deliber-
ate—it allowed the class to meet the demands of a public
THE FORUM 113

examination. The fact that there was a deviation for that purpose
has, however, been well acknowledged, as in Prabhu (1987).
In view of these facts, Beretta’s conclusions that “tasks such as
lecturettes and letter writing were dispensed with” in order to
“protect learners from the risk of linguistic error attendant upon
production” (p. 299), and that this was “an unacknowledged move
to eliminate the possibility of a focus on form” (p. 301), are ill-
founded. Project teaching consistently concentrated on comprehen-
sion rather than production as being of value to acquisition, and this
was fully acknowledged from the early days, as Beretta himself
notes. The lecturette and letter writing did not characterize project
teaching as such in the early years, but only examination-oriented
teaching with one particular class at one particular time. Beretta was
aware that project teaching had been modified for one class in
Bangalore to suit the public examination, as can be seen from
Beretta (1984): “In the Bangalore school, 3 years of structural
teaching were followed by 2 years of CTP [Communicational
Teaching Project], then 1 year of modified CTP before the
examination” (p. 19). He could also have seen from the RIE
newsletters that the same Malleswaram class that Carroll recorded
in 1981 had been receiving the more normal project teaching (based
on maps and timetables) earlier: The RIE newsletter series (Vol. 2,
No. 2, October 1980) reports on the teaching done with that class
from June to September, 1980. Perhaps Beretta might have paused
and asked himself why the teaching evidenced in Carroll’s
recordings was so unusual, before basing a large generalization on
it.
Incidentally, I do not think that a reduction of the scope for
learners’ errors is anything peculiar to the project’s teaching. A
reduction (indeed, a total prevention, if possible) of learners’ errors
has been an important principle of teaching based on structural
practice, and it marks a good deal of what is called functional or
communicative practice as well. While it is achieved, in these latter
forms of teaching, by substituting reproduction by learners (in one
form or another) for genuine production, the Bangalore project’s
teaching saw value in comprehension rather than in reproduction.
In any case, Beretta’s analysis shows a clear difference between
the more comprehension-based lessons and the more production-
based ones only in respect to the incidence of errors, not in respect
to their treatment. The treatment of errors for all of the lessons
showed that there was more correction for content than for
language, and a wide difference in the modes of treatment
employed with the two kinds of error, leading Beretta to remark
that “there is obviously a massive difference in generaI approach”
114 TESOL QUARTERLY

(p. 296). It does, that is to say, make sense to talk of incidental
correction, in contrast to systematic correction, whether teaching is
as comprehension-based as in the tasks on timetables and maps, or
more production-based as with lecturettes and letter writing. I
therefore do not understand the logic of Beretta’s concluding
sentence:
An important message of this study is that teachers who are involved in
content-based curricula and who are interested in changing correction
habits might wish to consider whether they are prepared to engage only
in tasks that inhibit learner production in order to ensure that form does
not become a focus. (p. 301)
REFERENCES
Beretta, A. (1984). Evaluation of the Bangalore/Madras communicational
teaching project. Mimeo, University of Edinburgh.
Prabhu, N. S. (1987). Second language pedagogy, Oxford: Oxford
University Press.
The Author Responds. . .

ALAN BERETTA
Michigan State University
I stand by my paper, but choose to abstain from further written

debate (here and elsewhere) because the matters raised require
discussion of issues that do not serve to advance understanding of
substantive questions (such as the treatment of error).
THE FORUM 115

Vol 24 1

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Vol 24 1

Transféré par

Droits d'auteur :

Formats disponibles

TABLE OF CONTENTS

TESOL QUARTERLY Volume 24, Number 1 ❑ Spring 1990

A Journal for Teachers of English to Speakers of Other Languages

To print, select PDF page

BRIEF REPORTS AND SUMMARIES

Information for Contributors 117

The titles of two articles in this issue are framed as questions,

What is the Role of Language

The applied branch of linguistics known as language planning

The African National Congress (ANC) has identified language as

LANGUAGE PLANNING BEFORE LP

APPLIED LINGUISTICS AND LANGUAGE PLANNING

LANGUAGE IN POST-APARTHEID SOUTH AFRICA

LANGUAGE PLANNING IN POST-APARTHEID SOUTH AFRICA 11

LANGUAGE PLANNING IN POST-APARTHEID SOUTH AFRICA 13

LP for the Sociolinguist

LANGUAGE PLANNING IN POST-APARTHEID SOUTH AFRICA 15

LANGUAGE PLANNING IN POST-APARTHEID SOUTH AFRICA 17

LANGUAGE PLANNING IN POST-APARTHEID SOUTH AFRICA 19

LANGUAGE PLANNING IN POST-APARTHEID SOUTH AFRICA 21

A Context-Adaptive Model for

The literature on the evaluation of language teaching programs has

Evaluation, the systematic attempt to examine what happens in,

Step 1: Audience and Goals

staff, program developers or researchers from other settings, or a

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 25

Step 2: Context Inventory

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 27

Reliable and valid measures. For this dimension, available language

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 29

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 31

Step 3: Preliminary Thematic Framework

Step 5: Data Collection

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 33

CONTEXT A SUMMATIVE REPORT

well as to elaborate the context inventory and to revise or elaborate

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 3.5

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 37

Step 7: Evaluation Report

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 39

Alkin, M. C., & Wooley, D. C. (1970). A framework for the evaluation of

A CONTEXT-ADAPTIVE MODEL FOR PROGRAM EVALUATION 41

Managing the Complexity of

Although previous research in ESL composition suggests a link

Until quite recently, process-oriented research in ESL writing

L2 Research on Transfer in Writing

REVISING ACROSS LANGUAGES 45

REVISING ACROSS LANGUAGES 47

REVISING ACROSS LANGUAGES 49

sentence from Anna’s first paper in English has been transcribed,

There was also a noticeable difference in the number of L1 and

REVISING ACROSS LANGUAGES 51

a Revisions divided by the total number of words generated by a subject in either L1 or L2

In addition, a comparison of the frequencies of revision per 100

Linguistic and Discourse Features

When the types of linguistic processes were analyzed, substitu-

REVISING ACROSS LANGUAGES 53

REVISING ACROSS LANGUAGES 55

REVISING ACROSS LANGUAGES 57

REVISING ACROSS LANGUAGES 59

Another Turn in the Conversation:

This study addresses a controversy in cloze testing. At issue is

This paper contributes to what is known about cloze procedure,