Alderson, J. C. 1 988: Innovation in language testing: can the
microcomputer he l p Lancaster: University of Lancaster. iv + 44 pp. (Special Report No. 1 : Language Testing Update) In the area of computers and language testing, much attention has been focused of late on delivery systems (notably computerized adaptive testing), as well as on related research and theoretical issues (Bailey, Dale and Clifford, 1987; Stansfield, 1986; see also Weiss, 1983). By contrast, the Alderson volume sets out to describe innovative item types that are facilitated by the computer. Not at all erring on the 'grand side' (Pollitt, 1987), the volume under review has an unusually restricted aim: it examines the potential of CBELT (computer-based English language testing) exams that are delivered as well as scored by computer. The focus is on computer- assisted rather than on adaptive testing, on the microcomputer rather than the main-frame, on testing technique rather than test analysis or delivery. But the restricted scope of this computer testing survey seems unneces- sarily limiting. Throughout the text, one has the impression of a tension between the self-imposed CBELT constraint of totally machine-scored pro- cedures on the one hand, and tantalizing options involving human inter- action and human scoring on the other. And well there might be. Another area in which computers have been utilized in the broad field of language is that of translation. Early efforts to manage the entire process by machine have largely been abandoned in favour of computer-assisted translation. This speeds up translation and provides for much improved accuracy and consistency, by allowing for considerable human interaction and initiative. It does not seem unreasonable to encourage this same latitude in computer applications to language testing. Despite this limitation, the text is very useful indeed. It is based on 'research' (I would prefer 'exploration') sponsored by the British Council and carried out at the University of Lancaster by a 10-member team, includ- ing such luminaries as Dick Allwright, Geoffrey Leech, and the writer of the report, Charles Alderson. Chapters 1 and 2 are introductory. The initial chapter outlines the objectives of the team's investigation, looking at limita- tions t o date in CBELT t est i ~g and summarizing interesting options for the future. The second chapter defines CBELT, including its limitations, and surveys technological options ranging from computerized spelling checkers to video-disc sequences. Chapters 3 and 4 help place in perspective the specific techniques discussed later in the report, but they also go beyond this by indicating specific 120 Book review technological procedures and techniques available on the computer. Chapter 3 discusses and illustrates possible advantages of the computer in language testing, including diagnosis and tailoring (through branching). Chapter 4 explores possible CBELT applications in listening, reading and writing. Chapters 5, 6 and 7 provide systematic presentations of CBELT tech- niques. Chapter 5 looks at CBELT options which could be employed in con- junction with CALL software presently available from the British Council and Cambridge University Press. Chapter 6, the most detailed and lengthy in the report, turns from CBELT and test content to CBELT and test method or technique. Here Heaton's paradigm of objective-test items is followed, illustrating how the computer might produce each of his seven item types. The short Chapter 7 looks beyond item types to innovations involving matters ranging from graphics to student feedback. The concluding section, Chapter 8, examines seven characteristics of computers which provide the potential for innovation in language testing. There is much to recommend in the report. While upbeat about the poten- tial of microcomputer applications in English language testing, the investiga- tors probe some of the limitations of the new technology. Not inclined to be prescriptive, the research team raise a number of interesting questions: for example, might the massive data that can be collected on a given student or item make profiles almost uninterpretable? Won't the constraints of machine scoring severely limit the nature of the evaluation process, perhaps causing us to take a step backward to exclusively multiple-choice techniques? Might 'help' options on the computer reduce a desired level of risk taking? Is everything that we can do on a computer worth doing? Moreover, limitations of CBELT in the testing of writing and speaking are clearly acknowledged. While the 'main concern' of the investigation was 'the possibility for innovation in test types through CBELT', the published report explores a wide range of computer options available to test makers. It is these special computer options - far more than any innovative test types - that are of the most value in the report, for they hold out the promise of improving the effectiveness of conventional as well as of innovative items. For example, Alderson and his team demonstrate how the computer can remind examinees (as often as necessary) of test instructions, so that the exam might better reflect the candidates' actual skills. They point out how the computer can readily show the consequences of one's selecting a given multiple-choice option; this procedure alone holds great promise for improving the effective- ness of objective items. Interesting adjuncts that are discussed include a built-in speller, thesaurus, and 'help' menu. It is shown how computers can restrict what the sudent is exposed to, utilize video sequences to simulate real-life situations, and provide great flexibility in how and what language is presented. In addition, various nonmultiple-choice options are presented, such as dictation, trans- lation, C-Tests, and intrusive word (e.g. cloze elide or editing); while these are not innovative test types, they do represent a reasonably broad scope of options available on the computer. In addition, the report demonstrates ways in which the computer can facilitate examinee initiatives while taking Book review 121 a test, an important ingredient in the development of future communicative language exams. The text will also be appreciated for its rich array of examples and its dozens of video-screen displays which exemplify the test items under discus- sion. Commendable, too, are the organization and delimitation of the inves- tigation and the clarity and practicality of this useful 43-page report. Although potentially helpful to those planning to develop computerized language tests, this report could be improved in the following areas. First of all, some recommended testing techniques and processes may be debatable to employ. For instance, the word-scramble approach to sentence construction can be inefficient, and ineffective for those who might conceptualize a perfectly good sentence that does not include all the words in the test writer's model. Moreover, while the provision of immediate feedback on perfor- mance could easily be built into a computer test, this would probably not be advisable while the examination is in progress, due to the anxiety that might well be aroused in many examinees. In addition, the categories to which various techniques or question types are assigned need some modifying, particularly if the report is to be used by relatively unsophisticated test developers. For example, I feel that dictation would be more appropriately classified under general proficiency than under listening, and intrusive word also under general proficiency, rather than under reading. Also, one would want to qualify the 'writing test' categoriza- tion of the partial parser. Since so much effort seems to have gone into the investigation reported in this text, a few more details regarding the feasibility of certain options seems logical. For instance, the cost of suggested video-disc sequences and graphics - per minute, say - could help one to determine whether or not these are a possibility for the near future. Fascinating as these are, the cost may be pro- hibitive (the cost of merely pressing a single disc, not t o mention the very high cost of filming and producing a video-disk, is in the range of $1,800 or 1 loo). Caution is likewise called for in relation to implementing diagnostic testing. While the computer is obviously capable of reporting individual per- formance on matters such as prepositions and phrasal verbs (as mentioned in the report), the test would have to be rather lengthy indeed to achieve adequate statistical reliability for a satisfactory variety of grammatical structures. And even though the report is relatively short, an index would have been useful. A more important area of concern is the book's overview on computers and language testing. Applications of item response theory are numerous and highly significant, yet are scarcely mentioned in this report. IRT-based test evaluation including bias identification and appropriateness measure- ment deserves mention. IRT-based computerized adaptive testing likewise begs attention, particularly since it qualifies as CBELT measurement: an exam that is 'delivered by computer snd also scored by computer'. Its potential for increased precision, its invariant scaling and its efficiency make the adaptive or tailored test one of the truly significant testing achievements of the decade. Although the emphasis in this text is not on test delivery, 122 Book review computerized-adaptive testing merits more attention than it is receives. But an even more significant concern is the nature of the 'innovative' test types presented - the principal focus of the book. In the introduction, the author refers to the 'very conservative' techniques used to date in computer tests. He adds: 'Unless we are careful, there is every danger that the increased popularity of CBELT might represent . . . a step backwards in language test- ing'. Yet the techniques that receive the focus in this report are almost exclu- sively conservative sentence-level, multiple-choice formats, adaptations of Heaton's seven objective-test item types. Discourse-level material is essen- tially absent, as is any truly innovative item type. Nevertheless, the investigation was certainly worth doing, and the result- ing report deserves very much to be read and used. Even though the item types described are not particularly innovative, the host of testing procedures described in the book are innovative. And they hold the promise of comple- menting CALL, assisting our research, easing the anxiety of the testing situation, providing increased flexibility in the measurement process, and spurring student initiative. References Bailey, K., Dale, T. and Clifford, R.T. 1987: Language testing research. Monterey, CA: Defense Language Institute. Pollitt, A. 1987: Review of Henning, G., A guide to language testing: development-evaluation-research. Language Testing 4 , 233-36. Stansfield, C. W. 1986: Technology and language testing. Washington, DC: TESOL. Weiss, D.J., editor, 1983: New horizons in testing latent trait test theory and computerized adaptive testing. New York: Academic Press. Brigham Young University Harold S. Madsen