TO +PPROACHES LANGUAGETESTING

Abstract. - Ram fuhishGiri'

'What does it mean to know a lhe fundamental question in any language testing theory is language?' In attempting to answer the question. numerous language testing theorists have developed theoretical models of language, and have described tinglage abitty, competence, communicafive competence, and performance. They have adhered that understanding of these witt eyble. a test developer to des9n, develop and use iarguage teststhat match their descriptions, and therefore, meet the criteria of validity, authenticit.v and.ripticability of target language use.Out of the numerous models emerge approaches.to lan*yqse tginng whi'ch-p*ia" an"unierstanding if how to view a language and language abilities aia no, bestio assess them. The present arficle discasses some of the most prevalent approaches in the etd of language f testing, specially the second or foreign language tesnng, and provides a bnef ciitique eich-of them. Filtally, it presents a case as to what may be,-in o iituonon where there for are too manv approaches to choosefrom, appropriate for testing EFL in counfties like Nepal, l. Introduction

developer viewslanguage. That is to say,a view of what language hasdirlct bearingon is what is testedand how it is tested.Language competencb tanguage oi abilityis the ,r-ujo, concernin all language teaching testingapproaihes or because is, obviously,whai is thit beingtested. some approaches propose measure in isolationor in test conditions; to it other's to seeit in the contextof all other factorsthat impactit. A structuralist try view of language' example, for is.likelyto emphasise accuracy, therefore and exert an approach to language testing which divideslanguage systems inio segments order for developing in separate testsfor them-A singletrait view, on the other hand,treatslanguage a as unitary ability and recommends integratiVe structure.A multi-methoa, an test iulil-trait view of language, however,.-.views. languageas a combinationof knowledgeor competencies coupled with the abilityto implementthat knowtedge language in appropnate anygiven to contex!, provides and abroadbasisfor both the dev-lopmeniunJ brunguagetests. ur. ' The aim'of this article is to discusssome of the language testing approaches particularlysecondor foreign language testing approaches hive beendominantlyin that practicein the language-testing in the tast ndy years.There ut. ri* field r."iions of this artiile, eachone devoted a separate ofupptou.h", representing separate to set a schoolof thought'Eachsectiondiscusses the concerned how schoolortrtinun! hnguageanc language "i;, such (construct)andwhat approaches suggest measure abilities they to abilities.

The design for developing language tests is based on and influenced by how a test

Mr' Giri is Lecturs at the Departrnent Engiish Language of Education,Tribhuvan university, Kirtipur. presently,he is a research scholarat the Schoolof Education,Victoria u*versiq,. Melbourne. Australia.
Journal ofNELTA LbL7,No.1&2
December,2002

l2lRam Ashish Giri

as in discussed this articleemerge, it was indicatedearlier,out of the The approaches of testing models.Discussions such modelshave numerousand often diverselanguage purport of the present article. For a deliberatelybeen excludedas it is beyondthe and see testingrnodels, Canale Swain(1980)andMcNamara1996). of discussion language 2. The Approaches Approaches 2.1 The Discrete-point in attemptto test knowledgeof a language approaches testinglanguage to Discrete-point consists on are Suchapproaches based the theorythat language segments. decontextualised sounds,'etc.)and different skills, for example, different parts (grammar,vocabulary, of that can be readingand writing, and theseare madeup of elements listening,speaking, test can compriseof a large numberof They assume that a language tested separately. discretetest items, which, as a group, cover all aspectsof languageability. Language systems of of testing,thus,is testingthe abilityof handling differentelements the language assumption that on and of and development individualdkills separately independently the masteryof that elementor skill eachitem gives preciseinformationabout'a candidate's 2000). his abilityasa whole(Banerjee andthat the total scoredescribes or herlanguage
language (a systemof systems) Elements skills (tested & separately /objectively) Aggregated score(Language ability interpretedin terms of the score)

Language*r

(reflectsone's Score Ability in language)

testing(ba$edon the ideasfrom Lado 1961) to approach language Figure 1: The discrete-point

that language restson the premise and theoryof teaching testinglanguage The structuralist levelsof of several and distribution as a set of habitsinvolvesmattersof form, meaning relevantand scientific, and objective, be should precise, stnicture, the testingof language to language discrete levels is equivalent at of that testing knowledgeof the elements of test A of testingknowledge language. typicaldiscrete-point consists (a) test of a specific point or phonological.item a vocabulary or such as a syntactic of component language of iterq (b) a readingtest in which comprehension a text is testedthrough testingthe in items contained the text, and (c) a writing test in which a of understanding specific provided. already item or information to is candidate required write on or abouta specific of Obviously,the focus is on the achievernent accuracythrough reproductionof of languageelementsindependent their context rather than on fluency and isolated of construction meaning.
Jownalof NELTA

VoL7,No.1&2

Decanba,2002

Approaches to Longuage Testingil3

The discretepoint tests, through they are highly reliable,are not valid tests because they are irrelevant, does not meet all the requiiementsof testing, are too artificial to be authentic(Weir 1990) and therefore, do not give information about how the candidateuses languagein real communisative situation (Davies 1990). Knowledge of languageelements has no practical import4nce without the knowledge of languageuse. So atomistic types of testing which test l4nguage elementsin isolation has no practical value becauseit gives only limited information about the test's languageabiiity (Spolsky 1975), and no attention to the way languagb elementsinteract in b larger context of communication.The discrete point test is, therefore, ineffective becausecrucial propertiesof languageare lost when its elementsare separated(Oller 1979). 2.2 The Integrative Approaches Integrative approaches,based mainly on the work of Oller (1979), emerged as an oppositionto the discretepoint approaches, and promotedthe notion that language ability is a unitary factor rather than a divisibleconstruct.Accordingto the Unitary Competence trIypothesis(UCH), underlying languagebehaviourscan be specifiedon the basis of the candidate's Pragmatic Expectancy Grammar which is a psychological representation of the languageuser's ability to expect utterancesin a given context (Shohamy 1996), and which is the chief mechanismunderlying the skills of thinking, understanding,speaking, reading and writing. Integrative tests, such as cloze and dictation, are said to operationalisethe candidate's language ability, which includes linguistic as well as extra linguistic knowledge to reconstitute the meaning of a text. That is to say, such tests integratea number of languagesub-skillssuch.asgrammaticalknowledge,the ability to extract information, lexical knowledge and the ability to construct a discourse. A cloze test, for example, tests grammatical knowledge and ability to extract and predict meaning from a text.^A dictation tests a candidate'slistening ability, ability to decode in writing, and therefore, their ability of writing skills and at the same timq their ability in receptivephonology. A writing test can test their knowledgeof grammar,ability to construct a discourseand ability to managetheir language resources(Banerjee2000). Similarly,a readingtest can test a candidate's cornprehension, ability to extract,construct, establishand guessmeaning,knowledge of grammar,and lexical knowledge. The integrativeapproaches language to testingare basedon the principleof component factor. analysis in that scores from tests or sectors such as voca6ulary, grammar, phonology, reading, dictation cloze and composition are analysed and conelated to determinethe candidate'slanguageproficiency. Integrativetests are commonly used in prbficiencytestsbecause they rEflectreal language more closely(Banerjee use 200b)

.tl Communicativefticusprelude
Sub-test: Writing

Figure2: The Designof the Integrative Test(adapted tromCarroll andHall 198S: 8)

Journal ofNELTA VoL7,No.1&2

December.2002

14/Ram Ashish Giri

tests,which emphasise mastery language in the total socialcontext, the of Integrative use aspect of language.It recognises attempt to measurethe total communicative that linguistic performance involves the individual's capability of mobilising his or her linguisticcompetence perfonnance and abilities anintegrated (Howard1980:275). in way Oller'sUCH doesnot explicitlyimpiywhat is traditionallytermedas 'competence and 'performance'.However,an analysis the ftatures of UCH reveals of that they sharesome features 'performance' Oller writesthat 'the objectof interestis language it is usedfor of as purposes- for getting and giving informationabout facts or statesof communicative attitudes towardsthosefactsor states affairs.(Oller 1979:16). affairs,andfor expressing of (1972)concept abilityfor usedefined rulesof processes reflectHyme's These clearly of as performance. NcNamara(1996) writes that there are two aspects Oiler's model that of (i) relate to performance. Naturalness and real time processing language of elements in relationship with the discourse, and (ii) relatingthe sequencing language of elements to mapping. extra- linguisticcontextvia pragmatic Oller's Pragmatic Expectancy Grammar, which is knowledggof the language is systems, constructed and modified in the courseof language acquisition. Language proficiencyconsistsof such expectancy generatingsystem. Languagetesting, then, is primarily a task of assessing efficiencyof the Pragmatic ExpectancyGramrnar in the process constructing.A language in orderto be valid in which the learneris in the of test terms of the theoretical constructwill haveto involve and challenge efficiencyof the the (Oller 1979). Corder(1973:351) grammar learner's developing callsthis phenomenon "the transitionalcompetence" constantlyevolving, dynamicability in the target language. a The role of the test,ther4is to showhowfar the student's language movedtowards-an has approximation the nativespeakers' of systern. for Oiler'sUCH hasbeenquestioned the fact that the datafor his research camefrom which focus on conscious explicit features language academic tests and of ratherthan the features, functionaland sociolinguistic and that the data is not conclusive supportthe to (Bachman existence unitary competence of and Paimer 1996). Oller's approach implies that basedon the unitarycompetenoe, language a learnershouldbe ableto usethe target languageequally well in ali its manifestations. However, it is not usually the case. ESL/EFL practitionerseverywhereexperience that non-Englishspeakingbackground (NESB) learners good at one skill or aspect havedifficulties someother aspects are but in (Bachman1989).Also, Oller'stechnique component of factor analysis explorethe to (Chalhoub-Deville underlyingstructureof his data is inappropriate 1997). One other problemof integrative testsis that the results reported a singlescorethoughthetasks are as in the tests combinedifferentcomponents language of ability. It is thereforedifficult to interpretthe scoreasto what it actually means.

Jownalof

NELTA

t b l . 7 , N oI .& 2

Decenber.2002

Approaches to Language Testing/l5

2.3The Proficiency Approaches The term ' second for_eign or language proficiency' usedin two differentways.Firstly,it is is.related directlyto'ability'and is defined the degrbe competence"or as of capability a in given language demonstrated an individualin a liven point tf time independent a by of specificcourseor textbooksor teachingmethods.S""ondly, it relatesto th; extent and adequacy an individual's of control or masteryof target language all kinds of socialor in interactive situations including work settings demoistrated tests.The former meaning as in of proficiency denotes competence wherealthelaterone specifies performpnce What makesa language test communicative non-communicative the problems, or is tasksandthe manipulations language of (Davies1990).To Davies, purpose skills the and contextof testingarethe mostreleyant features communicative of language testing.He. in a similarfashion Hymes's to dichotomy knowledgeand ability for use, distinguishes of two aspects communicative of competence 'knowl elge that' , whiohrefersto t<noivteAge in the Hymes's dichotomy, and'knowledgeltorr,',*nLn refersto abilityfor use.Davies retainsthe term 'knowledge',to refer to knowledgebut usesthe p."n;i;*/ ;; 'control' to referto performance (Davies 1990: "# 162). behaviouror product of behaviourvaluedin . Proficiencyfor Daviesis an observable itself' By control,he means that a candidate needs demonstrate only knowledge to not of language also skill in the use of that knowledge settings but in which are,to someextent, communicative.
Competense Knowledge that General Indirect tests Proficiencvtests performarice

Construct, predictive validity Content validitv competence Context specific, work sample ITlglfo.ral TOEFL interviews, SLC , Free
modelof language proficieney profrciency figure 3: Davies's and testing(based the ideasfrom on Davies t t, t -

Knowledge &ow Specific Directtests Achievement tests

The figure 3 aboveshowsthat Daviesviewscompetence a general, as abstract ability,and therefore, cannotbe testeddirectly.Proficiency it tests,suth i'r rorrr" .arr-orrry provide approximations a testee's of competince indirectiy All'proficiencytests,in this sense, are indirect'Performance, the othlrihand,. contex rpriinr, and on is canbe measured directly throughperformance tests such as free interviews. is, trrougtr" ti possible have semito directtestssuchas simulations role playswhich and generaland specifictesting contexts. "ornbin., All tests,in this sense the term, are proficiencytests. The of discrete-point approach mentored Lado (1961),integrative by approiches proioundedby CanouanoHa1 (19g5),
JoranalolNEL?A lbl7,No.I&2
De.canbq,2002

16/Ram Ashieh Glri

(19'68), theoretical and oller (lg7g), the pragmatic approach suggestedby Daviqs ^by-cumminj .the tu:h. as ACTEFL 1t99+;, rating-scale'approachet frameworks advocated frameworks developed by Hilgfotis, Bailey and assessment guid.tin., and ASLp& and to proficiency assessment. 5t.* (19g1, quoted in Chalhoub-Deville 1997)) are approaches and the focus thej lie The differen..i in these approaches in the ways they view language discussion of the attach while developing-ianguage tests. A brief comparison and ' is approaches given in the final sectionof the articie' 2.4 The Performance APProaches to approaches languagetesting developed out of the uncertainties The performance-based testing field in 1980smainly becauseof the non-existenceof a that existed in the tanguage long history and a well-developed theorelicJ model, in spite of the fact that they have a definitions and widespreadusagein other fields. So, pragmaticconsiderations operational result, language guided the devJlopmentand use of languagetests(Shohamy 1996). As a 'theory-free', performance-baied,task-driven, and were consideredto be tests were communicative,functional, direct and authentic(Shohamy1996.145). to approaches languagetesting, thus, developedin responseto The performance-based practice of needs in specific-pu.pot" contexts, diawing on the theory and practical the approach performanc. urr.ru..nt in vocational fields (McNamara 1996)' However, testing mainly after the ieceived a particular interest in communicative language madein the field by Hymes(1972) theoreticaladvance and were developed usedon the basisof (a) purposeof Languagetestson this approach (c) samplingof tesi task, (d) simulationof situation,and (e) testing, (b) testin! context, ,ou"rig. of languige content and of languageabilitiesand skills.
Rater Scale

Y (score)-)

Performanc. l*rtn, Instrument A

. Candidate

from McNamata1996) testing(adapted to approach language Figure4: The performance

of by as The test performance, the figure 4 aboveshoys,is analysed means a rating scale, and a whichprovides score a description. that very popula\-?fe (a) they became approaches based why performance The reasons in language (b) describing terms, of are free from compliiations theoreticai-complicated (c) rating scalesdescribed terms was easy to iomprehend, behavioural/functional generic, descriptiveand.the way, (d) rating scaleswere languagein communicable criteria in them and were taken from everydaylife situations,apd . (e) hieiarc-hical p.rCI.tun.e based tests have high face validity high wash back and high ciient acceptability.
Journal ofNELTA Vol. 7, No. I & 2 Decenber,2002

Approaches to Language Testing/l?

The very simplistic approach to tests becomesa weaknessof the perfbrmance-based approachesto languagetesting. As it was said earlier, performancebasedtests focus on tasks rather than constructs. Constructswhich refer to theories of competence, knowledge, communication, and skills are overlooked causing,therefore, a lack of construct validity becauseit is the constructsthat should drive the development,scoring, and interpretation of performanceassessment. The performanceapproaches languageassessment, to thus, are simplistic, narrow and are not based on theory of construct, but on a niurow view of communication(ShahomyI 996). .i A theoretical model of communicative competencebased on a more comprehensive understandingof the.factors underlying ability for use is needed which should guide the selectionand construction of.relwant test tasks as well as the rationale for scoring criteria and rubrics.

2.5 The Communicative Approaches The communicative approach language to testingis basedon the assumption language that is learnedto express different kinds of functions, and emphasises need of testing the language using processes communication by of such as using language appropriately in differenttypesof situations interactwith peopleand performon a wide rangeof tasks, to Suchtestsemphasise authenticity of test methods,authenticity of test materialsand authenticity of responsesso that the test outcomesgive a real picture of a test's communicativecompetence. Thus, the focus is on qualitative evaluation of the candidate's language knowledge ratherthanrthe. quantftative assessment linguistic of (Savignon features 2000) , Communicative language testingapproaches becamb commonpracticein late 1980's a particularly andearly 1990's, afterthe muchtalkedabbutworks of Canale Swain1980. and (1990)who responded the concern 1970s andBachman to of that,language produced on : of non-communicative tests was artificial and non-representative the test's actual communicative (Shahomy competence 1996).So, therewas a call for directtestsin which boththe testformatandprocedure duplicated closelyaspossible reallife situations. as the Bachman and Palmer's(1996) multi-method rnulti-trait approach has been particularly. populbr amongESLIEFL practitioners the US 'and Auitralia because in this provides elaborate complehensive approach an and represgntation language proficiency. of Bachman and Palmeruse the term "framework"to refer to what otherscall 'model' (McNamara1996).Bachman's theoretical frameworkof communicative languageability (CLA) consistsboth knowledge(competence) and the capacltyfor implementingoi executing that competence appropriate, in contexrualised communicative language use. tt shouldbe notedthat that thereis language abilitythat is non-communicative. language The abilitythat is non-communicative not of muchuseexceptperhaps research is for purposes. Every language learned cosmunication. is for Therefore, communicative ability is what is

Jownalof NELTA

VoL 7, No. I l&,2

Decenbq,2002

lE/Ram tuhish Giri

important. The writers make distinctions between linguistic, commuiicative and pragmatic competencies. The CLA framework includesthree components: 1. Langtage competence:Language competencecomprisesessentiallyof a set of specific knowledge componentsthat are utilised in communicationvia language2. Strategic competence: Strategic competence characterisesthe mental capacity for implementing the components of language competence in contextualised communicative language use. Strategic competence thus provides the means for relating language cornpetencies to features of the context of situation in which language use takes place and to the language user's knowledge structures (socioculturalknowledge,'realworld' knowledge),and 3. Psychophysiologicalniechanisms: Psycho-physiologicalmechanisms refer to the neurological and psychological processesinvolved in the actual execution of language (soundlight). as a physicalphenomenon

PRAGMATIC COMPE

Svnt. Phor/Graph

Cohesion.

Rhet. Org I .

deat tr{arip Heu Imag Semit Smit Semit Cultual Fuc. Fuc. Func. Func. todial toReg. toNsr Rdtefig. or wiety of Socech

Figure 5: Frameworkof Communicative lamguage Abilify (Bachmanand Palmer 1996)

The'knowledge of languageand the knowledge.of the world contributeto the development strategic of abilitiesleadingto psychological physiological processing and in order to make utterances appropriatefor a give context of situation.The languagd competence, the figure aboveshows,canbe brokendown to other sub-competencies. as The frameworkcoversthreebasicareas, knowledge language language of or competence, cognitiveaspectof ability for useof strategic competence, modalities performance and of physiological or psycho mechanism. Language competence consists two mainaspects organisational (a) of competence the knowledge of creating or recognising grammatically correct utterances and
Journal ofNELTA

VoL7,Nal&2

December.2002

Approeches to Language Testing/l9

comprehendingtheir prepositional content (grammatical competence) or of organising them to form text (textual competence)and (b) pragmatic competence-knowledgi of thi pragmatic conventions for performing felicitous language functions (illocutionary competence) and knowledge of sociolinguistic rules of appropriateness(sociolinguisti-c competence). Strategic competence is a general ability of assessingand planning to determine and execute means of achieving communicative goals. ttris is ability, not knowledge. Pragmatic competenceinvolves an understandingof the social rules governing the use of language.The speaker/writermust know what is socio-linguistically acceptable or appropriate in a given context, and be sensitiveto varieties of language,to registers, naturalnessand to cultural references.It is not easy to assesspragmaiic -ompetence not only becauseit is difficult to match utteranceswith contexts, but ilso becauseit is more difficult to elicit than to measureit. Organisationalcompetence,thus, has tended to be the locus most languagetests because is quantifiable. it The framework seems'rich enough to conceptualise the issues involved in second language perfbrmance situation and comprehensiveto incorporate every major aspect of the languageability; and it appearsto be a still-evolving frimework oi second language 'a ability which provides map rather than a prescription of language ability, setting out those areasof ability that needto be taken into accountin secondlanguageurr.rrrn.nt. There are problems with this approach. Firstly it is not easy to identify test performancewith trait or ability becausea performancecan be associatedwith ability if the test task is 'direct'. However, given the fact that languagetests, like all mental measures, are indirect, the task of identifying performancewith trait is problematic(Davies 1995). Secondly approachis primarily concernedwith construct nanAity and disregards the face, context, and predictive validity The approachsuggestscoveragein tenhs o? language components, skills and abilities,and recommends content of communicative language tests to be motivating, substantive, integrated and interactive; and topics, opinions,-ideas, reading passages texts, dictation and composition to be appropriateand 6e based on a commontheme(Bachman 1990:320) The last recommendationgiven above may ensurecoverage and authenticity but it still does not ensurethat the test will have content validity, whiih is a necessary quality of a test' In order for such a test to have content validity the test constnrctionshouldbe based on a thorough analysisof the target domain upon which the test content is to be based (Davies1995) 2.6 The Rating-scales Approaches - ACTFL and ASLpR Rating scalesas a basis for assessing languageproficiency are becoming widespreadin the recenttimes because people'sincreasing of interestin transparency educational in systems. Rating scales are useful becausethey describe what the attainment of a given level of language proficiency means in practice CNorth 2000). Rating scale alpp.oaches to proficiencyassessment becomingmore and more popular also becaus. are ruiing scales(a)
Journal ofNELTA

LbL7.No.I&2

December, 2002

20/Ram Ashbh Giri

a provide a basis for the learnerto comparehim/herse[ 0) establish framework of in and for achievement termswhich are meaningful user-friendly all reference describe to judged ratings,especially the and of stakeholders, (c) increase reliabilityof subjectively productive languageskills, and (d) provide a commonbands and meaningfor sugh judgements. Rating scalesalso set learning'goalsand descriptionof proficiencyfor learners allow the resultsto be measured against targets.They alsooffer a profile the and which becomea basis for selection" of a candidate'sability in the target language, recruitmentor a remedialprogramme the perspective for employment organi'sations or institutes trainingor education Today,the most popularrating scales the AustralianSecond are language Proficiency (ILR) scales, the American Ratings(ASLPR),the Interagency Language Roundtable and (ACTFL) Guidelines, of Proficiency Councilof Teaching ForeignLanguage which are all derived from the sarnesource,the rating scaledeveloped the US Foreign Service by Institute (FSI). The ILR scales for the US government are employees only and not for a common learner English.So,it is excluded from the discussion of here. (ACTFL f9S2) 2.6.1The AmericanCouncil of Teaching ForeignLanguages of (1982) describe test's languagg The ACTFL Guidelines proficiencyon a scaleranging a from novice to superior.The descriptors represent degrees proficiencyin real world of communication describe and how language learners typicallyfunctionalongthe rangeof competence. trisenctional framework of the scale(seefigure6 below) consists (a) The of functionswhich tells what a test can do ,at what level, (b) content,which sayswhat contentsor topics can serve as a basisfor assessment the level in question,and (c) at accuracy, which describes what quality or featurethe candidate's language will contain andat whatlevel.
Language FunctionS 0-0+ Low Novice
l-l+ Intermediate

Accuracy/'Description of TexV Content LearnerLaneuaee

2-2+ Advanced
J-JT

Suoerior versionof ACTFL Scale(adapted from ACTFL Guidelines 1982)) Figure6 Summary

The problemswith ACTFL Guidelines that they (a) take a generic,educated are native (b) the of speakeras a criterion againstwhich to measure proficiency, NESB learners, promote artificial context where the interactional skills and the set of roles that a can play are limited, and (c) do not correspond findingsin L2 acquisition candidate to (Brindley 1995).Furthermore, data collectionand analysis procedures how they the and
JownalofNELTA VoL7,No.1&2 Decentbs,2002

Approaqhes to L,anguage Testing/2l

were initially turned into performancedescriptorsare questionable,and so is the validity of the guidelines bgcausetliere is no empirical evidencebn how the scale descriptors have 'key-tasks' to levels without a principled been calibrated. Also, the guidelines allocate basis(North 2000): 2.6.2 Australian SecondLanguage Proficiency Ratings (ASLPR' 1984) ASLPR is an elaborateprocedureto measuregeneralproficiency in a languagelearnedas a second or. foreign language. It is descriptive scale in the sense that it consists of descriptions of language behaviour at nine levels, from zero to nine. In other words, ASLP& as it is shown in figure 7 below, consists of general descriptions of observable 'examples of-specific tasks' that exemplifuthe sorts of behaviour that are behaviour with observable(Ingram iSSSl. ASLPR has been developed in English, French, Italian, and Japanese Spanish,and its use has expandedrapidly in Australia.
Proficiencylevels 0 Zero or Initial proficiencv l-l+ ElementaryorMinimum Survivd proficiencY
JI

Skill-wise GeneialDescription

Example SoecificTasks

of

Comments

Minimum SocialProficiency 3 Minimum Vocational Proficiencv

4 Vocational Proficiency

5 Proficiencv Native-like
fromIngram1984) Versionof ASLPR(adapted Figure7: A Summary

ASLPR describeslanguagebehavioursin general proficiency levels from zero to native and a test's proficiencyis statedin a profile. separately like. Each macro-skillis described . Each profile has a numerical value which are accompanied by descviptions. The behaviouris in terms of the tasks he of are statements what the candidate's descriptions can carry out and how he can carry them out. The numerical scoresare interpreted directly from the descriptionsso that the reader knows what sorts of things the candidatecan do in the targetlanguage. However, ASLPR is not a test instrumentbut a rating scalewhich is basedon, as it is can that the assessor saidearlieron IRL oral interview.The problemsarethe scalesuggests providesno guidancefor that, nor doesit say develophis or her own test tasksbut the scale how the new test taskscanbe validated.

Journal ofNELTA

VoL7,No.1&2

December,2002

22lRarn Ashish Giri

Sum
Approaches Discrete point Approaches Lado1961 Views on Language Proficiencv Formalknowledge grammatical competence Languagea s_vstem of habitsof communicative Componential analysis Overallcontrol Considerations Test for Comments DeveloDment Indirectreference to No test of interactive performance ability Overallcontrol No consideration of Cross cultural understandins authenticityof materialsl materials response Learningapproach (linguistic) Performance approach (language work) at A combination skills and of work sampledornains Focus academic on conscious andexplicit features of language Componential analysis Reliable, valid Componential analysis of testperformance Lacksfunctionalsocial aspect Componential analysis of testperformance Componential analysis of testperformance

Pragmatic Approaches (1998) Davies

interactive (a) Approaches Oller(1979) Interactive Approaclies (b) Carroll(1961, 72\. 1985 Communicative Approaches Bachman 1990

Unitary abiliw Abstract. general

Componential Abilities

Domains. skills Integration throughtest specification test and modes Cornponential Reallife approach Too elaborate/ Abilities Integration approach comprehensive Contexualisation language of resultingin largetests Testmethods shouldmirror aspects ofperformance undertest Domains skills and PerformanceCapabilities involved Specific purpose testing Focus taskfulfilment on based in language use Directtests and qualitl'of task Approaches Language knowledge Analysisof skills and execution McNamara 1996 underlying components of pedormance performance Work sampieapproach of Qualityof executive UALP/tsICS Cummins 1984 Cognitiveand academicaily aspects of language proficienry, Communicative fluenry Abstract,general Surface fluenry The extentto which Academicand communicative measure language of perfonnance proficienryshould Aclive and cognitive relateacademic participation a task in achievemenl anci contexl embedded activitv proficiency language

Assessment General language Focus conscious on and Framework proficienry explicit Gatures of Hinofotis,Bailey Componential ability langr:age andStem1981; proficiency Language Contefi Soecific Deliverv
Journal oINELTA

Contextspecific

lbl. 7, No. I & 2

December,2002

Approaches to Language Testingl23

Assessment Cornmunication Framework ChalhoubDeville1997 Rating-scales General Language domains Approaches Language components ACTEFL/ Componential ability Skills.Descriptions levels of ASLPR lgure U: Summaryof Language Testing Testi Approactres ,:

Scales. testlicklF not guidelines test for

development

Languagetestersthroughout the world have developed and used different approaches to testing second or foreign languageproficiency. r'rom discrete-point upfr*ch of 1960s (Lado 1961) to context specificassessment framework of 1990sicrruri,ouu-Devilie lg97), numerous approaches,termed differently, have appeared at different points of time. However, as the summarychart aboveshows,despiiethe use of differentterminology,the mentors/writersdiffer on how to test rather than on what to test. For example.whether they use the term, unitary (oller l97g) or integrative(Carroll and Flall i9g5), whether they use componentialabilities(Bachman 1990), ibst.a.t, general, cognitive (Cummins 19g4), or underlyingability (McNamara 1996),they refer to'it. sarneihing - the ability or capabiiity or knowledgeunderlying typesof performance all whetherunder socialor test conditions. 'how' so far as of testingis concerned, some of the writers have gone for a linguistic framework (e.g. Lado) to teit the overall control of language, while others have taken the domains/skills ftamework (e.g., carroll, Bachman).Ho"*e,rer, what may be idealistically appealingto most languagetesters is a combination of the i*o o, *o..-upproaches or frameworks,for example, domain/skiliframewo.t the *c ttr. performance approachfor it to be appropriate the pirrposeand contextof testing(Davies to rgg5). The context specific proficiency assessment (Hinofotis et al lggl) needs a special mentionhere because is an e*emplarycaseof it how a tester can take a theoreticalmodel of languagetesting and make it context specific on the basis of the immediatepurpose, testingconditionsand other constraints. The.ProficiencyAssessment Framework (PAF) (Hinofotis, Bailey and Stern 19g1 in chalhoub-Deville 1997) developedto assesstit. esr oral proficiency .*a! of foreign teaching assistants The rateis, under..this empirical research, !FTA) were required to provide two types of ratings, firstly their overall impression of the FTA,s performance,and then theirjustifications written ,o.*rntr. in Threemain categories.of FTA's performance the were identifiedfor further elaboration, namely language proficiency, celivery, and iortnuni.ution tr anguage proficiency included ability in vocaburary,grammar, pronunciation, and flow of speech. Derivery comprised personality. and non-linguistic faciorr ru.t u, iye conract, gestures, confidence in manner and enthusiasm where as communicationincludld featureslike explanation, use of supportingevidence,clarity of expressionand abilityio r.rur, to students.

Jcurnal ofNELTA

VoI7,No.1&2

Decemba,2002

24lRsm Ashish Gtri

A rating scalewas provided to the ratefs during the secondround of viewing the tapes. A series Jf t.gt.rrion analysis revealed that all main categories and sub-catdgories ' contributed to the evaluation.The FTA's categories canlbe related to the corresponding componentsof Bachman's(1990) CLA frampwor\. For example,the grammar component ana ine communicationcategoriescan relate to grammatical competenceand illocutionary competenceof the CLA framework respectively,though in much curtailed forms. Framework (Igg7) adaptedthe Profidiency Assessment Similarly, Chaulhoub-Deville (pAF) for assessingstudents' oral proficiency for everyday communication. He too of of confirmedthe conclusion Hinbfotis et al's (1981) study that there is a colrespondence pAF categories with the components of CLA framework. Thus, the assessment frameworks (Hinofotis el at, and Chaulhoub-Deville) provide empirical justification for adapting specific aspects of the more general theoretical models to reflect the c of characteristics specifi contexts. above,it can be concludedthat (a) most language To sum up, basedon the discussions testing approaches favour componential approach to language proficiency though the iegarding the nature of the componentsremains debatableto this date, (b) many questi,cn approaches have strong theoretical foundation but scanty empirical support, (c) same 111od.ltare based on empirical data but suffers from poor statistical support, (d) some' approaches are to pbstract and generic or too elaborate and comprehensiveto be of imhediate practical value, and finally (e) adaptation of an approach or approachesto suit to the specific characteristicsand purpose without altering:the theoretical foundation is possiblythe best option for a practicalpurpose' References:
CU TestConstuctionand Evaluation.Cambridge: P. Language D. J.. Alderson, C. Clapham. Wall .199'5. of .1987. Reviews English languageProficie'ncvTests. Alderson.J, C., K.j. Krahnke and C. Stansfield DC. TESOL. Washington View Testsfrom a Communicative Proficienc.v Bachman,L. 1989. An Examinationof SomeLanguage Point.ERIC.2001. Bachman.L. and. A. Palmer .1996.LanguageTestingin Practice. Designing and Developing Useful Zesls. Oford: OUP. Language Bachman.L. f. tSge. The Test of English as a Foreign Languageas a Measureof Communicative and Language Testing: A Coltection of Papersfrom the Annual Technolog,, Competence. ELTS. 7: 69-88. New Jersey, Princeton. C. Research. Stansfield. on , Colloquium Lartgua[eTesting Testing. Oxford: OI-,,?. in Considqrations Language Fundamental L. Bachman, F. i990. ATesting. A. in Studies Language Tests. Usefirllanguage and L. Bachman. F. 1999.Designing Developing CUP: 109-118' Cambridge: T. Edler,N. Iwashita. McNamara. B. C. Davies. Baneqee,J. 2000. IntegratedTests.In Byram, M. (ed) .2000. RoutledgeEncyclopaediaof Language and Teaching Learnirg.London:Routledge Teaching NationalCentrefor EnglishLangrrage in Alsessment ,4ction.Sydney: Language Brindley,G. 1995. Macquarie University. andResearch. proficiency and Proficienry.Language for Framervork Language Canale.M. 1994.On SomeTheoretical Avon. Multilingual Matters. Achievement. Rivera:Clevedon, C. Academic Language to Approaches Second for Theoretical Bases Communicative M. Canale. and M. Swain.1980. I/l: l'11. AppliedLingttistics Teaching Testing. and Press. lesrs. Oxford: Pergamon ltlake YourOwnlanguage Carroll.B. and P. Hall .L985.
Journal ofNELTA VoL 7. No. I & 2 December,2002

Approaches to l-rnguage Testing/25

Chalhoub-Deville,M. 1997.TheoreticalModels, Assess*"nt frameworksand Test Construction Language . Testing ll 3-22. 14 Coombe, 1998.CurrentTrendsin EnglishLanguage C. Testing.ERIC. Corder,S.P.1973. Introduci Applied Linguistics. ng, London:Penguin. Crynmtns,J. 1994. Wanted: A Theoretical Framework for Relating Proficiencyto Achievementamong Bilingual Students. Language ProficiencyandAcademic Achievemenl. Rivera.Clwedon.Avon C. Multilingual Matters:2-19. Davies,A. (ed.) 1968.Language Testing Symposium. Psycholinguistic A Approac&. Oxford:OUP. Davies,A. 1990.Principlesof LanguageTesting. Oford: Black Basil. Davies, A. 1991. Issuesin Language Testing.ESLDILLS Project Seminar,Melbourne,University of Melbourne. Davies, A. 1994. Testing CommunicativeLanguageor Testing LanguageCommunicatively;What? How? MelbournePapersin LanguageTesting 4/L: l-20. Davies, A. 1995.Testing CommunicativeLanlrage or Testing l-angnge Communicatively:What? How? MelbournePapersin LanguageTesting.4l: l-20. Davies, andA. Brown .1990.Designing A. Instruments MeasureLanguage to Proficiency.IWG/Assessment of ProficiencyLevels in JFL, Canberra. Davies,A., et al. 1984.Surveyof English LonguageTeachingin Nepal. Kathmandu:British CounciUODA and HMGAI, Ministry of Educationand Culture, HMG[{. Departmentof Immigration and Ethnic Affairs .1984.Report on theformal trialting on theASLpR. Canbera: Australian Government Publishing Service. Dhakal, T. P. 2000. Effectivenessof Discrete Point Tests and Integrative Tests as Measuresof English LanguageProficiency:A ComparativeSudy. Departmentof EngtishLanguage Education,Faiutty of Education Kathinandu: Tribhuvan University. Giri, R. A. 1996.ClozeTestsin the ForeignLanguage Classroonn. Journal of NELTA I/l:26-31. Howard, F. '1980. Testing CommunicativeProficiency in French as a Slcond Language:A Searchfor Procedures. Canadi Modern LanguagbRevicw 36/2: 272-2g0. an Hymeg, D. Y?, On Communicatire Competence. Prideand Holmes(ds.), Sociotinguistics:269-293. In Ingram, P, Y_81 Using Proficienly Rating Scales with High SchoolforeignlanguagJlearners. ERIC ED 24976',7 Ingranr, D. E. 1985. Assessing Proficiency:An Overview of SomeAspectsof Testing.Modelting and Assessing Sebond_Language Acquisition.K. a. P. Hyltenstar4M. Clwedon,Engl-and, Multifnggal Matters.l8z 215-276. Lado,R. 1961. Language Testing. London: Longman. McNamara, T.F. 1996.'Me-asuring secondLanguage'performance. London:-Longman North, 8'2000. The Developmentof a CommonFrameworkScale of Language-Profciency.Oxford: peier Lang. oller. J. 1979 LangyageTestsat schools: A pragmatic Approach. London: Longman. Savignon, 2000.Communicative-Language S. Tdching. InByram, M. (ed.).2006.Routtedge Encyclopaedia Shohamy;E. 1994. The Rgle o-fLanguage Testsin The Constructionand Validation of SecondLanguage AcquisitionThgrrieg,Methodolog,, Second in Language Acquisition.E. S. G. a. A. C. C. Tion-e. Cambridge: CUP: 133-141. Shohamy, 1996.Competence Performance E. and in.LanguageTesting. in G. Brown, K. Malmkjaer and J. Williams (ds.) Performanceand Cohjcetence Seiona unluage Acquisition. Cambriige: ClIp: ii I 3 8 - 1 l5 . Singh, G. B. 1996.SLCbamination inNepa,l. Kathrnandu:Research Centrefor EducationalInnovationand Development Spolslcy'_B'1975. LanguageTesting: Art or Science? Key Note Addressat AssociationInternatronalede Linguistique Appliquec'e World Congress, France, _ iq lpolrt y, B. 1993.Testingthe English of Foreignstudents 1930,ERIC. 2002. Development ObjectiveLanguageTesting.Oxford: OUp. of !-rylttry, B. 1995'Measuredl|lordg: Tlqe weir, c. J. 1990.communicativeLanguageTesting.New ybrk: prentice Hall.
JournalofNEL[A VoL7,No.1&2

Decsnbq,2002