*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

TEST CONSTRUCTION AND EVALUATION

1. PRINCIPLES OF TEST DESIGN

Language testing at any level is a highly complex undertaking that must be based on theory as well as practice. The guiding principles that govern good test design, development and analysis are validity, reliability, practicality and washback.
1.1. Validity The term validity refers to the extent to which a test measures what it says it measures. In other words, test what you teach, how you teach it. Types of validity include:

Content validity means that the test assesses the course content and outcomes using formats
familiar to the students. Construct validity refers to the fit between the underlying theories and methodology of language learning and the type of assessment. Face validity means that the test looks as though it measures what it is supposed to measure. This is an important factor for both students and administrators. Other types of validity are more appropriate to large-scale assessment. 1.2. Reliability Reliability refers to the consistency of test scores. It simply means that a test would give similar results if it were given at another time. Three important factors affect test reliability.

Test factors such as the formats and content of the questions and the length of the exam must
be consistent. Administrative factors are also important for reliability. These include the classroom setting (lighting, seating arrangements, acoustics, lack of intrusive noise etc.) and how the teacher manages the exam administration. Affective factors in the response of individual students can also affect reliability. Test anxiety can be alleviated by coaching students in good test-taking strategies. 1.3. Practicality

Classroom teachers are well familiar with practical issues, but they need to think of how practical matters relate to testing. A good classroom test should be teacher-friendly. A teacher should be able to develop, administer and mark it within the available time and with available resources.

2. STAGES IN TEST DESIGN Good design consists of a number of stages which progressively elaborate the design. Good test design consists of a number of stages which progressively elaborate the design of tests: 2.1. Construct Specifications: What theoretical model is going to be tested? (skills / structures /functions / vocabulary) 2.2. Format Selection: What types of exercises are available for testing the model? (close-ended tests / open-ended response tests / restricted-response tests) 2.3. Item Writing: To what extent are the instructions clear and precisely focused?

1. SYLLABUS BREAKDOWN 2. TIME ALLOCATION AND TEST PAPERS COEFFICIENT 3. EXAM SECTIONS AND WEIGHTING 4. SECTIONS AND RUBRICS

4.1.1. Reading performance standards Test designers should take into account the five standards areas as stated in the Guidelines (2007). Students have to demonstrate their level of attainment through performing reading tasks as specified in the syllabus and Guidelines: Skim a text for general idea/main idea; Scan a text for specific information; Make inferences: informational, explanatory, propositional and pragmatic; Infer word meaning from the context; Identify discourse markers and their functions in the text; Distinguish main idea from detail; Distinguish fact from opinion; Distinguish cause from effect; Identify the authors attitude/intention Transfer information from text to chart, diagram or table. Re-arrange the ideas or topics discussed in the text. Evaluate ideas expressed in a text (i.e. critical reading). Draw conclusions from the text.

4.1.2. Text types and topics Reading texts could be in the form of newspaper articles, letters, biographies, encyclopedia entries, film/book reviews, report and advertisement. They can be narrative, informative, descriptive, or argumentative texts. Texts designed for testing reading need to be of appropriate level of difficulty.

4.1.3. Text length Streams Arts Humanities All science, technical and Original streams 4.1.4. Test techniques The following techniques should allow testers to evaluate testees various reading competencies. Test techniques should vary so that no particular test technique would become predictable. Wh/questions True/False + appropriate justification Sentence completion Summary cloze Gap filling Information transfer Matching elements Extracting information from the text Ordering elements of the text (events, arguments, etc.) Number of words 400 - 450 350 - 400 250 - 300

Test techniques should lead to clear and reliable scoring for the sake of equity. Three to six of the above techniques must appear in the reading comprehension section to guarantee appropriate coverage of the reading performance standards.

4.1.5. Rubrics Rubrics are those characteristics of the test that guide the testees and inform them about the procedures for performing test tasks. They also inform testees about how the test will be scored. For the sake of standardization, the following rubrics should appear in the test paper:

Which of the following would be the best title for the passage? Which of the following could be inferred from the text? Which of the following questions does the passage answer? Are these sentences true or false? Justify Answer these questions in your own words Complete these sentences in your own words Fill in the blanks with a word from the text Fill in each blank with an expression from the text Find in the text words or expressions that mean the same as.. What do the underlined words in the text refer to? Pick out from the text expressions or phrases which show that Match the main idea with the corresponding paragraph In what order do the following ideas/arguments/events appear in the text? What is the writers opinion/attitude/tone/intent? Choose the right answer from the list given Fill in the chart with the right information from the text

It comprises three content areas: vocabulary, grammar and rhetorical functions. It aims at assessing students ability to use language accurately and appropriately.

4.2.1. Vocabulary Vocabulary test items should be related to the topics and themes dealt with in the syllabus and textbook(s). It should particularly focus on the following areas: collocations Word associations word definitions synonyms/ antonyms poly-words word formation suffixes, prefixes and their meanings phrasal verbs idiomatic expressions. 4.2.2. Grammar Grammar test items assess students ability to recognize, understand and apply rules and their grammatical forms. They also evaluate their competence in using those forms accurately and appropriately. The test items should be realistic and contextualised. The grammar test component must include items related to the following list of structures: Tenses (including past perfect, past perfect continuous and future perfect); Wishes(present and past), If only (past), and conditionals (including type 3); Reporting commands, statements, and questions in the present and past; Modals: present and past; Infinitive vs. Gerund; Phrasal verbs with particular focus on the meaning of particles ; Passive with different tenses; Passive with modals; Restrictive and non-restrictive clauses; Linking words. 4.2.3. Rethorical Functions The rhetorical functions to be tested at this level are the following: Defining Cause and effect Purpose Addition Concession.

4.2.4. Language Test Techniques

A variety of test techniques should be included in the test paper. The overuse of some of these techniques at the expense of others must be avoided. Correct form Sentence transformation/ rewriting Sentence completion Gap filling Matching Multiple choice Cloze procedure with list or with first letter given Word definitions Classifying/categorising Providing synonyms, antonyms or poly-words. 4.2.5. Rubrics The rubrics below are to be used when testing grammar, vocabulary and rhetorical functions.
PUT THE VERBS BETWEEN BRACKETS IN THE CORRECT TENSE. FILL IN THE GAP(S) WITH THE APPROPRIATE WORD OR EXPRESSION FROM THE LIST. FILL IN THE GAP WITH AN APPROPRIATE WORD OR EXPRESSION. FILL IN THE GAP WITH AN APPROPRIATE PHRASAL VERB. FILL IN THE GAP WITH THE APPROPRIATE COLLOCATION. MATCH THE WORDS THAT GO TOGETHER TO MAKE APPROPRIATE COLLOCATIONS. REWRITE THE SENTENCES BEGINNING WITH THE WORDS GIVEN. REWRITE THE SENTENCES AS INDICATED. MATCH THE WORDS WITH THE CORRESPONDING DEFINITIONS/ SYNONYMS/ ANTONYMS. REPLACE THE UNDERLINED WORD WITH THE APPROPRIATE SYNONYM. SELECT THE ONE WORD/PHRASE THAT CLOSELY MATCHES THE MEANING OF THE
UNDERLINED WORD.

PROVIDE THE APPROPRIATE WORD(S) FOR THE FOLLOWING DEFINITION. GIVE THE CORRECT FORM OF THE WORDS BETWEEN BRACKETS. JOIN THE PAIRS OF SENTENCES WITH THE LINKING WORDS GIVEN. MATCH EACH SENTENCE WITH ITS APPRORIATE FUNCTION.

4.3.1. Writing performance standards The writing section assesses students writing skill and provides an indication of how well learners communicate in the written language. Testers can determine learners achievement in writing through real-life tasks, bearing in mind the performance standards stated in the syllabus and guidelines: using adequate and relevant content; using an appropriate text structure according to the conventions of the targeted genre ; selecting an appropriate style according to audience; using cohesive devices and transitions to organize content at the level of paragraph(s) and whole text effectively; using mechanics (spelling, punctuation, capitalization) and applying the appropriate structures and vocabulary to get the message across; using a variety of sentence structures: simple, complex sentences, active and passive forms, parallelism, etc.;

The following genres are highly recommended in the testing of writing: formal/ informal letters, film review, book review, narrative, descriptive, expository text, and argumentative texts.

Topics designed for testing writing should be bias-free and should target stimulating issues related to the themes familiar to the students and/or dealt with in the textbooks. Writing tasks should be well defined; i.e. students should know exactly what they are supposed to write. The audience and purpose should also be specified clearly in the task. Additional useful notes could be provided with the writing prompts to help students demonstrate their competence to the fullest.

4.3.2. The writing length The writing task must specify the number of words as indicated in the table below: Arts approximately 250 words Humanities approximately 200 words All science, technical and original streams approximately 150 words

-------------------------------------------------------------------------------------------------------------------------------I was walking around on a Saturday afternoon. It was cold and it was starting to rain. I was looking for a restaurant or a caf where I could sit and have something to drink. When I saw one, I crossed the street and I was there. I sat at a round table and asked for a cup of coffee. While I was waiting for my order, I realised that there were other people in the place, but I sensed loneliness. I saw their bodies, but I couldn't feel their souls. That was because their souls didn't belong to them; instead they belonged to the Internet. I stood up and walked between the tables. When I came to the biggest computer, I saw a thin, small man sitting in front of it. "I'm Steve", he finally answered after I had asked him a couple of times what his name was. "I can't talk with you. I'm busy," he said. I thought he was working, and I apologised. He didn't say anything. Before going back to my seat, I wanted to see what he was doing. I stood behind him. He was not working. He was chatting online with somebody - probably someone he didn't know--and, at the same time, he was playing a computer game a war game. I was astonished. Why didn't Steve want to talk to me? I tried again to communicate with that computer geek, but not a word came out of his mouth. I touched his shoulder, but there was no reaction. I was getting upset. I put my hand in front of the monitor, and he started shouting, "leave me alone!" I took a few steps back. I was ashamed. I wanted to look and see if all those people in the caf were looking at me, so I raised my head, but there was nobody interested in what was happening. I walked back to my table, and I noticed everybody was using a computer. At that moment, I realised that the people in that place were having a cup of coffee and a nice conversation with their machines, not with people. All of them were more interested in having a relationship with the computer, particularly Steve. I wouldn't want to imagine the future of human beings if they preferred sharing their lives with machines instead of people. I was worried and I sank in my thoughts. I didn't even realise that the coffee was bad, just as Steve didn't even realise that there was a person next to him.

I. COMPREHENSION (15 POINTS) BASE YOUR ANSWERS ON THE TEXT

A. CHOOSE THE BEST TITLE FOR THE PASSAGE. TICK THE APPROPRIATE BOX (2 pts )

Computer games Computer programming Computer addiction B. ARE THESE SENTENCES TRUE OR FALSE? JUSTIFY. ( 3 pts ) 1. The writer went to a caf on a sunny day. . . 2. Steve answered the writers question immediately. . . 3. The people in the caf didnt care about what was going on between Steve and the writer. . . C. ANSWER THESE QUESTIONS. (3 pts) 1. Why did the writer apologise? .... . 2. What was Steve busy doing? . . 3. Why did Steve start shouting? .... . D. COMPLETE THESE SENTENCES (3 pts) 1. The writer felt lonely in the caf although... 2. When the writer saw that Steve was too busy, he wanted to ...... 3. The writer cant imagine a future where people .

E. WHAT DO THE UNDERLINED WORDS IN THE TEXT REFER TO? (2 pts) 1. they: .. 2. machines: . F. FIND WORDS IN THE TEXT THAT MEAN THE SAME AS: (2 pts ) 1. surprised (paragraph 2): . 2. computer screen (paragraph 3): .

II. LANGUAGE (15 POINTS)

A. FILL IN THE GAPS WITH THE CORRECT PHRASAL VERBS. (2 pts) set up come back calm down look up fill in apply for I need to .. this application form. I want to .. a visa to Canada. Im going to study management there and hope to .a new business when I to Morocco. B. PUT THE VERBS IN BRACKETS IN THE CORRECT TENSE OR FORM. (3 pts)

Bob and David enjoy (watch) ...science fiction films. Yesterday, they (go) to the cinema, but when they arrived, they found that the film (already/start) . So, they decided to have a walk along the coast.
C. JOIN THE PAIRS OF SENTENCES WITH THE WORDS BETWEEN BRACKETS. (2 pts)

1. Jamal spent all the afternoon chatting. He didnt have time to do his homework. (because) 2. My friends decided to go on an excursion to the Atlas Mountains. They might face difficulties. (despite)
D. REWRITE THE SENTENCES BEGINNING WITH THE WORDS GIVEN. (4pts)

1. What a pity! I didnt send them the message on time. If only 2. Why dont you eat more vegetables and fruit? the doctor said. The doctor advised the patient.. 3. You must send your application before the end of March. Your application .. 4. I dont have a modem, so I cant have access to the Internet. If
E. GIVE THE CORRECT FORM OF THE WORDS BETWEEN BRACKETS. (2 pts; 0.5 pt each)

Active (citizen) .......................... can be defined as the (involve) ....................... of citizens in public life through their (participate) ...................................... in the communities' projects that can (contribution).. to the improvement of people's living conditions.
F. MATCH THE EXPRESSIONS WITH THEIR APPROPRIATE FUNCTIONS. (2 pts) 1. Youd better do some exercises in the workbook. 2. Could you close the window, please? a. requesting b. apologising c. suggesting

III.WRITING (10 POINTS) Write an email to your friend describing the best place you have ever visited. (Approximately 250 words) Here are some cues to help you:

- Name of the place (city, village, countryside, forest, beach, ..)

- Time of the visit (weekend, Sunday, holidays, summer , ) - Characteristics (attractive, nice, interesting, modern, traditional, . ) - Your feelings (happy, satisfied, unforgettable, memorable, . ) ..

APPLICATIONS

10

TASK 1: The text

Consider the appropriateness of the text (for the target level) in the light of the following guidelines:

1. Is the text at an appropriate level for the students (language/concepts)? 2. Is the text culturally biased? Does its cultural load impair comprehension? 3. Does the sampling of the text suit syllabus topics? 4. Is the text similar to the reading material students are familiar with? 5. Does the text lend itself to a sufficient number of comprehension tasks/questions? 6. Does the text allow the assessment of high order comprehension skills/subskills? 7. Is the text well organized, logically sound and factually correct? 8. Does the text require outside information for its full comprehension? 9. Is the text taken for a known/familiar source? 10. Is the text too long/too short?

TASK 2: Comprehension Questions

Consider the appropriateness of the comprehension questions in the light of the following guidelines:

1. Are the objectives behind the questions clear (specification of skills/subskills to assess) ? 2. Can any questions be answered without recourse to the reading passage? 3. Do the questions test significant material in the text? 4. Are the questions written in simple language? 5. Are there questions on every part of the text? 6. Are the questions worded in a language that is similar to that of the text? 7. Do any questions give away the answers to other questions? 8. Are the questions independent? 9. Are the questions varied in type? 10. Are any questions overlapping? 11. Do any questions have more than one possible answer? 12. Are any questions centred on the same information in the text?

TASK 3: LANGUAGE
Consider the appropriateness of the language questions in the light of the following guidelines:

11

1. Do the items assess key elements in the syllabus? 2. Are the stems short/simple/clear and with enough context? 3. Is the language of the stem natural? 4. Is the grammatical structure of the stem familiar to the students? 5. Is the item embedded in a context? 6. Is the context natural and meaningful? 7. Are the content and reliability level of the items suitable for the average student? 8. Are there many possible answers to any items? 9. Is there any mismatch between the way items are tested and the way they have been taught?

TASK4: WRITING
Consider the appropriateness of the writing task in the light of the following guidelines:

1. Is the wording of the topic clear and in simple/familiar language to the student? 2. Is the writing task authentic? 3. Does the task specify the audience/ purpose and text type? 4. Is the input concise and with enough information for the accomplishment of the task? 5. Do students know exactly what is expected of them in terms of content/size? 6. Is the task based on a topic which members of a sex may be favored? 7. Work out a scoring scheme based on the writing task to be performed

Scoring criteria

Scores

12