Académique Documents
Professionnel Documents
Culture Documents
ASSESSMENT IN EDUCATION
1.0
INTRODUCTION
The item analysis is an important phase in the development of an exam program. In this phase statistical methods are used to identify any test items that are not working well. If an item is too easy, too difficult, failing to show a difference between skilled and unskilled examinees, or even scored incorrectly, an item analysis will reveal it. The two most common statistics reported in an item analysis are the item difficulty, which is a measure of the proportion of examinees who responded to an item correctly, and the item discrimination, which is a measure of how well the item discriminates between examinees who are knowledgeable in the content area and those who are not. An additional analysis that is often reported is the distractor analysis. The distractor analysis provides a measure of how well each of the incorrect options contributes to the quality of a multiple choice item. Once the item analysis information is available, an item review is often conducted.
Item analysis is a method that is used in education to evaluate test item. This can ensure that questions are in appropriate standard and measure the effectiveness of individual test item. Item analysis is purposed to improve test items and identify unfair or biased item. Item analysis is conducted to avoid the chance of tests that are too difficult (and have an insufficient floor) tend to lead to frustration and lead to deflated scores, and tests that are too easy (and have an insufficient ceiling) facilitate a decline in motivation and lead to inflated scores. Item analysis can be analyzed by computing: Difficulty Index, Discrimination Index, Validity Coefficient, and Effectiveness of Distraction.
1.1
Difficulty Index
According to Wilson (2005), item difficulty is the most essential component of item analysis. Item difficulty is determined by the number of people who answer a particular test item correctly. It is important for a test to contain items of various difficulty levels in order to distinguish between students who are not prepared at all, students who are fairly prepared, and students who are well prepared.
UNITAR International University
EPM5153
ASSESSMENT IN EDUCATION
To compute level of difficulty we use the formula: Difficulty Index (p) = C/T p = Difficulty Index C = the number of students who answer item X correctly T = the number of total students who answer item X For example: There are 12 students who answer an item X, 4 of whom can answer the item correctly. So that, the level of difficulty is: Level of difficulty (p) = 4 / 12 = 0.33
The highest score for p is 1.0 and the lowest score is 0 p always has positive value The higher score for p the easier item The lower score for p the harder item
According to Allen & Yen (1986) to avoid a test is too difficult or too easy, each items in a test should have a difficulty range from 0.3 to 0.7. It is used to differentiate between individuals level of knowledge, ability, and preparedness.
1.2
Discrimination Index
EPM5153
ASSESSMENT IN EDUCATION
Discrimination goes beyond determining the proportion of people who answer correctly and looks more specifically at who answers correctly. In other words, item discrimination determines whether those who did well on the entire test did well on a particular item. An item should in fact be able to discriminate between upper and lower scoring groups. To compute discrimination index, the first thing we have to do is dividing test-takers into three groups, upper group, average group and lower group. Then using this formula: Discrimination index (D) = CU CL (1/2)TUL D = Discrimination index Cu = Number of Upper 25% students who answered the item correctly CL = Number of Lower 25% students who answered the item correctly
1.3
Multiple choice tests have one question and several options of answer. Among the options, there is only one answer which is correct, and the other options are wrong answers. Those wrong answers in multiple choice tests are called distraction. A distraction is called effective when there are a lot of students choosing it. According to Fernandez (1984) a distraction can be called as a good distraction when there are 2% of test takers choosing it.
1.4
Summary
In the item analysis phase of test development, statistical methods are used to identify potential item problems. The statistical results should be used along with substantive attention to the item content to determine if a problem exists and what should be done to correct it. Items that are functioning very poorly should usually be removed from consideration and the exams rescored
EPM5153
ASSESSMENT IN EDUCATION
before the test results are released. In other cases, items may still be usable, after modest changes are made to improve their performance on future exams.
2.0
EPM5153
ASSESSMENT IN EDUCATION
SUBJECT THEME TOPIC CLASS AGE NUMBER OF STUDENTS STUDENTS ABILITY LEARNING OUTCOMES
English Language World of knowledge Family 5 Perdana 11 years old 12 students ( 6 boys and 6 girls) Mixed Ability At the end of this topic, students will be able to : 1 2 3 Answer the comprehension questions based on the text given. Complete the sentences with the suitable words. Write an essay using the word provided.
LEARNING SKILLS
3.8 Read and understand simple factual texts for main ideas, supporting details, sequence and cause and effects. 4.3 4.4 Complete texts with the missing word, phrase or sentences. Construct simple and compound sentences with guidance and independently.
EPM5153
ASSESSMENT IN EDUCATION
2.2
Each question in this paper is followed by four possible answers : A, B, C and D. For each question, choose only one answer.
EPM5153
ASSESSMENT IN EDUCATION
1. Pak Kadir uses a _______________ to cut the long grass in his garden. A hoe B rake C sickle D wheelbarrow Answer : C (Level of knowledge : Knowledge domain, level 1)
2. The goat and its ____________ are walking to the field. A kids B cubs C foals D calves Answer : A (Level of knowledge : Knowledge domain, level 1)
3. Puan Ani gives a sharp knife to the maid and keeps the _______ one in the drawer. A fresh B blunt C neat D new Answer : B (Level of knowledge : Knowledge domain, level 1)
EPM5153
ASSESSMENT IN EDUCATION
4. _______________ Rahman ____________ Meng Choo came to the meeting yesterday. A If .. so B Either . or C Neither nor D Although . Because Answer : C (Level of knowledge : Knowledge domain, level 2)
Choose the sentence with the correct punctuation. 5. A Mei Ling and her parents are going to singapore. B Mei Ling and her Parents are going to Singapore. C Mei Ling and her parents are going to Singapore. D Mei Ling and her parents, are going to Singapore. Answer : C (Level of knowledge : Analysis domain, level 2)
Questions 6-10 Read the dialogue below carefully and answer the questions that follow. Eiman Grandmother Anyss Grandmother Anyss Grandmother Eiman : Grandma, where does Uncle Latif live ? : He is in Australia with his family. : Why do they live so far away ? : Thats because he is married to an Australian who owns a restaurant there. : How many children do they have ? : Three. So, you have three cousins who live far away. : What about Aunt Maznah ?
8
EPM5153
ASSESSMENT IN EDUCATION
Grandmother Anyss
: She works for an oil company in Miri and lives there with her family. : I hope Aunt Maznah wont leave us.Grandmother : I hope so too, but she has just got a job in Kuching after completing her studies at University Malaysia Sarawak.
: I want to study at a university too. I want to be a scientist. : I want to be an English teacher. : Thats good. If both of you work hard, you will be successful. : Thank you, Grandma. We will study very hard and we wont disappoint you.
6.
How many children does grandmother have ? A Two B Three C Four D Five Answer : B (Level of knowledge : Analysis domain, level 2)
7. Who is grandmother talking to ? A Her grandchildren B Her children C Her parents D Her cousins Answer : A (Level of knowledge : Comprehension domain, level 2)
EPM5153
ASSESSMENT IN EDUCATION
9. The word disappoint in the dialogue means A feels proud B be a loving person C happy and enjoyable D makes someone feels sad Answer : D (Level of knowledge : Comprehension domain, level 2)
10. Eiman wants to be a scientist because he wants to A work in an oil company. B study at a university. C invent new things. D live in Australia Answer : D (Level of knowledge : Comprehension domain, level 1)
10
EPM5153
ASSESSMENT IN EDUCATION
2.3
Question 1
Distracter Index
= ___2__ = 2 = 0.67 (1/2) (6) 3 Comments Option A is a better distracter The item is moderately difficult. The item has low discrimination.
11
EPM5153
ASSESSMENT IN EDUCATION
Question 1
Question 2
Distracter Index
Comments
12
EPM5153
ASSESSMENT IN EDUCATION
Option D has very low distraction but is a better distracter compared to B and C
Question 3
Distracter Index
Comments
13
EPM5153
ASSESSMENT IN EDUCATION
Students Upper 25% Average 50% Lower 25% Total Computation Comments 3 6 3 12 B* 3 1 0 4 A 0 0 0 0 0.00
Index 0.00 0.19 0.20 0.39 0.40 0.59 0.60 0.79 0.80 1.00
Interpretation Very low distraction Low distraction Moderate distraction High distraction Very high distraction
Option D has very high distraction but is better distracter compared A and C..
Question 4
Distracter Index
Comments
14
EPM5153
ASSESSMENT IN EDUCATION
Students Upper 25% Average 50% Lower 25% Total Computation Comments 3 6 3 12 C* 2 0 0 2 A 0 0 0 0 0.00
Index 0.00 0.19 0.20 0.39 0.40 0.59 0.60 0.79 0.80 1.00
Interpretation Very low distraction Low distraction Moderate distraction High distraction Very high distraction
Option C has very high distraction but is a better distracter compared to A and D
Question 5
Distracter Index
Comments
15
EPM5153
ASSESSMENT IN EDUCATION
Students Upper 25% Average 50% Lower 25% Total Computation Comments 3 6 3 12 C* 2 2 1 5 A 1 1 0 2 0.17
Index 0.00 0.19 0.20 0.39 0.40 0.59 0.60 0.79 0.80 1.00
Interpretation Very low distraction Low distraction Moderate distraction High distraction Very high distraction
Question 6
Distracter Index
Comments
Options
Index
Interpretation
16
EPM5153
ASSESSMENT IN EDUCATION
3 6 3 12
B* 1 0 0 1
A 2 6 3 11 0.92
C 0 0 0 0 0.00
D 0 0 0 0 0.00
0.00 0.19 0.20 0.39 0.40 0.59 0.60 0.79 0.80 1.00
Very low distraction Low distraction Moderate distraction High distraction Very high distraction
Question 7
Distracter Index
Comments
17
EPM5153
ASSESSMENT IN EDUCATION
3 6 3 12
3 4 1 8
0 2 2 4 0.33
0 0 0 0 0.00
0 0 0 0 0.00
0.00 0.19 0.20 0.39 0.40 0.59 0.60 0.79 0.80 1.00
Very low distraction Low distraction Moderate distraction High distraction Very high distraction
Question 8
Distracter Index
Comments
Question 8
Options A B D
Index
Interpretation
18
EPM5153
ASSESSMENT IN EDUCATION
3 6 3 12
2 2 0 4
0 0 1 1 0.08
1 4 2 7 0.58
0 0 0 0 0.00
0.00 0.19 0.20 0.39 0.40 0.59 0.60 0.79 0.80 1.00
Very low distraction Low distraction Moderate distraction High distraction Very high distraction
Question 9
Distracter Index
Comments
19
EPM5153
ASSESSMENT IN EDUCATION
Question 10
Distracter Index
Comments
20
EPM5153
ASSESSMENT IN EDUCATION
Comments
3.0 3.1
You are advised to spend about 35 minutes on this question. Rahmans hobby is collecting butterflies. With the help of the pictures and notes given below, write a composition in not less than 100 words.
catch butterfly own garden use butterfly net long stick net attached
use net catch butterfly handle gently not tear out wings
pour drop methylated spirit onto head preserve insect dry pin onto board (40 marks)
____________________________________________________________________________ ____________________________________________________________________________
UNITAR International University
21
EPM5153
ASSESSMENT IN EDUCATION
3.2
Subjective answer
Rahman has an unusual hobby. It is collecting butterflies. He catches the butterflies in his own garden which is full of beautiful flowers. In order to catch the butterflies, Rahman uses a butterfly net. He uses a long stick with a net attached to one end. First, Rahman catches a butterfly with his net. He must handle the butterfly gently so that he does not tear out its wings. Then, he pours a small drop of methylated spirit onto the butterflys head. This is to preserve the insect. When the butterfly is dry, he pins it onto a board. Rahman arranges the butterflies according to types, colours and sizes. He enjoys collecting butterflies because it is fun and is also a cheap hobby.
22
EPM5153
ASSESSMENT IN EDUCATION
4.0
NO. STUDENTS' NAME 1 Andrea Helena Ejin 2 Marytheresa Raj 3 Ezron Zabion 4 Jerrel Jammy 5 Daphne Sharlene Albert 6 Eldressa Peter 7 8 9 10 Marcella Bansing Allyana Hamzah Aizzat Mohd Arif Beyreand John
MARKS 90 85 80 75 70 68 68 60 60 58 45 45
GRED REPORTS A Outstanding performance A A B B B B B B C D D Very good performance Very good performance Satisfactory performance Satisfactory performance Satisfactory performance Satisfactory performance Satisfactory performance Satisfactory performance Weak performance Very weak performance Very weak performance
COMMENTS Keep up the good work Need to improve a little bit more Need to improve a little bit more Need to work harder to improve Need to work harder to improve Need to work harder to improve Need to work harder to improve Need to do more exercises Need to do more exercises Need to do more exercises Need to attend extra classes Need to attend extra classes
23
EPM5153
ASSESSMENT IN EDUCATION
5.0
CONCLUSION
Evaluation / testing requires some systematic process from beginning to end. With this systematic steps will the formation of a set of quality tests. This is due to give the test is part of the teaching and learning process. In conducting the test, the teacher must identify the questions are constructed to meet all the objectives of teaching and learning that have been implemented.
If you look at the results of Difficulty Index and Discrimination Index of paperwork we provide this course, the items / questions are constructed is good and suitable for use by students for testing. This is because both the index does not show any difficulty / extreme differences.
Therefore, every teacher must have sufficient knowledge in the construction field test the pupils / students can master the skills learned to be more effective, and the items were appropriate and challenge the ability of the student to be able to grow their thinking.
24
EPM5153
ASSESSMENT IN EDUCATION
REFERENCES Anderson, L. W., & Krathwohl, D. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives . New York: Longman. Clark, D. (2010). Bloom's taxonomy of learning domains: The three types of learning. Big Dog & Little Dog's Performance Juxtaposition . Edmonds, WA: Author. from http://www.nwlink.com/~donclark/hrd/bloom.html Forehand, M. (2005). Bloom's taxonomy: Original and revised.. In M. Orey (Ed.), Emerging perspectives on learning, teaching, and technology . Retrieved from http://projects.coe.uga.edu/epltt Huraian Sukatan Pelajaran (2003). English Language Year 5. Kuala Lumpur : Pusat Perkembangan Kurikulum. Lian, C. (2011). Temanku Ceria Bahasa Inggeris Year 5, Grammar and Writing. Selangor : InfoDidik Mertler, Craig A. (2001). Designing scoring rubrics for your classroom. Practical Assessment, Research & Evaluation, Moskal, Barbara M. (2000). Scoring rubrics: what, when and how? Practical Assessment, Research & Evaluation Yap, J (2011). Zoom In Topical Drill English Year 5. Petaling Jaya : Setia Emas Retrieved
25
EPM5153
ASSESSMENT IN EDUCATION
APPENDIX 1
Holistic Rubric
Grade A (90-100) Score Criteria The A argument essay is exceptional in every way. The essay is well organized and all claims are supported. It begins with a solid introduction that contains a clear thesis, is followed by body paragraphs that contain clear topic sentences with clear and detailed support, and ends with an effective conclusion. Content is thorough and lacking in no area. There are no (or few) errors in tone, format, mechanics, grammar, and content. B (80-89) The B essay is above adequate in most areas. In the areas where it is not above adequate, it is still entirely acceptable. The majority of the essay is clear, focused, and well detailed, but there may be a few areas requiring further development. While it may contain a few errors with tone, mechanics, grammar, and/or content, these errors are not egregious enough to detract from the overall point being made. C (70-79) The C essay is adequate in most areas, but exceptional in none. The thesis is clear although probably lacking in both control and command. Organization may be a slight problem but can be fixed. The paragraphs provide support but are generally underdeveloped. There may be multiple errors in tone, format, mechanics, grammar, and content, but these errors do not, for the most part, detract from the overall writing. D (60-69) The D essay is lacking in a majority of areas. It is generally unorganized and unfocused. The thesis is neither clear nor controls the entire essay. Most of the essay is underdeveloped. There are frequent errors in tone, format, mechanics, grammar, and/or content that distract from the content being provided. Its only saving grace is that, despite all of the errors, there appears to be a legitimate effort put forth by the writer. F (0-59) The F essay generally needs little explanation. There are significant problems throughout. The thesis is often lacking, and the argument, if there is one, wanders and is unorganized. The essay shows no understanding of basic essay structure, and there are significant errors in tone, format, mechanics, grammar, and/or content. The effort on the part of the writer is questionable, at best.
26
EPM5153
ASSESSMENT IN EDUCATION
27