EVALUATION

Test Item Construction

Essay Questions

Sylvester Saimon Simin

Keningau Teachers Training College
 Characteristics of Essay Questions
• Freedom of response
– As with short-answer items, students produce their own
answers. However, they are free to decide how to approach
the problem, what factual information to use, how to
organize the answer, and what degree of emphasis to give
each aspect of the response. Thus it is useful for measuring
the ability to organize, integrate, and express ideas.
 Comparison of Selection-Type Items and Essay
Selection-Type Items Essay Questions
• Good for measuring outcomes at • Inefficient for measuring
Learning the knowledge, comprehension, knowledge outcomes
outcomes and application levels • Best for ability to organize,
measured • Inadequate for organizing and integrate and express ideas.
expressing ideas
The use of a large number of The use of a small number of items
Sampling of items results in broad coverage, limits coverage, which makes
content which makes representative representative sampling of content
sampling of content feasible infeasible.
Preparation of good items is Preparation of good items is
Preparation
difficult and time consuming. difficult but easier than selection-
of Items
type items.
Objective, simple, and highly Subjective, difficult and less
Scoring
reliable. reliable.
Factors Reading ability and guessing. Writing ability and bluffing.
distorting
scores
Probable Encourages students to Encourages students to organize,
effect on remember, interpret, and use the integrate, and express their own
learning ideas of others. ideas.
 Types of Essay Questions
Restricted-Response Questions
– Places strict limits on the answer; the subject matter are narrowly
defined by the problem, the specific form of the answer is commonly
indicated (eg: ‘list’, ‘define’, ‘give reasons’), use of introductory
material or special directions
• Eg: Describe the relative merits of selection-type tests items and essay
questions for measuring learning outcomes at the comprehension level.
Confine your answer to one page.
• Eg: Mr. Roger, a Form 1 science teacher, wants to measure his students’
‘ability to interpret scientific data’ with a paper-and-pencil test.
(a) Describe the steps that Mr. Roger should follow.
(b) Give reasons to justify each step.
– Advantages:
• Prepared more easily, related more directly to specific learning outcomes,
scored more easily
– Disadvantages
• Little opportunities for students to demonstrate their abilities to organize, to
integrate and develop new pattern of response
– Useful for comprehension, application and analysis levels of learning
 Types of Essay Questions
Extended-Response Questions
– Almost unlimited freedom to determine the form and scope of their
responses.
– In some instances practical limits are imposed such as time or page
limits, but restrictions on the material to be included in the answer
and on the form of response are held to a minimum.
• Eg: For a course that you are teaching or expect to teach, prepare a
complete plan for assessing student acheivement. Be sure to include the
procedures you would follow, the instruments you would use, and the
reasons for your choices. (synthesis outcome)
• Eg: (The student is given a complete acheivement test that includes errors
or flaws in the directions, in the test items, and in the arrangement of the
items.) Write a critical evaluation of this test using as evaluative criteria the
rules and standards for test construction described in your textbook.
Include a detailed analysis of the test’s strengths and weaknesses and an
evaluation of its overall quality and probable effectiveness. (Evaluation
outcome)
– Advantages:
• Creative integration of ideas, overall evaluation of materials, and a broad
approach to problem solving.
– Disadvantages
• To evaluate the answers with sufficient reliability to provide a useful
measure of learning
– Useful for synthesis and evaluation levels of learning
 Guidelines for Writing Essay Questions

1. Use essay questions to measure complex learning

outcomes only.
– Essay test are suitable for situations that require students
to give reasons, explain relationships, describe data,
formulate conclusions.
– Where supplying the answer is vital, a properly constructed
restricted-response question is more appropriate
– At the synthesis & evaluation level, such as the production
of a complete work (eg. a plan of operation) or an overall
evaluation of a work (eg. evaluation of a novel or an
experiment) requires the use of extended-response
questions.
 Guidelines for Writing Essay Questions
2. Relate the questions as directly as possible to the
learning outcomes being measured.
– Each question should be specifically designed to measure
one or more well-defined outcomes. Thus, as with objective
items, a precise description of the performance to be
measured will help determine the content and form of the
item.
– The extended-response item requires greater freedom of
response and involves a number of learning outcomes. If
the task is prescribed too rigidly in the question, the
student’s freedom is apt to be infringed upon. One solution
is to indicate the criteria used in evaluating the answer eg.
‘your answer will be evaluated in terms of its
comprehensiveness, the relevance of its arguments, the
appropriateness of its examples, and the skill with which it
is organized’.
 Guidelines for Writing Essay Questions

3. Formulate questions that present a clear task to be performed.

Learning
Sample Terms
Outcomes
Comparing compare, classify, describe, distinguish between, explain, outline,
summarize
Interpreting convert, draw, estimates, illustrate, interpret, restate, summarize,
translate
Inferring derive, draw, estimate, extend, extrapolate, predict, propose, relate
Applying arrange, compute, describe, demonstrate, illustrate, rearrange, relate
summarize
Analyzing break down, describe, diagram, differentiate, divide, list, outline,
separate
Creating compose, design, devise, draw, formulate, make up, present, propose
Synthesizing arrange, combine, construct, design, rearrange, regroup, relate, write
Generalizing construct, develop, explain, formulate, generate, make, propose, state
Evaluating appraise, criticize, defend, describe, evaluate, explain, judge, write

Types of Complex Outcomes and Related Terms for Writing Essay Questions
 Guidelines for Writing Essay Questions

4. Do not permit a choice of questions

unless the learning outcome requires
it.
 Guidelines for Writing Essay Questions

5. Provide ample time for answering and

suggest a time limit on each question.
 Rules for Scoring Essay Questions
1. Evaluate answers to essay questions in terms of
the learning outcomes being measured.
2. Score restricted-response answers by the point
method, using a model answer as a guide.
3. Grade extended-response answers by the rating
method, using defined criteria as a guide.
4. Evaluate all of the students’ answers to one
question before proceeding to the next question.
5. Evaluate answers to essay questions without
knowing the identity of the writer.
6. Whenever possible, have two or more persons
grade each answer.
 Checklist for Evaluating Essay Questions
1. Is this type of item appropriate for measuring the
intended learning outcome?
2. Does the item task match the learning task to be
measured?
3. Is the question designed to measure complex learning
outcomes?
4. Does the question make clear what is being measured
and how the answer will be evaluated?
5. Has terminology been used that clarifies and limits the
task (eg: ‘describe’, not ‘discuss’)
6. Are all students required to answer the same questions?
7. Has an ample time limit been indicated for each
question?
8. Have adequate provisions been made for scoring
answers (eg: model answers or criteria for evaluating)?
 Common Types of Bluffing in Answering Essay Questions
1. Student repeat the question in statement form (slightly
paraphrased) and tells how important the topic is (e.g., “The
role of assessment I teaching is extremely important. It is hard
to imagine effective instruction witout it, etc.”).
2. Student writes on a well-known topic and fits it to the question
(e.g., a student who knows testing well but knows little about
performance assessment and is asked to compare the two
might describe testing in considerable detail and frequently
state that performance assessment is much superior for
evaluating the type of learning measured by the test).
3. Student liberally sprinkles the answer with basic concepts
whether they are understood or not (e.g., asked to write about
any assessment techniques the importance of ‘validity’ and
‘reliability’ is mentioned frequently).
4. Student includes the teacher’s basic beliefs wherever possible
(e.g., “The intended learning outcomes must be stated in
performance terms before this type of test is constructed or
selected”).
* Bluffing is most effective where plans have not been made for careful scoring of the
answers.