Review: Writing Good Multiple Choice Test Questions

Aaron Outland
TED 690
6/20/16

Review: Writing Good Multiple Choice Test Questions

Outland 1

Abstract
When developing assessments for students, it is crucial for teachers to construct effective
assessments. It is also crucial that teachers understand the strengths and weaknesses of different
types of assessments. Doing this will allow teachers to properly assess student performance, and
avoid pitfalls associated with different types of assessments. This paper reviews literature from
Vanderbilt Center for Teaching concerning multiple choice test questions. From it I have derived
general rules for writing multiple choice test questions.

Review: Writing Good Multiple Choice Test Questions

Outland 2

This article provides guidelines to educators for constructing an effective stem,

constructing effective alternatives, several additional guidelines for multiple choice questions.
The article indicates that there are three primary strengths of properly constructed multiple
choice assessment: versatility, reliability, and validity.
Strengths and Weaknesses of Multiple Choice
Multiple choice test questions are versatile because they, "can be written to assess various
levels of learning outcomes, from basic recall to application, analysis, and evaluation" (Brame
2016). However, they are limited in their ability to measure many other student skills, such as
organizing thoughts, articulating explanations, or giving creative answers. Multiple choice tests
should not be used to measure these learning outcomes.
They are considered reliable because of the consistency of their measurements. This
reliability is "enhanced when the number of MC items focused on a single learning objective is
increased" (Brame 2016). Their reliability is also increased by the objectivity of scorer
responses.
Validity is the degree to which a test measures the learning outcomes it purports to
measure. Because students can typically answer a multiple choice item much more quickly than
an essay question, tests based on multiple choice items can typically focus on a relatively broad
representation of course material, thus increasing the validity of the assessment.
Constructing an Effective Stem
Stems are the "question" part of multiple choice questions. The stem should have
meaning of its own. For example "Which of the following is a true statement?" has no meaning
on its own, and should be avoided (Brame, 2016). Similarly, stems should be a complete

Review: Writing Good Multiple Choice Test Questions

Outland 3

question or a partial sentence; however, a question is preferential because it has greater internal
meaning.
Effective stems avoid confusing the test taker. The stem should not contain irrelevant
material; this leads to confusion and corruption of the test's reliability. Likewise, stems should
avoid negative statements, since these also lead to confusion. If negative statements are used,
they should be emphasized using capitalization or italics.
Constructing Effective Alternatives
Alternatives are the "answer" part of multiple choice questions. False alternatives should
serve as distracters. Therefore they should all be plausible. Otherwise a test taker will easily
ignore them. Wordy alternatives should be avoided, since they "assess students' reading ability
rather than their attainment of the learning objectives" (Brame, 2016).
Student should not be able to tell the correct answer from reading the alternatives.
Therefore, they should be mutually exclusive, homogenous in content, have grammar consistent
with the stem, parallel form, similar length, and similar language. It is best to present alternatives
in a logical order, such as alphabetical or numerical "to avoid a bias toward certain positions"
that might give hints to the correct answer (Brame, 2016).
Additional Guidelines for Multiple Choice Questions
The article offers additional guidelines for multiple choice questions. Firstly, the number
of alternatives can vary between questions, since "there is little difference in difficulty,
discrimination, and test score reliability among items containing two, three, and four distractors"
(Brame, 2016). Sophisticated test takers can use the internal logic of a complex multiple choice
question to achieve an answer. This should be avoided, since it measures test taking ability and

Review: Writing Good Multiple Choice Test Questions

Outland 4

not achievement of learning goals. Sophisticated test takers can also use information from one
question to answer another, so content should be independent between questions.
Final Take-Aways
Fro this study, I derived about nine separate rules for writing good multiple choice
questions. 1) Multiple choice tests are not able to measure some types of skills, and should only
be used to assess certain levels of learning outcomes. 2) Multiple choice tests provide consistent
results. More questions provide greater reliability of results. 3) Multiple choice questions are
composed of "stems" (questions) and "alternatives" (answers). 4) Multiple choice tests can
assess a broad area of content knowledge quickly. 5) Effective stems should be a question or a
partial sentence with internal meaning. 6) Stems should not contain irrelevant material, and they
should not be phrased negatively. 7) Alternatives should contain homogenous content, and they
should be stated clearly and concisely. 8) Stems should be mutually exclusive, and presented in a
logical order. 9) Avoid complex multiple choice items, including "all of the above" and "none of
the above."

Review: Writing Good Multiple Choice Test Questions

Outland 5

Source
Brame, C. J. (2016). Writing Good Multiple Choice Test Questions. Retrieved June 20, 2016,
from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-testquestions/