FormScanner: Open-Source Solution for Grading Multiple-Choice Exams

Chadwick Young, Glenn Lo, and Kaisa YoungAlberto Borsetta

Citation: Phys. Teach. 54, (2016); doi: 10.1119/1.4937970

View online: http://dx.doi.org/10.1119/1.4937970
View Table of Contents: http://aapt.scitation.org/toc/pte/54/1
Published by the American Association of Physics Teachers
FormScanner: Open-Source Solution
for Grading Multiple-Choice Exams
Chadwick Young, Glenn Lo, and Kaisa Young, Nicholls State University, Thibodaux, LA
Alberto Borsetta, Independent Software Developer

T
he multiple-choice exam remains a staple for many dows or Macintosh computers; very little technical knowl-
introductory physics courses.1 In the past, people edge is required to use the software. Among the open-source
have graded these by hand or even flaming needles.2 OMR packages available, it is the most effective and user
Today, one usually grades the exams with a form scanner that friendly. The FormScanner software and many user resources
utilizes optical mark recognition (OMR). Several companies are available at this website: http://www.formscanner.org.
provide these scanners and particular forms, such as the
eponymous Scantron. OMR scanners combine hardware Create the form
and softwarea scanner and OMR programto read and Users create the forms for FormScanner and print them on
grade student-filled forms. plain paper. The bubbles on the forms must be evenly spaced,
Many schools are now up- and the form may have one or
grading their photocopiers more columns. Responses can
to newer models that include also be listed in a grid with rows
multi-page scanners, which can and columns for a single question.
scan hundreds of documents FormScanner only requires the
in just a few moments. The form to have black rings in each
scanned images can be saved of the page corners (see Fig. 1).
as a portable document format Several forms are available on the
(PDF) file or some other image website. Users can also submit
format (JPG, PNG, BMP, etc.). their forms for others to use. An
The photocopiers fill the same Excel template is also available, so
role as the hardware compo- users can customize forms.
nent of the OMR scanners.
Some companies now offer Scan the forms
standalone OMR software for The first step in grading the
use with scanners. The packag- forms is to scan them with either
es take the scanned images and a single- or multi-page scanner.
write the students bubbled If the scanner creates a PDF file,
responses to a spreadsheet. the user should convert the pages
Some of these packages are free. of the PDF to individual image
For example, Moodle has of- files. XnConvert is the preferred
fered an OMR extension with converter package, and detailed
its popular course management instructions for its use are on the
Fig. 1. The form can have multiple columns, areas for written
software, but these packages are responses, or other personalization. The four dark rings allow website.
often difficult to install and use. FormScanner to define the region to be scanned.
Other packages, such as Ricohs FormScanner
Remark Office OMR, are user The user selects one of the
friendly and effective but, cost- images and uses it to create a
ing thousands of dollars, pro- template for scoring the forms.
hibitively expensive. The inputs include the number of
FormScanner is a new, questions and the number of pos-
open-source, and free OMR sible responses for each question.
software package. FormScan- Then, with the mouse, the user
ner reads scanned images selects the positions of the first
of a user-created form and and last questions; FormScanner
produces a spreadsheet with interpolates the positions for all of
each students responses to the the responses. FormScanner has
questions on the form. It is Java Fig. 2. FormScanner detects the students filled bubbles and a couple of other parameters
based and can be used on Win- writes the results to a file. threshold and density describe

34 The Physics Teacher Vol. 54, January 2016 DOI: 10.1119/1.4937970

which marks the software interprets as responsesbut the
user need not adjust them. The users guide and video tutori-
als on the website explore the procedure in greater detail.
After saving the template, the user opens the images of the
forms with FormScanner. The software can either process
the images all at once and immediately write the results to a
spreadsheet or allow the user to inspect each image with its
assigned responses. Figure 2 shows the results; the red dots
show where FormScanner has assigned a student response.
FormScanner writes the results to a comma-separated text
file that can be opened in any spreadsheet software.
Further, FormScanner has some excellent features. The
software can read and interpret QR or barcodes, for those ex-
ams that need these identifying marks. It can be run from the
command line, for batch processing of many forms. Form-
Scanner can also interpret a grid of responseswhen the op-
tions are in both rows and columns; this feature is useful for
presenting surveys. Finally, FormScanner does allow multiple
responses, so questions with several answers are possible.
The software greatly exceeds the capability of most schools
OMR scanners.
Analysis of grades
FormScanner reports students responses for each ques-
tion, so instructors can complete detailed item analysis.3 A
spreadsheet macro, ProcessBubbles (available on the web-
site), takes the output of FormScanner and the instructor-
provided key. The macro sorts the results by student ID and
produces a detailed analysis of the results that includes item
facility, quartiles, averages, and discrimination for each ques-
tion. The spreadsheet can also account for partial credit when
multiple responses are allowed and gives the option to deduct
points when students answer incorrectly.
Advantages/disadvantages
The instructor reaps many advantages. First, FormScanner
provides item analysisthe ability to record which students
missed which questionsthat is often not available on many
basic OMR scanners. This ability can be very useful for ana-
lyzing assessment results.4 Second, FormScanner reduces the
amount of time inputting grades. In five minutes or less, the
instructor has each students grade organized by name and
identification number. Third, the students do not have to
purchase or remember to bring forms; the forms can be sta-
pled to the exam. Fourth, the instructor has greater flexibility
in creating the forms. One may include honor statements,
equations, or free-response questions on the form. Next, the
student may use any writing instrument; #2 pencils are not
required.
The Physics Teacher Vol. 54, January 2016
Further, FormScanner is open source and free. The open-
source nature of FormScanner is crucial to its success. By
inviting collaboration in its development, Alberto Borsetta,
who is the primary developer, allows others to innovate and
share with the community. Because FormScanner is without
cost, users who might not otherwise be able to purchase such
software can use it freely. Particularly, this is important in the
education sector, where funding is sometimes deficient, but
FormScanner also fills an important role in the civic func-
tionssuch as elections and pollsof poor and underdevel-
oped nations. The authors and, particularly, Alberto Borsetta
stand behind the open-source nature of FormScanner.
The disadvantages are less numerous. FormScanner does
require some mastery of new software packages; new users
should expect to spend about 90 minutes installing software
and learning the process. Once instructors have completed
the process a couple of times, they achieve mastery. Next, cor-
rect answers are not provided on the form; however, this dis-
advantage forces the students to review their answers more
thoroughly. Also, as with any OMR scanners, FormScanner
might not always recognize partially filled bubbles. However,
FormScanner does show great reliability; after scanning over
a thousand exams during the past academic year, FormScan-
ner produced less than a half-dozen errors due to improper
marking or poor erasures. Finally, the most frustrating draw-
back: the instructor does not get to chat with others while
waiting to scan his or her forms.
References
1. S-Y. Lin and C. Singh, Can free-response questions be ap-
proximated by multiple-choice equivalents? Am. J. Phys. 81,
624 (Aug. 2013).
2. Sister M. Richard, Apparatus for Teaching Physics: Grading
multiple choice examinations, Phys. Teach. 2, 228 (May 1964).
3. A. Hobson and D. Ghoshal, Flexible scoring for multiple-
choice exams, Phys. Teach. 34, 284 (May 1996).
4. D. Hestenes, M. Wells, and G. Swackhamer, Force Concept
Inventory, Phys. Teach. 30, 141158 (March 1992).
Chad Young is an astronomer and an associate professor of physics at
Nicholls State University.
chad.young@nicholls.edu
Alberto Borsetta is a software engineer at a leading Italian soft-
ware house. He develops open source software and is the creator of
FormScanner.
Glenn Lo is a physical chemist and professor of chemistry at Nicholls
State University.
Kaisa Young is an astronomer and an assistant professor of physics at
Nicholls State University.
35