CIA: A Computer Program for Classical Item Analysis

Seock-Ho Kim
The University of Georgia
March 19, 1999
Copyright (c) 1999 by Seock-Ho Kim. This computer program and the
manual may be reproduced for noncommercial use without the author's
permission.
1. Introduction
The computer program CIA provides results of classical item
analysis. The classical item statistics can be obtained using CIA
are as follows:
(1) item difficulty
(2) item discrimination indices; the point biserial and
biserial correlations
(3) proportions, point biserial and biserial correlations for
distractors
CIA can also provide summary statistics of test scores, the
alpha reliability estimate and the standard error of measurement as
well as the quintile information. Please note that this manual
does not contain equations or descriptions of item statistics and
other statistics or estimates. There are many measurement texts
that contain the equations of the statistics and methods used in
this manual. Henrysson (1971) presented an extensive discussion of
the item statistics. Also Allan and Yen (1979), Crocker and Algina
(1986), Gulliksen (1987), Lord and Novick (1968), and Magnusson
(1967) contain descriptions and equations of nearly all statistics
and methods used in the program. For quintile information, see
Coffman (1971) and Wainer (1989). For the entry level introduction
to item analysis, refer to Payne (1997), Sax (1997), and Thorndike
(1997).
It is assumed that the user has a data file. The data file
contains basically an identification variable and a set of item
responses from a test with multiple-choice or true-false (i.e.,
dichotomously scored) items. The item responses can be 1 to 9 or
A to I that depends on the total number of choices. Note that the
lower characters (e.g., a to i) cannot be used. The total possible
number of choice of an item must be less than or equal to nine.
Each item can have a different number of choices. Before using the
data, it is recommended that a blank response be classified as
either the omitted response or the unreached response, and be
recoded using respective alphabet characters (e.g., O for the
omitted response and U for the unreached response). All
other unspecified responses will be treated as the invalid
responses.
The correlation indices of item discrimination can be
corrected for the spuriousness. In addition, CIA computes
optionally the quintile information. The user is queried by the
program to ascertain the availability of these options.

The program was written in IBM Professional FORTRAN for the

IBM personal computers. It was written in a DOS mode. The program
runs in an interactive mode with the user answering a series of
questions and providing the necessary information. The set of
responses can also be put in a batch file and executed via the
redirect feature of DOS.
The distributed diskette contains the following seven files:
CIA.FOR (source code), CIA.OBJ (object code), CIA.EXE (executable
code), CIA.TEX (this manual in LaTeX), CIA.TXT (this manual in
ASCII), SAMPLE.INP (input stream), SAMPLE.DAT (data file),
SAMPLE.OUT (output file), and SAMPLE.SCO (score file).
2. Using the Program
The computer generated prompts and responses the user may give
are indicated in this section by the UPPER characters. No default
values are assumed. Some acceptable responses are given in the
parentheses following the questions. The program is executed by
typing CIA and the following occurs:
TYPE A TITLE FOR THE COMPUTER RUN:
You may enter a title that is less than 80 characters in
length.
ENTER THE NUMBER OF ITEMS:
The maximum number of items is 200.
TYPE A CHARACTER FOR THE OMITTED ITEM:
The character O or other characters can be used.
TYPE A CHARACTER FOR THE UNREACHED ITEM:
The character U or other characters can be used.
ENTER THE NUMBER OF ID CHARACTERS:
The total number of characters used for identification (ID) is
entered as an integer number. The number must be less than
200.
ENTER THE NUMBER OF EXAMINEES:
The maximum number of examinees is 10,000.
NEED NON-SPURIOUS CORRELATIONS? (Y/N):
To correct spurious correlation indices of item
discrimination. The input is either Y (i.e., Yes) or N
(i.e., No).
NEED QUINTILE GROUP RESULTS? (Y/N):
If you need quintile information, you should type {\tt Y}.
When you have a reasonably large data set, this option will be
very informative.
ENTER THE NAME OF THE DATA FILE:
The name of the data file containing the raw item responses
should be entered. The file should be in ASCII. Use a
standard DOS file name.
ENTER THE FORMAT OF THE DATA:
The first part contains the format of ID, and the second part
contains the FORTRAN format of the item responses in the data
file. The total length of the format statement should be less
than 80 characters.
ENTER THE KEY OF ITEM ITEMS:
You should type in Alphabet characters (e.g., A to I) or
numbers (e.g., 1 to 9) for correct choices for all items

sequentially. The total number of the keys should be the same

as the total number of items that you have specified earlier.
ENTER THE NUMBERS OF CHOICES:
The numbers of the choices of the items should be specified
sequentially. Note that each item may have a different number
of choices.
ENTER THE NAME OF THE OUTPUT FILE:
The name of the output file that will contain the item
analysis results should be specified.
ENTER THE NAME OF THE SCORE FILE:
The file that contains the scored results of examinees should
be specified.
The amount of time these calculations take for a given
computer system depends upon the number of items and the particular
options are employed. The sample run presented in the next section
using a data set of 14 items and 765 examinees took less than three
seconds on a desktop computer with a 90 Mhz Pentium chip.
3. Sample Run of the Program
3.1. Sample Data
This section provides a sample run of the CIA program. The
sample data provided in the distribution diskette are used for the
sample run. The followings the first ten examinees responses:
ANDY
CATHERIN
MINDY
TOBY
ERIC
VINCE
AARON
AMY
RICHARD
JENNIFER

22341253153121
32332542232544
51341413125135
22452551222135
22521432524145
22341252151243
22543413353111
51133511421341
22243442323131
22142122133145

Overall there are 765 examinees in the file. The first eight
characters are examinees first names. These are used as the
identification (ID) variable. There are two blank columns between
ID and the actual item responses. There are 14 items and each item
have five choices. Please note that the blank responses appeared
in the data file have been replaced by O and U. All other illegal
responses will be treated as the invalid responses. If you have
some blank or other responses, but do not want to differentiate
them as the omitted, unreached, and invalid responses, you may
analyze them as invalid responses.

3.2. Sample Run

Execution begins by typing CIA. The following contain all
responses to the questions to analyze the sample data:
TYPE A TITLE FOR THE COMPUTER RUN:
SAMPLE RUN
ENTER THE NUMBER OF ITEMS:
14
TYPE A CHARACTER FOR THE OMITTED ITEM:
O
TYPE A CHARACTER FOR THE UNREACHED ITEM:
U
ENTER THE NUMBER OF ID CHARACTERS:
8
ENTER THE NUMBER OF EXAMINEES:
765
NEED NON-SPURIOUS CORRELATIONS? (Y/N):
Y
NEED QUINTILE GROUP RESULTS? (Y/N):
Y
ENTER THE NAME OF THE DATA FILE:
SAMPLE.DAT
ENTER THE FORMAT OF THE DATA:
(8A1,2X,14A1)
ENTER THE KEY OF ITEM ITEMS:
22343413353511
ENTER THE NUMBERS OF CHOICES:
55555555555555
ENTER THE NAME OF THE OUTPUT FILE:
SAMPLE.OUT
ENTER THE NAME OF THE SCORE FILE:
SAMPLE.SCO
The FORTRAN format statement of the data indicates there are
eight alpha-numeric characters for ID (i.e., 8A1), two blank
columns between ID and the set of item responses (i.e., 2X), and 14
items on the test each occupies one column in alpha-numeric
characters (i.e., 14A1). It is very important to note that the
data can be written as either digits from 1 to 9 or upper
characters from A to I.

3.3. Use of a Batch Input File

It is also possible to use a batch input file and the DOS
redirection feature. You may make an input file that contains
answers to the CIA questions. You may invoke the program by typing
CIA < SAMPLE.INP. The sample input file for the above run is
as follows:
SAMPLE RUN
14
O
U
8
765
Y
Y
SAMPLE.DAT
(8A1,2X,14A1)
22343413353511
55555555555555
SAMPLE.OUT
SAMPLE.SCO
3.4. Sample Outputs
These are two outputs; one that contains the item and test
analysis results (e.g., SAMPLE.OUT) and the other that contains the
scoring results for examinees (e.g., SAMPLE.SCO). The followings
are selective parts of the outputs. The scoring results are
presented in the end of this section. In SAMPLE.OUT, the first
part contains item analysis results; the second part contains
summary statistics and reliability results; and the last part
contains the quintile group information.
In the first part of SAMPLE.OUT, PROP indicates the proportion
of examinees who selected the specific response. RPBI is the point
biserial correlation whereas RBIS is the biserial correlation. RES
indicates the response. The key response has the plus sign. Note
that there are also the omitted, unreached, and invalid response
categories. The item difficulty and item discrimination indices,
that is, the PROP, RPBI, and RBIS from the key response are
presented in the left side of the output.
ITEM
1

PROP
0.868

RPBI
0.468

RBIS
0.741

RES
1
2+
3
4
5
O
U
I

PROP
0.050
0.868
0.012
0.005
0.059
0.007
0.000
0.000

RPBI
-0.277
0.468
-0.166
-0.110
-0.270
-0.112
0.000
0.000

RBIS
-0.587
0.741
-0.582
-0.528
-0.542
-0.491
0.000
0.000

[ITEM 2 ... ITEM 14]

The second part contains the summary statistics of the total
score and the reliability analysis results. The summary statistics
are the number of examinees, the number of items, the mean,
variance, standard deviation, minimum, and maximum of the total

score. The alpha and the standard error of measurement are also
reported. In addition, the mean item difficulty, the mean point
biserial correlation, and the mean biserial correlation are
reported. The end of this part, you can find the title of the run,
and date and time.
N PERSONS
N ITEMS
MEAN
VARIANCE
SD
MINIMUM
MAXIMUM
ALPHA
SEM
MEAN P
MEAN RPBI
MEAN RBIS

765
14
9.31765
8.09646
2.84543
1.00000
14.00000
0.70633
1.54196
0.66555
0.45488
0.61451

SAMPLE RUN
CLASSICAL ITEM ANALYSIS
MONTH= 3 DAY=19 YEAR=1999 TIME=12:57
The last part of SAMPLE.OUT contains the quintile group
analysis. Based on the total score, five groups are formed. The
quintile groups are denoted as Q1 (low) to Q5 (high). Note that
the end of the output, the quintile range (i.e, the range of the
total score for each of the five quintile groups) can be found.
The marginal sum as well as the average score of the examinees who
selected the specific response category can be found. The
following is the cross classification of quintile group and item
response for item 1.
ITEM
1

RES
Q1
1
23
2+
97
3
3
4
2
5
24
O
4
U
0
I
0
SUM
153
[ITEM 2 ... ITEM 14]
QUINTILE RANGE
Q1
LOW,HIGH
1,7

Q2
7
130
2
1
12
1
0
0
153

Q3
7
136
2
1
7
0
0
0
153

Q4
1
150
1
0
1
0
0
0
153

Q5
0
151
1
0
1
0
0
0
153

Q2
7,9

Q3
9,10

Q4
10,12

Q5
12,14

SUM
38
664
9
4
45
5
0
0
765

AVERAGE
5.868
9.837
5.000
5.000
6.244
5.400
0.000
0.000
9.318

The scoring results are reported for examinees. The ten

examinees' scoring results from SAMPLE.OUT are reported below. It
contains the number of correct responses (R), the number of omitted
responses (O), the number of unreached responses (U), the number of
invalid responses (I), and the identification (ID) for each
examinees. In the end of the file, the title of the run, date and
time can be found.
R
O
U
I
ID
8
0
0
0
ANDY
3
0
0
0
CATHERIN
5
0
0
0
MINDY
2
0
0
0
TOBY
3
0
0
0
ERIC
5
0
0
0
VINCE
12
0
0
0
AARON
3
0
0
0
AMY
8
0
0
0
RICHARD
4
0
0
0
JENNIFER
[OTHER EXAMINEES SCORES]
SAMPLE RUN
CLASSICAL ITEM ANALYSIS
MONTH= 3 DAY=19 YEAR=1999 TIME=12:57

References
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement
theory. Monterey, CA: Brooks/Cole
Coffman, W. E. (1971). The achievement tests. In W. H. Angoff
(Ed.), The College Board Admissions Testing Program: A
Technical report on research and development activities
relating to the Scholastic Aptitude Test and Achievement
Tests (pp. 49-77). New York: College Entrance Examination
Board.
Crocker, L., & Algina, J. (1986). Introduction to classical and
modern test theory. New York: Holt, Rinehart and Winston.
Gulliksen, H. (1987). Theory of mental tests. Hillsdale, NJ:
Erlbaum. (Reprinted from Theory of mental tests, by H.
Gulliksen, 1950, New York: Wiley)
Henrysson, S. (1971). Gathering, analyzing, and using data on test
items. In R. L. Thorndike (Ed.), Educational measurement
(2nd ed., pp. 130-159). Washington, DC: American Council on
Education.
Hopkins, K. D. (1998). Educational and psychological measurement
and evaluation (8th ed.). Boston: Allyn and Bacon.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of
mental test scores. Reading, MA: Addison-Wesley.
Magnusson, D. (1967). Test theory (H. Mabon, Trans.). Reading,
MA: Addison-Wesley. (Original work published 1966)
Payne, D. A. (1997).
Wadsworth.

Applied educational assessment.

Belmont, CA:

Sax, G. (1997). Principles of educational and psychological

measurement and evaluation (4th ed.). Belmont, CA: Wadsworth.
Thorndike, R. M. (1997). Measurement and evaluation in psychology
and education (6th ed.). Upper Saddle River, NJ: Merrill/
Prentice-Hall.
Wainer, H. (1989). The future of item analysis.
Educational Measurement, 26, 191-208.

Journal of