Académique Documents
Professionnel Documents
Culture Documents
1. Specifications
2. Summary statistics
3. Item-by-item results
The statistical output is also recorded in a comma-separated value (CSV) file of the same name.
Specifications
The Windows paths for the input files used in this analysis were:
The Windows paths for the output files produced by this analysis were:
output-iteman
output-iteman.csv
Table 1 presents the specifications and basic information concerning the analysis. This provides
important documentation of the setup of the program for historical purposes.
Table 1: Specifications
Specification Value Specification Value
Number of examinees 100 Total Items 18
Scored Items 18 Pretest Items
Multiple Choice Items 13 Polytomous Items 5
Number of Domains 1 External scores No
Minimum P -0,00 Maximum P 1,00
Minimum item mean 0,00 Maximum item mean 15,00
Minimum item correlation -0,00 Maximum item correlation 1,00
ITEMAN 3.0 Header No Exclude omits from option statistics No
Number of ID columns 6 ID begins in column 1
Responses begin in column 7 Omit character O
Not Admin character N Produce quantile tables Yes
Correct for spuriousness Yes Produce quantile plots Yes
Specification Value Specification Value
Save data matrix No Include omit codes in matrix N/A
Include Not Admin codes in matrix N/A Include scaled scores for N/A
Scaling function N/A Scaled score setting 1 N/A
Scaled score new SD N/A Dichotomous Classification N/A
Classify based on N/A Cutpoint N/A
Low group label Pass High group label Fail
Data is delimited by Tab Test for DIF No
Group status is in column N/A Ability levels for DIF N/A
Group 1 code N/A Group 2 code N/A
Group 1 label N/A Group 2 label N/A
Summary statistics
Table 2 presents the summary statistics of the test for the scored items. Definitions of these statistics
are found in the Iteman manual.
Table 3 presents a reliability analysis of the tests. Alpha (also known as KR-20) is the most commonly
used index of reliability, and is therefore used to calculate the standard error of measurement (SEM)
on the raw score scale. Also presented are three configurations of split-half reliability, first as
uncorrected correlations, and then as Spearman-Brown (S-B) corrected correlations. This is because
an uncorrected split-half correlation is referenced to a "test" that only contains half as many items as
the full test, and therefore underestimates reliability.
Table 3: Reliability
Score Alpha SEM Split-Half Split-Half Split-Half S-B S-B First- S-B Odd-
(Random) (First-Last) (Odd-Even) Random Last Even
Scored 0,802 1,841 0,672 0,589 0,730 0,804 0,741 0,844
items
Table 4 presents the item statistics and flags for the item(s) that were flagged during the analysis
Score Frequency
-0 1
1 4
2 4
3 9
4 9
5 12
6 6
7 5
8 11
9 7
10 8
11 11
12 2
13 3
14 1
15 3
16 1
17 0
18 2
19 1
Figure 2 displays the distribution of the P values for the dichotomously scored items (correct/incorrect).
Table 6 displays the frequency distribution of the P values shown in Figure 2.
Figure 3 displays the distribution of the Item Means (rating scale or partial credit items only). Table 7
displays the frequency distribution of the Item Means shown in Figure 3.
Figure 3: Item Means for the scored items
Figure 4 displays the distribution of the Point-Biserial Correlations for the dichotomously scored items
(correct/incorrect). Table 8 displays the frequency distribution of the Point-Biserial correlations shown
in Figure 4.
Figure 4: Rpbis for the scored items
Figure 5 displays the distribution of the Item-Total Pearson r Correlations for polytomously scored
items (rating scale or partial credit). Table 9 displays the frequency distribution of Pearsons r
correlations shown in Figure 5.
Figure 5: R for the scored items
Figure 6 displays the scatterplot of P (difficulty) by Rpbis (discrimination) for the dichotomously scored
items (correct/incorrect).
Figure 6: P by Rpbis
Figure 7 displays the scatterplot of the Item Means (difficulty) by R (discrimination) for polytomously
scored items (rating scale or partial credit).
1. Item information table: records the information supplied by the control file (or Iteman 3 header) for
this item.
3. Option statistics: detailed statistics for each item, which helps diagnose issues in items with poor
statistics.
4. Quantile plot data: the values used to create the quantile plot.
The item statistics table presents overall item statistics in the first row of numbers. The two most
important item-level statistics for dichotomously scored (correct/incorrect) items are the P value and
the point-biserial correlation, which represent the difficulty and discrimination of the item, respectively.
For polytomously scored (rating scale or partial credit) items, the difficulty is represented by the mean
(average) item score, while the discrimination is represented by a Pearson r correlation.
The P value is the proportion of examinees that answered an item in the keyed direction. P ranges
from 0 to 1. A high value (0.95) means that an item is easy, a low value (0.25) means that the item is
difficult. The point-biserial correlation (Rpbis) is a measure of the discriminating, or differentiating,
power of the item. Rpbis ranges from -1 to 1. A negative Rpbis is indicative of a bad item as lower
scoring examinees are more likely than higher scoring examinees to respond in the keyed direction.
For rating scale or partial credit items, the mean item score ranges from the minimum to the maximum
of the scale. For example, if the item has a rating scale of 1 to 5, the possible range for the mean is 1
to 5. The Pearson r is similar to the Rpbis in that it ranges from -1 to 1, with a positive r indicating that
the item correlates well with total score.
The option statistics table presents statistics for each individual option (alternative). The key thing to
examine in this portion of the table is that no distractors have a higher Rpbis than the correct answer.
That indicates that higher scoring examinees are selecting the incorrect answer, which therefore might
be arguably correct.
The quantile plot data table simply presents the values calculated to create the quantile plot. Because
it contains the same information, the quantile plot itself presents a useful picture of the item's
performance, but this table can be used to examine that performance in detail to help diagnose
possible issues.
Item information
Seq. ID Key Scored Num Options Domain Flags
1 CR01 + Yes 3 1
Item statistics
N Mean Total R Total Eta Alpha w/o
100 1,020 0,454 0,535 0,790
Option statistics
Option Weight N Prop. Rpbis Rbis Mean SD Color
0 0 25 0,250 -0,423 -0,576 3,760 2,681 Maroon
1 1 48 0,480 0,078 0,098 7,792 3,678 Green
2 2 27 0,270 0,325 0,436 10,481 4,173 Blue
Omit 0
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,330 0,236 0,307 0,802
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 4 0,040 -0,215 -0,489 3,000 2,449 Maroon
2 36 0,360 0,160 0,206 8,028 3,791 Green
3 16 0,160 -0,254 -0,382 4,875 2,849 Blue
4 33 0,330 0,236 0,307 9,515 4,255 Olive **KEY**
Omit 11 0,110 -0,113 -0,188 5,273 3,467
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,780 0,299 0,418 0,798
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 5 0,050 -0,158 -0,334 4,000 3,162 Maroon
2 4 0,040 -0,192 -0,437 3,000 1,633 Green
3 78 0,780 0,299 0,418 8,359 3,868 Blue **KEY**
4 5 0,050 0,085 0,180 8,200 6,943 Olive
Omit 8 0,080 -0,156 -0,284 3,250 2,315
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,870 0,354 0,563 0,796
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 3 0,030 -0,161 -0,404 3,000 3,000 Maroon
2 1 0,010 -0,016 -0,060 6,000 0,000 Green
3 87 0,870 0,354 0,563 8,184 4,069 Blue **KEY**
4 3 0,030 -0,220 -0,552 1,667 1,155 Olive
Omit 6 0,060 -0,133 -0,265 3,167 2,137
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,720 0,436 0,582 0,790
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 6 0,060 -0,257 -0,513 2,833 2,483 Maroon
2 72 0,720 0,436 0,582 8,847 4,002 Green **KEY**
3 4 0,040 -0,160 -0,363 3,750 2,217 Blue
4 10 0,100 -0,145 -0,248 5,100 2,961 Olive
Omit 8 0,080 -0,138 -0,252 3,875 2,031
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,420 0,629 0,794 0,777
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 10 0,100 -0,202 -0,346 4,800 1,619 Maroon
2 8 0,080 -0,193 -0,351 4,625 3,503 Green
3 9 0,090 -0,045 -0,078 6,556 3,046 Blue
4 42 0,420 0,629 0,794 10,881 3,473 Olive **KEY**
Omit 31 0,310 -0,261 -0,342 4,839 2,684
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,450 0,539 0,678 0,783
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 8 0,080 -0,266 -0,485 3,625 1,847 Maroon
2 45 0,450 0,539 0,678 10,333 3,997 Green **KEY**
3 8 0,080 -0,198 -0,361 4,500 2,449 Blue
4 24 0,240 -0,181 -0,248 5,833 2,869 Olive
Omit 15 0,150 -0,124 -0,189 5,400 2,667
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,540 0,506 0,636 0,785
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 2 0,020 -0,167 -0,482 2,500 0,707 Maroon
2 54 0,540 0,506 0,636 9,759 3,976 Green **KEY**
3 10 0,100 -0,032 -0,055 6,600 3,340 Blue
4 3 0,030 -0,213 -0,534 2,333 2,309 Olive
Omit 31 0,310 -0,258 -0,339 4,710 2,224
Not Admin 0
Item statistics
N Mean Total R Total Eta Alpha w/o
100 0,370 0,587 0,600 0,777
Option statistics
Option Weight N Prop. Rpbis Rbis Mean SD Color
0 0 76 0,760 -0,581 -0,798 5,947 3,011 Maroon
1 1 11 0,110 0,266 0,443 10,909 2,448 Green
2 2 13 0,130 0,490 0,779 13,769 4,086 Blue
Omit 0
Not Admin 0
Item statistics
N Mean Total R Total Rbis Alpha w/o
100 0,080 0,433 0,787 0,794
Option statistics
Option Weight N Prop. Rpbis Rbis Mean SD Color
0 0 92 0,920 -0,431 -0,787 6,924 3,626 Maroon
1 1 8 0,080 0,431 0,787 14,250 4,062 Green
Omit 0
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,550 0,416 0,523 0,791
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 4 0,040 -0,275 -0,625 1,750 1,500 Maroon
2 55 0,550 0,416 0,523 9,418 3,743 Green **KEY**
3 5 0,050 -0,152 -0,321 4,400 0,894 Blue
4 9 0,090 -0,231 -0,408 4,111 3,219 Olive
Omit 27 0,270 -0,085 -0,114 6,185 3,783
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,220 0,443 0,620 0,790
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 22 0,220 0,443 0,620 11,545 4,554 Maroon **KEY**
2 10 0,100 -0,033 -0,057 6,900 3,985 Green
3 11 0,110 -0,026 -0,043 7,000 3,098 Blue
4 5 0,050 -0,135 -0,284 5,000 2,550 Olive
Omit 52 0,520 -0,189 -0,237 6,269 3,267
Not Admin 0
Item statistics
N Mean Total R Total Eta Alpha w/o
100 0,200 0,528 0,527 0,785
Option statistics
Option Weight N Prop. Rpbis Rbis Mean SD Color
0 0 82 0,820 -0,503 -0,737 6,402 3,359 Maroon
1 1 16 0,160 0,411 0,620 11,938 3,492 Green
2 2 2 0,020 0,304 0,879 17,500 3,536 Blue
Omit 0
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,140 -0,078 -0,122 0,814
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 11 0,110 -0,102 -0,169 6,182 3,545 Maroon
2 35 0,350 0,490 0,630 10,114 3,644 Green
3 14 0,140 -0,078 -0,122 7,571 4,879 Blue **KEY**
4 6 0,060 -0,064 -0,127 6,333 4,546 Olive
Omit 34 0,340 -0,226 -0,292 5,441 3,077
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,160 0,111 0,167 0,807
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 16 0,160 0,111 0,167 9,375 3,879 Maroon **KEY**
2 15 0,150 0,275 0,421 10,000 4,536 Green
3 8 0,080 0,038 0,070 7,875 5,793 Blue
4 13 0,130 -0,122 -0,193 6,077 4,699 Olive
Omit 48 0,480 -0,151 -0,190 6,437 3,162
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,080 0,356 0,649 0,797
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 8 0,080 0,356 0,649 13,250 6,319 Maroon **KEY**
2 8 0,080 -0,096 -0,176 6,125 4,224 Green
3 11 0,110 -0,110 -0,183 6,182 2,714 Blue
4 14 0,140 0,108 0,168 8,500 3,436 Olive
Omit 59 0,590 -0,105 -0,133 6,932 3,581
Not Admin 0
Item statistics
N P Total Rpbis Total Rbis Alpha w/o
100 0,070 0,096 0,182 0,805
Option statistics
Option N Prop. Rpbis Rbis Mean SD Color
1 13 0,130 0,280 0,445 10,385 5,723 Maroon
2 7 0,070 0,096 0,182 9,857 4,120 Green **KEY**
3 15 0,150 -0,045 -0,070 7,000 3,891 Blue
4 3 0,030 -0,019 -0,048 7,000 7,211 Olive
Omit 62 0,620 -0,143 -0,183 6,790 3,417
Not Admin 0
Item statistics
N Mean Total R Total Eta Alpha w/o
100 0,510 0,484 0,525 0,788
Option statistics
Option Weight N Prop. Rpbis Rbis Mean SD Color
0 0 66 0,660 -0,390 -0,505 5,970 3,291 Maroon
1 1 17 0,170 -0,007 -0,011 7,941 3,391 Green
2 2 17 0,170 0,500 0,741 13,059 3,648 Blue
Omit 0
Not Admin 0