Chapter

11
Basic Psychometrics
 Outline
Psychometrics
The Classical Test Theory
Item Response Theory
 Psychometrics
Measurement of psychological a9ributes
Develop tests and scales
Psychological tes>ng: development, obtaining data, and
interpreta>on of test
Psychometrics: Science of measuring mental capaci>es.
Developing sta>s>cal theory of mental measurement
Classical test theory (CCT)
Item response theory (IRT)
Generalizability theory
 Classical Test Theory

Raw score (X) is a func>on of true score and

random error
X =T +e
Raw score is random variable
True score is expected value of this probability
distribu>on K
= X k Pk = E ( X )
k=1
 Classical Test Theory
( )
T j = E X j = Xj
Tj is true score of examinee j
ej = X j Tj
The ej is error associated with examinee j

ej () (
= E e = E X T
j j j )
( )
E ej = Tj Tj = 0
 Assump:ons
The assump:ons of the CTT are as follows:
The expecta>on of error is zero. E(ej) = 0
The true score and the error scores are
independent. The correla>on between error score
and true score in the popula>on is zero. T.e = 0.00
When the scores of examinee are obtained on two
separate tests, then the errors of the two test in the
popula>on are uncorrelated. e1.e2 = 0.00
 Reliability, Reliability Coecient,
and CTT
Reliability is conceptualized as correla>on between the true
score and the raw score
X X = (T T ) + ( e e )

XT =
( X ) (T )
X T

N X T

(T ) + ( e ) (T )
XT =
T e T

N X T
Reliability, Reliability Coecient,
and CTT
Hence,
XT =
( T ) ( e ) (T )
+
T
2
e T

N X T N X T
( e e )(T T ) = 0
( T ) 2

N X T 2 = T
T
N
2
XT = T
X T
T
XT =
X
 Methods of Assessment of
Reliability
Procedures based on two test administra>ons:
Test-retest Method
Alternate form reliability

Methods for single administra>on of the test
Split-half method
Cronbach Alpha: method based on item covariances
 Test-retest Methods
Test has a single form
Two administra>ons of the single form
Same set of par>cipants
Two administra>on with a temporal gap
Pearsons product moment correla>on
between the two administra>ons
 Alternate Form reliability
Test has two equivalent forms
Correla>on between the two forms
Two forms can be used interchangeably
For two forms to be equivalent
similar means, similar variances, similar standard
errors of measurement, similar reliability and similar
rela>ons with other constructs
Less popular
Single Administra:on: Split-half
Method
Steps:
Odd items are grouped into rst group and even items are
grouped into second
randomly assign items to two halves
Rank-order items on the basis of diculty (p) and then alternate
item goes to rst half and remaining goes to second half.
By matching content: when content diers in specic respects, it is
necessary to match content.
2 AB
XX ' =
1 + AB
Cronbach Alpha: Method Based on
Item Covariances
Lee Cronbach (1951)

=
k
1 2
2
i


k 1 X
where
is Cronbach Alpha reliability,
k is number of items in the test,
i2 is a variance of item i,
X2 is a variance of the total test.
 Item Response Theory
Lord and Novik (1968)
Trait is called a latent trait ()
Theta () refers to the ability of the person
The probability of response to an item on the test is
func>on of individuals theta level
This probability is modeled by various IRT models
Dichotomous responses, polytomous responses, nominal
category model, ra>ng scale models, graded response
model, par>al credit models, sequen>al model for
ordered responses, and models for >me limit tests
One Parameter Logis:c Model
one parameter in the model is diculty
diculty is labeled as b (Mean=0, var=1.00)
theta is also scaled with mean 0 and variance 1.00
e( b)
P(X = 1 ,b) =
1+ e( b)
1
P( i ) =
{
1+ e D( bi ) }
where D is 1.702, e = 2.718, b is item diculty, and theta
is the ability of the examinee.
One PL IRT Model
Two Parameter Logis:c Model
1PL model assumed equal slope for all the items
two parameter logis>c model has slope parameter set
free. Discrimina>on parameter or slope parameter is
called a a( b)
e
P(X = 1 ,a,b) = a( b)
1+ e
1
P( i ) =
{1+ e Dai ( b )
}
Three Parameter Logis:c Model
3PL IRT Model is parameter c
Parameter c is a guessing parameter
a( b)
e
P(X = 1 ,a,b,c) = c + (1 c) a( b)
1+ e

1 1 c
P(i ) = c+(1 c) = c+
{1+ e } {1+ e }
Dai ( b ) Dai ( b )
Three Parameter Logis:c Model
 Samejimas Graded Response
Model
Modeling the polytomous response

Pxij ( X ij = xij | i ) = P (i ) P
*
xij
*
xij +1 (i )
Da j ( j bxij )
e
P (i ) = P ( X ij xij | i ) =
*
xij Da j ( j bxij )
1+ e
Samejima Graded Responses
Model Graph
 Item Informa:on and Test
Informa:on Func:on
Item%and%test%information%function%
%
Item%Information%function%for%first%three% Test%information%function%for%the%
items%% items%of%LSAT%

%
%
%
 Choosing a Model

Model es>mates are maximum likelihood

es>mates

Goodness-of-t chi-square test

LR test