30280 Applications for Management

Lecture 16
Scale construction and reliability analysis
Agenda

 Multidimensional scale of measurement

 Reliability evaluation
 Scale construction
 Surrogate variable
 Composite measure (index)
 Factor scores

Università Bocconi, 30280, BIEM, slide 2

Latent variables
 In social sciences, the concepts of interest
often cannot be measured directly.
 Examples are intelligence, political attitudes,
preferences, socio-economic status, happiness,
quality of life.
 Since, such concepts cannot be directly
observed they are called latent variables or
constructs.
 Variables that can be observed and measured
are then used so to get information on the
latent variables.
Università Bocconi, 30280, BIEM, slide 3
 Multidimension
 In general the concept of interest is
multidimensional, in the sense that it needs several
latent variables to be represented and consequently
several manifest variables need to be jointly
analysed.
o Intelligence can be studied by measuring variables that
can be influenced by the latent variables: scores in
multiple tests (e.g., verbal, mathematical, spatial
exercises).
o Socio economic status can be measured by collecting
information on several variables as household income,
occupation and education of the members.

Università Bocconi, 30280, BIEM, slide 4

 Multivariate metric scales
 Information on the latent variables is collected by
measuring several manifest variables.
 The scale of measurement being used is a
multivariate metric scale.
 Metric because each manifest variable is measured
by means of questions, in general closed-ended
ones, having numerical answers.
 Multivariate because the respondent is asked to
answer to several questions (a battery of
questions), each collecting information on the same
concept.

Università Bocconi, 30280, BIEM, slide 5

Likert Scale
 Rensis Likert, 1932, used to collect data on attitudes and
beliefs.
 A composite scale is made up by several questions, items
 For each item, the respondent is asked to express his
degree of agreement or disagreement with respect to a
given statement.
 An odd number of response levels is used. Often, the
response levels range from 1 to 5.
 A typical five-level Likert item is: 1. Strongly disagree;
2. Disagree; 3. Neither Agree nor disagree; 4. Agree; 5.
Strongly agree.
 Several scales use up to 10 levels.

Università Bocconi, 30280, BIEM, slide 6

Università Bocconi, 30280, BIEM, slide 7
Comments
Risk of response bias
 Most people have a tendency to answer to all items in a
similar way, e.g.
 central tendency: using only the central points

 using only the extreme points

 favouring points to the left or the right of the scale

Controversial
 The difference between numbered points is not equal

(and is not equal to 1)

 Technically, these scales are ordinal. (What does a

mean score on this scale mean?)

Università Bocconi, 30280, BIEM, slide 8

Association
 The items are used to gather information on the same
attitude or belief.

 Each item generates an ordinal variable, referred to as an

indicator variable.

 There are then several variables measuring the same

concept.

 The better the multivariate scale (battery of items) is

designed, the more the variables will be associated.

Università Bocconi, 30280, BIEM, slide 9

Scale reliability

 Is our scale a good measure?

 The reliability of a multidimensional scale
has to be assessed.

Università Bocconi, 30280, BIEM, slide 10

Techniques available to measure
reliability
 The most common method is Cronbach’s
alpha

 It is a correlation-based measure of the

“internal consistency” among the items of
the scale.

Università Bocconi, 30280, BIEM, slide 11

 Cronbach’s alpha
For items scored in the same direction, Cronbach’s
alpha has the following expression:

 The higher the inter-correlations between items, the

higher is Cronbach’s alpha, the more reliable the scale.

Università Bocconi, 30280, BIEM, slide 12

 Cronbach’s alpha

 Note that alpha will increase

 for given mean correlation if the number of items in
the scale (k) increases.
 for given number of items, if the mean correlation
increases.
 A multi-item scale can be improved by removing
items weakly correlated or by adding items.
Università Bocconi, 30280, BIEM, slide 13
Rules of thumb
Cronbach Alpha Coefficient Size

Coefficient Range Strength of Association

< .6 Poor
.6 to < .7 Moderate
.7 to < .8 Good
.8 to < .9 Very Good
> = .9 Excellent

Università Bocconi, 30280, BIEM, slide 14

Cronbach’s alpha in stata

Università Bocconi, 30280, BIEM, slide 15

Cronbach’s alpha in stata

Università Bocconi, 30280, BIEM, slide 16

Example - Students’ satisfaction

 Can we measure overall student

satisfaction?

 How can we build a reliable index of overall

student satisfaction?

 Can we identify determinants of students

satisfaction?

Università Bocconi, 30280, BIEM, slide 17

Example - Students’ satisfaction

 Can we measure overall student

satisfaction?

 How can we build a reliable index of overall

student satisfaction?

 Can we identify determinants of students

satisfaction?

Università Bocconi, 30280, BIEM, slide 18

Students’ satisfaction

Università Bocconi, 30280, BIEM, slide 19

Students’ satisfaction

Università Bocconi, 30280, BIEM, slide 20

Students’ satisfaction
ssc install labsumm (package to label the summary table)

Università Bocconi, 30280, BIEM, slide 21

 Students’ satisfaction Moderate strength
of the association
Reliability – Cronbach’s Alpha between the
variables

Item-rest corr. Is the correlation of the item to the summated scale score

Università Bocconi, 30280, BIEM, slide 22

Students’ satisfaction

 Can we measure overall student

satisfaction?

 How can we build a reliable index of overall

student satisfaction?

 Can we identify determinants of students

satisfaction

Università Bocconi, 30280, BIEM, slide 23

 Scale construction
The issue is then how to extract and combine the information
collected on the indicators so to get information on the latent
variables.

There are several approaches:

1. Proxy (surrogate) variable: one single manifest variable
is chosen to represent the underlying latent variable.
2. Index or composite measure: scores are assigned to
each level and the easiest combination is obtained by
adding them up or by averaging them. The average can be
an arithmetic or a weighted mean, in this case the choice
of the weight is an additional issue.
3. Factor scores: synthetic measure worked out by running
a factor analysis.
Università Bocconi, 30280, BIEM, slide 24
1. Students’ satisfaction:
Surrogate Variable

Università Bocconi, 30280, BIEM, slide 25

 1. Students’ satisfaction:
Surrogate Variable

Teaching
Education
Campus computer
Campus overall
Social overall

• Two factors (Academic satisfaction, Satisfaction with campus life)

• Choose:
• Teaching – Overall (highest loading on factor 1) as surrogate for
factor 1
• Campus – Computers (highest loading on factor 2) as
surrogate for factor 2
Università Bocconi, 30280, BIEM, slide 26
Select surrogate variables
Advantages & disadvantages

Advantage:
 Simple to administer and interpret
Disadvantages:
 It does not address the issue of measurement error
encountered when using single measures
 Measurement error= the degree to which the observed
values are not representative of the actual values due to
any number of reasons, such as actual errors (data entry
errors) to the inability of individuals to provide accurate
information.
 It runs the risk of potentially misleading results by
selecting only a single variable to represent a perhaps
more complex result.
Università Bocconi, 30280, BIEM, slide 27
 2. Students’ satisfaction:
Index
 Let’s use as index of Students satisfaction the sum of
the scores of the 5 items.
 Summated scale.

 Another option is to work out the arithmetic mean of

the items.
 Note that in both cases items enter the synthetic
measure with the same weight.
Università Bocconi, 30280, BIEM, slide 28
Using an index
advantages & disadvantages

Advantage:
 It overcomes to some extent measurement error
 Represents multiple aspects of a concept in a
single measure

Disadvantage:
 Relative weight of different items is usually not
taken into account

Università Bocconi, 30280, BIEM, slide 29

3. Students’ satisfaction:
A model based measure

 Let us run a factor analysis on the 5 items with

the aim of working out new variable(s)
summarising the information provided by the 5
items so to derive one or more measures of
overall satisfaction.

 Aiming at a data reduction, we run factor

analysis with PCA extraction.

Università Bocconi, 30280, BIEM, slide 30

 Factor analysis of student satisfaction
with PCA
The first two
components have
eigenvalues
greater than 1 and
account for 65% of
the total variance

Teaching Rotated F1: “Academic

Education satisfaction”
Campus computer Rotated F2: “Satisfaction with
Campus overall
campus life”
Social overall
Università Bocconi, 30280, BIEM, slide 31
 Factor scores
 A factor score is the value taken on a given statistical unit
by the variable that we create as a result of factor
analysis.
 In our case we have two work out factor scores for two
variables (academic satisfaction, and satisfaction with campus life).

 They are obtained as a weighted combination of the

values of the items, where weights can be estimated by
means of different methods.
 The weights are reported in the table Component (or
Factor) Score Coefficient Matrix.
 In the case of a factor analysis with PCA all three types of
methods deliver the same weights. Università Bocconi, 30280, BIEM, slide 32
 Component score coefficient
matrix
Optional technical note: Factor score
coefficients are calculated by multiplying
the inverse correlation matrix by the factor
loading matrix
Factor scores are derived from a
factor score coefficient matrix
which provides the “weights” for
each variable
Fi = w1i X1+ w2i X2+ w3i X3+ w4i X4+ w5i X5
FAC1_1i = 0.51*X1i + 0.087*X2i + 0.55*X3i
+ 0.08X4i -0.23X5i
FAC2_1i = -0.09*X1i + 0.24*X2i - 0.16*X3i
+ 0.46X4i +.67X5i
Università Bocconi, 30280, BIEM, slide 33
A factor score for each case is
calculated by regression

Università Bocconi, 30280, BIEM, slide 34

Histogram of factor scores

Università Bocconi, 30280, BIEM, slide 35

New variables from factor scores
advantages & disadvantages

Advantages:
 Represent all variables loading on the factor
 Very easy to calculate
 Accounts for more variance in the data than
other methods
Disadvantages:
 Validity: There may be ‘contamination’ of
concepts when contributions of variables that
measure other concepts are included
 Specific to one sample; cannot be replicated
across samples
Università Bocconi, 30280, BIEM, slide 36
Students’ satisfaction

 Can we measure overall student

satisfaction?

 Can we build a reliable index of overall

student satisfaction?

 Can we identify determinants of students

satisfaction?

Università Bocconi, 30280, BIEM, slide 37

Scores as dependent variables

 Rotated F1: “Academic satisfaction”

 Rotated F2: “Satisfaction with campus life”

 Do they depend on gender, age, overseas student,

average grade, percentage of course completed?

Università Bocconi, 30280, BIEM, slide 38

Regression on F1 (Academic satisfaction)

Regression

Università Bocconi, 30280, BIEM, slide 39

Regression on F2
(Satisfaction with campus life)

Regression

Università Bocconi, 30280, BIEM, slide 40