Procedures of Exploratory Factor Analysis

Let's work our way up the right-hand side of the "SEM Pyramid of Success," examining how the Pearson
correlation gives rise to exploratory factor analysis (EFA).
Starting out with a fairly large set of variables (usually single items), EFA will arrange the variables into
subsets, where the variables within each subset are strongly correlated with each other. These subsets are
organized along axes (the plural of "axis," not the "axe" like a hatchet).
You could have a one-factor (one-dimensional) solution, in which case all the variables will be capable of
being located along a single line (e.g., across from left to right, with low scores to the left and high scores
to the right). Or there could be a two-factor (two-dimensional) solution, where the axes are across and upand-down. Three-factor (three-dimensional) solutions are harder to describe verbally, so let's look at
a picture. These examples hold only as long as the axes areorthogonal (at 90-degree angles) to each
other (which denotes completely uncorrelated factors), an issue to which we'll return. Solutions can also
exceed three factors, but we cannot visualize four spatial dimensions (at least I can't).
In conducting factor analyses with a program such as SPSS, there arethree main steps, at each of which
a decision has to be made:
(1) One must first decide what extraction method to use (i.e., how to "pull out" the dimensions). The two
best-known approaches arePrincipal Axis Factoring (PAF; also known as common factor analysis)
and Principal Components Analysis (PCA). There's only one difference, computationally, between
PAF and PCA, as described in this document, yet some authors portray the two techniques as being very
different (further, PCA is technically not a form of factor analysis, but many researchers treat it as such).
(2) Second, one must decide how many factors to retain. There is no absolute, definitive answer to this
question. There are various tests, including the Kaiser Criterion (how many factors or components
haveeigenvalues greater than or equal to 1.00) and Scree Test (an "elbow curve," where one looks for
drop-off in the sizes of the eigenvalues).
The book Does Measurement Measure Up?, by John Henshaw, addresses the indeterminacy of factor
analysis in the context of intelligence testing as follows: "Statistical analyses of intelligence test data...
have been performed for a long time. Given the same set of data, one can make a convincing, statistically
sound argument for a single, overriding intelligence (sometimes called the g factor) or an equally sound
argument for multiple intelligences. In Frames of Mind, Howard Gardner argues that 'when it comes to
the interpretation of intelligence testing, we are faced with an issue of taste or preference rather than one
on which scientific closure is likely to be reached' " (p. 95).
(3) The axes from the original solution will not necessarily come close to sets of data points (loosely
speaking, it's like the best-fitting line in a correlational plot). The axes can be rotated to put them into
better alignment with the data points. The third decision, therefore, involves the choice of rotation
method. Two classes of rotation methods areorthogonal (as described above) and oblique (in which the
axes are free to intersect at other than 90-degree angles, which allows the factors to be correlated with
each other). Mathworks has a web document on factor rotation, including a nice color-coded depiction of
orthogonal and oblique rotation. (As of January 2015, the graphics do not show up in the Mathworks
document; however, I had previously saved a copy of the factor-rotation diagram, which I reproduce
below.)

From Mathworks, Factor Analysis

***
The particular combination of Principal Components Analysis for extraction, the Kaiser Criterion to
determine the number of factors, and orthogonal rotation (specifically one called Varimax) is known as
the "Little Jiffy" routine, presumably because it works quickly. I've always been a Little Jiffy guy myself
(and have written a song about it, below), but in recent years, Little Jiffy has been criticized, both
collectively and in terms of its individual steps.
An article by K.J. Preacher and R.C. MacCallum (2003) entitled "Repairing Tom Swifts Electric Factor
Analysis Machine" (explanation of "Tom Swift" reference) gives the following pieces of advice (shown in
italics, with my comments inserted in between):
Three recommendations are made regarding the use of exploratory techniques like EFA and PCA. First,
it is strongly recommended that PCA be avoided unless the researcher is specifically interested in data
reduction... If the researcher wishes to identify factors that account for correlations among [measured
variables], it is generally more appropriate to use EFA than PCA...
Another article we'll discuss (Russell, 2002, Personality and Social Psychology Bulletin) concurs that
PAF is preferable to PCA, although it acknowledges that the solutions produced by the two extraction
techniques are sometimes very similar. Also, data reduction (i.e., wanting to present results in terms of,
say, three factor-based subscales instead of 30 original items) seems to be a respectable goal, for which
PCA appears appropriate.
Second, it is recommended that a combination of criteria be used to determine the appropriate number
of factors to retain... Use of the Kaiser criterion as the sole decision rule should be avoided altogether,
although this criterion may be used as one piece of information in conjunction with other means of
determining the number of factors to retain.
I concur with this, and Russell's recommendation seems consistent with this.
Third, it is recommended that the mechanical use of orthogonal varimax rotation be avoided... The use
of orthogonal rotation methods, in general, is rarely defensible because factors are rarely if ever

uncorrelated in empirical studies. Rather, researchers should use oblique rotation methods.
As we'll see, Russell has some interesting suggestions in this area.
One final area, discussed in the Russell article, concerns how to create subscales or indices based on your
factor analysis. Knowing that certain items align well with a particular factor (i.e., having highfactor
loadings), we can either multiply each item by its factor loading before summing the items
(hypothetically, e.g., [.35 X Item 1] + [.42 X Item 2] + [.50 X Item 3].......) or just add the items up with
equal (or "unit") weighting (Item 1 + Item 2 + Item 3). Russell recommends the latter. It should be noted
that, if one obtains a varimax-rotated solution, the newly created subscales will only have zero correlation
(independence or orthogonality) with each other if the items are weighted by exact factor scores in
creating the subscales.
I have created a diagram to explicate factor scoring in greater detail.Here's another perspective on the
issue of factor scores (see the heading "Factor Scores/Scale Scores" when the new page opens).

2 parte

Latent and Manifest Variables in SEM

(Updated February 5, 2015)

To begin our coverage of SEM, we'll discuss the conceptual basis of latent variables and their manifest
(measured) indicators. The diagram below, which I developed several years ago, provides an "everyday"
illustration.

There are some additional analogies that can be drawn upon:

In Freudian psychology, a distinction is made between the latent and manifest content of dreams.
In biology, a distinction is made between genotype and phenotype.

The book Does Measurement Measure Up?, by John Henshaw, provides a concise summary of how
observable, measurable manifestations are used to infer underlying, unobservable propensities: "Aristotle
and others have contrasted the observed behavior of an individual with the underlying capacity on which
that behavior depended. Intelligence, as one of those underlying capacities, is an ability that may or may
not always be observed in everyday life. This underlying capacity must be deduced from observed
behaviors" (p. 92)