Factor Analysis

A technique that is often used to create new variables that summarize the information that might be available in the original variables.

One basic objective of FA is to determine whether the response variables exhibit patterns of relationships with each other, such that the variables can be partitioned into subsets of variables. The variables in a subset are highly correlated with each other and the variables in different subsets have low correlation with each other. Thus, FA is often used to study the correlational structure of the variables in a data set.

Another basic purpose of FA is to derive, create or develop a new set of uncorrelated variables, called underlying factors, with the hope that these new variables will give a better understanding of the data being analyzed. Reasonable interpretations are to be given for the newly created factors subjectively by the researchers, rather than objectively by statistical methods.

To summarize, the goals of FA include the following:

1. To determine whether a smaller set of uncorrelated variables exists that will explain the relationships that exist between the original variables

2. To determine the number of underlying factors

3. To interpret these new factors
Cont

4. To evaluate individuals or experimental units in the data set on these factors 5. To use these factors in other statistical analyses of the data

Warning FA creates a new set of uncorrelated variables from a set of correlated variables. Hence, if the original variables are already uncorrelated, then there is a little reason to consider carrying a FA.

Factor Analysis Model

Suppose one observes a p-variate response vector X from a population that has mean and variance-covariance matrix . The general FA model assumes that there are m underlying factors (m < p) denoted by f1, f2, fm.

Certain concepts
Factor Loadings Non uniqueness in the factor loading matrix - Use of Factor Rotation Choosing appropriate number of factors
Eigenvalues > 1 Cumulative percentage variation > specified percentage (80% or 90%)Factor Rotation

EXAMPLE 1 In a consumer-preference study, a random sample of customers were asked to rate several attributes of a new product. The response on a 7-point semantic differential scale were tabulated and the attribute correlation matrix is constructed. The correlation matrix is given as
Attributes (Variables) 1. Taste 2. Good buy for money 3. Flavour 4. Suitable for snacks 5. Provides lots of energy 1 1.00 0.02 1.00 0.96 0.13 1.00 0.42 0.71 0.50 1.00 0.01 0.85 0.11 0.79 1.00 2 3 4 5

Interpret the correlation matrix R, to show whether to go for Factor Analysis.

The factor loadings (obtained by Principal Component Analysis), the cumulative proportion of total sample variance is given below.
Variables Factor 1 1 2 3 4 5 0.56 0.78 0.65 0.94 0.80 Factor 2 0.82 - 0.53 0.75 - 0.11 - 0.54 0.931 After factor rotation (Varimax) Factor 1 0.02 0.94 0.13 0.84 0.97 0.507 Factor 2 0.99 - 0.01 0.98 0.33 - 0.02 0.935

Cumulative 0.571 proportion of total sample variance

Interpret the factor loadings.

EXAMPLE 2
Stock prices on weekly basis is taken for 5 stocks. The Factor Analysis results are shown in the table below. Variable Factor 1 Factor 2

1. Allied Chemical 2. Du Pont 3. Union Carbide 4. Exxon 5. Texaco Cumulative proportion of total sample variance
Interpret the factor loadings.

0.783 0.773 0.743 0.713 0.712 0.57

- 0.417 - 0.458 - 0.434 0.472 0.524 0.83