On the plausibility of unitary language proficiency factor

Hossein Farhady

On the Plausibility of the Unitary Language Proficiency Factor*

Hossein Farhady University of California, Los Angeles University for Teacher Education, Tehran, Iran

Editors Introduction In Chapter 2, Hossein Farhady explains certain fundamental facts about the most widely accepted approaches to exploratory factor analysis. He is concerned to demonstrate the inappropriateness of principal components analysis as applied in Chapter 1 (and elsewhere) to the problem of testing for the existence of an exhaustive general factor of language proficiency. He flatly rejects the strongest possible version of a general factor hypothesis, but he does not deny the possibility of a nonexhaustive general factor. Farhady does insist, however, on more standard factoring methods with communality estimates on the diagonal of the original correlation matrix (rather than unities) followed by an orthogonal rotational procedure to achieve a terminal solution.

Introduction
The complexities and intricacies of human intellectual capacity have been a subject of discussion for several centuries. The diversity of functions that the human mind is capable of performing has led scholars to formulate numerous theories. Many synonymous and/or overlapping terms have been coined to represent the underlying traits of human performance. All these theories, I believe, should be subjected to scientific scrutiny in order for them to be judged empirically valid. Otherwise, confusion and uncertainty will continue to overshadow systematicity and reality. Probably, one of the best ways to investigate the plausibility of a given theory is to test the hypotheses generated from the theory. And one of the most defensible ways of testing a hypothesis may be attempting to quantify the relationship between the variables in the hypothesis. Thus, quantification, testing, and measurement constitute a necessary part of experimental investigation of theories. Language is one of the most unique characteristics of human beings and is involved in almost all mental activities in one way or another. Various theories have been developed to explain and/or account for numerous facets of language behavior. Language structures, language use and functions, language acquisition and/or learning, and language instruction are some of the interrelated domains of investigation. To evaluate the extent and nature of learner competence in specific areas, various types of tests have been developed. The construction of tests for various skills, modes, and components of language, however, has led to a confusing situation in language testing.

84

On the plausibility of unitary language proficiency factor

Hossein Farhady

One reason for confusion may be the complex interrelationships among language processing tasks. For example, skills such as reading, writing, listening, and speaking, or processes such as recognition, comprehension, and production are so closely interrelated that separating them from each other (not to mention from other skills) is an almost impossible task. For this reason, many tests may be required to obtain a comprehensive picture of the degree of a learners competence in language behavior. Another source of confusion for the would-be tester of abilities is the plethora of tests with different names and supposed functions. But the tester must beware of the fact that bearing a certain name does not guarantee that the test actually measures whatever it is named. For example, calling an instrument a test of listening comprehension does not guarantee that it is actually measuring listening comprehension. More importantly, a listening comprehension test probably taps not one and only one aspect of language behavior but rather a combination of many elements. Language processing in any modality is probably a more integrated phenomenon. The diversity of the dimensions of language behaviors, as well as of the tests, and the interrelationship among the dimensions have led to duplication of efforts as well as arbitrary categorization of hypothesized traits. Of course, it is not an easy task to isolate and identify the traits. It is possible, however, to utilize statistical methods to determine the degree of relationship and/or overlap among tests. This approach will help eliminate redundancy, and it will aid in the development of tests that are representative of groups of traits (Guilford, 1954). In this way, the task of assessing human capabilities in general, and language abilities in particular, will be simplified. Fortunately, research in language testing has been moving in the desired direction. Recent investigations have been carried out to simplify the task of language testing by determining representative tests. The most common statistical technique used to examine the traits underlying language tests has been factor analysis. Factor analysis is a whole array of interrelated statistical procedures which allow researchers to investigate the intercorrelations among observed variables and to group them in relation to one or more underlying hypothetical factors. However, because of its versatility, factor analysis has been overused and some of its fundamental assumptions have been overlooked. One of the major issues that has emerged from factor-analytic studies has been the unitary language proficiency hypothesis the claim that there is a unitary factor of language proficiency which accounts for almost all variations in almost all language processing tasks. Oller has presented data from numerous sources in support of this hypothesis (Oller, Chapter 1, 1978, 1979b; Oller & Perkins, 1978, 1980). Though these reports have been based on generally well-designed and carefully conducted research, questions have been raised about the appropriateness of the interpretations offered for the results obtained (Vollmer, 1979, 1980, 1981; Briere, 1980; Abu-Sayf, et al. 1979; Farhady, 1979, 1980a, 1980b; and also see the other chapters of this section and their references). The purpose of this paper, then, is to critically examine the previous reports and question the correctness of the statements made about the unitary factor hypothesis. In order to provide an accurate perspective on the issue, some theoretical and technical

85

On the plausibility of unitary language proficiency factor

Hossein Farhady

clarifications are warranted. First, a brief explanation of some concepts such as reliability, communality, and specificity, which are crucial to explaining the results of factor analysis, will be provided. Second, a brief description of the theory of factor analysis and various alternative factor-analytic methods will be given and data from different sources will be compared in order to justify the most appropriate method of data analysis. And finally, the implications of alternative interpretations of the relevant findings will be discussed.

Definition of Terms
In order to clarify the process of factor analysis, it is necessary to explain the terms reliability and communality, as well as the process of analyzing test variance into common, unique, specific, and error components. Because each of these concepts has often been the topic of technical papers and books, I will try to avoid theoretical complexities and to explain the functions and relationships of the terms in a relatively non-technical way. Reliability Reliability refers to the consistency of scores obtained from an instrument on its repeated administrations to the same person or group of persons. Psychometrically, reliability is the proportion of standardized variance which can be consistently and systematically obtained. If the scores on a test are standardized, the total variance produced by that test will be unity (i.e., total variance = 1). Thus, the reliability will be that portion of unity which is consistently observed. If a test is perfectly reliable, the reliability coefficient will equal unity. In most cases, however, the reliability coefficient will be considerably greater than zero but less than unity. The difference between unity and the reliability coefficient is referred to as the error variance. It is variance that cannot be attributed to nonrandom sources. So, we have: Reliability (rel.) + error variance (Ve) = 1 or rel. = 1 Ve For example, if the reliability coefficient for a test is reported to be .81, the error variance will be: Ve = 1 - .81 = .19 Communality Communality (h2) refers to the amount of variance which is shared by two or more variables. Its magnitude for two tests is simply the square of the correlation coefficient between the two tests. For example, if the correlation coefficient between two tests is .80, the common variance between them will be: h2 = (.80)2 = .64

86

On the plausibility of unitary language proficiency factor

Hossein Farhady

This means that 64 percent of the variance is common between the two tests and is accounted for by either of these test. If more than two tests are involved, which is usually the case, communality is computed by methods more complex than simple correlation, though the concept remains unchanged. Specific and Unique Variance Depending on the degree of correlation between two or more measures and depending on their respective reliabilities, there may be in each test a portion of reliable variance over and above the communality. This portion of variance by definition is not shared by any other test in the analysis. This component is referred to as the specific variance (Vs) and its value is the difference between the reliability (the maximum amount of explainable variance) and the communality (common variance among the tests) of the test in question: Vs = rel. h2 For example, if two highly reliable tests (.90 and .95, respectively) are moderately correlated (.75), there will be a considerable amount of specific variance in each Vs = rel. h2 Vs (for test 1) = .90 (.70)2 = .90 - .49 = .41 and Vs (for test 2) = .95 - .49 = .46 Therefore, there is a close relationship between the components of reliable, common, and specific variance. One of the uses of factor analysis is to decompose test variance into factor components in a meaningful way. It should be noted that in addition to reliable variance, there is always (in fallible tests) error variance as well. The combination of the Vs, which is unique to a particular test, with its error variance (Ve) gives the total unique variance Vu. This quantity, Vu, is to be differentiated from the specific variance: Unique variance (Vu) = Vs + Ve Different Components of Variance Thus, the variance in a test can be decomposed into three components, common variance, specific variance, and error variance: Total variance (Vt) = h2 + Vs + Ve To determine the value of h2, we simply compute the sum of squares of factor loadings, which are in fact correlation coefficients among factors and variables. For example, from a variable, its common variance (i.e., that shared with other variables) will be: h2 = a2 + b2 + c2

87

On the plausibility of unitary language proficiency factor

Hossein Farhady

Two tests can be said to provide the same information if their values on a, b, and c are the same and if the sum of squared loadings on a, b, and c for each test is equal to the reliability of that test. In such a case, the specific variance for each test will be zero, and the two tests can be used interchangeably. The next section describes the principles of factor analysis and how different methods operate to deal with different components of variance.

Principles of Factor Analysis

Factor analysis, in oversimplified terms, is a body of statistical procedures, based on correlation coefficients, used to investigate underlying patterns of interrelationships among observed variables. The main purpose of factor-analytic methods is to associate the variables with a smaller number of traits in order to define the variables in a more precise way. Factor analysis can be used to test a certain theory, in which case, it is referred to as confirmatory, or it may be used to seek out a convenient model for the structuring of variables, in which case, it is referred to as exploratory factor analysis. As with other statistical methods, factor analysis depends heavily both on rigorous mathematical foundations and on theoretical interpretations. The computational side of factor analysis is determined with mathematical precision, but how to interpret the findings falls within the scope of theory formation. An example may help clarify the point. Obtaining a certain correlation coefficient relates to the mathematical dimension. For instance, a given correlation coefficient may be determined mathematically to be statistically significant at a particular probability level. However, just what that significant correlation coefficient means cannot be determined by statistics alone. We must formulate some theoretical explanation based on sound, but not mathematically rigorous, reasoning. It is usually on the theoretical side that controversies arise because theory formation is influenced by many extraneous factors such as the predispositions of the investigator, the purpose of the analysis, the statistical hypotheses, the expected and hoped-for implications of results, and so forth. In simple terms, the theory dimension is partly a matter of taste rather than mathematical rigor. Unfortunately, divergence between mathematical and interpretive dimensions of factor analysis may be more marked than in other statistical procedures. Controversies among scholars in interpreting the results of factor analysis have led to the development of various factor-analytic methods and auxiliary techniques. Although all the common methods can be mathematically justified, their outcomes may vary significantly in regard to theoretical interpretations. Thus, numerous conflicts arise. Some of the common disagreements on the interpretive side of factor analysis concern (1) how to do the initial factor extraction, (2) how to decide on the number of factors to be extracted, and (3) how to arrive at a final solution by applying rotational techniques to the extracted factor structures. In spite of disagreements on these issues, there are procedures upon which most scholars agree though it is true that their agreement consists of suggested preferences rather than mathematical necessities.

88

On the plausibility of unitary language proficiency factor

Hossein Farhady

Extracting Initial Factors Until recently, the centroid method was the most commonly used method for extracting initial factors. However, owing to the increased availability of computers, the principal-axes method, which is mathematically more sophisticated (though computationally more complex) than the centroid method, has become the most often used method of initial factor extraction. The principal-axes method in actual practice consists of two different techniques: principal component analysis (PCA) and principal factor analysis (PFA). There are two major differences between PCA and PFA. In PCA, the values in the diagonal entries of the correlation matrix are somewhat arbitrarily set at unity. This means that all of the variance generated by the tests used to obtain the correlations is entered into the analysis. Thus, common, specific, and error variance will be used by PCA to define the factors (in this case, they should be called components). In PFA, on the other hand, estimated communalities are assigned to the diagonal cells of the original correlation matrix. Thus, specific and error variance components are not included in the analysis. A PCA matrix is illustrated in Table 1 and a PFA matrix in Table 2. The second major difference between PCA and PFA is the process of factor extraction. In PCA, the factor loadings are extracted form the 1s in the diagonals. In PFA, by contrast, an iterative (successive approximation) method is used to refine estimates of the communalities to some predefined level of accuracy, and then, these values are placed in the diagonal of the correlation matrix. The iterative approach is designed to obtain the best possible estimates of the communalities for various steps of factor extraction. There are actually several acceptable ways of accomplishing this (Harman, 1976).
Table 1. Correlation Matrix for Principal-Component Method Variable 1 2 3 4 5 6 7 1 1 r21 r31 r41 r51 r61 r71 2 r12 1 r32 r42 r52 r62 r72 3 r13 r23 1 r43 r53 r63 r73 4 r14 r24 r34 1 r54 r64 r74 5 r15 r25 r35 r45 1 r65 r75 6 r16 r26 r36 r46 r56 1 r76 7 r17 r27 r37 r47 r57 r67 1

Table 2. Correlation Matrix for Principal-Factor Method Variable 1 2 3 * r12 r13 1 r21 * r23 2 r31 r32 * 3 r41 r42 r43 4 r51 r52 r53 5 r61 r62 r63 6 r71 r72 r73 7 *Communalities are assigned to these cells. 4 R14 R24 R34 * R54 R64 R74 5 r15 r25 r35 r45 * r65 r75 6 r16 r26 r36 r46 r56 * r76 7 r17 r27 r37 r47 r57 r67 *

89

On the plausibility of unitary language proficiency factor

Hossein Farhady

Since the main purpose of examining the underlying factor patterns of a set of variables is usually to analyze the common variance among these variables, most factor analysts prefer PFA over PCA for initial factor extraction because PFA uses only common variance among the variables in the analysis while systematically discards uniquenesses (i.e., both specific and error variances). Comrey (1973), one of the advocates of this method, argues against PCA and for PFA: Inserting values in the diagonal cells that exceeds the correct communalities results in extraction of extra specific and error variance. The more the diagonal cells are inflated, the more pronounced is this distortion. The result of such distortion is to obtain factors that mix up common and unique variance in an inextricable way that obscures the view of what the variables have in common with each other (p.98). Comrey is saying that PCA (using unities in the diagonal) will not provide as accurate a picture of factor patterns as PFA will. In other words, the magnitudes of factor loadings will almost always be inflated in a PCA solution.i Therefore, it seems reasonable to prefer PFA. Once the method of extracting the initial factors is decided, the next step is to determine the number of factors to be extracted. Determining the Number of Factors We must bear in mind that one of the purposes of factor analysis is to reduce the number of observed variables. Unfortunately, there is no mathematically determinate method for deciding on the number of factors to be extracted. However, there are some guidelines that may help investigators to make sound decisions on this question. Obviously, the maximum number of factors extracted must be less than the number of variables included in the analysis or the procedure will defeat its own purpose. To reduce the number of factors, two common approaches are followed: we may rely on a specified eigenvalue and/or a specified factor loading. The eigenvalue of a factor is the sum of squared loadings of input measures explained by that factor. It is therefore an index of the relative importance of the factor in explaining the total variance of all the variables. If a given factor were a perfect explanatory variable, it would have an eigenvalue equal to the number of variables. Since factors are extracted initially one by one, if the required magnitude of the eigenvalue is specified in advance, then, any factor with an eigenvalues less than the required amount will be excluded from the analysis. For example, if the eigenvalue is specified as unity, the factor extraction process will terminate when the eigenvalue for the next factor is less than 1. It should be noted that in a PFA solution, it makes sense to use a cutoff eigenvalue of less than unity because the communalities of interest (that is, the values in the diagonals) are less than unity.ii Therefore, it is reasonable to use a value less than the traditionally used unity value to terminate factor extraction in any PFA solution. Another basis for limiting the number of initial factors is to examine the strength of the loadings of variables on those factors. It is commonly accepted that if the loading of a variable on a factor is less than .30, that factor loading can be safely ignored.

90

On the plausibility of unitary language proficiency factor

Hossein Farhady

Thus, factors with loadings of .30 or less can be eliminated, and this criterion can be used to terminate factor extraction. When a judgment must be made concerning a larger or smaller number of initial factors, it seems reasonable to start with the larger number because it is better to examine all the meaningful factors at the start and then eliminate unnecessary ones rather than to exclude meaningful factors from the beginning without careful examination. Eliminating any factor without sufficient care may well result in loss of information and distortion of the final outcome. Rotating the Initial Factors Probably, the most important step in factor analysis is the rotation of the initial factor structures. Regardless of the method of factor extraction and the number of factors to be extracted, almost all factor analysts unanimously agree that in order to obtain psychologically meaningful factor patterns, the initial factor structures should be rotated.iii Although unrotated factors are mathematically as accurate as rotated factors, they are hardly as useful for scientific purposes (Comrey, 1973; Nunnally, 1967; Guilford, 1954; Guilford & Fruchter, 1973; Harman, 1976, and also see their references). The major reason for using rotation is to achieve a simpler factor structure, preferably with each variable loading primarily on only one factor, and each factor accounting for a maximum of the variance generated by the variables that load on it. The PFA technique extracts the first factor in such a way as to account for the maximum amount of variance in each and all of the variables. A given variable may actually be better explained by two factors other than the first, but the first factor may still account for a substantial portion of variance in that variable which may cause it to look uncorrelated with the other factors. The procedure of getting the initial factors is such that the second factor will necessarily account for less total variance (i.e., have a smaller eigenvalue) than the first factor, and the third will explain less than the second, and so forth. The procedure of factor extraction (by PFA or PCA) is similar to a stepwise regression. The first factor variance is extracted, and so on. The algorithm will cause each successive factor to account for less total variance than its predecessor. This means that the initial unrotated factors may not give the best picture of the factor structure. At each step, variance from many different common factor sources is being extracted because the factor vector is placed in such a way that as many of the variables as possible have substantial projections on it (Comrey, 1973, p.103). Figure 1 illustrates how the first factor may in effect usurp variance that would otherwise fall to two other uncorrelated factors, namely, X and Y in the figure which fall on the horizontal and vertical axes at 45-degree angles from factor 1 which would be extracted by PFA. It is an unfortunate possibility that in the first step of PFA, the first factor can be, and usually is, a composite of variance components usurped in just this way from clearly distinct factors. In such cases, taking the first factor as a general factor is a mistake. There is no easy solution for the problem, but the one approach that most factor analysts recommend is the rotation of the initial factor structures.iv Though there is no

91

On the plausibility of unitary language proficiency factor

Hossein Farhady

mathematical justification for rotating the factors, rotating the initial factors is expected to lead to psychologically more interpretable factor structures. For example, if we rotate the factors illustrated in Figure 1 (approximately 45 degrees), we obtain two distinct and probably more meaningful factors at positions X and Y. Figure 2 shows the result of such a rotation. The language variables in Figure 2 load heavily on factor 1 and the body-measurement variables load heavily on factor 2. This seems to make more sense theoretically than having a general factor as shown in Figure 1.

92

On the plausibility of unitary language proficiency factor

Hossein Farhady

Of course, determining the kind of rotation technique to use will depend on many considerations such as the nature of the variables included in the analysis and the kinds of interpretations that the researcher might want to make. However, for many reasons, orthogonal solutions which arrive at uncorrelated rotated factors are preferred (see Hinofotis, Chapter 8, for discussion of orthogonal versus oblique rotations). To examine the influence of rotation, it will be helpful to investigate differences in the outcome of rotated and unrotated solutions. Data are presented below from various versions of the UCLA English as a Second Language Placement Examination (ESLPE) administered in the fall quarters of 1977, 1978, and 1979. A principal factor solution with iterations was used to extract the initial factors. Then, the number of factors was set at four after eliminating non-significant factors. The results of rotated and unrotated solutions are presented in pairs of tables for each of the three data sets to demonstrate as clearly as possible the difference in the factor structures obtained. Examining these pairs of tables, it is clear that in unrotated factor structures (Tables 3a, 4a, and 5a), as we would expect, the first factor in each case, accounts for a large proportion of the total variance. We might therefore be tempted to say that no other factors are needed to account for a large proportion of the total variance. We might therefore be tempted to say that no other factors are needed to account for the data. After all, the additional factors do not account for a significant amount of variance if the factor structure is left in its initial form. But as we saw in Figure 1, this may be an artifactual result. Indeed, when the initial factors are rotated, entirely different factor patterns appear.v The rotated factor structures (Tables 3b, 4b, and 5b) show that the first unrotated factor was highly inflated and that the subsequent factors were correspondingly deflated, giving us a distorted view. Factor 1 is not so powerful as it

93

On the plausibility of unitary language proficiency factor

Hossein Farhady

seems in Tables 3a, 4a, and 5a, and factors 2, 3, and 4 are not so weak as they seem. This comes clear in the rotated solution given in Tables 3b, 4b, and 5b. It should be noted that the total amount of variance accounted for in either type of solution is unchanged from the initial to the rotated solution. They are explaining in fact the same total variance but distributing it in different ways. The crucial point is that the first factor can pull out variance from several unrelated variables and may therefore lead to misinterpretations. In an unrotated factor structure, the first factor by the very nature of the extraction procedure, will account for the greatest amount of variance possible and thus will be a composite of multiple unrelated factors.
Table 3a. Unrotated Factor Matrix Using Principal Factor with Iteration for the Fall 77 ESLPE Subtests Subtests Spelling Punctuation Dictation Cloze Listening comprehension Verbs Prepositions Articles Vocabulary romance Vocabulary germanic Reading comprehension F1 .69 .40 .86 .79 .71 .80 .80 .72 .77 .81 .88 F2 .59 .63 * * * * * * * * * F3 * * * * .35 * * * .37 * * F4 * * * * * * * * * * *

Table 3b. Varimax Rotated Factor Matrix Using Principal Factor with Iteration for the Fall 77 ESLPE Subtests Subtests Spelling Punctuation Dictation Cloze Listening comprehension Verbs Prepositions Articles Vocabulary romance Vocabulary germanic Reading comprehension *Loadings less than .30 are deleted. F1 * * .45 .50 .37 .49 .67 .46 .77 .52 .59 F2 .83 .74 .52 .58 * * * * * * * F3 * * .55 .35 .78 .45 .39 .52 * .42 .38 F4 .36 * * * * .49 * .34 * .62 .34

Table 4a. Unrotated Factor Matrix Using Principal Factor with Iteration for the Fall 78 ESLPE Subtests Subtests Cloze Dictation Listening 1 Listening 2 Reading 1 Verbal 1 Prepositions Articles Verbal 2 Reading 2 Listening 3 F1 .42 .46 .71 .80 .80 .72 .77 .81 .88 .63 .76 F2 .83 .51 * * * * * * * * * F3 * * .35 * * * * * * * * F4 * * * * * * * * * .36 *

94

On the plausibility of unitary language proficiency factor

Hossein Farhady

Table 4b. Varimax Rotated Factor Matrix Using Principal Factor with Iteration for the Fall 78 ESLPE Subtests Subtests Cloze Dictation Listening 1 Listening 2 Reading 1 Verbal 1 Prepositions Articles Verbal 2 Reading 2 Listening 3 Loadings less than .30 are deleted. F1 * * .76 .75 .33 * .50 .36 .38 .30 .67 F2 * * * .35 .49 .73 .54 .63 .74 * .31 F3 .90 .69 * * * * * * * * * F4 * * * * .54 * * .37 .33 .61 .31

Table 5a. Unrotated Factor Matrix Using Principal Factor with Iteration for the Fall 79 ESLPE Subtests Subtests Cloze Dictation LCV LCW RC Grammar (verbs) Grammar (prepositions) Grammar (others) Functional LCV = listening comprehension: visual LCW = listening comprehension: written RC = reading comprehension F1 .80 .81 .64 .72 .74 .81 .83 .79 .80 F2 * * .45 .36 * * * * * F3 -.30 * * * * * * * *

Table 5b. Varimax Rotated Factor Matrix Using Principal Factor with Iteration for the Fall 79 ESLPE Subtests Subtests Cloze Dictation LCV LCW RC Grammar (verbs) Grammar (prepositions) Grammar (others) Functional *Loadings less than .30 are deleted. LCV = listening comprehension: visual LCW = listening comprehension: written RC = reading comprehension F1 .51 .35 * .35 .53 .78 .75 .70 .66 F2 * .58 .74 .71 .31 * .39 * .38 F3 .65 .55 * * .46 .30 * .35 .30

This might raise controversies about the utility of rotation. One might argue that as long as the amount of explained variance remains the same and/or increases insignificantly, there is no need to rotate the factors. However, how the variance is

95

On the plausibility of unitary language proficiency factor

Hossein Farhady

explained logically outranks the question of the magnitude of the explainable variance.vi The reason is that decomposing the common variance into appropriate factor loadings is precisely the purpose of factor analysis. An analogy may help clarify the point. If someone sees a green area in the mountains from a distance, the most likely initial reaction is that the green area is a bunch of trees. Even if the person knew how many trees were there, this would not allow him to conclude that all the trees were of the same kind. The green area might be composed of trees of different kinds. Without a closer look, it might be quite incorrect to claim that they are all of the same kind. The same thing may happen in factor analysis. The first factor may be the most obvious sign of interrelationship among the variables, but without further and more detailed examination, to claim that there is only one factor would also be misleading. The data presented here are consistent with almost all previous reports on the results of factor analyses. Whenever the investigator(s) did not rotate the initial factor matrix, the first factor appeared to be so strong that the researcher inferred, mistakenly, the existence of one and only one general factor accounting for essentially all the explainable variance in the data. However, when the same researchers rotated the initial factors, various factor patterns appeared from the same data (see Oller & Hinofotis, 1980; Scholz, et al. 1980; and other entries in the same volume). Thus, the hypothesis of a unitary language proficiency factor will not be supported if one follows the appropriate steps in conducting the relevant factor analyses. If, on the other hand, one uses incomplete methods, it will appear, in study after study, that the first factor, whatever it may be called, is the only factor underlying the variables. Therefore, previous interpretations of unrotated factor matrices are called into question and further investigation is required to determine the actual composition of language proficiency. If the appropriate steps are followed, one will be drawn to conclude that the unitary language proficiency hypothesis is not plausible. However, it is still possible that owing to the nature of language skills and their interrelationships, a general factor may exist that accounts for a considerable amount of variance in a large variety of language processing tasks. But this is not the same as arguing for a unitary factor. A general factor would not exhaust all of the reliable variance. Only that portion that all language skills share will be manifested in the common factor. This does not mean, however, that a specific factor related to individual language skills does not exist. These specific factors can be expected to account for a portion of variance over and above the common variance. Another way to test for the existence of specific variance and thus to examine the plausibility of an exhaustive unitary language proficiency factor is to compare the magnitude of the reliability coefficients with the values of communalities. The difference between the two, if substantial, will directly reject this hypothesis. A close examination of the data, presented either here or in the literature, indicates that there is specific variance for almost all the tests included in the analyses. Regardless of the accuracy of the type of factor analysis being used, and regardless of

96

On the plausibility of unitary language proficiency factor

Hossein Farhady

the application of rotation to the initial factor structures, the differences between the reliability coefficients and the reported communalities are evidence for the existence of specific variances, which refutes the unitary hypothesis. For the purposes of illustration, the communalities and the reliability coefficients for the data discussed earlier in this paper are reported in Table 6, 7, and 8. Reliability coefficients are calculated using KR-21 for cloze and dictation, and Cronbachs alpha for other subtests. It should be noted that internal consistency reliability coefficients for cloze and dictation are not strictly appropriate, as I have argued elsewhere (Farhady, 1979, 1980a, and see also Chapters 14 and 17 of this volume). However, since such estimates would err on the high side for cloze and dictation, it would only strengthen the argument at stake here to use methods giving lower reliabilities and thus higher specificities. It can be observed from the data presented in Tables 6, 7, and 8 that most of the tests have specific variances. In some of them, the specificity is substantial. Of course, these results should be compared with those from other studies, but it seems to me that looking at language tests from the unitary perspective is not defensible.
Table 6. Communalities and Reliability Coefficients for the Fall 77 ESLPE Subtests Subtests Spelling Punctuation Dictation Cloze Listening comprehension Verbs Prepositions Articles Vocabulary romance Vocabulary germanic Reading comprehension * KR-21. : Alpha. Communality .88 .57 .81 .72 .81 .74 .69 .62 .68 .87 .67 Reliability .88* .60* .93* .83* .84 .86 .77 .70 .88 .89 .89 Specificity .00 .03 .12 .11 .03 .12 .08 .08 .20 .02 .22

Table 7. Communalities and Reliability Coefficients for the Fall 78 ESLPE Subtests Subtests Cloze Dictation Listening 1 Listening 2 Reading 1 Verbal 1 Prepositions Articles Verbal 2 Reading 2 Listening 3 * KR-21. : Alpha. Communality .87 .56 .69 .76 .71 .66 .62 .70 .84 .56 .70 Reliability .97* .98* .74 .77 .85 .71 .68 .72 .89 .67 .73 Specificity .10 .42 .05 .01 .14 .05 .06 .02 .05 .11 .03

97

On the plausibility of unitary language proficiency factor

Hossein Farhady

Table 8. Communalities and Reliability Coefficients for the Fall 79 ESLPE Subtests Subtests Communality Cloze .70 Dictation .75 LCV .62 LCW .66 RC .57 Grammar (verbs) .75 Grammar (prepositions) .76 Grammar (others) .67 Functional .66 * KR-21. : Alpha. LCV = listening comprehension: visual LCW = listening comprehension: written RC = reading comprehension Reliability .71* .93* .67 .74 .78 .77 .77 .78 .78 Specificity .01 .18 .05 .08 .21 .02 .01 .11 .12

Conclusions
Various conclusions can be drawn. First, factor analysis is complex and should be applied with great care. Second, the most often recommended technique for factor extraction is the principal-factor solution, which uses the communalities, estimated through iteration, in the diagonals of the correlation matrix. Third, initial factor structures should be rotated in order to obtain maximally meaningful and interpretable factor patterns. And finally, the results should be interpreted only after the completion of the sequence of required steps. This implies that the results obtained from incomplete factor analyses are questionable and need to be reanalyzed and reinterpreted in terms of the steps recommended in this paper and in many standard resource books on factor analysis (see Harman, 1976). Therefore, alternatives other than the unitary-factor hypothesis, some of which were discussed in this paper, and perhaps others not touched on here, should be pursued to improve our knowledge of the nature of language proficiency.

I wish to thank Frances Hinofotis, Andrew Cohen, and Ebrahim Maddahian for their comments on an earlier draft of this paper.

* This is the revised version of the paper printed in W.J. Oller, Jr. (ed.) (1983). Issues in language testing research. Rowley, Mass.: Newbury House Publishers, Inc.

Editors Notes
In Chapter 9, below, Upshur & Homburg discuss the results of applying a statistical correction for the inflation of PCA loadings. Their method comes form Kazelskis (1978). It yields corrected estimates which are quite comparable with those obtained from PFA. However, it should be noted that the adjustments do not by any means remove the general factor that is in dispute. What they do is reduce its apparent strength, but only slightly.
ii
i

This stems from the logic that an eigenvalue which accounts for less variance than that which is found in any single input variable is of little or no interest. This is in keeping with the general aim of factor analysis to reduce the number of constructs to be taken into account. In PCA, with unities on the

98

On the plausibility of unitary language proficiency factor

Hossein Farhady

diagonal of the correlation matrix, if the allowed eigenvalues were much less than unity (or in PFA where communalities are placed on the diagonal, if they were much less than the mean communality for the input variables), the number of factors extracted would exceed the number of variables to be taken into account. But what if only one factor appears? In such a case, rotation makes no sense. Rotation is sensible only if there are two or more factors over which the variance from the several contributing variables may be distributed in a meaningful way. However, one of the problems which the experts readily acknowledge is that there are always innumerable ways of doing this (Harman, 1976, p.19). Not all of them will be equally appealing in theory, but all will be equally defensible mathematically. The potential escape from this aspect of the rotational dilemma was thought to be a virtue of the single factor solutions in Chapter 1. However, it is acknowledged that factoring methods without rotation are applicable only when in fact only one factor can be discerned even then, PFA is to be preferred over PCA, as Farhady argues. The confirmatory techniques discussed in Chapters 6 and 7 below are also promising methods which offer more appropriate model testing capabilities. The use of any rotational procedure, however, presupposes a multiple-factor solution. However, the assumption underlying the factoring done in Chapter 1 was to attempt to assess the strength of that hypothesized factor. Must we reject as unconscionable any basis for looking for just such a general factor? Was not even the strongest version of this possibility worth examining if only to rule it out? However, the appearance of the different patterns in the rotated solutions is also a function of the statistical procedure to some extent, is it not, just as the large first factor is in the unrotated solutions? Therefore, isnt it possible to prefer rotated solutions without necessarily rejecting the possibility of a general factor?
vi

iii

iv

But isnt the magnitude of explainable variance always an important issue? Is any theorist interested in elegant explanations of negligible factors?

99