For instance, it is not reasonable to interpret a low correlation between two tests as an indication that the two tests are measuring different skills and also that neither of them is reliable (or that one of them is not reliable). Since reliability is prerequisite to validity, a given statistic cannot be taken as an indication of low reliability and high validity (yet this is sometimes suggested in the literature as we will see bellow). The observed low correlation could result if both tests were in fact measures of the same basic factor but were both relatively unreliable measures of that factor. B. Two common misinterpretation of correlations A low correlation between a Grammar test and a vocabulary test might well be the product of poor tests rather than an independence of hypothesized components of proficiency.
In an important sense, knowing a word is knowing how
to use it in a meaningful context that is subject to the normal syntactic (and other) constraints of a particular language. B. Two common misinterpretation of correlations Is reading comprehension subtest more of a measure of reading ability that it is of writing ability or grammar knowledge or vocabulary or mere test-taking ability or general proficiency factor or intelligence? The fact that the reliability coefficients are higher than correlations between different part scores is no proof that the tests are measuring different kinds of knowledge. In fact they may be measuring the same kinds of knowledge and their low intercorrelations may indicate merely that they are not doing as they could. B. Two common misinterpretation of correlations In any event, it is axiomatic that validity cannot exceed reliability indeed the general rule of thumb is that validity coefficients are not expected to exceed the square of the reliabilities of the intercorrelated test (Tate, 1965). If a certain test has some error variance in it and a certain other test also has some error variance in it, the error is apt to be compounded in their intercorrelation. Therefore, the correlation between two tests can hardly be expected to exceed their separate reliabilities. It can equal them only in the very special case that the tests are measuring exactly the same thing. B. Two common misinterpretation of correlations Surprisingly high correlations have been observed between a wide variety of testing techniques with a whole family of procedures under the general rubric of cloze testing, dictation, elicited imitation, essay writing, and oral interview. Populations have ranged from children and adult second language learners, to children and adult tested in their native language. What then can be made of such high correlations? B. Two common misinterpretation of correlations Two interpretations have been offered: 1. One of them argues that the strong correlations previously observed between cloze and dictation, for instance, are merely indications of the reliability of both procedures and proof in fact that they are both measuring basically the same thing 2. A second interpretation is that the high correlations between diverse tests must be taken as evidence not only of reliability but also of substantial test validity.