Vous êtes sur la page 1sur 14

64 Academy of Management Perspectives November

General Mental Ability, Job Performance, and Red


Herrings: Responses to Osterman, Hauser, and
Schmitt
by Frank Schmidt, Huy Le, In-Sue Oh, and Jonathan Shaffer

Executive Overview
We respond to the three comments on our recent article in the August issue of Academy of Management
Perspectives (Le, Oh, Shaffer, & Schmidt, 2007), which highlights the importance of methodological
advances in human resource research. By concentrating on tangential aspects of our article, these
comments miss its central points. Further, the comments (by Paul Osterman, Neal Schmitt, and Robert M.
Hauser) either involve quibbles and misinterpretations of research evidence or are downright erroneous.
We clarify the misunderstandings by providing evidence that is well-established through decades of
research in the area of industrial/organizational psychology. By doing so, we re-emphasize that (a) the utility
and validity of general mental ability are among the most robust findings in psychological research, (b) such
findings were realized by the development of the meta-analysis method, and most important, (c) organi-
zations can benefit greatly by utilizing findings obtained from meta-analysis.

I
n our recent article in Academy of Management (Hauser, 2007; Osterman, 2007; Schmitt, 2007),
Perspectives (Le, Oh, Shaffer, & Schmidt, 2007), who, while not disagreeing with our central mes-
we discussed the importance of methodological sage regarding the implications of the develop-
advances— especially the development of meta- ments of meta-analysis to organizations, con-
analysis methods—to the practices of personnel tended that our example “overemphasized” the
selection and demonstrated how such develop- importance of GMA in personnel selection. In the
ments can help bridge the long-standing gaps be- current article, we respond to the comments and
tween academic research and organizational prac- show that they are not only tangential to the
tices. The utility of general mental ability central messages in Le et al. (2007), but also
(GMA), one of the most well-established findings potentially misleading to readers about the utility
in industrial/organizational psychology, was en- of GMA and the usefulness of meta-analysis.
listed as an example to illustrate the fact that
meta-analytically derived findings can provide
Response to Osterman
guidance enabling practitioners to achieve posi-

T
he purpose of our article was to describe the
tive outcomes for their organizations. The article
usefulness of meta-analysis in the area of per-
was met with strong reactions from three com-
sonnel selection. We discuss the impact of
mentators from different areas of research
meta-analysis on conclusions about a variety of
selection methods. But Osterman’s (2007) com-
Author Note: Except for the first author, the other three authors mentary strangely consists almost entirely of at-
contributed to the article equally. Authorship is listed based on alphabet-
ical order. tempts to cast doubt on the validity and value of
*Frank Schmidt (frank-schmidt@uiowa.edu) is the Ralph L. Sheets Professor, Department of Management and Organizations, Tippie
College of Business, University of Iowa.
Huy Le (hale@mail.ucf.edu) is Assistant Professor, Department of Psychology, University of Central Florida.
In-Sue Oh (in-sue-oh@uiowa.edu) and Jonathan Shaffer (jonathan-shaffer@uiowa.edu) are Ph.D. students, Department of Management
and Organizations, Tippie College of Business, University of Iowa.

Copyright by the Academy of Management; all rights reserved. Contents may not be copied, e-mailed, posted to a listserv, or otherwise transmitted without the copyright holder’s express written
permission. Users may print, download, or e-mail articles for individual use only.
2007 Schmidt, Le, Oh, and Shaffer 65

tests of GMA in employee selection. We also note fact, when job sample measures (simulations of
that he engages in an ad hominem attack on one the job tasks—such as the “Excel spreadsheets,
of the authors of our article, by implying that sewing stitches, welding” referred to by Oster-
Schmidt’s (2002) citation of The Bell Curve is man)—are used instead of ratings, the validities
racist. He states that The Bell Curve is “seen by are larger, not smaller (e.g., cf. Hunter, 1986;
many sober scholars as racist (Fisher, 1996).” If so, Schmidt, Hunter, & Outerbridge, 1986). Validi-
how could it be cited approvingly in one of the ties for predicting objectively measured perfor-
top-rated American Psychological Association mance in job training programs are also larger.
(APA) journals (Journal of Personality and Social These findings clearly undercut Osterman’s argu-
Psychology, the 86th issue [2004]: Special Section: ment. Osterman also argues that the existence of
Cognitive Abilities: 100 Years After Spearman ethnic, gender, or other biases in ratings would
[1904])? Is APA racist? In fact, The Bell Curve is a invalidate validity estimates based on such ratings.
serious work of scholarship and is certainly ac- Actually, such biases would generally result in an
cepted as such by scholars in the relevant area of underestimation of validity. For example, suppose
individual differences psychology. Those who a supervisor is biased against female employees
have condemned it as “racist” are by no stretch of and so rates them lower. This is a systematic bias
the imagination “sober scholars.” We have read of the type discussed by Osterman. The effect of
many of these critiques and have found them to be this is to lower the validity estimate from what it
generally ideologically and politically motivated would have been had the supervisor not been
rather than scholarly and objective. Be that as it biased. If such biases routinely exist (and the
may, ad hominem attacks are not appropriate in literature is far from showing this), then this could
scholarly exchanges. be one explanation for why validities are larger
What about Osterman’s specific comments on when job performance is measured via an objec-
the role of GMA in selection? First, he incorrectly tive job simulation than when ratings are used.
states that our position is that “other approaches Osterman next states that even if we accept
to selection and promotion are biased and not research evidence on GMA validity, GMA still
helpful.” He states that one might legitimately does not account for much variance in job perfor-
wonder “whether other approaches might fruit- mance, pointing out that a validity of .57 “ex-
fully be utilized.” In fact, our article discusses a plains about 30% of job performance.” There are
variety of selection methods other than GMA four errors contained in Osterman’s statement.
that contribute to selection validity. For example, First (and least important), the percentage is 32%,
we cite Schmidt and Hunter (1998), which pre- not 30% (.57 squared ⫽ .324). Second, this is not
sents calibrated validities for many non-GMA se- a percentage “of job performance” but a percent-
lection methods and demonstrates that many of age of the variance across individuals in job perfor-
them are useful in personnel selection. Contrary mance—which is not at all the same thing. Third,
to Osterman’s implication, we do not in our arti- by all standards in psychological and social sci-
cle state or imply that GMA is the only useful ence, 30% of variance is a large effect (Cohen,
selection tool— only that it is the most valid one. 1977), not a small one. Finally and most impor-
Osterman next devotes considerable space to tant, percent variance accounted for is irrelevant
an attempt to discredit supervisory ratings of job in personnel selection. More than 60 years ago,
performance, the most commonly used measure of Brogden (1946; 1949) showed that the economic
job performance in validity studies. His assump- value (utility) of a selection method is directly and
tion is that if these ratings have accuracy prob- linearly related to the validity—and not to the
lems, then we must throw out the evidence for the square of validity. Brogden showed that a validity
validity of GMA. However, multi-method ap- of, for example, .57 produces 57% of the utility
proaches in science can be very useful. Conclu- gains that would be realized if one had (hypothet-
sions about GMA validity do not change when ically) perfect validity (a validity of 1.00). This
other measures of job performance are used. In fact is built into the current selection utility
66 Academy of Management Perspectives November

model, which was presented in our original article. system, to increase mean performance and produc-
An examination of the triple product utility tivity substantially, as detailed in Hunter and
model presented there shows it contains the uns- Schmidt (1982). It appears that Osterman over-
quared validity coefficient. In light of these errors, looked this important point. In light of this, it is
Osterman’s comments on this issue do not appear clear that the implication by Osterman that our
to have merit. article implies that other (i.e., non-selection)
The next contention by Osterman is that em- means of improving employee job performance are
ployers do not reward GMA monetarily, basing not useful is incorrect.
this conclusion on a single study by Neal and In summary, our analysis of Osterman’s com-
Johnson (1996). This study is contrary to a whole ments suggests to us that his criticisms are without
series of studies that find the opposite, many of merit. It appears that many of Osterman’s errors
which are cited in Schmidt and Hunter (2004). stem from a lack of familiarity with the research
For example, it has been found that among broth- literature related to personnel selection and indi-
ers raised in the same family, those with higher vidual differences—a very large literature.
GMA scores as youths go on to earn considerably
higher incomes later in life (Murray, 1998). It has
also been found that among new entries to the Response to Hauser

A
labor market, those with higher GMA scores tend s we stated earlier in our response to Osterman
to move up the occupational ladder and those (2007), the focus of our original article was
with lower GMA scores tend to move downward not the utility of general mental ability, which
on the occupation ladder (Wilk, Desmarais, & is widely accepted; rather, it was the value of using
Sackett, 1995; Wilk & Sackett, 1996). There are meta-analysis methods to answer questions rele-
many, many other such studies (e.g., Dreher & vant to organizations. Our demonstration of util-
Bretz, 1991; Judge, Higgins, Thoresen, & Barrick, ity analysis was just a vehicle for communicating
1999; Scullin, Peters, Williams, & Ceci, 2000). our message about the importance of meta-analy-
Because of this serious oversight by Osterman, we sis to organizational research and practices.
again see his comments as superficial and lacking Hauser (2007), however, focuses his entire cri-
credibility. tique on utility estimates. Indeed, he states that
In his final criticism, Osterman makes an error his reservations about our article “have nothing to
also made by Neal Schmitt (2007): the error of do with statistical uncertainty or the use of meta-
assuming that large validities for GMA imply that analysis. My skepticism arises from weak evidence
interventions such as training, employee socializa- and vague assumptions under-girding their esti-
tion, good supervision, etc., cannot produce any mates, questionable survey methods, and the ab-
important improvements in job performance. sence of convincing external validation” (p. 24).
What Osterman overlooks is that validity esti- Contrary to Hauser’s position, the utility estima-
mates refer to the prediction of individual differ- tion procedure described in our article has been
ences in performance in a context in which many supported by a vast body of research dating as far
other variables are held constant. That is, on any back as Brogden (1946, 1949). Because Hauser’s
given job when a validity study is conducted, remarks may mislead readers of this journal, we
these factors— quality of supervision, performance feel it is important to respond to his comments,
incentives, organizational climate, etc., which can even though estimation of the utility of GMA is
be considered as potential interventions—are not the main purpose of our original article.
held constant and hence cannot and do not pro- Hauser (2007) comments on all the compo-
duce any variance in employee job performance. nents of the utility equation (Brogden, 1949).
So even if, hypothetically, GMA accounted for all First, he questions the appropriateness of the pro-
of the variability in job performance (i.e., valid- cedure for estimating the SDy based on supervi-
ity ⫽ 1.00), it would still be possible for organi- sors’ ratings as described by Schmidt and col-
zational interventions, such as a new incentive leagues (Schmidt, Hunter, McKenzie, &
2007 Schmidt, Le, Oh, and Shaffer 67

Muldrow, 1979).1 He writes: “Without further Roth, Pritchard, Stout, & Brown, 1994). These
evidence, I am not persuaded by the validity of studies provide additional evidence that we hope
this protocol” (p. 25). Initially, we were perplexed would persuade Hauser of the validity of the pro-
by this comment but then noticed Hauser’s admis- tocol for estimating SDy by supervisors.
sion that other than checking the citations used in Hauser (2007) is further concerned about the
our original article, he “made no effort to search relevance of the SDy estimate because it was de-
the research literature for other relevant findings.” rived over 20 years ago, in the 1980s. He asks the
There are numerous studies (e.g., Becker & question: “Given the massive changes that have
Huselid, 1992; Bobko, Karren, & Parkington, transformed the American economy in the past
1983; Burke & Frederick, 1986; Eaton, Wing, & five decades, are these findings relevant to the
Mitchell, 1985; Reilly & Smither, 1985) that entire, contemporary workforce, as Le et al. would
compare the SDy provided by the procedure we have us believe?” (p. 25). This, we believe, is a
described to those obtained by different, more valid question. However, Hauser fails to further
sophisticated or objective procedures (e.g., cost- examine the implications of these changes for the
accounting procedures, Rothe, 1978; CREPID SDy estimate. It is generally agreed that the
[Cascio-Ramos Estimate of Performance In Dol- changes we have seen in the economy and in jobs
lars], Cascio & Ramos, 1986; direct estimates of make jobs more complex, challenging, and cogni-
the dollar value, Becker & Huselid, 1992). The tively demanding (Johnston & Packer, 1987; Na-
general conclusion obtained from these studies is tional Commission on Excellence in Education,
that different methods generally converge on sim- 1983). And the more complex the jobs become,
ilar estimates of the standard deviation of perfor- the larger is the variability of job performance
mance in dollar value for jobs and organizations. (Hunter, Schmidt, & Judiesch, 1990). As a result,
Similar estimates of the SDy have also been found the utility of GMA based on the value of SDy
in samples from other countries (e.g., from a Brit- suggested in Le et al. (2007) may actually be
ish sample, Smith, 1989). A more recent review underestimated.
(Roth, Bobko, & Mabon, 2002) reaffirms this Hauser (2007) also raises several questions
general finding. In fact, in some cases objective, about the assumptions underlying Brogden’s
direct procedures result in larger values for SDy in (1949) utility formula. Specifically, he challenges
comparison to values estimated by the Schmidt et the assumptions of normality (“Why would it be
al. (1979) procedure. For example, Becker and appropriate to assume that assessment scores are
Huselid (1992) reported direct estimates of the normally distributed, regardless of the job in ques-
SDy which range from 74% to 100% of mean tion?”), linearity (“or that effects of cognitive abil-
salary, as compared to the value of 40% suggested ity on performance are the same throughout abil-
by Schmidt et al. (1979), suggesting that the value ity distributions of differing applicant pools?”),
of 40% used in Le et al. (2007) may actually and random selection of applicants above cutoff
underestimate the utility of GMA. The reason we scores (“or that the application of criteria supple-
used the 40% rule in our original article is that it menting test performance would lead to random
has been found to be more easily understood and selection among applicants above the cut score?”).
accepted by managers (Hazer & Highhouse, Regarding the normality assumption, it should be
1997). There are also studies that examine the noted that the formula does not require this as-
factors supervisors take into account when esti- sumption (Schmidt et al., 1979). This assumption
mating the SDy (e.g., Cesare, Blankenship, & Gi- is introduced in most presentations of the model
annetto, 1994; Mathieu & Tannenbaum, 1989; only as a convenient way to estimate the average
standardized scores of those who were hired (Z៮ s).
1
Here Hauser makes a mistake when stating “the ratio of average salary This value, however, can be directly estimated
to mean output, which Schmidt and Hunter take to be exactly two” (p.25). from selected applicants, simply by arithmetically
This is not correct because the ratio, which was obtained from relevant
economic indices, is actually .57, not “exactly two” (see Schmidt & Hunter, computing the average test z-score of those hired.
1983). The assumption of linear relationship between
68 Academy of Management Perspectives November

ability and job performance has been empirically searchers are likely to use to make the correction.
confirmed and is well accepted in the literature Later, Sackett and Ostgaard (1994) empirically
(Coward & Sackett, 1990; Hawk, 1970), so the estimated the standard deviations of applicants for
fact that Hauser challenged this assumption with- many jobs and found that they are typically only
out either a thoughtful rationale or new empirical slightly smaller than that in the norm population.
evidence suggesting otherwise is quite confusing This finding led these researchers to refute Harti-
to us. Finally, the question about random selec- gan and Wigdor’s suggestion because it would
tion of applicants above a cut score is not relevant result in much more serious downward bias in
because we did not make such an assumption. estimation of validities as compared to the slight
Instead, our basis for the utility formula is that upward bias if range restriction correction is made
applicants are selected top-down based on test based on the SD obtained in the norm population.
scores. Of course, underestimation of validity leads to
However, it is at this point that Hauser’s con- underestimation of utility. In the case of the Har-
tention that the utility values we presented may tigan and Wigdor (1989) report, those underesti-
be overestimated has merit. As also noted in our mations were very substantial.
following response to Schmitt’s (2007) comments, Hauser (2007) further questions the appropri-
selection is rarely top-down on the predictor. Any ateness of using the validity coefficient rxy in the
other use of the test in selection, including ran- formula because the criterion used for estimating
dom selection from among applicants whose this validity is “not ordinarily a dollar value, but
scores are above a cutoff score, will result in re- some other measure of the quality or quantity of
duced utility. So it is true that the conventional work.” This is a valid question, but again, it ig-
practice of illustrating utility gains from selection nores evidences in the literature showing the link
by assuming top-down selection tends to overstate between rating of job performance and economic
such economic gains. However, the utility realized value (Becker & Huselid, 1992; Cesare, Blanken-
in real-world settings will still be substantial. Fur- ship, & Giannetto, 1994). Hauser goes on to
ther, the overestimation of utility due to assuming suggest an alternative, direct procedure to esti-
top-down selection is likely canceled out by the mate the link between GMA and some economic
underestimation of SDy, as noted earlier. outcome. Ironically, the only piece of empirical
The last component in the utility formula that evidence suggested by Hauser (2007) is poten-
Hauser (2007) criticized is the validity coefficient, tially misleading. He uses the Wisconsin Longitu-
rxy. As we noted in our original article, validity dinal Study sample to illustrate the effect of GMA
varies by type of selection method but is generally measured in adolescence on subsequent earnings.
one of the most robust and generalizable estimates Hauser then shows that GMA is predictive of
in the literature (Schmidt & Hunter, 1998). earnings, but that the effect disappears when ed-
Hauser alleges that “the NRC report remains valu- ucational attainment (measured by years of
able as a critical review of research on validity schooling) is taken into account. The finding,
generalization, and it rejects the claim that test- while interesting, is not new, as it had been found
based hiring could yield massive economic gains” in previous research (e.g., Scullin et al., 2000).
(p. 26). This statement, again, reveals that Hauser The more serious problem, however, is that such a
did not attempt to search for relevant evidence finding is not relevant to the question at issue. To
supporting his claims. There is in fact empirical estimate the utility of GMA for jobs in organiza-
evidence suggesting that some of the conclusions tions, we need to know the relationship between
reached in the NRC report are no longer relevant. GMA and performance for people within jobs (or
For example, Hartigan and Wigdor (1989) stated applying for a job). Hauser’s analysis provides the
that no correction for range restriction should be relationship between GMA and earnings obtained
made because the SD of the predictor (GMA test) from different jobs across different organizations.
in the applicant pools are generally smaller than Such information may have social and political
the SD in the norm population that most re- implications but is not relevant to jobs within
2007 Schmidt, Le, Oh, and Shaffer 69

organizations. The general finding is that for ap- edges that he is not very well informed about the
plicants to jobs, amount of education has low relevant literature, and we agree with his self-
validity for predicting job performance (cf. assessment. However, it is problematic to make
Schmidt & Hunter, 1998, for a summary of this comments based on such deficient knowledge.
research). Many of the questions he raises have been asked
Hauser appears to discount the effect of GMA and answered in the relevant literature decades
in favor of educational attainment in interpreting ago. Most of the evidence pertaining to Hauser’s
the study he cites. But research has shown that challenges comes not from obscure journals or
educational attainment partially mediates the ef- unpublished papers but from premier journals in
fect of GMA on people’s incomes as well as their psychology and management. Thus, it is puzzling
social status (Scullin et al., 2000). That is, GMA that Hauser ignores this evidence. Hauser’s com-
is a cause of educational level and income. As ments, if taken at their face value by readers, may
such, it should not be surprising that the effects of confuse managers and lead to a discounting of the
GMA on income or any other criterion are massive evidence from meta-analytic studies,
smaller when educational attainment is controlled thereby further aggravating the problem of bridg-
for (that is, statistically held constant). ing academic research findings and organizational
The findings cited by Hauser are different from practices. This was the very problem that was the
findings in the personnel selection literature, focus of our original article.
which clearly show that GMA is by far more valid
in predicting job performance than years of edu- Response to Schmitt

O
cation (e.g., Berry, Gruys, & Sackett, 2006; f the three writers who commented on our
Hunter & Hunter, 1984; Schmidt & Hunter, article, only Neal Schmitt has the training,
1998). This discrepancy can be reconciled if we knowledge base, and experience needed to
note that results from Hauser (2007) and Scullin comment on the research basis of our original
et al. (2000) are based on incomes obtained from article. The other two, by their own admission, are
different jobs, whereas available estimates of valid- not familiar with the relevant literature and have
ities for GMA and educational attainment no training or background in industrial/organiza-
(Hunter & Hunter, 1984) are from employees tional psychology. Because Neal Schmitt is a well-
within jobs. For the former, many non-g factors respected industrial/organizational psychologist
may influence people’s selection of jobs, which with a long list of notable contributions to this
attenuates the effect of GMA on incomes. Also, it field, we believe it is important to examine his
is well known that jobs at the same status or comments closely.
mental complexity level differ markedly in in- Schmitt states at the outset that he agrees with
come (e.g., judges vs. practicing lawyers, or clergy the main contention of our original article that
vs. investment managers). Also, many jobs have the field of personnel selection has been “com-
educational thresholds that limit the ranges of pletely transformed by the development of meta-
educational attainment among applicant and in- analysis.” He states that meta-analysis has swept
cumbents, reducing the relationship between aside earlier beliefs in the situational specificity of
amount of education and job performance. That validity of employment selection methods and
is, within the group applying for a given job the shown that selection test validities generalize
variability in education is much less than it is in widely. In addition, he states that the research on
the general population. At the same time, the utility analysis that we described in our original
variability in GMA is only slightly smaller in the article has shown that the use of valid tests has
applicant groups in comparison to the general real and substantial impacts on employee produc-
population (Sackett & Ostagaard, 1994). tivity. In short, in terms of approaches, methods,
In conclusion, we find many of Hauser’s (2007) and general conclusions, he endorses the content
comments subjective and not based on the evi- and thrust of our original article.
dence in the relevant literature. Hauser acknowl- What, then, are his reservations? His criticisms
70 Academy of Management Perspectives November

consist of technical quibbles about the specific of criterion reliability in most primary studies are
values of criterion reliability and range restriction quite inaccurate due to the substantial amounts of
ratios that we have employed in our meta-analyses sampling error in small-sample studies. Because
of validities. It is important to note at the outset these estimates are likely for this reason to be
that the differences he refers to would make little inaccurate, it is deceptive to refer to them as the
difference in estimates of validity and no differ- “actual” values, while implying that the far more
ence in conclusions about the presence and gen- accurate estimates from meta-analyses (Rothstein,
eralizability of validity. In this connection, it is 1990; Viswesvaran et al., 2005) are somehow not
important to remember that for practical purposes actual values. Note that because of the large
in organizations the two most important questions amount of sampling error, use of values from pri-
are (a) does a procedure have substantial validity? mary studies would likely result in reduced accu-
and (b) is the validity generalizable across employ- racy in meta-analysis results. In taking this posi-
ers, time periods, task differences in jobs, etc.? tion, Schmitt appears to be endorsing the
Even if we accepted all of Schmitt’s reservations situational specificity hypothesis. Schmitt agrees
(which we don’t), the answers to these two critical with our original article that the theory of situa-
questions would not change. In fact, this much is tional specificity of test validities has been discon-
implied in his opening comments endorsing the firmed by the findings of meta-analysis. Yet he
thrust and main content of our original article. seems here to be endorsing the idea that inter-
Nevertheless, we believe it is important to ex- rater reliabilities of job performance ratings are
amine his comments closely, because we believe situationally specific! We cannot see any other
they are often based on misunderstandings and basis for his contention that reliability estimates
inadequate information, and these misunderstand- from individual studies are the “actual” values,
ings may be shared by others. For example, while the much more precise estimates from large-
Schmitt states that “the appropriate conceptual- sample meta-analyses are not actual values.
ization of reliability has been disputed (Murphy & This same principle applies to estimates of
DeShon, 2000).” This statement is true but mis- range restriction ratios (ux values), which also are
leading. There has long been majority, if not uni- much affected by sampling error in typical small-
versal, agreement that the appropriate type of sample primary studies. (As explained in our orig-
reliability for ratings is inter-rater reliability, and inal article, ux is the ratio of the incumbent test
the psychometric and substantive foundation for standard deviation to the applicant pool standard
this is well established (see Schmidt, Viswesvaran, deviation, and is the index of the amount of range
& Ones, 2000; Thorndike, 1949; Viswesvaran, restriction on the predictor.) In particular, the
Schmidt, & Ones, 2005; and the sources cited in number of incumbents on which these estimates
these works). His citation of the single article that are based is typically quite small and may fluctuate
takes the opposite view implies that there is wide- across time (due, for example, to employee turn-
spread disagreement on this point and is therefore over), creating substantial instability of estimates.
misleading. As with criterion reliability estimates, Schmitt
In connection with the correction for unreli- seems to assume that the values of ux reported in
ability in measures of job performance, as assessed primary studies are “the actual values” and are
by supervisory ratings, Schmitt agrees that our more accurate than distributions of ux values that
estimates of criterion reliability are based on con- are derived from large typical selection databases
siderable empirical evidence and cites the Roth- and therefore are less influenced by sampling er-
stein (1990) meta-analysis as an example of such rors. In our judgment, this assumption is not likely
evidence. But he is concerned that “the actual to hold true. It is important to understand the
reliability values are often not provided in primary context of usage of ux distributions: The meta-
studies” [italics added], implying that only these analysis contains many studies (perhaps 100 or
values should be used in meta-analyses. The use of 200), and the ux distribution estimates the mean
the word actual is inappropriate, because estimates and variability of range restriction across these
2007 Schmidt, Le, Oh, and Shaffer 71

studies. The most important characteristic of this more accurate estimate of operational validity
distribution is its mean, because this is the ux than the initially observed value of .26. This point
value used to correct the mean observed validity. is important because of the emphasis Schmitt has
The variability of the ux distribution explains placed on focusing interpretation on uncorrected
some of the variance in the observed validities, validity estimates (discussed below).
but only a very small amount. Hence the key Another example of a quibble is Schmitt’s ci-
question is whether the average ux values are tation of Sackett and Ostgaard (1994). As noted
accurate. earlier in our response to Hauser, this study found
Neal Schmitt’s main concern is that the aver- that applicant pools (people applying for a partic-
age estimates of ux used in our research might be ular job) were 3% to 10% less variable than large
too small, resulting in estimates of validity that are norm groups (consisting of all people who applied
too high. His concerns here really do appear to for any job). Hunter (1983) used the standard
amount to quibbles, because the estimates we deviation (SD) in such a norm group as his esti-
have used are quite close to those produced by mate of the applicant pool SD, resulting in a small
large empirical databases. For example, based on a overestimation of the amount of range restriction
large database, Alexander et al. (1989) obtained a for the large sample of jobs (k ⫽ 425) he studied.
median ux value of .70. Based on the large U.S. Applying the Sackett and Ostegaard findings to
Employment Service database, Hunter (1983) the Hunter (1983) data results in ux values rang-
found an average ux value of .67. These empiri- ing from .69 to .74 —as compared to the value of
cally determined values are not far from the judg- .67 used by Hunter in his research. This small
mentally estimated value of .60 used in some of difference makes little difference in validity esti-
our earlier studies and criticized by Schmitt. Just mates and no difference at all in general conclu-
as important is the fact that the mean ux value of sions about the presence and generalizability of
.63 reported in Schmidt, Shaffer, and Oh (in substantial validities. In fact, this difference is so
press) is the average across several meta-analyses small that we have cited the Sackett and Ostgaard
in which all ux values were empirically determined (1994) study as showing that there is little error in
based on data from employment settings (Bertua, Hunter’s analysis (e.g., see Schmidt et al., in
Anderson, & Salgado, 2005; Hunter, 1986; Sal- press).
gado et al., 2003b). The grand mean of all the So it is clear that Schmitt is focusing on very
above ux values is .65. It is important to note that small inaccuracies (if there are any inaccuracies).
for our conclusions to be substantially in error Ironically, he endorses the making of very large
there has to be an important error in the estimates errors in validity estimates, quoting with approval
of ux used. As stated above, the fundamental con- the statement from Hartigan and Wigdor (1989,
clusions remain unchanged whether one uses p. 170) that “In the absence of data to estimate
mean ux values of .70, .67, .63, or .60, as shown in the effect, no correction should be made.” Harti-
the following analysis. The average initially ob- gan and Wigdor rejected Hunter’s corrections for
served (i.e., uncorrected) validity for GMA pre- range restriction because of the potential small
dicting job performance across eight meta-analy- error in his ux values discussed in the previous
ses is .26 (Schmidt et al., in press). Correcting for paragraph. They argued that it is preferable to
direct range restriction (and criterion unreliabil- make no correction at all for range restriction
ity), the validity estimates for these four ux values rather than to make corrections that might be a
are, respectively, .48, .50, .52, and .54. If we cor- few percentage points off. Of course, making no
rect for indirect range restriction the correspond- correction at all leads to massive errors of estima-
ing values are .53, .55, .59, and .62. In both cases, tion— often by 50% or more! For instance, in the
all validity estimates are substantial and do not examples described above, the lowest estimate of
differ very much. Fundamental conclusions do not mean validity under correction for indirect range
change as a function of these small differences in restriction is .53. The uncorrected value of .26
ux values. In addition, any of these values is a far underestimates this value by 51%.
72 Academy of Management Perspectives November

Thus we interpret Schmitt’s position here as initially observed validities impossible. For both
implying that it is better to accept really large these reasons, we do not see how this approach
estimation errors than to have small estimation can be scientifically viable.
errors! We do not see how such a position can be Schmitt points out that most selection is not
justified. In fact, this position has been explicitly top-down on a single test or other predictor, a
refuted by other researchers, including Sackett point we made very strongly in our earlier research
and Ostgaard (1994), whose finding that appli- (Hunter, Schmidt, & Le, 2006; Schmidt, Oh, &
cant pool SD are generally smaller than the SD in Le, 2006). In fact, we showed in these articles that
the norm groups (mentioned earlier) was cited by the fact that selection is almost never top-down
Schmitt to support his position of no correction on a single predictor is the reason why the cor-
for range restriction. After considering the impli- rection for direct range restriction produces down-
cations of their findings and comparing them with wardly biased validity estimates. But Schmitt also
the alternative solution supported by Hartigan states that our estimates of range restriction are
and Wigdor (1989), Sackett and Ostgaard con- “frequently based on the assumption that selec-
clude: “This suggests that the Hartigan and Wig- tion is top down” (p. 21). In validity generaliza-
dor decision to make no range restriction correc- tion studies, we do not and have not made this
tion does underestimate validity to a substantial assumption. This misunderstanding may stem
degree” (p. 683). from the fact that in presenting distributions of ux
It is important to bear in mind that no esti- values in validity generalization studies we often
mates in any science are ever perfectly accurate. provided the top-down selection ratios that would
Perfect accuracy is never possible, even in physics produce those ux values (e.g., see Pearlman,
or chemistry. The magnitude of errors is critical. Schmidt, & Hunter, 1980). However, these val-
Approximately accurate estimates that lead to ues, and in particular the mean ux value (.60),
correct decisions are far preferable to large errors were based on our direct observations of ux values
that lead to false conclusions and incorrect deci- in published validity studies and unpublished
sions. In discussions of these issues, it is important technical reports. Also, in an early article on
to keep this “big picture” in mind. The position statistical power in validity studies (a different
that Schmitt implicitly adopts is that unless one subject) (Schmidt, Hunter, & Urry, 1976), we
can have complete certainty that the corrections presented tables showing the ux values corre-
for the recognized large downward biases in ini- sponding to various top-down selection ratios. In
tially observed validities are exactly accurate, no that study, our estimates of sample sizes required
corrections should be made for these biases. This for different levels of statistical power were in fact
actually means that no bias corrections can ever based on the assumption of top-down selection
be made, because the artifact values from each (i.e., direct range restriction). A later study (Raju,
individual primary study that Schmitt favors for Edwards, & Loverde, 1985) presented values
making the corrections are estimated with consid- based on indirect range restriction, and these re-
erable inaccuracy. In fact, Schmitt may accept the quired sample sizes are somewhat smaller. In re-
conclusion that no corrections should ever be ferring to the assumption of top-down selection,
made, because in other contexts he has stated that Schmitt mentions its effect on utility estimates. In
researchers should focus on the initially observed explaining and illustrating the selection utility
(uncorrected) validity estimates (e.g., see Morge- model, we have often—for the sake of conve-
son, Campion, Hollenbeck, Murphy, & Schmitt, nience in presentation—assumed top-down selec-
2007; in press), estimates that are recognized in tion. This was in fact done in the target article, Le
the relevant professional standards as substantially et al., 2007. Since top-down selection on a single
downwardly biased (AERA-APA-NCME Stan- predictor is rare, Schmitt is correct in implying
dards, 1999; SIOP Principles, 2003). Furthermore, that the utility values presented in Le et al. (2007)
different estimates are downwardly biased to dif- may be too large. However, even allowing for this,
ferent degrees, making meaningful comparisons of the utility in most real-world contexts would still
2007 Schmidt, Le, Oh, and Shaffer 73

be quite large—though it would be hard to get large are well developed and accepted in the
exact estimates. Furthermore, as noted earlier in methodological literature and appear to also be
our response to Hauser (2007), the value of the accepted in principle by Schmitt. His argument
standard deviation of job performance (SDy⫽ against the resulting validity estimates is based on
40% of average salary) used in our utility calcula- his challenges to the specific values of ux and
tions is probably underestimated. Such underesti- criterion reliability used to make these correc-
mation is likely to offset the overestimation due to tions, and we have seen that his concerns in this
the top-down selection assumption in our utility respect are quite overstated. They are essentially
estimation. More important, it should be remem- quibbles, as he presents no other form of evidence
bered that the purpose of utility analysis is deci- against our estimates. There have been many in-
sion making about selection procedures, and so stances in the history of science in which results
the key question is whether utility is substantial. that initially seemed “incredulous” to many scien-
The utility literature shows that this is in fact the tists were later shown to be accurate and were
case, and Schmitt accepts this in his opening accepted. For example, this was true for the pre-
comments. Also, an important point is that in diction of the bending of light by gravity of Ein-
comparing the utility of different predictors or stein’s Theory of General Relativity, for the the-
predictor combinations, any overestimation due ory that stomach ulcers could be caused by a
to the assumption of top-down selection is can- bacterium, and for the theory that humans and
celed out when utility differences are examined. apes parted ways only 5 million years ago (instead
In most articles on selection utility, we have of 20 million years ago).
been careful to state that top-down selection is It is important to remember that validity esti-
not required by the model and that the mean mates refer to the prediction of individual differ-
standardized predictor score for those selected (Z៮ s) ences in performance in a context in which many
can be computed arithmetically regardless of how other variables are held constant—supervisory
employees are selected. In any event, the ux values style, performance incentives, organizational cli-
used in our validity generalization studies are not mate, etc. That is, on any given job when a
based on the assumption that that selection is top validity study is conducted, these factors—which
down. They are based either on empirical data or can be considered as interventions— do not vary.
on summary estimates based on our professional Even if, hypothetically, individual differences in
observations of ux values across many studies. This traits accounted for all the variance in perfor-
latter is the origin of the .60 value criticized by mance across employees on a given job, it would
Schmitt and discussed above. still be possible for interventions, such as a new
We could not replicate Schmitt’s multiple cor- motivational system, to increase mean perfor-
relations of .88 and .92 for predicting job perfor- mance substantially, as discussed by Hunter and
mance from a combination of GMA and person- Schmidt (1982). So large validity coefficients do
ality for complex jobs, which he claims to have not preclude large performance increases from
computed from data in one of our unpublished other sources. Hence there are no grounds for
studies. But the key psychological fact here is that Schmitt’s statement that if our validity values are
validity coefficients larger than about .70 accurate, the researchers studying organizational
(whether univariate validities or multiple correla- behavior and organizational development should
tions) appear to strike Schmitt as “incredulous” “find different objects for their investigations,
(p. 22). That is, it seems that Schmitt’s intuition since these individual difference correlates leave
tells him that the real values cannot be this large. little variance to explain.”
But where is the evidence that subjective intu- Schmitt argues that if test validities were as
ition is a good guide? The logic, the psychomet- large as we conclude them to be, differences in
rics, and the necessity of the corrections for rec- firm performance between companies using selec-
ognized biases in observed validities that produce tion tests and those not using them would be
the estimates that Schmitt feels intuitively are too much larger than the differences that research has
74 Academy of Management Perspectives November

been able to demonstrate. As a partial explanation tion. But the evidence indicates that the level of
for why these firm performance differences are not range restriction is just as great in predictive stud-
larger, he points out that organizations using se- ies as in concurrent studies (Pearlman, et al.,
lection tests may not, because of affirmative ac- 1980), as shown by the fact that mean observed
tion reasons, be willing to give the test scores full validities are the same in the two study designs.
weight in hiring decisions. But the number of Also, in the case of mental ability measures, the
factors that affect firm accounting performance evidence is so strong that predictive and concur-
and stock price is so large that the generally ac- rent validity studies provide identical validity es-
cepted view is that it is amazing that research has timates on average that this fact is included in the
been able to demonstrate any impact at all of good professional standards for selection of industrial/
selection and/or other HR practices on firm per- organizational psychologists (Principles, 2003). So
formance (Huselid, 1995). These include all the these are not real issues.
conditions of elevated uncertainty faced by busi- In summary, Neal Schmitt accepts and en-
nesses today: abrupt changes in market demand, dorses the main thrust and content of our original
unforeseen technological changes that cause prod- article and its conclusions. Except for his com-
uct obsolescence, acquisitions and mergers, criti- ments on the effects of the top-down assumption
cal bad decisions by a CEO, financial panics, and on selection utility estimates, his objections are
so on. The list is almost endless. So this argument basically quibbles about technical details that,
by Schmitt appears to us to be naive. even if accepted at face value, would not change
Schmitt’s next argument against our conclu- any of our fundamental conclusions. In addition,
sions is based on the fact that much of the validity we have shown that these quibbles are sometimes
data in our studies was collected decades ago. He based on misunderstandings and mistaken beliefs
speculates that changes in the world of work (the and therefore are not credible even as quibbles.
greater use of work teams, less job security, im- Yet Schmitt’s comments have to be viewed with
proved measures of noncognitive measures, etc.) seriousness, because others in the I/O psychology
may mean that our findings are no longer correct. field may passively accept them. Therefore, we
However, he offers no evidence that such changes believe the analysis of Schmitt’s reservations we
have affected the validity of mental ability mea- have presented here is important.
sures. And there is much evidence to the contrary.
In particular, recent meta-analyses of GMA (e.g., Conclusion

W
Bertua, et al., 2005; Salgado et al., 2003a; 2003b) hen we consider our article (Le et al., 2007);
report validities (observed and corrected) that are the three commentaries on that article by
quite similar to those reported by Hunter (1986); Schmitt, Osterman, and Hauser; and our
Pearlman et al. (1980); Schmidt, Hunter, Pearl- response to these commentaries, it occurs to us
man, and Shane (1979); and other early meta- that readers may well take this as yet another case
analyses. In fact, one the most striking aspects of in which “the experts cannot agree, so why pay
this area of research has been the consistency of any attention to what they say?” We have there-
validity findings across time (and across cultures: fore been careful to point out that of the three
European nations and the U.S. and Canada). This commentators, only one—Neal Schmitt—is an
finding suggests that changes over time in the expert in the relevant area of personnel selection.
nature of work and jobs, including changes in the We have carefully considered and, we believe,
employee-employer psychological contract, have successfully addressed his reservations, showing
not resulted in any noticeable changes in the role most of them to be quibbles. We have also ad-
that mental ability plays in job performance. This dressed the comments by Osterman and Hauser in
is an important oversight on Schmitt’s part. order to prevent readers from being misled. In
Schmitt also argues that future studies should summary, we believe that we have shown that the
be predictive, on the assumption that predictive conclusions we reached in our original article (Le
studies eliminate (or at least reduce) range restric- et al., 2007) are correct and valid.
2007 Schmidt, Le, Oh, and Shaffer 75

References sub(y) method, information frame, and focal interven-


tion. Journal of Applied Psychology, 82, 104 –112.
Alexander, R. A., Carson, K. P., Alliger, G. M., & Cron- Hunter, J. E. (1983). Test validation for 12,000 jobs: An
shaw, S. F. (1989). Empirical distributions of range re- application of job classification and validity generalization
stricted SDx values in validity studies. Journal of Applied analysis to the General Aptitude Test Battery (GATB),
Psychology, 74, 253–258. Test Research Report No. 45, U.S. Department of Labor,
American Educational Research Association, American U.S. Employment Service, Washington, DC.
Psychological Association, and the National Council on Hunter, J. E. (1986). Cognitive ability: Cognitive aptitude,
Measurement in Education. (1999). Standards for educa- job knowledge, and job performance. Journal of Voca-
tional and psychological testing (5th ed.). Washington, DC:
tional Behavior, 29, 340 –362.
American Educational Research Association.
Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of
Becker, B. E., & Huselid, M. A. (1992). Direct estimates of
alternative predictors of job performance. Psychological
SDy and the implications for utility analysis. Journal of
Bulletin, 96, 73–98.
Applied Psychology, 77, 227–233.
Hunter, J. E., & Schmidt, F. L. (1982). Quantifying the
Bertua, C., Anderson, N., & Salgado, J. F. (2005). The
effects of psychological interventions on employee job
predictive validity of cognitive ability tests: A UK meta-
performance and workforce productivity. American Psy-
analysis. Journal of Occupational and Organizational Psy-
chologist, 38, 473– 478.
chology, 78, 387– 409.
Hunter, J. E., Schmidt, F. L., & Judiesch, M. K. (1990).
Bobko, P., Karren, R., & Parkington, J. J. (1983). Estimation
of standard deviations in utility analysis: An empirical Individual differences in output variability as a function
test. Journal of Applied Psychology, 68, 170 –176. of job complexity. Journal of Applied Psychology, 75,
Brogden, H. E. (1946). On the interpretation of the corre- 28 – 42.
lation coefficient as a measure of predictive efficiency. Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications
Journal of Educational Psychology, 37, 65–76. of direct and indirect range restriction for meta-analysis
Brogden, H. E. (1949). When testing pays off. Personnel methods and findings. Journal of Applied Psychology, 91,
Psychology, 2, 171–183. 594 – 612.
Burke, M. J., & Frederick, J. T. (1986). A comparison of Huselid, M. A. (1995). The impact of human resource
economic utility estimates for alternative SDy estima- management practices on turnover, productivity, and
tion procedures. Journal of Applied Psychology, 71, 334 – corporate financial performance. Academy of Manage-
339. ment Journal, 38, 635– 672.
Cascio, W. F., & Ramos, R. A. (1986). Development and Judge, T. A., Higgins, C.A., Thoresen, C. J., & Barrick,
applications of a new method of assessing job perfor- M. R. (1999). The Big Five personality traits, general
mance in behavioral/economic terms. Journal of Applied mental ability, and career success across the life span.
Psychology, 71, 20 –28. Personnel Psychology, 52, 621– 652.
Cesare, S. J., Blankenship, M. H., & Giannetto, P. W. Le, H., Oh, I.-S., Shaffer, J. A., & Schmidt, F. L. (2007).
(1994). A dual focus of SDy estimations: A test of the Implications of methodological advances for the practice
linearity assumption and multivariate application. Hu- of personnel selection: How practitioners benefit from
man Performance, 7, 235–255. recent developments in meta-analysis. Academy of Man-
Cohen, J. (1977). Statistical power analysis for the behavioral agement Perspectives, 21(3), 6 –15.
sciences (rev. ed.). New York: Academic Press. Johnston, W. B., & Packer, A. H. (1987). Workforce 2000:
Coward, W. M., & Sackett, P. R. (1990). Linearity of Work and workers for the 21st century. Indianapolis, IN:
ability-performance relationships: A reconfirmation. Hudson Institute.
Journal of Applied Psychology, 75, 297–300. Mathieu, J. E., & Tannenbaum, S. I. (1989). A process-
Dreher, G. F., & Bretz, R. D. (1991). Cognitive ability and tracing approach toward understanding supervisors’ SDy
career attainment: The moderating effects of early career estimates: Results from five job classes. Journal of Occu-
success. Journal of Applied Psychology, 76, 392–397. pational Psychology, 62, 246 –256.
Eaton, N. K., Wing, H., & Mitchell, K. J. (1985). Alternate Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollen-
methods of estimating the dollar value of performance. beck, J. R., Murphy, K., & Schmitt, N. (2007). Recon-
Personnel Psychology, 38, 27– 40. sidering the use of personality tests in personnel con-
Hartigan, J. A., & Wigdor, A. K. (Eds.). (1989). Fairness in texts. Personnel Psychology, 60, 683–729.
employee testing: Validity generalization, minority is- Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollen-
sues, and the General Aptitude Test Battery. Washing- beck, J. R., Murphy, K., & Schmitt, N. (in press). Are we
ton, DC: National Academy Press. getting fooled again? Coming to terms with limitations
Hauser, R. M. (2007). Will practitioners benefit from meta- in the use of personality tests for personnel selection.
analysis? Academy of Management Perspectives, 21(3), Personnel Psychology.
24 –28. Murphy, K. R., & DeShon, R. P. (2000). Interrater corre-
Hawk, J. (1970). Linearity of criterion-GATB aptitude re- lations do not estimate the reliability of job performance
lationships. Measurement and Evaluation in Guidance, 2, ratings. Personnel Psychology, 53, 873–900.
249 –251. Murray, C. (1998). Income and inequality. Washington, DC:
Hazer, J. T., & Highhouse, S. (1997). Factors influencing AEI Press.
managers’ reactions to utility analysis: Effects of SD- National Commission on Excellence in Education. (1983).
76 Academy of Management Perspectives November

A nation at risk: The imperative for educational reform. ability in the world of work: Occupational attainment
Washington, DC: U.S. Department of Education. and job performance. Journal of Personality and Social
Neal, D. A., & Johnson, W. R. (1996). The role of premar- Psychology, 86, 162–173.
ket factors in Black-White wage differences. Journal of Schmidt, F. L., Hunter, J. E., Pearlman, K., & Shane, G. S.
Political Economy, 104, 869 – 895. (1979). Further tests of the Schmidt-Hunter Bayesian
Osterman, P. (2007). Comment on Le, Oh, Shaffer, and Validity Generalization Model. Personnel Psychology, 32,
Schmidt. Academy of Management Perspectives, 21(3), 257–281.
16 –18. Schmidt, F. L., Hunter, J. E., McKenzie, R. C., & Muldrow,
Pearlman, K., Schmidt, F. L., & Hunter, J. E. (1980). Va- T. W. (1979). Impact of valid selection procedures on
lidity generalization results for tests used to predict job workforce productivity. Journal of Applied Psychology, 64,
proficiency and training success in clerical occupations. 609 – 626.
Journal of Applied Psychology, 65, 373– 406. Schmidt, F. L., Hunter, J. E., & Outerbridge, A. N. (1986).
Raju, N. S., Edwards, J. E., & Loverde, M. A. (1985). Impact of job experience and ability on job knowledge,
Corrected formulas for computing sample sizes under work sample performance, and supervisory ratings of
indirect range restriction. Journal of Applied Psychology, performance. Journal of Applied Psychology, 71, 432– 439.
70, 565–566. Schmidt, F. L., Hunter, J. E., & Urry, V. W. (1976). Sta-
Reilly, R. R., & Smither, J. W. (1985). An examination of tistical power in criterion-related validity studies. Journal
two alternative techniques to estimate the standard de- of Applied Psychology, 61, 473– 485.
viation of job performance in dollars. Journal of Applied Schmidt, F. L., Oh, I.-S., & Le, H. (2006). Increasing the
Psychology, 70, 651– 661. accuracy of corrections for range restriction: Implications
Roth, P. L., Bobko, P., & Mabon, H. (2002). Utility for selection procedure validities and other research re-
analysis: A review and analysis at the turn of the century. sults. Personnel Psychology, 59, 281–305.
In N. Anderson & D. Ones (Eds.), Handbook of indus- Schmidt, F. L., Shaffer, J. A., & Oh, I.-S. (in press). In-
trial, work, and organizational psychology (pp. 363–384). creased accuracy of range restriction corrections: Impli-
London: Sage Publications. cations for the role of personality and general mental
Roth, P. L., Pritchard, R. L., Stout, J. D., & Brown, S. H.
ability in job and training performance. Personnel Psy-
(1994). Estimating the impact of variable costs on SDy
chology.
in complex situations. Journal of Business and Psychology,
Schmidt, F. L., Viswesvaran, C., & Ones, D. S. (2000).
8, 437– 454.
Reliability is not validity and validity is not reliability.
Rothe, H. F. (1978). Output rates among industrial employ-
Personnel Psychology, 53, 901–912.
ees. Journal of Applied Psychology, 63, 40 – 46.
Rothstein, H. R. (1990). Interrater reliability of job perfor- Schmitt, N. (2007). The value of personnel selection: Re-
mance ratings: Growth to asymptote level with increas- flections on some remarkable claims. Academy of Man-
ing opportunity to observe. Journal of Applied Psychology, agement Perspective, 21(3), 19 –23.
75, 322–327. Scullin, M. H., Peters, E., Williams, W. M., & Ceci, S. J.
Sackett, P. R., & Ostgaard, D. J. (1994). Job-specific appli- (2000). The role of IQ and education in predicting later
cant pools and national norms for cognitive ability tests: labor market outcomes: Implications for affirmative ac-
Implications for range restriction corrections in valida- tion. Psychology, Public Policy, and Law, 6, 63– 89.
tion research. Journal of Applied Psychology, 79, 680 – Smith, M. (1989). Some British data concerning the stan-
684. dard deviation of performance. Journal of Occupational
Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Psychology, 62, 189 –190.
Fruyt, F. (2003a). International validity generalization of Society for Industrial and Organizational Psychology.
GMA and cognitive abilities: A European community (2003). Principles for the validation and use of personnel
meta-analysis. Personnel Psychology, 56, 573– 605. selection procedures (4th ed.). Bowling Green, OH: Soci-
Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., de ety for Industrial and Organizational Psychology.
Fruyt, F., & Rolland, J. P. (2003b). A meta-analytic Viswesvaran, C., Schmidt, F. L., & Ones, D. S. (2005). Is
study of general mental ability validity for different oc- there a general factor in ratings of job performance? A
cupations in the European community. Journal of Applied meta-analytic framework for disentangling substantive
Psychology, 88, 1068 –1081. and error influences. Journal of Applied Psychology, 90,
Schmidt, F. L. (2002). The role of general cognitive ability 108 –131.
and job performance: Why there cannot be a debate. Wilk, S. L., Desmarais, L. B., & Sackett, P. R. (1995).
Human Performance, 15, 187–202. Gravitation to jobs commensurate with ability: Longitu-
Schmidt, F. L., & Hunter, J. E. (1998). The validity and dinal and cross-sectional tests. Journal of Applied Psychol-
utility of selection methods in personnel psychology: ogy, 80, 79 – 85.
Practical and theoretical implications for 85 years of Wilk, S. L., & Sackett, P. R. (1996). Longitudinal analysis
research findings. Psychological Bulletin, 124, 262–274. of ability-job complexity fit and job change. Personnel
Schmidt, F. L., & Hunter, J. E. (2004). General mental Psychology, 49, 937–967.

Vous aimerez peut-être aussi