Dr.

Stavroula Leka Introduction to Work Psychology

CHAPTER 4: Psychological Testing

I/O psychologists generally agree that no other selection technique is superior to intelligence or cognitive ability tests for selecting the best employees. They carry fewer risks of adverse impact in hiring decisions, they are good predictors of performance on the job and in training programs for many kinds of occupations. Even NFL, football teams administer psychological tests to prospective players. The New York Giants team has used personality tests to detect whether a player would tend to be overly aggressive, undisciplined or likely to cause trouble on and off the field. Characteristics of Psychological Tests Carefully developed and researched psychological tests have several characteristics. A proper test is: standardized, objective, based on sound norms, reliable, and valid. Standardization Standardization refers to the consistency or uniformity of the conditions and procedures for administering a test. Every person taking the test reads or listens to the same instructions, is allowed the same amount of time in which to respond, and is situated in a similar physical environment > any change in the testing procedure may produce a change in individual test performance. E.g. if the air-conditioner has broken down on a particular day, then the ppl perfoming the test that day may not do as well as others. The test administrators should be trained to maintain proper procedures. Objectivity Refers primarily to the scoring of the test results. The scoring process must be free of subjective judgment or bias on the part of the scorer.

judgment or bias. They contain multiple-choice and true-false items and scoring is a mechanical process that requires no special training or knowledge.

Objective tests: tests for which the scoring process is free of personal

Subjective tests: tests that contain essay questions; the scoring process can
be influenced by the personal characteristics and attitudes of the scorer. Test Norms

__________________________________________________________________________________ _ Chapter 4 / Page 1 of 14

Dr. Stavroula Leka Introduction to Work Psychology

To interpret the results of a psychological test, a frame of reference or point of comparison must be established so that the performance of one person can be compared with the performance of others. This is accomplished by means of test norms: the distribution of test scores of a large group of people similar in nature to the job applicants being tested. The scores of this group, called the standardization sample (the group of subjects used to establish test norms), serve as the point of comparison for determining the relative standing of the persons being tested. E.g. a high school graduate applies for a job that requires mechanical skills and achieves a score of 82 on a test of mechanical ability. This score alone tells us nothihg about the level of the applicants skill, but if we compare that score of 82 with the test norms the distribution of scores on the test from a large group of high school graduates then we can ascribe some meaning to the individual score. If the mean of the test norms is 80 and the standard deviation is 10, we know immediately that an applicant who scores 82 has only an average or moderate amount of mechanical ability. With this comparative information, we can evaluate objectively the applicants chances of succeeding on the job relative to the other applicants tested. The most widely used psychological tests have sets of norms for different age groups, races, sexes, and levels of education. Reliability Reliability: refers to the consistency or stability of a response on a test. If a group takes a cognitive ability test one week and achieves a mean score of 100, and repeats the test a week later and achieves a mean score of 72, there is sth wrong. We would describe the test as unreliable bacause it yields inconsistent measurements. It is common to find slight variation in test scores when a test is retaken at a later date, but if the fluctuation is great, it suggests that sth is amiss with the test or the scoring method.

involves administering a new test twice to the same group of subjects and correlating the two sets of scores. The closer the correlation coefficient (called in this case the reliability coefficient) approaches a perfect positive correlation of +1.00, the more reliable the test is considered to be. This method has limitations: it is uneconomical to ask employees to take time from their jobs to take the test twice. Also, the effects of learning (remembering the questions) and the influences of other experiences between the 2 testing sessions may cause the group to score higher the second time.

The test-retest method: a method for determining test reliability that

__________________________________________________________________________________ _ Chapter 4 / Page 2 of 14

Dr. Stavroula Leka Introduction to Work Psychology

involves administering similar forms of a new test to the same group of subjects and correlating the two sets of scores. Disadvantage: it is difficult and costly to develop two separate and equivalent tests.

The equivalent-forms method: a method for determining test reliability that

administering a new test to a group of subjects, dividing in half the total number of items, and correlating the two sets of scores. Less time-consuming than the other methods bacause only one administration of the test is required. There is no opportunity for learning or recall to influence the second score. In choosing a test for employee selection, the reliability coefficient ideally should exceed +.80, although, in practice, a coefficient of approximately +.70 is considered acceptable. Validity It is the most important requirement for a psychological test or any other selection device. Validity: the determination of whether a psychological test or selection device measures what it is intended to measure. Several different kinds of validity.

Split-halves method: a method for determining test reliablity that involves

Criterion-related validity: a type of validity concerned with the relationship

between test scores and job performance. E.g. a psychologist working for the Air Force develops a test of radar operator proficiency. The test will be considered valid if it measures those skills needed for competent performance on the job. One way to determine this is to correlate test scores with some measure of subsequent job performance. If persons who score high on the radar operator proficiency test also perform well on the job and if those who score low on the test perform poorly on the job, then the correlation coefficient (called in this case the validity coefficient) between test scores and job performance will be high. We will know that the test truly measures the skills needed to be a good radar operator and can be considered a valid predictor of job success. Validity coefficients of +.30 to +.40 are considered acceptable for employee selection tests. It is rare to find a test with a validity coefficient greater that +.50.

2 approaches in establishing criterion-related validity: predictive validity and concurrent validity.

__________________________________________________________________________________ _ Chapter 4 / Page 3 of 14

Dr. Stavroula Leka Introduction to Work Psychology

Predictive validity: an approach to establishing criterion-related

validity in which a new test is administered to all job applicants; all applicants are hired, regardless of test scores, and at a later date their test scores are correlated with a measure of job performance. In most corporations, top management is not at favor of this approach because some people hired will turn out to be poor workers.

Concurrent validity: a type of validity that involves administering a

test to employees already on the job and correlating their scores with job performance data. The disadvantage of this method is that by testing current workers, the validation sample contains only the better employees. Poorer workers will have already quit or been fired, demoted or transferred. Therefore, it is difficult to establish whether the test is discriminating between good and poor workers. Another problem is that applicants for a job and employees already on the job have different motivations and are likely to perform differently on psychological tests. A study of highway maintenance workers showed that applicants demonstrated significantly higher levels of effort and motivation to perform well on an employment test than did current workers (Arvey, Strickland, Drauden & Martin, 1990). The job performance measure, or criterion, most often used in the establishment of criterion-related validity is ratings assigned by a supervisor to an employees present level of job performance. Such ratings are made routinely as part of the employee performance appraisal process > research has shown that supervisor ratings are valid predictors of job success, but periodic reappraisals are necessary.

Rational Validity: The type of validity that relates to the nature, properties,

and content of a test, independent of its relationship to measures of job performance.

2 approaches to establishing the rational validity of a test are: content validity and construct validity.

In content validity, the test items are assessed to ensure that they
adequately sample the knowledge or skills the test is designed to measure. This assessment can be accomplished by conducting a job analysis and determining if the test items are related to all those abilities needed to perform the job. E.g. if your professor says you will be tested on the first 3 chapters of the book, questions about information from other chapters would not be considered content valid).

__________________________________________________________________________________ _ Chapter 4 / Page 4 of 14

Dr. Stavroula Leka Introduction to Work Psychology

Construct validity is a type of validity that attempts to determine the

psychological characteristics measured by a test. We can correlate scores on the new test with scores on established tests that measure certain variables or constructs (e.g. intelligence or motivation). If the correlation is high, then we can have some confidence that the new test is measuring the trait it claims to measure.

Face validity: a subjective impression of how well the items on a

psychological test appear to be related to the requirements of the job. It is not a statistical measure. Applicants may not take questions seriously if they think that they are not related to their job. E.g. a study of 154 managers and recruiters found that cognitive ability tests containing concrete items such as vocabulary and mathematical word problems were judged to be significantly more job-related (higher in face validity) than personality tests or cognitive ability tests with abstract items (Smither, Reilly, Millsap, Pearlman & Stoffey, 1993). People who perceive the selection tests to be high in face validity also rate the organization as more attractive and report they are more willing to recommend the organization to others as a place to work. The best psychological tests include in their manuals the results of validation studies. Without this information, the personnel or human resources manager can have little confidence that the tests in the companys employee selection program are actually measuring the qualities and abilities being sought in new employees. The average test validation has been estimated at more than $400,000, and the process requires more than a year to complete. Validity Generalization Validity Generalization: the idea that tests valid in one situation may also be valid in another situation. On the basis of large-scale reanalyses of previous validation studies, I/O psychologists have concluded that tests valid in one situation may also be valid in another situation > once established, the validity of a test can be generalized. If a test is valid for one job, it will be valid for others of the same or similar nature. A test valid for one company will be valid in other companies and for one ethnic group will be valid for other ethnic groups. The notion of validity generalization has not been accepted by all psychologists. Some critics question the feasibility of meta-analysis in general as well as the procedures for carrying it out. Others cast doubt on the accuracy and the conclusions drawn from validity generalization studies and others suggest that the findings of the early validity generalization studies were overly optimistic.

__________________________________________________________________________________ _ Chapter 4 / Page 5 of 14

Dr. Stavroula Leka Introduction to Work Psychology

If tests no longer require expensive validation procedures for every job in every company, then organizations can improve their selection programs while saving time and money. Psychological testing and fair employment practices Increase in validity research to document whether a test is discriminatory > if studies show that applicants of all races who score below a certain level on a test perform poorly on the job, then the test is not discriminatory by race. Research confirms that tests with high validity coefficients have relatively low levels of adverse impact. Criterion-related validation procedures (correlating test scores with job performance measures) are required when possible. A test found to be valid for one group is expected to be valid for another. Racial differences among average test scores are not the result of test bias but arise from educational, social and cultural differences. Differences shown mainly for cognitive ability tests. Race banding: a controversial practice of grouping test scores for minority job applicants to equalize hiring rates. E.g. a companys psychologist supervising the employee selection program could examine the test scores and decide to band or group toogether all applicant scores between, say, 91 and 100, calling the range of scores band 1. Band 2 would encompass all socres between 81 and 90, band 3 the scores between 71 and 80 and so on. For the purpose of hiring equality by race, then, all applicants included in band 1 would be considered equal in terms of the ability being tested. No distinction would be drawn between the applicant with a score of 100 and the applicant with a score of 92. Thus, all applicants in band 1 would be eligible for selection. Further, the order of selection within a band would not necessarily be by test score but rather could be by race. So, a firm might choose to select members of a minority group in the interests of diversity. But, the result might be that only members of the minority group are screened in if the number of openings to be filled is equal to or less than the number of minority group members in the top band. Testing for disabled persons > Many psychological tests have been adjusted or modified for disabled persons. For visually handicapped persons, test questions can be presented orally, in large print, or in braille, and people can be allowed more time in which to complete a test. Test questions relating to color, shape and texture cannot be used with persons who were born without sight and have never seen colors and objects. Applicants with hearing disabilities can be given written instead of oral test instructions.

__________________________________________________________________________________ _ Chapter 4 / Page 6 of 14

Dr. Stavroula Leka Introduction to Work Psychology

Establishing a testing program The first requirement is to investigate the nature of the job for which testing will be used. Once job and worker analyses have been conducted, the appropriate tests to measure the behaviors and abilities necessary for job success must be carefully chosen or developed. I/O psychologists have to find appropriate tests to test these abilities. They can either use tests already available or develop new tests specifically for the job and company in question. The best tests include information on reliability, validity and test norms. There are several issues to consider in deciding whether to use a published test or develop a new one > Cost is always important: it is less expensive to purchase a test than to construct one, especially if a small number of employees are to be selected. Time is also important. If the organization needs qualified workers as soon as possible, management may be unwilling to wait for a test to be developed. If an organization decides to develop its own test for a particular job, the I/O psychologists (internal or external) must prepare a list of suitable test items and then validate the test to determine whether it measures what it is supposed to measure. An item analysis will be conducted to evaluate how effectively each test item discriminates between those who score high on the total test and those who score low. This evaluation involves correlating a persons response on each item with the response on the test as a whole. A perfectly valid test question is one that was answered correctly by everyone who who scored high on the complete test and was answered incorrectly by everyone who scored low. Only items with a high correlation coefficient will be retained for the final version of the test. The level of difficulty for each item must be determined. Not all items must have the same level of difficulty. Then the predictive validity of the test must be determined. That is, the scores on the test must be correlated with actual performance on the job. The more common method is to test workers already on the job. The content validity of a test should be also demonstrated: it must be shown that the items deal with abilities that are directrly related to the job. Once validity and reliability of the new test have been found to be satisfactory, a cutoff score must be set. No one scoring below this level will be hired. The cutoff score depends partly on the available labor supply. The greater the number of applicants, the more selective a company can be. The higher the cutoff score, the higher the quality of the applicants hired. Most of the procedures for establishing cutoff scores involve job analyses and criterionrelated validity studies to determine the minimum acceptable level of job performance. One frequently used technique asks subject matter experts to assess the probability that a minimally competent pesron would answer each test item correctly. Types of Psychological Tests
__________________________________________________________________________________ _ Chapter 4 / Page 7 of 14

Dr. Stavroula Leka Introduction to Work Psychology

I/O psychologists categorize tests in 2 ways: 1. how they are constructed and administered, and 2. the skills and abilities they are designed to measure. Individual and Group Tests Group tests: tests designed to be administered to a large number of people at the same time. They can be given to 20, 200 or 2,000 applicants simultaneously. The only limitation is the size of the testing facility. Individual tests: psychological tests designed to be administered to one person at a time. More costly than group tests > used less frequently, usually only for selecting senior management personnel. More popular for vocational guidance and counseling and for diagnostic work with patients. Computer-Assisted Tests Computer-assisted tests: a means of administering psychological tests to large groups of applicants in which an applicants response determines the level of difficulty of succeeding items. They are designed for large scale group testing. Each applicant takes the test at a video display terminal > sometimes called tailored testing because the test is tailored or adapted to the person taking it. In computer-assisted testing you dont have to waste time answering questions below your level of ability. The computer program begins with a question of average difficulty. If you answer correctly, it proceeds to questions of greater difficulty. If not, it asks you less difficult questions. Advantages: - Testing can be done at any time in the selection process. - It is not dependent on finding a qualified test administrator and scheduling a testing session. - Because a range of abilities can be measured in a relatively short time, there is less opportunity for the applicants interest and motivation to diminish. - Fatigue and boredom are reduced, and immediate results are available to the human resources department.

Comparisons of the same tests given in paper-and-pencil format and computeradministered versions show little difference in the resulting scores. Research has shown that most ppl react favorably to computer-assisted testing. On personality tests, they tend to provide more accurate information. For high-level managerial jobs, the company may have an I/O psychologist prepare a summary of the test scores and, based on those scores, a description and judgment of the applicants abilities. Computer-generated reports are also available. Computerized
__________________________________________________________________________________ _ Chapter 4 / Page 8 of 14

Dr. Stavroula Leka Introduction to Work Psychology

test interpretation eliminates the possibility of personal bias in evaluating the test results. Speed and Power Tests The difference between the 2 is the time allotted for completion of the tests. Speed tests: tests that have a fixed time limit, at which point everyone taking the test must stop. Power tests: tests that have no time limit; applicants are allowed as much time as needed to complete the test. A power test often contains more difficult items than a speed test. Large-scale testing programs often include speed tests because all test forms can be collected at the same time. For some jobs, working speed is an important component of successful performance. A power test would not be able to evaluate this skill properly. Paper-and-Pencil and Performance Tests Paper-and-pencil tests: psychological tests in printed form; answers are recorded on a standard answer sheet. Most often and most standard group tests of cognitive abilities, interests and personality belong to this category. Some behaviors or characteristics are not so easily evaluated by paper-and-pencil means. E.g. the mechanical skills of applicants applying for jobs with an appliance repair company are better tested by having the applicants perform the appropriate mechanical operations than by having them answer questions about the nature of those operations. Performance tests: the assessment of complex skills, such as word processing or mechanical ability, for which paper-and-pencil tests are not appropriate. For the assessment of more complex skills, expensive equipment may be needed. Tests of Knowledge, Skills and Abilities Distinction in terms of the characteristics or behaviors they are designed to measure. The basic categories are tests of: cognitive abilities, interests, aptitudes, motor skills, and personality. Cognitive Abilities

__________________________________________________________________________________ _ Chapter 4 / Page 9 of 14

Dr. Stavroula Leka Introduction to Work Psychology

Cognitive ability tests (intelligence tests): tests of intelligence or mental ability. Widely used for employee selection > e.g. a survey of personnel practices in departments of law enforcement found that 92% used intelligence to select police officers (Ash, Slora & Britton, 1990). Group intelligence tests, the kind given most often, are a rough screening device. They are brief, take little time to complete and can be administered to large groups. They are rapidly and easily scored by hand or machine. Research has shown that cognitive ability tests are highly valid for predicting success in job training programs and in actual job performance. They correlate highly with measures of job performance such as supervisor ratings, production quantity and quality data, and work samples. Also, the high predictive validities of cognitive ability tests, have been shown to remain stable up to 5 years. Also, cognitive ability is a valid predictor of employee mobility > A study of more than 11,000 employees between the ages of 24 and 30 found that over a 5-year period, ppl tended to gravitate to jobs that suited their level of cognitive ability, whether high or low (Wilk, Desmarais & Sackett, 1995).

Interests

Interest tests: psychological tests to assess a persons interests and preferences; used primarily for career counseling. Include items about daily activities from among which applicants select their preferences. The rationale is that if a person exhibits the same pattern of interests and preferences as people who are successful in a given occupation, then the chances are high that the person will find satisfaction in that occupation. It is important to note that just because a person shows a high degree of interest in a particular job, there is no guarantee that s/he has the ability to be successful in that job. What interest tests show is that the persons interests are compatible with the interests of successful ppl in that career. If a test shows that a person has no interest in a field, then the chances of succeeding in it are limited. One problem with using them for employee selection is the possibility of faking of responses.

Aptitudes

__________________________________________________________________________________ _ Chapter 4 / Page 10 of 14

Dr. Stavroula Leka Introduction to Work Psychology

Aptitude tests: tests to measure specific abilities such as mechanical or clerical skills. May measure perceptual speed and accuracy, attention to detail, the capacity to visualize and manipulate objects in space, principles of mechanical operation, ability to operate computers.

Motor Skills

Motor skills tests: tests to measure abilities involving muscle coordination, finger dexterity and eye-hand coordination. Many jobs in industry and the military require abilities. These tests may simulate assembly-line conditions.

Personality

Certain personality characteristics can contribute to job satisfaction and job performance. E.g. empathy and nurturance are important traits for counselors to have; autonomy and persistence are vital to investigative reporters; and orderliness and precision are needed by accountants. Personality tests: assessments of personal traits and feelings. Personality test scores have been found to correlate with job success. Their predictive validities can be as high as those for assessment centers. They are popular selection techniques and their use is expected to increase. 2 approaches to personality assessment are self-report inventories and projective tests.

Self-report personality inventories: personality assessment tests that include

questions dealing with situations, symptoms and feelings; applicants are asked to indicate how well each item describes themselves or how much they agree with each item.

Honesty is the main difficulty. Research suggests though that few job applicants deliberately distort their responses in a socially desirable direction and that validity measures remain stable regardless of any possible distortion. Predictive validities for self-report personality inventories are generally in the low to moderate scale, but using them to assess the so-called Big Five factors of personality has shown much higher validities. These basic factors have been identified as extraversion, agreeableness, conscientiousness (or the will to achieve), neuroticism and openness to experience. 2 of those factors, conscientiousness and extraversion, have been found to be particularly effective in predicting job performance.
__________________________________________________________________________________ _ Chapter 4 / Page 11 of 14

Dr. Stavroula Leka Introduction to Work Psychology

Conscientiousness includes such characteristics as being responsible and dependable, able to plan, organized and achievement-oriented. Etxraversion, which includes the characteristics of sociability, talkativeness, ambition, assertiveness and high activity level, correlates highly with success for salespersons and managers. A third factor, openness to experience, has been found to predict training proficiency for all tested occupations. Studies of managers who make hiring decisions confirm that conscientiousness (along with general mental ability) is the most important attribute related to hirability.

Projective techniques: a personality assessment technique in which

applicants project their feelings onto an ambiguous stimulus such as an inkblot.

The task is to tell what is seen in the figure or picture. The rationale is that people will project their thoughts, wishes and feelings onto the stimulus in an effort to give it meaning. These tests cannot be faked because there are no right and wrong answers. They are time-consuming and must be administered individually. Examiners must be thoroughly trained and experienced. Because few of the tests have objective scoring keys, there is ample opportunity for subjective bias to affect the scoring and interpretation of the results. Although projective techniques are used for employee selection at the executive level, research shows that they have low validity.

Integrity tests

Integrity tests: paper-and-pencil tests to predict and detect employee dishonesty. There are 2 types of integrity tests: overt integrity tests, which directly assess attitudes toward theft, and personality-oriented integrity tests, which measure general delinquency, impulse control and conscientiousness. Both types of test appear to be valid predictors of theft and of such counterproductive behaviors as absenteeism, drug abuse and malingering. They have also been found predictive of general job performance as assessed by supervisor ratings.

Limitations of psychological testing programs

__________________________________________________________________________________ _ Chapter 4 / Page 12 of 14

Dr. Stavroula Leka Introduction to Work Psychology

Uncritical use: a personnel manager may choose a test because it is

new, without investigating the tests norms, reliability or validity. A test may continue to be used despite negative information about its dependability because the employment office is unaware of the research. Often there are no data to support a test because no research has ever been conducted on it.

Rejection of qualified applicants: even the best psychological tests are

not perfectly valid. No validity coefficients for tests reach a perfect positive correlation of +1.00. There is always some margin for error in the prediction of job success. These errors are considerably reduced when tests with the highest predictive validities are used. Also, a selection program should not be based on a single technique.

Faking: is less of a problem than previously believed and has a

negligible effect on validity.

Attitudes toward testing: psychological tests have never been popular

with ppl who are required to take them and whose future is dependent on the outcome. Many applicants experience considerable test anxiety (especially females and older applicants). Effects of negative attitudes: applicants who react negatively may become so offended that they withdraw from the application procedure and develop a hostile attitude toward the organization. They may spread that attitude to friends and acquaintances, possibly causing a number of potential applicants to avoid that company. They may even file a lawsuit against the company. Negative attitudes toward psychological tests may also lower applicants motivation to perform well on the tests, which in turn, reduces the predictive validity of the tests.

Ethical issues: APA > 1. Test users: persons who administer and interpret psychological tests should be
aware of the principles of psychological measurement and validation and the limitations of test interpretation. They must avoid bias and should always consider more than one means of assessment. They must adhere to standardized test administration procedures and make every effort to achieve accuracy in recording and scoring test results. 2. Test security: actual test questions should never be reprinted in a public medium such as a newspaper or magazine. It is permissible to publish sample questions (similar to real questions) but not items used in the actual scoring of a test. Tests should be sold only to professionals who will safeguard their use. 3. Test interpretation: test scores should be given only to those qualified to interpret them. They should not be given to anyone outside the human interpret
__________________________________________________________________________________ _ Chapter 4 / Page 13 of 14

Dr. Stavroula Leka Introduction to Work Psychology

department, such as the applicants potential supervisor, unless the supervisor has the training necessary to interpret the scores. The person being tested has the right to know the test score and what it means. 4. Test publication: tests should not be released for use without adequate background research to support the test developers claims. Informative and current test manuals containing reliability, validity and normative data should be made available. Advertisements should describe tests accurately, without emotional or persuasive appeals.

Privacy issues: is the use of questions about personal or intimate issues

appropriate? Critics say it is an unnecessary invasion of privacy. Personal questions about sex, religion, political beliefs and health have been successfully challenged in court as unwarranted invasions of privacy. Even if such questions could be shown to be related to job performance, there remains the issue of just how much of ourselves we should be required to reveal to a potential employer.

__________________________________________________________________________________ _ Chapter 4 / Page 14 of 14