Original Articles and Reviews

Construct-Driven Development
of a Video-Based Situational
Judgment Test for Integrity
A Study in a Multi-Ethnic Police Setting
Lonneke A. L. de Meijer,1 Marise Ph. Born,1 Jaap van Zielst,2
and Henk T. van der Molen1
1

Erasmus University, Rotterdam, The Netherlands

Police Academy of The Netherlands, Apeldoorn, The Netherlands

Abstract. In a eld study conducted in a multi-ethnic selection setting at the Dutch police, we examined the construct validity of a video-based
situational judgment test (SJT) aimed to measure the construct of integrity. Integrity is of central importance to productive work performance of
police ofcers. We used a sample of police applicants, which consisted of a Dutch ethnic majority group and an ethnic minority group. The ethnic
minority applicants came from one of the four largest ethnic minority groups in The Netherlands, namely groups with a Dutch Antillean, a
Moroccan, a Surinamese, or a Turkish background. A critical issue is the often-found construct-heterogeneity of SJTs. However, we found that a
construct-driven approach may be fruitful in the development of SJTs aiming to measure one single construct. Conrming our expectations, we
found support for the construct validity of the SJT intended to measure the construct of integrity. These results held across ethnic majority and
ethnic minority applicants. Therefore, the SJT is a promising test for personnel selection in a multi-ethnic setting.
Keywords: situational judgment test, integrity, construct validity, ethnicity

Situational judgment tests (SJTs) typically consist of scenarios of hypothetical work situations in which a problem has
arisen. Accompanying each scenario are multiple possible
ways to respond to the hypothetical situation. The test taker
is then asked to judge the possible courses of action.
Although SJTs have been in use since the 1920s, they have
become increasingly popular in personnel selection and in
the research literature during the last two decades (e.g., Chan
& Schmitt, 1997, 2005; Dalessio, 1994; McDaniel, Hartman,
Whetzel, & Grubb, 2007; Olson-Buchanan et al., 1998;
Weekley & Jones, 1997, 1999). Several characteristics of
the SJT have caused its revival. First, McDaniel et al.
(2007) meta-analytically showed the criterion-related validity
and the incremental validity of SJTs over and above a composite of cognitive ability tests and personality questionnaires in predicting job performance. Second, SJTs have
been found to have less adverse impact on ethnic minority
groups than more traditionally used cognitive ability tests
(Clevenger, Pereira, Wiechmann, Schmitt, & Harvey, 2001;
Motowidlo, Dunnette, & Carter, 1990; Nguyen & McDaniel,
2003; OConnell, Harman, McDaniel, Grubb, & Lawrence,
2007; Olson-Buchanan et al., 1998; Weekley & Jones,
1997, 1999). Finally, new technology has made the development of SJTs based on video material possible. The videobased SJT appears to have several advantages compared to
2010 Hogrefe Publishing

the paper-and-pencil SJT, such as a higher criterion-related

validity (Lievens & Sackett, 2006), less adverse impact,
and higher realism of the test leading to more reliable respondent reactions (Chan & Schmitt, 1997; Richman-Hirsch,
Olson-Buchanan, & Drasgow, 2000).
Even though SJTs have a series of advantages, important
questions still persist. A critical issue is the often-found construct-heterogeneity of SJTs and the difculty of developing
an SJT that measures one specic construct. Summarizing
the empirical literature, McDaniel and Nguyen (2001)
showed that SJTs are not one-dimensional construct tests,
but should be considered as a measurement method capable
of measuring several constructs. According to McDaniel and
Nguyen, empirical evidence indicates that the constructs
measured by SJTs can be limited to cognitive ability or
g (observed r :36; McDaniel, Morgeson, Bruhn
Finnegan, Campion, & Braverman, 2001), and the personality dimensions conscientiousness (observed r :26), agreeableness (observed r :25), and emotional stability
(observed r :31; McDaniel & Nguyen, 2001). Although
we agree with the viewpoint that SJTs are measurement
methods, we question whether SJTs are limited to measuring
g, conscientiousness, agreeableness, or emotional stability.
We argue that SJTs can also be built to measure other
constructs.
European Psychologist 2010; Vol. 15(3):229236
DOI: 10.1027/1016-9040/a000027

230

L. A. L. de Meijer et al.: The Development of a SJT for Integrity

The present study aims to demonstrate that a constructdriven development of SJTs is possible. We developed an
SJT intended to measure the concept of integrity and based
on video scenarios (i.e., a video-based SJT for integrity). We
collected eld data in a multi-ethnic setting during Dutch
police ofcer selection. Therefore, the construct validity of
the SJT was examined for both the ethnic majority and
the ethnic minority group. The largest ethnic minority
groups in The Netherlands are groups with a Dutch
Antillean, a Moroccan, a Surinamese, and a Turkish background. We will rst discuss the concept of integrity and of
integrity tests. Second, the Integrity-SJT will be introduced
and the hypotheses will be described. Finally, the relevance
of the Integrity-SJT for personnel selection will be dealt with.

The Concept of Integrity and Integrity Tests

For the following two reasons, more and more attention is
given to integrity during personnel selection and for job performance. First, measures of integrity have shown to be predictive of organizational outcomes, from theft to job
performance (Ones, Viswevaran, & Schmidt, 1993). Second,
integrity tests have also shown to predict incrementally over
and above measures of cognitive ability (Schmidt & Hunter,
1998).
Integrity is difcult to dene and appears to consist of
various dimensions (Jones, Brasher, & Huff, 2002; Van
Iddekinge, Taylor, & Eidson, 2005). Often-found examples
of dimensions of integrity are honesty, work values, customer service, and drug avoidance. Violations of integrity
at the Dutch police involve, among other things, corruption,
fraud and theft, accepting dubious gifts and services, misuse
of authority, and misuse of information (Naeye, Huberts,
Van Zweden, Busato, & Berger, 2004), which can be viewed
as dimensions of police integrity. Because of the impact that
these integrity violations may have on the police organization, it is important to determine an applicants integrity
by means of a police ofcer selection measure.
Broadly, there are two types of integrity tests. Tests using
items that focus on attitudes toward theft and other dishonest
behaviors are referred to as overt integrity tests, whereas
tests developed to assess broad personality traits that predict
counterproductive behaviors are referred to as personalitybased integrity tests or so-called disguised purpose
tests (Ones & Viswesvaran, 1998). In general, integrity tests
have been found to positively relate to the Big Five
personality dimensions of conscientiousness (observed
r = .26), agreeableness (observed r = .23), and emotional
stability (observed r = .18; Ones, 1993, in Wanek, 1999).
Integrity tests have negligible correlations with cognitive
ability (Ones & Viswesvaran, 1998).

An Integrity-SJT
The SJT that was developed for the Dutch police consists of
videos of critical situations in each of which police-integrity
violations are presented. Little is known in the literature
European Psychologist 2010; Vol. 15(3):229236

about construct-driven development of SJTs. We know of

one empirical study; Becker (2005) argued that his SJT
was based on an explicit, clear denition of integrity and
was intended to capture specic integrity values rather than
general personality traits or other variables that are related
to, but are not synonymous with, integrity. He stated that a
clear a priori denition of integrity was necessary, but did
not examine the construct validity of his SJT. Becker found
that the SJT was a moderately valid predictor of outcomes in
real-world settings, such as promotion, career progress, and
status as a team leader. As these criteria are general in nature,
they do not demonstrate explicitly the SJT as an instrument
measuring integrity. Building on Becker (2005), we developed an SJT intended to measure integrity and investigated
its construct validity. To this end, we investigated the relationship between the SJT score and actual integrity-related
variables, instead of examining the relationship between
the SJT score and general work-related outcomes, as Becker
did. Additionally, we examined the construct validity of the
Integrity-SJT in a multi-ethnic sample.

Overview of Hypotheses
Since both integrity tests and SJTs have shown to be related
to conscientiousness, agreeableness, and emotional stability
(McDaniel & Nguyen, 2001; Ones, 1993, in Wanek,
1999), examining correlations between the present SJT and
these three Big Five dimensions solely will not give much
insight into the question whether the SJT indeed measures
integrity. If, for instance, correlations around .25 are found
in the present study, would this mean that the SJT measures
integrity or would this mean that the test is yet another multidimensional SJT? Therefore, the SJTs convergent validity
was examined by means of the relationship between the SJT
score and several integrity-related measures, namely the
dimension Honesty-Humility of the HEXACO-model (Lee
& Ashton, 2004) and cognitive distortions by means of the
How-I-Think questionnaire (HIT questionnaire; Barriga,
Gibbs, Potter, & Liau, 2001). Also, the discriminant validity
of the SJT is investigated by means of a cognitive ability test.
In the following, we will state the hypotheses and the
arguments for these hypotheses. The rst hypothesis states
that scores on the Integrity-SJT will be related to scores
on other integrity-related dimensions (Hypothesis 1).
A dimension that has shown a strong resemblance to the
concept of integrity is the sixth factor of the recently introduced HEXACO-model of personality (Lee & Ashton,
2004; Lee, Ashton, Morrison, Cordery, & Dunlop, 2008).
This sixth factor is labeled Honesty-Humility and is typically described as honesty, fairness, sincerity, modesty, and
lack of greed. Lee, Ashton, and De Vries (2005) argue that
the dimension Honesty-Humility has a clear conceptual link
to integrity, since both consist of admissions of wrongdoing such as theft, fraud, sabotage, and alcohol and drug
abuse (p. 182). Hence, they investigated the relationship
between Honesty-Humility on the one hand and workplace
delinquency and scores on an overt integrity test on the other
hand. They found correlations of .47 for workplace
2010 Hogrefe Publishing

L. A. L. de Meijer et al.: The Development of a SJT for Integrity

delinquency and .53 for integrity. Therefore, we expect that

the score on the Integrity-SJT will be substantially correlated
to the score on the dimension Honesty-Humility.
The HIT questionnaire is a measure of self-serving cognitive distortions (Barriga et al., 2001). Self-serving cognitive
distortions are inaccurate or biased ways of attending to or
conferring meaning upon experiences associated with externalizing behavior. An example of a person showing self-serving cognitive distortions is someone who has been stealing
something from a shop but who blames the shop owner for
making stealing possible. Barriga and Gibbs (1996) argued
that self-serving cognitive distortions should correlate with
measures of antisocial behavior, such as theft, fraud, aggressive behavior, and disobedience. They found a correlation of
.54 between scores on the HIT questionnaire and aggressive
behavior, and a correlation of .46 between scores on the HIT
questionnaire and delinquent behavior. Therefore, we expect
that the score on the Integrity-SJT will also be substantially
correlated to scores on the HIT questionnaire.
The second hypothesis states that scores on the IntegritySJT will not be related to scores on a cognitive ability test
(Hypothesis 2). Ones and Viswesvaran (1998) showed that
integrity has a negligible correlation with cognitive ability.
Since it is integrity that the SJT intends to measure, we
expect a negligible correlation between the SJT score and
scores on the cognitive ability test.
Finally, the construct validity of the Integrity-SJT will be
examined in a group of Dutch ethnic majority applicants as
well as a group of ethnic minority applicants. Little is known
about potential differences in construct validity across ethnic
groups. There has been some cross-cultural research on
translated integrity tests and potential differences in criterion-related validity (i.e., Fortmann, Leslie, & Cunningham,
2002; Marcus, Lee, & Ashton, 2007). In general, the criterion-related validity ndings point toward equivalence
across cultures. This was examined with North American
integrity tests that were appropriately translated predicting
(counterproductive) work behavior in Canada, Central and
South America, South Africa, and Germany. Equivalence
in construct validity between ethnic groups differing in mastery of the language used in the test, however, has to our
knowledge not been examined until this point in time.
Therefore, the central question in this present study is
whether the correlations between the Integrity-SJT on the
one hand and other integrity-related measures and the cognitive ability test on the other hand are similar across the ethnic majority and minority group (Research Question).

Method
Sample and Procedure
Data came from ethnic majority and ethnic minority applicants who applied for a position at the Police Academy of
The Netherlands in the period from June 2006 to August
2006. The dataset consisted of 203 applicants (59% male;
Mage = 23.34, SD = 5.98), of which 151 were ethnic majority
2010 Hogrefe Publishing

231

applicants (58% male, Mage = 23.32, SD = 6.03) and 52 were

ethnic minority applicants (62% male, Mage = 23.39,
SD = 5.89). The ethnic minority applicants had a Dutch
Antillean, Moroccan, Surinamese, or Turkish background.
Applicants who are interested in a job as police ofcer rst
apply to the local police force where they want to work after
training. For the selection procedure, the local police forces
routinely send all applicants to the national Police Centre
for Competence Assessment and Monitoring (CCM). During
the selection process of the CCM, several instruments are
used. In the present study, we only used the scores on the cognitive ability test. Next to applicants scores on the cognitive
ability test, scores on the Integrity-SJT, on the in-depth
Honesty-Humility interview, and on the HIT-questionnaire
were used.

Measures
Situational Judgment Test for Integrity
SJTs typically consist of hypothetical scenarios describing
interpersonal work situations in which a problem has arisen.
The scenario may represent an actual situation on the target
job or a situation constructed in such a manner that it is psychologically identical to an actual work situation (Chan &
Schmitt, 1997). Scenarios within the test are usually developed on the basis of a critical-incident analysis involving
subject matter experts.
An approach analogous to earlier SJT studies was used
for the development of the Integrity-SJT (see, e.g., Weekley
& Jones, 1997; for an example of an SJT item, see
Appendix A). First, we collected realistic critical incidents
regarding interactions between police ofcers and civilians
or among police colleagues from 15 experienced police ofcers (both policemen and policewomen; both ethnic majority and minority police ofcers; police experts had around
15 years of police work experience). All incidents focused
on integrity violations and potential reactions to these violations. For example, several incidents dealt with resisting
fraudulent people or situations. Second, critical incidents
that were similar were grouped and scenarios were written
about each of these groups of critical incidents. Simultaneously, with the help of the experienced police ofcers,
four response options were derived for each scenario. This
procedure resulted in 14 SJT items (a scenario including
its four response options is labeled item) that were
pilot-tested in a written version of the test. Third, after examining the descriptives and the factor-analytic results of the
pilot-study data (N = 228, 72% male, Mage = 24.08;
SD = 6.78), 3 of the 14 SJT items were eliminated.
Subsequently, a video-based version of the test was developed. Both professional actors and police ofcers were
trained to act in scenarios. After this training, the scenarios
were videotaped in a professional manner. Finally, a panel
of experts was asked to ll out the video-based SJT in order
to develop a scoring key. The expert panel consisted of 50
experienced police ofcers with on average 14.06 years of
work experience (SD = 6.38) and with different ethnic backgrounds, namely 10 ethnic majority experts, 10 Antillean
European Psychologist 2010; Vol. 15(3):229236

232

L. A. L. de Meijer et al.: The Development of a SJT for Integrity

experts, 10 Moroccan experts, 10 Surinamese experts, and 10

Turkish experts. They had to evaluate each response option on
its effectiveness given the situation presented in the scenario.
Agreement among the experts in effectiveness ratings was
generally satisfactory (mean intraclass correlation
(ICC) = .70), both within ethnic groups (mean ICC = .69)
and between ethnic groups (mean ICC = .69). Since
agreement among experts was satisfactory, the scoring key
was set at the modus of the total expert group. The absolute
difference between the scoring key of a given item response
option and the applicant response formed the applicant score,
varying from 4 (largest difference between expert and applicant response) to 0 (no difference between expert and applicant response). The applicant score was subtracted from 4,
in order to have an intuitively logical range from 0 (lowest
possible integrity score) to 4 (highest possible integrity score).
In its nal form, the video-based SJT consists of short,
videotaped scenarios of key integrity issues that police ofcers are likely to encounter with civilians or with police colleagues. A narrator introduces each scenario. Per SJT item,
the scene freezes at an important point and the applicant has
to answer the responses related to the scene presented. The
11 items have four response options each. Applicants have
to evaluate each response option in terms of its effectiveness
within the given situation. This response instruction generally is known as a knowledge response instruction.
The SJT structure was analyzed with structural equation
modeling using Amos 6.0 (Arbuckle, 2005), consisting of
one factor and four subfactors. The model showed a good
t to the data (v2 [df = 1] = 1.09, ns; TLI = .96;
CFI = 1.00; RMSEA = .02). The four subfactors represent
meaningful clusters of response options (i.e., Factor 1 represented applicant scores on the response option that can generally be described as It is alright for this time. Factor 2:
It is not permitted!! (in a stern way), Factor 3: These are
the rules, so it is not allowed. (in a more friendly way), and
Factor 4: It is not allowed and I have to report it to the
supervisor!), which all loaded signicantly (.28 < b < .55,
p < .05) on one general SJT factor. The error terms of Factor 1 and Factor 3 appeared to be somewhat negatively correlated (r = .20, ns). The internal consistency of the SJT
was .69. In further analyses, the total SJT score was used.

Other Integrity Measures

In-Depth Interview
The in-depth interview was built around the sixth factor of
the HEXACO-model (Lee & Ashton, 2004) HonestyHumility and its four subdimensions: Modesty, Honesty,
Morality, and Avoidance of materialism (for denitions,
see Appendix B). The interviews were semistructured and
behaviorally based. The interviewer and an assessor, who
was present during the interview, independently made ratings on a 7-point Likert scale ranging from 1 (extremely
weak) to 7 (excellent), on the factor Honesty-Humility and
each of the four subdimensions. Interrater reliabilities ranged
from .63 to .78 (N = 203). The interview ratings were used
European Psychologist 2010; Vol. 15(3):229236

for further analyses. The model of the in-depth interview,

consisting of one Honesty-Humility factor and four subdimensions, showed a good t to the data (v2 [df = 2] = 2.12,
ns; TLI = 1.00; CFI = 1.00; RMSEA = .01). All subdimensions loaded signicantly (.45 < b < .83, p < .001) on the
Honesty-Humility factor. In further analyses, the dimension
Honesty-Humility was used.

How-I-Think Questionnaire
To measure applicants cognitive distortions, the Dutch
translation (translated from English by Utrecht University,
The Netherlands) of the HIT questionnaire (Barriga et al.,
2001) was used. The HIT questionnaire was developed to
measure two broad dimensions, namely cognitive distortions
and behavioral referents, each consisting of four subdimensions. Cognitive Distortions consist of the subdimensions
Self-centered, Blaming others, Minimizing/Mislabeling,
and Assuming the Worst. Behavioral Referents consist of
the subdimensions Opposition-Deance, Physical Aggression, Lying, and Stealing (for denitions, see Appendix B).
The alpha reliability of the dimension Cognitive Distortions
was .90 and of the dimension Behavioral Referents was .89,
based on the present sample. The alpha reliabilities of the
subdimensions varied from .70 for Blaming others to .79
for Stealing. Two models were tested in accordance with
Barriga et al. (2001), namely: each consisting of one dimension (i.e., Cognitive Distortions or Behavioral Referents)
and four subdimensions. Both models showed a good t
to the data (v2 [df = 2] = 8.82, p < .05, and 1.66, ns;
TLI = .94 and 1.00; CFI = .99 and 1.00; and
RMSEA = .05 and .00). All Cognitive Distortions subdimensions loaded signicantly (.80 < b < .90, p < .001)
on the Cognitive Distortions dimension. All Behavioral Referents subdimension loaded signicantly (.75 < b < .90,
p < .001) on the Behavioral Referents dimension. However,
because the two dimensions Cognitive Distortions and
Behavioral Referents were highly correlated (r = .99,
p < .001), further analyses were conducted with one general
HIT scale consisting of the mean of the two dimensions.

Cognitive Ability Test

The police intelligence test (PIT; Rijks Psychologische
Dienst, 1975) is a cognitive ability test and consists of
107 items divided over six subtests: Verbal Comprehension,
Inductive Reasoning, Numerical Reasoning, Word Fluency,
Spatial Ability, and Picture Arrangement. The time limit is
51 min. Applicants completed the PIT in Dutch. Prior
research by Lem and Van Doorn (2000) indicated alpha reliabilities varying from .69 for Series of Numbers, to .87 for
Folding Figures. A study by Van der Maesen (1992) showed
a corrected predictive validity coefcient of .39 (N = 162)
for the total PIT score predicting work performance. The
model of the test, consisting of one factor for general cognitive ability and six subtests, showed a good t to the data
(v2 [df = 9] = 19.42, p < .05; TLI = .86; CFI = .94; and
2010 Hogrefe Publishing

L. A. L. de Meijer et al.: The Development of a SJT for Integrity

RMSEA = .08). All cognitive ability subtests loaded

signicantly (.23 < b < .68, p < .05) on general cognitive
ability. In further analyses, the total score on the cognitive
ability test was used.

Analyses
Preliminary Analyses
Because response styles can affect answers on questionnaires
(e.g., Van de Vijver & Leung, 1997), structural equivalence
(i.e., absence of bias) was checked across the ethnic majority
and minority group for each measure separately before conducting further analyses. In accordance with Van de Vijver
and Leung (1997), structural equivalence across cultures is
interpreted as follows: A test measures the same trait crossculturally, but not necessarily on the same quantitative scale.
Using an equal measurement weights model in Amos 6.0
(Arbuckle, 2005), no signicant differences between factor
structures of the measures were found between the ethnic
majority group and the minority group (for detailed information, please contact the rst author).
Main Analyses
Using Amos 6.0 (Arbuckle, 2005), a general model was
tested to examine the correlations between the IntegritySJT and the two integrity-related instruments and between
the Integrity-SJT and the cognitive ability test. The model
showed a good t to the data (v2 [df = 4] = 4.73, ns;
TLI = .89; CFI = .98; RMSEA = .03). Multigroup analysis
was conducted to test for cross-ethnic invariance in correlations. Correlations between the SJT, the in-depth interview,
and the HIT-questionnaire were calculated to examine the
convergent validity of the SJT. The correlation between
the SJT and the cognitive ability test was calculated to investigate its discriminant validity.

Results
First, we expected that scores on the Integrity-SJT would
correlate with scores on other integrity-related tests (Hypothesis 1). Second, we expected that scores on the Integrity-SJT
would not correlate with scores on the cognitive ability test
(Hypothesis 2). Finally, we examined potential correlational
differences between the ethnic majority group and the ethnic
minority group (Research Question).
With regard to the Research Question, a multigroup analysis was conducted. A model in which the covariances were
held constant across the ethnic majority and minority group
was compared to the unconstrained model. No signicant
difference was found between the two models (Dv2
[Ddf = 8] = 10.26, ns), meaning that no signicant difference in covariances was found between the two ethnic
groups. To investigate the correlations between scores on
2010 Hogrefe Publishing

233

the SJT and the other measures, therefore, one group without
distinguishing between ethnic majority and minority applicants was used.
With regard to the integrity-related measures, the
observed correlations with the SJT were .23 (p < .05) for
Honesty-Humility and .36 (p < .001) for the HIT questionnaire. On cognitive ability, the correlation with the
SJT was .13 (ns). Thus, concerning the convergent-validity
evidence, the correlations were moderate to high (Cohen,
1988) between the SJT and the in-depth Honesty-Humility
interview and between the SJT and the HIT questionnaire.
Concerning the discriminant-validity results, the correlation
between the SJT and the cognitive ability test was small
(Cohen, 1988). These ndings support Hypotheses 1 and
2 and support the notion that the SJT seems to measure
integrity in both the ethnic majority and minority group.

Discussion
Situational judgment tests recently have gained in popularity
because of its incremental validity over and above cognitive
ability tests and because of its smaller adverse impact against
ethnic minority groups (Clevenger et al., 2001; Motowidlo
et al., 1990; Nguyen & McDaniel, 2003; Weekley & Jones,
1997, 1999). New technology has made the development of
video-based SJTs possible, which show even higher criterion-related validity (Lievens & Sackett, 2006) and less
adverse impact (Chan & Schmitt, 1997) than paper-andpencil SJTs. Furthermore, more and more attention is given
to measuring integrity of applicants during personnel selection because it has shown to be predictive of counterproductive work behavior varying from theft to job performance
(Ones et al., 1993). It is of central importance to determine
the integrity as a work style for police ofcer positions. In
fact, integrity is seen as the most important work style of
police ofcers according to O*Net (O*Net Online, 2009,
May 6). Because of the impact that integrity violations have
on the police organization, a video-based SJT intended to
measure integrity was developed. In a eld study conducted
in a multi-ethnic setting at the Dutch police, we examined the
construct validity of this Integrity-SJT. We investigated convergent and discriminant validity of the SJT, including potential correlational differences between the ethnic majority
group and the ethnic minority group. The largest ethnic
minority groups in The Netherlands have a Dutch Antillean,
a Moroccan, a Surinamese, or a Turkish background.
Firstly, we found no signicant differences in correlations
between the ethnic majority and minority group. Secondly,
correlations between the SJT and the integrity-related measures the in-depth Honesty-Humility interview and the
HIT questionnaire were moderate to high (Cohen,
1988). Finally, the correlation between the SJT and the cognitive ability test was low (Cohen, 1988). These results are in
line with our expectations (Hypotheses 1 and 2) and demonstrate the construct validity of the SJT measuring integrity.
The construct Honesty-Humility includes subdimensions such as Morality (i.e., being able to avoid fraud and
European Psychologist 2010; Vol. 15(3):229236

234

L. A. L. de Meijer et al.: The Development of a SJT for Integrity

corruption and unwilling to take advantage of other individuals or of society at large) and Honesty (i.e., being genuine in
interpersonal relations and unwilling to manipulate others).
The HIT questionnaire contains subdimensions such as
Opposition-Deance (i.e., being disrespectful for rules, laws,
or authorities), Stealing, and Lying. Therefore, the moderate
to high correlations between the SJT and the HonestyHumility interview and between the SJT and the HIT questionnaire showed support for the construct validity of the
SJT.
Additionally, we found a negligible relationship between
the SJT and the cognitive ability test. Regarding integrity
tests, Ones and Viswesvaran (1998) showed that they have
negligible correlations with cognitive ability. Since integrity
was the intended SJT construct, we expected a small correlation between the SJT score and scores on the cognitive
ability test. This was what we found, providing more evidence for the construct validity of the present SJT.

multi-ethnic setting during Dutch police ofcer selection,

we examined the construct validity of a video-based SJT
measuring integrity. We investigated (1) the convergent
and discriminant validity of the SJT and (2) potential correlational differences between the Dutch ethnic majority and
ethnic minority group. First, we found support for the
construct validity of the Integrity-SJT. Second, we found
no signicant differences in correlations between the ethnic
majority and minority group.

Acknowledgments
We acknowledge Hans van Loon and Hellen Westerveld for
their contribution to the execution of this study.

References
Limitations
Our study had some limitations. First, the small sample
size of ethnic minority applicants resulted in small power
concerning the multigroup analysis. With regard to the ethnic minority group, a larger sample size would have
allowed to draw rmer conclusions. Also, a larger sample
size of ethnic minorities would allow a further differentiation within the ethnic minority group. De Meijer, Born,
Terlouw, and Van der Molen (2006) showed that large
score differences on various selection measures exist
between ethnic minority groups, which might be explained
by differences in history and culture between the ethnic
groups. Investigating these ethnic minority groups separately may result in more useful information compared to
merely contrasting the ethnic majority to minority group
and not taking into account potential differences between
ethnic groups.
Second, we did not have criterion data at our disposal to
investigate the criterion-related validity of the present SJT.
Although the construct-validity results are promising, we do
not know whether the present SJT is able to predict job performance, workplace (dis)honesty, theft, fraud, etc. Since little is
known about SJTs measuring a single construct, in general,
and their criterion-related validity, specically, future research
should be focused on these types of SJTs and their predictive
power. Furthermore, SJTs intended to measure a single construct should be developed in different companies, in different
settings, and on different job levels to be able to properly generalize the ndings in the present study.

Conclusion
A critical issue regarding SJTs is the often-found constructheterogeneity. However, we argue that a construct-driven
approach may be fruitful in the development of SJTs measuring one single construct. In a eld study conducted in a
European Psychologist 2010; Vol. 15(3):229236

Arbuckle, J. L. (2005). Amos 6.0 [Computer software]. Chicago:

Smallwaters.
Barriga, A. Q., & Gibbs, J. C. (1996). Measuring cognitive
distortions in antisocial youth: Development and preliminary
validation of the How-I-Think questionnaire. Aggressive
Behavior, 22(3), 333343.
Barriga, A. Q., Gibbs, J. C., Potter, G. B., & Liau, A. K. (2001).
The How-I-Think questionnaire: Manual. Champaign, IL:
Research Press.
Becker, T. E. (2005). Development and validation of a situational
judgment test of employment integrity. International Journal
of Selection and Assessment, 13(3), 225232.
Chan, D., & Schmitt, N. (1997). Video-based versus paper-andpencil method of assessment in situational judgment tests:
Subgroup differences in test performance and face validity
perception. Journal of Applied Psychology, 82(1),
143159.
Chan, D., & Schmitt, N. (2005). Situational judgment tests. In
A. Evers, N. Anderson, & O. Voskuijl (Eds.), Handbook of
personnel selection (pp. 219246). Oxford: Blackwell.
Clevenger, J., Pereira, G. M., Wiechmann, D., Schmitt, N., &
Harvey, V. S. (2001). Incremental validity of situational
judgment tests. Journal of Applied Psychology, 86(3),
410417.
Cohen, J. (1988). Standard power analysis for the behavioral
sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Dalessio, A. T. (1994). Predicting insurance agent turnover using
a video-based situational judgment test. Journal of Business
and Psychology, 9(1), 2332.
De Meijer, L. A. L., Born, M. Ph., Terlouw, G., & Van der
Molen, H. T. (2006). Applicant and method factors related to
ethnic score differences in personnel selection: A study at the
Dutch police. Human Performance, 19(3), 219251.
Fortmann, K., Leslie, C., & Cunningham, M. (2002). Crosscultural comparisons of the Reid Integrity Scale in Latin
America and South Africa. International Journal of Selection
and Assessment, 10(1/2), 98108.
Jones, J. W., Brasher, E. E., & Huff, J. W. (2002). Innovations in
integrity-based personnel selection: Building a technologyfriendly assessment. International Journal of Selection and
Assessment, 10(1/2), 8797.
Lee, K., & Ashton, M. C. (2004). Psychometric properties of the
HEXACO personality inventory. Multivariate Behavioral
Research, 39(2), 329358.
Lee, K., Ashton, M. C., & De Vries, R. E. (2005). Predicting
workplace delinquency and integrity with the HEXACO and
2010 Hogrefe Publishing

L. A. L. de Meijer et al.: The Development of a SJT for Integrity

ve-factor models of personality structure. Human Performance, 18(2), 179197.

Lee, K., Ashton, M. C., Morrison, D. L., Cordery, J., & Dunlop,
P. D. (2008). Predicting integrity with the HEXACO personality model: Use of self- and observer reports. Journal of
Occupational and Organizational Psychology, 81(1),
147167.
Lem, J., & Van Doorn, E. (2000). Voortgangsrapportage.
Onderzoek kenmerkende voorspellers politie [Progress
report. Study noticeable predictors police]. Culemborg,
The Netherlands: Meurs Personeelsadvies.
Lievens, F., & Sackett, P. R. (2006). Video-based versus written
situational judgment tests: A comparison in terms of
predictive validity. Journal of Applied Psychology, 91(5),
11811188.
Marcus, B., Lee, K., & Ashton, M. C. (2007). Personality
dimensions explaining relationships between integrity tests
and counterproductive behavior: Big Five, or one in
addition? Personnel Psychology, 60(1), 134.
McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb,
W. L. III (2007). Situational judgment tests, response
instructions, and validity: A meta-analysis. Personnel Psychology, 60(1), 6391.
McDaniel, M. A., Morgeson, F. P., Bruhn Finnegan, E.,
Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarication of the literature. Journal of Applied Psychology, 86(4),
730740.
McDaniel, M. A., & Nguyen, N. T. (2001). Situational judgment
tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9(1/2), 103113.
Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An
alternative selection procedure: The low-delity simulation.
Journal of Applied Psychology, 75(6), 640647.
Naeye, J., Huberts, L., Van Zweden, C., Busato, V., & Berger, B.
(2004). Integriteit in het dagelijkse politiework [Integrity
during daily police work]. Amsterdam: Vrije Universiteit.
Nguyen, N. T., & McDaniel, M. A. (2003). Response instructions and racial differences in a situational judgment test.
Applied H.R.M. Research, 8(1), 3344.
Occupational Information Network (O*NET). (2009, May 6).
OnLine developed for the US Department of Labor by the
National O*NET Consortium. Retrieved May 6, 2009, from
http://www.online.onetcenter.org.
OConnell, M. S., Hartman, N. S., McDaniel, M. A., Grubb,
W. L., & Lawrence, A. III (2007). Incremental validity of
situational judgment tests for task and contextual performance. International Journal of Selection and Assessment,
15(1), 1929.
Olson-Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D.,
Keenan, P. A., & Donovan, M. A. (1998). An interactive
video assessment of conict resolution skills. Personnel
Psychology, 51(1), 124.
Ones, D. S. (1993). The construct validity evidence for integrity
tests. University of Iowa. Iowa City: Unpublished doctoral
dissertation.
Ones, D. S., & Viswesvaran, C. (1998). Gender, age, and race
differences on overt integrity tests: Results across four largescale job applicant data sets. Journal of Applied Psychology,
83(1), 3542.
Ones, D. S., Viswesvaran, C., & Schmidt, F. L. (1993). Comprehensive meta-analysis of integrity test validities: Findings
and implications for personnel selection and theories of job
performance [Monograph]. Journal of Applied Psychology,
78(4), 679703.
Richman-Hirsch, W. L., Olson-Buchanan, J. B., & Drasgow, F.
(2000). Examining the impact of administration medium on
examinee perceptions and attitudes. Journal of Applied
Psychology, 85(6), 880887.
2010 Hogrefe Publishing

235

Rijks Psychologische Dienst. (1975). Politie intelligentie test

[Police intelligence test]. The Hague, The Netherlands: RPD.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of
selection methods in personnel selection: Practical and
theoretical implications of 85 years of research ndings.
Psychological Bulletin, 124(2), 262274.
Van der Maesen, P. E. A. M. (1992). Het rendement van
personeelsselectie [The efciency of personnel selection].
The Netherlands: Rijksuniversiteit Groningen, Unpublished
doctoral dissertation.
Van de Vijver, A. J. R., & Leung, K. (1997). Methods and data
analysis for cross-cultural research. Thousand Oaks, CA:
Sage.
Van Iddekinge, C. H., Taylor, M. A., & Eidson, C. E., Jr. (2005).
Broad versus narrow facets of integrity: Predictive validity and
subgroup differences. Human Performance, 18(2), 151177.
Wanek, J. E. (1999). Integrity and honesty testing: What do we
know? How do we use it? International Journal of Selection
and Assessment, 7(4), 183195.
Weekley, J. A., & Jones, C. (1997). Video-based situational
judgment testing. Personnel Psychology, 50(1), 2549.
Weekley, J. A., & Jones, C. (1999). Further studies of situational
tests. Personnel Psychology, 52(3), 679700.

Received October 10, 2007

Accepted September 15, 2009
Published online March 16, 2010
About the authors
Dr. Lonneke A. L. de Meijer is assistant professor of Industrial
and Organizational Psychology at Erasmus University, Rotterdam, The Netherlands. Her research examines the role of ethnic
diversity in personnel selection, with a special focus on the
development of innovative tests.
Prof. Dr. Marise Ph. Born is professor of Personnel Psychology
at Erasmus University Rotterdam, The Netherlands, and President of the International Test Commission. Her research focuses
on personnel selection, test development, and cross-cultural
psychology.
Dr. Jaap van Zielst is head of the national Police Centre for
Competence Assessment and Monitoring at the Police Academy
of The Netherlands.
Prof. Dr. Henk T. van der Molen is Dean of the Faculty of Social
Sciences and professor of Psychology at Erasmus University,
Rotterdam, The Netherlands. His research interests are (effectiveness of) communication skills training programs, problembased learning, and innovations in education.

Lonneke A.L. de Meijer

Institute of Psychology
Erasmus University
Rotterdam FSW
Woudestein T13-24
P.O. Box 1738
NL-3000 DR Rotterdam
The Netherlands
Tel. +31 10 408 8678
Fax +31 10 408 9009
E-mail demeijer@fsw.eur.nl
European Psychologist 2010; Vol. 15(3):229236

236

L. A. L. de Meijer et al.: The Development of a SJT for Integrity

Appendix A

Possible Reactions of Police Officer 2

Example of Integrity-SJT Item

1. Dont worry about it! Police cars are covered with

scratches. (Factor 1, It is alright for this time.)
2. O. . . Im sorry. If I were you, I would report it to the
chief. (Factor 3, These are the rules, so it is not
allowed. (in a friendly way))
3. Well, thats pretty stupid of you!! You have to report it
to the chief! (Factor 2, It is not permitted! (in a stern
way))
4. The only thing you can do is to report it to the chief!
And if youre not going to do it, I will!! (Factor 4,
It is not allowed and I have to report it to the
supervisor!)

Description of Situation
A police ofcer (police ofcer 1) comes to work on his
motorbike. When he enters the parking garage of the police
station he accidentally hits a police car, causing a big scratch
on the police car. Shortly after, he meets a colleague (police
ofcer 2) and tells her what happened.
Police Officer 1
Hi! Listen: I just entered the parking garage with my
motorbike and caused a big scratch on one of the police cars.
I feel really bad about it and, actually, I dont know what to
do.

Appendix B
Integrity-Related Dimensions and Their Descriptions
Dimension
In-depth interview
Modesty

Description
Being modest, unassuming, and seeing oneself as an ordinary person
without any claim to special treatment

Honesty

Being genuine in interpersonal relations and unwilling to manipulate others.

Morality

Being able to avoid fraud and corruption and unwilling to take advantage
of other individuals or of society at large

Avoidance of materialism

Being uninterested in possessing lavish wealth,

luxury goods, and signs of high social status

How-I-Think questionnaire
Self-centered

According status to ones own view, expectations, needs, rights,

immediate feelings, and desires to such a degree that the legitimate views, etc.,
of others are scarcely considered or are disregarded altogether

Blaming others

Misattributing blame to outside sources or misattributing blame for

ones victimization or other misfortune to innocent others

Minimizing/mislabeling

Depicting antisocial behavior as causing no real harm or referring

to others with a belittling or dehumanizing label

Opposition-deance

Being disrespectful for rules, laws, or authorities

Note. Denitions of the facets of the in-depth Integrity interview are from Lee and Ashton (2004) and denitions of the (sub-)
dimensions of the HIT questionnaire are from Barriga et al. (2001). Denitions of the subdimensions Physical Aggression, Stealing,
and Lying of the HIT were not listed here, because we assumed that they are self-explanatory.

European Psychologist 2010; Vol. 15(3):229236

2010 Hogrefe Publishing