Vous êtes sur la page 1sur 32

Cross-Cultural Research

http://ccr.sagepub.com/

Cross-Cultural Research on the Reliability and Validity of the


Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT)
Jahanvash Karim and Robert Weisz
Cross-Cultural Research published online 2 August 2010
DOI: 10.1177/1069397110377603
The online version of this article can be found at:
http://ccr.sagepub.com/content/early/2010/07/27/1069397110377603

Published by:
http://www.sagepublications.com

On behalf of:
Society for Cross-Cultural Research

Additional services and information for Cross-Cultural Research can be found at:
Email Alerts: http://ccr.sagepub.com/cgi/alerts
Subscriptions: http://ccr.sagepub.com/subscriptions
Reprints: http://www.sagepub.com/journalsReprints.nav
Permissions: http://www.sagepub.com/journalsPermissions.nav

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

377603

CCR

Cross-Cultural Research OnlineFirst, published on August 2, 2010 as


doi:10.1177/1069397110377603

Cross-Cultural Research
on the Reliability and
Validity of the MayerSalovey-Caruso Emotional
Intelligence Test (MSCEIT)

Cross-Cultural Research
XX(X) 131
2010 SAGE Publications
Reprints and permission: http://www.
sagepub.com/journalsPermissions.nav
DOI: 10.1177/1069397110377603
http://ccr.sagepub.com

Jahanvash Karim and Robert Weisz

Abstract
Despite the rather large literature concerning emotional intelligence, the
vast majority of studies concerning development and validation of emotional
intelligence scales have been done in the Western countries. Hence, a major
limitation in this literature is its decidedly Western focus. The aim of this
research was to assess the psychometric properties of the Mayer-SaloveyCaruso Emotional Intelligence Test (MSCEIT) in a cross-cultural comparative
context involving the collectivist Pakistani (Eastern culture) and the individualist
French (Western culture) students. With the exception of significant mean
differences on the MSCEIT scores between two cultures, the results concerning
the validity of the MSCEIT generalized nicely across both cultures. The results
from multisample analysis revealed that the MSCEIT has the property of factorial
invariance across both cultures, including invariance of factor loadings, unique
variances, and factor variance. For both Pakistani and French students, the
MSCEIT scores were distinguishable from the Big Five personality dimensions,
self-report emotional intelligence measures, and cognitive intelligence.
Furthermore, in both cultures, the MSCEIT scores failed to demonstrate
incremental validity against well-being measures, after controlling for cognitive
intelligence and the Big Five personality dimensions. Finally, within each sample,
females significantly scored higher than males on the MSCEIT total scores.
Keywords
Emotional intelligence; validity
Universit de Paul Czane, France
Corresponding Author:
Jahanvash Karim, CERGAM, IAE dAix en Provence, Universit de Paul Czane, France, Clos
Guiot PuyricardBP
30063, Aix-en-Provence Cedex 2 13089, France
Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010
Email: J_vash@hotmail.com

Cross-Cultural Research XX(X)

Emotional intelligence (EI) exists and has significant impacts on individual


and organizational outcomes, ranging from individual performance, health,
and psychological well-being, to customer satisfaction and organizational
performance (Joseph & Newman, 2010; Schutte, Malouff, Thorsteinsson,
Bhullar, & Rooke, 2007). Indeed, Few fields of psychological investigation
appear to have touched so many disparate areas of human endeavor,
since its inception, as has emotional intelligence (Matthews, Zeidner, &
Roberts, 2004, p. 4). However, despite notable advances in the field, the
psychometric properties of EI instruments have seldom been examined with
demand and rigor across cultures, often leaving open questions of structural
and measurement equivalence. If an EI measure fails to show comparable
psychometric properties across different cultures, then its utility as a
psychological construct is questionable (Ekermans, 2009; Gangopadhyay &
Mandal, 2008; Palmer, Gignac, Ekermans, & Stough, 2008). The current study
sought to address this concern by simultaneously assessing the psychometric
properties of the Mayer-Salovey-Caruso Emotional Intelligence Test
(MSCEIT: Mayer, Salovey, & Caruso, 2002) in two distinct cultural groups:
the collectivist Pakistani and the individualist French. The collectivists tend
to view themselves as members of an extended family (or organization), and
place group interests ahead of individual needs. In contrast, Individualists
tend to believe that personal goals and interests are more important than
group interests (Hofstede, 1980).
A cross-cultural design is an answer to the call made by various researchers (e.g., Ekermans, 2009; Gangopadhyay & Mandal, 2008) for more systematically investigating cultural differences to determine whether the structure
of EI replicates across distinct cultures and whether correlates of EI are culture-specific or they cut across cultural boundaries. More specifically, the
current study had five main objectives. First, we compared participants EI
levels across both cultures. Second, we evaluated the structural equivalence
of the MSCEIT across both cultures. Third, we assessed the discriminant
validity of the MSCEIT vis--vis cognitive intelligence (the Ravens
Advanced Progressive Matrices), self-report or mixed model EI measures
(the SREIT and the TEIQue), and the Big Five personality measures. Fourth,
we assessed whether MSCEIT accounts for incremental variance in subjective well-being (i.e., positive affect, negative affect, and satisfaction with life)
and psychological distress above and beyond Big Five and cognitive intelligence in both cultures. Finally, we examined whether there are gender differences on the MSCEIT within each culture.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

Approaches to EI
Two complementary conceptualizations of EIthat is, mixed model framework and ability model frameworkexist side by side in the literature. The
proponents of ability EI framework view EI as a traditional intelligence,
resembling other standard intelligences (e.g., verbal, numerical, figural), comprising of a set of skills that combines emotions with cognition measured
through objective tests akin to IQ tests ([MSCEIT: Mayer et al., 2002).
Proponents of mixed models, by contrast, view EI as an eclectic mix (Mayer,
Salovey, &Caruso, 2008) of traits, many dispositional, such as self-esteem,
happiness, impulsiveness, self-management, and optimism, rather than as ability based. For example, Petrides and Furnham (2003) defined the construct as
a constellation of behavioral dispositions and self-perceptions concerning
ones ability to recognize, process, and utilize emotion-laden information. It
encompasses empathy, impulsivity, and assertiveness as well as elements of
social intelligence and personal intelligence (p. 278). Thus, within these models, a large number of traits are amassed and mixed in with a few socioemotional abilities (Mayer, Roberts, & Barsade, 2008; Mayer, Salovey et al.,
2008). Researchers in the mixed model framework have typically used selfreport measures to assess EI (e.g., Self-report Emotional Intelligence Test
[SREIT]: Schutte et al., 1998; Trait Emotional Intelligence Questionnaire
[TEIQue]: Petrides, Prez-Gonzalez, & Furnham, 2007). Furthermore, mixed
model measures of EI can typically be organized into one of two complementary types: self-report ability EI or self-report mixed EI (Joseph & Newman,
2010). The former includes self-report EI measures that are based on ability EI
model (e.g., Self-Rated Emotional Intelligence Scale [SREIS]: Brackett,
Rivers, Shiffman, Lerner, & Salovey, 2006; Self-report Emotional Intelligence
Test [SREIT]: Schutte et al., 1998). The latter includes measures which focus
on noncognitive factors such as social skills, self-esteem, and personality
dimensions (e.g., TEIQue: Petrides et al., 2007). In sum, currently we have
three distinct construct-method pairings of EI: performance-based ability EI,
self-report ability EI, and self-report mixed EI.

Mayer-Salovey-Caruso Emotional Intelligence


Test (MSCEIT: Mayer et al., 2002)
One of the more widely known ability EI models was developed by Mayer
and Salovey (1997), who defined EI as the ability to perceive emotions,
to access and generate emotions so as to assist thought, to understand emotions and emotional knowledge and to reflectively regulate emotions so as to

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Cross-Cultural Research XX(X)

promote emotional and intellectual growth (p. 10). In line with this operational definition, the structure of Mayer and Saloveys model is mutifactorial, comprising four conceptually related abilities arranged hierarchically
from the most basic to more psychologically complex. These include (a)
Perceiving Emotions, or the ability to identify emotions in oneself and others; (b) Using Emotions, or the ability to use emotions to impact cognitive
processes. This requires the ability to mobilize the appropriate emotions and
feelings to assist in certain cognitive activities such as reasoning, problemsolving, and decision making; (c) Understanding Emotions, or the ability to
comprehend how emotions combine and how emotions progress by transitioning from one emotion to another; and (d) Managing Emotions, or the
ability to reflectively regulate emotions and emotional relationships.
The Mayer-Salovey-Caruso Emotional Intelligence Test Version 2.0
(MSCEIT; Mayer et al., 2002) is the direct operationalization of Mayer and
Saloveys (1997) ability EI model. In line with ability EI conceptualization,
the MSCEIT measures ones capacity to reason with emotional content and to
use the emotional content to enhance thought. The MSCEIT differs from the
mixed model or trait measures of EI (self-reporting EI measures) as a result of
the nature and style of the assessment. Respondents are asked to solve emotional problems (e.g., how to resolve a conflict with a spouse) rather than
being asked to self-perceive and rate the extent to which their emotional skills
are being used (e.g., rating oneself on 7-point Likert-type scale).
The MSCEIT includes two tasks as measures of each branch: Perceiving
Emotions (faces and pictures); Using Emotions (sensations and facilitation);
Understanding Emotions (blends and changes); and Managing Emotions
(emotion management and emotional relationships). The four branches may
be further grouped into two EI areas: Experiential EI (Perceiving Emotions
and Using Emotions) and Strategic EI (Understanding Emotions and
Managing Emotions).
Scores on the MSCEIT can be obtained through consensus and expert
scoring methods. Consensus scores reflect the proportion of respondents in a
large normative sample who endorsed each MSCEIT response. A score for an
individual is computed by comparing his or her responses to that of the normative sample. In contrast, expert scores reflect the proportion of 21 emotion
experts who endorsed each response. It is worth to mention that the scoring
methods of the MSCEIT have been the subject of debate and controversy
(e.g., Matthews et al., 2002; Matthews, Roberts, & Zeidner, 2004). Both consensus and expert norms correlate highly (Mayer, Salovey, Caruso, &
Sitarenios, 2003). Mayer et al. (2003) reported acceptable reliabilities for the

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

MSCEIT. The MSCEIT full-test split-half reliability was 0.93 for general and
0.91 for expert consensus scoring. The reliability for four branch scores of
Perceiving, Facilitating, Understanding, and Managing ranged between 0.76
and 0.91. The individual task reliabilities ranged from a low of 0.55 to a high
of 0.88.

Overview of the Current Study


Country Differences on the MSCEIT
Individualism-Collectivism is a major dimension of cultural variable postulated by many theorists (e.g., Hofstede, 1980). This dimension focuses on the
degree a society reinforces individual or collective actions, achievements,
and interpersonal relationships. Collectivism typifies societies of a more collective nature, close ties between individuals, collective goals, and dependence on groups; while individualistic cultures stress individual goals and
independence. For this study, French and Pakistani cultures were selected
because cross-cultural research predominantly involves the comparison of
Eastern and Western cultures. According to Hofstedes (1980) cultural
dimensions, Pakistan is a typical representative of the classical Eastern culture. France is considered as a prototype of the classical Western culture.
According to Hofstedes rankings (see www.geert-hofstede.com), Pakistan
ranks 14 on individualism which is much lower than the world average of 50,
reflecting an orientation toward a collectivistic culture. France ranks 71,
indicating a society with more individualistic attitudes.
Some research indicates that cultural differences (individualist vs. collectivist) exist across a wide range of emotion-related abilities that essentially
comprise the construct of the ability EI. For example, compared to collectivists, people from individualistic cultures are better at recognizing and understanding emotions (Matsumoto, 1989; 1992), are more likely to express their
emotions (Fernandez, Carrera, Sanchez, Paez, & Candia, 2000), and are better able to regulate their emotions ( Gross & John , 2003). Thus the processes
underlying the ability EI factors and their manifestations may differ across
cultures as a consequence of the role culture plays in the development, display, and interpretation of emotions. Therefore, the first goal was to examine
whether there are cultural differences on the MSCEIT scores across French
(individualists) and Pakistani (collectivists) cultures. Based on literature on
emotions, it was expected that the participants in the French sample would
score higher on the MSCEIT than the participants in the Pakistani sample.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Cross-Cultural Research XX(X)

Structural Equivalence
The evidence for structural equivalence can be established by replicating the
factor structure of the MSCEIT and demonstrating that the MSCEIT possesses robust internal reliability across cultures (Ekermans, 2009).
Regarding the factorial validity of the MSCEIT, Mayer et al. (2003) have
demonstrated that four-factor models provide good fit to the data, suggesting
that this model provides viable representation of the tests underlying factor
structure. However, some have argued that four-factor solution is not preferable due to high correlations between branches perceiving emotions and
using emotions (Fan, Jackson, Yang, Tang, & Zhang, 2010; Roberts, Schulze,
OBrien, MacCann, Reid, & Maul, 2006; Rode et al., 2007; Rossen, Kranzler,
& Algina, 2008) or between using emotions and managing emotions (Palmer,
Gignac, Manocha, & Stough, 2005).
Further to this, an important research question that has yet to be systematically examined is whether the ability EI construct generalizes across different
cultures. So far, all factor structures of the MSCEIT have evolved only on the
basis of studies done in Western (predominantly individualistic) cultures and
none has assessed the factor structure of the MSCEIT in Eastern (primarily
collectivistic) cultures. It is argued that the construct of EI needs to be validated in the East (collectivist culture; Gangopadhyay & Mandal, 2008).
When tests are transported from one culture to another, the comparability of
psychological measurements across different cultures should be investigated.
A lack of evidence for measurement invariance across cultures could point
toward bias at the construct level (Ekermans, 2009) and obviates the ability
of the measure to be used in comparisons among different cultural groups.
For instance, one of the major objectives of any cross-cultural study is to
compare the mean level of a certain construct across cultural groups.
Interpretation of the mean differences may be problematic unless the underlying constructs are the same or invariant across cultural groups. Therefore,
if the MSCEIT is used to compare mean differences across cultures, the
MSCEIT should have the same meaning across cultural groups. In sum, if
equivalence assumptions remain untested, the practical utility of EI when
utilized across different cultural groups may be questionable (Ekermans,
2009).
The MSCEIT normative sample is based on data collected from more than
5,000 participants, including individuals from both individualistic societies
(e.g., United States, United Kingdom, Canada) as well as from collectivistic
societies (e.g., the Philippines, India, and Slovenia; Papadogiannis, Logan, &

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

Sitarenios, 2009). Thus one can argue that the structure of the MSCEIT will
replicate identical across cultures because of the heterogeneous nature of normative sample, including collectivists and individualists. It is expected that
ability EI factors are culturally universal and have comparable functions
across cultures.

Discriminant Validity
Discriminant validity is observed when the scores from an EI inventory are
found not to correlate with an inventory that is theoretically postulated to be
unrelated to EI (Gignac, 2009).
MSCEIT and self-report EI measures. Since self-report EI measures assess
emotion-related, self-perceived abilities and traits rather than cognitive abilities per se (as in ability EI), mixed EI and ability EI should be regarded distinct (Mayer, Roberts et al.,2008; Mayer, Salovey et al., 2008; Petrides, &
Furnham, 2003). Research has consistently supported this distinction by
revealing low correlations between the MSCEIT and various self-report
mixed EI measures (e.g., Brackett & Mayer, 2003; Livingstone, & Day, 2005;
OConnor & Little, 2003) and self-report ability EI measures (Brackett &
Mayer, 2003; Brackett et al., 2006; Joseph & Newman, 2010).
MSCEIT and Cognitive Intelligence. Empirically, there appears to be sufficient discriminant validity between the MSCEIT and various general intelligence measures (Papadogiannis et al., 2009). Various studies have indicated
low to moderate correlations between MSCEIT and measures tapping crystallized intelligence (Gc; e.g., Farrelly & Austin, 2007; Livingstone & Day,
2005; OConnor & Little, 2003; Rode et al., 2007). Interestingly, the MSCEIT
has shown no relation to Ravens Progressive Matrices (Raven, Raven, &
Court, 2003; e.g., Ciarrochi, Chan, & Caputi, 2000; Fabio & Palazzeschi,
2009; Farrelly & Austin, 2007), placing ability EI closer to crystallized
(rather than fluid, i.e., Gf) intelligence within Gf/Gc theory (for details see
Farrelly & Austin, 2007).
MSCEIT and the Big Five. Various studies have well-documented a nonsignificance or low correlations between the MSCEIT and the Big Five personality dimensions, thus providing evidence for the discriminant validity of the
MSCEIT (e.g., Joseph & Newman, 2010; OConnor & Little, 2003; Rode
et al., 2007).
Therefore, the third goal in the present study was to examine the relationship of scores on the MSCEIT with scores on the TEIQue, the SREIT, Ravens

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Cross-Cultural Research XX(X)

Progressive Advance Matrices, and the Big Five personality dimensions. It


was expected that the scores on the MSCEIT would be unrelated to the scores
on the SREIT, TEIQue, Ravens Progressive Matrices, and the Big Five personality dimensions.

Incremental Validity
There are many reasons to believe that EI plays an important role in predicting ones subjective sense of well-being and positive mental health. For
example, emotionally intelligent individuals (a) are better able to draw on
positive emotions, which help them to handle anxiety and tolerate distress
even when faced with episodes of negative emotional experiences (Tugade
& Fredrickson, 2001); (b) are more likely to use strategies such as eliciting
social support and disclosure of feelings, in place of maladaptive coping
strategies, such as rumination (Matthews, Emo, Funke, Zeidner, Roberts, &
Costa, 2006); (c) are more likely to retrieve positive memories during mood
induction as an aid to mood regulation (Ciarrochi et al., 2000); (d) have an
advantage in terms of greater social competence, richer social networks, and
more effective coping strategies (Salovey, Bedell, Detweiler, & Mayer,
2000); and (e) are better able to identify and interpret cues that inform selfregulatory actions to nurture positive affect and avoid negative affect (Mayer
& Salovey, 1997). In a study conducted on undergraduate students, Brackett
et al. (2006) found positive relationship between the MSCEIT and psychological well-being and life satisfaction.
The fourth goal of this study was to examine whether scores on the
MSCEIT predict scores on measures assessing subjective well-being (positive affect, negative affect, and life satisfaction) and psychological distress
after controlling for the influence of personality and cognitive intelligence.
As discussed above, MSCEIT is unrelated to fluid intelligence (Gf) and Big
Five personality dimensions; it is expected that MSCEIT will exhibit significant incremental validity over Big Five personality traits and cognitive intelligence (i.e., Ravens Advanced Progressive Matrices).

Known Group Validation


Gender differences have been reported consistently in emotions research. For
example, compared with men, women are more accurate in judging the emotional meaning from nonverbal cues (Hall & Matsumoto, 2004), have more

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

complex knowledge (Ciarrochi, Hynes, & Crittenden, 2005), use more emotion regulation strategies (Garnefski, Teerds, Kraaij, Legerstee, & Van den
Kommer, 2004), experience their emotions more intensely (Gross, & John,
1998), show greater emotional awareness (Barrett, Lane, Sechrest, &
Schwartz, 2000), and tend to be more empathic than men (Mehrabian,
Young, & Sato, 1988). More important, the MSCEIT manual (Mayer et al.,
2002) and a recent meta-analytic study by Joseph and Newman (2010) suggest that women score higher on the MSCEIT four factors than men do.
Therefore, the fifth and final research goal was to examine whether there are
gender differences on scores on the MSCEIT. Based on the literature review,
it was expected that women would score higher on the MSCEIT than men
would.

Method
Participants
Participants of this study included 192 students from two nonnative English
speaking national cultures: 111 from a university in Aix-en-Provence, France
(49 males, 62 females), and 81 from a large university in the province of
Balochistan, Pakistan (52 males and 29 females). To attain sample equivalency, participants in both cultures were recruited from the management
sciences subject pool fully conversant with English language and were
enrolled in programs where the medium of instruction was English. As all
students (in both cultures) indicated that they had good command of English
and were able to complete the instruments in the English language, they
completed the English versions of all instruments. Participants received class
credit for their participation. The average age of the participants was
29.46 (SD= 8.46). The French sample included 60% students from regular
master programs and 40% from executive programs. The Pakistani sample
included 73.5% from regular master programs and the rest were from executive programs.
All participants took the MSCEIT and other tests in two testing sessions,
each lasting 90 min. Eight classes participated in the study. The number of
students for each group was between 20 and 40. The questionnaires were
presented in the same order in all groups. All participants were treated in
accordance with the Ethical principles of Psychologists and Code of
Conduct (American Psychological Association, 2002).

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

10

Cross-Cultural Research XX(X)

Measures
TEIQue. The TEIQue (Petrides et al., 2007) is predicated on trait EI theory,
which conceptualizes EI as a personality trait, located at the lower levels of
personality hierarchies (Petrides & Furnham, 2003). The sampling domain of
the TEIQue comprises 15 emotion-related behavioral dispositions (e.g., happiness, self-control, self-motivation) thought to affect the ways individuals
cope with demands of the situation. These 15 emotion-related behavioral dispositions (traits) are theoretically arranged into four broader or major conceptual components. These include (a) well-being, representing how
successfully one is able to enjoy life and maintains a positive disposition
(e.g., On the whole, Im pleased with my life); (b) self-control, representing the ability to regulate ones impulsions and emotions as well as managing
emotional pressures (e.g., I usually find it difficult to regulate my emotions); (c) emotionality, representing the ability to identify and express feelings and to use these faculties to maintain close relationships with others
(e.g., Im normally able to get into someones shoes and experience their
emotions); and (d) sociability, representing interpersonal skills and functioning to assert oneself as well as to influence others emotions and decisions
(e.g., I find it difficult to bond well even with those close to me). The
TEIQue is comprised of 153 items with 7-point scale (strongly disagree to
strongly agree).
Affectivity. Affectivity was measured by 20 items Positive and Negative
Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988). PANAS is
composed of two 10-item mood scales one to measure positive affectivity
and the other to measure negativity affectivity. The higher scores on both
positive affectivity and negative affectivity items indicate the tendency to
experience a positive and negative mood. The 10 positive affective states
were motivated, excited, feel strong, enthusiastic, proud, alert, inspired,
determined, attentive, and active. The 10 negative affective states were distressed, upset, guilty, scared, hostile, irritable, ashamed, nervous, jittery, and
afraid. Respondents were requested to rate the statement on a 5-point scale
(not at all to extremely) by comparing themselves during the past 2 weeks
with their usual selves.
Psychological distress. Psychological distress was measured by Chans
(2005) 20-item General Health Questionnaire. This scale measures psychological distress in terms of current nonpsychotic symptoms in the five symptom areas represented by scales of health concerns (Felt exhausted) , sleep

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

11

problems ( Early awakening), anxiety (Afraid of everything), dysphoria


(Not enjoying activities), and suicidal ideas (Thoughts of ending life).
Respondents were requested to rate each symptom statement on a 5-point
scale (not at all to extremely) by comparing themselves during the past 2
weeks with their usual selves.
Personality. The 50-item version of the International Personality Item Pool
(IPIP; Goldberg et al., 2006), Big-Five Factor markers, was used to assess
personality. The scale contains 10 items for each of the Big-Five personality
factors: Extraversion (E; I am the life of the party), Agreeableness (A;Take
time out for others), Conscientiousness (C; Pay attention to details), Emotional Stability (ES; Seldom feel blue), and Intellect (I; Spend time reflecting on things). Participants were requested to read the 50 items comprising
the IPIP questionnaire and to mark each one according to how much they
believed it described them on a 5-point scale from very inaccurate to very
accurate.
Self-report emotional intelligence test (SREIT). The 33-item emotional intelligence scale (SREIT; Schutte et al. 1998) is a unidimensional self-report
measure of EI based on Salovey and Mayers (1990) ability model of EI. It
has previously demonstrated good reliability and has been shown to be predictive of various outcomes (Schutte et al. 1998). Respondents indicate their
level of agreement with each of 33 statements on a 7-point scale. Examples
of items are I am aware of the non-verbal messages I send to others, When
I am in a positive mood, I am able to come up with new ideas, and I help
other people feel better when they are down.
Life satisfaction. The Satisfaction with Life Scale (SWLS; Diener, Emmons,
Larsen, & Griffin, 1985) is a subjective self-report measure of life satisfaction. Respondents indicate their level of agreement with each of five statements on a 7-point scale. Examples of items are In most ways my life is
close to my ideal and I am satisfied with my life.
Ability EI. Emotional intelligence ability was measured with the MSCEIT
(Mayer et al., 2002). The MSCEIT is a 141-item test that measures how well
people perform tasks and solve emotional problems on eight tasks that are
divided into four classes or branches of abilities, including (a) perceiving
emotions, (b) facilitating thought, (c) understanding emotions, and (d) managing emotions. For current study, expert scores for the MSCEIT were
requested from the test publisher. Analysis of the data by the test publisher
provides 15 scores, including one for each task, one for each branch, one for
each area, and one for total EI.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

12

Cross-Cultural Research XX(X)

Cognitive intelligence. The 48-item Ravens Standard Progressive Matrices


Test (Raven et al., 2003) was used to measure cognitive intelligence. This test
is designed to measure Spearmans g factor and has now been recognized
as one of the purest measures of g available. The test consists of 48 questions
and presents people with a series of patterns, each of which has one part or
piece missing. The task in each case is to select from a set of eight alternatives the piece that will complete the pattern correctly.

Results
Table 1 presents descriptive statistics for the MSCEIT and other variables for
French and Pakistani participants. According to Mayer et al. (2003) suggestions, to accommodate for item heterogeneity, split-half reliabilities were
employed for the total, area, and branch levels. MSCEIT full-test split-half
reliability was r = 0.84 for the French sample and 0.85 for the Pakistani
sample. The two experiential and strategic area score reliabilities were r =
0.86 and 0.63, and r = 0.88 and 0.82 for the French and Pakistani samples,
respectively. The four branch scores of perceiving, using, understanding, and
managing ranged between r = 0.51 and 0.87 and 0.74 and 0.86 for the French
and Pakistani samples, respectively (see Table 1). The individual task reliabilities ranged from a low of 0.45 to a high of 0.80 for the French sample
and from a low of 0.50 to a high of 0.82 for the Pakistani sample.

Country Differences on the MSCEIT


To obtain an overall picture of possible cross-cultural differences on the
MSCEIT, we conducted a series of independent sample t tests on MSCEIT
branch, area, and total scores. There were indeed several significant crosscultural differences. French participants performed better than their Pakistani
counterparts on perceiving emotions (t = 2.39, p < .05, Cohens d = .35),
using emotions (t = 2.06, p < 0.05, Cohens d = 0.30), understanding emotions (t = 6.24, p < .001, Cohens d = 0.92), managing emotions (t = 5.05,
p < .001, Cohens d = 0.75), experiential EI (t = 2.50, p < 0.05, Cohens d =
0.37), strategic EI (t = 7.15, p < 0.001, Cohens d = 1.06), and total ability EI
(t = 5.38, p < .001, Cohens d = 0.79).

Factorial Invariance
As can be seen in Table 2, each task correlated mostly highly with its sister
subscale with which it combines (e.g., the Faces and Pictures subscales

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Table 1. Means, Standard Deviations, Skewness, and Kurtosis for French and Pakistani Samples
France
M (SD)
Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Age
Gender
RAPM
Extraversion
Agreeableness
Conscientiousness
Emotional stability
Intellect
Positive affect
Negative affect
Psychological distress
Life satisfaction
SREIT
Well-being (TEIQue)
Self-control (TEIQue)
Emotionality (TEIQue)
Sociability (TEIQue)
Total TEIQue
Faces
Facilitation
Changes

30.75 (7.99)
.82
0.44 (0.49)
.23
38.42 (4.80)
.87
3.44 (0.74)
.27
4.06 (0.51)
.62
3.37 (0.72)
.57
3.19 (0.68)
.06
3.69 (0.57)
.47
3.60 (0.55)
.85
2.11 (0.63)
.53
1.83 (0.53)
.81
4.60 (1.22)
.40
5.04 (0.54)
.07
5.22 (0.78) 1.19
4.13 (0.73)
.14
4.72 (0.68)
.16
4.71 (0.65)
.32
4.69 (0.47)
.37
99.18 (14.75)
.20
97.81 (15.51)
.06
89.31 (10.40)
.88

Pakistan
K
0.28
1.19
1.20
0.62
0.65
0.20
20
0.82
1.50
0.08
0.03
0.08
0.05
1.21
0.19
0.74
0.12
0.68
0.14
0.69
1.15

Reliability

.87
.80
.84
.79
.80
.78
.81
.86
.86
.85
.83
.70
.71
.70
.82
.80
.51
.45

M (SD)
27.85 (8.39)
0.63 (0.48)
27.57 (8.06)
3.29 (0.72)
4.06 (0.58)
3.57 (0.58)
2.98 (0.71)
3.46 (0.53)
3.54 (0.57)
1.98 (0.69)
2.04 (0.75)
4.82 (1.26)
5.32 (0.64)
5.05 (0.71)
4.24 (0.61)
4.64 (0.57)
4.97 (0.68)
4.58 (0.48)
95.49 (15.51)
99.65 (16.42)
82.92 (11.85)

S
1.19
0.57
0.93
0.04
0.69
0.07
0.22
0.03
0.35
0.81
1.12
0.45
0.28
0.35
0.26
0.11
0.25
0.44
0.13
0.05
0.07

Reliabilitya

2.15
1.15
0.54
0.16
0.34
0.42
0.30
0.65
0.32
0.66
1.19
0.29
0.60
0.22
0.91
0.01
0.05
0.20
0.52
0.32
0.13

.80
.81
.72
.81
.71
.76
.82
.91
.80
.90
.75
.68
.62
.76
.86
.82
.61
.68

13
(continued)

14
Table 1 (continued)
France
Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

M (SD)

Pakistan
K

Reliabilitya

M (SD)

Reliabilitya

Emotion management
Pictures
Sensation
Blends
Social management
Perceiving emotions
Using emotions
Understanding emotions
Managing emotions
Experiential EI
Strategic EI

87.35 (8.84)
95.50 (11.02)
90.24 (14.39)
90.01 (11.63)
87.34 (10.16)
97.31 (13.48)
91.45 (15.41)
87.57 (10.38)
85.58 (9.35)
93.69 (12.68)
84.79 (8.16)

.72
.53
.87
.10
.10
.33
1.01
.51
.23
.25
.05

0.68
0.95
0.64
0.59
0.34
0.62
1.12
0.11
0.67
0.07
0.54

.53
.75
.50
.46
.48
.87
.56
.60
.51
.86
.63

79.67 (10.16)
92.20 (13.86)
83.50 (13.92)
79.88 (11.54)
81.46 (14.63)
92.51 (13.97)
86.65 (16.64)
77.62 (11.56)
77.41 (13.04)
88.50 (15.46)
74.88 (11.03)

0.57
0.70
0.33
0.07
0.62
0.26
0.18
0.33
0.33
0.25
0.05

0.60
0.86
0.11
0.05
0.72
0.45
0.20
0.20
0.14
0.14
0.02

.62
.85
.66
.50
.63
.86
.80
.74
.80
.88
.82

Total ability EI

86.34 (10.70)

.07

0.04

.84

76.88 (13.62)

0.01

0.40

.85

Note: N = 111, 81, respectively. S = skewness; K = Kurtosis; SREIT = Self-Report Emotional Intelligence Test; RAPM = Ravens advanced progressive
matrices.
a.Split-half reliabilities are reported at the total test, area, and branch score levels due to item heterogeneity. Coefficient alpha reliabilities are reported at the subtest level due to item homogeneity.
*p < .05. **p < .01. ***p < .001.

Karim and Weisz

15

which measure Perceiving emotions).Table 3 presents goodness-of-fit indices for the models examined with CFA (N =192). Based on modification
indices, correlated errors were modeled between sensation and emotion management tasks. As can be seen, the four-factor model indicated satisfactory
levels of fit, 2(13) = 24.33; NFI = 0.92; TLI = 0.91; CFI = 0.96; RMSEA =
0.06 (0.02-0.10). This model was significantly better fitting than the twofactor, 2(5) = 35.15, p < .001 and both three-factor models, 2(3) = 9.13,
p < 0.05 and 2(3) = 25.89, p < 0.05, respectively. In addition, all factor
loadings for this model were positive and significant (range = 0.50 to 0.92).
Correlations among the four-factors (perceiving, using, understanding, and
managing) ranged from 0.52 to 0.75. Thus a four-factor model served as a
base line model for subsequent multisample analyses.
Next, invariance across cultures was tested on four levels of nested models. Each model had more constraints than the previous one (Table 3). First, a
multisample analysis with the unconstrained model (Model 1: configural
invariance) showed an acceptable baseline model for both French and
Pakistani samples. This showed that French participants and Pakistani participants shared the same MSCEIT underlying factor pattern and that corresponding tasks loaded on the same factors. Then, to test the invariance of the
factor loadings (metric invariance) across cultures, factor loadings were constrained to be equal across the two groups (M2). The results revealed that this
constrained model fit the data well. The chi-square difference test between
configural invariant model (M1) and metric invariant model (M2) was not
significant ( 2(4) = 2.84, p > 0.05), suggesting that factor loadings of both
groups were invariant. Next, in addition to the factor loadings, unique variances of each task were constrained to be equal across the groups (M3).
The chi-square difference test between this model and M2 was significant
(2(9) = 17.92, p < 0.05), suggesting that models are not completely invariant once setting equal error variances. Subsequent analyses revealed that
relaxing constraint on error variance of sensation task yielded a substantial
and significant improvement in model fit. The chi-square difference test
between this M3.1 and Model 2 was not significant (2(8) = 14.08, p > .05).
In sum, except for error variance of sensation task, the error variances in this
four-factor model did not vary with culture. Finally, besides the constrained
mentioned, factor covariances were also constrained to be equal across the
two groups (M4). Multisample analysis revealed that this constrained model
was acceptable. However, the chi-square difference test between this M4 and
M3.1 was significant (2(6) = 13.34, p < .05), suggesting that models are not
completely invariance once constraining the covariances across cultures.
Subsequent analyses revealed that, relaxing constraint on covariance between

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

16

Table 2. Intercorrelations Among MSCEIT Scales for French and Pakistani Samples
1

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

1. Faces Task
2. Facilitation Task
3. Changes Task
4. Emotion
Management
Task
5. Pictures Task
6. Sensation Task
7. Blends Task

10

11

12

13

14

15

.27**

.37**
.39***

.19
.10
.41***

.25*
.37**
.30**
.16

.44***
.45***
.50***
.39***

.19
.40***
.45***
.30**

.30**
.30**
.50***
.56***

.87***
.38**
.40***
.23*

.45***
.72***
.52***
.33**

.34**
.47***
.84***
.42***

.30**
.27*
.52***
.81***

.82***
.59***
.52***
.32**

.35**
.45***
.78***
.67***

.65***
.57***
.75***
.57***

.21

.02
.40***

.41***
.35**
.30**

.64***
.44***
.15

.26*
.92***
.45***

.19
.54***
.85***

.36**
.41***
.35**

.56***
.72***
.32**

.32**
.56***
.73***

.49***
.72***
.58***

.42***

.38***

.47***

.92***

.46***

.77***

.71***

.47***

.33**

.39***

.91***

.40***

.73***

.59***

.42***
.51***

.78***
.50***

.60***
.89***

.77***
.78***

.47***

.83***

.75***

.55***

.87***
.88***

.23*
.15
.01

.16
.22*

.08

.45***
.01
.09

.25**
.23*
.17

.22*
.33**
.35***

.02
.23*
.07

.20
.04

.04

.22*

.24*

.13

.40***

.36***

.21*

.20*

.89***

.29**

.22*

.03

.78***

.09

.10

.35***

.11
.15

.66***
.20*

.30**
.80***

.29**
.09

.29**
.17

.85***
.22*

.09
.83***

.27**
.21*

.21*
.20*

.23*

8. Social
Management
Task
9. Perceiving
Emotion
10. Using Emotion
11. Understanding
Emotion
12. Managing
Emotion
13. Experiential EI
14. Strategic EI

.15

.28**

.13

.75***

.25**

.25**

.18

.89***

.24*

.33**

.19*

.75***
.20*

.53***
.29**

.32**
.64***

.18
.45***

.72***
.26**

.50***
.30**

.11
.71***

.38***
.65***

.86***
.28**

.64***
.34**

.26**
.82***

.34**
.68***

.37***

15. Total EI

.60***

.51***

.56***

.36***

.61***

.48***

.47***

.60***

.72***

.60***

.63***

.60***

.85***

Note: N = 111, 81, respectively. Correlations for the French sample are below the diagonal and for the Pakistani sample are above the diagonal.
*p < .05. **p < .01. ***p < .001.

.80***

17

Karim and Weisz


Table 3. CFA and Multisample Goodness-of-Fit Indices for the MSCEIT Across
Cultures
2

Model

df

NFI

TLI

CFI

RMSEA
[95%C.I]

Confirmatory factor analyses (N = 192)


One-factor
69.39
19
.00
.78
.75
.83
.11 [.08, .14]
Two-factor
59.88
18
.00
.81
.78
.78
.11 [.08, .14]
Three-factora
33.46
16
.00
.89
.89
.94
.07 [.03, .11]
Three-factorb
50.22
16
.00
.84
.80
.88
.10 [.07, .14]
Four-factor
24.33
13
.028
.92
.91
.96
.06 [.02, .10]
Multigroup comparison factor analyses (French sample: N = 111; Pakistani sample: N = 81)
M1
Configural
40.45
26
.035
.94
.05 [.01, .08]
invariance
M2
Metric
43.29
30
.055
.95
.04 [.00-.07]
invariance
2M2 vs. M1
2.84
4
.58
M3
Invariant
61.21
39
.01
.91
.05 [.02-.08]
uniqueness
2 M3 vs. M2
17.92
9
.03
M3.1
sensation, free
57.35
38
.02
.93
.05 [.02-.07]
2 M3.1 vs. M2
14.06
8
.08
M4
Invariant
70.69
44
.007
.90
.05 [.03-.08]
factor
covariances
2 M4 vs. M3.1
13.34
6
.03
M4.1
Covariance
64.44
43
.01
.92
.05 [.01-.08]
between
understand
and managing
emotions set
free
2 M4.1 vs. M3.1

7.09

.21

Note: For both groups correlated errors were modeled between sensation and emotion management tasks. NFI = normed fit index; TLI = Tucker-Lewis index; CFI = comparative fit index;
RMSEA = root mean square error of approximation. NFI, CFI, and TLI > .90 and RMSEA < .08
are considered acceptable (Bentler & Bonett, 1980; Browne & Cudeck, 1993).
a.Three-factor oblique model comprising the perceiving and using allowed to load on a single
factor (Fan et al., 2010; Roberts et al., 2006; Rode et al., 2008; Rossen et al., 2008).
b.Three-factor oblique model comprising the using and managing allowed to load on a single
factor (Palmer et al., 2005).

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

18

Cross-Cultural Research XX(X)

Figure 1. The parameter estimates of complete invariance model (Model 4.1)


Note:Values in parenthesis represent standardized estimates for the Pakistani sample.

understanding and managing branches yielded a substantial and significant


improvement in model fit (M4.1). The chi-square difference test between
M4.1 and Model 3.1 was not significant (2(5) = 7.09, p > .05).Therefore,
the hypothesis of partially invariant covariances between cultures was tenable. In sum, multisample CFA analyses revealed that, with few exceptions,
the factor loadings, unique variances, and factor covariances were invariant
across cultures. The parameter estimates of complete invariant model (M4.1)
are presented in Figure 1.

Discriminant Validity
MSCEITs discriminant validity was examined by assessing the correlations
between scores obtained on the MSCEIT and scores obtained for the measures assessing self-report mixed model measure (TEIQue), self-report ability EI measure (SREIT), Ravens Advanced Progressive Matrices, and
personality. According to Papadogiannis et al. (2009), those that intercorrelate between r =0.00 and 0.25 are considered unrelated to minimally related

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Table 4. Zero-Order Correlations Between the MSCEIT and Other Variables in the Study
France
Branch scores

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Age
Gender
RAPM
E
A
C
ES
I
PA
NA
PD
SWL
SREIT
TEIQue
Well-being
Self-control
Emotionality
Sociability
Global trait EI

Pakistan
Area scores

Branch scores

Per

Use

Und

Man

Exp

Stg

Total

.03
.03
.06
.01
.03
.10
.29**
.17
.01
.25**
.19*
.17
.08

.03
.09
.00
.01
.16
.13
.08
.11
.08
.10
.10
.14
.11

.15
.11
.11
.11
.10
.00
.09
.04
.17
.08
.07
.06
.05

.06
.19*
.06
.14
.13
.08
.14
.04
.20*
.15
.12
.22*
.02

.01
.08
.04
.00
.10
.12
.30**
.08
.03
.24**
.20*
.23*
.01

.11
.20*
.10
.00
.14
.05
.01
.07
.02
.12
.12
.19*
-.03

.06
.18
.03
.00
.14
.10
.20*
.07
.01
.23*
.21*
.25**
.02

.00
.04
.12
.10
.01

.29**
.15
.00
.02
.19*

.17
.17
.00
.04
.15

.23*
.05
.13
.08
.17

.26** .17
.16
.03
.07
.08
.05
.07
.20*
.08

.25**
.11
.09
.01
.17

Per

Use

Und

.26*
.12
.11
.31** .25* .12
.22*
.27*
.43**
.04
.01
.01
.24*
.34** .28*
.11
.08
.08
.01
.01
.19
.14
.40** .31**
.02
.07
.17
.15
.05
.21
.10
.03
.26*
.09
.03
.02
.09
.07
.18
.13
.04
.17
.17
.18

.25*
.04
.15
.21
.22*

.40**
.16
.28*
.16
.34**

Area scores
Man

Exp

.18
.01
.29**
.11
.25*
.06
.17
.31**
.06
.35**
.36**
.01
.16

.22*
.15
.22*
.33** .09
.23*
.26*
.41** .36**
.03
.05
.01
.33** .30** .35**
.11
.09
.11
.01
.20
.11
.28** .37** .38**
.02
.15
.11
.13
.29** .26*
.09
.32** .24*
.08
.015
.07
.09
.20
.15

.28**
.23*
.38**
.34**
.40**

.21
.04
.19
.21
.22*

Stg

.38**
.22**
.36**
.28
.41**

Total

.33**
.16
.32**
.27*
.36**

19

Note. Per = perceiving emotions; Use = using emotions; Und = understanding emotions; Man = managing emotions; Exp = experiential EI; Stg = strategic EI;
RAPM = Ravens advanced progressive matrices; E = extraversion; A = agreeableness; C = conscientiousness; ES = emotional stability; I = intellect; PA = positive affect; NA = negative affect; PD = psychological distress. SWL = satisfaction with life; TEIQue = Trait emotional intelligence questionnaire.
*p < .05. **p < .01.

20

Cross-Cultural Research XX(X)

with one another; r = 0.25 to 0.50 indicate minimal to moderate overlap; r =


0.50 to 0.75 indicate moderate to highly related concepts; and r = 0.75 to
1.00 indicate that instruments share common themes and arguably assess the
same underlying constructs.
MSCEIT and TEIQue. For the participants in the French sample, nonsignificant to moderate correlations were observed between the MSCEIT branches
and the TEIQue factors, with the greatest correlation found between
MSCEITs managing emotions branch and TEIQues well-being factor (r =
0.29, p < 0.01). Likewise, for the participants in the Pakistani sample, the
greatest of correlation was observed between understanding emotions branch
of the MSCEIT with the well-being factor of the TEIQue (r = 0.40, p < 0.01).
However, the number of low to moderate significant correlations in the Pakistani sample (13) was higher than that for the French sample (i.e., only 4). In
sum, the performance-based ability EI in both cultures was found to be independent of the TEIQue. (see Table 4).
MSCEIT and SREIT. Nonsignificant correlations were observed between
the MSCEIT factors and the SREIT in both cultures.
MSCEIT and Cognitive Intelligence. For participants in the French sample, all
correlations between the MSCEIT scores and cognitive intelligence revealed
to be nonsignificant. However, for the Pakistani sample, low to moderate
significant correlations were observed between the cognitive intelligence and
the MSCEIT scores, with the greatest correlation observed between Strategic
EI and Ravens Advances Progressive Matrices (r = 0.41, p < .01).
MSCEIT and the Big Five. For participants in the French sample, among the
Big Five personality dimensions, only emotional stability revealed to be a
significant correlate of perceiving emotions (r = 0.29, p < 0.01), experiential
EI (r = 0.30, p < 0.01), and total ability EI (r = 0.20, p < 0.05). In contrast, for
participants in the Pakistani sample, low to moderate correlations were
observed between the Big Five dimensions of agreeableness and intellect and
the MSCEIT scores. Correlations between agreeableness and MSCEIT scores
ranged from 0.24 (perceiving emotions) to 0.35 (total ability EI), whereas
correlations between intellect and the MSCEIT scores ranged from 0.14 (perceiving emotions) to 0.38 (total ability EI).

Incremental Validity
We used the hierarchical multiple regression technique to test for the incremental validity of the MSCEIT. The Big Five personality dimensions and the

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

21

Karim and Weisz


Table 5. Predicting Satisfaction With Life, PA, NA, and Psychological Distress:
Results for French Sample
Life
satisfaction

Step
1

R2

PA

E
A
C
ES
I
RAPM
Perceiving
Using
Understanding

.11
.21*** .34
.14
.06
.19*
.42
.25**
.11
.07
.29
.25**
.01
.10
.02 .01
.04
.04
.02
.11

Managing
Total R2

.08
.24

.12
.47

Final F (df)

3.01**
(10, 94)

9.01***
(10, 99)

Psychological
distress

NA
R2

R2

R2

.45*** .09
.40*** .14
.30***
.04
.14
.10
.03
.59***
.47***
.03
.02
.05
.13
.02 .06
.03 .11
.03
.03
.09
.13
.10
.00
.43
7.56***
(10, 99)

.06
.33
5.08***
(10, 99)

Note: RAPM = Ravens advanced progressive matrices; E = extraversion; A = agreeableness;


C = conscientiousness; ES = emotional stability; I = intellect; PA = positive affect; NA =
negative affect.
*p < 0.05. **p < 0.01. ***p < 0.001.

scores on the Ravens Advanced Progressive Matrices were entered into the
equation first (Step 1). At Step 2, the scores for the MSCEIT four branches
were entered. As can be seen in Tables 5 and 6, none of the MSCEIT
branches was found to be a significant predictor of satisfaction with life, PA,
NA, and psychological distress.

Known Group validation


For the French sample, females scored higher than males on managing emotions, strategic EI, and total MSCEIT, whereas for the Pakistani sample,
females scored higher than males on perceiving emotions, using emotions,
experiential EI, and total MSCEIT (see Table 7).

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

22

Cross-Cultural Research XX(X)

Table 6. Predicting Satisfaction With Life, PA, NA, and Psychological Distress:
Results for Pakistani Sample
Life
satisfaction

Step
1

E
A
C
ES
I
RAPM
Perceiving
Using
Understanding
Managing
Total R2
Final F (df)

.06
.02
.20
.12
.34
.25
.12
.04
.04
.09
.20
1.73
(10, 66)

R2
.19*

.01

PA

Psychological
distress

NA
R2

R2

R2

.08
.28** .04
.45*** .01
.31***
.02
.23
.17
.44**
.15
.07
.01
.53***
.46***
.29*
.10
.05
.05
.01
.06
.03
.04 .04
.05
.04
.07
.20
.18
.14
.26
.04
.09
.03
.23
.29
.22
.50
.38
3.16**
(10, 66)

6.70***
(10, 66)

4.20***
(10, 66)

Note: RAPM = Ravens advanced progressive matrices; E = extraversion; A = agreeableness;


C = conscientiousness; ES = emotional stability; I = intellect; PA = positive affect; NA =
negative affect. = standardized regression weights.
*p < .05. **p < .01. ***p < .001.

Discussion
Cross-cultural validity of EI scales is a constant concern in organizational
behavior research (Gangopadhyay & Mandal, 2008; Palmer et al., 2008). The
present study attempted to evaluate an ability-based measure of EI (the
MSCEIT) cross-culturally. To our knowledge, this study is the first to provide evidence of the factorial invariance, discriminant and incremental validity of the MSCEIT across cultures.
The first objective of this study was to assess the mean differences on the
MSCEIT across two cultures. French participants had higher scores than their
Pakistani counterparts on branch, area, and total MSCEIT scores. This
accords well with findings showing that people from individualistic societies
are better at perceiving, understanding, expressing, and regulating emotions
(e.g., Fernandez et al., 2000; Gross & John, 2003; Matsumoto, 1989; 1992).
This finding suggests that care must be taken when selecting people from

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Table 7. Mean Differences Across Genders Within Each Sample


France
Females

Pakistan
Males

Females

Males

Perceiving
Using
Understanding
Managing
Experiential EI
Strategic EI

97.73 (13.25)
92.80 (14.24)
88.66 (11.05)
87.16 (9.62)
94.70 (12.53)
86.29 (8.41)

96.78 (13.89)
89.74 (16.74)
86.19 (9.40)
83.57 (8.67)
92.42 (12.90)
82.89 (7.49)

0.36
1.03
1.24
2.04*
0.93
2.20*

.07
.20
.24
.39
.18
.42

98.31 (12.34)
92.34 (16.89)
79.55 (10.80)
77.72 (13.29)
95.41 (13.98)
76.20 (10.57)

89.28 (13.89)
83.48 (15.78)
76.55 (11.93)
77.25 (13.03)
84.65 (15.20)
74.15 (11.31)

2.91**
2.36*
1.11
0.15
3.16**
0.80

.68
.55
.26
.04
.73
.19

Total EI

89.05 (10.57)

83.17 (10.58)

1.99*

.56

81.13 (13.40)

74.51 (13.29)

2.14*

.50

Note: For the French sample, females = 62 and males = 49 and for the Pakistani sample, females = 29 and males = 52. d = Cohens d.
*p < .05. **p < .01. ***p < .001.; Values in parentheses represent standard deviations

23

24

Cross-Cultural Research XX(X)

pool of individuals from diverse cultures because results may be biased


toward individuals from individualistic societies.
The second objective of this study was to investigate whether the MSCEIT
measures the same construct in both cultures. Multigroup CFA analyses
revealed that the MSCEIT has the same theoretical latent structure, the same
strength of the relationships among factors and tasks, and the same reliability
of tasks regardless of the country. Therefore, the MSCEIT across both cultures can be interpreted in the same way. Significant mean differences and
multigroup CFA analyses across both cultures provide support for the assertion made by Palmer et al. (2008) that, EI factors are culturally universal and
have comparable functions across cultures. However, the processes underlying these factors and their manifestation may differ across cultures as a consequence of the role culture plays in the development, display, and
interpretation of emotions (p. 35). It is worth mentioning that the similarities
in the factor structure across both cultures can be attributed to the shift in
cultural values of Pakistan. Pakistani youth (present study sample-university
students) are much different from previous generations. They have been
raised differently. This youth is the best educated and most culturally diverse
generation. A combination of Western-style of education (mostly American
and U.K.-based curriculum), widespread use of Internet, and currently vibrant
print and electronic media in the country has made this generation exceedingly tolerant and open-minded toward western lifestyles.
The third objective of this study was to assess the discriminant validity of
the MSCEIT vis--vis cognitive intelligence (the Ravens Advanced
Matrices), self-report EI measures (the SREIT and TEIQue), and the Big Five
personality measures. Consistent with past research relating self-report EI
with ability EI measures (e.g., Brackett & Mayer, 2003; Livingstone, & Day,
2005; Joseph & Newman, 2010; OConnor & Little, 2003), in both cultures,
the MSCEIT demonstrated a lack of convergence with the TEIQue and the
SREIT. This important finding suggests that the performance-based ability EI
measure (i.e., MSCEIT) and self-report measures are assessing different constructs. These findings support Petrides and Furnhams (2003) assertion that
the tendency to validate ability EI measure against another self-report measure is problematic given the obvious differences between measurement
methods. Low to moderate correlations were found between the MSCEIT
factors and cognitive intelligence in the Pakistani sample, whereas nonsignificant correlations were observed in the French sample. These findings support Mayer, Salovey, and Carusos (2004) assertion that EI is different from
other intelligences (p. 203). Finally, as predicted, this study revealed that
MSCEIT was mostly distinguishable from the Big Five personality

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

25

dimensions in both cultures. The correlations of the MSCEIT branches with


Big Five were mostly nonsignificant or low to moderate in both cultures.
These results were consistent with previous findings (Brackett & Mayer,
2003; OConnor, and Little, 2003; Rode et al., 2007) and provide support for
the assertion that the MSCEIT includes abilities to perceive, integrate, understand, and regulate emotions (Mayer & Salovey, 1997); therefore, it is
unlikely to be related to personality traits (Mayer, Roberts et al.,2008; Mayer,
Salovey et al., 2008).
The fourth objective of this study was to assess the incremental validity of
the MSCEIT. In line with previous studies (Livingstone & Day, 2005; Rode
et al., 2007; Rossen & Kranzler, 2009; Zeidner & Olnick-Shemesh, 2010),
the results indicated that the MSCEIT scales did not add to the prediction of
PA, NA, satisfaction with life, and psychological distress after controlling for
personality and cognitive intelligence in either of the sample. The lack of
incremental validity when explaining NA, PA, satisfaction with life, and psychological distress suggests that the MSCEIT may not increase our understanding of subjective well-being, after controlling for cognitive intelligence
and personality variables in the same analyses. Thus the results of this study
further highlight the serious problem associated with the predictive validity
of the MSCEIT (for further review please see Zeidner & Olnick-Shemesh,
2010) .
Finally, the fifth objective of this study was to obtain an overall picture of
possible gender differences on the MSCEIT within each sample. In the
French sample, women outperformed men on managing and total EI, whereas
in the Pakistani sample, women scored higher on perceiving, using, and total
EI. These findings support past research showing that women tend to be better at emotion-related abilities than men (Barrett et al., 2000; Ciarrochi et al.,
2005; Hall & Matsumoto, 2004; Garnefski et al., 2004; Gross & John, 1998;
Mayer et al., 2002; Joseph & Newman, 2010; Mehrabian, Young, & Sato,
1988; Palmer et al., 2005). This finding suggests gender bias in the MSCEIT;
therefore, according to some researchers, the MSCEIT should not be used for
personnel selection, unless research demonstrates that gender differences in
test performance reflect gender differences in job performance (Day &
Carroll, 2004).

Limitations and Future Directions


First, there is a possibility that individualism/collectivism along with other
cultural factors will vary within cultures because of demographic, regional,
class, and other differences within cultures. This leads to a limitation found

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

26

Cross-Cultural Research XX(X)

in much cross-cultural researchthat of generalizing across all cultural


groups or subcultures. The sample at a single university may not reflect the
culture of a heterogeneous nation. Therefore, results collected in big cities
could likely be different from those collected in small cities or villages. This,
of course, points to the need for future research to examine samples across
various subcultures.
Second, and most important, student samples do not represent the culture
as much as do more or less representative samples. Students may experience
different levels of EI, from a general working adult population (Day, Therrien,
& Carroll, 2005). The use of students limits the external validity of the results
(Wintre, North, & Sugar, 2001). Students are more internationally similar
than unselected members of a culture. This may have fostered similarities in
the results across both cultures. Therefore, it is important to examine the
validity of the MSCEIT using a larger sample that is more representative of
the general population.
Third, all participants responded to the MSCEIT in English. However, as
noted by one anonymous reviewer, even for the students taking courses in
English (or fully conversant with English as a second language), the vocabulary associated with emotion concepts may be somewhat obscure and/or such
students do not always possess the necessary emotion terminology. This may
have influenced the pattern of responses in the current study. For example,
participants in both cultures scored low on understanding and managing,
which are language-intensive. This, of course, points to the need for future
research to examine samples across native languages.
Finally, researchers have consistently expressed concerns about the
absence of scientific standards for determining the accuracy of consensus and
expert scores for the MSCEIT (for details see Matthews et al., 2002).
Moreover, American norm group scoring may not work well in other cultures
(Zeidner & Olnick-Shemesh, 2010). Therefore, future cross-cultural research,
instead of using American based scoring, should use proportion consensusscores with consensus weights determined from the local samples.
References
American Psychological Association .(2002). Ethical principles of psychologists and
codes of conduct. Washington, DC: APA.
Barrett, L. F., Lane, R. D., Sechrest, L., & Schwartz, G. E. (2000). Sex differences in
emotional awareness. Personality and Social Psychology Bulletin, 26, 1027-1035.
Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness-of-fit in the
analysis of covariance structures. Psychological Bulletin, 88, 588-606.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

27

Brackett, M. A., & Mayer, J. D. (2003). Convergent, discriminant, and incremental


validity of competing measures of emotional intelligence. Personality and Social
Psychology Bulletin, 29, 1147-1158.
Brackett, M. A., Rivers, S. E., Shiffman, S., Lerner, N., & Salovey, P. (2006). Relating
emotional intelligence abilities to social functioning: A comparison of self-report
and performance measures of emotional intelligence. Journal of Personality and
Social Psychology, 91 (4), 780-795.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K.
A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162).
Newbury Park, CA: Sage Publications.
Chan, D. W. (2005). Emotional Intelligence, social coping, and psychological distress among Chinese gifted students in Hong Kong. High Ability Studies, 16(2),
163-178.
Ciarrochi, J., Haynes, K., & Crittenden, N. (2005). Can men do better if they try
harder? Sex and motivational effects on emotional awareness. Cognition and
Emotion, 19, 133-141.
Ciarrochi, J. V., Chan, A. Y., & Caputi, P. (2000). A critical evaluation of the emotional intelligence construct. Personality and Individual Differences, 28, 539-561.
Day, A. L., & Carroll, S. A. (2004). Using an ability-based measure of emotional intelligence to predict individual performance, group performance, and group citizenship behaviors. Personality and Individual differences, 36, 1443-1458.
Day, A. L., Therrien, D. L., & Carroll, S. A. (2005). Predicting psychological health:
Assessing the incremental validity of emotional intelligence beyond personality,
Type A behaviour, and daily hassles. European Journal of Personality, 19,
519-536.
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction with
Life Scale. Journal of Personality Assessment, 49, 71-75.
Ekermans, G. (2009). Emotional intelligence across cultures: Theoretical and methodological considerations. In, C. Stough, D. H. Saklofske, & J. D. A. Parker (Eds),
Assessing emotional intelligence: Theory, research, and applications (pp. 259290). New York: Springer.
Fabio, A. Di., & Palazzeschi, L. (2009). An in-depth look at scholastic success: Fluid
intelligence, personality traits or emotional intelligence? Personality and Individual Differences, 46, 581-585.
Fan, H., Jackson, T., Yang, X., Tang, W., & Zhang, J. (2010). The factor structure
of the Mayer-Salvoey-Caruso Emotional Intelligence Test V 2.0 (MSCEIT): A
meta-analytic structural equation modeling approach. Personality and Individual
Differences, 48, 781-785.
Farrelly, D., & Austin, E. J. (2007). Ability EI as an intelligence? Associations of
the MSCEIT with performance on emotion processing and social tasks and with
cognitive ability. Cognition and Emotion, 21 (5), 1043-1063.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

28

Cross-Cultural Research XX(X)

Fernandez, I., Carrera, P., Sanchez, F., Paez, D., & Candia, L. (2000). Differences
between cultures in emotional verbal and non-verbal reactions. Psicothema, 12,
83-92.
Gangopadhyay, M., & Mandal, M. K. (2008). Emotional Intelligence-A universal or
culture-specific construct? In R. J. Emmerling, V. K. Shanwal, & M. K. Mandal
(Eds.), Emotional intelligence: Theoretical and cultural perspective (pp. 115134). New York: Nova Science Publishers, Inc.
Garnefski, N., Teerds, J., Kraaij, V., Legerstee, J., & Van den Kommer, T. (2004).
Cognitive emotion regulation strategies and depressive symptoms: Differences
between males and females. Personality and Individual differences, 36, 267-276.
Gignac, G. E. (2009). Psychometric and the measurement of emotional intelligence.
In, C. Stough, D. H. Saklofske, & J. D. A. Parker (Eds), Assessing emotional
intelligence: Theory, research, and applications (pp. 9-40). New York: Springer.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger,
C. R., & Gough, H. C. (2006). The International Personality Item Pool and the
future of public-domain personality measures. Journal of Research in Personality,
40, 84-96.
Gross, J. J., & John, O. P. (2003). Individual differences in two emotion regulation
processes: Implications for affect, relationships, and wellbeing. Journal of Personality and Social Psychology, 85, 348-362.
Gross, J. J., & John, O. P. (1998). Mapping the domain of expressivity: Multimethod
evidence for a hierarchical model. Journal of Personality and Social Psychology,
74, 170-191.
Hall, J. A. & Matsumoto, D. (2004). Gender differences in judgments of multiple
emotions from facial expressions. Emotion, 4 (2), 201-206.
Hofstede, G. (1980). Cultures consequences: International differences in workrelated values. Newbury Park, CA: Sage.
Joseph, D. L., & Newman, D. A. (2010). Emotional intelligence: An integrative metaanalysis and cascading model. Journal of Applied Psychology, 95 (1), 54-78.
Livingstone, H. A., & Day, A. L. (2005). Comparing the construct and criterionrelated validity of ability-based and mixed-model measures of emotional intelligence. Educational and Psychological Measurement, 65(5), 757-779).
Matsumoto, D. (1989). Cultural influences on the perception of emotion. Journal of
Cross-Cultural Psychology, 20, 92-105.
Matsumoto, D. (1992). American-Japanese cultural differences in the recognition of
universal facial expressions. Journal of Cross-Cultural Psychology, 23 (1), 72-84.
Matthews, G., Emo, A. K., Funke, G., Zeidner, M., Roberts, R. D., & Costa, P. T.
(2006). Emotional intelligence, personality, and task-induced stress. Journal of
Experimental Psychology, 12 (2), 96-107.
Matthews, G., Roberts, R. D., & Zeidner, M. (2004). Seven myths about emotional
intelligence. Psychological Inquiry, 15, 179-196.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

29

Matthews, G., Zeidner, M., & Roberts, R. D. (2002). Emotional intelligence: Science
and myth. Cambridge, MA: MIT Press.
Mayer, J. D., Roberts, R. D., & Barsade, S. G. (2008). Human Abilities: Emotional
Intelligence. Annual Review of Psychology, 59, 507-536.
Mayer, J. D. & Salovey, P. (1997). What is emotional intelligence? In P. Salovey &
D. J. Sluyter (Eds.), Emotional Development and Emotional Intelligence: Educational Implications. New York: Basic Books, pp. 3-27.
Mayer, J. D., Salovey, P., & Caruso, D. R. (2002). Mayer-Salovey-Caruso Emotional
Intelligence Test (MSCEIT) item booklet, version 2.0. Toronto, Onatario, Canada:
MHS.
Mayer, J. D., Salovey, P., & Caruso, D. R. (2004). Emotional intelligence: Theory,
findings, and implications. Psychological Inquiry, 15, 197-215.
Mayer, J. D., Salovey, P., & Caruso, D. R. (2008). Emotional intelligence: New ability
or eclectic traits? American Psychologist, 63, 503-517.
Mayer, J. D., Salovey, P., Caruso, D., & Sitarenios, G. (2003). Measuring emotional
intelligence with the MSCEIT V2.0. Emotion, 3, 97-105.
Mehrabian, A., Young, A. L., & Sato, S. (1988). Emotional empathy and associated
individual differences. Current Psychology Research and Reviews, 7, 221-240.
OConnor, R. M., & Little, I. S. (2003). Revisiting the predictive validity of emotional
intelligence: Self-report versus ability-based measures. Personality and Individual Differences, 35, 1893-1902.
Palmer, B. R., Gignac, G., Manocha, R., & Stough, C. (2005). A psychometric evaluation of the MayerSaloveyCaruso Emotional Intelligence Test Version 2.0. Intelligence, 33, 285-305.
Palmer, B. R., Gignac, G., Ekermans, G., & Stough, C. (2008). A comprehensive
framework for emotional intelligence. In R. J. Emmerling, V. K. Shanwal, & M.
K. Mandal (Eds.), Emotional Intelligence: Theoretical and Cultural Perspective
(pp. 17-38), New York: Nova Science.
Papadogiannis, P. K., Logan, D., & Sitarenios, G. (2009). An ability model of emotional intelligence: A rationale, description, and application of the Mayer Salovey
Caruso Emotional Intelligence Test (MSCEIT). In, C. Stough, D. H. Saklofske,
& J. D. A. Parker (Eds), Assessing emotional intelligence: Theory, research, and
applications (pp. 259-290). New York: Springer.
Petrides, K. V., & Furnham, A. (2003). Trait emotional intelligence: Behavioral validation in two studies of emotion recognition and reactivity to mood induction.
European Journal of Personality, 17, 39-57.
Petrides, K. V., Prez-Gonzalez, J. C., & Furnham, A. (2007). On the criterion and
incremental validity of trait emotional intelligence. Cognition and Emotion, 21,
26-55.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

30

Cross-Cultural Research XX(X)

Raven, J., Raven, J.C., & Court, J.H. (2003). Manual for Ravens Progressive Matrices and Vocabulary Scales. Section 1: General Overview. San Antonio, TX: Harcourt Assessment.
Roberts, R. D., Schulze, R., OBrien, K., MacCann, C., Reid, J., & Maul, A. (2006).
Exploring the validity of the MayerSaloveyCaruso Emotional Intelligence Test
(MSCEIT) with established emotions measures. Emotion, 6, 663-669.
Rode, J. C., Mooney, C. H., Arthaud-day, M. L., Near, J. P., Rubin, R. S., Baldwin,
T. T., & Bommer, W. H. (2007). An examination of the structural, discriminant,
nomological, and incremental predictive validity of the MSCEIT V2.0. Intelligence, 36(4), 350-366.
Rossen, E., & Kranzler, J. H. (2009). Incremental validity of Mayer-Salovey-Caruso
Emotional Intelligence Test version 2.0 (MSCEIT) after controlling for personality and intelligence. Journal of Research in Personality, 43, 60-65.
Rossen, E., Kranzler, J. H., & Algina, J. (2008). Confirmatory factor analysis of the
Mayer-Salovey-Caruso Emotional Intelligence Test V 2.0 (MSCEIT). Personality
and Individual Differences, 44, 1258-1269.
Salovey, P., Bedell, R. T., Detweiler, J., & Mayer, J. (2000). Current directions in emotional intelligence research. In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of emotions (pp. 504-520). New York: Guilford.
Salovey, P. & Mayer, J. D. (1990) Emotional intelligence. Imagination, Cognition and
Personality, 9,185-211.
Schutte, N. S., Malouff, J. M., Hall, L. E., Haggerty, D. J., Cooper, J. T., Golden, C. J.,
et al. (1998). Development and validation of a measure of emotional intelligence.
Personality and Individual Differences, 25, 167-177.
Schutte, N.S., Malouff, J.M., Thorsteinsson, E.B., Bhullar, N., & Rooke, S.E. (2007).
A meta-analytic investigation of the relationship between emotional intelligence
and health. Personality and Individual Differences, 42, 921-933.
Tugade, M. M., & Fredrickson, B. L. (2002). Positive emotions and emotional intelligence. In L. Feldman Barrett & P. Salovey (Eds.), The wisdom of feelings (pp.
319-340). New York: Guilford.
Watson, D., Clark, L.A., & Tellegen, A. (1988). Development and validation of brief
measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063-1070.
Wintre, M. G., North, C., & Sugar, L. A. (2001). Psychologists response to criticisms
about research based on undergradute participants: A developmental perspective.
Canadian Psychology, 42(3), 216-225.
Zeidner, M., & Olnick-Shemesh, D. (2010). Emotional intelligence and subjective
well-being revisited. Personality and Individual Differences, 48, 431-435.

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010

Karim and Weisz

31

Bios
Jahanvash Karim is a PhD student studying emotional intelligence at the
IAE dAix-en-Provence, Universit de Paul Czane, Aix-en-Provence, France. His
research includes cross-cultural analyses of emotional intelligence measures.
Robert Weisz is full professeur, specialist in organizational behavior & organizational development, at the IAE dAix-en-Provence, Universit de Paul Czane, Aixen-Provence, France. He is adjunct professor at the HEC (Paris) for the international
programs. He also teaches at other well-known business schools such as Monash BS
(Melbourne), WHU (Koblenz), and Steinbeis (Berlin).

Downloaded from ccr.sagepub.com by Jahanvash Karim on September 13, 2010