International Journal of Testing

ISSN: 1530-5058 (Print) 1532-7574 (Online) Journal homepage: http://www.tandfonline.com/loi/hijt20

Introducing Emotioncy as a Potential Source of

Test Bias: A Mixed Rasch Modeling Study

Reza Pishghadam, Purya Baghaei & Zahra Seyednozadi

To cite this article: Reza Pishghadam, Purya Baghaei & Zahra Seyednozadi (2017) Introducing
Emotioncy as a Potential Source of Test Bias: A Mixed Rasch Modeling Study, International Journal
of Testing, 17:2, 127-140, DOI: 10.1080/15305058.2016.1183208

To link to this article: http://dx.doi.org/10.1080/15305058.2016.1183208

Published online: 23 May 2016.

Submit your article to this journal

Article views: 68

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=hijt20

Download by: [151.238.209.154] Date: 24 May 2017, At: 09:35

International Journal of Testing, 17: 127–140, 2017
Copyright C International Test Commission
ISSN: 1530-5058 print / 1532-7574 online
DOI: 10.1080/15305058.2016.1183208

Introducing Emotioncy as a Potential

Source of Test Bias: A Mixed Rasch
Modeling Study
Reza Pishghadam
Language Education, Ferdowsi University of Mashhad, Mashhad, Iran

Purya Baghaei
Language Education, Islamic Azad University Mashhad Branch, Mashhad,
Iran

Zahra Seyednozadi
Language Education, Ferdowsi University of Mashhad, Mashhad, Iran

This article attempts to present emotioncy as a potential source of test bias to inform
the analysis of test item performance. Emotioncy is defined as a hierarchy, ranging
from exvolvement (auditory, visual, and kinesthetic) to involvement (inner and arch),
to emphasize the emotions evoked by the senses. This study hypothesizes that when
individuals have high levels of emotioncy for specific words, their test performance
may systematically change, resulting in test bias. To this end, 355 individuals were
asked to take a 40-item vocabulary test along with the emotioncy scale. Mixed Rasch
model was employed to flag differential item functioning items. Results illustrated
that the test takers with high emotioncy toward specific words outperformed the ones
in the low-emotioncy group, characterizing emotioncy as a potential source of test
bias.

Keywords: DIF, emotioncy, test bias, mixed Rasch model, exvolvement

INTRODUCTION

Since assessments are significant in gaining information about test takers’ knowl-
edge and progress, any extraneous factor affecting assessment is expected to be

Correspondence should be sent to Reza Pishghadam, Language Education, Ferdowsi University

of Mashhad, Mashhad, Iran. E-mail: pishghadam@um.ac.ir
Color versions of one or more of the figures in the article can be found online at
www.tandfonline.com/hijt.
128 PISHGHADAM, BAGHAEI, AND SEYEDNOZADI

identified and eliminated to have a valid test. One of the factors that may jeopar-
dize the results of any educational assessment is test bias (Shohamy, 1997). Tests
are considered biased when they yield systematic differences in the results of the
test-takers. Item bias occurs when some examinees are less likely to answer items
than other examinees because of certain construct irrelevant item or examinee
characteristics. That is, if an item does not display differential item functioning
(DIF), it is not biased either. But if an item does show DIF the presence of bias
is not certain (Zumbo, 2007). Therefore, DIF is a precondition of bias and can
indicate it. DIF analysis is a statistical concept but bias analysis is substantive,
leading to the conceptual reasons why DIF has occurred. Therefore, DIF analysis
is a preliminary step that could inform bias analysis. In other words, test perfor-
mance should not be affected by examinees’ individual differences other than the
language ability of interest. Some of the irrelevant factors that might affect test
performance, according to Bachman (1990), are cultural background, background
knowledge, cognitive characteristics (e.g., field independence and ambiguity toler-
ance), native language, ethnicity, sex, and age. Biased tests may cause universities
and employers to accept unqualified individuals and reject qualified ones (McNa-
mara & Roever, 2006). Fair test development is important in language education,
and there is a need for research on different sources of bias to be detected and
eliminated. According to the Joint Committee on Testing Practices (1988), test
developers must attempt to make tests fair to all test takers of different races,
genders, and cultural backgrounds (as cited in Kunnan, 2000).
In order to have a fair test and to identify the probable sources of bias, re-
searchers have introduced DIF (McNamara & Roever, 2006). DIF analyses de-
termine whether individuals from one group answer an item more easily than
equally knowledgeable persons from another group (Zumbo, 2007). Elder (1997)
elaborated that the study of test bias is concerned with diagnosing and, if probable,
reducing the effect of any confounding variables on test scores by making changes
to the test. She maintained that confounding variables such as method effects,
background knowledge, and so on, within the test may systematically falsify the
test takers’ real ability. In the same vein, McNamara and Roever (2006) indicated
that “test-inherent bias” falsifies measurement by allowing test scores influence
systematically (p. 81). They also mentioned that bias is a general term for group
characteristics which can cause the scores to be distorted. Several studies have
been conducted on potential sources of bias (e.g., ethnicity, sex, native language,
and age), but psychological sources of bias have been neglected to a considerable
extent (Aryadoust, 2012; Chen & Henning, 1985; Geranpayeh, 2001; Geranpayeh
& Kunnan, 2007; Harpole et al, 2014; Pena, Iglesias, & Lidz, 2001). The way
people experience the world through their senses may be a possible source of
bias. Pishghadam, Jajarmi, and Shayesteh (in press) using the term “sensory rel-
ativism” have shown that individuals construct their own realities of the world
through the senses they receive information. According to them, irrespective of
 EMOTIONCY AS TEST BIAS 129

having positive, negative, or neutral emotions, each sense may generate a specific
amount of emotional power and intensity. Since not all individuals in a society
can experience all concepts directly and some may have only hear about or see
some of these concepts, it seems that this sense-emotion relationship can make
a difference in the ways individuals conceptualize different concepts. In other
words, individuals’ emotional relations may be a predictor of their success, and in
accordance may be a source of bias. In this respect, Pishghadam, Adamson, and
Shayesteh (2013) found that degree of the lexical emotions the test takers have
toward different words can lead to effective learning. Inspired by Greenspan’s
(1992) Developmental, Individual differences, Relationship-based (DIR) model
of first language (L1) acquisition, Pishghadam, Tabatabaeyan, and Navari (2013)
proposed a new emotion orientation in second/foreign language education. The
model asserts that individuals have different levels of emotions toward different
linguistic items. Pishghadam, Tabatabaeyan and colleagues (2013) coined the tech-
nical term “emotioncy,” which emphasizes the emotional potency of the words.
Emotioncy refers to the emotions evoked by the senses from which each person
may receive information: hearing about or seeing something may generate differ-
ent degrees of emotions. With that in mind, the present study intends to introduce
and examine the new concept of emotioncy as a possible source of test bias. Indeed,
given that emotioncy is yet in its infancy and requires much labor to flourish, we
did not measure emotions directly; hence, drawing on the recent studies conducted
on the concept (e.g., Pishghadam, Jajarmi et al., in press; Pishghadam, Shayesteh,
& Rahmani, in press) we are of the view that senses are rather associated with
emotions. It is thus hypothesized that the higher the level of emotioncy for words
in the native language, the more probable the test takers recall the same words in
English.

Emotioncy
Pishghadam, Adamson and colleagues (2013) stated that holding stronger emo-
tions toward second/foreign language lexical items leads to a deeper understanding,
and if a learner encounters a word with little or no emotional relations, language
learning may be hindered. In this respect, emotioncy was considered to measure
quantitatively the sense and emotion relations. Pishghadam, Tabatabaeyan and as-
sociates (2013) used emotioncy, where the focus of attention was on the emotional
representations linked to words. In fact, emotioncy may be able to determine the
salience of words and serve as a criterion for vocabulary teaching (Pishghadam
& Shayesteh, in press). Pishghadam (2015) also put emotioncy on a “hierar-
chical order of null, auditory, visual, kinesthetic, inner, and arch emotioncies”
(p. 1) and assigned each level a specific score. As illustrated in Figure 1, a high
level of emotioncy can lead to a high level of comprehension, or in general a
high level of learning and retention. In other words, the lower level emotioncies
130 PISHGHADAM, BAGHAEI, AND SEYEDNOZADI

FIGURE 1
Emotioncy levels.
Adapted from “Emotioncy in Language Education: From Exvolvement to Involvement,” by R.
Pishghadam, 2015, October, Paper presented at the 2nd Conference of Interdisciplinary
Approaches to Language Teaching, Literature, and Translation Studies. Iran, Mashhad).

(auditory, visual, and kinesthetic) lead to exvolvement since they engage students’
emotions from outside, while the upper level emotioncies (inner and arch) lead
to involvement as they engage learners from inside. For instance, between the
words “banana” and “chopsticks” an African child learns banana more easily,
since unlike chopsticks, he has touched, smelled, and tasted banana (Pishghadam
& Shayesteh, in press). As another example, it is supposed that a person who
graduated in engineering may have a high level of emotioncy for the word “steel”
compared to a person from a major unrelated to the word.
Pishghadam and Shayesteh (in press) noted that individuals with different
genders, cultural/educational backgrounds, or socioeconomic status may display
different rates of emotional familiarity with specific types of linguistic items. In
fact, emotions may be the result of individuals’ distinctive reactions to people,
objects, or words, and emotional reactions are not brought about just by the
language but also by the experiences conveyed with them (Pishghadam, Adamson
et al., 2013). One point that should be clarified is the distinction between
familiarity and emotioncy. Despite the familiarity that deals with cognitive
aspects of learning and prior knowledge in information processing (Piaget, 1954),
 EMOTIONCY AS TEST BIAS 131

emotioncy underscores the role of emotions generated by sensory inputs. Since

each sense has a distinct storage area in the brain and stimulates a specific part
of it (Wagner et al., 1998), cognition and emotion may be affected by the type
of sensory input from which one receives information. In a recent study on the
concept of phlebotomy, Pishghadam, Jajarmi and colleagues (in press) found
that people with different levels of emotioncy have shown different cognitive and
emotional reactions to the concept. Their outcomes have revealed that unlike the
exvolved individuals (auditory, visual, and kinesthetic emotioncies) who used
more associations and hedges in their interviews on phlebotomy, the involved
ones (inner and arch emotioncies) used more analogies and fewer hedges. In fact,
while the exvolved individuals formed hyper-/hypo-realities, which generated
more exaggerated (distal) emotions, the involved ones formed realities that
evoked less exaggerated (proximal) emotions. Thus, Pishghadam, Adamson and
associates (2013), underlining the role of localization, argued that due to the fact
that the test takers with different backgrounds may be more familiar with specific
types of concepts, and accordingly have stronger emotions toward them, cultures,
regions, and languages should be considered in language teaching and testing.
As an instance, within the context of Iran, due to the geographical diversities, an
individual coming from the northern areas of the country may be more emotionally
familiar with words like tree, jungle, and rain, while an individual coming from
the western areas of the country is likely to be more emotionally familiar with
words such as winter, snow, and mountain. With that in mind, we hypothesized
that English language learners who have high levels of emotioncy of the concepts
in their native language are more successful in recalling the same English words.
It means that since individuals come from different backgrounds (gender, social
class, level of education, etc.), they may have heard, seen, touched, experienced, or
done research into a specific concept in their native language, which can make a
difference in learning the same concept/word in a second language. On the whole,
considering the fact that emotional relations may play a crucial role in language
testing, the present study intends to answer the following major research question:
Do English language learners recall the English words for which they have higher
levels of emotioncy in their native language?

METHODS

Participants
The sample comprised 355 adult participants (120 male and 235 female) aged 14
to 53 years (M = 28.18, SD = 5.57) with Persian as their first language. They
were learners of English with elementary to advanced proficiency levels, who were
studying at different English language institutes in Mashhad, Iran.
132 PISHGHADAM, BAGHAEI, AND SEYEDNOZADI

Measures
To gather the required data, the following instruments designed by the researchers,
were utilized: a 40-item multiple-choice vocabulary test (see Appendix A) and
a 40-item emotioncy scale (see Appendix B). Regarding the vocabulary test, the
participants were to find the Persian meanings of 40 English vocabulary words.
These vocabulary words were selected in a way to ensure that not all the test takers
had the same level of emotioncy for the words. For example, the participants
were asked the meanings of these vocabulary words: phlebotomy, caviar, karting,
gambling, alms, and so on.
The emotioncy scale is a self-report scale in Persian (the participants’ L1)
constructed to rate the participants’ familiarity with the 40 vocabulary words for
which they had tried to find their Persian meanings. The points on the scale were:
not familiar (0 point); heard (1 point); heard and seen (2 points); heard, seen, and
touched (3 points); heard, seen, touched, and used (4 points); and heard, seen,
touched, used, and done research on (5 points). The following example illuminates
the scale.

stationary bike:

· A person’s emotioncy rate who had heard about, seen, touched, used and done 01 2 3 4 5
research on it.
· A person’s emotioncy rate who had heard about, seen, touched, and used it. 0 1 2 3 4 5
· A person’s emotioncy rate who had heard about, seen, and touched it. 0 1 2 3 4 5
· A person’s emotioncy rate who had heard about, and seen it. 0 1 2 3 4 5
· A person’s emotioncy rate who had heard about it. 0 1 2 3 4 5
·A person’s emotioncy rate who doesn’t know what it is. 0 1 2 3 4 5

The scale, in fact, measured their emotioncy for the vocabularies they were
tested in the vocabulary test. The Cronbach’s alpha reliability of the vocabulary
and emotioncy scales were 0.83 and 0.88, respectively.

Procedure
To collect the data, first the vocabulary test was given to the participants. It took
an average of 20 minutes for each participant to take the test. The emotioncy scale
was administered to them afterward. The test takers were asked to rate their level
of familiarity with the vocabulary words they had encountered in the vocabulary
test on a five-point Likert scale.
The principal analysis undertaken in the study was the Mixed Rasch Model
(MRM), which identifies the latent classes for which the Rasch model fits sepa-
rately (Rasch, 1960). MRM is mainly applied in studying individual differences
 EMOTIONCY AS TEST BIAS 133

in strategy use and knowledge differences, as well as investigating unidimension-

ality and validation (Baghaei & Carstensen, 2013). Technically speaking, when
the data do not fit the standard unidimensional Rasch model, MRM identifies the
subsamples or latent classes within the entire sample which fit the Rasch model.
The subsamples that fit the Rasch model are in fact the latent classes which differ
qualitatively. In other words, instead of conducting DIF across manifest variables
such as sex or native language, MRM finds subgroups across which DIF exists.
After identifying the latent classes, the item contents have to be studied to find
out the nature of the qualitative differences between the classes that had caused
the DIF. When latent classes are identified, the next steps is to examine the nature
of the classes and figure out in what way the classes are different. In other words,
class membership should be linked to some qualitative differences in examinees.
Such differences could suggest crucial factors in the learning process and may
lead to substantive theories (Baghaei & Carstensen, 2013). In this study we use
MRM to identify the latent classes within our sample across whom DIF exists. In
the next step, we examine if the classes can be related to the learners differences
in emotioncy levels.

RESULTS

The sample was split into two parts based on their total emotioncy scores: those
above the mean (M = 107) and those below the mean. An independent samples
t-test showed that the vocabulary mean of high-emotioncy examinees (M = 21.04
SD = 4.63) differed significantly from that of low-emotioncy examinees (M =
14.94 SD = 8.06), t (284) = 8.75, p < 0.001, mean difference = 6.09, 95% CI:
4.72 to 7.46; eta squared = 0.18.
The data were analyzed with the mixed Rasch model (Rost, 1990) using WIN-
MIRA program (von Davier, 2001). Three models with one, two, and three latent
classes were fitted to the data (See Table 1). The two-class model with sizes
0.78 and 0.22 had the best fit according to Akaike Information Criterion (AIC,
Akaike, 1974), Bayesian Information Criterion (BIC, Schwarz, 1978), and Con-
sistent Akaike Information Criterion (CAIC, Bozdogan, 1987).

TABLE 1
Information Criteria for the Three Models

Model AIC BIC CAIC

One class 17007 17166 17207

Two class 13605 13927 14010
Three class 13602 13932 14022
134 PISHGHADAM, BAGHAEI, AND SEYEDNOZADI

FIGURE 2
Class specific item parameter profiles.

Class specific item profiles in Figure 2 were also studied. As the plot shows,
Class 1 members have found items 1, 2, 5, 6, 12, 13, 19, 20, 22, 23, 29, 30, 31, 32,
33, 35, 36, 37, and 40 easier while Class 2 members have found the rest of the items
easier, except item 8 (caviar), which has the same level of difficulty in both classes
since it is a cognate in Persian and English. Table 2 shows item difficulties and their
standard errors within each latent class. D is the difference between item difficulty
estimates in the two latent classes and p shows the statistical significance of the
difference. As the table shows, 38 of the 40 differences are statistically significant
(p < 0.05).
The finding shows that the unidimensional Rasch model does not fit as the two-
class model was shown to be the best description of the data. When a two-class
model fits, it means that the probability of a correct reply depends both on the
person ability and the group membership (Baghaei & Carstensen, 2013). This is
the evidence of the invalidity of the test for the entire sample.
The two classes were compared in terms of emotioncy for the words they have
found easier. Findings revealed that Class 1 members have significantly greater
emotioncy scores (M = 51.67; SD = 13.11) on the items that they have found
easier than Class 2 members (M = 47.26; SD = 2.72), who found those items
harder; t (338) = 2.94, p < 0.001, Cohen’s d = 0.46.
 EMOTIONCY AS TEST BIAS 135

TABLE 2
Differential Item Functioning by Latent Class

Class 1 Class 2
Item Estimate SE Emotioncy Mean Estimate SE Emotioncy Mean D p

1 −1.01 0.14 3.34 1.74 0.24 2.99 −2.75 0.00

2 −0.44 0.13 3.26 1.2 0.25 2.46 −1.64 0.00
3 0.38 0.14 2.04 −2.21 0.27 3.97 2.59 0.00
4 −0.16 0.13 1.34 −2.61 0.31 4.00 2.46 0.00
5 0.37 0.14 1.84 1.21 0.26 2.49 −0.85 0.00
6 −1.27 0.14 3.59 1.62 0.24 0.99 −2.89 0.00
7 −0.1 0.13 3.35 −2.58 0.31 4.00 2.48 0.00
8 −1.15 0.14 2.29 −1.15 0.27 3.58 0 1.00
9 −0.01 0.14 2.66 −2.5 0.3 4.00 2.49 0.00
10 2.83 0.2 1.34 −1 0.23 4.00 3.82 0.00
11 0.71 0.14 3.05 −2.02 0.26 4.01 2.73 0.00
12 −1.15 0.14 2.47 1.69 0.24 2.03 −2.83 0.00
13 −0.23 0.13 2.26 2.38 0.28 2.00 −2.61 0.00
14 0.38 0.14 1.91 −2.21 0.27 4.06 2.59 0.00
15 −0.1 0.13 2.33 −1.99 0.3 4.08 1.89 0.00
16 −0.26 0.13 1.83 −2.55 0.32 4.17 2.29 0.00
17 −0.48 0.13 1.61 −2.88 0.34 4.04 2.4 0.00
18 −1.25 0.14 3.64 −3.39 0.44 4.05 2.15 0.00
19 2.26 0.21 1.93 4.46 0.66 1.99 −2.21 0.00
20 0.42 0.14 2.77 2.9 0.33 2.00 −2.48 0.00
21 −0.61 0.13 2.35 −2.98 0.35 4.01 2.38 0.00
22 −0.17 0.13 3.59 2.43 0.28 2.99 −2.6 0.00
23 −1.39 0.14 2.65 1.52 0.23 2.00 −2.91 0.00
24 0.19 0.14 3.19 −2.37 0.29 4.00 2.56 0.00
25 −0.17 0.13 2.61 −2.64 0.31 4.00 2.47 0.00
26 −1.07 0.14 2.55 −3.37 0.41 4.01 2.3 0.00
27 0.23 0.14 3.84 −2.33 0.28 4.10 2.56 0.00
28 2.46 0.19 1.86 −1.15 0.23 4.00 3.61 0.00
29 0.93 0.15 2.07 3.29 0.39 2.03 −2.37 0.00
30 −0.5 0.13 3.49 2.17 0.26 3.00 −2.67 0.00
31 0.19 0.14 2.54 2.72 0.31 2.99 −2.53 0.00
32 −0.43 0.13 2.70 2.23 0.27 2.97 −2.65 0.00
33 −1.43 0.14 3.56 1.51 0.23 2.99 −2.94 0.00
34 0.82 0.15 1.72 −1.94 0.26 3.99 2.75 0.00
35 −1.43 0.14 3.31 1.51 0.23 2.99 −2.94 0.00
36 0.21 0.14 2.55 1.28 0.26 3.23 −1.07 0.00
37 0.6 0.14 2.23 1.28 0.27 3.22 −0.68 0.02
38 1.75 0.18 2.16 1.28 0.27 2.47 0.46 0.15
39 0.88 0.15 2.09 −1.91 0.25 4.05 2.79 0.00
40 0.25 0.14 1.54 2.77 0.31 1.94 −2.52 0.00
136 PISHGHADAM, BAGHAEI, AND SEYEDNOZADI

Likewise, an independent samples t-test showed that Class 2 members have

significantly greater emotioncy scores (M = 79.02; SD = 1.51) on the items they
have found easier than Class 1 members on those same items (M = 47.44; SD =
11.36), t (307) = 44.85, p < 0.001, Cohen’s d = 3.89. This finding reveals that
the source of DIF across the two latent classes is the examinees’ emotioncy for
the items.
Table 2 indicates that item difficulty estimates match emotioncy means. That
is, when an item is easier for one of the latent classes (lower estimate) the mean
of the class on the emotioncy scale for the item is higher.
Content analysis of the items showed that the majority of the items the Class 2
members have found easier relate to masculine things such as screwdriver, stock
market, depositor, and so on, and the items Class 1 members have found easier are
more feminine words such as hair dye, knitting, curling iron, etc. Out of the 120
male examinees, 83 belong to Class 2 and 37 are in Class 1. Out of the 235 female
examinees, 142 belong to Class 1 and 93 belong to Class 2. A Chi-square test for
independence (after Yates correction) showed a significant relationship between
sex and class membership, χ 2 (1, n = 355) = 26.65, p < 0.001, phi = −0.28.

DISCUSSION

As mentioned earlier, examining sources of bias is an essential part of language

test validation (Bachman, 1991) and DIF research is one way of detecting these
sources of group differences. However, only a limited number of DIF factors
such as gender, race, and L1 background have been investigated thus far. In fact,
the emotional aspect of test takers’ characteristics has been neglected in research
studies (Garret & Young, 2009). Therefore, this study aimed at introducing and
examining the new concept of emotioncy as a possible source of bias. In particular,
it investigated whether DIF varied as a function of test takers’ emotioncy. In so
doing, two tests were designed and validated by the researchers: a vocabulary
test and an emotioncy scale, which examined the levels of test takers’ emotioncy
toward those vocabularies.
The results indicated that the test takers with high level of emotioncy out-
performed the ones with low level of emotioncy toward those specific words.
This suggests that the test takers with high emotioncy level learned and retained
the words longer than their counterparts with low emotioncy level. This find-
ing extends the work of Pishghadam and Shayesteh (in press) by showing that
higher emotioncy is associated with individuals’ finding certain words easier to
recall. Moreover, it endorses the notion that having emotional relations with some
specific language items is beneficial to in-depth learning (Pishghadam, Tabataeyan
et al., 2013; Pishghadam, Shayesteh, & Rahmani, in press). Further, this finding
 EMOTIONCY AS TEST BIAS 137

may be in line with that of Bown and White (2010) who suggested that there is a
need for more attention to the learners’ emotional experiences in the process of
second/foreign language learning.
One important point that should be emphasized here is that unlike the other
sources of test bias, which have a static nature and are resistant to change (gender or
race), emotioncy has a dynamic nature and can change from one test performance
to another. Emotioncy can help test takers have more inner or arch emotioncies
for the objects. It implies that we can ask the test takers to experience the new
concepts by themselves in order to increase their level of emotioncies. As the
outcomes of the study showed, even in one gender some individuals may have
high levels of emotioncy for some words, which can make them more successful
on the item. As Pishghadam and Shayesteh (in press) have argued, emotioncy
can be affected by individuals‘ access to economic and cultural capital. It goes
without saying that the more individuals have economic and cultural capital,
the more opportunities they will have to be involved in experiencing different
objects, and accordingly increase their level of emotioncy up to the inner or arch
ones.
Based on the findings of this study, bias can originate from the senses as
we receive information. The exvolved side of emotioncy (auditory, visual, and
kinesthetic) generates distal emotions in individuals which may lead to shallow
processing of the input, while the involved side (inner and arch) evokes proxi-
mal emotions which are close to reality, and directly engage individuals in deep
processing of data. As per our findings, if individuals are deeply involved in learn-
ing concepts, they may commit information more effectively to their long-term
memory. Therefore, individuals‘ deprivation of inner or arch emotioncies may
result in test unfairness. In fact, those learners who do not have access to some
objects, they cannot experience them directly. These individuals may have to hear
about or see the objects, which may lead to low levels of emotioncy for these
objects.
Overall, the implication of this study is that emotioncy as a source of sensory
test bias can change test takers‘ performance. Compared to other types of test bias,
emotioncy is more volatile and difficult to predict. Test takers should be required
to take an emotioncy scale for every test to flag DIF or maybe the test should cover
more diverse content areas to include words from all social classes, professions,
and so on. Test developers should be more cognizant of the concept of emotioncy
and attempt to understand this specific bias caused by the sensory involvement. In
fact, emotioncy, as a social-emotional concept, implies that one specific group of
individuals for some social, political, financial, and cultural reasons may be allowed
to have more access to different resources and be directly involved in experiencing
them, while others may have exvolved experiences (hear, see, and touch). It is our
hope that this newly developed concept can shed more light on the DIF research
138 PISHGHADAM, BAGHAEI, AND SEYEDNOZADI

and test bias understanding. Since our study was done on a vocabulary test in one
culture, we suggest that other studies be conducted to measure the emotioncy levels
of other tests in different cultures. Moreover, the concept of emotioncy as described
here applies to reading comprehension tests. Test takers with high emotioncy lev-
els for the text content may outperform their peers at the same level of reading
ability. Future research should focus on emotioncy as a source of bias in reading
comprehension tests. Last but not least, this study adopted an implied perspective
to delineate emotioncy as a probable source of test bias. As already mentioned, we
did not measure emotions directly and we just inferred that different levels of emo-
tioncy might evoke different amounts and types of emotions which can relativize
cognition. Further research is highly recommended to measure emotions along
with senses to come up with a more exact and comprehensive outlook toward the
issue.

ACKNOWLEDGMENTS

The authors wish to express their sincere gratitude to the editor and anonymous
reviewers of this manuscript who assisted us to fine-tune our paper. We give our
special thanks to the participants who willingly took part in this study.

REFERENCES

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic
Control, 19(6), 716–723.
Aryadoust, V. (2012). Differential Item functioning in while-listening performance tests: The case of
the International English Language Testing System (IELTS) listening module. International Journal
of Listening, 26(1), 40–60.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University
Press.
Bachman, L. F. (1991). What does language testing have to offer? TESOL Quarterly, 25(4), 671–704.
Baghaei, P., & Carstensen, C. H. (2013). Fitting the mixed Rasch model to a reading comprehension
test: Identifying reader types. Practical Assessment, Research & Evaluation, 18(5), 1–13.
Bown, J., & White, C. J. (2010). Affect in a self-regulatory framework for language learning. System,
38(3), 432–443.
Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): The general theory
and its analytic extensions. Psychometrika, 52, 345–370.
Chen, Z., & Henning, G. (1985). Linguistic and cultural bias in language proficiency tests. Language
Testing, 2(2), 155–163.
Elder, C. (1997). What does test bias have to do with fairness? Language Testing, 14(3), 261–277.
Garrett, P., & Young, R. F. (2009). Theorizing affect in foreign language learning: An analysis of one
learner’s responses to a communicative Portuguese course. The Modern Language Journal, 93(2),
209–226.
 EMOTIONCY AS TEST BIAS 139

Geranpayeh, A. (2001). Country bias in FCE listening comprehension (Cambridge ESOL Internal
Research & Validation Rep. No. 271). Cambridge, UK: Cambridge University.
Geranpayeh, A., & Kunnan, A. J. (2007). Differential item functioning in terms of age in
the certificate in advanced English examination. Language Assessment Quarterly, 4(2), 190–
222.
Greenspan, S. I. (1992). Infancy and early childhood: The practice of clinical assessment and in-
tervention with emotional and developmental challenges. Madison, CT: International Universities
Press.
Harpole, J. K., Levinson, C. A., Woods, C. M., Rodebaugh, T. L., Weeks, J. W., Brown, P. J., &
Liebowitz, M. (2014). Assessing the straightforwardly-worded brief fear of negative evaluation
scale for differential item functioning across gender and ethnicity. Journal of Psychopathology and
Behavioral Assessment, 37(2), 306–317.
Kunnan, A. J. (2000). Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in
language assessment (pp. 1–13). Cambridge, U.K.: CUP.
McNamara, T., & Roever, C. (2006). Psychometric approaches to fairness: Bias and DIF. In R. Young
& N. C. Ellis. (Eds.), Language testing: The social dimension (pp. 81–128). Oxford: Blackwell
Publishing.
Pena, E., Iglesias, A., & Lidz, C. S. (2001). Reducing test bias through dynamic assessment of children’s
word learning ability. American Journal of Speech-Language Pathology, 10(2), 138–154.
Piaget, J. (1954). The construction of reality in the child. New York: Basic Books.
Pishghadam, R. (2015, October). Emotioncy in language education: From exvolvement to involvement.
Paper presented at 2nd conference of Interdisciplinary Approaches to Language Teaching, Literature,
and Translation Studies. Iran, Mashhad.
Pishghadam, R., Adamson, B., & Shayesteh, S. (2013). Emotion-based language instruction (EBLI)
as a new perspective in bilingual education. Multilingual Education, 3(9), 1–16.
Pishghadam, R., Jajarmi, H., & Shayesteh, S. (in press). Conceptualizing sensory relativism in light
of emotioncy: A movement beyond linguistic relativism. International Journal of Society, Culture,
and Language.
Pishghadam, R., & Shayesteh, S. (in press). Emotioncy: A post-linguistic approach toward vocabulary
learning and retention. Sri Lanka Journal of Social Sciences.
Pishghadam, R., Shayesteh, S., & Rahmani, S. (in press). Contextualization-emotionalization interface:
A case of teacher effectiveness. International and Multidisciplinary Journal of Social Sciences.
Pishghadam, R., Tabatabaeyan, M. S., & Navari, S. (2013). A critical and practical analysis of first
language acquisition theories: The origin and development. Iran, Mashhad: Ferdowsi University of
Mashhad Publications.
Rasch, G. (1960). Probabilistic models for some intelligence and achievement tests. Copenhagen:
Danish Institute for Educational Research.
Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis.
Applied Psychological Measurement, 14(3), 271–282.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.
Shohamy, E. (1997). Testing methods, testing consequences: Are they ethical? Are they fair? Language
Testing, 14(3), 340–349.
von Davier, M. (2001). WINMIRA [Computer Software]. Groningen, the Netherlands: ASC- Assess-
ment Systems Corporation. USA and Science Plus Group.
Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, et al. (1998). Building
memories: Remembering and forgetting of verbal experiences as predicted by brain activity. Science,
281, 1188–1191.
Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is
now, and where it is going. Language Assessment Quarterly, 4(2), 223–233.
140 PISHGHADAM, BAGHAEI, AND SEYEDNOZADI

APPENDICES

Appendix A. Sample Items of the Vocabulary Test

Appendix B. Sample Items of the Emotioncy Scale

a. cement
0) I don’t know what it is 1) I have seen it 2) I have heard about it
3) I have touched it 4) I have used it 5) I have done research on it
b. stationary bike
0) I don’t know what it is 1) I have seen it 2) I have heard about it
3) I have touched it 4) I have used it 5) I have done research on it
c. nail polish
0) I don’t know what it is 1) I have seen it 2) I have heard about it
3) I have touched it 4) I have used it 5) I have done research on it
d. stethoscope
0) I don’t know what it is 1) I have seen it 2) I have heard about it
3) I have touched it 4) I have used it 5) I have done research on it