Facial Perception

Oliver Wilhelm, University Ulm, Ulm, Germany

Andrea Hildebrandt, Ernst-Moritz-Arndt University Greifswald, Greifswald, Germany
Werner Sommer, Humboldt-Universität zu Berlin, Berlin, Germany
Ó 2015 Elsevier Ltd. All rights reserved.

Abstract

In this article, we summarize popular theories of face identity and facial expression processing. Next, we present some general
findings on individual differences in this field including considerations regarding more common and specific face identity
and facial expression processing assessment principles. We then discuss available measures and present a task-taxonomy,
followed by a short summary of key findings in the field. We then present a summary of research results concerning
neural underpinnings in identity and emotion processing. Finally, we conclude with some remarks about the emerging field
of identity and expression investigations and some research heuristics.

Faces provide indispensable information for human communi-
cation. They inform the perceiver about the familiarity and
identity of others and about those persons’ emotional states
and intentions. The mechanisms of facial information process-
ing and the respective neural correlates received wide attention
in the last years. Most of this research is based on experimental
findings in healthy participants and evidence from persons
with innate or acquired disorders of face processing. This article
will focus on variability in these abilities across persons and the
structure of individual differences in facial information pro-
cessing, after providing a summary of theories built upon
experimental and clinical research.
Theories of Face Identity and Facial
Expression Processing
Functional models of facial information processing (e.g.,
Haxby et al., 2000; Young and Bruce, 2011) postulate a hierar-
chical structure, including view-centered pictorial and view-
independent structural encoding of the face, access to stored
representations of known faces in memory, and to associated
(biographical) knowledge about the person and his or her
name. In addition to identity processing, these models allow
predictions about how the face processing system deals with
variable information provided by faces – in particular,
emotional expressions, facial movements accompanying
speech, and gaze direction. Further, socially relevant informa-
tion that can be derived from the invariant face structure inde-
pendent of the person’s identity – i.e., age, gender, and
judgments about attractiveness – is mapped in these models
as visually derived semantic codes. Early theorizing assumed
independent streams of identity versus expression-related
information. This initial idea was recently modified in favor
of partial dependence views (e.g., Calder, 2011; Young and
Bruce, 2011). However, it is not yet fully clarified how these
systems interact.
Research on the general physiological implementation of
face-based processes distinguishes between a core system and
an extended system (Haxby et al., 2000). The core system
consists of three bilateral, hierarchically organized brain
regions. During initial stages of face perception, facial features
are processed in the inferior occipital gyri. The inferior occipital
gyri provide input to the lateral fusiform gyrus, where invariant
aspects of faces – arguably cues of unique identities – are
processed. This region has been found to respond more strongly
to the perception of faces than to the perception of nonface
objects (Kanwisher, 2000). Based on this and similar findings,
the so-called fusiform face area (FFA), situated in the fusiform
gyrus, has been postulated to be a specialized module for face
perception. If attention is directed to changeable aspects of
faces, like emotion expression or lip movement, the superior
temporal sulci are activated. Visually derived semantic codes,
biographical, and name information are processed in the
extended system (Haxby et al., 2000). Beyond the core system,
an extended neural system – specialized on different cognitive
functions – is considered to interact with face-responsive
regions (Haxby et al., 2000). We highlight four such systems.
First, the intraparietal sulcus is considered to be in charge of
processing spatially directed attention and thought to aid the
perception of eye gaze direction. Second, the audiovisual inte-
gration in speech perception (see the McGurk effect), which is
supported by the interplay of the auditory and superior
temporal cortex. Third, during emotion processing the superior
temporal sulcus maintains connections with the amygdala,
insula, and limbic system, which are the neural systems
involved in the processing of emotion. Fourth, access to
semantic and biographical information during face processing
appears to be mediated by anterior temporal regions. Taken
together, face processing is supported by multiple brain regions
that are spatially disparate, yet act together in a coordinated
way. The mechanisms of facial information processing are
supposed to be hierarchically organized and actual models
assume an interplay between the identity and expression pro-
cessing routes. Furthermore, Haxby et al. (2000) emphasize
that brain systems for facial information processing do not
work in isolation but contribute to other cognitive functions
such as object recognition.
Individual Differences in Face Identity
and Expression Processing
Neurocognitive theories of face identity and facial expression
processing describe general mechanisms and the functional
organization of the facial information processing system. The

676 International Encyclopedia of the Social & Behavioral Sciences, 2nd edition, Volume 8 http://dx.doi.org/10.1016/B978-0-08-097086-8.25060-5

taxonomy, stability, and predictive validity of individual
differences in these cognitive mechanisms have been rather
recently – at least partly – addressed as a set of independent
research questions.
Given the neuroanatomic evidence on at least partly distinct
brain localizations of core aspects of face identity and facial
expression processing one might be tempted to infer that the
abilities to process neutral and emotional faces are highly
distinct and independent. However, all of the proposed tasks
for measuring facial information processing have attributes of
prototypical ability tests: Participants are willing to do their
best, there is an objective correct response, and performance
is assessed as degree to which responses are correct. It is well
known that any two such measures will covary positively in
unselected samples – a phenomenon labeled positive manifold
in ability research. Therefore, it is unrealistic to expect that (1)
measures of face perception that capture individual differences
will be completely independent of general cognitive abilities
measures and (2) measures of facial emotion processing will
be orthogonal to measures of face identity processing. Addi-
tionally, when taking a closer look at emotion recognition tasks
Adolphs (2002) concluded that emotion identification may be
carried out as a combination of processes reflecting perception,
semantic knowledge, and reasoning. The independence and
redundance of performance in a variety of measures is best
investigated by studying their factorial structure.
How do we understand the emotional states of others?
Lipps (1907) suggested that persons imitate (mimic) observed
affective behavior and that this mimicry elicits a corresponding
state in the observer, facilitating or enabling affect under-
standing. This idea is echoed in modern embodied simulation
theories. Indeed, facial mimicry of emotional expressions is
frequently observed, although in some cases only with the
aid of electromyographic recordings. Mimicry reactions are
often involuntary, automatic, and occur within a fraction of
a second (Dimberg et al., 2000). The idea that simulation is
fundamental for the perception of emotions in others is in
line with findings that hindering incidental mimicry leads to
performance impairment in emotion perception (e.g., Neal
and Chartrand, 2011). However, whether persons who show
more incidental mimicry are also better emotion perceivers
and whether they do so in all situations is less clear. Hess
and colleagues did not observe such a relationship between
mimicry and emotion perception in young adults (e.g., Hess
and Blairy, 2001). Therefore, facial mimicry may not be neces-
sary for the classification of prototypical emotional displays
but may be helpful for social binding or when emotion recog-
nition is difficult. In line with the latter view, Künecke et al.
(2014), using a multivariate approach, showed a positive rela-
tionship between the degree of facial corrugator response and
the ability of emotion perception.
In the popular four-branch theory of emotional abilities,
face-based emotion processing is included as one of the key
components of emotional intelligence (Mayer et al., 2008).
In the component process models (Scherer, 2007) face-based
emotion processing is included as receptive decoding. Wilhelm
and colleagues (2010) claimed that face identity perception
and recognition are core aspects of such emotional abilities.
If emotional intelligence exists (Borsboom et al., 2004) indi-
vidual differences in perceiving neutral and emotional faces
Facial Perception 677
ought to be considered lower order factors in a higher order
model of socioemotional abilities. Based upon the process
theories delineated in the introduction facial information-
related decoding abilities may be further subdivided into
perceptual and mnestic aspects of performance. Additionally,
it is not yet settled whether or not the perception of
neutral and emotional faces needs to be distinguished on an
individual differences level. Importantly, broader theories of
socioemotional abilities that postulate a series of emotion
receptive performance components ought to include more or
less specific abilities of processing identity and emotion in
the voice, gestures, and postures. Obviously, systematically
deriving ability measures of such components that are
grounded in basic research and incorporating them into factor
models of socioemotional abilities is a major challenge for
future research.
Apart from distinguishing perceptual and mnestic demands
of tasks challenging either identity or expression processing of
faces, there is one more crucial task attribute requiring consid-
eration: whether performance is expressed as speed or accu-
racy. In experimental research many of the tasks used to
provoke face identity and facial expression processing are
easy to solve. Comparisons between experimental conditions
are then usually based on the latency of responses. Within
ability research such measures would be considered processing
speed tasks because participants from the application popula-
tion solve a very high proportion of trials correctly and indi-
vidual differences are primarily manifest in the response
latency. It is important to note that easy tasks and hard tasks
usually load on different ability factors. Latency-related ability
factors may or may not arise as a function of the information
content. Indeed, measures challenging the speed of object
processing are essentially perfectly collinear with measures
challenging the speed of face identity and facial expression
processing (Hildebrandt et al., 2012, 2013). This is not true
for measures of face identity and facial expression processing
scored by the accuracy of responses (Wilhelm et al., 2010).
Given the distinction between speed and accuracy tasks that
is well established in ability research and the high collinearity
of speed-related abilities that lack facets of identity or emotion
processing, we recommend avoiding measures of face identity
and facial expression processing exclusively challenging the
speed of performance.
After narrowing down the face identity and expression
processing ability domains based on well-funded experimental
research, it is instrumental to consider prior work on socioe-
motional abilities from other fields of individual differences
research. The so-called jingle fallacy refers to instances in
which constructs are deemed the same, although they only
share a similar verbal label. Conversely, the jangle fallacy
refers to instances in which constructs are deemed different
although the only obvious distinction is the verbal label
established within a scientific community. The recently
popularized theory of mind and measures arguably capturing
individual differences in this ability are probably an instance
of a jangle fallacy. The Reading the Mind in the Eyes Test
displays photographs of the eye region of 36 actors and
actresses. Participants are to select which of several response
options best matches what those persons think or feel.
Obviously, the stimuli and the task presented here are akin
678 Facial Perception
to provoking facial expression processing. It is therefore
important to solidly clarify the status of individual
differences in such theory of mind tasks. Presupposing
adequate psychometric characteristics of the measures, we
would predict very high correlations between theory of mind
and emotion identification (i.e., a jangle fallacy). In that
case, the theory of mind ability may not need distinct
theorizing.
Measures of Face Identity and Facial
Expression Processing
One way to investigate the status quo of individual differences
research in this area is to go through available ability measures
and search for tasks that use faces as stimuli. We will describe
a few such measures – beginning with tasks challenging iden-
tity processing and proceeding to measures of facial expression
processing – and highlight their positive and negative attri-
butes. It is important to note that we restrict the presentation
here to protypical ability measures; meaning the behavior of
participants is assessed by comparing responses to an objective
standard of performance.
In the famous faces test, participants are asked to pick the
correct name out of a few alternatives for each famous face
shown. As a measure of identity recognition, this type of test
is conceptually problematic because it neglects differences
between persons in prior exposure to the face stimuli and
knowledge about the displayed famous persons. In the War-
rington Recognition Memory for Faces Test no control con-
cerning stimulus attributes irrelevant for face cognition, like
clothing, hair, or paraphernalia such as glasses, is applied.
Therefore, these irrelevant stimulus attributes might be used
to solve the task. Presumably, this is specifically true for partic-
ipants with severe problems in processing faces, such as proso-
pagnosic patients. In the Benton Facial Recognition Test,
participants are presented a target photo and are asked to select
the target identity from six simultaneously presented response
alternatives. In some items participants need to identify an
identical picture, in others they have to pick the correct identity
when pictures are taken from different angles or in different
lighting conditions. Although stimulus control is much better
here, participants can (and do) still rely on eyebrows and hair
lines in processing faces. The objections raised above also
apply to most of the available measures of socioemotional
abilities and learning and recognition of face-tasks included
in many intelligence and memory test batteries. Additionally,
in many of these tasks components unrelated to identity and
expression processing contribute to performance.
In order to solve issues with face external stimulus compo-
nents, Duchaine and Nakayama (2006) developed the
Cambridge Face Perception Task (CFPT) and the Cambridge
Face Memory Test (CFMT). These tests include only face
internal features and the perception task successfully mini-
mizes mnestic demands. The CFPT applies morphing: the crea-
tion of new faces through blending two different faces into
each other. Target faces are morphed with several other faces
varying the similarity of the morphed faces with the target
face. The target face is presented along with the morphs and
participants are asked to order the morphs according to their
similarity with the target face. In the CFMT, participants are
asked to learn and – after a very brief retention interval – recog-
nize pictures of six faces in different conditions (identical,
different angles, different lighting, and degraded through
visual noise). Both of the Cambridge tests are certainly supe-
rior to many earlier measures.
Despite this somewhat mixed assessment concerning
ability tasks relying on neutral faces as stimuli, recent develop-
ments in the field are encouraging. On the one side, we recently
presented a task battery that allows measuring individual
differences in perceiving and recognizing neutral faces and
were able to establish and replicate the distinction of these
abilities from object recognition and established ability factors
(Wilhelm et al., 2010). Within the task battery, an additional
aim was to distinguish between tasks challenging either speed
or accuracy of processing – a prominent distinction in intelli-
gence research outlined above (Carroll, 1993). The multitask
approach for measuring face identity processing-related abili-
ties presented by Wilhelm et al. (2010) includes task para-
digms widely used in experimental research on face
processing (Part–Whole, Composite Faces, and the Inversion
paradigms), different memory paradigms (delayed non-
matching, immediate recall, delayed recall, implicit
memory), and over 500 face identities across all tasks. The
factorial structure of the tests has been established across
a series of independent samples.
With respect to facial expression processing, many
measures have an intrinsic component of perceptual speed
because stimulus exposition is limited in time (i.e., Japanese
and Caucasian Brief Affective Recognition Test (Matsumoto
et al., 2000)). The popular Diagnostic Analysis of Nonverbal
Accuracy (DANVA; Nowicki and Carton, 1993), which chal-
lenges recognition of happiness, sadness, anger, and fear in
emotional faces, also has a strictly time-limited stimulus
exposition. The manipulations of expression intensity in the
DANVA would probably allow the derivation of accuracy
scores to express performance. However, the tasks have not
yet been probed as measures of accuracy.
The Profile of Nonverbal Sensitivity (Rosenthal et al., 1979) is
another popular measure, which uses recordings of faces,
voices, and bodies as stimuli to assess emotion recognition
from multimodal and dynamic stimuli. The use of dynamic
facial stimuli is an interesting aspect of the measure but an
important drawback here is the limitation to only two emotion
categories (positive versus negative).
One out of several measures designed for the assessment of
potential disorders of affect processing is the Florida Affect
Battery (Bowers et al., 1999), which uses static facial, vocal,
and cross-modal stimuli and multiple methods
(discrimination, naming, selection, and matching). This test
is arguably too easy for unimpaired subjects.
The Multimodal Emotion Recognition Test relies on 10 cate-
gories of emotion expressions but the measure is limited to
affect naming as a single observational method. The Geneva
Emotion Recognition Test (Schlegel et al., 2014) features
dynamic and multimodal actor portrayals of 14 emotions –
amusement, anger, disgust, despair, pride, anxiety, interest,
irritation, joy, fear, pleasure, relief, surprise, and sadness –
but exclusively relies on affect naming.

Two tasks challenging perceptual matching and explicit
labeling of expression with satisfying psychometric back-
ground were recently presented (Palermo et al., 2013). Both
tasks build upon the six basic emotions (anger, disgust, fear,
happiness, sadness, and surprise). The first task applies an
odd-man-out paradigm (i.e., participants have to indicate
which out of three simultaneously presented pictures
displays a different emotion). The second task requires
participants to explicitly identify the emotion in a target face
by selecting one of six written labels.
The validity and utility of highly specific information pro-
cessing paradigms like the Emotional Stroop Test, for example,
(Matthews et al., 2003) is still unclear.
Most of the above approaches do not allow abstracting
from task-specific variance – a crucial step toward studying
latent variables (ability estimates) rather than manifest raw
scores. Unfortunately, multivariate approaches are more often
an exception than the rule in the field. Additionally, stimulus
sampling seems to be particularly relevant when creating items
and compiling tests (Judd et al., 2012). Very recently Wilhelm
et al. (2014) proposed a task battery for measuring facial
expression perception and recognition. The same distinction
of speed versus accuracy-based measures for face identity pro-
cessing has also been implemented for the expression process-
ing tasks. The battery includes five tasks for measuring facial
expression perception and identification that are based on
the composite paradigm, the face inversion paradigm,
a measure of emotion discrimination in morphed continua
between facial expressions that share action units, and a visual
search paradigm with emotional expressions of different inten-
sity. The battery also includes recognition measures that
successfully avoid confusing identity and emotion processing
and applies different pictures in the learning versus recognition
stages. In order to avoid biases arising through applying the
same methods for performance assessment repeatedly, multi-
variate achievement tests should vary memory cues, viewing
angles, expression intensity, and the like.
Summary of Key Findings
Theories of face-based receptive communicative abilities – as
reviewed above – are mainly originating from the cognitive-
experimental literature and were investigated primarily with
experimental, neurophysiological, and neurobiological data.
However, the structure and predictive validity of mental
functions such as face perception, face memory, emotion
perception, and emotion memory can only be determined by
studying individual differences in a variety of adequate tasks.
Overarching theories based on such data – for example, the
four-branch theory of emotional intelligence (e.g., Mayer
et al., 2008) – are conceptually and theoretically
controversial, mainly because their grounding in more basic
science is insufficient. Based on the theoretical distinctions
and models reviewed above, we postulate a taxonomic
model of individual differences in facial information
processing that is – arguably – supported by available data.
Following the well-established distinction in the field, a
first differentiation of emotional abilities is between recep-
tive (decoding) and productive (encoding) performance.
Facial Perception 679
A second critical distinction is between speed performance
(i.e., latencies of correct responses) and the accuracy of
performance (i.e., proportion of correct responses in tasks
that are so hard that some participants cannot solve them
regardless of how much time is available). A third important
categorization refers to predominant perceptual or mnestic
demands of tasks.
Several larger studies critically test these distinctions. For
example, based on two larger studies Wilhelm et al. (2010)
provided evidence for three factors of face cognition: (1) the
accuracy of face perception, (2) the accuracy of face memory,
and (3) the speed of perceiving and recognizing unfamiliar
faces. These factors were replicated in an independent sample
of younger adults with new identities as face stimuli
(Hildebrandt et al., 2013). These studies show a clear distinc-
tion between speed and accuracy factors, which were nearly
completely independent of each other. The shared variance
between the performance accuracy and speed of face cognition
was less than 5%. This finding has a strong implication for
theories and research on face processing, because it shows
that research outcomes can tell completely different stories,
depending on whether speed or accuracy is analyzed. Addi-
tionally, these studies show unequivocally that perception of
and memory for faces are distinguishable abilities on the basis
of accuracy indicators, but not on the basis of speed indicators.
The expectation of a distinction between perception and
memory was based on the assumption stressed in face process-
ing models, that at a perceptual stage structural encoding of
faces is central, whereas at a memory stage, the access to face
representations stored in memory is critical. Obviously,
memory performance will partly depend on successful percep-
tion. Interestingly, such a perception versus memory differen-
tiation is not feasible on the basis of speed data. Individual
differences in the swiftness of recognizing well-learned faces
are perfectly collinear with face perception speed. In order to
further clarify the status of the speed factor of face cognition,
the convergent validity of this factor was studied. On the level
of latent variables the speed of face cognition was not distinct
from the speed of processing facial emotional expressions and
both facial speed factors could not be distinguished from the
speed of object cognition. Therefore, the swiftness of
performance in these three domains has zero specificity
(Hildebrandt et al., 2013).
With respect to the two factors that describe and explain
performance in the perception and memory accuracy tasks
there is strong and replicated evidence that they need to be
dissociated from general cognition, object cognition, imme-
diate and delayed memory for words, numbers and figures,
and mental speed. In several samples of young adults, a large
amount of variance that is specific for the face cognition factors
was replicated. In the first of these studies there was 46% face
perception specific variability and 52% face memory specific
variability. In an additional study (Hildebrandt et al., 2011),
covering an age range from 18 to 88 years, there was no moder-
ator effect of age on the relationship between face perception
and face memory versus general cognition, memory, and
mental speed. Therefore, this study shows the absence of
aging-related dedifferentiation between face cognition on
one side and several factors of general cognitive functioning
on the other side. Thus, the specificity of face cognition did
680 Facial Perception
not diminish across adult age, despite the expected substantial
age-related performance declines. Taken together these results
support the assumption that individual differences in face
cognition constitute abilities with substantial uniqueness
across the adult range.
Conceptually, face perception and face memory can be
considered necessary but not sufficient for emotion perception
(identification) and memory for emotional expressions. Their
empirical distinctness is an essential question to be addressed.
Contemporary neurocognitive models of face processing
describe partial overlap between the identity and expression
processing routes. Recently, Palermo et al. (2013) established
correlational evidence in favor of a common first step of iden-
tity and expression processing. A factorial model that approx-
imates models of facial information processing may have
a nested structure. A theoretical model is depicted in Figure 1.
To simulate the hierarchical structure of processing face-
related information – differentiated through successive pro-
cessing stages in neurocognitive models – the model first
postulates a general factor of face perception. This general
factor captures interindividual variability of face perception
tasks and accounts for face perception-related variability in
all other indicator tasks measuring face memory and facial
emotion processing. Second, to represent a partial overlap of
the routes processing face identity versus expression that also
holds at later stages, the factorial model includes a broad
face memory factor that accounts for variability in all memory
tasks. Importantly, this factor is purified from variance due to
perceptual processing. Two further more specific factors
account for variability due to processing emotion-related
information only.
The model (1) implies a partial dependency of emotion
processing from identity processing and (2) allows for speci-
ficity of emotion processing over and above identity cognition.
However, the degree of specificity of emotion processing-
related factors partialled for face identity processing-related
variance is still an unanswered research question. The postu-
lated model needs to be tested on the basis of accuracy data
because speed measures of facial emotion processing and the
speed of face cognition are perfectly collinear with speed
measures lacking interpersonally relevant content.
Figure 1 Schematic theoretical model of facial identity and emotion processing; ovals represent latent variables and squares depict indicators used
to measure the latent variables.
Once a nontrivial degree of specificity of emotion process-
ing-related factors is established, the emotion specificity is an
important research question. It will be interesting to see how
strongly indicators of performance in different emotion cate-
gories (happiness, surprise, fear, sadness, disgust, and anger)
will be related with each other. Although it has been argued
that persons may differ in their accuracy to identify specific
emotional expressions, because persons may be predomi-
nantly faced with different affects during early socialization
(e.g., Tomkins and McCarter, 1964), first evidence suggests
a lack of emotion category specificity of individual differ-
ences (Schlegel et al., 2014). Multivariate research designs
will be able to provide more conclusive evidence concerning
this question.
Neural Underpinnings of Individual
Difference in Face Cognition
The neural mechanisms underlying individual differences in
face cognition may provide a deeper understanding of why
people differ in these abilities. Performance measures used to
derive ability factors are end products of the information pro-
cessing occurring within the neurocognitive system. In
contrast, neuroscientific methods can provide information
about the success, organization, and timing of intermediate
processing steps that lead to the manifest, end products of
performance.
Neuroimaging studies on individual differences in face pro-
cessing-related abilities are surprisingly rare and confined to
small samples of usually not more than 30 participants. The
responsivity in bilateral anterior temporal poles and right ante-
rior hippocampus correlated positively with self-rated
preexperimental familiarity of face stimuli. Rotshtein et al.
(2007) reported that the behavioral impact of second-order
(configural) information of faces correlated with the
activation of bilateral inferior occipital gyrus and right
fusiform gyrus. Both studies rely only on small samples. For
a mixed group of normal and prosopagnosic participants Furl
et al. (2011) reported that better performance in facial
identity processing was positively associated (1) with face

selectivity in fusiform gyri bilaterally and left anterior temporal
lobe, (2) with peak face selectivity in the right and left
individually defined FFA, and (3) with the size of the right FFA.
Efforts to tap the neurocognitive mechanisms underlying
face cognition in large samples using multivariate modeling
are currently restricted to the analysis of event-related brain
potentials (ERPs). ERPs are complex responses of the brain
to discrete events, such as the presentation of a stimulus, and
consist of distinct components that can be related to specific
cognitive subprocesses. Amplitudes and latencies of ERPs
reflect the intensity and the timing of the processes reflected
in these components. Therefore, it is of interest whether
individual differences in specific ability factors are related to
amplitudes and latencies of specific performance-relevant
ERPs. Of particular interest here are ERPs that have been
related to the putative stages of face processes as postulated
in the functional models outlined above.
The occipital P100 is related to the processing of domain-
general, low-level stimulus features and is characterized by
a positive peak at occipital scalp sites around 75–135 ms; it
is thought to be generated in extrastriate visual brain areas.
The N170 is characterized by a negative peak at
occipitotemporal sites around 150–190 ms, typically larger
for faces than for other objects, but still unaffected by the
familiarity of faces. Therefore, it is considered to reflect the
configural encoding of facial features and their integration
into a holistic percept. Various brain areas have been
proposed as sources of the electrical N170, including
fusiform, lingual, or occipitotemporal gyri. The Early
Repetition Effect (ERE) or N250r is usually operationalized
as the difference wave between ERPs to primed and
unprimed faces in priming tasks; it is most pronounced at
temporal-parietal sites around 260–330 m and presumably
originates in the fusiform gyrus. The ERE and Late Repetition
Effect (LRE) (see below) are more pronounced for familiar as
compared to unfamiliar faces and for personally familiar
than for famous faces. Hence, the ERE is associated with the
activation of structural representations of faces in long-term
memory and with the identification of familiar faces (e.g.,
Schweinberger and Burton, 2011). Finally, the LRE or N400
– also typically observed in priming experiments – has been
related to the activation of person-related knowledge in long-
term memory (e.g., Schweinberger and Burton, 2011). It is
characterized as the difference wave between ERPs to primed
and unprimed faces, usually consisting of a parietal positivity
between 330 and 500 m. According to the model for familiar
face recognition (see above) the neural generators of the LRE
are considered to be part of the extended neural system
involved in the retrieving of person-related knowledge (i.e.,
anterior temporal cortex, precuneus, posterior cingulate
cortex).
Kaltwasser et al. (2014) investigated how these ERP compo-
nents are related with face cognition abilities and with general
cognitive functioning. All ERP components could be modeled
as unidimensional reliable latent variables. Several such exper-
imentally independent ERPs were only moderately correlated,
suggesting that they ought to be considered indicators of
specific, essentially independent neurocognitive processes.
Shorter latencies of the N170 component along with shorter
latencies and larger amplitudes of the ERE and LRE for learned
Facial Perception 681
faces have been associated with better face cognition perfor-
mance. The amplitude of the ERE for learned faces was the
strongest predictor of face cognition. The behavior versus
ERP latency associations were not face specific, because they
diminished after partialing general cognitive functioning.
A few studies investigated the neural underpinnings of indi-
vidual differences in emotional facial expressions. Kim et al.
(2003) reported that more intense subjective experience of
surprise is correlated with stronger amygdala activation as
measured in fMRI. Using an extreme group design Corden
et al. (2006) found less activation of amygdala, fusiform,
and anterior temporal cortex in response to neutral faces for
participants scoring low on a fear recognition task as compared
to a group with normal scores in this task. An ERP study by
Tamamiya and Hiraki (2013) found that N170 amplitude pre-
dicted the recognition accuracy of happy and angry faces.
Conclusions
Research on face identity and facial expression processing is
a rapidly moving field with exciting new developments in
many areas. These developments include elaborations on
neurocognitive models into the more applied field of indi-
vidual differences. One consequence of this elaboration is
that more and better measurement instruments will be avail-
able for research purposes and applied assessment. Research
results emerging from this field also have the potential to
dramatically improve our current concepts of socioemotional
abilities. In opposition to ex-cathedra ideas about the status
and organization of socioemotional abilities, an approach
grounded in neurocognitive work on identity and emotion
processing is obviously a more solid foundation of
psychological research. Based upon such a foundation,
a variety of intriguing research questions can be proposed.
Some of these questions concerning face identity and facial
expression processing not addressed in the present
contribution include behavior genetics, molecular genetics,
sex differences, relevance for a variety of mental disorders,
developmental trajectories in childhood, youth, and aging,
training and interventions based upon identities and
expressions, and many more. Research in all of these areas
hinges upon adequate measurement instruments. Due to the
relevance of these instruments, we would like to conclude
with statements that might seem evident, obvious, and
somewhat boring but nevertheless have not been adhered to
sufficiently in the past.
The unfolding individual differences research agenda in the
field of face identity and facial expression processing is best
advised to adhere to the following simple heuristics: Human
abilities should be measured with ability indicators. Self-
report measures are not ability indicators. Therefore, self-
report measures should not be used to study socioemotional
abilities. Theoretical claims about abilities should be based
on evidence that shows adequate coherence and
dependability within measures. Several such measures
designed to capture the same ability should show coherence
and dependability across measures. Once combinations of
established abilities are partialed from such a set of measures
a nontrivial amount of residual variance should be present.
682 Facial Perception
This residual variance should predict criteria or outcomes of
relevance. The presently available evidence summarized in
this contribution endorses the idea that some of the
proposed socioemotional ability factors are distinct from
established ability factors. This is no small achievement
given the fate of many initially intriguing but soon
abandoned ideas brought forward in ability research.
See also: Emotion and Expression; Emotion, Perception and
Expression of; Emotional Intelligence and Competencies;
Emotional Regulation; Emotions, Sociology of; Emotions:
Methods of Assessment; Facial Emotion Expression, Individual
Differences in; Factor Analysis and Latent Structure Analysis:
Confirmatory Factor Analysis; Factor Analysis and Latent
Variable Models in Personality Psychology; Personality
Assessment; Personality, Trait Models of; Social Intelligence
and Competencies; Social and Emotional Development in the
Context of the Family.
Bibliography
Adolphs, R., 2002. Neural systems for recognizing emotion. Current Opinion in
Neurobiology 12, 169–177.
Borsboom, D., Mellenbergh, G.J., van Heerden, J., 2004. The concept of validity.
Psychological Review 111, 1061–1071.
Bowers, D., Blonder, L.X., Heilman, K.M., 1999. Florida Affect Battery, A Manual.
Centre for Neuropsychological Studies, Cognitive Science Laboratory, University of
Florida.
Calder, A.J., 2011. Does facial identity and facial expression recognition involve
separate visual routes? In: Calder, A.J., Rhodes, G., Johnson, M.H., Haxby, J.V.
(Eds.), The Oxford Handbook of Face Perception. Oxford University Press, Oxford,
pp. 427–448.
Carroll, J.B., 1993. Human Cognitive Abilities: A Survey of Factor-Analytic Studies.
Cambridge University Press, Cambridge, United Kingdom.
Corden, B., Critchley, H.D., Skuse, D., Dolan, R.J., 2006. Fear recognition ability
predicts differences in social cognitive and neural functioning in men. Journal of
Cognitive Neuroscience 18, 889–897.
Dimberg, U., Thunberg, M., Elmehed, K., 2000. Unconscious facial reactions to
emotional facial expressions of emotions. Psychological Science 11, 86–89.
Duchaine, B., Nakayama, K., 2006. The Cambridge Face Memory Test: results for
neurologically intact individuals and an investigation of its validity using inverted
face stimuli and prosopagnosic participants. Neuropsychologia 44, 576–585.
Furl, N., Garrido, L., Dolan, R.J., Driver, J., Duchaine, B., 2011. Fusiform gyrus face
selectivity relates to individual differences in facial recognition ability. Journal of
Cognitive Neuroscience 23, 1723–1740.
Haxby, J.V., Hoffman, E.A., Gobbini, M.I., 2000. The distributed human neural system
for face perception. Trends in Cognitive Sciences 4, 223–233.
Hess, U., Blairy, S., 2001. Facial mimicry and emotional contagion to dynamic
emotional facial expressions and their influence on decoding accuracy.
International Journal of Psychophysiology 40, 129–141.
Hildebrandt, A., Wilhelm, O., Schmiedek, F., Herzmann, G., Sommer, W., 2011. On
the specificity of face cognition compared to general cognitive functioning across
adult age. Psychology and Aging 26, 701–715.
Hildebrandt, A., Schacht, A., Sommer, W., Wilhelm, O., 2012. Measuring the speed
of recognising facially expressed emotions. Cognition and Emotion 26, 650–666.
Hildebrandt, A., Wilhelm, O., Herzmann, G., Sommer, W., 2013. Face and object
cognition across adult age. Psychology & Aging 28, 243–248.
Judd, C.M., Westfall, J., Kenny, D.A., 2012. Treating stimuli as a random factor in
social psychology: a new and comprehensive solution to a pervasive but largely
ignored problem. Journal of Personality and Social Psychology 103, 54–69.
Kaltwasser, L., Hildebrandt, A., Recio, G., Wilhelm, O., Sommer, W., 2014. Neuro-
cognitive mechanisms of individual differences in face cognition: a replication and
extension. Cognitive, Affective, and Behavioral Neuroscience 14, 861–878.
Kanwisher, N., 2000. Domain specificity in face perception. Nature Neuroscience 3,
759–763.
Kim, H., Somerville, L.H., Johnstone, T., Alexander, A.L., Whalen, P.J., 2003. Inverse
amygdala and medial prefrontal cortex responses to surprised faces. Neuroreport
14, 2317–2322.
Künecke, J., Hildebrandt, A., Recio, G., Sommer, W., Wilhelm, O., 2014. Facial EMG
responses to emotional expressions are related to emotion perception ability. PLOS
ONE 9 (1): e84053.
Lipps, T., 1907. Das Wissen von fremden Ichen. In: Lipps, T. (Ed.), Psychologische
Untersuchungen. Engelmann, Leipzig, pp. 694–722.
Matthews, G., Roberts, R.D., Zeidner, M., 2003. Development of emotional
intelligence: a skeptical- but not dismissive perspective. Human Development 46,
109–114.
Matsumoto, D., LeRoux, J., Wilson-Cohn, C., Raroque, J., Kooken, K., Ekman, P.,
Yrizarry, N., Loewinger, S., Uchida, H., Yee, A., Amo, L., Goh, A., 2000. A new
test to measure emotion recognition ability: Matsumoto and Ekman’s Japanese
and Caucasian Brief Affect Recognition Test (JACBART). Journal of Nonverbal
Behavior 24, 179–209.
Mayer, J.D., Roberts, R.D., Barsade, S.G., 2008. Human abilities: emotional
intelligence. Annual Review of Psychology 59, 507–536.
Neal, D.T., Chartrand, T.L., 2011. Embodied emotion perception: amplifying and
dampening facial feedback modulates emotion perception accuracy. Social
Psychological and Personality Science 2, 673–678.
Nowicki, S., Carton, J., 1993. The measurement of emotional intensity from facial
expressions; the DANVA FACES 2. Journal of Social Psychology 133,
749–751.
Palermo, R., O’Connor, K.B., Davis, J.M., Irons, J., McKone, E., 2013. New tests to
measure individual differences in matching and labelling facial expressions of
emotion, and their association with ability to recognise vocal emotions and facial
identity. PLoS ONE 8, e68126.
Rotshtein, P., Geng, J.J., Driver, J., Dolan, R.J., 2007. Role of features and
second-order spatial relations in face discrimination, face recognition, and
individual face skills: behavioral and functional magnetic resonance imaging
data. Journal of Cognitive Neuroscience 19, 1435–1452.
Rosenthal, R., Hall, J.A., DiMatteo, M.R., Rogers, P.L., Archer, D., 1979. Sensitivity to
Nonverbal Communication: The PONS Test. John Hopkins University Press,
Baltimore.
Schlegel, K., Grandjean, D., Scherer, K.R., 2014. Introducing the Geneva emotion
recognition test: an example of Rasch-based test development. Psychological
Assessment 26, 666–672.
Scherer, K., 2007. Componential emotion theory can inform models of emotional
competence. In: Matthews, G., Zeidner, M., Roberts, R.D. (Eds.), Science of
Emotional Intelligence: Knowns and Unknowns. Oxford University Press, Oxford,
pp. 101–126.
Schweinberger, S.R., Burton, A.M., 2011. Person perception 25 years after Bruce
and Young (1986): an introduction. British Journal of Psychology 102,
695–703.
Tamamiya, Y., Hiraki, K., 2013. Individual differences in the recognition of facial
expressions: an event-related potentials study. PLoS ONE 8.
Tomkins, S., McCarter, R., 1964. What and where are the primary affects? Some
evidence for a theory. Perceptual and Motor Skills 18, 119–158.
Wilhelm, O., Hildebrandt, A., Manske, K., Schacht, A., Sommer, W., 2014. Test
battery for measuring the perception and recognition of facial expressions of
emotion. Frontiers in Psychology, Emotion Science 5, 404.
Wilhelm, O., Herzmann, G., Kunina, O., Danthiir, V., Schacht, A., Sommer, W., 2010.
Individual differences in face cognition. Journal of Personality and Social
Psychology 99, 530–548.
Young, A.W., Bruce, V., 2011. Understanding person perception. British Journal of
Psychology 102, 959–974.
Relevant Websites
http://www.faceblind.org/ – Prosopagnosia Research Centers at Harvard University
and University College London.
http://www.face-rec.org/ – Face Recognition Homepage.
http://scienceaid.co.uk/psychology/cognition/face.html – Science aid: face recognition.
http://ukcatalogue.oup.com/product/9780199559053.do – Oxford Handbook of Face
Perception.