Event-related Brain Potential Studies in Language

Angela D. Friederici, PhD

Address Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, PO Box 500 355, Leipzig 04303, Germany. E-mail: angelafr@cbs.mpg.de Current Neurology and Neuroscience Reports 2004, 4:466470 Current Science Inc. ISSN 1528-4042 Copyright 2004 by Current Science Inc.

A review of the four relevant language-related components in event-related brain potentials (ERPs) is provided. The different ERP components are functionally specified: the N400 component reflects semantic processes, the ELAN reflects early syntactic processes, the P600 reflects late syntactic reanalysis, and the CPS reflects aspects of prosodic processing. The neural generators of these components are discussed as well, both in the context of available brain imaging data and ERPs from lesion patient studies.

Introduction
Our understanding of the neural basis of language has made considerable progress over the past few decades. Although the language-brain relationship was originally described on the basis of studies correlating language behavior with brain lesions, the advent of new methods has allowed us to functionally specify particular brain areas within a neural network and also to describe the temporal dynamics of the activations within the network. Brain imaging methods such as functional magnetic resonance imaging and positron emission tomography enable scientists to identify brain areas that are active during language processing. Based on these methods, a fairly complete description of the neural network underlying language processing has been provided [1,2]. Language processing, however, consists of various subprocesses that specialize in areas such as phonologic analysis, meaning extraction, and the build-up of grammatical relations that have to be well coordinated in time in order to achieve comprehension. Although they are high in spatial resolution, the functional magnetic resonance imaging and positron emission tomography methodologies cannot capture language processing as it unfolds over time, millisecond by millisecond. As the different subprocesses take place in a partly cascading and partly parallel fashion, a second approach is needed that enables an adequate description of the neuro-

dynamics underlying language processing. Methods able to register the brain's activity millisecond by millisecond are electroencephalography (EEG) and magnetoencephalography (MEG). These methods measure the postsynaptic neural activity of large neural populations in the millisecond range. Such neural activity, when time-locked to particular stimulus events, is referred to as event-related brain potentials (ERPs) or event-related magnetic fields. The localization of the activity registered by EEG and MEG cannot be determined univocally because the signal is measured at the scalp. The source of activation, therefore, is calculated by means of mathematical models. The mathematically best solution, however, may not always be the best physiologic solution, in particular when two sources are spatially close. Thus, neurotopographic information from fMRI studies may be used to physiologically constrain the MEG dipole modeling [3]. Another way to bring together the temporal and spatial parameters of brain activation is to apply ERPs and eventrelated magnetic fields to patients with circumscribed brain lesions. The presence or absence of a particular ERP component allows us to make some conclusions about the source of the component and the linguistic process it reflects [4,5].

Language-related ERP Components

Within the language domain, four different ERP components have been identified, each reflecting a particular subprocess. Two of these components are correlated with syntactic processes, one with semantic processes, and one with prosodic processes.

Lexicalsemantic processes The N400 component The first component found to reflect a particular linguistic subprocess is the N400, a negative waveform peaking at around 400 ms after the onset of the critical stimulus. This component has been identified to be correlated with lexicalsemantic processes. Kutas and Hillyard [6,7] observed such a negative wave for sentence-final words that mismatched the preceding context semantically (eg, he spread the warm bread with socks). This N400 component is broadly distributed over the right and the left hemisphere, and often larger over the right hemisphere when words are presented visually. When words are

Event-related Brain Potential Studies in Language Friederici

467

presented auditorily, the N400 appears to be earlier and its distribution is more symmetric, sometimes even larger over the left than over the right hemisphere [8]. The N400 has been taken to reflect lexicalsemantic processes [6]. The N400 component has been found in different languages including English, French, Dutch, German, Hebrew, and even American Sign Language [9]. When words appear in sentential context, the amplitude of the N400 component varies inversely with the semantic expectancy of a word in a given context [7]. An N400, however, is not only observed in sentential context, but also when a word is presented in the context of another word (ie, two words are presented one after the other as a prime-target pair). The amplitude of the N400 is larger when primes and targets are semantically unrelated than when semantically related [8,10,11]. Moreover, it has been shown that the amplitude of the N400 also varies as a function of a word's frequency in general language use [12]. The question of whether the N400 reflects processes of lexical access or rather processes of lexical integration was investigated in a semantic word priming experiment. Chwilla et al. [13] presented word pairs that were either semantically related or not. In a decision task, participants had to decide either whether the letter sequence presented was a word or not, or whether it was written in small or capital letters. The N400 for the target was reduced as a function of semantic relatedness in the lexical decision task, but not in the physical judgment task. From these findings, it was concluded that the N400 reflects aspects of lexical integration. There is, however, evidence from semantic priming studies that suggest that the N400 does reflect some aspects of automatic spreading activation as well as meaning integration [1416]. Localizing the N400 generators There has been discussion as to which brain areas generate the N400 component. Given the finding of left hemispheric involvement for lexicalsemantic processes from lesion and brain imaging studies, the finding that the N400 is mostly distributed over the posterior portion of the right hemisphere appears counterintuitive. However, MEG studies using equivalent current dipole modeling approaches suggest that the N400 is more likely generated bilaterally in the vicinity of the auditory cortex [17,18]. Data from a study with intracortical recordings suggest that the superior temporal sulcus and additional frontal areas are involved in the generation of the N400 [19]. This is in partial agreement with brain imaging studies that report activation in the left superior temporal gyrus (STG), posterior and anterior to the Heschl's gyrus [1], for lexicalsemantic processes. The left inferior frontal gyrus only appears to come into play when strategic semantic tasks are performed [20,21].

hension. First, syntactic information is crucial at an initial processing phase during which the incoming information is structured into phrases on the basis of word category information (eg, noun, verb, and so forth) [22,23]. Second, after the initial local phrase structure is built, relations between phrases need to be established in order to identify who is doing what to whom. Whenever the words in the sentence are not in their canonical order (eg, object-first sentences, center-embedded sentences) different markers (eg, subjectverb agreement, case marking) are used for the assignment of grammatical (ie, thematic) roles. Third, syntactic processes come into play again at a final phase during which structural, lexicalsemantic, and thematic information have to be integrated to achieve comprehension. Each of these syntactic subprocesses leaves its traces in the ERP. The ELAN component An early left anterior negativity (ELAN) has been found to correlate with early structure-building processes. An early ERP study investigated the processing of syntactic violations in a number of different sentence structures during reading [24]. The violation of phrase structure (eg, Max's of proof the theorem) elicited a left anterior negativity (around 125 ms) that was followed by a left temporoparietal negativity between 350 and 500 ms. In this study, the stimulus material was presented visually, but in a rapid serial visual mode. A more recent study using a similar visual presentation mode to investigate phrase structure violations in German found an ELAN between 100 and 200 ms, but only when stimulus items were presented in a high visual contrast condition [25]. Studies using a visual presentation technique with long pauses between words did not find negativities in the early time domain, but rather a LAN between 300 and 700 ms [26]. An ELAN between 100 and 200 ms, however, has been reported for phrase structure violations in the auditory domain. Friederici et al. [27] and Hahne and Friederici [28] presented syntactically correct and incorrect sentences as connected speech. Syntactic incorrectness was realized as a word category violation (eg, Der Freund wurde im besucht/ The friend was in the visited). In German, the case-marked preposition im necessarily requires a noun phrase to follow. In the first [27] study, an ELAN present at around 180 ms was followed by a second negativity between 300 and 500 ms. The ELAN was interpreted to reflect highly automatic processes of initial structure building (ie, so-called first pass parsing processes) [23]. Support for this proposal was provided by a study that found the ELAN to be unaffected by attentional factors [28]. The observed independence of the ELAN from attentional variation indicates that the processes reflected by the ELAN are indeed highly automatic. The literature reviewed so far may suggest that negativities elicited by phrase structure violations always have a short latency after stimulus onset as long as the input is fast. However, languages differ with respect to where in the word

Syntactic processes Syntactic processes are multilayered because syntax becomes relevant at several different phases during language compre-

468

Behavior

the critical word category information becomes available. The latency of the ELAN, therefore, should be independent of when this information becomes available ( ie , early or late in the word). This was confirmed in a study by Friederici et al. [29] in which the phrase structure violation was realized by words in which the word category information was marked in the suffix (eg, the verb refined versus the noun refinement). The observed LAN was late when measured from the word onset, but early when measured from the word category identification point (ie, at the end of the word stem). The LAN component A number of studies have investigated morpho-syntactic aspects relevant for the identification of the grammatical relation between words and found a LAN between 300 and 500 ms. Most studies used a word-by-word visual presentation paradigm. The violation types investigated in the studies cluster around two phenomena: agreement information (mostly subject-verb agreement) and verb-argument structure information. Agreement violations were investigated in English [7], Dutch [30,31], and German [32]. With the exception of one study [30], all reported a LAN that was followed by a late positivity. Most of these negativities displayed a centrofrontal or frontal maximum, often with a left dominance. A similar pattern was also found for inflectional errors in an auditory sentence comprehension study in which the violation was not realized as a violation of tense and mode [27]. The combined data suggest that morphosyntactic violations realized as an incongruency of inflection elicit a LAN followed by a late positivity, independent of the input modality. Violations of verb-argument structure are also correlated with a LAN [33]. A similar LAN followed by a late positivity was reported by Coulson et al. [34] for incorrectly case-marked elements causing a mismatch between the verb's argument structure and its argument. In more recent studies, however, an N400 component was observed systematically in German sentences with case marking errors that signal incorrect thematic and thereby semantic roles [35]. It can be concluded that in languages that mark case unambiguously, thematic roles can be derived directly from the case information itself. Bornkessel et al. [36] proposed that the parser implements two parallel processing routes: one route that considers case and maps thematic roles directly whenever possible, and one route that considers other morphosyntactic information (eg, subject-verb agreement) and is used whenever no case information is available. The P600 component A late syntax-related ERP component, a positive wave after 600 ms (P600), has been observed with violations of structural preferences, outright syntactic violations, and difficulty of syntactic integration.

Violations of structural preferences Such violations can be realized in temporarily ambiguous structures that are disambiguated at some point in the sentence to mean the nonpreferred reading. Osterhout and Holcomb [37] reported a late centro-parietal positivity around 600 ms for so-called garden-path sentences (eg, The broker persuaded to sell the stock) at the disambiguating element (in this example, the word to), which indicates that the underlying structure of the sentences is not a simple subject-verb-object structure. This positivity was labeled the P600 component. From this and other studies [38,39], Osterhout et al. [39] developed the view that the P600 is a marker of the garden-path effect and present whenever the parser has to revise a structure. Thus, the P600 may be considered to reflect processes of structural reanalysis. Friederici and Mecklinger [40] formulated the assumption that the latency of the positivity may vary as a function of difficulty to recover from a garden-path. They found a very early positivity (P345) for easy-to-revise object relatives, but a later positivity (P600) for difficult-to-revise object-first complement sentences [41]. An alternative interpretation of the early positivity (P345) versus the late positivity (P600) on the basis of more recent data, however, is given by Bornkessel et al. [36] who interpret the P345 to reflect thematic reanalysis and the P600 to reflect structural reanalysis. Outright syntactic violations There are quite a number of ERP studies in different languages investigating the processing of syntactic violations and thus, processes of syntactic repair. Most of these report a biphasic pattern with a P600 following a LAN. The P600 has been found to co-occur with a variety of syntactic anomalies [27,30,39,4143], but also with outright phrase structure violations [24,40], subjacency violations [24,44] and agreement violations [27,30,31,34]. Syntactic integration The function of the P600 was further investigated by Kaan et al. [45]. They constructed sentences that varied in the difficulty of integration while keeping all other aspects constant and found a P600 for the difficult-to-integrate element. On the basis of this finding they argued that the P600 is a marker for syntactic integration difficulty. The neural basis of syntax-related ERP components One method used to study the neural basis of the ELAN and the P600 has been the application of the ERP method to patients with particular brain regions. The ELAN was found selectively absent in patients with lesions in the left anterior cortex [46] and those with lesions in the anterior temporal lobe [5], but present in those with lesions in the left basal ganglia [46]. In contrast, the P600 was selectively reduced or absent in patients with lesions in the basal ganglia [42,46,47]. These data suggest that the early structure-building processes

Event-related Brain Potential Studies in Language Friederici

469

are supported by a temporo-frontal network involving the left anterior portion of the STG and the left inferior frontal gyrus, whereas late processes of syntactic repair seem to involve the basal ganglia. The localization of the ELAN found support in a study using the method of dipole modeling of MEG data [2].

elicited by IPh. These latter processes, which appear to be subserved by a right hemispheric network, interact early with syntactic processes during auditory comprehension.

References and Recommended Reading

Papers of particular interest, published recently, have been highlighted as: Of importance Of major importance
1. Bookheimer S: Functional MRI of language: new approaches to understanding the cortical organization of semantic processing. Annu Rev Neurosci 2002, 25:151188. Friederici AD: Towards a neural basis of auditory sentence processing. Trends Cogn Sci 2002, 6:7884. Friederici AD, Wang Y, Herrmann CS, et al.: Localization of early syntactic processes in frontal and temporal cortical areas: a magnetoencephalographic study. Hum Brain Mapp 2000, 11:111. Friederici AD, Kotz SA: The brain basis of syntactic processes: functional imaging and lesion studies. Neuroimage, Special Issue 2003, 20:S8S17. Kotz SA, Friederici AD: Electrophysiology of normal and pathological language processing. J Neurolinguistics 2003, 16:4358. Kutas M, Hillyard SA: Reading senseless sentences: brain potentials reflect semantic incongruity. Science 1980, 207:203205. Kutas M, Hillyard SA: Event-related brain potentials to grammatical errors and semantic anomalies. Memory Cognit 1983, 11:539550. Holcomb PJ, Neville HJ: Semantic priming in visual and auditory language lexical decision: a between modality comparison. Lang Cogn Proc 1990, 5:281312. Van Petten C: Words and sentences: event-related brain potential measures. Psychophysiology 1995, 32:511525. Bentin S, McCarthy G, Wood CC: Event-related potentials, lexical decision and semantic priming. Electroenceph Clin Neurophysiol 1985, 60:343355. Rugg MD: The effects of semantic priming and word repetition on event-related potentials. Psychophysiology 1985, 22:642647. Van Petten C, Kutas M: Interactions between sentence context and word frequency in event-related brain potentials. Memory Cognit 1990, 18:380393. Chwilla DJ, Brown C, Hagoort P: The N400 as a function of level processing. Psychobiology 1995, 32:274285. Besson M, Kutas M, Van Petten C: An event-related potential (ERP) analysis of semantic congruity and repetition effects in sentences. J Cogn Neurosci 1992, 4:132149. Deacon D, Hewitt S, Yang CM, Nagata M: Event-related potential indices of semantic priming using masked and unmasked words: evidence that the N400 does not reflect a post-lexical process. Cognit Brain Res 2000, 9:137146. Holcomb PJ: Automatic and attentional processing: an eventrelated brain potential analysis of semantic priming. Brain Lang 1988, 35:6685. Halgren E, Dhond P, Christensen N, et al.: N400-like magnetoencephalography responses modulated by semantic context, word frequency, and lexical class in sentences. Neuroimage 2002, 17:11011116. Helenius P, Salmelin R, Service E, Connolly JF: Distinct time courses of word and context comprehension in the left temporal cortex. Brain 1998, 121:11331142. Halgren E, Baudena P, Heit P, et al.: Spatio-temporal stages in face and word processing. I. Depth-recorded potentials in the human occipital, temporal and parietal lobes. J Physiol Paris 1994, 88:150.

Prosodic processes The CPS component During comprehension of a spoken sentence, the listener not only relies on semantic and syntactic information, but also on suprasegmental phonologic information (ie, prosody or sentence melody). Prosody indicates not only which element is stressed, but also marks intonational phrase boundaries (IPh). Functionally, each IPh is a syntactic boundary (though not vice versa) and, therefore, particularly relevant for sentence parsing. Physically, each IPh is realized by variation in the pitch contour (F0) along with a pause and a lengthening of the syllable prior to the pause. The processing of IPhs has been found to correlate with a specific ERP component, a positive shift following the end of an IPh called the Closure Positive Shift (CPS) [48]. It has been shown that the CPS is present even when the pause at the IPh is deleted and the boundary is only indicated by the F0 variation and the prefinal lengthening [48]. Moreover, it has also been demonstrated that the CPS can be elicited even in the absence of segmental information (ie, in hummed sentences) [49]. This is strong evidence that the CPS is a reflection of purely prosodic processes. The data concerning the possible neural source of the CPS are sparse. There is, however, some indication from functional magnetic resonance imaging studies that prosodic information is processed in a network in the right hemisphere, comprising temporal and inferior frontal regions [50].

2. 3.

4.

5.

6.

7.

8.

9. 10.

11.

12.

Conclusions
Event-related brain potential studies involving both healthy participants and patients with specific brain lesions allow the following conclusions to be drawn with respect to the language-brain relationship. Processes of semantic integration take place around 400 ms after the word onset (N400) and appear to be supported by temporal brain regions. Syntactic processes are functionally subdivided into different phases. First, initial phrase structure-building processes around 200 ms (ie, ELAN) are supported by the anterior portion of the left STG and the frontal operculum. Second, left processes of assigning the grammatical relations between words and their thematic roles are reflected in a LAN between 300 and 500 ms. Later processes of syntactic integration and repair are reflected by the P600 component present beyond 600 ms. Prosodic processes, which are most relevant during spoken sentence comprehension, are correlated with a CPS

13. 14.

15.

16.

17.

18.

19.

470

Behavior

Thompson-Schill SL, DEsposito M, Aguirre GK, Farah MJ: Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. Proc Natl Acad Sci U S A 1997, 94:1479214797. 21. Fiez JA: Phonology, semantics, and the role of the left inferior prefrontal cortex. Hum Brain Mapp 1997, 5:7983. 22. Frazier L: Sentence processing: a tutorial review. In Attention and Performance 12: The Psychology of Reading. Edited by Hove CM. United Kingdom: Erlbaum; 1987:559586. 23. Friederici AD: The time course of syntactic activation during language processing: A model based on neuropsychological and neurophysiological data. Brain Lang 1995, 50:259281. 24. Neville HJ, Nicol J, Barss A, et al.: Syntactically based sentence processing classes: evidence from event-related brain potential. J Cogn Neurosci 1991, 3:151165. 25. Gunter TC, Friederici AD, Hahne A: Brain responses during sentence reading: visual input affects central processes. Neuroreport 1999, 10:31753178. 26. Mnte TF, Heinze HJ, Mangun GR: Dissociation of brain activity related to syntactic and semantic aspects of language. J Cogn Neurosci 1993, 7:3350. 27. Friederici AD, Pfeifer E, Hahne A: Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. Cognit Brain Res 1993, 1:183192. 28. Hahne A, Friederici AD: Electrophysiological evidence for two steps in syntactic analysis: Early automatic and late controlled processes. J Cogn Neurosci 1999, 11:194205. 29. Friederici AD, Hahne A, Mecklinger A: The temporal structure of syntactic parsing: Early and late event-related brain potential effects elicited by syntactic anomalies. J Exp Psychol Learn Mem Cogn 1996, 22:12191248. 30. Hagoort P, Brown C., Groothusen J: The syntactic positive shift as an ERP measure of syntactic processing. Lang Cogn Processes 1993, 8:439483. 31. Gunter TC, Stowe LA, Mulder G: When syntax meets semantics. Psychophysiology 1997, 34:660676. 32. Penke M, Weyerts H, Gross M, et al.: How the brain processes complex words: An event-related potential study of German verb inflections. Cognit Brain Res 1997, 6:3752. 33. Rsler F, Friederici AD, Ptz P, Hahne A: Event-related brain potentials while encountering semantic and syntactic constraint violations. J Cogn Neurosci 1993, 5:345362. 34. Coulson S, King JW, Kutas M: Expect the unexpected: eventrelated brain response to morphosyntactic violations. Lang Cogn Processes 1998, 13:2158. 35. Frisch S, Schlesewsky M: The N400 reflects problems of thematic hierarchizing. Neuroreport 2001, 12:33913394. This report indicates for the first time that the N400 does not only reflect semantic processes, but that a similar ERP pattern is observed for problems in hierarchizing.

20.

Bornkessel I, Schlesewsky M, Friederici AD: Beyond syntax: Language-related positivities reflect the revision of hierarchies. Neuroreport 2002, 13:361364. 37. Osterhout L, Holcomb PJ: Event-related potentials and syntactic anomaly. J Memory Lang 1992, 31:785804. 38. Osterhout L, Holcomb PJ: Event-related potentials and syntactic anomaly: Evidence of anomaly detection during the perception of continuous speech. Lang Cogn Processes 1993, 8:413437. 39. Osterhout L, Holcomb PJ, Swinney DA: Brain potentials elicited by garden-path sentences: Evidence of the application of verb information during parsing. J Exp Psychol Learn Mem Cognition 1994, 20:786803. 40. Friederici AD, Mecklinger A: Syntactic parsing as revealed by brain responses: first-pass and second-pass parsing processes. J Psycholinguist Res 1996, 25:157176. 41. Mecklinger A, Schriefers H, Steinhauer K, Friederici AD: Processing relative clauses varying on syntactic and semantic dimensions: an analysis with event-related potentials. Memory Cognit 1995, 23:477494. 42. Friederici AD, Kotz SA, Steinhauer K, von Cramon DY: The neural basis of the prosody-syntax interplay: the role of the corpus callosum. Brain Lang 2003, 87:133134. 43. Osterhout L, Holcomb PJ: Event-related brain potentials elicited by syntactic anomaly. J Memory Lang 1992, 31:785806. 44. McKinnon R, Osterhout L: Constraints on movement phenomena in sentence processing: evidence from event-related brain potentials. Lang Cogn Processes 1996, 11:495523. 45. Kaan E, Harris A, Gibson E, Holcomb P: The P600 as an index of syntactic integration difficulty. Lang Cogn Processes 2000, 15:159201. 46. Friederici AD, von Cramon DY, Kotz SA: Language related brain potentials in patients with cortical and subcortical left hemisphere lesions. Brain 1999, 122:10331047. 47. Frisch S, Kotz SA, von Cramon DY, Friederici AD: Why the P600 is not just a P300: the role of the basal ganglia. Clin Neurophysiol 2003, 114:336340. 48. Steinhauer K, Alter K, Friederici AD: Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nat Neurosci 1999, 2:191196. 49. Pannekamp A, Toepel U, Alter A, et al.: Prosody driven sentence processing: an ERP study. J Cogn Neurosci 2004, In press. This paper demonstrates that the CPS reflects the processing of IPhs, even in the absence of lexical information. 50. Meyer M, Alter K, Friederici AD, et al.: FMRI reveals brain regions mediating slow prosodic modulations in spoken sentences. Hum Brain Mapp 2002, 17:7388. This fMRI study provides evidence that the suprasegmental prosodic processes are supported by the right hemisphere.

36.