Vous êtes sur la page 1sur 283

Media Borders, Multimodality and Intermediality

This page intentionally left blank


Media Borders,
Multimodality and
Intermediality
Edited by

Lars Elleström
Editorial Board: Jørgen Bruhn, Siglind Bruhn, Claus Clüver,
Christina Ljungberg, Silvestra Mariniello, Jürgen E. Müller
and Valerie Robillard
Introduction, selection and editorial matter © Lars Elleström 2010
Individual chapters © contributors 2010
All rights reserved. No reproduction, copy or transmission of this
publication may be made without written permission.
No portion of this publication may be reproduced, copied or transmitted
save with written permission or in accordance with the provisions of the
Copyright, Designs and Patents Act 1988, or under the terms of any licence
permitting limited copying issued by the Copyright Licensing Agency,
Saffron House, 6-10 Kirby Street, London EC1N 8TS.
Any person who does any unauthorized act in relation to this publication
may be liable to criminal prosecution and civil claims for damages.
The authors have asserted their rights to be identified
as the authors of this work in accordance with the Copyright,
Designs and Patents Act 1988.
First published 2010 by
PALGRAVE MACMILLAN
Palgrave Macmillan in the UK is an imprint of Macmillan Publishers Limited,
registered in England, company number 785998, of Houndmills, Basingstoke,
Hampshire RG21 6XS.
Palgrave Macmillan in the US is a division of St Martin’s Press LLC,
175 Fifth Avenue, New York, NY 10010.
Palgrave Macmillan is the global academic imprint of the above companies
and has companies and representatives throughout the world.
Palgrave® and Macmillan® are registered trademarks in the United States,
the United Kingdom, Europe and other countries.
ISBN-13: 978–0–230–23860–2 hardback
This book is printed on paper suitable for recycling and made from fully
managed and sustained forest sources. Logging, pulping and manufacturing
processes are expected to conform to the environmental regulations of the
country of origin.
A catalogue record for this book is available from the British Library.
A catalog record for this book is available from the Library of Congress.
10 9 8 7 6 5 4 3 2 1
19 18 17 16 15 14 13 12 11 10
Printed and bound in Great Britain by
CPI Antony Rowe, Chippenham and Eastbourne
Contents

List of Figures vii


Acknowledgements viii
Notes on Contributors ix

Introduction 1
Lars Elleström

Part I Media, Modalities and Modes


1 The Modalities of Media: A Model for Understanding
Intermedial Relations 11
Lars Elleström

Part II Media Borders of Qualified Media


2 Border Talks: The Problematic Status of Media Borders in the
Current Debate about Intermediality 51
Irina O. Rajewsky

3 Intermedial Topography and Metaphorical Interaction 69


Axel Englund

4 Intermedial Strategies in Multimedia Art 81


Christina Ljungberg

Part III Combinations and Integrations of Media


5 ‘Media’ before ‘Media’ were Invented: The Medieval Ballad
and the Romanesque Church 99
Sigurd Kværndrup

6 The Intermediality of Field Guides: Notes Towards a Theory 111


Håkan Sandgren

7 Media on the Edge of Nothingness: Visual Apostrophes in


Lettrism 124
Sami Sjöberg

v
vi Contents

Part IV Mediations and Transformations of Media


8 Penrose, ‘Seeing is Believing’: Intentionality, Mediation and
Comprehension in the Arts 137
Siglind Bruhn
9 Beyond Definition: A Pragmatic Approach to Intermediality 150
Valerie Robillard
10 Translating Sounds: Intermedial Exchanges in Amy Lowell’s
‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’ 163
Regina Schober
11 ‘Transgenic Art’: The Biopoetry of Eduardo Kac 175
Claus Clüver
12 Photo/graphic Traces in Dubravka Ugrešić’s The Museum of
Unconditional Surrender 187
Katalin Sándor
13 The Dance of Intermediality: Attempt at a Semiotic Approach
of Medium Specificity and Intermediality in Film 199
Hajnal Király
14 Media in the Cinematic Imagination: Ekphrasis and the
Poetics of the In-Between in Jean-Luc Godard’s Cinema 211
Ágnes Pethő

Part V The Borders of Media Borders


15 Heteromediality 225
Jørgen Bruhn
16 Intermediality Revisited: Some Reflections about Basic
Principles of this Axe de pertinence 237
Jürgen E. Müller

Index 253
List of Figures

1 The modalities and modes of media 36


2 The first measures of Maurice Ravel’s piano piece Gaspard
de la nuit 73
3 Lucia Leão, Hermenetka (2005–2007): ‘What is the
Mediterranean for you?’ 92
4 The ballad dance sublime; the church of Ørslev, Denmark,
ca. 1300 100
5 An excerpt of Alain Satié’s ‘Sur le pont d’Avignon’ 126
6 The ‘musical icon’ of John Tavener’s Mary of Egypt 140
7 John Tavener, Mary of Egypt: schematic representation of the
hand bells of ‘unearthly stillness’ 141
8 Messiaen’s favourite vertically-symmetric chord: the
generation of its spatial pattern and its distribution
on a keyboard 147
9 The differential model 152
10 Calum Colvin, Fragments of Ancient Poetry IV 155
11 Eduardo Kac, Genesis, 1999. Transgenic work with
artist-created bacteria, ultraviolet light, Internet, video (detail) 177
12 Greimas’s semiotic square of cognitive modalities, from
A. Greimas (1987) 206
13 Frames from Satan’s Tango by Béla Tarr (1994) 207
14 Fernsehstube, hall and audience 238
15 A ‘married couple’ in First and Second Life 249

vii
Acknowledgements

The editor and publishers wish to thank the following for permission to
reproduce copyright material:

S. Fischer Verlag for Paul Celan, ‘Anabasis’, from Paul Celan, Die
Niemandsrose,  c 1963 S. Fischer Verlag GmbH, Frankfurt am Main
Calum Colvin for his artwork Fragment IV from the series Ossian, Fragments
of Ancient Poetry (2002)
Eduardo Kac for his artwork Genesis (1999)
Lucia Leão for her internet art work Hermenetka (2005–2007)
Alain Satié for his poem ‘Sur le pont d’Avignon’ (1970)
The journal Amerikastudien for parts of Regina Schober, ‘Amy Lowell’s
Peasant Dance’, Amerikastudien 53(2) (2008)

Every effort has been made to trace rights holders, but if any have been inad-
vertently overlooked the publishers would be pleased to make the necessary
arrangements at the first opportunity.
Unless otherwise noted, all translations are by the authors of the essays.

viii
Notes on Contributors

Jørgen Bruhn is Assistant Professor in the Department of Comparative Liter-


ature, Linnaeus University, Sweden. He has published books and articles on
Marcel Proust, M. M. Bakhtin, medieval literature, the history of the novel
and the theory and practice of intermedial studies.

Siglind Bruhn is a musicologist, concert pianist and interdisciplinary


scholar. A full-time researcher at the University of Michigan’s Institute for
the Humanities and a former guest researcher at the Sorbonne’s Institute
for the Aesthetics of Contemporary Arts, she has authored thirty book-
length monographs, primarily in the field of twentieth-century music and
its relationship with literature, art and religion (her most recent publications
in English include a book trilogy on Olivier Messiaen’s musico-symbolic
language). She is a co-editor of the Pendragon Press book series Interplay:
Music in Interdisciplinary Dialogue. In 2001 she was elected to the European
Academy of Arts and Sciences; in 2008 she received an honorary doctorate
from Växjö University, Sweden.

Claus Clüver is Professor Emeritus of Comparative Literature, Indiana Uni-


versity. He has also taught at New York University and in Germany, Sweden,
Denmark and repeatedly in Portugal and Brazil. His publications include
over 30 essays on intermediality and interarts studies, especially on concrete
and visual poetry, intersemiotic transposition, ekphrasis and representation
in the arts. He is co-editor of The Pictured Word (1998), Signs of Change:
Transformations of Christian Traditions and their Representation in the Arts,
1000–2000 (2004), Orientations: Space / Time / Image / Word (2005) and Inter-
midialidade (Belo Horizonte, Brazil, 2006). In 2003 he received an honorary
doctorate from Lund University, Sweden.

Lars Elleström is Professor of Comparative Literature, Linnaeus University,


Sweden. He organizes the Forum for Intermedial Studies, Linnaeus Univer-
sity, and chairs the board of the Nordic Society for Intermedial Studies.
Elleström has written and edited several books, including Divine Madness:
On Interpreting Literature, Music, and the Visual Arts Ironically (Bucknell Uni-
versity Press, 2002). He has also published numerous articles on poetry,
intermediality, gender and irony.

Axel Englund is a Doctoral Candidate in Comparative Literature at


Stockholm University, where he also teaches metrics and modernist exile
literature. He has previously taught music theory and computer notation

ix
x Notes on Contributors

at Lund University. Englund holds degrees in Musicology (BA, Lund


University), Comparative Literature (MA, Lund University) and Composi-
tion (MFA, Lund University), and in 2009 he held a visiting scholarship at
Columbia University. He is currently working on a dissertation dealing with
music in and around the poetry of Paul Celan and has recently published
articles on this topic in Seminar: A Journal of Germanic Studies and Perspectives
of New Music.

Hajnal Király is a freelance film critic, presently translator at the Cardiff


(UK) office of Thomson Reuters news agency. Her PhD dissertation deals with
alternative discourses of the film–literature relationship, her wider research
interests being medium theory, intermediality and the issues of a suit-
able approach to painting–film–literature interaction. Her publications have
appeared in English and in Hungarian, several of them in essay collections
on intermediality edited by Ágnes Pethő. Additionally, she has published
articles in film periodicals in both Romania and Hungary.

Sigurd Kværndrup was educated in Danish and theology at the Univer-


sity of Copenhagen where he taught history of Danish literature from 1969
to 2006. Now he is working as lecturer at the gymnasium in Nyköbing
Falster and as researcher at the Linnaeus University, Sweden. He was head
of Skaelskoer Folk High School between 1984 and 1990. In 1996 he earned
his PhD at the University of Southern Denmark, Odense, with the disser-
tation, ‘Twelve Principles in Saxo: An Analysis of Gesta Danorum’, printed
in Copenhagen 1999, and he received the degree of D Phil with the the-
sis, The East-Nordic Ballad: Oral Theory and Textual Analysis, Copenhagen
2006. Kværndrup was co-editor of the Anthology of Northern Literature 1–11
(1972–1983) and editor of The History of Danish Literature 1–9 (1984–1985).

Christina Ljungberg is Adjunct Professor of English and American Liter-


ature at the University of Zurich. Her research interests focus on visuality
and narrativity, intermediality, and iconicity. She is the author and editor
of To Fit, to Join, and to Make, The Crisis of Representation: Semiotic Founda-
tions and Manifestations in Culture and the Media (with W. Nöth) and Insistent
Images (with E. Tabakowska and O. Fischer). She has just completed Redefin-
ing Literary Semiotics (with J. D. Johansen and H. Veivo) and Signergy (with
M. Beukes, J. Conradie and O. Fischer) and is currently working on her book
Creative Dynamics: Diagrammatic Strategies in Narrative.

Jürgen E. Müller is Professor and Chair of the Media Department at the


University of Bayreuth, heading Media Lab and Campus TV projects. He has
a background in literary studies, sociology, linguistics and French studies,
and his research spans a number of areas including the history and theory of
modern media, intermediality, media culture, French cinema and reception
Notes on Contributors xi

theory. Among his publications, the book Intermedialität. Formen moderner


kultureller Kommunikation (1996) can be noted. He is one of the founders of
the CRI (International Center for Research on Intermediality). In 2005 he
organized, in cooperation with the CRI, the international congress ‘Media
Encounters’ at the University of Bayreuth.

Ágnes Pethő is Associate Professor at the Babeş-Bolyai University and the


Sapientia Hungarian University of Transylvania in Cluj-Napoca, Romania.
Her research interests include the relationship of cinema with the other
arts, cinematic intermediality and self-reflexivity. She is the editor of the
volume Words and Images on the Screen: Language, Literature, Moving Pictures,
published by Cambridge Scholars Publishing, 2008. Her other major publi-
cations in Hungarian include Múzsák tükre. Az intermedialitás és az önreflexió
poétikája a filmben [Mirror of the Muses: The Poetics of Intermediality and Self-
Reflexivity in Cinema, 2003] and the edited volumes of essays Képátvitelek.
Tanulmányok az intermedialitás tárgyköréből [Image Transfers: Essays on Inter-
mediality, 2002], Köztes képek. A filmelbeszélés színterei [Images In-Between:
Scenes of Filmic Narration, 2003] and Film. Kép. Nyelv [Film. Image. Language,
2007].

Irina O. Rajewsky is Associate Professor of Italian and French Literature at


the Freie Universität Berlin, associated with the Institute for Romance Lan-
guages and Literature and with the Friedrich Schlegel Graduate School of
Literary Studies. Her main areas of research are inter and transmediality,
literary theory (in particular narratology, intertextuality, fictionality, meta-
phenomena) and performativity. Currently she is conducting a research
project on ‘Mediality – Transmediality – Narration: Perspectives of a Trans-
generic and Transmedial Narratology (Film, Theatre, Literature)’. Her publi-
cations include Intermedialität (Tübingen, 2002) and Intermediales Erzählen
in der italienischen Literatur der Postmoderne (Tübingen, 2003); she is also
co-editor of Im Zeichen der Fiktion. Aspekte fiktionaler Rede aus historischer und
systematischer Sicht (Stuttgart, 2008).

Valerie Robillard is Associate Professor in the Department of English Lan-


guage and Culture at the University of Groningen. She has published
numerous articles in the area of Interarts Studies and co-edited Pictures Into
Words: Theoretical and Descriptive Approaches to Ekphrasis (Amsterdam, VU
Press, 1998). Her current research interests include intermediality in the per-
forming arts and in fashion and she is closely involved with the didactic
aspects of word and image studies in both academic and public contexts.

Håkan Sandgren is Senior Lecturer in Swedish and Comparative Literature


at Kristianstad University College, Sweden, where he teaches literature and
film and holds a position as Head of Teacher Staff. He is the author of
xii Notes on Contributors

Landskap på jorden och i drömmen. Studier i Folke Isakssons lyrik [Landscapes


on Earth and in Dreams: Studies in Folke Isaksson’s Poetry] (Lund University
Press, 1999), and was the first literary scholar to introduce ecocriticism and
green studies in Sweden. Sandgren is currently working on a book-length
study of Swedish ecopoetry in the twentieth century.

Katalin Sándor is currently Assistant Lecturer at the Faculty of Letters


of Babeş–Bolyai University, Cluj-Napoca, Romania. Her research interests
include theories of intermediality, visual poetry, ekphrasis, photography
in literature and visual culture. She has published her studies in various
conference volumes and periodicals (Iskolakultúra, Magyar Műhely, Nyelv-
és Irodalomtudományi Közlemények, Studia Universitatis Babeş–Bolyai–European
Studies, Korunk, A Hét).

Regina Schober has completed her PhD thesis on intermediality in Amy


Lowell’s poetry at Leibniz University, Hanover, in 2009. Her research
interests include intermedial relations between music and literature and
American modernist poetry.

Sami Sjöberg is a researcher at the University of Helsinki in the field of


comparative literature, currently working in the research project ‘Literature,
Transcendence, Avant-Garde’, funded by the Academy of Finland. His main
research subject is the avant-garde movement Lettrism, on which he has pub-
lished numerous articles in Finnish literary journals and textbooks. His areas
of expertise are the contemporary philosophical and literary manifestations
of medieval cabbalism, visual poetry and the concept of nothingness.
Introduction
Lars Elleström

Let us take a look at something. Let us see what can be found in something
that could be anything, for instance, something like an X. What is it? Is it
a sign? Yes, at least it can be a sign. The moment we decide that it means
something, it becomes a sign. The easiest way to make the X mean some-
thing is to put it in some sort of context, for instance, a conventional sign
system: the Greek or the Latin alphabets, the sign systems of mathematics or
logic, or some other more or less settled scheme. We know what to do with
X in the word ‘mix’ and we know that X, in an equation such as ‘3 + X = 5’,
means the unknown number.
There are very many alphabets, systems and schemes, however, that can
make signs ‘readable’ and, furthermore, signs often become signs not in
determined sign systems but in contexts that are tied only marginally or
indirectly to the prevailing conventions. Perhaps it is not even a very good
idea to say that all signs are read, since the acts of interpretation are much
diversified. If we stubbornly persist in ‘taking a look’ at the X, we have to
admit that even a simple little sign such as this one leads a rather compli-
cated double or triple life. Actually, both well-defined and temporary signs,
as well as sign systems and specific media productions and works of art,
can be said to share a lot of properties with and, hence, also overlap with
other signs, systems and types. The general characteristics of X, shared by
innumerable other signs, can be summarized as follows.
It has a material interface. The reader of this Introduction is probably
confronted with some sort of flat surface: a paper, hopefully in a book, or
a computer screen. This kind of material interface allows for a variety of
appearing content. You can put almost anything in a book, as long as it
is flat and static, and you can put almost anything on a screen, including
things that move, as long as it is flat. This flat surface is, first and foremost,
seen by the reader, spectator or interpreter. Of course, it is not forbidden
to use the other senses as well when interacting with the displayed con-
tent of screens and pages, but in this case nothing very significant would be
added by way of, for instance, trying to taste the X that can be seen in this

1
2 Introduction

sentence. Nevertheless, many signs that are part of conventional systems


have sounds attached to them. When reading this English text, most people
form the sound ‘eks’, silently or aloud, when seeing the mark X. This sound
is, however, a secondary aspect since it is not part of the material interface
but a result of the interpretative interaction with it. In a Greek context, the
appearance of the visually identical symbol X would generally lead to the
sound ‘chi’ (pronounced as the German ich backwards).
Spatiotemporally, this text is static since it remains the same once it has
been mediated into a printed or electronically fixed product. It takes time
to read it, however, and the alphabetic form invites the reader to decode
it according to certain rules including, among other things, standards for
sequential reading. On the level of what it represents, this text also cre-
ates temporal relations. As regards my writing about primary and secondary
sensorial experiences, for example, it is implied that in the described, con-
ceptual ‘world’ that the linguistic expressions refer to, the visual comes first
and the aural at a later stage in the interpretative process. There is thus some
sort of virtual time in the description. The manner in which the equation
‘3 + X = 5’ is outlined also implies a temporal sequence mirrored by the
reading order: first Linda catches three fishes and then an unknown number
of fishes, and when she returns at the end of the day she has five fishes.
The X, as a written sign, is obviously spatial, both in itself and as part of
conventional sign systems or unfixed context. As a visual sign, it stretches
out in four directions with a junction in the middle and it is generally con-
ceived as a combination of two intersecting lines. Its spatiality is normally,
and certainly in the context of this Introduction, two-dimensional, but also
three-dimensional objects, such as Ronald Bladen’s minimalistic but never-
theless very large aluminium sculpture The X, can be seen as parts of the
wide sign type of X. X-like figures that we see in paintings, drawings and
photographs can acquire virtual, three-dimensional spatiality, but the X in
Bladen’s sculpture, as well as the letters of many alphabetical toy bricks, is
three-dimensional in itself. It must also be recognized that there is an impor-
tant aspect of spatiality involved in the activity of ‘taking a look’ at X. To take
a look at X means both actually to process the information perceived by our
photosensitive receptors and to form cognitive, spatial structures when con-
sidering the many aspects of what X can be understood to be and how this
being can be understood. To ‘see’ is to form mental structures in cognitive
space.
The X actually means nothing, however, until it is interpreted as some
kind of sign. When reading about ‘mixed vegetables’ in a cookbook most
people automatically and rationally see the little X as a conventional sign
that is part of a word with a specific meaning: blended, composed of dif-
ferent elements and so forth. If, in the same cookbook, the word ‘mixed’
is shaped by various vegetables with a stalk of asparagus and a stick of cel-
ery forming the X, there is an iconic element attached to the conventional
Lars Elleström 3

sign: the word ‘mixed’ is no longer arbitrarily connected to its meaning since
it actually also resembles what it represents: mixed vegetables. If the cap-
tion of an image of mixed vegetables forming the words ‘mixed vegetables’
says that ‘mixed vegetables are good for your health’ we assume that the
text, because of its closeness to the image, can be read as a part of what the
image expresses. Moreover, we can assume also that vegetables that are not

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
depicted, but are nevertheless closely related, are actually as healthy as the
ones depicted, or we can assume that the ones that occur more frequently are
actually healthier than the others. All in all, the interpretative acts that form
the meaning of this page in the cookbook, that is, the interpretative acts that
actually put the sign-functions into action, or even create the signs, can be
said to be based on convention, resemblance and contiguity.
We must now remember that the X is merely an example. It is something
that could be anything. Yet, considering its conventional attributes and sig-
nificance, it is a funny little sign. When we ponder on, say, Ronald Bladen’s
X or the many visually striking Xs in popular culture (graffiti, record sleeves,
movie posters and so forth), a whole range of potential meanings pops up:
the X can represent meetings and mixes, but it also implies erasure and the
forbidden. Nonetheless, X is also the sign of presence. It can mean that some-
thing is valid or present and it is the only alphabetic letter that may also be
written analphabetically? Even if you cannot ‘read’ the X, you can see its
iconic potential. The X always carries visual aspects that may be activated
as iconic significance as soon as the interpreter finds an opportunity. When
there is a point in it, the X looks like a pair of crossed fingers, branches or
vegetables, and since the wings of the windmill definitely look like an X,
the X also looks like the wings of a windmill. Ian Hamilton Finlay is said to
have written a poem called ‘The Windmill’s Song’ that reads like this: ‘X’.
The poem certainly represents the same kind of windmills that can be seen
in, say, many paintings by Jan Brueghel, both the elder and the younger. It
would thus be easy to put the wings into the sequence of conventionally
formed letters. Don Quijote, as we know, had windmills on his mind, and
if we spell his name in the original way, and add a slight emphasis by way
of capitalizing the crucial letter, we suddenly have something that might be
called a concrete poem: ‘Don QuiXote’. In this version of his name, we can
actually see, iconically, that he is full of windmills.
The X, whether seen as a letter, a conventional, non-alphabetical sign, a
poem, an image, a sculpture, a part of a linguistic sequence, a graphic mark
on a surface, a sound, a part of a work of art or a media production, or
a sign used in completion of the income-tax return, is obviously part of a
context where all ‘texts’ and ‘systems’, in the widest senses of the words,
overlap. It would be very wrong to say that all systems are the same; that
there is no difference at all between the way meaning is produced when we
find the X in an exhibition hall and in a form respectively, but if we are to
talk about borders we are better off talking about border zones rather than

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


4 Introduction

strictly demarcated borders. Hence, I would say that all kinds of sign systems
and also specific media productions and works of art must be seen as parts
of a very wide field including not least the material, sensorial, spatiotempo-
ral and semiotic aspects. In my essay in this volume, I call these the four
‘modalities’ of media. By investigating the modalities one clearly sees that
all forms of art, media, languages, communication and messages have some

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
characteristics in common which make it possible for something like the X
to hover between different systems and simultaneously be part of various
frameworks without losing its relative stability.
For various reasons it is convenient to talk about ‘media’ when discussing
these issues. The notion of a medium is fairly inclusive and offers a way to
bring together scholarly efforts within a considerable number of disciplines.
The phenomenon whereby the properties of all media partly intersect and
the study of this same phenomenon are called intermediality. As might be
expected, however, the term intermediality has been used in many ways that
are not always entirely compatible. Sometimes the study of intermediality is
seen as the study of a group of media and art forms that fall outside the
borders of the ‘normal’ or ‘established’ media. In this collection of essays,
however, most of the writers, at least most of the time, see intermedial-
ity as the precondition for all mediality. Even so, it may occasionally be
appropriate to highlight the fact that some media may be perceived as more
border-crossing than others.
Our small survey of the X has hopefully demonstrated that media actually
cross each other rather than border each other. Presumably we will never
stop talking about media borders, but in this volume they will be partly
crossed out.

Media Borders, Multimodality and Intermediality is a collection of essays that


have found their final forms in dialog with each other. The initial ques-
tion posed was simply ‘what is intermediality?’ To answer this question, one
must also ask what a medium is and where we find the ‘gaps’ that inter-
mediality bridges. Clearly, the supposedly crossed borders must be described
before one can proceed to the ‘inter’ of intermediality. We also set out to
investigate how the two notions of intermediality and multimodality, rarely
discussed together, are related. The aim was to facilitate communication and
theoretical cross-fertilization over the borders between the aesthetic disci-
plines, media and communication studies, linguistics, and so forth. That
turned out to be a lucky strike, I would say, but the edges of the concep-
tual tools had to be sharpened in order to work properly. Further questions
that we all had in common from the beginning, and that are clearly vis-
ible in many of the essays, concerned the relations between notions such
as technical medium, medium, art form and genre. We wanted to inves-
tigate what happens, from a historical and social point of view, when
new media and art forms emerge and are delimited and to determine the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 5

theoretical and practical implications of describing something as intermedial


or multimodal.
The essays thus have common starting-points and they have been influ-
enced by each other. We have sought to avoid confusing conflicts between
terminologies and the research angles are relatively compatible, but there
is no absolute harmony between the essays. They are written by individual

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
scholars who rarely agree on the best way to deal with tricky queries. They
are, however, focused on a coherent range of questions and topics that are
believed to be important.
Of course, the titles of the volume’s sections only partly mirror the con-
tent of the essays, but they might give a hint of the main focus. The first
essay, ‘The Modalities of Media: A Model for Understanding Intermedial
Relations’, is my own attempt to build a theoretical framework that explains
how all media are related to each other: what they have in common and
in which ways they differ. The key to this understanding is to be found in
the four modalities that were briefly mentioned earlier in this Introduction.
I also find it imperative to distinguish between three aspects of the notion of
medium. Basic media are simply defined by their modal properties whereas
qualified media are also characterized by historical, cultural, social, aesthetic
and communicative facets. Technical media are any objects, or bodies, that
‘realize’, ‘mediate’ or ‘display’ basic and qualified media.
Part II, ‘Media Borders of Qualified Media’, contains three essays. Irina
O. Rajewsky’s ‘Border Talks: The Problematic Status of Media Borders in the
Current Debate about Intermediality’ discusses the many critical approaches
that make use of the notion of intermediality. Rajewsky focuses on the
assumption of tangible borders between individual media and the recent
questioning of precisely this fundamental premise of discernible media bor-
ders. Yet, Rajewsky argues, media borders and medial specificities are of
crucial importance for art practice, and she specifies how media can be
conceived as distinct and how they actually come into play in concrete
intermedial practices. In ‘Intermedial Topography and Metaphorical Inter-
action’, Axel Englund also highlights the tendency to think of arts and
media in terms of geographic areas delineated by definable borders, and
consequently of intermedial studies as a kind of topographical description.
Focusing on relations between music and literature, his essay points to some
of the implications of this topographical model, and contrasts them with
another way of conceptualizing intermedial relations, namely as metaphor-
ical phenomena. Englund argues that many musico-literary artefacts can be
successfully read as a metaphorical interaction between their musical and
verbal elements. Christina Ljungberg’s semiotically-oriented essay ‘Inter-
medial Strategies in Multimedia Art’ proposes that intermediality either
concerns the transgression of the borders between conventionally distinct
media of communication or the iconic enactment of one medium within
another. Ljungberg argues that both of these instances of intermediality are

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


6 Introduction

highly performative, as we are confronted with hybrid forms that gener-


ate something new and unique, that they are strongly self-reflexive, since
they focus attention both on their own mode of production and on their
own semiotic character, and that they constitute a highly effective com-
munication strategy, as they give receivers access to different levels of
meaning.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Part III, ‘Combinations and Integrations of Media’, includes three essays
that mainly draw attention to media as multimodal products of combined
and integrated modalities and other media aspects. In ‘ “Media” before
“Media” Were Invented: The Medieval Ballad and the Romanesque Church’,
Sigurd Kværndrup approaches the medieval ballad as an intermedial art form
and proposes a new theory of one aspect of its origin. Kværndrup builds
on Marshall McLuhan’s theories about the development of ‘mass media’
during the Middle Ages and relates these to Romanesque church-building.
Håkan Sandgren’s ‘The Intermediality of Field Guides: Notes Towards a The-
ory’ is an essay informed by ecocriticism that investigates the field guide to
birds, a hitherto unexplored area for the scholar of intermediality. In the
field guide, descriptive prose is combined with images, maps and transcrip-
tions of birdsong. A particular form of transcription offers examples of how
the author tries to transcribe bird song to text, an endeavour that some-
times approaches the domain of concrete poetry. In Sami Sjöberg’s ‘Media
on the Edge of Nothingness: Visual Apostrophes in Lettrism’, the author is
interested in how the Lettrists set out to challenge the limits of media and
representation. The techniques applied by this avant-garde group resulted
in works blending poetry, narrative fragments and imaginary signs. Sjöberg
proposes two new concepts for analysis: the ‘visual apostrophe’ locates omis-
sions in the text, where invented signs either replace the omitted part of
the text or supplement the work in an ambiguous relation to language, and
‘meontologization’ refers to the dynamics of signification and nothingness
engaged in the artwork.
The volume’s fourth and most extensive part is called ‘Mediations and
Transformations of Media’ and it includes seven essays that deal with the
large and multifaceted field of media transformations. Beginning with the
model of the much-discussed relation between creator, work and beholder,
Siglind Bruhn suggests a triangular concept of expressive intent in her essay
‘Penrose, “Seeing is Believing”: Intentionality, Mediation, and Comprehen-
sion in the Arts’. Regarding the rarely considered discrepancy between the
human mind’s limited capacity to grasp the various media equally and
the near-unlimited scope of creative options, Bruhn draws attention to
areas where the chain of mediation is less than straightforward in cases of
both extreme and ostensibly quite accessible messages. Finally, she presents
an unusual three-tiered ‘transmedialization’ that proceeds from a visual
through a verbal and on to a musical representation. In ‘Beyond Definition:
A Pragmatic Approach to Intermediality’, Valerie Robillard argues that defi-
nitions, although essential in laying out common terms of discourse, do not

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 7

fully contribute to our understanding or articulation of the various types


and degrees of medial interaction. The purpose of her essay is to demon-
strate the need to employ ontological systems in the delineation of medial
types, systems that have proven indispensable to other disciplines (such as
the natural sciences and linguistics) in determining the relative positions of
concepts and categories with respect to one another. Robillard proposes the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
use of a pragmatic intertextual model to delineate and differentiate types of
medial interaction and to demonstrate the difference between the textual
aspects of intermediality (the ‘message’) and the medial aspects (‘materials’).
Regina Schober’s essay ‘Translating Sounds: Intermedial Exchanges in Amy
Lowell’s “Stravinsky’s Three Pieces ‘Grotesques’, for String Quartet” ’ calls
attention to notions such as ‘transcription’ and ‘intermedial translation’ that
imply dynamic processes of movement and transfer between medial bor-
ders. More specifically, Schober discusses a verbal representation of a musical
work by American modernist poet Amy Lowell as a paradigmatic example
of an experimental transformation of one medium into another. Following
Roman Jakobson’s term ‘intersemiotic translation’ as well as recent attempts
to broaden the terms ‘translation’ and ‘transcription’, she comprehends
Lowell’s poem as both a literary and a cultural translation of Stravinsky’s
Three Pieces for String Quartet from 1914. In ‘ “Transgenic Art”: The Biopo-
etry of Eduardo Kac’, Claus Clüver investigates a very different sort of media
transformation resulting in poetry: the ‘transgenic’ artwork Genesis (1999)
by the Brazilian poet and artist Eduardo Kac, who has radically explored the
possibilities of contemporary media technology for art-making. Genesis was
based on an artificial gene created from a phrase from the biblical Genesis
represented in Morse code that was then converted into a DNA sequence
according to a special code, mutated and eventually ‘retranslated’. In his
essay, Clüver sets out to explore the functions of the work, the demands it
makes on the receiver and its implications for the discourse on media and
intermediality and for the notion of ‘poetry’.
In Katalin Sándor’s essay, ‘Photo/graphic Traces in Dubravka Ugrešić’s
The Museum of Unconditional Surrender’, Ugrešić’s novel is interpreted as a
text very much concerned with, shaped by and disrupted into (textual)
photographs, albums, museums, archives, cards, collections and memories.
Sándor focuses mainly on those passages which deal with photographic rep-
resentation and practices, as well as with a specific photograph incorporated
both visually and textually into the exile-narrative. She looks for the medial
traces of photography and explains how the photographic medium becomes
a kind of apparatus for the textual process of remembrance. In ‘The Dance
of Intermediality: Attempt at a Semiotic Approach of Medium Specificity
and Intermediality in Film’, Hajnal Király also analyses a specific work of
art: Hungarian director Béla Tarr’s Satan’s Tango, a seven-and-a-half-hour
movie merging two cinematographic trends beginning with the 1990s that
have raised the problematic notions of medium specificity and intermedial-
ity in films: the so-called writer’s movies and the contemplative, extremely

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


8 Introduction

slow-paced movies, mostly coming from the Far East, defying all the com-
plex narratorial accomplishments of the film medium. Király argues that the
first kind of movie systematically overturns the strict delimitation between
literature and film, the idea of conceptuality of the first and visuality of the
latter, and that the second kind of movie continuously turns the running
time of the narrative into an almost static, ‘plastic’ visual work of art: a

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
picture. In an essay by Ágnes Pethő, ‘Media in the Cinematic Imagination:
Ekphrasis and the Poetics of the In-Between in Jean-Luc Godard’s Cinema’,
the work of Godard, another film director, is the centre of attention. Pethő
states that his films have long been associated with the idea of cinematic
intermediality and that both his fiction films and his cinematic essays can
be considered a sort of direct theory, or ‘archeology’ of cinema as a medium.
The essay attempts an application of the notion of ekphrasis, traditionally
understood as verbal descriptions of images, to the medium of cinema. A few
major conditions for the perception of cinematic ekphrasis are outlined and,
from the variety of intermedial relations in Godard’s films that can be called
ekphrastic, four types are charted and exemplified.
The fifth and last part of the volume is called ‘The Borders of Media
Borders’. The title is an attempt to capture the more general perspectives
introduced by the section’s two essays. In his contribution, ‘Heteromedial-
ity’, Jørgen Bruhn proposes a widened concept of intermediality constructed
on the principle of multimodality. He suggests we begin considering all cul-
tural texts as mixed media and that we start understanding the particular
constellations of modalities as tensional relationships in the sense that the
mixing of modalities, and consequently of media, always refers to a wider
historical and ideological context. An adoption of this heteromedial view-
point will, he argues, transfer studies of intermediality to the very centre
of future humanistic studies. In ‘Intermediality Revisited: Some Reflections
about Basic Principles of this Axe de pertinence’, Jürgen E. Müller asserts that
the research axis of intermediality actually keeps numerous scholars at uni-
versities and research centres all over the globe busy. The variety of aspects
of the concept of intermediality makes it very difficult or almost impossible
to present some sort of general overview with regard to all options, Müller
argues, without opening a sort of academic bookkeeper discourse on differ-
ent terminological, theoretical, methodological and historical items. Instead,
he develops some ‘aphorisms’ on the state of affairs of intermedial studies
and some perspectives for a historical approach. Müller is concerned with
the question of when a ‘new’ medium becomes a ‘new’ medium and, aided
by the test case of television, he discusses intermediality as a process. Fur-
thermore, the concept of intermediality is compared with intertextuality,
interartiality and hybridity. Finally Müller asks what the substance of inter-
medial studies actually is. Hopefully, not only Müller but also the volume as
a whole provides a good answer to that question.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Part I
Media, Modalities and Modes

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
This page intentionally left blank

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


1
The Modalities of Media: A Model for
Understanding Intermedial Relations

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Lars Elleström

What is the problem?

Scholars have been debating the interrelations of the arts for centuries. Now,
in the age of electronic and digital media, the focus of the argumentation
has somewhat shifted to the intermedial relations between various arts and
media. One important move has been to acknowledge fully the materiality of
the arts: like other media, they are dependent on mediating substances. For
this reason, there is a point in not isolating the arts as something ethereal but
rather in seeing them as aesthetically developed forms of media. Still, most of
the issues discussed within the interart paradigm are also highly relevant to
intermedial studies. One such classical locus of the interart debate concerns
the relation between the arts of time (music, literature, film) and the arts of
space (the visual arts). In the eighteenth century, Gotthold Ephraim Lessing
argued famously in Laocoön that there are, or rather should be, clear differ-
ences between poetry and painting,1 but for the moment there is a tendency
rather to deconstruct the dissimilarities of various arts and media. W. J. T.
Mitchell is perhaps the most influential contemporary critic of attempts
to find clear boundaries between arts and media. Many important distinc-
tions have thus been made, and then successfully erased; much taxonomy
has been construed, and then torn down, and this process has led to many
valuable insights – Is that not enough? What is the problem?
The problem is that intermediality has tended to be discussed without
clarification of what a medium actually is. Without a more precise under-
standing of what a medium is, one cannot expect to comprehend what
intermediality is. This is not only a terminological problem. On the con-
trary, the understanding of what a medium is and what intermedial relations
actually consist of has vital implications for each and every inquiry in old
and new fields of study concerning the arts and media: ekphrasis, cinema,
illustration, visual poetry, remediation, adaptation, multimedia and so on.
I find it as unsatisfying to continue talking about ‘writing’, ‘film’, ‘perfor-
mance’, ‘music’ and ‘television’ as if they were like different persons that

11

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


12 Media, Modalities and Modes

can be married and divorced2 as to find repose in a belief that all media are
always fundamentally blended in a hermaphroditical way. The crucial ‘inter’
of intermediality is a bridge, but what does it bridge over? If all media were
fundamentally different, it would be hard to find any interrelations at all;
if they were fundamentally similar, it would be equally hard to find some-
thing that is not already interrelated. Media, however, are both different and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
similar, and intermediality must be understood as a bridge between medial
differences that is founded on medial similarities.
The most important aim of this essay is to present a theoretical frame-
work that explains and describes how media are related to each other: what
they have in common, in what ways they differ and how these differences
are bridged over by intermediality. In order to accomplish this, it must be
understood that the concept of medium generally includes several types or
levels of mediality that have to be correlated with each other. ‘Medium’,
of course, is a term widely employed and it would be pointless to try to
find a straightforward definition that covers all the various notions that
lurk behind the different uses of the word. Dissimilar notions of medium
and mediality are at work within different fields of research and there is
no reason to interfere with these notions as long as they fulfil their specific
tasks. Instead, I will circumscribe a concept that is applicable to the issue
of intermediality. Since intermediality will be understood as a general con-
dition for understanding communicative and aesthetic mechanisms, events
and devices, rather than a peripheral exception to ‘regular’ mediality, such a
concept must actually include most of the media notions circulating in the
academic world. Hence, I will not produce a two-line definition of ‘medium’.
I find such definitions counterproductive when it comes to complex con-
cepts and any clear-cut definition of medium can only capture fragments of
the whole conceptual web. Instead, I will try to form a model that preserves
the term medium and yet qualifies its use in relation to the different aspects
of the conceptual web of mediality. As a term, ‘medium’ should thus be
divided into subcategories to cover the many interrelated aspects of the mul-
tifaceted concept of medium and mediality. As my arguments unfold, I will
distinguish between ‘basic media’, ‘qualified media’ and ‘technical media’.
Basic and qualified media are abstract categories that help us understand
how media types are formed by very different sorts of qualities, whereas
technical media are the very tangible devices needed to materialize instances
of media types. Consequently, when talking about a medium without spec-
ifications, the term can refer to both a media category and a specific media
realization.
Evidently, it is important to note that qualified, basic and technical media
are not three separate types of media. Instead, they are three complementary,
theoretical aspects of what constitutes media and mediality. The wide con-
cept of medium that will be presented here thus comprises several intimately
related yet divergent notions that will be terminologically distinguished.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 13

I believe that intermediality cannot fully be understood without grasping


the fundamental conditions of every single medium and these conditions
constitute a complex network of both tangible qualities of media and various
perceptual and interpretive operations performed by the recipients of media.
For my purpose, media definitions that deal only with the physical aspects
of mediality are too narrow, as are media definitions that strongly emphasize

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the social construction of media conceptions. Instead, I will emphasize the
critical meeting of the material, the perceptual and the social. Media of pro-
duction and storage are not really relevant for the forthcoming discussion
and although I recognize the relevance of the aspect of communication in
its widest sense, my aim is not to discuss intermediality within the frame-
work of communication models. Instead, I want to treat mediality from a
hermeneutical point of view. I bracket much of the conditions of media
production and focus on the perception, conception and interpretation of
media as material interfaces situated in social, historical, communicative and
aesthetic circumstances.
The material of my theoretical framework consists of the notions of
modality and mode. Intermedial studies have their historical roots in aes-
thetics, philosophy, semiotics, comparative literature, media studies and, of
course, interart studies.3 During the last few decades, however, the notion of
multimodality has also gained ground, while the roots of this new plant have
grown in different soils; social semiotics, education, medicine and language
and communication studies. There are seldom cross-references between the
two research fields of intermedial and multimodal studies and the notions
of intermediality and multimodality are surprisingly seldom related to each
other.4 Also, in qualified texts of recent date, it is far from clear how ‘inter-
medial’, ‘multimodal’, ‘intermodal’ and ‘multimedial’ are related.5 Since it is
a waste of intellectual energy to develop two closely related research fields
separately, it is a matter of priority to straighten things out as far as core
concepts and basic terminology are concerned.

What is a medium?

Medium means ‘middle’, ‘interval’, ‘interspace’ and so on. The standard


definition found in dictionaries stresses that a medium is a channel for
the mediation of information and entertainment. Art might be seen as a
complex blend of information and entertainment (Horace’s utile dulci) so
it should be fully possible to include the art forms among other media. As
we know, however, the term ‘medium’ is used in many related but different
ways and it is also applied in contexts that are not relevant here. According
to Marshall McLuhan’s influential ideas, media are the ‘extensions of man’
and he suggestively argues that not only the spoken word, the photograph,
comics, the typewriter and television are media, but also are money, wheels
and axes.6 Within the framework of McLuhan’s own sociological theory, this

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


14 Media, Modalities and Modes

notion works rather well, but in order to take the step from ‘medium’ to
‘intermediality’, more accuracy is needed.
The term ‘modality’ is related to ‘mode’ and these terms are also widely
used in different fields. A ‘mode’ is a way to be or to do things. In the con-
text of media studies and linguistics, ‘multimodality’ sometimes refers to the
combination of, say, text, image and sound, and sometimes to the combi-
nation of sense faculties; the auditory, the visual, the tactile and so forth.7

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
In the work of Gunther Kress and Theo van Leeuwen, a mode is understood
as any semiotic resource, in a very broad sense, that produces meaning in
a social context; the verbal, the visual, language, image, music, sound, ges-
ture, narrative, colour, taste, speech, touch, plastic and so on. This approach
to multimodality has its pragmatic advantages but it produces a rather indis-
tinct set of modes that are very hard to compare since they overlap in many
ways that are in dire need of further theoretical discussion.8
It is no wonder, then, that the discourses on media and modalities tend to
be either separated or mixed up. Why bother to combine, or to keep apart,
notions that seem to be fuzzy in rather similar ways? A medium is a channel,
one might say, and of course there are many media, that is, modes of medi-
ating information and entertainment. In ordinary situations, this language
use is rather unproblematic. If one wants to understand the complexity of
individual media in a more precise way, however, I think it is wise to dif-
ferentiate between medium/intermediality and mode/multimodality. As far
as I can tell, there is nothing in the etymology of the words ‘medium’ and
‘mode’, or in the established conceptual uses of them, that clearly deter-
mines how they should be related to each other, so here I will see it as my
task to raise a theoretical construction and propose how to use the central
terms in relation to each other.
Earlier efforts to describe the relations between different media and art
forms as a rule start off with conceptual units such as image, music, text,
film, verbal media or visual media, presuming that it is appropriate to com-
pare these entities. The complexity resulting from such comparisons is often
slightly confusing, I would say, because of two limitations. The first problem
is that the units compared are often treated as fundamentally different media
with little or nothing in common. Thus, every intermedial relation seems
to be more or less an anomaly where the supposedly essentially different
characteristics of allegedly separate media are presumed to be more or less
transformed, combined or blended in a unique way. Mitchell has success-
fully criticized this way of thinking by pointing to the way various important
traits are in fact shared by art forms that are generally seen as opposites, yet
Mitchell’s discourse is also paradoxically but profoundly trapped in the tra-
dition of treating art forms as separate entities. In spite of the efforts to erase
most of the differences between poetry and painting, he anthropomorphizes
the two art forms and emphasizes the ‘struggle’ between them, which makes
it difficult to grasp the exact nature of the similarities of media as conceived

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 15

by Mitchell.9 Media are both similar and different and one cannot compare
media without clarifying which aspects are relevant to the comparison and
exactly how these aspects are related to each other.10
The second problem with many comparisons between conceptual units
such as ‘dance’ and ‘literature’ is that the materiality of media is generally not
distinguished from the perception of media. This is understandable since it is,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
in practice, impossible to separate the two. For human beings, nothing exists
outside perception. Nevertheless, it is crucial to discriminate theoretically
between the material and the perception of the material if one wants to
understand how media can be related to each other. One must be able to
determine to what extent certain qualities belong to the material aspects
of a medium and to what extent they are part of the perception. This is a
slippery business, no doubt, but one must acknowledge that, for instance,
the quality of ‘time’ in a movie is not the same as the ‘time’ that is necessary
to contemplate a still photograph, and that ‘time’ can be said to be present
in many forms in one and the same medium. If one avoids taking notice of
this intricacy, one is left with a featureless mass of only seemingly identical
media that cannot be compared properly.
I therefore consider it a matter of urgency to put forward a model that
starts at the other end, so to speak: not with the units of established media
forms, or with efforts to distinguish between specific types of intermedial
relations between these recognized media, but with the basic categories of
features, qualities and aspects of all media. My point of departure will be
what I call the modalities of media. The modalities are the essential cor-
nerstones of all media without which mediality cannot be comprehended
and together they build a medial complex integrating materiality, percep-
tion and cognition. Separately, these modalities constitute complex fields of
research and they are not related to the established media types in any def-
inite or definitive way; however, I believe that they are indispensable in all
efforts to describe the character of every single medial expression. They are
all very familiar although their interrelations have not been systematically
accounted for. I call them the material modality, the sensorial modality, the
spatiotemporal modality and the semiotic modality, and they are to be found
on a scale ranging from the tangible to the perceptual and the conceptual.
Media and art forms are constantly being described and defined on the
basis of one or more of these modalities.11 The categories of materiality, time
and space, the visual and the auditory, and natural and conventional signs,
have been reshaped over and over again, but they tend to be mixed up in
fundamental ways. Hence, in insightful essays, such as Jiří Veltruský’s ‘Com-
parative Semiotics of Art’, it remains unclear what the ‘material’ of an art
form is.12 According to Veltruský, materials can be divided into the ‘audi-
tory and visual’; the material of music is said to be ‘tones’ and the material
of literature is said to be ‘language’. Furthermore, the material of literature
is supposed to oscillate ‘between materiality and immateriality’.13 Although

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


16 Media, Modalities and Modes

this categorization is fairly representative, it is not at all illuminating. The


category of ‘material’ is fundamentally untenable since it includes aspects of
the arts that cannot be treated as equals; tones, language and even the imma-
terial. ‘Tones’ must be seen as related primarily to the sensorial modality
whereas ‘language’ must be understood in semiotic terms; however, language
actually also consists of some sorts of ‘tones’. What the ‘immaterial’ material

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
is, I do not know. Perhaps the most common mistake in intermedial com-
parisons is to confuse the notions of ‘visual’ and ‘iconic’: the visual is about
using a specific sense, as will be discussed later, whereas the iconic is semiosis
based on similarity (that only sometimes can be seen).
I thus propose that we distinguish between the four modalities mentioned
above to enable a clearer view of how media are constituted by both the
physical realities and the cognitive functions of human beings. I want to
stress that all media, as I understand the concept, are necessarily realized in
the form of all four modalities; hence, it is not enough to consider only one
or a few of them if one wants to grasp the character of a particular medium.
In this respect, there is a fundamental difference between my approach and
the systematic, often hierarchic, but necessarily simplistic classifications and
divisions of the arts that were put forward from the eighteenth century
and well into the twentieth century.14 The proposed model can be used to
highlight both crucial divergences and fundamental parallels between all
sorts and variants of media forms, which gives a firm ground for understand-
ing, describing and interpreting the most elementary intermedial relations.
Of course, the complexity of the innumerable intermedial relations that can
be derived from the four modalities, not least from the semiotic modality,
can only be hinted at.
When I speak of modalities henceforth, I mean these four necessary cate-
gories in the area of the medium ranging from the material to the mental,
and when I speak of modes, I mean the variants of the modalities as described
below. Entities such as ‘text’, ‘music’, ‘gesture’ or ‘image’ are not seen as
modalities or modes. The modalities are obviously interrelated and depen-
dent on each other in many ways, but nevertheless they can be rather clearly
separated theoretically. Also, the modes are entangled with each other in
many different ways, depending on the character of the medium.
Before discussing the four modalities, a preliminary distinction must be
made. All media need technical media to be realized. Our knowledge of the
outer world is always limited by and dependent on our senses but, unless
one gives oneself up to solipsism, one must assume that all media have a
material ground. The notion of a technical medium will be discussed and
defined later in this essay, since a more delineated explanation of what a
technical medium is requires an understanding of the four modalities; here,
it must suffice to say that a technical medium is not the same as the material
modality. The modes of the material modality, like the modes of the three
other modalities, must be understood as latent properties of media, whereas

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 17

the technical medium is the actual material medium, the ‘form’, that realizes
and manifests the latent properties of media, the ‘content’.

The four modalities of media

The order in which the four modalities will be presented is not arbitrary.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
I would not say that the order from the material, the sensorial and the spa-
tiotemporal to the semiotic is ‘temporal’ or ‘hierarchical’ in any clear-cut
way, but I do suggest that there is a point in starting with the material aspect
since this is what would exist even if all living creatures were to be wiped
out from the surface of our planet. The sensorial is the next stage since it is
a prerequisite for the more ‘advanced’ spatiotemporal and semiotic modali-
ties. Without sensory impressions there cannot be any conceptions of time,
space or meaning. The semiotic modality is the ‘last’ modality since it can
be said to include, or at least be based on, the other three. It is hence also
the most complex modality.
The material modality can thus be defined as the latent corporeal interface
of the medium. The material interface of television programs and motion
pictures, for instance, consists of a more or less flat surface of changing
images (in a wide sense of the notion) combined with sound waves. The
interface of most kinds of written text also consists of a flat surface, but the
appearance of the surface is not changing. The interface of music and radio
theatre consists of sound waves. Regular theatre, on the other hand, must be
understood as a combination of several interfaces: sound waves, surfaces that
are both flat and not flat and that have both a changing and static character,
and also the very specific corporeal interface of human bodies. The interface
of sculptures normally consists of extended, generally solid materiality.
The materialities of media can differ in many ways that cannot always
be clearly separated, of course, but I think it is proper to make an approx-
imate distinction between three modes of the material modality: human
bodies, other materiality of a demarcated character such as flat surfaces
and three-dimensional objects, and material manifestations of a less clearly
demarcated character such as sound waves and different sorts of laser or light
projections.
The sensorial modality is the physical and mental acts of perceiving the
present interface of the medium through the sense faculties. Media cannot
be realized: that is, cannot mediate, unless they are grasped by one or more
of our senses. Usually, we talk about the five senses of humans, which may
here be described as the five main modes of the sensorial modality: seeing,
hearing, feeling, tasting and smelling. Still, the issue is, as usual, more com-
plex. At least three levels of the sensorial must be discerned. The first level is
sense-data that originate from objects, phenomena and occurrences but that
can never be captured in isolation without a perceiving and interpreting
agent. Often, but far from always, sense-data tend to cause inter-subjective

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


18 Media, Modalities and Modes

sensations. The sense-data of media come from the realized material


interface. The second level of the sensorial consists of our receptors: cells that
when stimulated cause nerve impulses that are transferred to a nervous sys-
tem. The third level is the sensation, meaning the experienced effect of the
stimulation. All our sensations consist of integrated experiences of the way
a variety of receptors perceive and interpret an array of sense-data.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
The exact nature of sense-data and sensations, and the relation between
them, is very much disputed, whereas the physical receptors have been stud-
ied and described in detail.15 Exteroceptors register changes in the external
environment, interoceptors are sensible to the internal conditions and proprio-
ceptors give us information regarding length and tension in muscle fibres and
sinews. Our five senses are thus actually, to be more precise, the five sense
organs that register changes in the exterior environment: eyes, ears, olfactory
organ, gustatory organ and skin.16 For the moment we witness an increased
interest in the interoceptors and the proprioceptors but most media are still
primarily understood as exterior channels of information. Chiefly sight and
hearing, the two cognitively most advanced faculties, deserve our attention
in the context of media and arts, but not exclusively. Music and speech are
first and foremost heard, but there is a clear physical link between exterior
hearing and inner balance that cannot be ignored. A sculpture is mainly
seen, but it is impossible to grasp its entity without moving and hence also
involving the inner senses. Even if one does not actually touch its surface
one sees and indirectly feels its tactile qualities. The reactivation of memo-
ries of sensorial experiences plays a certain part in the perception of media.
Reading a text, for instance, often involves the creation and recollections of
visual experiences that are very remote from the way the alphabetic letters
look, and it also involves an inner hearing of the sounds of the words. New
sensations are thus frequently a complex web of perceived and conceived
sense-data combined with retrieved sensations.
Sense-data cannot be grasped, cannot be conceived as sensation, unless
they are given some sort of form, Gestalt, in the act of perception. The
spatiotemporal modality of media covers the structuring of the sensorial
perception of sense-data of the material interface into experiences and con-
ceptions of space and time. Media, like all objects and phenomena, receive
their multilayered spatiotemporal qualities in the act of perception and
interpretation; thus, the spatiotemporal cannot be identified with the prop-
erties covered by the material modality, although there is certainly a strong
link between these two modalities. I basically adhere to Kant’s idea that space
and time are a priori sensory intuitions ‘that must precede all empirical intu-
ition (i.e., the perception of actual objects)’.17 Thus, because of cognitive
conditions, all media necessarily in some respect receive both spatial and
temporal qualities. Furthermore, the principles of physics teach that the spa-
tiotemporal relationship is indeed very complex: time and space interact not
only on the level of perception but as physical phenomena as such, but we

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 19

do not have to bother about that when it comes to media modalities. In this
context, it suffices to state that all media have aspects of the two basic modes
of space and time which must theoretically be kept apart in some respects
and brought together in other respects. The closer we come to the sense-data,
the more time and space seem to be able to be considered separately and the
more they can be said to be part of the material modality; the closer we come

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
to the sensations, the more the very distinction between space and time loses
its relevance. This critical difference is often overlooked, which has led to
some confusion in the discussion of intermediality.
Spatiotemporal perception can be said to consist of four dimensions;
width, height, depth and time. The corporeal interface of a photograph
has only two dimensions; width and height. A sculpture has three mate-
rial dimensions, all of them spatial; width, height and depth. A dance has
four dimensions; width, height, depth and time. Every dance performance
has a beginning, an extension and an end situated in the dimension of time,
while a photograph, as long as it exists, simply exists. If one closes one’s
eyes in the middle of a dance performance, something is missed and the
spatiotemporal form cannot be grasped in its entirety. If one closes one’s
eyes while watching a photograph, nothing is missed and the spatial form
remains intact. In this respect, considering the material modality through
the spatiotemporal modality, there are very distinct and certainly relevant
spatiotemporal differences between media.
Hence, media that lack the fourth dimension, time, can be said to be
static, considered as material objects: their sense-data remain the same. For
media that do incorporate the dimension of time in their physical manifesta-
tion, meaning that their sense-data change, some further distinctions can be
made. Motion pictures and recorded music, for instance, have fixed sequen-
tiality. Hypertexts and much music accompanying computer games can be
said to have partially fixed sequentiality. Mobile sculptures, truly improvised
music and a performance broadcast live on television have (at least poten-
tially) non-fixed sequentiality. There are certainly no definite borders between
these categories, and for some media one must also consider the semiotic
modality in order to understand the spatiotemporal nature of the medium.
Listening to a recorded poem is like listening to recorded music: the inter-
face of the medium must be said to have fixed sequentiality. Listening to a
poem being read live is to perceive a medium hovering between the fixed
sequence and the non-fixed sequence. Reading a printed poem is to per-
ceive a medium with a clearly spatial material interface, but as soon as the
conventional semiotic aspect of language is considered, the perception also
incorporates temporality and fixed sequentiality (for most standard poems)
or at least partly fixed sequentiality (for poems lacking clearly distinguish-
able lines). However, this kind of sequentiality, being attributed not to the
material interface but to the realization of sequential sign systems, has a less
definite character.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


20 Media, Modalities and Modes

The most basic form of spatiality is hence the manifestation of the mate-
rial modality in terms of physical width, height and depth, but that is far
from the whole story since our cognition to a large extent works in terms
of spatiality. Also, abstract concepts and experiences of time have spatial
characteristics. Thinking in terms of spatiality is a fundamental trait of
the human mind that has a significant effect on the way we perceive and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
describe media. Experiences and interpretations of, for instance, narratives
and music are also conceived of as spatial relations and patterns.18
Some such conceptions are closely connected to certain types of primarily
visual sense-data. The notion of virtual space covers the effects of media that
are not three-dimensionally spatial on the level of the material interface but
that nevertheless receive a spatial character of depth in the perception and
interpretation. Paintings and photographs actually have only two dimen-
sions, width and height, but often, by means of resemblance of certain visual
qualities in the perceived world they give the illusion of a third, depth,
which creates a virtual space in the mind of the beholder. The interface of
a movie, correspondingly, has three dimensions: width, height and (fixed
sequential) time, but usually an illusion of depth is created. The virtual space
created by a computer is undoubtedly slightly different, since we can choose
to a certain extent how to move within it, but it nevertheless consists of
width, height and (partly fixed sequential) time, together also creating the
illusion of depth. Indeed, verbal narratives also create various sorts of virtual
spatiality in the mind of the listener or reader – not only abstract, conceptual
spatiality but virtual worlds within which the reader can navigate.19
Consequently, at least three levels of spatiality in media can be discerned:
space as a trait of the interface of the medium (the material modality consid-
ered through the spatiotemporal modality), space as a fundamental aspect
of all cognition and space as an interpretive aspect of what the medium
represents (virtual space).
Temporality in media can be understood in a similar way. The most fun-
damental form of time consists of the way the medium’s material modality
is manifested through its sense-data. Some media have corporeal interfaces
that are simply not temporal. Yet, it is important to note that all media are
obviously realized in time: all perception and interpretation of media and
what they mediate are necessarily inscribed in time, which complicates the
modal relations between time and space. Also, media that are not basically
temporal become situated in time as soon as they catch our attention, which
of course has implications for our conception and interpretation of such
media.
As a counterpart to virtual space, the notion of virtual time might further-
more be introduced. Some specific media have spatial characteristics that
encourage the interpretation of the spatial in terms of time passing. To some
extent, there are conventions that make us look at pictures, in a comic
strip for instance, in a certain temporal order. However, this is not a case
of virtual time but rather an instance of pictorial sequentiality produced by

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 21

merging conventions of decoding symbolic and iconic signs. Virtual time


is rather characterized by the capacity of individual pictures to depict not
only one static moment but a series of occurrences.20 Interpretations of still
images of what we, on iconic grounds, take to be moving objects or creatures
always include an interpretation of where the object or creature was ‘before’
and ‘after’ the frozen time in the image.21 Some still images, for instance

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
photographs, may have qualities in the image, such as objects represented
with blurred contours or stretched and transparent objects, that we take to
be indexical depictions of objects moving in space and time.22 These ‘illu-
sions’ of partly fixed sequential time might be called virtual time, which is
the case also for all sorts of time represented by verbal narration. In short,
virtual space and virtual time can be said to be manifest in the perception
and interpretation of a medium when what is taken to be the represented
spatiotemporal state is not the same as the spatiotemporal state of the repre-
senting material modality considered through the spatiotemporal modality.23
Again, we have at least three levels of temporality in media: time as a trait of
the interface of the medium (the material modality considered through the
spatiotemporal modality), time as a necessary condition of all perception
and time as an interpretive aspect of what the medium represents (virtual
time).24
Consequently, there are certainly fundamental differences between media
when it comes to time and space. If one does not acknowledge these dif-
ferences, one cannot understand the complexity of interpreting media in
terms of clashes, fusions and mutual exchanges between the categories of
time and space. The difference between media with various forms of spa-
tiotemporal interfaces is never dissolved, of course, but it is certainly crucial
to note the tension created in a medium lacking, for instance, temporal qual-
ities in the interface, and yet provoking temporal aspects in the perception
and interpretation.
So far nothing has been said about meaning, which I think primarily
belongs to the semiotic modality. Since the world is meaningless in itself,
meaning must be understood as the product of a perceiving and conceiving
subject situated in social circumstances. All meaning is the result of an inter-
preting mind attributing significance to states of affairs, actions, occurrences
and artefacts. In its widest sense, semiotics is a theoretical field aiming at
understanding how the processes of signification work. For me, the most
prolific endeavours of semiotics are those bordering on hermeneutics, such
as the pragmatic sign discussions of Charles Sanders Peirce. Following Peirce,
meaning can be described as the result of sign functions, and although there
are no signs until some interpreter has attributed significance to them, one
can distinguish between different sorts of signs, or sign functions.
The material interfaces of media have no meaning in themselves, of
course, but the process of interpretation already begins in the act of
perception. Conception and cognition do not come after perception; rather,
all our sensations are the results of an interpreting, meaning-seeking mind.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


22 Media, Modalities and Modes

The moment we become aware of a visual sensation, for instance, the sen-
sation is already meaningful at a basic level. Seeing a dancer is to become
aware of a visual sensation of a body being inscribed in a spatiotemporal
continuum. The sensation may also include apprehended similarities with
other phenomena in the world and gestures that we recognize from other
performances. The dancer may wave her arms like a bird, jump like a frog

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
and then bow. What we take to be imitations of animals may be described
as iconic sign functions, whereas the bow is primarily a conventional sign
denoting ‘the end’.
The semiotic modality thus involves the creation of meaning in the
spatiotemporally conceived medium by way of different sorts of think-
ing and sign interpretation. The creation of meaning already starts in the
unconscious apprehension and arrangement of sense-data perceived by the
receptors and it continues in the conscious act of finding relevant con-
nections within the spatiotemporal structure of the medium and between
the medium and the surrounding world. There are two different but com-
plementary ways of thinking: on the one hand, some cognitive functions
are mainly directed by propositional representations, while other cogni-
tive functions mainly rely on pictorial representations.25 Brain research has
shown that to a great extent the two ways of thinking can be located in
the two cerebral hemispheres. We think both in an abstract way and in a
concrete (visual and spatial) way. These interrelated but nevertheless differ-
ent ways of cognition are deeply correlated, I would say, with the semiotic
categories. Earlier, it was common to distinguish between conventional or
arbitrary signs and natural signs. Peirce’s most important trichotomy – sym-
bol, index and icon – has the advantage of avoiding the slightly misleading
idea that some signs exist ‘in nature’, but obviously the symbol is a con-
ventional sign, as Peirce states, and the index and the icon are in a way
natural signs. The indexical sign function is based on cause and closeness,
while iconicity is based on similarity: capacities that are part of the outer
world as it is perceived and conceived by us.26 In semiotic terms, thinking
based on propositional representations can be described as meaning cre-
ated by conventional, symbolic sign functions, whereas thinking based on
pictorial representations can be described as meaning created by indexical
and iconic sign functions. The indexical and the iconic sign functions are
deeply related to the way the mind conceives sense-data as spatiotempo-
ral structures, which is why especially this kind of meaning is the result
of interpretation also on the subliminal level. The spatiotemporal struc-
tures conceived by our mind are ‘designed’ to be meaningful – not in a
propositional way, but in a pictorial way.
I thus propose that convention (symbolic signs), resemblance (iconic
signs) and contiguity (indexical signs) should be seen as the three main
modes of the semiotic modality. According to Peirce, who stresses that the
determinate aspect of all signs are ‘in the mind’ of the interpreter, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 23

three modes of signification are always mixed, but often one of them can
be said to dominate.27 In most written texts, the symbolic sign functions
of the letters and words dominate the signification process. Instrumental
music and all kinds of visual images (for instance, drawings, figures, tables
and photographs) are generally dominated by iconic signs, although pho-
tos also have an important indexical character. The iconic qualities of music

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
and images differ, of course, since the musical signs are auditory signs that
mainly refer to motions, emotions, bodily experiences and cognitive struc-
tures, while the visual signs of images largely refer to other visual entities,
but all of these sign functions are based on resemblance.28 The semiotic char-
acter of all media is exceedingly complex but there is no doubt about the
basic semiotic differences between, for instance, a written text and a moving
image.
These semiotic modes, together with the spatiotemporal, the sensorial
and the material modes, form the specific character of every medium. Let
us briefly and rudimentarily examine a few examples. Traditional sculpture
has a three-dimensional, solid and static material interface. It is primar-
ily perceived visually but it also has tactile qualities. Generally, the iconic
sign function dominates. An animated movie consists of a fixed sequence
of moving images and sounds. Its corporeal interface is a flat surface with
visual qualities combined with sound waves, and the combination of two-
dimensional images and sound often creates an effect of virtual space. The
images are first and foremost iconic and they lack the specific indexical char-
acter of images produced by ordinary movie cameras. The sound generally
consists of voices, sound effects and music: the musical sounds, but often
also much of the voice qualities, are very much iconic, while the parts of
the voices that can be discerned as language are mainly decoded as conven-
tional signs. Printed poetry has a solid, two-dimensional material interface,
or a sequential combination of such interfaces (if realized in the technical
medium of a book). It is perceived by the eyes, but also when read silently
it becomes apparent that it also has latent auditory qualities in the conven-
tional system of signification called language. Most poetry gains its meaning
through these conventional signs, but there may also be substantial por-
tions of iconicity in both the visual form of the text and the silent, inner
sound experiences produced by the mind. In terms of spatiotemporality,
printed poetry is essentially spatial. Very rarely, virtual space is perceived
as a result of illusive depth in the two-dimensional visual appearance of the
poem, whereas virtual space in the sense of illusionary worlds is often cre-
ated. Printed poems that are dominated by readable words, rather than, for
instance, clusters of letters, are indirectly (partly) sequential since the con-
ventional signs (partly) determine the temporal realization of the written
language.
As one can see from these few examples, the modes of different media
clearly differ and the modalities always interact in more or less complex

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


24 Media, Modalities and Modes

ways. Since the modalities cannot be seen as isolated entities, the proposed
model offers no simple, mechanical way of checking off the modes of the
modalities, one after another, but it suggests a method of investigating
minutely the features of various media and how they may be interpreted.
The model roughly supports ideas about media always containing other
media (McLuhan29 ), or media always being mixed media (Mitchell30 ), but

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
it also accounts for, in some detail, what it can mean to say that media are
always entangled in each other, and in which respects, in fact, media are
not contained by or mixed with other media: it is about media necessar-
ily sharing the four basic modalities, but having the shifting modes of the
modalities only more or less in common; it is also about seemingly having
modes in common when in fact many media features come into existence
on different levels, ranging from the material interface to the perception and
interpretation of the medium.
There are thus media similarities and media dissimilarities. All media are
mixed in different ways. Every medium consists of a fusion of modes that are
partly, and in different degrees of palpability, shared by other media. Every
medium has the capacity of mediating only certain aspects of the total real-
ity. Since the world, or rather our perception and conception of the world,
is utterly multimodal, all media are more or less multimodal on the level
of at least some of the four modalities, meaning that they in some respect
include, for instance, both the visual and the auditory mode, both the iconic
and the symbolic mode, or both the spatial and the temporal mode (materi-
ally or virtually). I think it is fair to say that all media are multimodal as far
as the spatiotemporal and the semiotic modalities are concerned, whereas
some media, such as computer games and theatre, are multimodal on the
level of all four modalities.

The two qualifying aspects of media

The four modalities are thus necessary aspects of all conceivable media,
but it is not always sufficient to consider only the modes of the modali-
ties to reach a proper understanding of how media are actually realized and
understood. A deeper understanding of individual media realizations, their
infinitely many qualities and their way of taking part in a world of constant
change, requires additional perspectives. There are at least two other aspects
involved in media constructions and media definitions. These aspects com-
plement the modalities, but they are also to some extent involved in the
character of the modes. I propose they be called qualifying aspects of media.
The first of these two qualifying aspects is the origin, delimitation and use
of media in specific historical, cultural and social circumstances. This may be
called the contextual qualifying aspect. Modal combinations and blends can be
performed in very many ways and often there is no manner of automatically
deciding, on the basis of the modal properties, where the limits of a medium

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 25

are to be found. That can be determined only by way of investigating his-


torically determined practices, discourses and conventions. We tend to talk
about a medium as something that begins to be used in a certain way, or
gains certain qualities, at a certain time and in a certain cultural and social
context.31 ‘Visual art’, ‘Morse messages’, ‘sign language’ and ‘e-mail’ are not
eternal media although they may be neatly described as far as the modal

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
properties are concerned – they appear and (perhaps eventually) disappear
and they are intelligible only in certain cultural and social contexts. Some-
times it is a more or less radical change on the material and technical level,
such as the invention of a new printing technique or a new technological
device, that triggers the genesis of what is taken to be new media. Sometimes
it is rather old techniques that are seen as new media when adopted in new
contexts, as when photographs are exhibited at galleries and museums or
when letters are used to perform ‘mail art’.
The second of the two qualifying aspects that define media includes aes-
thetic and communicative characteristics. This may be called the operational
qualifying aspect.32 There is a strong tendency towards treating a medium
as a medium, or an art form as one form of art, only when certain qual-
itative aspects can be identified. Such aspects are, of course, not eternally
inscribed but formed by conventions.33 In fact, Lessing’s notorious assertions
concerning the rigorous difference between poetry and painting are clearly
normative and deal with qualifying aspects of the arts of time and space.
Lessing’s claims regarding very distinct differences between the temporal
art of poetry representing action and the spatial art of painting represent-
ing objects do not really concern the basic, modal aspects of media. He
recognizes important semiotic differences between the arts, of course, but
constantly demonstrates not least how (allegedly bad) poetry can represent
objects. Poetry, however, should not be as ‘speech and its signs in general’
he claims.34 According to Lessing, then, the restrictions concerning spa-
tiality and temporality in poetry and painting, respectively, are primarily
a question of qualifying aspects.
Another example of how the operational qualifying aspect works would
be ‘cinema’ which, it has been argued, did not become ‘cinema’ the day the
technique was invented.35 Cinema, like other new media, borrowed aesthetic
and communicative characteristics belonging to old media, and although
the first films also had distinct communicative and aesthetic characteris-
tics, of course, it took a while before the many qualifying characteristics
of the mediated content developed into recognizable media forms. Eventu-
ally, there came to be two notions attached to the same term: cinema as a
set of techniques and cinema as a multifaceted qualified medium developed
within the frames of, but not determined by, the technical aspects.36 Music,
on the other hand, can be mediated by a variety of technical media, but most
people would not include simply any kind of sound in the notion of music.
Music, as an art form, a qualified medium, must be produced within assured

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


26 Media, Modalities and Modes

communicative circumstances and fulfil certain conventional aesthetic crite-


ria to be accepted as music. These circumstances and criteria vary, no doubt,
but if they were to be annihilated, few people would find it meaningful still
to talk about the medium of music.
Dance is also a qualified medium governed by aesthetic standards, yet this
art form is closely related to gesture, which might be seen as a qualified

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
medium of another kind. The primary modes involved in both dance and
gesture are the body, visuality and spatiotemporality. As far as the semiotic
modality is concerned, both dance and gesture include iconic and symbolic
signs. I presume there is a tendency towards more indexical sign-functions
in gesture but the main difference between dance and gesture is to be
found in the operational qualifying aspect: dance adheres to conventions
of aesthetic expression whereas gesture is primarily part of communicative
situations.
These two qualifying aspects often interact, of course (and I guess it would
be feasible to split them into three, four or even more specific aspects). As
emphasized by Jürgen E. Müller, the aesthetic and the communicative fea-
tures of a medium often arise, or become gradually accepted, or disappear,
at a certain moment in history and in certain socio-cultural circumstances.37
The relativity in many definitions of particular media is thus strongly related
to the relativity of defining genres and subgenres of media. A genre cannot
be circumscribed as an abstract entity without considering how both ‘form’
and ‘content’ are related to both aesthetic and social changes and sometimes
it is an open question whether a new aesthetic or communicative practice
should be called a medium or a genre.38
The two qualifying aspects thus cannot be left out when trying to
delineate the contours of a medium. A painting consists of paint on a two-
dimensional (or weakly three-dimensional) surface that can be seen (and
to a lesser degree felt and smelled). Generally, the iconic signs dominate.
The iconic signs, together with conventions for representation, very often
make us perceive virtual space in the depiction. In order to be counted as a
painting instead of only paint spread around, however, the picture must be
produced and presented within generally accepted social and artistic frames
and it should have some aesthetic qualities. None of these qualifying aspects
are truly stable, though. Like all art forms and other qualified media, the
nature of ‘painting’ can only be circumscribed ad hoc. The modalities of the
shifting notions of painting are rather stable, however, and provide a useable
starting point for discussing the limits of the medium. If the material surface
of an alleged painting is strongly three-dimensional, it can consequently be
argued straightforwardly, on the basis of conventional genre and media bor-
ders, that it in fact should be seen as a sculpture due to its material modality.
Of course, this ‘redefinition’ leaning on modality properties may have an
impact on the way the painting or sculpture is conceived when taking into
account the qualifying aspects of media definitions.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 27

All of the four modalities, and as a rule also the two qualifying aspects,
must hence be considered when attempting to find the core of one medium
or another – if there is one. However, I think there is a lot to gain in acknowl-
edging not only the existence of modalities and qualifying aspects but also
their different natures. There is no point in comparing different media if the
media in question are described or defined on the basis of only a selection of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
modalities and qualifying aspects that are not properly related to each other.
There is a point in defining ‘music’ as a medium since it can be delimited
rather unambiguously by way of the four modalities and the two qualifying
aspects, notwithstanding the open character of the aesthetic qualities. ‘Liter-
ature’ and ‘alphabetic text’ are not media as such though, I would say, since
there is a distinct and extensive modal difference between the material, sen-
sorial and spatiotemporal modalities of visual text and auditory text. ‘Visual
text’ and ‘visual literature’ (based on printed or otherwise inscribed signs),
however, might be seen as media, since they are both categories that include
fairly similar medial objects (if ‘visual text’ is understood to be a written
sequence of linguistic signs on a spatial surface). On the other hand, there
is a difference between the media ‘visual text’ and ‘visual literature’: visual
literature is heavily dependent on the two qualifying aspects while visual
text is a sort of medium that can largely be defined by way of only the four
modalities. Media that are mainly identified by their modal appearances I
propose to call basic media. Art forms and other cultural media types always
rely strongly on the two qualifying aspects and hence can be called qualified
media.
The distinction between basic media and qualified media is not absolute
and, since the modes of the modalities are not easily isolated entities, there
is no definite set of basic media, I think. However, if we define ‘text’ as
any conventional sign-system, media such as ‘auditory text’, ‘tactile text’,
‘still image’, ‘moving image’, ‘iconic body performance’ and ‘organized non-
verbal sound’ would be examples of what can be seen as basic media. ‘Visual
text’, however, should be seen as a cluster of basic media that differ depend-
ing on whether they are produced by material signs or body movements,
whether they are fixed in space or inscribed in a temporal flow and per-
haps also whether they consist of singular sign units or sequences of signs.39
Apart from being defined by the two qualifying aspects, qualified media can
consist of both single basic media, for instance documentary photography
being based on still images, and combinations of basic media, for instance
motion pictures being primarily based on moving images, auditory text and
non-verbal sounds.

What is intermediality?

It has been argued, for good reason, that intermediality is a result of


constructed media borders being trespassed; indeed, there are no media

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


28 Media, Modalities and Modes

borders given by nature, but we need borders to talk about intermediality.


Werner Wolf emphasizes that media borders are created by conventions and
Christina Ljungberg stresses the performative aspect of border crossings.40
Intermediality would thus be something that sometimes ‘happens’; an effect
of unconventional ways of performing medial works.
Media borders are of at least two kinds, however: media differ partly

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
because of modal dissimilarities and partly because of divergences concern-
ing the qualifying aspects of media and the conventionality of media borders
is mainly a facet of the qualifying aspects.41 Intermedial relations between
basic media such as ‘moving image’ and ‘still image’ can thus be relatively
clearly described within the framework of the four modalities, whereas inter-
medial relations between qualified media such as ‘auditory literature’ and
‘music’ to a great extent also rely on the two qualifying aspects. In the
first case, the border between the two basic media of ‘moving image’ and
‘still image’ is mainly to be found in the spatiotemporal modality, since still
images are spatial whereas moving images are both spatial and temporal.
In the second case, the border between ‘auditory literature’ and ‘music’ is
partly of a modal character, considering that all literature is primarily (but
not exclusively) symbolic and music is primarily (but not exclusively) iconic,
and partly of a qualified character, since the boundaries between what is
counted as literature and music are also largely dependent on cultural and
aesthetic conventions. A ‘normal’ reading of a poem is generally seen as lit-
erature, whereas a singing performance of the same poem counts as music –
and there are many performance variants in between the literary and the
musical that cannot be classified as either literature or music in a clear-cut
way since there is no definite border to be crossed. Sometimes it is rather a
question of whether the poem is being performed in a ‘poetry reading’ or
a ‘concert’. This cultural and aesthetic ambiguity of the difference between
auditory literature and music is clearly linked to the semiotic modality, how-
ever. Also, a rather neutral reading of a poem has some iconic potential, and
what is taken to be the increasing ‘musicality’ of a more varied, rhythmic
and melodic reading is in fact strongly linked to an increase of the iconic
sign function.
Both kinds of media borders, the modal and the qualified, can be crossed
in two rather dissimilar ways. I think it is appropriate to distinguish between,
on the one hand, combination and integration of (basic or qualified) media
and, on the other hand, mediation and transformation of (basic or quali-
fied) media.42 Theatre, for instance, normally combines and integrates, to
varying degrees, basic media such as auditory text, still image and body per-
formance. The aesthetic aspects of these combinations and integrations of
basic media are part of how theatre is understood and defined as a qual-
ified medium. Each basic medium has its own modal characteristics and
when combined and integrated according to certain qualitative conventions
the result is what we call ‘theatre’, consisting of different kinds of material

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 29

interfaces, appealing to both the eye and the ear, being both profoundly
spatial and temporal, producing meaning by way of all kinds of signs and,
certainly, being circumscribed by way of historical and cultural conventions
and aesthetic standards. Theatre may thus be said to be a qualified medium
that is very much multimodal and also, in a way, very much intermedial
since it combines and integrates a range of basic and qualified media.43 The

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
pop song, to take another example, is a qualified medium that includes
the two basic media ‘auditory text’ and ‘organized non-verbal sound’. The
consequences of the combination and integration of these two basic media
are not as far-reaching as the combination of several basic media in the-
atre. Auditory text and organized non-verbal sound have the same material
interface: sound waves that are taken in by the organs of hearing. Their
way of being fundamentally temporal, but also to a certain degree spatial,
is similar. The difference between auditory text and organized non-verbal
sound is clearly to be found in the semiotic modality: the process of sig-
nification in auditory texts is mainly a question of decoding conventional
signs, whereas the meaning of the organized non-verbal sound first and
foremost is a result of interpreting the sounds in terms of resemblance and
contiguity.
An unqualified combination and integration of these two basic media is
not enough to produce a pop song, however. Normally, both the auditory
text and the non-verbal sound need to have certain qualities that confer on
them not only the value of ‘lyrics’ and ‘music’ but also of ‘pop lyrics’ and
‘pop music’. The qualities of qualified media become even more qualified,
so to speak, when aspects of genre are involved; a genre might therefore be
called a sub-medium. Indeed, we usually deem that the lyrics produced by
the singer are in themselves music, as is the sound produced by the mechan-
ical and electronic instruments. The integration of the two basic media in a
pop song is consequently in effect very deep, since the two media are more
or less identical when it comes to three of the four modalities, and concern-
ing the fourth modality, the semiotic, it is perfectly normal to integrate the
symbolic and the iconic sign-processes in the interpretation of both litera-
ture and music. Texts are generally more symbolic and music is generally
more iconic, but the combination and integration of words and music stim-
ulates the interpreter to find iconic aspects in the text and to realize the
conventional facets of the music.
Whether it is relevant to talk about the combination and integration of
media is thus a question of degree: media that share no or few modes,
such as music and visual literature, can only be combined or weakly inte-
grated,44 whereas media that have many modes in common may be deeply
integrated. In fact, one may certainly say that media consisting of many dif-
ferent modes in a way are ‘integrated’ or even ‘mixed’ already as ‘isolated’
media, as Mitchell emphasizes.45 However, it is imperative to note that every
medium is modally ‘mixed’ in a way that is more or less unique, allowing

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


30 Media, Modalities and Modes

different kinds of intermedial mixtures with other media consisting of


dissimilar modal combinations.46
Similarly, whether it is reasonable to talk about the mediation or transfor-
mation of media is a question of grade. In order to understand this properly,
the notion of technical medium, which has already been used tentatively,
must be discussed further. I define a technical medium as any object, physi-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
cal phenomenon or body that mediates, in the sense that it ‘realizes’ and
‘displays’ basic and qualified media.47 In this sense, paper is a technical
medium since it can mediate written words, whereas a pen, which can only
produce and not display written words, is not a technical medium. A gui-
tar, however, which can both produce and at the same time realize musical
sound, can also be said to be a technical medium if one considers espe-
cially its sound-realizing aspects.48 Basic and qualified media can exist only
as ideas without technical media. A technical medium can thus be described
as realizing ‘form’ while basic and qualified media are latent ‘content’. The
crucial connection between the ‘form’ and the ‘content’ of media is found in
the relation between the technical medium and the material modality: the
material modality of a medium consists of a latent corporeal interface that
can be realized in actual manifestations by technical media.
Like all form–content relations, the relation between technical media and
the material modality is very tight: the theoretical distinction can and must
be made, but in practice the two cannot be separated. For instance, the mate-
rial modality of sculpture consists of (an idea of) extended, generally solid
materiality that can be realized by technical media such as bronze, stone or
plaster. As an abstract notion, sculpture is not connected to specific technical
media. Actual sculptures, however, are always necessarily realized by particu-
lar technical media, for instance, metal or plastics. Accordingly, when talking
about media, many aspects are involved: ‘a medium’ may mean both a basic
or qualified medium with latent qualities and a particular realization of a
basic or qualified medium in a specific technical medium. We generally say
that both ‘sculpture’ and ‘a sculpture’ are media, although it would perhaps
be more lucid to say that the latter is an instance of a medium or a ‘medial
configuration’ in the phrasing of Irina Rajewsky.49 Hence, intermediality is
both about abstract relations between basic and qualified media and about
connections between and features of specific works, performances and media
products.
Every technical medium can be identified according to the range of basic
media it has the capacity of mediating: that is, which modal variants of
the four modalities it can mediate. The defining features of a technical
medium are its capacity to realize specific material interfaces and the per-
ceiver’s capacity to interact with these interfaces and with other users of the
medium, whereas the more or less hidden technical properties of the tech-
nical medium (the means of production and storage in a wide sense) are
of subordinate interest as far as this proposed conception is concerned.50

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 31

Mass media should be understood as a kind of technical media that have


the capacity of permitting, say, ‘simultaneous participation of many people
in some significant pattern of their own corporate lives’, as McLuhan sug-
gests.51 The television set is an illuminating case in point: it is a technical
medium able to mediate a range of basic media, primarily ‘moving images’,
‘auditory texts’ and ‘organized non-verbal sound’, but also ‘still images’ and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
various sorts of ‘visual texts’. There is a decisive difference between televi-
sion screens being able to transmit images in colour or in black-and-white
only, since the latter sort limits the range of potential interfaces. In con-
trast, the procedural difference between analogue and digital technologies
has no importance in itself when focusing on how the senses meet the
material impact. The computer, another technical medium, can mediate the
same basic media as the television set. Furthermore, it provides the opportu-
nity to interact with the material interfaces and to communicate with other
computer users. The orchestra is a technical medium that realizes ‘organized
non-verbal sound’. The singer is a technical medium, being able to mediate
both ‘auditory texts’ and ‘organized non-verbal sound’, and certainly also
‘body performance’. Stereo equipment is a rather ‘pure’ technical medium
that mediates, without having the capacity to produce, ‘auditory texts’ and
‘organized non-verbal sound’. Some technical media combine the human
and the non-bodily materiality: a man playing a Jew’s harp might be seen as
a cyborg able to mediate unusual fusions of ‘auditory texts’ and ‘organized
non-verbal sound’.
Every technical medium, accordingly, can fully mediate certain basic and
qualified media but only partly mediate other media. Basic and qualified
media can hence be mediated more or less completely and successfully by
different technical media. A theatre performance can only be realized by
a combination of technical media such as, for instance, human bodies, an
orchestra and properties. A television set, which mediates a feature film very
well (except for the size of the screen), is only capable of partly mediating a
theatre performance: the complex corporeal interface of the theatre appeal-
ing to many senses is reduced to a flat screen and a concentrated source
of sounds and the true visual three-dimensional spatiality is transformed
to virtual spatiality. A solo dance is mediated quite well by a television set,
not very well by still photographs and only in a radically altered form by
a radio – all depending on the shifting modal capacities of the technical
media. Of course, qualified media can be mediated many times by a row of
technical media, which might be called remediation.52
Some terms, we must remember, hover in a slightly confusing way
between denoting technical and qualified media. We have already noted
that ‘cinema’ did not become cinema in a qualified way the day the technol-
ogy was invented. The term ‘photography’, which I recently used to denote
rather vaguely a technical medium, is also the name of a qualified medium
which has in fact been mediated by various technical media through history.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


32 Media, Modalities and Modes

Cameras are technical devices of production (with the capacity to register


light chemically or physically) which can be said to be attached, more or
less distantly, to technical media with shifting properties, for instance, silver-
plated sheet copper, photographic paper or a screen (a computer screen or a
display on the camera itself).
Certain technical media can mediate basic or qualified media that may

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
represent other technical media, which is the case, for instance, when we
see a book or a dancing body on the television screen, but I would not
say that a technical medium as such can be mediated by another technical
medium. The technical medium of a body can be represented on the tele-
vision screen, but it is rather the qualified medium of dance that is being
mediated. Similarly, seeing a representation of a book on the screen has
very little in common with interacting with a real book since the techni-
cal medium book is not mediated. However, the basic media that a book can
mediate – certain visual texts and still images – can also be mediated very
well by the television screen. Seeing parts of a book in a television program
may thus be described as seeing a representation of the technical medium
‘book’ mediating certain basic media which are actually also being mediated
by the technical medium ‘television set’. To put it more straightforwardly,
the technical medium ‘television set’ mediates the qualified medium ‘tele-
vision program’ that represents the technical medium ‘book’ that mediates
the basic medium (visual, verbal, static) ‘text’. If one brackets a few links in
the chain one can also say, correctly, that the technical medium ‘television
set’ mediates the basic medium (visual, verbal, static) ‘text’.
It is thus important to realize that mediation and representation are
closely associated and yet distinct. Mediation is a relation between tech-
nical media and basic or qualified media whereas representation (in this
context) is a relation between basic or qualified media and what they sig-
nify (which may be almost anything, including technical media and other
qualified media). The issue of representation thus belongs to the semiotic
modality, which is only one of the many aspects of media and mediation.
Sometimes, however, when the process of mediation is very smooth; that is,
when the material, the sensorial and the spatiotemporal modalities do not
cause any friction in the mediating procedure, representation and mediation
seem to come very close: a photograph of a landscape painting is definitely
a question of mediation and when asked what the photograph represents
one is inclined to say ‘a landscape’ whereas it actually represents ‘a paint-
ing’: the photograph mediates a painting that represents a landscape. To
be even more detailed: the technical photographic medium, for instance,
photographic paper, mediates the qualified medium of photography that
represents the technical medium of a coloured surface that mediates the
qualified medium of, say, oil painting that represents a landscape. No doubt
it is easier, and often sufficient, to simply say that the photographic image

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 33

represents a landscape, but when wrestling with intermedial issues there are
no short cuts!53
The relation between technical media and basic media is thus a ques-
tion of technical media being able or not being able to mediate certain
modes of the modalities. Consequently, an important facet of the relation
between technical media and qualified media is about technical media also

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
being fit to realize the qualifying aspect of aesthetic and communicative
characteristics. As I stated earlier, all qualified media are characterized by
their origin, delimitation and use in specific historical, cultural and social
circumstances (the contextual qualifying aspect). Since the existence of spe-
cific technical media is an essential facet of every historical moment and
cultural space, all qualified media (qualified ideas of mediality) are more or
less strongly determined by specific technical media (realizations of medial-
ity). Some qualified media are actually fundamentally linked to irreplaceable
technical media. Hence, technical media inevitably also play a crucial part
in the forming of the characteristic aesthetic and communicative qualities
of qualified media (the operational qualifying aspect). Oil painting can be
described as a qualified medium characterized not only by certain modes
but also by unique aesthetic qualities linked to the technical medium of oil
colour, which was invented and developed at a certain time and in a certain
cultural context. Similarly, qualified media types such as computer games
are inconceivable without the resource of recently invented technical media
that allow advanced interaction with the displayed interface.
When the mediation of basic and qualified media through technical media
is restricted by the modal capacities of the technical media, or when the tech-
nical media allow of modal expansion, that is, when the mediation brings
about more or less radical modal changes, it may rather be described as trans-
formation. A solo dance being mediated by a radio is drastically transformed
by the mediation. I can think of two ways to perform such mediation: either
the dance is transformed to auditory text or to organized non-verbal sound.
In both cases, the human body of the dancer, which is normally consid-
ered to be a vital part of dance, is substituted by sound waves and the visual
mode is substituted by the auditory mode. In the case of a dance being trans-
formed to an auditory text, the spatiotemporal furthermore is reduced to a
primarily temporal mode, and, perhaps most importantly, the iconic mode
is transformed to a symbolic mode.
The qualified medium photography being mediated by a book is another
example of potentially radical mediation. Again, I can think of at least
two ways of mediating photography by way of the technical medium of
a book. If reproduced as an image, the book is an ideal technical medium
for mediating a photograph, and certainly other still images as well, present-
ing virtually no modal limitations at all compared with how photography
is usually presented. In fact, the book must be seen as one of the technical

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


34 Media, Modalities and Modes

media that originally determined the qualifying aspects of photography. If


mediated as a verbal description, however, the book offers the possibility
of a smooth mediation of visual texts. However, the visual text is in itself
a radical transformation of the photography: the two-dimensional spatiality
(involving virtual depth) of the primarily iconic and indexical still image has
been transformed to two-dimensional spatiality that, because of the mainly

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
symbolic character of the visual text, also involves the temporal aspect of
fixed sequentiality.
It is hence not necessarily the technical medium that ‘forces’ the transfor-
mation. Ekphrasis, for instance, is part of the general habit of transforming
basic and qualified media to other basic and qualified media, which is some-
times a result of the modal constraints of technical media (as when a football
match is covered on the radio) and sometimes a result of aesthetic or com-
municative choices to take advantage of modal possibilities (as when a
verbal narrative is transformed to a symphonic poem or when a movie is
transformed to a computer game). The classical ekphrasis, a poem describ-
ing a painting, is characterized by a certain kind of medial transformation:
the factual space and the virtual time of the painting’s visual iconicity are
being transformed to the virtual space of the poem’s sequentially arranged
symbolic signs.54
Obviously, there are very many kinds of intermedial transformations.55
Sometimes these involve fairly clear and complete relations between specific
art works or media productions, as when a particular newspaper article is
clearly recognizable in its Internet version (although with fewer words and
added animations and hyperlinks) or when a specific novel can be identified
as the source of a movie (although the narrative has been abridged and visual
and iconic qualities have been added). Sometimes it is rather a question of
less definitive and fragmentary media traits that travel between modes and
media types, as when musical form is traced in a short story or when visual
traits associated with comic strips can be said to have found their way to the
moving images of motion pictures.
In her book Intermedialität, Irina O. Rajewsky operates, on different levels
of distinction, with three notions that I find relevant when discussing media
transformation: transmediality (Transmedialität, phenomena that are not
media specific, such as parody), medial transposition (Medienwechsel, media
transformations such as adaptation) and intermedial references (intermediale
Bezüge, for instance narrativization of music or musicalization of fiction).56
These distinctions are valuable as long as one does not force them. In prac-
tice, however, it is not always clear when a medium is actually a distinct
transformation of another medium, exactly when some of the indistinct
media borders have been transgressed, or which traits are to be considered as
belonging to the one medium or the other. Intermedial transformations can
only partly be described as a fixed set of media relations. Ultimately, it is a
question of hermeneutics: when finding traces of another medium, whether

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 35

it is a basic medium or a specific qualified medium, it sometimes makes sense


to say that the initial medium is very much recognizable and that it has been
transformed to the other medium. Sometimes, if the connection seems more
fragile, one might prefer to say that the one medium simply refers to the
other. The only method of deciding whether it is a case of ‘strong’ transfor-
mation or ‘weak’ reference is to interpret. Actually, it even makes sense also

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
to talk about the transformation of fictive media. There is no fundamental
difference between the descriptions of, for example, an existing photo and
a non-existent photo. At times, the most important issue is not at all to
determine the transformational direction and specific relation between two
specific media instances but rather to compare traits, structures and forms
of meaning that are to be found in many qualified media, within a specific
historical context or crossing historical and social boundaries, where they
‘circulate’ without being definitely linked to the one or the other medium.
Such transmedial phenomena are best captured by transmedial notions and
concepts.57

What is the conclusion?

The starting point of this essay was the simple idea that if the notion of
medium is not specified, the notion of intermediality cannot be understood
properly. Media must be understood to be both similar and different and
the notion of multimodality can be used to describe in a rather strict way
what the many similarities and differences are. There are four modalities
that underlie all conceivable media but each modality encloses several modes
that vary between media. The modes of the modalities are not always easily
detectable properties; rather, they are to be found on a scale from the mate-
rial to the perceptual and the conceptual. Understanding the modal qualities
of media is not a question of simple observation; it also includes cognition
and interpretation.
The model presented for understanding intermedial relations is a bottom-
up model. Instead of beginning with a small selection of established media
and their interrelations, which is the usual method, it starts with the modal-
ities and modes that are shared by all media. The relations between the four
modalities and the modes that are most easy to track down are illustrated in
Figure 1.
Apart from the modalities, two qualifying aspects must be considered in
order to understand the notion of medium. The contextual qualifying aspect
is the origin, delimitation and use of media in specific historical, cultural
and social circumstances. The operational qualifying aspect is the aesthetic and
communicative characteristics of media. What I propose to call basic media,
are defined by the four modalities whereas qualified media are defined by the
four modalities and the two qualifying aspects. All qualified media are based
on one or more basic media.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


36 Media, Modalities and Modes

Modality What the modality is The most important modes of the


modality

Material modality The latent corporeal interface of  human bodies


the medium; where the senses  other demarcated materiality
meet the material impact  not demarcated materiality
Sensorial modality The physical and mental acts of  seeing

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
perceiving the interface of the  hearing
medium through the sense  feeling
faculties  tasting
 smelling
Spatiotemporal The structuring of the sensorial  space manifested in the
modality perception of the material material interface
interface into experiences and  cognitive space (always
conceptions of space and time present)
 virtual space
 time manifested in the material
interface
 perceptual time (always
present)
 virtual time
Semiotic modality The creation of meaning in the  convention (symbolic signs)
spatiotemporally conceived  resemblance (iconic signs)
medium by way of different sorts  contiguity (indexical signs)
of thinking and sign interpretation

Figure 1 The modalities and modes of media

Thus, intermedial relations can be found both between basic media and
between qualified media. Intermedial relations consequently include both
modal relations and qualified relations founded on conventions and a range
of historically changing circumstances. Furthermore, intermediality is about
both basic and qualified media as such and about specific works and per-
formances. Intermedial relations have been categorized in many intricate
systems but for the purpose of this essay I find it sufficient to differenti-
ate between two main types, each holding a variety of merging variants: on
the one hand, combination and integration of media and, on the other hand,
mediation and transformation of media. Depending on their modal charac-
ter and to a certain extent also on their qualifying properties, media can be
both rather loosely combined and intimately integrated. Media consisting of
many modes are in a way ‘integrated’ in themselves.
Basic and qualified media must be understood as abstractions that need
technical media to be materially realized. The material modality is the latent
corporeal interface of a media type that can be realized in actual manifes-
tations by technical media. The relation between technical media and basic
media is consequently very much a question of technical media being able

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 37

to or not being able to mediate certain modes, and all technical media must
be defined in relation to the range of basic media they have the capacity of
mediating. When mediation involves more or less radical modal changes, it
is feasible to say that it involves transformation.
Doubtless, there are other kinds of intermedial relations that have not
found their way into my model but might be compatible with it. For

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
instance, the relations between various technical media, regardless of their
mediated ‘content’, may well be described partly in terms of modal differ-
ences. Needless to say, the many abstract examples of media characteristics
and intermedial relations that I have briefly discussed have forced some
rather brusque simplifications, but my point has been not so much to exactly
circumscribe certain media characteristics and specific intermedial relations
as to provide a model and a rudimental method for such business.
What is a medium? The confusion around this question, and the incom-
patibility of many of the suggested answers, is largely caused, I think, by the
shifting approaches of different scholars and research traditions. Technical
aspects as well as modal and qualifying aspects have been emphasized in
diverse and often exclusive ways in the efforts to find narrow and hence
efficiently operable definitions of the notion of medium.58 One alterna-
tive has been to lean on conceptions of media that are open-ended and
mind-triggering but difficult to handle in the context of intermediality. The
advantage of rather seeing a medium as a complex of interrelated facets –
the technical, the modal and the qualifying aspects – is that such a notion
sets certain limits while at the same time it incorporates most of the actual
comprehensions of mediality.
What is multimodality? To say that a medium is multimodal if it com-
bines, for instance, solid materiality, visuality, spatiality and iconicity, is a
truism since there simply are no media that are not being realized by at least
one mode of each modality. Multimodality in a more qualified sense must
hence mean that a medium includes many modes within the same modality.
However, all media are at least slightly multimodal as far as the spatiotem-
poral and the semiotic modalities are concerned, whereas some media are
multimodal on the level of all four modalities. It can thus be argued that
multimodality is very much about really observing and emphasizing the
very common and perfectly normal multimodal characteristics of media.
Certainly, multimodality is a very general phenomenon that may also be
studied outside the context of media.
What is then intermediality? The many possible intermedial relations
within and between media have been discussed in some detail so far and
it has become clear, I think, that intermediality is a notion that cannot
be understood without the notions of modality, mode and multimodal-
ity. Intermediality might be described as ‘intermodal relations in media’ or
‘media intermultimodality’. I do not expect these terrible terms to win gen-
eral praise but I think there is a point in seeing intermediality as a complex
set of relations between media that are always more or less multimodal.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


38 Media, Modalities and Modes

I have hesitated to describe certain kinds of media as intermedial ‘in


themselves’. Many media, if not all, are indeed multimodal ‘in themselves’,
but when also considering the qualifying aspects of media, things become
more complex. Media characterized by strong multimodality may be said to
be intermedial in themselves in the sense that certain modal ‘borders’ are
crossed. However, theatre and computer games, two examples of strongly

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
multimodal media, are conventionally understood and rather well defined as
qualified media, so in that sense they are coherent media rather than exam-
ples of pronounced intermedial crossings of conventional borders, although
they may be said to fuse a multitude of qualified media that also exist in their
own right: music, for instance. It is thus necessary to acknowledge that qual-
ified media are conventionally circumscribed, but one must also realize that
the circumscriptions of qualified media in themselves create complex net-
works of conventional media borders. All qualified media overlap, and some
conventional media are totally engulfed by other conventional media. Con-
sequently, one is actually not much helped by the notion that intermediality
is the crossing of conventional media borders.
The point is that both multimodality and intermediality are to be found
everywhere. One can thus say that everything is intermedial and multimodal,
which is definitely true in a way, but that might come dangerously close to
saying that nothing is intermedial or multimodal. Given the fact that quali-
fied media are changing entities, I still find it most profitable to hold on to
the idea that all ‘mediality’ involve ‘intermediality’. I do not believe that it
is necessary or even possible to circumscribe a specific corpus of intermedial
works or a set of fixed relations between media, although I find many of the
scholarly systems of intermedial ‘works’ and ‘relations’ very valuable.59 Of
course, it is essential to discuss which media, and which relations between
media, might be of specific intermedial relevance, but I think it is pointless
to try to establish clear borders – they are bound to be crossed. Who would,
today, dream of deciding the exact delimitations of ‘art’ or even ‘visual art’ in
order to fix the area of investigation for art historians? Who would want the
objects of ‘popular culture’ to be narrowly defined in order to select suitable
objects for, say, cultural studies to interpret? The nature of intermedial rela-
tions, as they have been described here, is thus only seemingly exact and one
must realize that they can be pinned down only to a certain extent. Interme-
dial analysis cannot live without her twin sister intermedial interpretation.
Intermediality is thus certainly about specific intermedial relations but it is
also, and perhaps primarily, I would say, about studying all kinds of media
with a high level of awareness of the modalities of media and the crucial
modal differences and similarities of media. What makes intermedial studies
important is that they offer insights into the medial nature of all media, not
only a selection of peripheral media.60 The objects of intermedial studies may
well be, for instance, media that have been categorized as ‘intermedial’ in
themselves but they may also be what have been taken to be ‘normal’ media.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 39

The outcome of the studies does not so much depend on the objects of study
as on the way the studies are performed. Some studies of pattern poems or
newspapers may totally lack intermedial relevance, whereas other studies of
written prose texts or photographs may be bursting with intermedial aspects.

Notes

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
1. G. E. Lessing (1984) Laokoön: An Essay on the Limits of Painting and Poetry, trans-
lated, with an introduction and notes, by E. A. McCormick (Baltimore MD and
London: Johns Hopkins University Press). Lessing states that the poet can treat
‘two kinds of beings and actions, visible and invisible’, whereas in painting ‘every-
thing is visible’ (pp. 66, 76). It is certainly questionable to propose that painting
cannot deal with the ‘invisible’, but what is by far the most important for Lessing
is to be normative rather than descriptive: the good poet should not deal with
the visible unless it is inscribed in time in the form of action, whereas the good
painter should not deal with action at all, but only with visible objects that are
not inscribed in time. Actually, he once states that ‘signs existing in space can
express only objects whose wholes or parts coexist, while signs that follow one
another can express only objects whose wholes or parts are consecutive’, which
sounds very definite, but his conclusion deals with the ‘true subjects’ of poetry
and painting (p. 78) and in the rest of the essay he constantly refers to examples
of, for instance, poetry describing static objects, which is of course fully possi-
ble but not, according to Lessing, recommendable (p. 85). A philosopher such as
Susanne K. Langer is much more consistent when it comes to upholding tenable
borders between the arts. See S. K. Langer (1957) ‘Deceptive Analogies: Specious
and Real Relationships among the Arts’, Problems of Art: Ten Philosophical Lectures
(New York: Charles Scribner’s Sons), pp. 75–89.
2. Cf., for instance, Jörg Helbig’s recent taxonomy of intra, inter, trans and mul-
timedial relations in J. Helbig (2008) ‘Intermedialität – eine spezifische Form
des Medienkontakts oder globaler Oberbegriff? Neue Überlegungen zur System-
atik intersemiotischer Beziehungen’ in J. E. Müller (ed.) Media Encounters and
Media Theories (Münster: Nodus Publikationen). It works quite well as a very
rough model and it is representative of the interart tradition where the media
are very much seen as more or less separate entities. Its value is nevertheless
severely reduced because of the idea that media can be understood as ‘distinct
sign systems’ (p. 83) with fixed ‘medial borders’ (p. 79), each medium having its
‘medial surface’ (p. 85). Cf. also Axel Englund’s critical discussion, in this volume,
of ‘topographic’ ways of defining and delimiting media.
3. See the very comprehensive overview of the development of the research field
intermedial studies in C. Clüver (2007) ‘Intermediality and Interarts Studies’ in
J. Arvidson, M. Askander, J. Bruhn and H. Führer (eds) Changing Borders: Con-
temporary Positions in Intermediality (Lund: Intermedia Studies Press), pp. 19–37.
Also Irina Rajewsky’s overview of the field, from the point of view of comparative
literature and media studies, is valuable: I. O. Rajewsky (2008) ‘Intermedialität
und remediation: Überlegungen zu einigen Problemfeldern der jüngeren Interme-
dialitätsforschung’ in J. Paech and J. Schröter (eds) Intermedialität Analog/Digital:
Theorien – Methoden – Analysen (Munich: Wilhelm Fink), pp. 47–60. Many of the
studies of ‘intermediality’ are, in fact, considering both the theoretical frame-
works and the objects of research, rather studies of ‘interartiality’, for instance

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


40 Media, Modalities and Modes

T. Eichner and U. Bleckmann (eds) (1994) Intermedialität: Vom Bild zum Text (Biele-
feld: Aisthesis Verlag) and W. Wolf (1999) The Musicalization of Fiction: A Study in
the Theory and History of Intermediality (Amsterdam and Atlanta, GA: Rodopi).
4. Mikko Lehtonen, however, arguing eloquently for the introduction of the per-
spective of intermediality in cultural studies, has published an essay in a journal
of media and communication studies where the notions of multimodality and
intermediality are combined: M. Lehtonen (2001) ‘On No Man’s Land: Theses

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
on Intermediality’, Nordicom Review 22, 71–83. Lehtonen is not aware of the
research tradition of intermediality described by Clüver and hence he more or
less has to invent the topic. Although he uses the notions in a different way from
that proposed in this essay, the relation between multimodality and interme-
diality is accurately described: ‘multimodality always characterizes one medium
at a time. Intermediality, again, is about the relationships between multimodal
media’ (p. 75). Lehtonen is not the only one who has invented intermedi-
ality anew. See, for instance, L. M. Semali and A. W. Pailliotet (eds) (1999)
Intermediality: The Teachers’ Handbook of Critical Media Literacy (Boulder CO and
Oxford: Westview Press), where media literacy is discussed in terms of interme-
diality without knowledge of the existing research fields of intermediality and
multimodality.
5. See S. Moser (2007) ‘Iconicity in Multimedia Performance: Laurie Anderson’s
White Lily’ in E. Tabakowska, C. Ljungberg and O. Fischer (eds) Insistent Images,
in Iconicity in Language and Literature 5 (Amsterdam: John Benjamins), p. 323.
In another essay, S. Moser (2007) ‘Media Modes of Poetic Reception: Reading
Lyrics Versus Listening to Songs’, Poetics 35, 277–300, the author uses the terms
‘modality’ and ‘mode’ in a perfectly comprehensible but not systematic way:
‘modalities of poetic language (print/song) and corresponding modes of recep-
tion (reading/listening)’ (p. 277); ‘Songs are a multisensorial mode of linguistic
communication’ (p. 278); ‘lyrics occur in different media modalities, namely
oral . . . printed . . . and audiovisual’ (p. 278) and so forth. The ‘intermedia prac-
tice’ of popular songs is said to enact and embody ‘the interplay and integration of
oral, literate and audiovisual modes of linguistic communication’ (p. 283).
6. M. McLuhan (1994) Understanding Media: The Extensions of Man, Introduction by
Lewis H. Lapham (Cambridge MA and London: MIT Press), p. 24.
7. Cf. B. Granström, D. House and I. Karlsson (eds) (2002) Multimodality in Language
and Speech Systems (Dordrecht, Boston MA and London: Kluwer Academic Pub-
lishers). In the ‘Introduction’ by the editors, it is stated that multimodality is, in
essence, ‘the use of two or more of the five senses for the exchange of informa-
tion’ (p. 1). In many of the essays in the same volume, however, modalities are
also understood as gesture, speech, writing and so forth. In one of them, N. O.
Bernsen (2002) ‘Multimodality in Language and Speech Systems: From Theory
to Design Support Tool’, a medium is circumscribed as ‘the physical realisation of
some presentation of information at the interface between human and system’,
meaning in effect that media are defined by the ‘sensory modalities’ (p. 94).
8. G. Kress and T. van Leeuwen (2001) Multimodal Discourse: The Modes and Media of
Contemporary Communication (London: Hodder Arnold), pp. vii, 3, 20, 22, 25, 28,
67, 80; G. Kress and T. van Leeuwen (2006) Reading Images: The Grammar of Visual
Design, 2nd edn (London and New York: Routledge), pp. 46, 113, 177, 214. In
spite of claims of systematic analysis, the fundamental notion of multimodality
is still circumscribed rather haphazardly by researchers following Kress and van
Leeuwen, for instance J. A. Bateman (2008) Multimodality and Genre: A Foundation

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 41

for the Systemic Analysis of Multimodal Documents (Basingstoke and New York:
Palgrave Macmillan), who takes modes such as text, image, diagram, the visual,
the spatial and the verbal to contribute to multimodality (pp. 1, 7), although
these overlapping modes are never clearly related to each other as far as semiotic,
sensorial or spatiotemporal aspects are concerned.
9. In W. J. T. Mitchell (1986) Iconology: Image, Text, Ideology (Chicago and London:
University of Chicago Press), his fundamental and very traditional dichotomies

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
are text/image, verbal/pictorial and poetry/painting. This is understandable in
the light of the historical tradition that he wrestles with, but these dichotomies
must be overcome in order to grasp fully the similarities of media. By way of con-
stantly reinforcing these dichotomies through the figure of paragone (the ‘battle’
of the arts), which is of course necessary to achieve the historical understanding
of culturally constructed differences between poetry and painting (which seems
to be Mitchell’s most important aim), he thus in a way reinstates the differences
that he simultaneously deconstructs on the ahistorical level. In M. Bal (1991)
Reading Rembrandt: Beyond the Word-Image Opposition (Cambridge: Cambridge Uni-
versity Press), the author convincingly and brilliantly demonstrates that ‘word’
and ‘image’ are interrelated and integrated in complex ways, but she continues
to operate with the dichotomy ‘verbal/visual’ in spite of the modal incommensu-
rability of the two notions (the verbal and the visual are not to be understood as
media-specific characteristics, though).
10. Cf. the detailed and often enlightening comparison of literature and music in
W. Wolf (1999) The Musicalization of Fiction, and the excellent comparison of
prose fiction and fiction film in C. Johansson (2008) Mimetiskt syskonskap: En
representationsteoretisk undersökning av relationen fiktionsprosa-fiktionsfilm [Mimetic
Sisterhood: A Representation Theoretical Study of the Interrelations of Prose
Fiction and Fiction Film] (Stockholm: Acta Universitatis Stockholmiensis).
11. An important early thinker who saw things more clearly than most was Moses
Mendelssohn. In ‘On the Main Principles of the Fine Arts and Sciences’ [Über die
Hauptgrundsätze der schönen Künste und Wissenschaften, 1757], translated by
D. O. Dahlstrom, in D. O. Dahlstrom (ed.) (1997) Philosophical Writings (Cam-
bridge: Cambridge University Press), Mendelssohn builds a typology with the
aid of distinctions such as ‘natural’ versus ‘arbitrary’ signs, ‘the sense of hear-
ing’ versus ‘the sense of sight’ and signs that are represented ‘successively’ versus
‘alongside one another’ (pp. 177–9). The typology is sketchy but instructive since
Mendelssohn clearly realizes that the borders of the arts ‘often blur into one
another’ (p. 181). In modern times, Wendy Steiner has provided one of the
most nuanced and constructive accounts of many of the problems connected
to the spatiotemporal and semiotic aspects of interart comparison where she
manages to avoid most of the common pitfalls: W. Steiner (1982) The Colors of
Rhetoric: Problems in the Relation between Modern Literature and Painting (Chicago
and London: Chicago University Press). In W. J. T. Mitchell (1987) ‘Going Too
Far with the Sister Arts’ in J. A. W. Heffernan (ed.) Space, Time, Image, Sign: Essays
on Literature and the Visual Arts (New York: Peter Lang), the author discusses ‘four
basic ways in which we theoretically differentiate texts from images’. Three of
these ways are ‘perceptual mode (eye versus ear)’, ‘conceptual mode (space ver-
sus time)’ and ‘semiotic medium (natural versus conventional signs)’. He argues
that ‘there is no essential difference between poetry and painting, no difference,
that is, given for all time by the inherent natures of the media, the objects they
represent or the laws of the human mind’ (pp. 2–3). Mitchell demonstrates very

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


42 Media, Modalities and Modes

well that ideological considerations often permeate much of the rhetoric concern-
ing medial differences, and that there are important similarities, but he does not
really suggest that there are no differences. Although it is important not to exag-
gerate the differences between media, I would say that it is fully possible ‘to give
a theoretical account of these differences’ (p. 2), essential or not, which Mitchell
sincerely doubts.
12. J. Veltruský (1981) ‘Comparative Semiotics of Art’ in W. Steiner (ed.) Image and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Code (Ann Arbor: Michigan Studies in the Humanities).
13. Ibid., p. 110.
14. See the enlightening chapter, ‘Philosophical Classifications of the Arts’ in
T. Munro (1969) The Arts and Their Interrelations, revised and enlarged edi-
tion (Cleveland OH and London: Press of Case Western Reserve University),
pp. 157–208.
15. Living creatures have photosensitive, chemosensitive, mechanosensitive, elec-
trosensitive and thermosensitive receptors. The photoreceptors of human beings
are found in the eyes, while other creatures have them in the skin. Our
chemoreceptors are located in the organs of taste and smell, but also in certain
blood vessels. Mechanoreceptors that register changes of position and pres-
sure are in the organs of balance and hearing, and in the skin we also have
mechanoreceptors that register touch, pressure and vibrations. Similarly, electro
and thermoreceptors are located at various places in the body.
16. These sense organs do not, however, simply consist of five different kinds of recep-
tors. Both the olfactory and the gustatory organs consist of chemoreceptors, and
the skin consists of both mechanoreceptors and thermoreceptors. Moreover, the
sense organs and the different kinds of receptors do not work in isolation. Strong
sound, for instance, can both be heard and felt by the whole body, although
human beings do not have the very sensitive external sense organ of fishes, the
side line, which registers all kinds of sounds, vibrations and movements in the
fish’s environment.
17. I. Kant (1997) ‘Prolegomena to any future Metaphysics that will be able to come forward
as Science’, with selections from the ‘Critique of Pure Reason’, translated and edited
by G. Hatfield (Cambridge: Cambridge University Press), Prolegomena p. 35, § 10.
Cf. T. Munro (1969) The Arts and Their Interrelations, pp. 362–3, 399–406.
18. The notion of ‘spatial thinking’, inspired by R. Arnheim (1969) Visual Think-
ing (Berkeley and Los Angeles CA: University of California Press) is discussed in
L. Elleström (2002) Divine Madness: On Interpreting Literature, Music, and the Visual
Arts Ironically (Lewisburg PA and London: Bucknell University Press), pp. 184–93,
219–24 and in L. Elleström (forthcoming) ‘Iconicity as Meaning Miming Mean-
ing, and Meaning Miming Form’ in M. Beukes, J. Conradie, O. Fischer and
C. Ljungberg (eds) Signergy, in Iconicity in Language and Literature 7 (Amsterdam:
John Benjamins). For spatiality in music, see R. P. Morgan (1980) ‘Musical
Time/Musical Space’ in W. J. T. Mitchell (ed.) The Language of Images (Chicago
and London: University of Chicago Press), pp. 259–70. For spatiality in literature,
see W. J. T. Mitchell (1980) ‘Spatial Form in Literature: Toward a General Theory’
in the same volume, pp. 281–6. In this inspiring essay, Mitchell clearly sees the
complexities of the spatiotemporal and he is eager to make the important point
that space and time are closely interrelated. However, he makes no distinction
between, for instance, ‘spatial forms’ and the ‘experience’ of spatial forms, which
somewhat reduces the compass of his arguments.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 43

19. Cf. J. Frank (1991) ‘Spatial Form in Modern Literature’ [1945] in The Idea of Spa-
tial Form (New Brunswick and London: Rutgers University Press), pp. 5–66. The
notion of virtual worlds has since then been extensively explored within the field
of cognitive poetics.
20. See the enlightening discussions in J. A. W. Heffernan (1987) ‘The Temporaliza-
tion of Space in Wordsworth, Turner, and Constable’ in J. A. W. Heffernan (ed.)
Space, Time, Image, Sign: Essays on Literature and the Visual Arts (New York: Peter

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Lang), pp. 64–5.
21. Lessing acknowledges that there are represented bodies in painting ‘which, by
their position, permit us to conjecture an action’: G. E. Lessing (1984) Laokoön,
p. 77. He actually clearly states (which seems to be at odds with his earlier
one-sided assertion concerning what ‘signs existing in space can express’) that
‘painting too can imitate actions, but only by suggestion through bodies’ (p. 78).
22. Cf. E. H. Gombrich (1980) ‘Standards of Truth: The Arrested Image and the Mov-
ing Eye’ in W. J. T. Mitchell (ed.) The Language of Images (Chicago and London:
University of Chicago Press), pp. 208–17.
23. Langer uses the term ‘virtual time’ to denote the aspect of time in both music and
the plastic arts: Langer (1957) ‘Deceptive Analogies’, pp. 81–3.
24. Also Wendy Steiner comes to the conclusion that space and time ‘in fact relate
to three very different aspects of the work’ (1982, p. 50). Their aspects are sim-
ilar but not identical to the three levels that I discern. Cf. the detailed and
mostly convincing discussions in J. Levinson and P. Alperson (1991) ‘What
Is a Temporal Art?’ Midwest Studies in Philosophy 16, 439–50. Levinson and
Alperson, too, conclude that there are three main variants of temporality
in arts.
25. For references to relevant research, see Elleström (forthcoming) ‘Iconicity as
Meaning Miming Meaning, and Meaning Miming Form’.
26. C. Hartshorne and P. Weiss (eds) (1960) Collected Papers of Charles Sanders Peirce
Volume II, Elements of Logic (Cambridge MA and London: The Belknap Press of
Harvard University Press), pp. 156–73.
27. Ibid., p. 135.
28. See Elleström (2009) ‘Iconicity as Meaning Miming Meaning, and Meaning Mim-
ing Form’. I am well aware of the lack of consensus, not least when it comes to
the question of meaning in music, but my point is that no matter how you define
the semiotic character of a qualified medium it must include semiotic differences
that are at least partly media specific. Even if one does not accept the notion of
musical iconicity one must admit that there is a fundamental difference between
the way music and, say, literature produce meaning.
29. McLuhan (1994) Understanding Media, pp. 8, 305.
30. ‘all media are mixed media, combining different codes, discursive conventions,
channels, sensory and cognitive modes’: W. J. T. Mitchell (1994) ‘Beyond Com-
parison: Picture, Text, and Method’ in Picture Theory: Essays on Verbal and Visual
Representation (Chicago and London: University of Chicago Press), p. 95; ‘All
media are, from the standpoint of sensory modality, “mixed media” ’ and ‘the
very notion of a medium and of mediation already entails some mixture of sen-
sory, perceptual and semiotic elements’: W. J. T. Mitchell (2005) ‘There Are no
Visual Media’, Journal of Visual Culture 4, pp. 257, 260. Cf. the briefer comments
in W. J. T. Mitchell (2005) What do Pictures Want? The Lives and Loves of Images
(Chicago and London: Chicago University Press), pp. 215, 350.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


44 Media, Modalities and Modes

31. Joseph Garncarz rightly argues that one must see media ‘not only as textual sys-
tems, but as cultural and social institutions’: J. Garncarz (1998) ‘Vom Varieté zum
Kino: Ein Plädoyer für ein erweitertes Konzept der Intermedialität’ in J. Helbig
(ed.) Intermedialität: Theorie und Praxis eines interdisziplinären Forschungsgebiets
(Berlin: Erich Schmidt Verlag), p. 253.
32. I refrain from trying to say exactly what ‘aesthetic and communicative character-
istics’ are. Actually, any understanding of these characteristics is part of the way

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the operational qualifying aspect works: all changes in aesthetic conceptions, and
even a denial of the point of thinking in terms of aesthetics, take part of the
forming of qualified media.
33. Cf. Wolf’s definition of a medium as ‘a conventionally distinct means of com-
munication, specified not only by particular channels (or one channel) of
communication but also the use of one or more semiotic systems serving for the
transmission of cultural “messages” ’: Wolf (1999) The Musicalization of Fiction,
pp. 35–6.
34. Lessing (1984) Laokoön, p. 85.
35. See A. Gaudreault and P. Marion (2002) ‘The Cinema as a Model for the Genealogy
of Media’, translated by Timoty Barnard, Convergence 8(4), 12–18. Cf. the case
of video which was first launched as a technical medium and then eventually
gave birth to a qualified medium with specific aesthetic qualities, as described in
Y. Spielmann (2008) Video: The Reflexive Medium, translated by Anja Welle and
Stan Jones (Cambridge MA and London: MIT Press).
36. The terminological problem is aggravated by the shifting use of similar words
in different languages. Hajnal Király has suggested to me that ‘movie’ would be
the closest denomination for the technical aspect and that ‘cinema’ most often
rather refers to the sociological, institutional and cultural, that is the contextual
qualifying aspect, while ‘film’ is associated with the aesthetically mature medium,
determined by the operational qualifying aspect.
37. Discussing television, Müller demonstrates how social, cultural and historical
aspects of what I would call the qualified medium of television interact with
aesthetic and communicative aspects. All these qualifying aspects are developed
well after the step-by-step emergence of the technological prerequisites of the
medium. See J. E. Müller, this volume, and idem (2008) ‘Perspectives for an Inter-
media History of the Social Functions of Television’ in J. E. Müller (ed.) Media
Encounters and Media Theories (Münster: Nodus Publikationen), pp. 201–15. Cf.
also J. E. Müller (2008) ‘Intermedialität und Medienhistoriographie’ in J. Paech
and J. Schröter (eds) Intermedialität Analog/Digital: Theorien – Methoden – Analysen
(Munich: Wilhelm Fink), pp. 31–46.
38. Cf. the genre discussion in Rajewsky, this volume.
39. If we were to assume, in a very simplified way, that the most important modes
could be isolated, say (human bodies, other demarcated materiality and not
demarcated materiality), (seeing, hearing and feeling), (space and time) and (sym-
bols, icons and indices), and that these modes could be mixed following the
principle that there must be at least one mode per modality, and that there
can also be all sorts of combinations of modes within the same modality, the
possible amount of combinations would be 7×7×3×7=1029. Needless to say, it
would be ridiculous to speak of 1029, or more, types of basic media. We have
to settle with the fact that some basic modal combinations are commonly dis-
tinguishable at a certain time and that the future may hold new conventions
and technical solutions that make novel basic media discernible, such as a basic

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 45

medium consisting of, say, not demarcated materiality that can be both seen
and felt, that is perceived as both a spatial extension and a temporal flow pro-
ducing mainly iconic meaning. Assuming that a technical medium capable of
mediating such a basic medium were invented, one may expect that a range of
qualified media soon would be developed forming aesthetic and communicative
conventions and eventually giving rise to more or less demarcated genres and
subgenres.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
40. ‘ “Intermediality” can therefore be defined as a particular relation (a relation that
is “intermedial” in the narrow sense) between conventionally distinct media of
expression or communication’: Wolf (1999) The Musicalization of Fiction, p. 37.
See also C. Ljungberg, this volume.
41. A similar conclusion is drawn by Irina Rajewsky in her essay in this volume.
42. Cf. Hans Lund’s heuristic distinction between three kinds of word–picture rela-
tions: combination, integration and transformation, in H. Lund (1992) Text as
Picture: Studies in the Literary Transformation of Pictures, translated by Kacke Götrick
(Lewiston NY, Queenston Ontario and Lampeter UK: Edwin Mellen Press), pp.
5–9. Instead of combination and integration, Claus Clüver distinguishes between
multimedia texts (separable texts), mixed-media texts (weakly integrated texts)
and intermedia texts (fully integrated texts): Clüver (2007) ‘Intermediality and
Interarts Studies’, p. 19. Wolf’s distinction between ‘overt’ and ‘covert’ interme-
diality partly corresponds to my distinction between combination and integration
of media and mediation and transformation of media. However, I find it deeply
problematic to state that an artefact of ‘overt’ intermediality is distinguished by
qualities that are ‘immediately discernible on its surface’ and by way of limit-
ing ‘overt’ intermediality to cases where ‘the signifiers of two media are apparent
and distinct’ the category becomes very narrow: Wolf (1999) The Musicalization
of Fiction, pp. 40, 50. What is, actually, the surface of a multimodal medium, and
what does it mean to say that the signifiers of a medium are apparent? Only when
it comes to technical media, actual physical objects or phenomena, is it possi-
ble to talk about overt or direct co-presence of media, and since most qualified
media are mediated by various and also altering technical media the distinctions
overt–covert and direct–indirect have limited value when it comes to qualified
media.
43. Theatre is thus definitely extremely multimodal and it integrates many basic and
qualified media, but it is an overstatement to say that ‘theatre is a hypermedium
that incorporates all arts and media’. See F. Chapple and C. Kattenbelt (2006)
‘Key Issues in Intermediality in Theatre and Performance’ in F. Chapple and
C. Kattenbelt (eds) Intermediality in Theatre and Performance (Amsterdam and New
York: Rodopi), p. 20. Cf. C. Kattenbelt (2006) ‘Theatre as the Art of the Performer
and the Stage of Intermediality’, in the same volume, p. 32.
44. Also, a simple juxtaposition involves some kind of integration. If an image and a
visual text are combined, for instance, it matters which one of them is above or
to the left of the other.
45. Mitchell mainly deals with ‘verbal and visual representation’, as in Mitchell
(1994) ‘Beyond Comparison’.
46. In his later writings, Mitchell’s notion of mixed media becomes more articu-
lated. In Mitchell (2005) ‘There are no Visual Media’, he also straightforwardly
acknowledges the differences between media and states that, ‘If all media are
mixed media, they are not all mixed in the same way, with the same propor-
tions of elements’ (p. 260). This is a very important step towards the possibility

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


46 Media, Modalities and Modes

of creating ‘a more nuanced taxonomy of media based in sensory and semiotic


ratios’ (p. 264). However, the relation between the sensorial and the semiotic
modalities (in my terminology) is not always apparent in Mitchell’s account.
He argues that ‘Subtitles, intertitles, spoken and musical accompaniment made
“silent” film anything but silent’ (p. 258), but to me it is not obvious in which
way subtitles and intertitles break the silence. Although one may take account
of the perceiver’s subvocalization, the basic difference between silent film and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
sound film remains clear, if not unambiguous. His assertion that photography is
so riddled with language ‘that it is hard to imagine what it would mean to call
it a purely visual medium’ (p. 260) implies that a particular kind of mainly sym-
bolic semiosis affects the sensorial perception, which is obviously not the case.
Symbolic language that can be directly seen on or otherwise deduced from the
screen or surface of moving or static images still has its origin in the visual (and
has effect only on our photoreceptors), although it is not iconic, in the semiotic
sense of the term – but the modes of the semiotic modality are something other
than the modes of the sensorial modality. In spite of its conventional signs, nei-
ther silent film nor photography can be heard, smelled or tasted – they can only
be seen and, in a rudimentary way felt, but that has hardly any bearing on the
aspects of texts and language. Mitchell’s important main point is, it seems, to
emphasize the blurring of modal borders in the perception of media, but I think
one must also emphasize those borders that do exist in spite of the perceptual
and cognitive operations of the recipients.
47. Cf. Müller’s distinction between ‘technical conditions’ and ‘medial products’
in J. E. Müller (1996) Intermedialität: Formen moderner kultureller Kommunikation
(Münster: Nodus Publikationen), p. 23; see also pp. 81–2.
48. As I define the notion of technical medium, it is narrower than, for instance,
the notion of ‘physical media’ as circumscribed in C. Clüver (2007) ‘Intermedi-
ality and Interarts Studies’, p. 30. Devices used for the realization of media, but
not tools used only for the production or storage of media, are technical media.
The brush and the typewriter are tools of production that are separated from the
material manifestations of media and cannot be seen as technical media accord-
ing to my definition, although they count as physical media in Clüver’s sense.
Oil on canvas and ink on paper, however, are technical media. The flute and the
video camera are partly tools for production and partly devices for the realization
of media and can hence also be seen as technical media. Some technical media,
such as ink on paper, both store and display basic and qualified media, whereas
a computer hard disk, a device for storage only, is not a technical medium in the
sense that is emphasized here.
49. See Rajewsky, this volume.
50. In other contexts, of course, it is vital to consider not only the display but also the
production and storage of basic and qualified media. When discussing qualified
media such as art forms, for instance, many distinctive features that can be seen
as operational qualifying aspects are connected to the production and storage of
media. Traditional live theatre is produced and displayed by a range of technical
media, the bodies of the actors being the most important, but it should not,
and actually it cannot, be stored. A filmed theatre performance can be stored, but
what is being stored is, as a matter of fact, not the performance, but a transformed
version with very different modal and qualified qualities. A painting, on the other
hand, is not produced by, for instance, oil paint and canvas, but the paint and
canvas both store and display the painting. A motion picture is stored by technical

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Lars Elleström 47

equipment that is connected to and yet distinct from the screen that displays a
copy of the film.
51. McLuhan (1994) Understanding Media, p. 245.
52. Cf. the much broader notion of remediation in J. D. Bolter and R. Grusin (1999)
Remediation: Understanding New Media (Cambridge MA and London: MIT Press).
53. The lack of distinction between various forms of ‘representation’, ‘remediation’
and simply ‘similarity’ is perhaps the major obstacle in Bolter and Grusin (1999)

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Remediation. It is a very inspiring book, full of interesting observations relevant
for intermedial studies, but the authors’ notions of media and remediation are
conspicuously vague. In their view, a ‘medium’ seems to be both all kinds of
modalities, as understood in the tradition of Kress and van Leeuwen, and all
kinds of media as (not) defined by McLuhan. According to Bolter and Grusin,
all sorts of media can remediate all sorts of media, whether they are techni-
cal, qualified or something else: ‘our identity’ can be remediated by the internet
(p. 231), ‘the fatal stillness of Antonioni’s films’ can be remediated by a computer
game (p. 268) and ‘the printed book’ is remediated by hypertext (p. 272). Alto-
gether, their account gives a good view of the complexity of media relations, but
no theoretical tools to deal with it. Cf. J. Schröter (2008) ‘Das ur-intermediale
Netzwerk und die (Neu-)Erfindung des Mediums im (digitalen) Modernismus:
Ein Versuch’ in J. Paech and J. Schröter (eds) Intermedialität Analog/Digital: The-
orien – Methoden – Analysen (Munich: Wilhelm Fink), pp. 579–601, whose notion
of ‘Transformational-ontologische Intermedialität’ is severely limited by its close
association to the notion of representation (pp. 589–90).
54. There has been some debate concerning the proper delimitations of the notion
of ekphrasis. In S. Bruhn (2000) Musical Ekphrasis: Composers Responding to Poetry
and Painting (Hillsdale NY: Pendragon Press), the author demonstrates convinc-
ingly that the notion has an unexplored capacity to explain much more than
literary transformations of images. Cf. S. Bruhn (ed.) (2008) Sonic Transformations
of Literary Texts: From Program Music to Musical Ekphrasis (Hillsdale NY: Pendragon
Press).
55. Yvonne Spielmann discusses several ways of understanding intermedial trans-
formation in Y. Spielmann (1998) Intermedialität: Das System Peter Greenaway
(Munich: Wilhelm Fink). However, her main arguments are based on notions
and distinctions that I find problematic for reasons declared earlier in this essay:
‘textuality’ versus ‘visuality’ and ‘monomediality’ versus ‘multimediality’ versus
‘intermediality’.
56. I. O. Rajewsky (2002) Intermedialität (Tübingen and Basel: A. Francke Verlag),
pp. 12–3, 16–7. Rajewsky does not consider ‘Transmedialität’ to be a case of
intermediality. Cf. the notion of ‘intermedial translation’ introduced by Regina
Schober in this volume.
57. The term ‘media circuit’ is telling. It is used in M. P. Punzi (ed.) (2007) Literary
Intermediality: The Transit of Literature through the Media Circuit (Bern: Peter Lang),
where ‘intermediality’ mainly refers to the phenomenon here characterized as
the transformation of media. Cf. the way the notion of irony is transformed
in the discourses on various arts, as described in L. Elleström (2002) Divine
Madness, or the way narration can be understood in the context of different
media; Marie-Laure Ryan talks about ‘transmedial narratology’ in M.-L. Ryan
(2004) ‘Introduction’ in M.-L. Ryan (ed.) Narrative Across Media: The Languages of
Storytelling (Lincoln and London: University of Nebraska Press), p. 35. The trans-
medial notions of complexity, integration and rhythm are discussed as examples

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


48 Media, Modalities and Modes

of ‘inter-medial factors’ in T. M. Greene (1940) The Arts and the Art of Criti-
cism (Princeton: Princeton University Press), pp. 213–26. Ultimately, also notions
such as reception and interpretation are transmedial, of course, and it is vital to
recognize that interpretive strategies, contextualizations and ways to communi-
cate the outcome of interpretation cross all media borders. This is a crucial point
in Bal (1991) Reading Rembrandt.
58. This problem is emphasized in Müller (1996) Intermedialität, pp. 81–2.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
59. Cf. the enlightening overview of intermedial positions and issues in I. O.
Rajewsky (2005) ‘Intermediality, Intertextuality, and Remediation: A Literary
Perspective on Intermediality’, Intermédialités (6), pp. 43–64.
60. This is very much stressed by Jørgen Bruhn in his contribution in this volume.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Part II
Media Borders of Qualified Media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
This page intentionally left blank

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


2
Border Talks: The Problematic Status
of Media Borders in the Current
Debate about Intermediality

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Irina O. Rajewsky

The debate about intermediality is characterized by a variety of


heterogeneous approaches, spanning a wide range of subject matter and
research perspectives. A large number of critical approaches make use of
the concept, each with their own premises, methodology, terminology
and delimitations. Likewise, the specific objectives pursued by different
disciplines in conducting intermedial research (for example, media stud-
ies, literary, theatre and film studies, art history, musicology, philosophy
or sociology) vary considerably. While some approaches focus on general
media-historical developments or genealogical relations between media, on
medial transformation processes, on the very formation of a given medium
or on the process of medialization as such, others aim at questions of
media recognition (Medienerkenntnis) or at understanding general functions
of media. Further approaches, mostly coming from the realm of literary
studies and related fields, such as my own, emphasize various forms and
functions of concrete intermedial practices in specific individual texts, films,
theatre performances, paintings and so on. Considering this background, it
is not surprising that the question – still declared ‘fundamental’ in 2001 – of
what ‘the concept of intermediality actually means’1 now seems outmoded
and has been reformulated with regard to various intermediality conceptions
and their respective heuristic potential.2
While significant differences between the various research traditions
become apparent as soon as one looks at them in some detail, there still
seems to be a (more or less) general agreement on the definition of interme-
diality in its broadest sense. Generally speaking, and according to common
understanding, ‘intermediality’ refers to relations between media, to medial
interactions and interferences. Hence, ‘intermediality’ can be said to serve
first and foremost as a flexible generic term ‘that can be applied, in a broad
sense, to any phenomenon involving more than one medium’3 and thus to
any phenomenon that – as indicated by the prefix inter – in some way takes

51

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


52 Media Borders of Qualified Media

place between media. Accordingly, the crossing of media borders has been
defined as a founding category of intermediality.4
Obviously, such a broad definition of intermediality, which seeks to do
justice to this concept precisely in its quality as collective term, ultimately
tends to be rather bland.5 Nevertheless, it is worthwhile to briefly ponder
this broad definition of intermediality because it steers our attention to one

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of the central basic assumptions of intermediality research, which is also the
focus of this essay. A broad conception of this kind – and, as a consequence,
any more narrowly defined conception of intermediality – evidently pro-
ceeds from the assumption of tangible borders between individual media, of
medial specificities and differences. In fact, any reference to intermediality
implicitly presumes that it is indeed possible to delimit individual media,
since we can hardly talk about intermediality unless we can discern and
apprehend distinguishable entities between which there could be some kind
of interference, interaction or interplay.6 Quite recently, however, it is pre-
cisely this fundamental premise of discernible media borders that has been
called into question. As a consequence, the very concept of intermediality
itself has come under scrutiny.
There are two principal arguments, frequently entangled, that are
advanced against the aforementioned premises. Firstly, approaches to inter-
mediality are criticized for widely ignoring the constructed character of
any conception of ‘a medium’ and of any reference to ‘individual media’
(Einzelmedien) that, from the other point of view, are conceived of as purely
discursive strategies.7 Accordingly, approaches to intermediality are quite
frequently associated with dubious essentialist views.8 Secondly, the assump-
tion of medial delimitations and the criterion of a medial border crossing are
called into question by referring to various kinds of performances or artistic
‘events’ of the past decades that manifest ‘a still growing tendency towards
an annulment, a dissolution of the boundaries between different art forms’.9
As the synopsis of the 2006 conference of the German Society for Theater
Studies put it:

After decades in which film and video have been seen with such fre-
quency on stage . . . and in which the traditional borders of the arts
disciplines have often been completely blurred, we have to ask whether
the criterion of medial border crossing can still be so readily applied.10

Similarly, according to Erika Fischer-Lichte, the concept of intermediality


quite problematically ‘presupposes that it is possible to distinguish clearly
between the different media in play’.11 It is for this reason that Fischer-Lichte
is reluctant to adopt approaches of this kind for the investigation of the
artistic practices referred to above.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 53

I do not intend to rebut the aforementioned basic considerations with


which, taken as such and detached from any judgmental conclusions, I think
everybody will agree. The assumption of media borders and of medial delimi-
tations, and (along with it) any reference to ‘individual media’ should indeed
be handled with care, and it is true that quite a few approaches to intermedi-
ality, especially approaches coming from literary studies, often lack a careful

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
discussion of media theories. At the same time, however, I would like to
point out that any kind of theoretical dismantling of the term ‘intermedial-
ity’ is confronted with concrete intermedial practices in the arts for which, as
I will illustrate later on, media borders and medial specificities are indeed of
crucial importance. As I will show in more detail, this is not contradicted
by those medial configurations which tend to show the aforementioned
tendency towards the blurring or dissolving of borders between different
media or art forms. Likewise, with reference to theatre, the fact that films
and videos have been seen on stage for decades and that digital techniques
are increasingly used hardly gives cause to renounce the basic assumption
of current concepts of intermediality regarding the crossing of media bor-
ders. On the contrary, the fact that theatre is able to integrate various medial
forms of articulation and to present them on stage is made possible precisely
by the medial conditions and the fundamentally plurimedial structure of
this medium. Despite all medial expansion, theatre is still conventionally
perceived – and has been perceived for centuries – as a distinct individual
medium. It thus has medially based as well as conventionally drawn borders
(which are obviously subject to historic transformation and must in part be
seen as fluid).
This calls to our attention crucial aspects which should be considered with
respect to the fundamental question of the delimitability of individual media
and thus with respect to the status of medial specificities, differences and
borders in the context of intermedial practices. The first aspect is the specific
way in which medial differences, borders and the crossing of borders come
into play in a given medial configuration. The second aspect concerns the
historical processes of the development and differentiation of so-called indi-
vidual media. Finally, there is the justifiably designated ‘construct’ character
of media conceptions.
As I have elaborated in prior publications,12 in this context we should first
of all bear in mind that in dealing with medial configurations, we never
encounter ‘the medium’ as such, for instance, film as medium or writing
as medium, but only specific individual films, individual texts and so on.
Whether, following the work of Niklas Luhmann, we distinguish between
‘medium’ and ‘form’, as proposed, for example, by Joachim Paech,13 whether
we distinguish between ‘medium’ and ‘medial configuration’, as I will be
doing, or whether we use other terms and concepts, we must necessarily
take into account that we always only encounter concrete medial forms of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


54 Media Borders of Qualified Media

articulation, which moreover are characterized by a multilayered and mul-


timodal complex mediality. Already this very basic and simple observation
leads to the conclusion that to speak of ‘a medium’ or of ‘individual media’
ultimately refers to a theoretical construct, to a ‘theoretical abstraction’
(Abstraktionsleistung), as Sybille Krämer calls it.14 Moreover, even a slightly
more precise engagement with the variability of media conceptions brings

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
to light the constructed character of each of them. As the debate stands
today, it is hardly necessary to discuss the issue further. The question of
how a medium should be defined and delimited from other media is of
course always dependent on the historical and discursive contexts and the
observing subject or system, taking into account technological change and
relations between media within the overall media landscape at a given point
in time.
Analogous conditions apply, for example, to genre conceptions, to intro-
duce a comparative parameter, and yet, as opposed to recent questionings of
the concept of intermediality, not only is a heuristic potential still attributed
to conceptions of genres as well as of genre mixes or of an undermining
of generic boundaries. Rather, literary studies, for example, have sufficiently
clarified that – in spite of their constructedness and historical variability –
genre conventions, just like discourse traditions, play a decisive role in con-
veying meaning to literary texts. This holds true both for their production
and for their reception.
As these considerations already indicate, neither the fact that we are
always dealing only with specific individual medial configurations, nor the
constructedness and historicity of media conceptions, should lead us to
the conclusion that we ought to cease altogether to speak of (historically
transformable) medial specificities and differences, of media borders and
eventually of intermedial strategies and practices. Rather, we should ask our-
selves what exactly we mean when we talk about ‘individual media’, medial
specificities or of crossing media borders in this context. Drawing borders of
this kind clearly cannot be a matter of ‘fixed’ and ‘stable’ borders between
‘fixed’ and ‘stable’ entities, but if not this, what then? This is a question
I shall investigate in more detail, starting from concrete artistic intermedial
practices. For this purpose I first introduce some basic differentiations com-
monly used in current approaches to intermediality, albeit with different
terminology and subcategories.

Even if we confine ourselves to an understanding of intermediality as a crit-


ical category for the concrete analysis of individual medial configurations,
as I will be doing, one still has to cope with a vast and quite heteroge-
neous range of subject matter. In fact, in literary studies as well as in fields
such as art history, music, theatre and film studies, there is a repeated
focus on an entire range of phenomena qualifying as intermedial. Exam-
ples include those phenomena which for a long time have been designated

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 55

by terms such as filmic writing, ekphrasis, musicalization of literature, as


well as such phenomena as film adaptations of literary works, novelizations,
visual poetry, illuminated manuscripts, Sound Art, opera, comics, multime-
dia shows, multimedial computer ‘texts’ or installations and so forth. All
of these phenomena have to do in some way with a crossing of borders
between media and are in so far characterized by a quality of intermedi-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ality in the broad sense. However, it is also immediately apparent that the
intermedial quality of a film adaptation, for example, is hardly compara-
ble – or is comparable only in the broadest sense – with the intermediality
of so-called filmic writing and that both of these are quite distinct from, say,
book illustrations or Sound Art installations. If the use of intermediality as
a category for the description and analysis of particular phenomena is to
be productive, we should therefore distinguish groups of phenomena, each
of which exhibits a distinct intermedial quality and – what is even more
important in the present context – a particular way of crossing media bor-
ders. This allows for drawing distinctions between individual subcategories
of intermediality and for developing a uniform theory for each of them. At
the same time, it shows that any reference to ‘the criterion of a medial bor-
der crossing’ blurs fundamental differences between specific ways in which
such a border crossing manifests itself in given intermedial practices. Such
differences have implications both for theorizing the phenomena in ques-
tion as well as for the concrete analysis of given medial configurations and
their overall signification of course, in the concrete analysis also differing
functions of intermedial strategies have to be taken into account. A closer
look at the intermedial practices in question points to three groups of
phenomena:15

1. Intermediality in the narrower sense of medial transposition (Medien-


wechsel), also referred to as medial transformation, as, for example, film
adaptations of literary texts, novelizations and so forth.
2. Intermediality in the narrower sense of media combination (Medien-
kombination), which includes phenomena such as opera, film, theatre,
illuminated manuscripts, computer or Sound Art installations, comics,
or, to use another terminology, so-called multimedia, mixed-media and
intermedia forms.16
3. Intermediality in the narrower sense of intermedial references (interme-
diale Bezüge), for example, references in a literary text to a specific film,
film genre or film qua medium (that is, so-called filmic writing), like-
wise references in a film to painting, or in a painting to photography
and so on.

A somewhat more detailed scrutiny of these three groups of phenom-


ena reveals that we are dealing here with qualitatively different concep-
tions of intermediality. The first category, medial transposition, entails

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


56 Media Borders of Qualified Media

a production-oriented, ‘genetic’ conception of intermediality. Here the


intermedial quality – the criterion of a medial border crossing – has to do
with the way in which a medial configuration comes into being, that is,
with the transformation of a given medial configuration (a text, film and
so on) or of its substratum into another medium. The ‘original’ text, film
and so on, is the ‘source’ of the newly formed medial configuration, whose

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
formation is based on a media-specific and obligatory intermedial transfor-
mation process. The category of medial transposition can therefore, to use
Werner Wolf’s terminology, also be described as a form of ‘extracomposi-
tional intermediality’.17 Unlike medial transposition, both the second and
the third category, media combination and intermedial references, aim at
an intracompositional intermediality, that is, at a ‘direct or indirect partic-
ipation of more than one medium’ not only in the formation process, but
‘in the signification and/or structure of a given semiotic entity’.18 Conse-
quently, there is a fundamental difference between medial transposition on
the one hand and media combination and intermedial references on the
other. As Wolf goes on to say – and what is particularly relevant for analy-
sis – ‘[e]xtracompositional intermediality as such does not necessarily affect
the meaning or outer appearance of particular works or performances, while
intracompositional intermediality does’.19
If we focus on media combinations and intermedial references, that is, on
the different forms of intracompositional intermediality, we note a further
significant difference for which once again the moment of crossing media
borders becomes most important. This can already be derived from Wolf’s
assertion cited above: the issue is whether we are dealing with a direct or an
indirect ‘participation of more than one medium in the signification and/or
structure of a given semiotic entity’. In the following I will illustrate this
central point with examples from dance theatre and photorealistic painting.

Film, theatre, opera or, more recently, Sound Art, are evidence that media
combinations, from a historical perspective, quite frequently result in the
development of new forms which somewhere in the course of this pro-
cess are themselves conventionally perceived as distinct art or media genres.
The plurimedial structure, then, is a characteristic and constitutive feature
of these newly emerged genres. Correspondingly, dance theatre is defined
by a plurimedial structure which manifests itself, not least, in the combi-
nation of theatrical and dance elements and structures. This fundamental
plurimediality can, of course, be augmented, as has been shown especially
in more recent productions, which increasingly tend to involve digital and
other technical media. One example is Wim Vandekeybus’ production Blush
(Wim Vandekeybus/Ultima Vez, Brussels 2002). This production involves
pre-produced film sequences that are projected on a huge screen during the
live performance. In these film sequences the ensemble’s dancers are seen
swimming and acting under water. Drawing on the identity of the dancers

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 57

on stage and in the filmic underwater world, the interplay of film and live
performance creates highly effective moments: repeatedly dancers seem to
jump directly ‘into’ the screen and thus seemingly ‘into’ the film, where,
now in their filmic embodiment, they seem to continue their movements
without interruption. This effect is made possible by a projection screen that
actually consists of several panels put next to one another, leaving some

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
‘slots’ that allow the dancers to jump behind the screen. Yet, since the live
action on stage and the pre-produced action on film are exactly synchro-
nized, the dancers effectively seem to jump into the filmic underwater world
and thus into the water; an impression that is enhanced by visual and audi-
ble ‘splash’ effects occurring in the film sequences when the dancers seem to
hit the water. In this way the identity of the dancers on stage and on film, as
well as their seemingly continuous movements, create the illusion of a con-
tinuity of what happens on stage and on film. At the same time, two worlds,
two medial ‘realities,’ two time levels and two medial forms of embodiment
are set against one another.
Focusing on the distinction between different categories of intermedial-
ity outlined above, it is important that in this case – an instance of media
combination – the various medial forms of articulation are all present in
their own materiality and contribute to the constitution and signification
of the entire performance in their own media-specific way. Here, applying
Wolf’s terminology, several media come into play in a direct way. Such a way
of employing and staging media in dance performances is in stark contrast
to a sequence from the dance theatre production Bodies (Körper) by Sasha
Waltz (premiered in Berlin, 2000).20 At the beginning of this sequence, a
huge picture-frame-like construction is erected on the stage, equipped with
a transparent front and an opaque panel in the back. Positioned between the
transparent front pane and the back panel, and supporting themselves in the
air by pressing their limbs against the two supports, the dancers move very
slowly, heads up and heads down, in every possible direction; as if weight-
less and freed from the necessity to touch ground. With several other factors
contributing to its overall effect, such as the lighting, the dancers’ costumes
recalling loincloths and bodies seemingly cut off at the borders of the frame,
this sequence as a whole reminds the viewer of a painting, maybe even more
specifically of a mannerist one.
Here, different medial forms of articulation are not combined with one
another, as in media combination; instead, the means and instruments of
dance theatre itself – bodies, costumes, movements, lighting, stage props and
so on – are employed and fashioned in a way that corresponds to and resem-
bles elements, structures and representational practices of painting, thus
creating an illusion of painterly qualities. (Put in terms of cognitive research,
the spectator is cued to apply a painterly bound frame.) The evocation of
painting is not achieved simply by means of subjective associations that
may (or may not) be elicited in the spectator’s mind. Rather, the placement

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


58 Media Borders of Qualified Media

onstage of the oversized frame – a device that is iconically related to a picture


frame and that effectively frames the action taking place onstage – explicitly
designates painting as the medial system being referred to and thus marks
the overall mise-en-scène as an intermedial reference to painting. In this way,
the sequence as a whole constitutes itself (and is received by the viewer) in
relation to painting, simulating but at the same time also expanding the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
representational modes of the medium referred to. It is as if dance theatre
turned into painting, yet explicitly pointing to a medial difference, to its
own mediality and to the so-called ‘as if’ character of the whole procedure:
it is as if we saw a painting that, supplementing pictorial stasis with the
movements of the dancers, is not only populated by bodies in flesh and
blood, but also put into motion: a tableau vivant animé.
The same kind of intermedial technique underlies photorealistic painting,
to provide a further example with which the basic mechanisms of interme-
dial references can be visualized particularly well.21 Here again, it is not two
or more different forms of medial articulation that are present in their own
specific materiality. Instead, what we are dealing with is nothing other than
painting – but a kind of painting which inevitably evokes in the viewer the
impression of a photographic quality.22 Evidently, here too another medium
is ‘brought into play’, but this only in an indirect way, as in the case of
the sequence of Sasha Waltz’ Bodies. It is not photography which manifests
itself materially; rather painting’s own instruments and means are applied
and shaped in such a way that experiences, or ‘frames’, are evoked in the
observer that are medially bound to photography, leading to an illusion, an
‘as if’, of a photographic quality. Consequently, in the case of intermedial
references, only one conventionally distinct medium manifests itself in its
specific materiality and mediality. Historically, this medium may of course
have emerged through the conflation of different medial forms of articu-
lation and thus, as in the case of dance theatre, can exhibit a (potentially
expandable) plurimedial structure.
Thus, as already indicated above, the moment of crossing media borders
in the case of intermedial references is brought to bear in quite another
way than with media combination. A given photorealistic painting as such
exhibits less a crossing than a ‘playing around’ its own medial borders in the
direction of the system referred to: photorealistic painting constitutes itself
in relation to photography and appears to us ‘as a photo’, but it still remains
a painting. However, this effect comes about precisely due to the fact that it
refers to another medial system: with the medium’s own specific means and
instruments, elements and/or structures of another conventionally distinct
medium are thematized, evoked or, as in the case of the sequence from Bod-
ies and photorealistic painting, simulated23 – and this is certainly the most
interesting variant of this kind of intermedial strategy. The media borders’
cross-over moment in the case of intermedial references therefore does not
affect the material manifestation of various media within a given medial

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 59

configuration, but rather the specific quality of the reference itself. No matter
how corresponding strategies in a given medial configuration may be func-
tionalized in detail, in any case additional layers of meaning will be opened
up in this way which must be taken into account in the analysis.

In the sequence of Bodies and in photorealistic painting, the medial differ-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ence between the referencing medium and the medium referred to (that is,
between live performance and painting, and between painting and photog-
raphy, respectively), becomes apparent in quite an obvious way, as is typical
of intermedial references in general. Yet, especially in the domain of so-called
media combinations, we can also find concrete intermedial configurations
that do in fact, quite explicitly, show the constructedness of medial delimi-
tations and thus of any notion of an ‘individual medium’. Here, we must
consider the wide range of possible realizations of media combinations,
reaching from a contiguity or coexistence up to a more or less ‘genuine’
integration or interplay of the medial forms of articulation which in its
‘purest’ form would privilege none of its constitutive elements.24 Depend-
ing on the specific form of intermedial relations, such interplay may be
experienced as a synthesis or fusion of different modes of medial articula-
tion, but also as an oscillating ‘in-between-ness’, something actually situated
between two or more medial forms. The latter becomes especially apparent
in certain kinds of Sound Art installations, for instance, installations of the
Hamburg artist Andreas Oldörp. In Oldörp’s works sounds are created in a
natural, mechanical way, using either air- or steam-driven organ pipes or gas
flames in glass cylinders, so-called singing flames.25 Thus, in different con-
stellations, acoustic architectures develop that are distinguished to a great
extent through their hybrid quality. First and foremost, this is achieved in
that the sculptural, material constructions in the given space are neither
combined with independent sounds nor – as in many other sound installa-
tions – with sounds (re)produced electronically. Rather, the ‘sculpture’ itself
simultaneously functions as a sound-producing instrument. The recipient’s
apprehension of sound and his or her perception of the sculptural object are
in fact no longer separable from each other. The material constructions are
always at the same time ‘sculpture’ and sound instrument.
Hence, in such Sound Art works architectural space, material object and
sound (and ultimately the peripatetic recipient him or herself), are related in
such a way that the resulting effect exists and can only ever be experienced
in the ongoing process of their interplay, as transitory and not repeatable.
What develops, then, is less the impression of a synthesis, or ‘merging’, of
the various medial aspects and qualities than the impression of an oscillat-
ing ‘between-the-media’. The movements of the visitors in the architectural
space – their approaching some of the sound-producing objects while mov-
ing away from others – are continuously accompanied by changes of the
sound and of the sound perception. Thus, the medial aspect of sound, its

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


60 Media Borders of Qualified Media

materiality and mechanical generation, is again and again raised into the
visitors’ consciousness. By the same token, the sculpture-like material con-
structions are held present in their visual-aesthetic dimension and in their
relation to the architectural space, but they are at the same time also the
source of sound. In this way, it is precisely the smooth transitions and
consequently the impossibility of a clear demarcation between the media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
involved that become evident here. In that the visitors’ seeing and hearing
habits are irritated and raised into their consciousness, in that space, sound
and material constructions are related to each other in an unusual way and
can be apprehended only through their interplay, any kind of ‘clear’ borders
between individual media – and equally of distinct qualities in our sensual
perception – is presented as being constructed (as opposed to being ‘naturally
given’).
To recall the arguments that have been advanced against the premise
of discernible media borders, what we see here is an artistic practice that
definitely manifests the above quoted ‘still growing tendency towards a
dissolution of the boundaries between different art forms’ or media. In
fact, media combinations expose – or at least can expose – the construct-
edness of delimitations of individual media. At the same time, however,
it is also the oscillating interplay between two or more ‘entities’ as such
which media combinations of this kind bring to the recipient’s conscious-
ness. What I would like to emphasize here is that this oscillation per se,
and any apprehension of it, necessarily presumes commonly held distinc-
tions between the different media in play. If boundaries between different
art forms are said to be dissolved, and if delimitations of media are reflected
upon as constructs, what is necessarily presumed in the first place are a priori,
conventional delimitations of those media or art forms.

With this we arrive at the following point: concrete intermedial configura-


tions show, for one, the constructedness of medial delimitations and (along
with it) of any reference to ‘individual media’. For another, however, we
may also conclude that by means of intermedial strategies, the possibility
per se of delimiting different media re-enters the picture; that is, we may
conclude that the ‘idea’ of one or another individual medium can be, and
actually frequently is, called up in the recipient. We have seen this in an espe-
cially evident way in the case of the reference to painting in Bodies and to
photography in photorealistic painting, references which in their functional
potential necessarily depend on a perceptible medial difference.
I am deliberately emphasizing the notion of an ‘idea’ (Vorstellung) – not
least on the basis of conventional attributions – which the recipient asso-
ciates with a given medium. This is because the ‘idea’ of a particular medium
is relevant to the general question of how a medium can be defined and
discerned from other media, of how, for instance, the ‘filmic’, ‘painterly’
or ‘musical’ quality can be defined to which a given medial configuration

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 61

could refer. It is obvious that the answer to this question necessarily depends
on the historical and discursive contexts and on the observing subject or
system; furthermore, we must take into account the historical and contex-
tual variability and conventionality of any defining characteristics attributed
to a given medium. Hence, if I have been speaking all along of conven-
tional delimitations and conventionally distinct media, I have been doing

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
this very consciously. In my view, the functioning of intermedial config-
urations is always based on relations between media or ‘medialities’ that
are conventionally perceived as distinct,26 or, to put this in other terms, it is
based on the possibility of calling up specific medially bound frames in the
recipient. Despite their conventionality and constructedness, these variable
conventional ‘ideas’ and concepts associated with specific individual media
are nevertheless at one’s disposal, both for the production and inner func-
tioning and the reception of a given medial configuration. Conventional
and constructed as they may be, they are still available for partaking in
the constitution of a media product’s overall signification. Who (at least
in the Western world), to give just one more example, does not think of
theatre when watching Lars von Trier’s Dogville (Denmark 2003)? And who
does not, at the same time, apprehend the filmic quality of this interme-
dial experiment that draws, among other things, precisely on the medial
difference between film and theatre?
In fact, even Sound Art, by emphasizing the constructedness of medial
delimitations, at the same time uncovers such delimitations as they are con-
ventionally drawn (at least at a certain, and moreover context-related, point
in time). Ultimately then, here too, there is assumed a delimitability, or bet-
ter, an established, conventional delimitation of media. If the drawing of
borders between media is reflected upon as a construct and finally transferred
into an oscillating ‘between-the-media’, which applies at least to Oldörp’s
works, this nonetheless refers back at the same time to conventionally drawn
borders between the media. What is thus reflected upon, and what is in
fact made useful for the potential effects of a Sound Art installation on the
recipient, are prior, commonplace ideas about ‘individual media’ and medial
differences, that is, specific medially-bound frames which are called up in
the recipient, but which can also be modified and displaced, not least by the
medial performance itself.27
Moreover, the example of Sound Art shows the important role played by
processes of habitualization and conventionalization in relation to draw-
ing medial borders: if works of art that are now labelled ‘Sound Art’ were
initially seen as an attempt to embark in a new direction and to disrupt
established borders (also and particularly by the artists themselves) and
could thus be understood as an artistic practice taking place at the mar-
gins of current artistic and media conceptions, most recently Sound Art
is perceived as an increasingly established art and media genre in its own
right. While for this genre the interplay of sound and material constructions

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


62 Media Borders of Qualified Media

still remains constitutive, such interplay is meanwhile hardly perceived as


a destabilization or disturbance of established borderlines, conventions and
norms. Rather, the combination of sound and material constructions has
since become a ‘normal’, established element of a genre, which like all other
genres has certain prescriptive and restrictive rules. Hence, with the con-
ventionalization and establishment of Sound Art as an art and media genre

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
which in itself is conventionally perceived as distinct, the original destabiliz-
ing and disturbing moment of this form of medial border crossing has been
lost or, through habitualization, has at least been muffled. Thus, diachron-
ically viewed, practices of border crossings or of dissolutions of established
borders – as far as they are accompanied by a sustainable conventionalization
and habitualization – may result in other constructions, other borders that
again are perceived as conventional, and in turn modified or even entirely
new conceptions of individual media and art forms.

Up to this point, I have been mainly focusing on the constructed character


of any conception of ‘a medium’ and on the fact that when we talk about
‘individual media’ we are actually talking about media that are convention-
ally perceived as distinct. These considerations can be taken one step further
by highlighting an aspect that is usually overlooked, or at least not made
explicit, in related arguments. Here again, intermedial references come to
the fore, such as the reference to painting in Sasha Waltz’ Bodies or the ref-
erence to photography in photorealistic painting. The above remarks on
these examples concentrated on differences between instances of media
combination and intermedial references. Yet, those remarks actually not
only delineate the features of intermedial references as opposed to prac-
tices of media combination, but also as opposed to intramedial references, as
for instance intertextual relations, or, say, film–film and music–music rela-
tions. In this context the criterion of a medial border crossing is again of
primary importance. In the case of intramedial references the referencing
itself remains within one medium and consequently does not involve any
kind of medial difference. Hence, intramedial references, quite significantly,
do not come along with a medial border crossing. Instead, in the case of
intermedial references a medial difference does come into play; and more pre-
cisely, a medial difference that – as a matter of fact – cannot be effaced. What
can be achieved by intermedial references is an (more or less pronounced,
yet necessarily asymptotical) approximation to the medium referred to; an
overall actualization or realization of the other medial system is impossi-
ble. Due to its material and medial conditions, dance theatre cannot truly
become painting – just as painting itself can never become genuinely pho-
tographic, even though this is suggested at times by photorealistic painting.
Here medial specificities and borders emerge, which make clear that certain
basic medial constraints must be considered. In spite of all the construct-
edness and conventionality of the derived medial conceptions, these basic
medial constraints cannot be neglected.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 63

At the same time such medial constraints point to the fundamentally dif-
ferent status of medial restrictions and possibilities on the one hand and
generic restrictions and possibilities on the other. Thus, media specificities
can be differentiated from genre specificities. While genre specificities and
the prescriptive and restrictive rules of a genre are based solely on conven-
tions that can be played with, undermined and transcended without any

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
problems, medial specificities entail material and operative restrictions that
can be played with, but cannot be undermined with the use of the respective
media-specific means and instruments. To stress this again, painting cannot
become genuinely photographic, just as literary texts cannot really become
filmic or musical. What can be achieved in this respect is only an illusion,
an ‘as if’ of the other medium.
Consequently, if I have been emphasizing the constructed character of any
conception of ‘a medium’, this constructedness itself is actually confronted
with some limits; limits that are inherent to the materiality and mediality
of any form of medial articulation. Accordingly, my above remarks on the
potential capacity of intermedial practices to alter established prior ‘ideas’
of a given medium need to be specified: Intermedial strategies, if deployed
in the appropriate manner, can indeed be the means through which com-
monplace ideas of a given medium as well as conventionally drawn borders
between, and delimitations of, different media can be apprehended, criti-
cally reflected upon, and even displaced and undermined – yet always only
in keeping with the basic material conditions of the medial configurations
in question. In contrast to genres, conceptions of (individual) media are
not based solely on conventions. Rather, they are additionally based on
given material and operative conditions, which to a large extent are sub-
ject to historical, often technological, change, yet to some extent are also
transhistorically valid. In fact, we note a dependency between genre con-
ventions and their respective medial disposition: The emergence of certain
genre conventions is at least also bound to technical requirements and thus
to medially determined limits and possibilities underlying the respective
genre.28

Putting medial boundaries, differences and differentiations at the centre of


interest as I have been doing may be conceived as ‘strange’ or somehow
démodé, as there seems to be a general tendency in recent research to discard
thinking in differences as ontological or essentialist and thus outdated. Cur-
rently, efforts are being made to strengthen common and crossover features –
not only in intermediality studies, but also, for instance, in the young field
of so-called transgeneric and transmedial narratology.29 Contrary to this ten-
dency, I have advanced the thesis that medial differences and the notion of
media borders play a crucial and extremely productive role in the context of
intermedial practices. In this context it should be emphasized again that the
premise of discernible media borders and of the possibility of distinguishing

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


64 Media Borders of Qualified Media

between individual media, ultimately inherent to any conception of inter-


mediality, is not challenged even by such artistic or cultural practices that
show the tendency to blur, or to entirely dissolve and overcome, estab-
lished borders and delimitations of different art forms and media. Rather,
the respective practices constitute themselves – either intentionally or, so to
speak, necessarily – in relation to just these established borders and delim-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
itations. In other words, they necessarily constitute themselves in relation
to, and within the scope of, the overall medial and discursive landscape at a
given point in time, including the respective delimitations of conventionally
distinct art forms and media.
Thus, the ‘construct’ character and the historical variability of media con-
ceptions are evoked, aspects which intermediality research must of course
take into account. This is exactly why I have been speaking of individual
media that (at a given point in time) are conventionally perceived as distinct,
and of medially bound frames. These frames – if marked in an appropri-
ate manner – can be called up in the recipient and as such are available
for partaking in the constitution of a medial configuration’s overall signi-
fication. At the same time, however, we must also take into account that
any artistic or, more generally speaking, any cultural practice depends on,
and is determined by, its very mediality and materiality, rendering the con-
structedness of media conceptions and especially the constructed character
of medial delimitations at least in part relative. Borders between different
medial forms of articulation cannot ‘be drawn differently’ in every respect 30 -
and it is especially for this reason that media conceptions, as constructed
and conventional as they may be, still can be assigned a different status
from genre conceptions.
Considering the initial question and main thesis of this essay, what is most
important here is that, independently of which particular kind of inter-
medial practices we are dealing with, the effect potential of intermedial
practices is always in some way based on medial borders and differences.
Hence, in contrast to recent challenges concerning the criterion of a medial
border crossing, on the basis of concrete intermedial practices, and thus
starting from the objects of investigation as such, it is precisely the concept
of the border which can be strengthened. In my view, the concept of a bor-
der is the precondition for techniques of crossing or challenging, dissolving
or emphasizing medial boundaries, which can consequently be experienced
and reflected on as constructs and conventions. It is only due to our con-
structing borders in the first place that we are able to become aware of ways
of transcending or subverting those very boundaries or of ways of highlight-
ing their presence, of probing them, or even of dissolving them entirely. At
the same time, it is precisely these acts of transcending, subverting, probing
or highlighting which draw attention to the conventionality and (relative)
constructedness of these boundaries. My thesis thus encompasses the idea
of fostering a process of rethinking the notion of boundaries: it should be

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 65

shifted from taxonomies to the dynamic and creative potential of the bor-
der itself. The borders or – perhaps better – ‘border zones’ between media
can thus be understood as enabling structures, as spaces in which we can
test and experiment with a plethora of different strategies.

Notes

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
1. B. Ochsner and C. Grivel (2001) ‘Einleitung’ in B. Ochsner and C. Grivel (eds)
Intermediale: Kommunikative Konstellationen zwischen Medien (Tübingen: Stauffen-
burg), p. 4; my emphasis.
2. Cf. in more detail I. Rajewsky (2008) ‘Intermedialität und remediation: Über-
legungen zu einigen Problemfeldern der jüngeren Intermedialitätsforschung’ in
J. Paech and J. Schröter (eds.) Intermedialität analog/digital: Theorien, Methoden,
Analysen (München: Fink).
3. W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History of
Intermediality (Amsterdam and Atlanta GA: Rodopi), pp. 40–1.
4. Cf. I. Rajewsky (2002) Intermedialität (Tübingen and Basel: Francke), pp. 11–15.
5. A broad intermediality concept of this kind allows for making fundamental dis-
tinctions, namely between intra, inter and (ultimately) transmedial phenomena,
at the same time representing a transmedially useful category. Yet such a broad
concept does not permit us to derive a single theory that would uniformly apply
to the entire, heterogeneous subject matter covered by all the different concep-
tions of intermediality, nor does it help us to characterize more precisely any
one individual phenomenon on its own distinct formal terms. Accordingly, in
order to cover and to uniformly theorize specific intermedial manifestations, more
narrowly conceived (and often mutually contradictory) conceptions of interme-
diality have been introduced, each of them with its own explicit or implicit
premises, methods, interests and terminologies (cf. in more detail I. Rajewsky
(2005) ‘Intermediality, Intertextuality, and Remediation: A Literary Perspective
on Intermediality’, Intermédialités/Intermedialities 6, pp. 43–65).
6. The same is applicable for any reference to ‘interart relations’, as talking about
interart relations likewise presupposes discernible borders between different art
forms.
7. This was, for instance, one of the theses of the final panel discussion of the
8th Biennial International Conference of the German Society for Theater Studies
with a special focus on ‘Theater & the Media’, held at Erlangen, 12–15 October,
2006; cf. H. Schoenmakers et al., (eds) (2008) Theater und Medien/Theater and
the Media: Grundlagen – Analysen – Perspektiven. Eine Bestandsaufnahme (Biele-
feld: transcript), p. 26 and pp. 545–60. A similar thesis has been advanced
by Voßkamp and Weingart: ‘Not only within the traditional text-image gen-
res (emblems, advertising, press photography, comics, etc.), but also when it is
presumably “only” a matter of images or texts, monomedial assumptions about
the picturality of the picture or about the textuality of the text can ultimately
be seen as untenable essentialisms. In the sense of W. J. T. Mitchell’s asser-
tion that “all media are mixed media” it is assumed here that medial purity
precepts should themselves be understood as discursive effects and thus, not
least, as the result of procedures of power, of inclusion and exclusion’: (2005)
Transkriptionen 5, p. 31. See also W. Voßkamp and B. Weingart (2005) ‘Sicht-
bares und Sagbares: Text-Bild-Verhältnisse’ in W. Voßkamp and B. Weingart

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


66 Media Borders of Qualified Media

(eds) Sichtbares und Sagbares: Text-Bild-Verhältnisse (Cologne: DuMont), pp. 9–10.


Also compare in this context Mitchell’s own remarks on this subject: ‘[T]he
interaction of pictures and texts is constitutive of representation as such: all
media are mixed media, and all representations are heterogeneous; there are no
“purely” visual or verbal arts, though the impulse to purify media is one of the
central utopian gestures of modernism’: W. J. T. Mitchell (1994) Picture Theory:
Essays on Verbal and Visual Representation (Chicago and London: The University

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of Chicago Press), p. 5. As in the following I will continue to talk about ‘individ-
ual media’, what should be emphasized here is that, in my own understanding,
the term ‘individual medium’ (Einzelmedium) does not per se point to any kind
of ‘monomediality’ or medial ‘purity’. Instead, in my view, what is at issue here
are media that are conventionally perceived as distinct from other media (cf. in
more detail below). Hence, so-called individual media can indeed be characterized
by a plurimedial structure as, for instance, film or theatre. Moreover, individual
media – and this is also true of media often termed ‘monomedial’, such as (lit-
erary) texts – are always to be conceived of as multimodal (cf. Lars Elleström’s
contribution to the present volume). In fact, Mitchell’s famous dictum might be
reformulated into a slightly smoother ‘all media are multimodal (media)’. Signif-
icantly, Mitchell himself has recently rephrased his dictum, specifying that ‘[a]ll
media are, from the standpoint of sensory modality, mixed media’ (W. J. T. Mitchell
(2007) ‘There Are No Visual Media’ in O. Grau (ed.) MediaArtHistories (Cambridge
and London: MIT Press), p. 395; my emphasis).
8. Cf. note 7.
9. Quoted from the grant for the International Research Training Group InterArt/
Interart Studies, an international graduate school that has been established at the
Freie Universität Berlin in October 2006 (spokesperson: Erika Fischer-Lichte).
10. Quoted from the synopsis of the conference, cf. http://www.theater-
medien.de/kongress/sektionen.html, date accessed 4 May 2009.
11. As note 9.
12. Cf. especially I. Rajewsky (2004) ‘Intermedialität ‘light’? Intermediale Bezüge und
die ‘bloße Thematisierung’ des Altermedialen’ in R. Lüdeke and E. Greber (eds)
Intermedium Literatur: Beiträge zu einer Medientheorie der Literaturwissenschaften
(Göttingen: Wallstein).
13. Cf. J. Paech (2002) ‘Intermediale Figuration – am Beispiel von Jean-Luc Godards
Histoire(s) du Cinéma’ in A. J. Lehmann and I. Maassen (eds) Mediale Performanzen:
Historische Konzepte und Perspektiven (Freiburg i. B.: Rombach).
14. Cf. S. Krämer (2004) ‘Kulturanthropologie der Medien: Thesen zur Einführung’,
Paragrana: Internationale Zeitschrift für historische Anthropologie 13, pp. 130–3.
15. Cf. in more detail Rajewsky (2002), (2004) and (2005). With respect to this tri-
partite division, it is important to note that a single medial configuration may
certainly fulfil the criteria of two or even of all three of the intermedial categories
outlined below.
16. Cf. below, note 24.
17. W. Wolf (2005) ‘Intermediality’ in D. Herman and M.-L. Ryan (eds) The Routledge
Encyclopedia of Narrative Theory (London: Routledge), p. 253, cf. also p. 254.
18. Wolf (2005) ‘Intermediality’, p. 253.
19. Ibid., p. 254.
20. For a more detailed discussion (including illustrations) cf. Rajewsky (2002) and
(2005).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Irina O. Rajewsky 67

21. With respect to ‘photorealism’ and related concepts cf., for example, C. Lindey
(1980) Superrealist Painting & Sculpture (New York: Morrow); L. K. Meisel (1980)
Photorealism (New York: Abrams).
22. The depictions of streets and buildings by the American painter Richard Estes
serve as a paradigmatic example of this (see, for instance, Café Express, 1975, oil
on canvas).
23. My use of the term ‘simulation’ is not intended in the sense the word is used in

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
media studies (that is, to designate mathematical simulation processes); rather, it
connotes a simulation in the literal sense of the word.
24. Cf. also Wolf (1999) The Musicalization of Fiction, pp. 40–1. At the extreme outer
pole of this subcategory of intermediality are phenomena which, taking recourse
to another terminology, can also be designated as intermedia configurations. The
term ‘intermedia’ was first brought into play by Dick Higgins’ 1966 pioneering
essay ‘Intermedia’ (Something Else Newsletter 1, 1 (1966), reprinted in D. Higgins
(1984) Horizons: The Poetics and Theory of the Intermedia (Carbondale: Southern
Illinois UP)), in which Higgins expresses his conviction that ‘much of the best
work being produced today seems to fall between media’ (p. 18). This understand-
ing of the term has become relevant for attempts to delimit so-called intermedia
configurations from mixed-media and multimedia configurations. Higgins uses
‘intermedia’ to refer to works ‘in which the materials of various more estab-
lished art forms are “conceptually fused” rather than merely juxtaposed’ (E. Vos
(1997) ‘The Eternal Network: Mail Art, Intermedia Semiotics, Interarts Studies’
in U.-B. Lagerroth, H. Lund and E. Hedling (eds) Interart Poetics: Essays on the
Interrelations of the Arts and Media (Amsterdam and Atlanta, GA: Rodopi), p. 325);
the quality of medial juxtaposition is ascribed (with certain fine distinctions) to
mixed-media and multimedia configurations (see C. Clüver (2001) ‘Inter textus/
inter artes/inter media’, Komparatistik: Jahrbuch der Deutschen Gesellschaft für All-
gemeine und Vergleichende Literaturwissenschaft (2000/2001), pp. 14–50). Hence, as
can be seen in certain forms of visual poetry or corporate logos, in intermedia con-
figurations the materials of various individual media become inextricably bound
to, or even ‘merged with’, one another (cf. ibid., p. 36).
25. For a more detailed discussion (including illustrations) cf. Rajewsky (2002),
pp. 20–2, 164–6.
26. Cf. Wolf’s definition of ‘medium’ in Wolf (1999) The Musicalization of Fic-
tion, p. 40; see also W. Wolf (2002) ‘Intermedialität: Ein weites Feld und eine
Herausforderung für die Literaturwissenschaft’ in H. Foltinek and C. Leitgeb
(eds) Lite-ra-tur-wis-sen-schaft intermedial – interdisziplinär (Vienna: Verlag der
Österreichischen Akademie der Wissenschaften), p. 165.
27. Here a performative potential of intermedial strategies becomes apparent (cf. also
below).
28. In narrative film, for example, the formation of certain narrative conventions,
such as specific ways of introducing flashbacks, can at least also be traced back to
the fact that film, as far as the image track is concerned, is medially restricted to
present-tense narration. Hence, to visually convey analepses, film narration must
necessarily rely on certain filmic codes and conventions.
29. Cf. in more detail I. Rajewsky (2007) ‘Von Erzählern, die (nichts) vermitteln:
Überlegungen zu grundlegenden Annahmen der Dramentheorie im Kontext einer
transmedialen Narratologie’, Zeitschrift für Französische Sprache und Literatur 117,
pp. 25–68.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


68 Media Borders of Qualified Media

30. Compare in this connection general considerations on the constructed character


of border drawings, discussed particularly in the 1990s with special attention to
the observing subject or system. Referring to the concept of hybridity, Irmela
Schneider has pointed out that to the extent that differences are conceived
as observer-dependent differentiations, these are at the same time recognized
as modifiable (cf. I. Schneider (1997) ‘Von der Vielsprachigkeit zur “Kunst der
Hybridation”: Diskurse des Hybriden’ in I. Schneider and C. W. Thomsen (eds)

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Hybridkultur: Medien, Netze, Künste (Cologne: Wienand)). Accordingly, differences
and borders are not seen and discussed as ‘natural’ but as set or drawn borders,
which could also be set and drawn differently.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


3
Intermedial Topography and
Metaphorical Interaction

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Axel Englund

As the title of this volume illustrates, we often tend to think of arts and
media in terms of geographic areas delineated by borders, and consequently
of interartial and intermedial studies as a kind of topographical description,
a charting of territories and their positions in relation to each other. This
conception is often a helpful way of imagining the arts and media, and one
which is deeply connected to the way they have been functioning for the last
couple of hundred years or so. My point in this essay is not to argue that the
topographical model ought to be discarded, but to contrast it with a different
model, namely the conceptualization of intermedial relations in terms of
metaphoricity.1 My examples will concern the interplay between music and
literature, specifically works of Western art music (Ravel and Mozart) and
poetry (Bertrand and Celan) that bring the ‘other’ medium into play.
Most modern theories conceive of metaphoricity not in terms of substitu-
tion, but as a tension between separate elements that cannot be paraphrased
without losing its essential meaning.2 In other words, metaphor has a
cognitive import in its own right, and what it communicates cannot be
exhaustively recast in any literal wording. In this context I will be mak-
ing use of the conception of metaphor advocated by Max Black, which he
calls the ‘interaction view’ of metaphor.3 In short, a metaphor consists of
two separate subjects – Black labels them principal subject and subsidiary sub-
ject – interacting with each other. Each subject has a system of associated
implications (which might be roughly described as a list of ideas commonly
associated with the subject) and when placed together in a metaphor, the
implications of the primary subject are affected so that analogies with sub-
sidiary subject are foregrounded. As a result, our perception of the primary
subject is altered by the metaphorical interaction. As the word ‘interaction’
implies, however, this is not the whole truth: our perception of the sec-
ondary subject is altered as well, since those implications that are analogous
to implications of the primary subject are foregrounded by the metaphor.
In short, the metaphor organizes our conception of its two constituent sub-
jects. My discussion of particular examples departs from the idea that the

69

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


70 Media Borders of Qualified Media

‘music’ part and the ‘literature’ part of many musico-literary artefacts can
be interpreted as the constituent subjects of a metaphor. The emphasis on
the interpretative activity is important: what we are dealing with is less
an inherent quality exhibited by intermedial works than a fertile strat-
egy for approaching these works. At the same time, as has been suggested
by theoretical concepts as different as Gadamer’s Wirkungsgeschichte and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Derrida’s dissemination, a definitive distinction between text and interpre-
tation, between internal and external, is neither possible nor desirable.
What implications does Black’s theory have when applied to intermedial
relations? For one thing, his conception of metaphor suggests that the sub-
jects involved in the interaction are best regarded not as ‘things’ but as
‘systems of things’,4 or, even better, each subject could be conceived of as
a ‘system of ideas’.5 Such a system, he stresses, is ‘not sharply delineated,
and yet sufficiently definite to admit of detailed enumeration’.6 Hence, if
the interacting subjects are music and literature, these are to be thought of
as systems of ideas rather than one sharply delineated ‘thing’ with a defin-
able essence. On a general level, these ideas could be the physical form
of presentation (literature is stereotypically presented as a book, music as
performed on musical instruments), the semiotic mode (literature stereotypi-
cally employs a symbolic mode of reference, music stereotypically consists of
abstract sound or, in some cases, sound with iconic reference) or the sensory
channels through which they reach the percipient (a text is stereotypically
read, a musical piece heard). These notions, and many others, are all part of
the system of ideas connected to the words ‘music’ and ‘literature’. This way
of considering music and literature is fully analogous to definitions of media
not according to a single, palpable essence, but rather as the configuration
of a number of distinguishable modalities.7 It is not, however, entirely com-
patible with the topographical model, since it tells us that there are indeed
no distinct borders. Any alteration of the configuration of modalities would
entail a slight movement on the map, and it would be impossible to pin-
point the place in which one medial territory ends and another starts. Most
epochs, particularly modernity and post-modernity, have seen a number of
works that do not allow themselves readily to be placed on the map in its
contemporary state – liminal cases that, in terms of the topographical model,
would have to be regarded as a no man’s land or disputed zones. Much like
nations, then, the systems of ideas that constitute arts and media are histor-
ically contingent and if one wishes to speak about their borders, it needs to
be kept in mind that they are subject to gradual but constant change.
In Black’s view, we understand a metaphor by searching for analogies
between the ideas associated with the two subjects. The ideas, according
to Black, may be either commonplaces about the subject or ideas ‘estab-
lished ad hoc by the writer’.8 This double-edgedness is an important part
of applying the metaphorical perspective to musico-literary artefacts. Any
musico-literary metaphor would activate the system of ideas that make up

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Axel Englund 71

our conception of ‘music’ and ‘literature’, provoking us to find analogies


between them. These are the kind of general ideas mentioned in the preced-
ing section. However, such general ideas are not the only ones put into play
by the interaction, unless the metaphor in question is simply the sentence
‘literature is music’. In addition, each work incorporating music and litera-
ture also contains its own unique configuration of material ideas, by which

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
I mean its particular words, sentences, notes, chords, rhythms, large-scale
structures – in short, all that constitutes the physiognomic identity of the
given work. The ‘system of ideas’ associated with principal and subsidiary
subject respectively thus has a very wide spectrum, incorporating every-
thing from concrete material shapes to semantic connotations. Since all such
aspects are potentially essential parts of the artwork – a phenomenon akin to
what has been termed ‘repleteness’ by Goodman9 – they may consequently
be activated and reorganized by the metaphorical interplay.
Furthermore, much as there are no watertight seals between langue and
parole, there are none between these medial-general and work-specific ideas.
When a work is understood in terms of a metaphorical interaction between
music and literature, both categories, as they are configured in the partic-
ular artefact, partake of the reorganization of the systems of ideas, altering
our conception of categories as such. As Irina Rajewsky stresses, we never
encounter the abstract notion ‘the medium’ as such, but always as artic-
ulated in an individual work.10 If metaphorical interaction is based on a
tension between the principal and subsidiary subject, it also implies a ten-
sion between our preconceptions of the subjects involved and the way in
which they actually appear in the present context, the former demanding
continual revision when faced with the latter. The far-reaching consequence
of this perspective is that each musico-literary work can be read as a meta-
medial utterance with the inherent potential of altering the system of ideas
known as ‘music’ and ‘literature’, thus redrawing the borders of medial ter-
ritories. The way in which the ‘music-part’ and the ‘literature-part’ relate in
a given work has the power to change our conception of the way ‘music’
and ‘literature’ relate in general. Just as a metaphor changes our conception
of the world, the metaphorical dynamic of musico-literary artefacts changes
our conception of media. From this perspective, the topographical model
becomes untenable: a work of verbal or musical art is not an object located
in one territory or another, but is a shaping force in our conception of these
‘territories’.
Two further comments should be made here regarding the concept
‘metaphor’, namely regarding the criteria of two ‘systems of ideas’ such as
‘music’ and ‘literature’ being read as interacting according to the rules of
metaphor. First, metaphor is commonly thought of as a trope based on sim-
ilarity. It is also, however, fundamentally dependent on difference. If love
is said to be ‘a red, red rose’, we are dealing with metaphor. Where there
is no difference and no term is alien to the other, there is no metaphor,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


72 Media Borders of Qualified Media

but the pointless statement that ‘a rose is a flower’, or even a figural short-
circuit such as Gertrude Stein’s ‘a rose is a rose is a rose is a rose’. Now, if
metaphoricity presupposes difference, the relation between music and lit-
erature can never be described as metaphorical as long as the arts are not
conceived as different and distinguishable from each other. For instance, the
use of the Greek term mousike techne, subsuming dance and drama as well

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
as music and poetry, renders impossible any notion of metaphorical inter-
action between the arts – they were not thought of as separate at all and
cannot constitute the ‘distinct subjects’ described by Black. One step in this
historical development was taken when poetry began to be written to be
read silently rather than performed in song. A definite separation did not
come about until the mid eighteenth century, when aesthetics became a
philosophical field and instrumental music gradually began to be conceived
of as a genre of its own, distinct from language.11 The difference needed to
create a metaphorical tension between the concept of ‘music’ and the con-
cept of ‘literature’ was thus guaranteed: the arts became each other’s Other,
which opened up the possibility of cross-illumination. As is well known,
this possibility was not left unexploited: especially in Germany, the advent
of Romanticism saw a proliferation of the use of the other art as a model for
that which music or literature should aspire to. If the notion of literature and
music as independent and fundamentally different arts was necessary for a
metaphorical interaction to take place between them, it stands to reason that
it also promoted the topographical model, which to a certain degree coun-
teracts the metaphorical perspective: differences easily came to be thought
of as borders delineating territories.
If difference is one prerequisite of the fecundity of the metaphorical per-
spective, the pretension to identity is the other, in the textbook example of
metaphor usually brought about by the use of the copula ‘be’. Max Black
exemplifies his interaction view with the sentence ‘man is wolf’, which in
essence consists of nothing but a principal and a subsidiary subject and the
verb suggesting an identity between them. This ‘is’ need not be explicit; it
can equally well be implied by a genitive (‘the chess game of the cold war’)
or predicate (‘the chairman ploughed through the discussion’). Intermedial
works that somehow suggest an identity between music and literature seem
to encourage a metaphorical interpretation. The pretension to identity, one
might argue, is the necessary catalyst for a metaphorical understanding of
the intermedial relations. In a work bringing music and literature together,
then, this understanding does seem to require a meta-level: an identity
between the arts has to be suggested by the artefacts themselves.
How, then, does this suggestion come about? The most obvious place to
look for such a pretension to identity is the title of a work, whose most
important function, as Genette has pointed out in his meticulous discussion
on paratexts, is to designate the work, to give it an identity.12 If the iden-
tity suggested by the title clearly belongs to the other art, its metaphorical

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Axel Englund 73

quality is indisputable; it is, in fact, a clear illustration of the Aristotelian


definition of metaphor as ‘giving the thing a name that belongs to some-
thing else’.13 It is tantamount to saying that ‘this piece of music is a poem’
or ‘this poem is a piece of music’. A quick look at the works commonly
studied by musico-literary research today confirms a dominance of works
signalling their intermedial status in this fashion.14 A similar intermedial

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
signal can be found in the titles of instrumental pieces commonly studied
in musico-literary contexts, where many titles are borrowed from a specific
literary work rather than a genre, which nevertheless has the effect of ascrib-
ing the music to a medium other than its own. Such is the case with my first
example: Ravel’s piano piece Gaspard de la nuit, whose title is borrowed from
Aloysius Bertrand’s collection of poems.
Siglind Bruhn has carried out a beautiful analysis of the second piece from
Gaspard de la nuit, ‘Le gibet’, which, like the whole suite, has borrowed its
name from Bertrand.15 Rather brutally oversimplified, this poem has the
form of an inquiry as to the origin of a sound. This inquiry is repeated
through five strophes, all containing the interrogative ‘serait-ce’ and the sug-
gestion of a possible answer. These are the chirping of a cicada, the buzzing
of a fly and other insect activities. In the sixth strophe, the answer is given: it
is a bell on the city walls sounding the death-knell for the hanged man. Now,
as can be seen in Figure 2, Ravel’s piece begins with a B flat in two octaves,

Figure 2 The first measures of Maurice Ravel’s piano piece Gaspard de la nuit

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


74 Media Borders of Qualified Media

which persists throughout the piece, although in different metric positions.


It is subsequently surrounded by different kinds of music, a technique that
is suggested already in the first measures of the piece.
The bell tolling throughout the poem (which we recognize only in ret-
rospect) and the insistent repeated pitch of the piano piece is an obvious
analogy, and one that Bruhn’s analysis elaborates on. She interprets the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
gradually developing relation between this note and its surroundings as anal-
ogous to the relation between the hanged man and the insects that become
ever more encroaching as the poem progresses.16
The metaphorical perspective would not oppose this interpretation – it
would acknowledge the specific analogies discussed by Bruhn, although
in slightly different terms. It would, for instance, speak of the interaction
between music and poem as endowing the repeated note (an idea in the sys-
tem of the principal subject, Ravel’s music) with the sinister timbre of a death
knell (which is an idea in the system of the subsidiary subject, Bertrand’s
poem). These are, in the terms suggested above, the specific or material ideas,
but the metaphorical perspective would also add a further notion to support
the interpretation. The general concepts of ‘music’ and ‘literature’ form the
background against which the material analogies are understood, and the
individual work of art is constantly measured against these pre-established
notions of the media it involves. For instance, the notion of referentiality,
undeniably an important component in the system of ideas thought of as
‘literature’, is projected upon the system of ideas known as ‘music’. This
analogous implication organizes our conception of Ravel’s music, so that
the idea of referentiality – which is sometimes thought to be a part of music,
but which is not central to it – is foregrounded. By dint of the metaphori-
cal interaction between ‘literature’ and ‘music’, the latter is endowed with
the former’s tendency to signify – although in a semiotic modality focused
on the iconic rather than the symbolic mode – which makes the interpreta-
tion of the B flat as a death-knell, as well as its further implications for the
interpretation of the piece, all the more pertinent. Heard in this way, this
composition becomes a meta-utterance on the character of music and its
relation to verbal art, taking part in the historical process of medial evolution
by emphasizing their common ground in the semiotic modality.
Furthermore, due to its paratexts, Ravel’s piece is particularly suscepti-
ble to a metaphorical reading: ‘Le gibet’ is an excellent illustration of the
pretension to identity between music and poetry encouraging metaphorical
interaction. First, the title uses the name of a piece of literature to designate
a piece of music. Second, Ravel has furnished his score with the subtitle,
‘Trois Poèmes pour Piano d’après Aloysius Bertrand’. These paratexts make
the metaphorical suggestion ‘this piano piece is a poem’, indeed a specific
poem, which necessitates an interpretation of Ravel’s piece in the light of
the poem ‘Le gibet’. The paratext, then, constitutes a link to the important
intertext of Bertrand’s poem, which points to a crucial issue: the paratexts

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Axel Englund 75

play an essential part as the catalysts of metaphorical interaction. In fact,


music without verbal elements has very limited means – if any – to make
a meta-reflective suggestion of its own being literature. Therefore, if there
is no title or other authorial paratext (such as epigrams, program notes and
so on) suggesting the relation to a literary artefact, a piece of instrumental
music cannot be recognized as being based on such an artefact.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
This notion is related to the question of how to categorize a piece like
Ravel’s. As Bruhn points out, several labels exist already: she discusses
transposition, transformation, transcription, translation and transmutation,
before settling with transmedialization.17 In other words, what is continually
altered is the ending of the word, whereas the prefix is retained throughout.
Now, ‘trans’ etymologically implies something being moved from one place
to another, as is exemplified by Rajewsky’s definition of the term ‘transme-
diality’ as the appearance of the same matter in different media, where ‘the
assumption of a contact-giving medium of origin is not important or possi-
ble, and would not be relevant for the constitution of meaning in the current
medial product’.18 If this were the case, then, we would not need the source
medium to access this something once it had been carried across into the
target medium, just as the point of a translation is to make a text accessible
to those who do not have access to the original language.
Quite obviously, this is not the case in our example. Without the title
designating at once Ravel’s music and Bertrand’s text, and thus bringing the
latter into the interpretation, we would have no particular reason to hear this
piano piece in the light of this poem rather than any other. The repeated B
flat would hardly be endowed with the ominous character of a death-knell,
or at least it might equally well be heard as a number of other things. This
fact might be illustrated, for instance, by the similarly repeated A flat of
Chopin’s op. 28:15, which, due to the paratext of the piece, is often inter-
preted as a drop of rain. Any term using the prefix ‘trans’, then, denies the
necessity of keeping ‘target’ and ‘source’ simultaneously in mind, by implying
that the notes themselves are able to express the content of a poem without
being illuminated by a verbal paratext. Even if one adopts a postmodernist
perspective from which the notion of ‘music itself’ as absolutely distinguish-
able from ‘language’ appears highly suspect, the paramount importance of
paratextual elements in generating meaning has to be acknowledged.19
The notion of a musico-literary artefact as transmedialized is potentially
hypostasized by the topographical model, since it suppresses the notion of
simultaneity and interaction by emphasizing the notion of moving from one
place, art or medium into another, leaving the original behind. Rajewsky
notes this aspect implicitly in referring to transmedial works as Wander-
phänomene – phenomena of wandering. In this intermedial vagabondage,
the border is crossed and the property of another area brought into foreign
territory. The notion of media as adjacent, geographical areas, then, suggests
that an artefact of the verbal field can be transported into a musical field, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


76 Media Borders of Qualified Media

core of its identity remaining unaltered even when the original place is out
of sight. By contrast, the notion of musico-literary relations as metaphorical
interaction between two subjects or system of ideas, stresses the fact that the
original, verbal artefact and the musical piece need to be held in mind simul-
taneously. For instance, Paul Ricœur, quoting Douglas Berggren, holds that
the ‘possibility of comprehension of metaphorical construing requires . . . the
ability to entertain two different points of view at the same time’.20 Black,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
too, underscores the simultaneous appearance of principal and subsidiary
subject, speaking of the latter as a filter through which the former is per-
ceived.21 Ravel’s ‘Le gibet’, from this perspective, is not a content transported
from literature into music, but a piece of music demanding to be perceived
through the filter of a piece of poetry.
Unlike music, literature does not need to rely on its paratexts in order to
create the pretension to identity needed to set off a metaphorical interaction
between itself and another art. A piece of verbal writing, in other words, has
the ability to meta-reflectively thematize its own medial status and make the
metaphorical claim of belonging to another medium. For my next example,
I turn to Paul Celan, whose poem ‘Anabasis’ from the 1963 collection Die
Niemandsrose contains the following lines:

sekundenschön hüpfenden secondlovely skipping


Atemreflexen -: Leucht- breathreflexes -: light-
glockentöne (dum-, chimenotes (dum-,
dun-, un-, dun-, un-,
unde suspirat unde suspirat
cor), cor),
aus- re-
gelöst, ein- leased, re-
gelöst, unser.22 deemed, ours.

One could spend a great deal of time with these lines, not to mention with
the poem from which they have been severed here, but let us focus on what
they say about the relation between language and music.23 First, the Latin
phrase – meaning approximately ‘whence our hearts sigh’ – comes from
Mozart’s motet for soprano and orchestra, Exsultate, Jubilate.24 Immediately
preceding this quotation, three syllables that seem to be emptied of referen-
tial content present themselves. Importantly, these two modalities – abstract,
organized sound and referential language – are joined by a gradual, almost
seamless modulation: ‘dum-, / dun-, un-, / unde’. The Mozart quotation as
well as the sounds empty of referential meaning – tönend bewegte Formen,
to quote Hanslick’s famous definition of music – are contained within a
parenthesis, which seems to suggest that they exemplify the preceding word,
‘light- / chimenotes’. Thus, the poem makes a claim to verbal imitation of
musical sounds. Moreover, the gradual transition from an abstract-auditive

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Axel Englund 77

to a semantic modality in combination with the evocation of Mozart con-


tributes to the suggestion of an identity between music and language: the
border, as it were, is being blurred. Alternatively, these verses mimic the blur-
ring of the border – there is never any doubt as to the fact that ‘Anabasis’
is a verbal text rather than a piece of music.25 Thus the interplay between
identity and difference needed for a metaphorical interaction to be set off is

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
present in these lines.
There are, in fact, further reasons for interpreting this gesture as a preten-
sion to identity between music and literature. One of them lies in a general
tendency in Celan’s poetry to be meta-reflective: one of the main issues of his
oeuvre, only partly stemming from his engagement with Adorno’s famous
caveat, is the possibility of German poetry in the wake of Auschwitz. In view
of this, every single element of his poems can potentially be understood
as turned towards poetic language itself, questioning, problematizing and
analysing it, and musical references such as the ones displayed in ‘Anaba-
sis’ are no exception. Although a particularly salient feature of Celan’s work,
the inclination of poetry towards poetology is of course a very widespread
practice in modernist as well as postmodernist literature. A further histori-
cal note could be added: in the chaotic age of modernity, one of the most
conspicuous techniques in music as well as literature has been the juxta-
position of seemingly disparate and unrelated elements, disregarding those
laws of causality to which art had hitherto adhered. In poetry, paratactic
collocations as well as a number of other ways of deconstructing syntactic
structures could be mentioned, and in music the dissolving of tonality and
highly fragmented formal language exemplifies a similar tendency. In short,
the relations between the elements of a work of art became less and less obvi-
ous, and had to be inferred by the percipient. Although he preferred the term
imago, the following brief and well-known poem by Ezra Pound is doubtless
an example of metaphorical interaction in Black’s sense: ‘the apparition of
these faces in the crowd; / Petals on a wet, black bough’.26 Even though no
explicit statement of identity is made, the juxtaposition clearly prompts us
to look for analogous implications and to regard the depiction of the sub-
way scenery through the filter of the second image. The proliferation of this
kind of writing and composing, one might argue, has made the percipients
of modernist art attuned to the search for relations other than causal ones.
There is no reason why this change of attitude should not affect our under-
standing of the juxtaposition of music and literature within the same work.
The question of the relation between the medial components of a piece of
text or music has become increasingly important in modernism, which is
patently illustrated by verses such as those of Celan’s quoted above.
Having thus argued that these lines from ‘Anabasis’ are another exam-
ple of metaphorical interaction between literature and music, as well as an
illustration of a verbal text setting off such interaction without the aid of
paratexts, I would briefly describe the resulting interaction. Celan’s verses

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


78 Media Borders of Qualified Media

themselves would have to be the principal subject of this metaphor, whereas


the secondary subject would be the andante movement of Mozart’s motet.
One very general notion included in the system of ideas known as ‘music’
is that of a strong connection to emotions, a commonplace that remains
widespread to this day. This idea finds its analogy in the ‘cor’ mentioned
in the text, as well as in the sounds ‘dum-, /dun-’, which might be inter-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
preted as an onomatopoetic reference to the beats of the sighing heart (or,
for that matter, to the sound of chimes). Also, the religious character of
the Mozart motet activates the traditional notion of music functioning as a
link between humanity and divinity, fundamentally important to Friedrich
Hölderlin among others. In Celan’s text, this notion is mirrored by the elated
mood and almost epiphanic connotations of the lines ‘re- / leased, re- /
deemed, ours’.
A more specific, material idea lies in the adverb ‘secondlovely’, which
could be taken as referring not only to the time unit of a second, thus stress-
ing the fundamentally temporal quality of musical appreciation, but also
the musical interval of a major or minor second. If one examines Mozart’s
Exsultate, Jubilate at the entrance of the quoted phrase – that is, the music
evoked by the textual quotation – 21 intervals out of 25 turn out to be sec-
onds (the remaining being a fifth and three thirds).27 The predominance of
seconds is of course a very common phenomenon in the melodic idiom of
Mozart. Thus it seems reasonable to understand this musical interval as one
of the connotations of ‘secondlovely’. Furthermore, the term ‘breathreflexes’
actualizes the notion of breath, which is not only a central notion in Celan’s
poetics,28 but also inextricably linked to the human voice, be it speaking or,
as in Mozart’s piece, singing.
In this example, it is rather obvious that we are not dealing with a ‘trans’
anything: music is not carried into the poem. Without the aid of a paratex-
tual marker, Mozart’s music is evoked as an intertext through which we may
perceive the non-referential syllables as well as the Latin phrase – Mozart’s
motet serves as a metaphorical filter through which certain aspects of Celan’s
verses might be read.
By using examples from the categories labelled ‘literature in music’ and
‘music in literature’ in Steven Paul Scher’s classic essay,29 I have aimed to
suggest a further important advantage of the metaphorical perspective: it is
able to serve as an umbrella concept for both of these types, since they can
both be regarded as metaphorical constellations. This approach, I believe,
might serve as a possible starting point for the collocation and comparative
study of works from all three of Scher’s categories, a kind of intermedial
examination that is arguably hard to access and of which we have not yet
seen many convincing examples.
I have also tried to shed light on some implications of the topograph-
ical model which describes media as adjacent areas separated by borders
and gaps, and contrast this model with an understanding of musico-literary

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Axel Englund 79

phenomena in terms of metaphorical interaction. For an artefact to open


itself up to such an interpretation, it needs a meta-reflective element, giv-
ing rise to a pretension to identity between music and literature. In verbal
artefacts, this pretension can be expressed either by the text proper or a para-
text. In musical artefacts, by contrast, a paratextual marker such as a title,
a program note or an epigraph, is perhaps the only way. Hence, a literary

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
content or form cannot be transported into music and recognized as such
without the simultaneous presence of the original work, a fact blurred by
the topographical model as well as by the use of the ‘trans’ prefix. Although
efficient in many ways, I have argued that it suppresses the notion of simul-
taneity. The metaphorical perspective, then, requires a suspension of the
topographical model in order for the arts to exist simultaneously in the
mind of the percipient. Moreover, I have argued that, like Black’s subjects,
these arts are better understood as systems of ideas than as neatly mapped
out medial areas delineated by clear borders. Through the metaphorical
interaction, in accordance with Black’s perspective, the ideas that constitute
analogies between the subjects come to the fore in our understanding of
the musico-literary artefact. These are general, notional ideas as well as spe-
cific, material ones and the reorganization of the systems of ideas to which
the analogies between them give rise can be read as meta-medial utterances,
potentially altering our view of ‘music’ and ‘literature’ as such. Each instance
of a musico-literary metaphor actively partakes in the historically develop-
ing notions of arts and media, reflecting on and reshaping their imaginary
territories, and thus continuously rendering obsolete each new map of the
intermedial topography before it is finished.

Notes
1. The understanding of musico-literary relations as metaphorical has cropped up in
recent intermedial studies by Lawrence Kramer, Eric Prieto and Michael Spitzer,
whose work has given important incentives to this essay. Cf. L. Kramer (1995)
Classical Music and Postmodern Knowledge (Berkeley CA: University of California
Press), pp. 69–71; E. Prieto (2002) Listening In: Music, Mind, and the Modernist Nar-
rative (Lincoln NE: University of Nebraska Press); M. Spitzer (2004) Metaphor and
Musical Thought (Chicago: Chicago University Press).
2. Cf. for instance M. Black (1962) ‘Metaphor’ in Models and Metaphors (Ithaca NY:
Cornell University Press), pp. 25–47; P. Ricœur (1979) ‘The Metaphorical Process
as Cognition, Imagination, and Feeling’ in S. Sacks (ed.) On Metaphor (Chicago:
The University of Chicago Press), pp. 141–57.
3. Black (1962) ‘Metaphor’, pp. 38–46.
4. Ibid., p. 44.
5. Ibid., p. 40.
6. Ibid., p. 41.
7. Cf. Elleström, this volume.
8. Black (1962) ‘Metaphor’, p. 44.
9. N. Goodman (1976) Languages of Art (Indianapolis: Hackett), pp. 229–30.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


80 Media Borders of Qualified Media

10. Cf. Rajewsky, this volume.


11. Cf., for instance, C. Dalhaus (2002) ‘Die Idee der absoluten Musik’ in Gesammelte
Schriften 4 (Laaber: Laaber), pp. 11–128.
12. G. Genette (1997) Paratexts: Thresholds of Interpretation, translated by Jane E. Lewin
(Cambridge: Cambridge University Press), p. 76.
13. Aristotle (1995) The Complete Works of Aristotle, Jonathan Barnes (ed.) (Princeton:
Princeton University Press), vol. 2, p. 2332.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
14. Cf. for instance the works studied in W. Wolf (1999) The Musicalization of Fic-
tion: A Study in the Theory and History of Intermediality (Amsterdam and New York:
Rodopi), pp. ix–x.
15. S. Bruhn (1999) ‘Piano Poems and Orchestral Recitations: Instrumental Music
Interprets a Literary Text’ in W. Bernhart, S. P. Scher, and W. Wolf (eds) Word and
Music Studies: Defining the Field (Amsterdam and New York: Rodopi) pp. 277–99.
16. Ibid., pp. 288–91.
17. Ibid., pp. 296–7.
18. I. Rajewsky (2002) Intermedialität (Tübingen: A. Francke), p. 13.
19. Cf. Lawrence Kramer’s use of the term ‘designator’ in Kramer (1995) Classical
Music, p. 69.
20. Ricœur (1979) ‘The Metaphorical Process’, p. 152.
21. Black (1962) ‘Metaphor’, pp. 39–41,
22. P. Celan (1992) Gesammelte Werke in Sieben Bänden, B. Allemann and S. Reichert
(eds) (Frankfurt am Main: Suhrkamp), vol. 1, p. 256. Die Niemandsrose  c 1963
S. Fischer Verlag GmbH, Frankfurt am Main.
23. For further discussion of the musico-literary aspects of this poem cf. A. Englund
(2009) ‘Modes of Musicality in Paul Celan’s Die Niemandsrose’, Seminar: A Journal
of Germanic Studies 45(2), pp. 138–58.
24. O. Pöggeler (1993) ‘Die göttliche Tragödie: Mozart in Celans Spätwerk’ in
C. Jamme and O. Pöggeler (eds) Der glühende Leertext: Annäherungen an Paul Celans
Dichtung (München: Weilhelm Fink), p. 68.
25. For equivalent examples involving the visual arts, cf. Rajewsky’s discussion of
intermedial reference in this volume.
26. E. Pound (2003) Poems and Translations (New York: Library of America), p. 287.
27. Cf. W. A. Mozart (1963) Geistliche Gesangswerk: Werkgruppe 3 (Kassel: Bärenreiter),
p. 167.
28. Cf., for instance, the famous formulation from ‘Der Meridian’: ‘Poetry: that can
signify a turn of breath’ (Celan (1992) Gesammelte Werke, vol. 3, p.195).
29. S. P. Scher (1982) ‘Literature and Music’ in J.-P. Barricelli and J. Gibaldi (eds)
Interrelations of Literature (New York: Modern Language Association), pp. 225–45.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


4
Intermedial Strategies
in Multimedia Art

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Christina Ljungberg

Laurie Anderson’s multimedia performance White Lily opens with a


computer-animated projection of a figure running in slow motion. Anderson
then enters into view backwards against the animated runner who disap-
pears to the left, while Anderson moves across to centre stage to electronic
music punctuated by clock chimes. Dressed in a white suit and accompanied
by her silhouette shadow generated by a strong circular projection, Ander-
son has her movements doubled by the shadow, as she presents a short text
about her memory of a brief conversation in Rainer Werner Fassbinder’s1 film
Berlin Alexanderplatz.2 This short sequence is reshaped into a poem recited by
Anderson, at the same time as she makes punctuating gestures with her right
arm, ending by making a backwards sign. The performance concludes with
the projection of the central symbol, the white lily, held by the hand of a
white silhouette in a still that is left standing against the dark, as Anderson
moves out of the centre and disappears to the right. The animated figure
then reappears and fades out, with only the music playing in the dark.
With its intricate intermingling of image, sound and gesture, the per-
formance of White Lily3 is a complex multimedial restaging of the short
scene in Fassbinder’s film, in which Anderson directly addresses a number of
questions concerning mediality, modality and art form. Anderson’s reconfig-
uration of the scene in Fassbinder’s movie alludes to Alfred Döblin’s famous
1929 novel Berlin Alexanderplatz4 which was the source of the film Fassbinder
made for television in 1980. Its impetus as an intermedial work is partly
drawn from Döblin’s text, which is in itself both intermedial and intertex-
tual. With its strikingly cinematic and journalistic character, Döblin’s work
is a modern epic, which, not unlike James Joyce’s Ulysses, rewrites antique
myth into a modernist urban setting and which, influenced by both Futur-
ism and Dadaism, tries to capture not only the visual mosaic of the city’s
frenetic and oscillating surfaces but also its polyvocal discourses. This is the
cultural memory that Fassbinder recreates in his epic 15-hour film, which
transforms the (mainly verbal) montage technique Döblin employs in his
large 400-page polyphonic novel into an audiovisual masterpiece, and which

81

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


82 Media Borders of Qualified Media

Anderson reconfigures in her 1 minute and 16 seconds performance of a key


event in Fassbinder’s film and Döblin’s novel.
These intersemiotic transformations, from a dialogic and intermedial
novel to, initially, a brilliant filmic visualization of Döblin’s imagination and,
eventually, to Anderson’s masterpiece, that is, from one medium to another
to yet another, raise the question of the notion of ‘medium’ in an especially

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
pointed and forceful manner. What is a medium? A key concept in semi-
otics, one way of defining ‘medium’ is to say that it is the necessary channel
or conduit of communication which allows the transmission of a message to
a receiver. As Winfried Nöth5 suggests, even the air functions as a medium,
since it carries the sound waves from speaker to listener. This basic meaning
resonates with the definitions given by the Oxford English Dictionary (OED),
which describes it as a means by which something is expressed, communi-
cated or achieved; a substance through which a force or other influence is
transmitted; a form of storage for computer software, such as magnetic tape
or disks; a liquid with which pigments are mixed to make paint; with the
plural ‘mediums’, a person claiming to be able to communicate between the
dead and the living, that is, between our lived world and an imaginary one;
or the middle state between two extremes. Although the first sense given by
the OED may be the most commonly used in contemporary life, all these
aspects of ‘medium’ suggest a dynamic and fluid meaning, characterizing a
transient function more than a fixed or stable property.
In Peircean semiotics, a sign is itself a medium since it dialogically inter-
acts in its various modes, the iconic, the indexical and the symbolic, in
an ongoing flow of signs mediating between the life-world we live in and
our interpretation of it. Although all sign aspects are necessary for a sign
to function, they are differently foregrounded in the various sign systems.
If music is predominantly iconic, photography and film are, due to their
mode of production, mainly indexical media, whereas painting and ver-
bal communication, insofar as they depend on cultural conventions, are
symbolic.
In communication and media studies, the concept of ‘media’ is used to
refer to the classical mass media newspaper, book, radio, popular music,
film and television. More recently, the concept has been extended to cover
writing or even speech in general, music, painting, photography, video, the
Internet or computer games, which no longer qualify as media interacting
with the ‘masses’. Intermediality, then, concerns the transgression of the
borders between such media, for instance, between different sign systems
and/or the iconic enactment of one medium within another. It also involves
the sensorial modality of a specific medium, mainly the visual, oral or tactile
(the use of olfactory signs, as in Divine’s Odorama, was a short, even if rather
successful, attempt) and the semiotic register of sign functions.
What happens to these modalities and to these functions when var-
ious media interact? What precisely constitutes the result, that is, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Christina Ljungberg 83

phenomenon we call intermediality, which concerns the negotiations of the


borders between various media. What do these ‘border talks’, as Irina Rajew-
sky (in this volume) calls them, effect in such intermedial transgressions?
What are the possibilities and limitations that such intersemiotic transla-
tions from one art to the other bring into being and how is one medium
reflected in another? I will argue that these instances of intermediality are

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
• radically performative, as we are confronted with hybrid forms that
generate something new and unique
• strongly self-reflexive, since they focus attention both on their own mode
of production and on their own semiotic specificity, which is heightened
by the increasing digitalization of interacting media
• a highly effective communication strategy, as they give readers, viewers
and listeners access to different levels of meaning.

These are the issues at stake in my contribution, which will discuss two very
different examples of intermedial art mapping time and space by the per-
former, artist and writer Laurie Anderson and digital artist Lucia Leão. What
I want to explore are the strategies they use to achieve their unique and
innovative intermedial effects.

The medium as sign

As mentioned at the outset, in a technical or material sense, ‘medium’ can be


described as the channel enabling communication between a sender and a
receiver. In its broader semiotic sense, however, the sign itself, as defined by
C. S. Peirce, functions as a medium. In the triadic semiotics of Peirce, a sign is
anything that stands for (represents) something, called its object, to generate
another sign as its interpretant. According to this definition, the sign is itself
a mediator or medium, acting, so to speak, as a translator between its object
and its so-called interpretant, which is the result of its interpretation. The
sign is therefore defined in terms of a triadic process called semiosis, or sign
generation. According to one of the most quoted definitions, a sign is ‘some-
thing that stands for something to somebody in some respect or capacity’.6
The sign initiates a process which makes it interact relationally or function-
ally with its object. Signs are not necessarily material objects, nor even a class
of objects7 : they exist in the mind of their interpreters, in other words, they
have a cognitive effect on their interpreters. In signification, a sign dialogi-
cally interacts with its various sign aspects, the iconic, the indexical and the
symbolic, in an ongoing flow of signs mediating between the life-world we
live in and our interpretation of it.
Although all sign aspects are necessary for a sign to function, these
aspects – iconic, indexical or symbolic – are, as mentioned above, differ-
ently foregrounded in various sign systems. That is why painting which, as a

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


84 Media Borders of Qualified Media

visual sign, would seem to be primarily indexical as it is always embodied


in some singular materiality8 displays mostly symbolic aspects: paintings
adhere to the styles and cultural (and ideological) conventions dominant
in the period in which they were executed. Not even so-called ‘pure paint-
ing’, which was advanced at the end of the nineteenth century as a form
separate from so-called ‘narrative painting’ and supposedly demonstrating

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
‘pure opticality’, is ‘pure’ but is, as W. J. T. Mitchell suggests, the ‘discourses
of theory, of idealist and critical philosophy’.9
Recognizing these underlying discourses is as vital for understanding
modernist painting as the knowledge of the Western narrative canon and
familiarity with its myths required for comprehending classical narrative
painting. Music, another example, and the most iconic of all medial forms as
it comes to us in the form of mere quality, can also be indexical and, though
marginally, symbolic. It is indexical not only because it always indicates a
certain style or genre (insofar as the recognition of its characteristics depends
on similarity, it is again iconic), but also because its tonal flow calls us to
‘things in this [particular] world’. 10 Symbolic aspects appear, for instance, in
the case of a national anthem.11 That is why, just as there are no ‘pure’ sign
forms, there are no ‘purely’ visual, verbal or aural media: in Mitchell’s words,
‘all media are mixed media’.12 This does not take away a medium’s partic-
ular characteristics, its ‘specificity’ as it were, but instead, enables media
mixtures and innovations and the transformations of old media by new
techniques.
What happens when various sign systems interact, which is the phe-
nomenon that we call intermediality? What does it do to the specific
character of each particular system? It would have to be assumed that, as in
all border transgression, transformations and substitutions take place. One
of the oldest intermedial relationships is the evolution from image over pic-
torial writing to ideographical writing systems, which, as has recently been
shown, developed according to underlying principles governing the shapes
of human signs.13 And is not also narrative language in itself intermedial, as
we structure our sentences in verbal diagrams and as spatial diagrams on the
page? It would thus seem that intermediality is intrinsic to narrative texts,
whether in the form of rhetorical structure14 or in the interplay with other
sign systems such as photographs or maps.15 This explains, I would argue,
why forms of self-referentiality and iconicity are intrinsic to literary texts –
and vice versa.
This intermediality demonstrates how such a seemingly simple trans-
gression is inherently performative, since it creates something new and
unprecedented, at the same time as one medium is reflected in the other.
In intermedial art forms, this relationship becomes radicalized, as the dif-
ference in interaction in multimedia art and performance determines the
different degrees of performativity and self-referentiality of the work of art
and its communicative effect.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Christina Ljungberg 85

Intermediality and performativity

What precisely is meant by performative here, and what is its relationship to


intermediality? If ‘performance’ is an execution of an action, the fulfilment
of a claim, promise and so on or a presentation to an audience, ‘performa-
tive’ is the very expression that effectuates it. Both concepts entail memory.
As Mieke Bal points out, in a performance, playing a part or role requires

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
memorizing that part or that score and practicing the gestures, expressions
and diction suiting the role, just as a performative act needs a cultural – and
appropriate – context to function. This subordinates individual intention
to social convention and makes performative acts instances of an ‘endless
process of repetition involving similarity and difference, and therefore rel-
ativizing and enabling social change and subjects’ interventions, in other
words, agency’.16
The concept of performativity and its link with performance has also been
theorized by the theatre historian Erika Fischer-Lichte17 who argues that it is
the transformative potential arising from the shared ritual practices surround-
ing a theatre performance that makes them both self-referential and capable
of constituting reality. She differentiates three different kinds of performa-
tivity: in the weak sense, that something is done by someone saying it; in
the strong sense, whereby language creates a new reality against a backdrop
of stable conventions; and in the radical sense, by which all these processes
create a new social reality.18 Intermediality always entails performativity in
the radical sense owing to its hybridity.
Within our cultural life, performative utterances and acts bring something
into being. In the simplest cases, in uttering words (for instance, ‘I promise
you to be there at seven’) an act is performed, one entailing various con-
sequences and expectations. The performance of a traditional ritual (for
instance, a marriage ceremony or christening) is one thing, that of a yet
to be instituted form of ritual and performance quite another. Of course, the
latter are enacted against the vast, vague background of our complex, myriad
inheritances. Even so, they bring something into being, not in accord with
traditional forms of ritual, but as an attempt to establish what is not now
recognized or authorized.
What is attempted in the quiet dramas of quotidian life is also under-
taken in more manifest ways in numerous contemporary artworks, which
therefore also involve analysis and interpretation. But is not the very act
of analysis in itself a performative act in which a new, further developed
object of study is created? As C. S. Peirce points out, an object exercises
influence over the sign in so far that it can ‘guide or constrain’ the process
of semiosis or sign generation.19 This is what Bal refers to when she points to
the fact that the analysis mutates from just applying theory to ‘a performa-
tive interaction between object (including those of its aspects that remained
invisible before the encounter), theory and analyst’ making the very process

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


86 Media Borders of Qualified Media

of interpretation part of the object for the analyst to investigate. In this way,
‘objects enable reflection and speculation, and can contradict projection and
wrong-headed interpretation . . . and thus constitute a theoretical object with
philosophical relevance’.20
Such awareness is particularly called for in the analysis of complex
multimedia performances such as White Lily, in which the relationship

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
between performativity and performance indeed becomes radicalized since
the switching between different media automatically generates new objects
and new realities for their ‘analysts’, that is, the audience trying to grasp
the intricate layers of meaning. Anderson’s performance – and also the work
of Lucia Leão – therefore poses interesting questions as to what happens
when various media interact and what such intermedial relationships effect.
Anderson’s profound and multilayered transformation of Fassbinder’s film
based on Döblin’s novel into her own performance demonstrates exem-
plarily the extent to which intermediality involves the transformations of
myths, poetry or prose, modes of narration and styles of writing into other
forms such as paintings, films, video arts, performances or other works of art
which in some way quote, adapt, rework or just allude to literary texts. At
the same time intermediality also refers to the very act or process of trans-
formation that literature undergoes under the influence or in contact with
other art forms as it absorbs and adapts motives, plots and even modes of
writing of the visual arts.21 It also takes up intermedial dialogues with the
other art forms by reflecting, in writing, on what these interactions bring,
as well as producing new hybrid and multimedial forms of art, of which
Döblin’s intermedial novel Berlin Alexanderplatz is a particularly pertinent
example.
In Anderson’s performance, the interplay among the visual, the vocal and
the gestural is highlighted by the complex interaction between the tech-
nical media, the qualified media and their various modes,22 performatively
calling something entirely new and different into being. Anderson’s brilliant
syncopation of a key event in Fassbinder’s film and Döblin’s novel abstracts
the core issues of time and temporality in her ‘pretexts’, transforming them
into a multi-layered mediation about time, memory and mortality. The split
focus on Anderson and her shadow is iconically matched by the sound ten-
sion simultaneously created between the electronic background music with
its effects of a bell that chimes and Anderson’s voice dramatically reciting
her memory:

What Fassbinder film is it?


The one-armed man comes into a flower shop and says:
What flower expresses days go by
and they just keep going by endlessly
pulling you into the future?

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Christina Ljungberg 87

Days go by
endlessly
endlessly pulling you
into the future.
And the florist says:
White lily.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Anderson’s condensation of the question of time and memory into her one
minute and 16 seconds refiguration of this key event in Fassbinder’s film
and Döblin’s novel brings up the question of the relationship between per-
formance, performativity and cultural memory in a very intriguing manner.
The event itself, the quotidian act in which a man comes into a flower shop
to buy flowers, may seem banal enough. It constitutes, however, a crucial
trait in the portrayal of the hulking, child-like ex-convict Franz Biberkopf.
When this scene takes place, Franz has recently come out of prison, fully
convinced that from now on he will become an ‘honest soul’ and lead
‘a decent life’.23 But every time his life seems to be taking a turn for the bet-
ter, he is betrayed by men he trusts and considers his friends. Such a betrayal
is implied in Anderson’s performance of the flower shop scene, when Franz
wants to make a decent gesture towards a woman who has been nice to him.
In Döblin’s story and Fassbinder’s film, Franz meets a girl, Polish Lina, whose
uncle, Otto Lüders, lets him into his business as a door-to-door shoelace
salesman. On one of his peddling rounds, Franz meets a lonely widow, whose
husband he physically resembles, and has sex with her. She is grateful and
generous to him, but Franz’s childish delight at his good luck has him tell
Lüders about it. Lüders immediately pays a visit to the widow, first to insult
her, saying that word about her ‘generosity’ is afoot, and then to rob her.
Franz, unsuspectingly looking forward to his next dalliance with her, goes
to buy her flowers. This is where Fassbinder’s poetic imagination stages the
short scene that Anderson is alluding to, a scene that does not occur in the
novel which only tells us that Biberkopf ‘slowly walks up the stairs with
a bouquet wrapped in oiled paper,’24 only to have the door slammed shut
in his face, and, disappointed and angry, throws the flowers into the gut-
ter. Only later, receiving a letter from the widow, does he learn of Lüder’s
betrayal, which demolishes him: when Lina comes home, she finds him sit-
ting in his room, apathetically picking his alarm clock to pieces, as if time
could be stopped and turned back to be repackaged, reordered and replayed.
In his film, Fassbinder superimposes Biberkopf’s visit to the flower shop
onto the underlying issue of time and temporality. Franz enters the shop and
asks the florist what flowers express the meaning ‘that the past always runs
after you and drives you further and further to where there is no future’. To
his surprise, the florist suggests white carnations, which to Franz symbolize
‘flowers of death’. Fassbinder then has Biberkopf buy the widow a bouquet
of red roses instead – which then end up the gutter when he is refused entry

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


88 Media Borders of Qualified Media

to the widow’s apartment. At this time, Franz is not yet the ‘one-armed man’
in Anderson’s memory – he still has both his arms. Moreover, not only does
Anderson change the white carnation into a white lily but she also weaves
the flower shop scene in Fassbinder’s film together with a later betrayal,
which costs Franz his arm and nearly his life when he is thrown out of a car
by another envious ‘friend’ – who later proceeds to kill his girlfriend, making

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Franz temporarily insane. By mapping the two betrayals, temporally distinct
but causally related incidents (as both concern betrayal by ‘friends’ whom
Franz naïvely trusted) onto one another, Anderson skilfully and poetically
syncopates Franz’s existence as an agent caught up in fields of intersecting
forces beyond his control, a victim of his time and circumstances. By only
‘learning things that he had rather not wanted to learn’25 too late, and not
being able to adapt to the ruthless life of the modern metropolis, he shows
himself as too naïve, too innocent and above all, too trusting.
The network thus spun over Alexanderplatz could therefore be looked at
as an ‘agential space’, a space in which agents are at once caught up tran-
scending their immediate control and implicated in the effective exercise of
their somatic, social agency. In other words, these agents are such situated
and embodied forces that the exercise of agency is best understood in terms
of introducing disturbances into this field or as tracing these intersecting
force patterns. This notion would seem to be appropriate to describe both
Fassbinder’s film and Döblin’s multiperspectival story about the interaction
of people caught in the corrupt urban landscape of Weimar-era Germany
and which is brought to such a fulcrum in Anderson’s performance.

Intermediality and self-referentiality

Intermediality also displays degrees of self-referentiality. Iconic self-reference


is typical of the aesthetic sign. One of the characteristics of the aesthetic sign
is that it calls attention to various aspects of itself, above all to its sensu-
ous qualities and formal structures, its actual materiality and its rhetorical
strategies. At least, this becomes evident and explicable when we approach
self-reference in general and the self-referentiality of aesthetic signs accord-
ing to C. S. Peirce’s doctrine of signs. In particular, his second trichotomy of
signs (icon, index and symbol), which is based upon the character of the rela-
tionship between a sign and its dynamical object, is especially illuminating
here, since it provides

1) a way of understanding reference in terms of indexicality (the indexical


sign being defined as that in which there is a spatiotemporal or causal
relationship between sign and object) and
2) a way of understanding self-referentiality (at least in part) in terms of
iconicity. All iconic signs are self-referential, which could appear para-
doxical since signs should really stand in for something else. The reason

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Christina Ljungberg 89

why signs can represent other signs is because, in Peircean semiotics, the
object ‘does not need to be a piece of the so-called real world at all,
since signs or ideas can be the object of a sign. The object of the sign
is something which precedes and thus determines the sign in the pro-
cess of semiosis as a previous experience or cognition of the world.’26
The sign’s referent (Peirce’s object of the sign) can be another sign, and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
self-reference can be a chain of signs referring to other signs.

Oral interpretation, as in the case of Anderson’s performance, utilizes the


self-referencing qualities of language and literature which involve the inter-
preter as both actor and reader. In the case of digital media, self-reference is
coupled to the mathematical generation of numerical images. These images
are in perpetual metamorphosis, ‘oscillating’ between the actual image on
the screen and the virtual image or potential set of images.27 Because there
is no analogy between the algorithms that generate it and the image on the
computer screen, this image is highly iconic. It generates experiences that are
not ‘real’ but formalized and repeatable calculations, which makes the syn-
thetic image synonymous with virtuality and simulation. That is why the
digitalization of pictures and films has contributed strongly to the increase
of self-reference in the media – at the same time as it has liberated the media
from ‘the bonds of factual reference to a world which they used to depict’.28
Quotations, allusions, adaptations, influences and borrowing from texts,
films or any other medium also generate intertextual self-reference.29 When
various media are involved, for instance a film referring to a painting (Girl
with a Pearl Earring) or a novel in a film (for example, Conrad’s Heart of Dark-
ness in Francis Ford Coppola’s Apocalypse Now, which has the novella lying
on the protagonist’s bedside table), there is intermedial self-reference: reference
from one media to the other. Such borrowings from other texts or media are
alloreferential, which means that ‘the object of the quoting sign is a quoted
sign from which it differs’,30 since one medium refers to another. The differ-
ence here is that when one film makes an intertextual reference to its own
cinematic medium by quoting another film, and not to the world which
they ultimately represent, they are examples of intertextual self-reference, in
particular since quotations always entail repetition and sameness, which is
‘the source of iconic self-reference’.31
As discussed above, White Lily is a highly complex intertextual act since its
‘pretexts’ resonate throughout the performance, albeit mainly for an audi-
ence familiar with Fassbinder’s film and even more so for those who have
read Döblin’s novel. Quotations and allusions are generally considered ref-
erential, since they refer to something else, an object, in a different context.
That would make Anderson’s literary quotations referential and in this sense
indexical since they have a particular context in mind. However, whenever
her music refers to other pieces of music and her visuals to other visuals (and
not to any world beyond the world of music or visual representation), her

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


90 Media Borders of Qualified Media

‘quotations’ of other musical and visual works are self-referential since they
are references from the media to the media. Furthermore, Anderson’s work is
pervaded with repetitions and recursions of words, phrases or ideas, which
are typical and striking forms of self-reference – whether in music, texts,
images or films – since they always refer back to the preceding instances.
So are the reuses and quotes of her own work, which make up much of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Anderson’s œuvre.
Self-referentiality is something that digital art shares with other post-
modern art forms. Hybrid forms of art and media heighten the degree
of self-referentiality: switching between or among various media not only
forces its viewing or, rather, participating, audience32 to make comparisons
among them but also exposes the particularities of the various semiotic sys-
tems that each medium embodies. Virtual reality requires both specific aids
and a technological environment, which calls attention to the necessary pro-
cedures involved. Focusing attention on the artist and her bodily self, as both
generating and participating in the work of art, also increases self-reference.
However, these works have a marked indexical ingredient, too, in the sense
of referring to other ‘real’ works, contexts or bodies. Even in virtual reality,
an awareness of the physical body is necessary for orienting ourselves in and
understanding the particular digital work of art. Hence, there are varying
degrees and forms of self-reference characteristic of various types of digital
art and media. Virtual reality is not separate or even separable from embod-
ied reality, but is rather an attenuated and reconfigured form – or array of
forms – of our embodied being in interaction, intermedially, with anything
and everything.
Another factor contributing to the self-referentiality of digital art is the
emphasis artists put on the very process involved in producing art by more
or less smoothly integrating the various media into an intermedial whole. As
Mitchell points out (referring to McLuhan), what is important is the ‘ratio’
in which the various ‘ingredients’ are mixed and timed, questions of dom-
inance/subordination, the phenomenon of synesthesia, or what Mitchell
calls ‘nesting’, in which one medium appears within another as its con-
tent, or ‘braiding’, the seamless weaving together of disjunctive elements
into a ‘seemingly continuous narrative’.33 His own emphasis on process is
in effect an interrogation of time, of the textures and forms of temporality,
but since the flux of time leaves sedimented layers of meaning and identity,
these sedimentations deserve our critical attention – not only the result of
the temporal flux but also the power of this flux to unsettle, to disturb. This
does not emanate from the artist only but is created in the space between
bodily co-presence between actor and spectator, and in the filmed record-
ing of a performance. Anderson’s multimedial performance can be seen as
‘braiding’ since one medium appears within the other as its content though
in a highly complex form: the scene from Döblin’s novel transformed into
Fassbinder’s film, then the filmed recording of Anderson’s performance of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Christina Ljungberg 91

her memory of Fassbinder’s film. There is however also a ‘braiding’ taking


place in the viewer’s mind in the act of interpretation as this utterly con-
densed performance must be mapped onto our own personal and cultural
maps of knowledge and experience. Viewed thus, there is an expansive
network of performative relationships, which contributes to the artwork’s
particular power.
Interactive works of art such as Lucia Leão’s Hermenetka34 would seem to

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
offer a different kind of self-referentiality and intermediality, as it offers the
user the possibility to produce his or her own map dynamically in an ever-
changing cartographic configuration. The meshwork Leão spans across her
Mediterranean map is a project of Net Art that generates fortuitous cartogra-
phies from search engines in various databases. Hermenetka is a telescope
word coined from associations to Hermes, god of communication and com-
merce, protean shape-changer and inventive creator as well as the interpreter
and intermediary between the divinities and the mortals; Net, to internet,
and Ka to the divine concept in ancient Egyptian mythology representing
the life-force and consciousness. Leão chose the geographical area and the
cultural concept of the ‘Mediterranean’ because of its etymological mean-
ing as the ‘sea between territories’ or its cultural meaning of being a ‘sea
in the middle of the Earth’ – both literally and metaphorically – and there-
fore being a ‘space in-between’ – which is thus embodied in unbounded
cybernetic flows and data exchanges. The project’s basic intent is to generate
plural cartographies of the seas of data populating cyber culture.
Hermenetka is constructed as two types of mappings. The first enables the
generation of a map in real-time centring on the concept of the Mediter-
ranean: countries, cultures, histories and so on. The map is entered by
clicking on one of the various topics circulating in hypertext on the map sur-
face; frontiers, myths, limits, plurality, territories, flows, memory, flavours,
aroma or rhythm. You can then explore each site’s layered images by double-
clicking them – ‘rhythm/pulse’, for instance, begins with Luciano Berio,
Italian composer, with a link to the Wikipedia entry on Berio, the Albanian
singer Edi Zara and the Spanish rap artist La Mala Rodriguez. ‘[L]imits’ intro-
duces us to the singer Fortuina, then links us up with Wilson Sukorski’s
poetic Noosphere, focusing on boundary-crossing artists presented in con-
tinuously shifting images such as Okay Temiz, the Turkish jazz percussionist
whose unique ethno-jazz is the result of his crossing both physical and cul-
tural borders and confronting different music styles, or the Franco-Algerian
musician Rachid Taha. All these entries are accompanied by artists’ websites
or biographies, geographical maps, blogs, bibliographies and photologs. At
the time of downloading, the music databank provided 33 different kinds
of music from all parts of the Mediterranean zone which, apart from the
front map with its meshwork opening itself to the manifold images, is the
only ‘constant’ element in this dynamically altering interactive mapping
enterprise.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


92 Media Borders of Qualified Media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Figure 3 Lucia Leão, Hermenetka (2005–2007): ‘What is the Mediterranean for you?’
Courtesy of the artist

The second kind of mapping consists in answering the question ‘What is


the Mediterranean for you?’ in an html-window (see Figure 3). This encour-
ages map users to contribute their own images, texts and links, triggering
further cyberspace research to be entered on the site, thus constantly com-
posing a unique and instantaneous map. In both mappings, images are
generated at random and composed of different sizes and levels of transpar-
ent overlaying of images and texts. It is hard to escape the impression that
the context here is that of advertising propelling us into the illusion that
the world and everything in it is simply what we – with enough resources
and ingenuity – are able to make it into. At the same time, it is indicative of
the extent to which the ethos of consumerism permeates our sensibility as
the interactive character of the site flatters the informed and enlightened
consumer as setting the terms of interaction. But these are in fact con-
stantly negotiated and renegotiated, and here we, too, are agents in a field
of crisscrossing forces in which we can never achieve full control.
Populating the various territories with a seemingly unending supply of
sources of historical entanglements, myths and sensorial inputs as streams
in which we are caught up and carried along in the form of visual and
aural media both deconstructs known space and reconstructs it all over
again. The interaction between the various media – images, film, television,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Christina Ljungberg 93

music, books, the internet – uses the visual, the aural and its evocation
of smell, taste and tactility, thus playing on our sensory apparatus. The
sensorial inputs in their attempts to evoke flavours and aromas play no
small part here, making the site both performative and self-referential. More-
over, it is continuous, as the various levels flow into one another; it is also
dynamic, constantly developing new, augmented alternative cartographies

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of images and sounds from visitors’ suggestions and new combinations with
constantly changing perspectives and positions.
To conclude, what becomes obvious here is that an intermedial analy-
sis demands a multi-level specification of the particular elements of matter,
energy, skill and technology that are involved in medium-specific practices.
However, considering the extent to which we are caught up in media and
the ways that our actions and reactions are functions of the energies and
trajectories of the media themselves, an investigation into what spaces of
interaction, what degrees of performativity and of self-referentiality are gen-
erated in such reconfigurations is even more called for. This becomes yet
more urgent since our involvement is such that both our identity and agency
can be defined in and through immersion in media, which makes necessary a
penetrating understanding of the relation not only of practitioner to practice
but also of the interaction among media effects.
This interplay results in both artworks in something new and unique and
is therefore radically performative. Anderson’s condensation of Fassbinder’s
transformation of Döblin’s novel produces a performance which is both pre-
cise and poetic and will also differ whenever it is experienced. This makes
each viewing instantaneous since it projects new space emanating from the
cultural, political and social memory provided by the ‘pretexts’ it evokes.
This space is liminal and unique; apparently, it is also immediate, though
upon analysis it turns out to be a complex mediated affair. Such uniqueness
also applies to interactive digital art works like Leão’s dynamic mapping: its
fortuitous cartographies consist of an immense amount of incalculable com-
binations and variations as well as being potentially unlimited in the sense
that, whenever a new element is added to it, the constellations change. The
‘actual’ map is in continuous metamorphosis, responding to the input of
the user or interactor. They may seem to involve different degrees of perfor-
mativity – one theatrical, related to the space generated in the performative
interaction between performer and audience, the other in the dynamic space
produced in the interactive response between the mapper and the computer.
Nevertheless, they are not only both radically performative but also pro-
duced by not entirely dissimilar processes, as they both address time and
cultural (as well as personal) memory. Both take place within a certain rit-
ual: Laurie Anderson’s polyphonic amalgamation of various cultures, media
and time is played out to an audience as a live performance with all its
inherent rituals – even as a video recording. In a sense, this also pertains
to Lucia Leão’s Hermenetka, which involves a ‘cyberart ritual’ (turning on

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


94 Media Borders of Qualified Media

the computer, opening the website, clicking on the various links). Its per-
formative relationship is rather similar to the one created by Anderson’s
performance during which the audience as much as the artist becomes
responsible for what is brought into being by such dynamic works of art.
After all, the cybercartographer equally perceives and experiences the map-
pings from the point of his or her subjectivity and maps them into his or

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
her mental geography. Moreover, both in Anderson’s White Lily and Leão’s
Hermenetka self-reference is used as an aesthetic sign whose formal structures
and rhetorical strategies evoke the materiality of the performance, but in
different ways. That is also why they involve different degrees of intermedi-
ality: as we have seen in White Lily, Anderson’s multimodal Gesamtkunstwerk
which consists of invisibly sutured ingredients, there is a self-referential use
not only of syntax and sound in her lyrics (prosody, assonance, allitera-
tion and the sound editor) but also in the ways she deploys music, gesture
and visual animation. Whereas Leão’s cybermapping is closer to the concept
of ‘nesting’, in which one medium appears within another as its content,
Anderson’s intermediality could be defined as ‘braiding’, the multimodal
but seamless weaving together of disjunctive elements into a seemingly
continuous narrative.
Intermedial works of art therefore not only involve intermedial border
negotiations and media transgressions, they also concern performativity and
high degrees of self-reflexivity. Furthermore, it is precisely the efficacy of
these dialogic negotiations that makes them meta-negotiations, or are they
re-negotiations? Are not the terms on which we undertake such negotia-
tions always already set by existing structures of power, thereby forcing us to
renegotiate the very framework of negotiation itself at the same time as we
are renegotiating some specific point of negotiation or contestation? All this
may explain why intermediality is such an efficient, complex, elusive and
exhilarating communicative strategy.

Notes
1. Anderson’s first performance appeared one year after Fassbinder’s death, which
hardly seems coincidental.
2. R. W. Fassbinder (1980/2006) Berlin Alexanderplatz (Munich: Süddeutsche Rund-
funk Cinematek).
3. L. Anderson (1986) Home of the Brave (Warner Bros).
4. A. Döblin (1929) Berlin Alexanderplatz, translated by E. Jolas (London:
Continuum).
5. W. Nöth (2000) Handbuch der Semiotik, 2nd edn (Stuttgart: Metzler), p. 467.
6. C. S. Peirce (1931–1958) Collected Papers vols. 1–6, C. Harthorne and P. Weiss (eds),
vols. 7–8, A. W. Burks (ed.) (Cambridge MA: Harvard University Press), 2.228.
7. W. Nöth (1990) Handbook of Semiotics (Bloomington: Indiana University Press),
p. 42.
8. L. Santaella (2001) Matrizes da linguagem e do pensamento (São Paulo: Iluminuras).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Christina Ljungberg 95

9. W. J. T. Mitchell (2005) ‘There are no Visual Media’, Journal of Visual Culture 4(2),
pp. 257–66.
10. V. Colapietro (forthcoming) ‘Pointing Things out: Exploring the Indexical Dimen-
sions of Literary Texts’ in H. Veivo, C. Ljungberg and J. D. Johansen (eds) Redefin-
ing Literary Semiotics (Newcastle-on-Tyne: Cambridge Scholars Press), pp. 109–33.
11. Nöth (2000) Handbuch der Semiotik, p. 436.
12. Mitchell (2005) ‘There are no Visual Media’, p. 258.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
13. M. Changizi, Z. Qiong, Y. Hao and S. Shinsuke (2006) ‘The Structures of Letters
and Symbols throughout Human History are Selected to Match those Found in
Objects in Natural Scenes’, American Naturalist 167(5) E, 117–39.
14. See M. Nänny (2002) ‘Ikonicitet’ in H. Lund (ed.) Intermedialitet (Lund: Studentlit-
teratur), pp. 131–38.
15. C. Ljungberg (2004) ‘Between Reality and Representation’, Peirce and the Notion of
Representation [Special Issue] VISIO 9, pp. 67–78.
16. M. Bal (2005) Travelling Concepts in the Humanities: A Rough Guide (New York:
Routledge), pp. 175–6.
17. E. Fischer-Lichte (2005) Theatre, Sacrifice, Ritual: Exploring Forms of Political Theatre
(London: Routledge).
18. E. Fischer-Lichte (2004) ‘Culture as Performance’, keynote lecture at the Sympo-
sium Performativity: A Paradigm for the Studies of Art and Culture in Copenhagen,
30 November.
19. V. Colapietro (1997) ‘Peircean Reflections on Gendered Subjects’ in C. W. Spinks
and J. Deely (eds) Semiotics 1996 (New York: Peter Lang), pp. 178–88.
20. M. Bal (2003) ‘Visual Essentialism and the Object of Visual Culture’, Journal of
Visual Culture 2(5), p. 24.
21. W. Nöth and L.Santaella (forthcoming) ‘Literature and the Other Arts: The Point
of View of Semiotics’ in L. Block de Behar (ed.) The Role of Comparative Literature in
the Sharing of Knowledge and in the Preservation of Cultural Diversity (Oxford: Eolss).
22. Cf. S. Moser’s (2007) analysis in E. Tabakowska, C. Ljungberg and O. Fischer,
Insistent Images (Amsterdam: John Benjamins).
23. Döblin (1929) Berlin Alexanderplatz, p. 2.
24. Ibid., p. 83.
25. Ibid., p. 2.
26. W. Nöth (2007) ‘Self-Reference in the Media: The Semiotic Framework’ in W. Nöth
and N. Bishara (eds) Self-Reference in the Media (Berlin: Mouton de Gruyter), p. 19.
27. L. Santaella (1997) ‘The Prephotographic, the Photographic, and the Postphoto-
graphic Image’ in W. Nöth (ed.) Semiotics of the Media: State of the Art, Projects and
Perspectives (Berlin: Mouton de Gruyter), pp. 121–32.
28. Nöth (2007) ‘Self-Reference in the Media’, p. 3.
29. Ibid., p. 19.
30. Ibid.
31. Ibid.
32. Not only can digital art forms be accessed anywhere and at all times but they also
demand interactivity on the part of the addressee.
33. Mitchell (2005) ‘There Are No Visual Media’, p. 262.
34. L. Leão (2005) Hermenetka, http://www.lucialeao.pro.br/hermenetka.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
This page intentionally left blank

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Part III
Combinations and Integrations
of Media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
This page intentionally left blank

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


5
‘Media’ before ‘Media’ were
Invented: The Medieval Ballad
and the Romanesque Church

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Sigurd Kværndrup

My principal aim is to demonstrate a special variant of the spatiotemporal


modality, which I propose to call the ‘numinous mode’, a result of the inter-
face between a sacred and a secular space, a meeting that took place in the
very Romanesque church building (cf. Elleström, this volume). Understand-
ing this mode is necessary in order to grasp how a seemingly secular art form
such as the medieval ballad could be created and performed in the church,
as we can see happen in a Danish church painting from around 1300: the
ballad dance in the church of Ørslev (see Figure 4).
It would hardly be controversial to describe the ballad as a pan-European
art form with roots in the Middle Ages, but I understand it as an interme-
dial art form in close connection with the church of the early High Middle
Ages. This is controversial for at least two reasons. Firstly, one consequence
is an early dating of the art form, whereas the tendency in much interna-
tional ballad research over the last 30 years has been to deny the ballad’s
ancient origins. Secondly, the ballad is normally understood as an aspect of
a non-Christian sub-culture in this Catholic age.1 In the Middle Ages, most
art forms were born as intermedial and were performed in direct contact
with a public which often functioned as active co-artists in the performance.
The church interior, which in cathedrals unites many different media and
modalities into a grandiose unity, could probably be looked upon as an early
mass medium, as it communicated its messages to many people at the same
time, and not only through the spoken word. This way of thinking about
the notion of medium is originally due to the works of Marshall McLuhan,
which shall be considered below.

Understanding medieval media

Before the publication in 1964 of Marshall McLuhan’s Understanding Media,


the normal state of affairs in medieval studies was ignorance about media.
Certain Nordic philologists began to use the notion in the late sixties in

99

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


100 Combinations and Integrations of Media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Figure 4 The ballad dance sublime; the church of Ørslev, Denmark, ca. 1300

relation to, for example, Icelandic family sagas. This orientation towards
the notion of medium, no doubt, was inspired by American media theory –
regarded by some with considerable scepticism – while others found that
McLuhan’s essay contained very interesting perspectives. What made this
highly regarded work a new beginning in fields like media and mass com-
munication? First of all, Understanding Media offered a universal notion for
medium in defining it as ‘the extensions of man’, tools to prolong the phys-
ical and spiritual capacity of mankind.2 In this way, McLuhan built up his
notion of media from the bottom, for not only the book, the painting, the
radio and other electronic media are extensions of man, but so too are the
wheel, the house and the bridge. In addition to this, he offered a provoca-
tive and inspiring motto, ‘the medium is the message’, which threatened to
drive classical humanists mad. For humanists the content was the message.
McLuhan defended his motto this way: ‘This is merely to say that the per-
sonal and social consequences of any medium – that is, of any extension of
ourselves – result from the new scale that is introduced into our affairs by
each extension of ourselves, or by any new technology’.3
Looking back upon the development in media technology over the last 30
years, one can hardly disagree with McLuhan that all of these new ‘exten-
sions of man’ have carried with them new messages, new publics and new
possibilities, as well as a radical change in culture, which became part of ‘the
global village’.

The writing and the law

No doubt McLuhan also has something to offer the study of medieval media
reality, primarily in the ninth chapter of his book, titled ‘An Eye for an

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sigurd Kværndrup 101

Ear’, which describes writing as a medium. Its main point is that the pho-
netic alphabet, which was re-diffused during the Middle Ages, is essential
for the transition from local tribal societies to European feudalism based on
commonly accepted law. Ever since the ninth century, manuscripts of law
were produced in greater amounts, not for public enlightenment but as holy
tokens, supporting the authority of thing-men who had knowledge of the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
powerful script. Authorities used the medium of writing to create common
standards of law, administration of justice and, thereby, social order, based
on feudal principles in the broadest sense. Apart from this historical aspect
of Understanding Media, McLuhan’s greatest importance for the study of pre-
modern culture and communication lies in the way he opens up a new field
of studies by breaking down the division between what is normally called
media and other forms of ‘extensions of man’, first and foremost in the field
of architecture.4
It seems that McLuhan anticipates a main line, one also found in interme-
dial theory, which is characterized by this very broad sociological notion of
media that are treated in rather mechanistic terms. Of course, this causes
problems for any intermedial research that is interested in showing the
boundaries between media and the interaction between them. Therefore, his
approach requires some demarcation, for if we go back to McLuhan’s notion
of media, it is generalized to a level where medium is more or less the same as
the mediaeval notion of Ars (art and handicraft). Ars also covers the wheel,
the house and the bridge – everything that man through his or her God-
given, creative power could add to Natura, God’s creation. Even though the
Middle Ages did not normally distinguish between craft (everyday creativ-
ity) and art (exceptional and highly qualified forms of expression), this is a
necessary distinction from the point of view of media theory. More specif-
ically, it was also with the rediscovery of ideas from Antiquity that efforts
were made to develop a narrower concept of art in the High Middle Ages,
based on the Greek doctrine of the nine muses, including those for the epic,
dance, comedy, tragedy, pantomime and last but not least, the song-dance
(Erato).5 Aristotelian aesthetics also enjoyed some diffusion towards the four-
teenth century, and in Dante Alighieri we already see traces of this. In his
De vulgari eloquentia, Dante supplies us with a very important key to under-
standing why ballad texts are not normally recorded during the Middle Ages:
he compares balata with canzona, and says that the latter is a finer art than
the former and it is therefore written down by ‘friends of the book’. The
canzona has a text which may stand alone, whereas the balata needs plausores
(dancers and clappers) to be performed.6 Thus, the ‘ballata’ was born as an
intermedial construction, and in Dante’s eyes, it is, therefore, of less value
than a real poetic text.
In the twelfth century the handwritten, vernacular liber was spreading all
over Europe and was later taken over by Scandinavian writers. The book to
the modern mind is a mass medium. It is an interesting fact that ‘a book’
in old Scandinavian sources is not something that one reads from; in the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


102 Combinations and Integrations of Media

almost totally illiterate North, the book was understood as a holy object on
which one could swear an oath in front of the altar – in other words, it was
also looked upon as a ‘medium’, but this was understood as a channel to the
godhead.7 Thus, some notion of medium did exist and was used as part of
the scholastic speculations about the spiritual world. It was obvious to every-
body who lived in the ‘modern’ world, that is, in early feudal society, that

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
there was a need for mediation to the faraway and sometimes invisible lords
of this world – be they divine or human, mediation was the central element
in communication. In early feudal society one met the vertical distribution
of power through vassals and servants, mediating between power and peo-
ple, and parallel to this in the spiritual world, one could find a special sort
of ‘media’, the saints and not least the Virgin Mary, called the Mediatrix of
all graces, an advocate contacted in prayers. Here we see the original meaning
of medium, which, in Latin, means to go-between two extremes; medium is
the bridge of communication between two parties who cannot communicate
directly, such as servant and king, man and God.
By using the Latin understanding of medium, it becomes possible to rec-
ognize a fundamental feature of McLuhan’s method, in which he stresses the
fact that the notion of metaphor in Greek means ‘to carry over’ or ‘to trans-
port’. From this perspective, the meaning of ‘metaphor’ becomes the same
as ‘medium’ in classical culture, a bridge. For McLuhan, the inventor of the
wheel was able to think metaphorically about running man, thus creating
a new ‘extension of man’, and therefore he can write: ‘All media are active
metaphors in their power to translate experience into new forms’.8
Medium and metaphor – the combination of Latin and McLuhan’s ideas
shows how these two notions are closely related: they both have to do
with the connection between a message, a channel and an interpreter. But
whereas ‘medium’ in the Middle Ages primarily referred to a spiritual, even a
transcendent, communicative function, the concept of medium in our time
is material, perhaps to the extent that we are inclined to forget this original
dimension of the concept.

The early medieval church as mass medium

After the huge Viking halls had disappeared, the church buildings became
the greatest ‘extensions of man’, though the notion of kyrka (church) comes
from Greek kyriakon meaning ‘an extension of the Lord’ (kyrios). Its holiness
was originally differentiated, however, into two very distinct parts: towards
the east the choir with apse ascends, while the longhouse or nave (from
the Latin word for ‘ship’) was directed towards the west and the sunset. In
the early High Middle Ages, this division had particularly significant conse-
quences for the church as a sort of mass medium: a rood screen or ‘triumphal
wall’ divided the Romanesque church into two separate parts, as does the
icon wall in contemporary Orthodox churches; the choir was accessible from

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sigurd Kværndrup 103

the nave through a door which was opened at Eucharist and at Easter. Only
the ecclesiastical personnel had direct access to the choir, which was, there-
fore, also called the presbyterium. Furthermore, the choir was conceived as
a parallel to the Holy of Holies in the Jewish temple, but whereas the Lord
was believed to be present in the Jewish temple, the choir was thought of
as a bridge to the Lord, metaphorically the ‘mountain’ on which the Lord

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of Hosts revealed himself to Moses. Therefore the choir was, ideally, to be
lifted a few yards above the ground and entered by steps, a metaphor for the
celestial staircase which the patriarch Jacob beheld in his dream. In this way
there was room for a crypt (from Greek kryptos, hidden), an intimate under-
ground room with an altar, a saint’s sepulchre and often tombs for noble
people; here, the laity might have access to a holy room – being a metaphor
for the access Maria Magdalena had to the empty tomb of Christ.
In the Jewish temple, access to the Holy of Holies was absolutely closed by
a heavy curtain and barred by two cherubim, similar to those who guarded
the gate to Paradise. Only the high priest entered once a year at the feast of
atonement. To the Christians, admission was symbolically obtained when
the curtain to the Holy of Holies was split from top to bottom at the death
of Christ, thus reopening the gate to Paradise (Matthew 27:51–5). The
aristocratic paintings in the Romanesque choir would normally depict the
creation, the history of salvation and the Kingdom of God coming to Earth.
The only choir painting that was sometimes visible to the common congre-
gation represented the Majestas Domini, the victorious Christos Pantokrator,
sitting in a mandorla on the rainbow above the globe as the true ruler of
heaven and earth, with his personal weapon, the book, in his left hand, and
reading ‘Sum Via, Veritas et Vita’ (John 14:6), his right hand lifted with a typ-
ical rhetorical gesture, teaching the world a new road to truth and life – by
use of the alphabet.
Thus, while the idea behind the Romanesque choir was deeply rooted in
Mosaic religion, the nave was a typical Roman edifice. It was normally built
on the model of the secular congregation hall of antiquity, the basilica, since
the prince or emperor (Greek basileus) had raised seats at the western end of
the nave, while there were no chairs for ‘the people on the floor’. The nave
functioned as a congregational house, and there was access for both men
and women. The sexes were separated, however, and each had an entrance:
the north door for women, the south door, which was through the porch, for
men. In the western end of the nave stood the font, whose natural waters
gave the congregation access to life and to the common ‘ship’, where the
priest delivered his sermon in the vernacular, while the mass was held in the
holy language, Latin.
The nave was not permeated with the same sacred secrets as the choir;
we know that the nave was also used for meetings, feasts, defence and for
chorea (antiphon combined with circle dance). In this division of the church
into two parts (largely forgotten today) lies the explanation of why carols

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


104 Combinations and Integrations of Media

and ballads might be performed in the church itself. This is only revealed,
however, in the later pictorial development of the nave during the Gothic
period.

The essential features of the iconic development

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
In the Romanesque period (before 1200), the nave was mainly decorated
with stylized sacred paintings, and they mostly told stories from the Old
Testament which were ante-types for the ‘new covenant’. Additionally, the
nave often showed pictures of the 12 apostles and the four evangelists. The
pictures were clearly inspired by the philosophy of the period, conceptual
realism.9 With the more powerfully ostentatious Gothic church, drafting
the church interior as a more or less holy unity, the building of bipartite
churches ended – and, with that, the construction of crypts. In many church
naves, the old wooden ceilings were replaced by vaults, giving the nave a
choir-like character, and the whitewashed walls offered room for new paint-
ings. The triumphal wall was partly broken down, leaving a triumphal arch
of sorts, upon which Christ was seen judging mankind at Doomsday, often
with a dramatic chasm of hell flaming to the left and to the right the peaceful
city of Jerusalem reserved for the redeemed.
In the Gothic period, not only the building but also the function and
ideology of the church decoration changed fundamentally. With a popular
didactic purpose, the paintings become intermedial in a new way. They now
describe human life as an eternal struggle between good and evil, between
virtues and vices, and the intermediality is completed with bands of text,
which interpret the pictures with pedagogical messages. Thus, it might be
said that the Gothic style of decoration took its departure from everyday
life, as did, in fact, nominalism, the typical philosophy of the later Mid-
dle Ages. At the same time, the Gothic decorations lent weight to secular
authorities, concerning the formulation of common laws. The pictorial pro-
grammes behind the new decorations were, to some extent, international,
such as Biblia Pauperum. Certain local, vernacular stories might also be used
as sources, especially at the western end of the nave and in the porch where
the road back to ‘the World’ might be anticipated with frescoes of the con-
ditions ‘out there’, where Satan was lord. Here, elements from the grotesque
world of the jocular ballads may be seen. During a period around 1300, how-
ever, one could also see paintings of a virtuous lifestyle, which might have
been borrowed from knightly ballads and novels.

The numinous mode and the sublime

The Romanesque church interior could thus be described as a compound


medium, built of architectural metaphors and sacred icons. By combining

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sigurd Kværndrup 105

the four modalities – the material, the sensorial, the spatiotemporal and the
semiotic – within the same church, a special mode of numinous experience is
created: a sensorial perception beyond words and, therefore, perhaps a fifth
modality arising out of the meeting between sacred and secular space. In
looking at the material and sensorial modalities in the church, one expe-
riences an enormous and static space, unlike in any other building. The

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
windows are small, so the sensorial impression of the interior is dark and
more intimate than the impression given from the outside – in accordance
with the wish to keep a solemn and normally silent atmosphere that could
sometimes be broken with the aural impression of plainsong. The church
was never heated, so in winter the congregation would not only have to
get used to the darkness but also to the cold. The paintings became gradu-
ally visible, opening up a unique spatiotemporal reality as the supplicant’s
eyes adjusted to the darkness, and this widened the spatial perspective with
a fourth and temporal dimension: most of the paintings portrayed holy
people from the past and told stories from the Bible; a few gave contempo-
rary impressions, but the most alarming were those glimpses of Doomsday
images from the future.
While the sacred space in the choir was mastered by the educated elite,
the nave was for lay people. The elite lived in a meditative and somber real-
ity, which was almost as static as the building, while the laity was part of a
movable and transitory physical world. Through the congregational choros
and antiphon in the nave, dynamic life was sometimes given to the church,
but in a stylized manner that combined the static spatial order with a tem-
poral perspective, simply by telling stories that were not ancient, but that
mirrored contemporary life, as did many church paintings. The ballad is one
of these art forms that could be performed in this setting, sub limes, meaning
under the border to the holy space and to God.10
The church also gave room to a rich semiotic modality, a symbolic
dimension expressed in a holy order defined through conventional signs,
assembled in an all-encompassing symbolical dimension, which was sub-
stantiated in the philosophy of conceptual realism. This brings Platon-
ism to mind, and historically this way of thinking was indeed partially
rooted in Hellenistic Neoplatonism. Its basic idea – expressed in Anselm of
Canterbury’s famous proof of God’s existence – is that mankind, made in
God’s image, is able to create and live by certain concepts and ideas that
are even more real than earthly reality (hence conceptual realism).11 As a
fallen creature under the influence of Satan, man may also dedicate him-
self to certain false ideas, for instance, one which states that God does not
exist. This realism meant that earthly conditions were understood in analogy
with those of heaven – with certain reservations, stemming from original
sin and the unspeakable greatness of God. This duality of truth and illu-
sions, or misconceptions, is depicted visually in many churches – and in
ballads.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


106 Combinations and Integrations of Media

The intermedial relation between choir (‘heaven’) and nave (‘Earth’)


may be interpreted from this observation: the nave is a multimedial and
multimodal space that is not holy itself, but gains its special atmosphere
and meaning from its meeting with a sacred space, thus creating the numi-
nous mode. This ‘medium border’ inspires man to prayer, incantation and
thanksgiving, and such spaces may function as performance arena both for

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the enacting of everyday stories (ballads), for misconceptions and ideas con-
cerning the Kingdom of God that the congregation had or was taught by
the priests and auctores. The road from idea to action (song, drama, art)
passed through realist philosophy, being an invisible, spiritual medium that
forged a bridge from heaven to Earth. An example may shed light on this
notion: music up to around 1300 was also built on the philosophy of con-
ceptual realism. The rich ecclesiastical singing imitated two sorts of heavenly
song: firstly the angelic choirs, imagined to be performed with polyphony,
secondly the spherical music of the seven planets.12 Both musical ideas
also inspired European composers, throughout time, to compose sublime,
polyphonic music.

The ballad and the problem of cultural interpretation

As an art form, the ballad in the Faroe Islands and the Balkans today com-
bines antiphonal singing, epic poetry and ring dance; it is, in short, inher-
ently an intermedial art form where song and text are always integrated,
while there is some uncertainty as to whether the dance was originally inte-
grated or just combined with the songs. My position is that the ballad is
originally and primarily a choral dance (Greek: choros), closely related to the
carol, a French term that indeed derives from choros. I understand the ballad
as a total work of art, a Gesamtkunstwerk, which integrates four arts that we
would probably call media today: chain dance, antiphonal song, poetry and
image.13
The majority of the East Nordic ballads belong to the genre called ‘ballads
of chivalry’, their narrative universe seemingly links them to courtly cul-
ture, a culture that has been viewed as being relatively independent of the
Catholic Church. If the Middle Ages was a non-secularized period, then, the
ballad was interpreted as one side of a residual and relatively secular and ver-
nacular aspect of medieval culture. This view seemed to be fully corroborated
by another ballad genre, the jocular ballad, which does not appear to have
any affinity at all with Catholic piety; on the contrary, since M. Bakhtin this
genre has been portrayed as an impious, carnivalesque culture of laughter.14
How, then, should we understand the ballads of the supernatural, which
often have a Christian theme, to say nothing of the legendary ballads, which
can even have the distinct elements of the late medieval sermon?
This dichotomous way of interpreting medieval artefacts is outmoded –
as medievalists will know; but it is far from being eradicated, least of all in

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sigurd Kværndrup 107

ballad studies. For my own part, it was a lecture about the carnival, deliv-
ered by Bishop Joseph Ratzinger at the Saga Conference at the University of
Munich in 1976, that was the most important eye-opener. Here, the future
Pope Benedict XVI interpreted the carnival in biblical terms and based on
the Catholic tradition, which created, and still energetically maintains, this
festive practice. Since ancient times, it has comprised customs such as noisy

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
processions whose participants can sometimes perform naked or dressed as
absurdly as they please, while vigorously beating empty barrels and shak-
ing rattles. They thereby illustrate the anti-numinous point of the carnival,
which is to show off the naked, biological human being, without Christian
charity and without knowledge of God. Bishop Ratzinger demonstrated that
this pre-Lenten festival of the forsaking of meat (carne vallo) has its bibli-
cal foundation in the words of Paul in 1 Corinthians 13: ‘Though I speak
with the tongues of men and angels, and have not charity, I am become as
sounding brass, or a tinkling cymbal’. Obviously, it was not the Church that
invented the pagan art forms through which the carnival expresses itself,
but the relationship illustrates that the Catholic Church in the Middle Ages
was surprisingly spacious.

Choros as an ecclesiastical medium

The choir was a kind of mirror of heaven and when joyful events happened
in heaven, such as the coronation of Mary as the heavenly bride of Christ, it
was written that the angels danced circle dances (chorea) in joy. As an analogy
to this, the congregation could dance chorea of joy in the nave, which was
seen as the antechamber of paradise.15 Does that mean that people could
dance and sing carols in the church itself? Certainly; but such activities were
forbidden by numerous councils and bishops, revealing that they did occur
anyway.16
Carol (from Greek choros) integrates circle dance with a text that is sung in
interaction between one or two solo voices, and burdens that are performed
by all as a choric response to the solo. The strophic Nordic ballad genres are
normally of exactly the same form. The response burdens may emphasize
a particular point of view on the outcome of the story and contain more
or less hidden references to the context of the performance.17 At Yuletide,
this dance activity was especially widespread. One well-known Christmas
carol, ‘A Child is Born in Bethlehem’, has been translated from a medieval
antiphony and is performed in the East Nordic rhymed ballad form of the
short-lined type, a couplet with two burdens and with four rhythmic beats
in each line, alternating with three beats. As a form, this strophe is also
known from popular legendary ballads like ‘Jungfru Maria och Jesus’ (‘The
Virgin Mary and Jesus’), which was widespread over Scandinavia. The idea
that governs this ballad is that the child Jesus only grows up and recognizes
his mission on Earth the moment he runs away from his mother. She is

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


108 Combinations and Integrations of Media

anxiously chasing around, searching everywhere, to find her boy (here the
E version from Västmanland in 1830):

Jungfru Maria till Bethlehem gick (four beats) (The Virgin Mary went to
Bethlehem)
–Låfvadt vare Guds heliga namn! (burden with four beats) (Praise to God’s

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
holy name)
Der födde hon Jesum kär sonen sin (four beats) (There she gave birth to
Jesus, her dear son)
–Och så den helige and! (burden with three beats) (And likewise The Holy
Ghost)

E. L. Backman demonstrates in his Religious Dances in the Christian Church


and in Popular Medicine (1945, English version 1952) that the ecclesiastical
choir-dances may be traced back to the Christian church of Antiquity.18 They
had their roots in Greek and Jewish festive traditions but there are also exam-
ples from the New Testament that might serve as models, one of these being
in St Luke’s Gospel (7:32) where Jesus says to the Jews: ‘We have piped unto
you, and ye have not danced’. One of the strengths of Backman’s thesis is
that he has read the scriptures of the Fathers and translated their statements
about dancing. It is quite clear that most of the Fathers are ambivalent, for
though dancing is a pious way of expressing the joy of God, according to
the Bible, how it is practiced by former European pagans is another matter –
particularly its content and effect. Dancing for joy in heaven is found in a
church painting in the church of Ørslev in southern Zealand from around
1300. Here, an open chain of beautiful virgins and noble knights is dancing
the choros before Christ and the Virgin Mary, who are symbolically united
in the Holy Marriage. The painting very clearly connotes the universe of the
ballad, for on the next wall the same painter has shown vernacular wooing
and wedding images from the motif world of the knightly ballad. The great
Swedish painter Albertus Pictor (ca. 1440–1507) painted such ring dances
in three Upplandic churches: 19 in Husby-Sjutolft one sees the same sort of
dance of joy, but here the occasion is the birth of Christ.
The ecclesiastical choros must have been one of the origins of the medieval
ballad, but certainly not the only one, and time and again the Church turned
against the art form and banned it. However, this should not lead ballad
researchers to forget that this old intermedial art form also had its place
in the church.20 Indeed, the folk ballad in stanza-form was originally an
ecclesiastical medieval carol. Like the many people originally baptized in the
church and who later left it, however, the ballad also parted from its eccle-
siastical origin. The carol integrates chain dance with a text that is sung in
interaction between one or two solo voices, and burdens that are performed

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sigurd Kværndrup 109

by everybody. The folk ballad makes exactly the same performance. When
dealing with the early media of the Middle Ages, one has to take into account
primarily that they were born as intermedial forms and, secondarily, that
integration and even transformation of media from one cultural sphere to
another was of much greater importance than one would imagine today –
the Church being the most important melting pot for media exchange before

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the Late Middle Ages, when the market place took over.

Notes
1. I have at least one forerunner in this undertaking, namely the American bal-
lad scholar L. Pound (1921) Poetic Origins and the Ballad (New York: Macmillan).
Louise Pound took the standpoint that the earliest Anglo-Saxon ballads were of
ecclesiastical origin.
2. The full title of the work is: M. McLuhan (1964) Understanding Media: The
Extensions of Man (New York: McGraw-Hill).
3. Ibid., p. 7.
4. Cf. C. Dreyer (1997) ‘Architecture as a Mass Medium’ in W. Nöth (ed.) Semiotics of
the Media: State of the Art, Projects, and Perspectives (Berlin and New York: Mouton
de Gruyter).
5. Cf. U. Eco (1986) Art and Beauty in the Middle Ages (New Haven CT: Yale University
Press).
6. Dante Alighieri (1966) De vulgari eloquentia: Über das Dichten in der Muttersprache
(Darmstadt: Wissenschaftliche Buchgesellschaft), p. 54.
7. See further description by T. Damsgaard Olsen (1984) in S. Kaspersen,
S. Kværndrup, L. Lönnroth and T. Damsgaard Olsen, Dansk litteraturhistorie 1.
800–1480 (Copenhagen: Gyldendal), p. 13.
8. McLuhan (1964) Understanding Media, p. 57.
9. See M. H. Carré (1946) Realists and Nominalists: An Introduction to Medieval Views
of Knowledge (London: Oxford University Press).
10. It is quite likely that through that numinous mode, art (in the modern sense of
the word) was separated from ars, thus creating that gulf between handicraft of
lower and higher or even sublime value, which is so characteristic of European
culture since the Renaissance.
11. The proof is very short: ‘And, indeed, we believe that thou art a being than which
nothing greater can be conceived’: Saint Anselm (1962) Basic Writings, translated
by S. W. Deane, 2nd edn (La Salle IL: Open Court Publishing Company), p. 7.
12. One of the first examples of imagined polyphony is depicted in Le chevalier au lion
by Chrétien de Troyes around 1180, in the scene where a knight is experiencing
a miracle by a magic fountain.
13. These may be based on ekphrasis of everyday images, produced by women in dif-
ferent visual media. I argue to this effect in S. Kværndrup (2006) Den østnordiske
ballade – oral teori og tekstanalyse: Studier i Danmarks gamle Folkeviser (Copenhagen:
Museum Tusculanum) (The East-Nordic Ballad: Oral Theory and Textual Analy-
sis). Danmarks gamle Folkeviser means ‘The Ancient Popular Ballads of Denmark’,
vol. 1–12, 1853–1972 (Copenhagen: Universitets-Jubiläets danske Samfund).
14. Cf. O. Solberg (1993), Den omsnudde verda: Ein studie i dei norske skjemtebal-
ladane (Oslo: Solum) (The World Upside Down: A Study of the Norwegian
Jocular Ballads) and T. Olofsson (2008) ‘Den magstarka käringen: Om roliga

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


110 Combinations and Integrations of Media

medeltidsballader’ (The Unbelievable Woman: About Funny Medieval Ballads)


in G. Byrman (ed.) En värld för sig själv: Nya studier i medeltida ballader (Växjö
University Press) (A Special World: New Studies in Medieval Ballads).
15. Each ballad type is a unique organism, and each ballad version is a unique poem.
In B. R. Jonsson, S. Solheim and E. Danielson (1978) The Types of the Scandinavian
Medieval Ballad (TSB) (Oslo: Universitetsforlaget), p. 14, the editors write: ‘A ballad
is an “idea” in the Platonic sense; there is no archetype; from the point of view of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the folklorist, every text is as good as any other.’
16. Cf. C. Sachs (1937) World History of the Dance (New York: W. W. Norton & Co).
P. Bourcier (1978) Histoire de la dance en Occident (Paris: Le Seuil), gives many
examples of ecclesiastical prohibitions against carolling in church, beginning as
early as 774, when Pope Zacharias decreed a ban against ‘les chants et caroles des
femmes à l’église’ (p. 52). Bourcier adds that the persistence of the bans proves
the persistence of the dance (p. 53).
17. In the ballad content the dual reality is very often pictured, and in quite a few bur-
dens the duality between ‘world’ and ‘heaven’ is expressed, for instance in SMB
48, where the first burden sings, ‘Om sommaren’ (In summertime), and the sec-
ond ‘I himmelen är en stor glädje’ (In heaven there is great joy). SMB is short for
B. R. Jonsson, M. Jersild (tunes) and S.-B. Jansson (1983–2001), Sveriges Medeltida
Ballader 1–5, (Stockholm: Almquist & Wiksell International).
18. E. L. Backman (1945) Den religiösa dansen inom kristen kyrka och folkmedicin
(Stockholm: Norstedt). English version: (1952) Religious Dances in the Christian
Church and in Popular Medicine (London: George Allen and Unwin).
19. In Härkeberga and Täby the paintings are rustic ring dances around the golden
calf; the hand position of the dancers reveals that they are dancing a ballad.
Albertus Pictor has painted four clear motifs from ballads in Floda church.
20. Some researchers of ballad tunes saw a near relationship between ecclesiastic
tonalities and some ballad tunes. The relationship may be interpreted to the effect
that the origin in church music can be glimpsed behind several generations of
singers who have transformed the tunes beyond recognition.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


6
The Intermediality of Field Guides:
Notes Towards a Theory

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Håkan Sandgren

In field guides the mediating practices used are called media combinations
(see Lars Elleström and Irina Rajewsky, this volume), that is, a multifaceted
medium which contains two or more kinds of media modes organized and
combined in such a way as to create a certain perceptive and cognitive effect.
It is therefore not of any major relevance in this context to speak of bor-
der crossings, due to the fact that the genre makes use of the differences
and delimitations of the media contained in it, rather than transgress-
ing or dissolving them. The media transgressions made while using a field
guide nevertheless seem mainly to be a side effect of our way of perceiv-
ing the media modes it contains. Our attention wanders from the object
we would like to identify to image, text, maps and out to the environment
surrounding us, and eventually back to the physical object we are trying to
identify, in a process that at least superficially seems to be related to the
cognitive activity triggered when decoding a page containing both text and
image.1
That the field guide should be considered an art form is a more dubious
proposition, because the purpose of a field guide is never restricted to evok-
ing aesthetical effects or giving the user an artistic experience by means of
different media. If a field guide, nevertheless, in some way or other, should
be considered a work of art by a recipient, this must be seen as a side
effect, not as its primary aim. It is, though, in constant change; primarily
the changes affect those aspects of the medium which Elleström in this vol-
ume calls ‘contextual qualifying aspects’ and ‘operational qualifying aspects’
due to the fact that the field guide has become a necessary part of a social
context, for example, the birdwatching community. Birdwatchers have cre-
ated a discourse including a qualitative stratification of the different items of
the genre due to their interest in how well the field guides relate to current
research and to contemporary reports of bird observation. Apart from this,
the genre is also defined by an instrumental, or rather operational, mul-
timodal form, which is a necessary condition of the genre to function as
intended: as a medium used for identification of natural objects in the field.

111

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


112 Combinations and Integrations of Media

That said, it is obvious that studies of field guides could be carried out in
the same way as we study other forms of intermedial and multimodal forms
of communication. Owing to the historical and popular dominance of bird
guides, this essay is based mainly on examples taken from this quite abun-
dant sub-genre. Hopefully, this will not affect the general conclusions being
made, for, in any field guide, we will find combinations and interactions

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of images and prose, describing the object in a simple way, sometimes in
narrative form, with genre-specific types of verbal figures. Description is by
way of a visual, textual or auditory medium invoking different kinds of cog-
nitive processes and experiences of a physical reality, as convincingly and
unambiguously as possible.2
The field guide typically contains a short introduction, a user’s guide to
the best way to identify the objects in question. If the field guide is a guide
to flora or fauna, the evolutionary relation between orders, families, gen-
era, species, sub-species and races is explained. In the introduction we will
probably find images, illustrations, maps and sometimes sonograms, the
latter serving as visual explications of bird song.3 Field guides of animals
and plants are often organized in line with a taxonomical order based on
the interrelation between the different species. Although based on scientific
research, intradisciplinary discussions are rare in field guides, rather, recent
research results tend to ‘trickle down’ to the books, however slowly, and
sometimes randomly, and will sometimes be included in new, revised edi-
tions.4 The guides are often equipped with so-called keys; this is especially
true of plant guides, where keys present stylized images to illustrate similar-
ities and differences in the shape and form of the different kinds of leaves
and flowers.
For the ordinary user, identification is equivalent to naming the object.
Scientific naming of birds follows a strict but ever-changing nomenclature,5
but for the common user the popular name is often the most often searched
for, a fact that has encouraged the authors of guides to ‘invent’ popular
names for organisms.
The argument of this essay is that the intermediality of a field guide is
based on the cooperation, dialogue and mediating process between image,
text, map and transcription or interpretation of bird sound; mediated
description is connected to the natural ‘field’. Not all of the mediated aspects
need to be simultaneously activated by the reader and observer, but the user
has to take part in an intermedial process and make use of all different kinds
of media in a field guide, while he or she compares the medium with the
physical reality. For a field guide to work properly we must assume that it
combines in a pragmatic way different media modes in an act of gaining the
highest possible informative effect. As Elleström proposes in this volume,
media could be understood in terms of four kinds of modalities, and the
field guide is very much defined by the semiotic modality, but as is stated in
this essay, it is a genre of combined media, taken into active use in the field,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Håkan Sandgren 113

which means that some specific kind of material mode also must be
associated with the genre. This is due to the fact that in the case of the field
guide the function, not the aesthetic value or effect of it, is of primary inter-
est. If such a standpoint is taken, the reader of the field guide will first and
foremost become a user who by cognitive processes treats the different kinds
of information which text, image, transcription and maps are communicat-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ing, in such a way that he or she succeeds in his or her goal of identifying
the object.

The image

The image serves a certain purpose: to assist the user in identifying a specific
species. Because of this, distinctiveness is of the utmost importance. Images
in bird guides often take the form of black-on-white silhouettes showing typ-
ical traits such as the shape of a wing or tail.6 Drawings in black and white
may be used as illustrations to a piece of narrative, as part of ‘how to use
this book’ passages7 or to give an approximate image of a very rare species.8
The main bulk of images in a field guide, however, are paintings – in water-
colour, oil, gouache or acrylic. In older guides the artist often used stuffed
birds or preserved skins as models; today, live birds, caught for ring marking
or observed in the field, are preferred. Birds in different variants of plumage
are illustrated due to the fact that they could be confused with other, similar
species, or differ considerably within the species. Variants of plumage used
in a field guide include adult, sub-adult, male, female or wearing breeding
plumages, winter plumage, moult and so on; some common birds, such as
the mallard, show a definite difference between male and female plumage
during breeding season. Other birds are similar, regardless of sex, but carry
a winter and summer plumage, and different kinds of plumage at different
ages. The object could be placed in a stylized natural habitat to give the
observer an idea of where and how the bird is most likely to be discovered;
more commonly the object is seen against a neutral background. The most
frequent way to present a species visually is by a so-called ‘mug shot’.9 In a
‘mug shot’ the bird is seen in profile against a neutral background. Certain
species may be painted in motion – for example, ducks, which on the plates
seem to swim upon a water surface, a position that obscures their legs from
view.10 Some images take into account a certain wing or tail pattern which
can only be observed when the bird is in flight.11 The way the artist uses light
is very important; objects must be shown in an even light to facilitate identi-
fication and the artist’s choice of perspective thus becomes an interpretative
act which differs from the way a nature painter would choose to construe
and depict a natural object as, for instance, partly dissolved, out of focus
or in shade. Some artistic possibilities are definitely excluded for the artist
illustrating a field guide: an artist may create an image of a moving bird in
its natural habitat, but he or she cannot take such liberties as to cover parts

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


114 Combinations and Integrations of Media

of the object or to deviate from what we may call the standard observation
position. Images in field guides are referential and they guide identification –
that is their main purpose. Identification is, in turn, based upon recognition,
an act in which the observer must be able to connect the visual artefact to
the natural object. This is why distinct and separating traits are highlighted
in the image and sometimes noted by a discreet marker, such as an arrow or
a thin line.12

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
The text

The style in field guides is matter-of-fact and the text has to act together with
the images to make identification possible. This does not stop the author
from using rhetorical tricks, sometimes quite artfully, or using narratives.13
A standard text accompanying an image gives us facts about the length of
the bird from beak to the end of the tail, wingspan and whether there is a
considerable difference in size between the sexes. Short notes indicating how
common or uncommon the species are, following a defined categorization,
such as ‘common’, ‘scarce’ and ‘very rare’, are necessary. Description of the
bird’s habitat is essential, as is information about its most common mating
localities. The text is often supported by a map which shows the ranges of
the species, during breeding season or during migration. To serve its goal,
the text is primarily neutral and descriptive. Sometimes the writer, when
verbally mediating the bird’s behaviour, gives it poignant, human-like traits.
The bird could also be associated with concepts of conscience or states of
mind. In this particular context, anthropocentrism and anthropomorphism
signify the way the text describes the bird as a part of an environment dom-
inated by humans. Anthropocentrism can take many forms, but its main
purpose is to describe bird behaviour in such a way that it is possible for
us, as Homo sapiens, to understand it. Identification of a species is facili-
tated by knowing how the bird will react when a human is in the vicinity;
the difference in human-induced behaviour could help us determine which
species we encounter.14 Now and again we will find examples of animal-
centrism in a bird guide, a concept that in this context would try to help
describe the behaviour of the bird in the presence of other birds (for exam-
ple, raptors). The text often contains a verbal interpretation of the bird’s
behaviour, appearance, qualities and particularities. It has to be as trans-
parent as possible by creating a verbal representation of the natural object,
sometimes with the help of tropes and figures typical of the genre. This leads
the writer to focus on differentiating details and traits, thereby helping the
user to identify the species by separating it from other similar ones. ‘Field
marks’ – prominent and distinctive details in the plumage, the shape of the
bill, song or calls – make it possible for the observer to separate one species
from another.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Håkan Sandgren 115

Transcription of bird sounds

A common feature in bird guides is the writer’s attempt to translate or


transcribe the sounds of birds into verbal equivalents or onomatopoetic
combinations of vowels and/or consonants, put together in such a way as to
make it possible for the user to compare his or her sound impressions with
the phonetic effects invoked by the arrangement of letters. Transcription of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
bird song is a subject under constant discussion,15 and verbal representation
of bird calls has a long history, for instance, in poetry.16 When the sound
of the call or song is described rather than transcribed, the use of metaphors
and similes is predominant in the verbal rendering. This should not come
as a surprise considering that the ordinary usage of these figures tends to
establish a resemblance between two objects – in this case the authentic
sounds of the bird and the textual ‘picture’ created by the writer to conjure
up these sounds in the mind of the reader. The metaphors and similes most
often used are taken from the sphere of nature; surprisingly often a compar-
ison is done by using other birds’ calls, a feature that seems to presuppose
a thoroughly ‘informed reader’. Occasionally the vehicle of the metaphor
is taken from culture, such as ‘unnatural’ sounds (‘rubbing rubber tubing’),
musical instruments (‘jarred trumpet fanfares’) or sounds made by humans
(‘a laughing tremolo’).17 In cases when transcription is made, the interpreter
makes use of the onomatopoetic potential of letters. Transcription is based
on more or less successful representation of natural sound (for example,
‘ksch i’i’i’t’,’küll-küll-küll’, ‘ptrr’).

Examples discussed

The two examples chosen as a ground for a discussion are generally consid-
ered to be the most important of Swedish field guides. A classic Swedish field
guide is Erik Rosenberg’s Fåglar i Sverige [The Birds of Sweden]. The popularity
of this book has led to a number of revisions since it was first published in
1953 and new species have been added continuously. The illustrations were
executed by the famous Swedish nature painter Harald Wiberg, an artist in
his own right.18 Apart from the fact that Rosenberg’s book was both a result
of and a cause for the rising popularity of birdwatching in Sweden, it became
part of the environmental movement. In this book, Rosenberg, as he did
in his other work, pointed out changes in population range that he knew
were due to the impact of mankind on the environment.19 It is of some
importance to point out that this field guide, like almost every other field
guide, limits its scope to a well-defined geographical-political unit. Rosen-
berg’s book contains a preface, a piece on the habitats of birds and a survey
of taxonomy. The main part of the book presents the birds on colour plates
with texts accompanying the images. A curious feature is the particular
chapter in which Rosenberg (1972) addresses the reader and urges him or

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


116 Combinations and Integrations of Media

her to inform the writer of observations concerning the red-flanked bluetail


(Tarsiger cyanurus), a rare bird that had recently been reported, though with
some uncertainty. Indeed, I have seen no similar passage in any of the field
guides with which I am familiar, but it informs us of the kind of communi-
cation necessary to make a guide work. The reader (that is, user) is addressed
in a way that presupposes a committed user, who goes out in the field to

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
observe and identify natural objects and thereby, in one way or another, put
them into some kind of mental archive.20
Lars Jonsson’s Fåglar i Europa med Nordafrika och Mellanöstern [Birds of
Europe, with North Africa, and the Middle East], the first edition of which was
published in 1992, differs in many ways from the work of Rosenberg, as
will be discussed shortly. It is important to note that Jonsson is both illus-
trator and writer, thus creating a closer affinity between image and text. In
the Preface, Jonsson stresses that he has taken on the subject of writing a
field guide mainly from an artist’s perspective.21 As the title indicates, Jons-
son’s field guide covers a larger geographical area than Rosenberg’s and as a
result more species are included. Even though the geographical boundaries
are somewhat arbitrary, the question of political borders is of no importance
here. In addition, Jonsson’s field guide is more in line in its scope and ambi-
tion with important international ones such as Birds of Eastern and Central
North America and Collins Bird Guide. Jonsson’s field guide goes into greater
detail than Rosenberg’s for two reasons: it covers a larger area and it strives
to satisfy the demands of current birdwatching, a popular pursuit that is in
constant development.

Images compared

To study different kinds of image usage in field guides, I will take as examples
the pages in Rosenberg and Jonsson showing the pintail (Anas platyrhyn-
chos) and the (Northern) shoveler (Anas clypeata). Wiberg’s illustrations are
plain ‘mug shots’ and are apparently modelled on stuffed specimens or pho-
tographs.22 The eight birds (on the single plate there are seven different
species) are seen in profile against a white background, floating tranquilly
on an imagined surface of water and arranged so as to appear at the same
distance from the viewer, which makes differences in size apparent on the
page. A kind of symmetry is created by the fact that four birds seem to swim
to the left of the page, while the remaining four swim to the right. All birds,
apart from a female mallard, are male specimens clad in breeding plumage.
Names in the vernacular in capital letters appear on the page, as do the signs
for male ♂ and female ♀. Jonsson’s images are of a totally different kind.23
Seven birds are seen on the page, but they are of only two species – pintail
and shoveler – seen in different kinds of plumage. Symmetry is achieved by
letting the four centrally positioned birds – male and female pintail and male
and female shoveler in eclipse plumage – swim in opposite directions. The

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Håkan Sandgren 117

image signals depth by presenting two pintails in the upper part of the pic-
ture, as if they are swimming at some distance away from the other birds. The
most significant part of the page is the image of the male shoveler in breed-
ing plumage. Distinct from the other birds, it stands on the shore showing
its red duck’s feet. Its head is turned towards the onlooker, which makes its
already prominent beak even more significant. As a way to underline move-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ment and realism, Jonsson has placed the objects in a sketchy, but distinctive
habitat – a lake and its shore – and the conceived movements of water are
underlined by a stylized surface, rings in the water and reflections.
The two examples chosen show how field guides can vary in the way they
present their visual material. This material can be of a very neutral kind,
without any artistic ambition, or it can come close to a work of art. The
function of the image, though, is always to make identification possible.

Text comparison

An intermedial relation exists between text and image when different cat-
egories of media are not only combined with each other, but also work as
complementary factors; the text focuses on differential visual patterns, and
the image draws attention to traits commented on in the text. A plate in a
field guide is always in need of this kind of intermedial interaction. Let us
see how a common bird – the European robin (Erithacus rubecula rubecula) –
is verbally presented in the books studied. In Rosenberg the bird is described
in the following way: ‘The bird is curious and will happily jump out of the
shrubbery to peacefully observe the human . . . Sometimes he curtsies . . . At
dusk he is at his most spirited humor’.24 We can notice how the writer in
the cited passage is using anthropomorphic expressions which mediate the
bird into a human context. This device could be explained as being a way
to enhance the possibilities of identification, enabling the observer to name
the bird with the help of verbal description. Possibly this is facilitated if the
bird’s behaviour is vivified by comparing it with human behaviour. Identi-
fication of the European robin seldom presents a problem, but occasionally
the bird is hidden and detection must be done by way of recognizing its
calls. Here a comparison between Rosenberg’s and Jonsson’s ways of song
transcription and interpretation will be carried out, starting with Rosenberg:

The song of the robin is instantly recognizable; it resembles a musical


fantasy, a few high-pitched, extorted chirps, followed by a long, extended,
trembling, quivering coloratura, which after a while almost dies away,
and then, a sudden shake-up and after that a pearl-string of crystal clear
diminutive noises.25

Note how the passage starts by pointing out the song as ‘instantly recogniz-
able’, a conventional concept in this particular context. The way the bird’s

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


118 Combinations and Integrations of Media

specificity is pointed out makes it easier for the identifier to reduce the range
of possibilities. A description of the song follows, finding its rich metaphors
from what could be a haut-bourgeois musical performance where the bird
song is compared to classical music (‘musical fantasy’, ‘coloratura’) and to
the accessories worn by the conventionally dressed audience, and found in
the environment of the event (‘pearl-string’, ‘crystal’). The sound of the bird

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
has little to do with music, but the metaphorical transformation of the sound
makes it possible for the listener to recognize the bird, provided the fanci-
ful description is interpreted by an imaginative and creative mind, that is.
Jonsson’s description is more low-key:

Its song is unmistakable; a murmuring rivulet of crystal clear tones with


capricious changes of meter. Of a defining character is a discreet ticking
noise: ‘tick’, often repeated ‘tick-ick-ick . . . ’ as when a mechanical play-
thing is wound up, often heard from a shrub . . . Also has a thin, sucking
‘siih’ and from night time travellers a thin ‘sie’ or ‘sisie’.26

In this passage the simile makes use of natural elements that act as vehicles
in the comparison (‘rivulet’). This places the bird in a different context from
the Rosenberg example. Jonsson, too, is using musical metaphors by point-
ing towards changes in meter, and to mechanics. ‘Crystal clear’ is a metaphor
which finds its comparative point in a natural object, but this phrase being
a cliché, connoting certain kinds of singing (bel canto, for instance), it is
associated with a cultural artefact. Of further interest in this example is
the way this very problematic transformational mediation develops some
kind of intertextuality, based on a given set of metaphors and similes. Both
Rosenberg and Jonsson use the word ‘crystal’ to describe the characteristics
of the European robin’s song, showing us a kind of verbal tradition or con-
ventionality even in this genre. In the cited passage by Jonsson we can also
notice the anthropomorphic and anthropocentric traits.
It is informative to compare Rosenberg’s and Jonsson’s transcriptions of a
quite different and more exclusive species than the robin. The chosen bird
is a warbler, a shy and quite plain kind of bird that is seldom seen, which
forces the listener to identify the species by its characteristic song. This is
how Rosenberg describes the song of the icterine warbler (Hippolais icterina):

As a singer the Hippolais is in the same advanced class as the bluethroat


and the marsh warbler; he’s a virtuoso, and a composer of masterful stro-
phes. The song consists of short, swift melodies, e.g.: t r a l l a l í t r a l l a l á
mixed with mimicry of other species’ sounds, such as the barn swallow’s
s i f l i t , the common tern’s k i t t , the chaffinch’s f i n k , the oystercatcher’s
k u b i k , all of them repeated several times. From time to time a chorus
is inserted, a scraping g í e g í e g í e (with the i-note similar to the sound

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Håkan Sandgren 119

of a wasp) and h i p p o l ý i t - g í e g í e g í e h i p p o l ý i t . The song is excep-


tionally sprightly . . . When the species is nesting the adult ones nervously
fly around in the trees surrounding the unwanted visitor, swinging their
heads from side to side in a funny way while they are heard to utter a
melodic h y i t or a quick thrust t a c k t a c k t a c k h i p p o l ý t .27

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
The text contains many verbal figures and tropes which characterize this
kind of mediation from one medium to another, which invites the author
to use a kind of verbal equivalent to translate the song into words. As a
start we may observe Rosenberg once again using words and phrases taken
from the discourse associated with classical music: ‘singer’, ‘advanced class’,
‘composer’, ‘melodies’ and so on. In addition, his description presupposes
knowledge of other birds’ singing to be fully decipherable. This is a peculiar
use of intertextuality limited to the bird guide. To complicate things even
further, the authentic song of the icterine warbler includes auditory mimicry,
a fact that makes the bird an intertextualist in its own right. The sound of
the wasp noted in the passage is not a mock sound, but a simile invented by
the author, but it is nevertheless interesting because the simile’s vehicle is
the sound of another organism. We can also recognize examples of classical
forms of anthropocentrism in the passage (‘the unwanted visitor’). Jonsson
describes the song of the icterine warbler in this way:

The song is masterful, varied, and rich in mimicry. Every strophe is


repeated 2–5 times, the beginning of the strophes often starts with a
starling-like sound ‘shrr, shrr, shrr . . . ’ Through it all a typical, whining,
violin-like tone is heard, like a soft, dwindling ‘sjuh liu’. The gathering
call, a soft yet explosive ‘tett e’eytt’ or ‘hippol’yit’ is also woven into the
song . . . a nasal ‘tjepp’ or ‘the’, agitated often combined with a ‘the the
the te-lü’, the short staccato sounds house sparrow-esque.28

There is no need to go into detail when commenting on Jonsson’s descrip-


tion of the warbler’s sounds, but, as we can see, the species pointed to as
interartial comparison are different. The transcription is quite close to Rosen-
berg’s, but the system of notation is somewhat different. The combined
vowel and consonant sound ‘hippol’ yit’ is also the onomatopoetic scientific
name of the genus to which this warbler belongs (Hippolais).

Ecocritical and intermedial conclusions

Whether or not field guides should be considered an art form may be a mat-
ter of debate, but I will not venture up that road, limiting myself to making a
humble statement pointing out the necessary existence of creativity, rhetor-
ical and metaphorical fancifulness, and an intertextual and intermedial skill
in the producer of a field guide. That the genre is a true intermedial medium

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


120 Combinations and Integrations of Media

is beyond doubt, though. The user must make use of image, text, maps and
recurrent observation of the field surrounding him or her for the field guide
to have any apparent meaning. It is therefore important to stress the neces-
sity of media combination and the cognitive operations performed by the
user to make this combinatory mediation of visual, semantic and audible
modes work in a satisfactory way. If we take this discussion further, we may

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
come to the conclusion that nature, in itself, is not only the basis of infor-
mation being processed by the different kinds of modes represented in the
genre, but actually a necessary element (though probably not a mode) which
in combination with the aforementioned ones makes up the specific charac-
teristics of the particular medium ‘the field guide’ (cf. Lars Elleström, in this
volume). Consequently questions concerning the processes of transferring
physical experiences to verbal and visual media should always be the focal
point of the producer and the user of a field guide.
I have come to this intermedial subject by way of ecocriticism.
Ecocriticism is a school of cultural criticism that takes an interest in how
humans’ relation to and interactions with nature are mediated in cultural
artefacts. Field guides are both explicitly and implicitly engaged in this
mediation. From an ecocritical standpoint, Lyon (1995) tries to construe
a taxonomy of nature writing by creating a subject-based spectrum. This
spectrum ranges from descriptive texts to texts dealing mainly with philo-
sophical treatises on humanity’s relation to physical nature. At the end of the
axis of describability sits the field guide, a place it shares with the scientific
paper. The main goal of both genres is to submit, and mediate, information,
and the philosophical, personal or literary traits we find in them are mainly
accidental. Lyon calls this dimension of nature writing ‘natural history infor-
mation’.29 Primarily physical nature is what the artefact is trying to mediate
and guide us towards. Nature becomes the most important, albeit absent, fea-
ture in the different kinds of intermedial relations created by a field guide.
The relation to this mediated medium, nature, is far from uncomplicated, as
Dana Phillips has pointed out in an essay, where ecocriticism and field guides
are discussed. Phillips wants to tell us that a field guide is an open work,
meaning that the user has to divert his or her attention from the guide, and
its combination of media, to the field, the reality in which primary obser-
vation takes place, and then back to the guide in an ongoing process of
signification. In this way, writes Phillips, the user must become a ‘reader’ in
that he or she has to master the techniques of identifying such features as
the birds’ behaviour, their habitat and their different ways of flying. By this
thought experiment, the physical nature where the observer is situated will
be defined as a medium in which birds and other organisms are signs, or can
be combined into signs, and as an effect of this be ‘read’ by the observer.

If that description does not settle the issue of a bird’s identity, then the
birdwatcher must resort to the habitat maps in the back of the book

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Håkan Sandgren 121

(included in its recent editions). ‘Field-marks,’ it should be clear, are not


limited to visual features, but also comprise things like geographic range,
habitat preferences, typical behaviours such as interspecies flocking, and
flight patterns. Such being the case, the birdwatcher must become a
‘reader’.30

The image, the text, the transcriptions and the maps all work together, cre-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ating a unified perspective, but to reach this unifying point, verbal and
visual representation must at the same time be complementary media, for
instance, by focusing on specific and differentiating details in the object.
This urges the artist or illustrator to be utterly aware of ‘the moment of
choice’31 because the importance of positive identification, recognition and
referentiality are crucial aspects of every field guide. The artist’s or the writer’s
‘interpretation’ of the natural object has one main objective, namely to make
naming and identification possible. The kind of artistry that the producer
must strive for is exactitude and faithfulness in the way the natural object
is mediated into the media used, and to reach this objective is truly an
utterly refined form of artistry.32 In this lies a paradox: the mediated image
or description of, for instance, a bird that we would find in a field guide –
no matter how faithful to reality it is – will never replicate the visual data we
conceive as a bird when observing it in the field. First of all the image and
the description found in the guide are merely rough ideas of how a natural
object probably will appear when we find it in front of our eyes. Second,
the field as medium does not allow the kind of scrutiny made possible by
the guide, and its lifeless objects. In the field, the object is ever-moving, sit-
ting in the shade, partly covered or fleeing. Mediated in the medium of the
field guide the object is once and for all given to us. Apart from this, nature
persistently eludes us with its constant flux; what yesterday could be said to
denote ‘the bird species of Europe’ is today pointing to a different part of the
total quantity of bird species, a totality which in itself has to be an approx-
imation. Nevertheless, nature is always there when we use a field guide; it
is the element we have to come to terms with as users of the genre, and as
producers of it. Nature is always present in a field guide, a necessary part
of the way in which the creators and users of the genre combine the media
modes included therein, and as a goal to which our perception is directed
while making use of the guide.

Notes
1. J. Holšánová (2008) Discourse, Vision, and Cognition (Amsterdam: J. Benjamins).
2. W. Wolf and W. Bernhart (eds) (2007) Description: In Literature and Other Media
(Amsterdam: Rodopi).
3. For example, R. T. Peterson, G. Mountfort and P. A. D. Hollom (1985) Europas
fåglar: En fälthandbok (Stockholm: Norstedt), B. King, E. C. Dickinson and M. W.
Woodcock (1991) A Field Guide to the Birds of South-East Asia (London: Collins),

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


122 Combinations and Integrations of Media

L. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern (Stockholm:


Wahlström & Widstrand), E. Rosenberg (1995) Fåglar i Sverige (Stockholm:
Norstedts) and L. Svensson, P. J. Grant, K. Mullarney and D. Zetterström (1999)
Fågelguiden: Europas och Medelhavsområdets fåglar i fält (Stockholm: Albert Bon-
niers förlag).
4. J. Karnicky (2007) ‘Included in this Classification: Encoding American Birds’
(unpublished paper delivered at the conference of the Society for Literature,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Science, and the Arts, Portland, Maine, November 2007).
5. For the birds of Northern Europe, see K. H. Voous (1977) List of Recent Holarctic
Bird Species (London: BOU) and M. Beaman (1994) Palearctic Birds (Stonyhurst:
Harrier).
6. For example, Peterson et al. (1985) Europas fåglar, inside of cover; King et al.
(1991) A Field Guide to the Birds of South-East Asia, p. 219 and R. T. Peterson and
V. M. Peterson (2002) A Field Guide to the Birds of Eastern and Central North America
(Boston: Houghton Mifflin), inside of cover.
7. For example, Peterson and Peterson (2002) A Field Guide to the Birds of Eastern and
Central North America, pp. 8ff.; Jonsson (1992) Fåglar i Europa med Nordafrika och
Mellanöstern, pp. 7, 11, 21.
8. E. Rosenberg (1972) Fåglar i Sverige (Stockholm: Almqvist & Wiksell).
9. A term coined by D. Phillips (1999) ‘Ecocriticism, Literary Theory, and the Truth
of Ecology’, New Literary History 30(3), p. 593. Roger Tory Peterson’s famous guides
set the standard for this way of illustrating birds.
10. Peterson and Peterson (2002) A Field Guide to the Birds of Eastern and Central North
America, p. 73, and Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern,
p. 113.
11. Peterson and Peterson (2002) A Field Guide to the Birds of Eastern and Central
North America, pp. 106, 108, and Jonsson (1992) Fåglar i Europa med Nordafrika
och Mellanöstern, pp. 166f.
12. Peterson et al. (1985) Europas fåglar, and Peterson and Peterson (2002) A Field
Guide to the Birds of Eastern and Central North America.
13. Rosenberg (1972) and (1995) Fåglar i Sverige.
14. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 202.
15. A. Saunders (1951) A Guide to Bird Song: Descriptions and Diagrams of the Songs and
Singing Habits of Land Birds and Selected Species of Shore Birds (New York: Garden
City); L. Lutwack (1994) Birds in Literature (Gainsville FL: University Press of
Florida); D. Rothenberg (2006) Why Birds Sing: A Journey into the Mystery of Bird
Song (New York: Basic Books).
16. Lutwack (1994) Birds in Literature, pp. 1–17 and Rothenberg (2006) Why Birds Sing,
pp. 15 ff.
17. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern.
18. I. Storm (1985) Harald Wiberg: Ett konstnärsliv i naturen (Stockholm: LT) and
B. Svanberg (1991) Harald Wiberg: En konstnär (Umeå: SIH läromedel).
19. C. Curry-Lindahl (1952) Festskrift tillägnad Erik Rosenberg på 50-årsdagen 19 17/8 52
(Stockholm: Svenska naturskyddsföreningen) and C. Thor (2002) Fågelliv betyder
orden: Vandringar i Erik Rosenbergs riken (Örebro: Gullars).
20. The relation between user and writer, and questions concerning the social func-
tion of birdwatching, are studied by M. Cocker (2001) Birders: Tales of a Tribe (New
York: Grove Press) and J. Karnicky (2004) ‘What Is the Red Knot Worth? Valuing
Human/Avian Interaction’, Society & Animals 12, pp. 253–66.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Håkan Sandgren 123

21. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 5, see also
L. Jonsson (2002) Birds and Light: The Art of Lars Jonsson (London: Christopher
Helm).
22. Rosenberg (1995) Fåglar i Sverige, p. 48.
23. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 93.
24. Rosenberg (1995) Fåglar i Sverige, p. 315.
25. Ibid., p. 316.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
26. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 384.
27. Rosenberg (1995) Fåglar i Sverige, p. 349.
28. Jonsson (1992) Fåglar i Europa med Nordafrika och Mellanöstern, p. 434.
29. T. J. Lyon (1995) ‘A Taxonomy of Nature Writing’ in C. Glotfelty and H. Fromm
(eds) The Ecocriticism Reader: Landmarks in Literary Ecology (Athens: University of
Georgia Press), pp. 276–81.
30. Phillips (1999) ‘Ecocriticism, Literary Theory, and the Truth of Ecology’, p. 594.
31. E. Hodnett (1982) Image and Text: Studies in the Illustration of English Literature
(London: Scholar Press).
32. S. Jay Gould gives special attention to this skill in (2000) Wonderful Life: The
Burgess Shale and the Nature of History (London: Vintage), p. 100.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


7
Media on the Edge of Nothingness:
Visual Apostrophes in Lettrism

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Sami Sjöberg

It is commonly recognized that intermedial works of art may mediate


information in ways that individual media are not capable. The interme-
dial work plays with different modes of signification and produces effects
impossible for language alone. Peter Wagner defines intermediality as the
practice of describing one medium by means of another.1 Wagner refers
solely to ekphrasis, although intermediality is a complex phenomenon and
requires that ‘medium’ is defined. Lars Elleström defines ‘medium’ based on
the shared, yet dissimilar, qualities of media as communication.2 A ‘medium’
is a construct born and delimited in the process of definition. Thus inter-
mediality – the engagement of more than one medium – highlights the
constructedness of medial borders and establishes a kind of ‘border zone’.3
Accordingly, media can be mixed in ways that struggle to overcome the
construct nature of ‘intermedia’. Because the practice of applying many
media differs from the scholarly term describing this practice, this essay4
seeks to ‘renegotiate’ intermediality in terms of artworks radically disrupting
the coherence of the work itself. Coherence is always relative and the success
of such disruption debatable, but by realizing these limitations we can study
works that play with intermediality; while by doing so, artworks may also
demonstrate the term’s inadequacy due to its constructedness. This essay
argues that intermedial works play with nothingness and question the valid-
ity of signification in general by showing its limits with obscure techniques,
including invented signs.
The themes of incoherence and nothingness form the base of the œuvres
of the French Lettrist movement. The Lettrists’ interest in nothingness
derives from preceding developments in philosophy and art. In France, the
interwar era had made the theme of nothingness prominent in Sartrean
existentialism as also in the Dadaist works of Tristan Tzara.5 Influenced by
the trend in philosophy and aesthetics, Isidore Isou (Ion-Isidor Goldstein
1925–2007), the Romanian-born founder of Lettrism, published the man-
ifesto of Lettrist poetry (Le Manifeste de la poésie lettriste, 1947), claiming
that words could express none of the individual’s feelings, because words

124

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sami Sjöberg 125

were stereotypes.6 Isou criticized the inadequate means to mediate radically


differing experiences.
There can be no certainty, however, regarding the fundamental similar-
ity or dissimilarity of emotions due to the structure of communication.
Language is based on concepts that are already generalizations, hence
no comparison of the experiential is possible without objectification into

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
language. Nevertheless, Isou’s definition of language as stereotypical com-
munication manifests his profound distrust of this medium.7 He appears to
consider language as inevitably exterior to the individual. Isou’s Manifesto
implies that language sets the frames for the artist’s creative efforts although
it should be the other way around.8 Therefore he focused on what language
could not mediate: nothingness and the incomprehensible.
Furthermore, Isou tests how these motifs could be expressed in writing,9
seeking to subvert the notional language of logos to individual use of anti-
conceptual language. Although Isou’s point of departure is utopian and
highly problematic, his quest for individual expression results in a unique
response to the modern crisis of language: ‘That what for preceding authors
represented the dimension of silence and the unknown will become, thanks to
me, the dimension of the known and of the new kind of speech’.10 Isou con-
siders this ‘new speech’ the means to overcome the limits of language, which
do not correlate with those of experience. However, Isou’s ‘new speech’ is
inevitably in relation to language: what is defined as incomprehensible is
incomprehensible only when one seeks to communicate it through notional
language. Hence writing requires autonomy, distance from notional lan-
guage, and Isou seeks to emphasize the act of writing so that Lettrist writing
approaches writing on canvas.11 However, although he tries to recontextual-
ize writing, Isou does not abandon the book, and hence Lettrist works create
an intermedial tension between the written and the visual.12
By prioritizing the act of writing Isou simultaneously effaces the medi-
ating role of language. For him writing apparently deals with the arousal
of connotations and pre-linguistic affects, toying with the reader’s ‘semi-
otic imagination’, engaging possible interpretations such that the markings
acquire a plenitude of meaning – they can mean anything. This study con-
centrates rather on those qualities in Lettrist writing that seem to precede
and/or resist objectification into language (see Figure 5). The concrete effects
of such resistance are markings that appear meaningless and thus emphasize
the medium itself. Lettrist writing must be treated as a special case of writing
because it functions differently from notional language: its visual elements
emphasize the presence of the text.
Visuality was highlighted from 1953 onwards by hypergraphics (hyper-
graphie), enabling artists to combine all known writing systems without a
particular ‘grammar’ and invent ‘signs’ of their own.13 The invented mark-
ings14 are a method for distancing the act of writing from language. They
are applied to express what defies expression in notional language, in the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


126 Combinations and Integrations of Media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Figure 5 An excerpt of Alain Satié’s ‘Sur le pont d’Avignon’. Courtesy of the artist

sense of being supposedly outside the production and application of con-


cepts. In this light, invented markings are not based on ciphers and are
at best quasi-iconic with no recognizable resemblance or convention to
connect them with pre-linguistic experiences. The invented markings thus
highlight the individual writer’s ‘presence’ in Lettrist writing, producing a
tension when text is combined with invented markings. In such cases, Let-
trist writing fragments conventional text by means of another medium. I call
these interventions visual apostrophes. Besides indicating a missing letter or a
word in the text, the original literal meaning of the word apostrophe (Greek:
apostrophos ‘turning away’) is also preserved. In conventional text, the visual
apostrophe represents a turn away from notional language that makes room
for another medium. However, the visual apostrophe maintains its relation
to language, because often the apostrophe is linguistically motivated: the
apostrophe can replace a word or a letter and can hence be regarded as
intermedial.
Furthermore, there is a third, subtler meaning of apostrophe. The Greeks
used it to refer to an orator’s ‘turning aside’ to address some individual.15
In this context, the hypergraphics address the individual reader by show-
ing him or her the limits of notional language, thereby leading him or her
to reconstruct the meaning of invented markings. I suggest that this mean-
ing of the apostrophe requires a departure from the intermedial approach,
although the aforementioned definitions are necessary steps towards the
other approach, meontologization or the dynamics of signification and noth-
ingness, which is studied in the third part of this essay.

The restrictions of the intermedial

Critical inquiry may not always benefit the complexity of an artwork, espe-
cially in the case of intentional ambiguity. This is because intermediality

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sami Sjöberg 127

makes certain fundamental ontological assumptions affecting the depiction


of a work. One such assumption is that in the work there are at least two
media differing in their ways of signification. For instance, verbal and visual
signs differ in their mode of signification. In Lettrism, intermediality refers
to the interaction between the two distinct media, namely the written and
the pictorial. Hence, in order to approach such difficult cases of interme-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
diality as that of hypergraphics , the work must be subjected to a certain
deontologization.
Deontologization refers to the lowering or eventual removal of media
borders, yet, as W. J. T. Mitchell argues, ‘all media are mixed media, and
all representations are heterogeneous’.16 Mitchell means that writing too is
visual and material. In addition, no modern pictorial work can avoid the
text.17 Consequently, every intermedial work is subject to some degree of
deontologization. In the context of this essay, however, deontologization
is insufficient because Lettrist writing is not based on shared unproblem-
atic representations. Hypergraphics suggests a manifold relation to language.
On the one hand, hypergraphics is a form of visual expression, but visual
only in the limits imposed by written language. Invented markings have
the semblance of signs, because they are black and white and appear in
contexts where written signification is found (see Figure 5). In short, they
emulate the conventions of writing. On the other hand, invented mark-
ings fragment the text with their apparent lack of meaning. Once grasped
in the context of language, the lack of linguistic signification is supple-
mented by invented markings apparently concealing their meaning. This
supplementation engages the reader’s semiotic imagination. Yet the materi-
ality of hypergraphics arouses a feeling of the presence of the text and of a
presence ‘behind’ the text: someone in particular wrote these. Therefore, the
more notional language fails, the more the presence of the unobjectified is
pronounced.
Such presence manifests in the following visual apostrophe by Isou:

The individual wanted to see the surfaces it spoke of and


calculate them visually.18

Although the particular apostrophe is not as advanced as those applying


hypergraphics, it nevertheless emphasizes the co-presence of the visual and
the written in the script.19 No linguistic meaning is lost by the introduc-
tion of the black rectangle. The fragment is intermedial – in the Wagnerian
sense – in that the text emphasizes seeing and materiality through ekphra-
sis. The notional language discusses the visual element before the rectangle
is introduced. In addition, the fragment also works as an iconotext. Wag-
ner writes: ‘Iconotext refers to an artefact in which the verbal and the visual
signs mingle to produce rhetoric that depends on the co-presence of words
and images’.20 Thus the rectangle is included in the sentence as a visual sign,
which acquires that quality through the text. Yet its mode of signification is

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


128 Combinations and Integrations of Media

ambivalent. The rectangle may be either a mere self-referential presentation


as in both cases above or it may be depicted as representation. In the latter
case, an interpretation based on iconotextuality recognizes the black rect-
angle as comparable to note clusters in early twentieth-century modernist
music.21 As such, the rectangle is a ‘condensation’ of every possible sign
superimposed on each and obliterating individual markings.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
These different modes of signification are not contradictory. The nature
of the visual apostrophe is such that it presents a pause or a cut in the
text, it shuns one medium in order to make way for another, but it may
leave a trace of what is removed from the first. For example, consider
replacing the rectangle with an adverb, such as ‘automatically’ (automa-
tiquement). This supplementation is not linguistically necessary but possible
and exemplifies how easily the rectangle is interpreted as something other
than itself. Therefore, the lack of language or the traces of it are ambiguous
in visual apostrophes and the ambiguity arouses the interpreter’s semiotic
imagination.
The possibility of replacing the absent signified is insufficiently described
by any deontologization of intermediality, because deontologization ignores
the potential signification. The co-presence of different ways of signification
occurs because coherent sign categories are based on certain distinct ways
of signification. Deontologization explores both the reciprocal differences
between sign categories (that is, verbal, visual) and their co-presence, but
excludes the lack of a clearly defined signified.
Hypergraphic writing entails that the lack of signified not only prevents
the definition of an unambiguous semiotic category, but this may also be
deemed an intrusion of ‘pre-linguistic elements’ into notional language. This
is to say that hypergraphics explores the possibility of ‘notionless’ writing.
However, such a formulation is clearly paradoxical, because writing is by def-
inition the mediation of ideas through concepts. Hypergraphics plays with
this key idea in communication and deliberately engages the interpreter to
seek meaning through connotations and associations, which the work nei-
ther affirms nor negates. This strategy of ambiguous signification suggests
that the loosely definable connection of hypergraphics to existing sign cate-
gories should be analysed as such instead of cataloguing the various possible
interpretations. Hence I coin a new term, meontologization.22 It refers to the
study of what remains outside notional language; to what has not yet been
objectified or is beyond objectification. The perception of such a lack of
concepts as nothingness is practical, because nothingness cannot be made
into an object of thought. This is to say that nothingness does not lend
itself to semiotic imagination, and as a counterweight to interpretation, it
shapes the coherence of the artwork. As deontologization refers to dimin-
ishing the limits of what is present (signified), meontologization focuses
on the disappearance of distinct limits between what in representation is
present and absent. In other words, instead of the iconotextual co-presence

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sami Sjöberg 129

of the verbal and the visual, meontologization is the ‘cooperation’ of pres-


ence and absence manifest in verbal and visual signs. By meontologization
hypergraphics is understood as invented markings that are notionless, but
have the potential to change. Meontologization prioritizes the actual coher-
ence of the work and its possible disruptions instead of the hermeneutical
assumption of its unity, which may lead to hasty interpretations.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
How to do nothing with words

Although hypergraphics supposedly refers to what falls outside notional


language, Isou’s theory is in many ways problematic. Due to his utopian
attitude he sought to make the incomprehensible palpable by ‘writing noth-
ingness’ (écrire le rien).23 Isou was undoubtedly referring to experiences
beyond the limits of communication, yet he did not discuss the obvious
problem of mediating something unobjectified. The existence of unmedi-
ated experiences is debatable, because if we experience something we do
not simultaneously conceptually reflect it.24 Although the possible existence
of such experiences is not denied, their mediation is certainly impossible,
because it would require the objectification of the very experiences one seeks
not to objectify. Mediation necessitates objectification, because the experi-
ences cannot be present as such. In addition, the claim to ‘write nothingness’
is obviously paradoxical; how can material production, such as writing, pro-
duce nothingness? Undoubtedly these are problematical issues, which Isou
ignores. However, intermediality or iconotextuality is a characteristic result
of Isou’s search to express nothingness. When the signification of invented
markings or any kind of signs is effaced, their visuality is emphasized. The
reader then seeks to make hypergraphics meaningful as presentations – as
abstract forms – or to interpret them as representations.
Due to the ambiguity of signification in Lettrist writing, Heidegger’s view
of nothingness corresponds to visual apostrophes. He stated that noth-
ingness can be surmised but not logically proven.25 That is, nothingness
responds poorly to analytical thinking based on notional language. Hyper-
graphics is writing that defies interpretation and this resistance is both
intentional and significant to the coherence of the work. In this case, hyper-
graphics is interpreted as an effort to mediate experiences that cannot be
made into objects of thought, the challenge being not to interpret nothing-
ness as something else. Arguably the visual apostrophe can be approached
as nothingness. Its structure proves that what is taken away from represen-
tation is also a representation – as a trace of representation. Representation
is the only way to approach nothingness, because nothingness cannot be
presented. Meontologization further allows depiction of the trace of repre-
sentation, the absence of a given signified, as an intentional effect in the
work. In visual apostrophes, meontologization may indicate an absence or
predict the anxiety about nothingness.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


130 Combinations and Integrations of Media

Alain Satié’s work, ‘Sur le pont d’Avignon’ (On the bridge of Avignon)
(1970, see Figure 5), exemplifies these functions.26 There are three discernible
levels of writing and meontologization elaborates their interaction. Firstly,
the typewritten text forms the common level open to all. Secondly, the
handwritten headline forms the individual level, because although written
in French, another person cannot reproduce Satié’s handwriting. Thirdly,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the private level is formed by invented markings that disrupt sentences and
even individual words. In this work hypergraphics is not intended to widen
signification but to underline the shortcomings of notional language.
However, the work sporadically applies methods pointing to the absence
of a particular element. For example, the fragment between the fourth and
fifth lines (compa[ ]ables) suggests that the invented markings can be replaced
only by the letter ‘r’, thus forming a readable word in French, compara-
bles (comparable). The length of the ‘imaginary cartouche’ suggests a longer
replacement than ‘r’ due to the sign-like nature of the fragment, which the
reader identifies with language.27
As the ‘r’ seems to be the only logical replacement, in this case the hyper-
graphics leads us to search for an absent particularity. Such absence depends
on our expectations of finding something that is not there.28 Therefore the
nothingness in comparables is inherently particular, meaning that there is
a trace of something particular that is absent. In some visual apostrophes
the lack of language refers to absence, as in the example above. Moreover,
invented markings resemble representations because the reader replaces the
fragment with what it seems to stand for. The text around the visual frag-
ment is motivated by the hypergraphics charged with semantic content. In
addition to the meontological cooperation of presence and absence, the co-
presence of words and images is essential. Hence, these examples may be
called iconotextual hypergraphics. The relation to other objects is established
and the hypergraphic fragment becomes a quasi-linguistic sign. Thus visual
apostrophes utilize the particularity of absence: the interaction between the
common and the private levels is necessary, because individually the private
level structures only an unfinished signification. Here something present
mediates the absence of some particular, but what is missing can still be
depicted.
The reconstruction of other parts of the composition is not equally
uncomplicated, however. In places a whole word or multiple words are miss-
ing. For example, the invented markings on the second and third line in
Satié’s work suggest that the visual apostrophe either replaces multiple words
or none at all. However, in the case of multiple missing words, the inter-
pretation is more ambiguous than the previous one. Hence no univocal
supplementation but only indeterminate possibilities can be found.
What if there is no appropriate unambiguous particular replacement?
In the case of such profound indeterminacy, invented markings would
not represent the absence of something in particular, but a more general
nothingness. Here, Heidegger’s notion of nothingness has a structural

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sami Sjöberg 131

homology with the visual apostrophe. According to him, beyond the limits
of a whole there ‘is’ nothingness.29 Heidegger understood beings and things
as finite, and therefore nothingness draws their very limits. Nothingness
constitutes being and this constitution is known in anxiety. The anxiety
derives from the limitedness of our existence and our awareness thereof.
Therefore nothingness emerges as ‘a slipping away of the whole’.30

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
The slippage undermines the coherence of the finite (an individual, an
artwork). The hypergraphics evokes nothingness by showing the reader the
limits of his or her being. The implied existence of personal experiences is
revealed by invented markings, but the experiences remain radically other.
According to this approach, invented markings are a vestige of the presence
of the unknown other (writer, meaning) within the text. Moreover, visual
apostrophes are not mere changes in media or modes of signification, but
breaks in the coherence of the text revealing the absence of the writer and
exposing the reader to the lack of language and meaning. The slippage of the
whole results once the limits of being are foregrounded and we grasp that
there is something beyond our field of knowledge.
By effacing meaning, though, Lettrist writing undermines the capability
of language to correspond with the world it describes. Satié’s work does not
radically question the limits of being but those of notional language. Hence,
the work’s ‘meaningless’ visual apostrophes in fact criticize the assumption
that the individual and experiential could be fully communicated through
language. Such examples of visual apostrophes may be called notionless
hypergraphics. In this case, the hypergraphics ‘deconstructs’ the co-presence
essential to the iconotext because it establishes no coherent sign category.31
Therefore, in notionless hypergraphics the verbal and visual signs do not
mingle, but proclaim their autonomy.
Iconotextual hypergraphics is subordinate to notional language because
it replaces a grammatical sign (a letter, a word) in the text. However, if
hypergraphics does not acquire equal grammatical motivation, the hierar-
chical relation between notional language and hypergraphics is the opposite.
Notionless hypergraphics cuts the text, opening it to the reader’s semiotic
imagination. The reader is called to fill in the blank, which means that
through interpretations alone the blank can potentially become something
other than nothingness. Furthermore, the lack of meaning in the artwork
is in itself meaningful, thanks to its intentionality. Although hypergraphics
challenges the capabilities of writing to serve as a medium and communi-
cate by mediating information, meontologization highlights hypergraphics
as a medium that mediates the ‘presence’ of the unobjectified in the artwork.
This unobjectified, however, can only be detected through the dynamics of
meontologization, which is present in the visual apostrophe.
Fundamentally, Satié’s work is fragmentary if the aspects of absence and
nothingness are not taken into account and if iconotextual and notionless
hypergraphics are not regarded as separate. The title refers to an impassable
bridge. Arguably in Satié’s piece language also forms a bridge that enables

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


132 Combinations and Integrations of Media

one to cross to the other bank at the end of the text, yet invented mark-
ings subject the bridge of language to constant disruption. To ensure a safe
crossing, the actual bridge requires its missing pieces and likewise, meontol-
ogization supplements the bridge of language by showing what is not there.

Conclusion

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Lettrist visual apostrophes demonstrate the absence and potential of noth-
ingness by showing us our limits. By shunning language with invented
markings, there is a feeling of concealed expression. This concealment causes
the unfinished signification to emphasize the materiality of hypergraphics.
The pure materiality of writing results in what Isou called ‘hidden signifi-
cation’ (signification cachée).32 This hiddenness not only highlights absence,
but also reveals the limits of being and, henceforth, arouses the anxiety of
things eluding us.
The meontologization of Lettrist writing causes an effect of deficiency, as
hypergraphics points out the limits of notional language. Such is the ulti-
mate contribution of hypergraphics, although it disregards Isou’s utopian
claims. We must realize that language exists because the things it denotes
are not present. This practical view assumes that language enables us to dis-
cuss concrete absent things or abstractions that were never there. According
to this idea, hypergraphics reveals the fundamental emptiness of representa-
tions in language by invented markings and unfinished significations, which
highlight the unattainability of the intended meaning. In other words,
although the intended meaning is always inaccessible, hypergraphics fore-
grounds the separate realities of individuals instead of producing a fictive
one. Thus the empty representations evoke an anxiety of different, yet simul-
taneous, realities where the slipping away of the whole becomes evident.
However, hypergraphics implies another result of meontologization:
invented markings merely propose their own meaninglessness without
affirming it. Accordingly, it may be concluded that unfinished signification
is the very possibility to represent. From the perspective of notional language
hypergraphics signals that something is taken away – from representation –
in which case the possibility to represent is simultaneously a vestige. As
such, the pronounced presence of hypergraphics brings about the possibil-
ity of experiences that cannot be made into objects of thought. Therefore,
to neglect the meontologization in artworks is to neglect the philosophical
power of works of art.

Notes
1. P. Wagner (1996) Icons, Texts, Iconotexts: Essays on Ekphrasis and Intermediality
(Berlin and New York: Walter de Gruyter), pp. 10–11, 17.
2. Cf. Elleström and Jørgen Bruhn, in this volume.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Sami Sjöberg 133

3. Cf. Rajewsky and Sándor, in this volume.


4. This essay is a part of the project Literature, Transcendence, Avant-Garde, funded by
the Academy of Finland (1121211).
5. Although Isou never produced a coherent theory of nothingness, he drew on
the prevailing philosophical accounts of nothingness. See I. Isou (2000) Amos ou
introduction à la métagraphologie (first published 1953) (Marseille: la Termitière),
p. 26.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
6. I. Isou (1947) Introduction à une nouvelle poésie et à une nouvelle musique (Paris:
Gallimard), pp. 11–12.
7. Isou criticized media as means of communication, but the concept ‘medium’ was
understood rather conventionally in Lettrism.
8. See Isou (1947) Introduction à une nouvelle poésie et à une nouvelle musique, pp.
11–12.
9. I. Isou (2000) La dictature lettriste: Cahiers d’un nouveau régime artistique (first
published 1946) (Paris: Cahiers de l’externite), p. 16.
10. ‘Ce qui pour des auteurs antérieurs représentait la dimension du silence et de
l’inconnu deviendra, grâce à moi, la dimension du connu [et] de la parole nova-
trice’. I. Isou (2003) La créatique ou la novatique, 1941–1976 (Romainville: Éditions
Al Dante), p. 23.
11. See Isou (2000) Amos ou introduction à la métagraphologie, p. 26.
12. The genre of the work is also bound to the technical requirements of a given
medium (cf. Rajewsky, in this volume).
13. R. Sabatier (1989) Le lettrisme, les creations et les createurs (Nice: Z’éditions), pp.
27–8.
14. I call ‘signs’ invented by the writer ‘invented markings’ to avoid confusing their
sign-like nature with actual signs. ‘Hypergraphics’ refers to the combination of
every kind of sign system and therefore I rather apply ‘invented markings’, which
also emphasizes the act of mark-making.
15. J. D. Kneale (1991) ‘Romantic Aversions: Apostrophe Reconsidered’, English
Literary History 58, pp. 141–65.
16. W. J. T. Mitchell (1995) Picture Theory: Essays on Verbal and Visual Representation
(Chicago and London: The University of Chicago Press), p. 5.
17. Cf. Jørgen Bruhn, in this volume.
18. ‘L’individu voulait voir les surfaces dont il parlait . . . et les calculer visuellement’:
Isou (2000) Amos ou introduction à la métagraphologie, p. 20.
19. Isou’s apostrophe had numerous antecedents, such as Man Ray’s anonymous
poem (1924), consisting of black lines that form the poem, and a book by Robert
Carlton Brown (1886–1959), Gems: A Censored Anthology (1931). See J. Rasula and
S. McCaffery (2001) Imagining Language: An Anthology (Cambridge MA: MIT Press),
pp. 201, 288.
20. Wagner (1996) Icons, Texts, Iconotexts, p. 16. Wagner’s definition of iconotext is
congruent with some approaches to intermediality introduced in this volume (cf.
Jørgen Bruhn). However, I retain iconotext in order to distinguish between the
numerous intermedial ways in which hypergraphics criticizes language.
21. Henry Cowell pioneered the technique in The Tides of Manaunaun (1917). In
France, especially Olivier Messiaen applied note clusters in the early 1940s.
22. Meontology, from Greek to mē on ‘nonbeing’.
23. Isou (2000) La dictature lettriste, p. 16. Isou may have absorbed this rhetoric
from negative theology, which also struggled with the mediation of inexpressible
experiences. In addition, linguistic negation and absurd texts diffuse ambiguous

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


134 Combinations and Integrations of Media

uses of language. However, in nonsense literature and negative theology, for


instance, distrust of language does not manifest as radically as in invented mark-
ings. Here I treat as nonsense and negative theology such texts that rely on
conventional language although invented markings can be regarded as non-
sensical and were also utilized in different branches of mysticism. However, it
is difficult, if not impossible, to depict the genre of texts that apply invented
markings.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
24. Cf. R. K. C. Forman (1997) The Problem of Pure Consciousness: Mysticism and
Philosophy (Oxford: Oxford University Press).
25. See M. Heidegger (1993) Basic Writings (London: Routledge).
26. The title has a double reference to the real world. The bridge of Avignon is a
historical site and ‘Sur le pont d’Avignon’ is a traditional French song.
27. This is a case of confusion between ‘visual’ and ‘iconic’ (cf. Elleström, in this
volume).
28. See J.-P. Sartre (1943) L’être et le néant: Essai d’ontologie phenomenologique (Paris:
Gallimard), pp. 42–4.
29. The ‘whole’ refers to all entities whose meaning to us constitutes our world. Cf.
Heidegger (1993) Basic Writings.
30. Ibid., p. 102.
31. For a similar non-totalizing strategy cf. Sándor, in this volume.
32. Isou (1947) Introduction à une nouvelle poésie et à une nouvelle musique, p. 18.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Part IV
Mediations and Transformations
of Media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
This page intentionally left blank

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


8
Penrose, ‘Seeing is Believing’:
Intentionality, Mediation and
Comprehension in the Arts

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Siglind Bruhn

In 1934, the Swedish artist Oscar Reutersvärd created, with an uncommon


arrangement of cubes, what has become known as an ‘impossible triangle’.
In 1980, the Swedish government honoured the inventive artist by commis-
sioning photographs of three of his works for Swedish stamps. The stamps
were produced in 1982 and issued for about two years.
Twenty years after his Swedish counterpart, in 1954, the British mathe-
matical physicist Roger Penrose (born 1931), after attending a lecture by the
Dutch graphic artist M. C. Escher, rediscovered the impossible triangle, drew
it in what is today its most familiar form, and described it in a 1958 article
published in the British Journal of Psychology. Penrose’s impossible triangle,
unlike Reutersvärd’s earlier version, was drawn in perspective, which added
a further paradox to the figure.
Penrose described the ‘tribar’ as ‘impossibility in its purest form’. It does
indeed contain various dimensions of impossibility: its three straight beams
meet at right angles at the vertices of the triangle they form, and the three
surfaces of each beam are also at 90 degree angles to one another. In the
context of intermediality the object seems thought-provoking for its strictly
non-hierarchical nature. No face of any of the three arms can be said to be
privileged in that it is ‘always front’ or ‘tracing the inner circle’. Instead, each
of them constantly changes its spatial designation and its apparent visual
reality, as so many surfaces in M. C. Escher’s drawings typically do. This
essay, then, is a meditation on various art forms, media and modalities –
thoughts that are inspired by this and other triangles. In particular, I hope
to incite discussion of a number of questions concerning the angles linking
the three beams in another strange triangle – what is often referred to as
the ‘artistic triangle’. I will end by analysing and interpreting the three-step
intermedial transposition of a work created by an uncle of the designer of
the tribar, the British Surrealist painter Roland Penrose.1

137

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


138 Mediations and Transformations of Media

Various artistic triangles: options at the creator’s angle

Some time ago I undertook an in-depth study of an opera by the Ameri-


can composer, Philip Glass. The opera is entitled Satyagraha. In terms of the
dramatic action developed on stage, it deals with the early life of Mahatma
Gandhi and his development of the movement of non-violent resistance in
South Africa. The lyrics, however, are entirely taken from the ancient Indian

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
poem, Bhagavad Gita. Thus my task, before even commencing the analysis of
Glass’s music, was to understand how the ancient poetic dialogue between
a divine avatar and the warrior relates to an early twentieth-century story of
racial discrimination, and how particular verses from the Gita may inform
and reflect this particular instance from Gandhi’s struggle for human dignity
under a white supremacist government.
In the middle of my several weeks of reading up on the Bhagavad Gita, and
entirely unrelated to my work on Glass, I was invited to participate in a panel
discussion on the topic of the relation between the Trinity and the ‘artistic
triangle’, which is understood to consist of (a) artistic creator, (b) performer
or mediator and (c) audience or beholders. While I knew, of course, which
Trinity was being referred to, I found myself inspired to think of the question
in terms of the Hindu trinity. This in turn led me to a different kind of
artistic triangle, which I have since come to consider a foil in an impossible
dimension of the well-rehearsed one, linking (a) art as an offering with (b) art
that is passionately involved in the world and (c) art that ‘constructively
destroys’ or dethrones.
The first deity in the Hindu trinity is Brahma, the Creator. Brahma cre-
ates the world as a kind of offering. Having created it, Brahma is no longer
involved in the world, neither imposing rules nor fixing problems, neither
punishing nor rewarding. The second deity in the Hindu trinity is Vishnu,
the Preserver. Vishnu is supremely concerned with the world, incarnating
again and again to help remedy what human beings invariably mess up.
(Krishna, one of the protagonists in the Bhagavad Gita, and Rama are perhaps
the best known incarnations or avatars of Vishnu.) Finally there is Shiva, the
Constructive Destroyer. Shiva destroys so that new things may arise; it is said
that Shiva ‘dances the world to pieces’. Shiva is also very much interested in
abstractions and in philosophical conceptualizations.
The artistic triangle that emerged in the context of my reflections on
the Hindu trinity may be seen as intersecting with the more familiar tri-
angle of artistic creator, performer or mediator and audience. One corner
of my alternative artistic triangle would be taken up by art and music that
serve as an offering; Byzantine icons and religious mosaics come to mind
just as much as hymns whose composers have remained anonymous. This
kind of artistic attitude relates to Brahma. A second corner of my triangle
would be occupied by art and music that is passionately and compassion-
ately involved with the human world. This includes art exploring human

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Siglind Bruhn 139

bodies (such as Michelangelo’s Vitruvian Man) and art representing the world
in which humans live – from the interiors of Vermeer and other Flemish
artists to Romantic representations of small man in overwhelming nature,
music that incorporates human dances, folksongs and the like, all the way to
the nineteenth-century dramas and novels. This attitude relates to Vishnu.
Finally, there is the artistic attitude that ‘constructively destroys’: art that

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
dethrones the idea of representation, music that dethrones overtone-based
harmonies, literature that dethrones the idea of a logically developed plot
or a fully impersonated protagonist with whom readers can identify. This
art corresponds to Shiva in that it may become increasingly concerned with
conceptualizations, either by turning to abstract modes of communication
or by engaging irony, as Lars Elleström has shown so persuasively in his book
Divine Madness.2
The beauty of this triangle is that just as the three corners are not unre-
lated points in space but linked as the three sides of a strictly contained
geometric figure, the three attitudes are intricately interrelated. A work of
abstract music may well be created in the spirit of an offering, as may be a
representational still-life and so on, but then, Brahma, Vishnu and Shiva are
really three aspects of a godhead that is essentially One: Trimurti.

The limits of perception: obstacles at the beholder’s angle

With regard to what is more commonly called the artistic triangle, one ques-
tion has haunted me for a long time: how and to what extent does it matter
whether or not the audience can see, hear or in any other way grasp details,
structures and intentions, or whether these only become apparent with the
aid of notation or analysis? In a masterpiece of literature, music or plastic
art, a first exposure will typically only reveal a general impression and a frag-
ment of the extant details. Renewed exposure and atypical approaches can
and usually will take the beholder’s appreciation to ever deeper levels. This
includes reading a work of literature not consecutively but selectively or by
comparing disjunct passages, looking at the score of a musical composition
rather than only listening to the sounding realization, voicing the thoughts
that possibly inform the visual images in a painting and perhaps extrapolat-
ing from them into realms beyond the visually represented. There is thus a
development on the beholder’s side from perception to an understanding of
the deeper signification and possibly on to a sense of a work’s transcendent
meaning. From a first glimpse at this third corner, considerations will return
to the level of perception and retrace the triangle a number of times, in the
manner that Gadamer described for the hermeneutic circle.
If one were to arrange artworks along a spectrum that spans from those to
which access is immediate all the way to those that appear exceedingly com-
plex, veiled and layered, one would find oneself confronted with extreme
questions. At the one end there are creations, presumably in any medium

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


140 Mediations and Transformations of Media

and modality, which may reveal no more at prolonged or repeated expo-


sure than at the very first glance. At the other end, there are works whose
allusions and significations can only be ‘understood’ by specialists. Works
belonging to the one extreme may collapse the artistic triangle to a two-
dimensional relationship, as there may be no need or justification for any
‘mediation’, and where a musical composition still requires a performer in

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
order to be heard, this unfortunate go-between may seem flattened to a pres-
ence that is, psychologically and spiritually, paper-thin. At the latter end,
one may wonder where creative artists on the one hand and audiences on
the other draw the line between what can be understood and what needs to
be understood?

Case study 1: sensually or intellectually inaccessible


spiritual signifiers

I would like to give two examples from my own field, music. In both cases,
neither the signifier nor the signified are utterly mysterious for anybody,
even those who know almost nothing about music. One of the phenomena
is impossible to grasp without conscious knowledge, the other is only ever
felt subconsciously.
In 1991, the British composer John Tavener (born 1944) composed an
opera about a fifth-century desert hermit revered as St Mary of Egypt.3 The
work bears the subtitle ‘an ikon (sic) in music and dance’, thus alerting us to
the fact that it combines various art forms. It involves not only music, drama
and stage design as all operas typically do, but also includes dance, as some
works did since the beginnings of the genre in the early seventeenth cen-
tury. Moreover, this opera makes reference to a kind of visual representation
that has no connection to music-drama: the icon. Tavener, a member of the
Orthodox Church since 1977, and his librettist, Mother Thekla, the abbess of
an Orthodox convent, had many levels in mind when they decided to use
an icon as a frame for their representation of an ancient penitent. On the
literal level, they envisage a three-panelled icon on stage. At the beginning
of the performance, the outer panels are closed in on the central panel (see
Figure 6).

Mary Mary and Zosíma Zosíma

the penitent meeting in the the righteous


whore desert monk

(acts I + II) (acts III–V) (acts I + II)

Figure 6 The ‘musical icon’ of John Tavener’s Mary of Egypt

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Siglind Bruhn 141

When the two wings open a few minutes into Act I, the space inside them
is occupied by the two main characters: on the left, Mary who gives herself
up to the pleasures of a sensuous life in Alexandria, and on the right, a
devout monk in his monastery in Palestine who will eventually learn from
her that there is a devotion greater and more God-pleasing than his self-
righteous piety. The separate, parallel arrangement in the triptych’s two

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
opened side panels is maintained throughout the first half of the opera. The
work’s second half then takes place in the central panel, which represents the
desert where the penitent’s and the monk’s worlds intersect. Shortly before
the end of the final act, the icon panels close again, presumably allowing us
to comprehend that while the particular story is completed, the larger one
(and the music, for a little while) continues.
When Tavener refers to his work as a ‘living icon’, he has more in mind
than simply the visual aspect. The music is as stylized as the stage setting and
acting, described as using ‘primary colors as the icon painters do’.4 Tavener’s
‘musical icon’ presents a fascinating compendium of musical symbols. The
entire composition is crafted from a limited catalogue of ten components,
distinguished by tonal arrangement, metric organization, melodic features
and timbral realization, with very little overlap between one group and the
other. Of the ten musical elements, five are introduced ‘before the curtain
rises’ as it were, that is, before the icon’s side panels on stage swing open and
reveal the protagonists. These five musical elements must thus be compre-
hended as signifiers of an eternal, never-changing, divinely inspired universe
that exists before, during and after the particular human story. Once the
stage scenery has become visible, they are complemented by five thematic
components that embody the players in the dramatic action.
The first musical symbol that listeners consciously perceive while the stage
still suggests – to those who have eyes to see – the extra-dramatic world
of the closed icon, is a sequence of chiming sounds created by hand bells
struck with soft sticks. The performance indication requests that they ring
‘with unearthly stillness’ and allows freely chosen distances of 1–2 seconds
between the individual strokes. The sequence, encompassing 25 notes, is
built as a palindrome, that is, in perfect horizontal symmetry (see Figure 7).
Musical palindromes are interesting above all for their spiritual signifi-
cance. In the realm of human experience, the irreversibility that defines
all acts, be they physical or linguistic, the irreversibility that defines the
course of a day or life and the expected execution of a plan, are of a quality

Figure 7 John Tavener, Mary of Egypt: schematic representation of the hand bells of
‘unearthly stillness’

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


142 Mediations and Transformations of Media

intrinsically different from reminiscences, regrets, nostalgia and other acts


or feelings turned toward the past. Imagining a point where this distinction
no longer applies means leaving the realm of time as humans know it –
that time which, together with space, provides the coordinates for life in
our universe. On this ground, musical palindromes offer themselves as sig-
nifiers of timelessness, of the annihilation of time or, in a religious context,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of eternity. The device is also associated with connotations of perfection, an
essential quality of the Divine.
If the effect of symmetrical designs in works of visual art is one of balance
and aesthetic pleasure, the result of similar mirroring processes in art forms
that evolve in time is quite different. Whenever the segment that will turn
into its own retrograde comprises more than five or seven – at most, nine –
notes, the resulting symmetry is beyond the grasp of human listeners. In
other words: the distinguishing attribute of this melodic line has a profound
spiritual meaning, which however lies outside the perceptive faculties of all
human beholders. Although each note, and the succession, is easy to hear,
the line’s overall shape, and with it its transcendent message, are not. (Spiri-
tually speaking, it is of course utterly appropriate that we should not be able
to perceive what is by definition outside of human experience.)
The opera’s very first signifier escapes listeners for a different reason. It is
in fact the most pervasive sound phenomenon in the composition: a pro-
tracted drone on the pitch F. This sound seems to emerge imperceptibly
from the silence preceding the composition and to vanish equally gently
back into this silence. In the music of the Byzantine rite, such a drone is
used to accompany liturgical singing.5 With regard to the spiritual message
conveyed in this opera, it stands for the eternal presence of God, imper-
turbable no matter what happens. With regard to Tavener’s ‘musical icon’,
it is analogous to the golden background that surrounds painted images in
the icons of the Orthodox rite. Just as the faithful praying in front of a visual
icon will focus on the stylized features of the saint and largely neglect the
backdrop, so also in this music: the drone is so unimposing that it seems to
fade out of the hearing range of listeners. It is only interrupted when, on
three successive occasions, Mary threatens to become entirely disconnected
from God. In these instances, even lay listeners who claim to have had no
conscious awareness that there ever was such a backdrop sound comment
that ‘something intangible has changed’.

Interpretation or digression: risks and freedoms


at the mediator’s angle

In the two examples discussed above and in many others, performers and
conductors can do nothing to facilitate the audience’s access to the signi-
fication that is being offered. In other cases, interpretation makes all the
difference, and it therefore matters what and how much the interpreter

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Siglind Bruhn 143

understands. In particular, genres involving a multiplicity of media raise


the question of the role of choreographers and opera directors. Many
an operatic staging captures the audience so powerfully through visual
images – especially where video projections are added to real-time actions
on stage – that the music is relegated to the role of a cinematographic sound
track: a sonic tapestry that is meant to highlight the emotional qualities of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
a story presumably told primarily through verbal and visual means. While
that may be fine in the case of works that are composed with just this effect
in mind, one wonders whether it is equally appropriate in the case of operas
whose music speaks a language rich in hermeneutic baggage? Does not an
approach privileging visual effects thwart the audience’s potential access to
the musically conveyed messages?
A related question concerns choreographers wedded to the late-twentieth-
century concept of ‘non-redundancy’ and therefore striving for, as one of
them explained to me, ‘a dance that dialectically engages rather than dupli-
cates the music’.6 After the first thrill of this ‘dialectic engagement’ wears
off, such choreographers often receive feedback telling them that the dis-
juncture they create is such that spectators can only take in the dance if
they tune out the music as one would the muzak in a restaurant during an
interesting conversation with a friend.
Reflections on the immediate versus the successively revealed disclosure
of a work’s message inevitably lead to the role of the mediator. If the
performance of a work, while no doubt a creative act, is meant as an
‘interpretation’ that facilitates rather than veils the beholders’ access to the
conveyed signification, then musicians, actors and dancers, would ideally
invite their audiences to respond to the primary artwork’s content, rather
than to their skills. This raises a question regarding a medium that seldom
speaks through interpreters: plastic art. Is the fact that visual artists do not
(normally) rely on, much less depend on, interpreting mediators, only an
advantage or can it also sometimes foreclose avenues of access? How do
artists feel about the (still fairly unusual, always secondary, but often most
interesting) interpretation of a work of theirs in poetry, music or dance?
And this is only one of the many ways in which a primary representation is
re-presented in another medium?
In the final component of my essay, I will briefly discuss a musical piece
based on a poem that responds to a painting, in other words: a song whose
lyrics are an ekphrasis and whose music adds a novel perspective to what
James Heffernan has succinctly called ‘a verbal representation of visual
representation’.7

Case study 2: musical signifiers enriching an ekphrastic poem

In 1937, the English painter Roland Penrose, who had worked in the
Parisian Surrealist circles for many years and introduced Surrealism to the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


144 Mediations and Transformations of Media

British Isles, completed a canvas alternatively entitled Voir c’est croire (See-
ing is Believing) or L’Île invisible (The Invisible Isle).8 The painting shows
simultaneity of night and day. At the top, a dark sky with a new moon,
a few stars and white clouds seem to bleed out into the shoulders of a
young woman whose head hangs upside-down, covering much of the cen-
tral section, which is bathed in sunlight. Her wavy blond hair spills into

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the ocean below her and forms the backdrop and visual enclosure for a
small, rocky island occupied by more buildings than the limited space seems
able to hold. One distinguishes a port with the masts of sailing boats.
The reflections on the water suggest bright weather; at the same time, the
sky at the right-hand side releases a sheet of rain. From the barely sug-
gested shore in the foreground, two hands in a size roughly corresponding
to the scale of the woman’s head reach up – one real, the other a mere
shadow. Their gesture is vertically directed toward the lovely face but hor-
izontally separated from it by the expanse of water and the depth of the
island.9
The French composer Olivier Messiaen, a contemporary of Penrose, saw a
reproduction of the painting in the Swiss art magazine Forme et Couleur and
included his response to it as number ten in a cycle of 12 songs for dramatic
soprano and piano, Harawi, composed in the summer of 1945. Messiaen
loved the poetry of Paul Éluard, Pierre Reverdy and André Breton. For the
five song cycles he composed during the years 1936–48, he wrote his own
poetry, in a style that became ever more Surrealist. His interest in Surrealist
art was of a very particular kind, however: a devout Catholic fascinated with
miracles and supernatural occurrences of any kind (from fairytales to biblical
stories), he searched for the religious dimension in non-real representations.
Here is the poem Messiaen wrote in response to Penrose’s painting, together
with my translation.

X – ‘Amour oiseau d’étoile’ ‘Love Star-Bird’


Oiseau d’étoile, Star-bird,
Ton œil qui chante, Your eye that sings,
Vers les étoiles, Toward the stars,
Ta tête à l’envers sous le ciel. Your head reversed under the sky.
Ton œil d’étoile, Your star-eye,
Chaînes tombantes, Falling chains,
Vers les étoiles, Toward the stars,
Plus court chemin de l’ombre au ciel. Shortest path from the shadow to the sky.
Tous les oiseaux des étoiles, All star-birds,
Loin du tableau mes mains chantent. Far from the scene my hands sing.
Étoile, silence augmenté du ciel. Star, increased silence of the sky.
Mes mains, ton œil, ton cou, le ciel. My hands, your eye, your neck, the sky.
(Olivier Messiaen, ‘Amour oiseau d’étoile’, from Harawi: chant
d’amour et de mort)

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Siglind Bruhn 145

Messiaen’s poem suggests that he interpreted Penrose’s painting as symbol-


izing the relationship between earth/water and sky/heaven, between body
and spirit. He describes the young woman’s head as ‘reversed under the
heaven’, believes her eyes to be ‘singing toward the stars’ and her freely
falling hair (‘falling chains’) to be mediating between the painting’s two
worlds. He identifies with the hands in the foreground, which he knows

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
to be ‘far from the scene’ defined by the woman linking night sky and sun-
lit island. The poem’s final line traces the central images once again in their
upward-leading path: ‘My hands, your eye, your neck, the sky’.
The poetic text also includes several of Messiaen’s favourite terms, even
though what they describe is not evident or at least not explicit in Pen-
rose’s painting: bird, song, star and silence. In the beginning, the poetic
voice addresses the young woman as ‘oiseau d’étoile’ (star-bird); later, as
the speaker turns to the hands that represent him, he takes up the image
by alluding to ‘Tous les oiseaux des étoiles’ (all star-birds). The inner con-
nection between eye, singing and star, established in the symbolic imagery
of the woman’s eye singing ‘toward the stars’, finds its equivalent on the
interpreter’s side when his observation that his hands sing leads immedi-
ately back to the stars. The attitude expressed in the longingly raised hands
and their distance to the intuited closeness ‘of shadow and heaven’ accounts
for the increased silence of the heaven.
The music Messiaen composed for this song imports its own nuances of
interpretation. Some of them are universal, others could be described as a
kind of idiosyncratic semiosis typical of this composer at the period when
he wrote this song cycle, and a third category relies on our recognition of
a specific musical quotation (or rather, a paraphrase) of material for which
Messiaen is well known.
I would like to begin with the third category, so that I can lead readers
from the specific to the universal. One year before he composed the song
cycle Harawi, Messiaen had written a cycle of twenty pieces for piano solo
entitled Vingt Regards sur l’Enfant-Jésus. That cycle thus celebrates various
contemplations of the child in the manger or gazes cast on the infant Jesus.
The principal cyclical theme in this composition is labelled in the score:
thème de Dieu, ‘theme of God’. Its basic phrase consists of five chords. The
melody harmonized with these chords is very simple, consisting of a three-
note repetition, a climax a little higher and a return to the earlier pitch.
The rhythm of this little phrase is also very simple: attacks of the three-note
repetition are all equal in length, the climax is twice as long and the final
relaxation, three times as long. The tempo of this piece – as it were: the
temporal dimension of God’s eyes falling on the infant Jesus – is extremely
slow: Messiaen imagines each of the repeated, ‘faster’ note values to last for
three normal heart beats.
In the song ‘Amour oiseau d’étoile’, the main line – ‘Oiseau d’étoile’ –
is composed in the same key as ‘Regard du Père’ and has almost exactly

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


146 Mediations and Transformations of Media

the same melodic line as the ‘theme of God’, except that the note repe-
tition occurs on the level of the climax, not on that of the release. Also,
the rhythm is slightly changed: instead of beginning with three beats of
equal length followed by ever longer note values, the duration now increases
incrementally all along (that is, instead of counting 1-1-1-2- -3- - one now
counts 1-2- -3- - -4- - - -5- - - - -). This is the thematic link: a kind of varied quo-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
tation. There is also a direct structural link: the initial movement of the
piano cycle, devoted to ‘The Father’s gaze on the infant Jesus’ and focus-
ing entirely on the ‘theme of God’, is formed in accordance with the triad of
the Pindaric ode, with two stanzas (strophe and antistrophe) rounded off by a
substantial coda (epode). The same layout is found in the song ‘Amour oiseau
d’étoile’.
The second realm of musical signification in Messiaen’s song on the poem
he wrote in response to Penrose’s painting is a case of what I like to call
idiosyncratic semiosis. This aspect begins on the level of the poem and con-
tinues in the music. Olivier Messiaen was a bona fide ornithologist who is
today recognized worldwide as the person who has most accurately tran-
scribed the song of hundreds of birds. From the 1950s onward, Messiaen
wrote a number of compositions exclusively devoted to the musical repre-
sentation of bird calls, with some sonic descriptions of the birds’ character-
istic habitat: Le Merle noir (The Blackbird), 1951, for flute and piano; Réveil
des oiseaux (Awakening of the Birds), 1953, for orchestra, piano and game-
lan; Oiseaux exotiques (Exotic Birds), 1956, for orchestra, piano and gamelan;
Catalogue d’oiseaux (Catalog of Birds), 1958, for piano; La Fauvette des jardins
(The Garden Warbler), 1970, for piano; Des Canyons aux étoiles (From the
Canyons to the Stars), 1974, for orchestra and soloists; Petites Esquisses
d’oiseaux (Little Bird Sketches), 1985, for piano; Un Vitrail et des oiseaux
(A Stained-glass Window and Birds), 1986, for orchestra, piano and gamelan.
All through his life, Messiaen interpreted the music of songbirds as a sign
of God’s presence in nature. During the 1940s in particular, however, when
the song under discussion originated and his birdsong transcriptions were
not yet fully developed, he often used birds to signify human silence, espe-
cially the silence filled with awe in view of the transcendent.10 Indeed, ‘birds’
appear in the title of Messiaen’s poem, ‘Amour oiseau d’étoile’ (‘Love Star-
bird’) and recur in several lines, although I have not been able to detect a
single bird in Penrose’s painting. For birds, one may thus read ‘silence’. In
the music of the Penrose-inspired song, the birds rejoice in the piano after
every single phrase.
The most significant representative of the first, universal category of
interpretation in this song is an element of musical symmetry. This time,
however, the symmetry is vertical and not horizontal, spatial and not tem-
poral. Thus there is no constitutional hurdle for human understanding, as
in the case of a sequence read backward. While most of us will still not
necessarily grasp details about the relative distance between simultaneously

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Siglind Bruhn 147

major with in
triad added first

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
sixth inversion

Figure 8 Messiaen’s favourite vertically-symmetric chord: the generation of its spatial


pattern and its distribution on a keyboard

sounded pitches when hearing a chord, it is a strange fact that almost all
people can, somehow, hear and recognize the effect created, especially the
one achieved in the most frequently used vertically-symmetric chord. The
technical description of this chord is not essential here; for musicians, its
basic form is a major triad with added sixth, in first inversion. For an easy
way of making this vertical symmetry of sounds visual, it is helpful that
Messiaen uses only, and always when he uses this chord, the one in F# major,
and in this key, the chord even looks symmetric on any keyboard of a piano,
organ, xylophone and so on (see Figure 8).
In the piano piece ‘Regard du Père’, the very same chord is heard as a
concluding chord repeated 24 times. A little later in the piano cycle, the
chord also participates in another prominent theme, one that Messiaen’s
score identifies as thème d’amour, ‘theme of love’. As a result of this dual
appearance in the gaze of the loving heavenly Father and in the ‘theme of
love’, I like to speak of this chord itself – the chord that is vertically sym-
metrical both in its interval structural and in its visual appearance on the
keyboard – as the ‘chord of love’.
This brings me to another word in Messiaen’s poem that is not entirely
explained by Penrose’s painting: the word ‘love’, which appears only, but
very prominently, at the outset of the title ‘Amour oiseau d’étoile’. If the
remainder of Messiaen’s poem does not explicitly stress the quality of love,
Messiaen’s music does so all the more. For listeners, the predominant sym-
bolic message the composer conveys in this piece resides in the conclusion
of each phrase. The first ten segments of the song (eight in the stanzas and
two more at the beginning of the coda) each conclude with two F-major
six-five chords; the coda’s two final segments each add one more instance
of the characteristic chord. More than half of these harmonically equivalent
chords are cast in a vertically symmetrical format, thus appearing as variants
of Messiaen’s ‘chord of love’.
One could expand this symmetrical chord by repeating one or more
pitches on either side. Messiaen does so with considerable regularity, mostly
adding the same number of pitches above the piano chord’s highest and

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


148 Mediations and Transformations of Media

below its lowest note. The composer uses the vertically symmetrical chord
of love at the end of every vocal phrase, in different expansions. In the coda’s
long final phrase, the chord on the downbeat of m. 23, which provides
the backdrop for the most extensive bird song in this piece while separat-
ing the summary remark (‘My hands, your eye, your neck, the sky’) from
the body of the poem, presents the most expansive of the vertical symme-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
tries: a chord of ten vertically symmetrical pitches. Thus the word ‘amour’,
prominent in the song’s title but not verbally present in the poetic text that
is sung, is embodied (and even ubiquitous) in the music. The musical lan-
guage confirms the composer’s trust that, while the union of the two lovers
suggested in Penrose’s canvas may seem unattainable, they themselves are
reliably supported by God’s love.

Conclusion

As I have attempted to show, many of the triangles that are habitually con-
structed in the field of the arts have at least one angle that may make the
necessity of closure and geometric logic seem ‘impossible’. This is true not
only for the notorious ‘artistic triangle’ supposedly linking the three partici-
pants involved in the life of an artwork – creator, mediator and beholder, but
it is also true in the case of triangles defining each of these participants indi-
vidually. The example I fashioned with the Hindu Trinity as a metaphoric
trio shows this with regard to three essentially different creative aims; my
case study from Tavener’s opera documents three conditions in which the
limits of human sense perception displace the beholders’ appreciation from
the conscious to the subconscious realm; my example from Messiaen’s art
song demonstrates at once the extraordinary richness of an artistic media-
tor’s input and the highly individual reading of the first work of art he or
she may incite in the final appreciators by presenting it through the lens of
his or her own, secondary medium.
Were one to attempt to come full circle, from the piece of music interpret-
ing a poem that in turn responds to a painting and now back to Penrose’s
(and Reutersvärd’s) ‘impossible triangle’, one could observe that the repeated
mediation does indeed lead to an interestingly twisted appearance of the
three-dimensional object: returning to the point of departure after having
travelled through the three hinged legs, one notices with wonder that none
of the surfaces is where it was before. Mediation – in the various meanings of
this term – is a creative act that changes its object’s appearance and message.

Notes
1. Sir Roger Penrose, OM, FRS (born 8 August 1931) is an English mathematical
physicist and Emeritus Rouse Ball Professor of Mathematics at the Mathematical
Institute, University of Oxford. Sir Roland Penrose (1900–1984) was an English
artist, historian and poet. He was a major promoter and collector of modern art,
an associate of the Surrealists in the United Kingdom and a friend of Picasso.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Siglind Bruhn 149

2. L. Elleström (2002) Divine Madness: On Interpreting Literature, Music, and the Visual
Arts Ironically (Lewisburg PA and London: Bucknell University Press).
3. John Tavener, Mary of Egypt: An ikon in Music and Dance, was composed in 1991.
The score was published by Chester Music, London, in 1992; a compact disc
recording was issued by Collins Classics in 1993 (no. 70232).
4. ‘John Tavener in conversation with Michael Stewart on Mary of Egypt’, recorded
interview added as track 20 to the Collins Classics recording of Mary of Egypt.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
5. For more details on this liturgical music, see K. Levy (2001) ‘Music of the
Byzantine rite’ in S. Sadie (ed.) The New Grove Dictionary of Music and Musicians,
2nd edn (London: Macmillan) pp. 553–66.
6. Peter Sparling, professor of dance at the University of Michigan School of
Music and choreographer of the Peter Sparling Dance Company, in a private
conversation with the author, fall 2003.
7. See J. A. W. Heffernan (1993) Museum of Words: The Poetics of Ekphrasis from
Homer to Ashbery (Chicago: University of Chicago Press), p. 3. In a more encom-
passing wording, Claus Clüver defines ekphrasis as ‘the verbal representation
of a real or fictitious text composed in a non-verbal sign system’; cf. C. Clüver
(1997) ‘Ekphrasis Reconsidered: On Verbal Representations of Non-Verbal Texts’
in U.-B. Lagerroth, H. Lund and E. Hedling (eds) Interart Poetics: Essays on the
Interrelations of the Arts and Media (Amsterdam: Rodopi), p. 26.
8. The original was long believed to have been lost during World War II. However,
in a recent book by Penrose’s son, it is listed as ‘held in a private collec-
tion’: A. Penrose (2001) Roland Penrose: The Friendly Surrealist. A Memoir (Munich:
Prestel), p. 48.
9. A full-colour reproduction of Roland Penrose’s oil-on-canvas composition of 1937
Voir c’est croire (L’Île invisible) can be found at http://home.vrweb.de/∼ edition-
gorz/bruhn5-06.pdf (p. 215), date accessed 13 August 2009.
10. Cf. both ‘les oiseaux du silence’, mentioned in the commentary for the Regard du
Fils sur le Fils (no. V), and Regard du silence (no. XVII).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


9
Beyond Definition: A Pragmatic
Approach to Intermediality

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Valerie Robillard

Theories [are] instruments, not answers to enigmas, in which we


can rest. We don’t lie back upon them, we move forward, and, on
occasion, make nature over again by their aid. Pragmatism unstiff-
ens all our theories, limbers them up and sets each one at work.
—William James1

As media continue to merge in new and interesting ways, the various


theories and definitions formulated to describe and articulate the nature and
complexity of their interactions have generated compelling, and often con-
flicting, critical discourses. Indeed, the current plethora of perspectives on
‘intermediality’ not only demonstrates the slipperiness of the term but also
suggests that there may be more than one theoretical inroad by which to
fully understand the multiplicity of intermedial operations. Current research
into intermediality, with some notable exceptions,2 has primarily focused
on defining the terms of the field; however, it is becoming increasingly
clear that definitions, although essential in laying out common terms of
discourse, do not fully contribute to our understanding, or articulation, of
the various types and degrees of medial interaction.
The purpose of this essay is to demonstrate the need to employ ontological
systems to delineate medial types, systems that have proven indispensable to
other disciplines (such as the natural sciences and linguistics) in determining
the relative positions of concepts and categories with respect to one another.
Although the notion of ‘exact disciplines’ can be debated, the sciences have
proven to be more easily ordered into categories than have the arts, as the
latter are closely tied to fluctuating cultural and aesthetic codes.
This essay will explore the advantages of applying a system of categories
to intermedial inquiry by focussing on the intertextual exchange between
verbal and visual texts within the hybrid artwork. The advantage of explor-
ing intermediality (partly) in terms of intertextuality lies in the fact that
the interaction between the visual and verbal arts can be considered in
terms of their interreferentiality and semiotic encoding. As Peter Wagner has

150

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Valerie Robillard 151

suggested, the discourse on intermediality needs a reunification of writing


and painting under ‘the common banner of representation’ and an:

intermedial exploration of the working of both linguistic and pictorial


signs in one medium. Intertextuality in art thus becomes a possibility in
research . . . in a more extensive variety of ekphrasis, broadly understood,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
which considers visual art as a space serving as both meeting ground and
battle ground for encoded, rhetorical, sign systems that refer to texts and
images.3

Wagner’s argument for subsuming verbal and visual operations under the
heading of ‘intertextuality’ offers intriguing possibilities for the categoriza-
tion of distinct intermedial types. Heinrich Plett, for example, has suggested
that intermediality be considered a sub-category of intertextuality, as ‘it is
not usually single signifiers which are exchanged for others but themes,
motifs, scenes or even moods of a pretext which take shape in a different
medium.’4 Both Wagner and Plett make a good case for considering inter-
textuality within the intermedial paradigm (although the hierarchy implied
by Plett will be questioned later in this essay).
To address the operation of intertexuality within an intermedial frame-
work, I will explore one small theoretical corner of the intermedial debate,
that of ‘ekphrasis’ and ‘illustration’, both of which, as I have argued
elsewhere,5 can be analysed in terms of their intertextuality. I hope to
demonstrate that such complex combinations require a broad epistemo-
logical basis and ontological system by which to explore and articulate the
variety and types of their interaction.

Categorizing intermedial operations

Why employ categories in aesthetic discourse since, by their very nature,


categories are imperfect representations of phenomena? Can the demand
that categories be ‘mutually exclusive’ (the ‘bottom line’ for scientific mod-
els) be useful (or even possible) for studies in the humanities whose objects
are often far more ‘subjective’ than the empirical sciences? In other words,
categories, because of their perceived rigidity, may seem inappropriate for
organizing concepts that are generated by cultural or aesthetic codes. On
the other hand, categories, unless they are mutually exclusive, will not
accurately represent difference.6
In spite of the obvious pitfalls associated with classifying intermedial
types, the formulation of a system of categories which takes into account
the ‘slippage’ associated with cultural or aesthetic encoding might success-
fully contribute to understanding the operations and interrelations of media
through isolating and categorizing their various domains and characteristics.
The typology presented here, which I have called ‘The Differential Model’, is

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


152 Mediations and Transformations of Media

divided into categories and sub-categories and focuses on types and degrees
of intertextual relationships.7 This model, originally designed for the pur-
pose of delineating and differentiating logo-centric ekphrastic operations,
has been altered to accommodate intertextual operations on both sides of
the relationship (verbal-visual, visual-verbal).

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Differential model

This section will present a conceptual framework devised to differentiate


types of medial interactions, most specifically between the verbal and visual
arts. As, similar to verbal ekphrasis, the majority of ‘illustrated’ texts enlarge,
reduce or in some way alter their pretexts – and for similar reasons – these
operations can also be traced within a categorical framework. The typol-
ogy in Figure 9 is designed to be (a) non-hierarchical and (b) one whose
categories serve as vehicles for making generalizations about an artwork’s
representative, referential and interactive capacity.
This model differentiates between explicitly-marked visual or verbal texts
and those which signal a more subtle, associative relationship with their
pre-texts. Each heading is divided into subheadings. The first category in the
typology, labelled as ‘Referentiality’, reflects the assumption that ekphrastic
texts, because they are specific to particular works of art, will, in the rhetor-
ical spirit of ekphrasis, mark their pre-texts to some degree. In a poem, for
example, the name of the artwork or artist might be explicitly mentioned in
the title or elsewhere in the text. Similarly, in a visual work of art the verbal
pre-text might be mentioned in a label next to the painting or attached to
the frame, or, as in some modernist texts, appear within the painting itself.

naming

Referentiality allusion

indeterminate marking

selectivity
Re-presentation
structurality

mythos/topos
Association
dialogicity

Figure 9 The differential model

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Valerie Robillard 153

The second sub-category, ‘allusion’, subsumes more subtle references in


which the pre-text is called to mind through the presence of some (generally)
familiar aspect present in the target text. Finally, the sub-category ‘indeter-
minate marking’ accounts for the presence of some reference that would not
be generally understood as connected to a pre-text, but would be accessible
to a viewer (or ‘ideal reader’) who has subject-specific knowledge or a par-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ticular ‘cultural memory’. The second category, labelled ‘Re-presentation’,
differentiates between the ways in which the textual details of the verbal or
visual pre-text are present in the target text. The sub-category, ‘selectivity’,
refers to the detail from the pre-text that is transferred to the target text;8 the
second sub-category, ‘structurality’, refers to the manner in which a text fol-
lows the physical organization of its pre-text. (In the visual arts, for example,
this is often encountered in the triptych, where the narrative structure of the
original story is followed in a linear fashion; in the verbal arts, a poet might
organize the objects of the poem around the structural conventions of, say,
a cubist painting.) Finally, the ‘Associative’ category accounts for those texts
concerned with conventions or ideas associated with the visual or verbal
pre-text (mythos/topos) or are ‘dialogical’ in that they deliberately establish
a tension (both semantic and ideological) between the original text and the
new, wherein the former is cast in a new, opposing framework.9 (A filmic
example of ‘reframing’ would be Peter Greenaway’s Prospero’s Books, based
on Shakespeare’s play The Tempest.) This final category is linked to the ref-
erential properties of the text (Category I), as reception of these intertextual
elements will depend on level of ‘readership’.10 The following section will
explore the possibilities of such a typology by considering the complexi-
ties of the Scottish artist Calum Colvin’s Fragments of Ancient Poetry (2002),
focussing on the manner in which both verbal and visual texts interact to
convey meaning.11

The intermedial artwork

Colvin’s Fragments of Ancient Poetry, a series of photographs based on James


MacPherson’s 1760 text of the same title, is unique in its movement
toward a complete intermedial performance, as it is situated, according to
Colvin, ‘on the boundaries of Painting, Photography, Sculpture and Elec-
tronic Imaging’.12 Through his employment of a wide variety of media,
Colvin establishes a multi-levelled relationship with the original text. Before
analysing Colvin’s work as an intermedial ‘event’, however, I would first
like to address the aesthetic conventions that it challenges – namely those
connected with ‘illustration’.
Most definitions of illustration suggest that the translation of the verbal to
the visual necessitates some level of exemplification of the pre-text in which
the visual becomes an emblem for the verbal. Yet nearly all (art) historical
periods have produced a wide variety of visual approaches to verbal texts

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


154 Mediations and Transformations of Media

that far exceed simple transcriptions. Meyer Schapiro has suggested a vari-
ety of ways in which illustrations engage their visual pre-texts, ranging from
those that ‘enlarge’ the verbal text, adding details, figures, settings and so
on, that are not present in the original, to those which are extreme reduc-
tions of the text in which only a few details are represented. These artistic
choices are often the function of cultural, social and/or aesthetic concerns of
a particular time and place, or reflect individual artistic style or ideology.13

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
I would like to postulate that, as these operations apply equally to verbal
and visual ‘texts’, both can be subsumed under the heading of ‘ekphra-
sis’ (in a departure from the logo-centric origin of the term). Proceeding
from this understanding of the term, the textually-based aspects of Colvin’s
Ossian series, therefore, can be classified as a type of ‘visual ekphrasis’ which
belongs to the overarching notion of ‘intertextuality’. As will become clear,
however, this intertextual aspect accounts for only a small part of the inter-
medial and multimedial nature of Colvin’s Fragments, which are, as Tom
Normand points out, ‘created as constructed sets, painted with iconic sub-
jects, decorated with symbolic references, and, finally, photographed. The
photographic images are subsequently digitized and presented on canvas.
Sculpture, environment, collage, painting, photography, and computer-art
combine in a paradoxical and fantastic vision.’14 What are the implications
of attempting to categorize both textual and extra-textual media in works
such as Colvin’s?
Colvin’s Fragments depicts the discrepancies between a culture’s percep-
tion of its past and modern realities through exploring the epic nature
of MacPherson’s Fragments of Ancient Poetry, supposedly the poet Ossian’s
account of the heroic deeds of Fingal, a third-century Scottish king. In
spite of ongoing scepticism concerning authorship, this literary work not
only had an important impact on Romantic sensibility, but has become
an integral part of Scottish identity. These epics, which ‘conjured a world
of heroic northern warriors whose savagery was tempered by distinguished
codes of honour, sentiment, and a recognizable morality’,15 form the basis
of Colvin’s intermedial artwork. In a series of eight photographs, Colvin
addresses the relevance of these epic conventions to contemporary Scottish
identity by juxtaposing these with objects from Scottish popular culture and
more recent literary past. The importance of Colvin’s artwork to interme-
dial studies in general (and ‘visual ekphrasis’ in particular) is significant, as
it far exceeds simple exemplification of its pre-text and the variants of illus-
tration suggested by Schapiro. According to Iain Gale, Colvin ‘wants to peal
away the layers of historical image-making which obscure Scottish culture’
and with works such as Fragments, ‘he is reflecting on how Scotland has
become a nation of stereotypes’.16 Colvin achieves this dialogical effect not
only by juxtaposing fragments of popular culture with subtle references to
the Ossianic tradition, but also by making use of wider references to cultural

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Valerie Robillard 155

archetypes and literary texts associated thematically with the issues raised
by the original: Macpherson’s Fragments of Ancient Poetry.
In Fragments, Colvin arranges several objects against a stone backdrop,
objects that serve as literary references (primarily to MacPherson, Walter
Scott and Robert Burns) or perform a metaphorical function (see Figure 10).
Colvin begins the series with a Maori head projected onto a movie screen,
an image suggesting the sense of a Celtic past.17 Over a sequence of eight

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
photographs, Colvin transforms this central image into a cultural cliché:
a tipsy Scotsman sporting a ‘Jimmy hat’. Eventually, this image too disap-
pears, ‘crumbling into dust, becoming the stuff of legend’.18 Further, these
photographs contain oblique references to the epic tradition embodied by
the Ossian texts through the incorporation of fragments of Colvin’s por-
traits of Robert Burns and Walter Scott: an eye borrowed from Burns and an
ear from Scott, ironically placing the heroic epic against the romanticized

Figure 10 Calum Colvin, Fragments of Ancient Poetry IV. Courtesy of the artist

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


156 Mediations and Transformations of Media

Highland adventures of Scott and the lyrics of Burns. Also included in this
stone landscape are modern icons that establish a similar ironic connection
between the past and present, between ancient and popular culture: a slide
projector showing images from the film Brigadoon, a record player (with the
drunken Scot on the turntable – later to develop into the central image) – and
a souvenir Celtic lollipop. These are just three objects among the many that

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Colvin adds to this ruined landscape, but they are sufficient for our discus-
sion here: the intermedial tension achieved by placing literary and cultural
icons (both past and present) within a shared visual space.

Intertextual components of Fragments

To what extent is Colvin’s Fragments ‘intertextual’? More importantly, what


aspects of this work cannot be contained in such a text-based framework?
I will explore these questions by turning to the ‘Differential Model’. The
first question that we will want to ask is this: in what way is Colvin’s art-
work ‘referential’? In other words, does it clearly indicate that it is based on
Macpherson’s Fragments of Ancient Poetry? Because the pre-text is identified
in the title of Colvin’s work, referentiality is high. We also find several liter-
ary allusions and some intriguing indeterminate marking that are indexical
of the original text. Furthermore, the inclusion of the (partial) images of
Burns and Scott place the work within more than one literary context, and
these visual allusions fall under the sub-category of ‘dialogicity’, which will
be discussed below.
Moving to the ‘Re-presentative’ category, it is clear that few details of the
original text have been selected. However, what at first glance appears to
be a series of still-life images turns out to contain a narrative progression
achieved by a rearrangement of objects and the transformation of one image
into another. Although the artwork does not follow Ossian’s narrative line, it
nevertheless literally performs the larger issue of cultural change: the ‘mea-
sured disintegration’19 of a people that governs Macpherson’s text and both
artworks in Colvin’s series (Blind Ossian and Fragments). This brings us to the
final, ‘Associative’, category.
If Colvin’s Fragments does not explicitly engage its pre-text, either through
reference or re-presentation, it does, nevertheless, have a highly sym-
bolic and metaphorical relationship to its pre-text and this relationship is
‘associative’ through its play on cultural memory and associated myths.
Furthermore, we can claim that dialogicity in this work is high since the
issue of Scottish identity lies at the heart of the artwork’s conception and,
most importantly, its construction. Recalling that dialogicity refers to the
way in which a verbal or visual text ‘reframes’ its pre-text or places it in
another context, the use of Macpherson’s text is dialogical in that it raises
the issue of the demise of mythic stature in the modern world. Aside from its
dialogical stance, Colvin’s Fragments is concerned with the ‘mythos/topos’

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Valerie Robillard 157

that generated Macpherson’s text and its cultural and literary aftermath
primarily by stressing the absence of ancient warriors and a noble race.
The placement of the head of Walter Scott in this construction is asso-
ciative in the sense that, influenced by the romanticism of the Ossian
fragments, Scott popularized Highland history through his novels, yet was
also regarded as responsible for its trivialization and for generating much

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Scottish ‘kitsch’ (tartan kilts, Jimmy hats and so on), criticized as ‘the fatal
legacy of Scott’s achievement’.20 The inclusion of the film Brigadoon (more
romantic kitsch) in all eight plates serves to underline the overall theme
of a lost or misunderstood national identity. Finally, the work is replete with
semiotic codes for the modern world: for example, the ‘Celtic’ lollipop wrap-
per, the drunken Scot and the Jimmy hat, become the ‘potent symbol of
Scotland’s self-denigrating identity crisis’.21 In view of the fact that Colvin
uses Macpherson’s text as a template upon which to explore questions con-
cerning cultural heritage and its deconstruction, we can arguably conclude
that the intertextual relationship of Colvin’s visual text to its verbal pre-
text is primarily ‘associative’ (more specifically, dialogical) and not in any
way depictive – ultimately rendering an artwork that is a far cry from the
conventional notion of ‘illustration’.

Intertextuality? Intermediality? Multimediality?

The complexity of Colvin’s Fragments raises interesting questions concern-


ing the ways in which we define intermediality and categorize its operations.
If, following Plett, we understand intermediality to function as a variant of
intertextuality, then we will be concerned with textual semiotics and with
the manner in which the diverse elements of a pre-text are incorporated into
the new text (in this case, the artwork). In applying the ‘Differential Model’
to Colvin’s Fragments, however, we find that there is much present in this
complex work that cannot be accounted for within an intertextual frame-
work. Although the iconic (and ‘found’) objects that have been added to
Colvin’s artwork are not related to Macpherson’s Fragments of Ancient Poetry
but to contemporary popular culture, these additions can be categorized
as ‘dialogical’, as the placement of ‘life media’ in juxtaposition with other
(oblique) references to Macpherson’s text render a kind of spatiotemporal
contextual ekphrasis, as defined by Hans Lund. According to Lund, time-
contextual ekphrasis is generated by references to other artworks or texts
which arouse a reader’s associations or ‘memories’ and it is the reader ‘who
links the verbalized image to the memory of other images’.22 Spatiocontex-
tual images are linked to each other through ‘the physical space in which
they are contemplated’.23 In the case of Colvin’s Fragments, the placement
of various objects within the shared visual space evoke several levels of read-
ing on the part of the viewer and the resulting dialogical effect is crucial to
Colvin’s overall theme of cultural loss.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


158 Mediations and Transformations of Media

However, what the above analysis has demonstrated is that there is no


vehicle within an intertextual system of categories to account for the mate-
riality of Colvin’s artwork nor is there a way to categorize the process of
composition, which involves the interaction of a number of media. As we
know, Colvin’s work is based on a complex process: an image first con-
structed in three-dimensional space is painted, photographed and finally

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
destroyed. All that remains are the photographs: ‘layers and traces that
must be uncovered by the reader’.24 Colvin also produces meaning in his
entire Ossian series by the use of computer imaging. It has been argued that
Colvin’s employment of the computer for the series addresses an important
and controversial aspect of the original text: that Colvin was ‘inspired to res-
urrect Macpherson’s texts by the dubious authenticity of the Ossian myths
and their similarity to the deceptions possible with modern digital photogra-
phy, which also distort and subtly change images of reality’.25 Colvin’s work,
therefore, not only takes a dialogical stance towards Macpherson’s original
text but also takes issue with the very process of its making.
Colvin’s Fragments become even more interesting, for this study, when
placed against major theories of intertextuality, namely those of Michael
Riffaterre, whose reader-oriented theories focus on the hypogram. As Colvin
points out, his use of the fixed viewpoint of the camera is designed:

to collect the manipulated and constructed image in order to create elab-


orate narratives. These narratives have the quality of being both open
and closed. They are closed in that they clearly refer to given icons
and archetypes of Western culture, but open in that they accommo-
date any number of potential readings. These readings, in turn, reflect
the contemporary cultural climate and the unique authorial role of the
viewer.26

This notion of the ‘closed’ and ‘open’ narrative recalls Riffaterre’s differenti-
ation between obligatory intertextuality, which demands that the reader take
account of some ‘hypogrammatic origin’, and aleatory intertextuality, which
allows the reader to read a text through the prism of all and any familiar
texts.27 Furthermore, Riffaterre has posited three kinds of ‘semantic indirec-
tion’ by which the reader/viewer is led to consider the hypograms that lie
behind the target text. These are: (1) displacement, where the sign shifts
meaning, as in metaphor and metonymy; (2) distortion, which functions
through ambiguity, contradiction or nonsense; and (3) creation, in which
textual space serves as a principle of organization for making signs out of
linguistic (and semiotic) items that may not be meaningful otherwise.28
Colvin’s series clearly makes use of all forms of ‘semantic indirection’, as set
forth by Riffaterre, and in so doing has achieved the creative manipulation
of visual space necessary to form a symbolic link between past and present,
text and ‘life media’, and visual and verbal texts. As these links depend on

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Valerie Robillard 159

the translation of one text to another, we can conclude that there is much
in the series that is intertextual. However, there is more than one opera-
tion at work that cannot be contained (or explained) within an intertextual
framework. The materials and media employed are not ‘texts’ and therefore
are not referential; furthermore, in a work such as Colvin’s where media are
used and then erased to produce the final artwork, the work bears traces of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
process which need to be accounted for. Here is where the slipperiness of ter-
minology and categories comes into play. As it is clear that overall meaning
in Colvin’s Fragments depends on both textual and medial interaction, the
question arises as to how one system of categories or set of definitions can
contain the interactive potential of such hybrid works.

Categories and medial difference

We must be systematic, but we must keep our systems open.


—Alfred North Whitehead29

Categories delineating medial types are, almost by definition, bound to be


somewhat inaccurate in their attempt to contain various genres as well as
to keep apace with the ever-developing and innovative ‘cross-over’ experi-
ments of contemporary artists and writers. As Dick Higgins suggests, ‘much
of the best work being produced today seems to fall between media’.30 This
essay has argued in favour of the categorization of one aspect of interme-
diality: the (inter)textual aspects of hybrid artworks. Yet even the notion of
‘text’ is problematic, as our agreement as to what constitutes a ‘text’ remains
elusive. According to Winfred Nöth (quoting Petöfi), ‘since textuality “is not
an inherent property of certain objects, but is rather a property assigned
to objects by those producing or analysing them,” it is not surprising that
semioticians of the text have been unable to agree on a definition and on
criteria of their object of research’. However, as Nöth goes onto explain, ‘In
the pragmatic view, the text is defined by criteria of communication . . . the
pragmatic criteria determine the text within its situational context. This con-
text comprises textual and extra-textual phenomenon’.31 Taken strictly, the
notion of ‘text’ may apply only to written texts or, taken more widely, to the
visual arts, or it may be extended to all types of communicative processes
such as film, performance (and gesture), iconic objects and so on. It is this
wider sense of the word that gives us the freedom to explore the intertextual
(and therefore encoded) nature of the composite artwork and to differen-
tiate this type of medium from those which are ‘extratextual phenomena’.
In other words, we are in a position to distinguish the ‘message’ from the
‘material’. Of course, this differentiation has raised serious questions of its
own. If we can find ways to categorize intertextual operations within the
intermedial framework, how do we account for the ‘material’ (extratextual)
aspects of these composite artworks? For example, the depiction of a camera

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


160 Mediations and Transformations of Media

and film screen within Colvin’s Fragments bears an encoded message relevant
to its context and therefore is intertextual; the use, however, of a camera to
construct this work is purely material and calls for different categories alto-
gether, those that take process into account. For example, I have referred
throughout this essay to Colvin’s Fragments as ‘photographs’, yet photogra-
phy is only one medium used for the total artwork. How do we account for

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
other medial traces within a categorical framework? This brings us back to
the problem of categories in general.
Returning to Plett’s suggestion that intermediality be considered a sub-
category of intertextuality, the whole issue of ‘genus, species, class’ is raised
in which we are again confronted with definition and nomenclature. Plett
here addresses an important question of hierarchy: which is the largest, most
inclusive category and which the sub-category? In other words, if interme-
diality is the overarching term, where does intertextuality, clearly a crucial
element in intermedial discourse, come into play? Perhaps a way forward
might be to reverse Plett’s hierarchy so that ‘Intermediality’ becomes the
overarching concept, with one of its major categories labelled ‘Intertextual-
ity’, which will account for those aspects of the composite artwork that are
communicative and encoded; a sub-category of Intertextuality, then, might
be labelled as ‘ekphrasis’, to be shared with sub-categories that account for
other types of intertextual exchange (verbal–verbal or visual–visual trans-
ference for example). In this vein, a second category under Intermediality
might then be classified as ‘multimedial’ (a term explored by Jürgen Müller
in this volume and elsewhere). This category would include the materials
employed (film, paint, computer technology and so on). The manner in
which these could be placed into sub-categories might be drawn from cogni-
tive categories such as those formulated by Lars Elleström (in this volume),
which differentiate between types of sensory impact generated by the medial
‘event’. It is perhaps this type of category which could account for the lay-
ers (and traces) of media used by Colvin to create the final artwork, as these
elements are not encoded messages but pure materials and therefore sen-
sory. Once we have been able to place the various medial types within a
categorical framework, we might then be in a position to address more gen-
eral questions, such as those raised by Werner Wolf on determining ‘medial
dominance’.32
It is clear that what categories offer us is the mechanism by which to
schematically represent intertextual varieties and medial oppositions. Sim-
ply stated, the sciences (including linguistics) have taught us the usefulness
of models in organizing experience and knowledge. Whether or not cat-
egories can be as accurately employed in literary discourse as they are in
other disciplines, they do, nevertheless, bring us one step closer to a shared
language and ontological system by which to advance our discourse on inter-
medial topics. According to Nelson Goodman, understanding the world of
art is very much like understanding the world of science, as it always requires

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Valerie Robillard 161

interpretation of symbols and their relations among themselves.33 Devel-


oping a shared system of categories, which would have the advantage of
delineating subtype/super-types and articulating part–whole relationships,
is as relevant in determining the positions of the aesthetic concepts with
respect to one another as it is in categorizing genus and species within the
more ‘exact’ disciplines. Recalling William James’s statement on pragmatism

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
which began this essay, devising pragmatic categories need not set artifi-
cial parameters around our topic nor limit its extension to other potential
developments within intermedial discourse. The real challenge, as I see it,
is to find useful frameworks by which to articulate fundamental differences
among the media before we can begin to explore the nature and extent of
their interaction.

Notes
1. B. Kuklick (ed.) (1981) W. James, Pragmatism (Indianapolis and Cambridge:
Hackett Publishing Co.) (first published in 1907), p. 28.
2. See, for example, essays by Lars Elleström, Jørgen Bruhn and Jürgen Müller in this
volume.
3. P. Wagner (1996) Icons, Texts, Iconotexts: Essays on Ekphrasis and Intermediality
(Berlin: Walter de Gruyter), p. 36.
4. H. Plett (1991) ‘Intertextualities’ in Intertextuality: Research in Text Theory (Berlin:
De Gruyter), p. 20. In his essay, Plett suggests an intertextual framework based on
‘verbal and non-verbal signifiers, including the pictorial as a form of intertextual
discourse’.
5. V. Robillard (1998) ‘In Pursuit of Ekphrasis: An Intertexutual Approach’ in
V. Robillard and E. Jongeneel (eds) Pictures Into Words: Theoretical and Descriptive
Approaches to Ekphrasis (Amsterdam: VU University Press), pp. 53–72.
6. This problem has been addressed by Elinor Rosch, who has suggested in her pro-
totype theory that some disciplines need to employ ‘fuzzy categories’ to describe
relations between concepts. See G. Lakoff (1987) Women, Fire, and Dangerous
Things: What Categories Reveal About the Mind (Chicago and London: University
of Chicago Press), p. 15.
7. The typology employed here was developed to delineate ekphrastic types inspired
by Manfred Pfister’s notions of intertextuality. See Robillard (1998) ‘In Pursuit of
Ekphrasis’, pp. 60–2.
8. The terms ‘selectivity’ and ‘dialogicity’ have been adapted from Manfred Pfister’s
Intertextual Scales set out in his article, ‘Konzepte der Intertextualitität’ in
U. Broich and M. Pfister (eds) (1985) Intertextualität: Formen, Funktionen, anglis-
tische Fallstudien (Tübingen: Max Niemeyer).
9. Ibid.
10. This is where the idea of ‘mutual-exclusivity’ of a textually-based model breaks
down. A future restructuring of such a model would need to provide several
branches that account for the indeterminacy of reception.
11. One of Scotland’s leading contemporary artists, Calum Colvin is Professor of
Fine Arts at the University of Dundee. His works have been exhibited in venues
such as the Metropolitan Museum of Modern Art in New York, the Museum

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


162 Mediations and Transformations of Media

of Fine Art in Houston, Texas, the Victoria and Albert Museum in London, the
Scottish National Portrait Gallery in Edinburgh, which exhibited Colvin’s Ossian
series, as well as the Gallery of Modern Art in Glasgow (The Mad Hatters’ Review,
http://www.madhattersreview.com/issue7/contributors7.shtml, date accessed 13
August 2009). The Ossian series includes Blind Ossian and Fragments of Ancient
Poetry. I would like to express my appreciation to Calum Colvin for generously
granting permission to use his images as the basis for this essay.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
12. Quoted after J. Lawson, executive curator, National Portrait Gallery, Edinburgh,
http://www.calumcolvin.com/thumbs8.htm, date accessed 13 August 2009.
13. M. Schapiro (1973) Words and Pictures: On the Literal and the Symbolic in the
Illustration of a Text (The Hague and Paris: Mouton), pp. 9–16.
14. T. Normand (2002) Calum Colvin: Fragments of Ancient Poetry (Edinburgh: Scottish
National Portrait Gallery), p. 11.
15. Ibid.
16. I. Gale, Scotland on Sunday, 22 September, 2002.
17. It has been suggested that the head might be seen as an ‘ironic leitmotif repre-
senting the fallibility of history and the problem, even the absurdity, of those
narratives that search for essences, particularly racial essences’: Normand (2002)
Calum Colvin, p. 22.
18. P. Miller, Glasgow Herald, 14 September, 2002.
19. Normand (2002) Calum Colvin, p. 22.
20. Ibid., p. 44.
21. Gale, Scotland on Sunday, 22 September, 2002.
22. H. Lund (1998) ‘Ekphrastic Linkage and Contextual Ekphrasis’ in V. Robillard
and E. Jongeneel (eds) Pictures Into Words: Theoretical and Descriptive Approaches to
Ekphrasis (Amsterdam: VU University Press), pp. 173–88 at pp. 176–7.
23. Ibid., p. 177.
24. Gale, Scotland on Sunday, 22 September, 2002.
25. Miller, Glasgow Herald, 14 September, 2002.
26. Quoted in Calum Colvin, http://www.calumcolvin.com, date accessed 13 August
2009.
27. M. Worton and J. Still (eds) (1990) Intertextuality: Theory and Practices (Manchester
and New York: Manchester University Press), p. 26.
28. Riffaterre has set out these theories most notably in (1980) Semiotics of Poetry
(London: Methuen) and Text Production (1983) (New York: Columbia University
Press).
29. Quoted in J. F. Sowa (2003) Ontology, http://www.jfsowa.com/ontology/.
30. D. Higgins, Synesthesia and Intersenses: Intermedia, http://www.ubu.com/papers/
higgins_intermedia.html, date accessed 13 August 2009. Originally published
1966 in Something Else Newsletter I.
31. W. Nöth (1990) Handbook of Semiotics (Bloomington and Indianapolis: Indiana
University Press), pp. 329–32.
32. W. Wolf (2009) The Musicalization of Fiction: A Study in the Theory and History of
Intermediality (Amsterdam and Atlanta GA: Rodopi), p. 37.
33. N. Goodman (1998) Languages of Art: An Approach to the Theory of Symbols
(Indianapolis: Hackett Publishing).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


10
Translating Sounds: Intermedial
Exchanges in Amy Lowell’s
‘Stravinsky’s Three Pieces

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
“Grotesques”, for String Quartet’
Regina Schober

‘Imagine Media!’ This motto gives reason to reflect upon the concept of
mediality, the problem of medial boundaries and possible spaces for inter-
medial exchange and cross-fertilization.1 The word ‘imagine’, however, also
points to the permeability and constructedness of allegedly established
media boundaries, calling into question the validity of such commonly
accepted notions as, for instance, ‘literature’, ‘music’ or ‘art’. Apart from
drawing attention to the artificiality and arbitrary nature of media defi-
nition and classification, this slogan also contains an inherent notion of
intermediality, of a reciprocal relationship between different media, as we
conventionally understand them. The question of what a certain medium is
always depends on the conception, the ‘imagination’ of the beholder – be it
the recipient or the medium itself. For not only do we as scholars, performing
artists or audience have a specific understanding of what we recognize to be
a certain medium, but also the medium itself intrinsically incorporates and
conveys its ‘image’, its version of mediality, either by self-referentially and
self-reflexively pointing to its own medial status, by referring to other media
and thus determining its own medial boundaries, or simply by displaying
performative features of its own mediality in dissociation from other media.
In the case of intermedial ‘imagination’, a medium defines itself by its own
potentials, but also with regard to its limitations. Especially when turning
to another medium, it automatically embarks on a dialogical contemplation
about possible borrowings, impulses and insights which could contribute to
enhancing its own (limited) expressive power.
With the Modernist movement, self-reflexive and experimental tenden-
cies increasingly occurred in all media. Ezra Pound’s appeal to ‘MAKE IT
NEW’2 is paradigmatic for the general Modernist quest for revolutionizing
the arts by leaving trodden paths, dispensing with outdated aesthetic models
and venturing into new expressive fields. One of the crucial elements in the

163

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


164 Mediations and Transformations of Media

process of the revolution of the arts at the turn of the twentieth century was
the questioning and expansion of established media boundaries. More than
ever before media turned to other media, assimilating media-external tech-
niques and qualities into their own medial realm. In fact, Modernist artists
were particularly drawn to the musical medium. Walter Pater’s famous asser-
tion that ‘all art constantly aspires towards the condition of music’3 reflects

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
a common tendency among Modernist writers, but also painters, to turn to
music as an aesthetic model.4
In this essay I will discuss a very specific case of intermedial ‘imagina-
tion’ on the basis of a musico-literary experiment of American Modernist
poet Amy Lowell. I will show how Lowell’s poem ‘Stravinsky’s Three Pieces
“Grotesques”, for String Quartet’ takes up a particular piece of music and
verbalizes it within the poetic medium.5 By ‘imagining’ its own mediated
version of another medium, the poem engages in what I call ‘intermedial
translation’ and thus creates not only a concept of the medium music, but
at the same time also self-reflexively redefines, or ‘imagines’ its own medial
condition.

As Jane Ambrose reveals in ‘Amy Lowell and the Music of her Poetry’,6
Amy Lowell held a lifelong interest in music as poetry’s ‘sister art’. Sev-
eral of Lowell’s poems either allude to music or display musical analogies
on a structural level. In 1915, Lowell heard Igor Stravinsky’s ‘Three Pieces
“Grotesques”, for String Quartet’7 performed in concert in Boston, which
inspired her to convert an entire work of art from one medium into another,
thus performing an act of intermedial translation. This transformatory pro-
cess falls under the category of what Lars Elleström calls ‘mediation and
transformation of media’,8 but has also been described by terms such as
‘intermedial transposition’,9 ‘Medienwechsel’,10 adaptation, transposition d’art
and so on.11 This example of intermedial translation corresponds to the
notion of ekphrasis, as it is a ‘verbal representation of a real . . . text composed
in a non-verbal sign system’.12 However, in this context I would like to use
the term intermedial ‘translation’, for it emphasizes the tension between the
notion of accuracy on the one hand and creative originality on the other
hand. Furthermore, the idea of ‘translation’ implies a perspective that is not
restricted to the semiotic level, in terms of a simple shift between different
sign systems. As with any translation process, intermedial translation also
has considerable cultural and aesthetic implications resulting from the inter-
action of two culturally and historically embedded artefacts.13 As I will show,
Amy Lowell not only transgresses medial boundaries by referring to and
engaging with another medium. She also raises our awareness with regard
to the very translatability and exchangeability between different media by
highlighting at once transmedial relations as well as media-specific distinc-
tions and divergences. The concept of intermedial translation will thus also

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Regina Schober 165

serve to reflect upon general processes of transfer and appropriation between


the two media: music and poetry.

Processes of translation undeniably played an important role in Modernist


literary production. Modernist writers not only produced a great number of
translations, they also frequently drew on foreign, often classical, writings

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
which were integrated into their own texts, either as translations or in their
original form.14 The practice of translation did not solely serve hermeneutic
purposes, but instead participated in the Modernists’ experimental urge to
employ alternative materials in a general rewriting process. In Modernism,
translation was no longer regarded as a purely functional and inferior form
of writing, but rather attained the ‘status [of] a primary mode of Modernist
literary production in its own right’.15 Along the lines of T. S. Eliot’s notion
of tradition and originality, translation functioned ‘as a kind of dynamic
procedural lens through which the Modernists could at once view both the
past as well as other cultures and . . . focus their images of these traditions in
their own times’.16 Obviously, translation became an integral part of Mod-
ernist literary practice, in that it transformed already existing texts, which
Eliot calls ‘tradition’, into ‘the new’.17 Translation thus grew to be an inte-
gral part of the original and creative writing process, while the boundaries
between original and translated text became increasingly blurred.
Modernist writers held the strong view that only through an active
engagement with the ‘other’ texts and media could the ‘self’ be compre-
hended and developed. In his famous essay ‘The Task of the Translator’, Wal-
ter Benjamin describes the aim of a good translation to ‘express the central
reciprocal relationship between languages’.18 In highlighting the reciprocity
of the translation process and the repercussions which the translation has on
the original text by giving it an ‘afterlife’,19 Benjamin’s considerations pro-
vide a valuable basis for the question of intermedial translation processes.
Although translation studies have conventionally been concerned with writ-
ten texts only, the concept of translation could as well prove suitable for the
context of intermedial relationships. If translation is regarded as a trans-
formation of a pretext of a particular system into another text or system,
certain intermedial processes could be considered as just another form of
‘translating’ from one medium to another.
For a long time, ‘translation’ in the literary or linguistic context has com-
monly only been associated with the transfer of a text from one language
to another.20 According to the definition in the Oxford English Dictionary,
there is, however, already an indication of the notion of ‘intermedial trans-
lation’ in the nineteenth century, when ‘translation’ is referred to as ‘the
expression or rendering of something in another medium or form, e.g. of a
painting by an engraving or etching’.21 This early use of the concept of inter-
medial translation has been taken up by Roman Jakobson in his essay ‘On

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


166 Mediations and Transformations of Media

Linguistic Aspects of Translation’, which differentiates between three types


of translation; ‘intralingual’, ‘interlingual’ and ‘intersemiotic’ translation.22
While only the second type corresponds to the commonly accepted notion
of translation between different languages, the third concept, ‘intersemiotic
translation’, implies the translation between different semiotic systems, for
example, between written texts and film or music. Even though Jakobson’s

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
‘intersemiotic translation’ is somehow restricted in that the source medium
is always a written text, we owe it to him that the concept of translation is
extended from its strictly linguistic to a broader semiotic context, including
transformations and intersections not only between different languages but,
more generally, between any different sign systems.
Recent translation studies have increasingly followed this expanded and
often metaphorical understanding of translation, disassociating it from its
strictly linguistic meaning and instead concentrating more on the univer-
sally and cross-disciplinarily applicable transformational processes involved.
As maintained by Mieke Bal and Joanne Morra, ‘today translation is gaining
ground as a crucial trope, idea, concept, metaphor and mode of interpre-
tation’.23 In accord with Shirley Chew and Alistair Stead, who understand
translation ‘in its transferred or metaphorical senses, most fundamentally as
a process of change or a passage from one state to another’,24 I would like to
place the idea of translation, ‘translate’ it, into the context of intermediality.
Intermedial translation25 , as I understand it, can thus be regarded as a generic
term for various different expressions which have been applied to describe
the processes of change from one (medial) state to another, to use Chew
and Stead’s turn of phrase, such as ‘transposition’, ‘transcription’ and ‘adap-
tation’. Whether the aim of intermedial translation is to make the original
medium ‘readable’ or to try experimentally to make a text fitting for another
medial context is a matter that is still open to debate, as is the question of
the degree to which the transformation process also implies modification
and alteration, and to what extent intermedial translation is either total or
partial.
The fact that a translation, as an original artwork in itself, is always a modi-
fied ‘version’ of the original, containing features quite distinct and divergent
from the reference medium, applies, of course, all the more to the case of
intermedial translation. Even if the target medium employs trans or interme-
dial components which let it appear like the original medium, there is always
a certain degree of modification and change. Not the medium itself, but an
‘imagined’ version of the medium, realized by means of another medium, is
the outcome of the intermedial translation process.

In order to elucidate further the concept of intermedial translation, I will


illustrate my theoretical considerations by means of a musico-poetic tran-
scription by Amy Lowell. The musical pieces she refers to, Igor Stravinsky’s
Three Pieces for String Quartet, were written in 1914 and were originally titled

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Regina Schober 167

‘Dance’, ‘Excentrique’ and ‘Cantique’. The pieces were supplemented by a


short introduction written by the composer himself, which was read out
before the performance and which most likely formed the basis of Low-
ell’s poem. The introduction described the setting of the pieces as a peasant
dance, a grotesque Pierrot scene and a tragic funeral service.26 Although an
approximate narrative thread is discernible throughout the three pieces, they
can well be regarded individually.27 In this context, I will mainly restrict my

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
analysis to the first of the three pieces.
If we take a closer look at the different translation processes involved in
this musico-poetic experiment, it is especially striking how accurately the
poem takes up the music and represents it by both ‘explicit’ and ‘implicit’
intermedial references.28 Lowell both imitates the music with regard to its
compositional material and describes her sound impressions by adding a
semantic sphere to the music. The vivid sound imitations, descriptions and
thematic associations are all the more remarkable, since Lowell must have
been able to listen to the music just once.29 In her poetic transcription of
Stravinsky’s music, Lowell retains the overall title of the pieces,30 as well
as its three-part structure, although she interprets the three autonomous
pieces as movements of a larger whole by grouping them together. The first
of Stravinsky’s Three Pieces, originally entitled ‘Dance’, calls to mind a folk-
loristic peasant dance, first and foremost evoked by the seemingly ‘simple’
melody with its relatively small pitch range and its repetitive and cyclical
structure. The first part of Lowell’s poem takes up this thematic setting of a
peasant dance in a ‘market-place’ (line 11) with ‘sabots slapping the worn,
old stones’ (line 12), articulating a potential program of the music on the
basis of very accurate aural impressions.
The kinetic nature of the piece is supported by its rhythmic layout which
is characterized by the recurrence of extremely regular ostinati which are,
however, set against each other and thus create rhythmical displacement
and instability. Lowell’s poem captures this sense of rhythmic disruption,
which is clearly perceivable in the music, by bringing up the image of bulky
sabots dancing on cobblestones: ‘Clumsy and hard they are, / And uneven, /
Losing half a beat’ (lines 15–17). Written in free verse, the poem is liberated
from regular meter, so that particular rhythmic elements of the music can be
imitated.31 The basic rhythmical unit of the music, the alternation of a single
quarter and double eighth note, is reflected in the repeated words ‘sharp
(and) cutting, sharp (and) cutting’ or, more vividly, in the onomatopoetic
‘Bump! Bump! Tong-ti-bump!’ (line 7), and ‘shuffle, rap’ (line 23).
While Lowell’s poem clearly suggests a peasant dance scene, Stravin-
sky’s piece, with its lack of clear structure and melodic circularity, seems
almost constructed. Rainer Sievers is right in observing a certain mechani-
cal quality about the piece.32 Like a perpetual motion machine, the melody
could continue endlessly, repeating the patterns over and over, until it sud-
denly comes to a halt in a rather abrupt manner.33 Obviously, the artistic

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


168 Mediations and Transformations of Media

alienation of a traditional folk dance is as evident as its imitation. Stravinsky


does not attempt to reproduce authentic folk music, but by combining it
with new compositional techniques, he treats it in an experimental way in
order to create a new, idiosyncratic means of expression in line with his
Modernist aesthetics. By employing folk elements, but at the same time
dissociating himself from these influences, Stravinsky performs a typical

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Modernist move. Furthermore, through his alienation of the traditional
material towards a mechanical and machine-like gestus, he establishes a link
between artistic modernism and the notion of ‘modernization’ in terms of
technological and industrial progress.

Stravinsky’s experimentalism also shows in his unconventional treatment of


tonal colour. The traditional chamber music formation of a string quartet is
transformed into a folk band, in which the typically delicate string instru-
ments produce harsh and vulgar sounds. The almost excruciating crudeness
of the sounds are described by Lowell as ‘sharp and cutting’ (line 4), probably
particularly referring to the dissonant minor ninth played in the opening by
the viola, portrayed as a ‘screeching thread’ (line 3) which is so discordant
that ‘it hurts’ (line 5). Following Stravinsky’s detailed performing instruc-
tions such as ‘sur le sol’ (on the G-string), ‘glissez avec toute la longueur de
l’archet jusqu’ à la fin’ (use the entire bow) or ‘au talon’ (at the frog), the
instruments generate a breathy and at times raucous timbre. The crudest
sound is presented by the second violin. The four notes, which display the
loudest dynamics (ff ) of the piece, are again played ‘sur le sol du talon’ and,
additionally, extremely secco (‘excessivement sec’), a sound effect created by
the sequence of four up/downbows and staccatissimo accents (wedge-shaped
accents as opposed to the dotted accents, such as in bars 5/6). Taken as
a whole, the sound quality of the piece is dominated by rough noises, as
Rudolf Stephan observes:

Already the complexion is singular: Noises such as glissandi, harsh grace


notes, scratching at the bridge, playing at the frog, striking the string with
the wood, pizzicato etc. are predominant. This prominence of noises has
its function: the compositional detail is not to be lost in the comforting
sound of the quartet.34

The prevalence of the compositional material, indicating a high degree of


self-referentiality, is certainly characteristic of Stravinsky’s avant-garde aes-
thetics. The music’s harsh and unrefined sounds, however, also serve to
represent the allegedly primitive and unrefined sound quality of peasant
music which is portrayed in the piece.
While the sound descriptions in Lowell’s poem start off as rather accurate
attributions to musical instruments, they increasingly develop into rather
free associations regarding the market place setting. There are ‘thin Spring

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Regina Schober 169

leaves’ that ‘shake to the banging of shoes’ (lines 20–21) as well as ‘little
pigs’ voices’ (line 25), obviously an association evoked by the croaking and
grunting sound of the second violin.

Given the precision with which the poem represents the music, would it
be right, however, to speak of Lowell’s poem as a mere replica of the music

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
in another medium? Does the poetic adaptation simply involve a one-to-
one reproduction without any degree of originality? Certainly not, even
if Lowell, with regard to this translational experiment poses the question:
‘Could I reproduce the effect of the music in another medium?’35 In fact,
this question provides a first clue. The reproduction of the music itself is
not implied, but only the reproduction of its effect. Verbally imitating music
by means of employing the transmedial features of rhythm and sound, the
poem virtually mediates between the two art forms by taking up elements,
or, according to Elleström,36 medial modes which are innate to both media
and thus bridge the narrow gap between the already closely related art forms.
Elleström’s assertion, that ‘media are both different and similar’, is perfectly
illustrated by this example. The poem abounds with musical effects – in this
regard it blends in with many other Modernist poems. On the other hand,
however, the poem does not become music, nor can it be merely reduced
to its experimental quality of transforming sounds into words, since it is
important to maintain that intermedial translation, as I have claimed above,
includes more than the structural and formal, but also touches upon on the
cultural and aesthetic realm of a medium.

Interestingly, Lowell’s poem refers to a contemporary piece of music by a


composer who was associated with the European avant-garde. Stravinsky’s
pieces must have seemed extremely modern, unconventional and eccentric
at the time, especially to the rather conservative audience of the Boston
cultural elite. Taking up this particular piece of music, the poem thus self-
reflexively positions itself in an avant-garde context. Besides, the poem takes
up certain cultural themes which are inherent in the music or its titles and
which are then developed and elaborated upon by means of semanticiza-
tion and cultural association. By turning to what is considered European
Modernism, the poem consequently not only engages in intermedial trans-
lation, but at the same time also in a form of cultural translation. As has
already become apparent in the analysis, one of the culturally relevant top-
ics involved in both the music and this part of the poem is the discourse on
folklore and primitivism in European and American Modernism.
Stravinsky overtly employs compositional devices associated with folk
music. The notion of a peasant dance, which is evoked by both the origi-
nal title ‘Dance’ and Stravinsky’s introduction, is thus also inherent in the
compositional structure of the music, however alienated by Modernist com-
positional techniques. Stravinsky uses folkloristic elements not mainly as

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


170 Mediations and Transformations of Media

a source of compositional material, but possibly to try to revive facets of


a departing culture of the ‘Old Russia’, which he expected to find in the
allegedly primitive Russian folk music, while at the same time embedding it
within his experimental compositional practices.37 Elements of primitivism
also enter Lowell’s poem. The depiction of the folk dance suggests not only
an atmosphere full of different noises which appear simultaneously and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
thus evoke a chaotic scenery, but also give the impression of sensual and
physical ecstasy. Allusions to ‘delirium flapping its thigh-bones’ (line 33),
for instance, or ‘coarse stuffs and hot flesh weaving together’ (line 39) cre-
ate an unrefined, sexual and almost vulgar atmosphere. The scene depicted
expresses a high amount of energy, caused by the rhythmic use of words
and the dynamic imagery employed. Not only the sexual imagery, but also
the prominence of colours, such as ‘red, blue, yellow’ (line 34), blurred in
a delirium-like state of ‘steaming’ ‘drunkenness’ (line 35) until they appear
again in a different order ‘red, yellow, blue’ (line 36), add to an atmosphere
frequently associated with a primitivist setting in literary modernism.
That Lowell was familiar with the primitivist discourse of her time
becomes obvious in her essay, ‘Some Musical Analogies in Modern Poetry’:

It is obviously impossible to go back to the beginning of any art. That


rhythm was the starting place of all, seems, however, practically indis-
putable. It would appear as though the more simple rhythms should have
developed first, but that is lost in the mists of time. And it is a strange fact
that now the savage or semi-civilized races employ exceedingly subtle
rhythmic effects. . . . the complicated syncopation of the American negro
has captivated the world under the vulgar and misleading name of rag-
time. That he never learnt this from us is plain . . . At what period did
civilized man lose this power of retaining psychological beats in his head
without necessarily expressing them?38

This passage shows that Lowell, not being free from the prevalent racial ide-
ologies of early Modernism, links her concept of the ‘primitive’ with the
idea of a primal and an instinctive sense of rhythm, which has been lost in
the civilization process and which has to be retained and rediscovered. The
inherent irony of primitivism, the paradox between idealizing the ‘primi-
tive’ as a means of criticizing one’s own society while on the other hand
establishing and maintaining the border between ‘us’ and ‘them’ can well be
observed in Lowell’s rhetoric, which draws a clear line between the ‘Ameri-
can negro’ and ‘us’. Lowell’s fascination for the ‘primitive’ is not particularly
differentiated, for she subsumes various ethnic groups under the concept of
the ‘savage’; the Egyptians she encountered during a boat trip on the Nile,
the American Indian or the ‘American negro’. Because Lowell, as a Boston
Brahmin, had no direct affiliation with the supposed primitive element of
African-American culture, it was probably as ‘exotic’ and ‘foreign’ to her as
African or Chinese cultures.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Regina Schober 171

This is exactly where Lowell’s primitivism differs from Stravinsky’s interest


in folk music. Although both utilize ‘primitive’ elements as compositional
material, above all that of rhythm, their concepts of ‘primitivism’ are
grounded on totally different premises. Whereas Stravinsky uses materials
from the Russian folk tradition, belonging to his ‘own’ culture, Lowell has
in mind foreign, remote cultures, the ‘other’. Her poem ‘Stravinsky’s Three

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Pieces “Grotesques”’ does not make explicit references to the Russian cultural
context, but rather considers and portrays Russian folklore as one of many
‘primitive’ traditions. Russian folklore elements, which play a crucial part
in Stravinsky’s Three Pieces, turn into ‘primitive’ elements, which are almost
exchangeable, in Lowell’s text. While Stravinsky is interested in traces of a
particularly Russian culture, Lowell engages in a more universal search for a
primal state of being as a means of reinforcing modern civilization. Unlike
Stravinsky, she does not turn to the ‘primitive’ in order to return to her
own cultural identity, but rather to depart from it in favour of a turn to an
imaginary pre-civilized state of mankind.
To conclude, Amy Lowell’s poem ‘Stravinsky’s Three Pieces for String Quar-
tet’ exhibits processes of intermedial translation which engage both on a
formal and cultural or aesthetic level. The poem verbally represents a piece
of music by imitating and semanticizing it. Lowell’s translation not only
attempts an accurate representation of the original, but also foregrounds
the experimental borrowing of the material of the musical medium and
thus self-reflexively comments on its own aesthetics, while emphasizing
and appreciating its expressive potential. The poem also translates and thus
transforms the music’s cultural ‘meaning’. In the imaginative description of
the folk dance scene, Lowell creates her own response to the notion of the
primal inherent in Stravinsky’s music. The Russian folk element captured in
the music is translated into Lowell’s own cultural context and her particular
version of ‘primitivism’. However, the processes of intermedial translation
are not unidirectional. Not only can Lowell’s poem be understood as a poten-
tial program to the music, but the translation process itself elucidates and
reflects upon the very quality of mediality. By ‘translating’ a piece of music,
the verbal medium creates its own imagined version of the ‘other’ medium
and at the same time also of its own medial status. Thus, in translating from
music to literature, Lowell’s poem both overcomes and transcends medial,
cultural and aesthetic boundaries, but at the same time illuminates medial
differences, pointing to the very essence of their materiality.

Notes
1. ‘Imagine Media!’ is the title of the eighth conference of the Nordic Society for
Intermedial Studies, held at Växjö University (from 2010: Linnaeus University),
Sweden, 25–8 October 2007, where a preliminary version of this essay was first
presented.
2. E. Pound (1977) ‘LIII’ in The Cantos of Ezra Pound (New York: New Directions
Books), p. 53.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


172 Mediations and Transformations of Media

3. W. Pater (1910) The Renaissance: Studies in Art and Poetry (London: MacMillan),
p. 135.
4. For the influence of music on Modernist writers, see B. Bucknell (2001) Literary
Modernism and Musical Aesthetics: Pater, Pound, Joyce, and Stein (Cambridge: Cam-
bridge University Press). With regard to music’s impact on Modernist painting,
see K. von Maur (1999) Vom Klang der Bilder (Munich: Prestel).
5. A. Lowell (1955) ‘Stravinsky’s Three Pieces “Grotesques”, for String Quartet’, The

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Complete Poetical Works of Amy Lowell (Boston MA: Houghton Mifflin Company),
pp. 148–9.
6. J. P. Ambrose (1989) ‘Amy Lowell and the Music of Her Poetry’, The New England
Quarterly 62(1), 45–62.
7. I. Stravinsky (1922), ‘Trois Pièces pour Quatuor à Cordes’, F. H. Schneider (ed.)
(Edition Russe le Musique, printed by arr. Boosey & Hawkes).
8. Cf. Lars Elleström, in this volume.
9. W. Wolf (2002) ‘Intermediality Revisited: Reflections on Word and Music Rela-
tions in the Context of a General Typology of Intermediality’ in S. M. Lodato,
S. Aspden and W. Bernhart (eds) Word and Music Studies: Essays in Honor of Steven
Paul Scher on Cultural Identity and the Musical Stage (Amsterdam: Rodopi), pp.
13–34.
10. I. O. Rajewsky (2002) Intermedialität (Tübingen and Basel: Francke).
11. Cf. Irina Rajewsky, in this volume, for an overview of the terms describing
processes of media transformation.
12. C. Clüver (1997) ‘Ekphrasis Reconsidered: On Verbal Representations of Non-
Verbal Texts’ in U.-B. Lagerroth, H. Lund and E. Hedling (eds) Interart Poetics:
Essays on the Interrelations of the Arts and Media (Amsterdam: Rodopi), p. 26.
S. Bruhn (2001) discusses the term ekphrasis in the context of intermedial studies
in ‘A Concert of Paintings: Musical Ekphrasis in the Twentieth Century’, Poetics
Today 22(3), 551–605.
13. Recent studies of intermediality have increasingly focused their attention on
cultural implications and possible functions of intermediality. Significant contri-
butions to this field include E. Hedling and U.-B. Lagerroth (eds) (2002) Cultural
Functions of Intermedial Exploration (Amsterdam: Rodopi) and L. Eckstein and
C. Reinfandt (2006) ‘On Dancing about Architecture: Words and Music between
Cultural Practice and Transcendence’, Zeitschrift für Anglistik und Amerikanistik
54(1), pp. 1–8. Also see Jørgen Bruhn’s contribution to this volume for ideological
and cultural implications of intermedial studies.
14. S. G. Yao (2002) Translation and the Languages of Modernism: Gender, Politics,
Language (New York: Palgrave Macmillan), p. 5.
15. Ibid., p. 157.
16. Ibid., p. 7.
17. T. S. Eliot (1975) ‘Tradition and the Individual Talent’ in F. Kermode (ed.), Selected
Prose of T. S. Eliot (London: Faber and Faber), pp. 37–58.
18. W. Benjamin (2004) ‘The Task of the Translator’, translated by Harry Zohn, in L.
Venuti (ed.) The Translation Studies Reader (New York: Routledge), p. 17.
19. Ibid., p. 16.
20. The term ‘translation’ is also used in natural sciences, the science of religion and
astrology; however, in all cases the basic notion of the term is the transformation
of one state into another.
21. ‘Translation’ in Oxford English Dictionary, 2nd edn (Oxford University Press 1989,
reprinted 2007).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Regina Schober 173

22. R. Jakobson (2004) ‘On Linguistic Aspects of Translation’ in L. Venuti (ed.), The
Translation Studies Reader (New York: Routledge), p. 139.
23. M. Bal and J. Morra (2007) ‘Editorial: Acts of Translation’, Journal of Visual Culture
6 (5), p. 5.
24. S. Chew and A. Stead (1999) ‘Introduction’ in S. Chew and A. Stead (eds) Trans-
lating Life: Studies in Transpositional Aesthetics (Liverpool: Liverpool University
Press), p. 2.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
25. The term ‘intermedial translation’ has been used sporadically, for example, in the
context of visual culture by Bal and Morra and by Dick Higgins with regard to
sound poetry (‘A Taxonomy of Sound Poetry’). However, it has so far not been
applied to the musico-literary context.
26. According to a review of the concert by Philip Hale in The Boston Herald (3 Decem-
ber 1915), a text was read in connection with the pieces, which unfortunately
does not seem to exist anymore.
27. Although calling the pieces ‘Grotesques’, Stravinsky probably considered them as
individual works grouped together arbitrarily, for the orchestrated version of 1928
(Four Studies for Orchestra) contains an additional fourth piece titled ‘Madrid’.
28. Concerning the terms ‘explicit’ and ‘implicit’ intermedial references, see
W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History of
Intermediality (Amsterdam and Atlanta GA: Rodopi).
29. Since the contemporary pieces were played from manuscript by the players, Low-
ell did not have access to a score – the first printed edition was not published
until 1922.
30. Considering some of the ideas put forward by Axel Englund in this volume, the
explicit and exact reference to the music’s title could be regarded as indicative of
a metaphorical relationship between poem and music, since the poem represents
and thereby purports to be music, its medial ‘other’.
31. The direct imitation of musical rhythms corresponds with Lowell’s understand-
ing of poetic rhythm as set out in her essay ‘Some Musical Analogies’ (at p. 139,
see below note 34). Experimenting with the possibilities of free verse, Lowell
takes music, especially that of Debussy, as a model. Due to its neglect of sylla-
bles in favour of equally long time-units, free verse acquires a musical flow which
is not dominated by regular meters, but instead by larger time divisions. As a
result, single word lines, which are often onomatopoeic sound imitations, such
as ‘Whee-e-e!’ (line 12) are of the same length as lines with more words and sylla-
bles and thus create a musical resonance and reverberation. Due to their exposed
position and the combination of the voiced glide /w/ and the long, shrill /i/, they
have a similar effect as the accented quarter notes of the first violin, slurred into
an eighth note g (for example, bars 7/8). The rhythmical effect of free verse is
supported by a frequent use of sharp consonants, especially fricatives, affricates
and stops as in ‘screeching thread’ (line 3), ‘sabots slapping’ (line 13) or ‘shaking
and cracking’ (line 14).
32. R. Sievers (1996) Igor Strawinsky: Trois Pièces Pour Quatuor à Cordes: Analyse und
Deutung (Wiesbaden, Leipzig and Paris: Breitkopf & Härtel), p. 71.
33. At the end of the piece Stravinsky demands sans ralentir in order to make it
explicit that a ritardando is not intended.
34. R. Stephan (1972) ‘Aus Igor Stravinskys Spielzeugschachtel’ in L. U. Abraham (ed.)
Erich Doflein: Festschrift zum 70. Geburtstag (7. August 1970) (Mainz: Schott), p. 27.
35. A. Lowell (1920) ‘Some Musical Analogies in Modern Poetry’, The Musical
Quarterly 6(1), 148.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


174 Mediations and Transformations of Media

36. Cf. Lars Elleström, in this volume.


37. As demonstrated in R. Taruskin (1996) Stravinsky and the Russian Traditions:
A Biography of the Works through Mavra, 2 vols (Berkeley CA: University of Cali-
fornia Press), Stravinsky’s works are undeniably strongly influenced by Russian
folk music, although the composer himself denied any influence of that kind. His
explicit detachment from those influences might be explained with his antipathy
towards Red Russia and the socialist call for a nationalist art. Yet, according to

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Stephen Walsh, Stravinsky developed a ’sudden and intensive interest in folk
poetry’ (Walsh (1999) Stravinsky: A Creative Spring: Russia and France, 1882–1934
(New York: Knopf), p. 56) during his Swiss period, which not only affected his
Three Pieces, but also larger works such as Les Noces (1914 and 1923) and Renard
(1916). As a result of his involuntary exile, Stravinsky developed an interest
for his home country, which Walsh calls a ‘wartime quest for an idealized folk
modernism’ (1999, p. 537).
38. Lowell (1920), ‘Some Musical Analogies’, pp. 130–1.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


11
‘Transgenic Art’: The Biopoetry
of Eduardo Kac

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Claus Clüver

The various forms of ‘New Media Poetry’ documented and theorized in 1996
in an issue of Visible Language edited by Eduardo Kac1 included such forms as
digital poetry in a number of forms and genres (some of it interactive), other
forms of cyberliterature, videopoetry and holographic poetry (or holopo-
etry). The concept of ‘poetry’ involved in this practice and its theorization
expanded the convention of considering as ‘poetry’ all forms of manipula-
tion of and experimentation with the verbal medium and its written and
aural representations which dated back to the beginning of the twentieth
century and had come to be labelled as visual, concrete or sound poetry,
respectively. The Brazilian poet and artist Eduardo Kac (b. 1962) first gained
international recognition in the early 1980s with his computer-generated
holopoetry. Over the past 20 years, he has more radically explored the pos-
sibilities of contemporary media technology for artmaking, a development
documented in Telepresence & Bio Art: Networking Humans, Rabbits, & Robots
(2005), a collection of Kac’s essays published between 1992 and 2002,2 with
the essays grouped under the headings ‘Telecommunications, Dialogism and
Internet Art’, ‘Telepresence Art and Robotics’ and ‘Bio Art’. The most noto-
rious of his ‘bio art’ creations is the GFP Bunny (2000), the green fluorescent
rabbit Alba, a product of what Kac calls ‘transgenic art’ – ‘a new art form
based on the use of genetic engineering to create unique living beings’.3
The first of Kac’s ‘transgenic’ artworks, Genesis (1999), was also his first
‘biopoem’. Kac has written an essay4 detailing the conception and execution
of the work and reflecting on its implications. In describing it I shall have
to rely heavily on Kac’s own description. The work was executed in three
phases. The first led to the creation of an ‘artist’s gene’ – ‘a synthetic gene
that I invented and that does not exist in nature’.5 It involved the transfor-
mation of a verbal text into DNA. The text chosen was a statement by the
God of the biblical Genesis as rendered in English in the King James Ver-
sion of the Bible: ‘Let man have dominion over the fish of the sea and over
the fowl of the air and over every living thing that moves upon the earth’
(Genesis 1.26).

175

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


176 Mediations and Transformations of Media

The ironic appropriateness of choosing these lines to be transformed into a


gene is obvious. Kac has pointed out that the choice of a translation, instead
of the original Hebrew, and of the King James Version with its claims, its
complicated history and its ideological contexts, was deliberate. Next, the
letters of the alphabet were replaced by the symbols of the Morse code. Kac
explains that ‘Morse code was chosen partly because, as first employed in

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
radiotelegraphy, it represents the dawn of the information age – the genesis
of global communications’.6 I do not know what other code could have pro-
vided Kac with such a suitable base for conversion into DNA. Kac used the
Morse dashes and dots for two of the four letters in his own conversion code:
dashes represented the letter T (thymine), dots the letter C (cytosin); word
spaces represented the letter A (adenine), letter spaces the letter G (guanine).
The entire DNA resulting from this conversion represents the new synthetic
gene. Kac reports that he had to ‘e-mail the gene to a company specializ-
ing in DNA synthesis’ and two weeks later received a ‘package with a vial
containing millions of copies of the gene’. He realized that

by itself the gene cannot do anything because . . . to be meaningful it


needs a context. The context of the gene is the body of an organism,
and the context of the organism is its environment. In the case of my
Genesis, the organisms are bacteria . . ., and their environment is at once
their dish, the gallery, and the Internet.7

The Petri dish with the bacteria into which the Genesis gene was introduced
was displayed in a gallery. In fact, it contained two kinds of bacteria, both
genetically engineered to glow: one kind, which contained the synthetic
gene, emitted blue light, the other kind, which did not, emitted yellow light.
Spectators could not only observe these bacteria in their dish, they could also
turn on an ultraviolet light that would cause real biological mutations in the
bacteria.
The entire work exists as a multimedia installation in a blackened room
(see Figure 11). According to Kac, it employs

a petri dish with the bacteria, a flexible microvideo camera, a UV light


box, and a microscope illuminator. This set is connected to a video projec-
tor and two networked computers. One computer works as a Web server
(streaming live video and audio) and handles remote requests for UV acti-
vation. The other computer is responsible for DNA music synthesis. The
original music, which employs the Genesis gene, was composed by Peter
Gena. The local video projection shows a larger-than-life image of the
bacterial division and interaction seen through the microvideo camera.8

In the image of the installation reproduced below, the wall to the right of
the projection shows a large-scale painted inscription of the biblical text,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Claus Clüver 177

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Figure 11 Eduardo Kac, Genesis, 1999. Transgenic work with artist-created bacte-
ria, ultraviolet light, Internet, video (detail). Collection Instituto Valenciano de Arte
Moderno (IVAM), Valencia, Spain.9 Courtesy of the artist

opposite the projection is the text in Morse code, and on the left wall the
DNA. Even thus made visible and tangible through the employment of many
media, the project still seems to require a verbal narrative in the form of
annotations or a descriptive text to enlighten the viewer about the project
and the processes involved.
Kac reports that, at the end of the first exhibition of Genesis in 1999, the
gene ‘was decoded and read back in plain English’, but the mutations had
caused slight changes: ‘The mutated sentence read: LET AAN HAVE DOMIN-
ION OVER THE FISH OF THE SEA AND OVER THE FOWL OF THE AIR AND
OVER EVERY LIVING THING THAT IOVES UA EON THE EARTH’.10
There were two other phases to the project that I will not describe in any
detail here. One dealt with the protein produced by the synthetic gene. It
involved ‘the visualization of the three-dimensional structure of proteins
produced by sequenced genes’ and investigated ‘the logic, the methods, and
the symbolism’ of the process ‘as well as its potential as a domain of art
making’. The last phase ‘focused on giving tangible expression to important
aspects of the genomic and proteomic developments of Genesis’.11 A pair of
large black granite tablets contains the laser-etched texts that were painted
on the walls in the installation, as well as the inversion, with the mutated
DNA sequence on top, their Morse equivalent below and the changed ver-
bal text at the bottom. According to the artist, the ‘triadic configuration of
[these] Encryption Stones critically exposes the intersemiotic operations that
lie at the heart of the contemporary understanding of life processes’.12 There
were four additional sets of objects that made aspects of the project tangible
in more material form.13

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


178 Mediations and Transformations of Media

In his essay, the artist claims that

[a] critical stance is manifested throughout the Genesis project by follow-


ing scientifically accurate methods in the real production and visualiza-
tion of a gene and a protein that I have invented and that have absolutely
no function or value in biology. Rather than explicating or illustrating

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
scientific principles, the Genesis project complicates and obfuscates the
extreme simplification of standard molecular biology descriptions of life
processes, reinstating social and historical contextualization at the core of
the debate.14

Instead of entering into the ongoing discussion of the scientific and ethical
implications of this transgenic artwork and of the others Kac has created,
I shall briefly explore, in the context of the present project, the intermedial
processes in its constitution, among which transposition and transformation
are dominant. I shall focus on the first phase, and more specifically on the
processes leading to the Petri dish with the glowing bacteria. I shall consider
the ways in which the entire work as a set of intermedial transformations and
ongoing processes is communicated to the reader/observer, and its implica-
tions for the discourse on media and intermediality and for the notion of
‘poetry’.
The initial medium is verbal language as represented alphabetically in
writing, using modern spelling and no punctuation. The fact that the text
is a specific and familiar translation leads to reflections about the changes
of meaning that will have occurred in its transformations from its earliest
oral formulation and about other versions of this passage in the same lan-
guage and in others. These considerations will also involve the realization
that any other version would have led to a different DNA – and the ques-
tion whether this would have changed the final result in any significant way,
given the self-contained nature of the project. The slight changes of the orig-
inal text that would have occurred by the time of the re-‘translation’ of the
mutated genes would be different but of no imaginable consequence, espe-
cially since any decoding, including that of the original text, would always
have different and uncontrollable results, which would also depend on the
time allowed for the process of mutation.
The re-presentation of the alphabetic English version of the biblical text
in the signs of Morse code does not necessarily depend on a written text,
although seeing the letters will undoubtedly help; nor does the text in
Morse code need to be written out. In fact, the code was invented to send
telegraphic signals, electronic impulses received primarily as sound (and
replicable through knocking or other rhythmically produced sounds). How-
ever, the subsequent transformation into a genetic code, represented by a
string of capital letters of the Latin alphabet, is difficult to imagine without
the model of a visual version of the Morse text.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Claus Clüver 179

It is only when the three texts are seen on the walls of the gallery space
that we are physically confronted with the fact that writing is a medium
of its own, though usually intimately connected with verbal language. The
letterforms used for the verbal text suggest to the modern reader the era
when the translation was produced, and the font used for the DNA string will
be associated with contemporary technology. The visual version of the text

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
in Morse code of the very same text spelled out in letters, signals a change
in the production and means of communication: the verbal message is the
same, and there has been no change in alphabet, but the lines and dots and
spaces serve only to visualize the groups of electronic impulses representing
the letters conveyed over a distance. Moreover, as painted on the wall (not
shown in the illustration) and etched into the Encryption Stones, the visual
symbols do not indicate the spaces between the letters in the encoded text.
In my view the representation of letters in Morse code, even as visu-
alized for this project, does not constitute a form of writing. That may
sound like a precarious statement, because it may seem to question the
status of so much writing that occurs in electronic media and is perceived
in virtual space. Media poetry ‘takes language beyond the confines of the
printed page’, as Kac wrote in his ‘Introduction’ to New Media Poetry, ‘simply
because the textual aspirations of the authors cannot be physically real-
ized in print’.15 Nevertheless, in its concrete manifestations the physical
appearance and organization of letters and letter combinations and their
placement in (three-dimensional, dynamic) space are manipulated with
even greater abandon and inventiveness than previously happened in print
and manuscript and calligraphy, with transformations only now made pos-
sible by the new media, and approaching illegibility even more freely than
some two-dimensional experimental poetry. All these possibilities for manip-
ulating written signs that are inherent in any writing system and have been
explored for centuries in many cultures are independent of verbal language,
even when the writing serves to communicate a verbal text, and quite fre-
quently in order to enhance such a text. The choices of typeface, letter size
and visual arrangement of the biblical text above the Morse code version,
and the DNA string below it, are a modest indication of the effectiveness
of writing as a medium. The dots and dashes and spaces of the Morse code
version do not invite or even permit such manipulation; and while the phys-
ical (or, to use Lars Elleström’s term, ‘technical’) media may be changed,
the sequence of the signs cannot. Moreover, removed from that particu-
lar context, they fail to function altogether, while there have been many
stimulating visual compositions made of elements of writing without any
ostensible connection to verbal language, especially in the past hundred
years.16
In this reading, a written verbal text constitutes a combination of two
media, although this fact only becomes significant when the shape, size
and placement of letters attract attention to themselves as constituents of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


180 Mediations and Transformations of Media

the text’s meaning. Since the visual codes on which such a text draws are
not part of the verbal sign system, we would be reading an intermedia
text. Contrary to multimedia and mixedmedia texts, the media involved
in intermedia texts (for example, many logotypes and graffiti, concrete and
holographic poetry) are fused and not separated by any boundaries.17
All written languages are based on particular codes that have developed

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
over long periods of time. In alphabetic languages, the letters are written
signs that must be distinguishable one from the other but are independent
of the specific fonts or typefaces used in the production and manipulation
of ‘writing’. It is the alphabetic version of the biblical text, which can also be
spelled out orally, that serves as the basis of the text in Morse code. The text’s
transformation into a version in Morse code, which is entirely reversible,
does not affect its status as a verbal text at all, but it makes it difficult if
not impossible to read, especially as painted on the wall, without separation
of the coded letter signs. However, if properly transmitted by telegraph, the
signals will be received, and can be decoded and pronounced, as letters. It is
best to consider the Morse code as exactly that, and the transformation as
a switch of codes of the representation of alphabetical verbal signs, and not
as a change of media, even though the physical media involved are differ-
ent: the voice or visual signs in manuscript or print on one side, telegraphic
media for the text in Morse code on the other.
The next step in the sequence of transpositions, however, is entirely differ-
ent, even though its visual representation may at first sight show similarities
to the original written text. The uninterrupted chain of capital letters is
the result of an identification of the four elements of the visual version of
the biblical text in Morse code (the dots, the dashes, the pauses between
letters and the pauses between words) with the four elements that consti-
tute the genetic code. These elements have names, and the initial letters
of these names function, as capitals, to identify them. The capitals have
been assigned, apparently in an arbitrary fashion, to the four elements of
the Morse code, and the exact sequence in which these elements appear
in the transformed biblical text thus determines the DNA of the gene that
results from this process. While the DNA is represented in alphabetical form,
the gene itself is an entirely different medium. Is it, in fact, a medium
altogether?
Even though it was formulated over two decades ago, and in spite of the
current general agreement that any single-phrase definition of ‘medium’ is
bound to be inadequate, it may still be useful in this context to cite the def-
inition proposed in 1988 by Rainer Bohn, Eggo Müller and Rainer Ruppert,
which I have translated as: ‘That which mediates for and between humans a
(meaningful) sign (or a combination of signs) with the aid of suitable trans-
mitters across temporal and/or spatial distances’,18 for it provides a few terms
to be considered in our investigation. If we apply it to the artificial genes
contained in the bacteria in the Petri dish, our question about the medial

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Claus Clüver 181

nature of the gene does not find a straightforward answer. The blue bacteria
convey to us, as they were instructed to, that they contain the gene. They
do that both in the gallery and electronically whenever the work is installed
in a gallery and the installation is accessible via the internet, including the
possibility of clicking a button to turn on the ultraviolet light. The bacteria
might thus be considered suitable transmitters of a meaningful sign invisible

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
to the naked eye – genes constituted by the transformation of the original
text, in a process that can be reversed. Are the bacteria transmitters of signs,
or are they themselves the sign, the media configuration considered as a
‘biopoem’? If the latter is the case, then what is the medium in which they
can be configured as a sign?
The core of the actual work, the product of several transformative inter-
semiotic processes, consists in the synthetic genes inhabiting the bacteria
in the dish. However, it is crucial that the disk also holds bacteria that do
not contain the artificial gene, because the project requires the possibility
of mutations under the influence of ultraviolet light. The medial core con-
figuration, thoroughly manipulated and controlled, is thus the Petri dish
with two different-coloured sets of bacteria, one of which contains the arti-
ficial gene, accessible by the light. The medium enabling this configuration
is any suitable controllable environment where bacteria can interact under
certain conditions. Such an environment ordinarily does not serve to enable
the transmission of meaningful signs between humans – part of the defini-
tion cited above. Arguably it becomes a ‘medium’ only by being made to
function as an integral part of the Genesis project. The complex sign com-
municated by this particular laboratory setting is primarily self-referential: it
concerns the use of ordinary biological procedures to make us aware of the
possibility of genetic engineering through the creation of a gene resulting
from particular transformative processes, and through the consequent, less
controllable life processes resulting in mutations that can be demonstrated
by decoding. That is indeed all the re-translation can tell us, besides the fact
that the original text has been substantially preserved. The actual changes
that have occurred will inevitably make the text less intelligible, and their
extent, while otherwise uncontrollable, will depend on the time allowed for
the process of mutation to take place.19
The gene itself is entirely inaccessible to the gallery visitor. Its ‘meaning’ in
this specific case is completely artificial; in ordinary biological circumstances
the ‘meaning’ of a genetic code is understood and used in totally different
contexts. The alphabetical representation of the DNA, by itself unintelligi-
ble to the ordinary viewer, nevertheless makes it accessible in a different
medium – writing – which also mediates the verbal text from which it origi-
nated via its translation into Morse code (also contained in the display and
known to most viewers, at least in principle). However, its crude representa-
tion in a substantially different medium, alphabetic writing, indicates only
the first step in the process of the gene’s production. On the other hand, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


182 Mediations and Transformations of Media

fact that the reverse process of decoding will end up in a very similar chain
of capital letters that can then be re-translated, via the Morse code, into an
intelligible and identical (though slightly distorted) verbal phrase indicates
that the letter chain, though not a part of the genetic medium, is an integral
part of the constitutive transformative process.
Any understanding of the projected image of the bacterial interaction will

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
depend on an entirely different readerly competence. Again, most viewers
will bring some knowledge of these processes to the experience, in different
degrees; and these differences will be reflected in the readings of the entire
project that viewers will produce. A few may even come close to the artist’s
own assessment, who states emphatically that he is

interested in creating artworks that reflect on the multiple social implica-


tions of genetics, from unacceptable abuse to its hopeful promises, from
the notion of ‘code’ to the question of translation, from the synthesis
of genes to the process of mutation, from the metaphors employed by
biotechnology to the fetishization of genes and proteins, from simple
reductive narratives to complex views that account for environmental
influences. The urgent task is to unpack the implicit meanings of the
biotech revolution and, through art making, contribute to the creation
of alternative views.20

For viewers interested in questions of intermediality the large-scale projec-


tion of the bacteria in the Petri dish will raise a set of additional questions
and trigger reflections concerning the media-specificity of the enlarged liv-
ing organisms moving around and emitting the two kinds of light. As we
observe them, they are entirely the creatures of human imaginative activity
exploiting recent scientific discoveries. It is this knowledge, derived from an
external source, which makes us see them not only as biological specimens
but as elements of a media discourse. We know that they owe their exis-
tence to a profound shift of codes in which the original meaningful sign,
the verbal text, was obliterated to be reconstituted as a ‘biopoem’, a living
organism that yet contains in its genetic structure that very text, subject
only to the mutations provoked by bacterial interactions. While these inter-
actions can be influenced by human intervention (activating the ultraviolet
light), the poem-carrying bacteria in the dish constitute a medial configu-
ration whose behaviour follows its own rules or, as Kac put it somewhat
anthropomorphically, its own ‘interests’: ‘the biopoem’s internal interests as
a living creature (which are independent of you and me, or even of the ver-
bal components of the poem), i.e., the poem, being alive, has interests that
go beyond its status as a poem’.21 The poet offers another observation about
a consequence of transforming a verbal text into a living organism, this one
concerned with what Elleström (in this volume) calls the ‘spatiotemporal
modality’:

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Claus Clüver 183

There is also the fundamental question of what I call ‘biological time’,


i.e., contrary to the temporal structures already known in poetry (oral
performance, silent reading, simultaneities, recording and manipulation
of voice, use of video editing, etc.), the biopoem evolves according to its
own time, from a dynamic relationship between its internal metabolism
and its response to the environment, to environmental conditions,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
which include the care we give to it (or not). Biological time gives
the poem its own irreducible pace, following the rhythm of life as we
live it.22

If the bacteria in the dish constitute the actual biopoem, it is a text made
up by the fusion of at least two usually very distinct media and is as such
an intermedia text. One can argue, however, that what is really communi-
cated is the creative concept, especially since that which makes it a poem,
the presence of a radically transformed verbal text in the DNA (which can
nevertheless be recovered by a process of decoding), is invisible to the eye
(and thus, like other concept art, it raises questions about the ‘sensorial
modality’, to use another of Elleström’s terms). On the other hand, the
entire project is not only a multimedia installation appealing primarily to
the sense of vision and exchanging the materiality of the printed page
for the painted shapes on the walls, which thus appear materially simi-
lar to the projections of the enlarged image of the Petri dish, while the
technical media retain the most solid material presence. The materiality of
the work’s other aspects in the various media involved is also made tan-
gible by the additional objects created as part of the work but not of the
installation.
There is another medium involved in the work as installation: electronic
music, on which Eduardo Kac offers no further comments. It is apparently
generated by the DNA and does not exist independently.23 It most likely
does not directly and immediately affect our understanding of the trans-
formation project; but being surrounded by Peter Gena’s ‘gene sound’ will
certainly affect our experience, and on reflection we realize that the genesis
of a sonorous dimension in yet another medium is a further demonstration
of the interplay of natural language, genetics and binary logic that forms
‘the triple system of Genesis’, according to the artist, who sees it as ‘the key
to understanding the future’.24
At the core of the work is the act of what Irina Rajewsky, following oth-
ers, has called ‘medial transposition’: ‘the “original” text, film, etc., is the
“source” of the newly formed media product, whose formation is based
on a media-specific and obligatory intermedial transformation process.’25
Her classification of this as a ‘production-oriented, “genetic” conception
of intermediality’ seems particularly appropriate in this case, but the case
is also unusual, and not only because the target medium is not normally
involved in intermedial operations or an intermedial discourse. While in

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


184 Mediations and Transformations of Media

such media transpositions as film adaptations of literary texts, elements


of the source text will always be incorporated, more or less transformed,
into the new media constellation, in the case of Genesis the entire source
text, radically transformed, has been incorporated into the biopoem – in
fact, its presence makes it a poem, but also an intermedia text. As such,
it falls into Rajewsky’s second category of intermediality, that of ‘media

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
combination’.
Intermediality serves in this project as a bridge between usually unrelated
discourses. What enables the inclusion of the genetic medium in the game of
intermedial transformations, at least in this case, is the fact that the gene is
the carrier, and indeed a version, of a verbal text. Mimicking all genetic engi-
neering, the conversion of this arrogant ancestral self-definition attributed
to their god in an imaginative act that questions it by carrying the phrase
to an unexpected extreme can indeed be considered as another extension
of poetry. Kac told me again in 2007 that in the end he considers himself a
poet. In ‘Biopoetry’, a contribution to a book currently in press,26 Kac pro-
poses ‘the use of biotechnology and living organisms in poetry as a new
realm of verbal, paraverbal and nonverbal creation’ and outlines 20 projects,
including a few, like Genesis, already completed. Project number three reads
thus:

Marine mammal dialogical interaction: compose sound text by manipu-


lating recorded parameters of pitch and frequency of dolphin communi-
cation, for a dolphin audience. Observe how a whale audience responds
and vice-versa.

Here any kind of human interaction, a fundamental component in the def-


inition of ‘medium’ cited above, is at best secondary to the communication
among other mammals. If we again consider the project a specimen of con-
cept art where the idea is more interesting than its execution, we remain
entirely in the human sphere, of course. Whether executed or not, how-
ever, the substitution of any element of verbal language by human-induced
sounds of dolphin language takes the concept of ‘poetry’ well beyond the
presence and role of verbal language in Genesis and also beyond most if not
all boundaries of ‘poetry’ accepted in the work of experimental poets of the
last hundred years, from the inarticulate vocal productions of Dada perform-
ers and concrete ‘sound poets’ to the illegibility of certain printed texts or
calligraphic productions. What connects this particular project with such
radical versions of ‘poetry’ as well as some types of media poetry is the non-
metaphorical use of ‘language’, albeit that of non-human mammals. Poetry
can exist in many media, and its boundaries are constantly being redefined.
On the other hand, ‘biopoetry’ occupies only one area in the apparently
thriving realm of ‘bio-art’.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Claus Clüver 185

Notes
1. E. Kac (ed.) (1996) New Media Poetry: Poetic Innovation and New Technologies. Special
issue of Visible Language, 30(2). For the revised and expanded edition published
in 2007, Kac decided to drop ‘New’ from the title, ‘because now, ten years later,
digital and electronic media are no longer new in society in general or in poetry
in particular’ (E. Kac (ed.) (2007) Media Poetry: An International Anthology (Bristol,

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
UK and Chicago: Intellect), p. 7).
2. A collection of essays published between 1982 and 1988 in Portuguese appeared
in 2004 as E. Kac (2004) Luz & Letra: ensaios de arte, literatura e comunicação (Rio
de Janeiro: Contracapa).
3. E. Kac (2005) ‘Transgenic Art’ in E. Kac, Telepresence & Bio Art: Networking Humans,
Rabbits, & Robots (Ann Arbor MI: The University of Michigan Press), p. 236.
4. E. Kac (2005) ‘Genesis’ in Kac, Telepresence & Bio Art, pp. 249–63.
5. Ibid., p. 249.
6. Ibid., p. 251. In a long note, Kac added that he ‘employed Morse code not out of a
technical need but as a symbolic gesture meant both to expose the continuity of
ideology and technology and to reveal important aspects of the rhetorical strate-
gies of molecular biology’ (p. 261). Morse was an ardent nativist: ‘the nativist
platform was racist, anti-immigrant, anti-Catholic, and anti-Semitic.’ To Kac, ‘the
translation of the KJV Genesis passage into Morse code represents the continuity
from fierce British colonialism to the bigotry of nativist ideology’ (p. 261).
7. Ibid., p. 251.
8. The image was taken at the installation of Genesis in the Julia Friedman Gallery
in Chicago. An eight-page fold-out catalogue, Eduardo Kac: Genesis (May 4–June 2,
2001), contains an essay by David Hunt, ‘Eduardo Kac: Metaphor into Motif’, and
colour reproductions of all the elements of the work, including the text faces of
the two Encryption Stones.
9. Ibid., p. 251.
10. Ibid., p. 254.
11. Ibid., pp. 254–5.
12. Ibid., p. 256.
13. There is also Transcription Jewels, ‘a sculpture encased in a custom-made round
wooden box’ and consisting of a small glass bottle containing purified Genesis
DNA and ‘an equally small gold cast of the three-dimensional structure of the
Genesis protein’ (Kac (2005) Telepresence & Bio Art, p. 257). There is further a set
of Fossil Folds, a series of sculpted tablets based on Kac’s ‘artist’s protein’ (ibid.),
a five-page portfolio called The Book of Mutations and In Our Own Image, ‘a pair
of digital video-sculptures that present, respectively, moving images of Genesis
bacteria and the Genesis three-dimensional protein’ (ibid., p. 260). In 2001, all
these elements were brought together in the solo exhibition in the Julia Friedman
Gallery.
14. Ibid., p. 249.
15. Kac (1996) Visible Language 30(2), 98; reprinted in Kac (2007) Media Poetry, p. 11.
16. Examples of such compositions can be found, among many other sources, in
R. Kostelanetz (ed.) (1970) Imaged Words & Worded Images (New York: Outerbridge
& Dienstfrey); K. P. Dencker (ed.) (1972) Text-Bilder: Visuelle Poesie international:
Von der Antike bis zur Gegenwart (Köln: DuMont Schauberg); F. W. Block et al. (eds)
(1990) Transfutur: Visuelle Poesie aus der Sowjetunion, Brasilien und deutschsprachigen

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


186 Mediations and Transformations of Media

Ländern (Kassel: Jenior und Pressler); J. Peignot (ed.) (1993) Typoésie (Paris:
Imprimerie nationale).
17. My definitions of these categories were first published in Swedish in 1993.
They were restated in C. Clüver (2001), ‘Inter textus/inter artes/inter media’ in
M. Schmitz-Emans and U. Lindemann (eds) Komparatistik 2000/2001: Jahrbuch
der Deutschen Gesellschaft für Allgemeine und Vergleichende Literaturwissenschaft
(Heidelberg: Synchron Publishers) (in German) and again in C. Clüver (2008)

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
‘Intermediality and Interarts Studies’ in J. Arvidson, M. Askander, J. Bruhn and
H. Führer (eds) Changing Borders: Contemporary Positions in Intermediality (Lund:
Intermedia Studies Press).
18. R. Bohn, E. Müller and R. Ruppert (1988) ‘Die Wirklichkeit im Zeitalter ihrer tech-
nischen Fingierbarkeit’ in R. Bohn, E. Müller and R. Ruppert (eds) Ansichten einer
künftigen Medienwissenschaft (Berlin: Ed. Sigma Bohn), p. 10.
19. Gallery visitors and internet viewers can interfere in the process by switching on
the ultra-violet light; but it would be misleading to think of this as an interactive
situation, because these viewers do not respond to any sign coming from the dish.
Kac apparently attributes a greater significance to these viewer-induced mutated
texts: ‘The ability to change the sentence is a symbolic gesture: it means that we
do not accept its meaning in the form we inherited it, and that new meanings
emerge as we seek to change it’ (‘Artist’s Statement’ quoted by Jennifer Eberbach
(2007) ‘Eduardo Kac’ in B. Stirrat and L. Stephenson (eds) Human Nature. Exhi-
bition catalogue (Bloomington: Indiana University School of Fine Arts Gallery),
p. 66). Kac’s Genesis was shown as an installation in the SoFA Gallery’s ‘Human
Nature II’ exhibition, 9 February–9 March 2007.
20. Kac (2005) Telepresence & Bio Art, p. 255.
21. E-mail to the author.
22. E. Kac, e-mail to the author, 20 April 2009.
23. The 2001 catalogue of the installation in the Julia Friedman gallery notes on
p. 8 that ‘The Genesis Net installation has original DNA music by composer Peter
Gena’.
24. Kac likens it to the three writing systems recorded on the Rosetta stone, by which
Jean François Champollion, in the nineteenth century, produced ‘the key to
understanding the past’, because it made the Egyptian hieroglyphs readable (Kac
(2005) Telepresence & Bio Art, p. 254).
25. I. O. Rajewsky (2005), ‘Intermediality, Intertextuality, and Remediation: A Liter-
ary Perspective on Intermediality’, Intermédialités: histoire et théorie des arts, des
lettres et des techniques / Intermedialities: History and Theory of the Arts, Literature and
Techniques VI, p. 51; cf. Rajewsky, this volume.
26. E. Kac (forthcoming) ‘Biopoetry’ in S. A. Glaser (ed.) Media inter Media: Essays in
Honor of Claus Clüver (Amsterdam and New York: Rodopi), pp. 283–8.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


12
Photo/graphic Traces in Dubravka
Ugrešić’s The Museum of Unconditional
Surrender

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Katalin Sándor

In Dubravka Ugrešić’s novel, The Museum of Unconditional Surrender,1 the


reader ‘enters’ the text through a photograph, the single photograph of
the text: in different editions this can be found either on the cover or on
the first page of the book. What follows is a disjointed archive, the first per-
son narrative of a female Croatian intellectual in exile after the dissolution
of the former Yugoslavia, a memory-work about a dismembered cultural, his-
torical and geographical ‘body’, about personal and collective pasts.2 Ugrešić
herself retells in an essay3 the story of Simonides, the ‘inventor of memory’
from Cicero’s De oratore: during a banquet the hall collapsed on the guests,
disfiguring their bodies. Only Simonides was able to identify, to re-member
the unrecognizable fragments by using space as a mnemo-technical strat-
egy, remembering where the guests had sat at the table. Ugrešić’s novel itself
seems to work as a somewhat Simonidesian experiment, often appropriat-
ing photography, albums, collage, film, cards as ‘mnemonic’, (re)mediating
strategies for remembering the dismembered body of the past, which appears
not as something to be accessed but as something to be (re)constructed,
(re)assembled, interrogated. The wrecks of/from the past are recollected in a
text that remains unsettlingly dissected, traversed by white lines of interrup-
tion. In this process, photography is not primarily relevant as a document, as
an indexical evidence of/from the past (being often connected to oblivion4
and to questionable referentiality), but as an intermedial modality to frame,
thematize and/or critically reflect on identity, on the representability of per-
sonal and historical past, on remembrance and forgetting, on what Ricoeur
would call ‘the enigma of memory as presence of the absent encountered
previously’.5
The text of this novel is connected to photography in many respects:
(1) the photographic medium and the scopic experience is thematized
throughout the narrative, the text itself recurrently foregrounding

187

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


188 Mediations and Transformations of Media

photography and albums as its own self-interpreting, metadiscursive


patterns; (2) the single photograph of the book is rewritten and displaced in
two similar ekphrastic texts; (3) the narrator repeatedly adopts the position
and the lens of the camera as representational modalities, making textual
‘clicks’ and inserting ‘snapshots’ into the text. This is at times connected to
a narratorial attitude of limiting her role to that of a collector, withhold-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ing commentary and judgement, sketching a picture, inserting a quotation,
a piece from a story and leaving it to the reader (‘let him be patient: the
connections will establish themselves of their own accord.’6 ); (4) the text
interrupted by textual ‘snapshots’, by numbered fragments and quotations,
is not only a discourse on albums, but also an unfastened album-like arrange-
ment; (5) photography and albums are also intermedial practices in relation
to which historiographic and biographic representation are problematized
by foregrounding their mediated and mediating-constitutive aspect precisely
through the ‘objective’ of the camera.
Thematizing photography is also a way to expose the medial heterogeneity
of discourse. In this sense, the text might be called ‘photo/graphic’ – alluding
to Mitchell’s notion of the ‘image/text’ which designates ‘a problematic
gap, cleavage, or rupture in representation’7 , and ‘an unstable dialectic that
constantly shifts its location in representational practices, breaking both
pictorial and discursive frames’.8 In Ugrešić’s text (which might also be an
‘imagetext’ – as Mitchell9 understands it – combining text and photogra-
phy), the ‘photo/graphic’ seems to work as a gap or break in the text, an
interruption or an unsettling enigma retaining silence within discourse, or a
mediating modality that shapes remembrance and representation.
The photo/graphic could be an instance of intermediality as Paech defines
it: intermedial processes manifest themselves as events of difference, ‘as con-
figurations or as transformative inscriptions of mediality in a work, text or
intertext’.10 The in-between rhetoric of intermediality produces medial dis-
placements, figurations of medial difference and the ‘re-transcription of the
medium as form’ might insert ‘breaks, gaps and intervals’11 into represen-
tation. Thus the term ‘photo/graphic’ (written with a slash) does not refer
only to the single photograph in the book, but also to the way in which the
text (desires or) aims at approaching (and thus simultaneously displacing or
erasing) a medial ‘other’, as well as to the way photographic techniques of
seeing, framing, focusing, clicking become ‘readable’ as figurative, textual
transcriptions of the absent medium of photography or the absent pages
of albums. Paech distinguishes this kind of intermediality as ‘symbolic’ (as
opposed to ‘material’), since the medium is not present in another in its
materiality, but ‘on the level of depiction’12 , of textual (re)constitution. How-
ever, the novel which ‘cites’ photography verbally but also incorporates a
‘visible’ photograph, could be located at times in-between symbolic and
material intermediality.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Katalin Sándor 189

Alternating between temporal unfolding and the open, spatial


co-ordination of collage, this ‘mosaic’ or ‘album-novel’ can be read as
a discursive attempt to recollect the dismembered past, and (re)read the
present within historical and discursive conditions that do not offer any
reference point around which a coherent narrative or identity could be
constructed. The text on exile and displacement seems to be a text in

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
displacement as well: several fragments are dislocated as if in transit,
circulating and returning in the process of textual repetition. In this non-
identical recurrence, the narrator experiences different modes of identi-
fication and self-effacement in different geographical and socio-cultural
settings: Belgrade, Berlin, New York, Lisbon and so on. The process of tex-
tual, narrative displacements is often figured by ‘the map of the unreal’13 ,
by the interim dream-rhetoric of emigration, through the figures of hetero-
topic places like museums, mirrors, swaying houses, albums or the mutant,
divided city of Berlin. These also outline a non-linear, self-quoting and self-
inspecting discourse (at times more essayistic than narrative) in which some
short textual fragments seem to return as the traces, specters or echoes of
others.
A further metadiscursive pattern exposes the text as non-totalizing (and
‘visibly’ fabricated) discourse through ‘exhibiting’ the stomach contents of
Roland, a walrus formerly living at the Berlin Zoo. This unusual display,
which enlists the most common and yet the most unimaginable objects from
the pink cigarette lighter to a baby’s shoe, becomes a site of de and recon-
textualized ready-mades, reflecting on the divergent, collage-like process in
the (personal and collective) practices of remembering, forgetting, contex-
tualizing and meaning-making. As Rothberg notes about collective memory
(and memory in general), ‘Memories emerge in the interplay between dif-
ferent pasts and a heterogeneous present. I call this the multidirectionality
of memory: the interference, overlap, and mutual constitution of seemingly
distinct collective memories . . .’.14 The text abounds in sites and tropes of
heterogeneity, multidirectionality and collage: Roland’s exhibited stomach,
the Berlin flea-market, an artist’s studio, the German capital as a divided,
mutant museum city, the tarot cards through which the stories of the eight
friends and the eight different ‘histories’ of Yugoslavia are told, all scatter
any singular, reliable narrative about the past whose coherent representabil-
ity is contested throughout the text. Sontag claims that photographs expose
history, present and past, as a set of anecdotes and ‘faits divers’15 . In this
sense, the decontextualizing, fragmenting aspect and the contingency of
photography (and collage) might also be relevant from a historiographic
point of view: it questions the possibility and the politics of a master dis-
course, of an exclusive, hegemonic, uncritical and/or ethnocentric narrative
about the recent past of former Yugoslavia by leaving room for multiple
histories contesting and constituting each other.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


190 Mediations and Transformations of Media

A photograph at the centre?

In the processes of political, cultural and biographical dispersal, the narrator


seems to be fixated on an old photograph of three unknown women bathers
from the beginning of the twentieth century, an enigmatic image torn away
from any specific context. The only clue is that the photograph was taken
near the Pakra river, close to the narrator’s birthplace. The narrator takes the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
photograph everywhere and uses it in her intimate rituals of remembrance.
This fetishization of the photo, which may already mark a displacement
(being an exile’s surrogate for a loss), could possibly place it at the very core
of the text. An especially interesting approach to this novel, that of Ilinca
Iuraşcu, defines the position of the photograph as more or less central, as
if it were the generator of the narrative: a ‘pivotal element of the Museum’
around which the text ‘will gravitate’16 ; the ‘ “silence” of the image will be
interrupted within the narrative discourse and insistently counteracted by
the language of gendered self-representations’.17
Nevertheless from another point of view the photo can be just an image,
a recurrent one in the album-like arrangement of the text, a trace through
which the past might address the viewer and might itself be addressed. At
one point, the photograph is called ‘an old yellowing photograph’18 in a list
of other random bits of memory. The novel allegorized through the exhib-
ited stomach of the walrus – which frames and rearranges the contingent –
resists being read as a centred, hierarchical structure. The text evades any
centralizing reading, the photograph of the unknown swimmers ‘disappears’
and reappears among other memory-fragments, or in the ekphrastic tran-
scriptions which multiply its readability and (in)accessibility: it is a fetish,
but also a laterna magica of textual anamnesis, a context for (textually) expos-
ing other images, just an old yellowing photograph, or the only recurrent
(and ‘transportable’) reality in the continuous dislocations of exile.
When the narrator’s mother and her missing photographs are thematized,
we might recall Barthes’s Camera Lucida19 and the way Mitchell comments
on it. According to Mitchell, in Barthes’s text the missing and only textually-
recuperated photograph of the mother seems to be placed at the core of the
imaginary labyrinth made of all the photographs of the world; the text is a
thread that leads to the centre of the labyrinth, which is an absence, a lack,
a loss.20
With Ugrešić we also have absent, only textually-recuperated photographs
of the mother, who is very much preoccupied with rearranging the family
albums, the ‘archives’ of her own past. The constitutive aspect of remem-
brance is also relevant in the way in which the process of looking at the
past and present and anticipating a future is connected to different modal-
ities of identity-construction. The mother’s face and her photographs are
figured as sites for practices of identification (and sometimes anxieties), as
mirrors returning the narrator’s glimpse from the future, the glimpse of a

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Katalin Sándor 191

future (‘older’) self, a subject yet-to-be, splitting the sense of presence, one-
ness and sameness. The absence of the mother’s photograph is also linked
to the narrator’s reluctance to take pictures, to the perception of the photo-
graph as an act which prepares the body as a mummified trace – as Bazin21
would suggest – freezing the temporal existence of the subject into a still,
lifeless picture, into absence and ‘Death’.22 When the sick mother asks the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
narrator to take a picture of her, the latter is terrified by the thought that it
would be her last photograph.23 Through the lens of the camera she watches
her mother, a helpless prey, and is afraid of pressing the shutter to take a
picture, which could bring the loss of the mother by preparing for or mul-
tiplying her absence. The camera is conceptualized by the latent metaphor
of the weapon, as the lens of death, producing a picture and/but erasing a
subject. The desire to preserve the body as a photographic trace is overcome
by the anxiety that the photo will become a memorial image irrespective of
any intentionality that produced it.
Recurrent as they may be, in Ugrešić’s text none of the visual or textual
fragments can acquire the position that encapsulates all the others in the
rhetoric of reading: the exhibition of the stomach of the walrus as a metadis-
cursive frame encourages not the trope of the labyrinth with a centre (even
if that is an absence) but rather the pattern of collage in which the text does
not lead the reader on a trajectory like Ariadne’s thread; a collage with mul-
tiple nodes in which the old yellowing photograph and the absent photos
of the mother intersect with other texts or textually (re)constructed images.

Rewriting the ‘fetish’

The old yellowing photograph on the cover of the book is thematized several
times, in various allusions or in two almost identical ekphrastic transcrip-
tions. In both texts, the photograph is referred to as a ‘little fetish object’,24
and this is related to the way in which it is used by the narrator. In the the-
oretical discourses on photography there seems to be a kind of overlapping
in the conceptualization of this medium: one might notice the recurrence of
certain features, such as fetishizing fragmentation, metonymy, cut, rupture,
displacement, lack or absence. In Bazin’s realistic reading, the photograph
is conceptualized as an act of mummifying time,25 for Sontag photos are
memento mori or a ‘thin slice’26 of time and space, Barthes also relates it
to lack, absence, death and a missing mother,27 and Mitchell, commenting
on Said and Mohr (especially on the latter’s photographs about anonymous
Palestinian exiles), observes that ‘photography re-doubles the exile of image
from referent’.28
However, naming the photograph through the concept of the fetish is
also likely to invoke psychoanalytic, Marxist and feminist discourses on
fetishization. Relying on such discourses, Iuraşcu rightfully observes that the
photograph is the site for the fragmentation and fetishization of the female

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


192 Mediations and Transformations of Media

body, for its reduction to an ‘item of visual inspection and exchange’29 by


a phallocentric scopic regime, as well as a site for reifying gendered, racial,
national otherness within hegemonic power relations. Still – according to
Iuraşcu – photography can be read in a subversive way by incorporating
the image into a linguistic framework through commentary and caption.
This is how ‘the photographic image may be allowed to “speak”, ( . . . ) and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
therefore to do away with the confines of “silence”, “death” and “absence”
which have been shown to define the feminine subject within the phallocen-
tric scopic regime’.30 We can completely agree with Iuraşcu that we should
understand identity, memory, image and ideology in terms of technolog-
ical constructedness, which opens them up to contestation. Nevertheless,
she assigns the capacity of contesting, subverting the ideologically inscribed
photo only to language and discourse, and seems to preserve to some extent
the ideological pattern of dualistic oppositions in which the image is essen-
tialized as static, silent, ideologically-fixed, feminine and lacking agency,
whereas language is connected to voice, temporality, agency, intervention
and the capability of subversion. Such an approach might play down the
potentiality of photographs and albums to ‘intervene’ in the text as themes,
representational frames and gaps, textual ‘clicks’ or mutable snapshot-like
descriptions within a constant textual (!) displacement towards a medial
‘other’.

Photography and the photo/graphic: reading, looking


at and diving into the picture

Ugrešić’s text on the old photograph (and photography in general) outlines


certain ways in which the narrator uses the image, as well as the way her gaze
adopts the perspective of the camera-eye. In the narrator’s scopic experience
the photo is not reduced to a single (fetishizing) use: sometimes it is a ‘little
fetish object’ whose real meaning is not known, a piece of material which
keeps returning in the displacements of exile; sometimes it is an enigmatic,
seductive image, or a laterna magica for unleashing memory-work: ‘like a
lamp lit in a murky window, a heartening secret gesture with which I draw
pictures out of indifferent whiteness’.31
In the ekphrastic description,32 when the photo is ‘read’ as a picture, the
narrator tries to ‘decipher’ it by applying historical and iconographic pre-
texts and codes: she recognizes (through discursive knowledge) the four
gourds as old-fashioned swimming rings and sees the shape of the women’s
arms as wings. The photo seems to be read through the iconography of the
angel (the angel of history or that of oblivion), a recurrent trope in the text
which also echoes the reminiscences of Wim Wenders, Walter Benjamin,
Rilke and Klee, as well as allusions to Christian tradition. However, the con-
ceptual indeterminacy of the photograph as a kind of unsettling silence
cannot entirely be broken, it seems to resist utterance. At one point, the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Katalin Sándor 193

narrator enunciates (only) what is not there, what does not appear as an
attempt to preserve a gap, a void – namely, the absent, invisible something
the women seem to be waiting for: ‘Around them hovers an oniric haze full
of restrained light. They seem to be expecting something. For some reason
I am certain that what they are expecting is not the click of the camera.’33
The women’s look (‘expecting something’) undoes the frame of the pho-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
tograph and exposes the act of taking pictures as decontextualization, as
ripping out from a temporal and spatial continuum. The ekphrastic remedi-
ations of the photo might work in this way as never fully completed attempts
to reread, recontextualize the image and invest it with meaning. But just as
the women hide their hands in the photograph, something remains ripped
away, unsaid and unsettling, in excess to meaning or to the readable. This
may be somewhat similar to the way in which Barthes’s punctum34 or wound
in photography touches the viewer by eluding or disturbing its studium, its
culturally coded and intelligible meaning.
In the use of the photograph as a means for exposing different images
from or of the past, the photograph cracks and is looked through (and not
looked at) as a medium of anamnesis: the narrator’s gaze searches a temporal
and spatial fissure, ‘a hidden passage’35 to step out of linear or historical time
and homogeneous space into the multidirectionality of memory.
The narrator’s scopic experience is described not only as a self-interpreting,
self-inspecting, critical practice, but also as a self-effacing, immersive pro-
cess, a surrender to the image. This is articulated in expressions that make
the direction and the activity of the gaze indeterminate: ‘attracts my atten-
tion, hypnotically’, ‘stare at it’, ‘not thinking about anything’, ‘dive into
them’, ‘plunge attentively into’ and ‘slip’.36 Ambivalence also persists in
the way the photograph is dealt with throughout the text: aside from the
more theoretical-essayistic, sometimes even too-persistently self-reflective,
demystifying practice of thematizing photography, there is also a more ama-
teur, naïve, non-professional surrender to the seductive power of the oneiric,
enigmatic old photograph. The narrator admits that she has no rational
explanation for carrying this photo everywhere. She also makes a distinction
between photographic ‘amateurism’ and ‘professionalism’: the advantage of
the former over the latter ‘is contained in the point of indistinct pain, pain
which an amateur work (like extrasensory perception) can touch and thus
provoke the same reaction in the observer/reader’.37 This process of losing
one’s sense of self, of giving up to the (power of the) image, is somewhat
similar to Barthes’s approach in which he disclaims a more professional, sci-
entific, critical discourse in order to recuperate something from the magic of
photography38 through being touched, attracted, wounded by the picture.
However, the desire to dissolve the medium and immerse in some memory
behind it, is continually counterbalanced by multiple framing, by disclosing
the act of mediation: the photo placed in a window frame (‘in the left-hand
corner of the window, where the end of the lake can be seen’39 ) is exposed

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


194 Mediations and Transformations of Media

as a result of mediation and framing – a static, black-and-white square inter-


rupting the continuity of the moving image of the lake which is itself an
already framed, pre-mediated sight. The photograph as a requisite of exile
transforms shifting contexts and is transformed by them.
The oscillation between the (cultural) readability of the photo and its
unreadable wound-like, absorbing effect between its studium and punctum

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
might also be observed (in a different sense) in the way the narrator’s voice
continually alternates between a more critical, self-reflexive, distancing dis-
course exposing its own fabricated, ideological aspect (Was is Kunst? is a
recurrent question) and a more intimate, confessional, immersive way of
recounting the past.

Remediation, discursive remedy or the (critical) intervention


of the image?

Conceptualizing the photograph as fetish does not automatically position


the working of language as subversive and contesting, but rather as a dialec-
tical relation, in which the image can also elude or (actively) intervene in
verbal discourse. The narrator calls the photograph ‘a little fetish object’,
which might position it as a substitute for the loss in what she considers to
be the ‘neurosis’40 of exile. Elizabeth Grosz asks in Lesbian Fetishism what the
difference is ‘between the psychotic and the fetishist if both share disavowal
and a rejection of a piece of reality’.41 She concludes that the difference
‘seems to lie in the opposition between hallucination and substitution’42
as far as the absent reality is concerned. In this instance, the narrator who
uses the photograph as a fetish does not (cannot) hallucinate the presence
of a lost reality, but continually substitutes and displaces it in the process of
the desire to return. Or, as Edward Said notes in his comments on the pho-
tographs of Palestinian exiles: ‘You learn to transform the mechanics of loss
into a constantly postponed metaphysics of return’.43
The dialectical relation between photograph and text cannot be exhausted
by the discursive, ‘voiced’ deconstruction of the (fetishizing) photographic
representation. Iuraşcu’s most challenging approach seems to assign agency,
voice and deconstructive potential only to language and discourse: ‘the
death of the image allows for the emergence of narrative re/collection’44
and ‘the photograph gains a voice which speaks “in memories” and there-
fore disrupts the silence imposed by the camera eye’.45 In this interpretation
the critical, active, temporal aspect of speech, discourse and narration might
seem to be overemphasized as opposed to the so-called static, freezing, fix-
ating aspect of the (photographic) image which is unable to speak for itself.
Language is conceptualized not only in terms of remediation but rather as a
kind of remedy to the ideological constraints imposed by the photograph: it
is words and discourse that contest the reifying representations of the past,
the female body and the subject.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Katalin Sándor 195

However, from a different point of view, photography also seems to


‘intervene’ in discourse, sometimes even in a quite ‘visible’ way: the text
transcribes photographic techniques of image-making, often giving up nar-
rative linearity and inserting textual clicks and interruptions; it performs a
multidirectional memory-work which invokes the representational frame-
work of albums, museums, artistic installations, flee-markets and collage.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
The narrator’s gaze often appropriates the perspective of the camera in ‘tak-
ing’ textual pictures, exposing the medial awareness and the constitutive
aspect of the point of view and of framing, as well as the contextualizing,
mediating role of other pictures and texts (present and past) in the act of per-
ception and image-making. Thus the description of three women bathing in
the Adriatic in the narrator’s present is related to the photograph of the three
women from the beginning of the twentieth century: the depiction of the
scene is already ‘photo/graphic’, produced by an ‘inner click’46 through the
‘lens’ of the yellowing photograph. This textual snapshot produced through
the memory of a picture could be read in-between discursive and pictorial
frames: it produces a short, cut-out, mutable fragment in the text, the action
is freeze-framed by a click, and/but the words ‘click’ and ‘record’ are the
medial metaphors of the act of writing. Thus we cannot reduce the image–
text relations to the one-way appropriation, subversion and voicing of the
image by the text. The slash in ‘photo/graphic’ would signal a split within
discourse, a textual displacement towards a medial ‘other’, as well as the
modality in which the text transcribes photographic techniques figuratively,
producing and erasing the signs of their ‘presence’.
By citing the poetics of the album and photography, the text is cut up into
fragments, whose arrangement allows for exchangeability and fluid struc-
tures. Whether it is the self or history to be narrated, the text works as an
open, rearrangable archive. Being traversed by the white, unfilled lines of
interruption, it remains critical towards the concept of any homogeneous,
exclusive, self-sufficient historical truth. One of Iuraşcu’s most relevant con-
clusions is that ‘the “productive look” can act as a trigger for the resurgence
of “othered”, marginal hi/stories which have been “muted” by the discourses
of dominant ideologies’.47
The narration of the past displays – often through the absent lens of the
camera or through the poetics of photography and albums – an awareness
of its own partial, mediated and mediating aspect and also of the ‘plu-
ral’ realities it tries to recount. In Part III the eight friends’ stories named
after tarot cards suggest an understanding of history, of personal and col-
lective past in which different truths can be asserted simultaneously. The
non-hierarchical working of collage or the continually rearranged albums
and cards as strategies of relativization do not remove the problem of truth
(and, consequently, falsehood) from the scene, but necessarily multiply it
and expose it as relational, constructed and open to negotiation. Such prob-
lems are also addressed by contemporary theories of historiography, which,

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


196 Mediations and Transformations of Media

according to Gábor Gyáni, do not abandon the problem of distinguishing


truth and falsehood (this being related to the specific status of historiogra-
phy within other collective practices of remembrance), but accept the fact
that different truths can be affirmed simultaneously.48
The textual anamnesis, alternating between self-reflective, critical distance
and immersive proximity, makes collages and albums rather than unam-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
biguous judgements; it scatters the fragments of life-stories through collages,
albums and snapshots whose ‘silence’ may nevertheless ‘speak’ for itself. The
text does not complete a synthesis of heterogeneous memories; rather, it
struggles with something which might (still) be outside the representable.
Even if the narrator calls this something a ‘still verifiable’49 reality, she does
not transform the verifiable into an explanatory narrative, into ‘the story of
the local apocalypse’.50 If, in Mohr’s photographs ‘photography re-doubles
the exile of image from referent’51 , then in The Museum of Unconditional
Surrender (photo/graphic) anamnesis – which produces a sense of otherness
about the past – redoubles the exile of the remembering subject from her
memories, making it difficult, if not impossible, to feel ‘at ease, at home
(heimlich) in the enjoyment of the past revived’.52

Coda

The image–text relations in the text call for an understanding of interme-


diality in which the question of medial difference and aporetical tension
(between and within media) is not necessarily settled. Therefore, the present
essay does not attempt to straighten out the dialectical, equivocal rhetoric
of the photo/graphic image/text. Neither does it try to reinforce the ideolog-
ical opposition between a silent, static inert image and a temporal, voicing,
empowering discourse. Rather, it places the problem back into the alle-
gorical stomach of the walrus. Just like the ‘ready-mades’ in that unusual
collage, image and text may relate to each other in multiple and unpre-
dictable ways, and at the same time may intersect with other questions: the
different modalities of the subject’s (self)constitution and (self)effacement;
the discursive and medial conditions of reminiscence, of historiographic
and biographic representation; the contingency of photographic and lin-
guistic archives; the multidirectional and medially heterogeneous work of
remembrance. A non-totalizing (intermedial) approach should not nec-
essarily reconcile but rather enter the ‘image/text’ aporia and trace the
continuous dislocations of discursive and pictorial frames.

Notes
1. D. Ugrešić (1999) The Museum of Unconditional Surrender, translated by Celia
Hawkesworth (New York: New Directions).

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Katalin Sándor 197

2. Whether this text can be read as autobiographical – a possibility apparently ‘dis-


couraged’ in the text but, consequently, foregrounded as a problem – would be a
relevant question to ask, but it is beyond the limits of this essay.
3. D. Ugrešić (2001) ‘Elkobzott emlékezet’ [Forfeit Memory], Magyar Lettre Inter-
nationale, 40, http://www.c3.hu/scripta/lettre/lettre40/ugresic.htm, date accessed
5 May 2007.
4. Photographs not only ‘archive’ the past, but may also erase it: the framing act

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of photography also performs an exclusion, a rubbing out of what has been
excluded: ‘I tried to remember something else, but my memories stayed tena-
ciously fixed on the contents of the photographs’: Ugrešić (1999) The Museum of
Unconditional Surrender, p. 24.
5. P. Ricoeur (2006) Memory, History, Forgetting (Chicago and London: University of
Chicago Press), p. 39.
6. Ugrešić (1999) The Museum of Unconditional Surrender, unpaginated.
7. W. J. T. Mitchell (1994) Picture Theory: Essays on Verbal and Visual Representation
(Chicago and London: The University of Chicago Press), p. 89.
8. Ibid., p. 83.
9. Ibid., p. 89.
10. J. Paech (2000) Artwork – Text – Medium: Steps en Route to Intermediality,
http://www.uni-konstanz.de/FuF/Philo/LitWiss/MedienWiss/Texte/interm.html.
11. Ibid.
12. Ibid.
13. Ugrešić (1999) The Museum of Unconditional Surrender, p. 113.
14. M. Rothberg (2006) ‘Between Auschwitz and Algeria: Multidirectional Memory
and Counterpublic Witness’, Critical Inquiry 33 (Autumn), 162.
15. S. Sontag (1999) A fényképezésről [On Photography] (Budapest: Európa Kiadó),
p. 33.
16. I. Iuraşcu (2001) Re/collecting Gendered Memory: Photo/graphic Constructions of
Remembrance in Dubravka Ugresic’s The Museum of Unconditional Surrender and
Leslie Marmon Silko’s Storyteller. Master of Arts thesis (Budapest, Central European
University) p. 14.
17. Ibid., p. 3.
18. Ugrešić (1999) The Museum of Unconditional Surrender, p. 9.
19. R. Barthes (2000) Világoskamra. [Camera Lucida] (Budapest: Európa Kiadó).
20. Mitchell (1994) Picture Theory, p. 305.
21. A. Bazin (1967) What is Cinema? Vol. 1. (Berkeley CA: University of California
Press).
22. Barthes (2000) Világoskamra, p. 35.
23. Ugrešić (1999) The Museum of Unconditional Surrender, p. 20.
24. Ibid., pp. 4, 169.
25. Bazin (1967) What is Cinema?
26. Sontag (1999) A fényképezésről, p. 33.
27. Barthes (2000) Világoskamra.
28. Mitchell (1994) Picture Theory, p. 316.
29. Iuraşcu (2001) Re/collecting Gendered Memory, p. 5.
30. Ibid., p. 10.
31. Ugrešić (1999) The Museum of Unconditional Surrender, p. 173.
32. Ibid., pp. 169–70.
33. Ibid., p. 170.
34. Barthes (2000) Világoskamra, p. 31.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


198 Mediations and Transformations of Media

35. Ugrešić (1999) The Museum of Unconditional Surrender, p. 4.


36. Ibid., pp. 4, 169.
37. Ibid., p. 28.
38. Barthes (2000) Világoskamra, p. 23.
39. Ugrešić (1999) The Museum of Unconditional Surrender, p. 4.
40. Ibid., p. 113.
41. E. Grosz (1993) ‘Lesbian Fetishism?’ in E. Apter and W. Pietz William (eds)

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Fetishism as Cultural Discourse (Ithaca NY and London: Cornell University Press),
p. 108.
42. Ibid., p. 109.
43. E. Said, quoted in Mitchell (1994) Picture Theory, p. 314.
44. Iuraşcu (2001) Re/collecting Gendered Memory, p. 25.
45. Ibid., p. 26.
46. Ugrešić (1999) The Museum of Unconditional Surrender, p. 6.
47. Iuraşcu (2001) Re/collecting Gendered Memory, p. 26.
48. G. Gyáni (2007) ‘Hamis és igaz a történelemben’ in idem, Relatív történelem
[Relative history] (Budapest: Typotex Kiadó), p. 219.
49. Ugrešić (1999) The Museum of Unconditional Surrender, p. 194.
50. Ibid.
51. Mitchell (1994) Picture Theory, p. 316.
52. Ricoeur (2006) Memory, History, Forgetting, p. 39.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


13
The Dance of Intermediality: Attempt
at a Semiotic Approach of Medium
Specificity and Intermediality in Film

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Hajnal Király

During the centuries following the Italian Renaissance, numerous philoso-


phers, theologians, literary men and artists found it necessary to delimitate
poetic and visual arts, and, accordingly, to establish an accurate hierarchy
of them. Intriguingly enough, this comparative tradition has persisted after
the advent of the film, considered from the beginning – although pejora-
tively – a ‘mixed art’. The long-lived textual era, though it managed to level
the differences between different arts by imposing a universal terminology
and interpretation methodology (considering all works of art as simply texts,
that is, as readable sign systems), mostly provided close readings of isolated
texts, without attempting to place them in a wider, cultural and specific sign
system, characteristic for different arts or media. As Mitchell puts it in his
Picture Theory, the ‘pictorial turn’ has engendered

‘a postlinguistic, postsemiotic rediscovery of the picture as a complex


interplay between visuality, apparatus, institutions, discourse, bodies and
figurality. It is the realization that spectatorship (the look, the gaze, the
glance, the practices of observation, surveillance, and visual pleasure)
may be as deep a problem as various forms of reading (decipherment,
decoding, interpretation, etc.), and that visual experience or “visual
literacy” might not be fully explicable on the model of textuality’.1

There is instead – we could add with Murray Krieger – an increased preoc-


cupation with the socio-cultural-political subtext of works of art.2 Moreover,
the late twentieth century witnessed the overturning of the classical nar-
rative of art history (with, among others’, A. C. Danto’s subversive essays
on ‘the end of art’3 ). Different arts are no longer simply responding exter-
nal theories, recipes, discourses, manifestos, illustrations of what they then
become, but they tend to be the discourses themselves, and, most importantly
on themselves, their own mediality or/and intermediality. Marshall McLuhan’s
famous utterance ‘the medium is the message’ has never been this actual: the

199

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


200 Mediations and Transformations of Media

medium is not only mediating, holding the message, but it is the message (of)
itself.
Beginning with the 1990s, two powerful cinematographic trends have
addressed – from various institutional backgrounds – the problem of
medium specificity and intermediality in films: the so-called ‘writer’s
movies’4 and the contemplative, extremely slow-paced movies, mostly com-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ing from the Far East, defying all complex narratological accomplishments
of the film-medium. The writer’s movies, many of them from the popu-
lar and ‘midcult’ register – for example, Shakespeare in Love (John Madden
1996), Quills (Philip Kaufman 2000), Adaptation (Spike Jonze 2003), Tupsy-
Turvy (Mike Leigh 1999), A Cock and a Bull Story (Michael Winterbottom
2005) – represent writing and observing (spectatorship) as complementary
and reciprocally conditioning activities. We are often seeing the writer strug-
gling to write the story we are watching; or, conversely, we are witnessing the
effort of a director to mediate between the writer and spectators. This is no
longer simply a ‘self-reflective’, stylistically identifiable feature (formerly an
exclusive characteristic of arthouse movies), but a mediatic gesture of self-
awareness: both the writer and director are stepping over the limits of their
own medium to satisfy the expectation of their spectators. Film does not
need to define itself as ‘art’ anymore – the urge to delimitate itself from other
arts, to prove its representational competence has expired. Instead it regards
itself as a medium, inseparable – as W. J. T. Mitchell puts it in his Picture
Theory – from its socio-cultural, institutional background and foreground,
its spectators and the various discourses of spectatorship.5 This accentuated
self-awareness, which deals with its own intermediality or mixed mediality
(so ferociously attacked by the long lasting comparative tradition of ut pic-
tura poesis . . .) , is systematically overturning the strict delimitation between
literature and film along with the idea of conceptuality of the first and visual-
ity of the latter. In fact, the conceptuality of film has found defenders from
the very beginnings of its history (the avant-gardes, Russian film theories, the
Nouvelle Vague, the semiologic approach and, lately, the cognitive theory of
David Bordwell6 ) – but this time the ‘meaning making’ is often happening
(is being modelled) in the film diegesis itself.
The huge amount of visually extremely rich movies coming from the Far
East, with their intensely chronotopic imagery and slow-paced, minimalistic
narrative, on the other hand, continuously turn the ‘running time’ of the
narrative into ‘space’, a static, ‘plastic’ visual work of art: a picture (for exam-
ple, Zhang Ke Jia: Still Life, 2006). This is already an intermedial relationship
involving film and painting: we are watching the film as we contemplate a
painting, ‘scanning’ and ‘making meaning’ of all the details included in the
frame. These versions of intermediality in films – the self-conscious inclusion
of literary and painterly modalities – are not loudly proclaimed as they once
were in the form of manifestos, as with isolated works of Jean-Luc Godard
or Peter Greenaway, but appear as naturally integrated in films with the
most varied institutional backgrounds, belonging already to the everyday

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Hajnal Király 201

experience of the film medium. Films have become our medium, our ‘natural
environment’. This is partly due to the ‘nature’ of the medium: as Joachim
Paech puts it in his medium–form comparison, the medium is not observ-
able in itself, only the form is, and the medium appears in whatever form it
makes possible. ‘The medium can only be observed in its Other’. Its disap-
pearance aids the other’s emergence, in which it participates in a ‘parasitic

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
way’. He continues:

The only possibility to reach the medium behind the form consists in self-
observation of the observation and the re-entry of the medium as form
or as a back link, in which mediality as the constitutive difference in the
oscillation between medium and form becomes observable as the ‘para-
sitic third’, whose background noise renders the event of the difference,
thus, the message, perceptible and comprehensible.7

However, as already mentioned above, the ‘noise’ made by this parasitic


third, the medium or the ‘intrusion’ of another medium, is becoming less
and less perceptible: the ‘surprising’ forms of montage, superimpositions,
framing techniques or sound-effects are now common characteristics for
any film register – popular, midcult or arthouse movie. Instead, the film
medium tends to appear in the complete lack of these learned or techno-
logical features – or, according to Paech’s terminology, in the ‘breaks, gaps
and intervals’ of the form: extreme (narratological) minimalism, uncom-
fortably long shots and almost complete lack of dialogue. Together with
this obvious return to its origins – considered a sign of maturity by Rudolf
Arnheim and Erwin Panofsky, among others – along with a growing inter-
est in cultural and post-colonial studies, film is also acquiring an increased
socio-cultural responsibility. It is showing, unmasking, symbolically repre-
senting cultural, social, political reality. These minimalist movies rely on the
aesthetics of the frame, instead of that of the cut, and the changed role of
the spectator consists of ‘scanning’ and interpreting the signs and symbols
it contains. It is a more active form of spectatorship: a continuous effort of
meaning-making instead of losing ourselves in a perfect diegesis. We become
increasingly aware of the presence of the medium and our role as specta-
tors. The – almost forgotten – semiotic analysis appears to gain relevance in
the interpretation of these symbols, but, this time, it is strongly related to
cultural discourses of the film as medium. As Marc Laverette argues in his
contribution to Image and Narrative,8 the necessity of a discipline embracing
both semiotic and medium theory principles – a so-called ‘semiotic medi-
atics’ – has never been so actual. The delay of this merger is possibly due
to the fact that, with the emergence of medium theory, semiotics is already
considered exhausted and old-fashioned:

Medium theorists need to incorporate semiotics into their paradigm to


gain a respect for content and the overarching importance of meaning.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


202 Mediations and Transformations of Media

And while medium theory needs semiotics to better understand the signs
of life, semioticians need medium theory in order to better understand
the ‘allness’ of our signified environment.

Below I attempt a semiotic analysis, going beyond close textual reading,


of Hungarian director Béla Tarr’s adaptation of Satan’s Tango (1994).9 This

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
seven-and-a-half hour long film merges both trends mentioned above;
the writer’s movie, raising the issue of film–literature relationship and of
the adaptation, and the slow-paced, contemplative trend incorporating the
painterly tradition. Interestingly enough, in this ménage à trois or complex
intermedial relationship, the painting mediates between the other two: the
aesthetics of the frame, chronotopes such as the perspective, the circle,
the interior–exterior (house and road) opposition and the threshold, are all
transmediatic symbols which reconcile mediatic differences between litera-
ture and film. This approach, using Greimas’s semiotic square model, aims to
contribute to the methodology of analysing medium specificity and mixed
medium or intermediality, in the spirit of an old or new discipline: semiotic
mediatics.

Defining (inter)mediality: the tango metaphor

As often happens at the very beginning of a new theory or discipline,


metaphors fill the gap of a missing terminology: early film theory is crowded
with musical, painterly and architectural metaphors meant to grasp the
‘essence’ of the new medium. Similarly, intermediality has been described
as a ‘dialogue’, ‘in-between-ness’, ‘translation’, ‘oscillation’ or ‘flickering’
between the involved media. As we know from W. J. T. Mitchell, there are
always at least two media involved, for every medium is mixed by nature.
Béla Tarr’s adaptation raises a series of issues related to mediality, medium
specificity, limits and limitations of media, medium experience and interme-
diality, the boundaries of media involved in adaptations, challenging clas-
sical oppositions such as spatiality–temporality, or perceptual–conceptual.
Accordingly, the ‘tango’ or ‘dance’ metaphor from the title can be regarded
not only as a thematic, but also, at the same time, as an extended theoretic
figure. As a thematic figure, it interprets the hesitation of the characters to
step out from their lethargic environment (an abandoned farm) which keeps
them prisoner, and from the moral devaluation presented in the brughelian
dance party just before the devilish pact with the false prophet (another
possible meaning of ‘Satan tango’).
As a theoretical figure it presents, first of all, the experience of the medium
as a dance of the spectator. Satan’s Tango crosses the conventional, institu-
tional boundaries of the medium by exposing the viewer to a contradictory
medium experience: at the beginning, the low-key images and an extremely
slow camera movement make it difficult to watch, increasing the awareness

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Hajnal Király 203

of our physical reality and body; then there is a gradual absorption due to
the hypnotic pace – or dance – of camera movement. We forget about our
body, being here as spectators of the movie and then in-there, as one of the
film’s observing characters. We continuously step in and step out, engaged in a
tango between the film diegesis and our reality. The camera itself engages in
a continuous oscillation between stasis and slow movement, a tango which

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
emphasizes the spatio-temporality of the film medium. Moreover, due to its
unusual length, the film is screened with three intervals, and the screen-
ing room along with the entire institutional apparatus (a small art-movie
theatre in Budapest, which had Satan’s Tango on its weekly program) thus
becomes a ‘frame’, enabling us to step out to our reality, and then back to
this overwhelming medium experience. In this film, the strong presence of
the medium overflows the form and appears, as Joachim Paech puts it, as a
parasitic third, a noise disabling the creation of a perfect illusionary world.
The form processes are continuously interrupted by breaks, gaps caused
by the extremely long shots and slow camera movements. Moreover, our
motionless bodies in the dark screening room are perfect analogies of those
of the characters who are sitting in the dark interiors, staring helplessly at
the window, the only source of light.
Then, there is another dance: the film, as an adaptation of Laszlo Kraszna-
horkai’s homonymous novel,10 also marks a relevant mediatic turn, another
border-crossing, strongly connected to the socio-political changes induced
by the fall of the communist regimes in the Eastern Bloc. Krasznahorkai’s
acclaimed novel appeared in 1987, shortly before the events of 1989, and it
thematizes, among other things, the ultimate responsibility of the writer to
document the events witnessed. In the iconophobic era of communism, in
which all images of reality were banned except those that were constructed,
writing and literature became the medium able to confront the interdiction,
in this case disguised in a parable about a group of people forgotten on a
farm, waiting for the arrival of a false prophet, who then takes them to the
town and turns them into informers for the secret police.
Tarr was immediately interested in adapting the novel, but it took him
almost eight years to realize this project, partly due to a lack of institutional
and material support persisting after the change of the political regime. Turn-
ing the parable into the reality of film images, shot on an abandoned farm,
thus using the medium as an unveiling manifesto, Tarr was one of the first
film directors in Hungary to take a position towards the recent past and the
present. As a film, Satan’s Tango became as thoroughly intertwined with the
socio-political message of its time as had the novel. The medium is the mes-
sage: it mirrors the most actual preoccupations, needs and aspirations of its
time. The message is not only the content: it is identical with the inseparable
unity of form and content.
While the novel thematized the responsibility of writing, its adaptation
reflectively represents the observer, spectator and director. Moreover, and

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


204 Mediations and Transformations of Media

most excitingly, the emblematic positions of the writer and director, rep-
resenting the two different media, are here melded in the character of
the doctor. This proves to be a self-reflective gesture, which mirrors the
intentional annihilation of medium borders in this adaptation. The doctor
switches deliberately from observing or watching (not surprisingly, the only
point-of-view shots in the film are his) to taking notes. Watching or observ-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ing and writing or reading are shown as complementary activities: in this
respect, his name – ‘the doctor’ – (we know all the names of the other char-
acters) appears to be symbolic, as he is ‘healing’ the old aversions between
poesis and visual arts. This constant stepping back and forth or intermedi-
ality as a dance or tango is, at the same time, a figurative representation of
the relationship between the writer Krasznahorkai and the film-director in
making this film. In this respect, this adaptation is more than a simple case
of what Joachim Paech calls material intermediality – a technical transpo-
sition – but is also a symbolic one, due to this double representation (the
picture shows itself), or mise en abyme. On the other hand, we see the doc-
tor writing the novel, the adaptation of which the film is: a reinforcement
of Marshal McLuhan’s principle that the old medium is always becoming the
content of the new one.
A further gesture denies a delimitation of the two media involved, and
thus excludes the ‘ut pictura poesis’ comparative method: nothing in the
film’s credentials alludes to adaptation or to any transformational works
undergone. Moreover, the writer of the original novel appears, along with
Ágnes Hranitzky and director Béla Tarr, as the author of the movie. This
gesture overtly replaces the notion of the author, related to the concept
of artwork, with that of author function, defined by Barthes and Foucault:
this function, instead of granting the integrity of one specific text – the
novel or the film – is responsible for the connections between different
texts, and ensures the functioning and status of a discursive set in a given
society and culture.11 It is the principle of cohesion in an intertextual
web. Thus, this collective authorship participates in a collective, intermedial
‘meaning-making’.

Another tango: space and time, description and narration


in the novel

Lászlo Krasznahorkai’s novel already challenges the limits of the poetic


medium and its temporality due to an original approach to description and
narration. The central narrative feature of Satan’s Tango is the ‘fight’ between
these two: the world of silent, static objects is repeatedly overtaking that
of human action, which freezes all the time into stasis. The kitchen is set-
ting off, like a car, and the silent objects around the protagonists suddenly
start a nervous dialogue. The temporality of poetry and all literature, so
ferociously defended by the classic discourse of ut pictura poesis seems to

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Hajnal Király 205

be overturned by this kind of narrative, celebrating the spatial effect of the


freezing movement and the temporary extension of the description.
The floating character of the rigorously detailed description and the ten-
dency of the action to freeze in a tableau vivant is characteristic of what
literary criticism calls – in connection with the work of Gabriel García
Márquez – ‘magic realism’, often cited as reference for the narrative style

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
of Lászlo Krasznahorkai. This magic realism appears as an uncanny fusion
between the preference for detail and materialism of the nineteenth-century
narrative and the visionary images of the ancient myths. On this basis
the narrative of Satan’s Tango may be called chronotopic: the transmediatic
notion of the bakhtinian chronotope12 is a concrete, substantial unity of
spatial and temporal characteristics. Time becomes form and space extends
into a temporal continuity, space and time become reflections of each other.
The characteristics of time are revealed by space and vice versa, space is mea-
sured and filled with content by time. The house, the threshold, the road are
considered typically chronotopic, transmediatic forms, susceptible to such
narrative categories as strange world, public life, encounter, time of adven-
ture and such semantic contents as knowledge, power and desire. In Satan’s
Tango, the mobility of characters is reduced to the house (the space of the
observer) where they sit all day long, watching ‘how the damned life goes on’
and the muddy road (space of action), flowing away as a river (of time) after
the autumn rains, makes it impossible to leave the house. Thus, the mov-
ing, acting character is replaced here by the observer, who participates, as
Greimas puts it in his structural semantics, in the ‘spectacle of knowledge’.13
The News that They are Coming, Knowing Something, The Perspective, when
from the Front, The Perspective, when from Behind, The Circle Closes – as these
titles from the table of contents of the novel show, news and knowledge
become central semantic contents, together with the chronotopic perspec-
tive, a major compositional element from the time of Renaissance painting
(in which spatial and temporal aspects are intertwined) and also a symbolic
form to express longing and desire.

Reading out the picture: ‘meaning making’ in the film

In Tarr’s film, perspective becomes a symbolic correspondent to the semantic


content of knowing something, ‘thinking in perspective’, making plans or
longing. Accordingly, all roads in the film are represented in perspective and
the characters moving on them are shown from behind, except for the last
trip of the doctor to the church, when we see him coming back, from the
front, as he refuses to leave the farm: the circle closes.
The house and the road also become basic chronotopic components of a
systematic language, as different frames – windows, doors, thresholds – and
‘liminas’ between interior spaces and exterior world, powerful visual sym-
bols expressing the protagonists’ cognitive modalities, their ability to ‘see

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


206 Mediations and Transformations of Media

through’ (that is, to understand their situation) and, consequently, to ‘step


through’ (to change their lives). Through these visual elements all specific
mediatic features such as the role of point-of-view in literary and film narra-
tion seem to dissolve into a more general language of visual media and visual
anthropology. Thus, the semantic contents of understanding and power are
not connected to the points-of-view of certain characters, but are represented

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
through their relationship to frames and thresholds. This pictorial conven-
tion is represented, for example, by seventeenth-century Dutch painting,
one of the most prominent examples being the series by Pieter de Hooch,
showing the relationship of different characters of the household (a child, a
maid) and to the outer world, in accordance with contemporary social codes.
As Wolfgang Kemp has demonstrated, this convention is already present in
the works of Giotto and his contemporaries.14 Similarly, in Satan’s Tango,
the characters’ relationships to frames or thresholds represents their ability
or inability to understand the satanic plan of Irimias (allusion to prophet
Jeremiah) and, accordingly, to step out of this trap.
Greimas’s semiotic square seems to be a model capable of systematizing
these different attitudes, which are responsible for the dramatic turns in the
plot. This model, as well as the actantial system, is organized around the
key notions of ‘knowledge’, ‘power’ and ‘desire’, and explores the ‘meaning
of human actions’ and ‘how man and the world are’. Claude Gandelman, in
his Reading Pictures, Viewing Texts uses the semiotic square to demonstrate the
cognitive and passage modalities identifiable in seventeenth-century Dutch
paintings.15 According to this model, the distribution of the characters on
the semiotic square ‘to be able to see (through)’ would be as illustrated
in Figure 12.

To be able to see To be able not to see


(Freedom) (Independence)
Irimias, the doctor, Futaki, Essie The doctor
After her flight from home

To be not able not to see To be not able to see


(Obedience) (Powerlessness)
The informers: The Schmidts Essie on No man’s land
The Halics family, the teacher The farm’s inhabitants
before their exodus

Figure 12 Greimas’s semiotic square of cognitive modalities, from A. Greimas (1987)


‘The Interaction of Semiotic Constraints’ in On Meaning: Selected Writings in Semiotic
Theory. Trans. P. J. Perron and F. H. Collins. (Minneapolis: University of Minnesota
Press)

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Hajnal Király 207

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Figure 13 Frames from Satan’s Tango by Béla Tarr (1994). Window- and doorframes,
‘liminas’: visual schemata expressing the semantic contents of ‘knowing something’
and ‘passing through’

Similarly, the semiotic square of passage modalities would appear as fol-


lows: ‘To be able to pass’ or freedom (Irimias, the doctor, Futaki, Essie after
her flight from home), ‘to be able not to pass’ or independence (the Doctor
and Essie), ‘to be not able not to pass’ or obedience (the Schmidts, the Halics
family, the teacher, Essie on the no man’s land) and ‘to be not able to pass’ or
powerlessness (Essie on the no man’s land and the farm’s inhabitants before
their exodus) (see Figure 13).
This model not only ensures a categorization of different attitudes, but it
is also able to draw the turning points of the narration, which, according
to its classical definition, always has to include development. As depicted
here, the majority of characters remain blindly obedient (unable not to see
and not to step out, or more precisely, even though they step out in the
end, nothing changes), while the diabolic freedom of the false prophets and
the independence of the outsider doctor remain unchanged. Intriguingly,
the only development occurs in the case of Essie, the little girl considered
mentally disabled: she manages to step out from her obedient position
and gain her independence and absolute freedom through suicide and a
miraculous ascension, but this narration is cut short by the middle of both
the novel and the film. This is as if to emphasize that narrativity is not
an inherent specificity of the film medium, but a learned one. The ten-
sion is not created on the level of the story, but is due to a continuous
struggle between action (connected to the road) and observation (associ-
ated with the house), narration and description, and (de)monstration. The
drama of hesitation – Satan’s Tango – is not only that of characters, but
evidently that of the ‘two faces’ of the medium: showing (or opening a
window to) reality (the documentary tradition established by the Lumière
brothers) and telling a story, creating a diegesis, the illusion, the magic of
another world often associated with the pioneering work of Meliès. The
film chooses the ‘descriptive mode’, thus creating or modelling a visionary
world. According to Panofsky, the visionary has been drawn into images
by perspective: ‘Perspective seals off religious art from the realm of the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


208 Mediations and Transformations of Media

magical, dogmatic and symbolic and opens it to something entirely new:


the realm of the visionary, where the miraculous becomes a direct experi-
ence of the beholder; to the realm of psychological, the miraculous finds
its last refuge in the soul of the human being represented in the work
of art.’16
At the end of the film the doctor sets off to find out whether the bell he

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
hears ringing is real or just an illusion. This can be conceived of as another
statement in favour of the reality paradigm of film theory: films exist – as
other works by Tarr exemplify – to show and document reality, not to cre-
ate an illusion of it. After his return to the house (the place of observation)
we see him boarding up the window: the fact that the closing of the win-
dow marks the end of the film, presents the medium as another frame erected
between us and reality, enabling or not enabling us to see through – the old
question of the medium transparency – or to pass into its diegetic world.
Again, as Panofsky puts it in his Perspective as symbolic form, it is the perspec-
tive that transformed the entire picture into a window, ‘and we are meant
to believe we are looking through this window into a space’.17 At the same
time, this scene metaphorically reinforces the inseparability of the media
involved in the adaptation.
Intermediality here appears as a space of close interaction between the
writer and director, resulting in a transmediatic chronotopic language,
the only kind adequate to present a parable on the human condition.
This transmediatic chronotopic and symbolic language has been realized
with the involvement of codes from a third medium: painting (such as
composition, framing, perspective). As I already mentioned above, this is
not an isolated tendency in contemporary film production, especially if
we consider the wave of films coming from China, Korea, Hong Kong
and Japan, currently conquering European cinemas. Their narratological
minimalism and visual elegance – reminding us of the best tradition of
European film (the work of Michelangelo Antonioni, Andrej Tarkovskij) –
brings a refreshing new view to an exhausted Western production. This
seems to reinforce the principle that the return to the beginnings is a
sign of maturity of any art, already independent from the necessity to
show technological competence. Similarly, the concept of medium is no
longer reduced to technology. As Erwin Panofsky summarizes in his essay
on perspective:

Such reversals, which are often associated with a transfer of artistic ‘lead-
ership’ to a new country or a new genre, create the possibility of a new
edifice out of the rubble of the old; they do this precisely by abandoning
what has been already achieved, that is, by turning back to appar-
ently more ‘primitive’ modes of representation. These reversals lay the
groundwork for a creative reengagement with older problems, precisely
by establishing a distance from those problems.18

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Hajnal Király 209

This observation coincides with Rudolf Arnheim’s prophetic conclusion of


his desperate essay ‘New Laokoön’, on the incompatibility of film and sound:
as these ‘hybrid’ forms are always unstable, there is always a hope of return-
ing to ‘pure forms’.19 As we have seen, the second century of cinema has
proven itself promising in terms of ‘mediatic purification’. Narratology tends
to lose terrain, and the need of a semiotic mediatics has never been more

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
urgent.

Notes
1. W. J. T. Mitchell (1994) Picture Theory: Essays on Verbal and Visual Representation
(Chicago: University of Chicago Press), pp. 16.
2. M. Krieger (1992) Ekphrasis: The Illusion of the Natural Sign (Baltimore MD and
London: Johns Hopkins University Press).
3. See, for example, A. C. Danto (1987) The State of the Art (New York: Prentice Hall
Press) and more recently, A. C. Danto (1997) After the End of Art: Contemporary Art
and the Pale of History (Princeton: Princeton University Press).
4. See, for example, P. Arthur’s essay (2005) ‘The Written Scene: Writers as Figures
of Cinematic Redemption’ in R. Stam and A. Raengo (eds) Film and Literature
(London: Blackwell), pp. 331–42.
5. I am not dealing here separately with the technical aspects of the medium, as I
consider it thoroughly intertwined with the aesthetical, cultural and sociological
ones. In fact, all technical improvements (the colour, the size of the screen, the
television) have added new aesthetical dimensions to the film. As I point out in
my essay, film often tends to ‘deny’ its technical specificity (often identified with
‘medium specificity’) by turning towards the conventions of other media, such as
painting or photography. Interestingly enough, in this case, the lack of learned
conventions becomes a qualifying factor.
6. He is considered the founder of cognitive film theory, relying on cognitive psy-
chology. See, among others: D. Bordwell (1985) Narration in the Fiction Film
(Madison: University of Wisconsin Press); D. Bordwell (1989) Making Meaning:
Inference and Rhetoric in the Interpretation of Cinema (Cambridge MA: Harvard Uni-
versity Press); D. Bordwell and K. Thompson (2000) Film History: An Introduction
(New York: McGraw-Hill) and most recently, D. Bordwell (2008) Poetics of Cinema
(Berkeley CA: University of California Press).
7. J. Paech (2000) Artwork – Text – Medium: Steps en Route to Intermediality,
http://www.unikonstanz.de/FuF/Philo/LitWiss/MedienWiss/Texte/interm.html.
8. M. Leverette (2003) Towards an Ecology of Understanding: Semiotics, Medium The-
ory and the Uses of Meaning, http://www.imageandnarrative.be/mediumtheory/
marclaverette.htm.
9. Béla Tarr (1994) Sátántangó. Writers: László Krasznahorkai and Béla Tarr. Cast:
Mihály Víg, Putyi Horváth, László Lugossy, Éva Albert Albert Almássy, János
Derzsi, Irén Szajki, Alfréd Járai, Miklós B. Székely, Erika Bók, Peter Berling.
10. Krasznahorkai László (1993) Sátántangó (Budapest: Széphalom Könyvkiadó).
Available in French translation: Tango de Satan (2000) translated by Joelle
Dufemilly (Paris: Gallimard).
11. See R. Barthes (1993)[1960] ‘Authors and Writers’, in S. Sontag (ed.) A Roland
Barthes Reader (London: Vintage), pp. 185–93, and M. Foucault (1991) ‘What is

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


210 Mediations and Transformations of Media

an Author?’ in P. Rabinow (ed.) The Foucault Reader: An Introduction to Foucault’s


Thought (London: Penguin Books), pp. 101–20.
12. See M. M. Bakhtin (1984) ‘Forms of Time and of the Chronotope in the Novel:
Notes toward a Historical Poetics’ in C. Emerson and M. H. Austin (eds) The
Dialogic Imagination: Four Essays (Austin: University of Texas Press) pp. 84–258.
13. A. Greimas (1987) ‘The Interaction of Semiotic Constraints’ in On Meaning:
Selected Writings in Semiotic Theory, translated by P. J. Perron and F. H. Collins

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
(Minneapolis: University of Minnesota Press), pp. 48–62.
14. See W. Kemp (1996) Die Räume der Maler: Zur Bilderzählung seit Giotto (München:
Verlag C. H. Beck).
15. See C. Gandelman (1991) Reading Pictures, Viewing texts (Bloomington and
Indianapolis: Indiana University Press), pp. 14–55.
16. E. Panofsky (1997) Perspective as Symbolic Form, translated by C. S. Wood (New
York: Zone Books). Originally published as Die Perspektive als ‘symbolische Form’ in
the Vortrage der Bibliothek Warburg 1924–25 (Leipzig and Berlin, 1927), pp. 258–
330.
17. See ibid., p. 27.
18. Ibid., p. 47.
19. R. Arnheim (1957) ‘A New Laocoon: Artistic Composites and the Talking Film’
in Film as Art (Berkeley and Los Angeles CA: University of California Press),
pp. 199–230.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


14
Media in the Cinematic Imagination:
Ekphrasis and the Poetics of the
In-Between in Jean-Luc Godard’s

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Cinema
Ágnes Pethő

A love of cinema desires only cinema, whereas passion is excessive:


it wants cinema but it also wants cinema to become something else, it
even longs for the horizon where cinema risks being absorbed by
dint of metamorphosis, it opens up its focus onto the unknown.
—Serge Daney: The Godard Paradox1

Intermediality, the cinematic ‘in-between’ and ekphrasis

The complex mediality of cinema is unique among all other arts in its para-
doxes and raises a constant challenge not only to theorists who try to define
its characteristics but also to filmmakers who consciously explore its bound-
aries. On the one hand, cinema is the most transparent or ‘invisible’ medium
possible, operating with moving pictures that result in the illusion of real-
ity (we seem to see the things themselves and not their representation), and
engaging all our senses in their perception. On the other hand, it is also the
most abstract and constructed medium possible that has no palpable mate-
rial form (all the sensual complexity of the cinematic image being nothing
but an illusion). More importantly, from a media theory point of view, the
moving picture as a medium can remediate all other media forms used by
human communication. The mixed mediality of cinema – although it has
often been described in terms of the Gesamtkunstwerk ideal – is not a result
of an additive process (a unity of moving pictures, language, sound and so
on), but consists of a very unstable set of interrelationships that has under-
gone many changes in its configuration throughout its technical and stylistic
history.2
Cinematic experience itself can be defined by the tensions of being in a
state of ‘in-between’: in between reality and fantasy, in between empirical

211

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


212 Mediations and Transformations of Media

experience and conscious reflection, in between words and images, in


between the different art forms and in between media. The mediality of cin-
ema can always be perceived as intermediality, as its meanings are always
generated by the media relations that weave its fabric of significations. Cin-
ema can be defined as an impossible, heterotopic space where intermedial
processes take place, and where figurations of medial differences are played
out.3 Moreover Foucault’s ‘heterotopia’4 seems adequate to describe not only

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the way in which the images are being situated in a ‘place without place’,
but also more generally, to describe cinema’s ‘place’ itself among the arts
and media.
Jean-Luc Godard’s films have long been associated with the idea of inter-
mediality, in fact it seems that no theory of cinematic intermediality can be
forged without references to his works.5 Godard has discovered in cinema
a ‘space’ in which all other forms of representations can be inscribed and
all other media can be re-mediated. In his films images are always closely
related to words6 and cinema is always conceived in a dynamic relation with
the other arts, a relation that connects Godard’s cinema to a more general
artistic tradition: the phenomenon known as ekphrasis.
Ekphrasis, as we know, is a rhetorical device elaborated in Antiquity con-
sisting in the detailed description of a gallery of paintings or a group of
statues, a case where a verbal text is produced in competition with the plas-
tic arts. In essence, it is generally understood to stand for the urge of an
artist working in the medium of language to express whatever falls beyond
the realm of language, to use linguistic expressivity as a ‘tactile’ or visual
sense and thus cross over into the domains of the visible. Ekphrasis has been
a much debated question in literature, but its applicability to questions of
cinema has not been thoroughly investigated. Nevertheless, in a medium so
tied up with all other forms of human expression, questions of media bor-
ders are bound to emerge. We can say that certain tendencies in film history
undoubtedly have aspects that can be related to what theorists call ‘ekphras-
tic impulse’, a tendency to challenge cinema’s conventionally established
perceptive frames, and therefore it seems that the possibilities of a theory
of cinematic ekphrasis are worth exploring. What should be clarified first of
all, however, is why this particular term should be taken into consideration,
and not the term ‘remediation’ that Bolter and Grusin consider as denot-
ing a very similar process within media relations.7 W. J. T. Mitchell defines
ekphrasis as ‘the verbal representation of visual representation’,8 Bolter and
Grusin call it ‘the representation of one medium in another remediation’.9
Are these two terms interchangeable, as Bolter and Grusin seem to suggest?
Mitchell explains quite clearly that there is a possibility of overgeneralizing
the term by considering ekphrasis as the name of an overarching principle,
and he finally gives a definition that clearly concentrates on the presence
of some kind of representation both as signifier (verbal representation) and
as signified (visual representation). Bolter and Grusin in their more general

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Ágnes Pethő 213

media theoretical framework consider ekphrasis as a case of media being


incorporated, repurposed by other media. Taking into consideration both
viewpoints, I consider that we should not merge the idea of ekphrasis with
the idea of remediation or use the two terms as synonyms, but we should
consider them as complementary terms. Given also the fact that the idea of
ekphrasis is usually linked more closely not only to the idea of representa-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
tion but also to the aesthetic value of texts, whenever the relations of cinema
and the other arts, or the representations of other arts are involved, the term
‘ekphrasis’ seems more adequate; and whenever we can speak of more gen-
eral media relations (like the use of written or verbal language within a film,
for example), the term ‘remediation’ would be more suitable.
So what would be the main characteristics of a cinematic ekphrasis? Do
all artworks represented in a film result in an ekphrasis? When can we con-
sider that film attempts to ‘challenge its own boundaries’? To list only a few
important aspects, the following conditions for the relevance of the term
‘cinematic ekphrasis’ can be named:

(a) A film cannot be called ekphrastic simply whenever it includes an


embedded representation of another artwork.10 A condition for inter-
preting it as ekphrasis is that this embedded art form should go beyond
the function of a diegetic representation (for example, a painting on the
wall) and should be manifest as a medium that is different from that
of the cinematic image in which it is embedded. In short, an ekphra-
sis requires the perception of intermedial relations, as ‘transformative
inscriptions’ or ‘figurations’11 of mediality in a work.
(b) Cinema can also be perceived as ekphrastic not merely through the
media differences of embedded other media forms, but on a more gen-
eral level, in any case when cinema explicitly attempts to rival another
art form (or style developed in another art form). Expressionist films in
which we have the characteristically painted settings would be a good
example of such an attempt to transform the moving pictures into a
sequence of moving paintings.
(c) One of the most important features of cinematic ekphrasis is that in
fact, in cinema, we can usually speak of multiple or multidimensional
ekphrastic tendencies in which one medium opens up the cinematic
expression in order to mediate towards the ekphrastic assimilation of
another. In most of the cases when cinema imitates another art form this
imitation is not the primary ‘target’ of an ekphrastic impulse, but a vehi-
cle, a ‘mediator’ towards yet another medium, the essence of which is
perceived as something ‘beyond’ concrete expression, something ‘infig-
urable’. Common examples of this are the so-called ‘picto-films’, which
have acquired something of the status of a sub-genre among literary
adaptations, and in which a sense of ‘literariness’ is conveyed through
imitations of paintings or painterly styles.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


214 Mediations and Transformations of Media

An investigation into the ekphrastic aspects of Godard’s films seems to be


extremely fruitful, as his films can be considered ekphrastic not merely
because they often transpose representations from other arts onto the
screen and foreground essential features of cinematic intermediality, but also
because some of his films include explicit quotations from ekphrastic litera-
ture and thus engage in a multiple or meta-ekphrastic cinematic discourse.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
In what follows from the variety of intermedial relations that can be con-
nected to the principles of ekphrasis in Godard’s films, I will outline four
such types:

1. A multiplication of media layers ‘opening up’ towards each other and


remediating each other.
2. Ekphrasis as a ‘figure of oblivion’. Ekphrasis via media erasures.
3. The function of ekphrastic metaphors.
4. The ‘museum of memory’ and the deconstruction of ekphrasis in later
works.

The vertigo of media: ekphrasis and mise en abyme

In Jean-Luc Godard’s films there are instances of cinematic intermediality in


which one medium becomes the mirror of the other in some way. In other
words we can speak of an intermedial mise en abyme. One of the best known
examples of this is Godard’s early masterpiece Vivre sa vie (1962, translated
as A Life of her Own/Her Life to Live) which also includes a direct reference to
the ekphrastic tradition itself.
Here, in the final scene, a young man reads out a fragment from Edgar
Allen Poe’s short story ‘The Oval Portrait’, which includes an ekphrasis of a
painting. Poe’s story is about a man reading a book about a painter painting
a portrait of the beloved woman. The story within Godard’s story is also a
story within a story. Moreover, the images are over-codified by the voice-over
narration (in fact Godard’s own voice). The woman listening to the story
of the painting, Nana (Anna Karina) becomes herself virtually a cinematic
painting, losing all connections with ‘real life’ just like Poe’s model, whose
life is paradoxically stolen away and transformed into the painted image.
Throughout the film the protagonist is shown in a Brechtian split between
actress (Anna Karina, casting occasional direct glances at the camera, implic-
itly at her husband-author) and role (Nana).12 In the ‘Oval Portrait’ sequence
we are shown the face of Nana against a blank background, the camera
outlining her portrait by nailing her to the wall, rendering her a helpless
object of representation. All of this is additionally doubled by the mirror-
like presence of a photograph of Liz Taylor pinned to the wall, while Nana
herself is ‘painting’ her own lips in a mirror. In earlier sequences in the film,
we see her in turn either as a mirror image of Dreyer’s Jeanne D’Arc, or
a variation on the topic of ‘lonely courtesan in a Parisian café’ (so famil-
iar in French impressionist painting) or as an intertextual variation on

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Ágnes Pethő 215

Hitchcock’s enigmatic multiple identity woman in Vertigo (1958). Godard’s


film foregrounds cinema’s ‘ekphrastic impulse’ which aims at rivalling the
other arts by remediating traditional forms of portraiture both in the visual
arts and in literature. The embedded representations flaunt cinema’s mul-
tiple mediality, but they also result in an endless process of signification,
an endless attempt at ‘figurating’ the ‘infigurable’ identity and beauty of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Nana/Anna Karina. The ultimate image of Nana/Anna Karina that we get is
placed somewhere in an impossible space between art and reality, between
one medium and another, and in this way the film offers instead of the
images of Nana’s life (as the title would suggest), the paradoxes of the life of
the images of Nana/Anna Karina.
Similarly, in other Godard films the numerous reflections of characters
in paintings, posters, comic book drawings, genre film iconography, literary
figures and so on, can be seen in parallel with the remediational logic of
traditional literary ekphrasis.

Ekphrasis as a ‘figure of oblivion’

Besides this ekphrastic model of multiple remediations we have several


instances in Godard’s films in which quotations from post-Romantic French
poetry are used both as a reference to a model for a relationship to cul-
tural heritage that Godard adopts and also a means of infusing cinematic
language with poetry. The poets referred to by Godard usually belong to a lit-
erary phenomenon that Harald Weinrich13 described as the ‘art of oblivion’.
The traditional ‘art of memory’ (ars memoria) present in European culture
since Antiquity was based on the principle of preservation of knowledge by
way of mnemonic devices of visualization (association of images and places,
for instance). In contrast, the poetry included in the ‘art of oblivion’ was
mainly concerned with the renewal of poetic language, of finding new ways
of reinforcing the power of words. The birth of a new poetic language is often
allegorized in the poems of Verlaine, Rimbaud, Mallarmé or Valéry in a series
of images that emphasize the magic moments of loss of consciousness that
break up the process of remembering. Poetry derives its power from purifica-
tion by way of oblivion paradoxically in the presence of ‘frozen images of the
past’. Godard’s Bande à part (A Band of Outsiders, 1964) and Pierrot le fou (Pier-
rot Gone Mad, 1965) are full of quotations from these poets. The cited texts
enrich the image and direct our attention towards another medium (poetry),
distancing the image from the real life location and weaving around it a tex-
ture of pure imagination. There is no break in the pictorial flow, but another
verbal picture hovers over the image that we actually see. The scenes gain an
inner vibration. Godard’s ambition equals that of the quoted poets: he hopes
to enhance the expressivity of the cinematic image, and thus give birth to a
new kind of cinematic ‘language’ by remediating the poetry onto the screen.
In A Band of Outsiders there is actually a character whose name is Arthur
Rimbaud. One of the first scenes in which he appears, together with his

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


216 Mediations and Transformations of Media

opening remark, ‘It’s cold and lonely here’, paraphrases Verlaine’s famous
poem, Sentimental dialogue: ‘In the old and frozen park, two figures have just
passed’. The words that we hear from his friend as well as Godard’s own
voice-over are also a mixture of quotations from Verlaine. The quotations
are presented in a Brechtian manner, the words do not sound poetic at all:
the poetic images seem to be ‘frozen’ into the text of a casual dialogue and

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
voice-over narrative. At the same time, the image they are looking at resem-
bles a painting, the river and the branches of the trees reflecting in the water,
the fog veiling the landscape could well be an ekphrastic paraphrase of a
number of poems by Verlaine, Rimbaud or Mallarmé. The spectacular land-
scape in the background appears as a frozen surface beyond which images of
literature or painting can be sensed, but which is ignored by the two men.
In a traditional (literary) ekphrasis we have an active, speaking subject who
contemplates a passive (and usually silent) object of the gaze.14 In Godard’s
film we have an active, verbal component (speech) and a passive visual com-
ponent (image) which are placed in the same cinematic frame and can both
separately be called ekphrastic, but which are not in a direct ekphrastic con-
nection with each other. They become interrelated only on a secondary
level, where we recognize the type of poetry quoted here and project the
images from this poetry onto the screen. The painterly setting ‘erases’ the
medium of language (as these pictures replace the poetic imagery that they
‘translate’). The dialogue and voice-over narrative ‘erases’ the medium of the
(poetic) image (the unmarked quotations become depoeticized as they are
woven into the casual dialogue). The opening up of the image into ekphras-
tic dimensions of poetry by way of both quoting its lines and ‘forgetting’
about them – first of all by ‘hiding’ them within the dialogue (that is always
primarily decoded in its relevance regarding the diegesis) and also by way of
remediating aspects of these poetic texts into the images – is paradoxical but
effective.
There is another scene in Band of Outsiders that could also be linked to
the ekphrastic tradition. Godard includes a concrete defiant gesture in his
film that can be interpreted as an erasure of a traditional cultural space
that usually hosts ekphrastic meditations. I am referring, of course, to the
famous scene at the Louvre. The three young protagonists race through
the Louvre in a record time of 9 minutes 45 seconds without looking at
the masterpieces hanging there. The scene, which ‘forgets’ about canonized
contexts and traditional visual artistry in favour of youthful spontaneity,
ultimately expresses nothing else but this: the bursting energy of an act of
sheer inspiration – something that could be a driving force behind any work
of art.
Another example of ekphrastic ‘erasures’ is Pierrot le fou (1965), at the end
of which Pierrot paints his face blue and thus makes it resemble both the
Picasso painting and the portrait of Rimbaud shown earlier in the film. He
is thus literally transformed into a cubist image even before, in the viewers’

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Ágnes Pethő 217

imagination, his face is blown up into pieces similar to cubist portraits. The
scene can also be interpreted as a concealed reference to Verlaine’s poem,
‘Pierrot’, in which a frightening face of a scarecrow seems to be blown up, his
eyes sizzling in their hollow sockets. Once more, we have multiple erasures:
first Godard erases the filmic character and remediates it as a painted portrait,
and when Pierrot blows up his face as an image the scene also screens a

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
literary allusion, opening up the image for literature.

Ekphrastic metaphors pointing to the Other of the filmic image

Sometimes the mutual erasure, replacement or displacement of text and


image and an opening up towards the infigurable is achieved by a single
ekphrastic metaphor in Godard’s films. This is the case with literary quota-
tions of only a few words that are included within Godard’s films time and
again without any specific mark or clue. On one level these short quota-
tions which are introduced without reference to their source act as a kind of
poetic unconscious of the images. On another level these unmarked inclu-
sions of poetic texts have the effect of what Foucault described in his Las
Meninas essay in connection with the interdiction of using proper names: ‘If
one wishes to keep the relation of language to vision open, if one wishes to
treat their incompatibility as starting point for speech instead of an obstacle
to be avoided, so as to stay as close as possible to both, then one must erase
those proper names and preserve the infinity of the task.’15
In certain cases, however, in Godard’s films exactly the opposite happens,
as Godard randomly drops in references to specific names of authors and
characters or titles of whole literary works. At the beginning of Pierrot le
fou, for instance, the main character, Pierrot complains vehemently about
the fact that people no longer think of Balzac when dialling the area code
on their phone. This principle of associating literature by way of a one-word
reference with commonplace, everyday phenomena is characteristic of many
of Godard’s films.16 In the same Pierrot le fou, for instance, we hear the nar-
rator say: ‘Marianne had the eyes of both Aucassin and Nicolette’. Likewise,
in the Band of Outsiders, Godard tells us that the characters ‘stopped at a
bookstand and Franz bought the novel which reminded him of Odile’. The
reference acts as a sort of ‘ekphrastic metaphor’, as we have one word acting
as a metaphor that refers to a whole literary text. It does not suggest one par-
ticular image, but points to something too complex to be captured within
a single image, therefore ultimately unimaginable (we may either not know
the texts referred to or know them and then the meanings generated are
virtually infinite). However, this placement into a narrative textual context,
this mise en histoire can also parody clichés of narrative cinema, which con-
ventionally works by dissolving images within the process of storytelling.
As we have learned from the cognitive theories of cinema, the classical
dynamics of filmic narrative always consist in images having the role of

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


218 Mediations and Transformations of Media

directing the viewer’s attention toward the construction of a coherent story.


In this case the concrete image is projected into a void that the viewer is con-
fused about how to fill. Classical ekphrasis operates with the absence of the
image as the Other of the text,17 Godard plays with the absence of the text.
Such ekphrastic metaphors also work in the direction of painting. In A
Band of Outsiders, for instance, we hear this: ‘The Seine resembled a Corot’.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
In the Le petit soldat (Little Soldier, 1963) the narrator similarly says that the
‘somber blue sky reminded me of a painting by Paul Klee’. The images of the
foggy river or of the streets at night that appear throughout the film may
resemble a painting in general but the particular image it is spoken over
may or may not. The image is nonetheless displaced, the word (Corot or
Klee) projects it into an impossible space between cinema and painting, just
as the earlier example of Marianne’s eyes being compared to Aucassin and
Nicolette’s projected the singular concrete image against medieval narrative
literature and the whole myth surrounding it. Also in the Little Soldier there
is another piece of conversation with a similar logic. First Bruno tells Veron-
ica that she reminds him of a character in a Giraudoux play and later he
muses upon the question of whether the colour of girl’s eyes is Renoir-grey
or Velázquez-grey. Susan Sontag considers that such references are effective
because the viewer cannot verify them.18 We must add that they cannot
be verified not because of the ignorance of the viewer, but because of the
structure of the reference: the concrete name is referring to a whole range of
possible literary works or paintings.
Henk Oosterling compares the experience of intermedial in-between to
Barthes’ notion of ‘punctum’. He says: ‘The spectator is hit: affected and
moved by the punctum’. He considers that this resembles ‘the impossible
experience of the breaks between two media’.19 In all these examples, the
intermedial reference not only underscores the medial difference (a radical
alterity) between cinema, literature or painting, but also identifies in these
Others of cinema something that is beyond perception, yet essential in the
filmic image. This intermedial opening up of the image achieves the ideal
expressed more explicitly by Godard in his later works that ‘the real cin-
ema is a cinema that you cannot see’.20 Oosterling considers this aspect
as characteristic for the reception of intermediality itself, a process that
entails that ‘the sensible, as a reflective sensibility, balances between pres-
ence and absence: going back and forth from one medium to the other, it
is a movement in which positions are articulated in the awareness that they
are principally relational and provisional’.21

The ‘Museum of Memory’ and the calligrammatic


rewriting of ekphrasis

As we have seen in the previous examples there appears to exist a permanent


duality in Godard’s cinema: the almost tactile quality of the photographic

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Ágnes Pethő 219

image in cinema, the sensual presence of things doubled with the absence of
the physical reality that the image represents, or in certain cases, a void in
the signification that can be pointed out by techniques of intermedial mise
en abyme or ekphrastic metaphors mirroring the other arts and thus keep-
ing the relationship between signifier and signified infinitely open, making
the cinematic image reach beyond its own media boundaries and into the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
domain of the unnamable.
In earlier examples, the filmic image is placed in the ekphrastic, imag-
inary space created by fragments of poetic language and visual imagery
which activate poetic sensitivity but erase literature or even painting as a
directly perceivable medium and the institution of museum as a place for
the arts or even as a place for meditation upon the arts. In the cinematic
essays of the later period the idea of the museum is revived, but more in
the spirit of Malraux’s musée de la memoire, a virtually never-ending flow of
texts and reproductions of images that generate an also endless number of
associations.
In Godard’s later films ekphrasis acts as a generative principle. One of the
first examples of this can be seen in Letter to Jane: An Investigation About a
Still, a film made in 1972 with the co-authorship of Jean-Pierre Gorin, in
which Godard and Gorin meditate upon a photograph of militant actress
Jane Fonda seen in the company of Vietnamese people during the Vietnam
war. The process works both ways in late Godard films: there is a surge of
texts interpreting pictures and pictures anchoring the meanings of texts.
There is, however, an important deviation from the principle of ekphrasis
involved here, namely, the fact that these texts and images are always con-
jured up not in each other’s absence, but in each other’s presence. So the
underlying principle can be called ekphrastic, but otherwise we witness a
more explicit word and image relationship in which the two media come to
be mutually overwritten and intertwined.
The masterpiece in this respect and the ultimate ekphrastic work of
Godard’s is undoubtedly the series of essays entitled Histoire(s) du Cinema
(Histories of Cinema, a project that he worked on between 1989 and 1999).
The ekphrastic nature of the film was consciously explored by Godard who
conceived of the project first as the publication in book form of a series of lec-
tures delivered at the request of the Conservatoire d’Art Cinématographique
in Montréal, and then released together with the film version, an art book
of reproductions and a boxed set of five CDs containing an edited version
of the soundtrack (further multiplying the intermedial ‘trans-forms’ of the
cinematic project). Godard’s Histoire(s) du cinema can be considered a medi-
tation upon the archaeology of cinema, discovering in it layers of mediality
and culture. From this perspective cinema is not defined by its storytelling
capacity, but most of all by its possibilities of transcendence, of mediat-
ing first of all reality and/or memory. It is also clear from the beginning
that for Godard mediation in the movies means remediation. As Jacques

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


220 Mediations and Transformations of Media

Rancière wrote, ‘Histories of Cinema is wholly woven out of . . .“pseudo-


metamorphoses,” . . . imitations of one art by another’.22 Cinema appears as
painting in movement, as a musical composition of shadows, forms and
colours.
Medially speaking, it is cinema deconstructed into a constellation of
words and images. But which is the medium that ekphrastically reflects the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
other? It is not the usual case in which literature is seen through cinema
or vice versa, it is not even the case in which a newer medium remedi-
ates an older one, but quite the opposite: cinema seen through the filter
of a seemingly archaic medium of moving pictures. However, this is a form
that was constructed in retrospection, a form that has never existed as such,
never existed as a vehicle for cinematic storytelling; it can only be called
‘archaic’ because the techniques used were already available at the earliest
stages of cinema. Godard took great care in using the most ‘primitive’ tech-
niques possible: photographic inserts, slow motion, shadows projected on a
wall and so on, deliberately avoiding the use of more modern technology,
while repeatedly showing an old-fashioned editing table and a typewriter as
each other’s metaphors in representing the kind of ekphrastic filmic writing
he clearly prefers. Accordingly, text and image become equally important
and subjected to the same visual compositional principles of fade, dissolve,
superimposition. Text penetrates the image, and similarly rhythm and visu-
ality appear as key aspects of language. Viewed from a closer perspective, the
medium of the Histoire(s) is derived on the one hand from photomontage, and
on the other hand from calligrammatic writing. A calligram-like shot (in fact
a detail of the cover of Samuel Beckett’s book The Image) that we can see in
part 1.B of the cycle entitled ‘Une histoire seule’ is perhaps emblematic for
Godard’s technique. We see the graphic signs of the word ‘image’ appear-
ing as the pupil of an eye in the midst of a white circle of light. The word
seems to concentrate the meaning of the image, or the other way round,
we can also say that it is the image, the bright circle of light that reveals the
graphic signs as text. Moreover, the gesture of this cinematic découpage of the
detail of a book cover is also somewhat ekphrastic, as it tears out the word
from its literary context and transposes it onto the screen as an autonomous
image, thus it actually performs not merely a decontextualization, but also,
a multiple intermedial transfer and plunges the title of a well known author
into the realm of common language (‘without proper names’, as indicated
by Foucault) and pure visuality. By way of the cinematic calligram the aura
of the author (and artwork) is lost, the aura of the word in its infinite pos-
sible relations and that of the image – that is ‘worth a thousand words’ – is
regained.
As a whole, Histoire(s) du cinema accomplishes a uniquely paradoxical
fusion of photographic collage and calligrammatic text with the musical
and spiritual aspects of cinematic montage, and this intermedium is the one
that mirrors what cinema is supposed to stand for in-between the arts.23 By

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Ágnes Pethő 221

deliberately using techniques that can be labelled as primitive or archaic,


Godard presents the ‘tangible’ mediality of cinema as it once was: having
its roots in photographic representation and indexicality, with its infinite
connections with the arts and culture; the mediality of a cinema that came
to be displaced, ‘out of time’, with the dawn of the digital era. No wonder
that some of the most important literary quotations in the Histoire(s) come

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
from Proust’s À la recherche du temps perdu, as Godard forces us to experi-
ence once more the cinematic image in all its ‘palpable’, sensual glory and
mystique. This process of rendering the invisible mediality of cinema itself
visible by way of a complex set of ekphrases and (calligrammatic) remedi-
ations is in fact a feature that is consistent with the essential principle of
any Godard film: in it we experience a cinema coming to terms with its
own (inter)medial processes, a cinema that never ceases to open up towards
other arts and media in a constant quest to become – to quote one section
of Histoire(s) du cinema, borrowing a phrase from Malraux – no less than ‘the
currency of the absolute’ (‘la monnaie de l’absolu’).

Notes
1. S. Daney (2004) ‘The Godard Paradox’ in Michael Temple, James S. Williams and
Michael Witt (eds) Forever Godard (London Black Dog Publishing), p. 68 (italics
mine).
2. Technically we can think of such major changes as the shift from silent films to
the talkies, or stylistically we can keep in mind that the medial characteristics of
a neorealist film are very different from the multimedial extravaganza of a Peter
Greenaway film, for example, which consciously constructs its visual texture as
an interart palimpsest.
3. See Joachim Paech’s theory of cinematic intermediality in this respect: (1998)
‘Intermedialität: Mediales Differenzial und transformative Figurationen’ in
J. Paech (ed.) Intermedialität: Theorie und Praxis eines interdisziplinären Forschungs-
gebiets (Berlin: Erich Schmidt), pp. 14–30. Cf. also Müller in the present volume.
4. M. Foucault (1984) ‘Dits et écrits: Des espaces autres’ Architecture, Mouvement,
Continuité 5 (octobre), pp. 46–9.
5. Cf. J. E. Müller (1996) Intermedialität: Formen moderner kultureller Kommunikation
(Münster: Nodus Publikationen).
6. I have elaborated on the questions of the multiplicity of word and image rela-
tions in Godard’s cinema in the essay (2008) ‘The Screen is a Blank Page: Jean-Luc
Godard’s Word and Image Plays’ in Ágnes Pethő (ed.) Words and Images on
the Screen: Language, Literature, Moving Pictures (Newcastle-upon-Tyne: Cambridge
Scholars Publishing), pp. 159–87.
7. Cf. J. D. Bolter and R. Grusin (1999) Remediation: Understanding New Media
(Cambridge MA: The MIT Press).
8. W. J. T. Mitchell (1994) Picture Theory: Essays on Verbal and Visual Representation
(Chicago and London: University of Chigaco Press), p. 152.
9. Bolter and Grusin (1999) Remediation, p. 45.
10. L. M. Sager Eidt’s book on ekphrasis (2008) Writing and Filming the Painting:
Ekphrasis in Literature and Film (Amsterdam and New York: Rodopi), includes

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


222 Mediations and Transformations of Media

examples of filmic transpositions of paintings but is far from offering a more


nuanced examination of ekphrastic phenomena within cinema.
11. Cf. Paech (1998) ‘Intermedialität’.
12. The role itself, through the name Nana, is a hint at a literary text, Zola’s novel
having the same title, Nana (the protagonist of which is also a prostitute who has
ambitions of working in show business).
13. Cf. H. Weinrich (1997) Lethe: Kunst und Kritik des Vergessens (Munich: C. H. Beck

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
Verlag).
14. ‘The “self” is understood to be an active, speaking, seeing subject, while the
“other” is projected as a passive, seen, and (usually) silent object’: Mitchell (1994)
Picture Theory, p. 157.
15. M. Foucault (2002) The Order of Things: An Archaeology of the Human Sciences
(London: Routledge), p. 10.
16. It is also consistent with Tamar Yacobi’s views according to which ekphrasis can
consist of a single ‘ekphrastic simile’ of no more that one phrase, as this functions
as an ‘abbreviated reference to a whole pictorial set of works which silently refers
the reader to the original itself for details and extensions’: T. Yacobi (1997) ‘Verbal
Frames and Ekphrastic Figuration’ in U.-B. Lagerroth, H. Lund and E. Hedling
(eds) Interart Poetics: Essays on the Interrelation of the Arts and Media (Amsterdam
and Atlanta GA: Rodopi), p. 42.
17. ‘The ekphrastic image acts, in other words, like a sort of unapproachable and
unpresentable “black hole” in the verbal structure, entirely absent from it, just
shaping and affecting it in fundamental ways’: Mitchell (1994) Picture Theory,
p. 158.
18. S. Sontag (2002) ‘Godard’, Styles of Radical Will (New York: Picador), pp. 147–93.
19. H. Oosterling (2003) ‘Sens(a)ble Intermediality and Interesse: Towards the Ontol-
ogy of the In-Between’, Intermédialités, 1(Printemps), pp. 37–8.
20. Cf. Histoire(s) du cinéma: Une vague nouvelle.
21. Oosterling (2003) ‘Sens(a)ble Intermediality and Interesse’, p. 43.
22. J. Rancière (2007) The Future of the Image (London: Verso), p. 41.
23. I am not disputing Paech’s argument that the film’s main figuration is the medial
difference between video as ‘individual’ medium (as video-graphic ‘writing’, a
medium suitable for personal archives) and the dreamlike medium of film, I
merely suggest that there are some other figurations that contribute to the medial
complexity of Godard’s ekphrastic cinematic language. Cf. J. Paech (2002) ‘Inter-
mediale Figuration – am Beispiel von Jean-Luc Godards Histoire(s) du Cinéma’ in
J. Eming, A. Jael Lehmann and I. Maassen (eds) Mediale Performanzen (Freiburg:
Rombach), pp. 275–97.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Part V
The Borders of Media Borders

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
This page intentionally left blank

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


15
Heteromediality
Jørgen Bruhn

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
In the first part of this essay1 I will offer a very short description of what has
been identified as interart studies and intermediality studies. After this I will
propose, by way of Lars Elleström’s model, a new multimodal definition of
medium. This, in turn, will lead me to my attempt to rethink the field of
intermediality studies by way of a new concept, heteromediality. In the second
part of my essay I will describe two future possibilities for intermediality
studies: either the field may try to establish itself as an academic discipline
by strengthening its formalistic foundations, or, on the other hand, a field
of investigation based on a more critical and ideological discourse can be
imagined. I shall argue in favour of the second possibility.
First, a few pragmatic definitions: when talking about ‘text’ I refer to the
semiotic idea of ‘complex signs or sign combinations’;2 ‘art’ refers to the
conventionally defined forms of music, painting, literature, architecture and
so on. I use ‘ideology’ as a term expressing a relatively coherent value sys-
tem, though not necessarily organized in a political system (consequently it
does not refer to the Marxist idea of ideology as merely ‘false conceptions’).
‘Intermedial’ or ‘intermediality’ refers to objects and phenomena whereas
‘intermediality studies’ refers to the activity of investigating intermedial
phenomena. The concept of medium, as mentioned above, I shall try to
define below.

Part I

Interart studies and intermediality studies


Various strands of cultural studies have, since the beginning of the twen-
tieth century, investigated the relations between the arts. One comparative
tradition, often called interart studies, with subcategories such as ‘word and
image studies’ and ‘music and image studies’ has been an important sub-
field of both comparative literature and art history. The three arts most
often referred to are literature, the visual arts and music, with their pre-
sumably basic components of words, images and sound. Interart studies

225

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


226 The Borders of Media Borders

basically dealt with the relations between the arts, and the object of research
of such studies has been conceptualized along different lines, but often focus
was placed on, for instance, studies of ekphrasis, of the so-called artistic
Doppelbegabungen, or adaptations from music to poetry. However, under the
general impression of both the basic tenets of Cultural Studies from the
1960s and onwards (the critique of the traditional hierarchy of the arts),

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
and the new trends in artistic and technological products (hybrids in the
arts such as performance and happenings, the development of new digital
media), interart studies has since the 1980s, roughly, been supplanted by
‘intermediality studies’.3
A number of definitions of intermedial studies have been suggested. The
shortest is, I believe, that of Mikko Lehtonen, stating that intermediality
is ‘intertextuality transgressing media boundaries’.4 Intermediality studies is
a rather young field of investigation and consequently the object as well
as the theory and methods of the field are still relatively loosely defined,
and in conferences as well as in publications, the discussions are still at a
fundamental (but often sophisticated) level concerning the basic elements
of the field: what is a medium? What is the difference between interartiality
and intermediality and multimodality? These are questions which show that
it is a discipline still to define its own object and limits.
Despite the relatively weak foundations of the field, several schematiza-
tions have been suggested, among others by Werner Wolf in The Musical-
ization of Fiction: A Study in the Theory and History of Intermediality (1999)
and Hans Lund in Intermedialitet. Ord, bild och ton i samspel [Intermediality.
Word, Image, Sound in Collaboration] (2002).5 Lund suggests a useful divi-
sion where the field of intermediality is divided into combination (with the
subcategories ‘interreference’ and ‘co-existence’), integration and transforma-
tion.6 Interart studies and intermedial studies have created very important
work. Siglind Bruhn, Claus Clüver, Hans Lund, Werner Wolf and numerous
other researchers have shown the necessity of interdisciplinary work as a
revolt against inexpedient academic borders, but these researchers also share
what I choose to call a ‘formalistic vein’ in their work.
Despite the impressive list of results, contemporary intermediality stud-
ies suffers from two problems which I think may be solved by a change
of direction. First of all, by creating a new multimodal theory of medium,
the traditional concept of arts or medium in intermediality studies can be
rethought in order to establish a solid ground for future studies. This is my
main methodological point. Second, and I discuss this in the second part of
my essay, I will argue that in order to avoid intermediality studies remaining
a rather formalistically biased field of study, intermediality studies should
take notice of the ideologically interested trends of modern cultural think-
ing and contemporary philosophy. In order to suggest a solution to the first
problem (creating a definition of medium suitable for future intermedial-
ity studies), I turn to Lars Elleström. In Part II of the essay I return to the
question of intermediality and ideology.

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Jørgen Bruhn 227

Defining medium
The important strategic move from interart studies to intermediality studies
consisted, as mentioned above, in the broadening of the field of investiga-
tion from the traditional arts to, in principle, all existing media. Therefore,
intermediality studies can analyse the entire traditional field of aesthetic
objects as well as stamps, corporate logos, advertising, the medieval Chris-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
tian mass and the opening of the Olympic Games.7 Defining ‘medium’ has
proved difficult, however. ‘Curiously’, Werner Wolf has noted, ‘problems of
definition and typology have not hindered intermediality research. The most
obvious among these is the problem of defining “medium” itself.’8 Wolf
suggests

a broad concept of medium: not in the restricted sense of a technical or


institutional channel of communication but as a conventionally distinct
means of communication or expression characterized not only by partic-
ular channels (or one channel) for the sending and receiving of messages
but also by the use of one or more semiotic systems.9

At first view this is an attractive definition because of the pragmatic idea


that medium should be defined by ‘conventions’. The definition tends to be
rather conservative, however, because it ends up saying, basically, that media
is the same as the arts and that leaves the problem unresolved.
Elleström’s definition of media, described in detail elsewhere in this vol-
ume, consists of a mixture of modalities. In his model, the problematic
essentialism (theatre is x, painting is y) is avoided, and instead an open and
mixed construction is proposed. As shown in his contribution to this vol-
ume, Elleström operates with four necessary conditions for every medium:
(1) Material modality concerning the material manifestation of the medium;
(2) Sensorial modality concerning the human sensory channels affected by
the medium; (3) Spatiotemporal modality concerning the time-space ratio of
the medium and (4) Semiotic modality concerning types of signification using
Peirce’s distinction between iconic, symbolic and indexical signification. The
four necessary conditions (modalities) constitute what Elleström calls ‘basic
media’, such as still image, written words, oral words and organized sound.
These ‘basic media’ will, however, enter into more elaborate culturally and
aesthetically conventional forms, which Elleström chooses to call ‘qualified
media’.

The implications of a multimodal concept of medium


The idea that every medium consists of a number of elements called
basic modalities common to all media (albeit always in specific, concrete
constellations) means that these modalities (in a particular combination)
also form the basis of other media. Consequently, multimodality is a fact
of any conceivable text in any conceivable medium. The idea that texts are
mixed is of course banal when dealing with openly mixed media such as the

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


228 The Borders of Media Borders

mixture of sound, image, words, music in modern cinema or the pictures


and words in the picture book. The new and less obvious insight is that the
mixed character of texts is also a fact of the texts and media which have
traditionally been considered pure, without traces of other media. The main
point is that even the apparently monomedial text always consists of several
modalities.

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
An example may clarify my point: normally, literature has entered inter-
mediality studies via research on traditional ekphrasis, iconic projection
(Hans Lund10 ) adaptation theory or the ‘musicalization of fiction’ (Wolf).
My point, developing Elleström’s suggestions, would be that any literary text
shares modalities with other media – not only the obvious and important
cases mentioned here. Let me exemplify with Iain Banks’ The Wasp Factory
(1984), a gothic horror story set in Scotland. Traditional intermedial stud-
ies would probably dismiss this text as being out of reach of an intermedial
analysis, but a multimodal-media model would be able to identify relevant
aspects of such a text. Here I choose not to focus on the (short) descriptions
of music in the novel (ekphrasis of music) or the descriptions of the land-
scape (close to ekphrasis proper): nor will I stress the passages that may have
been signs of a ‘filmization’ of literature, or the crucial passages where the
protagonist’s father turns out to behave like not only a tyrannical patriarch
but also an artist who has chosen to use his own son as, literally, a model
to be sculpted. Instead I quote the final lines of the novel: ‘Poor Eric came
home to see his brother, only to find (Zap! Pow! Dams burst! Bombs go off!
Wasps fry: ttssss!) he’s got a sister’.
The prose here would not enter any of the traditional categories of inter-
mediality, but I maintain that the passage by way of its multimodality is
necessarily what I will define below as a heteromedial text. In this brief
excerpt I would stress the obvious sonoric aspects of the sequence ‘(Zap! Pow!
Dams burst! Bombs go off! Wasps fry: ttssss!)’. It is a kind of onomatopo-
etikon mixed with ‘normal’, non-iconic prose. The literary text (which is
supposed to consist of symbolic signs (in Peirce’s systematization)) suddenly
approached the sound-iconic state, thus using traits of sonorous art forms
such as music, or spoken poetry, or performance and theatre. I would also
refer to the parenthesis in this fragment of sounding and silent words: a
parenthesis is of course a symbolic sign signifying a kind of double textual
dimension inside the text and as such is a familiar trait in (written) litera-
ture.11 A pair of parentheses is also an iconic visual sign: we can see how
it effectively fences in and bars off the passage from the surrounding text,
so that we read the symbolic sign as well as see the iconic sign, probably
because the text wishes to operate in two different dimensions at the same
time. Even the added italics and repeated consonants of ttssss create a kind
of mixture of visual and sonorous and symbolical sign production.
This example is meant to illustrate my main point, namely that the pure,
distinct medium, and the equivalent to this on the level of specific texts, is a

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Jørgen Bruhn 229

historical as well as an ontological illusion. Such a pure medium or text has


never existed, and it even appears to be a logical impossibility. I think that
this might be what W. J. T. Mitchell has in mind when he claims that ‘the
attempt to grasp the unitary, homogeneous essences of painting, photogra-
phy, sculpture, poetry, etc., is the real aberration’ and that the conception
of purity and unity of media ‘is both impossible and utopian’ and therefore

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
media ought not to be investigated as an existing fact but as the result of
ideological construction and evaluation.12 Pure media may be, and has been,
an ideal in specific historical periods, and in particular ideological surround-
ings, but it is never a real, existent phenomenon. Consequently, research –
and teaching – should take as its starting point the fact that ‘all arts are
“composite” arts (both text and image); all media are mixed media, combin-
ing different codes, discursive conventions, channels, sensory and cognitive
modes’.13 Thus the implications of a new, multimodal concept of medium
is that interartial and intermedial models are transgressed and the meetings
of media are no longer reserved for the privileged exceptions but become a
condition of every text.

Defining ‘heteromediality’?
This new, multimodal definition of medium raises not only a number of
analytical and epistemological questions but also a basic terminological
question: is ‘intermediality’ still the best term to describe the multimodal
character of all media and, consequently, the a priori mixed character of
all conceivable texts? The term intermediality is too limited to satisfy the
demands of the new multimodal theory of medium. Therefore I will sug-
gest a new umbrella term, heteromediality, to describe any conceivable text,
whereas I reserve the term intermediality to parts of heteromediality.
With ‘heteromediality’ the focus shifts from the comparisons between
media and art forms, roughly consisting of the numerous possibilities sug-
gested in a diagram by Lund, for instance, where music represents poetry,
novel becomes movie, words and picture combine on the poster and so
on, to a method investigating the expanded field of media relations (in
Elleström’s terminology: media modalities) inside the text. With hetero-
medial studies we shift the focus from relations between media (always
concretized in forms, in ‘texts’), to medial relations within texts, in other
words. Therefore the prefix ‘hetero’ (Greek, ‘different’ or ‘other’) is more
suitable than ‘inter’ for these investigations.
The term heteromediality raises another terminological problem, though,
because the term is already in use in Werner Wolf’s schematization of inter-
medial relations. Wolf does not use ‘heteromediality’ in The Musicalization
of Fiction: A Study in the Theory and History of Intermediality from 1999, but
in a more recent article, in the Routledge Encyclopedia of Narrative Theory,14
‘heteromediality’ occupies a minor (and purely separating) role and is
defined as follows: ‘Intermediality, in contrast [to intertextuality], applies

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


230 The Borders of Media Borders

in its broadest sense to any transgression of boundaries between media and


thus is concerned with “heteromedial” relations between different semi-
otic complexes or between different parts of a semiotic complex’ (p. 252).
Heteromedial in Wolf’s definition can thus be translated simply as ‘different
media’ in contrast to traditional text-to-text intertextuality, which Wolf calls
a ‘homomedial’ relation (p. 252).15

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
My definition of the term differs clearly from Wolf’s. With the prefix
‘hetero’ I wish to signal the internal mixed character being a condition of
any text, as opposed to ‘inter’ signalling ‘media transgressing borders’. My
proposal, in other words, aims at creating a new, universal concept of text,
the heteromedial text. However it will be useful to operate with pragmatic
subcategories of the heteromedial text, in order not to create one monolithic
concept in danger of not explaining anything at all. Therefore, there will
still be a point in using the terms intermedial text (examples of which are
listed in Lund’s diagram, where media are defined as conventionally distinct
forms in contradistinction to the multimodal definition) and intertextual-
ity (the theoretical concept designating the fact that all text are dialogically
connected but without in this concept taking the specific media-specificities
into consideration). Intertextuality defines the overall phenomenon of texts
being mosaics of other texts (according to the now classic definitions of
Barthes and Kristeva); heteromediality defines the existence of several medial
modalities in all conceivable texts. Intermediality, then, is my term for one
particular subgenre of heteromediality, characterized by the traces of more
than one medium (either in combination, transformation or integration fol-
lowing Lund). Therefore all texts are heteromedial, and they will always
cite and will be cited by other texts (intertextuality); but only part of the
immense category of heteromediality is intermedial in the restricted sense
of the word.
In the following I use the term heteromedial studies for intermedial stud-
ies as such, thus piously hoping that my proposed terminology may prove
useful; when I use the term intermedial studies or intermediality I refer to
current or previous research.

Part II

Heteromediality and ideology


Until now I have described and discussed the development and inherent
problems of interart studies (the hierarchical and traditional system of the
arts) and intermediality studies (the problematic definitions of medium and
the tendency to diminish the real extension of intermedial relations) and I
have suggested a multimodal-media model that might constitute the basis
for what I propose to call heteromedial studies. Perhaps the most crucial
question for future heteromediality studies, however, concerns the interest-
ing and even necessary discussion around the Erkenntnisinteresse (cognitive

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Jørgen Bruhn 231

interest) of the field: What kind of knowledge does the ‘discipline’ want to
produce?
When I consider the present state of intermediality studies, it seems to
me that two possible alternatives for heteromediality studies arise. On the
one hand, I see a rather formalistically focused alternative which might be
described as a sophisticated development of the New Criticism idea of the

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
text or the work. Within the frame of this first alternative, heteromediality
studies can try to strengthen the analytical foundations of traditional inter-
mediality studies in order to establish itself as a conventional, respectable
discipline. This might happen through a sober and scientifically valid con-
struction of a methodology and an acknowledged object of investigation.
Such a direction would develop heteromediality studies into a genuinely
analytic and historically informed discipline based on solid, formalistic
investigative goals.
As far as I can see, this has been the underlying ambition of a number of
eminent researchers of intermediality studies who combine an impressive
philological or musicological training with an intermediality perspective.
Their analysis, based on a semiotic framework, is usually directed towards
relations between the inner working and the outer surface of specific, rel-
atively autonomously conceived works of art. This first line of research I
choose to call the ‘formalistic’ line of intermediality studies.16 Heterome-
diality studies might choose another direction, however. This will be the
case if heteromediality researchers engage with positions in modern cultural
theory and philosophy which hold particular interests in the ideological
dimensions of cultural production and reception.
Investigations into and reflections concerning the ideological and political
aspects of form have been crucial to a number of the discussions in literary
theory in the twentieth century, with inspiration from German neo-Marxism
and British Cultural Studies,17 while more contemporary trends such as New
Historicism and post-colonial studies have been important movements con-
cerning ideology and the arts. Both movements have designed sophisticated
theories of the text’s relation to the surrounding world, and both tend to
focus on the inner contradictions of literary texts. These contradictions are
considered to be (often unconscious) expressions of the surrounding soci-
ety’s ideological tensions and constellations of power. Kiernan Ryan claims
that the deepest aim of New Historicism ‘is to dethrone and demystify
the privileged literary work: to destroy its immunity to infection by cir-
cumstance and other kinds of texts and to rob it of political innocence
by exposing its discreet commitments, its subtle collusions in the cultural
struggle for power’18 – an apt description for a future heteromedial research
strategy, I would say.
Behind many of the analytic and theoretical endeavours in Anglo-Saxon
theory we find traces of modern French thinking, and this magnificent tradi-
tion of considering art as an ideological fact should enter the reflexive space
of future heteromediality studies. Heteromedial studies should engage with

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


232 The Borders of Media Borders

crucial names in what may be called the (heterogeneous) tradition combin-


ing art with a wider ‘critical’ context concerning ideological questions of
gender, class, race or struggles between different aesthetic positions. Such a
‘tradition’ may be said to consist of the theoretical writings and the impor-
tant concepts of the generations following after Jean-Paul Sartre and Georges
Bataille. There is a long list of thinkers deeply influenced by psychoanalyt-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
ical thinking, phenomenology and critical theory, and common to these
thinkers is a capacity to work across established academic and epistemologi-
cal boundaries (psychology, philosophy, literary theory and so on) as well as
analysing freely across the traditional boundaries of the arts. Aspects of the
work of these thinkers ought to be incorporated into heteromediality stud-
ies to such a degree that every reflection on aesthetic questions also entails
a critical dimension (in the sense of the word mentioned above) in order to
avoid heteromediality becoming a formalistic, descriptive discipline.
My suggestion is that the idea of the inherently mixed conception of
medium (leading to my concept of heteromediality) must be combined with
the ideological aspect. Therefore my credo directing my own way of prac-
ticing heteromedial studies is that the particular constellation of mixed media
in a text often expresses a tension which in a more or less opaque way relates to
the historical context of the text in question. This, I think, is what heteromedi-
ality as a critical concept would be able to facilitate in the study of cultural
history, and the distinctive traits of heteromediality studies would be the
urgent knowledge of the mixedness of media as well as an investigation of
the conflicts or tensions creating the particular mixture.

Cook and Mitchell


I would like to point to a few fruitful positions (outside the self-confessed
intermediality researchers) that might help to establish a sound and strong
critical version of future heteromediality studies, namely the work of W. J. T.
Mitchell and Nicholas Cook.
Mitchell combines the lines I have mentioned above: a multimodal-
ity concept of the work of art with an a priori ideological grounding. He
works through a number of relations between different media, in particu-
lar word and image relations, not in order to compare media but because
he wishes to investigate what kinds of non-likeness relations can be estab-
lished between media, and the ideological implications of these relations.
His starting point is that every cultural product is ‘mixed media’ and that this
relation expresses a particular ideological and/or philosophical constellation
connected to the social environment and history of media. At every moment
of history, particular hopes and fears can be detected in the background
of media, and according to Mitchell any conceivable art work expresses a
tension or a stride between media which mirrors a more comprehensive
ideological battle of the social surroundings or the epoch of the work, for

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Jørgen Bruhn 233

instance, connected to ideological conflicts concerning gender, class, eth-


nicity or nationality. Mitchell offers a reconsideration of the often claimed
essential differences between different media and art forms and he manages
to show that the concept of ‘borders’ between ‘pure’ media is a question of
ideology, not a matter of essential definitions.19
Whereas Mitchell has tried to construe an analytical strategy (but never a

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
strict methodology) based on avoiding the lazy comparison and the inven-
tion of a more fruitful relation between verbal and visual elements of texts,
Nicholas Cook has – from a musicologist’s point of view – made a compara-
ble strategic move.20 Cook wishes to contribute to ‘the current reformulation
of music theory in a manner that loosens the grip of the ideology of musi-
cal autonomy – the compulsory (and compulsive) cult of what Peter Kivy
calls “music alone” ’.21 Consequently, Cook wants to tear down the purist
idea of ‘pure’ or ‘absolute’ music. Interestingly, he tries to drive through his
argument by way of focusing on aspects that seem to lie as far away as possi-
ble from ‘absolute’ music, but when doing this he still manages to reach his
main thesis: that absolute music cannot be absolutely pure. Even the purest
musical text is a mixed text, to rephrase Cook’s idea in Mitchell’s terms. Anal-
ysis of the covers of compact discs, television advertisements of cars and
music videos underlines the way that music always, directly or indirectly,
bears traces of the multimodal. Thus, as I would state, Cook demonstrates
the real but suppressed heteromedial quality of musical products. Cook is
fascinated with ‘a hierarchy [of media] whose levels are at war with one
another’.22 A thorough analysis of a music video of the pop singer Madonna
(‘where the effect is to destabilize the hierarchy of media’)23 can in, Cook’s
analysis, achieve far reaching results.

Conclusion: the centrality of marginality


In this essay I hope to have shown how a new, multimodal version of
medium may imply a substantial rethinking of the discipline of intermedial-
ity studies. I propose the term heteromediality to signify that ‘all media are
mixed media’ and that the particular mixing of media has its roots in wider
contexts (ideological, historical, aesthetic and so on). Furthermore, I have
put forward my own suggestion as to how heteromedial studies might be
conducted, namely by way of letting French thinking from the period after
World War II balance the otherwise formalistic tendency in much of the
research that has been done in intermedial studies. The points that I have
been trying to make lead me to the following concluding remarks concern-
ing the ‘position’ of heteromediality studies in the future landscape of the
humanities.
Since the mid-twentieth century, research of the marginalized, the
Other(s), the liminal and the suppressed has, paradoxically, occupied a cen-
tral role in the humanities. The privileged perspective of the marginalized

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


234 The Borders of Media Borders

has been expressed by a number of different scholars, or rather, some schol-


ars show how the marginal may produce important insights into the centre
of a research field. Nicholas Cook’s efficient if also slightly surprising take
on this question, in Analyzing Musical Multimedia (1998), consists, as men-
tioned above, in using an advertisement for a Volvo car or a Madonna music
video as a lever to create a kind of deconstruction of the high-cultural eval-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
uation and understanding of ‘pure’ music. The centrality of the marginal is
almost a trademark of the work of W. J. T. Mitchell. I would like to mention
two instances of this rhetorical and epistemological tendency in his work,
and both examples have gained a central position in academic discussions,
partly following Mitchell’s lead. First I turn to his discussion of ekphrasis in
‘Ekphrasis and the Other’ from Picture Theory. In this text Mitchell claims,
and he might be right, that research into ekphrasis, despite the volumi-
nous size, is still in a ‘minority’ position and that there is a certain amount
of obscurity attached to the term and the research field.24 Despite this, or
should I say because of this, the term and concept of ekphrasis might form
the foundation for a new theory of description in literary studies, nothing
less. The reason is that ekphrasis, itself a description, is typical of all other
representations of reality. In another essay25 Mitchell chooses the concept
‘metapicture’, a picture that reflects upon itself as pictorial statement and
representation, to do the same job. Metapictures are apparently radical pic-
tures in the history of western representations: from Magritte’s pipe picture
to Wittgenstein’s beloved duck-rabbit drawing, to mention just a few. Once
again it turns out that the marginal has a central position: ‘The metapicture
is a piece of movable cultural apparatus, one which may serve a marginal
role as illustrative device or a central role as a kind of summary image, what
I have called a “hypericon” that encapsulates an entire episteme, a theory
of knowledge.’26 The marginal metapicture turns out to be the typical visual
representation.
From the point of view of heteromediality studies these claims for cen-
trality should cause us to consider, once again, whether heteromediality
studies does not deserve a much more privileged position in contempo-
rary, and future, studies in the humanities. If we once again ponder on
Hans Lund’s diagram over relations between the arts one is struck by the
clarity of the division and the comprehensive examples of intermedial rela-
tions, and most teachers of basic courses in intermediality will probably feel
rather satisfied with themselves if their students navigate in a reasonable
way in this diagram. Nevertheless the scheme has one crucial flaw, I think,
namely that it implicitly states that there exist media and art forms that are
‘un-intermedial’. This is not the case, as I have argued above, where I have
referred to the dictum ‘all media are mixed media’ and by naming the study
of this fact ‘heteromediality studies’. When understood in a fruitful way
the idea of heteromediality (that all texts are modally mixed, and that this
impure state implies some kind of ideological tension) might influence not

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


Jørgen Bruhn 235

only academic research but even institutional divisions and the unhealthy
‘compartmentalization’ of the humanities. In other words, Lund’s schemati-
zation establishes intermediality studies as a marginalized field inside the
studies of the humanities, whereas I believe that heteromediality studies
break loose from this position and become exactly the opposite: the basic,
central conceptual scheme underlying all studies in the (aesthetic part of

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
the) humanities.

Notes
1. Parts of this essay were presented at the Imagine Media! conference at Växjö Uni-
versity (from 2010: Linnaeus University), Sweden, in October 2007. An early
version appeared in Tidskrift för litteraturvetenskap (2008) 1. A number of ideas
put forth here have been developed in the collaborative edition of J. Arvidson,
M. Askander, J. Bruhn and H. Führer (eds) (2007) Changing Borders: Contemporary
Positions in Intermediality (Lund: Intermedia Studies Press) and some ideas corre-
spond to ideas put forward in the foreword to Changing Borders, pp. 13–19. I also
wish to thank Lars Elleström, Henriette Thune and Jan Lundquist for valuable
comments on this essay.
2. The succinct definition is from Claus Clüver, ‘Intermediality and Interart Studies’
in Arvidson et al. (2007) Changing Borders, pp. 19–37 at p. 20, note 5.
3. See Claus Clüver’s historical overview, ibid.
4. Mikko Lehtonen (2000) ‘On No Man’s Land: Theses on Intermediality’, Nordicom
Review 4, 71–83 at 71.
5. W. Wolf (1999) The Musicalization of Fiction: A Study in the Theory and History
of Intermediality (Amsterdam and Atlanta, GA: Rodopi); H. Lund (ed.) (2002)
Intermedialitet: Ord, bild och ton i samspel (Lund: Studentlitterature).
6. Hans Lund (ed.) (2002) Intermedialitet, as translated in Changing Borders, p. 15.
7. The last two examples are Claus Clüver’s. Clüver believes that these are highly sig-
nificant (but as yet almost un-analysed) examples of intermediality in contrast to
interart phenomena. See the final remarks in ‘Intermediality and Interart Studies’,
Changing Borders, p. 34.
8. See Werner Wolf’s entry on ‘Intermediality’ in D. Herman, M. Jahn and
M.-L. Ryan (eds) (2005) Routledge Encyclopedia of Narrative Theory (London & New
York: Routledge).
9. Quoted in I. Rajewsky (2002) Intermedialität (Tübingen & Basel: A. Francke Verlag),
p. 7.
10. See Hans Lund (1992) Text as Picture: Studies in the Literary Transformation of
Pictures (Lewiston, Ontario: Edwin Mellen Press).
11. A standard definition of the content of the parenthesis is that it can be omitted
without altering the overall meaning of the sentence.
12. W. J. T. Mitchell (1994) ‘Beyond Comparison’ in Picture Theory (Chicago: Chicago
University Press), pp. 107, 96.
13. Ibid., pp. 94–5.
14. Herman et al. (eds) (2005).
15. In Werner Wolf (1999) The Musicalization of Fiction. Werner Wolf distinguishes
between intertextuality (being a text-to-text relation) and intermediality (being
a ‘special relation between media’) (p. 46). One may appreciate the clear

10.1057/9780230275201 - Media Borders, Multimodality and Intermediality, Edited by Lars Elleström


236 The Borders of Media Borders

distinction, but in my definition of medium such a distinction cannot hold,


simply because any text is per definition never not only textual.
16. For an overview of this tradition, see Rajewsky (2002) Intermedialität. Perhaps my
classification is about to turn into an anachronism: in the present volume both
Siglind Bruhn and Claus Clüver do not confine themselves to semiotic or for-
malistic issues but instead incorporate ideological and religious aspects in their
presentations. Recently, Marion Froger and Jürgen E. Müller (2007) edited Inter-

Copyright material from www.palgraveconnect.com - licensed to University of South Florida - PalgraveConnect - 2011-05-25
médialité et socialite: Histoire et gégraphie d’un concept (Münster: Nodus) which,
despite its title and the interesting contributions, offers little when it comes to
changing the overall direction of intermediality studies (‘socialité’ is more or less
synonymous with ‘history’ or ‘context’).
17. See Simon During’s preface in S. During (ed.) (1999) The Cultural Studies Reader,
2nd edn (London & New York: Routledge), for an overview of the problems
inherent in the different periods of Cultural Studies. For an attempt to combine
Cultural Studies with intermediality studies, see W. Huber, E. Keitel and G. Süss
(eds) (2007) Intermedialities (Trier: Wissenschaftlicher Verlag).
18. R. Kiernan (1996) New Historicism and Cultural Materialism (Oxford and New York: