Vous êtes sur la page 1sur 6

The Art of Interaction

Ernest Edmonds
Creativity and Cognition Studios
University of Technology, Sydney
POBox 123 Broadway
NSW 2007

Interactive art has become much more common as a result of the many ways in which the
computer and the Internet have facilitated it. Issues relating to Human-Computer Interaction are as
important to interactive art making as issues relating to the colours of paint are to painting. It is not
that HCI and art necessarily share goals. It is just that much of the knowledge of HCI and its
methods can contribute to interactive art making. This paper reviews recent work that looks at
these issues in the art context. In interactive digital art, the artist is concerned with how the
artwork behaves, how the audience interacts with it and, ultimately, in participant experience and
their degree of engagement. The paper looks at these issues and brings together a collection of
research results and art practice experiences that together help to illuminate this significant new
and expanding area. In particular, it is suggested that this work points towards a much needed
critical language that can be used to describe, compare and discuss interactive digital art.

Engagement, Art, Interaction.

1. INTRODUCTION As is well known to HCI practitioners, however, we

do not have a simple cookbook of recipes for
Digital art is increasingly interactive. Some of it is interaction and experience design. Rather, we have
built on notions that come from computer games methods that involve research and evaluation with
and much of it is intended to engage the audience users as part of the design process. The
in some form of interactive experience that is a key implications of this point for art practice are, in
element in the aesthetics of the art. themselves, interesting. The art making process
needs to accommodate some form of audience
Issues relating to Human-Computer Interaction research within what has often been a secret and
(HCI) are as important to interactive art making as private activity.
issues relating to the colours of paint are to
painting. This paper reviews recent work that looks The paper looks at these issues and brings
at these issues in the art context. The concerns of together a collection of research results and art
experience design and understanding of user, or practice experiences that together help to illuminate
audience, and engagement are especially relevant this significant new and expanding area. In
ones. We are not so concerned with task analysis, particular, it is suggested that this work points
error prevention or task completion times, however, towards a much needed critical language that can
as with issues such as pleasure, play and long term be used to describe, compare and discuss
engagement. interactive digital art.

In interactive digital art, the artist is concerned with

how the artwork behaves, how the audience 2. INTERACTION AND PERCEPTION
interacts with it (and possibly with one another
through it) and, ultimately, in participant experience Perception is an active process (Norwich, 1982).
and their degree of engagement. In one sense, Even when we stand still and look at the Mona Lisa
these issues have always been part of the artists our perceptual system, the part of the brain behind
world but in the case of interactive art they have the eyes, is actively engaging with the painting.
become both more explicit and more prominent However, we do not change the painting in any
within the full cannon of concern. way. As we look longer it may seem to change and
we sometimes say that we see more in it, but it is
Whilst HCI in its various forms can offer results that our perception of it that is changing. This change
at times help the artist, it seems that the concerns process is most often mentioned in relation to
in interactive art, rather like those in computer works such as those by Rothko where at first it may
games design, go beyond traditional HCI. Hence, seem as if there is nothing much to see but the
we need to focus on issues that are in part new to more we look the more we perceive. Campbell-
or emerging in HCI research. Johnston commented that as you gaze into the
[Rothko] canvases you see that their surfaces are
modulated. Different patterns and intensities and research into human behaviour and this research
tones emerge. (Campell-Johnston, 2008). Marcel may also be something that requires skilled input
Duchamp went so far as to claim that the audience from an expert other than the artist and
completes the artwork. The active engagement with technologist/scientist themselves.
the work by the viewer is the final step in the
creative process. As Duchamp put it, the spectator A significant feature is the nature of the
adds his contribution to the creative act collaboration between artist, researcher and
(Duchamp, 1957). From this perspective, audience technologist. There are many ways in which it can
engagement with an artwork is an essential part of work, but it seems that the notion of the researcher
the creative process. The audience is seen to join and technologist being assistants to the artist is
with the artist in making the work. This position less and less common. Partnerships are often
became a particularly significant one for artists in formed in which the roles are spread across the
second half of the twentieth century. team. Sometimes, for example, a technologist may
be named as a co-author of the work (Candy and
Since the 1960s an increasing number of artists Edmonds, 2002).
have been taking active engagement further. Most
famously, in the period of happenings, direct and
physical audience participation became an integral 3. ART, GAMES AND PLAY
part of the artwork or performance (Stanford,
1995). Situations were set up, by the artists, in that The computer game arose from the technological
the audience were meant to engage by actually opportunities that have emerged. In fact computer
taking part and so explicitly determine the work. games and interactive art often have much in
The artwork itself is changed by the audience. common.
Indeed, the activity of engagement became part of
the artwork. Often with the help of electronics, The intention in a game can be quite different to the
members of the audience were able to touch an intention in an artwork, but both may involve the
artwork and cause it to change. Art became audience/player/user in intense interaction with a
interactive. See, for example, Frank Poppers book computer-controlled device (call it artwork or game)
on the subject (Popper, 2007). Sometimes we talk that is driven by some form of pleasure or curiosity.
about observably interactive art just to be clear that The human, confronted with the artwork (or game)
the interactive activity is not just in someones head takes an action that the work responds to. Typically
but can be seen in terms of movement, sound or a sequence of actions and responses develop and
changing images. continue until a goal is reached or the human is
satisfied or bored. The nature of play, as found in a
Interactive art has become much more common as game, is not infrequently the subject of an artists
a result of the many ways in which the computer interactive work and so game and artwork come
and the internet have facilitated it. The computer, together at times. Although this is no problem for
as a control device, can manage interactive artists, as recently as 2000 it was still a problem for
processes in ways never seen before. Today, we curators. In the UKs Millennium Dome (Millenium
are often hardly aware of the computers that we Dome, 2010) all of the interactive art was shown in
use at all. They operate our watches, our washing the Play Zone and none of it was included in the list
machines, our telephones, our cars and a high of artworks on show. Exhibiting interactive art is still
percentage of the other devices that we use. It is somewhat problematic, but the issues that the artist
not a big step, therefore, to find that the artworks faces go beyond that because their practice has to
that we engage with also sometimes have change in order to deal with interaction.
computers behind them.
In the context of making interactive art, Costello
There is another area in which interaction, or at has argued that the nature of play can best be
least the use of computers, has brought changes to understood through a taxonomy that she has
creative practice. The complexity of computer termed a pleasure framework (Costello, 2007).
systems and the many sub-areas of specialist She has synthesized a collection of research
knowledge required for their full exploitation have results relating to pleasure into thirteen categories.
increased the need for collaboration by the artist She describes these categories as follows:-
with others. The artist today is often a member of a
collaborative team and the role artist is even Creation is the pleasure participants get from
shifting to be applicable to the whole team or at having the power to create something while
least beyond one individual. A technical expert, for interacting with a work. It is also the pleasure
example, may often make creative contributions participants get from being able to express
and may, as a result, be named as a co-author of themselves creatively.
the resulting artwork. The collaboration may not be Exploration is the pleasure participants get from
limited to technical matters. There is a need for exploring a situation. Exploration is often linked
with the next pleasure, discovery, but not always. however, other layers may add depth to the
Sometimes it is fun to just explore. experience. The boundaries between games and
Discovery is the pleasure participants get from art can be very grey and, for the purposes of this
making a discovery or working something out. paper, it may be assumed that the complete
Difficulty is the pleasure participants get from art/game gamut is often best seen as one.
having to develop a skill or to exercise skill in order
to do something. Difficulty might also occur at an
intellectual level in works that require a certain 4. ART AND EXPERIENCE DESIGN
amount of skill to understand them or an aspect of
their content. In making interactive art, the artist goes beyond
Competition is the pleasure participants get from considerations of how the work will look or sound.
trying to achieve a de-fined goal. This could be a The way that it interacts with the audience is a
goal that is defined by them or it might be one that crucial part of its essence. The core of the art is in
is defined by the work. Completing the goal could the works behaviour more than in any other
involve working with or against another human aspect. The creative practice of the artist who
participant, a perceived entity within the work, or chooses this route is, therefore, quite different to
the system of the work itself. that of a painter, for example. A painting is static
Danger is the pleasure of participants feeling and so, in so far as a painter considers audience
scared, in danger, or as if they are taking a risk. reaction, the perception of colour relationships,
This feeling might be as mild as a sense of unease scale, figurative references and so on will be of
or might involve a strong feeling of fear. most interest. In the case of interactive art,
Captivation is the pleasure of participants feeling however, it will be the audience response to the
mesmerized or spellbound by something or of works behaviour that will be of most concern.
feeling like another entity has control over them. Audience engagement will not be seen in terms of
Sensation is the pleasure participants get from the just how long they look. It will be in terms of what
feeling of any physical action the work evokes, e.g. they do, how they develop interactions with the
touch, body movements, hearing, vocalising etc. piece and so on.
Sympathy is the pleasure of sharing emotional or
physical feelings with something. A painter might not explicitly consider the viewer at
Simulation is the pleasure of perceiving a copy or all. It is quite possible to paint a picture by only
representation of some-thing from real life. considering the properties of the paint, the colours
Fantasy is the pleasure of perceiving a fantastical and the forms constructed with them. In an
creation of the imagination. interactive work, on the other hand, as behaviour is
Camaraderie is the pleasure of developing a sense central to its very existence, the artist can hardly
of friendship, fellowship or intimacy with someone. ignore audience engagement within the making
Subversion is the pleasure of breaking rules or of process. This is where the most significant
seeing others break them. It is also the pleasure of implications of interactive art for creative practice
subverting or twisting the meaning of something or lies. As we know from the world of HCI, reliable
of seeing someone else do so. predictions of human behaviour in relation to
interactive systems are not available, except in
For further discuaaion, see Costello and certain very simple cases. Observation, in some
Edmondss paper (Costello and Edmonds, 2007). sense, of an interactive system in action is the only
Each of the categories of pleasure represents a way to understand it. Consider, for example, the
form of interaction with its own characteristics. issues identified in Costellos categories described
Each has to be considered in its own way, above. The artist has to find ways of incorporating
providing a context in which appropriate interaction observation of some kind into practice. This is an
design decisions can be made. In Costellos work, extension of the role of research in practice.
the framework has been applied in the design and
development of interactive artworks. For her, play A significant feature of the increasing role of
and pleasure formed the goals of the artwork or, at research has been the need for artists to try their
least, the nature of the interactive experience being works out with the public before completion.
addressed (Costello, 2009). Because an interactive work is not complete
without participants and because the nature of the
The subject of the art in such cases is play and interactive experience may depend significantly on
pleasure and the works engage the audience in context, an artist cannot finish the work alone in the
playful behaviours. The aesthetic results, of-course, studio. This can be seen as a problem in that
may be important in other respects. Art is many- showing a half finished work may be quite
layered and we certainly must not assume that the unattractive to the creator, however there seems to
significance of playful art is limited to play itself. In be no easy way out of the situation.
games, on the other hand, the top level of interest
may represent the point of the system. Even then,
An example of an approach to dealing with the emphasise the design of the interface, as the early
problem is Beta_Space. The Powerhouse Museum HCI work used to do, but looks as human
Sydney and the Creativity and Cognition Studios, experience and how the design of the behaviour of
University of Technology, Sydney have the system influences it.
collaborated to create Beta_Space, an
experimental exhibition environment where the One specific common area of interest between
public can engage with the latest research in art interactive art and experience design research is
and technology. It shows interactive artworks in engagement. Do people become engaged with the
development that are ready for some kind of artwork? Is that engagement sustained? What are
evaluation and/or refinement in response to the factors that influence the nature of the
participant engagement. The works shown are at engagement? Does engagement relate to pleasure,
different stages, from early prototype to end frustration, challenge or anger, for example? Of-
product. In all cases engagement with the public course, the artist can use themselves as subject
can provide critical information for further iterations and rely on their own reactions to guide their work.
of the artwork or of the research (Edmonds, Bilda Much art is made like that, although asking the
and Muller, 2009). Evaluation methods drawn, in opinion of expert peers, at least, is also normal.
various ways, from Human-Computer Interaction However, understanding audience engagement
are employed to provide the artist with a valuable with interactive works is quite a challenge and
understanding of their work in action. There are a needs more extensive investigation than
number of different perspectives that need to be introspection.
taken into account, including artist, curator and
researcher (Muller, Edmonds and Connell, 2006). Bilda has developed a model of the engagement
The key step has been to incorporate HCI research process in relation to audience studies with a range
into the interactive art making process. of artworks in Bela_Space (Bilda, Edmonds and
Candy, 2008). The process is illustrated in figure 1.


As above, one important area that contributes to

creative practice in art is HCI, or interaction design
in particular. As with gaming, it is not that HCI and
art necessarily share goals. It is just that much of
the knowledge of HCI and, perhaps more
significantly, its methods can contribute to
interactive art making. From HCI we know how
easy it is for a designer to shape software in ways
that seem easy to use to them but that are a
mystery to others. It is normally seen as an issue of
distinguishing between the model of the system
held by the various players: programmer, designer Figure 1. Model of engagement: Interaction modes
and user (Norman, 1988). Such confusion often and phases
happens when the designer makes an unconscious
assumption that is not shared by others. For Note that the engagement mode shifts in terms of
example, when an item is dragged over and audience interaction from unintended actions
dropped on a wastebin icon, it will normally be through deliberate ones that can lead to a sense of
made ready to be deleted but retained for the control. In some works it moves on into modes with
moment. People new to computers sometimes more exploration and uncertainty. Four interaction
assume that it is lost forever and so are nervous phases were identified; adaptation, learning,
about using it, leading to behaviours unexpected by anticipation and deeper understanding.
the designer. The same kind of thing can happen
with interactive art. The artist may or may not mind Adaptation: Participants adapt to the changes in
but they do need to be aware of such issues and the environment; learning how to behave and how
make conscious decisions about them. to set expectations, working with uncertainty. This
phase often occurs from unintended mode through
There is a growth area in HCI research and to deliberate mode.
practice known as experience design, as
discussed, for example, by Shedroff (Shedroff, Learning: Participants start developing and an
2001). This is particularly important because it internal/mental model of what the system does, this
represents a collection of methods and approaches also means that they develop (and change)
that concentrate on understanding expectations, emotions, and behaviours, accesses
audience/participant/user experience. It does not memories and beliefs. In this phase the participant
interprets exchanges, explores and experiments aspect of engagement might typically be found in
relationships between initiation and feedback from the learning phase of Bildas model.
the system. Therefore they develop expectations
on how to initiate certain feedback and Another form of engagement is one that extends
accumulates interpretations of exchanges. This over long periods of time, where one goes back for
phase can occur from deliberate mode to repeated experiences such as seeing a favourite
intended/in control mode. play in many performances throughout ones life.
These relaters are factors that enable the hot spot
Anticipation: In this phase, participants know what to remain hot on repeated visits to the exhibition. A
the system will do in relation to initiation, in other good set of relaters meet the highest approval in
words they predict the interaction. Intention is more the world of museums and galleries. This aspect of
grounded compared to the previous phases. This engagement might typically be found in the deeper
phase can occur from deliberate to intended/in understanding phase of Bildas model. We often
control mode. find that this long-term form of engagement is not
associated with a strong initial attraction.
Deeper understanding: Participants reach a more Engagement can grow with experience. These
complete understanding of the artwork and what issues are ones that the interactive artist needs to
his or her relationship is to the artwork. In this be clear about and the choices have significant
phase participants judge and evaluate at a higher, influence on the nature of the interaction employed.
conceptual level. They may discover a new aspect We saw above that Costello, for example, takes a
of an artwork or an exchange not noticed before. particular (but not exclusive) interest in sustainers
This phase can occur from intended/in control of engagement in her art. A description of a
mode to intended/uncertain mode. process of developing an artwork in order to
encourage engagement has been given by this
Comparing these phases with the pleasure author (Edmonds, 2006).
framework discussed above, we can see that the
categories may be most likely to be found in Most artists would probably say that they aimed for
different phases. For example, discovery might be their work to encourage long-term engagement with
common in the learning phase, whilst subversion their audience. Much interactive art, however,
might be more likely in the later phases. seems to emphasise attraction and immediate
engagement. Why is this? There are two possible
In designing for engagement, the artist needs to reasons for the focus on the immediate. One is the
consider where they sit in this space and what kind seductive appeal of direct interaction that has been
of engagement or engagement process they are so powerfully exploited in computer games. There
concerned with. is no doubt that the model of the game is
interesting. However, it also represents a challenge
There are many forms of engagement that may or to the artist taking the long-term view. How is the
may not be desired in relation to an artwork interactive artwork going to retain its interest once
(Edmonds, Muller and Connell, 2006). For the initial pleasure has worn off? An answer may
example, in museum studies people talk about be implied in the second reason for the emphasis
attractors, attributes of an exhibit that encourage on the immediate, which is an emphasis on the
the public to pay attention and so become action-response model of interaction discussed in
engaged. They have attraction power, in Bollo the next section..
and Dal Pozzolos term (Bollo and Dal Pozzolo,
2005). In a busy public place, be it museum or bar,
there are many distractions and points of interest. 6. CONCLUSION
The attractor is some feature of the interactive art
system that is inclined to cause passers by to pay So where has this discussion led us? By drawing
attention to the work and at least approach it, look from the HCI and psychological work on interaction
at it or listen for a few moments. we can begin to develop a critical language that
can enable discussion of interactive art and can
The immediate question arises of how long such provide a framework that informs creative practice
engagement might last and we find that the in the area. Whereas a painter might be able to
attributes that encourage sustained engagement think in terms of hue, texture and so on, the
are not the same as those that attract. Sustainers interactive artist also needs to think in terms of
have holding power and create hot spots, in Bollo forms of engagement, behaviours etc. Colour, for
and Dal Pozzolos term. So, presuming that the example, is hard enough, but we know much more
attractors have gained attention, it is necessary to about that than about interaction and so the role of
start to engage the audience in a way that can research, in some form, within creative practice
sustain interest for a noticeable period of time. This involving interaction becomes significant.
Interactive art is as valid as any other form. In Conference on Arts and Cultural
making it, the artist deals with the same issues and Management,Montreal,
faces much the same challenges as in any other Campbell-Johnston, R. (2008.) Mark Rothko at
kind of art. However, each form and each medium Tate Modern. The Times, London September 24,
has its own set of specific problems and this one is 2008.
no exception. Interactive behaviour and http://entertainment.timesonline.co.uk/tol/arts_and_
engagement are key. For the artist, it is not entertainment/visual_arts/article4811134.ece.
necessarily a matter of coming to clear [accessed 22.3.2010]
understandings, however. It might equally be a Candy, L and Edmonds, E. A. (2002). Explorations
matter of providing the kind of challenge to our in Art and Technology. Springer-Verlag, London.
beliefs and assumptions that makes understanding Costello, B. (2007). A Pleasure Framework.
even harder than we thought. Leonardo, 40 (4). 370-1.
Costello, B. (2009). Gestural Interfaces that
We see that a range of audience experience issues Stimulate Creative Play. PhD Thesis. University of
are important for the interactive artist and that Technology, Sydney
research into them is a significant part of the art Costello, B., Edmonds, E.A. (2007). A Study in
making process. A range of these issues have Play, Pleasure and Interaction Design, in
been identified, including a set of pleasure Proceedings of Designing Pleasurable Products
categories, an articulation of a developing and Interfaces, (Helsinki, 2007), ACM Press, NY
engagement process and different kinds of NY, pp76-91
engagement over different periods of time. Artists Duchamp, M. (1957) The Creative Act Talk given
are actively exploring both these factors and new in 1957 reprinted in The Essential Writings of
methods that can be employed as part of artistic Marcel Duchamp, editors Sanouillet, M. and
practice in order to deal with them. It is suggested Peterson, E. Thames and Hudson, London. 1975.
that researchers in HCI and, in particular 138-140.
experience design, might usefully consider these Sandford, M. (1995). Happenings and Other Acts
concerns within art to see to what extent they might (Worlds of Performance) Routledge, NY NY.
contribute to the broader study of interaction, user Popper, F. 2007. From Technological to Virtual Art.
and audience engagement. MIT Press, Cam, MA.
Edmonds, E. A. (2006). Abstraction and
Of particular interest from an art world point of Interaction: An Art System for White Noise.
view, is that we can see the lists of issues that have Computer Graphics, Imaging and Visualisation -
been presented as the beginnings of a language Techniques and Applications, Ebad Banissi, et al
with which to discuss the characteristics of (eds). IEEE Computer Society Conference
interactive artworks, the intentions of the artists and Publishing Services. Los Alamitos, CA. 423-7.
the reactions of audiences. The work described in Edmonds, E. A., Bilda, Z. & Muller, L. (2009). Artist,
this paper, therefore, has the potential to go evaluator and curator: three viewpoints on
beyond its immediate implications for art practice interactive art, evaluation and audience experience.
and experience design research. It could be used Digital Creativity, 20, 141 - 151.
as the basis for the development of a critical Edmonds, E. A., Muller, L, and Connell, M. (2006).
framework that extends visual art analysis to fully On Creative Engagement, Visual Communication.
embrace interactive digital art. The same 5 (3). 307-22.
framework will be valuable in experience design. Millenium Dome. (2010)
The potential of the work discussed here goes [accessed 22.3.2010]
beyond the scope that space allows for this paper Muller, L., Edmonds, E. A., Connell, M. (2006).
and must be left as the subject of later publication. Living Laboratories for Interactive Art. CoDesign:
International Journal of CoCreation in Design and
the Arts, 2(4), 195207.
7. REFERENCES Norman, D. (1988). The Design of Everyday
Things. Doubleday, NY NY.
Bilda, Z. Edmonds, E. Candy, L. (2008). Designing Norwich, K. H. 1982. Perception as an Active
for creative engagement, Design Studies, 29, Process Mathematics and Computers in
Issue 6, 525-40. Simulation. Volume 24, Issue 6, 535-53
Bollo, A. and Dal Pozzolo, L. (2005). Analysis of Shedroff, N. (2001). Experience Design. New
Visitor Behaviour inside the Museum: An Empirical Rider, Berkeley, CA.
Study, Proceedings of the 8th International