Blending Realities in Game Space

Blending Realities in Game Space
TROY INNOCENT Center for Electronic Media Art, Monash University
Modes of representation in digital games are explored in terms of game aesthetics, structure, and logic. A model for world-making is developed that identies key components of digital media language and their relationship to shifting modes of representation. These ideas are demonstrated through three art works created by the Idea-ON>!, Iconica, and Semiomorph. The nal work, Semiomorph, combines the theory of semiotic morphism with gameplay to create a digital game that generates blended realities. This work is described and analyzed in detail to illustrate the proposed model. Categories and Subject Descriptors: H.5.1 [Information Interfaces and Presentation]: Multimedia Information SystemsArticial, augmented, and virtual realities; I.2.11 [Articial Intelligence]: Distributed Articial IntelligenceMultiagent systems; J.5 [Computer Applications]: Arts and HumanitiesFine arts General Terms: Experimentation Additional Key Words and Phrases: Language of computers, semiotic morphism, digital iconography, ludology, media arts, digital games, articial life ACM Reference Format: Innocent, T. 2008. Blending realities in game space. ACM Comput. Entertain. 6, 3, Article 35 (October 2008), 15 pages. DOI = 10.1145/1394021.1394028 http://doi.acm.org/10.1145/1394021. 1394028
35
1. INTRODUCTION Electronic spaces can manifest complex relationships between their representation and the underlying system that denes them. Elements such as iconography, synthetic materials, skins, and emulated media are used to represent the virtual world built in electronic space. Playing with the relationship between the representation and the system develops its capacity as a dynamic, expressive generator of new meanings. This opens up opportunities for the exploration of virtual world not as simulation of the real, but as a medium for the development of other coherent, alternative worlds whose representation is uid and
Authors address: Centre for Electronic Media Art, Monash University; email: Troy.Innocent@ artdes.monash.edu.au Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or direct commercial advantage and that copies show this notice on the rst page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specic permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212) 869-0481, or permissions@acm.org. C 2008 ACM 1544-3574/2008/10-ART35 $5.00 DOI = 10.1145/1394021.1394028 http://doi.acm.org/ 10.1145/1394021.1394028
ACM Computers in Entertainment, Vol. 6, No. 3, Article 35, Publication date: October 2008.
35:2
Troy Innocent
mutable. The intersection of the audiovisual language of computers, articial life, generative systems, and virtual world design combines this uid representation with the denatured logic of the computer. The dynamic and engaging nature of game space makes it an ideal site for the investigation of these themes. Such ideas have been explored in a series of worlds that are structurally different, but they all connect uid representations of electronic spaces with gameplay, interactivity, and system design. Artefact: Semiomorph is a digital game that explores semiotic morphism, a systematic translation between sign systems in which signied messages can be mapped onto various signiers, multiplying and mutating instances of semiosis. Iconica is an articial world made of language, where iconic elements are the basic building blocks of a world literally made of language. Six elements from this language relate to unique pictorial styles and soundscapes used to represent the world, ranging from plastic knowbots and surreal iconography to electronic abstraction and the dirt of the real world. Finally, Idea-ON>! is a database of ideas and experiences contained within a collection of virtual worlds that explore the nature of electronic space. This investigation engages with digital games on a number of levels. The unique aesthetics of games are explored through an exploration of their iconography, articiality, and different modes of representation. Game logic as it is expressed in articial intelligence and behavioral systems drives the actions of the entities that populate these worlds. Finally, game spaces are rich in the endemic properties and characteristics of electronic space that inform the language of computers. An approach to the creation of new media art based on structures, strategies, and aesthetics of digital games has been developed for exploring the potential of these areas. This approach works towards discovering opportunities for communication and expression that are unique to the medium of electronic space. Three major components of this approach will be articulatedthe art practice of world-making, the nature of the computer as emulator/shape-shifter, and gameplay/modeling as a generative meaning system. 2. WORLD-MAKING Making digital games can be seen as an extension of the practice of worldmaking. Artists may take on the role of world buildercreating coherent alternative worlds from patterns of information. Steven Holtzman describes this practice in terms of how via a system of symbols to represent a view of reality, the artist shares his consciousness of aspects of that reality. [Holtzman 1994]. The representations of these worlds are made of meshes, geometric primitives, textures, materials, sound, music, text, graphics, animation, and other media elements. However, they are also dened in terms of the relationships between these elementstheir behavior, spatial location, connection to parameters in the world, the meaning each representation is intended to signify, and so on. The user/player is factored into this system through their agency and affect on the virtual world. This role may range from passive observer to active engagement. This multilevel system offers rich opportunities for new models of communication and expression.
35:3
In the construction of a virtual world, a number of decisions need to be made so that it works as a simulation. It is impossible from a computation and design point of view to model every aspect of reality in a virtual world. So particular aspects of reality need to be selected and simulated in that world. The result is a subset of reality that emphasizes a particular experience or way of being. A driving game is focused on the experience of driving, a rst-person shooter on hunt and kill type behavior, and so on. Factoring gameplay into the practice of world-making extends this practice by involving the player/user into the experience. Virtual worlds that set up situations and allow the player to act out scenarios are naturally quite different than static, virtual worlds that enable the navigation of a single point of view through space. The rules of the game set up the potential for many different outcomes in the virtual world. Conventions and rules for the appearance, placement, and behavior of virtual objects in space are beginning to emerge. Spatial cues suggest pathways for navigation through worlds. The player is addressed as an agent in the narrative through techniques for structuring space and time. Iconography works across mediaimage, sound, and action. All of these represent new ways of signifying meaning, and comprise part of digital media language. Formalising and dening electronic space seems to be an impossible task. Every discovery leads to another that opens up new possibilities, every limitation that is understood reveals further glitches in the system. In my exploration of electronic space, I have focused on creating personal spaces that are particular to my own intuitions and experience. The collection of icons, sounds, game characters, and spaces represent a personal language in response to the themes that have motivated the work. In terms of perception, these worlds can be seen as being real, symbolic, or in-between. They may be seen real, in that they are represented using all perceptual cues of real experience such as perspectival viewpoint, spatialized sound, immediate feedback, light, and other phenomena. However, they are constructed from codesymbols and relationships between symbolsthat is represented by such realistic simulation. Hence they could be described as the symbolic made real: the world model/abstraction that forms the underlying system is experienced in a mode of perception usually activated by the real world. In this way, the idea of experiences and worlds that were only possible in theory, dreams, and the human imagination are actualized in the abstract space of the computer. Factoring the computer into the process of generating these spaces extends this to the potential expression of a nonhuman imagination as well. 3. REPRESENTATION: SURFACE/STYLE/SKINS Looking deep into the structure of the computer, it is widely recognized that its essential nature is a network of connected bits that have no inherent meaning. Meanings are assigned to this structure at many levels, but these are not permanent and can be changed at any time. The structure becomes a series of relationships among languages, images, sounds, texts, and other elements mapped
35:4
Troy Innocent
over time and space. In places made of language, concepts become worlds that can be navigated and restructured, dependent on any possible variables. These spaces are made of media. They can emulate a multitude of different media, old and new, and switch between multiple surfaces or skins in an instant. Representations can shift from one medium to the nextimage may become sound and vice versa. The uid nature of connections and relationships in electronic space and the capacity of that space to emulate a wide range of possible modes of representation give rise to a unique opportunity. The same underlying system or structure may be represented in any number of possible ways. Not only is the system shifting and changing, but the mode of representation becomes an active, dynamic, mutable element opening up new expressive possibilities. In the case of games, this system is typically the game logic and simulation that models the world. So this potential is extended to include the mutation and shape-shifting of that world in terms of its structure and the representation of that structure. The separation of system and representation is part of a wider cultural logic that is characteristic of digital media. Generally, we are familiar with the idea of interface being a representation of data, and that being digital means being able to reinvent yourself at the click of a mouse: morphing effortlessly from calculator to spreadsheet to word processor to video-editing console and back again. [Johnson 1997]. By way of example, game modications involve the replacement of textures, models, sounds, animations, maps, and other elements in order to create new levels of a game, or entirely new scenarios represented through the original game engine. Typically, the rules of play and the underlying world remain unchanged, although more sophisticated game mods do alter the functionality of the game. Modications by artists may subvert the meaning of a game, or transform it into a musical instrument. This has also spawned new areas of activity, most notably the practice of lm-making using game engines, known as machinima [Machinima n.d.]. 4. GAME SPACES A model of game spaces that reects these ideas and themes could be summarized as having the following characteristics: (1) (2) (3) (4) symbolic world dened in virtual space; mediated by simulation; interaction enables control/feedback; and gameplay denes logic of symbolic world.
These can be seen as the basic elements of game language that most contemporary digital games operate with. So we may look at digital games in terms of interaction in terms of the many levels of the interface; simulation in terms of the modeling of certain phenomena; gameplay in the rules of operation and constraints; and the idea of a symbolic alternative world. The layers of meaning that are generated by digital games are largely the result of the connection and interplay between these elements.
35:5
This article looks at the subset of digital games that are situated in virtual worlds; this term itself is quite open to interpretation, so virtual worlds will be dened as electronic spaces with the following characteristics (described in more detail in an earlier paper which explored the nature of electronic space [Innocent 2003]): Feedback loop. The player has direct control of the game and it provides immediate feedback. This realtime interaction results in communication between the player and the game as a continuous stream of events. Mode of perception. Game spaces trigger many of the perceptual cues of real spaces. Spatialized sound, physics simulations, 3D perspective, and the players ability to manipulate the environment create a sense of actually being in the game world. Transmutational space. Despite its reliance on the simulation of a real world, the space of games is highly mutable and subject to change. Representations may be redened in a literal sense through changes in parameters of the simulation, by actions of the players, or direct modication of the software. World as sign system. The nature of the computational medium is a symbolic, abstract realm that requires everything to be formally dened. These denitions can be analyzed in terms of relations between signs, and so the virtual world literally becomes the embodiment of the sign system. 5. SHIFTING MODES OF REPRESENTATION Further potential in new forms of communication and expression may be found through exploration of the separation between system and representation that occurs in digital media. Outside the realm of the realism typical of computer simulation, alternative worlds based on shifting modes of representation can be dened in game space. In this model, the system is dened as both the elements of the virtual world and the relationships between these elements in terms of the art practice of world-making. The representation of this system is recognized as a collection of media that may be mapped and connected to the various elements dened in the system. The connection between system and representation occurs within the electronic game space through its aesthetics, logic, and structure. Three works that use different modes of deliverya CD-ROM, an interactive articial life simulation, and a 3D gameengage with these themes and use this particular model of world-making as the strategy for their conception and construction. Each work deals with the shifting modes of representation and the blending of multiple realities that can occur in game spaces. Idea-ON>! (Figure 1) is an electronic database, a database of ideas and experiences, each one contained in a place in the world. A kind of catalogue of experience or repository for characters, stories, sounds, and images which have been collected from my subconscious over time, grouped into four new realities. Visit the Techno Garden to nd morphing plants and a fertility goddess, or experience simulated wealth and human mutations in the Cybaroque world; touch an iconbody during the ritual in the prelinguistic world and see a short dance; or talk with new communication forms in the Communications space.
35:6
Troy Innocent
Fig. 1. Worlds and modes of representation in Idea-ON>!
Fig. 2. Iconica elements.
Visiting the Idea-ON>! installation can be likened to visiting a sacred site where spirits and myths reside. The information space inside the computer becomes a dreaming or meditative space, a manifestation of the subconscious where the objective contents of thoughts are stored for others to explore and experience or add upon if they desire. Similar in the way prelinguistic societies would have a shared body of myths and legends that made up their perception of the universe, a world like Idea-ON>! jumbles together many things, towards a prototype of a dreamlike, surreal, communal cyberspace in which people dream, create, imagine, and play with thought and form. Iconica (Figures 2, 3 and 4) is an interactive project, typically experienced as a gallery installation, which enables visitors to interact with an articial world made of language. The work has the capacity to evolve, change, and mutate through human interaction and its own evolutionary process. Visitors to the world can create, construct, and manipulate objects, inuence the evolution of societies, and discover new language elements. Communication with the residing lifeforms occurs via the iconic language on which the world is based. The multiplicity of Iconica is experienced through this interactionsimultaneously a cyberspace, a mindspace, an abstract world, and a stylized reality. Iconica uses a generative system to create unique images and soundtracks for each of the lifeforms generated by its articial life model. The audiovisual language it draws upon for these representations relates to different graphic styles used in digital media that signify various views of reality. Synthetic forms are displayed as clusters of shiny geometric primitives, information forms are displayed as numbers and bits, and so on. These shifts in representation are an active use of semiotic morphism to create a multidimensional space in which different types of representation coexist and overlap in the same space. Another level of the work is the relationship between the iconic language used to represent the world of Iconica, the articial life model, and the way this language is used to interface with the world. The articial life model operates using symbols (Figure 3). Energy forms, mutation forms, body structures, spaces, and so on, are all dened as symbolic entities in the articial life model.
35:7
Fig. 3. Anatomy of an entity in Iconica.
Fig. 4. Selected entities from Iconica, with descriptions.
An iconic equivalent for each of these abstract symbols used in the code of the world can be found in the iconic language used by the world. A simple grammatical structure is also used by the iconic language to describe relationships between elements in the world, such as which element a lifeform is made of, whether it is hungry, and so on. The users communication with lifeforms is
35:8
Troy Innocent
also achieved through the iconic language of the world. As a result, the user slowly learns this alternative language through interaction with the work. This engagement results in a different kind of immersion that comes from the involvement with the system and language of an articial world. 6. SEMIOMORPH This model and the investigation of shifting modes of representation has been explored further in the third work, Semiomorph, which looks at the intersection of semiotic morphism with digital games. Semiomorph is a third-person 3D digital game that is situated in a virtual world that morphs and shifts its representation to the player. Virtual worlds have the innate ability to shift and change, mutate and transmutate, and layer multiple modes of representation. Semiotic morphism is a model for translating the same data between these different modes of representation through computation. Potentially, hybrid modes of representation may be computed by the system. It uses algebraic semiotics [Innocent 2003] to adapt semiotic structures into functions and theorems using algebra and set theory. Once dened in this way, semiotic morphism allows the signied message to be mapped to various signiers resulting in a system that generates different instances of semiosis. This strategy reects the nature of the computer both as a manipulator of language and computational machine. [Innocent 2003] Semiomorph [Innocent 2003] explores this idea by connecting gameplay with shifts in representation on the basis of Goguens theory of semiotic morphism [Goguen n.d.]. The gameplay is described by the author as follows:
The goal of the game is to collect enough energy points to create a semiomorph and move onto the next stage. The kind of energy you collect will shift your mode of representation between word, diagram, icon and simulation. This in turn will change the rules of play and your effect on the world. You must avoid opposing entities and blast icons. Power-ups make you invincible for a short period of time. Muticons switch your mode of representation instantaneously.
The resulting space is one that is constantly shifting and changing, blending several forms of representation into one experience. Although the underlying structure of the space does not change, the different instances of the game objects may be simultaneously represented as text, diagram, iconographics, or simulated reality. This can be seen in the various representations of game elements (Figure 5). Each mode is personied by a group of game characters competing for screentime. Both the objects and the space can be represented using any of the four modes of representation, either at a local or a global level. Semiotic morphism is used to generate the representation of the game space and as the basis of the gameplay. Semiomorph explores semiotic morphism in an immersive environment, using the visual language of video and computer games as the starting point for its sign system. This environment consists of a 3D space, game objects and
35:9
Fig. 5. Table of game elements in Semiomorph.
characters, and the player. Each of these may be represented using one of four systems (Figure 6): (1) Word: text labels appear on spaces and game objects; represented by the Mt.Ke,I-t character. (2) Diagram: spaces and game objects are represented in wireframe mode; represented by the D-Glypha character. (3) Icon: simple stylized representation of space and game objects; represented by the Specular character. (4) Simulation: spaces and objects are texture-mapped and made more complex to increase realism; represented by the Realamon character. In terms of production, the game system was initially specied and designed in terms of its structure. Once this was in place, creating the elements for the game became a process of lling a database of 3D objects, textures, entities,
35:10
Troy Innocent
Fig. 6. Semiomorph game space in four modes of representation.
spaces, and sounds. Each media element was determined by the intersection of the game objects with the four modes of representation. The musician, Ollie Olsen, who composed the music and sound effects, worked with tables of game elements indexed by the different modes of representation. He interpreted these into the language of music and sound, creating distinct timbres and compositional strategies for each of word, diagram, icon, and simulation. 7. SEMIOMORPH GAME SPACES The potential of the Semiomorph space is explored further in four separate stages of the game (Figure 7). These stages are played through in the same order as they appear here. Each stage uses the same system to dene the game world, but interprets this system in different ways to express each mode of representation. 7.1 Icon Mode / SUPER PLASTIC PIXELTM Stage This stage characterizes the iconic mode of representation in Semiomorph. Referencing the plastic reality of game spaces such as Sonic the Hedgehog and Super Mario World, it is a stylized 3D world made of colorful surfaces and simple 3D forms. Perfectly rounded hills are dotted with owers, trees, and clusters of energy forms. Everything is what it is. This is a world of icons and it is
35:11
Fig. 7. Four stages of the Semiomorph game.
Fig. 8. Icon: SpecularTM and friends.
represented as such. However, all of these element are subject to shifts in representation, so this simple world is quickly broken down as hills become words, are rendered as wireframe, or texture-mapped. Three entities (Figure 8) the iconic mode of representation. Playful, superdeformed gures that reference the hypercute game worlds that this stage is
35:12
Troy Innocent
Fig. 9. Word: Mt-Ke,I-tTM and friends.
built upon. The music and sound are a collection of bleeps and simple melodies in a retro video game style. The iconic entity, Specular, may easily defeat word entities but is vulnerable to attacks from diagram entities. SUPER PLASTIC PIXELTM is a landscape set up for free exploration. The player begins in the center of the world on top of a hill, from which radiating paths of energy forms may be followed. Around the corners of the world, the power-ups may be found. Collections of blocks break up the space, creating smaller zones of activity close to the larger hills and sky that dene the boundaries of this stage. The structure allows long views through the space depicting the rolling hills populated by icons and entities.
7.2 Word Mode: M3TA+TM Stage An interpretation of ideas and themes from the realm of hypertext is the starting point for the word mode of representation. The walls, oor, and other structures of this stage are textured with moving surfaces lled with words describing these structures. These collections of descriptions and labels shift and change in relation to the different modes of representation as word intersects with diagram, icon, and simulation. Mt-Ke,I-t (Figure 9) is the character the belongs in this realm. This humanoid gure has a simple symbol for a head, vectors for hands and feet, and a body made of text and hyperlinks. In the game dynamics, word beats simulation but icon beats word. Sound for this stage consists of synthesized voice and abstract electronic sounds mixed in a fragmented, chaotic compositional style. The space of this stage is fragmented, consisting of different planes intersected by angled walls. The walls appear solid, but the player may walk through them and suddenly nding himself in another space altogether. In many cases, this results in a relocation on the vertical plane as well as a shift from one level to the next. Although the lack of solid walls means that the player can move anywhere without restriction, the dense layers of text in the space give a feeling of claustrophobia and intensity.
35:13
Fig. 10. Diagram: D-GlyphaTM and friends.
7.3 Diagram Mode: INFOS CORPTM Stage Information graphics, data glyphs, and other forms of machine communication form the basis of the aesthetics and structure of this stage (Figure 10). Grids, lines, and simple geometries are mapped onto surfaces to create layers of information space. Many game elements are switched to wireframe mode, creating overlapping visual patterns of geometric forms. When the modes of representation start shifting, these visual patterns are mixed with solid surfaces of color, realistic textures, and elds of text, resulting in a hybrid information space. A data glyph given life, D-Glypha (Figure 10) roams this space. Simple matrices of pixels have been extruded into a three-dimensional form that appears to be a giant robot that menacingly patrols its territory. Machine rhythms and bursts of data can be heard throughout this world. As a diagram character, D-Glypha easily defeats icon characters, but must avoid simulation types. This stage is built on a grid with the level based on the maze structure of Pacman. The player moves through long passages full of energy forms that are broken up by grids of blocks. Power-ups are in the four corners of the world. Visibility is limited as the viewpoint of the player is bound by the walls of the maze. 7.4 Simulation Mode: SOFT RAININGTM Stage The particular kind of realism typical of video games is the basis of this stage of Semiomorph. All surfaces are mapped with photo-realistic textures, and a soft rain falls over the natural, undulating landscape. Pools of water, large trees, and rocks appear throughout a space that is bounded by mountains. Particle systems, procedural textures, and reection maps are used on the game elements to give them a realistic appearance. This illusionary representation is disrupted as the other modes of representation break it up into wireframes, at planes of color, and text. A collection of surreal animals (Figure 11) personify this mode of representation. They crawl about the space in different colors and markings denoting
35:14
Troy Innocent
Fig. 11. Simulation: RealamonTM and friends.
different variations of the species. These Realamon are related to the Pokemon virtual creatures. When interacting with the other characters, the realism of simulation defeats diagram, but word defeats simulation mode. This stage is a landscape lled with natural forms, but strangely populated by game icons and blocks. These are scattered about at random to further develop the feel of a natural, organic space. The movement of the player is not restricted by many obstacles, and large sweeping views of the space are common. 8. CONCLUDING REMARKS Game spaces are ideally suited to artworks that are based on a model of worldmaking and on the exploration of coherent, alternative worlds that investigate the uid, mutable representation of these spaces. The potential for this kind of expression is expanded by connecting the modes of representation with gameplay (Semiomorph) or a generative system (Iconica). Such practice could be described in terms of blending realities. These ideas may also be explored further in terms of the overlap between the virtual and the real. Many of the game objects from Semiomorph have been replicated to scale as plastic models in the real world. Thus these sculptural objects, called ofine artefacts are a literal expression of blending reality. Finally, the increased perception of virtual spaces as being integral to human experience extends this blend of the virtual and the real even further. As these systems of world construction evolve into more complex forms through developments in articial intelligence and articial life, nding new ways to represent and express this complexity will be needed.
REFERENCES GOGUEN, J. (n.d.) Semiotic morphisms. http://www.cs.ucsd.edu/users/goguen/papers/sm/smm. html. Accessed Feb. 2002. HOLTZMAN, S. 1994. Digital Mantras: The Languages of Abstract and Virtual Worlds. MIT Press, Cambridge, MA, p. 210. INNOCENT, T. 2003. Exploring the nature of electronic space through semiotic morphism. In Proceedings of the MelbourneDAC, the 5th International Digital Arts and Culture Conference (May 1923).
35:15
JOHNSON, S. 1997. Interface Culture: How New Technology Transforms the Way We Create and Communicate. HarperEdge, p. 147. MACHINIMA. (n.d.) http://www.machinima.com/. Accessed Oct. 2003.
Received January 2008; accepted June 2008

Blending Realities in Game Space

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Blending Realities in Game Space

Transféré par

Droits d'auteur :

Formats disponibles

Blending Realities in Game Space

TROY INNOCENT Center for Electronic Media Art, Monash University