International Conference on Convergence and Hybrid Information Technology 2008

A Smart Communication System for Avatar Agents in Virtual Environment

Soo-Hyun Park, Seung-Hyun Ji, Dong-Sung Ryu and Hwan-Gue Cho Dept. of Computer Engineering Pusan National University, Republic of Korea {shpark, shji, dsryu}@pearl.cs.pusan.ac.kr, hgcho@pusan.ac.kr

Abstract
This paper propose a virtual communication system for avatar agents using geometry topology. Current virtual communication systems have several problems and limitations since they use only 2-dimensional dialog box and word balloons though they provide 3-dimensional virtual environment to communicate in virtual space. To solve these problems, modern virtual communication systems have developed articial chatting techniques such as One-to-One Chatting and Group Chatting. We propose a novel communication system for avatar agents using geometry topology information such as distances and directions between avatar agents without articial chatting techniques. The main contribution of our work is threefold. First, we display Partial Talk and Complete Talk by using geometry topology so that make more realistic and natural conversation without articial complex communication architecture. Second, we design a greedy algorithm for placing the word balloons in 3-dimensional virtual space. Finally, we propose a method to analyze order of dialogs and to construct talk sequence graph.

The Internet is evolving to become the cyber space or virtual world such as Second Life [6]. Today, the success and momentum of virtual environments are undeniable. The market for Massively Multiplayer Online Game(MMOG) is estimated to be worth more than one billion US dollars [13]. However, these virtual world systems have several problems and limitations to communicate in their virtual space. Since they use only 2-dimensional dialog box and word balloons though they provide 3-dimensional virtual environment to communicate in virtual space. To solve these problem, modern virtual environment systems have developed some articial chatting techniques such as One-to-One Chatting and Group Chatting. However, these techniques decreased reality of virtual world and make conversations articial. We propose a more realistic smart virtual communication system for avatar agents using geometry topology such as distance and direction between agents. Figure 1 shows result screen of our system.

1. Motivation
Computing technologies are changing the way that people communicate. As the internet services have grown in popularity, another form of communication: electronic chatting rooms are introduced [5]. Although these electronic chatting programs originally supported only textbased communications, fast modems and network connection techniques have enabled the developers of these chatting programs to create a richer user experience through the addition of graphics [1, 3, 8]. Participants in these graphical chatting rooms communicate with one another not only by typing text, but also by changing the gesture or facial expression of an avatar.

Figure 1. Result scene of our system. There

are several chatting agents in chatting room and they are chatting in two groups. Each agent can see only their group conversation clearly.

978-0-7695-3328-5/08 $25.00 2008 IEEE DOI 10.1109/ICHIT.33 10.1109/ICHIT.2008.186

119

In our system, participants can see conversation clearly of only near group agents but can not see conversation of others. They just know others are talking about something. We propose a virtual communication system basic idea about realistic conversation in real world. Our system proposes those three in the followings: We display a more realistic conversation in virtual space through Partial Talk and Complete Talk by using geometry topology, so make conversation more natural without articial chatting techniques. We propose a greedy algorithm for placing word balloons in 3-dimensional virtual space. We propose a method to analyze order of dialogs and to construct talk sequence graph.

(a)

(b)

Figure 2. Communication using Comic Chat.

Comic Chat automatically represents communications in the form of comics. It provides selection of word balloons, characters, gestures and semantic panels. It uses a greedy algorithm for placing the word balloons.

2. Previous Work
Internet chatting services originally supported only textbased communications. But fast modems and network connection techniques have enabled the developers of these chatting programs to create a richer user experience through the addition of graphics or simple images. Today, some chatting programs provide voice chatting and web-camera chatting. These chatting programs are evolving to become more realistic and interesting virtual worlds. Comic Chat uses a greedy algorithm for placing the bodies of word balloons, since word balloons are critical in Comic Chat. Once the algorithm chooses a word balloons location, it never changes it. However, the word balloon tail placement is not greedy. Instead of greedy algorithm, the algorithm delays placing the tails until all the word balloon bodies have been placed. Comic Chat provides three different balloon types, including speech balloon, thought balloon and whisper balloon. Comic Chat also provides several characters and their gestures. When a participant types in dialogue, Comic Chat uses this dialogue to determine a default gesture and expression. Now, Comic Chat is not available but it is meaningful that it provides the graphical history of the conversation.

2.1

Text-based Chatting System

Text-based chatting is a way of communicating by sending text messages to people in the same chatting room [10]. The oldest form of chatting systems are the variety of text-based chatting system. Freelancin Roundtable was a modem-based text chatting system in the 1980s [11]. Some chatting services such as Yahoo! use both text and voice simultaneously. These text-based chatting systems are simple and convenient as they can be used anywhere at anytime from One-to-One Chatting to Group Chatting.

2.3

Chatting System for Virtual Space

2.2

Template-based Chatting System

Fast modems and network connection techniques have enabled the developers of chatting programs to create a richer user experience through the addition of graphics or simple images. Especially, Comic Chat [5] is a system that automatically represents on-line communications in the form of comics. Comic Chat provides numerous aspects of comics generation, including word balloons, characters, gestures and semantic panels. It has several advantages over other graphical chatting programs, including the graphical history and a dynamic graphical presentation. Figure 2 shows communication result scenes using Comic Chat.

Chatting Systems for Virtual Space are characterized by using a 3D-Graphic representation of the avatar agents that can be moved in a virtual environment. These virtual worlds are important elements for variable internet medias such as massively multiplayer online games and educational material. Especially, the younger systems among the virtual world systems are Second Life [6] and IMVU [4]. Figure 3 shows communication results of Second Life and IMVU. The special feature of Second Life is Shout Word Balloon technique. If users use Shout Word Balloon technique, the messages of the users are sent more far and more stronger. IMVU also provides Shout Word Balloon, if the sentence contains exclamation mark, Shout Word Balloon is generated. However, both Second Life and IMVU use only two dimensional dialog box and word balloons though they provide three dimensional virtual en-

120

(a)

(b)

Figure 3. Communication using Virtual Reality Chatting System. (a) Second Life. (b) IMVU.

vironment to communicate in virtual space. Table 1 explains features of word balloon between Second Life and IMVU. Table 1. Comparing features of word balloon between Second Life and IMVU

agents into two groups. Group G1 consists of agent A and B and Group G2 consists of agent D, E, F and G. Chatting agent C at the passage sees Group G1 and Group G2 . Figure 5 shows the visibility graph of chatting room and their result scene. When a chatting agent types a dialogue, word balloon is generated. In order to develop the system effectively using the 3D topology geometry information of virtual world, we need to compute the relation between avatar agents and word balloons exactly. A set of relation pairs between avatar agent and word balloon in chatting room can be determined through the V CB computing process and these pairs are ordered according to their distances and directions. Figure 4 shows the structure of relation pairs between two word balloons Wa and Wb .

Default Form Additional Talk Past Talk Special Feature Drawback

Second Life Dialog box and word balloon The height gets longer Remove talk from top to bottom Speaking differs from Shouting Disregard geometry Less realistic

IMVU Dialog box and word balloon New word balloon is appended Remove balloons from top to bottom ! makes shout balloon Disregard geometry Less realistic

Figure 4. Example of relation pairs between Wa and Wb .

Current chatting systems for virtual world are using a similar method to increase reality but they use invariable word balloons and text messages without considering 3dimensional topology geometry information. In communication systems that uses only invariable word balloons, there are only sentences, called Complete Talk, that is totally visible or not visible. Since current chatting systems display only Complete Talk through the invariable word balloons, it is unrealistic comparing to real world. Our system displays Partial Talk that looked differently by the view direction, like a keyword recognition in real world communication.

(a)

(b)

(c)

(d)

3. Preliminary
Let us assume that we have a chatting room consists of walls and 7 chatting agents from A to G. The walls separate

Figure 5. Visibility Graph of Chatting Room.

(a)(b) Chatting agent C sees the Group G2 at time . (c)(d) Chatting agent C sees the Group G1 at time+ .

121

Each relation pair consists of two word balloons or avatar agents and each agent and word balloon consists of their position and view vector. Let an Wa and Wb be word balloons and lab be visibility line connected with center points of Wa , Wb . a and b are angles of Wa , Wb and lab . Denition 3.1 A VCB(Virtual Chatting Bandwidth) is communication capacity bandwidth between agents or word balloons. Higher V CB, agent can understand messages clearly. V CB between a and b is dened by V CB(a, b) = C1
(sin a sin b )k1 dist(a,b)k2 +C2 2

4. Optimization of 3D-Word Balloon Placement

4.1 Evaluation of Visibility

where a , b < . V CB(a, b) is maximum if a , b = and V CB(a, b) is zero if a , b < .

It is important to display the Partial Talk according to the V CB between word balloons. Previous chatting systems are using articial chatting techniques, such as Oneto-one Chatting and Group Chatting by using Complete Talk, that is totally visible or not visible. Instead, we display the Partial Talk according to the 3-dimensional topology geometry information. Figure 6 shows changes of V CB according to the distances.

Denition 3.2 A TSG(Talk Sequence Graph) is a Directed Acyclic Graph(DAG) with some sequencing on talk. Each talk consists of agent(ai ) and time stamp(ti ) is dened by T alk(ai , ti ) = message Each talk can be connected according to their V CB and time stamp(ti ).

Table 2. Feature examples of word balloons in virtual chatting room. (a) Agent A B A C A D A E A F A G B C B D B E B F B G C D C E C F C G D E D F D G E F E G F G lab 0.033 0.552 0.601 1.263 1.244 1.211 0.526 1.200 1.260 1.240 1.209 0.929 0.993 1.216 0.955 0.064 0.049 0.043 0.026 0.025 0.039 a 90 85 113 127 120 131 262 302 328 305 323 78 88 68 100 90 45 135 135 45 90 b 90 85 210 43 120 220 187 225 60 128 204 250 85 155 170 90 135 45 45 135 90 V CB 99.890 76.337 0.000 28.439 33.998 0.000 0.000 0.000 0.000 0.000 0.000 0.000 50.249 36.990 21.622 99.594 70.543 70.578 70.662 70.668 99.851

(b)

(c) Figure 6. Change of V CB according to the distance. (a) distance = 0.04 . (b) distance = 0.1. (c)
distance = 0.38.

Table 2 explains example features of word balloons in our virtual chatting room. We computed the V CB with xed parameters: C1 = 100, C2 = 1, k1 = 0.5, k2 = 2.

122

The sentences are looked differently by view directions and distances. We can see the sentences clearly at distance 0.04 in Figure 6(a), but we can not see clearly at distance 0.38 in Figure 6(c). For agents who do not look at each other, the sentences are displayed as Partial Talk. Figure 7 shows scenes of each view point in our virtual chatting room.

D. Agent D is connected with E, F , G in Figure 5, so D can see word balloons of E, F , G comparatively clearly. Figure 7(c), (d) are scenes of view point C. Agent C is connected with D, E, F , G at time and connected with A, B at time+ in Figure 5, so C can see word balloons of D, E, F , G at time and word balloons of A, B at time+ . But C can see small word balloons because of V CB according to distances and directions. C can not see conversation of group G1 , G2 , just know they are talking about something. Table 3 shows Talk recognition experience of each agent.

Table 3. Talk Recognition Experience of each agent. (a) (b) Agent A B C D E F G A B C D E F G

(c)

(d) The talk almost Complete Talk is indicated with . Partial Talk is . means can not see. It is natural to recognize the keywords of whole sentence compared with other communication systems that uses only Complete Talk, that is totally visible or not visible.

4.2
(e) (f)

3D-Word Balloon Placement

(g)

(h)

Before constructing word balloons, we apply a word balloon placement algorithm that determines placement of each balloon. In previous system, it is a simple problem to place word balloons in 2 dimensional layout. They considered only 2-dimensional word balloons. However, we need to place word balloons in 3-dimensional space. We use a greedy algorithm for placing word balloons. The algorithm nds a balloon which makes maximum total V CB and replaces the balloon. The following Algorithm 1 calculates the total V CB of each agent. Algorithm 1 Total Visibility Calculation Algorithm For all ai , bj S, calculate
ai , bj S

Figure 7. Different scenes of each agent. (a)

Scene of view point A. (b) Scene of view point B. (c) Scene of view point C at time . (d) Scene of view point C at time+ . (e) Scene of view point D. (f) Scene of view point E. (g) Scene of view point F. (e) Scene of view point G.

V (ai , bj )

if V isible(ai , bj ) = F alse then V (ai , bj ) = 0 else if a , b then V (ai , bj ) = 0 else V (ai , bj ) = C1

(sin a sin b )k1 dist(ai ,bj )k2 +C2

As a Figure 7, each agent sees different scenes and different word balloons according to their view vectors and distances between agents. Figure 7(e) is a scene of view point

123

Algorithm 2 Talk Sequence Graph Construction Algorithm procedure T SG(DL) V T alk(DL); N V iew(DL); T T ime(DL); E {(vi , vj )|vi , vj V, i = j}; for each edge do if visible(vi , vj ) = T RU E && < Tj Ti < && < Ni Nj < then E(vi , vj ) = T RU E return G(V, E); end procedure (a)

Table 4. Talk List Construction. Time 1 3 4 5 7 (b) Figure 8. Results of Word Balloon Placement.
(a) Before apply balloon placement. (b) After.

Agent G E D A F B D A B

8 9 10 12

Message Where will the conference be held? It is Kwangju. It is kyung-ju. isnt it? Hey SH! What did you have for lunch? No! It is Kwangju. Hey TJ! Are you ready to prepare about the conference? I ate Bulgogi for lunch. so so. Was the food delicious? Yes! very nice.

Using Algorithm 1, we nd a word balloon which has maximum incremental of total V CB and raises it up. Figure 8 shows results of word balloon placement.

5. Talk Sequence Graph Construction

Histories are crucial in chatting programs, since participants commonly interleave several tasks while chatting. However, most chatting programs provide only textual history transcripts and there is no history of the relation or context of conversation. To address this problem, we propose Talk Sequence Graph(TSG) construction algorithm. The following Algorithm 2 shows Talk Sequence Graph(TSG) construction algorithm. Edges of Talk Sequence Graph(TSG) are determined through the time stamps and range of vision according to the T ime with parameters , and V iew with parameters , . Table 4 shows example talk list and Figure 9 shows the Talk Sequence Graph constructed using Table 4. We assume that there is no talk that has same time stamp.

There is no ordering or sequencing in textual history transcripts in modern chatting systems. Most textual history transcripts of current chatting systems have just list of talk according to time stamp. However, we save Talk Sequence Graph(TSG) in stead of textual history transcripts and analyze a order of conversation and relation of talk. It helps participants to understand context of conversation.

6. Conclusion and Further Work

In previous communication systems for virtual world, there are only Complete Talk, that is totally visible or not visible. Since previous chatting systems display only Complete Talk through the invariable word balloons, modern virtual environment systems have developed some articial chatting techniques such as One-to-One Chatting and Group Chatting. However, these techniques decreased reality of virtual world and make conversations articial. In addition, most chatting programs provide only textual history transcripts without history of the relation or context of

124

Figure 9. TSG(Talk Sequence Graph).It is constructed according to the V CB and time stamps and helps participants can understand context of conversation.

conversation. To address this problem, we proposed Partial Talk that looked differently by the view direction and make natural One-to-One Chatting and Group Chatting without articial chatting techniques. Also we proposed a greedy algorithm for placing word balloons and Talk Sequence Graph(TSG) construction algorithm. Our proposed system has some limitations and drawbacks. The followings are further work of our work. We need to develop various types of word balloons, such as Shouting Balloon and Thought Balloon. Shouting means sending a message to participants very far, whispering means sending a message only to a friend subset of the participants. It is important to process sounds that its source is not in the range of view, such as call from behind and sound from speakers. It helps virtual worlds to raise their reality. We used a greedy algorithm for placing word balloons. However, we just assumed that balloons are just raised up. We need to develop a more efcient algorithm for placing word balloons.

[2] B.-K. Chun, D.-S. Ryu, W.-I. Hwang, and H.-G. Cho. An automated procedure for word balloon placement in cinema comics. LNCS(Lecture Notes in Computer Science) 2006, pages 576583, 2006. [3] M. Corp. Microsoft Introduces V-Chat Communications for MSN. The Microsoft Network, Microsoft Press Release. Redmond, WA 98052, 1995. [4] IMVU. http://imvu.com/. [5] D. Kurlander, T. Skelly, and D. Salesin. Comic chat. Proceedings of SIGGRAPH 1996, pages 225236, 1996. [6] S. Life. http://secondlife.com/. [7] T. Monahan, G. McArdle, and M. Bertolotto. Virtual reality for collaborative e-learning. Computers & Education, 50:13391353, 2008. [8] C. Morningstar and F. R. Farmer. The lessons of lucaslms habitat. Cyberspace: First Steps, pages 273301, 1991. [9] S. Pekkola. Critical approach to 3d virtual realities for group work. Proceedings of the second Nordic conference on Human-computer interaction, pages 1923, October 2002. [10] C. Room. http://en.wikipedia.org/wiki/Chat room/. [11] F. Roundtable. http://en.wikipedia.org/wiki/Freelancin Roundtable/. [12] B. shen Lin, B. Chen, H. min Wang, and L. shan Lee. A hierarchical tag-graph search scheme with layered grammar rules for spontaneous speech understanding. Pattern Recognition Letters, 23:819831, 2002. [13] E. K. Tansu Alpcan, Christian Bauckhage. Towards 3d internet: Why, what, and how? 2007 International Conference on Cyberworlds, pages 9599, 2007.

References
[1] W. Chat. http://www.worlds.net/products/wchat/readme.html.

125