Distributed Virtual Environments for Traiiining and Telecsllaboration

N.D. Georganas, FIEEE, E.M. Petriu, M. Cordea, and D. Ionescu

SITE .- School of Infonaation Technology and Engineering Universi~ty Ottawa, Canada of

A)bstract: This paper presents a brief review o f the

_ .
~

Distributed Krtuul Environment (DVE) applicat, ys in 1 training and telecollaboration. The second part f' the puper discusses systems architecture and avatar wner tracking aspectsjbr an experimental DVE.

1.. Introductijon
The basic idea of Distributed Krtual Enviro (DVEs) is simplc:: a simulated world runs not computer system, but on several, usiing a series of server applications, [l]. The computers are cor over a network and people using those computl able to interact and collaborate in re:al time, shar same virtual world. Tele-immersive DVEs recogr presence and miwements of indwiduals and track those imagzs, and then project them in rc multiple, ge0g;raphcally distributed im environments where individuals can interact wil other and with computer-generated models. Collaborative DVEs require an unconvenl broad range of iietworking, database, graphics, modeling, real-lime processing ;and user ir capabilities. The interest generated by DVEs is reflected multitude of the research projects pursued all o world, some of which are briefly presented here. Transvision: a Collaborative Augmented Testbed of Sony Computer Science Lab., Sor Japan, [2], is an attempt to use augmented virtual for collaborative designing. The user can sec palmtop video-see-through display a COI generated 3D model superimposed on the real view. The position and orientation of the disp tracked by the system such that the computer-ge model appears to occupy real space. Two 01 participants can share the same cornputer model
1

nents

one lientected s are g the .e the jects, listic, :rsive each

L

inally world :rface

n the

:r the
eality Co., eality on a merworld y are :rated more s ifit

were reaI.

Distributed Interactnve Virtual Environment (DIVE) developed by the Distributed Collaborative Enviroinments group of the Swedsh Institute of Computer Science, [3], is an internet multi-user DVE where participants navigate in 3D space and see, meet and interact with other users and applications. DIVE supp0rl.s the development of virtual environments, user interfaces and applications based on shared computer generated environments. Applications include virtual battlefields, spatial models of interaction. virtual agents, real-world robot control and multi-modal interaction. Social Virtual Reality Research, a project of the Mitsubishi Electric Research Laboratory (MERL), Cambridge, USA, [4], focuses on the interaction among people in virtual environments. It envisions group learning environments where people learn from each other, from teachers, and by interacting with computer simulations. SHAVE, a project of the SRI International, U S 4 [5], is a multi-user virtual environment that allows several users on the Internet to interact within a shared 3D world. The SHAVE system provides a framework for remote collaborative work within interactive and editable 3D worlds. ViItual Environment Technology Laboratory (VETL) of the {Jniversity of Houston and NASA Johnson Space Centre, USA, [6], does research and development focused on virtual environments for training, education, and scientiWengineering data visualization. NASA is exploring virtual reality technology to help international crews of the International Space Station to prepare for their missions The first demonstration of using a shared virtual environment across the Atlantic took pllace in 1995. Astronaut Bernard Harris at the NASAJJohnson Space Center in Houston, shared a virtual environment with the astronaut Ulf Merbold at the Fraunhofer Institute for Computer Graphics in Darmaadt, Germany.

0-7803-5276-9/99/$10.00 0 1939 IEEE

1847

COVEN (COllaborative Virtual ENvironments), a project of the Advanced Telecommunications Technologies and Services (ACTS) consortium led by the Thomson-CSF Central Research Laboratory, [7], brings together a wide range of expertise on communication infrastructures, Virtual Environments, Human-Computer Interaction and VR animation to provide support for a European distributed virtual environment. This project aims at developing a computational service for teleworking and virtual presence. The overall objective of the project is to provide the facilities needed to support future cooperative tele-working systems and to demonstrate the added value of networked VR for both professional users and home users. DVEs are well suited for collaborative design and interactive training of the industrial personnel for equipment operation and maintenance. Collaborative design work in VDE typically involves a small group of users, either synchronously or asynchronously, engaged in the construction, and manipulation of objects in the virtual world. Since the interfaces for three-dlmensional modeling in VR are still relatively imprecise compared to 2.5D CAD packages, most of the collaborative tasks in collaborative design involve evaluations of the design, and to a lesser degree, redesign or brainstorming for new design possibilities, [SI. As an example, the National Computational Science Alliance (NCSA) and Caterpillar Belgium S.A. have developed a system to allow remotely located engineers to work together on vehicle design review and redesign, [9]. Such a collaboration is needed for Caterpillar engineers in the U.S. and Europe who jointly design Caterpillar vehicles so that they meet customer demands and safety requirements for both markets.

2. Experimental DVE
An experimental DVE is currently under development in the MCRLab at the University of Ottawa as a test bed for training and telecollaboration, Figure 1. It will be used to study human-machine interfaces, object modeling and collaborative object manipulation, and the multimedia telecollaborationusing Internet tools and protocols, [lo]-[12]. Other issues of interest include: how participants should be represented in the collaborative environment; how to effectively transmit non-verbal cues that real-world collaborators so casually and effectively use; how to best transmit video and audlo via a channel that allows both public addressing as well as private conversations to occur; how to deal with different time-delays and how to sustain a virtual environment even when all its participants have left.

The human users in a collaborative environment are represented by virtual alter egos called avatars. We are using avatars based on muscle-based articulated models of the human face and body, [13]. Vision, force and tactile interfaces are used to provide the avatar-owners with a feeling about the interactions which their avatars are experiencing in the virtual environment, [ 141-[16]. As an expedient approach to the 3D object recognition and tracking problem we are using a permanent pseudo-random encodmg of the visible surfaces of the of the avatar-owners. This encoding has the notable advantage of a very compact encoding of the objects requiring only one symbol of code per quantization interval. It is a technique well suited for controllable environments which allow for object encoding and a priori mapping of all encoded object models in a data base, [16]. The window property of the pseudo-random codes allows identfication of the absolute coordinates of the encoded points on objects from a relatively small subset of symbols recognized on the object surface using monocular vision. Knowing how different surfaces of the geometric models of the encoded objects were originally mapped on the encoding array it is possible to recover the identity and the position, orientation and scale estimation parameters of the object containing the recovered window. The recovered movements of these actors are used to build a KDB for the dynamic behavior of the avatars evolving in the DVE. We are also using a feature-based technique for the real-time tracking of the human face position and orientation using vision, Figure 2. It consists of three steps (i) extraction of a set of 2D features from a set of monocular images, (ii) tracking the found features from one frame to another, and (iii) motion estimation. The feature detection is based on the shape and color constancy. An ellipse will search for the largest skinblob, and classifl it as a face. The maximum size of the ellipse is obtained, by searching for skin inside, non-slan outside. The advantage of using a shape for detection and further for tracking is that the tracker is not attracted by skin-like background regions. A wireframe mesh is then fitted on the 2D face using detected features as anchor points and least-square interpolating for the rest of the geometry. The model provides the depth parameters of a Kalman filter used to recover structure from motion. Most of the models used in this DVE are implemented in Java and VRML 2.0. Issues of causality and multimedia synchronization, areas that we have significant know-how and experience, are also studied. Sharing was solved using the Java-Enabled Telecollaboration System (JETS) developed at the MCRLab, [ 171.

1848

kornputer

I Animation

Object Shape &Behavior Models

Motion Tracking

FTRTIIAL SCENE / Th

I I

i I

_
Pseudo-Random Encoding -

6.

Figure 1. The architecture of the distributed viirtual environment.

1849

F i m e 2. Real-time tracking of the avatar-owner's face position and orientation.

Acknowledgment
This work was funded in part by the STENTOR New Media Fund.

[101 N.D. Georganas, "Multimedia Applications Development Experiences", J. Multimedia Tools and Applications, Vol. 4,No. 3, pp.313-332, 1997. [ I l l J.W. Wong, K.A. Lyons, R.J. Velthuys, G. v.Bochmann, E. Dubois, N.D. Georganas, G. Neufeld, M.T. Oszu, J. Brinskelle, D.F. Evans, A. Hafid, N. Hutchisosn, P. Iglinski, B. Kerherve, L. Lamont, D. Makaroff, and D. Szafkon, "Enabling Technology for Distributed Multimedia Applications", IBM Systems J. , Vol. 36, NO.4, pp.489-507, 1997. [ 121 W. Robbins and N.D. Georganas, "Collaborative Media Space Architectures", Proc. CCBR'98, Ottawa, June 1998. [13] H.J.W. Spoelder, E.M. Petriu, T. Whalen, D.C. Petriu, M. Cordea, "Knowledge-Based Animation of Articulated Anthropomorphic Models for Virtual Reality Applications," Proc. IMTC/99, IEEE Instrum. Meas. Technol. Con$, Venice, Italy, 1999. [14] G. Burdea,, E. Roskos, D. Silver, F. Thibaud., and R. Wolpov. "A Distributed Virtual Environment with Dextrous Force Feedback, Proc. Infomatique '92, International Conference Interjiace to Real and Virtual Worlds, pp. 255-26, Nanterre, France, 1992. [15] E. Petriu, W.S. McMath, S.K. Yeung, N. Trif, "Active Tactile Perception of Object Surface Geometric Profiles," IEEE Trans. Instrum. Meas., Vol. 41, No. 1, pp.87-92, 1992. [16] E. Petriu, D. Ionescu, D.C. Petriu, S.K. Yeung, Ph. Lavoie, N. Trif, "Absolute Position Measurement Applications of PseudeRandom Encoding," Proc. ETM'96 IEEE Intl. Workshop on Emeeent Technol.for Instrum. Meas., pp.119-126, Como, Italy, 1996. [17] SShirmohammadi and N.D.Georganas, "JETS: JavaEnabled TeleCollaboration System", Proc.IEEE Multimedia Systems'97., Ottawa, June 1997

References
[l] R.C. Waters and J.W. Barms, "The Rise of Shared Virtual Environments," IEEE Spectrum, Vo1.34, No. 3; pp. 18-25, March 1997. [2] http://www.csl.sony.co.jp/person/rekimoto/ transvision.html; [3] http://www.sics.se/dive/dive. html; 141 http://www.merl.codthreads/social/index.html;

[5] http://os.sri.com/research/shave.html; [6] h t t p : / / w . vet1.uh. edu/overview/overview2.html; [7J http://chinon.thomson-csf G/projects/coven/; [XI J. Leigh, A. Johnson and T.A. DeFanti, "CALVIN: an Immersimedia Design Environment Utilizing Heterogeneous Perspectives," Proc. IEEE International Conference on Multimedia Computing and Systems'96, Hiroshima, June 1996. [9] V.D. Lehner and T.A. DeFanti, "Distributed Virtual Reality: Supporting Remote Collaboration in Vehicle Design," IEEE Computer Graphics and Applications, 1997.

1850