Connecting the Physical World and Events Schedule of User Calendar for
Symbiotic Systems
Sidiq S. Hidayat1, Bong Keun Kim2, Tamio Tanikawa2, Kohtaro Ohba2
1
System and Information Engineering, University of Tsukuba, Japan, e-mail: s.hidayat@aist.go.jp
2
Intelligent Systems Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), Japan,
e-mail: {bk.kim, tamio.tanikawa, k.ohba}@aist.go.jp
Abstract—In this paper, connecting physical world and
events schedule in the smart environment and ubiquitous
robots are proposed to implement symbiotic robotic systems.
Symbiotic robotic systems consist of humans, robots, and smart
environment that cooperate symbiotically to perform a given
task. An embedded element to develop smart environment called
UFAM (Ubiquitous Functions Activation Module) is introduced.
Physical hyperlink is also implemented for object recognition
and localization based on the RFID system. We use Gdata as a
syndication format, to manage events on user calendar and
communicate with ubiquitous robots. Finally, an example of
symbiotic robotic application for weather forecast is described.
I. INTRODUCTION
Akun is a busy man, and he was going to participate in a
conference. He wrote his schedule, include when and where
the conference would be, using his handheld communicator
connected to the web-calendar. He also wrote when he would
be going back home. Before leaving home, he checked the
smart lamp showing weather forecast in various colors.
According to its colors, it would be high rain during his trip.
His mobile robot also understood that it would be rain. The
umbrella informed its location and the mobile robot then
moved to the corner and picked up the umbrella for him.
While he was going out and nobody was at home, the cleaner
robot was doing its schedule to cleaned floor and cleaned off
dinning table. When he was going back home, the door
detected his coming, and automatically opened. When he
entered the room, his motion was detected and the coffee
machine started to brew coffee.
Like many similar stories, this vignette illustrate a future
scenario in which ubiquitous robots technology are
pervasively present in our everyday environment, cohabiting
with humans and providing us some services to achieve better
quality of life. In this scenario, humans, robots and smart
environment work symbiotically to serve and support human
life.
They can be considered three-agent systems [4] in which
the agents are performing interdependent tasks. The human
create a plan, and then the robots arrange some tasks to
support and perform his/her plan. The environment, called
smart environment, which consist of several networked
devices, sensors network, seamlessly user interfaces, micro
computers, and ubiquitous robots should be sensible and
accessible at anytime and anywhere. Some of robots would
disappear in the environment in the same way as computers
should disappear according to the well known vision of
ubiquitous computing [1],[2].
The idea of merging the robotics technologies and smart
environment mentioned above, and their interaction with
human in daily life is called human robot symbiotic system.
This symbiotic interaction system is aiming that robots and
people can live at the place of everyday life together
(cohabitate) and to provide some services to what the human
need from their services or from their capabilities.
In this paper, to implement human-robot symbiotic system,
the concepts of symbiotic system with smart environment and
ubiquitous robots are introduced. RSS/Atom syndication and
GData protocol to implement tasks planning and scheduling
are briefly explained. Finally, an example of symbiotic
application for weather forecast is proposed.
A. Symbiotic Robotic System
In the ubiquitous robot environment, the robots, sensors,
actuators, and controllers are networked, distributed to space,
and emmbeded to everyday objects. In the same time, humans
are living with them, cohabiting, and doing interaction with
everyday objects and/or robots [5]. Both humans and robots
can control ubiquitous robot function to satisfy their
objectives. This interaction between human and robot is
called symbiotic robotic systems.
The symbiotic robotic system can be performed from
integration of three basic aspects [8]. First, using UFAM, the
objects in the environment possess capability for information
storage, processing, and communication. Therefore the robot
can easily interact with these objects through digital
interaction and physical interaction.
Second, in symbiotic robotic system robot model must
also have knowledge about the human’s position, intensions
and actions. The robot can perform these models through
combination of direct communication with object, and
linguistic communication with humans, and perceptual
observations.
Third, the robot must consider explicitly the actions
performed by the smart environment and by humans when
deciding its own actions during task execution. For instance,
the robot might ask a person which of two similar bottles
should be grasped. That is, the robot must adapt its own task
execution to the human’s expected actions to eliminate any
possibility of interference or collision. The services provided
by the robot is considered to makes the human’s state change
to be more comfortable, relaxed, healed, relieved, cheered,
luxury, satisfied, convenient, and happy, namely which turn
the humans into better states.
In the symbiotic robotic systems, each component works
to complement each other. The robot can ask the human to
remove an obstacle or to provide additional information for
disambiguiting a situation. The objects in the environment
have an RFID tag that can informs the robot its information,
i.e. its shape, material, size, weight, location, etc. This
information can help the robot achieve its tasks, and reduce
the requirements on robot sensors and actuators.
B. Syndication Format and Google Calendar
1. RSS/Atom Syndication
Syndication is an effective and popular method for
providing and aggregating content. There are two common
XML-based syndication formats for feeds and other data
interchange on the web: Atom and RSS. RSS has been around
longer, in various versions; Atom is more recent but has more
features.
In addition to its syndication format, Atom provides a
publishing protocol; it's an HTTP-based protocol for
publishing and editing web resources, e.g. using POST, PUT,
or DELETE commands respectively. This use of HTTP
requests for a publishing and editing protocol is in the spirit
of the REST approach to web service interfaces.
In various contexts, it's useful to be able to send a query to
a server and receive a response containing a list of matching
results. Unfortunately, neither Atom nor RSS provides a way
to do that, but GData does. We used this feature as an
example application of weather information and calendar
application.
2. GData Syndication Protocol
GData is a new protocol based on Atom 1.0 and RSS 2.0.
To acquire information from a service that supports GData,
we send an HTTP GET request; the service returns results as
an Atom or RSS feed. We can update data by sending an
HTTP PUT request, an approach based on the Atom
Publishing Protocol.
Because GData is built on basic technologies—HTTP and
common syndication models—we can send GData requests
and process the resulting feeds in a variety of ways:
traditional syndication aggregators/feed readers,
JavaScript/AJAX-based clients in a web browser, standalone
applications, or any other approach.
The GData protocol is language-neutral; we can write a
client in any programming language that lets issue HTTP
requests and parse XML-based responses. GData provides a
general model for feeds, queries, and results. We can use it to
send queries and updates to any service that has a GData
interface.
The following table lists GData features and indicates
whether they're available in other related technologies [9].
Using GData, we would connect events schedule of user
calendar as digital world in the internet, and some objects as
physical world in the smart environment to perform symbiotic
robotic systems.
TABLE I
GDATA FEATURES
Feature GData Atom RSS
Syndication Format Y Y Y
Queries Y N N
Updates Y Y N
Optimistic Concurrency Y N N
Authentication Y N N
3. Google Calendar
Google Calendar [11] allows client applications to view
and update calendar events in the form of Google data API
("GData") feeds. Client application can use the Google
Calendar data API to create new events, edit or delete
existing events, and query for events that match particular
criteria.
There are many possible uses for the Calendar data API.
For example, we can create a web front end for our group's
calendar that uses Google Calendar as a back end. Or we can
generate a public calendar for Google Calendar to display,
based on our organization's event database. Or we can search
relevant calendars to display a list of upcoming events on
those calendars.
For research purpose, we used Google Calendar because
GData is very excellent for syndication contents. Using
GData, we would connect events schedule of user calendar as
digital world in the internet, with some objects as physical
world in the smart environment to perform symbiotic robotic
systems.
C. Related Works
Recently, several research groups put out the vision of
symbiotic system and ubiquitous robots. Ueno [7] proposed a
concept of symbiotic information system (SIS) as well as
symbiotic robotics system as one of applications where hu-
man and robot can communicate with each other in human
way using speech and gesture. The objective of SIS is to al-
low non-expert users, who might not even be able to operate a
computer keyboard, to operate robots. Therefore, these ro-
bots are necessary to be equipped with natural interfaces
using speech and gesture. Our research approach with sym-
biotic system takes a different pose as we put much emphasis
on networked robots as well as on web applications interface.
We used seamless communication instead of verbal/non
verbal communication.
Kim [3] coined the term ‘ubiquitous robot’ as a robot in-
corporating three forms of robots: software robot, embedded
robot, and mobile robot; which can provide us with various
services by any device through any network, at any place
anytime in a ubiquitous space. Our approach with ubiquitous
robots is similar. It can be noted that we more concern with
the real application in the smart environment rather than a
simulation in the virtual space.
Saffiotti [8] proposed a concept of Ecology of Physically
Embedded Intelligent System, or PEISEcology. The concept
is a network of heterogeneous robotic devices (PEIS) perva-
sively embedded in the environment. This concept is very
similar with our research on ubiquitous robots and smart en-
vironment. Our research also integrate web service applica-
tion and syndication format as a part of smart environment.
II. METHODOLOGY
The various steps in the proposed human-robot symbiotic
system process are:
A) Smart object analysis to identify all of the properties
and actions that are belonged to the objects
B) Object perception and interaction by developing
properties and actions to provide information and
communication channel for the robot system
C) World modeling based on knowledge information
about the capabilities and state of the smart objects in the
environment
D) Task execution assigning tasks action for the robot
based on information of smart environment and human state.
A) Smart Object Analysis
This step involves a detail analysis to identify all the
properties and capabilities that are belonged to the objects.
Properties are defined as an attribute of the object that can be
monitored or observed by the robot or by the human.
Capabilities of objects are defined as the capability functions
of the objects that facilitate the interaction to the robot. Both
information of objects properties and objects capabilities are
stored into their tag device, therefore the robot can easily read
the information using tag reader. This information is used by
robot to decide what they should do to serve human need.
B) Object Perception and Interaction
In order to the robot have capabilities to get perception
and interaction of the objects, communication channel and
interaction methods between properties and actions have to
be defined.
The robot has two complementary ways to interact with
the objects. First is physical interaction by observing or
manipulating objects. And the second is digital interaction by
querying objects about their properties and their capabilities
actions, for example, information about shape, size and
weight and also store information about the objects; or by
asking them to perform particular actions such asking the
door to open.
Therefore the objects in the smart environment should
possess some capabilities for information storage, processing
and communication, so they have a presence in both the
physical and digital worlds.
C) World Modeling
The robot’s world model must include knowledge about
the capabilities and the state of the smart object in the
environment. It must also include knowledge about the
human’s position, intention, and possible future actions. The
robot can acquire these models through a combination of
direct communication with the objects, linguistic
communication with the humans, and perceptual
observations.
D) Task Execution
The robot must consider the action performed by the smart
environment and by human when deciding its own action
during task execution. The robot must adapt its own task
execution to the human’s expected actions to eliminate any
possibility of interference or collision.
III. IMPLEMENTATION
Implementation of the scenario mentioned in the previous
section is described here.
A) Physical Hyperlink
To link the physical objects, we developed and
implemented physical hyperlinks using two kinds of RFID
tags; active tags and passive tags. Our research group
developed Ubiquitous Functions Activation Module
(UFAM) as active tags for ubiquitous robots. This device is
shown in Fig.1.
Fig.1 Ubiquitous Function Activation Module (UFAM)
In general, UFAM can be embedded into every object in
the smart environment. Using UFAM, the objects in the en-
vironment possess capability for information storage, proc-
essing, and communication, so they have a presence in both
the physical and digital worlds. The robot can easily interact
with these objects through digital interaction and physical
interaction. For example when the UFAM tags included
objects information are attached to some object, robot can
easily understand object name, size, color, how to use or
manipulate it, etc.
B) Ubiquitous Robot
Ohara [6] describe robot are basically composed of
sensors, actuators and controllers. So if we extend this
concept for physical daily objects which has sensors,
actuators and controllers, therefore the environment becomes
one of the robots, so we can define this space/environment as
a Ubiquitous Robot, which is shown in Fig.2.
In the ubiquitous robot, the environment consists of a
number of UFAM tags which has some sensors and actuators,
and several passive tags embedded into every object in the
environment (door, windows, wall, floor, obstacles, objects,
etc). Each UFAM manages and processes local information
depending on specific place or objects. Robots can
communicate with the UFAM via a reader/writer device
provided with each robot, and extract/add/update the local
information in UFAM through local radio communication. Or,
the UFAM inform the robots of the necessary local
information for navigation or task execution, and mediate the
knowledge or commands stored by a robot to other robots.
 shown in Fig.3-4. Our smart environments consist of several
ubiquitous robots and smart objects. Several pasive tags and
active tags (UFAM) embedded into this environment.
Intelligent bookself and intelligent dining table have been
installed (Fig.5).

Fig. 2 Concept of Ubiquitous Robots

Ubiquitous robot can provide various functions, which is

not only an informative service but also physical services. Fig. 4 Smart Environment consisting of ubiquitous robot and smart objects
Ubiquitous robot will provide us with various services (any
service) anytime, at any place, by any device and through any Books in the book self and plates on the table were
network. Following the general concepts of ubiquitous embedded with pasive tags. About 400 pieces of RFID tags
were implanted in the floor for localization. We use laser
computing, ubiquitous robot will be seamless, calm,
range finder, odometry, StarLITE for more precise objects
context-aware, and networked [3].
localization.
C) Smart Environment
The aim of the smart environment is to reduce the
requirements on robot sensors and actuators and to advance
robotics capabilities.

(a)

Fig.3 Ubiquitous Robot in the Smart Environment

For instance, a smart environment can include devices for (b)

opening and closing doors and for moving objects; Fig. 5 Intelligent bookshelf (a) and intelligent dinning table
(b) have been developed in the Smart Environment
distributed Web cameras, RFID tags, or motes embedded in
everyday objects and storing information about those objects; D) User Calendar
gas sensors that are activated by smells; and touch sensors. We created user calendar using Google [12] account to
The computer chips are embedded in all the daily objects and manage user’ s events. The user creates an event through
thing in the smart environment. Using this technology, robot
several ways, such as from a web interface, email client, or
can easily control home appliances such as TV, lamps, air
from handheld communicator device.
conditioner, washing machine, refrigerator, doors, windows,
After an event was created, it will appear on user cal-
etc.
endar. The calendar also read weather information from
In the smart environment, informative objects in the
weather forecast web site, and put it on the calendar as a new
physical world have to be decoded into digital world in order
event. This event then will be aggregated by the middleware
to robot have capabilities to recognize its objects. The key
in the smart environment. This information will be used by
component is RFID (Radio Frequency Identification)
the robots to provide some services to the user.
technology. We have developed UFAM (Ubiquitous
We used Livedoor weather forecast [13] which has RSS
Function Activation Module), a tiny IC chips with erasable
syndication capability for local weather information (Tsu-
ROM, several I/O ports and RF communication module as
kuba/Tsuciura city, Japan). The original information is shown
one of important devices to support this smart environment.
in Fig.6 (a) and the extracted information appear in user
Proposed smart environment with ubiquitous robots is as
calendar is shown in Fig.6 (b).
E) The Mission
If the mission is to provide user same services depending
on weather forecast information, the sequence of information
management controlled by the ubiquitous robot is simplified
as follows:
1. The user wrote event schedules on a handheld communi-
cator and send it to his/her calendar on Google Calendar
2. A middleware in smart environment detected an “event”
from user’s calendar and using feed reader read weather
forecast for that day (Fig. 7).
3. Before the user leaving home, an active tag of middleware
communicate with passive tag on “magic” lamp and send
it weather information of that day.
4. The lamp displayed this information in different colors.
5. A middleware asked the robot to pick up an umbrella
Where
When
What
(a)
What
When
Where
(b)
Fig. 6 Livedoor weather information web site (a) and its information
appear in user calendar (b).
6. The active tag system on the robot communicates with the
passive tag system on the umbrella. Their active tag sys-
tem reads the current information of the floor tags;
therefore the robot and the umbrella know their current
position exactly.
7. The robot localizes the current position and composes the
path to navigate to the target position.
8. The robot navigates to the target position and compensates
the path following error continuously using the informa-
tion from the smart environment.
9. If the robot arrives at the target position, the robot carries
out the given task.
Similar steps can be arranged for other mission, for ex-
ample to clean off dinning table shown in Fig. 3.
℃ ℃
Clear 13 /4
Thu, February 15, 2007
Where: Tsuciura, Japan (map)
Fig. 7 Weather information on user’s calendar
IV. CONCLUSION
In this paper, we have presented a novel approach of
symbiotic robotic systems. Here connecting physical world
and events schedule in the smart environment and ubiquitous
robot are proposed to implement symbiotic robotic system.
To connect the physical world, active RFID tags (UFAM),
passive RFID tags, and many kinds of RFID/IRID are
developed and implemented in our smart environment.
Finally, symbiotic robotic application for weather forecast
has been proposed.
V. REFERENCES
[1]. Weiser, M., The computer for the 21st century, Scientific American,
Vol. 265, No. 3, pp. 94-104, Sep.,1991.
[2]. Weiser, M., “Some computer science Problems in ubiquitous com-
puting,” Communications of ACM, Vol. 36, No.7, pp. 75-84, Jul.,
1993.
[3]. Kim, J.H., Ubiquitous Robot:The Third Generation of Robotics,
Proceeding of the Second American University of Sharjah int’l
Symposium on Mechatronics, Sharjah, U.E.A., 2005.
[4]. Lee,K.H., Kim, J.H., A Suervey of Ubiquitous Space Where
Ubiquitous Robot Will Exist.
[5]. Kaplan, F., Everyday robotics: robots as everyday objects, Joint
sOc-EUSAI conference Grenoble, October 2004.
[6]. Ohara,K., Kim,B.K., Tanikawa,T., Ohba,K., Ubiquitous Spot Service
for Robot Environment.
[7]. Ueno, H., Symbiotic Information Systems – Towards a Hu-
man-Friendly Information Systems.
[8]. Coradeschi, S., Saffiotti, A., Symbiotic Robotic Systems: Human,
Robot, and Smart Environments, IEEE intelligent System, p82-84,
may/June 2006.
[9]. GData, http://code.google.com/apis/gdata/protocol.html
[10]. B.K. Kim, et al, Networked Robot in the Informative Spaces,
ICCAS2005, Gyeonggi-Do, Korea.
[11]. Google Calendar, http://code.google.com/apis/gdata/calendar.html.
[12]. Google Web site, http://www.google.com/.
[13]. Livedoor weather forecast for Tsukuba city, Japan,
http://weather.livedoor.com/area/8/55.html?r=ical20060916.