Académique Documents
Professionnel Documents
Culture Documents
Language Environments
Topic Area: Knowledge Management; Innovation and Technology
1. Introduction - The Problem of Foreign Languages for the Tourist Industry
The tremendous expansion in tourism in the two past decades, in line with declining
costs for travel, has allowed more people to travel who lack the advanced linguistic
skills necessary to optimally navigate foreign environments. This is particularly true in
regard to less well educated, nouveau riche travelers from newly industrializing
countries with a desire for foreign travel and experiences in exotic locations such as
Europe. These travelers, though perhaps capable of reading signs using what many
perceive to be difficult Chinese ideograms or some other script, could be entirely baffled
by signs using common Western languages such as French or English.
At the same time, more highly educated and affluent travelers from places such as
Europe and America are visiting tourist destinations where their skills are severely
challenged by languages that are quite different in form from the languages they are
familiar with. Although English has become a lingua franca for much of the tourist
industry, English and other Western European languages are still not yet a part of the
everyday environment for more than perhaps a quarter of the worlds population.
Consequently, attempts by more affluent travelers to achieve an authentic experience in
a radically foreign environment will inevitably be constrained by the limited linguistic
access afforded to the traveler. This difficulty is especially felt when alphabets are nonLatin in their form, or in environments where pictographs are used in place of alphabets.
The linguistic cues found on signs and other textual information sources in foreign
environments are hidden from the traveler as a result of linguistic differences, and
threaten to make the tourism experience difficult or even potentially dangerous.
Technology has stepped in to help with these problems, offering translation software,
electronic dictionaries, and the like, but problems still persist. Electronic dictionaries, for
example, are of little use when there is no alphabet to facilitate navigation, as is the
case for logographic/pictographic languages such as Chinese or Japanese, or in the
case where alphabets are non-Latin in form, such as for Thai or Korean. What is
needed is a tool to magically transform the environment into one that is linguistically
familiar to the traveler, without the need to actually alter the environment from the
perspective of the native. In this presentation we will look at how augmented reality (AR)
is making the whole world accessible to any traveler, regardless of native language or
foreign language experience.
This paper will, for practical reasons, focus only on the problems faced by foreign
travelers dealing with text or pictographic-based information sources. It is recognized
that a great deal of information is available only through verbal communication, and this
information will remain unavailable to the tourist as long as we lack real-time verbal
speak some English and thus might be able to assist our traveler. Finally, the OCR
software within the camera has translated the objects, or Japanese text, on the
package into Italian, the native language of the traveler, and our traveler now has found
the Japanese version of the familiar laxative product.
All of this AR magic is made possible as a result of something known as an augmented
reality browser, similar to the browsers used on personal computers, to navigate the
world through the lens of the smartphone camera. The video feed provided by the
smartphone, combined with a network Internet connection to a central database of
information, transforms the environment seen by the traveler through the browser into a
world filled with relevant information, in the native language of the traveler. Using the
markers, location, and/or objects we have just mentioned the AR browser is able to
generate the information they overlay on our environment.
3. AR Technology with Markers, Location, and Objects
Markers represent the least technologically advanced means for implementing AR. A
marker is something such as a barcode that is consciously added to the environment in
order to provide information. In most cases it requires that the environment be altered in
order to provide information that can be used within the environment, since barcodes
and other common forms of markers are not usually embedded on anything other than
products. One might imagine, for example, a sign with a barcode pasted to the bottom
of the sign. The barcode or marker can be scanned by the video feed from a
smartphone, and that code will be deciphered by the central database to overlay the
appropriate textual information on the browser window to the augmented reality world.
The barcode or marker might, in our sign example, indicate we are in a tsunami zone
and need to exercise caution. Perhaps we also noted the wave on the sign, but were
not certain if this indicated this was an ideal location for swimming or surfing, or if
danger was indicated. The amount of information embedded in a barcode can be large,
so we might also learn the sign was put in place last year, and therefore the information
is fairly current and probably worth considering.
Augmented reality applications based on location, as provided by gravimetric or GPS
technology, do not require the environment be changed in order to overlay information
on the environment. These applications are more flexible than marker-based AR, but
require greater technological sophistication from the AR device than marker-based
systems. They of course require that the AR device, such as the smartphone, be
equipped with a means to compute relative location on the basis of GPS signals. At
present this is one of the limiting factors in the adoption of AR, as commercial GPS
devices do not yet have the accuracy to locate objects as small as a sign with pinpoint
accuracy. Although the military is able to blow up ships, buildings, and other objects with
missiles coordinated by GPS signals, commercial users can presently expect nothing
more than to be able to identify buildings on the basis of location, and even in these
cases errors of up to hundreds of meters is not uncommon. Therefore, the type of
information that can be overlaid on an environment using location-based AR is still
limited by errors in GPS signal accuracy. Nonetheless, once a location is established,
AR technology can begin to provide the user with a wide variety of information about the
area. That information can come from databases used by Twitter, Facebook, or other
online sources of social information.
The final form of AR we are looking at involves object recognition, and the subset of
OCR-enabled translation. Although we do not need to know the location of the object to
provide this type of information, the processing requirements for translating information
using object recognition place a substantial strain on todays smartphones, and robust
network connections are also a necessity in most cases. In our sign example, no
barcode or marker would be required if we could rely on our smartphone and its OCR
software to translate the Japanese characters on the sign. There would be no need to
physically alter the environment in order for our visitor to have the same information,
through AR technology, native Japanese visitors to the area would have. Fortunately, a
great deal of effort has gone into developing OCR and object recognition algorithms,
and a number of smartphone applications have already been introduced.
4. Smartphone Translation Applications and Limited Augmented Reality
The potential benefits of augmented reality are so great because browsers will be able
to combine marker, location, and object recognition technologies to construct an entire
world of information unavailable to tourists who cannot read the language used in the
country where they are traveling. However, even today tourists have access to more
limited applications that use just a small spectrum of AR technology. Instead of viewing
the world through the augmented reality browser, for example, we are able to use our
smartphone cameras and software such as Word Lens, Camera Translator,
CamDictionary, Photo Translator, or similar applications to provide limited OCR-based
translation of signs. Even these more limited applications work remarkably well, with a
great number of language choices, and can be of tremendous assistance to travelers. It
is now possible to travel in Japan, China, or other exotic environments without worrying
that signs will be unintelligible without a working knowledge of the foreign language.
Lesser used languages, such as Lao or Cambodian, at present lack the population base
to make the development of OCR translation software commercially viable, but it is only
a matter of time before we will also have applications to make reading signs in Vientiane
or Phnom Penh a trivial matter.
Some countries, of course, already translate some of their signs into English,
particularly in large cities, and this translation already greatly facilitates ease of travel.
Travel to more remote and less densely populated locations, however, still presents
difficulties, as governments lack the economic resources to provide translations for each
and every sign that is put up. Moreover, while English translations are certainly
welcome, they are not a solution to the navigation problems of every foreign visitor. As
we pointed out at the beginning of this paper, due to declining transportation costs,
greater ease of travel, and higher incomes all over the world, more people who lack
basic foreign language skills are now traveling abroad. English translations might be
useful to more educated tourists, but of limited use in its raw form to these travelers.
Nonetheless, there is still a case to be made for providing English or some other
Western language translations on signs whenever possible, and particularly in countries
where text is not alphabet-based, or not Roman in its form. English itself is an object
that can easily be decoded by OCR software available today. The Italian visitor to
Cambodia, for example, might not be able to read a sign in the native Cambodian/
Khmer alphabet, but if that sign included an English translation an educated traveler
would probably be able to understand the information. With the assistance of an OCR
translation application, the English itself could even be translated into its Italian
equivalent to make the information available to anyone able to read Italian. Eventually,
when Khmer-Italian translation software is available the English language translations
would become unnecessary, but until that time they provide a method to make the
information accessible to the widest possible group of travelers.
5. Practical Augmented RealityApplications for Language Assistance in Tourism
The least technologically sophisticated augmented reality applications have the
potential to be most easily achieved, but they will not necessarily be implemented.
Attaching markers or barcodes to existing signs, for example, would allow for reliable
transmission of information in any language, providing tourists with translations for signs
such as high voltage or keep out. While some signs already provide pictographic
images to indicate no smoking or some similar concept, a bar code could provide
more sophisticated information in any language needed by the person using the
smartphone as a decoding device. It is quite easy, for example, to provide instructions
such as do not enter with a short barcode. One need not worry about the language of
the person scanning the bar code, since the smartphone would provide the correct
translation of the barcode information. Providing the same information in English would
also be useful, but to a person unable to understand English the barcode would be just
as useful.
Barcodes and markers can be decoded locally, with software residing on the
smartphone itself, if the set of phrases or languages required is limited. In this case no
Internet connection would be required. For true AR implementations, however, a
broadband Internet connection to enable access to a central database where a larger
set of information is required. For more sophisticated AR implementations, a variety of
databases will be queried, to provide even more information. In fact, the amount of
information available through AR browsers, even today, has the potential to overwhelm
the user with essentially irrelevant or misleading information. (Julier, Simon et.al., 2000)
What AR technology will eventually provide is a type of filter, or filtering algorithms, to
discard information we do not really require while gathering as much relevant
information as possible. We will also require a filter to authenticate information,
guaranteeing it is accurate, unbiased, and current, as shall be mentioned in the next
section of this paper.
The information problem presents itself because AR information will certainly be
primarily user-generated, as most Internet-based information is today. The most capable
augmented reality browsers in use today are constructed using something known as
ARML, or augmented reality markup language. As the name suggests, these browsers
use the same type of HTML/XML commands to construct their version of the real world
that web browsers use to construct our view of the World Wide Web. Developers of the
software and information that will be needed to make the augmented reality universal
language translation dream a reality will need to become proficient in ARML, and learn
how to work with at least one of the major AR platforms, such as Layar, Junaio, or
Wikitude. I believe the travel industry can expedite the adoption of AR by producing the
framework content creators will need to seed the AR world with information. It can also
provide some system to authenticate information in order to guarantee that users
receive accurate, unbiased, and current information. It would be in the best interests of
the travel industry to be proactive in its embrace of AR technology, working with major
AR players in order to help establish standards.
6. Future Challenges for Augmented Reality - Hardware Issues
Although it is most likely that augmented reality will initially become a part of the
travelers world through a device such as the smartphone, Arth and Schmalstieg (2011)
believe that smartphones are not yet truly ready to deliver full augmented reality to the
traveling masses. One basic problem smartphones have is they are not meant to
provide continuous camera operations for long periods of times due to limitations in
battery technology. I do not believe this is a serious problem, however, as I expect
augmented reality will be used only when needed, rather than on a continuous basis. I
believe that in the immediate future we will view the world through our own eyes for
most of the waking day, and only access an augmented reality version of the world
when we need specific information.
Other challenges for augmented reality relate to the slow processor speeds of
smartphones. OCR translation will certainly present the traveler with short time lags, as
a result of these slow processing speeds, making it impossible for road signs to be
translated instantaneously, for example, from a speeding car or train. Standing in front
of a sign, though, and waiting for a translation, will not be a great inconvenience for
most travelers.
A more serious issue relates to network speeds, and network availability. Network
connections will certainly be required for sophisticated AR applications to access all
information relevant to the traveler in the foreign environment, and these connections
should be as fast as possible to enable transfers of the large volume of information the
traveler will need to construct a complete augmented reality environment. Unfortunately,
due to the fact that augmented reality developments for smartphones are at the mercy
of hardware and network service providers, who make their decisions based on factors
that may not take the needs of augmented reality developers in account, this will be a
constraint on the use of augmented reality. We will certainly eventually begin to see
independent augmented reality devices brought to the market, perhaps in the form of
goggles, for example, that are developed specifically with the power and network needs
of augmented reality applications in mind.
response was an aberration. What I still will not know, however, is if the person posting
this review consistently produces these types of reviews. A good filter would allow me to
eliminate all information this person posted if I discovered this person had no credibility.
Augmented reality browsers will certainly filter the information they provides to the
traveler and, as is the case for Google and other search engines, the way in which the
filtering is performed will determine how useful the information will be. Filtering, though,
also represents a form of control over expression, and we need to consider the
implications of this form of censorship.
There are also important privacy concerns that are related to augmented reality
environments, and these need to be considered by anyone involved with this new
technology. In one sense we are all now consumers as well as products. This idea goes
beyond the term prosumer, used by Alvin and Heidi Toffler, which was coined to
express our dual function as producer and consumer. (Toffler, 2007) But as human
social products, our very location, our being, becomes the source for data consumed
by others. To what degree do I have the right to deny others knowledge about my
location, about who I am, and when I surrender this right by checking in with a service
such as FourSquare, do I lose all control over those rights? These are some of the
concerns I hope the tourist industry will address when it works with AR developers to
help develop standards and establish methods by which user-created information can
be evaluated for credibility, timeliness, and relevance.
8. Conclusions
A successful tourism experience presents the exotic in a form that is understandable
and in some way familiar to the traveler. It does not overwhelm the cognitive or sensory
inputs of the traveler. The widespread adoption of augmented reality browsers, and
development of the software and information needed to enable them in any foreign
environment, as demonstrated in this presentation, will make the linguistic difficulties
and associated cognitive overload of past travel a forgotten inconvenience. Merely
viewing the world through one of these augmented reality browsers, most likely through
a device such as a smartphone, will transform the world and the tourist experience for
the traveler.
References
Arth, Clemens and Dieter Schmalstieg. (2011) Challenges of Large-Scale Augmented
Reality on Smartphones, ISMAR 2011 Workshop: Enabling Large-Scale Outdoor
Mixed Reality and Augmented Reality.
Azuma, Ronald T. (1997) A Survey of Augmented Reality, Presence: Teleoperators
and Virtual Environments, 6: 4 (August), 355-385.
Azuma, Ronald, et. al. (2001) Recent Advances in Augmented Reality, IEEE Computer
Graphics and Applications, (November/December), 34-47.
Bimber, Oliver and Ramesh Raskar. (2005) Spatial Augmented Reality: Merging Real
and Virtual Worlds. Wellesley, MA: A. K. Peters, Ltd.
Cawood, Stephen and Mark Fiala. (2008) Augmented Reality: A Practical Guide.
Pragmatic Bookshelf.
Feiner, Steven, Blair MacIntyre, Tobias Hollerer and Anthony Webster. (1997) A Touring
Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban
Environment, Personal Technologies 1 (4), 208-217.
Furht, Borko, ed. (2011) Handbook of Augmented Reality. New York: Springer Science
and Media, LLC.
Hollerer, Tobias and Steven Feiner. (2004) Mobile Augmented Reality in
Telegeoinformatics: Location-Based Computing and Services. Hassan Karimi and Amin
Hammad, eds. Boca Raton, Florida: CRC Press, LLC.
Julier, Simon, Marco Lanzagorta, Yohan Baillot, Lawrence Rosenblum, Steven Feiner,
Tobias Hollerer, and Sabrina Sestito. (2000) Information Filtering for Mobile Augmented
Reality in Proceedings of the IEEE 2000 International Symposium on Augmented
Reality.
Kent, James. (2011) The Augmented Reality Handbook - Everything you need to know
about Augmented Reality.
Madden, Lester. (2011) Professional Augmented Reality Browsers for Smartphones:
Programming for juniao, Layar and Wikitude. West Sussex, UK: John Wiley & Sons.
Toffler, Alvin and Heidi. (2007) Revolutionary Wealth: How It Will Be Created and How It
Will Change Our Lives. New York: Doubleday.
van Krevelen, D.W.F. and Ronald Poelman. (2010) A Survey of Augmented Reality
Technologies, Applications and Limitations, International Journal of Virtual Reality 9 (2),
1-20.