Part VI Cross-Cutting Issues

Cooperative GMS ICT Framework

Case study: Construction of Lao Lexical Resources
Vincent Berment1, Houmphanh Thongvilu2
1
GETA, CLIPS, Joseph Fourier University
385 avenue de la Bibliothèque Saint-Martin-d'Hères, France, 38400
vincent.berment@imag.fr

2
LaoNet- Lao Human Resources Network
Melbourne, Australia
pan@Lao.Net

Abstract development team with a free and potentially much

The case study of Lao resources development offers a
window into a process for GMS organization to play an
active role in regional ICT development, to enhance the
organization cooperation with some common platform
and tools, for the benefit of its member countries, to
reduce the digital and information divide. It is observed
that most of the languages spoken in the GMS region
(e.g. Khmer, Lanna, Lao, Lü, Tai Dam) are still weekly
computerized or neglected. Among the different steps,
one of the heaviest tasks is probably the development of
lexical resources. In this paper, we present Montaigne-
Lao, an approach for building miscellaneous Lao
resources based on a cooperative effort on the web. We
then discuss how it can contribute to a collaborative
network for the GMS languages.
Keywords: Natural Language Processing, Generalized
Linguistic Contribution, Minority language, Internet,
Word Processing, Virtual Keyboard, Dictionary,
Machine Translation, Unsegmented Writing
Generalized linguistic contribution
During the last ten years, significant efforts have been
driven to produce efficient linguistic tools for
previously unsupported languages. However, this is still
limited to several tens of languages (less than 1%;
Berment 2002b). In particular, most of the languages in
the Greater Mekong Subregion (GMS) are still poorly
computerized even if the most affluent GMS countries
such as China, Thailand and Vietnam have greatly
benefited from their language computerization. The
developing nations in the GMS have limited resources
and more immediate priorities so they can not afford
significant research and development in computational
linguistics. The profit driven research and development
also often sees these needs being unanswered.
Our point of view is that computational linguistic
resources can be efficiently obtained by a collaborative
work on the web (Boitet 1999), replacing a local
larger distributed team. This idea of a “Generalized
Linguistic Contribution” (GLC) on the web, already
present in an early Montaigne project (1996), has been
applied by Oki to the Japanese language1 (Shimohata
2001), by NII and NECTEC to the Saikam Japanese-
Thai dictionary project2 and by miscellaneous
contributors to the Papillon multilingual dictionary
project3. Before that, in 1995, the LaoNet cyber-team
already achieved the development of the so-called Lao
Romanization Transcription (LRT)4.
In this paper, we first present Montaigne-Lao, a
cooperative project for developing Lao language
resources and which initial framework is formed by a
discussion forum, a dictionary with translation support
environment, an open source virtual keyboard, as well
as two ongoing projects, text to speech and Lao New
Coding. In a second part, we discuss the perspective of
a Montaigne-GMS project that would be based on a
principle of exchange.
The efficiency of the presented GLC concept can be
witnessed by the fact that the authors of this paper never
met before the Regional Conference on Digital GMS in
2003, though they worked together for more than one
year and achieved significant progress thanks to their
exchanges.
1
: http://www.yakushite.net/
2
: http://saikam.nii.ac.jp/
3
: http://www.papillon-dictionary.org/
The Papillon project already includes six languages: English,
French, Japanese, Malay, Thai and Vietnamese.
4
: http://www.lao.net/html/LRT.html

342 Berment, V. and Thongvilu, H.

 Part VI Cross-Cutting Issues
A case study: the Montaigne-Lao project5
1. General description
Basically, the Montaigne project’s idea is to offer a free
cooperative work facility on the web for developing
linguistic resources and machine translation tools. The
Montaigne-Lao project is the first implemented step of
this concept and has Lao as target language.
One important question is how to go about
developing a collaborative effort targeting specific
needs, or in other terms how to harvest the goodwill and
to channel the energies towards producing workable
solutions. The cyber-forum described in the next section
is the way we propose for fostering such participation
and for guiding it. It is the basis of an ICT cyber-
cooperation.
More generally, we describe here three already
existing components of the Montaigne-Lao project:
Forum,
Dictionary,
Virtual keyboard,
and two projects:
Text to speech,
Lao New Coding.
We would like to emphasize that, if these existing or
prospective developments have initially been driven
separately6, our discussions on the web allowed the
achievement of synergies and of technical emulation.
2. Forum & mailing list: a method to emulate
the cooperation
It is often said that the Internet has modernized the
workgroup relationship across cyberspace. However a
forum does not necessarily become a project team.
What are the ingredients that would attract common
minds to consolidate the efforts instead of projects
being worked on in isolation, duplicating efforts and
resources?
The keys are: central contact point, shared goals,
commitment, and participants. Obviously participants
are very important as they are the how and the fuel to
keep the central contact forum going. Even if some
people think it is a waste of effort, it can quickly expand
and sometimes replace the traditional networking of
peer groups of people.
To create such a forum is to enlist a group of people
with the same interest, Internet news-groups are a good
place to start for recruits as well as private mailing lists.
If such contact point does not exist, then individuals or
organizations should allocate resources to set it up, to
5
: http://sabaidi.imag.fr/
6
: Mostly in France: dictionary and LaoWord/virtual
keyboard and in Australia: forum, Lao new coding and text to
speech.
foster dialogue among peers, to encourage people to
take ownership of the issues and problems and
thereafter to search and identify means and ways of
realizing the goals.
The initial step requires coaching and mentorship by
an experienced teamwork overseer to manage the
evolution of the group and keep the group focus.
As in any community, cyber-communities suffer
from weak "signal to noise ratio" and effectiveness. It is
often the case that groups of people would break away
into teams that are more compatible working on some
common goals.
The driving force is not always money or fame, but
the desire for the common good, so the 'why' would be
answer by the 'why not'. Often the effort of participants,
mostly in time, could expand their horizon and bring
new challenges, as well as unforeseen solutions.
The resourcefulness and contact of the mentors, who
could see to the group needs, could steer the group up to
some level of self-management. "If there is a will there
is a way" aided by good resource would makes it easy.
Once the means and tools are available for wider
participation, the virtual workgroup could be replicated
to other fields. The trust of the process would bring peer
groups to work on common problems for mutual
bilateral benefits.
LaoNet forum was modeled on such cyber
cooperation; with its limited resource, despite many
difficulties and lack of financial resources we have
managed to achieve some milestone results: Lao
Romanization for Transliteration (LRT) 1995; STEA
Lao IT Technology seminar-VTE August 1995, leading
to the introduction of Internet into Laos; first Lao
History Symposium at Berkeley USA in March 2003;
STEA PAN-Laos ICT affairs seminar VTE March
2003. The forum has brought many, many people to
exchange ideas across the cyber-space without having
ever needed to meet physically to work together.
3. Dictionary and Translation Support
Environment (DTSE)
3.1. Functional architecture
The three following tasks participate to an initial step of
the Montaigne-Lao dictionary project:
a) A group of skilled persons build lexical entries,
partly starting from existing paper dictionaries.
b) A panel of specialists derives a reference on-line
dictionary from the gathered lexical entries.
c) Various tools and applications are developed to
utilize the resource.
An original point is that, in parallel with this work of
specialists, a lexical contribution coming from a
worldwide public is also compiled. This process relies
on a translation support environment provided on the
web site. In particular, a Lao-French bilingual editor is
proposed to the visitors together with a word for word
translation service.
Berment, V. and Thongvilu, H. 343
 Part VI Cross-Cutting Issues
Bilingual editor
Word for word translation
To help them working their personal translations out,
the users thus can ask for a word for word translation of
their Lao text. But in case a word is missing or is not
correctly translated, they also can add new or improved
entries into a personal dictionary that will be used in
priority for their later translations. These personal
dictionaries, as well as the personal bilingual texts
themselves, can only be seen by their owner and are
stored on the server. Then, in order to improve the
reference on-line dictionary (and therefore the word for
word translation service), the users can propose the
entries of their personal dictionaries to be transferred
into the main dictionary. This transfer is done under
the control of the linguists' panel.
Functional architecture
344 Berment, V. and Thongvilu, H.
The Montaigne-Lao dictionary is planned to be
exported towards the Papillon dictionary7.
3.2. Lexicographic issues
From a lexicographic point of view, the project follows
the explanatory and combinatorial lexicology (ECL)
concepts8 so the lexicographic content of the entries
includes the lexical item itself and its part of speech,
optional examples and idioms as well as a semantic
formula, the government pattern and lexical functions.
Lexical items input page
However, fields have been added in order to derive
other applications from the database (paper dictionaries
or machine translation tools). The initial lexicographic
effort has been handled by a group of lecturers and
students of the Institut National des Langues et des
Civilisations Orientales9 (Paris, France) and are
currently being reviewed by the panel of linguists.
3.3. Technical architecture
The dictionary is stored as a MySQL database table as
well as the contributors’ profiles. C code is used for
segmenting Lao texts into words10 and for sorting the
dictionary11. It uses a syllable recognition technology
(Berment 1998) and a longest matching algorithm (e.g.
Meknavin et al. 1997). Text input is possible with the
two currently used Lao keyboard layouts thanks to
JavaScript and to TextArea or Input HTML forms
controls. The LaoUniKey input tool presented in the
next section will be used when the text input will be
done in Unicode directly.
7
: http://www.papillon-dictionary.org/
8
: On this matter, see André Clas, Igor Mel'cuk and Alain
Polguère's book, Introduction à la lexicographie explicative et
combinatoire, Duculot 1995
9
: http://www.inalco.fr
10
: Lao is written from left to right with an alphabet deriving
from Indian scripts. A major characteristic of Lao writing is
that words are not separated with spaces, like Khmer, Thai or
Burmese writings.
11
: Another important characteristic of Lao writing is that
some vowels are placed before the consonant. This contributes
to make the automatic sort of Lao dictionaries more complex.
 Part VI Cross-Cutting Issues
4. Virtual Keyboard (VK)
Deriving from the Lao word processor called LaoWord
which is an add-in for Microsoft Word, a virtual
keyboard was developed to provide a general Unicode
input device for Lao: LaoUniKey12. LaoUniKey is part
of the Montaigne project both because it is available on-
line but also in the sense that its source code is available
under GPL license.
Its code has already been used as a basis of an
ongoing Montaigne-Lao project addressing an improved
Lao keyboard evaluator. Due to a lack of technology
native to operating systems that can handle foreign
language script, the enhanced-VK is designed to create
some smart pre-emptive keyboard across application
and platforms. In the first place, it simply interprets and
maps the keystrokes to character codes (8 bits or 16 bits
Unicode). By implementing some keystrokes buffering,
keystrokes history recall could be implemented. By
adding a lookup of keystrokes translation table, macro
programming of keystrokes is achieved. The macro
single keystroke could be expanded to a stream of
keystrokes, paving a way to implement multi elements
vowel sound of a phonetic virtual keyboard.
By enhancing the keystrokes buffering with some
intelligent rules or a dictionary lookup words correction
could be preempted.
The virtual keyboard is written in C language and is
limited to Windows platforms at the moment .Later on,
by implementing it by using a platform-independent
language such as Java, it will be possible to create a
smart keyboard across platforms.
5. Text To Speech (TTS)
A text to speech synthesizer (TTS) is currently being
worked on to broaden the appeal and usefulness of the
resource, to attract an audience that would sustain and
contribute to the growth of the online dictionary. It is
planned that the service will be an extension of the
dictionary server, where a client application could make
a word to sound lookup request. Such approach would
omit a need for voice synthesizer engine at the client
end. It will also foster simple voice capable front-end
applications e.g. language teaching web page.
The voice synthesizer is based on the public domain
Festival system (http://www.festvox.org/). Although a
full native Lao words to sound library is yet to be
constructed, an intermediate solution is adopted to
demonstrate the concept. With Thai TTS already being
more advanced we hope that a future GMS forum could
see Lao TTS benefiting from NECTEC TTS work.
The current implementation is done with the Lao text
being transcribed using roman characters employing an
earlier work of romanization for transliteration (LRT).
The transcription engine was carefully coded closely
to Lao grammar so to guarantee a reversal transcription.
12
: LaoUniKey's technical principles are described in
(Berment 2002a) at section 4: "An input method using hooks".
Having the text transcribed, the romanized Lao text
is fed into a Festival voice synthesizer engine. Minor
sound rules modification of Standard English sound
library is sufficient to make the engine cope with
producing a workable Lao text to voice system.
Future Lao TTS work could share the same basis of
this approach in terms of word construction, word
break, sentence formation, and utterance rules.
6. Lao New Coding (LNC)
6.1 Lao writing basic
Lao writing is derived from Indian Sanskrit script and
many words are borrowed to build Lao vocabulary.
Word could be made of a single or multiple syllables.
The syllable is formed by: [consonant/cluster
consonants][vowel/derived vowel] [consonant modifier
(optional)][tone (optional)]; this structure is being
referred to as a word-clusters here within.
Lao vowels compose of multiple sub-elements; each
element occupies a predefined position within the word-
cluster, some elements proceed and displace consonant
when invoke. Such symbols compounding is not
supported for Lao within any operating system, the lack
of standards had bred many incoherent implementations
that had confused the computerization of Lao language.
It is interesting to note that the term "Lao grammar"
is classically defined as the rules for Lao speech and
writing, words construction and little inter-linkage of
words. Multi-words words are built by joining a number
of words. Lao grammar does not have extensive word
identifier to explicitly define words relationship in the
multi-words construction.
Traditionally, a Lao text is not punctuated, other than
being organized in paragraphs of long continuous
stream of characters.
So when 'Lao grammar' is referred to in this text, we
are referring to the word-clusters construction rule.
6.2 Current technology
In current technology, Lao words require some five
bytes on average. Keystrokes or bytes stream of Lao
words require a correct order of sequence to correctly
position the elements within the word-cluster. Possible
solution is to assign extra bytes that would preserve
elements' attributes and position information. The recent
font standard, OpenType13 developed by Adobe and
Microsoft, combines TrueType and Postscript
technologies to provide new typographic features such
as the desirable capability of elements compounding,
promises a possible solution to Lao script problems.
Reliance on keystrokes sequence alone could
complicate lexical search and matching of words.
6.3 New encoding
LNC offers a new way of correctly storing symbol
elements' position at bits level within the word-cluster,
in two to three bytes compare with current technology
13
: http://www.adobe.com/type/opentype/main.html
Berment, V. and Thongvilu, H. 345
 Part VI Cross-Cutting Issues

of on average using some five bytes. By efficiently 1. Introduction

using less byte for data representation and storage, LNC
In the Greater Mekong Subregion, there are many
can also be regarded as a compression algorithm and
writings deriving from Indian scripts and sharing a
technique that is tightly related to Lao grammar.
common feature: they are unsegmented in the sense that
LNC is pure research work, and has not yet proven
words boundaries are not marked.
in any product. It is not designed to replace characters
Having the same origin would mean possible many
encoding such as MS code-page, or Unicode, but rather
cross adaptations of already developed solutions. Such
complementing them in a way that data representation is
as with font development, various existing fonts could
compacted and fast to transmit. A cross conversion
be cloned for another character set.
engine or driver program could transparently translate
The cyber-synergy described above for the Lao
the data. However raw LNC data would not be readily
language computerization (forum, lexical resources,
readable. LNC will be an optimized use of memory in
open source, downloads) may already exist for other
applications e.g. Virtual Keyboard buffers, with fast
languages and could be generalized. This section
word search and recall. LNC can guarantee a unique
discusses several possible topics where synergies could
representation of the entered words disregarding the
be found. The implementation phase could be supported
sequence of keystrokes.
by specific project grant and sponsorship. We propose
to generalize an existing Lao word processing facility to
7. Available on-line resources other unsegmented languages of the GMS. We further
Before developing dictionaries, text-to-speech or other highlight issues and problems relating to implementing
more sophisticated resources, a written language first an effective online lexical resource.
needs the basic tools that allow it to be written in
computers. For languages such as Lao which has its 2. GMSWord platform proposal
own alphabet, those primary tools are fonts and virtual
In this section, we propose to reuse the LaoWord17
keyboards.
software and to build a GMSWord from its features.
Lao language computerization has been industry
LaoWord is an add-in for Microsoft Word that performs
driven, implemented with varying degrees of success
specifically Lao word processing functions18:
and acceptance. It is mainly based on Microsoft
1. Virtual keyboards:
Windows platform, with two across applications
Miscellaneous virtual keyboards are available, both
products dominating the Lao MS Windows system:
adapted to national physical keyboards (e.g. French,
LaoWord14 and LSWIN15. On Unix platform the project
US, Danish) and with different layouts (traditional
of adding foreign languages support or internationalize
typewriters or intuitive layouts).
(I18N)16 or localization of Lao language at commands
2. Non-standardized font encoding management:
and messages level is approaching fruition. Thereafter
Several encodings have been used by font's designers
applications that are compiled with I18N awareness
making text input and font change quite complex.
would have full support of the language.
LaoWord contains the necessary software for
The MS Windows LaoWord and LSWIN solutions
providing the user with a virtual encoding standard:
are based on mainly solving keyboard input of Lao
he doesn't have to matter which font he uses both
language with their respective keyboard translation
during text input and font changes.
application: LaoKey and Keyman. Both input
3. Text selections by entire syllables:
applications have been updated to allow Unicode entries
The absence of spaces between Lao words makes
(LaoUniKey is the version for LaoWord).
Microsoft Word selections rather chaotic with the
Lao TrueType fonts have been developed since
Lao unsegmented writing. The LaoWord syllabic
1992, mainly thanks to Fontographer. Most of them are
selection mode makes the selections of Lao texts
free and can be downloaded from the Internet. Credits
more natural (syllable by syllable), for example
go to many font designers (such as John Durdin, author
before a copy of text into the clipboard.
of LSWIN) who had made serious attempt in making
4. A small Lao-French / Lao-English lexicon:
Lao language work across applications and the artful
The French or English translation of a Lao word can
skill for rendering practical fonts despite Lao small
be obtained by selecting the word and clicking on the
market. Useful Lao language computerization resource
translation button.
could be found at http://park.lao.net/references/.
5. Several romanizations:
The automatic romanization of a Lao sample can be
obtained by selecting the text and clicking on the
A scheme for computerizing GMS
unsegmented languages 17
: See http://www.LaoSoftware.com/ or
http://sabaidi.imag.fr/.
14 18
: http://www.LaoSoftware.com : See LaoWord user's manual for a more detailed
15
: http://www.tavultesoft.com/ description. It is included in the LaoWord package
16
: http://www.kde.org/international/ (http://www.LaoSoftware.com/).

346 Berment, V. and Thongvilu, H.

 Part VI Cross-Cutting Issues
romanization button. Three different romanizations
are proposed: free (i.e. only making use of the Latin
alphabet), a phonetic and a reversible transcriptions
both using the International Phonetic Alphabet.
6. Lao table sort:
Tables with text in Lao cannot be directly sorted with
Microsoft Word means. LaoWord allows a sort of
Lao tables. It can use any of the different rules found
in the dictionaries (selectable).
7. Miscellaneous Lao text finishing functions:
Lao making-up into pages,
Height selection (high/low) for accents,
Height selection (high/low) for "sala u",
Ligature of accented vowels.
While Microsoft Windows still represent the
majority of the available platforms, GMSWord would
offer a suitable effective tool for the GMS languages
with unsegmented writings. The same methodology
being implemented for the different languages, it would
be more easy to define common communication tools.
Note that a BurmaWord has already been prototyped
for testing the Burmese Unicode capability.
3. Character encoding standard, data format
3.1 Reducing the brakes on development
Within the past ten years despite the neglects and small
commercial viability, language such as Lao had
received enough interests and support to emerge with a
workable set of tools. However due to uncoordinated
efforts, many diverging technologies and conventions
exacerbate the problems. Several issues had been
identified that had hampered the development of this
Lao lexical on-line resource:
Lack of workable Lao standard character coding.
Lack of Lao script support native to current
operating system affecting keyboard input.
Resource: human and financial.
We present hereafter some solutions for Lao that we
currently discuss in the forum or already use and that
could also be applied to other languages19.
3.2 Lao standard character coding
Code-page scheme of MS Windows to display foreign
character sets combined with the lack of a Lao workable
character standard had created a situation where an
exchange of Lao data does not always convey the
relevant information. This is an issue that the Lao
lexical online resource has to develop some standard or
means of conversion:
Adopt Unicode,
Provide utilities for conversion,
Develop a convention to embed font information, to
achieve some data uniformity.
3.3 Lao script support, keyboard input
19
: See also (Berment 2002b) for a general approach.
As previously described in section 4 (Virtual
Keyboard), the implementation already exists for Lao
and is fully adaptable to all GMS languages.
3.4 Resource: human and financial
Currently most of the work on Lao lexical development
is driven by a group of volunteers. To fast track its
development would require the support of resourceful
hosts who share our vision. The goodwill and the
human resource do exist, ready for the initiative to
gather the efforts.
3.5 Data format
The data record format has to be carefully designed to
compromise between compactness and flexible data
fields of information. As Lao text is made of continuous
stream of characters, it is hard for unfamiliar reader to
dissect the word(s) boundary. It is further complicated
by a lack of clear rule base multi-words construction.
To record all Lao words would mean wasting of
resource in duplicating many word entries. Retrieving
words from a dictionary would then be pure strings
search rather than rule base.
By having a rule based database could create a more
concise and smarter dictionary. Therefore data records
need to be selective in how words and what relevant
information are stored, how multiple-words are linked.
It is seen to be necessary to introduce word identifiers
and deriving words dependencies and relationship.
Another aspect of Lao words is dialectic variants, with
different intonation and consonants used in words.
Incorporating all these issues, a possible data records
format is such:
1) Single word Unicode encoded,
2) International phonetic notation,
3) Romanized scheme wording,
4) Variants spelling (intonation, and consonant),
5) Word identifier/type (e.g. adj., noun etc.),
6) Links to multiple wordings,
7) Description,
8) Comment.
To ensure a quality words retrieval, a client-server
architecture is a key feature that would allow
maintaining a uniform and centralized database.
Centralized database of significant hits answering to
remote requests could suffer some poor performance.
To improve on availability, it is possible to distribute
traffic load to multiple slave servers that replicate from
and update to a single master database.
By centralizing the database it would be much easier
to collect statistical data of the words access and usage.
Such data is relevant in further enhance the design of
more intelligent lexical rules.
The implementation of a client-server architecture
would achieve an across platforms solution, favoring
HTML, SSI, PHP, JavaScript and compiled C code used
as CGI technology.
Conclusion
Berment, V. and Thongvilu, H. 347
 Part VI Cross-Cutting Issues
Computerization of a language is a pride for its speakers
and a natural right for all linguistic groups to possess
the same minimum set of tools for effective
communication of the time. Dialogues break down the
barrier of prejudices and ignorance. It establishes a
working relationship build on trust. Trust and
understanding ensure cooperation for bilateral benefits.
Though the Montaigne project started with an
application to the Lao language, its principles can be
applied to any of the GMS languages.
References
Berment V. 1998. Prolégomènes graphotaxiques du
laotien. DEA dissertation, Inalco, Paris, France.
Berment V. 2002a. Several Technical Issues for
Building New Lexical Bases. Papillon seminar 2002.
Berment V. 2002b. Several directions for minority
languages computerization. COLING 2002.
348 Berment, V. and Thongvilu, H.
Boitet C. 1999. A research perspective on how to
democratize machine translation and translation aids
aiming at high quality final output. MT Summit 1999.
Meknavin S., Charoenpornsawat P. and Kijsirikul B.
1997. Featured-based Thai word segmentation. Natural
Language Processing Pacific Rim Symposium 1997.
Shimohata S., Kitamura M., Sukehiro T. and Murata T.
2001. Collaborative Translation Environment on the
Web. MT Summit 2001.
Thongvilu H. (LaoNet) Lao Romanisation for
Transliteration STEA Lao IT technology seminar –
Vientiane August 1995.
Thongvilu H. Lao New Coding (LNC) PAN-Laos
National ICT affairs seminar – Vientiane 10th March
2003. (to be published)