Testing International Software

I have several friends who are STEs (Software Test Engineers) who will take exception to the
placement of this chapter at the end, wondering why I am choosing to send testers to the "back of
the bus," as it were. There are two very good reasons for this:

• Testing is obviously something that is usually done after a product is developed rather
than before, which makes putting this chapter at the end sensible.
• The job of the tester overlaps so much of the information contained in this book that if I
had not separated it there would have been a million "note to test:" references throughout
the book.

This chapter, therefore, will serve as a central place that someone who is a tester but who has
never had to handle i18N issues before might not want to skip. If you are a developer, you should
also take a look, just in case your testers decide to buy this book and you want to keep down your
bug count!

The best place to start is in deciding what to test.

Planning Out What to Test?

My first bullet point up above is a little glib because testing does not have to wait until the product
is done; it starts at the beginning as the tester reviews specifications and makes estimates as to
how long it will take to test the product that is being described. This is a crucial factor in
determining what features should be cut and perhaps entertained for future versions when there
is more time. This section discusses the issues to keep in mind.

There are many different configurations that tests can be run under, and I have tried to
summarize them in Table 16.1.

Table 16.1 The Test Environments to Consider

Category Operating System Settings Hardware

1 NT/Win2K Regional Settings set to other U.S./Any

locales

2 NT/Win2K Category 1 plus locale U.S./Any

keyboard installed

3 Win2K Category 2 plus changed U.S./Any

default system language

4 All Fully localized OS (on Win2K, U.S./Any

swap language via MUI)

5 All Fully localized OS Locale specific

Obviously, testing with the Category 5 in Table 16.1 will find the largest number of bugs related to
i18N, but it is often not practical to have to install multiple language versions of the operating
systems to do this. There are three points you will want to keep in mind:

• Testing under Windows 2000 is the smartest and most cost-effective way to test the
behavior under your other application because you can change regional options and
default system language so easily. You can further extend the ability to compare
language with Windows 2000 terminal services.
• As you move through the four options in Table 16.1, the amount of work it will take to test
the product will increase significantly (tenfold or more by the time you reach Category 5).
• The number of bugs you find that you would not have found with one of the
configurations given earlier in the table will decrease significantly. (Typically, less than 1%
of all defects that might exist are found under Category 5.)

With these points made, I will now look at each category individually. The decision for how much
to cover is often not up to a tester directly, but the tester's input on how much work will be
required to cover the product can be vital, so the information must be carefully considered and
balanced in relation to other factors such as market requirements.

Category 1: Regional Settings

The most important bugs that will be found here relate to the basic i18N issues in Part I of the
book: Can data be correctly input and is the data that is input subsequently handled and output
correctly? Does this apply to the various problems and issues with date/time values, numeric
values, and currencies? What happens when calendar settings change?

This testing should center on the control panel. Often, the best way to test your application is to
have unique and bizarre language choices for your regional settings-perhaps even customizing
formats at times. When you report bugs, make sure to accurately describe the settings you used
because they can be very difficult to remember when you change them often. There is nothing
more frustrating to a tester to have found a bug and then not be able to find it again!

In addition to the general testing of different settings, choosing to focus some of your time on the
actual configurations that you know your users will have is also valuable. But do not think that the
general testing is a waste of time; the best place to find such bugs is in locales that neither
developer nor tester was really thinking about. It will also give you more confidence when new
markets are considered.

Category 2: Input Locales and Data Storage

All the bugs in Category 1 are, of course, possible to find; in addition, you will be looking for bugs
related to the input of native characters and their persistent storage. Generally speaking, the
amount of work to do Category 2 is already being expended for Category 1, so it makes sense to
consider fuller coverage for roughly the same cost in terms of planning.

Of course, the scenarios multiply as soon as you agree to cover more, so you want to make sure
that you have people in your test organization who are ready and able to change their settings
constantly-in possibly more disruptive ways than Category 1 would ever be.

Category 3: Default System Language

All the bugs in Categories 1 and 2 are possible to find, and a whole new array of bugs is possible
as you look for bugs related to native characters on other code pages and how the application
under test (AUT) behaves on these locales.

It makes sense to plan on testing specific families of languages, such as the following. (Each item
in this list will include the types of bugs found in other families.)

• At least one language on the same code page that requires a different amount of space
to be taken by text-German or Dutch is a good example if you are starting from the
United States. The U.S. is a good example if you are on German or Dutch and want to
make sure that dialogs are acceptable when less space is being utilized.
• At least one language on another code page-Greek, Russian, and Turkish are good
examples if you are starting from the U.S. or Western Europe.
• At least one Asian language-Chinese, Japanese, and Korean are good examples, with
Japanese being the most common choice.
• At least one bidirectional language-Arabic and Hebrew are good examples, although
Arabic will often find more bugs, due to the regular nature of Hebrew characters with little
to no shaping rules that affect the characters in the language.
• At least one complex scripts language-Thai and Vietnamese are good examples because
they have some of the complex script shaping or line-breaking rules in which it is good to
test the application's behavior.
• At least one Unicode-only language-Hindi is a good example.

Obviously, you do not necessarily need to pick all these categories. For example, unless you plan
to invest a great deal of time in Part II of this book and in support for multinational applications,
you need to spend as much time on the Unicode-only languages.(You can assume that they will
simply not work!)

Category 4: Faux-Localized Operating System

This category can, of course, find all the bugs in Categories 1, 2, and 3 (although how much of
Category 3 is covered depends on your language choices). Generally, the additional types of
bugs that you will find relate to components (including common dialogs and localized components
such as Internet Explorer or Microsoft Office) provided by the operating system, which can vary
across languages.

Windows 2000 Multilangauge User Interface (MUI) and Office 2000 Language Packs are the
ideal (and, in fact, only) way to test this category, but it is worth considering whether you have
strong dependencies on such components in your AUT.

Category 5: Truly Localized Operating System

This category is the ultimate level of testing which can find bugs in all the other categories (how
many Category 2 bugs are covered depends on your language choices). The resources required
to test this scenario are often too large to realistically consider, although if you have a large
enough product and are planning to automate the testing of it.

An MSDN Universal or Professional subscription is required here because it can get you localized
versions of all the operating systems. Spending the time to learn how automated setups work is a
good idea for languages that you do not know because you will not have to spend as much time
worrying about error messages that you do not understand during setup (something I know from
personal experience!).
Choosing Which Category from Table 16.1 to Support

The choice of which categories to support can be a complex one, and you will seldom find a more
complex problem in need of cost/benefit analysis. Generally, doing the majority of testing in
Category 2 with spot-checking in Category 3 provides the best balance. As you become more
experienced with the developers and testers in your organization, you will not only become more
sophisticated in testing the product, but your developers will better understand the issues and
they will produce fewer i18N bugs.

Installation of the Application Under Test (AUT)

Although this step seems trite, I can assure you that it is not. It is important to install on a
localized operating system and deal with situations such as the fact that common directories (for
example, Program Files) might have localized names. This can underscore two different types of
problems:

• The program might not be installing to the proper location.

• The program might have problems installing to locations that include native characters in
the path.

These are the sorts of issues that you can test with every install.

For your test environment when setting up applications that use SQL Server, your test server
should be case sensitive because this can often discover problems such as those with the
Turkish I that I discuss in Chapter 4, "User Interface Issues," and Chapter 12, " Jet, SQL Server,
and Other Databases." Choosing different sorts for the server can find problems as well. Proper
set up of your test SQL Server can avoid many problems that would otherwise be found by your
customers. Regularly reinstalling SQL Server with different sorts is also a good way to find bugs
that would otherwise be missed.

Execution of Your International Test Cases

The subject of doing breakouts that list all the important scenarios to test (also known as a test
breakout, which contains test cases) is obviously beyond the scope of this book. However, I will
list many of the issues to keep in mind as you create the test cases for the AUT.

Things to Keep in Mind for i18N

For globalized applications, here is a sample of the types of issues you will want to consider:

• Check every place that you can enter a date, time, currency, or number in every place
that accepts these values in the AUT. Make sure that you can enter them with the correct
format. (See the Regional Settings dialog to find these items for a given locale.)
• Check places where the AUT is doing comparisons, validations, or calculations on dates,
times, currency, or numbers. Accepting correctly formatted values doesn’t guarantee that
the calculations on these values understand the format.
• Check every place that you can enter native characters into text fields. Be sure to enter a
mix of non-English and English characters.
• Make sure that native characters can be displayed properly after they are entered. If the
native characters require a special font (for example, Japanese), is your application
installing this font on the machine? (Or are you guaranteed that it will be available for
every Windows installation?) If the native characters need a special font to display, you
 need to be sure to check every location where text is displayed in your GUI. (Typically,
each control has a font property that may be set individually.)
• Enter the maximum number of characters (using native characters) into each different
“size” control in the GUI. Because Unicode and DBCS characters take up twice as much
room, controls might appear to work for small strings, but are not allocating enough
space for large strings.
• Look our for display problems relating to bad Unicode-to-ANSI conversions, which
characteristically show a question mark (?) instead of a native character.
• Along the same line, look out for display problems relating to bad font choices, which
usually show vertical bars or boxes instead of valid letters.
• Generally speaking, garbage text is due to incorrect conversions from ANSI to Unicode,
and you should look out for this.
• To the extent that you can, it is important to make sure that data that should be sorted is,
in fact, appropriately sorted.
• If your application has any persistent storage (files, database, and so forth), be sure to
test that native characters can "round-trip"; that is, they can be entered, persisted to
storage, reloaded, and displayed correctly in the GUI.
• If the bug also happens with pure English data on an English system, it is not an i18N
problem, and you should not label it as such.

Things to Keep in Mind for L10N

In a localized application, there is a whole range of test cases that must also be considered, such
as the following:

• Check each dialog, and each place where text appears for English strings.
• Check the accelerator key (the letter that has the underscore that enables a user to hit
ALT+<LETTER> to access that control) for each control and menu item on each dialog.
(Frequently when localizing, several controls will be assigned the same letter, which
effectively disables the accelerators for the duplicated controls.)
• In a more compact source language, check the each place where text appears to make
sure that it’s not cut off. For example, most non-English words are much longer than their
English counterparts.
• Bring up each error message to make sure that the strings are being handled correctly. If
possible, have a native language speaker read the dialogs and ensure that they make
sense. Many error messages are concatenated from several English strings (each of
which might not be correctly translated to the locale), and are then presented to the user
as one string (which most likely will not make sense in another language). I discuss this
problem in more detail in Part III, "Localization."
• Review your application for pictures that substitute for words with a native speaker, if
possible. Pictures typically aren’t localized, but might not connote the same thing or be
appropriate for the given locale. This is discussed in Part III.

Things to Keep in Mind for Complex Scripts in General

• When dealing with languages that consider some character combinations to be illegal,
look for a small dotted circle, which is the standard indication for a character that is
expecting another character to be there.
 • If you do not understand a language, be careful in trying to decide whether the way the
language is handled in your application is incorrect. Someone who knows that language
can be crucial here.
• Watch carefully for clipping of text both above and below the line.
• When dealing with languages such as Thai, make sure that you have someone who can
help you understand whether word breaking is happening properly.
• Arabic scripts have a natural flow of a sentence such as ،‫ والقتراحات‬،‫ والسئلة‬،‫أرسل كافة الشوائب‬
‫ والتعليقاتإلى العنوان‬،‫والشكاوى‬: (In English, this means "Send all bugs, questions, complaints, and
comments to:".) You would not expect many breaks between letters in a word (much like
cursive script in Latin text), so be on the lookout for words that seem mostly broken into
individual letters or characters.
• Make sure that someone who knows the language can verify the sorting behavior for it.
• If you are on Windows 2000 and want to verify proper keyboard behavior with a Hindi
keyboard and a Unicode control, you can try typing v+c+m+d+l+s. This will become the
word नमसते, which is the Hindi word Namaste, a common greeting. A similar test for Tamil
can be accomplished by typing b+C+k+d+k+c+d. This will become the word வணக்கம,
which is the Tamil word Vanakam, a similar greeting. If its letters or any of the shapes
look different, complex script support is not working properly.
• You can get similar words in all languages you need to support from customers or
localizers.

Where We Just Were...

This chapter discusses many of the issues that come up in testing international applications,
issues that must be carefully considered if you are expanding into global markets. It is important
for a tester to keep in mind that he or she is, in many ways, the most important person in a
company that is expanding its market. Testers are the ones who can help realistically assess the
markets that can be covered, and the success or failure in global markets will depend largely on
how good a job the testers do in making sure that the developer did a good job.

This makes a great note on which to close the book-with a product that the tester signs off on and
development is proud of, and that is embraced by customers around the world! Although the
reality can sometimes fall short, by achieving a global perspective, you can guarantee that you
will produce products that come much closer to your goal.