MoRE Phase III Final Report

MoRÉ: Mobile Reference Environment
Rapid Prototyping, Spring 2006

Phase 3 Report
May 12, 2006
Editors: Brian Ellis, Jae Choung Ha, Megan Hyland, Aashni
Shah
HCI Group
Brian Ellis
Anand Gopalkrishnan
Yong Woo Rhee
Aashni Shah
Wen Shu Tang Lu
Daniel Zinow
Search Group
Tabreez Govani
Melanie Haskell
Sachin Kulkarni
Kevin Smith
Capturing Experience Group

Jae Choung Ha
Leon Mai
Jonathan Tran
Platform Group
Mohammad
Ahmad
Megan Hyland
Zack Menegakis
Bryon Smith
Adam Wolbach
– Implementation and Integration RPCS – Spring 2006
1
Table of Contents
1 Introduction......................................................................................................... 4
1.1 Purpose ........................................................................................................... 4
1.2 Background..................................................................................................... 4
1.3 Group Members .............................................................................................. 5
1.4 Overview ........................................................................................................ 6
2 Conceptual Design............................................................................................... 7
2.1 Problem Definition.......................................................................................... 7
2.1.1 Problem Scenario .................................................................................... 7
2.1.2 Key System Requirements......................................................................10
2.2 Initial Solution Concepts ................................................................................12
2.2.1 Table of Selected Technologies ..............................................................12
2.2.2 Visionary Scenario .................................................................................12
2.3 Conceptual Design .........................................................................................13
2.3.1 Solution Scenario Selection Criteria .......................................................13
2.3.2 Product Design Specifications ................................................................14
2.3.3 Search ....................................................................................................14
2.3.4 Experience Capture ................................................................................14
2.3.5 Platform .................................................................................................15
3 System Tutorial and Usage Scenario .................................................................15

3.1 Summary of Integrated User Interactions........................................................15
3.1.1 User Interaction Issues and solution based on Input Device ....................15
3.1.2 User Interaction Issues and Solution Based on System and User .............17
3.2 Usage Scenario...............................................................................................17
4 Implementation and Integration........................................................................22

4.1 Unifying User Interaction...............................................................................22
4.1.1 User Interaction Design and Tutorial ......................................................22
4.1.2 User Studies Results or Walkthroughs ....................................................25
4.2 Search ............................................................................................................27
4.2.1 Functionality ..........................................................................................27
4.2.2 Experimental Measurements...................................................................29
4.2.3 Software Architecture.............................................................................33
4.2.4 Software Modules and Status..................................................................34
4.3 Experience Capture ........................................................................................38
4.3.1 Functionality ..........................................................................................38
4.3.2 User Studies Results ...............................................................................39
4.3.3 Experimental Measurements...................................................................40
4.3.5 Software Modules and Status..................................................................43
4.4 Platform .........................................................................................................49
4.4.1 Functionality ..........................................................................................49
4.4.2 Component Specifications and Pictures ..................................................49
4.4.3 Hardware Architecture............................................................................58
2

4.4.5 Hardware on Body..................................................................................61
4.4.6 Hardware Modules and Status ................................................................65
4.5 Conclusions....................................................................................................67
4.5.1 Summary of Key Design Issues ..............................................................67
4.5.2 Lessons Learned .....................................................................................71
5 Project Management ..........................................................................................73

5.1 Implementation and Integration Phase Results................................................73
5.1.1 Summary of Logbook Hours...................................................................73
5.2 Summary of the Entire Project........................................................................75
5.2.1 Task Dependency Chart..........................................................................75
5.2.2 Summary of Logbook Hours...................................................................75
5.3 Suggestions for Improving the Class ..............................................................78
6 Glossary of Terms ..............................................................................................79
7 Appendix.............................................................................................................80
7.1 Hardware Specification Tables .......................................................................80
7.2 Task Dependency Chart .................................................................................87
7.3 List of Figures................................................................................................88
7.4 List of Tables .................................................................................................88
3
1 Introduction
1.1 Purpose
Mechanical systems have followed two parallel trends in the last few decades: they have
become exponentially more complex, and have become increasingly common in various
and myriad unexpected environments, many of which are far less controlled and
accessible than a typical office. This reality has forced technicians who are assigned the
task of repairing these machines into a dilemma: how does one access all the information
one needs to repair a system with thousands of parts, while remaining mobile enough to
service machinery that is in an inconvenient or unconventional location?
The traditional solutions to this problem have been ad-hoc, ranging from reams of paper
manuals to (more recently) laptop computers, and all have generally involved a fervent
hope that the reference material one needs is present and can be searched through before
one dies of starvation. Clearly, a better solution is required that takes into account the
nature of mobile repair and allows technicians to find a balance between adaptability and
well-preparedness. Further, an optimal solution would itself be adaptable to other
purposes: the reasons that one might need access to information on the go are
innumerably many, and the current tools — including laptops, personal digital assistants,
and iPods — fall short. Laptops, with their full-size keyboards, delicate screens, and
often dubious portability, simply aren’t agile enough; PDAs are streamlined for input, not
output, and often lack the screen size and real estate to efficiently display large amounts
of textual and graphical information; iPods suffer from the screen size problem as well,
and lack the software and hardware extensibility necessary to cope with the informational
needs of professionals (such as an Internet connection and the ability to display data that
are not strictly textual).
The MoRÉ platform was thus developed as an alternative interface designed specifically
for individuals needing quick, ready access to large amounts of data while eschewing
traditional keyboard-and-mouse input models that would compromise portability and
efficiency.
1.2 Background
The Rapid Prototyping for Computer Systems class at Carnegie Mellon was contacted in
early 2006 by Dick Martin of Inmedius Software, Inc. to design and prototype a mobile
system that would allow United States Navy repair technicians maintaining F/A-18
Hornet fighter aircraft to more efficiently conduct their repairs. Initially, it was hoped
that the class would have access to the repair procedure and training manuals for the
aircraft themselves; due to issues of timing and material availability, however, the only
material available as of the date the class was to begin were training and reference
manuals for the IETM (Interactive Electronic Technical Manual) software system, made
by Boeing, which is itself a software environment designed to allow repair technicians
maintaining F/A-18s to more efficiently conduct their repairs.
4
This extra degree of separation from the repair process provided a unique challenge: it
was unlikely that a repair technician would need a mobile unit to access information
about the IETM, since the IETM itself could only be accessed from a stationary computer
(or, at best, a semi-mobile laptop). In designing a mobile environment for accessing the
IETM documentation, however, it was hoped that the problem could be generalized and
that the mobile unit could pave the way for the replacement of the very system it had
been created to document. This somewhat ironic arrangement has come to be referred to
as a “meta-problem”, since the immediate goal of the project was to solve a problem with
a system designed to solve a different problem, with the longer-term goal of solving the
more fundamental problem itself.
1.3 Group Members

The MoRÉ development group was composed of four major teams, plus a fifth that was
assembled from the other four during the final phases of development and testing. The
teams, their responsibilities, and their members were as follows:
Search Team
The Search team was responsible for creating and deploying a search engine and
interface that could be used effectively under the constraints imposed by the mobile
unit’s input devices, as well as organizing and indexing the data that were to be made
available through the mobile platform.
Members:
Brian Ellis
Anand Gopalkrishnan
Tabreez Govani
Melanie Haskell
Sachin Kulkarni
Kevin Smith.
Experience Capture Team

The Experience Capture team was responsible for exploring ways of leveraging the
experience of previous users of the system to improve the experience of current users,
and examining the behavior of users to tailor their experience to their particular needs.
Members:
Jae Choung Ha
Leon Mai
Aashni Shah
Wen Shu Tang
Jonathan Tran
5
Platform Team
The Platform team was responsible for determining, assembling, and manufacturing the
hardware equipment needed to realize the mobile unit, as well as providing a software
interface between the hardware devices and the software running on each device.
Members:
Mohammad Ahmad
Megan Hyland
Zack Menegakis
Bryon Smith
Adam Wolbach
Mechanical Engineer: Jules Henry
Human-Computer Interaction Team

The Human-Computer Interaction team was responsible for defining the initial project
specification and hardware and software requirements, baseline and visionary scenarios,
exploring and incorporating prior research, user interface design of software and
hardware interfaces, and the content of the final prototype demonstration.
Members:
Brian Ellis
Anand Gopalkrishnan
Yong Woo Rhee
Aashni Shah
Wen Shu Tang
Daniel Zinzow
Renderer Team
The Renderer team was responsible for creating a framework that could display the
information contained in the manuals to the user in a consistent and visually accessible
manner, and for integrating the various software components together into a cohesive
system.
Members:
Brian Ellis
Tabreez Govani
Jae Choung Ha
Melanie Haskell
Jonathan Tran
Wen Shu Tang
1.4 Overview
The MoRÉ system provides an efficient, portable means of accessing reference material
or any other dense, inter-related set of informational units. It can be used for extended
periods of time without requiring the presence of an electrical outlet, does not necessitate
a keyboard or any other means of text entry (such as handwriting recognition or an
onscreen keyboard), leaves both hands free, and has many other advantages over any
other common means of accessing such information. Its software interface is streamlined
6
to facilitate searching and leverages the inherent connections and relationships between
different pieces of data to assemble all relevant details together, resulting in a smaller
requirement of effort in order to find a particular piece of information. Most importantly,
it is eminently scalable to address the “meta-problem” of mobile information retrieval in
a professional context.
2 Conceptual Design
2.1 Problem Definition
2.1.1 Problem Scenario
To demonstrate the current manner in which aircraft maintenance tasks are performed, a
"baseline scenario" was presented following an imaginary but typical technician through
the steps of using the present-day Automated Maintenance Environment to perform a
simple maintenance procedure.
Consider the example of a maintainer named Mark. Mark is a rigger, a specific kind of
technician whose responsibilities include entering observed problems with a plane into
the Automated Maintenance Environment's Debrief system. Today, Mark is waiting out
on the tarmac for an F/A-18 to land. Once it does, he will remove a small card from the
nose of the plane and carry it back to the operations building, where he will place it into
the Data Acquisition System (DAS). The DAS unit will generate a report detailing all the
problems the card detected while it was inside the plane. Each of these problems is
referred to as a "BIT discrepancy".
Once the DAS report has been generated, Mark switches over to the Debrief system to
examine the discrepancies and file work orders for the problems revealed in the report.
He goes through the steps of loading the data into the Debrief system and finds himself
on the Discrepancy Review screen (Figure 1). Mark is fairly new to his job, and his
training process, though thorough, was a great deal of information to process all at once,
and Mark soon finds himself confused. In particular, he finds the label for a pop-up menu
in the lower right-hand corner of the screen – "Verified/Observed" – entirely inscrutable.
Mark spies a binder on the desk claiming to be a reference manual, and starts flipping
through it, hoping to find an answer. Many pages later, he finally arrives at the right
chapter and section and sees a picture of the Discrepancy Review screen (Figure 2). To
his dismay, however, he finds that the label for that pop-up menu is not the same in the
reference manual as it is on his screen. The reference manual calls it "Create Work
Order". Needless to say, this does little to alleviate his confusion, so he decides to consult
the training manual – a document with which he is already familiar with from his
orientation instead. He brings up the training manual, which is an interactive Web site, on
his computer screen and begins clicking his way through the various chapters, pages,
demonstrations, quizzes, and other content. He can only move one page or section at a
time due to the design of the manual, which was intended to act as a tutorial rather than a
7
reference, so getting to the right chapter, section, and subsection takes some time and
perseverance. Eventually, he comes across the picture he is looking for and reads the
description for the Verified/Observed label (Figure 3).
The training manual (Figure 3) seems to imply that the pop-up menu does indeed create a
work order, but after the inaccuracy of the reference manual, Mark is no longer so sure he
can trust the material to be up-to-date. As a last resort, Mark gets up from his desk and
hunts down the resident expert on Debrief, a technician named Ray who has been on the
job for many years. Mark describes the problem to Ray, who responds almost
immediately, "Oh, yeah, they changed that text last year. It does the same thing though.
Just always set that to 'YES' unless the BIT code is wrong." Mark thanks Ray for the help
and goes back to his desk, realizing all at once that his confusion, and the failure of the
manuals to relieve it, has cost him 35 minutes he could otherwise have been using to do
his job.
Figure 1: Discrepancy Review Screen in Debrief System
8
Figure 2: Reference Manual Explanation
9
Figure 3: Training Manual Explanation
2.1.2 Key System Requirements

To fulfill the needs to meet the design, some basic functional requirements will be
presented to appropriately address the problem at hand. For example, we have seen that
any system containing small parts that could fall into a plane or get lost are inherently
dangerous, and therefore infeasible. As a corollary to this rule, the system must also not
protrude from the mechanic's body, lest some component get caught on a piece of
equipment. Being tethered to a jet fighter as it prepares for take-off is not a pleasant
experience by any stretch. Furthermore, one must also consider the harsh conditions in
which mechanics often find themselves working. Any system designed for those
conditions must therefore be rugged enough to withstand both high and low temperatures
and rough environments, including blowing sand and rain.
Similarly, a mechanic may be called upon to work under the glare of the desert sun or in
a dimly lit hangar within minutes of each other, so the system must adapt to various
lighting conditions without interfering with usability. In the interest of unobtrusive
portability, any large, flat screens or keyboards – even those that might attach to the body
or limbs of the mechanic cannot be used, especially since they may very well get
damaged. A keyboard as input mechanism would be infeasible as mechanics often times
use heavy work gloves. Having to remove one's gloves to interact with the system would
place an unjustifiable burden on the mechanic to find a place to put them. On the other
hand, any input or output devices incorporating an auditory component as an alternative
must take into account the high ambient noise conditions usually present near jet aircraft,
and function satisfactorily with ambient noise levels up to 85 dB or higher.
10
The software platform for an integrated interactive manual must also satisfy certain
functional requirements. Since a technician cannot be expected to type or enter search
terms every time they wish to see more information on a topic that is detailed in the page
they are currently looking at, the software must support dynamically generated links
between various types of content. For example, a page in the reference manual that
mentions a certain acronym may link to a page in the glossary that explains the meaning
of that acronym, and several pages in the training manual each of which also mention that
acronym. The manuals must also be clearly labeled for easy referencing for the
technicians. In other words, the user must be able to easily distinguish which of the two
manuals he or she is currently reading from.
For those cases where a technician needs to know about a concept or component that is
not reflected on the page he or she is currently perusing, the system must also support
some kind of open-ended search capabilities whereby a technician can enter certain
keywords to find relevant content. The software must also capture the experience of the
users in order to help future users to have an easier time finding what they need. In a
sense, the software must recognize the browsing experience of past users and give leads
to the direct results (what they were trying to find) to future users. This would facilitate
the browsing experience of the users. Lastly, in order to take advantage of shared
experience and expert practices, the system must enable users to leave annotations on
particular pages that expand or clarify the manual content there.
The physical system as a whole must also conform to other requirements, some of which
are not as obvious from the visionary scenario but all of which are important to the
satisfaction of the user. The entire system must be portable, including the display; this
requirement constitutes the primary endorsement for a head-mounted display. The typical
shift length of an aircraft technician is eight hours, with a half-hour lunch break halfway
through. Since the technician cannot necessarily drop whatever he or she is doing and
return to the hangar to recharge the unit, the battery life of the system must be at least
four hours. This assumes "hot-swapping" capabilities, whereby depleted batteries may be
changed out for charged ones without the device losing its state. If such capabilities are
not feasible, the minimum battery life requirement becomes eight hours. Lastly, but still
importantly, the device must be usable by everyone who might have a need for it. Just as
it must be operable with heavy work gloves from inside the cockpit of a plane, so too
must it be usable from a desk equipped with a keyboard and external monitor.
Maximizing the usability of the device will also maximize its utility.
11
2.2 Initial Solution Concepts

2.2.1 Table of Selected Technologies
Below is a list of selected technologies used in the final implementation of the design.
The following sections address user scenarios for use of the design, incorporating each of
these features, as well as more in-depth descriptions of the requirements, helping to
explain why these technologies were chosen for the final prototype.
Table 1: Selected Technologies

System Requirement Selected Technology
Mobile Computer Sony Vaio-U8G
User Input Device Custom Dial Device
Earphones David Clark H10-13.4
Headphones David Clark H10-13.4
USB Connections Targus USB 2.0 Mini Hub
Head Mounted Display Liteye HMD
Audio Connector Sound Professionals USB Audio Converter
Browser Custom Browser based on Firefox
2.2.2 Visionary Scenario

The initial visionary scenario from Phase I follows an imaginary but typical technician
through a routine day of using the manual system. The scenario gives enough detail to
show how it must work to provide the technicians with what they need, but not so much
detail as to explain every nuance of the system.
Our Technician, Mark, is standing aside while an F/A-18 comes down on the runway.
After the pilot gets out, he does his initial routine tasks and finds a pen lying in the floor
of the plane during his inspection. Knowing that foreign objects in the plane can cause
damage to sensitive systems, he enters this into the Debrief system as he heads back to
the hangar. While heading back, though, a sand storm starts up. Gladly, Mark's
equipment is ruggedized against the elements, but the same cannot be said of the $60
million aircraft. Worried that the blowing sand might cause damage to the plane while
Mark has the canopy open, he decides to move it into the hangar. As he brings the system
into the plane, it adjusts to the artificial lighting.
Now Mark can begin doing the debriefing on the plane. Unfortunately he is unsure of
how to enter the discrepancies he noticed and thus consults the reference manual. The
reference manual's explanation of entering non-BIT discrepancies is rather confusing, so
Mark decides to compare it with the training manual by following a dynamic link. The
training manual's detailed explanation quickly clears up his confusion, and to help any
novice in the future, Mark adds an audio annotation to the reference manual. This
annotation appears on the page as a selectable element.
12
Having finished doing the debrief Mark moves on to repairing the plane by bringing up
the list of repair tasks. Mark finds the correct repair procedure and starts going through
the steps. Now however, he needs to keep a constant watch on the hundreds of controls in
the cockpit so as not to accidentally change anything. Mark is very glad that he does not
have to removes his gloves to flip any pages. After the repair is done he powers up
several systems of the aircraft to test the changes made. Doing so causes a loud whine
that fills the hangar, but Mark is still able to continue working with no problem. He is
able to issue commands and receive feedback from his repair system without any
hindrance.
2.3 Conceptual Design

2.3.1 Solution Scenario Selection Criteria
Several of the functional requirements for the prototype hardware are derived almost
directly from the visionary scenarios. For example, we have seen that any system
containing small parts that could fall into a plane or get lost are inherently dangerous, and
therefore infeasible. As a corollary to this rule, the system must also not protrude from
the mechanic's body, lest some component get caught on a piece of equipment. Being
tethered to a jet fighter as it prepares for take-off is not a pleasant experience by any
stretch. The visionary scenario also illustrates the harsh conditions in which mechanics
often find themselves working. Any system designed for those conditions must therefore
be rugged enough to withstand both high and low temperatures and rough environments,
including blowing sand and rain.
In the interest of unobtrusive portability, any large, flat screens or keyboards – even those
that might attach to the body or limbs of the mechanic -- cannot be used, especially since
they may very well get damaged. A keyboard is also not usable with heavy work gloves
on, and having to remove one's gloves to interact with the system would place an
unjustifiable burden on the mechanic to find a place to put them. So we decided to add a
scrollable device to be attached to the chest of the technician for input and also use a high
intensity microphone and headphone for the purpose of audio input.
The software platform for an integrated interactive manual must also satisfy certain
functional requirements. Since a technician cannot be expected to type or enter search
terms every time they wish to see more information on a topic that is detailed in the page
they are currently looking at, the software is made to support dynamically generated links
between various types of content. Lastly, in order to take advantage of shared experience
and expert practices, the system must enable users to leave annotations on particular
pages that expand or clarify the manual content there. The annotations must be easily
accessible within the same page and while the content is being displayed. Furthermore,
the annotation facilitates collaboration within the aircraft maintenance workers to
facilitate. In other words; more experienced maintenance workers can share their
experience with others through annotations. Thus, a single pane was designed to support
audio annotations and also have it updated in the search links for other users to hear and
learn.
13
The typical shift length of an aircraft technician is eight hours. Since the technician
cannot necessarily drop whatever he or she is doing and return to the hangar to recharge
the unit, the battery life of the system must be at least four hours. Thus, our system is
intended to address the worst case scenario of when a worker is out on the field for the
entire duration of his or her shift. This assumes "hot-swapping" capabilities, whereby
depleted batteries may be changed out for charged ones without the device losing its
state. So to counter this problem a unique swap controller was designed to alternate
between three different set of battery packs.
2.3.2 Product Design Specifications

Each of the four major design groups helped to research and design a module of the
overall system prior to integration. Design choices and specifications for the final
integrated product are addressed in depth in Section 4.
2.3.3 Search
The Search group was responsible for mappings between the procedure and the training
manual and to ensure that searching through the manuals is easy and straightforward.
Depending on the input from experts and learning from users’ responses and usage
patterns (data provided by capturing experience), specialized links and annotations were
incorporated into individual pages. As such, the search group was responsible for
integrating all these components and presenting them in a unified view to the users. This
included retrieving relevant information from different functionalities and formatting
both manuals and finally rendering them into our custom browser to be presented to the
user. In addition, the limited input capabilities of the handheld means that we had to tailor
our methods accordingly. Thus, the scope of search group was pretty wide, including
interacting with the Hardware/Platform Group for the specific input interfaces, with the
Capturing Experience Group for experts’ data, annotations etc and with the HCI Group to
ensure that the users are able to effectively use the capabilities provided by our group.
2.3.4 Experience Capture

Based on the visionary scenarios presented in Phase I and Phase II, ideas were formed
about how to capture a user’s experience.
First, it was felt that knowledge from experts would be greatly helpful to a user who has a
question. The user may be a novice, or perhaps an experienced technician that has
switched to an unfamiliar area of maintenance, etc. Though this, the Expert Contact
Information module was formed. This component will make available enough
information to contact the expert, such as title, years of service, and contact information.
Another idea from the visionary scenario was that if a user was lost and confused in
dealing with the manual for a long time, then finally figures out the solution, they should
be able to provide their experience to other users. In this way they can help save their
colleagues time. To solve this, audio annotations will be available for users. Through
these, a user can add their own input to the manual, visible as links to files, so that other
14
users can learn and save time using them. The general category of task recognition was
proposed from the beginning. However, since robust task recognition is very difficult, a
simplified model is used for this prototype. The Cycling Detection module does task
recognition by identifying when a user is cycling. It is assumed that if a user is cycling
they are probably lost, so in a situation where they are cycling, the page renderer will
highlight help in the side pane.
It was noticed early on in the project that navigating through the manuals was not easy;
they were large and often confusing. Smart Links shorten the time it takes a user to find
the information they are looking for. Links will be generated either through user’s
previous usage experience or through indexed search results to facilitate browsing
experience. Finally, to enable many of these features, there is a User Login page to
appropriately record information from the user to enable some the features from capture
experience and help user in search for the content he or she is looking for.
Further detail on all these modules is provided in the Functionality section.
2.3.5 Platform
Based upon the Visionary Scenario, the Platform team was responsible for designing and
obtaining the necessary hardware to create a mobile computing unit and to enable
peripherals capable of collecting user input and delivering outputs such as display, and
speech annotations. With these requirements in mind, the Platform team identified
different technologies and specifications for the design, which was determined to include
a mobile computer, dial input device, headphones, microphone, and head mounted
display unit.
In order to implement the hardware architecture and to interact with the Software
development groups, a new dial device was designed to accommodate all of the new
searching features. A mobile computer running on the Windows XP platform made the
integration with the Software teams smooth. In addition, to meet the minimum system
time requirements, and to maximize user use, the group decided to implement a hot-swap
controller, enabling a user to utilize the runtime of several attached batteries without
having to physically switch the battery in the system. This increased time will add to
usability of the system, and add to the simplicity of use. Furthermore, all components can
be attached to a wearable suit composed of belts and holsters and will not interfere with
the mobility of the user.
3 System Tutorial and Usage Scenario

3.1 Summary of Integrated User Interactions
3.1.1 User Interaction Issues and solution based on Input Device
By far the most challenging UI issue in a mobile reference environment is the lack of a
character-based text input device. Much of the conventional wisdom about navigating
15
dynamic content assumes the existence of a keyboard or similar technology; if the

choices are too many to enumerate, the obvious fallback measure is to allow (some might
say “force”) the user to enter search terms directly. While this approach does enjoy a
certain ease of implementation, most modern attempts at information organization and
retrieval at least supplement text-based entry with some kind of relevance engine — it
could be that the user does not know precisely what he or she is looking for, and the
system could suggest pieces of information that have historically been relevant to the
concept at hand. Suggesting likely next steps does more than just save keystrokes; it can
demonstrably improve the quality of results and increase user satisfaction.
That being as it may, however, dropping support for text entry altogether is a very
dangerous move. Content, especially on the Internet, is typically presented as a graph of
associations between units of information, and text-based search provides the ability to
teleport to any node in that graph immediately. Remove that functionality and what
remains is traversal, which may be appropriate for small maps but becomes a complex
proposition for larger, less well-organized systems. The potential problems are manifold:
for one, traversal assumes the existence of a starting point, and the only nodes available
from that starting point are those directly adjacent to it. This is almost guaranteed to be a
small subset of all available nodes, and worse yet; it will probably be a set of closely
related nodes that have relatively few external connections. In the worst case, this leads to
another problem with traversal: the possibility of nodes or entire sub graphs that are
entirely disconnected from the rest of the system. Put more plainly, if nothing links
directly to a page, there is no way to access that page without text entry. We can see,
then, why removing text-based search is a risky move: it takes away our safety net and
requires us to be very careful with our relevance algorithms. If we systematically miss a
relevant result, we effectively ensure that that page will never be viewed unless it is
directly linked to by another relevant page. In the case of the manuals hat are viewed on
the mobile units, these direct links are few and far between at best, and entirely absent at
worst (as in the reference manual).
We can mitigate this effect by such tactics as setting the starting point of the traversal to a
“table of contents” page that actively links to every possible page, but this solution can be
high-maintenance and may constitute a special case; it is therefore essential that search
results be either exhaustive which can itself be problematic from a user interface point of
view, as no one wants to sort through a hundred or more search results), or highly
relevant at all times.
With such high stakes on relevant search results, it would be ill-advised to design a
system without some means of detecting a breakdown in quality and compensating, either
by showing more results that had not yet been visited or by broadening the scope of the
search to include pages the user might not have realized were important. The experience
capture subsystem is a step in this direction, and while a robust and adaptive mechanism
for tuning search results based on past history could very easily be a project in and of
itself, it is hoped that features such as cycle detection will help to make up for the lack of
text-based input.
16
3.1.2 User Interaction Issues and Solution Based on System and User
An important design requirement of the MoRÉ system is to allow the user maximal
control over his or her information context. Particularly, the system must avoid whisking
a hapless user away to parts unknown while his or her original question gets buried under
a ponderous browser history. Even mainstream browsers have dealt with this problem in
their own time; the solution was, at first, multiple windows, and then later multiple tabs,
but even before this browser took care not to change context without a good reason.
Imagine if the only way to go to a Web site was to click a link somewhere on the current
page that would display a new page, blank except for a text field in the middle, in which
you could enter the address of a new Web site to visit. While the example may seem silly,
in a system with a limited range of possible user gestures this approach might easily
become a design reality.
To prevent this, MoRÉ borrows a page from mainstream browsers and displays a sidebar
in which search results can be displayed. This encourages the user to click terms about
which they may be unfamiliar or confused: in many cases, the titles of the search results
may themselves serve to alleviate the user’s uncertainty, and in any event there is no
obligation to click on one: investigating a search result does not change the information
context. Similarly, the user may always choose to preserve their previous context as well
as entering the new one by opening the link in a new tab; this mimics the real-world
operation of opening two manuals side-by-side.
The use of multiple panes also permits preservation of context by providing a mechanism
whereby the user can switch their focus to another goal without losing their progress on
the first. For example, a user could scroll halfway down a lengthy page, click on a
keyword link to bring up search results in the search pane, then switch to the search pane
and open one of the results in a new tab. The user could then simply switch back to the
original content pane and continue reading the document, secure in the knowledge that
their search result page will be waiting for them once they are finished. The content view
would remain in its original state, preserving such properties as position on the page,
highlighted keywords, etc.
The next section talks about how the user would interact with the dial and the system.
3.2 Usage Scenario

Once the user has logs in and arrives at the Main page of the browser there are multiple
functionalities he could perform using the dial. Below is a short description of these
functionalities of the dial and its corresponding effect on the browser:
1. Scrolling Between Links:
You can scroll between the different smart links by just turning the rotating piece on the
dial. Turning the dial clockwise will scroll down the page while anti-clockwise will scroll
17
up the page. This will allow you to swap between different smart links on the particular
page with ease.
Dial Usage:
Desired Effect:
2. Clicking on a Smart-Links:
You can click on a smart link by just pressing on the center piece. This clicking can help
you bring up the search results on the right pane or can take you to a different page itself
entirely when clicking on a search result. In case of annotations it would bring up the
player to hear the voice annotations left behind.
Dial Usage:
18
Desired Effect: (clicking on the smart link “Work center” brings up results on the search
pane)
3. Switching Between Panes:
You can use the pane switch button to swap between different panes in the browser
window. This gives you and option to scroll through links in a different pane of the
browser.
Dial Usage:
Desired Effect: (Clicking on the pane switch button twice moves the highlight from
“Work center” in the main window to the first results in the search pane)
19
4. Bringing Up Links in New Tab:
By holding down both the left and right tab button the user can bring a desired search
result in a new tab entirely. Both the tab button should be held down simultaneously on
the desired search result.
Dial Usage:
Desired Effect: (holding down and releasing both the tab buttons on the first search
result of the search pane brings a new tab with the search result)
5. Switching between the Tabs:
The dial gives you the option of switching between the tabs using the left and right tab
buttons. The right tab lets you tab to the right, while the left allows you to tab to left.
This allows you to switch between multiple tabs thereby providing quick easy reference
to certain content.
Dial Usage:
20
Desired Effect: (holding either of the left or right lets you switch tabs)
6. Using the Mouse cursor (optional):
The dial gives you and option to move the mouse pointer or cursor of windows to either
point to the desired location or move the cursor to close the browser. This feature of the
dial can be used by move the cursor either in a 2-dimentional plane similar to the use of a
joystick.
Dial Usage:
7. Using the windows left Mouse Click (optional):
After the mouse cursor has been used to move the cursor to a desired location you can
use the windows mouse left-click option by holding down and releasing the pane switch
and select button. The click happens only upon simultaneous release of both the buttons.
Dial Usage:
21
4 Implementation and Integration

4.1 Unifying User Interaction
4.1.1 User Interaction Design and Tutorial
The user will use the dial as the main input device to interact with the user interface of
the manual integration software. The software provides four panes in which contents of
the manuals, options for the screen, results of user’s search queries, and annotation
recording interface are presented. The dial’s components include a wheel, a select button,
a pane switch button, two tabbing buttons, and a pointer knob. Each component of the
user interface and the dial are functionally connected to support the user’s process to
achieve his/her goal.
4.1.1.1 User Interface
Figure 4: User Interface
The main pane in Figure 4 displays the contents of a manual that the user had requested
for. Each word of the manual is a link that displays the related materials in the search
results pane upon selection. However, some of the common words such as the, and, and
of are not highlighted as a link since they do not represent or mean a specific topic related
to a maintenance task. This pane provides the user the capability of scrolling as the
contents may exceed a length that cannot be displayed at once on the screen.
The next pane in Figure 4, positioned on the top right section of the screen, provides two
options: the option for closing the current window and the other option for marking
22
whether the problem was solved. When multiple tabs had been created, the user can close
the current tab by clicking on the “Close this window” option. In the case of which only
one tab is present, this option will be grayed out and won’t be accessible to the user. This
pane is also used to indicate whether the material (contents of the manuals or expert’s
annotation) solved the user’s problem. The user can select this link if his/her question
was solved by the materials provided by the interface in result of his/her search query.
The large pane on the right section of the window in Figure 4 is reserved for displaying
the search results, definitions of key words, and smart links related to the topic displayed
in the main pane. To prevent cluttered information in this pane, not all search results are
displayed, but only the most relevant links are provided by the interface. The interface
provides the number of results returned, the origin of the link (reference or training
manual), and the title of the link. The definitions section mainly serves to provide what
the abbreviations mean, such as IETM, which refers to Interactive Electronic Training
Manual. Following section displays the smart links, which are the links to topics that are
related to the page displayed in the left. If an expert have had left an audio annotation for
the topic, it is also displayed in this pane. The information of the annotation including the
time it was recorded, the name of person who left the annotation, and the length of the
audio are also provided here. The user can select an annotation to refer to an expert’s
knowledge on the topic to solve a problem.
The bottom right pane in Figure 4 provides an interface for recording an audio annotation
relating to the topic. When a user desires to leave his/her knowledge that may be useful in
helping other users to solve their problems, he/she can select the record button and record
an audio message. Upon selection, the interface will display how long the it has been
recording and allow the user to stop recording by once again selecting the same button,
which now displays as a stop button.
4.1.1.2 The Dial
23
Figure 5: Dial Prototype
The most dominant feature of the dial, seen in Figure 5, is the wheel. By turning the
wheel clockwise/counter-clockwise when worn, the user can move the selection box to
the next/previous link accordingly. The wheel scrolls up/down the page when the
selection box reached the top/bottom of the displayed screen but not the actual
top/bottom of the pane. When the box reaches the top/bottom link on the pane,
continuously turning the wheel will move the selection box back to the bottom/top link of
the pane.
The inner area of the ring of the wheel (Figure 5) functions acts as the select button. After
moving the selection box to a desired link by turning the wheel, the user can select the
link by pressing any section of the area. This button will also allow the user to click on
different buttons positioned in the interface.
The center knob within the select button (Figure 5) has the capability to move the cursor
on the screen in eight-direction movement. However, this knob is not to be used as a
primary controller for the interface.
The large button placed on the upper section of the dial (Figure 5), is the pane switch
button. By pressing this button, the user can move the selection box to the next pane in
the current tab. For example, clicking this button while the selection button is in the
search results pane will move the selection box to the record annotation pane.
The two tab buttons on each side of the dial (Figure 5) have two functions. Pressing the
left/right tab button when worn by the user will open the tab accordingly. Second
functionality of the tab buttons is that when they are pressed (squeezed) together
simultaneously, a new tab is created with the materials related to the selected link.
24
4.1.2 User Studies Results or Walkthroughs

The following contains a scenario that walks through how the dial and computer interface
interact with each other and how this system is used by the user. The user in this example
is Mark, a maintenance worker at an airfield. He goes out into the field to fix the
problems with the airplanes. When done with his work, Mark returns to the office to
download all of the information on his device to the computer.
Mark has just finished a day’s work at the airfield. As he heads back to the office, he
wants to upload all of the procedures that were completed, part information that was
collected, and bookmarks made.
To follow the steps of doing this correctly, Mark goes to the Uploading section of the
reference manual. Mark scrolls through this document, which he does by rotating the
wheel of the dial. Rotating the wheel causes a highlight on the screen to move from link
to link on the screen. Everything is making sense until Mark reads “Uploads should only
be performed when running Work Center on a workstation.” Mark doesn’t really work
with the workstation too much, so he is kind of confused about this “Work Center.” To
find out more, Mark rotates the wheel until the highlight is on “Work Center.” To do a
search on Work Center, Mark presses down on the center button that is located within the
wheel. The search results appear in the pane on the right side that is just below the
“Solved Problem” pane. To get there Mark clicks down on the pane button with his finger
to move the highlight from the main pane to the “Solve Problem” pane and finally to the
“Search Results” pane. In the search results, Mark sees a link to the Work Center part of
the reference manual. He decides that this result looks like it would be helpful and he
rotates to the wheel until “[reference] Work Center” is highlighted. Thinking that he
might want to compare the information about Work Center from the reference manual
with the current screen, as there are more mentions of this Work Center on this page,
Mark squeezes both tab buttons into the dial. This action causes the reference section on
Work Center to appear in a new tab.
After scrolling a bit through the page, Mark now understands that the Work Center is the
software on the workstations (desktop computers) that is what grabs the data from his
wearable computer and transfers it to the workstation. To go back to the page on
Uploading, Mark presses in on the left tab button which brings him back to the first page.
He reads further along the page and realizes that there are a lot of mentions of how the
Work Center needs to be running for many things to happen. Mark is not sure how to
know if the Work Center is running and he definitely does not want to attempt to upload
just to fail. With this confusion in mind, Mark clicks on the right tab button to move back
to the Work Center page. Reading further along on this page, Mark finds out that to tell if
the Work Center is running he just needs to look at the desktop of the workstation and
look to see if the Work Center software is open. If it is open, then it is running.
Feeling confident he now understands this whole Work Center thing, Mark clicks on the
pane button to move to the “Close Window/Solved Problem” pane. He rotates the wheel
until “Solved Problem” is highlighted. Mark then presses on the center button located in
the wheel to indicate that this current page solved his problem. After he clicks on the
25
button, a red check mark appears next to “Solved Problem”. Mark then continues to
rotate the dial until “Closed Window” is highlighted. With this highlighted, Mark presses
the center button within the wheel to close this tab. Now Mark is left with the page about
Uploading and with no tabs in sight and no “Close Window” button (since the first page
cannot close at all).
Not wanting anybody else to have to go through the same struggle as he did, Mark clicks
on the pane button to move to the “Annotation” pane. With the highlight on top of the
record button, Mark presses the wheel down and the record button turns to a green stop
button and the length of the recording also appears
Mark goes ahead and explains what a Work Center is and how to know when it is
running. When he is finished, Mark presses down on the dial to stop the recording. A
note that an annotation was made by Mark on April 2, 2005, with a length of 2 minutes
appears underneath the search results and smart links.
Mark finishes reading about the uploading process and then actually performs the
process. With that done, Mark moves onto the next section “Downloading Portable
Computer to Work Station.”
This section describes how to download any annotations made and problems solved. To
do this, though, Mark needs to attach the portable computer to a dock. This involves
taking the portable computer out of the protective case attached to his leg, then open up
the computer connector, and finally to place this computer in a certain orientation into the
dock. Mark knows how to take the portable computer out of the case attached to his leg
but everything afterwards he has no clue. To figure out more about the portable computer
and how to attach it to this dock, Mark rotates the wheel so the word “Portable
Computer” is highlighted at which point he clicks down on the center button within the
wheel to bring up search results for this concept. There are so many search results so
Mark takes a look at the Smart Links and sees “Portable Computer and Dock Station.”
Mark rotates the dial to highlight this, and presses down on the dial to bring this
information on the screen. This page is very helpful, showing and explaining the parts of
the computer.
Even with the visuals and explanations, Mark is still confused as to how to open the
connector and he does not want to damage the controller so he decides to look for more
help. He sees that there is an annotation, so he clicks down on the pane button to the
“Search Results” pane, rotates the dial to the play button and clicks down on the wheel.
The annotation plays and informs Mark that the connector is covered by a cover that just
needs a hard push to open and that the computer is durable enough not to be damaged.
Mark feels confident that he can open this connector without any problems now.
With his problem solved, Mark clicks on the pane button until the highlight is in the
“Close Window/Solved Problem” pane. He then rotates the wheel until “Solved Problem”
is highlighted at which time he clicks on the center button within the wheel to indicate
26
that the current page solved his problem. A red check mark appears by “Solved Problem”
also. To go back to the “Downloading Portable Computer to Work Station” page, Mark
rotates the wheel to highlight “Close Window” and presses down on the dial to close the
current page. Mark is now back to the “Downloading Portable Computer to Work
Station.” From there he continues to read the instructions and follow them, successfully
downloading all of the annotations and solved problems to the Work Station.
4.2 Search
4.2.1 Functionality
Figure 6: MoRE Web Browser Interface
The search subsystem provides the user with their sole means of navigating through the
training and reference material. This reliance on the search mechanism is due to the lack
of a keyboard or other text-entry interface on the mobile unit; the user may only select
from the links that are presented to him or her at any time, and these links must therefore
always permit the user to access the largest possible subset of the manuals without
causing information overload. In general, this is accomplished by intelligently deciding
what is relevant to a particular page through the use of smart links, as well as providing a
list of topics relevant to specific important terms, called keywords, on the page.
When a user wishes to learn more about a topic related to the page he or she is currently
viewing in the web browser, he or she can click on a smart link to be taken to a page
related to the current page in some way, or can click on a keyword to see a list of search
results for that keyword in another pane. Because clicking on a smart link causes the
user’s primary focus to change to another page, it is therefore most appropriate for “near-
misses”, cases where the user has found a page related to the answer they are seeking but
27
not the exact page they wanted. Clicking on keywords, by contrast, is non-destructive to
the context: that is, the user stays on the same page and simply sees a list of pages related
to the keyword that was clicked.
The search subsystem also provides access to annotations, editions or additions to a page
that are input from the mobile unit itself (usually by way of an audio input, producing a
sound file that can be played back by others). These annotations are stored and
associated with their respective pages by the search framework.
Care has been taken to ensure that the search subsystem is as adaptable as possible.
Although there is currently no way of performing free-text searches (again due to the lack
of a keyboard or other text input device), the search framework provides low-level
functionality to permit such searches. If future versions of the reference environment
support speech recognition or some novel form of character-based input, supporting such
input should be straightforward and will not require the content of the manuals to be re-
indexed.
Figure 7: Search Functionality
The search subsystem is comprised of several components, which can be seen in Error!
Reference source not found.. The browser, as depicted in Figure 6 and explained
28
above, acts as the user’s interface to the MoRÉ system, and in addition to the browser,
there are a number of other components, transparent to users, which provide the
functionality previously described.
As can be seen from Figure 7, user requests must pass through a number of components
before presenting the user with the requested information. Every request must pass
through the web server and then the controller, which passes the request to the experience
capture logic unit before passing the request to the renderer. The request may be for a
new manual page, new search results, new smart links, or a new browser tab among other
things, and the renderer must generate the necessary response as determined by Error!
Reference source not found.. After receiving the updated view, the web server can
display the results to the user’s browser.
4.2.2 Experimental Measurements

For the search subsystem, the first experimental method consists of gathering data about
the manuals and using the data to design and run the next set of tests. The Indexer and
Search Engine are the components involved in this refinement process. Tests are
performed with only the training manual since the training manual provided is complete
and the procedures manual provided only includes a few sparse examples. The results of
the tests performed are used as if they are manual independent due to the belief that the
general content formatting and sentence structure is similar.
The second method of testing is performed by looking at how browsers currently

implement different functionality to see how much the team’s implementation can
leverage and customize current implementations.
4.2.2.1 Training Manual Statistics Provided by Indexer
The Indexer successfully captured a multitude of information regarding the training

manual. It cataloged information about all 1,176 training manuals pages provided. Such
information about the pages included page length and page path. The primary task of the
Indexer was to store information about the words on a page. A summary of the training
manual statistics can be found in Table 2.
Table 2: Test Results for Indexer Tests

Test Result
Number of Pages 1,362
Number of Instances of Words 96,614
Number of Unique Words 3,868
Occurrences of “the” 4,965
Occurrences of “maintenance” 1,279
Top 5 Words With Most Instances the, to, of,
maintenance, and
Number of Words that Only Appear Once 884
Number of Words that Only Appear On One Page 947
29
Number of Words that Appear on 10 Pages or Less 2,578

Percentage of Strong Words 9,5%
Percentage of Javascript Words 21.6%
Time to Index Entire Training Manual ~5 minutes
locally
~15 minutes
remotely
Number of Keywords 612
Approximately 96,614 instances of words were added to the index table in our database
and about 3,868 unique words. The types of information collected about each instance of
a word include the word, the page it was found on, the offset into the page, whether or
not it was found bolded, in a title, or strong, and whether or not it was found in javascript.
The word “the” was the most common with 4,965 instances, and the word “maintenance”
came in fourth with 1,279 instances, appearing on 386 pages. About 884 words only
appear once ever, 947 words only appear on one page, and 2,578 different words appear
on no more than 10 pages.
The first thing that these results indicate is that the amount of data that we are dealing
with is large enough that we have to be careful with what we store and how we store it
because the database will live on our mobile device. In our database, we use page_ids as
references instead of the path to the page, and the field sizes aren’t extraneous.
Additionally, we put extra time in creating intelligent schemas to avoid redundant
information while still keeping performance up.
Also, what this data indicates is that the distribution of words in the training manual is
extremely exponential, with a few words appearing ubiquitously and the majority of
words appearing on very few pages. Figure 8 shows this trend more specifically. The
distribution of the words helped in selecting thresholds for word importance, page
importance, and how relevant certain words are in determining the content of the page.
These concepts together form the basis of selecting keywords to create search links on
within each page. Keyword generation tests are addressed in the following section.
30
Frequency vs. Number of Words with Frequency
Amount of words with given frequency in

1800
1600
1400
training manual
1200
1000
800
600
400
200
0
0 1000 2000 3000 4000 5000
Frequency of words
Figure 8: Graph of Frequency of Words to Number of

Words with Given Frequency
The last section of Indexer statistics gathering was to see if previous hypotheses about the
importance of formatting on a page would prove to be a strong classifier of the
importance of a word on a page. Nearly 9.5% of words contained a bold or title markup,
and 21.6% were found in javascript. Since only a fraction of the words are bold or in a
title, these words should be weighted more heavily. Words appearing in javascript are
usually associated with specific instructions, so they should be given priority if the user is
searching for instructions. But since about one fifth of the training manual’s words appear
in javascript, the additional weighting should be only slight because the javascript
condition is not too unique.
The data that was gathered from the Indexer helped the search team to adjust the Search
Engine ranking parameters in hopes of providing more relevant results to the user.
Additionally, the data has helped the queries performed by the Search Engine to focus on
performance of search by taking advantage of the way the data is structured. Actual tests
on the Search Engine relevancy were not done since no user testing took place and the
result pages returned are difficult to analyze in their raw form. The next steps of Search
Engine and Indexer tests would include actual relevancy tests.
4.2.2.2 Keywords Results Provided by Indexer
The keywords’ testing that was performed provided an evolution of the idea of how
keywords should be captured. Given that the central requirement of a keyword was a
word that alone would be a useful search term for the user, the idea of how to find a
useful word is what these tests sought to accomplish.
The first idea of how to find a keyword was to use all words that appeared on only a
small fraction of documents (less than 11 pages). If the word didn’t appear too often, then
maybe a search on that word would be useful. This algorithm was implemented and
tested. Almost two-thirds of the different words in the training manual were on no more
than 10 pages. These words account for almost 9.7% of the total instances of all words. If
all these keywords were highlighted, than about one-tenth of all words would be
31
highlighted within a page on average. This may be a good set of words that are useful to
search on for the user.
The words that were added to the keywords table using the above criteria were looked at.
Unfortunately, words like “very,” “who,” and “regardless” appear in the list of keywords
along with more technical terms. While the user could search on these types of common
English words and get a small set of results which might seem relevant, these words
hardly describe the content they are found in and wouldn’t be suitable words to link the
content among pages. Therefore, it was decided that these types of words should be
eliminated.
The next test we ran was an algorithm to remove common English words from the set of
keywords to further refine the set. Comparing frequencies of words in the manuals to
words found in more typical English text, the test was able to select words that appeared
significantly more frequently in the manuals than they did in the English text. These
words differentiate the manual from typical English text, and thus keywords should be
contained in this set. This optimization resulted in a keyword list half the original size
(about one-twentieth of the total instances of all words) and upon inspection, the words in
the keyword list seemed fairly descriptive to the content they might be found in. Besides
further tweaking of thresholds, the basic algorithm of selecting a keyword was chosen as
a result of these tests.
4.2.2.3 Browser Feature Testing Results
Since the search team is building a customized browser using XML User interfaces
Language (XUL), and the Mozilla Browser supports this type of customization,
functionality in Mozilla was examined. Specifically, functionality dealing with the
response to user actions such as Tab, Enter, Space, Click, and Scroll was tested.
Tabbing in Mozilla moves the focus from one hyperlink to the next. When it reaches the
end of the hyperlinks for a particular frame, the focus is shifted to the next frame, and
tabbing starts moving through hyperlinks in the new frame. When this is complete for all
frames, tabbing then moves through different elements in the toolbar area. The
functionality that is sought from a customized browser is that tabbing moves through
hyperlinks in a particular frame, and when the end of the frame is reached, the next Tab
press resets the focus to the first hyperlink of the current frame. Additionally frame
switching other than tabbing is not mapped to a particular key currently. A customized
browser would need to assign a key to this particular operation.
The Space button acts as a page-down, while the Enter key acts as a click on the currently
selected hyperlink or a form submit, depending on how the page is coded. A click on a
link follows the link; a click on anywhere but a link does not follow a link but may
change the current frame focus if the click is in a frame that does not have focus. Lastly,
scrolling using the scroll bar or a mouse scroll wheel scrolls down the page in a smooth
fashion. When tabbing through the links of a frame, if a link receives focus and it is not
currently visible, the page will scroll until it is visible.
32
Based on the behavior that the customized browser should have and these observations of
how the Mozilla browser currently operates, the search team will implement slightly
different behavior for tabbing, the space bar, and the Enter key. Additional button press
messages will be handled in order to add frame switching, left and right tab window
switching, and controlled tab creation.
4.2.2.4 Renderer Feasibility Tests
Another type of test that was performed was checking the feasibility of central Renderer
functions. Specifically, a Customized Browser/Renderer prototype unit was developed
using Coacoa under Mac OS. Coacoa allowed the implementation to have Document
Object Model (DOM) control over the pages in the training manual, and thus control
enough to find, highlight, focus on, and allow selection of keywords on the page. The
prototype demonstrated tabbing through keyword links on the page, switching panes, and
navigating through content.
The prototype test demonstrated the feasibility of basic Renderer functionality and
provided confidence that customized browsing, as well as keyword detection,
highlighting, and selection could be implemented in our system on a general level.
33
4.2.3 Software Architecture

The search software is split into three main sections as shown in Figure 9. These sections
are the web browser, local web server and the database.
Figure 9: Software Architecture
Web Browser: This is a graphical user interface developed in XML user interface
language (XUL). This is the part of the system that the user interfaces with. It was
custom designed to handle input from the click wheel mouse however it will work with
any standard input as well.
Local Web Server: The local web server is composed of many different components.
1. Controller: Handles user login and sessions. Also parses HTTP requests from the
browser and forwards them to the appropriate modules.
2. Renderer: Dynamically builds the pages in both manuals by adding in the
navigation, panels, keyword links and annotations that are specific to the MoRÉ
system.
3. Indexer: parses both manuals (XML and HTML) and populates the database with
34
relevant information about the keywords and pages. It is highly configurable thus
allowing for the constraints that determine keywords to be adapted to any system.
4. Annotation Engine: Deals with playing and storing voice annotations (see section
4.3 Capturing Experience for more details)
5. Proxy: Can be used with conjunction with a central server in order to facilitate
seamless transfer of files to the client.
6. Capturing Experience Module: Tracks usage of the system and users’
experiences. (See section 4.3 Capturing Experience for more details)
7. Search Engine: Allows searching in both manuals. The engine pushes all the logic
unto the database to improve speed.
8. Database Access Controller: Provides a wrapper class to the database access to
eliminate long quoted SQL queries in code. Currently configured for use with
either CVS or MySQL databases but easily adapted to different database types.
Database: A MySQL database which stores all keywords, annotations, smart links, and
any other relevant information.
4.2.4 Software Modules and Status
4.2.4.1 Indexer:
Implementation of the indexer is complete and it has been extensively tested. Special care
has been taken to make it highly configurable. It is very easy to specify what tags are
important and to specify actions to be taken for those tags. It can parse XML as well as
HTML. Titles, words in JavaScript, words in bold, etc., are extracted from the manuals.
Positions of words within the page are recorded as well.
After the detailed architecture of the entire system was designed, the work on indexer
began. Exact details of what data is needed for search engine, was analyzed and
accordingly the indexer were designed. Care was taken to ensure that data from the
manuals is represented at a sufficient level of abstraction without overwhelming the other
components with excessive metadata. The cases of a word being:
- in the title of the page
- in bold
- in larger font
are all considered equal and represented in the database using a single column so that
other components depending on the indexer find it easier to extract the relative
importance of the word for that page. On the other hand, a word being in JavaScript (this
usually happens when point-by-point instructions are given) is represented separately.
The Capturing Experience group wanted a separate titles table and hence the indexer fills
that table in as well. The well-defined database schema acted as an effective interface
between various components of the Search group as well as the Capturing Experience
group.
35
4.2.4.2 Search Engine:

The search engine has been completely implemented and thoroughly tested. It has been
extensively tweaked to improve the accuracy of the results. Various configurable
parameters were fine tuned until a near-optimal configuration was obtained. It takes into
consideration a number of factors before deciding on the rank of a page for a given word.
The factors considered are as follows:
- Word count i.e. how many times the word occurs on the page
- Title/bold/larger font i.e. a higher weight is given if the word occurs in bold or
is in the title of the page or has a larger font
- Higher weight if the word occurs in JavaScript
- For multiple word queries, proximity is considered
Separate considerations are given depending on whether the search query/string is a

single word query or a multiple word query. The search logic has been carefully
optimized for single word queries and they run significantly faster due to ease of query
generation and use of precompiled statements. In case of multiple word searches,
proximity of the words has been considered as an important metric. If the words occur
closer in a certain page, then that page receives a higher weight. All these metrics are
finally combined using weighted addition to compute the rank of pages and the result
returned in decreasing order of rank.
In order to satisfy space constraints, creation of new tables was avoided. To increase
speed of execution, all the operations are pushed to the database. Each search operation
results in only one query being fired on the database and the results returned. Hence, no
extra computations are to be done in the program itself.
Different groups needed different number of results (Search group needed 10 results
while Capturing Experience preferred 3 results). Hence provisions had to be done to
make the number of results to be returned as a parameter which can be set prior to the
invocation of search operation. Alternatively, appropriate number of results can be
chosen from the list returned. The search engine has been made flexible in that regard.
4.2.4.3 Proxy:
The proxy has been divided into three broad parts:

- Fetching a file from local server, if present, else fetching it from the remote
server
- Taking care of automatic updates as well as forced updates
- Storing recorded annotations on the main server
File fetching has been done. The module checks if the file is present locally. If it is
unable to find it locally, it fetches it from the centralized server. This fetching is
particularly useful when the user clicks on an annotation and a sound file has to be
played.
36
Automatic updates are of two types: fetching of updated pages of manuals and fetching
annotations. Manual page fetch is typically done on a daily basis. Annotation fetching is
done every hour. The times were chosen depending on the size of data that may have to
be transferred and the frequency of updates. They are configurable, however. Forced
updates will fetch immediately without waiting for the hourly updates. Binary log
comparisons are used to minimize bandwidth used when transferring updates.
Implementation of updates is complete, but hasn’t been integrated with the rest of the
system for want of time. However, considering the clear interface designed, it is expected
to be fairly straightforward. Currently, the updates have to be run manually and the client
database synchronized with the central database.
Storage of annotations has been done in close association with the Capturing Experience
group since they were responsible for recording, storing and naming the annotations.
Interfaces in terms of the database schema were defined to make the two activities
relatively independent, and that proved to be a good idea.
4.2.4.4 Renderer:
The renderer is responsible for dynamically generating pages. Depending on the user
request, certain parts of the page may be changed. For example, a query to search for a
keyword may result in only the search pane getting modified. Thus, the renderer has to
rebuild the page and feed it to the browser for display. The renderer adds links to both the
manuals and avoids adding any code in HTML tags or CSS or JavaScript. The renderer
displays three panes on the right side of the page:
- Navigation pane
- Link pane
- Annotations pane
The navigation pane allows the user to close a window and indicate whether a page was
helpful in solving the problem. This input is fed to the Capturing Experience group’s
module. The link pane displays the search results as well as the static and dynamic smart
links. The annotations pane allows the user to record annotations.
The renderer modifies the resolution of the content so that objects and pages can be easily
viewed on the HUD. The images in the manuals have a resolution which is fit for a
desktop computer. Hence, this had to be explicitly taken care of.
Since the HTML of the manuals is horrible, a prototype was developed (using Cocoa)
and depending on the lessons learned, the final design was decided. The renderer is a
central component which calls the methods of other classes, fetches all the data it needs
and dynamically builds the pages.
37
The renderer has to interface extensively with Capturing Experience group’s modules.
For example, the smart links generated by CE will be fetched by the renderer and it has to
take actions if a user is found to be cycling. A clean interface was designed between the
renderer and CE modules.
4.2.4.5 Controller:
Controller distinguishes between different user actions like ‘search for a word’, ‘play an
annotation’, ‘login’ etc., and routes the request to the appropriate module. It allows users
to login and stores the session information in a Java bean so that the information can be
retrieved at any time.
The controller has wrapper classes for databases tables and queries. This makes
developers’ life easier by not having to worry about connection establishment or other
related issues. The controller also sends the request to the Capturing Experience Black
box so that every request is reflected in history of pages accessed and cycling detection
and similar other functions can be performed.
4.2.4.6 Customized browser:
Due to unconventional requirements, a customized browser was needed. It takes input

from renderer and displays the page on screen. Also, it is also responsible for accepting
user inputs and passing them on to the controller to handle them appropriately. The user
inputs come from the input dial. The browser supports custom navigation like switching
between panes and tabs.
The browser was created using XUL (XML User Interface Language) with the Mozilla
browser. The browser takes care that the rendered pages are displayed properly. Though
the browser was designed to work with the dial, it works with any standard input device
(mouse, keyboard, etc.) The browser is easy to install and runs on virtually any machine.
Collaboration with the Platform group was needed for the browser since inputs from the
dial had to be received. The signals sent on a particular button press were fixed and the
hence integration went on smoothly.
4.3 Experience Capture

4.3.1 Functionality
This section details the functional level of Capturing Experience Components. More
implementation details are available in the Software Modules section.
The goal of Capturing Experience Group is to capture the experience of aircraft

maintenance workers to assist them as well as to develop tools that will guide and help
users as they use the Procedural and Training Manuals. The Capturing Experiencing
38
Group came up with 6 specific components that will be beneficial to the users. The 5
components are Annotations, Cycling Detection, Expert Contact Information, Static and
Dynamic Smart Links, and User Login.
Annotations are commentaries that users will input for a manual page. Users using voice
recording manually input annotations. Along with the actual recording, the user name,
timestamp of the annotation will be displayed on the page. The recording is saved
compressed in GSM format, because a raw wave would be too large. After compression,
it is stored in a database. Another module known as the renderer will display the newly
formed web page, which will include the annotation. This information will be sent to the
central server eventually so other users may see it in their browser. Annotations can be
of variable length; the user can start and stop recording at anytime. There is, however, a
maximum file size of 5MB so that the user does not make ridiculously long annotations.
The Cycling Detection module identifies when a user is cycling between pages. The
definition of cycling is 3 instances of one URL and 2 instances of a different URL. For
example, these two examples below would be identified as cycling:
(1) Page1, Page2, Page1, Page2, Page1
(2) Page1, Page2, Page3, Page2, Page1, Page2
Once a user is identified as cycling between two pages, those two pages are stored in a
temporary data structure until the user is finished with what he/she is looking for. When
the user clicks on the “solved my problem” link, the pages that the user cycled between
along with the page that solved his/her problem will be stored into the local database.
Then future users will then be given the stored answer link when and only when the user
is cycling between the appropriate pages.
The Expert Contact Information module was designed to provide a list of other users that
a user can contact for help if their question cannot be answered by the manuals. Since
there are security issues involving the actual names of the aircraft maintenance, fake
names were substituted for real users on the login page. However, our client gave us
some titles that would be realistic and usable. The user name and title will first appear at
a login page, which will be the first thing displayed when a user desires to open a manual
up. Once the user chooses their name, their information will be logged throughout a
session. Along with having their names and titles on the login page, their information
may appear on certain pages in which they are experts. With their information,
background and contact information, other users will know who and how to contact
someone if they need help.
Dynamic Smart Links are dynamically generated links that will help users find what they
are looking for. It learns from other users, and is semi-automatic. A path is determined as
the URLs the user visits starting from the first page of the procedural manual until they
have found what they are looking for. Users specify when they find what they’re looking
for by pressing the “Solved my problem” link on the page. The paths of future users will
be compared to the paths from previous ones. The future users will be given links to
potential solutions, which will be displayed on the right pane. Through User Login,
modules will know which using is looking through the manual, and be able to record the
39
appropriate information. This is also important so that the identity of the user who
records an annotation is known. To login, users will first scroll through a displayed list
until they find their category. This makes another column appear with user names. The
user once again scrolls until they have found their name.
Static Smart Links are links that will always be on a page that will never change (until the
manual version is updated). They are links that correlate URLs from the procedural
manual to the training manual. These links are automatically generated at index time
using the titles and headings of each page, which are stored in a database in the local
server. Considering that there could be many results for a certain page, the best results are
displayed on procedures manual pages. For now we limited the links to top 3 results from
the search index. Results are based on number of hits per page.
4.3.2 User Studies Results

Due to the entire system not being complete until the day before the presentation, most
modules did not have any user testing done. However, here included are some proposed
methods of user testing for future purposes.
For annotations, a user should try to record an annotation. On the customized browser, an
important criterion would be to see if they could find and use the record link. A user
would determine whether the recording process had any difficulties and whether the
annotation played the recording. The user would also determine if the annotation was
helpful or irrelevant to the page also to see the amount of annotations recorded over a
period of time. These tests would be run over a variety of users ranging from experts and
novices in the system.
For cycling detection, users did not necessarily consider the sequence they went through
as cycling between two pages. The users tried out different definitions of cycling to see
if the definition of cycling matched to their expectations, however this was not fully
resolved. The definition that had the most hits of what a user defined as cycling is if a
user has been to a page 3 times and another page 2 times in a sequence of 10 URLs.
However, this user study was only done with 2 people that are novices of the manuals.
With more user testing, a more accurate depiction would be met. Another idea that was
suggested was to change the name of the module to “Hunting Detection” to determine
when a user is trying to find certain information as compared to determining if they are
lost in the system.
For expert contact information, user testing can be conducted by having users browse the
manual to see if the contact information for each page was relevant to what was on the
page.
For dynamic Smart Links, users were not able to determine whether a Smart Link was
relevant to what they were looking for until they selected them, especially since link titles
were often the same among two or three links. Link titles were extracted from the pages
that the links refer to. So if multiple pages had the same title, which is common in pages
in the training manual, links to those pages had the same title as well.
40
For static Smart Links, user testing can be conducted using subjects who are intimately
knowledgeable with both manuals so that they can identify static links that draw strong
relations from those ones that do not. Furthermore, users can replace those links that draw
relations with the ones that do and attempt to fix them in the database.
4.3.3 Experimental Measurements

Due to the entire system not being complete until the day before the presentation, most of
the components of capturing experience were not tested with the complete system.
However, if testing was done on the components in Phase III, they were usually done
apart from the complete system.
For displaying and playing annotations, there were a number of things to be sensitive to.
On the displaying side, it was necessary to be sure that a page displayed annotations only
if a page had annotations. A quick check on the main pane of the browser was required
initially to also make sure that the format: the link, the user id, and timestamp, was being
properly displayed in a page with an annotation. To check the correctness and usability
of playing annotations, it was necessary to check that the correct annotation was played
when the annotation link was clicked, and that it was possible to close the Windows
Media Player window once playing had finished. It was also necessary to ensure that the
system was configured so that the player would open without any intermediate (such as
whether to open the file or save it) messages, and that the volume is adjusted properly.
See the “Software Modules and Status” section on annotations for more details on these
configurations. In terms of recording annotations, it was important to check that
recording started and stopped when the user instructed the system to, and that the file
lengths reflected that. It was also necessary to check that the filename was correct, the
file’s path was correct, and the user and timestamp, which were both used in the
filename, were accurately saved. Lastly, there was a check to see that the file ended up in
the directory it should have, and that the proper fields were updated in the database.
Cycling Detection module was tested extensively using different test cases. Before the
renderer was fully functional a simple main function was implemented. In the main
function test cases were used to see if the cycling detection was correctly implemented.
Considering the input into the cycling detection module was the URL as a string type,
strings were used in the main function. These were the test cases ran for cycling
detection:
1.) A.com, B.com, C.com, B.com, A.com, B.com, C.com, D.com, F.com
- To see if the module detected that the user is cycling between B.com and
A.com and if B.com, A.com and F.com was stored in the
“cyclingdetection” table in the database
2.) A.com, B.com, C.com, B.com, A.com, B.com, C.com, D.com, F.com (4 times)
- To see if the rank of F.com incremented
3.) A.com, B.com, C.com, B.com, A.com, B.com, C.com, D.com, G.com
- To see if G.com was stored in the same row as the previous cycled pages
(B.com, A.com) under a new column in the “cyclingdetection” table in the
database and not in a new row
41
4.) A.com, B.com, C.com, B.com, A.com, B.com, C.com, D.com, G.com (3 times)
- To see if the rank of G.com incremented
5.) A.com, B.com, C.com, B.com, A.com, B.com, C.com, D.com, H.com
- To see if G.com was stored in the same row as the previous cycled pages
(B.com, A.com) under a new column in the “cyclingdetection” table in the
database and not in a new row
6.) A.com, B.com, C.com, B.com, A.com, B.com, C.com, D.com, H.com (5 times)
- To see if the rank of H.com incremented
7.) A.com, B.com, C.com, B.com, A.com, B.com, C.com, D.com, Evict.com
- To see if the lowest ranked Smart Link was deleted and Evict.com was
added in its place
8.) J.com, K.com, L.com, M.com, N.com, D.com, B.com, J.com, K.com, J.com
- To see if the queue evicts J.com and K.com because the sequence is 12 deep
9.) J.com, K.com, L.com, M.com, L.com, M.com, O.com, M.com, Z.com
- To see if a new row was created with M.com, L.com and Z.com stored in
the database
The same sequences were performed when the integrated system was functional enough
for cycling detection to be tested. The cycling detection module completed each test
successfully.
In order to test the dynamic Smart Links, the users simply browsed through the manual
while viewing various topics. The URLs of the pages viewed were tracked manually, and
the problem-solved link was selected at various points. Since the data is stored in the
local database, the database records were verified against the expected data, including the
weights of edges between pages. Pages in the manuals were viewed again to verify that
the expected Smart Links were displayed. The local web server was also reloaded to test
that data persisted between uses. After the initial testing and debugging phase, all the
tested features behaved as expected.
Expert Contact Information was not tested extensively due to the fact that it had
dependencies with many other components. However, these would have been the
approaches taken for testing. Having people test out the annotation module would have
tested the displaying of his/her name with a user’s annotations. Along with this, the
actual titles of users would have been tested to see if they match and correlate with the
page.
The testing of the static Smart Links is fairly straightforward since it depends solely on
accessing the database and appropriately using the search function. Links are stored in the
database beforehand using the search function. When static Smart Links are needed from
a page, a simple lookup is performed to return the page ids and their respective URLs.
This process of accessing the pages has been successfully tested since it only involves
wrapper functions to connect and retrieve data from the database. A series of previously
entered data were pre-populated in the database and retrieved successfully using the static
link module. These links then serve as output of the capture experience module, which is
then passed to the renderer.
42
The pre-population of the database through extensive search of all pages has not been
tested. This is mainly due to the tight schedule of integration where certain aspects of the
software system were given more priority due their immediate need during the
demonstration. Populating the database using the StaticLinkUpdater function would
allow the static smart link module to function in a more complete manner.

In the capturing experience portion of the software architecture, there are six main
components: the audio annotation maker, the Cycling Detection sub module, the static
Smart Links generator, the dynamic Smart Links generator, and the User Login. These
components are contained within the capturing experience module of the client web
server. The only exception is the user login, which is handled at the top-level entry point
upon receiving requests from the user.
Page Request
Handler
New Annotation?
Yes No
History
Tracking
Annotations
Cycling Dynamic Smart
Detection Link Generation
Smart Links Static

Synthesizer Smart
Links
Renderer
Figure 10: Diagram of Software Architecture for Capturing

Experience
43
When the user attempts to view a page in one of the manuals, the web browser sends an
HTTP Get request to the web server on the client. Upon receiving this page request, it
notifies the capturing experience module. This page is then stored in an internal data
structure to track the history of page views of the user. Both the cycling detection sub
module and the dynamic Smart Link sub module use the resulting data structure. The
cycling detection sub module uses its heuristics to determine if the user is cycling based
on the recent history before passing on any additional links as help. The dynamic Smart
Link generator uses the history and its own processing to determine related links based on
past usage. The receipt of the message that the user has solved his problem is merely a
special case of a page request since all requests are sent through the web browser. The
resulting dynamic Smart Links are intelligently combined with the statically generated
Smart Links stored in a database and any additional help from the cycling detection
module. The result is then passed on to the renderer for display.
The audio annotation maker is also part of the capturing experience module. Based on
the request sent from the browser, the capturing experience module detects whether the
user is attempting to record a new annotation. The custom browser interacts with the user
directly via a side pane. When a request for a new annotation is made, the page request
handler dispatches to the audio annotation maker, which inserts the annotation into the
database for retrieval by the renderer.
The login page is a special page that, when viewed by the user, creates a cookie which is
sent along with each subsequent request to the web server. Data is stored along with the
annotations when they are created to allow the renderer to display which user created
each annotation.
The Static Smart Links database is populated by the static Smart Links generator. Before
being deployed for the first time or whenever the pages of the manuals are updated, this
sub module is executed once after the search indexer is run.
4.3.5 Software Modules and Status

As discussed in the functionality section, the Capturing Experience Group has 5 main
modules. The 6 components are Annotations, Cycling Detection, Expert Contact
Information/User Login and Static and Dynamic Smart Links.
4.3.5.1 Annotations
4.3.5.1.1 Displaying and Playing Annotations

When a user arrives at a manual page, the renderer calls annotation methods to query the
database and return information about the annotations on a page using the page id, which
is generated for every page in the manual. If there are no annotations, nothing is
returned, and the page will not appear differently to the user. If there are annotations, the
annotation method returns the user who recorded it, the time it was recorded, the path and
filename. This information is displayed at the bottom of the main page as a bulleted html
list, ordered with the most recent annotation shown first. The word “Annotation”, in an
example below, is an html link which links directly to where the annotation file is stored
on the web server.
44
Figure 11: Screenshot of a Training Manual Page with

Annotations
When the annotation link is clicked in the Moré system, the annotation plays in Windows
Media Player. Windows Media Player can then be closed by using the joystick and click
capabilities on the dial to navigate to the close button in the top right corner of the
Windows Media Player. While the above describes how it is played in the current
system, this could also be configured in different ways. For example, the installation-
default setting for the browser when clicking a link to an audio file is to ask whether to
open it or save it to disk. This was configured to always open without asking. Another
way that playing could be done differently is to play the file in another player. To do
this, the other player, such as Winamp, would have to be installed, and then configured
such that it is the default player.
As of the final demo, displaying and playing worked properly. They were included in the
final demo.
4.3.5.1.2 Recording Annotations

A major change that occurred in Phase III was how recording would be started and
stopped. In Phase II, a Java GUI button was used. When the button was pressed,
methods would be called to start or stop the threads that did recording as appropriate. In
Phase III, this was revised into a link in the annotation pane of the browser window,
because of the difficulty and complexity of integrating this Java button version into Moré.
When the link is pressed, the Capturing Experience controller calls the Java annotation
recording modules, which spawns a new thread to start recording. When the link in the
browser is pressed again, the thread is interrupted and recording stops. This gives the
user the ability to control the length of their annotation. There is also a maximum file
size of 5MB to both prevent overly long recordings, and as a safeguard to have recording
45
stop by itself in case the user forgets to stop it. Information about the annotation, most
notably—the page id, the user id, the time it was recorded, the path and filename, are
saved into to the database. The annotation itself is compressed and saved into a .GSM
format, which in tests reduced the file to about 10 times that of its original raw wave
format. Future work in a system with multiple users would include functionality to
upload these annotations to a central server on an interval basis, so that they can be
shared with other users.
The non-integrated version of the annotation modules worked as expected on a Moré

student’s home machine. However, the desktop machine in the lab that was being used
(and that had the configured web server and database) for integration could not support
the recording Java code because it did not have audio input devices required by internal
libraries. This was likely because the machine did not have a sound card, or other
devices that supported similar functionality. Thus, it was not possible to test integrated
recording modules on the machine that was used for integration. When the web server,
database, and the class files were migrated to the Vaio for final testing, the recording
module unfortunately did not work reliably. In some installations it worked properly,
recording from the microphone in the headset for a user-controlled time, then
compressing, naming, and saving the module properly and storing appropriate
information in the database. However, there were also times when it did not work,
halting in the section of the code where it searches for recording devices. At the end, the
reason for this was uncertain, as the device configuration used in the Java code matched
one that the Vaio did support, according to a tool from Sun, which was tested on the
Vaio. One reason suggested was that since the code was being compiled on the machine
that did not support the recording modules, which were then copied onto the Vaio, this
may have caused corruption in the class files. There was not a member knowledgeable
about how Java compiles and links to clarify whether this was possible or not.
4.3.5.2 Cycling Detection
The Cycling Detection module is fully functional. It uses the recent history of pages
viewed by the user to attempt to determine when the user is confused or needs additional
help. It uses heuristics that recognize certain patterns in the recent history. The definition
of cycling as of now is whenever a user has a sequence that includes three instances of
one page and two instances of another. This is a good but not completely accurate
indication that the user is cycling or switching between viewing the two pages. The
module was implemented in a manner that easily allows modifications to the definition of
cycling. A concern that came up was in the scenario that a user has a long sequence of
URLs he/she visited. To avoid generating a false positive of cycling in this case, only a
small constant number of pages, such as the last ten viewed, are taken into consideration.
Furthermore, the history is not stored between different user sessions.
The module uses a queue to store URLs that the user has gone to. The size of the queue is
currently 10, but may be increased or decreased depending on feedback from future user
testing. Before the module adds a URL into the queue, it checks the URL to see if there
are instances of itself in the queue. If there are 2 instances of it, then it checks to see if a
46
second URL, which is different from the first, is already stored in the queue. The idea
behind this is to see when the user has gone between two pages a couple of times. Here is
a diagram that shows this process:
Input: URL2 Input: URL2

URL1 0 URL1 0
URL2 1 URL2 1
Sees the user has Sees the user has
already gone to URL3 2 gone to another URL3 2
this URL twice URL2 3
URL twice URL2 3
before URL1 4
URL1 4
5 5
6 6
7 7
8 8
9 9
Figure 12: Diagram of How Cycling Detection stores the

URLs
Once a user has cycled between two pages, those two pages are temporarily stored in
another data structure until the user presses the “solved my problem” link. When that is
clicked, all the URLs in the data structure along with the URL the user was on when
he/she clicked on the “solved my problem” link are stored in a table named
“cyclingdetection” in a local database. The URL the user was on is stored under the
Smart Link column. Every different page that a user has cycled between is stored in a
new row in the database. There is a limit of three Smart Links per cycled page. Each
Smart Link has a rank which will determine which one will be replaced if a new Smart
Link is added when there are already three Smart Links stored for given cycled pages.
Here is an example of what happens internally when a user browses through certain
sequences of pages in the training and procedural manuals.
Sequences a user went through:
- A.com, B.com, C.com, B.com, A.com, B.com, H.com
- A.com, B.com, C.com, B.com, A.com, B.com, G.com
- A.com, B.com, C.com, B.com, A.com, B.com, K.com
- A.com, B.com, C.com, B.com, A.com, B.com, H.com
- A.com, B.com, C.com, B.com, A.com, B.com, G.com
- A.com, B.com, C.com, B.com, A.com, B.com, Z.com
In the first sequence, A.com and B.com are stored in a row under column name
“cycledpage1” and “cycledpage2”. Also H.com is stored as a Smart Link under column
name “smartlink1” and the rank is stored as 1 under column name “rank1”.
47
In the second sequence, G.com is stored under the column name “smartlink1” and the
rank is stored as 1 under column name “rank2”.
In the third sequence, K.com is stored under the column name “smartlink1” and the rank
is stored as 1 under column name “rank3”
In the fourth sequence, rank under column name “rank1” will be incremented to 2.
In the fifth sequence, rank under column name “rank2” will be incremented to 2.
In the sixth sequence, K.com is replaced with Z.com because it has the lowest rank and
the rank will be stored as 1 under column name “rank3”.
As the user cycles through the same pages as the ones in the database, the appropriate
Smart Link will display in the right pane under the search results with the other Smart
Links.
4.3.5.3 Dynamic Smart Links
The dynamic Smart Links generation module is fully functional. It stores its internal
graph data structure – the minimum amount of data needed to generate Smart Links –
into the local database. Whenever a problem is solved and the graph is updated, the
database is also updated. Upon instantiation, the data structure is loaded from the
database. So if synchronizing between the local database and the central server database
is implemented in the future, Smart Links will be shared seamlessly between users.
Figure 13: Screenshot of a Training Manual Page with

Smart Links
48
4.3.5.4 Expert Contact Information
This module mainly consists of getting information from the client about what
information is useful to have, and making it available in the system. Due to security
reasons, the client could not provide actual user data. Consequently imaginary users
were created. The data used are: user First and Last names, Titles, years in service and
contact information. All this information is stored in the database to be used by the
browser. Also, a login page was created using html which would be the first page
displayed for the user. He/she would need to choose their name to continue on to the
manuals.
4.3.5.5 Static Smart Links
The static smart links are intended to provide the users with alternative suggestions of
possible related pages in the other manual. This comes especially in handy when the
dynamic link module cannot provide the user with suggestions due to lack of training on
the module. Applying a search function on the current page of the manual produces the
static smart links. In particular, the title of the current page is used as input for the search;
the search function then returns a series of links based on these indexed titles. These links
would point the user to related pages in the other manual. This process of producing static
smart links is done offline, through an updater that would index through all the pages and
use their titles to perform search functions. The search would return related page ids and
three of them will be converted into their respective URLs and stored into the database.
Thus, within the database, there will be three URLs (generated by search) correspond to
one URL. After the database is populated with correlations between the manuals, static
link look up can be carried out in an online fashion when the user is running the More
System. A simple lookup of the current page id is performed in the database, which
would in turn retrieve up to three related links. These links are then ordered in an array
structure and used as output for the static link module. The implementation of this
module is complete and integrated with the rest of the system.
4.4 Platform
4.4.1 Functionality
The functionality of the mobile computing unit is to exist as a hardware unit, capable of
delivering the manuals and experience captured to the user. The unit must also support
gathering spoken input annotations to be added to the manuals, user input to navigate the
pages of the manuals, display the computing units screen to a head-mounted display, and
to allow the user to hear spoken annotations added to the manuals. This high-level
functionality was achieved using a Sony Vaio mobile computing unit, a Liteye HMD for
display, a David Clark headset and earphones and a custom-made dial input device. The
Dial is intended to be a pure input device that allows the user to control the mobile
49
computer while in many different orientations and possibly wearing gloves. Functionality
for the Dial will include the ability to left click, scroll through menus and links on a page,
navigate the screen with a joystick, move through tabs, create tabs, and switch page
frames.
The mobile platform that MoRÉ runs on is a fully-functional computing unit, with
primary and secondary input methods, displays, and audio outputs. The Sony Vaio
mobile computer runs the Windows XP Professional operating system and can run any
ordinary Windows application. The platform can take audio recordings via a microphone
attached to the composite Head-Mounted Display / David Clark Headset or a secondary
microphone in the USB audio adapter, and playback recordings through the David Clark
headset or a speaker internal to the Sony Vaio. The platform can visually display the
MoRÉ application through the primary Head-Mounted Display on an 800x600 SVGA
screen, or also display on the included LCD attached to the Sony Vaio at 640x480 VGA,
800x600 SVGA (native), and 1024x768 XGA resolutions. The platform can also receive
input primarily through the custom Dial, as well as via a touch-screen, buttons and a
virtual keyboard on the Vaio itself.
4.4.2 Component Specifications and Pictures

4.4.2.1 Dial
The Dial electronics design is based on a Freescale Semiconductor MC908JB16DWE
microcontroller with the following features:
• High-performance M68HC08 architecture
• Fully upward-compatible object code with M6805, M146805, and M68HC05 families
• Low-power design; fully static with stop and wait modes
• 6-MHz internal bus frequency
• 16,384 bytes of on-chip FLASH memory with security feature
• 384 bytes of on-chip random access memory (RAM)
• Up to 21 general-purpose input/output (I/O) pins, including:
o 15 shared-function I/O pins
o 8-bit keyboard interrupt port
o 10mA high current drive for PS/2 connection on 2 pins (with USB module disabled)
o 1 dedicated I/O pin, with 25mA direct drive for infrared LED (32-pin package)
o 6 dedicated I/O pins, with 25mA direct drive for infrared LED on 2 pins and 10mA direct drive for
normal LED on 4 pins (28-pin package)
• Two 16-bit, 2-channel timer interface modules (TIM1 and TIM2) with selectable input capture, output
compare, PWM capability on each channel, and external clock input option (TCLK)
• Universal Serial Bus specification 2.0 low-speed functions:
o 1.5Mbps data rate
o On-chip 3.3V regulator
o Endpoint 0 with 8-byte transmit buffer and 8-byte receive buffer
o Endpoint 1 with 8-byte transmit buffer
o Endpoint 2 with 8-byte transmit buffer and 8-byte receive buffer
• Serial communications interface module (SCI)
• Dual clock generator modules (CGM) (32-pin package)
• In-circuit programming capability using USB communication or standard serial link on PTA0 pin
• System protection features:
o Optional computer operating properly (COP) reset
o Optional Low-voltage detection with reset
o Illegal op-code detection with reset
50
o Illegal address detection with reset

• Master reset pin with internal pull-up and power-on reset
• IRQ interrupt pin with internal pull-up and Schmitt-trigger input
• 32-pin low-profile quad flat pack (LQFP) and 28-pin small outline
• integrated circuit package (SOIC)
• Features of the CPU08 include the following:
o Enhanced HC05 programming model
o Extensive loop control functions
o 16 addressing modes (eight more than the HC05)
o 16-bit index register and stack pointer
o Memory-to-memory data transfers
o Fast 8 × 8 multiply instruction
o Fast 16/8 divide instruction
o Binary-coded decimal (BCD) instructions
o Optimization for controller applications
o Third party C language support
The MC908JB16DWE microcontroller is a 28-pin SOIC surface-mount device and is

clocked via 12MHz microprocessor crystal. The MC908JB16DWE microcontroller
supports in circuit programming (ICP) through its USB interface port and is powered
from +5VDC direct from host USB interface.
The MC908JB16DWE microcontroller performs all sensing, control and supervisory and
communications functions. Pin interfaces and functions are contained in the Appendix.
(USB & SCI Communications Ports)

Two communications port interfaces are provided to the microcontroller; Universal Serial
Bus (USB) and non-buffered RS-232 Serial Communications Interfaces (SCI). Both USB
and SCI communications ports are available on connectors J2 and J1 respectively with
pinouts, as described in the Appendix.
(RKJXM Rotary Encoder/8-Position Joystick Switch)

A specialized multi-purpose Rotary Encoder / Joystick component (RKJXM Series
manufactured by ALPS) is utilized to provide mechanical detente and encoding of
rotational dial, and 8-position joystick with pushbutton. The RKJXM is a low cost rotary-
encoder/joystick with quadrature output at 15 pulses per revolution and 8-position
momentary joystick with pushbutton. The RKJXM is a through-hole PCB mountable
device and is based on mechanical switch closure. Filter and conditioning circuitry is
required to effectively eliminate contact-bounce and EMI/RFI pickup. All switch and
encoder signals are buffered from EMI/RFI injection and contact bounce using discrete
low-pass RC filter elements.
(DHC3 Series Pushbutton: Left, Right and Middle User Input)

The Dial device includes three user pushbutton inputs that may be programmed for any
standard mouse or keyboard function. DHC3 Snap-Action pushbuttons manufactured by
Cherry Switch Corporation are utilized for this function. The DHC3 pushbuttons are
SPDT contacts with only the normally open contact used. When a switch is depressed, a
low-level appears at the corresponding microcontroller pin. In non-depressed state, a high
level appears at the corresponding microcontroller pin via internal or external pull-up
51
resistor. All switch and encoder signals are buffered from EMI/RFI injection and contact
bounce using discrete low-pass RC filter elements.
(STAT1 & STAT2 LED User Supervisory Indicators)

Two Super-Bright user supervisory LED indicators are implemented on the Dial PCB.
These LED’s are intended for direct user output device for indication of various device
status or feedback functions. The specific function of each LED is determined via
microcontroller programming. One blue and one red LED are included on the
component-side of the Dial PCB, the local of these LED’s facilitates user viewing
through transparent mechanical housing about the middle user pushbutton location.
The red and blue LED’s are individually controller via microcontroller pins PTD0 and
PTD1 pins respectively. Microcontroller pins PTD0 and PTD1 must be driven low in
output LED-drive configuration mode to turn on LED’s.
See Microcontroller Pin Function & Description table for additional operational
information.
(Microcontroller Reset)
A PCB mounted momentary pushbutton switch is implemented to allow user-initiated
microcontroller reset function during device programming development. This reset
pushbutton is not intended to be used during normal Dial field operation.
Features
• Powerful MC908JB16DWE microcontroller
• Miniature 80% surface mount PCB implementation
• Very low-cost implementation
• Intrinsic Safety elements included on power and serial communications
interfaces
• High-degree of manufacturability
• Simple & reliable design
• Powered via Host USB port
• Low-power
Fully programmable user input/output device
52
Figure 14: Dial Final Product
4.4.2.2 Vaio Ruggedized Enclosure

The Enclosure was meant to house the processing unit in a safe environment from heat,
poor weather, dust, and any other interactions with the physical environment. It was
implemented in less than 0.1 inch thick aluminum, measuring 7.75 x 6.38 x 2.38 cubic
inches with a foam padded interior to protect the Vaio and allow heat to be dispersed
from the unit. In addition, the enclosure was designed to fit entirely inside the carrying
holster, as shown in Section 4.4.5.
53
Figure 15: Vaio Ruggedized Casing
4.4.2.2 Hot-Swap Controller

The Hot-Swap Controller (HSC) is based on the non-standard Sony Li-Ion Smart Battery
Pack. At the heart of the HSC design is a Freescale MC68HC908SR12 microcontroller.
The microcontroller monitors the standard Serial Module Bus (SMB) communications
interface between the Sony Vaio U PC and the smart battery pack to determine battery
state and charge capacity. Solid-state power and signal switches are implemented on the
HSC that isolate or connect the PC to selected battery pack under microcontroller
command and supervisory. The HSC may then make intelligent decisions about battery
status and switch between pack/HSC assemblies as appropriate. The HSC also includes
circuit implementations to independently monitor battery pack current and cell voltage, in
addition provisions for shared internal power bus between HSC devices and interrupt
request bus to synchronously communicate events between devices. The HSC also
included specialized power switching circuitry based on a dual-MOSFET switch and
analog amplifier that limits current during pack switching.
(MC68HC908SR12 Microcontroller)
The Hot-Swap Controller (HSC) electronics design is based on a Freescale
Semiconductor MC68HC908SR12 microcontroller with the following features:
• High-performance M68HC08 architecture
• Fully upward-compatible object code with M6805, M146805, and M68HC05 Families
• Maximum internal bus frequency:
o 8-MHz at 5V operating voltage
o 4-MHz at 3V operating voltage
• Clock input options:
o RC-oscillator
o 32kHz crystal-oscillator with 32MHz internal phase-lock-loop
• 12k-bytes user program FLASH memory with security1 feature
• 512 bytes of on-chip RAM
• Two 16-bit, 2-channel timer interface modules (TIM1 and TIM2) with selectable input capture, output
compare, and PWM capability on each channel
• Timebase module
• 3-channel, 8-bit high speed PWM (125kHz) with independent counters and automatic phase control
• Serial communications interface module (SCI)
• System Management Bus (SMBus), version 1.0/1.1 (Multi-master IIC bus)
• 14-channel, 10-bit analog-to-digital converter (ADC), with auto-scan mode for 4 channels
• Current sensor with programmable amplifier
• Temperature sensor (–20°C to +70°C)
54
• IRQ1 external interrupt pin with integrated pullup

• IRQ2 external interrupt pin with programmable pullup
• 8-bit keyboard wakeup port with integrated pullup
• 31 general-purpose input/output (I/O) pins and 2 dedicated pins:
o 31 shared-function I/O pins
o Two dedicated analog input pins
• Low-power design (fully static with Stop and Wait modes)
• Master reset pin (with integrated pullup) and power-on reset
• System protection features
o Optional computer operating properly (COP) reset
o Low-voltage detection with optional reset
o Illegal opcode detection with reset
o Illegal address detection with reset
• 48-pin low quad flat pack (LQFP) and 42-pin shrink dual-in-line package (SDIP)
• Features of the CPU08 include the following:
o Enhanced HC05 programming model
o Extensive loop control functions
o 16 addressing modes (eight more than the HC05)
o 16-bit Index register and stack pointer
o Memory-to-memory data transfers
o Fast 8 × 8 multiply instruction
o Fast 16/8 divide instruction
o Binary-coded decimal (BCD) instructions
o Optimization for controller applications
o Efficient C language support
The MC68HC908SR12 microcontroller is a 48-pin low quad flat pack (LQFP) surface-
mount device and is clocked via 32KHz microprocessor crystal. The MC68HC908SR12
microcontroller supports in circuit programming (ICP) through its SCI interface port and
is powered from 3.3VDC direct from host battery pack power interface.
The MC68HC908SR12 microcontroller performs all sensing, battery control, supervisory

and communications functions. Microcontroller pin interfaces and functions are shown in
the Appendix.
(SMB & SCI Communications Ports)

Both Serial Module Bus (SMB) and Serial Communications Interface (SCI)
communications port interfaces are provided to the microcontroller. The SCI port is
implemented as user development interface and the SMB port is dedicated to smart
battery communications. The SCI communication port supports standard RS-232 and
includes a level translator and full RS-232 signal levels. The RS-232 translator
automatically enters a low-power shuts-down mode when valid RS-232 signal levels are
not detected at the host interface connector. RS-232 serial communication interface is
available on connectors J4 with the following pinout, as shown in the Appendix.
The Serial Module Bus (SMB) channel 0 communications port is dedicated to smart
battery communications between the Vaio U PC and the battery pack. The HSC SMB
port may communicate directly with the battery pack (when Vaio U PC electrical
interfaces are disabled) or utilized to monitor SMB communication between pack and
PC. SMB serial communication interface is available on connectors J1, J3, J5 and J6 with
the pinouts available from the main schematic or main connector tables.
55
(Battery Pack Current Monitor)

The MC68HC908SR12 microcontroller includes an analog section specifically designed
to monitor battery charge and discharge current. This function allows an optional means
for the HSC to manage and assess battery state. Battery current measurement is facilitated
by a low-side current-sense resistor, the voltage from this sense resistor is monitored via
microcontroller OPIN1 (BATT_AMP pin. See MC68HC908SR12 microcontroller
datasheet (section 14 Analog Module) for additional details and information on battery
charge and discharge monitor.
(HSC Temperature Monitor)

The MC68HC908SR12 microcontroller includes an analog section specifically designed
to monitor ambient temperature. This function allows an optional means for the HSC to
manage and assess battery state. Temperature measurement is facilitated by a
semiconductor temperature sensor internal to the MC68HC908SR12 package, the voltage
from this temperature sensor is monitored via microcontroller ADC. See
MC68HC908SR12 microcontroller datasheet (section 14 Analog Module) for additional
details and information on temperature monitor.
(Battery Pack Voltage Monitor)

Battery pack voltage may be monitored by the MC68HC908SR12 microcontroller ADC
and additional conditioning, switching and scaling circuitry implemented on the HSC. It
is important to note that battery pack voltage and Vaio PC supply voltages are measured
separately and can differ depending on whether pack is currently providing power to the
Vaio PC. If an alternate pack is currently supplying current, the battery pack and PC
voltage will differ, but if the bidirectional MOSFET power switch is turned on; battery
pack and PC voltages will be identical.
Battery pack voltage is monitored at MC68HC908SR12 ADC pin ATD4, This pin
reflects scaled voltage supplied from battery pack connected to HSC independent from
Vaio PC voltage. Note PTD6 (VMON_SW) must be high logic level to enable analog
voltage feed to this pin. Battery voltage appearing at microcontroller ADT4 pin is scaled
by implemented discrete circuitry and may be characterized using following equation.
ADT 4(VDC ) = BATT (VDC ) * 0.4
See MC68HC908SR12 microcontroller datasheet section 15 (Analog to Digital

Converter) for addition details and information.
(Vaio Supply Voltage Monitor)

Vaio U PC supply voltage may be monitored by the MC68HC908SR12 microcontroller
ADC and additional conditioning, switching and scaling circuitry implemented on the
HSC. It is important to note that battery pack voltage and Vaio PC supply voltages are
measured separately and can differ depending on whether pack is currently providing
power to the Vaio PC. If an alternate pack is currently supplying current, the battery pack
56
and PC voltage will differ, but if the bidirectional MOSFET power switch is turned on;
battery pack and PC voltages will be identical.
Vaio U PC supply voltage is monitored at MC68HC908SR12 ADC pin ATD3, This pin
reflects scaled voltage supplied to the Vaio U PC independent from battery pack voltage.
Note PTD6 (VMON_SW) must be high logic level to enable analog voltage feed to this
pin. Vaio U PC supply voltage appearing at microcontroller ADT3 pin is scaled by
implemented discrete circuitry and may be characterized using following equation.
ADT 3(VDC ) = Vaio U PC (VDC ) * 0.4
See MC68HC908SR12 microcontroller datasheet section 15

(Analog to Digital Converter) for addition details and information.
(Dual MOSFET Power Switch)

A specialized dual MOSFET based bidirectional power switch has been implemented on
the HSC. This power switch consists of dual series common-drain configured power
MOSFET’s. The power switch includes a specialized analog circuit that monitors the
voltage appearing on the Vaio U PC supply (when power switch is off) and matches the
monitored voltage at the common drain connection. This function prevents dramatic
current spikes in the event MOSFET is switched on when battery and Vaio supply
voltages differ significantly (e.g. when a fresh battery pack is switched on over a depleted
pack).
MC68HC908SR12 microcontroller pin PTD1 (PWR_SW) controls power switch on/off
function. High logic level on microcontroller PTD1 output pin enables power switch
between battery pack and Vaio PC. Low logic level disables power switch between
battery pack and Vaio PC.
(MOSFET Communications & Signal Switch)

Multiple MOSFET based bidirectional switches have been implemented on the HSC.
These low-current communications and signal switches are realized using standard
bidirectional analog switches. MC68HC908SR12 microcontroller pin PTD0 (COM_SW)
controls communications and signal switch on/off function. High logic level on
microcontroller PTD0 output pin enables communications and signal switches between
battery pack and Vaio PC. Low logic level disables communications and signal switches
between battery pack and Vaio PC.
(Power Bus & 3.3VDC Regulator)

When multiple HSC and battery pack assemblies are implemented within a given system.
It is desirable to minimize power draw by placing HSC and battery pack in low-power
standby mode. The Sony battery pack automatically enters low-power mode when the
pack is disabled (i.e. either not electrically connected to the Vaio PC or not enabled by
the HSC microcontroller PTD3 output pin (/BAT_ON). The HSC on the other hand must
be placed in low-power mode through software action, however with the battery power
switch tuned off; the microcontroller would not be powered. It is also important that the
microcontroller is ready at all times to resume operation (from low-power mode) within
milliseconds to immediately supply power to the Vaio U PC in the event another
57
HSC/pack assembly is either user removed or becomes sufficiently low in charge

capacity.
In order to maintain microcontroller power at all times; a power bus is implemented that
connected common power between all HSC’s within a given network. This power bus is
active with battery terminal voltage from any battery pack currently switched on and
supplying power to the Vaio U PC. Safety rectifiers are included at on each HSC
implementation to prevent current-flow on power bus between discrete battery packs.
With the power bus always powered; all HSC’s within a given network are provided
power from a single enabled battery pack. HSC/pack assemblies not currently powering
Vaio UPC would enter low-power states ready to resume programming upon bussed /IRQ
request.
3.3VDC voltage regulation is implemented on each HSC, this regulated and conditioned
voltage is derived from the common power buss battery terminal voltage. Most HSC
systems are directly supplied voltage from the 3.3VDC regulator implementation.
(System-Wide Bussed IRQ)

A bussed interrupt request signal is implemented between all HSC’s within a given
system. The bussed /IRQ is interconnected between all HSC devices in system and driven
low when any HSC IRQ_EN (PTD2) is driven high. The /IRQ bus is directly connected
to each HSC microcontroller /IRQ1 (/IRQ) pin, and may be used to provide synchronous
signaling between HSC devices in an interconnected system. The primary use of this
bussed interrupt request is to signal sleeping microcontrollers to wake-up from low-
power modes and resume power management functions (e.g. when a given HSC
determined that its battery pack is sufficiently low in charge capacity, the /IRQ bus is
used to signal fresh HSC/pack assemblies to take over as primary power supply). Note
the microcontroller /IRQ1 (/IRQ) pin is used to monitor the /IRQ bus, when an HSC
microcontroller initiates an /IRQ bus request; the PTD2 output pin is used. High logic
level on microcontroller PTD2 pin initiates system-wide /IRQ to all connected HSC
devices. Logic low on PTD2 pin disables all /IRQ interrupts. PTD2 pin should only be
pulsed high for short durations when an /IRQ is to be initiated system-wide.
(Microcontroller Battery ON/OFF Switch)

The battery pack may be switched on and off under microcontroller control independent
of electrical power and communications connection with Vaio U PC. Microcontroller
PTD3 (/BAT_ON) output pin is utilized for this function. Low logic level on this pin
turns on battery pack connected to HSC. High logic level on PTD3 pin turns off battery
pack connected to HSC. This function is required when battery pack and HSC are not
currently supplying Vaio PC with power; for minimum power draw when pack is not
currently being used as Vaio U PC primary power source. This function may also be used
to allow HSC microcontroller interrogation of battery to determine status and charge state
via USB communications port.
(Crystal Oscillator)
58
A 32KHz ceramic resonator is used for primary microcontroller time-base. Discrete

termination and loading elements are implemented on the HSC PCB in support of
resonator operation. The HSC microcontroller includes internal PLL implementation
which multiplies the 32KHz clock to derive desired bus frequency.
(/LED_RED & /LED_BLU LED User Supervisory Indicators)

Two Super-Bright user supervisory LED indicators are implemented on the Hot-Swap
Controller PCB. These LED’s are intended for direct user output device for indication of
various device status or feedback functions. The specific function of each LED is
determined via microcontroller programming. One blue and one red LED are included on
the component-side of the Hot-Swap Controller PCB; the local of these LED’s facilitates
user viewing through transparent mechanical housing.
The red and blue LED’s are individually controller via microcontroller pins PTA4 and
PTA5 pins respectively. Microcontroller pins PTA4 and PTA5 must be driven low in
output LED-drive configuration mode to turn on LED’s.
See Microcontroller Pin Function & Description table for additional operational
information.
(Microcontroller Reset)
A PCB mounted momentary pushbutton switch is implemented to allow user-initiated
microcontroller reset function during device programming development. This reset
pushbutton is not intended to be used during normal Hot-Swap Controller field operation.
4.4.3 Hardware Architecture

The MoRÉ Platform Architecture consists of a combination of custom hardware
components and premium commercial devices. The system is centered around a
computationally powerful, yet small in size and lightweight computer, the Sony Vaio-
U8G. The requirements of the MoRÉ software demand a computer capable of running the
Microsoft Windows operating system, the ability to interface wirelessly with a remote
server, have the means of presenting audiovisual information, and interface with a variety
of custom and commercial devices. The Sony Vaio-U8G meets these and other functional
necessities, and with a long battery life of 4-5 hours, light weight of only 1.2 lbs. and size
of only 6.57 x 4.25 x 1.03 inches.
There are three main pieces of hardware: the integrated headset for display/audio, the
Sony Vaio holster and enclosure, and the Dial and its harness. The Vaio interfaces with
two other primary devices, the combined audiovisual headset and the custom Dial
pointing device, through a small Targus USB 2.0 Hub, which gets its power from the
Vaio. The hub sends and receives all user data to and from the Vaio through a single USB
cable, and uses three out of its four I/O communications ports for interfacing with the
various other devices. One of these three connections is used to power the Liteye Head-
Mounted Display (HMD), and the other two connections are used as data links for the
Sound Professionals USB Audio Converter, and the Custom Dial Pointing Device (Figure
16).
59
The Audiovisual Head Unit consists of the Liteye Head-Mounted Display and David
Clark H10-13.4 headset. As mentioned earlier, the HMD gets its power from the hub
through a USB cable, and it receives the signal for the user’s visual computer display
through a VGA cable that is connected directly to the Vaio. The Liteye Head-Mounted
Display is physically attached to the David Clark headset, which provides the user with
audio information from the software, and provides the capability for the system to receive
voice data from the user, if allowed for. This earphones/microphone headset plugs into
1/8 inch stereo audio ports on the Sound Professionals USB Audio Converter. The David
Clark headset provides the added bonus of hearing protection from high volumes of
noise, from 23 decibels, to be specific. The small USB audio converter, powered through
the hub, is capable of converting raw stereo audio signals into a standard audio data
stream interpreted by the operating system.
And lastly, the custom-made Dial Pointing Device also interfaces with the system via a
USB cable connected to the hub. As described in this report, the Dial’s unique
functionality is utilized by the custom browser that displays the manual and other
relevant information.
60
Figure 16: Hardware System Architecture
The Vaio interfaces with two other primary devices, the combined audiovisual headset
and the custom dial pointing device, through a small Targus USB 2.0 Hub, which gets its
power from the Vaio. The hub sends and receives all user data to and from the Vaio
through a single USB cable, and uses three out of its four I/O communications ports for
interfacing with the various other devices. One of these three connections is used to
power the Liteye Head-Mounted Display (HMD), and the other two connections are used
as data links for the Sound Professionals USB Audio Converter, and the Custom Dial
Pointing Device.

The software in the dial is very complicated. It is not a simple task to program a
microcontroller correctly and this task is made more difficult by the fact that this
microcontroller is a USB microcontroller. In our microcontroller the first chunk of
software that had to be mastered was the in-circuit programming portion (ICP). This
software is done in assembly language and is built upon existing icp code provided by the
61
manufacturer. Understanding how the microcontroller ICP architecture works was the
most challenging part, once that was figured two lines of assembly had to be modified so
that the assembly code jumped to the correct memory location upon being plugged in and
also could have its memory erased and then reprogrammed if the code had to be changed.
The second code chunk that had to be written was the USB code. This code sends USB
messages from the microcontroller to the PC. This code was also very difficult because
of the complexity of the USB code. Making this code was very difficult so we relied on
existing USB code for a similar microcontroller to speed our progress. Making this code
work was more challenging that first glance because the memory mapping was different
for the two microcontrollers. The last chunk of code was user code which consisted a big
polling while loop that checked to see which buttons are being pressed and then combines
that with the USB code to send keyboard messages to the PC.
Once the microcontroller had been programmed to conform to Human Interface Device
(HID) USB standards, it was able to communicate without having to write additional
drivers with the Windows XP operating system running on the Vaio. This enabled the
Dial to communicate with the custom browser developed by the Software teams, which
runs on top of XP, as shown in Figure 17.
Button- Windows Moré

Polling Virtual Application
Loop Keyboard
User User
Code
USB Protocol USB Host
Library Controller
Cable
Kernel Level Kernel
Dial Vaio
Figure 17: Dial Firmware Architecture
4.4.5 Hardware on Body

After completing the Hardware and Software design phases, design engineering and
implementation for the wearables of the system, including implementation and user
testing. The integrated headset covers most of the head, including the right eye and both
ears. It is relatively light. A more in-depth description of the design is contained below.
62
Figure 18: Integrated HMD and Headphones
The Vaio/holster wraps around the waist and can be attached to either leg. The product
weighs several pounds together. It also encumbers that leg somewhat by sticking out
several inches from the body.
Figure 19: Vaio Holster
63
The Dial’s harness wraps around the chest with two belts, placing the Dial directly on the
user’s chest which allows for equal usage for left and right-handed users.
Figure 20: Dial Harness
4.4.4.1 Audiovisual Head Unit Human-Interface Considerations

The design of the Audiovisual Head Unit needs to correct for imperfections in the
physical interaction between the Liteye HMD and the David Clark headset. Both devices
were designed independently from one-another at different companies, and were not
intended to be used in conjunction with any other apparatus on the head. The Platform
team would have preferred alternative designs from both companies, but such substitute
styles were either not in stock or did not exist.
The problems encountered were as follows. For one, both devices, in their unmodified
form, were competing for physical space above the right ear. The headset needed
significant volume surrounding the ear, and the HMD protruded from its head-mount,
directly above the right ear. The second problem was that the two devices had trouble
being mounted to the head at the same time. There was either difficulty in seeing the
display screen, or physical discomfort to the user. The simple, if non-ideal, solution to
these problems was to detach the display screen from its base, and mount it directly onto
one of the protruding screws on the headphones. This provides an interim solution to this
problem, until a more sophisticated HMD can be obtained from the company Liteye.
64
Figure 21: HMD and Headphones - Before and After Views
4.4.4.2 Power Savings versus Power-up Times in Various Modes of Operation

The Platform team was able to complete preliminary studies on the effects of putting the
system into various power-saving modes, as opposed to fully turning off and restarting
the system. The Sony Vaio U8G comes with similar power-saving features that are on
laptops on the market, such as “stand-by” and “hibernate”. In stand-by mode, the Vaio
will shut down most of its systems, including the hard drive and LCD display, and stop
sending any signals to and from the microprocessor and wired or wireless communication
channels. It will, however, maintain some power on the system board, and keep power to
its RAM. This allows for a speedy time in returning to where the user stopped working
with the system. Depending on which applications are running, and which resources the
Vaio needs in returning to work with these applications (hard drive data or establishing a
wireless network connection), this resume time can range between 5 and 9 seconds.
Further studies collecting data as to how much power-savings this mode provides need to
be conducted, but initial estimates indicate that up to 10%, or higher, of a fully-charged
battery could be expended over the course of an 8-hour workday, in providing this
capability.
The second power-saving mode is called “hibernate”. Upon receiving the signal for
hibernation, Windows begins storing pertinent data to the hard disk, primarily consisting
of all information actively being used in memory, such as Kernel memory and
information stored in RAM. And when the user wishes to resume use of the system,
Windows reads back this stored information and writes it to the appropriate memory
locations. In order to use this mode, the Vaio must have sufficient hard disk space
available to store this data from memory, usually ranging in the hundreds of megabytes.
However, this power-saving mode provides complete savings of the battery, as no energy
is spent in trying to keep any Vaio systems on. No component requires power in this
mode, as all data necessary to continue work, is permanently stored on the hard drive.
Other than needing adequate room on the hard drive, the Vaio also requires additional
time to fully resume operations after hibernation. Depending on how much information
needs to be read back by the operating system and rewritten to memory, resuming work
65
after hibernation will usually take between 8 and 15 seconds, but sometimes longer. If a
reasonable means of allowing for hibernation during a normal workday can be developed,
then this method of power-saving is well-worth the wait.
4.4.4.3 Usability Tests: Webbing Setup Time
Introductory studies on the way that individual users attach the various components to
their bodies have also been performed. There are three major webbing units in this
system, including the Audiovisual Head Unit, Dial Strap, and Han Solo Belt, so named
for the movie character who wore a quasi-belt-gun-holster. The head unit is put on in the
exact same manner as any typical pair of headphones, and the display is adjusted into
place with the hands. The dial strap is put on over the head, and then tightened with the
Velcro belt across the stomach. And finally, the Han Solo belt secures the Vaio and
surrounding ruggedized enclosure to the leg with two Velcro straps, but allows the weight
to be carried by the subject’s waist, via the belt. In this way, the lightweight Vaio is
locked in-place on the leg, but the user is not discomforted by its weight since it hangs
from the waist.
Test subjects were given these three components, and asked to put them all onto their
bodies, and plug in all of the devices into the appropriate ports. The system was already
turned on, and the appropriate software was already functioning. These users were not
asked to startup the system, place the Vaio into the ruggedized enclosure, nor asked to
use the devices, other than to put them on. These subjects had never worn the system
before, and they were only given a picture and a brief description of how the system
should be worn.
It was found that new users took on average just over 3 minutes to complete the task. And
as these users continued to try putting on all of the webbing components in More
iterations, the times were reduced to a little less than 2 minutes. Thanks to the simplified
design of only having three major articles of webbing, easily recognizable I/O ports for
all of the devices, and having these ports housed in one easy place on the waist, setup
times for both veterans and novices are minimized.
4.4.6 Hardware Modules and Status

As of the end of the Spring 2006 semester, the status of the Platform system is as follows:
The Dial is a completed prototype as of the end of the Spring 2006 semester and provides
keyboard input to any host to which it is connected via the USB protocol. The hardware
contained within the designed Dial enclosure is pictured below. Additional information
about the hardware connections and pins is available in the Appendix.
66
Figure 22: Dial PCB Component-side and Solder-side Views
The Enclosure is a completed prototype as of the end of the Spring 2006 semester and
both secures and protects the Sony Vaio unit, as well as allowing heat to escape and
placing the unit directly on the thigh. Pictures of the completed design are shown above.
The Hot-Swap controller has completed the hardware design phase and is in the process
of implementation. Layout and PCB production have been completed as of the end of the
semester (Figure 23). The Hot-Swap Controller Printed Circuit Board (PCB) was fully
implemented using Cadence ORCAD Schematic Capture CIS and Layout Engineers
Edition. The PCB is a double sided, 62mill thickness G10 circuit board with full ground
planes on both component and solder surfaces. Additional information describing the
microcontroller and connections of the Hot-Swap hardware are contained in the
Appendix.
Future work to improve upon the existing prototype includes the completion of the
firmware design of the Hot-Swap Controller.
Figure 23: Hot-Swap PCB Component-side and Solder-side

Views
67
Key issues that remain include no preexisting hot-swap firmware for the microcontroller
used, as well as no battery clips to attach to the batteries and holster belt. Figure 24
contains a prototype for the proposed Hot-Swap Controller, which would contain three
batteries to be located on the small of the back. When programming for the controller is
completed, the physical design of the casing and on-the-body testing can be completed
for the design.
Figure 24: Proposed Hot-Swap Controller Design
4.5 Conclusions
4.5.1 Summary of Key Design Issues
4.5.1.1 Capturing Experience
There were many design issues that came up in Phase III. One was to implement our
modules according to when and how the controller and renderer call the capturing
experience modules. Depending on the sequence of calls, the capturing experience
modules had to be adjusted. Also the modules had to be designed in a way that they
would be easily altered depending on the user testing we were planning on doing.
For cycling detection, determining the definition of cycling was an issue that came up
often along with having links that are useful to the user. Also an issue came up that in the
initial use of the Moré application, the dynamic smart links would be the same links as
the cycling detection ones. Only when more users use the system and the ranking of
smart links in the two modules differ will there be separate links.
The main design issues with dynamic Smart Links were to allow users to shortcut to
useful pages in the manuals without much burden. Based on experience, if users have to
do too much for a feature to function properly, they will probably not use it at all. So
heuristics were devised to attempt to infer when users had found useful pages, which was
the main issue in creating automatically generated Smart Links. For example, one
68
heuristic was that when a user spends a long time viewing a single page without
navigating away, it is likely that the user found useful information. Or by mapping pages
into a topic hierarchy based on the table of contents, one heuristic was to infer that the
user had found what he was looking for when he navigated to a page that was about a
different topic or in a different sub tree in the hierarchy. However, to maintain a
simplified approach for a rapid prototype, a link was used to require manual user input to
specify when he found what he was looking for. This does not bring much burden onto
the user, but it allows for dynamically generated Smart Links that are otherwise created
completely automatically.
4.5.1.2 The Dial
In order to facilitate fast design and implementation cycle, a direct to PCB approach was
taken. This approach eliminates the typical ‘breadboard’ engineering prototype cycle
using large pinned devices, and instead implements a prototype PCB based on final
design component selection and footprint requirements. Direct to PCB prototyping is an
effective and efficient engineering methodology that includes the following benefits:
• Forces thorough and competent design practices in schematic-entry
phase
• Matures detailed design in schematic-entry phase
• Allows use of single set of procured components
• Prototypes circuit, mechanical and PCB layout designs in single
engineering phase
• Allows analysis and testing of matured design in single engineering
phase
• Lowers risk of design anomalies
The direct to PCB engineering approach often requires only a single revision cycle to
produce error-free circuit design and PCB implementations.
Component Selection
The use of miniature surface-mount (SMD) electronic components was pursued early in
the engineering design process. SMD components offer the following benefits:
• Extremely space efficient
• Readily available (industry standard)
• Low-cost
• Easy to hand solder
• Facilitate low-noise characteristics
• Allow efficient miniature PCB layout
• Lower cost of PCB production and assembly
USB Interface
One key design issue was the port interface of the Dial. We assumed from very early on
that the Dial would be USB-compatible, as USB ports are widely available and fairly
universal in new computers, have mountains of background information available, and
69
have a strong development community. The Sony Vaio computer has one USB port, but
since several More devices use USB for data and power, the solution was to utilize a
miniature, unpowered USB hub.
The choice to conform to the USB interface influenced the electrical engineering
by leading to the choice of a Freescale USB microcontroller to run the Dial. It also
influenced the software engineering by allowing the Dial to conform to the USB Human
Interface Device (HID) class of devices, which encompass keyboards, mice, joysticks,
and other More exotic input devices. This was a natural fit given the goal of the Dial to
function as a rugged, orientation-independent input device. Later, the design decision to
emulate a USB keyboard was made when it was discovered that working USB keyboard
code existed for a close relative of our Freescale microcontroller. As a result, the Dial is
simply a functional USB keyboard that sends specific characters when different functions
are performed – the characters are interpreted by the MoRÉ application to become
meaningful.
Microcontroller Selection
Various microcontrollers were considered for use for dial implementation. These
included microcontrollers from various manufacturers; Atmel, PIC, Analog Devices,
Freescale (Motorola), Cypress and Mitsubishi. The final microcontroller selection was
weighted and based on the following ordered requirements:
• USB Hardware Facilities
• I/O Pin Count and Functionality
• Availability of Driver Software Libraries
• Rich Programming Environment & Infrastructure
• Product Availability
• Package Configuration
• Cost
• Low-Power
The Freescale (Motorola) 8-bit HC08 based MC68HC908JB16JDW

microcontroller was selected for final design implementation due to built-in USB module,
highly-flexible digital I/O, vast software library availability, ease of programming and
support, surface-mount 28-pin package, extremely low power and cost.
Peripheral Component Selection

One of the most important aspects of Dial design simplification was the selection of
specific peripheral components. These peripheral components include the following:
• Integrated low-cost miniature joystick, pushbutton and quadrature
encoder device
• Vertical and horizontal miniature tactile pushbutton
• Super-bright SMD LED’s
Passive RC Filter Elements

Passive RC filter elements were included between all switch, quadrature-encoder and
microcontroller inputs. These passive elements eliminate electrical noise generated from
mechanical switch contact-bounce, prevent EMI/RFI injection, protect microcontroller
70
CMOS digital inputs from static-discharge damage and reduce software overhead
associated with signal conditioning. Filter elements are implemented using discrete SMD
components and include nominal time constants of 1.0ms.
4.5.1.3 Hot Swap Controller
In order to facilitate fast design and implementation cycle, a direct to PCB approach was
taken. This approach eliminates the typical ‘breadboard’ engineering prototype cycle
using large pinned devices, and instead implements a prototype PCB based on final
design component selection and footprint requirements. Direct to PCB prototyping is an
effective and efficient engineering methodology that includes the following benefits:
• Forces thorough and competent design practices in schematic-entry
phase
• Matures detailed design in schematic-entry phase
• Allows use of single set of procured components
• Prototypes circuit, mechanical and PCB layout designs in single
engineering phase
• Allows analysis and testing of matured design in single engineering
phase
• Lowers risk of design anomalies
The direct to PCB engineering approach often requires only a single revision cycle to
produce error-free circuit design and PCB implementations.
Component Selection
The use of miniature surface-mount (SMD) electronic components was pursued early in
the engineering design process. SMD components offer the following benefits:
• Extremely space efficient
• Readily available (industry standard)
• Low-cost
• Easy to hand solder
• Facilitate low-noise characteristics
• Allow efficient miniature PCB layout
• Lower cost of PCB production and assembly
Microcontroller Selection
Various microcontrollers were considered for use for Hot-Swap Controller
implementation. These included microcontrollers from various manufacturers; Atmel,
PIC, Analog Devices, Freescale (Motorola), Cypress and Mitsubishi. The final
microcontroller selection was weighted and based on the following ordered requirements:
• I/O Pin Count and Functionality
• Rich Programming Environment & Infrastructure
• Product Availability
• Package Configuration
• Cost
• Low-Power
71
The Freescale (Motorola) 8-bit HC08 based MC68HC908SR12CFA microcontroller was

selected for final design implementation due to highly-flexible digital I/O, vast software
library availability, ease of programming and support, surface-mount 48-pin package,
extremely low power and cost.
Circuit Protection Elements

Due to the high-potential for electrostatic-shock damage and EMI/RFI injection
associated with a pluggable device like the Hot-Swap Controller, passive circuit
protection elements were included on all electrical interfaces to external components.
Protective elements are based on simple series resistor and fast shunt-zener diode
combinations. High current paths are protected with shunt-zener diode elements only.
Modular Design Approach

Hot Swap Controllers are designed in a manner consistent with flexibility and
modularity. A given system implementation may serially cascade as many Hot-Swap
Controller nodes as desired to extend battery capacity. Each Hot-Swap Controller is
identical in hardware design and includes interface connectors to enable serial
interconnectivity between nodes, primary load and charging facilities.
Low Power Mode

Hot Swap Controller electronics design includes provisions to enter standby low-power
mode under microcontroller command during periods when controller is not the primary
power source. In this mode the Hot Swap Controller draws only micro-amps of battery
current effectively preventing primary battery drain during idle periods. Hot Swap
Controllers share a common interrupt request signal that may be used to wake-up
controller from a standby state.
Interface Compatibility
Hot Swap Controllers are designed for direct interconnection between the Vaio PC and
the primary battery pack. In order to achieve this requirement, standard battery connector
interfaces are included for both Vaio and battery pack interface points. The standard
battery connectors are manufactured by Tyco Electronics (formerly Amp Inc.), and are
strategically located to enable efficient sandwiching between PC and battery pack.
4.5.2 Lessons Learned

The main challenge faced by the HCI group throughout the project is devising an
application interface that would work along with a custom input device (the dial). Many
parameters had to be considered in great detail for the design, this includes: real estate of
the screen, easy accessibility of the buttons and most importantly the location and manner
of triggering the buttons such that they are intuitive to users when interacting with our
custom software interface.
72
The main lesson learned throughout the project is the lack of user study tests which are
crucial to a good UI design. User testing is particularly important when one is trying to
design a customized system for a very small target of end users. Unfortunately, we had
no access to aircraft maintenance workers nor did we have hands on experience with
tasks that they would perform. Much of the design was carried out based on the intuition
of the group members however the access to user subjects and perhaps even test
environment under which the prototype would be deployed would greatly help in creating
a better design.
Throughout the project, in particular for Phase II and III, the HCI team had a difficult
time in identifying our roles as HCI. Although each the members were very involved
with their assigned sub-system, many of us were unsure of our role as HCI members.
Perhaps, this can be clarified for future classes so
The experience with coming up with a script was also somewhat challenging. For MoRÉ,
the system was not fully integrated to have a detailed draft of the script at an earlier stage.
Many details in the script that were planned for the demo had to be revised to adapt to the
working version of the system. Perhaps for future classes, scripting the demo could
closely related to integration and become a simultaneous process running parallel with
integration.
Many lessons were learned the hard way in Phase III by the Capturing Experience Group.
Assuming components would be integrated easily was a bad assumption make. Each of
the components worked individually, however after the initial integration a lot of
problems arose. Also, Phase III showed how important testing is. Whether it is airplanes
or software, testing and observing expected results is the only way to know whether a
prototype works. No matter how careful one is, inevitably there will be bugs or other
flaws.
Something that was needed but never done was to list issues that arose of each module.
Many problems were overlooked and forgotten. More specifically, when the recording
module was being implemented in Java in Phase III, part of the code necessarily required
the hardware to be able to record using a set of configurations. Since getting this to work
on a personal machine did not require a great amount of tweaking, because settings,
configurations, and hardware setup could be modified freely, it was forgotten until Phase
III, when it became a problem. In sum, it was learned that it is important to record things
that may be a problem in the future, and also if there are things that are dependent on an
environment, which may be different from the one it is being implemented and tested on.
Also, better coordination between different groups is key to a successful prototype. For
example, it was/is important to make sure everyone is not only using compatible versions
and also for all modules that had a dependency on another group’s module to understand
what will be inputted or outputted from each module. It is definitely something to watch
out for in the future when it may be much harder to transition.
73
One of the hardest-hitting lessons learned by the Platform group was that it turned out to
be basically impossible to write USB firmware from scratch, so it is important to choose
microcontrollers and chips that have available, working code bases. This wasn’t factored
in when the electrical engineering portions of the Dial and Hot-Swap Belt were done, and
almost became a critical issue with the system as there wasn’t enough time to write code
from scratch.
Even with months of planning and scrutinizing of plans in the project, it is really amazing
how a single minor problem can raise the possibility of disaster. The problems that the
platform team found in getting the dial to work as planned were so simple, yet required
so much effort to overcome. Although there was code readily available for our PCB
board, it took a great deal of time to locate it and to edit and conform it to our purposes.
And it took a number of our team members to complete the task. This hindered us in
another aspect of the project, which was the programming of the hot-swap controllers. So
a lesson that can be learned from this is that the proper research needs to be allocated as
soon as possible to clarifying assumptions made in the first few steps of the project. And
if at all possible, this research (in this case, the locating of proper code) should come even
before the creation of designs and plans of specific components. Of course, it was
difficult to do so in such a short period of time, but it is nevertheless the biggest lesson
that I have taken away from the class.
Another lesson is that project engineering design and implementation approach

methodologies were highly-successful with realization of revision A PCB hardware. Both
Dial and Hot-Swap controller electronics designs, PCB-layout and component assembly
phases were completed within schedule and functioned as per design specification
without circuit revisions. Fundamentally the fast-prototype direct to PCB approach was
reinforced as a viable and efficient means to design and implement custom electronics for
rapid-prototype projects.
5 Project Management
5.1 Implementation and Integration Phase Results
5.1.1 Summary of Logbook Hours
Table 3: HCI Group Phase 3 Worklog

Task Dan Yong Brian Anand Wen Shu Aashni Total
Administrative Tasks 1:00 1:00 2:00 5.00
Class 3:30 15:00 16:00 11:00 17:00 4:00 66.30
Coding 13:00 37:00 21:30 5:00 41.00
Group Meeting 7:30 12:00 1:30 11:10 12:00 23:30 67.00
Research 30:15 4:30 4:00 38.45
Design Engineering 39:00 22:30 61.30
74
User Interface Design

User Testing 6:00 6.00
Preparing Specs 5:30 3:00 8.30
Purchasing Activities 6:00 6.00
Writing 11:00 10:30 13:00 24:00 11:00 92.30
Reports/Presentations 23.00
Staff/Leaders Meeting 0:45 2:30 3.15
Architecture Design
Working with client 1:00 1.00
Total 54:00 92:30 66:00 92:10 64:00 52:00
Table 4: Search Group Phase 3 Worklog

Task Melanie Sachin Kevin Tabreeze Total
Administrative Tasks 8:00 - 8:00
Architecture Design - 1:30 1:30
Class 11:30 12:00 23:30
Coding 37:00 26:30 63:30
Data Migration and Restoration 8:00 8:00 16:00
Design Engineering - 11:00 11:00
Field Testing - - -
Group Meeting 10:00 4:30 14:30
Preparing Specs - 3:30 3:30
Purchasing Activities - - -
Research - 6:00 6:00
Staff/Leaders Meeting - - -
User Interface Design - - -
Writing Reports/Presentations 17:00 5:00 22:00
Total 91:30 78:00 169:30
Table 5: Capturing Experience Group Phase 3 Worklog

Ha, Jae Mai, Leon Tran, Jonathan
Administrative Tasks - - 3:45
Architecture Design - - -
Class - - 5:30
Coding 74:30 74:25 48:20
Design Engineering - 1:30 -
Field Testing - 7:00
Group Meeting - - 8:35
Preparing Specs - - -
Research - - -
Staff/Leaders Meeting - 1:25 10:00
75
User Interface Design - - -
Writing 10:00 6:30 14:45

Reports/Presentations
Total 84:30 83:50 87:40
Table 6: Platform Group Phase 3 Worklog

Task Adam Megan Zack Mohammad Bryon Total
Administrative Tasks 4:00 1:15 - - - 5:15
Architecture Design 1:00 - - 3:00 - 4:00
Class 13:40 14:00 24:00 12:50 9:00 73:30
Coding 31:00 18:00 - 34:00 - 83:00
Design Engineering 9:00 - 12:30 - 12:00 33:30
Field Testing - 2:00 - - 2:00
Group Meeting 5:30 4:00 10:00 3:00 4:00 26:30
Preparing Specs - - - -
Purchasing Activities - 12:00 - - 12:00
Research 3:00 3:30 12:15 3:00 10:30 32:15
Staff/Leaders 2:00 - - - -
Meeting
Working with client 1:00 - 0:30 - 1:30
Writing 23:00 18:30 4:00 - 11:00 57:30
Total 93:10 60:15 76:45 56:20 47:30 323:50
5.2 Summary of the Entire Project

5.2.1 Task Dependency Chart
The Task Dependency Chart shows the different tasks allocated per group along the
timeline of Phases 2 and 3. It has been added to the Appendix Section of the report. The
groups tried their best to stick to the Chart’s timeline as it was possible. One major
change in the chart is the time allotted for integration, which was greatly shortened due to
time constraints in preparing for the final presentation.
5.2.2 Summary of Logbook Hours

Table 7: HCI Group Semester Worklog
Task Dan Yong Brian Anand Wen Aashni Total
Shu
Administrative Tasks 1:30 - 4:30 2:00 0:30 - 8:30
Class 20:45 40:10 38:30 26:30 44:00 23:30 192.45
Coding - - 33:30 37:00 33:00 7:00 110.30
Data Migration & - - - 4:00 - - 4:00
76
Restoration
Field Testing - - - 5:00 - - 5:00
Group Meeting 37:30 42:00 18:30 30:10 31:00 23:30 182.00
Research 21:45 14:30 - - 1:00 7:00 43.75
Design Engineering - 39:00 22:30 - - - 61.3
User Interface Design - 11:00 - - 4:30 - 15.30
Preparing Specs - 9:30 1:00 3:00 - - 13.30
Purchasing Activities - 6:00 - - - - 6.00
Writing 38:45 31:30 39:30 39:30 26:00 206.35
Reports/Presentations 34:00
Staff/Leaders Meeting 0:45 1:00 1:00 4:00 7:00 - 13.45
Architecture Design - - - 2:00 2:30 - 4.30
User Testing 10:00 - - 6:00 2:00 - 18.00
Working with client - - 1:00 - - 1:00 2.00
Total 131:00 194:40 163:00 172:40 151:30 96:00
Table 8: Search Group Semester Worklog

Task Melanie Sachin Kevin Tabreeze Total
Administrative Tasks 8:00 4:05 - 12:05
Architecture Design - 4:00 1:30 5:30
Class 35:35 30:40 26:40 92:55
Coding 62:00 28:50 54:30 145:20
Data Migration and 8:00 - 8:00 16:00
Restoration
Design Engineering - 6:15 11:00 17:15
Field Testing - - - -
Group Meeting 21:30 16:45 18:30 56:45
Preparing Specs - 1:05 3:30 4:35
Research 4:30 14:40 34:00 53:10
Staff/Leaders 5:30 2:45 - 8:15
Meeting
User Interface 1:00 - - 1:00
Design
Writing 26:00 20:00 22:30 68:30
77
Total 166:35 129:05 180:10 475:50
Table 9: Capturing Experience Group Semester Worklog

Ha, Jae Mai, Leon Tran, Jonathan
Administrative Tasks 4:15 1:00 15:30
Architecture Design - - 0:45
Class 13:40 20:10 22:40
Coding 116:30 105:25 55:45
Design Engineering 4:00 1:30 2:30
Field Testing - - 7:00
Group Meeting 23:30 19:25 44:00
Preparing Specs 11:30 - 1:50
Research 23:00 24:00 2:05
Staff/Leaders Meeting 3:45 2:25 8:20
User Interface Design - - 0:20
Writing 41:35 29:25 35:35
Total 241:45 203:20 196:20
Table 10: Platform Group Semester Worklog

Task Adam Megan Zack Mohammad Bryon Total
Administrative Tasks 23:45 8:55 6:00 3:30 - 42:10
Architecture Design 1:00 - - 3:00 - 4:00
Class 35:55 39:45 54:00 35:15 35:00 199:55
Coding 31:00 23:30 - 45:50 10:00 110:20
Design Engineering 9:00 - 28:00 - 73:20 110:20
Field Testing - - 22:00 - - 22:00
Group Meeting 15:35 12:10 13:00 11:45 19:00 71:30
Preparing Specs 1:00 - 2:30 - 1:00 4:30
Purchasing Activities 0:30 - 12:00 - 3:00 15:30
Research 15:45 16:15 22:00 21:35 48:30 124:05
Staff/Leaders 5:30 - 1:00 - 0:05 6:35
Meeting
Working with client - 1:00 - 0:30 - 1:30
Writing 35:00 38:00 14:15 17:30 33:00 127:45
Total 174:00 139:35 174:45 138:55 222:55 851:10
78
5.3 Suggestions for Improving the Class

To improve RPCS, we would suggest emphasizing very early in the course how
important it is to look down the road at every step – as a group, we wasted a lot of time
choosing one microcontroller when there was much more code available for another from
a nearby grad student. This could have saved us a ton of time and headaches
programming. The main issue here is that the amount of programming required and the
difficulty of implementation was not considered during the hardware implementation
phase, because students have little experience in completing end-to-end projects such as
in this course. Instructors and Teaching Assistants should encourage students to begin
software research throughout the hardware design phase so that these considerations are
taken into account before the design is complete and it is too late. Fortunately, the group
was able to devote enough time and research into finding base code and developing a
new implementation. Other groups may not be so fortunate if not guided early.
79
6 Glossary of Terms
MoRÉ – Mobile Reference Environment
IETM - Interactive Electronic Training Manual
DAS – Data Acquisition System
Vaio – Sony Vaio Mobile Computer
HID – Human Interface Device
ICP – In-circuit programming
80
7 Appendix
7.1 Hardware Specification Tables
Tab le 11: MC908 JB 16DW E Microco ntro ller P in Fu nct ion & De scr ip tio n
Microcontroller Legend Description Configuration Function
Pin
1 VSS Microcontroller N/A Tied to PCB GND

negative supply
potential
2 OSC1 External oscillator N/A Used with external 12MHz
input crystal and related termination
elements
3 OSC2 External oscillator N/A Used with external 12MHz
output crystal and related termination
elements
4 VREG Microprocessor N/A Used to bias USB D- signal for
3.3VDC regulated USB low-speed interface host
output detection
5 VDD Microcontroller N/A Tied to PCB +5VDC
positive supply
potential
6 PTD0 STAT1 Output Red LED drive. Drive this pin
low to turn on red LED. Note
LED drive port option must be
configured
7 PTD1 STAT2 Output Blue LED drive. Drive this pin
low to turn on blue LED. Note
LED drive port option must be
configured
8 PTD2 ENCA Input: Rotational encoder quadrature A
Note active pull- signal. This signal is routed to
up resistor must be both PTD2 for level detection
enabled and PTE1 for interrupt detection
9 PTD3 ENCB Input: Rotational encoder quadrature B
10 PTD4 Not Used Input This pin must be configured as
input
11 PTE1 ENCA Input: Rotational encoder quadrature A
12 PTE3 D+ I/O USB Data+ differential
communications data
13 PTE4 D- I/O USB Data- differential
communications data
14 PTC0 TxD Output CMOS level RS-232 Transmit
15 /IRQ Interrupt Request Input –active low Not Used
16 PTC1 RxD Input CMOS level RS-232 Receive
81
input
18 PTA7 LF Input: Dial left user pushbutton. This
Note active pull- pin is normally high and driven
up resistor must be low when the user right
enabled pushbutton is depressed
19 PTA6 MID Input: Dial middle user pushbutton.
Note active pull- This pin is normally high and
up resistor must be driven low when the user
enabled middle pushbutton is depressed
20 PTA5 RT Input: Dial right user pushbutton. This
Note active pull- pin is normally high and driven
up resistor must be low when the user right
enabled pushbutton is depressed
21 PTA4 PUSH Input: Dial right joystick depression
Note active pull- pushbutton. This pin is normally
up resistor must be high and driven low when the
enabled user joystick pushbutton is
depressed
22 PTE2 ENCB Input: Rotational encoder quadrature B
23 PTE0 Not Used Input This pin must be configured as
input
24 PTA3 DIRD Input: Joystick direction D input. This
Note active pull- pin is normally high and is
enabled joystick is depressed in the D
direction
25 PTA2 DIRC Input: Joystick direction C input. This
enabled joystick is depressed in the C
direction
26 PTA1 DIRB Input: Joystick direction B input. This
enabled joystick is depressed in the B
direction
27 PTA0 DIRA Input: Joystick direction A input. This
enabled joystick is depressed in the A
direction
28 /RST Microcontroller I/O – active low This pin is connected to an
Reset external momentary pushbutton
that generates an active-low
microcontroller reset when
depressed
82
Tab le 12: J1 & J2 S CI and USB In terf ace

Molex Part # 53048-0410
J2 USB Interface
Pin Number Signal Legend Signal Description
1 +5VDC USB Power
2 D- Data +
3 D+ Data -
4 GND USB Ground
Molex Part # 53048-0410

J1 SCI Interface
1 +5VDC SCI Power
2 RxD TTL Level Receive Data
3 TxD TTL Level Transmit Data
4 GND SCI Ground
Tab le 13: MC68 H C90 8SR 12 Microcon tro ller P in Func tio n & De script ion
Microcontroller Legend Description Configuration Function
Pin
1 PTC3 Not Used Input This pin must be configured as

input with active pullup disabled to
prevent power draw
2 NC No Connection No Connection
3 PTD0 COM_SW Output High logic level on this pin enables
communication between Vaio PC
and battery pack. Low logic level
disables communication between
Vaio PC and battery pack
4 VDD Microcontroller N/A Tied to PCB +3.3VDC Voltage
positive supply Regulator. This voltage is
potential regulated from the common system
power bus and is present whenever
a battery pack (from any HSC) is
present in system
5 OSC1 External N/A Implemented with external 32KHz
oscillator input crystal and related termination
elements
6 OSC2 External N/A Implemented with external 32KHz
oscillator output crystal and related termination
elements
7 VSS Microcontroller N/A Tied to PCB GND
negative supply
potential
8 PTD1 PWR_SW Output High logic level on this pin enables
battery power between Vaio PC
and battery pack. Low logic level
disables battery power between
Vaio PC and battery pack
9 /IRQ1 /IRQ Input System-wide interrupt request.
This pin is connected between all
HSC devices in system and driven
low when any HSC IRQ_EN
83
(PTD2) is driven high.

10 PTD2 IRQ_EN Output High logic level on this pin
initiates system-wide /IRQ to all
connected HSC devices. Logic low
on this pin disables all /IRQ
interrupts. This pin should only be
pulsed high for short durations
when an /IRQ is to be initiated
system-wide.
11 /RST Microcontroller I/O – active This pin is connected to an external
Reset low momentary pushbutton that
generates an active-low
microcontroller reset when
depressed
12 PTD3 /BAT_ON Output Low logic level on this pin turns on
battery pack connected to HSC.
High logic level on this pin turns
off battery pack connected to HSC.
This function is required when
battery pack and HSC are not
currently supplying Vaio PC with
power. For minimum power draw
from idle pack.
13 SDA0 SDA0 I/O SMB Serial Data pin channel 0.
This pin is connected to the battery
pack SDA communications pin and
used to communicate with battery
or monitor SMB communications
between Vaio and battery pack.
14 SCL0 SCL0 I/O SMB Serial Clock pin channel 0.
This pin is connected to the battery
pack SCL communications pin and
used to communicate with battery
or monitor SMB communications
between Vaio and battery pack.
15 SDA1/TxD Tx/SDA1 I/O This pin is dual function: RS-232
TxD when a valid RS-232 voltage
level is detected at the RS-232
RxD connector pin. And SMB port
SDA otherwise (SMB
communications between HSC
devices). Note PTD7 input (RDY)
signal is high logic level when
valid RS-232 voltage level is
detected at the RS-232 RxD
connector pin.
16 SCL1/RxD Rx/SCL1 I/O This pin is dual function: RS-232
RxD when a valid RS-232 voltage
level is detected at the RS-232
RxD connector pin. And SMB port
SCL otherwise (SMB
communications between HSC
devices). Note PTD7 input (RDY)
signal is high logic level when
84

connector pin.
input
input
19 PTD6 VMON_SW Output High logic level on this pin enables
battery voltage and PC voltage
monitors at ATD3 & ATD4 ADC
pins. Low logic level on this pin
disables battery voltage and PC
voltage monitors at ATD3 &
ATD4 ADC pins.
input
input
input
23 PTA7 Not Used Input This pin must be configured as
input
25 PTD7 RDY Input Logic high on RDY pin indicates
connector pin. SDA1/TxD and
SCL1/RxD mode is determined by
logic level on RDY pin. With RDY
high: RS-232 mode should be
invoked, otherwise USB mode
should be configured.
input
27 PTB6 Not Used Input This pin must be configured as
input
input
input
30 OPIN1 BATT_AMP Input This pin is used by the
microcontroller internal amplifier
to monitor voltage across the
battery current sense resistor for
detection of battery charge and
discharge current
31 VSSAM GND NA Analog Module Ground Reference
input
input
34 ATD1 Not Used Input This pin must be configured as
input
35 VREFL VREFL Input ADC VREF Low reference.
Connected to GND
36 VREFH VREFH Input ADC VREF High reference.
85
Connected to VDD via RC LPF

37 ATD3 PC_V Analog Input Vaio PC supply voltage monitor.
This pin reflects scaled voltage
supplied to PC independent from
battery pack voltage. Note PTD6
(VMON_SW) must be high logic
level to enable analog voltage feed
to this pin. ADT3 Voltage =
Voltage at PC * 0.4
38 ATD4 BAT_V Analog Input Battery pack supply voltage
monitor. This pin reflects scaled
voltage supplied from battery pack
connected to HSC independent
from Vaio PC voltage. Note PTD6
(VMON_SW) must be high logic
level to enable analog voltage feed
to this pin. ADT4 Voltage =
Battery V * 0.4
input
40 PTA4 /LED_RED Output Red LED drive. Drive this pin low
to turn on red LED. Note LED
drive port option must be
configured
42 VSSA VSSA Analog Ground Reference.
Connected to PCB ground
43 VDDA VDDA Analog supply voltage. Connected
to regulated +3.3VDC via RC LPF.
input
input
input
47 PTA5 /LED_BLU Output Blue LED drive. Drive this pin low
to turn on blue LED. Note LED
drive port option must be
configured
48 CGMXFC CGMXFC NA External filter interface pin
Tab le 14: J4 S CI I nt erface

Molex Part # 53048-0410
J4 RS-232 SCI Interface
1 No Connection No Connection
2 RxD RS-232 Receive Data
3 TxD RS-232 Transmit Data
4 GND RS-232Ground
86
Tab le 15: Pr imary Con nector Pinout s

Vaio U PC Connector J1 Interface Molex: 6-1447143-0
Pin Legend Type Description
1 GND_PC NA Battery Negative Potential
2 /EN_PC I/O Battery Pack Enable Pin. Ground this pin through 1K to turn on battery
pack
3 /OK_PC I/O This pin is low when battery pack fuses are OK
4 SCL0_PC I/O Battery pack & PC SMB communications Serial Clock pin
5 SDA0_PC I/O Battery pack & PC SMB communications Serial Data pin
6 VBATT_PC NA Switched Battery Positive Potential
Inter-Board Interface Connectors J5 & J6 Molex: 22-05-7105

1 /EN_PC I/O Battery Pack Enable Pin. Ground this pin through 1K to turn on battery
pack
2 /OK_PC I/O This pin is low when battery pack fuses are OK
3 SCL0_PC I/O Battery pack & PC SMB communications Serial Clock pin
4 SDA0_PC I/O Battery pack & PC SMB communications Serial Data pin
5 SDA1 I/O Inter-Module SMB communications Serial Clock pin
6 SCL1 I/O Inter-Module SMB communications Serial Data pin
7 /IRQ_BUS I/O Inter-Module Interrupt Request Bus
8 GND NA Battery Negative Potential
9 VBATT_PC NA Switched Battery Positive Potential
10 PWR_BUS NA Inter-Module Power Bus
Battery Pack Connector J3 Interface Molex: 6-1447143-9

1 GND_BATT NA Battery Negative Potential
2 /EN_ BATT I/O Battery Pack Enable Pin. Ground this pin through 1K to turn on battery
pack
3 /OK_ BATT I/O This pin is low when battery pack fuses are OK
4 SCL0_ BATT I/O Battery pack & PC SMB communications Serial Clock pin
5 SDA0_ BATT I/O Battery pack & PC SMB communications Serial Data pin
6 VBATT_ NA Switched Battery Positive Potential
BATT
87
7.2 Task Dependency Chart

Figure 25: Task Dependency Chart
88
7.3 List of Figures

Figure 1: Discrepancy Review Screen in Debrief System................................................ 8
Figure 2: Reference Manual Explanation ........................................................................ 9
Figure 3: Training Manual Explanation..........................................................................10
Figure 4: User Interface .................................................................................................22
Figure 5: Dial Prototype.................................................................................................24
Figure 6: MoRE Web Browser Interface........................................................................27
Figure 7: Search Functionality .......................................................................................28
Figure 8: Graph of Frequency of Words to Number of Words with Given Frequency ....30
Figure 9: Software Architecture.....................................................................................33
Figure 10: Diagram of Software Architecture for Capturing Experience ........................43
Figure 11: Screenshot of a Training Manual Page with Annotations ..............................44
Figure 12: Diagram of How Cycling Detection stores the URLs ....................................46
Figure 13: Screenshot of a Training Manual Page with Smart Links ..............................48
Figure 14: Dial Final Product.........................................................................................52
Figure 15: Vaio Ruggedized Casing...............................................................................53
Figure 16: Hardware System Architecture .....................................................................60
Figure 17: Dial Firmware Architecture ..........................................................................61
Figure 18: Integrated HMD and Headphones .................................................................62
Figure 19: Vaio Holster .................................................................................................62
Figure 20: Dial Harness .................................................................................................63
Figure 21: HMD and Headphones - Before and After Views..........................................64
Figure 22: Dial PCB Component-side and Solder-side Views ........................................66
Figure 23: Hot-Swap PCB Component-side and Solder-side Views ...............................66
Figure 24: Proposed Hot-Swap Controller Design..........................................................67
Figure 25: Task Dependency Chart ................................................................................87
7.4 List of Tables

Table 1: Selected Technologies......................................................................................12
Table 2: Test Results for Indexer Tests ..........................................................................29
Table 3: HCI Group Phase 3 Worklog ...........................................................................73
Table 4: Search Group Phase 3 Worklog........................................................................74
Table 5: Capturing Experience Group Phase 3 Worklog ................................................74
Table 6: Platform Group Phase 3 Worklog.....................................................................75
Table 7: HCI Group Semester Worklog .........................................................................75
Table 8: Search Group Semester Worklog .....................................................................76
Table 9: Capturing Experience Group Semester Worklog ..............................................77
Table 10: Platform Group Semester Worklog ................................................................77
Table 11: MC908JB16DWE Microcontroller Pin Function & Description .....................80
Table 12: J1 & J2 SCI and USB Interface ......................................................................82
Table 13: MC68HC908SR12 Microcontroller Pin Function & Description ....................82
Table 14: J4 SCI Interface .............................................................................................85
Table 15: Primary Connector Pinouts.............................................................................86
89

MoRE Phase III Final Report

Transféré par

Informations du document

Description originale:

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

MoRE Phase III Final Report

Transféré par

Droits d'auteur :

Formats disponibles

MoRÉ: Mobile Reference Environment

Rapid Prototyping, Spring 2006

Capturing Experience Group

3 System Tutorial and Usage Scenario .................................................................15

4 Implementation and Integration........................................................................22

4.4.4 Software Architecture.............................................................................60

5 Project Management ..........................................................................................73

1.3 Group Members

Experience Capture Team

Human-Computer Interaction Team

Figure 1: Discrepancy Review Screen in Debrief System

Figure 2: Reference Manual Explanation

Figure 3: Training Manual Explanation

2.1.2 Key System Requirements

2.2 Initial Solution Concepts

Table 1: Selected Technologies

2.2.2 Visionary Scenario

2.3 Conceptual Design

2.3.2 Product Design Specifications

2.3.4 Experience Capture

Further detail on all these modules is provided in the Functionality section.

3 System Tutorial and Usage Scenario

dynamic content assumes the existence of a keyboard or similar technology; if the

3.2 Usage Scenario

1. Scrolling Between Links:

3. Switching Between Panes:

4. Bringing Up Links in New Tab:

5. Switching between the Tabs:

6. Using the Mouse cursor (optional):

7. Using the windows left Mouse Click (optional):

4 Implementation and Integration

4.1.1.1 User Interface

Figure 4: User Interface

4.1.1.2 The Dial

Figure 5: Dial Prototype

4.1.2 User Studies Results or Walkthroughs

Figure 6: MoRE Web Browser Interface

Figure 7: Search Functionality

4.2.2 Experimental Measurements

The second method of testing is performed by looking at how browsers currently

4.2.2.1 Training Manual Statistics Provided by Indexer

The Indexer successfully captured a multitude of information regarding the training

Table 2: Test Results for Indexer Tests

Number of Words that Appear on 10 Pages or Less 2,578

Frequency vs. Number of Words with Frequency

Amount of words with given frequency in

Figure 8: Graph of Frequency of Words to Number of

4.2.2.2 Keywords Results Provided by Indexer

4.2.2.3 Browser Feature Testing Results

4.2.2.4 Renderer Feasibility Tests

4.2.3 Software Architecture

Figure 9: Software Architecture

4.2.4 Software Modules and Status

4.2.4.2 Search Engine:

Separate considerations are given depending on whether the search query/string is a

The proxy has been divided into three broad parts:

4.2.4.6 Customized browser:

Due to unconventional requirements, a customized browser was needed. It takes input

4.3 Experience Capture

The goal of Capturing Experience Group is to capture the experience of aircraft

4.3.2 User Studies Results

4.3.3 Experimental Measurements

4.3.4 Software Architecture