Vous êtes sur la page 1sur 6

Indoor Navigation using AR in Warehouses

CS 4605 -- Project 20: Report

Sarthak Srinivas ​Zack Littke-Smith
ssrinivas32@gatech.edu zls3@gatech.edu

Sarah Marland Daniel Marcos

smarland3@gatech.edu danielmarcos@gatech.edu

ABSTRACT Order picking in warehouses is currently

Order picking is the process of finding and dominated by paper-based systems, which are
picking items inside a warehouse. This process clunky, unwieldy, and largely ignorant of
accounts for up to 55% of the operational cost technological advancements. Simply put, it can
of a warehouse1. Travel between different be improved. A more technologically-advanced
packages further accounts for 55% of order system could improve picking times in a
picking costs2. Despite variance in how densely low-density environment (an environment
packed orders are between different which has picks spread relatively thinly)
warehouses (referred to as pick density in greatly, consequently reducing a great deal of
industry), most warehouses still use old the running costs to warehouses by making
systems such as static maps and paper to pick every order-picking employee much more
orders. A more performant order-picking efficient. Currently, entirely-robotic systems
process could optimize this large warehouse don’t have enough dexterity or adaptability to
cost, saving employee time and company accommodate warehouse order picking, and
money. The goal of this project was to likely won’t for the next few decades. This
determine whether heads-up displays on leaves a large gap of time in which the
augmented reality headsets could decrease obsoleteness of current paper-based systems
order picking time or increase accuracy in will becoming increasingly glaring. One
low-density picking environments, thus available solution is the use of augmented
reducing travel time. We developed AR reality (AR) to make order picking more
software on Google Glass to create a visual portable and dynamic. The aim of this study
and intuitive order-picking experience, and was to develop a reasonable AR system and
tested it with books at the Georgia Tech study its effectiveness against a paper-based
library. We found that using AR headsets like system in order picking. More specifically, the
Google Glass reduced the time taken to pick goal was to determine if the use of an AR head
books by 38.7%, on average. They are also display could increase the speed of order
easier to use and more comfortable than paper picking or increase the accuracy of order
pick lists, and our participants said that they picking in low-density picking environments. In
would prefer to use AR headsets if they ever order to determine a solution to this problem,
had to pick items in the future. our process included the following:
1. Create a test environment to
INTRODUCTION accurately simulate a warehouse
order-picking scenario.
2. Create a minimalistic user interface to
​Bartholdi, John J., and Steven T. Hackman. ​Warehouse & visualize order picking with the goal of
Distribution Science: Release 0.98.​ Supply Chain and minimizing the amount of time spent
Logistics Institute, 2017.
​Bartholdi, John J., and Steven T. Hackman. ​Warehouse & on the task.
Distribution Science: Release 0.98​. Supply Chain and
Logistics Institute, 2017.

3. Randomly assign participants to a low density picking environments). We
paper-based or AR order picking implemented an HUD on Google Glass, and
condition and measure the average tested it against a paper-based pick list in the
amount of time each group takes to Georgia Tech library.
complete a series of picks.
4. Analyze the user data and determine Our first step was defining a warehouse-like
the effectiveness of an AR picking environment we could use to test our product
system against a paper-based one. and run studies. We chose the science-fiction
section of the Georgia Tech library as it was
BACKGROUND large enough for picking objects that were far
Baumann [1] demonstrated it is possible to away from each other, yet feasible for
improve order picking speed by up to 37% by implementation and research. We wrote a
using a head mounted display to display visual web-scraper to collect book data from the
cues for order picking]. By utilizing Google library’s website, and manually mapped each
Glass, our application aimed to improve the book to its physical location. However, we ran
speed of order picking in warehouses with a into the problem of inconsistencies between
head-worn display. ​Our user interface took scraped data from the online library catalog
small inspiration from the ones described by and real books inside the library. To address
Schwerdtfeger & Klinker [2] and Guo et al. [6], this, we ended up manually scanning 1 book
who described two possible interfaces to per shelf in the science fiction section of the
display visual cues for low density picking library, or about 288 books total. This became
environments using Augmented Reality. our master database of books in the library.
Bartholdi & Hackman [3] provided details
about costs associated with order picking in a With our test environment and data prepared,
warehouse, which helped us understand the we moved on to designing Pick by Paper and
value of a better solution that we hoped to Pick by AR. For Pick by Paper, we created a
create and test. Haynes [4] described the pick list by exporting 33 random books from
optimal placement of head-mounted displays our master database into a CSV file resembling
to ensure optimal comfort for wearers, Figure 1 below.
especially for industrial tasks.

Our interface also took small inspiration from

Weaver et al. [5], who described and analyzed
various interfaces for picking in high density
picking environments, as well as sources 7 to
11 which provided additional background
information on the same topic. All of these
studies together helped form our
understanding of the problem at hand, how an
interface for an effective solution might look,
how a significant reduction in order-picking
time could be achieved, and finally how to test
that solution in a realistic environment.

Our goal was to determine if the presence of
visual information for routing on a head-up AR
display decreases the amount of time it takes a
human to pick orders in a warehouse or
warehouse-lime environment (specifically in
Figure 1: Example paper pick list

ecologically valid. Finally, we found that we
For Pick by AR, we first investigated what the had trouble reading the titles of books for
right approach would be for picking books smaller volumes, and that we had to pull out
inside the library. We gathered feedback and many books in a shelf to actually read the book
iterated over several possible user interfaces title and author. Therefore, we repicked most
(UI) to find a design that showed useful of our books to be large hard-bound books
information without being too busy. We whose titles and authors could be read on the
evaluated the user experience (UX) to make spine of the book itself.
the interaction intuitive: our final design
required a single tap to find items in their Once we worked out most of the kinks in our
locations, and another to move to the next application, we deployed the app to Glass
pick. devices and printed CSV files to form paper
pick lists. We then conducted a user study to
After finalizing our UI and UX, we implemented compare our solution to traditional paper pick
an order-picking Android app for Google Glass. lists.
We first created a real-time database on
Firebase to host the book catalogue and RESULTS
experimental data. Then, we developed the To compare Pick by Paper with Pick by AR, the
Glass app using Java and the Android and team conducted a lab-based user study with 6
Glass APIs. Like Pick by Paper, we ensured that participants. ​Before the user study the
the pick list for Google Glass also consisted of participant was familiarized with the layout of
33 books. the science fiction section of the library. The
participant was also told to simply slide the
book in and out of the library shelf, since we
saw no benefit from actually picking out then
replacing the book. Finally, we explained how
instructions would be delivered to the
participant from the AR headset or the paper
pick list. During the user study the participant
Figure 2: The two views of our final UI for Pick would first pick using the paper pick list, get a
by AR five minute break, then pick from the AR
headset. Besides timing Pick by AR through
After preparing Pick by Paper and in-built logging capabilities inside the Google
Pick-by-HUD, we eliminated glitches in our Glass application, each study was also filmed
experiment design by picking a few sample and timed. Participants were also asked about
books by ourselves. We did this to avoid using their experience at the end of the test.
up valuable time when our participants came
to test. We discovered that randomly picking During the user study, two parameters were
out books in both our paper and AR pick list measured or observed. The first was the time
was not a good approach, since some pick lists taken to pick all books in the pick list. The
clustered books closely together while other second was the average task workload. Ideally,
pick lists scattered books far apart by pure we should have also been able to measure the
chance. To solve this, we modified the accuracy of using both methods. However,
algorithm to assign 33 books out of the total since the books being picked were guaranteed
288 lexicographically instead of randomly. We to be in place, we had an accuracy of 100% for
also realized that our original pick lists, which both methods.
consisted of between 10 and 14 books, did not
match the length of pick-lists in industry, Pick by Paper
which ranged from 30 to 50 items per picker. In Pick by Paper, participants reported that it
Therefore, the length of our pick lists was was easy to lose track of which book was being
adjusted to pick 33 items per pick to be

picked since rows in a pick list look largely that Figure 3: One of the authors using Pick by AR
same. Some participants doubled back after
they realized they accidentally skipped a row. Timing Data
One participant reported that concentrating on On average, it took 18.0 seconds to pick a
keeping track of picks prevented her from book using Pick by Paper. By contrast, it took
focusing on finding the book. All participants 10.8 seconds to pick a book using Pick by AR.
said they did not like using paper to pick Specific results of the user studies are
books. tabulated and visualized below:

Subject Paper HUD

(mm:ss) (mm:ss)

1 11:34 6:13

2 9:27 6:48

3 8:57 4.24

4 9:49 6:38

5 9:25 5:55

6 10:01 6:16
Figure 4: One of the authors using Pick by
Figure 3: Time taken to pick 33 books
Pick by AR
In Pick by AR, participants reported an easier Subject Difference % change
experience understanding where books were (mm:ss) (%)
located. One participant said that once she
understood the visualization of which shelf a 1 5:21 46.3
book was placed on, she stopped looking at the
2 2:39 28.0
aisle and shelf numbers on the shelf entirely
and relied completely on our interface. By 3 4:33 50.8
contrast, all participants said they required a
longer time to get used to the glass 4 3:11 32.4
visualization. This invisible training cost must
be taken into account in the cost of using the 5 3:30 37.2
system. 6 3:45 37.4

Figure 4: Time difference in approaches

The average percent change in time between

Pick by Paper and Pick by HUD is 38.7%.

environments. We believe that although travel
time was mostly identical, the speedups we
observed could be attributed to reduced
context switching and enhanced user
understanding of the task at hand.

We believe there are many other parameters

we can measure in a future study. First, we
could expand the methods being examined to
include Pick by Voice and Pick by RFID, two
other methods commonly used in industry. We
Figure 5: Comparison of Pick by Paper and Pick believe it would have been really interesting to
by AR quantify how much mental effort was exerted
by each participant so we could further
Evaluation investigate how cognitively demanding each
By examining the quantitative data and method is – this could have been accomplished
qualitative feedback from participants, we by using NASA-TLX. In future studies, we also
hypothesize that traditional paper pick lists want to increase the number of participants we
present too much information to the work with to 12 so that we could contribute to
participant at the start of the experiment, HCI workshops or conferences. We realized
which means a lot of mental effort needs to be upon conversing with the other order picking
exerted to keep track of the current pick, group in the class that we did not account for
which prevents the participant from focus on the time taken to learn both interfaces at the
where the next book actually is. By forcing the start of the study. We believe we can
participant to only focus on one book at a time, normalize all effects associated with learning to
we think that we greatly reduced the cognitive use a system by making participants pick
load on the participant. sample pick paths on both paper and AR
headsets so they grow used to the system. We
Since pick lists between Pick by Paper and Pick could then plot these different trials to show
by AR were similar, we believe there were how quickly participants learn an order picking
negligible differences between travel times system and therefore what the cost of learning
between different order picking methods as a system is.
hypothesized in the beginning of this study.
Rather, we think the large differences in REFERENCES
between trials comes from the ease of use and [1] Baumann, H. (2013). Order picking
the contextual design of our AR interface. We supported by mobile computing. ​Doctoral
think the visual mapping of shelves is easier to Dissertation (on pp. 39-43). University of
understand than a linear alphanumeric listing Bremen​.
of locations, and that by forcing the participant (​https://elib.suub.uni-bremen.de/edocs/00102
to pick one book at a time, the participant is 979-1.pdf​).
not given the opportunity to be distracted and [2] Schwerdtfeger, B., & Klinker, G. (2008,
can focus effectively at the task at hand. September). Supporting order picking with
augmented reality. In ​Mixed and Augmented
From the qualitative and quantitative data we International Symposium on (pp. 91-94). IEEE.
have collected from the study, we have shown (​https://mediatum.ub.tum.de/doc/1094499/fil
that Pick by AR is faster than Pick by Paper in e.pdf​).
low pick density environments. This is novel [3] Bartholdi, J. J., & Hackman, S. T. (2017).
because previous order picking research were Warehouse & Distribution Science: Release
only conducted in high pick density 0.98 (p. 13). Supply Chain and Logistics

[4] Haynes, M. (2017). Lateral Positioning of a
Monocular Head-Worn Display. ​Doctoral
Dissertation. Georgia Institute of Technology.
[5] K. Weaver, H. Baumann, T. Starner, H.
Iben, and M. Lawo. “An empirical task analysis
of warehouse order picking using
head-mounted displays.” ACM CHI, pp.
1695-1704, 2010.
[6] A. Guo, S. Raghu, X. Xie, S. Ismail, X. Luo,
J. Simoneau, S. Gilliland, H. Baumann, C.
Southern, T. Starner. “A Comparison of Order
Picking Assisted by Head-Up Display (HUD),
Cart-Mounted Display (CMD), Light, and Paper
Pick List.” ACM ISWC, Seattle, WA, September
[7] A. Guo, X. Wu, Z. Shen, T. Starner, S.
Gilliland, and H. Baumann. “Order Picking with
Displays.” IEEE Computer. 48(6): pp. 16-24,
June 2015.
[8] H. Baumann, T. Starner, H. Iben, A.
Lewandowski, and P. Zschaler. “Evaluation of
graphical user-interfaces for order picking
using head-mounted displays.” ACM ICMI, pp.
[9] X. Wu, M. Haynes, Y. Zhang, Z. Jiang, Z.
Shen, A. Guo, T. Starner, S. Gilliland.
“Comparing Order Picking Assisted by Head-Up
Display versus Pick-by-Light with Explicit Pick
Confirmation.” ACM ISWC. Osaka, Japan,
September 2015.
[10] X. Wu, M. Haynes, A. Guo, and T.
Starner. “A comparison of order picking
methods augmented with weight checking
error detection.” ISWC. Heidelberg, Germany,
September 2016.
[11] H. Baumann, T. Starner, and P. Zschaler.
“Studying Order Picking in an Operating
Automobile Manufacturing Plant.” IEEE ISWC,
Newcastle, UK, June 2012.