Andrzejczak

Effect of testing location on usability testing performance, participant stress levels, and subjective
testing experience
2010
journal of systems and software

1258-1266

abstract:
impact of testing localities on stress & UX not clear
this compares two groups in traditional lab & in field: complete simple & complex task
times measured, # of critical incidents reported, & user-reported anxity scored

Task times slower in real world—ascribed to gear used/methods & did not have any meaningful impact
on real world application
Critical incident counts did not vary/differ
Stress levels did not vary/differ significantly

CLAIM: similar user testing experience exists in remote & traditional usability lab tests

this is interesting. I am curious to see if they mention/discuss the whole testing/experimenter bubble
which Jambon brings up in one of his articles—and how that can work to establish or mimic the lab
testing environment unlike when users are operating alone, without experimenters (like the skiing), in
the environment. So, we shall see.I

According to page 1261, the observer is not physically present when the user completes the task.

1259
reference to Quesenberry's (2001) Es
effective, efficient, engaging, error tolerant, easy to learn

Andreason 2007: remote testing methods comparison

Bruun et al 2009 remote, asynchronous testing methods
Petrie et al 2006 vlaue of remote testing to working with disabilities

Critical Incident Technique: directive Framework (Flanagan 1954)

-observe behaviors that contribute to success/failure of an individual operating a system or performing
a task
Castillo 1997 took this to study interface difficulties

1260
task complexity is related to user's psychological state (Campbell 1988)

1263
testing location did not impact task times significantly
testing location had no effect on number of critical incidents reported
testing location did not impact the amount of stress
related to dissertation
this implies that testing location does not have an important or relevant impact upon the results. The
users, however, are operating as individuals in these situations and they, again, are just attempting to
complete a task.Sure one is simple and the other is arguably complex; however this does not take into
account the whole group dynamic account which is probably a significant aspect to working with
mobile devices and tourism. Similarly, it is working in an environment where people probably know
and are comfortaable in that environment. Operating or doing a task where you know the layout is one
thing, but it seems like you could argue that another form of usability testing could take place in a
space or arena that the user did not know and then see how much more significantly the different in
outside environment impacted the user. That is, does a known external or remote site not impact a
person nearly as much as a totally unknown location?

How does this link to usability and tourist devices?

I think part of it is asking where is the sharing, the multiple users, the confusion, and the collaborative
elements to it.
Second, I'd like to know what is being built or created with these tests and tasks as compared to what
I'm starting to see in terms of the tourist stuff.
Third,, I'm curious about the differences arising between lab and remote testing. Also, I wonder if
usability testing has worked very much with groups of users.