Precise Selection

Techniques for
Multi-Touch Screens
Hrvoje Benko
Andy D. Wilson
Patrick Baudisch

Columbia University and Microsoft Research

CHI 2006
Selecting a small target is
very HARD!

CHI 2006 2
 Small target size comparison
 Average finger ~ 15 mm wide

Target
Width Width Width
UI element
(abstract screen) 17” screen 30” screen
1024x768 1024x768

Close
6 mm 10.8 mm
button 18 pixels
(40% of finger) (66% of finger)

Resize
1.34 mm 2.4 mm
handle 4 pixels
(9% of finger) (16% of finger)

CHI 2006 3
 Touchscreen Issues

1. Finger >>> Target

2. Finger occludes the target
3. Fingers/hands shake and jitter
4. Tracking can be noisy (e.g. video)
5. No hover state (hover == drag)

CHI 2006 4
 Previous Work
 Solutions based on single touch interfaces and
complex on-screen widgets:

Sears, A. and Shneiderman, B. Albinsson, P. A. and Zhai, S.

“High Precision Touchscreens: “High Precision Touch Screen
Design Strategies and Interaction.” (CHI ’03)
Comparisons with a Mouse.”
(’91)

CHI 2006 5
 Dual Finger Selections

 Multi-touch techniques
 Single fluid interaction
 no lifting/repositioning of fingers
 Design guidelines:
 Keep simple things simple.
 Provide an offset to the cursor when so
desired.
 Enable user controlled control-display
ratio.

CHI 2006 6
 Simulating Hover State

 Extension of the “area==pressure” idea

(MacKenzie and Oniszczak, CHI 1998)
 Problem:
 LARGE area difference  reliable clicking
 SMALL movement (i.e. SMALL area
difference)  precise and accurate clicking

CHI 2006 7
 SimPress (Simulated Pressure)

 Clicking gesture –
“finger rocking”
 Goal:
 Maximize ∆ touch
area
 Minimize ∆ cursor
location

CHI 2006 8
 SimPress Cursor Placement

Center-of-Mass Cursor Top Middle Cursor

 Large ∆ touch area  Large ∆ touch area

 Large ∆ cursor loc.  Small ∆ cursor loc.

CHI 2006 9
SimPress in Action

CHI 2006 10
 Dual Finger Selections

1. Offset
2. Midpoint
3. Stretch
4. X-Menu
5. Slider

Primary finger  cursor position & click

Secondary finger  cursor speed or C/D

CHI 2006 11
 Dual Finger Offset

 Fixed offset WRT

finger
 Ambidextrous
control

CHI 2006 12
 Dual Finger Midpoint

 Cursor  ½
distance between
fingers
 Variable speed
control
 Max speed
reduction is 2x
 Dead spots on
screen!

CHI 2006 13
 Dual Finger Stretch

 Inspired by ZoomPointing
(Albinsson & Zhai,‘03)
 Primary finger  anchor
 Secondary finger
 defines the zooming area
 scales the area in all
directions away from the
anchor

CHI 2006 14
 Dual Finger Stretch

 Offset is
preserved after
selection!

CHI 2006 15
 Zooming Comparison

 Bounding Box Zoom  “Stretch” Zoom

 Fingers placed OFF  Primary finger placed
target ON target
 Target distance  Same motion = 2x
increases w/ zoom zoom

CHI 2006 16
Dual Finger X-Menu

 Crossing Menu (no buttons/no clicks)

 4 speed modes
 2 helper modes
 Cursor notification widget
 Eyes-free interaction
 Freezing cursor
 Quick offset setup
 Eliminate errors in noisy conditions
 Helpers:
 Snap – Remove offset
 Magnification Lens

CHI 2006 17
Dual Finger X-Menu

CHI 2006 18
Dual Finger X-Menu
with Magnification Lens

CHI 2006 19
 Dual Finger Slider

Freeze
Slow 10X
Slow 4X
Normal
Snap

CHI 2006 20
Dual Finger Slider

CHI 2006 21
 Multi-Touch Table Prototype

 Back projected
diffuse screen
 IR vision-based
tracking
 Similar to
TouchLight
(Wilson, ICMI’04)

CHI 2006 22
 User Experiments

 Measure the impact of a particular technique on

the reduction of error rate while clicking
 2 parts:
 Evaluation of SimPress clicking
 Comparison of Four Dual Finger Techniques
 Task:
 Reciprocal target selection
 Varying the square target width
 Fixed distance (100 pixels)
 12 paid participants (9 male,3 female, ages 20–
40), frequent computer users, various levels of
touchscreen use

CHI 2006 23
 Part 1: SimPress Evaluation

 Within subjects
repeated measures
100
90
design
Percent of Trials ± SEM

80
70
 5 target widths:
60
50
 1,2,4,8,16 pxls
40
30
 Hypothesis: only 16
20
10
pxls targets are
0 reliably selectable
1 2 4 8 16
Target Width (pixel)
 Results: 8 pixel
targets still have
F(4,44)=62.598, p<0.001 ~10% error rate

CHI 2006 24
 Part 2: Comparison of 4 Dual
Finger Selection Techniques
 Compare: Offset, Stretch, X-Menu, Slider
 Varying noise conditions
 Inserted Gaussian noise: σ=0, 0.5, 2
 Within subjects repeated measures design:
 3 noise levels x 4 techniques x 4 target widths
(1,2,4,8 pxls)
 6 repetitions  288 trials per user
 Hypotheses:
 Techniques that control the C/D will reduce the
impact of noise
 Slider should outperform X-Menu

CHI 2006 25
 Part 2: Error Rate Analysis

 Interaction of Noise x Technique

70 low medium high

Erro rR a te (% ) ± SE M

60
50
40
30
20
10
0
Ofs et X-Menu Slider Stretch

F(6,66)= 8.025, p<0.001

CHI 2006 26
 Part 2: Error Rate Analysis

 Interaction of Width x Technique

Offset X-Menu Slider Stretch

100
E rror Rate (% ) ± S E M

80

60

40

20

0
W-1 W-2 W-4 W-8

F(9,99)=29.473, p<0.001

CHI 2006 27
 Part 2: Movement Time
Analysis
 Analysis on median
Offset X-Menu Slider Stretch
times
 Stretch is ~ 1s faster
7
Missing than Slider/X-Menu
Movement Time (s) ± SEM

6 (t(11)=5.011, p<0.001)
5  Slider similar
4 performance to X-
3 Menu
2
1
0
W-1 W-2 W-4 W-8

CHI 2006 28
 Subjective Evaluation

 Post-experiment questionnaire (5 pt Likert

scale)
 Most mental effort: X-Menu (~2.88)
 Hardest to learn: X-Menu ( ~2.09)
 Most enjoyable: Stretch (~4.12), Slider (~4.08)
 No significant differences WRT fatigue

Best Technique for Noise Condition Overall Preference

Offset XMenu Slider Stretch 8

12 7
10 6
8 5
4
6
3
4 2
2 1
0 0
Low Noise Medium Noise High Noise Offset X-Menu Slider Stretch

CHI 2006 29
 Conclusions and Future Work

 Top performer & most preferred: Stretch

 Slider/X-Menu
 Comparable error rates to Stretch
 No distortion of user interface
 Cost: ~1s extra
 Freezing the cursor (positive feedback)
 Like “are you sure?” dialog for clicking…
 Possible future SimPress extensions:
 Detect user position/orientation
 Stabilization of the cursor

CHI 2006 30
Questions
 Multi-Touch Tabletops

 MERL DiamondTouch (Dietz & Lehigh,

’01)
 SmartSkin (Rekimoto, ’02)
 PlayAnywhere and TouchLight (Wilson,
’04, ’05)

CHI 2006 32
 ANOVA Table

Source df F p

Noise (N) (2,22) 20.24 <0.001

Technique (T) (3,33) 169.14 <0.001

Width (W) (3,33) 150.40 <0.001

NxT (6,66) 8.03 <0.001

TxW (9,99) 29.47 <0.001

NxW

NxTxW

CHI 2006 33