Software for the Quantification of

Reading Patterns using Gaze Tracking

Final Presentation by: David Young
Group 27 (Maeve Woeltje, Paras Vora)

Presentation Overview
Reminders
Need + Project Scope
Specific Design Requirements

Introduce Major Software Components and Flow-Path

Run Time and Hardware Considerations
Analytical Efforts for Algorithm Design Improvements
Future Directions
Conclusion

Need
Consumer Grade Reading Performance Feedback
Inexpensive Software-Hardware Package

Give readers an idea of their reading thoroughness

Accessible at home to amateur users

Scope
Evaluate Reading Patterns
Eye Tracking Software

Provide Feedback
Helpful for the user in quantifying reading gaze patterns

Specific Design Requirements

Total Parts cost < $200 (software + hardware)
Ease of Use
Intuitive GUI and menu system

Calibration under 30 seconds

Session Data Size under 5MB
Provide Useful Feedback

Reading Speed
Skimmed, skipped, re-read passages
Identification of points of distraction
Output vocabulary lists
Performance history

Software Flow-path
MainApp

GUI Scene
Controllers

Database
Controller

TextFormat

TheEyeTribe
API

Basic Metrics
Generation

GUI .FXML
Layouts

H2 SQL
Database
HeatMap
Generator

User
Interaction

Comprehensive
Analysis
Generator

TheEyeTribe
SDK

TheEyeTribe
Hardware

User Gaze
Interaction

Design Overview

Gaze Input (HW + Software)

Main App Controller
Database
GUI Layouts and Controllers
Text Formatting
Analysis Generation
Basic Metrics
Comprehensive Analysis
Heatmaps

Handling Gaze Input

APPLICATION

SDK
Open API
NETWORK

EyeTribe SERVER
HW
Images from: <dev.theeyetribe.com>

TRACKER

Main App Controller

Launches the Application
Launches other software components at the
appropriate time
Functions like Telephone Operator
Middle-man connecting different software
components (permits communication and
specialization)

Database Design (SQL Schema)

H2 SQL
Tables

users
texts
sessions
sessionData
basicSessionMetrics
sessionAnalyses

Creating a Table:

Database Control
databaseController Class:
Connects to Existing Database
Creates new DB if none exist

Logs in a valid user

Handles all Queries
Data Injection
Data Retrieval

Serves as only access to database

GUI Layouts + Controllers

Every GUI page is comprised of
FXML file
Function Extensible Markup Language
Handles layout design
Like HTML for webpages

Page Controller
Instantiates page
Dynamically updates information
Responds to User Input

Text Formatting
Graphical Stylized Text Display
Holds individual line layouts in a List

Hit Detection
Determines line #, then performs single line hit detection
Calls insert gaze data

Text Retrieval
Retrieve a words text (or a whole line) given specific character
indices

Analysis Generation (1/3)

basicSessionMetricsGenerator class:
Retrieves raw gaze data from a session
Generates basic metrics useful for the user, and for more analysis

Session Duration
Word Count
Average Reading Speed
Fraction of Time Focused
Total # of Saccades/Fixations
Average Spatial Saccade Length (in Characters)
Average Temporal Saccade and Fixation Lengths (in ms)

Analysis Generation (2/3)

sessionAnalysisGenerator class:
Receives parsed data from basicMetricsGenerator
Performs rapid analysis of gaze patterns
Calculates fixations per interest area
Line, words, and variable length interest areas

Sorts interest areas by fixation count

Identifies or Indicates:
Sections of extended focus
Sections skimmed and skipped
Vocab lists of difficult words

Analysis Generation (3/3)

heatmapGenerator class:
Prepares lists for 2 sets of data (all data & fixated only)
Java Heatmap creates a heatmap grid.
Overlays the generated heatmap onto a screenshot of the
session text.
Stores the completed heatmaps as images by sessionID.

Software Flow-path
MainApp

GUI Scene
Controllers

Database
Controller

TextFormat

TheEyeTribe
API

Basic Metrics
Generation

GUI .FXML
Layouts

H2 SQL
Database
HeatMap
Generator

User
Interaction

Comprehensive
Analysis
Generator

TheEyeTribe
SDK

TheEyeTribe
Hardware

User Gaze
Interaction

Analytical Approach to Improving Our

Nave Algorithms
Differentiating Saccades + Fixations
Dispersion Based Identification:
Dynamic window defines boundaries of a fixation, saccades are
defined as jumps between windows.

Creating dynamic interest areas

Change based on fixation location

4 spaces to left, 20 spaces to right (avg field of perception)
Makes it easier to distinguish skimming from faster reading
Fixation count in such a defined interest area is more informative
than individual words or lines (research backed)

Run Time + HW Consideration

Run Time Considerations:
SQL asymptotic complexity: insert O(1), search (sub linear to linear)
Algorithm design
Avoid redundancy and share resources
Only one complete iteration over data where possible
Intelligent sorting

Functionally fast.
Real-time data recording while reading (injection)
Analysis ( < 5 seconds)

Hardware Considerations:
Computers than run Java
Program is mainly serial

Future Directions
What weve accomplished so far
Proof of concept with basic analyses
All local machine based
Robust framework for expansion (~ 16k lines of code)

In development/partially finished
Improvements on existing analysis algorithms
Regression identification and quantification

Future
More capable and efficient algorithms
Optimizations
Cloud based storage and aggregate data analysis

Conclusion (1/2)
How we are different
Intelligent connection to viewed medium
Industry (consumer grade) vs. research oriented
Existing relevant patents have no product or different aims

What we learned
Start coding as early as possible
Experiment, prototype ideas (naively if need be)
Jump in and get feet wet

Then design optimized algorithms

Time to Debug >> Time to Design

Learned a lot about eye movement while reading.
Enormous amount of research accomplished to improve our nave
approaches

Conclusion (2/2)
As a proof of concept our prototype is successful:

Hardware ($110) < $200 design spec

GUI is extremely intuitive and easy to use
Feedback is nearly instant, and conveys meaningful information
Negligible hardware limitations

Envisioned Final Design:

Completely Debugged
Improved Algorithms
Increase Quantity of feedback
Make product truly Consumer Friendly

QUESTIONS?