CHAPTER 13:

Information Search

Designing the User Interface:

Strategies for Effective Human-Computer Interaction

Fifth Edition

Ben Shneiderman & Catherine Plaisant

in collaboration with
Maxine S. Cohen and Steven M. Jacobs
Addison Wesley
is an imprint of

© 2010 Pearson Addison-Wesley. All rights reserved.

 Information Search
• Introduction
• Searching in Textual Documents and Database
Querying
• Multimedia Document Searches
• Advanced Filtering and Search Interfaces

1-2

© 2010 Pearson Addison-Wesley. All rights reserved. 13-2

 Information Search (cont.)
• Information search should be a joyous experience,
but it takes a skilled searcher with robust tools to
perform an effective search
• Information retrieval and database management
have evolved into:
- information gathering, seeking, filtering,
collaborative filtering, sense making, and visual
analytics.
- data mining from data warehouses and data
marts
- knowledge networks or semantic webs
• All the above is complicated by the increased
volume of material to search 1-3

© 2010 Pearson Addison-Wesley. All rights reserved. 13-3

 Search terminology
• Task objects (such as movies for rent) are stored in
structured relational databases, textual document
libraries, or multimedia document libraries
• A structured relational database consists of
relations and a schema to describe the relations
• Relations have items (usually called tuples or
records), and each item has multiple attributes (often
called fields), which each have attribute values
• A textual document library consists of a set of
collections (typically up to a few hundred collections
per library) plus some descriptive attributes or
metadata about the library (for example, name,
location, owner)
1-4

© 2010 Pearson Addison-Wesley. All rights reserved. 13-4

 Search terminology (cont.)
• Task actions are decomposed into browsing or searching

• Here are some examples of task actions:

- Specific fact finding (known-item search)
• Find the e-mail address of the President of the United
States.
- Extended fact finding
• What other books are by the author of “Jurassic Park”?
- Exploration of availability
• Is there new work on voice recognition in the ACM
digital library?
- Open-ended browsing and problem analysis
• Is there promising new research on fibromyalgia that
might help my patient?

1-5

© 2010 Pearson Addison-Wesley. All rights reserved. 13-5

 Search terminology (cont.)
• Once users have clarified their information needs,
the first step towards satisfying those needs is
deciding where to search
• Supplemental finding aids can help users to clarify
and pursue their information needs, e.g. table of
contents or indexes
• Additional preview and overview surrogates for
items and collections can be created to facilitate
browsing

1-6

© 2010 Pearson Addison-Wesley. All rights reserved. 13-6

 Searching in Textual Documents
and Database Querying
• Expert users can use SQL:
SELECT DOCUMENT#
FROM JOURNAL-DB
WHERE (DATE >= 2004 AND DATE <= 2008)
AND (LANGUAGE = ENGLISH OR FRENCH)
AND (PUBLISHER = ASIST OR HFES OR ACM)
• SQL has powerful features, but it requires
training
• While SQL is a standard, form fill-in
queries have simplified query formulation

1-7

© 2010 Pearson Addison-Wesley. All rights reserved. 13-7

 Searching in Textual Documents
and Database Querying (cont.)
• Other methods include:
- Natural language queries
- Form fill-in
- Query by example (QBE)-database querylanguage
for relational databases
-  It is the first graphical query language, using visual
tables where the user would enter
commands, example elements and conditions.
• Evidence shows that users perform better and have
higher satisfaction when they can view and control
the search

1-8

© 2010 Pearson Addison-Wesley. All rights reserved. 13-8

 1-9

© 2010 Pearson Addison-Wesley. All rights reserved. 13-9

 Searching in Textual Documents
and Database Querying (cont.)

1-10

© 2010 Pearson Addison-Wesley. All rights reserved. 13-10

 Five-phase framework to clarify user
interfaces for textual search
1. Formulation: expressing the search
2. Initiation of action: launching the search
3. Review of results: reading messages and
outcomes
4. Refinement: formulating the next step
5. Use: compiling or disseminating insight

1-11

© 2010 Pearson Addison-Wesley. All rights reserved. 13-11

 Five-phase
framework to
clarify user
interfaces for
textual search
(cont.)

1-12

© 2010 Pearson Addison-Wesley. All rights reserved. 13-12

 Searching in Textual Documents
and Database Querying (cont.)

As users press keys on the keypad (left figure), the digits are shown and a search is
implicitly initiated to display the list of names in the address book that match the
series of keys pressed. On the right figure, red wedges at the edge of the screen hint
at the locations of off-screen results on a map (Gustafson) 1-13

© 2010 Pearson Addison-Wesley. All rights reserved. 13-13

 Searching in Textual Documents
and Database Querying (cont.)

A search for “user interface”

powered by Endeca
(http://www.lib.ncsu.edu) returns
144 results grouped into 10
pages. The menu at the upper
right allows users to sort
results by relevance or by date,
while on the left a summary of
the results organized
by Subject, Genre, or Format
provides an overview of the
results and facilitates
further refinement of the search.

1-14

© 2010 Pearson Addison-Wesley. All rights reserved. 13-14

 Multimedia Document Searches

• Image search
• Map search
• Design or diagram search
• Sound search
• Video search
• Animation search

1-15

© 2010 Pearson Addison-Wesley. All rights reserved. 13-15

 Multimedia document searches
(cont.)
• Image Search:
– Finding photos with images such as the Statue of Liberty is
a challenge
• Query-by-Image-Content (QBIC) is difficult
• Search by profile (shape of lady), distinctive features (torch),
colors (green copper)
– Use simple drawing tools to build templates or profiles to
search with
– More success is attainable by searching restricted
collections
• Search a vase collection
• Find a vase with a long neck by drawing a profile of it
– Critical searches such as fingerprint matching requires a
minimum of 20 distinct features
– For small collections of personal photos effective browsing
and lightweight annotation are important
1-16

© 2010 Pearson Addison-Wesley. All rights reserved. 13-16

 Multimedia document searches
(cont.)
• Map Search
– On-line maps are plentiful
– Search by latitude/longitude is the structured-database
solution
– Today's maps are allow utilizing structured aspects and
multiple layers
• City, state, and site searches
• Flight information searches
• Weather information searches
• Mapquest, Google Maps, etc.
– Mobile devices can allow “here” as a point of reference

1-17

© 2010 Pearson Addison-Wesley. All rights reserved. 13-17

 Map search:
 Computer-generated maps are increasingly available online.

 Locating a map by latitude and longitude is the structured database

solution, but searching by features is now possible because geographical
information systems preserve the structural aspects and the multiple layers in
maps.

 eg : Through mobile devices users can locate an Italian restaurant within two
hundred yards of any station on a given subway line.

18
 Multimedia document searches
(cont.)
• Design/Diagram Searches
– Some computer-assisted design packages support search of designs
– Allows searches of diagrams, blueprints, newspapers, etc., e.g. search for a red
circle in a blue square or a piston in an engine
– Document-structure recognition for searching newspapers
• Sound Search
– MIR supports audio input
– Search for phone conversations may be possible in future on speaker independent
basis
• Video Search
– Provide an overview
– Segmentation into scenes and frames
– Support multiple search methods
– Infomedia project
• Animation Search
– Prevalence increased with the popularity of Flash
– Possible to search for specific animations like a spinning globe
– Search for moving text on a black background

1-19

© 2010 Pearson Addison-Wesley. All rights reserved. 13-19

Design or diagram search :
 Some computer-assisted design packages offer
users limited search capabilities within a single
design or across design collections.

 Document structure recognition and search tools

also supports searching.

eg : finding red circles inside blue squares

for newspaper, front pages with headlines should
not have advertisements.

20
Sound Search :
 Music Information Retrieval(MIR) systems can now use audio input, where
users can query with musical content.

 Users can sing or play a theme, hook(clip), or riff from the desired piece of
music, and the system returns the most similar items.

 Finding a spoken word or phrase in databases of telephone conversations is

still difficult but it is becoming possible.

Video Search:
The video should be segmented into screens or cuts and allow scene
skipping. Gaining an overview of a two-hour video by a timeline of scenes enables
better understanding, editing and selection.

eg : information project is an example library of digital video.

21
Animation search :
With the success of Flash, it might become possible to specify for certain kinds of
animation, such as spinning globes or morphing faces.

General :
latitude (φ) is a geographic coordinate that specifies the north-south position of a
point on the Earth's surface.

Longitude is a geographic coordinate that specifies the east-west position of a

point on the Earth's surface.

Riff, a short, repeated musical phrase.

22
 Multimedia Document Searches
(cont.)

1-23

© 2010 Pearson Addison-Wesley. All rights reserved. 13-23

 Advanced Filtering and Search
Interfaces
• Filtering with complex Boolean queries
• Automatic filtering
• Dynamic queries
• Faceted metadata search
• Query by example
• Implicit search
• Collaborative filtering
• Multilingual searches
• Visual field specification
1-24

© 2010 Pearson Addison-Wesley. All rights reserved. 13-24

 Advanced filtering
and search interfaces (cont.)
For advanced uses there are alternatives to form fill-in query interfaces:

• Filtering with complex Boolean queries

– Problem with informal English, e.g. use of ‘and’ and ‘or’
– Venn diagrams, decision tables, and metaphor of water flowing have not worked for
complex queries
• Dynamic Queries - Adjusting sliders, buttons, etc and getting immediate
feedback
– “Direct manipulation” queries
– Use sliders and other related controls to adjust the query
– Get immediate (less than 100 msec) feedback with data
– Dynamic HomeFinder and Blue Nile
– Hard to update fast with large databases
• Query previews present an overview to give users information and the
distribution of data and thereby eliminate undesired items
• Faceted metadata search
– Integrates category browsing with keyword searching
– Flameco

1-25

© 2010 Pearson Addison-Wesley. All rights reserved. 13-25

 Advanced filtering
and search interfaces (cont.)
• Collaborative Filtering
– Groups of users combine evaluations to help in finding items in a
large database
– User "votes" and his info is used for rating the item of interest, e.g.
a user rating six restaurants highly is given a list of restaurants
also rated highly by those who agree the six are good
• Multilingual searches
– Current systems provide rudimentary translation searches
– Prototypes of systems with specific dictionaries and more
sophisticated translation
• Visual searches
– Specialized visual representations of the possible values, e.g. dates
on a calendar or seats on a plane
– On a map the location may be more important than the name
– Implicit initiation and immediate feedback

1-26

© 2010 Pearson Addison-Wesley. All rights reserved. 13-26

 Advanced Filtering and Search
Interfaces (cont.)

Blue Nile (bluenile.com) uses

dynamic queries to narrow
down the results of
searches. Here, the double-
sided sliders were adjusted
to show only lower-priced
diamonds with very good cut
and high carat ratings.

1-27

© 2010 Pearson Addison-Wesley. All rights reserved. 13-27

 Advanced Filtering and Search
Interfaces (cont.)

Flamenco (http://flamenco.berkeley.edu/) is an example of a faceted metadata search. Facets include Media,

Location, Date, Themes, and so on. Here, two attribute values are selected (Date = 20th century and Location =
Europe) with results grouped by location. The image previews are updated immediately as constraints are added or
removed (another example of implicit query initiation). Clicking on a group heading such as “Belgium/Flanders” refines
1-28
the query into that category, while clicking on “All” dates relaxes the date constraint.
© 2010 Pearson Addison-Wesley. All rights reserved. 13-28
 Advanced Filtering and Search
Interfaces (cont.)

Using The Hive Group’s treemap

(http://www.hivegroup.com/), users
can review all waterproof binoculars in
the catalog of Amazon.com products
and browse the items in the list,
grouped by manufacturer. Each box
corresponds to a pair of binoculars,
and the size of the box is proportional
to its price. Green boxes are best-
sellers. Users can filter the results
using the dynamic query sliders on the
right. Here all the binoculars with less
than three user reviews have been
filtered out, leaving only 61 binoculars
to consider.

1-29

© 2010 Pearson Addison-Wesley. All rights reserved. 13-29

 Advanced Filtering and Search Interfaces

Filtering with complex Boolean queries:

 Commercial information system such as DIALOG and FirstSearch permit
complex boolean expressions with parenthesis.

 A part of confusion stems from informal English usage.

eg : a query such as “List all employees who live in New york and Boston”
-- in English, usage of ‘ and ‘ expands the message but in SQL, it restricts the
message by applying both the conditions.

 Venn diagrams, decision tables, and metaphors have been used, but these
representations becomes clumsy when query complexity increases.

30
 Advanced Filtering and Search Interfaces
Automatic Filtering :
• Another form of filtering is to apply a user-constructed set of keywords to dynamically generated information, such as incoming e-mail
messages , newspaper stories or scientific journal articles.

• Users can be notified by e-mail that a relevant document has appeared or results can be collected into a file until users seek them out.
These approaches are called selective dissemination of information(SDI) which was used in the earliest days.

Dynamic Queries :
• Dynamic queries also called direct-manipulation queries, since they share the same concepts of visual display of actions and
objects, the use of rapid, incremental and reversible actions and the immediate display of feedback.

• Additional benefits : prevention of syntax errors and an encouragement of exploration.

31
 Advanced Filtering and Search Interfaces
• ORs and ANDs can be used in queries.

• The early Dynamic Home Finder used dynamic queries for finding homes for sale.

• Searching in online databases can also be done with dynamic queries.

• Query previews provides an interactive overview of the data available.

• This overview allows users to gain useful information about the distribution of the
data available over a few selected attributes and to rapidly eliminate undesired
items.

32
 Advanced Filtering and Search Interfaces

Faceted(one of the small, polished plane surfaces of a cut gem)

metadata search :
• This type of search interface integrates category browsing with keyword
searching.

• This interface makes use of hierarchical faceted metadata presented as

simultaneous menus and dynamically generated numerical query previews.

• It allows users to navigate explicitly along multiple conceptual dimensions

that describes the images and to progressively narrow or expand the scope of
the query while browsing images.

• eg : architectural photo browsing , users can look for photos of modern

homes.

33
 Advanced Filtering and Search Interfaces
• Many search interfaces are now using selection in multiple menus as their primary
search interfaces, but they often allow refinements in one menu at a time.

Collaborative filtering :
• The social form of filtering allows group of users to combine their evaluations
to help one another find interesting items in large collections.

• Each user rates items in terms of their interest.

• This method can also applied to movies, music, restaurants and so on.

• Dozens of systems have been built for organizational databases, news files,
music groups and World Wide Web pages.

34
 Advanced Filtering and Search Interfaces
Multilingual searches :
Current web search engines merely provide basic translation tools, but prototypes of
multilingual information systems allow users to select appropriate dictionaries,
restrict keyword translations and use more powerful translation systems to carefully
identify documents that justify the cost of high-quality professional translation.

Visual Searches :
• The specification of query fields can sometimes be simplified by using specialized
visual representations of the possible values.

• When there are no natural representations of the choices, information

visualization techniques can be used.

• eg : A tree map can be used to represent the product catalog.

• Visual search interfaces help users refine their needs. They are attractive and
reduce error messages such as “data out of range” while providing information
about data availability and a feeling of thoroughness to users.

35
 Advanced Filtering and Search Interfaces

Visual search interfaces can become powerful dynamic – query interfaces ,

while the addition of abstract data previews and overviews transforms visual
search interfaces into potent information-visualization tools that help users
explore the data visually before any search is even specified.

36