BCS HIGHER EDUCATION QUALIFICATIONS

Level 5 Diploma in IT
March 2012
EXAMINERS REPORT
Principles of User Interface Design
Examiners General Comments
Candidates must use the indicated mark allocations for each section of the question
as a guide to the amount of detail required by an answer and tailor their answer
accordingly. Any time and effort spent simply re-stating the question asked at the
beginning of an answer is wasted.
It is very important that the answers presented are relevant to the question asked,
and address the context and the specific issues raised in the question, or marks
cannot be earned. Candidates should not give very general descriptions of generic
design or evaluation techniques in response to questions that ask for how they would
approach highly specific scenarios - which usually have a very narrow context.

Section A
A1.

a)
b)

c)

Describe what is meant by the term Web Accessibility.

(3 marks)
Describe how you would use expert user evaluation to evaluate the
accessibility of a web site.
(12 marks)
You have been asked to design a web site for a pensioner/retired
persons charity. As most of the users will be elderly, you need to
consider accessibility issues.
Describe FIVE accessibility issues you will address when doing your
design.
(10 marks)

Answer Pointers
These are open-ended questions and students answers will vary, but should be
along the following lines. Any reasonable answer will be accepted and marks will be
given for clear explanations and good examples.
a)

The description should include the elements in the following W3C definition:
Web accessibility means that people with disabilities can use the Web. More
specifically, Web accessibility means that people with disabilities can perceive,
understand, navigate, and interact with the Web, and that they can contribute
to the Web. Web accessibility also benefits others, including older people with
changing abilities due to ageing. Web accessibility encompasses all

disabilities that affect access to the Web, including visual, auditory, physical,
speech, cognitive, and neurological disabilities.
b)

Answers should include most of the following points:

Use latest guidelines from W3C - Web Content Accessibility Guidelines
(WCAG) and other useful sites;
Online evaluation tools can be used to help with evaluation, but they cannot
determine if a site meets accessibility guidelines by themselves;
Either less formal preliminary review or more formal conformance evaluation
can be done.
A preliminary review aims to identify and recommend fixes for any
accessibility issues and problems with the site, while the more formal
conformance evaluation measures if the website meets certain accessibility
standards (e.g. conforms to WCAG Guidelines).
The procedure is much the same for both.
Use a representative sample pages containing examples of all elements and
layouts in the site such as forms, tables, images and scripts:
- examine with a range of browsers;
- manipulate colours, fonts, images, sounds to see effect;
- test script functionality;
- test navigation using selection by other than mouse clicks e.g. tabs
- use a voice browser and text browser to check information available
through these;
- use automated tests to validate markup;
- test with at least 2 online automated accessibility evaluation tools;
- see if the text on the web site is clear and simple (e.g. use Clear and
Appropriate Language and Design (CLAD) test).
Use automated system to see how website appears to users with various
types of colour blindness.
Summarise findings and make recommendations.

c)

Age-related considerations:
age-related macular degeneration (AMD) is the most common cause of
blindness and visual impairment in people over 60;
cataracts are the commonest cause of blindness in the world;
visual changes in the elderly often result in increased susceptibility to glare
and decreased contrast acuity;
neurological symptoms in the elderly are common: e.g. decreased cognitive
function, difficulty concentrating, easily distracted, impaired memory,
(making it difficult to follow navigation and complete tasks), mobility
problems, poor sight and hearing, and tremors;
joint stiffness or functional impairment.
Issues to be addressed in the design can include:
use larger fonts or make font size scalable e.g. using CSS;
use good color contrast or provide high contrast style;

 ensure users able to customise color settings/font size via browser

settings;
including instructions on customising fonts and colors in browser settings;
color should not used as the only visual means of conveying information;
make link and navigation targets more prominent e.g. larger font;
avoid distracting users with any unnecessary movement or sound on the
web pages;
for audio/audiovisual media - provide transcripts, captions, and keep any
background sound low;
for non-text elements e.g. images - provide text alternatives for people
using text-to-speech (speech synthesis) software;
make all functionality available via the keyboard i.e. not requiring a mouse e.g. tabbing between navigation or form elements.
Examiners Guidance Notes
In Question A1, there was generally a good understanding of the term Web
Accessibility, and most candidates covered the essential points. However in
answering the second, few were able to give a detailed description of how they
would go about the specific evaluation described.
Such an expert evaluation would not make of either normal or disabled users as
subjects, but be conducted by you, the expert. In particular, using focus groups
would not be appropriate. Use would always be made of any relevant generally
accepted or statutory guidelines or regulations covering this area.
Answers to this question required answers that addressed the specific accessibility
issues related to the scenarios, and general usability issues were not relevant to the
questions asked and therefore did not earn any marks.
For the record, although elderly people may have a number of problems related to
poor vision, colour blindness is not an accessibility issue that is specifically relevant
to this age group.

A2.

a)

In the context of usability testing describe what is meant by task-based

testing.
(2 marks)

b)

You have been hired by FasterFlyer airline to test the usability of a

mobile application which allows users to check in for their flight from
their mobile device. They set up a dummy flight and bookings for you to
use in your testing.
i)

ii)

Describe how you would undertake a task-based usability test,

including the metrics you would gather.
(18 marks)
What would you consider when deciding how many users you
needed for the test?
(5 marks)

Answer Pointers
These are open-ended questions and students answers will vary, but should be
along the following lines. Any reasonable answer will be accepted and marks will be
given for clear explanations and good examples.
a)

Task-based testing is where the user is given a task or tasks to perform using
the system and the usability determined from their efforts. This method does
not concern itself with the underlying functionality of the system, just the ability
of a user to perform given tasks.

b)

1. Undertake a task analysis and document the task/sub-tasks supported by

the app;
2. Decide which tasks to test - in this case it may include sub-tasks such as
reserving seats, requesting special meals etc. Ideally tasks should cover
all relevant functionality;
3. Determine the versions of the app and platforms/devices that it runs on;
4. Ensure your testing includes all versions and platforms/devices and
modes of access - 2G, 3G, wireless etc.;
5. Design protocol for user including scenario;
6. Decide what metrics to collect:
basic user demographic and characteristics;
device/platform/version details;
access method details;
time taken to complete task and subtasks;
errors made - number and type;
recovery from errors;
use of help;
usability questionnaire such as SUS/Sneidermans.
7. Draw up data collection protocol;
8. Brief user
9. Get user data + consent etc.
10. Get user to perform tasks;
11. Observe/time/count - help from other observers where necessary;
12. Makes notes
13. Feedback from user/SUS etc.

c)

Consider how many platforms/device/version combinations you have

Consider how many tasks to test
Whether each user could do more than one task/platforms/device/version
combination without learning being a factor
What metrics you are collecting - rule-of-thumb the more metrics the more
users required
The magic number - 30 users

Examiners Guidance Notes

This question was about a very well known but specific evaluation technique, so any
answers given that were outside of this context could not earn marks.

Good answers addressed all the points asked for in the question and demonstrated
that the candidate had thought about the issues in the context of the scenario
described.

A3.

a)

b)

c)

Describe what is meant by a mental model in the context of user

centred design.
(3 marks)
You are doing a survey on what people use the Internet for at home.
You must decide on whether to use an in-person or a postal
questionnaire. Describe FIVE issues you would consider when deciding
which to use.
(10 marks)
Describe how you would do a cognitive walkthrough as part of the
evaluation of a website used to sell books.
(12 marks)

Answer Pointers
These are open-ended questions and students answers will vary, but should be
along the following lines. Any reasonable answer will be accepted and marks will be
given for clear explanations and good examples.
a)

A mental model is the personal cognitive model that a user has of how a
system functions.

b)

c)

In-person Questionnaires

Postal Questionnaires

Relatively more expensive as

need administrator
May need more than one person
administering
Sample size often limited by time
Enables direct contact with user
- can clarify/follow up/expand
answers
- can ensure user answers
fully/correctly
- can gain additional information
by observing user
answering questions
Get returns from all participants
Data available immediately

Relatively cheap
Can usually be run by single
person
Can usually use large samples
No direct contact with users
- cannot easily clarify details/ask
supplementary
questions
- no control over answers
- cannot observe user

Return rates often poor

Delay in obtaining data

A cognitive walkthrough is a formalised way of anticipating a users thoughts

and actions when using the website.
Some knowledge of who the users of the website are is required. It can be
gained through surveys of the site, site statistics or customer data. Site
statistics can also give indications of traffic flow and tasks done on the site.

Tasks the website is intended to support are select the and each action a user
has to take to do the task is walked through using a protocol. The user actions
are anticipated, taking into account the user profiles already obtained.
Prompts and feedback provided by the website are also taken into account.
Any problems with the interface are documented, and solutions to them
determined.
A walkthrough does not test real users on the system but is an evaluation tool
that helps apply design expertise to the evaluation of the interface and
because an entire classes of users can be represented, the walkthrough can
be used to identify many more problems than a single, unique user in a single
test session.
Examiners Guidance Notes
In this question, good answers addressed the required number of points asked for in
a practical way and in the context of the scenario. General descriptions of what the
Internet might be used for at home were not relevant and could not earn marks.
It is important to appreciate that the question did not call for the candidate to choose
the most appropriate survey method, merely to discuss relevant issues to be
considered.
It should be noted that focus groups have no place in carrying out an evaluation
using a cognitive walkthrough.

Section B
B4.

Heuristic evaluation is commonly used in industry and commerce as a

convenient and useful way of determining the usability of products.
a)

Briefly explain what heuristic evaluation is and how is it used.

b)

Discuss the main advantages of using heuristic evaluation.

c)

(5 marks)
Discuss the disadvantages and limitations of heuristic evaluation.
(15 marks)

(5 marks)

Answer Pointers
a)
Heuristic evaluation is a form of Usability Inspection and was first put forward
by Nielsen who lists 10 general heuristics. The method relies on evaluators
inspecting a system using the 10 heuristics and identifying problems that arise
from comparing the system with the heuristics.
Candidates are NOT being asked to list the 10 original heuristics but if any are
listed marks will be given for that. The heuristics are:
1.

Visibility of system status

2.
3.
4.
5.
6.
7.
8.
9.
10.

Match between system and the real world

User control and freedom
Consistency and standards
Error prevention
Recognition rather than recall
Flexibility and efficiency of use
Aesthetic and minimalist design
Help users recognize, diagnose, and recover from errors
Help and documentation

b)

The advantages are that it is easy to use, does not take a long time and can
be applied by anyone. Many users obtain Nielsenss original 10 heuristics and
apply these. This has made heuristic evaluation very popular and usage is
common in industry and commerce.

c)

The 10 original heuristics are not defined in detail but each one is generally
described. What is needed is an agreed set of operational definitions that
allow quantification of some kind. Therefore the application of the heuristics
depends on the evaluators own understanding of what each heuristic means.
There is no way to assess statistical validity or reliability. (5 marks)
The method relies totally on the experience of the evaluator, i.e. results differ
depending on whether the evaluator is experienced or not. Contributes more
to training evaluators than to system design. (5 marks)
Leading Human Factors experts such as Carroll and Rosson have gone as far
to say that heuristics evaluation contributes nothing of use. This has led other
researchers and practitioners to argue that heuristic evaluation is potentially
dangerous in that the approach can easily result in misleading data that can
have detrimental effect on companies that are using the method. (5 marks)

Examiners Guidance Notes

This question yielded a number of good answers from candidates that showed good
preparation and an understanding of relevant background materials. Candidates
should be aware that there is no real substitute for having read and studied
background material in preparation for an examination. Heuristics are now commonly
used in practice and as a consequence there is a wealth of easily accessible
material available. It is also important to be clear that a critical discussion is needed
in parts of this question and candidates should not be afraid to be critical.

B5.

The use of scenarios is now common in user centred design approaches and
is seen as an important component of modern user interface design.
a)
b)
c)

Briefly outline the main characteristics of scenario-based design.

(5 marks)
Explain the main issues that scenarios are meant to address.
(5 marks)
i)
In scenario-based design, what is meant by problem scenario?
(5 marks)

ii)

Write a short problem scenario that addresses how a user might

download a PDF file from a web site.
(10 marks)

Answer Pointers
a)
The main characteristics are that scenarios are used to exemplify typical user
behaviours. Scenarios are used to exemplify users and stakeholders, typical
behaviours, special situations etc. (3 marks)
There is also an emphasis on activity, information and interaction. The
approach value adds UCD but does not replace it. (2 marks)
b)

Essentially the inclusion and use of scenarios is all about improving

communications with users. It enhances the elicitation of information from
users but also facilitates user involvement in the design process. Anyone can
write a scenario. Scenarios can be regarded as a communications bridge
between users and designers.

c)

i)

A problem scenario is a short verbal description that describes a typical

situation or typical usage. (2 marks)
The word problem in problem scenario is not necessarily negative but
refers to problem domain. This can be misleading. (3 marks)

ii)

Open ended and any reasonable attempt will be awarded with full
marks. As an example, a typical scenario is provided below:
Scenario 1: Satoshi taking the TOEFL
Satoshi had just received his SAT score in the mail. He was not quite
satisfied with his verbal score so he decided to talk to his counsellor
Leilani one day. Knowing that English was his second language, Leilani
recommended Satoshi to take the TOEFL (Test of English as a Foreign
Language). Satoshi did not know much about the TOEFL, so Leilani
gave him the exam booklet as well as a TOEFL study guide in her
office. Satoshi thanked her for the help and took the study guide home.
Because he did not know any of his friends taking the exam in the near
future, he would have to study the exam by himself.

Examiners Guidance Notes

While a number of candidates provided reasonable answers there was generally a
lack of depth and detail. A common issue was thinking that problem scenarios must
refer to negative issues because of the word problem. This is not correct as in the
context of scenario-based design, problem refers to problem domain so a
problem scenario covers ordinary usage and issues as well as problems. Some
answers demonstrated generally that in order to attain high marks, candidates must
study and be prepared.

B6.

A games company wants to design a new hand-held console that will allow it
to compete with mobile phones. The new console will allow users to play
games but will also enable users to search and purchase from eCommerce
web sites. The company want to adopt user centred design for the
development of the new hand-held console.
Storyboards originate in the film industry where they are used to plan scenes.
However, storyboards are also a useful way to design user interfaces and
human-computer interaction.
a)

b)

Briefly explain the advantages and disadvantages of using storyboards

for designing user interfaces and human-computer interaction.
(10 marks)
Write a storyboard with 4-6 frames that shows a possible user interface
and interaction for the new hand-held games console. The device will
use interaction through touch and gesture and will not have a keyboard.
The storyboard should show possible interface layouts and interaction
based on this scenario:
A user has just bought the new games console, taken it out of the box
and switched it on. When the console starts up it begins with a screen
that asks the user to provide information such as Users Name, Users
Age, Address, Telephone Number and Email Address. The user must
provide this information before moving to the next stage.
[Note: your answer should focus on interaction and dialogue, NOT
on layout.]
(15 marks)

Answer Pointers
a)
The main advantages of storyboards are their visual clarity and hence they
are easy to understand (2 marks). These factors make storyboards very
powerful tools for communicating with users (2 marks) and the basis of
evaluation studies (1 mark).
The main disadvantages of storyboards are that there is a great deal of
variation in how they are produced no agreed formalism exists or is agreed
(2 marks). While storyboards can provide a detailed picture of both layout and
dialogue they dont provide a strong bridge between design and
implementation (2 marks) another formalism (e.g. UML) is needed (1 mark).
b)

Unseen. Synthesis & problem solving

Answers will be assessed and marks awarded on the basis of the candidate
producing a storyboard that is reasonable in terms of interaction, dialogue and
layout but with least emphasis on visual look and feel and layout.
The key elements that earn marks will be clear labeling of frames, logical and
reasonable transitions that are clearly labeled. Overall annotation is also

important, i.e. justification. General readability and clarity will also be

rewarded.
Examiners Guidance Notes
Many candidates provided good answers to this question, in particular well designed
and presented storyboards. Other answers demonstrated a lack of study and
preparation. As with all questions, candidates need to be well prepared and there is
much material generally available as storyboards are widely used. To be awarded
high marks for storyboards candidates need to ensure that frames are well labeled
as regards objects and transitions. It is better to annotate heavily than to produce
sparse descriptions of frames, their content and transition conditions.