Académique Documents
Professionnel Documents
Culture Documents
Date:
To:
From:
Subject:
Attached is the full report for our usability study, Evaluating the Usability of TIDAL: A
Recommendation Report. As of November 22nd, we completed the four tasks associated with
our study which included: identifying TIDALs iconography, navigating to a specific song and
listening to it, creating a playlist with previously unheard holiday music, and finding similar
artists based on a favorite artist.
To perform these tasks, we conducted heuristic evaluations individually and then collectively
as a team. We followed these heuristic evaluations with a preliminary usability study plan after
receiving critique and feedback we produced a final usability study plan. Similarly, we created a
preliminary usability test kit and once we received feedback created a final version of our
usability test kit. At this point we conducted our usability study with eight participants followed
by a preliminary analysis report. After presenting our study to a group of our peers we created
the final recommendation report which can be found enclosed.
Our findings concluded there were three key areas TIDAL should focus on to improve the
usability of their web-based music streaming service. The first being a general confusion of
what the iconography found on TIDALs web-based platform represented. Icon
misidentification from our participants ranged from 38% to 100% depending on the ambiguity of
the given icon. The one exception was the video icon which had a zero percent
misidentification rate from our participants. The second key area identified for improvement
was the My Music section which confused participants as they did not completely understand
what My Music represented, purchased music, favorite music, etc. This section of the service
also had two Playlists sections which further confused participants as they were unsure as to
why two sections had this same label. The third key area of improvement was identified as a
general mismatch between the platforms information architecture/navigation and users
mental model. This was identified as participants looked for information where they expected
to find it but were disappointed to find that information was found elsewhere on the service.
Based on our findings, we recommend TIDAL to introduce more intuitive and consistent icons,
provide greater differentiation of My Music from the rest of the left navigation menu, and
providing the ability for users to find content through multiple click paths and not only one click
path.
Abstract
Evaluating the Usability of TIDAL:
A Recommendation Report
Prepared by:
On October 5th 2016, Dr. Elizabeth Sanocki, Senior Lecturer at the University of Washington,
Seattle, approved our proposal to evaluate the usability of TIDAL by taking participants through
a series of tasks focused around previously identified usability heuristics. To perform this
research, we created a usability study plan, a usability test kit, conducted participant field
studies, and ultimately presented our findings to a group of our peers. These findings included
unintuitive iconography, unclear My Music section on the left navigation menu, and a
mismatch between participants mental models and TIDALs content and/or navigation.
Specifically, we found icons that yielded an identification error rate of 38% for the Playlist
icon, 38% for the Article icon, 50% for the Artists icon and 100% for the Genre icon. In
terms of the unclear My Music section we found participants were not sure if this section was
reserved for purchased music or favorite music. All of the participants also had a problem with
the two Playlists sections on the left navigation menu and could not identify why there would
be a need to have two or how the two were differentiated. We recommend TIDAL to utilize
more intuitive and consistent iconography throughout its platform to ensure easy
understanding of what each icon represents. Moreover, we recommend TIDAL further
differentiate the subsections of My Music to facilitate easy understanding of its main
purposes. Lastly, we recommend TIDAL to provide multiple click paths to reach important
content such as Similar Artists instead of the current single click path as this makes it difficult
for users to find and explore everything TIDAL has to offer.
Keywords: TIDAL, usability, mental model, iconography, sub-genres, breadcrumbs, music
streaming, navigation,
Table of Contents
List of Illustrations
.......4
Executive Summary
........5
Introduction
.......6
Research Methods
.......7
Results
....1 2
Iconography
......1 2
My Music
.....1 5
Mental Model Mismatch .....1 7
Backwards Navigation ......2
0
Subgenres
.....2 2
Conclusion
.....2
4
References
.....2
5
3
List of Illustrations
Figure 1: Featured Panel
..1 2
Figure 2: Illustration of Icons
.....1 3
Figure 3: Display of Genre Icon
............14
Figure 4: Highlight of Duplicate Playlists Sections
.....16
Figure 5: Highlight of Duplicate Albums
....1 9
Figure 6: Navigation Bar
.........20
Figure 7: Christmas Music Search Results
...2
3
4
Executive Summary
On October 5th
2016, we, a group of four University of Washington students, obtained approval
to conduct a usability study of TIDAL, the music streaming service. TIDAL is a subscription
based music streaming service that combines lossless audio, HD music videos, and curated
editorials to provide a unique music streaming service. TIDAL has struggled as a service to
increase and retain its user base, our study looked at various aspects of the service to identify
the greatest issues affecting TIDALs user base. Some of these aspects included the ambiguity,
organization and accuracy of TIDAL through an evaluation of its web-based platform.
To perform this research, we familiarized ourselves with the service and began evaluating how
the service approached navigation, content and music information. Then, we formally
conducted heuristic evaluations to identify the specifics of the services usability problems.
We found one of the biggest usability problems participants faced initially was false
identification of the iconography used throughout the music streaming service. Specifically, the
genre icon had a participant identification error rate of 100%, the artist icon had a 50% error
rate, the playlist icon had a 38% error rate as well as the article icon. The only icon that was
accurately identified was that of the video icon which also seemed to be the only intuitive icon
5
depicting a video camera which certainly helped participants accurately identify it. We also
found a significant mismatch between users mental model and various navigation and content
elements on TIDAL such as the two Playlist sections on the left navigation menu, the Similar
Artists tab being at the top of an artists page vs the bottom where participants expected it to
be.
Based on these findings, we recommend more intuitive and consistent icons be utilized
throughout the music stream service. We also recommend a greater differentiation between
the My Music section and the rest of the left navigation menu. Lastly, we recommend
important content such as Similar Artists be found through various click paths and not strictly
one to provide users a better chance of finding this important content.
Introduction
Product and Feature Overview
TIDAL is a music streaming service that provides High Fidelity sound quality, along with High
Definition music videos and music reviews curated by music journalists. The purpose for this
study was to evaluate the ambiguity, navigation, and organization of the desktop (website)
version of TIDAL. We did this by providing tasks to build a playlist and prompt navigation of
different sections of the site.
Key Tasks
We analyzed TIDALs interface by asking users to browse through the web apps main
pages. We developed tasks that took participants through the main features and
functionality of TIDAL. For example, we asked them to identify the icons on TIDALs
home page, we asked them to choose a song from a specific artist, we had participants
create a holiday playlist of songs, and finally we had them look up artists that they liked
and listen to artists that TIDAL suggested for them based on their preferences.
Usability Questions
Our tasks and our study as a whole revolved around three main usability questions. These
questions were:
Research Methods
Our group previously identified in our heuristic evaluations that there were many problem
areas with TIDAL concerning its interface including extraneous registration screens, ambiguous
icons, information organization, behavioral bugs, etc., and we wanted to test these aspects with
potential users to validate/invalidate our findings. We conducted this study to come up with
design recommendations for improving the way people navigate and discover music on
TIDALs online web platform. We are providing these recommendations to TIDAL stakeholders
in order to help TIDAL become a rising contender in the competitive music streaming market.
Research Questions
1.
Ambiguity: How well can users recognize iconography? Will they correctly interpret the
icons on the Featured section?
2. Organization: Is Tidals content organized in a way that matches users mental model?
Will users be able to effectively navigate to their destination?
3. Accuracy: Will users effectively discover music that appeals to their preferences as part
of their music streaming experience on TIDAL?
Participant Characteristics
7
We looked for participants that were both familiar and unfamiliar with using music streaming
services. Unfamiliar users were defined as those who listen to music streaming services less
than 5 hours/week, while Familiar users were individuals that use a music streaming service
more than 5 hours/week). We recruited 4 participants for each group, leading to 8 participants
total. We also made sure to find users that listened to artists featured on TIDAL in order to test
the validity of TIDALs artist suggestion function.
Study
We used Morae and Morae Observer to record user sessions and to gather data on the amount
of time to complete tasks, number of screens to complete tasks, etc. Screen recording was a
main method of recording data. It was used to gather the efficiency of navigation of our
participants by counting the number of pages it takes for the participant to complete a task.
The recordings were also used to measure the efficiency of tasks by recording the amount of
time it took participants to complete a task and recording the amount of time a the participants
spent on each page. Time was disregarded for tasks that did not measure navigation
efficiency, such as when users are asked to interpret icons or count the number of suggested
artists that they like.
Screening
1.
Participants were recruited based on if they fit the characteristics that we discussed in
the Participant Characteristics section, and we set up a meet time that works for them
and the group.
Welcome
1. Upon arrival, participants were be greeted and eased into the evaluation area.
2. They were be reassured about the nature of the evaluation, more specifically, that it is
the product they will be using that is being tested, rather than them.
3. We explained to them about how the session will play out. Namely, that they will be
given separate note cards with different tasks on them. We then had them complete
these tasks and then talk about their experience with them in a post-session interview.
Tasks
Post-Test Interview
Task Lists
1.
Exploration of Icons: Explore the home page and take in the icons. As you go, identify
the icons you find.
a. Prompted to main Page.
b. Scroll over Features.
c. Notice that all features have icon on the top right.
d. As you scroll, name the icon you see on each feature and what you think the
icon indicates. You do not need to repeat naming and identifying an icon if it
shows up multiple times in the cue. Only once is fine.
2. Your friends are all raving about a Beyonce song that was released last year. They all
insisted that you go and listen to Beyonces Hold Up. You finally give in and decide to
go listen to the song.
a. Navigate to the Tidal website then,
b. Find Hold Up by Beyonce (example song).
c. Play Hold Up by Beyonce (example song).
3. Your friend was playing Christmas songs when you were visiting their place. You didnt
like the Christmas songs being played. Create a playlist of three of your prefered
Christmas songs and play them.
a. Use the site to find your preferred songs
b. Add songs to a playlist. Name this playlist Xmas Faves
c. After all songs are added, play the playlist.
4. You are beginning to become bored of your music, but you are interested in finding
similar songs. You decide to stretch out to find some new music.
a. Begin from the artist page that corresponds to your favorite artist mentioned in
screening.
b. Find Similar artists.
c. Count how many suggestions for similar artists there are, and tell us the number.
d. How many of the artists prompted have you listened to and enjoyed? Tell us the
number.
Study Design - Users carried out the tasks in the same order. The tasks do not necessarily
build upon each other, but participants may have become more familiar with the platform as
they continued to use it. For example, we asked users to identify icons first before they were
exposed to them throughout the site. To avoid similar issues that may come up, we prioritized
basic tasks first.
Background Questions:
1. Ask what music streaming services they have experience with/currently use.
2. If they currently use music streaming services, ask them how many hours they use them
per week.
3. Find out what kind of music they listen to, including genres, artists, songs, etc. Develop
a list of content to verify that it is on TIDAL later.
Ask them to rate their experience with each task and with TIDAL overall (on a scale of
1-10) in terms of satisfaction.
Ask if there were any tasks or moments where they were confused about what to do or
where to go.
Ask about how it was finding songs and accessing their playlists.
Ask about their thoughts on the icons they identified and saw during their session.
Take them back to some of the pages that they navigated to and ask about their
thought process when going through the pages and carrying out the tasks.
Ask about what they think about the content layout of the main page, sidebar, and
subpages.
Facilitation Approach(es)
We had users test the product individually. We seated users at the PC and gave them their first
task to complete. We then started recording their interaction and timed how long it took them
to complete the task once they were ready to begin. We recorded their interaction time and
observe the route they took for each task. We did this for each task and ended each session by
talking to users about their thoughts on the experience. We asked them about any
10
observations we made during their session and had them share their opinions about certain
parts of TIDALs features and UI. This helped us to obtain qualitative information about the
motives behind their actions and the factors that influenced their thoughts. In combination with
the rest of the data we collected, we gained a better understanding of how TIDALs layout
impacts the user experience.
Data Collected
Through our study, we collected various forms of data including both quantitative and
qualitative data. The quantitative data we collected includes: time on each task, page
click-throughs, number of errors in icon recognition exercise, similar artist suggestion accuracy
percentage, and likert scale ratings concerning the effectiveness of TIDALs UI. In terms of
qualitative data, we obtained demographic information, familiarity with other streaming
services, pain points they experienced during the usability study, what users enjoyed while
using the service and their overall thoughts of using TIDAL. By aggregating all of this data we
were able to get a more complete picture of TIDALs usability.
Data Evaluation
We then evaluated how successfully users, both familiar and unfamiliar with streaming music
applications, found content within the TIDAL desktop application. Symbol ambiguity was
assessed based on each users ability to correctly interpret TIDALs icons. Organization
success was determined by the number of screens users navigated to and the time taken to
complete each task. And finally, TIDALs content association accuracy was evaluated based on
how many similar artist suggestions were in line with user preferences. Success was
determined by the number of screens users navigated to while completing navigation tasks,
the amount of time taken on the each task, and correct recognition of icon functionality.
Information from post-test interviews was used to gather qualitative data from each user. We
then combined our qualitative and quantitative user data to determine how well users were
able to navigate TIDAL, as well to identify certain issues with TIDALs interface.
Reporting
We utilized the structure of the class for most of our reporting which includes: a project
declaration, project proposal, heuristic evaluation, usability study plan, usability test kit,
preliminary analysis, final presentation and final report. Each of these items are intended to
achieve specific usability study goals. The project declaration was intended to give us a
general idea of the problem space we were thinking about working within. The project
proposal was a more in depth look at the problem space and the specifics regarding our
plan/approach in addressing TIDAL as a service. The heuristic evaluation allowed us to define
our users and their goals along with defining the tasks they would attempt. The heuristic
11
evaluation also gave us the opportunity to complete walkthroughs, taking tasks step-by-step
through the users point of view to help us identify any problems based off a set of
predetermined heuristics.
The usability study plan was a detailed plan outlining the specifics of our study, so much so,
that another researcher could use it to execute a study comparable to the study our team will
be doing. The usability test kit includes the complete set of non-product items we will use to
interact with participants during our study. The preliminary analysis includes information
regarding our study once they have been completed along with brief descriptions of the
product tested, participants and preliminary analysis of our findings. The final presentation and
report include a complete synthesis and analysis of our study from beginning to end,
aggregated in a visually compelling and clear format to best depict our findings.
Results
After conducting the studies, we synthesized our data and identified the main issues that
existed across the usability sessions. We combined our notes and observations and coded
them by severity to figure out the most important and actionable problems to address. From
this process, we concluded that participants had issues with specific features on TIDAL.
Namely, participants were unable to identify the meanings of icons in TIDALs Featured Panel,
they had trouble interacting with TIDALs My Music section, the organization of content did
not match the Mental Model of several users, TIDALs navigation menu often went
unidentified and unused, and it was often difficult for participants to find music they liked due
to a lack of subgenres.
Iconography
Iconography is used throughout TIDAL to mark different sections of the main menu for
navigation, and used to mark media on the Featured Page. For the Featured Page, icons are
shown on the right side of each content panel, without any text describing what the icon is
(Figure 1). In order to assess how intuitive these icons were at categorizing TIDALs content, we
had users manually scroll through the features panel, and verbally identify what they thought
each icon meant and why.
12
Figure 1. Featured Panel in TIDAL including icons displayed on the top right of every panel.
Issue Statement: Users had trouble identifying what icons in the Featured Panel
stood for, and sometimes categorized multiple icons as the same thing.
Figure 2. Illustration of Icons on Featured Panel, with their actual meanings above and error rates and subject
identification below
The other two icons, Playlist and Article, had 3 out of 8 participants incorrectly identify the icon
meaning. Because both of these Icons had a similar visual of a document, people often
identified these icons similarly. 2 participants thought that the Playlist icon could be an article,
and visa versa. The music note on the Playlist icon seemed to cause confusion as to what
exactly it contributed, give that TIDAL is a music streaming site.
The icon that all eight participants correctly identified was the Music Video icon. The music
video icon was noted by a video camera (shown above). All participants tried to use context
clues by reading the information displayed on each panel before identifying the icon meaning.
13
Figure 3. Display of Genre icon alongside an actual Album Title on Featured Section.
With this kind of inconsistency, participants were confused about the Genre icons actual utility,
especially since the visual panel informed them of an Album and not any Genre related
material.
14
Since the Playlist and Article icons looked similar, this was also cause for redundant
identification responses. Both icons had a document format visual to distinguish it from other
icons, and because of this both gave way to user error.
Iconography Recommendations
The main recommendation would be to redesign icons so that the meanings are more intuitive
with user expectation. We can do this by looking at other music streaming such as Spotify,
iTunes, and Pandora, and seeing how aspects like Genre and Artists and commonly portrayed.
Icons should also only be used with content it is associated with. So Genre icons shouldnt be
used to categorize albums, only genre-related information. Also finding a way to distinguish
Playlist from Article visually would also minimize redundant identification. If all of these are
incorporated, it would limit misinterpretation of icons and the content that is identified through
these icons.
My Music
The My Music section on TIDAL holds music the user favorites. The My Music section has
five subsections: playlists, albums, tracks, videos, and artists. Each of these subsections begin
empty and filled by the user adding content to them by favoriting content in a number of ways.
The My Music is located at the bottom of the left navigation area of the home page of TIDAL.
Our study revealed misunderstandings with the My Music section that lead to user errors in
navigation.
15
Our study also identified the information that was stored in the My Music section to be
ambiguous to our participants. The ambiguousness of what was in the My Music section was
explored in task 3 and our post study interviews. Participants were confused by the option to
click the star located on the album art and the right side of a track. Clicking the star brings up
an alert box that simply says the item starred has been added to My Music. But for our
participants it was unclear where in My Music the starred selection was added. There are five
option as to where the item could be located; six if you include the play queue.
My Music Recommendations
For the My Music section as a whole, the location was not the biggest detractor and main
cause of confusion. The location fits within the design scheme of the website well. Moving the
My Music section would immediately differentiate it from the rest of the navigation bar
located on the left of the page, but such a move would ultimately have the main navigation
located in two different areas of the page. Thus, moving the My Music section is not
recommended. We recommend three options: renaming the My Music section, indenting the
My Music section, and making the My Music section collapsible. The first two options are
the most important. Renaming each of the buttons in the My Music section to include the
word My before each categorical name will address the confusion with there being two
playlists in the left navigation area. This would differentiate the two playlists, one being the
playlists of TIDAL and the other being the playlists belonging to the user. Secondly, indenting
all of the text for the buttons in the My Music section would help the user to intuitively
differentiate the My Music section at a glance. This will help users more quickly understand
what is on the page without having to read the text in the buttons. The third option, was to
make the My Music section collapsible. Two out of the eight participants mentioned they
would like this capability. Making the My Music collapsible is not an immediate need to help
distinguish the My Music section of the navigation bar, but it can help users navigated the site
better by not overloading the user with extra information. Therefore, a collapsible My Music
section is a secondary recommendation to renaming the buttons to include the word My
before each category and indenting the text of each button in the My Music section.
In connection to renaming the My Music buttons mentioned previously, the alert boxes that
pop up when music has been starred can be more descriptive to tell users where exactly
information can be located will help to eliminate confusion with what is being held in each of
the My Music categories. We recommend the alert boxes say directly where information is
being stored. Instead the alert boxes simply saying, Added to My Music, the alert should say,
Added to My Tracks or Added to My Artists, depending on which categorical button the
information will be stored. The direct wording of the alert box will tell the users where their
music is going and help build a strong mental model of the website.
interaction. And they also identify many of the usability problems they see are from
mixed-up models. We are using mismatch to describe the mixed-up models of the Nielsen
Norman Group.
Issue Statement: Participants expected to find related artist information near the
bottom of the page.
18
refining their search terms. This did not stop users from completing their tasks but it did add
unnecessary steps for them.
guessed as to which option was the one they wanted. In the secondary situation, search results
were not tailored to match a specific search. Instead, the results followed a rigid hierarchy.
Matching the mental model of users can eliminate the thinking currently needed to sift through
search results.
Backwards Navigation
20
Issue Statement: Users could not figure out how to utilize TIDALs navigational
breadcrumbs to go to previous pages.
computers display folder-traversing history, it may not be natural or expected for a music
streaming service. Users did look in the top-left corner for some way to go back to a previous
page, so the placement of the menu is intuitive, but the way the menu itself is presented is not.
Subgenres
A genre is a category used to group similar media based on certain characteristics. In the case
of TIDAL, music is categorized by the normal categories like, Jazz, Hip Hop/Rap, R&B/Soul,
Classical, Rock, Metal, etc. A subgenre is a subcategory of the first categorization of media.
Currently, TIDAL does not support subgenres. Subgenres can help users parse through larger
categories in order to find specific content.
why they chose that particular album the participant, much like before, responded by saying
that it was first one listed. Participants had no way to break down this large holiday music
category.
Subgenres Recommendations
The main recommendation for TIDAL is to add sub-genres to help users find the type of music
they are looking for. The addition of sub-genres in general album search results can minimize
the subsequent searching for users, which would also create better flow through the site.
Participant 1 even mentioned after task 4 that they would have like to see subgenres of listed
tracks in order to decide for themselves if the music was similar to their favorite artist. Although
23
adding subgenres does not give users direct control over the search results being displayed, it
does give the users control over the content they explore. The simple addition of sub-genres
will help TIDAL users feel empowered when navigating the site.
Conclusion
From our studies, we were able to determine key usability findings related to TIDALs UI and
specifically its organization, accuracy, and ambiguity issues. Our key findings involved icon
ambiguity, redundancy with TIDALs My Music section, the organization of content was
inconsistent with users Mental Model. Additional findings included that TIDALs navigation
menu often went unidentified and unused, and there was difficulty for participants to find music
they liked due to a lack of subgenres.
For the iconography, it was recommended that TIDAL look at existing music streaming services
to find more intuitive icons for labeling their content, and also find a way to distinguish
redundant looking icons. Distinguishing the My Music menu from the main menu through
highlighting or collapsable capabilities could help users distinguish which menu has their
content, and which one contains TIDALs inventory. Additionally, Adding My to all of the
subcomponents of the My Music menu could also help visually indicate a difference to users. In
order to be consistent with users mental models, Similar Artists should also be located in the
bottom of the page so it is in line with user expectations. Also search results should aim to be
tailored to specific user preferences and search frequencies. Other key recommendations
include differentiating the navigation menu from the rest of TIDALs content on their website
and also incorporating various click paths for navigation, as well as including sub-genres to
help user easily navigate a plethora of music genres and suggestions.
By incorporating our design recommendations to address their organization, accuracy, and
ambiguity issues, TIDAL can improve the user experience of their service and allow for a more
streamlined UI. With these improvements, TIDAL can be a standout competitor in the music
streaming market, and potentially attract a wide variety of users as well.
24
References
Nielsen, J. (2010, October 18). Mental Models and User Experience Design. Retrieved December
13, 2016, from https://www.nngroup.com/articles/mental-models/
SCRIPT
Thanks again for agreeing to participate in our study. Ill be guiding you through the session
today along with __________, who will also be taking notes as we go along. Ill be reading off
of this opening script to ensure that every participant is introduced to the procedure in the
same way.
The purpose of this study is to see what people think about TIDAL, an online music streaming
service that you may or not be familiar with. One important thing to understand with is that we
are not testing you and your ability to go through the tasks, were just studying TIDAL itself. So
you shouldnt feel pressured as you use the site because theres no wrong way to do it.
None of us had any part in designing the product so we wont be offended if you find it difficult
to use. We are students trying to analyze TIDALs design and find ways to improve it so we
encourage you to share your honest thoughts.
25
If you have any questions during the study feel free to ask us. We may not be able to help
immediately during the study, since were interested to see how people use TIDAL without
guidance, but we can try to answer them in-between tasks or after the session. Also, if you
need to take a break at any time during the study just let us know.
In case youve noticed we do have a microphone on and we will be recording what you say
during the sessions and in an interview afterwards. We will also be recording your screen with
your permission. If you are ok with this, we have a consent form for you to sign. The data we
collect will only be viewed by the people working on this project for reporting purposes. There
are four people on the team including me and ___________, and the other two will be
watching the session from another screen. Before I explain how the session will work, do you
have any questions?
Well give you a notecard relating to a single task. These cards will have scenarios and
instructions on them for you to complete.
You are allowed to read the information on the notecard before you start, and let us
know when you are ready to begin the task.
Let us know when youve completed each task so that we can give you another
notecard with another task to complete.
After youve completed all of the tasks we will then have an interview about your
experience throughout the session.
As you perform each task we encourage you to describe your thought process aloud. This
practice, known as think aloud, will help us to learn a lot more about your experience than
your actions can. Are you comfortable with doing this?
To practice this, wed like you to go through a quick thought exercise for us. Are you ready?
Ask them to count how many windows are in their home, and to take us through their
mental process.
Thank you. Hopefully that helped you to warm up. If you forget to think aloud during the
exercise, we may remind you every now and then.
Do you have any questions before we get started?
26
27
28
29
30
31
32
33