Executive Summary

Our testing of the YMCA Twin Cities website focused on the About Us portion of the left navigation bar, which consists of the links labeled About Us, Community Programs, Donate, and Volunteer. Using the information available through these links, as well as information we inferred that users might believe to be available through them, we developed eight scenarios. The evaluation process consisted of these scenarios and a background questionnaire. In order to make meaningful, effective recommendations, we gathered both qualitative and quantitative information during the evaluations. Our scenarios were designed to answer the following questions: How well does the content of the About Us section match users expectations? Can users find specific community programs? Can users find specific information about the YMCAs policies? Can users determine that Community Programs includes scholarship information?

Evaluations were conducted in the Usability Testing Laboratory at the University of Minnesota with eight participant evaluators, adults of varying ages living in the Twin Cities area. The information and observations recorded during these sessions yielded the following results: Participants generally found the homepage to be attractive and easy to read. Participants appreciated that the Google search toolbar was available if they got stuck. The extent to which they used it varied widely; some never used it, while others used it in over half their tasks. After completing scenarios in which they were required to visit branch-specific sites, participants rarely realized that the home label on the upper navigation bar would take them to the branch-specific home page, and not to the YMCA Twin Cities home page. Some became confused that the home page looked different. Few realized that the logo in the upper left corner was a link to the home page or saw the link labeled home at the bottom of each page. Participants were able to find addresses and contact information for specific YMCA locations without difficulty. Participants were generally able to find the YMCA mission statement without difficulty, although rarely immediately. Participants were generally able to find information on the Ys Start program, but did not find it easily.

Participants were generally unable to find contact information for programs offered through the University YMCA. Every participant started this task by going through the Volunteer link. Participants were generally willing to spend much more time looking for information on disability accommodations at a particular branch than for any other information that they did not ultimately find, which could indicate that they expected it to be available. Participants were generally able to find the YMCAs privacy policy regarding credit card information, but were not always able to find it immediately. Participants were generally able to find information on scholarships toward membership costs at the YMCA, though were often unsure about how to apply for them.

Based on the above and other findings, we would like to offer the following recommendations: To make the information under the headings more accessible, menu headings should have drop down boxes so that users are able to see the subcategories within each category of the menu, if applicable. Even though the label Kids, Teens + Family leads to a list of classes, it should have a quick link to information on the Ys Start program, since the path is logical for a majority of the evaluators. Since Volunteer can be found under About Us, information about tutoring should be in this heading as well. Privacy Policy information about payment methods should also be included under Membership, since those most likely to sign up for a membership would need information about how secure their information is going to be. Instead of putting general information on disability accommodations under About Us, users should be able to determine the disability accommodations for a YMCA when they select a location under Locations + Hours. Though quick links for Scholarships/Pricing Plans are available under Membership and About Us, the wording may be better changed to Financial Assistance, Scholarships + Personal Pricing Plans.

Table of Contents
Executive Summary Purpose Research Questions and Objectives Target Participants and Actual Participants Methods Overview Results and Findings Recommendations Appendices Test Plan Data Worksheets Signed Consent Forms 1 4 4 4 8 10 14 15

Purpose
The purpose of this usability evaluation is to measure first-time users ease for distinguishing and locating information on after school and community programs on the www.ymcatwincities.org web site. Is the YMCA website taskbar organized to be intuitive for a novice internet user to distinguish between what programs are listed under after school and community?

Research Questions & Test Objectives

The purpose and research questions for this usability test are listed below.

Team Obvious Purpose: How well can users understand which information is located under the About
Us section?

Research Question
RQ1: How well does the content of the About Us section match users expectations? RQ2: Can users find specific community programs? RQ3: Can users find the YMCAs policies? RQ4: Can users determine that community programs includes scholarship information?

Objectives
Users expect to find contact information, hours, mission statement, and a location for a specific YMCA within 45 seconds. Users can find information about the Ystart program within 45 seconds. Users can find information about the YMCAs Disability Accommodations within 45 seconds. Users can find information about scholarships through the YMCA within 45 seconds.

User Profile
Target Participants
For this evaluation, we will use eight to ten participants who live in the Minneapolis/St. Paul area. Eight participants will actually participate in the tasks, while two will remain on call in case a scheduled participant cannot make it to the evaluation. The evaluation will take place on March 27, 2010 from 12:00 pm to 4:00 pm, and on March 28, 2010 from 5:30 pm to 9:30 pm at the Usability Lab on the U of M East Bank campus. Each participant will have an hour to complete the evaluation. We estimate the target audience for ymcatwincities.org to be a diverse group of users from different demographic categories that include education, age, gender, geographic location, 4

language, socioeconomic status, online experience, frequency of YMCA use, and family size. Our estimated distribution for YMCA users is outlined in the Figure 1.1. We expect that the general audience for the website will consist of many different types of internet users and should be designed for the lowest common denominator of those users; the novice who has little to no website/online experience (see Figure 1.1). Figure 1.1 [enter label here] User characteristic Education Range 4th to 10th grade education High school Some college Bachelors Masters degree or higher 10 to 17 18 to 40 41 to 60 61+ Male Female Urban Suburban Rural English Spanish Hmong Somali $10,000 $15,000 annually $15,001 $25,000 annually $25,001 $40,000 annually $41,001 $60,000 annually $60,001 $80,000 annually $80,000 and above annually Daily (many websites) Daily (few websites) Weekly (many websites) Weekly (few websites) Monthly Biannually Daily Three times per week Once per week Once per month Twice per year Percentage 5% 30% 20% 40% 5% 25% 35% 30% 10% 50% 50% 30% 50% 20% 75% 15% 5% 5% 15% 15% 20% 25% 15% 10% 35% 20% 25% 10% 5% 5% 5% 40% 40% 10% 5%

Age

Gender Geographic location Language

Socioeconomic status (household income)

Online experience

Frequency of YMCA use

Number of people in your household

1 2 3 4 5 6 7 8+

15% 15% 15% 15% 15% 15% 5% 5%

Actual Participants
Highlights from the participant background questionnaire: All participants lived in a suburban neighborhood All participants were Caucasian All participants reported English as their primary language None of the participants were members of the YMCA Only participant 6 had previously used the YMCA website The charts and graphs below highlight the areas with the most differential changes per question:

Participants
4 3 2 1 0 Male Female Participants

4 3 2 1 0

Reported Net Use

Self Descriped Internet Use

Education
1 2 Bachelors 1 5 Graduate Degree High School 1 1

Income
$80,000 + $25,001 40,000 $9,999 <

Internet Use Frequency

weekly-few sites weekly 1 3 daily-many sites daily-few sites

1 3

Methods Overview
The ymcatwincities.org usability test consisted of a background questionnaire to discover where the test subjects fit in our user profile, eight scenarios to test the users ability to understand where information is located on the website, and debriefing questions to establish the users initial reactions to the website and subjective suggestions for its improvement.

Quantitative & Qualitative Results

Average Time Spent Per Scenario
200 180 160 140 120 100 80 60 40 20 0 173.625 139.5 113 97 51.125 117.75 102.625 Seconds 130.25

S1

S2

S3

S4

S5

S6

S7

S8

Overall, the average amount of time spent per scenario far surpassed our expectations of finding information in 45 seconds or less. Scenario number one, in which participants were asked to find a specific YMCA location and its contact information, was the only scenario to be completed relatively quickly in all cases, but is still around 6.125 seconds longer than wed like. A rule of thumb in the field of usability states that users will spend about 45 seconds or less searching for information on a given website. As you can see, scenarios two through eight averaged around two minutes before users either completed or gave up on the scenario. Therefore, we conclude that the information in those scenarios, which all could have been found under the About Us tab on the left navigation, is lost to the user. We suggest a total revamp of the About Us section.

Number of Issues Per Scenario

8 8 8 7 6 5 4 3 2 1 0 S1 S2 S3 S4 S5 S6 S7 S8 1 1 2 2 4 Issues

100% 8 7 6 5 4 3 2 1 0 S1

Completion Rates Per Scenario

87.5% 87.5% 75% 75%

50% Completed

0% S2 S3 S4

0% S5 S6 S7 S8

As highlighted in the graphs above, information in the About Us section is not easily located. At least one issued occurred in each scenario we offered, the most severe being scenario three (finding information on Y-start) and scenario five (finding disability accommodations information). The section that follows highlights the scenarios in regards to their corresponding research question. We will point out scenario specific problems, site problems, and make recommendations.

Results Per Research Question

Research Question 1: How well does the content of the About Us section match users expectations?
Scenario 1: You are an avid user of the YMCAs Shoreview location but are moving and would like to check out their Woodbury location. Find the YMCA Woodburys address and contact information. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done. Scenario 1 Results: All participants were able to complete the task and find a specific YMCA location rather quickly and effectively (see graph below). The position of the Locations button on the left navigation is appropriately placed. However, many users did hover their mouse over the About Us section while browsing for the locations. Additionally, the pop up menus including hours and contact information on the locations map is appropriate for quick referencing. We suggest no change for this particular scenario.

Completed
8 6 4 2 0 S1 S2 Completed

Scenario 2: You are interested in joining the YMCA in your area but want to know if their mission statement involves Christian principles. Find the YMCA mission statement to see whether it does. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done. Scenario 2 Results: The Mission Statement is currently located under the About Us tab on the YMCA website. Most participants, except for one, were able to locate the Mission Statement. However many participants did not understand that About Us was a clickable link, resulting in Google searches and random clicks, causing the average time for completion to be 1 minute and 53 seconds.

10

Research Question 2: Can users find specific community programs?

Scenario 3: Your neighbors daughter is enrolled in the Ys Start program. She speaks highly of it and you would like to know if it is a good fit for your child. Find information about the Y-start programs focus. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done. Scenario 3 Results: As seen in the graph below, none of our participants could find the Ys Start program. This indicates a significant problem with the Ys Starts categorization under the Programs tab. Because Ys Start is a program for school-age children, it would be more suitable the title of School-Age Children, not under the general Programs tab. Once again, the Programs tab looks like a heading, and not a clickable link.

Completed
8 7 6 5 4 3 2 1 0 S3 S4

Completed

Scenario 4: You are currently attending the University of Minnesota and are interested in tutoring at the YMCA. Find out who you should contact to become a tutor at the YMCA. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done. Scenario 4 Results: Only four of our participants completed this task. The Y-tutors program is a volunteer program. It should be located under the volunteer link under the About Us tab. However, the volunteer link lacks any concrete links to specific volunteer programs. We suggest recategorizing all volunteer programs and including them under the Volunteer link.

11

Research Question 3: Can users find the YMCAs policies?

Scenario 5: You are planning a party at the Elk River YMCA and are wondering if Grandpa Joe will be able to make it around the building with his wheelchair without problems. Find information regarding disability accommodations for the Elk River location. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done. Scenario 5 Results: Not a single participant completed this task (see results below). We consider this an egregious error. Disability accommodation is an equal access right issue and is pertinent to the YMCA and its mission. Currently, information on disability accommodations is only available by contacting a YMCA location. That information can be found under the Membership page, or under the About Us tab. We suggest including a handicap symbol in the pop-up menu for specific YMCAs that are handicap accessible. Additionally, we suggest disability accommodations be eliminated from either About Us or the Membership page to erase crossover and confusion. However, we must take partial fault for this question because we asked for a specific location which may have led our users astray.

Completed
8 6 4 2 0 S5 S6 Completed

Scenario 6: You are going to register your son for after-school childcare at the YMCA in your area, but are hesitant to disclose your credit card information online. Find the YMCAs privacy policy to confirm that they have a section on credit card security. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done. Scenario 6 Results: Most users were able to find this information; however, they required an average of 1 minute and 42 seconds to find it, makingmultiple clicking errors in the process. Users rights to privacy and company policies are a must. We believe this information is hard to find because it lies beneath the fold of the screen. We suggest putting the bottom links at the top of the homepage. Additionally, a link to credit card security should be added to all forms requiring credit card information.

12

Research Question 4: Can users determine that Community Programs includes scholarship information?
Scenario 7: Money is tight this month because of working fewer hours at your job. Find out if the YMCA has any programs that may allow for a more economical price-point given your financial circumstances. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done. Scenario 7 Results: Our participants had a hard time locating scholarship information, and only 75% did so successfully. The information is currently located under both the Membership Page (as a quick link) and under the About Us page. We believe the label Scholarships/Personal Pricing Plans does not register sufficiently with users, and that Scholarships/Reduced Rates/Financial Assistance could be a better term. Additionally, this information appears to be more pertinent under the Membership page and should not be included in About Us.

Completed
8 6 4 2 0 S7 S8 Completed

Scenario 8: You have heard that there are several requirements to getting assistance with membership costs at the Y, and you want to know learn how you can take advantage of this assistance. Find the requirements for the financial assistance programs on the YMCA website. Please think out loud. When you have finished the task, or believe the task is unable to be completed, please say, Im done.

13

Recommendations for the Site

How well does the content of the About Us section match users expectations? Several evaluators did not realize that the headings were links and commented on the lack of legibility with the current color scheme. To make the information under the headings more accessible, menu headings should have dropdown boxes so users are able to see the subcategories within each category of the menu, if applicable.

Can users find specific community programs? In general, users did not find the Ys Start program without using the Google search box because they did not understand that YsStart is a community program. Even though Kids, Teens + Family is a section for classes, it should have a quick link to get to YsStart since the path is logical for a majority of the evaluators. In general, users were not able to find information about tutoring within the About Us section, even though information to volunteer is available. Since Volunteer can be found under About Us, information about tutoring should be in this heading as well.

Can users find the YMCAs policies? Only one of the evaluators had issues finding the Privacy Policy for information about credit card security. Privacy Policy information about payment methods should also be included under Membership, since those most likely to sign up for a membership would need information about how secure their information is going to be. Task 5 advised evaluators to find disability accommodations at a particular location of the YMCA. Instead of putting Disability Accommodations under About Us, users should be able to determine the disability accommodations for a YMCA when they select a location under Locations + Hours.

Can users determine that community programs includes scholarship information? Scholarship often confused evaluators attempting to find information on financial assistance. Though Quick Links for Scholarships/Pricing Plans are available under Membership and About Us, a better phrasing may be Financial Assistance, Scholarships + Personal Pricing plans. Once the evaluators found Scholarships/Pricing Plans, they were often confused about how to submit applications materials. One evaluator commented that the option to complete applications online instead of visiting a location should be available.

14

[Test plan goes here. Since this is a rough draft, I thought it would get unwieldy if I added it to the same file. It will be exactly the same as the test plan we turned in, except with a few edits as recommended by Lee-Ann. We can talk about it. Also, the background questionnaires and signed consent forms will be added as appendices too, although obviously in the physical test results report only.]

15