Balanced Scorecard Application

Designed by Team One Ronald Chan Charles Bergman Trevor Obrien Alvaro Parra Supervised by: Dr. Ling Rothrock Kelly Sprehn

December 10

2010

This report describes the design process, calculations and business strategies for Kaiser Permanente, an esteemed health care insurance provider.

Contents
Executive Summary .................................................................................................................................. 3 Introduction ............................................................................................................................................... 4 Background ............................................................................................................................................... 5 Detailed Solution ...................................................................................................................................... 6 Organizational Sales Pitch ...................................................................................................................... 17 Conclusion .............................................................................................................................................. 18 Bibliography ................................................................................................................................................ 19

Executive Summary
Team One has been invited to re-design an interface that assists business executives in their daily operation while monitoring a companys progress, there are four main area of focus during the development of this application, they are:

Ability to detect problematic measures Ability to detect contributors of underperforming measures at the lowest level Ability to view variance reports for each measure Ability to view the status of any initiatives to improve performance After conceptualization, the first prototype was created. In the first prototype, the

interface employs a light blue color scheme and it is contrasted with grey boxes and black texts. This color scheme was selected in order to avoid fatigue during long usage period and it is aesthetically pleasing to most users. The texts of important measures are also color coded in order to notify the problematic areas, and the progress of each initiatives. Finally, navigational buttons were placed on the bottom of the screen for the ease of access, and they provide guidance to the users as they advance through the interface. After the first prototype was produced, the team conducted a usability analysis, which identified flawed areas of the interface. The usability analysis showed that only approximately 41% and 6% of users were able to find the variance report and the initiatives within the application, therefore the team focused on improving these two areas. The resulting interface (prototype 2) retains some of the features from prototype 1, such as color schemes and the buttons. The second prototype also created more buttons, instead of underlined texts that signify links, therefore the users can navigate through the application much easier.

Finally, with such a detailed project, Team one has decided that if the application is chosen, it will take 7-8 weeks to deliver the final product, with the cost of approximately 30,000 dollars.

Introduction
In todays fast paced and competitive society, business executives are responsible for numerous tasks in their daily operation. Therefore, in order to improve efficiency and efficacy, Kaiser Permanente, a healthcare company, prepares to launch an interface that is compatible to a mobile device. The newly designed application will assist business executives to review the status of the company, provide information on the underperforming measures and inform the executives of any initiatives. Due to demanding schedules of business executives, the application will also focus on efficiency; it will not only provide information needed, it will provide information with accuracy and speed. Ultimately, this application will monitor a companys performance while increasing productivity.

Background
Kaiser Permanente, an esteemed healthcare company, would like to purchase a newly designed application that would allow its managing executives to review the status of the company. In order to ensure the quality of the application, Kaiser Permanente had sent out invitations for bids to various design companies, and Team One was among the chosen candidates. Before creating the application, Kaiser Permanente has several specific guidelines for the interface, and it is outlines as follow: Ability to detect problematic measures Ability to detect contributors of underperforming measures at the lowest level Ability to view variance reports for each measure Ability to view the status of any initiatives to improve performance In order to create an application, the team followed an intuitive process. First, the team performed a task analysis, which includes hierarchical task analysis, in order to understand the needs of the customers. Secondly, the team conceptualized, or brainstormed, ideas that are related to the application. Then, a prototype of the application is created. To further improve the application, the team conducted a usability analysis, where different aspects of the application are ranked, and the lowest ranked attributes will be re-designed. Finally, the application will be tested again to ensure its quality.

After the design process, Team One will present the interface to their colleagues and business executives. These executives will then rank all the applications presented and choose the one best suited to their daily operation.

Detailed Solution
Before creating any interface, Team One must identify the navigation style of the application. This can be easily achieved conducting a hierarchical task analysis, also known as HTA. A HTA can help the team to identify the steps it takes a user to reach a certain level in the interface. Team Ones HTA is as follow:

0: Improve Company Status

1: Wait for program to load

2: Choose Deficit

3: Choose Metric to Improve

4: Determine the Variance

5: Determine initiatives

Click deficit button

Look for alarming color (Red)

Scan scorecard and look for variance

Scan scorecard and look for initiatives

Click underperforming measure

Prototype 1: To create the first prototype, four main goals must be satisfied. First, the interface must be able to select a problematic measure on the chosen scorecard, therefore the application must be able to transfer the information from the company website to the handheld device with coherently. Next, the interface must be able to recognize at the contributors of a given

problematic measure at the lowest level, therefore the interface should allow the users to select a measure quickly and easily to identify the lowest core contributor of a particular measure. The third goal was to be able to see the variance report of each measure. Finally, the interface must be able to monitor the status of any initiatives to improve performance. In addition, both the variance report and the initiatives must be presented in an efficient way. With all these goals in mind, Team One decided on a few ways to achieve these goals. First, the application employs a color coded system that is easy to understand. This was shown at the beginning of our application where the users have to select deficits or surplus scores. These were colored red and green respectively, as shown below:

Figure 1: A choosing screen where the user will choose whether to view deficit scores of surplus scores, it employs a specific color scheme.

This scheme was carried out throughout the rest of the application. Each underperforming measure was colored in red and the measures that were performing well are colored in green. In addition, the color yellow was used for the measures that did not meet the goal but were relatively close. This allowed the user to easily identify how each area of focus was performing with quickness and ease. The interface also had a color scheme for the background and text, it employs a light blue background to contrast a gray boarder and black text. These colors we found to be appealing to the users while decreasing avoiding fatigue during long usage period. This color was use consistently throughout the application; therefore users did not have constantly adjust to a changing color scheme. Font size was also considered during the design process since this is a work centered application. Team One decided that the size of the font to be 16 point for the standard text and 20 bold point font for the numbers shown in the variance initiatives reports. This decision was reached because in a touch screen handheld device, the users would be able to read and select items with their fingers easily and accurately. A screenshot of the application is shown below:

Figure 2: Sample screenshot of the interface. The variance report was another goal that needed to be met in this application. After the team decided on a basic structure, the layout of the content is designed. For the variance report, it was displayed with each measure that was shown on the screen. It starts with a basic overview of the variance while looking at all various areas at the start of the application. If the users advance to a deeper level of a particular measure, the detail increases by showing the goal, target, and actual values of the report. These numbers were also color coded according its performance and they were set at the 20 bold point font so that the users can clearly see and touch it in order to navigate throughout the application. Next, the team focused on the applications ability to identify the specific problematic measure and finding the contributor of underperformance to that measure at the lowest level. The

color scheme was used again to label each measure based on how each measure performs. For example, the red colored measure signifies a problematic measure, and click on the red button will direct the user to a lower level, which is a list of contributors to the underperforming measure. This link corresponds to the link mentioned before about the variance since they are next to each other on the application. As the user advance through the application by selecting the red colored measures, he can find his way to the lowest level quickly. To notify the user when he is at the lowest level, all the information will be presented. An example of the variance report and the initiatives is shown below:

Figure 3: Screen shot of a variance report. Finally, there are navigational buttons added on the bottom of each page to guide the user through the interface. These buttons will not only increase efficiency, it will decrease confusion while using the application. An example of the buttons layout is shown in figure 3.

Usability Analysis In order to test the effectiveness of our app, two questionnaires were used. The first questionnaires purpose was to determine how effective our app was for first time users to visually find the critical metrics of A problematic measure, contributors to underperformance at the lowest level, variance reports, and initiatives to improve performance. In order to test this, 4 random pages of our application were shown to an audience for ~10 seconds, to see if the audience could quickly and effectively find the critical metrics. The random pages were placed upon a grid such that the four slides were broken up into 8 sections (figure 1); the audience would write down the number that corresponded to the grid section in which they observed the critical metric. If the audience member was unable to find the metric, the number 9 would be written to show that. The results were analyzed in such a way that a correct grid number written down was given a 1, while an incorrect grid number was given a 0. From an average of the entire audience, a percentage could be calculated that estimated how many people could effectively find the metric in question. The results can be seen in table 1, and it shows that 87.5% of people could find a problematic measure, 66.7% of people could find the contributor to underperformance at the lowest level, only 41.7% of people could find the variance reports, and an abysmal 8.3% of people could find initiatives to improve performance

Figure 1: Questionnaire 1 design

Table 1: % of population able to quickly find the objective.

Q1 1 1 1 1 1 1 1 0

Q2 1 0 1 1 1 1 1 0

Q3 1 0 0 1 0 0 1 0

Q4 0 1 0 0 0 0 0 0

1 0 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0.875

1 0 0 0 0 0 1 1 1 1 1 0 1 1 1 1 0.666666667

1 0 0 0 0 0 1 0 1 0 0 1 0 1 1 1 0.416666667

0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0.083333

The second questionnaire was more standard, and was opinion-based on how well people thought our app performed. 13 statements were posed, and we asked raters to mark how they felt about the statement, either strongly agree, slightly agree, neutral, slightly disagree, or strongly disagree.

Due to the fact that all of the questions were positive, I was able to change these rankings into numbers, with a strongly disagree being a 1, a slightly disagree being a 2, to strongly agree being a 5. These numbers were then averaged across all participants in order to get a general overview of how the populace felt about our application. As shown in Table 2, the averages were mostly between slightly agree and strongly agree with 2 exceptions.

Table 2: results of questionnaire 2 Question The App clearly communicates the purpose. The App clearly communicates information about Active Strategy. The content is well written. The App appropriately reveals content through examples. It is easy to find archived data in the App. The links are well designed. The navigation is clear. In terms of use and placement, graphics are clear and appropriate. Graphics are designed appropriately. Window titles are appropriate. In terms of staging and pop-ups, information is presented clearly. 1 2 3 4 5 6 7 Average 4 5 3 5 4 3 5 4.142857 3 5 4 5 4 4 5 4.285714 5 5 4 5 4 3 5 4.428571 3 5 3 4 4 4 4 3.857143 4 5 4 4 3 4 5 4.142857 5 4 4 4 4 4 5 4.285714 5 5 3 2 3 4 5 3.857143 4 5 3 4 5 4 5 4.285714 5 5 3 4 5 3 5 4.285714 5 5 4 4 5 4 5 4.571429 5 5 4 4 5 3 5 4.428571

In terms of information graphics (graphs, etc), information is presented clearly. 4 5 5 4 5 4 5 4.571429

Overall, the results from questionnaire two were favorable, but question four and seven were lower than our goal of an overall slightly agree or better ranking. These two questions were The App appropriately reveals content through examples and The navigation was clear.

We designed our second prototype with these two results in mind, in order to improve the factors that caused the low scores to be given. The results from questionnaire one were much less favorable, as only one of the four critical measures could be found with a success rate over our goal of 85%. Due to this, we attempted to make the navigation more clear, and more obviously state the critical measures in the second prototype. Prototype #2 After reviewing the usability results, it was found that many people had trouble finding some aspects that were critical to revising a scorecard. People did not understand where to go for certain things or how to get there. Because of these problems a second prototype was made with improvements in order to make the app more effective and user friendly. In order to make the app more user friendly, the app was updated to make navigation more intuitive and easier to understand. In the first version the only way to get to a deeper level was to click on the name of the department the user wanted to see or the button that was next to it. The problem with this was that it was not obvious that the button or the name were clickable. In order to fix this the group encased all of the measures and color coded them as well. If the measure was in a red box, then that is a problematic measure but if the link was in yellow or green it was performing as expected or performing better than expected, respectively. After experimenting with this navigability for a while, the group also decided that it would be easier for a user to get to where they want if more than just the measure was clickable. So in addition to encasing the links names the group decided to make all the numbers associated with the measure, such as variances, also clickable. All of these clickable features led

to the measure they corresponded to, making it easier to get to what the user is looking for. An example screenshot is shown below:

Figure 4: Example screenshot with new buttons for each link Another problem that the users were having was finding the initiatives of each measure because the original app did not include the initiatives. In order to make it easy for the user to fin the initiatives the app includes them on the same page as the variance reports for each measure. The initiatives were not formatted much differently than they are on the website because the executives will already be used to seeing this format from the site and will not be confused when looking at them. The only difference from the site is the colors in order to match the apps background and fonts. The initiatives were placed in between the Measure Information and the Variance report in order to help distinguish the separation between the two.

Figure 5: screenshot of prototype 2 with initiatives

Organizational Sales Pitch

Because Team One is a fairly new company, it does not have a large amount of employees. However, most of the employees are recent college graduates who are selected through a tough employment process where the employment standards are high, therefore they possess the latest skills, education, daring creativity and great work ethics. In addition, Team One also has several senior staff such as the renowned Dr. Ling Rothrock and widely recognized industrial engineer Kelly Sprehn supervising the employees young and raw talent. When a versatile and talented team is combined with experienced supervisors, Team One is undoubtedly one of the best design companies in the design world. It does not only have the elements needed to succeed, it will do so while catering to the needs of its customers.

Conclusion
First and foremost, Team One would like to express gratitude to Kaiser Permanente and Active Strategy at being afforded the opportunity to submit a proposal for the Active Strategy project, thank you for having the confidence in a smaller company with such a rewarding project. During the development of the interface, the design process was closely followed. The interface designed caters to the needs of the users, while serving a greater purpose, which is monitoring the progress of a company. Although the initial designed was flawed, the team countered the problem with a usability analysis, it allowed the team to fix these flaws and create a better application. In conclusion, the application designed was efficient, accurate and it possesses all the elements a business executive need in order to improve the companys performance. Finally, if any additional information is needed, please do not hesitate to contact one of our employees, the team will then arrange a meeting with you to discuss any potential problems or ideas for improvement. Once again, thank you for the opportunity.

Bibliography
Lehto, M. R. (2008). Hujman factors and Ergonomics for Engineers. CRC Pubflishing.