Interactive TV and user experience Does interactivity contribute to a greater television user experience?

A Human-Computer Interaction perspective

Petter Bae Brandtzg Research Scientist, SINTEF ICT Pb 124, Blindern N-0314 Oslo, Norway pbb@sintef.no Jan Heim Chief Research Scientist, SINTEF ICT Pb 124, Blindern N-0314 Oslo, Norway jheim@sintef.no Asbjrn Flstad Research scientist, SINTEF ICT Pb 124, Blindern N-0314 Oslo, Norway asf@sintef.no

Abstract
This paper describes the evaluation of usability and user experience of the interactive TV (iTV) services provided by the Norwegian company Zonavi on Canal Digital during broadcast of the FIFA football World Cup 2002 to Norwegian viewers. The research examines the relationship between people's use of the iTV, and the total user experiences of the event. A questionnaire survey before and after the World Cup championships investigated both user expectations and user experiences of iTV during the World Cup. The target samples of the surveys were (1) users of iTV with access to World Cup, and (2) users watching the football event without an interactive TV service. Respondents with access to iTV during the football World Cup reported a greater total experience of the games compared to respondents with traditional TV. A greater total experience of the championships was associated with how entertaining they found the interactive services. The results are controlled to such factors as interest in football, whether they saw all the games they wanted to see, and how much they were looking forward to the games. These results are discussed in the context of new media trends and previous research. Key words: interactivity; television; usability; user experience, HCI

Introduction

TV is moving from being a passive medium to being and interactive medium. According to Strategy Analytics (NUA, 2002) European consumer spending on interactive TV (iTV) applications and services is set to rise by 121 per cent between 2002 and 2008. Enhanced applications such as viewer voting and showing participation will account for 48 per cent of the European market, while interactive games are expected to account for a further 31 per cent. The success or failure of iTV will, however, depend on the user experience of iTV. At present we know fairly little about the user experiences of iTV. Television represents a familiar technology to most people, but is traditionally a passive medium. Television as an active technology (iTV) will mean a shift in psychological orientation, from a passive to an active mental mode (Gunter, Nicholas, Huntingdon & Williams, 2002). The achievement of this objective will depend on how people perceive and experience the interactive services or how the design activities are communicating with the iTV users. In the field of iTV there is widespread debate about terminology. In the present paper iTV is given a broad definition: The TV content and services which are available for digital viewers to navigate through on their TV screen (BBC, 2004). The iTV content and services to be presented and discussed in this paper are related to a specific television event the broadcast of the FIFA World Cup 2002 to the Norwegian audience, where some viewers had access to particular iTV features or services via Canal Digital as shown in the figure sample below (Figure 1). The applications main functions were to 1) see highlights from other matches (video on demand); 2) change camera angle; 3) see earlier matches in repeat; 4) retrieve information about certain players and teams; and 5) participate in contests or football quizzes during the matches.

Figure 1: Interactive features during World Cup at Canal Digital. From the menu the iTV user can choose between the: Main sending ,Overview, Korea, France, Highlights, Multi-cameras and Information Services This interactivity may contribute to a more varied viewers possibilities and viewer-control over content and services compared to viewers who dont have iTV. Interactivity is said to have the potential to make a deep impact on the user experience of television, with giving the viewer the opportunity to extend their use of the television to activities more familiar from the Internet (Campbell, 2000). ITV features make it possible to browse information on topics of interest, personalize viewing choices, participate in user contests, and play an increasingly active role in broadcast programs. But, does interactivity contribute to greater television user experiences? The study described in this paper aimed to answer this critical question.

ITV and user experience

One theoretical approach to television use is the media research perspective (e.g. Tan, 1985). This approach is concerned with identifying how people use the media to gratify their needs or why people use particular media rather than on how content and services are experienced by the user. In this study, and as an alternative to the media research perspective, the Human Computer Interaction (HCI) perspective is to investigate how the technology meets the needs of the user and how the user actually experiences the technology. Design activities for new technology have been called a process of communication among various categories of viewers (Erickson, 1995). However, the HCI tradition has originally focused on the general working population and the traditional ICT (Information and Communication Technology) user (Krogstie, Brandtzg, Heim & Opdahl, 2003), where the focus of interest have been on user performance in terms of more effective and efficient task completion (Jordan, 1997). Thus, in later years, theory and methods of HCI have also come to comprise user experience and everyday technology use (Blythe, Overbeeke, Monk & Wright, 2003). TV is a product that must meet the needs of a broad range of viewers (Eronen, 2003). iTV services will surely address both traditional ICT-users and new groups of users, such as children, youth and the elderly. Products or services available for these user groups do not only require a formal task to be performed, they also need to provide a notion of having fun or obtaining an optimal experience (Brandtzg, Flstad & Heim, 2003); the emotions as well as the functionality associated with the product or services should meet the users expectations (Jordan, 2003). There is, however, little research related to the use and user experience of iTV (Huntington, Nicholas, Williams & Gunter, 2003).

A Study of iTV Use and User Experience

This paper describes user data from a study of iTV use during the broadcast of the FIFA World Cup 2002 to Norwegian viewers. The research examines the relationship between people's use of the iTV, and the total experiences of the event. The important question was to find and examine factors that are related to a subjective

feeling of a great experience and the role of interactivity in particular. It improves on earlier research by using a research design in a real life context - such as the World Cup. The inclusion of expectations in this study is done because the target sample of this study were two different user groups of the World Cup television event; those with and without iTV-services. The users expectation towards the product is based on the information the user has received before actual use, possibly through commercial information material or more objective reviews. It seems reasonable to believe that there is an interaction effect between expectations and actual experiences. Since many people with interactive services had bought interactive services to get mere access to the World Cup1 these people might also had higher expectations and interest towards the World Cup. And, since high user expectations and interest may generate greater user experience, it was necessary to measure both expectations before and after the World Cup to control for this. This approach might have given two comparable samples with and without iTV services, in regards to football interests and expectations, when measuring the total experience of the World Cup event.

3.1 Hypotheses
The following hypotheses were formulated for the study: ITV users will perceive a greater total experience of the World Cup than users with classic TV (non-iTV users). ITV users who perceive interactive services as entertaining will also report a greater experience of the World Cup event.

The influence of iTV services on the World Cup user experience was investigated through two questionnaire surveys to obtain data on both user expectations and user experience: 1. 2. The first survey was conducted prior to the onset of the World Cup, to collect data on user expectations to the championships and the related TV services. The second survey was conducted between three and four weeks later, between the semi-finals and the finals, to collect data on user experience.

Only the participants of the first survey were invited to participate in the second survey.

3.2

Sample

The group of participants (n 5066) included persons both with and without access to interactive TV services, but all participants had access to the World Cup. Target sample of the surveys were in more detail two different user groups of the World Cup television event: 1. 699 iTV users: Users of iTV with access to World Cup transmissions with iTV services. World Cup transmissions on digital TV with interactive services were a pay service (Canal Digital); users with access to these were likely to be highly interested in football. 18 per cent of the iTV-users were women and 82 per cent were men. The largest group was people in the age group 18-25 years, but differed only marginally from the age group 26-35 years. 4367 Non-iTV users: Users of television services with access to the to the World Cup transmissions, but without interactive TV services. There were 40 per cent women and 60 per cent men among the Non-iTV users. The largest group was people in the age group 26-35 years old.

2.

The two groups of target participants should be matched on all important variables except access to iTV-services. These two groups made it possible to compare data on user experience between iTV users and classical TV viewers.
1

In the weeks before the FIFA World Cup event there was commercial campaign for buying the licence to watch the World Cup on interactive TV.

The participants were recruited on the basis of a customer address list from the digital TV provider Canal Digital (snail.mail survey), and through an advertisement in the sports pages of the main Norwegian Internet newspaper www.vg.no (web survey).

3.3

Distribution and response rate

Snail-mail and web distributed the pre-World Cup survey. The paper-based questionnaire was distributed by snailmail to 500 randomly selected persons from the iTV provider's customer list. The response rate of the pre-World Cup paper survey was 35.8 per cent. The pre-World Cup web questionnaire was advertised as having 100,000 hits at the Internet newspaper web-site. The response rate relative to the number of times the advertisement presented was 5 per cent. The post-World Cup paper questionnaire was distributed by snail-mail to all participants returning the pre-World Cup paper questionnaire. Response rate: 53 per cent. Links to the post-World Cup web questionnaire were distributed to all participants answering the pre-World Cup web questionnaire. Response rate: 45 per cent. Participation in the surveys was rewarded with tickets in a lottery for the respective survey. The prize in both lotteries was a holiday travel gift card, valued at 1350 Euro.

3.4

Survey measurements

In the pre-World Cup questionnaire, the survey participants were asked to report how many matches they were going to see during the World Cup (None; 1-5; 6-10; 11-20; 21-30; 31-64). Their expectations to the World Cup in general were reported on scales ranging from -5 (disagree) to 5 (agree) for the following statements: I am looking forward to the World Cup I am interested in football I believe the World Cup will be an exciting experience for me I believe I will get to see all the World Cup matches I want to see I will enjoy the World Cup

In the post-World Cup questionnaire, the participants were asked to report how many matches they actually had seen during the World Cup. Their experience of the World Cup in general was reported on similar scales as for reporting their expectations. The statements responded to were: The football World Cup met my expectations I have become more interested in football due to the World Cup The football World Cup was an exciting experience I saw all the World Cup matches I wanted to The football World Cup was engaging All things considered, the World Cup was a great experience to me

Participants of the post-World Cup survey reporting to have access to interactive TV services, were asked to report their user experience of these services on the scales presented in the table below (Table 1). Table 1: Items rating on user experience of iTV-services in the survey
Not boring Not entertaining Not exciting Not interesting Not funny Not annoying Not useful -5 -5 -5 -5 -5 -5 -5 -4 -4 -4 -4 -4 -4 -4 -3 -3 -3 -3 -3 -3 -3 -2 -1 -2 -1 -2 -1 -2 -1 -2 -1 -2 -1 -2 -1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 2 2 2 2 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 4 4 5 5 5 5 5 5 5 Very boring Very entertaining Very exciting Very interesting Very funny Very annoying Very useful

Participants of the post-World Cup survey reporting to have access to interactive TV services, were also asked to report the following on scales ranging from 1 (disagree) to 7 (agree): I will use the interactive TV services also during the next major sport event I will recommend the interactive TV services to my friends I would have chosen to follow the World Cup on the TV channel with interactive TV services, even if the matches were accessible on other TV channels Using the interactive TV services made the World Cup a greater experience for me

The main iTV services accessible for the iTV users are described in more detail in the introduction to this paper. All quantitative analyses were accomplished by SPSS.

Analysis and results

4.1 Hypothesis number 1

As a first step, we analyzed hypothesis number one: iTV users will perceive a greater total experience of World Cup than users with classic TV (non-iTV users). We compared this by using the user experience measurement as described in the method, and in figure 2 below. To obtain a sample of participants to be as similar as possible; the analysis was controlled with regard to the respondents interest in football. This analyse included only respondents that saw 20 matches or more during the World Cup, this to achieve a more representative result of the possible differences between the two groups of Television users (with and without iTV-services) regarding their experiences of the World Cup. A shown in the figure 2, and despite the controlled football interest, respondents with access to iTV during the football World Cup reported a greater total experience of the games.

Figure 2 A comparison between non-iTV users and iTV user the World Cup on user experiences factors

4.2 Hypothesis number 2

The next step was to analyze the second hypothesis: TV-users who perceive interactive services as entertaining will also report a greater experience of World Cup, this by conducting a regression analysis. However, before the

regression analysis it was necessary to accomplish five factor analyses of separate parts in the questionnaires, both before and after the football World Cup. 44 variables were included in these analyses. Factors were rotated to oblique solutions with the Oblimin method. 10 factors were extracted. One variable was treated as dependent and nine as independent. . Dependent variable Questions or variables related to the general experience of World Cup were the basis for the dependent variable. The dependent variable was a very general factor based on all respondents (N=2271) of the second questionnaire (the experience survey obtained after the World Cup games). See the following six questions on: The football World Cup met my expectations.(described in detail in the method section). This was labelled as the General football experience factor, the highest loading items were The football World Cup was a great experience and The football World Cup was an exiting experience both loadings in the pattern matrix were 0,92. Independent variables A total of 9 factor score were entered at the beginning of the analysis, and the inclusion level was set at p=0,02. The following factors as shown in the table 3 below were entered as independent variables in the regression analysis. The listwise missing value option was used, which resulted in a total of 114 valid cases. Three variables had significant contributions: 1. 2. 3. Looking forwards to the World Cup (obtained before the games) Saw all the matches I wanted (obtained after the games) Interactive services were entertaining (obtained after the games)

Table 3: Dependent and independents variables in the regression analysis

Mean Dependent variable: I am in general satisfied with World Cup Expectations of user experiences Expectations of functionality Expectations of usability User experience Usability experiences Interactive services as entertaining Interactive services as boring Looking forward to WORLD CUP Believe I will see all the matches I wanted too I saw all the matches I wanted to 0,60 0,02 0,09 0,04 -0,15 0,13 -0,05 0,05 0,84 0,68 0,39 Std. Deviation 0,79 1,00 1,00 0,99 1,11 1,02 1,03 1,05 0,34 0,76 0,99 N 114 114 114 114 114 114 114 114 114 114 114

The four variables (1 dependent, 3 independent) had all significant skewness as shown in Table 4; therefore the transformations were used in order to get distributions closer to normality. First all variables were subtracted from the maximum value plus 1. Then square root transformations were performed on the dependent variable, Saw all the matches I wanted and Interactive services were entertaining. For the Looking forwards to the World Cup an inverse transformation was employed.

Table 4: Skewness and Kurtosis for variables in the regression model

Skewness Statistic General football experience factor (Dependent) Interactive services were entertaining Looking forwards to the WORLD CUP Saw all the matches I wanted 0,49 0,06 0,44 0,21 Std. Error 0,14 0,14 0,14 0,14 0,29 -0,03 -0,79 -1,10 Kurtosis Statistic Std. Error 0,28 0,28 0,28 0,28

It is evident that perfect normal distributions were not obtained. However the robustness of the regression model and the fact that estimates of coefficients do not requires normality, justified usage of a regression analysis.

Table 5: Results of regression analysis N=308 , R=0,632. *P< 0.001

Unstandardized Coefficients B (Constant) Interactive services were entertaining Looking forwards to the WORLD CUP Saw all the matches I wanted 0,953 0,263 0,370 0,268 Std. Error 0,091 0,040 0,049 0,041 0,309 -0,338 0,310 6,572* 7,541* 6,622* Standardized Coefficients Beta T

To substantiate that the correlations involved were not spurious as a result of skewed distributions, non-parametric statistics were employed in addition to the regression analysis. N=308. (Results based on non-transformed factor scores). Table 6: Pearson Correlation Coefficients between the dependent variable (General football experience factor) and the three predictor variables. *P < 0.001
Pearson Correlation Coefficient 0,42* 0,35* 0,45*

Interactive services were entertaining Looking forwards to the WORLD CUP Saw all the matches I wanted

As shown in Table 5 and Table 6 the respondents with access to iTV differ in their assessment of the total experience dependent on how entertaining they found the interactive services. People finding interactive services more entertaining report a significant higher experience of the World Cup event. This is not due to differences whether they saw all the games they wanted or how much they were looking forwards to the games.

Discussion and conclusion

The purpose of this study was to examine if there was a relationship between use of iTV and user experience. There are two main findings in this study that relate to the relationship between interactivity and total user experience: 1. Respondents with access to iTV during the World Cup reported a greater total experience of the games. This is not due to their greater interest in football or whether they saw all the matches they wanted to during the World Cup, even though both sources contribute independently to a greater experience. Respondents with access to iTV differ in their assessment of the total experience dependent on how entertaining they found the interactive services. This is either not due to differences whether they saw all the games they wanted or how much they were looking forward to the games.

2.

However, this study tells little about why respondents with iTV reported a greater experience of World Cup than those without iTV. One explanation may be that interactivity in it self is as a source toward a greater user experience. Firstly, this finding in the results; where the total experience is dependent on how entertaining the users found the iTV-services. Secondly, as pointed out in the introduction, interactivity may contribute to more varied viewer possibilities and a greater degree of empowerment over content and services than viewers of traditional television (Campbell, 2000). This is the main point to why iTV is said to have the potential to make a deep impact on the user experience of television. The importance of interactivity on user experience is also corresponding with previous research user control or empowerment is regarded as an important aspect of an enjoyable experience (Brandtzg et al., 2003).

Access to various possibilities and user agency may also be explained by a new trend in the media society in general. Newer media technologies, such as games technologies and Internet services, are designed for interactivity, a range of different user choices and interpersonal communication. Several data for both Internet usage and computer game usage support the claim that these are very popular interactive activities. This trend may influence peoples use of and approach to media technologies as such, including television, which in turn may facilitate other kinds of user demands and user needs in comparison to traditional television viewing. However, it should be noted that traditional TV not has been a very interactive media and is surely a more passive medium than new interactive media such as electronic games and the Internet. Therefore, previous studies of viewing ordinary television have reported the feeling of being more passive and less challenged and alert, while simultaneously concentrating less and using fewer skills than in almost any other daily activity except resting and doing nothing (Csikszentmihalyi, 1975). The results in our study of iTV as greater experience than classical TV may therefore reflect these new users demands and needs. There is a distinction between iTV and other kind of interactive technologies such as the PC. The two pieces of equipment are used in different ways and in a different environment - the TV tends to be viewed at a distance of a couple of metres and in a social and domestic environment unlike the PC which tends to viewed by individuals who are close to the screen (see Nielsen, 1997). Previous research has found that watching the television is a social activity to be shared with other family members in the living room (Bovill & Livingston, 2001). Another explanation for the level of user experience and iTV may therefore be the social factor. We know that people with iTV services during World Cup was popular among friends and often a social happening, because few others had access to Word Cup. A weakness is the sample in this study, were men is dominating. It has long been recognized that new forms of ICTs are not taken up by everyone at the same rate. There are innovators and early adaptors who are more venturesome and quick to start using new technologies. These are often characterized to be young, better off and male (Brandtzg, Endestad, Heim, Kaare & Torgesen, 2004). Among iTV users in the UK, men (46 %) reportedly used interactive services more than did women (29 %) (Gunter et al., 2002). However, in this study we controlled for factors like football interest and how many matches they saw, this because these factors was of more importance than gender when measuring the total experience of watching the World Cup. In conclusion, empowering users/viewers with the ability to get the information or programming what they want when they want may be a key to making iTV work. The interactive service provided during World Cup 2002 was a step in the right direction to empowering the users. However, important future research should examine in more depth why television interactivity contribute to a greater user experience. To realise this more HCI research is needed in the domestic environment or more specific in the living room. This is not necessary a well-structured context with an identified pattern and will require methods who is able to address users in a not controllable context. Evaluation tools appropriate in field research is needed. In the end these methods and tools should involve the capability to analyse more than just usability. It is necessary to measure entertainment experiences and be able to identify the audience needs for stimulation.

Acknowledgement

Thanks to the Norwegian iTV company Zonavi - www.zonavi.com - for funding the research in this study. Thanks to our colleagues at SINTEF ICT, Trond Schliemann, Anne Lund and Jan Hvard Skjetne.

References
BBC Commissioning (2004). What is interactive TV? Available 15.11.04: http://www.bbc.co.uk/commissioning/questions/interactive.shtml Blythe, M. A., Overbeeke, K., Monk, A. F., & Wright, P.C (Eds.). (2003). Funology: from usability to enjoyment. Dordrecht, the Netherlands: Kluwer. Bovill, M., & Livingstone, S. (2001) Bedroom Culture and the Privatization of Media Use in S. Livingstone and M. Bovill (Eds.), Children and Their Changing Media Environment, pp. 179-201. London: Lawrence Erlbaum Associates.

Brandtzg, P. B., Flstad, A. , & Heim, J. (2003). Enjoyment. Lessons from Karasek. Pp 55-65, in Blythe, M.A., Overbeeke, K., Monk, A.F. and Wright, P.C. (Eds.), Funology: from usability to enjoyment. Dordrecht, the Netherlands: Kluwer. Brandtzg, P. B., Endestad, T., Heim, J., Kaare, B. H. & Torgersen, L. (2004). Media technology and different patterns of use among children in Norway. Paper presented at the Conference Digital Generation Children, young people and new media. Centre for the Study of Children, Youth and Media Institute of Education, University of London 26-29. July 2004. Campbell, A. (2000). Usability Factors in Interactive Television. Retrieved May 29, 2004 from www.alastc.co.uk/usable_itv.pdf Csikszentmihalyi, M. (1975). Beyond boredom and anxiety: The experience of play in work and games. San Francisco: Jossey-Bass. Eronen, L. (2003). Digital television for all: User preferences and designers views on what would suite the user. Pp. 179-186 in N. Carbonell, C. Sthephanidis (Eds.), User interfaces of all, LNCS 2615..Berlin: SpringerVerlag. Erickson, T (1995). Notes on design practice: Stories and prototypes as catalysts for communication. Pp.37-58 in Carroll, J. M. (Ed.), Scenario-Based design. Envisioning work and technology in system development. New York: Wiley. Gunter, B, Nicholas, D, Huntingdon, P,, & Williams, P. (2002). Online versus offline research: implications for evaluating digital media. Aslib Proceedings, 54(4) pp229-239 Huntington P, Nicholas D, Williams P., & Gunter, B. (2003). An evaluation of a health video on demand service available to the public via interactive digital television. Libri 53(4), 266-281 Jordan, P. W. (2003). "Foreword" in Blythe, M.A., Overbeeke, K., Monk, A.F. and Wright, P.C. (Eds.), Funology: from usability to enjoyment. Dordrecht, the Netherlands: Kluwer. Jordan, P. W. (1997). The four pleasures taking human factors beyond usability. Proceedings of the 13th Triennial Congress of the International Ergonomics Association, 2, 364-365. Helsinki: Finnish Institute for Occupational Health. Krogstie, J., Brandtzg, P. B., Heim, J., & Opdahl, A. L (2003) Usable mCommerce Systems: The Need for Modeling-Based Approaches. Pp 190-205, in Ee-Peng Lim and Keng Siau (Eds.), Advances in Mobile Commerce Technologies. USA: Idea Group Publishing, Nielsen, J. (1997). TV meets the Web. Retrieved April 15, 2004 from http://www.useit.com/alertbox/9072b.html NUA survey (2002) Revenues from iTV set to soar in Europe. Retrieved May 16, 2003 from http://www.nua.ie/surveys/index.cgi?f=VS&art_id=905358371&rel=true Tan, A. (1985). Mass Communication Theories and Research. New York: John Wiley & Sons.