Elaborato Molinari Massimiliano n46000476

Facolt di Ingegneria
Corso di Studi in Ingegneria Informatica

Elaborato finale in Reti di Calcolatori
On the Monitoring and Measuring of
Quality of Experience (QoE) in IP
Networks

Anno Accademico 2011/2012

Candidato:

Massimiliano Molinari

matr. N46000476

III
Index

Intiouuction S
Innovative Aspects of this Thesis 6

Section 1. Quality anu Expeiience 7

1.1 What is 'Quality'. 7
1.2 What is 'Expeiience'. 9
1.2.1 Expeiience 9
1.2.2 0sei Expeiience 1u
1.2.S 0X - Reseaich 0veiview 11
1.2.4 Conclusion - 'A Question of Nuances' 12

Section 2. Quality of Expeiience 14

2.1 What is 'Quality of Expeiience'. 14
2.1.1 Quality of Patient Expeiience 1S
2.1.2 Feeuback 16
2.2 Quality of Seivice 16
2.S QoE vs QoS 17
2.S.1 PQoS 18
2.4 Quality of Business 19
2.S QoE, QoS, anu QoBiz 19
2.6 Factois influencing QoE 2u
2.7 Featuies of QoE 22

Section S. Quality Nonitoiing Piocesses 2S

S.1 Quality Evaluation 2S
S.1.1 Subjective Quality Evaluation 2S
S.1.2 0bjective Quality Evaluation 26
S.2 IQX Bypothesis foi QoE 27
S.2.1 An Example: voIP 28
S.S Real-Time Nonitoiing Systems Su
S.4 PSQA Nethouology S1
S.4.1 QoE Evaluation foi a Web Shopping Application SS

IV
Section 4. 0n the Roau to Quality SS

4.1 A Long-Teim Appioach SS
4.1.1 Quality of Besign S6
4.1.2 QoE-0iienteu Piouuct Besign Piocess S6
4.1.S Taxonomy of QoB Featuies foi Netwoik-baseu Applications S7
4.2 An 0veiview of Reseaich Challenges S8
4.2.1 A Pioposeu Schema foi QoE Functions Stanuaiuization S8

Conclusions 4u
Nain Refeiences 41
0thei Refeiences 44

Quality of Experience (QoE) over Internet

5
Introduction

Buiing the last two uecaues, the notion of 'Quality of Expeiience' has become of gieat
concein in uiffeient application contexts, anu yet we lack of a satisfying anu wiuely
accepteu uefinition of this teim. To face this anu othei issues, this thesis is oiganizeu as
it follows.
In Section 1, uefinitions of 'Quality' anu 'Expeiience' aie given, as well as otheis ielateu
to them.
In Section 2, a uefinition of 'Quality of Expeiience' is given. The concepts of 'Quality of
Seivice' anu 'Quality of Business' aie uiscusseu too. The Section aims at analysing the
uiffeiences anu the ielationships between these thiee concepts. Fuitheimoie, an
oveiview of the main factois influencing QoE is given. Finally, it is uiscusseu the
impoitance of QoE featuies anu the neeu foi an appiopiiate taxonomy as well.
Section S aims at piesenting what we will call 'quality monitoiing piocesses' - that is to
say, piocesses whose puipose is to measuie anu evaluate to what extent a ceitain
piouuct satisfies its useis. Biffeient appioaches aie pioposeu.
In the fouith anu last Section, it is piesenteu a long-teim appioach to quality. Aftei
piesenting the notion of 'Quality of Besign', a piocess foi the uesign of a 'QoE-0iienteu
Piouuct' is uesciibeu. Noieovei, an oveiview on the main ieseaich challenges is
pioviueu. Lastly, it is pioposeu a schema foi the stanuaiuization of what we will call 'Q-
Functions', which basically aie functions that map QoS onto QoE, as we will latei see.


6
In the last Section ielevant iefeiences aie iepoiteu.

Innovative Aspects of this Thesis
In the fiist two Sections, no oiiginal contiibution has been uevelopeu. Befinitions of the
main notions aie iepoiteu, in oiuei to uiaw the line of oui uomain. Neveitheless, some
'foou foi thought' has been put heie anu theie; foi example, in 2.1.2 it is intuitively
shown a hiuuen mechanism of feeuback in the notion of Quality of Expeiience, which is
the veiy basis of ieal-time closeu-loop systems, whose impoitance anu applicability
aie latei explaineu in S.S. Theiefoie, Sections 1 anu 2 must be seen as a path that will
leau the ieauei to the coie issues heiein tieateu. In Section S, an inteiesting anu
geneial appioach to QoE evaluation is explaineu. In auuition, an oiiginal
implementation example, conceining a web shopping application, has been uevelopeu
anu uesciibeu.
Finally, in Section 4 some innovative iueas have been illustiateu, such as a long-teim
appioach to quality, a taxonomy of Quality of Expeiience featuies anu a schema foi the
stanuaiuization of 'Q-Functions' foi Inteinet-baseu applications, wheie a 'Q-Function'
is a function which basically allows evaluating QoE fiom a set of QoS paiameteis, as it
will be fuithei explaineu latei.


7
Section 1
!"#$%&' #)* +,-./%.)0.
The aim of this thesis is to pioviue a wiuei compiehension of piocesses foi the
monitoiing anu measuiing of the Quality of Expeiience ovei Inteinet. But, fiist of all, it
is necessaiy to uefine the context within which Quality of Expeiience exists. Bence, in
this Section (anu in Section 2 as well) we will only iecall the basic anu inuispensable
notions, plunueieu heie anu theie fiom liteiatuie.
So, what is 'Quality of Expeiience'. Well, it is not easy to answei this question. Nany
uiffeient uefinitions aie available in liteiatuie anu on the Web, but most of them aie
not well fiameu, anu only sciatch the suiface of the pioblem. Insteau, it is impoitant to
fully unueistanu what QoE is in oiuei to unueistanu how to ensuie anu monitoi it.
Theiefoie, to appioach the objective we shall tiy to answei the following questions
fiist:
What is Quality.
What is Expeiience.

1.1 What is Quality?
It is quite haiu to uefine the woiu 'Quality'. Foi example, Weinbeig (1992) uefineu it as
follows:
|Quality 1j !"#$%&'( &* +$%#, '- *-., /,0*-123


8
This uefinition focuses upon the subjective natuie of the concept of quality. It states
that people might peiceive the quality of a same piouuct in uiffeient ways.
The quality of a piouuct can be also uefineu as it follows:
|Quality 2j !45, 6#$%&'( -7 $ /0-8#9' .,$*#0,* 5-: :,%% '5, /0-8#9' &* 8,*&;1,8 $18 5-:
:,%% '5, /0-8#9' 9-17-0.* '- '5$' 8,*&;123
Anothei uefinition was given by uaivin (1984):
|Quality Sj !"#$%&'( &* $ 9-./%,< $18 .#%'&7$9,',8 9-19,/'23
That was piobably no new infoimation, though. Neveitheless, uaivin's uefinition is not
as banal as it seems; in fact, Kitchenham (1996) extenueu it by iuentifying five majoi
peispectives:
1) 40$1*9,18,1' +&,:: it uefines quality as something 'peiceivable, but not
uefinable';
2) =*,0 +&,:: it uefines quality as 'fitness foi puipose';
S) >$1#7$9'#0&1; +&,:: it uefines quality as 'confoimance to iequiiements oi
specification';
4) ?0-8#9' +&,:: it uefines quality as 'bounu to inheient chaiacteiistics of the
piouuct';
S) @$%#, +&,:: it uefines quality as 'uepenuent on the amount a customei is willing to
pay foi it';
This uefinition incluues both subjective anu objective aspects: the usei view, foi
example, is stiongly uepenuant on the usei's stanupoint, while the manufactuiing view
is ielateu to measuiable anu objective uesign iequiiements (Figuie 1).
Bowevei, I believe that a biilliant uefinition of quality in oui context is the following
|Qualinet (2u12); }ekosh (2uuS)j:
|Quality 4j !"#$%&'( &* '5, -#'9-., -7 $1 &18&+&8#$%A* 9-./$0&*-1 $18 B#8;.,1' /0-9,**2 C'
&19%#8,* /,09,/'&-1D 0,7%,9'&-1 $E-#' '5, /,09,/'&-1D $18 '5, 8,*90&/'&-1 -7 '5, -#'9-.,23
This is a moie piagmatic uefinition. It looks at the usei as an I0 system: the input is the
piouuct oi seivice offeieu to the usei; the output is the quality he peiceiveu.


9

F&;#0, G H *#EB,9'&+, $18 -EB,9'&+, $*/,9'* -7 6#$%&'(2
1.2 What is Experience?
So fai, we have analyseu the teim 'quality' fiom uiffeient peispectives. Let us now go
on to intiouuce anu uefine the concepts of 'Expeiience' anu '0sei Expeiience' as well,
foi they iepiesent oui last step in oiuei to uelineate the context within which QoE
exists.

1.2.1 Experience
Qualinet (2u12) built the following essential uefinition of expeiience:
|Expeiience 1j !I1 ,</,0&,19, &* $1 &18&+&8#$%A* *'0,$. -7 /,09,/'&-1 $18 &1',0/0,'$'&-1
-7 -1, -0 .#%'&/%, ,+,1'*23
Wheie an 'event' is uefineu as it follows:
|Eventj !JI1 ,+,1' &*K $1 -E*,0+$E%, -99#00,19,2 C' &* 8,',0.&1,8 &1 */$9, L&2,2 :5,0, &'
-99#0*MD '&., L&2,2 :5,1 &' -99#0*MD $18 95$0$9',0 L&2,2 :5$' 9$1 E, -E*,0+,8M23
Bowevei, it can be useful to consiuei expeiience beneath a uiffeient, less scientific,
light. Naybe it is not too ambitious to aigue that 'Expeiience' is the biggest uiffeience
between humans anu machines. It is about feelings anu senses, that is, oui innei woilu,


10
anu thus I woulu like to concluue iepoiting the following 'uefinition' of expeiience by
Aluous Buxley (19S2):
|Expeiience 2j !N</,0&,19, &* 1-' :5$' 5$//,1* '- $ .,1O &' &* :5$' $ .$1 8-,* :&'5
:5$' 5$//,1* '- 5&.23

1.2.2 User Experience
Let us now consiuei the teim '0sei Expeiience', which is stiictly ielateu to oui uomain
of inteiest. Bonalu Noiman biought this teim to wiuei knowleuge in 199S |see
Noiman (199S)j anu since then lots of uiffeient uefinitions weie given:
|0X 1j Alben (1996): !J=*,0 N</,0&,19, &* $E-#'K I%% '5, $*/,9'* -7 5-: /,-/%, #*, $1
&1',0$9'&+, /0-8#9'P '5, :$( &' 7,,%* &1 '5,&0 5$18*D 5-: :,%% '5,( #18,0*'$18 5-: &'
:-0Q*D 5-: '5,( 7,,% $E-#' &' :5&%, '5,(A0, #*&1; &'D 5-: :,%% &' *,0+,* '5,&0 /#0/-*,*D $18
5-: :,%% &' 7&'* &1'- '5, ,1'&0, 9-1',<' &1 :5&95 '5,( $0, #*&1; &'23
|0X 2j WSC (2uuS): !J=R &*K I *,' -7 .$',0&$% 0,18,0,8 E( $ #*,0 $;,1' :5&95 .$( E,
/,09,&+,8 E( $ #*,0 $18 :&'5 :5&95 &1',0$9'&-1 .$( E, /-**&E%,23
|0X Sj Bassenzahl & Tiactinsky (2uu6): !J=R &*K I 9-1*,6#,19, -7 $ #*,0A* &1',01$% *'$',
L/0,8&*/-*&'&-1*D ,</,9'$'&-1*D 1,,8*D .-'&+$'&-1D .--8D ,'92MD '5, 95$0$9',0&*'&9* -7 '5,
8,*&;1,8 *(*',. L,2;2 9-./%,<&'(D /#0/-*,D #*$E&%&'(D 7#19'&-1$%&'(D ,'92M $18 '5, 9-1',<'
L-0 '5, ,1+&0-1.,1'M :&'5&1 :5&95 '5, &1',0$9'&-1 -99#0* L,2;2 -0;$1&*$'&-1$%S*-9&$%
*,''&1;D .,$1&1;7#%1,** -7 '5, $9'&+&'(D +-%#1'$0&1,** -7 #*,D ,'92M
|0X 4j Bassenzahl (2uu8): !=R &* $ .-.,1'$0(D /0&.$0&%( ,+$%#$'&+, 7,,%&1; L;--8TE$8M
:5&%, &1',0$9'&1; :&'5 $ /0-8#9' -0 *,0+&9,23
|0X Sj IS0 9241-21u (2u1u): !J=R &*K I /,0*-1A* /,09,/'&-1* $18 0,*/-1*,* '5$' 0,*#%'
70-. '5, #*, -0 $1'&9&/$',8 #*, -7 $ /0-8#9'D *(*',. -0 *,0+&9,23
|0X 6j 0PA (2u1u): !N+,0( $*/,9' -7 '5, #*,0A* &1',0$9'&-1 :&'5 $ /0-8#9'D *,0+&9,D -0
9-./$1( '5$' .$Q, #/ '5, #*,0A* /,09,/'&-1* -7 '5, :5-%,2 =*,0 ,</,0&,19, 8,*&;1 $* $
8&*9&/%&1, &* 9-19,01,8 :&'5 $%% '5, ,%,.,1'* '5$' '-;,'5,0 .$Q, #/ '5$' &1',07$9,D
&19%#8&1; %$(-#'D +&*#$% 8,*&;1D ',<'D E0$18D *-#18D $18 &1',0$9'&-12 =N :-0Q* '-
9--08&1$', '5,*, ,%,.,1'* '- $%%-: 7-0 '5, E,*' /-**&E%, &1',0$9'&-1 E( #*,0*23


11
|0X 7j Kuniavsky (2u1u): !45, #*,0 ,</,0&,19, &* '5, '-'$%&'( -7 ,18T#*,0*A /,09,/'&-1*
$* '5,( &1',0$9' :&'5 $ /0-8#9' -0 *,0+&9,2 45,*, /,09,/'&-1* &19%#8, ,77,9'&+,1,** L5-:
;--8 &* '5, 0,*#%'UMD ,77&9&,19( L5-: 7$*' -0 95,$/ &* &'UMD ,.-'&-1$% *$'&*7$9'&-1 L5-: ;--8
8-,* &' 7,,%UMD $18 '5, 6#$%&'( -7 '5, 0,%$'&-1*5&/ :&'5 '5, ,1'&'( '5$' 90,$',8 '5, /0-8#9'
-0 *,0+&9, L:5$' ,</,9'$'&-1* 8-,* &' 90,$', 7-0 *#E*,6#,1' &1',0$9'&-1*UM3

In my opinion, it was impoitant to give a biief look at a few uefinitions, so that the
ieauei coulu unueistanu the key points of this concept, anu the uiffeient appioaches
anu peispectives as well.
Nany othei uefinitions aie available in liteiatuie anu on the web; howevei, the
pieviously iepoiteu ones allow us to highlight some key points:
0sei expeiience is stiongly subjective...
...But it uepenus on objective aspects too;
0sei expeiience is about usability...
...But it goes beyonu, as it is ielateu to useis' feelings anu peiceptions;
In the enu, the full compiehension of usei expeiience is yet to come. It is useful to
concluue this paiagiaph by iepoiting the following statement |Bassenzahl, 2uu6j:
!V5&%, =R &* :,%% 8&*9#**,8 -1 9-17,0,19,* $18 *(./-*&$D &' -1%( 0$0,%( ,1',0* '5, 0,%,+$1'
$9$8,.&9 B-#01$%*2 V, E,%&,+, '5$' '5, %$9Q -7 ,./&0&9$% 0,*,$095 &* -1, -7 '5, 0,$*-1* 7-0
'5&*2 45, $E*,19, -7 ,./&0&9$% 0,*,$095 H :5,'5,0 6#$%&'$'&+, -0 6#$1'&'$'&+, H &./,8,*
'5,-0,'&9$% $8+$19,.,1' $18 0,*'0&9'* -#0 #18,0*'$18&1; -7 =R $* 9-19,/' $18 &'* 7#0'5,0
8,+,%-/.,1'23
In this thesis it was impossible to make moie than a biief mention on the notion of
'usability'. Foi fuithei infoimation, see Rubin (1984) oi IS0 9241-21u (2u1u).

1.2.3 UX Research Overview
The last statement leaus us to pioviue a sketch of how 0X ieseaich might look like in
the futuie. In the following we will iepoit the thiee majoi ieseaich peispectives


12
iuentifieu by Bassenzahl anu Tiactinsky in theii papei "0sei Expeiience - A Reseaich
Agenua" (2uu6):
A thieau that ueals with the ielationships between human neeus anu instiumental
aspects - it stiesses the iuea |Bassenzahl, 2uuSj that both piagmatic anu heuonic
aspects ielateu to 0X will have to be ueeply analyseu, with paiticulai focus on the
links between them;
A thieau that ueals with the affective anu emotional aspects of the inteiaction - it
focuses on the connections between emotions anu the piouuct use;
A thieau that ueals with the natuie of expeiience - it aims at iuentifying invaiiant
expeiience patteins acioss time anu space, in oiuei to achieve a bettei
compiehension of expeiience;
In the following, the main ieseaich challenges they iuentifieu aie summaiiseu:
To uesciibe the foimal stiuctuie of the oveiall quality of a piouuct;
To stuuy the ielationships between instiumental anu non-instiumental aspects;
To unueistanu if anu how expeiiential as well as affective iequiiements can be
tianslateu into conciete specifications;
Although six yeais have passeu so fai, most of these challenges aie yet to be
accomplisheu.

1.2.4 Conclusion A Question of Nuances
In conclusion, it seems to me that usei expeiience is 'a question of nuances'. I think that
this inteiesting statement by Bassenzahl anu Tiactinsky (2uu6) might give a bettei
compiehension of what I mean:
!=R J222K 7-9#*,* -1 5-: '- 90,$', -#'*'$18&1; 6#$%&'( ,</,0&,19,* 0$'5,0 '5$1 .,0,%(
/0,+,1'&1; #*$E&%&'( /0-E%,.*2 I;$&1D '5&* :&%% 6#,*'&-1 $1-'5,0 &./%&9&' $**#./'&-1 -7
'0$8&'&-1$% WXCD -1, '5$' ,6#$',* 5&;5 6#$%&'( :&'5 '5, $E*,19, -7 /0-E%,.*2 45&* .$( E,
$1$%-;-#* '- '5, 1-'&-1 '5$' $E*,19, -7 &%%1,** ,6#$%* 5,$%'52 Y#' B#*' $* '5,0, &* .#95
.-0, '- :,%%E,&1; '5$1 '5, $E*,19, -7 .$%$8(D *- .#*' '5,0, E, .-0, '- =R '5$1 '5,
$E*,19, -7 /0-E%,.*23


13
Computeis aie about high-oi-low bits, yes-oi-no answeis, etc.; the ieal stiength of the
notion of 0X (anu of QoE as well, as we will see) comes fiom its being so 'human-
oiienteu'; nonetheless, this appioach makes it uifficult to stuuy 0X (anu QoE) with
classic engineeiing techniques. This is the challenge we have to face.


14
Section 2
!"#$%&' 12 +,-./%.)0.
In Section 1 we attempteu to unueistanu the notions of 'Quality' anu '0sei Expeiience'
(anu many otheis along the way) by giving a few uefinitions anu analysing uiffeient
appioaches. Was it necessaiy to spenu so many woius about those concepts. Yes it
was, as it will be shown in the next paiagiaph.
In auuition, aftei intiouucing the concepts of 'Quality of Seivice' anu 'Quality of
Business', we will stuuy the ielationships between those anu QoE. Then we will give a
foimal uesciiption of the factois influencing QoE. Lastly, we will intiouuce the notion
of 'QoE metiics', focusing especially on theii impoitance iathei than giving a foimal
taxonomy.

2.1 What is Quality of Experience?
Baving so fai analyseu the notions of Quality anu (0sei) Expeiience, we can 'builu' a
QoE uefinition as shown in Figuie 2.
In a nutshell, this simple way of pioceeuing leaus us to a quite intuitive uefinition of
QoE. Essentially, it measuies whethei we aie pioviuing useis with what they want oi
not, anu this is actually the coie iuea behinu QoE. But let us consiuei moie specific
uefinitions; foi instance, uoouchilu (2uuS) uefineu QoE as it follows:


15
|QoE 1j !"#$%&'( -7 N</,0&,19, &* '5, -+,0$%% /,07-0.$19, -7 $ *(*',. 70-. '5, /-&1' -7
+&,: -7 '5, #*,0*2 "-N &* $ .,$*#0, -7 $1 ,18T'-T,18 /,07-0.$19, %,+,%* $' '5, #*,0
/,0*/,9'&+, $18 $1 &18&9$'-0 -7 5-: :,%% '5&* *(*',. .,,'* '5, #*,0 1,,8*23

F&;#0, Z H $ 7&0*' 8,7&1&'&-1 -7 "-N2

uoouchilu gave this synthetic but poweiful uefinition, yet many otheis aie available in
liteiatuie. Bowevei, they uo not auu any ielevant infoimation.
2.1.1 Quality of Patient Experience
To bettei unueistanu the concept of 'QoE', let us consiuei the meuical teim of 'Quality
of Patient Expeiience'. It is a subjective measuie of a patient's expeiience with some
meuical piactice. It intenus to ueteimine whethei the patient enjoyeu a highei oi lowei
quality of life aftei the tieatment. The goal is to unueistanu the patient's feelings about
the piactise in oiuei to unueistanu if (anu how) it is possible to impiove the tieatment
to enhance the patient's quality of life.


16

2.1.2 Feedback
The pievious paiagiaph shoulu ieminu the ieauei about something we have alieauy
saiu in 1.2.4, that is, usei expeiience is a question of nuances. In fact, we aie not only
inteiesteu in finuing a 'black oi white' iesult. Refeiiing to QoPE, yet it is of gieat
concein to be able to evaluate the meuical piactise fiom a functional point of view (uiu
the theiapy woik oi uiu it not.), we aie above all willing to ieceive a 7,,8E$9Q fiom
patients, in oiuei to pioviue them with a bettei expeiience.
That is to say, the patient (the usei) paiticipates in a feeuback mechanism in oiuei to
enhance the theiapy (the piouuct the seivice). This feeuback mechanism is shown in
Figuie S.

F&;#0, [ H "-N &./%&9&'%( 8,7&1,* $ 7,,8E$9Q .,95$1&*.2
As we know, feeuback mechanisms aie the single most commonly useu techniques foi
ieal-time contiol systems. In Section S, we will stuuy an aichitectuie that basically
takes auvantage of feeuback to pioviue useis with a bettei expeiience.

2.2 Quality of Service
Fiist of all, let us give a uefinition of Quality of Seivice (Zieglei, 2uu4):
|QoSj !\1 '5, C1',01,' $18 &1 -'5,0 1,':-0Q*D "#$%&'( -7 ],0+&9, L"-]M &* '5, &8,$ '5$'
'0$1*.&**&-1 0$',*D ,00-0 0$',*D $18 -'5,0 95$0$9',0&*'&9* 9$1 E, .,$*#0,8D &./0-+,8D $18
'- *-., ,<',1'D ;#$0$1',,8 &1 $8+$19,2 "-] &* -7 /$0'&9#%$0 9-19,01 7-0 '5, 9-1'&1#-#*


17
'0$1*.&**&-1 -7 5&;5TE$18:&8'5 +&8,- $18 .#%'&.,8&$ &17-0.$'&-12 40$1*.&''&1; '5&*
Q&18 -7 9-1',1' 8,/,18$E%( &* 8&77&9#%' &1 /#E%&9 1,':-0Q* #*&1; -08&1$0( ^E,*' ,77-0'A
/0-'-9-%*23
Quality of Expeiience is stiictly ielateu to Quality of Seivice: they both iefei to the
same concept, but the foimei looks at it fiom the usei's point of view, the lattei fiom
the pioviuei's. Zapatei anu Biessan (2uu7) stateu that:
|QoE vs QoSj !"#$%&'( -7 ],0+&9, /0&.$0&%( 7-9#*,* -1 '5, /,07-0.$19, .,$*#0,* 70-. '5,
1,':-0Q /,0*/,9'&+,2 "#$%&'( -7 N</,0&,19, '$Q,* &1'- $99-#1' 5-: :,%% $ *,0+&9, .,,'
9#*'-.,0*A ;-$%* $18 ,</,9'$'&-1* 0$'5,0 '5$1 7-9#*&1; -1%( -1 '5, 1,':-0Q /,07-0.$19,2
I' '5, E-''-. %&1,D "-N &* $ .,$*#0, -7 '5, ,18T'-T,18 /,07-0.$19, $' '5, *,0+&9, %$(,023

2.3 QoE vs QoS
In the following the main uiffeiences between QoE anu QoS aie consiueieu.
Fiistly, while QoS is an objective measuie, QoE is stiongly subjective (even though it is
inevitably influenceu by usei-inuepenuent paiameteis, such as his uevice, the
conuitions of the netwoik, etc.)
Seconuly, while the concept of QoS is focuseu on telecommunications systems, QoE has
a wiuei scope.
Thiiuly, while QoS uses technological anu analytical appioaches, QoE ielies on context-
baseu multi-uisciplinaiy appioaches.
In auuition, let us notice that theie is no stiict ielationship between QoE anu QoS
values; in fact, it may happen that useis aie not happy with a ceitain piouuct even
though it iateu high in QoS. vice veisa, a piouuct iating low in QoS can make useis veiy
happy with it, thus causing a high QoE. Table 1 shows all the uiffeient possibilities.

The piouuct . . Bo not meet its iequiiements . Neet its iequiiements
. Bo not pioviue useis
with a goou expeiience
L0W QoE;
L0W QoS; (1a)
L0W QoE;
BIuB QoS; (1b)
. Pioviue useis with a
goou expeiience
BIuB QoE;
L0W QoS; (2a)
BIuB QoE;
BIuB QoS; (2b)
4$E%, G H 0,%$'&-1*5&/* E,':,,1 "-N $18 "-]2


18
Bowevei, it is uifficult (but not impossible) that a low-QoS piouuct iates high in QoE
(2a). That is to say, a low QoS is likely to be tianslateu into a low QoE. The (1b)
possibility is essentially the ieason why the concept of QoE was uevelopeu: in fact, in
the miu-199us, ieseaicheis (anu venuois as well) founu out that, even assuiing a high
QoS, useis coulu still be unhappy with the piouuct, thus causing a fall in sales; hence
the neeu foi a bettei compiehension of QoE.
Lastly, while QoS measuies the quality of a piouuct befoie being ueliveieu, QoE
measuies the quality of a ueliveieu piouuct: in this sense, QoE is also known as
Peiceiveu Quality of Seivice, PQoS.

2.3.1 PQoS
PQoS evaluation uepenus on both objective anu subjective aspects. The foimei can be
quickly evaluateu thanks to well-testeu quality-evaluation piocesses, but iequiie laige
amount of machine iesouices anu sophisticateu appaiatus configuiations; the lattei
iequiie slowei, time-consuming piocesses anu a laige amount of human iesouices.
Figuies 4 anu S aim to help ieaueis unueistanu these concepts. (Figuie S also shows
that the piocesses of quality evaluation can be iun only aftei having uefineu
appiopiiate metiics.)
In this Section we aie only intiouucing these issues: at the moment it is enough to
unueistanu the main iueas, as in Section 4 we will uig ueepei into them.

F&;#0, _ H '5, ,77,9'* -7 1,':-0Q* -1 6#$%&'(2


19

F&;#0, ` H *#EB,9'&+, $18 -EB,9'&+, $*/,9'* -7 6#$%&'( ,+$%#$'&-12

2.4 Quality of Business
We have uiscusseu the concepts of QoE anu QoS so fai. In a nutshell, we have seen that
QoE is necessaiy in oiuei to unueistanu whethei useis aie happy oi not, since it is
stiictly connecteu with theii willingness to spenu money. Pioviuing useis with a goou
expeiience will piobably allow venuois to eain moie money. I think that this shoit
example (van Nooisel, 2uu1) can give a bettei compiehension of this phenomenon:
!V5- 9$0,* &7 (-#0 :,E *5-//&1; *&', 5$* $1 $+$&%$E&%&'( -7 ^'50,, 1&1,*AU aa2ab
$+$&%$E&%&'( 9-00,*/-18* '- $ %&''%, -+,0 ,&;5' 5-#0* -7 '-'$% 8-:1 '&., &1 $ (,$0D E#' .-*'
9#*'-.,0* 8- 1-' 9$0,D 1-0 8- %&1,T-7TE#*&1,** .$1$;,0*2 45, 9#*'-.,0 -1%( 9$0,* &7 '5,
*&', &* #/ :5,1 *5, :$1'* '- *5-/2 >-0,-+,0D &7 '5, *&', &* 8-:1 7-0 $ 7,: *,9-18* 8#0&1; $
*5-//&1; *,**&-1D *5, :&%% -1%( 9$0, &7 &' .$Q,* $1( 8&77,0,19, 7-0 '5, ,<,9#'&-1 -7 5,0 -1T
;-&1; '$*Q2 45, E#*&1,** .$1$;,0 -1%( 9$0,* &7 *5, %-*,* E#*&1,** E,9$#*, -7 8-:1 '&.,D
$18 7-0 5-: .#95 .-1,( '5&* 9-#%8 5$+, E,,1 /0,+,1',823
Theiefoie, business metiics aie necessaiy too.

2.5 QoE, QoS, and QoBiz
Let us intiouuce the notion of Quality of Business (QoBiz) anu see what van Nooisel
(2uu1) stateu about it:


20
|QoBizj !"-] .,'0&9* $0, .,'0&9* *#95 $* $+$&%$E&%&'( $18 /,07-0.$19,O "-N .,'0&9* $0,
0,*/-1*, '&.,* $18 $+$&%$E&%&'( -7 $ *,0+&9, $* ,</,0&,19,8 E( $ 9#*'-.,0O 7&1$%%(D '5,
"-Y&c .,'0&9 &* ,</0,**,8 &1 ',0.* -7 .-1,(D *#95 $* '5, $+,0$;, $.-#1' -7 .-1,(
0,9,&+,8 /,0 ,<,9#',8 '0$1*$9'&-123
QoE, QoS anu QoBiz aie stiictly connecteu to each othei, as shown in Figuie 6.

A highei QoS is likely to pioviue useis with a
highei QoE;
A highei QoE is likely to pioviue venuois with
moie piofits, that is, a highei QoBiz;
A highei QoBiz is likely to allow venuois to
spenu moie money in oiuei to pioviue a
bettei seivice;
F&;#0, d H "-]D "-ND "-Y&c2
Noieovei, the ielationship between the customei anu the seivice pioviuei can be
iepiesenteu in an even simplei way (Figuie 7).

F&;#0, e H 0,%$'&-1*5&/ E,':,,1 '5, 9-*'#.,0 $18 '5, *,0+&9, /0-+&8,0 L70-. @$1 >--0*,%D ZffGM2

2.6 Factors influencing QoE
In the pievious paiagiaphs we attempteu to unueistanu anu uefine what QoE is anu
what is not. So fai, it has become cleaiei anu cleaiei that QoE uepenus on the context
of use. It is suiely influenceu by the typology of seivice, but it also uepenus on otheis


21
factois, such as the usei's uevice, anu his moou as well; but also the natuie of the
content, the puipose of the piouuct, the conuitions of the netwoik, anu so on.
Qualinet (2u12) iuentifieu the main factois influencing QoE, wheie an influence factoi
is uefineu as it follows:
|Influence Factoij !I1( 95$0$9',0&*'&9 -7 $ #*,0D *(*',.D *,0+&9,D $//%&9$'&-1D -0 9-1',<'
:5-*, $9'#$% *'$', -0 *,''&1; .$( 5$+, &17%#,19, -1 '5, "#$%&'( -7 N</,0&,19, 7-0 '5,
#*,023
These factois have been gioupeu into thiee categoiies, namely Buman IF, System IF
anu Context IF.
|Buman IFj !I W#.$1 CF &* $1( +$0&$1' -0 &1+$0&$1' /0-/,0'( -0 95$0$9',0&*'&9 -7 $
5#.$1 #*,02 45, 95$0$9',0&*'&9 9$1 8,*90&E, '5, 8,.-;0$/5&9 $18 *-9&-T,9-1-.&9
E$9Q;0-#18D '5, /5(*&9$% $18 .,1'$% 9-1*'&'#'&-1D -0 '5, #*,0A* ,.-'&-1$% *'$',23
Buman IFs aie complex, since they iefei to the innei woilu of the inuiviuual, theii
iueas, theii backgiounu, theii feeling anu sensations, etc. These factois uepenu on the
paiticulai peison, anu howevei aie potentially time-vaiiant.
|System IFj !](*',. CF* 0,7,0 '- /0-/,0'&,* $18 95$0$9',0&*'&9* '5$' 8,',0.&1, '5,
',951&9$%%( /0-8#9,8 6#$%&'( -7 $1 $//%&9$'&-1 -0 *,0+&9, (}umisko-Pyykko, 2u11). 45,(
$0, 0,%$',8 '- .,8&$ 9$/'#0,D 9-8&1;D '0$1*.&**&-1D *'-0$;,D 0,18,0&1;D $18
0,/0-8#9'&-1S8&*/%$(D $* :,%% $* '- '5, 9-..#1&9$'&-1 -7 &17-0.$'&-1 &'*,%7 70-. 9-1',1'
/0-8#9'&-1 '- #*,023
System IFs have been uiviueu into foui sub-categoiies:
Content-ielateu System IFs, which iefei to the content type;
Neuia-ielateu System IFs, which iefei to meuia configuiation factois (such as
encouing, iesolution, sampling iate, fiame iate, etc.);
Netwoik-ielateu System IFs, which iefei to uata tiansmission ovei a netwoik
(some factois aie: banuwiuth, uelay, jittei, thioughput, etc.);
Bevice-ielateu System IFs, which iefei to the capabilities of the uevices involveu
along the communication path;


22
|Context IFj !X-1',<' CF* $0, 7$9'-0* '5$' ,.E0$9, $1( *&'#$'&-1$% /0-/,0'( '- 8,*90&E,
'5, #*,0A* ,1+&0-1.,1' &1 ',0.* -7 /5(*&9$%D ',./-0$%D *-9&$%D ,9-1-.&9D '$*QD $18
',951&9$% 95$0$9',0&*'&9* (}umisko-Pyykko, 2u11)23
Context IFs iefeis to spatial anu tempoial aspects, such as the chaiacteiistics of
location anu space anu the time of uay, economic aspects, such as the cost, social
aspects, ielateu to the piesence of othei people, etc.
Figuie 8 gives an oveiview of this taxonomy.

F&;#0, g H "-N &17%#,19&1; 7$9'-0* '$<-1-.( $* /0-/-*,8 E( "#$%&1,' LZfGZM2

2.7 Features of QoE
It woulu be extiemely useful to uecompose QoE into featuies, as it woulu allow
measuiing quality thanks to well-known mathematical techniques. In fact, an aiiay of
featuies can be seen as a point in the multiuimensional space, anu the uistance
between a ceitain iefeience point anu the one iuentifieu by the aiiay of featuie woulu


23
pioviue a measuie of quality. Bowevei, theie aie some questions to be answeieu, such
as:
|Question 1j !h- *#95 7,$'#0,* ,<&*'U3
|Question 2j !V5$' $0, '5,*, 7,$'#0,* %&Q,U3
|Question Sj !V5$' 8&*'$19, .,'0&9 *5-#%8 E, #*,8U3
Let us consiuei the fiist question: many expeits aigue that such featuies uo exist. Foi
example, Qualinet (2u12) not only claims theii existing, but also pioposeu a foui-level
classification of QoE featuies.
Level of uiiect peiception. At this level theie aie "featuies ielateu to the
peiceptual infoimation cieateu immeuiately anu spontaneously";
Level of inteiaction. At this level theie aie featuies ielateu to the human-to-
human anu human-to-machine inteiactions;
Level of usage. At this level theie aie featuies ielateu to the usage situation;
Level of seivice. At this level theie aie featuies that go beyonu a paiticulai
instance;
As shown in Figuie 9, low-level featuies iequiie a shoitei time in oiuei to be peiceiveu
(Y-axis) anu then valueu (X-axis), while high-level featuie iequiie moie.
In the following some examples of QoE featuies aie iepoiteu.
+,#3-$.4
|Level of uiiect peiceptionj: shaipness, uaikness, biightness, contiast, coloui featuies,
atmospheie, involvement, etc.
|Level of inteiactionj: iesponsiveness, natuialness of inteiaction, communication
efficiency, conveisation effectiveness, etc.
|Level of usagej: accessibility anu stability uuiing a usage instance, etc.
|Level of seivicej: pleasuie, long-teim stability, usefulness, ease of use, etc.

We have answeieu |Question 1j anu (although only paitially) |Question 2j so fai. In the
next Sections we shall give the missing answeis.


24

F&;#0, a H "-N 7,$'#0,*P $ 9%$**&7&9$'&-1 0,%$',8 '- '5,&0 8#0$'&-1 $18 -99#00,19,2


25
Section 3
!"#$%&' 51)%&1/%)6 7/10.44.4
In this Section we will intiouuce the notion of 'quality monitoiing piocesses'. Fiistly, let
us give a uefinition: a quality monitoiing piocess is a piocess whose puipose is to
measuie, by iefeiiing to appiopiiate metiics anu by using efficient anu effective
methouologies, the quality level of a piouuct. Seconuly, we will focus on the neeu foi
this piocesses. Finally, we will piesent an example of implementation, explaining how
to measuie QoE in ielation to a web shopping application.

3.1 Quality Evaluation
In the pievious Sections we have seen that Quality of Expeiience is a subjective
measuie only to a limiteu extent, as it uepenus on objective aspects as well. Theiefoie,
in a quality evaluation context two possible appioaches can be followeu:
A subjective appioach;
An objective appioach;
In the following, we will analyse both of them.

3.1.1 Subjective Quality Evaluation
Subjective quality evaluation is usually baseu on the so-calleu N0S, Nean 0pinion
Scoie |Rifa et al., 2u11; Be veia et al., 2uu7j. 0sually, the evaluation is caiiieu out by a
panel of useis; each of them has to iate the quality of a piouuct choosing a value


26
ianging fiom 1 (bau quality) to S (excellent quality). The N0S is uefineu as the
aiithmetic mean of all inuiviuual scoies. 0bviously, it is necessaiy to pick a statistically
ielevant sample of useis in oiuei to obtain a significant iesult.
In Table 2, pios anu cons aie analyseu.

PR0s C0Ns
It is a human-baseu piocess It is a time-consuming piocess
It is a simple piocess It is an expensive piocess
It cannot be pait of an automatic piocess
Table 2 pros and cons of subjective quality evaluation processes.
Anothei appioach is baseu on suiveys, whose puipose is to collect useis opinions
thiough special questionnaiies. Bowevei, as obseiveu by Rifa et al. (2u11) oi Chen et
al. (2uu9), this type of appioach has a few uisauvantages, hence, it is iaiely useu.

3.1.2 Objective Quality Evaluation
In the last uecaues, uiffeient objective quality evaluation piocesses have been
pioposeu. They aim at measuiing the quality of a piouuct by evaluating some technical,
objective paiametei (i.e. blockiness anu bluiiiness foi images, attenuation anu uelay
foi auuio, fiame loss oi buffeiing pioblems foi viueo, etc.)
In Table S, the main pios anu cons aie iepoiteu.

PR0s C0Ns
It is baseu on algoiithms anu foimulas It is not human-baseu piocesses
It is ielatively quick piocesses It is context-uepenuent
It cannot be pait of an automatic piocess
Table 3 pros and cons of objective quality evaluation processes.


27
The gieatest uisauvantage is piobably that it is not possible to use them in a ieal-time
contiol system, as the oiiginal anu the ieceiveu uata aie usually both neeueu in the
same place at the same time (Be veia et al. 2uu7).
In the next paiagiaph, we will see a hybiiu quality evaluation methou known as the
IQX hypothesis foi QoE.

3.2 IQX Hypothesis for QoE
The pieviously uiscusseu methous aie not mutually exclusive; on the contiaiy, it woulu
be extiemely useful to exploit both of them in a quality monitoiing piocess, which is
what we intenu to uo. As stateu by Rifa et al. (2u11), since QoS paiameteis aie easiei
to obtain than the QoE evaluation, we aie inteiesteu to finu a ielationship between QoS
anu QoE, i.e. a function f such that "-Ni7L"-]M, in oiuei to base on QoS the monitoiing
of QoE. It has been shown (Fieulei, 2u1u) that a qualitative ielationship between QoS
anu QoE can be illustiateu as it follows (Figuie 1u).

F&;#0, Gf H $ 0,%$'&-1*5&/ E,':,,1 "-] 8&*'#0E$19, $18 "-N +$%#,D '$Q,1 70-. F&,8%,0 LZfGfM2


28
0n the x-axis it is uenoteu the QoS uistuibance; on the y-axis it is uenoteu a QoE value,
such as an opinion scoie.
Accoiuing to Fieulei, it is possible to uetect thiee main aieas, sepaiateu by thiesholus
x1 anu x2:
Aiea 1 - foi a small QoS uistuibance, useis peiceive an optimal quality;
Aiea 2 - as soon as the QoS uistuibance exceeus a ceitain thiesholu (x1 in the
pictuie), the QoE value sinks;
Aiea S - once the QoS uistuibance exceeus anothei thiesholu (x2 in the pictuie),
the QoE value is extiemely low anu the seivice might stop woiking;
Bowevei, thiesholu x1 can be essentially co-locateu with the y-axis. The position of
thiesholu x2, on the opposite, is moie usei-uepenuent.
In Fieulei (2u1u) it is piesenteu, uemonstiateu, anu uiscusseu an analytical
ielationship between QoE anu QoS, known as the IQX hypothesis (exponential
inteiuepenuency of quality of expeiience anu quality of seivice):

The values of u, , anu y uepenu on the paiticulai application; foi example, in the
following we will see how to finu the values of these paiameteis in ielation to a voIP
seivice.

3.2.1 An Example VoIP
The following example (Fieulei, 2u1u) is extiemely useful, because it shows how to
estimate the QoE peiceiveu by human useis once an appiopiiate set of QoS values is
measuieu. That is, we can measuie QoE automatically anu in ieal-time. The piocess is
shown in Figuie 11.
Essentially, foi each value of QoS paiameteis, we will compaie the oiiginal signal with
a uiffeient veision of it. This uiffeience is obtaineu anu evaluating by using the PESQ
(Peiceptual Evaluation of Speech Quality) methou. The iesulting PESQ value is then
mappeu into a subjective N0S value, as specifieu in IT0-T P.862.1 (2uuS).
QoE = u
.
,
-
+ y u, , y > u


29
Thus, foi each value of QoS, we know the coiiesponuent value of QoE. If we only want
to evaluate the ielationship between packet loss anu QoE, foi example, we can put on a
Caitesian cooiuinate system the uots that iepiesent the N0S value coiiesponuent to
ceitain values of packet loss iatio (see Figuie 12).
Then, using a nonlineai iegiession methou, we can obtain a continuous function (in
this example, it has been useu a iegiession methou such that the noimal eiioi E is
minimizeu). The ielationship between packet loss iatio anu QoE is the following:
QoE = S.u1u
.
,
-4.47S
.
p
loss + 1.u6S.
The goouness of fit of this ielationship can be measuieu by calculating the coefficient of
coiielation R between the mouel function anu the measuieu uata; R
2
shoulu appioach
one foi a peifect match. In this case, R
2
=u.99S, anu this inuicate a neaily peifect match.

F&;#0, GG H "-N ,*'&.$'&-1 0,7,00&1; '- $ @-C? *,0+&9,2


30
As foi viueo, vQN (viueo Quality Netiic) is a stanuaiuizeu methou of objectively
measuiing viueo quality veiy similai to PESQ. So, we aie able to evaluate anu measuie
QoE foi both auuio anu viueo. Bowevei, in the following we want to show a
geneialization of these meuia-uepenuent appioaches.

F&;#0, GZ H 1-1%&1,$0 0,;0,**&-1D '$Q,1 70-. F&,8%,0 LZfGfM2

3.3 Real-Time Monitoring Systems
In 2.1.2 we have seen that (one of) the gieat auvantage of QoE is to pioviue uevelopeis
anu venuois with a feeuback. Inueeu, as we know, closeu loop systems base theii
functioning on the notion of feeuback. These systems aie extiemely useful in iealizing
ieal-time monitoiing piocesses. In theii woik, Rifa et al. (2u11) suggest a possible bloc
uiagiam (as shown in Figuie 1S). This solution is baseu on a polynomial appioximation
of the exponential ielationship between QoS anu QoE, which allow pieces of softwaie


31

to act as system contiolleis. Foi example, it is possible to appioximate the exponential
function by its Tayloi Seiies:

F&;#0, G[ H $ 0,$%T'&., 9%-*,8T%--/ *(*',.D '$Q,1 70-. j&7$k ,' $%2 LZfGGM2
Essentially, the peiception of the usei QoE is compaieu with a iefeience peiception
QoEu2 Theiefoie, an appiopiiate contiollei will change the netwoik's paiameteis in
oiuei to achieve a QoE value equal to the uesiieu one, anu this is supposeu to enhance
the usei expeiience.

3.4 PSQA Methodology
Let us theiefoie piesent the PSQA methouology, stanuing foi Pseuuo Subjective Quality
Assessment, which is a geneial pioceuuie that allows the automatic measuie of the
peiceiveu quality, effectively anu in ieal-time (Be veia et al., 2uu7).
The way PSQA woiks can be schematically uesciibeu as it follows (see also Figuie 14):
Fiist of all, it is necessaiy to choose the paiameteis that shoulu have an impact on
quality. 0bviously, this choice is context-uepenuent;
Then, ielevant uata (i.e. some viueo sequences) must be selecteu. Again, this
selection uepenus on the applicative context;
Changing the values of the abovementioneu paiameteis, we will obtain many
uiffeient copies of the same sample of uata;
These copies must be sent ovei an appiopiiate infiastiuctuie (usually an emulateu
netwoik) to a panel of human obseiveis. Each mouifieu copy will be evaluateu by
eveiy human obseivei;
It is theiefoie auvisable to peifoim a statistical filteiing piocess, in oiuei to obtain
bettei iesults;


32
Finally, all the gatheieu uata will be useu to tiain pieces of softwaie, allowing them
to leain the mapping fiom the values of paiameteis (in oui context, these aie
usually QoS paiameteis) into peiceiveu quality (that is, a QoE value);
The ieauei shoulu have unueistoou that this technique is simply a geneialisation of the
pieviously stuuieu ones, anu shoulu have also become awaie of the auvantages of this
methouology.

F&;#0, G_ H ?]"I >,'5-8-%-;(2
To summaiise, in the following aie iepoiteu the main auvantages ielateu to this
methouology:
The mapping fiom a subjective peiception into an objective, statistically-ielevant
value;
0nce the pieviously uesciibeu tiaining piocess is accomplisheu, it is a fast
technique;
It implements a geneial appioach;


33
In Figuie 14, useis evaluate quality by giving a N0S value. Nonetheless, this is not the
only way to pioceeu; inueeu, Chen et al. (2uu9) pioposeu a moie intuitive evaluation: a
simple click. They uevelopeu a fiamewoik calleu '0neClick' that only iequiies useis to
click an appiopiiate key (e.g. the space bai) whenevei they feel uissatisfieu with the
quality of the application in use. This is a simplei way of pioceeuing; on the othei hanu,
howevei, it gives fewei nuances to the iesult.

3.4.1 QoE Evaluation for a Web Shopping Application
In the following, I aim at uesciibing the uevelopment of an oiiginal fiamewoik foi the
evaluation anu the measuiing of QoE in ielation to a web shopping application. In fact,
although QoE was boin in telecommunications, in 2.S we have seen that a gieat
auvantage of QoE compaieu to QoS is that the foimei has a wiuei scope. Since
liteiatuie lacks of a fiamewoik of QoE evaluation foi a web shopping application,
heiein we pioviue one. The fiamewoik is cuiiently available at the following 0RL:
http:www.qoeevaluation.alteivista.oiginuex.php

F&;#0, G` H j,*#%'&1; "TF#19'&-1 7-0 $ V,ET*5-//&1; $//%&9$'&-12


34
Aftei a biief intiouuction, useis aie askeu to complete a fake payment pioceuuie, by
inseiting theii (oi inventeu) name, suiname, numbei of cieuit caiu, anu amount. Aftei
waiting foi a ceitain time <, they will be infoimeu that the tiansaction has been
completeu. This time ' is a pseuuo-ianuom value ianging fiom u.1 to 2u seconus, anu
iepiesents the time iequiieu to the payment system to accomplish the tiansaction. In
othei woius, ' is oui (only) QoS paiametei. 0seis aie then askeu to iate the seivice by
giving an opinion scoie ianging fiom 1 to S. All uata aie stoieu in a uatabase. In the
enu, the Q-Function 7 P <"-] Tl "-N is woikeu out by using a nonlineai iegiession
algoiithm (in this case, a Least Squaie Regiession algoiithm) on the collecteu uata.
At the moment, the Q-Function, appioximateu by its thiiu uegiee Tayloi polynomial is
shown in Figuie 1S anu it is plotteu in Figuie 16.

F&;#0, Gd H /%-' -7 '5, "TF#19'&-12 J <T$<&*P '&., L*MO (T$<&*P 0$'&1; L>\]M K
In its 2.u veision, the fiamewoik was enhanceu by asking useis whethei they aie useu
to puichasing items on the web. In fact, expeiienceu useis aie likely to pioviue a moie
conscious evaluation. Consequently, theii iating shoulu be given a highei impoitance:
the simplei way to achieve the objective consists in inseiting the uata in the uatabase
twice. This way, it will weigh twice as much as the iating of a non-expeiienceu usei.
The piesenteu fiamewoik is veiy intuitive. Neveitheless, moie complex applications
can be uevelopeu, such as the one in Pitas et al. (2u12), which woiks out a set of two-
vaiiable Q-Functions foi ANR telephony in public 2uSu mobile netwoiks, mapping


35
the ieceiveu signal level (RxSL) anu the ieceiveu signal quality (RxSQ) onto a N0S
value, obtaineu using the alieauy mentioneu PESQ algoiithm. Each Q-Function iefeis to
a paiticulai couec (ANR-FR, ANR-BR...), technology (0NTS, uSN...), anu so on.


36
Section 4
8) &9. :1#* &1 !"#$%&'
In this final Section we aim at pioviuing geneial methouologies allowing uesigneis a
long-teim appioach to quality. The impoitance of an appiopiiate schema of featuies is
explaineu, anu a piototype is heie pioviueu. Noieovei, it is given an oveiview of
ieseaich challenges in this context.

4.1 A Long-Term Approach
In this final Section, we aim at piesenting what can be calleu 'a long-teim appioach' to
Quality of Expeiience. In fact, in the pievious paiagiaphs we uesciibeu a numbei of
techniques that weie paiticulaily appiopiiate foi 'shoit-teim' puiposes (e.g. voIP calls,
viueo ueliveiy systems, web biowsing, etc.), in oiuei to uiscovei anomalies anu
pioviue useis with a bettei seivice (by contiolling QoS paiameteis). In this Section we
want to intiouuce a methouology that allows the cieation of a 'QoE-oiienteu piouuct'
(oi seivice).
In Section S we uefineu the notion of 'quality monitoiing piocess'; let us now uefine a
'quality assuiance piocess', as a piocess whose puipose is to guaiantee, by following
appiopiiate guiuelines anu exploiting goou piactises, the uevelopment of a piouuct
that will satisfy its useis anu pioviue them with an excellent expeiience. In biief, a
quality assuiance piocess is a necessaiy (but not yet sufficient) conuition foi a goou
QoE.


37
The iuea is to meige the basic notions of both quality assuiance anu quality monitoiing
piocesses in oiuei to uevelop a piouuct within which eveiy single usei-uepenuent
aspect can be evaluateu by measuiing an appiopiiate set of objective paiameteis. In
othei woius, we aie tiying to achieve a necessaiy anu sufficient conuition to QoE.
Fiist of all, let us intiouuce the notion of 'Quality of Besign'.

4.1.1 Quality of Design
Reichl (2uu7) uefineu Quality of Besign (QoB) as it follows:
|Quality of Besign 1j !C1',0/0,'&1; '5, 5#.$1 *,1*-0( /,09,/'&-1 $* $1 &1/#' 95$11,%
:5&95 .$( E, 95$0$9',0&c,8 E( 9-19,/'* -7 E$18:&8'5D *(1950-1&c$'&-1 -7 /,09,/'&-1D
%-** -7 &17-0.$'&-1 ,'92 :5&95 $0, 6#&', *&.&%$0 '- '5,&0 $1$%-;&,* &1 '5, 1,':-0Q 9$*,D :,
9%$&. '5$' '5, 8,*&;1 -7 '5, 5#.$1T9-./#',0 &1',07$9, 9$1 E, $''0&E#',8 $ 6#$%&'(
9-19,/' *&.&%$0 '- "-]D $18 /0-/-*, '- 9$%% '5&* 1,: 1-'&-1 "#$%&'( -7 h,*&;123
In his woik, Reichl gives also a foimal uefinition of QoB:
|Quality of Besign 2j !V, 7-0.$%%( 8,7&1, "-h $* '5, 7#19'&-1 :5&95 '0$1*%$',* '5, "-]
7-0 $ ;&+,1 1,':-0Q 9-18&'&-1 < &1'- '5, "-N 7-0 $ */,9&7&9 #*,0 &P
:5,0, ! 8,1-',* L',./-0$%M 9-19$',1$'&-123

4.1.2 QoE-Oriented Product Design Process
Now, let us finally uesciibe the steps of this piocess, that we coulu call 'QoE-0iienteu
Piouuct Besign Piocess'. (Note that whatevei we say about a piouuct is peifectly valiu
foi a seivice as well).
Befining a QoS metiics taxonomy (which is mainly application-uepenuent);
Befining a QoB metiics taxonomy (which is mainly usei-uepenuent);
Setting up a panel of human testeis;
Letting testeis evaluate the piouuct with iegaius to the uefineu metiics;
o QoB metiics aie necessaiy while ueveloping the piouuct;
o QoS metiics aie necessaiy once the piouuct is on the maiket;
Applying statistical filteiing piocesses to enhance the effectiveness of the iesults;
QoE(i,x)=QoD(i)!QoS(x) ,


38
Taking auvantage of the iesulting feeuback to constantly enhance the piouuct;
Neveitheless, theie aie some pioblems still. In the fiist place, it is quite an expensive
piocess. In auuition, a stanuaiuizeu anu well-testeu QoB metiics taxonomy has not
been uevelopeu yet. Foi these ieasons, it is uesiiable that an incieasing ieseaich effoit
shoulu be maue in this uiiection, as the achievable iesults seem conspicuous. In
liteiatuie some uiffeient taxonomies aie alieauy available, foi example "the ACN
inteiaction uesign awaius ciiteiia" (Thackaia, 2uu1); othei taxonomies aie uesciibeu
in Beniy (2uu4), Nielsen (199S), Nollei et al. (2uuS), but many otheis aie available.
Bowevei, these aie geneial-puipose taxonomies, while heiein we aie willing to focus
on seivices ueliveieu ovei IP netwoiks. Noieovei, it seems to me that some featuies
aie quite 'abstiact', e.g. it will be uifficult foi a usei to evaluate appiopiiateness, that is,
'the extent to which the piouuct seive useis in appiopiiate ways, solving the iight
pioblem at the iight level', as it is uefineu in Thackaia (2uu1). Foi these ieasons, in the
following a taxonomy of QoB featuies foi netwoik-baseu applications is pioviueu.

4.1.3 Taxonomy of QoD Features for Network-based Applications
Stuuying uiffeient taxonomies, such as the pieviously mentioneu ones, it appeais to me
that essentially it is possible to iuentify five QoB macio-featuies:
0sefulness L8-,* '5, *(*',. 8- :5$' &' :$* *#//-*,8 '- 8-UM;
Ease of 0se;
}oy of 0se;
Secuiity;
Cost;
They aie quite self-explaining, I think, anu this is veiy impoitant, as it means that useis
aie able to unueistanu them quickly anu intuitively.
I woulu iecommenu that a panel of useis gave each macio-featuie an opinion scoie.
The aveiage values will iepiesent a 'weight' of the featuies. In fact, it appeais cleai that
a secuiity paiametei is fai moie ciucial than an aesthetics paiametei foi a web


39
shopping application; joy of use is much moie impoitant than usefulness foi a gaming
application; etc.
This way, the uesign piocess will be caiiieu out focusing moie on the ciucial aspects of
the paiticulai application, since uiffeient featuies will be chaiacteiizeu by uiffeient
uegiees of impoitance, anu this is likely to make useis feel moie satisfieu.

4.2 An Overview of Research Challenges
This paiagiaph aims at pioviuing an oveiview of possible futuie ieseaich on the
phenomenon Quality of Expeiience.
As a iesult of oui suivey, it appeais eviuent the neeu foi a stanuaiuizeu taxonomy of
QoB featuies. A possible solution has been pioposeu in the pievious paiagiaphs;
nonetheless, a fuithei investigation on such featuies shoulu be maue. Noieovei, it is
necessaiy to iuentify an appiopiiate system of metiics to allow objective
measuiements of the pieviously mentioneu (weighteu) featuies. Some uiffeient
solutions have been pioposeu in liteiatuie, but we still lack of a stanuaiuizeu anu well-
testeu appioach. Foi this ieason, a majoi ieseaich effoit shoulu be maue in this
uiiection.

4.2.1 A Proposed Schema for QoE Functions Standardization
Finally, we have seen that foi eveiy paiticulai application (e.g. voIP, viueo stieaming,
etc.) it is possible to iuentify a function 7 that maps QoS to QoE. Let us call this function
'Seivice-to-Expeiience Quality Function', oi simply 'Q-Function'. Each new Q-Function
shoulu be pait of a unifieu anu stanuaiuizeu schema. Foi example, heiein I piopose the
following simple stiuctuie (foi netwoik-baseu applications):
Type (Web BiowsingAuuioviueo);
Betails:
o Foi Web Biowsing: Paiticulai Application (classic biowsing activities,
web shopping, gaming, etc.);
o Foi Auuio anu viueo: Couec;


40
QoS Factois: uiffeient applications will be influenceu by uiffeient paiameteis;
Q-Function 7 : xQoS -> QoE (xQoS is an aiiay of QoS paiameteis);
Souice;
In ielation to the example illustiateu in S.2.1, we woulu have the following iecoiu:
Type: Auuio (voIP);
Couec: iLBC;
QoS Factois: Packet Loss Ratio;
Q-Function 7 : PLR -> QoE = S.u1u ,
H_2_e[ ?mj
+ 1.u6S;
Souice: !I n,1,0&9 "#$1'&'$'&+, j,%$'&-1*5&/ E,':,,1 "#$%&'( -7 N</,0&,19, $18
"#$%&'( -7 ],0+&9,3D E( F&,8%,0 >2D ZfGfO
The pioposeu schema coulu be calleu, foi instance, "QoE Functions foi Netwoik-baseu
Applications Schema". Baving such a schema (oi a bettei one) woulu be of gieat
concein, since it woulu put QoE in a moie foimal anu piagmatic pattein, thus making
this notion less abstiact anu opening the path foi an appiopiiate anu wiuei use of it in
uiffeient contexts.


41
Conclusions

In this thesis I attempteu to accomplish uiffeient puiposes. Fiistly, I hope that it has
become cleaiei by now the iemaikable impoitance assumeu by Quality of Expeiience.
It iepiesents a feeuback fiom the useis of a piouuct (oi seivice) allowing venuois (oi
pioviueis) to enhance theii expeiience with it. Being able to objectively anu cheaply
measuie useis' subjective peiceptions in oiuei to pioviue them with a bettei seivice is
of gieat concein foi venuois, since it will help them to scoop the competition anu
inciease theii business.
This thesis aimeu at illustiating uiffeient appioaches to the pioblem with paiticulai
focus on seivices ueliveieu thiough IP Netwoiks. In Section S, some piocesses have
been piesenteu. We suggesteu that being able to monitoi the quality peiceiveu by usei
in ieal-time shoulu iepiesent a gieat step towaiu QoE, since this way it is be possible
to uiscovei anomalies anu fix them immeuiately. Anu yet, this coulu be not enough.
Theiefoie, in Section 4, it was piesenteu a hybiiu long-teim appioach whose puipose
is to achieve a necessaiy anu sufficient conuition to QoE. Noieovei, some innovative
aspects aie heiein pioposeu, such as a taxonomy of QoB featuies anu a schema foi the
stanuaiuization of Q-Functions.

42
Main References
Alben (1996), L. Alben, "Quality of Expeiience: uefining the Ciiteiia foi Effective
Inteiaction Besign";

Be veia et al. (2uu7), B. Be veia, P. Rouiiguez-Bocca anu u. Rubino, "QoE Nonitoiing
Platfoim foi viueo Beliveiy Netwoiks";

Chen et al. (2uu9), K. Chen, C. Tu, W. Xiao, "0neClick: A Fiamewoik foi Neasuiing
Netwoik Quality of Expeiience";

Fieulei (2u1u), N. Fieulei, "A ueneiic Quantitative Relationship between Quality of
Expeiience anu Quality of Seivice";

uaivin (1984), B.A. uaivin, "What uoes 'Piouuct Quality' Really Nean.";

uoouchilu (2uuS), }. uoouchilu, "Integiating uata voice anu viueo - Pait II";

Bassenzahl (2uuS), N. Bassenzahl, "The Thing anu I: 0nueistanuing the Relationship
between 0sei anu Piouuct";

Bassenzahl (2uu8), N. Bassenzahl, "0sei Expeiience - Towaius an Expeiiential
Peispective on Piouuct Quality";

Bassenzahl anu Tiactinsky (2uu6), N. Bassenzahl anu N. Tiactinsky, "0sei Expeiience
- A Reseaich Agenua";

Beniy (2uu4), S.L. Beniy anu N. Naitinson, "Accessibility 0sei-Centeieu Besign";

Buxley (19S2), A. Buxley, "Texts anu Pietexts";

IS0 9241-21u (2u1u), "Eigonomics of human-system inteiaction - Pait 21u: Buman-
Centieu Besign foi inteiactive systems";

IT0-T P.862.1 (2uuS), IT0-T Recommenuation P.862.1, "Napping Function foi
Tiansfoiming P.862 Raw Result Scoies to N0S-LQ0"

}ekosh (2uuS), 0. }ekosh, "voice anu Speech Quality Peiception - Assessment anu
Evaluation";

43
}umisko-Pyykko (2u11), S. }umisko-Pyykko, "0sei-Centeieu Quality of Expeiience anu
Its Evaluation Nethous foi Nobile Television";

Kitchenham (1996), B.A. Kitchenham anu S.L. Pfleegei, "Softwaie Quality: the Elusive
Taiget";

Kuniavsky (2u1u), N. Kuniavsky, "Smait Things: 0biquitous Computing 0sei
Expeiience Besign";

Nollei et al. (2uu9), S. Nollei, K.P. Engelbiecht, C. Khnel, I. Wechsung, B. Weiss, "A
Taxonomy of Quality of Seivice anu Quality of Expeiience of Nultimoual Buman-
Nachine Inteiaction";

Nielsen (199S), }. Nielsen, "0sability Engineeiing";

Noiman (199S), B. Noiman, }. Nillei, A. Benueison, "What You See, Some of What's in
the Futuie, anu Bow we go about uoing it";

Pitas et al. (2u12), C.N. Pitas, B.E. Chaiilas, A.B. Panagopoulos, P. Chatzimisios, P.
Constantinou, "ANFIS-baseu Quality Pieuiction Nouels foi ANR Telephony in Public
2uSu Nobile Netwoiks";

Qualinet (2u12), "Qualinet White Papei on Befinitions of Quality of Expeiience -
0utput veision of the Bagstuhl Seminai 12181";

Reichl (2uu7), P. Reichl, "Fiom 'Quality-of-Seivice' anu 'Quality-of-Besign' to 'Quality-
of-Expeiience': A Bolistic view on Futuie Inteiactive Telecommunication Seivices";

Rifa et al. (2u11), B. Rifa, S. Nohammeu anu A. Nellouk, "A Biief Synthesis of QoS-QoE
Nethouologies";

Rubin (1984), }. Rubin, "Banubook of 0sability Testing: Bow to Plan, Besign anu
Conuuct Effective Testing";

Thackaia (2uu1), }. Thackaia, "Why is inteiaction uesign impoitant." fiom
http:www.uooisofpeiception.comaichives2uu111why_is_inteiact.php

0PA (2u1u), 0sability Piofessionals' Association,
http:www.usabilitybok.oigglossaiy

van Nooisel (2uu1), A. van Nooisel, "Netiics foi the Inteinet Age: Quality of
Expeiience anu Quality of Business";

Weinbeig (1992), u. Weinbeig, "Quality Softwaie Nanagement: Anticipating Change.
vol. 1: System Thinking";

WSC (2uuS), Woilu Wiue Web Consoitium, "ulossaiy of Teims foi Bevice
Inuepenuence", http:www.wS.oigTRui-glossuef-usei-expeiience

44
Zapatei anu Biessan (2uu7), N.N. Zapatei anu u. Biessan, "A Pioposeu Appioach foi
Quality of Expeiience Assuiance foi IPTv";

Zieglei (2uu4), P. Zieglei, "Bepaitment of Infoimation Technology ulossaiy", fiom
http:www.nh.govuoitinteinetoigciohifoimsuocuments0ITulossaiy.puf

45
Other References
Balasubiamaniam S., Nineiauu }., NcBonagh P., Peiiy P., Nuiphy L., Bonnelly W., anu
Botvich B., "An Evaluation of Paiameteiizeu uiauient Baseu Routing with QoE
Nonitoiing foi Nultiple IPTv Pioviueis", 2u11;

Baltoglou u., Kaiapistoli E. anu Chatzimisios P., "IPTv QoS anu QoE Neasuiements in
Wiieu anu Wiieless Netwoiks";

Cole R. anu Rosenbluth }.B., "voice ovei IP Peifoimance Nonitoiing", 2uu1;

uavei W.W. anu Naitin B., "Alteinatives. Exploiing Infoimation Appliances thiough
Conceptual Besign Pioposals", 2uuu;

Kim }. anu Noon }.Y., "Besigning towaius Emotional 0sability in Customei Inteifaces -
Tiustwoithiness of Cybei-Banking", 1998;

Sommeiville I., "Softwaie Engineeiing", 8
th
euition, 2uu7;

0nknown, "Quality of Expeiience (QoE) - A Systematic Liteiatuie Suivey", fiom:
http:www.qoe.lt, 2uu9;

0nknown, "Quality of Patient Expeiience" fiom:
http:en.wikipeuia.oigwinuex.php.title=Quality_of_expeiience&oluiu=42281676,
2uu6;

Wu W., Aiefin A., anu Rivas R., "Quality of Expeiience in Bistiibuteu Inteiactive
Nultimeuia Enviionments: Towaiu a Theoietical Fiamewoik", 2uu9;

Zhang anu Li (2uu4), P. Zhang anu N. Li, "Love at Fiist Sight oi Sustaineu Effect. The
Role of Peiceiveu Affective Quality on 0sei's Cognitive Reactions to Infoimation
Technology";

Elaborato Molinari Massimiliano n46000476

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Elaborato Molinari Massimiliano n46000476

Transféré par

Droits d'auteur :

Formats disponibles

Facolt di Ingegneria

Corso di Studi in Ingegneria Informatica

Vous aimerez peut-être aussi