Department of Industrial Engineering, University or Trento

Experimental Statistics and Descriptive and Inferential

Statistics
Design of Experiments - I —Paolo Bosetti, 2016
 Course Contents
❖ Principles of Descriptive Statistics
❖ Inferential Statistics: hypothesis testing quantile-
quantile plots, operating characteristic curves
❖ Linear regression, linear models, and analysis of
variance (ANOVA)
❖ Design of Experiments: factorial plans, blocking,
fractional factorial plans, analysis of factorial plans,
model adequacy checking
 Software

❖ GNU-R (open source: www.r-project.org)

❖ Minitab (payware)
❖ Matlab (limited support)
❖ Octave (more limited support)
Multiplicity and Randomness

Descriptive Statistics Summarizing the main

features of a sample
 Probability Distributions
8
< 0 6 p(yi ) 6 1
P (y = y i ) = p(y i )
: P
i p(yi ) = 1

❖ Discrete Distributions
❖ Continuous Distributions
8
< 0 6 f (y) Rb
P (a 6 y 6 b) = f (y)dy
: R1 a
1
f (y)dy = 1
 Probability Distributions
P (y = yi ) = p(yi )
p(yi )

❖ Discrete Distributions y1 yn yi
y2
❖ Continuous Distributions f (y)

P (a y b))

a b y
 Mean and Variance
⇥
⇤⇥ y i p(y i )
µ =: E(y) = i
⇥
yf (y)dy

❖ Expected Value
Operators
❖ Variance

⇥
⇤⇥ (y i µ)2
p(y i )
⇥ =: V (y) =
2 i
⇥
(y µ) 2
f (y)dy
2
= E[(y µ) ]
2
 Fundamental Relationships
E(c) = c (1)
E(y) = µ (2)
E(cy) = cE(y) = cµ (3)
V (c) = 0 (4)
V (y) = ⇥ 2
(5)
V (cy) = c2 V (y) = c2 ⇥ 2 (6)
E(y1 + y2 ) = E(y1 ) + E(y2 ) = µ1 + µ2 (7)
Cov(y1 , y2 ) = E[(y1 µ1 )(y2 µ2 )] (8)
V (y1 + y2 ) = V (y1 ) + V (y2 ) + 2 Cov(y1 , y2 ) (9)
V (y1 y2 ) = V (y1 ) + V (y2 ) 2 Cov(y1 , y2 ) (10)
 Population and Samples

❖ Given a population of N elements and a sample of n

elements
✓ ◆
N N!
❖ Possible different samples: n
=
(N n)!n!
Pn
yi
❖ Sample mean: ȳ = i=1
n
Pn
Sample variance:
2
❖ 2 i=1 (y i ȳ)
S =
n 1
 Estimate & Estimators
❖ Sample mean and variance are estimators of the
population
❖ Estimators are random variables themselves
❖ A single numerical value of an estimator is an estimate
❖ Estimators must have minimum variance and must be
unbiased
❖ Estimators are also called statistics
 Degrees of Freedom
❖ DoF of a statistic are the number of independent
elements in it:
✓P 2
◆ ✓ ◆
2 (yi ȳ) SS
=E =E ; ⌫=n 1
n 1 ⌫
n
X n
X
(yi ȳ) = (yi ) nȳ = 0
i=1 i=1

❖ Note that if µ is known:

Pn
i=1 (yi µ)2
Ṡ 2 = ; ⌫=n
n
n
X n
X
(yi µ) = (yi ) nµ 6= 0
i=1 i=1
Dice Man's detail, sculpture by Tony Cragg, 2011

Probability For Representing Continuous

Distributions
and Discrete Variables
 Binomial Distribution (d)

❖ Consider n Bernoulli trials such

that:

0.15 0.10
0.080.15
❖ trials z1,…,zn are independent

Binom(20,0.5)
Geom(20,0.5)

0.10
Poisson(5)

0.10
0.06
❖ a trial outcome can be either

0.04
0.05
0.05
success or failure

0.02
0.00
0.00
0 5 10 15 20
❖ probability of success p is 0 5 10
xx
15 20

constant
 Binomial Distribution (d)

x = z 1 + z 2 + · · · + zn
x ⇠ Binom(n, p) when :

0.15
✓ ◆
n x
p (1 p)n x , x 2 {0, 1, . . . , n}

Binom(20,0.5)
f (x) =

0.10
x

0.05
n = Number of trials

0.00
p = Probability of success for each trial 0 5 10 15 20

2
µ = np, = np(1 p) x
 Poisson Distribution (d)

0.15 0.10
0.080.15
❖ In a binomial experiment, when

Binom(20,0.5)
Geom(20,0.5)
n becomes large (infinite) and

0.10
Poisson(5)

0.10
0.06
the distribution mean remains

0.04
0.05
0.05
constant:

0.02
0.00
0.00
0 5 10 15 20
0 5 10 15 20
xx

x ⇠ Binom(n, /n) = Poisson( ), where = np

n!1
 Poisson Distribution (d)

x ⇠ Poisson( ) when :

0.15 0.10
x

0.080.15
e
f (x) =
x!

Binom(20,0.5)
Geom(20,0.5)

0.10
Poisson(5)

0.10
0.06
0.04
0.05
= Probability of success

0.05
within an interval

0.02
0.00
0.00
2
E(x) = µ = ; V (x) = = 0
0 5
5 10
10 15
15 20
20
xx
 Geometric Distribution (d)

0.080.15 0.10
x ⇠ Geom(p) when :

Binom(20,0.5)
Geom(20,0.5)
p)x 1

0.10
f (x) = p(1 , x 2 {1, 2, . . . }

Geom(0.1)

0.06
0.04
0.05
p = Probability of success for each trial

0.02
0.00
0 5 10 15 20
0 5 10 15 20
xx
 Uniform Distribution (d, c)

1.0
0.8
y ⇠ U (a, b) when :
⇢

0.6
U(0,1)
0 y < a, y > b
f (y) =

0.4
1/(b a) a 6 y 6 b

0.2
0.0
0.0 0.5 1.0 1.5 2.0

y
 Normal Distribution (c)

0.4
y ⇠ N (µ, 2 ) when :

0.3
1 1 y µ 2
f (y) = p e 2 [ ]

N(0,1)

0.2
2⇡
y µ

0.1
Note : ⇠ N (0, 1)

0.0
-3 -2 -1 0 1 2 3

y
 Central Limit Theorem

If x1 , x2 , . . . , xn are n independent and identically distributed random vari-

ables with E(xi ) = µ and V (xi ) = 2 (both finite), and y = x1 + x2 + · · · + xn ,
then
y nµ
zn = p
n 2
has an approximate N (0, 1) distribution, in the sense that, if Fn (z) is the dis-
tribution function of zn and (z) is is the distribution function of the N (0, 1)
random variable, then
Fn (z)
lim =1
n!1 (z)
 Chi-square Distribution (c)
2
y⇠ k when :
2 2 2
y= z1 + z 2 + · · · + zk , zi ⇠ N (0, 1)

0.20
1

0.15
f (y) = y k/2 1
e y/2
2k/2 (k/2)

0.10
2
k
χ
µ=k

0.05
2
= 2k
Pn

0.00
SS i=1 (yi ȳ)2 2 0 10 20 30 40
Note : 2
= 2
⇠ n 1 y
 Student’s Distribution (c)
y ⇠ tk when :
z 2
y=p ; z ⇠ N (0, 1), x ⇠ k

0.5
x/k 106

0.4
[(k 1)/2] 1
f (y) = p 5

0.3
[(y 2 /k) + 1](k+1)/2
k⇡ (k/2)

tk
1

0.2
µ=0

0.1
2 k
=

0.0
k 2 -3 -2 -1 0 1 2 3

Note : tk = N (0, 1)
k!1
 Snedecor’s F Distribution (c)

y ⇠ Fu,v when :

1.0
xu /u

0.8
2 2
y= ; xu ⇠ u , xv ⇠ v

0.6
xv /v

F10,10

0.4
u+v u u/2 (u/2) 1
2 v y
f (y) =

0.2
u v u (u+v)/2
vx + 1

0.0
2 2 0 2 4 6 8

y
 Distribution of Mean

Given a sample X1 , . . . , Xn obtained from a N (µ, 2 ) distribution, from

thePCentral Limit theorem it follows that the mean value of the sample, X̄ =
1 n 2
n i=1 X i , is distributed as N (µ, /n). Consequently:

X̄ µ
p ⇠ N (0, 1)
/ n
Moreover, if 2 is unknown it can be replaced with the sample variance S 2 ,
and in this case it holds:

X̄ µ
p ⇠ tn 1
S/ n
From Sample to Population

Induction of population
Inferential Statistics properties from sample
observation
Difference in Sample Statistics
N (µ1 , 1)
2
N (µ2 , 2)
2

µ1 µ2 y
Treatment 1 Treatment 2

H0 : µ1 = µ2
H1 : µ1 = µ2
 Error Probability
↵ = P (type I error) = P (reject H0 |H0 is true)
= P (type II error) = P (fail to reject H0 |H0 is false)
Power = 1 = P (reject H0 |H0 is false)
↵ 6= 1

❖ Type I is a false alarm H0 True H0 False

Reject Type I
Correct
❖ Type II is a missing alarm H0 error
Accept Type II
Correct
❖ Power is the alarm reliability H0 error
 Student’s (t)-Test
H0 : µ1 = µ2
William S. Gosset

H1 : µ1 = µ2 (a.k.a. Student)

ȳ1 ȳ2 (n1 1)S1

2
+ (n2 1)S2
2
t0 = q ⇠ tn1 +n2 2;
2
Sp =
Sp 1
+ 1 n1 + n2 2
n1 n2

if |t0 | > t↵/2,n1 +n2 2 then H0 is FALSE

Quantile function: the value of tn that has a probability less than α/2
 Distribution Cumulative prob.
0.4

1.0
0.8
0.3

pnorm(x, low = F)

0.6
dnorm(x)

0.2

0.4
0.1

0.2
0.0
0.0

-3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3
Cumulative prob. Quantile function
x x
1.0

2
“Upper Tail”
0.8

1
pnorm(x, low = F)

0.6

0
x
0.4

-1
0.2

-2
0.0

-3 -2 -1 0 1 2 3 0.0 0.2 0.4 0.6 0.8 1.0

x pnorm
 Student’s (t)-Test
µ1 6= µ2 µ1 = µ2
0.4

0.4
0.3

0.3
dt(x, k)

dt(x, k)
0.2

0.2
0.1

0.1
0.0

0.0
-4 -2 0 2 4 -4 -2 0 2 4

t t

p>↵ p<↵
One Sample, Unknown Variance

Hypoteses Test Statistics Reject H0 when

H0 : µ = µ0
|t0 | > t↵/2,n
H1 : µ = µ0 1

H0 : µ = µ0 ȳ p
µ0
t0 = t0 < t↵,n
H1 : µ < µ0 S/ n 1

H0 : µ = µ0
t0 > t↵,n
H1 : µ > µ0 1
Two Samples, Unknown Variance
2
1 = 2
2

Hypoteses Test Statistics Reject H0 when

H0 : µ 1 = µ 2
|t0 | > t↵/2,⌫
H1 : µ1 6= µ2 ȳ1 ȳ2
t0 = p
Sp 1/n1 + 1/n2
H0 : µ1 = µ2
⌫ = n 1 + n2 2
H1 : µ1 < µ2
(n1 1)S12 + (n2 1)S22
Sp2 =
n1 + n2 2
t0 > t↵,⌫
H0 : µ1 = µ2
H1 : µ1 > µ2
Two Samples, Unknown Variance
2
1 6= 2
2

Hypoteses Test Statistics Reject H0 when

H0 : µ 1 = µ 2
|t0 | > t↵/2,⌫
H1 : µ1 6= µ2 ȳ1 ȳ2
t0 = p
H0 : µ1 = µ2 S12 /n1 + S22 /n2
H1 : µ1 < µ2 (S12 /n1 + S22 /n2 )2
⌫= (S12 /n1 )2
+
(S22 /n2 )2 t0 > t↵,⌫
H0 : µ1 = µ2 n1 1 n2 1

H1 : µ1 > µ2
 Known Pop. Variance
Hypoteses Test Statistics Reject H0 when
H0 : µ = µ 0
|Z0 | > Z↵/2
H1 : µ 6= µ0 ȳ
µ0
One Sample

H0 : µ = µ 0 Z0 = p
/ n Z0 < Z↵
H1 : µ < µ 0
H0 : µ = µ 0 ) Z0 ⇠ N (1, 0)
H1 : µ > µ 0
Z0 > Z↵
H0 : µ 1 = µ 2
|Z0 | > Z↵/2
Two Samples

H1 : µ1 6= µ2
ȳ1 ȳ2
H0 : µ 1 = µ 2 Z0 = q 2 2 Z0 < Z↵
H1 : µ 1 < µ2 1
n1 + 2
n2
H0 : µ 1 = µ 2
H1 : µ 1 > µ2
Z0 > Z↵
 Paired t-Test
Statistical model:
⇢
i = 1, 2
yij = µi + j + ✏ij ;
j = 1, 2, . . . , n

Di↵erence between pairs is:

dj = y1j y2j ; j = 1, 2, . . . , n

The expected value of di↵erences is:

µd = E(dj )
= E(y1j y2j )
= E(y1j y2j )
= E(y1j ) E(y2j )
= (µ1 + j ) (µ2 + j)
= µ1 µ2
 Paired t-Test
Testing the hypothesis H0 : µ1 = µ2 means to test the couple of hypotheses:

H0 : µ d = 0
H1 : µd 6= 0

The test statistics is:

d¯
t0 = p
Sd / n
where:
X n
1
d¯ = dj
n j=1

and: sP sP
¯2 2
P 2
j (dj d) d
j j 1/n( j d j )
Sd = =
n 1 n 1
 Paired t-Test

The null-hypothesis H0 : µd = 0 can thus be

rejected when:

|t0 | > t↵/2,n 1

 Caveats

❖ In two-samples t-test:
❖ variances of populations are equal
❖ observations are independent random
variables N ID(µ, 2
)
❖ populations are normally distributed
 Inference on Variance

One Sample Two Samples

H0 : 2
= 2
0 H0 : 2
1 = 2
2
Test
H1 : 2
= 2
0 H1 : 2
1 = 2
2

2 SS (n 1)S 2 2 S12
Statistics 0 = 2 = 2 ⇠ n 1 F 0 = 2 ⇠ Fn 1 1,n2 1
0 0 S2

Conf. (n 1)S 2 (n 1)S 2 S12 2

S12
2 6 2
6 2 2 F↵,n1 1,n2 1 6 1
2 6 2 F1 ↵,n1 1,n2 1
Interval ↵/2,n 1 1 ↵/2,n 1 S2 2 S2
 Samples Independence
d1 = N (3, 1)
d2 = N (5, 1)
d3 = 2 + d1 + N (0, 0.5)

COV=
COV=-0.308016063713454
1.14033728906712
0.23899288508923 COV=
COV=0.991968132659053
1.14033728906712

777
77 7

666
66 6

555
d2

d3
d2

d3
55 5

44 4
44 4

33 3
3

1.5
1 2.0 22.5 3.03 3.5 4.0
4 4.5 5 1.5
1 2.0 22.5 3.03 3.5 4.0
4 4.5 5

d1 d1
 Normality Check
Normal Q-Q plot for d1 Normal Q-Q plot for d3
5

7
6
4
Sample Quantiles

Sample Quantiles

5
3

4
Normal Q-Q plot for U(1,5)
2

3
5
1

-2 -1 0 1 2 -2 -1 0 1 2
4
Sample Quantiles

Theoretical Quantiles Theoretical Quantiles

3
2
1

-2 -1 0 1 2

Theoretical Quantiles
 How to Build Q-Q Plots
# y f(x<y) q(f)

1 12.33 0.08 -1.43 Normal Q-Q Plot

2 12.34 0.20 -0.85

17
3 13.28 0.32 -0.47

16
Sample Quantiles (y)

15
4 14.31 0.44 -0.15

14
5 14.55 0.56 0.15

13
6 16.16 0.68 0.47
-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5
7 16.52 0.80 0.85
Theoretical Quantiles (q(f))

8 16.98 0.92 1.43

after Blom, G. (1958), Statistical estimates and transformed

(k a)/(n + 1 2a) k = 1, . . . , n; a = 3/8
beta variables, Wiley, used in MATLAB and R
 Goodness-of-Fit Test
❖ We might wish to test the hypothesis that a population
is normal (or uniform, or t-shaped, or …)
❖ build a set of k bins expected to have equal frequency
Ei in the target distribution
❖ count the sample frequency Oi in each bin
k
X 2
2 (Oi Ei ) 2 2 2
X0 = ⇠ k p 1, P = P( k p 1 > 0)
i=1
Ei
where p is the number of parameters
of the hypotesized distribution
 Operating Characteristic Curves
Operating Characteristic Curves Operating Characteristic Curves
Two samples, @5% One sample, @5%
1.0

1.0
P(Type II error), or (1-Power)

P(Type II error), or (1-Power)

0.8

0.8
2
2 3
3 4
0.6

0.6
4 5
5 10
0.4

0.4
10 20
20 50
50 100
0.2

0.2
100
0.0

0.0
0 1 2 3 4 5 6 0 1 2 3 4 5 6

d d

|µ1 µ2 | |µ µ0 |
d= ; d=
2 2
 Confidence Interval

If ✓ is an unknown parameter, we are interested in defining two statistics L

and U such that
P (L 6 ✓ 6 U ) = 1 ↵
The interval L 6 ✓ 6 U represents the confidence interval for the parameter
✓.
 Confidence Interval
Lets X1 , . . . , Xn represent a N ID(µ, ) sample. If:

X n
1
X̄ = Xi
n i
Pn 2
(X i X̄)
S2 = i
n 1
then
X̄ µ
t0 = p ⇠ tn 1
S/ n
is distributed as a Studen’s T with n 1 degrees of freedom. If c is the 95-th
percentile of that distribution, then:

P r( c < t0 < c) = 0.90

and, consequently:
p p
P r(X̄ cS/ n < µ < X̄ + cS/ n) = 0.90
 Confidence Interval

When the di↵erence between two variables is of interest, the confidence in-
terval can be described by the statistics

(ȳ1 ȳ2 ) (µ1 µ2 )

q ⇠ tn1 +n2 2
1 1
Sp n1 + n2

in fact:
0 1
(ȳ1 ȳ2 ) (µ1 µ2 )
P @ t↵/2,n1 +n2 2 6 q 6 t↵/2,n1 +n2 2
A=1 ↵
1 1
Sp n1 + n2
 Confidence Interval
And thus, since:
✓ r
1 1
P (ȳ1 ȳ2 ) t↵/2,n1 +n2 2 Sp + 6 µ1 µ2 6
n1 n2
r ◆
1 1
6 (ȳ1 ȳ2 ) + t↵/2,n1 +n2 2 Sp + =1 ↵
n1 n2

it results that the two statistics

q
1 1
L = (ȳ1 ȳ2 ) t↵/2,n1 +n2 2 Sp n1 + n2
q
1 1
U = (ȳ1 ȳ2 ) + t↵/2,n1 +n2 2 Sp n1 + n2

represent the limits of the confidence interval for the parameter µ1 µ2 .

 Outliers Detection

❖ Chauvenet criterion:
❖ sample ➜ ",σ2 ➜ P(max(distance)/σ2) ➜ reject if
P<0.5
❖ Grubbs’ test
v
max |Yi Ȳ | u
i=1,...,n n 1u t2↵/(2n),n
G0 = , if G0 > t 2
then reject H0
s n n 2+ t2↵/(2n),n 2
 Pearson’s Chi-square test
Freq None Some
Heavy 7 1 3
Never 87 18 84
Occas 12 3 4
Regul 9 1 7

❖ We have a contingency table (2-D array)

❖ We test the null hypothesis that the two margins of the
table refer to independent variables
Pearson’s Chi-square test
X (Oij Ei,j )2
2
=
ij
Eij
Eij = N p·j pi·
X c
Oi· Oij
pi· = =
N j=1
N
r
X
O·j Oij
p·j = =
N i=1
N