Vous êtes sur la page 1sur 16

Chapter 1

We need some basic knowledge to understand the topics in analysis of variance.

Vectors:
A vector Y is an ordered n-tuple of real numbers. A vector can be expressed as row vector or a column
vector as
y1

y
Y = 2

yn

is a column vector of order n 1 and

Y ' = ( y1 , y2 ,..., yn )
is a row vector of order 1 n.
If all yi = 0 for all i = 1,2,,n then Y ' = (0, 0,..., 0) is called the null vector.

If
x1

x2
=
X =
, Y

xn
then

y1

y2
=
, Z

yn

z1

z2

zn

x1 + y2

x2 + y2

X +Y
=
=
, kY

xn + yn

ky1

ky2

kyn

X + (Y + Z ) = ( X + Y ) + Z
X '(Y + Z )= X ' Y + X ' Z
k=
( X ' Y ) (=
kX ) ' Y X '(kY )
k ( X + Y ) = kX + kY
X ' Y= x1 y1 + x2 y2 + ... + xn yn

where k is a scalar.

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

Orthogonal vectors:
Two vectors X and Y are said to be orthogonal if X=
' Y Y=
'X 0.

The null vector is orthogonal to every vector X and is the only such vector.

Linear combination:
If x1 , x2 ,..., xm are m vectors and k1 , k2 ,..., km are m scalars, then
m

t = ki xi
i =1

is called the linear combination of x1 , x2 ,..., xm .

Linear independence
If X 1 , X 2 ,..., X m are m vectors then they are said

m

k X
i =1

m

k x
i =1

i i

= 0 then

Any set of non-null pair-wise orthogonal vectors is linearly independent.

If m > 1 vectors are linearly dependent, it is always possible to express at least one of them as a
linear combination of the others.

Linear function:
Let

vector of scalars and X = ( x1 , x2 ,..., xm ) be a m 1 vector of

variables, then K ' Y = ki yi is called a linear function or linear form. The vector K is called the
i =1

x1

1 m
1
x2 1 1' X
x =
xi
=
=
(1,1,...,1)

m m
m i =1
m

xm

where 1'm is a m 1 vector of all elements unity.

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

Contrast:
m

i =1

k
i =1

= 0.

For example, the linear functions

x1 x2 , 2 x1 3 x2 + x3 ,

x
x1
x2 + 3
2
3

are contrasts.

x
i=

or to

1 m
x.i .
m i =1

Every contrast in x1 , x2 ,..., xn can be written as a linear combination of (m - 1) contrasts

x1 x2 , x1 x3 ,..., x1 xm .

Matrix:
A matrix is a rectangular array of real numbers. For example
a11

a21

am1

a22 ... a2 n

am 2 ... amn

A = diag (a11 , a22 ,..., amm ).

If m = n (square matrix) and aij = 0 for i > j , then A is called an upper triangular matrix.
On the other hand if m = n and aij = 0 for i < j then A is called a lower triangular matrix.

If A is a m n matrix, then the matrix obtained by writing the rows of A and

columns of A as columns of A and rows of A respectively is called the transpose of a matrix A
and is denoted as A ' .

If A = A ' then A is skew symmetric matrix.

A matrix whose all elements are equal to zero is called as null matrix.

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

An identity matrix is a square matrix of order p whose diagonal elements are unity (ones) and all
the off diagonal elements are zero. It is denotes as I p .

If A and B are matrices of order m n then ( A + B) ' =A '+ B '.

If A and B are the matrices of order m x n and n x p respectively and k is any scalar, then

(=
kA) B A=
(kB) k=
( AB) kAB.

If the orders of matrices A is m x n, B is n x p and C is p x q then ( AB)C = A( BC ).

A AI
=
A. .
If A is the matrix of order m x n then I m=
n

Trace of a matrix:
The trace of n n matrix A , denoted as tr(A) or trace(A) is defined to be the sum of all the diagonal
n

i =1

If A is of order m n and B is of order n m , then

tr ( AB) = tr ( BA) .

If A is n n matrix and P is any nonsingular n n matrix then

tr ( A) = tr ( P 1 AP).
If P is an orthogonal matrix than tr ( A) = tr ( P ' AP ).
*

If A and B are n n matrices, a and b are scalars then

tr (aA + bB=
) a tr ( A) + b tr ( B) .

If A is an m n matrix, then

( A ' A) tr=
( AA ')
tr=

=j 1 =i 1

2
ij

and

tr=
( A ' A) tr=
( AA ') 0 if and only if A = 0.

If A is n n matrix then

tr ( A ') = trA .

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

Rank of matrices
The rank of a matrix A of m n is the number of linearly independent rows in A.
Let B be any other matrix of order n q.

Rank A is equal to the maximum order of all nonsingular square sub-matrices of A.

rank
=
( AA ') rank=
( A ' A) rank
=
( A) rank ( A ') .

A is of full column rank if rank(A) = n < m.

Inverse of a matrix
The inverse of a square matrix A of order m, is a square matrix of order m, denoted as A1 , such that
1
1
A=
A AA
=
Im.

( A1 ) 1 = A.

If A is non singular, then ( A ') 1 = ( A1 ) '

If A and B are non-singular matrices of same order, then their product , if defined, is also
nonsingular and ( AB) 1 = B 1 A1.

Idempotent matrix:
2
A square matrix A is called idempotent if A
=
AA
= A. .

If A is an n n idempotent matrix with rank(A) = r n. Then

eigenvalues of A are 1 or 0.

trace
=
( A)

If A is idempotent then (I A) is also idempotent and A(I - A) = (I - A)A = 0.

rank
=
( A)

r.

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

If A is a given matrix of order m n and X and Y are two given vectors of order m 1 and n 1
respectively
m

X ' AY = aij xi y j
=i 1 =j 1

If A is square matrix of order m and X = Y , then

X ' AX= a11 x12 + .... + amm xm2 + (a12 + a21 ) x1 x2 + ... + (am 1,m + am ,m 1 ) xm 1 xm .

If A is symmetric also, then

X ' AX= a11 x12 + .... + amm xm2 + 2a12 x1 x2 + ..... + 2am 1,m xm 1 xm
m

= aij xi x j
=i 1 =j 1

The matrix A is called the matrix of quadratic form.

The quadratic form X ' AX and the matrix A of the form is called.
Positive definite if X ' AX > 0 for all x 0 .
Positive semi definite if X ' AX 0 for all x 0 .
Negative definite if X ' AX < 0 for all x 0 .
Negative semi definite if X ' AX 0 for all x 0 .

and if aii = 0 then aij = 0 for all j, and

a ji = 0 for all j.

If P is any nonsingular matrix and A is any positive definite matrix (or positive semi-definite
matrix) then P ' AP is also a positive definite matrix (or positive semi-definite matrix).

A matrix A is positive definite if and only if there exists a non-singular matrix P such that
A = P ' P.

If A is m n matrix and rank ( A=

) m < n then AA ' is positive definite and A ' A is positive
semidefinite.

If A m n matrix and rank ( A ) = k < m < n , then both A ' A and AA ' are positive semidefinite.

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

Simultaneous linear equations

The set of m linear equations in n unknowns x1 , x2 ,..., xn and scalars aij

and bi ,

=
i 1,=
2,..., m, j 1, 2,..., n of the form
a11 x1 + a12 x2 + .... + a1n xn =
b1
a21 x1 + a22 x2 + .... + a2 n xn =
b2

am1 x1 + am 2 x2 + .... + amn xn =

bm
can be formulated as

AX = b

where A is a real matrix of known scalars of order m n called as coefficient matrix, X is n 1 real
vector and b is n 1 real vector of known scalars given by

a21 a22 ... a2 n

A
, is an m n real matrix called as coefficient matrix,

am1 am 2 ... amn

x1
b1

x2

b2 , is an m 1 real vector.
X=
is an n 1 vector of variables and b =

xn
bm

If A is n n nonsingular matrix, then AX = b has a unique solution.

Let B = [A, b] is an augmented matrix. A solution to AX = b exist if and only if rank(A) = rank(B).

If A is an m n matrix of rank m , then AX = b has a solution.

Linear homogeneous system AX = 0 has a solution other than X = 0 if and only if rank(A) < n.

Let the n n matrix be partitioned as A = [a1 , a2 ,..., an ] where ai is an n 1 vector of the

elements of ith column of A. A necessary and sufficient condition that A is an orthogonal matrix
is given by the following:

ai' ai 1=
(i )=
for i 1, 2,..., n
'
j 1, 2,..., n.
(ii ) a=
0 for i =
ia j

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

Orthogonal matrix
A square matrix A is called an orthogonal matrix if A=
' A AA
=' I or equivalently if A1 = A '.

If A is an n n matrix and let P is an n n orthogonal matrix, then the determinants of A and

P ' AP are the same.

Random vectors:
Let Y1 , Y2 ,..., Yn be n random variables then Y = (Y1 , Y2 ,..., Yn ) '

The covariance matrix or dispersion matrix of Y is

Cov(Y1 , Y2 ) ... Cov(Y1 , Yn )
Var (Y1 )

Cov(Y2 , Y1 ) Var (Y2 ) ... Cov(Y2 , Yn )

Var (Y ) =

Var (Yn )
Cov(Yn , Y1 ) Cov(Yn , Y2 ) ...

which is a symmetric matrix

If Y1 , Y2 ,..., Yn are independently distributed, then the covariance matrix is a diagonal matrix.

n

If Y1 , Y2 ,..., Yn are n random variables, and k1 , k2 ,.., kn are scalars , then

kY
i =1

i i

is called a linear

function of random variables Y1 , Y2 ,..., Yn .

n

If Y

(Y
=
(k1 , k2 ,..., kn ) ' then K ' Y = kiYi .
1 , Y2 ,..., Yn ) ', K
i =1

( K ' Y ) K=
' E (Y )

k E (Y ) and
i =1

the variance of K ' Y is Var ( K ' Y ) = K 'Var (Y ) K .

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

A

= ( 1 , 2 ,..., n ) and dispersion matrix if its probability density function is

=
f (Y / , )

1
(2 ) n /2

n /2

exp (Y ) ' 1 (Y )
2

assuming is a nonsingular matrix.

Chi-square distribution

If Y1 , Y2 ,..., Yk are independently distributed following the normal distribution random variables
k

with common mean 0 and common variance 1, then the distribution of

Y
i =1

is called the 2 -

The probability density function of 2 -distribution with k degrees of freedom is given as

f 2 ( x)
=

k
1
1
x
2
x
exp ; 0 < x <
k /2
(k / 2)2
2

If Y1 , Y2 ,..., Yk are independently distributed following the normal distribution with common mean
1
0 and common variance 2 , then 2

Y
i =1

has 2 distribution with k degrees of freedom.

If the random variables Y1 , Y2 ,..., Yk are normally distributed with non-null means 1 , 2 ,..., k but
k

common variance 1, then the distribution of

Y
i =1

has noncentral

2 distribution with k

degrees of freedom and noncentrality parameter = i2 .

i =1

If Y1 , Y2 ,..., Yk are independently and normally distributed following the normal distribution with
means 1 , 2 ,..., k but common variance 2 then

Y
i =1

k

i =1

2
i

Var (U ) = 2k .

If U has a noncentral Chi-square distribution with k degrees of freedom and

) 2 k + 4 .
noncentrality parameter then E (U )= k + and Var (U=
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

If

distributed random variables with

each U i having a

noncentral Chi-square distribution with ni degrees of freedom and non centrality parameter

i , i = 1, 2,..., k

then

U
i =1

has noncentral Chi-square

distribution with

n
i =1

degrees of

i =1

Let

and positive

definite covariance matrix . Then X ' AX is distributed as noncentral 2 with k degrees of

freedom if and only if A is an idempotent matrix of rank k.

Let X = ( X 1 , X 2 ,..., X n ) has a multivariate normal distribution with mean vector and positive
definite covariance matrix . Let the two quadratic forms X ' A1 X

is distributed as 2 with n1 degrees of freedom and noncentrality parameter

' A1 and
X ' A2 X is distributed as 2 with

degrees of freedom and noncentrality parameter

' A2 .
0
Then X ' A1 X and X ' A2 X are independently distributed if A1A2 =

t-distribution

If
X has a normal distribution with mean 0 and variance 1,
Y has a 2 distribution with n degrees of freedom, and
X and Y are independent random variables,
then the distribution of the statistic T =

X
is called the t-distribution with n degrees of
Y /n

freedom. The probability density function of t is

n +1

2
=
fT (t )
n
n
2

If the mean of X is

t2
1 +
n

n +1

- < t <

X
Y /n

distribution with n degrees of freedom and noncentrality parameter .

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

10

F-distribution

freedom respectively, then the distribution of the statistic F =

X /m
is called the F-distribution
Y /n

m + n m

2 n
=
fF ( f )
m n

2 2

m /2

m2

m
1 + n

m+n

; 0< f <

If X has a noncentral Chi-square distribution with m degrees of freedom and noncentrality

parameter ;Y has a 2 distribution with n degrees of freedom, and X and Y are independent
random variables, then the distribution of F =

X /m
is the noncentral F distribution with m and
Y /n

n degrees of freedom and noncentrality parameter .

Linear model:
Suppose there are n observations. In the linear model, we assume that these observations are the values
taken by n random variables Y1 , Y2 ,.., Yn satisfying the following conditions:
1.

E (Yi )

is

linear

combination

of

unknown

parameters

1 , 2 ,..., p ,
) xi11 + xi 2 2 + ... + xip p , =
E (Yi=
i 1, 2,..., n
where xij ' s are known constants.
2. Y1 , Y2 ,..., Yn are uncorrelated and normality distributed with variance Var (Yi ) = 2 .
The linear model can be rewritten by introducing independent normal random variables following

N (0, 2 ) ,as
Yi= xi11 + xi 2 2 + .... + xip p + i , i= 1, 2,..., n.

These equations can be written using the matrix notations as

=
Y X +
where Y is a n 1 vector of observation, X is a n p matrix of n observations on each of X 1 , X 2 ,..., X p
variables, is a p 1 vector of parameters and is a n 1 vector of random error components with

~ N (0, 2 I n ). Here Y is called study or dependent. variable, X 1 , X 2 ,..., X p are called explanatory
or independent variables and 1 , 2 ,..., p are called as regression coefficients.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

11

Alternatively since Y ~ N ( X , 2 I ) so the linear model can also be expressed in the expectation form
as a normal random variable Y with

E ( y) = X
Var ( y ) = 2 I .
Note that and 2 are unknown but X is known.

Estimable functions:
A linear parametric function

' of the parameter is said to be an estimable parametric function or

estimable if there exists a linear function of random variables ' y of Y where Y = (Y1 , Y2 ,..., Yn ) ' such
that

E ( ' y ) = '
with = (1 , 2 ,..., n ) ' and = (1 , 2 ,..., n ) ' being vectors of known scalars.

Best linear unbiased estimates (BLUE)

The unbiased minimum variance linear estimate 'Y of an estimable function ' is called the best
linear unbiased estimate of ' .

the BLUE of (a11 + a2 2 ) ' .

If

'

is estimable, its best estimate is ' where is any solution of the equations

X ' X = X 'Y .

Least squares estimation

The least squares estimate of is =
Y X + is the value of which minimizes the
error sum of squares ' .

Let
S ==
' (Y X ) '(Y X )
=
Y 'Y 2 ' X 'Y + ' X ' X .

Minimizing S with respect to involves

S
=0

X 'X =
X 'Y
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

12

which is termed as normal equation. This normal equation has a unique solution given by

= ( X ' X ) 1 X ' Y
assuming

rank ( X ) = p .

Note that

2S
= X 'X
'

is a positive

definite matrix. So

= ( X ' X ) 1 X ' Y is the value of which minimizes ' and is termed as ordinary least squares
estimator of .

In this case, 1 , 2 ,..., p are estimable and consequently all the linear parametric function are
estimable.

E ( ) ( X=
' X ) 1 X ' E (Y ) (=
X ' X ) 1 X ' X
=

Var ( )

If ' and ' are the estimates of ' and ' respectively, then

(=
X ' X ) 1 X 'Var (Y ) X ( X ' X ) 1 2 ( X ' X ) 1

=
Var ( ' ) =
'Var ( ) 2 [ '( X ' X ) 1 ]

E (Y X ) =
0.

Linear model with correlated observations:

In the linear model

=
Y X +
with E ( ) = 0, Var ( ) = and is normally distributed, we find

E (Y ) = X , Var (Y ) =

Assuming to be positive definite, so we can write

=P ' P

Y X + by P, we get
where P is nonsingular matrix. Premultiplying =
PY PX + P
=
or =
Y* X * + *
where
Y * PY
, X * PX =
and * P .
=
=

=
E ( *0 0=
and Var ( *) 2 I .
Note that in this model

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

13

Distribution of 'Y :

Y X + , ~ N (0, 2 I )
In the linear model =

consider a linear function 'Y which is normally

distributed with

E ( ' Y ) = ' X ,
Var ( ' Y ) = 2 ( ' ).
Then

'Y
' X
~ N
,1 .
'
'

Further,

( ' Y ) 2
has a noncentral Chi-square distribution with one degrees of freedom and noncentrality
2 '

parameter

( ' X ) 2
.
2 '

Degrees of freedom:
A linear function 'Y of the observations ( 0) is said to carry one degrees of freedom. A set of
linear functions L ' Y where L is r x n matrix, is said to have M degrees of freedom if there exists M
linearly independent functions in the set and no more. Alternatively, the degrees of freedom carried by
the set L ' Y equals rank ( L) . When the set L ' Y are the estimates of ' , the degrees of freedom of the
set L ' Y will also be called the degrees of freedom for the estimates of ' .
Sum of squares:
If 'Y is a linear function of observations, then the projection of Y on is the vector
this projection is called the sum of squares (SS) due to ' y is given by

Y '
. . The square of
'

( ' Y ) 2
. Since 'Y has one degree of
'

freedom, so the SS due 'Y to has one degree of freedom.

The sum of squares and the degrees of freedom arising out of the mutually orthogonal sets of functions can be
added together to give the sum of squares and degrees of freedom for the set of all the function together and
vice versa.
Let X = ( X 1 , X 2 ,..., X n ) has a multivariate normal distribution with mean vector and positive definite
covariance matrix . Let the two quadratic forms.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

14

X ' A, X is distribution 2 with n1 degrees of freedom and noncentrality parameter ' A1 and

X ' A2 X is distributed as 2 with n2 degrees of freedom and noncentrality parameter ' A2 .

0.
Then X ' A1 X and X ' A2 X are independently distributed if A1A2 =

Fisher-Cochran theorem
If X = ( X 1 , X 2 ,..., X n ) has multivariate normal distribution with mean vector and positive definite
covariance matrix and let

X ' 1 X = Q1 + Q2 + ... + Qk
where

Ai ) N=
1, 2,..., k .
i, i

Then

Qi ' s are independently distributed

noncentral Chi-square distribution with N i degrees of freedom and noncentrality parameter ' Ai if
k

and only if

N
i =1

= N is which case

' 1 =
' Ai .
i =1

Derivatives of quadratic and linear forms:

Let X = ( x1 , x2 ,..., xn ) ' and f(X) be any function of n independent variables x1 , x2 ,..., xn , then
f ( X )
x
1

f ( X )
f ( X )

= x2 .
X

f ( X )
x
n

If K = (k1 , k2 ,..., kn ) ' is a vector of constants, then

K ' X
=K
X
If A is an m n matrix, then

X ' AX
= 2( A + A ') X .
X

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

15

Let Y is an n 1 vector having multivariate normal distribution N ( , I ) and B be an m n

matrix. Then the m 1 vector linear form BY is independent of the quadratic form Y ' AY if BA =
0 where A is a symmetric matrix of known elements.

Let Y is an n 1 vector having multivariate normal distribution N ( , ) with rank () =n . If

BA =
0 , then the quadratic form Y ' AY is independent of linear form BY where B is an m n

matrix.

Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur

16