Académique Documents
Professionnel Documents
Culture Documents
Vectors:
A vector Y is an ordered n-tuple of real numbers. A vector can be expressed as row vector or a column
vector as
y1
y
Y = 2
yn
Y ' = ( y1 , y2 ,..., yn )
is a row vector of order 1 n.
If all yi = 0 for all i = 1,2,,n then Y ' = (0, 0,..., 0) is called the null vector.
If
x1
x2
=
X =
, Y
xn
then
y1
y2
=
, Z
yn
z1
z2
zn
x1 + y2
x2 + y2
X +Y
=
=
, kY
xn + yn
ky1
ky2
kyn
X + (Y + Z ) = ( X + Y ) + Z
X '(Y + Z )= X ' Y + X ' Z
k=
( X ' Y ) (=
kX ) ' Y X '(kY )
k ( X + Y ) = kX + kY
X ' Y= x1 y1 + x2 y2 + ... + xn yn
where k is a scalar.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
Orthogonal vectors:
Two vectors X and Y are said to be orthogonal if X=
' Y Y=
'X 0.
The null vector is orthogonal to every vector X and is the only such vector.
Linear combination:
If x1 , x2 ,..., xm are m vectors and k1 , k2 ,..., km are m scalars, then
m
t = ki xi
i =1
Linear independence
If X 1 , X 2 ,..., X m are m vectors then they are said
k X
i =1
k x
i =1
i i
= 0 then
If m > 1 vectors are linearly dependent, it is always possible to express at least one of them as a
linear combination of the others.
Linear function:
Let
variables, then K ' Y = ki yi is called a linear function or linear form. The vector K is called the
i =1
m m
m i =1
m
xm
Contrast:
m
k
i =1
= 0.
x1 x2 , 2 x1 3 x2 + x3 ,
x
x1
x2 + 3
2
3
are contrasts.
x
i=
or to
1 m
x.i .
m i =1
x1 x2 , x1 x3 ,..., x1 xm .
Matrix:
A matrix is a rectangular array of real numbers. For example
a11
a21
am1
a22 ... a2 n
am 2 ... amn
If m = n (square matrix) and aij = 0 for i > j , then A is called an upper triangular matrix.
On the other hand if m = n and aij = 0 for i < j then A is called a lower triangular matrix.
A matrix whose all elements are equal to zero is called as null matrix.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
An identity matrix is a square matrix of order p whose diagonal elements are unity (ones) and all
the off diagonal elements are zero. It is denotes as I p .
If A and B are the matrices of order m x n and n x p respectively and k is any scalar, then
A AI
=
A. .
If A is the matrix of order m x n then I m=
n
Trace of a matrix:
The trace of n n matrix A , denoted as tr(A) or trace(A) is defined to be the sum of all the diagonal
n
tr ( AB) = tr ( BA) .
tr ( A) = tr ( P 1 AP).
If P is an orthogonal matrix than tr ( A) = tr ( P ' AP ).
*
tr (aA + bB=
) a tr ( A) + b tr ( B) .
If A is an m n matrix, then
( A ' A) tr=
( AA ')
tr=
=j 1 =i 1
2
ij
and
tr=
( A ' A) tr=
( AA ') 0 if and only if A = 0.
If A is n n matrix then
tr ( A ') = trA .
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
Rank of matrices
The rank of a matrix A of m n is the number of linearly independent rows in A.
Let B be any other matrix of order n q.
rank
=
( AA ') rank=
( A ' A) rank
=
( A) rank ( A ') .
Inverse of a matrix
The inverse of a square matrix A of order m, is a square matrix of order m, denoted as A1 , such that
1
1
A=
A AA
=
Im.
( A1 ) 1 = A.
If A and B are non-singular matrices of same order, then their product , if defined, is also
nonsingular and ( AB) 1 = B 1 A1.
Idempotent matrix:
2
A square matrix A is called idempotent if A
=
AA
= A. .
eigenvalues of A are 1 or 0.
trace
=
( A)
rank
=
( A)
r.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
Quadratic forms:
If A is a given matrix of order m n and X and Y are two given vectors of order m 1 and n 1
respectively
m
X ' AY = aij xi y j
=i 1 =j 1
X ' AX= a11 x12 + .... + amm xm2 + (a12 + a21 ) x1 x2 + ... + (am 1,m + am ,m 1 ) xm 1 xm .
X ' AX= a11 x12 + .... + amm xm2 + 2a12 x1 x2 + ..... + 2am 1,m xm 1 xm
m
= aij xi x j
=i 1 =j 1
The quadratic form X ' AX and the matrix A of the form is called.
Positive definite if X ' AX > 0 for all x 0 .
Positive semi definite if X ' AX 0 for all x 0 .
Negative definite if X ' AX < 0 for all x 0 .
Negative semi definite if X ' AX 0 for all x 0 .
a ji = 0 for all j.
If P is any nonsingular matrix and A is any positive definite matrix (or positive semi-definite
matrix) then P ' AP is also a positive definite matrix (or positive semi-definite matrix).
A matrix A is positive definite if and only if there exists a non-singular matrix P such that
A = P ' P.
If A m n matrix and rank ( A ) = k < m < n , then both A ' A and AA ' are positive semidefinite.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
and bi ,
=
i 1,=
2,..., m, j 1, 2,..., n of the form
a11 x1 + a12 x2 + .... + a1n xn =
b1
a21 x1 + a22 x2 + .... + a2 n xn =
b2
AX = b
where A is a real matrix of known scalars of order m n called as coefficient matrix, X is n 1 real
vector and b is n 1 real vector of known scalars given by
A
, is an m n real matrix called as coefficient matrix,
b2 , is an m 1 real vector.
X=
is an n 1 vector of variables and b =
xn
bm
Let B = [A, b] is an augmented matrix. A solution to AX = b exist if and only if rank(A) = rank(B).
Linear homogeneous system AX = 0 has a solution other than X = 0 if and only if rank(A) < n.
ai' ai 1=
(i )=
for i 1, 2,..., n
'
j 1, 2,..., n.
(ii ) a=
0 for i =
ia j
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
Orthogonal matrix
A square matrix A is called an orthogonal matrix if A=
' A AA
=' I or equivalently if A1 = A '.
Random vectors:
Let Y1 , Y2 ,..., Yn be n random variables then Y = (Y1 , Y2 ,..., Yn ) '
Var (Y ) =
Var (Yn )
Cov(Yn , Y1 ) Cov(Yn , Y2 ) ...
If Y1 , Y2 ,..., Yn are independently distributed, then the covariance matrix is a diagonal matrix.
kY
i =1
i i
is called a linear
If Y
(Y
=
(k1 , k2 ,..., kn ) ' then K ' Y = kiYi .
1 , Y2 ,..., Yn ) ', K
i =1
k E (Y ) and
i =1
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
1
(2 ) n /2
n /2
exp (Y ) ' 1 (Y )
2
Chi-square distribution
If Y1 , Y2 ,..., Yk are independently distributed following the normal distribution random variables
k
Y
i =1
is called the 2 -
f 2 ( x)
=
k
1
1
x
2
x
exp ; 0 < x <
k /2
(k / 2)2
2
If Y1 , Y2 ,..., Yk are independently distributed following the normal distribution with common mean
1
0 and common variance 2 , then 2
Y
i =1
If the random variables Y1 , Y2 ,..., Yk are normally distributed with non-null means 1 , 2 ,..., k but
k
Y
i =1
has noncentral
2 distribution with k
If Y1 , Y2 ,..., Yk are independently and normally distributed following the normal distribution with
means 1 , 2 ,..., k but common variance 2 then
Y
i =1
i =1
2
i
Var (U ) = 2k .
) 2 k + 4 .
noncentrality parameter then E (U )= k + and Var (U=
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
If
each U i having a
noncentral Chi-square distribution with ni degrees of freedom and non centrality parameter
i , i = 1, 2,..., k
then
U
i =1
distribution with
n
i =1
degrees of
i =1
Let
and positive
Let X = ( X 1 , X 2 ,..., X n ) has a multivariate normal distribution with mean vector and positive
definite covariance matrix . Let the two quadratic forms X ' A1 X
' A1 and
X ' A2 X is distributed as 2 with
' A2 .
0
Then X ' A1 X and X ' A2 X are independently distributed if A1A2 =
t-distribution
If
X has a normal distribution with mean 0 and variance 1,
Y has a 2 distribution with n degrees of freedom, and
X and Y are independent random variables,
then the distribution of the statistic T =
X
is called the t-distribution with n degrees of
Y /n
n +1
2
=
fT (t )
n
n
2
If the mean of X is
t2
1 +
n
n +1
- < t <
X
Y /n
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
10
F-distribution
X /m
is called the F-distribution
Y /n
2 n
=
fF ( f )
m n
2 2
m /2
m2
m
1 + n
m+n
; 0< f <
X /m
is the noncentral F distribution with m and
Y /n
Linear model:
Suppose there are n observations. In the linear model, we assume that these observations are the values
taken by n random variables Y1 , Y2 ,.., Yn satisfying the following conditions:
1.
E (Yi )
is
linear
combination
of
unknown
parameters
1 , 2 ,..., p ,
) xi11 + xi 2 2 + ... + xip p , =
E (Yi=
i 1, 2,..., n
where xij ' s are known constants.
2. Y1 , Y2 ,..., Yn are uncorrelated and normality distributed with variance Var (Yi ) = 2 .
The linear model can be rewritten by introducing independent normal random variables following
N (0, 2 ) ,as
Yi= xi11 + xi 2 2 + .... + xip p + i , i= 1, 2,..., n.
=
Y X +
where Y is a n 1 vector of observation, X is a n p matrix of n observations on each of X 1 , X 2 ,..., X p
variables, is a p 1 vector of parameters and is a n 1 vector of random error components with
~ N (0, 2 I n ). Here Y is called study or dependent. variable, X 1 , X 2 ,..., X p are called explanatory
or independent variables and 1 , 2 ,..., p are called as regression coefficients.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
11
Alternatively since Y ~ N ( X , 2 I ) so the linear model can also be expressed in the expectation form
as a normal random variable Y with
E ( y) = X
Var ( y ) = 2 I .
Note that and 2 are unknown but X is known.
Estimable functions:
A linear parametric function
estimable if there exists a linear function of random variables ' y of Y where Y = (Y1 , Y2 ,..., Yn ) ' such
that
E ( ' y ) = '
with = (1 , 2 ,..., n ) ' and = (1 , 2 ,..., n ) ' being vectors of known scalars.
If
'
is estimable, its best estimate is ' where is any solution of the equations
X ' X = X 'Y .
Let
S ==
' (Y X ) '(Y X )
=
Y 'Y 2 ' X 'Y + ' X ' X .
S
=0
X 'X =
X 'Y
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
12
which is termed as normal equation. This normal equation has a unique solution given by
= ( X ' X ) 1 X ' Y
assuming
rank ( X ) = p .
Note that
2S
= X 'X
'
is a positive
definite matrix. So
= ( X ' X ) 1 X ' Y is the value of which minimizes ' and is termed as ordinary least squares
estimator of .
In this case, 1 , 2 ,..., p are estimable and consequently all the linear parametric function are
estimable.
E ( ) ( X=
' X ) 1 X ' E (Y ) (=
X ' X ) 1 X ' X
=
Var ( )
If ' and ' are the estimates of ' and ' respectively, then
(=
X ' X ) 1 X 'Var (Y ) X ( X ' X ) 1 2 ( X ' X ) 1
=
Var ( ' ) =
'Var ( ) 2 [ '( X ' X ) 1 ]
E (Y X ) =
0.
=
Y X +
with E ( ) = 0, Var ( ) = and is normally distributed, we find
E (Y ) = X , Var (Y ) =
Y X + by P, we get
where P is nonsingular matrix. Premultiplying =
PY PX + P
=
or =
Y* X * + *
where
Y * PY
, X * PX =
and * P .
=
=
=
E ( *0 0=
and Var ( *) 2 I .
Note that in this model
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
13
Distribution of 'Y :
Y X + , ~ N (0, 2 I )
In the linear model =
distributed with
E ( ' Y ) = ' X ,
Var ( ' Y ) = 2 ( ' ).
Then
'Y
' X
~ N
,1 .
'
'
Further,
( ' Y ) 2
has a noncentral Chi-square distribution with one degrees of freedom and noncentrality
2 '
parameter
( ' X ) 2
.
2 '
Degrees of freedom:
A linear function 'Y of the observations ( 0) is said to carry one degrees of freedom. A set of
linear functions L ' Y where L is r x n matrix, is said to have M degrees of freedom if there exists M
linearly independent functions in the set and no more. Alternatively, the degrees of freedom carried by
the set L ' Y equals rank ( L) . When the set L ' Y are the estimates of ' , the degrees of freedom of the
set L ' Y will also be called the degrees of freedom for the estimates of ' .
Sum of squares:
If 'Y is a linear function of observations, then the projection of Y on is the vector
this projection is called the sum of squares (SS) due to ' y is given by
Y '
. . The square of
'
( ' Y ) 2
. Since 'Y has one degree of
'
14
X ' A, X is distribution 2 with n1 degrees of freedom and noncentrality parameter ' A1 and
Fisher-Cochran theorem
If X = ( X 1 , X 2 ,..., X n ) has multivariate normal distribution with mean vector and positive definite
covariance matrix and let
X ' 1 X = Q1 + Q2 + ... + Qk
where
Then
noncentral Chi-square distribution with N i degrees of freedom and noncentrality parameter ' Ai if
k
and only if
N
i =1
= N is which case
' 1 =
' Ai .
i =1
f ( X )
f ( X )
= x2 .
X
f ( X )
x
n
K ' X
=K
X
If A is an m n matrix, then
X ' AX
= 2( A + A ') X .
X
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
15
matrix.
Analysis of Variance | Chapter 1 | Linear Algebra, Matrix Theory and Dist. | Shalabh, IIT Kanpur
16