Revision

Use SSE - Sum of squared errors - to nd estimates of 0 and 1 . SSE =

n i=1

(Yi Yi )2

Minimise this w.r.t. 0 and 1 to get 0 and 1 . X 0 = Y 1 X , 1 = i Yi nX Y = SXY SXX Xi2 nX 2 SXX and SXY can be written in a variety of forms.

1/9

Example 1 Bond strength (Y ) vs Age in months (X )

Aim: predict strength of bond between two surfaces given the age of the bond.

2600

2400

Shear Strength

2200

1800

2000

1600 0

10

12

14

16

18

20

22

24

26

Age (months)

SXX = 1097.538, SXY = 40603.2, X = 13.37, Y = 2134. 1 = 36.99, 0 = 2629.36 =

2/9

Example 2 GNP (Y billions USD) vs Population (X millions)

GNP (Y ) 193 295 386 401 628 497 Pop (X ) 111.4 115 120 112 132.3 129.3
Yi Y Xi X (Yi Y )(Xi X ) (Xi X )2

1. Sketch a graph of the data. 2. Calculate X and Y . 3. Fill in the rest of the table. 4. Calculate 0 and 1 .
3/9

Linear estimators
1 = = = =
(X X ) n i (Xj X )2 j=1

i=1 (Xi X )(Yi Y ) n 2 j=1 (Xj X ) n i=1 (Xi X )Yi n 2 j=1 (Xj X ) n Xi X n 2 Yi j=1 (Xj X ) i=1 n i=1

ki Yi

where ki =

is a known constant.

Thus 1 is a linear combination of the Yi = a linear estimator Exercise: show that 0 is a linear estimator.
4/9

Properties of estimators
An estimator is a sample statistic and as a result it has a sampling distribution. If the mean of the sampling distribution is not equal to the parameter being estimated then the estimator is biased; otherwise it is unbiased.
Unbiased estimate Biased estimate

True Value

Apart from being unbiased it is also desirable for an estimate to have the smallest possible variance.
Smaller variance

Bigger variance True Value

5/9

Properties of the LS estimators

Assumptions
Yi = 0 + 1 Xi + i , i = 1, . . . , n with IE(i ) = 0, Var (i ) = 2 i, Cov (i , j ) = 0 i = j

The Gauss-Markov theorem

The LS estimates are the best (minimum variance) linear unbiased estimators (BLUE) among all unbiased linear estimators.

IE(0 ) = 0 , IE(1 ) = 1 = both unbiased 2 2 2 2 Var (0 ) = + X , Var (1 ) =

n SXX SXX

Cov (1 , 0 ) =

X 2 SXX
6/9

Note: design considerations

2 To minimise Var (1 ) = S it appears that we should maximise SXX . XX If it is possible for us to choose the X values, we could put half of them at their maximum and half at their minimum possible values.

But . . . does not allow us to check whether linearity is an appropriate t.

7/9

Point estimation of mean response

Regression function: IE(Y ) = 0 + 1 X Estimated (tted) regression function: Y = 0 + 1 X

Y is a point estimator of mean response when predictor is equal to X .

Y is an unbiased estimator of IE(Y ). If (Xi , Yi ) belongs to the data set, Y = 0 + 1 X is the tted value and i = Yi Yi is the estimated residual.

Note: Yi = 0 + 1 Xi + i = i = Yi IE(Yi ). Hence i = Yi Yi is an estimate of i .

8/9

Diagram

9/9