MATHEMATICS

MTHC02C02
Lecture 6
What is Regression
( ) y f x =
error
Given (x
0
,y
0
), (x
1
,y
1
), (x
n
,y
n
), such that we know that the best fit is the
function y=f(x). This best fit function will be calculated by minimizing the squares
of the errors as following

error
( , )
i i
x y
( )
2
( )
i i
i
E y f x =

The squared errors function

will be
Our target is to find the
conditions under which this
function is minimum. In order
to do so we will follow the
method known by The Least
Squares Method.
Linear Least Squares
Given (x
0
,y
0
), (x
1
,y
1
), (x
n
,y
n
), such that the we know that the best fit is the
linear equation y = a
0
+a
1
x. This best fit function will be calculated by minimizing
the squares of the errors as following
( )
( )
2
0 1 0 1
2
0 1
( , ) ( )
i i
i
i i
i
E a a y a a x
y a a x
= +
=

The squared errors function

will be
x
y
error
0 1
y a a x = +
To get the values of a
0
and a
1

at which the error function is
minimum we must have that,
0 1
0 0
E E
a a
c c
= =
c c
Then
0 1
0
0 1
1
2 ( ) 0
2 ( ) 0
i i
i
i i i
i
E
y a a x
a
E
x y a a x
a
c
= =
c
c
= =
c

Hence
0 1
2
0 1
0
0
i i
i i i
i i i i
i i i
y a a x
x y a x a x
=
=


Then to find a
0
and a
1
we need to solve the equations
0 1
2
0 1
i i
i i
i i i i
i i i
n a x a y
x a x a x y
+ =
+ =


Eq(1)
Eq(2)
Multiplay Eq(1) by and Eq(2) by and subtract, then we get
2
i
i
x
i
i
x

2
2 2
0 i i i i i i i
i i i i i i
n x x a x y x x y
(
| |
=
(
|
\ .
(


Thus
2
0 2
2
i i i i i
i i i i
i i
i i
x y x x y
a
n x x

=
| |

|
\ .


Similarly Multiplay Eq(1) by and Eq(2) by and subtract, then we get
i
i
x

n
1 2
2
i i i i
i i i
i i
i i
n x y x y
a
n x x

=
| |

|
\ .


And thus we can use the equation
to represent the linear relation between the
data points
0 1
y a a x = +
Example 1
Given the data in the following table calculate the curve of best fit linear
module to that data.
Solution
x
y
0.5 0.8 1.5 2.5 4
1.1 2.4 5.3
7.6 8.9
4.5
9.3
The linear best fit will be such that
0 1
y a a x = +
2
0 2
2
i i i i i
i i i i
i i
i i
x y x x y
a
n x x

=
| |

|
\ .


1 2
2
i i i i
i i i
i i
i i
n x y x y
a
n x x

=
| |

|
\ .


Now we have to calculate the following table
x y
2
x
xy
0.5
0.8
1.5
2.5
4
4.54
1.1
2.4
5.3
7.6
8.9
9.3
The Sum =
13.8 34.6
0.25
0.64
2.25
6.25
16
20.25
45.64
0.55
0.512
7.95
15.625
35.6
41.85
102.087
Then we have
0
2
(45.64)(34.6) (13.8)(102.087)
2.0425
(6)(45.64) (13.8)
a

= =

1
2
(6)(102.087) (13.8)(34.6)
1.6192
(6)(45.64) (13.8)
a

= =

Thus the best linear fit for these data points is

(2.0425) (1.6192) y x = +
Example 2
The torque, T needed to turn the torsion spring of a mousetrap through
an angle , is given in the below table.
u
Angle, in
Radians
Torque, T in
N-m
0.698132 0.188224
0.959931 0.209138
1.134464 0.230052
1.570796 0.250965
1.919862 0.313707
Data points for Angle vs. Torque data
0.1
0.2
0.3
0.4
0.5 1 1.5 2
(radians)
T
o
r
q
u
e

(
N
-
m
)
Find the constants for the model given by and hence use it to
find the torque needed to turn the spring through an angle 2.3 radians
0 1
T a au = +
Solution
The linear best fit will be such that
0 1
T a au = +
2
0 2
2
i i i i i
i i i i
i i
i i
T T
a
n
u u u
u u

=
| |

|
\ .


1 2
2
i i i i
i i i
i i
i i
n T T
a
n
u u
u u

=
| |

|
\ .


u T
2
u
T u
0.698132
0.959931
1.134464
1.570796
1.919862
0.188224
0.209138
0.230052
0.250965
0.313707
The Sum =
6.2831 1.1921
0.487388
0.921468
1.287
2.4674
3.6859
8.8491
0.131405
0.200758
0.260986
0.394215
0.602274
1.5896
Now we have to calculate the
following table
Then we have
0
2
(8.8491)(1.1921) (6.2831)(1.5896)
0.11767
(5)(8.8491) (6.2831)
a

= =

1
2
(5)(1.5896) (6.2831)(1.1921)
0.096091
(5)(8.8491) (6.2831)
a

= =

Thus the best linear fit for these data points is

(0.11767) (0.096091) T u = +
Now at we have
(0.11767) (0.096091)(2.3) 0.3386793 T = + =
2.3 u =
Non linear Least Squares
For this model we can continue by taking Ln both sides and then deal with the
problem as a linear model
(1) The Exponential model on the form
bx
y ae =
Nothing that
bx
bx
Ln y Lnae
Ln y Lna Lne
Ln y Lna bx
=
= +
= +
So we can take
0 1
Y Ln y X x
a Lna a b
= =
= =
0 1
Y a a X = +
Example 3
Many patients get concerned when a test involves injection of a radioactive
material. For example for scanning a gallbladder, a few drops of Technetium-
99m isotope is used. Half of the Technetium-99m would be gone in about 6
hours. It, however, takes about 24 hours for the radiation levels to reach what
we are exposed to in day-to-day activities. Below is given the relative intensity
of radiation as a function of time.
t (hrs) 0 1 3 5 7 9
1.000 0.891 0.708 0.562 0.447 0.355

The relative radiation intensity is related to time by the equation

bt
ae =
Find the values of the constants in the previous relation and hence calculate
the radiation intensity after 24 hours
Solution
We have
bt
bt
Ln Lnae
Ln Lna Lne
Ln Lna bt

=
= +
= +
So we can take
0 1
y Ln x t
a Lna a b
= =
= =
0 1
y a a x = +
Such that
2
0 2
2
i i i i i
i i i i
i i
i i
x y x x y
a
n x x

=
| |

|
\ .


1 2
2
i i i i
i i i
i i
i i
n x y x y
a
n x x

=
| |

|
\ .


Now we have to calculate the following table
The Sum =
Then we have
0
2
(165)( 2.876) (25)( 18.98)
0.00025
(6)(165) (25)
a

= =

Thus the constants of the exponential

regression will be
0
1
0.99975
0.1151
a
a e
b a
= =
= =
x t = y Ln =
2
x
xy
0
1
3
5
7
9
25
0.891
0.708
0.562
0.447
0.355
1
___
0
0.345
0.115
0.576
0.805
0.035
2.876
0
1
9
25
49
81
165
0
0.115
1.035
2.88
5.635
9.315
18.98
1
2
(6)( 18.98) (25)( 2.876)
0.115
(6)(165) (25)
a

= =

Now the best exponential fit to the given data points is

0.1151
0.99975
t
e

=
0.1151(24)
2
0.99975
6.3124 10
e

=
=
Thus the relative intensity of radiation after 24 hours will be
For this model we can continue by taking Ln both sides and then deal with the
problem as a linear model
(2) The Power model on the form
b
y ax =
Nothing that
b
b
Ln y Lnax
Ln y Lna Lnx
Ln y Lna bLn x
=
= +
= +
So we can take
0 1
Y Ln y X Lnx
a Lna a b
= =
= =
0 1
Y a a X = +
(3) The Quadratic model on the form
2
0 1 2
y a a x a x = + +
( )
( )
2
2
0 1 2 0 1 2
2
2
0 1 2
( , , ) ( )
i i i
i
i i i
i
E a a a y a a x a x
y a a x a x
= + +
=

The squared errors function will be

To get the values of a
0
, a
1
and a
2
at which the error function is
minimum we must have that,
0 1 2
0 0 0
E E E
a a a
c c c
= = =
c c c
Then
2
0 1 2
0
2
0 1 2
1
2 2
0 1 2
2
2 ( ) 0
2 ( ) 0
2 ( ) 0
i i i
i
i i i i
i
i i i i
i
E
y a a x a x
a
E
x y a a x a x
a
E
x y a a x a x
a
c
= =
c
c
= =
c
c
= =
c

Then to find a
0
, a
1
and a
2
we need to solve the equations
2
0 1 2
2 3
0 1 2
2 3 4 2
0 1 2
i i i
i i i
i i i i i
i i i i
i i i i i
i i i i
n a x a x a y
x a x a x a x y
x a x a x a x y
+ + =
+ + =
+ + =



2
0 1 2
y a a x a x = + +
For more details about this lecture you may use the following:

Our text book, chapter 20
Numerical Analysis by: Burden and Fares, Brooks/Cole Publications
Numerical Recipes in C; The Art of Scientific Computing by: William H. Press,
Brian P. Flannery, Saul A. Teukolsky and William T. Vetterling, Cambridge
University Press Publications
http://en.wikipedia.org/wiki/Linear_regression
http://en.wikipedia.org/wiki/Regression_analysis
http://www.law.uchicago.edu/files/files/20.Sykes_.Regression.pdf
http://www.statsoft.com/textbook/multiple-regression/
http://www.codecogs.com/code/maths/approximation/regression/linear.php
http://answers.google.com/answers/threadview/id/605807.html