CISE301_Topic1 1

CISE-301: Numerical Methods

Topic 1:
Introduction to Numerical Methods and Taylor Series
Lectures 1-4:
CISE301_Topic1 2
Lecture 1
Introduction to Numerical Methods

What are NUMERICAL METHODS?

Why do we need them?

Topics covered in CISE301.

Reading Assignment: Paes 3!10 o" te#t$oo%
CISE301_Topic1 3
Numerical Methods
Numerical Methods:
&'orithms that are (sed to o$tain n(merica'
so'(tions o" a mathematica' pro$'em.
h! do "e need them#
1. )o ana'ytica' so'(tion e#ists*
2. &n ana'ytica' so'(tion is di""ic('t to o$tain
or not practica'.
CISE301_Topic1 +
What do we need?
$asic Needs in the Numerical Methods:

Practica',
Can $e comp(ted in a reasona$'e amo(nt o" time.

&cc(rate,

-ood appro#imate to the tr(e va'(e*

In"ormation a$o(t the appro#imation error

./o(nds* error order*0 1.
CISE301_Topic1 2
Outlines of the Course

Tay'or Theorem

)(m$er
3epresentation

So'(tion o" non'inear

E4(ations

Interpo'ation

)(merica'
5i""erentiation

)(merica' Interation

So'(tion o" 'inear

E4(ations

6east S4(ares c(rve

"ittin

So'(tion o" ordinary

di""erentia' e4(ations

So'(tion o" Partia'

di""erentia' e4(ations
CISE301_Topic1 7
Solution of Nonlinear Equations

Some simp'e e4(ations can $e so'ved ana'ytica''y,

8any other e4(ations have no ana'ytica' so'(tion,

1
! 1 " #
! !" 1 " $ $ $
solution %nal&tic
' $
#
#

t

+ +
x and x
roots
x x
solution anal&tic No
' ( #
# )

+
x
e x
x x
CISE301_Topic1 9
Methods for Sol*in+ Nonlinear
Equations
o
$isection Method
o
Ne"ton%Ra&hson Method
o
Secant Method
CISE301_Topic1 :
Solution of S&stems of Linear
Equations
un,nowns- 1''' in equations 1'''
ha*e we if do What to
1 # . #
( # .
: as it sol*e can We
( #

1 #
# # # 1
# 1
# 1

+
+
+
x x
x x x x
x x
x x
CISE301_Topic1 ;
Cramer/s 0ule is Not 1ractical
this- compute to &ears 1' than more needs computer super %
needed- are tions multiplica 1' #- s&stem. ' 2& ' a sol*e To
tions- multiplica
1!N3 1!"N "N need we un,nowns. N with equations N sol*e To
pro2lems- lar+e for practical not is 0ule s Cramer4 5ut
#
# 1
1 1
( 1
1
. 1
# 1
1 1
# (
1
: s&stem the sol*e to used 2e can 0ule s Cramer4
#'
(
# 1

+
x x
CISE301_Topic1 10
Methods for Sol*in+ S&stems of Linear
Equations
o
Nai'e (aussian Elimination
o
(aussian Elimination "ith Scaled
)artial )i'oting
o
Algorithm *or Tri%diagonal E+uations
CISE301_Topic1 11
Cur*e 6ittin+

-iven a set o" data,

Se'ect a c(rve that $est "its the data. <ne

choice is to "ind the c(rve so that the s(m
o" the s4(are o" the error is minimi=ed.
7 ' 1 #
& '-( 1'- #1-


CISE301_Topic1 12
8nterpolation

-iven a set o" data,

>ind a po'ynomia' P(x) whose raph

passes thro(h a'' ta$('ated points.
7
i
' 1 #
&
i
'-( 1'- 1(-


ta2le in the is ! "
i i i
x if x P y
CISE301_Topic1 13
Methods for Cur*e 6ittin+
o
Least S+uares
o
Linear Regression
o
Nonlinear Least S+uares )ro,lems
o
Inter&olation
o
Ne"ton )ol!nomial Inter&olation
o
Lagrange Inter&olation
CISE301_Topic1 1+
8nte+ration

Some "(nctions can $e interated

ana'ytica''y,
?
: solutions anal&tical no ha*e functions man& 5ut
$
#
1
#
)
#
1
'

1
#

1
#

dx e
x xdx
a
x
CISE301_Topic1 12
Methods for Numerical 8nte+ration
o
U&&er and Lo"er Sums
o
Tra&e-oid Method
o
Rom,erg Method
o
(auss .uadrature
CISE301_Topic1 17
Solution of Ordinar& 9ifferential Equations
onl&- cases special
for a*aila2le are solutions %nal&tical :
equations- the satisfies that function a is
' ! ' " ; 1 ! ' "
' ! " ! " ! "
: equation al differenti the o solution t %
x(t)
x x
t x t x t x

+ +

CISE301_Topic1 19
Solution of 1artial 9ifferential Equations
Partia' 5i""erentia' E4(ations are more
di""ic('t to so've than ordinary di""erentia'
e4(ations,
! sin" ! ' . " . ' ! . 1 " ! . ' "
' #
#
#
#
#
x x u t u t u
t
u
x
u

+

CISE301_Topic1 1:
Summar&

Numerical Methods:
&'orithms that are
(sed to o$tain
n(merica' so'(tion o" a
mathematica' pro$'em.

e need them "hen

)o ana'ytica' so'(tion
e#ists or it is di""ic('t
to o$tain it.

So'(tion o" )on'inear E4(ations

So'(tion o" 6inear E4(ations

C(rve >ittin

6east S4(ares

Interpo'ation

)(merica' Interation

)(merica' 5i""erentiation

So'(tion o" <rdinary 5i""erentia'

E4(ations

So'(tion o" Partia' 5i""erentia'

E4(ations
Topics Covered in the Course
CISE301_Topic1 1;

)(m$er 3epresentation

)orma'i=ed >'oatin Point 3epresentation

Sini"icant 5iits

&cc(racy and Precision

3o(ndin and Choppin

Reading Assignment: Chapter 3
Lecture 2
Num2er 0epresentation and
%ccurac&
CISE301_Topic1 20
0epresentin+ 0eal Num2ers

?o( are "ami'iar with the decima' system,

5ecima' System, /ase @ 10 * 5iits .0*1*0*;1

Standard 3epresentations,
# 1 ' 1 #
1' ( 1' $ 1' # 1' 1 1' $( - 1#

+ + + +
part part
fraction inte+ral si+n
( $ - # 1 t
CISE301_Topic1 21
Normali<ed 6loatin+ 1oint 0epresentation

)orma'i=ed >'oatin Point 3epresentation,

Scienti"ic )otation, E#act'y one non!=ero diit appears

$e"ore decima' point.

&dvantae, E""icient in representin very sma'' or very

'are n(m$ers.
e7ponent si+ned : . '
e7ponent mantissa si+n
1'
$ # 1
-
n d
n
f f f f d
t
t
t
CISE301_Topic1 22
5inar& S&stem

/inary System, /ase @ 2* 5iits A0*1B

e7ponent si+ned mantissa si+n
# - 1
$ # 1
n
f f f f
t
t
1'
! =#( - 1 "
1'
!

# 1
#
# '
1
# 1 1 "
#
! 1'1 - 1 "

+
CISE301_Topic1 23
6act

)(m$ers that have a "inite e#pansion in one n(m$erin

system may have an in"inite e#pansion in another
n(m$erin system,

?o( can never represent 1.1 e#act'y in $inary system.

# 1'
---! '11'' '''11''11' - 1 " ! 1 - 1 "
8EEE >($ 6loatin+?1oint Standard

Sin'e Precision .32!$it representation1

1!$it Sin C :!$it E#ponent C 23!$it >raction

5o($'e Precision .7+!$it representation1

1!$it Sin C 11!$it E#ponent C 22!$it >raction

CISE301_Topic1 2+
S E#ponent
:
>raction
23
S E#ponent
11
>raction
22
.contin(ed1
CISE301_Topic1 22
Si+nificant 9i+its

Sini"icant diits are those diits that can $e

(sed with con"idence.

Sin'e!Precision, 9 Sini"icant 5iits

1.192+;+0 D 10
!3:
to 3.+02:230 D 10
3:

5o($'e!Precision, 12 Sini"icant 5iits

2.222093:0 D 10
!30:
to 1.9;97;310 D 10
30:
CISE301_Topic1 27
0emar,s

)(m$ers that can $e e#act'y represented are ca''ed

machine n(m$ers.

5i""erence $etween machine n(m$ers is not (ni"orm

S(m o" machine n(m$ers is not necessari'y a machine

n(m$er
CISE301_Topic1 29
Calculator E7ample

S(ppose yo( want to comp(te,

3.29: E 2.13;
(sin a ca'c('ator with two!diit "ractions
3.29
E
2.13 9.70
@
9.7233+2
True ans"er:
CISE301_Topic1 2:
$@-)
Si+nificant 9i+its ? E7ample
CISE301_Topic1 2;
%ccurac& and 1recision

&cc(racy is re'ated to the c'oseness to the tr(e

va'(e.

Precision is re'ated to the c'oseness to other

estimated va'(es.
CISE301_Topic1 30
CISE301_Topic1 31
0oundin+ and Choppin+

3o(ndin, 3ep'ace the n(m$er $y the nearest

machine n(m$er.

Choppin, Throw a'' e#tra diits.

CISE301_Topic1 32
0oundin+ and Choppin+
CISE301_Topic1 33
Can $e comp(ted i" the tr(e va'(e is %nown,

1'' :
*alue true
ion appro7imat *alue true
Error 0elati*e 1ercent %2solute
ion appro7imat *alue true
Error True %2solute
t

t
E
Error 9efinitions A True Error
CISE301_Topic1 3+
When the tr(e va'(e is not %nown,

1'' :
estimate current
estimate pre*ious estimate current
Error 0elati*e 1ercent %2solute Estimated
estimate pre*ious estimate current
Error %2solute Estimated

a
a
E

Error 9efinitions A Estimated Error

CISE301_Topic1 32
We say that the estimate is correct to n
decima' diits i",
We say that the estimate is correct to n
decima' diits rounded i",
n
1' Error
n
1'
#
1
Error
Notation
CISE301_Topic1 37
Summar&

)(m$er 3epresentation
)(m$ers that have a "inite e#pansion in one n(m$erin system
may have an in"inite e#pansion in another n(m$erin system.

)orma'i=ed >'oatin Point 3epresentation

E""icient in representin very sma'' or very 'are n(m$ers*

5i""erence $etween machine n(m$ers is not (ni"orm*

3epresentation error depends on the n(m$er o" $its (sed in

the mantissa.
CISE301_Topic1 39
Lectures 3-4
Taylor Theorem

8otivation

Tay'or Theorem

E#amp'es
Reading assignment: Chapter +
CISE301_Topic1 3:
Moti*ation

We can easi'y comp(te e#pressions 'i%e,

? ! = - ' sin" . $-1 compute &ou do Bow 5ut.
! $ " #
1'
#
+

x
wa&? practical a this 8s
sin"'-=!? compute to
definition the use we Can
0.6
a
b
CISE301_Topic1 3;
0emar,

In this co(rse* a'' an'es are ass(med to

$e in radian (n'ess yo( are to'd otherwise.

CISE301_Topic1 +0
Ta&lor Series
C
D
'
! "
'
! "

! "
#
! # "
4
! " ! "
3
1
! "
: write can we con*er+e. series the 8f
! " ! "
3
1

--- ! "
3
! "
! "
3 #
! "
! " ! " ! "
: a2out ! " of e7pansion series Ta&lor The

+ + + +

k
k k
k
k k
a x a f
k
x f
a x a f
k
Series Taylor
or
a x
a f
a x
a f
a x a f a f
a x f
CISE301_Topic1 +1
Maclaurin Series

8ac'a(rin series is a specia' case o" Tay'or

series with the center o" e#pansion a @ 0.
C
D
'
! "

! "
#
! # "
4
! ' "
3
1
! "
: write can we con*er+e. series the 8f
---
3
! ' "
3 #
! ' "
! ' " ! ' "
: ! " of e7pansion series n Maclauri The

+ + + +
k
k k
x f
k
x f
x
f
x
f
x f f
x f
CISE301_Topic1 +2
Maclaurin Series A E7ample 1
D- 7 for con*er+es series The
---
3 3 #
1
3
! ' "
3
1

1 1 ! ' " ! "
1 ! ' " ! "
1 ! ' " 4 ! " 4
1 ! ' " ! "
#
D
'
D
'
! "
! " ! "
! # " ! # "
C C
<
+ + + +





x x
x
k
x
x f
k
e
k for f e x f
f e x f
f e x f
f e x f
k
k
k
k k x
k x k
x
x
x
x
e x f ! " of e7pansion series n Maclauri O2tain
CISE301_Topic1 +3
Ta&lor Series
E7ample 1
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
0
0.5
1
1.5
2
2.5
3
1
1+x
1+x+0.5x
2
exp(x)
CISE301_Topic1 ++
Maclaurin Series A E7ample #
D- 7 for con*er+es series The
----
3 > 3 ( 3 3
! ' "
! sin"
1 ! ' " ! cos" ! "
' ! ' " ! sin" ! "
1 ! ' " 4 ! cos" ! " 4
' ! ' " ! sin" ! "
> (
D
'
! "
! " ! "
! # " ! # "
C
<
+ +

x x x
x x
k
f
x
f x x f
f x x f
f x x f
f x x f
k
k
k
: ! sin" ! " of e7pansion series n Maclauri O2tain x x f
CISE301_Topic1 +2
-4 -3 -2 -1 0 1 2 3 4
-4
-3
-2
-1
0
1
2
3
4
x
x-x
3
/3!
x-x
3
/3!+x
5
/5!
sin(x)
CISE301_Topic1 +7
Maclaurin Series A E7ample
D- for con*er+es series The
----
3 = 3 $ 3 #
1 ! "
3
! ' "
! cos"
' ! ' " ! sin" ! "
1 ! ' " ! cos" ! "
' ! ' " 4 ! sin" ! " 4
1 ! ' " ! cos" ! "
= $ #
D
'
! "
! " ! "
! # " ! # "
C
<
+ +

x
x x x
x
k
f
x
f x x f
f x x f
f x x f
f x x f
k
k
k
! cos" ! " : of e7pansion series Maclaurin O2tain x x f
CISE301_Topic1 +9
Maclaurin Series A E7ample $
( )
( )
( )
1 E E for con*er+es Series
--- 7 7 7 1
7 1
1
: of E7pansion Series Maclaurin
= ! ' "
1
=
! "
# ! ' "
1
#
! "
1 ! ' " 4
1
1
! " 4
1 ! ' "
1
1
! "
of e7pansion series n Maclauri O2tain
#
! "
$
! "
! # "

! # "
#
<
+ + + +

x
f
x
x f
f
x
x f
f
x
x f
f
x
x f
x 1
1
f(x)
CISE301_Topic1 +:
E7ample $ ? 0emar,s

Can we app'y the series "or #F1??

Gow many terms are needed to et a ood

appro#imation???
These 4(estions wi'' $e answered (sin
Tay'orHs Theorem.
CISE301_Topic1 +;
Ta&lor Series A E7ample (
--- ! 1 " ! 1 " ! 1 " 1 : ! 1 " E7pansion Series Ta&lor
= ! 1 "
=
! "
# ! 1 "
#
! "
1 ! 1 " 4
1
! " 4
1 ! 1 "
1
! "
1 at of e7pansion series Ta&lor O2tain
#
! "
$
! "
! # "

! # "
#
+ +

x x x a
f
x
x f
f
x
x f
f
x
x f
f
x
x f
a
x
1
f(x)
CISE301_Topic1 20
Ta&lor Series A E7ample =
--- ! 1 "

1
! 1 "
#
1
! 1 " : E7pansion Series Ta&lor
# ! 1 " 1 ! 1 " . 1 ! 1 " 4 . ' ! 1 "
#
! " .
1
! " .
1
! " 4 . ! ln" ! "
! 1 " at ! ln" of e7pansion series Ta&lor O2tain
#
! " ! # "

! "
#
! # "
+

x x x
f f f f
x
x f
x
x f
x
x f x x f
a x f(x)
CISE301_Topic1 21
Con*er+ence of Ta&lor Series

The Tay'or series converes "ast ."ew terms

are needed1 when x is near the point o"
e#pansion. I" |x-a| is 'are then more terms
are needed to et a ood appro#imation.
CISE301_Topic1 22
Ta&lor/s Theorem
- and 2etween is ! "
!3 1 "
! "
: where
! "
3
! "
! "
: 2& +i*en is ! " of *alue the then and containin+ inter*al an on
1! " ---. #. 1. orders of s deri*ati*e possesses ! " function a 8f
1
! 1 "
'
! "
C
x a and a x
n
f
R
R a x
k
a f
x f
x f x a
n x f
n
n
n
n
n
k
k
k

+
+

+
+
(n+1) terms Truncated
Taylor Series
Remainder
CISE301_Topic1 23
Ta&lor/s Theorem
- applica2le not is Theorem Ta&lor
defined- not are s deri*ati*e
its and function the then . 1 8f
- 1 E E if ' e7pansion of point the with
1
1
: for theorem s Ta&lor4 appl& can We

<

x
x a
x
f(x)
CISE301_Topic1 2+
Error Term
- and 2etween all for
! "
!3 1 "
! "
: on 2ound upper an deri*e can we
error. ion appro7imat a2out the idea an +et To
1
! 1 "
x a of values
a x
n
f
R
n
n
n

+
+

CISE301_Topic1 22
Error Term ? E7ample
( ) '( 1$#=@ - @ # - '
!3 1 "
! "
!3 1 "
! "
1 F G ! " ! "

1
# - '
1
! 1 "
# - ' ! " ! "

+

+
+
+
E R
n
e
R
a x
n
f
R
n for e f e x f
n
n
n
n
n
n x n

? # - ' ' at
e7pansion series Ta&lor its of ! " terms $ first the
2& ! " replaced we if error the is lar+e Bow

x when a
n
e x f
x
CISE301_Topic1 27
%lternati*e form of Ta&lor/s Theorem
h x x where h
n
f
R
h R h
k
x f
h x f
h x x
n x f Let
n
n
n
n
n
k
k
k
+
+

+ +
+
+
+
+

and 2etween is
!3 1 "
! "
si<e! step "
3
! "
! "
: then and containin+ inter*al an on
1! " ---. #. 1. orders of s deri*ati*e ha*e ! "
1
! 1 "
'
! "

CISE301_Topic1 29
Ta&lor/s Theorem A %lternati*e
forms
- and 2etween is
!3 1 "
! "
3
! "
! "
.
- and 2etween is
! "
!3 1 "
! "
! "
3
! "
! "
1
! 1 "
'
! "
1
! 1 "
'
! "
h x x where
h
n
f
h
k
x f
h x f
h x x x a
x a where
a x
n
f
a x
k
a f
x f
n
n n
k
k
k
n
n n
k
k
k
+
+
+ +
+

+
+
+
+

+
+

CISE301_Topic1 2:
Mean Halue Theorem
! " ! " 4
. . ' for Theorem s Ta&lor4 Ise : 1roof
! " 4
! . " e7ists there then
! . " inter*al open the on defined is deri*ati*e its and
J . K inter*al closed a on function continuous a is ! " 8f
a b f f(a) f(b)
b h x a x n
a b
f(a) f(b)
f
b a
b a
b a x f
+
+

CISE301_Topic1 2;
%lternatin+ Series Theorem
term omitted 6irst :
n terms! first the of "sum sum 1artial :
con*er+es series The
then
' lim
8f
S
: series + alternatin he Consider t
1
1
$ # 1
$ # 1
+
+

'

'

+ +
n
n
n n n
n
a
S
a S S
and
a
and
a a a a
a a a a

CISE301_Topic1 70
%lternatin+ Series A E7ample
3 >
1
3 (
1
3
1
1 ! 1 " s
3 (
1
3
1
1 ! 1 " s
: Then
' lim
: since series + alternatin con*er+ent a is This
3 >
1
3 (
1
3
1
1 ! 1 " s : usin+ computed 2e can sin"1!
$ # 1

,
_

,
_

+ +

in
in
a and a a a a
in
n
n

CISE301_Topic1 71
E7ample >
? 1 with e appro7imat to
used are terms 1! " when 2e error the can lar+e Bow
e7pansion! of center "the ( - ' at ! " of
e7pansion series Ta&lor the O2tain
1 #
1 #

+

+
+
x e
n
a e x f
x
x
CISE301_Topic1 72
E7ample > A Ta&lor Series
---
3
! ( - ' "
# ---
3 #
! ( - ' "
$ ! ( - ' " #
! ( - ' "
3
! ( - ' "
# ! ( - ' " # ! "
$ ! ( - ' " $ ! "
# ! ( - ' " 4 # ! " 4
! ( - ' " ! "
#
#
# # #
D
'
! "
1 #
# ! " 1 # ! "
# ! # " 1 # ! # "
# 1 #
# 1 #
C
+

+ +

+ +

+
+
+
+
+
k
x
e
x
e x e e
x
k
f
e
e f e x f
e f e x f
e f e x f
e f e x f
k
k
k
k
k
x
k k x k k
x
x
x
( - ' . ! " of e7pansion series Ta&lor O2tain
1 #

+
a e x f
x
CISE301_Topic1 73
E7ample > A Error Term
!3 1 "
ma7
!3 1 "
! ( - ' "
#
!3 1 "
! ( - ' 1 "
#
! ( - ' "
!3 1 "
! "
# ! "

1 #
J 1 . ( - ' K
1
1
1
1 # 1
1
! 1 "
1 # ! "
+

+
+
+
+ +
+
+
+
n
e
Error
e
n
Error
n
e Error
x
n
f
Error
e x f
n
n
n
n
n
n
x k k