9/11/2012

WEEK 1

Introduction to Numerical Methods

Mathematical modeling
Approximation and round off errors
Truncation errors and Taylor Series

LESSON OUTCOMES
At the end of this topic, the students will be able:

To

describe

numerical

techniques

as

compared to analytical methods

To use Taylor series expansion to approximate

a function

To perform error analysis associated with

numerical methods

9/11/2012

Why Numerical Method?

Could handle large systems of equations,
nonlinearity and complex geometries that is not
common
It provide approximate solutions to many of the
engineering problems.

What is Numerical Method

Powerful analysis tool in problem solving and
understanding problem in mathematical language
Techniques by which mathematical problems are
formulated, so that they can be solved with
arithmetic operations
The role of numerical method in solving engineering
problem:
PROBLEM
FORMULATION
Fundamental laws are used to
develop mathematical
equations that can represent
the specific problem

SOLUTION
Suitable numerical methods
are then selected to solve
the mathematical equations

INTERPRETATION
The results obtained can
then be used to
predict/analyze/understand
the specific problem better
5

Mathematical modeling
Is the use of mathematics to

Describe real world phenomena

Investigate important questions about the observed world
Explain real world phenomena
Test ideas
Make predictions about the real world

The real world refers to

Engineering
Physiology
Wildlife management
Economics
Etc

Physics
Ecology
Chemistry
Sports

9/11/2012

A mathematical model is represented as a functional

relationship of the form

Dependent variable
Observed behaviour/state/phenomenon of a system
Characteristic that reflects behaviour or state of the system
i.e. y, f(x), f(t)

Independent variable
Dimension that determine a system i.e. time, t , x

Parameter
Quantity that serves to relate to functions and variables
Reflective of the systems properties or composition

Forcing functions
External influence that acts on system i.e. acceleration gravity, g
8

Example

Apply Newtons second law,

And also can write as

F = ma

dv
m F
dt
or
m

d 2x
F
dt 2

Eq. relates a linear position x

(dependent variable) to the applied
force, F (forcing function) and the
time, t (independent variable). The
mass, m is the only parameter in
the above model.
9

9/11/2012

Example of mathematical modeling

10

Example
Assume that interested to predict the velocity of the falling parachutist with time

Use fundamental knowledge to find a

mathematical equation correlates the velocity
to the various forces acting on the parachutist
Newtons 2nd law of Motion
the time rate change of momentum of a

body is equal to the resulting force acting on

it.
The model is formulated as
F = ma
F=net force acting on the body (N)
m=mass of the object (kg)
a=its acceleration (m/s2)
11

acceleration a

dv
dt

dv
dt

F FD FU

FD downwardforceof gravity

FU upward forceof air resistance

FD mg
FU cv
Model relates acceleration of falling
object to the forces acting on it,

dv
dt

mg cv

(differential equation)

12

9/11/2012

Exact or analytical solution: it exactly satisfies the original equation

dependent variable

forcing functions

t,s

v,(m/s)

0.00

16.40

27.77

35.64

41.10

10

44.87

12

47.87

53.39

parameter

independent variable

Analytical solution of the parachutist

13

Unfortunately, there are many mathematical

models that cannot be solved exactly.

Numerical solution that approximates the exact solution

14

dv

mg cv

dt
dv

dt
v( t

i 1 )

v( t

v( t )
i

ti 1 ti

m
i 1 )

v( t )
i

ti 1 ti
g

c
m

v(t )
i

v(ti1 ) v(ti ) g v(ti ) (ti 1 ti )

m

The use of finite difference to approximate the

first derivative of v with respect to t

Approximate or
numerical solution
15

9/11/2012

Numerical solution

t,s

v,(m/s)

0.00

19.60

32.00

39.85

44.82

10

47.97

12

49.96

53.39

v(ti1 ) v(ti ) g v(ti ) (ti 1 ti )

m

Comparison between the exact and

numerical solution

16

Approximation and Roundoff Errors

Significant figures
Numbers to be used in confidence

98

2 significant figures

98.09

4 significant figures

0.0098

2 significant figures
17

Important of significance figures in numerical methods:

Numerical methods yield approximate results, therefore,
need to develop criteria to specify the confident in
approximate result.
Although quantities such as , e, or 7 represent specific
quantities, they cannot be expressed exactly by a limited
number of digits. Computers retain only a finite number of
significant figures.
= 3.141592653589793238462643

18

9/11/2012

Accuracy and Precision

Increasing precision

Increasing accuracy

How closely individual

computed or measured
values agree with each
other

How closely a computed

or measured value agree
with the true value

Inaccurate & imprecise

accurate & imprecise

Inaccurate & precise

accurate & precise

19

Error definitions
Error

Truncation errors ~ result

when approximations are
used to represent exact
mathematical procedures
Round-off errors ~ result
when numbers having a
limited significant figures
are used to represent
exact numbers

True value

Approximation
value
True value = Error + Approximation value

20

True value = error + approximation value

(1)

True error (Et)= true value approximation

(2)

(3)

t designates true percent relative error

21

9/11/2012

Calculation of errors
True value of length of a bridge is 10,000 cm.
When you measure, the length recorded is
9,999 cm. Compute the true error and true
percentage relative error of the bridge.

22
Answer: 1 cm and 0.01%

However, in actual situation, true value is rarely available.

Therefore, need to estimate the true value approximation
(4)

In numerical method, iterative approach is used to

compute answer, in which error is estimated as the
difference between previous and current approximations.
(5)

The signs of error can be negative or positive,

Absolute value of error, |a| need to be lower than
prespecified percent tolerance, s
n is significant figures.
23

Error estimates for iterative method

Suppose that we have exponential function as,

Starting with the simplest version, ex=1, add

terms to estimate e0.5. Compute true (t) and
approximate error (a) after each term is added
until |a| falls below s , conforming to 3
significant figures. Note that true value of e0.5
is 1.648721.
24

9/11/2012

Answer

25

Round-off errors

Results when numbers having limited significant

figures are used to represent exact numbers

ln 2 = 0.693 147 180 559 945 309 41...

A device only shows 8
significant numbers, so
round-off error is discrepancy
introduced by omission of
significant figures.
Round-off error for this case is
0.00000000055994530941
26

Other example of roundoff error

27

9/11/2012

Truncation errors and Taylor series

Truncation errors
Truncation error is the discrepancy introduced by the fact that
numerical methods may employ approximations to represent exact
mathematical operations and quantities.
Truncation error are errors resulted from using an approximation
in place of an exact mathematical procedure.
The difference between the calculated value using exact
mathematical equation and approximation mathematical equation.

28

Taylor series
Provides a means to predict a function value at one point in terms of the
function value and its derivative at another point

Zero order
First order
Second
order

nth order

29

Taylor series by defining a step size h = xi+1 - xi

(6)

Rn

f ( n 1) ( ) n 1
h
(7)
(n 1)!

30

10

9/11/2012

31

Remainder for the Taylor series Expansion

Remainder term, Rn, accounts for all terms from (n+1) to infinity.
f ( n 1) ( ) n 1
h
(n 1)!

Rn

It also usually expressed as:

Rn O(h n1 )
Let say we truncated the Taylor series expansion after zero-order term to
yield
f ( xi1 ) f ( xi )
Remainder term, Rn for zero order version

R0 f '( xi ) h

f ' '( xi )
2!

h2

f 3( xi ) 3
h ...
3!

Let truncate the remainder itself,

R0 f '( xi ) h
This result is still inexact because neglected second and higher order terms.
32

Alternative simplification that tranforms the approximation into an

equivalence based on graphical insight
derivative mean-value theorem states that if a function f(x) and its
derivative are continous over interval from xi to xi+1,
there exist at least one point on the function that has a slope, designated by
f(), parallel to line joining f (xi) and f(xi+1)
Thus,

So,
Zero order version

First order version

33

11

9/11/2012

Numerical
differentiation
Forward finite divided difference

Backward finite divided difference

Centered finite divided difference

34

Forward finite divided difference approximation of first derivative

(14)

Where,

(15)

Backward finite divided difference approximation of first derivative

f(xi ) f(xi 1 ) R1

h
h
f
f'(xi ) i O(h)
h
f'(xi )

Where,

(16)

h xi xi 1

Centered finite divided difference approximation of first derivative

(17)
Where,

h xi 1 xi xi xi 1

35

Example
Use forward and backward difference approximations of O(h) and a centered
difference approximation of O(h2) to estimate the first derivative of

f(xi ) 0.1x 4 0.15x3 0.5x 2 0.25x 1.2

at x = 0.5 using a step size h = 0.5. Repeat using h = 0.25. Also calculate the
true percent relative error for each approximation.

36

12

9/11/2012

Exercise
Use forward and backward and a centered difference to estimate the first
derivative of the function

f(xi ) 0.5x3 0.1x 2 0.8x 0.7

at x = 0.5 using a step size h=0.5. Repeat using h = 0.25. Also calculate the
true percent relative error for each approximation.
Ans:
h=0.5
FDM:1.525
BDM:0.875
CDM:1.200
h=0.25
FDM:1.26875
BDM:0.94375
CDM:1.10625

41.9%
18.60%
11.63%
18.02%
12.21%
2.91%
37

Higher derivatives

Second forward finite difference approximation of higher derivatives

Second backward finite difference approximation of higher derivatives

Second centred finite difference approximation of higher derivatives

38

Error propagation
This section is to study how errors in numbers can
propagate through mathematical functions. If we multiply
two numbers that have errors, we would like to estimate
the error in the product.

If a function f is dependent on
(a) a single independent variable x : f(x)
(b) two independent variables x and y : f(x, y)
(c) several independent variables x1, x2, x3, ... ,xn : f(x1 , x2 ,..., xn ).

39

13

9/11/2012

Function of a single variable.

Let

x be the true value and

x* be an approximate value of x

Then, TSE for f(x) computed near f(x*) is given by

f(x) f(x*) f'(x * )( x x*)

f ' ' ( x*)

( x x*)2 ...
2

Truncating after the first derivative term and rearranging the remaining terms
to give

f(x) f(x*) f ' (x*)( x x*)

f(x*) f ' (x*) x *

(21)

where

f ' (x*) f ( x) f ( x*) is an estimate of the error of the function

x* x x * is an estimate of the error of independent variable x
Eq.(21) provides 2 capabilities:
1. to approximate the error in f(x) knowing its derivative.
2. to approximate the error in the independent variable x.
40

Example
Given a value of x* = 2.5 with an error of x* = 0.01, estimate the
resulting error in the function, f(x)=x3.
Solution

f(x*) f ' (x*) x *

So,
f(x*) 3(2.5) 2 (0.01) 0.1875
Because f (2.5) 15.625, it can predict that
f( 2.5 ) 15.625 0.1875
Or the true value lies between 15.4375 and 15.8125. In fact, if x ~2.49,
f(x) could be 15.4382 and if x ~ 2.51, it would be 15.8132.
The first order error analysis provides a fairly close estimate of the true error.
41

Exercise
Knowing a value of x* = 2.0 with an error of x* = 0.01, estimate the resulting
error in the function

f(x) = 0.5x3-0.1x2+0.8x-0.7

Ans: f(2.0)=4.5 0.064

42

14

9/11/2012

Function of a more than One variable.

(22)

(23)

Refer section 4.2.2

for examples

43

Relative error
Refer section 4.2.3
for example

Condition number

=
=
44

Condition no equals 1 indicates that functions

relative error is identical to the relative error in
x
Condition no greater than 1 indicates relative
error is amplified.
Condition no less than 1 indicates relative
error is attenuated.
Function with very large values are said to be
ill-conditioned.
45

15

9/11/2012

Total Numerical Error

Total numerical errors = truncation error + round off error
Roundoff error by increase no. of significant figures or
reduce no. of computation in analysis
Truncation error by decreasing step size (h) or increase
no. of computation in analysis

46

Control numerical error

avoid subtract 2 nearly equal numbers to
avoid loss of significance
Use Taylor series for truncation and roundoff
error analysis
Perform numerical experiments
- repeat computation with different step size or method
and compare results

47

16