2993 Spring 2005

Introduction to Numerical Analysis for Engineers
Fundamentals of Digital Computing
Digital Computer Models

Convergence, accuracy and stability
Number representation
Arithmetic operations
Recursion algorithms
Error propagation numerical stability

Error estimation
Error cancellation
Condition numbers
Error Analysis
13.002
Numerical Methods for Engineers
Lecture 1

Continuous Model
Differential Equation
Differentiation
Integration
Difference Equation
w(x,t)
x
System of Equations
Discrete Model
Linear System of Equations
Solving linear
equations
xn
m
n
w(x,t)
Eigenvalue Problems
Non-trivial Solutions
Root finding
Accuracy and Stability => Convergence

13.002
Lecture 1
Floating Number Representation
Examples
m
b
e
Mantissa
Base
Exponent
Decimal
Binary
Convention
Decimal
Binary
Max mantissa
Min mantissa
Max exponent
General
13.002
Min exponent
Lecture 1
Arithmetic Operations
Number Representation
Addition and Subtraction
Absolute Error
Shift mantissa of largest number
Relative Error
Result has exponent of largest number

Absolute Error
Relative Error
Unbounded
Multiplication and Division
Relative Error
13.002
Bounded
Lecture 1
Recursion
Numerically evaluate square-root
Initial guess
Test
a=26;
n=10;
MATLAB script
g=1;
sqr.m
sq(1)=g;
for i=2:n
sq(i)= 0.5*(sq(i-1) + a/sq(i-1));
end
hold off
plot([0 n],[sqrt(a) sqrt(a)],'b')
hold on
plot(sq,'r')
plot(a./sq,'r-.')
plot((sq-sqrt(a))/sqrt(a),'g')
grid on
Mean of guess and its reciprocal
Recursion Algorithm
13.002
Lecture 1
Recursion
Horners Scheme
Evaluate polynomial
horner.m
% Horners scheme
% for evaluating polynomials
a=[ 1 2 3 4 5 6 7 8 9 10 ];
n=length(a) -1 ;
z=1;
b=a(1);
% Note index shift for a
for i=1:n
b=a(i+1)+ z*b;
end
p=b
Horners Scheme
General order n
>> horner
p =
55
Recurrence relation
>>
13.002
Lecture 1
Recursion
Order of Operations Matter
0
Result of small, but significant

term destroyed by subsequent
addition and subtraction of almost
equal, large numbers.
Remedy:
Change order of additions
N=20; sum=0; sumr=0;

b=1; c=1; x=0.5;
recur.m
xn=1;
% Number of significant digits in computations
dig=2;
ndiv=10;
for i=1:N
a1=sin(pi/2-pi/(ndiv*i));
a2=-cos(pi/(ndiv*(i+1)));
% Full matlab precision
xn=xn*x;
addr=xn+b*a1;
addr=addr+c*a2;
ar(i)=addr;
sumr=sumr+addr;
z(i)=sumr;
% additions with dig significant digits
add=radd(xn,b*a1,dig);
add=radd(add,c*a2,dig);
% add=radd(b*a1,c*a2,dig);
% add=radd(add,xn,dig);
a(i)=add;
sum=radd(sum,add,dig);
y(i)=sum;
end
sumr
'
i
delta
Sum
delta(approx) Sum(approx)'
res=[[1:1:N]' ar' z' a' y']
hold off
a=plot(y,'b'); set(a,'LineWidth',2);
hold on
a=plot(z,'r'); set(a,'LineWidth',2);
a=plot(abs(z-y)./z,'g'); set(a,'LineWidth',2);
legend([ num2str(dig) ' digits'],'Exact','Error');
13.002
Lecture 1
recur.m
>> recur
b = 1; c = 1; x = 0.5;
dig=2
i
delta
Sum
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
7.0000
8.0000
9.0000
10.0000
11.0000
12.0000
13.0000
14.0000
15.0000
16.0000
17.0000
18.0000
19.0000
20.0000
0.4634
0.2432
0.1226
0.0614
0.0306
0.0153
0.0076
0.0037
0.0018
0.0009
0.0004
0.0002
0.0001
0.0000
0.0000
-0.0000
-0.0000
-0.0000
-0.0000
-0.0000
0.4634
0.7065
0.8291
0.8905
0.9212
0.9364
0.9440
0.9478
0.9496
0.9505
0.9509
0.9511
0.9512
0.9512
0.9512
0.9512
0.9512
0.9512
0.9512
0.9512
delta(approx) Sum(approx)
res =
13.002
0.5000
0.2000
0.1000
0.1000
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.5000
0.7000
0.8000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
0.9000
Lecture 1
Introduction to Numerical Analysis for

Engineers
Fundamentals of Digital Computing

Convergence, accuracy and stability
Number representation
Arithmetic operations
Recursion algorithms
Error propagation numerical stability

Error estimation
Error cancellation
Condition numbers
Error Analysis
13.002
Lecture 2
Floating Number Representation
Examples
m
b
e
Mantissa
Base
Exponent
Decimal
Binary
Convention
Decimal
Binary
Max mantissa
Min mantissa
Max exponent
General
13.002
Min exponent
Lecture 2
Error Analysis
Number Representation
Addition and Subtraction
Absolute Error
Shift mantissa of largest number
Relative Error
Result has exponent of largest number

Absolute Error
Relative Error
Unbounded
Multiplication and Division
Relative Error
13.002
Bounded
Lecture 2
Error Propagation
Spherical Bessel Functions
Solutions
13.002
Lecture 2
Error Propagation
Spherical Bessel Functions
Forward Recurrence
Forward Recurrence
Unstable
Backward Recurrence
Millers algorithm
Stable
N ~ x+20
13.002
Lecture 2
Error Propagation
Eulers Method
Example
Discretization
Finite Difference (forward)
Recurrence
euler.m
Central Finite Difference
13.002
Lecture 2
Error Propagation
Absolute Errors
'y~f (x)'x
Function of one variable
'x = x - x
General Error Propagation Formula

x
13.002
Lecture 2
Error Propagation
Example
Error Propagation Formula
Multiplication
=>
=>
=>
=>
Relative Errors Add for Multiplication
13.002
Lecture 2
Error Propagation
Expectation of Errors
Addition
Standard Error
Truncation
Error Expectation
Rounding
Standard Error better measure

of expected errors
13.002
Lecture 2
Error Propagation
Error Cancellation
Max. error
Error cancellation
Stand. error
13.002
Lecture 2
Error Propagation
Condition Number
y = f(x)
x = x(1 + D)
y = y(1 + E)
Problem Condition Number

Problem ill-conditioned
Error cancellation example
Well-conditioned problem
13.002
Lecture 2
Error Propagation
Condition Number
Problem Condition Number
K A is algorith condition number, which

may be much larger than the K P due
to limited number representation.
Solution
Higher precision
Rewrite algorithm
4 Significant Digits
Well-conditioned Algorithm
Algorithm Condition Number

13.002
Lecture 2

Systems of Linear Equations
Mathews
Cramers Rule
Gaussian Elimination
Numerical implementation
3.3-3.4
Numerical stability
Partial Pivoting
Equilibration
Full Pivoting
Multiple right hand sides

Computation count
LU factorization
Error Analysis for Linear Systems
Condition Number
Special Matrices
Iterative Methods
13.002
3.3-3.5
Jacobis method
Gauss-Seidel iteration
Convergence
3.5
3.4
3.6
Lecture 3

Cramers Rule
Example, n=2
Cramers Rule, n=2
Cramers rule inconvenient for n>3
13.002
Lecture 3

Reduction
Step 0
13.002
Lecture 3

Reduction
Step 1
13.002
Lecture 3

Reduction
Step n-1
Reduction
Step k
Back-Substitution
13.002
Lecture 3

Step k
Partial Pivoting by Columns
Row k
Pivotal Elements
Row i
Required at each step!
13.002
Lecture 3

Reduction
Step k
New Row k
Pivotal Elements
New Row i
Required at each step!
13.002
Lecture 3

Example, n=2
Cramers Rule - Exact
2-digit Arithmetic
100% error
n=3
a = [ [0.01 1.0]' [-1.0 0.01]']
tbt.m
b= [1 1]'
r=a^(-1) * b
x=[0 0];
m21=a(2,1)/a(1,1);
tbt.m
a(2,1)=0;
a(2,2) = radd(a(2,2),-m21*a(1,2),n);
b(2)
= radd(b(2),-m21*b(1),n);
x(2)
= b(2)/a(2,2);
x(1)
= (radd(b(1), -a(1,2)*x(2),n))/a(1,1);
x'
13.002
1% error
Lecture 3

Interchange Rows
Example, n=2
2-digit Arithmetic
1% error
1% error
13.002
Lecture 3

Multiply Equation 1 by 200
Example, n=2
2-digit Arithmetic
100% error
1% error
Equations must be normalized for

partial pivoting to ensure stability
This Equilibration is made by
normalizing the matrix to unit norm
13.002
Infinity-Norm Normalization
Two-Norm Normalization
Lecture 3

Interchange Unknowns
Example, n=2
Pivoting by Rows
2-digit Arithmetic
1% error
Full Pivoting
Find largest numerical value in same row and column and interchange
Affects ordering of unknowns
13.002
Lecture 3

Numerical Stability
Partial Pivoting
Equilibrate system of equations

Pivoting by Columns
Simple book-keeping
Solution vector in original order
Full Pivoting
Does not require equilibration

Pivoting by both row and columns
More complex book-keeping
Solution vector re-ordered
Partial Pivoting is simplest and most common

Neither method guarantees stability
13.002
Lecture 3

Variable Transformation
Example, n=2
2-digit Arithmetic
1% error
100% error
13.002
Lecture 3

How to Ensure Numerical Stability
System of equations must be well conditioned
Investigate condition number
Tricky, because it requires matrix inversion (next class)
Consistent with physics
E.g. dont couple domains that are physically uncoupled
Consistent units
E.g. dont mix meter and Pm in unknowns
Dimensionless unknowns
Normalize all unknowns consistently
Equilibration and Partial Pivoting, or Full Pivoting
13.002
Lecture 3

Cramers Rule
3.3-3.4
Numerical stability
3.3-3.5
Partial Pivoting
Equilibration
Full Pivoting

Computation count
LU factorization
Condition Number
Special Matrices
Iterative Methods
13.002
Mathews
Jacobis method
Convergence
3.5
3.4
3.6
Lecture 4

Multiple Right-hand Sides
Reduction
Step k
Computation Count
Reduction Step k
Operations
k
Total Computation Count
Reduction
Back Substitution
n-k
n
Reduction for each right-hand side inefficient.
However, RHS may be result of iteration and unknown a priori
(e.g. Eulers method) -> LU Factorization
13.002
Lecture 4

LU Factorization
The coefficient Matrix A is decomposed as
where
and
is a lower triangular matrix

is an upper triangular matrix
Then the solution is performed in two simple steps
1.
Forward substitution
2.
Back substitution
How to determine
13.002
and
Lecture 4

LU Factorization
j
Change in reduction steps 1 - i-1:
Total change above diagonal
Total change below diagonal
Define
After reduction step i-1:
Above and on diagonal
=>
Unchanged after step i-1

Below diagonal
Become and remain 0 in step j
13.002
Lecture 4
Matrix product

LU Factorization
Sum stops at diagonal
Upper triangular
Lower triangular
Below diagonal
i
Above diagonal
13.002
k
Lecture 4

LU Factorization
GE Reduction directly yields LU factorization
Compact storage
Lower triangular
Upper triangular
=
Lower diagonal implied
13.002
Lecture 4

Pivoting in LU Factorization
Before reduction, step k
Forward substitution, step k
Interchange rows i and k
Pivoting if
Interchange rows i and k
else
Pivot element vector
13.002
Lecture 4
Linear Systems of Equations

Error Analysis
Linear systems
How is the relative error of x
dependent on errors in b?
Condition number
Example
The condition number K is a

measure of the amplification of the
relative error by the function f(x)
Small changes in b give large changes in x

The system is ill-Conditioned
13.002
Lecture 4

Error Analysis
Vector and Matrix Norm
Properties
Perturbed Right-hand Side
Subtract original equation
Relative Error Magnification
Condition Number
13.002
Lecture 4

Error Analysis
Vector and Matrix Norm
Perturbed Coefficient Matrix
Subtract unperturbed equation
Properties
Relative Error Magnification
Condition Number
13.002
Lecture 4
Ill-Conditioned System
n=4
a = [ [1.0 1.0]' [1.0 1.0001]']
b= [1 2]'
tbt6.m
ai=inv(a);
a_nrm=max( abs(a(1,1)) + abs(a(1,2)) ,
abs(a(2,1)) + abs(a(2,2)) )
ai_nrm=max( abs(ai(1,1)) + abs(ai(1,2)) ,
abs(ai(2,1)) + abs(ai(2,2)) )
k=a_nrm*ai_nrm
r=ai * b
x=[0 0];
m21=a(2,1)/a(1,1);
a(2,1)=0;
a(2,2) = radd(a(2,2),-m21*a(1,2),n);
b(2)
= radd(b(2),-m21*b(1),n);
x(2)
x(1)
x'
= b(2)/a(2,2);
= (radd(b(1), -a(1,2)*x(2),n))/a(1,1);
Ill-conditioned system
13.002
Lecture 4
Well-Conditioned System
4-digit Arithmetic
n=4
a = [ [0.0001 1.0]' [1.0 1.0]']
b= [1 2]'
ai=inv(a);
a_nrm=max( abs(a(1,1)) +
abs(a(2,1)) +
ai_nrm=max( abs(ai(1,1))
abs(ai(2,1))
k=a_nrm*ai_nrm
tbt7.m
abs(a(1,2)) ,
abs(a(2,2)) )
+ abs(ai(1,2)) ,
+ abs(ai(2,2)) )
r=ai * b
x=[0 0];
m21=a(2,1)/a(1,1);
a(2,1)=0;
a(2,2) = radd(a(2,2),-m21*a(1,2),n);
b(2)
= radd(b(2),-m21*b(1),n);
x(2)
x(1)
x'
= b(2)/a(2,2);
= (radd(b(1), -a(1,2)*x(2),n))/a(1,1);
Algorithmically ill-conditioned
Well-conditioned system
13.002
Lecture 4

Cramers Rule
3.3-3.4
Numerical stability
3.3-3.5
Partial Pivoting
Equilibration
Full Pivoting

Computation count
LU factorization
Condition Number
Special Matrices
Iterative Methods
13.002
Mathews
Jacobis method
Convergence
3.5
3.4
3.6
Lecture 5

Tri-diagonal Systems
Forced Vibration of a String
f(x,t)
xi
y(x,t)
Finite Difference
Discrete Difference Equations
Harmonic excitation
f(x,t) = f(x) cos(Zt)
Matrix Form
Boundary Conditions
Tridiagonal Matrix
Symmetric, positive definite: No pivoting needed
13.002
Lecture 5

General Tri-diagonal Systems
LU Factorization
13.002
Lecture 5

LU Factorization
Reduction
Forward Substitution
Back Substitution
LU Factorization:
Forward substitution:
Back substitution:
Total:
13.002
2*(n-1) operations
n-1 operations
n-1 operations
4(n-1) ~ O(n) operations
Lecture 5

Special Matrices
p
General, Banded Coefficient Matrix
Banded Symmetric Matrix
0
p super-diagonals
q sub-diagonals
w = p+q+1 bandwidth
13.002
b is half-bandwidth
Lecture 5

Special Matrices
Banded Coefficient Matrix
No Pivoting
0
=
0
0
13.002
Lecture 5

Special Matrices
With Pivoting
q
q
0
=
0
13.002
Lecture 5

Special Matrices
Compact Storage
Diagonal
p
0
j
13.002
n2
j-i
n(p+2q+1)
Lecture 5

Special Matrices
Sparse and Banded Coefficient Matrix
Skyline Systems
..
..
Skyline
..
Storage
Pointers
1 4 9 11 16 20
..
Skyline storage applicable when no pivoting is needed, e.g. for banded,

symmetric, and positive definite matrices: FEM and FD methods. Skyline
solvers are usually based on Choleski factorization
13.002
Lecture 5

Special Matrices
Symmetric, Positive Definite Coefficient Matrix
No pivoting needed
Choleski Factorization
where
13.002
Lecture 5

Cramers Rule
3.3-3.4
Numerical stability
3.3-3.5
Partial Pivoting
Equilibration
Full Pivoting

Computation count
LU factorization
Condition Number
Special Matrices
Iterative Methods
13.002
Mathews
Jacobis method
Convergence
3.5
3.4
3.6
Lecture 6

Iterative Methods
Sparse, Full-bandwidth Systems
0
x
x
x
0
Rewrite Equations
x
Iterative, Recursive Methods
0
Jacobis Method
0
0
13.002
Gauss-Seidelss Method
Lecture 6

Iterative Methods
Convergence
Jacobis Method
Iteration Matrix form
Iteration Matrix form
Decompose Coefficient Matrix

Convergence Analysis
with
/
Sufficient Convergence Condition
13.002
Lecture 6

Iterative Methods
Stop Criterion for Iteration
Jacobis Method
Diagonal Dominance
13.002
Lecture 6
vib_string.m
n=99;
L=1.0;
h=L/(n+1);
k=2*pi;
kh=k*h
x=[h:h:L-h]';
a=zeros(n,n);
f=zeros(n,1);
o=1
Off-diagonal values
a(1,1) =kh^2 - 2;
a(1,2)=o;
for i=2:n-1
a(i,i)=a(1,1);
a(i,i-1) = o;
a(i,i+1) = o;
end
a(n,n)=a(1,1);
a(n,n-1)=o;
nf=round((n+1)/3);
nw=round((n+1)/6);
nw=min(min(nw,nf-1),n-nf);
figure(1)
hold off
nw1=nf-nw;
nw2=nf+nw;
f(nw1:nw2) = h^2*hanning(nw2-nw1+1);
subplot(2,1,1); plot(x,f,'r');
% exact solution
y=inv(a)*f;
subplot(2,1,2); plot(x,y,'b');
13.002
% Iterative solution using Jacobi and Gauss-Seidel

b=-a;
c=zeros(n,1);
for i=1:n
b(i,i)=0;
for j=1:n
b(i,j)=b(i,j)/a(i,i);
c(i)=f(i)/a(i,i);
end
end
nj=100;
xj=f;
xgs=f;
figure(2)
nc=6
col=['r' 'g' 'b' 'c' 'm' 'y']
hold off
for j=1:nj
xj=b*xj+c;
xgs(1)=b(1,2:n)*xgs(2:n) + c(1);
for i=2:n-1
xgs(i)=b(i,1:i-1)*xgs(1:i-1) + b(i,i+1:n)*xgs(i+1:n) +c(i);
end
xgs(n)= b(n,1:n-1)*xgs(1:n-1) +c(n);
cc=col(mod(j-1,nc)+1);
subplot(2,1,1); plot(x,xj,cc); hold on;
subplot(2,1,2); plot(x,xgs,cc); hold on;
hold on
end
Lecture 6
vib_string.m
o=1.0
Iterative Solutions
Exact Solution
13.002
Lecture 6
vib_string.m
o = 0.5
Iterative Solutions
Exact Solution
13.002
Lecture 6

Mathews
Roots of Non-linear Equations

Herons formula
Stop criteria
General method
Convergence
Examples
2.1-2.3
Newton-Raphsons Method
2.4
Secant Method
2.4
Multiple roots
Bisection
2.4
2.2
Convergence Speed
Examples
Convergence and efficiency

Examples
13.002
2.1-2.4
Lecture 7
Roots of Nonlinear Equations
Example Square root

Herons Principle
a=2;
n=6;
heron.m
g=2;
% Number of Digits
dig=5;
sq(1)=g;
for i=2:n
sq(i)= 0.5*radd(sq(i-1),a/sq(i-1),dig);
end
'
i
value
'
[ [1:n]' sq']
hold off
hold on
plot(sq,'r')
plot(a./sq,'r-.')
grid on
Guess root
Mean is better guess

(
)/2
value
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
2.0000
1.5000
1.4167
1.4143
1.4143
1.4143
Iteration Formula
13.002
)/2
Lecture 7

Stop-criteria
Unrealistic stop-criteria
Realistic stop-criteria
Machine
Accuracy
Use combination of the two criteria

f(x)
f(x)
flat f(x)
steep f(x)
Cannot require
Cannot require
G
13.002
Lecture 7

General Method
Example: Cube root
Non-linear Equation
Goal: Converging series
Rewrite Problem
Example
% f(x) = x^3 - a = 0
% g(x) = x + C*(x^3 - a)
cube.m
a=2;
n=10;
g=1.0;
C=-0.1;
sq(1)=g;
for i=2:n
sq(i)= sq(i-1) + C*(sq(i-1)^3 -a);
end
hold off
plot([0 n],[a^(1./3.) a^(1/3.)],'b')
hold on
plot(sq,'r')
plot( (sq-a^(1./3.))/(a^(1./3.)),'g')
grid on
Iteration
13.002
Lecture 7

General Method
Convergence
Define k such that if
then
Convergence Criteria
Apply successively
13.002
Convergence
Lecture 7

General Method
Convergence
y=x
Convergent
y=g(x)
Mean-value Theorem
x1
x0
Convergence
y
>
y=x
y=g(x)
Divergent
x0 x1
13.002
Lecture 7

General Method
n=10;
g=1.0;
cube.m
C=-0.21;
sq(1)=g;
for i=2:n
sq(i)= sq(i-1) + C*(sq(i-1)^3 -a);
end
hold off
f=plot([0 n],[a^(1./3.) a^(1/3.)],'b')
set(f,'LineWidth',2);
hold on
f=plot(sq,'r')
f=plot( (sq-a^(1./3.))/(a^(1./3.)),'g')
legend('Exact','Iteration','Error');
f=title(['a = ' num2str(a) ', C = ' num2str(C)])
set(f,'FontSize',16);
grid on
Example: Cube root
Rewrite
Convergence
Converges more rapidly for small
13.002
Lecture 7

General Method
Converging, but how close?
Absolute error
General Convergence Rule
13.002
Lecture 7

Newton-Raphson Method
Non-linear Equation
Convergence Criteria
f(x)
Fast Convergence
Newton-Raphson Iteration
13.002
Lecture 7

Example Square Root
Newton-Raphson
a=26;
n=10;
sqr.m
g=1;
sq(1)=g;
for i=2:n
sq(i)= 0.5*(sq(i-1) + a/sq(i-1));
end
hold off
hold on
plot(sq,'r')
plot(a./sq,'r-.')
grid on
Same as Herons formula
13.002
Lecture 7

Approximate Guess
a=10;
n=10;
div.m
g=0.19;
sq(1)=g;
for i=2:n
sq(i)=sq(i-1) - sq(i-1)*(a*sq(i-1) -1) ;
end
hold off
plot([0 n],[1/a 1/a],'b')
hold on
plot(sq,'r')
plot((sq-1/a)*a,'g')
grid on
legend('Exact','Iteration','Error');
title(['x = 1/' num2str(a)])
Newton-Raphson
13.002
Lecture 7

Convergence Speed
Taylor Expansion
Second Order Expansion
Relative Error
Quadratic Convergence
General Convergence Rate
Convergence Exponent
13.002
Lecture 7

Secant Method
1.
In Newton-Raphson we have to evaluate 2 functions
2.
may not be given in closed, analytical form, i.e. it may be a

result of a numerical algorithm
Approximate Derivative
f(x)
Secant Method Iteration

x
Only 1 function call per iteration:

13.002
Lecture 7

Secant Method
Convergence Speed
Absolute Error
Error Exponent
Taylor Series 2nd order
1
Relative Error
Error improvement for each function call
Secant Method
Newton-Raphson
Exponents called Efficiency Index
13.002
Lecture 7

Multiple Roots
p-order Root
Newton-Raphson
=>
f(x)
Convergence
x
Slower convergence the higher the order of the root
13.002
Lecture 7

Bisection
Algorithm
f(x)
n = n+1
x
yes
no
13.002
Less efficient than Newton-Raphson and

Secant methods, but often used to isolate
interval with root and obtain approximate
value. Then followed by N-R or Secant
method for accurate root.
Lecture 7

Bisection
% Root finding by bi-section
f=inline(' a*x -1','x','a');
bisect.m
a=2
figure(1); clf; hold on
x=[0 1.5]; eps=1e-3;
err=max(abs(x(1)-x(2)),abs(f(x(1),a)-f(x(2),a)));
while (err>eps & f(x(1),a)*f(x(2),a) <= 0)
xo=x; x=[xo(1) 0.5*(xo(1)+xo(2))];
if ( f(x(1),a)*f(x(2),a) > 0 )
x=[0.5*(xo(1)+xo(2)) xo(2)]
end
x
err=max(abs(x(1)-x(2)),abs(f(x(1),a)-f(x(2),a)));
b=plot(x,f(x,a),'.b'); set(b,'MarkerSize',20);
grid on;
end
Algorithm
n = n+1
yes
no
13.002
Lecture 7

Mathews
Interpolation
4.1-4.4
Lagrange interpolation
Triangular families
Newtons iteration method
Equidistant Interpolation
Numerical Differentiation
Numerical Integration
4.3
4.4
4.4
4.4
6.1-6.2
7.1-7.3
Error of numerical integration
13.002
Lecture 8
Numerical Interpolation
Given:
Find
for
Purpose of numerical Interpolation

1. Compute intermediate values of a sampled function
2. Numerical differentiation foundation for Finite
Difference and Finite Element methods
3. Numerical Integration
13.002
Lecture 8
Polynomial Interpolation
f(x)
Coefficients: Linear System of Equations

x
Interpolation
Interpolation function
13.002
Lecture 8
Examples
f(x)
f(x)
Linear Interpolation
13.002
Quadratic Interpolation
Lecture 8
Taylor Series
Remainder
f(x)
Requirement
p(x)
f(x)
Ill-conditioned for large n
x
13.002
Polynomial is unique, but how do

we calculate the coefficients?
Lecture 8
Lagrange Polynomials
f(x)
1
k-3 k-2 k-1
k k+1 k+2
Difficult to program
Difficult to estimate errors
Divisions are expensive
Important for numerical integration
13.002
Lecture 8
Triangular Families of Polynomials
Ordered Polynimials
Special form convenient

for interpolation
where
Coefficients
found by recursion
13.002
Lecture 8
Triangular Families of Polynomials
Polynomial Evaluation
Horners Scheme
Remainder Interpolation Error
13.002
Lecture 8
Newtons Iteration Formula
Standard triangular family of polynomials
Newtons Computational Scheme
Divided Differences
13.002
Lecture 8
f(x)
f(x)
13.002
Lecture 8
Equidistant Newton Interpolation
Equidistant Sampling
13.002
Divided Differences
Stepsize Implied
Lecture 8
function[a] = interp_test(n)
%n=2
h=1/n
xi=[0:h:1]
f=sqrt(1-xi.*xi) .* (1 - 2*xi +5*(xi.*xi));
%f=1-2*xi+5*(xi.*xi)-4*(xi.*xi.*xi);
c=newton_coef(h,f)
m=101
x=[0:1/(m-1):1];
fx=sqrt(1-x.*x) .* (1 - 2*x +5*(x.*x));
%fx=1-2*x+5*(x.*x)-4*(x.*x.*x);
y=newton(x,xi,c);
hold off; b=plot(x,fx,'b'); set(b,'LineWidth',2);
hold on; b=plot(xi,f,'.r') ; set(b,'MarkerSize',30);
b=plot(x,y,'g'); set(b,'LineWidth',2);
yl=lagrange(x,xi,f);
b=plot(x,yl,'xm'); set(b,'Markersize',5);
b=legend('Exact','Samples','Newton','Lagrange')
b=title(['n = ' num2str(n)]); set(b,'FontSize',16);
function[y] = newton(x,xi,c)
% Computes Newton polynomial
% with coefficients c
n=length(c)-1
m=length(x)
y=c(n+1)*ones(1,m);
for i=n-1:-1:0
cc=c(i+1);
xx=xi(i+1);
y=cc+y.*(x-xx);
end
13.002
function[c] = newton_coef(h,f)
% Computes Newton Coefficients
% for equidistant sampling h
n=length(f)-1
c=f; c_old=f; fac=1;
for i=1:n
fac=i*h;
for j=i:n
c(j+1)=(c_old(j+1)-c_old(j))/fac;
end
c_old=c;
end
Lecture 8

Mathews
Interpolation
4.1-4.4
Lagrange interpolation
Triangular families
Newtons iteration method
Equidistant Interpolation
4.3
4.4
4.4
4.4
6.1-6.2
7.1-7.3
Error of numerical integration
13.002
Lecture 9
Taylor Series
n=1
f(x)
First order
h
13.002
Lecture 9
Second order
f(x)
n=2
Second Derivatives
n=2
Forward Difference
n=3
Central Difference
13.002
Lecture 9
Lagrange Interpolation
f(x)
Equidistant Sampling
Integration Weights (Cotes Numbers)
Properties
13.002
Lecture 9
f(x)
n=1
Trapezoidal Rule
Simpsons Rule
f(x)
n=2
13.002
Lecture 9
Simpsons Rule
Error Analysis
f(x)
Local Error
Global Error
Trapezoidal Rule
Local Absolute Error
N Intervals
Local Error
Global Error
13.002
Lecture 9

Mathews
Ordinary Differential Equations

Initial Value Problems
Eulers Method
Taylor Series Methods
9
9.1
9.2
9.4
Error analysis
Runge-Kutta Methods
13.002
9.5
Systems of differential equations

Boundary Value Problems
9.7
9.8
Shooting method
Direct Finite Difference methods
9.8
9.9
Lecture 10

y
Linear Differential Equation
Non-Linear Differential Equation
non-linear in y
Linear differential equations can often be solved analytically

Non-linear equations require numerical solution
13.002
Lecture 10

Eulers Method
Example
Discretization
Finite Difference (forward)
Recurrence
euler.m
13.002
Lecture 10

Truncate series to k terms
Initial Value Problem

Taylor Series
Derivatives
Choose Step Size h
+
Partial Derivatives
Discretization
Recursion Algorithm
with
Local Error
13.002
Lecture 10

General Taylor Series Method
Example Eulers Method
Example
Error Analysis?
Eulers Method
13.002
Lecture 10

Derivative Bounds
Error Analysis
Error Estimates and Convergence
Eulers Method
O(h)
13.002
Lecture 10

Example Eulers Method Error Analysis
Exact solution
Derivative Bounds
Error Bound
13.002
Lecture 10

Runge-Kutta Methods
Taylor Series Recursion
Substitute k2 in Runge Kutta
Runge-Kutta Recursion
Match 2nd order Taylor series
Match a,b,DE to match Taylor series amap.
13.002
Lecture 10

Runge-Kutta Methods
Initial Value Problem
y
mid-point
2nd Order Runge-Kutta

x
4th
13.002
Order Runge-Kutta
Predictor-corrector method
Lecture 10

Runge-Kutta Methods
Eulers Method
Recurrence
4th Order Runge-Kutta
h=1.0;
x=[0:0.1*h:10];
rk.m
y0=0;
y=0.5*x.^2+y0;
figure(1); hold off
a=plot(x,y,'b'); set(a,'Linewidth',2);
% Euler's method, forward finite difference
xt=[0:h:10]; N=length(xt);
yt=zeros(N,1); yt(1)=y0;
for n=2:N
yt(n)=yt(n-1)+h*xt(n-1);
end
hold on; a=plot(xt,yt,'xr'); set(a,'MarkerSize',12);
% Runge Kutta
fxy='x'; f=inline(fxy,'x','y');
[xrk,yrk]=ode45(f,xt,y0);
a=plot(xrk,yrk,'.g'); set(a,'MarkerSize',30);
a=title(['dy/dx = ' fxy ', y_0 = ' num2str(y0)])
set(a,'FontSize',16);
b=legend('Exact',['Euler, h=' num2str(h)],
'Runge-Kutta (Matlab)'); set(b,'FontSize',14);
Matlab ode45 automatically ensures convergence

Matlab inefficient for large problems > Convergence Analysis
13.002
Lecture 10

Mathews

Eulers Method
9
9.1
9.2
9.4
Error analysis
Runge-Kutta Methods
13.002
9.5
Systems of differential equations

9.7
9.8
Shooting method
Direct Finite Difference methods
9.8
9.9
Lecture 11

Higher Order Differential Equations
Convert to 1st Order System
Initial
Conditions
Matrix form
Solved using e.g. Runge-Kutta (ode45)

13.002
Lecture 11

Shooting Method
Boundary
Conditions
Shooting Method
Initial value Problem

Solve by Runge-Kutta
Shooting Iteration
13.002
Lecture 11

Direct Finite Difference Methods
Boundary
Conditions
Discretization
Finite Differences
13.002
Lecture 11

Direct Finite Difference Methods
Boundary value Problem
Matrix Equations
Finite Differences
Substitute Finite Differences
Linear Differential Equations

-1
Solve using standard linear system solver
Difference Equations
13.002
N-1 equations, N-1 unknowns
Lecture 11

Finite Difference Methods
Forced Vibration of a String
f(x,t)
xi
y(x,t)
Finite Difference
Discrete Difference Equations
Harmonic excitation
f(x,t) = f(x) cos(Zt)
Matrix Form
Boundary Conditions
Tridiagonal Matrix
Symmetric, positive definite: No pivoting needed
13.002
Lecture 11

Finite Difference Methods
Boundary Conditions with Derivatives
N-1
Central Difference
N+1
Central Difference
Difference Equations
Backward Difference
O(h3 )
General Boundary Conditions
O(h4 )
O(h 2)
13.002
Finite Difference Representation
Add extra point - N equations, N unknowns
Lecture 11

Mathews
Minimization Problems
Least Square Approximation
Normal Equation
Parameter Estimation
Curve fitting
Optimization Methods
Simulated Annealing
Traveling salesman problem
Genetic Algorithms
13.002
Lecture 12
Minimization Problems
Data Modeling Curve Fitting
Linear Model
Non-linear Model
Minimimize Overall Error
Objective: Find c that minimizes error
13.002
Lecture 12
Linear Measurement Model

n model parameters
m measurements
Overdetermined System
m measurements
n unknowns
m>n
Least Square Solution
Minimize Residual Norm
n
13.002
Lecture 12
Theorem
A
Proof
q.e.d
Normal Equation
)
Symmetric n x n matrix. Nonsingular if columns of A are

linearly independent
13.002
Lecture 12

Parameter estimation
Example
Island Survey
Measured Altitude Differences
C
D
Points D, E, and F at sea level. Find altitude

of inland points A, B, and C.
A=[ [1 0 0 -1 0 -1]' [0 1 0 1 -1 0]' [0 0 1 0 1 1]']
b=[1 2 3 1 2 1]';
C=A'*A
c=A'*b
% Least square solution
z=inv(C)*c
lstsq.m
% Residual
r=b-A*z
rn=sqrt(r'*r)
13.002
Normal Equation
Residual Vector
Lecture 12

Curve Fitting
% Quadratic data model
curve.m
fxy='a*x.^2+b'
f=inline(fxy,'x','a','b');
x=[0:0.01:1]; x=reshape([x' x']',1,2*length(x));
n=length(x); y=zeros(n,1);
a=5; b=3;
% Generate noisy data
amp=0.05*(max(f(x,a,b))-min(f(x,a,b)));
for i=1:n
y(i) =f(x(i),a,b)+random('norm',0,amp);
end
figure(1); clf; hold off; p=plot(x,y,'.r');
set(p,'MarkerSize',10)
% Non-linear, quadrati model
A=ones(n,2); A(:,1)=f(x,1,0)'; bb=y;
%Normal matrix
C=A'*A; c=A'*bb;
z=inv(C)*c
% Residuals
r=bb-A*z; rn=sqrt(r'*r)/n
hold on; p=plot(x,f(x,z(1),z(2)),'b'); set(p,'LineWidth',2)
% Linear model
A(:,1)=x';
C=A'*A; c=A'*bb;
z=inv(C)*c
% Residuals
r=bb-A*z; rn=sqrt(r'*r)/n
hold on; p=plot(x,z(1)*x+z(2),'g'); set(p,'LineWidth',2)
p=legend('Data','Non-linear','Linear'); set(p,'FontSize',14);
13.002
Lecture 12
Optimization Problems
Non-linear Models
Non-linear models
E(c)
Minimimize Overall Error
Local
Minimum
Measured values
Global
Minimum
Model Parameters
xi
Non-linear models often have multiple, local minima. A locally linear, least square
approximation may therefore find a local minimum instead of the global minimum.
13.002
Lecture 12
Optimization Algorithms
Simulated Annealing
Boltzman Probability Distribution
Analogy: Freezing of a Liquid
Energy probabilistically distributed among all states.

Higher energy states possible even at low temperature!
High temperature T
Optimization Problem: Minimize residual energy

Simulated thermodynamic system changes
its energy from
to
with probability
Low temperature T
Lower energy always accepted

Higher energy accepted with probability p:
Allows escape from local minimum
Crystal: Minimum energy of system.

Slow cooling -> global minimum: crystal.
Fast cooling -> local minimum: glass.
13.002
1.
2.
3.
4.
Elements of Metropolis algorithm

Description of possible system configurations
Random number generator for changing parameters
Cost function energy E
Control parameter temperature T.
Lecture 12
Simulated Annealing
Example: Traveling Salesman Problem
Objective:
Visit N cities across the US in arbitrary
order, in the shortest time possible.
Metropolis Algorithm
1.
2.
3.
4.
Adapted by MIT OCW.
Cost function: Distance Traveled
Penalty for crossing Mississippi
East:
West:
13.002
Configuration: Cities I = 1,2, N. Order

can vary
Rearrangements: Change the order of
any two cities.
Cost function: Distance traveled,
number of Mississippi crossings.
Annealing schedule. Experimentation
with cooling schedule. T held constant
for e.g. 100 re-orderings (heat-bath
method).
=1
= -1
Lecture 12
Simulated Annealing
Example: Traveling Salesman Problem
% Travelling salesman problem
salesman.m
% Create random city distribution
n=20; x=random('unif',-1,1,n,1); y=random('unif',-1,1,n,1);
gam=1; mu=sign(x);
% End up where you start. Add starting point to end
x=[x' x(1)]'; y=[y' y(1)]'; mu=[mu' mu(1)]';
figure(1); hold off; g=plot(x,y,'.r'); set(g,'MarkerSize',20);
c0=cost(x,y,mu,gam); k=1; % Boltzman constant
nt=50; nr=200; % nt: temp steps. nr: city switches each T
cp=zeros(nr,nt);
iran=inline('round(random(d,1.5001,n+0.4999))','d','n');
for i=1:nt
T=1.0 -(i-1)/nt
for j=1:nr
% switch two random cities
ic1=iran('unif',n); ic2=iran('unif',n);
xs=x(ic1); ys=y(ic1); ms=mu(ic1);
x(ic1)=x(ic2); y(ic1)=y(ic2); mu(ic1)=mu(ic2);
x(ic2)=xs; y(ic2)=ys; mu(ic2)=ms;
p=random('unif',0,1); c=cost(x,y,mu,gam);
if (c < c0 | p < exp(-(c-c0)/(k*T))) % accept
c0=c;
else
% reject and switch back
xs=x(ic1); ys=y(ic1); ms=mu(ic1);
x(ic1)=x(ic2); y(ic1)=y(ic2); mu(ic1)=mu(ic2);
x(ic2)=xs; y(ic2)=ys; mu(ic2)=ms;
end
cp(j,i)=c0;
end
figure(2); plot(reshape(cp,nt*nr,1)); drawnow;
figure(1); hold off; g=plot(x,y,'.r'); set(g,'MarkerSize',20);
hold on; plot(x,y,'b');
g=plot(x(1),y(1),'.g'); set(g,'MarkerSize',30);
p=plot([0 0],[-1 1],'r--'); set(g,'LineWidth',2); drawnow;
end
13.002
function [c] = cost(x,y,mu,gam)

n=length(x);
c=0;
for i=1:n-1
c =c+sqrt((x(i+1)-x(i))^2
+(y(i+1)-y(i))^2)
+ gam*(mu(i+1)-mu(i));
end
cost.m
Lecture 12

! "#%$'&()*+

, -$'&/.
0 $1243&/.5
6879-$:79*;97=<
>.$'&?2 ! @>A!.-B&C7D=
EGFHJIKIMLONQP:RSHTPVUXWZY1[G\]N_^`Ybac[dN'Y1[D\KN'egfh[1Ydi]NjUlkmUonqp
r5RsUutvHwxHzy{[|UlNq\qR}e
E
99
.79 )&?! A $'&/.>$1243&/.5

2%$:7$1 & !
27=< ${A*
!= 7 Ah o#

7Z7D*O)7= 2%$:7"&/4 .. 7 34A!D$:h 7
Ah5oA*>$:7DAB&?!
< s& 2$:! A=` ?& !&?!$'&? =A.B&C79
*/ =&B&/7D= 7 -h-)779 A=`
, -*/)
5A)]{h.z , Ah , -*/)
, -*/) M/ 2$7"&s 27=< ${A4*
2%$:7$1 &! '!-A4$: 5A!.3 =
-$qA.z-A$: 4 .*O*Z%$d$1B
B 9 4K9(
$'&? &524*/&? -$'&
- d Z z

M &s.79*+7D A22%$: 79A-2A

- d Z
z D
- d Z
z

F
B 9 4K9(
, -*/) 792$17$h
%l
-h
% z%

7DA . A= , -*/) = .*.A*/7$K
- _Q

|5z
7 hz&/D 4 2 */.-3 $:A*J &
24*/.= &? -3 "%$'&()4*
KO9( _
, -*/) "#$'&(4)*+ A= )h&? < &-3 *+{$
!3 $: 7 -3 .3$1!.-$ .!4 ) *-{$
&/&7$ A $.7$:h
4 * -3 $ .3$1!.-$ $ z&/D&_.
K xKojdxxKx'dxToKx'lux
xKojdxxKx'dxToK
K
, -*/) & . = `hz&T"
K h
K =
! %$: !7# -3! 4
d> 4KO9(

K

h

K 1
'

K

u
G "!$#%'&%b () T d
'
* *
K
+
-,M., 0/9
M2#43
1
!3 {A4.zB&C79 6 5! *(& -3 " %$'&()4* & -3!

< 7$d52.
K 7
8`
9 '
:T &d% u; (`] u 4!
<
'

!3 {A4.zB&C79 6 5! (* & -3! z&>= ! 7$
4*O*7Z.!B&/7D 7 D
7 A$ "#% $'&()*+
K 7
8`
* G
<
'
lK1'
?;
()
()
()
()
AB

o
J
'
K

L
'J
()
M b{
ANAx

x oK1'
K
b'A) Do1 È KGF

*
HI:`x
*
HI:`x
*
HI:`x
*
HI:`x
9
HI:`x
:G
OJ
( )A
C)A
AB
P
-,M., 0/9
M2#43
1
3! .79 4 - . ) Ah= 7 $: 7"!

"#%$'&()*+ o$:79 -3 < 7d$ 2#.
K Q]Kxd <
K 7
8`
9 '
:T &d%u;(`] u4!
b
K

z < &-3 !7 $ A *+- *O*457DA$

"#%$'&()*+
K Q]Kxd
K 7
8`
9 '
:T &d%u;(`] u4!
K
R

u MT
/T3T/
3! { A4.zB&C79 5 .3#4 -3! 2$=.%&z&C79 7

- 3 79A -24A
K E
'
UVWJ
K TdbG A xj=$E
'
X ' OxJxKxKOKO

K TdbG A Y8d
A Z
E
'
X ' JKI

K TdbG A xj [
E
'
V ' Ox JKKx KO] O]oI

K TdbG A N A!Z
E
'
xx xG
\
]3^/
_T
/
K 8loWE TdqGA
H_;`K <a
?xNA
W: AIE: A qqGAb
< x
QK1 E:AGA%]j
1
<ac< C uT 'l 1 ':(`K ETT Q'd
H_;`x <a
L) (l :u Al Y7'AeQW8 (NA7lKb %NTdbA W: A
E: A
YEKG ) qqGAB KxK;7]!
e
He_'`x <a
#I{;:`;Ab ?xG
?Nf%_'` a
He_'`x <a
?;f%_'` a
?Q]xKu
e'Fu
E`GA
TdqGA
7WA
8
e;e'AB]
c _ * M
He_'`x <a
?Q]xKu
e'Fu
E`GA
TdqGA
7WA
8
' %'%'A%g
He_'`x <a
?;f%_'` a
@
Hx]G
AB1 E` 1 A qqNA 7B'AI8 %'%WABg
c _ * M
He_'`x <a
@
Hx]G
AB1 E` 1 A qqNA 7B'AI8 j e;%WAB]
He_'`x <a
f@h
fL
FTu Q o qqN Ab
a 8
He_'`x <a

;)b
(`K i j bT (`]b k d ET%1 Au
Td
E`
'A%'&g i8lL TG ABW& b{ {u o]Gb A%g
G
%Glu ) Ed A% d ;dTu"
He_'`x <a
C < *l
H%;FTu
dbN
A q xKKd% 'T Q]' A%g
He_'`x <a
D_b
m < D a ?':EET L FAx 1 jGTKu% ]
c __
?G@
He_'`x <a
mI: A%
AI8
FAx 1 jGTKu% ( Q'k Gn
L
< EE{G
He_'`x <a
` <a
Fe|G AB' ( ) G A%] L |ox G AG ue g
E'I
d -., K
- qp _ z

p
-
- dsr r ut wv x
z yTv zx T{v: x v p|x
T{
v;} ~4xr B v;} X|x Bv;} X|x`

}}}}
}
}}}}=
}}}}
}
pM]T
~- p
}
} |B]p]~
-
h
~ }}}
}
91-~~
EKE
K 3u B _d

- d pD
-

~
~
B
-

pB

}
p]~
- d' ~9
-

~
zD

}
- w

-
Ej
(D B#d`Z(D {u# 039u B _d
! % z
%l
!
K < % v]h Co%j

J |Y]h
K < ;C

@oxd
:G
xT
xd
GAoe;F % b' :'A LKx4
K < ;C
'
jJ

V

K < q'
'
EjF
} 9 B -O
7 .$ "!h.-7$ < &-3 &?&B&(*4"#*OA

&?!.$: ! # *"4*A ] XAh -3!
.79*+7D !7#B&C79
F%]! F
F
x=
![F
4 24*
K

K
F
!

U!U6!
Dox':uT
A
8{': I8
UW

V

Dox':uT
A
8{': I8
U

J

Dox'
:uT K A
8{' : I8 W
UK
O

U

6
4$4&! .79*+7D !7#B&C79 & B&(*7

=T$'&?! -*s4)
E
uT39
hz&/. 2*+7{B&? .79 4

&?! u2 ! .79*+7$
3#4. 7 )4AB&Q 579A4$ 24*7#
z&? 5! -
M-{B&? -3! 4& *&? &h % 4
E

3 ( D -.,
D
@79A4$ u2 7
^v
"z
24*+7 -3 "h.z7$
rXdx
y^vV
*+7 !
"=hw
2*+7# < &-3

& (* &?!!$
rdx
y vV
*(&?!!$ !
z y zyTv
*+7
2*+7h
rdx
2*+7# < &-3

& +* 7
rdx
l&
"
l&
"
2*+7# < &-3 )7-3
E'+
-39 9
%$'&C79A= *&? d2X2*+7# ) 7D* 4 .7D*7$

B ) 79)&?!h < &-3 u v rr;x < 3!$: &
7$ .3#%$1.z$ T-$'&? !4 o$:79 -3
j79*O*+7=< &? .3#%$1!.-$h
xxK;7
G' A
QN)Tb
d
xTx'
(`W
:
78'A
(`] QWk
Q

(
7
k
1 A
Qb
QKK
Fj1u
k
E'
:
x
b
WAd
N AAu
T 8T;A
T 8u
F

j

!
j
jj
@ 7$ -4 24*4-3! j79*O*+7=< &? 24*7# 7
"=h^ A=z&? )*OA! 24*Ah

`KNAFUi)Ui
E
( 4
EP
9>( -O 9 9

3 ( 9 --,
9
v = -
%
! B&-*+
hz
*s4)*
hz
*s4)*
h-^v: sr r hz r x 2A hz

v: s r x &? .$: .7Z7$:&?# < 3$:
9

& -3! *+7=< $ *+5 4
& -3 A242$ $'&/D3 7 -3! .$:

A)&+"&/4 -3 < &?47h<
EWR
9>( -O 9 9

3 ( 9 --,
9
.3#44 %
%
%
4DA*=2=.z $QB&C7
! D$'&/ *&?
zy `
5 4**+7h< 7DA 7 A4*JB&24*+ 2*+7

7D -3 5A )42*+7
#4)*+ = 7Z7D
A=z&? 7DA=!
7 y-%` 5 g! D

7 2$Q (* &?
d5A..&+"! .!4*O* -A $q -3 2%:$ 7D2$ u
74D*+g
7D 7$ n7
E'\

3 (

3 }
3 } 9

K F !!14x
K ) F" G|FB
K Q]L
QK]xd%
A
8
QW:{xTb
K Ex;AFi)
K A%'AKmxNA L Fe 1|F &%g[FUq
K FK'(o FU
K )K'(o )U
K K%'
K E{%1 A jd'E E`KNA6V'E
K 'E E`KNAV'E
TI G:WA ' AWAej
KKK 1 eK
A
e'I:
Plot of x*sin(x) vs. x

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
x
0.6
0.7
0.8
0.9
xI
-
-
-
-
-
-
-
-

=

D 9
3 (
3 ( 9
9
}wX} ] T
T{vV x
T{
v x
w
vV r sr -
=zwv'} ] r } X rX
= zw
v'} ] r } | ]~r=

v
x
=r
r
T {vV x -
x

v x x
v x yz`
0.9
0.8
y1 = x sin(x)
0.7
y2 = sin(x) _._
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
x
0.6
0.7
0.8
0.9
E
-, hy
K A ALFA4iUKi) F 1|F jjjB

K A ALFA4iUi) G|F 4]B
K TGA4AI
Doxxq
1
@xuqld
`dbx
@FA' A
UO
JOK U Kxo UKOJK
H' A < K
`dbx
H' A * 1
fo; &G A%Q
H' A?;T
W K
H' A ;I8A
`qqo
fld%
;j Ax < ;ub A ]IIA
m WAB'
U K ]
`lN
AN AB' ]
?NAo%
1 ) F G| F jjj

'A% TG A
u
A%Qo < 'b A bK]
CI: AA'
#l; 7] HeQj
D;8]qxTb

Do YEE1
G
AdxL
:E A%Y(K
mu' A
Kou
a )IE
AG FA
u
#N A
(`]
'
K uG A4A i Hlj A? N{g i W Ke
K uG A4A] i Hlj A? N{g i W Ke
xGxo

K
-, hy
0.9
0.8
y1 = x sin(x)
0.7
y2 = sin(x) _._
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
x
0.6
0.7
0.8
0.9
KF
K
K
K
K
K
K
K
K
K
- 9
/9^39( 9
`x;A`xI
Ex;AFi)
FK'(o FUB )K;(lo )B xeb=

A%'AK
mxN A L Fe 1|F &%g[FUq
:(E`x;
A`'
Ex;
AFi)Te
FK'
(o FUB )K;(lo )B xeb=
A%'AK
mxN A L 1| F &BgZFUq
E{%
1 A jd' E E`KN A V' E
:(E
Plot of x*sin(x) vs. x

1
0.8
0.6
0.4
0.2
0
0
0.1
0.2
0.3
0.4
0.5
x
0.6
0.7
0.8
0.9
0.7
0.8
0.9
Plot of sin(x) vs. x

1
0.8
0.6
0.4
0.2
0
0
0.1
0.2
0.3
0.4
0.5
x
0.6
d 1
D4OO
, -*/) .79 4 j7$ 4h.&z&C79hh

|u6

r
! `
NA
TL ) 'AoTb:bV4Q!VB
;
Txxq4VL)KL ]
x T L)
YEU A
8 G) G AI8 j A
8 q AdTL)
e
x u
YEU A
8 G) G AI8 j A
8 q Ad TL)
e
'T

G&b"q
dKU B

K TGA
'
] j;H;(%jO]
K NA
K TL ) 'AoTb:bV4Q!VB
K ; TxKd4 VG )K L e K TL)
YEU A
8 G ) G AI8 j A
8 q AdTL)
e
x u
YEU A
8 G ) G AI8 j A
8 q Ad TL)
e
'T
AI8
G ) G AI8 j A
8 q AldTL) dK

G&b"q
dKU B
d 1
( 03
, -*/) .79 4 j7$ *7792=

5 ! 65
4 42 *
QK1 E:A
{ Q'Aldex
7B'A
8
q
KxWE
Y
T
Q;Alq% uou
q
b
!
{ Q;Ade
o T Q'Alq%x
b{
T
Q;Alq% uo
QK1 E:A
{ Q'Alde
x 7B'A
8 78]] K x'E
q
T
Q;Alq% uou
78]K
q
e
W

N
T Q'Alq%
x T Q;Ad% uo -
b{
T
Q;Alq% uo
+
K3
/% O
, -*/) .$'&?2 ! oA.zB&C79h $ .!4**+h
) h.A=` -3! 3"! -3! 5Ag
.$'&?2 %$:
>
*+ -3 .7D&? A!.
7 -*/4) .D
7 !h
@>A!.-B&C7D= %$: ! -3 $-A4$d "4*A
3! )&/44 & $:!. )d< .$'&?2 4
o A!.-B&C79h & -3 "#$'&(4)*+ .$:!h &? oA.zB&C79h
%$: *+7.*"#$j&4)*+< 3$:!= "#$'&(4)*+ .$:=
&? .$'&?2 %$: *7D)#4*S
!3 -*/) 7Z7D*O)7= %$: .7D*O*h.zB&/7D= 7

Ah{A* !
j$ &B&?! 5 79A4$ h7 < .$'&?2 4 { A4.zB&/7D=
& !=&/$ 7 $: .%&/ 7 hA -*s4)
P
9
3 (
9 K3
| K4 j E`] GAK;( Qb%E A
a 8
Qb%E A
Nk | E`K E`KNA L A
8l G
G:x
Q'AB'n A Y:u x'E8l TIGd
1
AI8
7dIkE Q
(lITd
A
8l

QqeYE A

Q]xx]u
A
!U6 !1h
)
1| x' E8 A
Ex;
AAUi)
FK'
(o AB| Q6q
)K'
(o A)4Au G| oW E8 ;A6B
A%'AK
| EK Ex; A ( ) xKx'dx ]K
OKg
K%
' j
K | E`K
K 8loW E | EK
|
EK4 j E`] G AK; ( Qb% E A
a 8
Qb%
E A N k | E`K E`KN A L A
8l G
G:x
Q'AB'n A Y:u x' E8l TI Gd
1
AI8
7dI
kE Q (lI Td A
8l
Qqe YE A Q]xx]u
K
E
R
uT3 3 d / #

3 ({
simple plot by jleonard 2/23/97

1
0.8
0.6
ty(t) = sin(alpha*t)
0.4
0.2
0
0.2
0.4
0.6
0.8
1
0
0.1
0.2
0.3
0.4
0.5
0.6
time (sec)
Y%VZY
wB;G''4G
We
0.7
0.8
0.9

XeNN
\

3
3

!
+-,./10*2 34/-5-6 287 9':;86 4

< 6=:19 4
0 :>2=2?6.@
E F 5D6G7H@:>.=A IKJ*L-MDJ*L-N=O
3 P IBM-J=J=JDJRQ
5-:K9*<SA?: P IBTDJDJ*L>3UQ
2WV628: P XZYDNDJU[\JU]\JÎ_[`N-Ja b 0^/ L IHcDJRQ
; P M b 0^/-L=5-:K9*<SA?:dQ
: P Je]J=f=gGLDMdQ
2WV628:Df P M=NU]f b 5-:K9S<*A?: L XZM b :8a_Q
"#
287
$ &%'%()!*
A4/=+B2=./1<=C*2?6
38/ihCS.?6jX1IWa
,-53
<D0 P ,K0=<6D:K9kX`2-V46>2?:ml ;nl :8a_Q
<D0_I P <=0kQ
05=7i2UX`2WV628:*b8IHcDJ*LB0^/jloIHJ b 5=7hpIHJqX<D0(a=a
:>r/D+
X4s\YDN-J NDJ YWc=J J-tâuQ
+6>2vXwh4,W:ml x\yD2/D,H;zx=l s`Y-NDJU[\f=JU[`N-J-tâuQ
hD./BA 7H@
V7=5A 7H@
05=7i2UX4s{2-V46>28:-f 2WV6>2?:DfWtUl s\Y-cDJ JWtUl x|YR]Wx}a
05=7i2UX4s\Y2WV6>2?:Df Yi2-V46>28:-f-tvl s\Y-c=J J-tvl x~Yv]x}a
2/B285D6qX8sx\3 P IBM-J ;DS
P x=l@=CW9^MS+H2G.vX|M b :4aul
x9
<46=:19?^/BAD2WV P ?Yqx=l@DCW9^MS+B2=.vX\2-V628:-fa_l
r?5D:B<6G5X*x2-V628: XÀ?6>h=.?6D6S+*a_x}auQ
5D:B<6G5X*x47>.i9':=5*/*6A +W7BC*.4,-6 5D686G5 XwA-aux}a_Q
Y%VZY
wB;G''4G
We
/1@
,-5D:*+=+
] ]=]
D
xAS6hex\tâuQ

XeNN

$"&
3>C=@,i2/W7B@
E
<=0
Z
,K0=<46=:K9kX`2WV6>2?:ml;zl~:8a
==-[ ?6=:19 04:>2=2?6.@ 3S7>. ,G/i.8,HC5D:. 0'/D+B287B@ 2=.?:B@'+iAC^,W6.el

>
C^+=/K@*h +H28:i@*A?:>.=A <6S+D+W6=5 3>C=@^,B2/W7B@ 9'7>AS6G5v] V6>.?6 :.?6 2D47
C^+W:hS6S+d[
<D0 P ,H0D<6D:K9X\2-V46>28:dl;nl~:4a
<D0 P ,H0D<6D:K9X\2-V46>28:dl;8:4a
E
E
E
E
E
E
==- .?6>2WCS.@'+ 2-V6 @7.9^:G5*/iS6>A <6D:K9 3CD@^,B2/-7H@ 3S7>.

>
8:>86B@=C9S<6>. ; :i@*A 2=.?:B@'+iAC^,W6. .S:A4/KC'+ : :>2 : <7>.S6S+=/BhWVG2
:B@Sh85-6 2WV6>2?: X`.?:>A/W:B@'+*aj]
:BC*2-V7>.z[G.S:A?5D6 k]7.?:B@klH =6D: D.?:B@*2el IBNDNDf
E
E
E
/i3
@4:.Dh/K@
.S6AC^,-6>AD=.?6W
<D0
fRl :
IpQ 6B@SA
;b-:GbG+=/1@nX\2-V628:8a_Q
X|M*bH<46S+D+W6G5X1Ipl~:i<^+
X`.?6AC^,W6AD=.?6â=a]wL.?6AC^,W6A==.?6Waj]}HMmQ
Y%VZY
wB;G''4G
We

XeNN
%
#
$ %%()!*
f = 120 kHz D = 0.037 m beamwidth = +9.899 deg

0
10
Normalized source level (dB)
20
30
40
50
60
70
80
90
60
30
0
theta (degrees)
30
KD}S`1K-i>B1DBv1>-H4G
60
90
q=
*4
D
Issued: February 3rd, 2005

13.002 Introduction to Numerical Methods for Engineers
In-class programming exercises
1. Let a be a positive real number, and let the sequence of real numbers xi be given by
x0 = 1,
xi+1 =
a
1
(xi + ),
2
xi
for i = 0, 1, 2, 3, . . ..
The value xi will converge to a as i Write a program that reads in the value of a
interactively and uses this algorithm to compute the square root of a.
Test your program as you vary the maximum number of iterations of the algorithm is
increased from 1, 2, 3, . . . and determine how many signicant digits of precision that you
obtain for each. How many iterations are necessary to reach the machine precision of
matlab?
2. Write a program to evaluate e by the series:

e=1+1+
1
1
1
1
+ + + + ...
2! 3! 4! 5!
Test your program as you increase the number of terms in the series. Determine how many
signicant digits of precision that you obtain in your answer as a function of the number
of terms in the series. How many terms are necessary to reach machine precision?
3. Consider the function x sin(x) 1.

(a.) How many roots does this function have in the interval [0, ]?
(b.) Write a matlab program to nd the root(s) using Newton-Raphson iteration with
appropriate starting values.
(c.) Make a graph of relative error vs. iteration step for all roots.
(d.) How many iterations are needed to reach an error of less than 108 ?
13.002J/10.002J Introduction to Numerical Analysis for Engineers

Problem Set 1
Issued: February 3, 2005
Due: February 10, 2005
Do the following problems from Mathews and Fink:
1.2.4 (a) and (b) (on page 23)
1.2.5 (a) and (b)
1.2.13 (b) (on page 24)
1.3.1 (b) and (c) (on page 37)
1.3.12 (on page 39)
1.3.13 (a)
Programming exercise 1 (on page 39)
Programming exercise 2
13.002
PS #1 Solution
1.2.4. (a)
(1.0110101)2 = (2^0)+(2^-2)+(2^-3)+(2^-5)+(2^-7)= 1.41406250000000
1.2.4. (b)
(11.0010010001)2 = (2^1)+(2^0)+(2^-3)+(2^-6)+(2^-10)= 3.14160156250000
1.2.5. (a)
2 (1.0110101) 2 = 1.41421356237309 1.41406250000000=

= 1.510623730900385e-004 (absolute error)
1.510623730900385e-004
= 1.068172283940988e-004 (relative error)
1.41421356237309
1.2.5. (b)
(11.0010010001) 2 = 3.14159265358979- 3.14160156250000

= -8.908910209992627e-006 (absolute error)
8.908910209992627e-006
= 2.835794194964366e-006 (relative error)
3.14159265358979
1.2.13. (b)
1 1 1
+ )+ =
10 3 5
((0.1101) 2 23 + (0.1011) 2 21
) = (0.1110) 2 21
(
(0.1110) 2 21 + (0.1101) 2 22 = (0.1010) 2 20

19
(0.1010) 2 20
30
0.63333333333333- 0.6250000000000= 0.0083333333333 (absolute error)
0.0083333333333
= 0.01315... (relative error)
0.63333333333333
1.3.1.(b)
98350-98000 = 350 (absolute error)
350
= 0.00355871886121 (relative error) (2 significant digits)
98350
1.3.1.(c)
0.000068-0.00006= 8e-006 (absolute error)

8e-006
= 0.117647058 (relative error) (no significant digits)
0.000068
1.3.12
x1new =
x2new
b + b 2 4ac b + b 2 4ac
2c
=
2
2a
b + b 4ac b + b 2 4ac
b b 2 4ac b b 2 4ac
2c
=
=
2
2a
b b 4ac b b 2 4ac
1.3.13. (a)
x 2 1, 000.001x + 1 = 0
b + b 2 4ac
x1 =
= 1000
2a
2c
x2 =
= 0.001
b b 2 4ac
Programming Exercise 1
% script M-file findroots.m
a=input(Enter the value of "a" from ax^2+bx+c=0 :);
b=input(Enter the value of "b" from ax^2+bx+c=0 :);
c=input(Enter the value of "c" from ax^2+bx+c=0 :);
if b>= 0;
sign=1;
else sign=-1;
end;
q=-0.5*(b+sign*sqrt((b^2)-4*a*c));
x1=q/a;
x2=c/q;
xx1=num2str(x1);
xx2=num2str(x2);
disp([X1 is equal to , xx1])
disp([X2 is equal to ,xx2])
Programming Exercise 2
x(1)=1/2;r(1)=0.994; p(1)=1; p(2)=0.497; q(1)=1; q(2)=0.497;
for n=2:11
x(n)=(1/2)*x(n-1);
end
for n=2:11
r(n)=(1/2)*(r(n-1));
end
for n=3:11
p(n)=(3/2)*p(n-1)-(1/2)*p(n-2);
end
for n=3:11
q(n)=(5/2)*q(n-1)-q(n-2);
end
h=1:11;
figure(1)
plot(h, x(h)-r(h),bd,h, x(h)-p(h),r+,h, x(h)-q(h),g)
grid on
legend(r(n),p(n),q(n))
fprintf(n
x(n)
r(n)
p(n)
q(n)\n)
for i = h
fprintf(%2d %+10.8f %+10.8f %+10.8f %+10.8f\n, i, x(i), p(i), r(i), q(i))
end
fprintf( n x(n)-r(n) x(n)-p(n)
x(n)-q(n)\n)
for i = h
fprintf(%2d %+10.8f %+10.8f %+10.8f\n, i, x(i)-r(i), x(i)-p(i), x(i)-q(i))
end
n
1
2
3
4
5
6
7
8
9
10
11
x(n)
+0.50000000
+0.25000000
+0.12500000
+0.06250000
+0.03125000
+0.01562500
+0.00781250
+0.00390625
+0.00195313
+0.00097656
+0.00048828
r(n)
p(n)
+1.00000000 +0.99400000
+0.49700000 +0.49700000
+0.24550000 +0.24850000
+0.11975000 +0.12425000
+0.05687500 +0.06212500
+0.02543750 +0.03106250
+0.00971875 +0.01553125
+0.00185938 +0.00776563
-0.00207031 +0.00388281
-0.00403516 +0.00194141
-0.00501758 +0.00097070
n
1
2
3
4
5
6
7
8
9
10
11
>>
x(n)-r(n)
-0.49400000
-0.24700000
-0.12350000
-0.06175000
-0.03087500
-0.01543750
-0.00771875
-0.00385938
-0.00192969
-0.00096484
-0.00048242
x(n)-p(n)
-0.50000000
-0.24700000
-0.12050000
-0.05725000
-0.02562500
-0.00981250
-0.00190625
+0.00204687
+0.00402344
+0.00501172
+0.00550586
q(n)
+1.00000000
+0.49700000
+0.24250000
+0.10925000
+0.03062500
-0.03268750
-0.11234375
-0.24817188
-0.50808594
-1.02204297
-2.04702148
x(n)-q(n)
-0.50000000
-0.24700000
-0.11750000
-0.04675000
+0.00062500
+0.04831250
+0.12015625
+0.25207813
+0.51003906
+1.02301953
+2.04750977
Amendment to Problem Set 4
The previous equation has been changed to

be:
1
0 x1
1
x
2
= e

1
e
1
2 e
x
3
1.
=
0
1
0
x1
1
1 1 1
x = 1
1

2 1
x3
1
x1 =
3
x2 = 2
x3 = 6
1
0
x1

1
0
1 0 1
x = 0
1

2 0 x3

0
x1
= 2
x2
=
1
x3 = 2
2.
clear
clc
alfa=100;
A=[exp(-alfa) 1 0; -1 exp(-alfa) -1; 1 -2
exp(-alfa)];
%A=[1 exp(-alfa) 0; exp(-alfa) -1 -1; -2 1
exp(-alfa)];
oA=A;
B=[1; exp(-alfa); exp(-alfa)];
oB=B;
[mm,n]=size(A);
A=[A B];
L=zeros(mm,n);
for i=1:mm-1
for j=i+1:mm
m(j,i)=A(j,i)/A(i,i);
L(j,i)=m(j,i);
for k=i:n+1
A(j,k)=A(j,k)-m(j,i)*A(i,k);
end
end
end
U=A(:,1:n);
L=L+eye(mm,n);
B=A(:,n+1);
x=zeros(n,1);
for j=mm:-1:1
x(j)=(B(j)-A(j,j+1:n)*x(j+1:n))/A(j,j);
end
x
oB-oA*x
3.
=0
x1
= 3
x2
= 2
x
3
= 6
B Ax = 0 The solutions are exactly correct.
= 5
x1
= 1
.9933
x2
= 0
.9866
x
3
= 1
.9934
B Ax 0 Solutions are very good
approximation.
= 10
x1 = 2
x2
= 0
.9999
x
3
= 2

approximation
= 20
x1
= 2
x2
= 1
x
3
= 2

approximation
= 40
x1
= 0
x2
= 1
x
3 = 0
Ax =
0
Big errors exist. Without

pivoting, its not stable now.
4.
clear
clc
alfa=40;
%A=[1 exp(-alfa) 0; exp(-alfa) -1 -1; -2 1
exp(-alfa)];
A=[exp(-alfa) 1 0; -1 exp(-alfa) -1; 1 -2
exp(-alfa)]; %original one.
oA=A;
B=[1; exp(-alfa); exp(-alfa)];
oB=B;
[mm,n]=size(A);
A=[A B];
L=zeros(mm,n);
for i=1:mm-1
%pivoting begins from here.
[Y I]=max(abs(A(i:mm,i)));
temp_store1=A(I,:);
temp_store2=B(I);
A(I,:)=A(i,:);
B(I)=B(i);
A(i,:)=temp_store1;
B(i)=temp_store2;
%pivoting ends from here.
for j=i+1:mm
m(j,i)=A(j,i)/A(i,i);
L(j,i)=m(j,i);
for k=i:n+1
A(j,k)=A(j,k)-m(j,i)*A(i,k);
end
end
end
U=A(:,1:n);
L=L+eye(mm,n);
B=A(:,n+1);
x=zeros(n,1);
for j=mm:-1:1
x(j)=(B(j)-A(j,j+1:n)*x(j+1:n))/A(j,j);
end
x
oB-oA*x
5. switch x1 and x2
13.002
Introduction to Numerical Methods for Engineers
Take Home Exam
Issued: Thursday, Mar. 10, 2005
Due: Friday, Mar. 18, 2005
Problem 1.
Problem 2.
13.002J - Introduction to Numerical Analysis for Engineers

Class Survey - Spring 2005
Name:
Course:
E-mail:
What programming subjects have you taken at MIT?
1.00
6.001
10.001
Self-Assessment Quiz (no grade)
A. Please circle the number representing your level of knowledge and under
standing in each of the following areas (0: no knowledge, 10: Expert):
Linear algebra
Dierentiation
Integration
Ordinary Dierential equations
Fortran
C
C++
Java
Scheme
Matlab
Other prog. language (specify)
0
0
0
0
0
0
0
0
0
0
0
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
4
5
5
5
5
5
5
5
5
5
5
5
6
6
6
6
6
6
6
6
6
6
6
7
7
7
7
7
7
7
7
7
7
7
8
8
8
8
8
8
8
8
8
8
8
9
9
9
9
9
9
9
9
9
9
9
10
10
10
10
10
10
10
10
10
10
10
B. Create a Matlab script that computes and plots the following two repre
sentations of the sine function for small arguments:
f (x) = sin x
f (x) =
(1)
1 107 round(107 cos2 x)
(2)
for x = 10n/100 , n = 700, , 300. The function round will round the
argument to the nearest integer. Describe what you think is going on.
6

2993 Spring 2005

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

2993 Spring 2005

Transféré par

Droits d'auteur :

Formats disponibles

Introduction to Numerical Analysis for Engineers

Fundamentals of Digital Computing

Digital Computer Models

Error propagation numerical stability

Numerical Methods for Engineers

Digital Computer Models

Accuracy and Stability => Convergence

Numerical Methods for Engineers

Floating Number Representation

Addition and Subtraction

Result has exponent of largest number

Multiplication and Division

Numerical Methods for Engineers

Numerically evaluate square-root

Mean of guess and its reciprocal

Numerical Methods for Engineers

Numerical Methods for Engineers

Result of small, but significant

N=20; sum=0; sumr=0;

Numerical Methods for Engineers

Numerical Methods for Engineers

Introduction to Numerical Analysis for

Digital Computer Models

Error propagation numerical stability

Numerical Methods for Engineers

Floating Number Representation

Addition and Subtraction

Result has exponent of largest number

Multiplication and Division

Numerical Methods for Engineers

Numerical Methods for Engineers

Numerical Methods for Engineers

Central Finite Difference

Numerical Methods for Engineers

Function of one variable

General Error Propagation Formula

Numerical Methods for Engineers

Relative Errors Add for Multiplication

Numerical Methods for Engineers

Standard Error better measure

Numerical Methods for Engineers

Numerical Methods for Engineers

Problem Condition Number

Numerical Methods for Engineers

K A is algorith condition number, which

Algorithm Condition Number

Numerical Methods for Engineers

Introduction to Numerical Analysis for Engineers

Multiple right hand sides

Numerical Methods for Engineers

Systems of Linear Equations

Cramers Rule, n=2

Cramers rule inconvenient for n>3

Numerical Methods for Engineers

Systems of Linear Equations

Linear System of Equations

Numerical Methods for Engineers

Systems of Linear Equations

Numerical Methods for Engineers

Systems of Linear Equations

Numerical Methods for Engineers

Systems of Linear Equations