Week2 6

Outline
• Performance of Algorithms
• Performance Prediction (Order of Algorithms)
Data Structures – Week #2 • Examples
• Exercises
Algorithm Analysis • Recursion
& • Recurrences
Recursion
November 11, 2005 Borahan Tümer, Ph.D. 2
Performance of Algorithms Performance Assessment

• Algorithm is a finite sequence of instructions that the • The actual execution time of an algorithm is hard to assess if
computer follows to solve a problem. one does not know the intimate details of the computer
• Algorithms solving the same problem may perform differently. architecture, the operating system, the compiler, the quality of
Depending on resource requirements an algorithm may be the program and the other factors.
feasible or not. To find out whether or not an algorithm is
usable or relatively better than another one solving the same • Execution time may be found for a given algorithm using
problem, its resource requirements should be determined. some special performance programs called benchmarks.
• The process of determining the resources of an algorithm is • Second alternative for performance assessment is to find the
called algorithm analysis. growth rate of an algorithm.
• Two essential resources, hence, performance criteria of • The execution time is a function of the input size such as the
algorithms are number of elements in an array, the number of records in a file
– execution time etc...
– memory space used.
November 11, 2005 Borahan Tümer, Ph.D. 3 November 11, 2005 Borahan Tümer, Ph.D. 4
Assessment Tools “Big O” Notation by words

• We can use the concept “the growth rate or order of • Expressed by words; O(f(n)) is the set of all functions
an algorithm” to assess both criteria. However, our t(n) mapping (->) integers (N) to nonnegative real
main concern will be the execution time. numbers (R*) such that (|) there exists a positive real
• Notation for the “order of” concept constant c (c R+) and there exists an integer
• Mathematically expressed, the “order of” or “Big O” constant n0 (n0 N) such that for all values of n
(O()) concept is as follows: greater than or equal to the constant (n t n0), the
function values of t(n) are less than or equal to the
• Let f: N->R* be an arbitrary function. function values of f(n) multiplied by the constant c
• O(f(n)) = {t: N->R* | (c R+)(n0 N)(n t n0) (t(n)d cf(n)).
[t(n)d cf(n)]}, • In other words, O(f(n)) is the set of all functions t(n)
– where R* is the set of nonnegative real numbers and R+ is bounded above by a positive real multiple of f(n),
the set of strictly positive real numbers (excluding 0). provided n is sufficiently large (greater than n0).
Execution time of various structures Execution time of various structures
• Simple Statement • O(f1(n)+}+fs(n))=O(max(f1(n),},fs(n))) ???
O(1), executed within a constant amount of time
• Proof:
irresponsive to any change in input size.
• Decision (if) structure t(n) O(f1(n)+}+fs(n)) t(n)d
if (condition) f(n) else g(n)
c[f1(n)+…+fs(n)] d
O(if structure)=max(O(f(n)),O(g(n)) sc*max [f1(n),…, fs(n)],sc another constant.
• Sequence of Simple Statements t(n) O(max(f1(n),},fs(n)))
O(1), since O(f1(n)+}+fs(n))=O(max(f1(n),},fs(n))) Hence, hypothesis follows.
Execution Time of Loop Structures Examples

• Loop structures’ execution time depends upon Find the execution time t(n) in terms of n!
whether or not their index bounds are related for (i=0; i<=n; i++)
to the input size. for (j=0; j<=n; j++)
statement block;
• Assume n is the number of input records
for (i=0; i<=n; i++)
• for (i=0; i<=n; i++) {statement block}, O(?) for (j=0; j<=i; j++)
• for (i=0; i<=m; i++) {statement block}, O(?) statement block;
for (i=0; i<=n; i++)

for (j=0; j<=n; j*=2)
statement block;
Exercises Recursion
Find the number of times the statement block is executed! Definition:
for (i=0; i<=n; i++)
for (j=0; j<=i; j*=2)
Recursion is a mathematical concept referring to
statement block; programs or functions calling or using itself.
for (i=0; i<=n; i*3)

for (j=0; j<=n; j*=2)
A recursive function is a functional piece of code
statement block; that invokes or calls itself.
Recursion Recursion… cont’d
Concept: • Base case: the simplest version of the problem
• A recursive function divides the problem into that is not further reducable. The function
two conceptual pieces: actually knows how to solve this version of the
– a piece that the function knows how to solve (base problem.
case),
– a piece that is very similar to the original problem, • To make the recursion feasible, the latter piece
hence still unknown how to solve by the function must be slightly simpler.
(call(s) of the function to itself).
Recursion Examples Towers of Hanoi… cont’d

• Towers of Hanoi Algorithm:
Hanoi(n,i,j)
• Story: According to the legend, the life on the // moves n smallest rings from rod i to rod j
world will end when priests in a Far-Eastern F0A0 if (n > 0) {
temple move 64 disks stacked on a peg in an //moves top n-1 rings to intermediary rod (6-i-j)
F0A2 Hanoi(n-1,i,6-i-j);
decreasing order in size to another peg. They
//moves the bottom (nth largest) ring to rod j
are allowed to move one stack at a time and a F0A5 move i to j
larger disk can never be placed over a smaller // moves n-1 rings at rod 6-i-j to destination rod j
one. F0A8 Hanoi(n-1,6-i-j,j);
F0AB }
Function Invocation in MM Function Invocation (Call) in MM

• Code and data are both in MM.
• Hanoi function is called by the instruction at
MM cell C0D5 with arguments (4,1,3).
• Program counter is a register in ȝP that holds
MM address of next instruction to execute.
• If current instruction is a function call, the
serial flow of execution is interrupted.
Function Call in MM… cont’d Towers of Hanoi… cont’d
Example: Hanoi(4,i,j)
413
312 323
• Following problems arise: 213 221
123
112
– how to keep the return address from the function called 013 021
(Hanoi) back to the caller function (C0D8 at main and 1o2

032
2o3
013
both F0A5 and F0AB at Hanoi); 1o3 2o1
123 131
– how to store the values of variables local to caller function. 021 032
2o3 3o1
• Both problems are solved by keeping the return 013 021
1o2 2o3
address and local variables’ values in a portion of the 232 213
main memory called system stack. 131

032
112
013
3o1 1o2
• Another register called Stack Pointer points to the 021 032
address pushed most recently to system stack. Return 3o2

112
1o3
123
addresses are retrieved from system stack in a last-in- 013
1o2
021
2o3
first-out (LIFO) fashion. We will see stacks later. 032 013
1o3
Towers of Hanoi… cont’d Recursion Examples

o o o o o
• Fibonacci Series
– tn= tn-1 + tn-2; t0=0; t1=1
• Algorithm
o o o o o o
long int fib(n)
{
if (n==0 || n==1)
o o o o o o
return n;
else
return fib(n-1)+fib(n-2);
o
}
Fibonacci Series… cont’d Fibonacci Series… cont’d

• Tree of recursive • Redundant function calls slow the execution
function calls for fib(5) down.
• Any problems???
• A lookup table used to store the fibonacci

values already computed saves redundant
function executions and speeds up the process.
• Homework: Write fib(n) with a lookup table!

Recurrences or Difference Equations Homogeneous Recurrences
• Homogeneous Recurrences We are looking for solutions of the form:
• Consider a0 tn + a1tn-1 + … + ak tn-k = 0. tn = xn
• The recurrence Then, we can write the recurrence as
– contains ti values which we are looking for. a0 xn + a1xn-1+ … + ak xn-k = 0
– is a linear recurrence (i.e., ti values appear alone, no th
• This k degree equation is the characteristic
powered values, divisions or products)
equation (CE) of the recurrence.
– contains constant coefficients (i.e., ai).
– is homogeneous (i.e., RHS of equation is 0).
Homogeneous Recurrences Inhomogeneous Recurrences

If ri, i=1,…, k, are k distinct roots of a0 xk + a1 xk-1+ … + Consider
ak = 0, then
k
• a0 tn + a1tn-1 + … + ak tn-k = bn p(n)
tn ¦c r
i 1
i i
n
• where b is a constant; and p(n) is a polynomial
in n of degree d.
If ri, i=1,…, k, is a single root of multiplicity k, then
k
tn ¦c n
i 1
i
i 1 n
r
Inhomogeneous Recurrences Generalized Solution for Recurrences

Generalized Solution for Recurrences
Consider a general equation of the form If ri, i=1,…, k, are k distinct roots of (a0 xk + a1 xk-1+
(a0 tn + a1tn-1 + … + ak tn-k ) = b1n p1(n) + b2n p2(n) + … + ak)=0
…
We are looking for solutions of the form:
k
tn = xn tn ¦c r
i 1
i i
n
ck 1b1n ck 2 nb1n / ck 1 d1 n d1 1b1n ck 2 d1 b2n ck 3 d1 nb2n / ck 2 d1 d 2 n d 2 1b2n
Then, we can write the recurrence as
(a0 xk + a1 xk-1+ … + ak ) (x-b1) d1+1 (x-b2) d2+1 …= 0;
where di is the polynomial degree of polynomial pi(n).
This is the characteristic equation (CE) of the recurrence.
Examples Examples
Homogeneous Recurrences Homogeneous Recurrence
Example 1. Example 2.
tn + 5tn-1 + 4 tn-2 = 0; sol’ns of the form tn = xn
tn-6 tn-1+12tn-2-8tn-3=0; tn = xn
xn + 5xn-1+ 4xn-2 = 0; n-2 trivial sol’ns
xn-6xn-1+12xn-2-8xn-3= 0; n-3 trivial sol’ns
(x2+5x+4) = 0; characteristic equation (CE)
x1=-1; x2=-4; nontrivial sol’ns
CE: (x3-6x2+12x-8) = (x-2)3= 0; by polynomial division
tn = c1(-1)n+ c2(-4)n ; general sol’n x1= x2= x3 = 2; roots not distinct!!!
tn = c12n+ c2n2n + c3n22n; general sol’n
Examples Examples
Homogeneous Recurrence Example 3… cont’d
Example 3. We use as many ti values
tn = tn-1+ tn-2; Fibonacci Series
as ci
xn-xn-1-xn-2 = 0; CE: x2-x-1 = 0; 0 0
§1 5 · §1 5 ·
1 r 5 ; distinct roots!!! t0 0 c1 ¨¨ ¸ c2 ¨
¸ ¨ 2 ¸
¸ c1 c2 0 c1 c2
x1, 2 © 2 ¹ © ¹
2 n n 1 1
§1 5 · §1 5 · t1
§1 5 ·
1 c1 ¨¨
§1 5 ·
¸ c2 ¨ ¸
§1 5 ·
c1 ¨¨
§1 5 ·
¸ c1 ¨ ¸
¨ 2 ¸ c1
1
, c2
1
tn c1 ¨¨ ¸ c2 ¨
¸ ¨ 2 ¸
¸ © 2 ¹
¸ ¨ 2 ¸
© ¹ © 2 ¹
¸
© ¹ 5 5
© 2 ¹ ; general sol’n!!
© ¹
We find coefficients ci using initial values t0 and t1 of Check it out using t2!!!
Fibonacci series on the next slide!!! n n
1 §1 5 · 1 §1 5 ·
tn ¨ ¸ ¨ ¸
5 ¨© 2 ¸¹ 5 ¨© 2 ¸¹
Examples Examples
Example 3… cont’d Example 4.
tn = 2tn-1 - 2tn-2; n t 2; t0 = 0; t1 = 1;
What do n and tn represent? CE: x2-2x+2 = 0;
Complex roots: x1,2=1ri
n is the location and tn the value of any Fibonacci number in the series.
What is the meaning of tn in recursive function As in differential equations, we represent the complex roots as a
fib(n) on page 22 ? vector in polar coordinates by a combination of a real radius r
and a complex argument T:
tn is the number of times the condition in the if structure (chosen as a barometer) is checked in fib(n).
z=r*eT i;
Here,
1+i=2 * e(S/4)i
1-i=2 * e(-S/4)i
Examples Examples
Example 4… cont’d Inhomogeneous Recurrences
Solution: Example 1. (From Example 3)
tn = c1 (2)n/2 e(nS/4)i + c2 (2)n/2 e(-nS/4)i; We would like to know how many times fib(n)
From initial values t0 = 0, t1 = 1, on page 22 is executed in terms of n. To find out:
tn = 2n/2 sin(nS/4); (prove that!!!) 1. choose a barometer in fib(n);
2. devise a formula to count up the number of
times the barometer is excuted.
Examples Examples
Example 1… cont’d Example 1… cont’d
In fib(n), the only statement is the if statement. tn - tn-1 - tn-2 = 1; inhomogeneous recurrence
Hence, if condition is chosen as the barometer. The homogeneous part comes directly from
Suppose fib(n) takes tn time units to execute, Fibonacci Series example on page 33.
where the barometer takes one time unit and the RHS of recurrence is 1 which can be expressed
function calls fib(n-1) and fib(n-2), tn-1 and tn-2, as 1nx0. Then, from the equation on page 29,
respectively. Hence, the recurrence to solve is CE: (x2-x-1)(x-1) = 0; from page 30,
n n
§1 5 · §1 5 ·
tn = tn-1 + tn-2 + 1 tn c1 ¨¨ ¸ c2 ¨
¸
¸
¨ 2 ¸ c31
n
© 2 ¹ © ¹
Examples Examples
Example 1… cont’d Example 1… cont’d
n n
§1 5 · §1 5 · n
§ 1 5 · §1 5 ·
n
tn c1 ¨¨ ¸ c2 ¨
¸
¸
¨ 2 ¸ c3
tn c1 ¨¨ ¸ c2 ¨
¸
¸
¨ 2 ¸ c3 ; t0 t1 1, t 2 3
© 2 ¹ © ¹
© 2 ¹ © ¹
5 1 5 1
Now, we have to find c1,…,c3. c1 ; c2 ; c3 1
5 5
n n
Initial values: for both n=0 and n=1, if condition is ª 5 1º § 1 5 · ª 5 1º § 1 5 ·
tn « »¨¨ ¸ «
¸ »¨¨ ¸ 1;
¸
¬ 5 ¼© 2 ¹ ¬ 5 ¼© 2 ¹
checked once and no recursive calls are done.
For n=2, if condition is checked once and recursive Here, tn provides the number of times the
calls fib(1) and fib(0) are done. barometer is executed in terms of n. Practically,
this number also gives the number of times fib(n) is
t0 = t1 = 1 and t2 = t0 + t1 + 1 = 3.
called.

Week2 6

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Week2 6

Transféré par

Droits d'auteur :

Formats disponibles

Outline

November 11, 2005 Borahan Tümer, Ph.D. 2

Performance of Algorithms Performance Assessment

Assessment Tools “Big O” Notation by words

Execution Time of Loop Structures Examples

for (i=0; i<=n; i++)

for (i=0; i<=n; i*3)

Recursion Examples Towers of Hanoi… cont’d

Function Invocation in MM Function Invocation (Call) in MM

(Hanoi) back to the caller function (C0D8 at main and 1o2

main memory called system stack. 131

address pushed most recently to system stack. Return 3o2

Towers of Hanoi… cont’d Recursion Examples

Fibonacci Series… cont’d Fibonacci Series… cont’d

• A lookup table used to store the fibonacci

• Homework: Write fib(n) with a lookup table!

Homogeneous Recurrences Inhomogeneous Recurrences

Inhomogeneous Recurrences Generalized Solution for Recurrences

Vous aimerez peut-être aussi