Vous êtes sur la page 1sur 35

# Algorithms and Data Structures

## SE240 - Spring 2009

Lecture 3
Chapter 3, 4.1, 4.2, 4.3
Dept. of Electrical and Computer Engineering
University of Waterloo
Asymptotic - Overview
 A way to describe behaviour of functions in the limit.

##  Focus on what’s important by abstracting away low-

order terms and constant factors.

##  A way to compare “sizes” of functions.

May 12, 2009 SE240 2
Asymptotic Notation

 Ο − notation ⎯
⎯→ Ο ≈≤ (big-oh, upper-bound)

 Ω − notation ⎯
⎯→ Ω ≈≥ (big-omega, lower bound)

 Θ − notation ⎯
⎯→ Θ ≈= (theta, tight bound)

 ο − notation ⎯
⎯→ ο ≈p (little-oh, upper-bound not tight)

 ω − notation ⎯
⎯→ ω ≈f (little-omega, lower-bound but not tight)

## May 12, 2009 SE240 3

Ο − Notation
Ο(g(n)) = { f (n) : there exist positive constants c and n0
such that 0 ≤ f (n) ≤ cg(n) for all n ≥ n0 }

Example:
2 n = Ο( n )
2 3

with c = 1, n0 = 2

## May 12, 2009 SE240 4

Ω − Notation
Ω (g(n)) = { f (n) : there exist positive constants c and n0
such that 0 ≤ cg(n) ≤ f (n) for all n ≥ n0 }

Example:

n = Ω(log n)
with c = 1, n0 = 16

## May 12, 2009 SE240 5

Θ − Notation
Θ(g(n)) = { f (n) : there exist positive constants c1, c2, and
n0 such that 0 ≤ c1 g(n) ≤ f (n) ≤ c2 g(n)
for all n ≥ n0 }
Example:
1 2
n − 3n = Θ(n 2 )
2
1 1
with c1 = , c2 = , n0 = 7
14 2
g(n) is an asymptotic tight bound for f(n)

Recurrences

## A recurrence is an equation or inequality

that describes a function in terms of its
value on smaller inputs.

Merge Sort
Θ(1) if n = 1;
T(n) =
2T(n/2) + Θ(n) if n > 1.

## May 12, 2009 SE240 7

Methods for Solving Recurrences

 Recursion-Tree Method

 Master Method

 Substitution Method

## May 12, 2009 SE240 8

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:

## May 12, 2009 SE240 9

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
T(n)

## May 12, 2009 SE240 10

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
T(n/4) T(n/2)

## May 12, 2009 SE240 11

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

## May 12, 2009 SE240 12

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2

Θ(1)

## May 12, 2009 SE240 13

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
(n/4)2 (n/2)2

Θ(1)

## May 12, 2009 SE240 14

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
5 n2
(n/4)2 (n/2)2
16
(n/16)2 (n/8)2 (n/8)2 (n/4)2

Θ(1)

## May 12, 2009 SE240 15

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
5 n2
(n/4)2 (n/2)2
16
25 n 2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
256

Θ(1)

## May 12, 2009 SE240 16

Example of Recursion Tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
5 n2
(n/4)2 (n/2)2
16
25 n 2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
256

Θ(1) Total = n
2
( ( ) + ( ) +L
1 + 16 + 16
5 5 2 5 3
16
)
= Θ(n2) Geometric Series
May 12, 2009 SE240 17
Appendix A: Geometric Series

n +1
1 − x
1 + x + x2 + L + xn = for x ≠ 1
1− x

1
1+ x + x +L =
2
for |x| < 1
1− x

Master Method

## The master method applies to recurrences of

the form
T(n) = a T(n/b) + f (n) ,
where a ≥ 1, b > 1, and f is asymptotically
positive.

## May 12, 2009 SE240 19

Idea of Master Theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

#leaves = ah
= alogbn nlogbaΤ (1)
Τ (1)
= nlogba
May 12, 2009 SE240 20
Master Method Intuition

## Three common cases:

¾ Running time dominated by cost at leaves
¾ Running time evenly distributed throughout the tree
¾ Running time dominated by cost at the root

## Consequently, to solve the recurrence, we need

only to characterize the dominant term

)

## May 12, 2009 SE240 21

Three Common Cases

## 1. f (n) = O(nlogba – ε) for some constant ε > 0.

• f (n) grows polynomially slower than nlogba
(by an nε factor).
Solution: T(n) = Θ(nlogba) .
Running time dominated by cost at leaves

## May 12, 2009 SE240 22

Idea of Master Theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

C ASE 1:
CASE 1: The
The weight
weight increases
increases
geometrically
geometrically from
from the
the root
root to the nlogbaΤ (1)
to the
Τ (1) leaves.
leaves. The
The leaves
leaves hold
hold aa constant
constant
fraction
fraction of
of the
the total
total weight.
weight. Θ(nlogba)
May 12, 2009 SE240 23
Three Common Cases (cont.)

## Compare f (n) with nlogba:

2. f (n) = Θ(nlogba ).
• f (n) and nlogba grow at similar rates.
Solution: T(n) = Θ(nlogba lgn) .

## May 12, 2009 SE240 24

Idea of Master Theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

C ASE 2:
CASE 2: The
The weight isis
weight nlogbaΤ (1)
Τ (1) approximately
approximately the the same
same on
on
each
each of
of the
the log
logbbnn levels.
levels.
Θ(nlogbalg n)
May 12, 2009 SE240 25
Three Common Cases (cont.)
Compare f (n) with nlogba:
3. f (n) = Ω(nlogba + ε) for some constant ε > 0.
• f (n) grows polynomially faster than nlogba (by
an nε factor),
and f (n) satisfies the regularity condition that
a f (n/b) ≤ c f (n) for some constant c < 1.
Solution: T(n) = Θ( f (n)) .
Running time dominated by cost at the root
May 12, 2009 SE240 26
Idea of Master Theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

C ASE 3:
CASE 3: The
The weight
weight decreases
decreases
geometrically
geometrically from
from the
the root
root to the nlogbaΤ (1)
to the
Τ (1) leaves.
leaves. The
The root
root holds
holds aa constant
constant
fraction
fraction of
of the
the total
total weight.
weight. Θ( f (n))
May 12, 2009 SE240 27
Master Method Strategy

## 1. Extract a, b, and f(n) from a given recurrence

logb a
2. Determine n
logb a
3. Compare f(n) and n asymptotically

and apply it

## May 12, 2009 SE240 28

Examples

 T(n) = 4T(n/2) + n
a = 4, b = 2 ⇒ nlogba = n2; f (n) = n.
CASE 1: f (n) = O(n2 – ε) for ε = 1.
∴ T(n) = Θ(n2).

 T(n) = 4T(n/2) + n2
a = 4, b = 2 ⇒ nlogba = n2; f (n) = n2.
CASE 2: f (n) = Θ(n2).
∴ T(n) = Θ(n2lg n).
May 12, 2009 SE240 29
Examples

 T(n) = 4T(n/2) + n3
a = 4, b = 2 ⇒ nlogba = n2; f (n) = n3.
CASE 3: f (n) = Ω(n2 + ε) for ε = 1
and 4(n/2)3 ≤ cn3 (reg. cond.) for c = 1/2.
∴ T(n) = Θ(n3).

## May 12, 2009 SE240 30

Substitution Method

## 1. Guess the form of the solution.

2. Verify by induction.
3. Solve for constants.
Example: T(n) = 4T(n/2) + n
 [Assume that T(1) = Θ(1).]
 Guess O(n3) . (Prove O and Ω separately.)
 Assume that T(k) ≤ ck3 for k < n .
 Prove T(n) ≤ cn3 by induction.
May 12, 2009 SE240 31
Example of Substitution
T (n) = 4T ( n / 2) + n
≤ 4c ( n / 2) 3 + n
= (c / 2) n3 + n
= cn3 − ((c / 2) n3 −n ) desired – residual
≤ cn 3 desired
whenever (c/2)n3 – n ≥ 0, for example,
if c ≥ 2 and n ≥ 1.
residual

## May 12, 2009 SE240 32

Example (cont.)
 We must also handle the initial
conditions, that is, ground the induction
with base cases.
 Base: T(n) = Θ(1) for all n < n0, where n0
is a suitable constant.
 For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if
we pick c big enough.

## This bound is not tight!

May 12, 2009 SE240 33
A Tighter Upper Bound?
We shall prove that T(n) = O(n2).
Assume that T(k) ≤ ck2 for k < n:
T ( n) = 4T ( n / 2) + n
≤ cn 2 + n
≤ cn2

## May 12, 2009 SE240 34

A Tighter Upper Bound
IDEA: Strengthen the inductive hypothesis.
 Subtract a low-order term.
Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n.
T (n) = 4T ( n / 2) + n
≤ 4(c1 (n / 2) 2 − c2 (n / 2))+ n
= c1n 2 − 2c2 n + n
= c1n 2 − c2 n − (c2 n − n)
≤ c1n 2 − c2 n if c > 1.
2
Pick c1 big enough to handle the initial conditions.
May 12, 2009 SE240 35