Vous êtes sur la page 1sur 11

Design and Analysis of Algorithms

Chapter 1

RAIK 283: Data Structures & Algorithms

RAIK 283: Data Structures & Algorithms

Analysis of Algorithms

b Giving credit

Dr. Ying Lu
ylu@cse.unl.edu
August 28, 2012

http://www.cse.unl.edu/~ylu/raik283/

Design and Analysis of Algorithms Chapter 2.1

where credit is due:

Most of the lecture notes are based on the slides


from the Textbooks companion website
http://www.aw--bc.com/info/levitin
http://www.aw
Several slides are from Jeff Edmonds of the
York University
I have modified them and added new slides

Design and Analysis of Algorithms Chapter 2.1

The Time Complexity of an Algorithm

The Time Complexity of an Algorithm

Specifies how the running time depends


on the size of the input

Purpose

Purpose
To estimate how long a program will run.
To estimate the largest input that can reasonably be
given to the program.
b To compare the efficiency of different algorithms.
algorithms.
b To help focus on the parts of code that are executed
the largest number of times.
b To choose an algorithm for an application.
b
b

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms


Purpose (Example)
b

Chapter 1
Purpose (Example)

Suppose a machine that performs a million floatingfloatingpoint operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?

Suppose a machine that performs a million floatingfloatingpoint operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?
1) If the algorithm requires n2 such operations:

1) If the algorithm requires n2 such operations:

0.0025 second

Design and Analysis of Algorithms Chapter 2.1

Purpose (Example)
b

Design and Analysis of Algorithms Chapter 2.1

Purpose (Example)

Suppose a machine that performs a million floatingfloatingpoint operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?

Suppose a machine that performs a million floatingfloatingpoint operations per second (106 FLOPS), then how
long an algorithm will run for an input of size n=50?

1) If the algorithm requires n2 such operations:

1) If the algorithm requires n2 such operations:

0.0025 second
2) If the algorithm requires 2n such operations:

2) If the algorithm requires 2n such operations:

0.0025 second

A) Takes a similar amount of time (t < 1 sec)


B) Takes a little bit longer time (1 sec < t < 1 year)
C) Takes a much longer time (1 year < t)

Design and Analysis of Algorithms Chapter 2.1

Time Complexity Is a Function

over 35 years!

Design and Analysis of Algorithms Chapter 2.1

10

Definition of Time?

Specifies how the running time depends on the size of the


input.
A function mapping
size of input

time T(n) executed .

Design and Analysis of Algorithms Chapter 2.1

11

Design and Analysis of Algorithms Chapter 2.1

12

Design and Analysis of Algorithms


Definition of Time

Chapter 1
Theoretical analysis of time efficiency

# of seconds (machine, implementation dependent).

Time efficiency is analyzed by determining the number of


repetitions of the basic operation as a function of input size

# lines of code executed.

# of times a specific operation is performed


addition).

Basic operation: the operation that contributes most


towards the running time of the algorithm.

(e.g.,

input size

T(n) copC(n)
running time

13

Design and Analysis of Algorithms Chapter 2.1

execution time
for basic operation

Number of times
basic operation is
executed
14

Design and Analysis of Algorithms Chapter 2.1

Input size and basic operation examples

Input size and basic operation examples

Problem

Problem

Input size measure

Basic operation

Input size measure

Search for key in a list of


n items

Search for key in a list of Number of items in the


n items
list: n

Multiply two matrices of


floating point numbers

Multiply two matrices of


floating point numbers

Compute an

Compute an

Graph problem

Graph problem
15

Design and Analysis of Algorithms Chapter 2.1

Basic operation
Key comparison

16

Design and Analysis of Algorithms Chapter 2.1

Input size and basic operation examples

Input size and basic operation examples

Problem

Basic operation

Problem

Search for key in a list of Number of items in the


n items
list: n

Key comparison

Search for key in list of n


Number of items in list n Key comparison
items

Multiply two matrices of


floating point numbers

Floating point
multiplication

Multiply two matrices of


floating point numbers

Dimensions of matrices

Floating point
multiplication

Compute an

Compute an

Floating point
multiplication

Graph problem

Graph problem

Input size measure

Dimensions of matrices

Design and Analysis of Algorithms Chapter 2.1

17

Input size measure

Design and Analysis of Algorithms Chapter 2.1

Basic operation

18

Design and Analysis of Algorithms


Input size and basic operation examples
Problem

Input size measure

Chapter 1
Theoretical analysis of time efficiency

Basic operation

Time efficiency is analyzed by determining the


number of repetitions of the basic operation as a
function of input size

Search for key in list of n


Number of items in list n Key comparison
items
Multiply two matrices of
floating point numbers

Dimensions of matrices

Floating point
multiplication

Compute an

Floating point
multiplication

Graph problem

#vertices and/or edges

Visiting a vertex or
traversing an edge

Design and Analysis of Algorithms Chapter 2.1

19

Which Input of size n?

Design and Analysis of Algorithms Chapter 2.1

Which Input of size n?

Efficiency also depends on the particular input


b For

Efficiency also depends on the particular input

instance: search a key in a list of n letters

Problem input: a list of n letters


How many different inputs?

Design and Analysis of Algorithms Chapter 2.1

For instance: search a key in a list of n letters


There are 26n inputs of size n.
Which do we consider
for the time efficiency C(n)?

21

Best--case, averageBest
average-case, worstworst-case

Design and Analysis of Algorithms Chapter 2.1

Worst case: W(n


W(n) maximum over inputs of size n

Best case:

Average case: A(n


A(n) average over inputs of size n

Problem: Given a list of n elements and a search key K, find


an element equal to K, if any.
Algorithm: Scan the list and compare its successive
elements with K until either a matching element is found
(successful search)
search) or the list is exhausted (unsuccessful
(unsuccessful
search))
search
Worst case

Best case

Average case

B(n) minimum over inputs of size n


B(n

NOT the average of worst and best case


Under some assumption about the probability distribution of all
possible inputs of size n, calculate the weighted sum of expected C(n)
(numbers of basic operation repetitions) over all possible inputs of size
n.

Design and Analysis of Algorithms Chapter 2.1

22

Example: Sequential search


b

20

23

Design and Analysis of Algorithms Chapter 2.1

24

Design and Analysis of Algorithms

Chapter 1

An example
b

b
b
b
b

Types of formulas for basic operation count

Compute gcd(m, n) by applying the algorithm based on


checking consecutive integers from min(m, n) down to
gcd(m, n)
Input size?
Best case?
Worst case?
Average case?

Design and Analysis of Algorithms Chapter 2.1

Exact formula
e.g., C(n
C(n) = n(n-1)/2

Formula indicating order of growth with specific


multiplicative constant
e.g., C(n
C(n) 0.5 n2

Formula indicating order of growth with unknown


multiplicative constant
e.g., C(n
C(n) cn2

25

Design and Analysis of Algorithms Chapter 2.1

26

Asymptotic growth rate

Types of formulas for basic operation count


b

Exact formula
e.g., C(n
C(n) = n(n-1)/2
Formula indicating order of growth with specific
multiplicative constant
e.g., C(n
C(n) 0.5 n2
Formula indicating order of growth with unknown
multiplicative constant
e.g., C(n
C(n) cn2

A way of comparing functions that ignores constant factors


and small input sizes

O(g
O(
g(n)):

(g(n)):

(g(n)):

Most important: Order of growth within a constant


multiple as n
Design and Analysis of Algorithms Chapter 2.1

27

Asymptotic growth rate

28

Table 2.1

A way of comparing functions that ignores constant factors


and small input sizes

O(gg(n)): class of functions f(n) that grow no faster than g(n)


O(

(g(n)): class of functions f(n) that grow at same rate as g(n)

(g(n)): class of functions f(n) that grow at least as fast as g(n)

Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

29

Design and Analysis of Algorithms Chapter 2.1

30

Design and Analysis of Algorithms

Chapter 1
Which are more alike?

Classifying Functions

Giving an idea of how fast a function


grows without going into too much detail.

32

Design and Analysis of Algorithms Chapter 2.1

Which are more alike?

Which are more alike?

Mammals

Design and Analysis of Algorithms Chapter 2.1

33

Which are more alike?

34

Design and Analysis of Algorithms Chapter 2.1

Classifying Animals
Vertebrates

Design and Analysis of Algorithms Chapter 2.1

Fish

Reptiles

35

Giraffe

Dogs

Design and Analysis of Algorithms Chapter 2.1

Mammals

Birds

Dogs

36

Design and Analysis of Algorithms


Which are more alike?

n1000

n2

Chapter 1
Which are more alike?

2n

n1000

n2

2n

Polynomials

37

Design and Analysis of Algorithms Chapter 2.1

Which are more alike?

1000n2

3n2

38

Design and Analysis of Algorithms Chapter 2.1

Which are more alike?

2n3

1000n2

3n2

2n3

Quadratic

Design and Analysis of Algorithms Chapter 2.1

39

Classifying Functions?

Classifying Functions
Functions

Functions

Polynomial

Exponential

Factorial

5 log n

Poly Logarithmic

41

Logarithmic

Constant
5
Design and Analysis of Algorithms Chapter 2.1

40

Design and Analysis of Algorithms Chapter 2.1

(log n)5

n5

25n

n!

Design and Analysis of Algorithms Chapter 2.1

42

Design and Analysis of Algorithms

Chapter 1
Classifying Functions

Classifying Functions?
Polynomial

Cubic

Logarithmic

Quadratic

43

Linear

Design and Analysis of Algorithms Chapter 2.1

Polynomial

5n

5n2

5n3

5n4

Design and Analysis of Algorithms Chapter 2.1

44

Poly Logarithmic

b log10n =

# digits to write n
= # bits to write n
= 3.32 log10n
1000
) = 1000 log(n)
b log(n
b log2n

Design and Analysis of Algorithms Chapter 2.1

Differ only by a
multiplicative
constant

45

Which grows faster?

log1000 n

(log n)5 = log5 n

Design and Analysis of Algorithms Chapter 2.1

46

Poly Logarithmic << Polynomial

n0.001

log1000 n << n0.001


For sufficiently large n

Design and Analysis of Algorithms Chapter 2.1

47

Design and Analysis of Algorithms Chapter 2.1

48

Design and Analysis of Algorithms


Which grows faster?

Linear << Quadratic

0.0001 n2

10000 n

Chapter 1

10000 n << 0.0001 n2


For sufficiently large n

49

Design and Analysis of Algorithms Chapter 2.1

Which grows faster?

n1000

Design and Analysis of Algorithms Chapter 2.1

50

Polynomial << Exponential

20.001 n

n1000 << 20.001 n


For sufficiently large n

51

Design and Analysis of Algorithms Chapter 2.1

Ordering Functions

Design and Analysis of Algorithms Chapter 2.1

52

Which Functions are Constant?

Functions

<<

<<

25n <<

Design and Analysis of Algorithms Chapter 2.1

Factorial

5 << 5 log n << (log n)5 << n5

<<

Exponential

<<

Polynomial

<<

Poly Logarithmic

Logarithmic

Constant

<<

5
1,000,000,000,000
0.0000000000001
-5
0
8 + sin(n)

n!
53

Design and Analysis of Algorithms Chapter 2.1

54

Design and Analysis of Algorithms


Which Functions are Constant?

Which Functions are Constant?


The running time of the algorithm is a Constant
if it does not depend significantly
on the size of the input.

Yes 5
Yes 1,000,000,000,000
Yes 0.0000000000001
Yes -5
Yes 0
No 8 + sin(n)

Design and Analysis of Algorithms Chapter 2.1

5
1,000,000,000,000
0.0000000000001
-5
0
8 + sin(n)
55

The running time of the algorithm is a Constant


It does not depend significantly
on the size of the input.

57

Which Functions are Quadratic?

Design and Analysis of Algorithms Chapter 2.1

58

Which Functions are Quadratic?


n2
0.001 n2
1000 n2
5n2 + 3n + 2log n

n2
0.001 n2
Lie in between
1000 n2
5n2 + 3n + 2log n

Design and Analysis of Algorithms Chapter 2.1

56

n2
0.001 n2
1000 n2
5n2 + 3n + 2log n

5
1,000,000,000,000
9
0.0000000000001
7
-5
0
Lie in between
8 + sin(n)
Design and Analysis of Algorithms Chapter 2.1

Design and Analysis of Algorithms Chapter 2.1

Which Functions are Quadratic?

Which Functions are Constant?

Yes
Yes
Yes
No
No
Yes

Chapter 1

Ignore low-order terms


Ignore multiplicative constants.
Ignore "small" values of n.
Write (n2).
59

Design and Analysis of Algorithms Chapter 2.1

60

10

Design and Analysis of Algorithms

Chapter 1

Examples
f(n)

g(n)

1)

ln2n

2)

nk

cn

3)

nsinn

4)

2n

2n/2

5)

nlgc

clgn

6)

O(g(n))?

(g(n))? (g(n))?

lg(n!) lg(nn)

Design and Analysis of Algorithms Chapter 2.1

61

11