Vous êtes sur la page 1sur 52

CA729 DESIGN AND

ANALYSIS OF ALGORITHMS

The Course
CA729 DESIGN AND ANALYSIS OF ALGORITHMS
Objective:
A rigorous introduction to the design and analysis
of algorithms
To learn about Time Complexity and various
algorithmic design methodologies

Textbook: Introduction to Algorithms, Cormen,


Leiserson, Rivest, Stein
Third edition

Instructor:

Algorithms as a
technology
Suppose computers were infinitely fast and
computer memory was free.
Would you have any reason to study algorithms?
The answer is yes
You would still like to demonstrate that your solution
method terminates and does so with the correct answer.

If computers were infinitely fast, any correct method


for solves the problem.
Implementation to be within the bounds of good software
engineering practice (i.e., well designed and documented)
Among many available algorithms, people use whichever
method was the easiest to implement.

Algorithms as a
technology
Of course, computers may be fast, but
they are not infinitely fast.
Memory may be cheap, but it is not free.
Computing time is therefore a bounded
resource, and so is space in memory.
These resources should be used
cleverly.
Algorithms that are efficient in terms of
time or space will help you do so.

Algorithms as a
technology
Algorithms devised to solve the same problem often differ
dramatically in their Efficiency.
Consider two algorithms
Insertion sort

c1 n 2

Merge sort

c2 n log n

C1 and C2 are not depends on n


Insertion sort usually has a smaller constant factor than
merge sort, so that c1 < c2
Constant factors can be far less significant in the running
time.
Insertion sort is usually faster than merge sort for small
input sizes.
For large n, merge sort runs faster.

Algorithms as a
technology
Consider two Computers

Computer A
Insertion Sort

One billion instructions


per second

100 times faster


C1 n2
c1 = 2
2n2

Computer B
Merge Sort

Ten million instructions


per second

C2 n log n
c2 = 50
50 n log n

Algorithms as a
technology
By using an algorithm whose running time grows more slowly,
even with computer B runs 20 times faster than computer A!
The advantage of merge sort is even more pronounced when
we sort ten million numbers:
Where insertion sort takes approximately 2.3 days, merge
sort takes under 20 minutes.
In general, as the problem size increases, so does the relative
advantage of merge sort increases.
The example above shows that algorithms, like computer
hardware, are a technology.
Total system performance depends on choosing efficient
algorithms as much as on choosing fast hardware.
Just as rapid advances are being made in other computer
technologies, they are being made in algorithms as well.

Insertion Sort

Sorting a hand of cards using insertion sort

Insertion Sort
Pseudocode

Insertion Sort
Example

Insertion Sort

At beginning of each iteration,


the subarray A[1j-1] constitutes sorted array
the subarray A[j+1n] corresponds to unsorted array

Loop Invariant:
At the start of each iteration elements A[1j-1] are the elements originally in positions from 1 to j-1
But, in sorted order

Correctness of Algorithm:
Loop invariant is used

Initialization:
J=2,
A[1j-1] = A[1]
Only one element

Maintenance:

For any j value, body of the for loop moves A[j-1], A[j-2], A[j-3],.
until find a proper position fie A[j].
Then inserts A[j] in correct position
Incrementing j for next iteration also preserves the loop invariant
Termination:

one position right

When for loop terminates?


Termination condition j > A.length = n
When j = n+1, we have the subarray A[1j-1] in sorted order which is equal to A[1]

Analysis of Algorithms
Analysis is performed with respect to a
computational model
We will usually use a generic uniprocessor
random-access machine (RAM)
All memory equally expensive to access
No concurrent operations
All reasonable instructions take unit time
Except, of course, function calls

Constant word size


Unless we are explicitly manipulating bits

Input Size
Time and space complexity
This is generally a function of the input
size
E.g., sorting, multiplication

How we characterize input size depends:

Sorting: number of input items


Multiplication: total number of bits
Graph algorithms: number of nodes & edges
Etc

Running Time
Number of primitive steps that are
executed
Except for time of executing a function
call
Most statements roughly require the
same amount of time
y=m*x+b
c = 5 / 9 * (t - 32 )
z = f(x) + g(y)

We can be more exact if need be

Analysis
Worst case
Provides an upper bound on running
time
An absolute guarantee

Average case
Provides the expected running time
Very useful, but treat with care: what is
average?
Random (equally likely) inputs
Real-life inputs

Analysis
Why it matters
Growth of functions

An Example: Insertion Sort


30

10

40

20

i = j = key =
A[j] =
A[j+1] =

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


30

10

40

20

i=2 j=1
A[j] = 30

key = 10
A[j+1] = 10

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


30

30

40

20

i=2 j=1
A[j] = 30

key = 10
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


30

30

40

20

i=2 j=1
A[j] = 30

key = 10
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


30

30

40

20

i=2 j=0
A[j] =

key = 10
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


30

30

40

20

i=2 j=0
A[j] =

key = 10
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=2 j=0
A[j] =

key = 10
A[j+1] = 10

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=3 j=0
A[j] =

key = 10
A[j+1] = 10

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=3 j=0
A[j] =

key = 40
A[j+1] = 10

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=3 j=0
A[j] =

key = 40
A[j+1] = 10

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=3 j=2
A[j] = 30

key = 40
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=3 j=2
A[j] = 30

key = 40
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=3 j=2
A[j] = 30

key = 40
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=4 j=2
A[j] = 30

key = 40
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=4 j=2
A[j] = 30

key = 20
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=4 j=2
A[j] = 30

key = 20
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=4 j=3
A[j] = 40

key = 20
A[j+1] = 20

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

20

i=4 j=3
A[j] = 40

key = 20
A[j+1] = 20

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

40

i=4 j=3
A[j] = 40

key = 20
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

40

i=4 j=3
A[j] = 40

key = 20
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

40

i=4 j=3
A[j] = 40

key = 20
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

40

i=4 j=2
A[j] = 30

key = 20
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

40

40

i=4 j=2
A[j] = 30

key = 20
A[j+1] = 40

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

30

40

i=4 j=2
A[j] = 30

key = 20
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

30

40

i=4 j=2
A[j] = 30

key = 20
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

30

40

i=4 j=1
A[j] = 10

key = 20
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

30

30

40

i=4 j=1
A[j] = 10

key = 20
A[j+1] = 30

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

20

30

40

i=4 j=1
A[j] = 10

key = 20
A[j+1] = 20

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

An Example: Insertion Sort


10

20

30

40

i=4 j=1
A[j] = 10

key = 20
A[j+1] = 20

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

Done!

Insertion Sort
What is the precondition
for this loop?

InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
}
}

Insertion Sort
InsertionSort(A, n) {
for i = 2 to n {
key = A[i]
j = i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] = A[j]
j = j - 1
}
A[j+1] = key
How many times will
}
this loop execute?
}

Insertion Sort
Statement Effort
InsertionSort(A, n) {
for i = 2 to n { c1n
key = A[i] c2(n-1)
j = i - 1; c3(n-1)
while (j > 0) and (A[j] > key) { c4T
A[j+1] = A[j] c5(T-(n-1))
j = j - 1
c6(T-(n-1))
} 0
A[j+1] = key
c7(n-1)
} 0
}
T = t2 + t3 + + tn where ti is number of while expression evaluations for
the ith for loop iteration

Analyzing Insertion Sort


T(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1))
+ c7(n-1)

= c8T + c9n + c10

What can T be?


Best case -- inner loop body never executed
ti = 1 T(n) is a linear function

Worst case -- inner loop body executed for all


previous elements
ti = i T(n) is a quadratic function

Average case
???

Analysis
Simplifications
Ignore actual and abstract statement
costs
Order of growth is the interesting
measure:
Highest-order term is what counts
Remember, we are doing asymptotic analysis
As the input size grows larger it is the high order
term that dominates

Upper Bound Notation


We say InsertionSorts run time is O(n2)
Properly we should say run time is in O(n2)
Read O as Big-O (youll also hear it as
order)

In general a function
f(n) is O(g(n)) if there exist positive constants
c and n0 such that f(n) c g(n) for all n n0

Formally
O(g(n)) = { f(n): positive constants c and n0
such that f(n) c g(n) n n0
David Luebke
51

Insertion Sort Is O(n )


2

Proof
Suppose runtime is an2 + bn + c
If any of a, b, and c are less than 0 replace the
constant with its absolute value
an2 + bn + c (a + b + c)n2 + (a + b + c)n + (a + b
+ c)

3(a + b + c)n2 for n 1


Let c = 3(a + b + c) and let n0 = 1

Question
Is InsertionSort O(n3)?
Is InsertionSort O(n)?

Vous aimerez peut-être aussi