Insertion Sort, Merge Sort and Quicksort

Sorting
Insertion sort - Mergesort Quicksort
Insertion Sort
Algorithm:
Move left-to-right through array.
Exchange next element with larger
elements to its left, one-by-one.
Example
InsertionSort( ElementType A[ ], int N)

{
int j, P;
Element Type Tmp;
for (P=1; P<N; P++)
{
Tmp = A[P];
for (j=P; j>0 && A[j-1]>Tmp; j--)
A[j] = A[j-1];
A[j] = Tmp;
}
}
4
Contd.....
Analyzing Insertion Sort

T(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1)
= c8T + c9n + c10
What can T be?

Best case -- inner loop body never executed
ti = 1 T(n) is a linear function
Worst case -- inner loop body executed for all previous

elements
ti = i T(n) is a quadratic function
Average case
???
Running time for insertion sort is O(n2)

7
Analysis
Simplifications
Ignore actual and abstract statement costs
Order of growth is the interesting measure:
Highest-order term is what counts
Remember, we are doing asymptotic analysis
As the input size grows larger it is the high order term that
dominates
Merge Sort
Apply divide-and-conquer to sorting problem
Problem: Given n elements, sort elements into
non-decreasing order
Divide-and-Conquer:
If n=1 terminate (every one-element list is already
sorted)
If n>1, partition elements into two or more subcollections; sort each; combine into a single sorted list
How do we partition?
9
Partitioning - Choice 1
First n-1 elements into set A, last element set B
Sort A using this partitioning scheme recursively
B already sorted
Combine A and B using method Insert() (= insertion

into sorted array)
Leads to recursive version of InsertionSort()
Number of comparisons: O(n2)
Best case = n-1
n
Worst case =
c i =
i=2
n(n 1)
2
10
Put element with largest key in B, remaining elements
in A
Sort A recursively
To combine sorted A and B, append B to sorted A
Use Max() to find largest element recursive
SelectionSort()
Use bubbling process to find and move largest element to
right-most position recursive BubbleSort()
All O(n2)
11
Lets try to achieve balanced partitioning

A gets n/2 elements, B gets rest half
Sort A and B recursively
Combine sorted A and B using a process
called merge, which combines two sorted
lists into one
How? We will see soon
12
13
MergeSort
14 23 45 98
33 42 67
14
14 23 45 98
33 42 67
Merge
15
14 23 45 98
33 42 67
6
Merge
16
14 23 45 98
6
33 42 67
14
Merge
17
14 23 45 98
6
33 42 67
14 23
Merge
18
14 23 45 98
6
33 42 67
14 23 33
Merge
19
14 23 45 98
6
33 42 67
14 23 33 42
Merge
20
14 23 45 98
6
33 42 67
14 23 33 42 45
Merge
21
14 23 45 98
6
33 42 67
14 23 33 42 45 67
Merge
22
14 23 45 98
6
33 42 67
14 23 33 42 45 67 98
Merge
23
void mergeSort(Comparable A[], int left, int right)

{
// sort a[left:right]
if (left < right)
{
// at least two elements

int mid = (left+right)/2;
//midpoint
mergeSort(A, left, mid);

mergeSort(A, mid + 1, right);
merge(A, B, left, mid, right); //merge from A to B
copy(B, A, left, right);
//copy result back to A
}
}
24
Evaluation
Recurrence equation:
Assume n is a power of 2
c1
if n=1
T(n) =
2T(n/2) + c2n
if n>1, n=2k
25
Solution
By Substitution:
T(n) = 2T(n/2) + c2n
T(n/2) = 2T(n/4) + c2n/2
T(n) = 4T(n/4) + 2 c2n
T(n) = 8T(n/8) + 3 c2n
T(n) = 2iT(n/2i) + ic2n
Assuming n = 2k, expansion halts when we get T(1) on right side; this happens
when i=k T(n) = 2kT(1) + kc2n
Since 2k=n, we know k=logn; since T(1) = c1, we get
T(n) = c1n + c2nlogn;
Thus an upper bound for TmergeSort(n) is O(nlogn)

26
Quicksort Algorithm
Given an array of n elements (e.g., integers):
If array only contains one element, return
Else
pick one element to use as pivot.
Partition elements into two sub-arrays:
Elements less than or equal to pivot
Elements greater than pivot
Quicksort two sub-arrays

Return results
27
Example
We are given array of n integers to sort:
40 20 10 80 60 50
30 100
28
Pick Pivot Element

There are a number of ways to pick the pivot element. In this
example, we will use the first element in the array:
40 20 10 80 60 50
30 100
29
Partitioning Array
Given a pivot, partition the elements of the array
such that the resulting array consists of:
1. One sub-array that contains elements >= pivot
2. Another sub-array that contains elements < pivot
The sub-arrays are stored in the original data array.

Partitioning loops through, swapping elements
below/above pivot.
30
pivot_index = 0
40
20
10
80 60 50
30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
31
1. While data[too_big_index] <= data[pivot]

++too_big_index
pivot_index = 0
40
20
10
80 60 50
30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
32

++too_big_index
pivot_index = 0
40
20
10
80 60 50
30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
33

++too_big_index
pivot_index = 0
40
20
10
80 60 50
30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
34

++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
pivot_index = 0
40
20
10
80 60 50
30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
35

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
80 60 50
30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
36

++too_big_index
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
pivot_index = 0
40
20
10
80 60 50
30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
37

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30 60 50
80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
38

++too_big_index
--too_small_index
swap data[too_big_index] and
data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 0
40
20
10
30 60 50
80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
39

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30 60 50
80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
40

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30 60 50
80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
41

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30 60 50
80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
42

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30 60 50
80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
43

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30 60 50
80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
44

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
45

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
46

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
47

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
48

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
49

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
50

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
51

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
52

++too_big_index
--too_small_index
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
53

++too_big_index
--too_small_index
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 0
40
20
10
30
50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
54

++too_big_index
--too_small_index
pivot_index = 4
20
10
30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index
too_small_index
55
Partition Result
7
20 10 30
[0] [1] [2]
40
50
60 80 100
[3] [4] [5] [6] [7] [8]
<= data[pivot]
> data[pivot]
56
Recursion: Quicksort Sub-arrays

7
20 10 30
[0] [1] [2]
40
50
60 80 100
[3] [4] [5] [6] [7] [8]
<= data[pivot]
> data[pivot]
57
58
Quicksort Analysis
Assume that keys are random, uniformly
distributed.
What is best case running time?
59
Quicksort Analysis

distributed.
Recursion:
1. Partition splits array in two sub-arrays of size n/2
2. Quicksort each sub-array
60
Quicksort Analysis

distributed.
Recursion:
Depth of recursion tree?

61
Quicksort Analysis

distributed.
Recursion:
Depth of recursion tree? O(log2n)

62
Quicksort Analysis

distributed.
Recursion:

Number of accesses in partition?
63
Quicksort Analysis

distributed.
Recursion:

Number of accesses in partition? O(n)
64
Quicksort Analysis

distributed.
Best case running time: O(n log2n)
65
Quicksort Analysis

distributed.
Worst case running time?
66
Quicksort: Worst Case

Assume first element is chosen as pivot.
Assume we get array that is already in
order:
pivot_index = 0
10
[0] [1] [2]

too_big_index
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]

too_small_index
67

++too_big_index
--too_small_index
pivot_index = 0
10
[0] [1] [2]

too_big_index
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]

too_small_index
68

++too_big_index
--too_small_index
pivot_index = 0
10
[0] [1] [2]

too_big_index
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]

too_small_index
69

++too_big_index
--too_small_index
pivot_index = 0
10
[0] [1] [2]

too_big_index
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]

too_small_index
70

++too_big_index
--too_small_index
pivot_index = 0
10
[0] [1] [2]

too_big_index
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]

too_small_index
71

++too_big_index
--too_small_index
pivot_index = 0
10
[0] [1] [2]

too_big_index
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]

too_small_index
72

++too_big_index
--too_small_index
pivot_index = 0
10
[0] [1] [2]

too_big_index
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]

too_small_index
73

++too_big_index
--too_small_index
pivot_index = 0
10
[0] [1] [2]

<= data[pivot]
12
13
50 57 63 100
[3] [4] [5] [6] [7] [8]
> data[pivot]
74
Quicksort Analysis

distributed.
Recursion:
1.
Partition splits array in two sub-arrays:
2.
one sub-array of size 0

the other sub-array of size n-1
Quicksort each sub-array
Depth of recursion tree?

75
Quicksort Analysis

distributed.
Recursion:
1.
2.

Depth of recursion tree? O(n)

76
Quicksort Analysis

distributed.
Recursion:
1.
2.


Number of accesses per partition?
77
Quicksort Analysis

distributed.
Recursion:
1.
2.


Number of accesses per partition? O(n)
78
Quicksort Analysis

distributed.
Worst case running time: O(n2)!!!
79
Quicksort Analysis

distributed.
Worst case running time: O(n2)!!!
What can we do to avoid worst case?
80
Improved Pivot Selection

Pick median value of three elements from data array:
data[0], data[n/2], and data[n-1].
Use this median value as pivot.
81
Improving Performance of
Quicksort
Improved selection of pivot.
For sub-arrays of size 3 or less, apply brute
force search:
Sub-array of size 1: trivial
Sub-array of size 2:
if(data[first] > data[second]) swap them
Sub-array of size 3: left as an exercise.

82

Insertion Sort, Merge Sort and Quicksort

Transféré par

Informations du document

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Insertion Sort, Merge Sort and Quicksort

Transféré par

Droits d'auteur :

Formats disponibles

Sorting

Insertion sort - Mergesort Quicksort

InsertionSort( ElementType A[ ], int N)

Analyzing Insertion Sort

What can T be?

Worst case -- inner loop body executed for all previous

Running time for insertion sort is O(n2)

Combine A and B using method Insert() (= insertion

Lets try to achieve balanced partitioning

void mergeSort(Comparable A[], int left, int right)

// at least two elements

mergeSort(A, left, mid);

//copy result back to A

Thus an upper bound for TmergeSort(n) is O(nlogn)

Quicksort two sub-arrays

Pick Pivot Element

The sub-arrays are stored in the original data array.

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]

1. While data[too_big_index] <= data[pivot]

[0] [1] [2] [3] [4] [5] [6] [7] [8]