Sorting

CS 2401 Spring 2012

Motivation
An initial sort of the data can significantly enhance the performance of an algorithm. Sorted data examples:
Words in dictionary Files in a directory Musical compact discs at a store

Sorting Algorithms
Bubble sort Insertion sort Selection sort Merge sort Quick sort

Bubble Sort
Repeatedly step through the list to be sorted, comparing each pair of adjacent items and swapping them if they are in the wrong order. The pass through the list is repeated until no swaps are needed. Runtime O(n2)

Example
http://www.cosc.canterbury.ac.nz/mukundan/ dsal/BSort.html

Code
public void bubbleSort(arr[]){ int temp; for(int j=size-1; j > 0; j--){ for(int i=0; i < j; i++){ if(arr[i] > arr[i+1]){ temp=arr[i]; arr[i]=arr[j]; arr[j]=temp; } } } }

Exercise
Trace Bubble Sort for the following number sequence:
6,15,3,7,23,5,0,8

Insertion Sort
A simple sorting algorithm that is appropriate for small inputs. Intuitively, take the number one at a time and build up another sorted list. Runtime is O(n2) for average, but in practice very fast on small arrays
quicksort uses insertion sort for low-n cases, typically n <= 10.

How to Sort
() (8,2,5,3,10,7,1,4,6,9) (8)(2,5,3,10,7,1,4,6,9) (2,8) (5,3,10,7,1,4,6,9) (2,5,8) (3,10,7,1,4,6,9) (2,3,5,8) (10,7,1,4,6,9) (2,3,5,8,10) (7,1,4,6,9) (2,3,5,7,8,10) (1,4,6,9) (1,2,3,5,7,8,10) (4,6,9) (1,2,3,4,5,7,8,10) (6,9) (1,2,3,4,5,6,7,8,10) (9) (1,2,3,4,5,6,7,8,9,10) ()

Idea
Spilt array into two regions, a sorted region (initially one element) and an unsorted region. In each iteration, move one element from the unsorted to the sorted region. Algorithm:
While unsorted region is not empty Let f be the first element of the unsorted region Shift right every element of the sorted region that is greater than f Insert f to the right of the first element that was not shifted

Example
http://www.cosc.canterbury.ac.nz/mukundan/ dsal/ISort.html

Exercise
Trace Insertion Sort for the following number sequence:
6,15,3,7,23,5,0,8

Selection Sort
Find the minimum value in the list Swap it with the value in the first position Repeat the steps above for the remainder of the list (starting at the second position and advancing each time) The list is divided into two parts
the sublist of items already sorted, which is built up from left to right and is found at the beginning. the sublist of items remaining to be sorted, occupying the remainder of the array.

Runtime: O(n2)

Example
http://www.cosc.canterbury.ac.nz/mukundan/ dsal/SSort.html

Exercise
Trace Selection Sort for the following number sequence:
6,15,3,7,23,5,0,8

Shellsort
1959 discovered by Donald Shell Sub-quadratic algorithm Performance is highly dependent on the increment sequence. Also called diminishing gap sort

Idea
Compare elements that are far apart Compare elements that are less apart So on Typically use incremental sequence of [1,3,5] Shell suggested starting gap at N/2 and half it until 1 is reached.

Example
http://www.akira.ruc.dk/~keld/algoritmik_e99 /Applets/Chap08/ShellSort/ShellSort.html

Example
http://www.youtube.com/watch?v=CmPA7zE8 mx0 Starting at N/2

Example

1-20

Pseudocode

Data Structures: A Pseudocode Approach with C, Second Edition

21

Analysis
Worst Case: O(n2)
N is an exact power of 2 all the large elements are in even indexed positions All the small elements are in odd indexed positions

Average: O(n3/2)
If consecutive increments are relatively prime

Best: O(n7/6)
Divide by 2.2 instead of 2.

Other.
http://en.wikipedia.org/wiki/Shellsort

Exercise
Trace ShellSort for the following number sequence:
6,15,3,7,23,5,0,8

Merge Sort
Recursive algorithm Typically used with linked lists For arrays, you need extra space to store intermediate results Runtime: O(nlogn)

Idea
Break the list into halves Recursively mergesort each half Base case is 0 or 1 elements Combine sorted halves Its like binary search, except that you sort instead of search

Example
http://www.cosc.canterbury.ac.nz/mukundan/ dsal/MSort.html

Example
http://www.youtube.com/watch?v=XaqR3G_ NVoo

Pseudo Code
MergeSort(list, first, last){ if list size <= 1 return find midpoint recursive MergeSort(first half) recursive MergeSort(second half) merge first and second half } Merge(list1, list2) { newList = new list() while (both lists have remaining elements) { compare first elements of list1 and list2 remove the lowest one from respective list add to newList } add all remaining elements from non-empty list to newList }

Analysis
How many levels of recursion? log n division levels log n merge levels 2 log n overall Each division level takes O(n) need to find midpoint (loop through LL) Each merge step take O(n) merging two lists requires one comparison/move for each element in the merged list (worst case) at each level, we have at most n elements in total Therefore, we have 2 n log n = O (n log n) complexity in the worst case n log n average case performance

Exercise
Trace Merge Sort for the following number sequence:
6,15,3,7,23,5,0,8

Quicksort
Another recursive algorithm Very fast in practice; one of the best known/most common methods

Process
Pick an element, called a pivot, from the list. Reorder the list so that all elements with values less than the pivot come before the pivot, while all elements with values greater than the pivot come after it (equal values can go either way). After this partitioning, the pivot is in its final position.
This is called the partition operation.

Process
Recursively sort the sub-list of lesser elements and the sub-list of greater elements.
The base case of the recursion are lists of size zero or one, which never need to be sorted.

Idea
Select a "pivot" Divide remaining elements into "lower" and "higher" sublists Recursively quicksort each sublist Combine: "lower" "pivot" "higher

Example
http://www.cosc.canterbury.ac.nz/mukundan/ dsal/QSort.html

Example
http://www.youtube.com/watch?v=ywWBy6J 5gz8

Pivoting
critical to algorithm
want one that divides list equally on average into halves

random selection works ok

"median of three" works even better

hard to randomize selection for LL implementation; leads to poorer performance (use merge sort instead)

Code
QuickSort(list, first, last) { if list size <= 1 return select pivot newPivotIndex = Partition(list, first, last, pivot) quickSort lower list quickSort higher list return } int Partition(list, first, last, pivot) { swap pivot into first position smallIndex = first for (first+1 to end of list) { if (element is less than pivot) { smallIndex++ swap element at smallIndex with current element } } swap pivot with smallIndex }

Analysis
Each partitioning step takes O(n) time
look at each element and compare to pivot may swap each element

How many recursive calls?

If the pivot is (approximately) in the middle, we divide the list in half each time O(log n) levels of recusion just like mergesort

Analysis
Worst case
pivot is min or max element; all remaining elements are in one list of the other in this case, we have n levels of recursion

Average case
n log n

Worst case
n^2

In practice, very fast Can implement tricks to avoid bad cases

Exercise
Trace QuickSort for the following number sequence:
6,15,3,7,23,5,0,8