The Heroic Tales of Sorting Algorithms

Notation: O(x) = Worst Case Running Time ;(x) = Best Case Running Time 5(x)!Best and Worst case are the same. Page numbers refer to the Preiss text book Data Structures and Algorithms with ObjectOrientated Design Patterns in Java. This page was created with some references to Paul's spiffy sorting algorithms page which can be found here. Most of the images scans of the text book (accept the code samples) were gratefully taken from that site. Sorting Implementation Summary Comments Type Page Algorithm On each pass the current Good for nearly Insertion item is inserted into the sorted lists, very sorted section of the list. It bad for out of starts with the last position order lists, due to of the sorted list, and the shuffling. moves backwards until it finds the proper place of the current item. That item is then inserted into that place, and all items after that are shuffled to the left to accommodate it. It is for Straight this reason, that if the list is 495 Insertion already sorted, then the sorting would be O(n) because every element is already in its sorted position. If however the list is sorted in reverse, it would take O(n2) time as it would be searching through the entire sorted section of the list each time it does an insertion, and shuffling all other elements down the list.. Stable? Asymptotic Complexities Yes

Best Case: O(n). Worst Case: O(n2)

This is an extension of the Straight Insertion as above, however instead of doing a linear search each time for the correct position, it does a binary search, which is O(log n) instead of O(n). The only problem is that it Binary always has to do a binary Insertion 497 search even if the item is in Sort its current position. This brings the cost of the best cast up to O(n log n). Due to the possibility of having to shuffle all other elements down the list on each pass, the worst case running time remains at O(n2). On each pass of the data, adjacent elements are compared, and switched if they are out of order. eg. e1 with e2, then e2 with e3 and so on. This means that on each pass, the largest element that is left unsorted, has been "bubbled" to its rightful place at the end of the array. However, due to the fact that all adjacent out of Bubble 499 order pairs are swapped, Sort the algorithm could be finished sooner. Preiss claims that it will always take O(n2) time because it keeps sorting even if it is in order, as we can see, the algorithm doesn't recognise that. Now someone with a bit more knowledge than Preiss will obviously see, that you can end the algorithm in the case when no swaps were made,

This is better Insertion than the Strait Insertion if the comparisons are costly. This is because even though, it always has to do log n comparisons, it would generally work out to be less than a linear search.

Yes

Best Case: O(n log n). Worst Case: O(n2)

In general this is Exchange better than Insertion Sort I believe, because it has a good change of being sorted in much less than O(n2) time, unless you are a blind Preiss follower.

Yes.

NOTE: Preiss uses a bad algorithm, and claims that best and worst case is O(n2). We however using a little bit of insight, can see that the following is correct of a better bubble sort Algorithm (which does Peake agree with?) Best Case: O(n). Worst Case:

thereby making the best case O(n) (when it is already sorted) and worst case still at O(n2). I strongly recommend looking at the diagram for this one. The code is also useful and provided below (included is the selectPivot method even though that probably won't help you understanding anyway). The quick sort operates along these lines: Firstly a pivot is selected, and Quicksort 501 removed from the list (hidden at the end). Then the elements are partitioned into 2 sections. One which is less than the pivot, and one that is greater. This partitioning is achieved by exchanging values. Then the pivot is restored in the middle, and those 2 sections are recursively quick sorted. This one, although not very efficient is very simply. Basically, it does n2 linear passes on the list, and on each pass, it selects the largest value, and swaps it with the last unsorted Straight element. Selection 511 This means that it isn't Sorting. stable, because for example a 3 could be swapped with a 5 that is to the left of a different 3.

O(n2)

A complicated but effective sorting algorithm.

Exchange

No

Best Case: O(n log n). Worst Case: O(n2) Refer to page 506 for more information about these values. Note: Preiss on page 524 says that the worst case is O(n log n) contradicting page 506, but I believe that it is O(n2), as per page 506. Unlike the Bubble sort this one is truly 5(n2), where best case and worst case are the same, because even if the list is sorted, the same number of selections must still be performed.

A very simple Selection algorithm, to code, and a very simple one to explain, but a little slow. Note that you can do this using the smallest value, and swapping it with the first unsorted element.

No

This uses a similar idea to This utilises, just Selection the Straight Selection about the only Sorting, except, instead of good use of using a linear search for the heaps, that is maximum, a heap is finding the constructed, and the maximum maximum can easily be element, in a max removed (and the heap heap (or the reformed) in log n time. minimum of a Heap Sort 513 This means that you will do min heap). Is in n passes, each time doing a every way as log n remove maximum, good as the meaning that the algorithm straight selection will always run in 5(n log sort, but faster. n) time, as it makes no difference the original order of the list.

No

Best Case: O(n log n). Worst Case: O(n log n). Ok, now I know that looks tempting, but for a much more programmer friendly solution, look at Merge sort instead, for a better O(n log n) sort .

2 Way Merge Sort

It is fairly simple to take 2 sorted lists, and combine the into another sorted list, simply by going through, comparing the heads of each list, removing the smallest to join the new sorted list. As you may guess, this is an O(n) operation. With 2 way sorting, we apply this method to an single unsorted list. In brief, the 519 algorithm recursively splits up the array until it is fragmented into pairs of two single element arrays. Each of those single elements is then merged with its pairs, and then those pairs are merged with their pairs and so on, until the entire list is united in sorted order. Noting that if there is every an odd number, an extra operation is added, where it is added

Now isn't this Merge much easier to understand that Heap sort, its really quite intuitive. This one is best explain with the aid of the diagram, and if you haven't already, you should look at it.

Yes

Best and Worst Case: 5(n log n)

Bucket Sort

Radix Sort

to one of the pairs, so that that particular pair will have 1 more element than most of the others, and won't have any effect on the actual sorting. Bucket sort initially creates This sufferers a Distribution No a "counts" array whose size limitation that is the size of the range of Radix doesn't, in all possible values for the that if the data we are sorting, eg. all possible range of of the values could be your numbers is between 1 and 100, very high, you therefore the array would would need too have 100 elements. 2 many "buckets" passes are then done on the and it would be list. The first tallies up the impractical. The occurrences of each of other limitation number into the "counts" that Radix array. That is for each doesn't have, that 526 index of the array, the data this one does is that it contains signifies the that stability is number of times that not maintained. It number occurred in list. does however The second and final pass outperform radix goes though the counts sort if the array, regenerating the list possible range is in sorted form. So if there very small. were 3 instance of 1, 0 of 2, and 1 of 3, the sorted list would be recreated to 1,1,1,3. This diagram will most likely remove all shadows of doubt in your minds. This is an extremely spiffy This is the god of Distribution Yes implementation of the sorting bucket sort algorithm. This algorithms. It time, several bucket like will search the sorts are performed (one largest list, with 528 for each digit), but instead the biggest of having a counts array numbers, and has representing the range of a is guaranteed all possible values for the O(n) time data, it represents all of the complexity. And possible values for each it ain't very

Best and Worst case:5(m + n) where m is the number of possible values. Obviously this is O(n) for most values of m, so long as m isn't too large. The reason that these distribution sorts break the O(n log n) barrier is because no comparisons are performed!

Best and Worst Case: 5(n) Bloody awesome!

individual digit, which in complex to decimal numbering is only understand or 10. Firstly a bucked sort is implement. performed, using only the My least significant digit to recommendations sort it by, then another is are to use this done using the next least one wherever significant digit, until the possible. end, when you have done the number of bucket sorts equal to the maximum number of digits of your biggest number. Because with the bucket sort, there are only 10 buckets (the counts array is of size 10), this will always be an O(n) sorting algorithm! See below for a Radix Example. On each of the adapted bucket sorts it does, the count array stores the numbers of each digit. Then the offsets are created using the counts, and then the sorted array regenerated using the offsets and the original data. Radix Sort Example: First Pass:
Data: 67 50 70 25 93 47 21 Buckets on index 0 1 count 2 1 offset 0 2 first 2 3 4 0 1 0 3 3 4 pass (least significant digit): 5 6 7 8 9 1 0 2 0 0 4 5 5 7 7

Data after first pass 50 70 21 93 25 67 47

That data is created by doing a single pass on the unsorted data, using the offsets to work out at where each item belongs. For example, it looks at the first one 67, then at the offsets for the digit 7, and inserts it into the 5th position. The offset at 7 is then incremented, so that the next value encountered which has a least significant digit of 7 is placed into the 6th position. Continuing the example, the number 50

would then be looked at, and inserted into the 0th position, its offset incremented, so that the next value which is 70 would be inserted into the 1st position, and so on until then end of the list. As you can see, this data is sorted by its least significant digit. Second Pass:
Data after first pass 50 70 21 93 25 67 47 Buckets on index 0 1 count 0 0 offset 0 2 first 2 3 4 2 0 1 3 3 4 pass (most significant digit): 5 6 7 8 9 1 1 1 0 1 4 5 5 7 7

Data after second pass (sorted) 21 25 47 50 67 70 93

Look at this diagram for another example, noting that the "offsets" array is unnecessary

Images:

Straight Insertion - Figure 15.2

Bubble Sorting - Figure 15.3

Quick Sorting - Figure 15.4

Program 15.7 AbstractQuickSorter code

Program 15.9 MedianOfThreeQuickSorter class selectPivot method

Straight Selection Sorting - Figure 15.5

Building a heap - Figure 15.7

Heap Sorting - Figure 15.8

Two-way merge sorting - Figure 15.10

Bucket Sorting - Figure 15.12

Radix Sorting - Figure 15.13