Heap and Merge Sorts Lecture 32 Robb T. Koether Hampden-Sydney College Mon, Apr 20, 2015 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 1 / 22
1 Sorting 2 Comparison of Run Times 3 The Merge Sort 4 The Heap Sort Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 2 / 22
Outline 1 Sorting 2 Comparison of Run Times 3 The Merge Sort 4 The Heap Sort Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 3 / 22
Total Order Relations Definition (Total Order Relation) A total order relation on a set A is a relation that has the following properties. Reflexive: For all a A, a a. Anti-symmetric: For all a, b A, if a b and b a, then a = b. Transitive: For all a, b, c A, if a b and b c, then a c. Comparable: For all a, b A, a b, or b a. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 4 / 22
Sorting The order is determined by an order operator: <, >, <=, or >=. These operators, together with == and!=, must define a total order on the class. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 5 / 22
Inefficient Sorting Algorithms Most elementary sorting algorithms are inefficient for long lists. Examples Bubble Sort. Selection Sort. Insertion Sort. These algorithms have run times of order O(n 2 ). They are fine for short lists (length < 10, 000) and ok for length < 100, 000. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 6 / 22
Efficient Sorting Algorithms The efficient sorting algorithms are more complicated. Examples Merge Sort Heap Sort Quick Sort These algorithms have run times of order O(n log n). They are fast for lists of any practical length. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 7 / 22
Outline 1 Sorting 2 Comparison of Run Times 3 The Merge Sort 4 The Heap Sort Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 8 / 22
Comparison of Algorithms How much faster is O(n log n) than O(n 2 )? Let s compare. Let A be an algorithm of order O(n 2 ). Let B be an algorithm of order O(n log n). Suppose both algorithms require 1 µsec to process a list of size n = 100. How long will they take to process lists of sizes 10 3, 10 4, 10 5, 10 6, 10 7, 10 8, and 10 9? Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 9 / 22
Comparison of Algorithms Algorithm A has run time Algorithm B has run time n 2 100 2 = 0.0001n2. n log n = 0.005n log n. 100 log 100 Evaluate these functions when n = 10 2, 10 3, 10 4, 10 5, 10 6, 10 7, 10 8, and 10 9. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 10 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs 10 3 100 µs 15 µs Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs 10 3 100 µs 15 µs 10 4 10 ms 200 µs Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs 10 3 100 µs 15 µs 10 4 10 ms 200 µs 10 5 1 s 2.5 ms Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs 10 3 100 µs 15 µs 10 4 10 ms 200 µs 10 5 1 s 2.5 ms 10 6 100 s 30 ms Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs 10 3 100 µs 15 µs 10 4 10 ms 200 µs 10 5 1 s 2.5 ms 10 6 100 s 30 ms 10 7 2.8 h 350 ms Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs 10 3 100 µs 15 µs 10 4 10 ms 200 µs 10 5 1 s 2.5 ms 10 6 100 s 30 ms 10 7 2.8 h 350 ms 10 8 11.6 d 4 s Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Comparison of Algorithms n Algorithm A Algorithm B 10 2 1 µs 1 µs 10 3 100 µs 15 µs 10 4 10 ms 200 µs 10 5 1 s 2.5 ms 10 6 100 s 30 ms 10 7 2.8 h 350 ms 10 8 11.6 d 4 s 10 9 3.2 y 45 s Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 11 / 22
Outline 1 Sorting 2 Comparison of Run Times 3 The Merge Sort 4 The Heap Sort Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 12 / 22
The Merge Sort Merging two sorted lists of length n has run time O(n). The run time of the Merge Sort is O(n log n). Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 13 / 22
The Merge Sort Algorithm Begin by considering the list to be a collection of sublists each of length 1. 50 30 70 20 80 40 10 60 Merge adjacent sublists in pairs. Continue to merge adjacent sublists until there remains only one sublist. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 14 / 22
The Merge Sort The Merge Sort 50 30 70 20 80 40 10 60 Begin with a list of size n = 8. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
The Merge Sort The Merge Sort 30 50 70 20 80 40 10 60 Pass #1: Merge the 1st and 2nd lists. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
The Merge Sort The Merge Sort 30 50 20 70 80 40 10 60 Pass #1: Merge the 3rd and 4th lists. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
The Merge Sort The Merge Sort 30 50 20 70 40 80 10 60 Pass #1: Merge the 5th and 6th lists. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
The Merge Sort The Merge Sort 30 50 20 70 40 80 10 60 Pass #1: Merge the 7th and 8th lists. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
The Merge Sort The Merge Sort 20 30 50 70 40 80 10 60 Pass #2: Merge the 1st and 2nd lists. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
The Merge Sort The Merge Sort 20 30 50 70 10 40 60 80 Pass #2: Merge the 3rd and 4th lists. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
The Merge Sort The Merge Sort 10 20 30 40 50 60 70 80 Pass #3: Merge the 1st and 2nd lists. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 15 / 22
Outline 1 Sorting 2 Comparison of Run Times 3 The Merge Sort 4 The Heap Sort Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 16 / 22
The Heap Sort Definition Heap Sort A heap is a binary tree that has the following structure: At each node, the value at that node is greater than or equal to each of its children. Every level except the lowest level is full. The nodes in the lowest level are as far to the left as possible. The Heap Sort algorithm uses a heap. The nodes are indexed from 0 to size - 1 in level order. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 17 / 22
A Heap 90 80 50 60 70 30 10 20 40 A heap Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 18 / 22
The Heap Sort algorithm is as follows. Heapify the tree (to be explained). Set end equal to size - 1. While end > 0, do the following. Swap the root value with the end value. Sift down the root value down through the tree (to be explained). Decrement end. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 19 / 22
The siftdown() Function The siftdown() function proceeds as follows. Begin at the root node. While the node value is smaller than at least one of its active children, Swap it with the larger of its children. Make that child node the current node. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 20 / 22
The heapify() Function The heapify() function proceeds as follows. Begin with node = end/2, i.e., the last node with a child. While node > 1, do the following Sift the node down. Decrement the node. Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 21 / 22
A Heap 40 80 50 60 70 30 10 20 90 Swap 90 and 40 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 80 40 50 60 70 30 10 20 90 Swap 80 and 40 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 80 70 50 60 40 30 10 20 90 Swap 70 and 40 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 20 70 50 60 40 30 10 80 90 Swap 80 and 20 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 70 20 50 60 40 30 10 80 90 Swap 70 and 20 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 70 60 50 20 40 30 10 80 90 Swap 60 and 20 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 10 60 50 20 40 30 70 80 90 Swap 70 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 60 10 50 20 40 30 70 80 90 Swap 60 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 60 40 50 20 10 30 70 80 90 Swap 40 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 30 40 50 20 10 60 70 80 90 Swap 60 and 30 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 50 40 30 20 10 60 70 80 90 Swap 50 and 30 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 10 40 30 20 50 60 70 80 90 Swap 50 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 40 10 30 20 50 60 70 80 90 Swap 40 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 40 20 30 10 50 60 70 80 90 Swap 20 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 10 20 30 40 50 60 70 80 90 Swap 40 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 30 20 10 40 50 60 70 80 90 Swap 30 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 10 20 30 40 50 60 70 80 90 Swap 30 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 20 10 30 40 50 60 70 80 90 Swap 20 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 10 20 30 40 50 60 70 80 90 Swap 20 and 10 Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22
A Heap 10 20 30 40 50 60 70 80 90 Done Robb T. Koether (Hampden-Sydney College) Heap and Merge Sorts Mon, Apr 20, 2015 22 / 22