Chapter 3: Sorting Algorithms
Introduction
Learning Outcomes
Evaluate a sort by time, extra memory, stability, whether it is in place, and whether it is comparison-based.
Trace bubble, insertion, selection and Shell sort on an array step by step and implement them in Java.
Explain how merge sort, quicksort and heapsort work; justify their O(n log n) and worst-case behavior.
Explain the comparison lower bound and counting and radix sort; choose the right algorithm for a problem.
Introduction
Sorting Algorithms
- From shopping lists to social media posts, the digital world is full of data.
- To use data efficiently we need to keep it organized.
- Sorting algorithms arrange the items of a list according to some criterion:
- numeric value,
- alphabetical order,
- date …
- The order can be ascending (smallest first) or descending.
- Formally: the input is ⟨a₁, a₂, …, aₙ⟩; the output is a permutation of the same numbers with a′₁ ≤ a′₂ ≤ … ≤ a′ₙ.
Introduction
Kinds of Sorting Algorithms
Different sorting algorithms work on different principles:
| Algorithm | Key idea |
|---|---|
| Bubble sort | Compares adjacent items and swaps them if they are out of order. |
| Insertion sort | Inserts the next item into its place in the sorted part. |
| Selection sort | Finds the smallest item and moves it to the front, then sorts the rest. |
| Shell sort | Insertion sort that starts with distant items and shrinks the gap. |
| Merge sort | Splits the array in half, sorts the halves, then merges them. |
| Quicksort | Partitions around a pivot and sorts the two parts recursively. |
| Heapsort | Builds a max-heap and repeatedly moves the largest item to the end. |
| Counting / Radix sort | No comparisons; counts the keys or distributes them digit by digit. |
| Timsort | A hybrid of insertion sort and merge sort; the object sort of Java and Python. |
Introduction
Evaluating a Sorting Algorithm
These bounds count one comparison as O(1); for strings of length L each comparison can cost O(L) (see Chapters 10–11).
Introduction
Stability: An Example
Records are sorted by the number (the key) only; the letter identifies the record. A stable algorithm keeps the records with key 3 in input order, A, C, F; the long-distance swaps of selection sort turn this into C, A, F. (This input is the example from the simulator lab.)
Bubble Sort
Bubble Sort
- On each pass the larger items move toward the end of the array, rising like bubbles in water.
- Adjacent items are compared in pairs.
- If the left one is larger, they are exchanged (swap).
- After pass i, the i largest items are in their final places; the next pass is one item shorter.
- If a pass makes no swaps, the array is sorted → early exit.
Bubble Sort · Step by step
Bubble Sort Step by Step: [3, 1, 5, 2, 6, 4]
Array: [3, 1, 5, 2, 6, 4]. Each pass compares adjacent pairs from left to right; the larger one moves right.
Pass 1, j = 0: 3 > 1? Yes → swap.
Swap: the array is now [1, 3, 5, 2, 6, 4].
Pass 1, j = 1: 3 > 5? No → stays.
Pass 1, j = 2: 5 > 2? Yes → swap.
Swap: the array is now [1, 3, 2, 5, 6, 4].
Pass 1, j = 3: 5 > 6? No → stays.
Pass 1, j = 4: 6 > 4? Yes → swap.
Swap: the array is now [1, 3, 2, 5, 4, 6].
Pass 1 done: the largest remaining item, 6, is in its final place (index 5).
Pass 2, j = 0: 1 > 3? No → stays.
Pass 2, j = 1: 3 > 2? Yes → swap.
Swap: the array is now [1, 2, 3, 5, 4, 6].
Pass 2, j = 2: 3 > 5? No → stays.
Pass 2, j = 3: 5 > 4? Yes → swap.
Swap: the array is now [1, 2, 3, 4, 5, 6].
Pass 2 done: the largest remaining item, 5, is in its final place (index 4).
Pass 3, j = 0: 1 > 2? No → stays.
Pass 3, j = 1: 2 > 3? No → stays.
Pass 3, j = 2: 3 > 4? No → stays.
Pass 3 made no swaps → the array is sorted and the algorithm stops early. Result: [1, 2, 3, 4, 5, 6].
Bubble Sort
Bubble Sort: Java Code
boolean swapped; for(int i = 0; i < n - 1; i++) { swapped = false; for(int j = 0; j < n - 1 - i; j++) { if(arr[j] > arr[j+1]) { int temp = arr[j]; arr[j] = arr[j+1]; arr[j+1] = temp; swapped = true; } } if(swapped == false) break; }
- Outer loop (i): at most n − 1 passes.
- Inner loop (j): adjacent pairs in the unsorted part; the last i items are already in place, hence
j < n − 1 − i. - The swap takes three assignments using
temp. - The
swappedflag: if a whole pass makes no swap, the array is sorted →break.
> (not ≥), equal items are never swapped: bubble sort is stable.Bubble Sort · Run the code step by step
Bubble Sort: Code Trace, arr = [5, 1, 9, 2, 10]
n = 5. The flag swapped is declared.
Outer loop i = 0: the flag is set to false.
j = 0: arr[0] = 5 > arr[1] = 1? Yes.
Swapped via temp = 5; the flag becomes true.
j = 1: arr[1] = 5 > arr[2] = 9? No.
j = 2: arr[2] = 9 > arr[3] = 2? Yes.
Swapped via temp = 9; the flag becomes true.
j = 3: arr[3] = 9 > arr[4] = 10? No.
End of pass: 10 is in place. Flag true → continue.
Outer loop i = 1: the flag is set to false.
j = 0: arr[0] = 1 > arr[1] = 5? No.
j = 1: arr[1] = 5 > arr[2] = 2? Yes.
Swapped via temp = 5; the flag becomes true.
j = 2: arr[2] = 5 > arr[3] = 9? No.
End of pass: 9 is in place. Flag true → continue.
Outer loop i = 2: the flag is set to false.
j = 0: arr[0] = 1 > arr[1] = 2? No.
j = 1: arr[1] = 2 > arr[2] = 5? No.
No swap in this pass: break. The array is sorted: [1, 2, 5, 9, 10].
boolean swapped; for(int i = 0; i < n - 1; i++) { swapped = false; for(int j = 0; j < n - 1 - i; j++) { if(arr[j] > arr[j+1]) { int temp = arr[j]; arr[j] = arr[j+1]; arr[j+1] = temp; swapped = true; } } if(swapped == false) break; }
Bubble Sort
Bubble Sort: Analysis
| Case | Input | Comparisons | Swaps | Time |
|---|---|---|---|---|
| Best | sorted | n − 1 (one pass, with flag) | 0 | Θ(n) |
| Average | random | ≈ n²/2 | ≈ n²/4 | Θ(n²) |
| Worst | reverse sorted | n(n − 1)/2 | n(n − 1)/2 | Θ(n²) |
- Very easy to understand and write.
- In place (O(1) extra memory) and stable.
- With the flag, recognizes sorted input in Θ(n).
- Θ(n²): very slow on large arrays.
- Makes many swaps (each inversion = 1 swap).
- In practice always slower than insertion sort.
An inversion is a pair i < j with arr[i] > arr[j]. Bubble and insertion sort fix exactly one inversion per step; a reverse-sorted array has n(n − 1)/2 inversions.
Insertion Sort
Insertion Sort
- Like sorting a hand of playing cards: each new card is slipped into its place among the sorted cards in your hand.
- The array has two parts: a sorted part (on the left) and an unsorted part.
- Take the first item of the unsorted part and insert it at its correct position in the sorted part.
- To make room, the larger items are shifted one place to the right.
- Invariant: at the start of each outer step,
arr[0..i−1]is sorted.
Insertion Sort · Step by step
Insertion Sort Step by Step: [3, 1, 5, 2, 6, 4]
Array [3, 1, 5, 2, 6, 4]. The first item alone is the sorted part; the rest is unsorted.
Take the first item of the unsorted part: key = 1.
3 > 1 → 3 is shifted right (index 0 → 1).
Reached the start of the array → 1 is inserted at index 0 (1 shift).
Take the first item of the unsorted part: key = 5.
3 ≤ 5 → 5 is inserted at index 2 (no shift needed).
Take the first item of the unsorted part: key = 2.
5 > 2 → 5 is shifted right (index 2 → 3).
3 > 2 → 3 is shifted right (index 1 → 2).
1 ≤ 2 → 2 is inserted at index 1 (2 shifts).
Take the first item of the unsorted part: key = 6.
5 ≤ 6 → 6 is inserted at index 4 (no shift needed).
Take the first item of the unsorted part: key = 4.
6 > 4 → 6 is shifted right (index 4 → 5).
5 > 4 → 5 is shifted right (index 3 → 4).
3 ≤ 4 → 4 is inserted at index 3 (2 shifts). Sorted: [1, 2, 3, 4, 5, 6].
Insertion Sort
Insertion Sort: Java Code
for(int i = 1; i < n; i++) { int key = arr[i]; int j = i - 1; while(j >= 0 && arr[j] > key) { arr[j+1] = arr[j]; j = j - 1; } arr[j+1] = key; }
- Start at i = 1: the one-element
arr[0..0]is already sorted. key: the item to insert. It is saved first because the shifts overwrite its cell.while: as long as the item on the left is larger, shift it one place right.- When the loop ends, the free slot is
j + 1.
Insertion Sort · Run the code step by step
Insertion Sort: Code Trace, arr = [5, 1, 9, 2, 10]
n = 5. Sorted part: just arr[0].
i = 1: key = arr[1] = 1, j = 0.
j = 0 ≥ 0 and arr[0] = 5 > 1 → enter the loop.
arr[1] = arr[0] = 5 (shift right), j = −1.
j = −1: the loop ends.
arr[0] = key = 1. Sorted part: [1, 5].
i = 2: key = arr[2] = 9, j = 1.
arr[1] = 5 ≤ 9: the loop ends.
arr[2] = key = 9. Sorted part: [1, 5, 9].
i = 3: key = arr[3] = 2, j = 2.
j = 2 ≥ 0 and arr[2] = 9 > 2 → enter the loop.
arr[3] = arr[2] = 9 (shift right), j = 1.
j = 1 ≥ 0 and arr[1] = 5 > 2 → enter the loop.
arr[2] = arr[1] = 5 (shift right), j = 0.
arr[0] = 1 ≤ 2: the loop ends.
arr[1] = key = 2. Sorted part: [1, 2, 5, 9].
i = 4: key = arr[4] = 10, j = 3.
arr[3] = 9 ≤ 10: the loop ends.
arr[4] = key = 10. Sorted part: [1, 2, 5, 9, 10].
i = 5 is not < n → the loop ends. Result: [1, 2, 5, 9, 10].
for(int i = 1; i < n; i++) { int key = arr[i]; int j = i - 1; while(j >= 0 && arr[j] > key) { arr[j+1] = arr[j]; j = j - 1; } arr[j+1] = key; }
Insertion Sort
Insertion Sort: Analysis
| Case | Input | Comparisons | Shifts | Time |
|---|---|---|---|---|
| Best | sorted | n − 1 | 0 | Θ(n) |
| Average | random | ≈ n²/4 | ≈ n²/4 | Θ(n²) |
| Worst | reverse sorted | n(n − 1)/2 | n(n − 1)/2 | Θ(n²) |
- Stable, in place, online.
- Adaptive: time Θ(n + number of inversions).
- Among the fastest on small arrays (n ≲ 20–50).
- Θ(n²) on large random arrays.
- An item moves only one cell per step; a distant item needs many shifts (→ Shell sort).
Over all inputs we can only say O(n²) and Ω(n); there is no single Θ. Timsort and many libraries hand small subarrays in quicksort and merge sort over to insertion sort.
Selection Sort
Selection Sort
- The array has two parts: sorted and unsorted.
- At each step:
- Find the smallest item in the unsorted part.
- Swap it with the item at the front of the unsorted part.
- After the swap it joins the sorted part.
- n − 1 passes in total; at most one swap per pass.
Selection Sort · Step by step
Selection Sort Step by Step: [3, 1, 5, 2, 6, 4]
Array [3, 1, 5, 2, 6, 4], length = 6. The sorted part starts empty.
Pass 1: the first unsorted item is the tentative minimum: min = 3.
1 < min (3)? Yes → min = 1.
5 < min (1)? No.
2 < min (1)? No.
6 < min (1)? No.
4 < min (1)? No.
The minimum (1) is swapped with the front of the unsorted part → [1, 3, 5, 2, 6, 4]; the sorted part grows by one.
Pass 2: the first unsorted item is the tentative minimum: min = 3.
5 < min (3)? No.
2 < min (3)? Yes → min = 2.
6 < min (2)? No.
4 < min (2)? No.
The minimum (2) is swapped with the front of the unsorted part → [1, 2, 5, 3, 6, 4]; the sorted part grows by one.
Pass 3: the first unsorted item is the tentative minimum: min = 5.
3 < min (5)? Yes → min = 3.
6 < min (3)? No.
4 < min (3)? No.
The minimum (3) is swapped with the front of the unsorted part → [1, 2, 3, 5, 6, 4]; the sorted part grows by one.
Pass 4: the first unsorted item is the tentative minimum: min = 5.
6 < min (5)? No.
4 < min (5)? Yes → min = 4.
The minimum (4) is swapped with the front of the unsorted part → [1, 2, 3, 4, 6, 5]; the sorted part grows by one.
Pass 5: the first unsorted item is the tentative minimum: min = 6.
5 < min (6)? Yes → min = 5.
The minimum (5) is swapped with the front of the unsorted part → [1, 2, 3, 4, 5, 6]; the sorted part grows by one.
After 5 passes the last item is in place by itself. Sorted array: [1, 2, 3, 4, 5, 6].
Selection Sort
Selection Sort: Java Code
public void sort(int[] arr) { int n = arr.length; for(int i = 0; i < n - 1; i++) { int minIdx = i; for(int j = i + 1; j < n; j++) { if(arr[j] < arr[minIdx]) { minIdx = j; } } int temp = arr[minIdx]; arr[minIdx] = arr[i]; arr[i] = temp; } }
minIdxholds the index of the smallest item (not its value): the swap at the end needs the index.- The inner loop scans the whole unsorted part; the scan is no shorter even if the input is sorted.
- One swap per pass: at most n − 1 in total.
Selection Sort · Run the code step by step
Selection Sort: Code Trace, arr = [5, 1, 10, 2, 9]
n = arr.length = 5.
i = 0: minIdx = 0 (arr[minIdx] = 5).
j = 1: arr[1] = 1 < arr[minIdx] = 5? Yes.
minIdx = 1 (new minimum: 1).
j = 2: arr[2] = 10 < arr[minIdx] = 1? No.
j = 3: arr[3] = 2 < arr[minIdx] = 1? No.
j = 4: arr[4] = 9 < arr[minIdx] = 1? No.
j = 5 → the inner loop ends. temp = 1; arr[1] ↔ arr[0] → [1, 5, 10, 2, 9].
i = 1: minIdx = 1 (arr[minIdx] = 5).
j = 2: arr[2] = 10 < arr[minIdx] = 5? No.
j = 3: arr[3] = 2 < arr[minIdx] = 5? Yes.
minIdx = 3 (new minimum: 2).
j = 4: arr[4] = 9 < arr[minIdx] = 2? No.
j = 5 → the inner loop ends. temp = 2; arr[3] ↔ arr[1] → [1, 2, 10, 5, 9].
i = 2: minIdx = 2 (arr[minIdx] = 10).
j = 3: arr[3] = 5 < arr[minIdx] = 10? Yes.
minIdx = 3 (new minimum: 5).
j = 4: arr[4] = 9 < arr[minIdx] = 5? No.
j = 5 → the inner loop ends. temp = 5; arr[3] ↔ arr[2] → [1, 2, 5, 10, 9].
i = 3: minIdx = 3 (arr[minIdx] = 10).
j = 4: arr[4] = 9 < arr[minIdx] = 10? Yes.
minIdx = 4 (new minimum: 9).
j = 5 → the inner loop ends. temp = 9; arr[4] ↔ arr[3] → [1, 2, 5, 9, 10].
i = 4 is not < n − 1 → done. Result: [1, 2, 5, 9, 10].
public void sort(int[] arr) { int n = arr.length; for(int i = 0; i < n - 1; i++) { int minIdx = i; for(int j = i + 1; j < n; j++) { if(arr[j] < arr[minIdx]) { minIdx = j; } } int temp = arr[minIdx]; arr[minIdx] = arr[i]; arr[i] = temp; } }
Selection Sort
Selection Sort: Analysis
| Measure | Value |
|---|---|
| Comparisons | (n − 1) + (n − 2) + … + 1 = n(n − 1)/2 on every input |
| Swaps | at most n − 1 |
| Time | best = average = worst = Θ(n²) |
| Extra memory | O(1), in place |
| Stable? | No (long-distance swaps) |
The minimum (1) is swapped with 2ₐ at the front: 2ₐ ends up behind 2ᵦ.
Not adaptive: it spends n(n − 1)/2 comparisons even on a sorted array.
Simple Algorithms · Comparison
Bubble, Insertion, Selection: Same Input
Cells: comparisons / swaps (or shifts), n = 6. Values were counted by running the code.
| Algorithm | Move type | [3, 1, 5, 2, 6, 4] | sorted 1…6 | reverse 6…1 |
|---|---|---|---|---|
| Bubble (with flag) | swap | 12 / 5 | 5 / 0 | 15 / 15 |
| Insertion | shift | 9 / 5 | 5 / 0 | 15 / 15 |
| Selection | swap | 15 / 5 | 15 / 0 | 15 / 3 |
- In bubble and insertion sort the number of moves = the number of inversions (5 for [3, 1, 5, 2, 6, 4]); a swap is 3 assignments, a shift is 1.
- Selection sort always makes n(n − 1)/2 = 15 comparisons regardless of the input, but it makes the fewest swaps.
- On sorted input, flagged bubble sort and insertion sort make only n − 1 = 5 comparisons (adaptive).
Shell Sort
Shell Sort
- D. Shell (1959): a generalization of insertion sort.
- First, items a gap apart are insertion-sorted; small items far away jump a long distance in one move.
- The gap shrinks (n/2, n/4, …); the last pass has gap = 1, i.e. plain insertion sort — but by now the array is nearly sorted.
- In place, unstable; the time depends on the gap sequence: worst case Θ(n²) with n/2 gaps, O(n1.5) with Hibbard's (2ᵏ − 1).
void shellSort(int[] arr) { int n = arr.length; for (int gap = n / 2; gap > 0; gap /= 2) for (int i = gap; i < n; i++) { int key = arr[i], j; for (j = i; j >= gap && arr[j - gap] > key; j -= gap) arr[j] = arr[j - gap]; arr[j] = key; } }
Shell Sort · Step by step
Shell Sort Step by Step: [6, 5, 3, 1, 8, 7, 2, 4]
Array [6, 5, 3, 1, 8, 7, 2, 4], n = 8. Gap sequence: n/2 = 4, 2, … , 1.
gap = 4: items 4 indices apart form a group (same color): [6, 8], [5, 7], [3, 2], [1, 4].
Each group is insertion-sorted (1 shift): [6, 8], [5, 7], [2, 3], [1, 4] → [6, 5, 2, 1, 8, 7, 3, 4].
gap = 2: items 2 indices apart form a group (same color): [6, 2, 8, 3], [5, 1, 7, 4].
Each group is insertion-sorted (6 shifts): [2, 3, 6, 8], [1, 4, 5, 7] → [2, 1, 3, 4, 6, 5, 8, 7].
gap = 1: items 1 index apart form a group (same color): [2, 1, 3, 4, 6, 5, 8, 7].
gap = 1 is plain insertion sort; since the array is nearly sorted, only 3 shifts were needed. Result: [1, 2, 3, 4, 5, 6, 7, 8].
Shell Sort · Run the code step by step
Shell Sort: Code Trace, arr = [5, 2, 6, 1, 4, 3]
arr = [5, 2, 6, 1, 4, 3], n = 6. The first gap is n / 2 = 3: items 3 apart, (0, 3), (1, 4), (2, 5), are insertion-sorted as separate groups.
i = 3: key = 1. arr[0] = 5 > 1 → moves to index 3. Stop at j = 0 and write 1 there.
i = 4: key = 4. arr[1] = 2 ≤ 4: no shift, 4 stays.
i = 5: key = 3. arr[2] = 6 > 3 → moves to index 5. Stop at j = 2 and write 3 there.
gap = 3 / 2 = 1: now plain insertion sort. i = 1: key = 2. arr[0] = 1 ≤ 2: no shift, 2 stays.
i = 2: key = 3. arr[1] = 2 ≤ 3: no shift, 3 stays.
i = 3: key = 5. arr[2] = 3 ≤ 5: no shift, 5 stays.
i = 4: key = 4. arr[3] = 5 > 4 → moves to index 4. arr[2] = 3 ≤ 4, stop; write 4 there.
i = 5: key = 6. arr[4] = 5 ≤ 6: no shift, 6 stays.
gap = 1 / 2 = 0: the loop ends and the array is sorted. Only 3 shifts in total; the gap-3 pass moved the small items forward in one go.
void shellSort(int[] arr) { int n = arr.length; for (int gap = n / 2; gap > 0; gap /= 2) for (int i = gap; i < n; i++) { int key = arr[i], j; for (j = i; j >= gap && arr[j - gap] > key; j -= gap) arr[j] = arr[j - gap]; arr[j] = key; } }
Merge Sort
Merge Sort
- A divide-and-conquer algorithm.
- Invented by John von Neumann in 1945.
- O(n log n) even in the worst case.
- Stable; merging needs extra memory (O(n)).
1. Divide: split the list into two sublists, and keep splitting until each has one item.
2. Conquer: sort each half the same way (recursively).
3. Combine: merge the sorted sublists until a single sorted list remains.
Merge Sort
Example: [38, 27, 43, 3, 9, 82, 10]
Seven items: since mid = ⌊(left + right)/2⌋, the left half gets one extra item ([38, 27, 43, 3] and [9, 82, 10]).
Merge Sort · Step by step
Merge Sort Step by Step: [6, 5, 3, 1, 8, 7, 2, 4]
Array [6, 5, 3, 1, 8, 7, 2, 4]: first split down to single items, then the sorted pieces are merged.
Split: [6, 5, 3, 1, 8, 7, 2, 4] → [6, 5, 3, 1] and [8, 7, 2, 4].
Split: [6, 5, 3, 1] → [6, 5] and [3, 1].
Split: [6, 5] → [6] and [5]. One-item pieces are already sorted.
Merge: [6] + [5] → [5, 6].
Split: [3, 1] → [3] and [1]. One-item pieces are already sorted.
Merge: [3] + [1] → [1, 3].
Merge: [5, 6] + [1, 3] → [1, 3, 5, 6].
Split: [8, 7, 2, 4] → [8, 7] and [2, 4].
Split: [8, 7] → [8] and [7]. One-item pieces are already sorted.
Merge: [8] + [7] → [7, 8].
Split: [2, 4] → [2] and [4]. One-item pieces are already sorted.
Merge: [2] + [4] → [2, 4].
Merge: [7, 8] + [2, 4] → [2, 4, 7, 8].
Merge: [1, 3, 5, 6] + [2, 4, 7, 8] → [1, 2, 3, 4, 5, 6, 7, 8]. The array is sorted.
Merge Sort
Merge Sort: Java Code
void mergeSort(int[] arr, int left, int right) { if (left < right) { int mid = left + (right - left) / 2; mergeSort(arr, left, mid); mergeSort(arr, mid + 1, right); merge(arr, left, mid, right); } }
void merge(int[] A, int p, int q, int r) { int[] L = Arrays.copyOfRange(A, p, q + 1); int[] R = Arrays.copyOfRange(A, q + 1, r + 1); int i = 0, j = 0, k = p; while (i < L.length && j < R.length) { if (L[i] <= R[j]) A[k++] = L[i++]; else A[k++] = R[j++]; } while (i < L.length) A[k++] = L[i++]; while (j < R.length) A[k++] = R[j++]; }
Writing left + (right − left) / 2 avoids the int overflow in (left + right) / 2.
Merge Sort · Run the code step by step
merge(): Merging Two Sorted Arrays
Copy L = A[p..q] = [1, 3, 5, 6] and R = A[q+1..r] = [2, 4, 7, 8] (both sorted). i = j = 0, k = p = 0.
L[0] = 1 ≤ R[0] = 2 → A[0] = 1; i++, k++.
L[1] = 3 > R[0] = 2 → A[1] = 2; j++, k++.
L[1] = 3 ≤ R[1] = 4 → A[2] = 3; i++, k++.
L[2] = 5 > R[1] = 4 → A[3] = 4; j++, k++.
L[2] = 5 ≤ R[2] = 7 → A[4] = 5; i++, k++.
L[3] = 6 ≤ R[2] = 7 → A[5] = 6; i++, k++.
L is exhausted (i = 4); the remaining R items are copied in order: A[6] = 7.
A[7] = 8. Merge done: [1, 2, 3, 4, 5, 6, 7, 8] (8 items, at most 7 comparisons).
void merge(int[] A, int p, int q, int r) { int[] L = Arrays.copyOfRange(A, p, q + 1); int[] R = Arrays.copyOfRange(A, q + 1, r + 1); int i = 0, j = 0, k = p; while (i < L.length && j < R.length) { if (L[i] <= R[j]) A[k++] = L[i++]; else A[k++] = R[j++]; } while (i < L.length) A[k++] = L[i++]; while (j < R.length) A[k++] = R[j++]; }
Merge Sort
Merge Sort: Analysis
Recurrence:
T(n) = 2 T(n/2) + Θ(n)
- The total merge work on each level is Θ(n).
- The number of levels is log₂ n (halving n down to 1).
- Total: Θ(n log n) — best = average = worst.
- Extra memory Θ(n) (auxiliary arrays) + a Θ(log n) recursion stack.
L[i] <= R[j]: ties are taken from the left → stable.
Quicksort
Quicksort
- C. A. R. Hoare (1959). Also divide and conquer, but most of the work happens in the divide step.
- Choose a pivot item.
- Split the array into two parts around the pivot (partitioning): ≤ x on the left, ≥ x on the right.
- Sort the two parts recursively. No merge step is needed.
Quicksort
The Partitioning Procedure
Lomuto: pivot = last item
partition(A, p, q): pivot = A[q] // choose pivot i = p - 1 // end of ≤ zone for j = p to q - 1: if A[j] <= pivot: i = i + 1 swap(A[i], A[j]) swap(A[i + 1], A[q]) // place the pivot return i + 1 // pivot index
Same idea: pivot = first item
PARTITION(A, p, q):
x = A[p] // pivot
i = p
for j = p + 1 to q:
if A[j] <= x:
i = i + 1
swap(A[i], A[j])
swap(A[p], A[i])
return iInvariant: A[p..i] ≤ pivot, A[i+1..j−1] > pivot, A[j..] not yet examined. One pass: Θ(n) time, O(1) extra memory.
Quicksort · Step by step
Partitioning: [6, 10, 13, 5, 8, 3, 2, 11]
Pivot x = A[p] = 6 (yellow). i = p; j starts at p + 1. Blue: ≤ x, red: > x, white: not yet seen.
A[j] = 10 > 6 → only j advances; the red zone grows.
A[j] = 13 > 6 → only j advances; the red zone grows.
A[j] = 5 ≤ 6 → i = 1; A[i] ↔ A[j]: [6, 5, 13, 10, 8, 3, 2, 11]. The blue zone grows by one.
A[j] = 8 > 6 → only j advances; the red zone grows.
A[j] = 3 ≤ 6 → i = 2; A[i] ↔ A[j]: [6, 5, 3, 10, 8, 13, 2, 11]. The blue zone grows by one.
A[j] = 2 ≤ 6 → i = 3; A[i] ↔ A[j]: [6, 5, 3, 2, 8, 13, 10, 11]. The blue zone grows by one.
A[j] = 11 > 6 → only j advances; the red zone grows.
The loop is done. The pivot is swapped with the last item of the blue zone: [2, 5, 3, 6, 8, 13, 10, 11]. Pivot 6 is now in its final place (index 3); everything to its left is ≤ 6, to its right ≥ 6.
Quicksort · Run the code step by step
partition(): Code Trace, arr = [2, 8, 7, 1, 3, 5, 6, 4]
pivot = arr[hi] = 4; i = lo − 1 = −1 (the ≤ pivot zone is empty).
j = 0: arr[0] = 2 ≤ 4? Yes.
i = 0; swap(arr, 0, 0) → [2, 8, 7, 1, 3, 5, 6, 4] (with itself).
j = 1: arr[1] = 8 ≤ 4? No → j advances.
j = 2: arr[2] = 7 ≤ 4? No → j advances.
j = 3: arr[3] = 1 ≤ 4? Yes.
i = 1; swap(arr, 1, 3) → [2, 1, 7, 8, 3, 5, 6, 4].
j = 4: arr[4] = 3 ≤ 4? Yes.
i = 2; swap(arr, 2, 4) → [2, 1, 3, 8, 7, 5, 6, 4].
j = 5: arr[5] = 5 ≤ 4? No → j advances.
j = 6: arr[6] = 6 ≤ 4? No → j advances.
The loop is done. swap(arr, i+1 = 3, hi = 7) → [2, 1, 3, 4, 7, 5, 6, 8].
return 3: pivot 4 is in its final place; left [0..2] ≤ 4, right [4..7] > 4.
int partition(int[] arr, int lo, int hi) { int pivot = arr[hi]; int i = lo - 1; for (int j = lo; j < hi; j++) { if (arr[j] <= pivot) { i++; swap(arr, i, j); } } swap(arr, i + 1, hi); return i + 1; }
Quicksort
Quicksort: Recursive Structure
QUICKSORT(A, p, r)
if p < r
q = PARTITION(A, p, r)
QUICKSORT(A, p, q − 1)
QUICKSORT(A, q + 1, r)
Initial call: QUICKSORT(A, 1, n)void quickSort(int[] arr, int lo, int hi) { if (lo < hi) { int p = partition(arr, lo, hi); quickSort(arr, lo, p - 1); quickSort(arr, p + 1, hi); } }
- The pivot (index q) is included in neither recursive call: it is already in place.
- Base case: a subarray of 0 or 1 items (p ≥ r) is already sorted.
- The Java version is 0-based: the initial call is
quickSort(arr, 0, n − 1).
Quicksort · Step by step
Quicksort Step by Step: [6, 10, 13, 5, 8, 3, 2, 11]
Array [6, 10, 13, 5, 8, 3, 2, 11]. Each call partitions its subarray with pivot = first item, then sorts both sides recursively.
Depth 0: [6, 10, 13, 5, 8, 3, 2, 11] is partitioned around 6 → the pivot lands at index 3; then [0..2] and [4..7] are sorted.
Depth 1: [2, 5, 3] is partitioned around 2 → the pivot lands at index 0; then [0..-1] and [1..2] are sorted.
Depth 2: [5, 3] is partitioned around 5 → the pivot lands at index 2; then [1..1] and [3..2] are sorted.
Depth 3: [1..1] has one item (3) → already sorted (p < r fails).
Depth 1: [8, 13, 10, 11] is partitioned around 8 → the pivot lands at index 4; then [4..3] and [5..7] are sorted.
Depth 2: [13, 10, 11] is partitioned around 13 → the pivot lands at index 7; then [5..6] and [8..7] are sorted.
Depth 3: [11, 10] is partitioned around 11 → the pivot lands at index 6; then [5..5] and [7..6] are sorted.
Depth 4: [5..5] has one item (10) → already sorted (p < r fails).
All calls are done; there is no merge step, and the array was sorted in place: [2, 3, 5, 6, 8, 10, 11, 13].
Quicksort · Run the code step by step
quickSort: Tracing the Calls, arr = [5, 2, 8, 1, 9, 3]
quickSort(0, 5): lo < hi. partition splits around pivot = 3 (last item): [2, 1] on the left, [5, 9, 8] on the right; the pivot lands at index p = 2. Then line 4: the left part.
quickSort(0, 1): lo < hi. partition splits around pivot = 1 (last item): [] on the left, [2] on the right; the pivot lands at index p = 0.
quickSort(0, -1): lo > hi, empty part → returns at once (base case).
quickSort(1, 1): the one-item part [2] is already sorted → returns. Call (0, 1) is done; line 5: (3, 5).
quickSort(3, 5): lo < hi. partition splits around pivot = 8 (last item): [5] on the left, [9] on the right; the pivot lands at index p = 4.
quickSort(3, 3): the one-item part [5] is already sorted → returns.
quickSort(5, 5): the one-item part [9] is already sorted → returns.
All calls have returned: arr = [1, 2, 3, 5, 8, 9]. 3 partition calls and 4 base-case calls; no pivot entered a subcall, each landed in its own place.
void quickSort(int[] arr, int lo, int hi) { if (lo < hi) { int p = partition(arr, lo, hi); quickSort(arr, lo, p - 1); quickSort(arr, p + 1, hi); } }
Quicksort
Quicksort: Analysis and Pivot Choice
| Case | When? | Recurrence | Time |
|---|---|---|---|
| Best | pivot is always the median | T(n) = 2T(n/2) + Θ(n) | Θ(n log n) |
| Average | random input / random pivot | expected ≈ 1.39 n log₂ n comparisons | Θ(n log n) |
| Worst | pivot always the min/max (e.g. a sorted array + last-item pivot) | T(n) = T(n − 1) + Θ(n) | Θ(n²) |
In place (O(log n) stack on average), unstable. Being cache-friendly, it is usually the fastest comparison sort in practice (Java Arrays.sort(int[]): dual-pivot quicksort).
Heapsort
Heapsort
- Sorts using the heap data structure; O(n log n) on average and in the worst case.
- Max-heap: every node is greater than or equal to its children → the largest item is at the root.
- The tree is not a separate structure; it is the array itself: the children of arr[i] are
2i + 1and2i + 2, its parent(i − 1) / 2.
- Turn the array into a max-heap.
- Move the largest item (the root) to the end of the array.
- Restore the max-heap on the rest (sift the root down).
- Repeat until all items are sorted.
Heapsort
Example: [4, 10, 3, 5, 1]
| Step | Array (heap | sorted) | Explanation |
|---|---|---|
| Array | [4, 10, 3, 5, 1] | |
| Build max-heap | [10, 5, 3, 4, 1] | root = largest |
| Root ↔ arr[4] | [1, 5, 3, 4] | [10] | 10 to the sorted part |
| Restore max-heap | [5, 4, 3, 1] | [10] | root sifted down |
| Root ↔ arr[3] | [1, 4, 3] | [5, 10] | 5 to the sorted part |
| Restore max-heap | [4, 1, 3] | [5, 10] | root sifted down |
| Root ↔ arr[2] | [3, 1] | [4, 5, 10] | 4 to the sorted part |
| Restore max-heap | [3, 1] | [4, 5, 10] | root sifted down |
| Root ↔ arr[1] | [1] | [3, 4, 5, 10] | 3 to the sorted part |
| Result | [1, 3, 4, 5, 10] |
Heapsort · Step by step
Phase 1: Building the Max-Heap — [6, 5, 3, 1, 8, 7, 2, 4]
View the array [6, 5, 3, 1, 8, 7, 2, 4] as a complete binary tree: the children of arr[i] are 2i+1 and 2i+2. Not a heap yet.
heapify(i = 3): 1 is smaller than its child → swap (1 ↔ 4). Array: [6, 5, 3, 4, 8, 7, 2, 1].
heapify(i = 2): 3 is smaller than its larger child → swap (3 ↔ 7). Array: [6, 5, 7, 4, 8, 3, 2, 1].
heapify(i = 1): 5 is smaller than its larger child → swap (5 ↔ 8). Array: [6, 8, 7, 4, 5, 3, 2, 1].
heapify(i = 0): 6 is smaller than its children → swap (6 ↔ 8). Array: [8, 6, 7, 4, 5, 3, 2, 1].
All internal nodes (i = 3 … 0) are done: max-heap [8, 6, 7, 4, 5, 3, 2, 1]. The largest item (8) is at the root.
Heapsort · Step by step
Phase 2: Move the Root to the End, Sift Down
Max-heap: [8, 6, 7, 4, 5, 3, 2, 1]. Each round swaps the root (the largest) with the heap's last item, and the heap shrinks by one.
Root 8 ↔ arr[7] = 1: 8 joins the sorted part (green). Heap size 7.
heapify(0) sifts the root down (1 ↔ 7; 1 ↔ 3) → heap: [7, 6, 3, 4, 5, 1, 2].
Root 7 ↔ arr[6] = 2: 7 joins the sorted part (green). Heap size 6.
heapify(0) sifts the root down (2 ↔ 6; 2 ↔ 5) → heap: [6, 5, 3, 4, 2, 1].
Root 6 ↔ arr[5] = 1: 6 joins the sorted part (green). Heap size 5.
heapify(0) sifts the root down (1 ↔ 5; 1 ↔ 4) → heap: [5, 4, 3, 1, 2].
Root 5 ↔ arr[4] = 2: 5 joins the sorted part (green). Heap size 4.
heapify(0) sifts the root down (2 ↔ 4) → heap: [4, 2, 3, 1].
Root 4 ↔ arr[3] = 1: 4 joins the sorted part (green). Heap size 3.
heapify(0) sifts the root down (1 ↔ 3) → heap: [3, 2, 1].
Root 3 ↔ arr[2] = 1: 3 joins the sorted part (green). Heap size 2.
heapify(0) sifts the root down (1 ↔ 2) → heap: [2, 1].
Root 2 ↔ arr[1] = 1: 2 joins the sorted part (green). Heap size 1.
One item is left in the heap, and it is in place. Sorted array: [1, 2, 3, 4, 5, 6, 7, 8].
Heapsort
Heapsort: Java Code and Analysis
void heapSort(int[] arr) { int n = arr.length; for (int i = n / 2 - 1; i >= 0; i--) // 1) build heap heapify(arr, n, i); for (int i = n - 1; i > 0; i--) { // 2) sort swap(arr, 0, i); // max to the end heapify(arr, i, 0); // sift root down } } void heapify(int[] arr, int n, int i) { while (true) { int largest = i, left = 2 * i + 1, right = 2 * i + 2; if (left < n && arr[left] > arr[largest]) largest = left; if (right < n && arr[right] > arr[largest]) largest = right; if (largest == i) return; // in place swap(arr, i, largest); i = largest; // keep sifting } }
- Building the heap: the leaves (i ≥ n/2) are already one-item heaps; sift down from i = n/2 − 1 toward the root. Total O(n) (most nodes are near the bottom).
- Sorting: n − 1 times, move the root to the end and sift down in O(log n) → O(n log n).
- O(n log n) even in the worst case, with O(1) extra memory (in place).
- Unstable: the root–last swap can reorder equal keys.
A heap can also be built by inserting items one by one (sift up); that too gives a valid max-heap, but takes O(n log n). The bottom-up construction used here (Floyd's method) is O(n).
Heapsort · Run the code step by step
Building a Max-Heap: Code Trace, arr = [3, 9, 2, 1, 4, 5]
arr = [3, 9, 2, 1, 4, 5]. The leaves (indices 3, 4, 5) are already one-item heaps. Building runs from i = n / 2 − 1 = 2 toward the root.
i = 2 (2): left = 5 (5) > 2 → largest = left; no right child.
largest ≠ i: swap(arr, 2, 5) → 5 moves up, 2 moves down; the loop continues with i = 5 (line 11).
i = 5 (2): no children (left = 11 ≥ n). largest == i → return, sifting ends.
Loop i = 1: heapify(arr, 6, 1).
i = 1 (9): left = 3 (1) is not > 9; right = 4 (4) is not > 9. No change: it is already larger than its children.
Loop i = 0: heapify(arr, 6, 0).
i = 0 (3): left = 1 (9) > 3 → largest = left; right = 2 (5) is not > 9.
largest ≠ i: swap(arr, 0, 1) → 9 moves up, 3 moves down; the loop continues with i = 1 (line 11).
i = 1 (3): left = 3 (1) is not > 3; right = 4 (4) > 3 → largest = right.
largest ≠ i: swap(arr, 1, 4) → 4 moves up, 3 moves down; the loop continues with i = 4 (line 11).
i = 4 (3): no children (left = 9 ≥ n). largest == i → return, sifting ends.
i = −1: the loop ends. [9, 4, 5, 1, 3, 2] is a max-heap: every parent is larger than its children, and the largest item, 9, is at the root.
for (int i = n / 2 - 1; i >= 0; i--) heapify(arr, n, i); void heapify(int[] arr, int n, int i) { while (true) { int largest = i, left = 2 * i + 1, right = 2 * i + 2; if (left < n && arr[left] > arr[largest]) largest = left; if (right < n && arr[right] > arr[largest]) largest = right; if (largest == i) return; // in place swap(arr, i, largest); i = largest; // keep sifting } }
Linear-Time Sorting
A Lower Bound for Comparison Sorting
- Any algorithm that only asks "ai ≤ aj?" can be drawn as a decision tree.
- Each leaf is an ordering (a permutation); to be correct the tree needs n! leaves.
- A binary tree of height h has at most 2h leaves: 2h ≥ n! ⇒ h ≥ log₂(n!) = Ω(n log n).
- Consequence: merge sort and heapsort are asymptotically optimal.
- Going faster needs information beyond comparisons: small integer keys → counting, radix.
Linear-Time Sorting
Counting Sort
- Items are never compared with each other.
- Input: A[1 … n], A[j] ∈ {1, 2, …, k}.
- Output: B[1 … n], sorted.
- Temporary: counters C[1 … k].
Idea: if we know how many items have key ≤ i, the last item with key i goes exactly at that position.
for i = 1 to k C[i] = 0 for j = 1 to n C[A[j]] = C[A[j]] + 1 // C[i] = |{key = i}| for i = 2 to k C[i] = C[i] + C[i−1] // C[i] = |{key ≤ i}| for j = n downto 1 B[C[A[j]]] = A[j] C[A[j]] = C[A[j]] − 1
Linear-Time Sorting · Step by step
Counting Sort Step by Step: A = [4, 1, 3, 4, 3]
Input A = [4, 1, 3, 4, 3], n = 5, keys {1, …, 4}. C: counter array (k = 4), B: output.
Loop 1: for i = 1 to k: C[i] = 0 — all counters are reset. Θ(k).
Loop 2, j = 1: A[1] = 4 → C[4]++ = 1. (C[i] = count of key i)
Loop 2, j = 2: A[2] = 1 → C[1]++ = 1. (C[i] = count of key i)
Loop 2, j = 3: A[3] = 3 → C[3]++ = 1. (C[i] = count of key i)
Loop 2, j = 4: A[4] = 4 → C[4]++ = 2. (C[i] = count of key i)
Loop 2, j = 5: A[5] = 3 → C[3]++ = 2. (C[i] = count of key i)
Loop 3, i = 2: C[2] = C[2] + C[1] = 1. (C[i] = count of keys ≤ i)
Loop 3, i = 3: C[3] = C[3] + C[2] = 3. (C[i] = count of keys ≤ i)
Loop 3, i = 4: C[4] = C[4] + C[3] = 5. (C[i] = count of keys ≤ i)
The prefix sums are ready: C = [1, 1, 3, 5]. C[i] is the position in B of the last item with key i.
Loop 4, j = 5: A[5] = 3 → B[C[3] = 3] = 3; C[3]−− = 2.
Loop 4, j = 4: A[4] = 4 → B[C[4] = 5] = 4; C[4]−− = 4.
Loop 4, j = 3: A[3] = 3 → B[C[3] = 2] = 3; C[3]−− = 1.
Loop 4, j = 2: A[2] = 1 → B[C[1] = 1] = 1; C[1]−− = 0.
Loop 4, j = 1: A[1] = 4 → B[C[4] = 4] = 4; C[4]−− = 3.
Result B = [1, 3, 3, 4, 4]. No comparisons were made; scanning from back to front keeps equal keys in order (stable).
Linear-Time Sorting
Counting Sort: Complexity and Java
| Loop | Work | Time |
|---|---|---|
| 1 | C[1..k] = 0 | Θ(k) |
| 2 | counting | Θ(n) |
| 3 | prefix sums | Θ(k) |
| 4 | place into B | Θ(n) |
| Total | Θ(n + k) | |
int[] countingSort(int[] arr) { // elements ≥ 0 if (arr.length == 0) return arr; int max = Arrays.stream(arr).max().getAsInt(); int[] count = new int[max + 1]; for (int x : arr) count[x]++; for (int i = 1; i <= max; i++) count[i] += count[i - 1]; int[] output = new int[arr.length]; for (int i = arr.length - 1; i >= 0; i--) { output[count[arr[i]] - 1] = arr[i]; count[arr[i]]--; } return output; }
- If k = O(n) it is Θ(n): it beats the Ω(n log n) bound because it makes no comparisons.
- Extra memory Θ(n + k); impractical if k is huge (e.g. 32-bit integers). The Java version handles only non-negative integers and uses the range 0 … max.
- Scanning the last loop from back to front makes it stable → the basis of radix sort.
Linear-Time Sorting
Radix Sort
- Sorts non-negative integers digit by digit, starting from the least significant digit (the ones): LSD.
- Each pass does a stable sort (counting sort, k = 10) on that digit.
- Stability is essential: sorting by the tens digit must preserve the order of the ones.
- d digits, base b: Θ(d · (n + b)). Θ(n) if d is constant.
- Extra memory Θ(n + b); not in place.
int[] radixSort(int[] arr) { // elements ≥ 0 if (arr.length == 0) return arr; int max = Arrays.stream(arr).max().getAsInt(); for (long exp = 1; max / exp > 0; exp *= 10) arr = countByDigit(arr, (int) exp); return arr; } int[] countByDigit(int[] arr, int exp) { int n = arr.length; int[] output = new int[n], count = new int[10]; for (int x : arr) count[(x / exp) % 10]++; for (int i = 1; i < 10; i++) count[i] += count[i - 1]; for (int i = n - 1; i >= 0; i--) { int b = (arr[i] / exp) % 10; output[--count[b]] = arr[i]; } return output; }
Linear-Time Sorting · Step by step
Radix Sort Step by Step
Input [170, 45, 75, 90, 802, 24, 2, 66]. The largest number is 802: 3 digits → 3 passes (starting from the least significant digit, LSD).
Pass 1: each number's ones digit (shown below) is the key; a missing digit counts as 0.
After the stable counting sort: [170, 90, 802, 2, 24, 45, 75, 66]. Numbers with equal digits kept their previous order.
Pass 2: each number's tens digit (shown below) is the key; a missing digit counts as 0.
After the stable counting sort: [802, 2, 24, 45, 66, 170, 75, 90]. Numbers with equal digits kept their previous order.
Pass 3: each number's hundreds digit (shown below) is the key; a missing digit counts as 0.
After the stable counting sort: [2, 24, 45, 66, 75, 90, 170, 802]. All digits processed → sorted.
Linear-Time Sorting
Radix Sort: Why Stability Is Needed
- The green columns are the suffixes sorted so far: after k passes the array is sorted by its last k digits.
- In pass 2, 167 and 467 have the same tens digit (6); the order from pass 1 (7 = 7, input order) is preserved.
- If the inner sort were unstable, 273 and 276, say, could swap in the last pass and ruin the result.
Linear-Time Sorting · Run the code step by step
countByDigit(arr, 1): arr = [32, 15, 41, 23, 11]
exp = 1: sorting by the ones digit. count[0..9] is zero, output is empty.
x = 32: digit (32 / 1) % 10 = 2 → count[2]++ = 1.
x = 15: digit (15 / 1) % 10 = 5 → count[5]++ = 1.
x = 41: digit (41 / 1) % 10 = 1 → count[1]++ = 1.
x = 23: digit (23 / 1) % 10 = 3 → count[3]++ = 1.
x = 11: digit (11 / 1) % 10 = 1 → count[1]++ = 2.
Prefix sums: count[b] is now the number of items with digit ≤ b; e.g. count[3] = 4. This is one more than the position in output of the last item with that digit.
i = 4: x = 11, b = 1. --count[1] = 1 → output[1] = 11.
i = 3: x = 23, b = 3. --count[3] = 3 → output[3] = 23.
i = 2: x = 41, b = 1. --count[1] = 0 → output[0] = 41. 41 and 11 share a digit; scanning right to left puts 41 (earlier in the input) to the left of 11: stable.
i = 1: x = 15, b = 5. --count[5] = 4 → output[4] = 15.
i = 0: x = 32, b = 2. --count[2] = 2 → output[2] = 32.
output = [41, 11, 32, 23, 15]: sorted by the ones digit. The next pass, exp = 10, looks at the tens digit; stability preserves the order of the ones.
int[] countByDigit(int[] arr, int exp) { int n = arr.length; int[] output = new int[n], count = new int[10]; for (int x : arr) count[(x / exp) % 10]++; for (int i = 1; i < 10; i++) count[i] += count[i - 1]; for (int i = n - 1; i >= 0; i--) { int b = (arr[i] / exp) % 10; output[--count[b]] = arr[i]; } return output; }
Linear-Time Sorting
Bucket Sort
- Suppose the input consists of numbers uniformly distributed over [0, 1).
- Create n buckets; a value x goes into bucket
⌊n · x⌋. - Sort each bucket (with insertion sort) and concatenate the buckets in order.
- Expected time Θ(n); worst case Θ(n²) if every item lands in one bucket.
- The tenth algorithm in the simulator; it makes an assumption about the distribution.
Timsort
Timsort: A Hybrid Sort
- Tim Peters (2002, Python). Java uses Timsort for object arrays (
Arrays.sort(Object[]),List.sort) and Python forsorted. - It exploits the sorted stretches (runs) common in real data: short runs are extended with insertion sort, and runs are combined with merge sort.
- Stable; O(n log n) worst case, Θ(n) on sorted input; O(n) extra memory.
static final int MIN_RUN = 4; void timSort(int[] arr) { int n = arr.length; for (int i = 0; i < n; i += MIN_RUN) // 1) runs insertionSort(arr, i, Math.min(i + MIN_RUN - 1, n - 1)); for (int size = MIN_RUN; size < n; size *= 2) // 2) merge for (int left = 0; left < n; left += 2 * size) { int mid = left + size - 1; int right = Math.min(left + 2 * size - 1, n - 1); if (mid < right) merge(arr, left, mid, right); } }
This version is simplified with fixed-length runs (real Timsort uses minrun 32–64, natural run detection and "galloping").
Timsort · Step by step
Timsort Step by Step (MIN_RUN = 4)
Array [5, 21, 7, 23, 19, 10, 3, 12, 8, 1], n = 10. The array is split into runs of 4 (colors).
Run [0..3] sorted with insertion sort: [5, 7, 21, 23].
Run [4..7] sorted with insertion sort: [3, 10, 12, 19].
Run [8..9] sorted with insertion sort: [1, 8].
size = 4: [5, 7, 21, 23] + [3, 10, 12, 19] merged → [3, 5, 7, 10, 12, 19, 21, 23].
size = 4: run [8..9] has no partner to merge with (mid = 11 ≥ right = 9) → stays as is.
size = 8: [3, 5, 7, 10, 12, 19, 21, 23] + [1, 8] merged → [1, 3, 5, 7, 8, 10, 12, 19, 21, 23]. The array is sorted.
Comparisons
Comparing the Sorting Algorithms
| Algorithm | Best | Average | Worst | Extra memory | Stable | In place |
|---|---|---|---|---|---|---|
| Bubble (with flag) | Θ(n) | Θ(n²) | Θ(n²) | O(1) | Yes | Yes |
| Insertion | Θ(n) | Θ(n²) | Θ(n²) | O(1) | Yes | Yes |
| Selection | Θ(n²) | Θ(n²) | Θ(n²) | O(1) | No | Yes |
| Shell (n/2 gaps) | Θ(n log n) | depends on gaps | Θ(n²) | O(1) | No | Yes |
| Merge | Θ(n log n) | Θ(n log n) | Θ(n log n) | Θ(n) | Yes | No |
| Quick | Θ(n log n) | Θ(n log n) | Θ(n²) | O(log n)* | No | Yes |
| Heapsort | O(n log n) | Θ(n log n) | Θ(n log n) | O(1) | No | Yes |
| Counting | Θ(n + k) | Θ(n + k) | Θ(n + k) | Θ(n + k) | Yes | No |
| Radix (LSD) | Θ(d(n + b)) | Θ(d(n + b)) | Θ(d(n + b)) | Θ(n + b) | Yes | No |
| Bucket | Θ(n) | Θ(n)† | Θ(n²) | Θ(n) | Yes | No |
| Timsort | Θ(n) | Θ(n log n) | Θ(n log n) | O(n) | Yes | No |
* recursion stack (if the smaller part is processed first). † expected, under a uniform distribution. k: key range, d: number of digits, b: base. Timsort's Θ(n) best case is for the real version with natural run detection.
Comparisons
How Many Comparisons? n = 1000
| Algorithm | Random | Sorted | Reverse sorted |
|---|---|---|---|
| Bubble (with flag) | 497,960 | 999 | 499,500 |
| Insertion | 252,205 | 999 | 499,500 |
| Selection | 499,500 | 499,500 | 499,500 |
| Shell (n/2) | 14,723 | 8,006 | 11,716 |
| Merge | 8,693 | 5,044 | 4,932 |
| Quick (last-item pivot) | 11,384 | 499,500 | 499,500 |
| Heapsort | 16,791 | 17,583 | 15,965 |
- Element comparisons, counted by running this chapter's code for n = 1000 (the random array was generated with a fixed seed).
- For reference: n² / 2 ≈ 500,000, n log₂ n ≈ 9,966. On sorted input, quicksort with a last-item pivot hits its worst case.
Comparisons
Which Algorithm When?
Bubble and selection sort are for teaching; selection sort only makes sense when writes are very expensive.
Summary
Summary
| Topic | Key points |
|---|---|
| Criteria | time (best/average/worst), extra memory, stability, in place, adaptivity |
| Bubble | adjacent swaps; largest at the end after each pass; best Θ(n) with a flag, otherwise Θ(n²); stable |
| Insertion | save the key, shift larger items, insert; Θ(n + inversions); best on small or nearly sorted data |
| Selection | find the minimum, move it to the front; always n(n − 1)/2 comparisons, ≤ n − 1 swaps; unstable |
| Shell | insertion sort with shrinking gaps; in place, unstable |
| Merge | split–sort–merge; T(n) = 2T(n/2) + Θ(n) = Θ(n log n); Θ(n) extra memory; stable |
| Quick | partition around a pivot; Θ(n log n) on average, Θ(n²) worst case; in place, unstable |
| Heapsort | build a max-heap in O(n), n − 1 times move the root to the end and sift down; O(n log n), O(1) extra memory |
| Linear | comparison lower bound Ω(n log n); counting Θ(n + k), radix Θ(d(n + b)), bucket Θ(n) expected |
Summary
Common Mistakes
j < n − 1 − i; writing j < n overflows at arr[j+1].key is not saved first, the first shift overwrites it.< instead of <= in merge, or scanning front to back in counting sort, reorders equal keys.quickSort(arr, lo, p) processes the pivot again; with two items left it can recurse forever.Summary
Sample Problems
- Apply bubble sort with a flag to [5, 2, 4, 6, 1, 3]. Write the array after each pass.
- How many comparisons and shifts does insertion sort need for the same array? How many swaps does selection sort make?
- What are the array and the return value after
partition(Lomuto, pivot = last item) runs on [2, 8, 7, 1, 3, 5, 6, 4]? - What max-heap does bottom-up construction build from [4, 10, 3, 5, 1]?
- For A = [2, 5, 3, 0, 2, 3, 0, 3] (keys 0 … 5), what is the array C after the third loop of counting sort?
- At most how many comparisons does merge sort need to sort 8 items?
Show answers
- [2, 4, 5, 1, 3, 6] → [2, 4, 1, 3, 5, 6] → [2, 1, 3, 4, 5, 6] → [1, 2, 3, 4, 5, 6] → [1, 2, 3, 4, 5, 6] (no swap in the last pass → stop).
- Insertion: 12 comparisons, 9 shifts (= number of inversions). Selection: 15 comparisons, 3 swaps.
- [2, 1, 3, 4, 7, 5, 6, 8], returns 3 (pivot 4 at index 3).
- [10, 5, 3, 4, 1]: first i = 1 (10 is already larger), then i = 0: 4 ↔ 10, 4 ↔ 5.
- C = [2, 2, 4, 7, 7, 8] (counts [2, 0, 2, 3, 0, 1] and their prefix sums).
- Merging m items on a level takes at most m − 1 comparisons: 4·1 + 2·3 + 1·7 = 17.
Summary
Exercises
- How many comparisons and swaps does bubble sort make on a reverse-sorted array of n = 6 items?
- Give a three-item example showing that selection sort is unstable.
- If insertion sort finds the correct position by binary search, what is its time complexity?
- What happens when quicksort (last-item pivot) is called on an already sorted array of 1000 items? How can it be fixed?
- Sort [329, 457, 657, 839, 436, 720, 355] with LSD radix sort; write the array after each pass.
- Propose an O(n log n) method for counting the inversions in an array.
Answers / hints
- Every pair is inverted on every pass: 5 + 4 + 3 + 2 + 1 = 15 comparisons and 15 swaps.
- [2ₐ, 2ᵦ, 1] → the first pass swaps 1 and 2ₐ → [1, 2ᵦ, 2ₐ].
- Comparisons drop to O(n log n), but the shifts are still Θ(n²) → Θ(n²) overall.
- Each partition splits off only the pivot: 999 + 998 + … + 1 = 499,500 comparisons and recursion 1000 deep. Use a random pivot or median of three.
- Ones: [720, 355, 436, 457, 657, 329, 839]; tens: [720, 329, 436, 839, 355, 457, 657]; hundreds: [329, 355, 436, 457, 657, 720, 839].
- Modify merge sort: whenever an item is taken from the right half, add the number of items left in the left half (L.length − i) to the inversion count; O(n log n) overall.
Summary
References
- Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., MIT Press, 2022 (Chapters 2, 6, 7, 8).
- Sedgewick and Wayne, Algorithms, 4th ed., Addison-Wesley, 2011 (Chapter 2: Sorting).
- D. E. Knuth, The Art of Computer Programming, Vol. 3: Sorting and Searching, 2nd ed., 1998.
- E. D. Demaine and C. E. Leiserson, MIT 6.046J Introduction to Algorithms lecture notes, 2005 (partitioning and counting sort examples).
- C. A. R. Hoare, "Quicksort", The Computer Journal 5(1), 1962; D. L. Shell, "A High-Speed Sorting Procedure", CACM 2(7), 1959.
- T. Peters, "listsort.txt" (Timsort design notes), CPython, 2002.