Chapter 19: Basic Sorting Algorithms
Introduction
Learning Outcomes
Distinguish records, keys, internal/external sorting, stability and in-place sorting.
Write selection, insertion and bubble sort in C and trace them step by step.
Count comparisons and exchanges; compute the best, worst and average case.
Explain when counting sort and bucket sort run in O(n).
Show why insertion sort with gaps beats plain insertion sort.
Prove the Ω(n lg n) bound for comparison sorting with a decision tree.
Sorting Concepts
Sorting: Records, Keys, Internal/External
- Records made of fields form a list when kept in memory and a file when kept outside it.
- The field(s) that identify a record uniquely form the key; sorting is done by key.
- Internal sorting: all the data is in memory. External sorting: auxiliary storage such as disk or tape is used; the number of disk accesses dominates.
Key: student number
Sorting Concepts
Stability and In-Place Sorting
Records with equal keys keep their relative order. Sort stably by first name, then by surname, and people with the same surname stay in first-name order (the basis of radix sort).
Uses only O(1) (by some definitions O(lg n)) extra memory beyond the input. Selection, insertion and bubble sort are in place; counting sort is not (arrays c and b).
Sorting Concepts
Common Setup: less and SWAP
#define N 15 #define less(A, B) ((A) < (B)) #define SWAP(A, B) do { int t_ = (A); (A) = (B); (B) = t_; } while (0)
int main(void) { int *a = malloc(N * sizeof *a); if (a == NULL) return 1; for (int i = 0; i < N; i++) a[i] = rand() % 50; selection(a, 0, N - 1); /* call the sort */ for (int i = 0; i < N; i++) printf("%d ", a[i]); free(a); return 0; }
Writing the macro as {int t = A; A=B; B=t;} fails to compile inside if (…) SWAP(…); else; do { … } while (0) closes that trap. The idiom if (less(A, B)) SWAP(A, B); swaps when A < B (compare-exchange).
Selection Sort
Selection Sort
void selection(int a[], int L, int R) { for (int i = L; i < R; i++) { int min = i; for (int j = i + 1; j <= R; j++) if (less(a[j], a[min])) min = j; SWAP(a[i], a[min]); } }
- Find the smallest element in the list.
- Swap it with the element in the first position.
- Repeat on the remaining sublist, starting from the 2nd element.
After each outer pass, a[L..i] is sorted and holds the smallest elements of the array.
To pick the minimum, the remaining sublist must be scanned to its end, even if the data is already sorted.
Selection Sort · Step through the code
Code Trace: selection(a, 0, 4)
Pass i = 0: the remaining sublist is a[0..4]. Initially the minimum's index is min = 0 (a[0] = 29).
a[1] = 10 < a[0] = 29 → new minimum: min = 1.
a[2] = 14 ≥ a[1] = 10 → min unchanged.
a[3] = 37 ≥ a[1] = 10 → min unchanged.
a[4] = 13 ≥ a[1] = 10 → min unchanged.
SWAP(a[0], a[1]): 10 is in place; a[0..0] is now sorted.
Pass i = 1: the remaining sublist is a[1..4]. Initially the minimum's index is min = 1 (a[1] = 29).
a[2] = 14 < a[1] = 29 → new minimum: min = 2.
a[3] = 37 ≥ a[2] = 14 → min unchanged.
a[4] = 13 < a[2] = 14 → new minimum: min = 4.
SWAP(a[1], a[4]): 13 is in place; a[0..1] is now sorted.
Pass i = 2: the remaining sublist is a[2..4]. Initially the minimum's index is min = 2 (a[2] = 14).
a[3] = 37 ≥ a[2] = 14 → min unchanged.
a[4] = 29 ≥ a[2] = 14 → min unchanged.
min = i, so the element is swapped with itself (no change); a[0..2] is sorted.
Pass i = 3: the remaining sublist is a[3..4]. Initially the minimum's index is min = 3 (a[3] = 37).
a[4] = 29 < a[3] = 37 → new minimum: min = 4.
SWAP(a[3], a[4]): 29 is in place; a[0..3] is now sorted.
void selection(int a[], int L, int R) { for (int i = L; i < R; i++) { int min = i; for (int j = i + 1; j <= R; j++) if (less(a[j], a[min])) min = j; SWAP(a[i], a[min]); } }
Selection Sort
Analysis of Selection Sort
(n−1) + (n−2) + … + 2 + 1 = n(n−1)/2 ≈ n²/2
- The number of comparisons is independent of the input: best = average = worst ≈ n²/2.
- The number of exchanges is n − 1: when the last element is reached, it is already in place.
- Complexity O(n²); O(1) extra memory (in place).
- Long-distance swaps make it not stable (the earlier 3a/3c example).
| n | comparisons | swaps |
|---|---|---|
| 5 | 10 | 4 |
| 10 | 45 | 9 |
| 100 | 4950 | 99 |
| 1000 | 499500 | 999 |
Insertion Sort
Insertion Sort: Two Versions
void insertion(int a[], int L, int R) { for (int i = L + 1; i <= R; i++) { int v = a[i]; // to insert int j = i - 1; // a[L..i-1] sorted while (j >= L && a[j] > v) { a[j + 1] = a[j]; // shift right j--; } a[j + 1] = v; } }
void insertion1(int a[], int L, int R) { for (int i = R; i > L; i--) // sentinel if (a[i] < a[i-1]) SWAP(a[i], a[i-1]); for (int i = L + 2; i <= R; i++) { int v = a[i], j = i; while (less(v, a[j - 1])) { a[j] = a[j - 1]; j--; } a[j] = v; } }
- Each pass takes one element from the input and inserts it at the right place in the sorted prefix (like arranging playing cards).
insertion1on the right is the improved version of the left one: it first moves the minimum toa[L]; thanks to this sentinel, the inner loop needs noj > Ltest.- Writing the condition as
j >= 0is wrong; if the subarray starts atL > 0it overruns. The correct test isj >= L.
Insertion Sort · Step through the code
Code Trace: insertion(a, 0, 4)
i = 1: v = a[1] = 2 is set aside; a[0..0] is sorted; find a place for v.
a[0] = 7 > v = 2 → 7 shifts one right (a[1]); j = -1.
j = −1, the start of the array → the loop ends; a[0] = 2. The sorted prefix is now a[0..1].
i = 2: v = a[2] = 9 is set aside; a[0..1] is sorted; find a place for v.
a[1] = 7 ≤ v → the loop ends; a[2] = 9. The sorted prefix is now a[0..2].
i = 3: v = a[3] = 5 is set aside; a[0..2] is sorted; find a place for v.
a[2] = 9 > v = 5 → 9 shifts one right (a[3]); j = 1.
a[1] = 7 > v = 5 → 7 shifts one right (a[2]); j = 0.
a[0] = 2 ≤ v → the loop ends; a[1] = 5. The sorted prefix is now a[0..3].
i = 4: v = a[4] = 1 is set aside; a[0..3] is sorted; find a place for v.
a[3] = 9 > v = 1 → 9 shifts one right (a[4]); j = 2.
a[2] = 7 > v = 1 → 7 shifts one right (a[3]); j = 1.
a[1] = 5 > v = 1 → 5 shifts one right (a[2]); j = 0.
a[0] = 2 > v = 1 → 2 shifts one right (a[1]); j = -1.
j = −1, the start of the array → the loop ends; a[0] = 1. The sorted prefix is now a[0..4].
void insertion(int a[], int L, int R) { for (int i = L + 1; i <= R; i++) { int v = a[i]; // to insert int j = i - 1; // a[L..i-1] sorted while (j >= L && a[j] > v) { a[j + 1] = a[j]; // shift right j--; } a[j + 1] = v; } }
Insertion Sort · Step through the code
Code Trace: insertion1(a, 0, 3) (with a sentinel)
a[3] = 2 ≥ a[2] = 1 → no swap. The minimum moves left.
a[2] = 1 < a[1] = 3 → swap. The minimum moves left.
a[1] = 1 < a[0] = 4 → swap. a[0] = 1 is now the minimum: the sentinel is ready.
i = 2: v = 3. a[0..1] is sorted.
v = 3 < a[1] = 4 → shift; j = 1. No bounds test: the sentinel a[0] will stop it.
a[0] = 1 ≤ v → stop; a[1] = 3.
i = 3: v = 2. a[0..2] is sorted.
v = 2 < a[2] = 4 → shift; j = 2. No bounds test: the sentinel a[0] will stop it.
v = 2 < a[1] = 3 → shift; j = 1. No bounds test: the sentinel a[0] will stop it.
a[0] = 1 ≤ v → stop; a[1] = 2.
void insertion1(int a[], int L, int R) { for (int i = R; i > L; i--) // sentinel if (a[i] < a[i-1]) SWAP(a[i], a[i-1]); for (int i = L + 2; i <= R; i++) { int v = a[i], j = i; while (less(v, a[j - 1])) { a[j] = a[j - 1]; j--; } a[j] = v; } }
Insertion Sort
Insertion Sort: Best and Worst Case
On sorted input, each element is compared only with the rightmost value of the sorted list; it is larger, so it stays put: n − 1 comparisons in total.
On reverse-sorted input, each element is compared with the whole sorted list and inserted at the far left; the sorted part shifts right every pass: 1 + 2 + … + (n−1) = n(n−1)/2.
a[j] > v is strict, so equal elements are not passed) and in place. On nearly sorted data it is almost linear.Insertion Sort
Average-Case Analysis: the Possible Cases for j = 3
Case 1: 8 > 10 is false → 1 comparison.
Case 2: 8 > 7 true, 6 > 7 false → 2 comparisons.
Case 3: 8 > 5 true, 6 > 5 true, 3 > 5 false → 3 comparisons.
Case 4: 8 > 2, 6 > 2, 3 > 2 all true; at j = −1, j >= 0 is false and a[j] > v is never tested, so it costs the same as the previous case → 3 comparisons.
Insertion Sort
Average Case: the Expected Number of Comparisons
E(Cj) = (1/(j+1))·[2j + Σk=1j−1 k] = (j+2)/2 − 1/(j+1)
E(Y) = Σj=1N−1 E(Cj) = (N² + 3N)/4 − HN ≈ N²/4
- Cj, the number of comparisons to insert element j, is a random variable; E(Cj) is its expected value.
- HN = 1 + 1/2 + … + 1/N is the harmonic number: HN = ln N + O(1).
- On average ≈ N²/4 comparisons and ≈ N²/4 half-exchanges (shifts, not swaps) → O(n²).
| N | N! | all permutations | formula | N²/4 |
|---|---|---|---|---|
| 3 | 6 | 2.667 | 2.667 | 2.25 |
| 4 | 24 | 4.917 | 4.917 | 4.00 |
| 5 | 120 | 7.717 | 7.717 | 6.25 |
| 6 | 720 | 11.050 | 11.050 | 9.00 |
| 7 | 5040 | 14.907 | 14.907 | 12.25 |
Table: all N! orderings were counted in Python; they match the formula exactly.
Bubble Sort
Bubble Sort
void bubble(int a[], int L, int R) { int i, j; for (i = L; i < R; i++) for (j = R; j > i; j--) if (a[j] < a[j-1]) SWAP(a[j], a[j-1]); }
Start: [7, 2, 9, 5, 1]. Each pass compares neighbors from right to left.
Pass i = 0: the smallest remaining value 1 bubbles up to a[0] → [1, 7, 2, 9, 5].
Pass i = 1: the smallest remaining value 2 bubbles up to a[1] → [1, 2, 7, 5, 9].
Pass i = 2: the smallest remaining value 5 bubbles up to a[2] → [1, 2, 5, 7, 9].
Pass i = 3: no swap; the smallest remaining value 7 is already in a[3] → [1, 2, 5, 7, 9].
Bubble Sort · Step through the code
Code Trace: bubble2(a, 0, 4) with Early Exit
Pass i = 0 begins; the swap flag is cleared.
a[4] < a[3] → swap: [2, 1, 3, 4, 5]; swapped = 1.
a[3] = 4 ≥ a[2] = 3 → no swap.
a[2] = 3 ≥ a[1] = 1 → no swap.
a[1] < a[0] → swap: [1, 2, 3, 4, 5]; swapped = 1.
swapped = 1 → the array cannot be assumed sorted yet; go to the next pass.
Pass i = 1 begins; the swap flag is cleared.
a[4] = 5 ≥ a[3] = 4 → no swap.
a[3] = 4 ≥ a[2] = 3 → no swap.
a[2] = 3 ≥ a[1] = 2 → no swap.
No swap in this pass → the array is sorted; early exit. Two useless passes were skipped.
void bubble2(int a[], int L, int R) { for (int i = L; i < R; i++) { int swapped = 0; // any swap this pass? for (int j = R; j > i; j--) if (less(a[j], a[j - 1])) { SWAP(a[j], a[j - 1]); swapped = 1; } if (!swapped) return; // sorted: exit early } }
Bubble Sort
Analysis of Bubble Sort
- Comparisons: n(n−1)/2 ≈ n²/2 in every case (without early exit).
- Exchanges: n²/2 in the worst case (reverse sorted); on average as many as the inversions, ≈ n²/4.
- Complexity O(n²); the early-exit version is O(n) on sorted input.
- Only neighbors are swapped and equal elements never are → stable, in place.
| n | comp. | swaps (worst) | swaps (avg.) |
|---|---|---|---|
| 4 | 6 | 6 | 3 |
| 5 | 10 | 10 | 5 |
| 6 | 15 | 15 | 7.5 |
| 7 | 21 | 21 | 10.5 |
Average column: all n! orderings were counted. Some sources (e.g. Sedgewick) also give n²/2 exchanges on average; the count shows the average is n²/4.
Bubble Sort
Inversions: a Common Measure
Inversion: a pair with i < j and a[i] > a[j]. [7, 2, 9, 5, 1] → (7,2), (7,5), (7,1), (2,1), (9,5), (9,1), (5,1)
- A neighbor swap (bubble) or a one-position shift (insertion) removes exactly one inversion.
- So in both, the number of swaps/shifts = the number of inversions: 7 for this array (the traces also made 7 shifts).
- A reverse-sorted array has n(n−1)/2 inversions; a random one has n(n−1)/4 on average (each pair is inverted with probability ½).
- Every algorithm that works on neighbors is Ω(n²) on average: Shell sort and quicksort, with long-distance swaps, break through this wall.
Counting Sort
Counting Sort
int counting_sort(const int a[], int b[], int r, int k) { int *c = malloc((k + 1) * sizeof *c); if (c == NULL) return -1; for (int i = 0; i <= k; i++) c[i] = 0; // L1 for (int j = 0; j <= r; j++) c[a[j]]++; // L2 for (int i = 1; i <= k; i++) // L3 c[i] += c[i - 1]; for (int j = r; j >= 0; j--) { // L4 b[c[a[j]] - 1] = a[j]; c[a[j]]--; } free(c); return 0; }
| name | meaning |
|---|---|
| a[] | input array |
| b[] | sorted output array |
| c[] | auxiliary count array (k+1 elements) |
| r | a[0..r] is sorted (r+1 elements) |
| k | values lie in 0…k |
- Harold H. Seward, 1954. Sorts integers from a known range.
- Values are never compared with each other → a non-comparison sort (bucket sort is in this class too).
int c[K]needs a compile-time constant size; here we usemalloc+ a check +free.
Counting Sort · Step through the code
Code Trace: counting_sort, Counting (L1–L3)
L1: c[0..10] is cleared. Here we will count how many times each value occurs.
L2: a[0] = 3 → c[3]++ = 1.
L2: a[1] = 0 → c[0]++ = 1.
L2: a[2] = 6 → c[6]++ = 1.
L2: a[3] = 9 → c[9]++ = 1.
L2: a[4] = 4 → c[4]++ = 1.
L2: a[5] = 3 → c[3]++ = 2. (A second 3!)
L2: a[6] = 10 → c[10]++ = 1.
L2: a[7] = 5 → c[5]++ = 1.
L3: c[1] += c[0] → 1: “1 element is ≤ 1”.
L3: c[2] += c[1] → 1: “1 element is ≤ 2”.
L3: c[3] += c[2] → 3: “3 elements are ≤ 3”. A 3 goes to position 2.
L3: c[4] += c[3] → 4: “4 elements are ≤ 4”. A 4 goes to position 3.
L3: c[5] += c[4] → 5: “5 elements are ≤ 5”. A 5 goes to position 4.
L3: c[6] += c[5] → 6: “6 elements are ≤ 6”. A 6 goes to position 5.
L3: c[7] += c[6] → 6: “6 elements are ≤ 7”.
L3: c[8] += c[7] → 6: “6 elements are ≤ 8”.
L3: c[9] += c[8] → 7: “7 elements are ≤ 9”. A 9 goes to position 6.
L3: c[10] += c[9] → 8: “8 elements are ≤ 10”. A 10 goes to position 7.
for (int i = 0; i <= k; i++) c[i] = 0; // L1 for (int j = 0; j <= r; j++) c[a[j]]++; // L2 for (int i = 1; i <= k; i++) // L3 c[i] += c[i - 1];
Counting Sort · Step through the code
Code Trace (continued): Placing into b with L4
After L3: c[x] = “number of elements ≤ x”. E.g. c[9] = 7 → a 9 goes to position 7 − 1 = 6.
L4: a[7] = 5, c[5] = 5 → b[4] = 5; c[5]-- = 4.
L4: a[6] = 10, c[10] = 8 → b[7] = 10; c[10]-- = 7.
L4: a[5] = 3, c[3] = 3 → b[2] = 3; c[3]-- = 2 (the next equal value lands one to the left).
L4: a[4] = 4, c[4] = 4 → b[3] = 4; c[4]-- = 3.
L4: a[3] = 9, c[9] = 7 → b[6] = 9; c[9]-- = 6.
L4: a[2] = 6, c[6] = 6 → b[5] = 6; c[6]-- = 5.
L4: a[1] = 0, c[0] = 1 → b[0] = 0; c[0]-- = 0.
L4: a[0] = 3, c[3] = 2 → b[1] = 3; c[3]-- = 1 (the next equal value lands one to the left).
Result b = [0, 3, 3, 4, 5, 6, 9, 10]. Scanning right to left kept the two 3s in their relative order: stable.
for (int j = r; j >= 0; j--) { // L4 b[c[a[j]] - 1] = a[j]; c[a[j]]--; }
Counting Sort
Counting Sort: Analysis and Limits
| loop | work | cost |
|---|---|---|
| L1 | clear c | O(k) |
| L2 | count | O(n) |
| L3 | prefix sums | O(k) |
| L4 | place | O(n) |
- Total O(n + k); O(n) if k = O(n). Extra memory O(n + k): not in place.
- L4 runs right to left, so it is stable, a property radix sort requires.
Shell and Bucket Sort · Step through the code
Code Trace: shell_sort(a, 8)
Gap h = 4: elements 4 apart are insertion-sorted separately.
i = 4: v = 14; shifted back by 4 once, a[0] = 14.
i = 5: v = 19; shifted back by 4 once, a[1] = 19.
i = 6: v = 27; shifted back by 4 once, a[2] = 27.
i = 7: v = 44; a[3] ≤ 44, stays in place.
Gap h = 2: elements 2 apart are insertion-sorted separately.
i = 2: v = 27; a[0] ≤ 27, stays in place.
i = 3: v = 10; shifted back by 2 once, a[1] = 10.
i = 4: v = 35; a[2] ≤ 35, stays in place.
i = 5: v = 33; a[3] ≤ 33, stays in place.
i = 6: v = 42; a[4] ≤ 42, stays in place.
i = 7: v = 44; a[5] ≤ 44, stays in place.
Last pass h = 1: plain insertion sort, but the array is now nearly sorted.
i = 1: v = 10; shifted back by 1 once, a[0] = 10.
i = 2: v = 27; a[1] ≤ 27, stays in place.
i = 3: v = 19; shifted back by 1 once, a[2] = 19.
i = 4: v = 35; a[3] ≤ 35, stays in place.
i = 5: v = 33; shifted back by 1 once, a[4] = 33.
i = 6: v = 42; a[5] ≤ 42, stays in place.
i = 7: v = 44; a[6] ≤ 44, stays in place.
void shell_sort(int a[], int n) { for (int h = n / 2; h > 0; h /= 2) // gap for (int i = h; i < n; i++) { int v = a[i], j = i; while (j >= h && a[j - h] > v) { a[j] = a[j - h]; // shift back h j -= h; } a[j] = v; } }
Shell and Bucket Sort
Shell Sort (1959)
- The weakness of insertion sort: an element moves only one position at a time; O(n²) on reverse-sorted input.
- Shell: first sort distant elements (gap h), then shrink h; the last pass uses h = 1, so the result is always correct.
- The cost depends on the gap sequence: worst case O(n²) for h = n/2, n/4, … (Shell's); O(n3/2) for Knuth's 1, 4, 13, 40, …
- In place, but long-distance moves make it not stable.
When properties of the data are known, Shell sort and bucket sort can be used together.
Short code and good practical speed: a favorite in embedded systems and for small to medium arrays.
Shell and Bucket Sort
Bucket Sort
Input: 10 numbers in [0, 1); 10 buckets, each 0.1 wide.
Distribute: value x goes into bucket ⌊10·x⌋ (e.g. .78 → 7, .17 → 1). O(n).
Each bucket is sorted on its own (by insertion sort): cheap, since buckets are small.
Concatenate the buckets in order → sorted output. O(n) on average for a uniform distribution.
Shell and Bucket Sort
Bucket Sort: an Implementation in C
typedef struct bnode { double x; struct bnode *next; } bnode; int bucket_sort(double a[], int n) { /* 0 <= a[i] < 1 */ bnode **bucket = calloc(n, sizeof *bucket); bnode *pool = malloc(n * sizeof *pool); if (!bucket || !pool) { free(bucket); free(pool); return -1; } for (int i = 0; i < n; i++) { /* scatter + sorted insert */ bnode **p = &bucket[(int)(a[i] * n)]; while (*p && (*p)->x <= a[i]) p = &(*p)->next; pool[i].x = a[i]; pool[i].next = *p; *p = &pool[i]; } int k = 0; /* gather the buckets */ for (int b = 0; b < n; b++) for (bnode *q = bucket[b]; q; q = q->next) a[k++] = q->x; free(bucket); free(pool); return 0; }
- n buckets; the value
a[i]is inserted in order into linked list⌊a[i]·n⌋(insertion sort within the bucket). - Thanks to
<=, equal values go to the end → stable. The nodes come from a singlepoolarray: two allocations instead of n separatemallocs, and one pair offrees.
Optimal Sorting Time
Optimal Sorting Time: the Decision Tree
Tests are ellipses, decisions (orderings) rectangles. For n = 3 there are 3! = 6 leaves.
Input a = 5, b = 8, c = 2. Root: a < b? 5 < 8 → yes.
b < c? 8 < 2 → no: c must come before b.
a < c? 5 < 2 → no.
Leaf c < a < b → 2, 5, 8. For this input, 3 comparisons = the length of the path.
less() call, each leaf an ordering of the input. The worst-case number of comparisons = the tree's height.Optimal Sorting Time
Theorem: the Lower Bound for Comparison Sorting
- There are at least n! leaves (one per ordering).
- A binary tree of height h has at most 2h leaves: 2h ≥ n! ⇒ h ≥ lg(n!).
- Stirling: ⌈lg(n!)⌉ = n lg n − n lg e + Θ(lg n) ≈ n lg n − 1.4427 n.
- Result: comparison sorting is Ω(n lg n); counting, bucket and radix sort make no comparisons, so the bound does not apply to them.
| n | ⌈lg n!⌉ | n lg n − 1.44n | n(n−1)/2 |
|---|---|---|---|
| 3 | 3 | 0.4 | 3 |
| 5 | 7 | 4.4 | 10 |
| 10 | 22 | 18.8 | 45 |
| 20 | 62 | 57.6 | 190 |
| 100 | 525 | 520.1 | 4950 |
Lab · C code step by step
Lab: Basic Sorts
Summary
Counts on the Same Input (n = 1000)
| algorithm | sorted: comp. / moves | reversed: comp. / moves | random: comp. / moves |
|---|---|---|---|
| Selection | 499,500 / 999 | 499,500 / 999 | 499,500 / 999 |
| Insertion | 999 / 0 | 499,500 / 499,500 | 247,876 / 246,884 |
| Bubble (early exit) | 999 / 0 | 499,500 / 499,500 | 499,035 / 246,884 |
| Shell (n/2, n/4, …) | 8,006 / 0 | 11,716 / 4,700 | 14,760 / 7,254 |
Counted by a Python simulation (random input: seed 19). Moves: swaps for selection and bubble, shifts for insertion and Shell.
Blind to the input: 499,500 comparisons and 999 swaps every time.
999 comparisons when sorted; ≈ n²/4 on random input; n²/2 when reversed.
Far less work than insertion on random input: long shifts remove inversions fast.
Summary
Complexity Summary
| algorithm | best | average | worst | extra memory | stable | in place |
|---|---|---|---|---|---|---|
| Selection | n²/2 | n²/2 | n²/2 | O(1) | no | yes |
| Insertion | O(n) | ≈ n²/4 | n²/2 | O(1) | yes | yes |
| Bubble (early exit) | O(n) | ≈ n²/2 comp. | n²/2 | O(1) | yes | yes |
| Shell (Knuth h) | O(n lg n) | — | O(n3/2) | O(1) | no | yes |
| Counting | O(n+k) | O(n+k) | O(n+k) | O(n+k) | yes | no |
| Bucket (uniform) | O(n) | O(n) | O(n²) | O(n) | yes* | no |
For selection, insertion and bubble the figures are comparison counts. * If a stable sort is used inside each bucket. Shell's average depends on the gap sequence and has no known closed form. The lower bound for comparison sorting is ⌈lg n!⌉ ≈ n lg n.
Summary
Which One When?
Selection: exactly n − 1 swaps.
Insertion: stable, in place, O(n) when nearly sorted; also used on the small pieces in quicksort.
Bubble with early exit: detects a sorted array in one pass; rarely used in production.
Counting: O(n + k), stable; if k ≫ n, memory and time are wasted.
Shell: a few lines of code, clearly faster than O(n²).
Bucket: O(n) on average.
Summary
Common Mistakes
min).v = a[i] before shifting in insertion sort: the first shift overwrites a[i].j >= 0: for a subarray a[L..R] the bound is j >= L.if (less(A, B)) SWAP(A, B): it swaps when A < B.c[].Summary
Exercises
- Write the array after each pass of selection sort on [5, 1, 4, 2, 8]. How many swaps were made?
- How many comparisons does insertion sort make on [1, 2, 3, …, n] and on [n, …, 2, 1]?
- How many passes does
bubble2run on [1, 2, 3, 5, 4]? - For a = [2, 5, 3, 0, 2, 3, 0, 3], k = 5, write the array c after L3 of counting sort.
- What is the minimum height of a decision tree for 3 elements? For 4?
- Find the smallest example showing that selection sort is not stable.
Show answers
- [1,5,4,2,8] → [1,2,4,5,8] → [1,2,4,5,8] → [1,2,4,5,8]; 4 passes, the first two are real swaps (the other two swap an element with itself).
- n − 1 when sorted; n(n−1)/2 when reverse sorted.
- 2 passes: the first swaps 5 and 4, the second makes no swap → early exit.
- Counts [2,0,2,3,0,1] → prefix sums [2,2,4,7,7,8].
- ⌈lg 3!⌉ = ⌈lg 6⌉ = 3; ⌈lg 4!⌉ = ⌈lg 24⌉ = 5.
- [2a, 2b, 1]: the first pass swaps 1 and 2a → [1, 2b, 2a].
Summary
Chapter Summary
Same class, different character: selection makes few swaps, insertion is fast on nearly sorted data, bubble can stop early.
The expected value over all n! orderings: ≈ n²/4 comparisons for insertion. Not the mean of best and worst!
Counting O(n+k) and bucket O(n) on average: knowledge of the data (range, distribution) buys speed.
Stability (order of equal keys) and in-place operation (extra memory) drive the choice of algorithm.
Insertion sort with gaps; the final pass with h = 1 guarantees correctness.
Comparison sorting cannot beat ⌈lg n!⌉ ≈ n lg n comparisons → the touchstone for advanced sorting algorithms.
Summary
References and Next Steps
- E. Horowitz, S. Sahni, S. Anderson-Freed, Fundamentals of Data Structures in C, 2nd ed., Chapter 7 (Sorting).
- R. Sedgewick, Algorithms in C, Parts 1–4, Chapter 6 (Elementary Sorting Methods): the macro approach, Shell sort.
- T. H. Cormen et al., Introduction to Algorithms (CLRS), Chapter 2 (insertion), 8 (lower bound, counting, bucket).