Chapter 3: Sorting Algorithms

Algorithms
Stability and in-place sortingBubbleInsertionSelectionShell MergeQuickHeapsortCounting and radixTimsortComparison
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Concepts

Evaluate a sort by time, extra memory, stability, whether it is in place, and whether it is comparison-based.

Simple algorithms

Trace bubble, insertion, selection and Shell sort on an array step by step and implement them in Java.

Divide and conquer, heaps

Explain how merge sort, quicksort and heapsort work; justify their O(n log n) and worst-case behavior.

Linear time and choosing

Explain the comparison lower bound and counting and radix sort; choose the right algorithm for a problem.

Introduction

Sorting Algorithms

  • From shopping lists to social media posts, the digital world is full of data.
  • To use data efficiently we need to keep it organized.
  • Sorting algorithms arrange the items of a list according to some criterion:
    • numeric value,
    • alphabetical order,
    • date …
  • The order can be ascending (smallest first) or descending.
  • Formally: the input is ⟨a₁, a₂, …, aₙ⟩; the output is a permutation of the same numbers with a′₁ ≤ a′₂ ≤ … ≤ a′ₙ.
Unsorted72941sort (ascending)Sorted12479
Sorted data makes many tasks easy: binary search in O(log n), finding duplicates, merging, the median, the k smallest or largest items.

Introduction

Kinds of Sorting Algorithms

Different sorting algorithms work on different principles:

AlgorithmKey idea
Bubble sortCompares adjacent items and swaps them if they are out of order.
Insertion sortInserts the next item into its place in the sorted part.
Selection sortFinds the smallest item and moves it to the front, then sorts the rest.
Shell sortInsertion sort that starts with distant items and shrinks the gap.
Merge sortSplits the array in half, sorts the halves, then merges them.
QuicksortPartitions around a pivot and sorts the two parts recursively.
HeapsortBuilds a max-heap and repeatedly moves the largest item to the end.
Counting / Radix sortNo comparisons; counts the keys or distributes them digit by digit.
TimsortA hybrid of insertion sort and merge sort; the object sort of Java and Python.

Introduction

Evaluating a Sorting Algorithm

TimeComparisons and swaps (moves) in the best, average and worst case: Θ(n²), Θ(n log n), Θ(n + k) …
Extra memorySpace needed beyond the input. An algorithm that uses O(1) extra memory sorts in place.
StabilityAn algorithm is stable if items with equal keys keep their relative order.
Comparison-based?Algorithms that only ask "a ≤ b?" need Ω(n log n) comparisons in the worst case. Counting and radix sort escape this bound.
AdaptivityAn algorithm that runs faster on nearly sorted input is adaptive: insertion sort, bubble sort with a flag, Timsort.
OnlineAn algorithm that can sort data as it arrives is online: insertion sort.

These bounds count one comparison as O(1); for strings of length L each comparison can cost O(L) (see Chapters 10–11).

Introduction

Stability: An Example

Input(3, A)(1, B)(3, C)(2, D)(1, E)(3, F)3s: ACFStable (insertion)(1, B)(1, E)(2, D)(3, A)(3, C)(3, F)3s: ACFUnstable (selection)(1, B)(1, E)(2, D)(3, C)(3, A)(3, F)3s: CAF

Records are sorted by the number (the key) only; the letter identifies the record. A stable algorithm keeps the records with key 3 in input order, A, C, F; the long-distance swaps of selection sort turn this into C, A, F. (This input is the example from the simulator lab.)

Stability matters in multi-key sorting: sort students by name, then (with a stable algorithm) by department, and each department stays alphabetical. Radix sort needs a stable inner sort for the same reason.

Bubble Sort

Bubble Sort

  • On each pass the larger items move toward the end of the array, rising like bubbles in water.
  • Adjacent items are compared in pairs.
  • If the left one is larger, they are exchanged (swap).
  • After pass i, the i largest items are in their final places; the next pass is one item shorter.
  • If a pass makes no swaps, the array is sorted → early exit.
before pass 1: adjacent pairs315264after pass 1132546largest at the end

Bubble Sort · Step by step

Bubble Sort Step by Step: [3, 1, 5, 2, 6, 4]

Start301152236445

Array: [3, 1, 5, 2, 6, 4]. Each pass compares adjacent pairs from left to right; the larger one moves right.

Pass 1jj+1301152236445

Pass 1, j = 0: 3 > 1? Yes → swap.

Pass 1jj+1103152236445swap

Swap: the array is now [1, 3, 5, 2, 6, 4].

Pass 1jj+1103152236445

Pass 1, j = 1: 3 > 5? No → stays.

Pass 1jj+1103152236445

Pass 1, j = 2: 5 > 2? Yes → swap.

Pass 1jj+1103122536445swap

Swap: the array is now [1, 3, 2, 5, 6, 4].

Pass 1jj+1103122536445

Pass 1, j = 3: 5 > 6? No → stays.

Pass 1jj+1103122536445

Pass 1, j = 4: 6 > 4? Yes → swap.

Pass 1jj+1103122534465swap

Swap: the array is now [1, 3, 2, 5, 4, 6].

End of pass 1103122534465

Pass 1 done: the largest remaining item, 6, is in its final place (index 5).

Pass 2jj+1103122534465

Pass 2, j = 0: 1 > 3? No → stays.

Pass 2jj+1103122534465

Pass 2, j = 1: 3 > 2? Yes → swap.

Pass 2jj+1102132534465swap

Swap: the array is now [1, 2, 3, 5, 4, 6].

Pass 2jj+1102132534465

Pass 2, j = 2: 3 > 5? No → stays.

Pass 2jj+1102132534465

Pass 2, j = 3: 5 > 4? Yes → swap.

Pass 2jj+1102132435465swap

Swap: the array is now [1, 2, 3, 4, 5, 6].

End of pass 2102132435465

Pass 2 done: the largest remaining item, 5, is in its final place (index 4).

Pass 3jj+1102132435465

Pass 3, j = 0: 1 > 2? No → stays.

Pass 3jj+1102132435465

Pass 3, j = 1: 2 > 3? No → stays.

Pass 3jj+1102132435465

Pass 3, j = 2: 3 > 4? No → stays.

End of pass 3102132435465

Pass 3 made no swaps → the array is sorted and the algorithm stops early. Result: [1, 2, 3, 4, 5, 6].

Bubble Sort

Bubble Sort: Java Code

boolean swapped;
for(int i = 0; i < n - 1; i++) {
  swapped = false;
  for(int j = 0; j < n - 1 - i; j++) {
    if(arr[j] > arr[j+1]) {
      int temp = arr[j];
      arr[j] = arr[j+1];
      arr[j+1] = temp;
      swapped = true;
    }
  }
  if(swapped == false) break;
}
  • Outer loop (i): at most n − 1 passes.
  • Inner loop (j): adjacent pairs in the unsorted part; the last i items are already in place, hence j < n − 1 − i.
  • The swap takes three assignments using temp.
  • The swapped flag: if a whole pass makes no swap, the array is sorted → break.
Because the test is > (not ≥), equal items are never swapped: bubble sort is stable.

Bubble Sort · Run the code step by step

Bubble Sort: Code Trace, arr = [5, 1, 9, 2, 10]

arr[]50119223104

n = 5. The flag swapped is declared.

arr[]50119223104

Outer loop i = 0: the flag is set to false.

arr[]jj+150119223104

j = 0: arr[0] = 5 > arr[1] = 1? Yes.

arr[]jj+110519223104swap

Swapped via temp = 5; the flag becomes true.

arr[]jj+110519223104

j = 1: arr[1] = 5 > arr[2] = 9? No.

arr[]jj+110519223104

j = 2: arr[2] = 9 > arr[3] = 2? Yes.

arr[]jj+110512293104swap

Swapped via temp = 9; the flag becomes true.

arr[]jj+110512293104

j = 3: arr[3] = 9 > arr[4] = 10? No.

arr[]10512293104

End of pass: 10 is in place. Flag true → continue.

arr[]10512293104

Outer loop i = 1: the flag is set to false.

arr[]jj+110512293104

j = 0: arr[0] = 1 > arr[1] = 5? No.

arr[]jj+110512293104

j = 1: arr[1] = 5 > arr[2] = 2? Yes.

arr[]jj+110215293104swap

Swapped via temp = 5; the flag becomes true.

arr[]jj+110215293104

j = 2: arr[2] = 5 > arr[3] = 9? No.

arr[]10215293104

End of pass: 9 is in place. Flag true → continue.

arr[]10215293104

Outer loop i = 2: the flag is set to false.

arr[]jj+110215293104

j = 0: arr[0] = 1 > arr[1] = 2? No.

arr[]jj+110215293104

j = 1: arr[1] = 2 > arr[2] = 5? No.

arr[]10215293104

No swap in this pass: break. The array is sorted: [1, 2, 5, 9, 10].

boolean swapped;
for(int i = 0; i < n - 1; i++) {
  swapped = false;
  for(int j = 0; j < n - 1 - i; j++) {
    if(arr[j] > arr[j+1]) {
      int temp = arr[j];
      arr[j] = arr[j+1];
      arr[j+1] = temp;
      swapped = true;
    }
  }
  if(swapped == false) break;
}

Bubble Sort

Bubble Sort: Analysis

CaseInputComparisonsSwapsTime
Bestsortedn − 1 (one pass, with flag)0Θ(n)
Averagerandom≈ n²/2≈ n²/4Θ(n²)
Worstreverse sortedn(n − 1)/2n(n − 1)/2Θ(n²)
Pros
  • Very easy to understand and write.
  • In place (O(1) extra memory) and stable.
  • With the flag, recognizes sorted input in Θ(n).
Cons
  • Θ(n²): very slow on large arrays.
  • Makes many swaps (each inversion = 1 swap).
  • In practice always slower than insertion sort.

An inversion is a pair i < j with arr[i] > arr[j]. Bubble and insertion sort fix exactly one inversion per step; a reverse-sorted array has n(n − 1)/2 inversions.

Insertion Sort

Insertion Sort

  • Like sorting a hand of playing cards: each new card is slipped into its place among the sorted cards in your hand.
  • The array has two parts: a sorted part (on the left) and an unsorted part.
  • Take the first item of the unsorted part and insert it at its correct position in the sorted part.
  • To make room, the larger items are shifted one place to the right.
  • Invariant: at the start of each outer step, arr[0..i−1] is sorted.
next103152236445← sortedunsorted →

Insertion Sort · Step by step

Insertion Sort Step by Step: [3, 1, 5, 2, 6, 4]

Start301152236445← sortedunsorted →

Array [3, 1, 5, 2, 6, 4]. The first item alone is the sorted part; the rest is unsorted.

i = 1i301152236445← sortedunsorted →key1

Take the first item of the unsorted part: key = 1.

i = 1j303152236445← sortedunsorted →key1

3 > 1 → 3 is shifted right (index 0 → 1).

i = 1j+1103152236445← sortedunsorted →

Reached the start of the array → 1 is inserted at index 0 (1 shift).

i = 2i103152236445← sortedunsorted →key5

Take the first item of the unsorted part: key = 5.

i = 2j+1103152236445← sortedunsorted →

3 ≤ 5 → 5 is inserted at index 2 (no shift needed).

i = 3i103152236445← sortedunsorted →key2

Take the first item of the unsorted part: key = 2.

i = 3j103152536445← sortedunsorted →key2

5 > 2 → 5 is shifted right (index 2 → 3).

i = 3j103132536445← sortedunsorted →key2

3 > 2 → 3 is shifted right (index 1 → 2).

i = 3j+1102132536445← sortedunsorted →

1 ≤ 2 → 2 is inserted at index 1 (2 shifts).

i = 4i102132536445← sortedunsorted →key6

Take the first item of the unsorted part: key = 6.

i = 4j+1102132536445← sortedunsorted →

5 ≤ 6 → 6 is inserted at index 4 (no shift needed).

i = 5i102132536445← sortedunsorted →key4

Take the first item of the unsorted part: key = 4.

i = 5j102132536465← sortedkey4

6 > 4 → 6 is shifted right (index 4 → 5).

i = 5j102132535465← sortedkey4

5 > 4 → 5 is shifted right (index 3 → 4).

i = 5j+1102132435465← sorted

3 ≤ 4 → 4 is inserted at index 3 (2 shifts). Sorted: [1, 2, 3, 4, 5, 6].

Insertion Sort

Insertion Sort: Java Code

for(int i = 1; i < n; i++) {
  int key = arr[i];
  int j = i - 1;
  while(j >= 0 && arr[j] > key) {
    arr[j+1] = arr[j];
    j = j - 1;
  }
  arr[j+1] = key;
}
  • Start at i = 1: the one-element arr[0..0] is already sorted.
  • key: the item to insert. It is saved first because the shifts overwrite its cell.
  • while: as long as the item on the left is larger, shift it one place right.
  • When the loop ends, the free slot is j + 1.
It shifts rather than swaps: one assignment per step instead of 3. That is why insertion sort is noticeably faster than bubble sort.

Insertion Sort · Run the code step by step

Insertion Sort: Code Trace, arr = [5, 1, 9, 2, 10]

arr[]50119223104← sortedunsorted →

n = 5. Sorted part: just arr[0].

arr[]ij50119223104← sortedunsorted →key1

i = 1: key = arr[1] = 1, j = 0.

arr[]j50119223104← sortedunsorted →key1

j = 0 ≥ 0 and arr[0] = 5 > 1 → enter the loop.

arr[]j+150519223104← sortedunsorted →key1

arr[1] = arr[0] = 5 (shift right), j = −1.

arr[]50519223104← sortedunsorted →key1

j = −1: the loop ends.

arr[]j+110519223104← sortedunsorted →

arr[0] = key = 1. Sorted part: [1, 5].

arr[]ij10519223104← sortedunsorted →key9

i = 2: key = arr[2] = 9, j = 1.

arr[]j10519223104← sortedunsorted →key9

arr[1] = 5 ≤ 9: the loop ends.

arr[]j+110519223104← sortedunsorted →

arr[2] = key = 9. Sorted part: [1, 5, 9].

arr[]ij10519223104← sortedunsorted →key2

i = 3: key = arr[3] = 2, j = 2.

arr[]j10519223104← sortedunsorted →key2

j = 2 ≥ 0 and arr[2] = 9 > 2 → enter the loop.

arr[]j+110519293104← sortedunsorted →key2

arr[3] = arr[2] = 9 (shift right), j = 1.

arr[]j10519293104← sortedunsorted →key2

j = 1 ≥ 0 and arr[1] = 5 > 2 → enter the loop.

arr[]j+110515293104← sortedunsorted →key2

arr[2] = arr[1] = 5 (shift right), j = 0.

arr[]j10515293104← sortedunsorted →key2

arr[0] = 1 ≤ 2: the loop ends.

arr[]j+110215293104← sortedunsorted →

arr[1] = key = 2. Sorted part: [1, 2, 5, 9].

arr[]ij10215293104← sortedunsorted →key10

i = 4: key = arr[4] = 10, j = 3.

arr[]j10215293104← sortedkey10

arr[3] = 9 ≤ 10: the loop ends.

arr[]j+110215293104← sorted

arr[4] = key = 10. Sorted part: [1, 2, 5, 9, 10].

arr[]10215293104

i = 5 is not < n → the loop ends. Result: [1, 2, 5, 9, 10].

for(int i = 1; i < n; i++) {
  int key = arr[i];
  int j = i - 1;
  while(j >= 0 && arr[j] > key) {
    arr[j+1] = arr[j];
    j = j - 1;
  }
  arr[j+1] = key;
}

Insertion Sort

Insertion Sort: Analysis

CaseInputComparisonsShiftsTime
Bestsortedn − 10Θ(n)
Averagerandom≈ n²/4≈ n²/4Θ(n²)
Worstreverse sortedn(n − 1)/2n(n − 1)/2Θ(n²)
Pros
  • Stable, in place, online.
  • Adaptive: time Θ(n + number of inversions).
  • Among the fastest on small arrays (n ≲ 20–50).
Cons
  • Θ(n²) on large random arrays.
  • An item moves only one cell per step; a distant item needs many shifts (→ Shell sort).

Over all inputs we can only say O(n²) and Ω(n); there is no single Θ. Timsort and many libraries hand small subarrays in quicksort and merge sort over to insertion sort.

Selection Sort

Selection Sort

  • The array has two parts: sorted and unsorted.
  • At each step:
    • Find the smallest item in the unsorted part.
    • Swap it with the item at the front of the unsorted part.
    • After the swap it joins the sorted part.
  • n − 1 passes in total; at most one swap per pass.
min102152336445swap← sortedunsorted →

Selection Sort · Step by step

Selection Sort Step by Step: [3, 1, 5, 2, 6, 4]

Start301152236445unsorted →

Array [3, 1, 5, 2, 6, 4], length = 6. The sorted part starts empty.

Pass 1min301152236445unsorted →min3

Pass 1: the first unsorted item is the tentative minimum: min = 3.

Pass 1min301152236445unsorted →min1

1 < min (3)? Yes → min = 1.

Pass 1minj301152236445unsorted →min1

5 < min (1)? No.

Pass 1minj301152236445unsorted →min1

2 < min (1)? No.

Pass 1minj301152236445unsorted →min1

6 < min (1)? No.

Pass 1minj301152236445unsorted →min1

4 < min (1)? No.

Pass 1103152236445swapunsorted →

The minimum (1) is swapped with the front of the unsorted part → [1, 3, 5, 2, 6, 4]; the sorted part grows by one.

Pass 2min103152236445← sortedunsorted →min3

Pass 2: the first unsorted item is the tentative minimum: min = 3.

Pass 2minj103152236445← sortedunsorted →min3

5 < min (3)? No.

Pass 2min103152236445← sortedunsorted →min2

2 < min (3)? Yes → min = 2.

Pass 2minj103152236445← sortedunsorted →min2

6 < min (2)? No.

Pass 2minj103152236445← sortedunsorted →min2

4 < min (2)? No.

Pass 2102152336445swap← sortedunsorted →

The minimum (2) is swapped with the front of the unsorted part → [1, 2, 5, 3, 6, 4]; the sorted part grows by one.

Pass 3min102152336445← sortedunsorted →min5

Pass 3: the first unsorted item is the tentative minimum: min = 5.

Pass 3min102152336445← sortedunsorted →min3

3 < min (5)? Yes → min = 3.

Pass 3minj102152336445← sortedunsorted →min3

6 < min (3)? No.

Pass 3minj102152336445← sortedunsorted →min3

4 < min (3)? No.

Pass 3102132536445swap← sortedunsorted →

The minimum (3) is swapped with the front of the unsorted part → [1, 2, 3, 5, 6, 4]; the sorted part grows by one.

Pass 4min102132536445← sortedunsorted →min5

Pass 4: the first unsorted item is the tentative minimum: min = 5.

Pass 4minj102132536445← sortedunsorted →min5

6 < min (5)? No.

Pass 4min102132536445← sortedunsorted →min4

4 < min (5)? Yes → min = 4.

Pass 4102132436455swap← sortedunsorted →

The minimum (4) is swapped with the front of the unsorted part → [1, 2, 3, 4, 6, 5]; the sorted part grows by one.

Pass 5min102132436455← sortedunsorted →min6

Pass 5: the first unsorted item is the tentative minimum: min = 6.

Pass 5min102132436455← sortedunsorted →min5

5 < min (6)? Yes → min = 5.

Pass 5102132435465swap← sortedunsorted →

The minimum (5) is swapped with the front of the unsorted part → [1, 2, 3, 4, 5, 6]; the sorted part grows by one.

Result102132435465

After 5 passes the last item is in place by itself. Sorted array: [1, 2, 3, 4, 5, 6].

Selection Sort

Selection Sort: Java Code

public void sort(int[] arr) {
  int n = arr.length;
  for(int i = 0; i < n - 1; i++) {
    int minIdx = i;
    for(int j = i + 1; j < n; j++) {
      if(arr[j] < arr[minIdx]) {
        minIdx = j;
      }
    }
    int temp = arr[minIdx];
    arr[minIdx] = arr[i];
    arr[i] = temp;
  }
}
  • minIdx holds the index of the smallest item (not its value): the swap at the end needs the index.
  • The inner loop scans the whole unsorted part; the scan is no shorter even if the input is sorted.
  • One swap per pass: at most n − 1 in total.
Where writes are expensive (e.g. flash memory), the small number of swaps is selection sort's only real advantage.

Selection Sort · Run the code step by step

Selection Sort: Code Trace, arr = [5, 1, 10, 2, 9]

arr[]50111022394

n = arr.length = 5.

arr[]i, minIdx50111022394unsorted →

i = 0: minIdx = 0 (arr[minIdx] = 5).

arr[]minIdxj50111022394unsorted →

j = 1: arr[1] = 1 < arr[minIdx] = 5? Yes.

arr[]minIdx, j50111022394unsorted →

minIdx = 1 (new minimum: 1).

arr[]minIdxj50111022394unsorted →

j = 2: arr[2] = 10 < arr[minIdx] = 1? No.

arr[]minIdxj50111022394unsorted →

j = 3: arr[3] = 2 < arr[minIdx] = 1? No.

arr[]minIdxj50111022394unsorted →

j = 4: arr[4] = 9 < arr[minIdx] = 1? No.

arr[]iminIdx10511022394swapunsorted →

j = 5 → the inner loop ends. temp = 1; arr[1] ↔ arr[0] → [1, 5, 10, 2, 9].

arr[]i, minIdx10511022394← sortedunsorted →

i = 1: minIdx = 1 (arr[minIdx] = 5).

arr[]minIdxj10511022394← sortedunsorted →

j = 2: arr[2] = 10 < arr[minIdx] = 5? No.

arr[]minIdxj10511022394← sortedunsorted →

j = 3: arr[3] = 2 < arr[minIdx] = 5? Yes.

arr[]minIdx, j10511022394← sortedunsorted →

minIdx = 3 (new minimum: 2).

arr[]minIdxj10511022394← sortedunsorted →

j = 4: arr[4] = 9 < arr[minIdx] = 2? No.

arr[]iminIdx10211025394swap← sortedunsorted →

j = 5 → the inner loop ends. temp = 2; arr[3] ↔ arr[1] → [1, 2, 10, 5, 9].

arr[]i, minIdx10211025394← sortedunsorted →

i = 2: minIdx = 2 (arr[minIdx] = 10).

arr[]minIdxj10211025394← sortedunsorted →

j = 3: arr[3] = 5 < arr[minIdx] = 10? Yes.

arr[]minIdx, j10211025394← sortedunsorted →

minIdx = 3 (new minimum: 5).

arr[]minIdxj10211025394← sortedunsorted →

j = 4: arr[4] = 9 < arr[minIdx] = 5? No.

arr[]iminIdx10215210394swap← sortedunsorted →

j = 5 → the inner loop ends. temp = 5; arr[3] ↔ arr[2] → [1, 2, 5, 10, 9].

arr[]i, minIdx10215210394← sortedunsorted →

i = 3: minIdx = 3 (arr[minIdx] = 10).

arr[]minIdxj10215210394← sortedunsorted →

j = 4: arr[4] = 9 < arr[minIdx] = 10? Yes.

arr[]minIdx, j10215210394← sortedunsorted →

minIdx = 4 (new minimum: 9).

arr[]iminIdx10215293104swap← sortedunsorted →

j = 5 → the inner loop ends. temp = 9; arr[4] ↔ arr[3] → [1, 2, 5, 9, 10].

arr[]10215293104

i = 4 is not < n − 1 → done. Result: [1, 2, 5, 9, 10].

public void sort(int[] arr) {
  int n = arr.length;
  for(int i = 0; i < n - 1; i++) {
    int minIdx = i;
    for(int j = i + 1; j < n; j++) {
      if(arr[j] < arr[minIdx]) {
        minIdx = j;
      }
    }
    int temp = arr[minIdx];
    arr[minIdx] = arr[i];
    arr[i] = temp;
  }
}

Selection Sort

Selection Sort: Analysis

MeasureValue
Comparisons(n − 1) + (n − 2) + … + 1 = n(n − 1)/2 on every input
Swapsat most n − 1
Timebest = average = worst = Θ(n²)
Extra memoryO(1), in place
Stable?No (long-distance swaps)
Input2ₐ2ᵦ1pass 1: swapOutput12ᵦ2ₐ

The minimum (1) is swapped with 2ₐ at the front: 2ₐ ends up behind 2ᵦ.

Not adaptive: it spends n(n − 1)/2 comparisons even on a sorted array.

Simple Algorithms · Comparison

Bubble, Insertion, Selection: Same Input

Cells: comparisons / swaps (or shifts), n = 6. Values were counted by running the code.

AlgorithmMove type[3, 1, 5, 2, 6, 4]sorted 1…6reverse 6…1
Bubble (with flag)swap12 / 55 / 015 / 15
Insertionshift9 / 55 / 015 / 15
Selectionswap15 / 515 / 015 / 3
3152645 inversions[3, 1, 5, 2, 6, 4]
  • In bubble and insertion sort the number of moves = the number of inversions (5 for [3, 1, 5, 2, 6, 4]); a swap is 3 assignments, a shift is 1.
  • Selection sort always makes n(n − 1)/2 = 15 comparisons regardless of the input, but it makes the fewest swaps.
  • On sorted input, flagged bubble sort and insertion sort make only n − 1 = 5 comparisons (adaptive).

Shell Sort

Shell Sort

  • D. Shell (1959): a generalization of insertion sort.
  • First, items a gap apart are insertion-sorted; small items far away jump a long distance in one move.
  • The gap shrinks (n/2, n/4, …); the last pass has gap = 1, i.e. plain insertion sort — but by now the array is nearly sorted.
  • In place, unstable; the time depends on the gap sequence: worst case Θ(n²) with n/2 gaps, O(n1.5) with Hibbard's (2ᵏ − 1).
void shellSort(int[] arr) {
  int n = arr.length;
  for (int gap = n / 2; gap > 0; gap /= 2)
    for (int i = gap; i < n; i++) {
      int key = arr[i], j;
      for (j = i; j >= gap && arr[j - gap] > key; j -= gap)
        arr[j] = arr[j - gap];
      arr[j] = key;
    }
}

Shell Sort · Step by step

Shell Sort Step by Step: [6, 5, 3, 1, 8, 7, 2, 4]

Start6051321384752647

Array [6, 5, 3, 1, 8, 7, 2, 4], n = 8. Gap sequence: n/2 = 4, 2, … , 1.

gap = 4: groups6051321384752647

gap = 4: items 4 indices apart form a group (same color): [6, 8], [5, 7], [3, 2], [1, 4].

gap = 4: after6051221384753647

Each group is insertion-sorted (1 shift): [6, 8], [5, 7], [2, 3], [1, 4] → [6, 5, 2, 1, 8, 7, 3, 4].

gap = 2: groups6051221384753647

gap = 2: items 2 indices apart form a group (same color): [6, 2, 8, 3], [5, 1, 7, 4].

gap = 2: after2011324364558677

Each group is insertion-sorted (6 shifts): [2, 3, 6, 8], [1, 4, 5, 7] → [2, 1, 3, 4, 6, 5, 8, 7].

gap = 1: groups2011324364558677

gap = 1: items 1 index apart form a group (same color): [2, 1, 3, 4, 6, 5, 8, 7].

gap = 1: after1021324354657687

gap = 1 is plain insertion sort; since the array is nearly sorted, only 3 shifts were needed. Result: [1, 2, 3, 4, 5, 6, 7, 8].

Shell Sort · Run the code step by step

Shell Sort: Code Trace, arr = [5, 2, 6, 1, 4, 3]

arr502162134435●●○○◆◆gap = 3: same marks form a group

arr = [5, 2, 6, 1, 4, 3], n = 6. The first gap is n / 2 = 3: items 3 apart, (0, 3), (1, 4), (2, 5), are insertion-sorted as separate groups.

gap = 3, i = 3, key = 1before502162134435after126543j

i = 3: key = 1. arr[0] = 5 > 1 → moves to index 3. Stop at j = 0 and write 1 there.

gap = 3, i = 4, key = 4before102162534435after126543j

i = 4: key = 4. arr[1] = 2 ≤ 4: no shift, 4 stays.

gap = 3, i = 5, key = 3before102162534435after123546j

i = 5: key = 3. arr[2] = 6 > 3 → moves to index 5. Stop at j = 2 and write 3 there.

gap = 1, i = 1, key = 2before102132534465after123546j

gap = 3 / 2 = 1: now plain insertion sort. i = 1: key = 2. arr[0] = 1 ≤ 2: no shift, 2 stays.

gap = 1, i = 2, key = 3before102132534465after123546j

i = 2: key = 3. arr[1] = 2 ≤ 3: no shift, 3 stays.

gap = 1, i = 3, key = 5before102132534465after123546j

i = 3: key = 5. arr[2] = 3 ≤ 5: no shift, 5 stays.

gap = 1, i = 4, key = 4before102132534465after123456j

i = 4: key = 4. arr[3] = 5 > 4 → moves to index 4. arr[2] = 3 ≤ 4, stop; write 4 there.

gap = 1, i = 5, key = 6before102132435465after123456j

i = 5: key = 6. arr[4] = 5 ≤ 6: no shift, 6 stays.

arr102132435465

gap = 1 / 2 = 0: the loop ends and the array is sorted. Only 3 shifts in total; the gap-3 pass moved the small items forward in one go.

void shellSort(int[] arr) {
  int n = arr.length;
  for (int gap = n / 2; gap > 0; gap /= 2)
    for (int i = gap; i < n; i++) {
      int key = arr[i], j;
      for (j = i; j >= gap && arr[j - gap] > key; j -= gap)
        arr[j] = arr[j - gap];
      arr[j] = key;
    }
}

Merge Sort

Merge Sort

  • A divide-and-conquer algorithm.
  • Invented by John von Neumann in 1945.
  • O(n log n) even in the worst case.
  • Stable; merging needs extra memory (O(n)).
How does it work?

1. Divide: split the list into two sublists, and keep splitting until each has one item.

2. Conquer: sort each half the same way (recursively).

3. Combine: merge the sorted sublists until a single sorted list remains.

3827433382743327383433273843splitsortmerge
A one-item list is sorted by definition; this is the recursion's base case. The real work happens in the merge step.

Merge Sort

Example: [38, 27, 43, 3, 9, 82, 10]

Input382743398210Divide382743398210Divide382743398210Divide382743398210Merge273834398210Merge327384391082Merge391027384382

Seven items: since mid = ⌊(left + right)/2⌋, the left half gets one extra item ([38, 27, 43, 3] and [9, 82, 10]).

Merge Sort · Step by step

Merge Sort Step by Step: [6, 5, 3, 1, 8, 7, 2, 4]

65318724

Array [6, 5, 3, 1, 8, 7, 2, 4]: first split down to single items, then the sorted pieces are merged.

6531872465318724

Split: [6, 5, 3, 1, 8, 7, 2, 4] → [6, 5, 3, 1] and [8, 7, 2, 4].

65318724653187246531

Split: [6, 5, 3, 1] → [6, 5] and [3, 1].

6531872465318724653165

Split: [6, 5] → [6] and [5]. One-item pieces are already sorted.

6531872465318724563165

Merge: [6] + [5] → [5, 6].

653187246531872456316531

Split: [3, 1] → [3] and [1]. One-item pieces are already sorted.

653187246531872456136531

Merge: [3] + [1] → [1, 3].

653187241356872456136531

Merge: [5, 6] + [1, 3] → [1, 3, 5, 6].

6531872413568724561387246531

Split: [8, 7, 2, 4] → [8, 7] and [2, 4].

653187241356872456138724653187

Split: [8, 7] → [8] and [7]. One-item pieces are already sorted.

653187241356872456137824653187

Merge: [8] + [7] → [7, 8].

65318724135687245613782465318724

Split: [2, 4] → [2] and [4]. One-item pieces are already sorted.

65318724135687245613782465318724

Merge: [2] + [4] → [2, 4].

65318724135624785613782465318724

Merge: [7, 8] + [2, 4] → [2, 4, 7, 8].

12345678135624785613782465318724

Merge: [1, 3, 5, 6] + [2, 4, 7, 8] → [1, 2, 3, 4, 5, 6, 7, 8]. The array is sorted.

Merge Sort

Merge Sort: Java Code

void mergeSort(int[] arr, int left, int right) {
  if (left < right) {
    int mid = left + (right - left) / 2;
    mergeSort(arr, left, mid);
    mergeSort(arr, mid + 1, right);
    merge(arr, left, mid, right);
  }
}
void merge(int[] A, int p, int q, int r) {
  int[] L = Arrays.copyOfRange(A, p, q + 1);
  int[] R = Arrays.copyOfRange(A, q + 1, r + 1);
  int i = 0, j = 0, k = p;
  while (i < L.length && j < R.length) {
    if (L[i] <= R[j]) A[k++] = L[i++];
    else              A[k++] = R[j++];
  }
  while (i < L.length) A[k++] = L[i++];
  while (j < R.length) A[k++] = R[j++];
}

Writing left + (right − left) / 2 avoids the int overflow in (left + right) / 2.

Merge Sort · Run the code step by step

merge(): Merging Two Sorted Arrays

L1356iR2478jA01234567k

Copy L = A[p..q] = [1, 3, 5, 6] and R = A[q+1..r] = [2, 4, 7, 8] (both sorted). i = j = 0, k = p = 0.

L1356iR2478jA101234567k

L[0] = 1 ≤ R[0] = 2 → A[0] = 1; i++, k++.

L1356iR2478jA1021234567k

L[1] = 3 > R[0] = 2 → A[1] = 2; j++, k++.

L1356iR2478jA10213234567k

L[1] = 3 ≤ R[1] = 4 → A[2] = 3; i++, k++.

L1356iR2478jA102132434567k

L[2] = 5 > R[1] = 4 → A[3] = 4; j++, k++.

L1356iR2478jA1021324354567k

L[2] = 5 ≤ R[2] = 7 → A[4] = 5; i++, k++.

L1356R2478jA10213243546567k

L[3] = 6 ≤ R[2] = 7 → A[5] = 6; i++, k++.

L1356R2478jA102132435465767k

L is exhausted (i = 4); the remaining R items are copied in order: A[6] = 7.

L1356R2478A1021324354657687

A[7] = 8. Merge done: [1, 2, 3, 4, 5, 6, 7, 8] (8 items, at most 7 comparisons).

void merge(int[] A, int p, int q, int r) {
  int[] L = Arrays.copyOfRange(A, p, q + 1);
  int[] R = Arrays.copyOfRange(A, q + 1, r + 1);
  int i = 0, j = 0, k = p;
  while (i < L.length && j < R.length) {
    if (L[i] <= R[j]) A[k++] = L[i++];
    else              A[k++] = R[j++];
  }
  while (i < L.length) A[k++] = L[i++];
  while (j < R.length) A[k++] = R[j++];
}

Merge Sort

Merge Sort: Analysis

Recurrence:

T(n) = 2 T(n/2) + Θ(n)

  • The total merge work on each level is Θ(n).
  • The number of levels is log₂ n (halving n down to 1).
  • Total: Θ(n log n) — best = average = worst.
  • Extra memory Θ(n) (auxiliary arrays) + a Θ(log n) recursion stack.
  • L[i] <= R[j]: ties are taken from the left → stable.
nnn/2n/2nn/4n/4n/4n/4n⋮⋮11111111nΣ = n · log₂ nlog₂ n levels
By the Master Theorem: a = 2, b = 2, f(n) = Θ(n) = Θ(nlog₂ 2) → case 2 → T(n) = Θ(n log n).

Quicksort

Quicksort

  • C. A. R. Hoare (1959). Also divide and conquer, but most of the work happens in the divide step.
  • Choose a pivot item.
  • Split the array into two parts around the pivot (partitioning): ≤ x on the left, ≥ x on the right.
  • Sort the two parts recursively. No merge step is needed.
≤ xx≥ xpivotleft part → sort recursivelyright part → sort recursively
After partitioning the pivot is in its final place: it sits at exactly that index in the sorted array.

Quicksort

The Partitioning Procedure

Lomuto: pivot = last item

partition(A, p, q):
  pivot = A[q]        // choose pivot
  i = p - 1           // end of ≤ zone
  for j = p to q - 1:
    if A[j] <= pivot:
      i = i + 1
      swap(A[i], A[j])
  swap(A[i + 1], A[q])  // place the pivot
  return i + 1          // pivot index

Same idea: pivot = first item

PARTITION(A, p, q):
  x = A[p]            // pivot
  i = p
  for j = p + 1 to q:
    if A[j] <= x:
      i = i + 1
      swap(A[i], A[j])
  swap(A[p], A[i])
  return i

Invariant: A[p..i] ≤ pivot, A[i+1..j−1] > pivot, A[j..] not yet examined. One pass: Θ(n) time, O(1) extra memory.

Quicksort · Step by step

Partitioning: [6, 10, 13, 5, 8, 3, 2, 11]

x = A[p] = 6ij6010113253843526117

Pivot x = A[p] = 6 (yellow). i = p; j starts at p + 1. Blue: ≤ x, red: > x, white: not yet seen.

j = 1ij6010113253843526117

A[j] = 10 > 6 → only j advances; the red zone grows.

j = 2ij6010113253843526117

A[j] = 13 > 6 → only j advances; the red zone grows.

j = 3ij6051132103843526117swap

A[j] = 5 ≤ 6 → i = 1; A[i] ↔ A[j]: [6, 5, 13, 10, 8, 3, 2, 11]. The blue zone grows by one.

j = 4ij6051132103843526117

A[j] = 8 > 6 → only j advances; the red zone grows.

j = 5ij6051321038413526117swap

A[j] = 3 ≤ 6 → i = 2; A[i] ↔ A[j]: [6, 5, 3, 10, 8, 13, 2, 11]. The blue zone grows by one.

j = 6ij6051322384135106117swap

A[j] = 2 ≤ 6 → i = 3; A[i] ↔ A[j]: [6, 5, 3, 2, 8, 13, 10, 11]. The blue zone grows by one.

j = 7ij6051322384135106117

A[j] = 11 > 6 → only j advances; the red zone grows.

Finally: A[p] ↔ A[i]i2051326384135106117swap

The loop is done. The pivot is swapped with the last item of the blue zone: [2, 5, 3, 6, 8, 13, 10, 11]. Pivot 6 is now in its final place (index 3); everything to its left is ≤ 6, to its right ≥ 6.

Quicksort · Run the code step by step

partition(): Code Trace, arr = [2, 8, 7, 1, 3, 5, 6, 4]

arr[]2081721334556647

pivot = arr[hi] = 4; i = lo − 1 = −1 (the ≤ pivot zone is empty).

arr[]j2081721334556647

j = 0: arr[0] = 2 ≤ 4? Yes.

arr[]i, j2081721334556647

i = 0; swap(arr, 0, 0) → [2, 8, 7, 1, 3, 5, 6, 4] (with itself).

arr[]ij2081721334556647

j = 1: arr[1] = 8 ≤ 4? No → j advances.

arr[]ij2081721334556647

j = 2: arr[2] = 7 ≤ 4? No → j advances.

arr[]ij2081721334556647

j = 3: arr[3] = 1 ≤ 4? Yes.

arr[]ij2011728334556647swap

i = 1; swap(arr, 1, 3) → [2, 1, 7, 8, 3, 5, 6, 4].

arr[]ij2011728334556647

j = 4: arr[4] = 3 ≤ 4? Yes.

arr[]ij2011328374556647swap

i = 2; swap(arr, 2, 4) → [2, 1, 3, 8, 7, 5, 6, 4].

arr[]ij2011328374556647

j = 5: arr[5] = 5 ≤ 4? No → j advances.

arr[]ij2011328374556647

j = 6: arr[6] = 6 ≤ 4? No → j advances.

arr[]i2011324374556687swap

The loop is done. swap(arr, i+1 = 3, hi = 7) → [2, 1, 3, 4, 7, 5, 6, 8].

arr[]i2011324374556687

return 3: pivot 4 is in its final place; left [0..2] ≤ 4, right [4..7] > 4.

int partition(int[] arr, int lo, int hi) {
  int pivot = arr[hi];
  int i = lo - 1;
  for (int j = lo; j < hi; j++) {
    if (arr[j] <= pivot) {
      i++;
      swap(arr, i, j);
    }
  }
  swap(arr, i + 1, hi);
  return i + 1;
}

Quicksort

Quicksort: Recursive Structure

QUICKSORT(A, p, r)
  if p < r
    q = PARTITION(A, p, r)
    QUICKSORT(A, p, q − 1)
    QUICKSORT(A, q + 1, r)

Initial call: QUICKSORT(A, 1, n)
void quickSort(int[] arr, int lo, int hi) {
  if (lo < hi) {
    int p = partition(arr, lo, hi);
    quickSort(arr, lo, p - 1);
    quickSort(arr, p + 1, hi);
  }
}
≤ pivot: lo … p − 1p> pivot: p + 1 … hiquickSort(arr, lo, p − 1)quickSort(arr, p + 1, hi)in place
  • The pivot (index q) is included in neither recursive call: it is already in place.
  • Base case: a subarray of 0 or 1 items (p ≥ r) is already sorted.
  • The Java version is 0-based: the initial call is quickSort(arr, 0, n − 1).

Quicksort · Step by step

Quicksort Step by Step: [6, 10, 13, 5, 8, 3, 2, 11]

QUICKSORT(A, 0, 7)6010113253843526117

Array [6, 10, 13, 5, 8, 3, 2, 11]. Each call partitions its subarray with pivot = first item, then sorts both sides recursively.

QUICKSORT(A, 0, 7)q2051326384135106117

Depth 0: [6, 10, 13, 5, 8, 3, 2, 11] is partitioned around 6 → the pivot lands at index 3; then [0..2] and [4..7] are sorted.

QUICKSORT(A, 0, 2)q2051326384135106117

Depth 1: [2, 5, 3] is partitioned around 2 → the pivot lands at index 0; then [0..-1] and [1..2] are sorted.

QUICKSORT(A, 1, 2)q2031526384135106117

Depth 2: [5, 3] is partitioned around 5 → the pivot lands at index 2; then [1..1] and [3..2] are sorted.

QUICKSORT(A, 1, 1)q2031526384135106117

Depth 3: [1..1] has one item (3) → already sorted (p < r fails).

QUICKSORT(A, 4, 7)q2031526384135106117

Depth 1: [8, 13, 10, 11] is partitioned around 8 → the pivot lands at index 4; then [4..3] and [5..7] are sorted.

QUICKSORT(A, 5, 7)q2031526384115106137

Depth 2: [13, 10, 11] is partitioned around 13 → the pivot lands at index 7; then [5..6] and [8..7] are sorted.

QUICKSORT(A, 5, 6)q2031526384105116137

Depth 3: [11, 10] is partitioned around 11 → the pivot lands at index 6; then [5..5] and [7..6] are sorted.

QUICKSORT(A, 5, 5)q2031526384105116137

Depth 4: [5..5] has one item (10) → already sorted (p < r fails).

Result2031526384105116137

All calls are done; there is no merge step, and the array was sorted in place: [2, 3, 5, 6, 8, 10, 11, 13].

Quicksort · Run the code step by step

quickSort: Tracing the Calls, arr = [5, 2, 8, 1, 9, 3]

201132539485arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

quickSort(0, 5): lo < hi. partition splits around pivot = 3 (last item): [2, 1] on the left, [5, 9, 8] on the right; the pivot lands at index p = 2. Then line 4: the left part.

102132539485arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

quickSort(0, 1): lo < hi. partition splits around pivot = 1 (last item): [] on the left, [2] on the right; the pivot lands at index p = 0.

102132539485arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

quickSort(0, -1): lo > hi, empty part → returns at once (base case).

102132539485arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

quickSort(1, 1): the one-item part [2] is already sorted → returns. Call (0, 1) is done; line 5: (3, 5).

102132538495arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

quickSort(3, 5): lo < hi. partition splits around pivot = 8 (last item): [5] on the left, [9] on the right; the pivot lands at index p = 4.

102132538495arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

quickSort(3, 3): the one-item part [5] is already sorted → returns.

102132538495arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

quickSort(5, 5): the one-item part [9] is already sorted → returns.

102132538495arr(0, 5)(0, 1)(3, 5)(0, -1)(1, 1)(3, 3)(5, 5)

All calls have returned: arr = [1, 2, 3, 5, 8, 9]. 3 partition calls and 4 base-case calls; no pivot entered a subcall, each landed in its own place.

void quickSort(int[] arr, int lo, int hi) {
  if (lo < hi) {
    int p = partition(arr, lo, hi);
    quickSort(arr, lo, p - 1);
    quickSort(arr, p + 1, hi);
  }
}

Quicksort

Quicksort: Analysis and Pivot Choice

CaseWhen?RecurrenceTime
Bestpivot is always the medianT(n) = 2T(n/2) + Θ(n)Θ(n log n)
Averagerandom input / random pivotexpected ≈ 1.39 n log₂ n comparisonsΘ(n log n)
Worstpivot always the min/max (e.g. a sorted array + last-item pivot)T(n) = T(n − 1) + Θ(n)Θ(n²)
Random pivotNo fixed input can force the bad case every time; Θ(n²) happens only with tiny probability, and the expected time is Θ(n log n).
Median of threeThe pivot is the median of the first, middle and last items; this avoids the bad case on sorted and reverse-sorted input, though crafted inputs can still cause Θ(n²).
Small partsSwitching to insertion sort when n ≲ 16 and recursing into the smaller side first (stack O(log n)) speeds things up in practice.

In place (O(log n) stack on average), unstable. Being cache-friendly, it is usually the fastest comparison sort in practice (Java Arrays.sort(int[]): dual-pivot quicksort).

Heapsort

Heapsort

  • Sorts using the heap data structure; O(n log n) on average and in the worst case.
  • Max-heap: every node is greater than or equal to its children → the largest item is at the root.
  • The tree is not a separate structure; it is the array itself: the children of arr[i] are 2i + 1 and 2i + 2, its parent (i − 1) / 2.
How it works
  1. Turn the array into a max-heap.
  2. Move the largest item (the root) to the end of the array.
  3. Restore the max-heap on the rest (sift the root down).
  4. Repeat until all items are sorted.
1005132431410051324314arr

Heapsort

Example: [4, 10, 3, 5, 1]

StepArray (heap | sorted)Explanation
Array[4, 10, 3, 5, 1]
Build max-heap[10, 5, 3, 4, 1]root = largest
Root ↔ arr[4][1, 5, 3, 4] | [10]10 to the sorted part
Restore max-heap[5, 4, 3, 1] | [10]root sifted down
Root ↔ arr[3][1, 4, 3] | [5, 10]5 to the sorted part
Restore max-heap[4, 1, 3] | [5, 10]root sifted down
Root ↔ arr[2][3, 1] | [4, 5, 10]4 to the sorted part
Restore max-heap[3, 1] | [4, 5, 10]root sifted down
Root ↔ arr[1][1] | [3, 4, 5, 10]3 to the sorted part
Result[1, 3, 4, 5, 10]
410351build heap105341[4, 10, 3, 5, 1]max-heap

Heapsort · Step by step

Phase 1: Building the Max-Heap — [6, 5, 3, 1, 8, 7, 2, 4]

60513213847526476051321384752647arr

View the array [6, 5, 3, 1, 8, 7, 2, 4] as a complete binary tree: the children of arr[i] are 2i+1 and 2i+2. Not a heap yet.

60513243847526176051324384752617arr

heapify(i = 3): 1 is smaller than its child → swap (1 ↔ 4). Array: [6, 5, 3, 4, 8, 7, 2, 1].

60517243843526176051724384352617arr

heapify(i = 2): 3 is smaller than its larger child → swap (3 ↔ 7). Array: [6, 5, 7, 4, 8, 3, 2, 1].

60817243543526176081724354352617arr

heapify(i = 1): 5 is smaller than its larger child → swap (5 ↔ 8). Array: [6, 8, 7, 4, 5, 3, 2, 1].

80617243543526178061724354352617arr

heapify(i = 0): 6 is smaller than its children → swap (6 ↔ 8). Array: [8, 6, 7, 4, 5, 3, 2, 1].

80617243543526178061724354352617arr

All internal nodes (i = 3 … 0) are done: max-heap [8, 6, 7, 4, 5, 3, 2, 1]. The largest item (8) is at the root.

Heapsort · Step by step

Phase 2: Move the Root to the End, Sift Down

80617243543526178061724354352617arr

Max-heap: [8, 6, 7, 4, 5, 3, 2, 1]. Each round swaps the root (the largest) with the heap's last item, and the heap shrinks by one.

106172435435261061724354352687arr

Root 8 ↔ arr[7] = 1: 8 joins the sorted part (green). Heap size 7.

706132435415267061324354152687arr

heapify(0) sifts the root down (1 ↔ 7; 1 ↔ 3) → heap: [7, 6, 3, 4, 5, 1, 2].

2061324354152061324354157687arr

Root 7 ↔ arr[6] = 2: 7 joins the sorted part (green). Heap size 6.

6051324324156051324324157687arr

heapify(0) sifts the root down (2 ↔ 6; 2 ↔ 5) → heap: [6, 5, 3, 4, 2, 1].

10513243241051324324657687arr

Root 6 ↔ arr[5] = 1: 6 joins the sorted part (green). Heap size 5.

50413213245041321324657687arr

heapify(0) sifts the root down (1 ↔ 5; 1 ↔ 4) → heap: [5, 4, 3, 1, 2].

204132132041321354657687arr

Root 5 ↔ arr[4] = 2: 5 joins the sorted part (green). Heap size 4.

402132134021321354657687arr

heapify(0) sifts the root down (2 ↔ 4) → heap: [4, 2, 3, 1].

1021321021324354657687arr

Root 4 ↔ arr[3] = 1: 4 joins the sorted part (green). Heap size 3.

3021123021124354657687arr

heapify(0) sifts the root down (1 ↔ 3) → heap: [3, 2, 1].

10211021324354657687arr

Root 3 ↔ arr[2] = 1: 3 joins the sorted part (green). Heap size 2.

20112011324354657687arr

heapify(0) sifts the root down (1 ↔ 2) → heap: [2, 1].

101021324354657687arr

Root 2 ↔ arr[1] = 1: 2 joins the sorted part (green). Heap size 1.

101021324354657687arr

One item is left in the heap, and it is in place. Sorted array: [1, 2, 3, 4, 5, 6, 7, 8].

Heapsort

Heapsort: Java Code and Analysis

void heapSort(int[] arr) {
  int n = arr.length;
  for (int i = n / 2 - 1; i >= 0; i--)   // 1) build heap
    heapify(arr, n, i);
  for (int i = n - 1; i > 0; i--) {      // 2) sort
    swap(arr, 0, i);                   // max to the end
    heapify(arr, i, 0);                // sift root down
  }
}
void heapify(int[] arr, int n, int i) {
  while (true) {
    int largest = i, left = 2 * i + 1, right = 2 * i + 2;
    if (left < n && arr[left] > arr[largest]) largest = left;
    if (right < n && arr[right] > arr[largest]) largest = right;
    if (largest == i) return;          // in place
    swap(arr, i, largest);
    i = largest;                       // keep sifting
  }
}
  • Building the heap: the leaves (i ≥ n/2) are already one-item heaps; sift down from i = n/2 − 1 toward the root. Total O(n) (most nodes are near the bottom).
  • Sorting: n − 1 times, move the root to the end and sift down in O(log n) → O(n log n).
  • O(n log n) even in the worst case, with O(1) extra memory (in place).
  • Unstable: the root–last swap can reorder equal keys.

A heap can also be built by inserting items one by one (sift up); that too gives a valid max-heap, but takes O(n log n). The bottom-up construction used here (Floyd's method) is O(n).

Heapsort · Run the code step by step

Building a Max-Heap: Code Trace, arr = [3, 9, 2, 1, 4, 5]

309122134455392145

arr = [3, 9, 2, 1, 4, 5]. The leaves (indices 3, 4, 5) are already one-item heaps. Building runs from i = n / 2 − 1 = 2 toward the root.

309122134455392145

i = 2 (2): left = 5 (5) > 2 → largest = left; no right child.

309152134425395142

largest ≠ i: swap(arr, 2, 5) → 5 moves up, 2 moves down; the loop continues with i = 5 (line 11).

309152134425395142

i = 5 (2): no children (left = 11 ≥ n). largest == i → return, sifting ends.

309152134425395142

Loop i = 1: heapify(arr, 6, 1).

309152134425395142

i = 1 (9): left = 3 (1) is not > 9; right = 4 (4) is not > 9. No change: it is already larger than its children.

309152134425395142

Loop i = 0: heapify(arr, 6, 0).

309152134425395142

i = 0 (3): left = 1 (9) > 3 → largest = left; right = 2 (5) is not > 9.

903152134425935142

largest ≠ i: swap(arr, 0, 1) → 9 moves up, 3 moves down; the loop continues with i = 1 (line 11).

903152134425935142

i = 1 (3): left = 3 (1) is not > 3; right = 4 (4) > 3 → largest = right.

904152133425945132

largest ≠ i: swap(arr, 1, 4) → 4 moves up, 3 moves down; the loop continues with i = 4 (line 11).

904152133425945132

i = 4 (3): no children (left = 9 ≥ n). largest == i → return, sifting ends.

904152133425945132

i = −1: the loop ends. [9, 4, 5, 1, 3, 2] is a max-heap: every parent is larger than its children, and the largest item, 9, is at the root.

for (int i = n / 2 - 1; i >= 0; i--)
  heapify(arr, n, i);

void heapify(int[] arr, int n, int i) {
  while (true) {
    int largest = i, left = 2 * i + 1, right = 2 * i + 2;
    if (left < n && arr[left] > arr[largest]) largest = left;
    if (right < n && arr[right] > arr[largest]) largest = right;
    if (largest == i) return;          // in place
    swap(arr, i, largest);
    i = largest;                       // keep sifting
  }
}

Linear-Time Sorting

A Lower Bound for Comparison Sorting

  • Any algorithm that only asks "ai ≤ aj?" can be drawn as a decision tree.
  • Each leaf is an ordering (a permutation); to be correct the tree needs n! leaves.
  • A binary tree of height h has at most 2h leaves: 2h ≥ n! ⇒ h ≥ log₂(n!) = Ω(n log n).
  • Consequence: merge sort and heapsort are asymptotically optimal.
  • Going faster needs information beyond comparisons: small integer keys → counting, radix.
≤>≤>≤>≤>≤>1:22:31:3⟨1,2,3⟩1:3⟨2,1,3⟩2:3⟨1,3,2⟩⟨3,1,2⟩⟨2,3,1⟩⟨3,2,1⟩3! = 6 leaves → height ≥ ⌈log₂ 6⌉ = 3

Linear-Time Sorting

Counting Sort

  • Items are never compared with each other.
  • Input: A[1 … n], A[j] ∈ {1, 2, …, k}.
  • Output: B[1 … n], sorted.
  • Temporary: counters C[1 … k].

Idea: if we know how many items have key ≤ i, the last item with key i goes exactly at that position.

for i = 1 to k
  C[i] = 0
for j = 1 to n
  C[A[j]] = C[A[j]] + 1   // C[i] = |{key = i}|
for i = 2 to k
  C[i] = C[i] + C[i−1]    // C[i] = |{key ≤ i}|
for j = n downto 1
  B[C[A[j]]] = A[j]
  C[A[j]] = C[A[j]] − 1

Linear-Time Sorting · Step by step

Counting Sort Step by Step: A = [4, 1, 3, 4, 3]

A:4134312345C:1234B:12345

Input A = [4, 1, 3, 4, 3], n = 5, keys {1, …, 4}. C: counter array (k = 4), B: output.

A:4134312345C:00001234B:12345

Loop 1: for i = 1 to k: C[i] = 0 — all counters are reset. Θ(k).

A:4134312345C:00011234B:12345

Loop 2, j = 1: A[1] = 4 → C[4]++ = 1. (C[i] = count of key i)

A:4134312345C:10011234B:12345

Loop 2, j = 2: A[2] = 1 → C[1]++ = 1. (C[i] = count of key i)

A:4134312345C:10111234B:12345

Loop 2, j = 3: A[3] = 3 → C[3]++ = 1. (C[i] = count of key i)

A:4134312345C:10121234B:12345

Loop 2, j = 4: A[4] = 4 → C[4]++ = 2. (C[i] = count of key i)

A:4134312345C:10221234B:12345

Loop 2, j = 5: A[5] = 3 → C[3]++ = 2. (C[i] = count of key i)

A:4134312345C:11221234B:12345

Loop 3, i = 2: C[2] = C[2] + C[1] = 1. (C[i] = count of keys ≤ i)

A:4134312345C:11321234B:12345

Loop 3, i = 3: C[3] = C[3] + C[2] = 3. (C[i] = count of keys ≤ i)

A:4134312345C:11351234B:12345

Loop 3, i = 4: C[4] = C[4] + C[3] = 5. (C[i] = count of keys ≤ i)

A:4134312345C:11351234B:12345

The prefix sums are ready: C = [1, 1, 3, 5]. C[i] is the position in B of the last item with key i.

A:4134312345C:11251234B:312345

Loop 4, j = 5: A[5] = 3 → B[C[3] = 3] = 3; C[3]−− = 2.

A:4134312345C:11241234B:3412345

Loop 4, j = 4: A[4] = 4 → B[C[4] = 5] = 4; C[4]−− = 4.

A:4134312345C:11141234B:33412345

Loop 4, j = 3: A[3] = 3 → B[C[3] = 2] = 3; C[3]−− = 1.

A:4134312345C:01141234B:133412345

Loop 4, j = 2: A[2] = 1 → B[C[1] = 1] = 1; C[1]−− = 0.

A:4134312345C:01131234B:1334412345

Loop 4, j = 1: A[1] = 4 → B[C[4] = 4] = 4; C[4]−− = 3.

A:4134312345C:01131234B:1334412345

Result B = [1, 3, 3, 4, 4]. No comparisons were made; scanning from back to front keeps equal keys in order (stable).

Linear-Time Sorting

Counting Sort: Complexity and Java

LoopWorkTime
1C[1..k] = 0Θ(k)
2countingΘ(n)
3prefix sumsΘ(k)
4place into BΘ(n)
TotalΘ(n + k)
int[] countingSort(int[] arr) { // elements ≥ 0
  if (arr.length == 0) return arr;
  int max = Arrays.stream(arr).max().getAsInt();
  int[] count = new int[max + 1];
  for (int x : arr) count[x]++;
  for (int i = 1; i <= max; i++) count[i] += count[i - 1];
  int[] output = new int[arr.length];
  for (int i = arr.length - 1; i >= 0; i--) {
    output[count[arr[i]] - 1] = arr[i];
    count[arr[i]]--;
  }
  return output;
}
  • If k = O(n) it is Θ(n): it beats the Ω(n log n) bound because it makes no comparisons.
  • Extra memory Θ(n + k); impractical if k is huge (e.g. 32-bit integers). The Java version handles only non-negative integers and uses the range 0 … max.
  • Scanning the last loop from back to front makes it stable → the basis of radix sort.

Linear-Time Sorting

Radix Sort

  • Sorts non-negative integers digit by digit, starting from the least significant digit (the ones): LSD.
  • Each pass does a stable sort (counting sort, k = 10) on that digit.
  • Stability is essential: sorting by the tens digit must preserve the order of the ones.
  • d digits, base b: Θ(d · (n + b)). Θ(n) if d is constant.
  • Extra memory Θ(n + b); not in place.
int[] radixSort(int[] arr) { // elements ≥ 0
  if (arr.length == 0) return arr;
  int max = Arrays.stream(arr).max().getAsInt();
  for (long exp = 1; max / exp > 0; exp *= 10)
    arr = countByDigit(arr, (int) exp);
  return arr;
}
int[] countByDigit(int[] arr, int exp) {
  int n = arr.length;
  int[] output = new int[n], count = new int[10];
  for (int x : arr) count[(x / exp) % 10]++;
  for (int i = 1; i < 10; i++) count[i] += count[i - 1];
  for (int i = n - 1; i >= 0; i--) {
    int b = (arr[i] / exp) % 10;
    output[--count[b]] = arr[i];
  }
  return output;
}

Linear-Time Sorting · Step by step

Radix Sort Step by Step

Input17045759080224266

Input [170, 45, 75, 90, 802, 24, 2, 66]. The largest number is 802: 3 digits → 3 passes (starting from the least significant digit, LSD).

exp = 1: ones digit1700455755900802224422666

Pass 1: each number's ones digit (shown below) is the key; a missing digit counts as 0.

sorted by the ones digit1700900802222244455755666

After the stable counting sort: [170, 90, 802, 2, 24, 45, 75, 66]. Numbers with equal digits kept their previous order.

exp = 10: tens digit1707909802020242454757666

Pass 2: each number's tens digit (shown below) is the key; a missing digit counts as 0.

sorted by the tens digit8020202424546661707757909

After the stable counting sort: [802, 2, 24, 45, 66, 170, 75, 90]. Numbers with equal digits kept their previous order.

exp = 100: hundreds digit8028202404506601701750900

Pass 3: each number's hundreds digit (shown below) is the key; a missing digit counts as 0.

sorted by the hundreds digit2024045066075090017018028

After the stable counting sort: [2, 24, 45, 66, 75, 90, 170, 802]. All digits processed → sorted.

Linear-Time Sorting

Radix Sort: Why Stability Is Needed

input456167276467273123ones273123456276167467tens123456167467273276hundreds123167273276456467
  • The green columns are the suffixes sorted so far: after k passes the array is sorted by its last k digits.
  • In pass 2, 167 and 467 have the same tens digit (6); the order from pass 1 (7 = 7, input order) is preserved.
  • If the inner sort were unstable, 273 and 276, say, could swap in the last pass and ruin the result.

Linear-Time Sorting · Run the code step by step

countByDigit(arr, 1): arr = [32, 15, 41, 23, 11]

arr3215412311count00010203040506070809output01234

exp = 1: sorting by the ones digit. count[0..9] is zero, output is empty.

arr3215412311count00011203040506070809output01234

x = 32: digit (32 / 1) % 10 = 2 → count[2]++ = 1.

arr3215412311count00011203041506070809output01234

x = 15: digit (15 / 1) % 10 = 5 → count[5]++ = 1.

arr3215412311count00111203041506070809output01234

x = 41: digit (41 / 1) % 10 = 1 → count[1]++ = 1.

arr3215412311count00111213041506070809output01234

x = 23: digit (23 / 1) % 10 = 3 → count[3]++ = 1.

arr3215412311count00211213041506070809output01234

x = 11: digit (11 / 1) % 10 = 1 → count[1]++ = 2.

arr3215412311count00213243445556575859output01234

Prefix sums: count[b] is now the number of items with digit ≤ b; e.g. count[3] = 4. This is one more than the position in output of the last item with that digit.

arr3215412311count00113243445556575859output0111234

i = 4: x = 11, b = 1. --count[1] = 1 → output[1] = 11.

arr3215412311count00113233445556575859output011122334

i = 3: x = 23, b = 3. --count[3] = 3 → output[3] = 23.

arr3215412311count00013233445556575859output41011122334

i = 2: x = 41, b = 1. --count[1] = 0 → output[0] = 41. 41 and 11 share a digit; scanning right to left puts 41 (earlier in the input) to the left of 11: stable.

arr3215412311count00013233444556575859output4101112233154

i = 1: x = 15, b = 5. --count[5] = 4 → output[4] = 15.

arr3215412311count00012233444556575859output410111322233154

i = 0: x = 32, b = 2. --count[2] = 2 → output[2] = 32.

arr3215412311count00012233444556575859output410111322233154

output = [41, 11, 32, 23, 15]: sorted by the ones digit. The next pass, exp = 10, looks at the tens digit; stability preserves the order of the ones.

int[] countByDigit(int[] arr, int exp) {
  int n = arr.length;
  int[] output = new int[n], count = new int[10];
  for (int x : arr) count[(x / exp) % 10]++;
  for (int i = 1; i < 10; i++) count[i] += count[i - 1];
  for (int i = n - 1; i >= 0; i--) {
    int b = (arr[i] / exp) % 10;
    output[--count[b]] = arr[i];
  }
  return output;
}

Linear-Time Sorting

Bucket Sort

  • Suppose the input consists of numbers uniformly distributed over [0, 1).
  • Create n buckets; a value x goes into bucket ⌊n · x⌋.
  • Sort each bucket (with insertion sort) and concatenate the buckets in order.
  • Expected time Θ(n); worst case Θ(n²) if every item lands in one bucket.
  • The tenth algorithm in the simulator; it makes an assumption about the distribution.
A = .78, .17, .39, .26, .72, .94, .21, .12, .23, .680∅1.12.172.21.23.263.394∅5∅6.687.72.788∅9.94Concatenate: .12, .17, .21, .23, .26, .39, .68, .72, .78, .94

Timsort

Timsort: A Hybrid Sort

  • Tim Peters (2002, Python). Java uses Timsort for object arrays (Arrays.sort(Object[]), List.sort) and Python for sorted.
  • It exploits the sorted stretches (runs) common in real data: short runs are extended with insertion sort, and runs are combined with merge sort.
  • Stable; O(n log n) worst case, Θ(n) on sorted input; O(n) extra memory.
static final int MIN_RUN = 4;
void timSort(int[] arr) {
  int n = arr.length;
  for (int i = 0; i < n; i += MIN_RUN)            // 1) runs
    insertionSort(arr, i, Math.min(i + MIN_RUN - 1, n - 1));
  for (int size = MIN_RUN; size < n; size *= 2)  // 2) merge
    for (int left = 0; left < n; left += 2 * size) {
      int mid = left + size - 1;
      int right = Math.min(left + 2 * size - 1, n - 1);
      if (mid < right) merge(arr, left, mid, right);
    }
}

This version is simplified with fixed-length runs (real Timsort uses minrun 32–64, natural run detection and "galloping").

Timsort · Step by step

Timsort Step by Step (MIN_RUN = 4)

Input, MIN_RUN = 45021172233194105361278819

Array [5, 21, 7, 23, 19, 10, 3, 12, 8, 1], n = 10. The array is split into runs of 4 (colors).

insertionSort(0, 3)5071212233194105361278819

Run [0..3] sorted with insertion sort: [5, 7, 21, 23].

insertionSort(4, 7)5071212233341051261978819

Run [4..7] sorted with insertion sort: [3, 10, 12, 19].

insertionSort(8, 9)5071212233341051261971889

Run [8..9] sorted with insertion sort: [1, 8].

size = 4: merge(0, 3, 7)3051721031241952162371889

size = 4: [5, 7, 21, 23] + [3, 10, 12, 19] merged → [3, 5, 7, 10, 12, 19, 21, 23].

size = 4: [8..9] alone3051721031241952162371889

size = 4: run [8..9] has no partner to merge with (mid = 11 ≥ right = 9) → stays as is.

size = 8: merge(0, 7, 9)1031527384105126197218239

size = 8: [3, 5, 7, 10, 12, 19, 21, 23] + [1, 8] merged → [1, 3, 5, 7, 8, 10, 12, 19, 21, 23]. The array is sorted.

Comparisons

Comparing the Sorting Algorithms

AlgorithmBestAverageWorstExtra memoryStableIn place
Bubble (with flag)Θ(n)Θ(n²)Θ(n²)O(1)YesYes
InsertionΘ(n)Θ(n²)Θ(n²)O(1)YesYes
SelectionΘ(n²)Θ(n²)Θ(n²)O(1)NoYes
Shell (n/2 gaps)Θ(n log n)depends on gapsΘ(n²)O(1)NoYes
MergeΘ(n log n)Θ(n log n)Θ(n log n)Θ(n)YesNo
QuickΘ(n log n)Θ(n log n)Θ(n²)O(log n)*NoYes
HeapsortO(n log n)Θ(n log n)Θ(n log n)O(1)NoYes
CountingΘ(n + k)Θ(n + k)Θ(n + k)Θ(n + k)YesNo
Radix (LSD)Θ(d(n + b))Θ(d(n + b))Θ(d(n + b))Θ(n + b)YesNo
BucketΘ(n)Θ(n)†Θ(n²)Θ(n)YesNo
TimsortΘ(n)Θ(n log n)Θ(n log n)O(n)YesNo

* recursion stack (if the smaller part is processed first). † expected, under a uniform distribution. k: key range, d: number of digits, b: base. Timsort's Θ(n) best case is for the real version with natural run detection.

Comparisons

How Many Comparisons? n = 1000

AlgorithmRandomSortedReverse sorted
Bubble (with flag)497,960999499,500
Insertion252,205999499,500
Selection499,500499,500499,500
Shell (n/2)14,7238,00611,716
Merge8,6935,0444,932
Quick (last-item pivot)11,384499,500499,500
Heapsort16,79117,58315,965
  • Element comparisons, counted by running this chapter's code for n = 1000 (the random array was generated with a fixed seed).
  • For reference: n² / 2 ≈ 500,000, n log₂ n ≈ 9,966. On sorted input, quicksort with a last-item pivot hits its worst case.

Comparisons

Which Algorithm When?

Small or nearly sortedInsertion sort: low overhead, adaptive, stable.
General purpose, speedQuicksort (random / median-of-three pivot); cache-friendly, in place.
Stability requiredMerge sort or Timsort (libraries' object sort).
Worst-case guarantee, little memoryHeapsort: O(n log n) and O(1) extra memory (introsort's fallback).
Small integer keysCounting or radix sort: Θ(n + k), Θ(d(n + b)).
Linked lists, external memoryMerge sort: needs no random access; large files on disk are merged piece by piece.

Bubble and selection sort are for teaching; selection sort only makes sense when writes are very expensive.

Summary

Summary

TopicKey points
Criteriatime (best/average/worst), extra memory, stability, in place, adaptivity
Bubbleadjacent swaps; largest at the end after each pass; best Θ(n) with a flag, otherwise Θ(n²); stable
Insertionsave the key, shift larger items, insert; Θ(n + inversions); best on small or nearly sorted data
Selectionfind the minimum, move it to the front; always n(n − 1)/2 comparisons, ≤ n − 1 swaps; unstable
Shellinsertion sort with shrinking gaps; in place, unstable
Mergesplit–sort–merge; T(n) = 2T(n/2) + Θ(n) = Θ(n log n); Θ(n) extra memory; stable
Quickpartition around a pivot; Θ(n log n) on average, Θ(n²) worst case; in place, unstable
Heapsortbuild a max-heap in O(n), n − 1 times move the root to the end and sift down; O(n log n), O(1) extra memory
Linearcomparison lower bound Ω(n log n); counting Θ(n + k), radix Θ(d(n + b)), bucket Θ(n) expected

Summary

Common Mistakes

Running off the arrayIn bubble sort the inner loop must be j < n − 1 − i; writing j < n overflows at arr[j+1].
Not saving the keyIn insertion sort, if key is not saved first, the first shift overwrites it.
Tracking the minimum by valueIn selection sort, without the index the swap cannot be done.
Breaking stabilityUsing < instead of <= in merge, or scanning front to back in counting sort, reorders equal keys.
Including the pivot in the recursionWriting quickSort(arr, lo, p) processes the pivot again; with two items left it can recurse forever.
"Quicksort is always n log n"On sorted input a fixed pivot gives Θ(n²) and recursion n deep (StackOverflowError).

Summary

Sample Problems

  1. Apply bubble sort with a flag to [5, 2, 4, 6, 1, 3]. Write the array after each pass.
  2. How many comparisons and shifts does insertion sort need for the same array? How many swaps does selection sort make?
  3. What are the array and the return value after partition (Lomuto, pivot = last item) runs on [2, 8, 7, 1, 3, 5, 6, 4]?
  4. What max-heap does bottom-up construction build from [4, 10, 3, 5, 1]?
  5. For A = [2, 5, 3, 0, 2, 3, 0, 3] (keys 0 … 5), what is the array C after the third loop of counting sort?
  6. At most how many comparisons does merge sort need to sort 8 items?
Show answers
  1. [2, 4, 5, 1, 3, 6] → [2, 4, 1, 3, 5, 6] → [2, 1, 3, 4, 5, 6] → [1, 2, 3, 4, 5, 6] → [1, 2, 3, 4, 5, 6] (no swap in the last pass → stop).
  2. Insertion: 12 comparisons, 9 shifts (= number of inversions). Selection: 15 comparisons, 3 swaps.
  3. [2, 1, 3, 4, 7, 5, 6, 8], returns 3 (pivot 4 at index 3).
  4. [10, 5, 3, 4, 1]: first i = 1 (10 is already larger), then i = 0: 4 ↔ 10, 4 ↔ 5.
  5. C = [2, 2, 4, 7, 7, 8] (counts [2, 0, 2, 3, 0, 1] and their prefix sums).
  6. Merging m items on a level takes at most m − 1 comparisons: 4·1 + 2·3 + 1·7 = 17.

Summary

Exercises

  1. How many comparisons and swaps does bubble sort make on a reverse-sorted array of n = 6 items?
  2. Give a three-item example showing that selection sort is unstable.
  3. If insertion sort finds the correct position by binary search, what is its time complexity?
  4. What happens when quicksort (last-item pivot) is called on an already sorted array of 1000 items? How can it be fixed?
  5. Sort [329, 457, 657, 839, 436, 720, 355] with LSD radix sort; write the array after each pass.
  6. Propose an O(n log n) method for counting the inversions in an array.
Answers / hints
  1. Every pair is inverted on every pass: 5 + 4 + 3 + 2 + 1 = 15 comparisons and 15 swaps.
  2. [2ₐ, 2ᵦ, 1] → the first pass swaps 1 and 2ₐ → [1, 2ᵦ, 2ₐ].
  3. Comparisons drop to O(n log n), but the shifts are still Θ(n²) → Θ(n²) overall.
  4. Each partition splits off only the pivot: 999 + 998 + … + 1 = 499,500 comparisons and recursion 1000 deep. Use a random pivot or median of three.
  5. Ones: [720, 355, 436, 457, 657, 329, 839]; tens: [720, 329, 436, 839, 355, 457, 657]; hundreds: [329, 355, 436, 457, 657, 720, 839].
  6. Modify merge sort: whenever an item is taken from the right half, add the number of items left in the left half (L.length − i) to the inversion count; O(n log n) overall.

Summary

References

  • Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., MIT Press, 2022 (Chapters 2, 6, 7, 8).
  • Sedgewick and Wayne, Algorithms, 4th ed., Addison-Wesley, 2011 (Chapter 2: Sorting).
  • D. E. Knuth, The Art of Computer Programming, Vol. 3: Sorting and Searching, 2nd ed., 1998.
  • E. D. Demaine and C. E. Leiserson, MIT 6.046J Introduction to Algorithms lecture notes, 2005 (partitioning and counting sort examples).
  • C. A. R. Hoare, "Quicksort", The Computer Journal 5(1), 1962; D. L. Shell, "A High-Speed Sorting Procedure", CACM 2(7), 1959.
  • T. Peters, "listsort.txt" (Timsort design notes), CPython, 2002.

The End

Chapter 3: Sorting Algorithms
Previous: Chapter 2: Algorithm Complexity · Next: Chapter 4: Searching Algorithms
1 / 1 Sercan KÜLCÜ. All rights reserved.