Chapter 7: Randomized Algorithms
Introduction
Learning Outcomes
Define a randomized algorithm and its expected running time; distinguish Las Vegas from Monte Carlo algorithms.
Write the Fisher–Yates shuffle, show why the naive shuffle is biased, and derive BogoSort's expected Θ(n · n!) time.
Trace random-pivot partitioning step by step; justify the expected O(n log n) and worst-case O(n²) bounds.
Compute the error probability of π estimation, the Miller–Rabin primality test and Karger's min-cut algorithm, and reduce it by repetition.
Introduction
Algorithms That Gamble
- Some problem-solving techniques resemble games of chance.
- The algorithm uses randomness when making decisions: it flips a coin, rolls a die, or picks a random element.
- Example game: roll two dice; the player wins if the sum is 7 or 11.
Introduction
Randomized Algorithms
- A randomized algorithm uses random numbers in addition to its input.
- Two runs on the same input may take different steps: the running time (and sometimes the output) can vary from run to run.
- The running time is now a random variable, so we study the expected time E[T(n)]. The expectation is over the algorithm's own coin flips, not over the input.
Introduction
Why Randomness?
Deterministic QuickSort takes O(n²) on a sorted array. With a random pivot, no input is always bad: the worst case arises only from unlucky coin flips.
A randomized algorithm is often much shorter and clearer than the best deterministic one (Karger's min cut, randomized QuickSelect).
Miller–Rabin decides whether a number with hundreds of digits is prime in milliseconds, with a tiny error probability.
Monte Carlo simulation, random sampling and leader election in distributed systems are hard without randomness.
The price: time or correctness is guaranteed only probabilistically, and we need a good random number generator.
Introduction
Probability Review
| Concept | Definition / rule | Example |
|---|---|---|
| Expected value | E[X] = Σ x · P(X = x) | expected value of a die: 3.5 |
| Linearity | E[X + Y] = E[X] + E[Y] (no independence needed) | sum of two dice: 7 |
| Indicator variable | XA = 1 (if A occurs), else 0 → E[XA] = P(A) | QuickSort: "were i and j compared?" |
| Geometric distribution | trials with success probability p: expected trials until the first success 1/p | BogoSort: p = 1/n! → n! shuffles |
| Independent repetition | k independent trials, each failing with probability p: all fail with pk | Miller–Rabin: (1/4)k |
| Markov's inequality | if X ≥ 0, P(X ≥ c · E[X]) ≤ 1/c | Las Vegas → Monte Carlo |
Types of Algorithms
Las Vegas and Monte Carlo
| Las Vegas | Monte Carlo | |
|---|---|---|
| Correctness | Always correct result | Wrong result with small probability |
| Time | Random; we analyze the expected time | Bounded (deterministic upper bound) |
| What is random | how long it runs | whether the answer is correct |
| Examples | Randomized QuickSort, QuickSelect, BogoSort, Fisher–Yates | Miller–Rabin, Karger's min cut, π estimation |
When Miller–Rabin says "composite", it is certainly right; only the answer "prime" can be wrong.
Numerical simulations such as π estimation can err in either direction; the error shrinks with the number of samples.
Types of Algorithms
Reducing the Error Probability
If one round errs with probability at most q, then after k independent rounds the error is ≤ qk.
- Miller–Rabin: q ≤ 1/4 → with k = 20 rounds, ≤ 4−20 ≈ 9 · 10−13.
- Karger: success ≥ 2/(n(n−1)); with n(n−1)/2 · ln n repetitions, failure ≤ 1/n.
- Las Vegas → Monte Carlo: stop the algorithm after 2 · E[T] steps. Markov: P(T ≥ 2 E[T]) ≤ 1/2 → "no answer" with probability at most 1/2.
- Monte Carlo → Las Vegas: if an answer can be verified quickly (e.g., multiply the factors found), repeat until it checks out. Expected repetitions = 1/p.
Miller–Rabin error bound 4−k (log scale): each round divides the bound by 4. k = 10 rounds suffice to get below 10−6.
Shuffling
Example: Shuffling
- Randomness is used to shuffle the elements of a list: a deck of cards, random test data, a playlist.
- Correctness: every element lands in every position with equal probability; more strongly, each of the n! permutations must have probability 1/n!.
- Running time: Fisher–Yates always performs exactly n − 1 swaps: Θ(n). What varies is the result, not the time.
- Built into Java:
Collections.shuffle(list)(Fisher–Yates).
| n | n! permutations |
|---|---|
| 3 | 6 |
| 4 | 24 |
| 10 | 3,628,800 |
| 52 | ≈ 8.07 · 1067 |
Shuffling · Run the code step by step
Fisher–Yates (Knuth) Shuffle
Start: a = [1, 2, 3, 4, 5]. Working from the end, each position receives a random element from those not yet placed.
i = 4: one of the 5 elements in a[0..4] will move to a[4].
rng.nextInt(5) → j = 1 (probability 1/5).
Swap a[4] ↔ a[1] → [1, 5, 3, 4, 2]. a[4] = 2 is now final (green).
i = 3: one of the 4 elements in a[0..3] will move to a[3].
rng.nextInt(4) → j = 3 (j = i: the element stays put).
Swap with itself: the array is unchanged; a[3] = 4 is final.
i = 2: one of the 3 elements in a[0..2] will move to a[2].
rng.nextInt(3) → j = 0 (probability 1/3).
Swap a[2] ↔ a[0] → [3, 5, 1, 4, 2]. a[2] = 1 is now final (green).
i = 1: one of the 2 elements in a[0..1] will move to a[1].
rng.nextInt(2) → j = 1 (j = i: the element stays put).
Swap with itself: the array is unchanged; a[1] = 5 is final.
Loop done: [3, 5, 1, 4, 2]. The number of possible draw sequences is 5 · 4 · 3 · 2 = 120 = 5!: each permutation arises exactly once → uniform distribution.
static void shuffle(int[] a) { for (int i = a.length - 1; i > 0; i--) { int j = rng.nextInt(i + 1); // 0 ≤ j ≤ i int t = a[i]; a[i] = a[j]; a[j] = t; } }
Shuffling
The Naive Shuffle Is Biased
// BIASED: for each i, pick from 0..n-1 for (int i = 0; i < n; i++) { int j = rng.nextInt(n); swap(a, i, j); }
- Each of the n draws takes n values → nn equally likely paths.
- n = 3: 33 = 27 paths, 3! = 6 permutations. 27 is not divisible by 6 → the permutations cannot be equally likely.
- The fix for this common pattern is Fisher–Yates:
nextInt(i + 1).
| result | paths | probability | should be |
|---|---|---|---|
123 | 4/27 | 0.148 | 1/6 ≈ 0.167 |
132 | 5/27 | 0.185 | 1/6 ≈ 0.167 |
213 | 5/27 | 0.185 | 1/6 ≈ 0.167 |
231 | 5/27 | 0.185 | 1/6 ≈ 0.167 |
312 | 4/27 | 0.148 | 1/6 ≈ 0.167 |
321 | 4/27 | 0.148 | 1/6 ≈ 0.167 |
All 27 draw sequences for [1, 2, 3] were enumerated (Fisher–Yates has 3 · 2 = 6 paths, one per permutation).
BogoSort
BogoSort
- One of the worst-performing sorting algorithms ("stupid sort", permutation sort).
- Basic idea: shuffle randomly until the list is sorted.
- A fun and instructive Las Vegas algorithm; never used in practice.
- Is the list sorted? Yes → stop.
- No → shuffle the list randomly and go back to step 1.
static void bogoSort(int[] a) { while (!isSorted(a)) shuffle(a); // Fisher–Yates } static boolean isSorted(int[] a) { for (int i = 0; i + 1 < a.length; i++) if (a[i] > a[i + 1]) return false; return true; }
BogoSort
BogoSort Example: [5, 2, 8, 4]
The list [5, 2, 8, 4]; each shuffle uses Fisher–Yates (a seeded run). Red: the first adjacent pair out of order.
Start [5, 2, 8, 4]: a[0] = 5 > a[1] = 2 → not sorted, shuffling begins.
Shuffle 1: [5, 4, 2, 8] → a[0] = 5 > a[1] = 4, not sorted.
Shuffle 2: [8, 4, 5, 2] → a[0] = 8 > a[1] = 4, not sorted.
Shuffle 3: [4, 2, 8, 5] → a[0] = 4 > a[1] = 2, not sorted.
Shuffle 4: [2, 8, 4, 5] → a[1] = 8 > a[2] = 4, not sorted.
Shuffle 5: [5, 8, 2, 4] → a[1] = 8 > a[2] = 2, not sorted.
Shuffle 6: [8, 4, 2, 5] → a[0] = 8 > a[1] = 4, not sorted.
Shuffle 7: [2, 8, 5, 4] → a[1] = 8 > a[2] = 5, not sorted.
Shuffle 8: [2, 4, 5, 8] → sorted! The algorithm stops. This run needed 8 shuffles; the expected number is 4! = 24.
BogoSort
Analysis of BogoSort
- With n distinct elements, a random shuffle is sorted with probability p = 1/n!.
- Trials are independent → geometric distribution: the expected number of shuffles is 1/p = n!.
- Each round: shuffle O(n) + check O(n) → expected time Θ(n · n!). This is the same order as O((n + 1)!), since (n + 1)! = (n + 1) · n!.
- Best case: the array is already sorted → one check, O(n).
- Worst case: unbounded. Unlucky shuffles can go on forever (with probability tending to 0). O((n + 1)!) bounds only the expected time, not the worst case.
| n | expected shuffles n! | time (10⁹ shuffles/s) |
|---|---|---|
| 5 | 120 | 0.12 µs |
| 10 | 3,628,800 | 3.6 ms |
| 12 | 479,001,600 | 0.48 s |
| 13 | 6,227,020,800 | 6.2 s |
| 15 | 1,307,674,368,000 | ≈ 22 minutes |
| 20 | 2,432,902,008,176,640,000 | ≈ 77 years |
Randomized QuickSort
Pivot Choice and Its Effect
| Pivot strategy | Bad input | Expected / typical | Worst |
|---|---|---|---|
| First or last element | sorted / reverse-sorted array (very common!) | O(n log n)* | O(n²) |
| Median-of-3 | specially crafted "killer" arrays | O(n log n)* | O(n²) |
| Random element | none: bad runs come only from bad luck | O(n log n) for every input | O(n²) |
| True median (BFPRT) | none | O(n log n) | O(n log n) |
- * For deterministic strategies, O(n log n) holds only if the input is randomly ordered (average case). With a random pivot, the expectation is over the algorithm's own choices (expected time).
- A good pivot improves the expected time but does not remove the worst case: O(n²) is still possible with a random pivot, just very unlikely.
- BFPRT has large constants; in practice a random or median-of-3 pivot is preferred.
Randomized QuickSort
QuickSort Run with Random Pivots
[6, 5, 3, 1, 8, 7, 2, 4]; pivots chosen at random: 3, 1, 4, 6, 8. Black: pivot · red outline: pointers i and j · green: final · faded: outside the active subarray.
Subarray a[0..7] = [6, 5, 3, 1, 8, 7, 2, 4]: the pivot is chosen at random → 3 (index 2, black box).
i moves right past elements smaller than 3 and stops at a[0] = 6; j moves left past elements larger than 3 and stops at a[6] = 2. i < j → swap.
Swap a[0] ↔ a[6] → [2, 5, 3, 1, 8, 7, 6, 4].
i moves right past elements smaller than 3 and stops at a[1] = 5; j moves left past elements larger than 3 and stops at a[3] = 1. i < j → swap.
Swap a[1] ↔ a[3] → [2, 1, 3, 5, 8, 7, 6, 4].
The pointers have crossed (i ≥ j): pivot 3 is in its final place (index 2). Only smaller elements lie to its left, only larger ones to its right.
Subarray a[0..1] = [2, 1]: the pivot is chosen at random → 1 (index 1, black box).
i moves right past elements smaller than 1 and stops at a[0] = 2; j moves left past elements larger than 1 and stops at a[1] = 1. i < j → swap.
Swap a[0] ↔ a[1] → [1, 2, 3, 5, 8, 7, 6, 4]. The pivot itself may move; we track its position.
The pointers have crossed (i ≥ j): pivot 1 is in its final place (index 0). Only smaller elements lie to its left, only larger ones to its right.
The one-element subarray a[1] = 2 is already sorted: final.
Subarray a[3..7] = [5, 8, 7, 6, 4]: the pivot is chosen at random → 4 (index 7, black box).
i moves right past elements smaller than 4 and stops at a[3] = 5; j moves left past elements larger than 4 and stops at a[7] = 4. i < j → swap.
Swap a[3] ↔ a[7] → [1, 2, 3, 4, 8, 7, 6, 5]. The pivot itself may move; we track its position.
The pointers have crossed (i ≥ j): pivot 4 is in its final place (index 3). Only smaller elements lie to its left, only larger ones to its right.
Subarray a[4..7] = [8, 7, 6, 5]: the pivot is chosen at random → 6 (index 6, black box).
i moves right past elements smaller than 6 and stops at a[4] = 8; j moves left past elements larger than 6 and stops at a[7] = 5. i < j → swap.
Swap a[4] ↔ a[7] → [1, 2, 3, 4, 5, 7, 6, 8].
i moves right past elements smaller than 6 and stops at a[5] = 7; j moves left past elements larger than 6 and stops at a[6] = 6. i < j → swap.
Swap a[5] ↔ a[6] → [1, 2, 3, 4, 5, 6, 7, 8]. The pivot itself may move; we track its position.
The pointers have crossed (i ≥ j): pivot 6 is in its final place (index 5). Only smaller elements lie to its left, only larger ones to its right.
The one-element subarray a[4] = 5 is already sorted: final.
Subarray a[6..7] = [7, 8]: the pivot is chosen at random → 8 (index 7, black box).
Pivot 8: i skips 7 and stops at 8; j is already at 8. i ≥ j → no swap, 8 is in its final place (index 7).
The one-element subarray a[6] = 7 is already sorted: final. The array is sorted: [1, 2, 3, 4, 5, 6, 7, 8].
Randomized QuickSort
Randomized QuickSort: Code
static Random rng = new Random(); static void quickSort(int[] a, int lo, int hi) { if (lo < hi) { int p = partition(a, lo, hi); // random quickSort(a, lo, p - 1); quickSort(a, p + 1, hi); } } static int partition(int[] a, int lo, int hi) { int r = lo + rng.nextInt(hi - lo + 1); swap(a, r, hi); // pivot to end int pivot = a[hi], k = lo; for (int i = lo; i < hi; i++) if (a[i] <= pivot) swap(a, i, k++); // smaller left swap(a, k, hi); // pivot in place return k; }
- The pivot is uniformly random. The output is always correct (Las Vegas); only the running time depends on the pivots.
partitionmoves the random pivot to the end, then runs Lomuto: for [6, 5, 3, 1, 8, 7, 2, 4], r = 2 (pivot 3) → [1, 2, 3, 6, 8, 7, 5, 4], returns 2.- Extra space: the recursion stack, expected depth O(log n); recursing on the smaller part first makes it O(log n) in the worst case too.
- With many duplicate keys, use three-way partitioning (<, =, >).
Randomized QuickSort
Analysis of QuickSort
| Case | Recurrence | Time |
|---|---|---|
| Best | T(n) = 2T(n/2) + Θ(n) | Θ(n log n) |
| Expected (random pivot) | T(n) = (1/n) Σq [T(q) + T(n−1−q)] + Θ(n) | Θ(n log n) |
| Worst | T(n) = T(n−1) + Θ(n) | Θ(n²) |
Probability of the worst case with a random pivot: the smallest or largest element must be chosen at every step → 2n−1/n! (≈ 2 · 10−13 for n = 20).
Randomized QuickSort
Expected Number of Comparisons
- Label the elements in sorted order: z1 < z2 < … < zn. Indicator Xij = 1 ⇔ zi and zj are compared.
- Two elements are compared only if one is the pivot. If the first pivot from {zi, …, zj} lies strictly between them, they are split apart and never compared.
- That first pivot is uniformly random → P(Xij = 1) = 2 / (j − i + 1).
- Linearity: E[X] = Σi<j 2/(j − i + 1) < 2n Hn ≈ 1.39 n log₂ n.
- Exact value: E[X] = 2(n + 1)Hn − 4n.
| n | random pivot (expected) | sorted input, first-element pivot |
|---|---|---|
| 10 | 24.4 | 45 |
| 100 | 648 | 4,950 |
| 1,000 | 10,986 | 499,500 |
| 106 | 2.48 · 107 | 5.0 · 1011 |
Randomized QuickSort
Randomized Selection (QuickSelect)
static int select(int[] a, int lo, int hi, int s) { if (lo == hi) return a[lo]; int p = partition(a, lo, hi); if (s == p) return a[p]; if (s < p) return select(a, lo, p - 1, s); return select(a, p + 1, hi, s); }
- Goal: find the s-th smallest element (e.g., the median) without sorting.
- It uses the same random partitioning but recurses on one side only.
- Expected time O(n) (≈ 3.39 n comparisons for the median); worst case O(n²).
- Example: [6, 5, 3, 1, 8, 7, 2, 4] with s = 2 (0-based). Partitioning around pivot 3 gives p = 2 → the answer 3 on the first try.
Randomized QuickSort · Run the code step by step
QuickSelect: Code Trace
Array [6, 5, 3, 1, 8, 7, 2, 4]; target s = 5 (6th smallest). The first call covers the whole array: lo = 0, hi = 7.
partition chose random pivot 3; after partitioning its final position is p = 2: left ≤ 3, right > 3.
s = 5 > p = 2 → the target is on the right: recurse only on [3, 7]; the left side is never examined again.
New call select(a, 3, 7, 5): 5 elements in range, lo ≠ hi.
partition chose random pivot 7; after partitioning its final position is p = 6: left ≤ 7, right > 7.
s = 5 < p = 6 → the target is on the left: recurse only on [3, 5].
New call select(a, 3, 5, 5): 3 elements in range, lo ≠ hi.
partition chose random pivot 5; after partitioning its final position is p = 4: left ≤ 5, right > 5.
s = 5 > p = 4 → the target is on the right: recurse only on [5, 5]; the left side is never examined again.
New call select(a, 5, 5, 5): 1 element in range, lo = hi.
lo = hi = 5: one element left → return a[5] = 6.
select(a, 0, 7, 5) = 6: position 5 (0-based) of the sorted array. The array was never fully sorted; only the needed parts were partitioned.
static int select(int[] a, int lo, int hi, int s) { if (lo == hi) return a[lo]; int p = partition(a, lo, hi); if (s == p) return a[p]; if (s < p) return select(a, lo, p - 1, s); return select(a, p + 1, hi, s); }
Monte Carlo Algorithms
Monte Carlo Estimate of π
- Throw uniformly random points into the unit square. The quarter circle has area π/4 and the square area 1 → P(inside) = π/4, so π ≈ 4 · inside / n.
- The error falls like 1/√n: one more correct digit needs 100 times as many points.
static double estimatePi(int n) { int inside = 0; for (int t = 0; t < n; t++) { double x = rng.nextDouble(), y = rng.nextDouble(); if (x * x + y * y <= 1) inside++; } return 4.0 * inside / n; }
300 points: 239 inside → π ≈ 4 · 239/300 = 3.187.
Monte Carlo Algorithms · Run the code step by step
Monte Carlo π: Code Trace
n = 8 points will be thrown; inside counts those landing inside the quarter circle. (A seeded Random makes the run reproducible.)
t = 0: (0.62, 0.19), x² + y² = 0.42 ≤ 1 → inside. inside = 1.
t = 1: (0.93, 0.09), x² + y² = 0.88 ≤ 1 → inside. inside = 2.
t = 2: (0.58, 0.49), x² + y² = 0.58 ≤ 1 → inside. inside = 3.
t = 3: (0.00, 0.49), x² + y² = 0.24 ≤ 1 → inside. inside = 4.
t = 4: (0.30, 0.71), x² + y² = 0.59 ≤ 1 → inside. inside = 5.
t = 5: (0.78, 0.72), x² + y² = 1.13 > 1 → outside. inside = 5.
t = 6: (0.35, 0.98), x² + y² = 1.08 > 1 → outside. inside = 5.
t = 7: (0.50, 0.36), x² + y² = 0.38 ≤ 1 → inside. inside = 6.
4.0 * 6 / 8 = 3.0. With 8 points the error is large (≈ 1.64/√8 ≈ 0.58); the estimate approaches π as n grows.
static double estimatePi(int n) { int inside = 0; for (int t = 0; t < n; t++) { double x = rng.nextDouble(), y = rng.nextDouble(); if (x * x + y * y <= 1) inside++; } return 4.0 * inside / n; }
Monte Carlo Algorithms
Try It: Monte Carlo π
| number of points n | 0 |
| inside | 0 |
| π estimate | — |
| error |π̂ − π| | — |
| standard deviation ≈ 1.64/√n | — |
Green: inside the quarter circle · red: outside. The plot shows the first 3000 points; counting continues over all points. 1.64 = 4 · √(p(1 − p)), p = π/4.
Monte Carlo Algorithms
Miller–Rabin Primality Test
- Fermat's little theorem: if n is prime and 1 < a < n, then an−1 ≡ 1 (mod n). If this fails, n is certainly composite; but some composites (Carmichael numbers) satisfy it for every base.
- Square-root property: if n is prime, x² ≡ 1 (mod n) holds only for x ≡ ±1.
- Write n − 1 = 2s · d (d odd); for a random a, examine the sequence ad, a2d, …, a2s−1d (mod n).
- If the sequence does not start with 1 and n − 1 never appears, a is a witness that n is composite. Otherwise n "passes" this round.
- For composite n, at most 1/4 of the bases are liars → with k independent rounds the error is ≤ 4−k. Each round takes O(log n) modular multiplications: O(k log³ n) bit operations in total.
Monte Carlo Algorithms
Miller–Rabin: Code
static boolean isPrime(long n, int k) { if (n < 4) return n == 2 || n == 3; if (n % 2 == 0) return false; long d = n - 1; int s = 0; // n-1 = 2^s·d while (d % 2 == 0) { d /= 2; s++; } for (int round = 0; round < k; round++) { long a = 2 + rng.nextLong(n - 3); // [2, n-2] long x = modPow(a, d, n); // a^d mod n if (x == 1 || x == n - 1) continue; boolean witness = true; for (int r = 1; r < s && witness; r++) { x = x * x % n; if (x == n - 1) witness = false; } if (witness) return false; // certainly composite } return true; // probably prime }
modPow: fast modular exponentiation (square-and-multiply), O(log d) multiplications.- The base a is drawn uniformly from [2, n − 2].
- With
long,x * xdoes not overflow only for n < 3 · 109; for larger n useBigInteger.modPoworisProbablePrime.
Monte Carlo Algorithms · Run the code step by step
Miller–Rabin: Code Trace (n = 561)
n = 561 = 3 · 11 · 17 is a Carmichael number: it passes the Fermat test for every base. n ≥ 4 and odd → the early checks do not return.
d = n − 1 = 560, s = 0. Now factor the 2s out of 560.
d is even → d /= 2, s++: d = 280, s = 1.
d is even → d /= 2, s++: d = 140, s = 2.
d is even → d /= 2, s++: d = 70, s = 3.
d is even → d /= 2, s++: d = 35, s = 4. d is odd: 560 = 2⁴ · 35.
Round 0: random base drawn, a = 2.
x = 2³⁵ mod 561 = 263 (by fast exponentiation).
x = 263 is neither 1 nor n − 1 = 560 → the round is not over; with witness = true we check the squares.
r = 1: x = 263² mod 561 = 166. Not 560.
r = 2: x = 166² mod 561 = 67. Not 560.
r = 3: x = 67² mod 561 = 1. Not 560. We reached 1 without seeing n − 1.
The sequence never reached 560: witness stayed true → return false. 561 is certainly composite; 67 is a square root of 1 other than ±1.
static boolean isPrime(long n, int k) { if (n < 4) return n == 2 || n == 3; if (n % 2 == 0) return false; long d = n - 1; int s = 0; // n-1 = 2^s·d while (d % 2 == 0) { d /= 2; s++; } for (int round = 0; round < k; round++) { long a = 2 + rng.nextLong(n - 3); // [2, n-2] long x = modPow(a, d, n); // a^d mod n if (x == 1 || x == n - 1) continue; boolean witness = true; for (int r = 1; r < s && witness; r++) { x = x * x % n; if (x == n - 1) witness = false; } if (witness) return false; // certainly composite } return true; // probably prime }
Monte Carlo Algorithms · Run the code step by step
modPow: 2³⁵ mod 561 Step by Step
Compute 2³⁵ mod 561. Since 35 = 32 + 2 + 1, 2³⁵ = 2³² · 2² · 2¹. Initially result = 1, a = 2.
Bit 0 = 1: result = 1 · 2 mod 561 = 2 (2¹ included). Then a = 2² mod 561 = 4.
Bit 1 = 1: result = 2 · 4 mod 561 = 8 (2² included). Then a = 4² mod 561 = 16.
Bit 2 = 0: result unchanged (8). Only square: a = 16² mod 561 = 256.
Bit 3 = 0: result unchanged (8). Only square: a = 256² mod 561 = 460.
Bit 4 = 0: result unchanged (8). Only square: a = 460² mod 561 = 103.
Bit 5 = 1: result = 8 · 103 mod 561 = 263 (2³² included). Then a = 103² mod 561 = 511.
d = 0: loop done. return 263 = 2³⁵ mod 561, using only 6 squarings + 3 multiplications instead of 34.
static long modPow(long a, long d, long n) { long result = 1; a %= n; while (d > 0) { if ((d & 1) == 1) // bit is 1 result = result * a % n; a = a * a % n; // square d >>= 1; // next bit } return result; }
Monte Carlo Algorithms
Miller–Rabin Examples
| n | n − 1 | a | ad, a2d, … (mod n) | result |
|---|---|---|---|---|
| 61 | 22 · 15 | 2 | 11, 60 | passes (may be prime) |
| 221 | 22 · 55 | 174 | 47, 220 | passes (may be prime) |
| 221 | 22 · 55 | 137 | 188, 205 | composite (a is a witness) |
| 561 | 24 · 35 | 2 | 263, 166, 67, 1 | composite (a is a witness) |
- 61 is prime: the sequence reaches n − 1 = 60.
- 221 = 13 · 17: a = 174 is a strong liar (the sequence reaches 220), while a = 137 is a witness. That is why we need several rounds.
- 561 = 3 · 11 · 17 is a Carmichael number: 2560 ≡ 1, so the Fermat test is fooled. Miller–Rabin sees that 67² ≡ 1: 67 is a square root of 1 other than ±1 → composite.
Monte Carlo Algorithms
Karger's Min-Cut Algorithm
Start: 6 vertices, 8 edges. The minimum cut is 2 (e.g., {3, 4} | the rest). Dark vertex: the supervertex formed by the last contraction.
Random edge (0, 2) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 7 edges.
Random edge (1, 5) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 6 edges.
Random edge (3, 4) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 5 edges.
Random edge (0, 1) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 2 edges. Two supervertices left → cut found = 2 (minimum; a successful run).
- Problem: find a cut with the fewest edges that splits the graph into two parts (global min cut).
- Algorithm: contract a uniformly random edge until 2 vertices remain; the remaining edges form a cut.
- One run finds a minimum cut with probability at least 2/(n(n−1)) (n = 6: ≥ 1/15). On this graph, 200,000 simulations gave ≈ 0.46.
- The output can be wrong (a larger cut) → Monte Carlo. E.g., if the first edges chosen are (1,2), (4,5), (3,4), (0,1), we get the cut {0,1,2} | {3,4,5} with 3 edges.
- n(n−1)/2 · ln n repetitions → failure ≤ 1/n; O(n4 log n) in total (simple implementation).
Monte Carlo Algorithms
Other Applications
Randomized scheduling (job assignment) and metaheuristics such as simulated annealing and genetic algorithms also use randomness to search; metaheuristics do not guarantee convergence to the optimum.
Monte Carlo Algorithms
Pseudorandom Numbers
- Computers produce pseudorandom numbers rather than true randomness: a deterministic sequence that looks random.
- Linear congruential generator (LCG): Xk+1 = (a · Xk + c) mod m.
java.util.Random: a = 25214903917, c = 11, m = 248. - The same seed gives the same sequence: runs with
new Random(42)are reproducible (debugging, experiments). - Use
SecureRandomfor cryptography andThreadLocalRandomin multithreaded code.
rng.nextInt(n)→ 0 … n − 1lo + rng.nextInt(hi − lo + 1)→ lo … hirng.nextDouble()→ [0, 1)rng.nextInt(6) + 1→ a die roll: 1 … 6
Summary
Summary
| Topic | Key points |
|---|---|
| Randomized algorithm | input + random bits; time/output are random variables; expectation over the algorithm's choices |
| Las Vegas | always correct, random time: QuickSort, QuickSelect, BogoSort |
| Monte Carlo | bounded time, small error probability; repetition drives it down to qk |
| Fisher–Yates | j = nextInt(i + 1), Θ(n), all n! outcomes equally likely; the naive shuffle is biased |
| BogoSort | p = 1/n!, expected Θ(n · n!), worst case unbounded |
| Randomized QuickSort | expected ≈ 1.39 n log₂ n comparisons, O(n log n) for every input; worst case O(n²) |
| Miller–Rabin | n − 1 = 2sd; error ≤ 4−k; "composite" is certain |
| Karger | random edge contraction; success ≥ 2/(n(n−1)); repeat many times |
| π estimation | 4 · inside / n; error ∝ 1/√n |
Summary
Common Mistakes
nextInt(n) for every i is biased (nn paths, n! permutations). Use nextInt(i + 1).int, x * x % n can overflow for n > 46,341. Use long, Math.multiplyHigh or BigInteger.modPow.Random; in Karger, pick the edge uniformly (choosing a vertex first and then a neighbor is not uniform).Summary
Sample Problems
- Fisher–Yates is applied to
[1, 2, 3, 4]; for i = 3, 2, 1 the drawn values of j are 1, 2, 0. What is the result? - For an array of 5 distinct elements, what is the probability that BogoSort finishes after the first shuffle, and what is the expected number of shuffles?
- In randomized QuickSort, with what probability are the smallest and largest elements compared? Two elements adjacent in sorted order?
- Miller–Rabin: run the test on n = 25 with bases a = 7 and a = 2.
- In a Monte Carlo run, 787 of 1000 points fell inside the quarter circle. What is the estimate of π?
Show answers
- i = 3, j = 1 → [1, 4, 3, 2]; i = 2, j = 2 → unchanged; i = 1, j = 0 → [4, 1, 3, 2].
- P = 1/5! = 1/120; expected number of shuffles 120 (geometric distribution).
- z1 and zn: 2/n (the first pivot must be one of the two). Neighbors zi, zi+1: 2/2 = 1, always compared.
- 24 = 23 · 3. a = 7: 73 ≡ 18, 18² ≡ 24 = n − 1 → passes (7 is a strong liar for 25). a = 2: 8, 14, 21 → 24 never appears → composite.
- π ≈ 4 · 787/1000 = 3.148.
Summary
Exercises
partitionis called on[3, 8, 2, 5, 1, 4, 7, 6]and drawsr = 5(pivot 4). What are the array after partitioning and the return value?- How many comparisons does QuickSort make on a sorted 8-element array when the first element is the pivot?
- How many Miller–Rabin rounds are needed to push the error probability below 10−6?
- On a 6-vertex graph (success ≥ 1/15), how many times must Karger be repeated to bring the failure probability below 1/6?
- Convert a Monte Carlo algorithm with success probability 1/3 and a verifiable result into a Las Vegas algorithm. What is the expected number of repetitions?
Show answers
- Pivot to the end: [3, 8, 2, 5, 1, 6, 7, 4]; the elements ≤ 4 (3, 2, 1) move left → [3, 2, 1, 4, 8, 6, 7, 5], returns 3.
- The part shrinks by one at each level: 7 + 6 + … + 1 = n(n − 1)/2 = 28.
- 4−k < 10−6 → k ≥ 10 (4−10 ≈ 9.5 · 10−7).
- (14/15)T ≤ e−T/15 ≤ 1/6 → T ≥ 15 ln 6 ≈ 26.9 → 27 repetitions suffice.
- Repeat until verification succeeds: the answer is always correct (Las Vegas); expected repetitions 1/(1/3) = 3.
Summary
References
- R. Motwani and P. Raghavan, Randomized Algorithms, Cambridge University Press, 1995.
- M. Mitzenmacher and E. Upfal, Probability and Computing, 2nd ed., Cambridge University Press, 2017.
- Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., 2022 (Chapters 5, 7, 9, 31).
- C. A. R. Hoare, "Quicksort", The Computer Journal 5(1), 1962.
- M. O. Rabin, "Probabilistic Algorithm for Testing Primality", Journal of Number Theory 12(1), 1980.
- D. R. Karger, "Global Min-cuts in RNC, and Other Ramifications of a Simple Min-Cut Algorithm", SODA, 1993.
- D. E. Knuth, The Art of Computer Programming, Vol. 2, 3rd ed., 1997 (Section 3.4.2: shuffling).