Chapter 7: Randomized Algorithms

Advanced Algorithms
Randomness and expected timeLas Vegas and Monte CarloFisher–Yates shuffleBogoSort Randomized QuickSortMonte Carlo estimate of πMiller–RabinKarger's min cut
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Concepts

Define a randomized algorithm and its expected running time; distinguish Las Vegas from Monte Carlo algorithms.

Shuffling and BogoSort

Write the Fisher–Yates shuffle, show why the naive shuffle is biased, and derive BogoSort's expected Θ(n · n!) time.

Randomized QuickSort

Trace random-pivot partitioning step by step; justify the expected O(n log n) and worst-case O(n²) bounds.

Monte Carlo

Compute the error probability of π estimation, the Miller–Rabin primality test and Karger's min-cut algorithm, and reduce it by repetition.

Introduction

Algorithms That Gamble

  • Some problem-solving techniques resemble games of chance.
  • The algorithm uses randomness when making decisions: it flips a coin, rolls a die, or picks a random element.
  • Example game: roll two dice; the player wins if the sum is 7 or 11.
P(7) = 6/36, P(11) = 2/36 → P(win) = 8/36 = 2/9 ≈ 0.22. Expected number of rolls until a win = 1 / (2/9) = 4.5.
= 7win: 7 or 11

Introduction

Randomized Algorithms

InputRandom bitsAlgorithm(decisions use chance)Output
  • A randomized algorithm uses random numbers in addition to its input.
  • Two runs on the same input may take different steps: the running time (and sometimes the output) can vary from run to run.
  • The running time is now a random variable, so we study the expected time E[T(n)]. The expectation is over the algorithm's own coin flips, not over the input.

Introduction

Why Randomness?

Protection against bad inputs

Deterministic QuickSort takes O(n²) on a sorted array. With a random pivot, no input is always bad: the worst case arises only from unlucky coin flips.

Simplicity

A randomized algorithm is often much shorter and clearer than the best deterministic one (Karger's min cut, randomized QuickSelect).

Speed

Miller–Rabin decides whether a number with hundreds of digits is prime in milliseconds, with a tiny error probability.

Sampling and symmetry breaking

Monte Carlo simulation, random sampling and leader election in distributed systems are hard without randomness.

The price: time or correctness is guaranteed only probabilistically, and we need a good random number generator.

Introduction

Probability Review

ConceptDefinition / ruleExample
Expected valueE[X] = Σ x · P(X = x)expected value of a die: 3.5
LinearityE[X + Y] = E[X] + E[Y] (no independence needed)sum of two dice: 7
Indicator variableXA = 1 (if A occurs), else 0 → E[XA] = P(A)QuickSort: "were i and j compared?"
Geometric distributiontrials with success probability p: expected trials until the first success 1/pBogoSort: p = 1/n! → n! shuffles
Independent repetitionk independent trials, each failing with probability p: all fail with pkMiller–Rabin: (1/4)k
Markov's inequalityif X ≥ 0, P(X ≥ c · E[X]) ≤ 1/cLas Vegas → Monte Carlo

Types of Algorithms

Las Vegas and Monte Carlo

Las VegasMonte Carlo
CorrectnessAlways correct resultWrong result with small probability
TimeRandom; we analyze the expected timeBounded (deterministic upper bound)
What is randomhow long it runswhether the answer is correct
ExamplesRandomized QuickSort, QuickSelect, BogoSort, Fisher–YatesMiller–Rabin, Karger's min cut, π estimation
One-sided error

When Miller–Rabin says "composite", it is certainly right; only the answer "prime" can be wrong.

Two-sided error

Numerical simulations such as π estimation can err in either direction; the error shrinks with the number of samples.

Types of Algorithms

Reducing the Error Probability

Amplification by repetition

If one round errs with probability at most q, then after k independent rounds the error is ≤ qk.

  • Miller–Rabin: q ≤ 1/4 → with k = 20 rounds, ≤ 4−20 ≈ 9 · 10−13.
  • Karger: success ≥ 2/(n(n−1)); with n(n−1)/2 · ln n repetitions, failure ≤ 1/n.
Converting between types
  • Las Vegas → Monte Carlo: stop the algorithm after 2 · E[T] steps. Markov: P(T ≥ 2 E[T]) ≤ 1/2 → "no answer" with probability at most 1/2.
  • Monte Carlo → Las Vegas: if an answer can be verified quickly (e.g., multiply the factors found), repeat until it checks out. Expected repetitions = 1/p.
110−110−210−310−410−510−610−71234567891011number of rounds k≈ 9.5 · 10⁻⁷

Miller–Rabin error bound 4−k (log scale): each round divides the bound by 4. k = 10 rounds suffice to get below 10−6.

Shuffling

Example: Shuffling

  • Randomness is used to shuffle the elements of a list: a deck of cards, random test data, a playlist.
  • Correctness: every element lands in every position with equal probability; more strongly, each of the n! permutations must have probability 1/n!.
  • Running time: Fisher–Yates always performs exactly n − 1 swaps: Θ(n). What varies is the result, not the time.
  • Built into Java: Collections.shuffle(list) (Fisher–Yates).
nn! permutations
36
424
103,628,800
52≈ 8.07 · 1067

Shuffling · Run the code step by step

Fisher–Yates (Knuth) Shuffle

1021324354

Start: a = [1, 2, 3, 4, 5]. Working from the end, each position receives a random element from those not yet placed.

1021324354i

i = 4: one of the 5 elements in a[0..4] will move to a[4].

1021j324354i

rng.nextInt(5) → j = 1 (probability 1/5).

1051324324

Swap a[4] ↔ a[1] → [1, 5, 3, 4, 2]. a[4] = 2 is now final (green).

10513243i24

i = 3: one of the 4 elements in a[0..3] will move to a[3].

10513243i = j24

rng.nextInt(4) → j = 3 (j = i: the element stays put).

1051324324

Swap with itself: the array is unchanged; a[3] = 4 is final.

105132i4324

i = 2: one of the 3 elements in a[0..2] will move to a[2].

10j5132i4324

rng.nextInt(3) → j = 0 (probability 1/3).

3051124324

Swap a[2] ↔ a[0] → [3, 5, 1, 4, 2]. a[2] = 1 is now final (green).

3051i124324

i = 1: one of the 2 elements in a[0..1] will move to a[1].

3051i = j124324

rng.nextInt(2) → j = 1 (j = i: the element stays put).

3051124324

Swap with itself: the array is unchanged; a[1] = 5 is final.

3051124324

Loop done: [3, 5, 1, 4, 2]. The number of possible draw sequences is 5 · 4 · 3 · 2 = 120 = 5!: each permutation arises exactly once → uniform distribution.

static void shuffle(int[] a) {
  for (int i = a.length - 1; i > 0; i--) {
    int j = rng.nextInt(i + 1); // 0 ≤ j ≤ i
    int t = a[i]; a[i] = a[j]; a[j] = t;
  }
}

Shuffling

The Naive Shuffle Is Biased

// BIASED: for each i, pick from 0..n-1
for (int i = 0; i < n; i++) {
  int j = rng.nextInt(n);
  swap(a, i, j);
}
  • Each of the n draws takes n values → nn equally likely paths.
  • n = 3: 33 = 27 paths, 3! = 6 permutations. 27 is not divisible by 6 → the permutations cannot be equally likely.
  • The fix for this common pattern is Fisher–Yates: nextInt(i + 1).
resultpathsprobabilityshould be
1234/270.1481/6 ≈ 0.167
1325/270.1851/6 ≈ 0.167
2135/270.1851/6 ≈ 0.167
2315/270.1851/6 ≈ 0.167
3124/270.1481/6 ≈ 0.167
3214/270.1481/6 ≈ 0.167

All 27 draw sequences for [1, 2, 3] were enumerated (Fisher–Yates has 3 · 2 = 6 paths, one per permutation).

BogoSort

BogoSort

  • One of the worst-performing sorting algorithms ("stupid sort", permutation sort).
  • Basic idea: shuffle randomly until the list is sorted.
  • A fun and instructive Las Vegas algorithm; never used in practice.
  1. Is the list sorted? Yes → stop.
  2. No → shuffle the list randomly and go back to step 1.
static void bogoSort(int[] a) {
  while (!isSorted(a))
    shuffle(a);         // Fisher–Yates
}
static boolean isSorted(int[] a) {
  for (int i = 0; i + 1 < a.length; i++)
    if (a[i] > a[i + 1]) return false;
  return true;
}
sorted?yesstopnoshufflep = 1/n! → expected n! rounds

BogoSort

BogoSort Example: [5, 2, 8, 4]

The list [5, 2, 8, 4]; each shuffle uses Fisher–Yates (a seeded run). Red: the first adjacent pair out of order.

50218243permutation matrixcolumn: position · row: rank

Start [5, 2, 8, 4]: a[0] = 5 > a[1] = 2 → not sorted, shuffling begins.

50412283permutation matrixcolumn: position · row: rank

Shuffle 1: [5, 4, 2, 8] → a[0] = 5 > a[1] = 4, not sorted.

80415223permutation matrixcolumn: position · row: rank

Shuffle 2: [8, 4, 5, 2] → a[0] = 8 > a[1] = 4, not sorted.

40218253permutation matrixcolumn: position · row: rank

Shuffle 3: [4, 2, 8, 5] → a[0] = 4 > a[1] = 2, not sorted.

20814253permutation matrixcolumn: position · row: rank

Shuffle 4: [2, 8, 4, 5] → a[1] = 8 > a[2] = 4, not sorted.

50812243permutation matrixcolumn: position · row: rank

Shuffle 5: [5, 8, 2, 4] → a[1] = 8 > a[2] = 2, not sorted.

80412253permutation matrixcolumn: position · row: rank

Shuffle 6: [8, 4, 2, 5] → a[0] = 8 > a[1] = 4, not sorted.

20815243permutation matrixcolumn: position · row: rank

Shuffle 7: [2, 8, 5, 4] → a[1] = 8 > a[2] = 5, not sorted.

20415283permutation matrixcolumn: position · row: rank

Shuffle 8: [2, 4, 5, 8] → sorted! The algorithm stops. This run needed 8 shuffles; the expected number is 4! = 24.

BogoSort

Analysis of BogoSort

  • With n distinct elements, a random shuffle is sorted with probability p = 1/n!.
  • Trials are independent → geometric distribution: the expected number of shuffles is 1/p = n!.
  • Each round: shuffle O(n) + check O(n) → expected time Θ(n · n!). This is the same order as O((n + 1)!), since (n + 1)! = (n + 1) · n!.
  • Best case: the array is already sorted → one check, O(n).
  • Worst case: unbounded. Unlucky shuffles can go on forever (with probability tending to 0). O((n + 1)!) bounds only the expected time, not the worst case.
nexpected shuffles n!time (10⁹ shuffles/s)
51200.12 µs
103,628,8003.6 ms
12479,001,6000.48 s
136,227,020,8006.2 s
151,307,674,368,000≈ 22 minutes
202,432,902,008,176,640,000≈ 77 years

Randomized QuickSort

Pivot Choice and Its Effect

Pivot strategyBad inputExpected / typicalWorst
First or last elementsorted / reverse-sorted array (very common!)O(n log n)*O(n²)
Median-of-3specially crafted "killer" arraysO(n log n)*O(n²)
Random elementnone: bad runs come only from bad luckO(n log n) for every inputO(n²)
True median (BFPRT)noneO(n log n)O(n log n)
  • * For deterministic strategies, O(n log n) holds only if the input is randomly ordered (average case). With a random pivot, the expectation is over the algorithm's own choices (expected time).
  • A good pivot improves the expected time but does not remove the worst case: O(n²) is still possible with a random pivot, just very unlikely.
  • BFPRT has large constants; in practice a random or median-of-3 pivot is preferred.

Randomized QuickSort

QuickSort Run with Random Pivots

[6, 5, 3, 1, 8, 7, 2, 4]; pivots chosen at random: 3, 1, 4, 6, 8. Black: pivot · red outline: pointers i and j · green: final · faded: outside the active subarray.

6051321384752647

Subarray a[0..7] = [6, 5, 3, 1, 8, 7, 2, 4]: the pivot is chosen at random → 3 (index 2, black box).

60i513213847526j47

i moves right past elements smaller than 3 and stops at a[0] = 6; j moves left past elements larger than 3 and stops at a[6] = 2. i < j → swap.

2051321384756647

Swap a[0] ↔ a[6] → [2, 5, 3, 1, 8, 7, 6, 4].

2051i3213j84756647

i moves right past elements smaller than 3 and stops at a[1] = 5; j moves left past elements larger than 3 and stops at a[3] = 1. i < j → swap.

2011325384756647

Swap a[1] ↔ a[3] → [2, 1, 3, 5, 8, 7, 6, 4].

2011325384756647

The pointers have crossed (i ≥ j): pivot 3 is in its final place (index 2). Only smaller elements lie to its left, only larger ones to its right.

2011325384756647

Subarray a[0..1] = [2, 1]: the pivot is chosen at random → 1 (index 1, black box).

20i11j325384756647

i moves right past elements smaller than 1 and stops at a[0] = 2; j moves left past elements larger than 1 and stops at a[1] = 1. i < j → swap.

1021325384756647

Swap a[0] ↔ a[1] → [1, 2, 3, 5, 8, 7, 6, 4]. The pivot itself may move; we track its position.

1021325384756647

The pointers have crossed (i ≥ j): pivot 1 is in its final place (index 0). Only smaller elements lie to its left, only larger ones to its right.

1021325384756647

The one-element subarray a[1] = 2 is already sorted: final.

1021325384756647

Subarray a[3..7] = [5, 8, 7, 6, 4]: the pivot is chosen at random → 4 (index 7, black box).

10213253i84756647j

i moves right past elements smaller than 4 and stops at a[3] = 5; j moves left past elements larger than 4 and stops at a[7] = 4. i < j → swap.

1021324384756657

Swap a[3] ↔ a[7] → [1, 2, 3, 4, 8, 7, 6, 5]. The pivot itself may move; we track its position.

1021324384756657

The pointers have crossed (i ≥ j): pivot 4 is in its final place (index 3). Only smaller elements lie to its left, only larger ones to its right.

1021324384756657

Subarray a[4..7] = [8, 7, 6, 5]: the pivot is chosen at random → 6 (index 6, black box).

1021324384i756657j

i moves right past elements smaller than 6 and stops at a[4] = 8; j moves left past elements larger than 6 and stops at a[7] = 5. i < j → swap.

1021324354756687

Swap a[4] ↔ a[7] → [1, 2, 3, 4, 5, 7, 6, 8].

102132435475i66j87

i moves right past elements smaller than 6 and stops at a[5] = 7; j moves left past elements larger than 6 and stops at a[6] = 6. i < j → swap.

1021324354657687

Swap a[5] ↔ a[6] → [1, 2, 3, 4, 5, 6, 7, 8]. The pivot itself may move; we track its position.

1021324354657687

The pointers have crossed (i ≥ j): pivot 6 is in its final place (index 5). Only smaller elements lie to its left, only larger ones to its right.

1021324354657687

The one-element subarray a[4] = 5 is already sorted: final.

1021324354657687

Subarray a[6..7] = [7, 8]: the pivot is chosen at random → 8 (index 7, black box).

1021324354657687

Pivot 8: i skips 7 and stops at 8; j is already at 8. i ≥ j → no swap, 8 is in its final place (index 7).

1021324354657687

The one-element subarray a[6] = 7 is already sorted: final. The array is sorted: [1, 2, 3, 4, 5, 6, 7, 8].

Randomized QuickSort

Randomized QuickSort: Code

static Random rng = new Random();

static void quickSort(int[] a, int lo, int hi) {
  if (lo < hi) {
    int p = partition(a, lo, hi); // random
    quickSort(a, lo, p - 1);
    quickSort(a, p + 1, hi);
  }
}

static int partition(int[] a, int lo, int hi) {
  int r = lo + rng.nextInt(hi - lo + 1);
  swap(a, r, hi);          // pivot to end
  int pivot = a[hi], k = lo;
  for (int i = lo; i < hi; i++)
    if (a[i] <= pivot)
      swap(a, i, k++);     // smaller left
  swap(a, k, hi);          // pivot in place
  return k;
}
  • The pivot is uniformly random. The output is always correct (Las Vegas); only the running time depends on the pivots.
  • partition moves the random pivot to the end, then runs Lomuto: for [6, 5, 3, 1, 8, 7, 2, 4], r = 2 (pivot 3) → [1, 2, 3, 6, 8, 7, 5, 4], returns 2.
  • Extra space: the recursion stack, expected depth O(log n); recursing on the smaller part first makes it O(log n) in the worst case too.
  • With many duplicate keys, use three-way partitioning (<, =, >).

Randomized QuickSort

Analysis of QuickSort

Best: balanced splitnnnnlog₂ n levels × n work = O(n log n)Worst: always the smallest/largest pivotnn−1n−2…1n levels: n + (n−1) + … + 1 = O(n²)
CaseRecurrenceTime
BestT(n) = 2T(n/2) + Θ(n)Θ(n log n)
Expected (random pivot)T(n) = (1/n) Σq [T(q) + T(n−1−q)] + Θ(n)Θ(n log n)
WorstT(n) = T(n−1) + Θ(n)Θ(n²)

Probability of the worst case with a random pivot: the smallest or largest element must be chosen at every step → 2n−1/n! (≈ 2 · 10−13 for n = 20).

Randomized QuickSort

Expected Number of Comparisons

  • Label the elements in sorted order: z1 < z2 < … < zn. Indicator Xij = 1 ⇔ zi and zj are compared.
  • Two elements are compared only if one is the pivot. If the first pivot from {zi, …, zj} lies strictly between them, they are split apart and never compared.
  • That first pivot is uniformly random → P(Xij = 1) = 2 / (j − i + 1).
  • Linearity: E[X] = Σi<j 2/(j − i + 1) < 2n Hn ≈ 1.39 n log₂ n.
  • Exact value: E[X] = 2(n + 1)Hn − 4n.
nrandom pivot (expected)sorted input, first-element pivot
1024.445
1006484,950
1,00010,986499,500
1062.48 · 1075.0 · 1011
z1z2z3z4z5z6z7z8first pivot{z₂, …, z₆}: 5 elementsPivot z₄ → z₂ goes left, z₆ goes right: they never meet.Only if z₂ or z₆ is the first pivot: P = 2/5.

Randomized QuickSort

Randomized Selection (QuickSelect)

static int select(int[] a, int lo, int hi, int s) {
  if (lo == hi) return a[lo];
  int p = partition(a, lo, hi);
  if (s == p) return a[p];
  if (s < p) return select(a, lo, p - 1, s);
  return select(a, p + 1, hi, s);
}
  • Goal: find the s-th smallest element (e.g., the median) without sorting.
  • It uses the same random partitioning but recurses on one side only.
  • Expected time O(n) (≈ 3.39 n comparisons for the median); worst case O(n²).
  • Example: [6, 5, 3, 1, 8, 7, 2, 4] with s = 2 (0-based). Partitioning around pivot 3 gives p = 2 → the answer 3 on the first try.

Randomized QuickSort · Run the code step by step

QuickSelect: Code Trace

6051321384752647sactive range [0, 7]

Array [6, 5, 3, 1, 8, 7, 2, 4]; target s = 5 (6th smallest). The first call covers the whole array: lo = 0, hi = 7.

1021326384755647sactive range [0, 7]

partition chose random pivot 3; after partitioning its final position is p = 2: left ≤ 3, right > 3.

1021326384755647sactive range [0, 7]

s = 5 > p = 2 → the target is on the right: recurse only on [3, 7]; the left side is never examined again.

1021326384755647sactive range [3, 7]

New call select(a, 3, 7, 5): 5 elements in range, lo ≠ hi.

1021326344557687sactive range [3, 7]

partition chose random pivot 7; after partitioning its final position is p = 6: left ≤ 7, right > 7.

1021326344557687sactive range [3, 7]

s = 5 < p = 6 → the target is on the left: recurse only on [3, 5].

1021326344557687sactive range [3, 5]

New call select(a, 3, 5, 5): 3 elements in range, lo ≠ hi.

1021324354657687sactive range [3, 5]

partition chose random pivot 5; after partitioning its final position is p = 4: left ≤ 5, right > 5.

1021324354657687sactive range [3, 5]

s = 5 > p = 4 → the target is on the right: recurse only on [5, 5]; the left side is never examined again.

1021324354657687sactive range [5, 5]

New call select(a, 5, 5, 5): 1 element in range, lo = hi.

1021324354657687sactive range [5, 5]

lo = hi = 5: one element left → return a[5] = 6.

1021324354657687s

select(a, 0, 7, 5) = 6: position 5 (0-based) of the sorted array. The array was never fully sorted; only the needed parts were partitioned.

static int select(int[] a, int lo, int hi, int s) {
  if (lo == hi) return a[lo];
  int p = partition(a, lo, hi);
  if (s == p) return a[p];
  if (s < p) return select(a, lo, p - 1, s);
  return select(a, p + 1, hi, s);
}

Monte Carlo Algorithms

Monte Carlo Estimate of π

  • Throw uniformly random points into the unit square. The quarter circle has area π/4 and the square area 1 → P(inside) = π/4, so π ≈ 4 · inside / n.
  • The error falls like 1/√n: one more correct digit needs 100 times as many points.
static double estimatePi(int n) {
  int inside = 0;
  for (int t = 0; t < n; t++) {
    double x = rng.nextDouble(), y = rng.nextDouble();
    if (x * x + y * y <= 1) inside++;
  }
  return 4.0 * inside / n;
}

300 points: 239 inside → π ≈ 4 · 239/300 = 3.187.

Monte Carlo Algorithms · Run the code step by step

Monte Carlo π: Code Trace

01points: 0inside: 0

n = 8 points will be thrown; inside counts those landing inside the quarter circle. (A seeded Random makes the run reproducible.)

01points: 1inside: 1x² + y² = 0.42≤ 1 → inside

t = 0: (0.62, 0.19), x² + y² = 0.42 ≤ 1 → inside. inside = 1.

01points: 2inside: 2x² + y² = 0.88≤ 1 → inside

t = 1: (0.93, 0.09), x² + y² = 0.88 ≤ 1 → inside. inside = 2.

01points: 3inside: 3x² + y² = 0.58≤ 1 → inside

t = 2: (0.58, 0.49), x² + y² = 0.58 ≤ 1 → inside. inside = 3.

01points: 4inside: 4x² + y² = 0.24≤ 1 → inside

t = 3: (0.00, 0.49), x² + y² = 0.24 ≤ 1 → inside. inside = 4.

01points: 5inside: 5x² + y² = 0.59≤ 1 → inside

t = 4: (0.30, 0.71), x² + y² = 0.59 ≤ 1 → inside. inside = 5.

01points: 6inside: 5x² + y² = 1.13> 1 → outside

t = 5: (0.78, 0.72), x² + y² = 1.13 > 1 → outside. inside = 5.

01points: 7inside: 5x² + y² = 1.08> 1 → outside

t = 6: (0.35, 0.98), x² + y² = 1.08 > 1 → outside. inside = 5.

01points: 8inside: 6x² + y² = 0.38≤ 1 → inside

t = 7: (0.50, 0.36), x² + y² = 0.38 ≤ 1 → inside. inside = 6.

01points: 8inside: 6π ≈ 4 · 6 / 8= 3.0

4.0 * 6 / 8 = 3.0. With 8 points the error is large (≈ 1.64/√8 ≈ 0.58); the estimate approaches π as n grows.

static double estimatePi(int n) {
  int inside = 0;
  for (int t = 0; t < n; t++) {
    double x = rng.nextDouble(), y = rng.nextDouble();
    if (x * x + y * y <= 1) inside++;
  }
  return 4.0 * inside / n;
}

Monte Carlo Algorithms

Try It: Monte Carlo π

number of points n0
inside0
π estimate—
error |π̂ − π|—
standard deviation ≈ 1.64/√n—

Green: inside the quarter circle · red: outside. The plot shows the first 3000 points; counting continues over all points. 1.64 = 4 · √(p(1 − p)), p = π/4.

Monte Carlo Algorithms

Miller–Rabin Primality Test

  • Fermat's little theorem: if n is prime and 1 < a < n, then an−1 ≡ 1 (mod n). If this fails, n is certainly composite; but some composites (Carmichael numbers) satisfy it for every base.
  • Square-root property: if n is prime, x² ≡ 1 (mod n) holds only for x ≡ ±1.
  • Write n − 1 = 2s · d (d odd); for a random a, examine the sequence ad, a2d, …, a2s−1d (mod n).
  • If the sequence does not start with 1 and n − 1 never appears, a is a witness that n is composite. Otherwise n "passes" this round.
  • For composite n, at most 1/4 of the bases are liars → with k independent rounds the error is ≤ 4−k. Each round takes O(log n) modular multiplications: O(k log³ n) bit operations in total.
ad²a2d²a4d²…²a2^(s−1)·dan−1→passes: the sequence starts with 1 or n − 1 appears somewherewitness: first term ≠ 1 and n − 1 never appears → n certainly composite
One-sided error: the answer "composite" is certain; the answer "prime" is only probably right.

Monte Carlo Algorithms

Miller–Rabin: Code

static boolean isPrime(long n, int k) {
  if (n < 4) return n == 2 || n == 3;
  if (n % 2 == 0) return false;
  long d = n - 1; int s = 0;     // n-1 = 2^s·d
  while (d % 2 == 0) { d /= 2; s++; }
  for (int round = 0; round < k; round++) {
    long a = 2 + rng.nextLong(n - 3); // [2, n-2]
    long x = modPow(a, d, n);    // a^d mod n
    if (x == 1 || x == n - 1) continue;
    boolean witness = true;
    for (int r = 1; r < s && witness; r++) {
      x = x * x % n;
      if (x == n - 1) witness = false;
    }
    if (witness) return false;   // certainly composite
  }
  return true;                  // probably prime
}
  • modPow: fast modular exponentiation (square-and-multiply), O(log d) multiplications.
  • The base a is drawn uniformly from [2, n − 2].
  • With long, x * x does not overflow only for n < 3 · 109; for larger n use BigInteger.modPow or isProbablePrime.

Monte Carlo Algorithms · Run the code step by step

Miller–Rabin: Code Trace (n = 561)

n − 1 = 560 = 20 · 560a^d→a^2d→a^4d→a^8d

n = 561 = 3 · 11 · 17 is a Carmichael number: it passes the Fermat test for every base. n ≥ 4 and odd → the early checks do not return.

n − 1 = 560 = 20 · 560a^d→a^2d→a^4d→a^8d

d = n − 1 = 560, s = 0. Now factor the 2s out of 560.

n − 1 = 560 = 21 · 280a^d→a^2d→a^4d→a^8d

d is even → d /= 2, s++: d = 280, s = 1.

n − 1 = 560 = 22 · 140a^d→a^2d→a^4d→a^8d

d is even → d /= 2, s++: d = 140, s = 2.

n − 1 = 560 = 23 · 70a^d→a^2d→a^4d→a^8d

d is even → d /= 2, s++: d = 70, s = 3.

n − 1 = 560 = 24 · 35a^d→a^2d→a^4d→a^8d

d is even → d /= 2, s++: d = 35, s = 4. d is odd: 560 = 2⁴ · 35.

n − 1 = 560 = 24 · 35a = 2a^d→a^2d→a^4d→a^8d

Round 0: random base drawn, a = 2.

n − 1 = 560 = 24 · 35a = 2263a^d→a^2d→a^4d→a^8d

x = 2³⁵ mod 561 = 263 (by fast exponentiation).

n − 1 = 560 = 24 · 35a = 2263a^d→a^2d→a^4d→a^8d

x = 263 is neither 1 nor n − 1 = 560 → the round is not over; with witness = true we check the squares.

n − 1 = 560 = 24 · 35a = 2263a^d→166a^2d→a^4d→a^8d

r = 1: x = 263² mod 561 = 166. Not 560.

n − 1 = 560 = 24 · 35a = 2263a^d→166a^2d→67a^4d→a^8d

r = 2: x = 166² mod 561 = 67. Not 560.

n − 1 = 560 = 24 · 35a = 2263a^d→166a^2d→67a^4d→1a^8d

r = 3: x = 67² mod 561 = 1. Not 560. We reached 1 without seeing n − 1.

n − 1 = 560 = 24 · 35a = 2263a^d→166a^2d→67a^4d→1a^8d67² ≡ 1 but 67 ≠ ±1 → a = 2 is a witness: 561 is composite

The sequence never reached 560: witness stayed true → return false. 561 is certainly composite; 67 is a square root of 1 other than ±1.

static boolean isPrime(long n, int k) {
  if (n < 4) return n == 2 || n == 3;
  if (n % 2 == 0) return false;
  long d = n - 1; int s = 0;     // n-1 = 2^s·d
  while (d % 2 == 0) { d /= 2; s++; }
  for (int round = 0; round < k; round++) {
    long a = 2 + rng.nextLong(n - 3); // [2, n-2]
    long x = modPow(a, d, n);    // a^d mod n
    if (x == 1 || x == n - 1) continue;
    boolean witness = true;
    for (int r = 1; r < s && witness; r++) {
      x = x * x % n;
      if (x == n - 1) witness = false;
    }
    if (witness) return false;   // certainly composite
  }
  return true;                  // probably prime
}

Monte Carlo Algorithms · Run the code step by step

modPow: 2³⁵ mod 561 Step by Step

d = 35 = 100011₂13201608041211result = 1a = 2a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

Compute 2³⁵ mod 561. Since 35 = 32 + 2 + 1, 2³⁵ = 2³² · 2² · 2¹. Initially result = 1, a = 2.

d = 35 = 100011₂13201608041211result = 2a = 4× a: 1 · 2 mod 561square: 2² mod 561a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

Bit 0 = 1: result = 1 · 2 mod 561 = 2 (2¹ included). Then a = 2² mod 561 = 4.

d = 35 = 100011₂13201608041211result = 8a = 16× a: 2 · 4 mod 561square: 4² mod 561a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

Bit 1 = 1: result = 2 · 4 mod 561 = 8 (2² included). Then a = 4² mod 561 = 16.

d = 35 = 100011₂13201608041211result = 8a = 256bit 0: no multiplysquare: 16² mod 561a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

Bit 2 = 0: result unchanged (8). Only square: a = 16² mod 561 = 256.

d = 35 = 100011₂13201608041211result = 8a = 460bit 0: no multiplysquare: 256² mod 561a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

Bit 3 = 0: result unchanged (8). Only square: a = 256² mod 561 = 460.

d = 35 = 100011₂13201608041211result = 8a = 103bit 0: no multiplysquare: 460² mod 561a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

Bit 4 = 0: result unchanged (8). Only square: a = 460² mod 561 = 103.

d = 35 = 100011₂13201608041211result = 263a = 511× a: 8 · 103 mod 561square: 103² mod 561a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

Bit 5 = 1: result = 8 · 103 mod 561 = 263 (2³² included). Then a = 103² mod 561 = 511.

d = 35 = 100011₂13201608041211result = 263a = 511a becomes 2^1, 2^2, 2^4, 2^8 … (mod 561) in turn

d = 0: loop done. return 263 = 2³⁵ mod 561, using only 6 squarings + 3 multiplications instead of 34.

static long modPow(long a, long d, long n) {
  long result = 1;
  a %= n;
  while (d > 0) {
    if ((d & 1) == 1)          // bit is 1
      result = result * a % n;
    a = a * a % n;             // square
    d >>= 1;                   // next bit
  }
  return result;
}

Monte Carlo Algorithms

Miller–Rabin Examples

nn − 1aad, a2d, … (mod n)result
6122 · 15211, 60passes (may be prime)
22122 · 5517447, 220passes (may be prime)
22122 · 55137188, 205composite (a is a witness)
56124 · 352263, 166, 67, 1composite (a is a witness)
  • 61 is prime: the sequence reaches n − 1 = 60.
  • 221 = 13 · 17: a = 174 is a strong liar (the sequence reaches 220), while a = 137 is a witness. That is why we need several rounds.
  • 561 = 3 · 11 · 17 is a Carmichael number: 2560 ≡ 1, so the Fermat test is fooled. Miller–Rabin sees that 67² ≡ 1: 67 is a square root of 1 other than ±1 → composite.

Monte Carlo Algorithms

Karger's Min-Cut Algorithm

012345

Start: 6 vertices, 8 edges. The minimum cut is 2 (e.g., {3, 4} | the rest). Dark vertex: the supervertex formed by the last contraction.

×20,21345

Random edge (0, 2) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 7 edges.

×30,21,534

Random edge (1, 5) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 6 edges.

×30,21,53,4

Random edge (3, 4) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 5 edges.

×20,1,2,53,4

Random edge (0, 1) contracted: its endpoints merge into one supervertex, edges between them (self-loops) are deleted, parallel edges (×k) are kept. Remaining: 2 edges. Two supervertices left → cut found = 2 (minimum; a successful run).

  • Problem: find a cut with the fewest edges that splits the graph into two parts (global min cut).
  • Algorithm: contract a uniformly random edge until 2 vertices remain; the remaining edges form a cut.
  • One run finds a minimum cut with probability at least 2/(n(n−1)) (n = 6: ≥ 1/15). On this graph, 200,000 simulations gave ≈ 0.46.
  • The output can be wrong (a larger cut) → Monte Carlo. E.g., if the first edges chosen are (1,2), (4,5), (3,4), (0,1), we get the cut {0,1,2} | {3,4,5} with 3 edges.
  • n(n−1)/2 · ln n repetitions → failure ≤ 1/n; O(n4 log n) in total (simple implementation).

Monte Carlo Algorithms

Other Applications

Graph coloringTries random colorings, searching for one in which no two neighbors share a color.
Randomized DFSVisits neighbors in random order to find connectivity and spanning trees (e.g., maze generation).
MatchingRandomized incremental algorithms add edges in random order to approach a maximum matching.
Approximate countingCount-Min Sketch uses random hash functions to estimate frequencies in large data streams with controlled error.
Matrix computationsRandomized matrix approximation yields low-rank approximations: dimensionality reduction, data compression.
Computational geometryRandomized incremental convex hull, closest pair, Delaunay triangulation: expected O(n log n).

Randomized scheduling (job assignment) and metaheuristics such as simulated annealing and genetic algorithms also use randomness to search; metaheuristics do not guarantee convergence to the optimum.

Monte Carlo Algorithms

Pseudorandom Numbers

  • Computers produce pseudorandom numbers rather than true randomness: a deterministic sequence that looks random.
  • Linear congruential generator (LCG): Xk+1 = (a · Xk + c) mod m. java.util.Random: a = 25214903917, c = 11, m = 248.
  • The same seed gives the same sequence: runs with new Random(42) are reproducible (debugging, experiments).
  • Use SecureRandom for cryptography and ThreadLocalRandom in multithreaded code.
Random ranges
  • rng.nextInt(n) → 0 … n − 1
  • lo + rng.nextInt(hi − lo + 1) → lo … hi
  • rng.nextDouble() → [0, 1)
  • rng.nextInt(6) + 1 → a die roll: 1 … 6

Summary

Summary

TopicKey points
Randomized algorithminput + random bits; time/output are random variables; expectation over the algorithm's choices
Las Vegasalways correct, random time: QuickSort, QuickSelect, BogoSort
Monte Carlobounded time, small error probability; repetition drives it down to qk
Fisher–Yatesj = nextInt(i + 1), Θ(n), all n! outcomes equally likely; the naive shuffle is biased
BogoSortp = 1/n!, expected Θ(n · n!), worst case unbounded
Randomized QuickSortexpected ≈ 1.39 n log₂ n comparisons, O(n log n) for every input; worst case O(n²)
Miller–Rabinn − 1 = 2sd; error ≤ 4−k; "composite" is certain
Kargerrandom edge contraction; success ≥ 2/(n(n−1)); repeat many times
π estimation4 · inside / n; error ∝ 1/√n

Summary

Common Mistakes

Naive shuffleSwapping with nextInt(n) for every i is biased (nn paths, n! permutations). Use nextInt(i + 1).
"A random pivot eliminates the worst case"No: the worst case is still O(n²). What changes is that no input is always bad, and the worst case becomes very unlikely.
Confusing average and expected timeThe average case assumes an input distribution; expected time is computed for each input over the algorithm's coin flips.
Treating a Monte Carlo answer as certainIn Miller–Rabin, "prime" means only probably prime; the number of rounds k sets the error bound.
OverflowWith int, x * x % n can overflow for n > 46,341. Use long, Math.multiplyHigh or BigInteger.modPow.
new Random() on every callCreates needless objects, and the seeds may be correlated. Share a single Random; in Karger, pick the edge uniformly (choosing a vertex first and then a neighbor is not uniform).

Summary

Sample Problems

  1. Fisher–Yates is applied to [1, 2, 3, 4]; for i = 3, 2, 1 the drawn values of j are 1, 2, 0. What is the result?
  2. For an array of 5 distinct elements, what is the probability that BogoSort finishes after the first shuffle, and what is the expected number of shuffles?
  3. In randomized QuickSort, with what probability are the smallest and largest elements compared? Two elements adjacent in sorted order?
  4. Miller–Rabin: run the test on n = 25 with bases a = 7 and a = 2.
  5. In a Monte Carlo run, 787 of 1000 points fell inside the quarter circle. What is the estimate of π?
Show answers
  1. i = 3, j = 1 → [1, 4, 3, 2]; i = 2, j = 2 → unchanged; i = 1, j = 0 → [4, 1, 3, 2].
  2. P = 1/5! = 1/120; expected number of shuffles 120 (geometric distribution).
  3. z1 and zn: 2/n (the first pivot must be one of the two). Neighbors zi, zi+1: 2/2 = 1, always compared.
  4. 24 = 23 · 3. a = 7: 73 ≡ 18, 18² ≡ 24 = n − 1 → passes (7 is a strong liar for 25). a = 2: 8, 14, 21 → 24 never appears → composite.
  5. π ≈ 4 · 787/1000 = 3.148.

Summary

Exercises

  1. partition is called on [3, 8, 2, 5, 1, 4, 7, 6] and draws r = 5 (pivot 4). What are the array after partitioning and the return value?
  2. How many comparisons does QuickSort make on a sorted 8-element array when the first element is the pivot?
  3. How many Miller–Rabin rounds are needed to push the error probability below 10−6?
  4. On a 6-vertex graph (success ≥ 1/15), how many times must Karger be repeated to bring the failure probability below 1/6?
  5. Convert a Monte Carlo algorithm with success probability 1/3 and a verifiable result into a Las Vegas algorithm. What is the expected number of repetitions?
Show answers
  1. Pivot to the end: [3, 8, 2, 5, 1, 6, 7, 4]; the elements ≤ 4 (3, 2, 1) move left → [3, 2, 1, 4, 8, 6, 7, 5], returns 3.
  2. The part shrinks by one at each level: 7 + 6 + … + 1 = n(n − 1)/2 = 28.
  3. 4−k < 10−6 → k ≥ 10 (4−10 ≈ 9.5 · 10−7).
  4. (14/15)T ≤ e−T/15 ≤ 1/6 → T ≥ 15 ln 6 ≈ 26.9 → 27 repetitions suffice.
  5. Repeat until verification succeeds: the answer is always correct (Las Vegas); expected repetitions 1/(1/3) = 3.

Summary

References

  • R. Motwani and P. Raghavan, Randomized Algorithms, Cambridge University Press, 1995.
  • M. Mitzenmacher and E. Upfal, Probability and Computing, 2nd ed., Cambridge University Press, 2017.
  • Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., 2022 (Chapters 5, 7, 9, 31).
  • C. A. R. Hoare, "Quicksort", The Computer Journal 5(1), 1962.
  • M. O. Rabin, "Probabilistic Algorithm for Testing Primality", Journal of Number Theory 12(1), 1980.
  • D. R. Karger, "Global Min-cuts in RNC, and Other Ramifications of a Simple Min-Cut Algorithm", SODA, 1993.
  • D. E. Knuth, The Art of Computer Programming, Vol. 2, 3rd ed., 1997 (Section 3.4.2: shuffling).

The End

Chapter 7: Randomized Algorithms
Next chapter: Chapter 8: Online Algorithms
1 / 1 Sercan KÜLCÜ. All rights reserved.