Chapter 7: Randomized Algorithms

Chapter 7: Randomized Algorithms · Study Questions
Las Vegas and Monte CarloFisher–Yates shuffleBogoSortRandomized QuickSortQuickSelectMonte Carlo πMiller–RabinKarger min-cut
Sercan KÜLCÜ · Giresun University · Computer Engineering

Multiple Choice · Basic Concepts

Question 1: Las Vegas or Monte Carlo?

Which of the following pairings is correct?

A: A Las Vegas algorithm always gives the correct result, with a random running time (QuickSort, QuickSelect, BogoSort); Fisher–Yates, whose time is a fixed Θ(n), belongs to this group too because it never gives a wrong result. A Monte Carlo algorithm has a bounded running time, but its result may be wrong with a small probability (Miller–Rabin, Karger, the π estimate).

Multiple Choice · Basic Concepts

Question 2: Expected Rolls Until a Win

Two dice are rolled; the player wins if the total is 6 or 8, otherwise rolls again. What is the expected number of rolls until a win?

36 equally likely outcomes; orange: total 6 or 8 (10 of them)

C: P(6) = 5/36, P(8) = 5/36 → p = 10/36. The trials are independent: geometric distribution, expected number of trials 1/p = 36/10 = 3.6. 0.28 is the success probability itself; 7.20 only counts a single total (5/36).

Step-by-Step Trace · Shuffling

Question 3: Tracing Fisher–Yates

shuffle is applied to the array [10, 20, 30, 40, 50]; the random number generator returns j = 2, 3, 0, 1 in turn. How does the array change at each step?

100201302403504

a = [10, 20, 30, 40, 50]. Going from the end, each position receives one of the elements not yet placed. In this trace nextInt returns 2, 3, 0, 1 in turn.

100201302j403504i

i = 4: dice.nextInt(5) → j = 2; each of the 5 elements in a[0..4] could have been chosen with probability 1/5.

100201502403304

a[4] ↔ a[2] are swapped → [10, 20, 50, 40, 30]. a[4] = 30 is now final.

100201502403i = j304

i = 3: dice.nextInt(4) → j = 3; each of the 4 elements in a[0..3] could have been chosen with probability 1/4.

100201502403304

j = i: the element is swapped with itself and the array does not change; a[3] = 40 is now final.

100j201502i403304

i = 2: dice.nextInt(3) → j = 0; each of the 3 elements in a[0..2] could have been chosen with probability 1/3.

500201102403304

a[2] ↔ a[0] are swapped → [50, 20, 10, 40, 30]. a[2] = 10 is now final.

500201i = j102403304

i = 1: dice.nextInt(2) → j = 1; each of the 2 elements in a[0..1] could have been chosen with probability 1/2.

500201102403304

j = i: the element is swapped with itself and the array does not change; a[1] = 20 is now final.

500201102403304

The loop ends at i = 0: [50, 20, 10, 40, 30]. The number of possible draw sequences is 5 · 4 · 3 · 2 = 120 = 5!: every permutation occurs exactly once → uniform distribution. The number of swaps is always n − 1: Θ(n).

static void shuffle(int[] a) {
  for (int i = a.length - 1; i > 0; i--) {
    int j = dice.nextInt(i + 1);  // 0 ≤ j ≤ i
    int t = a[i]; a[i] = a[j]; a[j] = t;
  }
}

Multiple Choice · Shuffling

Question 4: Why Is the Naive Shuffle Biased?

A naive shuffle that, for each i, swaps with j = dice.nextInt(n) is applied to n = 4 elements. How many equally likely draw paths are there, and why is the result biased?

D: Each of the 4 draws takes 4 values → 4⁴ = 256 equally likely paths; 4! = 24 permutations. 256/24 is not an integer, so some permutations occur more often. Fisher–Yates has 4 · 3 · 2 · 1 = 24 paths, each permutation exactly once.

Output Question · Shuffling

Question 5: nextInt(i) Instead of nextInt(i + 1)

shuffle method has a typo: instead of dice.nextInt(i + 1) it uses dice.nextInt(i). For the array [1, 2, 3], which results occur with what probabilities? Is this shuffle uniform?

Show answer

At i = 2, j ∈ {0, 1}; at i = 1, j = 0 is forced: only 2 · 1 = 2 paths.

1230132021302311/23121/23210
  • j = 0: [3, 2, 1] → then a[1] ↔ a[0] → [2, 3, 1]
  • j = 1: [1, 3, 2] → then → [3, 1, 2]

Each has probability 1/2; the other 4 permutations (including the sorted [1, 2, 3]) never occur. a[i] is always swapped with an element further to the left, so no element can stay in place: this is Sattolo's algorithm, and it produces only the (n − 1)! cyclic permutations. It is not a uniform shuffle.

Multiple Choice · BogoSort

Question 6: Expected Shuffles in BogoSort

An array of 6 distinct elements is sorted with BogoSort (shuffling: Fisher–Yates). What is the expected number of shuffles?

C: One shuffle comes out sorted with probability p = 1/6! = 1/720; the trials are independent → geometric distribution, expected 1/p = 720. Since each round is Θ(n), the expected time is Θ(n · n!). 120 = 5!, 5040 = 7!.

Classic Question · BogoSort

Question 7: The Worst Case of BogoSort

State the best-case, expected and worst-case running times of BogoSort, with reasons. Why is the expected Θ(n · n!) time of the same magnitude as O((n + 1)!), and why can no upper bound be given for the worst case?

Show answer
  • Best: the array is already sorted → a single isSorted call, O(n).
  • Expected: p = 1/n!, geometric distribution → n! shuffles; each round costs a shuffle O(n) + a check O(n) → Θ(n · n!). Since (n + 1)! = (n + 1) · n!, it is of the same order as n · n!.
  • Worst: the shuffles are independent; the probability of coming out unsorted for k rounds in a row is (1 − 1/n!)k > 0. For every k a run that long is possible → unbounded (even though its probability tends to 0).

That is why BogoSort is a Las Vegas algorithm: the result is always correct, the time is random.

Step-by-Step Trace · Randomized QuickSort

Question 8: Tracing Randomized Partition

partition(a, 0, 6) call runs on a = {8, 3, 6, 1, 7, 2, 5} and dice.nextInt(7) returns 2. How do the array, i and k change at each step?

803162r13742556

dice.nextInt(7) → r = 2: pivot a[2] = 6 (orange).

80315213742566

The pivot moves to the end: a[2] ↔ a[6] → [8, 3, 5, 1, 7, 2, 6].

80i,k315213742566

pivot = 6, k = 0: k is where the "≤ pivot" region (green) ends.

80k31i5213742566

i = 0: a[0] = 8 > 6 → stays in the "> pivot" region (blue).

3081k52i13742566

i = 1: a[1] ≤ 6 → swap(a, 1, 0), k = 1 → [3, 8, 5, 1, 7, 2, 6].

305182k13i742566

i = 2: a[2] ≤ 6 → swap(a, 2, 1), k = 2 → [3, 5, 8, 1, 7, 2, 6].

30511283k74i2566

i = 3: a[3] ≤ 6 → swap(a, 3, 2), k = 3 → [3, 5, 1, 8, 7, 2, 6].

30511283k7425i66

i = 4: a[4] = 7 > 6 → stays in the "> pivot" region (blue).

3051122374k8566i

i = 5: a[5] ≤ 6 → swap(a, 5, 3), k = 4 → [3, 5, 1, 2, 7, 8, 6].

30511223648576

The pivot settles at k = 4 → [3, 5, 1, 2, 6, 8, 7], return value 4. To its left there are only values ≤ 6, to its right only values > 6; high − low = 6 comparisons were made.

static int partition(int[] a, int low, int high) {
  int r = low + dice.nextInt(high - low + 1);
  swap(a, r, high);  // pivot to the end
  int pivot = a[high], k = low;
  for (int i = low; i < high; i++)
    if (a[i] <= pivot)
      swap(a, i, k++);  // smaller ones to the left
  swap(a, k, high);  // pivot into place
  return k;
}

Output Question · Randomized QuickSort

Question 9: Comparisons on a Sorted Array

A QuickSort that always picks the first element of the subarray as pivot sorts an already sorted array of 10 elements. How many comparisons are made? For the randomized-pivot version on the same array, what is the expected number of comparisons E[X] = 2(n + 1)Hn − 4n?

Show answer

The first element is always the smallest: one part is empty, the other is one element shorter. The comparisons are 9 + 8 + … + 1 = n(n − 1)/2 = 45.

Random pivot: H10 = 1 + 1/2 + … + 1/10 ≈ 2.929 → 2 · 11 · 2.929 − 40 ≈ 24.4. This value does not depend on the input: a sorted array is not "bad" for a random pivot.

For n = 1000 the difference becomes huge: 499,500 versus ≈ 10,986.

Multiple Choice · Randomized QuickSort

Question 10: Probability of Being Compared

In randomized QuickSort let the elements in sorted order be z₁ < z₂ < … < zn. What is the probability that z₂ and z₆ are compared with each other?

B: From the set {z₂, …, z₆} (5 elements), they are compared if the first pivot chosen is z₂ or z₆; if one of the elements z₃, z₄, z₅ in between is chosen first, they end up in separate parts. Uniform choice → 2/5. General rule: 2/(j − i + 1); adjacent elements (j = i + 1) are always compared.

Step-by-Step Trace · Randomized QuickSort

Question 11: Tracing QuickSelect

Consider a = [6, 5, 3, 1, 8, 7, 2, 4] and the call quickSelect(a, 0, 7, 5): partition draws r = 2, 6, 5 in turn. How do the calls proceed?

10low2132p6384755647high

The first call is on a = [6, 5, 3, 1, 8, 7, 2, 4]; partition draws r = 2 (pivot 3) → p = 2, the array is [1, 2, 3, 6, 8, 7, 5, 4].

10213263low84755647high

s = 5 ≠ 2 and s > p → descend only into the right part: quickSelect(a, 3, 7, 5). The left part (faded) is never examined again.

10213243low54p756687high

partition(a, 3, 7) draws r = 6 (pivot 5): the 4, being ≤ 5, goes to the left → p = 4, the array is [1, 2, 3, 4, 5, 7, 6, 8].

102132435475low6687high

s = 5 > p = 4 → quickSelect(a, 5, 7, 5).

102132435465low76p87high

partition(a, 5, 7) draws r = 5 (pivot 7) → p = 6, the array is [1, 2, 3, 4, 5, 6, 7, 8].

102132435465low7687

s = 5 < p = 6 → this time the left part: quickSelect(a, 5, 5, 5).

102132435465low7687

Base case: low == high → a[5] = 6 is returned.

10low2132435465p7687high

The value 6 returns unchanged along the call chain: the 0-based 5th smallest element (the 6th smallest) is 6. Since each call descends into only one side, the expected time is O(n).

static int quickSelect(int[] a, int low, int high, int s) {
  if (low == high) return a[low];
  int p = partition(a, low, high);
  if (s == p) return a[p];
  if (s < p) return quickSelect(a, low, p - 1, s);
  return quickSelect(a, p + 1, high, s);
}

Multiple Choice · Randomized QuickSort

Question 12: QuickSelect Complexity

Which statement about QuickSelect with a random pivot (quickSelect) is true?

B: Since only one side is descended into, the expected work decreases geometrically like n + n/2 + n/4 + … → O(n) (about 3.39 n comparisons for the median). With unlucky pivots the part shrinks by only one each time → O(n²). O(log n) is impossible because the partitioning alone is already Θ(n).

Step-by-Step Trace · Monte Carlo

Question 13: Tracing Monte Carlo π

System.out.println(estimatePi(6)) is run; the random points are given. How does inside change, and what is printed?

01

Unit square and quarter circle (x² + y² ≤ 1). nextDouble returns these points in turn: (0.3, 0.4), (0.9, 0.6), (0.5, 0.5), (0.8, 0.7), (0.1, 0.9), (0.7, 0.7).

01

t = 0: x² + y² = 0.25 ≤ 1 → inside (green), inside = 1.

01

t = 1: x² + y² = 1.17 > 1 → outside (red), inside is unchanged.

01

t = 2: x² + y² = 0.50 ≤ 1 → inside (green), inside = 2.

01

t = 3: x² + y² = 1.13 > 1 → outside (red), inside is unchanged.

01

t = 4: x² + y² = 0.82 ≤ 1 → inside (green), inside = 3.

01

t = 5: x² + y² = 0.98 ≤ 1 → inside (green), inside = 4.

01

The loop ends: 4 · 4 / 6 ≈ 2.667. With 6 points the estimate is rough: error ≈ 1.64/√n ≈ 0.67. Multiplying by 4.0 forces a double division (4 * inside / n with integer division would give 2).

static double estimatePi(int n) {
  int inside = 0;
  for (int t = 0; t < n; t++) {
    double x = dice.nextDouble(), y = dice.nextDouble();
    if (x * x + y * y <= 1) inside++;
  }
  return 4.0 * inside / n;
}

Multiple Choice · Monte Carlo

Question 14: One More Digit of Accuracy

A Monte Carlo π estimate with n = 10,000 points has an error of about 0.016. How many points are needed to reduce the error to about 0.0016?

D: The error shrinks like 1/√n: to cut the error to a tenth, √n must grow 10 times, i.e. n must grow 100 times → 1,000,000. With 100,000 points the error shrinks only by √10 ≈ 3.16. That is why Monte Carlo is slow when high accuracy is needed.

Step-by-Step Trace · Monte Carlo

Question 15: Tracing Miller–Rabin

isPrime(221, 2) is called; the random bases turn out to be a = 174 and a = 137 in turn. How does x change in each round, and what is the result?

n = 221 is odd and ≥ 4: the small-case and even-number checks are passed. (221 = 13 · 17, but the algorithm does not know that.)

n − 1 = 220 = 2² · 55 → d = 55, s = 2. Sequence to examine: a55, a110 (mod 221).

a^d ≡ 47green: n − 1 = 220 seen

round = 0, a = 174: x = 17455 mod 221 = 47. It is neither 1 nor 220 → witness = true.

a^d ≡ 47a^2d ≡ 220green: n − 1 = 220 seen

r = 1: x = 47² mod 221 = 2209 mod 221 = 220 = n − 1 → witness = false. 174 is a strong liar: 221 passes this round.

witness is false → no return, we move on to the second round. If a single round were enough, it would wrongly say "prime".

a^d ≡ 188green: n − 1 = 220 seen

round = 1, a = 137: x = 13755 mod 221 = 188 → witness = true.

a^d ≡ 188a^2d ≡ 205green: n − 1 = 220 seen

r = 1: x = 188² mod 221 = 205 ≠ 220. r = 2 < s is false, the inner loop ends; n − 1 never appeared.

witness true → false is returned: 137 is a witness and the answer "composite" is certain (one-sided error).

static boolean isPrime(long n, int k) {
  if (n < 4) return n == 2 || n == 3;
  if (n % 2 == 0) return false;
  long d = n - 1; int s = 0;  // n-1 = 2^s·d
  while (d % 2 == 0) { d /= 2; s++; }
  for (int round = 0; round < k; round++) {
    long a = 2 + dice.nextLong(n - 3);  // [2, n-2]
    long x = modPow(a, d, n);  // a^d mod n
    if (x == 1 || x == n - 1) continue;
    boolean witness = true;
    for (int r = 1; r < s && witness; r++) {
      x = x * x % n;
      if (x == n - 1) witness = false;
    }
    if (witness) return false;  // certainly composite
  }
  return true;  // probably prime
}

Multiple Choice · Monte Carlo

Question 16: How Many Rounds Are Needed?

In Miller–Rabin, a single round errs on a composite number with probability at most 1/4. What is the minimum number of independent rounds needed to push the error probability below 10−9?

A: 4−k < 10−9 ⇔ 4k > 10⁹; 415 = 230 ≈ 1.07 · 10⁹ is enough, 414 ≈ 2.7 · 10⁸ is not enough. 30 would assume 2−k per round; 9 would assume 10−k per round.

Multiple Choice · Monte Carlo

Question 17: Karger's Success Probability

With what minimum probability does a single run of Karger's random edge-contraction algorithm find a minimum cut in a graph with n = 5 vertices?

B: The lower bound is 2/(n(n − 1)) = 2/20 = 1/10. In each contraction the probability of not choosing an edge of the minimum cut is 1 − 2/k (k: remaining vertices); the product is 3/5 · 2/4 · 1/3 = 1/10. The true success probability can be higher depending on the graph.

Output Question · Monte Carlo

Question 18: Order of Karger Contractions

In the graph in the figure, Karger's algorithm contracts the edges (2, 3), (1, 2) and (4, 5) in turn (self-loops are deleted, parallel edges are kept). Which cut is found, and how many edges does it have? Is it a minimum cut?

12345
Show answer
  • (2, 3) → supervertex {2, 3}; the edge between them becomes a self-loop and is deleted. 6 edges remain.
  • (1, 2) → {1, 2, 3}; (1, 3) also becomes a self-loop. 4 edges remain.
  • (4, 5) → {4, 5}. Two supervertices remain.
{1, 2, 3}{4, 5}2–43–43–5

Cut {1, 2, 3} | {4, 5}: 3 edges (2–4, 3–4, 3–5). It is not a minimum cut: vertices 1 and 5 have degree 2, e.g. {1} | the rest is a cut of 2 edges. Because the output can be wrong, Karger is a Monte Carlo algorithm; it is repeated many times and the smallest result is taken.

The End

Chapter 7: Randomized Algorithms: Study Questions
Lecture slides: Chapter 7: Randomized Algorithms · Lecture notes
1 / 1 Sercan KÜLCÜ. All rights reserved.