Chapter 1: Divide and Conquer
Introduction
Learning Outcomes
Explain the steps of divide and conquer and decide whether a problem suits the approach.
Set up a recurrence and use the master theorem to find the running time of a divide-and-conquer algorithm.
Trace Quickhull for the convex hull and the strip method for the closest pair of points.
Show how Karatsuba (integers) and Strassen (matrices) gain speed by saving one recursive call.
Paradigm
The Divide–Break Down–Conquer Paradigm
Divide and conquer solves complex problems effectively in four steps:
- Divide: split the problem into smaller, manageable subproblems.
- Break down: keep splitting until the subproblems are small enough to solve directly (recursion).
- Conquer: solve each subproblem separately.
- Combine: merge the subproblem solutions into a solution of the original problem.
Paradigm
Why Divide and Conquer?
- It breaks a complex problem into small, manageable pieces that are easier to understand and solve.
- The same method (the same function) is reused at every level: short, reusable code.
- Uses: sorting, searching, computational geometry, big-number and matrix arithmetic, and game-playing AI.
Advantages
- Often an asymptotic gain: O(n²) → O(n log n), O(n) → O(log n).
- Independent subproblems can be solved in parallel.
- Small subproblems fit in cache: good memory locality.
- Correctness proofs by induction are easy.
Disadvantages
- Recursion overhead: call stack, extra memory (e.g., O(n) for merge sort).
- If subproblems overlap, the same work is repeated → exponential time.
- For small n the constant factors dominate; in practice, switch to a simple method below a threshold.
Paradigm
General Template and Recurrence
Result divideAndConquer(Problem p) {
if (p.size() <= THRESHOLD)
return solveDirectly(p); // base case
Problem[] subs = divide(p); // a subproblems of size n/b
Result[] sols = new Result[subs.length];
for (int i = 0; i < subs.length; i++)
sols[i] = divideAndConquer(subs[i]); // conquer: recurse
return combine(sols); // combine
}T(n) = a·T(n/b) + f(n)
a: number of subproblems · n/b: subproblem size · f(n): cost of dividing + combining
- Without a base case, the recursion never ends.
- The split should be balanced: n/2 + n/2 is good, 1 + (n − 1) is bad.
- Speedups usually come from reducing a (Karatsuba, Strassen) or shrinking f(n).
Paradigm · Run the code step by step
The Template in Action: Maximum Element
int findMax(int[] a, int lo, int hi) { if (lo == hi) return a[lo]; // base case int mid = (lo + hi) / 2; int left = findMax(a, lo, mid); int right = findMax(a, mid + 1, hi); return Math.max(left, right); // combine }
Array [5, 2, 9, 4, 7]; findMax(a, 0, 4) is called. The range has more than one element → mid = 2; the left half [0..2] is solved first.
findMax(0, 2): mid = 1 → left half [0..1] first, then right half [2..2].
findMax(0, 1): mid = 0 → left half [0..0] first, then right half [1..1].
findMax(0, 0): one element → base case, returns a[0] = 5.
findMax(1, 1): one element → base case, returns a[1] = 2.
findMax(0, 1): left = 5, right = 2 → combine: returns max = 5.
findMax(2, 2): one element → base case, returns a[2] = 9.
findMax(0, 2): left = 5, right = 9 → combine: returns max = 9.
findMax(3, 4): mid = 3 → left half [3..3] first, then right half [4..4].
findMax(3, 3): one element → base case, returns a[3] = 4.
findMax(4, 4): one element → base case, returns a[4] = 7.
findMax(3, 4): left = 4, right = 7 → combine: returns max = 7.
findMax(0, 4): left = 9, right = 7 → combine: returns max = 9. Result 9: n leaves, n − 1 comparisons; T(n) = 2T(n/2) + O(1) = O(n).
Paradigm
Classic Divide-and-Conquer Algorithms
| Algorithm | Divide | Combine | Recurrence | Time |
|---|---|---|---|---|
| Binary search | check the middle, discard one half | none | T(n/2) + O(1) | O(log n) |
| Merge sort | split the array in half | merge the sorted halves | 2T(n/2) + O(n) | O(n log n) |
| Quicksort | partition around a pivot | none (in place) | T(k) + T(n−k−1) + O(n) | avg. O(n log n) |
| Fast exponentiation | xⁿ = (x⌊n/2⌋)²·(x) | square | T(n/2) + O(1) | O(log n) |
| Quickhull | triangle with the farthest point | join the hull pieces | T(n₁) + T(n₂) + O(n) | avg. O(n log n) |
| Closest pair of points | split at the median x | scan the middle strip | 2T(n/2) + O(n) | O(n log n) |
| Karatsuba | split the number in half | add up 3 products | 3T(n/2) + O(n) | O(n1.585) |
| Strassen | split the matrix into 4 blocks | add up 7 products | 7T(n/2) + O(n²) | O(n2.807) |
Recurrences
The Master Theorem and This Chapter's Algorithms
T(n) = a·T(n/b) + f(n), a ≥ 1, b > 1. Compare f(n) with the total leaf work nlogb a: if f(n) is polynomially smaller, Θ(nlogb a) (case 1, leaves dominate); if it is of the same order, Θ(nlogb a · log n) (case 2); if it is polynomially larger and the regularity condition holds, Θ(f(n)) (case 3, root dominates).
| Algorithm | a | b | f(n) | nlogb a | Case | T(n) |
|---|---|---|---|---|---|---|
| Binary search (decrease and conquer), fast exponentiation | 1 | 2 | 1 | n⁰ = 1 | 2 | Θ(log n) |
| Merge sort, closest pair, balanced quicksort | 2 | 2 | n | n | 2 | Θ(n log n) |
| Karatsuba | 3 | 2 | n | n1.585 | 1 | Θ(n1.585) |
| Naive block matrix multiplication | 8 | 2 | n² | n³ | 1 | Θ(n³) |
| Strassen | 7 | 2 | n² | n2.807 | 1 | Θ(n2.807) |
All of the gain in Karatsuba and Strassen comes from reducing a by one: 4 → 3 and 8 → 7. Quickhull (T(n₁) + T(n₂) + n) and unbalanced quicksort (T(n − 1) + n) have unequal splits, so the master theorem does not apply; solve them with a recursion tree.
Basic Examples
Merge Sort: A Divide-and-Conquer View
Array: [38, 27, 43, 3, 9, 82, 10], n = 7.
Divide: mid = (0 + 6)/2 = 3 → [38 27 43 3] and [9 82 10].
The halves are split again: [38 27] [43 3] [9 82] [10].
One-element arrays are already sorted: the base case.
Combine: pairs are merged in order: [27 38] [3 43] [9 82] [10].
[3 27 38 43] and [9 10 82]: each merge takes as many steps as the two halves have elements.
Final merge: [3 9 10 27 38 43 82]. Each level costs O(n); ⌈log₂ 7⌉ = 3 levels.
- Divide: split the array in the middle, O(1). Conquer: sort both halves recursively. Combine: merge the sorted halves in one pass, Θ(n).
- T(n) = 2T(n/2) + Θ(n) → case 2 → Θ(n log n) in the best, average, and worst case alike. The real work happens in the merge.
Basic Examples
Quicksort: A Divide-and-Conquer View
Divide: partition around a pivot, Θ(n). Conquer: sort both parts recursively. Combine: nothing; the parts are sorted in place. The real work happens in the partition.
Array [10 80 30 90 40 50 70]; Lomuto partition: the last element is the pivot (70).
partition(arr, 0, 6), pivot 70 → placed at index 4. Left: [10 30 40 50], right: [90 80].
partition(arr, 0, 3), pivot 50 → placed at index 3. Left: [10 30 40], right: [—]. Unbalanced split!
partition(arr, 0, 2), pivot 40 → placed at index 2. Left: [10 30], right: [—]. Unbalanced split!
partition(arr, 0, 1), pivot 30 → placed at index 1. Left: [10], right: [—]. Unbalanced split!
partition(arr, 5, 6), pivot 80 → placed at index 5. Left: [—], right: [90]. Unbalanced split!
Sorted: [10 30 40 50 70 80 90]. No combine step; the work is done in the partition.
Basic Examples · Run the code step by step
Fast Exponentiation: Tracing 3¹³
long power(long x, int n) { if (n == 0) return 1; // base case long y = power(x, n / 2); if (n % 2 == 0) return y * y; // even return x * y * y; // odd }
The naive method uses 12 multiplications, this one 7; for n = 10⁶, 999,999 versus 27 → O(log n): T(n) = T(n/2) + O(1).
power(3, 13) is called → first power(3, 6) must be computed (n/2 = 6).
power(3, 6) is called → first power(3, 3) must be computed (n/2 = 3).
power(3, 3) is called → first power(3, 1) must be computed (n/2 = 1).
power(3, 1) is called → first power(3, 0) must be computed (n/2 = 0).
n = 0 → base case: returns 3⁰ = 1.
power(3, 1): y = 1; 1 is odd → x·y·y = 3·1·1 = 3.
power(3, 3): y = 3; 3 is odd → x·y·y = 3·3·3 = 27.
power(3, 6): y = 27; 6 is even → y·y = 27·27 = 729.
power(3, 13): y = 729; 13 is odd → x·y·y = 3·729·729 = 1594323.
Basic Examples
When Does It Fail? Overlapping Subproblems
long fib(int n) { if (n < 2) return n; return fib(n - 1) + fib(n - 2); }
- 15 calls for F(5): F(3) is computed 2 times, F(2) 3 times.
- The subproblems are not independent; they overlap → T(n) = T(n−1) + T(n−2) + O(1) = O(φⁿ) ≈ O(1.618ⁿ).
- Fix: store the results → dynamic programming, O(n).
Convex Hull
Convex Hull
- Let S be a set of n points in the plane. The convex hull of S is the smallest convex polygon containing S.
- Hammer a nail into each point and stretch a rubber band around them: the shape it takes is the hull.
- The hull's vertices are points of S; the output is the list of these vertices in counterclockwise order.
- Convex: the segment joining any two points inside lies entirely within the polygon.
Convex Hull
The Quickhull Algorithm
- The points with the smallest (A) and largest (B) x are on the hull.
- Line AB splits the points into two subsets: above and below it.
- In a subset, the point C farthest from AB is on the hull.
- Points inside triangle ACB are discarded: they cannot be on the hull.
- Recurse on the points outside AC and outside CB.
- Stop when no points remain; the points found form the hull.
Steps 1–2: the leftmost A and the rightmost B are on the hull; line AB splits the points into two sets (upper / lower).
Steps 3–5: C (13,27), farthest from AB in the upper set, is on the hull. The 4 points inside ACB are discarded; recurse outside AC and CB.
Step 6: recurse until no point is left outside → a 7-vertex hull.
Like quicksort (hence the name): the farthest point is the pivot, and the triangle's sides partition the rest.
Convex Hull
Geometry: Orientation Test and Distance
Quickhull needs only integer multiplication; no angles, square roots, or division:
long cross(Point A, Point B, Point P) { return (long)(B.x - A.x) * (P.y - A.y) - (long)(B.y - A.y) * (P.x - A.x); }
- cross > 0: P is to the left of A→B; < 0: to the right; = 0: on the line.
- |cross| = 2 × (area of triangle ABP) = |AB| × distance. Since |AB| is fixed, farthest point = largest cross.
Convex Hull · Run the code step by step
Quickhull: Code Trace
void quickHull(List<Point> P) { Point A = leftmost(P), B = rightmost(P); hull.add(A); hull.add(B); findHull(leftOf(P, A, B), A, B); // upper set findHull(leftOf(P, B, A), B, A); // lower set } void findHull(List<Point> S, Point A, Point B) { if (S.isEmpty()) return; // AB: hull edge Point C = farthest(S, A, B); // max cross hull.add(C); findHull(leftOf(S, A, C), A, C); // outside AC findHull(leftOf(S, C, B), C, B); // outside CB } // inside ACB: dropped
The leftmost A = (2,18) and the rightmost B = (35,11) are added to the hull.
8 points lie left of AB (upper), 6 right of it (lower). The upper set comes first.
|S| = 8: the farthest point C = (13,27) joins the hull. 4 points inside the triangle are discarded; 1 point lies outside AC, 2 outside CB.
|S| = 1: the farthest point C = (4,25) joins the hull. 0 points inside the triangle are discarded; 0 outside AC, 0 outside CB.
S is empty → (2,18)–(4,25) is a hull edge; return.
S is empty → (4,25)–(13,27) is a hull edge; return.
|S| = 2: the farthest point C = (28,23) joins the hull. 1 point inside the triangle is discarded; 0 outside AC, 0 outside CB.
S is empty → (13,27)–(28,23) is a hull edge; return.
S is empty → (28,23)–(35,11) is a hull edge; return.
|S| = 6: the farthest point C = (20,2) joins the hull. 4 points inside the triangle are discarded; 0 outside AC, 1 outside CB.
S is empty → (35,11)–(20,2) is a hull edge; return.
|S| = 1: the farthest point C = (10,5) joins the hull. 0 points inside the triangle are discarded; 0 outside AC, 0 outside CB.
S is empty → (20,2)–(10,5) is a hull edge; return.
S is empty → (10,5)–(2,18) is a hull edge; return.
Done: 7 vertices found, 9 points discarded. 12 findHull calls in total.
Convex Hull
Quickhull: A 104-Point Example
104 points. The leftmost (-14,46) and rightmost (106,20) are joined; 71 points lie above, 31 below.
Farthest from (-14,46)→(106,20): (54,120); 56 points inside the triangle discarded. Candidates left: 45.
Farthest from (-14,46)→(54,120): (8,84); 2 points inside the triangle discarded. Candidates left: 42.
Farthest from (-14,46)→(8,84): (-4,69); 0 points inside the triangle discarded. Candidates left: 41.
Farthest from (8,84)→(54,120): (42,111); 1 point inside the triangle discarded. Candidates left: 39.
Farthest from (54,120)→(106,20): (104,74); 6 points inside the triangle discarded. Candidates left: 32.
Farthest from (54,120)→(104,74): (77,103); 0 points inside the triangle discarded. Candidates left: 31.
Farthest from (106,20)→(-14,46): (11,-25); 25 points inside the triangle discarded. Candidates left: 5.
Farthest from (106,20)→(11,-25): (88,-5); 4 points inside the triangle discarded. Candidates left: 0.
Done: the hull of 104 points has 10 vertices. Most points are discarded at every step, so the work shrinks fast.
Convex Hull
Quickhull Analysis and Other Hull Algorithms
- Each call scans S once (farthest point + two partitions): O(|S|).
- T(n) = T(n₁) + T(n₂) + O(n), n₁ + n₂ ≤ n − 1.
- Average (balanced splits, many points discarded): O(n log n); often even faster for uniformly random points.
- Worst case: only one point is split off at each step → T(n) = T(n − 1) + O(n) = O(n²).
- Output-sensitive bound: O(n·h) for a hull with h vertices.
| Algorithm | Time |
|---|---|
| Brute force (every candidate edge) | O(n³) |
| Jarvis march (gift wrapping) | O(n·h) |
| Graham scan | O(n log n) |
| Andrew's monotone chain | O(n log n) |
| Quickhull | avg. O(n log n), worst O(n²) |
| Chan | O(n log h) |
Closest Pair of Points
Closest Pair of Points
Problem: among n points in the plane, find the pair with the smallest Euclidean distance d(p, q) = √((pₓ − qₓ)² + (py − qy)²).
double bruteForce(List<Point> p) { double best = Double.MAX_VALUE; for (int i = 0; i < p.size(); i++) for (int j = i + 1; j < p.size(); j++) best = Math.min(best, dist(p.get(i), p.get(j))); return best; }
- Brute force: n(n − 1)/2 pairs → O(n²). For n = 10⁶, ≈ 5·10¹¹ distances.
- Easy in one dimension: sort and compare only neighbors → O(n log n).
- In two dimensions, neighbors in x order are not enough; divide and conquer achieves O(n log n).
Closest Pair of Points
Divide and Conquer: The Middle Strip
- Sort the points once by x and once by y.
- Divide: split into two halves with the vertical line L through the middle point.
- Conquer: recursively find δL and δR in the halves; δ = min(δL, δR).
- Combine: a closer pair must cross L → check only the strip points with |x − L| < δ.
- Scan the strip in y order; compare each point only with the following points whose y differs by less than δ.
Closest Pair of Points · Run the code step by step
Closest Pair: Code Trace
double closest(List<Point> X, List<Point> Y) { int n = X.size(), mid = n / 2; if (n <= 3) return bruteForce(X); Point m = X.get(mid); // YL/YR: the halves of Y double dL = closest(X.subList(0, mid), YL); double dR = closest(X.subList(mid, n), YR); double d = Math.min(dL, dR); List<Point> s = new ArrayList<>(); // strip for (Point p : Y) // in y order if (Math.abs(p.x - m.x) < d) s.add(p); for (int i = 0; i < s.size(); i++) { Point p = s.get(i); for (int j = i + 1; j < s.size() && s.get(j).y - p.y < d; j++) d = Math.min(d, dist(p, s.get(j))); } return d; }
n = 10 > 3 → mid = 5, m = (12,10). Left: 5 points, right: 5.
n = 5 > 3 → mid = 2, m = (5,1). Left: 2 points, right: 3.
n = 2 ≤ 3 → brute force: (2,3), (3,4). Closest (2,3)–(3,4), d = 1.41.
n = 3 ≤ 3 → brute force: (5,1), (7,7), (12,30). Closest (5,1)–(7,7), d = 6.32.
Both halves returned: dL = 1.41, dR = 6.32 → d = 1.41.
Strip |x − 5| < 1.41: [(5,1)]; a single point → no comparisons. Returns 1.41.
n = 5 > 3 → mid = 2, m = (27,27). Left: 2 points, right: 3.
n = 2 ≤ 3 → brute force: (12,10), (17,17). Closest (12,10)–(17,17), d = 8.60.
n = 3 ≤ 3 → brute force: (27,27), (28,27), (40,50). Closest (27,27)–(28,27), d = 1.00.
Both halves returned: dL = 8.60, dR = 1.00 → d = 1.00.
Strip |x − 27| < 1.00: [(27,27)]; a single point → no comparisons. Returns 1.00.
Both halves returned: dL = 1.41, dR = 1.00 → d = 1.00.
Strip |x − 12| < 1.00: [(12,10), (12,30)]; the y gap exceeds d → no comparisons at all. Returns 1.00.
Closest Pair of Points
Closest Pair: Analysis
| Step | Cost | Note |
|---|---|---|
| Presort by x and by y | O(n log n) | only once, before the recursion |
| Divide (X in half, Y into YL/YR keeping the order) | O(n) | no re-sorting |
| Build the strip | O(n) | Y is already sorted by y |
| Scan the strip | O(7n) = O(n) | ≤ 7 neighbors per point |
| Total | O(n log n) | T(n) = 2T(n/2) + O(n) |
Karatsuba Multiplication
Karatsuba: 3 Multiplications Instead of 4
Split the n-digit numbers x, y at m = ⌊n/2⌋: x = a·10m + b, y = c·10m + d.
4 products: ac, ad, bc, bd
T(n) = 4T(n/2) + O(n) = O(n²) → no gain!
⇒ ad + bc = (a + b)(c + d) − ac − bd
3 products: z2 = ac, z0 = bd, z1 = (a + b)(c + d)
T(n) = 3T(n/2) + O(n) = O(n1.585)
| Example: 1234 · 5678 (m = 2, 10m = 100) | |
|---|---|
| a = 12, b = 34, c = 56, d = 78 | z2 = 12·56 = 672 · z0 = 34·78 = 2652 |
| z1 = (12 + 34)(56 + 78) = 46·134 = 6164 | z1 − z2 − z0 = 6164 − 672 − 2652 = 2840 (= ad + bc) |
| 672·10⁴ + 2840·10² + 2652 = 6,720,000 + 284,000 + 2652 = 7,006,652 ✓ | |
Karatsuba Multiplication · Run the code step by step
Karatsuba: Tracing 1234 · 5678
karatsuba(1234, 5678): n = 4, p = 100 → a = 12, b = 34, c = 56, d = 78. Three calls: (34, 78), (46, 134), (12, 56).
karatsuba(34, 78): n = 2, p = 10 → a = 3, b = 4, c = 7, d = 8. Three calls: (4, 8), (7, 15), (3, 7).
4 · 8: a one-digit factor → base case, directly 32.
7 · 15: a one-digit factor → base case, directly 105.
3 · 7: a one-digit factor → base case, directly 21.
21·100 + (105 − 21 − 32)·10 + 32 = 2100 + 520 + 32 = 2652.
karatsuba(46, 134): n = 3, p = 10 → a = 4, b = 6, c = 13, d = 4. Three calls: (6, 4), (10, 17), (4, 13).
6 · 4: a one-digit factor → base case, directly 24.
karatsuba(10, 17): n = 2, p = 10 → a = 1, b = 0, c = 1, d = 7. Three calls: (0, 7), (1, 8), (1, 1).
0 · 7: a one-digit factor → base case, directly 0.
1 · 8: a one-digit factor → base case, directly 8.
1 · 1: a one-digit factor → base case, directly 1.
1·100 + (8 − 1 − 0)·10 + 0 = 100 + 70 + 0 = 170.
4 · 13: a one-digit factor → base case, directly 52.
52·100 + (170 − 52 − 24)·10 + 24 = 5200 + 940 + 24 = 6164.
karatsuba(12, 56): n = 2, p = 10 → a = 1, b = 2, c = 5, d = 6. Three calls: (2, 6), (3, 11), (1, 5).
2 · 6: a one-digit factor → base case, directly 12.
3 · 11: a one-digit factor → base case, directly 33.
1 · 5: a one-digit factor → base case, directly 5.
5·100 + (33 − 5 − 12)·10 + 12 = 500 + 160 + 12 = 672.
672·10000 + (6164 − 672 − 2652)·100 + 2652 = 6720000 + 284000 + 2652 = 7006652.
long karatsuba(long x, long y) { if (x < 10 || y < 10) return x * y; // base case int n = Math.max(digits(x), digits(y)); long p = (long) Math.pow(10, n / 2); long a = x / p, b = x % p; // x = a·p + b long c = y / p, d = y % p; // y = c·p + d long z0 = karatsuba(b, d); long z1 = karatsuba(a + b, c + d); long z2 = karatsuba(a, c); return z2 * p * p + (z1 - z2 - z0) * p + z0; }
Karatsuba Multiplication
Karatsuba Analysis
- T(n) = 3T(n/2) + O(n) → case 1 → Θ(nlog₂3) ≈ Θ(n1.585).
- Addition, subtraction, and shifting by 10m cost O(n); cutting the products from 4 to 3 lowers the exponent from 2 to 1.585.
- The example makes 16 calls and 11 base-case products (the schoolbook method needs 16 one-digit products).
- Slower on small numbers because of the extra additions → use the schoolbook method below a threshold.
- Java's
BigInteger.multiplyuses Karatsuba and Toom-Cook for large numbers. Faster still: Schönhage–Strassen O(n log n log log n), Harvey–van der Hoeven O(n log n) (2019).
| n (digits) | n² | n1.585 | ratio |
|---|---|---|---|
| 10 | 100 | 38 | 2.6× |
| 100 | 10,000 | 1,479 | 6.8× |
| 1,000 | 1.0·106 | 56,871 | 17.6× |
| 10,000 | 1.0·108 | 2.2·106 | 45.7× |
| 1.0·106 | 1.0·1012 | 3.2·109 | 309.2× |
Strassen Multiplication
Matrix Multiplication: Classic and Block
for (int i = 0; i < n; i++) for (int j = 0; j < n; j++) for (int k = 0; k < n; k++) C[i][j] += A[i][k] * B[k][j];
n² cells, each needing n products → n³ products.
Split A, B, and C into four n/2 × n/2 blocks:
| C11 = A11·B11 + A12·B21 |
| C12 = A11·B12 + A12·B22 |
| C21 = A21·B11 + A22·B21 |
| C22 = A21·B12 + A22·B22 |
Strassen Multiplication
Strassen (1969): 7 Products Instead of 8
| Product | Definition |
|---|---|
| P1 | A11 · (B12 − B22) |
| P2 | (A11 + A12) · B22 |
| P3 | (A21 + A22) · B11 |
| P4 | A22 · (B21 − B11) |
| P5 | (A11 + A22) · (B11 + B22) |
| P6 | (A12 − A22) · (B21 + B22) |
| P7 | (A11 − A21) · (B11 + B12) |
| Result blocks |
|---|
| C11 = P5 + P4 − P2 + P6 |
| C12 = P1 + P2 |
| C21 = P3 + P4 |
| C22 = P5 + P1 − P3 − P7 |
7 products + 18 block additions/subtractions. The blocks are themselves multiplied with Strassen recursively; the base case is 1×1.
Strassen Multiplication
Strassen Step by Step: A 2 × 2 Example
| 1 | 3 |
| 7 | 5 |
| 6 | 8 |
| 4 | 2 |
| P1 | A11 · (B12 − B22) | 1 · (8 − 2) | 6 |
| P2 | (A11 + A12) · B22 | … | |
| P3 | (A21 + A22) · B11 | … | |
| P4 | A22 · (B21 − B11) | … | |
| P5 | (A11 + A22) · (B11 + B22) | … | |
| P6 | (A12 − A22) · (B21 + B22) | … | |
| P7 | (A11 − A21) · (B11 + B12) | … |
P1 = 1 · (8 − 2) = 6.
| P1 | A11 · (B12 − B22) | 1 · (8 − 2) | 6 |
| P2 | (A11 + A12) · B22 | (1 + 3) · 2 | 8 |
| P3 | (A21 + A22) · B11 | … | |
| P4 | A22 · (B21 − B11) | … | |
| P5 | (A11 + A22) · (B11 + B22) | … | |
| P6 | (A12 − A22) · (B21 + B22) | … | |
| P7 | (A11 − A21) · (B11 + B12) | … |
P2 = (1 + 3) · 2 = 8.
| P1 | A11 · (B12 − B22) | 1 · (8 − 2) | 6 |
| P2 | (A11 + A12) · B22 | (1 + 3) · 2 | 8 |
| P3 | (A21 + A22) · B11 | (7 + 5) · 6 | 72 |
| P4 | A22 · (B21 − B11) | … | |
| P5 | (A11 + A22) · (B11 + B22) | … | |
| P6 | (A12 − A22) · (B21 + B22) | … | |
| P7 | (A11 − A21) · (B11 + B12) | … |
P3 = (7 + 5) · 6 = 72.
| P1 | A11 · (B12 − B22) | 1 · (8 − 2) | 6 |
| P2 | (A11 + A12) · B22 | (1 + 3) · 2 | 8 |
| P3 | (A21 + A22) · B11 | (7 + 5) · 6 | 72 |
| P4 | A22 · (B21 − B11) | 5 · (4 − 6) | −10 |
| P5 | (A11 + A22) · (B11 + B22) | … | |
| P6 | (A12 − A22) · (B21 + B22) | … | |
| P7 | (A11 − A21) · (B11 + B12) | … |
P4 = 5 · (4 − 6) = −10.
| P1 | A11 · (B12 − B22) | 1 · (8 − 2) | 6 |
| P2 | (A11 + A12) · B22 | (1 + 3) · 2 | 8 |
| P3 | (A21 + A22) · B11 | (7 + 5) · 6 | 72 |
| P4 | A22 · (B21 − B11) | 5 · (4 − 6) | −10 |
| P5 | (A11 + A22) · (B11 + B22) | (1 + 5) · (6 + 2) | 48 |
| P6 | (A12 − A22) · (B21 + B22) | … | |
| P7 | (A11 − A21) · (B11 + B12) | … |
P5 = (1 + 5) · (6 + 2) = 48.
| P1 | A11 · (B12 − B22) | 1 · (8 − 2) | 6 |
| P2 | (A11 + A12) · B22 | (1 + 3) · 2 | 8 |
| P3 | (A21 + A22) · B11 | (7 + 5) · 6 | 72 |
| P4 | A22 · (B21 − B11) | 5 · (4 − 6) | −10 |
| P5 | (A11 + A22) · (B11 + B22) | (1 + 5) · (6 + 2) | 48 |
| P6 | (A12 − A22) · (B21 + B22) | (3 − 5) · (4 + 2) | −12 |
| P7 | (A11 − A21) · (B11 + B12) | … |
P6 = (3 − 5) · (4 + 2) = −12.
| P1 | A11 · (B12 − B22) | 1 · (8 − 2) | 6 |
| P2 | (A11 + A12) · B22 | (1 + 3) · 2 | 8 |
| P3 | (A21 + A22) · B11 | (7 + 5) · 6 | 72 |
| P4 | A22 · (B21 − B11) | 5 · (4 − 6) | −10 |
| P5 | (A11 + A22) · (B11 + B22) | (1 + 5) · (6 + 2) | 48 |
| P6 | (A12 − A22) · (B21 + B22) | (3 − 5) · (4 + 2) | −12 |
| P7 | (A11 − A21) · (B11 + B12) | (1 − 7) · (6 + 8) | −84 |
P7 = (1 − 7) · (6 + 8) = −84.
| C11 = P5 + P4 − P2 + P6 | 48 + (−10) − 8 + (−12) | 18 |
| C12 = P1 + P2 | 6 + 8 | 14 |
| C21 = P3 + P4 | 72 + (−10) | 62 |
| C22 = P5 + P1 − P3 − P7 | 48 + 6 − 72 − (−84) | 66 |
C = [[18, 14], [62, 66]]. Check: 1·6 + 3·4 = 18, 7·8 + 5·2 = 66 ✓. 7 products used (classic: 8).
Strassen Multiplication
Strassen Analysis and Practice
- T(n) = 7T(n/2) + Θ(n²) → case 1 → Θ(nlog₂7) ≈ Θ(n2.807).
- Scalar products for n = 2k: 7k (classic: 8k). For n = 1024: 7¹⁰ ≈ 2.8·10⁸ vs. 8¹⁰ ≈ 1.07·10⁹ → about 3.8 times fewer.
- If n is not a power of 2, the matrix is padded with zeros.
- Threshold: 18 extra additions and temporary matrices make it slow for small n; libraries switch to the classic method for n ≲ 64–128.
- Numerical stability: with floating-point numbers, the subtractions can amplify errors.
- Memory: temporary blocks at every level → extra O(n²).
- The theoretical record is ω < 2.372 (2024); the constants are huge, so it is not used in practice.
Wrap-Up
Naive Method vs. Divide and Conquer
| Problem | Naive method | Divide and conquer | Recurrence |
|---|---|---|---|
| Sorting | O(n²) (bubble sort) | O(n log n) | 2T(n/2) + n |
| Search in a sorted array | O(n) | O(log n) | T(n/2) + 1 |
| Exponentiation xⁿ | O(n) products | O(log n) | T(n/2) + 1 |
| Convex hull | O(n³) | avg. O(n log n) | T(n₁) + T(n₂) + n |
| Closest pair of points | O(n²) | O(n log n) | 2T(n/2) + n |
| Integer multiplication | O(n²) | O(n1.585) | 3T(n/2) + n |
| Matrix multiplication | O(n³) | O(n2.807) | 7T(n/2) + n² |
Where the gain comes from: either the work is split evenly and each level is combined in O(n) (sorting, closest pair), or the number of subproblems is reduced (Karatsuba, Strassen), or half is thrown away (search, exponentiation).
Summary
Summary
| Topic | Key points |
|---|---|
| Paradigm | divide → (break down) → conquer → combine; independent subproblems of the same kind; a base case is required |
| Analysis | T(n) = aT(n/b) + f(n); recursion tree; master theorem: compare f(n) with nlogb a |
| Basic examples | binary search and fast exponentiation O(log n); merge sort O(n log n); quicksort avg. O(n log n), worst O(n²) |
| Quickhull | endpoints A, B; farthest C (max cross); points inside the triangle discarded; avg. O(n log n), worst O(n²) |
| Closest pair | split by x; δ = min(δL, δR); ≤ 7 neighbors in the 2δ strip; O(n log n) |
| Karatsuba | ad + bc = (a + b)(c + d) − ac − bd; 3 products → O(n1.585) |
| Strassen | 7 block products → O(n2.807); C22 = P5 + P1 − P3 − P7 |
Summary
Common Mistakes
StackOverflowError. In Karatsuba: x < 10 || y < 10.Summary
Sample Problems
- Solve T(n) = 4T(n/2) + n with the master theorem.
- Solve T(n) = 2T(n/2) + n²; check the regularity condition.
- Compute 21 · 43 with Karatsuba (m = 1).
- A = (0,0), B = (10,0). Find cross(A, B, ·) for P = (3,4), Q = (5,−2), R = (7,6). Which point does Quickhull choose in the upper set?
- How many multiplications does
power(3, 10)perform? Compare with the naive method. - When merge sort sorts [5, 1, 4, 2, 3], which two sorted arrays are merged in the final merge?
Show answers
- a = 4, b = 2 → nlog₂4 = n²; f(n) = n is smaller → case 1 → Θ(n²).
- nlog₂2 = n; f(n) = n² is larger; 2·(n/2)² = n²/2 ≤ ½·n² ✓ → case 3 → Θ(n²).
- a = 2, b = 1, c = 4, d = 3: z2 = 8, z0 = 3, z1 = 3·7 = 21 → 21 − 8 − 3 = 10 → 800 + 100 + 3 = 903.
- P: 10·4 = 40 (left), Q: 10·(−2) = −20 (right), R: 10·6 = 60 (left). Largest cross in the upper set → R.
- n: 10 → 5 → 2 → 1 → 0. Multiplications: n=1: 2, n=2: 1, n=5: 2, n=10: 1 → 6 (naive: 9). Result 59049.
- mid = 2: [5, 1, 4] → [1, 4, 5] and [2, 3] → [1, 4, 5] + [2, 3] → [1, 2, 3, 4, 5].
Summary
Exercises
- Find the maximum element of an array with divide and conquer; solve the recurrence.
- Modify merge sort to count inversions (i < j, a[i] > a[j]). What is the result for [2, 4, 1, 3, 5]?
- Maximum subarray: give a divide-and-conquer solution for [−2, 1, −3, 4, −1, 2, 1, −5, 4] and its running time.
- Compute A = [[1, 2], [3, 4]] times B = [[5, 6], [7, 8]] with Strassen; try C22 with ± P7.
- Describe an input on which Quickhull takes O(n²) time.
- What would change in the closest pair algorithm if the base case were n ≤ 1 instead of n ≤ 3?
Answers / hints
- max(left half, right half); T(n) = 2T(n/2) + O(1) → case 1 → Θ(n).
- When the merge takes an element from the right, add the count of elements still in the left half; O(n log n). Result 3.
- max(left, right, crossing the middle); 2T(n/2) + O(n) = O(n log n). Result 6 ([4, −1, 2, 1]).
- P1..P7 = −2, 24, 35, 8, 65, −30, −22 → C = [[19, 22], [43, 50]]; C22 = 65 − 2 − 35 + 22 = 50 ✓ (6 with +P7 ✗).
- Points on a convex curve where the farthest point is always next to one end (exponentially spaced): T(n) = T(n − 1) + O(n).
- A one-point half returns ∞; the result is still correct (the strip finds the missed pair), but the number of calls grows.
Summary
References
- Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, 4th ed., 2022 (Chapter 4: Divide-and-Conquer; Chapter 33: Computational Geometry).
- Kleinberg and Tardos, Algorithm Design, 2005 (Chapter 5: Divide and Conquer).
- Sedgewick and Wayne, Algorithms, 4th ed., 2011.
- Barber, Dobkin, and Huhdanpaa, "The Quickhull Algorithm for Convex Hulls", ACM TOMS 22(4), 1996.
- Karatsuba and Ofman, "Multiplication of Multidigit Numbers on Automata", 1962; Strassen, "Gaussian Elimination is not Optimal", Numerische Mathematik 13, 1969.