Chapter 1: Divide and Conquer

Advanced Algorithms
Divide · Break Down · Conquer · CombineMaster theorem Merge sort and quicksortFast exponentiationQuickhull Closest pair of pointsKaratsubaStrassen
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Paradigm

Explain the steps of divide and conquer and decide whether a problem suits the approach.

Analysis

Set up a recurrence and use the master theorem to find the running time of a divide-and-conquer algorithm.

Geometric algorithms

Trace Quickhull for the convex hull and the strip method for the closest pair of points.

Fast multiplication

Show how Karatsuba (integers) and Strassen (matrices) gain speed by saving one recursive call.

Paradigm

The Divide–Break Down–Conquer Paradigm

Divide and conquer solves complex problems effectively in four steps:

  1. Divide: split the problem into smaller, manageable subproblems.
  2. Break down: keep splitting until the subproblems are small enough to solve directly (recursion).
  3. Conquer: solve each subproblem separately.
  4. Combine: merge the subproblem solutions into a solution of the original problem.
Problem (n)n/2n/2n/4n/4n/4n/4solutionsolutionSolution (n)1 Divide2 Break down3 Conquer4 Combine
Key: the subproblems are of the same kind as the original, only smaller, and independent of each other, so the same function calls itself recursively.

Paradigm

Why Divide and Conquer?

  • It breaks a complex problem into small, manageable pieces that are easier to understand and solve.
  • The same method (the same function) is reused at every level: short, reusable code.
  • Uses: sorting, searching, computational geometry, big-number and matrix arithmetic, and game-playing AI.

Advantages

  • Often an asymptotic gain: O(n²) → O(n log n), O(n) → O(log n).
  • Independent subproblems can be solved in parallel.
  • Small subproblems fit in cache: good memory locality.
  • Correctness proofs by induction are easy.

Disadvantages

  • Recursion overhead: call stack, extra memory (e.g., O(n) for merge sort).
  • If subproblems overlap, the same work is repeated → exponential time.
  • For small n the constant factors dominate; in practice, switch to a simple method below a threshold.

Paradigm

General Template and Recurrence

Result divideAndConquer(Problem p) {
  if (p.size() <= THRESHOLD)
    return solveDirectly(p);        // base case
  Problem[] subs = divide(p);       // a subproblems of size n/b
  Result[] sols = new Result[subs.length];
  for (int i = 0; i < subs.length; i++)
    sols[i] = divideAndConquer(subs[i]); // conquer: recurse
  return combine(sols);             // combine
}
Running time:
T(n) = a·T(n/b) + f(n)
a: number of subproblems · n/b: subproblem size · f(n): cost of dividing + combining
  • Without a base case, the recursion never ends.
  • The split should be balanced: n/2 + n/2 is good, 1 + (n − 1) is bad.
  • Speedups usually come from reducing a (Karatsuba, Strassen) or shrinking f(n).

Paradigm · Run the code step by step

The Template in Action: Maximum Element

int findMax(int[] a, int lo, int hi) {
  if (lo == hi) return a[lo];       // base case
  int mid = (lo + hi) / 2;
  int left = findMax(a, lo, mid);
  int right = findMax(a, mid + 1, hi);
  return Math.max(left, right);     // combine
}
50219243740..4

Array [5, 2, 9, 4, 7]; findMax(a, 0, 4) is called. The range has more than one element → mid = 2; the left half [0..2] is solved first.

50219243740..20..4

findMax(0, 2): mid = 1 → left half [0..1] first, then right half [2..2].

50219243740..10..20..4

findMax(0, 1): mid = 0 → left half [0..0] first, then right half [1..1].

50219243740→50..10..20..4

findMax(0, 0): one element → base case, returns a[0] = 5.

50219243740→51→20..10..20..4

findMax(1, 1): one element → base case, returns a[1] = 2.

50219243740→51→20..1→50..20..4

findMax(0, 1): left = 5, right = 2 → combine: returns max = 5.

50219243740→51→20..1→52→90..20..4

findMax(2, 2): one element → base case, returns a[2] = 9.

50219243740→51→20..1→52→90..2→90..4

findMax(0, 2): left = 5, right = 9 → combine: returns max = 9.

50219243740→51→20..1→52→90..2→93..40..4

findMax(3, 4): mid = 3 → left half [3..3] first, then right half [4..4].

50219243740→51→20..1→52→90..2→93→43..40..4

findMax(3, 3): one element → base case, returns a[3] = 4.

50219243740→51→20..1→52→90..2→93→44→73..40..4

findMax(4, 4): one element → base case, returns a[4] = 7.

50219243740→51→20..1→52→90..2→93→44→73..4→70..4

findMax(3, 4): left = 4, right = 7 → combine: returns max = 7.

50219243740→51→20..1→52→90..2→93→44→73..4→70..4→9

findMax(0, 4): left = 9, right = 7 → combine: returns max = 9. Result 9: n leaves, n − 1 comparisons; T(n) = 2T(n/2) + O(1) = O(n).

Paradigm

Classic Divide-and-Conquer Algorithms

AlgorithmDivideCombineRecurrenceTime
Binary searchcheck the middle, discard one halfnoneT(n/2) + O(1)O(log n)
Merge sortsplit the array in halfmerge the sorted halves2T(n/2) + O(n)O(n log n)
Quicksortpartition around a pivotnone (in place)T(k) + T(n−k−1) + O(n)avg. O(n log n)
Fast exponentiationxⁿ = (x⌊n/2⌋)²·(x)squareT(n/2) + O(1)O(log n)
Quickhulltriangle with the farthest pointjoin the hull piecesT(n₁) + T(n₂) + O(n)avg. O(n log n)
Closest pair of pointssplit at the median xscan the middle strip2T(n/2) + O(n)O(n log n)
Karatsubasplit the number in halfadd up 3 products3T(n/2) + O(n)O(n1.585)
Strassensplit the matrix into 4 blocksadd up 7 products7T(n/2) + O(n²)O(n2.807)

Recurrences

The Master Theorem and This Chapter's Algorithms

T(n) = a·T(n/b) + f(n), a ≥ 1, b > 1. Compare f(n) with the total leaf work nlogb a: if f(n) is polynomially smaller, Θ(nlogb a) (case 1, leaves dominate); if it is of the same order, Θ(nlogb a · log n) (case 2); if it is polynomially larger and the regularity condition holds, Θ(f(n)) (case 3, root dominates).

Algorithmabf(n)nlogb aCaseT(n)
Binary search (decrease and conquer), fast exponentiation121n⁰ = 12Θ(log n)
Merge sort, closest pair, balanced quicksort22nn2Θ(n log n)
Karatsuba32nn1.5851Θ(n1.585)
Naive block matrix multiplication82n²n³1Θ(n³)
Strassen72n²n2.8071Θ(n2.807)

All of the gain in Karatsuba and Strassen comes from reducing a by one: 4 → 3 and 8 → 7. Quickhull (T(n₁) + T(n₂) + n) and unbalanced quicksort (T(n − 1) + n) have unequal splits, so the master theorem does not apply; solve them with a recursion tree.

Basic Examples

Merge Sort: A Divide-and-Conquer View

Divide382743398210DivideDivideBaseCombineCombineCombine

Array: [38, 27, 43, 3, 9, 82, 10], n = 7.

Divide382743398210Divide382743398210DivideBaseCombineCombineCombine

Divide: mid = (0 + 6)/2 = 3 → [38 27 43 3] and [9 82 10].

Divide382743398210Divide382743398210Divide382743398210BaseCombineCombineCombine

The halves are split again: [38 27] [43 3] [9 82] [10].

Divide382743398210Divide382743398210Divide382743398210Base382743398210CombineCombineCombine

One-element arrays are already sorted: the base case.

Divide382743398210Divide382743398210Divide382743398210Base382743398210Combine273834398210CombineCombine

Combine: pairs are merged in order: [27 38] [3 43] [9 82] [10].

Divide382743398210Divide382743398210Divide382743398210Base382743398210Combine273834398210Combine327384391082Combine

[3 27 38 43] and [9 10 82]: each merge takes as many steps as the two halves have elements.

Divide382743398210Divide382743398210Divide382743398210Base382743398210Combine273834398210Combine327384391082Combine391027384382

Final merge: [3 9 10 27 38 43 82]. Each level costs O(n); ⌈log₂ 7⌉ = 3 levels.

  • Divide: split the array in the middle, O(1). Conquer: sort both halves recursively. Combine: merge the sorted halves in one pass, Θ(n).
  • T(n) = 2T(n/2) + Θ(n) → case 2 → Θ(n log n) in the best, average, and worst case alike. The real work happens in the merge.

Basic Examples

Quicksort: A Divide-and-Conquer View

Divide: partition around a pivot, Θ(n). Conquer: sort both parts recursively. Combine: nothing; the parts are sorted in place. The real work happens in the partition.

100801302903404505706pivot

Array [10 80 30 90 40 50 70]; Lomuto partition: the last element is the pivot (70).

100301402503704905806pivot

partition(arr, 0, 6), pivot 70 → placed at index 4. Left: [10 30 40 50], right: [90 80].

100301402503704905806pivot

partition(arr, 0, 3), pivot 50 → placed at index 3. Left: [10 30 40], right: [—]. Unbalanced split!

100301402503704905806pivot

partition(arr, 0, 2), pivot 40 → placed at index 2. Left: [10 30], right: [—]. Unbalanced split!

100301402503704905806pivot

partition(arr, 0, 1), pivot 30 → placed at index 1. Left: [10], right: [—]. Unbalanced split!

100301402503704805906pivot

partition(arr, 5, 6), pivot 80 → placed at index 5. Left: [—], right: [90]. Unbalanced split!

100301402503704805906

Sorted: [10 30 40 50 70 80 90]. No combine step; the work is done in the partition.

Balanced splitIf the pivot is near the median, T(n) = 2T(n/2) + O(n) → O(n log n) (average case).
Unbalanced splitSorted input with the last element as pivot: T(n) = T(n − 1) + O(n) → O(n²). Remedy: random pivot, median of three.

Basic Examples · Run the code step by step

Fast Exponentiation: Tracing 3¹³

long power(long x, int n) {
  if (n == 0) return 1;         // base case
  long y = power(x, n / 2);
  if (n % 2 == 0) return y * y; // even
  return x * y * y;             // odd
}

The naive method uses 12 multiplications, this one 7; for n = 10⁶, 999,999 versus 27 → O(log n): T(n) = T(n/2) + O(1).

n=13n=6n=3n=1n=03¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 13) is called → first power(3, 6) must be computed (n/2 = 6).

n=13n=6n=3n=1n=03¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 6) is called → first power(3, 3) must be computed (n/2 = 3).

n=13n=6n=3n=1n=03¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 3) is called → first power(3, 1) must be computed (n/2 = 1).

n=13n=6n=3n=1n=03¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 1) is called → first power(3, 0) must be computed (n/2 = 0).

n=13n=6n=3n=1n=0= 13¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

n = 0 → base case: returns 3⁰ = 1.

n=13n=6n=3n=1= 3n=0= 13¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 1): y = 1; 1 is odd → x·y·y = 3·1·1 = 3.

n=13n=6n=3= 27n=1= 3n=0= 13¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 3): y = 3; 3 is odd → x·y·y = 3·3·3 = 27.

n=13n=6= 729n=3= 27n=1= 3n=0= 13¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 6): y = 27; 6 is even → y·y = 27·27 = 729.

n=13= 1594323n=6= 729n=3= 27n=1= 3n=0= 13¹³ = 3·(3⁶)²; 3⁶ = (3³)²; 3³ = 3·(3¹)²

power(3, 13): y = 729; 13 is odd → x·y·y = 3·729·729 = 1594323.

Basic Examples

When Does It Fail? Overlapping Subproblems

F(5)F(4)F(3)F(2)F(1)F(0)F(1)F(2)F(1)F(0)F(3)F(2)F(1)F(0)F(1)
long fib(int n) {
  if (n < 2) return n;
  return fib(n - 1) + fib(n - 2);
}
  • 15 calls for F(5): F(3) is computed 2 times, F(2) 3 times.
  • The subproblems are not independent; they overlap → T(n) = T(n−1) + T(n−2) + O(1) = O(φⁿ) ≈ O(1.618ⁿ).
  • Fix: store the results → dynamic programming, O(n).
The Fibonacci recursion can be written in divide-and-conquer form and gives the right answer, but it is a bad example: divide and conquer is efficient only when the subproblems are independent.

Convex Hull

Convex Hull

  • Let S be a set of n points in the plane. The convex hull of S is the smallest convex polygon containing S.
  • Hammer a nail into each point and stretch a rubber band around them: the shape it takes is the hull.
  • The hull's vertices are points of S; the output is the list of these vertices in counterclockwise order.
  • Convex: the segment joining any two points inside lies entirely within the polygon.
010203001020
Uses: collision detection (games, robotics), geographic information systems, shape analysis in image processing, farthest pair of points, clustering.

Convex Hull

The Quickhull Algorithm

  1. The points with the smallest (A) and largest (B) x are on the hull.
  2. Line AB splits the points into two subsets: above and below it.
  3. In a subset, the point C farthest from AB is on the hull.
  4. Points inside triangle ACB are discarded: they cannot be on the hull.
  5. Recurse on the points outside AC and outside CB.
  6. Stop when no points remain; the points found form the hull.
010203001020AB

Steps 1–2: the leftmost A and the rightmost B are on the hull; line AB splits the points into two sets (upper / lower).

010203001020ABC

Steps 3–5: C (13,27), farthest from AB in the upper set, is on the hull. The 4 points inside ACB are discarded; recurse outside AC and CB.

010203001020

Step 6: recurse until no point is left outside → a 7-vertex hull.

Like quicksort (hence the name): the farthest point is the pivot, and the triangle's sides partition the rest.

Convex Hull

Geometry: Orientation Test and Distance

Quickhull needs only integer multiplication; no angles, square roots, or division:

long cross(Point A, Point B, Point P) {
  return (long)(B.x - A.x) * (P.y - A.y)
       - (long)(B.y - A.y) * (P.x - A.x);
}
  • cross > 0: P is to the left of A→B; < 0: to the right; = 0: on the line.
  • |cross| = 2 × (area of triangle ABP) = |AB| × distance. Since |AB| is fixed, farthest point = largest cross.
distanceABP: cross > 0 (left)Q: cross < 0 (right)
Example: A = (2,18), B = (35,11). For C = (13,27), cross = 33·9 − (−7)·11 = 374; for (28,23), 33·5 + 7·26 = 347. C has the largest value → C is the farthest point.

Convex Hull · Run the code step by step

Quickhull: Code Trace

void quickHull(List<Point> P) {
  Point A = leftmost(P), B = rightmost(P);
  hull.add(A); hull.add(B);
  findHull(leftOf(P, A, B), A, B); // upper set
  findHull(leftOf(P, B, A), B, A); // lower set
}
void findHull(List<Point> S, Point A, Point B) {
  if (S.isEmpty()) return; // AB: hull edge
  Point C = farthest(S, A, B); // max cross
  hull.add(C);
  findHull(leftOf(S, A, C), A, C); // outside AC
  findHull(leftOf(S, C, B), C, B); // outside CB
}  // inside ACB: dropped
010203001020AB

The leftmost A = (2,18) and the rightmost B = (35,11) are added to the hull.

010203001020AB

8 points lie left of AB (upper), 6 right of it (lower). The upper set comes first.

010203001020

|S| = 8: the farthest point C = (13,27) joins the hull. 4 points inside the triangle are discarded; 1 point lies outside AC, 2 outside CB.

010203001020

|S| = 1: the farthest point C = (4,25) joins the hull. 0 points inside the triangle are discarded; 0 outside AC, 0 outside CB.

010203001020

S is empty → (2,18)–(4,25) is a hull edge; return.

010203001020

S is empty → (4,25)–(13,27) is a hull edge; return.

010203001020

|S| = 2: the farthest point C = (28,23) joins the hull. 1 point inside the triangle is discarded; 0 outside AC, 0 outside CB.

010203001020

S is empty → (13,27)–(28,23) is a hull edge; return.

010203001020

S is empty → (28,23)–(35,11) is a hull edge; return.

010203001020

|S| = 6: the farthest point C = (20,2) joins the hull. 4 points inside the triangle are discarded; 0 outside AC, 1 outside CB.

010203001020

S is empty → (35,11)–(20,2) is a hull edge; return.

010203001020

|S| = 1: the farthest point C = (10,5) joins the hull. 0 points inside the triangle are discarded; 0 outside AC, 0 outside CB.

010203001020

S is empty → (20,2)–(10,5) is a hull edge; return.

010203001020

S is empty → (10,5)–(2,18) is a hull edge; return.

010203001020

Done: 7 vertices found, 9 points discarded. 12 findHull calls in total.

Convex Hull

Quickhull: A 104-Point Example

050100050100

104 points. The leftmost (-14,46) and rightmost (106,20) are joined; 71 points lie above, 31 below.

050100050100

Farthest from (-14,46)→(106,20): (54,120); 56 points inside the triangle discarded. Candidates left: 45.

050100050100

Farthest from (-14,46)→(54,120): (8,84); 2 points inside the triangle discarded. Candidates left: 42.

050100050100

Farthest from (-14,46)→(8,84): (-4,69); 0 points inside the triangle discarded. Candidates left: 41.

050100050100

Farthest from (8,84)→(54,120): (42,111); 1 point inside the triangle discarded. Candidates left: 39.

050100050100

Farthest from (54,120)→(106,20): (104,74); 6 points inside the triangle discarded. Candidates left: 32.

050100050100

Farthest from (54,120)→(104,74): (77,103); 0 points inside the triangle discarded. Candidates left: 31.

050100050100

Farthest from (106,20)→(-14,46): (11,-25); 25 points inside the triangle discarded. Candidates left: 5.

050100050100

Farthest from (106,20)→(11,-25): (88,-5); 4 points inside the triangle discarded. Candidates left: 0.

050100050100

Done: the hull of 104 points has 10 vertices. Most points are discarded at every step, so the work shrinks fast.

Convex Hull

Quickhull Analysis and Other Hull Algorithms

  • Each call scans S once (farthest point + two partitions): O(|S|).
  • T(n) = T(n₁) + T(n₂) + O(n), n₁ + n₂ ≤ n − 1.
  • Average (balanced splits, many points discarded): O(n log n); often even faster for uniformly random points.
  • Worst case: only one point is split off at each step → T(n) = T(n − 1) + O(n) = O(n²).
  • Output-sensitive bound: O(n·h) for a hull with h vertices.
AlgorithmTime
Brute force (every candidate edge)O(n³)
Jarvis march (gift wrapping)O(n·h)
Graham scanO(n log n)
Andrew's monotone chainO(n log n)
Quickhullavg. O(n log n), worst O(n²)
ChanO(n log h)
Lower bound: the hull can also sort (place the points on the parabola y = x²), so Ω(n log n) time is needed in the comparison model.

Closest Pair of Points

Closest Pair of Points

Problem: among n points in the plane, find the pair with the smallest Euclidean distance d(p, q) = √((pₓ − qₓ)² + (py − qy)²).

double bruteForce(List<Point> p) {
  double best = Double.MAX_VALUE;
  for (int i = 0; i < p.size(); i++)
    for (int j = i + 1; j < p.size(); j++)
      best = Math.min(best, dist(p.get(i), p.get(j)));
  return best;
}
  • Brute force: n(n − 1)/2 pairs → O(n²). For n = 10⁶, ≈ 5·10¹¹ distances.
  • Easy in one dimension: sort and compare only neighbors → O(n log n).
  • In two dimensions, neighbors in x order are not enough; divide and conquer achieves O(n log n).
← closest pair7 distances per point, 28 pairs in all
Applications: the two closest aircraft in air traffic control, clustering, collision detection, nearby facilities in geographic data.

Closest Pair of Points

Divide and Conquer: The Middle Strip

  1. Sort the points once by x and once by y.
  2. Divide: split into two halves with the vertical line L through the middle point.
  3. Conquer: recursively find δL and δR in the halves; δ = min(δL, δR).
  4. Combine: a closer pair must cross L → check only the strip points with |x − L| < δ.
  5. Scan the strip in y order; compare each point only with the following points whose y differs by less than δ.
L (x = median)pδδδleft halfright half
Why at most 7? Split the δ × 2δ rectangle above p into 8 squares of side δ/2. Two points in one square would lie in the same half at distance < δ, which is impossible → ≤ 1 point per square → p is compared with at most 7 points.

Closest Pair of Points · Run the code step by step

Closest Pair: Code Trace

double closest(List<Point> X, List<Point> Y) {
  int n = X.size(), mid = n / 2;
  if (n <= 3) return bruteForce(X);
  Point m = X.get(mid);  // YL/YR: the halves of Y
  double dL = closest(X.subList(0, mid), YL);
  double dR = closest(X.subList(mid, n), YR);
  double d = Math.min(dL, dR);
  List<Point> s = new ArrayList<>();  // strip
  for (Point p : Y)                   // in y order
    if (Math.abs(p.x - m.x) < d) s.add(p);
  for (int i = 0; i < s.size(); i++) {
    Point p = s.get(i);
    for (int j = i + 1; j < s.size()
         && s.get(j).y - p.y < d; j++)
      d = Math.min(d, dist(p, s.get(j)));
  }
  return d;
}
01020304002550x = 12

n = 10 > 3 → mid = 5, m = (12,10). Left: 5 points, right: 5.

01020304002550x = 5

n = 5 > 3 → mid = 2, m = (5,1). Left: 2 points, right: 3.

01020304002550d = 1.41

n = 2 ≤ 3 → brute force: (2,3), (3,4). Closest (2,3)–(3,4), d = 1.41.

01020304002550d = 6.32

n = 3 ≤ 3 → brute force: (5,1), (7,7), (12,30). Closest (5,1)–(7,7), d = 6.32.

01020304002550x = 5

Both halves returned: dL = 1.41, dR = 6.32 → d = 1.41.

01020304002550x = 5d = 1.41

Strip |x − 5| < 1.41: [(5,1)]; a single point → no comparisons. Returns 1.41.

01020304002550x = 27

n = 5 > 3 → mid = 2, m = (27,27). Left: 2 points, right: 3.

01020304002550d = 8.60

n = 2 ≤ 3 → brute force: (12,10), (17,17). Closest (12,10)–(17,17), d = 8.60.

01020304002550d = 1.00

n = 3 ≤ 3 → brute force: (27,27), (28,27), (40,50). Closest (27,27)–(28,27), d = 1.00.

01020304002550x = 27

Both halves returned: dL = 8.60, dR = 1.00 → d = 1.00.

01020304002550x = 27d = 1.00

Strip |x − 27| < 1.00: [(27,27)]; a single point → no comparisons. Returns 1.00.

01020304002550x = 12

Both halves returned: dL = 1.41, dR = 1.00 → d = 1.00.

01020304002550x = 12d = 1.00

Strip |x − 12| < 1.00: [(12,10), (12,30)]; the y gap exceeds d → no comparisons at all. Returns 1.00.

Closest Pair of Points

Closest Pair: Analysis

StepCostNote
Presort by x and by yO(n log n)only once, before the recursion
Divide (X in half, Y into YL/YR keeping the order)O(n)no re-sorting
Build the stripO(n)Y is already sorted by y
Scan the stripO(7n) = O(n)≤ 7 neighbors per point
TotalO(n log n)T(n) = 2T(n/2) + O(n)
Common pitfallRe-sorting the strip in every call: T(n) = 2T(n/2) + O(n log n) = O(n log² n).
In the examplethe closest pair of the 10 points is (27,27)–(28,27), d = 1.00. Brute force would compute 45 distances.

Karatsuba Multiplication

Karatsuba: 3 Multiplications Instead of 4

Split the n-digit numbers x, y at m = ⌊n/2⌋: x = a·10m + b, y = c·10m + d.

Naive divide and conquerx·y = ac·102m + (ad + bc)·10m + bd
4 products: ac, ad, bc, bd
T(n) = 4T(n/2) + O(n) = O(n²) → no gain!
Karatsuba (1962)(a + b)(c + d) = ac + (ad + bc) + bd
⇒ ad + bc = (a + b)(c + d) − ac − bd
3 products: z2 = ac, z0 = bd, z1 = (a + b)(c + d)
T(n) = 3T(n/2) + O(n) = O(n1.585)
x1234aby5678cd→z2 = a·c672z1 = (a+b)(c+d)6164z0 = b·d2652
Example: 1234 · 5678 (m = 2, 10m = 100)
a = 12, b = 34, c = 56, d = 78z2 = 12·56 = 672 · z0 = 34·78 = 2652
z1 = (12 + 34)(56 + 78) = 46·134 = 6164z1 − z2 − z0 = 6164 − 672 − 2652 = 2840 (= ad + bc)
672·10⁴ + 2840·10² + 2652 = 6,720,000 + 284,000 + 2652 = 7,006,652 ✓

Karatsuba Multiplication · Run the code step by step

Karatsuba: Tracing 1234 · 5678

1234·5678

karatsuba(1234, 5678): n = 4, p = 100 → a = 12, b = 34, c = 56, d = 78. Three calls: (34, 78), (46, 134), (12, 56).

1234·567834·78

karatsuba(34, 78): n = 2, p = 10 → a = 3, b = 4, c = 7, d = 8. Three calls: (4, 8), (7, 15), (3, 7).

1234·567834·784·8= 32

4 · 8: a one-digit factor → base case, directly 32.

1234·567834·784·8= 327·15= 105

7 · 15: a one-digit factor → base case, directly 105.

1234·567834·784·8= 327·15= 1053·7= 21

3 · 7: a one-digit factor → base case, directly 21.

1234·567834·78 = 26524·8= 327·15= 1053·7= 21

21·100 + (105 − 21 − 32)·10 + 32 = 2100 + 520 + 32 = 2652.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·134

karatsuba(46, 134): n = 3, p = 10 → a = 4, b = 6, c = 13, d = 4. Three calls: (6, 4), (10, 17), (4, 13).

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·1346·4= 24

6 · 4: a one-digit factor → base case, directly 24.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·1346·4= 2410·17

karatsuba(10, 17): n = 2, p = 10 → a = 1, b = 0, c = 1, d = 7. Three calls: (0, 7), (1, 8), (1, 1).

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·1346·4= 2410·170·7= 0

0 · 7: a one-digit factor → base case, directly 0.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·1346·4= 2410·170·7= 01·8= 8

1 · 8: a one-digit factor → base case, directly 8.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·1346·4= 2410·170·7= 01·8= 81·1= 1

1 · 1: a one-digit factor → base case, directly 1.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·1346·4= 2410·17 = 1700·7= 01·8= 81·1= 1

1·100 + (8 − 1 − 0)·10 + 0 = 100 + 70 + 0 = 170.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·1346·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 52

4 · 13: a one-digit factor → base case, directly 52.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·134 = 61646·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 52

52·100 + (170 − 52 − 24)·10 + 24 = 5200 + 940 + 24 = 6164.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·134 = 61646·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 5212·56

karatsuba(12, 56): n = 2, p = 10 → a = 1, b = 2, c = 5, d = 6. Three calls: (2, 6), (3, 11), (1, 5).

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·134 = 61646·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 5212·562·6= 12

2 · 6: a one-digit factor → base case, directly 12.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·134 = 61646·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 5212·562·6= 123·11= 33

3 · 11: a one-digit factor → base case, directly 33.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·134 = 61646·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 5212·562·6= 123·11= 331·5= 5

1 · 5: a one-digit factor → base case, directly 5.

1234·567834·78 = 26524·8= 327·15= 1053·7= 2146·134 = 61646·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 5212·56 = 6722·6= 123·11= 331·5= 5

5·100 + (33 − 5 − 12)·10 + 12 = 500 + 160 + 12 = 672.

1234·5678 = 700665234·78 = 26524·8= 327·15= 1053·7= 2146·134 = 61646·4= 2410·17 = 1700·7= 01·8= 81·1= 14·13= 5212·56 = 6722·6= 123·11= 331·5= 5

672·10000 + (6164 − 672 − 2652)·100 + 2652 = 6720000 + 284000 + 2652 = 7006652.

long karatsuba(long x, long y) {
  if (x < 10 || y < 10) return x * y; // base case
  int n = Math.max(digits(x), digits(y));
  long p = (long) Math.pow(10, n / 2);
  long a = x / p, b = x % p;   // x = a·p + b
  long c = y / p, d = y % p;   // y = c·p + d
  long z0 = karatsuba(b, d);
  long z1 = karatsuba(a + b, c + d);
  long z2 = karatsuba(a, c);
  return z2 * p * p + (z1 - z2 - z0) * p + z0;
}

Karatsuba Multiplication

Karatsuba Analysis

  • T(n) = 3T(n/2) + O(n) → case 1 → Θ(nlog₂3) ≈ Θ(n1.585).
  • Addition, subtraction, and shifting by 10m cost O(n); cutting the products from 4 to 3 lowers the exponent from 2 to 1.585.
  • The example makes 16 calls and 11 base-case products (the schoolbook method needs 16 one-digit products).
  • Slower on small numbers because of the extra additions → use the schoolbook method below a threshold.
  • Java's BigInteger.multiply uses Karatsuba and Toom-Cook for large numbers. Faster still: Schönhage–Strassen O(n log n log log n), Harvey–van der Hoeven O(n log n) (2019).
n (digits)n²n1.585ratio
10100382.6×
10010,0001,4796.8×
1,0001.0·10656,87117.6×
10,0001.0·1082.2·10645.7×
1.0·1061.0·10123.2·109309.2×

Strassen Multiplication

Matrix Multiplication: Classic and Block

Classic: Θ(n³)
for (int i = 0; i < n; i++)
  for (int j = 0; j < n; j++)
    for (int k = 0; k < n; k++)
      C[i][j] += A[i][k] * B[k][j];

n² cells, each needing n products → n³ products.

C11C12C21C22=A11A12A21A22×B11B12B21B22C11 = A11·B11 + A12·B21: 2 block products
Block divide and conquer

Split A, B, and C into four n/2 × n/2 blocks:

C11 = A11·B11 + A12·B21
C12 = A11·B12 + A12·B22
C21 = A21·B11 + A22·B21
C22 = A21·B12 + A22·B22
8 block products + 4 block additions: T(n) = 8T(n/2) + Θ(n²) → nlog₂8 = Θ(n³). Splitting alone is not enough; the number of products must go down.

Strassen Multiplication

Strassen (1969): 7 Products Instead of 8

ProductDefinition
P1A11 · (B12 − B22)
P2(A11 + A12) · B22
P3(A21 + A22) · B11
P4A22 · (B21 − B11)
P5(A11 + A22) · (B11 + B22)
P6(A12 − A22) · (B21 + B22)
P7(A11 − A21) · (B11 + B12)
Result blocks
C11 = P5 + P4 − P2 + P6
C12 = P1 + P2
C21 = P3 + P4
C22 = P5 + P1 − P3 − P7

7 products + 18 block additions/subtractions. The blocks are themselves multiplied with Strassen recursively; the base case is 1×1.

In C22, P7 is subtracted. Writing "+ P7" makes the program silently produce a wrong matrix; checking the implementation against the classic method on a small example catches the bug.

Strassen Multiplication

Strassen Step by Step: A 2 × 2 Example

A =
13
75
×
B =
68
42
P1A11 · (B12 − B22)1 · (8 − 2)6
P2(A11 + A12) · B22…
P3(A21 + A22) · B11…
P4A22 · (B21 − B11)…
P5(A11 + A22) · (B11 + B22)…
P6(A12 − A22) · (B21 + B22)…
P7(A11 − A21) · (B11 + B12)…

P1 = 1 · (8 − 2) = 6.

P1A11 · (B12 − B22)1 · (8 − 2)6
P2(A11 + A12) · B22(1 + 3) · 28
P3(A21 + A22) · B11…
P4A22 · (B21 − B11)…
P5(A11 + A22) · (B11 + B22)…
P6(A12 − A22) · (B21 + B22)…
P7(A11 − A21) · (B11 + B12)…

P2 = (1 + 3) · 2 = 8.

P1A11 · (B12 − B22)1 · (8 − 2)6
P2(A11 + A12) · B22(1 + 3) · 28
P3(A21 + A22) · B11(7 + 5) · 672
P4A22 · (B21 − B11)…
P5(A11 + A22) · (B11 + B22)…
P6(A12 − A22) · (B21 + B22)…
P7(A11 − A21) · (B11 + B12)…

P3 = (7 + 5) · 6 = 72.

P1A11 · (B12 − B22)1 · (8 − 2)6
P2(A11 + A12) · B22(1 + 3) · 28
P3(A21 + A22) · B11(7 + 5) · 672
P4A22 · (B21 − B11)5 · (4 − 6)−10
P5(A11 + A22) · (B11 + B22)…
P6(A12 − A22) · (B21 + B22)…
P7(A11 − A21) · (B11 + B12)…

P4 = 5 · (4 − 6) = −10.

P1A11 · (B12 − B22)1 · (8 − 2)6
P2(A11 + A12) · B22(1 + 3) · 28
P3(A21 + A22) · B11(7 + 5) · 672
P4A22 · (B21 − B11)5 · (4 − 6)−10
P5(A11 + A22) · (B11 + B22)(1 + 5) · (6 + 2)48
P6(A12 − A22) · (B21 + B22)…
P7(A11 − A21) · (B11 + B12)…

P5 = (1 + 5) · (6 + 2) = 48.

P1A11 · (B12 − B22)1 · (8 − 2)6
P2(A11 + A12) · B22(1 + 3) · 28
P3(A21 + A22) · B11(7 + 5) · 672
P4A22 · (B21 − B11)5 · (4 − 6)−10
P5(A11 + A22) · (B11 + B22)(1 + 5) · (6 + 2)48
P6(A12 − A22) · (B21 + B22)(3 − 5) · (4 + 2)−12
P7(A11 − A21) · (B11 + B12)…

P6 = (3 − 5) · (4 + 2) = −12.

P1A11 · (B12 − B22)1 · (8 − 2)6
P2(A11 + A12) · B22(1 + 3) · 28
P3(A21 + A22) · B11(7 + 5) · 672
P4A22 · (B21 − B11)5 · (4 − 6)−10
P5(A11 + A22) · (B11 + B22)(1 + 5) · (6 + 2)48
P6(A12 − A22) · (B21 + B22)(3 − 5) · (4 + 2)−12
P7(A11 − A21) · (B11 + B12)(1 − 7) · (6 + 8)−84

P7 = (1 − 7) · (6 + 8) = −84.

C11 = P5 + P4 − P2 + P648 + (−10) − 8 + (−12)18
C12 = P1 + P26 + 814
C21 = P3 + P472 + (−10)62
C22 = P5 + P1 − P3 − P748 + 6 − 72 − (−84)66

C = [[18, 14], [62, 66]]. Check: 1·6 + 3·4 = 18, 7·8 + 5·2 = 66 ✓. 7 products used (classic: 8).

Strassen Multiplication

Strassen Analysis and Practice

  • T(n) = 7T(n/2) + Θ(n²) → case 1 → Θ(nlog₂7) ≈ Θ(n2.807).
  • Scalar products for n = 2k: 7k (classic: 8k). For n = 1024: 7¹⁰ ≈ 2.8·10⁸ vs. 8¹⁰ ≈ 1.07·10⁹ → about 3.8 times fewer.
  • If n is not a power of 2, the matrix is padded with zeros.
  • Threshold: 18 extra additions and temporary matrices make it slow for small n; libraries switch to the classic method for n ≲ 64–128.
  • Numerical stability: with floating-point numbers, the subtractions can amplify errors.
  • Memory: temporary blocks at every level → extra O(n²).
  • The theoretical record is ω < 2.372 (2024); the constants are huge, so it is not used in practice.
Lesson: trading a product for additions lowers the exponent by reducing a by one. Same idea in Karatsuba (4 → 3) and Strassen (8 → 7).

Wrap-Up

Naive Method vs. Divide and Conquer

ProblemNaive methodDivide and conquerRecurrence
SortingO(n²) (bubble sort)O(n log n)2T(n/2) + n
Search in a sorted arrayO(n)O(log n)T(n/2) + 1
Exponentiation xⁿO(n) productsO(log n)T(n/2) + 1
Convex hullO(n³)avg. O(n log n)T(n₁) + T(n₂) + n
Closest pair of pointsO(n²)O(n log n)2T(n/2) + n
Integer multiplicationO(n²)O(n1.585)3T(n/2) + n
Matrix multiplicationO(n³)O(n2.807)7T(n/2) + n²

Where the gain comes from: either the work is split evenly and each level is combined in O(n) (sorting, closest pair), or the number of subproblems is reduced (Karatsuba, Strassen), or half is thrown away (search, exponentiation).

Summary

Summary

TopicKey points
Paradigmdivide → (break down) → conquer → combine; independent subproblems of the same kind; a base case is required
AnalysisT(n) = aT(n/b) + f(n); recursion tree; master theorem: compare f(n) with nlogb a
Basic examplesbinary search and fast exponentiation O(log n); merge sort O(n log n); quicksort avg. O(n log n), worst O(n²)
Quickhullendpoints A, B; farthest C (max cross); points inside the triangle discarded; avg. O(n log n), worst O(n²)
Closest pairsplit by x; δ = min(δL, δR); ≤ 7 neighbors in the 2δ strip; O(n log n)
Karatsubaad + bc = (a + b)(c + d) − ac − bd; 3 products → O(n1.585)
Strassen7 block products → O(n2.807); C22 = P5 + P1 − P3 − P7

Summary

Common Mistakes

Forgetting the base caseOr writing one that is never reached: infinite recursion, StackOverflowError. In Karatsuba: x < 10 || y < 10.
Unbalanced splitsThe last element as pivot on a sorted array, or a single point per step in Quickhull: T(n) = T(n − 1) + n = O(n²).
Overlapping subproblemsPlain recursive Fibonacci takes exponential time; store results with dynamic programming.
Underestimating the combine costSorting the strip in every call gives O(n log² n); 2T(n/2) + n log n ≠ Θ(n log n).
Misapplying the master theoremIt does not apply to T(n) = T(n − 1) + n, to unequal splits, or to f(n) = n/log n.
Sign errorsIn Strassen, C22 = P5 + P1 − P3 − P7; a single sign error silently produces a wrong matrix. Comparing with the classic method on a small example catches it.

Summary

Sample Problems

  1. Solve T(n) = 4T(n/2) + n with the master theorem.
  2. Solve T(n) = 2T(n/2) + n²; check the regularity condition.
  3. Compute 21 · 43 with Karatsuba (m = 1).
  4. A = (0,0), B = (10,0). Find cross(A, B, ·) for P = (3,4), Q = (5,−2), R = (7,6). Which point does Quickhull choose in the upper set?
  5. How many multiplications does power(3, 10) perform? Compare with the naive method.
  6. When merge sort sorts [5, 1, 4, 2, 3], which two sorted arrays are merged in the final merge?
Show answers
  1. a = 4, b = 2 → nlog₂4 = n²; f(n) = n is smaller → case 1 → Θ(n²).
  2. nlog₂2 = n; f(n) = n² is larger; 2·(n/2)² = n²/2 ≤ ½·n² ✓ → case 3 → Θ(n²).
  3. a = 2, b = 1, c = 4, d = 3: z2 = 8, z0 = 3, z1 = 3·7 = 21 → 21 − 8 − 3 = 10 → 800 + 100 + 3 = 903.
  4. P: 10·4 = 40 (left), Q: 10·(−2) = −20 (right), R: 10·6 = 60 (left). Largest cross in the upper set → R.
  5. n: 10 → 5 → 2 → 1 → 0. Multiplications: n=1: 2, n=2: 1, n=5: 2, n=10: 1 → 6 (naive: 9). Result 59049.
  6. mid = 2: [5, 1, 4] → [1, 4, 5] and [2, 3] → [1, 4, 5] + [2, 3] → [1, 2, 3, 4, 5].

Summary

Exercises

  1. Find the maximum element of an array with divide and conquer; solve the recurrence.
  2. Modify merge sort to count inversions (i < j, a[i] > a[j]). What is the result for [2, 4, 1, 3, 5]?
  3. Maximum subarray: give a divide-and-conquer solution for [−2, 1, −3, 4, −1, 2, 1, −5, 4] and its running time.
  4. Compute A = [[1, 2], [3, 4]] times B = [[5, 6], [7, 8]] with Strassen; try C22 with ± P7.
  5. Describe an input on which Quickhull takes O(n²) time.
  6. What would change in the closest pair algorithm if the base case were n ≤ 1 instead of n ≤ 3?
Answers / hints
  1. max(left half, right half); T(n) = 2T(n/2) + O(1) → case 1 → Θ(n).
  2. When the merge takes an element from the right, add the count of elements still in the left half; O(n log n). Result 3.
  3. max(left, right, crossing the middle); 2T(n/2) + O(n) = O(n log n). Result 6 ([4, −1, 2, 1]).
  4. P1..P7 = −2, 24, 35, 8, 65, −30, −22 → C = [[19, 22], [43, 50]]; C22 = 65 − 2 − 35 + 22 = 50 ✓ (6 with +P7 ✗).
  5. Points on a convex curve where the farthest point is always next to one end (exponentially spaced): T(n) = T(n − 1) + O(n).
  6. A one-point half returns ∞; the result is still correct (the strip finds the missed pair), but the number of calls grows.

Summary

References

  • Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, 4th ed., 2022 (Chapter 4: Divide-and-Conquer; Chapter 33: Computational Geometry).
  • Kleinberg and Tardos, Algorithm Design, 2005 (Chapter 5: Divide and Conquer).
  • Sedgewick and Wayne, Algorithms, 4th ed., 2011.
  • Barber, Dobkin, and Huhdanpaa, "The Quickhull Algorithm for Convex Hulls", ACM TOMS 22(4), 1996.
  • Karatsuba and Ofman, "Multiplication of Multidigit Numbers on Automata", 1962; Strassen, "Gaussian Elimination is not Optimal", Numerische Mathematik 13, 1969.

The End

Chapter 1: Divide and Conquer
Next chapter: Chapter 2: Greedy Algorithms
1 / 1 Sercan KÜLCÜ. All rights reserved.