Chapter 5: Branch and Bound

Chapter 5: Branch and Bound · Study Questions
Branching, bounding, pruningValid boundFIFO, LIFO, best-first0/1 knapsackTraveling salesmanAssignment problemInteger programmingMaximum clique
Sercan KÜLCÜ · Giresun University · Computer Engineering

Multiple Choice · Paradigm

Question 1: How Branch and Bound Differs

Which pruning rule distinguishes branch and bound from backtracking (maximization)?

B: Backtracking uses only the infeasibility and completeness rules. Branch and bound cuts a node even if it is feasible when bound(u) ≤ best: no solution below it can beat the best one. A TSP or clique search without the bound test is really backtracking.

Multiple Choice · Bound

Question 2: A Valid Bound

In a maximization problem, what does it mean for the value bound(u) computed for node u to be valid, and what happens if it is not?

C: The bound must be optimistic: no solution in the subtree can exceed it. If the bound falls below the true value, the branch containing the optimum may be pruned as "not good enough" and the result becomes wrong. A tight and cheap bound is a matter of efficiency; it does not have to equal the true optimum (the problem would already be solved then).

Multiple Choice · Bound

Question 3: A Pessimistic Bound

A student computes the knapsack bound as "add the profit of the remaining items that fit completely, do not add the fractional part". Which statement is true for this bound?

D: This value is the profit of a greedy 0/1 solution, that is, a lower bound: it can fall below the true optimum. In the example (p, w) = (20, 2), (30, 5), (35, 7), (12, 3), W = 9 it gives 20 + 30 = 50 at the root, but the optimum is 55; when best is 50, this "bound" cuts the branch of the optimum. An upper bound must be an optimistic value such as the fractional relaxation.

Multiple Choice · Search Strategies

Question 4: A Memory-Limited Strategy

Which search strategy should be chosen for branch and bound on a system with very limited memory?

A: LIFO (DFS) keeps only the siblings along the current path: memory is O(depth · branching). With FIFO and best-first, the number of live nodes can grow exponentially. Best-first usually expands the fewest nodes but its queue eats memory; LIFO, with an early complete solution, also raises best quickly.

Output Question · Search Strategies

Question 5: FIFO Branch and Bound

The knapsack example from the lecture: (p, w) = (40, 2), (30, 5), (50, 10), (10, 5), W = 16. The priority queue in the best-first code is replaced by an ArrayDeque using add()/poll() (FIFO). What does the program print (best, generated, expanded)? The root counts as generated (generated starts at 1).

Deque<Node> queue = new ArrayDeque<>();
queue.add(root);
while (!queue.isEmpty()) {
  Node u = queue.poll();        // first in
  if (u.bound <= best || u.level == n - 1)
    continue;
  expanded++;
  // take and skip: same code (generated += 2)
}
System.out.println(
    best + " " + generated + " " + expanded);
Show answer

Output: 90 13 6

  • FIFO expands nodes level by level (BFS): first the two children of the root, then their children…
  • Best-first generates 11 nodes and expands 5 of them in the same example; FIFO opens more nodes because it does not wait for the node with the largest bound.
  • The optimum is still 90 (items 1 and 3): the strategy changes only the amount of work and the memory.

Step-by-Step Trace · 0/1 Knapsack

Question 6: Tracing bound(root)

Items (p, w): (20, 2), (30, 5), (35, 7), (12, 3); W = 9. How is the fractional bound computed for the root node?

W = 9result = 0

Items in ratio order: (20, 2) 10 · (30, 5) 6 · (35, 7) 5 · (12, 3) 4; W = 9. Root: level −1, profit 0, weight 0 ≤ 9 → feasible.

W = 9result = 0

The bound starts from the current profit; j = level + 1 = 0 (item 1).

W = 9i1result = 20

0 + w[0] = 2 ≤ 9 → item 1 fits completely: totW = 2, result = 20.

W = 9i1i2result = 50

2 + w[1] = 7 ≤ 9 → item 2 fits too: totW = 7, result = 50.

W = 9i1i2result = 50

7 + w[2] = 14 > 9 → item 3 does not fit completely, the loop ends.

W = 9i1i2i3·2/7result = 60

Remaining capacity 2: 2/7 of item 3 → 2 · 35/7 = 10 is added, result = 60.

W = 9i1i2i3·2/7result = 60

bound(root) = 60: no 0/1 solution can earn more than 60. The true optimum is 55 (item 1 + 3, weight 9).

double bound(Node u) {
  if (u.weight > W) return 0;    // infeasible
  double result = u.profit;
  int totW = u.weight;
  int j = u.level + 1;
  while (j < n && totW + w[j] <= W) {
    totW += w[j];
    result += p[j];
    j++;
  }
  if (j < n)                      // fractional
    result += (W - totW)
             * (double) p[j] / w[j];
  return result;
}

Output Question · 0/1 Knapsack

Question 7: Bounds of Other Nodes

Items (p, w): (20, 2), (30, 5), (35, 7), (12, 3); W = 9. Compute the bounds of these nodes: C = "item 1 skipped" (level 0, profit 0, weight 0) and E = "item 1 taken, item 2 skipped" (level 1, profit 20, weight 2). What are the greedy choice by ratio and the optimum; is C pruned?

Show answer
  • bound(C) = 50: item 2 fits (30, weight 5); item 3 does not fit, 4 units remain → 4 · 35/7 = 20 → 30 + 20 = 50.
  • bound(E) = 55: 20 + item 3 (35, weight 9) = 55; the capacity is exactly full, the fraction for item 4 is 0.
  • Greedy (ratio order) takes 1 and 2: 50 (weight 7); 3 and 4 do not fit. The optimum is 1 + 3 = 55.
  • After best = 55 is found, bound(C) = 50 ≤ 55 → C is pruned: without item 1 no solution can exceed 55.

Step-by-Step Trace · 0/1 Knapsack

Question 8: Best-First Branch and Bound

Items (p, w): (20, 2), (30, 5), (35, 7), (12, 3); W = 9. Which nodes does best-first (LC) branch and bound expand, and in what order?

A60

Root A: profit 0, weight 0, bound = 60. Queue (largest bound first): A (60); best = 0. The number next to each label is the bound.

A60B60C50

A is removed (largest bound) and branched; best = 20. B (item 1 taken): b = 60 → queued; C (item 1 skipped): b = 50 → queued.

A60B60C50D60E55

B is removed (largest bound) and branched; best = 50. D (item 2 taken): b = 60 → queued; E (item 2 skipped): b = 55 → queued.

A60B60C50D60E55F✗G58

D is removed (largest bound) and branched; best = 50. F (item 3 taken): weight 14 > 9 → infeasible; G (item 3 skipped): b = 58 → queued.

A60B60C50D60E55F✗G58H✗I50

G is removed (largest bound) and branched; best = 50. H (item 4 taken): weight 10 > 9 → infeasible; I (item 4 skipped): b = 50 ≤ 50 → pruned.

A60B60C50D60E55F✗G58H✗I50J55K32

E is removed (largest bound) and branched; best = 55. J (item 3 taken): b = 55 ≤ 55 → pruned; K (item 3 skipped): b = 32 ≤ 55 → pruned.

A60B60C50D60E55F✗G58H✗I50J55K32

C is removed: bound = 50 ≤ best = 55 → pruned (it went stale while waiting). Queue empty: the best profit is 55. Of the 31 nodes of the full tree, 11 were generated and 5 expanded.

Node u = queue.poll(); // largest bound
if (u.bound <= best || u.level == n - 1)
  continue;
int i = u.level + 1;
Node take = new Node(i, u.profit + p[i],
                     u.weight + w[i]);
if (take.weight <= W && take.profit > best)
  best = take.profit;
take.bound = bound(take);
if (take.bound > best) queue.add(take);
Node skip = new Node(i, u.profit,
                       u.weight);
skip.bound = bound(skip);
if (skip.bound > best) queue.add(skip);

Multiple Choice · 0/1 Knapsack

Question 9: Test Again After Removal

Why is the test if (u.bound <= best) continue; done after poll() in the best-first code? The node was already tested when it was added to the queue.

B: The bound does not change; best does. In the knapsack example with W = 9, best was 20 when node C (bound 50) was added; when it was removed, best had become 55 and C was discarded without being expanded. Without the test the result is still correct, but unnecessary nodes are expanded.

Step-by-Step Trace · Traveling Salesman

Question 10: Tracing TSP Branch and Bound

Symmetric distances: d(0,1) = 4, d(0,2) = 9, d(0,3) = 5, d(1,2) = 3, d(1,3) = 7, d(2,3) = 6. Which branches does tsp(0, 0, 1) open and which does it prune?

4953760123

Start at city 0. The cheapest outgoing edge of each city is minOut = [4, 3, 3, 5]; the lower bound of the root is 4 + 3 + 3 + 5 = 15.

4953760123

0 → 1: next = 4; lb = 15 < ∞ → branch.

4953760123

1 → 2: next = 7; lb = 15 < ∞ → branch.

4953760123

2 → 3: next = 13; lb = 18 < ∞ → branch.

4953760123

Tour 0 → 1 → 2 → 3 → 0 = 18 < ∞ → best = 18.

4953760123

1 → 3: next = 11; lb = 19 ≥ best = 18 → pruned.

4953760123

0 → 2: next = 9; lb = 20 ≥ best = 18 → pruned.

4953760123

0 → 3: next = 5; lb = 16 < 18 → branch.

4953760123

3 → 1: next = 12; lb = 18 ≥ best = 18 → pruned.

4953760123

3 → 2: next = 11; lb = 17 < 18 → branch.

4953760123

2 → 1: next = 14; lb = 17 < 18 → branch.

4953760123

Tour 0 → 3 → 2 → 1 → 0 = 18 = best; not updated. Done: the shortest tour is 18 (0 → 1 → 2 → 3 → 0).

void tsp(int city, int cost, int count) {
  if (count == n) { // tour complete
    int tour = cost + d[city][0];
    if (tour < best) best = tour;
    return;
  }
  for (int s = 0; s < n; s++) {
    if (visited[s]) continue;
    int next = cost + d[city][s];
    visited[s] = true;
    if (lowerBound(s, next) < best)  // bound
      tsp(s, next, count + 1);
    visited[s] = false; // undo
  }
}

Multiple Choice · Traveling Salesman

Question 11: Validity of the TSP Lower Bound

Why is the bound lb(path) = cost(path) + minOut[last city] + Σ minOut[unvisited] valid?

D: To complete the tour, the last city and every unvisited city are left exactly once; each exit costs at least the cheapest edge of that city. So no completed tour can be cheaper than lb(path). The bound does not have to equal the true value; A is the nearest-neighbor heuristic and gives an upper value.

Classic Question · Traveling Salesman

Question 12: The Two Shortest Edges Bound

Symmetric distances: d(0,1) = 4, d(0,2) = 9, d(0,3) = 5, d(1,2) = 3, d(1,3) = 7, d(2,3) = 6. For this 4-city example, compute the root lower bound "half of the sum of the two shortest edges of each city". Compare it with the bound (15) based on the sum of the cheapest outgoing edges of each city.

Show answer
citytwo shortest edgessum
04, 59
13, 47
23, 69
35, 611

(9 + 7 + 9 + 11) / 2 = 36 / 2 = 18. Each city is entered by one edge and left by one edge; since each edge is counted at two cities, the total is divided by 2.

This bound equals the optimum (18): even at the root, all branches can be pruned once the first tour is found. The cheapest-outgoing-edge bound, 15, is looser. A tight bound prunes more but is a little more expensive at every node.

Step-by-Step Trace · Assignment Problem

Question 13: Tracing the Assignment Problem

Cost matrix [[9, 2, 7], [6, 4, 3], [5, 8, 1]] (row: person, column: job). Which branches does assign(0, 0) open and which does it prune?

927p0643p1581p2job 0job 1job 2

The cost of job j for person i is c[i][j]. The row minima are 2, 3, 1 → restMin = [6, 4, 1, 0] (from row i to the end). Lower bound of the root: 6.

927p0643p1581p2job 0job 1job 2

Person 0 → job 0: next = 9, lb = 9 + restMin[1] = 13 < ∞ → branch.

927p0643p1581p2job 0job 1job 2

Person 1 → job 1: next = 13, lb = 13 + restMin[2] = 14 < ∞ → branch.

927p0643p1581p2job 0job 1job 2

Person 2 → job 2: next = 14, lb = 14 + restMin[3] = 14 < ∞ → branch.

927p0643p1581p2job 0job 1job 2

Complete assignment [0, 1, 2], cost 14 → best = 14.

927p0643p1581p2job 0job 1job 2

Person 1 → job 2: next = 12, lb = 12 + restMin[2] = 13 < 14 → branch.

927p0643p1581p2job 0job 1job 2

Person 2 → job 1: lb = 20 ≥ best = 14 → pruned.

927p0643p1581p2job 0job 1job 2

Person 0 → job 1: next = 2, lb = 2 + restMin[1] = 6 < 14 → branch.

927p0643p1581p2job 0job 1job 2

Person 1 → job 0: next = 8, lb = 8 + restMin[2] = 9 < 14 → branch.

927p0643p1581p2job 0job 1job 2

Person 2 → job 2: next = 9, lb = 9 + restMin[3] = 9 < 14 → branch.

927p0643p1581p2job 0job 1job 2

Complete assignment [1, 0, 2], cost 9 → best = 9.

927p0643p1581p2job 0job 1job 2

Person 1 → job 2: next = 5, lb = 5 + restMin[2] = 6 < 9 → branch.

927p0643p1581p2job 0job 1job 2

Person 2 → job 0: lb = 10 ≥ best = 9 → pruned.

927p0643p1581p2job 0job 1job 2

Person 0 → job 2: lb = 11 ≥ best = 9 → pruned. Done: the best assignment is [1, 0, 2] (2 + 6 + 1), cost 9.

void assign(int i, int cost) { // person i
  if (i == n) {
    best = Math.min(best, cost);
    return;
  }
  for (int j = 0; j < n; j++) {
    if (taken[j]) continue;
    int next = cost + c[i][j];
    if (next + restMin[i + 1] >= best)
      continue; // bound
    taken[j] = true;
    assign(i + 1, next);
    taken[j] = false; // undo
  }
}

Output Question · Assignment Problem

Question 14: 4 × 4 Assignment

The assign method that works with branch and bound (a branch is cut if next + restMin[i + 1] ≥ best; a call counter and a complete-assignment counter were added) is run with the matrix below. What is the root lower bound (restMin[0]) and what does the program print?

int[][] c = {{7, 3, 8, 4},
             {5, 6, 2, 9},
             {4, 8, 5, 3},
             {6, 2, 7, 5}};
// restMin: suffix sums of the row minima
assign(0, 0);
System.out.println(restMin[0] + " " + best
    + " " + calls + " " + full);
Show answer

Output: 10 12 21 4 (restMin[0], best, calls, complete assignments).

  • The row minima are 3, 2, 3, 2 → root lower bound 10. The bound is valid: each person costs at least their own cheapest job.
  • Best assignment: person 0 → job 3 (4), 1 → job 2 (2), 2 → job 0 (4), 3 → job 1 (2) = 12.
  • Brute force tries 4! = 24 complete assignments; branch and bound reaches only 4 complete assignments in 21 calls.

Classic Question · Integer Programming

Question 15: ILP Branch and Bound Tree

Maximize Z = 5x + 8y; x + y ≤ 6, 5x + 9y ≤ 45, x, y ≥ 0 integer. Apply branch and bound with LP relaxations (branch on y first, then on x if it is fractional). What is the optimum? Why is rounding the LP solution not enough?

Show answer
nodeLP solution (x; y)Zdecision
root(2.25; 3.75)41.25y fractional → y ≤ 3 | y ≥ 4
y ≤ 3(3; 3)39integer → best = 39
y ≥ 4(1.8; 4)41x fractional → x ≤ 1 | x ≥ 2
y ≥ 4, x ≤ 1(1; 4.44)40.56y fractional → y ≤ 4 | y ≥ 5
… y ≤ 4(1; 4)3737 ≤ 39 → pruned
… y ≥ 5(0; 5)40integer, 40 > 39 → best = 40
y ≥ 4, x ≥ 2——infeasible

The optimum is x = 0, y = 5, Z = 40. Rounding: (2; 4) is infeasible (5·2 + 9·4 = 46 > 45), and (2; 3) gives Z = 34; the true optimum is far from the LP point. The LP value (41.25) is an upper bound for every subtree.

Multiple Choice · Integer Programming

Question 16: Branching on a Fractional Variable

At an ILP node the solution of the LP relaxation gives y = 2.6. Which two subproblems does branch and bound create, and why is no integer solution lost?

A: Branching discards the fractional region (2, 3); all integer points stay in one of the two subproblems. B loses the remaining values of y; C leaves the current LP solution in both subproblems (no progress); in D the subproblems overlap. The LP value of each subproblem is an upper bound for that branch.

Output Question · Maximum Clique

Question 17: What the Clique Bound Gains

Edges 0–1, 0–2, 1–2, 1–3, 2–3, 3–4, 2–4, 1–4. The clique method from the lecture is run with the bound line and with the bound line deleted; a counter increases at every call. What are the maximum clique and the call counts of the two versions?

01234
Show answer
  • Maximum clique: {1, 2, 3, 4} (4 vertices; every pair is adjacent).
  • Without the bound (backtracking): 20 calls; with the |K| + |cand| ≤ |best| bound: 15 calls.
  • After best = {1, 2, 3, 4} is found, branches such as K = {2}, cand = {3, 4} are cut immediately because 1 + 2 ≤ 4: no clique larger than 4 can come out of this branch.

Multiple Choice · Maximum Clique

Question 18: A Problem That Does Not Need Branch and Bound

For which of the following problems is branch and bound not needed?

C: Maximum flow has polynomial-time algorithms (Ford-Fulkerson, Edmonds-Karp). Branch and bound makes sense for NP-hard optimization problems (TSP, 0/1 knapsack, maximum clique, ILP); in the worst case it still runs in exponential time, but with a good bound it visits very few nodes in practice.

The End

Chapter 5: Branch and Bound: Study Questions
Lecture slides: Chapter 5: Branch and Bound · Lecture notes
1 / 1 Sercan KÜLCÜ. All rights reserved.