Chapter 5: Branch and Bound

Advanced Algorithms
State-space treeBounding and pruningFIFO, LIFO, least cost0/1 knapsack Traveling salesmanInteger programmingMaximum cliqueComparing paradigms
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Paradigm

Explain the state-space tree, branching, bounding and pruning steps of branch and bound; state how it differs from backtracking.

Bound design

Build an upper bound for maximization and a lower bound for minimization; justify why the bound must be valid (optimistic).

Search strategies

Trace FIFO, LIFO and least-cost (best-first) strategies on the same example; compare them by memory and node count; tell the design paradigms apart.

Applications

Solve 0/1 knapsack, traveling salesman, integer linear programming and maximum clique step by step with branch and bound.

Introduction

Weighing the Best Options

  • Solving a hard problem is sometimes like picking the right way at a fork in the road.
  • Branch and bound algorithms explore the possible solution paths systematically to find the best option.
  • Goal: an exact (optimal) solution to combinatorial optimization problems. Trying every candidate takes exponential or factorial time.
  • Idea: estimate in advance how good a branch can possibly be; if that is not good enough, never enter the branch.
n2ⁿ subsets(n−1)! tours
53224
101,024362,880
201,048,576≈ 1.2 · 10¹⁷
40≈ 1.1 · 10¹²≈ 2.0 · 10⁴⁶

Knapsack tries 2ⁿ subsets, the traveling salesman (n−1)! tours: brute force quickly becomes impossible.

Introduction

Branch and Bound Algorithms

  • Think of it as a tree that branches and gets pruned.
  • First the problem is split into smaller subproblems (branching).
  • For each subproblem the best reachable value is estimated (the bound).
  • Options whose bound is no better than the best solution found so far are discarded (pruning).
  • The process repeats for all remaining subproblems; as useless branches are pruned, the search gets shorter.
  • Applications: the traveling salesman, knapsack, scheduling and more.
✗✗✗bestpruned
The subtree of a pruned node (✗) is never generated.

Introduction

Basic Concepts

State-space treeRoot = no decisions made yet; each level is one decision (e.g. "take item i?"); leaves are complete solutions.
BranchingGenerating a node's subproblems (children). The node being expanded is the E-node (expanding node).
BoundAn optimistic value that no solution below the node can beat: an upper bound for maximization, a lower bound for minimization.
Live nodeA promising node that has been generated but not yet expanded. Live nodes wait in a queue.
Dead nodeA node that has been expanded, or pruned as infeasible or with an insufficient bound; it is never considered again.
best (incumbent)The value of the best complete solution found so far. Pruning decisions are made against it.
dead (expanded)E-node✗ bound ≤ bestbest (complete solution)live (in the queue)✗ infeasible

Paradigm

General Scheme (Maximization)

int best = -∞;
queue.add(root);
while (!queue.isEmpty()) {
  Node u = queue.remove();    // strategy
  if (bound(u) <= best) continue;
  for (Node c : u.children()) {
    if (!isFeasible(c)) continue;
    if (isComplete(c) && value(c) > best)
      best = value(c);       // new incumbent
    if (bound(c) > best)
      queue.add(c);          // live node
  }
}
return best;
  1. Select: remove a live node from the queue. The strategy decides which one (FIFO, LIFO, best bound).
  2. Re-test: best may have grown while the node was waiting; if its bound is no longer enough, discard it.
  3. Branch: generate the children; drop infeasible ones.
  4. Update: if a complete solution is better, raise best.
  5. Bound: enqueue only children whose bound can beat best.
remove u from queuebound(u) ≤ best?generate childrenfeasible? complete? bound?enqueue live nodesyes → discard (re-test)infeasible / bound ≤ best → pruneuntil the queue is empty

For minimization the directions flip: use a lower bound and prune when lb(c) ≥ best.

Paradigm

How to Choose a Bound

It must be valid (optimistic)For maximization bound(u) ≥ every solution in the subtree; for minimization lb(u) ≤ every solution. Otherwise the optimum may be pruned and the result is wrong.
It should be tightThe closer it is to the true value, the more branches are pruned. An infinitely large "bound" is valid but prunes nothing.
It should be cheapThe bound is computed at every node. A very expensive bound can cost more time than the nodes it prunes.
Build it by relaxationDrop a hard constraint and solve the easy problem: fractional knapsack, LP relaxation, cheapest outgoing edges.
Success depends heavily on a good bound. In minimization the lower bound is the minimum achievable in the best case; a weak lower bound makes the search longer.

Paradigm

When Is a Node Pruned?

1. InfeasibilityThe node violates a constraint: weight > capacity, a second visit to the same city, an LP with no solution. No valid solution lies below it.
2. Boundbound(u) ≤ best (maximization) or lb(u) ≥ best (minimization). No solution below it can beat the best.
3. CompletenessThe node is a complete solution (a leaf), or the relaxation's solution is already feasible (e.g. the LP solution is integral). Its value is compared with best; no branching is needed.

Backtracking uses only rule 1 (and completeness). What sets branch and bound apart is rule 2: a node is cut even when feasible if it is "not good enough".

Paradigm

Search Strategies: Which Live Node Next?

StrategyContainerHow it exploresPros / cons
FIFO branch and boundqueue (ArrayDeque)level by level, breadth-first (BFS)finds shallow solutions early; the number of live nodes can grow exponentially
LIFO branch and boundstack (or recursion)depth-first (DFS)memory O(depth); an early complete solution raises best quickly
Least cost (LC, best-first)priority queue (PriorityQueue)node with the most promising bound firstusually expands the fewest nodes; the queue consumes memory, each operation is O(log k)
  • All three strategies find the same optimum; what changes is the number of generated nodes and the memory.
  • In LC search the cost function ĉ(u) is the bound value (or the estimate f(u) + ĝ(u)); A* search is the path-finding version of this idea.

Paradigm

Least-Cost (LC) Search: ĉ(x) = f(x) + ĝ(x)

  • f(x): distance from the root (cost paid so far); ĝ(x): estimate of the remaining cost from x to a solution.
  • Expand the live node with the smallest ĉ. ĝ = 0 → only the cost paid: uniform-cost search (FIFO/BFS with unit steps when f is the depth); f = 0 → greedy best-first search.
  • In the 8-puzzle ĝ = number of misplaced tiles: each misplaced tile needs at least one move, so ĝ never overestimates.
  • Initially 5 and 8 are misplaced (ĝ = 2). The child with the smallest ĉ is “blank down”; its child “blank right” reaches the goal: 2 moves, only 2 expansions.
12346758ĉ = 0 + 213426758blank upĉ = 1 + 3 = 412345678blank downĉ = 1 + 1 = 212346758blank leftĉ = 1 + 3 = 412346758blank rightĉ = 1 + 3 = 4
Goal: 1 2 3 / 4 5 6 / 7 8 blank. Orange: the tile that moved.

Paradigm

Backtracking vs. Branch and Bound

FeatureBacktrackingBranch and bound
Problem typedecision / search: one or all valid solutions (n-queens, Sudoku)optimization: the best solution (largest profit, shortest tour)
Pruning criterionfeasibility: is a constraint violated?feasibility + bound: can it beat best?
Traversalalways DFS (recursion)DFS, BFS or best-first
Extra informationonly the partial solutionpartial solution + bound function + best
MemoryO(depth)O(depth) with DFS; queue size with BFS/LC
Worst caseexponentialstill exponential; far fewer nodes in practice

A TSP or maximum-clique search with no bound test is really backtracking; add a single bound line and it becomes branch and bound.

Paradigm

Pros and Cons

Pros

  • Finds the exact optimum of hard optimization problems.
  • Narrows the search space and discards useless options.
  • Ideal for problems that ask for the best route or selection.
  • Applies to a wide range of problems whose solutions can be represented as a tree.
  • If stopped early, it still holds a good solution and its bound (a quality guarantee).

Cons

  • Solution time can grow when there are very many options: exponential time in the worst case.
  • With FIFO and best-first strategies, memory can also grow exponentially.
  • Success depends on a good bound; a weak bound prunes almost nothing.
  • Needs a problem-specific bound and branching rule; harder to implement than backtracking.

0/1 Knapsack

Example: The 0/1 Knapsack Problem

  • Notation: profit pᵢ, weight wᵢ, capacity W, decision xᵢ ∈ {0, 1}; maximize Σ pᵢxᵢ subject to Σ wᵢxᵢ ≤ W.
  • Items are sorted by decreasing pᵢ/wᵢ ratio; the bound computation relies on this order.
  • The state-space tree is binary: at level i the left branch is "item i taken" (1), the right one "not taken" (0). The full tree has 2⁵ − 1 = 31 nodes.
item 1item 2item 3item 41 = take0 = skip
Itempᵢwᵢpᵢ/wᵢ
140220
23056
350105
41052

Capacity W = 16.

Optimum: items 1 and 3, profit 90 (weight 12). Branch and bound gets there without generating all 31 nodes.

0/1 Knapsack

Upper Bound: Fractional Knapsack Relaxation

  • Relaxation: drop the "items cannot be split" constraint. The greedy method solves the fractional knapsack optimally.
  • At node u (profit k, weight a, level i): add the remaining items in ratio order whole while they fit, then a fraction of the first item that does not fit:
bound(u) = k + Σj = i+1m−1 pⱼ + (W − a − Σ wⱼ) · pm / wm    (m: first item that does not fit)
Why valid?Every 0/1 solution is also a fractional solution. Fractional optimum ≥ 0/1 optimum, so the bound never understates the true best.
For the root40 + 30 = 70 (weight 7); item 3 does not fit, 9 units left → 9 · 50/10 = 45. bound(root) = 115. If a > W the node is infeasible (bound 0).

0/1 Knapsack · Run the code step by step

bound(root): Code Trace

double bound(Node u) {
  if (u.weight > W) return 0;   // infeasible
  double result = u.profit;
  int totW = u.weight;
  int j = u.level + 1;
  while (j < n && totW + w[j] <= W) {
    totW += w[j];
    result += p[j];
    j++;
  }
  if (j < n)                // fractional part
    result += (W - totW) * (double) p[j] / w[j];
  return result;
}
capacity W = 160481216no items yet

Root node: level −1, profit 0, weight 0. Since 0 ≤ 16, the node is feasible.

capacity W = 160481216j = 0: next is item 1

The bound starts from the current profit; j points to the next item (index 0 = item 1).

capacity W = 1604812160 + 2 ≤ 16 ?

0 + w[0] = 2 ≤ 16 → item 1 fits entirely.

capacity W = 1610481216item 1 added

totW = 2, result = 40, j = 1.

capacity W = 16104812162 + 5 ≤ 16 ?

2 + w[1] = 7 ≤ 16 → item 2 fits entirely too.

capacity W = 16120481216item 2 added

totW = 7, result = 70, j = 2.

capacity W = 161204812167 + 10 = 17 > 16

7 + w[2] = 17 > 16 → item 3 does not fit entirely; the loop ends.

capacity W = 16123 (9/10)04812169 units left: 9/10 of item 3

Remaining capacity 9: 9/10 of item 3 → 9 · 50/10 = 45 is added, result = 115.

capacity W = 16123 (9/10)0481216bound(A) = 115

bound(A) = 115: no solution below this node can earn more than 115.

0/1 Knapsack · Step by step

Best-First (LC) Branch and Bound: State-Space Tree

rootitem 1(40, 2)item 2(30, 5)item 3(50, 10)item 4(10, 5)A0 / 0s = 115

Root A: no item chosen (profit 0, weight 0). Fractional bound b = 115; best = 0. Queue: A (115).

rootitem 1(40, 2)item 2(30, 5)item 3(50, 10)item 4(10, 5)10A0 / 0s = 115B40 / 2s = 115C0 / 0s = 82

A is removed (largest bound) and branched. B (item 1 taken): best = 40, b = 115 → enqueued. C (no item 1): b = 82 → enqueued. Queue: B (115), C (82).

rootitem 1(40, 2)item 2(30, 5)item 3(50, 10)item 4(10, 5)1010A0 / 0s = 115B40 / 2s = 115C0 / 0s = 82D70 / 7s = 115E40 / 2s = 98

B is removed (largest bound) and branched. D (item 2 taken): best = 70, b = 115 → enqueued. E (no item 2): b = 98 → enqueued. Queue: D (115), E (98), C (82).

rootitem 1(40, 2)item 2(30, 5)item 3(50, 10)item 4(10, 5)101010A0 / 0s = 115B40 / 2s = 115C0 / 0s = 82D70 / 7s = 115E40 / 2s = 98F120 / 17infeasibleG70 / 7s = 80

D is removed (largest bound) and branched. F (item 3 taken): weight 17 > 16 → infeasible. G (no item 3): b = 80 → enqueued. Queue: E (98), C (82), G (80).

rootitem 1(40, 2)item 2(30, 5)item 3(50, 10)item 4(10, 5)10101010A0 / 0s = 115B40 / 2s = 115C0 / 0s = 82D70 / 7s = 115E40 / 2s = 98F120 / 17infeasibleG70 / 7s = 80H90 / 12s = 98I40 / 2s = 50

E is removed (largest bound) and branched. H (item 3 taken): best = 90, b = 98 → enqueued. I (no item 3): b = 50 ≤ 90 → pruned. Queue: H (98), C (82), G (80).

rootitem 1(40, 2)item 2(30, 5)item 3(50, 10)item 4(10, 5)1010101010A0 / 0s = 115B40 / 2s = 115C0 / 0s = 82D70 / 7s = 115E40 / 2s = 98F120 / 17infeasibleG70 / 7s = 80H90 / 12s = 98I40 / 2s = 50J100 / 17infeasibleK90 / 12s = 90

H is removed (largest bound) and branched. J (item 4 taken): weight 17 > 16 → infeasible. K (no item 4): b = 90 ≤ 90 → pruned. Queue: C (82), G (80).

rootitem 1(40, 2)item 2(30, 5)item 3(50, 10)item 4(10, 5)1010101010A0 / 0s = 115B40 / 2s = 115C0 / 0s = 82D70 / 7s = 115E40 / 2s = 98F120 / 17infeasibleG70 / 7s = 80H90 / 12s = 98I40 / 2s = 50J100 / 17infeasibleK90 / 12s = 90

C is removed: b = 82 ≤ best = 90 → pruned. G is removed: b = 80 ≤ best = 90 → pruned. Queue empty: best profit 90 (H: items 1 and 3). Only 11 of the full tree's 31 nodes were generated.

0/1 Knapsack

Best-First Branch and Bound: Java

int branchAndBound() {
  PriorityQueue<Node> queue = new PriorityQueue<>(
      (a, b) -> Double.compare(b.bound, a.bound));
  Node root = new Node(-1, 0, 0);
  root.bound = bound(root);
  queue.add(root);  int best = 0;
  while (!queue.isEmpty()) {
    Node u = queue.poll();             // largest bound
    if (u.bound <= best || u.level == n - 1) continue;
    int i = u.level + 1;
    // left child: take item i
    Node take = new Node(i, u.profit + p[i],
                         u.weight + w[i]);
    if (take.weight <= W && take.profit > best)
      best = take.profit;
    take.bound = bound(take);
    if (take.bound > best) queue.add(take);
    // right child: skip item i
    Node skip = new Node(i, u.profit, u.weight);
    skip.bound = bound(skip);
    if (skip.bound > best) queue.add(skip);
  }
  return best;
}
  • Node: level, profit, weight, bound (a boolean[] or bitmask can be added for the chosen items).
  • The comparator is reversed: the largest bound comes out first.
  • The test after poll discards nodes that went stale while waiting in the queue (C and G in the example).
  • For FIFO, ArrayDeque + add()/poll() is enough; for LIFO, push()/pop(); the rest of the code is the same.
  • Worst case O(2ⁿ) nodes, O(n) bound computation per node.

0/1 Knapsack · Run the code step by step

FIFO Branch and Bound: Code Trace

int fifoBranchAndBound() {
  ArrayDeque<Node> queue = new ArrayDeque<>();
  queue.add(root);  int best = 0;
  while (!queue.isEmpty()) {
    Node u = queue.poll();            // first in
    if (u.bound <= best || u.level == n - 1) continue;
    for (int take = 1; take >= 0; take--) { // take first
      Node c = child(u, take);        // computes bound(c)
      if (c.weight <= W && c.profit > best)
        best = c.profit;                // new incumbent
      if (c.bound > best) queue.add(c);
    }
  }
  return best;
}
A115in queuepruned / infeasiblebest

The same knapsack instance (W = 16), this time with a FIFO queue. Root A: profit 0, bound 115; only A is in the queue.

10A115B115C82in queuepruned / infeasiblebest

A leaves the front of the queue and branches: B (take: b = 115 → enqueued, best = 40); C (skip: b = 82 → enqueued).

1010A115B115C82D115E98in queuepruned / infeasiblebest

B leaves the front of the queue and branches: D (take: b = 115 → enqueued, best = 70); E (skip: b = 98 → enqueued).

101010A115B115C82D115E98L82M60in queuepruned / infeasiblebest

C leaves the front of the queue and branches: L (take: b = 82 → enqueued); M (skip: b = 60 ≤ 70, pruned).

10101010A115B115C82D115E98L82M60F0G80in queuepruned / infeasiblebest

D leaves the front of the queue and branches: F (take: weight 17 > 16, infeasible); G (skip: b = 80 → enqueued).

1010101010A115B115C82D115E98L82M60F0G80H98I50in queuepruned / infeasiblebest

E leaves the front of the queue and branches: H (take: b = 98 → enqueued, best = 90); I (skip: b = 50 ≤ 90, pruned).

1010101010A115B115C82D115E98L82M60F0G80H98I50in queuepruned / infeasiblebest

L is removed: its bound 82 ≤ best = 90 → discarded by the re-test (it looked promising when enqueued, but best grew while it waited).

1010101010A115B115C82D115E98L82M60F0G80H98I50in queuepruned / infeasiblebest

G is removed: its bound 80 ≤ best = 90 → discarded by the re-test (it looked promising when enqueued, but best grew while it waited).

101010101010A115B115C82D115E98L82M60F0G80H98I50J0K90in queuepruned / infeasiblebest

H leaves the front of the queue and branches: J (take: weight 17 > 16, infeasible); K (skip: b = 90 ≤ 90, pruned). Queue empty: best = 90 (items 1 and 3). 13 nodes generated, 6 expansions, at most 3 in the queue.

0/1 Knapsack

Three Strategies on the Same Example

123456
FIFO: 13 nodes, 6 expansions
123546
LIFO: 13 nodes, 6 expansions
12345
Best-first: 11 nodes, 5 expansions

Numbers give the expansion order; small circles are nodes generated but not expanded (red dashed: pruned); ring: the optimum (90).

MethodNodes generatedExpandedLargest queue
Brute force (full tree)3115—
FIFO branch and bound1363
LIFO branch and bound ("take" first)1363
Best-first (LC)1153

Traveling Salesman

Traveling Salesman Problem (TSP): Lower Bound

  • Minimization: we want the shortest tour; node bounds are lower bounds.
  • Node = a partial path starting at 0; children = the next unvisited city.
  • Lower bound: every city not yet left (the current city + the unvisited ones) must be left by at least one edge:
321521580123bold: each city'scheapest edge2 + 3 + 2 + 5 = 12
0123min
0—32152
13—2153
2221—82
31558—5

Symmetric 4-city distance matrix d.

lb(path) = cost(path) + minOut[last city] + Σv unvisited minOut[v]

Root: 2 + 3 + 2 + 5 = 12. Path 0 → 1: 3 + 3 + (2 + 5) = 13. No real tour can cost less than this sum, so the bound is valid.

Traveling Salesman · Run the code step by step

tsp(0, 0, 1): Bounded Depth-First Search

void tsp(int city, int cost, int count) {
  if (count == n) {           // tour complete
    int tour = cost + d[city][0];
    if (tour < best) best = tour;
    return;
  }
  for (int s = 0; s < n; s++) {
    if (visited[s]) continue;
    int next = cost + d[city][s];
    visited[s] = true;
    if (lowerBound(s, next) < best) // bound
      tsp(s, next, count + 1);
    visited[s] = false;      // undo
  }
}
0a=12

Start at city 0. The root's lower bound = 2 + 3 + 2 + 5 = 12.

30a=121a=13

0 → 1: next = 0 + 3 = 3; lb = 13 < ∞ → branch.

3210a=121a=132a=31

1 → 2: next = 3 + 21 = 24; lb = 31 < ∞ → branch.

32180a=121a=132a=313a=37

2 → 3: next = 24 + 8 = 32; lb = 37 < ∞ → branch.

32180a=121a=132a=313tour 47

Tour 0 → 1 → 2 → 3 → 0 = 32 + 15 = 47 < ∞ → best = 47.

321850a=121a=132a=313tour 473a=15

1 → 3: next = 3 + 5 = 8; lb = 15 < 47 → branch.

3218580a=121a=132a=313tour 473a=152a=18

3 → 2: next = 8 + 8 = 16; lb = 18 < 47 → branch.

3218580a=121a=132a=313tour 473a=152tour 18

Tour 0 → 1 → 3 → 2 → 0 = 16 + 2 = 18 < 47 → best = 18.

32185820a=121a=132a=313tour 473a=152tour 182a=12

0 → 2: next = 0 + 2 = 2; lb = 12 < 18 → branch.

3218582210a=121a=132a=313tour 473a=152tour 182a=121a=31✗

2 → 1: next = 23; lb = 31 ≥ best = 18 → pruned.

32185822180a=121a=132a=313tour 473a=152tour 182a=121a=31✗3a=18✗

2 → 3: next = 10; lb = 18 ≥ best = 18 → pruned.

3218582218150a=121a=132a=313tour 473a=152tour 182a=121a=31✗3a=18✗3a=25✗

0 → 3: next = 15; lb = 25 ≥ best = 18 → pruned.

3218582218150a=121a=132a=313tour 473a=152tour 182a=121a=31✗3a=18✗3a=25✗

Done: the shortest tour is 0 → 1 → 3 → 2 → 0 = 18; 10 nodes instead of 16.

Traveling Salesman · Run the code step by step

lowerBound(3, 8): Code Trace

int lowerBound(int last, int cost) {
  int lb = cost + minOut[last];
  for (int v = 0; v < n; v++)
    if (!visited[v])        // not visited
      lb += minOut[v];
  return lb;
}
321521580123path 0 → 1 → 3, cost 8vminOutvis.02✓13✓22—35✓

Step 1 → 3 of the TSP trace: path 0 → 1 → 3, cost 3 + 5 = 8; visited = {0, 1, 3}. The last city 3 must be left once more: lb = 8 + minOut[3] = 8 + 5 = 13.

321521580123path 0 → 1 → 3, cost 8vminOutvis.02✓13✓22—35✓

v = 0: visited (on the path) → no contribution; its outgoing edge is already in the cost.

321521580123path 0 → 1 → 3, cost 8vminOutvis.02✓13✓22—35✓

v = 1: visited (on the path) → no contribution; its outgoing edge is already in the cost.

321521580123path 0 → 1 → 3, cost 8vminOutvis.02✓13✓22—35✓

v = 2: not yet visited → must be left by at least minOut[2] = 2: lb = 15.

321521580123path 0 → 1 → 3, cost 8vminOutvis.02✓13✓22—35✓

v = 3: visited (on the path) → no contribution; 3's exit was added at the start.

321521580123path 0 → 1 → 3, cost 8vminOutvis.02✓13✓22—35✓

Returns lb = 15. Every tour in this branch is ≥ 15; the real tour 0 → 1 → 3 → 2 → 0 = 18. Since 15 < best = 47, the TSP trace entered the branch.

Traveling Salesman

A Tighter Lower Bound: Two Shortest Edges

  • A tour enters each city by one edge and leaves it by another: every city touches exactly two tour edges.
  • These two edges cannot be shorter than the two shortest edges touching the city. Summed over all cities, each edge is counted twice:
tour ≥ ½ · Σv (v's two shortest edges)
321521580123v two smallest sum0 2 + 3 = 51 3 + 5 = 82 2 + 8 = 103 5 + 8 = 1336 / 2 = 18

On the same matrix the “cheapest outgoing edge” bound gave 12; this bound gives 18, equal to the optimal tour 0 → 1 → 3 → 2 → 0 = 3 + 5 + 8 + 2 = 18. Since bound = optimum at the root, all remaining branches are pruned as soon as the first complete tour is found.

Traveling Salesman · Run the code step by step

twoEdgeBound(): Code Trace

double twoEdgeBound() {
  int sum = 0;
  for (int v = 0; v < n; v++) {
    int m1 = INF, m2 = INF;     // two smallest
    for (int u = 0; u < n; u++) {
      if (u == v) continue;
      if (d[v][u] < m1) { m2 = m1; m1 = d[v][u]; }
      else if (d[v][u] < m2) m2 = d[v][u];
    }
    sum += m1 + m2;
  }
  return sum / 2.0;  // each edge counted twice
}
321521580123m1 = — m2 = — sum = 0

Symmetric 4-city matrix. sum = 0; we look for the two shortest edges of each city.

321521580123m1 = 3 m2 = ∞ sum = 0

v = 0, u = 1: d = 3 < m1 = ∞ → m1 = 3 (the old m1 shifts to m2).

321521580123m1 = 2 m2 = 3 sum = 0

u = 2: d = 2 < m1 = 3 → m2 = 3, m1 = 2. The two minima shift along.

321521580123m1 = 2 m2 = 3 sum = 5

u = 3: 15 is larger than both, no change. sum += 2 + 3 = 5.

321521580123m1 = 3 m2 = 5 sum = 13

v = 1: edges 3, 21, 5 → m1 = 3, m2 = 5. sum = 13.

321521580123m1 = 2 m2 = 8 sum = 23

v = 2: edges 2, 21, 8 → 2 + 8. sum = 23.

321521580123m1 = 5 m2 = 8 sum = 36

v = 3: edges 15, 5, 8 → 5 + 8. sum = 36.

321521580123m1 = 5 m2 = 8 sum = 36

Returns 36 / 2 = 18. The bold edges form exactly the optimal tour (0-1-3-2-0), which is why the bound is tight. On a partial path the same idea applies with the path's edges fixed.

Traveling Salesman

Lower Bound from the Reduced Cost Matrix

  • Set the diagonal to ∞. Subtract the minimum of each row, then the minimum of each column.
  • Every tour uses exactly one cell in each row and each column; the total subtracted comes off every tour's cost → a lower bound.
  • Example: rows 2 + 3 + 2 + 5 = 12, 2 from column 3 → root bound 14.
  • Branching on edge i → j: set row i and column j to ∞, set (j, i) to ∞, and reduce the matrix again. Child bound = parent + reduced c[i][j] + the new reduction.
Branch from the rootbound
0 → 114 + 1 + 3 = 18
0 → 214 + 0 + 4 = 18
0 → 314 + 11 + 3 = 28
0123
0∞1011
10∞180
2019∞4
31003∞

Reduced matrix (total 14). Every row and column contains at least one 0.

Branch 0 → 3 with 28 is pruned as soon as the optimum 18 is found. The method also works for the asymmetric TSP (d[i][j] ≠ d[j][i]) (Little et al., 1963).

Traveling Salesman · Run the code step by step

reduce(): Row and Column Reduction

int reduce(int[][] m) {
  int sum = 0;
  for (int i = 0; i < n; i++) {       // rows
    int mn = rowMin(m, i);
    if (mn > 0 && mn < INF) {
      subtractRow(m, i, mn); sum += mn;
    }
  }
  for (int j = 0; j < n; j++) {       // columns
    int mn = colMin(m, j);
    if (mn > 0 && mn < INF) {
      subtractCol(m, j, mn); sum += mn;
    }
  }
  return sum;
}
01230∞321513∞2152221∞831558∞sum = 0

4 × 4 distance matrix with ∞ on the diagonal; sum = 0.

01230∞101313∞2152221∞831558∞sum = 2mn = 2

Row 0: minimum 2 → subtract 2 from the row: 1, 0, 13. sum = 2.

01230∞101310∞1822221∞831558∞sum = 5mn = 3

Row 1: minimum 3 → 0, 18, 2. sum = 5.

01230∞101310∞1822019∞631558∞sum = 7mn = 2

Row 2: minimum 2 → 0, 19, 6. sum = 7.

01230∞101310∞1822019∞631003∞sum = 12mn = 5

Row 3: minimum 5 → 10, 0, 3. Row reduction done: sum = 12 (the same as the cheapest-exit bound).

01230∞101310∞1822019∞631003∞sum = 12mn = 0

Column 0: minimum 0 → no change. Columns 1 and 2 already contain a 0.

01230∞101110∞1802019∞431003∞sum = 14mn = 2

Column 3: values 13, 2, 6 → minimum 2; subtracted: 11, 0, 4. sum = 14. The column step adds the fact that “each city is entered once” to the bound.

01230∞101110∞1802019∞431003∞sum = 14

Returns bound 14: tighter than 12, looser than the two-edge bound (18). The matrix is kept for computing the children's bounds.

Traveling Salesman

Three TSP Bounds: Tightness and Cost

cheapest outgoing edge12reduced matrix14two shortest edges18optimal tour18
BoundCost per nodeWhat it usesNote
Cheapest outgoing edgeO(n) (minOut[] precomputed)one exit from each citycheapest, loosest
Reduced matrixO(n²)one entry + one exit per cityalso valid for asymmetric TSP
Two shortest edgesO(n²)two edges touching each cityfor symmetric TSP
A tighter bound prunes more but costs more at each node. Even tighter bounds (1-trees, LP relaxation) are used on large instances; the choice balances nodes pruned against work per node.

Traveling Salesman

A Larger Example: 6 Cities

012345min
0—292021163116
129—1529284015
22015—15142514
3212915—121812
416281412—2212
53140251822—18
  • Backtracking without pruning tries all (n−1)! = 120 tours; with the lowerBound test the same answer is found with far fewer nodes.
  • Root lower bound: 16 + 15 + 14 + 12 + 12 + 18 = 87; optimum 115.
  • Best tour: 0 → 1 → 2 → 3 → 5 → 4 → 0 = 29 + 15 + 15 + 18 + 22 + 16 = 115.
VersionNodes generatedtsp calls
No pruning (backtracking)326326
Cheapest-exit bound210130
Tighter boundsFor the symmetric TSP, half the sum of each city's two shortest edges; or the row/column reduced cost matrix (Little et al., 1963). A tighter bound prunes more but costs more at each node.

Assignment Problem

Assignment Problem: n People, n Jobs

  • c[i][j]: the cost of person i doing job j. Give each person a different job so that the total is minimized; there are n! possible assignments.
  • State space: at level i, pick a free job for person i.
  • Lower bound: each remaining person pays at least the minimum of their own row (job clashes are ignored):
lb = next + restMin[i + 1],  restMin[i] = Σr ≥ i minj c[r][j]

Cut the branch if lb ≥ best. Row minima 4, 3, 2 → restMin = [9, 5, 2, 0]: the root bound is 9.

person \ job012min
05844
17363
22952
The assignment problem can also be solved in polynomial time (the Hungarian method, O(n³)). Here branch and bound just shows the method on a small, easy-to-trace example; it is truly needed for variants with extra constraints (e.g. some jobs cannot be done together).

Assignment Problem · Run the code step by step

assign(0, 0): Code Trace

void assign(int i, int cost) {
  if (i == n) {                     // complete assignment
    best = Math.min(best, cost);
    return;
  }
  for (int j = 0; j < n; j++) {
    if (taken[j]) continue;
    int next = cost + c[i][j];
    if (next + restMin[i + 1] >= best)
      continue;                     // bound
    taken[j] = true;
    assign(i + 1, next);
    taken[j] = false;               // undo
  }
}
job 0job 1job 2k0584k1736k2295best = ∞

restMin = [9, 5, 2, 0]; best = ∞. assign(0, 0) is called: jobs are tried in order for person 0.

job 0job 1job 2k0584k1736k2295best = ∞lb = 10

p0 → job 0: next = 5, lb = 5 + restMin[1] = 10 < ∞ → branch.

job 0job 1job 2k0584k1736k2295best = ∞lb = 10

p1: job 0 is taken, skipped. Job 1: next = 8, lb = 8 + 2 = 10 → branch.

job 0job 1job 2k0584k1736k2295best = 13

p2: only job 2 is free: next = 13 → i = 3, complete assignment: best = 13.

job 0job 1job 2k0584k1736k2295best = 13lb = 13

Back up. p1 → job 2: next = 11, lb = 11 + 2 = 13 ≥ 13 → cut.

job 0job 1job 2k0584k1736k2295best = 13lb = 13

p0 → job 1: next = 8, lb = 8 + 5 = 13 ≥ 13 → cut; the 2 assignments below it are never tried.

job 0job 1job 2k0584k1736k2295best = 13lb = 9

p0 → job 2: next = 4, lb = 9 < 13 → branch.

job 0job 1job 2k0584k1736k2295best = 13lb = 13

p1 → job 0: next = 11, lb = 13 ≥ 13 → cut.

job 0job 1job 2k0584k1736k2295best = 13lb = 9

p1 → job 1: next = 7, lb = 9 → branch.

job 0job 1job 2k0584k1736k2295best = 9

p2 → job 0: next = 9 → complete assignment, best = 9 (4 + 3 + 2). It equals the bound, so nothing better exists. 7 calls; a search without pruning would visit all 3! = 6 assignments (16 calls).

Other Applications

Integer Linear Programming (ILP)

  • Every variable of the linear program must be an integer. This is the most common industrial use of branch and bound (Land and Doig, 1960).
  • Bound: drop the integrality condition and solve the LP relaxation. The LP optimum is an upper bound on the ILP optimum.
  • Branching: for a fractional variable (y = 1.5), two subproblems: y ≤ 1 and y ≥ 2. The fractional region in between is discarded; no integer point is lost.
Example: maximize Z = 5x + 4y; 6x + 4y ≤ 24, x + 2y ≤ 6, x, y ≥ 0 integers.

Second example: Z = 3x + 2y; x + y ≤ 4, x ≤ 3, y ≤ 2. Brute force tries all 12 points of the grid 0 ≤ x ≤ 3, 0 ≤ y ≤ 2 → (3, 1), Z = 11. Here the LP relaxation also gives (3, 1): branch and bound ends at the root.

12345671234560xy6x+4y=24x+2y=6LP: (3, 1.5) Z = 21↑ (4; 0) Z = 20
Shaded: the LP feasible region; dots: integer solutions.

Other Applications · Step by step

ILP: Branching on LP Relaxations

x=3, y=1.5Z = 21

Root: the integrality condition is relaxed (LP relaxation). The LP optimum (3, 1.5), Z = 21 → upper bound. y is fractional → branch on y.

y ≤ 1y ≥ 2x=3, y=1.5Z = 21x=3.33, y=1Z = 20.67x=2, y=2Z = 18

Branch y ≤ 1: LP optimum (3.33, 1), Z = 20.67. x is fractional → open branches x ≤ 3 and x ≥ 4.

y ≤ 1y ≥ 2x ≤ 3x=3, y=1.5Z = 21x=3.33, y=1Z = 20.67x=2, y=2Z = 18x=3, y=1Z = 19

Branch x ≤ 3: the LP solution (3, 1) is integral → first incumbent, best = 19.

y ≤ 1y ≥ 2x ≤ 3x ≥ 4x=3, y=1.5Z = 21x=3.33, y=1Z = 20.67x=2, y=2Z = 18x=3, y=1Z = 19x=4, y=0Z = 20

Branch x ≥ 4: the LP solution (4, 0) is integral, Z = 20 > 19 → best = 20.

y ≤ 1y ≥ 2x ≤ 3x ≥ 4x=3, y=1.5Z = 21x=3.33, y=1Z = 20.67x=2, y=2Z = 18x=3, y=1Z = 19x=4, y=0Z = 20

Branch y ≥ 2: LP bound Z = 18 ≤ 20 → pruned. Optimum: x = 4, y = 0, Z = 20.

Rounding the LP optimum (3, 1.5) gives (3, 1) → Z = 19, or (3, 2) → infeasible. The true optimum (4, 0) cannot be found by rounding.

Other Applications

Maximum Clique

012345
A 6-vertex example graph; the largest clique is {0, 1, 2, 3, 4}.
  • Clique: a set of vertices in which every two are adjacent.
  • Bound: if |K| + |cand| ≤ |best|, the current clique cannot beat best however much it grows.
void clique(List<Integer> K, int start) {
  if (K.size() > best.size())
    best = new ArrayList<>(K);
  List<Integer> cand = new ArrayList<>();
  for (int v = start; v < n; v++)
    if (adjToAll(v, K)) cand.add(v); // adjacent to K
  if (K.size() + cand.size() <= best.size())
    return;                        // bound
  for (int v : cand) {
    K.add(v);
    clique(K, v + 1);
    K.remove(K.size() - 1);          // undo
  }
}
Versionclique calls
Backtracking without a bound36
With the |K| + |cand| bound17

Other Applications · Run the code step by step

clique({}, 0): Bounded Search Trace

void clique(List<Integer> K, int start) {
  if (K.size() > best.size())
    best = new ArrayList<>(K);
  List<Integer> cand = new ArrayList<>();
  for (int v = start; v < n; v++)
    if (adjToAll(v, K)) cand.add(v); // adjacent to K
  if (K.size() + cand.size() <= best.size())
    return;                        // bound
  for (int v : cand) {
    K.add(v);
    clique(K, v + 1);
    K.remove(K.size() - 1);        // undo
  }
}
012345|K| + |cand| = 0 + 6 = 6 ; |best| = 0

Six-vertex graph; K is empty, best is empty. All of 0–5 are candidates: 0 + 6 > 0, the bound passes; the first candidate 0 is added and we go deeper.

012345|K| + |cand| = 1 + 4 = 5 ; |best| = 1

K = {0} is larger than best → best = {0}. Candidates adjacent to all of K: {1, 2, 3, 4}; 1 + 4 > 1 → continue, the first candidate 1 is added.

012345|K| + |cand| = 2 + 3 = 5 ; |best| = 2

K = {0, 1} is larger than best → best = {0, 1}. Candidates adjacent to all of K: {2, 3, 4}; 2 + 3 > 2 → continue, the first candidate 2 is added.

012345|K| + |cand| = 3 + 2 = 5 ; |best| = 3

K = {0, 1, 2} is larger than best → best = {0, 1, 2}. Candidates adjacent to all of K: {3, 4}; 3 + 2 > 3 → continue, the first candidate 3 is added.

012345|K| + |cand| = 4 + 1 = 5 ; |best| = 4

K = {0, 1, 2, 3} is larger than best → best = {0, 1, 2, 3}. Candidates adjacent to all of K: {4}; 4 + 1 > 4 → continue, the first candidate 4 is added.

012345|K| + |cand| = 5 + 0 = 5 ; |best| = 5

K = {0, 1, 2, 3, 4}: best = {0, 1, 2, 3, 4} (5 vertices). No candidates left: 5 + 0 ≤ 5 → return.

012345|K| + |cand| = 4 + 0 = 4 ; |best| = 5

Undo, then try K = {0, 1, 2, 4}: cand {}, 4 + 0 = 4 ≤ 5 → pruned by the bound; this clique cannot beat 5.

012345|K| + |cand| = 3 + 1 = 4 ; |best| = 5

Undo, then try K = {0, 1, 3}: cand {4}, 3 + 1 = 4 ≤ 5 → pruned by the bound; this clique cannot beat 5.

012345|K| + |cand| = 2 + 0 = 2 ; |best| = 5

Likewise {0, 1, 4}: 3 + 0; {0, 2}: 2 + 2; {0, 3}: 2 + 1; {0, 4}: 2 + 0. All ≤ 5, pruned.

012345|K| + |cand| = 1 + 2 = 3 ; |best| = 5

The remaining root candidates 1–5 are tried one by one ({1}: 4, {2}: 3, {3}: 3, {4}: 2, {5}: 1); all ≤ 5 → pruned. Result {0, 1, 2, 3, 4}, 17 calls (36 without the bound).

Other Applications

A Tighter Bound for Clique: Coloring

  • Every pair of vertices in a clique is adjacent, so all vertices of a clique get different colors.
  • If the candidates can be colored with k colors, at most k of them can join the clique: prune if |K| + colors(cand) ≤ |best|.
  • Greedy coloring takes O(cand²) and, even if not optimal, gives a valid upper bound (extra colors only loosen the bound).
  • Example: the candidates form a 5-vertex ring, K is empty, best = 3. The old bound 0 + 5 > 3 → the branch is opened; the color bound 0 + 3 ≤ 3 → pruned (the largest clique in a ring is 2).
012343 colors:{0, 2} {1, 3} {4}|cand| = 5color bound = 3
Most of the fastest exact clique algorithms in practice (Tomita's MCQ / MCS family) rest on this idea: try candidates in color order and prune with the color bound.

Other Applications

Other Problems Solved by Branch and Bound

Hamiltonian pathA path visiting every vertex exactly once; the weighted version is solved with bounds much like TSP's.
Satisfiability (SAT)Assign truth values to variables; prune conflicting partial assignments (DPLL). For MAX-SAT the bound is the most clauses that can still be satisfied.
SchedulingMinimize makespan or tardiness; bound: the best-case load of the remaining jobs.
Bin packingPack items into as few bins as possible; lower bound ⌈Σ size / bin capacity⌉.
Maximum coverageChoose k sets covering the most elements; bound: the elements the largest remaining sets could add.
Assignment problemAssign n jobs to n people at minimum cost; lower bound: the minimum cost of each unassigned row.

Problems with polynomial-time algorithms, such as maximum flow, do not need branch and bound; the method makes sense for NP-hard problems.

Other Applications

Branch and Bound, A* and Depth-First Branch and Bound

FeatureBest-first branch and bound (LC)A*Depth-first branch and bound
Selectionbest bound (ĉ)smallest f = g + hstack / recursion
Bound / estimatevalid (optimistic) boundadmissible h (never overestimates)valid bound + best
Goaloptimization (best leaf)shortest path from start to goaloptimization
Memorylive nodes: can be exponentialopen list: can be exponentialO(depth)
First complete solutionusually late but goodthe first goal reached is optimalearly (a solution is always at hand)
  • A* is LC branch and bound searching for a path in a weighted graph: h(x) is a lower bound on the remaining cost from x to the goal (misplaced tiles in the 8-puzzle).
  • When memory runs out, the depth-first version or IDA* (depth-first search with an iteratively increasing bound threshold) is used.

Evaluation

Design Paradigms: A Cumulative Comparison

ParadigmDecisionSubproblems / traversalWhen?Example
Brute forcegenerate and test every candidatethe full search spacetiny search spaces8⁸ queen placements
Divide and conquerdivide, solve, combineindependent; top-downnon-overlapping subproblemsmerge sort, Strassen
Greedybest-looking choice; never undoneone subproblem after each choicegreedy-choice property holdsDijkstra, Huffman, Kruskal
Dynamic programmingtry all options, keep the bestoverlapping; mostly bottom-upoptimal substructure, polynomially many statesFibonacci, 0/1 knapsack, LCS
Backtrackingextend; undo if invalidDFS on the state-space treeconstraint satisfaction, all solutionsn-queens, Sudoku
Branch and boundbacktracking + pruning by boundDFS, BFS or best-firstNP-hard optimization, exact result0/1 knapsack, TSP
Independent subproblems → divide and conquer. Overlapping → dynamic programming. Locally best is globally best → greedy. NP-hard: backtracking / branch and bound for an exact answer, an approximation algorithm for a guaranteed one in polynomial time.

Summary

Summary

TopicKey points
Paradigmstate-space tree; select → branch → bound → prune; best = best complete solution so far
Boundvalid (optimistic), tight, cheap; from a relaxation. Upper for max, lower for min
Pruninginfeasible, bound ≤ best (lb ≥ best), complete / integral LP
StrategiesFIFO (queue), LIFO (stack, little memory), LC / best-first (priority queue, few nodes)
Knapsackfractional upper bound; bound(root) = 115, optimum 90; 11 nodes instead of 31
TSPlb = cost + cheapest exit edge of each unfinished city; 16 → 10 nodes for 4 cities
ILP / cliqueLP relaxation + branch on a fractional variable; clique: prune if |K| + |cand| ≤ |best|
Complexityexponential in the worst case; speed in practice depends on the bound
Paradigmsindependent → D&C; overlapping → DP; greedy choice → greedy; NP-hard exact → backtracking / B&B

Summary

Common Mistakes

Invalid (pessimistic) boundIn knapsack, summing only the items that fit whole is not an upper bound; the branch containing the optimum may be pruned.
Mixing up the directionPrune when bound ≤ best for maximization and lb ≥ best for minimization. Writing the reverse either misses the optimum or prunes nothing.
Not re-testing after removalThe node was promising when it was added; if best grew while it waited, it is now useless (C and G in the example).
Not sorting the itemsThe fractional bound is not a valid upper bound unless items are in decreasing order of pᵢ/wᵢ.
Updating best only at leavesIn knapsack every feasible node is a solution; updating early gives more pruning.
Confusing it with backtrackingCode without a bound test (e.g. a TSP search with no pruning) is not branch and bound; it just walks the whole tree.

Summary

Sample Problems

Knapsack: (p, w) = (20, 2), (30, 5), (35, 7), (12, 3); W = 9 (ratios 10, 6, 5, 4).

  1. Compute the root's fractional upper bound.
  2. What is the optimal profit, and which items are taken?
  3. What is the bound of the node "item 1 not taken" (0, 0)? Is it pruned once the optimum is found?
  4. In the 4-city TSP example, what is the lower bound of the path 0 → 3 → 1?
  5. In maximum clique, what happens when K = {3, 4}, cand = {5} and |best| = 5?
  6. Which search strategy should be used on a system with very little memory?
Show answers
  1. 20 + 30 = 50 (weight 7); the remaining 2 units from item 3: 2 · 35/7 = 10 → 60.
  2. Items 1 and 3: 55 (weight 9).
  3. Item 2 fits whole: 30 (weight 5); item 3 does not fit, 4 units left: 4 · 35/7 = 20 → 50. 50 ≤ 55 → pruned.
  4. cost 15 + 5 = 20; + minOut[1] = 3 + minOut[2] = 2 → 25 (≥ 18, pruned).
  5. 2 + 1 = 3 ≤ 5 → pruned; no clique larger than 5 can come from this branch.
  6. LIFO (DFS): only the siblings of the current path are kept; memory O(depth · branching).

Summary

Exercises

  1. Write the knapsack code with a FIFO queue; how many nodes are generated on the chapter's example?
  2. Add an int[] bestPath to the tsp method that records the best tour.
  3. Compute the "half the sum of the two shortest edges" bound at the root of the 4-city example.
  4. For the assignment problem with cost matrix [[9, 2, 7], [6, 4, 3], [5, 8, 1]], what are the root lower bound and the optimum?
  5. Prove that the fractional knapsack bound is always valid.
  6. In how many nodes does LP-based branch and bound solve the second ILP example (Z = 3x + 2y; x + y ≤ 4, x ≤ 3, y ≤ 2)?
Answers / hints
  1. ArrayDeque + poll(): 13 nodes, 6 expansions; the result is still 90.
  2. In the loop, path[count] = s; if tour < best, bestPath = path.clone().
  3. (2+3) + (3+5) + (2+8) + (5+8) = 36 → 36/2 = 18: equal to the optimum.
  4. Row minima 2 + 3 + 1 = 6; optimum 2 + 6 + 1 = 9.
  5. Every 0/1 solution is a fractional solution; greedy gives the fractional optimum, which is ≥ the 0/1 optimum.
  6. A single node: the LP relaxation is already integral, (3, 1), Z = 11.

Summary

References

  • E. Horowitz, S. Sahni and S. Rajasekaran, Computer Algorithms, 2nd ed., Silicon Press, 2008 (Chapter 8: Branch-and-Bound).
  • R. Neapolitan, Foundations of Algorithms, 5th ed., Jones & Bartlett, 2015 (Chapter 6).
  • A. Levitin, Introduction to the Design and Analysis of Algorithms, 3rd ed., Pearson, 2012 (Section 12.2).
  • A. H. Land and A. G. Doig, "An Automatic Method of Solving Discrete Programming Problems", Econometrica 28(3), 1960.
  • J. D. C. Little, K. G. Murty, D. W. Sweeney and C. Karel, "An Algorithm for the Traveling Salesman Problem", Operations Research 11(6), 1963.
  • R. Carraghan and P. M. Pardalos, "An Exact Algorithm for the Maximum Clique Problem", Operations Research Letters 9(6), 1990.

The End

Chapter 5: Branch and Bound
Previous: Chapter 4: Backtracking · Next: Chapter 6: Approximation Algorithms
1 / 1 Sercan KÜLCÜ. All rights reserved.