Chapter 6: Approximation Algorithms

Chapter 6: Approximation Algorithms · Study Questions
Approximation ratioTraveling salesman2-opt local searchJob schedulingVertex coverKnapsackPower Iteration
Sercan KÜLCÜ · Giresun University · Computer Engineering

Multiple Choice · Approximation Ratio

Question 1: The Guarantee for Maximization

A 2-approximation algorithm is used for a maximization problem (e.g. knapsack). If the optimal value of an input is C* = 90, what can be said for certain about the value C that the algorithm finds?

C: For maximization, a ρ-approximation means C ≥ C* / ρ: C ≥ 90 / 2 = 45. The algorithm may also find a better value: C can be anything between 45 and 90. So C = 45 and C ≤ 45 are not certain; the only certain upper bound is C ≤ C* = 90. C ≥ 180 applies the minimization formula (C ≤ ρ · C*) backwards.

Multiple Choice · Approximation Ratio

Question 2: Types of Approximation

Which statement about approximation algorithms and heuristics is true?

B: An FPTAS (e.g. knapsack, O(n³/ε)) is polynomial in both n and 1/ε. A PTAS is polynomial only in n and may be exponential in ε (C is reversed). Nearest neighbor has no constant ratio; even for metric TSP it can be Θ(log n). 2-opt reaches only a local optimum.

Classic Question · Approximation Ratio

Question 3: No Constant Ratio for General TSP

Explain, as a proof idea, why for general TSP (without the triangle inequality) there is no polynomial-time ρ-approximation algorithm for any constant ρ ≥ 1 unless P = NP. Use the Hamiltonian cycle problem.

Show answer

Suppose a polynomial-time ρ-approximation A exists. Build a TSP instance from a graph G with n vertices: edges of G get weight 1, and pairs not in G get weight ρn + 1.

abcdeweight ρn + 1bold: Hamiltonian cycle, n edges × 1 = n
  • If a Hamiltonian cycle exists in G, the optimal tour is n; A returns a tour of length at most ρn. That tour cannot use a heavy edge (a single heavy edge already costs ρn + 1 > ρn), so A's tour is a Hamiltonian cycle of G.
  • If no Hamiltonian cycle exists in G, every tour contains at least one heavy edge: length > ρn.

By looking at the tour length returned by A (is it ≤ ρn?), the NP-complete Hamiltonian cycle problem would be solved in polynomial time → P = NP. For metric TSP this construction is invalid because it violates the triangle inequality; there, ratios of 2 and 1.5 are possible.

Step-by-Step Trace · Traveling Salesman

Question 4: Tracing Nearest Neighbor

Rows of the distance matrix of 5 cities: 0: (0 3 4 2 7) · 1: (3 0 4 6 3) · 2: (4 4 0 5 8) · 3: (2 6 5 0 6) · 4: (7 3 8 6 0). Which tour and length does the call nearestNeighbor(d, 0) find?

01234

Start city 0 is visited; tour = [0], total = 0. The matrix is symmetric but not metric.

342701234

k = 1: unvisited cities in row 0: d[0][1] = 3, d[0][2] = 4, d[0][3] = 2, d[0][4] = 7 → the smallest is 3 (2).

201234

tour[1] = 3, total += 2 → 2; now cur = 3.

265601234

k = 2: unvisited cities in row 3: d[3][1] = 6, d[3][2] = 5, d[3][4] = 6 → the smallest is 2 (5).

2501234

tour[2] = 2, total += 5 → 7; now cur = 2.

254801234

k = 3: unvisited cities in row 2: d[2][1] = 4, d[2][4] = 8 → the smallest is 1 (4).

25401234

tour[3] = 1, total += 4 → 11; now cur = 1.

254301234

k = 4: unvisited cities in row 1: d[1][4] = 3 → the smallest is 4 (3).

254301234

tour[4] = 4, total += 3 → 14; now cur = 4.

2543701234

Return to the start: total += d[4][0] = 7 → 21. The tour is 0 → 3 → 2 → 1 → 4 → 0. The optimum by brute force is 19 (0 → 2 → 1 → 4 → 3 → 0): the expensive closing edge (7) stayed in.

int[] nearestNeighbor(int[][] d, int start) {
  int n = d.length, total = 0, cur = start;
  boolean[] visited = new boolean[n];
  int[] tour = new int[n + 1];
  tour[0] = start; visited[start] = true;
  for (int k = 1; k < n; k++) {
    int nearest = -1, minD = Integer.MAX_VALUE;
    for (int j = 0; j < n; j++)
      if (!visited[j] && d[cur][j] < minD) {
        minD = d[cur][j]; nearest = j; }
    tour[k] = nearest; visited[nearest] = true;
    total += minD; cur = nearest;
  }
  tour[n] = start; total += d[cur][start];
  return tour;
}

Output Question · Traveling Salesman

Question 5: From MST to Tour

In a metric TSP instance the minimum spanning tree found by Prim is as shown (w(T) = 19). The tree is traversed in preorder from root A, children in alphabetical order, skipping repeated cities. What tour is found? Which bounds on the tour length and on the optimum OPT hold for certain?

342532ABCDEFG
Show answer

Walking around twice: A B D B E B A C F G F C A (length 2 · 19 = 38).

Preorder tour with shortcuts: A → B → D → E → C → F → G → A.

  • Removing one edge from a tour leaves a spanning tree → w(T) = 19 ≤ OPT.
  • The triangle inequality guarantees that shortcuts never make the path longer → tour ≤ 2 · w(T) = 38 ≤ 2 · OPT.

So 19 ≤ OPT ≤ tour ≤ 38. The actual tour length depends on distances other than the edge weights (e.g. E–C, G–A); the bound still holds.

Multiple Choice · Traveling Salesman

Question 6: Metric TSP Results

Which statement about TSP approximation results is false?

D is false: nearest neighbor is a heuristic; even in the metric case its ratio can be Θ(log n), so there is no constant guarantee. Christofides (MST + perfect matching on the odd-degree vertices) gives 1.5, the MST-based method gives 2; Euclidean TSP has Arora's PTAS.

Multiple Choice · Local Search

Question 7: The Gain of a 2-opt Move

The tour contains edges (a, b) and (c, d); the points are a(0, 0), b(6, 8), c(6, 0), d(0, 8) and the distance is Euclidean. What is the gain Δ = d(a,c) + d(b,d) − d(a,b) − d(c,d) of the 2-opt move, and is the move made?

a(0,0)b(6,8)c(6,0)d(0,8)

A: d(a,b) = d(c,d) = √(36 + 64) = 10, d(a,c) = d(b,d) = 6 → Δ = 6 + 6 − 10 − 10 = −8 < 0, so the tour gets 8 shorter. The crossing edges (dashed) are replaced by the bold edges. +8 is a sign error; −4 comes from connecting with (a,d) and (b,c) (8 + 8 − 20).

Output Question · Local Search

Question 8: The Tour After a 2-opt Move

In a tour with 8 cities, the twoOpt move is accepted for i = 1, j = 5 (reverse(tour, i + 1, j), which reverses both ends inclusive). If before the move tour = {0, 1, 2, 3, 4, 5, 6, 7}, what is tour afterwards; which two edges are removed and which are added?

for (int i = 0; i < n - 1; i++)
  for (int j = i + 2; j < n; j++) {
    int a = tour[i], b = tour[i + 1];
    int c = tour[j], e = tour[(j + 1) % n];
    double delta = d[a][c] + d[b][e]
                - d[a][b] - d[c][e];
    if (delta < -1e-9) {  // shorter
      reverse(tour, i + 1, j);
      improved = true;
    }
  }
Show answer

tour = {0, 1, 5, 4, 3, 2, 6, 7}

0011524334256677tour[2..5] reversed
  • a = tour[1] = 1, b = tour[2] = 2, c = tour[5] = 5, e = tour[6] = 6.
  • Removed edges: (1, 2) and (5, 6); added edges: (1, 5) and (2, 6).
  • The path 2 … 5 in between is traversed in the opposite direction; in symmetric TSP the lengths of the inner edges do not change, so the gain is computed from the four edges alone (O(1)).

Step-by-Step Trace · Job Scheduling

Question 9: Tracing List Scheduling

Graham's list scheduling assigns each job to the currently least loaded machine. For the call listSchedule(new int[]{5, 3, 4, 6, 2, 7}, 3), how do the loads change, and what is Cmax?

M0M1M2036912

Three identical machines are empty: load = [0, 0, 0]. The jobs come in order p = [5, 3, 4, 6, 2, 7].

M0M1M2J1(5)036912

J1 (duration 5): with equal loads the first machine wins → load[0] = 0 + 5 = 5.

M0M1M2J1(5)J2(3)036912

J2 (duration 3): the least loaded machine is M1 (0) → load[1] = 0 + 3 = 3.

M0M1M2J1(5)J2(3)J3(4)036912

J3 (duration 4): the least loaded machine is M2 (0) → load[2] = 0 + 4 = 4.

M0M1M2J1(5)J2(3)J3(4)J4(6)036912

J4 (duration 6): the least loaded machine is M1 (3) → load[1] = 3 + 6 = 9.

M0M1M2J1(5)J2(3)J3(4)J4(6)J5(2)036912

J5 (duration 2): the least loaded machine is M2 (4) → load[2] = 4 + 2 = 6.

M0M1M2J1(5)J2(3)J3(4)J4(6)J5(2)J6(7)036912

J6 (duration 7): the least loaded machine is M0 (5) → load[0] = 5 + 7 = 12.

M0M1M2J1(5)J2(3)J3(4)J4(6)J5(2)J6(7)036912

Cmax = 12. Lower bound: OPT ≥ Σp / m = 27 / 3 = 9; indeed OPT = 9 with {7, 2}, {5, 4}, {6, 3}. Ratio 12/9 ≈ 1.33 ≤ 2 − 1/3.

int listSchedule(int[] p, int m) {
  int[] load = new int[m];  // machine loads
  for (int j = 0; j < p.length; j++) {
    int least = 0;  // first machine to become free
    for (int i = 1; i < m; i++)
      if (load[i] < load[least]) least = i;
    load[least] += p[j];
  }
  int cmax = 0;
  for (int y : load) cmax = Math.max(cmax, y);
  return cmax;
}

Output Question · Job Scheduling

Question 10: LPT Order

For jobs (5, 3, 4, 6, 2, 7) and m = 3, the jobs are first sorted by duration in decreasing order and then listSchedule is called (LPT). What are the machine loads and Cmax? Compare with list scheduling.

Show answer

Order 7, 6, 5, 4, 3, 2:

  • 7 → M0, 6 → M1, 5 → M2 (loads 7, 6, 5)
  • 4 → M2 (5) → 9; 3 → M1 (6) → 9; 2 → M0 (7) → 9

Loads [9, 9, 9], Cmax = 9 = OPT (lower bound 27/3 = 9). List scheduling in the given order finds 12 (loads [12, 9, 6]): the long job (7) left for last created an imbalance. In LPT the short jobs at the end repair the imbalance; ratio ≤ 4/3 − 1/(3m) = 11/9.

M0M1M27654329

Multiple Choice · Job Scheduling

Question 11: The Ratio of List Scheduling

Which approximation ratio is guaranteed on m = 4 identical machines by list scheduling (jobs in the given order)?

A: List scheduling is a (2 − 1/m)-approximation: 2 − 1/4 = 1.75. 1.25 is the bound of LPT for m = 4 (4/3 − 1/12); 1.33 is 4/3 independent of m; 2.00 is the limit as m → ∞.

Step-by-Step Trace · Vertex Cover

Question 12: The Vertex Cover 2-Approximation

vertexCover processes the edges of the graph in the figure in the given order. How does the cover grow, and how does the result compare with the optimum?

ABCDEF

The cover is empty. Edges in order: (A,B), (A,C), (B,D), (C,D), (C,E), (D,F), (E,F).

ABCDEF

(A, B): neither endpoint is in the cover → A and B are added. The edge joins the matching.

ABCDEF

(A, C): A is already in the cover → the edge is covered, skipped.

ABCDEF

(B, D): B is already in the cover → the edge is covered, skipped.

ABCDEF

(C, D): neither endpoint is in the cover → C and D are added. The edge joins the matching.

ABCDEF

(C, E): C is already in the cover → the edge is covered, skipped.

ABCDEF

(D, F): D is already in the cover → the edge is covered, skipped.

ABCDEF

(E, F): neither endpoint is in the cover → E and F are added. The edge joins the matching.

ABCDEF

Result: 6 vertices (all of them). The optimum is 3: {A, D, E} (green). Ratio 6/3 = 2: the bound is tight on this input. The chosen edges (A,B), (C,D), (E,F) form a matching with no common endpoints.

Set<Character> vertexCover(char[][] edges) {
  Set<Character> cover = new HashSet<>();
  for (char[] k : edges) {
    char u = k[0], v = k[1];
    if (!cover.contains(u) && !cover.contains(v)) {
      cover.add(u); cover.add(v); }  // both endpoints
  }
  return cover;
}

Multiple Choice · Vertex Cover

Question 13: Edge Order on a Path

The edges of the path 1–2–3–4–5–6–7–8 are given to vertexCover in this order: (2,3), (4,5), (6,7), (1,2), (3,4), (5,6), (7,8). How many vertices does the algorithm choose, and what is the optimum?

12345678

C: (2,3), (4,5), (6,7) are chosen → {2, 3, 4, 5, 6, 7}; each of the remaining four edges already has one endpoint covered. The optimum is 4 (e.g. {2, 4, 6, 7}); on a path with 7 edges, 3 vertices cover at most 6 edges. 8 vertices would result if the edges were given in path order: (1,2), (3,4), (5,6), (7,8).

Multiple Choice · Vertex Cover

Question 14: Reduction from 3-SAT

In the 3-SAT → vertex cover reduction (one edge per variable, one triangle per clause), for a formula with l = 3 variables and m = 4 clauses, what is the target cover size k?

C: k = l + 2m = 3 + 8 = 11: one endpoint from each variable edge, two vertices from each clause triangle. 7 = l + m (assuming one vertex per triangle), 15 = l + 3m (the whole triangle), 14 = 2l + 2m. The graph has 2l + 3m = 18 vertices.

Output Question · Knapsack

Question 15: Greedy by Ratio

Capacity W = 15. Items (value, weight): I1 (30, 10), I2 (20, 8), I3 (18, 7), I4 (6, 4). Taking items in decreasing order of value/weight ratio with the rule take if it fits, which items are chosen and what is the total value? What do the optimum and the improved rule max(greedy, most valuable single item) give?

Show answer
OrderItemRatioDecisionWeight
1I13.00taken10
2I32.57does not fit (17)10
3I22.50does not fit (18)10
4I41.50taken14

Greedy: I1 + I4 = 36. Optimum (DP): I2 + I3 = 38 (weight 15, completely full).

Improved rule: max(36, 30) = 36 ≥ 38/2 → the 2-approximation guarantee holds. In the 0/1 version, sorting by ratio is not optimal; in the fractional version it is (I1 + 5/7 · I3 = 30 + 12.86 = 42.86).

Multiple Choice · Knapsack

Question 16: The Improved Rule

W = 20; items (value, weight): (3, 1) and (20, 20). What values do ratio-ordered greedy and the rule max(greedy value, most valuable single item that fits) find, respectively?

D: The ratios are 3 and 1; greedy takes (3, 1) first and (20, 20) no longer fits → 3. The improved rule gives max(3, 20) = 20 = optimum. 23 would mean taking both items together (weight 21 > 20). In this example the ratio of pure greedy is 20/3, and it gets arbitrarily worse as W grows.

Step-by-Step Trace · Power Iteration

Question 17: Tracing Power Iteration

power(new double[][]{{2, 1}, {1, 3}}, 3) call: how do y, ‖y‖ and x change at each step?

v₁xx₁x₂

A = [[2, 1], [1, 3]], x = [1, 1] (as a direction in the figure). The dominant eigenvector is v₁ ≈ (0.526, 0.851), λ₁ = (5 + √5)/2 ≈ 3.618.

v₁yxx₁x₂

k = 0: y = A·x = (3.0000, 4.0000), ‖y‖ = 5.0000.

v₁xx₁x₂

Normalization: x = y / ‖y‖ = (0.6000, 0.8000). The length becomes 1.

v₁yxx₁x₂

k = 1: y = A·x = (2.0000, 3.0000), ‖y‖ = 3.6056 (= √13).

v₁xx₁x₂

Normalization: x = y / ‖y‖ = (0.5547, 0.8321). x approaches v₁.

v₁yxx₁x₂

k = 2: y = A·x = (1.9415, 3.0509), ‖y‖ = 3.6162.

v₁xx₁x₂

Normalization: x = y / ‖y‖ = (0.5369, 0.8437). x approaches v₁.

v₁xx₁x₂

After 3 steps x ≈ (0.5369, 0.8437) ≈ v₁; the estimate ‖A·x‖ goes 5 → 3.606 → 3.616 → approaching λ₁ = 3.618. The error shrinks at each step by a factor of ≈ λ₂/λ₁ = 1.382/3.618 ≈ 0.38.

double[] power(double[][] a, int steps) {
  int n = a.length;
  double[] x = new double[n];
  Arrays.fill(x, 1.0); // x₀ ≠ 0 and not orthogonal to v₁
  for (int k = 0; k < steps; k++) {
    double[] y = new double[n];
    for (int i = 0; i < n; i++)  // y = A·x
      for (int j = 0; j < n; j++)
        y[i] += a[i][j] * x[j];
    double norm = 0;
    for (double yi : y) norm += yi * yi;
    norm = Math.sqrt(norm);
    if (norm == 0) break; // A·x = 0
    for (int i = 0; i < n; i++)  // ‖x‖ = 1
      x[i] = y[i] / norm;
  }
  return x;
}

Multiple Choice · Power Iteration

Question 18: The Fastest Convergence

The two eigenvalues of largest absolute value (λ₁, λ₂) of four symmetric matrices are given. For which one does Power Iteration converge the fastest?

B: The error shrinks at each step by a factor of ≈ |λ₂ / λ₁|: 0.9; 0.25; 0.97; 0.6. The smallest ratio, 0.25, is the fastest. A large λ₁ (100) does not bring speed by itself; what matters is the ratio. For a negative λ₂ the absolute value is taken: |−2.9 / 3| ≈ 0.97 is very slow.

The End

Chapter 6: Approximation Algorithms: Study Questions
Lecture slides: Chapter 6: Approximation Algorithms · Lecture notes
1 / 1 Sercan KÜLCÜ. All rights reserved.