Chapter 3: Dynamic Programming

Advanced Algorithms
Overlapping subproblemsMemoization and tabulationFibonacci0/1 and unbounded knapsack Subset sumLongest increasing subsequenceLongest common substringLongest common subsequence
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Recognition

Spot overlapping subproblems and optimal substructure in a problem; decide when dynamic programming applies.

Formulation

Define the subproblem (state), write the recurrence and base cases, and fix the order in which the table is filled.

Implementation

Fill the tables for Fibonacci, 0/1 and unbounded knapsack, subset sum, LIS, longest common substring, and LCS by hand, and code them in Java.

Analysis

Compare memoization with tabulation; justify the O(n · W), O(m · n), O(n²), and O(n log n) bounds; reconstruct the solution from the table.

Basic Concepts

What Is Dynamic Programming?

  • Complex problems sometimes call for clever methods; dynamic programming is one of them.
  • It breaks a big problem into small, recurring subproblems and solves them step by step.
  • Each subproblem's solution is written into a table; when the same subproblem comes up again, the answer is read from the table, which saves time.
  • It is widely used for optimization problems (largest value, shortest path, fewest operations).
In short

DP = recursion + remembering. Every DP problem can also be solved by plain recursion; DP makes sure the same work is never done twice.

The name comes from Richard Bellman (1950s); "programming" here means planning with a table, not writing code.

Basic Concepts

Two Key Properties for DP

1. Overlapping subproblems

The recursive solution calls the same subproblem again and again. Example: computing fib(5) computes fib(3) twice and fib(2) three times.

If the subproblems are independent (as in merge sort), a table does not help; that is divide and conquer.

f(5)×1f(4)×1f(3)×2f(2)×3f(1)×5
2. Optimal substructure

An optimal solution of the problem can be built from optimal solutions of its subproblems.

Bellman's principle of optimality: every subsolution contained in an optimal solution is itself optimal for its subproblem.

A Bellman equation example (maximum subarray): if V(i) is the largest sum ending at i, then V(i) = max(a[i], V(i−1) + a[i]). The value is computed from the optimal value of the previous subproblem.

Basic Concepts

Steps in Designing a DP Solution

  1. Study the structure of an optimal solution: which subproblems must be solved?
  2. Write the state and recurrence: express the solution in terms of smaller subproblems (recurrence + base cases).
  3. Fix the order of computation: when a cell is computed, the cells it needs must be ready.
  4. Fix the table size: how many dimensions, how many values each? (e.g., (n+1) × (W+1))
  5. Fill the table: bottom-up (tabulation) or top-down (memoization).
  6. Reconstruct the solution: not just the value; the chosen items are found by reading the table backward.

Total time ≈ (number of states) × (time per state). This rule explains every bound in this chapter.

Basic Concepts

Memoization and Tabulation

Top-down: memoization
  • Write the recursive solution as is; store the results in a cache (array / HashMap).
  • Before a call, check the table: if the value is there, return it; otherwise compute and store it.
  • Only the subproblems actually needed are solved.
  • Deep recursion risks a stack overflow (StackOverflowError).
f0f1f2f3f4f5the f(5) call goes down
Bottom-up: tabulation
  • Start from the smallest subproblem and fill the table in order with loops.
  • No recursion and no call overhead; usually faster.
  • The fill order must be worked out correctly in advance.
  • If only the previous row is needed, memory can be reduced (e.g., O(W)).
f0f1f2f3f4f5f0, f1, … fill in order
Both solve each subproblem once; the asymptotic time is the same. They differ in order and style.

Basic Concepts

DP, Divide and Conquer, and Greedy

Divide and ConquerDynamic ProgrammingGreedy
Subproblemsindependentoverlappinga single choice per step
Choicenone; divide and combinetry every option, store the best in the tabletake the current best, never go back
Optimality—guaranteed if optimal substructure holdsonly with the greedy-choice property
Examplemerge sort, binary search0/1 knapsack, LCS, Bellman-Fordfractional knapsack, Dijkstra, Kruskal
Costrecurrence, master theoremnumber of states × work per stateusually O(n log n): sort + one pass
Memoryrecursion stackthe table (often O(n · W) or O(m · n))usually O(1) extra memory

Greedy can be seen as dynamic programming that looks at only one option per step: if that choice can be proved correct, no table is needed.

Basic Concepts

Applications of Dynamic Programming

Shortest pathsGPS devices and maps: Bellman-Ford, Floyd-Warshall.
String comparisondiff, spell checking: LCS, edit distance.
BioinformaticsDNA/protein alignment (Needleman-Wunsch, Smith-Waterman), RNA structure prediction.
Resource allocationKnapsack, budget and investment planning, production-line scheduling.
Compilers and numericsMatrix-chain multiplication, optimal binary search trees, parsing (CYK).
Artificial intelligenceChess-playing computers, machine translation, reinforcement learning (the Bellman equation).

Fibonacci

Fibonacci: Plain Recursion

The Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, …

F(0) = 0, F(1) = 1
F(n) = F(n−1) + F(n−2), n ≥ 2
static long fib(int n) {
  if (n <= 1)
    return n;         // base cases
  return fib(n - 1) + fib(n - 2);
}
  • A literal copy of the definition; correct but very slow.
  • Number of calls: T(n) = T(n−1) + T(n−2) + 1 → Θ(φⁿ), φ ≈ 1.618 (the golden ratio).
  • fib(40) makes about 331 million calls; fib(50) takes minutes.
  • Why? The same subproblems (fib(3), fib(2), …) are computed again and again: overlapping subproblems.

Fibonacci

The fib(5) Recursion Tree: Call Order

Orange: running call · blue: calls waiting on the stack · green: finished call (with its value).

f(5)f(4)f(3)f(2)f(1)f(0)f(1)f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(5) called. Stack: fib(5)

f(5)f(4)f(3)f(2)f(1)f(0)f(1)f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(4) called. Stack: fib(5) → fib(4)

f(5)f(4)f(3)f(2)f(1)f(0)f(1)f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(3) called. Stack: fib(5) → fib(4) → fib(3)

f(5)f(4)f(3)f(2)f(1)f(0)f(1)f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(2) called. Stack: fib(5) → fib(4) → fib(3) → fib(2)

f(5)f(4)f(3)f(2)f(1)f(0)f(1)f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(1) base case: returns 1 at once. Stack: fib(5) → fib(4) → fib(3) → fib(2) → fib(1)

f(5)f(4)f(3)f(2)f(1)=1f(0)f(1)f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(0) base case: returns 0 at once. Stack: fib(5) → fib(4) → fib(3) → fib(2) → fib(0)

f(5)f(4)f(3)f(2)=1f(1)=1f(0)=0f(1)f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(1) base case: returns 1 at once. Stack: fib(5) → fib(4) → fib(3) → fib(1)

f(5)f(4)f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(2) called — computed for the 2nd time! Stack: fib(5) → fib(4) → fib(2)

f(5)f(4)f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)f(1)f(0)f(3)f(2)f(1)f(0)f(1)

fib(1) base case: returns 1 at once. Stack: fib(5) → fib(4) → fib(2) → fib(1)

f(5)f(4)f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)f(1)=1f(0)f(3)f(2)f(1)f(0)f(1)

fib(0) base case: returns 0 at once. Stack: fib(5) → fib(4) → fib(2) → fib(0)

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(1)=1f(0)=0f(3)f(2)f(1)f(0)f(1)

fib(3) called — computed for the 2nd time! Stack: fib(5) → fib(3)

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(1)=1f(0)=0f(3)f(2)f(1)f(0)f(1)

fib(2) called — computed for the 3rd time! Stack: fib(5) → fib(3) → fib(2)

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(1)=1f(0)=0f(3)f(2)f(1)f(0)f(1)

fib(1) base case: returns 1 at once. Stack: fib(5) → fib(3) → fib(2) → fib(1)

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(1)=1f(0)=0f(3)f(2)f(1)=1f(0)f(1)

fib(0) base case: returns 0 at once. Stack: fib(5) → fib(3) → fib(2) → fib(0)

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(1)=1f(0)=0f(3)f(2)=1f(1)=1f(0)=0f(1)

fib(1) base case: returns 1 at once. Stack: fib(5) → fib(3) → fib(1)

f(5)=5f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(1)=1f(0)=0f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1

Result fib(5) = 5, 15 calls in total. Red boxes are repeats: fib(3) was computed 2 times, fib(2) 3 times, fib(1) 5 times.

Fibonacci

The Cost of Repeated Work

nF(n)plain recursive callsmemoized calls
55159
105517719
206,76521,89139
30832,0402,692,53759
40102,334,155331,160,28179
  • Plain recursion makes 2·F(n+1) − 1 calls: it grows as fast (exponentially) as the Fibonacci numbers themselves.
  • Yet there are only n+1 distinct subproblems: fib(0), fib(1), …, fib(n).
  • Solving each once and storing it cuts the calls to 2n − 1: from exponential to linear.
The essence of DP: when the number of distinct subproblems is small but the recursion calls them many times, storing results pays off hugely.

Fibonacci

Memoization: fib(5) Step by Step

f(5)memo—0—102030405

fib(5): not in the table; first call fib(4).

f(5)f(4)memo—0—102030405

fib(4): not in the table; first call fib(3).

f(5)f(4)f(3)memo—0—102030405

fib(3): not in the table; first call fib(2).

f(5)f(4)f(3)f(2)memo—0—102030405

fib(2): not in the table; first call fib(1).

f(5)f(4)f(3)f(2)f(1)=1memo—0—102030405

fib(1): base case, returns 1.

f(5)f(4)f(3)f(2)f(1)=1memo—0—102030405

fib(1) returned (a = 1); now call fib(0).

f(5)f(4)f(3)f(2)f(1)=1f(0)=0memo—0—102030405

fib(0): base case, returns 0.

f(5)f(4)f(3)f(2)=1f(1)=1f(0)=0memo—0—112030405

memo[2] = 1 + 0 = 1 is stored in the table and returned.

f(5)f(4)f(3)f(2)=1f(1)=1f(0)=0memo—0—112030405

fib(2) returned (a = 1); now call fib(1).

f(5)f(4)f(3)f(2)=1f(1)=1f(0)=0f(1)=1memo—0—112030405

fib(1): base case, returns 1.

f(5)f(4)f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1memo—0—112230405

memo[3] = 1 + 1 = 2 is stored in the table and returned.

f(5)f(4)f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1memo—0—112230405

fib(3) returned (a = 2); now call fib(2).

f(5)f(4)f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1memo—0—112230405

fib(2): memo[2] = 1 is already in the table; no recomputation.

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1memo—0—112233405

memo[4] = 2 + 1 = 3 is stored in the table and returned.

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1memo—0—112233405

fib(4) returned (a = 3); now call fib(3).

f(5)f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(3)=2memo—0—112233405

fib(3): memo[3] = 2 is already in the table; no recomputation.

f(5)=5f(4)=3f(3)=2f(2)=1f(1)=1f(0)=0f(1)=1f(2)=1f(3)=2memo—0—112233455

memo[5] = 3 + 2 = 5 is stored in the table and returned.

static long[] memo = new long[50];

static long fib(int n) {
  if (n <= 1) return n;          // base case
  if (memo[n] != 0) return memo[n];
  long a = fib(n - 1);
  long b = fib(n - 2);
  memo[n] = a + b;
  return memo[n];
}

Fibonacci

Tabulation: fibonacci(9) Step by Step

00010203040506070809

The fib array is created with n + 1 = 10 elements (all 0 initially in Java).

001123456789

The base cases are written: fib[0] = 0, fib[1] = 1. Empty cells are not computed yet.

0011123456789+

i = 2: fib[2] = fib[1] + fib[0] = 1 + 0 = 1

00111223456789+

i = 3: fib[3] = fib[2] + fib[1] = 1 + 1 = 2

001112233456789+

i = 4: fib[4] = fib[3] + fib[2] = 2 + 1 = 3

0011122334556789+

i = 5: fib[5] = fib[4] + fib[3] = 3 + 2 = 5

00111223345586789+

i = 6: fib[6] = fib[5] + fib[4] = 5 + 3 = 8

0011122334558613789+

i = 7: fib[7] = fib[6] + fib[5] = 8 + 5 = 13

001112233455861372189+

i = 8: fib[8] = fib[7] + fib[6] = 13 + 8 = 21

00111223345586137218349+

i = 9: fib[9] = fib[8] + fib[7] = 21 + 13 = 34

00111223345586137218349

The loop ends; fib[9] = 34 is returned. Each cell was computed once: O(n) time, O(n) memory.

static int fibonacci(int n) {
  if (n <= 1) return n;   // n = 0 and n = 1
  int[] fib = new int[n + 1];
  fib[0] = 0;   // base cases
  fib[1] = 1;
  for (int i = 2; i <= n; i++)
    fib[i] = fib[i - 1] + fib[i - 2];
  return fib[n];
}

Fibonacci

Fibonacci: O(1) Memory and Comparison

static long fib(int n) {
  if (n <= 1) return n;
  long prev = 0, curr = 1;
  for (int i = 2; i <= n; i++) {
    long next = prev + curr;
    prev = curr;
    curr = next;
  }
  return curr;
}
MethodTimeExtra memory
Plain recursionΘ(φⁿ)O(n) stack
MemoizationO(n)O(n) table + stack
TabulationO(n)O(n) table
Two variablesO(n)O(1)
Saving memory: fib[i] depends only on the last two values; there is no need to store the whole array. The same idea will return in the knapsack as "only the previous row".

0/1 Knapsack

The 0/1 Knapsack Problem

  • n items: each item has a value vᵢ and a weight (size) wᵢ.
  • The knapsack has capacity W.
  • Each item is either taken whole (1) or not taken (0); it cannot be split.
  • Goal: maximize the total value subject to total weight ≤ W.
15 kgcapacity4 ₺12 kg2 ₺2 kg2 ₺1 kg1 ₺1 kg10 ₺4 kg
Question: Which items should go into the knapsack in the picture?
AnswerAll except the 12 kg item: 2 + 2 + 1 + 10 = 15 ₺, 8 kg in total. Any choice that includes the 12 kg item is worth at most 4 + 2 + 2 = 8 ₺.

0/1 Knapsack

Why Not Brute Force or Greedy?

Brute force

2 options per item → 2ⁿ subsets. For n = 50, more than 10¹⁵ subsets: impossible in practice.

Greedy (value/weight ratio)

W = 50; items (60 ₺, 10 kg), (100 ₺, 20 kg), (120 ₺, 30 kg). Ratios 6, 5, 4 → greedy takes the first two: 160 ₺. Optimal: second + third = 220 ₺.

greedy10 kg20 kgoptimal20 kg30 kg160 ₺220 ₺W = 50
The greedy choice is optimal only when items can be split (fractional knapsack). In the 0/1 case, taking an item changes the later options: both choices must be tried → dynamic programming.

0/1 Knapsack

0/1 Knapsack: State and Recurrence

State: K[i][c] = the largest value achievable with the first i items and capacity c.

K[0][c] = 0, K[i][0] = 0
if wᵢ > c: K[i][c] = K[i−1][c]
otherwise: K[i][c] = max(K[i−1][c], vᵢ + K[i−1][c − wᵢ])
  • Skip it: the capacity stays the same; the best with the earlier items.
  • Take it: gain vᵢ and fill the remaining capacity c − wᵢ as well as possible with the earlier items.
  • Answer: K[n][W]. The table is (n+1) × (W+1), filled row by row.
K[i−1][c−wᵢ]K[i−1][c]K[i][c]skiptake: + vᵢrow i−1 (earlier items)row i

0/1 Knapsack

0/1 Knapsack: Filling the Table (W = 6)

c (capacity) →ivw0123456000000001540243033204210

The table is (n+1) × (W+1) = 5 × 7. Row 0 (no items) and column 0 (no capacity) are 0.

c (capacity) →ivw01234560000000015400243033204210

i=1, c=1: w=4 > 1, the item does not fit → copy the value above: K[1][1] = 0

c (capacity) →ivw012345600000000154000243033204210

i=1, c=2: w=4 > 2, the item does not fit → copy the value above: K[1][2] = 0

c (capacity) →ivw0123456000000001540000243033204210

i=1, c=3: w=4 > 3, the item does not fit → copy the value above: K[1][3] = 0

c (capacity) →ivw01234560000000015400005243033204210

i=1, c=4: max(skip = 0, take = 5 + K[0][0] = 5) = 5 (take)

c (capacity) →ivw012345600000000154000055243033204210

i=1, c=5: max(skip = 0, take = 5 + K[0][1] = 5) = 5 (take)

c (capacity) →ivw0123456000000001540000555243033204210

i=1, c=6: max(skip = 0, take = 5 + K[0][2] = 5) = 5 (take)

c (capacity) →ivw01234560000000015400005552430033204210

i=2, c=1: w=3 > 1, the item does not fit → copy the value above: K[2][1] = 0

c (capacity) →ivw012345600000000154000055524300033204210

i=2, c=2: w=3 > 2, the item does not fit → copy the value above: K[2][2] = 0

c (capacity) →ivw0123456000000001540000555243000433204210

i=2, c=3: max(skip = 0, take = 4 + K[1][0] = 4) = 4 (take)

c (capacity) →ivw01234560000000015400005552430004533204210

i=2, c=4: max(skip = 5, take = 4 + K[1][1] = 4) = 5 (skip)

c (capacity) →ivw012345600000000154000055524300045533204210

i=2, c=5: max(skip = 5, take = 4 + K[1][2] = 4) = 5 (skip)

c (capacity) →ivw0123456000000001540000555243000455533204210

i=2, c=6: max(skip = 5, take = 4 + K[1][3] = 4) = 5 (skip)

c (capacity) →ivw01234560000000015400005552430004555332004210

i=3, c=1: w=2 > 1, the item does not fit → copy the value above: K[3][1] = 0

c (capacity) →ivw012345600000000154000055524300045553320034210

i=3, c=2: max(skip = 0, take = 3 + K[2][0] = 3) = 3 (take)

c (capacity) →ivw0123456000000001540000555243000455533200344210

i=3, c=3: max(skip = 4, take = 3 + K[2][1] = 3) = 4 (skip)

c (capacity) →ivw01234560000000015400005552430004555332003454210

i=3, c=4: max(skip = 5, take = 3 + K[2][2] = 3) = 5 (skip)

c (capacity) →ivw012345600000000154000055524300045553320034574210

i=3, c=5: max(skip = 5, take = 3 + K[2][3] = 7) = 7 (take)

c (capacity) →ivw0123456000000001540000555243000455533200345784210

i=3, c=6: max(skip = 5, take = 3 + K[2][4] = 8) = 8 (take)

c (capacity) →ivw01234560000000015400005552430004555332003457842102

i=4, c=1: max(skip = 0, take = 2 + K[3][0] = 2) = 2 (take)

c (capacity) →ivw012345600000000154000055524300045553320034578421023

i=4, c=2: max(skip = 3, take = 2 + K[3][1] = 2) = 3 (skip)

c (capacity) →ivw0123456000000001540000555243000455533200345784210235

i=4, c=3: max(skip = 4, take = 2 + K[3][2] = 5) = 5 (take)

c (capacity) →ivw01234560000000015400005552430004555332003457842102356

i=4, c=4: max(skip = 5, take = 2 + K[3][3] = 6) = 6 (take)

c (capacity) →ivw012345600000000154000055524300045553320034578421023567

i=4, c=5: max(skip = 7, take = 2 + K[3][4] = 7) = 7 (skip)

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

i=4, c=6: max(skip = 8, take = 2 + K[3][5] = 9) = 9 (take)

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

The table is full. The answer is K[4][6] = 9. 4 × 6 = 24 cells in total, each O(1).

int knapsack(int[] v, int[] w, int W) {
  int n = v.length;
  int[][] K = new int[n + 1][W + 1];
  for (int i = 1; i <= n; i++)
    for (int c = 1; c <= W; c++) {
      int vi = v[i - 1], wi = w[i - 1];
      if (wi > c)            // does not fit
        K[i][c] = K[i - 1][c];
      else                   // skip / take
        K[i][c] = Math.max(K[i - 1][c],
                    vi + K[i - 1][c - wi]);
    }
  return K[n][W];
}

0/1 Knapsack

Reconstruction: Which Items Were Taken?

Rule: if K[i][c] ≠ K[i−1][c], item i was taken (the value rose only by taking it) → c ← c − wᵢ; if equal, it was not taken.

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

Reconstruction starts at cell K[4][6] = 9. Which items were taken?

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

K[4][6] = 9 ≠ K[3][6] = 8 → item 4 taken (v=2, w=1); c = 6 − 1 = 5

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

K[3][5] = 7 ≠ K[2][5] = 5 → item 3 taken (v=3, w=2); c = 5 − 2 = 3

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

K[2][3] = 4 ≠ K[1][3] = 0 → item 2 taken (v=4, w=3); c = 3 − 3 = 0

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

K[1][0] = K[0][0] = 0 → item 1 not taken; c is unchanged.

c (capacity) →ivw0123456000000001540000555243000455533200345784210235679

Items chosen: 2, 3, 4 → value 4 + 3 + 2 = 9, weight 3 + 2 + 1 = 6 = W.

0/1 Knapsack

0/1 Knapsack: Complexity and Memory

  • Time: O(n · W), memory: O(n · W) — (n+1)(W+1) cells, each O(1).
  • This time is pseudo-polynomial: it depends on the value of W, not on its bit length. With W = 10⁹, the table cannot be built.
  • 0/1 knapsack is NP-hard; DP is practical only when W is small.
  • Saving memory (code on the right): row i uses only row i−1 → a one-dimensional array, O(W). The capacity must be scanned from high to low; otherwise the same item is taken twice.
  • For reconstruction, the one-dimensional version also keeps a boolean[][] taken table.
int[] dp = new int[W + 1];
for (int i = 0; i < n; i++)
  for (int c = W; c >= w[i]; c--)
    dp[c] = Math.max(dp[c],
                     v[i] + dp[c - w[i]]);
return dp[W];

0/1 Knapsack · Run the code step by step

0/1 Knapsack: With a Single Row (W = 6)

dp00010203040506itemv=5 w=4v=4 w=3v=3 w=2v=2 w=1

Items (v, w): (5, 4), (4, 3), (3, 2), (2, 1); W = 6. One row instead of the 2D table: dp[c] = the best value with capacity c using the items so far.

dp00010203545556before0000000c: W → 4itemv=5 w=4v=4 w=3v=3 w=2v=2 w=1

Item 0 (v = 5, w = 4), c from 6 down to 4: dp[6]: 0 → 5, dp[5]: 0 → 5, dp[4]: 0 → 5. Thanks to the decreasing order, dp[c − 4] is always the old value (0): the item cannot be taken twice.

dp00010243545556before0000555c: W → 3itemv=5 w=4v=4 w=3v=3 w=2v=2 w=1

Item 1 (v = 4, w = 3), c from 6 down to 3: dp[3]: 0 → 4.

dp00013243547586before0004555c: W → 2itemv=5 w=4v=4 w=3v=3 w=2v=2 w=1

Item 2 (v = 3, w = 2), c from 6 down to 2: dp[6]: 5 → 8, dp[5]: 5 → 7, dp[2]: 0 → 3.

dp00213253647596before0034578c: W → 1itemv=5 w=4v=4 w=3v=3 w=2v=2 w=1

Item 3 (v = 2, w = 1), c from 6 down to 1: dp[6]: 8 → 9, dp[4]: 5 → 6, dp[3]: 4 → 5, dp[1]: 0 → 2.

dp00213253647596itemv=5 w=4v=4 w=3v=3 w=2v=2 w=1

dp[6] = 9 (items 1, 2, 3: 4 + 3 + 2), the same as the bottom-right cell of the 2D table; memory O(W) instead of O(n · W).

int[] dp = new int[W + 1];
for (int i = 0; i < n; i++)
  for (int c = W; c >= w[i]; c--)
    dp[c] = Math.max(dp[c],
          v[i] + dp[c - w[i]]);
return dp[W];

Unbounded Knapsack

The Unbounded Knapsack

  • Each item can be taken as many times as you like (unlimited copies); otherwise as in 0/1.
  • State: dp[i][c] = the largest profit with item types 0..i and capacity c.
dp[i][c] = max( dp[i−1][c] , profitᵢ + dp[i][c − weightᵢ] ) (if weightᵢ ≤ c)
The only difference from 0/1"Take it" looks at the same row (i), not the row above (i−1): a taken item can be taken again.
i−1isame row: c − weightᵢ
ExampleProfit = [15, 50, 60, 90], weight = [1, 3, 4, 5], capacity = 8. The table is 4 × 9; the first row can use only item 0 (profit 15, 1 kg).

Row indices start at 0 (take dp[−1][c] = 0).

Unbounded Knapsack

Unbounded Knapsack: Filling the Table

capacity →profitwt.i01234567815100503106042090530

The capacity-0 column is 0: no item fits.

capacity →profitwt.i0123456781510015503106042090530

i=0, c=1: only item 0 is available; its weight 1 fits → 15 + dp[0][0] = 15.

capacity →profitwt.i01234567815100153045607590105120503106042090530

i=0, c=2..8: dp[0][c] = 15 + dp[0][c−1] → c copies of item 0: 30, 45, …, 120. The same row is reused!

capacity →profitwt.i0123456781510015304560759010512050310156042090530

i=1, c=1: weight 3 > 1, does not fit → dp[1][1] = dp[0][1] = 15

capacity →profitwt.i012345678151001530456075901051205031015306042090530

i=1, c=2: weight 3 > 2, does not fit → dp[1][2] = dp[0][2] = 30

capacity →profitwt.i01234567815100153045607590105120503101530506042090530

i=1, c=3: max(dp[0][3] = 45, 50 + dp[1][0] = 50) = 50

capacity →profitwt.i0123456781510015304560759010512050310153050656042090530

i=1, c=4: max(dp[0][4] = 60, 50 + dp[1][1] = 65) = 65

capacity →profitwt.i012345678151001530456075901051205031015305065806042090530

i=1, c=5: max(dp[0][5] = 75, 50 + dp[1][2] = 80) = 80

capacity →profitwt.i012345678151001530456075901051205031015305065801006042090530

i=1, c=6: max(dp[0][6] = 90, 50 + dp[1][3] = 100) = 100

capacity →profitwt.i012345678151001530456075901051205031015305065801001156042090530

i=1, c=7: max(dp[0][7] = 105, 50 + dp[1][4] = 115) = 115

capacity →profitwt.i012345678151001530456075901051205031015305065801001151306042090530

i=1, c=8: max(dp[0][8] = 120, 50 + dp[1][5] = 130) = 130

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201590530

i=2, c=1: weight 4 > 1, does not fit → dp[2][1] = dp[1][1] = 15

capacity →profitwt.i0123456781510015304560759010512050310153050658010011513060420153090530

i=2, c=2: weight 4 > 2, does not fit → dp[2][2] = dp[1][2] = 30

capacity →profitwt.i012345678151001530456075901051205031015305065801001151306042015305090530

i=2, c=3: weight 4 > 3, does not fit → dp[2][3] = dp[1][3] = 50

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201530506590530

i=2, c=4: max(dp[1][4] = 65, 60 + dp[2][0] = 60) = 65

capacity →profitwt.i0123456781510015304560759010512050310153050658010011513060420153050658090530

i=2, c=5: max(dp[1][5] = 80, 60 + dp[2][1] = 75) = 80

capacity →profitwt.i0123456781510015304560759010512050310153050658010011513060420153050658010090530

i=2, c=6: max(dp[1][6] = 100, 60 + dp[2][2] = 90) = 100

capacity →profitwt.i0123456781510015304560759010512050310153050658010011513060420153050658010011590530

i=2, c=7: max(dp[1][7] = 115, 60 + dp[2][3] = 110) = 115

capacity →profitwt.i0123456781510015304560759010512050310153050658010011513060420153050658010011513090530

i=2, c=8: max(dp[1][8] = 130, 60 + dp[2][4] = 125) = 130

capacity →profitwt.i012345678151001530456075901051205031015305065801001151306042015305065801001151309053015

i=3, c=1: weight 5 > 1, does not fit → dp[3][1] = dp[2][1] = 15

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201530506580100115130905301530

i=3, c=2: weight 5 > 2, does not fit → dp[3][2] = dp[2][2] = 30

capacity →profitwt.i0123456781510015304560759010512050310153050658010011513060420153050658010011513090530153050

i=3, c=3: weight 5 > 3, does not fit → dp[3][3] = dp[2][3] = 50

capacity →profitwt.i012345678151001530456075901051205031015305065801001151306042015305065801001151309053015305065

i=3, c=4: weight 5 > 4, does not fit → dp[3][4] = dp[2][4] = 65

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201530506580100115130905301530506590

i=3, c=5: max(dp[2][5] = 80, 90 + dp[3][0] = 90) = 90

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201530506580100115130905301530506590105

i=3, c=6: max(dp[2][6] = 100, 90 + dp[3][1] = 105) = 105

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201530506580100115130905301530506590105120

i=3, c=7: max(dp[2][7] = 115, 90 + dp[3][2] = 120) = 120

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201530506580100115130905301530506590105120140

i=3, c=8: max(dp[2][8] = 130, 90 + dp[3][3] = 140) = 140

capacity →profitwt.i01234567815100153045607590105120503101530506580100115130604201530506580100115130905301530506590105120140

Answer dp[3][8] = 140: 90 + dp[3][3] = 90 + 50 → one item 3 (90, 5 kg) + one item 1 (50, 3 kg).

Unbounded Knapsack

Unbounded Knapsack: A One-Dimensional Solution

static int unboundedKnapsack(int[] w, int[] v, int W) {
  int[] dp = new int[W + 1];       // dp[c]: best with capacity c
  for (int c = 1; c <= W; c++)
    for (int j = 0; j < w.length; j++)
      if (w[j] <= c)
        dp[c] = Math.max(dp[c], dp[c - w[j]] + v[j]);
  return dp[W];
}
  • The rows can be merged: dp[c] = max(dp[c − wⱼ] + vⱼ) over all items.
  • For the example, dp = [0, 15, 30, 50, 65, 90, 105, 120, 140] — the last row of the table.
  • Time O(n · W), memory O(W).
Loop direction decides everythingIn a one-dimensional array, scanning capacity from low to high means dp[c − w] has already been updated in this round → the item can be taken again (unbounded). Scanning from high to low reads the old value → 0/1.

Unbounded Knapsack · Run the code step by step

unboundedKnapsack(w, v, 8)

dp0012345678itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90

Profit [15, 50, 60, 90], weight [1, 3, 4, 5], W = 8. dp[c] = the largest profit with capacity c; dp[0] = 0 and the others are not computed yet.

dp001512345678itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+15

c = 1: dp[0] + 15 = 15 → dp[1] = 15.

dp00151302345678itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+15

c = 2: dp[1] + 15 = 30 → dp[2] = 30. Item 0 is taken a second time: dp[1] was already updated in this round (unbounded).

dp0015130250345678itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+50

c = 3: dp[2] + 15 = 45, dp[0] + 50 = 50 → dp[3] = 50.

dp001513025036545678itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+15

c = 4: dp[3] + 15 = 65, dp[1] + 50 = 65, dp[0] + 60 = 60 → dp[4] = 65.

dp00151302503654905678itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+90

c = 5: dp[4] + 15 = 80, dp[2] + 50 = 80, dp[1] + 60 = 75, dp[0] + 90 = 90 → dp[5] = 90.

dp00151302503654905105678itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+15

c = 6: dp[5] + 15 = 105, dp[3] + 50 = 100, dp[2] + 60 = 90, dp[1] + 90 = 105 → dp[6] = 105.

dp00151302503654905105612078itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+15

c = 7: dp[6] + 15 = 120, dp[4] + 50 = 115, dp[3] + 60 = 110, dp[2] + 90 = 120 → dp[7] = 120.

dp00151302503654905105612071408itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90+50

c = 8: dp[7] + 15 = 135, dp[5] + 50 = 140, dp[4] + 60 = 125, dp[3] + 90 = 140 → dp[8] = 140.

dp00151302503654905105612071408itemw=1 v=15w=3 v=50w=4 v=60w=5 v=90

dp[8] = 140: 50 + 90 (3 kg + 5 kg). The equally profitable 15 + 15 + 50 + 60 weighs 9 kg and does not fit; the table tried all combinations in O(n · W).

static int unboundedKnapsack(
    int[] w, int[] v, int W) {
  int[] dp = new int[W + 1];  // dp[c]
  for (int c = 1; c <= W; c++)
    for (int j = 0; j < w.length; j++)
      if (w[j] <= c)
        dp[c] = Math.max(dp[c],
                  dp[c - w[j]] + v[j]);
  return dp[W];
}

Subset Sum

Subset Sum

1010005588662244target sum = 155 + 8 + 2 = 15✓✓✓
  • Given nonnegative integers (target ≥ 0), is there a subset whose sum equals the target exactly? (Decision problem: yes / no.)
  • State: dp[i][j] = can the first i elements make the sum j? (boolean)
dp[i][0] = true · dp[0][j] = false (j > 0)
dp[i][j] = dp[i−1][j] or ( aᵢ ≤ j and dp[i−1][j − aᵢ] )

It is the special case of knapsack with "value = weight": logical or replaces max.

Subset Sum

Subset Sum: Table (a = 10, 0, 5, 8, 6, 2, 4; target 15)

j (sum) →iaᵢ01234567891011121314150—✓···············110✓20✓35✓48✓56✓62✓74✓

Column 0 is always ✓ (the empty set sums to 0). In row 0 (no elements), no other sum is reachable (·).

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓35✓48✓56✓62✓74✓

i=1, a=10: copy row i−1, and mark ✓ every j with dp[i−1][j−10] ✓ → new sums reached: 10.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓48✓56✓62✓74✓

i=2, a=0: copy row i−1, and mark ✓ every j with dp[i−1][j−0] ✓ → no new sums (adding 0 changes nothing).

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓56✓62✓74✓

i=3, a=5: copy row i−1, and mark ✓ every j with dp[i−1][j−5] ✓ → new sums reached: 5, 15.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓62✓74✓

i=4, a=8: copy row i−1, and mark ✓ every j with dp[i−1][j−8] ✓ → new sums reached: 8, 13.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓74✓

i=5, a=6: copy row i−1, and mark ✓ every j with dp[i−1][j−6] ✓ → new sums reached: 6, 11, 14.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓

i=6, a=2: copy row i−1, and mark ✓ every j with dp[i−1][j−2] ✓ → new sums reached: 2, 7, 12.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

i=7, a=4: copy row i−1, and mark ✓ every j with dp[i−1][j−4] ✓ → new sums reached: 4, 9.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

dp[7][15] = dp[6][15] = ✓ → sum 15 is possible without 4; not chosen, move up.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

dp[6][15] = dp[5][15] = ✓ → sum 15 is possible without 2; not chosen, move up.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

dp[5][15] = dp[4][15] = ✓ → sum 15 is possible without 6; not chosen, move up.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

dp[4][15] = dp[3][15] = ✓ → sum 15 is possible without 8; not chosen, move up.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

dp[3][15] ✓ but dp[2][15] · → 5 chosen; j = 15 − 5 = 10

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

dp[2][10] = dp[1][10] = ✓ → sum 10 is possible without 0; not chosen, move up.

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

dp[1][10] ✓ but dp[0][10] · → 10 chosen; j = 10 − 10 = 0

j (sum) →iaᵢ01234567891011121314150—✓···············110✓·········✓·····20✓·········✓·····35✓····✓····✓····✓48✓····✓··✓·✓··✓·✓56✓····✓✓·✓·✓✓·✓✓✓62✓·✓··✓✓✓✓·✓✓✓✓✓✓74✓·✓·✓✓✓✓✓✓✓✓✓✓✓✓

Reconstruction finds the subset {5, 10}: 10 + 5 = 15. {5, 8, 2} and {5, 6, 4} also work; the table yields only one.

Subset Sum

Subset Sum: Code and Analysis

boolean subsetSum(int[] a, int target) {
  int n = a.length; // a[i] ≥ 0, target ≥ 0
  boolean[][] dp = new boolean[n + 1][target + 1];
  for (int i = 0; i <= n; i++)
    dp[i][0] = true;           // empty set
  for (int i = 1; i <= n; i++) {
    int x = a[i - 1];
    for (int j = 1; j <= target; j++)
      dp[i][j] = dp[i - 1][j]  // x not taken
        || (x <= j && dp[i - 1][j - x]);
  }
  return dp[n][target];
}
  • Time and memory: O(n · target) — again pseudo-polynomial.
  • The problem is NP-complete; if the target is large, the table cannot be built.
  • This example has 6 different solutions: {10, 5}, {5, 8, 2}, {5, 6, 4}, and each of them with 0 added.
  • One-dimensional version: scan j from high to low (as in 0/1).
  • Application: the partition problem — splitting an array with sum S into two equal halves = subset sum with target S/2.

Subset Sum · Run the code step by step

subsetSum({3, 34, 4, 12, 5, 2}, 9)

0123456789–0··········31··········342··········43··········124··········55··········26··········x

a = [3, 34, 4, 12, 5, 2], target = 9. dp[i][j]: can the first i elements make the sum j? In Java a boolean array starts out false (·).

0123456789–0T·········31T·········342T·········43T·········124T·········55T·········26T·········x

dp[i][0] = true in every row: the empty set gives sum 0. The rest of row 0 is false: with no elements there is no positive sum.

0123456789–0T·········31T··T······342T·········43T·········124T·········55T·········26T·········x

i = 1, x = 3: the row above carries down; in addition, every j with dp[0][j − 3] true gives a new sum: 3.

0123456789–0T·········31T··T······342T··T······43T·········124T·········55T·········26T·········x

i = 2, x = 34 > 9: x fits no j; the row is a copy of the one above.

0123456789–0T·········31T··T······342T··T······43T··TT··T··124T·········55T·········26T·········x

i = 3, x = 4: the row above carries down; in addition, every j with dp[2][j − 4] true gives new sums: 4, 7.

0123456789–0T·········31T··T······342T··T······43T··TT··T··124T··TT··T··55T·········26T·········x

i = 4, x = 12 > 9: x fits no j; the row is a copy of the one above.

0123456789–0T·········31T··T······342T··T······43T··TT··T··124T··TT··T··55T··TTT·TTT26T·········x

i = 5, x = 5: the row above carries down; in addition, every j with dp[4][j − 5] true gives new sums: 5, 8, 9. dp[5][9] = dp[4][4] → 4 + 5 = 9 found.

0123456789–0T·········31T··T······342T··T······43T··TT··T··124T··TT··T··55T··TTT·TTT26T·TTTTTTTTx

i = 6, x = 2: the row above carries down; in addition, every j with dp[5][j − 2] true gives new sums: 2, 6.

0123456789–0T·········31T··T······342T··T······43T··TT··T··124T··TT··T··55T··TTT·TTT26T·TTTTTTTTx

dp[6][9] = true: some subset sums to 9 (e.g., {4, 5} or {3, 4, 2}). The table has (n + 1) × (target + 1) = 70 cells.

boolean subsetSum(int[] a, int target) {
  int n = a.length; // a[i] ≥ 0, target ≥ 0
  boolean[][] dp =
      new boolean[n + 1][target + 1];
  for (int i = 0; i <= n; i++)
    dp[i][0] = true;        // empty set
  for (int i = 1; i <= n; i++) {
    int x = a[i - 1];
    for (int j = 1; j <= target; j++)
      dp[i][j] = dp[i - 1][j] // x not taken
        || (x <= j && dp[i - 1][j - x]);
  }
  return dp[n][target];
}

Longest Increasing Subsequence

Longest Increasing Subsequence

  • A subsequence keeps the order of the elements but may skip some. A substring / subarray must be contiguous.
  • LIS: the longest subsequence whose elements are strictly increasing.
0, 8, 4, 12, 2, 10, 6, 14, 1, 9, 5, 13, 3, 11, 7, 15 → LIS = 0, 2, 6, 9, 11, 15 (length 6)
0841221061419513311715
O(n²) DPdp[i] = length of the longest increasing subsequence ending at a[i]. dp[i] = 1 + max{ dp[j] : j < i, a[j] < a[i] }.
O(n log n): active listsKeep, for each length, the list with the smallest last element; place each new element by binary search.

This sequence has more than one LIS (e.g., 0, 4, 6, 9, 13, 15); the length is always unique.

Longest Increasing Subsequence

LIS: O(n²) Dynamic Programming

0123456789101112131415a0841221061419513311715dpprev

Start: every element alone is an increasing subsequence of length 1.

0123456789101112131415a0841221061419513311715dp1prev—

i=0: a[0]=0, no earlier elements → dp[0] = 1.

0123456789101112131415a0841221061419513311715dp12prev—0

i=1: a[1]=8; largest dp among smaller earlier elements (blue): j=0 (dp=1) → dp[1] = 2

0123456789101112131415a0841221061419513311715dp122prev—00

i=2: a[2]=4; largest dp among smaller earlier elements (blue): j=0 (dp=1) → dp[2] = 2

0123456789101112131415a0841221061419513311715dp1223prev—001

i=3: a[3]=12; largest dp among smaller earlier elements (blue): j=1 (dp=2) → dp[3] = 3

0123456789101112131415a0841221061419513311715dp12232prev—0010

i=4: a[4]=2; largest dp among smaller earlier elements (blue): j=0 (dp=1) → dp[4] = 2

0123456789101112131415a0841221061419513311715dp122323prev—00101

i=5: a[5]=10; largest dp among smaller earlier elements (blue): j=1 (dp=2) → dp[5] = 3

0123456789101112131415a0841221061419513311715dp1223233prev—001012

i=6: a[6]=6; largest dp among smaller earlier elements (blue): j=2 (dp=2) → dp[6] = 3

0123456789101112131415a0841221061419513311715dp12232334prev—0010123

i=7: a[7]=14; largest dp among smaller earlier elements (blue): j=3 (dp=3) → dp[7] = 4

0123456789101112131415a0841221061419513311715dp122323342prev—00101230

i=8: a[8]=1; largest dp among smaller earlier elements (blue): j=0 (dp=1) → dp[8] = 2

0123456789101112131415a0841221061419513311715dp1223233424prev—001012306

i=9: a[9]=9; largest dp among smaller earlier elements (blue): j=6 (dp=3) → dp[9] = 4

0123456789101112131415a0841221061419513311715dp12232334243prev—0010123062

i=10: a[10]=5; largest dp among smaller earlier elements (blue): j=2 (dp=2) → dp[10] = 3

0123456789101112131415a0841221061419513311715dp122323342435prev—00101230629

i=11: a[11]=13; largest dp among smaller earlier elements (blue): j=9 (dp=4) → dp[11] = 5

0123456789101112131415a0841221061419513311715dp1223233424353prev—001012306294

i=12: a[12]=3; largest dp among smaller earlier elements (blue): j=4 (dp=2) → dp[12] = 3

0123456789101112131415a0841221061419513311715dp12232334243535prev—0010123062949

i=13: a[13]=11; largest dp among smaller earlier elements (blue): j=9 (dp=4) → dp[13] = 5

0123456789101112131415a0841221061419513311715dp122323342435354prev—00101230629496

i=14: a[14]=7; largest dp among smaller earlier elements (blue): j=6 (dp=3) → dp[14] = 4

0123456789101112131415a0841221061419513311715dp1223233424353546prev—0010123062949611

i=15: a[15]=15; largest dp among smaller earlier elements (blue): j=11 (dp=5) → dp[15] = 6

0123456789101112131415a0841221061419513311715dp1223233424353546prev—0010123062949611

Largest dp = 6 (i=15). Follow the prev indices: 15 → 11 → 9 → 6 → 2 → 0 ⇒ 0, 4, 6, 9, 13, 15 (ties pick the first j; 0, 2, 6, 9, 11, 15 is also an LIS of length 6).

Orange: element being processed · blue: candidates with a[j] < a[i] · green: best previous element. O(n²) comparisons in total.

Longest Increasing Subsequence

LIS: The Active-Lists Method

For each length, keep the increasing list with the smallest last element; a smaller end is more open to future growth.

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,150

No lists yet: start the first active list with [0].

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 8

8 is larger than every end: copy the longest list and append 8 (length 2).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 40, 8dropped

4 extends the list ending in 0 → [0, 4]; [0, 8] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 40, 4, 12

12 is larger than every end: copy the longest list and append 12 (length 3).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 20, 4dropped0, 4, 12

2 extends the list ending in 0 → [0, 2]; [0, 4] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 20, 2, 100, 4, 12dropped

10 extends the list ending in 2 → [0, 2, 10]; [0, 4, 12] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 20, 2, 60, 2, 10dropped

6 extends the list ending in 2 → [0, 2, 6]; [0, 2, 10] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 20, 2, 60, 2, 6, 14

14 is larger than every end: copy the longest list and append 14 (length 4).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 2dropped0, 2, 60, 2, 6, 14

1 extends the list ending in 0 → [0, 1]; [0, 2] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 2, 60, 2, 6, 90, 2, 6, 14dropped

9 extends the list ending in 6 → [0, 2, 6, 9]; [0, 2, 6, 14] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 1, 50, 2, 6dropped0, 2, 6, 9

5 extends the list ending in 1 → [0, 1, 5]; [0, 2, 6] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 1, 50, 2, 6, 90, 2, 6, 9, 13

13 is larger than every end: copy the longest list and append 13 (length 5).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 1, 30, 1, 5dropped0, 2, 6, 90, 2, 6, 9, 13

3 extends the list ending in 1 → [0, 1, 3]; [0, 1, 5] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 1, 30, 2, 6, 90, 2, 6, 9, 110, 2, 6, 9, 13dropped

11 extends the list ending in 9 → [0, 2, 6, 9, 11]; [0, 2, 6, 9, 13] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 1, 30, 1, 3, 70, 2, 6, 9dropped0, 2, 6, 9, 11

7 extends the list ending in 3 → [0, 1, 3, 7]; [0, 2, 6, 9] of the same length is now redundant (larger end).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 1, 30, 1, 3, 70, 2, 6, 9, 110, 2, 6, 9, 11, 15

15 is larger than every end: copy the longest list and append 15 (length 6).

0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,1500, 10, 1, 30, 1, 3, 70, 2, 6, 9, 110, 2, 6, 9, 11, 15← an LIS (length 6)

The sequence is done: the longest active list is 0, 2, 6, 9, 11, 15 → LIS length 6.

Longest Increasing Subsequence

LIS: O(n²) Code

static int lis(int[] a) {
  int n = a.length, best = 0;
  int[] dp = new int[n];
  for (int i = 0; i < n; i++) {
    dp[i] = 1;               // on its own
    for (int j = 0; j < i; j++)
      if (a[j] < a[i] && dp[j] + 1 > dp[i])
        dp[i] = dp[j] + 1;   // extend
    best = Math.max(best, dp[i]);
  }
  return best;
}
  • The inner loop tries to extend the sequence of every earlier element smaller than a[i].
  • Time O(n²), memory O(n).
  • To build the sequence, store prev[i] = j, or scan the dp values downward from the end.
  • The answer is max dp[i], not dp[n−1]: an LIS can end anywhere.

Longest Increasing Subsequence · Run the code step by step

lis({5, 2, 8, 6, 3, 6, 9, 7})

adp5021826334659677

a = [5, 2, 8, 6, 3, 6, 9, 7]. dp[i] = length of the longest increasing subsequence ending at a[i]; not computed yet.

adp50218263346596771

i = 0, a[i] = 5: no smaller element on its left → dp[0] = 1 (on its own).

adp502182633465967711

i = 1, a[i] = 2: no smaller element on its left → dp[1] = 1 (on its own).

adp5021826334659677112

i = 2, a[i] = 8: smaller elements on the left: 5, 2. Extending the longest, j = 0 (dp = 1) → dp[2] = 2.

adp50218263346596771122

i = 3, a[i] = 6: smaller elements on the left: 5, 2. Extending the longest, j = 0 (dp = 1) → dp[3] = 2.

adp502182633465967711222

i = 4, a[i] = 3: smaller elements on the left: 2. Extending the longest, j = 1 (dp = 1) → dp[4] = 2.

adp5021826334659677112223

i = 5, a[i] = 6: smaller elements on the left: 5, 2, 3. Extending the longest, j = 4 (dp = 2) → dp[5] = 3.

adp50218263346596771122234

i = 6, a[i] = 9: smaller elements on the left: 5, 2, 8, 6, 3, 6. Extending the longest, j = 5 (dp = 3) → dp[6] = 4.

adp502182633465967711222344

i = 7, a[i] = 7: smaller elements on the left: 5, 2, 6, 3, 6. Extending the longest, j = 5 (dp = 3) → dp[7] = 4.

adp502182633465967711222344

best = 4 = max dp[i] (not dp[n − 1]). A sample LIS: 2, 3, 6, 9 (the chain with dp values 1, 2, 3, 4).

static int lis(int[] a) {
  int n = a.length, best = 0;
  int[] dp = new int[n];
  for (int i = 0; i < n; i++) {
    dp[i] = 1;         // on its own
    for (int j = 0; j < i; j++)
      if (a[j] < a[i] &&
          dp[j] + 1 > dp[i])
        dp[i] = dp[j] + 1;  // extend
    best = Math.max(best, dp[i]);
  }
  return best;
}

Longest Increasing Subsequence

LIS: O(n log n) — The Tails Array and Binary Search

static int lisFast(int[] a) {
  int[] tails = new int[a.length];
  int len = 0;           // list count
  for (int x : a) {
    int lo = 0, hi = len; // tails[lo] >= x
    while (lo < hi) {
      int mid = (lo + hi) / 2;
      if (tails[mid] < x) lo = mid + 1;
      else hi = mid;
    }
    tails[lo] = x;       // replace / extend
    if (lo == len) len++;
  }
  return len;
}
  • Only the last elements of the active lists are stored; tails[] is always increasing, so the position is found by binary search → O(n log n).
  • For the sample sequence, tails = [0, 1, 3, 7, 11, 15]: the length is right (6), but this is not an LIS (7 comes after 11).
  • To build the sequence, also store each element's predecessor index.

Longest Increasing Subsequence · Run the code step by step

lisFast({5, 2, 8, 6, 3, 6, 9, 7})

atails502182633465967701234567len = 0

tails[k] = the smallest last element of an increasing subsequence of length k + 1. tails[0..len−1] is always increasing; initially len = 0.

atails5021826334659677501234567len = 1

x = 5 is larger than every end: binary search gives lo = len = 0 → tails[0] = 5, the list grows (len = 1).

atails5021826334659677201234567len = 1

x = 2: the first position with tails[lo] ≥ x is lo = 0 (1 comparison) → 2 replaces 5; len is unchanged, but the length-1 sequence now ends with a smaller number.

atails50218263346596772081234567len = 2

x = 8 is larger than every end: binary search gives lo = len = 1 → tails[1] = 8, the list grows (len = 2).

atails50218263346596772061234567len = 2

x = 6: the first position with tails[lo] ≥ x is lo = 1 (2 comparisons) → 6 replaces 8; len is unchanged, but the length-2 sequence now ends with a smaller number.

atails50218263346596772031234567len = 2

x = 3: the first position with tails[lo] ≥ x is lo = 1 (2 comparisons) → 3 replaces 6; len is unchanged, but the length-2 sequence now ends with a smaller number.

atails502182633465967720316234567len = 3

x = 6 is larger than every end: binary search gives lo = len = 2 → tails[2] = 6, the list grows (len = 3).

atails5021826334659677203162934567len = 4

x = 9 is larger than every end: binary search gives lo = len = 3 → tails[3] = 9, the list grows (len = 4).

atails5021826334659677203162734567len = 4

x = 7: the first position with tails[lo] ≥ x is lo = 3 (2 comparisons) → 7 replaces 9; len is unchanged, but the length-4 sequence now ends with a smaller number. Result 4; tails = [2, 3, 6, 7] need not be an LIS.

static int lisFast(int[] a) {
  int[] tails = new int[a.length];
  int len = 0;      // number of lists
  for (int x : a) {
    int lo = 0, hi = len;
    while (lo < hi) {
      int mid = (lo + hi) / 2;
      if (tails[mid] < x) lo = mid + 1;
      else hi = mid;
    }
    tails[lo] = x;    // place / extend
    if (lo == len) len++;
  }
  return len;
}

Longest Common Substring

Longest Common Substring

  • The longest run of contiguous characters shared by two strings.
  • Example: "abdca" and "cbda" → common substrings "a", "b", "d", "c", "bd" → answer "bd", length 2.
  • State: L[i][j] = length of the longest common piece ending exactly at position i of Y and position j of X.
if Y[i−1] = X[j−1]: L[i][j] = L[i−1][j−1] + 1
otherwise: L[i][j] = 0 (contiguity breaks, the counter resets)
Answer = the largest value in the table (not the last cell!)
XabdcaYcbda"bd": contiguouslength 2

Brute force: search for each of the O(m²) substrings of X in Y → O(m² · n). DP: O(m · n) time.

Longest Common Substring

Common Substring: Filling the Table

Columns X = "abdca", rows Y = "cbda". Orange: cell being computed · blue: the diagonal cell used.

abdca0000000c10b20d30a40012345

Row 0 and column 0 are zero: nothing is shared with the empty string.

abdca0000000c100b20d30a40012345

'c' ≠ 'a' → L[1][1] = 0 · best: 0

abdca0000000c1000b20d30a40012345

'c' ≠ 'b' → L[1][2] = 0 · best: 0

abdca0000000c10000b20d30a40012345

'c' ≠ 'd' → L[1][3] = 0 · best: 0

abdca0000000c100001b20d30a40012345

Y[0]='c' = X[3]='c' → L[1][4] = L[0][3] + 1 = 1 · best: 1

abdca0000000c1000010b20d30a40012345

'c' ≠ 'a' → L[1][5] = 0 · best: 1

abdca0000000c1000010b200d30a40012345

'b' ≠ 'a' → L[2][1] = 0 · best: 1

abdca0000000c1000010b2001d30a40012345

Y[1]='b' = X[1]='b' → L[2][2] = L[1][1] + 1 = 1 · best: 1

abdca0000000c1000010b20010d30a40012345

'b' ≠ 'd' → L[2][3] = 0 · best: 1

abdca0000000c1000010b200100d30a40012345

'b' ≠ 'c' → L[2][4] = 0 · best: 1

abdca0000000c1000010b2001000d30a40012345

'b' ≠ 'a' → L[2][5] = 0 · best: 1

abdca0000000c1000010b2001000d300a40012345

'd' ≠ 'a' → L[3][1] = 0 · best: 1

abdca0000000c1000010b2001000d3000a40012345

'd' ≠ 'b' → L[3][2] = 0 · best: 1

abdca0000000c1000010b2001000d30002a40012345

Y[2]='d' = X[2]='d' → L[3][3] = L[2][2] + 1 = 2 · best: 2

abdca0000000c1000010b2001000d300020a40012345

'd' ≠ 'c' → L[3][4] = 0 · best: 2

abdca0000000c1000010b2001000d3000200a40012345

'd' ≠ 'a' → L[3][5] = 0 · best: 2

abdca0000000c1000010b2001000d3000200a401012345

Y[3]='a' = X[0]='a' → L[4][1] = L[3][0] + 1 = 1 · best: 2

abdca0000000c1000010b2001000d3000200a4010012345

'a' ≠ 'b' → L[4][2] = 0 · best: 2

abdca0000000c1000010b2001000d3000200a40100012345

'a' ≠ 'd' → L[4][3] = 0 · best: 2

abdca0000000c1000010b2001000d3000200a401000012345

'a' ≠ 'c' → L[4][4] = 0 · best: 2

abdca0000000c1000010b2001000d3000200a4010001012345

Y[3]='a' = X[4]='a' → L[4][5] = L[3][4] + 1 = 1 · best: 2

abdca0000000c1000010b2001000d3000200a4010001012345

The largest value is L[3][3] = 2; 2 steps back along the diagonal: X[1..2] = "bd". The last cell, L[4][5] = 1, is not the answer.

Longest Common Subsequence

Longest Common Subsequence

  • The longest subsequence common to two sequences, chosen in order (with skips allowed).
  • Example: "abdca" and "cbda" → "bda", length 3 (the common substring was only 2). The LCS need not be unique, but its length is: AGGTAB · GXTXAYB → GTAB (4); GTAB is a subsequence but not a substring.
  • Uses: diff / git comparison, similarity of DNA sequences, plagiarism detection.
if Y[i−1] = X[j−1]: L[i][j] = L[i−1][j−1] + 1 (take the common character)
otherwise: L[i][j] = max( L[i−1][j] , L[i][j−1] ) (skip one)
Answer = L[n][m] (the last cell)
MatchMismatchAnswer
Common substringdiagonal + 10maximum of the table
Common subsequence (LCS)diagonal + 1max(up, left)last cell

Longest Common Subsequence

LCS: Code Trace

abdca0000000c10b20d30a40012345

Array L is (n+1) × (m+1) = 5 × 6; in Java every cell starts at 0.

abdca0000000c100b20d30a40012345

'c' ≠ 'a' → max(up 0, left 0) = 0

abdca0000000c1000b20d30a40012345

'c' ≠ 'b' → max(up 0, left 0) = 0

abdca0000000c10000b20d30a40012345

'c' ≠ 'd' → max(up 0, left 0) = 0

abdca0000000c100001b20d30a40012345

'c' = 'c' → diagonal + 1: L[1][4] = 0 + 1 = 1

abdca0000000c1000011b20d30a40012345

'c' ≠ 'a' → max(up 0, left 1) = 1

abdca0000000c1000011b200d30a40012345

'b' ≠ 'a' → max(up 0, left 0) = 0

abdca0000000c1000011b2001d30a40012345

'b' = 'b' → diagonal + 1: L[2][2] = 0 + 1 = 1

abdca0000000c1000011b20011d30a40012345

'b' ≠ 'd' → max(up 0, left 1) = 1

abdca0000000c1000011b200111d30a40012345

'b' ≠ 'c' → max(up 1, left 1) = 1

abdca0000000c1000011b2001111d30a40012345

'b' ≠ 'a' → max(up 1, left 1) = 1

abdca0000000c1000011b2001111d300a40012345

'd' ≠ 'a' → max(up 0, left 0) = 0

abdca0000000c1000011b2001111d3001a40012345

'd' ≠ 'b' → max(up 1, left 0) = 1

abdca0000000c1000011b2001111d30012a40012345

'd' = 'd' → diagonal + 1: L[3][3] = 1 + 1 = 2

abdca0000000c1000011b2001111d300122a40012345

'd' ≠ 'c' → max(up 1, left 2) = 2

abdca0000000c1000011b2001111d3001222a40012345

'd' ≠ 'a' → max(up 1, left 2) = 2

abdca0000000c1000011b2001111d3001222a401012345

'a' = 'a' → diagonal + 1: L[4][1] = 0 + 1 = 1

abdca0000000c1000011b2001111d3001222a4011012345

'a' ≠ 'b' → max(up 1, left 1) = 1

abdca0000000c1000011b2001111d3001222a40112012345

'a' ≠ 'd' → max(up 2, left 1) = 2

abdca0000000c1000011b2001111d3001222a401122012345

'a' ≠ 'c' → max(up 2, left 2) = 2

abdca0000000c1000011b2001111d3001222a4011223012345

'a' = 'a' → diagonal + 1: L[4][5] = 2 + 1 = 3

abdca0000000c1000011b2001111d3001222a4011223012345

The table is full: L[4][5] = 3. 20 cells, each O(1) → O(m · n).

static int lcs(String x, String y) {
  int m = x.length(), n = y.length();
  int[][] L = new int[n + 1][m + 1];
  for (int i = 1; i <= n; i++)
    for (int j = 1; j <= m; j++)
      if (y.charAt(i - 1) == x.charAt(j - 1))
        L[i][j] = L[i - 1][j - 1] + 1;
      else
        L[i][j] = Math.max(L[i - 1][j],
                           L[i][j - 1]);
  return L[n][m];
}

Longest Common Subsequence

LCS: Reconstructing the Solution

abdca0000000c1000011b2001111d3001222a4011223012345

Reconstruction starts at the last cell, L[4][5] = 3; the characters found are collected from back to front.

abdca0000000c1000011b2001111d3001222a4011223012345

'a' = 'a' → the character joins the LCS ("a"); move diagonally.

abdca0000000c1000011b2001111d3001222a4011223012345

'd' ≠ 'c'; left (2) > up (1) → move left.

abdca0000000c1000011b2001111d3001222a4011223012345

'd' = 'd' → the character joins the LCS ("da"); move diagonally.

abdca0000000c1000011b2001111d3001222a4011223012345

'b' = 'b' → the character joins the LCS ("bda"); move diagonally.

abdca0000000c1000011b2001111d3001222a4011223012345

'c' ≠ 'a'; up (0) ≥ left (0) → move up.

abdca0000000c1000011b2001111d3001222a4011223012345

The border is reached. Green cells are matches: LCS = "bda", length 3. Reconstruction takes O(m + n).

Longest Common Subsequence

Example: LCS of DNA Sequences

ACCGGTGGACAATTCAGGAAAGAGATATGCAC
  • The lines joining matched characters do not cross: the relative order is preserved. Every matching that obeys this rule is a common subsequence.
  • DP result: LCS = GGAAATTCA, length 9.
Matching by eye is misleading. For instance, AAAATTCA (8 characters) is also a valid common subsequence, but not the longest; the DP table gives the right answer.

Longest Common Subsequence

Try It: LCS and Common Substring Tables

X has at most 9 characters, Y at most 7. Green cells show the reconstruction path.

Other Problems

Other Classic DP Problems

ProblemState and recurrenceTime
Edit distanceD[i][j] = min(D[i−1][j] + 1, D[i][j−1] + 1, D[i−1][j−1] + [aᵢ ≠ bⱼ]); D[i][0] = i, D[0][j] = jO(m · n)
Coin changefewest coins: P[x] = 1 + min P[x − c] (c ≤ x); P[0] = 0O(k · X)
Maximum subarrayV(i) = max(a[i], V(i−1) + a[i]) — Kadane; answer max V(i)O(n)
Matrix-chain multiplicationM[i][j] = min { M[i][k] + M[k+1][j] + pᵢ₋₁ · pₖ · pⱼ : i ≤ k < j }O(n³)
Traveling salesman (Held-Karp)dp[S][i] = shortest path visiting set S and ending at i; dp[S][i] = min dp[S − i][j] + d(j, i)O(n² · 2ⁿ)
Bellman-Forddₖ(v) = shortest path with at most k edges; dₖ(v) = min(dₖ₋₁(v), dₖ₋₁(u) + w(u, v))O(V · E)
Optimal binary search treee[i][j] = min { e[i][r−1] + e[r+1][j] : i ≤ r ≤ j } + w(i, j)O(n³)

Summary

Summary

ProblemState / recurrenceTime · memory
FibonacciF(n) = F(n−1) + F(n−2); memoization or a tableO(n) · O(1)
0/1 knapsackK[i][c] = max(K[i−1][c], vᵢ + K[i−1][c−wᵢ])O(nW) · O(W)
Unbounded knapsackdp[i][c] = max(dp[i−1][c], pᵢ + dp[i][c−wᵢ])O(nW) · O(W)
Subset sumdp[i][j] = dp[i−1][j] ∨ dp[i−1][j−aᵢ]O(n · target)
LISdp[i] = 1 + max dp[j] (j < i, a[j] < a[i]); or active listsO(n²) / O(n log n)
Common substringif equal, diagonal + 1, else 0; answer is the table maximumO(mn)
LCSif equal, diagonal + 1, else max(up, left); answer is the last cellO(mn)
DP = overlapping subproblems + optimal substructure. Time ≈ number of states × work per state. The solution itself is found by reading the table backward.

Summary

Common Mistakes

Forgetting the base casesIf K[0][c], dp[i][0], or fib(0)/fib(1) is initialized wrongly, the whole table shifts.
Off-by-one indicesThe table has size n+1 but the array has n: row i uses v[i − 1]. Writing v[i] overflows on the last item.
Wrong loop directionIn the one-dimensional 0/1 knapsack, scanning capacity from low to high takes an item more than once (it becomes unbounded).
Substring ≠ subsequenceFor the common substring, a mismatch writes 0 and the answer is the table maximum; for LCS, write max(up, left) and the answer is the last cell.
Assuming greedy worksGreedy is not optimal for the 0/1 knapsack or for making change (6 with {1, 3, 4}).
Mistaking O(nW) for polynomialW is a numeric value; the input size is log W bits. For large W, the table cannot be built.

Summary

Sample Problems

  1. 0/1 knapsack: W = 5; items (v, w) = (3, 2), (4, 3), (5, 4), (6, 5). What is the largest value?
  2. What is the length of LCS("ABCBDAB", "BDCABA")? Write one LCS.
  3. Coins {1, 3, 4}, amount 6: how few coins suffice? How many does greedy use?
  4. What are the LIS length and one LIS of [10, 22, 9, 33, 21, 50, 41, 60]?
  5. What are the longest common substring (contiguous) and LCS lengths for "ABAB" and "BABA"?
  6. How many calls does plain recursive fib(6) make? How many with memoization?
Show answers
  1. 7: item 1 + item 2 (w = 2 + 3 = 5, v = 3 + 4).
  2. 4; e.g., "BCBA" (or "BDAB", "BCAB").
  3. DP: 2 coins (3 + 3). Greedy: 4 + 1 + 1 = 3 coins.
  4. 5: 10, 22, 33, 50, 60 (or 10, 22, 33, 41, 60).
  5. Common substring 3 ("ABA" or "BAB"); LCS is also 3. Skipping characters gives nothing longer here.
  6. 2 · F(7) − 1 = 2 · 13 − 1 = 25 calls; with memoization 2 · 6 − 1 = 11.

Summary

Exercises

  1. Write the 0/1 knapsack with memoization (top-down).
  2. In how many ways can the amount 5 be paid with coins {1, 2, 5}?
  3. Find the longest common substring of "ababc" and "babca".
  4. Does {3, 34, 4, 12, 5, 2} have a subset summing to 9? To 30?
  5. What is the maximum subarray of [−2, 1, −3, 4, −1, 2, 1, −5, 4] (Kadane)?
  6. Weights {2, 3, …, 8}, values {3, 5, 8, 6, 10, 11, 15}, W = 10: what are the 0/1 and unbounded results?
Answers / hints
  1. Integer[][] memo; f(i, c): 0 if i = n; otherwise max(f(i+1, c), v[i] + f(i+1, c − w[i])), storing the result.
  2. 4: 5, 2+2+1, 2+1+1+1, 1·5 (outer loop over coins: d[x] += d[x − c]).
  3. "babc", length 4.
  4. 9: yes (4 + 5). 30: no (without 34 the total is 26 < 30).
  5. 6: [4, −1, 2, 1].
  6. 0/1: 18 (e.g., w 4 + 6 → 8 + 10). Unbounded: 19 (e.g., w 4 + 4 + 2 → 8 + 8 + 3).

Summary

References

  • Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, 4th ed., MIT Press, 2022 (Chapter 14: Dynamic Programming).
  • Kleinberg and Tardos, Algorithm Design, Pearson, 2005 (Chapter 6).
  • Dasgupta, Papadimitriou, and Vazirani, Algorithms, McGraw-Hill, 2006 (Chapter 6).
  • R. Bellman, Dynamic Programming, Princeton University Press, 1957.

The End

Chapter 3: Dynamic Programming
Previous chapter: Chapter 2: Greedy Algorithms · Next chapter: Chapter 4: Backtracking
1 / 1 Sercan KÜLCÜ. All rights reserved.