Chapter 2: Algorithm Complexity
Output Question · The Notion of Complexity
Question 1: Counting Steps
If each basic operation counts as 1 unit (assignment, comparison, increment, return), how many operations are made on the n = 10 element array a = {7, 1, 9, 4, 2, 8, 6, 3, 5, 0} for (a) target = 4 and (b) a target that is not in the array?
for (int i = 0; i < a.length; i++) { if (a[i] == target) return i; } return -1;
Show answer
| Operation | (a) target = 4 | (b) absent |
|---|---|---|
| i = 0 | 1 | 1 |
| i < a.length | 4 | n + 1 = 11 |
| a[i] == target | 4 | n = 10 |
| i++ | 3 | n = 10 |
| return | 1 | 1 |
| Total | 13 | 33 = 3n + 3 |
4 is found at index 3: four comparison rounds, three increments. If the target is at index k, 3k + 4 operations are made. Worst case: 3n + 3 = Θ(n).
Multiple Choice · The Notion of Complexity
Question 2: Average Number of Comparisons
A linear search runs on 9 elements. The target is always present, equally likely at every position. How many comparisons (a[i] == target) are made on average?
B: A target at position k is found with k comparisons: (1 + 2 + … + 9)/9 = 45/9 = (n + 1)/2 = 5. 4.5 = n/2 is a common mistake (counting from 0 instead of 1); 9 is the unsuccessful (worst) case. The average is also Θ(n).
Multiple Choice · The Notion of Complexity
Question 3: Memory of f3
What are the time and extra memory (including the call stack) complexities of f3(n)?
public long f3(int n) { if (n <= 1) { return 1; } return f3(n - 1) + f3(n - 1); }
D: 2ⁿ − 1 calls are made, but the calls finish in sequence: the second f3(n − 1) does not start until the first has finished completely and its frames are popped. At most n frames are on the stack at once (f3(n), f3(n − 1), …, f3(1)). Time and memory can grow differently.
Multiple Choice · Asymptotic Notation
Question 4: Which One Is False?
For f(n) = 3n² + 5n + 2, which of the following is false?
C: Θ requires both a lower and an upper bound; f(n) ≥ c · n³ does not hold for any c > 0 at large n (f(n)/n³ → 0). n³ is a loose upper bound (A is true), n is a loose lower bound (B is true); the only tight bound is n².
Output Question · Asymptotic Notation
Question 5: The Smallest n₀
For f(n) = 2n² + 7n + 3, c = 3 is chosen. What is the smallest n₀ such that f(n) ≤ 3n² holds for every n ≥ n₀? What is n₀ if c = 4 is chosen?
Show answer
c = 3: 2n² + 7n + 3 ≤ 3n² ⇔ n² − 7n − 3 ≥ 0 ⇔ n ≥ (7 + √61)/2 = 7.41 → n₀ = 8.
| n | 3n² | f(n) | |
|---|---|---|---|
| 7 | 147 | 150 | ✗ |
| 8 | 192 | 187 | ✓ |
c = 4: 2n² − 7n − 3 ≥ 0 ⇔ n ≥ (7 + √73)/4 = 3.89 → n₀ = 4 (n = 3: 36 < 42 ✗; n = 4: 64 ≥ 63 ✓). As c grows, n₀ shrinks.
Output Question · Loop Analysis
Question 6: A Doubling Counter
How many times do the calls foo(8) and foo(100) print "Hello"? What is the time complexity?
public void foo(int n) { for (int i = 1; i <= n; i = i * 2) { System.out.println("Hello"); } }
Show answer
- n = 8: i = 1, 2, 4, 8 → 4 times (log₂ 8 + 1).
- n = 100: i = 1, 2, 4, 8, 16, 32, 64 → 7 times (⌊log₂ 100⌋ + 1); 128 > 100.
After k rounds, i = 2ᵏ; the loop runs while 2ᵏ ≤ n → ⌊log₂ n⌋ + 1 rounds → O(log n). If the condition were i < n, it would run 3 times for n = 8.
Step-by-Step Trace · Loop Analysis
Question 7: Tracing f2(4)
In f2(4), how often does the inner loop run in each outer iteration, and what is returned?
x = 0. The outer loop runs i = 0 to 3; the inner bound is i · i each time.
i = 0: j < 0 is false at once → inner loop runs 0 times; x = 0.
i = 1: inner loop runs i · i = 1 times → x = 1.
i = 2: inner loop runs i · i = 4 times → x = 5.
i = 3: inner loop runs i · i = 9 times → x = 14.
i = 4 < 4 is false → the loop ends; return 14 = 0 + 1 + 4 + 9 = (n − 1)n(2n − 1)/6 = 3 · 4 · 7 / 6. Total ≈ n³/3 → Θ(n³).
public int f2(int n) { int x = 0; for (int i = 0; i < n; i++) { for (int j = 0; j < i * i; j++) { x++; } } return x; }
Step-by-Step Trace · Loop Analysis
Question 8: A Geometric Sum
What does func(10) print? Which class does the total number of inner-loop iterations belong to?
n = 10, count = 0. The outer loop doubles i on every iteration: 1, 2, 4, 8, …
i = 1 < 10 → the inner loop runs 1 time; count = 1.
i = 2 < 10 → the inner loop runs 2 times; count = 3.
i = 4 < 10 → the inner loop runs 4 times; count = 7.
i = 8 < 10 → the inner loop runs 8 times; count = 15.
i = 16 < 10 is false → the loop ends; 15 is printed. 1 + 2 + 4 + 8 = 2⁴ − 1 < 2n: although the outer loop runs log n times, the total is Θ(n).
void func(int n) { int count = 0; for (int i = 1; i < n; i *= 2) { for (int j = 0; j < i; j++) { count++; } } System.out.println(count); }
Multiple Choice · Loop Analysis
Question 9: A Doubling Outer Loop
What is the time complexity of the following function?
public void foo(int n) { for (int i = 1; i < n; i = i * 2) { for (int j = 0; j < n; j++) { System.out.println(i + " " + j); } } }
D: The outer loop runs ⌈log₂ n⌉ times (i = 1, 2, 4, …); the inner bound depends on n, not i, so it runs n times each time. For n = 16, 4 · 16 = 64 lines. With j < i the total would be 1 + 2 + 4 + … < 2n, i.e. Θ(n).
Multiple Choice · Loop Analysis
Question 10: The Tight Bound of f6
Which statement is true for f6, written with the loop for (int i = 0; i < n; i = 1 << i) (ignoring overflow)?
D: The update i ← 2ⁱ makes i grow like a power tower: 0, 1, 2, 4, 16, 65536. The number of iterations is the iterated logarithm (log* n). A would be right for the loop i = 2 · i; in f6 i itself becomes the exponent.
Output Question · Analyzing Recursion
Question 11: Two Calls vs. One Call
What does f3(5) return, and how many calls are made in total (including the first)? If the last line is changed to return 2 * f3(n - 1);, what are the return value and the number of calls?
public long f3(int n) { if (n <= 1) { return 1; } return f3(n - 1) + f3(n - 1); }
Show answer
- f3(5) = 16 (= 2ⁿ⁻¹), 31 calls (= 2ⁿ − 1): each call spawns two calls.
- 2 * f3(n − 1) gives: the value is again 16, but only 5 calls: f3(5), f3(4), …, f3(1).
The recurrence drops from 2T(n − 1) + c to T(n − 1) + c: Θ(2ⁿ) → Θ(n). Solving the same subproblem twice is wasteful; computing the result once and reusing it is enough.
Multiple Choice · Analyzing Recursion
Question 12: Unrolling a Recurrence
If the recurrence T(n) = 2T(n − 1) + c, T(1) = c is solved by unrolling, which closed form is correct?
B: T(n) = 2T(n − 1) + c = 4T(n − 2) + 3c = 8T(n − 3) + 7c = … = 2ᵏ T(n − k) + (2ᵏ − 1)c. At k = n − 1, T(1) = c: 2ⁿ⁻¹c + (2ⁿ⁻¹ − 1)c = (2ⁿ − 1)c → Θ(2ⁿ). Because it is a subtractive recurrence, the master theorem does not apply.
Classic Question · Analyzing Recursion
Question 13: T(n) = T(n − 1) + n
Solve the recurrence T(n) = T(n − 1) + n, T(1) = 1 by unrolling. Why can't the master theorem be applied? In the worst case of which algorithm does this recurrence arise?
Show answer
T(n) = n + T(n − 1) = n + (n − 1) + T(n − 2) = … = n + (n − 1) + … + 2 + T(1) = n(n + 1)/2 → Θ(n²).
The master theorem requires the form T(n) = aT(n/b) + f(n); here the subproblem is n − 1, not n/b (a subtractive recurrence).
In quicksort where the pivot is always the smallest or largest element, every partition takes Θ(n) and separates only the pivot: T(n) = T(n − 1) + Θ(n). Insertion sort on reverse-sorted input gives the same sum.
Classic Question · Master Theorem
Question 14: Five Recurrences
Solve with the master theorem; for each, give a, b, log_b a, the degree c of f(n), and the case: (a) T(n) = 3T(n/3) + n, (b) T(n) = 9T(n/3) + n, (c) T(n) = T(n/2) + n, (d) T(n) = 4T(n/2) + n², (e) T(n) = 2T(n/2) + n log n.
Show answer
| a, b | log_b a | c | Case | Result | |
|---|---|---|---|---|---|
| (a) | 3, 3 | 1 | 1 | 2 | Θ(n log n) |
| (b) | 9, 3 | 2 | 1 | 1 | Θ(n²) |
| (c) | 1, 2 | 0 | 1 | 3 | Θ(n) |
| (d) | 4, 2 | 2 | 2 | 2 | Θ(n² log n) |
| (e) | 2, 2 | 1 | 1 (· log n) | extended | Θ(n log² n) |
(c) In Case 3, regularity: a · f(n/b) = n/2 ≤ ½ · n, k = ½ < 1 ✓. In (d) every level does n² work and there are log₂ n levels. (e) falls into none of the three basic cases; if f(n) = Θ(n^log_b a · logᵏ n) then T(n) = Θ(n^log_b a · logᵏ⁺¹ n).
Multiple Choice · Master Theorem
Question 15: Eight Subproblems
What is the solution of the recurrence T(n) = 8T(n/2) + n²?
A: log₂ 8 = 3 > c = 2 → Case 1 (leaves dominate): Θ(n^log₂ 8) = Θ(n³). This is divide-and-conquer matrix multiplication (four blocks, 8 multiplications), no better than the classic three loops. Strassen, with 7 multiplications, gets Θ(n^2.807).
Multiple Choice · Master Theorem
Question 16: A Recurrence That Does Not Apply
To which of the following recurrences does none of the three cases of the master theorem (in its basic form) apply?
C: Compare n^log₂ 2 = n with f(n) = n / log n: f(n) is smaller than n, but only by a log factor, not by a polynomial factor (n^ε); it fits neither Case 1 nor Case 2. A: Case 1 → Θ(n²); B: Case 3 → Θ(n²); D: log₄ 16 = 2 = c → Case 2 → Θ(n² log n).
Step-by-Step Trace · Application: f4 and f5
Question 17: Recursion Tree of f4(4)
In the call f4(4), in what order is the recursion tree built, how many calls are made, and what is the result?
f4(4): 4 > 1 → on line 5, the left call f4(4 / 2) = f4(2) is made first.
f4(2): 2 > 1 → f4(1) is called.
f4(1): base case, 1 is returned.
The second f4(1) also returns 1 → f4(2) = 1 + 1 = 2.
f4(4) calls the right f4(2): the same subtree is expanded a second time.
Two more f4(1) → this f4(2) also returns 2.
f4(4) = 2 + 2 = 4. 7 = 2n − 1 calls in total; level costs c, 2c, 4c: leaves dominate → Θ(n). Stack depth is only log₂ n + 1 = 3.
public int f4(int n) { if (n <= 1) { return 1; } return f4(n / 2) + f4(n / 2); }
Output Question · Application: f4 and f5
Question 18: Values of f5
What do f5(4) and f5(32) return? (f1(n) returns n.) Compare the result with n log₂ n + n.
public int f5(int n) { if (n <= 1) { return 1; } return f1(n) + f5(n / 2) + f5(n / 2); }
Show answer
- f5(1) = 1, f5(2) = 2 + 1 + 1 = 4
- f5(4) = 4 + 4 + 4 = 12 = 4 · 2 + 4
- f5(8) = 32, f5(16) = 80 → f5(32) = 32 + 80 + 80 = 192 = 32 · 5 + 32
For n = 2ᵏ, f5(n) = n log₂ n + n: each of the log₂ n levels does n work, and the leaves add n. T(n) = 2T(n/2) + Θ(n) → Case 2 → Θ(n log n).
Step-by-Step Trace · Amortized Analysis
Question 19: Dynamic Array: 9 Insertions
9 elements are inserted into an empty array of capacity 1 that doubles when full. What is the cost of each insertion (writes + copies) and the total cost?
Insertion 1: capacity 1, array empty → 1 write. Cost = writes + number of copied elements.
Insertion 2: the array was full → capacity 1 → 2, 1 copy + 1 write = 2.
Insertion 3: the array was full → capacity 2 → 4, 2 copies + 1 write = 3.
Insertion 4: there is room → just 1 write.
Insertion 5: the array was full → capacity 4 → 8, 4 copies + 1 write = 5.
Insertion 6: there is room → just 1 write.
Insertion 7: there is room → just 1 write.
Insertion 8: there is room → just 1 write.
Insertion 9: the array was full → capacity 8 → 16, 8 copies + 1 write = 9. Total 24 < 3 · 9 = 27 → amortized Θ(1) per insertion.
void append(int x) { if (size == data.length) { int[] newData = new int[2 * data.length]; for (int i = 0; i < size; i++) newData[i] = data[i]; // copy data = newData; } data[size++] = x; // write }
Multiple Choice · Amortized Analysis
Question 20: Amortized vs. Average
What is the difference between amortized analysis and average-case analysis?
A: Amortized cost is the total of a worst-case operation sequence divided by the number of operations; it assumes no input distribution. The average case is the expected time over a probability distribution of inputs (so D is wrong). In a dynamic array one insertion can be Θ(n), yet the amortized cost is Θ(1).