Chapter 2: Algorithm Complexity

Chapter 2: Algorithm Complexity · Study Questions
Time and memoryBest / worst / average caseO, Ω and ΘLoop analysisRecurrence relationsMaster theoremf1–f6Amortized analysis
Sercan KÜLCÜ · Giresun University · Computer Engineering

Output Question · The Notion of Complexity

Question 1: Counting Steps

If each basic operation counts as 1 unit (assignment, comparison, increment, return), how many operations are made on the n = 10 element array a = {7, 1, 9, 4, 2, 8, 6, 3, 5, 0} for (a) target = 4 and (b) a target that is not in the array?

for (int i = 0; i < a.length; i++) {
  if (a[i] == target)
    return i;
}
return -1;
Show answer
Operation(a) target = 4(b) absent
i = 011
i < a.length4n + 1 = 11
a[i] == target4n = 10
i++3n = 10
return11
Total1333 = 3n + 3

4 is found at index 3: four comparison rounds, three increments. If the target is at index k, 3k + 4 operations are made. Worst case: 3n + 3 = Θ(n).

Multiple Choice · The Notion of Complexity

Question 2: Average Number of Comparisons

A linear search runs on 9 elements. The target is always present, equally likely at every position. How many comparisons (a[i] == target) are made on average?

B: A target at position k is found with k comparisons: (1 + 2 + … + 9)/9 = 45/9 = (n + 1)/2 = 5. 4.5 = n/2 is a common mistake (counting from 0 instead of 1); 9 is the unsuccessful (worst) case. The average is also Θ(n).

Multiple Choice · The Notion of Complexity

Question 3: Memory of f3

What are the time and extra memory (including the call stack) complexities of f3(n)?

public long f3(int n) {
  if (n <= 1) {
    return 1;
  }
  return f3(n - 1) + f3(n - 1);
}

D: 2ⁿ − 1 calls are made, but the calls finish in sequence: the second f3(n − 1) does not start until the first has finished completely and its frames are popped. At most n frames are on the stack at once (f3(n), f3(n − 1), …, f3(1)). Time and memory can grow differently.

Multiple Choice · Asymptotic Notation

Question 4: Which One Is False?

For f(n) = 3n² + 5n + 2, which of the following is false?

C: Θ requires both a lower and an upper bound; f(n) ≥ c · n³ does not hold for any c > 0 at large n (f(n)/n³ → 0). n³ is a loose upper bound (A is true), n is a loose lower bound (B is true); the only tight bound is n².

Output Question · Asymptotic Notation

Question 5: The Smallest n₀

For f(n) = 2n² + 7n + 3, c = 3 is chosen. What is the smallest n₀ such that f(n) ≤ 3n² holds for every n ≥ n₀? What is n₀ if c = 4 is chosen?

Show answer

c = 3: 2n² + 7n + 3 ≤ 3n² ⇔ n² − 7n − 3 ≥ 0 ⇔ n ≥ (7 + √61)/2 = 7.41 → n₀ = 8.

n3n²f(n)
7147150✗
8192187✓

c = 4: 2n² − 7n − 3 ≥ 0 ⇔ n ≥ (7 + √73)/4 = 3.89 → n₀ = 4 (n = 3: 36 < 42 ✗; n = 4: 64 ≥ 63 ✓). As c grows, n₀ shrinks.

Output Question · Loop Analysis

Question 6: A Doubling Counter

How many times do the calls foo(8) and foo(100) print "Hello"? What is the time complexity?

public void foo(int n) {
  for (int i = 1; i <= n; i = i * 2) {
    System.out.println("Hello");
  }
}
Show answer
  • n = 8: i = 1, 2, 4, 8 → 4 times (log₂ 8 + 1).
  • n = 100: i = 1, 2, 4, 8, 16, 32, 64 → 7 times (⌊log₂ 100⌋ + 1); 128 > 100.

After k rounds, i = 2ᵏ; the loop runs while 2ᵏ ≤ n → ⌊log₂ n⌋ + 1 rounds → O(log n). If the condition were i < n, it would run 3 times for n = 8.

n = 100: values of i1248163264

Step-by-Step Trace · Loop Analysis

Question 7: Tracing f2(4)

In f2(4), how often does the inner loop run in each outer iteration, and what is returned?

inner-loop iterations (i²)0i=00i=10i=20i=3

x = 0. The outer loop runs i = 0 to 3; the inner bound is i · i each time.

inner-loop iterations (i²)0i=00i=10i=20i=3

i = 0: j < 0 is false at once → inner loop runs 0 times; x = 0.

inner-loop iterations (i²)0i=01i=10i=20i=3

i = 1: inner loop runs i · i = 1 times → x = 1.

inner-loop iterations (i²)0i=01i=14i=20i=3

i = 2: inner loop runs i · i = 4 times → x = 5.

inner-loop iterations (i²)0i=01i=14i=29i=3

i = 3: inner loop runs i · i = 9 times → x = 14.

inner-loop iterations (i²)0i=01i=14i=29i=3

i = 4 < 4 is false → the loop ends; return 14 = 0 + 1 + 4 + 9 = (n − 1)n(2n − 1)/6 = 3 · 4 · 7 / 6. Total ≈ n³/3 → Θ(n³).

public int f2(int n) {
  int x = 0;
  for (int i = 0; i < n; i++) {
    for (int j = 0; j < i * i; j++) {
      x++;
    }
  }
  return x;
}

Step-by-Step Trace · Loop Analysis

Question 8: A Geometric Sum

What does func(10) print? Which class does the total number of inner-loop iterations belong to?

inner-loop iterations (= i)0i=10i=20i=40i=8

n = 10, count = 0. The outer loop doubles i on every iteration: 1, 2, 4, 8, …

inner-loop iterations (= i)1i=10i=20i=40i=8

i = 1 < 10 → the inner loop runs 1 time; count = 1.

inner-loop iterations (= i)1i=12i=20i=40i=8

i = 2 < 10 → the inner loop runs 2 times; count = 3.

inner-loop iterations (= i)1i=12i=24i=40i=8

i = 4 < 10 → the inner loop runs 4 times; count = 7.

inner-loop iterations (= i)1i=12i=24i=48i=8

i = 8 < 10 → the inner loop runs 8 times; count = 15.

inner-loop iterations (= i)1i=12i=24i=48i=8

i = 16 < 10 is false → the loop ends; 15 is printed. 1 + 2 + 4 + 8 = 2⁴ − 1 < 2n: although the outer loop runs log n times, the total is Θ(n).

void func(int n) {
  int count = 0;
  for (int i = 1; i < n; i *= 2) {
    for (int j = 0; j < i; j++) {
      count++;
    }
  }
  System.out.println(count);
}

Multiple Choice · Loop Analysis

Question 9: A Doubling Outer Loop

What is the time complexity of the following function?

public void foo(int n) {
  for (int i = 1; i < n; i = i * 2) {
    for (int j = 0; j < n; j++) {
      System.out.println(i + " " + j);
    }
  }
}

D: The outer loop runs ⌈log₂ n⌉ times (i = 1, 2, 4, …); the inner bound depends on n, not i, so it runs n times each time. For n = 16, 4 · 16 = 64 lines. With j < i the total would be 1 + 2 + 4 + … < 2n, i.e. Θ(n).

Multiple Choice · Loop Analysis

Question 10: The Tight Bound of f6

Which statement is true for f6, written with the loop for (int i = 0; i < n; i = 1 << i) (ignoring overflow)?

D: The update i ← 2ⁱ makes i grow like a power tower: 0, 1, 2, 4, 16, 65536. The number of iterations is the iterated logarithm (log* n). A would be right for the loop i = 2 · i; in f6 i itself becomes the exponent.

Output Question · Analyzing Recursion

Question 11: Two Calls vs. One Call

What does f3(5) return, and how many calls are made in total (including the first)? If the last line is changed to return 2 * f3(n - 1);, what are the return value and the number of calls?

public long f3(int n) {
  if (n <= 1) {
    return 1;
  }
  return f3(n - 1) + f3(n - 1);
}
Show answer
  • f3(5) = 16 (= 2ⁿ⁻¹), 31 calls (= 2ⁿ − 1): each call spawns two calls.
  • 2 * f3(n − 1) gives: the value is again 16, but only 5 calls: f3(5), f3(4), …, f3(1).

The recurrence drops from 2T(n − 1) + c to T(n − 1) + c: Θ(2ⁿ) → Θ(n). Solving the same subproblem twice is wasteful; computing the result once and reusing it is enough.

Multiple Choice · Analyzing Recursion

Question 12: Unrolling a Recurrence

If the recurrence T(n) = 2T(n − 1) + c, T(1) = c is solved by unrolling, which closed form is correct?

B: T(n) = 2T(n − 1) + c = 4T(n − 2) + 3c = 8T(n − 3) + 7c = … = 2ᵏ T(n − k) + (2ᵏ − 1)c. At k = n − 1, T(1) = c: 2ⁿ⁻¹c + (2ⁿ⁻¹ − 1)c = (2ⁿ − 1)c → Θ(2ⁿ). Because it is a subtractive recurrence, the master theorem does not apply.

Classic Question · Analyzing Recursion

Question 13: T(n) = T(n − 1) + n

Solve the recurrence T(n) = T(n − 1) + n, T(1) = 1 by unrolling. Why can't the master theorem be applied? In the worst case of which algorithm does this recurrence arise?

Show answer

T(n) = n + T(n − 1) = n + (n − 1) + T(n − 2) = … = n + (n − 1) + … + 2 + T(1) = n(n + 1)/2 → Θ(n²).

The master theorem requires the form T(n) = aT(n/b) + f(n); here the subproblem is n − 1, not n/b (a subtractive recurrence).

In quicksort where the pivot is always the smallest or largest element, every partition takes Θ(n) and separates only the pivot: T(n) = T(n − 1) + Θ(n). Insertion sort on reverse-sorted input gives the same sum.

Classic Question · Master Theorem

Question 14: Five Recurrences

Solve with the master theorem; for each, give a, b, log_b a, the degree c of f(n), and the case: (a) T(n) = 3T(n/3) + n, (b) T(n) = 9T(n/3) + n, (c) T(n) = T(n/2) + n, (d) T(n) = 4T(n/2) + n², (e) T(n) = 2T(n/2) + n log n.

Show answer
a, blog_b acCaseResult
(a)3, 3112Θ(n log n)
(b)9, 3211Θ(n²)
(c)1, 2013Θ(n)
(d)4, 2222Θ(n² log n)
(e)2, 211 (· log n)extendedΘ(n log² n)

(c) In Case 3, regularity: a · f(n/b) = n/2 ≤ ½ · n, k = ½ < 1 ✓. In (d) every level does n² work and there are log₂ n levels. (e) falls into none of the three basic cases; if f(n) = Θ(n^log_b a · logᵏ n) then T(n) = Θ(n^log_b a · logᵏ⁺¹ n).

Multiple Choice · Master Theorem

Question 15: Eight Subproblems

What is the solution of the recurrence T(n) = 8T(n/2) + n²?

A: log₂ 8 = 3 > c = 2 → Case 1 (leaves dominate): Θ(n^log₂ 8) = Θ(n³). This is divide-and-conquer matrix multiplication (four blocks, 8 multiplications), no better than the classic three loops. Strassen, with 7 multiplications, gets Θ(n^2.807).

Multiple Choice · Master Theorem

Question 16: A Recurrence That Does Not Apply

To which of the following recurrences does none of the three cases of the master theorem (in its basic form) apply?

C: Compare n^log₂ 2 = n with f(n) = n / log n: f(n) is smaller than n, but only by a log factor, not by a polynomial factor (n^ε); it fits neither Case 1 nor Case 2. A: Case 1 → Θ(n²); B: Case 3 → Θ(n²); D: log₄ 16 = 2 = c → Case 2 → Θ(n² log n).

Step-by-Step Trace · Application: f4 and f5

Question 17: Recursion Tree of f4(4)

In the call f4(4), in what order is the recursion tree built, how many calls are made, and what is the result?

f4(4)

f4(4): 4 > 1 → on line 5, the left call f4(4 / 2) = f4(2) is made first.

f4(2)f4(4)

f4(2): 2 > 1 → f4(1) is called.

f4(2)f4(1)=1f4(4)

f4(1): base case, 1 is returned.

f4(2)=2f4(1)=1f4(1)=1f4(4)

The second f4(1) also returns 1 → f4(2) = 1 + 1 = 2.

f4(4)f4(2)=2f4(1)=1f4(1)=1f4(2)

f4(4) calls the right f4(2): the same subtree is expanded a second time.

f4(1)=1f4(1)=1f4(4)f4(2)=2f4(1)=1f4(1)=1f4(2)=2

Two more f4(1) → this f4(2) also returns 2.

f4(1)=1f4(1)=1f4(4)=4f4(2)=2f4(1)=1f4(1)=1f4(2)=2

f4(4) = 2 + 2 = 4. 7 = 2n − 1 calls in total; level costs c, 2c, 4c: leaves dominate → Θ(n). Stack depth is only log₂ n + 1 = 3.

public int f4(int n) {
  if (n <= 1) {
    return 1;
  }
  return f4(n / 2) + f4(n / 2);
}

Output Question · Application: f4 and f5

Question 18: Values of f5

What do f5(4) and f5(32) return? (f1(n) returns n.) Compare the result with n log₂ n + n.

public int f5(int n) {
  if (n <= 1) {
    return 1;
  }
  return f1(n) + f5(n / 2) + f5(n / 2);
}
Show answer
  • f5(1) = 1, f5(2) = 2 + 1 + 1 = 4
  • f5(4) = 4 + 4 + 4 = 12 = 4 · 2 + 4
  • f5(8) = 32, f5(16) = 80 → f5(32) = 32 + 80 + 80 = 192 = 32 · 5 + 32

For n = 2ᵏ, f5(n) = n log₂ n + n: each of the log₂ n levels does n work, and the leaves add n. T(n) = 2T(n/2) + Θ(n) → Case 2 → Θ(n log n).

Step-by-Step Trace · Amortized Analysis

Question 19: Dynamic Array: 9 Insertions

9 elements are inserted into an empty array of capacity 1 that doubles when full. What is the cost of each insertion (writes + copies) and the total cost?

capacity 1e1

Insertion 1: capacity 1, array empty → 1 write. Cost = writes + number of copied elements.

capacity 2e1e2

Insertion 2: the array was full → capacity 1 → 2, 1 copy + 1 write = 2.

capacity 4e1e2e3

Insertion 3: the array was full → capacity 2 → 4, 2 copies + 1 write = 3.

capacity 4e1e2e3e4

Insertion 4: there is room → just 1 write.

capacity 8e1e2e3e4e5

Insertion 5: the array was full → capacity 4 → 8, 4 copies + 1 write = 5.

capacity 8e1e2e3e4e5e6

Insertion 6: there is room → just 1 write.

capacity 8e1e2e3e4e5e6e7

Insertion 7: there is room → just 1 write.

capacity 8e1e2e3e4e5e6e7e8

Insertion 8: there is room → just 1 write.

capacity 16e1e2e3e4e5e6e7e8e9

Insertion 9: the array was full → capacity 8 → 16, 8 copies + 1 write = 9. Total 24 < 3 · 9 = 27 → amortized Θ(1) per insertion.

void append(int x) {
  if (size == data.length) {
    int[] newData = new int[2 * data.length];
    for (int i = 0; i < size; i++)
      newData[i] = data[i];   // copy
    data = newData;
  }
  data[size++] = x;       // write
}

Multiple Choice · Amortized Analysis

Question 20: Amortized vs. Average

What is the difference between amortized analysis and average-case analysis?

A: Amortized cost is the total of a worst-case operation sequence divided by the number of operations; it assumes no input distribution. The average case is the expected time over a probability distribution of inputs (so D is wrong). In a dynamic array one insertion can be Θ(n), yet the amortized cost is Θ(1).

The End

Chapter 2: Algorithm Complexity: Study Questions
Lecture slides: Chapter 2: Algorithm Complexity · Lecture notes
1 / 1 Sercan KÜLCÜ. All rights reserved.