Chapter 2: Algorithm Analysis

Data Structures
Algorithms and efficiencyExperimental analysisCounting primitive operations O, Ω, ΘLoop patternsLinear and binary searchSpace complexityCorrectness and robustness
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Measuring and counting

Time an algorithm experimentally and analyze it by counting its primitive operations.

Asymptotic notation

Express time and space complexity with O, Ω, and Θ.

From code to complexity

Read best-, average-, and worst-case cost off loop patterns.

Comparing algorithms

Compare algorithms for the same problem by cost, correctness, and robustness.

Algorithms and Measurement

Algorithms

  • An algorithm is a step-by-step procedure for solving a problem: input → steps → output.
  • One problem can have many algorithms, and some are more efficient than others.
  • Choosing the right one matters: a good algorithm does the job with fewer resources.
input5291algorithm (sorting)finite · definite · effective stepsoutput1259

Some common algorithms:

SortingPuts data in a given order.bubble sort, merge sort
SearchingFinds a given item in a data set.linear search, binary search
GraphSolves problems on graphs.shortest paths, network flow

Algorithms and Measurement

What Makes a Good Algorithm

A good algorithm solves its problem effectively. We judge it by four criteria:

Time complexityMeasured by running time or by the number of operations.
Space complexityMeasured by the amount of memory the algorithm uses.
CorrectnessIt must give the right result for every valid input; wrong answers make it useless.
RobustnessIt should cope with a wide range of inputs and situations.

Algorithms and Measurement · Measuring

Experimental Analysis: System.nanoTime()

for (int n = 1000; n <= 64000; n *= 2) {
  int[] a = new int[n];
  for (int i = 0; i < n; i++) a[i] = i; // distinct
  long start = System.nanoTime();
  hasDuplicates(a);                          // timed work
  long elapsed = System.nanoTime() - start;  // ns
  System.out.println(n + " " + elapsed / 1e6 + " ms");
}
MeasureSubtracting two System.nanoTime() readings gives elapsed ns (not wall-clock time).
RepeatJIT warm-up, garbage collection, and other processes add noise: run several times, take the median.
PlotPlot time against n and see how the time changes when n doubles.
Careful: new int[n] is all zeros → hasDuplicates (an O(n²) duplicate check) returns true on its first comparison. To time the worst case, use distinct data.

Algorithms and Measurement · Measuring

Doubling Experiments and the Limits of Measurement

2×4×6×8×2.0×O(n)2.2×O(n log n)4.0×O(n²)8.0×O(n³)n = 1000 → 2000: how many times longer?

Limits of experimental analysis (Goodrich §4.1):

  • Results depend on the hardware, the JVM, and the load; compare two algorithms in the same environment.
  • Only a limited set of inputs can be tried; an important one may be missed.
  • The algorithm must be fully implemented before it can be timed.
Doubling experiment: if T(2n)/T(n) ≈ 2k, then T(n) ≈ c·nk. To get past these limits, analyze by counting primitive operations: hardware-independent, valid for all inputs, no code to run.

Algorithms and Measurement · Counting steps

Counting Primitive Operations: arrayMax()

int arrayMax(int[] a) {
  int currentMax = a[0];
  for (int i = 1; i < a.length; i++) {
    if (a[i] > currentMax)
      currentMax = a[i];
  }
  return currentMax;
}
  • Primitive operation: assignment, comparison, arithmetic, array indexing, method call / return.
  • Each takes constant time → we only count how many times each one runs.
LineStatementCount
2currentMax = a[0]1
3i = 11
3i < a.lengthn
4a[i] > currentMaxn − 1
5currentMax = a[i]0 … n − 1
3i++n − 1
7return1
Best: 3n + 1 (a[0] is the largest, no updates) · Worst: 4n (increasing array) → Θ(n) either way. Constants depend on the hardware, so we drop them.

Algorithms and Measurement · Step through the code

Code Trace: arrayMax(a)

0318223944a[0..0] scannedcurrentMax3

currentMax = a[0] = 3: the first item is the largest so far.

0318223944▲ ia[0..0] scannedcurrentMax3

i = 1, 1 < 5 → continue. Invariant: currentMax = max(a[0..0]).

0318223944▲ ia[0..0] scannedcurrentMax3

a[1] = 8 > 3 → true.

0318223944▲ ia[0..1] scannedcurrentMax8

currentMax = 8: a new maximum.

0318223944▲ ia[0..1] scannedcurrentMax8

i = 2, 2 < 5 → continue. Invariant: currentMax = max(a[0..1]).

0318223944▲ ia[0..2] scannedcurrentMax8

a[2] = 2 > 8 → false; currentMax is unchanged.

0318223944▲ ia[0..2] scannedcurrentMax8

i = 3, 3 < 5 → continue. Invariant: currentMax = max(a[0..2]).

0318223944▲ ia[0..2] scannedcurrentMax8

a[3] = 9 > 8 → true.

0318223944▲ ia[0..3] scannedcurrentMax9

currentMax = 9: a new maximum.

0318223944▲ ia[0..3] scannedcurrentMax9

i = 4, 4 < 5 → continue. Invariant: currentMax = max(a[0..3]).

0318223944▲ ia[0..4] scannedcurrentMax9

a[4] = 4 > 9 → false; currentMax is unchanged.

0318223944a[0..4] scannedcurrentMax9

i = 5: 5 < 5 is false → loop ends. The whole array has been scanned.

0318223944a[0..4] scannedcurrentMax9

return 9. n − 1 = 4 comparisons for n = 5 → Θ(n).

int arrayMax(int[] a) {
  int currentMax = a[0];
  for (int i = 1; i < a.length; i++) {
    if (a[i] > currentMax)
      currentMax = a[i];
  }
  return currentMax;
}

Asymptotic Notation

Time Complexity and Big O

  • Expresses an algorithm's running time as a function of the input size n.
  • Big O gives an upper bound on running time; it is usually quoted for the worst case.
  • O(1): the operation count does not grow with the input size, though the time need not be identical for every input.
  • O(n): time grows linearly with the input.
Operations as n grows
Only the fastest-growing term matters in Big O; constants and lower-order terms are dropped:
3n2 + 5n + 2 → O(n2)  ·  4n + 100 → O(n)  ·  7 → O(1)

Asymptotic Notation

O, Ω, and Θ: Definition and Graphs

Definition: f(n) = O(g(n)) ⇔ there exist c > 0 and n₀ ≥ 1 such that f(n) ≤ c·g(n) for every n ≥ n₀. For Ω use ≥; for Θ, both.
2040600481216n →n₀ = 5c·g(n) = 4nf(n) = 2n + 10

Big O: 2n + 10 ≤ 4·n for n ≥ n₀ = 5 → f(n) = O(n) (c = 4). For small n (n < 5) f(n) is larger, which does not matter.

2040600481216n →c·g(n) = 2nf(n) = 2n + 10

Big Omega: 2n + 10 ≥ 2·n for every n ≥ 1 → f(n) = Ω(n): f grows no more slowly than n (lower bound).

2040600481216n →n₀ = 54n2nf(n)

Big Theta: 2n ≤ f(n) ≤ 4n for n ≥ 5 → f(n) = Θ(n): squeezed between two lines, it grows exactly like n.

Asymptotic Notation

Asymptotic Bounds and Cases

O(g(n)) · Upper boundFor large enough n, the work grows no faster than g(n).
Ω(g(n)) · Lower boundFor large enough n, the work grows no more slowly than g(n).
Θ(g(n)) · Tight boundThe upper and lower bounds match; the growth rate is in the same class as g(n).
AnalysisQuestionLinear search
Best caseHow many operations on the most favorable valid input?Θ(1) if the first item is the target
Average caseWhat is the expected cost under an assumed input distribution?Θ(n) if all positions are equally likely
Worst caseWhat cost is guaranteed for every valid input?Θ(n) if the target is last or absent
Amortized analysis: A single operation may be expensive, so we divide the total cost of a long sequence of operations by their number. In a growable array, for example, a resize costs O(n), yet appending is amortized O(1).

Asymptotic Notation · Case analysis

Best, Average, and Worst Case

Linear search on [4, 8, 15, 16, 23, 42]: comparisons for each possible target:

0246481516234250worst = 6average ≈ 3.86best = 1target (50 is not in the array)comparisons
  • For the same n, the cost depends on the input.
  • The worst case is a guarantee and easy to compute → it is what we usually report.
  • The average assumes a probability distribution (here 7 equally likely targets, including the absent 50: 27/7 ≈ 3.86).
  • The best case alone is misleading.

Asymptotic Notation

Big O Examples

NotationNameExampleWhy?
O(1)ConstantAccessing the first array elementAccess time is the same whatever the array size
O(log n)LogarithmicBinary search in a sorted arrayEach step halves the search range
O(n)LinearScanning an array end to endEach item is processed once
O(n log n)LinearithmicMerge sortDivide and conquer: log n levels, n work per level
O(n2)QuadraticComparing all pairs of itemsTwo nested loops
O(2n)ExponentialListing all subsetsA set of n items has 2n subsets

Asymptotic Notation

What Happens as n Grows?

Complexityn = 10n = 100n = 1,000
log2 n3.36.610
n101001,000
n log2 n336649,966
n210010,0001,000,000
2n1,024≈ 1.3·1030≈ 1.1·10301
At 109 operations per second, 2n steps for n = 100 take about 4·1013 years. The universe is ≈ 1.4·1010 years old.

From Code to Complexity · Counting steps

From Loop Patterns to Complexity

PatternBody runsn = 16Class
for (i = 0; i < n; i++)n16O(n)
for (i = 0; i < n; i += 2)n / 28O(n)
for (i = 1; i < n; i *= 2)⌈log₂ n⌉4O(log n)
nested: i < n, j < nn²256O(n²)
nested: i < n, j < in(n−1)/2120O(n²)
two consecutive i < n loops2n32O(n)
Consecutive blockscosts add, and the largest wins: O(n) + O(n²) = O(n²)
Nested loopscosts multiply: outer × inner (sum instead if the inner loop depends on the outer)

From Code to Complexity · Step through the code

Code Trace: countSteps(16) — i doubles every round

12345678910111213141516n

countSteps(16): how many doublings does it take to reach n?

12345678910111213141516n

Counter k = 0.

12345678910111213141516▲ in

i = 1 < 16 → body runs.

12345678910111213141516▲ in

k++ → k = 1.

12345678910111213141516▲ in

i = 2 < 16 → body runs.

12345678910111213141516▲ in

k++ → k = 2.

12345678910111213141516▲ in

i = 4 < 16 → body runs.

12345678910111213141516▲ in

k++ → k = 3.

12345678910111213141516▲ in

i = 8 < 16 → body runs.

12345678910111213141516▲ in

k++ → k = 4.

12345678910111213141516▲ i = n

i *= 2 → i = 16; 16 < 16 is false → loop ends.

12345678910111213141516n

return 4 = log₂ 16. Making n 1000 times larger adds only ~10 rounds → O(log n).

int countSteps(int n) {
  int k = 0;
  for (int i = 1; i < n; i *= 2)
    k++;
  return k;
}

From Code to Complexity

From Code to Complexity

O(1)
int first(int[] a) {
    return a[0];
}
O(n)
int sum(int[] a) {
    int s = 0;
    for (int x : a)  // n times
        s += x;
    return s;
}
O(n²)
boolean hasDuplicates(int[] a) {
    for (int i = 0; i < a.length; i++)
        for (int j = i + 1; j < a.length; j++)
            if (a[i] == a[j])
                return true;
    return false;
}
// outer loop n, inner ≈ n/2
// → n(n−1)/2 comparisons

Rule: drop constants and lower-order terms. n(n−1)/2 = n²/2 − n/2 → O(n²)

From Code to Complexity · Step through the code

Code Trace: sum()

071229

main calls sum(a), and a new call frame is pushed.

071229

Inside the method, a refers to the same array (no copy is made).

071229

The running total starts at s = 0.

071229

Round 1: x = a[0] = 7.

071229

s += x → s = 0 + 7 = 7.

071229

Round 2: x = a[1] = 2.

071229

s += x → s = 7 + 2 = 9.

071229

Round 3: x = a[2] = 9.

071229

s += x → s = 9 + 9 = 18.

071229

No items left; the loop exits.

071229

return 18: the frame is popped and the value goes back to the caller.

071229

println prints 18. The loop body ran n = 3 times → O(n).

int sum(int[] a) {
    int s = 0;
    for (int x : a) {
        s += x;
    }
    return s;
}

// in main:
System.out.println(sum(a));

From Code to Complexity · Step through the code

Code Trace: linearSearch(a, 23)

205182123164235386567728919

Searching a sorted array for 23. Linear search ignores the order and checks items one by one from the start.

205182123164235386567728919▲ i

a[0] = 2 ≠ 23 → i moves on. Each step rules out only one item.

205182123164235386567728919▲ i

a[1] = 5 ≠ 23 → i moves on.

205182123164235386567728919▲ i

a[2] = 8 ≠ 23 → i moves on.

205182123164235386567728919▲ i

a[3] = 12 ≠ 23 → i moves on.

205182123164235386567728919▲ i

a[4] = 16 ≠ 23 → i moves on.

205182123164235386567728919▲ i

a[5] = 23 == 23 → return 5. 6 comparisons; with the target last or absent, n = 10 → O(n).

int linearSearch(int[] a, int target) {
    for (int i = 0; i < a.length; i++)
        if (a[i] == target) return i;
    return -1;
}

From Code to Complexity · Step through the code

Code Trace: binarySearch(a, 23)

021528312416523638756872991▲ lo▲ hi

The search range is the whole array: lo = 0, hi = 9.

021528312416523638756872991▲ lo▲ hi

0 ≤ 9 → range nonempty, round 1 begins.

021528312416523638756872991▲ lo▲ hi▲ mid

mid = (0 + 9) / 2 = 4, a[4] = 16.

021528312416523638756872991▲ lo▲ hi

16 < 23 → the target is in the right half: lo = mid + 1 = 5. Left half discarded.

021528312416523638756872991▲ lo▲ hi

5 ≤ 9 → range nonempty, round 2 begins.

021528312416523638756872991▲ lo▲ hi▲ mid

mid = (5 + 9) / 2 = 7, a[7] = 56.

021528312416523638756872991▲ lo▲ hi

56 > 23 → the target is in the left half: hi = mid − 1 = 6. Right half discarded.

021528312416523638756872991▲ lo▲ hi

5 ≤ 6 → range nonempty, round 3 begins.

021528312416523638756872991▲ lomid▲ hi

mid = (5 + 6) / 2 = 5, a[5] = 23.

021528312416523638756872991▲ lomid▲ hi

23 == 23 → found, return 5. Only 3 rounds; linear search needed 6 comparisons. Each round halves the range → O(log n).

int binarySearch(int[] a, int target) {
    int lo = 0, hi = a.length - 1;
    while (lo <= hi) {
        int mid = (lo + hi) / 2;
        if (a[mid] == target) return mid;
        if (a[mid] < target) lo = mid + 1;
        else hi = mid - 1;
    }
    return -1;
}

From Code to Complexity · Step through the code

Code Trace: binarySearch(a, 40) — target absent

021528312416523638756872991▲ lo▲ hi

This time 40 is absent: the worst case. The range is the whole array.

021528312416523638756872991▲ lo▲ hi

0 ≤ 9 → range nonempty, round 1.

021528312416523638756872991▲ lo▲ hi▲ mid

Round 1: mid = (0 + 9) / 2 = 4, a[4] = 16.

021528312416523638756872991▲ lo▲ hi

16 < 40 → lo = 5; left side discarded.

021528312416523638756872991▲ lo▲ hi

5 ≤ 9 → range nonempty, round 2.

021528312416523638756872991▲ lo▲ hi▲ mid

Round 2: mid = (5 + 9) / 2 = 7, a[7] = 56.

021528312416523638756872991▲ lo▲ hi

56 > 40 → hi = 6; right side discarded.

021528312416523638756872991▲ lo▲ hi

5 ≤ 6 → range nonempty, round 3.

021528312416523638756872991▲ lomid▲ hi

Round 3: mid = (5 + 6) / 2 = 5, a[5] = 23.

021528312416523638756872991▲ lohi

23 < 40 → lo = 6; left side discarded.

021528312416523638756872991▲ lohi

6 ≤ 6 → range nonempty, round 4.

021528312416523638756872991▲ lohimid

Round 4: mid = (6 + 6) / 2 = 6, a[6] = 38.

021528312416523638756872991▲ hi▲ lo

38 < 40 → lo = 7; left side discarded.

021528312416523638756872991▲ hi▲ lo

7 ≤ 6 is false → range empty, loop ends.

021528312416523638756872991▲ hi▲ lo

return -1. Even in the worst case, only 4 rounds = ⌊log₂ 10⌋ + 1; linear search would need 10 comparisons.

int binarySearch(int[] a, int target) {
    int lo = 0, hi = a.length - 1;
    while (lo <= hi) {
        int mid = (lo + hi) / 2;
        if (a[mid] == target) return mid;
        if (a[mid] < target) lo = mid + 1;
        else hi = mid - 1;
    }
    return -1;
}

From Code to Complexity · Step through the code

Code Trace: hasDuplicates()

04172137▲ i

Outer loop: i = 0 (a[i] = 4). j starts at i + 1.

04172137▲ i▲ j

a[0] = 4 vs. a[1] = 7 → different, j moves on.

04172137▲ i▲ j

a[0] = 4 vs. a[2] = 1 → different, j moves on.

04172137▲ i▲ j

a[0] = 4 vs. a[3] = 7 → different, j moves on.

04172137▲ i

Outer loop: i = 1 (a[i] = 7). j starts at i + 1.

04172137▲ i▲ j

a[1] = 7 vs. a[2] = 1 → different, j moves on.

04172137▲ i▲ j

a[1] = 7 and a[3] = 7 are equal → return true. The worst case (no duplicates) takes n(n−1)/2 = 6 comparisons → O(n²).

boolean hasDuplicates(int[] a) {
    for (int i = 0; i < a.length; i++)
        for (int j = i + 1; j < a.length; j++)
            if (a[i] == a[j])
                return true;
    return false;
}

From Code to Complexity · One problem, two algorithms

Example Problem: Prefix Averages

Input x[0..n−1]; output a[j] = (x[0] + … + x[j]) / (j + 1). Used for moving averages and financial trend analysis (Goodrich §4.3).

x04182632a4.0a[0] = (4) / 1 = 4.0

a[0] = average of x[0] … x[0] = 4.0.

x04182632a4.06.0a[1] = (4 + 8) / 2 = 6.0

a[1] = average of x[0] … x[1] = 6.0.

x04182632a4.06.06.0a[2] = (4 + 8 + 6) / 3 = 6.0

a[2] = average of x[0] … x[2] = 6.0.

x04182632a4.06.06.05.0a[3] = (4 + 8 + 6 + 2) / 4 = 5.0

a[3] = average of x[0] … x[3] = 5.0.

Two solutions: re-sum from scratch for each j (O(n²)) · carry a running sum (O(n)).

From Code to Complexity · Step through the code

Code Trace: prefixAverage1 — re-sum from scratch for each j

x04182632a

n = 4; the result array a is created, filled with 0.0.

x04182632atotal0.0

j = 0: total = 0 — each j re-sums from scratch.

x04182632atotal4.0

total += x[0] = 4 → total = 4.0.

x04182632a4.0total4.0

a[0] = 4.0 / 1 = 4.0.

x04182632a4.0total0.0

j = 1: total = 0 — each j re-sums from scratch.

x04182632a4.0total4.0

total += x[0] = 4 → total = 4.0.

x04182632a4.0total12.0

total += x[1] = 8 → total = 12.0.

x04182632a4.06.0total12.0

a[1] = 12.0 / 2 = 6.0.

x04182632a4.06.0total0.0

j = 2: total = 0 — each j re-sums from scratch.

x04182632a4.06.0total4.0

total += x[0] = 4 → total = 4.0.

x04182632a4.06.0total12.0

total += x[1] = 8 → total = 12.0.

x04182632a4.06.0total18.0

total += x[2] = 6 → total = 18.0.

x04182632a4.06.06.0total18.0

a[2] = 18.0 / 3 = 6.0.

x04182632a4.06.06.0total0.0

j = 3: total = 0 — each j re-sums from scratch.

x04182632a4.06.06.0total4.0

total += x[0] = 4 → total = 4.0.

x04182632a4.06.06.0total12.0

total += x[1] = 8 → total = 12.0.

x04182632a4.06.06.0total18.0

total += x[2] = 6 → total = 18.0.

x04182632a4.06.06.0total20.0

total += x[3] = 2 → total = 20.0.

x04182632a4.06.06.05.0total20.0

a[3] = 20.0 / 4 = 5.0.

x04182632a4.06.06.05.0

return a. Additions: 1 + 2 + 3 + 4 = 10 = n(n+1)/2 → O(n²).

double[] prefixAverage1(double[] x) {
  int n = x.length;
  double[] a = new double[n];
  for (int j = 0; j < n; j++) {
    double total = 0;
    for (int i = 0; i <= j; i++)
      total += x[i];
    a[j] = total / (j + 1);
  }
  return a;
}

From Code to Complexity · Step through the code

Code Trace: prefixAverage2 — running sum

x04182632atotal0.0

total is set to 0 once, before the loop, and carried across rounds.

x04182632atotal4.0

j = 0: total += x[0] = 4 → total = 4.0 (previous sum reused).

x04182632a4.0total4.0

a[0] = 4.0 / 1 = 4.0.

x04182632a4.0total12.0

j = 1: total += x[1] = 8 → total = 12.0 (previous sum reused).

x04182632a4.06.0total12.0

a[1] = 12.0 / 2 = 6.0.

x04182632a4.06.0total18.0

j = 2: total += x[2] = 6 → total = 18.0 (previous sum reused).

x04182632a4.06.06.0total18.0

a[2] = 18.0 / 3 = 6.0.

x04182632a4.06.06.0total20.0

j = 3: total += x[3] = 2 → total = 20.0 (previous sum reused).

x04182632a4.06.06.05.0total20.0

a[3] = 20.0 / 4 = 5.0.

x04182632a4.06.06.05.0

Same result with only 4 additions (prefixAverage1: 10). One loop → O(n).

double[] prefixAverage2(double[] x) {
  int n = x.length;
  double[] a = new double[n];
  double total = 0;
  for (int j = 0; j < n; j++) {
    total += x[j];
    a[j] = total / (j + 1);
  }
  return a;
}

From Code to Complexity · One problem, two algorithms

Comparing the Two Solutions

20406078123456789101112nprefixAverage1: n(n+1)/2prefixAverage2: n
nprefixAverage1prefixAverage2
105510
1005,050100
1,000500,5001,000
1,000,000500,000,500,0001,000,000
The gain comes from removing repeated work: prefixAverage2 remembers the previous sum. For n = 10⁶, that is 10⁶ additions instead of ~5·10¹¹.

Space and Correctness

Space Complexity

  • Measures how much memory an algorithm uses, usually in Big O notation.
  • Describes how memory use grows with the input size (usually not counting the input itself: extra space).
  • Memory efficiency matters on systems with limited memory.
O(1) · ConstantExample: creating one variable. Memory use does not change with the input size.s
O(n) · LinearExample: storing every item of an array, like the array a in prefixAverage.n cells
O(n2) · QuadraticExample: storing an n×n matrix. Memory grows with the square of n.n × n

Space and Correctness

Correctness

  • A correct algorithm produces the expected output for every valid input in finite time.
  • A fast algorithm that gives wrong answers is worthless.

How do we show correctness?

Test casesCompare outputs with known correct results.
Edge casesEmpty input, a single item, the largest/smallest values.
Loop invariantA proof via a property that holds at every step.
CounterexampleA single wrong result proves the algorithm incorrect.
0318223944▲ icurrentMax = max(a[0..2]) = 8Invariant:before each round, currentMaxis the largest in a[0..i−1].At exit i = n → correct result.

Space and Correctness

Robustness

  • An algorithm's ability to cope with unexpected situations.
  • A robust algorithm handles even out-of-range inputs without crashing or giving wrong results.

How do we test robustness?

Invalid inputCoping with inappropriate or unexpected input.null, negative size
Bad dataCoping with corrupt or missing data.missing rows, wrong format
Large data setsHandling very large inputs.10⁸ records
OverloadHow the algorithm behaves under heavy load.thousands of simultaneous requests

Summary

Common Mistakes

“O means worst case”O denotes an upper bound; best and worst case are a separate idea. Any case can be described with O, Ω, or Θ.
“O(1) is always fast”The constant can be large: an O(1) operation taking 1000 steps per call is slower than an n-step O(n) one while n < 1000.
“Nested loops mean n²”If the inner loop advances by j *= 2, it is n log n; if it runs a fixed number of times, O(n).
“The log base matters”log₂ n and log₁₀ n differ only by a constant factor: both are O(log n).
(lo + hi) / 2 overflowFor very large arrays the int sum overflows; lo + (hi - lo) / 2 is safe.
Comparing arrays with a == b== compares references; use Arrays.equals(a, b) for the contents.

Lab · Java code, step by step

Lab: Binary Search and Prefix Averages

Summary

Summary

TopicKey points
Experimental analysistime depends on hardware and input; doubling experiment: T(2n)/T(n) ≈ 2k → T(n) ≈ c·nk
Counting operationsassignment, comparison, arithmetic, array access; hardware-independent, valid for all inputs
O, Ω, Θupper, lower, and tight bounds; constant factors and lower-order terms are dropped
Casesbest, average, worst; the worst case for a guarantee, amortized for long sequences of operations
Loop patternssingle loop O(n) · nested O(n²) · halving O(log n) · outer n, inner log n → O(n log n)
Choosing an algorithmprefix averages O(n²) → O(n); binary search in a sorted array O(log n)
Space, correctnessextra memory is measured in Big O too; a fast algorithm that is not correct and robust is worthless

Summary

Operation Costs in Basic Data Structures

Data structureAccess item iSearchInsertDelete
ArrayO(1)O(n)O(n)O(n)
Linked listO(n)O(n)O(1)†O(1)†
Stacktop onlyO(n)O(1) pushO(1) pop
Queuefront onlyO(n)O(1) enqueueO(1) dequeue
Map / set (hash)—O(1)*O(1)*O(1)*

† If the position (the previous node) is known.   * Average case; O(n) in the worst case.

Summary

Sample Problems

  1. What is the time complexity of an algorithm that compares every pair of items?
  2. Which Θ class does f(n) = 5n³ + 2n² + 100 belong to?
  3. How many rounds does binary search take in the worst case on a sorted array of n = 1024 items?
  4. How many times does the loop for (int i = 1; i < n; i *= 2) run?
  5. What are the best and worst cases of linear search?
  6. Algorithm A takes 100n steps, algorithm B takes n². For which values of n is A faster?
Show answers
  1. n(n − 1)/2 comparisons → O(n2).
  2. Θ(n³): the fastest-growing term decides; constants are dropped.
  3. ⌊log₂ 1024⌋ + 1 = 11 rounds.
  4. i = 1, 2, 4, …: ⌈log₂ n⌉ rounds → O(log n).
  5. Best: the target is the first item, Θ(1). Worst: the target is last or absent, Θ(n).
  6. 100n < n² ⇔ n > 100: for large n, the slower-growing algorithm wins.

Summary

Exercises

  1. What is the Big O of this code?
    for (int i = n; i > 1; i /= 2)
      for (int j = 0; j < n; j++) s++;
  2. Show that f(n) = 3n² + 5n + 2 = O(n²) by giving c and n₀.
  3. In a doubling experiment, doubling n multiplies the time by ≈ 8. What class is T(n) in?
  4. How many additions do prefixAverage1 and prefixAverage2 each make for n = 5?
  5. After for (i = 0; i < n; i++) for (j = i; j < n; j++) s++;, what is s?
  6. An O(n²) algorithm takes 2 seconds for n = 1000. About how long will it take for n = 4000?
Answers / hints
  1. The outer loop runs ⌊log₂ n⌋ rounds, the inner loop n → O(n log n).
  2. For n ≥ 1, 5n ≤ 5n² and 2 ≤ 2n² → f(n) ≤ 10n²: c = 10, n₀ = 1.
  3. 2³ = 8 → O(n³) (cubic).
  4. prefixAverage1: 1+2+3+4+5 = 15; prefixAverage2: 5.
  5. n + (n − 1) + … + 1 = n(n + 1)/2 → O(n²).
  6. Quadrupling n multiplies the time by 4² = 16: ≈ 32 seconds.

Summary

Further Reading

  • Goodrich, Tamassia, and Goldwasser, Data Structures and Algorithms in Java, 6th ed., 2014.
  • Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, 4th ed., 2022.
  • Sedgewick and Wayne, Algorithms, 4th ed., 2011.

The End

Chapter 2: Algorithm Analysis
Next deck: Chapter 3: Arrays · Study questions · Glossary
1 / 1 Sercan KÜLCÜ. All rights reserved.