Chapter 4.1: Algorithms
Introduction
Learning Outcomes
Pin down the inputs, outputs and constraints.
Break the solution into clear, finite steps.
Trace and test the algorithm on examples.
Evaluate different searching and sorting approaches.
Problem Solving
Algorithm First, Code Second
- Code is one implementation of a solution idea, in one particular language.
- Code written without an algorithm leads to trial and error and hidden bugs.
- The right order: understand the problem, solve an example, write the method, then code it.
Problem Solving
Problem Definition
Which problem are we solving?
What data does the program receive?
What result must it produce?
What are the valid ranges and conditions?
Which small example confirms the expected result?
What happens on invalid or missing input?
Algorithm
Properties of a Good Algorithm
An algorithm is a finite sequence of well-defined steps that solves a given problem.
- It takes zero or more inputs and produces at least one output.
- Its steps are unambiguous, ordered and executable.
- It halts after a finite number of steps.
- It produces the correct result for every valid input.
- Its resource use stays acceptable as the input grows.
Algorithm
The Algorithm Design Cycle
- If a hand trace reveals a bug, fix it and go around again; once no bugs remain, start coding.
- When several solutions exist, compare their correctness, readability and efficiency.
Algorithm
Pseudocode
Pseudocode describes an algorithm's logic without tying it to a programming language.
START read A read B SUM = A + B print SUM END
- Use short, consistent phrases close to natural language.
- Indentation shows the scope of conditions and loops.
- The point is clarity, not syntax.
Examples
Example: Sort Two Numbers
read a, b if a > b then temp = a a = b b = temp print a, b
- Equal values need no swap.
- A swap takes three assignments.
- The output must satisfy a ≤ b.
temp? Writing a = b; b = a directly overwrites a's old value in the first assignment: for a = 7, b = 3 the result is 3 3.Examples · Step-by-step trace
Sort Two Numbers Step by Step (a = 7, b = 3)
Read the inputs: a = 7, b = 3. Goal: print the smaller one first.
Condition: 7 > 3 → true, so we enter the swap block.
temp = a = 7: back up a's value first so it isn't lost.
a = b = 3. The old 7 now lives only in temp.
b = temp = 7. Three assignments, and the values have swapped.
Output 3 7: a ≤ b holds. (With a = 2, b = 9 the condition is false and lines 3–5 are skipped.)
read a, b if a > b then temp = a a = b b = temp print a, b
Examples
Example: Factorial
read n if n < 0 then print "error" and stop result = 1 i = 2 while i ≤ n result = result × i i = i + 1 print result
- 0! and 1! are 1: the loop never runs and result stays 1.
- Loop invariant: at the start of each round, result = (i − 1)!
- On negative input, report an error and stop.
Examples · Step-by-step trace
Factorial Step by Step (n = 5)
Read n = 5. Expected: 5! = 1×2×3×4×5.
Condition: 5 < 0 → false; no error, continue.
A product starts at 1 (starting at 0 would make everything 0).
i = 2: multiplying by 1 changes nothing, so start at 2.
Condition: 2 ≤ 5 → true.
result = 1 × 2 = 2 (= 2!).
i = 3.
Condition: 3 ≤ 5 → true.
result = 2 × 3 = 6 (= 3!).
i = 4.
Condition: 4 ≤ 5 → true.
result = 6 × 4 = 24 (= 4!).
i = 5.
Condition: 5 ≤ 5 → true.
result = 24 × 5 = 120 (= 5!).
i = 6.
Condition: 6 ≤ 5 → false, so the loop ends.
Print 120. The loop ran n − 1 = 4 times.
read n if n < 0 then print "error" and stop result = 1 i = 2 while i ≤ n result = result × i i = i + 1 print result
Examples
Example: Fibonacci
read n
a = 0, b = 1
for round = 0 to n-1
print a
next = a + b
a = b
b = next- Keep the two previous values: a and b form a window that slides one step right each round.
- The temporary variable prevents losing information during the update.
Examples · Step-by-step trace
Fibonacci Step by Step (n = 8, rounds 0–2)
n = 8: print the first 8 Fibonacci numbers.
The first two terms: a = 0, b = 1.
Round 0 (0…7): print a = 0.
next = 0 + 1 = 1.
a = b = 1: the window slides.
b = next = 1. Order matters: without a = b first, the old b would be lost.
Round 1 (0…7): print a = 1.
next = 1 + 1 = 2.
a = b = 1: the window slides.
b = next = 2.
Round 2 (0…7): print a = 1.
next = 1 + 2 = 3.
a = b = 2: the window slides.
b = next = 3.
read n a = 0, b = 1 for round = 0 to n-1 print a next = a + b a = b b = next
Examples · Step-by-step trace
Fibonacci Step by Step (cont.: rounds 3–5)
Round 3 (0…7): print a = 2.
next = 2 + 3 = 5.
a = b = 3: the window slides.
b = next = 5.
Round 4 (0…7): print a = 3.
next = 3 + 5 = 8.
a = b = 5: the window slides.
b = next = 8.
Round 5 (0…7): print a = 5.
next = 5 + 8 = 13.
a = b = 8: the window slides.
b = next = 13.
read n a = 0, b = 1 for round = 0 to n-1 print a next = a + b a = b b = next
Examples · Step-by-step trace
Fibonacci Step by Step (cont.: rounds 6–7)
Round 6 (0…7): print a = 8.
next = 8 + 13 = 21.
a = b = 13: the window slides.
b = next = 21.
Round 7 (0…7): print a = 13.
next = 13 + 21 = 34.
a = b = 21: the window slides.
b = next = 34.
All 8 rounds done, so the loop ends. Output: 0 1 1 2 3 5 8 13. The last term printed is 13.
read n a = 0, b = 1 for round = 0 to n-1 print a next = a + b a = b b = next
Examples
Example: Palindrome
read text clean text by the rule lo = 0, hi = length - 1 while lo < hi if the characters differ, return false lo = lo + 1, hi = hi - 1 return true
- Comparison moves from both ends toward the middle.
- Decide the case and space rules up front (e.g. "Kayak" → "kayak").
- At most length / 2 comparisons.
Examples · Step-by-step trace
Palindrome Step by Step ("Kayak")
Read the text: "Kayak". The first letter is uppercase.
Rule: lowercase it and drop spaces → "kayak". Otherwise K ≠ k would be a mismatch.
The two ends: lo = 0, hi = 4.
Condition: 0 < 4 → true.
text[0] = 'k', text[4] = 'k' → equal, continue.
One step inward: lo = 1, hi = 3.
Condition: 1 < 3 → true.
text[1] = 'a', text[3] = 'a' → equal, continue.
One step inward: lo = 2, hi = 2.
Condition: 2 < 2 → false: we reached the middle, so the loop ends.
No mismatched pair → true: "Kayak" is a palindrome. Only 2 comparisons for 5 letters.
read text clean text by the rule lo = 0, hi = length - 1 while lo < hi if the characters differ, return false lo = lo + 1, hi = hi - 1 return true
Examples
Example: Prime Number
read n if n < 2 then print "not prime" and stop i = 2 while i × i ≤ n if n % i == 0 then print "not prime" and stop i = i + 1 print "prime"
- The search stops at the first divisor.
- Square-root bound: if n = a × b, then a and b cannot both exceed √n; any small divisor lies in the range i × i ≤ n.
- % gives the remainder of a division.
Examples · Step-by-step trace
Prime Number Step by Step (n = 29)
Read n = 29.
29 < 2 → false; not 0 or 1, continue.
i = 2: the first candidate divisor.
Condition: 2×2 = 4 ≤ 29 → true.
29 % 2 = 1 → not zero, so i is not a divisor.
i = 3.
Condition: 3×3 = 9 ≤ 29 → true.
29 % 3 = 2 → not zero, so i is not a divisor.
i = 4.
Condition: 4×4 = 16 ≤ 29 → true.
29 % 4 = 1 → not zero, so i is not a divisor.
i = 5.
Condition: 5×5 = 25 ≤ 29 → true.
29 % 5 = 4 → not zero, so i is not a divisor.
i = 6.
Condition: 6×6 = 36 ≤ 29 → false: i has passed √29, so there is no need to go on.
No divisor found → prime. Only 4 divisors tried (instead of 2…28). For n = 21, i = 3 gives 21 % 3 = 0 → not prime.
read n if n < 2 then print "not prime" and stop i = 2 while i × i ≤ n if n % i == 0 then print "not prime" and stop i = i + 1 print "prime"
Verify
Tracing an Algorithm
- Record each variable's value at every step in a trace table.
- Try normal, boundary and invalid inputs separately.
- When expected and actual results differ, find the first step where they diverge.
Trace table for factorial, n = 4 (i: value at the start of the round, result: value at the end):
| Step | i | result | i ≤ n |
|---|---|---|---|
| start | 2 | 1 | — |
| round 1 | 2 | 2 | true |
| round 2 | 3 | 6 | true |
| round 3 | 4 | 24 | true |
| exit | 5 | 24 | 5 ≤ 4 false |
Verify
Test Cases: Factorial
| Type | Goal | Input | Expected |
|---|---|---|---|
| Normal | Check typical use | n = 5 | 120 |
| Boundary | Try the lower bound | n = 0 | 1 (the loop never runs) |
| Invalid | Check error handling | n = -3 | "error" |
| Large | Detect overflow | n = 21 | 21! = 51,090,942,171,709,440,000 → does not fit in a 64-bit integer |
long) holds at most 9,223,372,036,854,775,807; 20! = 2,432,902,008,176,640,000 fits, 21! does not. Hence the constraint 0 ≤ n ≤ 20.Analysis
Same Problem, Two Algorithms: 1 + 2 + … + n
sum = 0
i = 1
while i <= n
sum = sum + i
i = i + 1A: a loop, n rounds
sum = n * (n + 1) / 2
B: Gauss's formula, one step
Analysis
Correctness and Efficiency
- Correctness means the algorithm produces the expected result.
- Time complexity describes how the operation count grows with input size, written in big-O notation: O(n), O(log n), O(n²).
- Space complexity measures the extra memory used.
- On small inputs, the fastest-looking method is not always the clearest one.
| n | ⌊log₂ n⌋ + 1 (binary search) | n (linear search) | n² (simple sort) |
|---|---|---|---|
| 10 | 4 | 10 | 100 |
| 100 | 7 | 100 | 10,000 |
| 1,000 | 10 | 1,000 | 1,000,000 |
| 1,000,000 | 20 | 1,000,000 | 1,000,000,000,000 |
Searching
Linear Search
int linearSearch(int[] arr, int target) { for (int i = 0; i < arr.length; i++) { if (arr[i] == target) return i; } return -1; }
- Compares elements from first to last; the data need not be sorted.
- Stops at the first match; returns -1 if there is none.
- Worst case examines all n elements: O(n).
The code is in Java.
Searching · Run the code step by step
Linear Search Step by Step (target = 5)
Call linearSearch(arr, 5). The array is not sorted, so we check one element at a time from the start.
i = 0: 0 < 5 → true.
arr[0] = 7 == 5 → false.
i = 1: 1 < 5 → true.
arr[1] = 3 == 5 → false.
i = 2: 2 < 5 → true.
arr[2] = 9 == 5 → false.
i = 3: 3 < 5 → true.
arr[3] = 5 == 5 → true → return 3. That took 4 comparisons; for target 4 it would return -1 after 5.
int linearSearch(int[] arr, int target) { for (int i = 0; i < arr.length; i++) { if (arr[i] == target) return i; } return -1; }
Searching
Binary Search
int binarySearch(int[] arr, int target) { int lo = 0, hi = arr.length - 1; while (lo <= hi) { int mid = lo + (hi - lo) / 2; if (arr[mid] == target) return mid; if (arr[mid] < target) lo = mid + 1; else hi = mid - 1; } return -1; }
- Works only on sorted data; each step halves the range.
- Worst case ⌊log₂ n⌋ + 1 rounds: O(log n).
lo + (hi - lo) / 2gives the same result as(lo + hi) / 2but does not overflow for very large indices.- Sorting first just to search once can raise the total cost.
Searching · Run the code step by step
Binary Search Step by Step (target = 34)
The search range is the whole array: lo = 0, hi = 8.
0 ≤ 8 → true: the range is not empty, round 1.
mid = 0 + (8 − 0) / 2 = 4, arr[4] = 21.
21 == 34 → false.
21 < 34 → the target is in the right half: lo = 5. The left half is gone.
5 ≤ 8 → true: the range is not empty, round 2.
mid = 5 + (8 − 5) / 2 = 6, arr[6] = 30.
30 == 34 → false.
30 < 34 → the target is in the right half: lo = 7. The left half is gone.
7 ≤ 8 → true: the range is not empty, round 3.
mid = 7 + (8 − 7) / 2 = 7, arr[7] = 34.
34 == 34 → return 7. Found in 3 rounds; linear search would have taken 8 comparisons.
int binarySearch(int[] arr, int target) { int lo = 0, hi = arr.length - 1; while (lo <= hi) { int mid = lo + (hi - lo) / 2; if (arr[mid] == target) return mid; if (arr[mid] < target) lo = mid + 1; else hi = mid - 1; } return -1; }
Searching · Run the code step by step
Binary Search: Target Not Found (target = 19)
The search range is the whole array: lo = 0, hi = 7. The target 19 is not in the array (it would sit between 17 and 21).
0 ≤ 7 → true: the range is not empty, round 1.
mid = 0 + (7 − 0) / 2 = 3, arr[3] = 17.
17 == 19 → false.
17 < 19 → true: the target can only be in the right half, lo = 4. The left half is gone.
4 ≤ 7 → true: the range is not empty, round 2.
mid = 4 + (7 − 4) / 2 = 5, arr[5] = 25.
25 == 19 → false.
25 < 19 → false, else: the target can only be in the left half, hi = 4. The right half is gone.
4 ≤ 4 → true: the range is not empty, round 3.
mid = 4 + (4 − 4) / 2 = 4, arr[4] = 21.
21 == 19 → false.
21 < 19 → false, else: the target can only be in the left half, hi = 3. The right half is gone.
4 ≤ 3 → false: lo has passed hi; the range is empty, so the loop ends.
return -1: target not found. The range emptied in 3 rounds, without checking all 8 elements.
int binarySearch(int[] arr, int target) { int lo = 0, hi = arr.length - 1; while (lo <= hi) { int mid = lo + (hi - lo) / 2; if (arr[mid] == target) return mid; if (arr[mid] < target) lo = mid + 1; else hi = mid - 1; } return -1; }
Searching
Comparing the Searches
| Property | Linear | Binary |
|---|---|---|
| Sorted data | Not needed | Required |
| Worst-case time | O(n) | O(log n) |
| Structure | Array or stream | Sorted, random-access structure |
| Use | Small or unsorted data | Many searches |
Searching
Searching: How Many Comparisons in the Worst Case?
The values were counted by actually running both algorithms with a missing target. For n = 1,000,000, linear search needs 1,000,000 comparisons; binary search only 20 rounds.
Sorting
Selection Sort
- Find the smallest element in the unsorted part.
- Swap it into the first position of that part.
- Always n(n−1)/2 comparisons → O(n²); at most n − 1 swaps; O(1) extra memory.
for i = 0 to n-2 minIdx = i for j = i+1 to n-1 if arr[j] < arr[minIdx] then minIdx = j swap arr[i] and arr[minIdx]
Sorting · Step-by-step trace
Selection Sort Step by Step
i = 0: the unsorted part is 0…4. Candidate minimum: arr[0] = 29.
arr[1] = 10 < arr[0] = 29 → true → minIdx = 1.
arr[2] = 14 < arr[1] = 10 → false.
arr[3] = 37 < arr[1] = 10 → false.
arr[4] = 13 < arr[1] = 10 → false.
Swap arr[0] and arr[1] → 10 is in place.
i = 1: the unsorted part is 1…4. Candidate minimum: arr[1] = 29.
arr[2] = 14 < arr[1] = 29 → true → minIdx = 2.
arr[3] = 37 < arr[2] = 14 → false.
arr[4] = 13 < arr[2] = 14 → true → minIdx = 4.
Swap arr[1] and arr[4] → 13 is in place.
i = 2: the unsorted part is 2…4. Candidate minimum: arr[2] = 14.
arr[3] = 37 < arr[2] = 14 → false.
arr[4] = 29 < arr[2] = 14 → false.
The minimum is already in place (minIdx = i), so the swap has no effect.
i = 3: the unsorted part is 3…4. Candidate minimum: arr[3] = 37.
arr[4] = 29 < arr[3] = 37 → true → minIdx = 4.
Swap arr[3] and arr[4] → 29 is in place. Sorted, with 10 comparisons = n(n−1)/2.
for i = 0 to n-2 minIdx = i for j = i+1 to n-1 if arr[j] < arr[minIdx] then minIdx = j swap arr[i] and arr[minIdx]
Sorting
Bubble Sort
- Compares adjacent elements and swaps them if they are out of order.
- Each pass carries the largest remaining element to the end; the next pass is one element shorter.
- If a pass makes no swaps, the array is sorted and we stop.
- O(n²) in general; best case (already sorted) is a single pass: O(n).
round = 0 repeat swapped = false for j = 0 to n-2-round if arr[j] > arr[j+1] then swap arr[j] and arr[j+1] swapped = true round = round + 1 until swapped is false
Sorting · Step-by-step trace
Bubble Sort Step by Step (pass 1)
round = 0. Adjacent pairs are compared from left to right.
Pass 1: swapped = false; j = 0…3.
arr[0] = 5 > arr[1] = 1 → true → swap.
Swapped: 5 moved one step right; swapped = true.
arr[1] = 5 > arr[2] = 4 → true → swap.
Swapped: 5 moved one step right; swapped = true.
arr[2] = 5 > arr[3] = 2 → true → swap.
Swapped: 5 moved one step right; swapped = true.
arr[3] = 5 > arr[4] = 8 → false, already in order.
Pass done: 8 is in its final place (green border). round = 1.
swapped = true → another pass is needed.
round = 0 repeat swapped = false for j = 0 to n-2-round if arr[j] > arr[j+1] then swap arr[j] and arr[j+1] swapped = true round = round + 1 until swapped is false
Sorting · Step-by-step trace
Bubble Sort Step by Step (cont.: pass 2)
Pass 2: swapped = false; j = 0…2.
arr[0] = 1 > arr[1] = 4 → false, already in order.
arr[1] = 4 > arr[2] = 2 → true → swap.
Swapped: 4 moved one step right; swapped = true.
arr[2] = 4 > arr[3] = 5 → false, already in order.
Pass done: 5 is in its final place (green border). round = 2.
swapped = true → another pass is needed.
round = 0 repeat swapped = false for j = 0 to n-2-round if arr[j] > arr[j+1] then swap arr[j] and arr[j+1] swapped = true round = round + 1 until swapped is false
Sorting · Step-by-step trace
Bubble Sort Step by Step (cont.: pass 3)
Pass 3: swapped = false; j = 0…1.
arr[0] = 1 > arr[1] = 2 → false, already in order.
arr[1] = 2 > arr[2] = 4 → false, already in order.
Pass done: 4 is in its final place (green border). round = 3.
swapped = false → no swaps at all, so the array is sorted; stop. 9 comparisons in total.
round = 0 repeat swapped = false for j = 0 to n-2-round if arr[j] > arr[j+1] then swap arr[j] and arr[j+1] swapped = true round = round + 1 until swapped is false
Sorting
Insertion Sort
- Like arranging playing cards in your hand: the left part is always sorted.
- Take the next element (the key), shift larger ones one step right, and drop the key into the gap.
- Worst case (reverse order) O(n²); on a sorted array each key needs one comparison: O(n). O(1) extra memory.
- Very fast on small or nearly sorted data.
for i = 1 to n-1 key = arr[i] j = i - 1 while j >= 0 and arr[j] > key arr[j+1] = arr[j] j = j - 1 arr[j+1] = key
Sorting · Step-by-step trace
Insertion Sort Step by Step
i = 1: take key = arr[1] = 3, leaving a gap. The left part (0…0) is sorted. j = 0.
j = 0 ≥ 0 and arr[0] = 4 > 3 → true.
4 moved one step right; the gap is now cell 0. j = -1.
j = -1 < 0 → false (we hit the left end), so the loop ends.
arr[0] = 3: the key drops into the gap. Now 0…1 is sorted.
i = 2: take key = arr[2] = 5, leaving a gap. The left part (0…1) is sorted. j = 1.
j = 1 ≥ 0 and arr[1] = 4 > 5 → false, so the loop ends.
arr[2] = 5: the key drops into the gap. Now 0…2 is sorted.
i = 3: take key = arr[3] = 1, leaving a gap. The left part (0…2) is sorted. j = 2.
j = 2 ≥ 0 and arr[2] = 5 > 1 → true.
5 moved one step right; the gap is now cell 2. j = 1.
j = 1 ≥ 0 and arr[1] = 4 > 1 → true.
4 moved one step right; the gap is now cell 1. j = 0.
j = 0 ≥ 0 and arr[0] = 3 > 1 → true.
3 moved one step right; the gap is now cell 0. j = -1.
j = -1 < 0 → false (we hit the left end), so the loop ends.
arr[0] = 1: the key drops into the gap. Now 0…3 is sorted. Done, with 5 comparisons.
for i = 1 to n-1 key = arr[i] j = i - 1 while j >= 0 and arr[j] > key arr[j+1] = arr[j] j = j - 1 arr[j+1] = key
Sorting
Comparing the Sorts
| Algorithm | Key idea | Worst | Best | Extra space |
|---|---|---|---|---|
| Selection | Select the smallest and place it | O(n²) | O(n²) | O(1) |
| Bubble | Swap neighbors | O(n²) | O(n) (with early exit) | O(1) |
| Insertion | Insert into the sorted part | O(n²) | O(n) | O(1) |
| Merge | Split, sort, merge | O(n log n) | O(n log n) | O(n) |
Sorting
Sorting: How Many Comparisons?
For each n, 20 random arrays were actually sorted by all four algorithms and the averages plotted. At n = 60, selection sort made 1770 comparisons; merge sort only 281.
Summary
Chapter Summary
- An algorithm is a clear, finite set of steps that turns input into output.
- Pseudocode writes an algorithm without committing to a language.
- Hand tracing and a variety of test cases help assess correctness.
- Linear search needs no sorting; binary search halves the range on sorted data.
- Besides correctness, algorithms are compared by time and memory cost.
Summary
Review Questions
- What belongs in a good problem definition?
- Why write pseudocode before choosing a programming language?
- How does the factorial algorithm handle an input of 0?
- Why does binary search need sorted data?
- What does comparing O(n) with O(log n) tell us?
- What is the key difference between selection sort and bubble sort?
Summary
Exercises
- Write pseudocode that prints three numbers (a, b, c) in ascending order; build a trace table for a = 9, b = 2, c = 5.
- Write an algorithm that sums the even numbers from 1 to n. Trace it for n = 0 and n = 7.
- Change the factorial algorithm to warn when n = 21 is entered (constraint 0 ≤ n ≤ 20).
- Sort [6, 2, 8, 3] by hand with selection, bubble and insertion sort; count the comparisons for each.
- Use binary search to look for 27 and 4 in [2, 5, 9, 14, 20, 27, 33]; write down lo, hi and mid in every round.
- Write an algorithm that sums the digits of a number (e.g. 472 → 13) using % and /.
Hints
- Apply the sort-two-numbers step three times (a-b, b-c, a-b).
- Start at i = 2 and step by i = i + 2; n = 7 → 12.
- Binary search: 27 → 2 rounds; 4 → 3 rounds, then -1.
Summary
References
- Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., MIT Press, 2022.
- Levitin, Introduction to the Design and Analysis of Algorithms, 3rd ed., Pearson, 2012.
- Sedgewick and Wayne, Algorithms, 4th ed., Addison-Wesley, 2011 (book site).
- Sedgewick and Wayne, Computer Science: An Interdisciplinary Approach, Addison-Wesley, 2016: 4.1 Performance, 4.2 Sorting and Searching.
- Oracle, Java SE 21 API: java.util.Arrays (sort, binarySearch).