Chapter 1: Introduction and Algorithm Analysis
Introduction
Learning Outcomes
Describe the five phases of the system life cycle and where data structure decisions are made.
Define algorithms and abstract data types (ADTs); build an interface in C with a header file.
Work out the space requirement and step count of a C function (sum, rsum, add).
Express step counts with O, Ω and Θ; separate the best, average and worst cases.
Explain how the common complexity classes pull apart as n grows.
Time a program with clock() and recognize the pitfalls (clock ticks, output).
Course Introduction · not in the books!
Textbooks and Resources
Kernighan and Ritchie, The C Programming Language. Written by C’s designers; keep it within reach at all times.
Horowitz, Sahni and Anderson-Freed, Fundamentals of Data Structures in C.
Three editions of one book: almost the same content, only the language of the code changes.
Further reading: Cormen et al. (CLRS), Introduction to Algorithms; Sedgewick, Algorithms in C.
Course Introduction · Reading
Reading: Loving Your Work
Lives today but thinks 5500 years back: “The pharaoh’s wife can’t have been buried there for that reason.”
Photographs for years to prove a black hole exists; thrilled when a star moves one pixel.
8 hours on the road, a 150 TL ticket, stones and sticks… Yet we call someone who travels as far for a conference a “fool”.
“The way to a lifetime of happiness is to love your work.” — Chinese proverb
Foundations
The System Life Cycle
Software goes through a process with the following phases:
Requirements: A project starts by specifying its purpose: which inputs are given, which outputs must be produced?
Analysis: The problem is broken into pieces. There are two approaches: bottom-up or top-down.
Design: Define the data objects (→ ADTs) and operations (→ algorithms); both are language-independent. More time spent on design means fewer errors.
Refinement and coding: Algorithms are coded on the chosen data structures. The data representation determines how efficient the algorithm is.
Verification: Correctness proofs, testing on varied data, and debugging. Each is a research area in its own right.
Horowitz–Sahni, Chapter 1
Foundations
Analysis: Bottom-Up or Top-Down?
Bottom-up
- Split the problem into pieces and focus on each one separately.
- Modules are visible right away; the problem is solved module by module.
- Preferred when we are experienced in the domain.
Top-down
- Start with a master plan: “What is the goal?”, “What is the final product?”
- Divide the problem into subproblems; this takes more time.
- Less backtracking and fewer errors; preferred when we lack experience.
Foundations
Coding and Verification
Mathematical techniques prove a program correct; proving a large project end to end is nearly impossible. Using proven algorithms reduces errors.
Needs working code and test data covering all scenarios. A program that runs without syntax errors is not necessarily correct.
When proofs or tests reveal errors, debugging retraces design and coding decisions.
A correct but slow program is of little value: correctness and efficiency are separate questions.
Foundations
What Is an Algorithm?
An algorithm is a finite sequence of instructions that accomplishes a particular task. Every algorithm must satisfy these criteria:
| Criterion | Meaning |
|---|---|
| Input | Zero or more quantities are supplied externally. |
| Output | At least one quantity is produced. |
| Definiteness | Each instruction is clear and unambiguous: no “add 6 or 7 to x”. |
| Finiteness | In every case, it terminates after a finite number of steps. |
| Effectiveness | Each instruction is basic enough to be carried out with pencil and paper. |
Horowitz–Sahni, Chapter 1
Foundations
Abstract Data Types
Data type = objects + operations. An ADT is a data type whose specification of objects and operations is separated from their representation and implementation.
Operation names, argument types, result type and what each operation does. This is all the user knows.
How the data is laid out in memory (struct, array…) and the code for the operations. Hidden from the user.
ADT Fraction
objects: num/den, den ≠ 0
operations:
create(p, q) → Fraction /* constructor */
add(a, b) → Fraction /* transformer */
equal(a, b) → Boolean /* observer */
print(a) /* reporter */
destroy(a)Horowitz–Sahni, Section 1.3
Foundations
Abstraction in C: Header File + struct
/* fraction.h : interface (WHAT?) */ #ifndef FRACTION_H #define FRACTION_H typedef struct fraction Fraction; /* opaque */ Fraction *fraction_create(int num, int den); Fraction *fraction_add(const Fraction *a, const Fraction *b); int fraction_equal(const Fraction *a, const Fraction *b); void fraction_print(const Fraction *f); void fraction_free(Fraction *f); #endif
/* fraction.c : implementation (HOW?) */ #include <stdio.h> #include <stdlib.h> #include "fraction.h" struct fraction { int num, den; }; Fraction *fraction_create(int num, int den) { Fraction *f; if (den == 0) return NULL; f = malloc(sizeof *f); if (f == NULL) return NULL; f->num = num; f->den = den; return f; }
typedef struct fraction Fraction; declares an incomplete (opaque) type: a file that includes fraction.h cannot see the fields and only holds a Fraction *. The fields stay hidden in fraction.c → C’s counterpart of an ADT.Foundations
The Rest of the Implementation, and Its Use
Fraction *fraction_add(const Fraction *a, const Fraction *b) { return fraction_create( a->num * b->den + b->num * a->den, a->den * b->den); } int fraction_equal(const Fraction *a, const Fraction *b) { return a->num * b->den == b->num * a->den; } void fraction_print(const Fraction *f) { printf("%d/%d\n", f->num, f->den); } void fraction_free(Fraction *f) { free(f); }
/* main.c : knows only the interface */ #include "fraction.h" #include <stddef.h> /* NULL */ int main(void) { Fraction *a = fraction_create(1, 2); Fraction *b = fraction_create(1, 3); Fraction *s = NULL; if (a && b) s = fraction_add(a, b); if (s) fraction_print(s); /* 5/6 */ fraction_free(a); fraction_free(b); fraction_free(s); /* free(NULL) is safe */ return 0; }
fraction_create returns NULL if the denominator is 0 or memory runs out; the caller checks.
Every result of create/add belongs to the caller and is released with fraction_free.
Selection Sort
Selection Sort
We want to sort an array into ascending order. Each pass finds the smallest of the remaining elements and moves it to the front (position i):
for (i = 0; i < n; i++) {
examine list[i] to list[n-1];
let list[min] be the smallest integer;
interchange list[i] and list[min];
}Selection Sort
Selection Sort in C
#include <stdio.h> #include <stdlib.h> #define MAX_SIZE 101 #define SWAP(x, y, t) ((t) = (x), (x) = (y), (y) = (t)) void sort(int list[], int n); int main(void) { int i, n, list[MAX_SIZE]; printf("Enter the array size: "); if (scanf("%d", &n) != 1 || n < 1 || n > MAX_SIZE) { printf("Invalid value of n!\n"); return 1; } for (i = 0; i < n; i++) list[i] = rand() % 100; sort(list, n); for (i = 0; i < n; i++) printf("%d ", list[i]); return 0; }
void sort(int list[], int n) { int i, j, min, temp; for (i = 0; i < n - 1; i++) { min = i; for (j = i + 1; j < n; j++) if (list[j] < list[min]) min = j; SWAP(list[i], list[min], temp); } }
Selection Sort · A common mistake
Where Does a Common Faulty sort() Go Wrong?
void sort(int list[], int n) { int i, j, min, temp; for (i = 0; i < n; i++) { min = i; for (j = 0; j < n; j++) if (list[j] > list[min]) { min = j; SWAP(list[i], list[min], temp); } } }
| Input | Version | Comp. | Swaps* |
|---|---|---|---|
| {29,10,14,37,13} | faulty | 25 | 7 |
| selection | 10 | 3 | |
| 100 random | faulty | 10000 | 2561 |
| selection | 4950 | 94 | |
| 500…1 (descending) | faulty | 250000 | 124750 |
| selection | 124750 | 250 |
j = 0 and the swap sits inside the loop. The correct version (j = i + 1, swap after the inner loop) is the one used in this chapter. *Swaps: SWAPs that exchange two distinct cells (i ≠ min).Selection Sort
SWAP: Macro or Function?
Macro
#define SWAP(x, y, t) \ ((t) = (x), (x) = (y), (y) = (t)) int temp; SWAP(list[i], list[min], temp);
- No call overhead: the preprocessor substitutes the text.
- The temporary
tis supplied by the caller.
Function
void swap(int *x, int *y) { int temp = *x; *x = *y; *y = temp; } swap(&list[i], &list[min]);
- Type-checked;
tempis local. - We must pass the addresses of the variables.
SWAP(a[i++], a[j], t) increments i twice! A function named SWAP would clash with the macro, so the function is called swap.Selection Sort · Run the code step by step
Code Trace: swap(&a, &b)
In main’s frame, a = 3 and b = 8.
The call copies addresses, not values. x → a, y → b.
temp = *x: read the value x points to (3).
*x = *y: the 8 that y points to is written where x points, i.e. into a.
*y = temp: 3 goes into b. The swap happened in main’s variables.
swap’s frame is gone (x, y, temp), but its effect remains.
void swap(int *x, int *y) { int temp = *x; *x = *y; *y = temp; } int main(void) { int a = 3, b = 8; swap(&a, &b); printf("%d %d\n", a, b); }
Performance Analysis
How Do We Judge a Program?
- Does it meet the specification?
- Does it work correctly?
- Is it documented (with comments) so others can use it?
- Does it use functions effectively to produce logical units?
- Is the code readable?
- Does it use primary and secondary memory efficiently?
- Is its running time acceptable?
Machine-independent: estimates of time and space (complexity).
Machine-dependent: the actual running time is measured.
Performance Analysis
Space Complexity
The amount of memory a program needs to run to completion: S(P) = c + SP(I)
Independent of input/output size: the code itself, simple variables, constants.
Depends on characteristics of input I (usually its size). This is what we analyze.
float abc(float a, float b, float c) { return a + b + b * c + (a + b - c) / (a + b) + 4.00; } /* fixed space only: S_abc(I) = 0 */
Performance Analysis
sum and rsum: Variable Space
float sum(float list[], int n) { float tempsum = 0; int i; for (i = 0; i < n; i++) tempsum += list[i]; return tempsum; } float rsum(float list[], int n) { if (n) return rsum(list, n - 1) + list[n - 1]; return 0; }
sumreceives the address of the first element → Ssum(n) = 0. If the elements were copied by value, Ssum(n) would be n.rsum(list, n)is called n+1 times in all; each call pushes a stack frame → Srsum(n) ∝ n.
Performance Analysis
Time Complexity
T(P) = compile time + run time. A program is compiled once; we care about the run time tP.
The time of each operation depends on the compiler and the machine; hard to know in advance.
Instead we count program steps: a meaningful, machine-independent unit.
Performance Analysis
Step Count Table: sum()
| Statement | s/e | Frequency | Total |
|---|---|---|---|
float sum(float list[], int n) { | 0 | 0 | 0 |
float tempsum = 0; | 1 | 1 | 1 |
int i; | 0 | 0 | 0 |
for (i = 0; i < n; i++) | 1 | n + 1 | n + 1 |
tempsum += list[i]; | 1 | n | n |
return tempsum; | 1 | 1 | 1 |
| Total | 2n + 3 |
for test runs n+1 times: n times “continue”, once “done”. Declarations count as zero steps.Performance Analysis · Run the code step by step
Code Trace: sum(list, 3) with count
Global count = 0. It goes up by one at every program step.
The assignment is one step → count = 1.
Loop test 0 < 3 is true → one step (count = 2).
tempsum += list[0] → 3; count = 3.
Loop test 1 < 3 is true → one step (count = 4).
tempsum += list[1] → 4; count = 5.
Loop test 2 < 3 is true → one step (count = 6).
tempsum += list[2] → 8; count = 7.
The final test 3 < 3 is false; it is a step too → count = 8.
return: count = 9 = 2·3 + 3, matching 2n + 3 from the table.
float sum(float list[], int n) { float tempsum = 0; count++; int i; for (i = 0; i < n; i++) { count++; /* loop test */ tempsum += list[i]; count++; } count++; /* last loop test */ count++; return tempsum; }
Performance Analysis
Step Count Table: rsum()
| Statement | s/e | Frequency | Total |
|---|---|---|---|
float rsum(float list[], int n) { | 0 | 0 | 0 |
if (n) | 1 | n + 1 | n + 1 |
return rsum(list, n-1) + list[n-1]; | 1 | n | n |
return 0; | 1 | 1 | 1 |
| Total | 2n + 2 |
rsum (2n+2) seems to take “fewer steps” than sum (2n+3), yet each recursive call (frame setup, return) costs more than one loop iteration.
What matters is how the step count grows as n grows: both are linear.
Performance Analysis · Run the code step by step
Code Trace: rsum(list, 3) and the call stack
rsum(list, 3): if test (count = 1); n = 3 ≠ 0.
count = 2; rsum(list, 2) must be computed first → new frame.
rsum(list, 2): if test (count = 3); n = 2 ≠ 0.
count = 4; rsum(list, 1) must be computed first → new frame.
rsum(list, 1): if test (count = 5); n = 1 ≠ 0.
count = 6; rsum(list, 0) must be computed first → new frame.
rsum(list, 0): if test (count = 7); n = 0 → base case.
count = 8; return 0. The stack is at its deepest: 4 frames.
Add list[0] = 3 to the returned value → rsum(list, 1) = 3.
Add list[1] = 1 to the returned value → rsum(list, 2) = 4.
Add list[2] = 4 to the returned value → rsum(list, 3) = 8. count = 8 = 2·3 + 2.
float rsum(float list[], int n) { count++; /* if test */ if (n) { count++; /* add + return */ return rsum(list, n - 1) + list[n - 1]; } count++; return 0; }
Performance Analysis
Step Count: Matrix Addition add()
| Statement | Frequency |
|---|---|
void add(int a[][MAX_SIZE], int b[][MAX_SIZE], int c[][MAX_SIZE], int rows, int cols) { | 0 |
int i, j; | 0 |
for (i = 0; i < rows; i++) | rows + 1 |
for (j = 0; j < cols; j++) | rows · (cols + 1) |
c[i][j] = a[i][j] + b[i][j]; | rows · cols |
| Total | 2·rows·cols + 2·rows + 1 |
Writing “O(rows, cols)” is wrong; the product is correct: O(rows·cols).
Performance Analysis
Best, Average and Worst Case
For the same n, the step count may also depend on the input itself. In linear search:
| Case | When? | Comparisons |
|---|---|---|
| Best | Key is in the first cell | 1 → Θ(1) |
| Average | Key present, every position equally likely | (n + 1) / 2 → Θ(n) |
| Worst | Key in the last cell or absent | n → Θ(n) |
CLRS Section 2.2
Asymptotic Notation
Big-O Notation
Definition: f(n) = O(g(n)) if there exist constants c > 0 and n0 such that f(n) ≤ c·g(n) for all n ≥ n0: “the algorithm has complexity of order g(n)”.
- Tsum(n) = 2n + 3 = O(n)
- Trsum(n) = 2n + 2 = O(n)
- Tadd = 2·rows·cols + 2·rows + 1 = O(rows·cols) → O(n²)
O is an upper bound; here “=” means “is a member of” and reads one way only.
Asymptotic Notation
Example: f(n) = 3n³ + 6n² + 7n = O(n³)
Bound the lower-order term 7n by n²: 7n ≤ n² whenever n ≥ 7.
6n² + n² = 7n².
Same trick once more: 7n² ≤ n³ for n ≥ 7.
We found c = 4 and n₀ = 7: the definition holds.
The running time never exceeds 4n³ steps (n ≥ 7).
If sorting 100 numbers takes 2 s, sorting 300 takes at most 2·3³ = 54 s (not 6 s!).
Asymptotic Notation
Lower Bound Ω and Tight Bound Θ
| Notation | Definition (for all n ≥ n₀) | Meaning |
|---|---|---|
| f(n) = O(g(n)) | f(n) ≤ c·g(n) | g is an upper bound on f |
| f(n) = Ω(g(n)) | f(n) ≥ c·g(n) | g is a lower bound on f |
| f(n) = Θ(g(n)) | c₁·g(n) ≤ f(n) ≤ c₂·g(n) | both O and Ω: a tight bound |
- 3n + 2 = Θ(n): 3n ≤ 3n + 2 ≤ 4n for n ≥ 2.
- 3n + 2 = O(n²) is also true but says little; we write the smallest g(n) we can.
- Selection sort makes n(n−1)/2 comparisons on every input → Θ(n²).
Asymptotic Notation
Practical Complexities
| Complexity | Name | n=1 | n=2 | n=4 | n=8 | n=16 | n=32 |
|---|---|---|---|---|---|---|---|
| 1 | Constant | 1 | 1 | 1 | 1 | 1 | 1 |
| lg n | Logarithmic | 0 | 1 | 2 | 3 | 4 | 5 |
| n | Linear | 1 | 2 | 4 | 8 | 16 | 32 |
| n·lg n | Linearithmic | 0 | 2 | 8 | 24 | 64 | 160 |
| n² | Quadratic | 1 | 4 | 16 | 64 | 256 | 1,024 |
| n³ | Cubic | 1 | 8 | 64 | 512 | 4,096 | 32,768 |
| 2ⁿ | Exponential | 2 | 4 | 16 | 256 | 65,536 | 4.29·109 |
| n! | Factorial | 1 | 2 | 24 | 40,320 | 2.09·1013 | 2.63·1035 |
Asymptotic Notation
As n Grows: Growth Chart
Vertical scale 0–150: lg n, n and n·lg n separate; n² climbs fast, while n³ and 2ⁿ are already off the chart.
Scale 0–4500: n³ = 1728 (n = 12), 2ⁿ = 4096 (n = 12); n! leaves the chart at n = 7 with 5040. The linear functions seem glued to the axis.
Asymptotic Notation
Running Times on a Machine Executing 10⁹ Instructions per Second
| n | n | lg n | n·lg n | n² | n³ | n⁴ | n¹⁰ | 2ⁿ |
|---|---|---|---|---|---|---|---|---|
| 10 | 0.01 μs | 0.00332 μs | 0.0332 μs | 0.1 μs | 1 μs | 10.0 μs | 10.0 s | 1.02 μs |
| 20 | 0.02 μs | 0.00432 μs | 0.0864 μs | 0.4 μs | 8 μs | 160 μs | 2.84 h | 1.05 ms |
| 30 | 0.03 μs | 0.00491 μs | 0.147 μs | 0.9 μs | 27.0 μs | 810 μs | 6.83 days | 1.07 s |
| 40 | 0.04 μs | 0.00532 μs | 0.213 μs | 1.6 μs | 64.0 μs | 2.56 ms | 4.05 mo | 18.3 min |
| 50 | 0.05 μs | 0.00564 μs | 0.282 μs | 2.5 μs | 125 μs | 6.25 ms | 3.1 yr | 13.0 days |
| 100 | 0.1 μs | 0.00664 μs | 0.664 μs | 10.0 μs | 1 ms | 100 ms | 31.7 cent. | 4.02·10¹¹ cent. |
| 1000 | 1 μs | 0.00997 μs | 9.97 μs | 1 ms | 1 s | 16.7 min | 3.17·10¹¹ cent. | 3.40·10²⁸² cent. |
| 10000 | 10.0 μs | 0.0133 μs | 133 μs | 100 ms | 16.7 min | 3.86 mo | 3.17·10²¹ cent. | 6.33·10²⁹⁹¹ cent. |
Each step takes 1 ns; 1 year = 365 days, 1 month = 30 days; cent. = centuries. For n = 10000, 2ⁿ is about 102992 centuries.
Asymptotic Notation
A First Look at Amortized Analysis
Some operations are occasionally expensive but usually cheap. Amortized analysis divides the total cost of a sequence of n operations by n. Example: an array whose capacity doubles when full (realloc); we trace it and count its copies in Chapter 2.
CLRS 3rd ed., Chapter 17 (Amortized Analysis)
Magic Square
The Magic Square Problem
Every row, column and diagonal: 65
Place the numbers 1 to n² in an n×n square so that every row, every column and both main diagonals have the same sum: n(n²+1)/2 (65 for 5×5).
Coxeter’s rule (n odd; also known as the Siamese or de la Loubère method): put 1 in the middle of the first row. Always move diagonally up and to the left; if that square is taken, move down from the current square instead. Moves off the square wrap around to the opposite edge. It finishes in n² steps → O(n²).
Magic Square
“Computers Are Fast, It’ll Solve It Instantly!”?
A common objection: a new 10 GHz computer will solve this right away; no need to bother with algorithms.
4.9 million centuries → about 490 thousand centuries. Still hopeless.
Coxeter’s rule: 25 steps, under a microsecond. Algorithms come before hardware.
Calculation: 25! / (10¹⁰ · 3.15·10⁹) ≈ 4.9·10⁵ centuries.
Magic Square · Run the code step by step
Code Trace: magic(3, sq)
Middle of the first row: i = 0, j = 1 → write 1.
s = 2: the up-left square (2, 0) is free (row wrapped → bottom row) → move there.
s = 3: the up-left square (1, 2) is free (column wrapped → rightmost column) → move there.
s = 4: the up-left square (0, 1) is taken (it holds 1) → one down: (2, 2).
s = 5: the up-left square (1, 1) is free → move there.
s = 6: the up-left square (0, 0) is free → move there.
s = 7: the up-left square (2, 2) is taken (it holds 4) → one down: (1, 0).
s = 8: the up-left square (0, 2) is free (column wrapped → rightmost column) → move there.
s = 9: the up-left square (2, 1) is free (row wrapped → bottom row) → move there.
s = 10 > 9: the loop ends. Every row, column and diagonal sums to 15 = 3·(9+1)/2.
void magic(int n, int sq[][MAX]) { int i = 0, j = (n - 1) / 2, s, r, c; sq[i][j] = 1; /* sq must be all zeros */ for (s = 2; s <= n * n; s++) { r = (i - 1 < 0) ? n - 1 : i - 1; /* up */ c = (j - 1 < 0) ? n - 1 : j - 1; /* left */ if (sq[r][c]) i = (i + 1) % n; /* down */ else { i = r; j = c; } sq[i][j] = s; } }
Performance Measurement
Performance Measurement: Two Methods
| 1 · Processor time (clock) | 2 · Calendar time (time) | |
|---|---|---|
| Start | start = clock(); | start = time(NULL); |
| Stop | stop = clock(); | stop = time(NULL); |
| Type | clock_t | time_t |
| Result (s) | (double)(stop - start) / CLOCKS_PER_SEC | difftime(stop, start) |
Number of processor clock ticks used by the program. Resolution is limited by the system’s tick interval.
Wall-clock time with one-second resolution. Fine for long jobs, too coarse for short ones.
CLK_TCK is an old, nonstandard name; standard C uses CLOCKS_PER_SEC. I/O devices (screen, disk, printer) are slow: keep printf calls outside the timed region.Performance Measurement
Exercise 1: Timing Selection Sort
#include <stdio.h> #include <time.h> #define MAX_SIZE 5001 #define ITERATIONS 20 void sort(int list[], int n); /* selection sort */ int main(void) { int i, j, list[MAX_SIZE]; int sizelist[] = { 0, 100, 300, 600, 900, 1200, 1500, 2000, 3000, 3100, 3200, 3300, 3400, 3500, 3600, 3700, 3800, 3900, 4000, 5000 }; clock_t start, stop; double ms; for (i = 0; i < ITERATIONS; i++) { for (j = 0; j < sizelist[i]; j++) list[j] = sizelist[i] - j; /* descending */ start = clock(); sort(list, sizelist[i]); stop = clock(); ms = 1000.0 * (stop - start) / CLOCKS_PER_SEC; printf("%4d elements: %6.1f ms\n", sizelist[i], ms); } return 0; }
- 20 values of n; for each, an array in descending order.
- Only the
sortcall lies between the twoclock()calls. - Writing
(int)(1000*(stop-start)/CLK_TCK)drops the fraction through integer division; double arithmetic with1000.0is more accurate.
Performance Measurement
Results and Inconsistencies
Dots: sample measurements · dashed: c·n² curve
- Everything up to 600 elements takes 0 ms?
- 3000 elements 110 ms, but 3100 only 93 ms?
- 3500, 3600 and 3700 all 125 ms?
stop − start = 0. On a system with a ~15.6 ms tick, results are rounded to multiples of 15–16 ms.Performance Measurement
Exercise 2: Sequential Search (with a Sentinel)
int seqsearch(int list[], int searchnum, int n) { int i; list[n] = searchnum; /* sentinel */ for (i = 0; list[i] != searchnum; i++) ; return (i < n) ? i : -1; }
start = clock(); for (j = 0; j < runtimes[i]; j++) pos = seqsearch(list, -1, sizelist[i]); stop = clock(); total = 1000.0 * (stop - start) / CLOCKS_PER_SEC; avg = 1000.0 * total / runtimes[i]; /* in μs */ list[sizelist[i]] = sizelist[i]; /* undo sentinel */
- Sentinel: the key is stored in
list[n], so the loop always stops and needs noi < ntest on each pass. - The array needs room for n + 1 elements (MAX_SIZE = 5001).
- We search for −1, which is never found → the worst case is measured.
- For each n the search is repeated
runtimes[i]times (500000 … 5000).
Performance Measurement · Run the code step by step
Code Trace: seqsearch({4, 8, 2}, 5, 3)
The array holds 3 elements, but room for list[3] is reserved. We search for 5.
Sentinel: list[3] = 5. The loop now stops here at the latest.
list[0] = 4 ≠ 5 → i++. (One test: no bounds check.)
list[1] = 8 ≠ 5 → i++. (One test: no bounds check.)
list[2] = 2 ≠ 5 → i++. (One test: no bounds check.)
list[3] = 5 → the loop stops, but this is the sentinel.
i = 3 is not < n → return −1: not found. n + 1 = 4 comparisons.
int seqsearch(int list[], int searchnum, int n) { int i; list[n] = searchnum; /* sentinel */ for (i = 0; list[i] != searchnum; i++) ; return (i < n) ? i : -1; }
Performance Measurement
Repeated Measurement: Average Search Time
Dots: measured averages · dashed: linear trend
- The averages are no longer 0 and grow linearly with n, as O(n) predicts.
- Fluctuation beyond 3000: with fewer repetitions (runtimes) the measurement gets coarse again.
- The test machine ticks 1000 times per second: 1 tick = 1 ms.
Homework
Homework (Recursion)
- Write a recursive function that computes n!.
- The Fibonacci numbers are f₀ = 0, f₁ = 1, fᵢ = fᵢ₋₁ + fᵢ₋₂ (i > 1). Write a recursive function that computes fᵢ.
- Given C(n, m) = C(n−1, m) + C(n−1, m−1), write a recursive function that computes C(n, m) = n! / (m!·(n−m)!).
Hints
- Base: n ≤ 1 → 1. Step: n · fact(n−1).
- Base: n ≤ 1 → n. Step: fib(n−1) + fib(n−2).
- Base: m == 0 or m == n → 1.
Homework
Solutions and Common Mistakes
long fact(long n) { if (n <= 1) return 1; return n * fact(n - 1); } long fib(long n) { if (n <= 1) return n; /* f0 = 0, f1 = 1 */ return fib(n - 1) + fib(n - 2); } long binom(int n, int m) { if (m == 0 || m == n) return 1; return binom(n - 1, m) + binom(n - 1, m - 1); }
A common mistake is writing fib(n-1) * fib(n-2); the product is always 0 or 1. By definition it must be a sum.
A common mistake is writing long binom() with no parameters; n and m must be parameters.
fact is O(n); fib and binom as written are exponential: the same subproblems are solved again and again.
Summary
Complexity Summary
| Function | Time | Extra space | Note |
|---|---|---|---|
sort (selection sort) | Θ(n²) | O(1) | n(n−1)/2 comparisons, ≤ n−1 swaps |
sum | Θ(n) | O(1) | 2n + 3 steps |
rsum | Θ(n) | O(n) | 2n + 2 steps, n+1 frames |
add (matrix addition) | Θ(rows·cols) | O(1) | Θ(n²) for a square matrix |
seqsearch | O(n) | O(1) | best Θ(1), worst n + 1 |
magic (Coxeter) | Θ(n²) | O(1) | brute force: (n²)! |
| Dynamic array insertion | O(1) amortized | O(n) | a single insertion is O(n) worst case |
fib (recursive) | exponential | O(n) | stack depth n |
Summary
Common Mistakes
clock() pair: the result is 0 or a multiple of the tick.printf in the timed region: I/O time swamps the algorithm.SWAP(a[i++], …).Summary
Exercises
- How many times does
x++execute infor (i = 0; i < n; i++) for (j = 0; j < i; j++) x++;? O(?) - Show that 10n² + 100n + 1000 = O(n²) by giving c and n₀.
- How many comparisons does selection sort make on an already sorted array? Does it have a best case?
- An O(n log n) algorithm takes 10 ms for n = 1000. About how long for n = 10⁶?
- Why is calling
rsumwith n = 10⁶ dangerous? - Write the first row of a 7×7 magic square using Coxeter’s rule (what must the sum be?).
Show answers / hints
- n(n−1)/2 → Θ(n²).
- 10n² + 100n + 1000 ≤ 1110n² for n ≥ 1: c = 1110, n₀ = 1 (other choices work too).
- Still n(n−1)/2; the comparison count does not depend on the input (only the swap count does).
- The ratio (10⁶·log 10⁶)/(10³·log 10³) = 2000 → ≈ 20 s.
- 10⁶ frames may overflow the stack; the iterative sum is safe.
- Sum 7·50/2 = 175; first row: 28 19 10 1 48 39 30.
Summary
Chapter Summary
Requirements → analysis → design → coding → verification. Deferring the implementation leads to better choices.
ADT = specification ⟂ implementation. In C: a header file + an opaque struct.
Space: c + SP(I). Time: program steps (sum 2n+3, rsum 2n+2).
O upper, Ω lower, Θ tight bound. The worst case is a guarantee; amortized analysis looks at the whole sequence.
Small n misleads; 2ⁿ and n! quickly become hopeless. Algorithms come before hardware.
clock() + repetition + averaging; keep I/O out of the timing.
Summary
References
- Horowitz, Sahni, Anderson-Freed, Fundamentals of Data Structures in C, 2nd ed., Chapter 1.
- Cormen, Leiserson, Rivest, Stein, Introduction to Algorithms, 3rd ed.: Chapters 2–3 (analysis, asymptotics), Chapter 17 (amortized analysis).
- Kernighan, Ritchie, The C Programming Language, 2nd ed.