Chapter 8: Online Algorithms
Introduction
Learning Outcomes
Distinguish the online and offline problem models; explain what an irrevocable decision is.
Define the competitive ratio; prove the 2 − 1/B ratio for the ski rental problem.
Count page faults step by step under FIFO, LRU and OPT; interpret why LRU is k-competitive and MTF is 2-competitive.
Implement a moving average, a running median and reservoir sampling in Java; analyze time and memory per element.
Introduction
Deciding Without the Full Picture
- Sometimes decisions must be made without all the information.
- Online algorithms:
- when information arrives piece by piece,
- they try to make the best decision with what is known so far.
- A decision is irrevocable: information that arrives later does not change past decisions.

Introduction
Online Algorithms
- Like walking through a maze:
- You cannot see the whole maze, only the path ahead.
- At each step you pick the best path with limited information.
- They allow fast decisions when information does not arrive all at once.
- Examples:
- Which road to take in a traffic jam?
- What is the next move in a chess game?
- Adjusting traffic lights to current traffic.

Introduction
Online vs. Offline Problem Model
The input is a request sequence σ = σ1, σ2, …, σn. The difference lies in when the algorithm sees it.
| Offline | Online | |
|---|---|---|
| Input | Fully known in advance | Arrives piece by piece; when σt arrives, σt+1 is unknown |
| Decision | Made after seeing the whole input | Must be made when σt arrives; irrevocable |
| Example | Sorting, Dijkstra, knapsack (DP) | Cache management, ski rental, stock trading |
| Optimal solution | OPT can be computed | OPT is known only in hindsight |
| Measure | Time / space complexity | Competitive ratio: ALG / OPT |
Introduction
The Online Algorithm Model
- Data arrives continuously; the algorithm processes each piece as it arrives and outputs immediately.
- The memory (buffer) cannot hold the whole input: the past survives only as a summary (a sum, a heap, a sample…).
- So the goal is little work per element (usually O(1) or O(log n)) and little memory.
Introduction
Applications
Common ground: the environment is dynamic, decisions are real-time, and knowledge of the future is incomplete.
Introduction
Advantages and Disadvantages
Advantages
- Work in dynamic environments; no need to wait for the input to end.
- Make real-time decisions with low latency.
- Can process very large, even infinite, streams with little memory.
- The competitive ratio gives a worst-case guarantee.
Disadvantages
- Wrong decisions in early steps can affect later ones.
- In chess, one wrong move can lose the game.
- No guarantee of finding the best global solution.
- Sensitive to the early input; harder to design and analyze.
Introduction
Design Approaches for Online Algorithms
Competitive Analysis
Competitive Analysis
- On every input, the online algorithm ALG is compared with the best offline algorithm OPT, which knows everything in advance.
- ALG is c-competitive if for every request sequence σALG(σ) ≤ c · OPT(σ) + αholds (α: a constant independent of σ; if α = 0, it is strictly competitive).
- The smallest such c is the competitive ratio: c = supσ ALG(σ) / OPT(σ).
- The closer the ratio is to 1, the less the missing knowledge of the future hurts.
Competitive Analysis
The Adversary Model
To lower-bound the competitive ratio, imagine an adversary playing against the algorithm: it generates the input and runs OPT.
Ski Rental
The Ski Rental Problem
- Skis can be rented for 1 TL per day or bought for B TL.
- The number of ski days d is unknown: each morning you learn only whether you ski today.
- Each day's decision: rent again or buy?
- OPT knows d: OPT(d) = min(d, B).
B = 10; x-axis: actual days d. Worst ratio at d = B: 19/10 = 2 − 1/B.
Same shape: hourly cloud server vs. a yearly plan; keeping a TCP connection open vs. closing it.
Ski Rental
Cost of the Strategies (B = 10)
OPT, knowing the future: if d < 10 it always rents (cost d); if d ≥ 10 it buys on day 1 (cost 10).
Always rent: cost d. As d grows the cost increases without bound while OPT stays at 10 → not competitive.
Buy on day 1: the cost is always 10. But if d = 1, OPT = 1 → ratio 10 = B. Bad.
Break-even: rent for 9 days, buy on day 10. While d < 10 it matches OPT; for d ≥ 10 the cost is 9 + 10 = 19.
Worst case d = 10: ALG = 19, OPT = 10 → ratio 1.9 = 2 − 1/B. No d is worse.
Ski Rental · Run the code step by step
Break-Even Strategy: Code Trace
B = 4 TL (to buy), rent 1 TL per day. The season will actually last d = 6 days; the algorithm does not know this and each morning sees only "am I skiing today?".
Day 1: 1 < B → rent. cost = 1.
Day 2: 2 < B → rent. cost = 2.
Day 3: 3 < B → rent. cost = 3.
Day 4 = B: rent so far is B − 1 = 3 TL; now buy: cost = 3 + 4 = 7.
break: the skis are ours now; days 5 and 6 are free.
ALG = 7, OPT = min(6, 4) = 4 (buy on day 1). Ratio 7/4 = 1.75 = 2 − 1/B. Whatever d is, the ratio never exceeds this.
int breakEven(int B, int d) { // d unknown int cost = 0; for (int day = 1; day <= d; day++) { if (day < B) { cost += 1; // rent } else { cost += B; // buy break; // no more fees } } return cost; }
Ski Rental
Why Nothing Does Better: Lower Bound
Every deterministic strategy is described by a single number: k = the day it buys.
- Adversary: the weather turns right after the purchase, i.e., d = k.
- ALG = (k − 1) rent + B, OPT = min(k, B).
- k < B: ratio (k − 1 + B) / k ≥ (2B − 1) / B.
- k ≥ B: ratio (k − 1 + B) / B ≥ (2B − 1) / B.
- Best is k = B: ratio 2 − 1/B.
| k | rent | ALG | OPT | ratio |
|---|---|---|---|---|
| 1 | 0 | 10 | 1 | 10 |
| 2 | 1 | 11 | 2 | 5.5 |
| 5 | 4 | 14 | 5 | 2.8 |
| 9 | 8 | 18 | 9 | 2 |
| 10 | 9 | 19 | 10 | 1.9 |
| 11 | 10 | 20 | 10 | 2 |
| 15 | 14 | 24 | 10 | 2.4 |
| 20 | 19 | 29 | 10 | 2.9 |
B = 10; d = k (the adversary's choice). The table shows that the worst ratio is smallest at k = B.
Ski Rental
Try It: Ski Rental
Strategy: rent for k − 1 days; on day k, if still skiing, buy. The worst ratio arises when the adversary picks d = k.
Paging
The Paging Problem
- Fast memory (the cache) holds k pages; slow memory is unbounded.
- If the requested page is in the cache, it is a hit (cost 0); otherwise a page fault (cost 1): the page is loaded.
- If the cache is full, which page should be evicted? Future requests are unknown: a typical online problem.
- Goal: minimize the total number of page faults.
Paging
Page Eviction Policies
| Policy | Which page does it evict? | Online? | Competitive ratio |
|---|---|---|---|
| FIFO | The page loaded earliest (a queue) | yes | k |
| LRU | The least recently used page | yes | k |
| FWF | On a fault with a full cache, evicts all pages (flush when full) | yes | k |
| LFU | The least frequently used page | yes | unbounded |
| LIFO | The page loaded most recently | yes | unbounded |
| OPT (MIN) | The page whose next use is farthest in the future (Belady, 1966) | no | 1 |
Paging · Step by step
FIFO Step by Step
FIFO: the cache behaves like a queue; first in, first out. Request sequence 7 0 1 2 0 3 0 4 2 3 0 3 2, k = 3. ✗: fault, ✓: hit.
Request 1: page 7 missing → page fault; loaded into empty frame F1. Total faults: 1.
Request 2: page 0 missing → page fault; loaded into empty frame F2. Total faults: 2.
Request 3: page 1 missing → page fault; loaded into empty frame F3. Total faults: 3.
Request 4: page 2 missing → page fault; 7, loaded earliest, is evicted. Total faults: 4.
Request 5: page 0 is cached → hit. The FIFO order does not change. Total faults: 4.
Request 6: page 3 missing → page fault; 0, loaded earliest, is evicted. Total faults: 5.
Request 7: page 0 missing → page fault; 1, loaded earliest, is evicted. Total faults: 6.
Request 8: page 4 missing → page fault; 2, loaded earliest, is evicted. Total faults: 7.
Request 9: page 2 missing → page fault; 3, loaded earliest, is evicted. Total faults: 8.
Request 10: page 3 missing → page fault; 0, loaded earliest, is evicted. Total faults: 9.
Request 11: page 0 missing → page fault; 4, loaded earliest, is evicted. Total faults: 10.
Request 12: page 3 is cached → hit. The FIFO order does not change. Total faults: 10.
Request 13: page 2 is cached → hit. The FIFO order does not change. Total faults: 10.
Paging · Step by step
LRU Step by Step
LRU: track each page's last use time; evict the oldest. Request sequence 7 0 1 2 0 3 0 4 2 3 0 3 2, k = 3. ✗: fault, ✓: hit.
Request 1: page 7 missing → page fault; loaded into empty frame F1. Total faults: 1.
Request 2: page 0 missing → page fault; loaded into empty frame F2. Total faults: 2.
Request 3: page 1 missing → page fault; loaded into empty frame F3. Total faults: 3.
Request 4: page 2 missing → page fault; 7, least recently used, is evicted. Total faults: 4.
Request 5: page 0 is cached → hit. 0 is now the most recently used page. Total faults: 4.
Request 6: page 3 missing → page fault; 1, least recently used, is evicted. Total faults: 5.
Request 7: page 0 is cached → hit. 0 is now the most recently used page. Total faults: 5.
Request 8: page 4 missing → page fault; 2, least recently used, is evicted. Total faults: 6.
Request 9: page 2 missing → page fault; 3, least recently used, is evicted. Total faults: 7.
Request 10: page 3 missing → page fault; 0, least recently used, is evicted. Total faults: 8.
Request 11: page 0 missing → page fault; 4, least recently used, is evicted. Total faults: 9.
Request 12: page 3 is cached → hit. 3 is now the most recently used page. Total faults: 9.
Request 13: page 2 is cached → hit. 2 is now the most recently used page. Total faults: 9.
Paging · Step by step
OPT Step by Step
OPT: knows the future; evicts the page whose next use is farthest away. Request sequence 7 0 1 2 0 3 0 4 2 3 0 3 2, k = 3. ✗: fault, ✓: hit.
Request 1: page 7 missing → page fault; loaded into empty frame F1. Total faults: 1.
Request 2: page 0 missing → page fault; loaded into empty frame F2. Total faults: 2.
Request 3: page 1 missing → page fault; loaded into empty frame F3. Total faults: 3.
Request 4: page 2 missing → page fault; 7 is never requested again → evicted. Total faults: 4.
Request 5: page 0 is cached → hit. Total faults: 4.
Request 6: page 3 missing → page fault; 1 is never requested again → evicted. Total faults: 5.
Request 7: page 0 is cached → hit. Total faults: 5.
Request 8: page 4 missing → page fault; 0, whose next use is farthest (request 11), is evicted. Total faults: 6.
Request 9: page 2 is cached → hit. Total faults: 6.
Request 10: page 3 is cached → hit. Total faults: 6.
Request 11: page 0 missing → page fault; 4 is never requested again → evicted. Total faults: 7.
Request 12: page 3 is cached → hit. Total faults: 7.
Request 13: page 2 is cached → hit. Total faults: 7.
Paging
Comparing the Three Policies
| Policy | Page faults | Hits | ALG / OPT | Information needed |
|---|---|---|---|---|
| FIFO | 10 | 3 | 1.43 | Load order (queue) |
| LRU | 9 | 4 | 1.29 | Last use time |
| OPT | 7 | 6 | 1 | All future requests |
- The first k = 3 faults are unavoidable (compulsory or cold misses): every policy pays them.
- LRU does well on sequences with locality of reference: a page used recently is likely to be used again soon. Here page 0 is requested often, so FIFO evicts it needlessly.
- Real systems use cheap approximations of LRU: the clock algorithm (second chance).
Paging · Run the code step by step
LRU: Code Trace
The cache is empty, faults = 0. The head of the list is the least recently used page, the tail the most recently used.
Page 1 is not in the list: remove returns false → else branch.
faults = 1. The cache has room (size 0 < 3) → no eviction.
Page 1 is appended at the tail.
Page 2 is not in the list: remove returns false → else branch.
faults = 2. The cache has room (size 1 < 3) → no eviction.
Page 2 is appended at the tail.
Page 3 is not in the list: remove returns false → else branch.
faults = 3. The cache has room (size 2 < 3) → no eviction.
Page 3 is appended at the tail.
Page 1 is in the list: remove returns true and removes it → hit.
Page 1 is appended at the tail: now the most recently used.
Page 4 is not in the list: remove returns false → else branch.
faults = 4. The cache is full → the head (least recently used) 2 is evicted.
Page 4 is appended at the tail.
Page 2 is not in the list: remove returns false → else branch.
faults = 5. The cache is full → the head (least recently used) 3 is evicted.
Page 2 is appended at the tail.
Sequence done: 5 page faults and 1 hit in 6 requests. Since remove scans the list, each request costs O(k); with a HashMap + doubly linked list it becomes O(1).
int lruFaults(int[] requests, int k) { var cache = new LinkedList<Integer>(); int faults = 0; for (int page : requests) { if (cache.remove((Integer) page)) { cache.addLast(page); // make newest } else { faults++; if (cache.size() == k) cache.removeFirst(); // drop oldest cache.addLast(page); } } return faults; }
Paging
LRU Cache: O(1) Implementation
class LruCache<K, V> extends LinkedHashMap<K, V> { private final int k; LruCache(int k) { super(16, 0.75f, true); // access order this.k = k; } @Override protected boolean removeEldestEntry( Map.Entry<K, V> eldest) { return size() > k; // evict beyond k } }
LinkedHashMap= hash table + doubly linked list.true(accessOrder): everyget/putmoves the entry to the end of the list.removeEldestEntry: once the size exceeds k, the head (least recently used) is removed.- Find, move, remove: all O(1); memory O(k).
false (insertion order): accesses do not change the order.Paging
Theorem: Deterministic Lower Bound k
- k + 1 distinct pages suffice. At every step the adversary requests the one page not in ALG's cache → ALG faults on every request.
- When OPT (evict farthest) faults, the page it evicts is not requested for at least k − 1 requests → OPT faults at most once every k requests.
- Hence ALG / OPT → k (Sleator and Tarjan, 1985).
Paging
Theorem: LRU (and FIFO) Are k-Competitive
- Split the sequence into phases: each phase is a maximal block with k distinct pages; the (k + 1)-th distinct page starts a new phase.
- LRU faults at most k times per phase: a page requested in a phase is not evicted before the phase ends (some page used less recently always exists). Each distinct page faults at most once.
- OPT faults at least once per two consecutive phases: from the 2nd request of one phase to the 1st request of the next, k + 1 distinct pages are requested; they do not fit in k frames.
- For m phases, LRU ≤ k · m and OPT ≥ m − 1 → LRU ≤ k · OPT + k.
Paging
Belady's Anomaly: More Memory, More Faults
- Sequence: 1 2 3 4 1 2 5 1 2 3 4 5.
- FIFO, k = 3: 9 faults; k = 4: 10 faults. The cache grew, and so did the faults!
- LRU on the same sequence: k = 3 → 10, k = 4 → 8. LRU and OPT are stack algorithms: the pages in k frames are a subset of those in k + 1 frames → no anomaly.
Paging · Run the code step by step
FIFO on Belady's Sequence: Code Trace
Belady's sequence 1 2 3 4 1 2 5 1 2 3 4 5, k = 3. The queue keeps the load order; the set answers "is it cached?" quickly.
p = 1 missing → fault 1 (compulsory); there is room, so it joins the queue.
p = 2 missing → fault 2 (compulsory); there is room, so it joins the queue.
p = 3 missing → fault 3 (compulsory); there is room, so it joins the queue.
p = 4 missing → fault 4. The cache is full: 1, loaded earliest, leaves and 4 joins the back of the queue.
p = 1 missing → fault 5. The cache is full: 2, loaded earliest, leaves and 1 joins the back of the queue.
p = 2 missing → fault 6. The cache is full: 3, loaded earliest, leaves and 2 joins the back of the queue.
p = 5 missing → fault 7. The cache is full: 4, loaded earliest, leaves and 5 joins the back of the queue.
p = 1 is cached → hit. In FIFO a hit does not change the queue order.
p = 2 is cached → hit. In FIFO a hit does not change the queue order.
p = 3 missing → fault 8. The cache is full: 1, loaded earliest, leaves and 3 joins the back of the queue.
p = 4 missing → fault 9. The cache is full: 2, loaded earliest, leaves and 4 joins the back of the queue.
p = 5 is cached → hit. In FIFO a hit does not change the queue order.
return 9. The same code with k = 4 gives 10 faults: more frames, more faults (Belady's anomaly).
int fifoFaults(int[] requests, int k) { Queue<Integer> queue = new LinkedList<>(); Set<Integer> cached = new HashSet<>(); int faults = 0; for (int p : requests) { if (cached.contains(p)) continue; // hit faults++; if (cached.size() == k) cached.remove(queue.poll()); // oldest queue.offer(p); cached.add(p); } return faults; }
Paging
Randomized Paging: The Marking Algorithm
Marking
- At the start of a phase, all pages are unmarked.
- The requested page is marked (hit or fault).
- On a fault, evict a random page among the unmarked ones.
- If all pages are marked, a new phase begins (step 1).
Results
- Expected ratio 2Hk against an oblivious adversary (Fiat et al., 1991).
- Hk = 1 + 1/2 + … + 1/k ≈ ln k. For k = 8, H8 ≈ 2.72 → ratio ≈ 5.4 (8 for deterministic).
- No randomized algorithm can beat Hk.
- LRU is also a marking algorithm (it picks the oldest unmarked page).
Paging · Run the code step by step
The Marking Algorithm: Code Trace
Requests 7 0 1 2 0 3 0 4 2 3 0 3 2, k = 3. p = 7 missing → fault 1; added to the cache and marked (•).
p = 0 missing → fault 2 (compulsory); there is room, so it is added and marked.
p = 1 missing → fault 3 (compulsory); there is room, so it is added and marked.
All pages were marked → new phase: marks are cleared. p = 2 missing → fault 4. 7 is evicted at random from the unmarked candidates [0, 1, 7].
p = 0 is cached → hit; it is just marked (•).
p = 3 missing → fault 5. 1 is evicted at random from the unmarked candidates [1].
p = 0 is cached → hit; it is just marked (•).
All pages were marked → new phase: marks are cleared. p = 4 missing → fault 6. 3 is evicted at random from the unmarked candidates [0, 2, 3].
p = 2 is cached → hit; it is just marked (•).
p = 3 missing → fault 7. 0 is evicted at random from the unmarked candidates [0].
All pages were marked → new phase: marks are cleared. p = 0 missing → fault 8. 2 is evicted at random from the unmarked candidates [2, 3, 4].
p = 3 is cached → hit; it is just marked (•).
p = 2 missing → fault 9. 4 is evicted at random from the unmarked candidates [4].
return 9. With these draws marking made 9 faults (LRU 9, FIFO 10, OPT 7); another seed may give a different result.
int marking(int[] requests, int k) { Set<Integer> cache = new HashSet<>(); Set<Integer> marked = new HashSet<>(); int faults = 0; for (int p : requests) { if (!cache.contains(p)) { faults++; if (cache.size() == k) { if (marked.size() == k) marked.clear(); // new phase List<Integer> cand = new ArrayList<>(cache); cand.removeAll(marked); // unmarked ones int i = rng.nextInt(cand.size()); cache.remove(cand.get(i)); } cache.add(p); } marked.add(p); } return faults; }
Paging
Try It: Paging Policies
Digits 0–9 separated by spaces (at most 16 requests). Orange cell: newly loaded page · green: hit. Enter = run again.
List Update
The List Update Problem
- A linked list of n elements; accessing the element at position i costs i (scanned from the head).
- After an access, the accessed element may be moved toward the front for free.
- Requests arrive online: which elements will be popular is unknown.
List Update · Step by step
Move-to-Front (MTF) Step by Step
Initial list A B C D E; requests E D E A D E.
Request E: at position 5 → cost 5; moved to the front. Total: 5.
Request D: at position 5 → cost 5; moved to the front. Total: 10.
Request E: at position 2 → cost 2; moved to the front. Total: 12.
Request A: at position 3 → cost 3; moved to the front. Total: 15.
Request D: at position 3 → cost 3; moved to the front. Total: 18.
Request E: at position 3 → cost 3; moved to the front. Total: 21. On the same sequence a list that never moves pays 24, and transpose pays 26.
| Strategy | E | D | E | A | D | E | Total |
|---|---|---|---|---|---|---|---|
| Move-to-front (MTF) | 5 | 5 | 2 | 3 | 3 | 3 | 21 |
| No moves | 5 | 4 | 5 | 1 | 4 | 5 | 24 |
| Transpose | 5 | 5 | 5 | 1 | 5 | 5 | 26 |
List Update
Theorem: MTF Is 2-Competitive
- Sleator and Tarjan (1985): for every request sequence σ, MTF(σ) ≤ 2 · OPT(σ) − m (m: number of requests; OPT's paid swaps count toward its cost).
- Proof idea: the potential function Φ = the number of inversions between the MTF list and the OPT list. On every access, MTF's actual cost + ΔΦ ≤ 2 · (OPT's cost) − 1. This is amortized analysis.
Transpose is not competitive
- Request the last two elements alternately: xn, xn−1, xn, … Each access swaps them, but they stay at positions n and n − 1 → ≈ n per request. MTF pays 2 per request after the first two.
Neither is frequency count
- Request x1 n times, x2 n − 1 times, …, then the same in reverse order. The counter keeps the old order for a long time; the cost can be a factor Θ(n) worse.
Data Streams
The Data Stream Model
- Elements x1, x2, … arrive one at a time; the stream may be very long or even infinite.
- Each element is seen once (a single pass); the past cannot be stored in full.
- An answer must be ready at any moment: "what is the average / median / a sample so far?"
| Problem | Stored summary | Per element | Memory |
|---|---|---|---|
| Moving average (last k) | Queue + sum | O(1) | O(k) |
| Running median | Max-heap + min-heap | O(log n) | O(n) |
| Reservoir sampling | k-element sample | O(1) | O(k) |
| Pattern matching in a stream | Last m characters + rolling hash | O(1) | O(m) |
| Running mean | n and μ | O(1) | O(1) |
Data Streams
Moving Average
- The average of the last k elements of a stream: smoothing sensor data, a 50-day moving average of a stock.
- The naive method sums k elements each time: O(k).
- The smart method: a sliding window. Keep the sum; add the new element as it enters and subtract the oldest as it leaves: O(1).
average = sum / min(t, k)
class MovingAverage { private final int k; private final Queue<Integer> window = new LinkedList<>(); private int sum = 0; MovingAverage(int k) { this.k = k; } double next(int value) { if (window.size() == k) sum -= window.poll(); window.offer(value); sum += value; return sum / (double) window.size(); } }
Data Streams · Run the code step by step
Moving Average: Code Trace
New value 4. The window holds 0 elements (< 3) → nothing to remove.
4 joins the queue and is added to the sum: sum = 4.
Average = 4 / 1 = 4.
New value 8. The window holds 1 element (< 3) → nothing to remove.
8 joins the queue and is added to the sum: sum = 12.
Average = 12 / 2 = 6.
New value 6. The window holds 2 elements (< 3) → nothing to remove.
6 joins the queue and is added to the sum: sum = 18.
Average = 18 / 3 = 6.
New value 2. The window is full (size 3 = k) → the oldest must go.
The oldest element 4 leaves the queue and is subtracted from the sum: sum = 14.
2 joins the queue and is added to the sum: sum = 16.
Average = 16 / 3 = 5.33.
New value 10. The window is full (size 3 = k) → the oldest must go.
The oldest element 8 leaves the queue and is subtracted from the sum: sum = 8.
10 joins the queue and is added to the sum: sum = 18.
Average = 18 / 3 = 6.
double next(int value) { if (window.size() == k) sum -= window.poll(); // oldest window.offer(value); sum += value; return sum / (double) window.size(); }
Data Streams
Running Median
- After each new element we want the median so far.
- Sorting every time: O(n log n). Inserting into a sorted array: O(n).
- Two heaps: the smaller half in a max-heap (left), the larger half in a min-heap (right).
- Invariants: every element of left ≤ every element of right; |left| = |right| or |left| = |right| + 1.
- The median is read from the heap tops in O(1); insertion takes O(log n).
new PriorityQueue<>((a, b) -> b - a).Data Streams · Run the code step by step
Running Median: Code Trace
x = 5: left is empty → goes into the left heap.
Sizes 1 and 0: balanced, no move.
Odd count: median = top of left = 5.
x = 15: 15 > 5 → goes into the right heap.
Sizes 1 and 1: balanced, no move.
Even count: median = (5 + 15) / 2 = 10.
x = 1: 1 ≤ 5 → goes into the left heap.
Sizes 2 and 1: balanced, no move.
Odd count: median = top of left = 5.
x = 3: 3 ≤ 5 → goes into the left heap.
Out of balance (|left| = 3, |right| = 1): the top of left, 5, moves to the right heap.
Even count: median = (3 + 5) / 2 = 4.
x = 8: 8 > 3 → goes into the right heap.
Out of balance (|left| = 2, |right| = 3): the top of right, 5, moves to the left heap.
Odd count: median = top of left = 5.
x = 7: 7 > 5 → goes into the right heap.
Sizes 3 and 3: balanced, no move.
Even count: median = (5 + 7) / 2 = 6.
void add(int x) { if (left.isEmpty() || x <= left.peek()) left.offer(x); // smaller half else right.offer(x); // larger half if (left.size() > right.size() + 1) right.offer(left.poll()); else if (right.size() > left.size()) left.offer(right.poll()); } double median() { if (left.size() == right.size()) return (left.peek() + right.peek()) / 2.0; return left.peek(); }
Data Streams
Reservoir Sampling
- Maintain a sample of k elements from a stream of unknown length, with every element equally likely to be chosen.
- Algorithm R (Vitter, 1985): put the first k elements into the reservoir (if the stream is shorter than k, return them all). Then for the i-th element (0-based, i ≥ k), pick j = random[0, i]; if j < k, set reservoir[j] ← xi.
- The i-th element enters with probability k / (i + 1); if it does, it replaces an element at a random slot.
Example: k = 2, n = 5, element x2.
The same holds for the first k elements: they enter with probability 1, and the product k/(k+1) · … · (n−1)/n = k/n. Memory O(k), O(1) per element.
Data Streams · Run the code step by step
Reservoir Sampling: Code Trace
The first k = 3 elements (10, 20, 30) go straight into the reservoir.
i = 3 (value 40): j = random[0, 3] → j = 1 (fixed for this trace).
j = 1 < 3 → r[1] = 40 (20 leaves). 40 entered with probability 3/4.
i = 4 (value 50): j = random[0, 4] → j = 4 (fixed for this trace).
j = 4 ≥ 3 → 50 does not enter; the reservoir is unchanged.
i = 5 (value 60): j = random[0, 5] → j = 0 (fixed for this trace).
j = 0 < 3 → r[0] = 60 (10 leaves). 60 entered with probability 3/6.
Stream done: the sample is [60, 40, 30]. Other values of j give other results; each element is chosen with probability 3/6 = 1/2.
List<Integer> reservoir(int[] stream, int k) { List<Integer> r = new ArrayList<>(); for (int i = 0; i < Math.min(k, stream.length); i++) r.add(stream[i]); for (int i = k; i < stream.length; i++) { int j = rng.nextInt(i + 1); // 0..i if (j < k) r.set(j, stream[i]); } return r; }
Data Streams · Run the code step by step
Pattern Matching in a Stream: Rolling Hash
Pattern "AB", m = 2, B = 101. pow = 101m−1 = 101, hp = 65 · 101 + 66 = 6631. The stream "CABDAB" arrives one character at a time; for each new character hnew = (hold − cout · Bm−1) · B + cin: O(1) instead of O(m) from scratch.
t = 0, c = C (67): the window is not full yet; h = 67.
t = 1: the window is full: "CA". h = 6832 ≠ hp → no match.
t = 2: C leaves (subtract 67 · 101), B enters. h = 6631 = hp and the characters agree → match: 1.
t = 3: A leaves (subtract 65 · 101), D enters. h = 6734 ≠ hp → no match.
t = 4: B leaves (subtract 66 · 101), A enters. h = 6933 ≠ hp → no match.
t = 5: D leaves (subtract 68 · 101), B enters. h = 6631 = hp and the characters agree → match: 4. Equal hashes are not enough: collisions are possible, so the characters are compared too.
void search(String stream, String pattern) { int m = pattern.length(), B = 101, h = 0, hp = 0; int pow = 1; // B^(m-1) for (int i = 0; i < m - 1; i++) pow *= B; for (int i = 0; i < m; i++) hp = hp * B + pattern.charAt(i); for (int t = 0; t < stream.length(); t++) { char c = stream.charAt(t); if (t >= m) h -= stream.charAt(t - m) * pow; h = h * B + c; // add incoming if (t >= m - 1 && h == hp && stream.startsWith(pattern, t - m + 1)) System.out.println("match: " + (t - m + 1)); } }
Incremental Learning · Step by step
Incremental Learning
At first the model is trained on the original data set: MODEL 1.
New data 1 arrives. Instead of retraining on all the data, the model is updated with the new data only.
MODEL 2 captures the information in both old and new data; the old data is not processed again.
New data 2 arrives; the same update is repeated.
MODEL 3 carries the information of all three data sets. The model is always up to date, and each update costs only as much as the new data.
Incremental Learning
Incremental Updates and Their Challenges
Running mean
The mean is updated with each new element; no need to store the past:
Stream 4, 8, 6: μ = 4 → 4 + (8 − 4)/2 = 6 → 6 + (6 − 6)/3 = 6.
Online gradient descent
Each example is used once; η is the learning rate.
Challenges
- Catastrophic forgetting: old knowledge is lost while adapting to new data.
- Concept drift: the data distribution changes over time; old data can mislead.
- Order effects: the model is shaped by the order in which data arrives.
- Noise: a single outlier can corrupt the model; a small η and a sliding window help.
Summary
Problems and Results
| Problem | Online algorithm | Result | Note |
|---|---|---|---|
| Ski rental | Rent for B − 1 days, then buy | 2 − 1/B | best deterministic; randomized e/(e−1) |
| Paging | LRU, FIFO, FWF | k | best deterministic |
| Paging | LFU, LIFO | unbounded | not competitive |
| Paging (randomized) | Marking | 2Hk | lower bound Hk |
| List update | Move-to-front (MTF) | 2 | transpose and frequency count are not competitive |
| Moving average | Queue + sum | O(1) | memory O(k) |
| Running median | Two heaps | O(log n) | median query O(1) |
| Reservoir sampling | Algorithm R | O(1) | each element with probability k/n |
The numbers in the top five rows are competitive ratios; those below are time per element.
Summary
Summary
| Topic | Key points |
|---|---|
| Model | the input arrives piece by piece; decisions are immediate and irrevocable |
| Competitive ratio | ALG(σ) ≤ c · OPT(σ) + α for every σ; worst-case analysis, adversary model |
| Ski rental | OPT = min(d, B); break-even achieves 2 − 1/B; no deterministic strategy does better |
| Paging | FIFO and LRU are k-competitive; OPT evicts the page needed farthest in the future; Belady's anomaly in FIFO |
| List update | cost = position; MTF is 2-competitive (potential function: inversions) |
| Data streams | single pass, little memory: sliding window, two heaps, reservoir, rolling hash |
| Incremental learning | the model is updated with new data; challenges: forgetting and concept drift |
Summary
Common Mistakes
nextInt(i) breaks the probabilities.Summary
Sample Problems
- How many page faults does OPT make on the request sequence 1 2 3 4 1 2 5 1 2 3 4 5 with k = 3?
- In ski rental B = 20. What is the ratio of the break-even strategy? What is the worst-case ratio of the strategy that buys on day 5?
- On the list A B C D, what is the total MTF cost of the requests D C D A?
- Write down the running medians for the stream 2, 9, 4, 7.
- In reservoir sampling with k = 2, with what probability does element 9 (0-based) enter the reservoir? If the stream ends after 10 elements, with what probability is this element in the final reservoir?
Show answers
- 7 faults: 1 2 3 (compulsory), 4 (evict 3), 5 (evict 4), 3 (evict 1 or 2), 4. FIFO makes 9, LRU 10.
- 2 − 1/20 = 1.95. k = 5: with d = 5, ALG = 4 + 20 = 24, OPT = 5 → 4.8.
- D: 4 → D A B C; C: 4 → C D A B; D: 2 → D C A B; A: 3 → A D C B. Total 13.
- 2 → 2; 2, 9 → 5.5; 2, 4, 9 → 4; 2, 4, 7, 9 → 5.5.
- Entering: k/(i + 1) = 2/10 = 1/5. Since it is the last element, it is also 1/5 = k/n at the end.
Summary
Exercises
- Implement FIFO paging in O(1) time with a
Queueand aHashSet. - Implement OPT (Belady) in Java. What is its time complexity?
- Find the ratio of the "buy on day B/2" strategy for ski rental.
- Why is adding
remove(x)to the running median hard? - Show that with k = 1, each element of a 3-element stream is chosen with probability 1/3.
- Find the maximum of the last k elements in amortized O(1) time.
Answers / hints
- Hit:
cached.contains(p). Fault: if full,cached.remove(queue.poll()); thenoffer+add. - Scan ahead on every fault: O(n² · k) in total; precomputing next-use indices and keeping them in a heap gives O(n log k).
- With d = B/2, ALG = (B/2 − 1) + B and OPT = B/2 → ratio 3 − 2/B. Worse than break-even (2 − 1/B).
- A heap cannot delete an arbitrary element quickly; use a
TreeMapwith counts, or lazy deletion. - 1st element: 1 · (1/2) · (2/3) = 1/3; 2nd element: (1/2) · (2/3) = 1/3; 3rd element: 1/3.
- A monotonically decreasing deque: a newcomer pops smaller elements from the back, and elements outside the window leave from the front; the front is the maximum.
Summary
References
- A. Borodin and R. El-Yaniv, Online Computation and Competitive Analysis, Cambridge University Press, 1998.
- D. D. Sleator and R. E. Tarjan, "Amortized Efficiency of List Update and Paging Rules", Communications of the ACM 28(2), 1985.
- L. A. Belady, "A Study of Replacement Algorithms for a Virtual-Storage Computer", IBM Systems Journal 5(2), 1966.
- A. Fiat, R. M. Karp, M. Luby, L. A. McGeoch, D. D. Sleator and N. E. Young, "Competitive Paging Algorithms", Journal of Algorithms 12(4), 1991.
- J. S. Vitter, "Random Sampling with a Reservoir", ACM Transactions on Mathematical Software 11(1), 1985.
- Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., 2022 (Chapter 27: Online Algorithms).