Chapter 9: String Matching and Compression
Introduction
Learning Outcomes
Define text, pattern, shift, prefix and suffix; state the string matching problem formally.
Run brute force, KMP, Boyer-Moore and Rabin-Karp step by step on an example and implement them in Java.
Explain what the LPS table, the bad-character table and the rolling hash save; compare best, average and worst cases.
Compress losslessly with RLE and LZW, decompress the result, and compute the compression ratio.
Introduction
String Algorithms
- Text is everywhere: email, chat messages, social media posts, news articles…
- Our computers handle enormous amounts of text every day. How do we organize and analyze it?
- String algorithms perform operations such as searching, replacing and comparing text.
- Two core problems: string matching (finding a pattern in a text) and string compression (storing text in less space).
Ctrl+F in editors, grep, search engines, virus signature scanning, plagiarism detection, gene search in DNA, file formats such as ZIP/GIF/PNG.
Introduction
Basic Concepts and Notation
- Alphabet Σ: the set of characters in use (e.g. {a, b, …, z}, {A, C, G, T}).
- Text T[0..n−1] and pattern P[0..m−1], usually m ≤ n.
- P occurs in T with shift s ⇔ T[s..s+m−1] = P. Valid shifts: 0 ≤ s ≤ n − m.
- Prefix: P[0..k−1]; suffix: P[m−k..m−1]. A proper prefix/suffix is one not equal to the whole string.
- Example: for "ababd" the prefixes are a, ab, aba, abab; the suffixes are d, bd, abd, babd.
The string matching problem: find all (or the first) shifts s at which P occurs.
Introduction
Chapter Map
Brute Force
The Brute-Force (Naive) Algorithm
- The simplest way to search for a pattern in a text, hence the name "naive".
- How it works:
- For each shift s = 0, 1, …, n − m, align the pattern under the text.
- Compare characters left to right; stop at the first mismatch.
- If all m characters match, report s; otherwise slide the pattern one position.
- No preprocessing and no extra memory.
Worst case: (n − m + 1) · m comparisons → O(n · m).
Best case: the first character mismatches at every shift → O(n).
n: text length, m: pattern length.
Brute Force
Example: "THIS IS A SIMPLE EXAMPLE"
Each row is one shift: green = match, red = mismatch. SIMPLE is found at the 11th shift (s = 10) after 18 comparisons in total (n = 24, m = 6).
Brute Force · Step through the code
"LO" in "HELLO WORLD": Code Trace
n = 11, m = 2. The pattern will be tried at shifts i = 0, 1, …, 9.
t[0] = 'H' ≠ p[0] = 'L' → mismatch; slide the pattern one to the right.
t[1] = 'E' ≠ p[0] = 'L' → mismatch; slide the pattern one to the right.
t[2] = 'L' matches p[0] = 'L' → j++.
t[3] = 'L' ≠ p[1] = 'O' → mismatch; slide the pattern one to the right.
t[3] = 'L' matches p[0] = 'L' → j++.
t[4] = 'O' matches p[1] = 'O' → j++.
j == m: pattern found at i = 3. 6 comparisons in total.
int bruteForce(String t, String p) { int n = t.length(), m = p.length(); for (int i = 0; i <= n - m; i++) { int j = 0; while (j < m && t.charAt(i + j) == p.charAt(j)) j++; if (j == m) return i; } return -1; }
Brute Force
Brute Force: Best, Average, Worst
| Case | When? | Comparisons | Time |
|---|---|---|---|
| Best | P[0] matches no character of the text (e.g. P = "xyz", T = "aaaa…") | n − m + 1 | O(n) |
| Average | Random text, large alphabet: a mismatch usually comes within the first 1–2 characters | ≈ n · σ/(σ−1) | O(n) |
| Worst | T = "aaaaaaaaaa", P = "aaab": m comparisons at every shift | (n − m + 1) · m | O(n · m) |
KMP
The Knuth-Morris-Pratt (KMP) Algorithm
- Finds occurrences of a pattern in a text.
- Developed by Donald Knuth and Vaughan Pratt and, independently, by James H. Morris; published jointly in 1977.
- Key idea: on a mismatch, use what we already know about the part of the pattern that has matched; the text pointer never moves back.
- That knowledge is precomputed from the pattern and stored in the LPS table (the prefix function π).
Preprocessing: O(m)
Search: O(n)
Total: O(n + m), even in the worst case
Extra memory: O(m)
KMP
KMP: How It Works
KMP
The LPS Table (Prefix Function)
lps[q] = length of the longest proper prefix of P[0..q] that is also a suffix of P[0..q].
- lps[0] = 0 (a single character has no proper prefix).
- Example: text "ababcababcabababd", pattern "ababd".
- Pattern table: a: 0, b: 0, a: 1, b: 2, d: 0.
| q | P[q] | lps[q] | Reason (P[0..q]) |
|---|---|---|---|
| 0 | a | 0 | "a" — no proper prefix |
| 1 | b | 0 | "ab" — a ≠ b |
| 2 | a | 1 | "aba" — "a" is both prefix and suffix |
| 3 | b | 2 | "abab" — "ab" is both prefix and suffix |
| 4 | d | 0 | "ababd" — no prefix ends in d |
KMP · Step through the code
Building the LPS Table: "ACABACACD"
lps[0] = 0 always. k = 0: length of the prefix matched so far.
p[0] = 'A' ≠ p[1] = 'C' → k = 0, lps[1] = 0.
p[0] = 'A' = p[2] = 'A' → k = 1, lps[2] = 1.
p[1] = 'C' ≠ p[3] = 'B' → fall back: k = lps[0] = 0.
p[0] = 'A' ≠ p[3] = 'B' → k = 0, lps[3] = 0.
p[0] = 'A' = p[4] = 'A' → k = 1, lps[4] = 1.
p[1] = 'C' = p[5] = 'C' → k = 2, lps[5] = 2. "AC" is both prefix and suffix.
p[2] = 'A' = p[6] = 'A' → k = 3, lps[6] = 3. "ACA" is both prefix and suffix.
p[3] = 'B' ≠ p[7] = 'C' → fall back: k = lps[2] = 1.
p[1] = 'C' = p[7] = 'C' → k = 2, lps[7] = 2. "AC" is both prefix and suffix.
p[2] = 'A' ≠ p[8] = 'D' → fall back: k = lps[1] = 0.
p[0] = 'A' ≠ p[8] = 'D' → k = 0, lps[8] = 0.
Result: lps = [0, 0, 1, 0, 1, 2, 3, 2, 0]. Constant amortized work per q → O(m).
int[] computeLPS(String p) { int m = p.length(), k = 0; int[] lps = new int[m]; for (int q = 1; q < m; q++) { while (k > 0 && p.charAt(k) != p.charAt(q)) k = lps[k - 1]; if (p.charAt(k) == p.charAt(q)) k++; lps[q] = k; } return lps; }
KMP
The Three Phases of a KMP Search
KMP
KMP Example: Searching for "ABCDABD"
lps("ABCDABD") = [0, 0, 0, 0, 1, 2, 0]. green = match, red = mismatch, blue = known from LPS (not compared again).
s = 0: ABC matched, t[3] = 'A' ≠ p[3] = 'D'. q = 3 → q = lps[2] = 0; the pattern slides 3 and t[3] is compared with p[0].
s = 3: ABCDAB matched, t[9] = 'A' ≠ 'D'. q = 6 → q = lps[5] = 2: "AB" is known to match; the pattern slides 4.
s = 7: AB is not compared again. t[9] = 'A' ≠ p[2] = 'C' → q = lps[1] = 0.
s = 9: ABCDAB matched, t[15] = 'C' ≠ 'D'. q = 6 → q = lps[5] = 2; the pattern slides 4.
s = 13: AB is known; CDABD matches too → pattern found at position 13. The text pointer never moved back.
24 character comparisons in total (n = 21); brute force makes 37 on the same text.
KMP · Step through the code
kmpSearch("ababcababcabababd", "ababd")
n = 17, m = 5, lps = [0, 0, 1, 2, 0], q = 0.
p[0] = t[0] = 'a' → q = 1.
p[1] = t[1] = 'b' → q = 2.
p[2] = t[2] = 'a' → q = 3.
p[3] = t[3] = 'b' → q = 4.
p[4] = 'd' ≠ t[4] = 'c' → q = lps[3] = 2 (i stays put).
p[2] = 'a' ≠ t[4] = 'c' → q = lps[1] = 0 (i stays put).
p[0] = 'a' ≠ t[4] = 'c' → q stays 0, i advances.
p[0] = t[5] = 'a' → q = 1.
p[1] = t[6] = 'b' → q = 2.
p[2] = t[7] = 'a' → q = 3.
p[3] = t[8] = 'b' → q = 4.
p[4] = 'd' ≠ t[9] = 'c' → q = lps[3] = 2 (i stays put).
p[2] = 'a' ≠ t[9] = 'c' → q = lps[1] = 0 (i stays put).
p[0] = 'a' ≠ t[9] = 'c' → q stays 0, i advances.
p[0] = t[10] = 'a' → q = 1.
p[1] = t[11] = 'b' → q = 2.
p[2] = t[12] = 'a' → q = 3.
p[3] = t[13] = 'b' → q = 4.
p[4] = 'd' ≠ t[14] = 'a' → q = lps[3] = 2 (i stays put).
p[2] = t[14] = 'a' → q = 3.
p[3] = t[15] = 'b' → q = 4.
p[4] = t[16] = 'd' → q = 5.
q == m → i − m + 1 = 12 is returned. The text was scanned only once.
int kmpSearch(String t, String p) { int n = t.length(), m = p.length(); int[] lps = computeLPS(p); int q = 0; // matched length for (int i = 0; i < n; i++) { while (q > 0 && p.charAt(q) != t.charAt(i)) q = lps[q - 1]; if (p.charAt(q) == t.charAt(i)) q++; if (q == m) return i - m + 1; } return -1; }
KMP
KMP: Why O(n + m)?
- The outer loop increments i n times; i never decreases.
- q grows by at most 1 per step → total increase ≤ n.
- Each fallback in the
whileloop decreases q by at least 1; q cannot go negative → total fallbacks ≤ total increase ≤ n. - Search: at most 2n comparisons → O(n). The same amortized argument gives O(m) for LPS.
Pros
- Linear even in the worst case
- Never backs up in the text: suits streaming data
Cons
- Usually slower than Boyer-Moore in practice
- Little to skip over with a small alphabet (DNA)
Boyer-Moore
The Boyer-Moore Algorithm
- Searches for a pattern in a text; developed by Robert S. Boyer and J Strother Moore in 1977.
- The pattern slides left to right, but characters are compared right to left.
- On a mismatch, shift by the larger of two rules: bad character and good suffix.
- Most text characters can be skipped without ever being read: very fast for long patterns over large alphabets (natural language);
grepand editor search belong to this family.
Best: O(n/m) — the first comparison fails at every attempt and we skip m.
Average: sublinear, faster than O(n) in practice.
Worst: O(n · m) with bad character alone; O(n + m) with good suffix + Galil's rule.
Boyer-Moore
Boyer-Moore: How It Works
- Preprocessing: build shift tables from the pattern: the last occurrence of each character in the pattern (bad character) and a shift amount for each suffix (good suffix).
- Search: align the pattern under the text; compare starting from the last character and moving left.
- Match check: if all characters match, report the position.
- Bad-character rule: shift according to the rightmost position in the pattern of the mismatched text character.
- Good-suffix rule: shift according to whether the matched suffix occurs elsewhere in the pattern.
Red numbers give the comparison order: only 6 comparisons on a 12-character text.
Boyer-Moore
The Bad-Character Rule
- Let p[j] ≠ t[s + j]; c = t[s + j] is the bad character.
- If c occurs in the pattern left of j: shift so that the rightmost c lines up under t[s + j].
- If c does not occur in the pattern (last = −1): shift the pattern entirely past c (j + 1).
- Formula: s += max(1, j − last[c]). If c last occurs right of j, then j − last[c] ≤ 0; hence the minimum of 1.
int[] lastOccurrence(String p) { int[] last = new int[256]; Arrays.fill(last, -1); for (int i = 0; i < p.length(); i++) last[p.charAt(i)] = i; return last; }
Boyer-Moore
The Good-Suffix Rule
- The suffix u = P[j+1..m−1] matched the text before the mismatch: the "good suffix".
- Case 1: if u occurs elsewhere in the pattern (preceded by a different character), shift so that this occurrence lines up under u in the text.
- Case 2: otherwise, align the longest prefix of P that equals a suffix of u.
- Case 3: if neither exists, shift the pattern by m.
- The table is precomputed in O(m) time (similar to KMP's LPS idea, but for suffixes).
"AB" matched, t[2] = 'A' ≠ p[2] = 'B'. Bad character: last[A] = 3 > j → only 1. Good suffix: "AB" also occurs at p[1..2] (preceded by 'C' ≠ 'B') → shift 2.
Boyer-Moore
Example: "HELLO" in "LO LOELLO O HELLO"
| c | H | E | L | O | other |
|---|---|---|---|---|---|
| last[c] | 0 | 1 | 3 | 4 | −1 |
Rule: s += max(1, j − last[c]). green = match, red = bad character.
Alignment 1 (s = 0): 2 characters matched, t[2] = '␣' ≠ p[2] = 'L'. '␣' is not in the pattern → j − (−1) = 3 → shift 3.
Alignment 2 (s = 3): t[7] = 'L' ≠ p[4] = 'O'. last['L'] = 3 → j − 3 = 1 → shift 1.
Alignment 3 (s = 4): 4 characters matched, t[4] = 'O' ≠ p[0] = 'H'. last['O'] = 4 ≥ j → at least 1 → shift 1.
Alignment 4 (s = 5): t[9] = '␣' ≠ p[4] = 'O'. '␣' is not in the pattern → j − (−1) = 5 → shift 5.
Alignment 5 (s = 10): t[14] = 'L' ≠ p[4] = 'O'. last['L'] = 3 → j − 3 = 1 → shift 1.
Alignment 6 (s = 11): t[15] = 'L' ≠ p[4] = 'O'. last['L'] = 3 → j − 3 = 1 → shift 1.
Alignment 7 (s = 12): all 5 characters match right to left → HELLO found at s = 12. 7 alignments, 17 comparisons in total.
Boyer-Moore
Horspool's Simplification and the Shift Table
- Horspool (1980): wherever the mismatch occurs, shift according to the text character under the last position of the window.
- Shift table: shift[c] = m − 1 − (rightmost position of c in P[0..m−2]); m if c does not occur there.
- HELLO: H → 4, E → 3, L → 1, O → 5 (the last character is excluded; O occurs nowhere else), other → 5.
- The same example needs only 5 alignments (7 with the basic bad-character rule).
| Alignment | s | Comparison | last char | shift |
|---|---|---|---|---|
| 1 | 0 | t[2] = '␣' ≠ p[2] | 'O' | 5 |
| 2 | 5 | t[9] = '␣' ≠ p[4] | '␣' | 5 |
| 3 | 10 | t[14] = 'L' ≠ p[4] | 'L' | 1 |
| 4 | 11 | t[15] = 'L' ≠ p[4] | 'L' | 1 |
| 5 | 12 | all match | — | found |
Horspool · Step through the code
horspool("LO LOELLO O HELLO", "HELLO")
n = 17, m = 5. All 256 cells of shift start at m = 5: on a character not in the pattern, the pattern jumps entirely past it.
Scan P[0..3], excluding the last character: H → 4, E → 3, L first 2 then 1 (the rightmost occurrence wins). O occurs only at the end, so it stays 5.
s = 0: 2 characters match from the right, then t[2] = '␣' ≠ p[2] = 'L' → mismatch.
The shift looks at the last character of the window, not at the mismatch: t[4] = 'O' → shift = 5 → s = 5.
s = 5: t[9] = '␣' ≠ p[4] = 'O' → mismatch.
The window's last character t[9] = '␣' is not in the pattern → shift = 5: the pattern jumps entirely past it, s = 10.
s = 10: t[14] = 'L' ≠ p[4] = 'O' → mismatch.
The shift looks at the last character of the window, not at the mismatch: t[14] = 'L' → shift = 1 → s = 11.
s = 11: t[15] = 'L' ≠ p[4] = 'O' → mismatch.
The shift looks at the last character of the window, not at the mismatch: t[15] = 'L' → shift = 1 → s = 12.
All five characters match right to left, j = −1 → return s = 12. 5 alignments, 11 comparisons in total.
int horspool(String t, String p) { int n = t.length(), m = p.length(); int[] shift = new int[256]; Arrays.fill(shift, m); for (int i = 0; i < m - 1; i++) shift[p.charAt(i)] = m - 1 - i; int s = 0; while (s <= n - m) { int j = m - 1; while (j >= 0 && p.charAt(j) == t.charAt(s + j)) j--; if (j < 0) return s; s += shift[t.charAt(s + m - 1)]; } return -1; }
Boyer-Moore
Example: Searching with the Last-Occurrence Table
Pattern abacab, last occurrence: a = 4, b = 5, c = 3, * = −1. The number above a cell is the comparison counter.
Alignment 1 (s = 0): t[5] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.
Alignment 2 (s = 1): t[4] = 'a' ≠ p[3] = 'c'. last['a'] = 4 ≥ j = 3 → 1 → shift 1.
Alignment 3 (s = 2): t[7] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.
Alignment 4 (s = 3): t[8] = 'd' ≠ p[5] = 'b'. 'd' is not in the pattern → j + 1 = 6 → shift 6.
Alignment 5 (s = 9): t[14] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.
Alignment 6 (s = 10): all 6 comparisons match → match at s = 10. We want all occurrences, so shift 1 and continue.
Alignment 7 (s = 11): t[16] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.
Alignment 8 (s = 12): t[17] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.
Alignment 9 (s = 13): t[16] = 'a' ≠ p[3] = 'c'. last['a'] = 4 ≥ j = 3 → 1 → shift 1.
Alignment 10 (s = 14): t[18] = 'b' ≠ p[4] = 'a'. last['b'] = 5 ≥ j = 4 → 1 → shift 1.
10 alignments, 20 comparisons in total (n = 20, m = 6): a character not in the pattern, such as 'd', yields a 6-position jump.
Boyer-Moore · Step through the code
boyerMoore("merhaba naber", "abe")
n = 13, m = 3. Last-occurrence table built; s = 0.
p[2] = 'e' ≠ t[2] = 'r'; last['r'] = -1 → s += max(1, 2 − (-1)) = 3.
p[2] = 'e' ≠ t[5] = 'b'; last['b'] = 1 → s += max(1, 2 − (1)) = 1.
p[2] = 'e' ≠ t[6] = 'a'; last['a'] = 0 → s += max(1, 2 − (0)) = 2.
p[2] = 'e' ≠ t[8] = 'n'; last['n'] = -1 → s += max(1, 2 − (-1)) = 3.
p[2] = t[11] = 'e' → j-- .
p[1] = t[10] = 'b' → j-- .
p[0] = t[9] = 'a' → j-- .
j < 0 → pattern found at s = 9 ("naber").
int boyerMoore(String t, String p) { int n = t.length(), m = p.length(); int[] last = lastOccurrence(p); int s = 0; while (s <= n - m) { int j = m - 1; while (j >= 0 && p.charAt(j) == t.charAt(s + j)) j--; if (j < 0) return s; s += Math.max(1, j - last[t.charAt(s + j)]); } return -1; }
Boyer-Moore
Boyer-Moore: Analysis and Example
- Best: text characters do not occur in the pattern → 1 comparison per alignment, skip m: ≈ n/m comparisons.
- Worst (bad character only): T = aaaa…a, P = baaa…a → m comparisons per alignment, shift 1: O(n · m).
- With a large alphabet and frequent mismatches, longer patterns give longer shifts (best ≈ n/m).
- With a small alphabet (DNA, binary data) the bad-character rule is weak; the good-suffix rule matters more.
Text: abccbaabccbaabcbcabbabcabc
Pattern: abcbcabbabcabc (m = 14)
Last-occurrence table: a: 11, b: 12, c: 13
Result: the pattern is found at position 12.
Rabin-Karp
The Rabin-Karp Algorithm
- Finds occurrences of a pattern in a text; developed by Michael O. Rabin and Richard M. Karp in 1987.
- Instead of comparing characters one by one, it compares the hash of the pattern with the hash of each length-m window of the text.
- Rolling hash: when the window slides one step right, the new hash is computed from the old one in O(1).
- Equal hashes do not guarantee a match: a spurious hit is possible → verify character by character.
Best: hashes rarely collide → O(n + m)
Average: O(n + m) in practice
Worst: a collision (spurious hit) at every window, constant verification → O(n · m)
Rabin-Karp
Rabin-Karp: How It Works and the Rolling Hash
- Hash: compute the hash of the pattern and of the first text window.
- Match check: substrings whose hash matches are candidates.
- Verification: compare a candidate window with the pattern character by character.
- Slide and rehash: if there is no match, slide the window one step right and update the hash.
- Repeat: continue across the whole text.
h(t[s..s+m−1]) = (t[s]·Bm−1 + t[s+1]·Bm−2 + … + t[s+m−1]) mod Q
Slide: hs+1 = (B · (hs − t[s] · Bm−1) + t[s+m]) mod Q — subtract the outgoing character, shift the rest one digit left, add the incoming character.
Rabin-Karp
A Simple Example: Sum Hash and Spurious Hits
Toy hash: the sum of the letter values. Searching for "TS" in "VUATS".
Window "VU": 5 + 1 = 6 ≠ 5 → hashes differ, no character comparison.
Window "UA": 1 + 4 = 5 = 5 → equal hashes, but "UA" ≠ "TS": a spurious hit; the verification is wasted.
Window "AT": 4 + 2 = 6 ≠ 5 → skip. New hash = old − outgoing (U=1) + incoming (T=2): O(1).
Window "TS": 2 + 3 = 5 → equal hashes, and the characters agree too: match (s = 3).
The sum hash is weak (it ignores letter order: "ST" also gives 5). That is why a position-weighted polynomial hash with a large prime Q is used.
Rabin-Karp · Step through the code
rabinKarp("abracadabra", "cad")
n = 11, m = 3, B = 256, Q = 101. h = 256² mod 101 = 88 (weight of the outgoing character).
ph = (99·256² + 97·256 + 100) mod 101 = 11; for the first window "abr", th = 4.
Window "abr": th = 4 ≠ 11 → characters not checked. Slide: 'a' out, 'a' in → new th = 30.
Window "bra": th = 30 ≠ 11 → characters not checked. Slide: 'b' out, 'c' in → new th = 17.
Window "rac": th = 17 ≠ 11 → characters not checked. Slide: 'r' out, 'a' in → new th = 41.
Window "aca": th = 41 ≠ 11 → characters not checked. Slide: 'a' out, 'd' in → new th = 11.
th = 11 = ph → character check with matchAt: "cad" = "cad" → return s = 4.
int rabinKarp(String t, String p) { int n = t.length(), m = p.length(); int B = 256, Q = 101, h = 1; for (int i = 0; i < m - 1; i++) h = (h * B) % Q; // B^(m-1) mod Q int ph = 0, th = 0; for (int i = 0; i < m; i++) { ph = (B * ph + p.charAt(i)) % Q; th = (B * th + t.charAt(i)) % Q; } for (int s = 0; s <= n - m; s++) { if (ph == th && matchAt(t, p, s)) return s; if (s < n - m) { th = (B * (th - t.charAt(s) * h) + t.charAt(s + m)) % Q; if (th < 0) th += Q; } } return -1; }
Rabin-Karp
The Sliding Window and Analysis
n − m + 1 = 7 windows; each computed from the previous one in O(1).
- Hashing cost: O(m) + (n − m) · O(1).
- With a large prime Q, a spurious hit has probability ≈ 1/Q → expected time O(n + m).
- Bad Q or adversarial input: every window is a candidate → O(n · m).
- Strength: many patterns of the same length can be searched in one pass (put their hashes in a set): plagiarism detection, document similarity.
- It also generalizes to 2D pattern search (a block within an image).
Matching Summary
Comparing String Matching Algorithms
| Algorithm | Key idea | Preprocessing | Average | Worst | Extra memory |
|---|---|---|---|---|---|
| Brute force | Tries every position | none | O(n)* | O(n · m) | O(1) |
| KMP | LPS table, never backs up in the text | O(m) | O(n + m) | O(n + m) | O(m) |
| Boyer-Moore | Right to left, bad character + good suffix | O(m + σ) | sublinear | O(n · m)** | O(m + σ) |
| Rabin-Karp | Compares rolling hashes | O(m) | O(n + m) | O(n · m) | O(1) |
* for random text over a large alphabet; on regular (repetitive) text it approaches O(n · m). ** with the bad-character rule alone; O(n + m) with good suffix + Galil's rule. σ: alphabet size.
Compression
String Compression Algorithms
- Goal: store or transmit text (or data) using fewer bits.
- Lossless compression: decompression gives back the original data exactly. RLE and LZW are lossless (text, programs, PNG/GIF).
- Lossy compression: perceptually unimportant detail is discarded (JPEG, MP3); not suitable for text.
- Compression ratio = compressed size / original size (smaller is better).
RLE
Run-Length Encoding (RLE)
- Run-length encoding replaces each sequence of repeated characters (a run) with a count + character pair.
- Steps: scan the text left to right → count consecutive repeats → combine count and character → store the result as the compressed text.
- The text is scanned once, so the time is O(n).
- Simple and effective, but useless or even harmful when there is little repetition: "ABCD" → "1A1B1C1D" (twice the size!).
- Used in: fax (black-and-white lines), BMP/PCX/TIFF, simple icons.
Text: AAAAABBBCCCCDDD (15 characters)
Compressed: 5A3B4C3D (8 characters)
Ratio: 8/15 ≈ 0.53
RLE · Step through the code
rleCompress("AAAAABBBCCCCDDD")
n = 15, result is empty.
New run: c = 'A', count = 1 (i = 0).
The while loop ran 4 times: count = 5, i = 4. Append "5A" to result.
New run: c = 'B', count = 1 (i = 5).
The while loop ran 2 times: count = 3, i = 7. Append "3B" to result.
New run: c = 'C', count = 1 (i = 8).
The while loop ran 3 times: count = 4, i = 11. Append "4C" to result.
New run: c = 'D', count = 1 (i = 12).
The while loop ran 2 times: count = 3, i = 14. Append "3D" to result.
Result: "5A3B4C3D" — 8 characters instead of 15.
String rleCompress(String s) {
StringBuilder result = new StringBuilder();
int n = s.length();
for (int i = 0; i < n; i++) {
char c = s.charAt(i);
int count = 1;
while (i + 1 < n &&
s.charAt(i + 1) == c) {
count++;
i++;
}
result.append(count).append(c);
}
return result.toString();
}RLE
Byte-Level RLE Encoding
Text: aaaaaaaaaabbbbbbececececececdddddddddddddddecb (46 bytes)
| Count–character pairs | (10, a)(6, b)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(15, d)(1, e)(1, c)(1, b) |
| RLE without flag (2 bytes per run) | 10 97 06 98 01 101 01 99 01 101 01 99 01 101 01 99 01 101 01 99 01 101 01 99 01 101 01 99 15 100 01 101 01 99 01 98 → 36 bytes |
| RLE with flag (flag = 255) | 255 10 97 255 06 98 101 99 101 99 101 99 101 99 101 99 101 99 255 15 100 101 99 98 → 24 bytes |
255 count char triple and leaves other characters as they are (the value 255 itself must be escaped).RLE
RLE on a Binary (Black-and-White) Image
- Each row is scanned left to right: W = white, B = black, followed by the run length.
- 42 pixels → 18 runs (each row is encoded separately). With only two colors, the color need not be written at all: runs simply alternate white/black (the fax standard uses this idea).
- Very effective for images with large flat areas (drawings, scanned text); useless for photographs.
RLE
Pixel-Level RLE (RGB)
Blue (0, 0, 255), black (0, 0, 0), yellow (255, 255, 0). Raw: 64 × 3 = 192 bytes. RLE: 33 runs × (1 count + 3 color) = 132 bytes.
| Count | R G B | Count | R G B | Count | R G B |
|---|---|---|---|---|---|
| 2 | 0 0 255 | 4 | 0 0 0 | 3 | 0 0 255 |
| 1 | 0 0 0 | 4 | 255 255 0 | 1 | 0 0 0 |
| 1 | 0 0 255 | 1 | 0 0 0 | 1 | 255 255 0 |
| 1 | 0 0 0 | 2 | 255 255 0 | 1 | 0 0 0 |
| 1 | 255 255 0 | 2 | 0 0 0 | 6 | 255 255 0 |
| 2 | 0 0 0 | 1 | 255 255 0 | 1 | 0 0 0 |
| 2 | 255 255 0 | 1 | 0 0 0 | 1 | 255 255 0 |
| 2 | 0 0 0 | 2 | 255 255 0 | 2 | 0 0 0 |
| 2 | 255 255 0 | 1 | 0 0 0 | 1 | 0 0 255 |
| 1 | 0 0 0 | 4 | 255 255 0 | 1 | 0 0 0 |
| 3 | 0 0 255 | 4 | 0 0 0 | 2 | 0 0 255 |
Read the table row by row (left to right, then the next row); if the same color continues at the end of an image row, the run carries over to the next row (e.g. 3 blue).
LZW
Lempel-Ziv-Welch (LZW) Compression
- Dictionary-based lossless compression: Terry Welch (1984) refined Lempel and Ziv's LZ78 method (1978).
- Identifies repeated strings and represents them by short codes (indices) into a dictionary.
- The dictionary is dynamic: it grows during compression and is never transmitted; the decompressor rebuilds the same dictionary on its own.
- Used in formats such as GIF, TIFF, PDF and Unix
compress. - As the dictionary grows, code width (bits) and memory grow and performance drops: in practice the dictionary is capped (GIF: 4096 codes = 12 bits) and reset when full.
Text: ABABCABABCA
Initial dictionary: A = 0, B = 1, C = 2
Compressed: 0 1 3 2 3 5 0
7 codes instead of 11 symbols.
LZW · Step through the code
lzw("ABABCABABCA")
The dictionary starts with the characters of the text: A = 0, B = 1, C = 2. w = "".
"A" is in dict (0) → w = "A"; look for a longer match.
"AB" not in dict → output code of w = "A": 0; add "AB" = 3 to dict; w = "B".
"BA" not in dict → output code of w = "B": 1; add "BA" = 4 to dict; w = "A".
"AB" is in dict (3) → w = "AB"; look for a longer match.
"ABC" not in dict → output code of w = "AB": 3; add "ABC" = 5 to dict; w = "C".
"CA" not in dict → output code of w = "C": 2; add "CA" = 6 to dict; w = "A".
"AB" is in dict (3) → w = "AB"; look for a longer match.
"ABA" not in dict → output code of w = "AB": 3; add "ABA" = 7 to dict; w = "A".
"AB" is in dict (3) → w = "AB"; look for a longer match.
"ABC" is in dict (5) → w = "ABC"; look for a longer match.
"ABCA" not in dict → output code of w = "ABC": 5; add "ABCA" = 8 to dict; w = "A".
Loop done: output the code of the leftover w = "A", 0. Output: 0 1 3 2 3 5 0.
List<Integer> lzw(String s) {
Map<String, Integer> dict =
initDict(s); // A=0, B=1, C=2
String w = "";
List<Integer> codes = new ArrayList<>();
for (char c : s.toCharArray()) {
String wc = w + c;
if (dict.containsKey(wc)) {
w = wc;
} else {
codes.add(dict.get(w));
dict.put(wc, dict.size());
w = "" + c;
}
}
if (!w.isEmpty())
codes.add(dict.get(w));
return codes;
}LZW
LZW Decompression
- The decompressor knows only the initial dictionary; it builds the rest itself, in the same order as the compressor.
- For each new code k: write e = dict[k]; add w + e[0] to the dictionary; set w = e.
- The decompressor lags one step behind. If k is not yet defined (k = the next code), e must be w + w[0] (the cScSc case).
- Output: A·B·AB·C·AB·ABC·A = ABABCABABCA ✓
| Code | Output | Added |
|---|---|---|
| 0 | A | — |
| 1 | B | 3 = AB |
| 3 | AB | 4 = BA |
| 2 | C | 5 = ABC |
| 3 | AB | 6 = CA |
| 5 | ABC | 7 = ABA |
| 0 | A | 8 = ABCA |
LZW
LZW Example: A DNA Sequence
Text: ATGATCATGAG (11 symbols)
Initial: A = 0, T = 1, G = 2, C = 3
Output: 0 1 2 4 3 4 6 2 (8 codes)
- Each row: the new string added to the dictionary and the code output at that moment.
- The second time "AT" appears, it is written as a single code (4); likewise "GA" (6).
- The final w = "G" is output after the loop (2).
| Code | Added to dictionary | Output |
|---|---|---|
| 4 | AT | 0 |
| 5 | TG | 1 |
| 6 | GA | 2 |
| 7 | ATC | 4 |
| 8 | CA | 3 |
| 9 | ATG | 4 |
| 10 | GAG | 6 |
| — | (final w = G) | 2 |
LZW
LZW with an ASCII Dictionary and the Special Case
- In practice the initial dictionary holds the 256 single-byte values (0–255; the first 128 are ASCII; a = 97, b = 98, c = 99); new strings are numbered from 256.
- Text:
aabaacababacb(13 characters) → 9 codes: 97 97 98 256 99 257 261 99 98. - "aba" = 261 is added while "ab" (257) is written and is used right in the next code.
- When the decompressor reads 261, its dictionary only goes up to 260 → special case: e = w + w[0] = "ab" + "a" = "aba".
| Output | Added to dictionary |
|---|---|
| 97 | 256 = aa |
| 97 | 257 = ab |
| 98 | 258 = ba |
| 256 | 259 = aac |
| 99 | 260 = ca |
| 257 | 261 = aba |
| 261 | 262 = abac |
| 99 | 263 = cb |
| 98 | (final w = b) |
LZW · Step through the code
lzwDecode([97 97 98 256 99 257 261 99 98])
The decompressor knows only the 256 single-byte values. The first code, 97, is always a single character: w = "a", output "a". Nothing from 256 up exists yet.
k = 97 (ASCII) → write e = "a"; add w + e[0] = "a" + "a" = 256 = "aa" to dict.
k = 98 (ASCII) → write e = "b"; add w + e[0] = "a" + "b" = 257 = "ab" to dict.
k = 256 (in dict: 256 = "aa") → write e = "aa"; add w + e[0] = "b" + "a" = 258 = "ba" to dict.
k = 99 (ASCII) → write e = "c"; add w + e[0] = "aa" + "c" = 259 = "aac" to dict.
k = 257 (in dict: 257 = "ab") → write e = "ab"; add w + e[0] = "c" + "a" = 260 = "ca" to dict.
k = 261, but the largest code in the dictionary is 260: special case. e = w + w[0] = "ab" + "a" = "aba"; 261 = "aba" is added too. The compressor used this code the moment it added it.
k = 99 (ASCII) → write e = "c"; add w + e[0] = "aba" + "c" = 262 = "abac" to dict.
k = 98 (ASCII) → write e = "b"; add w + e[0] = "c" + "b" = 263 = "cb" to dict.
The 9 codes turned back into the 13-character text aabaacababacb; the decompressor's entries 256–263 match the compressor's.
String lzwDecode(List<Integer> codes) {
List<String> dict =
initDict(); // 0..255: byte values
String w = dict.get(codes.get(0));
StringBuilder s = new StringBuilder(w);
for (int i = 1; i < codes.size(); i++) {
int k = codes.get(i);
String e = k < dict.size()
? dict.get(k)
: w + w.charAt(0); // special case
s.append(e);
dict.add(w + e.charAt(0));
w = e;
}
return s.toString();
}LZW
LZ78: The Ancestor of LZW
- LZ78 starts with an empty dictionary; its output is (index, character) pairs: the index of the longest dictionary match (0 if none) + the character that follows it.
- Text:
aabaacabcabcb→ (0,a) (1,b) (1,a) (0,c) (2,c) (5,b). - Welch's change (LZW): start the dictionary with every single character → the output consists of codes only; no characters need to be written.
- The LZ77 (sliding window) family underlies gzip/ZIP (DEFLATE) and PNG.
| Output | Dictionary |
|---|---|
| (0, a) | 1 = a |
| (1, b) | 2 = ab |
| (1, a) | 3 = aa |
| (0, c) | 4 = c |
| (2, c) | 5 = abc |
| (5, b) | 6 = abcb |
LZW · Step through the code
lz78("aabaacabcabcb")
The LZ78 dictionary starts empty (unlike LZW). w holds the longest piece found in the dictionary so far; for now it is empty.
w is empty and 'a' is not in dict → (0, a) (0 = empty prefix); add 1 = "a".
w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.
"ab" not in dict → write w = "a" (index 1) and c = 'b': (1, b); add 2 = "ab", reset w.
w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.
"aa" not in dict → write w = "a" (index 1) and c = 'a': (1, a); add 3 = "aa", reset w.
w is empty and 'c' is not in dict → (0, c) (0 = empty prefix); add 4 = "c".
w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.
w + c = "ab" is in dict → w = "ab"; no output yet, try a longer piece with the next character.
"abc" not in dict → write w = "ab" (index 2) and c = 'c': (2, c); add 5 = "abc", reset w.
w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.
w + c = "ab" is in dict → w = "ab"; no output yet, try a longer piece with the next character.
w + c = "abc" is in dict → w = "abc"; no output yet, try a longer piece with the next character.
"abcb" not in dict → write w = "abc" (index 5) and c = 'b': (5, b); add 6 = "abcb", reset w. End of text: 6 pairs.
List<String> lz78(String s) {
Map<String, Integer> dict =
new HashMap<>();
List<String> out = new ArrayList<>();
String w = "";
for (char c : s.toCharArray()) {
if (dict.containsKey(w + c)) {
w = w + c; // extend
} else {
int k = w.isEmpty() ? 0
: dict.get(w);
out.add("(" + k + ", " + c + ")");
dict.put(w + c, dict.size() + 1);
w = "";
}
}
if (!w.isEmpty()) // last piece
out.add("(" + dict.get(w) + ")");
return out;
}LZW
RLE vs LZW
| RLE | LZW | |
|---|---|---|
| Redundancy captured | consecutive repeats of one symbol | repeated strings (even if not consecutive) |
| Extra structure | none | dictionary (hash table / trie) |
| Time | O(n) | O(n) (O(1) average dictionary access with a trie or a (code, char)-keyed dictionary, not String concatenation) |
| Worst case | output doubles if nothing repeats | little gain on short/random data, code width grows |
| Used in | fax, BMP/PCX, simple graphics | GIF, TIFF, PDF, Unix compress |
Summary
Summary
| Topic | Key points |
|---|---|
| Brute force | try every shift left to right; no preprocessing — worst case O(n · m) |
| KMP | lps[q] = longest proper prefix that is also a suffix; on a mismatch q = lps[q − 1], i never backs up — O(n + m) |
| Boyer-Moore | right to left; s += max(1, j − last[c]) plus good suffix; best O(n/m), fastest in practice |
| Rabin-Karp | polynomial rolling hash, O(1) update; verify on equal hashes; expected O(n + m) |
| RLE | run → (count, symbol); O(n); expands data without repeats; improved by a flag byte |
| LZW | if w + c is in the dictionary, extend; else output code(w) and add w + c; the decompressor rebuilds the dictionary (special case: w + w[0]) |
Summary
Common Mistakes
i < n − m in brute force skips the last shift (s = n − m); it should be i <= n − m.max(1, …) and the pattern moves back or loops forever.% can return a negative value: if (th < 0) th += Q.Summary
Sample Problems
- Find the LPS table of the pattern "aabaaab".
- How many comparisons does brute force make for T = "aaaaaaaa" (n = 8), P = "aab"?
- Boyer-Moore (bad character only): T = "GCAATGCCTATGTGACC", P = "TATGTG". What is the first shift?
- Base 10, Q = 13, characters are digits: T = "31415", m = 3. Given h("314"), use the rolling formula to find h("141").
- Encode "WWWWBBBWWWWWWB" with RLE and compute the ratio.
- Compress "ABABABA" with LZW using the dictionary A = 0, B = 1.
Show answers
- [0, 1, 0, 1, 2, 2, 3]
- 6 shifts × 3 = 18 ("aa" matches at every shift, 'b' fails); no match.
- s = 0: p[5] = t[5] = 'G' and p[4] = t[4] = 'T' match; p[3] = 'G' ≠ t[3] = 'A' → last['A'] = 1 → 3 − 1 = 2.
- h("314") = 314 mod 13 = 2; h' = (10 · (2 − 3 · 9) + 1) mod 13 = (−249) mod 13 = 11 = 141 mod 13 ✓ (10² mod 13 = 9).
- 4W3B6W1B → 8 characters / 14 ≈ 0.57.
- AB = 2, BA = 3, ABA = 4 are added; output 0 1 2 4 (when decompressing, 4 is the special case: "AB" + "A").
Summary
Exercises
- Modify KMP search to return all occurrences in a list.
- Find the last-occurrence table of the pattern "abcbcabbabcabc" and the first shift in the text "abccbaabccbaabcbcabbabcabc".
- Design a Rabin-Karp method that searches for k patterns of the same length in a single pass.
- Write
rleDecode(String s), an RLE decoder that also handles multi-digit counts correctly. - Decompress the codes 0 1 3 2 3 5 0 (A = 0, B = 1, C = 2) with LZW.
- Give a pair T, P (n = 12, m = 4) that produces brute force's worst case; state the number of comparisons.
Answers / hints
- When q == m, continue with
list.add(i − m + 1); q = lps[q − 1];. - a: 11, b: 12, c: 13. s = 0: p[13] = 'c' ≠ t[13] = 'b' → 13 − 12 = 1.
- Put the pattern hashes in a
HashSet; look up each window hash in the set and verify on a hit: expected O(n + k · m). - Accumulate digits with
Character.isDigitasnum = num · 10 + digit; on a letter, write it num times. - A, B, AB, C, AB, ABC, A → ABABCABABCA.
- T = "aaaaaaaaaaaa", P = "aaab": 9 shifts × 4 = 36 comparisons.
Summary
References
- Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., 2022 (Chapter 32: String Matching).
- Sedgewick and Wayne, Algorithms, 4th ed., 2011 (Section 5.3 Substring Search, 5.5 Data Compression).
- Goodrich, Tamassia and Goldwasser, Data Structures and Algorithms in Java, 6th ed., 2014 (Chapter 13).
- D. E. Knuth, J. H. Morris, V. R. Pratt, "Fast Pattern Matching in Strings", SIAM J. Computing 6(2), 1977.
- R. S. Boyer, J S. Moore, "A Fast String Searching Algorithm", Communications of the ACM 20(10), 1977.
- R. M. Karp, M. O. Rabin, "Efficient Randomized Pattern-Matching Algorithms", IBM J. Res. Dev. 31(2), 1987.
- T. A. Welch, "A Technique for High-Performance Data Compression", IEEE Computer 17(6), 1984.