Chapter 9: String Matching and Compression

Algorithms
Brute forceKMP and the LPS tableBoyer-MooreRabin-Karp ComparisonRLELZW
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Concepts

Define text, pattern, shift, prefix and suffix; state the string matching problem formally.

Matching

Run brute force, KMP, Boyer-Moore and Rabin-Karp step by step on an example and implement them in Java.

Analysis

Explain what the LPS table, the bad-character table and the rolling hash save; compare best, average and worst cases.

Compression

Compress losslessly with RLE and LZW, decompress the result, and compute the compression ratio.

Introduction

String Algorithms

  • Text is everywhere: email, chat messages, social media posts, news articles…
  • Our computers handle enormous amounts of text every day. How do we organize and analyze it?
  • String algorithms perform operations such as searching, replacing and comparing text.
  • Two core problems: string matching (finding a pattern in a text) and string compression (storing text in less space).
Where?

Ctrl+F in editors, grep, search engines, virus signature scanning, plagiarism detection, gene search in DNA, file formats such as ZIP/GIF/PNG.

Introduction

Basic Concepts and Notation

  • Alphabet Σ: the set of characters in use (e.g. {a, b, …, z}, {A, C, G, T}).
  • Text T[0..n−1] and pattern P[0..m−1], usually m ≤ n.
  • P occurs in T with shift s ⇔ T[s..s+m−1] = P. Valid shifts: 0 ≤ s ≤ n − m.
  • Prefix: P[0..k−1]; suffix: P[m−k..m−1]. A proper prefix/suffix is one not equal to the whole string.
  • Example: for "ababd" the prefixes are a, ab, aba, abab; the suffixes are d, bd, abd, babd.
text Tababcababd0123456789pattern Pababdshift s = 5

The string matching problem: find all (or the first) shifts s at which P occurs.

Introduction

Chapter Map

Brute ForceChecks every position of the text for a match with the pattern. Makes the most comparisons.
Knuth-Morris-Pratt (KMP)Builds a table (LPS) up front, then scans the text without ever moving backwards, saving comparisons.
Boyer-MooreCompares right to left; the bad-character and good-suffix rules allow long jumps. Very effective for long patterns.
Rabin-KarpCompares hash values; as the window slides, the hash is updated in O(1). Fast with high probability.
RLE (Run-Length Encoding)Each run of repeated values is stored as a single value plus a repeat count.
LZW (Lempel-Ziv-Welch)Dictionary-based compression: repeated patterns become short codes in a dictionary that grows on the fly (GIF).

Brute Force

The Brute-Force (Naive) Algorithm

  • The simplest way to search for a pattern in a text, hence the name "naive".
  • How it works:
    1. For each shift s = 0, 1, …, n − m, align the pattern under the text.
    2. Compare characters left to right; stop at the first mismatch.
    3. If all m characters match, report s; otherwise slide the pattern one position.
  • No preprocessing and no extra memory.
Complexity

Worst case: (n − m + 1) · m comparisons → O(n · m).

Best case: the first character mismatches at every shift → O(n).

n: text length, m: pattern length.

TAAAAAAs=0AABs=1AABs=2AABs=3AAB4 shifts × 3 = 12 comparisons

Brute Force

Example: "THIS IS A SIMPLE EXAMPLE"

THIS␣IS␣A␣SIMPLE␣EXAMPLESIMPLESIMPLESIMPLESIMPLESIMPLESIMPLESIMPLESIMPLESIMPLESIMPLESIMPLE

Each row is one shift: green = match, red = mismatch. SIMPLE is found at the 11th shift (s = 10) after 18 comparisons in total (n = 24, m = 6).

Brute Force · Step through the code

"LO" in "HELLO WORLD": Code Trace

tHELLO␣WORLD012345678910

n = 11, m = 2. The pattern will be tried at shifts i = 0, 1, …, 9.

tHELLO␣WORLD012345678910pLOi = 0

t[0] = 'H' ≠ p[0] = 'L' → mismatch; slide the pattern one to the right.

tHELLO␣WORLD012345678910pLOi = 1

t[1] = 'E' ≠ p[0] = 'L' → mismatch; slide the pattern one to the right.

tHELLO␣WORLD012345678910pLOi = 2

t[2] = 'L' matches p[0] = 'L' → j++.

tHELLO␣WORLD012345678910pLOi = 2

t[3] = 'L' ≠ p[1] = 'O' → mismatch; slide the pattern one to the right.

tHELLO␣WORLD012345678910pLOi = 3

t[3] = 'L' matches p[0] = 'L' → j++.

tHELLO␣WORLD012345678910pLOi = 3

t[4] = 'O' matches p[1] = 'O' → j++.

tHELLO␣WORLD012345678910pLOi = 3

j == m: pattern found at i = 3. 6 comparisons in total.

int bruteForce(String t, String p) {
  int n = t.length(), m = p.length();
  for (int i = 0; i <= n - m; i++) {
    int j = 0;
    while (j < m &&
      t.charAt(i + j) == p.charAt(j))
      j++;
    if (j == m) return i;
  }
  return -1;
}

Brute Force

Brute Force: Best, Average, Worst

CaseWhen?ComparisonsTime
BestP[0] matches no character of the text (e.g. P = "xyz", T = "aaaa…")n − m + 1O(n)
AverageRandom text, large alphabet: a mismatch usually comes within the first 1–2 characters≈ n · σ/(σ−1)O(n)
WorstT = "aaaaaaaaaa", P = "aaab": m comparisons at every shift(n − m + 1) · mO(n · m)
Numeric example: n = 10, m = 4, T = a¹⁰, P = aaab → 7 shifts × 4 = 28 comparisons. The mismatch always comes at the last character.
The real flaw: what a mismatch teaches us (the characters that matched) is thrown away; the algorithm backs up in the text and rereads the same characters. KMP puts this information to use.

KMP

The Knuth-Morris-Pratt (KMP) Algorithm

  • Finds occurrences of a pattern in a text.
  • Developed by Donald Knuth and Vaughan Pratt and, independently, by James H. Morris; published jointly in 1977.
  • Key idea: on a mismatch, use what we already know about the part of the pattern that has matched; the text pointer never moves back.
  • That knowledge is precomputed from the pattern and stored in the LPS table (the prefix function π).
Complexity

Preprocessing: O(m)

Search: O(n)

Total: O(n + m), even in the worst case

Extra memory: O(m)

KMP

KMP: How It Works

1. PreprocessingFor each position of the pattern, record the length of the longest proper prefix that is also a suffix (LPS). On a mismatch, this table tells us where to resume in the pattern.
2. Pattern searchScan the text once, left to right. On a mismatch, the pattern pointer q falls back to the value read from the table (q = lps[q − 1]); the text pointer i stays put.
3. Match checkWhen q reaches m, every character of the pattern has matched: report position i − m + 1. To find all occurrences, continue with q = lps[m − 1].
TABABABC0123456PABABCq = 4:C ≠ AABABCq = lps[3] = 2"AB" is notrecheckedi = 4 stays put
Why is this correct? If P[0..q−1] has matched the text and P[q] mismatches, the next possible alignment is the one that overlaps the longest prefix of P[0..q−1] that is also its suffix. No shift in between can possibly match.

KMP

The LPS Table (Prefix Function)

lps[q] = length of the longest proper prefix of P[0..q] that is also a suffix of P[0..q].

  • lps[0] = 0 (a single character has no proper prefix).
  • Example: text "ababcababcabababd", pattern "ababd".
  • Pattern table: a: 0, b: 0, a: 1, b: 2, d: 0.
qP[q]lps[q]Reason (P[0..q])
0a0"a" — no proper prefix
1b0"ab" — a ≠ b
2a1"aba" — "a" is both prefix and suffix
3b2"abab" — "ab" is both prefix and suffix
4d0"ababd" — no prefix ends in d

KMP · Step through the code

Building the LPS Table: "ACABACACD"

PlpsACABACACD0123456780

lps[0] = 0 always. k = 0: length of the prefix matched so far.

PlpsACABACACD01234567800q

p[0] = 'A' ≠ p[1] = 'C' → k = 0, lps[1] = 0.

PlpsACABACACD012345678001qk

p[0] = 'A' = p[2] = 'A' → k = 1, lps[2] = 1.

PlpsACABACACD012345678001qk

p[1] = 'C' ≠ p[3] = 'B' → fall back: k = lps[0] = 0.

PlpsACABACACD0123456780010q

p[0] = 'A' ≠ p[3] = 'B' → k = 0, lps[3] = 0.

PlpsACABACACD01234567800101qk

p[0] = 'A' = p[4] = 'A' → k = 1, lps[4] = 1.

PlpsACABACACD012345678001012qk

p[1] = 'C' = p[5] = 'C' → k = 2, lps[5] = 2. "AC" is both prefix and suffix.

PlpsACABACACD0123456780010123qk

p[2] = 'A' = p[6] = 'A' → k = 3, lps[6] = 3. "ACA" is both prefix and suffix.

PlpsACABACACD0123456780010123qk

p[3] = 'B' ≠ p[7] = 'C' → fall back: k = lps[2] = 1.

PlpsACABACACD01234567800101232qk

p[1] = 'C' = p[7] = 'C' → k = 2, lps[7] = 2. "AC" is both prefix and suffix.

PlpsACABACACD01234567800101232qk

p[2] = 'A' ≠ p[8] = 'D' → fall back: k = lps[1] = 0.

PlpsACABACACD012345678001012320q

p[0] = 'A' ≠ p[8] = 'D' → k = 0, lps[8] = 0.

PlpsACABACACD012345678001012320

Result: lps = [0, 0, 1, 0, 1, 2, 3, 2, 0]. Constant amortized work per q → O(m).

int[] computeLPS(String p) {
  int m = p.length(), k = 0;
  int[] lps = new int[m];
  for (int q = 1; q < m; q++) {
    while (k > 0 &&
      p.charAt(k) != p.charAt(q))
      k = lps[k - 1];
    if (p.charAt(k) == p.charAt(q))
      k++;
    lps[q] = k;
  }
  return lps;
}

KMP

The Three Phases of a KMP Search

Phase 1 · Find the first characterAdvance the text pointer (i) until it matches the first symbol of the pattern.
Phase 2 · Advance togetherWhile symbols match, both pointers advance. On reaching the end of the pattern, return (text position − pattern position).
Phase 3 · Fall back via LPSOn a mismatch only the pattern pointer (q) falls back via LPS; after a match or on reaching 0, return to Phase 2 (Phase 1 if q = 0).
123matchmismatchq>0q=0

KMP

KMP Example: Searching for "ABCDABD"

lps("ABCDABD") = [0, 0, 0, 0, 1, 2, 0]. green = match, red = mismatch, blue = known from LPS (not compared again).

ABCABCDABABCDABCDABDE01234567891011121314151617181920ABCDABD

s = 0: ABC matched, t[3] = 'A' ≠ p[3] = 'D'. q = 3 → q = lps[2] = 0; the pattern slides 3 and t[3] is compared with p[0].

ABCABCDABABCDABCDABDE01234567891011121314151617181920ABCDABD

s = 3: ABCDAB matched, t[9] = 'A' ≠ 'D'. q = 6 → q = lps[5] = 2: "AB" is known to match; the pattern slides 4.

ABCABCDABABCDABCDABDE01234567891011121314151617181920ABCDABD

s = 7: AB is not compared again. t[9] = 'A' ≠ p[2] = 'C' → q = lps[1] = 0.

ABCABCDABABCDABCDABDE01234567891011121314151617181920ABCDABD

s = 9: ABCDAB matched, t[15] = 'C' ≠ 'D'. q = 6 → q = lps[5] = 2; the pattern slides 4.

ABCABCDABABCDABCDABDE01234567891011121314151617181920ABCDABD

s = 13: AB is known; CDABD matches too → pattern found at position 13. The text pointer never moved back.

24 character comparisons in total (n = 21); brute force makes 37 on the same text.

KMP · Step through the code

kmpSearch("ababcababcabababd", "ababd")

tababcababcabababd012345678910111213141516lps = [0, 0, 1, 2, 0]

n = 17, m = 5, lps = [0, 0, 1, 2, 0], q = 0.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[0] = t[0] = 'a' → q = 1.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[1] = t[1] = 'b' → q = 2.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[2] = t[2] = 'a' → q = 3.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[3] = t[3] = 'b' → q = 4.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[4] = 'd' ≠ t[4] = 'c' → q = lps[3] = 2 (i stays put).

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[2] = 'a' ≠ t[4] = 'c' → q = lps[1] = 0 (i stays put).

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[0] = 'a' ≠ t[4] = 'c' → q stays 0, i advances.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[0] = t[5] = 'a' → q = 1.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[1] = t[6] = 'b' → q = 2.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[2] = t[7] = 'a' → q = 3.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[3] = t[8] = 'b' → q = 4.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[4] = 'd' ≠ t[9] = 'c' → q = lps[3] = 2 (i stays put).

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[2] = 'a' ≠ t[9] = 'c' → q = lps[1] = 0 (i stays put).

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[0] = 'a' ≠ t[9] = 'c' → q stays 0, i advances.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[0] = t[10] = 'a' → q = 1.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[1] = t[11] = 'b' → q = 2.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[2] = t[12] = 'a' → q = 3.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[3] = t[13] = 'b' → q = 4.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[4] = 'd' ≠ t[14] = 'a' → q = lps[3] = 2 (i stays put).

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[2] = t[14] = 'a' → q = 3.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[3] = t[15] = 'b' → q = 4.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

p[4] = t[16] = 'd' → q = 5.

tababcababcabababd012345678910111213141516pababdilps = [0, 0, 1, 2, 0]

q == m → i − m + 1 = 12 is returned. The text was scanned only once.

int kmpSearch(String t, String p) {
  int n = t.length(), m = p.length();
  int[] lps = computeLPS(p);
  int q = 0;  // matched length
  for (int i = 0; i < n; i++) {
    while (q > 0 &&
      p.charAt(q) != t.charAt(i))
      q = lps[q - 1];
    if (p.charAt(q) == t.charAt(i))
      q++;
    if (q == m)
      return i - m + 1;
  }
  return -1;
}

KMP

KMP: Why O(n + m)?

  • The outer loop increments i n times; i never decreases.
  • q grows by at most 1 per step → total increase ≤ n.
  • Each fallback in the while loop decreases q by at least 1; q cannot go negative → total fallbacks ≤ total increase ≤ n.
  • Search: at most 2n comparisons → O(n). The same amortized argument gives O(m) for LPS.

Pros

  • Linear even in the worst case
  • Never backs up in the text: suits streaming data

Cons

  • Usually slower than Boyer-Moore in practice
  • Little to skip over with a small alphabet (DNA)

Boyer-Moore

The Boyer-Moore Algorithm

  • Searches for a pattern in a text; developed by Robert S. Boyer and J Strother Moore in 1977.
  • The pattern slides left to right, but characters are compared right to left.
  • On a mismatch, shift by the larger of two rules: bad character and good suffix.
  • Most text characters can be skipped without ever being read: very fast for long patterns over large alphabets (natural language); grep and editor search belong to this family.
Complexity

Best: O(n/m) — the first comparison fails at every attempt and we skip m.

Average: sublinear, faster than O(n) in practice.

Worst: O(n · m) with bad character alone; O(n + m) with good suffix + Galil's rule.

Boyer-Moore

Boyer-Moore: How It Works

  1. Preprocessing: build shift tables from the pattern: the last occurrence of each character in the pattern (bad character) and a shift amount for each suffix (good suffix).
  2. Search: align the pattern under the text; compare starting from the last character and moving left.
  3. Match check: if all characters match, report the position.
  4. Bad-character rule: shift according to the rightmost position in the pattern of the mismatched text character.
  5. Good-suffix rule: shift according to whether the matched suffix occurs elsewhere in the pattern.
TABDCABDCABDEs=0ABDE1no C → shift 4s=4ABDE2no C → shift 4s=8ABDE3456found

Red numbers give the comparison order: only 6 comparisons on a 12-character text.

Shift = max(bad-character shift, good-suffix shift). Both are "safe": no skipped alignment can contain a match.

Boyer-Moore

The Bad-Character Rule

patternabacab012345a4b5c3*−1last-occurrence table
  • Let p[j] ≠ t[s + j]; c = t[s + j] is the bad character.
  • If c occurs in the pattern left of j: shift so that the rightmost c lines up under t[s + j].
  • If c does not occur in the pattern (last = −1): shift the pattern entirely past c (j + 1).
  • Formula: s += max(1, j − last[c]). If c last occurs right of j, then j − last[c] ≤ 0; hence the minimum of 1.
int[] lastOccurrence(String p) {
  int[] last = new int[256];
  Arrays.fill(last, -1);
  for (int i = 0; i < p.length(); i++)
    last[p.charAt(i)] = i;
  return last;
}

Boyer-Moore

The Good-Suffix Rule

  • The suffix u = P[j+1..m−1] matched the text before the mismatch: the "good suffix".
  • Case 1: if u occurs elsewhere in the pattern (preceded by a different character), shift so that this occurrence lines up under u in the text.
  • Case 2: otherwise, align the longest prefix of P that equals a suffix of u.
  • Case 3: if neither exists, shift the pattern by m.
  • The table is precomputed in O(m) time (similar to KMP's LPS idea, but for suffixes).
tBBAABCABAB0123456789beforeCABABafterCABAB

"AB" matched, t[2] = 'A' ≠ p[2] = 'B'. Bad character: last[A] = 3 > j → only 1. Good suffix: "AB" also occurs at p[1..2] (preceded by 'C' ≠ 'B') → shift 2.

Boyer-Moore

Example: "HELLO" in "LO LOELLO O HELLO"

cHELOother
last[c]0134−1

Rule: s += max(1, j − last[c]). green = match, red = bad character.

LO␣LOELLO␣O␣HELLO012345678910111213141516HELLO

Alignment 1 (s = 0): 2 characters matched, t[2] = '␣' ≠ p[2] = 'L'. '␣' is not in the pattern → j − (−1) = 3 → shift 3.

LO␣LOELLO␣O␣HELLO012345678910111213141516HELLO

Alignment 2 (s = 3): t[7] = 'L' ≠ p[4] = 'O'. last['L'] = 3 → j − 3 = 1 → shift 1.

LO␣LOELLO␣O␣HELLO012345678910111213141516HELLO

Alignment 3 (s = 4): 4 characters matched, t[4] = 'O' ≠ p[0] = 'H'. last['O'] = 4 ≥ j → at least 1 → shift 1.

LO␣LOELLO␣O␣HELLO012345678910111213141516HELLO

Alignment 4 (s = 5): t[9] = '␣' ≠ p[4] = 'O'. '␣' is not in the pattern → j − (−1) = 5 → shift 5.

LO␣LOELLO␣O␣HELLO012345678910111213141516HELLO

Alignment 5 (s = 10): t[14] = 'L' ≠ p[4] = 'O'. last['L'] = 3 → j − 3 = 1 → shift 1.

LO␣LOELLO␣O␣HELLO012345678910111213141516HELLO

Alignment 6 (s = 11): t[15] = 'L' ≠ p[4] = 'O'. last['L'] = 3 → j − 3 = 1 → shift 1.

LO␣LOELLO␣O␣HELLO012345678910111213141516HELLO

Alignment 7 (s = 12): all 5 characters match right to left → HELLO found at s = 12. 7 alignments, 17 comparisons in total.

Boyer-Moore

Horspool's Simplification and the Shift Table

  • Horspool (1980): wherever the mismatch occurs, shift according to the text character under the last position of the window.
  • Shift table: shift[c] = m − 1 − (rightmost position of c in P[0..m−2]); m if c does not occur there.
  • HELLO: H → 4, E → 3, L → 1, O → 5 (the last character is excluded; O occurs nowhere else), other → 5.
  • The same example needs only 5 alignments (7 with the basic bad-character rule).
AlignmentsComparisonlast charshift
10t[2] = '␣' ≠ p[2]'O'5
25t[9] = '␣' ≠ p[4]'␣'5
310t[14] = 'L' ≠ p[4]'L'1
411t[15] = 'L' ≠ p[4]'L'1
512all match—found

Horspool · Step through the code

horspool("LO LOELLO O HELLO", "HELLO")

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16shiftH5E5L5O5other5

n = 17, m = 5. All 256 cells of shift start at m = 5: on a character not in the pattern, the pattern jumps entirely past it.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16shiftH4E3L1O5other5

Scan P[0..3], excluding the last character: H → 4, E → 3, L first 2 then 1 (the rightmost occurrence wins). O occurs only at the end, so it stays 5.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOs = 0shiftH4E3L1O5other5

s = 0: 2 characters match from the right, then t[2] = '␣' ≠ p[2] = 'L' → mismatch.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOHELLO→shiftH4E3L1O5other5

The shift looks at the last character of the window, not at the mismatch: t[4] = 'O' → shift = 5 → s = 5.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOs = 5shiftH4E3L1O5other5

s = 5: t[9] = '␣' ≠ p[4] = 'O' → mismatch.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOHELLO→shiftH4E3L1O5other5

The window's last character t[9] = '␣' is not in the pattern → shift = 5: the pattern jumps entirely past it, s = 10.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOs = 10shiftH4E3L1O5other5

s = 10: t[14] = 'L' ≠ p[4] = 'O' → mismatch.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOHELLO→shiftH4E3L1O5other5

The shift looks at the last character of the window, not at the mismatch: t[14] = 'L' → shift = 1 → s = 11.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOs = 11shiftH4E3L1O5other5

s = 11: t[15] = 'L' ≠ p[4] = 'O' → mismatch.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOHELLO→shiftH4E3L1O5other5

The shift looks at the last character of the window, not at the mismatch: t[15] = 'L' → shift = 1 → s = 12.

tL0O1␣2L3O4E5L6L7O8␣9O10␣11H12E13L14L15O16pHELLOs = 12shiftH4E3L1O5other5

All five characters match right to left, j = −1 → return s = 12. 5 alignments, 11 comparisons in total.

int horspool(String t, String p) {
  int n = t.length(), m = p.length();
  int[] shift = new int[256];
  Arrays.fill(shift, m);
  for (int i = 0; i < m - 1; i++)
    shift[p.charAt(i)] = m - 1 - i;
  int s = 0;
  while (s <= n - m) {
    int j = m - 1;
    while (j >= 0 &&
      p.charAt(j) == t.charAt(s + j))
      j--;
    if (j < 0) return s;
    s += shift[t.charAt(s + m - 1)];
  }
  return -1;
}

Boyer-Moore

Example: Searching with the Last-Occurrence Table

Pattern abacab, last occurrence: a = 4, b = 5, c = 3, * = −1. The number above a cell is the comparison counter.

abacaabadcabacabaabb0123456789101112131415161718191abacab

Alignment 1 (s = 0): t[5] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.

abacaabadcabacabaabb012345678910111213141516171819234abacab

Alignment 2 (s = 1): t[4] = 'a' ≠ p[3] = 'c'. last['a'] = 4 ≥ j = 3 → 1 → shift 1.

abacaabadcabacabaabb0123456789101112131415161718195abacab

Alignment 3 (s = 2): t[7] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.

abacaabadcabacabaabb0123456789101112131415161718196abacab

Alignment 4 (s = 3): t[8] = 'd' ≠ p[5] = 'b'. 'd' is not in the pattern → j + 1 = 6 → shift 6.

abacaabadcabacabaabb0123456789101112131415161718197abacab

Alignment 5 (s = 9): t[14] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.

abacaabadcabacabaabb0123456789101112131415161718198910111213abacab

Alignment 6 (s = 10): all 6 comparisons match → match at s = 10. We want all occurrences, so shift 1 and continue.

abacaabadcabacabaabb01234567891011121314151617181914abacab

Alignment 7 (s = 11): t[16] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.

abacaabadcabacabaabb01234567891011121314151617181915abacab

Alignment 8 (s = 12): t[17] = 'a' ≠ p[5] = 'b'. j − last['a'] = 5 − 4 = 1 → shift 1.

abacaabadcabacabaabb012345678910111213141516171819161718abacab

Alignment 9 (s = 13): t[16] = 'a' ≠ p[3] = 'c'. last['a'] = 4 ≥ j = 3 → 1 → shift 1.

abacaabadcabacabaabb0123456789101112131415161718191920abacab

Alignment 10 (s = 14): t[18] = 'b' ≠ p[4] = 'a'. last['b'] = 5 ≥ j = 4 → 1 → shift 1.

10 alignments, 20 comparisons in total (n = 20, m = 6): a character not in the pattern, such as 'd', yields a 6-position jump.

Boyer-Moore · Step through the code

boyerMoore("merhaba naber", "abe")

tmerhaba␣naber0123456789101112last: a = 0, b = 1, e = 2, other = −1

n = 13, m = 3. Last-occurrence table built; s = 0.

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

p[2] = 'e' ≠ t[2] = 'r'; last['r'] = -1 → s += max(1, 2 − (-1)) = 3.

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

p[2] = 'e' ≠ t[5] = 'b'; last['b'] = 1 → s += max(1, 2 − (1)) = 1.

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

p[2] = 'e' ≠ t[6] = 'a'; last['a'] = 0 → s += max(1, 2 − (0)) = 2.

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

p[2] = 'e' ≠ t[8] = 'n'; last['n'] = -1 → s += max(1, 2 − (-1)) = 3.

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

p[2] = t[11] = 'e' → j-- .

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

p[1] = t[10] = 'b' → j-- .

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

p[0] = t[9] = 'a' → j-- .

tmerhaba␣naber0123456789101112pabeslast: a = 0, b = 1, e = 2, other = −1

j < 0 → pattern found at s = 9 ("naber").

int boyerMoore(String t, String p) {
  int n = t.length(), m = p.length();
  int[] last = lastOccurrence(p);
  int s = 0;
  while (s <= n - m) {
    int j = m - 1;
    while (j >= 0 &&
      p.charAt(j) == t.charAt(s + j))
      j--;
    if (j < 0) return s;
    s += Math.max(1,
      j - last[t.charAt(s + j)]);
  }
  return -1;
}

Boyer-Moore

Boyer-Moore: Analysis and Example

  • Best: text characters do not occur in the pattern → 1 comparison per alignment, skip m: ≈ n/m comparisons.
  • Worst (bad character only): T = aaaa…a, P = baaa…a → m comparisons per alignment, shift 1: O(n · m).
  • With a large alphabet and frequent mismatches, longer patterns give longer shifts (best ≈ n/m).
  • With a small alphabet (DNA, binary data) the bad-character rule is weak; the good-suffix rule matters more.
Example

Text: abccbaabccbaabcbcabbabcabc

Pattern: abcbcabbabcabc (m = 14)

Last-occurrence table: a: 11, b: 12, c: 13

Result: the pattern is found at position 12.

Rabin-Karp

The Rabin-Karp Algorithm

  • Finds occurrences of a pattern in a text; developed by Michael O. Rabin and Richard M. Karp in 1987.
  • Instead of comparing characters one by one, it compares the hash of the pattern with the hash of each length-m window of the text.
  • Rolling hash: when the window slides one step right, the new hash is computed from the old one in O(1).
  • Equal hashes do not guarantee a match: a spurious hit is possible → verify character by character.
Complexity

Best: hashes rarely collide → O(n + m)

Average: O(n + m) in practice

Worst: a collision (spurious hit) at every window, constant verification → O(n · m)

Rabin-Karp

Rabin-Karp: How It Works and the Rolling Hash

  1. Hash: compute the hash of the pattern and of the first text window.
  2. Match check: substrings whose hash matches are candidates.
  3. Verification: compare a candidate window with the pattern character by character.
  4. Slide and rehash: if there is no match, slide the window one step right and update the hash.
  5. Repeat: continue across the whole text.
Polynomial hash (base B, prime modulus Q):
h(t[s..s+m−1]) = (t[s]·Bm−1 + t[s+1]·Bm−2 + … + t[s+m−1]) mod Q
Slide: hs+1 = (B · (hs − t[s] · Bm−1) + t[s+m]) mod Q — subtract the outgoing character, shift the rest one digit left, add the incoming character.

Rabin-Karp

A Simple Example: Sum Hash and Spurious Hits

Toy hash: the sum of the letter values. Searching for "TS" in "VUATS".

VUATS5 + 1 = 6pattern "TS"h = 2 + 3 = 5U=1 T=2 S=3 A=4 V=5

Window "VU": 5 + 1 = 6 ≠ 5 → hashes differ, no character comparison.

VUATS1 + 4 = 5pattern "TS"h = 2 + 3 = 5U=1 T=2 S=3 A=4 V=5

Window "UA": 1 + 4 = 5 = 5 → equal hashes, but "UA" ≠ "TS": a spurious hit; the verification is wasted.

VUATS4 + 2 = 6pattern "TS"h = 2 + 3 = 5U=1 T=2 S=3 A=4 V=5

Window "AT": 4 + 2 = 6 ≠ 5 → skip. New hash = old − outgoing (U=1) + incoming (T=2): O(1).

VUATS2 + 3 = 5pattern "TS"h = 2 + 3 = 5U=1 T=2 S=3 A=4 V=5

Window "TS": 2 + 3 = 5 → equal hashes, and the characters agree too: match (s = 3).

The sum hash is weak (it ignores letter order: "ST" also gives 5). That is why a position-weighted polynomial hash with a large prime Q is used.

Rabin-Karp · Step through the code

rabinKarp("abracadabra", "cad")

tabracadabra012345678910a=97 b=98 c=99 d=100 r=114

n = 11, m = 3, B = 256, Q = 101. h = 256² mod 101 = 88 (weight of the outgoing character).

tabracadabra012345678910window s = 0ph = h("cad") = 11, th = h("abr") = 4

ph = (99·256² + 97·256 + 100) mod 101 = 11; for the first window "abr", th = 4.

tabracadabra012345678910window s = 0th("abr") = 4 ≠ ph = 11

Window "abr": th = 4 ≠ 11 → characters not checked. Slide: 'a' out, 'a' in → new th = 30.

tabracadabra012345678910window s = 1th("bra") = 30 ≠ ph = 11

Window "bra": th = 30 ≠ 11 → characters not checked. Slide: 'b' out, 'c' in → new th = 17.

tabracadabra012345678910window s = 2th("rac") = 17 ≠ ph = 11

Window "rac": th = 17 ≠ 11 → characters not checked. Slide: 'r' out, 'a' in → new th = 41.

tabracadabra012345678910window s = 3th("aca") = 41 ≠ ph = 11

Window "aca": th = 41 ≠ 11 → characters not checked. Slide: 'a' out, 'd' in → new th = 11.

tabracadabra012345678910window s = 4th = 11 = ph → compare characters

th = 11 = ph → character check with matchAt: "cad" = "cad" → return s = 4.

int rabinKarp(String t, String p) {
  int n = t.length(), m = p.length();
  int B = 256, Q = 101, h = 1;
  for (int i = 0; i < m - 1; i++)
    h = (h * B) % Q;  // B^(m-1) mod Q
  int ph = 0, th = 0;
  for (int i = 0; i < m; i++) {
    ph = (B * ph + p.charAt(i)) % Q;
    th = (B * th + t.charAt(i)) % Q;
  }
  for (int s = 0; s <= n - m; s++) {
    if (ph == th && matchAt(t, p, s))
      return s;
    if (s < n - m) {
      th = (B * (th - t.charAt(s) * h)
            + t.charAt(s + m)) % Q;
      if (th < 0) th += Q;
    }
  }
  return -1;
}

Rabin-Karp

The Sliding Window and Analysis

Index 0abcdefabcIndex 1abcdefabcIndex 2abcdefabcIndex 3abcdefabcIndex 4abcdefabcIndex 5abcdefabcIndex 6abcdefabc

n − m + 1 = 7 windows; each computed from the previous one in O(1).

  • Hashing cost: O(m) + (n − m) · O(1).
  • With a large prime Q, a spurious hit has probability ≈ 1/Q → expected time O(n + m).
  • Bad Q or adversarial input: every window is a candidate → O(n · m).
  • Strength: many patterns of the same length can be searched in one pass (put their hashes in a set): plagiarism detection, document similarity.
  • It also generalizes to 2D pattern search (a block within an image).

Matching Summary

Comparing String Matching Algorithms

AlgorithmKey ideaPreprocessingAverageWorstExtra memory
Brute forceTries every positionnoneO(n)*O(n · m)O(1)
KMPLPS table, never backs up in the textO(m)O(n + m)O(n + m)O(m)
Boyer-MooreRight to left, bad character + good suffixO(m + σ)sublinearO(n · m)**O(m + σ)
Rabin-KarpCompares rolling hashesO(m)O(n + m)O(n · m)O(1)

* for random text over a large alphabet; on regular (repetitive) text it approaches O(n · m). ** with the bad-character rule alone; O(n + m) with good suffix + Galil's rule. σ: alphabet size.

Need a guaranteeKMP: streaming data, where the worst-case bound matters.
Long pattern, natural languageBoyer-Moore / Horspool: fastest in practice.
Many patternsRabin-Karp: many patterns in a single pass.

Compression

String Compression Algorithms

  • Goal: store or transmit text (or data) using fewer bits.
  • Lossless compression: decompression gives back the original data exactly. RLE and LZW are lossless (text, programs, PNG/GIF).
  • Lossy compression: perceptually unimportant detail is discarded (JPEG, MP3); not suitable for text.
  • Compression ratio = compressed size / original size (smaller is better).
RLEConsecutive repeats of the same value are stored as one value + a count.
LZWRepeated patterns become short codes in a dictionary that grows on the fly.

RLE

Run-Length Encoding (RLE)

  • Run-length encoding replaces each sequence of repeated characters (a run) with a count + character pair.
  • Steps: scan the text left to right → count consecutive repeats → combine count and character → store the result as the compressed text.
  • The text is scanned once, so the time is O(n).
  • Simple and effective, but useless or even harmful when there is little repetition: "ABCD" → "1A1B1C1D" (twice the size!).
  • Used in: fax (black-and-white lines), BMP/PCX/TIFF, simple icons.
Example

Text: AAAAABBBCCCCDDD (15 characters)

AAAAA5ABBB3BCCCC4CDDD3D

Compressed: 5A3B4C3D (8 characters)

Ratio: 8/15 ≈ 0.53

RLE · Step through the code

rleCompress("AAAAABBBCCCCDDD")

AAAAABBBCCCCDDD01234567891011121314

n = 15, result is empty.

AAAAABBBCCCCDDD01234567891011121314i

New run: c = 'A', count = 1 (i = 0).

AAAAABBBCCCCDDD01234567891011121314i

The while loop ran 4 times: count = 5, i = 4. Append "5A" to result.

AAAAABBBCCCCDDD01234567891011121314i

New run: c = 'B', count = 1 (i = 5).

AAAAABBBCCCCDDD01234567891011121314i

The while loop ran 2 times: count = 3, i = 7. Append "3B" to result.

AAAAABBBCCCCDDD01234567891011121314i

New run: c = 'C', count = 1 (i = 8).

AAAAABBBCCCCDDD01234567891011121314i

The while loop ran 3 times: count = 4, i = 11. Append "4C" to result.

AAAAABBBCCCCDDD01234567891011121314i

New run: c = 'D', count = 1 (i = 12).

AAAAABBBCCCCDDD01234567891011121314i

The while loop ran 2 times: count = 3, i = 14. Append "3D" to result.

AAAAABBBCCCCDDD01234567891011121314

Result: "5A3B4C3D" — 8 characters instead of 15.

String rleCompress(String s) {
  StringBuilder result = new StringBuilder();
  int n = s.length();
  for (int i = 0; i < n; i++) {
    char c = s.charAt(i);
    int count = 1;
    while (i + 1 < n &&
      s.charAt(i + 1) == c) {
      count++;
      i++;
    }
    result.append(count).append(c);
  }
  return result.toString();
}

RLE

Byte-Level RLE Encoding

Text: aaaaaaaaaabbbbbbececececececdddddddddddddddecb (46 bytes)

Count–character pairs(10, a)(6, b)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(1, e)(1, c)(15, d)(1, e)(1, c)(1, b)
RLE without flag
(2 bytes per run)
10 97 06 98 01 101 01 99 01 101 01 99 01 101 01 99 01 101 01 99 01 101 01 99 01 101 01 99 15 100 01 101 01 99 01 98 → 36 bytes
RLE with flag
(flag = 255)
255 10 97 255 06 98 101 99 101 99 101 99 101 99 101 99 101 99 255 15 100 101 99 98 → 24 bytes
In plain RLE, non-repeating stretches such as "ececec…" cost 2 bytes per character. RLE with a flag writes only long runs as a 255 count char triple and leaves other characters as they are (the value 255 itself must be escaped).

RLE

RLE on a Binary (Black-and-White) Image

W1B1W3B1W1B3W1B3B7W1B5W1W2B3W2W3B1W3
  • Each row is scanned left to right: W = white, B = black, followed by the run length.
  • 42 pixels → 18 runs (each row is encoded separately). With only two colors, the color need not be written at all: runs simply alternate white/black (the fax standard uses this idea).
  • Very effective for images with large flat areas (drawings, scanned text); useless for photographs.

RLE

Pixel-Level RLE (RGB)

Blue (0, 0, 255), black (0, 0, 0), yellow (255, 255, 0). Raw: 64 × 3 = 192 bytes. RLE: 33 runs × (1 count + 3 color) = 132 bytes.

CountR G BCountR G BCountR G B
20 0 25540 0 030 0 255
10 0 04255 255 010 0 0
10 0 25510 0 01255 255 0
10 0 02255 255 010 0 0
1255 255 020 0 06255 255 0
20 0 01255 255 010 0 0
2255 255 010 0 01255 255 0
20 0 02255 255 020 0 0
2255 255 010 0 010 0 255
10 0 04255 255 010 0 0
30 0 25540 0 020 0 255

Read the table row by row (left to right, then the next row); if the same color continues at the end of an image row, the run carries over to the next row (e.g. 3 blue).

LZW

Lempel-Ziv-Welch (LZW) Compression

  • Dictionary-based lossless compression: Terry Welch (1984) refined Lempel and Ziv's LZ78 method (1978).
  • Identifies repeated strings and represents them by short codes (indices) into a dictionary.
  • The dictionary is dynamic: it grows during compression and is never transmitted; the decompressor rebuilds the same dictionary on its own.
  • Used in formats such as GIF, TIFF, PDF and Unix compress.
  • As the dictionary grows, code width (bits) and memory grow and performance drops: in practice the dictionary is capped (GIF: 4096 codes = 12 bits) and reset when full.
Example

Text: ABABCABABCA

Initial dictionary: A = 0, B = 1, C = 2

A0B1AB3C2AB3ABC5A0

Compressed: 0 1 3 2 3 5 0

7 codes instead of 11 symbols.

LZW · Step through the code

lzw("ABABCABABCA")

ABABCABABCA0123456789100A1B2C

The dictionary starts with the characters of the text: A = 0, B = 1, C = 2. w = "".

ABABCABABCA012345678910c0A1B2C

"A" is in dict (0) → w = "A"; look for a longer match.

ABABCABABCA012345678910c0A1B2C3AB

"AB" not in dict → output code of w = "A": 0; add "AB" = 3 to dict; w = "B".

ABABCABABCA012345678910c0A1B2C3AB4BA

"BA" not in dict → output code of w = "B": 1; add "BA" = 4 to dict; w = "A".

ABABCABABCA012345678910c0A1B2C3AB4BA

"AB" is in dict (3) → w = "AB"; look for a longer match.

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC

"ABC" not in dict → output code of w = "AB": 3; add "ABC" = 5 to dict; w = "C".

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC6CA

"CA" not in dict → output code of w = "C": 2; add "CA" = 6 to dict; w = "A".

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC6CA

"AB" is in dict (3) → w = "AB"; look for a longer match.

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC6CA7ABA

"ABA" not in dict → output code of w = "AB": 3; add "ABA" = 7 to dict; w = "A".

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC6CA7ABA

"AB" is in dict (3) → w = "AB"; look for a longer match.

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC6CA7ABA

"ABC" is in dict (5) → w = "ABC"; look for a longer match.

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC6CA7ABA8ABCA

"ABCA" not in dict → output code of w = "ABC": 5; add "ABCA" = 8 to dict; w = "A".

ABABCABABCA012345678910c0A1B2C3AB4BA5ABC6CA7ABA8ABCA

Loop done: output the code of the leftover w = "A", 0. Output: 0 1 3 2 3 5 0.

List<Integer> lzw(String s) {
  Map<String, Integer> dict =
      initDict(s);  // A=0, B=1, C=2
  String w = "";
  List<Integer> codes = new ArrayList<>();
  for (char c : s.toCharArray()) {
    String wc = w + c;
    if (dict.containsKey(wc)) {
      w = wc;
    } else {
      codes.add(dict.get(w));
      dict.put(wc, dict.size());
      w = "" + c;
    }
  }
  if (!w.isEmpty())
    codes.add(dict.get(w));
  return codes;
}

LZW

LZW Decompression

  • The decompressor knows only the initial dictionary; it builds the rest itself, in the same order as the compressor.
  • For each new code k: write e = dict[k]; add w + e[0] to the dictionary; set w = e.
  • The decompressor lags one step behind. If k is not yet defined (k = the next code), e must be w + w[0] (the cScSc case).
  • Output: A·B·AB·C·AB·ABC·A = ABABCABABCA ✓
CodeOutputAdded
0A—
1B3 = AB
3AB4 = BA
2C5 = ABC
3AB6 = CA
5ABC7 = ABA
0A8 = ABCA

LZW

LZW Example: A DNA Sequence

Text: ATGATCATGAG (11 symbols)

Initial: A = 0, T = 1, G = 2, C = 3

Output: 0 1 2 4 3 4 6 2 (8 codes)

  • Each row: the new string added to the dictionary and the code output at that moment.
  • The second time "AT" appears, it is written as a single code (4); likewise "GA" (6).
  • The final w = "G" is output after the loop (2).
CodeAdded to dictionaryOutput
4AT0
5TG1
6GA2
7ATC4
8CA3
9ATG4
10GAG6
—(final w = G)2

LZW

LZW with an ASCII Dictionary and the Special Case

  • In practice the initial dictionary holds the 256 single-byte values (0–255; the first 128 are ASCII; a = 97, b = 98, c = 99); new strings are numbered from 256.
  • Text: aabaacababacb (13 characters) → 9 codes: 97 97 98 256 99 257 261 99 98.
  • "aba" = 261 is added while "ab" (257) is written and is used right in the next code.
  • When the decompressor reads 261, its dictionary only goes up to 260 → special case: e = w + w[0] = "ab" + "a" = "aba".
OutputAdded to dictionary
97256 = aa
97257 = ab
98258 = ba
256259 = aac
99260 = ca
257261 = aba
261262 = abac
99263 = cb
98(final w = b)

LZW · Step through the code

lzwDecode([97 97 98 256 99 257 261 99 98])

code970971982256399425752616997988sadict256 and up: still empty

The decompressor knows only the 256 single-byte values. The first code, 97, is always a single character: w = "a", output "a". Nothing from 256 up exists yet.

code970971982256399425752616997988saadict256aa

k = 97 (ASCII) → write e = "a"; add w + e[0] = "a" + "a" = 256 = "aa" to dict.

code970971982256399425752616997988saabdict256aa257ab

k = 98 (ASCII) → write e = "b"; add w + e[0] = "a" + "b" = 257 = "ab" to dict.

code970971982256399425752616997988saabaadict256aa257ab258ba

k = 256 (in dict: 256 = "aa") → write e = "aa"; add w + e[0] = "b" + "a" = 258 = "ba" to dict.

code970971982256399425752616997988saabaacdict256aa257ab258ba259aac

k = 99 (ASCII) → write e = "c"; add w + e[0] = "aa" + "c" = 259 = "aac" to dict.

code970971982256399425752616997988saabaacabdict256aa257ab258ba259aac260ca

k = 257 (in dict: 257 = "ab") → write e = "ab"; add w + e[0] = "c" + "a" = 260 = "ca" to dict.

code970971982256399425752616997988saabaacababadict256aa257ab258ba259aac260ca261aba

k = 261, but the largest code in the dictionary is 260: special case. e = w + w[0] = "ab" + "a" = "aba"; 261 = "aba" is added too. The compressor used this code the moment it added it.

code970971982256399425752616997988saabaacababacdict256aa257ab258ba259aac260ca261aba262abac

k = 99 (ASCII) → write e = "c"; add w + e[0] = "aba" + "c" = 262 = "abac" to dict.

code970971982256399425752616997988saabaacababacbdict256aa257ab258ba259aac260ca261aba262abac263cb

k = 98 (ASCII) → write e = "b"; add w + e[0] = "c" + "b" = 263 = "cb" to dict.

code970971982256399425752616997988saabaacababacbdict256aa257ab258ba259aac260ca261aba262abac263cb

The 9 codes turned back into the 13-character text aabaacababacb; the decompressor's entries 256–263 match the compressor's.

String lzwDecode(List<Integer> codes) {
  List<String> dict =
      initDict();      // 0..255: byte values
  String w = dict.get(codes.get(0));
  StringBuilder s = new StringBuilder(w);
  for (int i = 1; i < codes.size(); i++) {
    int k = codes.get(i);
    String e = k < dict.size()
        ? dict.get(k)
        : w + w.charAt(0); // special case
    s.append(e);
    dict.add(w + e.charAt(0));
    w = e;
  }
  return s.toString();
}

LZW

LZ78: The Ancestor of LZW

  • LZ78 starts with an empty dictionary; its output is (index, character) pairs: the index of the longest dictionary match (0 if none) + the character that follows it.
  • Text: aabaacabcabcb → (0,a) (1,b) (1,a) (0,c) (2,c) (5,b).
  • Welch's change (LZW): start the dictionary with every single character → the output consists of codes only; no characters need to be written.
  • The LZ77 (sliding window) family underlies gzip/ZIP (DEFLATE) and PNG.
OutputDictionary
(0, a)1 = a
(1, b)2 = ab
(1, a)3 = aa
(0, c)4 = c
(2, c)5 = abc
(5, b)6 = abcb

LZW · Step through the code

lz78("aabaacabcabcb")

sa0a1b2a3a4c5a6b7c8a9b10c11b12dictempty (index 0 = empty string)output

The LZ78 dictionary starts empty (unlike LZW). w holds the longest piece found in the dictionary so far; for now it is empty.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1aoutput(0,a)

w is empty and 'a' is not in dict → (0, a) (0 = empty prefix); add 1 = "a".

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1aoutput(0,a)

w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2aboutput(0,a)(1,b)

"ab" not in dict → write w = "a" (index 1) and c = 'b': (1, b); add 2 = "ab", reset w.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2aboutput(0,a)(1,b)

w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aaoutput(0,a)(1,b)(1,a)

"aa" not in dict → write w = "a" (index 1) and c = 'a': (1, a); add 3 = "aa", reset w.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4coutput(0,a)(1,b)(1,a)(0,c)

w is empty and 'c' is not in dict → (0, c) (0 = empty prefix); add 4 = "c".

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4coutput(0,a)(1,b)(1,a)(0,c)

w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4coutput(0,a)(1,b)(1,a)(0,c)

w + c = "ab" is in dict → w = "ab"; no output yet, try a longer piece with the next character.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4c5abcoutput(0,a)(1,b)(1,a)(0,c)(2,c)

"abc" not in dict → write w = "ab" (index 2) and c = 'c': (2, c); add 5 = "abc", reset w.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4c5abcoutput(0,a)(1,b)(1,a)(0,c)(2,c)

w + c = "a" is in dict → w = "a"; no output yet, try a longer piece with the next character.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4c5abcoutput(0,a)(1,b)(1,a)(0,c)(2,c)

w + c = "ab" is in dict → w = "ab"; no output yet, try a longer piece with the next character.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4c5abcoutput(0,a)(1,b)(1,a)(0,c)(2,c)

w + c = "abc" is in dict → w = "abc"; no output yet, try a longer piece with the next character.

sa0a1b2a3a4c5a6b7c8a9b10c11b12dict1a2ab3aa4c5abc6abcboutput(0,a)(1,b)(1,a)(0,c)(2,c)(5,b)

"abcb" not in dict → write w = "abc" (index 5) and c = 'b': (5, b); add 6 = "abcb", reset w. End of text: 6 pairs.

List<String> lz78(String s) {
  Map<String, Integer> dict =
      new HashMap<>();
  List<String> out = new ArrayList<>();
  String w = "";
  for (char c : s.toCharArray()) {
    if (dict.containsKey(w + c)) {
      w = w + c;               // extend
    } else {
      int k = w.isEmpty() ? 0
                          : dict.get(w);
      out.add("(" + k + ", " + c + ")");
      dict.put(w + c, dict.size() + 1);
      w = "";
    }
  }
  if (!w.isEmpty())        // last piece
    out.add("(" + dict.get(w) + ")");
  return out;
}

LZW

RLE vs LZW

RLELZW
Redundancy capturedconsecutive repeats of one symbolrepeated strings (even if not consecutive)
Extra structurenonedictionary (hash table / trie)
TimeO(n)O(n) (O(1) average dictionary access with a trie or a (code, char)-keyed dictionary, not String concatenation)
Worst caseoutput doubles if nothing repeatslittle gain on short/random data, code width grows
Used infax, BMP/PCX, simple graphicsGIF, TIFF, PDF, Unix compress
Both are lossless: decompression returns the original data exactly. No lossless method can shrink every input (pigeonhole principle).

Summary

Summary

TopicKey points
Brute forcetry every shift left to right; no preprocessing — worst case O(n · m)
KMPlps[q] = longest proper prefix that is also a suffix; on a mismatch q = lps[q − 1], i never backs up — O(n + m)
Boyer-Mooreright to left; s += max(1, j − last[c]) plus good suffix; best O(n/m), fastest in practice
Rabin-Karppolynomial rolling hash, O(1) update; verify on equal hashes; expected O(n + m)
RLErun → (count, symbol); O(n); expands data without repeats; improved by a flag byte
LZWif w + c is in the dictionary, extend; else output code(w) and add w + c; the decompressor rebuilds the dictionary (special case: w + w[0])

Summary

Common Mistakes

Loop boundWriting i < n − m in brute force skips the last shift (s = n − m); it should be i <= n − m.
Misdefining LPSThe pattern itself does not count: lps("aaaa") = [0, 1, 2, 3], not [1, 2, 3, 4]. On a KMP mismatch use lps[q − 1], not lps[q].
Negative shiftIn Boyer-Moore, j − last[c] can be negative (c right of j). Forget the max(1, …) and the pattern moves back or loops forever.
Trusting equal hashesIn Rabin-Karp, equal hashes do not mean a match: always verify. In Java, % can return a negative value: if (th < 0) th += Q.
Multi-digit counts in RLEDecoding "12A" by reading the count as one digit is wrong (one '2'?). Bound the count or store it as a byte.
LZW's final w and special caseForgetting to output the code of the leftover w after the loop; not handling a not-yet-defined code (w + w[0]) when decompressing.

Summary

Sample Problems

  1. Find the LPS table of the pattern "aabaaab".
  2. How many comparisons does brute force make for T = "aaaaaaaa" (n = 8), P = "aab"?
  3. Boyer-Moore (bad character only): T = "GCAATGCCTATGTGACC", P = "TATGTG". What is the first shift?
  4. Base 10, Q = 13, characters are digits: T = "31415", m = 3. Given h("314"), use the rolling formula to find h("141").
  5. Encode "WWWWBBBWWWWWWB" with RLE and compute the ratio.
  6. Compress "ABABABA" with LZW using the dictionary A = 0, B = 1.
Show answers
  1. [0, 1, 0, 1, 2, 2, 3]
  2. 6 shifts × 3 = 18 ("aa" matches at every shift, 'b' fails); no match.
  3. s = 0: p[5] = t[5] = 'G' and p[4] = t[4] = 'T' match; p[3] = 'G' ≠ t[3] = 'A' → last['A'] = 1 → 3 − 1 = 2.
  4. h("314") = 314 mod 13 = 2; h' = (10 · (2 − 3 · 9) + 1) mod 13 = (−249) mod 13 = 11 = 141 mod 13 ✓ (10² mod 13 = 9).
  5. 4W3B6W1B → 8 characters / 14 ≈ 0.57.
  6. AB = 2, BA = 3, ABA = 4 are added; output 0 1 2 4 (when decompressing, 4 is the special case: "AB" + "A").

Summary

Exercises

  1. Modify KMP search to return all occurrences in a list.
  2. Find the last-occurrence table of the pattern "abcbcabbabcabc" and the first shift in the text "abccbaabccbaabcbcabbabcabc".
  3. Design a Rabin-Karp method that searches for k patterns of the same length in a single pass.
  4. Write rleDecode(String s), an RLE decoder that also handles multi-digit counts correctly.
  5. Decompress the codes 0 1 3 2 3 5 0 (A = 0, B = 1, C = 2) with LZW.
  6. Give a pair T, P (n = 12, m = 4) that produces brute force's worst case; state the number of comparisons.
Answers / hints
  1. When q == m, continue with list.add(i − m + 1); q = lps[q − 1];.
  2. a: 11, b: 12, c: 13. s = 0: p[13] = 'c' ≠ t[13] = 'b' → 13 − 12 = 1.
  3. Put the pattern hashes in a HashSet; look up each window hash in the set and verify on a hit: expected O(n + k · m).
  4. Accumulate digits with Character.isDigit as num = num · 10 + digit; on a letter, write it num times.
  5. A, B, AB, C, AB, ABC, A → ABABCABABCA.
  6. T = "aaaaaaaaaaaa", P = "aaab": 9 shifts × 4 = 36 comparisons.

Summary

References

  • Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., 2022 (Chapter 32: String Matching).
  • Sedgewick and Wayne, Algorithms, 4th ed., 2011 (Section 5.3 Substring Search, 5.5 Data Compression).
  • Goodrich, Tamassia and Goldwasser, Data Structures and Algorithms in Java, 6th ed., 2014 (Chapter 13).
  • D. E. Knuth, J. H. Morris, V. R. Pratt, "Fast Pattern Matching in Strings", SIAM J. Computing 6(2), 1977.
  • R. S. Boyer, J S. Moore, "A Fast String Searching Algorithm", Communications of the ACM 20(10), 1977.
  • R. M. Karp, M. O. Rabin, "Efficient Randomized Pattern-Matching Algorithms", IBM J. Res. Dev. 31(2), 1987.
  • T. A. Welch, "A Technique for High-Performance Data Compression", IEEE Computer 17(6), 1984.

The End

Chapter 9: String Matching and Compression
Previous: Chapter 8: Network Flow · Next: Chapter 10: Advanced String Algorithms
1 / 1 Sercan KÜLCÜ. All rights reserved.