Chapter 10: Advanced String Algorithms
Introduction
Learning Outcomes
Compare strings in lexicographic order; run LSD and MSD radix sort step by step and explain their O(W · (N + R)) cost.
Read and write regular expressions; design and draw a finite state machine (DFA) and run it on an input.
Compute Levenshtein distance with a dynamic programming table and read the edit sequence back from it; compare it with Hamming, LCS and common-substring measures.
Build a suffix array and use it for search; compute and invert the Burrows–Wheeler transform.
Introduction
String Algorithms: Four Problem Families
Four families of algorithms that work on strings:
String Sorting
Lexicographic Order
- Lexicographic (alphabetical) order arranges items by the positions of their characters in the alphabet or code table.
- Comparison starts with the first characters; if they are equal, move on to the next ones.
- The first differing character decides: "apple" < "banana" because 'a' < 'b'.
- If one string runs out, the shorter one (the prefix) comes first: "car" < "cart".
| Comparison | Decided by |
|---|---|
| apple · banana | a < b (1st character) |
| apple · apricot | p < r (3rd character) |
| car · cart | "car" runs out |
| Zebra · apple | 'Z' (90) < 'a' (97) |
java.text.Collator.String Sorting · Step through the code
Example: Comparing "apple" and "apricot"
Call compare("apple", "apricot"). A negative result means a comes first, a positive one means b does (String.compareTo works the same way).
At most as many characters as the shorter string can be compared: n = min(5, 7) = 5.
k = 0 < n: the loop continues with character 1.
Read x = 'a' (97), y = 'a' (97).
'a' == 'a': equal characters decide nothing; move on to the next one.
k = 1 < n: the loop continues with character 2.
Read x = 'p' (112), y = 'p' (112).
'p' == 'p': equal characters decide nothing; move on to the next one.
k = 2 < n: the loop continues with character 3.
Read x = 'p' (112), y = 'r' (114).
'p' ≠ 'r': first differing character found (character 3).
return 112 − 114 = −2 < 0 → apple comes before apricot. The remaining characters are never examined.
int compare(String a, String b) { int n = Math.min(a.length(), b.length()); for (int k = 0; k < n; k++) { char x = a.charAt(k), y = b.charAt(k); if (x != y) return x - y; // first difference } return a.length() - b.length(); }
String Sorting
Lexicographic Order: Words and Numbers
- Numbers sorted as strings are compared character by character from the left: "11" < "2" because '1' < '2'. Likewise, 123, 45, 6, 789 are already in lexicographic order.
- For numeric order, convert to integers or left-pad with zeros to equal length ("02" < "11").
String Sorting
Comparison-Based String Sorting
void lexSort(String[] a) { int n = a.length; for (int i = 0; i < n - 1; i++) for (int j = i + 1; j < n; j++) if (a[i].compareTo(a[j]) > 0) { String tmp = a[i]; a[i] = a[j]; a[j] = tmp; } }
- Each position receives the lexicographically smallest of the remaining strings via swaps.
compareToreads at most W characters (W: longest string).- Time: O(N²) comparisons × O(W) = O(N² · W).
- With merge sort, O(N log N) comparisons → O(W · N log N).
| Input | Output |
|---|---|
| eabcdf, abcde, bcdea, cdeab, deabcg | abcde, bcdea, cdeab, deabcg, eabcdf |
| banana, orange, grape, apple, kiwi | apple, banana, grape, kiwi, orange |
Comparison sorts need Ω(N log N) comparisons. Radix sort makes no comparisons, so this lower bound does not apply to it.
String Sorting · Step by step
LSD Radix Sort: Strings
LSD: right to left, each pass a stable counting sort on that character (short strings get an empty character); equal keys keep the order of the previous pass. Example: cab, dab, bad, dad, fad, ace, bee, bed, add, ebb (W = 3).
Pass 1 (character 3): distribute into buckets by the last character; strings in the same bucket (cab, dab) keep their input order.
Pass 2 (character 2): by the middle character; in bucket 'a', cab, dab, bad, dad, fad keep the previous order (b < d).
Pass 3 (character 1): by the first character; the array is now in lexicographic order: ace, add, bad, bed, bee, cab, dab, dad, ebb, fad.
String Sorting
LSD Radix Sort: Java Code
void lsdSort(String[] a) { int W = maxLen(a); // longest string for (int d = W - 1; d >= 0; d--) countSort(a, d); // right to left } void countSort(String[] a, int d) { int[] cnt = new int[256]; // ASCII for (String s : a) cnt[ch(s, d)]++; for (int r = 1; r < 256; r++) cnt[r] += cnt[r - 1]; String[] aux = new String[a.length]; for (int i = a.length - 1; i >= 0; i--) // stable aux[--cnt[ch(a[i], d)]] = a[i]; System.arraycopy(aux, 0, a, 0, a.length); } int ch(String s, int d) { // short string → 0 return d < s.length() ? s.charAt(d) : 0; }
- Count:
cnt[c]is the number of strings whose d-th character is c. - Prefix sums:
cnt[c]now marks where bucket c ends. - Placement runs from back to front, so equal keys keep their order (stability).
chreturns 0 for a string that is too short: "car" comes before "cart".
One pass costs O(N + R), with alphabet size R = 256; W passes → O(W · (N + R)).
String Sorting
MSD Radix Sort: Left to Right
- Split all strings into buckets by their first character (B, C, M, W).
- Recursively sort each bucket on the next character.
- Stop at a one-element bucket or at the end of a string (
\0).
With long common prefixes MSD also reads every character. Since an R = 256 count array is expensive for small buckets, practical versions switch to insertion sort for small subarrays.
String Sorting
Time Complexity and Comparison
- LSD: W passes (W: length of the longest string), each pass over N strings + R buckets → O(W · (N + R)).
- With a fixed alphabet (R = 256) the cost is O(W · N): longest string length × number of items.
| Method | Time | Extra memory | Stable |
|---|---|---|---|
| Exchange sort + compareTo | O(N² · W) | O(1) | no |
| Merge sort + compareTo | O(W · N log N) | O(N) | yes |
| LSD radix sort | O(W · (N + R)) | O(N + R) | yes |
| MSD radix sort | O(N · W) worst | O(N + W · R) | yes |
Regular Expressions
Regular Expressions
- A regular expression (regex) is a string that describes a search pattern.
- Used to find, validate, extract or replace every part of a text that matches the pattern.
- It automates text processing: log analysis, form validation, data cleaning, lexical analysis (lexers) in compilers.
- Theory: every regular expression has an equivalent finite state machine (Kleene's theorem).
| Pattern | Text | Matches |
|---|---|---|
colou?r | color colour | color, colour |
gr(a|e)y | gray grey groy | gray, grey |
\d+ | 20.99 TL, 5 pcs | 20, 99, 5 |
\bcode\b | code coding | only "code" |
java.util.regex (Pattern, Matcher) and String.matches, replaceAll, split. Inside a Java string the backslash is doubled: "\\d+".Regular Expressions
Basic Operators and Concepts
| Op. | Meaning | Example | Found in | Not in |
|---|---|---|---|---|
. | Any single character (except newline) | a.c | abc, a7c | ac |
* | Zero or more of the preceding item | ab*c | ac, abbc | adc |
+ | One or more of the preceding item | ab+c | abc, abbc | ac |
? | Zero or one of the preceding item | ab?c | ac, abc | abbc |
[ ] | One character from the set | [ck]ar | car, kar | bar |
^ | Start of the string | ^ab | abc | cab |
$ | End of the string | ab$ | cab | abc |
( ) | Grouping; defines a subexpression | ^(ab)+$ | ab, abab | aba |
The columns show whether the string contains a part that matches the pattern (Matcher.find). *, +, ? apply only to the item right before them (a character, set or group).
Regular Expressions
More Operators and Special Characters
| Expression | Meaning |
|---|---|
a | a single 'a' character |
abc | exactly "abc" |
{ } | a given number of repeats: \d{4} |
| | alternatives: cat|dog |
\b | word boundary (start or end) |
\d | a digit: [0-9] |
\w | a word character: [a-zA-Z0-9_] |
\s | whitespace: space, tab, newline |
- Uppercase forms are complements:
\Da non-digit,\Wa non-word character,\Sa non-whitespace character. - A special character itself must be escaped:
\.a dot,\+a plus,\(a parenthesis. \wmatches a single character; write\w+for a word.
\w covers only ASCII letters by default: the 'ş' in "şeker" does not match. For Turkish text use \p{L} or Pattern.UNICODE_CHARACTER_CLASS.Regular Expressions
Character Classes
[abc] | 'a', 'b' or 'c' |
[a-z] | any lowercase letter |
[A-Z] | any uppercase letter |
[0-9] | any digit |
[a-zA-Z_] | a letter or underscore |
[^abc] | any character except 'a', 'b', 'c' |
[^0-9] | a non-digit character |
[^0-9]+ | "ab12cd" → "ab", "cd" |
^means "not" inside a bracket, at its start; outside, it marks the start of the string.- Inside a set,
.,*,+are not special:[.+]matches a dot or a plus. A hyphen (-) denotes a range; to match it literally, put it first or last. - The range
[a-z]is based on character codes; ç, ğ, ı, ö, ş, ü are not in it.
Regular Expressions
Repetition and Grouping
Quantifiers
a* | zero or more 'a' |
a+ | one or more 'a' |
a? | zero or one 'a' |
a{3} | exactly three 'a' |
a{2,4} | two to four 'a' |
a{2,} | at least two 'a' |
Grouping
(abc) | exactly "abc"; captured as a group |
(a|b|c) | 'a', 'b' or 'c'; captured as a group |
(ab)+ | "ab", "abab", … |
In Java a captured group is read with m.group(1).
a{2,4} takes the longest possible piece of "aaaaa" (aaaa). A ? after the quantifier picks the shortest match: in "<b>x</b>", "<.+?>" first takes only "<b>", not the whole text.Regular Expressions
Lookahead and Lookbehind
| Pattern | Name | Matches when | "ab ac ba" |
|---|---|---|---|
a(?=b) | lookahead | 'a' followed by 'b' | ab ac ba |
a(?!b) | negative lookahead | 'a' not followed by 'b' | ab ac ba |
(?<=b)a | lookbehind | 'a' preceded by 'b' | ab ac ba |
(?<!b)a | negative lookbehind | 'a' not preceded by 'b' | ab ac ba |
- Lookaround tests a position and consumes no characters: the match of
a(?=b)is just "a"; the 'b' is not part of it. - Example:
^(?=.*\d)(?=.*[A-Z]).{8,}$validates a password of at least 8 characters containing at least one digit and one uppercase letter.
Regular Expressions
Example: Validation Expressions
| Purpose | Regular expression |
|---|---|
^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$ | |
| Phone | ^(\+\d{1,3}[- ]?)?\d{10}$ |
| URL | ^https?:\/\/[^\s/$.?#].[^\s]*$ |
[…]+ user name, @, domain, \. a literal dot, an extension of at least 2 letters. ✓ ali.veli@giresun.edu.tr ✗ ali@tr ✗ a b@x.com(…)? country code: + and 1–3 digits, then a hyphen or space; then exactly 10 digits. ✓ +90 5321234567 ✓ 5321234567 ✗ 0532 123 45 67s? http or https; the first character cannot be whitespace, /, $, ., ?, #; then non-whitespace characters. ✓ https://giresun.edu.tr ✗ ftp://x.com ✗ "http:// x"These expressions are practical approximations; they do not cover the full standards (RFC 5322 email, RFC 3986 URI).
Regular Expressions
Parse Tree: A URL Example
A grammar for http://didit.csail.mit.edu:4949/:
url ::= 'http://' host
':' port '/'
host ::= word '.' host
| word
word ::= [a-z]+
port ::= [0-9]+http://([a-z]+\.)*[a-z]+:[0-9]+/. Nested structures such as balanced parentheses cannot be described by a regex; they need a context-free grammar and a parser.Regular Expressions
Regular Expressions in Java: Pattern and Matcher
String input = "Name: Ali Veli, Age: 30, Email: ali.veli@example.com";
Pattern p = Pattern.compile("Age: (\\d+),"); // compile
Matcher m = p.matcher(input);
if (m.find()) { // first match
int age = Integer.parseInt(m.group(1)); // group 1: "30"
System.out.println("Age: " + age);
}| Pattern | group(1) | Explanation |
|---|---|---|
Name: (.*?), | Ali Veli | lazy .*?: stops at the first comma |
Age: (\d+), | 30 | one or more digits |
Email: (.*?)$ | ali.veli@example.com | up to the end of the line |
- The same job can be done without regexes using
indexOfandsubstring: find the start marker (e.g. "Age: ") and cut up to the end marker (","). find()tries to match part of the text,matches()the whole of it.
Regular Expressions
Try It: Regex Tester
Uses the browser's JavaScript regex engine; its syntax is largely the same as Java's. Matches are highlighted and groups are listed.
Finite State Machines
Finite State Machines
- A finite state machine (FSM) is a mathematical model that processes a string one character at a time.
- It has finitely many states; its only memory is the current state.
- Simple and easy to understand, yet it handles complex string analysis tasks (lexical analysis, protocols, pattern search).
Finite State Machines
Kinds of FSM: DFA, NFA, ε-NFA
- All three recognize the same languages (the regular languages): subset construction turns any NFA into a DFA, though the number of states can grow to 2ⁿ in the worst case.
- grep, lex/flex and compiler lexers use this chain.
Finite State Machines
Using It: Simulating a DFA
- Enter the start state.
- For each character, update the state from the transition table.
- When the string ends, check whether the final state is an accept state.
boolean accepts(int[][] delta, boolean[] accept, String s) { int state = 0; // start state for (char c : s.toCharArray()) state = delta[state][c - '0']; // one transition return accept[state]; // accepting? }
int[|Q|][|Σ|]. Each character costs one table lookup: a string of length n is processed in O(n) time, with no backtracking.Finite State Machines
Example: A Machine That Recognizes "nice"
- A machine that recognizes "nice": each state stands for the prefix read so far (ε, n, ni, nic, nice).
- If the expected character does not arrive (≠n, ≠i, …), the machine moves to Error and stays there (a trap state).
- "nice" → 1 → 2 → 3 → 4 → 7: accept. "nine" → 1 → 2 → 3 → 6: reject (the third character 'n' ≠ 'c').
- Any character after Success → Error ("nicer" is rejected); self-loops that are not drawn are implied.
Finite State Machines · Step by step
Example: A DFA over a Binary Alphabet
Start: state A. Input: 10011.
Symbol 1 is 1: δ(A, 1) = B.
Symbol 2 is 0: δ(B, 0) = B.
Symbol 3 is 0: δ(B, 0) = B.
Symbol 4 is 1: δ(B, 1) = C.
Symbol 5 is 1: δ(C, 1) = D. Input exhausted; D is an accept state → accept.
| δ | 0 | 1 |
|---|---|---|
| → A | E | B |
| B | B | C |
| C | B | D |
| D * | D | A |
| E | E | F |
| F | F | A |
→ start, * accept. One state per cell: a DFA.
Finite State Machines
Example: An Automaton That Searches for "ACACAGA"
| text | G | A | C | A | C | A | C | A | G | A |
|---|---|---|---|---|---|---|---|---|---|---|
| state | 0 | 1 | 2 | 3 | 4 | 5 | 4 | 5 | 6 | 7 |
- State q = length of the longest prefix of the pattern that matches the end of the text read so far. All undrawn transitions go to 0. Reaching state 7 means a match (here at position 3).
- When the 7th character 'C' arrives, go back 5 → 4: the end of "ACACAC" can continue as the prefix "ACAC". This is the automaton form of KMP's prefix function: preprocessing O(m · |Σ|), search O(n).
Finite State Machines · Step through the code
dfaSearch(dfa, "GACACACAGA", 7)
The automaton for ACACAGA is built in advance. State q = length of the longest pattern prefix that matches the end of the text. Only forward transitions are drawn here; the back transitions are on the previous slide. Initially q = 0.
t[0] = 'G' extends no prefix of the pattern: q stays 0.
t[1] = 'A' is the next pattern character: 0 → 1.
t[2] = 'C' is the next pattern character: 1 → 2.
t[3] = 'A' is the next pattern character: 2 → 3.
t[4] = 'C' is the next pattern character: 3 → 4.
t[5] = 'A' is the next pattern character: 4 → 5.
t[6] = 'C' is not the expected 'G', but q is not reset: "ACACAC" ends with the prefix "ACAC" → 5 → 4. No backing up in the text.
t[7] = 'A' is the next pattern character: 4 → 5.
t[8] = 'G' is the next pattern character: 5 → 6.
'A' takes 6 → 7 = m: pattern found; return i − m + 1 = 9 − 7 + 1 = 3.
int dfaSearch(int[][] dfa, String t, int m) { int q = 0; // state for (int i = 0; i < t.length(); i++) { q = dfa[q][t.charAt(i)]; if (q == m) return i - m + 1; // match } return -1; }
Finite State Machines · Step through the code
Parsing Numbers with an FSM: Code Trace
Input "$20.99 or 5.". State START, buffer empty.
'$' is neither a digit nor a dot; state stays START, nothing happens.
'2' is a digit and the state is START → DIGIT; appended to the buffer.
'0' is a digit: state stays DIGIT; append to the buffer.
'.' arrives → move to DECIMAL; the dot is appended to the buffer.
'9' is a digit: state stays DECIMAL; append to the buffer.
'9' is a digit: state stays DECIMAL; append to the buffer.
' ' is not part of a number: the number ends, 20.99 is printed; back to START.
'o' is neither a digit nor a dot; state stays START, nothing happens.
'r' is neither a digit nor a dot; state stays START, nothing happens.
' ' is neither a digit nor a dot; state stays START, nothing happens.
'5' is a digit and the state is START → DIGIT; appended to the buffer.
'.' arrives → move to DECIMAL; the dot is appended to the buffer.
Input exhausted; the state is DECIMAL, so the buffer is printed: 5.. The trailing dot was included in the number (a flaw).
void findNumbers(String s) { State d = State.START; StringBuilder buf = new StringBuilder(); for (char c : s.toCharArray()) { boolean digit = Character.isDigit(c); if (digit) { if (d == State.START) d = State.DIGIT; buf.append(c); } else if (c == '.' && d != State.DECIMAL) { d = State.DECIMAL; buf.append(c); } else if (d != State.START) { // number ended System.out.println(buf); buf.setLength(0); d = State.START; } } if (d != State.START) System.out.println(buf); }
Edit Distance
Levenshtein Distance (Edit Distance)
- Levenshtein distance (V. Levenshtein, 1965) is a metric that measures how different two strings are.
- It is the minimum number of single-character edits needed to turn one string into the other.
- Used in autocorrect and predictive text, spell checking, DNA sequence comparison and fuzzy search.
- Metric properties: d(a, a) = 0, d(a, b) = d(b, a), triangle inequality.
Edit Distance
Edit Examples
HONDA → HYUNDAI: substitute O→Y, insert U, insert I: d = 3
substitution insertion deletion
- (a) rain → sain → shin → shine · (b) shine → rhine → raine → rain (delete e) · (c) shine → tshine → trhine → traine → train (delete e). A struck-through orange letter is deleted.
- The distance is symmetric: (a) and (b) are inverses of each other; insertions and deletions swap roles.
Edit Distance
Computing Levenshtein Distance
D[i][j] = minimum cost of turning the first i characters of a into the first j characters of b.
if ai = bj then D[i][j] = D[i−1][j−1]
otherwise D[i][j] = 1 + min( D[i−1][j] delete, D[i][j−1] insert, D[i−1][j−1] replace )
- Dynamic programming: each subproblem is solved once and stored in an (m+1) × (n+1) table; the answer is in the bottom-right corner.
- A cell depends only on its upper, left and upper-left neighbors → fill the table row by row.
- Time O(m · n); memory O(m · n), or O(min(m, n)) with two rows if only the distance is needed.
Edit Distance · Step by step
DP Table: CARS → PAIRS
a = CARS (rows), b = PAIRS (columns)
| ε | P | A | I | R | S | |
|---|---|---|---|---|---|---|
| ε | 0 | 1 | 2 | 3 | 4 | 5 |
| C | 1 | |||||
| A | 2 | |||||
| R | 3 | |||||
| S | 4 |
Start: first row 0..5 (ε → P, PA, … : insertions), first column 0..4 (C, CA, … → ε : deletions).
| ε | P | A | I | R | S | |
|---|---|---|---|---|---|---|
| ε | 0 | 1 | 2 | 3 | 4 | 5 |
| C | 1 | 1 | 2 | 3 | 4 | 5 |
| A | 2 | |||||
| R | 3 | |||||
| S | 4 |
Row 1 (C) filled. C ≠ P: D[1][1] = 1 + min(0, 1, 1) = 1 (substitution).
| ε | P | A | I | R | S | |
|---|---|---|---|---|---|---|
| ε | 0 | 1 | 2 | 3 | 4 | 5 |
| C | 1 | 1 | 2 | 3 | 4 | 5 |
| A | 2 | 2 | 1 | 2 | 3 | 4 |
| R | 3 | |||||
| S | 4 |
Row 2 (A) filled. A = A: D[2][2] = D[1][1] = 1, no added cost.
| ε | P | A | I | R | S | |
|---|---|---|---|---|---|---|
| ε | 0 | 1 | 2 | 3 | 4 | 5 |
| C | 1 | 1 | 2 | 3 | 4 | 5 |
| A | 2 | 2 | 1 | 2 | 3 | 4 |
| R | 3 | 3 | 2 | 2 | 2 | 3 |
| S | 4 |
Row 3 (R) filled. R = R: D[3][4] = D[2][3] = 2. A ≠ I: D[2][3] = 1 + min(1, 2, 3) = 2.
| ε | P | A | I | R | S | |
|---|---|---|---|---|---|---|
| ε | 0 | 1 | 2 | 3 | 4 | 5 |
| C | 1 | 1 | 2 | 3 | 4 | 5 |
| A | 2 | 2 | 1 | 2 | 3 | 4 |
| R | 3 | 3 | 2 | 2 | 2 | 3 |
| S | 4 | 4 | 3 | 3 | 3 | 2 |
Row 4 (S) filled. S = S: D[4][5] = D[3][4] = 2.
| ε | P | A | I | R | S | |
|---|---|---|---|---|---|---|
| ε | 0 | 1 | 2 | 3 | 4 | 5 |
| C | 1 | 1 | 2 | 3 | 4 | 5 |
| A | 2 | 2 | 1 | 2 | 3 | 4 |
| R | 3 | 3 | 2 | 2 | 2 | 3 |
| S | 4 | 4 | 3 | 3 | 3 | 2 |
Result D[4][5] = 2. Reading back: substitute C→P, A equal, insert I, R and S equal: CARS → PARS → PAIRS.
Edit Distance
Levenshtein: Java Code
int levenshtein(String a, String b) { int m = a.length(), n = b.length(); int[][] D = new int[m + 1][n + 1]; for (int i = 0; i <= m; i++) D[i][0] = i; // delete for (int j = 0; j <= n; j++) D[0][j] = j; // insert for (int i = 1; i <= m; i++) for (int j = 1; j <= n; j++) if (a.charAt(i - 1) == b.charAt(j - 1)) D[i][j] = D[i - 1][j - 1]; // equal else D[i][j] = 1 + Math.min(D[i - 1][j - 1], // replace Math.min(D[i][j - 1], // insert D[i - 1][j])); // delete return D[m][n]; }
- Table index i corresponds to string index i − 1: row 0 of D is for the empty prefix.
- Examples: kitten → sitting = 3, ercan → sercan = 1, apple → banana = 5.
Edit Distance · Step through the code
levenshtein("kar", "ark")
a = "kar" (rows), b = "ark" (columns). Column 0 holds i deletions and row 0 holds j insertions: the cost of converting to or from the empty prefix.
'k' ≠ 'a' → 1 + min(replace 0, insert 1, delete 1) = 1.
'k' ≠ 'r' → 1 + min(replace 1, insert 1, delete 2) = 2.
a[0] = b[2] = 'k' → free: D[1][3] = up-left = 2.
a[1] = b[0] = 'a' → free: D[2][1] = up-left = 1.
'a' ≠ 'r' → 1 + min(replace 1, insert 1, delete 2) = 2.
'a' ≠ 'k' → 1 + min(replace 2, insert 2, delete 2) = 3.
'r' ≠ 'a' → 1 + min(replace 2, insert 3, delete 1) = 2.
a[2] = b[1] = 'r' → free: D[3][2] = up-left = 1.
'r' ≠ 'k' → 1 + min(replace 2, insert 1, delete 3) = 2.
D[3][3] = 2: delete the leading k of "kar" (ar), then append k (ark). The green path shows these two edits.
int levenshtein(String a, String b) { int m = a.length(), n = b.length(); int[][] D = new int[m + 1][n + 1]; for (int i = 0; i <= m; i++) D[i][0] = i; // delete for (int j = 0; j <= n; j++) D[0][j] = j; // insert for (int i = 1; i <= m; i++) for (int j = 1; j <= n; j++) if (a.charAt(i-1) == b.charAt(j-1)) D[i][j] = D[i-1][j-1]; // equal else D[i][j] = 1 + Math.min(D[i-1][j-1], Math.min(D[i][j-1], D[i-1][j])); return D[m][n]; }
Edit Distance
Comparing Similarity Measures
| Measure | What it measures | Time | AGGTAB · GXTXAYB |
|---|---|---|---|
| Hamming | equal lengths; number of differing positions | O(n) | undefined (6 ≠ 7) |
| Levenshtein | number of inserts / deletes / substitutions | O(m · n) | 4 |
| LCS | longest common subsequence | O(m · n) | 4 (GTAB) |
| Longest common substring | longest contiguous common piece | O(m · n) | 1 (e.g. "G") |
- The LCS is the longest common subsequence that keeps relative order but need not be contiguous; it is found with a DP table like Levenshtein's (diagonal + 1 on a match, otherwise max(up, left)).
- If only insertions and deletions are allowed, the distance is m + n − 2 · LCS: 6 + 7 − 8 = 5; Levenshtein, which also allows substitutions, brings it down to 4.
- Longest common substring: on a match L[i][j] = L[i−1][j−1] + 1, otherwise 0; the answer is the largest value in the table (ercan · sercan → "ercan").
- A similarity ratio: 1 − d / max(m, n). kitten · sitting: 1 − 3/7 ≈ 0.57.
String Transforms
Suffix Arrays
- A suffix array holds the starting indices of all suffixes of a string, in sorted order.
- Each suffix runs from some position of the string to its end. A suffix array is a compact representation of all substrings: every substring is a prefix of some suffix.
| i | suffix |
|---|---|
| 0 | banana |
| 1 | anana |
| 2 | nana |
| 3 | ana |
| 4 | na |
| 5 | a |
| rank | SA | sorted suffix |
|---|---|---|
| 0 | 5 | a |
| 1 | 3 | ana |
| 2 | 1 | anana |
| 3 | 0 | banana |
| 4 | 4 | na |
| 5 | 2 | nana |
"banana" → SA = [5, 3, 1, 0, 4, 2]. Only n integers: O(n) memory versus the O(n²) nodes of a suffix trie. Uses: string search, sorting, genome sequence analysis, BWT and data compression.
String Transforms
Building a Suffix Array
| Method | Idea | Time |
|---|---|---|
| Brute force | Generate all suffixes and sort them with Arrays.sort; each comparison is O(n) | O(n² log n) |
| Manber–Myers (1990) | Prefix doubling: derive the order on 2k characters from the order on k characters without comparing letters, by radix sorting pairs of ranks | O(n log n) |
| Larsson–Sadakane (1999) | Same doubling; only groups not yet separated are re-sorted by comparing the ranks of later suffixes | O(n log n) |
| DC3 (2003), SA-IS (2009) | Recursively sort a subset of the suffixes and merge in the rest | O(n) |
String Transforms · Step by step
Prefix Doubling
The Manber–Myers idea on "banana": the order on 2k characters comes from the order on k characters. log₂ n rounds, each O(n) with radix sort.
| i | first 1 char | key | new rank |
|---|---|---|---|
| 1 | a | 'a' | 0 |
| 3 | a | 'a' | 0 |
| 5 | a | 'a' | 0 |
| 0 | b | 'b' | 1 |
| 2 | n | 'n' | 2 |
| 4 | n | 'n' | 2 |
k = 1: sort the suffixes by their first character: a → 0, b → 1, n → 2. Equal ranks are not yet separated.
| i | first 2 chars | key | new rank |
|---|---|---|---|
| 5 | a | (0, −1) | 0 |
| 1 | an | (0, 2) | 1 |
| 3 | an | (0, 2) | 1 |
| 0 | ba | (1, 0) | 2 |
| 2 | na | (2, 0) | 3 |
| 4 | na | (2, 0) | 3 |
k = 2: key = (rank[i], rank[i + 1]), or −1 if i + 1 ≥ n. No letters are compared; only pairs of numbers are sorted.
| i | first 4 chars | key | new rank |
|---|---|---|---|
| 5 | a | (0, −1) | 0 |
| 3 | ana | (1, 0) | 1 |
| 1 | anan | (1, 1) | 2 |
| 0 | bana | (2, 3) | 3 |
| 4 | na | (3, −1) | 4 |
| 2 | nana | (3, 3) | 5 |
k = 4: key = (rank[i], rank[i + 2]), or −1 if i + 2 ≥ n. No letters are compared; only pairs of numbers are sorted. All ranks distinct → SA = [5, 3, 1, 0, 4, 2].
String Transforms
Searching with a Suffix Array
- Since the suffixes are sorted, those that start with the pattern P form a contiguous range in SA.
- Both ends of the range are found by binary search: O(log n) steps × O(m) per comparison = O(m log n).
- Far less memory than a suffix trie: just n integers.
- Add the LCP array (longest common prefix of adjacent suffixes) and the longest repeated substring is found in O(n).
| rank | SA | suffix |
|---|---|---|
| 0 | 5 | a |
| 1 | 3 | ana |
| 2 | 1 | anana |
| 3 | 0 | banana |
| 4 | 4 | na |
| 5 | 2 | nana |
String Transforms · Step through the code
search("banana", sa, "an")
t = "banana", SA = [5, 3, 1, 0, 4, 2], p = "an". The suffixes starting with "an" form a contiguous range of the sorted array; two binary searches find its ends.
mid = 3: "banana" ≥ "an" → hi = mid. Range [0, 3).
mid = 1: "ana" ≥ "an" → hi = mid. Range [0, 1).
mid = 0: "a" < "an" → lo = mid + 1. Range [1, 1).
First loop done: the first suffix not less than "an" is at rank 1 → first = 1. For the second search, hi is reset to the end of the array.
mid = 3: "banana" does not start with "an" → hi = mid.
mid = 2: "anana" starts with "an" → lo = mid + 1.
Result [1, 3): ranks 1 and 2 → SA values 3 and 1. "an" occurs in banana at positions 3 and 1; 5 comparisons sufficed (each binary search takes at most ⌊log₂ 6⌋ + 1 = 3).
int[] search(String t, int[] sa, String p) { int lo = 0, hi = sa.length; while (lo < hi) { // first >= p int mid = (lo + hi) / 2; if (t.substring(sa[mid]) .compareTo(p) < 0) lo = mid + 1; else hi = mid; } int first = lo; hi = sa.length; while (lo < hi) { // not prefixed by p int mid = (lo + hi) / 2; if (t.startsWith(p, sa[mid])) lo = mid + 1; else hi = mid; } return new int[] { first, lo }; }
String Transforms
The Burrows–Wheeler Transform (BWT)
- The Burrows–Wheeler transform (BWT, 1994) rearranges the input: it changes no characters, only their positions.
- It brings equal characters together: characters that precede similar contexts end up next to each other.
- It improves the performance of compression algorithms.
- It is reversible: the original data can be recovered from the transform.
Steps
- Generate all cyclic rotations of the input string.
- Sort the rotations alphabetically.
- Form a new string from the last characters of the sorted rotations (column L).
- Store the row number of the original string in the sorted list (or append a unique
$).
String Transforms · Step by step
BWT Example: BANANA
Input: BANANA
1. All rotations: BANANA, ANANAB, NANABA, ANABAN, NABANA, ABANAN (moving the first character to the end).
2. Sorted: ABANAN, ANABAN, ANANAB, BANANA, NABANA, NANABA. The original string is in row 3 (counting from 0).
3. Last characters (column L): NNBAAA. Output: (NNBAAA, 3). Equal letters are grouped: NN, AAA.
String Transforms
BWT Example: abraca
- Last column of the sorted matrix: L = caraab.
- The original "abraca" is in row 1 of the sorted matrix (counting from 0); this index is stored for the inverse transform.
- The first column (F = aaabcr) is just L's letters in sorted order; knowing L gives F.
substring and sorts them with Arrays.sort → O(n² log n) time, O(n²) memory. In practice it is derived from the suffix array in O(n) via L[i] = T[SA[i] − 1] (T is the text with $ appended; L[i] = $ if SA[i] = 0; the formula is for the $ version). The $ and cyclic BWTs differ: cyclic BANANA → NNBAAA, BANANA$ → ANNB$AA.String Transforms
Inverting the BWT (LF Mapping)
| i | F | … | L |
|---|---|---|---|
| 0 | $ | banan | a |
| 1 | a | $bana | n |
| 2 | a | na$ba | n |
| 3 | a | nana$ | b |
| 4 | b | anana | $ |
| 5 | n | a$ban | a |
| 6 | n | ana$b | a |
- T = banana$ ($ smallest, unique) → BWT = annb$aa.
- F = sort(L) = $aaabnn.
- LF rule: the k-th 'c' in L is the same character as the k-th 'c' in F. In row i, L[i] immediately precedes F[i] in the text.
- Start at row 0, which begins with $; read L[i] and go to row LF(i):
L read: a n a n a b → reverse → banana
With the ranks (each character's occurrence number in L) precomputed, inversion takes O(n) time. Without $, store the original row index instead (3, as in the BANANA example).
String Transforms · Step through the code
inverseBWT("annb$aa")
L = "annb$aa" (the BWT of banana$). rank[i]: which occurrence of its character L[i] is in L (from 0). E.g. L[6] = a, the third a → 2.
C[c] = number of characters smaller than c = the row where c starts in column F: $ → 0, a → 1, b → 4, n → 5.
i = 0: L[0] = 'a' is prepended → s = "a". LF(0) = C['a'] + rank[0] = 1 + 0 = 1.
i = 1: L[1] = 'n' is prepended → s = "na". LF(1) = C['n'] + rank[1] = 5 + 0 = 5.
i = 5: L[5] = 'a' is prepended → s = "ana". LF(5) = C['a'] + rank[5] = 1 + 1 = 2.
i = 2: L[2] = 'n' is prepended → s = "nana". LF(2) = C['n'] + rank[2] = 5 + 1 = 6.
i = 6: L[6] = 'a' is prepended → s = "anana". LF(6) = C['a'] + rank[6] = 1 + 2 = 3.
i = 3: L[3] = 'b' is prepended → s = "banana". LF(3) = C['b'] + rank[3] = 4 + 0 = 4.
Rows 0 → 1 → 5 → 2 → 6 → 3 → 4 were visited; the next row would end in $, so the loop stops after n − 1 = 6 rounds. Result: banana.
String inverseBWT(String L) { // with $
int n = L.length();
int[] rank = new int[n];
int[] cnt = new int[256];
int[] C = new int[256];
for (int i = 0; i < n; i++)
rank[i] = cnt[L.charAt(i)]++;
for (int c = 1; c < 256; c++)
C[c] = C[c - 1] + cnt[c - 1];
StringBuilder s = new StringBuilder();
int i = 0; // $ row
for (int k = 0; k < n - 1; k++) {
char c = L.charAt(i);
s.insert(0, c);
i = C[c] + rank[i]; // LF(i)
}
return s.toString();
}Summary
Summary
| Topic | Key points |
|---|---|
| Lexicographic order | the first differing character decides; a prefix comes first; code order ≠ Turkish alphabet |
| Radix sort | LSD: stable counting sort right to left, O(W · (N + R)); MSD: from the left, recursing into buckets |
| Regular expressions | . * + ? [ ] ^ $ ( ) { } | \d \w \s; lookarounds consume no characters |
| FSM | (Q, Σ, δ, q₀, F); one transition per step in a DFA, O(n) scan; NFA ≡ DFA ≡ regex |
| Levenshtein | 1 + min(delete, insert, replace), or up-left if equal; O(m · n) |
| Similarity | Hamming on equal lengths O(n); LCS and common substring O(m · n); insert/delete-only distance = m + n − 2 · LCS |
| Suffix array | sorted suffix indices; brute force O(n² log n), doubling O(n log n); search O(m log n) |
| BWT | last column of the sorted rotations + row index; inverted via LF mapping; bzip2 |
Summary
Common Mistakes
Collator or normalization.a.b also matches "a+b"; <.*> swallows the whole line. Use \. and the lazy .*?. Expressions like (a+)+$ cause exponential backtracking.charAt(i − 1). Not initializing the first row/column breaks the result.$ makes the transform irreversible.Summary
Sample Problems
- Put the strings "10", "9", "100", "2" in lexicographic order.
- Apply LSD radix sort to sob, bat, cab, bot, tab; write the array after each pass.
- Which of "555-1234", "5551234", "55-12345" does
^\d{3}-\d{4}$match? - What is Levenshtein("sunday", "saturday")?
- What is Hamming("karolin", "kathrin")? Can the Levenshtein distance be larger?
- Find the BWT of "kayak" and the original row index.
Show answers
- "10" < "100" < "2" < "9" ("10" is a prefix of "100").
- Character 3: sob, cab, tab, bat, bot · character 2: cab, tab, bat, sob, bot · character 1: bat, bot, cab, sob, tab.
- Only "555-1234".
- 3: insert "a" and "t" (satunday), substitute n → r (saturday).
- 3 (r/t, o/h, l/r). No: with equal lengths, Hamming edits are just substitutions, so Levenshtein ≤ Hamming; here it is also 3.
- Sorted rotations: akkay, ayakk, kayak, kkaya, yakka → BWT = ykkaa, index 2.
Summary
Exercises
- Write MSD radix sort recursively.
- Write a regex that validates the Turkish mobile number format (0532 123 45 67).
- Draw a DFA that accepts binary strings with an even number of 1s and write its transition table.
- Compute Levenshtein with only two rows of memory, O(min(m, n)) extra space.
- The number-parsing FSM prints "5." on the input "… or 5.". Fix the machine.
- Find the suffix array of "mississippi".
Answers / hints
sort(a, lo, hi, d): distribute into buckets by the d-th character (strings that have ended go to bucket −1), then callsort(…, d + 1)on each bucket.^0\d{3} \d{3} \d{2} \d{2}$(for 05 numbers only:^05\d{2} …).- Two states: Even (start, accept) and Odd. On 0: stay; on 1: Even ↔ Odd.
- Arrays
prevandcurr, swapped at the end of each row. Use the shorter string for the columns. - Before printing, drop a trailing '.' from the buffer (or append the dot only when a digit follows).
- SA = [10, 7, 4, 1, 0, 9, 8, 6, 3, 5, 2] (i, ippi, issippi, ississippi, mississippi, pi, ppi, sippi, sissippi, ssippi, ssissippi).
Summary
References
- Sedgewick and Wayne, Algorithms, 4th ed., 2011 (5.1 String Sorts, 5.4 Regular Expressions).
- Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., 2022 (string-matching automata).
- Hopcroft, Motwani and Ullman, Introduction to Automata Theory, Languages, and Computation, 3rd ed., 2006.
- V. I. Levenshtein, "Binary codes capable of correcting deletions, insertions, and reversals", Soviet Physics Doklady 10(8), 1966.
- R. A. Wagner and M. J. Fischer, "The String-to-String Correction Problem", Journal of the ACM 21(1), 1974.
- U. Manber and G. Myers, "Suffix Arrays: A New Method for On-Line String Searches", SIAM Journal on Computing 22(5), 1993.
- M. Burrows and D. J. Wheeler, "A Block-sorting Lossless Data Compression Algorithm", DEC SRC Research Report 124, 1994.
- J. E. F. Friedl, Mastering Regular Expressions, 3rd ed., O'Reilly, 2006; Java
java.util.regex.Patterndocumentation.