Chapter 10: Advanced String Algorithms

Algorithms
Lexicographic orderRadix sortRegular expressionsFinite state machines Levenshtein distanceSimilarity measuresSuffix arraysBurrows–Wheeler transform
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Sorting

Compare strings in lexicographic order; run LSD and MSD radix sort step by step and explain their O(W · (N + R)) cost.

Parsing

Read and write regular expressions; design and draw a finite state machine (DFA) and run it on an input.

Similarity

Compute Levenshtein distance with a dynamic programming table and read the edit sequence back from it; compare it with Hamming, LCS and common-substring measures.

Transforms

Build a suffix array and use it for search; compute and invert the Burrows–Wheeler transform.

Introduction

String Algorithms: Four Problem Families

Four families of algorithms that work on strings:

String SortingLexicographic order: strings are arranged as in a dictionary by comparing character codes. Radix sort: groups strings without comparisons, treating each character as a digit.
ParsingRegular expressions: strings that describe a search pattern. Finite state machines (FSM): automata that read the input one character at a time and change state.
SimilarityLevenshtein distance: the minimum number of single-character edits that turn one string into another. LCS: the longest common subsequence, in order but not necessarily contiguous.
TransformationSuffix array: the starting indices of all suffixes in sorted order. Burrows–Wheeler transform (BWT): a reversible transform obtained by sorting rotations; the first stage of bzip2.

String Sorting

Lexicographic Order

  • Lexicographic (alphabetical) order arranges items by the positions of their characters in the alphabet or code table.
  • Comparison starts with the first characters; if they are equal, move on to the next ones.
  • The first differing character decides: "apple" < "banana" because 'a' < 'b'.
  • If one string runs out, the shorter one (the prefix) comes first: "car" < "cart".
ComparisonDecided by
apple · bananaa < b (1st character)
apple · apricotp < r (3rd character)
car · cart"car" runs out
Zebra · apple'Z' (90) < 'a' (97)
The computer's "alphabet" is the character code (ASCII / Unicode): every uppercase letter comes before every lowercase one, and ç, ğ, ı, ö, ş, ü come after z. For Turkish dictionary order use java.text.Collator.

String Sorting · Step through the code

Example: Comparing "apple" and "apricot"

abappleapricot0123456

Call compare("apple", "apricot"). A negative result means a comes first, a positive one means b does (String.compareTo works the same way).

abappleapricot0123456

At most as many characters as the shorter string can be compared: n = min(5, 7) = 5.

abappleapricot0123456k

k = 0 < n: the loop continues with character 1.

abappleapricot0123456k'a' = 97'a' = 97

Read x = 'a' (97), y = 'a' (97).

abappleapricot0123456k'a' = 97'a' = 97

'a' == 'a': equal characters decide nothing; move on to the next one.

abappleapricot0123456k

k = 1 < n: the loop continues with character 2.

abappleapricot0123456k'p' = 112'p' = 112

Read x = 'p' (112), y = 'p' (112).

abappleapricot0123456k'p' = 112'p' = 112

'p' == 'p': equal characters decide nothing; move on to the next one.

abappleapricot0123456k

k = 2 < n: the loop continues with character 3.

abappleapricot0123456k'p' = 112'r' = 114

Read x = 'p' (112), y = 'r' (114).

abappleapricot0123456k'p' = 112'r' = 114

'p' ≠ 'r': first differing character found (character 3).

abappleapricot0123456k'p' = 112'r' = 114

return 112 − 114 = −2 < 0 → apple comes before apricot. The remaining characters are never examined.

int compare(String a, String b) {
  int n = Math.min(a.length(), b.length());
  for (int k = 0; k < n; k++) {
    char x = a.charAt(k), y = b.charAt(k);
    if (x != y)
      return x - y;   // first difference
  }
  return a.length() - b.length();
}

String Sorting

Lexicographic Order: Words and Numbers

Word 1Word 2EDUCATIVEEDUCATED✓✓✓✓✓✓✗·'E' (69) < 'I' (73)→ Word 2 first
numericlexicographic23571113171923291113171922329357
  • Numbers sorted as strings are compared character by character from the left: "11" < "2" because '1' < '2'. Likewise, 123, 45, 6, 789 are already in lexicographic order.
  • For numeric order, convert to integers or left-pad with zeros to equal length ("02" < "11").

String Sorting

Comparison-Based String Sorting

void lexSort(String[] a) {
  int n = a.length;
  for (int i = 0; i < n - 1; i++)
    for (int j = i + 1; j < n; j++)
      if (a[i].compareTo(a[j]) > 0) {
        String tmp = a[i];
        a[i] = a[j];
        a[j] = tmp;
      }
}
  • Each position receives the lexicographically smallest of the remaining strings via swaps.
  • compareTo reads at most W characters (W: longest string).
  • Time: O(N²) comparisons × O(W) = O(N² · W).
  • With merge sort, O(N log N) comparisons → O(W · N log N).
InputOutput
eabcdf, abcde, bcdea, cdeab, deabcgabcde, bcdea, cdeab, deabcg, eabcdf
banana, orange, grape, apple, kiwiapple, banana, grape, kiwi, orange

Comparison sorts need Ω(N log N) comparisons. Radix sort makes no comparisons, so this lower bound does not apply to it.

String Sorting · Step by step

LSD Radix Sort: Strings

LSD: right to left, each pass a stable counting sort on that character (short strings get an empty character); equal keys keep the order of the previous pass. Example: cab, dab, bad, dad, fad, ace, bee, bed, add, ebb (W = 3).

beforecabdabbaddadfadacebeebedaddebbaftercabdabebbbaddadfadbedaddacebeebucketsb: cab dab ebbd: bad dad fad bed adde: ace bee

Pass 1 (character 3): distribute into buckets by the last character; strings in the same bucket (cab, dab) keep their input order.

beforecabdabebbbaddadfadbedaddacebeeaftercabdabbaddadfadebbaceaddbedbeebucketsa: cab dab bad dad fadb: ebbc: aced: adde: bed bee

Pass 2 (character 2): by the middle character; in bucket 'a', cab, dab, bad, dad, fad keep the previous order (b < d).

beforecabdabbaddadfadebbaceaddbedbeeafteraceaddbadbedbeecabdabdadebbfadbucketsa: ace addb: bad bed beec: cabd: dab dade: ebbf: fad

Pass 3 (character 1): by the first character; the array is now in lexicographic order: ace, add, bad, bed, bee, cab, dab, dad, ebb, fad.

String Sorting

LSD Radix Sort: Java Code

void lsdSort(String[] a) {
  int W = maxLen(a);               // longest string
  for (int d = W - 1; d >= 0; d--)
    countSort(a, d);               // right to left
}
void countSort(String[] a, int d) {
  int[] cnt = new int[256];        // ASCII
  for (String s : a) cnt[ch(s, d)]++;
  for (int r = 1; r < 256; r++) cnt[r] += cnt[r - 1];
  String[] aux = new String[a.length];
  for (int i = a.length - 1; i >= 0; i--)   // stable
    aux[--cnt[ch(a[i], d)]] = a[i];
  System.arraycopy(aux, 0, a, 0, a.length);
}
int ch(String s, int d) {          // short string → 0
  return d < s.length() ? s.charAt(d) : 0;
}
  • Count: cnt[c] is the number of strings whose d-th character is c.
  • Prefix sums: cnt[c] now marks where bucket c ends.
  • Placement runs from back to front, so equal keys keep their order (stability).
  • ch returns 0 for a string that is too short: "car" comes before "cart".

One pass costs O(N + R), with alphabet size R = 256; W passes → O(W · (N + R)).

String Sorting

MSD Radix Sort: Left to Right

BADGE\0BANNER\0COFFEE\0COMPARISON\0COMPUTER\0MIDNIGHT\0WANDER\0WARDROBE\0WORKER\0
  1. Split all strings into buckets by their first character (B, C, M, W).
  2. Recursively sort each bucket on the next character.
  3. Stop at a one-element bucket or at the end of a string (\0).
Green cells are the characters that must be read to fix the word's place (its distinguishing prefix). MSD examines only these: one character suffices for MIDNIGHT, five for COMPARISON / COMPUTER.

With long common prefixes MSD also reads every character. Since an R = 256 count array is expensive for small buckets, practical versions switch to insertion sort for small subarrays.

String Sorting

Time Complexity and Comparison

  • LSD: W passes (W: length of the longest string), each pass over N strings + R buckets → O(W · (N + R)).
  • With a fixed alphabet (R = 256) the cost is O(W · N): longest string length × number of items.
MethodTimeExtra memoryStable
Exchange sort + compareToO(N² · W)O(1)no
Merge sort + compareToO(W · N log N)O(N)yes
LSD radix sortO(W · (N + R))O(N + R)yes
MSD radix sortO(N · W) worstO(N + W · R)yes
When to use radix sort? LSD is ideal for short, fixed-length keys (license plates, postal codes, IP addresses, 3-letter codes). When lengths vary widely, prefer MSD or a comparison sort.

Regular Expressions

Regular Expressions

  • A regular expression (regex) is a string that describes a search pattern.
  • Used to find, validate, extract or replace every part of a text that matches the pattern.
  • It automates text processing: log analysis, form validation, data cleaning, lexical analysis (lexers) in compilers.
  • Theory: every regular expression has an equivalent finite state machine (Kleene's theorem).
PatternTextMatches
colou?rcolor colourcolor, colour
gr(a|e)ygray grey groygray, grey
\d+20.99 TL, 5 pcs20, 99, 5
\bcode\bcode codingonly "code"
In Java: java.util.regex (Pattern, Matcher) and String.matches, replaceAll, split. Inside a Java string the backslash is doubled: "\\d+".

Regular Expressions

Basic Operators and Concepts

Op.MeaningExampleFound inNot in
.Any single character (except newline)a.cabc, a7cac
*Zero or more of the preceding itemab*cac, abbcadc
+One or more of the preceding itemab+cabc, abbcac
?Zero or one of the preceding itemab?cac, abcabbc
[ ]One character from the set[ck]arcar, karbar
^Start of the string^ababccab
$End of the stringab$cababc
( )Grouping; defines a subexpression^(ab)+$ab, abababa

The columns show whether the string contains a part that matches the pattern (Matcher.find). *, +, ? apply only to the item right before them (a character, set or group).

Regular Expressions

More Operators and Special Characters

ExpressionMeaning
aa single 'a' character
abcexactly "abc"
{ }a given number of repeats: \d{4}
|alternatives: cat|dog
\bword boundary (start or end)
\da digit: [0-9]
\wa word character: [a-zA-Z0-9_]
\swhitespace: space, tab, newline
  • Uppercase forms are complements: \D a non-digit, \W a non-word character, \S a non-whitespace character.
  • A special character itself must be escaped: \. a dot, \+ a plus, \( a parenthesis.
  • \w matches a single character; write \w+ for a word.
In Java, \w covers only ASCII letters by default: the 'ş' in "şeker" does not match. For Turkish text use \p{L} or Pattern.UNICODE_CHARACTER_CLASS.

Regular Expressions

Character Classes

Character classes
[abc]'a', 'b' or 'c'
[a-z]any lowercase letter
[A-Z]any uppercase letter
[0-9]any digit
[a-zA-Z_]a letter or underscore
Negated character classes
[^abc]any character except 'a', 'b', 'c'
[^0-9]a non-digit character
[^0-9]+"ab12cd" → "ab", "cd"
  • ^ means "not" inside a bracket, at its start; outside, it marks the start of the string.
  • Inside a set, ., *, + are not special: [.+] matches a dot or a plus. A hyphen (-) denotes a range; to match it literally, put it first or last.
  • The range [a-z] is based on character codes; ç, ğ, ı, ö, ş, ü are not in it.

Regular Expressions

Repetition and Grouping

Quantifiers

a*zero or more 'a'
a+one or more 'a'
a?zero or one 'a'
a{3}exactly three 'a'
a{2,4}two to four 'a'
a{2,}at least two 'a'

Grouping

(abc)exactly "abc"; captured as a group
(a|b|c)'a', 'b' or 'c'; captured as a group
(ab)+"ab", "abab", …

In Java a captured group is read with m.group(1).

Greedy vs lazy: a{2,4} takes the longest possible piece of "aaaaa" (aaaa). A ? after the quantifier picks the shortest match: in "<b>x</b>", "<.+?>" first takes only "<b>", not the whole text.

Regular Expressions

Lookahead and Lookbehind

PatternNameMatches when"ab ac ba"
a(?=b)lookahead'a' followed by 'b'ab ac ba
a(?!b)negative lookahead'a' not followed by 'b'ab ac ba
(?<=b)alookbehind'a' preceded by 'b'ab ac ba
(?<!b)anegative lookbehind'a' not preceded by 'b'ab ac ba
  • Lookaround tests a position and consumes no characters: the match of a(?=b) is just "a"; the 'b' is not part of it.
  • Example: ^(?=.*\d)(?=.*[A-Z]).{8,}$ validates a password of at least 8 characters containing at least one digit and one uppercase letter.

Regular Expressions

Example: Validation Expressions

PurposeRegular expression
Email^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$
Phone^(\+\d{1,3}[- ]?)?\d{10}$
URL^https?:\/\/[^\s/$.?#].[^\s]*$
Email[…]+ user name, @, domain, \. a literal dot, an extension of at least 2 letters. ✓ ali.veli@giresun.edu.tr ✗ ali@tr ✗ a b@x.com
PhoneOptional (…)? country code: + and 1–3 digits, then a hyphen or space; then exactly 10 digits. ✓ +90 5321234567 ✓ 5321234567 ✗ 0532 123 45 67
URLs? http or https; the first character cannot be whitespace, /, $, ., ?, #; then non-whitespace characters. ✓ https://giresun.edu.tr ✗ ftp://x.com ✗ "http:// x"

These expressions are practical approximations; they do not cover the full standards (RFC 5322 email, RFC 3986 URI).

Regular Expressions

Parse Tree: A URL Example

url'http://'host':'port'/'word'.'host'4949''didit'word'.'host'csail'word'.'word'mit''edu'

A grammar for http://didit.csail.mit.edu:4949/:

url  ::= 'http://' host
         ':' port '/'
host ::= word '.' host
       | word
word ::= [a-z]+
port ::= [0-9]+
The grammar looks recursive, but it is only tail recursion; the equivalent regex is http://([a-z]+\.)*[a-z]+:[0-9]+/. Nested structures such as balanced parentheses cannot be described by a regex; they need a context-free grammar and a parser.

Regular Expressions

Regular Expressions in Java: Pattern and Matcher

String input = "Name: Ali Veli, Age: 30, Email: ali.veli@example.com";
Pattern p = Pattern.compile("Age: (\\d+),");  // compile
Matcher m = p.matcher(input);
if (m.find()) {                          // first match
  int age = Integer.parseInt(m.group(1)); // group 1: "30"
  System.out.println("Age: " + age);
}
Patterngroup(1)Explanation
Name: (.*?),Ali Velilazy .*?: stops at the first comma
Age: (\d+),30one or more digits
Email: (.*?)$ali.veli@example.comup to the end of the line
  • The same job can be done without regexes using indexOf and substring: find the start marker (e.g. "Age: ") and cut up to the end marker (",").
  • find() tries to match part of the text, matches() the whole of it.

Regular Expressions

Try It: Regex Tester

Uses the browser's JavaScript regex engine; its syntax is largely the same as Java's. Matches are highlighted and groups are listed.

Finite State Machines

Finite State Machines

  • A finite state machine (FSM) is a mathematical model that processes a string one character at a time.
  • It has finitely many states; its only memory is the current state.
  • Simple and easy to understand, yet it handles complex string analysis tasks (lexical analysis, protocols, pattern search).
Formal definition: M = (Q, Σ, δ, q₀, F)
q₀q₀q₁1001Q = {q₀, q₁}, Σ = {0, 1}F = {q₁} (double ring)strings "ending in 1"
States QThe distinct states the machine can be in.
Alphabet ΣThe set of input symbols.
Start state q₀Where processing begins.
Transitions δδ(state, symbol) → new state.
Accept states F ⊆ QIf the machine is in one of these when the string ends, the string is accepted.

Finite State Machines

Kinds of FSM: DFA, NFA, ε-NFA

DFADeterministic finite automaton: every (state, symbol) pair has exactly one transition. Simple to run: O(n).
NFANondeterministic finite automaton: a (state, symbol) pair may have zero, one or several transitions. The machine is in a set of states at once.
ε-NFAAn NFA that allows empty transitions (epsilon, ε) that read no symbol. Built directly from a regex (Thompson's construction).
Regexε-NFANFADFAThompsonε-closuresubset
  • All three recognize the same languages (the regular languages): subset construction turns any NFA into a DFA, though the number of states can grow to 2ⁿ in the worst case.
  • grep, lex/flex and compiler lexers use this chain.

Finite State Machines

Using It: Simulating a DFA

  1. Enter the start state.
  2. For each character, update the state from the transition table.
  3. When the string ends, check whether the final state is an accept state.
boolean accepts(int[][] delta, boolean[] accept,
                String s) {
  int state = 0;                    // start state
  for (char c : s.toCharArray())
    state = delta[state][c - '0'];  // one transition
  return accept[state];              // accepting?
}
The transition table has size int[|Q|][|Σ|]. Each character costs one table lookup: a string of length n is processed in O(n) time, with no backtracking.

Finite State Machines

Example: A Machine That Recognizes "nice"

nice≠n≠i≠c≠e1Start2n_found3i_found4c_found6Error7Success
  • A machine that recognizes "nice": each state stands for the prefix read so far (ε, n, ni, nic, nice).
  • If the expected character does not arrive (≠n, ≠i, …), the machine moves to Error and stays there (a trap state).
  • "nice" → 1 → 2 → 3 → 4 → 7: accept. "nine" → 1 → 2 → 3 → 6: reject (the third character 'n' ≠ 'c').
  • Any character after Success → Error ("nicer" is rejected); self-loops that are not drawn are implied.

Finite State Machines · Step by step

Example: A DFA over a Binary Alphabet

011111010000AEFDCB

Start: state A. Input: 10011.

011111010000AEFDCB

Symbol 1 is 1: δ(A, 1) = B.

011111010000AEFDCB

Symbol 2 is 0: δ(B, 0) = B.

011111010000AEFDCB

Symbol 3 is 0: δ(B, 0) = B.

011111010000AEFDCB

Symbol 4 is 1: δ(B, 1) = C.

011111010000AEFDCB

Symbol 5 is 1: δ(C, 1) = D. Input exhausted; D is an accept state → accept.

δ01
→ AEB
BBC
CBD
D *DA
EEF
FFA

→ start, * accept. One state per cell: a DFA.

Finite State Machines

Example: An Automaton That Searches for "ACACAGA"

ACACAGAAAAACC01234567
textGACACACAGA
state0123454567
  • State q = length of the longest prefix of the pattern that matches the end of the text read so far. All undrawn transitions go to 0. Reaching state 7 means a match (here at position 3).
  • When the 7th character 'C' arrives, go back 5 → 4: the end of "ACACAC" can continue as the prefix "ACAC". This is the automaton form of KMP's prefix function: preprocessing O(m · |Σ|), search O(n).

Finite State Machines · Step through the code

dfaSearch(dfa, "GACACACAGA", 7)

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

The automaton for ACACAGA is built in advance. State q = length of the longest pattern prefix that matches the end of the text. Only forward transitions are drawn here; the back transitions are on the previous slide. Initially q = 0.

ACACAGAG → 001234567tG0A1C2A3C4A5C6A7G8A9

t[0] = 'G' extends no prefix of the pattern: q stays 0.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

t[1] = 'A' is the next pattern character: 0 → 1.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

t[2] = 'C' is the next pattern character: 1 → 2.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

t[3] = 'A' is the next pattern character: 2 → 3.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

t[4] = 'C' is the next pattern character: 3 → 4.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

t[5] = 'A' is the next pattern character: 4 → 5.

ACACAGAC01234567tG0A1C2A3C4A5C6A7G8A9

t[6] = 'C' is not the expected 'G', but q is not reset: "ACACAC" ends with the prefix "ACAC" → 5 → 4. No backing up in the text.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

t[7] = 'A' is the next pattern character: 4 → 5.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

t[8] = 'G' is the next pattern character: 5 → 6.

ACACAGA01234567tG0A1C2A3C4A5C6A7G8A9

'A' takes 6 → 7 = m: pattern found; return i − m + 1 = 9 − 7 + 1 = 3.

int dfaSearch(int[][] dfa, String t, int m) {
  int q = 0;                     // state
  for (int i = 0; i < t.length(); i++) {
    q = dfa[q][t.charAt(i)];
    if (q == m)
      return i - m + 1;          // match
  }
  return -1;
}

Finite State Machines · Step through the code

Parsing Numbers with an FSM: Code Trace

$20.99 ve 5.STARTDIGITDECIMAL

Input "$20.99 or 5.". State START, buffer empty.

$20.99 ve 5.STARTDIGITDECIMAL

'$' is neither a digit nor a dot; state stays START, nothing happens.

$20.99 ve 5.STARTDIGITDECIMAL

'2' is a digit and the state is START → DIGIT; appended to the buffer.

$20.99 ve 5.STARTDIGITDECIMAL

'0' is a digit: state stays DIGIT; append to the buffer.

$20.99 ve 5.STARTDIGITDECIMAL

'.' arrives → move to DECIMAL; the dot is appended to the buffer.

$20.99 ve 5.STARTDIGITDECIMAL

'9' is a digit: state stays DECIMAL; append to the buffer.

$20.99 ve 5.STARTDIGITDECIMAL

'9' is a digit: state stays DECIMAL; append to the buffer.

$20.99 ve 5.STARTDIGITDECIMAL

' ' is not part of a number: the number ends, 20.99 is printed; back to START.

$20.99 ve 5.STARTDIGITDECIMAL

'o' is neither a digit nor a dot; state stays START, nothing happens.

$20.99 ve 5.STARTDIGITDECIMAL

'r' is neither a digit nor a dot; state stays START, nothing happens.

$20.99 ve 5.STARTDIGITDECIMAL

' ' is neither a digit nor a dot; state stays START, nothing happens.

$20.99 ve 5.STARTDIGITDECIMAL

'5' is a digit and the state is START → DIGIT; appended to the buffer.

$20.99 ve 5.STARTDIGITDECIMAL

'.' arrives → move to DECIMAL; the dot is appended to the buffer.

$20.99 ve 5.STARTDIGITDECIMAL

Input exhausted; the state is DECIMAL, so the buffer is printed: 5.. The trailing dot was included in the number (a flaw).

void findNumbers(String s) {
  State d = State.START;
  StringBuilder buf = new StringBuilder();
  for (char c : s.toCharArray()) {
    boolean digit = Character.isDigit(c);
    if (digit) {
      if (d == State.START) d = State.DIGIT;
      buf.append(c);
    } else if (c == '.' && d != State.DECIMAL) {
      d = State.DECIMAL;
      buf.append(c);
    } else if (d != State.START) {  // number ended
      System.out.println(buf);
      buf.setLength(0);
      d = State.START;
    }
  }
  if (d != State.START) System.out.println(buf);
}

Edit Distance

Levenshtein Distance (Edit Distance)

  • Levenshtein distance (V. Levenshtein, 1965) is a metric that measures how different two strings are.
  • It is the minimum number of single-character edits needed to turn one string into the other.
  • Used in autocorrect and predictive text, spell checking, DNA sequence comparison and fuzzy search.
  • Metric properties: d(a, a) = 0, d(a, b) = d(b, a), triangle inequality.
InsertionAdd a character: "sittin" → "sitting"
DeletionRemove a character: "shine" → "shin"
SubstitutionReplace one character with another: "kitten" → "sitten"
kitten → sitten (k→s) → sittin (e→i) → sitting (+g): d = 3. No shorter sequence exists; dynamic programming proves it.
kittensittingksiitttteinn–greplacereplaceinsert

Edit Distance

Edit Examples

HONDAHHOYUNNDDAAIHYUNDAI

HONDA → HYUNDAI: substitute O→Y, insert U, insert I: d = 3

rainsainshinshineshinerhineraineraineshinetshinetrhinetrainetraine(a) 3(b) 3(c) 4

substitution insertion deletion

  • (a) rain → sain → shin → shine · (b) shine → rhine → raine → rain (delete e) · (c) shine → tshine → trhine → traine → train (delete e). A struck-through orange letter is deleted.
  • The distance is symmetric: (a) and (b) are inverses of each other; insertions and deletions swap roles.

Edit Distance

Computing Levenshtein Distance

D[i][j] = minimum cost of turning the first i characters of a into the first j characters of b.

D[i][0] = i (i deletions) · D[0][j] = j (j insertions)
if ai = bj then D[i][j] = D[i−1][j−1]
otherwise D[i][j] = 1 + min( D[i−1][j] delete, D[i][j−1] insert, D[i−1][j−1] replace )
  • Dynamic programming: each subproblem is solved once and stored in an (m+1) × (n+1) table; the answer is in the bottom-right corner.
  • A cell depends only on its upper, left and upper-left neighbors → fill the table row by row.
  • Time O(m · n); memory O(m · n), or O(min(m, n)) with two rows if only the distance is needed.
D[i−1][j−1]D[i−1][j]D[i][j−1]D[i][j]replace / equaldeleteinsertup-leftup

Edit Distance · Step by step

DP Table: CARS → PAIRS

a = CARS (rows), b = PAIRS (columns)

εPAIRS
ε012345
C1
A2
R3
S4

Start: first row 0..5 (ε → P, PA, … : insertions), first column 0..4 (C, CA, … → ε : deletions).

εPAIRS
ε012345
C112345
A2
R3
S4

Row 1 (C) filled. C ≠ P: D[1][1] = 1 + min(0, 1, 1) = 1 (substitution).

εPAIRS
ε012345
C112345
A221234
R3
S4

Row 2 (A) filled. A = A: D[2][2] = D[1][1] = 1, no added cost.

εPAIRS
ε012345
C112345
A221234
R332223
S4

Row 3 (R) filled. R = R: D[3][4] = D[2][3] = 2. A ≠ I: D[2][3] = 1 + min(1, 2, 3) = 2.

εPAIRS
ε012345
C112345
A221234
R332223
S443332

Row 4 (S) filled. S = S: D[4][5] = D[3][4] = 2.

εPAIRS
ε012345
C112345
A221234
R332223
S443332

Result D[4][5] = 2. Reading back: substitute C→P, A equal, insert I, R and S equal: CARS → PARS → PAIRS.

Edit Distance

Levenshtein: Java Code

int levenshtein(String a, String b) {
  int m = a.length(), n = b.length();
  int[][] D = new int[m + 1][n + 1];
  for (int i = 0; i <= m; i++) D[i][0] = i;  // delete
  for (int j = 0; j <= n; j++) D[0][j] = j;  // insert
  for (int i = 1; i <= m; i++)
    for (int j = 1; j <= n; j++)
      if (a.charAt(i - 1) == b.charAt(j - 1))
        D[i][j] = D[i - 1][j - 1];            // equal
      else
        D[i][j] = 1 + Math.min(D[i - 1][j - 1], // replace
                  Math.min(D[i][j - 1],         // insert
                           D[i - 1][j]));       // delete
  return D[m][n];
}
  • Table index i corresponds to string index i − 1: row 0 of D is for the empty prefix.
  • Examples: kitten → sitting = 3, ercan → sercan = 1, apple → banana = 5.

Edit Distance · Step through the code

levenshtein("kar", "ark")

εarkε0123k1a2r3

a = "kar" (rows), b = "ark" (columns). Column 0 holds i deletions and row 0 holds j insertions: the cost of converting to or from the empty prefix.

εarkε0123k11a2r3

'k' ≠ 'a' → 1 + min(replace 0, insert 1, delete 1) = 1.

εarkε0123k112a2r3

'k' ≠ 'r' → 1 + min(replace 1, insert 1, delete 2) = 2.

εarkε0123k1122a2r3

a[0] = b[2] = 'k' → free: D[1][3] = up-left = 2.

εarkε0123k1122a21r3

a[1] = b[0] = 'a' → free: D[2][1] = up-left = 1.

εarkε0123k1122a212r3

'a' ≠ 'r' → 1 + min(replace 1, insert 1, delete 2) = 2.

εarkε0123k1122a2123r3

'a' ≠ 'k' → 1 + min(replace 2, insert 2, delete 2) = 3.

εarkε0123k1122a2123r32

'r' ≠ 'a' → 1 + min(replace 2, insert 3, delete 1) = 2.

εarkε0123k1122a2123r321

a[2] = b[1] = 'r' → free: D[3][2] = up-left = 1.

εarkε0123k1122a2123r3212

'r' ≠ 'k' → 1 + min(replace 2, insert 1, delete 3) = 2.

εarkε0123k1122a2123r3212

D[3][3] = 2: delete the leading k of "kar" (ar), then append k (ark). The green path shows these two edits.

int levenshtein(String a, String b) {
  int m = a.length(), n = b.length();
  int[][] D = new int[m + 1][n + 1];
  for (int i = 0; i <= m; i++)
    D[i][0] = i;            // delete
  for (int j = 0; j <= n; j++)
    D[0][j] = j;            // insert
  for (int i = 1; i <= m; i++)
    for (int j = 1; j <= n; j++)
      if (a.charAt(i-1) == b.charAt(j-1))
        D[i][j] = D[i-1][j-1];  // equal
      else
        D[i][j] = 1 + Math.min(D[i-1][j-1],
            Math.min(D[i][j-1], D[i-1][j]));
  return D[m][n];
}

Edit Distance

Comparing Similarity Measures

MeasureWhat it measuresTimeAGGTAB · GXTXAYB
Hammingequal lengths; number of differing positionsO(n)undefined (6 ≠ 7)
Levenshteinnumber of inserts / deletes / substitutionsO(m · n)4
LCSlongest common subsequenceO(m · n)4 (GTAB)
Longest common substringlongest contiguous common pieceO(m · n)1 (e.g. "G")
  • The LCS is the longest common subsequence that keeps relative order but need not be contiguous; it is found with a DP table like Levenshtein's (diagonal + 1 on a match, otherwise max(up, left)).
  • If only insertions and deletions are allowed, the distance is m + n − 2 · LCS: 6 + 7 − 8 = 5; Levenshtein, which also allows substitutions, brings it down to 4.
  • Longest common substring: on a match L[i][j] = L[i−1][j−1] + 1, otherwise 0; the answer is the largest value in the table (ercan · sercan → "ercan").
  • A similarity ratio: 1 − d / max(m, n). kitten · sitting: 1 − 3/7 ≈ 0.57.

String Transforms

Suffix Arrays

  • A suffix array holds the starting indices of all suffixes of a string, in sorted order.
  • Each suffix runs from some position of the string to its end. A suffix array is a compact representation of all substrings: every substring is a prefix of some suffix.
isuffix
0banana
1anana
2nana
3ana
4na
5a
rankSAsorted suffix
05a
13ana
21anana
30banana
44na
52nana

"banana" → SA = [5, 3, 1, 0, 4, 2]. Only n integers: O(n) memory versus the O(n²) nodes of a suffix trie. Uses: string search, sorting, genome sequence analysis, BWT and data compression.

String Transforms

Building a Suffix Array

MethodIdeaTime
Brute forceGenerate all suffixes and sort them with Arrays.sort; each comparison is O(n)O(n² log n)
Manber–Myers (1990)Prefix doubling: derive the order on 2k characters from the order on k characters without comparing letters, by radix sorting pairs of ranksO(n log n)
Larsson–Sadakane (1999)Same doubling; only groups not yet separated are re-sorted by comparing the ranks of later suffixesO(n log n)
DC3 (2003), SA-IS (2009)Recursively sort a subset of the suffixes and merge in the restO(n)
Prefix doubling runs in O(n log n) time, not linear. Linear-time construction came later, with algorithms such as DC3 and SA-IS.

String Transforms · Step by step

Prefix Doubling

The Manber–Myers idea on "banana": the order on 2k characters comes from the order on k characters. log₂ n rounds, each O(n) with radix sort.

ifirst 1 charkeynew rank
1a'a'0
3a'a'0
5a'a'0
0b'b'1
2n'n'2
4n'n'2

k = 1: sort the suffixes by their first character: a → 0, b → 1, n → 2. Equal ranks are not yet separated.

ifirst 2 charskeynew rank
5a(0, −1)0
1an(0, 2)1
3an(0, 2)1
0ba(1, 0)2
2na(2, 0)3
4na(2, 0)3

k = 2: key = (rank[i], rank[i + 1]), or −1 if i + 1 ≥ n. No letters are compared; only pairs of numbers are sorted.

ifirst 4 charskeynew rank
5a(0, −1)0
3ana(1, 0)1
1anan(1, 1)2
0bana(2, 3)3
4na(3, −1)4
2nana(3, 3)5

k = 4: key = (rank[i], rank[i + 2]), or −1 if i + 2 ≥ n. No letters are compared; only pairs of numbers are sorted. All ranks distinct → SA = [5, 3, 1, 0, 4, 2].

String Transforms

Searching with a Suffix Array

  • Since the suffixes are sorted, those that start with the pattern P form a contiguous range in SA.
  • Both ends of the range are found by binary search: O(log n) steps × O(m) per comparison = O(m log n).
  • Far less memory than a suffix trie: just n integers.
  • Add the LCP array (longest common prefix of adjacent suffixes) and the longest repeated substring is found in O(n).
rankSAsuffix
05a
13ana
21anana
30banana
44na
52nana
Example: P = "an" in "banana". The middle, rank 2 ("anana"), starts with "an"; the left- and right-boundary searches find the range [1, 2] → "an" occurs at 3 and 1.

String Transforms · Step through the code

search("banana", sa, "an")

rankSAsuffix05a13ana21anana30banana44na52nanalohi(end of array)

t = "banana", SA = [5, 3, 1, 0, 4, 2], p = "an". The suffixes starting with "an" form a contiguous range of the sorted array; two binary searches find its ends.

rankSAsuffix05a13ana21anana30banana44na52nanalohi

mid = 3: "banana" ≥ "an" → hi = mid. Range [0, 3).

rankSAsuffix05a13ana21anana30banana44na52nanalohi

mid = 1: "ana" ≥ "an" → hi = mid. Range [0, 1).

rankSAsuffix05a13ana21anana30banana44na52nanalo=hi

mid = 0: "a" < "an" → lo = mid + 1. Range [1, 1).

rankSAsuffix05a13ana21anana30banana44na52nanalohi(end of array)

First loop done: the first suffix not less than "an" is at rank 1 → first = 1. For the second search, hi is reset to the end of the array.

rankSAsuffix05a13ana21anana30banana44na52nanalohi

mid = 3: "banana" does not start with "an" → hi = mid.

rankSAsuffix05a13ana21anana30banana44na52nanalo=hifirst

mid = 2: "anana" starts with "an" → lo = mid + 1.

rankSAsuffix05a13ana21anana30banana44na52nanalo=hifirst

Result [1, 3): ranks 1 and 2 → SA values 3 and 1. "an" occurs in banana at positions 3 and 1; 5 comparisons sufficed (each binary search takes at most ⌊log₂ 6⌋ + 1 = 3).

int[] search(String t, int[] sa, String p) {
  int lo = 0, hi = sa.length;
  while (lo < hi) {   // first >= p
    int mid = (lo + hi) / 2;
    if (t.substring(sa[mid])
          .compareTo(p) < 0) lo = mid + 1;
    else hi = mid;
  }
  int first = lo;
  hi = sa.length;
  while (lo < hi) {   // not prefixed by p
    int mid = (lo + hi) / 2;
    if (t.startsWith(p, sa[mid]))
      lo = mid + 1;
    else hi = mid;
  }
  return new int[] { first, lo };
}

String Transforms

The Burrows–Wheeler Transform (BWT)

  • The Burrows–Wheeler transform (BWT, 1994) rearranges the input: it changes no characters, only their positions.
  • It brings equal characters together: characters that precede similar contexts end up next to each other.
  • It improves the performance of compression algorithms.
  • It is reversible: the original data can be recovered from the transform.

Steps

  1. Generate all cyclic rotations of the input string.
  2. Sort the rotations alphabetically.
  3. Form a new string from the last characters of the sorted rotations (column L).
  4. Store the row number of the original string in the sorted list (or append a unique $).
rotationssortedaba$ba$aa$ab$aba$abaa$ababa$ba$aFLL = "ab$a"
The bzip2 pipeline: BWT → Move-to-Front (repeats become small numbers) → Run-Length Encoding → Huffman coding.

String Transforms · Step by step

BWT Example: BANANA

Input: BANANA

rotationsBANANAANANABNANABAANABANNABANAABANAN012345

1. All rotations: BANANA, ANANAB, NANABA, ANABAN, NABANA, ABANAN (moving the first character to the end).

rotationsBANANAANANABNANABAANABANNABANAABANANsortedABANANANABANANANABBANANANABANANANABA012345

2. Sorted: ABANAN, ANABAN, ANANAB, BANANA, NABANA, NANABA. The original string is in row 3 (counting from 0).

sortedABANANANABANANANABBANANANABANANANABA012345NNBAAABWT = NNBAAA, row 3

3. Last characters (column L): NNBAAA. Output: (NNBAAA, 3). Equal letters are grouped: NN, AAA.

String Transforms

BWT Example: abraca

S = abraca (initial)abracabracaaracaabacaabrcaabraaabracsorted matrixaabracabracaacaabrbracaacaabraracaab← L
  • Last column of the sorted matrix: L = caraab.
  • The original "abraca" is in row 1 of the sorted matrix (counting from 0); this index is stored for the inverse transform.
  • The first column (F = aaabcr) is just L's letters in sorted order; knowing L gives F.
The direct method generates n rotations with substring and sorts them with Arrays.sort → O(n² log n) time, O(n²) memory. In practice it is derived from the suffix array in O(n) via L[i] = T[SA[i] − 1] (T is the text with $ appended; L[i] = $ if SA[i] = 0; the formula is for the $ version). The $ and cyclic BWTs differ: cyclic BANANA → NNBAAA, BANANA$ → ANNB$AA.

String Transforms

Inverting the BWT (LF Mapping)

iF…L
0$banana
1a$banan
2ana$ban
3anana$b
4banana$
5na$bana
6nana$ba
  • T = banana$ ($ smallest, unique) → BWT = annb$aa.
  • F = sort(L) = $aaabnn.
  • LF rule: the k-th 'c' in L is the same character as the k-th 'c' in F. In row i, L[i] immediately precedes F[i] in the text.
  • Start at row 0, which begins with $; read L[i] and go to row LF(i):
rows 0 → 1 → 5 → 2 → 6 → 3 → 4
L read: a n a n a b → reverse → banana

With the ranks (each character's occurrence number in L) precomputed, inversion takes O(n) time. Without $, store the original row index instead (3, as in the BANANA example).

String Transforms · Step through the code

inverseBWT("annb$aa")

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$abnC[c]: # of chars smaller than cs

L = "annb$aa" (the BWT of banana$). rank[i]: which occurrence of its character L[i] is in L (from 0). E.g. L[6] = a, the third a → 2.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than cs

C[c] = number of characters smaller than c = the row where c starts in column F: $ → 0, a → 1, b → 4, n → 5.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than csaa

i = 0: L[0] = 'a' is prepended → s = "a". LF(0) = C['a'] + rank[0] = 1 + 0 = 1.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than csnan

i = 1: L[1] = 'n' is prepended → s = "na". LF(1) = C['n'] + rank[1] = 5 + 0 = 5.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than csanaa

i = 5: L[5] = 'a' is prepended → s = "ana". LF(5) = C['a'] + rank[5] = 1 + 1 = 2.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than csnanan

i = 2: L[2] = 'n' is prepended → s = "nana". LF(2) = C['n'] + rank[2] = 5 + 1 = 6.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than csananaa

i = 6: L[6] = 'a' is prepended → s = "anana". LF(6) = C['a'] + rank[6] = 1 + 2 = 3.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than csbananab

i = 3: L[3] = 'b' is prepended → s = "banana". LF(3) = C['b'] + rank[3] = 4 + 0 = 4.

iLrankF0a0$1n0a2n1a3b0a4$0b5a1n6a2nC[c]$0a1b4n5C[c]: # of chars smaller than csbananab

Rows 0 → 1 → 5 → 2 → 6 → 3 → 4 were visited; the next row would end in $, so the loop stops after n − 1 = 6 rounds. Result: banana.

String inverseBWT(String L) {   // with $
  int n = L.length();
  int[] rank = new int[n];
  int[] cnt = new int[256];
  int[] C = new int[256];
  for (int i = 0; i < n; i++)
    rank[i] = cnt[L.charAt(i)]++;
  for (int c = 1; c < 256; c++)
    C[c] = C[c - 1] + cnt[c - 1];
  StringBuilder s = new StringBuilder();
  int i = 0;                   // $ row
  for (int k = 0; k < n - 1; k++) {
    char c = L.charAt(i);
    s.insert(0, c);
    i = C[c] + rank[i];        // LF(i)
  }
  return s.toString();
}

Summary

Summary

TopicKey points
Lexicographic orderthe first differing character decides; a prefix comes first; code order ≠ Turkish alphabet
Radix sortLSD: stable counting sort right to left, O(W · (N + R)); MSD: from the left, recursing into buckets
Regular expressions. * + ? [ ] ^ $ ( ) { } | \d \w \s; lookarounds consume no characters
FSM(Q, Σ, δ, q₀, F); one transition per step in a DFA, O(n) scan; NFA ≡ DFA ≡ regex
Levenshtein1 + min(delete, insert, replace), or up-left if equal; O(m · n)
SimilarityHamming on equal lengths O(n); LCS and common substring O(m · n); insert/delete-only distance = m + n − 2 · LCS
Suffix arraysorted suffix indices; brute force O(n² log n), doubling O(n log n); search O(m log n)
BWTlast column of the sorted rotations + row index; inverted via LF mapping; bzip2

Summary

Common Mistakes

Mistaking code order for the alphabet"Zebra" < "apple" and "şeker" > "zeytin". Case and Turkish letters need a Collator or normalization.
Sorting numbers as strings"10" < "9". For numeric order, convert to integers or zero-pad to equal length.
An unstable sub-sort in LSDEvery pass must be stable, or the order from earlier passes is lost. In counting sort, place items from back to front.
Unescaped dot, greedy .*a.b also matches "a+b"; <.*> swallows the whole line. Use \. and the lazy .*?. Expressions like (a+)+$ cause exponential backtracking.
Off-by-one in the DP tableThe table is (m+1) × (n+1); D[i][j] compares charAt(i − 1). Not initializing the first row/column breaks the result.
Confusing subsequence and substringLCS does not require contiguity; longest common substring does. In BWT, forgetting to store the row index or the $ makes the transform irreversible.

Summary

Sample Problems

  1. Put the strings "10", "9", "100", "2" in lexicographic order.
  2. Apply LSD radix sort to sob, bat, cab, bot, tab; write the array after each pass.
  3. Which of "555-1234", "5551234", "55-12345" does ^\d{3}-\d{4}$ match?
  4. What is Levenshtein("sunday", "saturday")?
  5. What is Hamming("karolin", "kathrin")? Can the Levenshtein distance be larger?
  6. Find the BWT of "kayak" and the original row index.
Show answers
  1. "10" < "100" < "2" < "9" ("10" is a prefix of "100").
  2. Character 3: sob, cab, tab, bat, bot · character 2: cab, tab, bat, sob, bot · character 1: bat, bot, cab, sob, tab.
  3. Only "555-1234".
  4. 3: insert "a" and "t" (satunday), substitute n → r (saturday).
  5. 3 (r/t, o/h, l/r). No: with equal lengths, Hamming edits are just substitutions, so Levenshtein ≤ Hamming; here it is also 3.
  6. Sorted rotations: akkay, ayakk, kayak, kkaya, yakka → BWT = ykkaa, index 2.

Summary

Exercises

  1. Write MSD radix sort recursively.
  2. Write a regex that validates the Turkish mobile number format (0532 123 45 67).
  3. Draw a DFA that accepts binary strings with an even number of 1s and write its transition table.
  4. Compute Levenshtein with only two rows of memory, O(min(m, n)) extra space.
  5. The number-parsing FSM prints "5." on the input "… or 5.". Fix the machine.
  6. Find the suffix array of "mississippi".
Answers / hints
  1. sort(a, lo, hi, d): distribute into buckets by the d-th character (strings that have ended go to bucket −1), then call sort(…, d + 1) on each bucket.
  2. ^0\d{3} \d{3} \d{2} \d{2}$ (for 05 numbers only: ^05\d{2} …).
  3. Two states: Even (start, accept) and Odd. On 0: stay; on 1: Even ↔ Odd.
  4. Arrays prev and curr, swapped at the end of each row. Use the shorter string for the columns.
  5. Before printing, drop a trailing '.' from the buffer (or append the dot only when a digit follows).
  6. SA = [10, 7, 4, 1, 0, 9, 8, 6, 3, 5, 2] (i, ippi, issippi, ississippi, mississippi, pi, ppi, sippi, sissippi, ssippi, ssissippi).

Summary

References

  • Sedgewick and Wayne, Algorithms, 4th ed., 2011 (5.1 String Sorts, 5.4 Regular Expressions).
  • Cormen, Leiserson, Rivest and Stein, Introduction to Algorithms, 4th ed., 2022 (string-matching automata).
  • Hopcroft, Motwani and Ullman, Introduction to Automata Theory, Languages, and Computation, 3rd ed., 2006.
  • V. I. Levenshtein, "Binary codes capable of correcting deletions, insertions, and reversals", Soviet Physics Doklady 10(8), 1966.
  • R. A. Wagner and M. J. Fischer, "The String-to-String Correction Problem", Journal of the ACM 21(1), 1974.
  • U. Manber and G. Myers, "Suffix Arrays: A New Method for On-Line String Searches", SIAM Journal on Computing 22(5), 1993.
  • M. Burrows and D. J. Wheeler, "A Block-sorting Lossless Data Compression Algorithm", DEC SRC Research Report 124, 1994.
  • J. E. F. Friedl, Mastering Regular Expressions, 3rd ed., O'Reilly, 2006; Java java.util.regex.Pattern documentation.

The End

Chapter 10: Advanced String Algorithms
Previous: Chapter 9: String Matching and Compression · Next: Chapter 11: Tries
1 / 1 Sercan KÜLCÜ. All rights reserved.