Chapter 16: Sets and Union-Find
Introduction
Learning Outcomes
Represent pairwise disjoint sets as trees whose links point from child to parent.
Write find and union with a single parent array.
Show which sequence of operations degenerates the tree and costs O(n²).
Apply the weighting rule and the collapsing rule; interpret the lemmas.
Explain the Tarjan–van Leeuwen result and the inverse Ackermann function.
Apply union-find to counting connected components and to Kruskal's algorithm.
Disjoint Sets
Union and find on sets
The universe is {0, 1, …, n−1} and the sets are pairwise disjoint: S₁ ∩ S₂ = S₂ ∩ S₃ = S₁ ∩ S₃ = ∅. For n = 10:
Each set is a tree; links point from children to parents (arrows go up).
Disjoint Sets
Two operations, and two ways to unite
Si ∪ Sj = {x : x ∈ Si ∨ x ∈ Sj}. After a union the two sets no longer exist separately: S₁ ∪ S₂ = {0, 6, 7, 8, 1, 4, 9}.
Returns the set that contains i (the set's name = the root of its tree).
The roots are 0, 4 and 2. To unite two sets, make one root's parent link point to the other root.
S₁ ∪ S₂: 4's parent becomes 0; the tree of S₂ becomes a subtree of 0.
S₂ ∪ S₁: this time 0's parent becomes 4. Both are correct; the choice affects the cost.
Disjoint Sets
A set is named by its root
| Set name | link (root) |
|---|---|
| S₁ | → 0 |
| S₂ | → 4 |
| S₃ | → 2 |
If the names are needed as text, a separate table keeps a link to each root.
For union and find we identify each set with the root of its tree: instead of “set S₁” we say “set 0”.
So find(8) == find(6) asks “are 8 and 6 in the same set?”
Array Implementation
How should we implement it?
| Method | Union | Find | Problem |
|---|---|---|---|
| Linked list | O(1) | O(n) | slow search |
| Keep it sorted | — | — | sorting is O(n lg n) |
| parent array (tree) | O(1) | tree depth | depth must stay small |
Array Implementation
The parent array: one link per index
- The parent of 1 and 9 is 4; of 3 and 5, 2; of 6, 7 and 8, 0.
- 0, 2 and 4 are roots:
parent[root] = −1(negative = root).
#define N 10 int parent[N] = { -1, 4, -1, 2, -1, 2, 0, 0, 0, 4 };
Array Implementation
First version: find1 and union1
int find1(int i) { for (; parent[i] >= 0; i = parent[i]) ; /* climb to a root */ return i; } void union1(int i, int j) { parent[i] = j; /* put tree i under j */ }
Follows parent links until it reaches a negative entry (the root) and returns the root.
Given two roots, makes one tree a subtree of the other: O(1).
i and j must be roots; otherwise call find1 first.
Array implementation · step through the code
Code trace: find1(9), union1(4, 0)
find1(9): which set is 9 in?
parent[9] = 4 ≥ 0 → i = parent[i] = 4.
parent[4] = −1 < 0: a root. return 4 → 9 is in set 4.
union1(4, 0): two roots are passed.
parent[4] = 0: tree 4 now hangs under 0. One assignment: O(1).
find1(9) again: the path is one node longer.
parent[9] = 4 ≥ 0 → i = 4.
parent[4] = 0 ≥ 0 → i = 0.
return 0: 9 is now in set 0. Cost = depth of the node.
int find1(int i) { for (; parent[i] >= 0; i = parent[i]) ; /* climb to a root */ return i; } void union1(int i, int j) { parent[i] = j; /* put tree i under j */ }
Analysis · step through the code
Code trace: union1(0,1) … union1(3,4), find1(0)
n = 5 singleton sets Si = {i}, all with parent[i] = −1.
union1(0, 1): parent[0] = 1. Each call is O(1).
union1(1, 2): parent[1] = 2. Each call is O(1).
union1(2, 3): parent[2] = 3. Each call is O(1).
union1(3, 4): parent[3] = 4. Each call is O(1).
The tree has become a chain of depth n. Now find1(0).
i = parent[0] = 1 (step 1).
i = parent[1] = 2 (step 2).
i = parent[2] = 3 (step 3).
i = parent[3] = 4 (step 4).
return 4 after n−1 = 4 steps. find1(1), (2), … take n−2, n−3, … steps: in total Σ = O(n²).
int find1(int i) { for (; parent[i] >= 0; i = parent[i]) ; /* climb to a root */ return i; } void union1(int i, int j) { parent[i] = j; /* put tree i under j */ }
Analysis
Analysis of find1 and union1
/* start with n singleton sets */
union1(0, 1);
union1(1, 2);
...
union1(n-2, n-1);
find1(0);
find1(1);
...
find1(n-1);- n−1 unions, each constant → O(n) in total; no problem there.
- The tree degenerates into a chain of depth n.
- For the element i at level n−i, find1 takes O(n−i): n−1 steps for 0, n−2 for 1, …
- One find1 can cost O(n); n of them cost Σ i = O(n²).
Analysis
An aside: testing the worst case
For worst-case analysis, code should be tested on the data that crashes it or makes it run longest. Finding that data is hard.
The chain-building union order is exactly such an input for find1; random tests easily miss it.
Weighting Rule
The weighting rule
Definition: if tree i has fewer nodes than tree j, make j the parent of i; otherwise make i the parent of j. (The tree with more nodes becomes the parent.)
Weighting Rule
union2: the root stores minus the node count
void union2(int i, int j) { /* weighting rule; i, j are roots */ int p = parent[i] + parent[j]; /* -(size) */ if (parent[i] > parent[j]) { /* i smaller */ parent[i] = j; /* i under j */ parent[j] = p; /* j's new size */ } else { /* j not larger */ parent[j] = i; parent[i] = p; } }
A root holds −(number of nodes): −1 for a singleton, −8 for an 8-element set.
>?The values are negative: parent[i] > parent[j] means i has fewer nodes (−2 > −4).
Still O(1), with no extra array.
Weighting rule · step through the code
Code trace: union2 (0,1) (2,3) (4,5) (6,7)
union2(0, 1): set 0 has 1 node, set 1 has 1 node.
p = -1 + (-1) = -2: the merged set has 2 nodes.
-1 > -1 is false (j is not larger) → else: j goes under i.
parent[1] = 0, parent[0] = -2. The deepest node is on level 2.
union2(2, 3): set 2 has 1 node, set 3 has 1 node.
p = -1 + (-1) = -2: the merged set has 2 nodes.
-1 > -1 is false (j is not larger) → else: j goes under i.
parent[3] = 2, parent[2] = -2. The deepest node is on level 2.
union2(4, 5): set 4 has 1 node, set 5 has 1 node.
p = -1 + (-1) = -2: the merged set has 2 nodes.
-1 > -1 is false (j is not larger) → else: j goes under i.
parent[5] = 4, parent[4] = -2. The deepest node is on level 2.
union2(6, 7): set 6 has 1 node, set 7 has 1 node.
p = -1 + (-1) = -2: the merged set has 2 nodes.
-1 > -1 is false (j is not larger) → else: j goes under i.
parent[7] = 6, parent[6] = -2. The deepest node is on level 2.
void union2(int i, int j) { /* weighting rule; i, j are roots */ int p = parent[i] + parent[j]; /* -(size) */ if (parent[i] > parent[j]) { /* i smaller */ parent[i] = j; /* i under j */ parent[j] = p; /* j's new size */ } else { /* j not larger */ parent[j] = i; parent[i] = p; } }
Weighting rule · step through the code
Code trace: union2 (0,2) (4,6) (0,4)
union2(0, 2): set 0 has 2 nodes, set 2 has 2 nodes.
p = -2 + (-2) = -4: the merged set has 4 nodes.
-2 > -2 is false (j is not larger) → else: j goes under i.
parent[2] = 0, parent[0] = -4. The deepest node is on level 3.
union2(4, 6): set 4 has 2 nodes, set 6 has 2 nodes.
p = -2 + (-2) = -4: the merged set has 4 nodes.
-2 > -2 is false (j is not larger) → else: j goes under i.
parent[6] = 4, parent[4] = -4. The deepest node is on level 3.
union2(0, 4): set 0 has 4 nodes, set 4 has 4 nodes.
p = -4 + (-4) = -8: the merged set has 8 nodes.
-4 > -4 is false (j is not larger) → else: j goes under i.
parent[4] = 0, parent[0] = -8. The deepest node is on level 4.
void union2(int i, int j) { /* weighting rule; i, j are roots */ int p = parent[i] + parent[j]; /* -(size) */ if (parent[i] > parent[j]) { /* i smaller */ parent[i] = j; /* i under j */ parent[j] = p; /* j's new size */ } else { /* j not larger */ parent[j] = i; parent[i] = p; } }
Weighting Rule
Question: how badly can the tree degenerate?
union2 traces). Each union deepens every node of the smaller tree by 1, but the tree at least doubles.Weighting Rule
Lemma: weighted union keeps the depth logarithmic
union2 operations, then the depth (number of levels) of T is at most ⌊lg m⌋ + 1.- Proof idea (induction): a node gets deeper by 1 only when its tree is attached to a tree that is no smaller.
- Then the tree containing the node at least doubles.
- A size can double at most lg m times before reaching m → depth ≤ ⌊lg m⌋ + 1.
Collapsing Rule
The collapsing rule
In the final tree, find(7) reaches the root in 3 steps. If a program calls find(7) over and over, the cost piles up. Even when the smaller tree goes under the larger one the depth can grow; since we cannot control the depth, we go one step further:
Collapsing Rule
find2: find the root, then link the path to it
int find2(int i) { int r, s; for (r = i; parent[r] >= 0; r = parent[r]) ; /* find the root */ while (i != r) { /* collapse path */ s = parent[i]; /* save the parent */ parent[i] = r; /* link i to root */ i = s; /* move up */ } return r; }
r finds the root (the negative entry).
Every node on the path is linked straight to r. It also runs once for a node already linked to the root (useless but harmless).
Later finds on the same path reach the root in one step.
Collapsing rule · step through the code
Code trace: find2(7)
find2(7).
parent[7] = 6 ≥ 0 → r = 6.
parent[6] = 4 ≥ 0 → r = 4.
parent[4] = 0 ≥ 0 → r = 0.
parent[0] = -8 < 0: root r = 0.
i = 7 ≠ r: save s = parent[7] = 6.
parent[7] = 0: 7 now points to the root; i = s = 6.
i = 6 ≠ r: save s = parent[6] = 4.
parent[6] = 0: 6 now points to the root; i = s = 4.
i = 4 ≠ r: save s = parent[4] = 0.
parent[4] = 0 (already linked to the root: one extra round); i = 0.
i == r: the loop ends, return 0. parent = [-8, 0, 0, 2, 0, 4, 0, 0].
int find2(int i) { int r, s; for (r = i; parent[r] >= 0; r = parent[r]) ; /* find the root */ while (i != r) { /* collapse path */ s = parent[i]; /* save the parent */ parent[i] = r; /* link i to root */ i = s; /* move up */ } return r; }
Collapsing rule · step through the code
Code trace: find2(7) again, then find2(5)
find2(7).
parent[7] = 0 ≥ 0 → r = 0.
parent[0] = -8 < 0: root r = 0.
i = 7 ≠ r: save s = parent[7] = 0.
parent[7] = 0 (already linked to the root: one extra round); i = 0.
i == r: the loop ends, return 0. parent = [-8, 0, 0, 2, 0, 4, 0, 0].
find2(5).
parent[5] = 4 ≥ 0 → r = 4.
parent[4] = 0 ≥ 0 → r = 0.
parent[0] = -8 < 0: root r = 0.
i = 5 ≠ r: save s = parent[5] = 4.
parent[5] = 0: 5 now points to the root; i = s = 4.
i = 4 ≠ r: save s = parent[4] = 0.
parent[4] = 0 (already linked to the root: one extra round); i = 0.
i == r: the loop ends, return 0. parent = [-8, 0, 0, 2, 0, 0, 0, 0].
int find2(int i) { int r, s; for (r = i; parent[r] >= 0; r = parent[r]) ; /* find the root */ while (i != r) { /* collapse path */ s = parent[i]; /* save the parent */ parent[i] = r; /* link i to root */ i = s; /* move up */ } return r; }
Collapsing Rule
“We had a for loop, and now we added a while!”
| Call | find1 (steps) | find2 (steps + relinks) |
|---|---|---|
| 1st find(7) | 3 | 3 + 3 |
| 2nd find(7) | 3 | 1 + 1 |
| find(7) 10 times | 30 | 6 + 9·2 = 24 |
| find(5) (afterwards) | 2 | 2 + 2, then 1 + 1 |
Collapsing Rule
Lemma (Tarjan and van Leeuwen): almost constant
k₁·(n + f·α(f+n, n)) ≤ T(f, u) ≤ k₂·(n + f·α(f+n, n))
- α is the inverse Ackermann function; it grows extremely slowly.
- Even for as many elements as there are atoms in the universe, α ≤ 4 in practice.
- Divided by the number of operations, the cost is almost O(1).
/* Ackermann: grows very fast */
A(m, n) = n + 1 m = 0
= A(m-1, 1) m > 0, n = 0
= A(m-1, A(m, n-1)) m > 0, n > 0Collapsing Rule
The Ackermann function and its inverse
| n = 0 | n = 1 | n = 2 | n = 3 | n = 4 | |
|---|---|---|---|---|---|
| m = 0 | 1 | 2 | 3 | 4 | 5 |
| m = 1 | 2 | 3 | 4 | 5 | 6 |
| m = 2 | 3 | 5 | 7 | 9 | 11 |
| m = 3 | 5 | 13 | 29 | 61 | 125 |
| m = 4 | 13 | 65533 | 265536−3 | … | … |
- A(5, 1) = 22…2 − 3 (a tower of 65536 twos): unimaginably large.
- Inverse: α(m, n) = min{ i ≥ 1 : A(i, ⌊m/n⌋) ≥ lg n }. Since A grows so fast, α grows very slowly.
The Ackermann function has several definitions. With the one above, A(3,4) = 125; with Horowitz's definition (A(1,j) = 2j …), A(3,1) = 16. The conclusion is the same either way: in practice α ≤ 4.
Collapsing Rule
Exam question: write the Ackermann function recursively
Solution
unsigned long ackermann(unsigned long m, unsigned long n) { if (m == 0) return n + 1; if (n == 0) return ackermann(m - 1, 1); return ackermann(m - 1, ackermann(m, n - 1)); }
A(1,2) = A(0, A(1,1)) = A(0, A(0, A(1,0))) = A(0, A(0, A(0,1))) = A(0, A(0, 2)) = A(0, 3) = 4. Even A(4, 2) never finishes on any computer; the stack overflows.
Recursion · step through the code
Code trace: ackermann(1, 2)
ackermann(1, 2) is called.
A(1,2): the inner A(1,1) must be computed first.
A(1,1): the inner A(1,0) must be computed first.
A(1,0): n = 0 → call A(0, 1).
A(0,1): m = 0 → return 2.
A(1,0) = A(0,1) = 2; return.
The inner call gave 2 → outer call A(0,2).
A(0,2): m = 0 → return 3.
A(1,1) = 3.
The inner call gave 3 → outer call A(0,3).
A(0,3): m = 0 → return 4.
A(1,2) = 4.
unsigned long ackermann(unsigned long m, unsigned long n) { if (m == 0) return n + 1; if (n == 0) return ackermann(m - 1, 1); return ackermann(m - 1, ackermann(m, n - 1)); }
Collapsing Rule
An aside: where do these lemmas come from?
Application · step through the code
Code trace: counting connected components
Edges u–v: (0,1), (2,3), (1,4), (3,2). How many connected components are there?
Every node is its own set: parent = −1, count = 6.
Edge (0, 1): find2(0) = 0, find2(1) = 1.
0 ≠ 1: different components → union2(0, 1), count = 5.
Edge (2, 3): find2(2) = 2, find2(3) = 3.
2 ≠ 3: different components → union2(2, 3), count = 4.
Edge (1, 4): find2(1) = 0, find2(4) = 4.
0 ≠ 4: different components → union2(0, 4), count = 3.
Edge (3, 2): find2(3) = 2, find2(2) = 2.
a == b = 2: 3 and 2 are already in one component; the edge closes a cycle and the count stays.
return 3: the components are {0,1,4}, {2,3}, {5}. Total O(n + m·α).
int components(int n, int m, int u[], int v[]) { for (int i = 0; i < n; i++) parent[i] = -1; int count = n; /* all separate */ for (int e = 0; e < m; e++) { int a = find2(u[e]), b = find2(v[e]); if (a != b) { /* new component */ union2(a, b); count--; } } return count; }
Applications
Where else is union-find used?
Derive the classes from statements i ≡ j (Horowitz's equivalence problem).
Answer “are these two computers connected?” online (Sedgewick's opening example).
Group adjacent pixels of the same color into regions (connected-component labeling).
Is there an open path from top to bottom of a grid? The core of Monte Carlo experiments.
Unification in type inference; alias analysis.
Tarjan's lowest common ancestor algorithm.
Lab · C code step by step
Lab: Counting Components with Union-Find
Summary
Exam question: contents of the parent array
(i) What does the
parent array contain initially?(ii) What does
parent contain after union2(4, 2); find2(5); union2(0, 4); find2(7); find2(3);? Note: if (i) is wrong, (ii) is not graded.Solution
parent = [-4, 4, -3, 2, -3, 2, 0, 0, 0, 4]: each root holds minus its node count (−4, −3, −3).- union2(4,2): equal (−3, −3) → 2 goes under 4, parent[4] = −6. find2(5): 5 → 2 → 4, so 5 is linked straight to 4. union2(0,4): −4 > −6 → 0 goes under 4, parent[4] = −10. find2(7): 7 → 0 → 4, so 7 is linked to the root. find2(3): 3 → 2 → 4, so 3 is linked to the root. Result:
[4, 4, 4, 4, -10, 4, 0, 4, 0, 4].
Summary
Complexity summary
| Version | Union | Find (worst) | n−1 unions + m finds |
|---|---|---|---|
| union1 + find1 | O(1) | O(n) | O(n + m·n) |
| union2 (weighted) | O(1) | O(lg n) | O(n + m lg n) |
| union2 + find2 | O(1) | O(lg n) per call | O(n + m·α(m+n, n)) |
Memory: just the parent array of n integers. Since α ≤ 4, the last row is linear in practice.
Summary
Common mistakes
parent[i] = j cuts off i's subtree.parent array with −1 at the start.Summary
Exercises
- n = 6, parent all −1. What is parent after
union2(0,1); union2(2,3); union2(0,2); union2(4,5)? - With the weighting rule, what is the maximum depth of a 16-node tree?
- Write a union rule that stores the height (rank) instead of the weight. What changes?
- Write find2 recursively (one-line path compression).
- Why does rejecting an edge in Kruskal's algorithm mean a cycle?
- What does it cost to count the components of a graph with n nodes and m edges?
Show answers
[-4, 0, 0, 2, -2, 4](root 0 has 4 nodes; root 4 has 2).- ⌊lg 16⌋ + 1 = 5 levels.
- The root stores −(height); on a tie one root goes under the other and the height grows by 1. The depth is still O(lg n); with collapsing, rank becomes an upper bound.
int find(int i) { return parent[i] < 0 ? i : (parent[i] = find(parent[i])); }- The ends are already in the same set, so a path of chosen edges joins them; the new edge would close that path into a cycle.
- O(n + m·α(m+n, n)), linear in practice.
Summary
Chapter summary
Disjoint sets = trees with child-to-parent links, named by their roots.
A single parent array; a negative value marks a root.
Linking without a rule can build a chain: n finds cost O(n²).
Smaller tree under larger: depth ≤ ⌊lg m⌋ + 1.
The path is linked to the root; later finds take one step.
Together: α ≤ 4 per operation, almost O(1).
Summary
References and next steps
- E. Horowitz, S. Sahni, S. Anderson-Freed, Fundamentals of Data Structures in C, Section 5.10 (Set Representation).
- T. H. Cormen et al., Introduction to Algorithms (CLRS), chapter “Data Structures for Disjoint Sets”.
- R. Sedgewick, Algorithms in C, Chapter 1 (connectivity with union-find).
- R. E. Tarjan, J. van Leeuwen, “Worst-case analysis of set union algorithms”, J. ACM 31(2), 1984.