Chapter 13: Priority Queues and Heaps

Data Structures and Algorithms with C
Priority queueMax / min heapComplete binary tree in an arrayInsert: sift upDelete: sift downComparisonHeapify · d-ary
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Define

Define priority queues and max/min heaps; tell a heap apart from a binary search tree.

Build the representation

Store a complete binary tree in a 1-based array: parent i/2, children 2i, 2i+1.

Insert and delete

Trace insert_max_heap and delete_max_heap step by step.

Analyze

Show why insert and delete are O(lg n) and why the best case is Ω(1).

Compare

Compare heaps with unsorted/sorted arrays and lists using a cost table.

Go further

Explain why bottom-up heap construction (heapify) is O(n); meet the d-ary heap.

Priority Queues

Priority queue: the highest-priority item leaves first

The item removed first is chosen not by arrival order but by the highest or lowest value.

  • Max priority queue: the highest value is served first.
  • Min priority queue: the lowest value is served first.
Queue (FIFO): arrival orderA:3B:9C:5D:7▲ front: first in (A) leaves← add at the rearMax priority queueA:3B:9C:5D:7▲ highest priority (B:9) leaves
Instructor“We say priority queue; let's see how well we know the queue. Sometimes knowing things can get in the way. In the queue we know, you add at the rear and take from the front. This one is different…”

Priority Queues

Why do we need priority queues?

1 · Photocopier

Every user pays a flat fee (3 pages or 30, same price). To maximize revenue, serve the shortest job first → a min priority queue.

2 · Airport

Among planes waiting to land, the one with the least fuel must go first → a min priority queue.

Real systems work the same way: an operating system, for example, keeps processes in a priority queue.

Instructor“We want to maximize our revenue. Giving priority to the shortest usage time maximizes it.”
Instructor“We can't say ‘You're at the back of the queue. You're low on fuel, but that's the queue. What can we do?’”

Priority Queues

Abstract data type: priority queue

/* max priority queue interface */
bool is_empty(void);
bool is_full(void);
void insert(element item);     /* O(lg n) */
bool pop_max(element *out); /* pop + return */
bool peek_max(element *top);  /* no removal */
Insert

The item goes into the structure together with its priority.

Delete

The highest-priority item is returned and removed from the structure.

A priority queue is an interface; many data structures can implement it. The heap is just one, but it balances both: insert and delete in O(lg n).
Instructor“Once you grasp where and how to use it, the heap is a very, very important data structure.”

Heap Structure

A heap lives in an array as a complete binary tree

141122731048566
0–114212374105866

Index 0 of the array is unused; the root is heap[1].

NodeIndex
root1
parent of ii / 2
left child2i
right child2i + 1
last itemn

In a 0-based array: parent (i−1)/2, children 2i+1, 2i+2.

The array is not sorted; the only guarantee is that the item at index 1 has the highest priority (is removed first). A complete binary tree has no gaps, so no pointers are needed.

Heap Structure

A heap means a complete binary tree, not a search tree

472655173449845293Binary search tree931842553524345496477178269A max heap with the same keys
  • Complete binary tree: every level is full, and the last level fills from left to right.
  • In a search tree left < root < right; in a heap only parent ≥ child (no order between left and right).
Instructor“‘Sir, could you draw a binary search tree?’ By popular demand, let's draw one; we are not confusing the two!”

Heap Structure

Definition: max heap and min heap

Max heap: a complete binary tree in which every node's key is no smaller than its children's.

Min heap: a complete binary tree in which every node's key is no larger than its children's.

1411227310485660–114212374105866916233540–192633453012520–130225maxheap examples
21724310485660–12273441058661012028335040–1102203834501112120–111221minheap examples

These definitions let us implement max/min priority queues.

Heap Structure

Quick check: which is a max heap?

5013024031042053560–150230340410520635(a)5013024033542051060–150230340435520610(b)91728314250–1927384152(c)

Is heap[k/2] ≥ heap[k] for every k ≥ 2? Only parent–child pairs matter; no order is required between siblings or cousins.

Answer
  1. (a) Yes: 50≥30, 50≥40, 30≥10, 30≥20, 40≥35.
  2. (b) No: 35 at index 4 is larger than its parent 30.
  3. (c) Yes: complete, and every parent ≥ child (it does not matter that 1 and 2 are smaller than 8).

Heap Structure

Definitions in C: array, counter and macros

#define MAX_ELEMENTS 200   /* max items + 1 */
#define HEAP_FULL(n) ((n) == MAX_ELEMENTS - 1)
#define HEAP_EMPTY(n)  (!(n))

typedef struct {
  int x;          /* priority (key) */
  /* other fields */
} element;

element heap[MAX_ELEMENTS];  /* [0] unused */
int n = 0;                    /* item count */
Capacity

Indices 1..199 are used; the heap is full when n == 199.

Parentheses in macros

(n) prevents precedence bugs in calls like HEAP_EMPTY(a+b).

Common mistake: declaring the array as element heap[MAX_ELEMENTS - 1] (199 items, indices 0..198). Before the heap counts as full, n can reach 199 and heap[199] writes past the end; the size must be MAX_ELEMENTS.

Insertion

Inserting into a max heap: take the new slot, then sift

1411227310485660–1142123741058667891011startn = 6

Start: 14, 12, 7, 10, 8, 6. We will insert 15, 109, 107, 3, 15 in turn.

1411227310485667hole0–1142123741058667891011+15n = 7item15

First a new slot opens: the first free position of the complete tree (7).

1411223104856677hole0–11421237410586677891011+15n = 7item15

15 > 7: parent 7 moves down; the hole rises to position 3.

1122143104856677hole0–114212314410586677891011+15n = 7item15

15 > 14: parent 14 moves down; the hole rises to position 1.

1511221431048566770–115212314410586677891011+15n = 7

15 is written at position 1. The heap property holds again.

Instructor“The slot for the new item is certainly there. No doubt about it. It goes there first. Then, uh-oh! It had to be larger than its child. So we move it to the right position.”

Insertion

insert_max_heap: shifting, not swapping

void insert_max_heap(element item) {
  if (HEAP_FULL(n)) {
    heap_full();             /* error message */
    return;
  }
  int i = ++n;
  while (i != 1) {            /* i == 1: at root */
    if (item.x <= heap[i / 2].x) break;
    heap[i] = heap[i / 2];  /* move parent down */
    i /= 2;                   /* go up a level */
  }
  heap[i] = item;
}
The hole method

No swaps: larger ancestors shift down one level each, and the item is written once at the end (a swap costs 3 assignments, a shift 1).

Loop condition

i != 1: the root has no parent. When item.x <= parent, the slot is found.

Full heap

heap_full() is an error-message routine; we return without inserting.

Insertion · run the code step by step

Code trace: insert_max_heap(15)

1411227310485660–1142123741058667891011insert(15)n = 6

insert(15) is called; the heap holds 6 items: 14, 12, 7, 10, 8, 6.

1411227310485660–1142123741058667891011insert(15)n = 6

HEAP_FULL(6) is false: 6 ≠ 199, there is room.

1411227310485667ii/20–1142123741058667891011▲ i▲ i/2insert(15)n = 7item15

i = ++n = 7: the first free position of the complete tree. The slot is certainly there, but first we compare with the parent; item is not written yet.

1411227310485667ii/20–1142123741058667891011▲ i▲ i/2insert(15)n = 7item15

i = 7 ≠ 1: not at the root yet; enter the loop.

1411227310485667ii/20–1142123741058667891011▲ i▲ i/2insert(15)n = 7item15

15 ≤ 7? No. The new item is larger than its parent: the parent must move down.

1411223104856677ii/20–11421237410586677891011▲ i▲ i/2insert(15)n = 7item15

heap[7] = heap[3]: the parent (7) shifted down one level. No swap; the hole (orange) moved up.

1411223104856677i0–11421237410586677891011▲ iinsert(15)n = 7item15

i /= 2 → i = 3: move to the parent above.

1411223104856677ii/20–11421237410586677891011▲ i▲ i/2insert(15)n = 7item15

i = 3 ≠ 1: not at the root yet; enter the loop.

1411223104856677ii/20–11421237410586677891011▲ i▲ i/2insert(15)n = 7item15

15 ≤ 14? No. The new item is larger than its parent: the parent must move down.

1122143104856677ii/20–114212314410586677891011▲ i▲ i/2insert(15)n = 7item15

heap[3] = heap[1]: the parent (14) shifted down one level. No swap; the hole (orange) moved up.

1122143104856677i0–114212314410586677891011▲ iinsert(15)n = 7item15

i /= 2 → i = 1: move to the parent above.

1122143104856677i0–114212314410586677891011▲ iinsert(15)n = 7item15

i == 1: reached the root; the loop ends.

151122143104856677i0–115212314410586677891011▲ iinsert(15)n = 7

heap[1] = item: 15 is in place. Heap: 15, 12, 14, 10, 8, 6, 7.

void insert_max_heap(element item) {
  if (HEAP_FULL(n)) {
    heap_full();             /* error message */
    return;
  }
  int i = ++n;
  while (i != 1) {            /* i == 1: at root */
    if (item.x <= heap[i / 2].x) break;
    heap[i] = heap[i / 2];  /* move parent down */
    i /= 2;                   /* go up a level */
  }
  heap[i] = item;
}

Insertion · run the code step by step

Code trace: insert_max_heap(109)

1511221431048566770–115212314410586677891011insert(109)n = 7

insert(109): larger than every item. Worst case: it will climb to the root.

1511221431048566770–115212314410586677891011insert(109)n = 7

HEAP_FULL(7) is false: 7 ≠ 199, there is room.

1511221431048566778ii/20–115212314410586677891011▲ i▲ i/2insert(109)n = 8item109

i = ++n = 8: the first free position of the complete tree. The slot is certainly there, but first we compare with the parent; item is not written yet.

1511221431048566778ii/20–115212314410586677891011▲ i▲ i/2insert(109)n = 8item109

i = 8 ≠ 1: not at the root yet; enter the loop.

1511221431048566778ii/20–115212314410586677891011▲ i▲ i/2insert(109)n = 8item109

109 ≤ 10? No. The new item is larger than its parent: the parent must move down.

1511221434856677108ii/20–11521231441058667781091011▲ i▲ i/2insert(109)n = 8item109

heap[8] = heap[4]: the parent (10) shifted down one level. No swap; the hole (orange) moved up.

1511221434856677108i0–11521231441058667781091011▲ iinsert(109)n = 8item109

i /= 2 → i = 4: move to the parent above.

1511221434856677108ii/20–11521231441058667781091011▲ i▲ i/2insert(109)n = 8item109

i = 4 ≠ 1: not at the root yet; enter the loop.

1511221434856677108ii/20–11521231441058667781091011▲ i▲ i/2insert(109)n = 8item109

109 ≤ 12? No. The new item is larger than its parent: the parent must move down.

1512143124856677108ii/20–11521231441258667781091011▲ i▲ i/2insert(109)n = 8item109

heap[4] = heap[2]: the parent (12) shifted down one level. No swap; the hole (orange) moved up.

1512143124856677108i0–11521231441258667781091011▲ iinsert(109)n = 8item109

i /= 2 → i = 2: move to the parent above.

1512143124856677108ii/20–11521231441258667781091011▲ i▲ i/2insert(109)n = 8item109

i = 2 ≠ 1: not at the root yet; enter the loop.

1512143124856677108ii/20–11521231441258667781091011▲ i▲ i/2insert(109)n = 8item109

109 ≤ 15? No. The new item is larger than its parent: the parent must move down.

1152143124856677108ii/20–11521531441258667781091011▲ i▲ i/2insert(109)n = 8item109

heap[2] = heap[1]: the parent (15) shifted down one level. No swap; the hole (orange) moved up.

1152143124856677108i0–11521531441258667781091011▲ iinsert(109)n = 8item109

i /= 2 → i = 1: move to the parent above.

1152143124856677108i0–11521531441258667781091011▲ iinsert(109)n = 8item109

i == 1: reached the root; the loop ends.

1091152143124856677108i0–110921531441258667781091011▲ iinsert(109)n = 8

heap[1] = item: 109 is in place. Heap: 109, 15, 14, 12, 8, 6, 7, 10.

void insert_max_heap(element item) {
  if (HEAP_FULL(n)) {
    heap_full();             /* error message */
    return;
  }
  int i = ++n;
  while (i != 1) {            /* i == 1: at root */
    if (item.x <= heap[i / 2].x) break;
    heap[i] = heap[i / 2];  /* move parent down */
    i /= 2;                   /* go up a level */
  }
  heap[i] = item;
}

Insertion

Successive insertions: +107, +3, +15

10911521431248566771080–110921531441258667781091011startn = 8

Now: 109, 15, 14, 12, 8, 6, 7, 10. Next come 107, 3 and 15.

10911521431248566771089hole0–110921531441258667781091011+107n = 9item107

First a new slot opens: the first free position of the complete tree (9).

10911521434856677108129hole0–11092153144125866778109121011+107n = 9item107

107 > 12: parent 12 moves down; the hole rises to position 4.

10912143154856677108129hole0–11092153144155866778109121011+107n = 9item107

107 > 15: parent 15 moves down; the hole rises to position 2.

10912143154856677108129hole0–11092153144155866778109121011+107n = 9item107

107 ≤ 109: the parent is large enough; stop.

109110721431548566771081290–110921073144155866778109121011+107n = 9

107 is written at position 2. The heap property holds again.

1091107214315485667710812910hole0–110921073144155866778109121011+3n = 10item3

First a new slot opens: the first free position of the complete tree (10).

1091107214315485667710812910hole0–110921073144155866778109121011+3n = 10item3

3 ≤ 8: the parent is large enough; stop.

109110721431548566771081293100–1109210731441558667781091210311+3n = 10

3 is written at position 10. The heap property holds again.

1091107214315485667710812931011hole0–1109210731441558667781091210311+15n = 11item15

First a new slot opens: the first free position of the complete tree (11).

1091107214315456677108129310811hole0–11092107314415586677810912103118+15n = 11item15

15 > 8: parent 8 moves down; the hole rises to position 5.

1091107214315456677108129310811hole0–11092107314415586677810912103118+15n = 11item15

15 ≤ 107: the parent is large enough; stop.

1091107214315415566771081293108110–110921073144155156677810912103118+15n = 11

15 is written at position 5. The heap property holds again.

Inserting 3 needs no shifts at all (best case, Ω(1)); 109 climbed all the way to the root (worst case, O(lg n)).

Insertion

Insertion cost: at most the height of the tree

level 01 nodelevel 12 nodeslevel 24 nodeslevel 38 nodeslevel 416 nodesnumber of levels ⌈lg(n+1)⌉
CaseExampleShiftsCost
Bestinserting 30Ω(1)
Worstinserting 109the heightO(lg n)
General—0 … ⌊lg n⌋O(lg n)
A complete binary tree with n items has ⌈lg(n+1)⌉ levels. The loop climbs one level per iteration, so the cost is at most lg n and at least 1: time complexity O(lg n).

Deletion

Deleting from a max heap: take the root, place the last item

3011522831141453667108990–130215328411514637681099startn = 9

Data: 30, 15, 28, 11, 14, 3, 6, 10, 9. The first item out is 30 at the root.

115228311414536671080–13021532841151463768109delete → 30n = 8 · out = 30temp9

The root (30) is taken. The last item 9 is removed and becomes the candidate for the hole at the root.

281152311414536671080–12821532841151463768109delete → 30n = 8 · out = 30temp9

The larger of the two children (28) is larger than temp (9) → moved up.

2811529311414536671080–1282153941151463768109delete → 30n = 8 · out = 30

9 settles at position 3.

At each step three values matter: the two children and the last item. The largest goes to the slot i points to; i marks the slot to fill, j the children.

Instructor“Just taking an item is easy! But the job doesn't end there. We must delete it and keep the max priority queue structure intact.”

Deletion

delete_max_heap: the corrected version

bool delete_max_heap(element *out) {
  if (HEAP_EMPTY(n)) return false;
  *out = heap[1];          /* root is returned */
  element temp = heap[n--];  /* find its slot  */
  int i = 1, j = 2;
  while (j <= n) {
    if (j < n && heap[j].x < heap[j + 1].x) j++;
    if (temp.x >= heap[j].x) break;
    heap[i] = heap[j];      /* move child up    */
    i = j; j *= 2;
  }
  heap[i] = temp;
  return true;
}
Common mistake

Writing return 0; for an empty heap is a compile error (an integer does not convert to a struct).

The fix

The suggested solution: return a bool (success/failure) and pass the removed item back through a pointer parameter.

Why j < n?

It guarantees the right child exists before heap[j+1] is read.

Deletion · run the code step by step

Code trace: delete_max_heap (30)

3011522831141453667108990–130215328411514637681099delete_max_heap()n = 9

The call: the heap holds 9 items; the root is 30. The removed item goes into e.

3011522831141453667108990–130215328411514637681099delete_max_heap()n = 9

HEAP_EMPTY(9) is false; there is an item to remove.

3011522831141453667108990–130215328411514637681099delete_max_heap()n = 9 · out = 30

*out = heap[1] = 30: by definition the first item out is the first item of the array.

115228311414536671080–13021532841151463768109delete_max_heap()n = 8 · out = 30temp9

temp = heap[n--] = 9, n = 8. The last item is removed so the tree stays complete; we search for its slot starting at the root.

11522831141453667108ij0–13021532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

i = 1 (candidate slot for temp), j = 2 (i's left child).

11522831141453667108ij0–13021532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

j = 2 ≤ n = 8: i has a child; enter the loop.

11522831141453667108ij0–13021532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

The right child is larger (15 < 28) → j++, j = 3.

11522831141453667108ij0–13021532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

9 ≥ 28? No: the larger child must move up.

28115231141453667108ij0–12821532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

heap[1] = heap[3]: 28 moved up one level; the hole sank to position 3.

28115231141453667108ij0–12821532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

i = 3, j = 6: continue from the moved value's old slot.

28115231141453667108ij0–12821532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

j = 6 ≤ n = 8: i has a child; enter the loop.

28115231141453667108ij0–12821532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

The right child is larger (3 < 6) → j++, j = 7.

28115231141453667108ij0–12821532841151463768109▲ i▲ jdelete_max_heap()n = 8 · out = 30temp9

9 ≥ 6? Yes: temp is ≥ both children → break.

281152931141453667108i0–1282153941151463768109▲ idelete_max_heap()n = 8 · out = 30

heap[3] = temp (9). Heap: 28, 15, 9, 11, 14, 3, 6, 10.

2811529311414536671080–1282153941151463768109delete_max_heap()n = 8 · out = 30

return true; the caller gets 30 in out. The path has ≤ ⌊lg n⌋ levels → O(lg n).

bool delete_max_heap(element *out) {
  if (HEAP_EMPTY(n)) return false;
  *out = heap[1];          /* root is returned */
  element temp = heap[n--];  /* find its slot  */
  int i = 1, j = 2;
  while (j <= n) {
    if (j < n && heap[j].x < heap[j + 1].x) j++;
    if (temp.x >= heap[j].x) break;
    heap[i] = heap[j];      /* move child up    */
    i = j; j *= 2;
  }
  heap[i] = temp;
  return true;
}

Deletion · run the code step by step

Code trace: the second delete_max_heap (28)

2811529311414536671080–128215394115146376810delete_max_heap()n = 8

The second call: 28, 15, 9, 11, 14, 3, 6, 10. This time the last item (10) sinks all the way to a leaf.

2811529311414536671080–128215394115146376810delete_max_heap()n = 8

HEAP_EMPTY(8) is false; there is an item to remove.

2811529311414536671080–128215394115146376810delete_max_heap()n = 8 · out = 28

*out = heap[1] = 28: by definition the first item out is the first item of the array.

11529311414536670–1282153941151463768delete_max_heap()n = 7 · out = 28temp10

temp = heap[n--] = 10, n = 7. The last item is removed so the tree stays complete; we search for its slot starting at the root.

1152931141453667ij0–1282153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

i = 1 (candidate slot for temp), j = 2 (i's left child).

1152931141453667ij0–1282153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

j = 2 ≤ n = 7: i has a child; enter the loop.

1152931141453667ij0–1282153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

Left child (15) ≥ right child (9): j stays 2.

1152931141453667ij0–1282153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

10 ≥ 15? No: the larger child must move up.

1512931141453667ij0–1152153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

heap[1] = heap[2]: 15 moved up one level; the hole sank to position 2.

1512931141453667ij0–1152153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

i = 2, j = 4: continue from the moved value's old slot.

1512931141453667ij0–1152153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

j = 4 ≤ n = 7: i has a child; enter the loop.

1512931141453667ij0–1152153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

The right child is larger (11 < 14) → j++, j = 5.

1512931141453667ij0–1152153941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

10 ≥ 14? No: the larger child must move up.

1511429311453667ij0–1152143941151463768▲ i▲ jdelete_max_heap()n = 7 · out = 28temp10

heap[2] = heap[5]: 14 moved up one level; the hole sank to position 5.

1511429311453667i0–1152143941151463768▲ idelete_max_heap()n = 7 · out = 28temp10

i = 5, j = 10: continue from the moved value's old slot.

1511429311453667i0–1152143941151463768▲ idelete_max_heap()n = 7 · out = 28temp10

j = 10 > n = 7: i is a leaf; leave the loop.

151142931141053667i0–1152143941151063768▲ idelete_max_heap()n = 7 · out = 28

heap[5] = temp (10). Heap: 15, 14, 9, 11, 10, 3, 6.

1511429311410536670–1152143941151063768delete_max_heap()n = 7 · out = 28

return true; the caller gets 28 in out. The path has ≤ ⌊lg n⌋ levels → O(lg n).

bool delete_max_heap(element *out) {
  if (HEAP_EMPTY(n)) return false;
  *out = heap[1];          /* root is returned */
  element temp = heap[n--];  /* find its slot  */
  int i = 1, j = 2;
  while (j <= n) {
    if (j < n && heap[j].x < heap[j + 1].x) j++;
    if (temp.x >= heap[j].x) break;
    heap[i] = heap[j];      /* move child up    */
    i = j; j *= 2;
  }
  heap[i] = temp;
  return true;
}

Deletion

Delete until the heap is empty: items come out in decreasing order

1511429311410536670–11521439411510637689staten = 7

Now: 15, 14, 9, 11, 10, 3, 6. Removed so far: 30, 28.

1411129364105360–114211394651063789delete → 15n = 6 · out = 15

15 removed; the last item 6 settled at position 4. Output order: 30, 28, 15

1111029364350–1112103946536789delete → 14n = 5 · out = 14

14 removed; the last item 3 settled at position 5. Output order: 30, 28, 15, 14

1016293340–11026394356789delete → 11n = 4 · out = 11

11 removed; the last item 3 settled at position 4. Output order: 30, 28, 15, 14, 11

9162330–192633456789delete → 10n = 3 · out = 10

10 removed; the last item 3 settled at position 3. Output order: 30, 28, 15, 14, 11, 10

61320–16233456789delete → 9n = 2 · out = 9

9 removed; the last item 3 settled at position 2. Output order: 30, 28, 15, 14, 11, 10, 9

310–1323456789delete → 6n = 1 · out = 6

6 removed; the last item 3 settled at position 1. Output order: 30, 28, 15, 14, 11, 10, 9, 6

0–123456789delete → 3n = 0 · out = 3

3 removed; 3 was the last item, and the heap is empty. Output order: 30, 28, 15, 14, 11, 10, 9, 6, 3

Instructor“Now it's a matter of pride! ☺”

Deletion

Deletion cost and the journey of the last item

It may stop midway

In the 1st deletion, 9 settled at position 3 without sinking three levels.

It may go to the bottom

In the 2nd deletion, 10 sank to a leaf (position 5): the worst case.

Cost

The loop goes down one level per iteration: at most ⌊lg n⌋ iterations, 2 comparisons each → O(lg n).

A by-product

Deleting n times yields the items in decreasing order: this is the idea behind heap sort.

Deletion's best case is Ω(1) too: if the last item is ≥ both children of the root it settles at once (e.g. when 2 items are left).

Deletion

Writing the routines better: a struct instead of globals

typedef struct { element *a; int n, capacity; } Heap;

Heap *heap_create(int capacity) {
  Heap *h = malloc(sizeof *h);
  if (h == NULL) return NULL;
  h->a = malloc((capacity + 1) * sizeof *h->a);
  if (h->a == NULL) { free(h); return NULL; }
  h->n = 0; h->capacity = capacity;
  return h;
}
void heap_destroy(Heap *h) { if (h) { free(h->a); free(h); } }
Suggestion

Report success with TRUE/FALSE; pass the item and n by reference.

Here

The array and n live in a struct: there can be several heaps; malloc is always checked.

Instructor“The code in the book is not very correct anyway. But since the definitions are correct, as far as I can see it is an ideal book for learning data structures…”

Deletion

The min versions: rename and flip the comparisons

void insert_min_heap(element item) {
  /* ... same full check ... */
  int i = ++n;
  while (i != 1) {
    if (item.x >= heap[i / 2].x) break; /* >= ! */
    heap[i] = heap[i / 2];
    i /= 2;
  }
  heap[i] = item;
}
WhereMaxMin
insert: stopitem <= parentitem >= parent
delete: childlargersmaller
delete: stoptemp >= [j]temp <= [j]
Instructor“Change the routine's name and flip the less-than and greater-than signs, and you have successfully written a min heap.”

Deletion

When priority changes over time: the planes' fuel

Observation

Each plane burns fuel at a different rate: fuel levels must be updated every second by an interrupt.

In the heap

An item whose priority rises moves up, one whose priority falls moves down: O(lg n). Position k is kept in an index table.

/* priority of the item at k rose: sift it up */
void increase_key(int k, int key) {
  element e = heap[k];
  e.x = key;
  while (k != 1 && e.x > heap[k / 2].x) {
    heap[k] = heap[k / 2];
    k /= 2;
  }
  heap[k] = e;
}
Instructor“One plane may burn 10 gallons (37.85 liters) of fuel a second, while another can perhaps fly an hour on 10 gallons.”

Comparison

Why a heap? Wouldn't a simple array or list do?

#StructureInsertDelete (max)Why
1Unsorted arrayΘ(1)Θ(n)Append at the end; finding the max scans everything.
2Unsorted listΘ(1)Θ(n)Insert at the front; still a full scan.
3Sorted array (ascending)O(n)Θ(1)binsearch O(lg n) + n shifts; the max is at the end.
4Sorted linked list (descending)O(n)Θ(1)Find the slot; delete from the front, update the head.
5Max heapO(lg n)O(lg n)Just one root–leaf path.

When best (Ω) and worst (O) cases have the same order, the bound is tight and written with Θ. Without binsearch, insertion into a sorted array is Θ(n), not just O(n).

Instructor“‘What if we keep the largest value in a max variable?’ What happens when that plane lands? Then you would need a sorted array.”

Comparison

Let's put numbers into the cost table

#InsertDelete
1 · unsorted array12,000,000
2 · unsorted list12,000,000
3 · sorted array2,000,0001
4 · sorted list2,000,0001
5 · max heap≈ 21≈ 21

Assume one instruction = 1 second; lg 2,000,000 ≈ 20.9 (rounded to 21).

Instructor“I analyze 5 different approaches without writing any code, without touching a key.”
Instructor“I love the number 2 million because its logarithm is easy to take. […] Just for fun, I run the program built on a max heap 40,000 times. It takes as long as running the other structures once.”

Comparison

“Who will notice my algorithm is bad?”

Instructor“You may say ‘Instructions take microseconds, not seconds.’ […] but once it is part of an operating system, what a difference! ‘Come on! A plane arrives at the airport every half hour, every hour. Who will notice my algorithm is bad?’ […] In work that demands expertise, code written that way has no academic or scientific value.”
Instructor“The problem should be like this: insert an item, give me the smallest right away. […] If anyone still says ‘Sir, I like the sorted linked list,’ there is nothing I can do.”
Choose the data structure by the mix of operations. For a problem with frequent inserts and frequent “give me the smallest”, a heap doing both in O(lg n) beats structures that do one in Θ(1) and the other in Θ(n).

Comparison

Going further: binomial and Fibonacci heaps

OperationBinary heapBinomial heapFibonacci heap*
insertO(lg n)O(lg n)O(1)
find minO(1)O(lg n)O(1)
delete minO(lg n)O(lg n)O(lg n)
decrease keyO(lg n)O(lg n)O(1)
merge two heapsO(n)O(lg n)O(1)

* Fibonacci heap times are amortized (averaged over a sequence of operations) (CLRS). They give a theoretical gain in algorithms such as Dijkstra and Prim that do many “decrease key” operations.

Instructor“This structure is the binary heap. Beyond it there are Fibonacci heap and binomial heap algorithms. Even faster. Those people did not settle for lg n. — Citius! Altius! Fortius! (Faster! Higher! Stronger!)”

Advanced Topics

Heapify: turning an array into a heap in place

void build_heap(element a[], int n) {
  for (int k = n / 2; k >= 1; k--) {
    element temp = a[k];
    int i = k, j = 2 * k;
    while (j <= n) {
      if (j < n && a[j].x < a[j + 1].x) j++;
      if (temp.x >= a[j].x) break;
      a[i] = a[j];
      i = j; j *= 2;
    }
    a[i] = temp;
  }
}
The idea

Leaves (k > n/2) are already one-item heaps. From k = n/2 up to the root, each node sifts down: a root whose two children are heaps gets fixed.

What do we gain?

n calls to insert take O(n lg n); bottom-up construction takes O(n).

The inner loop

Identical to the loop inside delete_max_heap (adjust in Horowitz).

Heapify · run the code step by step

Code trace: build_heap — k = 3 and k = 2

5132839415126770–152338495161277build_heap(a, 7)

Input: 5, 3, 8, 9, 1, 12, 7 (not a heap yet). Start at n/2 = 3.

5132839415126770–152338495161277▲ kbuild_heap(a, 7)k = 3

k = 3: fix the subtree rooted at 3. Green nodes are already roots of heaps (leaves and those fixed earlier).

51323941512677ij0–152338495161277▲ ki▲ jbuild_heap(a, 7)k = 3temp8

temp = a[3] = 8; i = 3, j = 6. The same sift-down as in deletion begins.

51323941512677ij0–152338495161277▲ ki▲ jbuild_heap(a, 7)k = 3temp8

Children 12 and 7: the larger is a[6] = 12.

51321239415677ij0–1523312495161277▲ ki▲ jbuild_heap(a, 7)k = 3temp8

8 < 12 → a[3] = a[6]; the hole sinks to position 6.

51321239415677i0–1523312495161277▲ k▲ ibuild_heap(a, 7)k = 3temp8

i = 6, j = 12: continue from the hole's new position.

51321239415677i0–1523312495161277▲ k▲ ibuild_heap(a, 7)k = 3temp8

j = 12 > n: i = 6 is a leaf; the loop ends.

513212394158677i0–152331249516877▲ k▲ ibuild_heap(a, 7)k = 3

a[6] = temp (8). The subtree rooted at 3 is now a heap: 5, 3, 12, 9, 1, 8, 7.

5132123941586770–152331249516877▲ kbuild_heap(a, 7)k = 2

k = 2: fix the subtree rooted at 2. Green nodes are already roots of heaps (leaves and those fixed earlier).

51212394158677ij0–152331249516877▲ ki▲ jbuild_heap(a, 7)k = 2temp3

temp = a[2] = 3; i = 2, j = 4. The same sift-down as in deletion begins.

51212394158677ij0–152331249516877▲ ki▲ jbuild_heap(a, 7)k = 2temp3

Children 9 and 1: the larger is a[4] = 9.

51921234158677ij0–152931249516877▲ ki▲ jbuild_heap(a, 7)k = 2temp3

3 < 9 → a[2] = a[4]; the hole sinks to position 4.

51921234158677i0–152931249516877▲ k▲ ibuild_heap(a, 7)k = 2temp3

i = 4, j = 8: continue from the hole's new position.

51921234158677i0–152931249516877▲ k▲ ibuild_heap(a, 7)k = 2temp3

j = 8 > n: i = 4 is a leaf; the loop ends.

519212334158677i0–152931243516877▲ k▲ ibuild_heap(a, 7)k = 2

a[4] = temp (3). The subtree rooted at 2 is now a heap: 5, 9, 12, 3, 1, 8, 7.

void build_heap(element a[], int n) {
  for (int k = n / 2; k >= 1; k--) {
    element temp = a[k];
    int i = k, j = 2 * k;
    while (j <= n) {
      if (j < n && a[j].x < a[j + 1].x) j++;
      if (temp.x >= a[j].x) break;
      a[i] = a[j];
      i = j; j *= 2;
    }
    a[i] = temp;
  }
}

Heapify · run the code step by step

Code trace: build_heap — k = 1

5192123341586770–152931243516877▲ kbuild_heap(a, 7)k = 1

k = 1: fix the subtree rooted at 1. Green nodes are already roots of heaps (leaves and those fixed earlier).

19212334158677ij0–152931243516877▲ ki▲ jbuild_heap(a, 7)k = 1temp5

temp = a[1] = 5; i = 1, j = 2. The same sift-down as in deletion begins.

19212334158677ij0–152931243516877▲ ki▲ jbuild_heap(a, 7)k = 1temp5

Children 9 and 12: the larger is a[3] = 12.

12192334158677ij0–1122931243516877▲ ki▲ jbuild_heap(a, 7)k = 1temp5

5 < 12 → a[1] = a[3]; the hole sinks to position 3.

12192334158677ij0–1122931243516877▲ k▲ i▲ jbuild_heap(a, 7)k = 1temp5

i = 3, j = 6: continue from the hole's new position.

12192334158677ij0–1122931243516877▲ k▲ i▲ jbuild_heap(a, 7)k = 1temp5

Children 8 and 7: the larger is a[6] = 8.

12192833415677ij0–112293843516877▲ k▲ i▲ jbuild_heap(a, 7)k = 1temp5

5 < 8 → a[3] = a[6]; the hole sinks to position 6.

12192833415677i0–112293843516877▲ k▲ ibuild_heap(a, 7)k = 1temp5

i = 6, j = 12: continue from the hole's new position.

12192833415677i0–112293843516877▲ k▲ ibuild_heap(a, 7)k = 1temp5

j = 12 > n: i = 6 is a leaf; the loop ends.

121928334155677i0–112293843516577▲ k▲ ibuild_heap(a, 7)k = 1

a[6] = temp (5). The subtree rooted at 1 is now a heap: 12, 9, 8, 3, 1, 5, 7.

1219283341556770–112293843516577build_heap(a, 7)

k = 0: the loop is done. The whole array is a max heap: 12, 9, 8, 3, 1, 5, 7.

void build_heap(element a[], int n) {
  for (int k = n / 2; k >= 1; k--) {
    element temp = a[k];
    int i = k, j = 2 * k;
    while (j <= n) {
      if (j < n && a[j].x < a[j + 1].x) j++;
      if (temp.x >= a[j].x) break;
      a[i] = a[j];
      i = j; j *= 2;
    }
    a[i] = temp;
  }
}

Advanced Topics

Why is heapify O(n)?

level 0 (root)1 node × at most 3 moves = 3level 12 nodes × at most 2 moves = 4level 24 nodes × at most 1 move = 4level 3 (leaves)8 nodes × at most 0 moves = 0

Half the nodes are leaves and never move; a quarter move at most 1 level, an eighth at most 2… Total work:

Σh≥0 ⌈n / 2h+1⌉ · O(h) = O(n · Σ h/2h) = O(2n) = O(n)

For the 15-node complete tree in the example, the total is at most 0·8 + 1·4 + 2·2 + 3·1 = 11 < n. Inserting one by one may instead lift low nodes all the way to the root: n lg n.

Advanced Topics

d-ary heap: every node has d children

95060180270320450530675740810965102511
Binary (d = 2)d-ary (0-based)
parent(i−1)/2(i−1)/d
children2i+1, 2i+2d·i+1 … d·i+d
height≈ lg n≈ logd n
insert / increase keyO(lg n)O(logd n) — faster
delete rootO(lg n)O(d · logd n) — compares d children

With many inserts/decrease-keys and few deletes (e.g. Dijkstra on dense graphs), choosing d > 2 pays off; a shallow tree is cache-friendly.

Advanced Topics

Where are priority queues used?

Operating systems

Scheduling processes and interrupts, timer events.

Graph algorithms

Dijkstra's shortest paths and Prim's MST: picking the nearest vertex.

Heap sort

Build a heap, remove the root n times: O(n lg n), in place.

Event simulation

Events are pulled from a min heap by timestamp.

k largest / smallest

A min heap of size k (a max heap for the smallest): O(n lg k) on streaming data.

Huffman coding

Repeatedly merges the two smallest frequencies.

Lab · C code step by step

Lab: Max Heap

Summary

Complexity summary (binary heap with n items)

OperationTimeExtra spaceNote
peek at the maxO(1)O(1)heap[1]
insertΩ(1) … O(lg n)O(1)sift up
delete the maxO(lg n)O(1)sift down
change a priorityO(lg n)O(1)if the position is known
bottom-up constructionO(n)O(1)in place
building by n insertsO(n lg n)O(1)
search for an arbitrary itemO(n)O(1)a heap is not a search structure

Summary

Common mistakes

1Mixing up the 1-based (i/2, 2i) and 0-based ((i−1)/2, 2i+1) formulas.
2Declaring the array one smaller than the capacity: heap[199] writes out of bounds.
3Writing return 0; in a routine that returns an element (compile error).
4Forgetting the j < n test and reading heap[j+1] (outside the heap; outside the array in build_heap).
5Picking the first child instead of the larger one when deleting: the heap property breaks.
6Treating a heap as a sorted array or a binary search tree.

Summary

Exam-style question: insert, insert, … delete, delete

With sample dataInto the max heap heap[1..6] = 14, 12, 7, 10, 8, 6 we insert 15, 109, 107, 3, 15 in order; then delete_max_heap is called twice.
(i) What does the array hold after the insertions? (ii) Which items are removed, and what is the final array? (iii) Which insertion shifted least, and which most?
Solution
  1. 109, 107, 14, 15, 15, 6, 7, 10, 12, 3, 8 (n = 11).
  2. Removed: 109 and 107; final array 15, 15, 14, 12, 3, 6, 7, 10, 8 (n = 9).
  3. Inserting 3 shifted nothing (Ω(1)); 109 made 3 shifts up to the root (O(lg n)).

Summary

Exercises

  1. Insert 20, 5, 30, 10, 40 into an empty max heap; write the array after each insertion.
  2. Write delete_max_heap for a min heap. Which lines change?
  3. At most how many shifts does an insertion make in a heap of n = 1000 items?
  4. In a 1-based heap, at which indices are the parent and the children of node 13?
  5. If successive inserts are applied to 5, 3, 8, 9, 1, 12, 7 instead of build_heap, is the result the same?
  6. Where can the smallest item of a heap be? What does finding it cost?
Show answers
  1. 20 → 20, 5 → 30, 5, 20 → 30, 10, 20, 5 → 40, 30, 20, 5, 10.
  2. Child choice [j] > [j+1], stopping condition temp.x <= heap[j].x; the name is delete_min_heap.
  3. ⌈lg 1001⌉ = 10 levels → at most 9 shifts (⌊lg 1000⌋ = 9).
  4. Parent 13/2 = 6; children 26 and 27 (if n ≥ 27).
  5. No; successive insertion gives 12, 8, 9, 3, 1, 5, 7. Both are valid heaps, but they differ.
  6. If the keys are distinct, only among the leaves (index > n/2); about n/2 leaves are scanned → O(n).

Summary

Chapter summary

Priority queue

The first item out has the highest (max) or the lowest (min) value.

Heap

Complete binary tree + parent ≥ child. In a 1-based array the parent is i/2, the children 2i and 2i+1.

Insertion

Open the first free slot, shift larger ancestors down: O(lg n).

Deletion

Take the root, find a slot for the last item starting at the root, move larger children up: O(lg n).

Why a heap?

Other structures push one operation to Θ(n); a heap keeps both logarithmic.

Beyond

Heapify in O(n), d-ary heaps, binomial and Fibonacci heaps.

Summary

References and next steps

  • E. Horowitz, S. Sahni, S. Anderson-Freed, Fundamentals of Data Structures in C, Section 5.6 (Heaps), Chapter 7 (Heap Sort).
  • T. H. Cormen et al., Introduction to Algorithms (CLRS), Chapter 6 (Heapsort, priority queues) and the chapter on Fibonacci heaps.
  • R. Sedgewick, Algorithms in C, Parts 1–4, Chapter 9 (Priority Queues and Heapsort).

The End

Chapter 13: Priority Queues and Heaps
Next chapter: Chapter 14: Hash Tables · Course page
1 / 1 Data Structures in C