Chapter 5: Polynomials and Sparse Matrices

Data Structures and Algorithms with C
Polynomial ADTArray of termspaddMultiplication with listsTriplestranspose · fast_transposeSparse multiplicationCSR
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

Learning Outcomes

Choosing a representation

Compare a coefficient array and an array of terms for polynomials in space and time.

Merge logic

Add two sorted term arrays with padd in O(n + m) and analyze it.

Multiplying with lists

Multiply polynomials stored as linked lists, combining terms with equal exponents.

Sparse matrices

Store a matrix as ⟨row, col, value⟩ triples and say when this pays off.

Transposing

Trace transpose and fast_transpose; compare O(cols·terms) with O(cols + terms).

Multiplying

Build sparse matrix multiplication with the help of a transpose; get to know CSR.

Polynomial ADT and Representation · Horowitz 2.4

A polynomial: a set of ⟨exponent, coefficient⟩ pairs

A(x) = anxn + … + a1x + a0; each term aixi has a coefficient and an exponent. Degree: the largest exponent with a nonzero coefficient.

OperationMeaning
Zero(), IsZero(p)build / test the zero polynomial
Coef(p, e)coefficient of the term with exponent e (0 if none)
LeadExp(p)largest exponent
Attach(p, c, e), Remove(p, e)add / remove a term
SingleMult(p, c, e)p · c xe
Add(p1, p2), Mult(p1, p2)sum, product
Addition

A(x) + B(x) = Σ (ai + bi)·xi: coefficients of equal exponents are added.

Multiplication

A(x)·B(x) = Σ (aixi · Σ bjxj): exponents add, coefficients multiply.

Polynomial ADT and Representation

Method 1 (bad): keep every coefficient

#define MAX_DEGREE 101   /* max degree + 1 */
typedef struct {
  int degree;               /* largest exponent */
  float coef[MAX_DEGREE];   /* coef[i]: coef of x^i */
} polynomial;
polynomial P;
Pro

With degree stored, the degree is not recomputed each time; Coef and addition are very simple.

Con

Terms with coefficient 0 are stored too: a big waste of memory for a sparse polynomial.

00102030405060…9901005For P(x) = 5x¹⁰⁰, coef[0..99] hold useless zeros: 101 floats, 1 useful

Polynomial ADT and Representation

Method 2 (better): only the nonzero terms

#define MAX_TERMS 101   /* max number of terms */
typedef struct {
  float coef;            /* coefficient */
  int expon;             /* exponent */
} polynomial;
polynomial P[MAX_TERMS]; /* all polynomials here */
int avail = 0;           /* first free slot */

Terms are kept in decreasing exponent order; each polynomial is a range [start, finish] of P.

From the instructor “Merge logic lets us build more efficient algorithms on sorted structures.”
coefexpon[0]320[1]25[2]40[3]15[4]103[5]32[6]10[7][8]▲ sA▲ fA▲ sB▲ fB▲ avail

A(x) = 3x²⁰ + 2x⁵ + 4  ·  B(x) = x⁵ + 10x³ + 3x² + 1

Adding Polynomials

Adding by merging with two pointers

  • Equal exponents: add the coefficients; if the sum ≠ 0, append it to the result; both pointers advance.
  • A’s is larger: “B has no term with A’s exponent” — B’s remaining terms are all smaller. Append A’s term; only A advances.
  • B’s is larger: symmetric; append B’s term.
  • When one runs out, copy the rest of the other as is.
#define COMPARE(x, y) \
  ((x) < (y) ? -1 : (y) < (x) ? 1 : 0)

void attach(float coef, int expon) {
  if (avail >= MAX_TERMS) {
    fprintf(stderr, "Too many terms!\n"); exit(1);
  }
  P[avail].coef = coef;
  P[avail++].expon = expon;
}
#define COPY(k) attach(P[k].coef, P[k].expon)
Writing attach(int coef, …) truncates a float coefficient to int (2.5 → 2). The parameter must be float.

Adding Polynomials · Run the code step by step

padd: A + B, term by term

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6][7][8][9]▲ sa▲ sb▲ avail

A = 4x³ + 2x² + 5 → P[0..2], B = 3x³ − 2x² + x → P[3..5]. The result D starts at avail = 6: *sd = 6.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6][7][8][9]▲ sa▲ sb▲ avail

COMPARE(3, 3) = 0: equal exponents.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6][7][8][9]▲ sa▲ sb▲ avail

c = 4 + (3) = 7.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7][8][9]▲ sa▲ sb▲ avail

c ≠ 0 → attach(7, 3): P[6] = 7x³.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7][8][9]▲ sa▲ sb▲ avail

Both pointers advance.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7][8][9]▲ sa▲ sb▲ avail

COMPARE(2, 2) = 0: equal exponents.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7][8][9]▲ sa▲ sb▲ avail

c = 2 + (-2) = 0.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7][8][9]▲ sa▲ sb▲ avail

c = 0 → the term is not appended (the x² terms cancel).

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7][8][9]▲ sa▲ sb▲ avail

Both pointers advance.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7][8][9]▲ sa▲ sb▲ avail

COMPARE(0, 1) = -1: B’s exponent is larger.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7]11[8][9]▲ sa▲ avail

B’s term (x) is appended to D; only sb advances.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7]11[8][9]▲ sa▲ avail

sb = 6 > fb = 5 → B is done; leave the loop.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7]11[8]50[9]▲ avail

The 5 left in A is copied as is.

coefexpon[0]43[1]22[2]50[3]33[4]-22[5]11[6]73[7]11[8]50[9]▲ avail

*fd = 8. D(x) = P[6..8] = 7x³ + x + 5.

void padd(int sa,int fa,int sb,int fb,int*sd,int*fd){
  float c; *sd = avail;               /* start of D */
  while (sa <= fa && sb <= fb)
    switch (COMPARE(P[sa].expon, P[sb].expon)) {
    case -1: COPY(sb); sb++; break;   /* B larger */
    case  1: COPY(sa); sa++; break;   /* A larger */
    case  0: c = P[sa].coef + P[sb].coef;
      if (c) attach(c, P[sa].expon); /* skip 0 */
      sa++; sb++;
    }
  for (; sa <= fa; sa++) COPY(sa);    /* rest of A */
  for (; sb <= fb; sb++) COPY(sb);    /* rest of B */
  *fd = avail - 1;
}

Adding Polynomials

The complexity of padd

  • n terms in A, m in B: every loop round advances at least one pointer → at most n + m rounds.
  • The copy loops for the leftovers add at most n + m in total → O(n + m).
  • Best case: all exponents shared (n = m); every round is case 0 and both pointers move → Ω(n).
  • Without sorting, every term would need a search/insert (with shifting) in the result → about O(n²).
From the instructor “If O(n+m) is asked and you can do better, all the better!”
The price of a shared P array

Results are always appended at avail; the space of dead polynomials is never reclaimed. When P fills up we need compaction — or a linked list.

Adding Polynomials · homework

HOMEWORK 2: pmult, adding and multiplying n polynomials

  1. Write a routine pmult that multiplies two polynomials (in the P array representation) and analyze its performance.
  2. How would you add n polynomials? Design the data structure.
  3. How would you multiply n polynomials? Design the data structure.
Solution
  1. For each term aixei of A, multiply B by it with SingleMult (m terms) and add it to the running result with padd. The running result has at most n·m terms → O(n·(nm + m)) = O(n²m) in total. With the smaller polynomial in the outer loop: O(min(n,m)²·max(n,m)).
  2. Keep the polynomials in P with arrays start[k], finish[k]. Repeated padd: O(n·N) in the worst case for N terms in total. Better: advance all n pointers together and pick the largest exponent at each step; with a priority queue (heap) that is O(N log n).
  3. Multiplying left to right makes the intermediate results grow; multiplying in pairs (tree-wise) keeps the sizes balanced. Use the same start/finish tables or linked lists for the intermediate results.

Polynomials as Linked Lists · Horowitz 4.4

Storing a polynomial as a linked list

typedef struct poly_node *poly_pointer;
typedef struct poly_node {
  int coef;
  int expon;
  poly_pointer link;
} poly_node;
  • Only the nonzero terms, in decreasing exponent order.
  • The number of terms need not be known in advance; no shared array, no avail, no compaction.
  • Inserting/deleting in the middle is O(1) by changing pointers (once the place is found).
coefexponlink3142810∅

a = 3x¹⁴ + 2x⁸ + 1

Addition is still a merge

The list padd also walks with two pointers: O(n + m); result nodes come from malloc and must be freed when done (Horowitz 4.4).

Polynomials as Linked Lists

Helpers: new_term and delete_next_term

/* make a term; link it after prev if given */
poly_pointer new_term(int expon, int coef,
                      poly_pointer prev) {
  poly_pointer p = malloc(sizeof *p);
  if (!p) { printf("Out of memory!\n"); exit(1); }
  p->expon = expon;  p->coef = coef;
  if (prev) {                /* middle or end */
    p->link = prev->link;    /* new node first */
    prev->link = p;          /* then prev */
  } else p->link = NULL;     /* stands alone */
  return p;
}
/* delete the node after prev */
void delete_next_term(
        poly_pointer prev) {
  poly_pointer victim =
        prev->link;
  if (!victim) return;
  prev->link =
        victim->link;
  free(victim);
}

The address is kept in victim first so it can be freed after the link is updated; “the leftover in memory must be cleaned up”.

Polynomials as Linked Lists

Multiplying two polynomials given as lists

poly_pointer place_term(poly_pointer r,
                        int expon, int coef) {
  while (r->link && r->link->expon > expon) r = r->link;
  if (!r->link || r->link->expon < expon)
    new_term(expon, coef, r);          /* new exponent */
  else if ((r->link->coef += coef) == 0)
    delete_next_term(r);               /* became 0 */
  return r;
}
  • Every term of A is multiplied by every term of B: exponents add, coefficients multiply.
  • The result list must stay sorted; if a term with the same exponent exists its coefficient is updated, and if it becomes 0 the term is deleted.
  • start: the products of p’s next term cannot begin before the product of the previous one with qstart → the search starts there.
  • For readability the inner loop body was moved into place_term.

Polynomials as Linked Lists · Run the code step by step

poly_mult: (x + 1)·(x − 1)

coefexponlinkrstart = NULL (empty list)A = x + 1 · B = x − 1

We multiply A = x + 1 by B = x − 1 (expected x² − 1). The result list is empty.

coefexponlinkrstart = NULL (empty list)p: x q: —

p = x. r = start = NULL (no list yet).

coefexponlinkrstart = NULL (empty list)p: x q: x

expon = 1 + 1 = 2, coef = 1·(1) = 1.

coefexponlink12∅▲ rstart, r, startp: x q: x

The list is empty → the first node is x²; rstart = r = start.

coefexponlink12∅▲ r, startp: x q: x

q was the first term → start = r = &(1,2): p’s next term will begin its search here.

coefexponlink12∅▲ r, startp: x q: −1

expon = 1 + 0 = 1, coef = 1·(-1) = -1.

coefexponlink12-11∅▲ r, startp: x q: −1

place_term: no term with exponent 1 after r → a new node −x is linked after r.

coefexponlink12-11∅▲ r, startp: 1 q: —

p = 1. r = start: the search begins at node &(1,2), not at the head.

coefexponlink12-11∅▲ r, startp: 1 q: x

expon = 0 + 1 = 1, coef = 1·(1) = 1.

coefexponlink12∅▲ r, startp: 1 q: x

place_term: a term with exponent 1 exists; its coefficient −1 + 1 = 0 → deleted with delete_next_term(r).

coefexponlink12∅▲ r, startp: 1 q: x

q was the first term → start = r = &(1,2): p’s next term will begin its search here.

coefexponlink12∅▲ r, startp: 1 q: −1

expon = 0 + 0 = 0, coef = 1·(-1) = -1.

coefexponlink12-10∅▲ r, startp: 1 q: −1

place_term: no term with exponent 0 after r → a new node −1 is linked after r.

coefexponlink12-10∅▲ rstart(x + 1)(x − 1) = x² − 1

Result: x² − 1. n·m = 4 products; −x and +x summed to 0 and were deleted.

poly_pointer poly_mult(poly_pointer pstart,
                       poly_pointer qstart) {
  poly_pointer p, q, r = NULL, start = NULL, rstart=NULL;
  for (p = pstart; p; p = p->link) {
    r = start;             /* search from here */
    for (q = qstart; q; q = q->link) {
      int expon = p->expon + q->expon;
      int coef = p->coef * q->coef;
      if (rstart) r = place_term(r, expon, coef);
      else rstart=r=start=new_term(expon, coef, NULL);
      if (q == qstart) start = r;  /* for next p */
    }
  }
  return rstart;
}

Polynomials as Linked Lists

Other solutions to the problem (A: n terms, B: m terms)

methodcomplexity
1Multiply everything, append each product at the end; then sort and combineO(n²m²)
2Multiply everything; scan from the head each time and insert in placeO(n²m²)
3Build a list for the product with each term of A and merge it into the result at each stepO(n²m)
4Method 3 with the smaller polynomial outsideO(min(n,m)²·max(n,m))
5A separate list for each term of A; merge them all (in pairs) at the endO(m·n·lg n)
6Method 5 with the smaller polynomial outsideO(m·n·lg min(m,n))
7Use an array whose n-th index holds the n-th exponent—
From the instructor “An array whose n-th index holds the n-th exponent? NEVER!”

Sparse Matrices

A sparse matrix: most entries are zero

00151020322405-15101130002000-60030000004910000050028000

Only 8 of the 36 cells are nonzero

A precise definition is hard; storing it in a 2D array wastes memory on many zeros. Solution: store each nonzero entry as a ⟨row, col, value⟩ triple.

#define MAX_TERMS 101
#define MAX_COL 50   /* max columns */
typedef struct {
  int row, col, value;
} term;
term M[MAX_TERMS];

M[0] is the header: number of rows, number of columns, number of nonzero entries. The triples are sorted by row, then by column.

Sparse Matrices

The triple table, and when it pays off

rowcolvalueM[0]668M[1]0015M[2]0322M[3]05-15M[4]1111M[5]123M[6]23-6M[7]4091M[8]5228
Memory

3·(t + 1) integers instead of rows·cols. They break even at t ≈ rows·cols/3, i.e. when a third of the matrix is nonzero. Denser than that, this structure is not used.

In this example

3·9 = 27 integers instead of 36.

ADT operations

Create(max_row, max_col), Transpose(a), Add(a, b), Multiply(a, b) — Horowitz ADT 2.3. Other representations exist too (linked, CSR…).

Transposing

Transpose: M[i][j] → M′[j][i]

We write the rows as columns:

M1304→M′1034
  • Simply swapping row and col in the triples is not enough: the result breaks the row order.
  • Swapping and then sorting, or inserting each entry by shifting, is expensive.
  • Solution: the rows of the transpose are the columns of M; scan column by column and emit them in order.
From the instructor “If you are shifting and moving data around, you are not doing the data structure justice!”
Starting the inner loop of transpose at j = 0 also scans the header M[0]. (M[0].col = number of columns never equals any i, so the result is fine, but it is wasted work.) It should be j = 1.

Transposing · Run the code step by step

transpose: scan every term for each column

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]T[2]T[3]T[4]0071020331040022000M (3×4)

Header: T is 4×3 (rows ↔ columns), still 4 terms.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]T[3]T[4]column 00071020331040022000M (3×4)

Column 0: M[1].col = 0 matches → T[1] = (0, 0, 7), cur = 2.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]T[3]T[4]column 00071020331040022000M (3×4)

Column 0: M[2].col = 3 ≠ 0 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]T[3]T[4]column 00071020331040022000M (3×4)

Column 0: M[3].col = 1 ≠ 0 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]T[4]column 00071020331040022000M (3×4)

Column 0: M[4].col = 0 matches → T[2] = (0, 2, 2), cur = 3.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]T[4]column 10071020331040022000M (3×4)

Column 1: M[1].col = 0 ≠ 1 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]T[4]column 10071020331040022000M (3×4)

Column 1: M[2].col = 3 ≠ 1 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]column 10071020331040022000M (3×4)

Column 1: M[3].col = 1 matches → T[3] = (1, 1, 4), cur = 4.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]column 10071020331040022000M (3×4)

Column 1: M[4].col = 0 ≠ 1 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]column 20071020331040022000M (3×4)

Column 2: M[1].col = 0 ≠ 2 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]column 20071020331040022000M (3×4)

Column 2: M[2].col = 3 ≠ 2 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]column 20071020331040022000M (3×4)

Column 2: M[3].col = 1 ≠ 2 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]column 20071020331040022000M (3×4)

Column 2: M[4].col = 0 ≠ 2 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]column 30071020331040022000M (3×4)

Column 3: M[1].col = 0 ≠ 3 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]303column 30071020331040022000M (3×4)

Column 3: M[2].col = 3 matches → T[4] = (3, 0, 3), cur = 5.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]303column 30071020331040022000M (3×4)

Column 3: M[3].col = 1 ≠ 3 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]303column 30071020331040022000M (3×4)

Column 3: M[4].col = 0 ≠ 3 → skip.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]3030071020331040022000M (3×4)

Done: T is sorted by row for free (M’s rows were sorted). 4 columns × 4 terms = 16 comparisons.

void transpose(term M[], term T[]) {
  int n = M[0].value, i, j, cur;
  T[0].row = M[0].col;  T[0].col = M[0].row;
  T[0].value = n;
  if (n > 0) {
    cur = 1;                       /* next in T */
    for (i = 0; i < M[0].col; i++)     /* each col */
      for (j = 1; j <= n; j++)         /* each term */
        if (M[j].col == i) {
          T[cur].row = M[j].col; T[cur].col = M[j].row;
          T[cur].value = M[j].value;  cur++;
        }
  }
}

Transposing

The complexity of transpose

  • The outer loop runs cols times, the inner terms times → O(cols · terms).
  • For a dense matrix terms = rows·cols → O(cols² · rows): worse than the O(rows·cols) transpose of a 2D array!
  • We saved space but lost time — worthwhile only when terms ≪ rows·cols.
  • The problem: scanning the same array M again for every column.
The way out

If we knew in advance where each term goes in T, we could scan M once and put each term straight into place.

Example

If row 0 of the transpose has 2 entries and row 1 has 3: row 0 starts at T[1], row 1 at T[3], row 2 at T[6].

Transposing

fast_transpose: count first, then place

void fast_transpose(term M[], term T[]) {
  int i, j, nc = M[0].col, nt = M[0].value;
  int row_terms[MAX_COL], start_pos[MAX_COL];
  T[0].row = nc;  T[0].col = M[0].row;  T[0].value = nt;
  if (nt <= 0) return;                    /* empty matrix */
  for (i = 0; i < nc; i++) row_terms[i] = 0;
  for (i = 1; i <= nt; i++) row_terms[M[i].col]++;
  start_pos[0] = 1;
  for (i = 1; i < nc; i++)
    start_pos[i] = start_pos[i-1] + row_terms[i-1];
  for (i = 1; i <= nt; i++) {
    j = start_pos[M[i].col]++;       /* take slot, bump */
    T[j].row = M[i].col;  T[j].col = M[i].row;
    T[j].value = M[i].value;
  }
}
rowcolvalueT[0]668T[1]0015T[2]0491T[3]1111T[4]213T[5]2528T[6]3022T[7]32-6T[8]50-15

In the 6×6 example row_terms = [2, 1, 2, 2, 0, 1], start_pos = [1, 3, 4, 6, 8, 8].

Transposing · Run the code step by step

fast_transpose (1): count and find the starts

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_termsstart_poscolumn 0column 1column 2column 3

Copy the header: T is 4×3 with 4 terms. nt > 0 → go on.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms0000start_poscolumn 0column 1column 2column 3

Zero row_terms: a counter for each row of T (= each column of M).

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms1000start_poscolumn 0column 1column 2column 3

M[1] is in column 0 → row_terms[0]++ = 1.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms1001start_poscolumn 0column 1column 2column 3

M[2] is in column 3 → row_terms[3]++ = 1.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms1101start_poscolumn 0column 1column 2column 3

M[3] is in column 1 → row_terms[1]++ = 1.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_poscolumn 0column 1column 2column 3

M[4] is in column 0 → row_terms[0]++ = 2.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos1column 0column 1column 2column 3

The terms of column 0 start at T[1]: start_pos[0] = 1.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos13column 0column 1column 2column 3

start_pos[1] = 1 + 2 = 3.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos134column 0column 1column 2column 3

start_pos[2] = 3 + 1 = 4.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos1344column 0column 1column 2column 3

start_pos[3] = 4 + 0 = 4 (column 2 is empty, so column 3 also starts at 4).

void fast_transpose(term M[], term T[]) {
  int i, j, nc = M[0].col, nt = M[0].value;
  int row_terms[MAX_COL], start_pos[MAX_COL];
  T[0].row = nc;  T[0].col = M[0].row;
  T[0].value = nt;
  if (nt <= 0) return;                  /* empty */
  for (i = 0; i < nc; i++) row_terms[i] = 0;
  for (i = 1; i <= nt; i++) row_terms[M[i].col]++;
  start_pos[0] = 1;
  for (i = 1; i < nc; i++)
    start_pos[i] = start_pos[i-1] + row_terms[i-1];
  /* phase 2: put the terms in place */

Transposing · Run the code step by step

fast_transpose (2): put the terms in place

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos1344column 0column 1column 2column 3rowcolvalueT[0]434T[1]T[2]T[3]T[4]

M is scanned once, start to end; each term goes straight to its place.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos2344column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]T[3]T[4]

M[1] = (0, 0, 7): j = start_pos[0]++ → j = 1, start_pos[0] = 2.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos2344column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]T[3]T[4]

T[1] = (0, 0, 7): row and column swapped.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos2345column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]T[3]T[4]303

M[2] = (0, 3, 3): j = start_pos[3]++ → j = 4, start_pos[3] = 5.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos2345column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]T[3]T[4]303

T[4] = (3, 0, 3): row and column swapped.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos2445column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]T[3]114T[4]303

M[3] = (1, 1, 4): j = start_pos[1]++ → j = 3, start_pos[1] = 4.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos2445column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]T[3]114T[4]303

T[3] = (1, 1, 4): row and column swapped.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos3445column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]303

M[4] = (2, 0, 2): j = start_pos[0]++ → j = 2, start_pos[0] = 3.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos3445column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]303

T[2] = (0, 2, 2): row and column swapped.

rowcolvalueM[0]344M[1]007M[2]033M[3]114M[4]202row_terms2101start_pos3445column 0column 1column 2column 3rowcolvalueT[0]434T[1]007T[2]022T[3]114T[4]303

Done — the same result as transpose. Two linear scans: O(cols + terms).

  /* phase 1: row_terms and start_pos ready */
  for (i = 1; i <= nt; i++) {
    j = start_pos[M[i].col]++;  /* read, bump */
    T[j].row = M[i].col;
    T[j].col = M[i].row;
    T[j].value = M[i].value;
  }
}
From the instructor “A programmer must know the operator precedence of their language well.”

Transposing

Analysis, and a version with one array

Loops: num_cols + num_terms + num_cols + num_terms (+ constants):

2·terms + 2·cols + 3 = O(cols + terms)

For a dense matrix O(rows·cols): the same order as transposing a 2D array. The price: two auxiliary arrays (O(cols) space).

From the instructor “Remember I said a fast transpose can be done with a single array? Here it is…”
/* turn row_terms into start_pos in place */
for (i = 0, pos = 1; i < nc; i++) {
  int c = row_terms[i];  /* terms in this column */
  row_terms[i] = pos;  /* now a start position */
  pos += c;
}
for (i = 1; i <= nt; i++) {
  j = row_terms[M[i].col]++;
  T[j].row = M[i].col;  T[j].col = M[i].row;
  T[j].value = M[i].value;
}

The fast transpose can also be written with a single array. It is exactly counting sort, with the column number as the key.

Sparse Matrix Multiplication · Horowitz 2.5

Dm×p = Am×n · Bn×p

dij = Σk=0n−1 aik · bkj

Row i of A and column j of B are “multiplied and summed”. The product of two sparse matrices need not be sparse:

A100100100·B111000000=D111111111
  • In the triples, row i of A is contiguous, but column j of B is scattered.
  • So we first take the transpose of B (fast_transpose): B’s columns become consecutive rows of Bᵀ.
  • Then each row of A is multiplied by each row of Bᵀ by merging on the col field.
Saying “first transpose the first matrix” is wrong; in Horowitz’s mmult (and logically) it is the second matrix B that is transposed.

Sparse Matrix Multiplication · adapted from Horowitz Program 2.9

Helpers: end of row, dot product, store

/* where the row of the term at s ends */
int row_end(term t[], int s) {
  int e = s;
  while (e <= t[0].value &&
         t[e].row == t[s].row) e++;
  return e;
}
/* row a[x..ex) · row b[y..ey) */
int dot(term a[], int x, int ex,
        term b[], int y, int ey) {
  int sum = 0;
  while (x < ex && y < ey)
    if (a[x].col < b[y].col) x++;
    else if (a[x].col > b[y].col) y++;
    else sum += a[x++].value * b[y++].value;
  return sum;
}
void store(term d[], int *nd,
      int r, int c, int s) {
  if (s == 0) return;
  if (++*nd >= MAX_TERMS) {
    fprintf(stderr, "Too many terms\n");
    exit(1);
  }
  d[*nd].row = r;
  d[*nd].col = c;
  d[*nd].value = s;
}

dot is a two-pointer merge like padd: only terms with the same k (col) are multiplied. A zero sum is not stored.

Sparse Matrix Multiplication · Run the code step by step

Sparse multiplication: row of A · row of Bᵀ

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]220d[1]d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

A = [[1,0,2],[0,3,0]], B = [[4,0],[0,5],[6,0]]; the transpose of B, bt, was computed beforehand.

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]220d[1]d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

Row 0 of A: a[1..2].

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]220d[1]d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

Column 0 of B = bt[1..2].

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]221d[1]0016d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

dot: 1·4 + 2·6 = 16 → d[1] = (0, 0, 16).

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]221d[1]0016d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

Column 1 of B = bt[3..3].

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]221d[1]0016d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

dot: no common col = 0 → zero, not stored.

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]221d[1]0016d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

Row 1 of A: a[3..3].

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]221d[1]0016d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

Column 0 of B = bt[1..2].

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]221d[1]0016d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

dot: no common col = 0 → zero, not stored.

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]221d[1]0016d[2]A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

Column 1 of B = bt[3..3].

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]222d[1]0016d[2]1115A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

dot: 3·5 = 15 → d[2] = (1, 1, 15).

rowcolvaluea[0]233a[1]001a[2]022a[3]113rowcolvaluebt[0]233bt[1]004bt[2]026bt[3]115rowcolvalued[0]222d[1]0016d[2]1115A (2×3)Bᵀ (B: 3×2)D = A·B (2×2)

Write the header: D is 2×2 with 2 terms. D = [[16, 0], [0, 15]].

void smult(term a[], term bt[], term d[]) {
  int ra, ea, rb, eb, nd = 0;
  for (ra = 1; ra <= a[0].value; ra = ea) {
    ea = row_end(a, ra);       /* row of A [ra,ea) */
    for (rb = 1; rb <= bt[0].value; rb = eb) {
      eb = row_end(bt, rb);    /* col of B [rb,eb) */
      store(d, &nd, a[ra].row, bt[rb].row,
            dot(a, ra, ea, bt, rb, eb));
    }
  }
  d[0].row = a[0].row;  d[0].col = bt[0].row;
  d[0].value = nd;
}

Sparse Matrix Multiplication · Horowitz 2.5

The complexity of sparse multiplication

  • Transpose: O(colsB + tB).
  • For each row i of A, Bᵀ is walked end to end: Σi tB = rowsA·tB.
  • For each pair (i, j) the terms of A’s row are walked too: Σj tA = colsB·tA.
  • Total: O(colsB·tA + rowsA·tB).
Classic multiplication

O(rowsA·colsA·colsB). For dense matrices (tA = rowsA·colsA) the sparse version reaches the same order with a larger constant.

When is it worth it?

When tA ≪ rows·cols and tB ≪ rows·cols. The result D is not guaranteed to be sparse: MAX_TERMS may be exceeded.

Other Representations

Compressed sparse row (CSR)

val0151222-15311435-6691728col_ind03512302row_ptr0356678row 1: val[3..4]

The same 6×6 matrix: 8 + 8 + 7 = 23 integers (27 as triples). Row i: val[row_ptr[i] .. row_ptr[i+1]−1]; the empty row 3 → two equal consecutive row_ptr values.

/* y = M * x  (matrix-vector product) */
for (i = 0; i < rows; i++) {
  y[i] = 0;
  for (k = row_ptr[i]; k < row_ptr[i+1]; k++)
    y[i] += val[k] * x[col_ind[k]];
}
Use

One of the most common formats in scientific computing; M·x is O(rows + terms). The column version: CSC.

Connection

row_ptr is the same idea as start_pos in fast_transpose (prefix sums).

Other Representations

Representing a sparse matrix with linked lists

  • In a triple array, insertion/deletion needs shifting and there is a MAX_TERMS limit.
  • In the linked representation each nonzero entry is a node; entries in the same row and in the same column are joined by circular lists, with row/column header nodes (Horowitz 4.7).
  • Insert/delete is an O(1) link change (once the place is found); traversal by row and by column is easy.
  • The price: two extra pointers per entry.
From the instructor “If I hold your homework over mine against the light… (there should be no difference)”
Where?

The linked implementation of sparse matrices is a direct application of linked lists.

Wrap-Up

Complexity Summary

OperationTimeNote
padd (n, m terms)O(n + m)sorted terms + merging
pmult (array, repeated padd)O(n²m)smaller one outside: O(min²·max)
poly_mult (linked list)O(n²m) worst casestart shortens the search
transposeO(cols·terms)O(cols²·rows) for a dense matrix
fast_transposeO(cols + terms)O(cols) extra space
Sparse multiplication (smult / mmult)O(colsB·tA + rowsA·tB)Bᵀ first
M·x with CSRO(rows + terms)row start array

Wrap-Up

Common Mistakes

1Keeping the terms unsorted but still writing padd with two pointers.
2Appending a term whose sum is 0, or not deleting it from the list.
3Truncating a float coefficient through an int parameter in attach; not checking avail for overflow.
4Only swapping row/col in a transpose and forgetting the order; mistaking the header M[0] for a term.
5Mixing up prefix and postfix in start_pos[M[i].col]++.
6Transposing A in sparse multiplication; assuming the result stays sparse.

Wrap-Up

Exercises

  1. Trace padd for A = 2x⁴ + 3x² − 1 and B = −3x² + x + 1. How many terms does D have?
  2. In the P array representation, at most how many attach calls does adding two 10-term polynomials make?
  3. In a 5×5 matrix, from how many nonzero entries on does the triple representation take more space than a 2D array?
  4. What are row_terms and start_pos for the 3×4 matrix with terms (0,1,5), (1,0,2), (1,3,4), (2,1,1)?
  5. How is fast_transpose written with a single array? Which sorting algorithm does it resemble?
  6. Which term does poly_mult delete when computing (x² + 1)(x² − 1)?
Hints / answers
  1. D = 2x⁴ + x (3x² − 3x² = 0 and −1 + 1 = 0 are not appended): 2 terms.
  2. 20 if no exponent is shared.
  3. 3(t + 1) > 25 → t ≥ 8: 8 or more nonzero entries.
  4. Columns 1, 0, 3, 1 → row_terms = [1, 2, 0, 1], start_pos = [1, 2, 4, 4].
  5. Turn row_terms into start positions in place and increment the same array; counting sort.
  6. x² · (−1) = −x² and 1 · x² = +x² add up to 0: the x² term is deleted → x⁴ − 1.

Wrap-Up

Chapter Summary

Representation matters

A coefficient array wastes space on a sparse polynomial; an array of terms keeps only the meaningful ones.

Order + merging

padd is O(n + m): walk two sorted arrays with two pointers.

Linked lists

Flexible when the number of terms is unknown; poly_mult combines equal exponents and deletes zeros.

Triples

⟨row, col, value⟩ + header M[0]; pays off while t < rows·cols/3.

Transpose

transpose is O(cols·t); fast_transpose counts and places in O(cols + t).

Multiplication

Bᵀ first, then row-by-row merging; CSR is the standard form of the same “row start” idea.

Wrap-Up

References and Next Steps

  • E. Horowitz, S. Sahni, S. Anderson-Freed, Fundamentals of Data Structures in C, 2nd ed., 2.4–2.5 (polynomials, sparse matrices), 4.4 and 4.7 (linked polynomials and sparse matrices).
  • Y. Saad, Iterative Methods for Sparse Linear Systems, 3.4 (CSR/CSC formats).

The End

Chapter 5: Polynomials and Sparse Matrices
Next chapter: Chapter 6: Stacks · Course page
1 / 1 Data Structures in C