Matrix Algorithms and Linear Algebra Basics

Matrices are both mathematical objects and data structures. Many algorithms in graphics, scientific computing, optimization, machine learning, graph processing, and dynamic programming rely on matrix operations.

Placement: this page is supplementary mathematical background rather than a standalone main chapter. It is most useful before or alongside graph algorithms, dynamic programming, optimization, and advanced algorithmic topics.

Matrix Representation

An m×n matrix has m rows and n columns. In programs it is commonly stored as a two-dimensional array.

A[i][j] = entry in row i, column j

Accessing a known element is O(1) in the usual array representation.

Addition

Matrices of the same dimensions are added element by element.

C[i][j] = A[i][j] + B[i][j]

For two m×n matrices, time complexity is Θ(mn).

Transpose

The transpose exchanges rows and columns.

Aᵀ[j][i] = A[i][j]

Creating an explicit transpose of an m×n matrix takes Θ(mn) time.

Identity Matrix

The n×n identity matrix I has 1 on the main diagonal and 0 elsewhere.

AI = IA = A

It plays the same multiplicative identity role that 1 plays for scalars.

Matrix Multiplication

If A is m×k and B is k×n, then C=AB is m×n.

C[i][j] = Σ A[i][t] · B[t][j], t=1..k

The classical triple-loop algorithm runs in Θ(mkn); for square n×n matrices this is Θ(n³).

Faster Multiplication

Matrix multiplication is an important example where asymptotic improvements are possible.

  • Classical multiplication: Θ(n³)
  • Strassen: O(n^log₂7) ≈ O(n^2.807)
  • Even faster asymptotic algorithms exist, though they are mostly theoretical or specialized.

Determinant and Invertibility

For a 2×2 matrix:

det([[a,b],[c,d]]) = ad − bc

A square matrix is invertible exactly when its determinant is nonzero. In numerical algorithms, however, computing an inverse explicitly is often not the preferred way to solve a linear system.

Gaussian Elimination

Systems of linear equations Ax=b are commonly solved by elimination rather than by explicitly forming A⁻¹.

For a dense n×n system, standard Gaussian elimination uses Θ(n³) arithmetic operations.

for pivot = 0 .. n-1:
    choose pivot row
    eliminate entries below pivot
back-substitute to recover x

Matrices and Graphs

An adjacency matrix represents a graph using a V×V matrix.

  • Space: Θ(V²)
  • Edge-existence test: O(1)
  • Useful for dense graphs

Matrix powers also encode walk information in graphs.

Algorithmic Applications

  • Graph adjacency and transitive closure
  • Dynamic programming tables
  • 3D transformations and graphics
  • Markov chains
  • Machine learning
  • Scientific computing
  • Fast exponentiation of linear recurrences

Interactive 2×2 Matrix Playground

Enter matrices as two rows separated by a newline and values separated by spaces.

Ready.