Essential Linear Algebra Formulas You Should Know

Why Linear Algebra Formulas Matter

Linear algebra is the math behind machine learning, computer graphics, engineering, and data science. If you're working with vectors, matrices, or systems of equations, these formulas are your toolkit. No theory-first approach here—just the formulas you'll actually use.

Matrix Basics

A matrix is a rectangular array of numbers. The size is defined by rows × columns. A 2×3 matrix has 2 rows and 3 columns.

Matrix Addition & Subtraction

You can only add or subtract matrices of the same size. Add corresponding elements together:

Formula: C = A + B means cᵢⱼ = aᵢⱼ + bᵢⱼ

Same logic applies for subtraction. That's it. Nothing complicated.

Scalar Multiplication

Multiply every element in a matrix by a constant (scalar):

Formula: cA = [c·aᵢⱼ]

Distribute the scalar across each entry. Works the same way as distributing any number.

Matrix Multiplication

This one trips people up. You're not multiplying element-by-element. You're computing dot products of rows and columns.

Formula: If C = AB, then cᵢⱼ = Σₖ aᵢₖ · bₖⱼ

The number of columns in A must match the number of rows in B. A 3×2 matrix times a 2×4 matrix gives a 3×4 result.

Key properties:

• AB ≠ BA in general (not commutative)
• A(BC) = (AB)C (associative)
• A(B + C) = AB + AC (distributive)

Transpose of a Matrix

The transpose flips rows and columns. Row 1 becomes Column 1.

Formula: (Aᵀ)ᵢⱼ = Aⱼᵢ

If A is m×n, then Aᵀ is n×m.

Special Matrices

• Symmetric: Aᵀ = A
• Skew-symmetric: Aᵀ = -A
• Identity: AI = IA = A (like 1 for multiplication)
• Orthogonal: Aᵀ = A⁻¹

Determinants

The determinant is a single number computed from a square matrix. It tells you if a matrix is invertible (non-zero determinant = invertible).

2×2 Determinant

Formula: det(A) = ad - bc for matrix [[a, b], [c, d]]

3×3 Determinant (Sarrus Rule)

Formula: det(A) = a(ei - fh) - b(di - fg) + c(dh - eg)

For larger matrices, use cofactor expansion or row reduction. The determinant scales by factor k if you multiply a row by k.

Key Determinant Properties

• det(AB) = det(A) · det(B)
• det(Aᵀ) = det(A)
• det(A⁻¹) = 1/det(A)
• det(I) = 1

Matrix Inverse

The inverse A⁻¹ satisfies AA⁻¹ = I. Only square matrices with non-zero determinants have inverses.

2×2 Inverse Formula

Formula: A⁻¹ = (1/det(A)) · [[d, -b], [-c, a]] for [[a, b], [c, d]]

Swap the diagonal elements, negate the off-diagonals, divide by determinant.

Properties

• (A⁻¹)⁻¹ = A
• (AB)⁻¹ = B⁻¹A⁻¹ (order reverses)
• (Aᵀ)⁻¹ = (A⁻¹)ᵀ

Vector Operations

Vectors are matrices with one row or one column.

Dot Product

Multiplies corresponding components and sums the results. Gives a scalar.

Formula: a · b = Σᵢ aᵢbᵢ = |a||b|cos(θ)

The dot product is zero when vectors are perpendicular (orthogonal).

Cross Product

Only for 3D vectors. Produces a vector perpendicular to both inputs.

Formula: a × b = (a₂b₃ - a₃b₂, a₃b₁ - a₁b₃, a₁b₂ - a₂b₁)

Use the right-hand rule to find direction.

Vector Norm (Magnitude)

Formula: ||a|| = √(a₁² + a₂² + ... + aₙ²)

For unit vectors, ||a|| = 1. Normalize by dividing by the norm.

Eigenvalues and Eigenvectors

This is where linear algebra gets interesting. An eigenvector doesn't change direction when you apply the matrix—it just scales.

Formula: Av = λv

λ is the eigenvalue, v is the eigenvector.

Finding Eigenvalues

Formula: det(A - λI) = 0

Solve this characteristic equation for λ. For a 2×2 matrix [[a, b], [c, d]]:

λ² - (a + d)λ + (ad - bc) = 0

Finding Eigenvectors

Plug each eigenvalue into (A - λI)v = 0 and solve for v.

Properties

• Sum of eigenvalues = trace (sum of diagonal)
• Product of eigenvalues = determinant
• Symmetric matrices always have real eigenvalues

Solving Linear Systems

Systems like Ax = b appear everywhere. Here are your tools.

Gaussian Elimination

Row reduce the augmented matrix [A|b] to row echelon form. Back-substitute to find solutions.

Cramer's Rule

For small systems only. Replace column i of A with b, divide by det(A).

Formula: xᵢ = det(Aᵢ)/det(A)

Computationally inefficient for large systems. Don't use this in practice.

LU Decomposition

Factor A = LU where L is lower triangular and U is upper triangular. Then solve Ly = b, then Ux = y.

Fast and reusable when solving the same A with different b values.

Quick Reference Table

Getting Started: How to Use These Formulas

Pick your problem type:

1. Matrix arithmetic — Check sizes first. You can't add 2×3 and 3×2. Multiply only when dimensions match.
2. Finding inverses — Start with the determinant. Zero means no inverse exists.
3. Solving systems — Gaussian elimination works for everything. LU decomposition if you need speed or multiple solutions.
4. Eigenvalue problems — Compute det(A - λI) = 0 first. Then substitute back for eigenvectors.

Write code or use NumPy/MATLAB for anything beyond 3×3. Manual computation beyond that is a waste of time.

Common Pitfalls

• Confusing matrix multiplication with element-wise multiplication
• Forgetting that AB ≠ BA
• Not checking if a matrix is invertible before trying to find the inverse
• Dropping signs when computing cross products or cofactors
• Using Cramer's Rule on systems larger than 3×3

Linear algebra rewards precision. One sign error cascades through the entire calculation. Check your work at each step.