Mathematics → Linear Algebra

Linear Algebra

Linear algebra is the study of vectors, matrices, and linear transformations. It is the language of machine learning, computer graphics, physics, and data science — arguably the most practically important branch of mathematics.

Vectors

A vector in $\mathbb{R}^n$ is an ordered list of $n$ real numbers, written as a column:

\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}

Operations

Addition

\mathbf{u} + \mathbf{v} = (u_1+v_1,\ u_2+v_2,\ \ldots)

Scalar multiplication

c\mathbf{v} = (cv_1,\ cv_2,\ \ldots)

Dot product

\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^n u_i v_i = |\mathbf{u}||\mathbf{v}|\cos\theta

Magnitude

|\mathbf{v}| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}

Two vectors are orthogonal (perpendicular) if and only if their dot product is zero: $\mathbf{u} \cdot \mathbf{v} = 0$ .

Matrices

Definition — Matrix

m \times n

matrix is a rectangular array of numbers with

m

rows and

n

columns. The entry in row

i

, column

j

is denoted

A_{ij}

A = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix} \quad \text{(2\times 3 matrix)}

A matrix represents a linear transformation: multiplying by $A$ maps vectors from $\mathbb{R}^n$ to $\mathbb{R}^m$ . Every linear transformation between finite-dimensional spaces can be represented as a matrix.

Matrix Operations

Multiplication

The product $AB$ of an $m\times n$ matrix $A$ and an $n\times p$ matrix $B$ is the $m\times p$ matrix with entries:

(AB)_{ij} = \sum_{k=1}^n A_{ik} B_{kj}

Matrix multiplication is not commutative: $AB \neq BA$ in general. It is associative: $(AB)C = A(BC)$ .

Transpose

The transpose $A^T$ is obtained by reflecting over the diagonal — rows become columns:

(A^T)_{ij} = A_{ji}

Inverse

A square matrix $A$ is invertible if there exists $A^{-1}$ such that $A A^{-1} = A^{-1} A = I$ , where $I$ is the identity matrix. $A$ is invertible if and only if $\det(A) \neq 0$ .

Determinants

The determinant is a scalar associated with every square matrix, measuring how much the matrix scales area (or volume in higher dimensions).

\det\begin{pmatrix}a & b \\ c & d\end{pmatrix} = ad - bc

\det\begin{pmatrix}a&b&c\\d&e&f\\g&h&i\end{pmatrix} = a(ei-fh) - b(di-fg) + c(dh-eg)

det(AB) = det(A)·det(B)

det(Aᵀ) = det(A)

det(A⁻¹) = 1/det(A)

A is invertible ⟺ det(A) ≠ 0

Linear Systems

A system of linear equations can be written compactly as $A\mathbf{x} = \mathbf{b}$ :

\begin{pmatrix}2 & 1\\5 & 3\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix} = \begin{pmatrix}4\\7\end{pmatrix}

If $A$ is invertible, the unique solution is $\mathbf{x} = A^{-1}\mathbf{b}$ . In practice, Gaussian elimination — row-reducing the augmented matrix $[A \mid \mathbf{b}]$ — is more efficient. A system has either zero, one, or infinitely many solutions.

Eigenvalues and Eigenvectors

Definition — Eigenvalue / Eigenvector

A nonzero vector

\mathbf{v}

is an eigenvector of matrix

A

with eigenvalue

\lambda

if:

A\mathbf{v} = \lambda\mathbf{v}

Geometrically:

A

stretches (or reflects)

\mathbf{v}

by the scalar factor

\lambda

, leaving its direction unchanged.

Eigenvalues are found by solving the characteristic equation:

\det(A - \lambda I) = 0

For each eigenvalue $\lambda$ , the corresponding eigenvectors are the nonzero solutions to $(A - \lambda I)\mathbf{v} = \mathbf{0}$ .

Example: For $A = \begin{pmatrix}3&1\\0&2\end{pmatrix}$ , the characteristic equation is $(3-\lambda)(2-\lambda)=0$ , giving eigenvalues $\lambda_1 = 3$ , $\lambda_2 = 2$ .

Eigenvalues and eigenvectors are fundamental to principal component analysis (PCA), Google's PageRank algorithm, quantum mechanics, and the analysis of differential equations.

Next: Probability →

On this page

Vectors

Matrices

Matrix Operations

Determinants

Linear Systems

Eigenvalues & Eigenvectors