IX · Linear algebra
Singular Value Decomposition
What it is
Any m×n matrix factors as A = UΣVᵀ where U, V are orthogonal and Σ is diagonal. Truncating Σ gives the best rank-k approximation in Frobenius norm.
Where it lives
Latent semantic analysis, recommender systems, image compression, regularised regression, every dimensionality reduction.
The key insight
The largest singular value tells you the matrix's 2-norm; truncating gives the best low-rank approximation. The math behind LSA, PCA, and modern embeddings.