Linear algebra at work
Linear algebra is the language of modern computation. This closing lesson connects the course's ideas to machine learning, graphics, engineering, and science.
Machine learning
Datasets are matrices with one row per example and one column per feature. Linear regression is least squares; principal component analysis is the SVD; neural networks compose linear maps (weight matrices) with non-linear activations, and training adjusts those matrices by gradient descent. Graphics processors and specialised AI chips are, at heart, machines for multiplying matrices fast.
Embeddings and similarity
Modern AI represents words, images, and documents as vectors called embeddings, often with hundreds or thousands of dimensions. Similar items have vectors pointing in similar directions, measured by cosine similarity, a normalised dot product. Search, recommendation, and retrieval-augmented language models find nearest neighbours among millions of such vectors.
Computer graphics and vision
3D graphics transforms vertices with 4 x 4 matrices in homogeneous coordinates for rotation, scaling, translation, and perspective projection. Computer vision estimates camera positions and reconstructs 3D scenes by solving linear systems and least-squares problems, often using the SVD, and image processing applies convolution, which is a linear operation.
Engineering and physics
Structural engineers solve stiffness systems Ku = f with millions of unknowns using finite element methods. Electrical engineers analyse circuits with nodal matrices. Control engineers use state-space models x' = Ax + Bu and test stability with eigenvalues. Quantum mechanics is formulated entirely in vector spaces of states and linear operators.
Signal processing and data compression
The discrete Fourier transform is multiplication by a special matrix whose columns are sampled complex exponentials; the fast Fourier transform (FFT), published by James Cooley and John Tukey in 1965, computes it in O(n log n) rather than O(n^2) time. JPEG compression uses the discrete cosine transform on 8 x 8 blocks, a change of basis in which images are sparse.
Networks and economics
Network analysis uses adjacency and Laplacian matrices and their eigenvalues for ranking, clustering, and spreading processes. Economics uses Leontief input-output models and linear programming, optimising linear objectives under linear constraints, solved by the simplex method (George Dantzig, 1947) and interior-point methods for logistics, scheduling, and resource allocation.
Cryptography and coding
Error-correcting codes, protecting data on phones, satellites, and storage, are subspaces over finite fields: linear codes such as Hamming and Reed-Solomon codes encode messages with generator matrices and detect errors with parity-check matrices. Post-quantum cryptography standards published in 2024 rely on lattice problems, hard questions about integer combinations of vectors.
Worked example
A recommendation system has a 4 x 3 ratings matrix for 4 users and 3 films. If its singular values are 10, 3, and 0.2, a rank-2 approximation keeps the two strongest 'taste' factors and discards the third as noise: the relative error in Frobenius norm is \(\frac{0.2}{\sqrt{100 + 9 + 0.04}} \approx 1.9\%\). Predicted ratings for unseen films come from the low-rank model, just as in the matrix factorisation methods popularised by the Netflix Prize.
Key takeaways
Machine learning treats data as matrices: regression is least squares, PCA is the SVD, and neural networks compose weight matrices with non-linearities, accelerated by matrix-multiplying hardware. Embeddings and cosine similarity power search and AI retrieval. Graphics uses 4 x 4 homogeneous transformations; vision solves least-squares problems with the SVD. Engineering solves huge stiffness and circuit systems and tests stability with eigenvalues; quantum mechanics is linear algebra. The FFT (Cooley and Tukey, 1965) and DCT-based JPEG change bases efficiently. Networks, Leontief models, and linear programming (Dantzig, 1947) apply matrices to systems and economies. Linear codes and lattice cryptography rely on vector spaces.
Flashcard
Rows are examples, columns are features
Check your understanding
In a neural network, what does each layer's weight matrix represent mathematically?