Matrices of linear maps and applications
Any linear map between finite-dimensional spaces can be represented by a matrix once bases are chosen. Choosing bases wisely reveals a map's structure.
Matrix relative to bases
Let T map V, with basis B, to W, with basis C. The matrix of T relative to B and C has columns \([T(\mathbf{b}_1)]_C, \dots, [T(\mathbf{b}_n)]_C\): the C-coordinates of the images of B's vectors. Then \([T(\mathbf{x})]_C = M[\mathbf{x}]_B\), so computing T reduces to multiplying coordinate vectors by M.
Example: differentiation
For D from quadratics to quadratics, with basis 1, t, \(t^2\): D(1) = 0, D(t) = 1, and \(D(t^2) = 2t\), with coordinates (0, 0, 0), (1, 0, 0), and (0, 2, 0). So the matrix is \(\begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 2 \\ 0 & 0 & 0 \end{bmatrix}\). Multiplying it by the coordinates (3, 5, 4) of \(3 + 5t + 4t^2\) gives (5, 8, 0), the coordinates of the derivative \(5 + 8t\).
Change of basis for operators
For a linear map T from V to itself, changing basis changes its matrix from A to \(P^{-1}AP\), where P converts new coordinates to old. Choosing a basis of eigenvectors, when possible, makes the matrix diagonal; the next units develop this, along with orthonormal bases that make P orthogonal and its inverse simply its transpose.
Projections
A projection P satisfies \(P^2 = P\): projecting twice changes nothing more. Orthogonal projection onto a line through a unit vector \(\mathbf{u}\) has matrix \(\mathbf{u}\mathbf{u}^T\). Projections appear in least squares, computer graphics (projecting 3D scenes onto a 2D screen), and signal processing, where noise is removed by projecting onto a subspace of clean signals.
Linear maps in data science
Neural networks alternate linear maps (weight matrices) with simple non-linear functions; most of their computation is matrix multiplication, which is why GPUs accelerate them. Dimensionality reduction maps high-dimensional data linearly into a few dimensions, and word embeddings show that some relationships between words behave approximately like vector differences.
Linear differential equations
The solutions of a linear homogeneous differential equation such as \(y'' + y = 0\) form a vector space, here with basis \(\cos t\) and \(\sin t\), because the differential operator is linear. Systems of linear differential equations \(\mathbf{x}' = A\mathbf{x}\) are solved using the eigenvalues and eigenvectors of A, linking this course to differential equations.
Affine maps
Maps of the form \(T(\mathbf{x}) = A\mathbf{x} + \mathbf{b}\), a linear map followed by a translation, are affine. They are not linear unless \(\mathbf{b} = \mathbf{0}\), but they keep straight lines straight and preserve ratios along lines. Graphics pipelines represent them as linear maps in homogeneous coordinates, so they can be composed by matrix multiplication.
Worked example
Projection onto the line through \(\mathbf{u} = \frac{1}{\sqrt{2}}(1, 1)\) has matrix \(\mathbf{u}\mathbf{u}^T = \frac{1}{2}\begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\). Projecting (3, 1) gives \(\frac{1}{2}(4, 4) = (2, 2)\), the closest point on the line y = x. Applying the matrix again leaves (2, 2) unchanged, consistent with \(P^2 = P\); its rank is 1 and its kernel is the line y = -x.
Key takeaways
Choosing bases B and C, a linear map T has a matrix whose columns are [T(b_i)]_C, and [T(x)]_C = M [x]_B. Differentiation on quadratics with basis 1, t, t^2 becomes a 3 x 3 matrix. For operators, changing basis gives P^{-1} A P; eigenvector and orthonormal bases simplify matrices. Projections satisfy P^2 = P; projection onto unit vector u has matrix u u^T. Neural networks, dimensionality reduction, and embeddings rely on linear maps. Linear differential equations have vector spaces of solutions. Affine maps Ax + b preserve lines and are handled with homogeneous coordinates.
Flashcard
Columns [T(b_i)]_C
Check your understanding
What is the matrix of differentiation on quadratics, with basis 1, t, t^2, applied to the coordinates of 2 + 3t + t^2?