Gram-Schmidt Orthogonalization Process
Any linearly independent set can be systematically converted into an orthonormal basis by subtracting out parallel projections.
Vector Projection
The foundation of orthogonalization is the projection of one vector onto another. Projecting vector v onto a non-zero vector u yields the component of v that lies along the direction of u. Subtracting this projection from v leaves an error vector that is strictly orthogonal to u.
\operatorname{proj}_{\mathbf{u}}(\mathbf{v}) = \frac{\langle \mathbf{v}, \mathbf{u} \rangle}{\langle \mathbf{u}, \mathbf{u} \rangle} \mathbf{u}Iterative Orthogonalization
Given a basis of vectors, the algorithm fixes the first vector as the starting direction. Each subsequent vector is made orthogonal to the subspace spanned by the preceding vectors by subtracting its projection onto each previously constructed orthogonal vector.
\mathbf{u}_k = \mathbf{v}_k - \sum_{j=1}^{k-1} \frac{\langle \mathbf{v}_k, \mathbf{u}_j \rangle}{\langle \mathbf{u}_j, \mathbf{u}_j \rangle} \mathbf{u}_jOrthonormalization
The set of vectors produced by the iterative subtraction step is mutually orthogonal but not necessarily unit length. Dividing each vector by its norm yields an orthonormal set spanning the exact same subspace, establishing the matrix QR decomposition.
\mathbf{e}_k = \frac{\mathbf{u}_k}{\|\mathbf{u}_k\|}Flashcard
Orthogonal projection of vector v onto direction u
Check your understanding
Given linearly independent vectors v1 = (1, 1) and v2 = (2, 0), what is the orthogonal vector u2 obtained by applying Gram-Schmidt with u1 = v1?