Chapter 14 - Gram-Schmidt Orthonormalization

Abstract

Listen

Translate article

This chapter develops the Gram-Schmidt process that takes a set of k linearly independent vectors and returns an orthonormal set of k vectors that spans the same subspace. The algorithm begins with the first vector, normalizes it, and then determines the remaining orthonormal sequence by successively subtracting from the next vector in the original sequence the projections of it onto the already computed orthonormal sequence. The difference is orthogonal to all the previously generated vectors, and the algorithm normalize it. There are two versions of the process, classical Gram-Schmidt (CGS) and modified Gram-Schmidt (MGS). During the execution of CGS, the generated vectors are often not quite orthogonal, due to rounding errors. This loss of orthogonality is particularly bad. The computation also yields poor results when some of the vectors are almost linearly dependent. For these reasons, it is said that the classical Gram-Schmidt process is numerically unstable. MGS performs better by subtracting projections off one at a time instead of all at once. By maintaining inner products and norms in a matrix R, Gram-Schmidt generates the reduced QR decomposition of a matrix, A = QR, m ≥ n, with A having full rank. There are two versions, the classical and the modified Gram-Schmidt QR decomposition. The modified version performs better. A stability analysis determines that the error in orthogonality of the vectors depends on the condition number of the matrix, making Gram-Schmidt method less stable than other methods covered in later chapters. There are many applications for the QR decomposition, particularly for the computation of eigenvalues and the solution of linear least-squares problems. In this chapter, it is shown how to use the QR decomposition to compute the absolute value of a determinant and a basis for the range of a matrix with linearly independent columns.