Chapter 3 - Some Fundamental Tools and Concepts from Numerical Linear Algebra

Abstract

This chapter presents the fundamental concepts and techniques of numerical linear algebra that are essential for in-depth understanding of computational algorithms for control problems. It introduces the basic concepts of floating point operations, numerical stability of an algorithm, conditioning of a computational problem, and their effects on the accuracy of a solution obtained by a certain algorithm. The chapter describes three important matrix factorizations that are LU, QR, and the singular value decomposition (SVD); their applications to solutions of algebraic linear systems; linear least-squares problems; and eigenvalue problems in details. The method of choice for the linear system problem is the LU factorization technique obtained by Gaussian elimination with partial pivoting. The method of choice for the symmetric positive definite system is the Cholesky factorization technique. The QR factorization of a matrix is introduced in the context of the least-squares solution of a linear system. However, it also forms the core of the QR iteration technique, which is the method of choice for eigenvalue computation. Two numerically stable methods for the QR factorization—namely, Householder's and Givens' methods are described in the chapter. Householder's method is slightly cheaper than Givens' method for sequential computations, but the latter has computational advantages in parallel computation setting. The SVD has nowadays become an essential tool for determining the numerical rank, the distance of a matrix from a matrix of immediate lower rank, and finding the orthonormal basis and projections.