Singular Value Decomposition

Abstract

In Chapter 3, we learned that certain types of matrices, which are referred to as positive semidefinite matrices, can be expressed in the following form: $$\displaystyle A= V \varDelta V^T $$ Here, V is a d × d matrix with orthonormal columns, and Δ is a d × d diagonal matrix with nonnegative eigenvalues of A. The orthogonal matrix V can also be viewed as a rotation/reflection matrix, the diagonal matrix Δ as a nonnegative scaling matrix along axes directions, and the matrix VT is the inverse of V . By factorizing the matrix A into simpler matrices, we are expressing a linear transform as a sequence of simpler linear transformations (such as rotation and scaling). This chapter will study the generalization of this type of factorization to arbitrary matrices. This generalized form of factorization is referred to as singular value decomposition.