Some properties of singular value decomposition and their applications to digital signal processing

Abstract

Singular value decomposition (SVD), one of the most basic and important tools of numerical linear algebra, is finding increasing applications in digital signal processing. In this paper, we first review some basic properties of SVD for matrices and then for linear operators between two Hilbert spaces. While SVD results for matrices are quite well known, applications of SVD on operators are less well known. Specifically, we propose the use of a representation for a stochastic linear system from the point of view of a stochastic SVD and show its relationship to the classical Karhunen-Loève expansion. Furthermore, issues related to practical use of time-averaging of realizations of random processes from the SVD point of view are also considered. Then algorithms for performing SVD are briefly discussed. These include the well known QR method, the modified Jacobi method of Nash, and an extension of the modified Jacobi method applicable to SVD of linear operators. Finally, we review various applications of finite rank approximation properties of SVD to digital signal processing problems in least-squares estimations, complexity reduction of FIR digital filters, matrix rank and system order determinations, and digital image processing.