SVD Computations on the Connection Machine CM-5/CM-5E: Implementation and Accuracy

Abstract

The singular value decomposition (SVD) plays an essential role in many applications. One technique for obtaining the SVD of a real dense matrix is to first reduce the dense matrix to bidiagonal form and then compute the SVD of the bidiagonal matrix. Guidelines followed to implement this approach efficiently on the Connection Machine CM-5/CM-5E are described. Timing results show that use of the described techniques yields up to 44% of peak performance in the reduction from dense to bidiagonal form. The singular values are achieved by applying the parallel bisection algorithm to the symmetric tridiagonal matrix obtained by doing a perfect-shuffle permutation of the rows and columns of [[0 BT];[B 0]], where B is the bidiagonal matrix. The singular vectors are computed using inverse iteration on the symmetric tridiagonal matrix. SVD experiments done on bidiagonal matrices are presented and illustrate that this approach yields very good performance as well as orthogonal singular vectors even for ill-conditioned matrices as long as the singular values are computed to very high accuracy. The main conclusion is that extra accuracy in the eigenvalue computation very often enhance orthogonality of the eigenvectors computed with inverse iteration to the point where reorthogonalization is not needed. The conclusions presented regarding the SVD's numerical properties are general to the algorithm and not specific to the Connection Machine implementation.