Quadratic optimization for simultaneous matrix diagonalization

Abstract

Simultaneous diagonalization of a set of matrices is a technique that has numerous applications in statistical signal processing and multivariate statistics. Although objective functions in a least-squares sense can be easily formulated, their minimization is not trivial, because constraints and fourth-order terms are usually involved. Most known optimization algorithms are, therefore, subject to certain restrictions on the class of problems: orthogonal transformations, sets of symmetric, Hermitian or positive definite matrices, to name a few. In this paper, we present a new algorithm called QDIAG that splits the overall optimization problem into a sequence of simpler second order subproblems. There are no restrictions imposed on the transformation matrix, which may be nonorthogonal, indefinite, or even rectangular, and there are no restrictions regarding the symmetry and definiteness of the matrices to be diagonalized, except for one of them. We apply the new method to second-order blind source separation and show that the algorithm converges fast and reliably. It allows for an implementation with a complexity independent of the number of matrices and, therefore, is particularly suitable for problems dealing with large sets of matrices