Riemannian Optimization on Tensor Products of Grassmann Manifolds: Applications to Generalized Rayleigh-Quotients

Abstract

We introduce a generalized Rayleigh-quotient $\rho_A$ on the direct product of Grassmannians $\mathrm{Gr}({\bf m},{\bf n})$ enabling a unified approach to well-known optimization tasks from different areas of numerical linear algebra, such as best low-rank approximations of tensors (data compression), geometric measures of entanglement (quantum computing), and subspace clustering (image processing). We compute the Riemannian gradient of $\rho_A$, characterize its critical points, and prove that they are generically nondegenerated. Moreover, we derive an explicit necessary condition for the nondegeneracy of the Hessian. Finally, we present two intrinsic methods for optimizing $\rho_A$—a Newton-like and a conjugated gradient—and compare our algorithms tailored to the above-mentioned applications with established ones from the literature.