Two Newton methods on the manifold of fixed-rank matrices endowed with Riemannian quotient geometries

Abstract

We consider two Riemannian geometries for the manifold $${\mathcal{M }(p,m\times n)}$$ of all $$m\times n$$ matrices of rank $$p$$ . The geometries are induced on $${\mathcal{M }(p,m\times n)}$$ by viewing it as the base manifold of the submersion $$\pi :(M,N)\mapsto MN^\mathrm{T}$$ , selecting an adequate Riemannian metric on the total space, and turning $$\pi $$ into a Riemannian submersion. The theory of Riemannian submersions, an important tool in Riemannian geometry, makes it possible to obtain expressions for fundamental geometric objects on $${\mathcal{M }(p,m\times n)}$$ and to formulate the Riemannian Newton methods on $${\mathcal{M }(p,m\times n)}$$ induced by these two geometries. The Riemannian Newton methods admit a stronger and more streamlined convergence analysis than the Euclidean counterpart, and the computational overhead due to the Riemannian geometric machinery is shown to be mild. Potential applications include low-rank matrix completion and other low-rank matrix approximation problems.