Rank $2r$ Iterative Least Squares: Efficient Recovery of Ill-Conditioned Low Rank Matrices from Few Entries

Abstract

We present a new, simple, and computationally efficient iterative method for low rank matrix completion. Our method is inspired by the class of factorization-type iterative algorithms, but substantially differs from them in the way the problem is cast. Precisely, given a target rank $r$, instead of optimizing on the manifold of rank $r$ matrices, we allow our interim estimated matrix to have a specific overparametrized rank $2r$ structure. Our algorithm, denoted R2RILS, for rank $2r$ iterative least squares, thus has low memory requirements, and at each iteration it solves a computationally cheap sparse least squares problem. We motivate our algorithm by its theoretical analysis for the simplified case of a rank 1 matrix. Empirically, R2RILS is able to recover ill-conditioned low rank matrices from very few observations---near the information limit---and it is stable to additive noise.