Majorized Proximal Alternating Imputation for regularized rank constrained matrix completion

Abstract

Low rank approximation in the presence of missing data is ubiquitous in many areas of science and engineering. As an NP-hard problem, spectral regularization (e.g., nuclear norm, Schatten p-norm) is widely used. However, solving this problem directly can be computationally expensive due to the unknown rank of variables and full rank Singular Value Decompositions (SVD). Bilinear factorization is efficient and effectively benefits from the fast numerical optimization. However, most of existing methods require explicit knowledge of the rank and often tend to be flatlining or over fitting. To balance and take advantage of both, we formulate the matrix completion problem under the regularized rank constrained conditions. We relax this problem to the surrogates with Majorization Minimization (MM). By employing a proximal alternating minimization method to optimize the corresponding surrogates, we propose an efficient and accurate algorithm named Majorized Proximal Alternating Imputation (MPA-IMPUTE) algorithm. The proposed method iteratively replaces the missing data at each step with the most recent estimate, and only involves simple matrix computation and small scale SVD. Theoretically, the proposed method is guaranteed to converge to the stationary point under mild conditions. Moreover, the proposed method can be generalized to a family of non-convex regularization problems. Extensive experiments over synthetic data and real-world dataset demonstrate the superior performance of our method.