Low-rank Matrix Recovery Problem Research Articles

Approximation Conjugate Gradient Method for Low-Rank Matrix Recovery

Large-scale symmetric and asymmetric matrices have emerged in predicting the relationship between genes and diseases. The emergence of large-scale matrices increases the computational complexity of the problem. Therefore, using low-rank matrices instead of original symmetric and asymmetric matrices can greatly reduce computational complexity. In this paper, we propose an approximation conjugate gradient method for solving the low-rank matrix recovery problem, i.e., the low-rank matrix is obtained to replace the original symmetric and asymmetric matrices such that the approximation error is the smallest. The conjugate gradient search direction is given through matrix addition and matrix multiplication. The new conjugate gradient update parameter is given by the F-norm of matrix and the trace inner product of matrices. The conjugate gradient generated by the algorithm avoids SVD decomposition. The backtracking linear search is used so that the approximation conjugate gradient direction is computed only once, which ensures that the objective function decreases monotonically. The global convergence and local superlinear convergence of the algorithm are given. The numerical results are reported and show the effectiveness of the algorithm.

Matrix recovery from nonconvex regularized least absolute deviations

In this paper, we consider the low-rank matrix recovery problem. We propose the nonconvex regularized least absolute deviations model via ℓ1−αℓ2 (0<α<1) minimization. We establish the theoretical analysis of the proposed model and obtain a stable error estimation. Our result is a nontrivial extension of some previous work. Different from most of the state-of-the-art methods, our method does not need any knowledge of standard deviation or any moment assumption of the noise. Numerical experiments show that our method is effective for many types of noise distributions.

Two novel models with nuclear norm for robust matrix recovery

In this paper, we investigate the low-rank matrix recovery problem from linear observations. Inspired by the matrix Lasso and Dantzig selector, we propose nuclear norm regularized models with two data fitting items: ℓ2-loss function and Dantzig selector. We establish the recovery errors for both constrained and unconstrained models under the matrix restricted isometry property frame. For the unconstrained model, we obtain a tighter error bound compared with the previous study by W. Wang et al. (2021) under certain conditions. Furthermore, we design an effective solving algorithm for the proposed unconstrained model. Our numerical experiments demonstrate that our method outperforms the Lasso and Dantzig selector models.

Low-rank matrix recovery problem minimizing a new ratio of two norms approximating the rank function then using an ADMM-type solver with applications

Since the nuclear norm may lead to suboptimal solutions to rank minimization problems, many nonconvex surrogates have been proposed to provide better rank function approximations. In this paper, we use the ratio of the nuclear norm and the Frobenius norm, denoted as N/F, as a new nonconvex surrogate of the rank function. The N/F can provide a close approximation for the matrix rank. In particular, the N/F is the same as the rank for rank 1 matrices. Furthermore, compared to other popular nonconvex approximations, the N/F model has no extra parameters to tune. Theoretically, we establish the exact recovery condition and the stable recovery condition for the N/F minimization problem with or without noise, respectively. Computationally, we use the alternating direction method of multipliers with a specific variable-splitting scheme to solve the N/F minimization problem. What is more, we incorporate a box constraint in the N/F model, and propose the ADMM-box to solve it. The convergence analysis of ADMM and ADMM-box are also given. Extensive experiments on both synthetic data and color images verify the effectiveness of our proposed methods.

Generalized Nonconvex Nonsmooth Low-Rank Matrix Recovery Framework With Feasible Algorithm Designs and Convergence Analysis.

Decomposing data matrix into low-rank plus additive matrices is a commonly used strategy in pattern recognition and machine learning. This article mainly studies the alternating direction method of multiplier (ADMM) with two dual variables, which is used to optimize the generalized nonconvex nonsmooth low-rank matrix recovery problems. Furthermore, the minimization framework with a feasible optimization procedure is designed along with the theoretical analysis, where the variable sequences generated by the proposed ADMM can be proved to be bounded. Most importantly, it can be concluded from the Bolzano-Weierstrass theorem that there must exist a subsequence converging to a critical point, which satisfies the Karush-Kuhn-Tucher (KKT) conditions. Meanwhile, we further ensure the local and global convergence properties of the generated sequence relying on constructing the potential objective function Particularly, the detailed convergence analysis would be regarded as one of the core contributions besides the algorithm designs and the model generality. Finally, the numerical simulations and the real-world applications are both provided to verify the consistence of the theoretical results, and we also validate the superiority in performance over several mostly related solvers to the tasks of image inpainting and subspace clustering.

An accelerated alternating directional method with non-monotone technique for matrix recovery

&lt;abstract&gt;&lt;p&gt;Low-rank and sparse structures have been frequently exploited in matrix recovery and robust PCA problems. In this paper, we develop an alternating directional method and its variant equipped with the non-monotone search procedure for solving a non-convex optimization model of low-rank and sparse matrix recovery problems, where the concerned matrix with incomplete data is separable into a low-rank part and a sparse part. The main idea is to use the alternating minimization method for the low-rank matrix part, and to use the non-monotone line search technique for the sparse matrix part to iteratively update, respectively. To some extent, the non-monotone strategy relaxes the single-step descent into a multi-step descent and then greatly improves the performance of the alternating directional method. Theoretically, we prove the global convergence of the two proposed algorithms under some mild conditions. Finally, the comparison of numerical experiments shows that the alternate directional method with non-monotone technique is more effective than the original monotone method and the previous method. The efficiency and effectiveness of the proposed algorithms are demonstrated by solving some instances of random incomplete matrix recovery problems and some problems of the background modeling in video processing.&lt;/p&gt;&lt;/abstract&gt;

Cooperative Electromagnetic Data Annotation via Low-Rank Matrix Completion

Electromagnetic data annotation is one of the most important steps in many signal processing applications, e.g., radar signal deinterleaving and radar mode analysis. This work considers cooperative electromagnetic data annotation from multiple reconnaissance receivers/platforms. By exploiting the inherent correlation of the electromagnetic signal, as well as the correlation of the observations from multiple receivers, a low-rank matrix recovery formulation is proposed for the cooperative annotation problem. Specifically, considering the measured parameters of the same emitter should be roughly the same at different platforms, the cooperative annotation is modeled as a low-rank matrix recovery problem, which is solved iteratively either by the rank minimization method or the maximum-rank decomposition method. A comparison of the two methods, with the traditional annotation method on both the synthetic and real data, is given. Numerical experiments show that the proposed methods can effectively recover missing annotations and correct annotation errors.

Sharp Restricted Isometry Property Bounds for Low-Rank Matrix Recovery Problems with Corrupted Measurements

In this paper, we study a general low-rank matrix recovery problem with linear measurements corrupted by some noise. The objective is to understand under what conditions on the restricted isometry property (RIP) of the problem local search methods can find the ground truth with a small error. By analyzing the landscape of the non-convex problem, we first propose a global guarantee on the maximum distance between an arbitrary local minimizer and the ground truth under the assumption that the RIP constant is smaller than 1/2. We show that this distance shrinks to zero as the intensity of the noise reduces. Our new guarantee is sharp in terms of the RIP constant and is much stronger than the existing results. We then present a local guarantee for problems with an arbitrary RIP constant, which states that any local minimizer is either considerably close to the ground truth or far away from it. Next, we prove the strict saddle property, which guarantees the global convergence of the perturbed gradient descent method in polynomial time. The developed results demonstrate how the noise intensity and the RIP constant of the problem affect the landscape of the problem.

Local and Global Linear Convergence of General Low-Rank Matrix Recovery Problems

We study the convergence rate of gradient-based local search methods for solving low-rank matrix recovery problems with general objectives in both symmetric and asymmetric cases, under the assumption of the restricted isometry property. First, we develop a new technique to verify the Polyak-Lojasiewicz inequality in a neighborhood of the global minimizers, which leads to a local linear convergence region for the gradient descent method. Second, based on the local convergence result and a sharp strict saddle property proven in this paper, we present two new conditions that guarantee the global linear convergence of the perturbed gradient descent method. The developed local and global convergence results provide much stronger theoretical guarantees than the existing results. As a by-product, this work significantly improves the existing bounds on the RIP constant required to guarantee the non-existence of spurious solutions.

GNMR: A Provable One-Line Algorithm for Low Rank Matrix Recovery

Low rank matrix recovery problems, including matrix completion and matrix sensing, appear in a broad range of applications. In this work we present GNMR -- an extremely simple iterative algorithm for low rank matrix recovery, based on a Gauss-Newton linearization. On the theoretical front, we derive recovery guarantees for GNMR in both the matrix sensing and matrix completion settings. A key property of GNMR is that it implicitly keeps the factor matrices approximately balanced throughout its iterations. On the empirical front, we show that for matrix completion with uniform sampling, GNMR performs better than several popular methods, especially when given very few observations close to the information limit.

Column $\ell_{2,0}$-Norm Regularized Factorization Model of Low-Rank Matrix Recovery and Its Computation

This paper is concerned with the column $\ell_{2,0}$-regularized factorization model of low-rank matrix recovery problems and its computation. The column $\ell_{2,0}$-norm of factor matrices is introduced to promote column sparsity of factors and low-rank solutions. For this nonconvex discontinuous optimization problem, we develop an alternating majorization-minimization (AMM) method with extrapolation, and a hybrid AMM in which a majorized alternating proximal method is proposed to seek an initial factor pair with less nonzero columns and the AMM with extrapolation is then employed to minimize of a smooth nonconvex loss. We provide the global convergence analysis for the proposed AMM methods and apply them to the matrix completion problem with non-uniform sampling schemes. Numerical experiments are conducted with synthetic and real data examples, and comparison results with the nuclear-norm regularized factorization model and the max-norm regularized convex model show that the column $\ell_{2,0}$-regularized factorization model has an advantage in offering solutions of lower error and rank within less time.

Linear convergence of Frank–Wolfe for rank-one matrix recovery without strong convexity

We consider convex optimization problems which are widely used as convex relaxations for low-rank matrix recovery problems. In particular, in several important problems, such as phase retrieval and robust PCA, the underlying assumption in many cases is that the optimal solution is rank-one. In this paper we consider a simple and natural sufficient condition on the objective so that the optimal solution to these relaxations is indeed unique and rank-one. Mainly, we show that under this condition, the standard Frank–Wolfe method with line-search (i.e., without any tuning of parameters whatsoever), which only requires a single rank-one SVD computation per iteration, finds an $$\epsilon $$ -approximated solution in only $$O(\log {1/\epsilon })$$ iterations (as opposed to the previous best known bound of $$O(1/\epsilon )$$ ), despite the fact that the objective is not strongly convex. We consider several variants of the basic method with improved complexities, as well as an extension motivated by robust PCA, and finally, an extension to nonsmooth problems.

Patch-based mesh inpainting via low rank recovery

Mesh inpainting aims to fill the holes or missing regions from observed incomplete meshes and keep consistent with prior knowledge. Inspired by the success of low rank in describing similarity, we formulate the mesh inpainting problem as the low rank matrix recovery problem and present a patch-based mesh inpainting algorithm. Normal patch covariance is adapted to describe the similarity between surface patches. By analyzing the similarity of patches, the most similar patches are packed into a matrix with low rank structure. An iterative diffusion strategy is first designed to recover the patch vertex normals gradually. Then, the normals are refined by low rank approximation to keep the overall consistency and vertex positions are finally updated. We conduct several experiments in different 3D models to verify the proposed approach. Compared with existing algorithms, our experimental results demonstrate the superiority of our approach both visually and quantitatively in recovering the mesh with self-similarity patterns.

Low rank matrix recovery with adversarial sparse noise* *This work was supported in part by National Natural Science Foundation of China under Grant numbers 12071426, 11971427, 11901518, the NSAF of China under grant number U21A20426, and the Fundamental Research Funds for the Central Universities under Grant number 2020XZZX002-03.

Many problems in data science can be treated as recovering a low-rank matrix from a small number of random linear measurements, possibly corrupted with adversarial noise and dense noise. Recently, a bunch of theories on variants of models have been developed for different noises, but with fewer theories on the adversarial noise. In this paper, we study low-rank matrix recovery problem from linear measurements perturbed by ℓ 1-bounded noise and sparse noise that can arbitrarily change an adversarially chosen ω-fraction of the measurement vector. For Gaussian measurements with nearly optimal number of measurements, we show that the nuclear-norm constrained least absolute deviation (LAD) can successfully estimate the ground-truth matrix for any ω < 0.239. Similar robust recovery results are also established for an iterative hard thresholding algorithm applied to the rank-constrained LAD considering geometrically decaying step-sizes, and the unconstrained LAD based on matrix factorization as well as its subgradient descent solver.

Blind Super-Resolution of Point Sources via Projected Gradient Descent

Blind super-resolution can be cast as a low rank matrix recovery problem by exploiting the inherent simplicity of the signal and the low dimensional structure of point spread functions. In this paper, we develop a simple yet efficient non-convex projected gradient descent method for this problem based on the low rank structure of the vectorized Hankel matrix associated with the target matrix. Theoretical analysis indicates that the proposed method exactly converges to the target matrix with a linear convergence rate under the similar conditions as convex approaches. Numerical results show that our approach is competitive with existing convex approaches in terms of recovery ability and efficiency.

Topological Interference Management With Inaccurate Topology of Network

In this letter, we investigate the topological interference management (TIM) problem in the presence of measurement error of the network topology. We formulate the TIM problem as a low-rank matrix recovery problem and then solve it through an alternating algorithm to recover the true network topology. Our numerical results demonstrate that the proposed low-rank matrix recovery model can significantly improve the achievable degrees of freedom of the network.

Error bound of critical points and KL property of exponent 1/2 for squared F-norm regularized factorization

This paper is concerned with the squared F(robenius)-norm regularized factorization form for noisy low-rank matrix recovery problems. Under a suitable assumption on the restricted condition number of the Hessian matrix of the loss function, we establish an error bound to the true matrix for the non-strict critical points with rank not more than that of the true matrix. Then, for the squared F-norm regularized factorized least squares loss function, we establish its KL property of exponent 1/2 on the global optimal solution set under the noisy and full sample setting, and achieve this property at its certain class of critical points under the noisy and partial sample setting. These theoretical findings are also confirmed by solving the squared F-norm regularized factorization problem with an accelerated alternating minimization method.

Low-rank matrix recovery with Ky Fan 2-k-norm

Low-rank matrix recovery problem is difficult due to its non-convex properties and it is usually solved using convex relaxation approaches. In this paper, we formulate the non-convex low-rank matrix recovery problem exactly using novel Ky Fan 2-k-norm-based models. A general difference of convex functions algorithm (DCA) is developed to solve these models. A proximal point algorithm (PPA) framework is proposed to solve sub-problems within the DCA, which allows us to handle large instances. Numerical results show that the proposed models achieve high recoverability rates as compared to the truncated nuclear norm method and the alternating bilinear optimization approach. The results also demonstrate that the proposed DCA with the PPA framework is efficient in handling larger instances.

Low-rank matrix recovery via regularized nuclear norm minimization

In this paper, we theoretically investigate the low-rank matrix recovery problem in the context of the unconstrained regularized nuclear norm minimization (RNNM) framework. Our theoretical findings show that, the RNNM method is able to provide a robust recovery of any matrix X (not necessary to be exactly low-rank) from its few noisy measurements b = A ( X ) + n with a bounded constraint ‖ n ‖ 2 ≤ ϵ , provided that the tk -order restricted isometry constant (RIC) of A satisfies a certain constraint related to t > 0 . Specifically, the obtained recovery condition in the case of t > 4 / 3 is found to be same with the sharp condition established previously by Cai and Zhang [10] to guarantee the exact recovery of any rank- k matrix via the constrained nuclear norm minimization method. More importantly, to the best of our knowledge, we are the first to establish the tk -order RIC based coefficient estimate of the robust null space property in the case of 0 < t ≤ 1 .

On the Simplicity and Conditioning of Low Rank Semidefinite Programs

Low rank matrix recovery problems appear widely in statistics, combinatorics, and imaging. One celebrated method for solving these problems is to formulate and solve a semidefinite program (SDP). I...