Low-rank Minimization Research Articles

Quartic First-Order Methods for Low-Rank Minimization

We study a general nonconvex formulation for low-rank minimization problems. We use recent results on non-Euclidean first-order methods to provide efficient and scalable algorithms. Our approach uses the geometry induced by the Bregman divergence of well-chosen kernel functions; for unconstrained problems, we introduce a novel family of Gram quartic kernels that improve numerical performance. Numerical experiments on Euclidean distance matrix completion and symmetric nonnegative matrix factorization show that our algorithms scale well and reach state-of-the-art performance when compared to specialized methods.

An Inexact Penalty Decomposition Method for Sparse Optimization.

The penalty decomposition method is an effective and versatile method for sparse optimization and has been successfully applied to solve compressed sensing, sparse logistic regression, sparse inverse covariance selection, low rank minimization, image restoration, and so on. With increase in the penalty parameters, a sequence of penalty subproblems required being solved by the penalty decomposition method may be time consuming. In this paper, an acceleration of the penalty decomposition method is proposed for the sparse optimization problem. For each penalty parameter, this method just finds some inexact solutions to those subproblems. Computational experiments on a number of test instances demonstrate the effectiveness and efficiency of the proposed method in accurately generating sparse and redundant representations of one-dimensional random signals.

Learning Deep Sparse Regularizers With Applications to Multi-View Clustering and Semi-Supervised Classification.

Sparsity-constrained optimization problems are common in machine learning, such as sparse coding, low-rank minimization and compressive sensing. However, most of previous studies focused on constructing various hand-crafted sparse regularizers, while little work was devoted to learning adaptive sparse regularizers from given input data for specific tasks. In this paper, we propose a deep sparse regularizer learning model that learns data-driven sparse regularizers adaptively. Via the proximal gradient algorithm, we find that the sparse regularizer learning is equivalent to learning a parameterized activation function. This encourages us to learn sparse regularizers in the deep learning framework. Therefore, we build a neural network composed of multiple blocks, each being differentiable and reusable. All blocks contain learnable piecewise linear activation functions which correspond to the sparse regularizer to be learned. Furthermore, the proposed model is trained with back propagation, and all parameters in this model are learned end-to-end. We apply our framework to multi-view clustering and semi-supervised classification tasks to learn a latent compact representation. Experimental results demonstrate the superiority of the proposed framework over state-of-the-art multi-view learning models.

Remote sensing image classification using subspace sensor fusion

The amount of remote sensing and ancillary datasets captured by diverse airborne and spaceborne sensors has been tremendously increased, which opens up the possibility of utilizing multimodal datasets to improve the performance of processing approaches with respect to the application at hand. However, developing a generic framework with high generalization capability that can effectively fuse diverse datasets is a challenging task since the current approaches are usually only applicable to two specific sensors for data fusion. In this paper, we propose an accurate fusion-based technique called SubFus with capability to integrate diverse remote sensing data for land cover classification. Here, we assume that a high dimensional multisensor dataset can be represented fused features that live in a lower-dimensional space. The proposed classification methodology includes three main stages. First, spatial information is extracted by using spatial filters (i.e., morphology filters). Then, a novel low-rank minimization problem is proposed to represent the multisensor datasets in subspaces using fused features. The fused features in the lower-dimensional subspace are estimated using a novel iterative algorithm based on the alternative direction method of multipliers. Third, the final classification map is produced by applying a supervised spectral classifier (i.e., random forest) on the fused features. In the experiments, the proposed method is applied to a three-sensor (RGB, multispectral LiDAR, and hyperspectral images) dataset captured over the area of the University of Houston, the USA, and a two-sensor (hyperspectral and LiDAR) dataset captured over the city of Trento, Italy. The land-cover maps generated using SubFus are evaluated based on classification accuracies. Experimental results obtained by SubFus confirm considerable improvements in terms of classification accuracies compared with the other methods used in the experiments. The proposed fusion approach obtains 85.32% and 99.25% in terms of overall classification accuracy on the Houston (the training portion of the dataset distributed for the data fusion contest of 2018) and trento datasets, respectively.

From group sparse coding to rank minimization: A novel denoising model for low-level image restoration

Recently, low-rank matrix recovery theory has been emerging as a significant progress for various image processing problems. Meanwhile, the group sparse coding (GSC) theory has led to great successes in image restoration (IR) problem with each group contains low-rank property. In this paper, we propose a novel low-rank minimization based denoising model for IR tasks under the perspective of GSC, an important connection between our denoising model and rank minimization problem has been put forward. To overcome the bias problem caused by convex nuclear norm minimization (NNM) for rank approximation, a more generalized and flexible rank relaxation function is employed, namely weighted nonconvex relaxation. Accordingly, an efficient iteratively-reweighted algorithm is proposed to handle the resulting minimization problem combing with the popular L1/2 and L2/3 thresholding operators. Finally, our proposed denoising model is applied to IR problems via an alternating direction method of multipliers (ADMM) strategy. Typical IR experiments on image compressive sensing (CS), inpainting, deblurring and impulsive noise removal demonstrate that our proposed method can achieve significantly higher PSNR/FSIM values than many relevant state-of-the-art methods.

Nonconvex nonsmooth low-rank minimization for generalized image compressed sensing via group sparse representation

Group sparse representation (GSR) based method has led to great successes in various image recovery tasks, which can be converted into a low-rank matrix minimization problem. As a widely used surrogate function of low-rank, the nuclear norm based convex surrogate usually leads to over-shrinking problem, since the standard soft-thresholding operator shrinks all singular values equally. To improve traditional sparse representation based image compressive sensing (CS) performance, we propose a generalized CS framework based on GSR model, which leads to a nonconvex nonsmooth low-rank minimization problem. The popular L2-norm and M-estimator are employed for standard image CS and robust CS problem to fit the data respectively. For the better approximation of the rank of group-matrix, a family of nuclear norms are employed to address the over-shrinking problem. Moreover, we also propose a flexible and effective iteratively-weighting strategy to control the weighting and contribution of each singular value. Then we develop an iteratively reweighted nuclear norm algorithm for our generalized framework via an alternating direction method of multipliers framework, namely, GSR-AIR. Experimental results demonstrate that our proposed CS framework can achieve favorable reconstruction performance compared with current state-of-the-art methods and the robust CS framework can suppress the outliers effectively.

Edge Computing-Empowered Large-Scale Traffic Data Recovery Leveraging Low-Rank Theory

Intelligent Transportation Systems (ITSs) have been widely deployed to provide traffic sensing data for a variety of smart traffic applications. However, the inevitable and ubiquitous missing data potentially compromises the performance of ITSs and even undermines the traffic applications. Therefore, accurate and real-time traffic data recovery is crucial to ITSs and its related services, especially for large-scale traffic networks. To leverage the characteristics in transportation networks for data recovery, we first conduct experimental explorations on a large-scale traffic dataset of an ITS and further quantify the spatiotemporal correlations of traffic data. Inspired by the observation results, we propose GTR , an edGe computing-empowered system for large-scale Traffic data recovery with low-Rank theory. GTR leverages the decentralized computing power of edge nodes to process massive traffic data from hundreds of traffic stations for accurate and real-time recovery. Specifically, we first propose a suboptimal edge node deployment algorithm with a theoretical performance guarantee, by exploiting the supermodularity in the NP-hard joint-optimization problem. Furthermore, to leverage the low-rank nature of traffic data, we transform the data recovery problem into a low-rank minimization problem, then utilize the fixed-point continuation iterative scheme to capture spatiotemporal correlations for accurate traffic recovery. Finally, the extensive trace-driven evaluations show that GTR only needs at most 5.7% extra total cost compared to the optimal deployment, while outperforming four baseline methods by 63.8% improvement in terms of traffic data recovery accuracy.

An Automated Unified Framework for Video Deraining and Simultaneous Moving Object Detection in Surveillance Environments

In many instances, outdoor surveillance systems suffer from atrocious weather conditions such as rain, since images or videos captured by such vision systems in rainy days may undergo severe visual dilapidations. This can cause a glitch in those algorithms which are further used for object detection and tracking. Therefore, an ancillary video processing algorithm namely, video deraining is necessary prior to the implementation of object detection and tracking. This indicates the requirement of a time-consuming and complicated two-step process for the detection of moving objects in a rainy environment. This paper proposes an automated single-stage formulation for the conventional three-stage procedure such as rain steak removal, original data recovery and moving object detection as simultaneous operation in the tensor framework. The brilliance of this work is confined in the efficient formulation of an operator termed as Slice Rotational Total Variation (SRTV), to unify the different rain patterns into a common pattern so that any form of rain pattern can be effectively removed from the rainy data in a visually appealing manner by preserving the important details of the data. In this paper, SRTV regularization and tensor low-rank minimization are utilized for the effective deraining as well as efficient retrieval of clean background. Besides, $l_{1}$ norm and Tensor Total Variation (TTV) regularizers together with SRTV regularizer are employed for the faithful detection of derained moving objects. The experimental results show that the proposed method outperforms the state of the art methods in terms of deraining capability and accurate moving object detection.

Robust approximations of low-rank minimization for tensor completion

Motivated by the nuclear norm of tensors and nonconvex approximations of matrix rank, we propose three robust approximations of multi-linear rank for tensor completion. For each method, we develop an efficient algorithm to solve the corresponding optimization problem. Besides, we prove that every cluster point of the sequence, generated by the respective algorithm, is a stationary point. To obtain a more robust reconstruction, we design an updating rule of parameters for each method. Our empirical experiments on real-world data show that the proposed methods deliver state-of-the-art performance in the reconstruction of low-rank tensors.

A SAR Image Despeckling Method Using Multi-Scale Nonlocal Low-Rank Model

Speckle noise is an inherent nature of synthetic aperture radar (SAR) images, which degrades the quality of the images and makes the interpretation of SAR images difficult. In this letter, we propose a despeckling method by simultaneously exploring low-rank prior and multi-scale prior of SAR images. Especially, we propose a low-rank minimization model by considering a data fidelity term derived from the Fisher–Tippett distribution and a weighted nuclear norm regularization term. Furthermore, we explore the multi-scale prior by selecting similar patches from different scales of the SAR image. The resulting optimization problem is solved by the alternating direction method of multipliers (ADMM). Experiments conducted on both simulated and real SAR images demonstrate that the proposed method can provide promising despeckling results in terms of speckle reduction and texture and edge details preservation.

SAR Image Despeckling Based on Nonlocal Low-Rank Regularization

In this paper, we propose a new synthetic aperture radar (SAR) image despeckling method based on the nonlocal low-rank minimization model. First, some similar image patches are selected for each pixel to construct the patch group matrix (PGM). Then, a new low-rank minimization model, called Fisher–Tippett distribution (FT)-weighted nuclear norm minimization (WNNM), is proposed to recover the underlying low-rank component from the PGM. Specifically, the FT-WNNM is developed by reformulating the despeckling problem as the maximizing a posterior probability problem. The new model consists of a data fidelity term and a regularization term (also called prior term). The data fidelity term is derived from the statistical distribution of SAR images in the logarithm domain, which is known as the Fisher–Tippett distribution, and the regularization term is the recent weighted nuclear norm. Then, the alternating direction method of multipliers (ADMM) is introduced to solve the corresponding optimization problem. Under ADMM framework, the resulting subproblems can be solved efficiently and the convergence can be guaranteed. Extensive experiments on both simulated and real SAR images demonstrate that the proposed method can achieve comparable or even better despeckling performance than some state-of-the-art despeckling algorithms.

K4SID: Large-Scale Subspace Identification With Kronecker Modeling

In this paper, we consider the identification of matrix state-space models (MSSM) of the following form: \begin{align*} \mathbf {X}(k+1) &= \mathbf {A}_2 \mathbf {X}(k) \mathbf {A}_1^T + \mathbf {B}_2 \mathbf {U}(k) \mathbf {B}_1^T \mathbf {Y}(k) &= \mathbf {C}_2 \mathbf {X}(k) \mathbf {C}_1^T + \mathbf {E}(k) \end{align*} for all time dependent quantities and matrices of appropriate dimensions. Due to the large size of these matrices, vectorization does not allow the use of standard multivariable subspace methods such as N4SID or MOESP. In this paper, the resulting Kronecker structure that appears in the system matrices due to vectorization is exploited for developing a scalable subspace-like identification approach. This approach consists of first estimating the Markov parameters associated to the MSSM via the solution of a regularized bilinear least-squares problem that is solved in a globally convergent manner. Second, a bilinear low-rank minimization problem is tackled which allows to write a three-dimensional low-rank tensor and consequently to estimate the state sequence and the lower-dimensional matrices $\mathbf {A}_1,\mathbf {A}_2,\mathbf {B}_1,\mathbf {B}_2,\mathbf {C}_1,\mathbf {C}_2$ . A numerical example on a large-scale adaptive optics system demonstrates the ability of the algorithm to handle the identification of state-space models within the class of Kronecker structured matrices in a scalable manner which results in more compact models.

Image Completion Using Low Tensor Tree Rank and Total Variation Minimization

Tensor completion recovers missing entries of multiway data. Most of the current methods exploit the low-rank tensor structure for image completion applications. In this paper, we simultaneously exploit the globally multidimensional structure and locally piecewise smoothness to further enhance the performance. In the proposed optimization model, the low tensor tree rank minimization is used for the global data structure, and the total variation minimization is used for the local structure. Two kinds of total variation functions are discussed. The optimization problem is transformed into several subproblems by alternating direction method of multipliers. The subproblem on low tensor tree rank minimization is solved by singular value thresholding, and the subproblem on total variation minimization can be solved by soft thresholding. Numerical experiments on color images and light field images demonstrate that the proposed method outperforms most of the state-of-the-art methods in terms of recovery accuracy and computational complexity.

Line survey joint denoising via low-rank minimization

Prestack seismic data denoising is an important step in seismic processing due to the development of prestack time migration. Reduced-rank filtering is a state-of-the-art method for prestack seismic denoising that uses predictability between neighbor traces for each single frequency. Different from the original way of embedding low-rank matrix based on the Hankel or Toeplitz transform, we have developed a new multishot gathers joint denoising method in a line survey, which used a new way of rearranging data to a matrix with low rank. Inspired by video denoising, each single-shot record in the line survey can be viewed as a frame in the video sequence. Due to high redundancy and similar event structure among the shot gathers, similar patches can be selected from different shot gathers in the line survey to rearrange a low-rank matrix. Then, seismic denoising is formulated into a low-rank minimization problem that can be further relaxed into a nuclear-norm minimization problem. A fast algorithm, called the orthogonal rank-one matrix pursuit, is used to solve the nuclear-norm minimization. Using this method avoids the computation of a full singular value decomposition. Our method is validated using synthetic and field data, in comparison with [Formula: see text] deconvolution and singular spectrum analysis methods.

Robust Structured Low-Rank Representation for Image Segmentation

Subspace clustering analysis algorithms are often employed when dealing with high-dimensional data. As a representative approach, Low-Rank Representation (LRR) of data has achieved great success for subspace segmentation tasks in applications such as image processing. The traditional LRR-related methods consist of two separate tasks: first, the affinity graph construction by using lowrank minimization techniques, and then the spectral clustering, which is done on the affinity graph to get the final segmentation. Since these two steps are independent of each other, this method does not guarantee that the results obtained by the algorithm are globally optimal. In this paper, a method called Robust Structured Low-Rank Representation (RSLRR) is proposed, by integrating the two above mentioned tasks and solve a joint optimization problem. This paper also puts forward a method to solve the joint optimization problem, which can efficiently get both the segmentation and the structured low-rank representation. Experiments on several standard datasets show that, compared with other algorithms, the algorithm proposed in this paper can achieve better clustering results.

Multiple view image denoising using 3D focus image stacks

In this paper, we introduce a novel multi-view image denoising algorithm using 3D focus image stacks (3DFIS) to exploit image redundancy within and across views. Robust disparity map is first estimated using the 3DFIS with texture-based view selection and patch-size variation scheme. Leveraging both 3DFIS and the estimated disparity map, the proposed algorithm effectively denoises the target view from multiple views through a low rank minimization approach that incorporates robust similarity metrics and occlusion handling techniques. The paper combs through a number of existing image denoising methods, including the preliminary results in our earlier research efforts, and then details the ways, means and merits of our proposed algorithm. With extensive experiments, we conclude that this novel algorithm is superior over various existing state-of-the-art approaches in terms of both visual and quantitative performance.

Performance Analysis of Joint-Sparse Recovery from Multiple Measurement Vectors via Convex Optimization: Which Prior Information is Better?

In sparse signal recovery of compressive sensing, the phase transition determines the edge, which separates successful recovery and failed recovery. The phase transition can be seen as an indicator and an intuitive way to judge, which recovery performance is better. Traditionally, the multiple measurement vectors (MMVs) problem is usually solved via $\ell _{2,1}$ -norm minimization, which is our first investigation via conic geometry in this paper. Then, we are interested in the same problem but with two common constraints (or prior information): prior information relevant to the ground truth and the inherent low rank within the original signal. To figure out which constraint is most helpful, the MMVs problems are solved via $\ell _{2,1}$ - $\ell _{2,1}$ minimization and $\ell _{2,1}$ -low rank minimization, respectively. By theoretically presenting the necessary and sufficient condition of successful recovery from MMVs, we can have a precise prediction of phase transition to judge, which constraint or prior information is better. All our findings are verified via simulations and show that, under certain conditions, $\ell _{2,1}$ - $\ell _{2,1}$ minimization outperforms $\ell _{2,1}$ -low rank minimization. Surprisingly, $\ell _{2,1}$ -low rank minimization performs even worse than $\ell _{2,1}$ -norm minimization. To the best of our knowledge, we are the first to study the MMVs problem under different prior information in the context of compressive sensing.

A note on patch-based low-rank minimization for fast image denoising

Patch-based low-rank minimization for image processing attracts much attention in recent years. The minimization of the matrix rank coupled with the Frobenius norm data fidelity can be solved by the hard thresholding filter with principle component analysis (PCA) or singular value decomposition (SVD). Based on this idea, we propose a patch-based low-rank minimization method for image denoising. The main denoising process is stated in three equivalent way: PCA, SVD and low-rank minimization. Compared to recent patch-based sparse representation methods, experiments demonstrate that the proposed method is rather rapid, and it is effective for a variety of natural grayscale images and color images, especially for texture parts in images. Further improvements of this method are also given. In addition, due to the simplicity of this method, we could provide an explanation of the choice of the threshold parameter, estimation of PSNR values, and give other insights into this method.

Low-rank based infrared spectral feature extraction framework for quantitative analysis

Feature extraction is a key problem in spectral analysis. Spectrum collected with spectrometer have latent low-rank component. If spectrum can be represented as a superposition of low-rank component and an approximation term, the spectrum feature is obtained. In this paper, a novel low-rank based infrared spectral feature extraction method is proposed. Employing a slide window to convert a single spectrum into a matrix, which can be decomposed as the superposition of a low-rank component and feature. In machine learning, nuclear norm is employed to approximate to low-rank minimization. Thus, the model can be written as a combination of the nuclear norm and an approximation term. We have proposed an efficient algorithm with singular value decomposition to the model. Solving the model, we obtain the latent low-rank component in spectrum. The feature is obtained via derivative of original and low-rank approximation. Then the quantitative analysis model is directly built with the feature. The advantage of proposed method is that extraction procedure of one spectrum is not affected by other spectrum. Extensive experiments are conducted with four public data sets and experimental results demonstrate that our proposed feature extraction method can lead to accuracy improvements over state-of-the-art methods.

A novel method for a class of structured low-rank minimizations with equality constraint

The positive semidefinite constraint and equality constraint arise widely in matrix optimization problems of different areas including signal/image processing, finance and risk management. In this paper, an inexact accelerated Augmented Lagrangian Method (ALM) relying on a parameter m is designed to solve the structured low-rank minimization with equality constraint, which is more general and flexible than the existing ALM and its variants. We prove a worst-case O ( 1 ∕ k 2 ) convergence rate of the new method in terms of the residual of the Lagrangian function, and we analyze that when m ∈ [ 0 , 1 ) the residual of our method is smaller than that of the traditional accelerated ALM. Compared with several state-of-the-art methods, preliminary numerical experiments on solving the Q-weighted low-rank correlation matrix problem from finance validate the efficiency of the proposed method.