Low-rank Minimization Research Articles

“Sparse + Low-Rank” tensor completion approach for recovering images and videos

Recovering color images and videos from highly undersampled data is a fundamental and challenging task in face recognition and computer vision. By the multi-dimensional nature of color images and videos, in this paper, we propose a novel tensor completion approach, which is able to efficiently explore the sparsity of tensor data under the discrete cosine transform (DCT). Specifically, we introduce two “sparse + low-rank” tensor completion models as well as two implementable algorithms for finding their solutions. The first one is a DCT-based sparse plus weighted nuclear norm induced low-rank minimization model. The second one is a DCT-based sparse plus p-shrinking mapping induced low-rank optimization model. Moreover, we accordingly propose two implementable augmented Lagrangian-based algorithms for solving the underlying optimization models. A series of numerical experiments including color image inpainting and video data recovery demonstrate that our proposed approach performs better than many existing state-of-the-art tensor completion methods, especially for the case when the ratio of missing data is high.

Image cartoon-texture decomposition by a generalized non-convex low-rank minimization method

Image cartoon-texture decomposition is an important problem in image processing. In recent years, by exploiting low-rank priors of images, low-rank minimization methods have been widely adopted for image cartoon-texture decomposition. Since matrix rank minimization is an NP-hard problem, the convex nuclear norm is often used as a substitute for the matrix’s rank to realize the low-rank minimization methods. In this paper, we utilize a generalized non-convex surrogate of the matrix rank function to develop a novel low-rank minimization model for image cartoon-texture decomposition. We design a proximal alternating algorithm to solve the non-convex model and further demonstrate the global convergence of the algorithm. Numerical experiments illustrate that the proposed method can show much better performances than the existing state-of-the-art methods for image cartoon-texture decomposition.

Quarter match non-local means algorithm for noise removal

The notion of improving plays out in many forms in our lives. We look for better quality, faster speed, and leisurelier connections. To achieve our desired goals, we must ask questions. How to make a process stronger? How to make a process more efficient? How to make a process more effective? Image denoising plays a vital role in many professions and understanding how noise can be present in images has led to multiple denoising techniques. These techniques include total variation regularization, non-local regularization, sparse representation, and low-rank minimization just to name a few. Many of these techniques exist because of the concept of improvement. First, we have a change (problem). This change invokes thoughts and questions. How these changes occur and how they are handled play an essential role in the realization or malfunction of that process. With this understanding, first, we look to fully understand the process to achieve success. As it relates to image denoising, the non-local means is incredibly effective in image reconstruction. In particular, the non-local means filter removes noise and sharpens edges without losing too many fine structures and details. Also, the non-local means algorithm is amazingly accurate. Consequently, the disadvantage that plagues the non-local means filtering algorithm is the computational burden and it is due to the non-local averaging. In this paper, we investigate innovative ways to reduce the computational burden and enhance the effectiveness of this filtering process. Research examining image analysis shows there is a battle between noise reduction and the preservation of actual features, which makes the reduction of noise a difficult task. For exploration, we propose a quarter-match non-local means denoising filtering algorithm. The filters help to classify a more concentrated region in the image and thereby enhance the computational efficiency of the existing non-local means denoising methods and produce an enriched comparison for overlying in the restoration process. To survey the constructs of this new algorithm, the authors use the original non-local means filtering algorithm, which is coined, “State of the Art” and other selective processes to test the effectiveness and efficiency of the new model. When comparing the original non-local means with the new quarter match filtering algorithm, on average, we can reduce the computational cost by half, while improving the quality of the image. To further test our new algorithm, medical resonance (MR) and synthetic aperture radar (SAR) images are used as specimens for real-world applications.

Accelerated matrix completion algorithm using continuation strategy and randomized SVD

The matrix completion (MC) problem aims to recover the complete matrix from the observed matrix with missing entries. The unknown matrix is generally assumed to have a low-rank structure. It is well known that the low-rank minimization problem is NP-hard, so the rank function of the matrix is approximated by a surrogate function. In this paper, we firs t apply the optimization transfer technique to formulate the minimization problem into a bi-variate minimization problem by adding an auxiliary variable. We apply the alternating minimization method to find the minimizer of the bi-variate minimization problem. For the auxiliary variable, the minimum can be formulated as a linear combination of the observation data and the original variable. For the original variable, the minimum is a generalized singular thresholding (GSVT) operator of the auxiliary variable. However, most existing iteration methods for matrix completion problem use fixed thresholding value and also suffer from full singular value decomposition (SVD), which need more computation time for iteration. Thus, they become inefficient for large-scale problems. In order to handle these limitations, we apply the continuation strategy and the randomized singular value decomposition (RSVD) to reduce the computational costs. The threshold value of the continuous strategy is reduced in each iteration until a given minimum thresholding value is reached, while the RSVD only needs to decompose small-scale matrices. Hence, using both strategies simultaneously can effectively reduce the computation time for iteration. Numerical experiments demonstrate that the proposed algorithm can effectively recover the missing entries both in synthetic and real data.

Nonconvex [formula omitted]- regularized nonlocal self-similarity denoiser for compressive sensing based CT reconstruction

Compressive sensing (CS) based computed tomography (CT) image reconstruction aims at reducing the radiation risk through sparse-view projection data. However, it is challenging to achieve satisfying image quality from incomplete projections. Recent works demonstrate the promising potential of nonconvex L1/2-norm in CS problem, while the applications on medical imaging are constrained by its nonconvexity. In this paper, we develop an L1/2-regularized nonlocal self-similarity (NSS) denoiser based CT reconstruction model, which combines with low-rank approximation and group sparse coding (GSC) framework. Concretely, we firstly split the CT reconstruction problem into two subproblems, then improve CT image quality furtherly using our proposed denoiser. Instead of optimizing the nonconvex problem under the perspective of GSC, we particularly reconstruct CT image via low-rank minimization based on two simple yet essential schemes, which build the equivalent relationship between GSC based denoiser and low-rank minimization. Furtherly, the weighted singular value thresholding (WSVT) operator is utilized to optimize the resulting nonconvex L1/2 minimization problem. Following this, our proposed denoiser is integrated with the CT reconstruction problem by alternating direction method of multipliers (ADMM) framework. Extensive experimental results on typical clinical CT images have demonstrated that our approach can further achieve better performance than popular approaches.

A new non-convex low rank minimization model to decompose an image into cartoon and texture components

Decomposition of an image into cartoon and texture components is frequently used in many image processing applications. Here, the cartoon component has been characterized by the frequently used total variation norm. However, it becomes very challenging to obtain the texture component due to the varying nature of the texture. In general, the texture component has oscillatory behavior locally or globally. Owing to this oscillatory behavior, the texture component has been characterized via low-rank regularization which is widely used to extract texture component from the image. In the works reported till now, convex nuclear norm has been frequently used as a surrogate of the matrix rank, which is suboptimal because of shrinking each singular value equally, while the non-convex surrogate of the rank treats each singular value adaptively. In this paper, we are introducing a new tightest non-convex surrogate of the rank that assigns different weights to each singular value. The new non-convex image decomposition minimization model provides us cartoon and texture components by minimizing the total variation norm and non-convex function simultaneously. This model can also work best for many image restoration problems such as image deblurring and inpainting. The conventional alternating direction method of multiplier (ADMM) has been exploited as the solver of the non-convex minimization model. The proposed model works well on both globally patterned and natural images. In the experimental section, we demonstrate the outperformance of the proposed model over the state-of-the-art methods.

Large-Scale Affine Matrix Rank Minimization With a Novel Nonconvex Regularizer.

Low-rank minimization aims to recover a matrix of minimum rank subject to linear system constraint. It can be found in various data analysis and machine learning areas, such as recommender systems, video denoising, and signal processing. Nuclear norm minimization is a dominating approach to handle it. However, such a method ignores the difference among singular values of target matrix. To address this issue, nonconvex low-rank regularizers have been widely used. Unfortunately, existing methods suffer from different drawbacks, such as inefficiency and inaccuracy. To alleviate such problems, this article proposes a flexible model with a novel nonconvex regularizer. Such a model not only promotes low rankness but also can be solved much faster and more accurate. With it, the original low-rank problem can be equivalently transformed into the resulting optimization problem under the rank restricted isometry property (rank-RIP) condition. Subsequently, Nesterov's rule and inexact proximal strategies are adopted to achieve a novel algorithm highly efficient in solving this problem at a convergence rate of O(1/K) , with K being the iterate count. Besides, the asymptotic convergence rate is also analyzed rigorously by adopting the Kurdyka- ojasiewicz (KL) inequality. Furthermore, we apply the proposed optimization model to typical low-rank problems, including matrix completion, robust principal component analysis (RPCA), and tensor completion. Exhaustively empirical studies regarding data analysis tasks, i.e., synthetic data analysis, image recovery, personalized recommendation, and background subtraction, indicate that the proposed model outperforms state-of-the-art models in both accuracy and efficiency.

A matrix rank minimization-based regularization method for image restoration

Image restoration is a fundamental problem of image processing. In recent years, low-rank minimization (LRM) methods have been extensively applied to this problem to improve the image restoration effects. Due to the theoretical difficulties, the matrix's rank is often replaced by its convex surrogates such as the nuclear norm and the solution can be obtained by solving the Euler-Lagrange equation. In this paper, we propose a model for image restoration by utilizing the matrix rank directly. We design a proximal alternating rank minimization algorithm for solving the model. In each subproblem of the proposed algorithm, the closed-form solution can be easily obtained. The global convergence of the algorithm is also demonstrated by applying the Kudyka-Łojasiewicz property. Numerical experiments show that the proposed method can recover images better than some current state-of-the-art methods in terms of both the recovered measure quantities and visual qualities.

Moving Object Detection in Dynamic Environment via Weighted Low-Rank Structured Sparse RPCA and Kalman Filtering

As a classical problem for computer vision, moving object detection (MOD) can be efficiently achieved by foreground and background separation. The Robust Principal Component Analysis (RPCA)-based method has been potentially utilized to solve the problem. However, the detection accuracy for RPCA-based method is limited for complex scenes with slow-motion. Besides, it is time consuming for the way to seek for background modeling based on solving a low-rank minimization problem, for which multiple frames of the videos are required as the input. Therefore, a real-time MOD framework (LSRPCA_KF) is proposed for the dynamic background, where a weighted low-rank and structured sparse RPCA algorithm is used to achieve background modeling for history data, while the online MOD is achieved by the background subtraction method and updated by the Kalman filter for every real time frame. Specifically, for the background model, a newly designed weight is incorporated to distinguish the significance of different singular values, and a structured sparse prior is added to penalty the spatial connection property of the moving object. Besides, the weighted low-rank and structured sparse RPCA model is efficiently solved by the Alternating Direction Method of Multipliers (ADMM) optimization algorithm. Experimental results demonstrated that better performance of our method with significantly reduced delay in processing and better detect the moving object has been achieved, especially for the dynamic background.

Phase retrieval from multiple FRFT measurements based on nonconvex low-rank minimization

Phase retrieval (PR) problems aim to recover the phase from intensity measurements and are relevant in many applications such as X-ray crystallography and diffraction imaging. Most existing phase retrieval algorithms apply mask-based Fourier transform (FT) measurements to provide redundant information for signal reconstruction. This gives rise to a series of problems with masks such as the choice of suitable masks. In this work, we prove the rationality of multiple fractional Fourier transform (MFRFT) measurements, which can be applied to replace mask-based FT measurements. In addition, based on the low rank property (rank-1) of the desired matrix, we present a new low-rank model for PR problems using truncated nuclear norm regularization (TNNR), called TNNR-MFRFT. Extensive experiments validate the superiority of our approach over the state-of-the-art algorithms.

Block-based spectral image reconstruction for compressive spectral imaging using smoothness on graphs.

A novel reconstruction method for compressive spectral imaging is designed by assuming that the spectral image of interest is sufficiently smooth on a collection of graphs. Since the graphs are not known in advance, we propose to infer them from a panchromatic image using a state-of-the-art graph learning method. Our approach leads to solutions with closed-form that can be found efficiently by solving multiple sparse systems of linear equations in parallel. Extensive simulations and an experimental demonstration show the merits of our method in comparison with traditional methods based on sparsity and total variation and more recent methods based on low-rank minimization and deep-based plug-and-play priors. Our approach may be instrumental in designing efficient methods based on deep neural networks and covariance estimation.

Multi-view unsupervised feature selection with tensor low-rank minimization

To describe objects more comprehensively and accurately, multi-view learning has attracted considerable attention. Recently, graph embedding based multi-view feature selection methods have been proposed and shown efficient in many real applications. The existing methods generally construct one common graph matrix to exploit the local structure of multi-view data via the linear weight fusion or learning one common graph matrix across all views. However, since all views share the identical graph structure, this emphasizes the consistency too much, resulting in restricting the diversity among different views. In this paper, a tensor low-rank constrained graph embedding method is proposed for multi-view unsupervised feature selection. To embody the view-specific information of each view, our model constructs the graph structure for each corresponding view, respectively. To capture the consistency across views, a tensor low-rank regularization constraint is imposed on the tensor data formed by these graph matrices. An efficient optimization algorithm with theoretical convergence guarantee is designed to solve the proposed method. Extensive experimental results validate that the proposed method outperforms some state-of-the-art methods. The code of our model can be found athttps://www.researchgate.net/publication/353902948_demoTLR.

RFI Suppression for SAR via a Dictionary-Based Nonconvex Low-Rank Minimization Framework and Its Adaptive Implementation

Synthetic aperture radar (SAR) frequently suffers from radio frequency interference (RFI) due to the simultaneous presence of numerous wireless communication signals. Recently, the narrowband RFI is found to possess the low-rank property benefiting from stable frequency occupancy, hence the reconsideration of RFI suppression as a joint sparse and low-rank optimization problem. The existing methods either use the non-sparse useful signal itself as the sparse regularizer, or employ the nuclear norm to approximate the rank function, which punishes all singular values with the same penalty via singular value thresholding (SVT), resulting in the improper punishment problem. Hence, both are consequentially subject to performance limitation. In this paper, a novel dictionary-based nonconvex low-rank minimization (DNLRM) optimization framework is proposed for RFI suppression, which concurrently considers the improvements for both the sparse regularizer and the low-rank regularizer. For the former, an over-completed dictionary is constructed, for which the sparse coefficient acts as the sparse regularizer. For the latter, the rank function is more accurately approximated by innovatively introducing the nonconvex function, for which the supergradient is synchronously used to generate the weighted penalty, thus solving the improper punishment problem. The derivation of the closed-form solution and the convergence analysis are described in detail. Additionally, the adaptive selection scheme for the model parameter is uniquely proposed for further ensuring the practicality of the DNLRM framework. The superiority of the proposed method is demonstrated via not only the RFI-free real SAR data combined with the measured RFI, but the RFI-contaminated real SAR data.

A Patch-Based Low-Rank Minimization Approach for Speckle Noise Reduction in Ultrasound Images

A Patch-Based Low-Rank Minimization Approach for Speckle Noise Reduction in Ultrasound Images

DLRP: Learning Deep Low-Rank Prior for Remotely Sensed Image Denoising

Remotely sensed images degraded by additive white Gaussian noise (AWGN) are not beneficial for the analysis of their contents. Such a phenomenon is usually modeled as an inverse problem which can be solved by model-based optimization methods or discriminative learning approaches. The former pursue their pleasing performance at the cost of a highly computational burden while the latter are impressive for their fast testing speed but are limited by their application range. To join their merits, this letter proposes a nonlocal self-similar (NSS) block-based deep image denoising scheme, namely deep low-rank prior (DLRP), which includes the following key points: First, the low-rank property of the neighboring NSS patches ordered lexicographically is utilized to model a global objective function (GOF). Second, with the aid of an alternative iteration strategy, the GOF can be easily decomposed into two independent subproblems. One is a quadratic optimization problem, and has a closed-form solution. While the other is a low-rank minimization denoising problem and is learned by deep convolutional neural network (DCNN). Then, the deep denoiser, acted as a modular part, is plugged into the model-based optimization method with adaptive noise level estimation to solve the inverse problem. In the experiments, we first discuss parameter setting and the convergence. Then, quantitative/qualitative comparisons of experimental results validate that the DLRP is a flexible and powerful denoising method to achieve competitive performance which even outperforms those produced by state-of-the-arts.

A fast low rank Vandermonde factorization reconstruction method for non-uniformly sampled 2D NMR spectroscopy

As an indispensable analytical method in the fields of chemistry and medicine, nuclear magnetic resonance (NMR) spectroscopy is widely adopted for analyzing the composition and structure of substances. However, the data acquisition period for high-resolution NMR spectra is extremely long. Therefore, non-uniform sampling is commonly adopted to reduce the measurement time and the complete NMR signal can be obtained by appropriate reconstruction algorithms. Recently, it was found that NMR signal can be reconstructed well by low-rank Hankel matrix completion. However, the computational time required for this method to reconstruct two-dimensional (2D) and higher dimensional signals is relatively long because it increases rapidly as spectral resolution improves. In this paper, we propose a novel low rank minimization method to alleviate this problem by utilizing the unexploited property that a 2D NMR signal can be decomposed into two Vandermonde matrices. However, minimizing the rank necessitates the time-consuming singular value decomposition (SVD). Thus we introduce a matrix bi-factorization idea into the method to reduce the time complexity and greatly mitigate the time required by the SVD. Experiments on synthetic data and realistic NMR spectroscopy data demonstrate that compared with the state-of-the-art nuclear norm based minimization and fast iterative hard thresholding algorithms, the computational time of the proposed algorithm is significantly reduced, and high reconstruction accuracy can be ensured at the same time.

Curve Approximation Models based on Statistical Distribution with Application to Photoplethysmography (PPG) Signal

Photoplethysmography (PPG) signal captures blood volume change in the arteries that carry blood. PPG signals are often used to check the cardiovascular health in patients. Health care automation have made more importance into study of PPG signal and its automatic prognosis and diagnosis. In this paper we aim to achieve motion artifact reduction using low rank minimization and PPG signal representation in mathematical form using statistical distribution models. The proposed approach has been tested for Gaussian, Bifurcated Gaussian, Exponentially broadened Gaussian and lognormal distributions. The accuracy of each distribution in modeling the PPG signal was also studied.

A new Lagrangian-Benders approach for a concave cost supply chain network design problem

This article presents an extended facility location model for multiproduct supply chain network design, that accounts for concave capacity, transportation, and inventory costs (induced by economies of scale, quantity discounts and risk pooling, respectively). The problem is formulated as a mixed-integer nonlinear program (MINLP) with linear constraints and a large number of separable concave terms in the objective function. We propose a solution approach that combines stabilized Lagrangian relaxation with a novel Benders decomposition. Lagrangian relaxation is used first to decompose the problem by potential warehouse location into low-rank concave minimization subproblems. Benders decomposition is then applied to solve the subproblems by shifting the concave terms to a low-dimensional master problem that is solved effectively through implicit enumeration. Finally, a feasible solution for the original problem is constructed from the partial subproblems solutions by solving a restricted set covering problem. The proposed approach is tested on several instances from the literature and compared against a state-of-the-art solver. Furthermore, a realistic case study is used to demonstrate the impact of concavities in the cost components with extensive sensitivity analyses.

An accelerated IRNN-Iteratively Reweighted Nuclear Norm algorithm for nonconvex nonsmooth low-rank minimization problems

Low-rank minimization problems arise in numerous important applications such as recommendation systems, machine learning, network analysis, and so on. The problems however typically consist of minimizing a sum of a smooth function and nonconvex nonsmooth composite functions, which solving them remains a big challenge. In this paper, we take inspiration from the Nesterov’s acceleration technique to accelerate an iteratively reweighted nuclear norm algorithm for the considered problems ensuring that every limit point is a critical point. Our algorithm iteratively computes the proximal operator of a reweighted nuclear norm which has a closed-form solution by performing the SVD of a smaller matrix instead of the full SVD. This distinguishes our work from recent accelerated proximal gradient algorithms that require an expensive computation of the proximal operator of nonconvex nonsmooth composite functions. We also investigate the convergence rate with the Kurdyka–Łojasiewicz assumption. Numerical experiments are performed to demonstrate the efficiency of our algorithm and its superiority over well-known methods.

A novel MR image denoising via LRMA and NLSS

Nonlocal self-similarity has been proven to be a useful tool for image denoising. For MR image denoising, the method combining the nonlocal self-similarity with the low-rank approximation has been recently attracting considerable attentions, due to its favorable performance. Since the original low-rank approximation problem is difficult to be solved, the frequently used method is to use the nuclear norm minimization for the matrix low-rank approximation. However, the solution obtained by nuclear norm minimization generally deviates from the solution of the original problem. In this paper, an approach for MR image denoising is proposed by combining a novel nonlocal self-similarity scheme with a novel low-rank approximation scheme. In proposed approach, a similarity evaluation with respect to the noise is proposed in the patch matching stage. To approximate the original low-rank minimization problem, the propose approach minimizes trace-based operator at each step. Every minimization is solvable and used to approximate the original low-rank minimization. An algorithm is established for this approximation, as well. Experimental results show that the proposed approach has a superior performance, comparing with some of the low-rank approximation methods, in both the objective quality metrics and visual inspections.