Low-rank Matrix Decomposition Research Articles

Randomized two-sided subspace iteration for low-rank matrix and tensor decomposition

The low-rank approximation of big data matrices and tensors plays a pivotal role in many modern applications. Although, a truncated version of the singular value decomposition (SVD) furnishes the best approximation, its computation is challenging on modern, multicore architectures. Recently, the randomized subspace iteration has shown to be a powerful tool in approximating large-scale matrices. In this paper we present a two-sided variant of the randomized subspace iteration. Novel in our work is the exploitation of the unpivoted QR factorization, rather than the SVD, for factorizing the compressed matrix. Hence our algorithm is a randomized rank-revealing URV decomposition. We prove the rank-revealingness of our algorithm by establishing bounds for the singular values as well as the other blocks of the compressed matrix. We further provide bounds on the error of the low-rank approximations of the proposed algorithm, in both 2- and Frobenius norm. In addition, we employ the proposed algorithm to efficiently compute low rank tensor decompositions: we present two randomized algorithms, one for the truncated higher-order SVD, and the other for the tensor SVD. We conduct numerical tests on (i) various classes of matrices, and (ii) synthetic tensors and real datasets to demonstrate the efficacy of the proposed algorithms.

SAR Jamming Suppression by Exploiting Polarized Similarity with Low-Rank and Sparse Matrix Decomposition

Optimal hyper-parameter selection for low-rank and sparse matrix decomposition (LRSMD) in synthetic aperture radar jamming suppression is usually challenging. This letter proposes an effective approach to LRSMD jamming suppression by exploiting the polarized similarity. A polarimetric ratio function is established to straightforwardly determine the rank hyper-parameter. The polarimetric ratio function is defined as the energy ratio of the decomposed low-rank jamming component to the sparse target signal, which is consistent across multiple polarized channels due to the equal jamming gain distinguished from the target polarized scattering. The algorithm optimally exploits the polarized similarity of jamming components to determine the rank hyper-parameter. It provides enhanced robustness and accuracy in SAR jamming removal, confirmed by synthetic experiments.

Algorithm for Multifocus Image Fusion Based on Low-Rank and Sparse Matrix Decomposition and Discrete Cosine Transform

Algorithm for Multifocus Image Fusion Based on Low-Rank and Sparse Matrix Decomposition and Discrete Cosine Transform

Enhanced Low-Rank Matrix Decomposition for High-Resolution UAV-SAR Imagery

Enhanced Low-Rank Matrix Decomposition for High-Resolution UAV-SAR Imagery

Nonmonotone variable projection algorithms for matrix decomposition with missing data

This paper investigates algorithms for matrix factorization when some or many components are missing, a problem that arises frequently in computer vision and pattern recognition. We demonstrate that the Jacobian used in the damped Wiberg (DW) method is exactly the same as that of Kaufman’s simplified variable projection (VP) algorithm. Our analysis provides a novel perspective on the efficiency of VP algorithms by improving the strong convexity of the approximate function. To enhance numerical stability, we set a lower bound on the damping parameter instead of adding a null space like the DW algorithm. Another challenge of low-rank matrix decomposition with missing data is the existence of many sharp local minima, which are often distributed in narrow valleys of the landscape of objection functions. Falling into such minima tends to result in poor reconstruction results. To address this issue, we design a non-monotonic VP algorithm, which can facilitate the algorithm to escape from sharp minima and converge to flatter minima. Numerical experiments confirm the effectiveness and efficiency of the proposed nonmonotone VP algorithm.

Compressed SVD-based L + S model to reconstruct undersampled dynamic MRI data using parallel architecture.

Magnetic Resonance Imaging (MRI) is a highly demanded medical imaging system due to high resolution, large volumetric coverage, and ability to capture the dynamic and functional information of body organs e.g. cardiac MRI is employed to assess cardiac structure and evaluate blood flow dynamics through the cardiac valves. Long scan time is the main drawback of MRI, which makes it difficult for the patients to remain still during the scanning process. By collecting fewer measurements, MRI scan time can be shortened, but this undersampling causes aliasing artifacts in the reconstructed images. Advanced image reconstruction algorithms have been used in literature to overcome these undersampling artifacts. These algorithms are computationally expensive and require a long time for reconstruction which makes them infeasible for real-time clinical applications e.g. cardiac MRI. However, exploiting the inherent parallelism in these algorithms can help to reduce their computation time. Low-rank plus sparse (L+S) matrix decomposition model is a technique used in literature to reconstruct the highly undersampled dynamic MRI (dMRI) data at the expense of long reconstruction time. In this paper, Compressed Singular Value Decomposition (cSVD) model is used in L+S decomposition model (instead of conventional SVD) to reduce the reconstruction time. The results provide improved quality of the reconstructed images. Furthermore, it has been observed that cSVD and other parts of the L+S model possess highly parallel operations; therefore, a customized GPU based parallel architecture of the modified L+S model has been presented to further reduce the reconstruction time. Four cardiac MRI datasets (three different cardiac perfusion acquired from different patients and one cardiac cine data), each with different acceleration factors of 2, 6 and 8 are used for experiments in this paper. Experimental results demonstrate that using the proposed parallel architecture for the reconstruction of cardiac perfusion data provides a speed-up factor up to 19.15× (with memory latency) and 70.55× (without memory latency) in comparison to the conventional CPU reconstruction with no compromise on image quality. The proposed method is well-suited for real-time clinical applications, offering a substantial reduction in reconstruction time.

Hyperspectral Anomaly Detection with Auto-Encoder and Independent Target

As an unsupervised data representation neural network, auto-encoder (AE) has shown great potential in denoising, dimensionality reduction, and data reconstruction. Many AE-based background (BKG) modeling methods have been developed for hyperspectral anomaly detection (HAD). However, their performance is subject to their unbiased reconstruction of BKG and target pixels. This article presents a rather different low rank and sparse matrix decomposition (LRaSMD) method based on AE, named auto-encoder and independent target (AE-IT), for hyperspectral anomaly detection. First, the encoder weight matrix, obtained by a designed AE network, is utilized to construct a projector for generating a low-rank component in the encoder subspace. By adaptively and reasonably determining the number of neurons in the latent layer, the designed AE-based method can promote the reconstruction of BKG. Second, to ensure independence and representativeness, the component in the encoder orthogonal subspace is made into a sphere and followed by finding of unsupervised targets to construct an anomaly space. In order to mitigate the influence of noise on anomaly detection, sparse cardinality (SC) constraint is enforced on the component in the anomaly space for obtaining the sparse anomaly component. Finally, anomaly detector is constructed by combining Mahalanobi distance and multi-components, which include encoder component and sparse anomaly component, to detect anomalies. The experimental results demonstrate that AE-IT performs competitively compared to the LRaSMD-based models and AE-based approaches.

A Fast and Super-Resolution Method of Vortex-Based Imaging

With characteristics of particular wavefronts and orthogonal modes, the sparse imaging method based on vortex waves could obtain high-resolution reconstruction. However, building a large-scale observation matrix is time-consuming and heavily burdens the whole imaging procedure. Inspired by the spherical harmonic expansion of plane waves, this paper provides a fast super-resolution imaging method based on vortex waves. Featuring spherical harmonic functions and a transformation, an orthogonal matching pursuit algorithm for high-resolution imaging exploits a low-rank matrix decomposition of the observation matrix. Simulation results show that the proposed method exhibits significant efficiency advantages while retaining high resolution, achieving an efficiency of up to 51.4% when applied to a large-scale observation matrix.

Through the Wall Radar Imaging via Kronecker-structured Huber-type RPCA

The detection of multiple targets in an enclosed scene, from its outside, is a challenging topic of research addressed by Through-the-Wall Radar Imaging (TWRI). Traditionally, TWRI methods operate in two steps: first the removal of wall clutter then followed by the recovery of targets positions. Recent approaches manage in parallel the processing of the wall and targets via low rank plus sparse matrix decomposition and obtain better performances. In this paper, we reformulate this precisely via a RPCA-type problem, where the sparse vector appears in a Kronecker product. We extend this approach by adding a robust distance with flexible structure to handle heterogeneous noise and outliers, which may appear in TWRI measurements. The resolution is achieved via the Alternating Direction Method of Multipliers (ADMM) and variable splitting to decouple the constraints. The removal of the front wall is achieved via a closed-form proximal evaluation and the recovery of targets is possible via a tailored Majorization–Minimization (MM) step. The analysis and validation of our method is carried out using Finite-Difference Time-Domain (FDTD) simulated data, which show the advantage of our method in detection performance over complex scenarios.

Robust fault detection for chemical processes based on dynamic low-rank matrix and optimized LSTM

The data collected by sensors in modern chemical process systems are always contaminated by industrial noise, so robust fault detection is an important technology to ensure process safety. However, the existing robust detection techniques have limitations in extracting dynamic and time series features of chemical process data. In this paper, a robust fault detection method based on dynamic low-rank matrix and optimized LSTM is proposed for dynamic chemical processes under noise background. First, a new low-rank matrix decomposition method, dynamic principal component pursuit (DPCP), is proposed for the dynamic characteristics of process data containing noise. The PCP is improved using the time delay excursion method, i.e., the DPCP is constructed by embedding an augmented dynamic matrix with a time lag factor on the PCP and solved using the alternating direction multiplier method. The purpose is to strip the useless noise from the useful detection information and extract the dynamic low-rank matrix. Second, an optimized LSTM (OPLSTM) is proposed for the time series features of dynamic low-rank matrix. vector factors for feature selection of memory cells in the LSTM are reweighted to balance the time series features of industrial processes and avoid heavy reliance on erroneous fault information. In addition, support vector data description (SVDD) is used to describe the hypersphere with clear boundary, and distance-based statistic is constructed to achieve fault detection with high fault detection rate and low false alarm rate. Finally, to assess the effectiveness of the proposed method, we performed extensive experiments on the Tennessee Eastman Process and the electrolytic aluminum process, and compared them with multiple methods. The experimental results show that the proposed method is effective and robust for chemical processes fault detection in the background of industrial noise.

Structurally incoherent adaptive weighted low-rank matrix decomposition for image classification

Structurally incoherent adaptive weighted low-rank matrix decomposition for image classification

Speech Enhancement Based on Constrained Low-rank Sparse Matrix Decomposition Integrated with Temporal Continuity Regularisation

Speech enhancement in strong noise condition is a challenging problem. Low-rank and sparse matrix decomposition (LSMD) theory has been applied to speech enhancement recently and good performance was obtained. Existing LSMD algorithms consider each frame as an individual observation. However, real-world speeches usually have a temporal structure, and their acoustic characteristics vary slowly as a function of time. In this paper, we propose a temporal continuity constrained low-rank sparse matrix decomposition (TCCLSMD) based speech enhancement method. In this method, speech separation is formulated as a TCCLSMD problem and temporal continuity constraints are imposed in the LSMD process. We develop an alternative optimisation algorithm for noisy spectrogram decomposition. By means of TCCLSMD, the recovery speech spectrogram is more consistent with the structure of the clean speech spectrogram, and it can lead to more stable and reasonable results than the existing LSMD algorithm. Experiments with various types of noises show the proposed algorithm can achieve a better performance than traditional speech enhancement algorithms, in terms of yielding less residual noise and lower speech distortion.

Topology identification under spatially correlated noise

This article addresses the problem of reconstructing the topology of a network of agents interacting via linear dynamics, while being excited by exogenous stochastic sources that are possibly correlated across the agents, from time-series measurements alone. It is shown, under the assumption that the correlations are affine in nature, such network of nodal interactions is equivalent to a network with added agents. The added agents are represented by nodes that are latent, where no corresponding time-series measurements are available; however, here all the exogenous excitements are spatially (that is, across agents) uncorrelated. Generalizing affine correlations, it is shown that, under polynomial correlations, the latent nodes in the expanded networks can be excited by clusters of noise sources, where the clusters are uncorrelated with each other. The clusters can be replaced with a single noise source if the latent nodes are allowed to have non-linear interactions. Finally, using the sparse plus low-rank matrix decomposition of the imaginary part of the inverse power spectral density matrix (IPSDM) of the time-series data, the topology of the network is reconstructed. Under non conservative assumptions, the correlation graph of the noise sources is retrieved.

A robust direction-of-arrival estimation method for impulsive noise environments

The performance of conventional direction-of-arrival (DOA) estimation methods degrades greatly when there are few snapshots and the noise model is mismatched, especially in impulsive noise environments. To solve this problem, this paper proposes a robust DOA estimation method, which is robust to the probability density function of impulsive noise. First, by exploiting the low-rank decomposition of the residual fitting error matrix and the characteristics of impulsive noise, three conditions of a good cost function in impulsive noise environments are proposed, and a unified cost function framework is established. Based on this, a robust cost function is designed to suppress the contribution of samples with impulsive noise to the cost function. Then, a bi-iterative complex fixed-point algorithm (BI-CFPA) is developed for the nonconvex optimization problem of signal subspace estimation based on the proposed robust cost function. Theoretical analyses indicate that the BI-CFPA has excellent numerical stability and low computational complexity. Finally, with the estimated signal subspace, the multiple signal classification technique is employed to obtain DOA estimates. Simulation results show that the proposed DOA estimator performs better than existing methods in three typical impulsive noise environments, especially under fairly strong impulsive noise.

Multi-view Ensemble Clustering via Low-rank and Sparse Decomposition: From Matrix to Tensor

As a significant extension of classical clustering methods, ensemble clustering first generates multiple basic clusterings and then fuses them into one consensus partition by solving a problem concerning graph partition with respect to the co-association matrix. Although the collaborative cluster structure among basic clusterings can be well discovered by ensemble clustering, most advanced ensemble clustering utilizes the self-representation strategy with the constraint of low-rank to explore a shared consensus representation matrix in multiple views. However, they still encounter two challenges: (1) high computational cost caused by both the matrix inversion operation and singular value decomposition of large-scale square matrices; (2) less considerable attention on high-order correlation attributed to the pursue of the two-dimensional pair-wise relationship matrix. In this article, based on low-rank and sparse decomposition from both matrix and tensor perspectives, we propose two novel multi-view ensemble clustering methods, which tangibly decrease computational complexity. Specifically, our first method utilizes low-rank and sparse matrix decomposition to learn one common co-association matrix, while our last method constructs all co-association matrices into one third-order tensor to investigate the high-order correlation among multiple views by low-rank and sparse tensor decomposition. We adopt the alternating direction method of multipliers to solve two convex models by dividing them into several subproblems with closed-form solution. Experimental results on ten real-world datasets prove the effectiveness and efficiency of the proposed two multi-view ensemble clustering methods by comparing them with other advanced ensemble clustering methods.

Accurate Multiobjective Low-Rank and Sparse Model for Hyperspectral Image Denoising Method

Due to the unavoidable influence of sparse and Gaussian noise during the process of data acquisition, the quality of hyperspectral images (HSIs) is degraded and their applications are greatly limited. It is therefore necessary to restore clean HSIs. In the traditional methods, low-rank and sparse matrix decomposition methods are usually applied to restore the pure data matrix from the observed data matrix. However, due to the fact that the optimization of the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${l}_{0}$ </tex-math></inline-formula> -norm for the sparse modeling is a nonconvex and NP-hard problem, convex relaxation and regularization parameters are usually introduced. However, convex relaxation often leads to inaccurate sparse modeling results, and the sensitive regularization parameters can lead to unstable results. Thus, in this article, to address these issues, an accurate multiobjective low-rank and sparse denoising framework is proposed for HSIs to achieve accurate modeling. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${l}_{0}$ </tex-math></inline-formula> -norm is directly modeled as the sparse noise and is optimized by an evolutionary algorithm, and the denoising problem is converted into a multiobjective optimization problem through simultaneously optimizing the low-rank term, the sparse term, and the data fidelity term, without sensitive regularization parameters. However, since the low-rank clean image and sparse noise of the HSI are encoded into a solution, the length of the solution is too long to be optimized. In this article, a subfitness strategy is constructed to achieve effective optimization by comparing the objective function values corresponding to each band for each solution. The experiments undertaken with simulated images in 11 noise cases and four real noisy images confirm the effectiveness of the proposed method.

Low Rank Matrix Factorization Algorithm Based on Multi-Graph Regularization for Detecting Drug-Disease Association.

Detecting potential associations between drugs and diseases plays an indispensable role in drug development, which has also become a research hotspot in recent years. Compared with traditional methods, some computational approaches have the advantages of fast speed and low cost, which greatly accelerate the progress of predicting the drug-disease association. In this study, we propose a novel similarity-based method of low-rank matrix decomposition based on multi-graph regularization. On the basis of low-rank matrix factorization with L2 regularization, the multi-graph regularization constraint is constructed by combining a variety of similarity matrices from drugs and diseases respectively. In the experiments, we analyze the difference in the combination of different similarities, resulting that combining all the similarity information on drug space is unnecessary, and only a part of the similarity information can achieve the desired performance. Then our method is compared with other existing models on three data sets (Fdataset, Cdataset and LRSSLdataset) and have a good advantage in the evaluation measurement of AUPR. Besides, a case study experiment is conducted and showing that the superior ability for predicting the potential disease-related drugs of our model. Finally, we compare our model with some methods on six real world datasets, and our model has a good performance in detecting real world data.

Non-contact heart rate estimation based on singular spectrum component reconstruction using low-rank matrix and autocorrelation.

The remote photoplethysmography (rPPG) based on cameras, a technology for extracting pulse wave from videos, has been proved to be an effective heart rate (HR) monitoring method and has great potential in many fields; such as health monitoring. However, the change of facial color intensity caused by cardiovascular activities is weak. Environmental illumination changes and subjects' facial movements will produce irregular noise in rPPG signals, resulting in distortion of heart rate pulse signals and affecting the accuracy of heart rate measurement. Given the irregular noises such as motion artifacts and illumination changes in rPPG signals, this paper proposed a new method named LA-SSA. It combines low-rank sparse matrix decomposition and autocorrelation function with singular spectrum analysis (SSA). The low-rank sparse matrix decomposition is employed to globally optimize the components of the rPPG signal obtained by SSA, and some irregular noise is removed. Then, the autocorrelation function is used to optimize the global optimization results locally. The periodic components related to the heartbeat signal are selected, and the denoised rPPG signal is obtained by weighted reconstruction with a singular value ratio. The experiment using UBFC-RPPG and PURE database is performed to assess the performance of the method proposed in this paper. The average absolute error was 1.37 bpm, the 95% confidence interval was -7.56 bpm to 6.45 bpm, and the Pearson correlation coefficient was 98%, superior to most existing video-based heart rate extraction methods. Experimental results show that the proposed method can estimate HR effectively.

Iteratively Reweighted Minimax-Concave Penalty Minimization for Accurate Low-rank Plus Sparse Matrix Decomposition.

Low-rank plus sparse matrix decomposition (LSD) is an important problem in computer vision and machine learning. It has been solved using convex relaxations of the matrix rank and l0-pseudo-norm, which are the nuclear norm and l1-norm, respectively. Convex approximations are known to result in biased estimates, to overcome which, nonconvex regularizers such as weighted nuclear-norm minimization and weighted Schatten p-norm minimization have been proposed. However, works employing these regularizers have used heuristic weight-selection strategies. We propose weighted minimax-concave penalty (WMCP) as the nonconvex regularizer and show that it admits an equivalent representation that enables weight adaptation. Similarly, an equivalent representation to the weighted matrix gamma norm (WMGN) enables weight adaptation for the low-rank part. The optimization algorithms are based on the alternating direction method of multipliers technique. We show that the optimization frameworks relying on the two penalties, WMCP and WMGN, coupled with a novel iterative weight update strategy, result in accurate low-rank plus sparse matrix decomposition. The algorithms are also shown to satisfy descent properties and convergence guarantees. On the applications front, we consider the problem of foreground-background separation in video sequences. Simulation experiments and validations on standard datasets, namely, I2R, CDnet 2012, and BMC 2012 show that the proposed techniques outperform the benchmark techniques.

Exemplar-based large-scale low-rank matrix decomposition for collaborative prediction

Recommender system is one of the most effective tools to solve the problem of information overload. As a popular method for recommendation system, regularized singular value decomposition (RSVD) has the advantage of prediction accuracy. However, with the growing of the size of rating matrix A, RSVD suffers from both ‘out of memory’ and high computational cost. To alleviate these disadvantages, we utilize CUR decomposition approach to reduce the memory consumption before RSVD is applied. Additionally, the rating matrix A often is sparse, so we propose a novel column sampling algorithm and sparseness measure to solve the data sparse problem. The main computation cost in the original RSVD is to compute and respectively. While replacing N by CUR or C in RSVD, we can reduce the computational cost from to (where c is the size of data subspace, m and n are the size of the input matrix, t is the number of iterations, is the number of features). Because CUR is explicitly expressed in terms of a small number of actual columns and actual rows of the original data matrix, the result of matrix decomposition has better interpretability. The advantage of what we devised CUR+RSVD and C+RSVD collaborative prediction approaches is that, they not only can deal with the large scale matrix rapidly but also preserve the sparsity of the original matrix, more interestingly, our approaches have higher prediction accuracy. Experimental results on Movielens, Joke data set, etc. show that what we proposed methods can handle the sparsity issue on large-scale low-rank matrix effectively. Compared with RSVD, it can achieve far better prediction accuracy and recommendation result, meanwhile save about 70% training time at the same dataset.