Low-rank Component Research Articles

Robust Low Transformed Multi-Rank Tensor Completion With Deep Prior Regularization for Multi-Dimensional Image Recovery

In this paper, we study the robust tensor completion problem in three-dimensional image data, where only partial entries are available and the observed tensor is corrupted by Gaussian noise and sparse noise simultaneously. Compared with the existing tensor nuclear norm minimization for the low-rank component, we propose to use the transformed tensor nuclear norm to explore the global low-rankness of the underlying tensor. Moreover, the plug-and-play (PnP) deep prior denoiser is incorporated to preserve the local details of multi-dimensional images. Besides, the tensor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> norm is utilized to characterize the sparseness of the sparse noise. A symmetric Gauss-Seidel based alternating direction method of multipliers is designed to solve the resulting model under the PnP framework with deep prior denoiser. Extensive numerical experiments on hyperspectral and multispectral images, videos, color images, and magnetic resonance image datasets are conducted to demonstrate the superior performance of the proposed model in comparison with several state-of-the-art models.

Tensor recovery based on rank adaptive and non-convex methods

The growing popularity of tensor singular value decomposition (T-SVD) in tensor recovery problems and the N-tubal rank can be applied to higher order tensors. However, this method faces new challenges as it does not fully use the rank prior information of tensors and lacks non-convex relaxation methods. Thus, this paper applies a new non-convex function, called Minimax Logarithmic Concave Penalty (MLCP), based on the N-tubal rank method. Two MLCP models based on N-tubal rank are introduced for solving low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA) problems, along with corresponding solving algorithms. The effectiveness and superiority of these models are demonstrated through experiments on real datasets.

Tensor robust principal component analysis based on Bayesian Tucker decomposition for thermographic inspection

Thermographic inspection is considered an effective and promising nondestructive testing tool because of its intuitiveness, wide range and noncontact property. Despite this, the detection of weak defects and the recovery of their shape remain difficult, particularly when the surface being inspected is the opposite of the surface being drilled. This study proposes a new tensor robust principal component analysis method based on Bayesian Tucker decomposition to improve the spatial resolution of thermography. A hierarchical form of a generalized Student-t prior is imposed on the model parameters in the Bayesian framework so as to approximate the low-rank component related to the defect feature. Through variational Bayesian inference, all model parameters are adaptively estimated. Based on two experimental data, it appears that the proposed method is capable of improving the spatial resolution and detection accuracy of the thermographic inspection system.

Joint low-rank and sparse decomposition for infrared and visible image sequence fusion

Image fusion for infrared and visible images is intended for integrating the daylight information and the hidden target information into a single synthetic image that is more suitable for high-level vision tasks, such as image segmentation, target recognition. Existing techniques that employ the same representation to extract the same characteristics of various source images may not be suitable for infrared and visible image sequence fusion due to ignoring the correlation information between the infrared and visible image sequences. In order to overcome this problem, this paper presents a new fusion method of infrared and visible image sequences via joint low-rank and sparse decomposition (JLRSD) model. First, infrared and visible image sequences are respectively decomposed via joint low-rank and sparse representation to obtain the common low-rank components, specific low-rank components and sparse low-rank components. Second, a novel weighted average technique with phase congruency and guided filtering is put forward to fuse the specific low-rank components. Then, the average fusion rule is adopted to fuse the sparse low-rank components, which aims to preserve more target information. Finally, the fusion results are obtained by summing the common low-rank component, the fused specific low-rank component, and the fused sparse low-rank component. As compared with the several existing fusion approaches, experimental results demonstrate that our method can provide superior achievement in terms of subjective evaluation and objective assessment.

Photon-counting spectral CT reconstruction with sparse and double low-rank components fusion

ObjectivePhoton-counting CT (computed tomography) has aroused more attention. The relatively high dose of X-ray increases concerns about radiation, dose reduction can mitigate radiation risk, however, projection data within energy channel suffers from strong noise, leading to degraded results. Besides, there is high correlation among channels since the multi-channel data is captured from the same object, which should be utilized to improve reconstruction quality. To suppress noise while preserving details in reconstruction and improve material decomposition accuracy, we proposed a novel optimization-based spectral CT reconstruction method with a fusion framework. MethodsTwo results were obtained by implementing two iterative reconstructions with different constraints and can be treated as complementary components. The first constraint is sparsity, the second constraint is double low-rank, moreover, a heuristic strategy is designed to solve double low-rank constrained optimization. We developed a guided filter based fusion framework to fuse sparse component and double low-rank component. ResultsExperimental results showed that our method achieved satisfying visualization, higher values of PSNR (peak signal-to-noise ratio) and SSIM (structure similarity), moreover, lower material decomposition error in this work.Conclusion.Our method can well perform spectral CT reconstruction under low-signal-to-ratio situation and it has decent performance in noise suppression, edge preserving and material discrimination.

Tensor completion via convolutional sparse coding with small samples-based training

Tensor data often suffer from missing value problems due to the complex high-dimensional structure while acquiring them. To complete the missing information, lots of Low-Rank Tensor Completion (LRTC) methods have been proposed, most of which depend on the low-rank property of tensor data. In this way, the low-rank component of the original data could be recovered roughly. However, the shortcoming is that the detailed information can not be fully restored, no matter the Sum of the Nuclear Norm (SNN) nor the Tensor Nuclear Norm (TNN) based methods. On the contrary, in the field of signal processing, Convolutional Sparse Coding (CSC) can provide a good representation of the high-frequency component of the image, which is generally associated with the detail component of the data. To this end, we propose two novel methods, LRTC-CSC-I and LRTC-CSC-II, which adopt CSC as a supplementary regularization for LRTC to capture the high-frequency components. Therefore, the LRTC-CSC methods can not only solve the missing value problem but also recover the details. Moreover, the regularizer CSC can be trained with small samples due to the sparsity characteristic. Extensive experiments show the effectiveness of LRTC-CSC methods, and quantitative evaluation indicates that the performance of our models are superior to state-of-the-art methods.

NUCLEAR NORM REGULARIZED QUANTILE REGRESSION WITH INTERACTIVE FIXED EFFECTS

This paper studies large N and large T conditional quantile panel data models with interactive fixed effects. We propose a nuclear norm penalized estimator of the coefficients on the covariates and the low-rank matrix formed by the interactive fixed effects. The estimator solves a convex minimization problem, not requiring pre-estimation of the (number of) interactive fixed effects. It also allows the number of covariates to grow slowly with N and T. We derive an error bound on the estimator that holds uniformly in the quantile level. The order of the bound implies uniform consistency of the estimator and is nearly optimal for the low-rank component. Given the error bound, we also propose a consistent estimator of the number of interactive fixed effects at any quantile level. We demonstrate the performance of the estimator via Monte Carlo simulations.

Fast, Efficient, and Viable Compressed Sensing, Low-Rank, and Robust Principle Component Analysis Algorithms for Radar Signal Processing

Modern radar signal processing techniques make strong use of compressed sensing, affine rank minimization, and robust principle component analysis. The corresponding reconstruction algorithms should fulfill the following desired properties: complex valued, viable in the sense of not requiring parameters that are unknown in practice, fast convergence, low computational complexity, and high reconstruction performance. Although a plethora of reconstruction algorithms are available in the literature, these generally do not meet all of the aforementioned desired properties together. In this paper, a set of algorithms fulfilling these conditions is presented. The desired requirements are met by a combination of turbo-message-passing algorithms and smoothed ℓ0-refinements. Their performance is evaluated by use of extensive numerical simulations and compared with popular conventional algorithms.

Optimal Portfolio Using Factor Graphical Lasso

Abstract Graphical models are a powerful tool to estimate a high-dimensional inverse covariance (precision) matrix, which has been applied for a portfolio allocation problem. The assumption made by these models is a sparsity of the precision matrix. However, when stock returns are driven by common factors, such assumption does not hold. We address this limitation and develop a framework, Factor Graphical Lasso (FGL), which integrates graphical models with the factor structure in the context of portfolio allocation by decomposing a precision matrix into low-rank and sparse components. Our theoretical results and simulations show that FGL consistently estimates the portfolio weights and risk exposure and also that FGL is robust to heavy-tailed distributions which makes our method suitable for financial applications. FGL-based portfolios are shown to exhibit superior performance over several prominent competitors including equal-weighted and index portfolios in the empirical application for the S&amp;P500 constituents.

Low-rank kernel regression with preserved locality for multi-class analysis

Kernel ridge regression (KRR) is a kind of efficient supervised algorithm for multi-class analysis. However, limited by the implicit kernel space, current KRR methods have weak abilities to deal with redundant features and hidden local structures. Thus, they may get indifferent results when applied to analyze the data with complicated components. To overcome this weakness and obtain better multi-class regression performance, we propose a new method named low-rank kernel regression with preserved locality (RLRKRR). In this method, data are mapped into an explicit feature space by using the random Fourier feature technique to discover the non-linear relationship between data samples. In addition, during the training of the regression coefficient matrix, the low-rank components of this explicit feature space are simultaneously extracted for reducing the effect of the redundancy. Moreover, the graph regularization is performed on the extracted low-rank components to preserve local structures. Furthermore, the l2,p norm is imposed on the regression error term for relieving the impact of outliers. Based on these strategies, RLRKRR is capable to achieve rewarding results in complicated multi-class data analysis. In the comprehensive experiments conducted on various types of datasets, RLRKRR outperforms several state-of-the-art regression methods in terms of classification accuracy (CA).

Low-Tubal-Rank tensor recovery with multilayer subspace prior learning

Currently, low-rank tensor recovery employing the subspace prior information is an emerging topic, which has attracted considerable attention. However, existing studies cannot flexibly and fully utilize the accessible subspace prior information, thereby leading to suboptimal restored performance. Aiming at addressing this issue, based on the tensor singular value decomposition (t-SVD), this article presents a novel strategy that integrates more than two layers of subspace knowledge about columns and rows of target tensor into one unified recovery framework. Specially, we first design a multilayer subspace prior learning scheme, and then apply it to two common low-rank tensor recovery problems, i.e., tensor completion and tensor robust component principal analysis. Crucially, we prove that our approach can achieve exact recovery of tensors under a significantly weaker incoherence assumption than the analogous conditions previously proposed. Furthermore, two efficient algorithms with convergence guarantees based on alternating direction method of multipliers (ADMM) are proposed to solve the corresponding models. The experimental results on synthetic and real tensor data show that the proposed algorithms outperform other state-of-the-art algorithms in terms of both qualitative and quantitative metrics.

Clutter Removal Method for GPR Based on Low-Rank and Sparse Decomposition With Total Variation Regularization

The performance of ground penetrating radar (GPR) target detection is seriously affected by the clutter. In this letter, an effective GPR clutter removal method is proposed based on low-rank and sparse decomposition with total variation regularization (LRSD-TVR). In the proposed method, a total variation (TV) regularization of sparse matrix is introduced to further remove the remaining clutter and to obtain a clearer target image. An iterative approach based on alternating direction method of multipliers (ADMM) is developed to solve the optimization problem of LRSD-TVR. In each iteration, the low rank component which corresponds to the clutter is computed by singular value decomposition (SVD) thresholding. Besides, the sparse component corresponding to the target is obtained by solving the sub-optimization problem reformulated in terms of TV component. The effectiveness of proposed method is verified by both numerical simulations and field experiments.

New Robust Sparse Convolutional Coding Inversion Algorithm for Ground Penetrating Radar Images

In this paper, we propose two algorithms to enhance the interpretability of the hyperbola in B-scans obtained with a Ground Penetrating Radar (GPR). These hyperbolas are the responses of buried objects or cavities. To correctly detect and classify them, a denoising is typically necessary for GPR images as the signal-to-noise ratio is low, and the various interfaces naturally present in the earth have a strong response. Both algorithms are based on a sparse convolutional coding model plus a low rank component. It is solved through an Alternating Direction Method of Multipliers (ADMM) framework. In order to take into account the presence of outliers and the artifacts caused by the acquisition, the second algorithm is based on the Huber norm instead of the classic <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm. These algorithms are tested on a real dataset labeled by geophysicists. The results show the denoising efficiency of this approach, and in particular the robustness of the second algorithm.

Double Auto-weighted Tensor Robust Principal Component Analysis.

Tensor Robust Principal Component Analysis (TRPCA), which aims to recover the low-rank and sparse components from their sum, has drawn intensive interest in recent years. Most existing TRPCA methods adopt the tensor nuclear norm (TNN) and the tensor ℓ1 norm as the regularization terms for the low-rank and sparse components, respectively. However, TNN treats each singular value of the low-rank tensor L equally and the tensor ℓ1 norm shrinks each entry of the sparse tensor S with the same strength. It has been shown that larger singular values generally correspond to prominent information of the data and should be less penalized. The same goes for large entries in S in terms of absolute values. In this paper, we propose a Double Auto-weighted TRPCA (DATRPCA) method. Instead of using predefined and manually set weights merely for the low-rank tensor as previous works, DATRPCA automatically and adaptively assigns smaller weights and applies lighter penalization to significant singular values of the low-rank tensor and large entries of the sparse tensor simultaneously. We have further developed an efficient algorithm to implement DATRPCA based on the Alternating Direction Method of Multipliers (ADMM) framework. In addition, we have also established the convergence analysis of the proposed algorithm. The results on both synthetic and real-world data demonstrate the effectiveness of DATRPCA for low-rank tensor recovery, color image recovery and background modelling.

Robust Tensor Decomposition Based Background/Foreground Separation in Noisy Videos and Its Applications in Additive Manufacturing

Background/foreground separation is one of the most fundamental tasks in computer vision, especially for video data. Robust PCA (RPCA) and its tensor extension, namely, Robust Tensor PCA (RTPCA), provide an effective framework for background/foreground separation by decomposing the data into low-rank and sparse components, which contain the background and the foreground (moving objects), respectively. However, in real-world applications, the video data is contaminated with noise. For example, in metal additive manufacturing (AM), the processed X-ray video to study melt pool dynamics is very noisy. RPCA and RTPCA are not able to separate the background, foreground, and noise simultaneously. As a result, the noise will contaminate the background or the foreground or both. There is a need to remove the noise from the background and foreground. To achieve the three components decomposition, a smooth sparse Robust Tensor Decomposition (SS-RTD) model is proposed to decompose the data into static background, smooth foreground, and noise, respectively. Specifically, the static background is modeled by the low-rank tucker decomposition, the smooth foreground (moving objects) is modeled by the spatio-temporal continuity, which is enforced by the total variation regularization, and the noise is modeled by the sparsity, which is enforced by the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> norm. An efficient algorithm based on alternating direction method of multipliers (ADMM) is implemented to solve the proposed model. Extensive experiments on both simulated and real data demonstrate that the proposed method significantly outperforms the state-of-the-art approaches for background/foreground separation in noisy cases. Note to Practitioners—This work is motivated by melt pool detection in metal additive manufacturing where the processed X-ray video from the monitoring system is very noisy. The objective is to recover the background with porosity defects and the foreground with melt pool in the presence of noise. Existing methods fail to separate the noise from the background and foreground since RPCA and RTPCA have only two components, which cannot explain the three components in the data. This paper puts forward a smooth sparse Robust Tensor Decomposition by decomposing the tensor data into low-rank, smooth, and sparse components, respectively. It is a highly effective method for background/foreground separation in noisy case. In the case studies on simulated video and X-ray data, the proposed method can handle non-additive noise, and even the case of high noise-ratio. In the proposed algorithm, there is only one tuning parameter <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\lambda $ </tex-math></inline-formula> . Based on the case studies, our method achieves satisfying performance by taking any <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\lambda \in [{0.2,1}]$ </tex-math></inline-formula> with anisotropic total variation regularization. With this observation, practitioners can apply the proposed method without extensive parameter tuning work. Furthermore, the proposed method is also applicable to other popular industrial applications. Practitioners can use the proposed SS-RTD for degradation processes monitoring, where the degradation image contains the static background, anomaly, and random disturbance, respectively.

Tensor Recovery Based on a Novel Non-Convex Function Minimax Logarithmic Concave Penalty Function.

Non-convex relaxation methods have been widely used in tensor recovery problems, compared with convex relaxation methods, and can achieve better recovery results. In this paper, a new non-convex function, Minimax Logarithmic Concave Penalty (MLCP) function, is proposed, and some of its intrinsic properties are analyzed, among which it is interesting to find that the Logarithmic function is an upper bound of the MLCP function. The proposed function is generalized to tensor cases, yielding tensor MLCP and weighted tensor Lγ -norm. Consider that its explicit solution cannot be obtained when applying it directly to the tensor recovery problem. Therefore, the corresponding equivalence theorems to solve the such problem are given, namely, tensor equivalent MLCP theorem and equivalent weighted tensor Lγ -norm theorem. In addition, we propose two EMLCP-based models for classic tensor recovery problems, namely low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA), and design proximal alternate linearization minimization (PALM) algorithms to solve them individually. Furthermore, based on the Kurdyka-Åasiwicz property, it is proved that the solution sequence of the proposed algorithm has a finite length and converges to the critical point globally. Finally, extensive experiments show that the proposed algorithm achieves good results, and it is confirmed that the MLCP function is indeed better than the Logarithmic function in the minimization problem, which is consistent with the analysis of theoretical properties.

Matrix Factorization With Framelet and Saliency Priors for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection aims to separate sparse anomalies from low-rank background components. A variety of detectors have been proposed to identify anomalies, but most of them tend to emphasize characterizing backgrounds with multiple types of prior knowledge and limited information on anomaly components. To tackle these issues, this paper simultaneously focuses on two components and proposes a matrix factorization method with framelet and saliency priors to handle the anomaly detection problem. We first employ a framelet to characterize nonnegative background representation coefficients, as they can jointly maintain sparsity and piecewise smoothness after framelet decomposition. We then exploit saliency prior knowledge to measure each pixel’s potential to be an anomaly. Finally, we incorporate the pure pixel index with Reed-Xiaoli’s method to possess representative dictionary atoms. We solve the optimization problem by using a block successive upper-bound minimization framework with guaranteed convergence. Experiments conducted on benchmark hyperspectral datasets demonstrate that the proposed method outperforms some state-of-the-art anomaly detection methods.

Anomaly Detection for Hyperspectral Imagery via Tensor Low-Rank Approximation With Multiple Subspace Learning

Hyperspectral anomaly detection (HAD) is regarded as an indispensable, pivotal technology in remote sensing and earth science domains. Nevertheless, most existing detection approaches for anomaly targets flatten 3-D hyperspectral images (HSIs) with spatial and spectral information into 2-D spectral vector data, which virtually breaks up the internal spatial structure in HSIs and degenerates the detection performance. To this end, we directly consider the HSI data cube as a 3-D tensor and develop a novel tensor low-rank approximation (TLRA) detection algorithm to separate the sparse anomalous component from the background with low-rank characteristics. Then, in light of the multi-subspace structure in heterogeneous backgrounds, we utilize multiple subspace learning (MSL) theory to encode the background tensor with a coefficient tensor and corresponding dictionary tensor. In addition, considering that different singular values indicate different information quantities and should be penalized to different extents, we introduce a tighter tensor rank surrogate named the ϵ-shrinkage tensor nuclear norm (ϵ-TNN) to recover the low-rank component more accurately. Meanwhile, concerning the sparse anomaly target, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> constraint is incorporated to represent the group sparsity of the abnormal component. Finally, an effective iterative optimization algorithm based on the alternating direction method of multipliers (ADMM) is devised to solve the proposed TLRA-MSL model. We conduct extensive experiments on six hyperspectral datasets to prove the effectiveness and robustness of our method. The experimental results illustrate that better detection performance is obtained using the proposed model compared with other state-of-the-art algorithms.

A Single-Image Noise Estimation Algorithm Based on Pixel-Level Low-Rank Low-Texture Patch and Principal Component Analysis.

Noise level is an important parameter for image denoising in many image-processing applications. We propose a noise estimation algorithm based on pixel-level low-rank, low-texture subblocks and principal component analysis for white Gaussian noise. First, an adaptive clustering algorithm, based on a dichotomy merge, adaptive pixel-level low-rank matrix construction method and a gradient covariance low-texture subblock selection method, is proposed to construct a pixel-level low-rank, low-texture subblock matrix. The adaptive clustering algorithm can improve the low-rank property of the constructed matrix and reduce the content of the image information in the eigenvalues of the matrix. Then, an eigenvalue selection method is proposed to eliminate matrix eigenvalues representing the image to avoid an inaccurate estimation of the noise level caused by using the minimum eigenvalue. The experimental results show that, compared with existing state-of-the-art methods, our proposed algorithm has, in most cases, the highest accuracy and robustness of noise level estimation for various scenarios with different noise levels, especially when the noise is high.

Topology Learning of Linear Dynamical Systems With Latent Nodes Using Matrix Decomposition

In this article, we present a novel approach to reconstruct the topology of networked linear dynamical systems with latent nodes. The network is allowed to have directed loops and bi-directed edges. The main approach relies on the unique decomposition of the inverse of power spectral density matrix (IPSDM) obtained from observed nodes as a sum of sparse and low-rank matrices. We provide conditions and methods for decomposing the IPSDM into sparse and low-rank components. The sparse component yields moral graph (MG) associated with the observed nodes, and the low-rank component retrieves parents, children and spouses (the Markov Blanket) of the hidden nodes. The article provides necessary and sufficient conditions for the unique decomposition of a given skew symmetric matrix into sum of a sparse skew symmetric and a low-rank skew symmetric matrices. For a large class of systems, the unique decomposition of imaginary part of the IPSDM of observed nodes, a skew symmetric matrix, into the sparse and the low-rank components is sufficient to identify the MG of the observed nodes as well as the Markov Blanket of latent nodes. For a large class of systems, all spurious links in the MG formed by the observed nodes can be identified. Assuming conditions on hidden nodes required for identifiability, links between hidden and observed nodes can be reconstructed, thus retrieving the exact topology of the network from the IPSDM. Moreover, for finite data, we provide bounds on entry-wise distance between the true and the estimated IPSDMs.