Tensor Robust Principal Component Analysis Research Articles

Weighted Robust Tensor Principal Component Analysis for the Recovery of Complex Corrupted Data in a 5G-Enabled Internet of Things

Technological developments coupled with socioeconomic changes are driving a rapid transformation of the fifth-generation (5G) cellular network landscape. This evolution has led to versatile applications with fast data-transfer capabilities. The integration of 5G with wireless sensor networks (WSNs) has rendered the Internet of Things (IoTs) crucial for measurement and sensing. Although 5G-enabled IoTs are vital, they face challenges in data integrity, such as mixed noise, outliers, and missing values, owing to various transmission issues. Traditional methods such as the tensor robust principal component analysis (TRPCA) have limitations in preserving essential data. This study introduces an enhanced approach, the weighted robust tensor principal component analysis (WRTPCA), combined with weighted tensor completion (WTC). The new method enhances data recovery using tensor singular value decomposition (t-SVD) to separate regular and abnormal data, preserve significant components, and robustly address complex data corruption issues, such as mixed noise, outliers, and missing data, with the globally optimal solution determined through the alternating direction method of multipliers (ADMM). Our study is the first to address complex corruption in multivariate data using the WTRPCA. The proposed approach outperforms current techniques. In all corrupted scenarios, the normalized mean absolute error (NMAE) of the proposed method is typically less than 0.2, demonstrating strong performance even in the most challenging conditions in which other models struggle. This highlights the effectiveness of the proposed approach in real-world 5G-enabled IoTs.

Tensor robust principal component analysis via dual lp quasi-norm sparse constraints

As a pioneering method, tensor robust principal component analysis (TRPCA) can separate an underlying low-rank component and a sparse component from the original data by minimizing a convex objective function composed of tensor nuclear norm and l1-norm. However, it has two limitations. One is that tensor nuclear norm, as a constraint on the low-rank component, treats all singular values uniformly, ignoring the differences among singular values. In essence, this constraint is a sparse constraint that is imposed on singular values of the low-rank component by l1-norm. The other is that l1-norm is used as a constraint for the sparse component. However, l1-norm is a loose constraint, leading to the solutions of TRPCA deviating from the authentic ones. To alleviate these issues, we propose a TPRCA model called p-TRPCA that utilizes the lp-norm to impose sparse constraints on both the singular values and the sparse component simultaneously. The lp quasi-norm (0<p<1) is a tighter constraint than l1-norm, which enhances the low-rankness and sparsity of the proposed model. To solve p-TRPCA, we present an effective algorithm based on the alternating direction method of multipliers and also analyze its convergence. Extensive experiments are performed on data simulation, image recovery, and background modeling. Experimental results show that our p-TRPCA outperforms TRPCA and its variants.

Tensor structure-based ground clutter suppression approaches for pulse doppler radar

ABSTRACT For pulse doppler (PD) radar, two tensor structure-based methods are proposed to improve the clutter suppression performance. One is tensor singular value decomposition (TSVD) and the other is tensor robust principal component analysis (TRPCA). Two algorithms both stack the range-Doppler matrices of multiple coherent processing intervals (CPI) into a tensor structure, which helps make the most of the temporal information across CPIs. TSVD utilizes the strong temporal correlation of ground clutter data to estimate the clutter subspace. Meanwhile, TRPCA exactly separates both low-rank clutter component and sparse target components from the time series range-Doppler tensor structure. Experimental results indicate that compared with conventional matrix-based schemes such as singular value decomposition (SVD) or singular value decomposition (RPCA), the proposed methods have significant advantages in keeping the target echo while suppressing the clutter. In addition, the computational complexity of each algorithm is analysed in detail.

Tensor robust PCA with nonconvex and nonlocal regularization

Tensor robust principal component analysis (TRPCA) is a classical way for low-rank tensor recovery, which minimizes the convex surrogate of tensor rank by shrinking each tensor singular value equally. However, for real-world visual data, large singular values represent more significant information than small singular values. In this paper, we propose a nonconvex TRPCA (N-TRPCA) model based on the tensor adjustable logarithmic norm. Unlike TRPCA, our N-TRPCA can adaptively shrink small singular values more and shrink large singular values less. In addition, TRPCA assumes that the whole data tensor is of low rank. This assumption is hardly satisfied in practice for natural visual data, restricting the capability of TRPCA to recover the edges and texture details from noisy images and videos. To this end, we integrate nonlocal self-similarity into N-TRPCA, and further develop a nonconvex and nonlocal TRPCA (NN-TRPCA) model. Specifically, similar nonlocal patches are grouped as a tensor and then each group tensor is recovered by our N-TRPCA. Since the patches in one group are highly correlated, all group tensors have strong low-rank property, leading to an improvement of recovery performance. Experimental results demonstrate that the proposed NN-TRPCA outperforms existing TRPCA methods in visual data recovery. The demo code is available at https://github.com/qguo2010/NN-TRPCA.

Superposed Atomic Representation for Robust High-Dimensional Data Recovery of Multiple Low-Dimensional Structures

This paper proposes a unified Superposed Atomic Representation (SAR) framework for high-dimensional data recovery with multiple low-dimensional structures. The data can be in various forms ranging from vectors to tensors. The goal of SAR is to recover different components from their sum, where each component has a low-dimensional structure, such as sparsity, low-rankness or be lying a low-dimensional subspace. Examples of SAR include, but not limited to, Robust Sparse Representation (RSR), Robust Principal Component Analysis (RPCA), Tensor RPCA (TRPCA), and Outlier Pursuit (OP). We establish the theoretical guarantee for SAR. To further improve SAR, we also develop a Weighted SAR (WSAR) framework by paying more attention and penalizing less on significant atoms of each component. An effective optimization algorithm is devised for WSAR and the convergence of the algorithm is rigorously proved. By leveraging WSAR as a general platform, several new methods are proposed for high-dimensional data recovery. The experiments on real data demonstrate the superiority of WSAR for various data recovery problems.

A fast correction approach to tensor robust principal component analysis

Tensor robust principal component analysis (TRPCA) is a useful approach for obtaining low-rank data corrupted by noise or outliers. However, existing TRPCA methods face certain challenges when it comes to estimating the tensor rank and the sparsity accurately. The commonly used tensor nuclear norm (TNN) may lead to sub-optimal solutions due to the gap between TNN and the tensor rank. Additionally, the ℓ1-norm is not an ideal estimation of the ℓ0-norm, and solving TNN minimization can be computationally intensive because of the tensor singular value thresholding (t-SVT) scheme. To address these issues, a method called fast correction TNN (FC-TNN) is proposed for TRPCA. In contrast to existing methods, FC-TNN introduces a correction term to bridge the gap between TNN and the tensor rank. Furthermore, a new correction term is employed for the ℓ1-norm to achieve the desired solution. To improve computational efficiency, the Chebyshev polynomial approximation (CPA) technique is presented for computing t-SVT without requiring tensor singular value decomposition (t-SVD). The CPA technique is incorporated into the alternating direction method of multipliers (ADMM) algorithm to solve the proposed model effectively. Theoretical analysis demonstrates that FC-TNN offers a lower error bound compared to TNN under certain conditions. Extensive experiments conducted on various tensor-based datasets illustrate that the proposed method outperforms several state-of-the-art methods.

Tensor recovery based on Bivariate Equivalent Minimax-Concave Penalty

In tensor recovery problem, larger singular values often correspond to the primary information of the image, such as contours, sharp edges and smooth areas. The minimum–maximum concave penalty (MCP) function has been effective in preserving larger singular values and achieving good tensor recovery results. However, in the process of solving this problem, singular values may vary during iterations, and the fixed parameters of the MCP function may not sufficiently preserve all the larger singular values, which may hinder the attainment of optimal results in tensor recovery. To overcome this challenge, we propose the Bivariate Equivalent Minimax-Concave Penalty (BEMCP) theorem, which allows the parameters to adapt to the changes in singular values and more comprehensively preserve the larger singular values. For low-rank tensor completion and tensor robust principal component analysis problems, we propose BEMCP-based models. Finally, experiments with various real-world data demonstrate that the proposed methods outperform state-of-the-art methods.

Tensor Recovery With Weighted Tensor Average Rank.

In this article, a curious phenomenon in the tensor recovery algorithm is considered: can the same recovered results be obtained when the observation tensors in the algorithm are transposed in different ways? If not, it is reasonable to imagine that some information within the data will be lost for the case of observation tensors under certain transpose operators. To solve this problem, a new tensor rank called weighted tensor average rank (WTAR) is proposed to learn the relationship between different resulting tensors by performing a series of transpose operators on an observation tensor. WTAR is applied to three-order tensor robust principal component analysis (TRPCA) to investigate its effectiveness. Meanwhile, to balance the effectiveness and solvability of the resulting model, a generalized model that involves the convex surrogate and a series of nonconvex surrogates are studied, and the corresponding worst case error bounds of the recovered tensor is given. Besides, a generalized tensor singular value thresholding (GTSVT) method and a generalized optimization algorithm based on GTSVT are proposed to solve the generalized model effectively. The experimental results indicate that the proposed method is effective.

Mechanical Fault Diagnosis With Noisy Multisource Signals via Unified Pinball Loss Intuitionistic Fuzzy Support Tensor Machine

In this paper, a challenging and significant intelligent fault diagnosis task is investigated, in which multisource sensor signals with intense noise and outlier disturbances are jointly analyzed. Such a scenario has hardly been considered in industrial research. To this end, we develop a novel tensor-based nonlinear classifier called unified pinball loss intuitionistic fuzzy support tensor machine (UPIFSTM), which can successfully solve the above tasks and improve the performance of fault diagnosis in practical applications. First, the noisy multisource signals are converted into time-frequency images and reconstructed into tensor samples to mine the time domain, frequency domain features and coupled structure information in the spatial domain. Next, we design two nonlinear forms of nonmembership functions in the tensor space, and obtain an intuitionistic fuzzy score for each training sample to enhance the robustness of the model. Subsequently, pinball loss function is introduced to better handle noise sensitivity and resampling instability problems. Note that, we employ the tensor robust principal component analysis method to accurately recover the low-rank tensors corrupted by sparse noise from the original tensors. Finally, two numerical examples are presented to verify the feasibility and validity of the proposed method.

Bayesian Dictionary Learning on Robust Tubal Transformed Tensor Factorization.

The recent study on tensor singular value decomposition (t-SVD) that performs the Fourier transform on the tubes of a third-order tensor has gained promising performance on multidimensional data recovery problems. However, such a fixed transformation, e.g., discrete Fourier transform and discrete cosine transform, lacks being self-adapted to the change of different datasets, and thus, it is not flexible enough to exploit the low-rank and sparse property of the variety of multidimensional datasets. In this article, we consider a tube as an atom of a third-order tensor and construct a data-driven learning dictionary from the observed noisy data along the tubes of the given tensor. Then, a Bayesian dictionary learning (DL) model with tensor tubal transformed factorization, aiming to identify the underlying low-tubal-rank structure of the tensor effectively via the data-adaptive dictionary, is developed to solve the tensor robust principal component analysis (TRPCA) problem. With the defined pagewise tensor operators, a variational Bayesian DL algorithm is established and updates the posterior distributions instantaneously along the third dimension to solve the TPRCA. Extensive experiments on real-world applications, such as color image and hyperspectral image denoising and background/foreground separation problems, demonstrate both effectiveness and efficiency of the proposed approach in terms of various standard metrics.

Video Hashing with Tensor Robust PCA and Histogram of Optical Flow for Copy Detection

Abstract This paper proposes a novel video hashing with tensor robust Principal Component Analysis (PCA) and Histogram of Optical Flow (HOF) for copy detection. In the proposed hashing, a video is divided into some video groups. For each video group, a low-rank secondary frame is constructed from the low-rank component decomposed by applying tensor robust PCA to the video group. Since the low-rank component can well indicate spatial-temporal intrinsic structure of the video group and it is slightly disturbed by digital operations, feature extraction from the low-rank secondary frames is discriminative and stable. Next, spatial features and temporal features are extracted from low-rank secondary frames by Charlier moments and HOF, respectively. Since the Charlier moments are robust to geometric transform and they can efficiently distinguish video frames with different contents, the use of Charlier moments can make robust and discriminative spatial features. As the HOF can measure the distribution of motion information between frames, the temporal features formed by HOFs can provide good discrimination. Hash is ultimately determined by quantizing the spatial and temporal features and concatenating the quantized results. Numerous experiments on open video datasets indicate that the proposed hashing is superior to some hashing baseline schemes in terms of classification and copy detection.

Adaptive weighted robust data recovery with total variation for hyperspectral image

This paper proposes an adaptive weighted robust data recovery method with total variation (TV) regularization for hyperspectral image (HSI). In the proposed method, the HSI recovery problem is modeled as a tensor robust principal component analysis optimization problem, which decomposes the received data into low-rank HSI data, outliers, and noise component, with tensor nuclear norm (TNN), ℓ1 norm, and ℓF norm constrained, respectively. Then, an adaptive weighted strategy is defined to impose on the TNN and outliers to flexibly use the priori information of singular values (SVs) and strengthen the sparsity of outliers, where the weights are adaptively conducted by the elements in SVs and outliers. Specifically, the weighted strategy in TNN retains larger SVs to determine the main information for data recovery, while penalizing smaller SVs to eliminate the influence of interference. And the weighted strategy on outliers enables the proposed method to more effectively measure the sparsity of outliers. Furthermore, TV regularization is introduced to extract the local information for data recovery by a difference operation along different modes, which also can help the proposed method avoid the detail loss caused by the weighted strategy. Experiments confirm that the proposed method significantly outperforms the existing methods.

Tensor recovery based on rank adaptive and non-convex methods

The growing popularity of tensor singular value decomposition (T-SVD) in tensor recovery problems and the N-tubal rank can be applied to higher order tensors. However, this method faces new challenges as it does not fully use the rank prior information of tensors and lacks non-convex relaxation methods. Thus, this paper applies a new non-convex function, called Minimax Logarithmic Concave Penalty (MLCP), based on the N-tubal rank method. Two MLCP models based on N-tubal rank are introduced for solving low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA) problems, along with corresponding solving algorithms. The effectiveness and superiority of these models are demonstrated through experiments on real datasets.

ITRPCA: a new model for computational drug repositioning based on improved tensor robust principal component analysis.

Background: Drug repositioning is considered a promising drug development strategy with the goal of discovering new uses for existing drugs. Compared with the experimental screening for drug discovery, computational drug repositioning offers lower cost and higher efficiency and, hence, has become a hot issue in bioinformatics. However, there are sparse samples, multi-source information, and even some noises, which makes it difficult to accurately identify potential drug-associated indications. Methods: In this article, we propose a new scheme with improved tensor robust principal component analysis (ITRPCA) in multi-source data to predict promising drug-disease associations. First, we use a weighted k-nearest neighbor (WKNN) approach to increase the overall density of the drug-disease association matrix that will assist in prediction. Second, a drug tensor with five frontal slices and a disease tensor with two frontal slices are constructed using multi-similarity matrices and an updated association matrix. The two target tensors naturally integrate multiple sources of data from the drug-side aspect and the disease-side aspect, respectively. Third, ITRPCA is employed to isolate the low-rank tensor and noise information in the tensor. In this step, an additional range constraint is incorporated to ensure that all the predicted entry values of a low-rank tensor are within the specific interval. Finally, we focus on identifying promising drug indications by analyzing drug-disease association pairs derived from the low-rank drug and low-rank disease tensors. Results: We evaluate the effectiveness of the ITRPCA method by comparing it with five prominent existing drug repositioning methods. This evaluation is carried out using 10-fold cross-validation and independent testing experiments. Our numerical results show that ITRPCA not only yields higher prediction accuracy but also exhibits remarkable computational efficiency. Furthermore, case studies demonstrate the practical effectiveness of our method.

Tensor robust principal component analysis based on Bayesian Tucker decomposition for thermographic inspection

Thermographic inspection is considered an effective and promising nondestructive testing tool because of its intuitiveness, wide range and noncontact property. Despite this, the detection of weak defects and the recovery of their shape remain difficult, particularly when the surface being inspected is the opposite of the surface being drilled. This study proposes a new tensor robust principal component analysis method based on Bayesian Tucker decomposition to improve the spatial resolution of thermography. A hierarchical form of a generalized Student-t prior is imposed on the model parameters in the Bayesian framework so as to approximate the low-rank component related to the defect feature. Through variational Bayesian inference, all model parameters are adaptively estimated. Based on two experimental data, it appears that the proposed method is capable of improving the spatial resolution and detection accuracy of the thermographic inspection system.

Robust implementation of foreground extraction and vessel segmentation for X-ray coronary angiography image sequence

The extraction of contrast-filled vessels from X-ray coronary angiography (XCA) image sequence has important clinical significance for intuitively diagnosis and therapy. In this study, the XCA image sequence is regarded as a 3D tensor input, the vessel layer is regarded as a sparse tensor, and the background layer is regarded as a low-rank tensor. Using tensor nuclear norm (TNN) minimization, a novel method for vessel layer extraction based on tensor robust principal component analysis (TRPCA) is proposed. Furthermore, considering the irregular movement of vessels and the low-frequency dynamic disturbance of surrounding irrelevant tissues, the total variation (TV) regularized spatial–temporal constraint is introduced to smooth the foreground layer. Subsequently, for vessel layer images with uneven contrast distribution, a two-stage region growing (TSRG) method is utilized for vessel enhancement and segmentation. A global threshold method is used as the preprocessing to obtain main branches, and the Radon-Like features (RLF) filter is used to enhance and connect broken minor segments. The final binary vessel mask is constructed by combining the two intermediate results. The visibility of TV-TRPCA algorithm for foreground extraction is evaluated on clinical XCA image sequences and third-party dataset, which can effectively improve the performance of commonly used vessel segmentation algorithms. Based on TV-TRPCA, the accuracy of TSRG algorithm for vessel segmentation is further evaluated. Both qualitative and quantitative results validate the superiority of the proposed method over existing state-of-the-art approaches.

Guaranteed Tensor Recovery Fused Low-rankness and Smoothness.

Tensor recovery is a fundamental problem in tensor research field. It generally requires to explore intrinsic prior structures underlying tensor data, and formulate them as certain forms of regularization terms for guiding a sound estimate of the restored tensor. Recent researches have made significant progress by adopting two insightful tensor priors, i.e., global low-rankness (L) and local smoothness (S), which are always encoded as a sum of two separate regularizers into recovery models. However, unlike the primary theoretical developments on low-rank tensor recovery, these joint "L+S" models have no theoretical exact-recovery guarantees yet, making the methods lack reliability in real practice. To this crucial issue, in this work, we build a unique regularizer termed as tensor correlated total variation (t-CTV), which essentially encodes both L and S priors of a tensor simultaneously. Especially, by equipping t-CTV into the recovery models, we can rigorously prove the exact recovery guarantees for two typical tensor recovery tasks, i.e., tensor completion and tensor robust principal component analysis. To the best of our knowledge, this should be the first exact-recovery results among all related "L+S" methods for tensor recovery. We further propose ADMM algorithms with fine convergence to solve the proposed models. Significant recovery accuracy improvements are observed in extensive experiments. Typically, our method achieves a workable performance when the missing rate is extremely large, e.g., 99.5%, for the color image inpainting task, while all its peers totally fail in such a challenging case. Code is released at https://github.com/wanghailin97.

Tensor Robust Principal Component Analysis With Side Information: Models and Applications

As a domain-dependent prior knowledge, side information has been introduced into Robust Principal Component Analysis (RPCA) to alleviate its degenerate or suboptimal performance in some real applications. It has recently realized that the natural structural information can be better retained if the observed data is kept in the original tensor form rather than matricizing it or other order reduction means. Hence, studies on RPCA of tensor version have attracted more and more attentions. To share the merits from both direct tensor modeling and side information, we propose three models to deal with the problem of Tensor RPCA with side information based on tensor Singular Value Decomposition (t-SVD). To solve these models, we develop an efficient algorithm with convergence guarantee using the well-known alternating direction method of multiplier. Extensive experimental studies on both synthetic and real-world tensor data have been carried out to demonstrate the superiority of the proposed models over several other state-of-the-arts. Our code is released at https://github.com/zsj9509/TPCPSF.

Doubly robust logistic regression for image classification

The growing prevalence of image data in engineering and medical applications motivates the need for classification performance that are robust against outliers. To facilitate efficient and data-driven classification and recovery method, in this paper, we propose a novel supervised learning strategy based on the robust principal component analysis for third-order tensor and minimum distance criterion L2E for logistic regression, which is named as doubly robust logistic regression. Our work applies the ADMM method to obtain updating algorithm, and its global convergence is established even though L2E loss function is non-convex. We also extend the estimation procedure to the case of incomplete observation in input matrix. The numerical experiments demonstrate the advantages of combining the logistic L2E with tensor robust principal component analysis which can not only increase the accuracy of classification but also improve the recovery accuracy of noisy image. Three real data analysis are further used to examine the outperformance of our proposed method over the stat-of-art.

Fast and provable tensor robust principal component analysis via scaled gradient descent

Abstract An increasing number of data science and machine learning problems rely on computation with tensors, which better capture the multi-way relationships and interactions of data than matrices. When tapping into this critical advantage, a key challenge is to develop computationally efficient and provably correct algorithms for extracting useful information from tensor data that are simultaneously robust to corruptions and ill-conditioning. This paper tackles tensor robust principal component analysis (RPCA), which aims to recover a low-rank tensor from its observations contaminated by sparse corruptions, under the Tucker decomposition. To minimize the computation and memory footprints, we propose to directly recover the low-dimensional tensor factors—starting from a tailored spectral initialization—via scaled gradient descent (ScaledGD), coupled with an iteration-varying thresholding operation to adaptively remove the impact of corruptions. Theoretically, we establish that the proposed algorithm converges linearly to the true low-rank tensor at a constant rate that is independent with its condition number, as long as the level of corruptions is not too large. Empirically, we demonstrate that the proposed algorithm achieves better and more scalable performance than state-of-the-art tensor RPCA algorithms through synthetic experiments and real-world applications.