Low-rank Tensor Representation Research Articles

A new nonconvex multi-view subspace clustering via learning a clean low-rank representation tensor

Abstract Recently, low-rank tensor representation has achieved impressive results for multi-view subspace clustering (MSC). The typical MSC methods utilize the tensor nuclear norm as a convex surrogate of the tensor multi-rank to obtain a low-rank representation, which exhibits limited robustness when dealing with noisy and complex data scenarios. In this paper, we introduce an innovative clean low-rank tensor representation approach that combines the idea of tensor robust principal component analysis (TRPCA) with a new nonconvex tensor multi-rank approximation regularization. This integration enhances the robustness of the low-rank representation, resulting in improved performance. Furthermore, to better capture the local geometric features, we employ a higher-order manifold regularization term. To effectively address our new model, we develop an iterative algorithm that has been proven to converge to the desired Karush-Kuhn-Tucker (KKT) point. The numerical experiments on widely used datasets serve to demonstrate the efficacy and effectiveness of our new method.

Joint local smoothness and low-rank tensor representation for robust multi-view clustering

Low-rank tensor representation (LRTR) has become a significant method for achieving improved multi-view clustering (MVC) performance. Generally, most LRTR methods impose a tensor low-rank constraint (TLRC) on a tensor, which is spliced by the representation matrix of each view, to explore the low-rank prior hidden in these representation matrices. However, two problems remain unsolved. On the one hand, these representation matrices still contain noise since the raw data usually possess noise; on the other hand, and more importantly, is there any other prior information that can be explored among these representation matrices? Since samples located in the same cluster are similar, the coefficients of their representation matrices should exhibit similar. That is, these representation matrices should be local smoothness (LS), which is also verified by our numerical tests. In addition, the use of the LS prior helps to denoise the noise contain in the data. Then, in this paper, we propose a joint LS and LRTR for robust MVC method (LS-LRTR), which can simultaneously exploit the low-rank and LS priors. Specifically, we utilize a TLRC to explore the low-rank prior in the representation matrices. Subsequently, to mine the LS prior and further reduce the influence of noise, we introduce the Total Variation (TV) norm to the constraint representation matrices. Then, we fuse the TLRC and TV norm into a unified framework. Additionally, we apply an Augmented Lagrange Multiplier to solve the optimization problem of LS-LRTR. Experiments conducted on several datasets indicate that LS-LRTR outperforms the state-of-the-art clustering methods.

Tensorized Incomplete Multi-view Kernel Subspace Clustering

Recently considerable advances have been achieved in the incomplete multi-view clustering (IMC) research. However, the current IMC works are often faced with three challenging issues. First, they mostly lack the ability to recover the nonlinear subspace structures in the multiple kernel spaces. Second, they usually neglect the high-order relationship in multiple representations. Third, they often have two or even more hyper-parameters and may not be practical for some real-world applications. To tackle these issues, we present a Tensorized Incomplete Multi-view Kernel Subspace Clustering (TIMKSC) approach. Specifically, by incorporating the kernel learning technique into an incomplete subspace clustering framework, our approach can robustly explore the latent subspace structure hidden in multiple views. Furthermore, we impute the incomplete kernel matrices and learn the low-rank tensor representations in a mutual enhancement manner. Notably, our approach can discover the underlying relationship among the observed and missing samples while capturing the high-order correlation to assist subspace clustering. To solve the proposed optimization model, we design a three-step algorithm to efficiently minimize the unified objective function, which only involves one hyper-parameter that requires tuning. Experiments on various benchmark datasets demonstrate the superiority of our approach. The source code and datasets are available at: https://www.researchgate.net/publication/381828300_TIMKSC_20240629.

CoNoT: Coupled Nonlinear Transform-Based Low-Rank Tensor Representation for Multidimensional Image Completion.

Recently, the transform-based tensor nuclear norm (TNN) methods have shown promising performance and drawn increasing attention in tensor completion (TC) problems. The main idea of these methods is to exploit the low-rank structure of frontal slices of the tensor under the transform. However, the transforms in TNN methods usually treat all modes equally and do not consider the different traits of different modes (i.e., spatial and spectral/temporal modes). To address this problem, we suggest a new low-rank tensor representation based on the coupled nonlinear transform (called CoNoT) for a better low-rank approximation. Concretely, spatial and spectral/temporal transforms in the CoNoT, respectively, exploit the different traits of different modes and are coupled together to boost the implicit low-rank structure. Here, we use the convolutional neural network (CNN) as the CoNoT, which can be learned solely from an observed multidimensional image in an unsupervised manner. Based on this low-rank tensor representation, we build a new multidimensional image completion model. Moreover, we also propose an enhanced version (called Ms-CoNoT) to further exploit the spatial multiscale nature of real-world data. Extensive experiments on real-world data substantiate the superiority of the proposed models against many state-of-the-art methods both qualitatively and quantitatively.

Robust Corrupted Data Recovery and Clustering via Generalized Transformed Tensor Low-Rank Representation.

Tensor analysis has received widespread attention in high-dimensional data learning. Unfortunately, the tensor data are often accompanied by arbitrary signal corruptions, including missing entries and sparse noise. How to recover the characteristics of the corrupted tensor data and make it compatible with the downstream clustering task remains a challenging problem. In this article, we study a generalized transformed tensor low-rank representation (TTLRR) model for simultaneously recovering and clustering the corrupted tensor data. The core idea is to find the latent low-rank tensor structure from the corrupted measurements using the transformed tensor singular value decomposition (SVD). Theoretically, we prove that TTLRR can recover the clean tensor data with a high probability guarantee under mild conditions. Furthermore, by using the transform adaptively learning from the data itself, the proposed TTLRR model can approximately represent and exploit the intrinsic subspace and seek out the cluster structure of the tensor data precisely. An effective algorithm is designed to solve the proposed model under the alternating direction method of multipliers (ADMMs) algorithm framework. The effectiveness and superiority of the proposed method against the compared methods are showcased over different tasks, including video/face data recovery and face/object/scene data clustering.

Tensor low-rank representation combined with consistency and diversity exploration

Tensor low-rank representation combined with consistency and diversity exploration

Partial Tubal Nuclear Norm-Regularized Multiview Subspace Learning.

In this article, a unified multiview subspace learning model, called partial tubal nuclear norm-regularized multiview subspace learning (PTN2MSL), was proposed for unsupervised multiview subspace clustering (MVSC), semisupervised MVSC, and multiview dimension reduction. Unlike most of the existing methods which treat the above three related tasks independently, PTN2MSL integrates the projection learning and the low-rank tensor representation to promote each other and mine their underlying correlations. Moreover, instead of minimizing the tensor nuclear norm which treats all singular values equally and neglects their differences, PTN2MSL develops the partial tubal nuclear norm (PTNN) as a better alternative solution by minimizing the partial sum of tubal singular values. The PTN2MSL method was applied to the above three multiview subspace learning tasks. It demonstrated that these tasks organically benefited from each other and PTN2MSL has achieved better performance in comparison to state-of-the-art methods.

Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery.

Since higher-order tensors are naturally suitable for representing multi-dimensional data in real-world, e.g., color images and videos, low-rank tensor representation has become one of the emerging areas in machine learning and computer vision. However, classical low-rank tensor representations can solely represent multi-dimensional discrete data on meshgrid, which hinders their potential applicability in many scenarios beyond meshgrid. To break this barrier, we propose a low-rank tensor function representation (LRTFR) parameterized by multilayer perceptrons (MLPs), which can continuously represent data beyond meshgrid with powerful representation abilities. Specifically, the suggested tensor function, which maps an arbitrary coordinate to the corresponding value, can continuously represent data in an infinite real space. Parallel to discrete tensors, we develop two fundamental concepts for tensor functions, i.e., the tensor function rank and low-rank tensor function factorization, and utilize MLPs to paramterize factor functions of the tensor function factorization. We theoretically justify that both low-rank and smooth regularizations are harmoniously unified in LRTFR, which leads to high effectiveness and efficiency for data continuous representation. Extensive multi-dimensional data recovery applications arising from image processing (image inpainting and denoising), machine learning (hyperparameter optimization), and computer graphics (point cloud upsampling) substantiate the superiority and versatility of our method as compared with state-of-the-art methods. Especially, the experiments beyond the original meshgrid resolution (hyperparameter optimization) or even beyond meshgrid (point cloud upsampling) validate the favorable performances of our method for continuous representation.

Error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding

Tensor based subspace clustering, a method aimed at partitioning multi-view data into distinct clusters through the tensor low-rank representation, has gained widespread popularity. Despite the effectiveness of tensor singular value decomposition (t-SVD) based nuclear norm in extracting high-dimensional information from multiple views, certain limitations persist. Firstly, while the commonly used l2,1 norm enhances clustering robustness, it remains susceptible to the impact of high-intensity noises. Secondly, the t-SVD based nuclear norm provides a biased estimation of tensor rank. Thirdly, existing methods either fail to preserve manifold structures from the original space or only extract binary relationships between data points. To address these challenges and enhance clustering performance in both high-intensity noisy and real-world conditions, an error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding (EMSC-NLTHG) method is proposed. Specifically, Cauchy loss function which reduces sensitivity to larger noises and outliers is introduced. Based on the Cauchy loss function, we develop the Cauchy pseudo norm to represent the reconstruction error in clustering to enhance the robustness. Moreover, rather than uniformly weighting all singular values in the t-SVD nuclear norm, a nonconvex tensor nuclear norm is adopted to add weights adaptively and approximate the true tensor rank. Additionally, hypergraph is embedded in the representation matrices to preserve local manifold structures and extract the complex multivariate relationships between data points. Extensive experiments demonstrate that the proposed EMSC-NLTHG method consistently outperforms state-of-the-art techniques by approximately 18% to 40% on datasets corrupted by high-intensity noises and around 1% to 27% on eight real-world datasets across six popular evaluation metrics.

Semi-supervised Symmetric Non-negative Matrix Factorization with Low-Rank Tensor Representation

Semi-supervised Symmetric Non-negative Matrix Factorization with Low-Rank Tensor Representation

Double Discrete Cosine Transform-Oriented Multi-View Subspace Clustering.

Low-rank tensor representation with the tensor nuclear norm has been rising in popularity in multi-view subspace clustering (MVSC), in which the tensor nuclear norm is commonly implemented using discrete Fourier transform (DFT). Unfortunately, existing DFT-oriented MVSC methods may provide unsatisfactory results since (1) DFT exploits complex arithmetic in the Fourier domain, usually resulting in high tubal tensor rank, and (2) local structural information is rarely considered. To solve these problems, in this paper, we propose a novel double discrete cosine transform (DCT)-oriented multi-view subspace clustering (D2CTMSC) method, in which the first DCT aims to derive the tensor nuclear norm without complex arithmetic while the second DCT aims to explore the local structure of the self-representation tensor, such that the essential low-rankness and sparsity embedding in multi-view features can be thoroughly exploited. Moreover, we design an effective alternating iteration strategy to solve the proposed model. Experimental results on four types of multi-view datasets (News stories, Face images, Scene images, and Generic objects) demonstrate the superiority of the D2CTMSC method compared with DFT-based methods and other state-of-the-art clustering methods.

Double-Factor Tensor Cascaded-Rank Decomposition for Hyperspectral Image Denoising

Hyperspectral image (HSIs) denoising is a preprocessing step that plays a crucial role in many applications used in Earth observation missions. Low-rank tensor representation can be utilized to restore mixed-noise HSIs, such as those affected by mixed Gaussian, impulse, stripe, and deadline noises. Although there is a considerable body of research on spatial and spectral prior knowledge concerning subspace, the correlation between the spectral continuity and the nonlocal sparsity of the spectral and spatial factors is not yet fully understood. To address this deficiency, in the present study, we determined the correlation between these factors using a cascaded technique, and we describe in this paper the double-factor tensor cascaded-rank (DFTCR) minimization method that was used. The information existing in the nonlocal sparsity property of the spatial factor was employed to promote a geometrical feature representation, and a tensor cascaded-rank minimization approach was introduced as a nonlocal self-similarity to promote restoration quality. The continuity between the difference and nonlocal gradient sparsity constraints of the spectral factor was also introduced to learn the basis. Furthermore, to estimate the solutions of the proposed model, we developed an algorithm based on the alternating direction method of multipliers (ADMM). The performance of the DFTCR method was tested by a comparison with eleven established denoising methods for HSIs. The results showed that the proposed DFTCR method exhibited superior performance in the removal of mixed noise from HSIs.

Multi-Dimensional Low-Rank with Weighted Schatten p-Norm Minimization for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection is an important unsupervised binary classification problem that aims to effectively distinguish between background and anomalies in hyperspectral images (HSIs). In recent years, methods based on low-rank tensor representations have been proposed to decompose HSIs into low-rank background and sparse anomaly tensors. However, current methods neglect the low-rank information in the spatial dimension and rely heavily on the background information contained in the dictionary. Furthermore, these algorithms show limited robustness when the dictionary information is missing or corrupted by high level noise. To address these problems, we propose a novel method called multi-dimensional low-rank (MDLR) for HSI anomaly detection. It first reconstructs three background tensors separately from three directional slices of the background tensor. Then, weighted schatten p-norm minimization is employed to enforce the low-rank constraint on the background tensor, and LF,1-norm regularization is used to describe the sparsity in the anomaly tensor. Finally, a well-designed alternating direction method of multipliers (ADMM) is employed to effectively solve the optimization problem. Extensive experiments on four real-world datasets show that our approach outperforms existing anomaly detection methods in terms of accuracy.

A low-rank isogeometric solver based on Tucker tensors

We propose an isogeometric solver for Poisson problems that combines (i) low-rank tensor techniques to approximate the unknown solution and the system matrix, as a sum of a few terms having Kronecker product structure, (ii) a Truncated Preconditioned Conjugate Gradient solver to keep the rank of the iterates low, and (iii) a novel low-rank preconditioner, based on the Fast Diagonalization method where the eigenvector multiplication is approximated by the Fast Fourier Transform. Although the proposed strategy is written in arbitrary dimension, we focus on the three-dimensional case and adopt the Tucker format for low-rank tensor representation, which is well suited in low dimension. We show by numerical tests that this choice guarantees significant memory saving compared to the full tensor representation. We also extend and test the proposed strategy to linear elasticity problems.

Semi-Supervised Subspace Clustering via Tensor Low-Rank Representation

In this letter, we propose a novel semi-supervised subspace clustering method, which is able to simultaneously augment the initial supervisory information and construct a discriminative affinity matrix. By representing the limited amount of supervisory information as a pairwise constraint matrix, we observe that the ideal affinity matrix for clustering shares the same low-rank structure as the ideal pairwise constraint matrix. Thus, we stack the two matrices into a 3-D tensor, where a global low-rank constraint is imposed to promote the affinity matrix construction and augment the initial pairwise constraints synchronously. Besides, we use the local geometry structure of input samples to complement the global low-rank prior to achieve better affinity matrix learning. The proposed model is formulated as a Laplacian graph regularized convex low-rank tensor representation problem, which is further solved with an alternative iterative algorithm. In addition, we propose to refine the affinity matrix with the augmented pairwise constraints. Comprehensive experimental results on eight commonly-used benchmark datasets demonstrate the superiority of our method over state-of-the-art methods. The code is publicly available at https://github.com/GuanxingLu/Subspace-Clustering.

Low-rank tensor methods for partial differential equations

Low-rank tensor representations can provide highly compressed approximations of functions. These concepts, which essentially amount to generalizations of classical techniques of separation of variables, have proved to be particularly fruitful for functions of many variables. We focus here on problems where the target function is given only implicitly as the solution of a partial differential equation. A first natural question is under which conditions we should expect such solutions to be efficiently approximated in low-rank form. Due to the highly nonlinear nature of the resulting low-rank approximations, a crucial second question is at what expense such approximations can be computed in practice. This article surveys basic construction principles of numerical methods based on low-rank representations as well as the analysis of their convergence and computational complexity.

Semi-supervised Multi-view Clustering Based on Non-negative Matrix Factorization and Low-Rank Tensor Representation

Semi-supervised Multi-view Clustering Based on Non-negative Matrix Factorization and Low-Rank Tensor Representation

Discovering Community Structure in Multiplex Networks via a Co-Regularized Robust Tensor-Based Spectral Approach

Complex networks arise in various fields, such as biology, sociology and communication, to model interactions among entities. Entities in many real-world systems exhibit different types of interactions, which requires modeling these type of systems properly. Multiplex networks are used to model these systems, as they can reflect the nodes’ pair-wise interactions as multiple distinct types of links across layers. Community detection is a widely studied application in network analysis as it provides insights into the structure and organization of the network. Even though multiple algorithms have been developed in the community detection field, many of them have a limited performance in the presence of noise. In this article, we develop a novel algorithm that combines tensor low-rank representation, spectral clustering and distance regularization to improve the accuracy in discovering communities in multiplex networks. The low-rank representation leads to reducing the noise and errors existing in the network and the optimization of an accurate consensus set of eigenvectors that reveals the communities in the network. Moreover, the proposed approach balances the agreement between the eigenvectors of each layer, i.e., individual subspaces, and the consensus set of eigenvectors, i.e., common subspaces, by minimizing the projection distance between them. The common and individual subspaces are computed efficiently through Tucker decomposition and modified spectral clustering, respectively. Finally, multiple experiments are conducted on real and simulated networks to evaluate the proposed approach and compare it to state-of-the-art algorithms. The proposed approach shows its robustness and efficiency in discovering the communities in multiplex networks.

Nonconvex low-rank tensor approximation with graph and consistent regularizations for multi-view subspace learning

Multi-view clustering is widely used to improve clustering performance. Recently, the subspace clustering tensor learning method based on Markov chain is a crucial branch of multi-view clustering. Tensor learning is commonly used to apply tensor low-rank approximation to represent the relationships between data samples. However, most of the current tensor learning methods have the following shortcomings: the information of the local graph is not taken into account, the relationships between different views are not shown, and the existing tensor low-rank representation takes a biased tensor rank function for estimation. Therefore, a nonconvex low-rank tensor approximation with graph and consistent regularizations (NLRTGC) model is proposed for multi-view subspace learning. NLRTGC retains the local manifold information through graph regularization, and adopts a consistent regularization between multi-views to keep the diagonal block structure of representation matrices. Furthermore, a nonnegative nonconvex low-rank tensor kernel function is used to replace the existing classical tensor nuclear norm via tensor-singular value decomposition (t-SVD), so as to reduce the deviation from rank. Then, an alternating direction method of multipliers (ADMM) which makes the objective function monotonically non-increasing is proposed to solve NLRTGC. Finally, the effectiveness and superiority of the NLRTGC are shown through abundant comparative experiments with various state-of-the-art algorithms on noisy datasets and real world datasets.

Online Tensor Low-Rank Representation for Streaming Data Clustering

Tensor low-rank representation (TLRR) has attracted increasing attention in recent years due to its effectiveness in capturing the intrinsic low-rank structure of tensor data. However, it is known that TLRR suffers from high computational complexity for large-scale data because it minimizes the tensor rank of the representation tensor whose size is proportional to the sample size. This paper develops a streaming algorithm—termed online low-rank tensor subspace clustering (OLRTSC)—for low-rank tensor data recovery and clustering based on the tensor singular value decomposition (t-SVD) algebraic framework. The key idea of OLRTSC is the non-convex reformulation of the objective function of TLRR by decomposing the tensor nuclear norm into an explicit tensor product of two low-rank tensors, which can be solved by leveraging the stochastic optimization technique. Compared to batch TLRR, OLRTSC is online by nature, can handle dynamic data, avoids computing t-SVD of the representation tensor, and reduces the storage cost significantly. This paper provides the theoretical guarantee that the sequence of the solutions produced by OLRTSC converges almost surely to a stationary point of the objective function. Moreover, an extension of OLRTSC is also proposed to handle the case when parts of the data are missing. Finally, experimental results on both synthetic and real data demonstrate the efficiency and robustness of the proposed approaches.