Tensor Nuclear Norm Research Articles

Joint Intra-view and Inter-view Enhanced Tensor Low-rank Induced Affinity Graph Learning

Graph-based and tensor-based multi-view clustering have gained popularity in recent years due to their ability to explore the relationship between samples. However, there are still several shortcomings in the current multi-view graph clustering algorithms. (1) Most previous methods only focus on the inter-view correlation, while ignoring the intra-view correlation. (2) They usually use the Tensor Nuclear Norm (TNN) to approximate the rank of tensors. However, while it has the same penalty for different singular values, the model cannot approximate the true rank of tensors well. To solve these problems in a unified way, we propose a new tensor-based multi-view graph clustering method. Specifically, we introduce the Enhanced Tensor Rank (ETR) minimization of intra-view and inter-view in the process of learning the affinity graph of each view. Compared with 10 state-of-the-art methods on 8 real datasets, the experimental results demonstrate the superiority of our method.

A Low Transformed Tubal Rank Tensor Model Using a Spatial-Tubal Constraint for Sample Clustering with Cancer Multi-omics Data

Background: Since each dimension of a tensor can store different types of genomics data, compared to matrix methods, utilizing tensor structure can provide a deeper understanding of multi-dimensional data while also facilitating the discovery of more useful information related to cancer. However, in reality, there are issues such as insufficient utilization of prior knowledge in multi-omics data and limitations in the recovery of low-tubal-rank tensors. Therefore, the method proposed in this article was developed. Objective: In this paper, we proposed a low transformed tubal rank tensor model (LTTRT) using a spatial-tubal constraint to accurately partition different types of cancer samples and provide reliable theoretical support for the identification, diagnosis, and treatment of cancer. Method: In the LTTRT method, the transformed tensor nuclear norm based on the transformed tensor singular value decomposition is characterized by the low-rank tensor, which can explore the global low-rank property of the tensor, resolving the challenge of the tensor nuclear norm-based method not achieving the lowest tubal rank. Additionally, the introduction of weighted total variation regularization is conducive to extracting more information from sequencing data in both spatial and tubal dimensions, exploring cross-correlation features of multiple genomic data, and addressing the problem of overlooking prior knowledge from various perspectives. In addition, the L1-norm is used to improve sparsity. A symmetric Gauss‒Seidel-based alternating direction method of multipliers (sGS-ADMM) is used to update the LTTRT model iteratively. Results: The experiments of sample clustering on multiple integrated cancer multi-omics datasets show that the proposed LTTRT method is better than existing methods. Experimental results validate the effectiveness of LTTRT in accurately partitioning different types of cancer samples. result: The experiments of sample clustering on multiple integrated cancer multi-omics datasets show that the proposed LTTRT method is better than existing methods. Experimental results validate the effectiveness of LTTRT in accurately partitioning different types of cancer samples. Conclusion: The LTTRT method achieves precise segmentation of different types of cancer samples.

Tensorized diversity and consistency with Laplacian manifold for multi-view clustering

The advantage of multi-view clustering lies in its ability to leverage the diversity and consistency among multiple views to better capture the intrinsic structure of the data. However, existing multi-view methods treat diversity and consistency as a set of opposing attributes, overlooking their inherent connections. Meanwhile, the complete information across multiple views is not fully utilized. To address these issues, this paper proposes the tensorized diversity and consistency with Laplacian manifold for multi-view clustering method (TDCLM). Specifically, starting from the self-expressive property of the original data, we obtain the diversity graphs and the consistency graph, and for the first time, we combined Laplacian manifold constraints to strengthen the relationship between diversity and consistency while jointly optimizing the diversity graphs and the consistency graph. Additionally, we innovatively combine the diversity graphs and the consistency graph into a tensor and subject it to the constraint of tensor nuclear norm. By doing so, we not only obtain the complete information between multiple views but also enable the mutual learning and mutual enhancement of the diversity graphs and the consistency graph. Finally, by adopting the augmented Lagrange multiplier method, we integrate the two steps into a comprehensive framework. The TDCLM shows a performance enhancement of up to 25.85%, with experimental results across diverse datasets demonstrating that the TDCLM algorithm surpasses the state-of-the-art algorithms. In other words, these experimental results validate the importance of obtaining complete information from multiple views and effectively leveraging the diversity and consistency inherent in this complete information. The code is publicly available at https://github.com/TongWuahpu/TDCLM.

A new nonconvex multi-view subspace clustering via learning a clean low-rank representation tensor

Abstract Recently, low-rank tensor representation has achieved impressive results for multi-view subspace clustering (MSC). The typical MSC methods utilize the tensor nuclear norm as a convex surrogate of the tensor multi-rank to obtain a low-rank representation, which exhibits limited robustness when dealing with noisy and complex data scenarios. In this paper, we introduce an innovative clean low-rank tensor representation approach that combines the idea of tensor robust principal component analysis (TRPCA) with a new nonconvex tensor multi-rank approximation regularization. This integration enhances the robustness of the low-rank representation, resulting in improved performance. Furthermore, to better capture the local geometric features, we employ a higher-order manifold regularization term. To effectively address our new model, we develop an iterative algorithm that has been proven to converge to the desired Karush-Kuhn-Tucker (KKT) point. The numerical experiments on widely used datasets serve to demonstrate the efficacy and effectiveness of our new method.

Flexible Dual Multi-Modal Hashing for Incomplete Multi-Modal Retrieval

Multimodal hashing aims to efficiently integrate multi-source data into a unified discrete Hamming space, facilitating fast similarity searches with minimal query and storage overhead. Traditional multimodal hashing assumes that data from different sources are fully observed, an assumption that fails in real-world scenarios involving large-scale multimodal data, thereby compromising conventional methods. To address these limitations during both training and retrieval, our approach manages dual-stage data missing, occurring in both phases. In this paper, we introduce a novel framework called Flexible Dual Multimodal Hashing (FDMH), which recovers missing data at both stages by jointly leveraging low-dimensional data relations and semantic graph structural relationships in multi-source data, achieving promising performance in incomplete multimodal retrieval. We transform the original features into anchor graphs and use existing modalities to reconstruct the anchor graphs of missing modalities. Based on these anchor graphs, we perform weight-adaptive fusion in the semantic space, supervised by original semantic labels, and apply a tensor nuclear norm to enforce consistency constraints on the projection matrices across different modalities. Furthermore, our method flexibly fuses existing and recovered modalities during retrieval. We validate the effectiveness of our approach through extensive experiments on four large-scale multimodal datasets, demonstrating its robust performance in real-world dual-missing retrieval scenarios.

A low-rank support tensor machine for multi-classification

In recent decades, there has been an increasing demand for effectively handling high-dimensional multi-channel tensor data. Due to the inability to utilize internal structural information, Support Vector Machine (SVM) and its variations struggle to classify flattened tensor data, consequently resulting in the ‘curse of dimensionality’ issue. Furthermore, most of these methods can not directly apply to multiclass datasets. To overcome these challenges, we have developed a novel classification method called Multiclass Low-Rank Support Tensor Machine (MLRSTM). Our method is inspired by the well-established low-rank tensor hypothesis, which suggests a correlation between each channel of the feature tensor. Specifically, MLRSTM adopts the hinge loss function and introduces a convex approximation of tensor rank, the order-d Tensor Nuclear Norm (order-d TNN), in the regularization term. By leveraging the order-d TNN, MLRSTM effectively exploits the inherent structural information in tensor data to enhance generalization performance and avoid the curse of dimensionality. Moreover, we develop the Alternating Direction Method of Multipliers (ADMM) algorithm to optimize the convex problem inherent in training MLRSTM. Finally, comprehensive experiments validate the excellent performance of MLRSTM in tensor multi-classification tasks, showcasing its potential and efficacy in handling high-dimensional multi-channel tensor data.

A robust low‐rank tensor completion model with sparse noise for higher‐order data recovery

AbstractThe tensor singular value decomposition‐based model has garnered increasing attention in addressing tensor recovery challenges. However, existing tensor recovery methods exhibit certain inherent limitations. Some ignore the simultaneous effects of noise and missing values, while most can't handle higher‐order tensors, which are not reflective of real‐world scenarios. The information redundancy within tensor data often leads to a prevailing low‐rank structure, making low‐rankness a vital prior in the tensor recovery process. To tackle this pressing issue, a robust low‐rank tensor recovery framework is proposed to rehabilitate higher‐order tensors corrupted by sparse noise and missing entries. In the model, the tensor nuclear norm derived for order‐d tensors (d 4) are employed as a representation of the low‐rank prior, while utilizing the ‐norm to model the sparse noise. To solve the proposed model, an efficient Alternating direction method of multipliers algorithm is developed. A series of experiments are performed on synthetic and real‐world datasets. The results show that the superior performance of the method compared with other algorithms dedicated to addressing order‐d tensor recovery challenges. Notably, in scenarios where the data is severely compromised (noise ratio 40%, sample ratio 70%), the algorithm consistently outperforms its competitors, achieving significantly improved results.

Spatio-temporal feature extraction network based multi-performance indicators synergetic monitoring method for complex industrial processes

In the context of smart manufacturing, modern industrial processes are becoming increasingly complex in terms of process flows, product varieties, and performance indicators (PIs). Performance-driven process monitoring attracts extensive attentions in recent years. However, most methods require spatio-temporal correspondence between process variables and PIs, and rarely consider the correlation among various PIs. In this paper, a spatio-temporal feature extraction network-based multi-performance indicators synergetic monitoring framework is presented. Firstly, considering the missing data in PI measurements, a weighted sum of tensor nuclear norm (WSTNN) based batch-process data completion approach is developed, which can adeptly handle local missing and incomplete data issues and establish the spatio-temporal correspondence for subsequent modeling. Secondly, for a specific PI, a new canonical variate analysis embedded spatio-temporal convolutional network (CVA-STCN) is designed to extract the PI-related features with spatio-temporal dependency. Thirdly, considering the dynamic interaction of multiple PIs, a third-order feature tensor is established to perform the future fusion, and the correlations among various PI-related features are explored via tensor decomposition. Finally, a hierarchical multi-performance indicators synergetic monitoring model is developed over several subspaces. The proposed method is verified on Tennessee Eastman process and an actual hot strip mill process. Overall, the monitoring performance of the proposed method outperforms the traditional ones, in terms of higher fault detection rates and lower false alarm rates. Moreover, the information provided by the multi-subspace synergetic monitoring charts can offer valuable guidance to field engineers.

Tensor double arc-tangent norm minimization for multi-view clustering

Tensor structures are widely utilized in clustering tasks due to their ability to represent high-order relationships among multi-view data. The rank and decomposition of tensors are essential characteristics that have recently received considerable attention. Most approaches approximate tensor rank using the tensor nuclear norm (TNN) and non-convex rank functions. We introduce a multi-view clustering method based on an enhanced self-expression with a tensor double arc-tangent norm. Initially leveraging tensor singular value decomposition, this method impartially approximates the true rank of the tensor, avoiding rank overestimation. The enhanced self-expression further utilizes the latent connections among the original data. Then, we propose an improved alternating direction method of multipliers (ADMM) for rapid optimization of the model. Experiments conducted on six datasets confirm the effectiveness of the proposed model.

Learning latent disentangled embeddings and graphs for multi-view clustering

Graph based methods have recently attracted much attention for multi-view clustering. Most existing methods seek the latent shared embeddings to learn a unified similarity graph or fuse multiple view-specific graphs to a consensus one for clustering, which may not sufficiently explore the common and complementary information among views. Besides, the high-order inter-view correlations are not fully investigated. To address these issues, this paper proposes a latent Disentangled Embeddings and GRaphs based multi-viEw clustEring (DEGREE) method, which considers the common and view-specific information in a latent subspace by explicit embedding disentanglement and multiple graphs learning. We assume that each view can be generated from a shared latent embedding and a corresponding view-specific embedding, which model the common information and exclusive complementary information among views, respectively. The intra-view and inter-view exclusivities among embeddings are encouraged by an orthogonality regularizer. To fully use the underlying information, we excavate the pairwise instance relations in both shared embedding and diverse view-specific embeddings by learning multiple graphs. Besides, a tensor singular value decomposition (t-SVD) based tensor nuclear norm regularizer is imposed on view-specific graphs, which helps to explore the high-order inter-view correlations. An alternative optimization algorithm is designed to solve the proposed model. Experimental evaluations on several popular datasets demonstrate that our DEGREE method outperforms the state-of-the-art methods.

TIHN: Tensor Improved Huber Norm for low-rank tensor recovery

Tensor Robust Principal Component Analysis (TRPCA) has received much attention in many real-world applications which aims to recover a low-rank tensor corrupted by sparse noise. Most existing TRPCA methods usually regularize the low-rank component by minimizing its Tensor Nuclear Norm (TNN). However, the original TNN shrinks all of the singular values with the same penalty which generally leads to suboptimal results. The reason is that the large singular values should be less penalized since they usually correspond to salient information of the real-world tensor. In this work, we develop a Tensor Improved Huber Norm (TIHN) which considers the difference between the singular values of tensor. The TIHN can achieve better performance by recovering the large and significant singular values exactly. We also establish the Huber TRPCA (HTRPCA) method by utilizing the proposed TIHN. In addition, an efficient optimization algorithm based on the half-quadratic theory and Alternating Direction Method of Multipliers (ADMM) framework is designed to implement the HTRPCA. Finally, experiments on color image recovery and video recovery validate the effectiveness of the proposed method.

Tensor-based global block-diagonal structure radiation for incomplete multiview clustering

Incomplete multiview clustering (IMVC) has attracted extensive attention in the field of machine learning due to its excellent performance in handling incomplete multiview data. However, existing IMVC methods have certain limitions:1) the completion methods are not flexible enough for cases where view information is arbitrarily missing; 2) the existing methods cannot fully exploit the information contained in the clustering indicator matrix, which is crucial to the clustering results. To solve these problems, we propose a novel method, i.e., tensor-based global block-diagonal structure radiation for incomplete multiview clustering (TGBSR). First, we utilize the information from known samples to construct similarity graphs and integrate these matrices into a third-order tensor. Second, to explore the consensus information among the different similarity graphs, we introduce the spectral embedding technique to obtain the clustering indicator matrix, i.e., U. Third, and more importantly, to fully explore the information contained in the clustering indicator matrix, we introduce a novel matrix, i.e., the UUT, which is low-rank and naturally has an excellent block diagonal structure, and then incorporate it as a new view into the aforementioned tensor, which is constrained by the tensor nuclear norm (TNN), to mine the high-order correlations among all the different views. Finally, we combine all these steps into a unified framework and develop an effective iterative optimization procedure to solve it. The experimental results on several well-known datasets demonstrate the effectiveness of our method. The code is publicly available at https://github.com/hhipp-999/TGBSR.

A nonlinear high-order transformations-based method for high-order tensor completion

The high-order tensor nuclear norm model (HTNN) has recently shown promising results in tensor completion problems. The HTNN-based approaches rely on the low-rank structure of tensor slices during reversible transformations. However, under reversible transformations, the low-rank configuration within the slice-wise modality of the tensor’s structure is not markedly pronounced. In order to more effectively describe the low-rank characteristics, we propose a model based on the nonlinear high-order transform-based tensor nuclear norm (NHTNN). Specifically, our framework consists of a linearly semi-orthogonal transformation along the high-dimensional modality and an element-wise nonlinear transformation. We introduce a model for tensor completion, grounded in the suggested measure of tensor low-rank, i.e., NHTNN. Utilizing this non-convex nonlinear model, we formulate a proximal alternating minimization (PAM) algorithm, establishing its convergence through a rigorous proof. In experiments on datasets such as hyperspectral videos (HSVs) and color videos (CVs), our approach demonstrates superior quantitative numerical results and qualitative visual effects compared to cutting-edge tensor completion techniques.

Efficient enhancement of low-rank tensor completion via thin QR decomposition.

Low-rank tensor completion (LRTC), which aims to complete missing entries from tensors with partially observed terms by utilizing the low-rank structure of tensors, has been widely used in various real-world issues. The core tensor nuclear norm minimization (CTNM) method based on Tucker decomposition is one of common LRTC methods. However, the CTNM methods based on Tucker decomposition often have a large computing cost due to the fact that the general factor matrix solving technique involves multiple singular value decompositions (SVDs) in each loop. To address this problem, this article enhances the method and proposes an effective CTNM method based on thin QR decomposition (CTNM-QR) with lower computing complexity. The proposed method extends the CTNM by introducing tensor versions of the auxiliary variables instead of matrices, while using the thin QR decomposition to solve the factor matrix rather than the SVD, which can save the computational complexity and improve the tensor completion accuracy. In addition, the CTNM-QR method's convergence and complexity are analyzed further. Numerous experiments in synthetic data, real color images, and brain MRI data at different missing rates demonstrate that the proposed method not only outperforms in terms of completion accuracy and visualization, but also conducts more efficiently than most state-of-the-art LRTC methods.

Low-Rank Tensor Completion Based on Self-Adaptive Learnable Transforms.

The tensor nuclear norm (TNN), defined as the sum of nuclear norms of frontal slices of the tensor in a frequency domain, has been found useful in solving low-rank tensor recovery problems. Existing TNN-based methods use either fixed or data-independent transformations, which may not be the optimal choices for the given tensors. As the consequence, these methods cannot exploit the potential low-rank structure of tensor data adaptively. In this article, we propose a framework called self-adaptive learnable transform (SALT) to learn a transformation matrix from the given tensor. Specifically, SALT aims to learn a lossless transformation that induces a lower average-rank tensor, where the Schatten- p quasi-norm is used as the rank proxy. Then, because SALT is less sensitive to the orientation, we generalize SALT to other dimensions of tensor (SALTS), namely, learning three self-adaptive transformation matrices simultaneously from given tensor. SALTS is able to adaptively exploit the potential low-rank structures in all directions. We provide a unified optimization framework based on alternating direction multiplier method for SALTS model and theoretically prove the weak convergence property of the proposed algorithm. Experimental results in hyperspectral image (HSI), color video, magnetic resonance imaging (MRI), and COIL-20 datasets show that SALTS is much more accurate in tensor completion than existing methods. The demo code can be found at https://faculty.uestc.edu.cn/gaobin/zh_CN/lwcg/153392/list/index.htm.

Low-Rank Tensor Regularized Views Recovery for Incomplete Multiview Clustering.

In real applications, it is often that the collected multiview data contain missing views. Most existing incomplete multiview clustering (IMVC) methods cannot fully utilize the underlying information of missing data or sufficiently explore the consistent and complementary characteristics. In this article, we propose a novel Low-rAnk Tensor regularized viEws Recovery (LATER) method for IMVC, which jointly reconstructs and utilizes the missing views and learns multilevel graphs for comprehensive similarity discovery in a unified model. The missing views are recovered from a common latent representation, and the recovered views conversely improve the learning of shared patterns. Based on the shared subspace representations and recovered complete multiview data, the multilevel graphs are learned by self-representation to fully exploit the consistent and complementary information among views. Besides, a tensor nuclear norm regularizer is introduced to pursue the global low-rank property and explore the interview correlations. An alternating direction minimization algorithm is presented to optimize the proposed model. Moreover, a new initialization method is proposed to promote the effectiveness of our method for latent representation learning and missing data recovery. Extensive experiments demonstrate that our method outperforms the state-of-the-art approaches.

CoNoT: Coupled Nonlinear Transform-Based Low-Rank Tensor Representation for Multidimensional Image Completion.

Recently, the transform-based tensor nuclear norm (TNN) methods have shown promising performance and drawn increasing attention in tensor completion (TC) problems. The main idea of these methods is to exploit the low-rank structure of frontal slices of the tensor under the transform. However, the transforms in TNN methods usually treat all modes equally and do not consider the different traits of different modes (i.e., spatial and spectral/temporal modes). To address this problem, we suggest a new low-rank tensor representation based on the coupled nonlinear transform (called CoNoT) for a better low-rank approximation. Concretely, spatial and spectral/temporal transforms in the CoNoT, respectively, exploit the different traits of different modes and are coupled together to boost the implicit low-rank structure. Here, we use the convolutional neural network (CNN) as the CoNoT, which can be learned solely from an observed multidimensional image in an unsupervised manner. Based on this low-rank tensor representation, we build a new multidimensional image completion model. Moreover, we also propose an enhanced version (called Ms-CoNoT) to further exploit the spatial multiscale nature of real-world data. Extensive experiments on real-world data substantiate the superiority of the proposed models against many state-of-the-art methods both qualitatively and quantitatively.

Poisson tensor completion with transformed correlated total variation regularization

Tensor completion involves recovering the underlying tensor from partial observations, and in this paper we focus on the point that these observations obey the Poisson distribution. To contend with this problem, we adopt a popular method that minimizes the sum of the data-fitting term and the regularization term under a uniform sampling mechanism. Specifically, we consider the negative logarithmic maximum likelihood estimate of the Poisson distribution as the data-fitting term. To effectively characterize the intrinsic structure of the tensor data, we propose a parameter-free regularization term that can simultaneously capture the low rankness and local smoothness of the underlying tensor. Here, the transformed tensor nuclear norm is used to explore the low rankness under suitable unitary transformations. We present theoretical derivations to demonstrate the feasibility of the proposed model. Furthermore, we develop an algorithm based on the alternating direction multiplier method (ADMM) to efficiently solve the proposed optimization problem, with its overall convergence being established. A series of numerical experiments show that proposed model yields a pleasing accuracy over several state-of-the-art models.

Coupled double self-expressive subspace clustering with low-rank tensor learning

In recent years, subspace clustering (SC) methods have been widely used in machine learning and computer vision. However, the self-expressive matrix obtained by the existing SC methods is not clean block diagram and still contains some of typical noise in data, which leads to a decrease in clustering performance. In addition, most SC methods do not make full use of the structural information of self-expressive matrices. To solve the above problems, we propose coupled double self-expressive subspace clustering with low-rank tensor learning (CDSESLRTL) method. Specifically, first, we use the self-expressive characteristics of the data to obtain the self-expressive matrix of the data. However, the structural information of the self-expressive matrix is not fully utilized, and the block diagonal structure is often not sufficiently clear. Then, we obtain a new self-expressive matrix by viewing the obtained self-expressive matrix as a feature representation of the data and use self-expressive characteristics again. Third, we stack the two self-expressive matrices into a tensor, which is constrained by the tensor nuclear norm (TNN). Then, the two self-expressive matrices can be learned in a mutually reinforcing manner, which not only gives these matrices a clearer block diagonal structure but also sufficiently utilizes their structural information. Finally, the combination of these three steps forms a framework, which is addressed using the augmented Lagrange multiplier (ALM) method. The experimental results on various datasets demonstrate that the CDSESLRTL algorithm outperforms the state-of-the-art algorithms.

Robust Tensor Learning for Multi-View Spectral Clustering

Tensor-based multi-view spectral clustering methods are promising in practical clustering applications. However, most of the existing methods adopt the ℓ2,1 norm to depict the sparsity of the error matrix, and they usually ignore the global structure embedded in each single view, compromising the clustering performance. Here, we design a robust tensor learning method for multi-view spectral clustering (RTL-MSC), which employs the weighted tensor nuclear norm to regularize the essential tensor for exploiting the high-order correlations underlying multiple views and adopts the nuclear norm to constrain each frontal slice of the essential tensor as the block diagonal matrix. Simultaneously, a novel column-wise sparse norm, namely, ℓ2,p, is defined in RTL-MSC to measure the error tensor, making it sparser than the one derived by the ℓ2,1 norm. We design an effective optimization algorithm to solve the proposed model. Experiments on three widely used datasets demonstrate the superiority of our method.