Tensor Singular Value Decomposition Research Articles

A Low Transformed Tubal Rank Tensor Model Using a Spatial-Tubal Constraint for Sample Clustering with Cancer Multi-omics Data

Background: Since each dimension of a tensor can store different types of genomics data, compared to matrix methods, utilizing tensor structure can provide a deeper understanding of multi-dimensional data while also facilitating the discovery of more useful information related to cancer. However, in reality, there are issues such as insufficient utilization of prior knowledge in multi-omics data and limitations in the recovery of low-tubal-rank tensors. Therefore, the method proposed in this article was developed. Objective: In this paper, we proposed a low transformed tubal rank tensor model (LTTRT) using a spatial-tubal constraint to accurately partition different types of cancer samples and provide reliable theoretical support for the identification, diagnosis, and treatment of cancer. Method: In the LTTRT method, the transformed tensor nuclear norm based on the transformed tensor singular value decomposition is characterized by the low-rank tensor, which can explore the global low-rank property of the tensor, resolving the challenge of the tensor nuclear norm-based method not achieving the lowest tubal rank. Additionally, the introduction of weighted total variation regularization is conducive to extracting more information from sequencing data in both spatial and tubal dimensions, exploring cross-correlation features of multiple genomic data, and addressing the problem of overlooking prior knowledge from various perspectives. In addition, the L1-norm is used to improve sparsity. A symmetric Gauss‒Seidel-based alternating direction method of multipliers (sGS-ADMM) is used to update the LTTRT model iteratively. Results: The experiments of sample clustering on multiple integrated cancer multi-omics datasets show that the proposed LTTRT method is better than existing methods. Experimental results validate the effectiveness of LTTRT in accurately partitioning different types of cancer samples. result: The experiments of sample clustering on multiple integrated cancer multi-omics datasets show that the proposed LTTRT method is better than existing methods. Experimental results validate the effectiveness of LTTRT in accurately partitioning different types of cancer samples. Conclusion: The LTTRT method achieves precise segmentation of different types of cancer samples.

Learnable Transform-Assisted Tensor Decomposition for Spatio-Irregular Multidimensional Data Recovery

Tensor decompositions have been successfully applied to multidimensional data recovery. However, classical tensor decompositions are not suitable for emerging spatio-irregular multidimensional data (i.e., spatio-irregular tensor), whose spatial domain is non-rectangular, e.g., spatial transcriptomics data from bioinformatics and semantic units from computer vision. By using preprocessing (e.g., zero-padding or element-wise 0-1 weighting), the spatio-irregular tensor can be converted to a spatio-regular tensor and then classical tensor decompositions can be applied, but this strategy inevitably introduces bias information, leading to artifacts. How to design a tensor-based method suitable for emerging spatio-irregular tensors is an imperative challenge. To address this challenge, we propose a learnable transform-assisted tensor singular value decomposition (LTA-TSVD) for spatio-irregular tensor recovery, which allows us to leverage the intrinsic structure behind the spatio-irregular tensor. Specifically, we design a learnable transform to project the original spatio-irregular tensor into its latent spatio-regular tensor, and then the latent low-rank structure is captured by classical TSVD on the resulting regular tensor. Empowered by LTA-TSVD, we develop spatio-irregular low-rank tensor completion (SIR-LRTC) and spatio-irregular tensor robust principal component analysis (SIR-TRPCA) models for the spatio-irregular tensor imputation and denoising respectively, and we design corresponding solving algorithms with theoretical convergence. Extensive experiments including the spatial transcriptomics data imputation and hyperspectral image denoising show SIR-LRTC and SIR-TRPCA are superior performance to competing approaches and benefit downstream applications.

Adaptive sampling with tensor leverage scores for exact low-rank third-order tensor completion

Tensor completion aims at estimating the missing entries from the incomplete observation. Under the tensor singular value decomposition framework, the exact recovery of a low-tubal-rank third-order tensor could be achieved via convex optimization with high probability if the tensor satisfies the tensor incoherence condition. In this work, we show that, when the random selection of entries is made adaptive to a distribution which is dependent on the coherence structure of the tensor, any low-tubal-rank tensor of the size n×n×n with tubal-rank r can be exactly recovered with high probability from as few as O(rn2log2⁡(n)) randomly chosen entries. In practice, tensor leverage scores are not known a priori, and we design a two-phase adaptive sampling strategy to obtain the leverage scores. Numerical experiments on synthetic and real-world third-order tensor data sets are used to validate our theoretical results and illustrate that the tensor recovery performance of the proposed two-phase adaptive sampling scheme is better than that of the other state-of-the-art methods.

Robust Hashing with Deep Features and Meixner Moments for Image Copy Detection

Copy detection is a key task of image copyright protection. Most robust hashing schemes do not make satisfied performance of image copy detection yet. To address this, a robust hashing scheme with deep features and Meixner moments is proposed for image copy detection. In the proposed hashing, global deep features are extracted by applying tensor Singular Value Decomposition (t-SVD) to the three-order tensor constructed in the DWT domain of the feature maps calculated by the pre-trained VGG16. Since the feature maps in the DWT domain are slightly disturbed by digital operations, the constructed three-order tensor is stable and thus the desirable robustness is guaranteed. Moreover, since t-SVD can decompose a three-order tensor into multiple low-dimensional matrices reflecting intrinsic structure, the global deep feature calculation from the low-dimensional matrices can provide good discrimination. Local features are calculated by the block-based Meixner moments. As the Meixner moments are resistant to geometric transformation and can efficiently discriminate various images, the use of the block-based Meixner moments can make discriminative and robust local features. Hash is ultimately determined by quantifying and combining global deep features and local features. The results of extensive experiments on open image datasets demonstrate that the proposed robust hashing outperforms some state-of-the-art robust hashing schemes in terms of classification and copy detection performances.

Adaptive variable sampling T-SSD algorithm

Addressing the challenges of non-unique decomposition outcomes and prolonged decomposition durations in the fault feature adaptive extraction algorithm based on tensor decomposition, this paper presents a novel algorithm called the adaptive variable sampling tensor singular spectrum decomposition (T-SSD) algorithm. The proposed approach centers on decomposing multichannel time series with adaptive sampling frequency, leveraging tensor singular value decomposition. Initially, the embedding dimension and the number of resampling points were optimized by power spectral density analysis and adaptive sampling algorithm. Subsequently, a third-order tensor is constructed based on the principle of tensor-tensor-preserving order multiplication, combining trajectory tensor construction and embedding dimension. Finally, the signal decomposition and reconstruction of multichannel component signals are achieved through adaptive sampling, interpolation, and complementary back steps. Experimental signal analysis indicates that this algorithm can better extract fault characteristics from signals compared to common signal processing algorithms. In comparison with the traditional T-SSD algorithm, this method significantly improves decomposition efficiency, particularly for low-frequency components. It effectively tackles the efficiency challenge caused by data redundancy, enabling the organic fusion and adaptive decomposition of multichannel signals.

Bayesian Dictionary Learning on Robust Tubal Transformed Tensor Factorization.

The recent study on tensor singular value decomposition (t-SVD) that performs the Fourier transform on the tubes of a third-order tensor has gained promising performance on multidimensional data recovery problems. However, such a fixed transformation, e.g., discrete Fourier transform and discrete cosine transform, lacks being self-adapted to the change of different datasets, and thus, it is not flexible enough to exploit the low-rank and sparse property of the variety of multidimensional datasets. In this article, we consider a tube as an atom of a third-order tensor and construct a data-driven learning dictionary from the observed noisy data along the tubes of the given tensor. Then, a Bayesian dictionary learning (DL) model with tensor tubal transformed factorization, aiming to identify the underlying low-tubal-rank structure of the tensor effectively via the data-adaptive dictionary, is developed to solve the tensor robust principal component analysis (TRPCA) problem. With the defined pagewise tensor operators, a variational Bayesian DL algorithm is established and updates the posterior distributions instantaneously along the third dimension to solve the TPRCA. Extensive experiments on real-world applications, such as color image and hyperspectral image denoising and background/foreground separation problems, demonstrate both effectiveness and efficiency of the proposed approach in terms of various standard metrics.

Tensor double arc-tangent norm minimization for multi-view clustering

Tensor structures are widely utilized in clustering tasks due to their ability to represent high-order relationships among multi-view data. The rank and decomposition of tensors are essential characteristics that have recently received considerable attention. Most approaches approximate tensor rank using the tensor nuclear norm (TNN) and non-convex rank functions. We introduce a multi-view clustering method based on an enhanced self-expression with a tensor double arc-tangent norm. Initially leveraging tensor singular value decomposition, this method impartially approximates the true rank of the tensor, avoiding rank overestimation. The enhanced self-expression further utilizes the latent connections among the original data. Then, we propose an improved alternating direction method of multipliers (ADMM) for rapid optimization of the model. Experiments conducted on six datasets confirm the effectiveness of the proposed model.

Learning latent disentangled embeddings and graphs for multi-view clustering

Graph based methods have recently attracted much attention for multi-view clustering. Most existing methods seek the latent shared embeddings to learn a unified similarity graph or fuse multiple view-specific graphs to a consensus one for clustering, which may not sufficiently explore the common and complementary information among views. Besides, the high-order inter-view correlations are not fully investigated. To address these issues, this paper proposes a latent Disentangled Embeddings and GRaphs based multi-viEw clustEring (DEGREE) method, which considers the common and view-specific information in a latent subspace by explicit embedding disentanglement and multiple graphs learning. We assume that each view can be generated from a shared latent embedding and a corresponding view-specific embedding, which model the common information and exclusive complementary information among views, respectively. The intra-view and inter-view exclusivities among embeddings are encouraged by an orthogonality regularizer. To fully use the underlying information, we excavate the pairwise instance relations in both shared embedding and diverse view-specific embeddings by learning multiple graphs. Besides, a tensor singular value decomposition (t-SVD) based tensor nuclear norm regularizer is imposed on view-specific graphs, which helps to explore the high-order inter-view correlations. An alternative optimization algorithm is designed to solve the proposed model. Experimental evaluations on several popular datasets demonstrate that our DEGREE method outperforms the state-of-the-art methods.

Cyclic tensor singular value decomposition with applications in low-rank high-order tensor recovery

The rapid advancements in emerging technologies have increased the demand for recovery tasks involving high-dimensional data with complex structures. Effectively utilizing tensor decomposition techniques to capture the low-rank structure of such data is crucial. Recently, the high-order t-SVD has demonstrated strong adaptability. However, this decomposition approach can only capture the low-rank correlation of two modes along other modes individually, while disregarding the structural correlation between different modes. In this paper, we propose a novel cyclic tensor singular value decomposition (CTSVD) method that effectively characterizes the low-rank structures of high-order tensors along all modes. Specifically, our method decomposes an order-N tensor into N factor tensors and one core tensor, connecting them using a defined mode-k tensor-tensor product (t-product). Building upon this, we establish the corresponding tensor rank and its convex relaxation. To address the issue of dimensional imbalance between adjacent modes in high-dimensional data, we propose and integrate a square reshaping strategy into the recovery models for tensor completion (TC) and tensor principal component analysis (TRPCA) tasks. Effective alternating direction method of multipliers (ADMM)-based algorithms are designed to these tasks. Extensive experiments on both synthetic and real data demonstrate that our methods outperform state-of-the-art approaches.

A Robust Dual Watermarking using Grey Wolf Optimization, Selective Encryption and Fast Flexible De-Noising Convolution Neural Network

Digital data interchange in IoT systems has flourished with the advancement of industrial internet technologies. Particularly, more and more digital images created by intelligent and industrial equipment are sent there are security concerns related to the website, server, and cloud. To accomplish this issue, in this article a secure watermarking approach is suggested in this research to effectively improve security, invisibility, and resilience at the same time. The adequate coefficient for information embedding is first determined using an assortment of transform domain techniques Discrete-Wavelet-Transform (DWT), Heisenberg- decomposition (HD), and Tensor-singular-value-decomposition (T-SVD). Using the grey wolf optimization (GWO) approach, we estimated the appropriate embedding factors to provide a reasonable compromise between robustness and invisibility. To enable the suggested approach to offer an additional level of security, a selective encryption technique is used on the watermark image. Moreover, FFDNet—a quick and adaptable de-noising convolutional-neural–network is working to increase the robustness-of-the suggested algorithm. The results demonstrate that the recommended watermarking method generates exceptional imperceptibility, resilience, and security against standard attacks. Additionally, the comparison demonstrates that the suggested algorithm performs better than alternative strategies. The following metrics were reached: 51.6966 dB, 0.9944, 0.9961, and 0.2849 for the peak-signal- to-noise ratio (PSNR), Structural-Similarity-Index (SSIM), number of changing pixels per second (NPCR), and unified-averaged-changed-intensity (UACI) average scores.

Robust Corrupted Data Recovery and Clustering via Generalized Transformed Tensor Low-Rank Representation.

Tensor analysis has received widespread attention in high-dimensional data learning. Unfortunately, the tensor data are often accompanied by arbitrary signal corruptions, including missing entries and sparse noise. How to recover the characteristics of the corrupted tensor data and make it compatible with the downstream clustering task remains a challenging problem. In this article, we study a generalized transformed tensor low-rank representation (TTLRR) model for simultaneously recovering and clustering the corrupted tensor data. The core idea is to find the latent low-rank tensor structure from the corrupted measurements using the transformed tensor singular value decomposition (SVD). Theoretically, we prove that TTLRR can recover the clean tensor data with a high probability guarantee under mild conditions. Furthermore, by using the transform adaptively learning from the data itself, the proposed TTLRR model can approximately represent and exploit the intrinsic subspace and seek out the cluster structure of the tensor data precisely. An effective algorithm is designed to solve the proposed model under the alternating direction method of multipliers (ADMMs) algorithm framework. The effectiveness and superiority of the proposed method against the compared methods are showcased over different tasks, including video/face data recovery and face/object/scene data clustering.

Adaptive Low-Rank Tensor Estimation Model Based Multichannel Weak Fault Detection for Bearings.

Multichannel signals contain an abundance of fault characteristic information on equipment and show greater potential for weak fault characteristics extraction and early fault detection. However, how to effectively utilize the advantages of multichannel signals with their information richness while eliminating interference components caused by strong background noise and information redundancy to achieve accurate extraction of fault characteristics is still challenging for mechanical fault diagnosis based on multichannel signals. To address this issue, an effective weak fault detection framework for multichannel signals is proposed in this paper. Firstly, the advantages of a tensor on characterizing fault information were displayed, and the low-rank property of multichannel fault signals in a tensor domain is revealed through tensor singular value decomposition. Secondly, to tackle weak fault characteristics extraction from multichannel signals under strong background noise, an adaptive threshold function is introduced, and an adaptive low-rank tensor estimation model is constructed. Thirdly, to further improve the accurate estimation of weak fault characteristics from multichannel signals, a new sparsity metric-oriented parameter optimization strategy is provided for the adaptive low-rank tensor estimation model. Finally, an effective multichannel weak fault detection framework is formed for rolling bearings. Multichannel data from the repeatable simulation, the publicly available XJTU-SY whole lifetime datasets and an accelerated fatigue test of rolling bearings are used to validate the effectiveness and practicality of the proposed method. Excellent results are obtained in multichannel weak fault detection with strong background noise, especially for early fault detection.

Incomplete Data Meets Uncoupled Case: A Challenging Task of Multiview Clustering.

Incomplete multiview clustering (IMC) methods have achieved remarkable progress by exploring the complementary information and consensus representation of incomplete multiview data. However, to our best knowledge, none of the existing methods attempts to handle the uncoupled and incomplete data simultaneously, which affects their generalization ability in real-world scenarios. For uncoupled incomplete data, the unclear and partial cross-view correlation introduces the difficulty to explore the complementary information between views, which results in the unpromising clustering performance for the existing multiview clustering methods. Besides, the presence of hyperparameters limits their applications. To fill these gaps, a novel uncoupled IMC (UIMC) method is proposed in this article. Specifically, UIMC develops a joint framework for feature inferring and recoupling. The high-order correlations of all views are explored by performing a tensor singular value decomposition (t-SVD)-based tensor nuclear norm (TNN) on recoupled and inferred self-representation matrices. Moreover, all hyperparameters of the UIMC method are updated in an exploratory manner. Extensive experiments on six widely used real-world datasets have confirmed the superiority of the proposed method in handling the uncoupled incomplete multiview data compared with the state-of-the-art methods.

Correction to: An Efficient Randomized Fixed-Precision Algorithm for Tensor Singular Value Decomposition

Correction to: An Efficient Randomized Fixed-Precision Algorithm for Tensor Singular Value Decomposition

Weighted Robust Tensor Principal Component Analysis for the Recovery of Complex Corrupted Data in a 5G-Enabled Internet of Things

Technological developments coupled with socioeconomic changes are driving a rapid transformation of the fifth-generation (5G) cellular network landscape. This evolution has led to versatile applications with fast data-transfer capabilities. The integration of 5G with wireless sensor networks (WSNs) has rendered the Internet of Things (IoTs) crucial for measurement and sensing. Although 5G-enabled IoTs are vital, they face challenges in data integrity, such as mixed noise, outliers, and missing values, owing to various transmission issues. Traditional methods such as the tensor robust principal component analysis (TRPCA) have limitations in preserving essential data. This study introduces an enhanced approach, the weighted robust tensor principal component analysis (WRTPCA), combined with weighted tensor completion (WTC). The new method enhances data recovery using tensor singular value decomposition (t-SVD) to separate regular and abnormal data, preserve significant components, and robustly address complex data corruption issues, such as mixed noise, outliers, and missing data, with the globally optimal solution determined through the alternating direction method of multipliers (ADMM). Our study is the first to address complex corruption in multivariate data using the WTRPCA. The proposed approach outperforms current techniques. In all corrupted scenarios, the normalized mean absolute error (NMAE) of the proposed method is typically less than 0.2, demonstrating strong performance even in the most challenging conditions in which other models struggle. This highlights the effectiveness of the proposed approach in real-world 5G-enabled IoTs.

NOODLE: Joint Cross-View Discrepancy Discovery and High-Order Correlation Detection for Multi-View Subspace Clustering

Benefiting from the effective exploration of the valuable topological pair-wise relationship of data points across multiple views, multi-view subspace clustering (MVSC) has received increasing attention in recent years. However, we observe that existing MVSC approaches still suffer from two limitations that need to be further improved to enhance the clustering effectiveness. Firstly, previous MVSC approaches mainly prioritize extracting multi-view consistency, often neglecting the cross-view discrepancy that may arise from noise, outliers, and view-inherent properties. Secondly, existing techniques are constrained by their reliance on pair-wise sample correlation and pair-wise view correlation, failing to capture the high-order correlations that are enclosed within multiple views. To address these issues, we propose a novel MVSC framework via joi N t cr O ss-view discrepancy disc O very an D high-order corre L ation d E tection ( NOODLE ), seeking an informative target subspace representation compatible across multiple features to facilitate the downstream clustering task. Specifically, we first exploit the self-representation mechanism to learn multiple view-specific affinity matrices, which are further decomposed into cohesive factors and incongruous factors to fit the multi-view consistency and discrepancy, respectively. Additionally, an explicit cross-view sparse regularization is applied to incoherent parts, ensuring the consistency and discrepancy to be precisely separated from the initial subspace representations. Meanwhile, the multiple cohesive parts are stacked into a three-dimensional tensor associated with a tensor-Singular Value Decomposition (t-SVD) based weighted tensor nuclear norm constraint, enabling effective detection of the high-order correlations implicit in multi-view data. Our proposed method outperforms state-of-the-art methods for multi-view clustering on six benchmark datasets, demonstrating its effectiveness.

Tensor structure-based ground clutter suppression approaches for pulse doppler radar

ABSTRACT For pulse doppler (PD) radar, two tensor structure-based methods are proposed to improve the clutter suppression performance. One is tensor singular value decomposition (TSVD) and the other is tensor robust principal component analysis (TRPCA). Two algorithms both stack the range-Doppler matrices of multiple coherent processing intervals (CPI) into a tensor structure, which helps make the most of the temporal information across CPIs. TSVD utilizes the strong temporal correlation of ground clutter data to estimate the clutter subspace. Meanwhile, TRPCA exactly separates both low-rank clutter component and sparse target components from the time series range-Doppler tensor structure. Experimental results indicate that compared with conventional matrix-based schemes such as singular value decomposition (SVD) or singular value decomposition (RPCA), the proposed methods have significant advantages in keeping the target echo while suppressing the clutter. In addition, the computational complexity of each algorithm is analysed in detail.

Randomized two-sided subspace iteration for low-rank matrix and tensor decomposition

The low-rank approximation of big data matrices and tensors plays a pivotal role in many modern applications. Although, a truncated version of the singular value decomposition (SVD) furnishes the best approximation, its computation is challenging on modern, multicore architectures. Recently, the randomized subspace iteration has shown to be a powerful tool in approximating large-scale matrices. In this paper we present a two-sided variant of the randomized subspace iteration. Novel in our work is the exploitation of the unpivoted QR factorization, rather than the SVD, for factorizing the compressed matrix. Hence our algorithm is a randomized rank-revealing URV decomposition. We prove the rank-revealingness of our algorithm by establishing bounds for the singular values as well as the other blocks of the compressed matrix. We further provide bounds on the error of the low-rank approximations of the proposed algorithm, in both 2- and Frobenius norm. In addition, we employ the proposed algorithm to efficiently compute low rank tensor decompositions: we present two randomized algorithms, one for the truncated higher-order SVD, and the other for the tensor SVD. We conduct numerical tests on (i) various classes of matrices, and (ii) synthetic tensors and real datasets to demonstrate the efficacy of the proposed algorithms.

Error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding

Tensor based subspace clustering, a method aimed at partitioning multi-view data into distinct clusters through the tensor low-rank representation, has gained widespread popularity. Despite the effectiveness of tensor singular value decomposition (t-SVD) based nuclear norm in extracting high-dimensional information from multiple views, certain limitations persist. Firstly, while the commonly used l2,1 norm enhances clustering robustness, it remains susceptible to the impact of high-intensity noises. Secondly, the t-SVD based nuclear norm provides a biased estimation of tensor rank. Thirdly, existing methods either fail to preserve manifold structures from the original space or only extract binary relationships between data points. To address these challenges and enhance clustering performance in both high-intensity noisy and real-world conditions, an error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding (EMSC-NLTHG) method is proposed. Specifically, Cauchy loss function which reduces sensitivity to larger noises and outliers is introduced. Based on the Cauchy loss function, we develop the Cauchy pseudo norm to represent the reconstruction error in clustering to enhance the robustness. Moreover, rather than uniformly weighting all singular values in the t-SVD nuclear norm, a nonconvex tensor nuclear norm is adopted to add weights adaptively and approximate the true tensor rank. Additionally, hypergraph is embedded in the representation matrices to preserve local manifold structures and extract the complex multivariate relationships between data points. Extensive experiments demonstrate that the proposed EMSC-NLTHG method consistently outperforms state-of-the-art techniques by approximately 18% to 40% on datasets corrupted by high-intensity noises and around 1% to 27% on eight real-world datasets across six popular evaluation metrics.

A Tensor Approach for Uncoupled Multiview Clustering.

Multiview clustering plays an important part in unsupervised learning. Although the existing methods have shown promising clustering performances, most of them assume that the data is completely coupled between different views, which is unfortunately not always ensured in real-world applications. The clustering performance of these methods drops dramatically when handling the uncoupled data. The main reason is that: 1) cross-view correlation of uncoupled data is unclear, which limits the existing multiview clustering methods to explore the complementary information between views and 2) features from different views are uncoupled with each other, which may mislead the multiview clustering methods to partition data into wrong clusters. To address these limitations, we propose a tensor approach for uncoupled multiview clustering (T-UMC) in this article. Instead of pairwise correlation, T-UMC chooses a most reliable view by view-specific silhouette coefficient (VSSC) at first, and then couples the self-representation matrix of each view with it by pairwise cross-view coupling learning. After that, by integrating recoupled self-representation matrices into a third-order tensor, the high-order correlations of all views are explored with tensor singular value decomposition (t-SVD)-based tensor nuclear norm (TNN). And the view-specific local structures of each individual view are also preserved with the local structure learning scheme with manifold learning. Besides, the physical meaning of view-specific coupling matrix is also discussed in this article. Extensive experiments on six commonly used benchmark datasets have demonstrated the superiority of the proposed method compared with the state-of-the-art multiview clustering methods.