Third-order Tensor Research Articles

Multi-view Ensemble Clustering via Low-rank and Sparse Decomposition: From Matrix to Tensor

As a significant extension of classical clustering methods, ensemble clustering first generates multiple basic clusterings and then fuses them into one consensus partition by solving a problem concerning graph partition with respect to the co-association matrix. Although the collaborative cluster structure among basic clusterings can be well discovered by ensemble clustering, most advanced ensemble clustering utilizes the self-representation strategy with the constraint of low-rank to explore a shared consensus representation matrix in multiple views. However, they still encounter two challenges: (1) high computational cost caused by both the matrix inversion operation and singular value decomposition of large-scale square matrices; (2) less considerable attention on high-order correlation attributed to the pursue of the two-dimensional pair-wise relationship matrix. In this article, based on low-rank and sparse decomposition from both matrix and tensor perspectives, we propose two novel multi-view ensemble clustering methods, which tangibly decrease computational complexity. Specifically, our first method utilizes low-rank and sparse matrix decomposition to learn one common co-association matrix, while our last method constructs all co-association matrices into one third-order tensor to investigate the high-order correlation among multiple views by low-rank and sparse tensor decomposition. We adopt the alternating direction method of multipliers to solve two convex models by dividing them into several subproblems with closed-form solution. Experimental results on ten real-world datasets prove the effectiveness and efficiency of the proposed two multi-view ensemble clustering methods by comparing them with other advanced ensemble clustering methods.

GND-PCA Method for Identification of Gene Functions Involved in Asymmetric Division of C. elegans

Due to the rapid development of imaging technology, a large number of biological images have been obtained with three-dimensional (3D) spatial information, time information, and spectral information. Compared with the case of two-dimensional images, the framework for analyzing multidimensional bioimages has not been completely established yet. WDDD is an open biological image database. It dynamically records 3D developmental images of 186 samples of nematodes C. elegans. In this study, based on WDDD, we constructed a framework to analyze the multidimensional dataset, which includes image segmentation, image registration, size registration by the length of main axes, time registration by extracting key time points, and finally, using generalized N-dimensional principal component analysis (GND-PCA) to analyze the phenotypes of bioimages. As a data-driven technique, GND-PCA can automatically extract the important factors involved in the development of P1 and AB in C. elegans. A 3D bioimage can be regarded as a third-order tensor. Therefore, GND-PCA was applied to the set of third-order tensors, and a set of third-order tensor bases was iteratively learned to linearly approximate the set. For each tensor base, a corresponding characteristic image is built to reveal its geometric meaning. The results show that different bases can be used to express different vital factors in development, such as the asymmetric division within the two-cell stage of C. elegans. Based on selected bases, statistical models were built by 50 wild-type (WT) embryos in WDDD, and were applied to RNA interference (RNAi) embryos. The results of statistical testing demonstrated the effectiveness of this method.

Convergence analysis of iterative methods for computing the T-pseudoinverse of complete full-rank third-order tensors based on the T-product

This paper proposes an iterative approach for estimating the T-pseudoinverse of a third-order tensor A. The T-pseudoinverse A† is defined as a generalization of the classical pseudoinverse for matrices. In this work, we present an efficient iterative method to estimate A† based on an iterative formula derived from Li and Li’s work on matrices. Additionally, we employ the T-product as the tensor multiplication operation. This iterative method avoids the tedious task of computing the T-pseudoinverse using singular value decomposition. Firstly, we demonstrate that if A is an invertible tensor, the proposed iterative method, represented by the sequence {Xk}k=0∞, converges to the inverse tensor of A, for a suitable initial value. Furthermore, for a complete full-rank tensor A, we propose a novel iterative method based on the sequence {Xk}k=0∞, that converges to A†, given an appropriate initial value. Numerical experiments are presented to demonstrate the accuracy of the proposed method.

Quantization and application of low-rank tensor decomposition based on the deep learning model.

Watching the presentation of a large-scale network is very important for network state tracking, performance optimization, traffic engineering, anomaly detection, fault analysis, etc. In this paper, we try to develop deep learning technology to solve the defect problem of tensor filling based on inner product interaction. To solve the limitations of the existing tensor-filling algorithms, a new neural tensor-filling (NTC) model is proposed. NTC model can effectively type the third-order communication between data landscapes through outer creation operation. It creates the third-order interaction mapping tensor. On this basis, the interaction between local features of the 3D neural network is studied. In this paper, another fusion neural tensor filling (Fu NTC) model is proposed to solve the problem that the NTC model can only extract the nonlinear complex structural information between potential feature dimensions. In the framework of the neural network, the NTC model and tensor decomposition model share the same potential feature embedding. It can effectively extract nonlinear feature information and linear feature information at the same time. It achieves higher precision data recovery.

Classification of the Traffic Status Subcategory with ETC Gantry Data: An Improved Support Tensor Machine Approach

Accurate and reliable traffic state identification is the prerequisite for developing intelligent traffic programs. With the improvement of intelligent traffic control measures, the traffic state of some highways has gradually stabilized. The current research on traffic state identification needs to fully meet the highly informative intelligent traffic system and traffic state subcategory analysis. To fill the gap above, we propose an improved support tensor machine (STM) method based on self-training and multiclassification for traffic state subcategory identification (ISTM) with ETC gantry data. This paper takes the excellent application of the support vector machine (SVM) in traffic state identification as the starting point of method design and extends to the STM. The ETC gantry data are represented as a third-order tensor model. This paper utilizes the similarity among tensor samples to construct the kernel function and recognize the traffic states. We simplify STM calculation with a one-against-one model and a self-training idea. An optimal fit of the characteristics is supplied by maximizing inter-subcategory tensor block distances and minimizing intra-subcategory tensor block distances throughout a joint utilization of the STM and multiscale training theories. The experiment in this paper uses ETC gantry data from the Jingtai highway in Shandong Province, and the findings reveal that the ISTM has optimum values of 0.2578 and 0.3254 for the SumD and 0.1718 and 0.1901 for the DBI as compared to K-mean clustering and the SVM. The ISTM trains the traffic state subcategory classifiers with high accuracy and strong generalization ability.

Joint range, angle and polarization estimation in polarimetric FDA-MIMO radar based on Tucker tensor decomposition

Frequency diverse array multiple-input multiple-output (FDA-MIMO) radar is an emerging technology to offer range-angle-dependent beampattern. Polarimetric FDA-MIMO radar can sense additional polarization information to improve target identification capability. In this article, we investigate the problem of joint range, angle and polarization parameter estimation in a monostatic polarimetric FDA-MIMO radar with an FDA at transmitter and a cross-dipole array at receiver. Unlike the conventional methods in which the multidimensional data structure is rearranged into vectors or matrices by stacking operation, we propose a Tucker tensor decomposition-based scheme, which can reserve the original data structure and avoid spoiling the inherent characteristics of interest, especially when the number of snapshots is small. The third-order tensor model of the observed data is constructed. Two approaches named as Tucker covariance reconstruction and Tucker signal subspace are presented using the fourth-order covariance tensor decomposition. The Cramér–Rao bound for range, angle and polarization estimation is also provided. Numerical experiments demonstrate the superiorities of the proposed approaches. Specifically, two targets with identical range and close angles are effectively distinguished.

Context-Aware Point-of-Interest Recommendation Based on Similar User Clustering and Tensor Factorization

The rapid development of big data technology and mobile intelligent devices has led to the development of location-based social networks (LBSNs). To understand users’ behavioral patterns and improve the accuracy of location-based services, point-of-interest (POI) recommendation has become an important task. In contrast to the general task of product recommendation, POI recommendation faces the problems of the sparsity and weak semantics of user check-in data. To address these issues, an increasing number of studies have improved the accuracy of POI recommendations by introducing contextual information such as geographical, temporal, textual, and social relations. However, the rich context also brings great challenges to POI recommendation, such as the low utilization rate of context information, difficulty in balancing the richness of contextual information, and the complexity of the recommendation matrix. Considering that similar users have more interest preferences in common than users generally have, the check-in information of similar users has greater reference meaning. Thus, we propose a personalized POI recommendation method named CULT-TF, which incorporates similar users’ contextual information into the tensor factorization model. First, we present a user activity model and a user similarity model, which integrate contextual information to calculate the user activity and similarity between users. According to user activity, the most representative active users are selected as user clustering centers, and then users are clustered based on user similarity into several similar user clusters (C). Next, we construct a third-order tensor (user-location-time matrix) for each user cluster by using user activity, POI popularity, and time slot popularity as the eigenvalues in the user (U), location (L), and time (T) dimensions, and the eigenvalue of each dimension is modeled by integrating contextual information of users’ check-in behavior at the user, location, and time levels. Similar user clustering reduces the number of users in tensor modeling, reducing the U dimension. To further reduce the complexity of the recommendation matrix, the reduction of the L dimension is achieved through ROI (region of interest) clustering, and the reduction of the T dimension is achieved through time slot encoding. Then, we use tensor factorization (TF) to obtain the recommendation results. Our method decreases the complexity of the tensor matrix and integrates rich contextual information on users’ check-in behavior. Finally, we conducted a comprehensive performance evaluation of CULT-TF using real-world LBSN datasets from Brightkite. The experimental results show that our proposed method performs much better than other recommendation methods in terms of precision and recall.

Hyperspectral Anomaly Detection Based on Regularized Background Abundance Tensor Decomposition

The low spatial resolution of hyperspectral images means that existing mixed pixels rely heavily on spectral information, making it difficult to differentiate between the target of interest and the background. The endmember extraction method is powerful in enhancing spatial structure information for hyperspectral anomaly detection, whereas most approaches are based on matrix representation, which inevitably destroys the spatial or spectral information. In this paper, we treated the hyperspectral image as a third-order tensor and proposed a novel anomaly detection method based on a low-rank linear mixing model of the scene background. The obtained abundance maps possessed more distinctive features than the raw data, which was beneficial for identifying anomalies in the background. Specifically, the low-rank tensor background was approximated as the mode-3 product of an abundance tensor and endmember matrix. Due to the distinctive features of the background’s abundance maps, we characterized them by tensor regularization and imposed low rankness through CP decomposition, smoothness, and sparsity. In addition, we utilized the ℓ1,1,2-norm to characterize the tube-wise sparsity of the anomaly, since it accounted for a small portion of the scene. The experimental results obtained using five real datasets demonstrated the outstanding performance of our proposed method.

Exploring Neural Mechanisms of Reward Processing Using Coupled Matrix Tensor Factorization: A Simultaneous EEG-fMRI Investigation.

It is crucial to understand the neural feedback mechanisms and the cognitive decision-making of the brain during the processing of rewards. Here, we report the first attempt for a simultaneous electroencephalography (EEG)-functional magnetic resonance imaging (fMRI) study in a gambling task by utilizing tensor decomposition. First, the single-subject EEG data are represented as a third-order spectrogram tensor to extract frequency features. Next, the EEG and fMRI data are jointly decomposed into a superposition of multiple sources characterized by space-time-frequency profiles using coupled matrix tensor factorization (CMTF). Finally, graph-structured clustering is used to select the most appropriate model according to four quantitative indices. The results clearly show that not only are the regions of interest (ROIs) found in other literature activated, but also the olfactory cortex and fusiform gyrus which are usually ignored. It is found that regions including the orbitofrontal cortex and insula are activated for both winning and losing stimuli. Meanwhile, regions such as the superior orbital frontal gyrus and anterior cingulate cortex are activated upon winning stimuli, whereas the inferior frontal gyrus, cingulate cortex, and medial superior frontal gyrus are activated upon losing stimuli. This work sheds light on the reward-processing progress, provides a deeper understanding of brain function, and opens a new avenue in the investigation of neurovascular coupling via CMTF.

A Randomized Singular Value Decomposition for Third-Order Oriented Tensors

The oriented singular value decomposition (O-SVD) proposed by Zeng and Ng provides a hybrid approach to the t-product-based third-order tensor singular value decomposition with the transformation matrix being a factor matrix of the higher-order singular value decomposition. Continuing along this vein, this paper explores realizing the O-SVD efficiently by drawing a connection to the tensor-train rank-1 decomposition and gives a truncated O-SVD. Motivated by the success of probabilistic algorithms, we develop a randomized version of the O-SVD and present its detailed error analysis. The new algorithm has advantages in efficiency while keeping good accuracy compared with the current tensor decompositions. Our claims are supported by numerical experiments on several oriented tensors from real applications.

Patient specific higher order tensor based approach for the detection and localization of myocardial infarction using 12-lead ECG

Myocardial Infarction (MI) is an emergency condition that requires immediate medical treatment. The rapid and accurate diagnosis of MI using a 12-lead electrocardiogram (ECG) is extremely important in a clinical study to save the patient’s life. The manual interpretation of MI using a 12-lead ECG is tedious and time-consuming. Therefore, a patient-specific software-based computer-aided diagnosis framework is helpful to detect and localize MI disease accurately. This paper proposes a patient-specific higher-order tensor-based approach to detect and localize MI automatically using 12-lead ECG recordings. The 12-lead ECG recordings are segmented into 12-lead ECG beats using the multi-lead fusion-based QRS detection algorithm. The fast and adaptive multivariate empirical mode decomposition (FA-MVEMD) based multiscale analysis method decomposes 12-lead ECG beat into a third-order tensor containing the information from the samples, beat, and intrinsic mode functions (IMFs). Furthermore, a fourth-order tensor is formulated by considering beats, samples, lead, and IMFs information of 12-lead ECG recording. The multilinear singular value decomposition (MLSVD) extracts features from the fourth-order tensors and third-order tensors of 12-lead ECG. The K-nearest neighbor (KNN), support vector machine (SVM), and stacked autoencoder-based deep neural network (SAE-DNN) models are used for the detection and localization of MI using fourth-order and third-order tensor domain features. The proposed approach is evaluated using 73 healthy control (HC) and 100 different types of MI-based 12-lead ECG recordings from a public database. The proposed approach has obtained the classification accuracy values of (98.84%, 98.27%, 98.27%) and (86.64%, 83.17%, and 81.98%) using (KNN, SVM, and SAE-DNN) models for MI detection, and localization, respectively using 30-min duration of 12-lead ECG recordings. For MI detection and localization, the suggested approach has obtained accuracy values of 96.53% and 93.32%, respectively, using the 4-s duration of 12-lead ECG recordings. Our approach outperformed existing MI detection and localization methods using 12-lead ECG recordings regarding classification performance.

A Fixed Point Iterative Method for Third-order Tensor Linear Complementarity Problems

A Fixed Point Iterative Method for Third-order Tensor Linear Complementarity Problems

A Tensor-based domain alignment method for intelligent fault diagnosis of rolling bearing in rotating machinery

Fault diagnosis of rolling bearings plays a pivotal role in modern industry. Most existing methods have two disadvantages: 1) The assumption that the training and test data obey the same distribution; and 2) They are designed for vector representation which is unable to characterize the important structure of the rolling bearings data of interest. To overcome these drawbacks, this paper proposes a novel tensor based domain adaptation method. Firstly, this method uses the time domain signals, the frequency domain signals, and the Hilbert marginal spectrum and integrates them into a third-order tensor model. Secondly, these three types of signals are split into two parts: the source and target domain data; all the representative features are identified in the source domain. Thirdly, a tensor decomposition method is used to decompose the features into a series of third-order tensors, and several alignment matrices are defined to align the representation of the two domains to the tensor invariant subspace. Then, the alignment matrices and the tensor subspace are jointly optimized to realize the adaptive learning. Finally, the feature tensor is reconstructed into a matrix form to realize the fault diagnosis through the classifier. Extensive experiments are conducted on a public dataset and a dataset collected from our own laboratory; experimental results show the satisfactory performance of the proposed method.

High Performance Hierarchical Tucker Tensor Learning Using GPU Tensor Cores

Extracting information from large-scale high-dimensional data is a fundamentally important task in high performance computing, where the hierarchical Tucker (HT) tensor learning approach (learning a tensor-tree structure) has been widely used in many applications. However, HT tensor learning algorithms are compute-intensive due to the “ <i>curse of dimensionality</i> ,” i.e., the time complexity grows exponentially with the order of the data tensor. The computation of HT tensor learning algorithms boils down to tensor primitives, which are amenable to computing on GPU tensor cores. Existing work does not support HT tensor learning using GPU tensor cores. There are three main challenges to address: 1) to accelerate tensor learning primitives using GPU tensor cores; 2) to implement the tensor learning algorithms using GPU tensor cores and multiple GPUs; 3) to support large-scale data tensors exceeding the GPU memory capacity. In this paper, we present efficient HT tensor learning primitives using GPU tensor cores and demonstrate three applications. First, we utilize GPU tensor cores to optimize HT tensor learning primitives, including tensor contractions, tensor matricizations and tensor singular value decomposition (SVD). We employ the optimized primitives to optimize HT tensor decomposition algorithms for Big Data analysis. Second, we propose a novel HT tensor layer for deep neural networks, whose training process only involves a forward pass without back propagation. The forward pass consists of tensor operations, thus further exploiting the computing power of GPU tensor cores. Third, we apply the optimized primitives to develop a tensor-tree structured quantum machine learning algorithm <i>tree-tensor network (TTN)</i> . Compared with TensorLy and TensorNetwork on NVIDIA A100 GPUs, our third-order HT tensor decomposition algorithm achieves up to <inline-formula><tex-math notation="LaTeX">$8.92 \times$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$6.42 \times$</tex-math></inline-formula> speedups, respectively, and our high-order case achieves up to <inline-formula><tex-math notation="LaTeX">$32.67 \times$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$23.97 \times$</tex-math></inline-formula> speedups, respectively. Our HT tensor layer for a fully connected neural network achieves <inline-formula><tex-math notation="LaTeX">$49.2 \times$</tex-math></inline-formula> compression at the cost of 0.5% drops in accuracy and <inline-formula><tex-math notation="LaTeX">$1.42 \times$</tex-math></inline-formula> speedup compared with the implementation on CUDA cores; for the AlexNet, our HT tensor layer achieves <inline-formula><tex-math notation="LaTeX">$9.45 \times$</tex-math></inline-formula> compression at the cost of 0.8% drops in accuracy and <inline-formula><tex-math notation="LaTeX">$1.87 \times$</tex-math></inline-formula> speedup compared with the implementation on CUDA cores. Our TTN algorithm achieves up to <inline-formula><tex-math notation="LaTeX">$11.17\times$</tex-math></inline-formula> speedup compared with TensorNetwork, indicating the potential of optimized tensor learning primitives for the classical simulation of quantum machine learning algorithms.

Dictionary Learning With Low-Rank Coding Coefficients for Tensor Completion.

In this article, we propose a novel tensor learning and coding model for third-order data completion. The aim of our model is to learn a data-adaptive dictionary from given observations and determine the coding coefficients of third-order tensor tubes. In the completion process, we minimize the low-rankness of each tensor slice containing the coding coefficients. By comparison with the traditional predefined transform basis, the advantages of the proposed model are that: 1) the dictionary can be learned based on the given data observations so that the basis can be more adaptively and accurately constructed and 2) the low-rankness of the coding coefficients can allow the linear combination of dictionary features more effectively. Also we develop a multiblock proximal alternating minimization algorithm for solving such tensor learning and coding model and show that the sequence generated by the algorithm can globally converge to a critical point. Extensive experimental results for real datasets such as videos, hyperspectral images, and traffic data are reported to demonstrate these advantages and show that the performance of the proposed tensor learning and coding method is significantly better than the other tensor completion methods in terms of several evaluation metrics.

Effective Incomplete Multi-View Clustering via Low-Rank Graph Tensor Completion

In the past decade, multi-view clustering has received a lot of attention due to the popularity of multi-view data. However, not all samples can be observed from every view due to some unavoidable factors, resulting in the incomplete multi-view clustering (IMC) problem. Up until now, most efforts for the IMC problem have been made on the learning of consensus representations or graphs, while many missing views are ignored, making it impossible to capture the information hidden in the missing view. To overcome this drawback, we first analyzed the low-rank relationship existing inside each graph and among all graphs, and then propose a novel method for the IMC problem via low-rank graph tensor completion. Specifically, we first stack all similarity graphs into a third-order graph tensor and then exploit the low-rank relationship from each mode using the matrix nuclear norm. In this way, the connection hidden between the missing and available instances can be recovered. The consensus representation can be learned from all completed graphs via multi-view spectral clustering. To obtain the optimal multi-view clustering result, incomplete graph recovery and consensus representation learning are integrated into a joint framework for optimization. Extensive experimental results on several incomplete multi-view datasets demonstrate that the proposed method can obtain a better clustering performance in comparison with state-of-the-art incomplete multi-view clustering methods.

Smooth Crack Band Model—A Computational Paragon Based on Unorthodox Continuum Homogenization

Abstract The crack band model, which was shown to provide a superior computational representation of fracture of quasibrittle materials (in this journal, May 2022), still suffers from three limitations: (1) The material damage is forced to be uniform across a one-element wide band because of unrestricted strain localization instability; (2) the width of the fracture process zone is fixed as the width of a single element; and (3) cracks inclined to rectangular mesh lines are represented by a rough zig-zag damage band. Presented is a generalization that overcomes all three, by enforcing a variable multi-element width of the crack band front controlled by a material characteristic length l0. This is achieved by introducing a homogenized localization energy density that increases, after a certain threshold, as a function of an invariant of the third-order tensor of second gradient of the displacement vector, called the sprain tensorη, representing (in isotropic materials) the magnitude of its Laplacian (not expressible as a strain-gradient tensor). The continuum free energy density must be augmented by additional sprain energy Φ(l0η), which affects only the postpeak softening damage. In finite element discretization, the localization resistance is effected by applying triplets of self-equilibrated in-plane nodal forces, which follow as partial derivatives of Φ(l0η). The force triplets enforce a variable multi-element crack band width. The damage distribution across the fracture process zone is non-uniform but smoothed. The standard boundary conditions of the finite element method apply. Numerical simulations document that the crack band propagates through regular rectangular meshes with virtually no directional bias.

Approximation strategy based on the T-product for third-order quaternion tensors with application to color video compression

Following up the ideas of T-product algebra for third-order real tensors, this paper aims to establish an approximation strategy based on the T-product for third-order quaternion tensors. First, we constructively prove the existence of the tensor-quaternion singular value decomposition (t-QSVD) for third-mode symmetric quaternion tensors, and provide a way to compute the t-QSVD via fast Fourier transforms. Then further, we propose a T-product based compression strategy for any given third-order quaternion tensor. Last, the effectiveness of the T-product based compression strategy is demonstrated by numerical simulations on color videos.

Transformed singular values and spectral norms of third-order tensors

The tensor-tensor product (T-product) and tensor singular value decomposition (T-SVD) for third-order tensors, which are based upon the general revertible linear transformation, are widely used in engineering and data science. This paper focuses on studying singular values and spectral norms of third-order tensors under unitary transformation. After presenting the decomposition for the third-order Hermitian tensors and the QR factorization of third-order tensors, with the help of the involved invertible linear transformation, several basic properties of singular values for third-order tensors are studied, which include the Lipschitz continuity that characterizes the singular values perturbation. Based upon these, two kinds of spectral norms for third-order tensors are introduced, one of which is the norm induced by Frobenius norm of tensors in the sense of the T-product.

Bi-Nuclear Tensor Schatten-p Norm Minimization for Multi-View Subspace Clustering.

Multi-view subspace clustering aims to integrate the complementary information contained in different views to facilitate data representation. Currently, low-rank representation (LRR) serves as a benchmark method. However, we observe that these LRR-based methods would suffer from two issues: limited clustering performance and high computational cost since (1) they usually adopt the nuclear norm with biased estimation to explore the low-rank structures; (2) the singular value decomposition of large-scale matrices is inevitably involved. Moreover, LRR may not achieve low-rank properties in both intra-views and inter-views simultaneously. To address the above issues, this paper proposes the Bi-nuclear tensor Schatten- p norm minimization for multi-view subspace clustering (BTMSC). Specifically, BTMSC constructs a third-order tensor from the view dimension to explore the high-order correlation and the subspace structures of multi-view features. The Bi-Nuclear Quasi-Norm (BiN) factorization form of the Schatten- p norm is utilized to factorize the third-order tensor as the product of two small-scale third-order tensors, which not only captures the low-rank property of the third-order tensor but also improves the computational efficiency. Finally, an efficient alternating optimization algorithm is designed to solve the BTMSC model. Extensive experiments with ten datasets of texts and images illustrate the performance superiority of the proposed BTMSC method over state-of-the-art methods.