Canonical Polyadic Research Articles

CP decomposition-based algorithms for completion problem of motion capture data

Motion capture (MoCap) technology is an essential tool for recording and analyzing movements of objects or humans. However, MoCap systems frequently encounter the challenge of missing data, stemming from mismatched markers, occlusion, or equipment limitations. Recovery of these missing data is imperative to maintain the reliability and integrity of MoCap recordings. This paper introduces a novel application of the tensor framework for MoCap data completion. We propose three completion algorithms based on the canonical polyadic (CP) decomposition of tensors. The first algorithm utilizes CP decomposition to capture the low-rank structure of the tensor. However, relying only on low-rank assumptions may be insufficient to deal with complex motion data. Thus, we propose two modified CP decompositions that incorporate additional information, SmoothCP and SparseCP decompositions. SmoothCP integrates piecewise smoothness prior, while SparseCP incorporates sparsity prior, each aiming to improve the accuracy and robustness of MoCap data recovery. To compare and evaluate the merit of the proposed algorithms over other tensor completion methods in terms of several evaluation metrics, we conduct numerical experiments with different MoCap sequences from the CMU motion capture dataset.

Tensor decomposition of EEG signals for transfer learning applications

ABSTRACT We address the recognized person-to-person Brain–Computer Interface (BCI) calibration problem and tackle session-dependency through the use of unsupervised canonical polyadic (CP) tensor decomposition. For a motor imagery task, the approach reveals universal structures within EEG data, common between subjects and prominent for a certain task. Further, we develop a novel similarity measure that includes weighting of the decomposition’s factor matrices, and argue that it is more representative than what has previously been presented in literature. The proposed similarity measure shows potential in a BCI classification task, i.e. drowsiness during simulated driving (average Pearson correlation of 0.6).

A canonical polyadic tensor basis for fast Bayesian estimation of multi-subject brain activation patterns.

Task-evoked functional magnetic resonance imaging studies, such as the Human Connectome Project (HCP), are a powerful tool for exploring how brain activity is influenced by cognitive tasks like memory retention, decision-making, and language processing. A fast Bayesian function-on-scalar model is proposed for estimating population-level activation maps linked to the working memory task. The model is based on the canonical polyadic (CP) tensor decomposition of coefficient maps obtained for each subject. This decomposition effectively yields a tensor basis capable of extracting both common features and subject-specific features from the coefficient maps. These subject-specific features, in turn, are modeled as a function of covariates of interest using a Bayesian model that accounts for the correlation of the CP-extracted features. The dimensionality reduction achieved with the tensor basis allows for a fast MCMC estimation of population-level activation maps. This model is applied to one hundred unrelated subjects from the HCP dataset, yielding significant insights into brain signatures associated with working memory.

Parametric channel estimation for RIS-assisted mmWave MIMO-OFDM systems with low pilot overhead

We consider the channel estimation problem in a millimeter-wave (mmWave) multiple-input multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) system aided by a passive reconfigurable intelligent surface (RIS). Initially, leveraging the inherent sparsity of mmWave channels, we decouple the channel estimation into the recovery of multipath parameters, including spatial frequencies of angles, delays, and complex gains. Subsequently, we employ two training schemes: the first involves a full row rank constrained RIS phase shift pattern, and the second incorporates a Kronecker structure constrained RIS phase shift pattern. These schemes reform channel estimation into estimating parameters from two fifth-order tensor signals, which admit constrained Canonical Polyadic decomposition. Moreover, both tensor signals share Vandermonde-constrained factor matrices, enabling algebraic algorithms for tensor decomposition problems and recovering channel multipath parameters. Theoretical analyses show that the first training scheme has a lower signal processing complexity for channel estimation, while the second attains a higher estimation accuracy. Also, we prove that our proposed two schemes have the same minimum pilot overhead, which is as low as proportional to the product of the number of paths in the two MIMO channels (transmitter-RIS and RIS-receiver). Numerical results validate the effectiveness of our proposed methods.

Sparse-View Spectral CT Reconstruction Based on Tensor Decomposition and Total Generalized Variation

Spectral computed tomography (CT)-reconstructed images often exhibit severe noise and artifacts, which compromise the practical application of spectral CT imaging technology. Methods that use tensor dictionary learning (TDL) have shown superior performance, but it is difficult to obtain a high-quality pre-trained global tensor dictionary in practice. In order to resolve this problem, this paper develops an algorithm called tensor decomposition with total generalized variation (TGV) for sparse-view spectral CT reconstruction. In the process of constructing tensor volumes, the proposed algorithm utilizes the non-local similarity feature of images to construct fourth-order tensor volumes and uses Canonical Polyadic (CP) tensor decomposition instead of pre-trained tensor dictionaries to further explore the inter-channel correlation of images. Simultaneously, introducing the TGV regularization term to characterize spatial sparsity features, the use of higher-order derivatives can better adapt to different image structures and noise levels. The proposed objective minimization model has been addressed using the split-Bregman algorithm. To assess the performance of the proposed algorithm, several numerical simulations and actual preclinical mice are studied. The final results demonstrate that the proposed algorithm has an enormous improvement in the quality of spectral CT images when compared to several existing competing algorithms.

Fake News Classification Using Tensor Decomposition and Graph Convolutional Network

With the widespread of fake news on social media, its impact has become a major concern of the public, so accurate detection methods are urgently needed. As content-based features are the most natural clues for fake news detection, many methods that rely solely on text content have been proposed. However, these methods rarely investigate the sentence interaction patterns of different news articles, and most of them do not consider fine-grained fake news classification. To overcome these issues, this article constructs a graph representation for news articles and employs a graph neural network (GNN) to classify fake news. The proposed method uses the local word co-occurrence information of sentences to obtain the interaction relationship between sentences, which is abstracted by the weight matrix of the graph representation. Accordingly, a third-order co-occurrence tensor is built, and the weight matrix is calculated based on the canonical polyadic (CP) decomposition of this tensor. Since our method considers the sentence interaction patterns, the computed representations can capture more accurate contextual information of news articles. The results on two real-world datasets demonstrate that our method outperforms the competing methods in both binary and multiclass classification tasks. In particular, for multiclass classification on the selected dataset with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$70$</tex-math> </inline-formula> % of the training set for training, the improvements of accuracy and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F1$</tex-math> </inline-formula> -score are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.82$</tex-math> </inline-formula> %p– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16.98$</tex-math> </inline-formula> %p and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.85$</tex-math> </inline-formula> %p– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16.65$</tex-math> </inline-formula> %p, respectively.

Spatiotemporal knowledge graph completion via diachronic and transregional word embedding

Knowledge Graph Completion (KGC) is an essential application in the field of knowledge graphs (KGs) that attempts to fill in the missing information in the process of KG modelling. With the popularity of temporal knowledge graphs (TKGs), a wide range of techniques based on temporal knowledge graph completion (TKGC) have appeared, solving the issue of real-world knowledge with temporal properties. However, there is little study on KGC with spatiotemporal attributes, some real-world data include both spatial and temporal attributes. Effectively handling the completion of missing entities or predicates in spatiotemporal knowledge graphs (STKGs) is an important challenge. Our study fills the gap in knowledge completion techniques in the field of STKG. We present a model for completion based on the well-known tensor factorization canonical polyadic (CP) decomposition. It introduces temporal and spatial attributes into the decomposition vector to achieve entity and link predictions. We name it diachronic and transregional word embedding (DT-WE), which includes two different modules: the embedding framework and the scoring module. Firstly, send the vectors to the embedding framework to get the new vector representation, then, we send it to the scoring module to compute, and finally, the resulting values are added together to compute the prediction probability. We conducted extensive experiments on three real-world STKGs: YAGO10K, Wikidata40K and Opensky. The results indicate that the newly introduced spatiotemporal attributes not only improve accuracy in predicting entities and predicates compared to temporal models but also achieve state-of-the-art performance with lower spatial complexity.

Temporal pattern-aware QoS prediction by Biased Non-negative Tucker Factorization of tensors

Dynamic quality of service (QoS) data contain rich temporal patterns of user-service interactions, which are vital for better understanding user behaviors and service conditions. Canonical polyadic (CP)-based latent factorization model has proven to be capable of capturing such patterns. However, it models the relations among latent features of user, service and time in a rigid and unnatural way, causing its failures in capturing the complex patterns when the target QoS data become massive. To address that issue, this paper proposes a Biased Non-negative Tucker Factorization of 3D tensors (BNTucF) model with the four-fold ideas: (1) utilizing a core tensor for modeling the complex interactions among latent features; (2) incorporating linear biases into the model for accurate descriptions on QoS fluctuation; (3) constraining the model to be non-negative for describing QoS non-negativity; (4) deducing a single latent factor-dependent, multiplicative updating scheme for training the model in an efficient density-oriented way. Empirical studies demonstrate that the proposed BNTucF can learn complex dynamic user-service interaction patterns more accurately, hence achieving accurate predictions on missing QoS data.

Stable Low-Rank CP Decomposition for Compression of Convolutional Neural Networks Based on Sensitivity

Modern convolutional neural networks (CNNs) play a crucial role in computer vision applications. The intricacy of the application scenarios and the growing dataset both significantly raise the complexity of CNNs. As a result, they are often overparameterized and have significant computational costs. One potential solution for optimizing and compressing the CNNs is to replace convolutional layers with low-rank tensor decomposition. The most suitable technique for this is Canonical Polyadic (CP) decomposition. However, there are two primary issues with CP decomposition that lead to a significant loss in accuracy. Firstly, the selection of tensor ranks for CP decomposition is an unsolved issue. Secondly, degeneracy and instability are common problems in the CP decomposition of contractional tensors, which makes fine-tuning the compressed model difficult. In this study, a novel approach was proposed for compressing CNNs by using CP decomposition. The first step involves using the sensitivity of convolutional layers to determine the tensor ranks for CP decomposition effectively. Subsequently, to address the degeneracy issue and enhance the stability of the CP decomposition, two novel techniques were incorporated: optimization with sensitivity constraints and iterative fine-tuning based on sensitivity order. Finally, the proposed method was examined on common CNN structures for image classification tasks and demonstrated that it provides stable performance and significantly fewer reductions in classification accuracy.

Machine learning with tree tensor networks, CP rank constraints, and tensor dropout.

Tensor networks developed in the context of condensed matter physics try to approximate order-N tensors with a reduced number of degrees of freedom that is only polynomial in N and arranged as a network of partially contracted smaller tensors. As we have recently demonstrated in the context of quantum many-body physics, computation costs can be further substantially reduced by imposing constraints on the canonical polyadic (CP) rank of the tensors in such networks. Here, we demonstrate how tree tensor networks (TTN) with CP rank constraints and tensor dropout can be used in machine learning. The approach is found to outperform other tensor-network-based methods in Fashion-MNIST image classification. A low-rank TTN classifier with branching ratio b = 4 reaches a test set accuracy of 90.3% with low computation costs. Consisting of mostly linear elements, tensor network classifiers avoid the vanishing gradient problem of deep neural networks. The CP rank constraints have additional advantages: The number of parameters can be decreased and tuned more freely to control overfitting, improve generalization properties, and reduce computation costs. They allow us to employ trees with large branching ratios, substantially improving the representation power.

An Approach for Using a Tensor-Based Method for Mobility-User Pattern Determining

Modern mobile networks exhibit a complex heterogeneous structure. To enhance the Quality of Service (QoS) in these networks, intelligent control mechanisms should be implemented. These functions are based on the processing of large amounts of data and feature extraction. One such feature is information about user mobility. However, directly determining user mobility remains challenging. To address this issue, this study proposes an approach based on multi-linear data processing. The user mobility is proposed to determine, using the multi-linear data, about the changing of the Signal-to-Interference-plus-Noise-Ratio (SINR). SINR varies individually for each user over time, relative to the network’s base stations. It is natural to represent these data as a tensor. A tensor-based preprocessing step employing Canonical Polyadic Decomposition (CPD) is proposed to extract user mobility information and reduce the data volume. In the next step, using the DBSCAN algorithm, users are clustered according to their mobility patterns. Subsequently, users are clustered based on their mobility patterns using the DBSCAN algorithm. The proposed approach is evaluated utilizing data from Network Simulator 3 (NS-3), which simulates a portion of the mobile network. The results of processing these data using the proposed method demonstrate superior performance in determining user mobility.

Performance optimization and parameters estimation for MIMO-OFDM dual-functional communication-radar systems

Dual-function communication radar systems use common Radio Frequency (RF) signals are used for both communication and detection. For better compatibility with existing communication systems, we adopt Multiple-Input Multiple-Output (MIMO) Orthogonal Frequency Division Multiplexing (OFDM) signals as integrated signals and investigate the estimation performance of MIMO-OFDM signals. First, we analyze the Cramer-Rao Lower Bound (CRLB) of parameter estimation. Then, the transmit powers over different subcarriers are optimized to achieve the best tradeoff between the transmission rate and the estimation performance. Finally, we propose a more accurate estimation method that uses Canonical Polyadic Decomposition (CPD) of the third-order tensor to obtain the parameter matrices. Due to the characteristic of the column structure of the parameter matrices, we only need to use DFT / IDFT to recover the parameters of multiple targets. The simulation results show that tensor-based estimation method can achieve a performance close to CRLB, and the estimation performance can be improved by optimizing the transmit powers.

3D-KCPNet: Efficient 3DCNNs based on tensor mapping theory

As the deep neural networks (DNNs) with satisfied expression ability usually require large scale to gain adequate performance, and deploying large-scale DNNs on resource-limited environments is still a challenge, neural network compression becomes a hot topic nowadays. Among the multiple compression methods, tensor decomposition reveals many specific advantages, such as regular data structure comes from linear algebra, convenient approach of training from scratch, ideal compression ratio, etc. Nevertheless, for some compact neural modules such as two-dimensional/three-dimensional (2D/3D) convolutional kernels, traditional tensor decomposition in the way of approximation has encountered intractable obstacles. Fortunately, some works utilize the tensor mapping approach, which just regards the data structure of tensor decomposition as neural layers, to reconstruct the convolutional kernel into a new lightweight module with several thinner convolutional kernels. The only two flies in the ointment are there lack necessary theories of tensor mapping, and there is still no tensor mapping way to compress high-order three-dimensional convolutional neural networks (3DCNNs). In this paper, we first deeply analyze the tensor mapping theory including convergence and precision, which separately establishes the rationality of tensor mapping and its superiority over the traditional tensor approximation, according to the Lottery Ticket Hypothesis. Then we propose an efficient method termed as 3D-KCPNet, to compress 3DCNNs based on the novel Kronecker canonical polyadic (KCP) tensor decomposition. The proposed method can not only compress the 3D convolutional kernels, but also convert a 3D convolution to efficient 1 × 1 × 1 and 2D depthwise convolutions. The experiments on the video recognition datasets VIVA Challenge, UCF11, UCF50, and UCF101 show that the accuracy of 3D-KCPNet can surpass its original baseline model and the corresponding tensor approximation model.

Creating High-Quality Synthetic Health Data: Framework for Model Development and Validation.

Electronic health records are a valuable source of patient information that must be properly deidentified before being shared with researchers. This process requires expertise and time. In addition, synthetic data have considerably reduced the restrictions on the use and sharing of real data, allowing researchers to access it more rapidly with far fewer privacy constraints. Therefore, there has been a growing interest in establishing a method to generate synthetic data that protects patients' privacy while properly reflecting the data. This study aims to develop and validate a model that generates valuable synthetic longitudinal health data while protecting the privacy of the patients whose data are collected. We investigated the best model for generating synthetic health data, with a focus on longitudinal observations. We developed a generative model that relies on the generalized canonical polyadic (GCP) tensor decomposition. This model also involves sampling from a latent factor matrix of GCP decomposition, which contains patient factors, using sequential decision trees, copula, and Hamiltonian Monte Carlo methods. We applied the proposed model to samples from the MIMIC-III (version 1.4) data set. Numerous analyses and experiments were conducted with different data structures and scenarios. We assessed the similarity between our synthetic data and the real data by conducting utility assessments. These assessments evaluate the structure and general patterns present in the data, such as dependency structure, descriptive statistics, and marginal distributions. Regarding privacy disclosure, our model preserves privacy by preventing the direct sharing of patient information and eliminating the one-to-one link between the observed and model tensor records. This was achieved by simulating and modeling a latent factor matrix of GCP decomposition associated with patients. The findings show that our model is a promising method for generating synthetic longitudinal health data that is similar enough to real data. It can preserve the utility and privacy of the original data while also handling various data structures and scenarios. In certain experiments, all simulation methods used in the model produced the same high level of performance. Our model is also capable of addressing the challenge of sampling patients from electronic health records. This means that we can simulate a variety of patients in the synthetic data set, which may differ in number from the patients in the original data. We have presented a generative model for producing synthetic longitudinal health data. The model is formulated by applying the GCP tensor decomposition. We have provided 3 approaches for the synthesis and simulation of a latent factor matrix following the process of factorization. In brief, we have reduced the challenge of synthesizing massive longitudinal health data to synthesizing a nonlongitudinal and significantly smaller data set.

A novel soft sensor method based on stacked fusion autoencoder with feature enhancement for industrial application

Soft sensor is pivotal in contemporary industrial processes. However, extracting effective feature representations from intricate process data remains a challenging task. To this end, a feature enhancement stacked fusion autoencoder (FE-SFAE) is proposed to leverage spatiotemporal characteristics and linear residual fusion for modeling. Firstly, the feature enhancement (FE) module that incorporates multidirectional delayed transform (MDT) and bilateral smoothing constraint canonical polyadic (BS-CP) decomposition is developed to alleviate the issues of time-lag and spatial dependence. Based on the FE module, complementary data can be generated for the subsequent stage. Secondly, to address the problem of variables being overly non-linearized, we propose a linear residual fusion gate mechanism. On this basis, we design the model named FE-SFAE that not only captures spatiotemporal features in complementary data but also balances the degree of network nonlinearization. Finally, the superiority of FE-SFAE is demonstrated through an industrial case involving the esterification process of polyester polymerization.

Optimizing Convolutional Neural Networks Utilizing Tensor Decomposition Techniques for Large-Scale Image Recognition Tasks

Convolutional Neural Networks (CNNs) are integral to numerous applications in today's digital landscape. However, their computational demands often result in slow operation, especially when resources are constrained. This study introduces two tensor decomposition methods aimed at optimizing the performance of CNNs by minimizing the total number of operations and weights, while maintaining accuracy. The first method employs Canonical Polyadic (CP) decomposition to divide the convolutional layer into multiple rank 1 tensors. The second method uses Tucker decomposition to break down the convolutional layer into a compact core tensor and several matrices. The effectiveness of these methods was evaluated on widely-used convolutional architectures, VGG16 and LeNet, by substituting their convolutional layers with a series of decomposed layers, implemented using PyTorch and TensorLy. The CP decomposition method achieved a computational speed-up of 43% with a minimal accuracy reduction of less than 0.12%. Similarly, Tucker decomposition resulted in a 37% speed-up with an accuracy decrease of less than 0.16%. These findings suggest that the proposed tensor decomposition methods can significantly enhance the efficiency of CNNs without significantly impacting their performance.

Exploring the feasibility of tensor decomposition for analysis of fNIRS signals: a comparative study with grand averaging method.

The analysis of functional near-infrared spectroscopy (fNIRS) signals has not kept pace with the increased use of fNIRS in the behavioral and brain sciences. The popular grand averaging method collapses the oxygenated hemoglobin data within a predefined time of interest window and across multiple channels within a region of interest, potentially leading to a loss of important temporal and spatial information. On the other hand, the tensor decomposition method can reveal patterns in the data without making prior assumptions of the hemodynamic response and without losing temporal and spatial information. The aim of the current study was to examine whether the tensor decomposition method could identify significant effects and novel patterns compared to the commonly used grand averaging method for fNIRS signal analysis. We used two infant fNIRS datasets and applied tensor decomposition (i.e., canonical polyadic and Tucker decompositions) to analyze the significant differences in the hemodynamic response patterns across conditions. The codes are publicly available on GitHub. Bayesian analyses were performed to understand interaction effects. The results from the tensor decomposition method replicated the findings from the grand averaging method and uncovered additional patterns not detected by the grand averaging method. Our findings demonstrate that tensor decomposition is a feasible alternative method for analyzing fNIRS signals, offering a more comprehensive understanding of the data and its underlying patterns.

Blockwise acceleration of alternating least squares for canonical tensor decomposition

AbstractThe canonical polyadic (CP) decomposition of tensors is one of the most important tensor decompositions. While the well‐known alternating least squares (ALS) algorithm is often considered the workhorse algorithm for computing the CP decomposition, it is known to suffer from slow convergence in many cases and various algorithms have been proposed to accelerate it. In this article, we propose a new accelerated ALS algorithm that accelerates ALS in a blockwise manner using a simple momentum‐based extrapolation technique and a random perturbation technique. Specifically, our algorithm updates one factor matrix (i.e., block) at a time, as in ALS, with each update consisting of a minimization step that directly reduces the reconstruction error, an extrapolation step that moves the factor matrix along the previous update direction, and a random perturbation step for breaking convergence bottlenecks. Our extrapolation strategy takes a simpler form than the state‐of‐the‐art extrapolation strategies and is easier to implement. Our algorithm has negligible computational overheads relative to ALS and is simple to apply. Empirically, our proposed algorithm shows strong performance as compared to the state‐of‐the‐art acceleration techniques on both simulated and real tensors.

Tracking online low-rank approximations of higher-order incomplete streaming tensors

In this paper, we propose two new provable algorithms for tracking online low-rank approximations of high-order streaming tensors with missing data. The first algorithm, dubbed adaptive Tucker decomposition (ATD), minimizes a weighted recursive least-squares cost function to obtain the tensor factors and the core tensor in an efficient way, thanks to an alternating minimization framework and a randomized sketching technique. Under the canonical polyadic (CP) model, the second algorithm, called ACP, is developed as a variant of ATD when the core tensor is imposed to be identity. Both algorithms are low-complexity tensor trackers that have fast convergence and low memory storage requirements. A unified convergence analysis is presented for ATD and ACP to justify their performance. Experiments indicate that the two proposed algorithms are capable of streaming tensor decomposition with competitive performance with respect to estimation accuracy and runtime on both synthetic and real data.

Computing vibrational energy levels using a canonical polyadic tensor method with a fixed rank and a contraction tree.

In this paper, we use the previously introduced Canonical Polyadic (CP)-Multiple Shift Block Inverse Iteration (MSBII) eigensolver [S. D. Kallullathil and T. Carrington, J. Chem. Phys. 155, 234105 (2021)] in conjunction with a contraction tree to compute vibrational spectra. The CP-MSBII eigensolver uses the CP format. The memory cost scales linearly with the number of coordinates. A tensor in CP format represents a wavefunction constrained to be a sum of products (SOP). An SOP wavefunction can be made more accurate by increasing the number of terms, the rank. When the required rank is large, the runtime of a calculation in CP format is long, although the memory cost is small. To make the method more efficient, we break the full problem into pieces using a contraction tree. The required rank for each of the sub-problems is small. To demonstrate the effectiveness of the ideas, we computed vibrational energy levels of acetonitrile (12-D) and ethylene oxide (15-D).