Canonical Polyadic Tensor Decomposition Research Articles

LINK PREDICTION USING TENSOR DECOMPOSITION

In recent years, tensor decomposition has gained increasing interest in the field of link prediction, which aims to estimate the likelihood of new connections forming between nodes in a network. This study highlights the potential of the Canonical Polyadic tensor decomposition in enhancing link prediction in complex networks. It suggests effective tensor decomposition algorithms that not only take into account the structural characteristics of the network but also its temporal evolution. During the process of tensor decomposition, the initial tensor is decomposed into two-way tensors, also known as factor matrices, representing different modes of the data. These factor matrices capture the underlying patterns or relationships within the network, providing insights into the structure and dynamics of the network. For evaluation, we examine a dataset derived from the WSDM. After preprocessing, the data is represented as a multi-way tensor, with each mode representing different aspects such as users, items, and time. Our primary objective is to make precise predictions about the links between users and items within specific time periods. The experimental results demonstrate that our approach significantly improves prediction accuracy for evolving networks, as measured by the AUC.

Accelerating probabilistic tensor canonical polyadic decomposition with nonnegative factors: An inexact BCD Approach

Recently, Bayesian modeling and variational inference (VI) were leveraged to enable the nonnegative factor matrix learning with automatic rank determination in tensor canonical polyadic decomposition (CPD), which has found various applications in big data analytics. However, since VI inherently performs block coordinate descent (BCD) steps over the functional space, it generally does not allow integration with modern large-scale optimization methods, making the scalability a critical issue. In this paper, it is revealed that the expectations of the variables updated by the VI algorithm is equivalent to the block minimization steps of a deterministic optimization problem. This equivalence further enables the adoption of inexact BCD method for devising a fast nonnegative factor matrix learning algorithm with automatic tensor rank determination. Numerical results using synthetic data and real-world applications show that the performance of the proposed algorithm is comparable with that of the VI-based algorithm, but with computation times reduced significantly.

Multi-level optimization of the canonical polyadic tensor decomposition at large-scale: Application to the stratification of social networks through deflation

Tensors are multi-dimensional mathematical objects that allow to model complex relationships and to perform decompositions for analytical purpose. They are used in a wide range of data mining applications. In social network analysis, tensor decompositions give interesting insights by taking into consideration multiple characteristics of data. However, the power-law distribution of such data forces the decomposition to reveal only the strong signals that hide information of interest having a lighter intensity. To reveal hidden information, we propose a method to stratify the signal, by gathering clusters of similar intensity in each stratum. It is an iterative process, in which the CANDECOMP/PARAFAC (CP) decomposition is applied and its result is used to deflate the tensor, i.e., by removing from the tensor the clusters found with the decomposition. As the CP decomposition is computationally demanding, it is also necessary to optimize its algorithm, to apply it on large-scale data with a reasonable execution time, even with the several executions needed by the iterative process of the stratification. Therefore, we propose an algorithm that uses both dense and sparse data structures and that leverages coarse and fine grained optimizations in addition to incremental computations in order to achieve large scale CP tensor decomposition. Our implementation outperforms the baseline of large-scale CP decomposition libraries by several orders of magnitude. We validate our stratification method and our optimized algorithm on a Twitter dataset about COVID vaccines.

Practical Leverage-Based Sampling for Low-Rank Tensor Decomposition

The low-rank canonical polyadic tensor decomposition is useful in data analysis and can be computed by solving a sequence of overdetermined least squares subproblems. Motivated by consideration of sparse tensors, we propose sketching each subproblem using leverage scores to select a subset of the rows, with probabilistic guarantees on the solution accuracy. We randomly sample rows proportional to leverage score upper bounds that can be efficiently computed using the special Khatri--Rao subproblem structure inherent in tensor decomposition. Crucially, for a $(d+1)$-way tensor, the number of rows in the sketched system is $O(r^d/\epsilon)$ for a decomposition of rank $r$ and $\epsilon$-accuracy in the least squares solve, independent of both the size and the number of nonzeros in the tensor. Along the way, we provide a practical solution to the generic matrix sketching problem of sampling overabundance for high-leverage-score rows, proposing to include such rows deterministically and combine repeated samples in the sketched system; we conjecture that this can lead to improved theoretical bounds. Numerical results on real-world large-scale tensors show the method is significantly faster than deterministic methods at nearly the same level of accuracy.

Fast CP‐compression layer: Tensor CP‐decomposition to compress layers in deep learning

Deep neural network (DNN) shows its powerful performance in terms of image classification and many other applications. However, as the number of network layers increases, it brings huge pressure on devices with limited resources. In this article, a novel network compression algorithm is proposed that compresses the original network by up to about 60 times. In particular, a tensor Canonical Polyadic(CP) decomposition based algorithm is proposed to compress the weight matrix in the fully connected(FC) layer and the convolution kernel in the convolution layer. Traditional tensor decomposition algorithms are usually to first pre-train the weights, and decompose the weights, finally perform fine-tuning on the factors in the second training phase. Instead, the decomposed factors are directly updated by performing tensor CP decomposition on weight without fine-tuning. The proposed algorithm is called Fast CP-Compression Layer method in this paper. Experiments show that the proposed algorithm cannot only reduce computing time and improve compression factor but also improve accuracy on some datasets.

Towards Flexible Sparsity-Aware Modeling: Automatic Tensor Rank Learning Using the Generalized Hyperbolic Prior

Tensor rank learning for canonical polyadic decomposition (CPD) has long been deemed as an essential yet challenging problem. In particular, since thetensor rank controls the complexity of the CPD model, its inaccurate learning would cause overfitting to noise or underfitting to the signal sources, and even destroy the interpretability of model parameters. However, the optimal determination of a tensor rank is known to be a non-deterministic polynomial-time hard (NP-hard) task. Rather than exhaustively searching for the best tensor rank via trial-and-error experiments, Bayesian inference under the Gaussian-gamma prior was introduced in the context of probabilistic CPD modeling, and it was shown to be an effective strategy for automatic tensor rank determination. This triggered flourishing research on other structured tensor CPDs with automatic tensor rank learning. On the other side of the coin, these research works also reveal that the Gaussian-gamma model does not perform well for high-rank tensors and/or low signal-to-noise ratios (SNRs). To overcome these drawbacks, in this paper, we introduce a more advanced generalized hyperbolic (GH) prior to the probabilistic CPD model, which not only includes the Gaussian-gamma model as a special case, but also is more flexible to adapt to different levels of sparsity. Based on this novel probabilistic model, an algorithm is developed under the framework of variational inference, where each update is obtained in a closed-form. Extensive numerical results, using synthetic data and real-world datasets, demonstrate the significantly improved performance of the proposed method in learning both low as well as high tensor ranks even for low SNR cases.

Tensor-based approach for blind separation of interleave-NOMA 5G system

Nowadays, wireless broadband communication has experienced tremendous growth. The requirement for various new and advanced multiple access techniques is constantly increasing. The interleave division multiple access (IDMA) technique has attracted the attention of the scientific community as an opportune code domain non-orthogonal multiple access (NOMA) for future communication standards. In this system, the detection of the signals received from different users is done by specific interleavers.In this paper, and regarding the transmission-rate loss caused by the use of spreading codes and interleavers in the IDMA systems, we are going to place ourselves in a blind separation context for IDMA signals. For this purpose, a study of a purely deterministic approach, which is the tensor decomposition, as well as a theoretical study of the Canonical Polyadic (CP) model’s existence and uniqueness issues, is also carried out. Then, a tensor modeling of the IDMA emitter and the proposal of a new form of the interleavers will be undertaken. The separation and equalization of received signals using the CP tensor decomposition compelled us to examine the methods utilized to carry out this decomposition, as well as suggest two iterative blind receivers that take use of the algebraic features of the suggested model. On the other hand, the application of an exact performance criterion will be performed, taken into account the scale and permutation indeterminacy constraints.

Dynamic spectrum cartography via canonical polyadic tensor decomposition

Spectrum cartography aims at estimating multidimensional radio map from limited geographical observations. Recovering unobserved elements in a radio map, from sequential spectrum observations over a long period in large geographic areas, poses major challenges to real-time requirement and storage. As historical information from dynamic spectrum observations is not well utilized, traditional spectrum cartography methods, which are statically applied on single or multiple time slots, are not well suitable for time-varying scenarios. To address these challenges, we propose two novel radio map reconstruction algorithms based on tensor canonical polyadic decomposition (CPD), called dynamic window size based CPD (DW-CPD) and incremental CPD (I-CPD). The dynamic window size of DW-CPD is derived based on Kruskal condition of tensor CPD, and I-CPD is derived based on an exponentially weighted least squares criterion. We prove that the solution of I-CPD converges to a stationary point of the tensor completion problem, which validates that the proposed I-CPD is suitable for applications in estimating dynamic CP factor. Simulations results show that DW-CPD and I-CPD based algorithms have similar estimation performance for dynamic spectrum cartography, and are better than the existing methods. Due to the overall advantage of performance, running time and storage, I-CPD based algorithm is preferable for real-time applications.

Computation of low-rank tensor approximation under existence constraint via a forward-backward algorithm

The Canonical Polyadic (CP) tensor decomposition has become an attractive mathematical tool in several fields during the last ten years. This decomposition is very powerful for representing and analyzing multidimensional data. The most attractive feature of the CP decomposition is its uniqueness, contrary to rank-revealing matrix decompositions, where the problem of rotational invariance remains. This paper presents the performance analysis of iterative descent algorithms for calculating the CP decomposition of tensors when columns of factor matrices are almost collinear – i.e. swamp problems arise. We propose in this paper a new and efficient proximal algorithm based on the Forward Backward splitting method. More precisely, the existence of the best low-rank tensor approximation is ensured thanks to a coherence constraint implemented via a logarithmic regularized barrier. Computer experiments demonstrate the efficiency and stability of the proposed algorithm in comparison to other iterative algorithms in the literature for the normal case, and also producing significant results even in difficult situations.

Dynamic brain effective connectivity analysis based on low-rank canonical polyadic decomposition: application to epilepsy.

In this paper, a new method to track brain effective connectivity networks in the context of epilepsy is proposed. It relies on the combination of partial directed coherence with a constrained low-rank canonical polyadic tensor decomposition. With such combination being established, the most dominating directed graph structures underlying each time window of intracerebral electroencephalographic signals are optimally inferred. Obtained time and frequency signatures of inferred brain networks allow respectively to track the time evolution of these networks and to define frequency bands on which they are operating. Besides, the proposed method allows also to track brain connectivity networks through several epileptic seizures of the same patient. Understanding the most dominating directed graph structures over epileptic seizures and investigating their behavior over time and frequency plans are henceforth possible. Since only few but the the most important directed connections in the graph structure are of interest and also for a meaningful interpretation of obtained signatures to be guaranteed, the low-rank canonical polyadic tensor decomposition is prompted respectively by the sparsity and the non-negativity constraints on the tensor loading matrices. The main objective of this contribution is to propose a new way of tracking brain networks in the context of epileptic iEEG data by identifying the most dominant effective connectivity patterns underlying the observed iEEG signals at each time window. The performance of the proposed method is firstly evaluated on simulated data imitating brain activities and secondly on real intracerebral electroencephalographic signals obtained from an epileptic patient. The partial directed coherence-based tensor has been decomposed into space, time, and frequency signatures in accordance with the expected ground truth for each consecutive sequence of the simulated data. The method is also in accordance with the clinical expertise for iEEG epileptic signals, where the signatures were investigated through a simultaneous multi-seizure analysis.

Robust brain network identification from multi-subject asynchronous fMRI data

We describe a novel method for robust identification of common brain networks and their corresponding temporal dynamics across subjects from asynchronous functional MRI (fMRI) using tensor decomposition. We first temporally align asynchronous fMRI data using the orthogonal BrainSync transform, allowing us to study common brain networks across sessions and subjects. We then map the synchronized fMRI data into a 3D tensor (vertices × time × subject/session). Finally, we apply Nesterov-accelerated adaptive moment estimation (Nadam) within a scalable and robust sequential Canonical Polyadic (CP) decomposition framework to identify a low rank tensor approximation to the data. As a result of CP tensor decomposition, we successfully identified twelve known brain networks with their corresponding temporal dynamics from 40 subjects using the Human Connectome Project's language task fMRI data without any prior information regarding the specific task designs. Seven of these networks show distinct subjects’ responses to the language task with differing temporal dynamics; two show sub-components of the default mode network that exhibit deactivation during the tasks; the remaining three components reflect non-task-related activities. We compare results to those found using group independent component analysis (ICA) and canonical ICA. Bootstrap analysis demonstrates increased robustness of networks found using the CP tensor approach relative to ICA-based methods.

Respiration Monitoring With RFID in Driving Environments

To improve driving safety and avoid accidents caused by driving fatigue, drowsiness detection aims to alarm the driver before he/she falls asleep. Since breathing rate is a key indicator of the drowsy state, respiration monitoring in the noisy driving environment is critical for developing an effective driving fatigue detection system. In this paper, we propose, for the first time, an RFID based respiration monitoring system for driving environments. The system estimates the respiration rate of a driver based on phase values sampled from multiple RFID tags attached to the seat belt, while exploiting the tag diversity to combat the strong noise in the driving environment. Both tensor completion and tensor Canonical Polyadic Decomposition (CPD) are applied to process the phase values, to overcome the influence of frequency hopping, random sampling, vehicle vibration, and other environmental movements. The proposed system is analyzed and implemented with commodity RFID devices. Its accurate and robust performance is demonstrated with extensive experiments conducted in a real driving car.

Reduced-order modelling of equations of state using tensor decomposition for robust, accurate and efficient property calculation in high-pressure fluid flow simulations

Computationally efficient, accurate and robust Computational Fluid Dynamics (CFD) simulations involving thermodynamic properties from Equations of State (EOS) are hindered by limitations dictated by coupling strategies between EOS and CFD codes. This is a key aspect for a wide range of Chemical Engineering designs with special emphasis on those involving transcritical flows. We introduce a ROM approach based on a non-structured and sparse implementation of the Canonical Polyadic Decomposition of tensors that target abovementioned requirements. It reaches a similar speed with regards to direct use of the full equation of state and provides mean errors about 1 %–5 % without limiting accuracy. Its implementation is done in a standard and portable way, avoiding the need of additional implementation and an easy coupling with open and commercial CFD codes. The method is tested here for CFD but it can be directly applied in any process simulation tool.

Using the proximal gradient and the accelerated proximal gradient as a canonical polyadic tensor decomposition algorithms in difficult situations

Canonical Polyadic (CP) tensor decomposition is useful in many real-world applications due to its uniqueness, and the ease of interpretation of its factor matrices. This work addresses the problem of calculating the CP decomposition of tensors in difficult cases where the factor matrices in one or all modes are almost collinear – i.e. bottleneck or swamp problems arise. This is done by introducing a constraint on the coherences of the factor matrices that ensures the existence of a best low-rank approximation, which makes it possible to estimate these highly correlated factors. Two new algorithms optimizing the CP decomposition based on proximal methods are proposed. Simulation results are provided and demonstrate the good behaviour of these algorithms, as well as a better compromise between accuracy and convergence speed than other algorithms in the literature.

Learning Nonnegative Factors From Tensor Data: Probabilistic Modeling and Inference Algorithm

Tensor canonical polyadic decomposition (CPD) with nonnegative factor matrices, which extracts useful latent information from multidimensional data, has found wide-spread applications in various big data analytic tasks. Currently, the implementation of most existing algorithms needs the knowledge of tensor rank. However, this information is practically unknown and difficult to acquire. To address this issue, a probabilistic approach is taken in this paper. Different from previous works, this paper firstly introduces a sparsity-promoting nonnegative Gaussian-gamma prior, based on which a novel probabilistic model for the CPD problem with nonnegative and continuous factors is established. This probabilistic model further enables the derivation of an efficient inference algorithm that accurately learns the nonnegative factors from the tensor data, along with an integrated feature of automatic rank determination. Numerical results using synthetic data and real-world applications are presented to show the remarkable performance of the proposed algorithm.

Research Spotlights

Tensors, or multiway arrays, are often used for storage of high-dimensional data. In order to have compression, completion, or interpretation of such data, the data tensor is factored according to a proposed model consistent with some belief about the data. The first Research Spotlights article, “Generalized Canonical Polyadic Tensor Decomposition," by David Hong, Tamara G. Kolda, and Jed A. Duersch, treats the problem of finding a low-rank tensor decomposition that minimizes a cost functional defined from a componentwise loss function. The difference between the proposed generalized canonical polyadic (GCP) decomposition and the well-known CP decomposition is the flexibility in the choice of loss function, which can be tailored to reflect the appropriate statistical likelihood of a model given the data: low rankness of the tensor is enforced as a constraint. The main result of the paper is in Theorem 3, which addresses the computation of the gradients of a cost function defined in terms of any loss function that is continuously differentiable with respect to its second argument. Their result reveals an elegant expression for the gradients that enables the use of existing kernels for finding CP decompositions to be employed in their algorithm to compute the GCP. The authors use extensive numerical testing, highlighting the flexibility of their algorithm and its ability to efficiently handle missing data while simultaneously illustrating the potential gain to the data analyst in choosing loss functions more consistent with the data. A variety of applications are considered, including analysis of a chat network and explaining factors for mouse neural activity. A “sticky” diffusion process, as defined in the second Research Spotlights article, “Sticky Brownian Motion and Its Numerical Solution," refers to solutions to stochastic differential equations (SDEs) that can spend finite time on a lower-dimensional boundary. Materials like concrete, paint, and toothpaste have something in common: in the limit, the dynamics of the collection of mesoscale particles of which each is comprised approaches a sticky diffusion process. Sticky Brownian motion (SBM), the simplest example of a sticky diffusion process, is the primary focus of the present work. The goals of authors Nawaf Bou-Rabee and Miranda C. Holmes-Cerfon are to call the attention of the greater applied mathematics community to the topic of sticky diffusions as well as to offer a numerical method to allow for simulation of a sticky diffusion with a relatively large time step. These goals are evident in the presentation of material. The mathematical setup of an SBM is given in section 2. Beginning with the Brownian dynamics equations and assumptions on the potential energy function, the dynamics of the probability density are discussed, including asymptotic behavior in the “sticky limit." The authors discuss how to find the generator of the SBM and give some examples. The generator is the basis for the numerical method they develop in section 3, which the authors claim is orders of magnitude faster than alternative methods to simulate a sticky Brownian motion. The authors complement their presentation with a mathematical literature review of some additional methods for characterizing SBM. The final section serves as a well-considered road map for the reader interested in extending this work to a higher-dimensional setting.

On accelerated methods for tensor canonical polyadic decomposition

On accelerated methods for tensor canonical polyadic decomposition

Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion

Hyperspectral (HS) super-resolution, which aims at enhancing the spatial resolution of hyperspectral images (HSIs), has recently attracted considerable attention. A common way of HS super-resolution is to fuse the HSI with a higher spatial-resolution multispectral image (MSI). Various approaches have been proposed to solve this problem by establishing the degradation model of low spatial-resolution HSIs and MSIs based on matrix factorization methods, e.g., unmixing and sparse representation. However, this category of approaches cannot well construct the relationship between the high-spatial-resolution (HR) HSI and MSI. In fact, since the HSI and the MSI capture the same scene, these two image sources must have common factors. In this paper, a nonlocal tensor decomposition model for hyperspectral and multispectral image fusion (HSI-MSI fusion) is proposed. First, the nonlocal similar patch tensors of the HSI are constructed according to the MSI for the purpose of calculating the smooth order of all the patches for clustering. Then, the relationship between the HR HSI and the MSI is explored through coupled tensor canonical polyadic (CP) decomposition. The fundamental idea of the proposed model is that the factor matrices in the CP decomposition of the HR HSI’s nonlocal tensor can be shared with the matrices factorized by the MSI’s nonlocal tensor. Alternating direction method of multipliers is used to solve the proposed model. Through this method, the spatial structure of the MSI can be successfully transferred to the HSI. Experimental results on three synthetic data sets and one real data set suggest that the proposed method substantially outperforms the existing state-of-the-art HSI-MSI fusion methods.

DOA and Polarization Parameters Estimation by Exploiting Canonical Polyadic Decomposition of Tensors

A new algorithm to estimate the direction of arrival (DOA) and polarization parameters of signals impinging on an array with electromagnetic (EM) vector-sensors is presented by exploiting the canonical polyadic decomposition (CPD) of tensors. In addition to spatial and temporal diversities, further information from the polarization domain is considered and used in this paper. Estimation errors of these parameters are evaluated by the Cramér-Rao lower bound (CRB) benchmark, in the presence of additive white Gaussian noise (AWGN). The superiority of the proposed algorithm is shown by comparing with the derivative algorithms of MUSIC and ESPRIT. In the proposed algorithm, the parameters can be estimated by virtue of the diversities of the spatial and polarization belonging to the factor matrices, rather than the conventional subspace which is the foundation of MUSIC and ESPRIT. Additionally, the classical CPD algorithm based on Alternating Least Squares (ALS) is introduced to verify the efficacy of the proposed CPD algorithm. Results demonstrate that when the number of snapshots is greater than 50, the proposed algorithm requires a smaller number of snapshots to achieve a high level of performance, compared against the subspace-based algorithms and the ALS-based algorithm. Furthermore, in the matter of the array with a small number of sensors, the discovered advantage concerning the Root Mean Square Error (RMSE) in estimating the DOA and the polarization state of the signal is noteworthy.

Fiber Sampling Approach to Canonical Polyadic Decomposition and Application to Tensor Completion

Tensor decompositions play an important role in a variety of applications, such as signal processing and machine learning. In practice, the tensor can be incomplete or very large, making it difficult to analyze and process using conventional tensor techniques. In this paper we focus on the basic canonical polyadic decomposition (CPD). We propose an algebraic framework for finding the CPD of tensors that have missing fibers. This includes extensions of multilinear algebraic as well as generic uniqueness conditions originally developed for the CPD of fully observed tensors. Computationally, we reduce the CPD of a tensor with missing fibers to relatively simple matrix completion problems via a matrix eigenvalue decomposition (EVD). Under the given conditions, the EVD-based algorithm is guaranteed to return the exact CPD. The derivation establishes connections with so-called coupled CPDs, an emerging concept that has proven to be of great interest in a range of array processing and signal processing applications. It will become clear that relatively few fibers are needed in order to compute the CPD. This makes fiber sampling interesting for large scale tensor decompositions. Numerical experiments show that the algebraic framework may significantly speed up more common optimization-based computation schemes for the estimation of the CPD of incomplete noisy data tensors.