Canonical Polyadic Decomposition Research Articles

Robust coupled tensor decomposition and feature extraction for multimodal medical data

High-dimensional and multimodal data to describe various aspects of a patient’s clinical condition have become increasingly abundant in the medical field across a variety of domains. For example, in neuroimaging applications, electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) can be collected simultaneously (i.e., EEG-fMRI) to provide high spatial and temporal resolution of a patient’s brain function. Additionally, in telemonitoring applications, a smartphone can be used to record various aspects of a patient’s condition using its built-in microphone, accelerometer, touch screen, etc. Coupled CANDECOMP/PARAFAC decomposition (CCPD) is a powerful approach to simultaneously extract common structures and features from multiple tensors and can be applied to these high-dimensional, multi-modal data. However, the existing CCPD models are inadequate to handle outliers, which are highly present in both applications. For EEG-fMRI, outliers are common due to fluctuations in the electromagnetic field resulting from interference between the EEG electrodes and the fMRI machine. For telemonitoring, outliers can result from patients not properly following instructions while performing smartphone-guided exercises at home. This motivates us to propose a robust CCPD (RCCPD) method for robust feature extraction. The proposed method utilizes the Alternating Direction Method of Multipliers (ADMM) to minimize an objective function that simultaneously decomposes a pair of coupled tensors and isolates outliers. We compare the proposed RCCPD method with the classical CP decomposition, the coupled matrix-tensor/tensor-tensor factorization (CMTF/CTTF), and the tensor robust CP decomposition (TRCPD). Experiments on both synthetic and real-world data demonstrate that the proposed RCCPD effectively handles outliers and outperforms the benchmarks in terms of accuracy.

Bayesian robust tensor completion via CP decomposition

The real-world tensor data are inevitably missing and corrupted with noise. Some models of the low-rank tensor factorization (LRTF) add an L1 norm or L2 norm to deal with the sparse or Gaussian noise. However, the real noise are usually complex. We propose a robust Bayesian tensor completion method, called MoG BTC-CP, which could impute the missing data and remove the complex noise simultaneously. The observed tensor is assumed to be the summation of a low-rank tensor and the noise. CP decomposition is proposed to extract the low-rank structure of the tensor. We assume that the noise follows a Mixture of Gaussian (MoG) distribution. A full Bayesian framework together with a Gibbs sampling algorithm is designed to estimate the model. Extensive experiments including synthetic data and real life applications show that MoG BTC-CP outperforms the recently published leading tensor completion and denoising methods.

Multiarea Distribution System State Estimation via Distributed Tensor Completion

This paper proposes a model-free distribution system state estimation method based on tensor completion using canonical polyadic decomposition. In particular, we consider a setting where the network is divided into multiple areas. The measured physical quantities at buses located in the same area are processed by an area controller. A three-way tensor is constructed to collect these measured quantities. The measurements are analyzed locally to recover the full state information of the network. A distributed closed-form iterative algorithm based on the alternating direction method of multipliers is developed to obtain the low-rank factors of the whole network state tensor where information exchange happens only between neighboring areas. The convergence properties of the distributed algorithm and the sufficient conditions on the number of samples for each smaller network that guarantee the identifiability of the factors of the state tensor are presented. To demonstrate the efficacy of the proposed algorithm and to check the identifiability conditions, numerical simulations are carried out using the IEEE 123-bus system and a large-scale real utility feeder.

Multispectral and Hyperspectral Image Fusion Based on Regularized Coupled Non-Negative Block-Term Tensor Decomposition

The problem of multispectral and hyperspectral image fusion (MHF) is to reconstruct images by fusing the spatial information of multispectral images and the spectral information of hyperspectral images. Focusing on the problem that the hyperspectral canonical polyadic decomposition model and the Tucker model cannot introduce the physical interpretation of the latent factors into the framework, it is difficult to use the known properties and abundance of endmembers to generate high-quality fusion images. This paper proposes a new fusion algorithm. In this paper, a coupled non-negative block-term tensor model is used to estimate the ideal high spatial resolution hyperspectral images, its sparsity is characterized by adding 1-norm, and total variation (TV) is introduced to describe piecewise smoothness. Secondly, the different operators in two directions are defined and introduced to characterize their piecewise smoothness. Finally, the proximal alternating optimization (PAO) algorithm and the alternating multiplier method (ADMM) are used to iteratively solve the model. Experiments on two standard datasets and two local datasets show that the performance of this method is better than the state-of-the-art methods.

Multi-level optimization of the canonical polyadic tensor decomposition at large-scale: Application to the stratification of social networks through deflation

Tensors are multi-dimensional mathematical objects that allow to model complex relationships and to perform decompositions for analytical purpose. They are used in a wide range of data mining applications. In social network analysis, tensor decompositions give interesting insights by taking into consideration multiple characteristics of data. However, the power-law distribution of such data forces the decomposition to reveal only the strong signals that hide information of interest having a lighter intensity. To reveal hidden information, we propose a method to stratify the signal, by gathering clusters of similar intensity in each stratum. It is an iterative process, in which the CANDECOMP/PARAFAC (CP) decomposition is applied and its result is used to deflate the tensor, i.e., by removing from the tensor the clusters found with the decomposition. As the CP decomposition is computationally demanding, it is also necessary to optimize its algorithm, to apply it on large-scale data with a reasonable execution time, even with the several executions needed by the iterative process of the stratification. Therefore, we propose an algorithm that uses both dense and sparse data structures and that leverages coarse and fine grained optimizations in addition to incremental computations in order to achieve large scale CP tensor decomposition. Our implementation outperforms the baseline of large-scale CP decomposition libraries by several orders of magnitude. We validate our stratification method and our optimized algorithm on a Twitter dataset about COVID vaccines.

Coupled canonical polyadic decomposition of multi-group fMRI data with spatial reference and orthonormality constraints

Multi-group fMRI data may possess different types of subjects, tasks, scans, etc. Fortunately, coupled canonical polyadic decomposition (CCPD) requires multiple tensor datasets to share one or more factor matrices. Considering that spatial variability is generally smaller than temporal variability, we attempt CCPD to decompose multi-group fMRI data into shared spatial maps (SMs), group-specific time courses (TCs) and subject intensities. As spatial references of interested components are generally available and the spatial orthonormality can reduce crosstalk among components, we propose a novel CCPD by adding spatial reference and orthonormality constraints. Specifically, based on accelerated alternating least squares, we further update shared SMs twice: 1) we orthonormalize shared SM components by orthogonal Procrustes solution; 2) after identifying the interested components by maximizing Pearson correlation coefficients between shared SMs and spatial references, we update interested shared SMs by minimizing the square error between magnitude part of normalized shared SMs and corresponding normalized spatial references. The results of two-group simulated and experimental task-related fMRI data as well as resting-state fMRI data with 24 healthy controls (HCs) and 24 schizophrenia patients (SZs) all show outperformed performance for the proposed method compared with unconstrained CCPD, CCPD with a spatial orthonormality constraint, widely-used tensor independent component analysis (ICA) and semi-blind group information guide ICA in both magnitude-only analysis and complex-valued analysis. Moreover, by using AdaBoost, resting-state group-specific TCs estimated by the proposed method significantly exhibit larger group differences, especially for the sensorimotor network, and thus provide a potential biomarker for schizophrenia.

Tensor-Based Learning for Detecting Abnormalities on Digital Mammograms.

In this study, we propose a tensor-based learning model to efficiently detect abnormalities on digital mammograms. Due to the fact that the availability of medical data is limited and often restricted by GDPR (general data protection regulation) compliance, the need for more sophisticated and less data-hungry approaches is urgent. Accordingly, our proposed artificial intelligence framework utilizes the canonical polyadic decomposition to decrease the trainable parameters of the wrapped Rank-R FNN model, leading to efficient learning using small amounts of data. Our model was evaluated on the open source digital mammographic database INBreast and compared with state-of-the-art models in this domain. The experimental results show that the proposed solution performs well in comparison with the other deep learning models, such as AlexNet and SqueezeNet, achieving 90% ± 4% accuracy and an F1 score of 84% ± 5%. Additionally, our framework tends to attain more robust performance with small numbers of data and is computationally lighter for inference purposes, due to the small number of trainable parameters.

HyperNTF: A hypergraph regularized nonnegative tensor factorization for dimensionality reduction

Tensor decomposition is an effective tool for learning multi-way structures and heterogeneous features from high-dimensional data, such as the multi-view images and multichannel electroencephalography (EEG) signals, are often represented by tensors. However, most of tensor decomposition methods are the linear feature extraction techniques, which are unable to reveal the nonlinear structure within high-dimensional data. To address such problem, a lot of algorithms have been proposed for simultaneously performs linear and non-linear feature extraction. A representative algorithm is the Graph Regularized Nonnegative Matrix Factorization (GNMF) for image clustering. However, the normal 2-order graph can only model the pairwise similarity of objects, which cannot sufficiently exploit the complex structures of samples. Thus, we propose a novel method, named Hypergraph Regularized Nonnegative Tensor Factorization (HyperNTF), which utilizes hypergraph to model the complex connections among samples and employs the factor matrix corresponding with last mode of Canonical Polyadic (CP) decomposition as low-dimensional representation of original data. Extensive experiments on synthetic manifolds, real-world image datasets, and EEG signals, demonstrating that HyperNTF outperforms the state-of-the-art methods in terms of dimensionality reduction, clustering, and classification.

Compression of hyperspectral images based on Tucker decomposition and CP decomposition.

Hyperspectral imagers are developing towards high resolution, high detection sensitivity, broad spectra, and wide coverage, which means that hyperspectral data are getting more and more substantial. This brings a great challenge to data storage and real-time transmission of hyperspectral data. A compression method based on Tucker decomposition and CANDECOMP/PARAFAC decomposition (TD-CP) is proposed. The hyperspectral data are treated as a third-order tensor. First, TD is performed on the hyperspectral data to obtain a core tensor and three factor matrices, and then CP decomposition is performed on the core tensor. Compared with principal component analysis (PCA)+JPEG2000, TD, and CP, TD-CP can retain spatial information and spectral information better at the same time, and running time is shorter.

Smoothed tensor quantile regression estimation for longitudinal data

As extensions of vector and matrix data with ultrahigh dimensionality and complex structures, tensor data are fast emerging in a large variety of scientific applications. In this paper, a two-stage estimation procedure for linear tensor quantile regression (QR) with longitudinal data is proposed. In the first stage, we account for within-subject correlations by using the generalized estimating equations and then impose a low-rank assumption on tensor coefficients to reduce the number of parameters by a canonical polyadic decomposition. To avoid the asymptotic analysis and computation problems caused by the non-smooth QR score function, kernel smoothing method is applied in the second stage to construct the smoothed tensor QR estimator. When the number of rank is given, a block-relaxation algorithm is proposed to estimate the regression coefficients. A modified BIC is applied to estimate the number of rank in practice and show the rank selection consistency. Further, a regularized estimator and its algorithm are investigated for better interpretation and efficiency. The asymptotic properties of the proposed estimators are established. Simulation studies and a real example on Beijing Air Quality data set are provided to show the performance of the proposed estimators.

Blind signal separation for coprime planar arrays: An improved coupled trilinear decomposition method

AbstractIn this study, the problem of blind signal separation for coprime planar arrays is investigated. For coprime planar arrays comprising two uniform rectangular subarrays, we link the signal separation to the tensor‐based model called coupled canonical polyadic decomposition (CPD) and propose an improved coupled trilinear decomposition approach. The output data of coprime planar arrays are modeled as a coupled tensor set that can be further interpreted as a coupled CPD model, allowing a signal separation to be achieved using coupled trilinear alternating least squares (TALS). Furthermore, in the procedure of the coupled TALS, a Vandermonde structure enforcing approach is explicitly applied, which is shown to ensure fast convergence. The results of Monto Carlo simulations show that our proposed algorithm has the same separation accuracy as the basic coupled TALS but with a faster convergence speed.

Multilayer Multiconfiguration Time-Dependent Hartree Study on the Mode-/Bond-Specific Quantum Dynamics of Water Dissociation on Cu(111).

In this work, full-dimensional (9D) quantum dynamics calculations on mode-/bond-specific surface scattering of a water molecule on a copper (111) rigid surface are performed through the multilayer multiconfiguration time-dependent Hartree (ML-MCTDH) method. To easily perform the ML-MCTDH calculations on such a triatomic molecule-surface system, we first choose specific Jacobi coordinates as a set of coordinates of water. Next, to efficiently perform the 9D ML-MCTDH wavepacket propagation, the potential energy surface is transferred to a canonical polyadic decomposition form with the aid of a Monte Carlo-based method. Excitation-specific dissociation probabilities of H2O on Cu(111) are computed, and mode-/bond-specific dynamics are demonstrated by comparison with a probability curve computed for a water molecule in the ground state. The dependence of the dissociation probability of the initial state of H2O is studied, and it is found that the excitation-specific dissociation probabilities can be divided into three groups. We find that the vibrationally excited states enhance the dissociation reactivity of H2O, while the rotationally excited states hardly influence it.

Tensor Decomposition-Based Channel Estimation for Hybrid mmWave Massive MIMO in High-Mobility Scenarios

Massive multiple-input multiple-output (MIMO) integrated with millimeter-wave (mmWave) can provide unprecedented performance improvement for realizing future wireless communications. However, acquiring accurate channel state information in wideband mmWave massive MIMO systems with hybrid transceiver architectures is even challenging, especially in high-mobility scenarios with severe Doppler effects. In this paper, we propose a tensor decomposition-based method to estimate the time-varying and frequency-selective (TVFS) mmWave MIMO channels. Specifically, by exploiting the sparse scattering nature of TVFS channels, we model the frequency-domain received signal as a third-order tensor that admits a canonical polyadic (CP) decomposition format. Then, we analyze the uniqueness condition of the proposed CP decomposition-based channel estimation problem and propose a novel estimator to acquire TVFS channel parameters including angle of departure/arrival (AoD/AoA), time delay, path gain, and the Doppler shift. To address the sophisticated coupling among unknown parameters, we further propose a joint AoD and Doppler shift estimation (JADE) algorithm that provides reliable initial and iteratively refined estimates. The derived analysis and simulation results verify that the proposed JADE algorithm achieves higher estimation accuracy and guarantees the superiority of the proposed TVFS channel estimator over existing schemes.

Practical Leverage-Based Sampling for Low-Rank Tensor Decomposition

The low-rank canonical polyadic tensor decomposition is useful in data analysis and can be computed by solving a sequence of overdetermined least squares subproblems. Motivated by consideration of sparse tensors, we propose sketching each subproblem using leverage scores to select a subset of the rows, with probabilistic guarantees on the solution accuracy. We randomly sample rows proportional to leverage score upper bounds that can be efficiently computed using the special Khatri--Rao subproblem structure inherent in tensor decomposition. Crucially, for a $(d+1)$-way tensor, the number of rows in the sketched system is $O(r^d/\epsilon)$ for a decomposition of rank $r$ and $\epsilon$-accuracy in the least squares solve, independent of both the size and the number of nonzeros in the tensor. Along the way, we provide a practical solution to the generic matrix sketching problem of sampling overabundance for high-leverage-score rows, proposing to include such rows deterministically and combine repeated samples in the sketched system; we conjecture that this can lead to improved theoretical bounds. Numerical results on real-world large-scale tensors show the method is significantly faster than deterministic methods at nearly the same level of accuracy.

Multi-Subject Analysis for Brain Developmental Patterns Discovery via Tensor Decomposition of MEG Data.

Identification of informative signatures from electrophysiological signals is important for understanding brain developmental patterns, where techniques such as magnetoencephalography (MEG) are particularly useful. However, less attention has been given to fully utilizing the multidimensional nature of MEG data for extracting components that describe these patterns. Tensor factorizations of MEG yield components that encapsulate the data's multidimensional nature, providing parsimonious models identifying latent brain patterns for meaningful summarization of neural processes. To address the need for meaningful MEG signatures for studies of pediatric cohorts, we propose a tensor-based approach for extracting developmental signatures of multi-subject MEG data. We employ the canonical polyadic (CP) decomposition for estimating latent spatiotemporal components of the data, and use these components for group level statistical inference. Using CP decomposition along with hierarchical clustering, we were able to extract typical early and late latency event-related field (ERF) components that were discriminative of high and low performance groups ([Formula: see text]) and significantly correlated with major cognitive domains such as attention, episodic memory, executive function, and language comprehension. We demonstrate that tensor-based group level statistical inference of MEG can produce signatures descriptive of the multidimensional MEG data. Furthermore, these features can be used to study group differences in brain patterns and cognitive function of healthy children. We provide an effective tool that may be useful for assessing child developmental status and brain function directly from electrophysiological measurements and facilitate the prospective assessment of cognitive processes.

Ranks with Respect to a Projective Variety and a Cost-Function

Let X⊂Pr be an integral and non-degenerate variety. A “cost-function” (for the Zariski topology, the semialgebraic one, or the Euclidean one) is a semicontinuous function w:=[1,+∞)∪+∞ such that w(a)=1 for a non-empty open subset of X. For any q∈Pr, the rank rX,w(q) of q with respect to (X,w) is the minimum of all ∑a∈Sw(a), where S is a finite subset of X spanning q. We have rX,w(q)<+∞ for all q. We discuss this definition and classify extremal cases of pairs (X,q). We give upper bounds for all rX,w(q) (twice the generic rank) not depending on w. This notion is the generalization of the case in which the cost-function w is the constant function 1. In this case, the rank is a well-studied notion that covers the tensor rank of tensors of arbitrary formats (PARAFAC or CP decomposition) and the additive decomposition of forms. We also adapt to cost-functions the rank 1 decomposition of real tensors in which we allow pairs of complex conjugate rank 1 tensors.

Multistatic MIMO radar target localization via coupled canonical polyadic decomposition

Target localization is considered based on a multistatic MIMO radar that consists of a single transmit array and multiple receive arrays. The output data of the multistatic MIMO radar is formulated into a set of tensors that together admits a coupled canonical polyadic decomposition (C-CPD) model, and a thorough study on the identifiability of the C-CPD model is provided. In addition, a C-CPD based computational framework for target localization is presented, consisting of coupled data compression in the preprocessing step, several (semi-)algebraic and optimization based C-CPD algorithms in different scenarios as the main body of the framework, and a data fusion approach that calculates the optimal position of each target by fusing the C-CPD results in the postprocessing stage. Due to the exploitation of coupling, the proposed C-CPD based framework is free of parameter pairing and can yield more accurate estimates than conventional single-set based methods. In addition, the proposed method does not require prior knowledge of probing waveforms, which can be non-orthogonal, and can work in difficult underdetermined cases. Experiments are provided to illustrate the performance of the proposed framework.

Representation of Diabatic Potential Energy Matrices for Multiconfiguration Time-Dependent Hartree Treatments of High-Dimensional Nonadiabatic Photodissociation Dynamics.

Conventional quantum mechanical characterization of photodissociation dynamics is restricted by steep scaling laws with respect to the dimensionality of the system. In this work, we examine the applicability of the multi-configurational time-dependent Hartree (MCTDH) method in treating nonadiabatic photodissociation dynamics in two prototypical systems, taking advantage of its favorable scaling laws. To conform to the sum-of-product form, elements of the ab initio diabatic potential energy matrix (DPEM) are re-expressed using the recently proposed Monte Carlo canonical polyadic decomposition method, with enforcement of proper symmetry. The MCTDH absorption spectra and product branching ratios are shown to compare well with those calculated using conventional grid-based methods, demonstrating its promise for treating high-dimensional nonadiabatic photodissociation problems.

Scalable tensor factorization for recovering multiday missing intramuscular electromyography data

To design a prosthetic hand which can classify movements based on the electromyography (EMG) signals, complete and good quality signals are essential. However, due to different reasons such as disconnection of electrodes or muscles fatigue the recorded EMG data can be incomplete, which degrades the classification of test movements. In this paper, we first acquire multiday intramuscular EMG (iEMG) signals (which are invasive) with higher Signal-to-Noise Ratio (SNR) compared to surface EMG (sEMG) signals; followed by application of matrix (non-negative matrix factorization – NMF) and tensor factorization methods (Canonical Polyadic Decomposition (CPD), Tucker decomposition (TD) & Canonical Polyadic-Weighted Optimization (CP-WOPT)) for recovering structured missing data i.e., chunks of missing samples in channels. Furthermore, we tested the scalability of NMF, CPD, TD and CP-WOPT by employing them on the large multiday (seven days) iEMG data where the size of missing data is increased from day 1 to day 7, and for each day a fixed percentage of missing data is introduced from 10% to worst case of 50%. Results show that CP-WOPT outperformed NMF, CPD and TD to recover large percentage of missing data in terms of Relative Mean Error (RME) even when 7 days of data is considered. CP-WOPT showed robustness even for the worse case even when 50% iEMG data is removed from day 1 to day 7 where it’s RME degraded slightly from 0.08 to 0.1.

Compressed Channel Estimation for IRS-Assisted Millimeter Wave OFDM Systems: A Low-Rank Tensor Decomposition-Based Approach

We consider the problem of downlink channel estimation for intelligent reflecting surface (IRS)-assisted millimeter Wave (mmWave) orthogonal frequency division multiplexing (OFDM) systems. By exploring the inherent sparse scattering characteristics of mmWave channels, we show that the received signals can be expressed as a low-rank third-order tensor that admits a tensor rank decomposition, also known as canonical polyadic decomposition (CPD). A structured CPD-based method is then developed to estimate the channel parameters. Our analysis reveals that the training overhead required by our proposed method is as low as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(U^{2})$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${U}$ </tex-math></inline-formula> denotes the sparsity of the cascade channel. Simulation results are provided to illustrate the efficiency of the proposed method.