CANDECOMP/PARAFAC Decomposition Research Articles

CP factor model for dynamic tensors

Abstract Observations in various applications are frequently represented as a time series of multidimensional arrays, called tensor time series, preserving the inherent multidimensional structure. In this paper, we present a factor model approach, in a form similar to tensor CANDECOMP/PARAFAC (CP) decomposition, to the analysis of high-dimensional dynamic tensor time series. As the loading vectors are uniquely defined but not necessarily orthogonal, it is significantly different from the existing tensor factor models based on Tucker-type tensor decomposition. The model structure allows for a set of uncorrelated one-dimensional latent dynamic factor processes, making it much more convenient to study the underlying dynamics of the time series. A new high-order projection estimator is proposed for such a factor model, utilizing the special structure and the idea of the higher order orthogonal iteration procedures commonly used in Tucker-type tensor factor model and general tensor CP decomposition procedures. Theoretical investigation provides statistical error bounds for the proposed methods, which shows the significant advantage of utilizing the special model structure. Simulation study is conducted to further demonstrate the finite sample properties of the estimators. Real data application is used to illustrate the model and its interpretations.

Visible light fingerprint database recovery algorithm based on CP decomposition.

Visible light communication(VLC) is a new method of indoor communication. It can provide an effective solution for indoor positioning. Fingerprint-based visible light positioning(VLP) has been widely studied for its feasibility and high accuracy. The acquisition of 'fingerprint database' is crucial for accurate VLP. However, sparse sensors such as photodiode(PD) can only be arranged because of the space-limited scenario and high costs. Correspondingly, it results in the loss of the fingerprint database. Therefore, it is indispensable to solve the problem of how to effectively and accurately recover the fingerprint database from measurements of sparsely arranged sensors. In this paper, we propose a spatio-temporal constraint tensor completion (SCTC) algorithm based on CANDECOMP/PARAFAC (CP) decomposition to recover the fingerprint database from measurements of sparsely arranged sensors. Specifically, we model the measurements from the spatial and temporal dimensions as a tensor, and formulate the optimization problem based on the low-rank feature of the tensor. To improve the recovery accuracy, spatial and temporal constraint matrices are introduced to effectively constrain the optimization direction when completing the tensor. Spatial constraint matrices are constructed by utilizing the mode-n expansion matrix of the tensor based on the undirected graph theory. Accordingly, the Toeplitz matrix is used as the temporal constraint matrix to excavate the temporal correlation of the tensor. Since the optimization problem is non-convex and difficult to solve, we introduce CP decomposition to decompose the tensor into several factor matrices. By solving the factor matrices, the original tensor is reconstructed. The performance of the proposed SCTC algorithm is confirmed via experimental measured data.

Probability-weighted tensor robust PCA with CP decomposition for hyperspectral image restoration

This paper presents a novel probability-weighted tensor robust principal component analysis (TRPCA) method based on CANDECOMP/PARAFAC decomposition (CPD) for hyperspectral image (HSI) restoration with low computational complexity from the coexistence of dense noise and sparse outliers. In the proposed method, via the CPD property, the tensor nuclear norm (TNN) optimization object is replaced as the CPD factor matrix of HSI with lower dimensions without losing low-rank property, so as to the computational complexity could be reduced during optimizing TNN in TRPCA model. To demonstrate the different effects of noise and outliers on HSI restoration, two weighted sets are defined with probability by the prior knowledge of the occurrence of outliers, which are used to analyze the probability whether the HSI is corrupted by dense noise or outliers. Via the probability theory, information entropy is employed to measure the uncertainty of noise and outlier occurrence in received data, so that to demonstrate their difference impacts on HSI restoration to improve the recovery accuracy. Theoretical analysis and simulation results show that the proposed method has lower computational complexity and better restoration performance compared with other related methods.

Unsupervised Feature Selection via Graph Regularized Nonnegative CP Decomposition.

Unsupervised feature selection has attracted remarkable attention recently. With the development of data acquisition technology, multi-dimensional tensor data has been appeared in enormous real-world applications. However, most existing unsupervised feature selection methods are non-tensor-based which results the vectorization of tensor data as a preprocessing step. This seemingly ordinary operation has led to an unnecessary loss of the multi-dimensional structural information and eventually restricted the quality of the selected features. To overcome the limitation, in this paper, we propose a novel unsupervised feature selection model: Nonnegative tensor CP (CANDECOMP/PARAFAC) decomposition based unsupervised feature selection, CPUFS for short. In specific, we devise new tensor-oriented linear classifier and feature selection matrix for CPUFS. In addition, CPUFS simultaneously conducts graph regularized nonnegative CP decomposition and newly-designed tensor-oriented pseudo label regression and feature selection to fully preserve the multi-dimensional data structure. To solve the CPUFS model, we propose an efficient iterative optimization algorithm with theoretically guaranteed convergence, whose computational complexity scales linearly in the number of features. A variation of the CPUFS model by incorporating nonnegativity into the linear classifier, namely CPUFSnn, is also proposed and studied. Experimental results on ten real-world benchmark datasets demonstrate the effectiveness of both CPUFS and CPUFSnn over the state-of-the-arts.

Identification of metabotypes in complex biological data using tensor decomposition

Differences in the physiological response to treatment, such as dietary intervention, has led to the development of precision approaches in nutrition and medicine to tailor treatment for improved benefits to the individual. One such approach is to identify metabotypes, i.e., groups of individuals with similar metabolic profiles and/or regulation. Metabotyping has previously been performed using e.g., principal component analysis (PCA) on matrix data. However, metabotyping methods suitable for more complex experimental designs such as repeated measures or cross-over studies are needed. We have developed a metabotyping method for tensor data, based on CANDECOMP/PARAFAC (CP) tensor decomposition. Metabotypes are inferred from CP scores using k-means clustering, and robustness is evaluated using bootstrapping of metabolites. As a proof-of-concept, we identified metabotypes from metabolomics data where 79 metabolites were analyzed in 8 time points postprandially in 17 overweight men that underwent a three-arm dietary crossover intervention. Two metabotypes were found, characterized by differences in amino acid metabolite concentration, that were differentially associated with baseline plasma creatinine (p = 0.007) and with the baseline metabolome (p = 0.004). These results suggest that CP decomposition provides a viable approach for metabotype identification directly from complex, high-dimensional data with improved biological interpretation compared to the more simplistic PCA approach. A simulation study together with results from measured data concluded that several preprocessing methods should be taken into consideration for CP-based metabotyping on complex tensor data.

Inertial accelerated SGD algorithms for solving large-scale lower-rank tensor CP decomposition problems

The stochastic gradient descent (SGD) method has been applied to the tensor CANDECOMP/PARAFAC (CP) decomposition problem to reduce the computational cost. However, SGD usually takes numerous iterations to converge. In this paper, we propose inertial accelerated SGD methods for the subproblems of CP decomposition concerning one block factor. We consider two methods for the selection of a block index in the subproblem: the stochastic manner with replacement and the randomly permuted order without replacement. First, we propose an inertial accelerated block-randomized SGD algorithm for CP decomposition (iBrasCPD) and show each cluster point is a stochastic stationary point. Then we introduce a randomized permutation inertial stochastic algorithm (RP-iSPALM) for randomized order without replacement. Under some mild conditions, we show the global convergence and convergence rate of RP-iSPALM under the Kurdyka-Łojasiewicz (KL) property. Numerical experiments on six real datasets demonstrate that our proposed algorithms are efficient and can achieve better performance than the existing state-of-the-art methods.

Time-Varying Channel Estimation Scheme for Uplink MU-MIMO in 6G Systems

Channel estimation is a challenging issue for millimeter wave (mmWave) and massive multiple-input multiple-output (MIMO) in the future sixth generation (6G) wireless systems, where the conventional estimation schemes may fail to track the fast varying channels, especially in high-speed mobile scenarios. In this paper, a novel tensor-based uplink channel estimation scheme is proposed for multi-user MIMO (MU-MIMO) systems over time-varying channels. In the proposed scheme, a low-overhead pilot transmission scheme is designed to track the varying channel. The received uplink signal at the base station (BS) is formulated as a third-order tensor which admits a CANDECOMP/PARAFAC (CP) model. The CP decomposition issue is then solved using blind matrix decomposition, in which the special structures of signals in the time dimension and the matrix subspace are utilized. By exploiting low-rank structure of the signal tensor, the channel parameters (angles of arrival/departure, path gains, and Doppler shifts) are estimated from the factor matrices. Moreover, the proposed scheme is theoretically analyzed and it is guaranteed with low pilot overhead. Simulation results verify that the proposed scheme can outperform the compressed sensing (CS) based scheme and the iteration-based scheme in terms of accuracy and stability. The uniqueness of the proposed scheme is also verified in our simulation.

Multi-level optimization of the canonical polyadic tensor decomposition at large-scale: Application to the stratification of social networks through deflation

Tensors are multi-dimensional mathematical objects that allow to model complex relationships and to perform decompositions for analytical purpose. They are used in a wide range of data mining applications. In social network analysis, tensor decompositions give interesting insights by taking into consideration multiple characteristics of data. However, the power-law distribution of such data forces the decomposition to reveal only the strong signals that hide information of interest having a lighter intensity. To reveal hidden information, we propose a method to stratify the signal, by gathering clusters of similar intensity in each stratum. It is an iterative process, in which the CANDECOMP/PARAFAC (CP) decomposition is applied and its result is used to deflate the tensor, i.e., by removing from the tensor the clusters found with the decomposition. As the CP decomposition is computationally demanding, it is also necessary to optimize its algorithm, to apply it on large-scale data with a reasonable execution time, even with the several executions needed by the iterative process of the stratification. Therefore, we propose an algorithm that uses both dense and sparse data structures and that leverages coarse and fine grained optimizations in addition to incremental computations in order to achieve large scale CP tensor decomposition. Our implementation outperforms the baseline of large-scale CP decomposition libraries by several orders of magnitude. We validate our stratification method and our optimized algorithm on a Twitter dataset about COVID vaccines.

Fast Tensor-Based Joint Estimation for Time Delay and Angle of Arrival in OFDM System

Nowadays, the joint estimation of time delay (TD) and angle of arrival (AOA) using conventional vector structure suffers from the considerable complexity of multidimensional spectrum search. Therefore, a fast estimation method using orthogonal frequency division multiplexing (OFDM) technology and uniform planar array (UPA) is proposed in this paper, which adopts low-complexity tensor-based operations and spatial-frequency features to reconfigure the channel frequency response. To begin with, the array response is integrated with the OFDM signal characteristics to build an extended array in tensor form. Afterwards, we process the covariance matrix of the tensor structure by CANDECOMP/PARAFAC decomposition (CPD) to separate the respective signal subspaces of TD and AOA estimates. Finally, we conduct a one-dimensional (1-D) spectrum search to locate the TD estimates and a two-dimensional (2-D) spectrum search to locate the AOA estimates. The simulated performance demonstrates that the proposed algorithm offers precise estimates at low signal-to-noise ratios in a multipath environment and outperforms traditional vector-based algorithms with respect to computational complexity.

Bayesian tensor logistic regression with applications to neuroimaging data analysis of Alzheimer's disease.

Alzheimer's disease (AD) can be diagnosed by utilizing traditional logistic regression models to fit magnetic resonance imaging (MRI) data of brain, which is regarded as a vector of covariates. But its parameter estimation is inefficient and computationally extensive due to ultrahigh dimensionality and complicated structure of MRI data. To overcome this deficiency, this paper proposes a tensor logistic regression model (TLRM) for AD's MRI data by regarding MRI tensor as covariates. Under this framework, a tensor candecomp/parafac (CP) decomposition tool is employed to reduce ultrahigh dimensional tensor to a high dimensional level, a novel Bayesian adaptive Lasso method is developed to simultaneously select important components of tensor and estimate model parameters by incorporating the Plya-Gamma method leading a closed-form likelihood and avoiding the usage of the Metropolis-Hastings algorithm, and Gibbs sampler technique in Markov chain Monte Carlo (MCMC). A tensor's product technique is utilized to optimize the calculation program and speed up the calculation of MCMC. Bayes factor together with the path sampling approach is presented to select tensor rank in CP decomposition. Effectiveness of the proposed method is illustrated on simulation studies and an MRI data analysis.

Local Unitary Equivalence of Generic Multi-qubits Based on the CP Decomposition

The CANDECOMP/PARAFAC (CP) decomposition is a generalization of the spectral decomposition of matrices to higher-order tensors. In this paper we use the CP decomposition to study unitary equivalence of higher order tensors and construct several invariants of local unitary equivalence for general higher order tensors. Based on this new method, we study the coefficient tensors of $3$-qubit states and obtain a necessary and sufficient criterion for local unitary equivalence of general tripartite states in terms of the CP decomposition. We also generalize this method to obtain some invariants of local unitary equivalence for general multi-partite qudits.

Reducing Cumulative Errors of Incremental CP Decomposition in Dynamic Online Social Networks

CANDECOMP/PARAFAC (CP) decomposition is widely used in various online social network (OSN) applications. However, it is inefficient when dealing with massive and incremental data. Some incremental CP decomposition (ICP) methods have been proposed to improve the efficiency and process evolving data, by updating decomposition results according to the newly added data. The ICP methods are efficient, but inaccurate because of serious error accumulation caused by approximation in the incremental updating. To promote the wide use of ICP, we strive to reduce its cumulative errors while keeping high efficiency. We first differentiate all possible errors in ICP into two types: the cumulative reconstruction error and the prediction error. Next, we formulate two optimization problems for reducing the two errors. Then, we propose several restarting strategies to address the two problems. Finally, we test the effectiveness in three typical dynamic OSN applications. To the best of our knowledge, this is the first work on reducing the cumulative errors of the ICP methods in dynamic OSNs.

Incremental CP Tensor Decomposition by Alternating Minimization Method

In practical applications, incremental tensors are very common: only a portion of tensor data is available, and new data are arriving in the next time step or continuously over time. To handle this type of tensors time-saving algorithms are required for online computation. In this paper, we consider incremental CP (CANDECOMP/PARAFAC) decomposition, which requires one to update the CP decomposition after new tensor data, together with the exiting tensor, are ready for analysis. There exist several incremental CP decomposition algorithms, but almost all of these algorithms assume that the number of CP decomposition components remains fixed in the incremental process. The main contribution of this paper is the study of how to add components in the incremental CP decomposition. We derive the coordinate representation of the incremental CP decomposition with respect to a special basis and show related properties of tensor rank and the uniqueness of such incremental CP decomposition. Under the framework of this representation, the proposed method can be solved by using an alternating minimization algorithm. Numerical examples are presented to show the good performance of the proposed algorithms in terms of computational time and data fitting compared with existing methods.

Joint estimation of carrier frequency and two-dimensional arrival angle based on L-shaped delay array modulation wideband converter

As the signal spectrum in modern information technology becomes wider and wider, multi-band signals are distributed in a frequency range of tens of GHz. It covers a very wide spectrum but each RF signal has a very narrow band, and the distribution location of the band (or carrier frequency) is completely unknown. For the receiver, the single-band signals transmitted together constitute a multi-band signal. The sampling rate required to jointly estimate the space domain and frequency domain parameters of these signals is getting higher and higher. Modulated wideband converter system is an analog information conversion system for multiband analog signals, which is based on compressed sensing theory and greatly reduces the sampling rate. First, we propose an L-shaped delay array structure based on modulated wideband converter, which can estimate carrier frequency and two-dimensional arrival angles with a small number of samples. Secondly, two parameter-estimating algorithms are proposed based on the proposed structure. One is based on the estimating of signal parameter via rotational invariance technique (ESPRIT), which requires a small number of computations and is suitable for real-time processing application scenarios; the other algorithm is based on CANDECOMP/PARAFAC (CP) technique, which has better robustness and is suitable for applications with low signal-to-noise ratio. The samples of the delay channels can be directly used to estimate the carrier frequencies, and then the two-dimensional arrival angles are calculated. No additional pairing issue is required between the parameters. Then we give the time complexity analysis and space complexity analysis of the two methods. It can be found that the computational complexity and space storage occupation of the method based on ESPRIT are lower than those of the CP decomposition method. Then the conditions for unique parameter estimation are given. Finally, simulation experiments show that the proposed methods can estimate the carrier frequencies and two-dimensional arrival angles from sub-Nyquist samples. It can be found that the estimation method based on CP decomposition is more robust than the method based on ESPRIT, but at the cost of increased complexity of the algorithm.

Randomized CP tensor decomposition

The CANDECOMP/PARAFAC (CP) tensor decomposition is a popular dimensionality-reduction method for multiway data. Dimensionality reduction is often sought after since many high-dimensional tensors have low intrinsic rank relative to the dimension of the ambient measurement space. However, the emergence of ‘big data’ poses significant computational challenges for computing this fundamental tensor decomposition. By leveraging modern randomized algorithms, we demonstrate that coherent structures can be learned from a smaller representation of the tensor in a fraction of the time. Thus, this simple but powerful algorithm enables one to compute the approximate CP decomposition even for massive tensors. The approximation error can thereby be controlled via oversampling and the computation of power iterations. In addition to theoretical results, several empirical results demonstrate the performance of the proposed algorithm.

Multi-Variations Activity Based Gaits Recognition Using Commodity WiFi

With the prevalence of commercial WiFi devices and the development of Internet of Things (IoT), researchers have extended the usage of WiFi from communication to sensing. Recently, device-free human activity recognition has been applied to support WiFi based remote control and human-computer interaction. However, prior works usually recognize each individual activity by extracting the feature of corresponding WiFi signals, and hence distinguishing differences between human activities. Once the activity has multiple variations of different body parts (such as head, arm and leg), distinguishing such sub-activities is extremely difficult. In this paper, we propose a Multi-variations Activity Recognition (MAR) system to identify multiple-variations of body parts. Our work is based on an observation that the Channel State Information (CSI) is sensitive to the activities of body parts. Firstly, CANDECOMP/ PARAFAC (CP) decomposition and Dynamic Time Warping (DTW) are applied to recognize multi-variations activities. Secondly, we theoretically analyse the uniqueness of CP decomposition. Then, we design specific experiment to verify the reliability and stability of uniqueness. Finally, we apply MAR in gaits recognition of multiple volunteers to evaluate the accuracy performance. The experiment results demonstrate that MAR achieves average 95% accuracy in gaits recognition.

Nonlocal Tensor-Ring Decomposition for Hyperspectral Image Denoising

Hyperspectral image (HSI) denoising is a fundamental problem in remote sensing and image processing. Recently, nonlocal low-rank tensor approximation-based denoising methods have attracted much attention due to their advantage of being capable of fully exploiting the nonlocal self-similarity and global spectral correlation. Existing nonlocal low-rank tensor approximation methods were mainly based on two common decomposition [Tucker or CANDECOMP/PARAFAC (CP)] methods and achieved the state-of-the-art results, but they are subject to certain issues and do not produce the best approximation for a tensor. For example, the number of parameters for Tucker decomposition increases exponentially according to its dimensions, and CP decomposition cannot better preserve the intrinsic correlation of the HSI. In this article, a novel nonlocal tensor-ring (TR) approximation is proposed for HSI denoising by using TR decomposition to explore the nonlocal self-similarity and global spectral correlation simultaneously. TR decomposition approximates a high-order tensor as a sequence of cyclically contracted third-order tensors, which has strong ability to explore these two intrinsic priors and to improve the HSI denoising results. Moreover, an efficient proximal alternating minimization algorithm is developed to optimize the proposed TR decomposition model efficiently. Extensive experiments on three simulated data sets under several noise levels and two real data sets verify that the proposed TR model provides better HSI denoising results than several state-of-the-art methods in terms of quantitative and visual performance evaluations.

Superpixel Tensor Model for Spatial–Spectral Classification of Remote Sensing Images

Nowadays, many methods of spatial–spectral classification have been developed and achieved good results for classification with high-resolution remotely sensed images, especially superpixel-based methods. However, these methods generally consider a superpixel as a group of pixels instead of one entity, ignoring the spectral–spatial entirety in the third-order RSI data cube. In order to fully exploit the third-order spectral–spatial information, in this paper, we propose a superpixel-based tensor model for RSI classification, where a multiattribute superpixel tensor (MAST) model is constructed on the top of multiattribute superpixel maps based on the concept of extended morphological profiles (EMAPs). In order to manage the adaptive spatial nature of superpixels, we develop an increment strategy to augment all superpixels with filling up their own envelop rectangles including three different ways, i.e., 0 vector, mean vector of all the pixels within the superpixel, or original pixels. Then, we use CANDECOMP/PARAFAC (CP) decomposition to obtain the features of the unified dimension from the MASTs of various sizes. Especially, CP decomposition can deal with missing data, so we also got a fourth means of constructing the MAST. Finally, base kernels calculated, respectively, from the original spectral feature, EMAP features and MAST features are learned by multiple kernel learning methods, with the optimal kernel fed to a support vector machine to complete the classification task. The experiments conducted on four real RSIs and compared with several well-known methods demonstrate the effectiveness of the proposed model.

Application of Nonnegative Tensor Factorization for Intercity Rail–Air Transport Supply Configuration Pattern Recognition

With the rapid expansion of the railway represented by high-speed rail (HSR) in China, competition between railway and aviation will become increasingly common on a large scale. Beijing, Shanghai, and Guangzhou are the busiest cities and the hubs of railway and aviation transportation in China. Obtaining their supply configuration patterns can help identify defects in planning. To achieve that, supply level is proposed, which is a weighted supply traffic volume that takes population and distance factors into account. Then supply configuration can be expressed as the distribution of supply level over time periods with different railway stations, airports, and city categories. Furthermore, nonnegative tensor factorization (NTF) is applied to pattern recognition by introducing CP (CANDECOMP/PARAFAC) decomposition and the block coordinate descent (BCD) algorithm for the selected data set. Numerical experiments show that the designed method has good performance in terms of computation speed and solution quality. Recognition results indicate the significant pattern characteristics of rail–air transport for Beijing, Shanghai, and Guangzhou are extracted, which can provide some theoretical references for practical policymakers.

Application of Non-Negative Tensor Factorization for Airport Flight Delay Pattern Recognition

With the rapid development of civil aviation transportation in China, huge demand growth has broken the balance between supply and demand, resulting in airspace congestion and increasing flight delays. The delays of large airports have been increasing year by year, which has seriously affected the air travel experience of passengers. Obtaining their flight delay patterns can help identify defects in flight scheduling and airspace utilization. The investigation based on the actual flight operation data of Tianjin Binhai International Airport (TSN) is conducted, in order to capture the relationship and impact between the factors such as traffic flow direction, airline attributes and hourly average delay distribution. Furthermore, Non-negative Tensor Factorization (NTF) is applied to pattern recognition by introducing CP (CANDECOMP/PARAFAC) decomposition and Block Coordinate Descent (BCD) algorithm for selected data set. Numerical experiments show that the designed method has good performance in terms of computation speed and solution quality. Recognition results indicate the significant pattern characteristics of the Tianjin airport delay are extracted, which can provide some new perspectives for air traffic management unit to alleviate airspace congestion and improve service quality.