Tensor Decomposition Method Research Articles

DRIM: A Web-Based System for Investigating Drug Response at the Molecular Level by Condition-Specific Multi-Omics Data Integration.

Pharmacogenomics is the study of how genes affect a person's response to drugs. Thus, understanding the effect of drug at the molecular level can be helpful in both drug discovery and personalized medicine. Over the years, transcriptome data upon drug treatment has been collected and several databases compiled before drug treatment cancer cell multi-omics data with drug sensitivity (IC50, AUC) or time-series transcriptomic data after drug treatment. However, analyzing transcriptome data upon drug treatment is challenging since more than 20,000 genes interact in complex ways. In addition, due to the difficulty of both time-series analysis and multi-omics integration, current methods can hardly perform analysis of databases with different data characteristics. One effective way is to interpret transcriptome data in terms of well-characterized biological pathways. Another way is to leverage state-of-the-art methods for multi-omics data integration. In this paper, we developed Drug Response analysis Integrating Multi-omics and time-series data (DRIM), an integrative multi-omics and time-series data analysis framework that identifies perturbed sub-pathways and regulation mechanisms upon drug treatment. The system takes drug name and cell line identification numbers or user's drug control/treat time-series gene expression data as input. Then, analysis of multi-omics data upon drug treatment is performed in two perspectives. For the multi-omics perspective analysis, IC50-related multi-omics potential mediator genes are determined by embedding multi-omics data to gene-centric vector space using a tensor decomposition method and an autoencoder deep learning model. Then, perturbed pathway analysis of potential mediator genes is performed. For the time-series perspective analysis, time-varying perturbed sub-pathways upon drug treatment are constructed. Additionally, a network involving transcription factors (TFs), multi-omics potential mediator genes, and perturbed sub-pathways is constructed, and paths to perturbed pathways from TFs are determined by an influence maximization method. To demonstrate the utility of our system, we provide analysis results of sub-pathway regulatory mechanisms in breast cancer cell lines of different drug sensitivity. DRIM is available at: http://biohealth.snu.ac.kr/software/DRIM/.

Robust Tensor Decomposition for Image Representation Based on Generalized Correntropy.

Traditional tensor decomposition methods, e.g., two dimensional principal component analysis and two dimensional singular value decomposition, that minimize mean square errors, are sensitive to outliers. To overcome this problem, in this paper we propose a new robust tensor decomposition method using generalized correntropy criterion (Corr-Tensor). A Lagrange multiplier method is used to effectively optimize the generalized correntropy objective function in an iterative manner. The Corr-Tensor can effectively improve the robustness of tensor decomposition with the existence of outliers without introducing any extra computational cost. Experimental results demonstrated that the proposed method significantly reduces the reconstruction error on face reconstruction and improves the accuracies on handwritten digit recognition and facial image clustering.

A new method of data missing estimation with FNN-based tensor heterogeneous ensemble learning for internet of vehicle

The Internet of Vehicles (IoV) can obtain traffic information through a large number of data collected by sensors. However, the lack of data, abnormal data, and other low-quality problems have seriously restricted the development and application of the IoV. To solve the problem of missing data in a large-scale road network, the previous research achievements show that tensor decomposition method has the advantages in solving multi-dimensional data imputation problems, so we adopt this tensor mode to model traffic velocity data. A new method of data missing estimation with tensor heterogeneous ensemble learning based on FNN (Fuzzy Neural Network) named FNNTEL is proposed in this paper. The performance of this method is evaluated by our experiments and analysis. The proposed method is applied to be tested by the real data captured in Guangzhou and Tianjin of China respectively. A large number of experimental tests show that the performance of the new method is better than other commonly used technologies and different missing data generation models.

Hybrid tensor decomposition in neural network compression

Deep neural networks (DNNs) have enabled impressive breakthroughs in various artificial intelligence (AI) applications recently due to its capability of learning high-level features from big data. However, the current demand of DNNs for computational resources especially the storage consumption is growing due to that the increasing sizes of models are being required for more and more complicated applications. To address this problem, several tensor decomposition methods including tensor-train (TT) and tensor-ring (TR) have been applied to compress DNNs and shown considerable compression effectiveness. In this work, we introduce the hierarchical Tucker (HT), a classical but rarely-used tensor decomposition method, to investigate its capability in neural network compression. We convert the weight matrices and convolutional kernels to both HT and TT formats for comparative study, since the latter is the most widely used decomposition method and the variant of HT. We further theoretically and experimentally discover that the HT format has better performance on compressing weight matrices, while the TT format is more suited for compressing convolutional kernels. Based on this phenomenon we propose a strategy of hybrid tensor decomposition by combining TT and HT together to compress convolutional and fully connected parts separately and attain better accuracy than only using the TT or HT format on convolutional neural networks (CNNs). Our work illuminates the prospects of hybrid tensor decomposition for neural network compression.

Identification of miRNA signatures for kidney renal clear cell carcinoma using the tensor-decomposition method

Cancer is a highly complex disease caused by multiple genetic factors. MicroRNA (miRNA) and mRNA expression profiles are useful for identifying prognostic biomarkers for cancer. Kidney renal clear cell carcinoma (KIRC), which accounts for more than 70% of all renal malignant tumour cases, was selected for our analysis. Traditional methods of identifying cancer prognostic markers may not be accurate. Tensor decomposition (TD) is a useful method uncovering the underlying low-dimensional structures in the tensor. The TD-based unsupervised feature extraction method was applied to analyse mRNA and miRNA expression profiles. Biological annotations of the prognostic miRNAs and mRNAs were examined utilizing the pathway and oncogenic signature databases DIANA-miRPath and MSigDB. TD identified the miRNA signatures and the associated genes. These genes were found to be involved in cancer-related pathways, and 23 genes were significantly correlated with the survival of KIRC patients. We demonstrated that the results are robust and not highly dependent upon the databases we selected. Compared with traditional supervised methods tested, TD achieves much better performance in selecting prognostic miRNAs and mRNAs. These results suggest that integrated analysis using the TD-based unsupervised feature extraction technique is an effective strategy for identifying prognostic signatures in cancer studies.

Generalized orthonormal moment tensor decomposition and its source-type diagram

Moment tensor decomposition is a method for deriving the isotropic (ISO), double-couple (DC), and compensated linear vector dipole (CLVD) components from a seismic moment tensor. Currently, there are two families of methods, namely, standard moment tensor decomposition and Euclidean moment tensor decomposition. Although both methods can usually provide workable solutions, there are some minor inconsistencies between the two methods: an equality inconsistency that occurs in standard moment tensor decomposition and the pure CLVD unity and flip basis inconsistency encountered in Euclidean moment tensor decomposition. Moreover, there is a sign problem when disentangling the CLVD component from a DC-dominated case. To address these minor inconsistencies, we propose a new moment tensor decomposition method inspired by both previous methods. The new method can not only avoid all these minor inconsistencies but also withstand deviations in ISO- or CLVD-dominated cases when using source-type diagrams.

Hierarchical low-rank and sparse tensor micro defects decomposition by electromagnetic thermography imaging system.

With the advancement of electromagnetic induction thermography and imaging technology in non-destructive testing field, this system has significantly benefitted modern industries in fast and contactless defects detection. However, due to the limitations of front-end hardware experimental equipment and the complicated test pieces, these have brought forth new challenges to the detection process. Making use of the spatio-temporal video data captured by the thermal imaging device and linking it with advanced video processing algorithm to defects detection has become a necessary alternative way to solve these detection challenges. The extremely weak and sparse defect signal is buried in complex background with the presence of strong noise in the real experimental scene has prevented progress to be made in defects detection. In this paper, we propose a novel hierarchical low-rank and sparse tensor decomposition method to mine anomalous patterns in the induction thermography stream for defects detection. The proposed algorithm offers advantages not only in suppressing the interference of strong background and sharpens the visual features of defects, but also overcoming the problems of over- and under-sparseness suffered by similar state-of-the-art algorithms. Real-time natural defect detection experiments have been conducted to verify that the proposed algorithm is more efficient and accurate than existing algorithms in terms of visual presentations and evaluation criteria. This article is part of the theme issue 'Advanced electromagnetic non-destructive evaluation and smart monitoring'.

Multi-Paradigm fMRI Fusion via Sparse Tensor Decomposition in Brain Functional Connectivity Study

Functional magnetic resonance imaging (fMRI) is a powerful technique with the potential to estimate individual variations in behavioral and cognitive traits. Joint learning of multiple datasets can utilize their complementary information so as to improve learning performance, but it also gives rise to the challenge for data fusion to effectively integrate brain patterns elicited by multiple fMRI data. However, most of the current data fusion methods analyze each single dataset separately and further infer the relationship among them, which fail to utilize the multidimensional structure inherent across modalities and may ignore complex but important interactions. To address this issue, we propose a novel sparse tensor decomposition method to integrate multiple task-stimulus (paradigm) fMRI data. Seeing each paradigm fMRI as one modality, our proposed method considers the relationships across subjects and modalities simultaneously. In specific, a third-order tensor is first modeled by using the functional network connectivity (FNC) of subjects in multiple fMRI paradigms. A novel sparse tensor decomposition with the regularization terms is designed to factorize the tensor into a series of rank-one components, which can extract the shared components across modalities as the embedded features. The L2,1-norm regularizer (i.e., group sparsity) is enforced to select a few common features among multiple subjects. Validation of the proposed method is performed on realistic three paradigm fMRI datasets from the Philadelphia Neurodevelopmental Cohort (PNC) study, for the study of the relationship between the FNC and human cognitive abilities. Experimental results show our method outperforms several other competing methods in the prediction of individuals with different cognitive behaviors via the wide range achievement test (WRAT). Furthermore, our method discovers the FNC related to the cognitive behaviors, such as the connectivity associated with the default mode network (DMN) for three paradigms, and the connectivity between DMN and visual (VIS) domains within the emotion task.

A tensor decomposition method for cycle-based traffic volume estimation using sampled vehicle trajectories

Cycle-based traffic volume estimation is important for the dynamic evaluation and optimization of signal control schemes at intersections. With the development of intelligent mobility and connected vehicle technologies, massive high-resolution vehicle trajectory data have become available, which can provide a rich source of information for estimating traffic flow parameters at signalized intersections. Existing methods for traffic volume estimation using sampled vehicle trajectories are commonly driven by shockwave or probabilistic models, in which the traffic volume estimation is usually modelled as a parameter estimation problem. These methods commonly require certain assumptions for the vehicle arrival distribution and a First-In-First-Out (FIFO) queuing rule. However, these model-driven methods have limitations when the vehicle arrival distribution is unknown or when there is more than one lane for the same movement at an intersection approach, as the FIFO rule is no longer true. In addition, the accuracy and stability of these models remain challenging under low penetration rates. In this paper, we propose a tensor decomposition method to estimate cycle-based traffic volume at signalized intersections using sampled vehicle trajectories. Unlike the existing model-driven methods, the proposed method is purely data-driven and does not rely on any prerequisite assumptions for the arrival distributions and queuing rules. In the proposed method, the traffic volume of each cycle is first divided into a known part and an unknown part, based on the queuing position of the last queued sample vehicle. Then, these two parts of the volume as a whole, along with two relevant traffic observations (i.e., the number of sampled vehicles and the observation duration), are then integrated into a three-dimensional tensor. This tensor can effectively preserve the temporal correlations between adjacent cycles and the interrelationships among the traffic observations. Finally, the problem of cycle-based traffic estimation is transformed into a tensor completion problem, and the Tucker decomposition method is adopted to solve the completion problem. The proposed method is evaluated using both empirical and simulation data. The results indicate that the proposed method is capable of producing accurate and reliable estimates for cycle-based traffic volumes even under low penetration rates, i.e., less than 5%.

Tensor methods for the Boltzmann-BGK equation

We present a tensor-decomposition method to solve the Boltzmann transport equation (BTE) in the Bhatnagar-Gross-Krook approximation. The method represents the six-dimensional BTE as a set of six one-dimensional problems, which are solved with the alternating least-squares algorithm and the discrete Fourier transform at N collocation points. We use this method to predict the equilibrium distribution (steady-state simulation) and a non-equilibrium distribution returning to the equilibrium (transient simulation). Our numerical experiments demonstrate Nlog⁡N scaling. Unlike many BTE-specific numerical techniques, the numerical tensor-decomposition method we propose is a general technique that can be applied to other high-dimensional systems.

Multipass SAR Interferometry Based on Total Variation Regularized Robust Low Rank Tensor Decomposition

Multipass SAR interferometry (InSAR) techniques based on meter-resolution spaceborne SAR satellites, such as TerraSAR-X or COSMO-Skymed, provide 3D reconstruction and the measurement of ground displacement over large urban areas. Conventional method such as Persistent Scatterer Interferometry (PSI) usually requires a fairly large SAR image stack (usually in the order of tens), in order to achieve reliable estimates of these parameters. Recently, low rank property in multipass InSAR data stack was explored and investigated in our previous work. By exploiting this low rank prior, more accurate estimation of the geophysical parameters can be achieved, which in turn can effectively reduce the number of interferograms required for a reliable estimation. Based on that, this paper proposes a novel tensor decomposition method in complex domain, which jointly exploits low rank and variational prior of the interferometric phase in InSAR data stacks. Specifically, a total variation (TV) regularized robust low rank tensor decomposition method is exploited for recovering outlier-free InSAR stacks. We demonstrate that the filtered InSAR data stacks can greatly improve the accuracy of geophysical parameters estimated from real data. Moreover, this paper demonstrates for the first time in the community that tensor-decomposition-based methods can be beneficial for large-scale urban mapping problems using multipass InSAR. Two TerraSAR-X data stacks with large spatial areas demonstrate the promising performance of the proposed method.

Tensor decomposition with relational constraints for predicting multiple types of microRNA-disease associations.

MicroRNAs (miRNAs) play crucial roles in multifarious biological processes associated with human diseases. Identifying potential miRNA-disease associations contributes to understanding the molecular mechanisms of miRNA-related diseases. Most of the existing computational methods mainly focus on predicting whether a miRNA-disease association exists or not. However, the roles of miRNAs in diseases are prominently diverged, for instance, Genetic variants of miRNA (mir-15) may affect the expression level of miRNAs leading to B cell chronic lymphocytic leukemia, while circulating miRNAs (including mir-1246, mir-1307-3p, etc.) have potentials to detecting breast cancer in the early stage. In this paper, we aim to predict multi-type miRNA-disease associations instead of taking them as binary. To this end, we innovatively represent miRNA-disease-type triples as a tensor and introduce tensor decomposition methods to solve the prediction task. Experimental results on two widely-adopted miRNA-disease datasets: HMDD v2.0 and HMDD v3.2 show that tensor decomposition methods improve a recent baseline in a large scale (up to $38\%$ in Top-1F1). We then propose a novel method, Tensor Decomposition with Relational Constraints (TDRC), which incorporates biological features as relational constraints to further the existing tensor decomposition methods. Compared with two existing tensor decomposition methods, TDRC can produce better performance while being more efficient.

The probabilistic tensor decomposition toolbox

This article introduces the probabilistic tensor decomposition toolbox - a MATLAB toolbox for tensor decomposition using Variational Bayesian inference and Gibbs sampling. An introduction and overview of probabilistic tensor decomposition and its connection with classical tensor decomposition methods based on maximum likelihood is provided. We subsequently describe the probabilistic tensor decomposition toolbox which encompasses the Canonical Polyadic, Tucker, and Tensor Train decomposition models. Currently, unconstrained, non-negative, orthogonal, and sparse factors are supported. Bayesian inference forms a principled way of incorporating prior knowledge, prediction of held-out data, and estimating posterior probabilities. Furthermore, it facilitates automatic model order determination, automatic regularization on factors (e.g. sparsity), and inherently penalizes model complexity which is beneficial when inferring hierarchical models, such as heteroscedastic noise modelling. The toolbox allows researchers to easily apply Bayesian tensor decomposition methods without the need to derive or implement these methods themselves. Furthermore, it serves as a reference implementation for comparing existing and new tensor decomposition methods. The software is available from https://github.com/JesperLH/prob-tensor-toolbox/.

A Comparison of Non-negative Tucker Decomposition and Parallel Factor Analysis for Identification and Measurement of Human EEG Rhythms

Abstract Analysis of changes in the brain neural electrical activity measured by the electroencephalogram (EEG) plays a crucial role in the area of brain disorder diagnostics. The elementary latent sources of the brain neural activity can be extracted by a tensor decomposition of continuously recorded multichannel EEG. Parallel factor analysis (PARAFAC) is a powerful approach for this purpose. However, the assumption of the same number of factors in each dimension of the PARAFAC model may be restrictive when applied to EEG data. In this article we discuss the potential benefits of an alternative tensor decomposition method – the Tucker model. We analyze situations, where in comparison to the PARAFAC solution, the Tucker model provides a more parsimonious representation of the EEG data decomposition. We show that this more parsimonious representation of EEG is achieved without reducing the ability to explain variance. We analyze EEG records of two patients after ischemic stroke and we focus on the extraction of specific sensorimotor oscillatory sources associated with motor imagery during neurorehabilitation training. Both models provided consistent results. The advantage of the Tucker model was a compact structure with only two spatial signatures reflecting the expected lateralized activation of the detected subject-specific sensorimotor rhythms.

Correlation-based initialization algorithm for tensor-based HSI compression methods

Tensor decomposition (TD) is widely used in hyperspectral image (HSI) compression. we note that the correlation in HSI may be helpful to improve the HSI compression performance of TD methods. The initialization of factor matrix in TD can determine the HSI compression performance. Meanwhile, it is worth noting that HSI is highly correlated in bands. In this paper, we propose a method called correlation-based TD initialization algorithm. By analyzing the optimization of TD, we find that the initialization of TD can be well approximated by a matrix decomposition. Thus, we adopt the mean band of HSI to approximate the initialization of TD. In accordance with the SVD result of the reference band, the initialized factor matrices of TD are produced. We compare our methods with other compression methods. The experimental results reveal that our correlation-based TD initialization method is capable of significantly reducing the computational cost of TD while keeping the initialization quality and compression performance.

Canonical polyadic decomposition (CPD) of big tensors with low multilinear rank

Tensor decomposition methods have been widely applied to big data analysis as they bring multiple modes and aspects of data to a unified framework, which allows us to discover complex internal structures and correlations of data. Unfortunately most existing approaches are not designed to meet the challenges posed by big data dilemma. This paper attempts to improve the scalability of tensor decompositions and makes two contributions: A flexible and fast algorithm for the CP decomposition (FFCP) of tensors based on their Tucker compression; A distributed randomized Tucker decomposition approach for arbitrarily big tensors but with relatively low multilinear rank. These two algorithms can deal with huge tensors, even if they are dense. Extensive simulations provide empirical evidence of the validity and efficiency of the proposed algorithms.

Embedding Traffic Network Characteristics Using Tensor for Improved Traffic Prediction

Techniques for using multi-way traffic patterns for traffic prediction is gaining importance. One of the possible techniques for representing the multi-way traffic patterns is tensors. Tensor decomposition is used to generate low-rank approximations of the original tensor that is subsequently used for traffic volume prediction. However, the existing tensor-based approaches do not consider certain important mutual relationships among the locations like temporal traffic reciprocity that can improve the prediction accuracy. In this paper, we introduce TeDCaN, a “Tensor Decomposition method with Characteristic Network” constraints that generate low rank approximations of the original tensor considering the traffic reciprocity at different pair of locations. Investigations using large traffic datasets from 2 different cities reveal that the prediction accuracy of TeDCaN considerably outperforms several state-of-art baselines for cases when complete traffic data is available as well as situations when a certain fraction of the data is missing - a likely scenario in many real datasets. We discover that TeDCaN achieves around 20% reduction in the RMSE scores as compared to the baselines. TeDCaN is applicable in many operations on such a big traffic network where the existing models would either be inapplicable or hard to perform. As one of the major yields, TeDCaN generates a “reduced dimensional network embedding” that captures the similarity of the nodes considering the traffic volume as well as the reciprocity of traffic between the nodes.

Beyond Unfolding: Exact Recovery of Latent Convex Tensor Decomposition Under Reshuffling

Exact recovery of tensor decomposition (TD) methods is a desirable property in both unsupervised learning and scientific data analysis. The numerical defects of TD methods, however, limit their practical applications on real-world data. As an alternative, convex tensor decomposition (CTD) was proposed to alleviate these problems, but its exact-recovery property is not properly addressed so far. To this end, we focus on latent convex tensor decomposition (LCTD), a practically widely-used CTD model, and rigorously prove a sufficient condition for its exact-recovery property. Furthermore, we show that such property can be also achieved by a more general model than LCTD. In the new model, we generalize the classic tensor (un-)folding into reshuffling operation, a more flexible mapping to relocate the entries of the matrix into a tensor. Armed with the reshuffling operations and exact-recovery property, we explore a totally novel application for (generalized) LCTD, i.e., image steganography. Experimental results on synthetic data validate our theory, and results on image steganography show that our method outperforms the state-of-the-art methods.

HO-OTSVD: A Novel Tensor Decomposition and Its Incremental Decomposition for Cyber–Physical–Social Networks (CPSN)

The development of social networks and ubiquitous sensing promotes the network space into a new stage, which integrates the cyber network, physical network, and social network into cyber–physical–social networks (CPSN). In this paper, we propose a CPSN-based service framework. The framework firstly represents CPSN as an adjacency tensor. Then, a novel tensor decomposition method named high-order orthogonal tensor singular value decomposition (HO-OTSVD) is proposed for knowledge discovery. To cope with the dynamic CPSN, an incremental HO-OTSVD (IHO-OTSVD) is developed to update the orthogonal tensor basis and the core tensor. Furthermore, we propose high-order bidiagonal Lanczos algorithm to cope with the orthogonalization of HO-OTSVD, wherein the complexity reduces from cubic execution time to quadratic execution time. Finally, we use a recommendation system as a case study to evaluate the effectiveness and efficiency of the proposed CPSN-based framework. The results show that HO-OTSVD method outperforms the existing methods.

Multi-Aspect Incremental Tensor Decomposition Based on Distributed In-Memory Big Data Systems

Abstract Purpose We propose InParTen2, a multi-aspect parallel factor analysis three-dimensional tensor decomposition algorithm based on the Apache Spark framework. The proposed method reduces re-decomposition cost and can handle large tensors. Design/methodology/approach Considering that tensor addition increases the size of a given tensor along all axes, the proposed method decomposes incoming tensors using existing decomposition results without generating sub-tensors. Additionally, InParTen2 avoids the calculation of Khari–Rao products and minimizes shuffling by using the Apache Spark platform. Findings The performance of InParTen2 is evaluated by comparing its execution time and accuracy with those of existing distributed tensor decomposition methods on various datasets. The results confirm that InParTen2 can process large tensors and reduce the re-calculation cost of tensor decomposition. Consequently, the proposed method is faster than existing tensor decomposition algorithms and can significantly reduce re-decomposition cost. Research limitations There are several Hadoop-based distributed tensor decomposition algorithms as well as MATLAB-based decomposition methods. However, the former require longer iteration time, and therefore their execution time cannot be compared with that of Spark-based algorithms, whereas the latter run on a single machine, thus limiting their ability to handle large data. Practical implications The proposed algorithm can reduce re-decomposition cost when tensors are added to a given tensor by decomposing them based on existing decomposition results without re-decomposing the entire tensor. Originality/value The proposed method can handle large tensors and is fast within the limited-memory framework of Apache Spark. Moreover, InParTen2 can handle static as well as incremental tensor decomposition.